Otherwise, e.g. if a message is a large GIF, but its viewtype is set to `Image` by the app, this GIF
will be recoded to JPEG to reduce its size. GIFs and other special viewtypes must be always detected
and sent as is.
as discussed in several chats, this PR starts making it possible to use
Webxdc as integrations to the main app. In other word: selected parts of
the main app can be integrated as Webxdc, eg. Maps [^1]
this PR contains two parts:
- draft an Webxdc Integration API
- use the Webxdc Integration API to create a Maps Integration
to be clear: a Webxdc is not part of this PR. the PR is about marking a
Webxdc being used as a Map - and core then feeds the Webxdc with
location data. from the view of the Webxdc, the normal
`sendUpdate()`/`setUpdateListener()` is used.
things are still marked as "experimental", idea is to get that in to
allow @adbenitez and @nicodh to move forward on the integrations into
android and desktop, as well as improving the maps.xdc itself.
good news is that we currently can change the protocol between Webxdc
and core at any point :)
# Webxdc Integration API
see `dc_init_webxdc_integration()` in `deltachat.h` for overview and
documentation.
rust code is mostly in `webxdc/integration.rs` that is called by other
places as needed. current [user of the API is
deltachat-ios](https://github.com/deltachat/deltachat-ios/pull/1912),
android/desktop will probably follow.
the jsonrpc part is missing and can come in another PR when things are
settled and desktop is really starting [^2] (so we won't need to do all
iterations twice :) makes also sense, when this is done by someone
actually trying that out on desktop
while the API is prepared to allow other types of integrations (photo
editor, compose tools ...) internally, we currently ignore the type. if
that gets more crazy, we probably also need a dedicated table for the
integrations and not just a single param.
# Maps Integration
rust code is mostly in `webxdc/maps_integration.rs` that is called by
`webxdc/integration.rs` as needed.
EDIT: the idea of having a split here, is that
`webxdc/maps_integration.rs` really can focus on the json part, on the
communication with the .xdc, including tests
this PR is basic implementation, enabling to move forward on
integrations on iOS, but also on desktop and android.
the current implementation allows already the following:
- global and per-chat maps
- add and display POIs
- show positions and tracks of the last 24 hours
the current maps.xdc uses leaflet, and is in some regards better than
the current android/desktop implementations (much faster, show age of
positions, fade out positions, always show names of POIs, clearer UI).
however, we are also not bound to leaflet, it can be anything
> [**screenshots of the current
state**](https://github.com/deltachat/deltachat-ios/pull/1912)
> 👆
to move forward faster and to keep this PR small, the following will go
to a subsequent PR:
- consider allowing webxdc to use a different timewindow for the
location
- delete POIs
- jsonrpc
[^1]: maps are a good example as anyways barely native (see android
app), did cause a lot of pain on many levels in the past (technically,
bureaucratically), and have a comparable simple api
[^2]: only going for jsonrpc would only make sense if large parts of
android/ios would use jsonrpc, we're not there
---------
Co-authored-by: link2xt <link2xt@testrun.org>
Let's add a 1-minute tolerance to `Params::MemberListTimestamp`.
This adds to the group membership consistency algo the following properties:
- If remote group membership changes were made by two members in parallel, both of them are applied,
no matter in which order the messages are received.
- If we remove a member locally, only explicit remote member additions/removals made in parallel are
allowed, but not the synchronisation of the member list from "To". Before, if somebody managed to
reply earlier than receiving our removal of a member, we added it back which doesn't look good.
Currently when a user sets up another device by logging in, a new key is created. If a message is
sent from either device outside, it cannot be decrypted by the other device.
The message is replaced with square bracket error like this:
```
<string name="systemmsg_cannot_decrypt">This message cannot be decrypted.\n\n• It might already help to simply reply to this message and ask the sender to send the message again.\n\n• If you just re-installed Delta Chat then it is best if you re-setup Delta Chat now and choose "Add as second device" or import a backup.</string>
```
(taken from Android repo `res/values/strings.xml`)
If the message is outgoing, it does not help to "simply reply to this message". Instead, we should
add a translatable device message of a special type so UI can link to the FAQ entry about second
device. But let's limit such notifications to 1 per day. And as for the undecryptable message
itself, let it go to Trash if it can't be assigned to a chat by its references.
In particular TLSRPT reports
contain files that may be interesting for admins.
Currently Delta Chat drops the attachment
so message appears as a text message without actual payload.
Before this fix actual contents of the message
reposted by Schleuder is considered a mailing list footer and removed,
not visible even in the "Show Full Message..." view.
With this change there will be two message bubbles,
one for header and one for the contents,
but it is still better than losing the contents completely.
Attempting to parse header part is out of scope for this change.
Put a copy of Message-ID into hidden headers and prefer it over the one in the IMF header section
that servers mess up with.
This also reverts "Set X-Microsoft-Original-Message-ID on outgoing emails for amazonaws (#3077)".
If a message is encrypted, but unsigned:
- Don't set `MimeMessage::from_is_signed`.
- Remove "secure-join-fingerprint" and "chat-verified" headers from `MimeMessage`.
- Minor: Preserve "Subject" from the unencrypted top level if there's no "Subject" in the encrypted
part, this message is displayed w/o a padlock anyway.
Apparently it didn't lead to any vulnerabilities because there are checks for
`MimeMessage::signatures.is_empty()` in all necessary places, but still the code looked dangerous,
especially because `from_is_singed` var name didn't correspond to its actual value (it was rather
`from_is_encrypted_maybe_signed`).
If a message is unsigned or signed with an unknown key, `MimeMessage::was_encrypted()` returns
false. So, it mustn't be checked when deciding whether to look into
`MimeMessage::decoded_data`. Looking through git history one can see that it's just a wrong check
left in the code for historical reasons.
Check if a sticker has at least one fully transparent corner and otherwise change the Sticker type
to Image. This would fix both Android and iOS at the same time and prevent similar bug on future
platforms that may get this bug like Ubuntu Touch.
If the Inbox is fetched before the Sentbox (as done currently), messages from the Sentbox will
correctly mingle with the Inbox messages in the end. So, this commit changes message ordering only
if we already have processed outgoing messages, e.g. if we just sent them in the chat as described
in #4621. Otherwise new incoming messages are displayed somewhere in the middle of the chat which
doesn't look usable.
Correctly handle messages with old timestamps for verified chats:
* They must not be sorted over a protection-changed info message
* If they change the protection, then they must not be sorted over existing other messages, because then the protection-changed info message would also be above these existing messages.
This PR fixes this:
1. Even seen messages can't be sorted into already-noticed messages anymore. **This also changes DC's behavior in the absence of verified 1:1 chats**. Before this PR, messages that are marked as seen when they are downloaded will always be sorted by their timestamp, even if it's very old.
2. protection-changed info messages are always sorted to the bottom.
**Edit:**
3. There is an exception to rule 1: Outgoing messages are still allowed to be sorted purely by their timestamp, and don't influence old messages. This is to the problem described at [*].
Together, these rules also make sure that the protection-changed info message is always right above the message causing the change.
[*] If we receive messages from two different folders, e.g. `Sent` and `Inbox`, then this will lead to wrong message ordering in many cases. I need to think about this more, or maybe someone else has an idea. One new idea that came to my mind is:
* Always sort noticed messages under the newest info message (this PR sorts them under the newest noticed message, master sorts them purely by their sent timestamp)
* Always sort unnoticed messages under the newest noticed message (that's the same behavior as in this PR and on master)
* Always sort protection-changed info messages to the bottom (as in this PR)
However, after a talk with @link2xt we instead decided to add rule 3. (see above) because it seemed a little bit easier.
simplify() is written to process incoming plaintext messages
and extract footers and quotes from them.
Incoming messages contain various quote styles
and simplify() implements heuristics to detects them.
If dehtml() output is processed by simplify(),
simplify() heuristics may erroneously detect
footers and quotes in produced plaintext.
dehtml() should directly detect quotes
instead of converting them to plaintext quotes
for parsing with simplify().
I.e. > 500K for the balanced media quality and 130K for the worse one. This can remove animation and
transparency from PNG/WebP, but then a user always can send an image as a file.
Also don't reduce wide/high images if they aren't huge. Among other benefits, this way most of PNG
screenshots aren't touched.
Also remove Exif from all images, not from JPEGs only.
Services like Lacre [1] on Disroot and Inbound Encryption on Posteo [2]
offer to encrypt all incoming messages with the provided OpenPGP
public key. Resulting messages are encrypted, but not end-to-end encrypted
and not signed by the sender, therefore should not have a padlock displayed.
However, such encrypted and unsigned message is also not an indication
of an error on ongoing attack, so we shoud not report this as a problem
to the user.
[1] https://lacre.io/
[2] https://posteo.de/en/help/how-do-i-activate-inbound-encryption-with-my-public-pgp-key
Note that if the message is encrypted, we don't check whether it's signed with an attached key
currently, otherwise a massive refactoring of the code is needed because for encrypted messages a
signature is checked and discarded first now.
Fix#3507
Note that this is not intended for a release at this point! We first have to test whether it runs stable enough. If we want to make a release while we are not confident enough in authres-checking, then we have to disable it.
BTW, most of the 3000 new lines are in `test_data/messages/dkimchecks...`, not the actual code
da3a4b94 adds the results to the Message info. It currently does this by adding them to `hop_info`. Maybe we should rename `hop_info` to `extra_info` or something; this has the disadvantage that we can't rename the sql column name though.
Follow-ups for this could be:
- In `update_authservid_candidates()`: Implement the rest of the algorithm @hpk42 and me thought about. What's missing is remembering how sure we are that these are the right authserv-ids. Esp., when receiving a message sent from another account at the same domain, we can be quite sure that the authserv-ids in there are the ones of our email server. This will make authres-checking work with buzon.uy, disroot.org, yandex.ru, mailo.com, and riseup.net.
- Think about how we present this to the user - e.g. currently the only change is that we don't accept key changes, which will mean that the small lock on the message is not shown.
- And it will mean that we can fully enable AEAP, after revisiting the security implications of this, and assuming everyone (esp. @link2xt who pointed out the problems in the first place) feels comfortable with it.
* add request_internet_access manifest option
* test request_internet_access
* update CHANGELOG
* force warning when internet access is enabled
if internet access is enabled,
show a warning instead of the normal summary
(the internet access is currently mainly to test out integrations
as maps for video chat; the summary is dispensable in the cases currently)
* adapt json-rpc's WebxdcMessageInfo
Google Workspace has an option "Append footer" which appends standard
footer defined by administrator to all outgoing messages. However,
there is no plain text part in encrypted messages sent by Delta Chat,
so Google Workspace turn the message into multipart/mixed MIME, where
the first part is an empty plaintext part with a footer and the second
part is the original encrypted message.
This commit makes Delta Chat attempt to repair such messages,
similarly to how it already repairs "Mixed Up" MIME structure in
`get_mixed_up_mime`.
* add min_api to manifest.toml, define WEBXDC_API_VERSION
* return an error instead of index.html if the webxdc requires a newer api
* add a test with an webxdc requiring a newer api
* update CHANGELOG
* let sending invalid webxdc fail
invalid webxdc can still be send as Viewtype::File, however
(maybe one want to discuss errors or so ;)
* clarify the supported zip compression methods
This is an irrecoverable error, dc_receive_imf must not fail on it
as it prevents last seen UID from advancing, so the same message
is prefetched on each iteration of IMAP loop.
