The motivation is to reduce code complexity, get rid of the extra IMAP connection and cases when
messages are added to chats by Inbox and Sentbox loops in parallel which leads to various message
sorting bugs, particularly to outgoing messages breaking sorting of incoming ones which are fetched
later, but may have a smaller "Date".
If we use modules (which are actually namespaces), we can use shorter names. Another approach is to
only use modules for internal code incapsulation and use full names like deltachat-ffi does.
Create unprotected group in test_create_protected_grp_multidev
The test is renamed accordingly.
SystemMessage::ChatE2ee is added in encrypted groups
regardless of whether they are protected or not.
Previously new encrypted unprotected groups
had no message saying that messages are end-to-end encrypted
at all.
This mechanism replaces `Chat-Verified` header.
New parameter `_verified=1` in `Autocrypt-Gossip`
header marks that the sender has the gossiped key
verified.
Using `_verified=1` instead of `_verified`
because it is less likely to cause troubles
with existing Autocrypt header parsers.
This is also how https://www.rfc-editor.org/rfc/rfc2045
defines parameter syntax.
If the contact is already introduced by someone,
usually by adding to a verified group,
it should not be reverified because of another
chat message is a verified group.
This usually results is verification loops
and is not meaningful because the verifier
likely got this same contact introduced
in the same group.
this PR adds a info message "messages are end-to-end-encrypted" also for
chats created by eg. vcards. by the removal of lock icons, this is a
good place to hint for that in addition; this is also what eg. whatsapp
and others are doing
the wording itself is tweaked at
https://github.com/deltachat/deltachat-android/pull/3817 (and there is
also the rough idea to make the message a little more outstanding, by
some more dedicated colors)
~~did not test in practise, if this leads to double "e2ee info messages"
on secure join, tests look good, however.~~ EDIT: did lots of practise
tests meanwhile :)
most of the changes in this PR are about test ...
ftr, in another PR, after 2.0 reeases, there could probably quite some
code cleanup wrt set-protection, protection-disabled etc.
---------
Co-authored-by: Hocuri <hocuri@gmx.de>
This change introduces a new type of contacts
identified by their public key fingerprint
rather than an e-mail address.
Encrypted chats now stay encrypted
and unencrypted chats stay unencrypted.
For example, 1:1 chats with key-contacts
are encrypted and 1:1 chats with address-contacts
are unencrypted.
Groups that have a group ID are encrypted
and can only contain key-contacts
while groups that don't have a group ID ("adhoc groups")
are unencrypted and can only contain address-contacts.
JSON-RPC API `reset_contact_encryption` is removed.
Python API `Contact.reset_encryption` is removed.
"Group tracking plugin" in legacy Python API was removed because it
relied on parsing email addresses from system messages with regexps.
Co-authored-by: Hocuri <hocuri@gmx.de>
Co-authored-by: iequidoo <dgreshilov@gmail.com>
Co-authored-by: B. Petersen <r10s@b44t.com>
If we receive a message from non-verified contact
in a non-protected chat with a Chat-Verified header,
there is no need to upgrade the chat
to verified and display an error.
If it was an attack, an attacker could
just not send the Chat-Verified header.
Most of the time, however, it is just
message reordering.
- **feat: add `AccountsChanged` and `AccountsItemChanged` events**
- **emit event and add tests**
closes#6106
TODO:
- [x] test receiving synced config from second device
- [x] bug: investigate how to delay the configuration event until it is
actually configured - because desktop gets the event but still shows
account as if it was unconfigured, maybe event is emitted before the
value is written to the database?
- [x] update node bindings constants
I.e. add the "Messages are guaranteed to be end-to-end encrypted from now on." message and mark the
chat as protected again because no user action is required in this case. There are a couple of
problems though:
- If the program crashes earlier than the protection is restored, the chat remains
protection-broken. But this problem already exists because `ChatId::set_protection()` is never
retried.
- If multiple old unverified messages are received, protection messages added in between don't
annihilate, so they clutter the chat.
If a message from an old contact's setup is received, the outdated Autocrypt header isn't applied,
so the contact verification preserves. But the chat protection breaks because the old message is
sorted to the bottom as it mustn't be sorted over the protection info message (which is `InNoticed`
moreover). Would be nice to preserve the chat protection too e.g. add a "protection broken" message,
then the old message and then a new "protection enabled" message, but let's record the current
behaviour first.
This fixes the bug that sometimes made QR scans fail.
The problem was:
When sorting headers into unprotected/hidden/protected, the From: header
was added twice for all messages: Once into unprotected_headers and once
into protected_headers. For messages that are `is_encrypted && verified
|| is_securejoin_message`, the display name is removed before pushing it
into unprotected_headers.
Later, duplicate headers are removed from unprotected_headers right
before prepending unprotected_headers to the message. But since the
unencrypted From: header got modified a bit when removing the display
name, it's not exactly the same anymore, so it's not removed from
unprotected_headers and consequently added again.
If a display name should be protected (i.e. opportunistically encrypted), only put the corresponding
address to the unprotected headers. We protect the From display name only for verified chats,
otherwise this would be incompatible with Thunderbird and K-9 who don't use display names from the
encrypted part. Still, we always protect To display names as compatibility seems less critical here.
When receiving a messge, overwrite the From display name but not the whole From field as that would
allow From forgery. For the To field we don't really care. Anyway as soon as we receive a message
from the user, the display name will be corrected.
Co-authored-by: iequidoo <dgreshilov@gmail.com>
When doing an AEAP transition, we mustn't just delete the old peerstate as this would break
encryption to it. This is critical for non-verified groups -- if we can't encrypt to the old
address, we can't securely remove it from the group (to add the new one instead).
This protects Bob (the joiner) of sending unexpected-unencrypted messages during an otherwise nicely
running SecureJoin.
If things get stuck, however, we do not want to block communication -- the chat is just
opportunistic as usual, but that needs to be communicated:
1. If Bob's chat with Alice is `Unprotected` and a SecureJoin is started, then add info-message
"Establishing guaranteed end-to-end encryption, please wait..." and let `Chat::can_send()` return
`false`.
2. Once the info-message "Messages are guaranteed to be e2ee from now on" is added, let
`Chat::can_send()` return `true`.
3. If after SECUREJOIN_WAIT_TIMEOUT seconds `2.` did not happen, add another info-message "Could not
yet establish guaranteed end-to-end encryption but you may already send a message" and also let
`Chat::can_send()` return `true`.
Both `2.` and `3.` require the event `ChatModified` being sent out so that UI pick up the change wrt
`Chat::can_send()` (this is the same way how groups become updated wrt `can_send()` changes).
SECUREJOIN_WAIT_TIMEOUT should be 10-20 seconds so that we are reasonably sure that the app remains
active and receiving also on mobile devices. If the app is killed during this time then we may need
to do step 3 for any pending Bob-join chats (right now, Bob can only join one chat at a time).
This is a test reproducing the problem
in <https://github.com/deltachat/deltachat-core-rust/issues/5339>.
Fix would be to avoid reordering on the server side,
so the test checks that the unverified message
is replaced with a square bracket error
as expected if messages arrive in the wrong order.
An unencrypted message with already known Autocrypt key, but sent from another address, means that
it's rather a new contact sharing the same key than the existing one changed its address, otherwise
it would already have our key to encrypt.
It was broken completely and before "fix: apply Autocrypt headers if timestamp is unchanged" that
didn't show up because the message from the second Bob's device never had "Date" greater than one
from the message sent before from the first device.
Add a new crate `deltachat_time` with a fake `struct SystemTimeTools` for mocking
`SystemTime::now()` for test purposes. One still needs to use `std::time::SystemTime` as a struct
representing a system time. I think such a minimalistic approach is ok -- even if somebody uses the
original `SystemTime::now()` instead of the mock by mistake, that could break only tests but not the
program itself. The worst thing that can happen is that tests using `SystemTime::shift()` and
checking messages timestamps f.e. wouldn't catch the corresponding bugs, but now we don't have such
tests at all which is much worse.
Test that on the second device of a protected group creator the first message is
`SystemMessage::ChatProtectionEnabled` and the second one is the message populating the group.
I.e. exclude from the list the following chats as well:
- Read-only mailing lists.
- Chats we're not a member of.
But as for ProtectionBroken chats, we return them, as that may happen to a verified chat at any
time. It may be confusing if a chat that is normally in the list disappears suddenly. The UI need to
deal with that case anyway.
If the Inbox is fetched before the Sentbox (as done currently), messages from the Sentbox will
correctly mingle with the Inbox messages in the end. So, this commit changes message ordering only
if we already have processed outgoing messages, e.g. if we just sent them in the chat as described
in #4621. Otherwise new incoming messages are displayed somewhere in the middle of the chat which
doesn't look usable.
Directly unwrap in TestContext::get_chat()
Turns out that all usages of get_chat() directly unwrapped, because in a
test it doesn't make sense to handle the error of there being no chat.
Don't show a contact as verified if their key changed in the meantime
If a contact's key changed since the verification, then it's very
unlikely that they still have the old, verified key. So, don't show them
as verified anymore.
This also means that you can't add a contact like this to a verified
group, which is good.
The documentation actually already described this (new) behavior:
```rust
/// and if the key has not changed since this verification.
```
so, this adapts the code to the documentation.
Correctly handle messages with old timestamps for verified chats:
* They must not be sorted over a protection-changed info message
* If they change the protection, then they must not be sorted over existing other messages, because then the protection-changed info message would also be above these existing messages.
This PR fixes this:
1. Even seen messages can't be sorted into already-noticed messages anymore. **This also changes DC's behavior in the absence of verified 1:1 chats**. Before this PR, messages that are marked as seen when they are downloaded will always be sorted by their timestamp, even if it's very old.
2. protection-changed info messages are always sorted to the bottom.
**Edit:**
3. There is an exception to rule 1: Outgoing messages are still allowed to be sorted purely by their timestamp, and don't influence old messages. This is to the problem described at [*].
Together, these rules also make sure that the protection-changed info message is always right above the message causing the change.
[*] If we receive messages from two different folders, e.g. `Sent` and `Inbox`, then this will lead to wrong message ordering in many cases. I need to think about this more, or maybe someone else has an idea. One new idea that came to my mind is:
* Always sort noticed messages under the newest info message (this PR sorts them under the newest noticed message, master sorts them purely by their sent timestamp)
* Always sort unnoticed messages under the newest noticed message (that's the same behavior as in this PR and on master)
* Always sort protection-changed info messages to the bottom (as in this PR)
However, after a talk with @link2xt we instead decided to add rule 3. (see above) because it seemed a little bit easier.
If the verification is broken, `can_send()` is false.
But if the user was typing a message right when a verification-breaking message came in, the UI still needs to be able to save it as a draft.
Steps to reproduce the bug:
- Set a draft
- Your chat partner breaks verification
- Go back to the chats list
- Go to the chat again
- Accept the breakage
- Expected: The draft is still there
- Bug behavior: The draft is gone
