Before, if the user fixed their clock incorrectly set to the future, they needed to delete
previously sent messages or wait until this future comes again so that new sent messages are added
to the bottom. Strictly speaking, the problem isn't fixable because we don't know if messages were
incorrectly timestamped into the future or they are timestamped correctly and the clock is now
incorrectly set to the past. Anyway, adding messages to the middle of the chat isn't a good way to
inform the user about the problem.
We already set sort_timestamp to 0 for "Messages are end-to-end encrypted."
since 8f1bf963b4.
Do this for "Others will only see this group after you sent a first message."
and "Messages in this chat use classic email and are not encrypted." as well
so no messages can be added on top.
This avoids creating 1:1 chat on a second device when joining a channel.
Now when joining a channel there may be no 1:1 chat with the inviter
when the channel is created. In this case we still create the channel
as unblocked even if 1:1 chat would be a contact request
because joining the channel is an explicit action
and it is not possible to add someone who did not scan a QR
to the channel manually.
The tests were originally generated with AI and then reworked.
Follow-up to https://github.com/chatmail/core/pull/7754 (c724e29)
This prevents the following attack:
/// Eve is subscribed to a channel and wants to know whether Alice is also subscribed to it.
/// To achieve this, Eve sends a message to Alice
/// encrypted with the symmetric secret of this broadcast channel.
///
/// If Alice sends an answer (or read receipt),
/// then Eve knows that Alice is in the broadcast channel.
///
/// A similar attack would be possible with auth tokens
/// that are also used to symmetrically encrypt messages.
///
/// To prevent this, a message that was unexpectedly
/// encrypted with a symmetric secret must be dropped.
If one of broadcast owner's devices didn't add a new subscriber for any reason, e.g. because of
missing SecureJoin messages, this device shall add "member added" messages when syncing the member
list from the `SetPgpContacts` message.
The test sometimes fails because of wrong message ordering for bob:
[...]
Waiting for the device of alice@example.org to reply… [NOTICED][INFO]
<Msg#2010🔒: (Contact#Contact#2001): hi [FRESH]
Msg#2008🔒: (Contact#Contact#2001): You joined the channel. [FRESH][INFO]
>Msg#2010🔒: (Contact#Contact#2001): hi [FRESH]
Msg#2011🔒: (Contact#Contact#2001): Member Me removed by alice@example.org. [FRESH][INFO]
This adds `SystemTime::shift(Duration::from_secs(1))` as a workaround.
Co-authored-by: Hocuri <hocuri@gmx.de>
Don't use first-person form in placeholder texts,
as these can be misleading when broadcasted to group.
Additionally ensures that broadcasted system messages
are not localized to not leak locally-set language
to the group chat.
Fixes#7930
Signed-off-by: Jagoda Ślązak <jslazak@jslazak.com>
Close https://github.com/chatmail/core/issues/7396. Before reviewing,
you should read the issue description of
https://github.com/chatmail/core/issues/7396.
I recommend to review with hidden whitespace changes.
TODO:
- [x] Implement the new protocol
- [x] Make Rust tests pass
- [x] Make Python tests pass
- [x] Test it manually on a phone
- [x] Print the sent messages, and check that they look how they should:
[test_secure_join_group_with_mime_printed.txt](https://github.com/user-attachments/files/24800556/test_secure_join_group.txt)
- [x] Fix bug: If Alice has a second device, then Bob's chat won't be
shown yet on that second device. Also, Bob's contact isn't shown in her
contact list. As soon as either party writes something into the chat,
the that shows up and everything is fine. All of this is still a way
better UX than in WhatsApp, where Bob always has to write first 😂
Still, I should fix that.
- This is actually caused by a larger bug: AUTH tokens aren't synced if
there is no corresponding INVITE token.
- Fixed by 6b658a0e0
- [x] Either make a new `auth_tokens` table with a proper UNIQUE bound,
or put a UNIQUE bound on the `tokens` table
- [x] Benchmarking
- [x] TODOs in the code, maybe change naming of the new functions
- [x] Write test for interop with older DC (esp. that the original
securejoin runs if you remove the &v=3 param)
- [x] From a cryptography perspective, is it fine that vc-request is
encrypted with AUTH, rather than a separate secret (like INVITE)?
- [x] Make sure that QR codes without INVITE work, so that we can remove
it eventually
- [x] Self-review, and comment on some of my code changes to explain
what they do
- [x] ~~Maybe use a new table rather than reusing AUTH token.~~ See
https://github.com/chatmail/core/pull/7754#discussion_r2728544725
- [ ] Update documentation; I'll do that in a separate PR. All necessary
information is in the https://github.com/chatmail/core/issues/7396 issue
description
- [ ] Update tests and other code to use the new names (e.g.
`request-pubkey` rather than `request` and `pubkey` rather than
`auth-required`); I'll do that in a follow-up PR
**Backwards compatibility:**
Everything works seamlessly in my tests. If both devices are updated,
then the new protocol is used; otherwise, the old protocol is used. If
there is a not-yet-updated second device, it will correctly observe the
protocol, and mark the chat partner as verified.
Note that I removed the `Auto-Submitted: auto-replied` header from
securejoin messages. We don't need it ourselves, it's a cleartext header
that leaks too much information, and I can't see any reason to have it.
---------
Co-authored-by: iequidoo <117991069+iequidoo@users.noreply.github.com>
Timestamp renewal was introduced in 1dbf924c6a "feat:
chat::resend_msgs: Guarantee strictly increasing time in the Date header" so that re-sent messages
can be deduplicated on the reciver side, but the deduplication logic doesn't depend on "Date"
anymore.
fix#7877
The bug was: If there is no chat description, and the chat description
is set to an empty string, the INSERT statement inserted a row with an
empty chat description, and therefore from the view of the INSERT
statement, something changed.
This PR fixes this by simply loading the chat description first, and
comparing it.
instead of Alice saying to Bob "You changed the chat description",
we now say "[Chat description changed, please update ...]
i was also considering to say "[Chat description changed to:\n\n...]"
but then there is no incentive for ppl to update, and chat descriptions
for chat creation would still be missing. and this is probably far more
often used.
successor of https://github.com/chatmail/core/pull/7829
fix https://github.com/chatmail/core/issues/7766
Implementation notes:
- Descriptions are only sent with member additions, when the description
is changed, and when promoting a previously-unpromoted group, in order
not to waste bandwith.
- Descriptions are not loaded everytime a chat object is loaded, because
they are only needed for the profile. Instead, they are in their own
table, and can be loaded with their own JsonRPC call.
---------
Co-authored-by: iequidoo <117991069+iequidoo@users.noreply.github.com>
resolves#7724: When forwarding a message with file to another profile, the file was not copied to the target $blobdir and so the forwarded message missed it
---------
Co-authored-by: Hocuri <hocuri@gmx.de>
This PR fixes a bug that old channel members were remembered in the
database even after they left the channel. Concretely, they remained in
the `past_members` table that's only meant for groups.
Though it was not a bad bug; we're anyways not cleaning up old contacts.
Add `chat::forward_msgs_2ctx()` which takes another context as a parameter and forwards messages to
it and its jsonrpc wrapper `CommandApi::forward_messages_to_account()`.
There are quite some unneeded stock strings; this PR removes some of
them. None of these stock strings were actually set by the UI, or even
have translations in Transifex.
- We don't have AEAP anymore.
- The "I added/removed member" and "I left the group" strings are
anyways not meant to be shown to the user. Also, starting to translate
them now would leak the device language.
BREAKING CHANGE: This can theoretically be a breaking change because a
UI could reference one of the removed stock strings, so I marked it as
breaking just in case.
Fix#7435
For most messages, `calc_sort_timestamp()` makes sure that they are at the correct place; esp. that they are not above system messages or other noticed/seen messages.
Most callers of `add_info_msg()`, however, didn't call `calc_sort_timestamp()`, and just used `time()` or `smeared_time()` to get the sort timestamp. Because of this, system messages could sometimes wrongly be sorted above other messages.
This PR fixes this by making the sort timestamp optional in `add_info_msg*()`. If the sort timestamp isn't passed, then the message is sorted to the bottom of the chat. `sent_rcvd_timestamp` is not optional anymore, because we need _some_ timestamp that can be shown to the user (most callers just pass `time()` there).
Follow-up for https://github.com/chatmail/core/pull/7042, part of
https://github.com/chatmail/core/issues/6884.
This will make it possible to create invite-QR codes for broadcast
channels, and make them symmetrically end-to-end encrypted.
- [x] Go through all the changes in #7042, and check which ones I still
need, and revert all other changes
- [x] Use the classical Securejoin protocol, rather than the new 2-step
protocol
- [x] Make the Rust tests pass
- [x] Make the Python tests pass
- [x] Fix TODOs in the code
- [x] Test it, and fix any bugs I find
- [x] I found a bug when exporting all profiles at once fails sometimes,
though this bug is unrelated to channels:
https://github.com/chatmail/core/issues/7281
- [x] Do a self-review (i.e. read all changes, and check if I see some
things that should be changed)
- [x] Have this PR reviewed and merged
- [ ] Open an issue for "TODO: There is a known bug in the securejoin
protocol"
- [ ] Create an issue that outlines how we can improve the Securejoin
protocol in the future (I don't have the time to do this right now, but
want to do it sometime in winter)
- [ ] Write a guide for UIs how to adapt to the changes (see
https://github.com/deltachat/deltachat-android/pull/3886)
## Backwards compatibility
This is not very backwards compatible:
- Trying to join a symmetrically-encrypted broadcast channel with an old
device will fail
- If you joined a symmetrically-encrypted broadcast channel with one
device, and use an old core on the other device, then the other device
will show a mostly empty chat (except for two device messages)
- If you created a broadcast channel in the past, then you will get an
error message when trying to send into the channel:
> The up to now "experimental channels feature" is about to become an officially supported one. By that, privacy will be improved, it will become faster, and less traffic will be consumed.
>
> As we do not guarantee feature-stability for such experiments, this means, that you will need to create the channel again.
>
> Here is what to do:
> • Create a new channel
> • Tap on the channel name
> • Tap on "QR Invite Code"
> • Have all recipients scan the QR code, or send them the link
>
> If you have any questions, please send an email to delta@merlinux.eu or ask at https://support.delta.chat/.
## The symmetric encryption
Symmetric encryption uses a shared secret. Currently, we use AES128 for
encryption everywhere in Delta Chat, so, this is what I'm using for
broadcast channels (though it wouldn't be hard to switch to AES256).
The secret shared between all members of a broadcast channel has 258
bits of entropy (see `fn create_broadcast_shared_secret` in the code).
Since the shared secrets have more entropy than the AES session keys,
it's not necessary to have a hard-to-compute string2key algorithm, so,
I'm using the string2key algorithm `salted`. This is fast enough that
Delta Chat can just try out all known shared secrets. [^1] In order to
prevent DOS attacks, Delta Chat will not attempt to decrypt with a
string2key algorithm other than `salted` [^2].
## The "Securejoin" protocol that adds members to the channel after they
scanned a QR code
This PR uses the classical securejoin protocol, the same that is also
used for group and 1:1 invitations.
The messages sent back and forth are called `vg-request`,
`vg-auth-required`, `vg-request-with-auth`, and `vg-member-added`. I
considered using the `vc-` prefix, because from a protocol-POV, the
distinction between `vc-` and `vg-` isn't important (as @link2xt pointed
out in an in-person discussion), but
1. it would be weird if groups used `vg-` while broadcasts and 1:1 chats
used `vc-`,
2. we don't have a `vc-member-added` message yet, so, this would mean
one more different kind of message
3. we anyways want to switch to a new securejoin protocol soon, which
will be a backwards incompatible change with a transition phase. When we
do this change, we can make everything `vc-`.
[^1]: In a symmetrically encrypted message, it's not visible which
secret was used to encrypt without trying out all secrets. If this does
turn out to be too slow in the future, then we can remember which secret
was used more recently, and and try the most recent secret first. If
this is still too slow, then we can assign a short, non-unique (~2
characters) id to every shared secret, and send it in cleartext. The
receiving Delta Chat will then only try out shared secrets with this id.
Of course, this would leak a little bit of metadata in cleartext, so, I
would like to avoid it.
[^2]: A DOS attacker could send a message with a lot of encrypted
session keys, all of which use a very hard-to-compute string2key
algorithm. Delta Chat would then try to decrypt all of the encrypted
session keys with all of the known shared secrets. In order to prevent
this, as I said, Delta Chat will not attempt to decrypt with a
string2key algorithm other than `salted`
BREAKING CHANGE: A new QR type AskJoinBroadcast; cloning a broadcast
channel is no longer possible; manually adding a member to a broadcast
channel is no longer possible (only by having them scan a QR code)
If we use modules (which are actually namespaces), we can use shorter names. Another approach is to
only use modules for internal code incapsulation and use full names like deltachat-ffi does.
Create unprotected group in test_create_protected_grp_multidev
The test is renamed accordingly.
SystemMessage::ChatE2ee is added in encrypted groups
regardless of whether they are protected or not.
Previously new encrypted unprotected groups
had no message saying that messages are end-to-end encrypted
at all.
It really depends on the screen,
but on Android phones with higher DPI
than laptop screen the contrast
looks a bit too low.
It should not be increased too high
because of clipping.
While testing the previous commit i understood that it's better to try giving different colors to
groups, particularly if their names are equal so that they visually differ, and at the same time
preserve the color if the group is renamed. Using `grpid` solves this. So let groups change colors
once and forever.
