When the published relay list changes, key-contacts are informed with an
unsigned message carrying the re-signed key, encrypted to a chunk of contacts
at a time. It is shaped like a receipt notification naming no message, so that
cores which know nothing about keyupdates trash it as well.
See the src/keyupdate.rs module docs for the design.
An unsigned message carries no intended recipient fingerprints,
so recipients of an encrypted unsigned message
learn nothing about other recipients from the PGP packets.
No functional changes:
Add a relay_addrs helper, share the protected headers and self-key rendering
of non-chat messages, and move insert_into_smtp from securejoin to smtp.
chatmail relays (filtermail) expect {vc,vg}-request
messages to be multipart/mixed with a single part.
Messages had this structure
before commit e0494b0b37
so we need to keep it for compatibility.
This replaces independent data structures
which allowed invalid state of having both the public keys
and the shared secret, or having a shared secret
but not encrypting the message because encryption_pubkeys
was None instead of Some(Vec::new()).
Creating QR code or sending a message will still fail
with a shorter error, but no info message will be created anymore.
Most users should have migrated the channels by recreating them by now.
This change is needed for moving loading of the shared secret
to earlier stages of message preparation,
otherwise mimefactory will have to create these info messages
when loading the message into memory.
"ex" means "used to be, but no longer is".
"ext" means "extended", which is the intended meaning.
"Ext" is more common not only in the Rust ecosystem but in general.
We already had several people (myself included) asking
what "ex" is supposed to mean.
To reproduce this commit, search and replace `_ex(?!\w)` -> `_ext`.
But don't change `list_transports_ex` because it's public API.
Unencrypted messages are no longer sent as multipart/mixed
with Message-ID header duplicated inside.
This was useful as non-standard protection
against servers that rewrite the Message-ID,
but now most messages are sent encrypted
and for them standard RFC 9788 header protection
works the same way.
the wire format is already prepared to carry information additionally to reactions,
so let the header name reflect that as well.
in practise, we might want to use a `pinned` flag very soon
in case pinned messages got resent in channels.
it makes sense if that flag goes to the message as such,
and not to an extra message.
and then it makes sense to not introduce a new things.
ftr, this is about resending state to other members,
it is not about syncing across devices for the same user.
the code as such is not changed by this PR,
it is only a wording preparation, of a header that is not even visible.
if we semantically put things other than reactions to the wire format,
we may want to move it out of `broadcast_reactions.rs`
this PR adds support for reactions in broadcast channels.
> the idea of broadcast reactions is that they are sent as usual from
subscribers to owner. after some time, the owner broadcasts them to all
subscribers, who only get to see reaction+count, not who-reacted-what
the PR is quite large, but a good share are tests and otherwise many
things are straight forward.
review should be done by-file, not by-commit. to make review easier,
here is a high-level overview:
first a change in the existing internal `Reactions` object was required.
before this PR, `Reactions` had a "contact to reaction map" only, and
the "frequencies map", that are actually mainly needed for UI, were
calculated as needed. with this PR, the "frequencies map" is the field
that always exist, the "contact map" is only available on top of that.
moreover, this PR shifts that part to the core, it was unfortunately in
the bindings before.
with that preparation things done as follows:
1. reactions from broadcast channel subscriber (`Chattype::InBroadcast`)
to broadcast channel owner (`Chattype::OutBroadcast`) are sent as usual,
only change for that step was to allow sending them at all
2. the owner receives reactions and saves them to the existing
`reactions` table as usual. additionally, the changed message is
remembered in `reactions_need_broadcast` table
3. in the IMAP loop, when ~10 minutes have passed, and
`reactions_need_broadcast` contains entries, a single, hidden message
with accumulated reactions is sent. this message may contain reactions
to different messages. for each message, all known reactions are sent as
reaction+count.
4. subscriber receive that message and save the accumulated reactions in
`reactions_broadcasted`
6. `get_message_reactions` is adapted so that `frequencies` are set
independently of who-reacted-what (the old and only field).
who-reacted-what is called `by_contact` now. it is always set for
compatibility reasons, however, it is not exhaustive for subscribers.
in general, UI should work with frequencies, the API itself, however,
has not changed.
other tweaks:
- outgoing channels are muted on creation, and UI shall allow to
unmute/mute them as all other chats. reason is that reactions are
notified, but in many cases not of large interest. this is also what
telegram is doing
- to have an intermediate feedback when reacting, the local state should
include ones own reaction, even if it is not yet broadcasted. for that,
we modify `reactions_broadcasted` using `modify_frequencies()` as needed
when sending an reaction. there are still some situations where the
update may not include ones own reaction, in this case it is added
lately by `refine_frequencies()`, so that `get_message_reactions()`
always contain SELF.
(in a first implementation, we always increased SELF reaction in
refine_frequencies(), however, that was worse and led to SELF counted
twice once the owner sent broadcast)
<details>
<summary>wire format</summary>
wire format is a JSON in the `Broadcast-Reactions:` header.
additionally, `Content-Disposition: reaction` is set to not show the
hidden message on existing devices.
using a header also allows us to broadcast reactions with resent channel
messages (on joining) later.
```
{
"messages": [
{
"id": "12345678",
"reactions": [
{ "emoji": "👍", "count": 4 },
{ "emoji": "🎉", "count": 2 }
]
},
{
"id": "23456789",
"reactions": []
}
]
}
```
for `id`, the wire format needs to use `rfc724_mid` as `msg_id` are
local only.
</details>
### known issues
- if the channel owner uses multiple devices, broadcasted reaction
updates are sent from each device. the updates are not that big, so that
is probably not a big deal. if it turns out that this is an issue, we
can think about fixes in another PR. might be done by restarting our
10-minute-wait once we see an update from another device
- we cannot set contact_id for DC_EVENT_REACTIONS_CHANGED - but i doubt
it was ever used
### for another pr
- ~~add `Broadcast-Reactions:` header also for resent channel messages,
so that new subscriber do not only get the latest messages, but also
their reactions. for that, the `Broadcast-Reactions:` header can go to
the corresponding message, no need to send extra messages. we would need
to change the sending part to send all reactions for a given message. on
receiving part, we need to make sure, `receive_broadcast_reactions()` is
called when the message actually exist.~~
EDIT: subsequent PR for resending broadcast reactions at
https://github.com/chatmail/core/pull/8496
- add api to allow only a subset of reactions, fiter incoming reactions
before broadcasting
### misc.
ui pr: https://github.com/deltachat/deltachat-ios/pull/3225 and
https://github.com/deltachat/deltachat-android/pull/4560 , which both
were tested successfully with this core PR already. desktop is meant to
be done once this is merged
---------
Co-authored-by: l <link2xt@testrun.org>
This change separates rendering into two separate steps:
1. Rendering of the message payload without the From, Date and Autocrypt headers.
2. Adding the From, Date and Autocrypt headers and possibly encrypting the message.
The goal is to have serializable result of the first step
that can be persisted in the database and sent later with any email address.
This way it will be possible to send queued messages over any relay.
This will make it possible not to remove all messages from the queue
when the sending relay is changed.
Currently changing `configured_addr` deletes everything from `smtp` table.
This change is however only a refactoring and does not implement any features.
This is not required by <https://datatracker.ietf.org/doc/html/rfc3156>.
Looks like Content-Description and Content-Disposition are
the same as what Thunderbird produces,
e.g. in `test-data/message/thunderbird_encrypted_signed.eml`,
and the same values also made it into Autocrypt spec.
Content-Description is likely not used by anyone.
For Content-Disposition specification see <https://datatracker.ietf.org/doc/html/rfc2183>.
While it is explicitly allowed to set the filename for inline attachment,
it does not look useful to specify generic "encrypted.asc" explicitly.
Content-Disposition for the second part results in some webmail clients
showing the second part as an attachment with "encrypted.asc" filename.
They will likely show it differently after this change,
e.g. with a different filename or actually inline.
E.g. rename location::is_sending_locations_to_chat to location::is_sending_to_chat.
Then import only the location module and use it with location:: prefix,
similarly to how e.g. std::fs or std::tokio::fs avoids using "fs" or "file"
in function names.
Last 10 messages in a broadcast channel are resent. They are sent and encrypted only to the new member, not to other subscribers.
Close#7678
Based on https://github.com/chatmail/core/pull/7854, with the following
changes:
- Refactor and simplify code, don't reuse the existing `get_chat_msgs()`
function
- Document that Param::Arg4 is also used for resent messages
cc818d9099
- It's unclear how exactly to resend webxdc status updates. After
discussing with @r10s, don't resend webxdc's at all for now.
38d57ebb30
- Don't set fake `msg_id` in resent messages
e7d0687d90
Setting the msg_id to `u32::MAX` is hacky, and may just as well break
things as it may fix things, because some code may use the msg.id to
load information from the database, like `get_iroh_topic_for_msg()`.
From reading the code, I couldn't find any problem with leaving the
correct `msg_id`, and if there is one, then we should add a function
parameter `is_resending` that is checked in the corresponding places.
Easiest to review file-by-file rather than individual commits, probably.
I'll squash-merge this.
---------
Co-authored-by: iequidoo <dgreshilov@gmail.com>
Broadcast subscribers can't change the chat name, so sending the "Chat-Group-Name{,-Timestamp}"
headers looks unnecessary. That could be useful for other subscriber's devices, but having only the
chat name isn't enough anyway, at least knowing the secret is necessary which is sent by the
broadcast owner.
This makes such contacts appear in the contact list. `IncomingTo` is used because
`ChatId::accept_ex()` does so for groups, so sending a group message is effectively accepting the
group again in regards to contact searchability.
This fixes up b549e7633d which made it impossible to find contacts
from groups even if we've written there.
Can be reviewed commit-by-commit.
This fixes another silly thing you can do with securejoinv3: show Bob a
QR code with auth token that is a broadcast channel secret of a known
channel, then never respond. Bob will decrypt messages from the channel
and drop them because they are sent by the "wrong" sender.
This can be avoided with domain separation, instead of
encrypting/decrypting securejoinv3 messages directly with auth token,
encrypt/decrypt them with `securejoin/<auth token>` as the secret or
even `securejoinv3/<alice's fingerprint>/<auth token>`. For existing
broadcast channels we cannot do this, but for securejoinv3 that is not
released yet this looks like an improvement that avoids at least this
problem.
Credits to link2xt for noticing the problem.
This also adds Alice's fingerprint to the auth tokens, which
was pretty easy to do. I find it hard to develop an intuition for
whether this is important, or whether we will be annoyed by it in the
future.
**Note:** This means that QR code scans will not work if one of the chat
partners uses a self-compiled core between c724e2981 and merging this PR
here. This is fine; we will just have to tell the other developers to
update their self-compiled cores.
The tests were originally generated with AI and then reworked.
Follow-up to https://github.com/chatmail/core/pull/7754 (c724e29)
This prevents the following attack:
/// Eve is subscribed to a channel and wants to know whether Alice is also subscribed to it.
/// To achieve this, Eve sends a message to Alice
/// encrypted with the symmetric secret of this broadcast channel.
///
/// If Alice sends an answer (or read receipt),
/// then Eve knows that Alice is in the broadcast channel.
///
/// A similar attack would be possible with auth tokens
/// that are also used to symmetrically encrypt messages.
///
/// To prevent this, a message that was unexpectedly
/// encrypted with a symmetric secret must be dropped.
Don't use first-person form in placeholder texts,
as these can be misleading when broadcasted to group.
Additionally ensures that broadcasted system messages
are not localized to not leak locally-set language
to the group chat.
Fixes#7930
Signed-off-by: Jagoda Ślązak <jslazak@jslazak.com>
