Mirrors/chatmail-core
1
0
Fork 0
mirror of https://github.com/chatmail/core.git synced 2026-05-01 20:36:31 +03:00
Code Issues Packages Projects Releases Wiki Activity

feat!: QR codes and symmetric encryption for broadcast channels (#7268)

Browse Source 
Follow-up for https://github.com/chatmail/core/pull/7042, part of
https://github.com/chatmail/core/issues/6884.

This will make it possible to create invite-QR codes for broadcast
channels, and make them symmetrically end-to-end encrypted.

- [x] Go through all the changes in #7042, and check which ones I still
need, and revert all other changes
- [x] Use the classical Securejoin protocol, rather than the new 2-step
protocol
- [x] Make the Rust tests pass
- [x] Make the Python tests pass
- [x] Fix TODOs in the code
- [x] Test it, and fix any bugs I find
- [x] I found a bug when exporting all profiles at once fails sometimes,
though this bug is unrelated to channels:
https://github.com/chatmail/core/issues/7281
- [x] Do a self-review (i.e. read all changes, and check if I see some
things that should be changed)
- [x] Have this PR reviewed and merged
- [ ] Open an issue for "TODO: There is a known bug in the securejoin
protocol"
- [ ] Create an issue that outlines how we can improve the Securejoin
protocol in the future (I don't have the time to do this right now, but
want to do it sometime in winter)
- [ ] Write a guide for UIs how to adapt to the changes (see
https://github.com/deltachat/deltachat-android/pull/3886)

## Backwards compatibility

This is not very backwards compatible:
- Trying to join a symmetrically-encrypted broadcast channel with an old
device will fail
- If you joined a symmetrically-encrypted broadcast channel with one
device, and use an old core on the other device, then the other device
will show a mostly empty chat (except for two device messages)
- If you created a broadcast channel in the past, then you will get an
error message when trying to send into the channel:

> The up to now "experimental channels feature" is about to become an officially supported one. By that, privacy will be improved, it will become faster, and less traffic will be consumed.
> 
> As we do not guarantee feature-stability for such experiments, this means, that you will need to create the channel again. 
> 
> Here is what to do:
>  • Create a new channel
>  • Tap on the channel name
>  • Tap on "QR Invite Code"
>  • Have all recipients scan the QR code, or send them the link
> 
> If you have any questions, please send an email to delta@merlinux.eu or ask at https://support.delta.chat/.


## The symmetric encryption

Symmetric encryption uses a shared secret. Currently, we use AES128 for
encryption everywhere in Delta Chat, so, this is what I'm using for
broadcast channels (though it wouldn't be hard to switch to AES256).

The secret shared between all members of a broadcast channel has 258
bits of entropy (see `fn create_broadcast_shared_secret` in the code).

Since the shared secrets have more entropy than the AES session keys,
it's not necessary to have a hard-to-compute string2key algorithm, so,
I'm using the string2key algorithm `salted`. This is fast enough that
Delta Chat can just try out all known shared secrets. [^1] In order to
prevent DOS attacks, Delta Chat will not attempt to decrypt with a
string2key algorithm other than `salted` [^2].

## The "Securejoin" protocol that adds members to the channel after they
scanned a QR code

This PR uses the classical securejoin protocol, the same that is also
used for group and 1:1 invitations.

The messages sent back and forth are called `vg-request`,
`vg-auth-required`, `vg-request-with-auth`, and `vg-member-added`. I
considered using the `vc-` prefix, because from a protocol-POV, the
distinction between `vc-` and `vg-` isn't important (as @link2xt pointed
out in an in-person discussion), but
1. it would be weird if groups used `vg-` while broadcasts and 1:1 chats
used `vc-`,
2. we don't have a `vc-member-added` message yet, so, this would mean
one more different kind of message
3. we anyways want to switch to a new securejoin protocol soon, which
will be a backwards incompatible change with a transition phase. When we
do this change, we can make everything `vc-`.



[^1]: In a symmetrically encrypted message, it's not visible which
secret was used to encrypt without trying out all secrets. If this does
turn out to be too slow in the future, then we can remember which secret
was used more recently, and and try the most recent secret first. If
this is still too slow, then we can assign a short, non-unique (~2
characters) id to every shared secret, and send it in cleartext. The
receiving Delta Chat will then only try out shared secrets with this id.
Of course, this would leak a little bit of metadata in cleartext, so, I
would like to avoid it.
[^2]: A DOS attacker could send a message with a lot of encrypted
session keys, all of which use a very hard-to-compute string2key
algorithm. Delta Chat would then try to decrypt all of the encrypted
session keys with all of the known shared secrets. In order to prevent
this, as I said, Delta Chat will not attempt to decrypt with a
string2key algorithm other than `salted`

BREAKING CHANGE: A new QR type AskJoinBroadcast; cloning a broadcast
channel is no longer possible; manually adding a member to a broadcast
channel is no longer possible (only by having them scan a QR code)
This commit is contained in:
Hocuri
2025-11-03 21:02:13 +01:00
committed by GitHub
parent 997e8216bf
commit 5034449009
43 changed files with 2635 additions and 475 deletions
Download Patch File Download Diff File Expand all files Collapse all files

261
src/receive_imf.rs
View File

@@ -14,7 +14,7 @@ use mailparse::SingleInfo;
use num_traits::FromPrimitive;
use regex::Regex;
use crate::chat::{self, Chat, ChatId, ChatIdBlocked, remove_from_chat_contacts_table};
use crate::chat::{self, Chat, ChatId, ChatIdBlocked, save_broadcast_secret};
use crate::config::Config;
use crate::constants::{self, Blocked, Chattype, DC_CHAT_ID_TRASH, EDITED_PREFIX, ShowEmails};
use crate::contact::{self, Contact, ContactId, Origin, mark_contact_id_as_verified};
@@ -25,8 +25,8 @@ use crate::ephemeral::{Timer as EphemeralTimer, stock_ephemeral_timer_changed};
use crate::events::EventType;
use crate::headerdef::{HeaderDef, HeaderDefMap};
use crate::imap::{GENERATED_PREFIX, markseen_on_imap_table};
use crate::key::self_fingerprint_opt;
use crate::key::{DcKey, Fingerprint};
use crate::key::{self_fingerprint, self_fingerprint_opt};
use crate::log::LogExt;
use crate::log::{info, warn};
use crate::logged_debug_assert;
@@ -43,7 +43,7 @@ use crate::simplify;
use crate::stats::STATISTICS_BOT_EMAIL;
use crate::stock_str;
use crate::sync::Sync::*;
use crate::tools::{self, buf_compress, remove_subject_prefix};
use crate::tools::{self, buf_compress, remove_subject_prefix, validate_broadcast_secret};
use crate::{chatlist_events, ensure_and_debug_assert, ensure_and_debug_assert_eq, location};
/// This is the struct that is returned after receiving one email (aka MIME message).
@@ -320,7 +320,7 @@ async fn get_to_and_past_contact_ids(
.await?;
}
}
ChatAssignment::Trash | ChatAssignment::MailingListOrBroadcast => {
ChatAssignment::Trash => {
to_ids = Vec::new();
past_ids = Vec::new();
}
@@ -385,7 +385,10 @@ async fn get_to_and_past_contact_ids(
)
.await?;
}
ChatAssignment::OneOneChat => {
// Sometimes, messages are sent just to a single recipient
// in a broadcast (e.g. securejoin messages).
// In this case, we need to look them up like in a 1:1 chat:
ChatAssignment::OneOneChat | ChatAssignment::MailingListOrBroadcast => {
let pgp_to_ids = add_or_lookup_key_contacts(
context,
&mime_parser.recipients,
@@ -681,12 +684,13 @@ pub(crate) async fn receive_imf_inner(
handle_securejoin_handshake(context, &mut mime_parser, from_id)
.await
.context("error in Secure-Join message handling")?
} else {
let to_id = to_ids.first().copied().flatten().unwrap_or(ContactId::SELF);
} else if let Some(to_id) = to_ids.first().copied().flatten() {
// handshake may mark contacts as verified and must be processed before chats are created
observe_securejoin_on_other_device(context, &mime_parser, to_id)
.await
.context("error in Secure-Join watching")?
} else {
securejoin::HandshakeMessage::Propagate
};
match res {
@@ -1370,6 +1374,7 @@ async fn do_chat_assignment(
create_or_lookup_mailinglist_or_broadcast(
context,
allow_creation,
create_blocked,
mailinglist_header,
from_id,
mime_parser,
@@ -1510,17 +1515,34 @@ async fn do_chat_assignment(
// (it can't be a mailing list, since it's outgoing)
if let Some(mailinglist_header) = mime_parser.get_mailinglist_header() {
let listid = mailinglist_header_listid(mailinglist_header)?;
chat_id = Some(
if let Some((id, ..)) = chat::get_chat_id_by_grpid(context, &listid).await?
if let Some((id, ..)) = chat::get_chat_id_by_grpid(context, &listid).await? {
chat_id = Some(id);
} else {
// Looks like we missed the sync message that was creating this broadcast channel
let name =
compute_mailinglist_name(mailinglist_header, &listid, mime_parser);
if let Some(secret) = mime_parser
.get_header(HeaderDef::ChatBroadcastSecret)
.filter(|s| validate_broadcast_secret(s))
{
id
} else {
chat_created = true;
let name =
compute_mailinglist_name(mailinglist_header, &listid, mime_parser);
chat::create_broadcast_ex(context, Nosync, listid, name).await?
},
);
chat_id = Some(
chat::create_out_broadcast_ex(
context,
Nosync,
listid,
name,
secret.to_string(),
)
.await?,
);
} else {
warn!(
context,
"Not creating outgoing broadcast with id {listid}, because secret is unknown"
);
}
}
}
}
ChatAssignment::AdHocGroup => {
@@ -1614,8 +1636,8 @@ async fn add_parts(
is_partial_download: Option<u32>,
mut replace_msg_id: Option<MsgId>,
prevent_rename: bool,
chat_id: ChatId,
chat_id_blocked: Blocked,
mut chat_id: ChatId,
mut chat_id_blocked: Blocked,
is_dc_message: MessengerMessage,
) -> Result<ReceivedMsg> {
let to_id = if mime_parser.incoming {
@@ -1648,6 +1670,18 @@ async fn add_parts(
for part in &mut mime_parser.parts {
part.param.set(Param::OverrideSenderDisplayname, name);
}
if chat.typ == Chattype::InBroadcast {
warn!(
context,
"Not assigning msg '{rfc724_mid}' to broadcast {chat_id}: wrong sender: {from_id}."
);
let direct_chat =
ChatIdBlocked::get_for_contact(context, from_id, Blocked::Request).await?;
chat_id = direct_chat.id;
chat_id_blocked = direct_chat.blocked;
chat = Chat::load_from_db(context, chat_id).await?;
}
}
}
@@ -2790,20 +2824,18 @@ async fn apply_group_changes(
!chat_contacts.contains(&ContactId::SELF) || chat_contacts.contains(&from_id);
if let Some(removed_addr) = mime_parser.get_header(HeaderDef::ChatGroupMemberRemoved) {
// TODO: if address "alice@example.org" is a member of the group twice,
// with old and new key,
// and someone (maybe Alice's new contact) just removed Alice's old contact,
// we may lookup the wrong contact because we only look up by the address.
// The result is that info message may contain the new Alice's display name
// rather than old display name.
// This could be fixed by looking up the contact with the highest
// `remove_timestamp` after applying Chat-Group-Member-Timestamps.
if !is_from_in_chat {
better_msg = Some(String::new());
} else if let Some(id) =
lookup_key_contact_by_address(context, removed_addr, Some(chat.id)).await?
} else if let Some(removed_fpr) =
mime_parser.get_header(HeaderDef::ChatGroupMemberRemovedFpr)
{
removed_id = Some(id);
removed_id = lookup_key_contact_by_fingerprint(context, removed_fpr).await?;
} else {
// Removal message sent by a legacy Delta Chat client.
removed_id =
lookup_key_contact_by_address(context, removed_addr, Some(chat.id)).await?;
}
if let Some(id) = removed_id {
better_msg = if id == from_id {
silent = true;
Some(stock_str::msg_group_left_local(context, from_id).await)
@@ -2819,8 +2851,8 @@ async fn apply_group_changes(
} else if let Some(key) = mime_parser.gossiped_keys.get(added_addr) {
// TODO: if gossiped keys contain the same address multiple times,
// we may lookup the wrong contact.
// This could be fixed by looking up the contact with
// highest `add_timestamp` to disambiguate.
// This can be fixed by looking at ChatGroupMemberAddedFpr,
// just like we look at ChatGroupMemberRemovedFpr.
// The result of the error is that info message
// may contain display name of the wrong contact.
let fingerprint = key.public_key.dc_fingerprint().hex();
@@ -2973,6 +3005,7 @@ async fn apply_group_changes(
if !added_ids.remove(&added_id) && added_id != ContactId::SELF {
// No-op "Member added" message. An exception is self-addition messages because they at
// least must be shown when a chat is created on our side.
info!(context, "No-op 'Member added' message (TRASH)");
better_msg = Some(String::new());
}
}
@@ -3175,6 +3208,7 @@ fn mailinglist_header_listid(list_id_header: &str) -> Result<String> {
async fn create_or_lookup_mailinglist_or_broadcast(
context: &Context,
allow_creation: bool,
create_blocked: Blocked,
list_id_header: &str,
from_id: ContactId,
mime_parser: &MimeMessage,
@@ -3200,25 +3234,19 @@ async fn create_or_lookup_mailinglist_or_broadcast(
};
if allow_creation {
// list does not exist but should be created
// Broadcast channel / mailinglist does not exist but should be created
let param = mime_parser.list_post.as_ref().map(|list_post| {
let mut p = Params::new();
p.set(Param::ListPost, list_post);
p.to_string()
});
let is_bot = context.get_config_bool(Config::Bot).await?;
let blocked = if is_bot {
Blocked::Not
} else {
Blocked::Request
};
let chat_id = ChatId::create_multiuser_record(
context,
chattype,
&listid,
name,
blocked,
create_blocked,
param,
mime_parser.timestamp_sent,
)
@@ -3230,13 +3258,6 @@ async fn create_or_lookup_mailinglist_or_broadcast(
)
})?;
chat::add_to_chat_contacts_table(
context,
mime_parser.timestamp_sent,
chat_id,
&[ContactId::SELF],
)
.await?;
if chattype == Chattype::InBroadcast {
chat::add_to_chat_contacts_table(
context,
@@ -3246,7 +3267,12 @@ async fn create_or_lookup_mailinglist_or_broadcast(
)
.await?;
}
Ok(Some((chat_id, blocked, true)))
context.emit_event(EventType::ChatModified(chat_id));
chatlist_events::emit_chatlist_changed(context);
chatlist_events::emit_chatlist_item_changed(context, chat_id);
Ok(Some((chat_id, create_blocked, true)))
} else {
info!(context, "Creating list forbidden by caller.");
Ok(None)
@@ -3399,19 +3425,77 @@ async fn apply_out_broadcast_changes(
) -> Result<GroupChangesInfo> {
ensure!(chat.typ == Chattype::OutBroadcast);
if let Some(_removed_addr) = mime_parser.get_header(HeaderDef::ChatGroupMemberRemoved) {
// The sender of the message left the broadcast channel
remove_from_chat_contacts_table(context, chat.id, from_id).await?;
let mut send_event_chat_modified = false;
let mut better_msg = None;
return Ok(GroupChangesInfo {
better_msg: Some("".to_string()),
added_removed_id: None,
silent: true,
extra_msgs: vec![],
});
if from_id == ContactId::SELF {
apply_chat_name_and_avatar_changes(
context,
mime_parser,
from_id,
chat,
&mut send_event_chat_modified,
&mut better_msg,
)
.await?;
}
Ok(GroupChangesInfo::default())
if let Some(added_fpr) = mime_parser.get_header(HeaderDef::ChatGroupMemberAddedFpr) {
if from_id == ContactId::SELF {
let added_id = lookup_key_contact_by_fingerprint(context, added_fpr).await?;
if let Some(added_id) = added_id {
if chat::is_contact_in_chat(context, chat.id, added_id).await? {
info!(context, "No-op broadcast addition (TRASH)");
better_msg.get_or_insert("".to_string());
} else {
chat::add_to_chat_contacts_table(
context,
mime_parser.timestamp_sent,
chat.id,
&[added_id],
)
.await?;
let msg =
stock_str::msg_add_member_local(context, added_id, ContactId::UNDEFINED)
.await;
better_msg.get_or_insert(msg);
send_event_chat_modified = true;
}
} else {
warn!(context, "Failed to find contact with fpr {added_fpr}");
}
}
} else if let Some(removed_fpr) = mime_parser.get_header(HeaderDef::ChatGroupMemberRemovedFpr) {
send_event_chat_modified = true;
let removed_id = lookup_key_contact_by_fingerprint(context, removed_fpr).await?;
if removed_id == Some(from_id) {
// The sender of the message left the broadcast channel
// Silently remove them without notifying the user
chat::remove_from_chat_contacts_table_without_trace(context, chat.id, from_id).await?;
info!(context, "Broadcast leave message (TRASH)");
better_msg = Some("".to_string());
} else if from_id == ContactId::SELF {
if let Some(removed_id) = removed_id {
chat::remove_from_chat_contacts_table_without_trace(context, chat.id, removed_id)
.await?;
better_msg.get_or_insert(
stock_str::msg_del_member_local(context, removed_id, ContactId::SELF).await,
);
}
}
}
if send_event_chat_modified {
context.emit_event(EventType::ChatModified(chat.id));
chatlist_events::emit_chatlist_item_changed(context, chat.id);
}
Ok(GroupChangesInfo {
better_msg,
added_removed_id: None,
silent: false,
extra_msgs: vec![],
})
}
async fn apply_in_broadcast_changes(
@@ -3422,6 +3506,16 @@ async fn apply_in_broadcast_changes(
) -> Result<GroupChangesInfo> {
ensure!(chat.typ == Chattype::InBroadcast);
if let Some(part) = mime_parser.parts.first() {
if let Some(error) = &part.error {
warn!(
context,
"Not applying broadcast changes from message with error: {error}"
);
return Ok(GroupChangesInfo::default());
}
}
let mut send_event_chat_modified = false;
let mut better_msg = None;
@@ -3435,11 +3529,58 @@ async fn apply_in_broadcast_changes(
)
.await?;
if let Some(_removed_addr) = mime_parser.get_header(HeaderDef::ChatGroupMemberRemoved) {
// The only member added/removed message that is ever sent is "I left.",
// so, this is the only case we need to handle here
if let Some(added_addr) = mime_parser.get_header(HeaderDef::ChatGroupMemberAdded) {
if context.is_self_addr(added_addr).await? {
let msg = if chat.is_self_in_chat(context).await? {
// Self is already in the chat.
// Probably Alice has two devices and her second device added us again;
// just hide the message.
info!(context, "No-op broadcast 'Member added' message (TRASH)");
"".to_string()
} else {
stock_str::msg_you_joined_broadcast(context).await
};
better_msg.get_or_insert(msg);
send_event_chat_modified = true;
}
}
if let Some(removed_fpr) = mime_parser.get_header(HeaderDef::ChatGroupMemberRemovedFpr) {
// We are not supposed to receive a notification when someone else than self is removed:
if removed_fpr != self_fingerprint(context).await? {
logged_debug_assert!(context, false, "Ignoring unexpected removal message");
return Ok(GroupChangesInfo::default());
}
chat::delete_broadcast_secret(context, chat.id).await?;
if from_id == ContactId::SELF {
better_msg.get_or_insert(stock_str::msg_you_left_broadcast(context).await);
} else {
better_msg.get_or_insert(
stock_str::msg_del_member_local(context, ContactId::SELF, from_id).await,
);
}
chat::remove_from_chat_contacts_table_without_trace(context, chat.id, ContactId::SELF)
.await?;
send_event_chat_modified = true;
} else if !chat.is_self_in_chat(context).await? {
chat::add_to_chat_contacts_table(
context,
mime_parser.timestamp_sent,
chat.id,
&[ContactId::SELF],
)
.await?;
send_event_chat_modified = true;
}
if let Some(secret) = mime_parser.get_header(HeaderDef::ChatBroadcastSecret) {
if validate_broadcast_secret(secret) {
save_broadcast_secret(context, chat.id, secret).await?;
} else {
warn!(context, "Not saving invalid broadcast secret");
}
}