5034449009
Follow-up for https://github.com/chatmail/core/pull/7042, part of https://github.com/chatmail/core/issues/6884. This will make it possible to create invite-QR codes for broadcast channels, and make them symmetrically end-to-end encrypted. - [x] Go through all the changes in #7042, and check which ones I still need, and revert all other changes - [x] Use the classical Securejoin protocol, rather than the new 2-step protocol - [x] Make the Rust tests pass - [x] Make the Python tests pass - [x] Fix TODOs in the code - [x] Test it, and fix any bugs I find - [x] I found a bug when exporting all profiles at once fails sometimes, though this bug is unrelated to channels: https://github.com/chatmail/core/issues/7281 - [x] Do a self-review (i.e. read all changes, and check if I see some things that should be changed) - [x] Have this PR reviewed and merged - [ ] Open an issue for "TODO: There is a known bug in the securejoin protocol" - [ ] Create an issue that outlines how we can improve the Securejoin protocol in the future (I don't have the time to do this right now, but want to do it sometime in winter) - [ ] Write a guide for UIs how to adapt to the changes (see https://github.com/deltachat/deltachat-android/pull/3886) ## Backwards compatibility This is not very backwards compatible: - Trying to join a symmetrically-encrypted broadcast channel with an old device will fail - If you joined a symmetrically-encrypted broadcast channel with one device, and use an old core on the other device, then the other device will show a mostly empty chat (except for two device messages) - If you created a broadcast channel in the past, then you will get an error message when trying to send into the channel: > The up to now "experimental channels feature" is about to become an officially supported one. By that, privacy will be improved, it will become faster, and less traffic will be consumed. > > As we do not guarantee feature-stability for such experiments, this means, that you will need to create the channel again. > > Here is what to do: > • Create a new channel > • Tap on the channel name > • Tap on "QR Invite Code" > • Have all recipients scan the QR code, or send them the link > > If you have any questions, please send an email to delta@merlinux.eu or ask at https://support.delta.chat/. ## The symmetric encryption Symmetric encryption uses a shared secret. Currently, we use AES128 for encryption everywhere in Delta Chat, so, this is what I'm using for broadcast channels (though it wouldn't be hard to switch to AES256). The secret shared between all members of a broadcast channel has 258 bits of entropy (see `fn create_broadcast_shared_secret` in the code). Since the shared secrets have more entropy than the AES session keys, it's not necessary to have a hard-to-compute string2key algorithm, so, I'm using the string2key algorithm `salted`. This is fast enough that Delta Chat can just try out all known shared secrets. [^1] In order to prevent DOS attacks, Delta Chat will not attempt to decrypt with a string2key algorithm other than `salted` [^2]. ## The "Securejoin" protocol that adds members to the channel after they scanned a QR code This PR uses the classical securejoin protocol, the same that is also used for group and 1:1 invitations. The messages sent back and forth are called `vg-request`, `vg-auth-required`, `vg-request-with-auth`, and `vg-member-added`. I considered using the `vc-` prefix, because from a protocol-POV, the distinction between `vc-` and `vg-` isn't important (as @link2xt pointed out in an in-person discussion), but 1. it would be weird if groups used `vg-` while broadcasts and 1:1 chats used `vc-`, 2. we don't have a `vc-member-added` message yet, so, this would mean one more different kind of message 3. we anyways want to switch to a new securejoin protocol soon, which will be a backwards incompatible change with a transition phase. When we do this change, we can make everything `vc-`. [^1]: In a symmetrically encrypted message, it's not visible which secret was used to encrypt without trying out all secrets. If this does turn out to be too slow in the future, then we can remember which secret was used more recently, and and try the most recent secret first. If this is still too slow, then we can assign a short, non-unique (~2 characters) id to every shared secret, and send it in cleartext. The receiving Delta Chat will then only try out shared secrets with this id. Of course, this would leak a little bit of metadata in cleartext, so, I would like to avoid it. [^2]: A DOS attacker could send a message with a lot of encrypted session keys, all of which use a very hard-to-compute string2key algorithm. Delta Chat would then try to decrypt all of the encrypted session keys with all of the known shared secrets. In order to prevent this, as I said, Delta Chat will not attempt to decrypt with a string2key algorithm other than `salted` BREAKING CHANGE: A new QR type AskJoinBroadcast; cloning a broadcast channel is no longer possible; manually adding a member to a broadcast channel is no longer possible (only by having them scan a QR code)
The chatmail core library implements low-level network and encryption protocols,
integrated by many chat bots and higher level applications,
allowing to securely participate in the globally scaled e-mail server network.
We provide reproducibly-built deltachat-rpc-server static binaries
that offer a stdio-based high-level JSON-RPC API for instant messaging purposes.
The following protocols are handled without requiring API users to know much about them:
-
secure TLS setup with DNS caching and shadowsocks/proxy support
-
safe and interoperable MIME parsing and MIME building.
-
security-audited end-to-end encryption with rPGP and Autocrypt and SecureJoin protocols
-
ephemeral Peer-to-Peer networking using Iroh for multi-device setup and webxdc realtime data.
-
a simulation- and real-world tested P2P group membership protocol without requiring server state.
Installing Rust and Cargo
To download and install the official compiler for the Rust programming language, and the Cargo package manager, run the command in your user environment:
$ curl https://sh.rustup.rs -sSf | sh
On Windows, you may need to also install Perl to be able to compile deltachat-core.
Using the CLI client
Compile and run the command line utility, using cargo:
$ cargo run --locked -p deltachat-repl -- ~/profile-db
where ~/profile-db is the database file. The utility will create it if it does not exist.
Optionally, install deltachat-repl binary with
$ cargo install --locked --path deltachat-repl/
and run as
$ deltachat-repl ~/profile-db
Configure your account (if not already configured):
Chatmail is awaiting your commands.
> set addr your@email.org
> set mail_pw yourpassword
> configure
Connect to your mail server (if already configured):
> connect
Export your public key to a vCard file:
> make-vcard my.vcard 1
Create contacts by address or vCard file:
> addcontact yourfriends@email.org
> import-vcard key-contact.vcard
List contacts:
> listcontacts
Contact#Contact#11: key-contact@email.org <key-contact@email.org>
Contact#Contact#Self: Me √ <your@email.org>
2 key contacts.
Contact#Contact#10: yourfriends@email.org <yourfriends@email.org>
1 address contacts.
Create a chat with your friend and send a message:
> createchat 10
Single#Chat#12 created successfully.
> chat 12
Selecting chat Chat#12
Single#Chat#12: yourfriends@email.org [yourfriends@email.org] Icon: profile-db-blobs/4138c52e5bc1c576cda7dd44d088c07.png
0 messages.
81.252µs to create this list, 123.625µs to mark all messages as noticed.
> send hi
List messages when inside a chat:
> chat
For more commands type:
> help
Installing libdeltachat system wide
$ git clone https://github.com/chatmail/core.git
$ cd deltachat-core-rust
$ cmake -B build . -DCMAKE_INSTALL_PREFIX=/usr
$ cmake --build build
$ sudo cmake --install build
Development
# run tests
$ cargo test --all
# build c-ffi
$ cargo build -p deltachat_ffi --release
Debugging environment variables
-
DCC_MIME_DEBUG: if set outgoing and incoming message will be printed -
RUST_LOG=async_imap=trace,async_smtp=trace: enable IMAP and SMTP tracing in addition to info messages.
Expensive tests
Some tests are expensive and marked with #[ignore], to run these
use the --ignored argument to the test binary (not to cargo itself):
$ cargo test -- --ignored
Fuzzing
Install cargo-bolero with
$ cargo install cargo-bolero
Run fuzzing tests with
$ cd fuzz
$ cargo bolero test fuzz_mailparse -s NONE
Corpus is created at fuzz/fuzz_targets/corpus,
you can add initial inputs there.
For fuzz_mailparse target corpus can be populated with
../test-data/message/*.eml.
Features
vendored: When using Openssl for TLS, this bundles a vendored version.
Update Provider Data
To add the updates from the
provider-db to the core,
check line REV= inside ./scripts/update-provider-database.sh
and then run the script.
Language bindings and frontend projects
Language bindings are available for:
- C [📂 source | 📚 docs]
- JS: [📂 source | 📦 npm | 📚 docs]
- Python [📂 source | 📦 pypi | 📚 docs]
- Go
- Free Pascal1 [📂 source]
- Java and Swift (contained in the Android/iOS repos)
The following "frontend" projects make use of the Rust-library or its language bindings:
-
Out of date / unmaintained, if you like those languages feel free to start maintaining them. If you have questions we'll help you, please ask in the issues. ↩︎