Simon Laux bfc58e9204 feat: receive pre-messages and adapt download on demand
fix python lint errors

receive pre-mesages, start with changes to imap loop.

refactor: move download code from `scheduler.rs` to `download.rs`, also
move `get_msg_id_by_rfc724_mid` to `MsgId::get_by_rfc724_mid`

`MAX_FETCH_MSG_SIZE` is no longer unused

Parse if it is a pre-message or full-message

start with receiving logic

get rid of `MsgId::get_by_rfc724_mid` because it was a duplicate of
`message::rfc724_mid_exists`

docs: add hint to `MimeMessage::from_bytes` stating that it has
side-effects.

receiving full message

send and receive `attachment_size` and set viewtype to text in
pre_message

metadata as struct in pre-message in header. And fill params that we can
already fill from the metadata. Also add a new api to check what
viewtype the message will have once downloaded.

api: jsonrpc: add `full_message_view_type` to `Message` and
`MessageInfo`

make PreMsgMetadata.to_header_value not consume self/PreMsgMetadata

add api to merge params

on download full message: merge new params into old params and remove
full-message metadata params

move tests to `src/tests/pre_messages.rs`

dynamically allocate test attachment bytes

fix detection of pre-messages. (it looked for the ChatFullMessageId
header in the unencrypted headers before)

fix setting dl state to avaiable on pre-messages

fix: save pre message with rfc724_mid of full message als disable
replacement for full messages

add some receiving tests and update test todo for premessage metadata

test: process full message before pre-message

test receive normal message

some serialization tests for PreMsgMetadata

remove outdated todo comment

test that pre-message contains message text

PreMsgMetadata: test_build_from_file_msg and test_build_from_file_msg

test: test_receive_pre_message_image

Test receiving the full message after receiving an edit after receiving
the pre-message

test_reaction_on_pre_message

test_full_download_after_trashed

test_webxdc_update_for_not_downloaded_instance

simplify fake webxdc generation in
test_webxdc_update_for_not_downloaded_instance

test_markseen_pre_msg

test_pre_msg_can_start_chat and test_full_msg_can_start_chat

test_download_later_keeps_message_order

test_chatlist_event_on_full_msg_download

fix download not working

log splitting into pre-message

add pre-message info to text when loading from db. this can be disabled
with config key `hide_pre_message_metadata_text` if ui wants to display
it in a prettier way.

update `download_limit` documentation

more logging: log size of pre and post messages

rename full message to Post-Message

split up the pre-message tests into multiple files

dedup test code by extracting code to create test messages into util
methods

remove post_message_view_type from api, now it is only used internally
for tests

remove `hide_pre_message_metadata_text` config option, as there
currently is no way to get the full message viewtype anymore

Update src/download.rs
resolve comment

use `parse_message_id` instead of removing `<>`parenthesis it manually

fix available_post_msgs gets no entries
handle forwarding and add a test for it.

convert comment to log warning event on unexpected download failure

add doc comment to `simple_imap_loop`

more logging

handle saving pre-message to self messages and test.
2025-12-09 14:29:17 +00:00
2023-10-29 13:13:44 +00:00
2025-12-05 21:35:21 +00:00
2025-12-05 21:35:21 +00:00
2023-07-04 00:52:31 +02:00
2023-10-29 13:13:44 +00:00

Chatmail logo

Rust CI dependency status

The chatmail core library implements low-level network and encryption protocols, integrated by many chat bots and higher level applications, allowing to securely participate in the globally scaled e-mail server network. We provide reproducibly-built deltachat-rpc-server static binaries that offer a stdio-based high-level JSON-RPC API for instant messaging purposes.

The following protocols are handled without requiring API users to know much about them:

Installing Rust and Cargo

To download and install the official compiler for the Rust programming language, and the Cargo package manager, run the command in your user environment:

$ curl https://sh.rustup.rs -sSf | sh

On Windows, you may need to also install Perl to be able to compile deltachat-core.

Using the CLI client

Compile and run the command line utility, using cargo:

$ cargo run --locked -p deltachat-repl -- ~/profile-db

where ~/profile-db is the database file. The utility will create it if it does not exist.

Optionally, install deltachat-repl binary with

$ cargo install --locked --path deltachat-repl/

and run as

$ deltachat-repl ~/profile-db

Configure your account (if not already configured):

Chatmail is awaiting your commands.
> set addr your@email.org
> set mail_pw yourpassword
> configure

Connect to your mail server (if already configured):

> connect

Export your public key to a vCard file:

> make-vcard my.vcard 1

Create contacts by address or vCard file:

> addcontact yourfriends@email.org
> import-vcard key-contact.vcard

List contacts:

> listcontacts
Contact#Contact#11: key-contact@email.org <key-contact@email.org>
Contact#Contact#Self: Me √ <your@email.org>
2 key contacts.
Contact#Contact#10: yourfriends@email.org <yourfriends@email.org>
1 address contacts.

Create a chat with your friend and send a message:

> createchat 10
Single#Chat#12 created successfully.
> chat 12
Selecting chat Chat#12
Single#Chat#12: yourfriends@email.org [yourfriends@email.org] Icon: profile-db-blobs/4138c52e5bc1c576cda7dd44d088c07.png
0 messages.
81.252µs to create this list, 123.625µs to mark all messages as noticed.
> send hi

List messages when inside a chat:

> chat

For more commands type:

> help

Installing libdeltachat system wide

$ git clone https://github.com/chatmail/core.git
$ cd deltachat-core-rust
$ cmake -B build . -DCMAKE_INSTALL_PREFIX=/usr
$ cmake --build build
$ sudo cmake --install build

Development

# run tests
$ cargo test --all
# build c-ffi
$ cargo build -p deltachat_ffi --release

Debugging environment variables

  • DCC_MIME_DEBUG: if set outgoing and incoming message will be printed

  • RUST_LOG=async_imap=trace,async_smtp=trace: enable IMAP and SMTP tracing in addition to info messages.

Expensive tests

Some tests are expensive and marked with #[ignore], to run these use the --ignored argument to the test binary (not to cargo itself):

$ cargo test -- --ignored

Fuzzing

Install cargo-bolero with

$ cargo install cargo-bolero

Run fuzzing tests with

$ cd fuzz
$ cargo bolero test fuzz_mailparse -s NONE

Corpus is created at fuzz/fuzz_targets/corpus, you can add initial inputs there. For fuzz_mailparse target corpus can be populated with ../test-data/message/*.eml.

Features

  • vendored: When using Openssl for TLS, this bundles a vendored version.

Update Provider Data

To add the updates from the provider-db to the core, check line REV= inside ./scripts/update-provider-database.sh and then run the script.

Language bindings and frontend projects

Language bindings are available for:

The following "frontend" projects make use of the Rust-library or its language bindings:

Description
Chatmail Rust Core library, used by Android/iOS/desktop chatmail apps, bindings and bots 📧
Readme MPL-2.0 103 MiB
Languages
Rust 74.4%
Tcl 9.1%
Python 8.8%
C 4.9%
DIGITAL Command Language 1.1%
Other 1.7%