4dd0ba2c72
With `ORDER BY` statement SQLite searches the `imap` table by `transport_id` and for each found row scans the whole `imap_markseen` table. Number of `imap` entries for each `transport_id` is usually large as we need to know which UIDs to delete on IMAP server when deleting a message. ``` sqlite> EXPLAIN QUERY PLAN SELECT imap.id, uid, folder FROM imap, imap_markseen WHERE imap.id = imap_markseen.id AND imap.transport_id=? AND target = folder ORDER BY folder, uid; QUERY PLAN |--SEARCH imap USING INDEX sqlite_autoindex_imap_1 (transport_id=?) `--SCAN imap_markseen ``` Without `ORDER BY` statement SQLite scans `imap_markseen` table which is expected to be small, and then searches `imap` table by `rowid` for each found result. ``` sqlite> EXPLAIN QUERY PLAN SELECT imap.id, uid, folder FROM imap, imap_markseen WHERE imap.id = imap_markseen.id AND imap.transport_id=? AND target = folder; QUERY PLAN |--SCAN imap_markseen `--SEARCH imap USING INTEGER PRIMARY KEY (rowid=?) ``` Query planning was tested with SQLite 3.52.0. It is possible to explictly make query planner move sorting to the last step with `ORDER +folder, +uid`, but this is not recommended in SQLite documentation (see <https://www.sqlite.org/optoverview.html#uplus>). It is also possible to add indexes, but indexes use space, adding them requires an SQL migration, and each index needs to be updated so it will slow down writes.
The chatmail core library implements low-level network and encryption protocols,
integrated by many chat bots and higher level applications,
allowing to securely participate in the globally scaled e-mail server network.
We provide reproducibly-built deltachat-rpc-server static binaries
that offer a stdio-based high-level JSON-RPC API for instant messaging purposes.
The following protocols are handled without requiring API users to know much about them:
-
secure TLS setup with DNS caching and shadowsocks/proxy support
-
safe and interoperable MIME parsing and MIME building.
-
security-audited end-to-end encryption with rPGP and Autocrypt and SecureJoin protocols
-
ephemeral Peer-to-Peer networking using Iroh for multi-device setup and webxdc realtime data.
-
a simulation- and real-world tested P2P group membership protocol without requiring server state.
Installing Rust and Cargo
To download and install the official compiler for the Rust programming language, and the Cargo package manager, run the command in your user environment:
$ curl https://sh.rustup.rs -sSf | sh
On Windows, you may need to also install Perl to be able to compile deltachat-core.
Using the CLI client
Compile and run the command line utility, using cargo:
$ cargo run --locked -p deltachat-repl -- ~/profile-db
where ~/profile-db is the database file. The utility will create it if it does not exist.
Optionally, install deltachat-repl binary with
$ cargo install --locked --path deltachat-repl/
and run as
$ deltachat-repl ~/profile-db
Configure your account (if not already configured):
Chatmail is awaiting your commands.
> set addr your@email.org
> set mail_pw yourpassword
> configure
Connect to your mail server (if already configured):
> connect
Export your public key to a vCard file:
> make-vcard my.vcard 1
Create contacts by address or vCard file:
> addcontact yourfriends@email.org
> import-vcard key-contact.vcard
List contacts:
> listcontacts
Contact#Contact#11: key-contact@email.org <key-contact@email.org>
Contact#Contact#Self: Me √ <your@email.org>
2 key contacts.
Contact#Contact#10: yourfriends@email.org <yourfriends@email.org>
1 address contacts.
Create a chat with your friend and send a message:
> createchat 10
Single#Chat#12 created successfully.
> chat 12
Selecting chat Chat#12
Single#Chat#12: yourfriends@email.org [yourfriends@email.org] Icon: profile-db-blobs/4138c52e5bc1c576cda7dd44d088c07.png
0 messages.
81.252µs to create this list, 123.625µs to mark all messages as noticed.
> send hi
List messages when inside a chat:
> chat
For more commands type:
> help
Installing libdeltachat system wide
$ git clone https://github.com/chatmail/core.git
$ cd deltachat-core-rust
$ cmake -B build . -DCMAKE_INSTALL_PREFIX=/usr
$ cmake --build build
$ sudo cmake --install build
Development
# run tests
$ cargo test --all
# build c-ffi
$ cargo build -p deltachat_ffi --release
Debugging environment variables
-
DCC_MIME_DEBUG: if set outgoing and incoming message will be printed -
RUST_LOG=async_imap=trace,async_smtp=trace: enable IMAP and SMTP tracing in addition to info messages.
Expensive tests
Some tests are expensive and marked with #[ignore], to run these
use the --ignored argument to the test binary (not to cargo itself):
$ cargo test -- --ignored
Fuzzing
Install cargo-bolero with
$ cargo install cargo-bolero
Run fuzzing tests with
$ cd fuzz
$ cargo bolero test fuzz_mailparse -s NONE
Corpus is created at fuzz/fuzz_targets/corpus,
you can add initial inputs there.
For fuzz_mailparse target corpus can be populated with
../test-data/message/*.eml.
Features
vendored: When using Openssl for TLS, this bundles a vendored version.
Update Provider Data
To add the updates from the
provider-db to the core,
check line REV= inside ./scripts/update-provider-database.sh
and then run the script.
Language bindings and frontend projects
Language bindings are available for:
- C [📂 source | 📚 docs]
- -> libdeltachat is going to be deprecated and only exists because Android, iOS and Ubuntu Touch are still using it. If you build a new project, then please use the jsonrpc api instead.
- JS: [📂 source | 📦 npm | 📚 docs]
- Python [📂 source | 📦 pypi | 📚 docs]
- Go [📂 source]
- Java and Swift (contained in the Android/iOS repos)
The following "frontend" projects make use of the Rust-library or its language bindings: