Compare commits

...
Author SHA1 Message Date
iequidoo 99c2febf13 feat: Disable wal_autocheckpoint
From https://www.sqlite.org/wal.html:
> The default strategy is to allow successive write transactions to grow the WAL until the WAL
  becomes about 1000 pages in size, then to run a checkpoint operation for each subsequent COMMIT
  until the WAL is reset to be smaller than 1000 pages. By default, the checkpoint will be run
  automatically by the same thread that does the COMMIT that pushes the WAL over its size
  limit. This has the effect of causing most COMMIT operations to be very fast but an occasional
  COMMIT (those that trigger a checkpoint) to be much slower.

And while autocheckpoint runs in the `PASSIVE` mode and thus doesn't block concurrent readers and
writers, in our design it blocks writers because it's done under `write_mutex` locked and thus may
cause the app to stuck for noticeable time. Let's disable autocheckpointing then, we can't rely on
it anyway. Instead, run a `TRUNCATE` checkpoint from `inbox_loop()` if the WAL is >= 4K pages and a
`PASSIVE` checkpoint otherwise.
2026-04-21 02:58:12 -03:00
iequidoo 76b51ff8da feat: Trigger housekeeping after opening db
Some migrations want housekeeping to run. Also if housekeeping failed before, fixing the reason and
restarting the program is the most natural way to retry it.
2026-04-19 23:38:18 -03:00
link2xt 3c25e4b726 api: add clear_all_relay_storage API 2026-04-19 13:01:47 +00:00
link2xt 8cd06bb785 fix: use write transaction in SpkiHashStore.cleanup()
query_map_vec() uses read-only connection,
so it cannot be used to delete rows.
2026-04-19 09:15:13 +00:00
link2xt bb816ff398 fix: do not sort prefetched messages by INTERNALDATE
Messages are iterated over in fetch_new_msg_batch()
and largest_uid_fetched variable is updated there
assuming that messages come in the order of increasing UID.
If UIDs are not increasing, it is possible
that largest_uid_fetched will be updated
even though smaller UID is not fetched yet
and the message will be lost.

INTERNALDATE sorting was introduced to
deal with email providers such as Gmail
that keep INTERNALDATE but not the UID
order when moving the messages.
Since we don't move the messages anymore
after commit 04c0e7da16,
there is no need for ordering by INTERNALDATE.
2026-04-19 09:00:40 +00:00
link2xt 9fcb26c849 chore(cargo): upgrade rand 0.8.5 to rand 0.8.6
This upgrade resolves RUSTSEC-2026-0097
2026-04-19 09:00:00 +00:00
B. Petersen d9474a678e fix python test 2026-04-18 23:45:35 +02:00
B. Petersen f1e1a240ac feat: webxdc sending contexts 2026-04-18 23:45:35 +02:00
link2xt 82924952fb feat: allow TLS connections with invalid certificate if the key is unchanged
This change weakens TLS checks.
Every time we make a successful TLS connection,
we remember public key hash from the certificate
in relation to the hostname.
If later we connect to the same hostname and the public key does not change,
we skip checking certificate chain.
This way we will still connect successfully
even if certificate expires or becomes invalid for another reason,
but keeps the key.

We always check that certificate corresponds to the hostname.
We also do this for certificates starting with _
where we allow self-signed certificates,
so self-signed certificates with mismatching domains are not allowed.
Previously we did not check this for domains starting with _.
2026-04-17 18:07:20 +00:00
link2xt 7daa6cc8d9 ci: update Rust to 1.95.0 2026-04-17 15:24:24 +00:00
link2xt b8cfee7e9e build: remove coredeps Dockerfile
It was used to build legacy CFFI Python binding wheels.
We are not building them anymore and it is not
tested that they can still be built.
2026-04-17 15:24:24 +00:00
link2xt 03fc2d26ee ci: remove Concourse CI pipelines
They were used to build legacy Python bindings,
but we stopped updating them and have no Concourse CI setup.
2026-04-17 15:24:24 +00:00
link2xt 942172a31a feat: remove MvboxMove and OnlyFetchMvbox 2026-04-16 16:42:40 +00:00
link2xt 04c0e7da16 feat: do not unconditionally watch mvbox for non-chatmail
Since commit 25750de4e1
released in 2.36.0 we do not move messages to mvbox without explicit
mvbox_move setting, so do not need to watch it as well
as long as other devices are updated to the same change.
2026-04-16 16:42:40 +00:00
iequidoo 4178671839 fix: Scale up contacts messaged in groups to IncomingTo
This makes such contacts appear in the contact list. `IncomingTo` is used because
`ChatId::accept_ex()` does so for groups, so sending a group message is effectively accepting the
group again in regards to contact searchability.

This fixes up b549e7633d which made it impossible to find contacts
from groups even if we've written there.
2026-04-16 12:56:41 -03:00
dependabot[bot] 49e8065b4c chore(deps): bump swatinem/rust-cache from 2.8.2 to 2.9.1
Bumps [swatinem/rust-cache](https://github.com/swatinem/rust-cache) from 2.8.2 to 2.9.1.
- [Release notes](https://github.com/swatinem/rust-cache/releases)
- [Changelog](https://github.com/Swatinem/rust-cache/blob/master/CHANGELOG.md)
- [Commits](https://github.com/swatinem/rust-cache/compare/779680da715d629ac1d338a641029a2f4372abb5...c19371144df3bb44fab255c43d04cbc2ab54d1c4)

---
updated-dependencies:
- dependency-name: swatinem/rust-cache
  dependency-version: 2.9.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-04-16 15:41:17 +00:00
Hocuri 1c24ad91eb feat: Remove the largely-unused ability to send multiple reactions to one message (#8131)
After talking with r10s:

For spring cleaning, remove the largely-unused things that can be done a
bit.

Most private messengers (WhatsApp/Signal/...) do not have this feature,
and we do not want to become Matrix where every client has different,
partly-incompatible features.
2026-04-16 14:30:05 +00:00
link2xt 8f7777b843 ci: upgrade cargo-deny-action to v2.0.17 2026-04-16 16:14:12 +02:00
link2xt caeddcd57b chore: add exception for RUSTSEC-2026-0097 2026-04-16 16:14:12 +02:00
link2xt 418fd92d6d build: upgrade rustls-webpki to 0.103.12
Ignore <https://rustsec.org/advisories/RUSTSEC-2026-0098>
and <https://rustsec.org/advisories/RUSTSEC-2026-0099>
for rustls-webpki 0.102.8 that is an iroh dependency.
2026-04-16 16:14:12 +02:00
iequidoo a70924a1d6 fix: fetch_single_msg(): Lock fetch_msgs_mutex before fetching
This is to avoid race conditions in `receive_imf` which also happened previously when
multi-transport was added.
2026-04-16 10:40:16 -03:00
iequidoo 8e91eecb3d feat: decide_chat_assignment: Log correct post_msg_exists value
rfc724_mid_exists() returns the message id not only when the message is fully downloaded, so we need
to check this additionally. Also improve test_receive_pre_message to check that receiving the
pre-message again doesn't affect the existing message.
2026-04-16 10:40:16 -03:00
iequidooandHocuri 89f948028d fix: Set Param::GuaranteeE2ee before preparing message blob (#8090)
Otherwise `Param::GuaranteeE2ee` is set only after rendering the message
and some UIs, e.g. DC Android, display the message as unencrypted while
preparing the blob and rendering and encrypting the message. NB: DC
Desktop doesn't display the message until `send_msg()` returns.

---------
Co-authored-by: Hocuri <hocuri@gmx.de>
2026-04-15 11:28:53 -03:00
72374 9adb71bf8f feat: Change multiplier to 7/8 when scaling down avatars
The resolution-limits for avatar-images are currently 512x512 or 256x256.
Reducing the resolution further, to 2/3 each step, can reduce the quality
much more than is necessary to fit within the file-size-limits,
which are currently 60 kB or 20 kB.

An image made entirely of noise (which results in unusually
large file-sizes), encoded with jpeg-quality 75,
and 4:2:2-colour-subsampling (the format currently used
for encoding images), can be below 60 kB at 227x227.
For the lower file-size-limit of 20 kB,
such images can be too large at 170x170, but fit at 149x149.
More normal images will have a lower file-size at the same resolution.

Before this change, the target-resolutions for resampling were:
512x512 ->                       341x341 ->                       227x227.

And for the lower file-size-limit:
256x256 ->                       170x170 ->                       113x113.

After this change, the target-resolutions for resampling will be:
512x512 -> 448x448 -> 392x392 -> 343x343 -> 300x300 -> 262x262 -> 229x229.

And for the lower file-size-limit, those will be:
256x256 -> 224x224 -> 196x196 -> 171x171 -> 149x149 -> 130x130 -> 113x113.

This does add 2 steps between the previous target-resolutions,
while still reaching target-resolutions close to the previous ones,
to reduce situations in which the quality will be lower than before.
2026-04-15 13:57:41 +02:00
72374 fe2ba05804 feat: Increase the resolution-limit WORSE_AVATAR_SIZE from 128 to 256
The file-size of many images will already be smaller than 20 kB,
when encoded at 256x256, and it can be a large improvement in quality.
2026-04-15 13:57:41 +02:00
link2xt 072fc34c77 fix: trash no-op messages about self being added to groups 2026-04-15 07:02:15 +00:00
link2xt cb00bd7043 refactor: replace HashSet with BTreeSet
This is needed to avoid golden tests being flaky
because of info message order being different
between runs.
2026-04-15 07:02:15 +00:00
link2xt f766c11075 perf: enable clippy::large_futures lint
Large size of Mimefactor.render() futures
increases the size of all callers
down to set_config() and background_fetch().
I also had to Box::pin one call to fetch_new_msg_batch
and one call to fetch_single_msg.
2026-04-15 05:33:52 +00:00
link2xt 89e450894d refactor: make HTML parser non-async 2026-04-14 12:43:46 +00:00
DarkCat09 6ef1f7d52b fix: restart io on transport deletion
Fixes #8038
2026-04-14 13:11:28 +02:00
dependabot[bot] 33dc3d20ad chore(deps): bump taiki-e/install-action from 2.64.0 to 2.74.0
Bumps [taiki-e/install-action](https://github.com/taiki-e/install-action) from 2.64.0 to 2.74.0.
- [Release notes](https://github.com/taiki-e/install-action/releases)
- [Changelog](https://github.com/taiki-e/install-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/taiki-e/install-action/compare/69e777b377e4ec209ddad9426ae3e0c1008b0ef3...85b24a67ef0c632dfefad70b9d5ce8fddb040754)

---
updated-dependencies:
- dependency-name: taiki-e/install-action
  dependency-version: 2.74.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-04-14 03:36:39 +00:00
dependabot[bot] b8a0c37efe chore(cargo): bump rand from 0.9.2 to 0.9.3
Bumps [rand](https://github.com/rust-random/rand) from 0.9.2 to 0.9.3.
- [Release notes](https://github.com/rust-random/rand/releases)
- [Changelog](https://github.com/rust-random/rand/blob/0.9.3/CHANGELOG.md)
- [Commits](https://github.com/rust-random/rand/compare/rand_core-0.9.2...0.9.3)

---
updated-dependencies:
- dependency-name: rand
  dependency-version: 0.9.3
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-04-14 03:36:12 +00:00
iequidoo ad7f873c68 fix: Ensure that message being sent is added to the bottom (#8027)
Before, if the user fixed their clock incorrectly set to the future, they needed to delete
previously sent messages or wait until this future comes again so that new sent messages are added
to the bottom. Strictly speaking, the problem isn't fixable because we don't know if messages were
incorrectly timestamped into the future or they are timestamped correctly and the clock is now
incorrectly set to the past. Anyway, adding messages to the middle of the chat isn't a good way to
inform the user about the problem.
2026-04-13 08:49:58 -03:00
link2xt 3236c8bbf4 chore: bump version to 2.50.0-dev 2026-04-13 10:14:01 +02:00
link2xt dab7ca19fe chore(release): prepare for 2.49.0 2026-04-13 10:10:07 +02:00
88 changed files with 1337 additions and 1648 deletions
+8 -8
View File
@@ -20,7 +20,7 @@ permissions: {}
env:
RUSTFLAGS: -Dwarnings
RUST_VERSION: 1.94.0
RUST_VERSION: 1.95.0
# Minimum Supported Rust Version
MSRV: 1.88.0
@@ -40,7 +40,7 @@ jobs:
- run: rustup override set $RUST_VERSION
shell: bash
- name: Cache rust cargo artifacts
uses: swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5
uses: swatinem/rust-cache@c19371144df3bb44fab255c43d04cbc2ab54d1c4
- name: Run rustfmt
run: cargo fmt --all -- --check
- name: Run clippy
@@ -59,7 +59,7 @@ jobs:
with:
show-progress: false
persist-credentials: false
- uses: EmbarkStudios/cargo-deny-action@3fd3802e88374d3fe9159b834c7714ec57d6c979
- uses: EmbarkStudios/cargo-deny-action@91bf2b620e09e18d6eb78b92e7861937469acedb
with:
arguments: --workspace --all-features --locked
command: check
@@ -91,7 +91,7 @@ jobs:
show-progress: false
persist-credentials: false
- name: Cache rust cargo artifacts
uses: swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5
uses: swatinem/rust-cache@c19371144df3bb44fab255c43d04cbc2ab54d1c4
- name: Rustdoc
run: cargo doc --document-private-items --no-deps
@@ -134,10 +134,10 @@ jobs:
shell: bash
- name: Cache rust cargo artifacts
uses: swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5
uses: swatinem/rust-cache@c19371144df3bb44fab255c43d04cbc2ab54d1c4
- name: Install nextest
uses: taiki-e/install-action@69e777b377e4ec209ddad9426ae3e0c1008b0ef3
uses: taiki-e/install-action@85b24a67ef0c632dfefad70b9d5ce8fddb040754
with:
tool: nextest
@@ -168,7 +168,7 @@ jobs:
persist-credentials: false
- name: Cache rust cargo artifacts
uses: swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5
uses: swatinem/rust-cache@c19371144df3bb44fab255c43d04cbc2ab54d1c4
- name: Build C library
run: cargo build -p deltachat_ffi
@@ -194,7 +194,7 @@ jobs:
persist-credentials: false
- name: Cache rust cargo artifacts
uses: swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5
uses: swatinem/rust-cache@c19371144df3bb44fab255c43d04cbc2ab54d1c4
- name: Build deltachat-rpc-server
run: cargo build -p deltachat-rpc-server
+1 -1
View File
@@ -25,7 +25,7 @@ jobs:
with:
node-version: 18.x
- name: Add Rust cache
uses: Swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5
uses: Swatinem/rust-cache@c19371144df3bb44fab255c43d04cbc2ab54d1c4
- name: npm install
working-directory: deltachat-jsonrpc/typescript
run: npm install
+62
View File
@@ -1,5 +1,66 @@
# Changelog
## [2.49.0] - 2026-04-13
### Features / Changes
- Flipped Exif orientations ([#8057](https://github.com/chatmail/core/pull/8057)).
### Fixes
- Determine whether a message is an own message by looking at signature. multiple devices can temporarly have different sets of self addresses, and still need to properly recognize incoming versus outgoing messages. Disclaimer: some LLM tooling was initially involved but i went over everything by hand, and also addressed review comments..
- Mark a message as delivered only after it has been fully sent out ([#8062](https://github.com/chatmail/core/pull/8062)).
- Do not create 1:1 chat on second device when scanning a QR code.
- Do not URL-encode proxy hostnames.
- Assign webxdc updates from post-message to webxdc instance.
- Let search also return hidden contacts if search value is an email address.
- Add missing `extern "C"` to `dc_array_is_independent`.
- Make start messages stick to the top of the chat.
- For bots, wait with emitting IncomingMsg until the Post-Msg arrived ([#8104](https://github.com/chatmail/core/pull/8104)).
- Trash message about group name change from non-member.
### API-Changes
- [**breaking**] remove `dc_msg_force_plaintext`.
- @deltachat/stdio-rpc-server: also export a class.
### CI
- Make sure `-dev` version suffix is not forgotten after release.
### Documentation
- Document that events are broadcasted to all event emitters.
- Fix broken link for i-d "Common PGP/MIME Message Mangling".
### Refactor
- ignore ForcePlaintext in saved messages chat.
- @deltachat/stdio-rpc-server: make `getRPCServerPath` and `startDeltaChat` synchronous.
- @deltachat/stdio-rpc-server: remove `await` from README example.
- less nested `remove_contact_from_chat`.
### Tests
- Add test for `tweak_sort_timestamp()`.
- Test that messages are only marked as delivered after being fully sent out ([#8077](https://github.com/chatmail/core/pull/8077)).
- Fix flaky `test_no_old_msg_is_fresh`: Wait for incoming message before sending outgoing one.
- Use TestContextManager in `test_keep_member_list_if_possibly_nomember`.
### Miscellaneous Tasks
- cargo: bump chrono from 0.4.43 to 0.4.44.
- cargo: bump tracing-subscriber from 0.3.22 to 0.3.23.
- cargo: bump tempfile from 3.26.0 to 3.27.0.
- cargo: bump pin-project from 1.1.10 to 1.1.11.
- cargo: bump tokio from 1.49.0 to 1.50.0.
- cargo: bump libc from 0.2.182 to 0.2.183.
- cargo: bump quote from 1.0.44 to 1.0.45.
- cargo: bump image from 0.25.9 to 0.25.10.
- cargo: bump proptest from 1.10.0 to 1.11.0.
- deps: bump dependabot/fetch-metadata from 2.4.0 to 3.0.0.
- bump version to 2.49.0-dev.
## [2.48.0] - 2026-03-30
### Fixes
@@ -8041,3 +8102,4 @@ https://github.com/chatmail/core/pulls?q=is%3Apr+is%3Aclosed
[2.46.0]: https://github.com/chatmail/core/compare/v2.45.0..v2.46.0
[2.47.0]: https://github.com/chatmail/core/compare/v2.46.0..v2.47.0
[2.48.0]: https://github.com/chatmail/core/compare/v2.47.0..v2.48.0
[2.49.0]: https://github.com/chatmail/core/compare/v2.48.0..v2.49.0
Generated
+35 -35
View File
@@ -550,7 +550,7 @@ dependencies = [
"bolero-kani",
"bolero-libfuzzer",
"cfg-if",
"rand 0.9.2",
"rand 0.9.4",
]
[[package]]
@@ -573,7 +573,7 @@ dependencies = [
"bolero-generator",
"lazy_static",
"pretty-hex",
"rand 0.9.2",
"rand 0.9.4",
"rand_xoshiro",
]
@@ -1307,7 +1307,7 @@ dependencies = [
[[package]]
name = "deltachat"
version = "2.49.0-dev"
version = "2.50.0-dev"
dependencies = [
"anyhow",
"astral-tokio-tar",
@@ -1360,8 +1360,8 @@ dependencies = [
"proptest",
"qrcodegen",
"quick-xml",
"rand 0.8.5",
"rand 0.9.2",
"rand 0.8.6",
"rand 0.9.4",
"ratelimit",
"regex",
"rusqlite",
@@ -1416,7 +1416,7 @@ dependencies = [
[[package]]
name = "deltachat-jsonrpc"
version = "2.49.0-dev"
version = "2.50.0-dev"
dependencies = [
"anyhow",
"async-channel 2.5.0",
@@ -1437,7 +1437,7 @@ dependencies = [
[[package]]
name = "deltachat-repl"
version = "2.49.0-dev"
version = "2.50.0-dev"
dependencies = [
"anyhow",
"deltachat",
@@ -1453,7 +1453,7 @@ dependencies = [
[[package]]
name = "deltachat-rpc-server"
version = "2.49.0-dev"
version = "2.50.0-dev"
dependencies = [
"anyhow",
"deltachat",
@@ -1482,7 +1482,7 @@ dependencies = [
[[package]]
name = "deltachat_ffi"
version = "2.49.0-dev"
version = "2.50.0-dev"
dependencies = [
"anyhow",
"deltachat",
@@ -1490,7 +1490,7 @@ dependencies = [
"human-panic",
"libc",
"num-traits",
"rand 0.9.2",
"rand 0.9.4",
"serde_json",
"thiserror 2.0.18",
"tokio",
@@ -2429,7 +2429,7 @@ dependencies = [
"idna",
"ipnet",
"once_cell",
"rand 0.9.2",
"rand 0.9.4",
"ring",
"thiserror 2.0.18",
"tinyvec",
@@ -2451,7 +2451,7 @@ dependencies = [
"moka",
"once_cell",
"parking_lot",
"rand 0.9.2",
"rand 0.9.4",
"resolv-conf",
"smallvec",
"thiserror 2.0.18",
@@ -2852,7 +2852,7 @@ dependencies = [
"hyper",
"hyper-util",
"log",
"rand 0.9.2",
"rand 0.9.4",
"tokio",
"url",
"xmltree",
@@ -2981,7 +2981,7 @@ dependencies = [
"pin-project",
"pkarr",
"portmapper",
"rand 0.8.5",
"rand 0.8.6",
"rcgen",
"reqwest",
"ring",
@@ -3056,7 +3056,7 @@ dependencies = [
"iroh-metrics",
"n0-future",
"postcard",
"rand 0.8.5",
"rand 0.8.6",
"rand_core 0.6.4",
"serde",
"serde-error",
@@ -3119,7 +3119,7 @@ checksum = "929d5d8fa77d5c304d3ee7cae9aede31f13908bd049f9de8c7c0094ad6f7c535"
dependencies = [
"bytes",
"getrandom 0.2.16",
"rand 0.8.5",
"rand 0.8.6",
"ring",
"rustc-hash",
"rustls",
@@ -3172,7 +3172,7 @@ dependencies = [
"pin-project",
"pkarr",
"postcard",
"rand 0.8.5",
"rand 0.8.6",
"reqwest",
"rustls",
"rustls-webpki 0.102.8",
@@ -3776,7 +3776,7 @@ dependencies = [
"num-integer",
"num-iter",
"num-traits",
"rand 0.8.5",
"rand 0.8.6",
"serde",
"smallvec",
"zeroize",
@@ -4206,7 +4206,7 @@ dependencies = [
"p256",
"p384",
"p521",
"rand 0.8.5",
"rand 0.8.6",
"regex",
"replace_with",
"ripemd",
@@ -4453,7 +4453,7 @@ dependencies = [
"nested_enum_utils",
"netwatch",
"num_enum",
"rand 0.8.5",
"rand 0.8.6",
"serde",
"smallvec",
"snafu",
@@ -4621,7 +4621,7 @@ checksum = "4b45fcc2344c680f5025fe57779faef368840d0bd1f42f216291f0dc4ace4744"
dependencies = [
"bitflags 2.11.0",
"num-traits",
"rand 0.9.2",
"rand 0.9.4",
"rand_chacha 0.9.0",
"rand_xorshift",
"regex-syntax",
@@ -4701,7 +4701,7 @@ dependencies = [
"bytes",
"getrandom 0.3.3",
"lru-slab",
"rand 0.9.2",
"rand 0.9.4",
"ring",
"rustc-hash",
"rustls",
@@ -4776,9 +4776,9 @@ dependencies = [
[[package]]
name = "rand"
version = "0.8.5"
version = "0.8.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "34af8d1a0e25924bc5b7c43c079c942339d8f0a8b57c39049bef581b46327404"
checksum = "5ca0ecfa931c29007047d1bc58e623ab12e5590e8c7cc53200d5202b69266d8a"
dependencies = [
"libc",
"rand_chacha 0.3.1",
@@ -4787,9 +4787,9 @@ dependencies = [
[[package]]
name = "rand"
version = "0.9.2"
version = "0.9.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6db2770f06117d490610c7488547d543617b21bfa07796d7a12f6f1bd53850d1"
checksum = "44c5af06bb1b7d3216d91932aed5265164bf384dc89cd6ba05cf59a35f5f76ea"
dependencies = [
"rand_chacha 0.9.0",
"rand_core 0.9.3",
@@ -5164,7 +5164,7 @@ dependencies = [
"once_cell",
"ring",
"rustls-pki-types",
"rustls-webpki 0.103.10",
"rustls-webpki 0.103.12",
"subtle",
"zeroize",
]
@@ -5201,9 +5201,9 @@ dependencies = [
[[package]]
name = "rustls-webpki"
version = "0.103.10"
version = "0.103.12"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "df33b2b81ac578cabaf06b89b0631153a3f416b0a886e8a7a1707fb51abbd1ef"
checksum = "8279bb85272c9f10811ae6a6c547ff594d6a7f3c6c6b02ee9726d1d0dcfcdd06"
dependencies = [
"ring",
"rustls-pki-types",
@@ -5322,7 +5322,7 @@ version = "0.17.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "22c3b0257608d7de4de4c4ea650ccc2e6e3e45e3cd80039fcdee768bcb449253"
dependencies = [
"rand 0.9.2",
"rand 0.9.4",
"substring",
"thiserror 1.0.69",
"url",
@@ -5581,7 +5581,7 @@ dependencies = [
"once_cell",
"percent-encoding",
"pin-project",
"rand 0.9.2",
"rand 0.9.4",
"sendfd",
"serde",
"serde_json",
@@ -5611,7 +5611,7 @@ dependencies = [
"chacha20poly1305",
"hkdf",
"md-5",
"rand 0.9.2",
"rand 0.9.4",
"ring-compat",
"sha1",
]
@@ -5843,7 +5843,7 @@ dependencies = [
"precis-core",
"precis-profiles",
"quoted-string-parser",
"rand 0.9.2",
"rand 0.9.4",
]
[[package]]
@@ -5870,7 +5870,7 @@ dependencies = [
"hex",
"parking_lot",
"pnet_packet",
"rand 0.8.5",
"rand 0.8.6",
"socket2 0.5.9",
"thiserror 1.0.69",
"tokio",
@@ -6247,7 +6247,7 @@ dependencies = [
"getrandom 0.3.3",
"http 1.1.0",
"httparse",
"rand 0.9.2",
"rand 0.9.4",
"ring",
"rustls-pki-types",
"simdutf8",
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "deltachat"
version = "2.49.0-dev"
version = "2.50.0-dev"
edition = "2024"
license = "MPL-2.0"
rust-version = "1.88"
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "deltachat_ffi"
version = "2.49.0-dev"
version = "2.50.0-dev"
description = "Deltachat FFI"
edition = "2018"
readme = "README.md"
+2 -8
View File
@@ -426,14 +426,6 @@ char* dc_get_blobdir (const dc_context_t* context);
* 1=send a copy of outgoing messages to self (default).
* Sending messages to self is needed for a proper multi-account setup,
* however, on the other hand, may lead to unwanted notifications in non-delta clients.
* - `mvbox_move` = 1=detect chat messages,
* move them to the `DeltaChat` folder,
* and watch the `DeltaChat` folder for updates (default),
* 0=do not move chat-messages
* - `only_fetch_mvbox` = 1=Do not fetch messages from folders other than the
* `DeltaChat` folder. Messages will still be fetched from the
* spam folder.
* 0=watch all folders normally (default)
* - `show_emails` = DC_SHOW_EMAILS_OFF (0)=
* show direct replies to chats only,
* DC_SHOW_EMAILS_ACCEPTED_CONTACTS (1)=
@@ -4242,6 +4234,8 @@ char* dc_msg_get_webxdc_blob (const dc_msg_t* msg, const char*
* true if the Webxdc should get internet access;
* this is the case i.e. for experimental maps integration.
* - self_addr: address to be used for `window.webxdc.selfAddr` in JS land.
* - is_app_sender: Define if the local user is the one who initially shared the webxdc application in the chat.
* - is_broadcast: Define if the app runs in a broadcasting context.
* - send_update_interval: Milliseconds to wait before calling `sendUpdate()` again since the last call.
* Should be exposed to `webxdc.sendUpdateInterval` in JS land.
* - send_update_max_size: Maximum number of bytes accepted for a serialized update object.
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "deltachat-jsonrpc"
version = "2.49.0-dev"
version = "2.50.0-dev"
description = "DeltaChat JSON-RPC API"
edition = "2021"
license = "MPL-2.0"
+14 -5
View File
@@ -318,6 +318,15 @@ impl CommandApi {
Ok(())
}
/// Requests to clear storage on all chatmail relays.
///
/// I/O must be started for this request to take effect.
async fn clear_all_relay_storage(&self, account_id: u32) -> Result<()> {
let ctx = self.get_context(account_id).await?;
ctx.clear_all_relay_storage().await?;
Ok(())
}
/// Get top-level info for an account.
async fn get_account_info(&self, account_id: u32) -> Result<Account> {
let context_option = self.accounts.read().await.get_account(account_id);
@@ -2377,12 +2386,12 @@ impl CommandApi {
Ok(message_id.to_u32())
}
/// Send a reaction to message.
/// Sends a reaction to message.
///
/// Reaction is a string of emojis separated by spaces. Reaction to a
/// single message can be sent multiple times. The last reaction
/// received overrides all previously received reactions. It is
/// possible to remove all reactions by sending an empty string.
/// A reaction is a string that represents an emoji.
/// You can call this function again to change the emoji;
/// the last sent reaction overrides all previously sent reactions.
/// It is possible to remove the reaction by sending an empty string.
async fn send_reaction(
&self,
account_id: u32,
+10 -2
View File
@@ -33,6 +33,12 @@ pub struct EnteredLoginParam {
/// Imap server port.
pub imap_port: Option<u16>,
/// IMAP server folder.
///
/// Defaults to "INBOX" if not set.
/// Should not be an empty string.
pub imap_folder: Option<String>,
/// Imap socket security.
pub imap_security: Option<Socket>,
@@ -85,6 +91,7 @@ impl From<dc::EnteredLoginParam> for EnteredLoginParam {
password: param.imap.password,
imap_server: param.imap.server.into_option(),
imap_port: param.imap.port.into_option(),
imap_folder: param.imap.folder.into_option(),
imap_security: imap_security.into_option(),
imap_user: param.imap.user.into_option(),
smtp_server: param.smtp.server.into_option(),
@@ -104,14 +111,15 @@ impl TryFrom<EnteredLoginParam> for dc::EnteredLoginParam {
fn try_from(param: EnteredLoginParam) -> Result<Self> {
Ok(Self {
addr: param.addr,
imap: dc::EnteredServerLoginParam {
imap: dc::EnteredImapLoginParam {
server: param.imap_server.unwrap_or_default(),
port: param.imap_port.unwrap_or_default(),
folder: param.imap_folder.unwrap_or_default(),
security: param.imap_security.unwrap_or_default().into(),
user: param.imap_user.unwrap_or_default(),
password: param.password,
},
smtp: dc::EnteredServerLoginParam {
smtp: dc::EnteredSmtpLoginParam {
server: param.smtp_server.unwrap_or_default(),
port: param.smtp_port.unwrap_or_default(),
security: param.smtp_security.unwrap_or_default().into(),
+9 -18
View File
@@ -24,6 +24,8 @@ pub struct JsonrpcReaction {
#[serde(rename = "Reactions", rename_all = "camelCase")]
pub struct JsonrpcReactions {
/// Map from a contact to it's reaction to message.
/// There is only a single reaction per contact,
/// but this contains a list of reactions for historical reasons.
reactions_by_contact: BTreeMap<u32, Vec<String>>,
/// Unique reactions and their count, sorted in descending order.
reactions: Vec<JsonrpcReaction>,
@@ -31,27 +33,16 @@ pub struct JsonrpcReactions {
impl From<Reactions> for JsonrpcReactions {
fn from(reactions: Reactions) -> Self {
let mut reactions_by_contact: BTreeMap<u32, Vec<String>> = BTreeMap::new();
for contact_id in reactions.contacts() {
let reaction = reactions.get(contact_id);
if reaction.is_empty() {
continue;
}
let emojis: Vec<String> = reaction
.emojis()
.into_iter()
.map(|emoji| emoji.to_owned())
.collect();
reactions_by_contact.insert(contact_id.to_u32(), emojis.clone());
}
let self_reactions = reactions_by_contact.get(&ContactId::SELF.to_u32());
let reactions_by_contact: BTreeMap<u32, Vec<String>> = reactions
.iter()
.map(|(key, value)| (key.to_u32(), vec![value.as_str().to_string()]))
.collect();
let self_reaction = reactions_by_contact.get(&ContactId::SELF.to_u32());
let mut reactions_v = Vec::new();
for (emoji, count) in reactions.emoji_sorted_by_frequency() {
let is_from_self = if let Some(self_reactions) = self_reactions {
self_reactions.contains(&emoji)
let is_from_self = if let Some(self_reaction) = self_reaction {
self_reaction.contains(&emoji)
} else {
false
};
@@ -37,6 +37,10 @@ pub struct WebxdcMessageInfo {
internet_access: bool,
/// Address to be used for `window.webxdc.selfAddr` in JS land.
self_addr: String,
/// Define if the local user is the one who initially shared the webxdc application in the chat.
is_app_sender: bool,
/// Define if the app runs in a broadcasting context.
is_broadcast: bool,
/// Milliseconds to wait before calling `sendUpdate()` again since the last call.
/// Should be exposed to `window.sendUpdateInterval` in JS land.
send_update_interval: usize,
@@ -60,6 +64,8 @@ impl WebxdcMessageInfo {
request_integration: _,
internet_access,
self_addr,
is_app_sender,
is_broadcast,
send_update_interval,
send_update_max_size,
} = message.get_webxdc_info(context).await?;
@@ -72,6 +78,8 @@ impl WebxdcMessageInfo {
source_code_url: maybe_empty_string_to_option(source_code_url),
internet_access,
self_addr,
is_app_sender,
is_broadcast,
send_update_interval,
send_update_max_size,
})
+1 -1
View File
@@ -54,5 +54,5 @@
},
"type": "module",
"types": "dist/deltachat.d.ts",
"version": "2.49.0-dev"
"version": "2.50.0-dev"
}
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "deltachat-repl"
version = "2.49.0-dev"
version = "2.50.0-dev"
license = "MPL-2.0"
edition = "2021"
repository = "https://github.com/chatmail/core"
+1 -1
View File
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "deltachat-rpc-client"
version = "2.49.0-dev"
version = "2.50.0-dev"
license = "MPL-2.0"
description = "Python client for Delta Chat core JSON-RPC interface"
classifiers = [
@@ -25,7 +25,14 @@ class Message:
return self.account._rpc
def send_reaction(self, *reaction: str) -> "Message":
"""Send a reaction to this message."""
"""
Sends a reaction to message.
A reaction is a string that represents an emoji.
You can call this function again to change the emoji;
the last sent reaction overrides all previously sent reactions.
It is possible to remove the reaction by sending an empty string.
"""
msg_id = self._rpc.send_reaction(self.account.id, self.id, reaction)
return Message(self.account, msg_id)
+12 -99
View File
@@ -1,104 +1,26 @@
import logging
import re
import time
import pytest
from imap_tools import AND, U
from deltachat_rpc_client import Contact, EventType, Message
from deltachat_rpc_client import EventType
def test_move_works(acfactory, direct_imap):
ac1, ac2 = acfactory.get_online_accounts(2)
ac2_direct_imap = direct_imap(ac2)
ac2_direct_imap.create_folder("DeltaChat")
ac2.set_config("mvbox_move", "1")
ac2.bring_online()
chat = ac1.create_chat(ac2)
chat.send_text("message1")
# Message is moved to the movebox
ac2.wait_for_event(EventType.IMAP_MESSAGE_MOVED)
# Message is downloaded
msg = ac2.wait_for_incoming_msg().get_snapshot()
assert msg.text == "message1"
def test_reactions_for_a_reordering_move(acfactory, direct_imap):
"""When a batch of messages is moved from Inbox to DeltaChat folder with a single MOVE command,
their UIDs may be reordered (e.g. Gmail is known for that) which led to that messages were
processed by receive_imf in the wrong order, and, particularly, reactions were processed before
messages they refer to and thus dropped.
"""
(ac1,) = acfactory.get_online_accounts(1)
def test_moved_markseen(acfactory, direct_imap, log):
"""Test that message already moved to DeltaChat folder is marked as seen."""
ac1 = acfactory.get_online_account()
addr, password = acfactory.get_credentials()
ac2 = acfactory.get_unconfigured_account()
ac2.add_or_update_transport({"addr": addr, "password": password})
ac2_direct_imap = direct_imap(ac2)
ac2_direct_imap.create_folder("DeltaChat")
ac2.set_config("mvbox_move", "1")
assert ac2.is_configured()
ac2.bring_online()
chat1 = acfactory.get_accepted_chat(ac1, ac2)
ac2.stop_io()
logging.info("sending message + reaction from ac1 to ac2")
msg1 = chat1.send_text("hi")
msg1.wait_until_delivered()
# It's is sad, but messages must differ in their INTERNALDATEs to be processed in the correct
# order by DC, and most (if not all) mail servers provide only seconds precision.
time.sleep(1.1)
react_str = "\N{THUMBS UP SIGN}"
msg1.send_reaction(react_str).wait_until_delivered()
logging.info("moving messages to ac2's DeltaChat folder in the reverse order")
ac2_direct_imap = direct_imap(ac2)
ac2_direct_imap.connect()
for uid in sorted([m.uid for m in ac2_direct_imap.get_all_messages()], reverse=True):
ac2_direct_imap.conn.move(uid, "DeltaChat")
logging.info("receiving messages by ac2")
ac2.start_io()
msg2 = Message(ac2, ac2.wait_for_reactions_changed().msg_id)
assert msg2.get_snapshot().text == msg1.get_snapshot().text
reactions = msg2.get_reactions()
contacts = [Contact(ac2, int(i)) for i in reactions.reactions_by_contact]
assert len(contacts) == 1
assert contacts[0].get_snapshot().address == ac1.get_config("addr")
assert list(reactions.reactions_by_contact.values())[0] == [react_str]
def test_move_works_on_self_sent(acfactory, direct_imap):
ac1, ac2 = acfactory.get_online_accounts(2)
# Create and enable movebox.
ac1_direct_imap = direct_imap(ac1)
ac1_direct_imap.create_folder("DeltaChat")
ac1.set_config("mvbox_move", "1")
ac1.set_config("bcc_self", "1")
ac1.bring_online()
chat = ac1.create_chat(ac2)
chat.send_text("message1")
ac1.wait_for_event(EventType.IMAP_MESSAGE_MOVED)
chat.send_text("message2")
ac1.wait_for_event(EventType.IMAP_MESSAGE_MOVED)
chat.send_text("message3")
ac1.wait_for_event(EventType.IMAP_MESSAGE_MOVED)
def test_moved_markseen(acfactory, direct_imap):
"""Test that message already moved to DeltaChat folder is marked as seen."""
ac1, ac2 = acfactory.get_online_accounts(2)
log.section("ac2: creating DeltaChat folder")
ac2_direct_imap = direct_imap(ac2)
ac2_direct_imap.create_folder("DeltaChat")
ac2.set_config("mvbox_move", "1")
ac2.set_config("delete_server_after", "0")
ac2.set_config("sync_msgs", "0") # Do not send a sync message when accepting a contact request.
ac2.add_or_update_transport({"addr": addr, "password": password, "imapFolder": "DeltaChat"})
ac2.bring_online()
ac2.stop_io()
@@ -108,6 +30,7 @@ def test_moved_markseen(acfactory, direct_imap):
idle2.wait_for_new_message()
# Emulate moving of the message to DeltaChat folder by Sieve rule.
log.section("ac2: moving message into DeltaChat folder")
ac2_direct_imap.conn.move(["*"], "DeltaChat")
ac2_direct_imap.select_folder("DeltaChat")
assert len(list(ac2_direct_imap.conn.fetch("*", mark_seen=False))) == 1
@@ -131,17 +54,11 @@ def test_moved_markseen(acfactory, direct_imap):
assert len(list(ac2_direct_imap.conn.fetch(AND(seen=True, uid=U(1, "*")), mark_seen=False))) == 1
@pytest.mark.parametrize("mvbox_move", [True, False])
def test_markseen_message_and_mdn(acfactory, direct_imap, mvbox_move):
def test_markseen_message_and_mdn(acfactory, direct_imap):
ac1, ac2 = acfactory.get_online_accounts(2)
for ac in ac1, ac2:
ac.set_config("delete_server_after", "0")
if mvbox_move:
ac_direct_imap = direct_imap(ac)
ac_direct_imap.create_folder("DeltaChat")
ac.set_config("mvbox_move", "1")
ac.bring_online()
# Do not send BCC to self, we only want to test MDN on ac1.
ac1.set_config("bcc_self", "0")
@@ -150,10 +67,7 @@ def test_markseen_message_and_mdn(acfactory, direct_imap, mvbox_move):
msg = ac2.wait_for_incoming_msg()
msg.mark_seen()
if mvbox_move:
rex = re.compile("Marked messages [0-9]+ in folder DeltaChat as seen.")
else:
rex = re.compile("Marked messages [0-9]+ in folder INBOX as seen.")
rex = re.compile("Marked messages [0-9]+ in folder INBOX as seen.")
for ac in ac1, ac2:
while True:
@@ -161,12 +75,11 @@ def test_markseen_message_and_mdn(acfactory, direct_imap, mvbox_move):
if event.kind == EventType.INFO and rex.search(event.msg):
break
folder = "mvbox" if mvbox_move else "inbox"
ac1_direct_imap = direct_imap(ac1)
ac2_direct_imap = direct_imap(ac2)
ac1_direct_imap.select_config_folder(folder)
ac2_direct_imap.select_config_folder(folder)
ac1_direct_imap.select_folder("INBOX")
ac2_direct_imap.select_folder("INBOX")
# Check that the mdn is marked as seen
assert len(list(ac1_direct_imap.conn.fetch(AND(seen=True), mark_seen=False))) == 1
@@ -9,10 +9,6 @@ def test_add_second_address(acfactory) -> None:
account = acfactory.new_configured_account()
assert len(account.list_transports()) == 1
# When the first transport is created,
# mvbox_move and only_fetch_mvbox should be disabled.
assert account.get_config("mvbox_move") == "0"
assert account.get_config("only_fetch_mvbox") == "0"
assert account.get_config("show_emails") == "2"
qr = acfactory.get_account_qr()
@@ -32,32 +28,10 @@ def test_add_second_address(acfactory) -> None:
account.delete_transport(second_addr)
assert len(account.list_transports()) == 2
# Enabling mvbox_move or only_fetch_mvbox
# is not allowed when multi-transport is enabled.
for option in ["mvbox_move", "only_fetch_mvbox"]:
with pytest.raises(JsonRpcError):
account.set_config(option, "1")
# show_emails does not matter for multi-relay, can be set to anything
account.set_config("show_emails", "0")
@pytest.mark.parametrize("key", ["mvbox_move", "only_fetch_mvbox"])
def test_no_second_transport_with_mvbox(acfactory, key) -> None:
"""Test that second transport cannot be configured if mvbox is used."""
account = acfactory.new_configured_account()
assert len(account.list_transports()) == 1
assert account.get_config("mvbox_move") == "0"
assert account.get_config("only_fetch_mvbox") == "0"
qr = acfactory.get_account_qr()
account.set_config(key, "1")
with pytest.raises(JsonRpcError):
account.add_transport_from_qr(qr)
def test_second_transport_without_classic_emails(acfactory) -> None:
"""Test that second transport can be configured if classic emails are not fetched."""
account = acfactory.new_configured_account()
@@ -147,44 +121,13 @@ def test_download_on_demand(acfactory) -> None:
assert msg.get_snapshot().download_state == dstate
@pytest.mark.parametrize("is_chatmail", ["0", "1"])
def test_mvbox_move_first_transport(acfactory, is_chatmail) -> None:
"""Test that mvbox_move is disabled by default even for non-chatmail accounts.
Disabling mvbox_move is required to be able to setup a second transport.
"""
account = acfactory.get_unconfigured_account()
account.set_config("fix_is_chatmail", "1")
account.set_config("is_chatmail", is_chatmail)
# The default value when the setting is unset is "1".
# This is not changed for compatibility with old databases
# imported from backups.
assert account.get_config("mvbox_move") == "1"
qr = acfactory.get_account_qr()
account.add_transport_from_qr(qr)
# Once the first transport is set up,
# mvbox_move is disabled.
assert account.get_config("mvbox_move") == "0"
assert account.get_config("is_chatmail") == is_chatmail
def test_reconfigure_transport(acfactory) -> None:
"""Test that reconfiguring the transport works
even if settings not supported for multi-transport
like mvbox_move are enabled."""
"""Test that reconfiguring the transport works."""
account = acfactory.get_online_account()
account.set_config("mvbox_move", "1")
[transport] = account.list_transports()
account.add_or_update_transport(transport)
# Reconfiguring the transport should not reset
# the settings as if when configuring the first transport.
assert account.get_config("mvbox_move") == "1"
def test_transport_synchronization(acfactory, log) -> None:
"""Test synchronization of transports between devices."""
@@ -18,6 +18,8 @@ def test_webxdc(acfactory) -> None:
"sourceCodeUrl": None,
"summary": None,
"selfAddr": webxdc_info["selfAddr"],
"isAppSender": False,
"isBroadcast": False,
"sendUpdateInterval": 1000,
"sendUpdateMaxSize": 18874368,
}
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "deltachat-rpc-server"
version = "2.49.0-dev"
version = "2.50.0-dev"
description = "DeltaChat JSON-RPC server"
edition = "2021"
readme = "README.md"
@@ -15,5 +15,5 @@
},
"type": "module",
"types": "index.d.ts",
"version": "2.49.0-dev"
"version": "2.50.0-dev"
}
+4
View File
@@ -23,7 +23,11 @@ ignore = [
# it is a transitive dependency of iroh 0.35.0
# which depends on ^0.102.
# <https://rustsec.org/advisories/RUSTSEC-2026-0049>
# <https://rustsec.org/advisories/RUSTSEC-2026-0098>
# <https://rustsec.org/advisories/RUSTSEC-2026-0099>
"RUSTSEC-2026-0049",
"RUSTSEC-2026-0098",
"RUSTSEC-2026-0099"
]
[bans]
+1 -1
View File
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "deltachat"
version = "2.49.0-dev"
version = "2.50.0-dev"
license = "MPL-2.0"
description = "Python bindings for the Delta Chat Core library using CFFI against the Rust-implemented libdeltachat"
readme = "README.rst"
-1
View File
@@ -522,7 +522,6 @@ class ACFactory:
ac = self.get_unconfigured_account()
assert "addr" in configdict and "mail_pw" in configdict, configdict
configdict.setdefault("bcc_self", False)
configdict.setdefault("mvbox_move", False)
configdict.setdefault("sync_msgs", False)
configdict.setdefault("delete_server_after", 0)
ac.update_config(configdict)
+10 -10
View File
@@ -52,19 +52,19 @@ class TestOfflineAccountBasic:
def test_set_config_int_conversion(self, acfactory):
ac1 = acfactory.get_unconfigured_account()
ac1.set_config("mvbox_move", False)
assert ac1.get_config("mvbox_move") == "0"
ac1.set_config("mvbox_move", True)
assert ac1.get_config("mvbox_move") == "1"
ac1.set_config("mvbox_move", 0)
assert ac1.get_config("mvbox_move") == "0"
ac1.set_config("mvbox_move", 1)
assert ac1.get_config("mvbox_move") == "1"
ac1.set_config("bcc_self", False)
assert ac1.get_config("bcc_self") == "0"
ac1.set_config("bcc_self", True)
assert ac1.get_config("bcc_self") == "1"
ac1.set_config("bcc_self", 0)
assert ac1.get_config("bcc_self") == "0"
ac1.set_config("bcc_self", 1)
assert ac1.get_config("bcc_self") == "1"
def test_update_config(self, acfactory):
ac1 = acfactory.get_unconfigured_account()
ac1.update_config({"mvbox_move": False})
assert ac1.get_config("mvbox_move") == "0"
ac1.update_config({"bcc_self": True})
assert ac1.get_config("bcc_self") == "1"
def test_has_bccself(self, acfactory):
ac1 = acfactory.get_unconfigured_account()
+1 -1
View File
@@ -1 +1 @@
2026-04-08
2026-04-13
-16
View File
@@ -48,19 +48,3 @@ the build machine (ask your friendly sysadmin on #deltachat Libera Chat) to type
This will **rsync** your current checkout to the remote build machine
(no need to commit before) and then run either rust or python tests.
# coredeps Dockerfile
`coredeps/Dockerfile` specifies an image that contains all
of Delta Chat's core dependencies. It is used to
build python wheels (binary packages for Python).
You can build the docker images yourself locally
to avoid the relatively large download:
cd scripts # where all CI things are
docker build -t deltachat/coredeps coredeps
Additionally, you can install qemu and build arm64 docker image on x86\_64 machine:
apt-get install qemu binfmt-support qemu-user-static
docker build -t deltachat/coredeps-arm64 --build-arg BASEIMAGE=quay.io/pypa/manylinux2014_aarch64 coredeps
-26
View File
@@ -1,26 +0,0 @@
# Concourse CI pipeline
`docs_wheels.yml` is a pipeline for [Concourse CI](https://concourse-ci.org/)
that builds C documentation, Python documentation, Python wheels for `x86_64`
and `aarch64` and Python source packages, and uploads them.
To setup the pipeline run
```
fly -t <your-target> set-pipeline -c docs_wheels.yml -p docs_wheels -l secret.yml
```
where `secret.yml` contains the following secrets:
```
c.delta.chat:
private_key: |
-----BEGIN OPENSSH PRIVATE KEY-----
...
-----END OPENSSH PRIVATE KEY-----
devpi:
login: dc
password: ...
```
Secrets can be read from the password manager:
```
fly -t b1 set-pipeline -c docs_wheels.yml -p docs_wheels -l <(pass show delta/b1.delta.chat/secret.yml)
```
-305
View File
@@ -1,305 +0,0 @@
resources:
- name: deltachat-core-rust
type: git
icon: github
source:
branch: main
uri: https://github.com/chatmail/core.git
- name: deltachat-core-rust-release
type: git
icon: github
source:
branch: main
uri: https://github.com/chatmail/core.git
tag_filter: "v*"
jobs:
- name: python-x86_64
plan:
- get: deltachat-core-rust
- get: deltachat-core-rust-release
trigger: true
# Build manylinux image with additional dependencies
- task: build-coredeps
privileged: true
config:
inputs:
# Building the latest, not tagged coredeps
- name: deltachat-core-rust
image_resource:
source:
repository: concourse/oci-build-task
type: registry-image
outputs:
- name: coredeps-image
path: image
params:
CONTEXT: deltachat-core-rust/scripts/coredeps
UNPACK_ROOTFS: "true"
BUILD_ARG_BASEIMAGE: quay.io/pypa/manylinux2014_x86_64
platform: linux
caches:
- path: cache
run:
path: build
# Use built image to build python wheels
- task: build-wheels
image: coredeps-image
config:
inputs:
- name: deltachat-core-rust-release
path: .
outputs:
# Binary wheels
- name: py-wheels
path: ./python/.docker-tox/wheelhouse/
platform: linux
run:
path: bash
args:
- -exc
- |
scripts/run_all.sh
# Upload x86_64 wheels and source packages
- task: upload-wheels
config:
inputs:
- name: py-wheels
image_resource:
type: registry-image
source:
repository: debian
platform: linux
run:
path: sh
args:
- -ec
- |
apt-get update -y
apt-get install -y --no-install-recommends python3-pip python3-setuptools python3-venv
python3 -m venv env
env/bin/pip install --upgrade pip
env/bin/pip install devpi
env/bin/devpi use https://m.devpi.net/dc/master
env/bin/devpi login ((devpi.login)) --password ((devpi.password))
env/bin/devpi upload py-wheels/*manylinux201*
- name: python-aarch64
plan:
- get: deltachat-core-rust
- get: deltachat-core-rust-release
trigger: true
# Build manylinux image with additional dependencies
- task: build-coredeps
privileged: true
config:
inputs:
# Building the latest, not tagged coredeps
- name: deltachat-core-rust
image_resource:
source:
repository: concourse/oci-build-task
type: registry-image
outputs:
- name: coredeps-image
path: image
params:
CONTEXT: deltachat-core-rust/scripts/coredeps
UNPACK_ROOTFS: "true"
BUILD_ARG_BASEIMAGE: quay.io/pypa/manylinux2014_aarch64
platform: linux
caches:
- path: cache
run:
path: build
# Use built image to build python wheels
- task: build-wheels
image: coredeps-image
config:
inputs:
- name: deltachat-core-rust-release
path: .
outputs:
- name: py-wheels
path: ./python/.docker-tox/wheelhouse/
platform: linux
run:
path: bash
args:
- -exc
- |
scripts/run_all.sh
# Upload aarch64 wheels
- task: upload-wheels
config:
inputs:
- name: py-wheels
image_resource:
type: registry-image
source:
repository: debian
platform: linux
run:
path: sh
args:
- -ec
- |
apt-get update -y
apt-get install -y --no-install-recommends python3-pip python3-setuptools python3-venv
python3 -m venv env
env/bin/pip install --upgrade pip
env/bin/pip install devpi
env/bin/devpi use https://m.devpi.net/dc/master
env/bin/devpi login ((devpi.login)) --password ((devpi.password))
env/bin/devpi upload py-wheels/*manylinux201*
- name: python-musl-x86_64
plan:
- get: deltachat-core-rust
- get: deltachat-core-rust-release
trigger: true
# Build manylinux image with additional dependencies
- task: build-coredeps
privileged: true
config:
inputs:
# Building the latest, not tagged coredeps
- name: deltachat-core-rust
image_resource:
source:
repository: concourse/oci-build-task
type: registry-image
outputs:
- name: coredeps-image
path: image
params:
CONTEXT: deltachat-core-rust/scripts/coredeps
UNPACK_ROOTFS: "true"
BUILD_ARG_BASEIMAGE: quay.io/pypa/musllinux_1_1_x86_64
platform: linux
caches:
- path: cache
run:
path: build
# Use built image to build python wheels
- task: build-wheels
image: coredeps-image
config:
inputs:
- name: deltachat-core-rust-release
path: .
outputs:
- name: py-wheels
path: ./python/.docker-tox/wheelhouse/
platform: linux
run:
path: bash
args:
- -exc
- |
scripts/run_all.sh
# Upload musl x86_64 wheels
- task: upload-wheels
config:
inputs:
- name: py-wheels
image_resource:
type: registry-image
source:
repository: debian
platform: linux
run:
path: sh
args:
- -ec
- |
apt-get update -y
apt-get install -y --no-install-recommends python3-pip python3-setuptools python3-venv
python3 -m venv env
env/bin/pip install --upgrade pip
env/bin/pip install devpi
env/bin/devpi use https://m.devpi.net/dc/master
env/bin/devpi login ((devpi.login)) --password ((devpi.password))
env/bin/devpi upload py-wheels/*musllinux_1_1_x86_64*
- name: python-musl-aarch64
plan:
- get: deltachat-core-rust
- get: deltachat-core-rust-release
trigger: true
# Build manylinux image with additional dependencies
- task: build-coredeps
privileged: true
config:
inputs:
# Building the latest, not tagged coredeps
- name: deltachat-core-rust
image_resource:
source:
repository: concourse/oci-build-task
type: registry-image
outputs:
- name: coredeps-image
path: image
params:
CONTEXT: deltachat-core-rust/scripts/coredeps
UNPACK_ROOTFS: "true"
BUILD_ARG_BASEIMAGE: quay.io/pypa/musllinux_1_1_aarch64
platform: linux
caches:
- path: cache
run:
path: build
# Use built image to build python wheels
- task: build-wheels
image: coredeps-image
config:
inputs:
- name: deltachat-core-rust-release
path: .
outputs:
- name: py-wheels
path: ./python/.docker-tox/wheelhouse/
platform: linux
run:
path: bash
args:
- -exc
- |
scripts/run_all.sh
# Upload musl aarch64 wheels
- task: upload-wheels
config:
inputs:
- name: py-wheels
image_resource:
type: registry-image
source:
repository: debian
platform: linux
run:
path: sh
args:
- -ec
- |
apt-get update -y
apt-get install -y --no-install-recommends python3-pip python3-setuptools python3-venv
python3 -m venv env
env/bin/pip install --upgrade pip
env/bin/pip install devpi
env/bin/devpi use https://m.devpi.net/dc/master
env/bin/devpi login ((devpi.login)) --password ((devpi.password))
env/bin/devpi upload py-wheels/*musllinux_1_1_aarch64*
-9
View File
@@ -1,9 +0,0 @@
ARG BASEIMAGE=quay.io/pypa/manylinux2014_x86_64
#ARG BASEIMAGE=quay.io/pypa/musllinux_1_1_x86_64
#ARG BASEIMAGE=quay.io/pypa/manylinux2014_aarch64
FROM $BASEIMAGE
RUN pipx install tox
COPY install-rust.sh /scripts/
RUN /scripts/install-rust.sh
RUN if command -v yum; then yum install -y perl-IPC-Cmd; fi
-20
View File
@@ -1,20 +0,0 @@
#!/usr/bin/env bash
set -euo pipefail
# Install Rust
#
# Path from https://forge.rust-lang.org/infra/other-installation-methods.html
#
# Avoid using rustup here as it depends on reading /proc/self/exe and
# has problems running under QEMU.
RUST_VERSION=1.94.0
ARCH="$(uname -m)"
test -f "/lib/libc.musl-$ARCH.so.1" && LIBC=musl || LIBC=gnu
curl "https://static.rust-lang.org/dist/rust-${RUST_VERSION}-$ARCH-unknown-linux-$LIBC.tar.gz" | tar xz
cd "rust-${RUST_VERSION}-$ARCH-unknown-linux-$LIBC"
./install.sh --prefix=/usr --components=rustc,cargo,"rust-std-$ARCH-unknown-linux-$LIBC"
rustc --version
cd ..
rm -fr "rust-${RUST_VERSION}-$ARCH-unknown-linux-$LIBC"
+1 -1
View File
@@ -465,7 +465,7 @@ impl<'a> BlobObject<'a> {
));
}
target_wh = target_wh * 2 / 3;
target_wh = target_wh * 7 / 8;
} else {
info!(
context,
+35 -16
View File
@@ -1,7 +1,7 @@
//! # Chat module.
use std::cmp;
use std::collections::{BTreeSet, HashMap, HashSet};
use std::collections::{BTreeSet, HashMap};
use std::fmt;
use std::io::Cursor;
use std::marker::Sync;
@@ -1188,7 +1188,6 @@ SELECT id, rfc724_mid, pre_rfc724_mid, timestamp, ?, 1 FROM msgs WHERE chat_id=?
/// prefer plaintext emails.
///
/// To get more verbose summary for a contact, including its key fingerprint, use [`Contact::get_encrinfo`].
#[expect(clippy::arithmetic_side_effects)]
pub async fn get_encryption_info(self, context: &Context) -> Result<String> {
let chat = Chat::load_from_db(context, self).await?;
if !chat.is_encrypted(context).await? {
@@ -1752,7 +1751,6 @@ impl Chat {
///
/// If `update_msg_id` is set, that record is reused;
/// if `update_msg_id` is None, a new record is created.
#[expect(clippy::arithmetic_side_effects)]
async fn prepare_msg_raw(
&mut self,
context: &Context,
@@ -2628,7 +2626,7 @@ pub async fn send_msg(context: &Context, chat_id: ChatId, msg: &mut Message) ->
if msg.state != MessageState::Undefined && msg.state != MessageState::OutPreparing {
msg.param.remove(Param::GuaranteeE2ee);
msg.param.remove(Param::ForcePlaintext);
msg.update_param(context).await?;
// create_send_msg_jobs() will update `param` in the db.
}
// protect all system messages against RTLO attacks
@@ -2733,7 +2731,19 @@ async fn prepare_send_msg(
None
};
// ... then change the MessageState in the message object
if matches!(
msg.state,
MessageState::Undefined | MessageState::OutPreparing
)
// Legacy SecureJoin "v*-request" messages are unencrypted.
&& msg.param.get_cmd() != SystemMessage::SecurejoinMessage
&& chat.is_encrypted(context).await?
{
msg.param.set_int(Param::GuaranteeE2ee, 1);
if !msg.id.is_unset() {
msg.update_param(context).await?;
}
}
msg.state = MessageState::OutPending;
msg.timestamp_sort = create_smeared_timestamp(context);
@@ -2779,15 +2789,13 @@ async fn render_mime_message_and_pre_message(
let mut mimefactory_post_msg = mimefactory.clone();
mimefactory_post_msg.set_as_post_message();
let rendered_msg = mimefactory_post_msg
.render(context)
let rendered_msg = Box::pin(mimefactory_post_msg.render(context))
.await
.context("Failed to render post-message")?;
let mut mimefactory_pre_msg = mimefactory;
mimefactory_pre_msg.set_as_pre_message_for(&rendered_msg);
let rendered_pre_msg = mimefactory_pre_msg
.render(context)
let rendered_pre_msg = Box::pin(mimefactory_pre_msg.render(context))
.await
.context("pre-message failed to render")?;
@@ -2802,7 +2810,7 @@ async fn render_mime_message_and_pre_message(
Ok((Some(rendered_pre_msg), rendered_msg))
} else {
Ok((None, mimefactory.render(context).await?))
Ok((None, Box::pin(mimefactory.render(context)).await?))
}
}
@@ -2931,11 +2939,24 @@ pub(crate) async fn create_send_msg_jobs(context: &Context, msg: &mut Message) -
msg.param.remove(Param::GuaranteeE2ee);
}
msg.subject.clone_from(&rendered_msg.subject);
// Sort the message to the bottom. Employ `msgs_index7` to compute `timestamp`.
context
.sql
.execute(
"UPDATE msgs SET pre_rfc724_mid=?, subject=?, param=? WHERE id=?",
"
UPDATE msgs SET
timestamp=(
SELECT MAX(timestamp) FROM msgs WHERE
-- From `InFresh` to `OutMdnRcvd` inclusive except `OutDraft`.
state IN(10,13,16,18,20,24,26,28) AND
hidden IN(0,1) AND
chat_id=?
),
pre_rfc724_mid=?, subject=?, param=?
WHERE id=?
",
(
msg.chat_id,
&msg.pre_rfc724_mid,
&msg.subject,
msg.param.to_string(),
@@ -3000,7 +3021,6 @@ pub async fn send_text_msg(
}
/// Sends chat members a request to edit the given message's text.
#[expect(clippy::arithmetic_side_effects)]
pub async fn send_edit_request(context: &Context, msg_id: MsgId, new_text: String) -> Result<()> {
let mut original_msg = Message::load_from_db(context, msg_id).await?;
ensure!(
@@ -3763,7 +3783,7 @@ pub(crate) async fn update_chat_contacts_table(
context: &Context,
timestamp: i64,
id: ChatId,
contacts: &HashSet<ContactId>,
contacts: &BTreeSet<ContactId>,
) -> Result<()> {
context
.sql
@@ -4589,7 +4609,6 @@ pub async fn save_msgs(context: &Context, msg_ids: &[MsgId]) -> Result<()> {
/// the copy contains a reference to the original message
/// as well as to the original chat in case the original message gets deleted.
/// Returns data needed to add a `SaveMessage` sync item.
#[expect(clippy::arithmetic_side_effects)]
pub(crate) async fn save_copy_in_self_talk(
context: &Context,
src_msg_id: MsgId,
@@ -5022,7 +5041,7 @@ async fn set_contacts_by_addrs(context: &Context, id: ChatId, addrs: &[String])
chat.typ == Chattype::OutBroadcast,
"{id} is not a broadcast list",
);
let mut contacts = HashSet::new();
let mut contacts = BTreeSet::new();
for addr in addrs {
let contact_addr = ContactAddress::new(addr)?;
let contact = Contact::add_or_lookup(context, "", &contact_addr, Origin::Hidden)
@@ -5030,7 +5049,7 @@ async fn set_contacts_by_addrs(context: &Context, id: ChatId, addrs: &[String])
.0;
contacts.insert(contact);
}
let contacts_old = HashSet::<ContactId>::from_iter(get_chat_contacts(context, id).await?);
let contacts_old = BTreeSet::<ContactId>::from_iter(get_chat_contacts(context, id).await?);
if contacts == contacts_old {
return Ok(());
}
+25
View File
@@ -5027,6 +5027,31 @@ async fn test_do_not_overwrite_draft() -> Result<()> {
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_outgoing_msg_after_another_from_future() -> Result<()> {
let mut tcm = TestContextManager::new();
let t = &tcm.alice().await;
let chat_id = t.get_self_chat().await.id;
// Simulate sending a message with clock set to the future.
SystemTime::shift(Duration::from_secs(3600));
let msg_id = send_text_msg(t, chat_id, "test".to_string()).await?;
SystemTime::shift_back(Duration::from_secs(3600));
let timestamp_sent: i64 = t
.sql
.query_get_value("SELECT timestamp_sent FROM msgs WHERE id=?", (msg_id,))
.await?
.unwrap();
// Let's have a check here that locally sent messages have zero `timestamp_sent`, it can be a
// useful invariant.
assert_eq!(timestamp_sent, 0);
let msg_id = send_text_msg(t, chat_id, "Fixed my clock".to_string()).await?;
assert_eq!(t.get_last_msg_in(chat_id).await.id, msg_id);
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_info_contact_id() -> Result<()> {
let mut tcm = TestContextManager::new();
+1 -40
View File
@@ -155,18 +155,6 @@ pub enum Config {
#[strum(props(default = "1"))]
MdnsEnabled,
/// True if chat messages should be moved to a separate folder. Auto-sent messages like sync
/// ones are moved there anyway.
#[strum(props(default = "1"))]
MvboxMove,
/// Watch for new messages in the "Mvbox" (aka DeltaChat folder) only.
///
/// This will not entirely disable other folders, e.g. the spam folder will also still
/// be watched for new messages.
#[strum(props(default = "0"))]
OnlyFetchMvbox,
/// Whether to show classic emails or only chat messages.
#[strum(props(default = "2"))] // also change ShowEmails.default() on changes
ShowEmails,
@@ -268,9 +256,6 @@ pub enum Config {
/// Configured folder for incoming messages.
ConfiguredInboxFolder,
/// Configured folder for chat messages.
ConfiguredMvboxFolder,
/// Unix timestamp of the last successful configuration.
ConfiguredTimestamp,
@@ -467,7 +452,6 @@ impl Config {
self,
Self::Displayname
| Self::MdnsEnabled
| Self::MvboxMove
| Self::ShowEmails
| Self::Selfavatar
| Self::Selfstatus,
@@ -476,10 +460,7 @@ impl Config {
/// Whether the config option needs an IO scheduler restart to take effect.
pub(crate) fn needs_io_restart(&self) -> bool {
matches!(
self,
Config::MvboxMove | Config::OnlyFetchMvbox | Config::ConfiguredAddr
)
matches!(self, Config::ConfiguredAddr)
}
}
@@ -594,13 +575,6 @@ impl Context {
.is_some_and(|x| x != 0))
}
/// Returns true if movebox ("DeltaChat" folder) should be watched.
pub(crate) async fn should_watch_mvbox(&self) -> Result<bool> {
Ok(self.get_config_bool(Config::MvboxMove).await?
|| self.get_config_bool(Config::OnlyFetchMvbox).await?
|| !self.get_config_bool(Config::IsChatmail).await?)
}
/// Returns true if sync messages should be sent.
pub(crate) async fn should_send_sync_msgs(&self) -> Result<bool> {
Ok(self.get_config_bool(Config::SyncMsgs).await?
@@ -682,8 +656,6 @@ impl Context {
| Config::ProxyEnabled
| Config::BccSelf
| Config::MdnsEnabled
| Config::MvboxMove
| Config::OnlyFetchMvbox
| Config::Configured
| Config::Bot
| Config::NotifyAboutWrongPw
@@ -706,11 +678,6 @@ impl Context {
pub async fn set_config(&self, key: Config, value: Option<&str>) -> Result<()> {
Self::check_config(key, value)?;
let n_transports = self.count_transports().await?;
if n_transports > 1 && matches!(key, Config::MvboxMove | Config::OnlyFetchMvbox) {
bail!("Cannot reconfigure {key} when multiple transports are configured");
}
let _pause = match key.needs_io_restart() {
true => self.scheduler.pause(self).await?,
_ => Default::default(),
@@ -789,12 +756,6 @@ impl Context {
.set_raw_config(key.as_ref(), value.map(|s| s.to_lowercase()).as_deref())
.await?;
}
Config::MvboxMove => {
self.sql.set_raw_config(key.as_ref(), value).await?;
self.sql
.set_raw_config(constants::DC_FOLDERS_CONFIGURED_KEY, None)
.await?;
}
Config::ConfiguredAddr => {
let Some(addr) = value else {
bail!("Cannot unset configured_addr");
+4 -4
View File
@@ -196,11 +196,11 @@ async fn test_sync() -> Result<()> {
sync(&alice0, &alice1).await;
assert_eq!(alice1.get_config_bool(Config::MdnsEnabled).await?, false);
for key in [Config::ShowEmails, Config::MvboxMove] {
let val = alice0.get_config_bool(key).await?;
alice0.set_config_bool(key, !val).await?;
{
let val = alice0.get_config_bool(Config::ShowEmails).await?;
alice0.set_config_bool(Config::ShowEmails, !val).await?;
sync(&alice0, &alice1).await;
assert_eq!(alice1.get_config_bool(key).await?, !val);
assert_eq!(alice1.get_config_bool(Config::ShowEmails).await?, !val);
}
// `Config::SyncMsgs` mustn't be synced.
+10 -27
View File
@@ -261,6 +261,7 @@ impl Context {
.await?;
send_sync_transports(self).await?;
self.quota.write().await.remove(&removed_transport_id);
self.restart_io_if_running().await;
Ok(())
}
@@ -315,31 +316,16 @@ impl Context {
(&param.addr,),
)
.await?
{
// Should be checked before `MvboxMove` because the latter makes no sense in presense of
// `OnlyFetchMvbox` and even grayed out in the UIs in this case.
if self.get_config(Config::OnlyFetchMvbox).await?.as_deref() != Some("0") {
bail!(
"To use additional relays, disable the legacy option \"Settings / Advanced / Only Fetch from DeltaChat Folder\"."
);
}
if self.get_config(Config::MvboxMove).await?.as_deref() != Some("0") {
bail!(
"To use additional relays, disable the legacy option \"Settings / Advanced / Move automatically to DeltaChat Folder\"."
);
}
if self
&& self
.sql
.count("SELECT COUNT(*) FROM transports", ())
.await?
>= MAX_TRANSPORT_RELAYS
{
bail!(
"You have reached the maximum number of relays ({}).",
MAX_TRANSPORT_RELAYS
)
}
{
bail!(
"You have reached the maximum number of relays ({}).",
MAX_TRANSPORT_RELAYS
)
}
let provider = match configure(self, param).await {
@@ -552,6 +538,7 @@ async fn get_configured_param(
.collect(),
imap_user: param.imap.user.clone(),
imap_password: param.imap.password.clone(),
imap_folder: Some(param.imap.folder.clone()).filter(|folder| !folder.is_empty()),
smtp: servers
.iter()
.filter_map(|params| {
@@ -644,10 +631,6 @@ async fn configure(ctx: &Context, param: &EnteredLoginParam) -> Result<Option<&'
progress!(ctx, 900);
let is_configured = ctx.is_configured().await?;
if !is_configured {
ctx.sql.set_raw_config("mvbox_move", Some("0")).await?;
ctx.sql.set_raw_config("only_fetch_mvbox", None).await?;
}
if !ctx.get_config_bool(Config::FixIsChatmail).await? {
if imap_session.is_chatmail() {
ctx.sql.set_raw_config("is_chatmail", Some("1")).await?;
@@ -811,7 +794,7 @@ pub enum Error {
mod tests {
use super::*;
use crate::config::Config;
use crate::login_param::EnteredServerLoginParam;
use crate::login_param::EnteredImapLoginParam;
use crate::test_utils::TestContext;
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
@@ -830,7 +813,7 @@ mod tests {
let entered_param = EnteredLoginParam {
addr: "alice@example.org".to_string(),
imap: EnteredServerLoginParam {
imap: EnteredImapLoginParam {
user: "alice@example.net".to_string(),
password: "foobar".to_string(),
..Default::default()
+1 -6
View File
@@ -199,7 +199,7 @@ pub const WORSE_IMAGE_BYTES: usize = 130_000;
// max. width/height and bytes of an avatar
pub(crate) const BALANCED_AVATAR_SIZE: u32 = 512;
pub(crate) const BALANCED_AVATAR_BYTES: usize = 60_000;
pub(crate) const WORSE_AVATAR_SIZE: u32 = 128;
pub(crate) const WORSE_AVATAR_SIZE: u32 = 256;
pub(crate) const WORSE_AVATAR_BYTES: usize = 20_000; // this also fits to Outlook servers don't allowing headers larger than 32k.
// max. width/height of images scaled down because of being too huge
@@ -210,11 +210,6 @@ pub const WORSE_IMAGE_SIZE: u32 = 640;
/// usage by UIs.
pub const MAX_RCVD_IMAGE_PIXELS: u32 = 50_000_000;
// Key for the folder configuration version (see below).
pub(crate) const DC_FOLDERS_CONFIGURED_KEY: &str = "folders_configured";
// this value can be increased if the folder configuration is changed and must be redone on next program start
pub(crate) const DC_FOLDERS_CONFIGURED_VERSION: i32 = 5;
// If more recipients are needed in SMTP's `RCPT TO:` header, the recipient list is split into
// chunks. This does not affect MIME's `To:` header. Can be overwritten by setting
// `max_smtp_rcpt_to` in the provider db.
-1
View File
@@ -2068,7 +2068,6 @@ pub(crate) async fn mark_contact_id_as_verified(
Ok(())
}
#[expect(clippy::arithmetic_side_effects)]
fn cat_fingerprint(ret: &mut String, name: &str, addr: &str, fingerprint: &str) {
*ret += &format!("\n\n{name} ({addr}):\n{fingerprint}");
}
+45
View File
@@ -4,6 +4,7 @@ use super::*;
use crate::chat::{Chat, get_chat_contacts, send_text_msg};
use crate::chatlist::Chatlist;
use crate::receive_imf::receive_imf;
use crate::securejoin::get_securejoin_qr;
use crate::test_utils::{self, TestContext, TestContextManager, TimeShiftFalsePositiveNote, sync};
#[test]
@@ -154,6 +155,50 @@ async fn test_get_contacts() -> Result<()> {
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_search_contacts_from_group() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
let bob = &tcm.bob().await;
let fiona = &tcm.fiona().await;
let alice_chat_id = chat::create_group(alice, "").await?;
let qr = get_securejoin_qr(alice, Some(alice_chat_id)).await?;
let bob_chat_id = tcm.exec_securejoin_qr(bob, alice, &qr).await;
tcm.exec_securejoin_qr(fiona, alice, &qr).await;
// Workaround for "member added" for fiona not sent to bob.
let gossip_period = alice.get_config_int(Config::GossipPeriod).await?;
SystemTime::shift(Duration::from_secs(gossip_period.try_into()?));
send_text_msg(alice, alice_chat_id, "hello".to_string()).await?;
let sent_msg = alice.pop_sent_msg().await;
bob.recv_msg(&sent_msg).await;
fiona.recv_msg(&sent_msg).await;
let contacts = Contact::get_all(bob, 0, None).await?;
let bob_alice_id = bob.add_or_lookup_contact_id(alice).await;
assert_eq!(contacts.len(), 1);
assert_eq!(contacts[0], bob_alice_id);
let contacts = Contact::get_all(fiona, 0, None).await?;
let fiona_alice_id = fiona.add_or_lookup_contact_id(alice).await;
assert_eq!(contacts.len(), 1);
assert_eq!(contacts[0], fiona_alice_id);
// Sending to the group adds new members to the contact list.
send_text_msg(bob, bob_chat_id, "hello".to_string()).await?;
fiona.recv_msg(&bob.pop_sent_msg().await).await;
let contacts = Contact::get_all(bob, 0, None).await?;
let bob_fiona_id = bob.add_or_lookup_contact_id(fiona).await;
assert_eq!(contacts.len(), 2);
assert_eq!(contacts[0], bob_alice_id);
assert_eq!(contacts[1], bob_fiona_id);
let contacts = Contact::get_all(fiona, 0, None).await?;
assert_eq!(contacts.len(), 1);
assert_eq!(contacts[0], fiona_alice_id);
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_is_self_addr() -> Result<()> {
let t = TestContext::new().await;
+25 -46
View File
@@ -20,16 +20,16 @@ use crate::constants::{self, DC_BACKGROUND_FETCH_QUOTA_CHECK_RATELIMIT, DC_VERSI
use crate::contact::{Contact, ContactId};
use crate::debug_logging::DebugLogging;
use crate::events::{Event, EventEmitter, EventType, Events};
use crate::imap::{FolderMeaning, Imap, ServerMetadata};
use crate::imap::{Imap, ServerMetadata};
use crate::key::self_fingerprint;
use crate::log::warn;
use crate::logged_debug_assert;
use crate::message::{self, MessageState, MsgId};
use crate::net::tls::TlsSessionStore;
use crate::net::tls::{SpkiHashStore, TlsSessionStore};
use crate::peer_channels::Iroh;
use crate::push::PushSubscriber;
use crate::quota::QuotaInfo;
use crate::scheduler::{ConnectivityStore, SchedulerState, convert_folder_meaning};
use crate::scheduler::{ConnectivityStore, SchedulerState};
use crate::sql::Sql;
use crate::stock_str::StockStrings;
use crate::timesmearing::SmearedTimestamp;
@@ -308,6 +308,13 @@ pub struct InnerContext {
/// TLS session resumption cache.
pub(crate) tls_session_store: TlsSessionStore,
/// Store for TLS SPKI hashes.
///
/// Used to remember public keys
/// of TLS certificates to accept them
/// even after they expire.
pub(crate) spki_hash_store: SpkiHashStore,
/// Iroh for realtime peer channels.
pub(crate) iroh: Arc<RwLock<Option<Iroh>>>,
@@ -511,6 +518,7 @@ impl Context {
push_subscriber,
push_subscribed: AtomicBool::new(false),
tls_session_store: TlsSessionStore::new(),
spki_hash_store: SpkiHashStore::new(),
iroh: Arc::new(RwLock::new(None)),
self_fingerprint: OnceLock::new(),
self_public_key: Mutex::new(None),
@@ -560,6 +568,15 @@ impl Context {
}
}
/// Requests deletion of all messages from chatmail relays.
///
/// Non-chatmail relays are excluded
/// to avoid accidentally deleting emails
/// from shared inboxes.
pub async fn clear_all_relay_storage(&self) -> Result<()> {
self.scheduler.clear_all_relay_storage().await
}
/// Restarts the IO scheduler if it was running before
/// when it is not running this is an no-op
pub async fn restart_io_if_running(&self) {
@@ -623,17 +640,10 @@ impl Context {
let mut session = connection.prepare(self).await?;
// Fetch IMAP folders.
// Inbox is fetched before Mvbox because fetching from Inbox
// may result in moving some messages to Mvbox.
for folder_meaning in [FolderMeaning::Inbox, FolderMeaning::Mvbox] {
if let Some((_folder_config, watch_folder)) =
convert_folder_meaning(self, folder_meaning).await?
{
connection
.fetch_move_delete(self, &mut session, &watch_folder, folder_meaning)
.await?;
}
}
let folder = connection.folder.clone();
connection
.fetch_move_delete(self, &mut session, &folder)
.await?;
// Update quota (to send warning if full) - but only check it once in a while.
// note: For now this only checks quota of primary transport,
@@ -644,7 +654,7 @@ impl Context {
DC_BACKGROUND_FETCH_QUOTA_CHECK_RATELIMIT,
)
.await
&& let Err(err) = self.update_recent_quota(&mut session).await
&& let Err(err) = self.update_recent_quota(&mut session, &folder).await
{
warn!(self, "Failed to update quota: {err:#}.");
}
@@ -884,23 +894,6 @@ impl Context {
Err(err) => format!("<key failure: {err}>"),
};
let mvbox_move = self.get_config_int(Config::MvboxMove).await?;
let only_fetch_mvbox = self.get_config_int(Config::OnlyFetchMvbox).await?;
let folders_configured = self
.sql
.get_raw_config_int(constants::DC_FOLDERS_CONFIGURED_KEY)
.await?
.unwrap_or_default();
let configured_inbox_folder = self
.get_config(Config::ConfiguredInboxFolder)
.await?
.unwrap_or_else(|| "<unset>".to_string());
let configured_mvbox_folder = self
.get_config(Config::ConfiguredMvboxFolder)
.await?
.unwrap_or_else(|| "<unset>".to_string());
let mut res = get_info();
// insert values
@@ -976,14 +969,6 @@ impl Context {
.await?
.to_string(),
);
res.insert("mvbox_move", mvbox_move.to_string());
res.insert("only_fetch_mvbox", only_fetch_mvbox.to_string());
res.insert(
constants::DC_FOLDERS_CONFIGURED_KEY,
folders_configured.to_string(),
);
res.insert("configured_inbox_folder", configured_inbox_folder);
res.insert("configured_mvbox_folder", configured_mvbox_folder);
res.insert("mdns_enabled", mdns_enabled.to_string());
res.insert("bcc_self", bcc_self.to_string());
res.insert("sync_msgs", sync_msgs.to_string());
@@ -1297,12 +1282,6 @@ ORDER BY m.timestamp DESC,m.id DESC",
Ok(list)
}
/// Returns true if given folder name is the name of the "DeltaChat" folder.
pub async fn is_mvbox(&self, folder_name: &str) -> Result<bool> {
let mvbox = self.get_config(Config::ConfiguredMvboxFolder).await?;
Ok(mvbox.as_deref() == Some(folder_name))
}
pub(crate) fn derive_blobdir(dbfile: &Path) -> PathBuf {
let mut blob_fname = OsString::new();
blob_fname.push(dbfile.file_name().unwrap_or_default());
+6 -5
View File
@@ -173,9 +173,7 @@ pub(crate) async fn download_msg(
if msg_transport_id != transport_id {
return Ok(None);
}
session
.fetch_single_msg(context, &server_folder, server_uid, rfc724_mid)
.await?;
Box::pin(session.fetch_single_msg(context, &server_folder, server_uid, rfc724_mid)).await?;
Ok(Some(()))
}
@@ -204,8 +202,11 @@ impl Session {
let mut uid_message_ids: BTreeMap<u32, String> = BTreeMap::new();
uid_message_ids.insert(uid, rfc724_mid);
let (sender, receiver) = async_channel::unbounded();
self.fetch_many_msgs(context, folder, vec![uid], &uid_message_ids, sender)
.await?;
{
let _fetch_msgs_lock_guard = context.fetch_msgs_mutex.lock().await;
self.fetch_many_msgs(context, folder, vec![uid], &uid_message_ids, sender)
.await?;
}
if receiver.recv().await.is_err() {
bail!("Failed to fetch UID {uid}");
}
+3 -7
View File
@@ -89,13 +89,9 @@ mod test_chatlist_events {
.get_matching(|evt| match evt {
EventType::ChatlistItemChanged {
chat_id: Some(ev_chat_id),
} => {
if ev_chat_id == &chat_id {
first_event_is_item.store(true, Ordering::Relaxed);
true
} else {
false
}
} if ev_chat_id == &chat_id => {
first_event_is_item.store(true, Ordering::Relaxed);
true
}
EventType::ChatlistChanged => true,
_ => false,
+17 -19
View File
@@ -86,8 +86,7 @@ impl HtmlMsgParser {
/// Function takes a raw mime-message string,
/// searches for the main-text part
/// and returns that as parser.html
#[expect(clippy::arithmetic_side_effects)]
pub async fn from_bytes<'a>(
pub fn from_bytes<'a>(
context: &Context,
rawmime: &'a [u8],
) -> Result<(Self, mailparse::ParsedMail<'a>)> {
@@ -99,14 +98,14 @@ impl HtmlMsgParser {
let parsedmail = mailparse::parse_mail(rawmime).context("Failed to parse mail")?;
parser.collect_texts_recursive(context, &parsedmail).await?;
parser.collect_texts_recursive(context, &parsedmail)?;
if parser.html.is_empty() {
if let Some(plain) = &parser.plain {
parser.html = plain.to_html();
}
} else {
parser.cid_to_data_recursive(context, &parsedmail).await?;
parser.cid_to_data_recursive(context, &parsedmail)?;
}
parser.html += &mem::take(&mut parser.msg_html);
Ok((parser, parsedmail))
@@ -120,8 +119,7 @@ impl HtmlMsgParser {
/// Usually, there is at most one plain-text and one HTML-text part,
/// multiple plain-text parts might be used for mailinglist-footers,
/// therefore we use the first one.
#[expect(clippy::arithmetic_side_effects)]
async fn collect_texts_recursive<'a>(
fn collect_texts_recursive<'a>(
&'a mut self,
context: &'a Context,
mail: &'a mailparse::ParsedMail<'a>,
@@ -129,7 +127,7 @@ impl HtmlMsgParser {
match get_mime_multipart_type(&mail.ctype) {
MimeMultipartType::Multiple => {
for cur_data in &mail.subparts {
Box::pin(self.collect_texts_recursive(context, cur_data)).await?
self.collect_texts_recursive(context, cur_data)?
}
Ok(())
}
@@ -138,7 +136,7 @@ impl HtmlMsgParser {
if raw.is_empty() {
return Ok(());
}
let (parser, mail) = Box::pin(HtmlMsgParser::from_bytes(context, &raw)).await?;
let (parser, mail) = HtmlMsgParser::from_bytes(context, &raw)?;
if !parser.html.is_empty() {
let mut text = "\r\n\r\n".to_string();
for h in mail.headers {
@@ -201,7 +199,7 @@ impl HtmlMsgParser {
/// Replace cid:-protocol by the data:-protocol where appropriate.
/// This allows the final html-file to be self-contained.
async fn cid_to_data_recursive<'a>(
fn cid_to_data_recursive<'a>(
&'a mut self,
context: &'a Context,
mail: &'a mailparse::ParsedMail<'a>,
@@ -209,7 +207,7 @@ impl HtmlMsgParser {
match get_mime_multipart_type(&mail.ctype) {
MimeMultipartType::Multiple => {
for cur_data in &mail.subparts {
Box::pin(self.cid_to_data_recursive(context, cur_data)).await?;
self.cid_to_data_recursive(context, cur_data)?;
}
Ok(())
}
@@ -271,7 +269,7 @@ impl MsgId {
let rawmime = rawmime?;
if !rawmime.is_empty() {
match HtmlMsgParser::from_bytes(context, &rawmime).await {
match HtmlMsgParser::from_bytes(context, &rawmime) {
Err(err) => {
warn!(context, "get_html: parser error: {:#}", err);
Ok(None)
@@ -300,7 +298,7 @@ mod tests {
async fn test_htmlparse_plain_unspecified() {
let t = TestContext::new().await;
let raw = include_bytes!("../test-data/message/text_plain_unspecified.eml");
let (parser, _) = HtmlMsgParser::from_bytes(&t.ctx, raw).await.unwrap();
let (parser, _) = HtmlMsgParser::from_bytes(&t.ctx, raw).unwrap();
assert_eq!(
parser.html,
r#"<!DOCTYPE html>
@@ -318,7 +316,7 @@ This message does not have Content-Type nor Subject.<br/>
async fn test_htmlparse_plain_iso88591() {
let t = TestContext::new().await;
let raw = include_bytes!("../test-data/message/text_plain_iso88591.eml");
let (parser, _) = HtmlMsgParser::from_bytes(&t.ctx, raw).await.unwrap();
let (parser, _) = HtmlMsgParser::from_bytes(&t.ctx, raw).unwrap();
assert_eq!(
parser.html,
r#"<!DOCTYPE html>
@@ -336,7 +334,7 @@ message with a non-UTF-8 encoding: äöüßÄÖÜ<br/>
async fn test_htmlparse_plain_flowed() {
let t = TestContext::new().await;
let raw = include_bytes!("../test-data/message/text_plain_flowed.eml");
let (parser, _) = HtmlMsgParser::from_bytes(&t.ctx, raw).await.unwrap();
let (parser, _) = HtmlMsgParser::from_bytes(&t.ctx, raw).unwrap();
assert!(parser.plain.unwrap().flowed);
assert_eq!(
parser.html,
@@ -358,7 +356,7 @@ and will be wrapped as usual.<br/>
async fn test_htmlparse_alt_plain() {
let t = TestContext::new().await;
let raw = include_bytes!("../test-data/message/text_alt_plain.eml");
let (parser, _) = HtmlMsgParser::from_bytes(&t.ctx, raw).await.unwrap();
let (parser, _) = HtmlMsgParser::from_bytes(&t.ctx, raw).unwrap();
assert_eq!(
parser.html,
r#"<!DOCTYPE html>
@@ -378,7 +376,7 @@ test some special html-characters as &lt; &gt; and &amp; but also &quot; and &#x
async fn test_htmlparse_html() {
let t = TestContext::new().await;
let raw = include_bytes!("../test-data/message/text_html.eml");
let (parser, _) = HtmlMsgParser::from_bytes(&t.ctx, raw).await.unwrap();
let (parser, _) = HtmlMsgParser::from_bytes(&t.ctx, raw).unwrap();
// on windows, `\r\n` linends are returned from mimeparser,
// however, rust multiline-strings use just `\n`;
@@ -396,7 +394,7 @@ test some special html-characters as &lt; &gt; and &amp; but also &quot; and &#x
async fn test_htmlparse_alt_html() {
let t = TestContext::new().await;
let raw = include_bytes!("../test-data/message/text_alt_html.eml");
let (parser, _) = HtmlMsgParser::from_bytes(&t.ctx, raw).await.unwrap();
let (parser, _) = HtmlMsgParser::from_bytes(&t.ctx, raw).unwrap();
assert_eq!(
parser.html.replace('\r', ""), // see comment in test_htmlparse_html()
r##"<html>
@@ -410,7 +408,7 @@ test some special html-characters as &lt; &gt; and &amp; but also &quot; and &#x
async fn test_htmlparse_alt_plain_html() {
let t = TestContext::new().await;
let raw = include_bytes!("../test-data/message/text_alt_plain_html.eml");
let (parser, _) = HtmlMsgParser::from_bytes(&t.ctx, raw).await.unwrap();
let (parser, _) = HtmlMsgParser::from_bytes(&t.ctx, raw).unwrap();
assert_eq!(
parser.html.replace('\r', ""), // see comment in test_htmlparse_html()
r##"<html>
@@ -434,7 +432,7 @@ test some special html-characters as &lt; &gt; and &amp; but also &quot; and &#x
assert!(test.find("data:").is_none());
// parsing converts cid: to data:
let (parser, _) = HtmlMsgParser::from_bytes(&t.ctx, raw).await.unwrap();
let (parser, _) = HtmlMsgParser::from_bytes(&t.ctx, raw).unwrap();
assert!(parser.html.contains("<html>"));
assert!(!parser.html.contains("Content-Id:"));
assert!(parser.html.contains("data:image/jpeg;base64,/9j/4AAQ"));
+47 -245
View File
@@ -27,13 +27,14 @@ use crate::calls::{
use crate::chat::{self, ChatId, ChatIdBlocked, add_device_msg};
use crate::chatlist_events;
use crate::config::Config;
use crate::constants::{self, Blocked, DC_VERSION_STR};
use crate::constants::{Blocked, DC_VERSION_STR};
use crate::contact::ContactId;
use crate::context::Context;
use crate::ensure_and_debug_assert;
use crate::events::EventType;
use crate::headerdef::{HeaderDef, HeaderDefMap};
use crate::log::{LogExt, warn};
use crate::message::{self, Message, MessageState, MessengerMessage, MsgId};
use crate::message::{self, Message, MessageState, MsgId};
use crate::mimeparser;
use crate::net::proxy::ProxyConfig;
use crate::net::session::SessionStream;
@@ -91,6 +92,9 @@ pub(crate) struct Imap {
oauth2: bool,
/// Watched folder.
pub(crate) folder: String,
authentication_failed_once: bool,
pub(crate) connectivity: ConnectivityStore,
@@ -162,7 +166,6 @@ pub enum FolderMeaning {
/// Spam folder.
Spam,
Inbox,
Mvbox,
Trash,
/// Virtual folders.
@@ -174,19 +177,6 @@ pub enum FolderMeaning {
Virtual,
}
impl FolderMeaning {
pub fn to_config(self) -> Option<Config> {
match self {
FolderMeaning::Unknown => None,
FolderMeaning::Spam => None,
FolderMeaning::Inbox => Some(Config::ConfiguredInboxFolder),
FolderMeaning::Mvbox => Some(Config::ConfiguredMvboxFolder),
FolderMeaning::Trash => None,
FolderMeaning::Virtual => None,
}
}
}
struct UidGrouper<T: Iterator<Item = (i64, u32, String)>> {
inner: Peekable<T>,
}
@@ -263,6 +253,11 @@ impl Imap {
let addr = &param.addr;
let strict_tls = param.strict_tls(proxy_config.is_some());
let oauth2 = param.oauth2;
let folder = param
.imap_folder
.clone()
.unwrap_or_else(|| "INBOX".to_string());
ensure_and_debug_assert!(!folder.is_empty(), "Watched folder name cannot be empty");
let (resync_request_sender, resync_request_receiver) = async_channel::bounded(1);
Ok(Imap {
transport_id,
@@ -273,6 +268,7 @@ impl Imap {
proxy_config,
strict_tls,
oauth2,
folder,
authentication_failed_once: false,
connectivity: Default::default(),
conn_last_try: UNIX_EPOCH,
@@ -490,7 +486,7 @@ impl Imap {
/// that folders are created and IMAP capabilities are determined.
pub(crate) async fn prepare(&mut self, context: &Context) -> Result<Session> {
let configuring = false;
let mut session = match self.connect(context, configuring).await {
let session = match self.connect(context, configuring).await {
Ok(session) => session,
Err(err) => {
self.connectivity.set_err(context, &err);
@@ -498,14 +494,6 @@ impl Imap {
}
};
let folders_configured = context
.sql
.get_raw_config_int(constants::DC_FOLDERS_CONFIGURED_KEY)
.await?;
if folders_configured.unwrap_or_default() < constants::DC_FOLDERS_CONFIGURED_VERSION {
self.configure_folders(context, &mut session).await?;
}
Ok(session)
}
@@ -518,15 +506,15 @@ impl Imap {
context: &Context,
session: &mut Session,
watch_folder: &str,
folder_meaning: FolderMeaning,
) -> Result<()> {
ensure_and_debug_assert!(!watch_folder.is_empty(), "Watched folder cannot be empty");
if !context.sql.is_open().await {
// probably shutdown
bail!("IMAP operation attempted while it is torn down");
}
let msgs_fetched = self
.fetch_new_messages(context, session, watch_folder, folder_meaning)
.fetch_new_messages(context, session, watch_folder)
.await
.context("fetch_new_messages")?;
if msgs_fetched && context.get_config_delete_device_after().await?.is_some() {
@@ -554,19 +542,9 @@ impl Imap {
context: &Context,
session: &mut Session,
folder: &str,
folder_meaning: FolderMeaning,
) -> Result<bool> {
let transport_id = session.transport_id();
if should_ignore_folder(context, folder, folder_meaning).await? {
info!(
context,
"Transport {transport_id}: Not fetching from {folder:?}."
);
session.new_mail = false;
return Ok(false);
}
let folder_exists = session
.select_with_uidvalidity(context, folder)
.await
@@ -589,9 +567,8 @@ impl Imap {
let mut read_cnt = 0;
loop {
let (n, fetch_more) = self
.fetch_new_msg_batch(context, session, folder, folder_meaning)
.await?;
let (n, fetch_more) =
Box::pin(self.fetch_new_msg_batch(context, session, folder)).await?;
read_cnt += n;
if !fetch_more {
return Ok(read_cnt > 0);
@@ -606,7 +583,6 @@ impl Imap {
context: &Context,
session: &mut Session,
folder: &str,
folder_meaning: FolderMeaning,
) -> Result<(usize, bool)> {
let transport_id = self.transport_id;
let uid_validity = get_uidvalidity(context, transport_id, folder).await?;
@@ -676,13 +652,7 @@ impl Imap {
info!(context, "Deleting locally deleted message {message_id}.");
}
let _target;
let target = if delete {
""
} else {
_target = target_folder(context, folder, folder_meaning, &headers).await?;
&_target
};
let target = if delete { "" } else { folder };
context
.sql
@@ -710,18 +680,9 @@ impl Imap {
// message, move it to the movebox and then download the second message before
// downloading the first one, if downloading from inbox before moving is allowed.
if folder == target
// Never download messages directly from the spam folder.
// If the sender is known, the message will be moved to the Inbox or Mvbox
// and then we download the message from there.
// Also see `spam_target_folder_cfg()`.
&& folder_meaning != FolderMeaning::Spam
&& prefetch_should_download(
context,
&headers,
&message_id,
fetch_response.flags(),
)
.await.context("prefetch_should_download")?
&& prefetch_should_download(context, &headers, &message_id, fetch_response.flags())
.await
.context("prefetch_should_download")?
{
if headers
.get_header_value(HeaderDef::ChatIsPostMessage)
@@ -984,6 +945,29 @@ impl Session {
Ok(())
}
/// Deletes all messages from IMAP folder.
pub(crate) async fn delete_all_messages(
&mut self,
context: &Context,
folder: &str,
) -> Result<()> {
let transport_id = self.transport_id();
if self.select_with_uidvalidity(context, folder).await? {
self.add_flag_finalized_with_set("1:*", "\\Deleted").await?;
self.selected_folder_needs_expunge = true;
context
.sql
.execute(
"DELETE FROM imap WHERE transport_id=? AND folder=?",
(transport_id, folder),
)
.await?;
}
Ok(())
}
/// Moves batch of messages identified by their UID from the currently
/// selected folder to the target folder.
async fn move_message_batch(
@@ -1671,13 +1655,8 @@ impl Session {
// Store new encrypted device token on the server
// even if it is the same as the old one.
if let Some(encrypted_device_token) = new_encrypted_device_token {
let folder = context
.get_config(Config::ConfiguredInboxFolder)
.await?
.context("INBOX is not configured")?;
self.run_command_and_check_ok(&format_setmetadata(
&folder,
"INBOX",
&encrypted_device_token,
))
.await
@@ -1722,117 +1701,6 @@ impl Session {
}
Ok(())
}
/// Attempts to configure mvbox.
///
/// Tries to find any folder examining `folders` in the order they go.
/// This method does not use LIST command to ensure that
/// configuration works even if mailbox lookup is forbidden via Access Control List (see
/// <https://datatracker.ietf.org/doc/html/rfc4314>).
///
/// Returns first found folder name.
async fn configure_mvbox<'a>(
&mut self,
context: &Context,
folders: &[&'a str],
) -> Result<Option<&'a str>> {
// Close currently selected folder if needed.
// We are going to select folders using low-level EXAMINE operations below.
self.maybe_close_folder(context).await?;
for folder in folders {
info!(context, "Looking for MVBOX-folder \"{}\"...", &folder);
let res = self.examine(&folder).await;
if res.is_ok() {
info!(
context,
"MVBOX-folder {:?} successfully selected, using it.", &folder
);
self.close().await?;
// Before moving emails to the mvbox we need to remember its UIDVALIDITY, otherwise
// emails moved before that wouldn't be fetched but considered "old" instead.
let folder_exists = self.select_with_uidvalidity(context, folder).await?;
ensure!(folder_exists, "No MVBOX folder {:?}??", &folder);
return Ok(Some(folder));
}
}
Ok(None)
}
}
impl Imap {
pub(crate) async fn configure_folders(
&mut self,
context: &Context,
session: &mut Session,
) -> Result<()> {
let mut folders = session
.list(Some(""), Some("*"))
.await
.context("list_folders failed")?;
let mut delimiter = ".".to_string();
let mut delimiter_is_default = true;
let mut folder_configs = BTreeMap::new();
while let Some(folder) = folders.try_next().await? {
info!(context, "Scanning folder: {:?}", folder);
// Update the delimiter iff there is a different one, but only once.
if let Some(d) = folder.delimiter()
&& delimiter_is_default
&& !d.is_empty()
&& delimiter != d
{
delimiter = d.to_string();
delimiter_is_default = false;
}
let folder_meaning = get_folder_meaning_by_attrs(folder.attributes());
let folder_name_meaning = get_folder_meaning_by_name(folder.name());
if let Some(config) = folder_meaning.to_config() {
// Always takes precedence
folder_configs.insert(config, folder.name().to_string());
} else if let Some(config) = folder_name_meaning.to_config() {
// only set if none has been already set
folder_configs
.entry(config)
.or_insert_with(|| folder.name().to_string());
}
}
drop(folders);
info!(context, "Using \"{}\" as folder-delimiter.", delimiter);
let fallback_folder = format!("INBOX{delimiter}DeltaChat");
let mvbox_folder = session
.configure_mvbox(context, &["DeltaChat", &fallback_folder])
.await
.context("failed to configure mvbox")?;
context
.set_config_internal(Config::ConfiguredInboxFolder, Some("INBOX"))
.await?;
if let Some(mvbox_folder) = mvbox_folder {
info!(context, "Setting MVBOX FOLDER TO {}", &mvbox_folder);
context
.set_config_internal(Config::ConfiguredMvboxFolder, Some(mvbox_folder))
.await?;
}
for (config, name) in folder_configs {
context.set_config_internal(config, Some(&name)).await?;
}
context
.sql
.set_raw_config_int(
constants::DC_FOLDERS_CONFIGURED_KEY,
constants::DC_FOLDERS_CONFIGURED_VERSION,
)
.await?;
info!(context, "FINISHED configuring IMAP-folders.");
Ok(())
}
}
impl Session {
@@ -1966,15 +1834,7 @@ async fn spam_target_folder_cfg(
return Ok(None);
}
if needs_move_to_mvbox(context, headers).await?
// If OnlyFetchMvbox is set, we don't want to move the message to
// the inbox where we wouldn't fetch it again:
|| context.get_config_bool(Config::OnlyFetchMvbox).await?
{
Ok(Some(Config::ConfiguredMvboxFolder))
} else {
Ok(Some(Config::ConfiguredInboxFolder))
}
Ok(Some(Config::ConfiguredInboxFolder))
}
/// Returns `ConfiguredInboxFolder` or `ConfiguredMvboxFolder` if
@@ -1985,16 +1845,12 @@ pub async fn target_folder_cfg(
folder_meaning: FolderMeaning,
headers: &[mailparse::MailHeader<'_>],
) -> Result<Option<Config>> {
if context.is_mvbox(folder).await? {
if folder == "DeltaChat" {
return Ok(None);
}
if folder_meaning == FolderMeaning::Spam {
spam_target_folder_cfg(context, headers).await
} else if folder_meaning == FolderMeaning::Inbox
&& needs_move_to_mvbox(context, headers).await?
{
Ok(Some(Config::ConfiguredMvboxFolder))
} else {
Ok(None)
}
@@ -2015,27 +1871,6 @@ pub async fn target_folder(
}
}
async fn needs_move_to_mvbox(
context: &Context,
headers: &[mailparse::MailHeader<'_>],
) -> Result<bool> {
let has_chat_version = headers.get_header_value(HeaderDef::ChatVersion).is_some();
if !context.get_config_bool(Config::MvboxMove).await? {
return Ok(false);
}
if has_chat_version {
Ok(true)
} else if let Some(parent) = get_prefetch_parent_message(context, headers).await? {
match parent.is_dc_message {
MessengerMessage::No => Ok(false),
MessengerMessage::Yes | MessengerMessage::Reply => Ok(true),
}
} else {
Ok(false)
}
}
/// Try to get the folder meaning by the name of the folder only used if the server does not support XLIST.
// TODO: lots languages missing - maybe there is a list somewhere on other MUAs?
// however, if we fail to find out the sent-folder,
@@ -2366,21 +2201,6 @@ async fn get_modseq(context: &Context, transport_id: u32, folder: &str) -> Resul
.unwrap_or(0))
}
/// Whether to ignore fetching messages from a folder.
///
/// This caters for the [`Config::OnlyFetchMvbox`] setting which means mails from folders
/// not explicitly watched should not be fetched.
async fn should_ignore_folder(
context: &Context,
folder: &str,
folder_meaning: FolderMeaning,
) -> Result<bool> {
if !context.get_config_bool(Config::OnlyFetchMvbox).await? {
return Ok(false);
}
Ok(!(context.is_mvbox(folder).await? || folder_meaning == FolderMeaning::Spam))
}
/// Builds a list of sequence/uid sets. The returned sets have each no more than around 1000
/// characters because according to <https://tools.ietf.org/html/rfc2683#section-3.2.1.5>
/// command lines should not be much more than 1000 chars (servers should allow at least 8000 chars)
@@ -2440,23 +2260,5 @@ impl std::fmt::Display for UidRange {
}
}
pub(crate) async fn get_watched_folder_configs(context: &Context) -> Result<Vec<Config>> {
let mut res = vec![Config::ConfiguredInboxFolder];
if context.should_watch_mvbox().await? {
res.push(Config::ConfiguredMvboxFolder);
}
Ok(res)
}
pub(crate) async fn get_watched_folders(context: &Context) -> Result<Vec<String>> {
let mut res = Vec::new();
for folder_config in get_watched_folder_configs(context).await? {
if let Some(folder) = context.get_config(folder_config).await? {
res.push(folder);
}
}
Ok(res)
}
#[cfg(test)]
mod imap_tests;
+8
View File
@@ -220,6 +220,8 @@ impl Client {
alpn(addr.port()),
logging_stream,
&context.tls_session_store,
&context.spki_hash_store,
&context.sql,
)
.await?;
let buffered_stream = BufWriter::new(tls_stream);
@@ -282,6 +284,8 @@ impl Client {
"",
tcp_stream,
&context.tls_session_store,
&context.spki_hash_store,
&context.sql,
)
.await
.context("STARTTLS upgrade failed")?;
@@ -310,6 +314,8 @@ impl Client {
alpn(port),
proxy_stream,
&context.tls_session_store,
&context.spki_hash_store,
&context.sql,
)
.await?;
let buffered_stream = BufWriter::new(tls_stream);
@@ -373,6 +379,8 @@ impl Client {
"",
proxy_stream,
&context.tls_session_store,
&context.spki_hash_store,
&context.sql,
)
.await
.context("STARTTLS upgrade failed")?;
+1 -5
View File
@@ -120,11 +120,7 @@ impl Session {
impl Imap {
/// Idle using polling.
pub(crate) async fn fake_idle(
&mut self,
context: &Context,
watch_folder: String,
) -> Result<()> {
pub(crate) async fn fake_idle(&mut self, context: &Context, watch_folder: &str) -> Result<()> {
let fake_idle_start_time = tools::Time::now();
info!(context, "IMAP-fake-IDLEing folder={:?}", watch_folder);
+22 -58
View File
@@ -100,23 +100,16 @@ fn test_build_sequence_sets() {
async fn check_target_folder_combination(
folder: &str,
mvbox_move: bool,
chat_msg: bool,
expected_destination: &str,
accepted_chat: bool,
outgoing: bool,
) -> Result<()> {
println!(
"Testing: For folder {folder}, mvbox_move {mvbox_move}, chat_msg {chat_msg}, accepted {accepted_chat}, outgoing {outgoing}"
"Testing: For folder {folder}, chat_msg {chat_msg}, accepted {accepted_chat}, outgoing {outgoing}"
);
let t = TestContext::new_alice().await;
t.ctx
.set_config(Config::ConfiguredMvboxFolder, Some("DeltaChat"))
.await?;
t.ctx
.set_config(Config::MvboxMove, Some(if mvbox_move { "1" } else { "0" }))
.await?;
if accepted_chat {
let contact_id = Contact::create(&t.ctx, "", "bob@example.net").await?;
@@ -161,64 +154,42 @@ async fn check_target_folder_combination(
assert_eq!(
expected,
actual.as_deref(),
"For folder {folder}, mvbox_move {mvbox_move}, chat_msg {chat_msg}, accepted {accepted_chat}, outgoing {outgoing}: expected {expected:?}, got {actual:?}"
"For folder {folder}, chat_msg {chat_msg}, accepted {accepted_chat}, outgoing {outgoing}: expected {expected:?}, got {actual:?}"
);
Ok(())
}
// chat_msg means that the message was sent by Delta Chat
// The tuples are (folder, mvbox_move, chat_msg, expected_destination)
const COMBINATIONS_ACCEPTED_CHAT: &[(&str, bool, bool, &str)] = &[
("INBOX", false, false, "INBOX"),
("INBOX", false, true, "INBOX"),
("INBOX", true, false, "INBOX"),
("INBOX", true, true, "DeltaChat"),
("Spam", false, false, "INBOX"), // Move classical emails in accepted chats from Spam to Inbox, not 100% sure on this, we could also just never move non-chat-msgs
("Spam", false, true, "INBOX"),
("Spam", true, false, "INBOX"), // Move classical emails in accepted chats from Spam to Inbox, not 100% sure on this, we could also just never move non-chat-msgs
("Spam", true, true, "DeltaChat"),
// The tuples are (folder, chat_msg, expected_destination)
const COMBINATIONS_ACCEPTED_CHAT: &[(&str, bool, &str)] = &[
("INBOX", false, "INBOX"),
("INBOX", true, "INBOX"),
("Spam", false, "INBOX"), // Move classical emails in accepted chats from Spam to Inbox, not 100% sure on this, we could also just never move non-chat-msgs
("Spam", true, "INBOX"),
];
// These are the same as above, but non-chat messages in Spam stay in Spam
const COMBINATIONS_REQUEST: &[(&str, bool, bool, &str)] = &[
("INBOX", false, false, "INBOX"),
("INBOX", false, true, "INBOX"),
("INBOX", true, false, "INBOX"),
("INBOX", true, true, "DeltaChat"),
("Spam", false, false, "Spam"),
("Spam", false, true, "INBOX"),
("Spam", true, false, "Spam"),
("Spam", true, true, "DeltaChat"),
const COMBINATIONS_REQUEST: &[(&str, bool, &str)] = &[
("INBOX", false, "INBOX"),
("INBOX", true, "INBOX"),
("Spam", false, "Spam"),
("Spam", true, "INBOX"),
];
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_target_folder_incoming_accepted() -> Result<()> {
for (folder, mvbox_move, chat_msg, expected_destination) in COMBINATIONS_ACCEPTED_CHAT {
check_target_folder_combination(
folder,
*mvbox_move,
*chat_msg,
expected_destination,
true,
false,
)
.await?;
for (folder, chat_msg, expected_destination) in COMBINATIONS_ACCEPTED_CHAT {
check_target_folder_combination(folder, *chat_msg, expected_destination, true, false)
.await?;
}
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_target_folder_incoming_request() -> Result<()> {
for (folder, mvbox_move, chat_msg, expected_destination) in COMBINATIONS_REQUEST {
check_target_folder_combination(
folder,
*mvbox_move,
*chat_msg,
expected_destination,
false,
false,
)
.await?;
for (folder, chat_msg, expected_destination) in COMBINATIONS_REQUEST {
check_target_folder_combination(folder, *chat_msg, expected_destination, false, false)
.await?;
}
Ok(())
}
@@ -226,16 +197,9 @@ async fn test_target_folder_incoming_request() -> Result<()> {
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_target_folder_outgoing() -> Result<()> {
// Test outgoing emails
for (folder, mvbox_move, chat_msg, expected_destination) in COMBINATIONS_ACCEPTED_CHAT {
check_target_folder_combination(
folder,
*mvbox_move,
*chat_msg,
expected_destination,
true,
true,
)
.await?;
for (folder, chat_msg, expected_destination) in COMBINATIONS_ACCEPTED_CHAT {
check_target_folder_combination(folder, *chat_msg, expected_destination, true, true)
.await?;
}
Ok(())
}
+4 -4
View File
@@ -16,7 +16,7 @@ use crate::net::session::SessionStream;
/// - Autocrypt-Setup-Message to check if a message is an autocrypt setup message,
/// not necessarily sent by Delta Chat.
/// - Chat-Is-Post-Message to skip it in background fetch or when it is > `DownloadLimit`.
const PREFETCH_FLAGS: &str = "(UID INTERNALDATE RFC822.SIZE BODY.PEEK[HEADER.FIELDS (\
const PREFETCH_FLAGS: &str = "(UID RFC822.SIZE BODY.PEEK[HEADER.FIELDS (\
MESSAGE-ID \
DATE \
X-MICROSOFT-ORIGINAL-MESSAGE-ID \
@@ -124,7 +124,7 @@ impl Session {
}
/// Prefetch `n_uids` messages starting from `uid_next`. Returns a list of fetch results in the
/// order of ascending delivery time to the server (INTERNALDATE).
/// order of ascending UIDs.
#[expect(clippy::arithmetic_side_effects)]
pub(crate) async fn prefetch(
&mut self,
@@ -142,10 +142,10 @@ impl Session {
let mut msgs = BTreeMap::new();
while let Some(msg) = list.try_next().await? {
if let Some(msg_uid) = msg.uid {
msgs.insert((msg.internal_date(), msg_uid), msg);
msgs.insert(msg_uid, msg);
}
}
Ok(msgs.into_iter().map(|((_, uid), msg)| (uid, msg)).collect())
Ok(Vec::from_iter(msgs))
}
}
+1
View File
@@ -17,6 +17,7 @@
clippy::cloned_instead_of_copied,
clippy::manual_is_variant_and
)]
#![cfg_attr(not(test), warn(clippy::large_futures))]
#![cfg_attr(not(test), warn(clippy::arithmetic_side_effects))]
#![cfg_attr(not(test), forbid(clippy::indexing_slicing))]
#![cfg_attr(not(test), forbid(clippy::string_slice))]
+44 -8
View File
@@ -56,9 +56,37 @@ pub enum EnteredCertificateChecks {
AcceptInvalidCertificates2 = 3,
}
/// Login parameters for a single server, either IMAP or SMTP
/// Login parameters for a single IMAP server.
#[derive(Debug, Default, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct EnteredServerLoginParam {
pub struct EnteredImapLoginParam {
/// Server hostname or IP address.
pub server: String,
/// Server port.
///
/// 0 if not specified.
pub port: u16,
/// Folder to watch.
///
/// If empty, user has not entered anything and it shuold expand to "INBOX" later.
pub folder: String,
/// Socket security.
pub security: Socket,
/// Username.
///
/// Empty string if not specified.
pub user: String,
/// Password.
pub password: String,
}
/// Login parameters for a single SMTP server.
#[derive(Debug, Default, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct EnteredSmtpLoginParam {
/// Server hostname or IP address.
pub server: String,
@@ -96,10 +124,10 @@ pub struct EnteredLoginParam {
pub addr: String,
/// IMAP settings.
pub imap: EnteredServerLoginParam,
pub imap: EnteredImapLoginParam,
/// SMTP settings.
pub smtp: EnteredServerLoginParam,
pub smtp: EnteredSmtpLoginParam,
/// TLS options: whether to allow invalid certificates and/or
/// invalid hostnames
@@ -111,6 +139,8 @@ pub struct EnteredLoginParam {
impl EnteredLoginParam {
/// Loads entered account settings.
///
/// This is a legacy API for loading from separate config parameters.
pub(crate) async fn load(context: &Context) -> Result<Self> {
let addr = context
.get_config(Config::Addr)
@@ -127,6 +157,10 @@ impl EnteredLoginParam {
.get_config_parsed::<u16>(Config::MailPort)
.await?
.unwrap_or_default();
// There is no way to set custom folder with this legacy API.
let mail_folder = String::new();
let mail_security = context
.get_config_parsed::<i32>(Config::MailSecurity)
.await?
@@ -185,14 +219,15 @@ impl EnteredLoginParam {
Ok(EnteredLoginParam {
addr,
imap: EnteredServerLoginParam {
imap: EnteredImapLoginParam {
server: mail_server,
port: mail_port,
folder: mail_folder,
security: mail_security,
user: mail_user,
password: mail_pw,
},
smtp: EnteredServerLoginParam {
smtp: EnteredSmtpLoginParam {
server: send_server,
port: send_port,
security: send_security,
@@ -354,14 +389,15 @@ mod tests {
let t = TestContext::new().await;
let param = EnteredLoginParam {
addr: "alice@example.org".to_string(),
imap: EnteredServerLoginParam {
imap: EnteredImapLoginParam {
server: "".to_string(),
port: 0,
folder: "".to_string(),
security: Socket::Starttls,
user: "".to_string(),
password: "foobar".to_string(),
},
smtp: EnteredServerLoginParam {
smtp: EnteredSmtpLoginParam {
server: "".to_string(),
port: 2947,
security: Socket::default(),
+4 -5
View File
@@ -1,7 +1,6 @@
//! # Messages and their identifiers.
use std::collections::BTreeSet;
use std::collections::HashSet;
use std::path::{Path, PathBuf};
use std::str;
@@ -200,7 +199,6 @@ SELECT ?1, rfc724_mid, pre_rfc724_mid, timestamp, ?, ? FROM msgs WHERE id=?1
}
/// Returns detailed message information in a multi-line text form.
#[expect(clippy::arithmetic_side_effects)]
pub async fn get_info(self, context: &Context) -> Result<String> {
let msg = Message::load_from_db(context, self).await?;
@@ -826,7 +824,6 @@ impl Message {
///
/// Currently this includes `additional_text`, but this may change in future, when the UIs show
/// the necessary info themselves.
#[expect(clippy::arithmetic_side_effects)]
pub fn get_text(&self) -> String {
self.text.clone() + &self.additional_text
}
@@ -1393,6 +1390,8 @@ pub enum MessageState {
/// For files which need time to be prepared before they can be
/// sent, the message enters this state before
/// OutPending.
///
/// Deprecated 2024-12-07.
OutPreparing = 18,
/// Message saved as draft.
@@ -1693,7 +1692,7 @@ pub(crate) async fn delete_msg_locally(context: &Context, msg: &Message) -> Resu
pub(crate) async fn delete_msgs_locally_done(
context: &Context,
msg_ids: &[MsgId],
modified_chat_ids: HashSet<ChatId>,
modified_chat_ids: BTreeSet<ChatId>,
) -> Result<()> {
for modified_chat_id in modified_chat_ids {
context.emit_msgs_changed_without_msg_id(modified_chat_id);
@@ -1723,7 +1722,7 @@ pub async fn delete_msgs_ex(
msg_ids: &[MsgId],
delete_for_all: bool,
) -> Result<()> {
let mut modified_chat_ids = HashSet::new();
let mut modified_chat_ids = BTreeSet::new();
let mut deleted_rfc724_mid = Vec::new();
let mut res = Ok(());
+11 -9
View File
@@ -463,18 +463,21 @@ impl MimeFactory {
.into_iter()
.filter(|id| *id != ContactId::SELF)
.collect();
if recipient_ids.len() == 1
&& !matches!(
msg.param.get_cmd(),
SystemMessage::MemberRemovedFromGroup | SystemMessage::SecurejoinMessage
)
&& !matches!(chat.typ, Chattype::OutBroadcast | Chattype::InBroadcast)
if !matches!(
msg.param.get_cmd(),
SystemMessage::MemberRemovedFromGroup | SystemMessage::SecurejoinMessage
) && !matches!(chat.typ, Chattype::OutBroadcast | Chattype::InBroadcast)
{
let origin = match recipient_ids.len() {
1 => Origin::OutgoingTo,
// Use the same origin as ChatId::accept_ex() does for groups.
_ => Origin::IncomingTo,
};
info!(
context,
"Scale up origin of {} recipients to OutgoingTo.", chat.id
"Scale up origin of {} recipients to {origin:?}.", chat.id
);
ContactId::scaleup_origin(context, &recipient_ids, Origin::OutgoingTo).await?;
ContactId::scaleup_origin(context, &recipient_ids, origin).await?;
}
if !msg.is_system_message()
@@ -1884,7 +1887,6 @@ impl MimeFactory {
}
/// Render an MDN
#[expect(clippy::arithmetic_side_effects)]
fn render_mdn(&mut self) -> Result<MimePart<'static>> {
// RFC 6522, this also requires the `report-type` parameter which is equal
// to the MIME subtype of the second body part of the multipart/report
-1
View File
@@ -269,7 +269,6 @@ impl MimeMessage {
///
/// This method has some side-effects,
/// such as saving blobs and saving found public keys to the database.
#[expect(clippy::arithmetic_side_effects)]
pub(crate) async fn from_bytes(context: &Context, body: &[u8]) -> Result<Self> {
let mail = mailparse::parse_mail(body)?;
+5 -1
View File
@@ -12,7 +12,7 @@ use tokio_io_timeout::TimeoutStream;
use crate::context::Context;
use crate::net::session::SessionStream;
use crate::net::tls::TlsSessionStore;
use crate::net::tls::{SpkiHashStore, TlsSessionStore};
use crate::sql::Sql;
use crate::tools::time;
@@ -130,6 +130,8 @@ pub(crate) async fn connect_tls_inner(
strict_tls: bool,
alpn: &str,
tls_session_store: &TlsSessionStore,
spki_hash_store: &SpkiHashStore,
sql: &Sql,
) -> Result<impl SessionStream + 'static> {
let use_sni = true;
let tcp_stream = connect_tcp_inner(addr).await?;
@@ -141,6 +143,8 @@ pub(crate) async fn connect_tls_inner(
alpn,
tcp_stream,
tls_session_store,
spki_hash_store,
sql,
)
.await?;
Ok(tls_stream)
+4
View File
@@ -87,6 +87,8 @@ where
"",
proxy_stream,
&context.tls_session_store,
&context.spki_hash_store,
&context.sql,
)
.await?;
Box::new(tls_stream)
@@ -99,6 +101,8 @@ where
"",
tcp_stream,
&context.tls_session_store,
&context.spki_hash_store,
&context.sql,
)
.await?;
Box::new(tls_stream)
+2 -2
View File
@@ -171,7 +171,6 @@ pub enum ProxyConfig {
}
/// Constructs HTTP/1.1 `CONNECT` request for HTTP(S) proxy.
#[expect(clippy::arithmetic_side_effects)]
fn http_connect_request(host: &str, port: u16, auth: Option<(&str, &str)>) -> String {
// According to <https://datatracker.ietf.org/doc/html/rfc7230#section-5.4>
// clients MUST send `Host:` header in HTTP/1.1 requests,
@@ -320,7 +319,6 @@ impl ProxyConfig {
/// config into `proxy_url` if `proxy_url` is unset or empty.
///
/// Unsets `socks5_host`, `socks5_port`, `socks5_user` and `socks5_password` in any case.
#[expect(clippy::arithmetic_side_effects)]
async fn migrate_socks_config(sql: &Sql) -> Result<()> {
if sql.get_raw_config("proxy_url").await?.is_none() {
// Load legacy SOCKS5 settings.
@@ -436,6 +434,8 @@ impl ProxyConfig {
"",
tcp_stream,
&context.tls_session_store,
&context.spki_hash_store,
&context.sql,
)
.await?;
let auth = if let Some((username, password)) = &https_config.user_password {
+45 -16
View File
@@ -6,13 +6,20 @@ use std::sync::Arc;
use anyhow::Result;
use crate::net::session::SessionStream;
use crate::sql::Sql;
use crate::tools::time;
use tokio_rustls::rustls;
use tokio_rustls::rustls::client::ClientSessionStore;
use tokio_rustls::rustls::server::ParsedCertificate;
mod danger;
use danger::NoCertificateVerification;
use danger::CustomCertificateVerifier;
mod spki;
pub use spki::SpkiHashStore;
#[expect(clippy::too_many_arguments)]
pub async fn wrap_tls<'a>(
strict_tls: bool,
hostname: &str,
@@ -21,10 +28,21 @@ pub async fn wrap_tls<'a>(
alpn: &str,
stream: impl SessionStream + 'static,
tls_session_store: &TlsSessionStore,
spki_hash_store: &SpkiHashStore,
sql: &Sql,
) -> Result<impl SessionStream + 'a> {
if strict_tls {
let tls_stream =
wrap_rustls(hostname, port, use_sni, alpn, stream, tls_session_store).await?;
let tls_stream = wrap_rustls(
hostname,
port,
use_sni,
alpn,
stream,
tls_session_store,
spki_hash_store,
sql,
)
.await?;
let boxed_stream: Box<dyn SessionStream> = Box::new(tls_stream);
Ok(boxed_stream)
} else {
@@ -94,6 +112,7 @@ impl TlsSessionStore {
}
}
#[expect(clippy::too_many_arguments)]
pub async fn wrap_rustls<'a>(
hostname: &str,
port: u16,
@@ -101,9 +120,11 @@ pub async fn wrap_rustls<'a>(
alpn: &str,
stream: impl SessionStream + 'a,
tls_session_store: &TlsSessionStore,
spki_hash_store: &SpkiHashStore,
sql: &Sql,
) -> Result<impl SessionStream + 'a> {
let mut root_cert_store = rustls::RootCertStore::empty();
root_cert_store.extend(webpki_roots::TLS_SERVER_ROOTS.iter().cloned());
let root_cert_store =
rustls::RootCertStore::from_iter(webpki_roots::TLS_SERVER_ROOTS.iter().cloned());
let mut config = rustls::ClientConfig::builder()
.with_root_certificates(root_cert_store)
@@ -127,20 +148,28 @@ pub async fn wrap_rustls<'a>(
config.resumption = resumption;
config.enable_sni = use_sni;
// Do not verify certificates for hostnames starting with `_`.
// They are used for servers with self-signed certificates, e.g. for local testing.
// Hostnames starting with `_` can have only self-signed TLS certificates or wildcard certificates.
// It is not possible to get valid non-wildcard TLS certificates because CA/Browser Forum requirements
// explicitly state that domains should start with a letter, digit or hyphen:
// https://github.com/cabforum/servercert/blob/24f38fd4765e019db8bb1a8c56bf63c7115ce0b0/docs/BR.md
if hostname.starts_with("_") {
config
.dangerous()
.set_certificate_verifier(Arc::new(NoCertificateVerification::new()));
}
config
.dangerous()
.set_certificate_verifier(Arc::new(CustomCertificateVerifier::new(
spki_hash_store.get_spki_hash(hostname, sql).await?,
)));
let tls = tokio_rustls::TlsConnector::from(Arc::new(config));
let name = tokio_rustls::rustls::pki_types::ServerName::try_from(hostname)?.to_owned();
let tls_stream = tls.connect(name, stream).await?;
// Successfully connected.
// Remember SPKI hash to accept it later if certificate expires.
let (_io, client_connection) = tls_stream.get_ref();
if let Some(end_entity) = client_connection
.peer_certificates()
.and_then(|certs| certs.first())
{
let now = time();
let parsed_certificate = ParsedCertificate::try_from(end_entity)?;
let spki = parsed_certificate.subject_public_key_info();
spki_hash_store.save_spki(hostname, &spki, sql, now).await?;
}
Ok(tls_stream)
}
+78 -11
View File
@@ -1,26 +1,93 @@
//! Dangerous TLS implementation of accepting invalid certificates for Rustls.
//! Custom TLS verification.
//!
//! We want to accept expired certificates.
use rustls::RootCertStore;
use rustls::client::{verify_server_cert_signed_by_trust_anchor, verify_server_name};
use rustls::pki_types::{CertificateDer, ServerName, UnixTime};
use rustls::server::ParsedCertificate;
use tokio_rustls::rustls;
#[derive(Debug)]
pub(super) struct NoCertificateVerification();
use crate::net::tls::spki::spki_hash;
impl NoCertificateVerification {
pub(super) fn new() -> Self {
Self()
#[derive(Debug)]
pub(super) struct CustomCertificateVerifier {
/// Root certificates.
root_cert_store: RootCertStore,
/// Expected SPKI hash as a base64 of SHA-256.
spki_hash: Option<String>,
}
impl CustomCertificateVerifier {
pub(super) fn new(spki_hash: Option<String>) -> Self {
let root_cert_store =
RootCertStore::from_iter(webpki_roots::TLS_SERVER_ROOTS.iter().cloned());
Self {
root_cert_store,
spki_hash,
}
}
}
impl rustls::client::danger::ServerCertVerifier for NoCertificateVerification {
impl rustls::client::danger::ServerCertVerifier for CustomCertificateVerifier {
fn verify_server_cert(
&self,
_end_entity: &CertificateDer<'_>,
_intermediates: &[CertificateDer<'_>],
_server_name: &ServerName<'_>,
end_entity: &CertificateDer<'_>,
intermediates: &[CertificateDer<'_>],
server_name: &ServerName<'_>,
// OCSP is a certificate revocation mechanism that is intentionally ignored.
// It is practically not used and is essentially deprecated
// in favor of Certificate Revocation Lists.
// Let's Encrypt has disabled OCSP responders in 2025:
// <https://letsencrypt.org/2025/08/06/ocsp-service-has-reached-end-of-life>.
// Theoretically checking of stapled OCSP responses could be implemented,
// but it is not interesting to implement it because it is not used
// by the servers: <https://github.com/rustls/webpki/issues/217>.
_ocsp_response: &[u8],
_now: UnixTime,
now: UnixTime,
) -> Result<rustls::client::danger::ServerCertVerified, rustls::Error> {
let parsed_certificate = ParsedCertificate::try_from(end_entity)?;
let spki = parsed_certificate.subject_public_key_info();
let provider = rustls::crypto::ring::default_provider();
if let ServerName::DnsName(dns_name) = server_name
&& dns_name.as_ref().starts_with("_")
{
// Do not verify certificates for hostnames starting with `_`.
// They are used for servers with self-signed certificates, e.g. for local testing.
// Hostnames starting with `_` can have only self-signed TLS certificates or wildcard certificates.
// It is not possible to get valid non-wildcard TLS certificates because CA/Browser Forum requirements
// explicitly state that domains should start with a letter, digit or hyphen:
// https://github.com/cabforum/servercert/blob/24f38fd4765e019db8bb1a8c56bf63c7115ce0b0/docs/BR.md
} else if let Some(hash) = &self.spki_hash
&& spki_hash(&spki) == *hash
{
// Last time we successfully connected to this hostname with TLS checks,
// SPKI had this hash.
// It does not matter if certificate has now expired.
} else {
// verify_server_cert_signed_by_trust_anchor does no revocation checking:
// <https://docs.rs/rustls/0.23.37/rustls/client/fn.verify_server_cert_signed_by_trust_anchor.html>
// We don't do it either.
verify_server_cert_signed_by_trust_anchor(
&parsed_certificate,
&self.root_cert_store,
intermediates,
now,
provider.signature_verification_algorithms.all,
)?;
}
// Verify server name unconditionally.
//
// We do this even for self-signed certificates when hostname starts with `_`
// so we don't try to connect to captive portals
// and fail on MITM certificates if they are generated once
// and reused for all hostnames.
verify_server_name(&parsed_certificate, server_name)?;
Ok(rustls::client::danger::ServerCertVerified::assertion())
}
+133
View File
@@ -0,0 +1,133 @@
//! SPKI hash storage.
//!
//! We store hashes of Subject Public Key Info from TLS certificates
//! after successful connection to allow connecting when
//! server certificate expires as long as the key is not changed.
use std::collections::BTreeMap;
use anyhow::Result;
use base64::Engine as _;
use parking_lot::RwLock;
use sha2::{Digest, Sha256};
use tokio_rustls::rustls::pki_types::SubjectPublicKeyInfoDer;
use crate::sql::Sql;
use crate::tools::time;
/// Calculates Subject Public Key Info SHA-256 hash and returns it as base64.
///
/// This is the same format as used in <https://www.rfc-editor.org/rfc/rfc7469>.
/// You can calculate the same hash for any remote host with
/// ```sh
/// openssl s_client -connect "$HOST:993" -servername "$HOST" </dev/null 2>/dev/null |
/// openssl x509 -pubkey -noout |
/// openssl pkey -pubin -outform der |
/// openssl dgst -sha256 -binary |
/// openssl enc -base64
/// ```
pub fn spki_hash(spki: &SubjectPublicKeyInfoDer) -> String {
let spki_hash = Sha256::digest(spki);
base64::engine::general_purpose::STANDARD.encode(spki_hash)
}
/// Write-through cache for SPKI hashes.
#[derive(Debug)]
pub struct SpkiHashStore {
/// Map from hostnames to base64 of SHA-256 hashes.
pub hash_store: RwLock<BTreeMap<String, String>>,
}
impl SpkiHashStore {
pub fn new() -> Self {
Self {
hash_store: RwLock::new(BTreeMap::new()),
}
}
/// Returns base64 of SPKI hash if we have previously successfully connected to given hostname.
pub async fn get_spki_hash(&self, hostname: &str, sql: &Sql) -> Result<Option<String>> {
if let Some(hash) = self.hash_store.read().get(hostname).cloned() {
return Ok(Some(hash));
}
match sql
.query_row_optional(
"SELECT spki_hash FROM tls_spki WHERE host=?",
(hostname,),
|row| {
let spki_hash: String = row.get(0)?;
Ok(spki_hash)
},
)
.await?
{
Some(hash) => {
self.hash_store
.write()
.insert(hostname.to_string(), hash.clone());
Ok(Some(hash))
}
None => Ok(None),
}
}
/// Saves SPKI hash after successful connection.
pub async fn save_spki(
&self,
hostname: &str,
spki: &SubjectPublicKeyInfoDer<'_>,
sql: &Sql,
timestamp: i64,
) -> Result<()> {
let hash = spki_hash(spki);
self.hash_store
.write()
.insert(hostname.to_string(), hash.clone());
sql.execute(
"INSERT OR REPLACE INTO tls_spki (host, spki_hash, timestamp) VALUES (?, ?, ?)",
(hostname, hash, timestamp),
)
.await?;
Ok(())
}
/// Removes stale entries from SPKI storage.
pub async fn cleanup(&self, sql: &Sql) -> Result<()> {
let now = time();
let removed_hosts = sql
.transaction(|transaction| {
let mut stmt = transaction
.prepare("DELETE FROM tls_spki WHERE ? > timestamp + ? RETURNING host")?;
let mut res = Vec::new();
for row in stmt.query_map((now, 30 * 24 * 60 * 60), |row| {
let host: String = row.get(0)?;
Ok(host)
})? {
res.push(row?);
}
// Fix timestamps that happen to be in the future
// if we had clock set incorrectly when the timestamp was stored.
// Otherwise entry may take more than 30 days to expire.
transaction.execute(
"UPDATE tls_spki SET timestamp = ?1 WHERE timestamp > ?1",
(now,),
)?;
Ok(res)
})
.await?;
let mut lock = self.hash_store.write();
for host in removed_hosts {
// We may accidentally remove a host that was added
// to the cache after SQL query but before we got
// the write lock on `hash_store`.
// It is unlikely and will only result
// in additional SQL query next time.
lock.remove(&host);
}
Ok(())
}
}
-1
View File
@@ -24,7 +24,6 @@ pub struct PlainText {
impl PlainText {
/// Convert plain text to HTML.
/// The function handles quotes, links, fixed and floating text paragraphs.
#[expect(clippy::arithmetic_side_effects)]
pub fn to_html(&self) -> String {
static LINKIFY_MAIL_RE: LazyLock<regex::Regex> =
LazyLock::new(|| regex::Regex::new(r"\b([\w.\-+]+@[\w.\-]+)\b").unwrap());
+4 -5
View File
@@ -17,7 +17,7 @@ use crate::config::Config;
use crate::contact::{Contact, ContactId, Origin};
use crate::context::Context;
use crate::key::Fingerprint;
use crate::login_param::{EnteredCertificateChecks, EnteredLoginParam, EnteredServerLoginParam};
use crate::login_param::{EnteredCertificateChecks, EnteredImapLoginParam, EnteredLoginParam};
use crate::net::http::post_empty;
use crate::net::proxy::{DEFAULT_SOCKS_PORT, ProxyConfig};
use crate::token;
@@ -41,7 +41,7 @@ pub(crate) const DCBACKUP_SCHEME_PREFIX: &str = "DCBACKUP";
/// Version written to Backups and Backup-QR-Codes.
/// Imports will fail when they have a larger version.
pub(crate) const DCBACKUP_VERSION: i32 = 4;
pub(crate) const DCBACKUP_VERSION: i32 = 5;
/// Scanned QR code.
#[derive(Debug, Clone, PartialEq, Eq)]
@@ -714,7 +714,6 @@ fn decode_account(qr: &str) -> Result<Qr> {
}
/// scheme: `https://t.me/socks?server=foo&port=123` or `https://t.me/socks?server=1.2.3.4&port=123`
#[expect(clippy::arithmetic_side_effects)]
fn decode_tg_socks_proxy(_context: &Context, qr: &str) -> Result<Qr> {
let url = url::Url::parse(qr).context("Invalid t.me/socks url")?;
@@ -824,7 +823,7 @@ pub(crate) async fn login_param_from_account_qr(
let param = EnteredLoginParam {
addr,
imap: EnteredServerLoginParam {
imap: EnteredImapLoginParam {
password,
..Default::default()
},
@@ -844,7 +843,7 @@ pub(crate) async fn login_param_from_account_qr(
let param = EnteredLoginParam {
addr: email,
imap: EnteredServerLoginParam {
imap: EnteredImapLoginParam {
password,
..Default::default()
},
+6 -3
View File
@@ -5,7 +5,9 @@ use anyhow::{Context as _, Result, bail};
use deltachat_contact_tools::may_be_valid_addr;
use super::{DCLOGIN_SCHEME, Qr};
use crate::login_param::{EnteredCertificateChecks, EnteredLoginParam, EnteredServerLoginParam};
use crate::login_param::{
EnteredCertificateChecks, EnteredImapLoginParam, EnteredLoginParam, EnteredSmtpLoginParam,
};
use crate::provider::Socket;
/// Options for `dclogin:` scheme.
@@ -178,14 +180,15 @@ pub(crate) fn login_param_from_login_qr(
} => {
let param = EnteredLoginParam {
addr: addr.to_string(),
imap: EnteredServerLoginParam {
imap: EnteredImapLoginParam {
server: imap_host.unwrap_or_default(),
port: imap_port.unwrap_or_default(),
folder: "INBOX".to_string(),
security: imap_security.unwrap_or_default(),
user: imap_username.unwrap_or_default(),
password: imap_password.unwrap_or(mail_pw),
},
smtp: EnteredServerLoginParam {
smtp: EnteredSmtpLoginParam {
server: smtp_host.unwrap_or_default(),
port: smtp_port.unwrap_or_default(),
security: smtp_security.unwrap_or_default(),
-2
View File
@@ -15,7 +15,6 @@ use crate::securejoin;
use crate::stock_str::{self, backup_transfer_qr};
/// Create a QR code from any input data.
#[expect(clippy::arithmetic_side_effects)]
pub fn create_qr_svg(qrcode_content: &str) -> Result<String> {
let all_size = 512.0;
let qr_code_size = 416.0;
@@ -176,7 +175,6 @@ async fn self_info(context: &Context) -> Result<(Option<Vec<u8>>, String, String
Ok((avatar, displayname, addr, color))
}
#[expect(clippy::arithmetic_side_effects)]
fn inner_generate_secure_join_qr_code(
qrcode_description: &str,
qrcode_content: &str,
+22 -22
View File
@@ -9,7 +9,6 @@ use async_imap::types::{Quota, QuotaResource};
use crate::chat::add_device_msg_with_importance;
use crate::config::Config;
use crate::context::Context;
use crate::imap::get_watched_folders;
use crate::imap::session::Session as ImapSession;
use crate::log::warn;
use crate::message::Message;
@@ -48,26 +47,24 @@ pub struct QuotaInfo {
async fn get_unique_quota_roots_and_usage(
session: &mut ImapSession,
folders: Vec<String>,
folder: &str,
) -> Result<BTreeMap<String, Vec<QuotaResource>>> {
let mut unique_quota_roots: BTreeMap<String, Vec<QuotaResource>> = BTreeMap::new();
for folder in folders {
let (quota_roots, quotas) = &session.get_quota_root(&folder).await?;
// if there are new quota roots found in this imap folder, add them to the list
for qr_entries in quota_roots {
for quota_root_name in &qr_entries.quota_root_names {
// the quota for that quota root
let quota: Quota = quotas
.iter()
.find(|q| &q.root_name == quota_root_name)
.cloned()
.context("quota_root should have a quota")?;
// replace old quotas, because between fetching quotaroots for folders,
// messages could be received and so the usage could have been changed
*unique_quota_roots
.entry(quota_root_name.clone())
.or_default() = quota.resources;
}
let (quota_roots, quotas) = &session.get_quota_root(folder).await?;
// if there are new quota roots found in this imap folder, add them to the list
for qr_entries in quota_roots {
for quota_root_name in &qr_entries.quota_root_names {
// the quota for that quota root
let quota: Quota = quotas
.iter()
.find(|q| &q.root_name == quota_root_name)
.cloned()
.context("quota_root should have a quota")?;
// replace old quotas, because between fetching quotaroots for folders,
// messages could be received and so the usage could have been changed
*unique_quota_roots
.entry(quota_root_name.clone())
.or_default() = quota.resources;
}
}
Ok(unique_quota_roots)
@@ -122,14 +119,17 @@ impl Context {
/// As the message is added only once, the user is not spammed
/// in case for some providers the quota is always at ~100%
/// and new space is allocated as needed.
pub(crate) async fn update_recent_quota(&self, session: &mut ImapSession) -> Result<()> {
pub(crate) async fn update_recent_quota(
&self,
session: &mut ImapSession,
folder: &str,
) -> Result<()> {
let transport_id = session.transport_id();
info!(self, "Transport {transport_id}: Updating quota.");
let quota = if session.can_check_quota() {
let folders = get_watched_folders(self).await?;
get_unique_quota_roots_and_usage(session, folders).await
get_unique_quota_roots_and_usage(session, folder).await
} else {
Err(anyhow!(stock_str::not_supported_by_provider(self)))
};
+44 -102
View File
@@ -1,6 +1,6 @@
//! # Reactions.
//!
//! Reactions are short messages consisting of emojis sent in reply to
//! Reactions are short messages representing an emoji sent in reply to
//! messages. Unlike normal messages which are added to the end of the chat,
//! reactions are supposed to be displayed near the original messages.
//!
@@ -11,7 +11,7 @@
//! [XEP-0444](https://xmpp.org/extensions/xep-0444.html) section
//! "3.2 Updating reactions to a message". Received reactions override
//! all previously received reactions from the same user and it is
//! possible to remove all reactions by sending an empty string as a reaction,
//! possible to remove the reaction by sending an empty string as a reaction,
//! even though RFC 9078 requires at least one emoji to be sent.
use std::cmp::Ordering;
@@ -29,9 +29,7 @@ use crate::events::EventType;
use crate::message::{Message, MsgId, rfc724_mid_exists};
use crate::param::Param;
/// A single reaction consisting of multiple emoji sequences.
///
/// It is guaranteed to have all emojis sorted and deduplicated inside.
/// A single reaction.
#[derive(Debug, Default, Clone, Deserialize, Eq, PartialEq, Serialize)]
pub struct Reaction {
/// Canonical representation of reaction as a string of space-separated emojis.
@@ -42,11 +40,8 @@ pub struct Reaction {
// FromStr requires error type and reaction parsing never returns an
// error.
impl From<&str> for Reaction {
/// Parses a string containing a reaction.
///
/// Reaction string is separated by spaces or tabs (`WSP` in ABNF),
/// but this function accepts any ASCII whitespace, so even a CRLF at
/// the end of string is acceptable.
/// Convert a `&str` into a `Reaction`.
/// Everything after the first whitespace is ignored.
///
/// Any short enough string is accepted as a reaction to avoid the
/// complexity of validating emoji sequences as required by RFC
@@ -57,42 +52,27 @@ impl From<&str> for Reaction {
/// such as sending large numbers of large messages, and should be
/// dealt with the same way, e.g. by blocking the user.
fn from(reaction: &str) -> Self {
let mut emojis: Vec<&str> = reaction
let reaction: &str = reaction
.split_ascii_whitespace()
.next()
.filter(|&emoji| emoji.len() < 30)
.collect();
emojis.sort_unstable();
emojis.dedup();
let reaction = emojis.join(" ");
Self { reaction }
.unwrap_or("");
Self {
reaction: reaction.to_string(),
}
}
}
impl Reaction {
/// Returns true if reaction contains no emojis.
/// Returns true if reaction contains no emoji.
pub fn is_empty(&self) -> bool {
self.reaction.is_empty()
}
/// Returns a vector of emojis composing a reaction.
pub fn emojis(&self) -> Vec<&str> {
self.reaction.split(' ').collect()
}
/// Returns space-separated string of emojis
/// Returns a string representing the emoji.
pub fn as_str(&self) -> &str {
&self.reaction
}
/// Appends emojis from another reaction to this reaction.
pub fn add(&self, other: Self) -> Self {
let mut emojis: Vec<&str> = self.emojis();
emojis.append(&mut other.emojis());
emojis.sort_unstable();
emojis.dedup();
let reaction = emojis.join(" ");
Self { reaction }
}
}
/// Structure representing all reactions to a particular message.
@@ -126,12 +106,10 @@ impl Reactions {
pub fn emoji_frequencies(&self) -> BTreeMap<String, usize> {
let mut emoji_frequencies: BTreeMap<String, usize> = BTreeMap::new();
for reaction in self.reactions.values() {
for emoji in reaction.emojis() {
emoji_frequencies
.entry(emoji.to_string())
.and_modify(|x| *x += 1)
.or_insert(1);
}
emoji_frequencies
.entry(reaction.as_str().to_string())
.and_modify(|x| *x += 1)
.or_insert(1);
}
emoji_frequencies
}
@@ -152,6 +130,11 @@ impl Reactions {
});
emoji_frequencies
}
/// Returns an iterator of the contacts that reacted and their corresponding reactions.
pub fn iter(&self) -> impl Iterator<Item = (&ContactId, &Reaction)> {
self.reactions.iter()
}
}
impl fmt::Display for Reactions {
@@ -223,7 +206,7 @@ async fn set_msg_id_reaction(
/// Sends a reaction to message `msg_id`, overriding previously sent reactions.
///
/// `reaction` is a string consisting of space-separated emoji. Use
/// `reaction` is a string consisting of a single emoji. Use
/// empty string to retract a reaction.
pub async fn send_reaction(context: &Context, msg_id: MsgId, reaction: &str) -> Result<MsgId> {
let msg = Message::load_from_db(context, msg_id).await?;
@@ -251,24 +234,12 @@ pub async fn send_reaction(context: &Context, msg_id: MsgId, reaction: &str) ->
Ok(reaction_msg_id)
}
/// Adds given reaction to message `msg_id` and sends an update.
///
/// This can be used to implement advanced clients that allow reacting
/// with multiple emojis. For a simple messenger UI, you probably want
/// to use [`send_reaction()`] instead so reacting with a new emoji
/// removes previous emoji at the same time.
pub async fn add_reaction(context: &Context, msg_id: MsgId, reaction: &str) -> Result<MsgId> {
let self_reaction = get_self_reaction(context, msg_id).await?;
let reaction = self_reaction.add(Reaction::from(reaction));
send_reaction(context, msg_id, reaction.as_str()).await
}
/// Updates reaction of `contact_id` on the message with `in_reply_to`
/// Message-ID. If no such message is found in the database, reaction
/// is ignored.
///
/// `reaction` is a space-separated string of emojis. It can be empty
/// if contact wants to remove all reactions.
/// `reaction` is string representing the emoji. It can be empty
/// if contact wants to remove the reaction.
pub(crate) async fn set_msg_reaction(
context: &Context,
in_reply_to: &str,
@@ -301,26 +272,9 @@ pub(crate) async fn set_msg_reaction(
Ok(())
}
/// Get our own reaction for a given message.
async fn get_self_reaction(context: &Context, msg_id: MsgId) -> Result<Reaction> {
let reaction_str: Option<String> = context
.sql
.query_get_value(
"SELECT reaction
FROM reactions
WHERE msg_id=? AND contact_id=?",
(msg_id, ContactId::SELF),
)
.await?;
Ok(reaction_str
.as_deref()
.map(Reaction::from)
.unwrap_or_default())
}
/// Returns a structure containing all reactions to the message.
pub async fn get_msg_reactions(context: &Context, msg_id: MsgId) -> Result<Reactions> {
let reactions: BTreeMap<ContactId, Reaction> = context
let mut reactions: BTreeMap<ContactId, Reaction> = context
.sql
.query_map_collect(
"SELECT contact_id, reaction FROM reactions WHERE msg_id=?",
@@ -332,6 +286,7 @@ pub async fn get_msg_reactions(context: &Context, msg_id: MsgId) -> Result<React
},
)
.await?;
reactions.retain(|_contact, reaction| !reaction.is_empty());
Ok(Reactions { reactions })
}
@@ -406,46 +361,32 @@ mod tests {
#[test]
fn test_parse_reaction() {
// Check that basic set of emojis from RFC 9078 is supported.
assert_eq!(Reaction::from("👍").emojis(), vec!["👍"]);
assert_eq!(Reaction::from("👎").emojis(), vec!["👎"]);
assert_eq!(Reaction::from("😀").emojis(), vec!["😀"]);
assert_eq!(Reaction::from("☹").emojis(), vec!["☹"]);
assert_eq!(Reaction::from("😢").emojis(), vec!["😢"]);
assert_eq!(Reaction::from("👍").as_str(), "👍");
assert_eq!(Reaction::from("👎").as_str(), "👎");
assert_eq!(Reaction::from("😀").as_str(), "😀");
assert_eq!(Reaction::from("☹").as_str(), "☹");
assert_eq!(Reaction::from("😢").as_str(), "😢");
// Empty string can be used to remove all reactions.
assert!(Reaction::from("").is_empty());
// Short strings can be used as emojis, could be used to add
// support for custom emojis via emoji shortcodes.
assert_eq!(Reaction::from(":deltacat:").emojis(), vec![":deltacat:"]);
assert_eq!(Reaction::from(":deltacat:").as_str(), ":deltacat:");
// Check that long strings are not valid emojis.
assert!(
Reaction::from(":foobarbazquuxaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa:").is_empty()
);
// Multiple reactions separated by spaces or tabs are supported.
assert_eq!(Reaction::from("👍 ❤").emojis(), vec!["❤", "👍"]);
assert_eq!(Reaction::from("👍\t❤").emojis(), vec!["❤", "👍"]);
// Multiple reactions separated by spaces or tabs are not supported.
assert_eq!(Reaction::from("👍 ❤").as_str(), "👍");
assert_eq!(Reaction::from("👍\t❤").as_str(), "👍");
// Invalid emojis are removed, but valid emojis are retained.
assert_eq!(
Reaction::from("👍\t:foo: ❤").emojis(),
vec![":foo:", "❤", "👍"]
);
assert_eq!(Reaction::from("👍\t:foo: ❤").as_str(), ":foo: ❤ 👍");
assert_eq!(Reaction::from("👍\t:foo: ❤").as_str(), "👍");
assert_eq!(Reaction::from("👍\t:foo: ❤").as_str(), "👍");
// Duplicates are removed.
assert_eq!(Reaction::from("👍 👍").emojis(), vec!["👍"]);
}
#[test]
fn test_add_reaction() {
let reaction1 = Reaction::from("👍 😀");
let reaction2 = Reaction::from("❤");
let reaction_sum = reaction1.add(reaction2);
assert_eq!(reaction_sum.emojis(), vec!["❤", "👍", "😀"]);
assert_eq!(Reaction::from("👍 👍").as_str(), "👍");
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
@@ -511,7 +452,7 @@ Content-Disposition: reaction\n\
assert_eq!(contacts.first(), Some(&bob_id));
let bob_reaction = reactions.get(bob_id);
assert_eq!(bob_reaction.is_empty(), false);
assert_eq!(bob_reaction.emojis(), vec!["👍"]);
assert_eq!(bob_reaction.as_str(), "👍");
assert_eq!(bob_reaction.as_str(), "👍");
// Alice receives reaction to her message from Bob with a footer.
@@ -730,7 +671,7 @@ Content-Disposition: reaction\n\
let bob_id = contacts.first().unwrap();
let bob_reaction = reactions.get(*bob_id);
assert_eq!(bob_reaction.is_empty(), false);
assert_eq!(bob_reaction.emojis(), vec!["👍"]);
assert_eq!(bob_reaction.as_str(), "👍");
assert_eq!(bob_reaction.as_str(), "👍");
expect_reactions_changed_event(&alice, chat_alice.id, alice_msg.sender_msg_id, *bob_id)
.await?;
@@ -769,15 +710,16 @@ Content-Disposition: reaction\n\
);
// Alice reacts to own message.
// Trying to set multiple reactions at once is not allowed.
send_reaction(&alice, alice_msg.sender_msg_id, "👍 😀")
.await
.unwrap();
let reactions = get_msg_reactions(&alice, alice_msg.sender_msg_id).await?;
assert_eq!(reactions.to_string(), "👍2 😀1");
assert_eq!(reactions.to_string(), "👍2");
assert_eq!(
reactions.emoji_sorted_by_frequency(),
vec![("👍".to_string(), 2), ("😀".to_string(), 1)]
vec![("👍".to_string(), 2)]
);
Ok(())
+45 -30
View File
@@ -1,7 +1,7 @@
//! Internet Message Format reception pipeline.
use std::cmp;
use std::collections::{BTreeMap, BTreeSet, HashSet};
use std::collections::{BTreeMap, BTreeSet};
use std::iter;
use std::sync::LazyLock;
@@ -773,6 +773,7 @@ pub(crate) async fn receive_imf_inner(
chat_id,
chat_id_blocked,
is_dc_message,
is_created,
)
.await
.context("add_parts error")?
@@ -1312,8 +1313,9 @@ async fn decide_chat_assignment(
..
} = &mime_parser.pre_message
{
let msg_id = rfc724_mid_exists(context, post_msg_rfc724_mid).await?;
if let Some(msg_id) = msg_id {
let post_msg_exists = if let Some((msg_id, not_downloaded)) =
message::rfc724_mid_exists_ex(context, post_msg_rfc724_mid, "download_state<>0").await?
{
context
.sql
.execute(
@@ -1321,8 +1323,10 @@ async fn decide_chat_assignment(
(rfc724_mid, msg_id),
)
.await?;
}
let post_msg_exists = msg_id.is_some();
!not_downloaded
} else {
false
};
info!(
context,
"Message {rfc724_mid} is a pre-message for {post_msg_rfc724_mid} (post_msg_exists:{post_msg_exists})."
@@ -1809,6 +1813,7 @@ async fn add_parts(
mut chat_id: ChatId,
mut chat_id_blocked: Blocked,
is_dc_message: MessengerMessage,
is_chat_created: bool,
) -> Result<ReceivedMsg> {
let to_id = if mime_parser.incoming {
ContactId::SELF
@@ -1872,7 +1877,16 @@ async fn add_parts(
apply_out_broadcast_changes(context, mime_parser, &mut chat, from_id).await?
}
Chattype::Group => {
apply_group_changes(context, mime_parser, &mut chat, from_id, to_ids, past_ids).await?
apply_group_changes(
context,
mime_parser,
&mut chat,
from_id,
to_ids,
past_ids,
is_chat_created,
)
.await?
}
Chattype::InBroadcast => {
apply_in_broadcast_changes(context, mime_parser, &mut chat, from_id).await?
@@ -1927,7 +1941,7 @@ async fn add_parts(
&& chat_id.get_ephemeral_timer(context).await? != ephemeral_timer
{
let chat_contacts =
HashSet::<ContactId>::from_iter(chat::get_chat_contacts(context, chat_id).await?);
BTreeSet::<ContactId>::from_iter(chat::get_chat_contacts(context, chat_id).await?);
let is_from_in_chat =
!chat_contacts.contains(&ContactId::SELF) || chat_contacts.contains(&from_id);
@@ -2462,7 +2476,7 @@ async fn handle_edit_delete(
// See `message::delete_msgs_ex()`, unlike edit requests, DC doesn't send unencrypted
// deletion requests, so there's no need to support them.
if part.param.get_bool(Param::GuaranteeE2ee).unwrap_or(false) {
let mut modified_chat_ids = HashSet::new();
let mut modified_chat_ids = BTreeSet::new();
let mut msg_ids = Vec::new();
let rfc724_mid_vec: Vec<&str> = rfc724_mid_list.split_whitespace().collect();
@@ -2889,8 +2903,7 @@ async fn create_group(
let mut chat_id = None;
let mut chat_id_blocked = Default::default();
if chat_id.is_none()
&& !mime_parser.is_mailinglist_message()
if !mime_parser.is_mailinglist_message()
&& !grpid.is_empty()
&& mime_parser.get_header(HeaderDef::ChatGroupName).is_some()
// otherwise, a pending "quit" message may pop up
@@ -3082,6 +3095,7 @@ async fn apply_group_changes(
from_id: ContactId,
to_ids: &[Option<ContactId>],
past_ids: &[Option<ContactId>],
is_chat_created: bool,
) -> Result<GroupChangesInfo> {
let from_is_key_contact = Contact::get_by_id(context, from_id).await?.is_key_contact();
ensure!(from_is_key_contact || chat.grpid.is_empty());
@@ -3094,7 +3108,7 @@ async fn apply_group_changes(
let mut better_msg = None;
let mut silent = false;
let chat_contacts =
HashSet::<ContactId>::from_iter(chat::get_chat_contacts(context, chat.id).await?);
BTreeSet::<ContactId>::from_iter(chat::get_chat_contacts(context, chat.id).await?);
let is_from_in_chat =
!chat_contacts.contains(&ContactId::SELF) || chat_contacts.contains(&from_id);
@@ -3172,8 +3186,8 @@ async fn apply_group_changes(
&& chat.member_list_is_stale(context).await?
{
info!(context, "Member list is stale.");
let mut new_members: HashSet<ContactId> =
HashSet::from_iter(to_ids_flat.iter().copied());
let mut new_members: BTreeSet<ContactId> =
BTreeSet::from_iter(to_ids_flat.iter().copied());
new_members.insert(ContactId::SELF);
if !from_id.is_special() {
new_members.insert(from_id);
@@ -3217,7 +3231,7 @@ async fn apply_group_changes(
)
.await?;
} else {
let mut new_members: HashSet<ContactId>;
let mut new_members: BTreeSet<ContactId>;
// True if a Delta Chat client has explicitly and really added our primary address to an
// already existing group.
let self_added =
@@ -3228,7 +3242,7 @@ async fn apply_group_changes(
false
};
if self_added {
new_members = HashSet::from_iter(to_ids_flat.iter().copied());
new_members = BTreeSet::from_iter(to_ids_flat.iter().copied());
new_members.insert(ContactId::SELF);
if !from_id.is_special() && from_is_key_contact != chat.grpid.is_empty() {
new_members.insert(from_id);
@@ -3279,7 +3293,7 @@ async fn apply_group_changes(
.await?;
}
let new_chat_contacts = HashSet::<ContactId>::from_iter(
let new_chat_contacts = BTreeSet::<ContactId>::from_iter(
chat::get_chat_contacts(context, chat.id)
.await?
.iter()
@@ -3287,27 +3301,23 @@ async fn apply_group_changes(
);
// These are for adding info messages about implicit membership changes.
let mut added_ids: HashSet<ContactId> = new_chat_contacts
let mut added_ids: BTreeSet<ContactId> = new_chat_contacts
.difference(&chat_contacts)
.copied()
.collect();
let mut removed_ids: HashSet<ContactId> = chat_contacts
let mut removed_ids: BTreeSet<ContactId> = chat_contacts
.difference(&new_chat_contacts)
.copied()
.collect();
if let Some(added_id) = added_id
&& !added_ids.remove(&added_id)
&& added_id != ContactId::SELF
{
// No-op "Member added" message. An exception is self-addition messages because they at
// least must be shown when a chat is created on our side.
info!(context, "No-op 'Member added' message (TRASH)");
better_msg = Some(String::new());
}
let id_was_already_added = if let Some(added_id) = added_id {
!added_ids.remove(&added_id)
} else {
false
};
if let Some(removed_id) = removed_id {
removed_ids.remove(&removed_id);
}
let group_changes_msgs = if !chat_contacts.contains(&ContactId::SELF)
&& new_chat_contacts.contains(&ContactId::SELF)
{
@@ -3316,6 +3326,11 @@ async fn apply_group_changes(
group_changes_msgs(context, &added_ids, &removed_ids, chat.id).await?
};
if id_was_already_added && group_changes_msgs.is_empty() && !is_chat_created {
info!(context, "No-op 'Member added' message (TRASH)");
better_msg = Some(String::new());
}
if send_event_chat_modified {
context.emit_event(EventType::ChatModified(chat.id));
chatlist_events::emit_chatlist_item_changed(context, chat.id);
@@ -3513,8 +3528,8 @@ async fn apply_chat_name_avatar_and_description_changes(
#[expect(clippy::arithmetic_side_effects)]
async fn group_changes_msgs(
context: &Context,
added_ids: &HashSet<ContactId>,
removed_ids: &HashSet<ContactId>,
added_ids: &BTreeSet<ContactId>,
removed_ids: &BTreeSet<ContactId>,
chat_id: ChatId,
) -> Result<Vec<(String, SystemMessage, Option<ContactId>)>> {
let mut group_changes_msgs: Vec<(String, SystemMessage, Option<ContactId>)> = Vec::new();
+11 -8
View File
@@ -3875,21 +3875,17 @@ async fn test_group_contacts_goto_bottom() -> Result<()> {
assert_eq!(Contact::get_all(bob, 0, None).await?.len(), 0);
bob_chat_id.accept(bob).await?;
let contacts = Contact::get_all(bob, 0, None).await?;
assert_eq!(contacts.len(), 2);
let bob_fiona_id = bob.add_or_lookup_contact_id(fiona).await;
// Fiona hasn't been online, so she goes after Alice.
assert_eq!(contacts.len(), 2);
assert_eq!(contacts[1], bob_fiona_id);
ChatId::create_for_contact(bob, bob_fiona_id).await?;
let bob_fiona_chat_id = ChatId::create_for_contact(bob, bob_fiona_id).await?;
let contacts = Contact::get_all(bob, 0, None).await?;
assert_eq!(contacts.len(), 2);
assert_eq!(contacts[0], bob_fiona_id);
send_text_msg(
bob,
bob_chat_id,
"Hi Alice, stay down in my contact list".to_string(),
)
.await?;
send_text_msg(bob, bob_chat_id, "Hi all".to_string()).await?;
bob.pop_sent_msg().await;
let contacts = Contact::get_all(bob, 0, None).await?;
assert_eq!(contacts[0], bob_fiona_id);
@@ -3905,6 +3901,13 @@ async fn test_group_contacts_goto_bottom() -> Result<()> {
bob.pop_sent_msg().await;
let contacts = Contact::get_all(bob, 0, None).await?;
let bob_alice_id = bob.add_or_lookup_contact_id(alice).await;
// As the group only contains Alice, the sent message promotes her in the contact list.
assert_eq!(contacts[0], bob_alice_id);
send_text_msg(bob, bob_fiona_chat_id, "Hi Fiona".to_string()).await?;
bob.pop_sent_msg().await;
let contacts = Contact::get_all(bob, 0, None).await?;
// Alice is still the 0th contact because Fiona hasn't been online.
assert_eq!(contacts[0], bob_alice_id);
Ok(())
}
+97 -157
View File
@@ -17,7 +17,7 @@ use crate::context::Context;
use crate::download::{download_known_post_messages_without_pre_message, download_msgs};
use crate::ephemeral::{self, delete_expired_imap_messages};
use crate::events::EventType;
use crate::imap::{FolderMeaning, Imap, session::Session};
use crate::imap::{Imap, session::Session};
use crate::location;
use crate::log::{LogExt, warn};
use crate::smtp::{Smtp, send_smtp_messages};
@@ -211,25 +211,19 @@ impl SchedulerState {
/// Indicate that the network likely has come back.
pub(crate) async fn maybe_network(&self) {
let inner = self.inner.read().await;
let (inboxes, oboxes) = match *inner {
let inboxes = match *inner {
InnerSchedulerState::Started(ref scheduler) => {
scheduler.maybe_network();
let inboxes = scheduler
scheduler
.inboxes
.iter()
.map(|b| b.conn_state.state.connectivity.clone())
.collect::<Vec<_>>();
let oboxes = scheduler
.oboxes
.iter()
.map(|b| b.conn_state.state.connectivity.clone())
.collect::<Vec<_>>();
(inboxes, oboxes)
.collect::<Vec<_>>()
}
_ => return,
};
drop(inner);
connectivity::idle_interrupted(inboxes, oboxes);
connectivity::idle_interrupted(inboxes);
}
/// Indicate that the network likely is lost.
@@ -256,6 +250,16 @@ impl SchedulerState {
}
}
pub(crate) async fn clear_all_relay_storage(&self) -> Result<()> {
let inner = self.inner.read().await;
if let InnerSchedulerState::Started(ref scheduler) = *inner {
scheduler.clear_all_relay_storage();
Ok(())
} else {
bail!("IO is not started");
}
}
pub(crate) async fn interrupt_smtp(&self) {
let inner = self.inner.read().await;
if let InnerSchedulerState::Started(ref scheduler) = *inner {
@@ -318,7 +322,10 @@ impl Drop for IoPausedGuard {
struct SchedBox {
/// Address at the used chatmail/email relay
addr: String,
meaning: FolderMeaning,
/// Folder name
folder: String,
conn_state: ImapConnectionState,
/// IMAP loop task handle.
@@ -330,8 +337,6 @@ struct SchedBox {
pub(crate) struct Scheduler {
/// Inboxes, one per transport.
inboxes: Vec<SchedBox>,
/// Optional boxes -- mvbox.
oboxes: Vec<SchedBox>,
smtp: SmtpConnectionState,
smtp_handle: task::JoinHandle<()>,
ephemeral_handle: task::JoinHandle<()>,
@@ -353,6 +358,7 @@ async fn inbox_loop(
let ImapConnectionHandlers {
mut connection,
stop_token,
clear_storage_request_receiver,
} = inbox_handlers;
let transport_id = connection.transport_id();
@@ -391,7 +397,14 @@ async fn inbox_loop(
}
};
match inbox_fetch_idle(&ctx, &mut connection, session).await {
match inbox_fetch_idle(
&ctx,
&mut connection,
session,
&clear_storage_request_receiver,
)
.await
{
Err(err) => warn!(
ctx,
"Transport {transport_id}: Failed inbox fetch_idle: {err:#}."
@@ -412,42 +425,30 @@ async fn inbox_loop(
.await;
}
/// Convert folder meaning
/// used internally by [fetch_idle] and [Context::background_fetch].
///
/// Returns folder configuration key and folder name
/// if such folder is configured, `Ok(None)` otherwise.
pub async fn convert_folder_meaning(
async fn inbox_fetch_idle(
ctx: &Context,
folder_meaning: FolderMeaning,
) -> Result<Option<(Config, String)>> {
let folder_config = match folder_meaning.to_config() {
Some(c) => c,
None => {
// Such folder cannot be configured,
// e.g. a `FolderMeaning::Spam` folder.
return Ok(None);
}
};
let folder = ctx
.get_config(folder_config)
.await
.with_context(|| format!("Failed to retrieve {folder_config} folder"))?;
if let Some(watch_folder) = folder {
Ok(Some((folder_config, watch_folder)))
} else {
Ok(None)
}
}
async fn inbox_fetch_idle(ctx: &Context, imap: &mut Imap, mut session: Session) -> Result<Session> {
imap: &mut Imap,
mut session: Session,
clear_storage_request_receiver: &Receiver<()>,
) -> Result<Session> {
let transport_id = session.transport_id();
// Clear IMAP storage on request.
//
// Only doing this for chatmail relays to avoid
// accidentally deleting all emails in a shared mailbox.
let should_clear_imap_storage =
clear_storage_request_receiver.try_recv().is_ok() && session.is_chatmail();
if should_clear_imap_storage {
info!(ctx, "Transport {transport_id}: Clearing IMAP storage.");
session.delete_all_messages(ctx, &imap.folder).await?;
}
// Update quota no more than once a minute.
if ctx.quota_needs_update(session.transport_id(), 60).await
&& let Err(err) = ctx.update_recent_quota(&mut session).await
//
// Always update if we just cleared IMAP storage.
if (ctx.quota_needs_update(session.transport_id(), 60).await || should_clear_imap_storage)
&& let Err(err) = ctx.update_recent_quota(&mut session, &imap.folder).await
{
warn!(
ctx,
@@ -473,6 +474,11 @@ async fn inbox_fetch_idle(ctx: &Context, imap: &mut Imap, mut session: Session)
last_housekeeping_time.saturating_add(constants::HOUSEKEEPING_PERIOD);
if next_housekeeping_time <= time() {
sql::housekeeping(ctx).await.log_err(ctx).ok();
} else {
let force_truncate = false;
if let Err(err) = ctx.sql.wal_checkpoint(ctx, force_truncate).await {
warn!(ctx, "wal_checkpoint() failed: {err:#}.");
}
}
}
Err(err) => {
@@ -494,7 +500,7 @@ async fn inbox_fetch_idle(ctx: &Context, imap: &mut Imap, mut session: Session)
.await
.context("Failed to register push token")?;
let session = fetch_idle(ctx, imap, session, FolderMeaning::Inbox).await?;
let session = fetch_idle(ctx, imap, session).await?;
Ok(session)
}
@@ -503,34 +509,19 @@ async fn inbox_fetch_idle(ctx: &Context, imap: &mut Imap, mut session: Session)
/// This function performs all IMAP operations on a single folder, selecting it if necessary and
/// handling all the errors. In case of an error, an error is returned and connection is dropped,
/// otherwise connection is returned.
async fn fetch_idle(
ctx: &Context,
connection: &mut Imap,
mut session: Session,
folder_meaning: FolderMeaning,
) -> Result<Session> {
async fn fetch_idle(ctx: &Context, connection: &mut Imap, mut session: Session) -> Result<Session> {
let transport_id = session.transport_id();
let Some((folder_config, watch_folder)) = convert_folder_meaning(ctx, folder_meaning).await?
else {
// The folder is not configured.
// For example, this happens if the server does not have Sent folder
// but watching Sent folder is enabled.
connection.connectivity.set_not_configured(ctx);
connection.idle_interrupt_receiver.recv().await.ok();
bail!("Cannot fetch folder {folder_meaning} because it is not configured");
};
let watch_folder = connection.folder.clone();
if folder_config == Config::ConfiguredInboxFolder {
session
.store_seen_flags_on_imap(ctx)
.await
.context("store_seen_flags_on_imap")?;
}
session
.store_seen_flags_on_imap(ctx)
.await
.context("store_seen_flags_on_imap")?;
// Fetch the watched folder.
connection
.fetch_move_delete(ctx, &mut session, &watch_folder, folder_meaning)
.fetch_move_delete(ctx, &mut session, &watch_folder)
.await
.context("fetch_move_delete")?;
@@ -564,7 +555,7 @@ async fn fetch_idle(
ctx,
"Transport {transport_id}: IMAP session does not support IDLE, going to fake idle."
);
connection.fake_idle(ctx, watch_folder).await?;
connection.fake_idle(ctx, &watch_folder).await?;
return Ok(session);
}
@@ -579,7 +570,7 @@ async fn fetch_idle(
ctx,
"Transport {transport_id}: IMAP IDLE is disabled, going to fake idle."
);
connection.fake_idle(ctx, watch_folder).await?;
connection.fake_idle(ctx, &watch_folder).await?;
return Ok(session);
}
@@ -595,69 +586,6 @@ async fn fetch_idle(
Ok(session)
}
/// Simplified IMAP loop to watch non-inbox folders.
async fn simple_imap_loop(
ctx: Context,
started: oneshot::Sender<()>,
inbox_handlers: ImapConnectionHandlers,
folder_meaning: FolderMeaning,
) {
use futures::future::FutureExt;
info!(ctx, "Starting simple loop for {folder_meaning}.");
let ImapConnectionHandlers {
mut connection,
stop_token,
} = inbox_handlers;
let ctx1 = ctx.clone();
let fut = async move {
let ctx = ctx1;
if let Err(()) = started.send(()) {
warn!(
ctx,
"Simple imap loop for {folder_meaning}, missing started receiver."
);
return;
}
let mut old_session: Option<Session> = None;
loop {
let session = if let Some(session) = old_session.take() {
session
} else {
info!(ctx, "Preparing new IMAP session for {folder_meaning}.");
match connection.prepare(&ctx).await {
Err(err) => {
warn!(
ctx,
"Failed to prepare {folder_meaning} connection: {err:#}."
);
continue;
}
Ok(session) => session,
}
};
match fetch_idle(&ctx, &mut connection, session, folder_meaning).await {
Err(err) => warn!(ctx, "Failed fetch_idle: {err:#}"),
Ok(session) => {
old_session = Some(session);
}
}
}
};
stop_token
.cancelled()
.map(|_| {
info!(ctx, "Shutting down IMAP loop for {folder_meaning}.");
})
.race(fut)
.await;
}
async fn smtp_loop(
ctx: Context,
started: oneshot::Sender<()>,
@@ -760,7 +688,6 @@ impl Scheduler {
let (location_interrupt_send, location_interrupt_recv) = channel::bounded(1);
let mut inboxes = Vec::new();
let mut oboxes = Vec::new();
let mut start_recvs = Vec::new();
for (transport_id, configured_login_param) in ConfiguredLoginParam::load_all(ctx).await? {
@@ -772,30 +699,17 @@ impl Scheduler {
task::spawn(inbox_loop(ctx, inbox_start_send, inbox_handlers))
};
let addr = configured_login_param.addr.clone();
let folder = configured_login_param
.imap_folder
.unwrap_or_else(|| "INBOX".to_string());
let inbox = SchedBox {
addr: addr.clone(),
meaning: FolderMeaning::Inbox,
folder,
conn_state,
handle,
};
inboxes.push(inbox);
start_recvs.push(inbox_start_recv);
if ctx.should_watch_mvbox().await? {
let (conn_state, handlers) =
ImapConnectionState::new(ctx, transport_id, configured_login_param).await?;
let (start_send, start_recv) = oneshot::channel();
let ctx = ctx.clone();
let meaning = FolderMeaning::Mvbox;
let handle = task::spawn(simple_imap_loop(ctx, start_send, handlers, meaning));
oboxes.push(SchedBox {
addr,
meaning,
conn_state,
handle,
});
start_recvs.push(start_recv);
}
}
let smtp_handle = {
@@ -822,7 +736,6 @@ impl Scheduler {
let res = Self {
inboxes,
oboxes,
smtp,
smtp_handle,
ephemeral_handle,
@@ -842,7 +755,7 @@ impl Scheduler {
}
fn boxes(&self) -> impl Iterator<Item = &SchedBox> {
self.inboxes.iter().chain(self.oboxes.iter())
self.inboxes.iter()
}
fn maybe_network(&self) {
@@ -865,6 +778,12 @@ impl Scheduler {
}
}
fn clear_all_relay_storage(&self) {
for b in &self.inboxes {
b.conn_state.clear_relay_storage();
}
}
fn interrupt_smtp(&self) {
self.smtp.interrupt();
}
@@ -896,7 +815,7 @@ impl Scheduler {
let timeout_duration = std::time::Duration::from_secs(30);
let tracker = TaskTracker::new();
for b in self.inboxes.into_iter().chain(self.oboxes) {
for b in self.inboxes {
let context = context.clone();
tracker.spawn(async move {
tokio::time::timeout(timeout_duration, b.handle)
@@ -998,6 +917,13 @@ struct SmtpConnectionHandlers {
#[derive(Debug)]
pub(crate) struct ImapConnectionState {
state: ConnectionState,
/// Channel to request clearing the folder.
///
/// IMAP loop receiving this should clear the folder
/// on the next iteration if IMAP server is a chatmail relay
/// and otherwise ignore the request.
clear_storage_request_sender: Sender<()>,
}
impl ImapConnectionState {
@@ -1009,11 +935,13 @@ impl ImapConnectionState {
) -> Result<(Self, ImapConnectionHandlers)> {
let stop_token = CancellationToken::new();
let (idle_interrupt_sender, idle_interrupt_receiver) = channel::bounded(1);
let (clear_storage_request_sender, clear_storage_request_receiver) = channel::bounded(1);
let handlers = ImapConnectionHandlers {
connection: Imap::new(context, transport_id, login_param, idle_interrupt_receiver)
.await?,
stop_token: stop_token.clone(),
clear_storage_request_receiver,
};
let state = ConnectionState {
@@ -1022,7 +950,10 @@ impl ImapConnectionState {
connectivity: handlers.connection.connectivity.clone(),
};
let conn = ImapConnectionState { state };
let conn = ImapConnectionState {
state,
clear_storage_request_sender,
};
Ok((conn, handlers))
}
@@ -1036,10 +967,19 @@ impl ImapConnectionState {
fn stop(&self) {
self.state.stop();
}
/// Requests clearing relay storage and interrupts the inbox.
fn clear_relay_storage(&self) {
self.clear_storage_request_sender.try_send(()).ok();
self.state.interrupt();
}
}
#[derive(Debug)]
struct ImapConnectionHandlers {
connection: Imap,
stop_token: CancellationToken,
/// Channel receiver to get requests to clear IMAP storage.
pub(crate) clear_storage_request_receiver: Receiver<()>,
}
+27 -85
View File
@@ -5,11 +5,10 @@ use std::{iter::once, ops::Deref, sync::Arc};
use anyhow::Result;
use humansize::{BINARY, format_size};
use crate::context::Context;
use crate::events::EventType;
use crate::imap::{FolderMeaning, get_watched_folder_configs};
use crate::quota::{QUOTA_ERROR_THRESHOLD_PERCENTAGE, QUOTA_WARN_THRESHOLD_PERCENTAGE};
use crate::stock_str;
use crate::{context::Context, log::LogExt};
use super::InnerSchedulerState;
@@ -67,40 +66,33 @@ enum DetailedConnectivity {
/// Connection is established and is idle.
Idle,
/// The folder was configured not to be watched or configured_*_folder is not set
NotConfigured,
}
impl DetailedConnectivity {
fn to_basic(&self) -> Option<Connectivity> {
fn to_basic(&self) -> Connectivity {
match self {
DetailedConnectivity::Error(_) => Some(Connectivity::NotConnected),
DetailedConnectivity::Uninitialized => Some(Connectivity::NotConnected),
DetailedConnectivity::Connecting => Some(Connectivity::Connecting),
DetailedConnectivity::Working => Some(Connectivity::Working),
DetailedConnectivity::InterruptingIdle => Some(Connectivity::Working),
DetailedConnectivity::Error(_) => Connectivity::NotConnected,
DetailedConnectivity::Uninitialized => Connectivity::NotConnected,
DetailedConnectivity::Connecting => Connectivity::Connecting,
DetailedConnectivity::Working => Connectivity::Working,
DetailedConnectivity::InterruptingIdle => Connectivity::Working,
// At this point IMAP has just connected,
// but does not know yet if there are messages to download.
// We still convert this to Working state
// so user can see "Updating..." and not "Connected"
// which is reserved for idle state.
DetailedConnectivity::Preparing => Some(Connectivity::Working),
DetailedConnectivity::Preparing => Connectivity::Working,
// Just don't return a connectivity, probably the folder is configured not to be
// watched, so we are not interested in it.
DetailedConnectivity::NotConfigured => None,
DetailedConnectivity::Idle => Some(Connectivity::Connected),
DetailedConnectivity::Idle => Connectivity::Connected,
}
}
fn to_icon(&self) -> String {
match self {
DetailedConnectivity::Error(_)
| DetailedConnectivity::Uninitialized
| DetailedConnectivity::NotConfigured => "<span class=\"red dot\"></span>".to_string(),
DetailedConnectivity::Error(_) | DetailedConnectivity::Uninitialized => {
"<span class=\"red dot\"></span>".to_string()
}
DetailedConnectivity::Connecting => "<span class=\"yellow dot\"></span>".to_string(),
DetailedConnectivity::Preparing
| DetailedConnectivity::Working
@@ -120,7 +112,6 @@ impl DetailedConnectivity {
DetailedConnectivity::InterruptingIdle | DetailedConnectivity::Idle => {
stock_str::connected(context)
}
DetailedConnectivity::NotConfigured => "Not configured".to_string(),
}
}
@@ -139,7 +130,6 @@ impl DetailedConnectivity {
DetailedConnectivity::InterruptingIdle
| DetailedConnectivity::Preparing
| DetailedConnectivity::Idle => stock_str::last_msg_sent_successfully(context),
DetailedConnectivity::NotConfigured => "Not configured".to_string(),
}
}
@@ -151,7 +141,6 @@ impl DetailedConnectivity {
DetailedConnectivity::Working => false,
DetailedConnectivity::InterruptingIdle => false,
DetailedConnectivity::Preparing => false, // Just connected, there may still be work to do.
DetailedConnectivity::NotConfigured => true,
DetailedConnectivity::Idle => true,
}
}
@@ -180,9 +169,6 @@ impl ConnectivityStore {
pub(crate) fn set_preparing(&self, context: &Context) {
self.set(context, DetailedConnectivity::Preparing);
}
pub(crate) fn set_not_configured(&self, context: &Context) {
self.set(context, DetailedConnectivity::NotConfigured);
}
pub(crate) fn set_idle(&self, context: &Context) {
self.set(context, DetailedConnectivity::Idle);
}
@@ -190,7 +176,7 @@ impl ConnectivityStore {
fn get_detailed(&self) -> DetailedConnectivity {
self.0.lock().deref().clone()
}
fn get_basic(&self) -> Option<Connectivity> {
fn get_basic(&self) -> Connectivity {
self.0.lock().to_basic()
}
fn get_all_work_done(&self) -> bool {
@@ -201,27 +187,14 @@ impl ConnectivityStore {
/// Set all folder states to InterruptingIdle in case they were `Idle` before.
/// Called during `dc_maybe_network()` to make sure that `all_work_done()`
/// returns false immediately after `dc_maybe_network()`.
pub(crate) fn idle_interrupted(inboxes: Vec<ConnectivityStore>, oboxes: Vec<ConnectivityStore>) {
pub(crate) fn idle_interrupted(inboxes: Vec<ConnectivityStore>) {
for inbox in inboxes {
let mut connectivity_lock = inbox.0.lock();
// For the inbox, we also have to set the connectivity to InterruptingIdle if it was
// NotConfigured before: If all folders are NotConfigured, dc_get_connectivity()
// returns Connected. But after dc_maybe_network(), dc_get_connectivity() must not
// return Connected until DC is completely done with fetching folders; this also
// includes scan_folders() which happens on the inbox thread.
if *connectivity_lock == DetailedConnectivity::Idle
|| *connectivity_lock == DetailedConnectivity::NotConfigured
{
*connectivity_lock = DetailedConnectivity::InterruptingIdle;
}
}
for state in oboxes {
let mut connectivity_lock = state.0.lock();
if *connectivity_lock == DetailedConnectivity::Idle {
*connectivity_lock = DetailedConnectivity::InterruptingIdle;
}
}
// No need to send ConnectivityChanged, the user-facing connectivity doesn't change because
// of what we do here.
}
@@ -234,9 +207,7 @@ pub(crate) fn maybe_network_lost(context: &Context, stores: Vec<ConnectivityStor
let mut connectivity_lock = store.0.lock();
if !matches!(
*connectivity_lock,
DetailedConnectivity::Uninitialized
| DetailedConnectivity::Error(_)
| DetailedConnectivity::NotConfigured,
DetailedConnectivity::Uninitialized | DetailedConnectivity::Error(_)
) {
*connectivity_lock = DetailedConnectivity::Error("Connection lost".to_string());
}
@@ -273,9 +244,8 @@ impl Context {
let stores = self.connectivities.lock().clone();
let mut connectivities = Vec::new();
for s in stores {
if let Some(connectivity) = s.get_basic() {
connectivities.push(connectivity);
}
let connectivity = s.get_basic();
connectivities.push(connectivity);
}
connectivities
.into_iter()
@@ -386,7 +356,7 @@ impl Context {
.map(|b| {
(
b.addr.clone(),
b.meaning,
b.folder.clone(),
b.conn_state.state.connectivity.clone(),
)
})
@@ -411,7 +381,6 @@ impl Context {
// [======67%===== ]
// =============================================================================================
let watched_folders = get_watched_folder_configs(self).await?;
let incoming_messages = stock_str::incoming_messages(self);
ret += &format!("<h3>{incoming_messages}</h3><ul>");
@@ -433,41 +402,14 @@ impl Context {
let folders = folders_states
.iter()
.filter(|(folder_addr, ..)| *folder_addr == transport_addr);
for (_addr, folder, state) in folders {
let mut folder_added = false;
if let Some(config) = folder.to_config().filter(|c| watched_folders.contains(c)) {
let f = self.get_config(config).await.log_err(self).ok().flatten();
if let Some(foldername) = f {
let detailed = &state.get_detailed();
ret += &*detailed.to_icon();
ret += " <b>";
if folder == &FolderMeaning::Inbox {
ret += &*domain_escaped;
} else {
ret += &*escaper::encode_minimal(&foldername);
}
ret += ":</b> ";
ret += &*escaper::encode_minimal(&detailed.to_string_imap(self));
ret += "<br />";
folder_added = true;
}
}
if !folder_added && folder == &FolderMeaning::Inbox {
let detailed = &state.get_detailed();
if let DetailedConnectivity::Error(_) = detailed {
// On the inbox thread, we also do some other things like scan_folders and run jobs
// so, maybe, the inbox is not watched, but something else went wrong
ret += &*detailed.to_icon();
ret += " ";
ret += &*escaper::encode_minimal(&detailed.to_string_imap(self));
ret += "<br />";
}
}
for (_addr, _folder, state) in folders {
let detailed = &state.get_detailed();
ret += &*detailed.to_icon();
ret += " <b>";
ret += &*domain_escaped;
ret += ":</b> ";
ret += &*escaper::encode_minimal(&detailed.to_string_imap(self));
ret += "<br />";
}
let Some(quota) = quota.get(&transport_id) else {
+48
View File
@@ -1584,3 +1584,51 @@ async fn test_auth_token_is_synchronized() -> Result<()> {
Ok(())
}
/// Tests that Bob deduplicates messages about being added to the group.
///
/// If Alice (inviter) has multiple devices,
/// Bob may receive multiple "member added" messages.
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_deduplicate_member_added() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice1 = &tcm.alice().await;
let alice2 = &tcm.alice().await;
let bob = &tcm.bob().await;
// Enable sync messages so Alice QR code tokens
// are synchronized.
alice1.set_config_bool(Config::SyncMsgs, true).await?;
alice2.set_config_bool(Config::SyncMsgs, true).await?;
tcm.section("Alice creates a group");
let alice1_chat_id = chat::create_group(alice1, "Group").await?;
tcm.section("Alice creates group QR code and synchronizes it");
let qr = get_securejoin_qr(alice1, Some(alice1_chat_id)).await?;
sync(alice1, alice2).await;
// Import Alice's key to skip key request step.
tcm.section("Bob imports Alice's vCard");
bob.add_or_lookup_contact_id(alice1).await;
tcm.section("Bob scans the QR code");
let bob_chat_id = join_securejoin(bob, &qr).await?;
tcm.section("Both Alice's devices add Bob to chat");
let sent = bob.pop_sent_msg().await;
alice1.recv_msg_trash(&sent).await;
alice2.recv_msg_trash(&sent).await;
let sent1 = alice1.pop_sent_msg().await;
let sent2 = alice2.pop_sent_msg().await;
let bob_rcvd = bob.recv_msg(&sent1).await;
assert_eq!(bob_rcvd.chat_id, bob_chat_id);
assert_eq!(bob_rcvd.text, "Member Me added by alice@example.org.");
// Second message is a no-op, so it is trashed.
bob.recv_msg_trash(&sent2).await;
Ok(())
}
-1
View File
@@ -9,7 +9,6 @@ use crate::tools::IsNoneOrEmpty;
/// This escapes a bit more than actually needed by delta (e.g. also lines as "-- footer"),
/// but for non-delta-compatibility, that seems to be better.
/// (to be only compatible with delta, only "[\r\n|\n]-- {0,2}[\r\n|\n]" needs to be replaced)
#[expect(clippy::arithmetic_side_effects)]
pub fn escape_message_footer_marks(text: &str) -> String {
if let Some(text) = text.strip_prefix("--") {
"-\u{200B}-".to_string() + &text.replace("\n--", "\n-\u{200B}-")
+1 -1
View File
@@ -580,7 +580,7 @@ async fn send_mdn_rfc724_mid(
)
.await?;
let encrypted = mimefactory.will_be_encrypted();
let rendered_msg = mimefactory.render(context).await?;
let rendered_msg = Box::pin(mimefactory.render(context)).await?;
let body = rendered_msg.message;
let mut recipients = Vec::new();
+32 -3
View File
@@ -11,11 +11,12 @@ use crate::log::warn;
use crate::net::dns::{lookup_host_with_cache, update_connect_timestamp};
use crate::net::proxy::ProxyConfig;
use crate::net::session::SessionBufStream;
use crate::net::tls::{TlsSessionStore, wrap_tls};
use crate::net::tls::{SpkiHashStore, TlsSessionStore, wrap_tls};
use crate::net::{
connect_tcp_inner, connect_tls_inner, run_connection_attempts, update_connection_history,
};
use crate::oauth2::get_oauth2_access_token;
use crate::sql::Sql;
use crate::tools::time;
use crate::transport::ConnectionCandidate;
use crate::transport::ConnectionSecurity;
@@ -111,10 +112,26 @@ async fn connection_attempt(
);
let res = match security {
ConnectionSecurity::Tls => {
connect_secure(resolved_addr, host, strict_tls, &context.tls_session_store).await
connect_secure(
resolved_addr,
host,
strict_tls,
&context.tls_session_store,
&context.spki_hash_store,
&context.sql,
)
.await
}
ConnectionSecurity::Starttls => {
connect_starttls(resolved_addr, host, strict_tls, &context.tls_session_store).await
connect_starttls(
resolved_addr,
host,
strict_tls,
&context.tls_session_store,
&context.spki_hash_store,
&context.sql,
)
.await
}
ConnectionSecurity::Plain => connect_insecure(resolved_addr).await,
};
@@ -240,6 +257,8 @@ async fn connect_secure_proxy(
alpn(port),
proxy_stream,
&context.tls_session_store,
&context.spki_hash_store,
&context.sql,
)
.await?;
let mut buffered_stream = BufStream::new(tls_stream);
@@ -273,6 +292,8 @@ async fn connect_starttls_proxy(
"",
tcp_stream,
&context.tls_session_store,
&context.spki_hash_store,
&context.sql,
)
.await
.context("STARTTLS upgrade failed")?;
@@ -299,6 +320,8 @@ async fn connect_secure(
hostname: &str,
strict_tls: bool,
tls_session_store: &TlsSessionStore,
spki_hash_store: &SpkiHashStore,
sql: &Sql,
) -> Result<Box<dyn SessionBufStream>> {
let tls_stream = connect_tls_inner(
addr,
@@ -306,6 +329,8 @@ async fn connect_secure(
strict_tls,
alpn(addr.port()),
tls_session_store,
spki_hash_store,
sql,
)
.await?;
let mut buffered_stream = BufStream::new(tls_stream);
@@ -319,6 +344,8 @@ async fn connect_starttls(
host: &str,
strict_tls: bool,
tls_session_store: &TlsSessionStore,
spki_hash_store: &SpkiHashStore,
sql: &Sql,
) -> Result<Box<dyn SessionBufStream>> {
let use_sni = false;
let tcp_stream = connect_tcp_inner(addr).await?;
@@ -336,6 +363,8 @@ async fn connect_starttls(
"",
tcp_stream,
tls_session_store,
spki_hash_store,
sql,
)
.await
.context("STARTTLS upgrade failed")?;
+28 -25
View File
@@ -9,7 +9,6 @@ use rusqlite::{Connection, OpenFlags, Row, config::DbConfig, types::ValueRef};
use tokio::sync::RwLock;
use crate::blob::BlobObject;
use crate::chat::add_device_msg;
use crate::config::Config;
use crate::constants::DC_CHAT_ID_TRASH;
use crate::context::Context;
@@ -18,13 +17,11 @@ use crate::ephemeral::start_ephemeral_timers;
use crate::imex::BLOBS_BACKUP_NAME;
use crate::location::delete_orphaned_poi_locations;
use crate::log::{LogExt, warn};
use crate::message::Message;
use crate::message::MsgId;
use crate::net::dns::prune_dns_cache;
use crate::net::http::http_cache_cleanup;
use crate::net::prune_connection_history;
use crate::param::{Param, Params};
use crate::stock_str;
use crate::tools::{SystemTime, delete_file, time};
/// Extension to [`rusqlite::ToSql`] trait
@@ -264,6 +261,12 @@ impl Sql {
info!(context, "Opened database {:?}.", self.dbfile);
*self.is_encrypted.write().await = Some(passphrase_nonempty);
// Some migrations want housekeeping to run. Also if housekeeping failed before, fixing the
// reason and restarting the program is the most natural way to retry it.
context
.set_config_internal(Config::LastHousekeeping, None)
.await?;
// setup debug logging if there is an entry containing its id
if let Some(xdc_id) = self
.get_raw_config_u32(Config::DebugLogging.as_ref())
@@ -663,8 +666,16 @@ impl Sql {
&self.config_cache
}
/// Attempts to truncate the WAL file.
pub(crate) async fn wal_checkpoint(&self, context: &Context) -> Result<()> {
/// Runs a WAL checkpoint operation.
///
/// * `force_truncate` - Force TRUNCATE mode to truncate the WAL file to 0 bytes, otherwise only
/// run PASSIVE mode if the WAL isn't too large. NB: Truncating blocks all db connections for
/// some time.
pub(crate) async fn wal_checkpoint(
&self,
context: &Context,
force_truncate: bool,
) -> Result<()> {
let lock = self.pool.read().await;
let Some(pool) = lock.as_ref() else {
// No db connections, nothing to checkpoint.
@@ -677,7 +688,7 @@ impl Sql {
readers_blocked_duration,
pages_total,
pages_checkpointed,
} = pool.wal_checkpoint().await?;
} = pool.wal_checkpoint(force_truncate).await?;
if pages_checkpointed < pages_total {
warn!(
context,
@@ -708,6 +719,7 @@ fn new_connection(path: &Path, passphrase: &str) -> Result<Connection> {
PRAGMA secure_delete=on;
PRAGMA soft_heap_limit = 8388608; -- 8 MiB limit, same as set in Android SQLiteDatabase.
PRAGMA foreign_keys=on;
PRAGMA wal_autocheckpoint=N;
",
)?;
@@ -830,12 +842,6 @@ pub async fn housekeeping(context: &Context) -> Result<()> {
);
}
maybe_add_mvbox_move_deprecation_message(context)
.await
.context("maybe_add_mvbox_move_deprecation_message")
.log_err(context)
.ok();
if let Err(err) = incremental_vacuum(context).await {
warn!(context, "Failed to run incremental vacuum: {err:#}.");
}
@@ -843,7 +849,8 @@ pub async fn housekeeping(context: &Context) -> Result<()> {
// bigger than 200M) and also make sure we truncate the WAL periodically. Auto-checkponting does
// not normally truncate the WAL (unless the `journal_size_limit` pragma is set), see
// https://www.sqlite.org/wal.html.
if let Err(err) = Sql::wal_checkpoint(&context.sql, context).await {
let force_truncate = true;
if let Err(err) = Sql::wal_checkpoint(&context.sql, context, force_truncate).await {
warn!(context, "wal_checkpoint() failed: {err:#}.");
debug_assert!(false);
}
@@ -883,6 +890,14 @@ pub async fn housekeeping(context: &Context) -> Result<()> {
.log_err(context)
.ok();
context
.spki_hash_store
.cleanup(&context.sql)
.await
.context("Failed to prune SPKI store")
.log_err(context)
.ok();
// Cleanup `imap` and `imap_sync` entries for deleted transports.
//
// Transports may be deleted directly or via sync messages,
@@ -913,18 +928,6 @@ pub async fn housekeeping(context: &Context) -> Result<()> {
Ok(())
}
/// Adds device message about `mvbox_move` config deprecation
/// if the user has it enabled.
async fn maybe_add_mvbox_move_deprecation_message(context: &Context) -> Result<()> {
if !context.get_config_bool(Config::OnlyFetchMvbox).await?
&& context.get_config_bool(Config::MvboxMove).await?
{
let mut msg = Message::new_text(stock_str::mvbox_move_deprecation(context));
add_device_msg(context, Some("mvbox_move_deprecation"), Some(&mut msg)).await?;
}
Ok(())
}
/// Get the value of a column `idx` of the `row` as `Vec<u8>`.
pub fn row_get_vec(row: &Row, idx: usize) -> rusqlite::Result<Vec<u8>> {
row.get(idx).or_else(|err| match row.get_ref(idx)? {
+60
View File
@@ -2316,6 +2316,66 @@ ALTER TABLE contacts ADD COLUMN name_normalized TEXT;
.await?;
}
inc_and_check(&mut migration_version, 150)?;
if dbversion < migration_version {
sql.execute_migration_transaction(
|transaction| {
let only_fetch_mvbox = transaction
.query_row(
"SELECT value FROM config WHERE keyname='only_fetch_mvbox'",
(),
|row| {
let value: String = row.get(0)?;
Ok(value)
},
)
.optional()?
.as_deref()
== Some("1");
if only_fetch_mvbox {
let mvbox_folder = transaction
.query_row(
"SELECT value FROM config WHERE keyname='configured_mvbox_folder'",
(),
|row| {
let value: String = row.get(0)?;
Ok(value)
},
)
.optional()?
.unwrap_or_else(|| "DeltaChat".to_string());
transaction.execute(
"UPDATE transports
SET entered_param=json_set(entered_param, '$.imap.folder', ?1),
configured_param=json_set(configured_param', '$.imap_folder', ?1)",
(mvbox_folder,),
)?;
}
Ok(())
},
migration_version,
)
.await?;
}
inc_and_check(&mut migration_version, 151)?;
if dbversion < migration_version {
sql.execute_migration(
"CREATE TABLE tls_spki (
host TEXT NOT NULL UNIQUE,
spki_hash TEXT NOT NULL, -- base64 of SPKI SHA-256 hash
timestamp INTEGER NOT NULL -- timestamp of the last time we have seen this key
) STRICT;
-- Index on host column is created implicitly because of UNIQUE constraint.
CREATE INDEX tls_spki_index_timestamp ON tls_spki (timestamp);
",
migration_version,
)
.await?;
}
let new_version = sql
.get_raw_config_int(VERSION_CFG)
.await?
+3 -3
View File
@@ -199,8 +199,8 @@ impl Pool {
Arc::clone(&self.inner).get(query_only).await
}
/// Truncates the WAL file.
pub(crate) async fn wal_checkpoint(&self) -> Result<WalCheckpointStats> {
wal_checkpoint::wal_checkpoint(self).await
/// Runs a WAL checkpoint operation.
pub(crate) async fn wal_checkpoint(&self, force_truncate: bool) -> Result<WalCheckpointStats> {
wal_checkpoint::wal_checkpoint(self, force_truncate).await
}
}
+24 -5
View File
@@ -26,26 +26,45 @@ pub(crate) struct WalCheckpointStats {
/// Number of checkpointed WAL pages.
///
/// It should be the same as `pages_total`
/// If TRUNCATE is forced, it should be the same as `pages_total`
/// unless there are external connections to the database
/// that are not in the pool.
pub pages_checkpointed: i64,
}
/// Runs a checkpoint operation in TRUNCATE mode, so the WAL file is truncated to 0 bytes.
pub(super) async fn wal_checkpoint(pool: &Pool) -> Result<WalCheckpointStats> {
/// Runs a WAL checkpoint operation.
///
/// * `force_truncate` - Force TRUNCATE mode to truncate the WAL file to 0 bytes, otherwise only run
/// PASSIVE mode if the WAL isn't too large.
pub(super) async fn wal_checkpoint(
pool: &Pool,
force_truncate: bool,
) -> Result<WalCheckpointStats> {
let t_start = Time::now();
// Do as much work as possible without blocking anybody.
let query_only = true;
let conn = pool.get(query_only).await?;
tokio::task::block_in_place(|| {
let (pages_total, pages_checkpointed) = tokio::task::block_in_place(|| {
// Execute some transaction causing the WAL file to be opened so that the
// `wal_checkpoint()` can proceed, otherwise it fails when called the first time,
// see https://sqlite.org/forum/forumpost/7512d76a05268fc8.
conn.query_row("PRAGMA table_list", [], |_| Ok(()))?;
conn.query_row("PRAGMA wal_checkpoint(PASSIVE)", [], |_| Ok(()))
conn.query_row("PRAGMA wal_checkpoint(PASSIVE)", [], |row| {
let pages_total: i64 = row.get(1)?;
let pages_checkpointed: i64 = row.get(2)?;
Ok((pages_total, pages_checkpointed))
})
})?;
if !force_truncate && pages_total < 4096 {
return Ok(WalCheckpointStats {
total_duration: time_elapsed(&t_start),
writers_blocked_duration: Duration::ZERO,
readers_blocked_duration: Duration::ZERO,
pages_total,
pages_checkpointed,
});
}
// Kick out writers. `write_mutex` should be locked before taking an `InnerPool.semaphore`
// permit to avoid ABBA deadlocks, so drop `conn` which holds a semaphore permit.
-10
View File
@@ -413,11 +413,6 @@ https://delta.chat/donate"))]
#[strum(props(fallback = "Messages in this chat use classic email and are not encrypted."))]
ChatUnencryptedExplanation = 230,
#[strum(props(
fallback = "You are using the legacy option \"Settings → Advanced → Move automatically to DeltaChat Folder\".\n\nThis option will be removed in a few weeks and you should disable it already today.\n\nIf having chat messages mixed into your inbox is a problem, see https://delta.chat/legacy-move"
))]
MvboxMoveDeprecation = 231,
#[strum(props(fallback = "Outgoing audio call"))]
OutgoingAudioCall = 232,
@@ -1225,11 +1220,6 @@ pub(crate) fn chat_unencrypted_explanation(context: &Context) -> String {
translated(context, StockMessage::ChatUnencryptedExplanation)
}
/// Stock string: `You are using the legacy option "Move automatically to DeltaChat Folder`…
pub(crate) fn mvbox_move_deprecation(context: &Context) -> String {
translated(context, StockMessage::MvboxMoveDeprecation)
}
impl Viewtype {
/// returns Localized name for message viewtype
pub fn to_locale_string(&self, context: &Context) -> String {
+2 -2
View File
@@ -19,7 +19,7 @@ use crate::token::Namespace;
use crate::tools::time;
use crate::transport::{ConfiguredLoginParamJson, sync_transports};
use crate::{message, stock_str, token};
use std::collections::HashSet;
use std::collections::BTreeSet;
/// Whether to send device sync messages. Aimed for usage in the internal API.
#[derive(Debug, PartialEq)]
@@ -383,7 +383,7 @@ impl Context {
}
async fn sync_message_deletion(&self, msgs: &Vec<String>) -> Result<()> {
let mut modified_chat_ids = HashSet::new();
let mut modified_chat_ids = BTreeSet::new();
let mut msg_ids = Vec::new();
for rfc724_mid in msgs {
if let Some(msg_id) = message::rfc724_mid_exists(self, rfc724_mid).await? {
-1
View File
@@ -567,7 +567,6 @@ impl TestContext {
.unwrap();
ctx.set_config(Config::BccSelf, Some("1")).await.unwrap();
ctx.set_config(Config::SyncMsgs, Some("0")).await.unwrap();
ctx.set_config(Config::MvboxMove, Some("0")).await.unwrap();
Self {
ctx,
+6
View File
@@ -118,7 +118,13 @@ async fn test_receive_pre_message() -> Result<()> {
let msg = bob.recv_msg(&pre_message).await;
assert_eq!(msg.download_state, DownloadState::Available);
assert_summary_texts(&msg, bob, "👤 test").await;
let chat_id = msg.chat_id;
assert!(bob.recv_msg_opt(&pre_message).await.is_none());
let msg = Message::load_from_db(bob, msg.id).await?;
assert_eq!(msg.download_state, DownloadState::Available);
assert_summary_texts(&msg, bob, "👤 test").await;
assert_eq!(msg.chat_id, chat_id);
Ok(())
}
+2 -3
View File
@@ -507,9 +507,8 @@ async fn test_verified_member_added_reordering() -> Result<()> {
assert_eq!(fiona_received_message.get_text(), "Hi");
// Fiona receives late "Member added" message
// and the chat becomes protected.
fiona.recv_msg(&alice_sent_member_added).await;
// Fiona receives late "Member added" message.
fiona.recv_msg_trash(&alice_sent_member_added).await;
Ok(())
}
-1
View File
@@ -687,7 +687,6 @@ fn extract_address_from_receive_header<'a>(header: &'a str, start: &str) -> Opti
})
}
#[expect(clippy::arithmetic_side_effects)]
pub(crate) fn parse_receive_header(header: &str) -> String {
let header = header.replace(&['\r', '\n'][..], "");
let mut hop_info = String::from("Hop: ");
+41 -9
View File
@@ -19,6 +19,7 @@ use crate::config::Config;
use crate::configure::server_params::{ServerParams, expand_param_vector};
use crate::constants::{DC_LP_AUTH_FLAGS, DC_LP_AUTH_OAUTH2};
use crate::context::Context;
use crate::ensure_and_debug_assert;
use crate::events::EventType;
use crate::login_param::EnteredLoginParam;
use crate::net::load_connection_timestamp;
@@ -163,22 +164,30 @@ pub(crate) struct ConfiguredLoginParam {
/// `From:` address that was used at the time of configuration.
pub addr: String,
/// List of IMAP candidates to try.
pub imap: Vec<ConfiguredServerLoginParam>,
// Custom IMAP user.
//
// This overwrites autoconfig from the provider database
// if non-empty.
/// Custom IMAP user.
///
/// This overwrites autoconfig from the provider database
/// if non-empty.
pub imap_user: String,
pub imap_password: String,
// IMAP folder to watch.
//
// If not stored, should be interpreted as "INBOX".
// If stored, should be a folder name and not empty.
pub imap_folder: Option<String>,
/// List of SMTP candidates to try.
pub smtp: Vec<ConfiguredServerLoginParam>,
// Custom SMTP user.
//
// This overwrites autoconfig from the provider database
// if non-empty.
/// Custom SMTP user.
///
/// This overwrites autoconfig from the provider database
/// if non-empty.
pub smtp_user: String,
pub smtp_password: String,
@@ -199,6 +208,13 @@ pub(crate) struct ConfiguredLoginParam {
pub(crate) struct ConfiguredLoginParamJson {
pub addr: String,
pub imap: Vec<ConfiguredServerLoginParam>,
/// IMAP folder to watch.
///
/// Defaults to "INBOX" if unset.
#[serde(skip_serializing_if = "Option::is_none")]
pub imap_folder: Option<String>,
pub imap_user: String,
pub imap_password: String,
pub smtp: Vec<ConfiguredServerLoginParam>,
@@ -545,6 +561,7 @@ impl ConfiguredLoginParam {
Ok(Some(ConfiguredLoginParam {
addr,
imap,
imap_folder: None,
imap_user: mail_user,
imap_password: mail_pw,
smtp,
@@ -577,11 +594,18 @@ impl ConfiguredLoginParam {
pub(crate) fn from_json(json: &str) -> Result<Self> {
let json: ConfiguredLoginParamJson = serde_json::from_str(json)?;
ensure_and_debug_assert!(
json.imap_folder
.as_ref()
.is_none_or(|folder| !folder.is_empty()),
"Configured watched folder name cannot be empty"
);
let provider = json.provider_id.and_then(|id| get_provider_by_id(&id));
Ok(ConfiguredLoginParam {
addr: json.addr,
imap: json.imap,
imap_folder: json.imap_folder,
imap_user: json.imap_user,
imap_password: json.imap_password,
smtp: json.smtp,
@@ -619,6 +643,7 @@ impl From<ConfiguredLoginParam> for ConfiguredLoginParamJson {
imap: configured_login_param.imap,
imap_user: configured_login_param.imap_user,
imap_password: configured_login_param.imap_password,
imap_folder: configured_login_param.imap_folder,
smtp: configured_login_param.smtp,
smtp_user: configured_login_param.smtp_user,
smtp_password: configured_login_param.smtp_password,
@@ -638,9 +663,16 @@ pub(crate) async fn save_transport(
add_timestamp: i64,
is_published: bool,
) -> Result<bool> {
ensure_and_debug_assert!(
configured
.imap_folder
.as_ref()
.is_none_or(|folder| !folder.is_empty()),
"Configured watched folder name cannot be empty"
);
let addr = addr_normalize(&configured.addr);
let configured_addr = context.get_config(Config::ConfiguredAddr).await?;
let mut modified = context
.sql
.execute(
+3
View File
@@ -34,6 +34,7 @@ async fn test_save_load_login_param() -> Result<()> {
},
user: "alice".to_string(),
}],
imap_folder: None,
imap_user: "".to_string(),
imap_password: "foo".to_string(),
smtp: vec![ConfiguredServerLoginParam {
@@ -142,6 +143,7 @@ async fn test_posteo_alias() -> Result<()> {
user: user.to_string(),
},
],
imap_folder: None,
imap_user: "alice@posteo.de".to_string(),
imap_password: "foobarbaz".to_string(),
smtp: vec![
@@ -272,6 +274,7 @@ fn dummy_configured_login_param(
},
user: addr.to_string(),
}],
imap_folder: None,
imap_user: addr.to_string(),
imap_password: "foobarbaz".to_string(),
smtp: vec![ConfiguredServerLoginParam {
+13
View File
@@ -111,6 +111,12 @@ pub struct WebxdcInfo {
/// Address to be used for `window.webxdc.selfAddr` in JS land.
pub self_addr: String,
/// Define if the local user is the one who initially shared the webxdc application in the chat.
pub is_app_sender: bool,
/// Define if the app runs in a broadcasting context.
pub is_broadcast: bool,
/// Milliseconds to wait before calling `sendUpdate()` again since the last call.
/// Should be exposed to `window.sendUpdateInterval` in JS land.
pub send_update_interval: usize,
@@ -923,6 +929,11 @@ impl Message {
let internet_access = is_integrated;
let self_addr = self.get_webxdc_self_addr(context).await?;
let is_app_sender = self.from_id == ContactId::SELF;
let chat = Chat::load_from_db(context, self.chat_id)
.await
.with_context(|| "Failed to load chat from the database")?;
let is_broadcast = chat.typ == Chattype::InBroadcast || chat.typ == Chattype::OutBroadcast;
Ok(WebxdcInfo {
name: if let Some(name) = manifest.name {
@@ -961,6 +972,8 @@ impl Message {
request_integration,
internet_access,
self_addr,
is_app_sender,
is_broadcast,
send_update_interval: context.ratelimit.read().await.update_interval(),
send_update_max_size: RECOMMENDED_FILE_SIZE as usize,
})
+40
View File
@@ -12,6 +12,7 @@ use crate::chatlist::Chatlist;
use crate::config::Config;
use crate::ephemeral;
use crate::receive_imf::receive_imf;
use crate::securejoin::get_securejoin_qr;
use crate::test_utils::{E2EE_INFO_MSGS, TestContext, TestContextManager};
use crate::tools::{self, SystemTime};
use crate::{message, sql};
@@ -2195,3 +2196,42 @@ async fn test_self_addr_consistency() -> Result<()> {
assert_eq!(db_msg.get_webxdc_self_addr(alice).await?, self_addr);
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_webxdc_info_app_sender() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
let bob = &tcm.bob().await;
// Alice sends webxdc in a group chat
let alice_chat_id = alice.create_group_with_members("Group", &[bob]).await;
let alice_instance = send_webxdc_instance(alice, alice_chat_id).await?;
let sent1 = alice.pop_sent_msg().await;
let alice_info = alice_instance.get_webxdc_info(alice).await?;
assert!(alice_info.is_app_sender);
assert!(!alice_info.is_broadcast);
// Bob receives group webxdc
let bob_instance = bob.recv_msg(&sent1).await;
let bob_info = bob_instance.get_webxdc_info(bob).await?;
assert!(!bob_info.is_app_sender);
assert!(!bob_info.is_broadcast);
// Alice sends webxdc to broadcast channel
let alice_chat_id = create_broadcast(alice, "Broadcast".to_string()).await?;
let qr = get_securejoin_qr(alice, Some(alice_chat_id)).await.unwrap();
tcm.exec_securejoin_qr(bob, alice, &qr).await;
let alice_instance = send_webxdc_instance(alice, alice_chat_id).await?;
let sent2 = alice.pop_sent_msg().await;
let alice_info = alice_instance.get_webxdc_info(alice).await?;
assert!(alice_info.is_app_sender);
assert!(alice_info.is_broadcast);
// Bob receives broadcast webxdc
let bob_instance = bob.recv_msg(&sent2).await;
let bob_info = bob_instance.get_webxdc_info(bob).await?;
assert!(!bob_info.is_app_sender);
assert!(bob_info.is_broadcast);
Ok(())
}