Compare commits

...
Author SHA1 Message Date
B. Petersen c8c33fbdeb feat: webxdc sending contexts 2026-04-18 12:03:01 +02:00
link2xt 82924952fb feat: allow TLS connections with invalid certificate if the key is unchanged
This change weakens TLS checks.
Every time we make a successful TLS connection,
we remember public key hash from the certificate
in relation to the hostname.
If later we connect to the same hostname and the public key does not change,
we skip checking certificate chain.
This way we will still connect successfully
even if certificate expires or becomes invalid for another reason,
but keeps the key.

We always check that certificate corresponds to the hostname.
We also do this for certificates starting with _
where we allow self-signed certificates,
so self-signed certificates with mismatching domains are not allowed.
Previously we did not check this for domains starting with _.
2026-04-17 18:07:20 +00:00
link2xt 7daa6cc8d9 ci: update Rust to 1.95.0 2026-04-17 15:24:24 +00:00
link2xt b8cfee7e9e build: remove coredeps Dockerfile
It was used to build legacy CFFI Python binding wheels.
We are not building them anymore and it is not
tested that they can still be built.
2026-04-17 15:24:24 +00:00
link2xt 03fc2d26ee ci: remove Concourse CI pipelines
They were used to build legacy Python bindings,
but we stopped updating them and have no Concourse CI setup.
2026-04-17 15:24:24 +00:00
link2xt 942172a31a feat: remove MvboxMove and OnlyFetchMvbox 2026-04-16 16:42:40 +00:00
link2xt 04c0e7da16 feat: do not unconditionally watch mvbox for non-chatmail
Since commit 25750de4e1
released in 2.36.0 we do not move messages to mvbox without explicit
mvbox_move setting, so do not need to watch it as well
as long as other devices are updated to the same change.
2026-04-16 16:42:40 +00:00
iequidoo 4178671839 fix: Scale up contacts messaged in groups to IncomingTo
This makes such contacts appear in the contact list. `IncomingTo` is used because
`ChatId::accept_ex()` does so for groups, so sending a group message is effectively accepting the
group again in regards to contact searchability.

This fixes up b549e7633d which made it impossible to find contacts
from groups even if we've written there.
2026-04-16 12:56:41 -03:00
dependabot[bot] 49e8065b4c chore(deps): bump swatinem/rust-cache from 2.8.2 to 2.9.1
Bumps [swatinem/rust-cache](https://github.com/swatinem/rust-cache) from 2.8.2 to 2.9.1.
- [Release notes](https://github.com/swatinem/rust-cache/releases)
- [Changelog](https://github.com/Swatinem/rust-cache/blob/master/CHANGELOG.md)
- [Commits](https://github.com/swatinem/rust-cache/compare/779680da715d629ac1d338a641029a2f4372abb5...c19371144df3bb44fab255c43d04cbc2ab54d1c4)

---
updated-dependencies:
- dependency-name: swatinem/rust-cache
  dependency-version: 2.9.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-04-16 15:41:17 +00:00
Hocuri 1c24ad91eb feat: Remove the largely-unused ability to send multiple reactions to one message (#8131)
After talking with r10s:

For spring cleaning, remove the largely-unused things that can be done a
bit.

Most private messengers (WhatsApp/Signal/...) do not have this feature,
and we do not want to become Matrix where every client has different,
partly-incompatible features.
2026-04-16 14:30:05 +00:00
link2xt 8f7777b843 ci: upgrade cargo-deny-action to v2.0.17 2026-04-16 16:14:12 +02:00
link2xt caeddcd57b chore: add exception for RUSTSEC-2026-0097 2026-04-16 16:14:12 +02:00
link2xt 418fd92d6d build: upgrade rustls-webpki to 0.103.12
Ignore <https://rustsec.org/advisories/RUSTSEC-2026-0098>
and <https://rustsec.org/advisories/RUSTSEC-2026-0099>
for rustls-webpki 0.102.8 that is an iroh dependency.
2026-04-16 16:14:12 +02:00
iequidoo a70924a1d6 fix: fetch_single_msg(): Lock fetch_msgs_mutex before fetching
This is to avoid race conditions in `receive_imf` which also happened previously when
multi-transport was added.
2026-04-16 10:40:16 -03:00
iequidoo 8e91eecb3d feat: decide_chat_assignment: Log correct post_msg_exists value
rfc724_mid_exists() returns the message id not only when the message is fully downloaded, so we need
to check this additionally. Also improve test_receive_pre_message to check that receiving the
pre-message again doesn't affect the existing message.
2026-04-16 10:40:16 -03:00
iequidooandHocuri 89f948028d fix: Set Param::GuaranteeE2ee before preparing message blob (#8090)
Otherwise `Param::GuaranteeE2ee` is set only after rendering the message
and some UIs, e.g. DC Android, display the message as unencrypted while
preparing the blob and rendering and encrypting the message. NB: DC
Desktop doesn't display the message until `send_msg()` returns.

---------
Co-authored-by: Hocuri <hocuri@gmx.de>
2026-04-15 11:28:53 -03:00
72374 9adb71bf8f feat: Change multiplier to 7/8 when scaling down avatars
The resolution-limits for avatar-images are currently 512x512 or 256x256.
Reducing the resolution further, to 2/3 each step, can reduce the quality
much more than is necessary to fit within the file-size-limits,
which are currently 60 kB or 20 kB.

An image made entirely of noise (which results in unusually
large file-sizes), encoded with jpeg-quality 75,
and 4:2:2-colour-subsampling (the format currently used
for encoding images), can be below 60 kB at 227x227.
For the lower file-size-limit of 20 kB,
such images can be too large at 170x170, but fit at 149x149.
More normal images will have a lower file-size at the same resolution.

Before this change, the target-resolutions for resampling were:
512x512 ->                       341x341 ->                       227x227.

And for the lower file-size-limit:
256x256 ->                       170x170 ->                       113x113.

After this change, the target-resolutions for resampling will be:
512x512 -> 448x448 -> 392x392 -> 343x343 -> 300x300 -> 262x262 -> 229x229.

And for the lower file-size-limit, those will be:
256x256 -> 224x224 -> 196x196 -> 171x171 -> 149x149 -> 130x130 -> 113x113.

This does add 2 steps between the previous target-resolutions,
while still reaching target-resolutions close to the previous ones,
to reduce situations in which the quality will be lower than before.
2026-04-15 13:57:41 +02:00
72374 fe2ba05804 feat: Increase the resolution-limit WORSE_AVATAR_SIZE from 128 to 256
The file-size of many images will already be smaller than 20 kB,
when encoded at 256x256, and it can be a large improvement in quality.
2026-04-15 13:57:41 +02:00
link2xt 072fc34c77 fix: trash no-op messages about self being added to groups 2026-04-15 07:02:15 +00:00
link2xt cb00bd7043 refactor: replace HashSet with BTreeSet
This is needed to avoid golden tests being flaky
because of info message order being different
between runs.
2026-04-15 07:02:15 +00:00
link2xt f766c11075 perf: enable clippy::large_futures lint
Large size of Mimefactor.render() futures
increases the size of all callers
down to set_config() and background_fetch().
I also had to Box::pin one call to fetch_new_msg_batch
and one call to fetch_single_msg.
2026-04-15 05:33:52 +00:00
link2xt 89e450894d refactor: make HTML parser non-async 2026-04-14 12:43:46 +00:00
DarkCat09 6ef1f7d52b fix: restart io on transport deletion
Fixes #8038
2026-04-14 13:11:28 +02:00
dependabot[bot] 33dc3d20ad chore(deps): bump taiki-e/install-action from 2.64.0 to 2.74.0
Bumps [taiki-e/install-action](https://github.com/taiki-e/install-action) from 2.64.0 to 2.74.0.
- [Release notes](https://github.com/taiki-e/install-action/releases)
- [Changelog](https://github.com/taiki-e/install-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/taiki-e/install-action/compare/69e777b377e4ec209ddad9426ae3e0c1008b0ef3...85b24a67ef0c632dfefad70b9d5ce8fddb040754)

---
updated-dependencies:
- dependency-name: taiki-e/install-action
  dependency-version: 2.74.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-04-14 03:36:39 +00:00
dependabot[bot] b8a0c37efe chore(cargo): bump rand from 0.9.2 to 0.9.3
Bumps [rand](https://github.com/rust-random/rand) from 0.9.2 to 0.9.3.
- [Release notes](https://github.com/rust-random/rand/releases)
- [Changelog](https://github.com/rust-random/rand/blob/0.9.3/CHANGELOG.md)
- [Commits](https://github.com/rust-random/rand/compare/rand_core-0.9.2...0.9.3)

---
updated-dependencies:
- dependency-name: rand
  dependency-version: 0.9.3
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-04-14 03:36:12 +00:00
iequidoo ad7f873c68 fix: Ensure that message being sent is added to the bottom (#8027)
Before, if the user fixed their clock incorrectly set to the future, they needed to delete
previously sent messages or wait until this future comes again so that new sent messages are added
to the bottom. Strictly speaking, the problem isn't fixable because we don't know if messages were
incorrectly timestamped into the future or they are timestamped correctly and the clock is now
incorrectly set to the past. Anyway, adding messages to the middle of the chat isn't a good way to
inform the user about the problem.
2026-04-13 08:49:58 -03:00
link2xt 3236c8bbf4 chore: bump version to 2.50.0-dev 2026-04-13 10:14:01 +02:00
link2xt dab7ca19fe chore(release): prepare for 2.49.0 2026-04-13 10:10:07 +02:00
DarkCat09 520cd0ede8 docs: fix broken link for i-d "Common PGP/MIME Message Mangling" 2026-04-12 00:32:51 +00:00
WofWca 5d5deedec3 refactor: less nested remove_contact_from_chat 2026-04-11 12:23:16 +04:00
link2xt f33e21ccb9 fix: trash message about group name change from non-member 2026-04-11 01:26:51 +00:00
link2xt 00c06c490b test: use TestContextManager in test_keep_member_list_if_possibly_nomember 2026-04-11 01:26:51 +00:00
Hocuri 8b58b16cb5 fix: For bots, wait with emitting IncomingMsg until the Post-Msg arrived (#8104)
I used some AI to draft a first version of this, and then reworked it.

This is one of multiple possibilities to fix
https://github.com/chatmail/core/issues/8041: For bots, the IncomingMsg
event is not emitted when the pre-message arrives, only when the
post-message arrives. Also, post-messages are downloaded immediately,
not after all the small messages are downloaded.

The `get_next_msgs()` API is deprecated. Instead, bots need to listen to
the IncomingMsg event in order to be notified about new events. Is this
acceptable for bots?

THE PROBLEM THAT WAS SOLVED BY THIS:

With pre-messages, it's hard for bots to wait for the message to be fully downloaded and then process it.

Up until now, bots used get_next_msgs() to query the unprocessed messages, then set last_msg_id after processing a message to that they won't process it again.

But this will now also return messages that were not fully downloaded.

ALTERNATIVES:

In the following, I will explain
the alternatives, and for why it's not so easy to just make the
`get_next_msgs()` API work. If it's not understandable, I'm happy to
elaborate more.

Core can't just completely ignore the pre-message for two reasons:
- If a post-message containing a Webxdc arrives later, and some webxdc updates arrive in the meantime, then these updates will be lost.
- The post-message doesn't contain the text (reasoning was to avoid duplicate text for people who didn't upgrade yet during the 2.43.0 rollout)

There are multiple solutions:
- Add the message as hidden in the database when the pre-message arrives.
  - When the post-message arrives and we want to make it available for bots, we need to update the msg_id because of how the `get_next_msgs()` API works. This means that we need to update all webxdc updates that reference this msg_id.
  - Alternatively, we could make webxdc's reference the rfc724_mid instead of the msg_id, so that we don't need stable msg_ids anymore
  - Alternatively, we could deprecate `get_next_msgs()`, and ask bots to use plain events for message processing again. It's not that bad; worst case, the bot crashes and then forgets to react to some messages, but the user will just try again. And if some message makes the bot crash, then it might actually be good not to try and process it again.
- Store the pre-message text and `PostMsgMetadata` (or alternatively, the whole mime) in a new database table. Wait with processing it until the post-message arrives.

Additionally, the logic that small messages are downloaded before post-messages should be disabled for bots, in order to prevent reordering.
2026-04-10 21:10:46 +02:00
link2xt d6971ee4ac fix: make start messages stick to the top of the chat
We already set sort_timestamp to 0 for "Messages are end-to-end encrypted."
since 8f1bf963b4.
Do this for "Others will only see this group after you sent a first message."
and "Messages in this chat use classic email and are not encrypted." as well
so no messages can be added on top.
2026-04-10 03:16:12 +00:00
DavidSM100 e3bf6bf352 refactor(@deltachat/stdio-rpc-server): remove await from README example 2026-04-09 14:58:55 +00:00
DavidSM100 4b81cd2fc8 api(@deltachat/stdio-rpc-server): also export a class
This is convenient for bots and libs for bots, so they can extend from this class directly
2026-04-09 14:58:55 +00:00
DavidSM100 be920bf3bf refactor(@deltachat/stdio-rpc-server): make getRPCServerPath and startDeltaChat synchronous 2026-04-09 14:58:55 +00:00
link2xt 602f0a088e ci: make sure -dev version suffix is not forgotten after release
Workflow checks that PRs are made only when current version ends with -dev
If this fails, a commit bumping the version to -dev should be pushed to main branch.
2026-04-09 14:34:23 +00:00
link2xt a2bb8962cb fix: add missing extern "C" to dc_array_is_independent
It was the only `unsafe fn` (not `unsafe extern "C" fn`) in lib.rs
2026-04-09 03:24:15 +00:00
link2xt 795fe9a38b chore: bump version to 2.49.0-dev 2026-04-08 22:27:29 +02:00
holger krekel 60bc4011f7 fix: let search also return hidden contacts if search value is an email address 2026-04-07 22:39:13 +02:00
link2xt f552cf93b4 fix: assign webxdc updates from post-message to webxdc instance 2026-04-07 19:14:44 +00:00
link2xt f75a7986b5 refactor: ignore ForcePlaintext in saved messages chat
ForcePlaintext was used for Autocrypt Setup Message,
there is no need to support it in saved messages chat anymore.
2026-04-07 17:00:47 +00:00
link2xt 3b8f1934f3 api!: remove dc_msg_force_plaintext
Message.force_plaintext() is still used in legacy SecureJoin steps
internally, so cannot be removed, but there is no need for public API.
2026-04-07 17:00:47 +00:00
dependabot[bot] c8716f50aa chore(deps): bump dependabot/fetch-metadata from 2.4.0 to 3.0.0
Bumps [dependabot/fetch-metadata](https://github.com/dependabot/fetch-metadata) from 2.4.0 to 3.0.0.
- [Release notes](https://github.com/dependabot/fetch-metadata/releases)
- [Commits](https://github.com/dependabot/fetch-metadata/compare/v2.4.0...v3.0.0)

---
updated-dependencies:
- dependency-name: dependabot/fetch-metadata
  dependency-version: 3.0.0
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-04-07 05:15:35 +00:00
link2xt d2097d3523 fix: do not URL-encode proxy hostnames 2026-04-06 13:16:28 +00:00
link2xt 1219cbe1a3 fix: do not create 1:1 chat on second device when scanning a QR code
This avoids creating 1:1 chat on a second device when joining a channel.
Now when joining a channel there may be no 1:1 chat with the inviter
when the channel is created. In this case we still create the channel
as unblocked even if 1:1 chat would be a contact request
because joining the channel is an explicit action
and it is not possible to add someone who did not scan a QR
to the channel manually.
2026-04-06 00:42:14 +00:00
iequidoo bc48b17e93 test: Fix flaky test_no_old_msg_is_fresh: Wait for incoming message before sending outgoing one
We don't want to send an outgoing message from the 2nd device (`ac1_clone`) before receiving the
incoming one and expect that the messages will be ordered correctly on the 1st device (`ac1`). Let's
ensure the correct message order locally in the first place. I checked logs of a failed test run and
it indeed happened that `ac1_clone` sent the message earlier, so it can't reference the incoming
message and `tweak_sort_timestamp()` does nothing on `ac1`, so the messages can't be ordered
correctly considering that smeared clocks on the devices are diferent.
2026-04-05 20:59:46 -03:00
Hocuri 7233b4b811 test: Test that messages are only marked as delivered after being fully sent out (#8077)
Test for https://github.com/chatmail/core/pull/8062. I checked that the
test fails without #8062.
2026-04-05 20:37:32 +00:00
iequidoo d1e0088201 feat: Flipped Exif orientations (#8057)
Before, sending of images flipped in Exif led to images having wrong orientation.
2026-04-05 17:04:17 -03:00
dependabot[bot] a5e41b0b49 chore(cargo): bump proptest from 1.10.0 to 1.11.0
Bumps [proptest](https://github.com/proptest-rs/proptest) from 1.10.0 to 1.11.0.
- [Release notes](https://github.com/proptest-rs/proptest/releases)
- [Changelog](https://github.com/proptest-rs/proptest/blob/main/CHANGELOG.md)
- [Commits](https://github.com/proptest-rs/proptest/compare/v1.10.0...v1.11.0)

---
updated-dependencies:
- dependency-name: proptest
  dependency-version: 1.11.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-04-05 17:03:06 -03:00
iequidoo 2f76fd98dd test: Add test for tweak_sort_timestamp()
The part of logic there adjusting the sort timestamp forward if the parent message has a greater
sort timestamp wasn't tested explicitly by any test. I only saw one unrelated "golden test" failure
when commented it out.
(Related to #8027)
2026-04-05 11:16:15 -03:00
dependabot[bot] 6235f2a01a chore(cargo): bump image from 0.25.9 to 0.25.10
Bumps [image](https://github.com/image-rs/image) from 0.25.9 to 0.25.10.
- [Changelog](https://github.com/image-rs/image/blob/v0.25.10/CHANGES.md)
- [Commits](https://github.com/image-rs/image/compare/v0.25.9...v0.25.10)

---
updated-dependencies:
- dependency-name: image
  dependency-version: 0.25.10
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-04-05 05:50:41 +00:00
dependabot[bot] ec5117a6c2 chore(cargo): bump quote from 1.0.44 to 1.0.45
Bumps [quote](https://github.com/dtolnay/quote) from 1.0.44 to 1.0.45.
- [Release notes](https://github.com/dtolnay/quote/releases)
- [Commits](https://github.com/dtolnay/quote/compare/1.0.44...1.0.45)

---
updated-dependencies:
- dependency-name: quote
  dependency-version: 1.0.45
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-04-02 15:45:56 +00:00
dependabot[bot] d6e3a8829b chore(cargo): bump libc from 0.2.182 to 0.2.183
Bumps [libc](https://github.com/rust-lang/libc) from 0.2.182 to 0.2.183.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Changelog](https://github.com/rust-lang/libc/blob/0.2.183/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.182...0.2.183)

---
updated-dependencies:
- dependency-name: libc
  dependency-version: 0.2.183
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-04-02 15:45:37 +00:00
dependabot[bot] 2340818488 chore(cargo): bump tokio from 1.49.0 to 1.50.0
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.49.0 to 1.50.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.49.0...tokio-1.50.0)

---
updated-dependencies:
- dependency-name: tokio
  dependency-version: 1.50.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-04-02 15:45:07 +00:00
dependabot[bot] f175d2fed9 chore(cargo): bump pin-project from 1.1.10 to 1.1.11
Bumps [pin-project](https://github.com/taiki-e/pin-project) from 1.1.10 to 1.1.11.
- [Release notes](https://github.com/taiki-e/pin-project/releases)
- [Changelog](https://github.com/taiki-e/pin-project/blob/main/CHANGELOG.md)
- [Commits](https://github.com/taiki-e/pin-project/compare/v1.1.10...v1.1.11)

---
updated-dependencies:
- dependency-name: pin-project
  dependency-version: 1.1.11
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-04-02 14:20:27 +00:00
dependabot[bot] d318bbb0f4 chore(cargo): bump tempfile from 3.26.0 to 3.27.0
Bumps [tempfile](https://github.com/Stebalien/tempfile) from 3.26.0 to 3.27.0.
- [Changelog](https://github.com/Stebalien/tempfile/blob/master/CHANGELOG.md)
- [Commits](https://github.com/Stebalien/tempfile/compare/v3.26.0...v3.27.0)

---
updated-dependencies:
- dependency-name: tempfile
  dependency-version: 3.27.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-04-02 14:15:16 +00:00
dependabot[bot] a0f14a5978 chore(cargo): bump tracing-subscriber from 0.3.22 to 0.3.23
Bumps [tracing-subscriber](https://github.com/tokio-rs/tracing) from 0.3.22 to 0.3.23.
- [Release notes](https://github.com/tokio-rs/tracing/releases)
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-subscriber-0.3.22...tracing-subscriber-0.3.23)

---
updated-dependencies:
- dependency-name: tracing-subscriber
  dependency-version: 0.3.23
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-04-02 14:14:51 +00:00
dependabot[bot] 7e49033f92 chore(cargo): bump chrono from 0.4.43 to 0.4.44
Bumps [chrono](https://github.com/chronotope/chrono) from 0.4.43 to 0.4.44.
- [Release notes](https://github.com/chronotope/chrono/releases)
- [Changelog](https://github.com/chronotope/chrono/blob/main/CHANGELOG.md)
- [Commits](https://github.com/chronotope/chrono/compare/v0.4.43...v0.4.44)

---
updated-dependencies:
- dependency-name: chrono
  dependency-version: 0.4.44
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-04-02 14:10:54 +00:00
Hocuri 626ac8161a fix: Mark a message as delivered only after it has been fully sent out (#8062)
Fix https://github.com/chatmail/core/issues/8042

The problem was that after receiving the bcc_self'ed pre-message in
`receive_imf`, the logic there only looked for a pending
`smtp`-table-entry that matches the rfc724_mid, and if there was none
then it thought "Great, apparently the message is fully sent out, we can
mark it as delivered!".

But with pre-messages, the same message can have two `smtp` entries (one
for the pre-message and one for the post-message), and the message
should only be marked as delivered once both of them are sent out.

Now, I changed the logic to look for all entries with the same msg_id.
This is actually the same SQL query used in smtp.rs, so, I extracted it
into a new function; feel free to suggest a better name for it.

I tested on Android that it now works fine.

I'll add a test in a follow-up PR.

There are a lot of other problems with sending large files, though:
- The pre-message is sent before the post-message, so that for the
receiver it looks as if the message arrived, but stays in
"downloading..." forever
- There is quite a time delay between clicking on "Send" and the
outgoing message appearing in the chat
- The message shortly gets a letter icon right after it is sent
- I'm wondering if there is a way to give feedback to the user
immediately if the message is too big
- It's unclear when exactly we want to send read receipts

I'll open a follow-up issue for these.
2026-04-02 15:12:17 +02:00
holger krekel 28cce5e31d fix: determine whether a message is an own message by looking at signature. multiple devices can temporarly have different sets of self addresses, and still need to properly recognize incoming versus outgoing messages. Disclaimer: some LLM tooling was initially involved but i went over everything by hand, and also addressed review comments. 2026-04-01 14:51:48 +02:00
link2xt 3b87e27f34 docs: document that events are broadcasted to all event emitters
Delta Chat for iOS already relies on this behavior,
so it cannot be practically changed.
2026-04-01 09:03:07 +00:00
link2xt 24b21c0588 chore(release): prepare for 2.48.0 2026-03-30 12:48:24 +02:00
link2xt eb666d4cc3 test: the message is sorted correctly in the chat even if it arrives late 2026-03-30 08:52:19 +00:00
link2xt ef265689dd fix: do not sort received messages below the last seen one 2026-03-30 08:52:19 +00:00
link2xt 49223792f9 fix: never sort the message before chat joining timestamp
This is to avoid sorting incoming messages that
are slightly in the past above system messages
about SecureJoin. SecureJoin messages are
timed according to smeared timestamp,
so even in the local tests they are in the future
by a few seconds.
2026-03-30 08:52:19 +00:00
Hocuri 920da083d1 fix: Manipulate sort_timestamp to not be 0 2026-03-30 08:52:19 +00:00
link2xt 8f1bf963b4 fix: always sort "Messages are end-to-end encrypted" notice to the beginning
We set timestamp of this info message to 0
to make it always appear in the beginning of the chat.
To avoid new chats being sorted to the end of the chatlist,
we ignore such 0 and use chat creation timestamp
when sorting the chatlist.
2026-03-30 08:52:19 +00:00
link2xt e33d50b4e0 test: use load_imf_email() more 2026-03-30 08:52:19 +00:00
link2xt f1dc03a4ee test: do not rely on loading newest chat in load_imf_email()
We know which message was added from the return value
of receive_imf(). It may be that the first chat
in the chatlist is not the one where the message was received
if there is a pinned chat or if
just received message is old.
2026-03-30 08:52:19 +00:00
link2xt 5d90cc7a2a test: remove test_old_message_5
It is not clear now what this is testing.
Golden test shows messages ordered
incorrectly according to the timestamps,
they should be ordered the other way round.

Comment talks about fetching from mvbox and inbox
in paralell which is a rare case that
could have happened if one message is left in the inbox
and the other message is a chat message moved to mvbox.
We never download anything that is not moved to the target folder.

The test also resides in "verified chats" tests
which are all legacy tests we kept after
replacing the concept of verified/protected chats
with key contacts in 2.x.
2026-03-30 08:52:19 +00:00
link2xt 68e630eb82 fix: remove migration 108
This removes migration added in 625887d249
2026-03-30 08:38:28 +00:00
iequidoo ef718bb869 fix: When receiving MDN, mark all preceding messages as noticed, even having same timestamp (#7928)
This fixes flaky JSON-RPC `test_multidevice_sync_seen`.
2026-03-29 11:50:50 -03:00
iequidoo f1860f90d4 feat: Log received message sort timestamp
This way it's easier to debug issues like `MsgsNoticed` not emitted for a chat.
2026-03-29 11:50:50 -03:00
link2xt a947f4296f refactor(securejoin): do not check for self address in forwarding protection
If our key is gossiped, the message is intended for us.
The check for address is redundant for incoming messages as
if we received the message then it was addressed to us.

This whole protection code can eventually be removed
as we have intended recipient fingerprints already,
it only protects against forwarding of messages
sent by old clients.
2026-03-28 16:20:39 +00:00
link2xt 8c3139f7a2 feat: add decryption error to the device message about outgoing message decryption failure 2026-03-28 13:27:15 +00:00
link2xt 3dd7defaa1 docs: add SQL performance tips to STYLE.md 2026-03-28 10:08:54 +00:00
link2xt 3096dd6027 ci: fix https://docs.zizmor.sh/audits/#bot-conditions 2026-03-28 09:29:12 +01:00
link2xt ee62d2d281 ci: use environment for js.jsonrpc.delta.chat deployment 2026-03-28 01:06:12 +01:00
link2xt 6095971f67 ci: use environment for cffi.delta.chat deployment 2026-03-27 16:37:58 +01:00
link2xt 32ff5b7a6b ci: use environment for rs.delta.chat deployment 2026-03-27 09:36:11 +00:00
link2xt b87805ab24 fix: cleanup imap and imap_sync records without transport in housekeeping
Previously transports deleted via sync messages left unused `imap` entries.
2026-03-26 16:24:11 +00:00
link2xt c8716ad85a fix: delete imap_markseen entries not corresponding to any imap rows 2026-03-26 16:24:11 +00:00
link2xt 4dd0ba2c72 fix: move sorting outside of SQL query in store_seen_flags_on_imap
With `ORDER BY` statement SQLite searches
the `imap` table by `transport_id` and for each found row
scans the whole `imap_markseen` table.
Number of `imap` entries for each `transport_id`
is usually large as we need to know
which UIDs to delete on IMAP server
when deleting a message.

```
sqlite> EXPLAIN QUERY PLAN
SELECT imap.id, uid, folder FROM imap, imap_markseen
WHERE imap.id = imap_markseen.id
AND imap.transport_id=?
AND target = folder
ORDER BY folder, uid;
QUERY PLAN
|--SEARCH imap USING INDEX sqlite_autoindex_imap_1 (transport_id=?)
`--SCAN imap_markseen
```

Without `ORDER BY` statement SQLite scans `imap_markseen`
table which is expected to be small,
and then searches `imap` table by `rowid` for each found result.

```
sqlite> EXPLAIN QUERY PLAN
SELECT imap.id, uid, folder FROM imap, imap_markseen
WHERE imap.id = imap_markseen.id
AND imap.transport_id=?
AND target = folder;
QUERY PLAN
|--SCAN imap_markseen
`--SEARCH imap USING INTEGER PRIMARY KEY (rowid=?)
```

Query planning was tested with SQLite 3.52.0.
It is possible to explictly make
query planner move sorting to the last step
with `ORDER +folder, +uid`, but this is not recommended
in SQLite documentation
(see <https://www.sqlite.org/optoverview.html#uplus>).

It is also possible to add indexes,
but indexes use space,
adding them requires an SQL migration,
and each index needs to be updated so it will slow down writes.
2026-03-26 16:24:11 +00:00
link2xt a24248a90b ci: update {c,py}.delta.chat website deployments
The host has been changed and the secrets are moved to environments.
2026-03-26 15:25:53 +00:00
iequidoo af16fc9038 fix: Make Message-ID of pre-messages stable across resends (#8007) 2026-03-25 23:32:33 -03:00
114 changed files with 2492 additions and 2191 deletions
+8 -8
View File
@@ -20,7 +20,7 @@ permissions: {}
env:
RUSTFLAGS: -Dwarnings
RUST_VERSION: 1.94.0
RUST_VERSION: 1.95.0
# Minimum Supported Rust Version
MSRV: 1.88.0
@@ -40,7 +40,7 @@ jobs:
- run: rustup override set $RUST_VERSION
shell: bash
- name: Cache rust cargo artifacts
uses: swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5
uses: swatinem/rust-cache@c19371144df3bb44fab255c43d04cbc2ab54d1c4
- name: Run rustfmt
run: cargo fmt --all -- --check
- name: Run clippy
@@ -59,7 +59,7 @@ jobs:
with:
show-progress: false
persist-credentials: false
- uses: EmbarkStudios/cargo-deny-action@3fd3802e88374d3fe9159b834c7714ec57d6c979
- uses: EmbarkStudios/cargo-deny-action@91bf2b620e09e18d6eb78b92e7861937469acedb
with:
arguments: --workspace --all-features --locked
command: check
@@ -91,7 +91,7 @@ jobs:
show-progress: false
persist-credentials: false
- name: Cache rust cargo artifacts
uses: swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5
uses: swatinem/rust-cache@c19371144df3bb44fab255c43d04cbc2ab54d1c4
- name: Rustdoc
run: cargo doc --document-private-items --no-deps
@@ -134,10 +134,10 @@ jobs:
shell: bash
- name: Cache rust cargo artifacts
uses: swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5
uses: swatinem/rust-cache@c19371144df3bb44fab255c43d04cbc2ab54d1c4
- name: Install nextest
uses: taiki-e/install-action@69e777b377e4ec209ddad9426ae3e0c1008b0ef3
uses: taiki-e/install-action@85b24a67ef0c632dfefad70b9d5ce8fddb040754
with:
tool: nextest
@@ -168,7 +168,7 @@ jobs:
persist-credentials: false
- name: Cache rust cargo artifacts
uses: swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5
uses: swatinem/rust-cache@c19371144df3bb44fab255c43d04cbc2ab54d1c4
- name: Build C library
run: cargo build -p deltachat_ffi
@@ -194,7 +194,7 @@ jobs:
persist-credentials: false
- name: Cache rust cargo artifacts
uses: swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5
uses: swatinem/rust-cache@c19371144df3bb44fab255c43d04cbc2ab54d1c4
- name: Build deltachat-rpc-server
run: cargo build -p deltachat-rpc-server
+2 -2
View File
@@ -10,11 +10,11 @@ permissions:
jobs:
dependabot:
runs-on: ubuntu-latest
if: ${{ github.actor == 'dependabot[bot]' }}
if: github.event.pull_request.user.login == 'dependabot[bot]' && github.repository == github.event.pull_request.head.repo.full_name
steps:
- name: Dependabot metadata
id: metadata
uses: dependabot/fetch-metadata@v2.4.0
uses: dependabot/fetch-metadata@v3.0.0
with:
github-token: "${{ secrets.GITHUB_TOKEN }}"
- name: Approve a PR
+23
View File
@@ -0,0 +1,23 @@
# Check that PRs are made against the -dev version.
#
# If this fails, push commit to update the version to -dev to main.
name: Check for -dev version
on:
pull_request:
permissions: {}
jobs:
check_dev_version:
name: Check that current version ends with -dev
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
with:
show-progress: false
persist-credentials: false
- name: Run version-checking script
run: scripts/check-dev-version.py
+1 -1
View File
@@ -25,7 +25,7 @@ jobs:
with:
node-version: 18.x
- name: Add Rust cache
uses: Swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5
uses: Swatinem/rust-cache@c19371144df3bb44fab255c43d04cbc2ab54d1c4
- name: npm install
working-directory: deltachat-jsonrpc/typescript
run: npm install
+43 -10
View File
@@ -1,16 +1,18 @@
name: Build & deploy documentation on rs.delta.chat, c.delta.chat, and py.delta.chat
name: Build & deploy documentation on rs.delta.chat, c.delta.chat, py.delta.chat and cffi.delta.chat
on:
push:
branches:
- main
- build_jsonrpc_docs_ci
permissions: {}
jobs:
build-rs:
runs-on: ubuntu-latest
environment:
name: rs.delta.chat
url: https://rs.delta.chat/
steps:
- uses: actions/checkout@v6
@@ -23,12 +25,15 @@ jobs:
- name: Upload to rs.delta.chat
run: |
mkdir -p "$HOME/.ssh"
echo "${{ secrets.KEY }}" > "$HOME/.ssh/key"
echo "${{ secrets.RS_DOCS_SSH_KEY }}" > "$HOME/.ssh/key"
chmod 600 "$HOME/.ssh/key"
rsync -avzh -e "ssh -i $HOME/.ssh/key -o StrictHostKeyChecking=no" $GITHUB_WORKSPACE/target/doc "${{ secrets.USERNAME }}@rs.delta.chat:/var/www/html/rs/"
rsync -avzh -e "ssh -i $HOME/.ssh/key -o StrictHostKeyChecking=no" $GITHUB_WORKSPACE/target/doc "${{ secrets.RS_DOCS_SSH_USER }}@rs.delta.chat:/var/www/html/rs.delta.chat/"
build-python:
runs-on: ubuntu-latest
environment:
name: py.delta.chat
url: https://py.delta.chat/
steps:
- uses: actions/checkout@v6
@@ -42,12 +47,15 @@ jobs:
- name: Upload to py.delta.chat
run: |
mkdir -p "$HOME/.ssh"
echo "${{ secrets.CODESPEAK_KEY }}" > "$HOME/.ssh/key"
echo "${{ secrets.PY_DOCS_SSH_KEY }}" > "$HOME/.ssh/key"
chmod 600 "$HOME/.ssh/key"
rsync -avzh -e "ssh -i $HOME/.ssh/key -o StrictHostKeyChecking=no" $GITHUB_WORKSPACE/result/html/ "delta@py.delta.chat:/home/delta/build/master"
rsync -avzh --delete -e "ssh -i $HOME/.ssh/key -o StrictHostKeyChecking=no" $GITHUB_WORKSPACE/result/html/ "${{ secrets.PY_DOCS_SSH_USER }}@py.delta.chat:/var/www/html/py.delta.chat"
build-c:
runs-on: ubuntu-latest
environment:
name: c.delta.chat
url: https://c.delta.chat/
steps:
- uses: actions/checkout@v6
@@ -61,12 +69,16 @@ jobs:
- name: Upload to c.delta.chat
run: |
mkdir -p "$HOME/.ssh"
echo "${{ secrets.CODESPEAK_KEY }}" > "$HOME/.ssh/key"
echo "${{ secrets.C_DOCS_SSH_KEY }}" > "$HOME/.ssh/key"
chmod 600 "$HOME/.ssh/key"
rsync -avzh -e "ssh -i $HOME/.ssh/key -o StrictHostKeyChecking=no" $GITHUB_WORKSPACE/result/html/ "delta@c.delta.chat:/home/delta/build-c/master"
rsync -avzh --delete -e "ssh -i $HOME/.ssh/key -o StrictHostKeyChecking=no" $GITHUB_WORKSPACE/result/html/ "${{ secrets.C_DOCS_SSH_USER }}@c.delta.chat:/var/www/html/c.delta.chat"
build-ts:
runs-on: ubuntu-latest
environment:
name: js.jsonrpc.delta.chat
url: https://js.jsonrpc.delta.chat/
defaults:
run:
working-directory: ./deltachat-jsonrpc/typescript
@@ -90,6 +102,27 @@ jobs:
- name: Upload to js.jsonrpc.delta.chat
run: |
mkdir -p "$HOME/.ssh"
echo "${{ secrets.KEY }}" > "$HOME/.ssh/key"
echo "${{ secrets.JS_JSONRPC_DOCS_SSH_KEY }}" > "$HOME/.ssh/key"
chmod 600 "$HOME/.ssh/key"
rsync -avzh -e "ssh -i $HOME/.ssh/key -o StrictHostKeyChecking=no" $GITHUB_WORKSPACE/deltachat-jsonrpc/typescript/docs/ "${{ secrets.USERNAME }}@js.jsonrpc.delta.chat:/var/www/html/js-jsonrpc/"
rsync -avzh --delete -e "ssh -i $HOME/.ssh/key -o StrictHostKeyChecking=no" $GITHUB_WORKSPACE/deltachat-jsonrpc/typescript/docs/ "${{ secrets.JS_JSONRPC_DOCS_SSH_USER }}@js.jsonrpc.delta.chat:/var/www/html/js.jsonrpc.delta.chat/"
build-cffi:
runs-on: ubuntu-latest
environment:
name: cffi.delta.chat
url: https://cffi.delta.chat/
steps:
- uses: actions/checkout@v6
with:
show-progress: false
persist-credentials: false
- name: Build the documentation with cargo
run: |
cargo doc --package deltachat_ffi --no-deps
- name: Upload to cffi.delta.chat
run: |
mkdir -p "$HOME/.ssh"
echo "${{ secrets.CFFI_DOCS_SSH_KEY }}" > "$HOME/.ssh/key"
chmod 600 "$HOME/.ssh/key"
rsync -avzh --delete -e "ssh -i $HOME/.ssh/key -o StrictHostKeyChecking=no" $GITHUB_WORKSPACE/target/doc/ "${{ secrets.CFFI_DOCS_SSH_USER }}@delta.chat:/var/www/html/cffi.delta.chat/"
-31
View File
@@ -1,31 +0,0 @@
# GitHub Actions workflow
# to build `deltachat_ffi` crate documentation
# and upload it to <https://cffi.delta.chat/>
name: Build & Deploy Documentation on cffi.delta.chat
on:
push:
branches:
- main
permissions: {}
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
with:
show-progress: false
persist-credentials: false
- name: Build the documentation with cargo
run: |
cargo doc --package deltachat_ffi --no-deps
- name: Upload to cffi.delta.chat
run: |
mkdir -p "$HOME/.ssh"
echo "${{ secrets.KEY }}" > "$HOME/.ssh/key"
chmod 600 "$HOME/.ssh/key"
rsync -avzh -e "ssh -i $HOME/.ssh/key -o StrictHostKeyChecking=no" $GITHUB_WORKSPACE/target/doc/ "${{ secrets.USERNAME }}@delta.chat:/var/www/html/cffi/"
+119
View File
@@ -1,5 +1,122 @@
# Changelog
## [2.49.0] - 2026-04-13
### Features / Changes
- Flipped Exif orientations ([#8057](https://github.com/chatmail/core/pull/8057)).
### Fixes
- Determine whether a message is an own message by looking at signature. multiple devices can temporarly have different sets of self addresses, and still need to properly recognize incoming versus outgoing messages. Disclaimer: some LLM tooling was initially involved but i went over everything by hand, and also addressed review comments..
- Mark a message as delivered only after it has been fully sent out ([#8062](https://github.com/chatmail/core/pull/8062)).
- Do not create 1:1 chat on second device when scanning a QR code.
- Do not URL-encode proxy hostnames.
- Assign webxdc updates from post-message to webxdc instance.
- Let search also return hidden contacts if search value is an email address.
- Add missing `extern "C"` to `dc_array_is_independent`.
- Make start messages stick to the top of the chat.
- For bots, wait with emitting IncomingMsg until the Post-Msg arrived ([#8104](https://github.com/chatmail/core/pull/8104)).
- Trash message about group name change from non-member.
### API-Changes
- [**breaking**] remove `dc_msg_force_plaintext`.
- @deltachat/stdio-rpc-server: also export a class.
### CI
- Make sure `-dev` version suffix is not forgotten after release.
### Documentation
- Document that events are broadcasted to all event emitters.
- Fix broken link for i-d "Common PGP/MIME Message Mangling".
### Refactor
- ignore ForcePlaintext in saved messages chat.
- @deltachat/stdio-rpc-server: make `getRPCServerPath` and `startDeltaChat` synchronous.
- @deltachat/stdio-rpc-server: remove `await` from README example.
- less nested `remove_contact_from_chat`.
### Tests
- Add test for `tweak_sort_timestamp()`.
- Test that messages are only marked as delivered after being fully sent out ([#8077](https://github.com/chatmail/core/pull/8077)).
- Fix flaky `test_no_old_msg_is_fresh`: Wait for incoming message before sending outgoing one.
- Use TestContextManager in `test_keep_member_list_if_possibly_nomember`.
### Miscellaneous Tasks
- cargo: bump chrono from 0.4.43 to 0.4.44.
- cargo: bump tracing-subscriber from 0.3.22 to 0.3.23.
- cargo: bump tempfile from 3.26.0 to 3.27.0.
- cargo: bump pin-project from 1.1.10 to 1.1.11.
- cargo: bump tokio from 1.49.0 to 1.50.0.
- cargo: bump libc from 0.2.182 to 0.2.183.
- cargo: bump quote from 1.0.44 to 1.0.45.
- cargo: bump image from 0.25.9 to 0.25.10.
- cargo: bump proptest from 1.10.0 to 1.11.0.
- deps: bump dependabot/fetch-metadata from 2.4.0 to 3.0.0.
- bump version to 2.49.0-dev.
## [2.48.0] - 2026-03-30
### Fixes
- Fix reordering problems in multi-relay setups by not sorting received messages below the last seen one.
- Always sort "Messages are end-to-end encrypted" notice to the beginning.
- Make Message-ID of pre-messages stable across resends ([#8007](https://github.com/chatmail/core/pull/8007)).
- Delete `imap_markseen` entries not corresponding to any `imap` rows.
- Cleanup `imap` and `imap_sync` records without transport in housekeeping.
- When receiving MDN, mark all preceding messages as noticed, even having same timestamp ([#7928](https://github.com/chatmail/core/pull/7928)).
- Remove migration 108 preventing upgrades from core 1.86.0 to the latest version.
### Features / Changes
- Improve IMAP loop logs.
- Add decryption error to the device message about outgoing message decryption failure.
- Log received message sort timestamp.
### Performance
- Move sorting outside of SQL query in `store_seen_flags_on_imap`.
### API-Changes
- Add JSON-RPC API `markfresh_chat()`.
- ffi: Correctly declare `dc_event_channel_new()` as having no params ([#7831](https://github.com/chatmail/core/pull/7831)).
### Refactor
- Remove `wal_checkpoint_mutex`, lock `write_mutex` before getting sql connection instead.
- Replace async `RwLock` with sync `RwLock` for stock strings.
- Cleanup remaining Autocrypt Setup Message processing in `mimeparser`.
- SecureJoin: do not check for self address in forwarding protection.
- Fix clippy warnings.
### CI
- Update {c,py}.delta.chat website deployments.
- Use environments for {rs,cffi,js.jsonrpc}.delta.chat deployments.
- Fix https://docs.zizmor.sh/audits/#bot-conditions.
### Documentation
- Add SQL performance tips to STYLE.md.
### Tests
- Remove `test_old_message_5`.
- Do not rely on loading newest chat in `load_imf_email()`.
- Use `load_imf_email()` more.
- The message is sorted correctly in the chat even if it arrives late.
### Miscellaneous Tasks
- cargo: update rustls-webpki to 0.103.10.
## [2.47.0] - 2026-03-24
### Fixes
@@ -7984,3 +8101,5 @@ https://github.com/chatmail/core/pulls?q=is%3Apr+is%3Aclosed
[2.45.0]: https://github.com/chatmail/core/compare/v2.44.0..v2.45.0
[2.46.0]: https://github.com/chatmail/core/compare/v2.45.0..v2.46.0
[2.47.0]: https://github.com/chatmail/core/compare/v2.46.0..v2.47.0
[2.48.0]: https://github.com/chatmail/core/compare/v2.47.0..v2.48.0
[2.49.0]: https://github.com/chatmail/core/compare/v2.48.0..v2.49.0
Generated
+46 -46
View File
@@ -550,7 +550,7 @@ dependencies = [
"bolero-kani",
"bolero-libfuzzer",
"cfg-if",
"rand 0.9.2",
"rand 0.9.4",
]
[[package]]
@@ -573,7 +573,7 @@ dependencies = [
"bolero-generator",
"lazy_static",
"pretty-hex",
"rand 0.9.2",
"rand 0.9.4",
"rand_xoshiro",
]
@@ -827,9 +827,9 @@ dependencies = [
[[package]]
name = "chrono"
version = "0.4.43"
version = "0.4.44"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fac4744fb15ae8337dc853fee7fb3f4e48c0fbaa23d0afe49c447b4fab126118"
checksum = "c673075a2e0e5f4a1dde27ce9dee1ea4558c7ffe648f576438a20ca1d2acc4b0"
dependencies = [
"iana-time-zone",
"num-traits",
@@ -1307,7 +1307,7 @@ dependencies = [
[[package]]
name = "deltachat"
version = "2.48.0-dev"
version = "2.50.0-dev"
dependencies = [
"anyhow",
"astral-tokio-tar",
@@ -1361,7 +1361,7 @@ dependencies = [
"qrcodegen",
"quick-xml",
"rand 0.8.5",
"rand 0.9.2",
"rand 0.9.4",
"ratelimit",
"regex",
"rusqlite",
@@ -1416,7 +1416,7 @@ dependencies = [
[[package]]
name = "deltachat-jsonrpc"
version = "2.48.0-dev"
version = "2.50.0-dev"
dependencies = [
"anyhow",
"async-channel 2.5.0",
@@ -1437,7 +1437,7 @@ dependencies = [
[[package]]
name = "deltachat-repl"
version = "2.48.0-dev"
version = "2.50.0-dev"
dependencies = [
"anyhow",
"deltachat",
@@ -1453,7 +1453,7 @@ dependencies = [
[[package]]
name = "deltachat-rpc-server"
version = "2.48.0-dev"
version = "2.50.0-dev"
dependencies = [
"anyhow",
"deltachat",
@@ -1482,7 +1482,7 @@ dependencies = [
[[package]]
name = "deltachat_ffi"
version = "2.48.0-dev"
version = "2.50.0-dev"
dependencies = [
"anyhow",
"deltachat",
@@ -1490,7 +1490,7 @@ dependencies = [
"human-panic",
"libc",
"num-traits",
"rand 0.9.2",
"rand 0.9.4",
"serde_json",
"thiserror 2.0.18",
"tokio",
@@ -2429,7 +2429,7 @@ dependencies = [
"idna",
"ipnet",
"once_cell",
"rand 0.9.2",
"rand 0.9.4",
"ring",
"thiserror 2.0.18",
"tinyvec",
@@ -2451,7 +2451,7 @@ dependencies = [
"moka",
"once_cell",
"parking_lot",
"rand 0.9.2",
"rand 0.9.4",
"resolv-conf",
"smallvec",
"thiserror 2.0.18",
@@ -2852,7 +2852,7 @@ dependencies = [
"hyper",
"hyper-util",
"log",
"rand 0.9.2",
"rand 0.9.4",
"tokio",
"url",
"xmltree",
@@ -2860,9 +2860,9 @@ dependencies = [
[[package]]
name = "image"
version = "0.25.9"
version = "0.25.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e6506c6c10786659413faa717ceebcb8f70731c0a60cbae39795fdf114519c1a"
checksum = "85ab80394333c02fe689eaf900ab500fbd0c2213da414687ebf995a65d5a6104"
dependencies = [
"bytemuck",
"byteorder-lite",
@@ -3260,9 +3260,9 @@ dependencies = [
[[package]]
name = "libc"
version = "0.2.182"
version = "0.2.184"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6800badb6cb2082ffd7b6a67e6125bb39f18782f793520caee8cb8846be06112"
checksum = "48f5d2a454e16a5ea0f4ced81bd44e4cfc7bd3a507b61887c99fd3538b28e4af"
[[package]]
name = "libm"
@@ -3483,9 +3483,9 @@ dependencies = [
[[package]]
name = "moxcms"
version = "0.7.5"
version = "0.8.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ddd32fa8935aeadb8a8a6b6b351e40225570a37c43de67690383d87ef170cd08"
checksum = "bb85c154ba489f01b25c0d36ae69a87e4a1c73a72631fc6c0eb6dde34a73e44b"
dependencies = [
"num-traits",
"pxfm",
@@ -4235,18 +4235,18 @@ dependencies = [
[[package]]
name = "pin-project"
version = "1.1.10"
version = "1.1.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "677f1add503faace112b9f1373e43e9e054bfdd22ff1a63c1bc485eaec6a6a8a"
checksum = "f1749c7ed4bcaf4c3d0a3efc28538844fb29bcdd7d2b67b2be7e20ba861ff517"
dependencies = [
"pin-project-internal",
]
[[package]]
name = "pin-project-internal"
version = "1.1.10"
version = "1.1.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6e918e4ff8c4549eb882f14b3a4bc8c8bc93de829416eacf579f1207a8fbf861"
checksum = "d9b20ed30f105399776b9c883e68e536ef602a16ae6f596d2c473591d6ad64c6"
dependencies = [
"proc-macro2",
"quote",
@@ -4615,13 +4615,13 @@ dependencies = [
[[package]]
name = "proptest"
version = "1.10.0"
version = "1.11.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "37566cb3fdacef14c0737f9546df7cfeadbfbc9fef10991038bf5015d0c80532"
checksum = "4b45fcc2344c680f5025fe57779faef368840d0bd1f42f216291f0dc4ace4744"
dependencies = [
"bitflags 2.11.0",
"num-traits",
"rand 0.9.2",
"rand 0.9.4",
"rand_chacha 0.9.0",
"rand_xorshift",
"regex-syntax",
@@ -4701,7 +4701,7 @@ dependencies = [
"bytes",
"getrandom 0.3.3",
"lru-slab",
"rand 0.9.2",
"rand 0.9.4",
"ring",
"rustc-hash",
"rustls",
@@ -4729,9 +4729,9 @@ dependencies = [
[[package]]
name = "quote"
version = "1.0.44"
version = "1.0.45"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "21b2ebcf727b7760c461f091f9f0f539b77b8e87f2fd88131e7f1b433b3cece4"
checksum = "41f2619966050689382d2b44f664f4bc593e129785a36d6ee376ddf37259b924"
dependencies = [
"proc-macro2",
]
@@ -4787,9 +4787,9 @@ dependencies = [
[[package]]
name = "rand"
version = "0.9.2"
version = "0.9.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6db2770f06117d490610c7488547d543617b21bfa07796d7a12f6f1bd53850d1"
checksum = "44c5af06bb1b7d3216d91932aed5265164bf384dc89cd6ba05cf59a35f5f76ea"
dependencies = [
"rand_chacha 0.9.0",
"rand_core 0.9.3",
@@ -5164,7 +5164,7 @@ dependencies = [
"once_cell",
"ring",
"rustls-pki-types",
"rustls-webpki 0.103.10",
"rustls-webpki 0.103.12",
"subtle",
"zeroize",
]
@@ -5201,9 +5201,9 @@ dependencies = [
[[package]]
name = "rustls-webpki"
version = "0.103.10"
version = "0.103.12"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "df33b2b81ac578cabaf06b89b0631153a3f416b0a886e8a7a1707fb51abbd1ef"
checksum = "8279bb85272c9f10811ae6a6c547ff594d6a7f3c6c6b02ee9726d1d0dcfcdd06"
dependencies = [
"ring",
"rustls-pki-types",
@@ -5322,7 +5322,7 @@ version = "0.17.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "22c3b0257608d7de4de4c4ea650ccc2e6e3e45e3cd80039fcdee768bcb449253"
dependencies = [
"rand 0.9.2",
"rand 0.9.4",
"substring",
"thiserror 1.0.69",
"url",
@@ -5581,7 +5581,7 @@ dependencies = [
"once_cell",
"percent-encoding",
"pin-project",
"rand 0.9.2",
"rand 0.9.4",
"sendfd",
"serde",
"serde_json",
@@ -5611,7 +5611,7 @@ dependencies = [
"chacha20poly1305",
"hkdf",
"md-5",
"rand 0.9.2",
"rand 0.9.4",
"ring-compat",
"sha1",
]
@@ -5843,7 +5843,7 @@ dependencies = [
"precis-core",
"precis-profiles",
"quoted-string-parser",
"rand 0.9.2",
"rand 0.9.4",
]
[[package]]
@@ -5988,9 +5988,9 @@ checksum = "55937e1799185b12863d447f42597ed69d9928686b8d88a1df17376a097d8369"
[[package]]
name = "tempfile"
version = "3.26.0"
version = "3.27.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "82a72c767771b47409d2345987fda8628641887d5466101319899796367354a0"
checksum = "32497e9a4c7b38532efcdebeef879707aa9f794296a4f0244f6f69e9bc8574bd"
dependencies = [
"fastrand",
"getrandom 0.3.3",
@@ -6144,9 +6144,9 @@ checksum = "1f3ccbac311fea05f86f61904b462b55fb3df8837a366dfc601a0161d0532f20"
[[package]]
name = "tokio"
version = "1.49.0"
version = "1.50.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "72a2903cd7736441aac9df9d7688bd0ce48edccaadf181c3b90be801e81d3d86"
checksum = "27ad5e34374e03cfffefc301becb44e9dc3c17584f414349ebe29ed26661822d"
dependencies = [
"bytes",
"libc",
@@ -6247,7 +6247,7 @@ dependencies = [
"getrandom 0.3.3",
"http 1.1.0",
"httparse",
"rand 0.9.2",
"rand 0.9.4",
"ring",
"rustls-pki-types",
"simdutf8",
@@ -6403,9 +6403,9 @@ dependencies = [
[[package]]
name = "tracing-subscriber"
version = "0.3.22"
version = "0.3.23"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2f30143827ddab0d256fd843b7a66d164e9f271cfa0dde49142c5ca0ca291f1e"
checksum = "cb7f578e5945fb242538965c2d0b04418d38ec25c79d160cd279bf0731c8d319"
dependencies = [
"matchers",
"nu-ansi-term",
+3 -3
View File
@@ -1,6 +1,6 @@
[package]
name = "deltachat"
version = "2.48.0-dev"
version = "2.50.0-dev"
edition = "2024"
license = "MPL-2.0"
rust-version = "1.88"
@@ -181,7 +181,7 @@ harness = false
anyhow = "1"
async-channel = "2.5.0"
base64 = "0.22"
chrono = { version = "0.4.43", default-features = false }
chrono = { version = "0.4.44", default-features = false }
deltachat-contact-tools = { path = "deltachat-contact-tools" }
deltachat-jsonrpc = { path = "deltachat-jsonrpc", default-features = false }
deltachat = { path = ".", default-features = false }
@@ -198,7 +198,7 @@ rusqlite = "0.37"
sanitize-filename = "0.6"
serde = "1.0"
serde_json = "1"
tempfile = "3.25.0"
tempfile = "3.27.0"
thiserror = "2"
tokio = "1"
tokio-util = "0.7.18"
+6
View File
@@ -68,6 +68,12 @@ keyword doesn't help here.
Consider adding context to `anyhow` errors for SQL statements using `.context()` so that it's
possible to understand from logs which statement failed. See [Errors](#errors) for more info.
When changing complex SQL queries, test them on a new database with `EXPLAIN QUERY PLAN`
to make sure that indexes are used and large tables are not going to be scanned.
Never run `ANALYZE` on the databases,
this makes query planner unpredictable
and may make performance significantly worse: <https://github.com/chatmail/core/issues/6585>
## Errors
Delta Chat core mostly uses [`anyhow`](https://docs.rs/anyhow/) errors.
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "deltachat_ffi"
version = "2.48.0-dev"
version = "2.50.0-dev"
description = "Deltachat FFI"
edition = "2018"
readme = "README.md"
+15 -45
View File
@@ -364,18 +364,14 @@ uint32_t dc_get_id (dc_context_t* context);
* To get these events, you have to create an event emitter using this function
* and call dc_get_next_event() on the emitter.
*
* Events are broadcasted to all existing event emitters.
* Events emitted before creation of event emitter
* are not available to event emitter.
*
* @memberof dc_context_t
* @param context The context object as created by dc_context_new().
* @return Returns the event emitter, NULL on errors.
* Must be freed using dc_event_emitter_unref() after usage.
*
* Note: Use only one event emitter per context.
* The result of having multiple event emitters is unspecified.
* Currently events are broadcasted to all existing event emitters,
* but previous versions delivered events to only one event emitter
* and this behavior may change again in the future.
* Events emitted before creation of event emitter
* may or may not be available to event emitter.
*/
dc_event_emitter_t* dc_get_event_emitter(dc_context_t* context);
@@ -430,14 +426,6 @@ char* dc_get_blobdir (const dc_context_t* context);
* 1=send a copy of outgoing messages to self (default).
* Sending messages to self is needed for a proper multi-account setup,
* however, on the other hand, may lead to unwanted notifications in non-delta clients.
* - `mvbox_move` = 1=detect chat messages,
* move them to the `DeltaChat` folder,
* and watch the `DeltaChat` folder for updates (default),
* 0=do not move chat-messages
* - `only_fetch_mvbox` = 1=Do not fetch messages from folders other than the
* `DeltaChat` folder. Messages will still be fetched from the
* spam folder.
* 0=watch all folders normally (default)
* - `show_emails` = DC_SHOW_EMAILS_OFF (0)=
* show direct replies to chats only,
* DC_SHOW_EMAILS_ACCEPTED_CONTACTS (1)=
@@ -3323,18 +3311,14 @@ void dc_accounts_set_push_device_token (dc_accounts_t* accounts, const
* This is similar to dc_get_event_emitter(), which, however,
* must not be called for accounts handled by the account manager.
*
* Events are broadcasted to all existing event emitters.
* Events emitted before creation of event emitter
* are not available to event emitter.
*
* @memberof dc_accounts_t
* @param accounts The account manager as created by dc_accounts_new().
* @return Returns the event emitter, NULL on errors.
* Must be freed using dc_event_emitter_unref() after usage.
*
* Note: Use only one event emitter per account manager.
* The result of having multiple event emitters is unspecified.
* Currently events are broadcasted to all existing event emitters,
* but previous versions delivered events to only one event emitter
* and this behavior may change again in the future.
* Events emitted before creation of event emitter
* are not available to event emitter.
*/
dc_event_emitter_t* dc_accounts_get_event_emitter (dc_accounts_t* accounts);
@@ -4250,6 +4234,10 @@ char* dc_msg_get_webxdc_blob (const dc_msg_t* msg, const char*
* true if the Webxdc should get internet access;
* this is the case i.e. for experimental maps integration.
* - self_addr: address to be used for `window.webxdc.selfAddr` in JS land.
* - app_sender_addr: address of the peer who initially shared the webxdc in the chat.
* Can be compared to self_addr to determine whether the app runs for the sender or a receiver.
* - can_only_send_updates_to_app_sender: true if updates sent by the local user
* will only be seen by the app sender.
* - send_update_interval: Milliseconds to wait before calling `sendUpdate()` again since the last call.
* Should be exposed to `webxdc.sendUpdateInterval` in JS land.
* - send_update_max_size: Maximum number of bytes accepted for a serialized update object.
@@ -4981,17 +4969,6 @@ uint32_t dc_msg_get_original_msg_id (const dc_msg_t* msg);
*/
uint32_t dc_msg_get_saved_msg_id (const dc_msg_t* msg);
/**
* Force the message to be sent in plain text.
*
* This API is for bots, there is no need to expose it in the UI.
*
* @memberof dc_msg_t
* @param msg The message object.
*/
void dc_msg_force_plaintext (dc_msg_t* msg);
/**
* @class dc_contact_t
*
@@ -5979,21 +5956,14 @@ void dc_event_channel_unref(dc_event_channel_t* event_channel);
* To get these events, you have to create an event emitter using this function
* and call dc_get_next_event() on the emitter.
*
* This is similar to dc_get_event_emitter(), which, however,
* must not be called for accounts handled by the account manager.
* Events are broadcasted to all existing event emitters.
* Events emitted before creation of event emitter
* are not available to event emitter.
*
* @memberof dc_event_channel_t
* @param The event channel.
* @return Returns the event emitter, NULL on errors.
* Must be freed using dc_event_emitter_unref() after usage.
*
* Note: Use only one event emitter per account manager / event channel.
* The result of having multiple event emitters is unspecified.
* Currently events are broadcasted to all existing event emitters,
* but previous versions delivered events to only one event emitter
* and this behavior may change again in the future.
* Events emitted before creation of event emitter
* are not available to event emitter.
*/
dc_event_emitter_t* dc_event_channel_get_event_emitter(dc_event_channel_t* event_channel);
+1 -11
View File
@@ -2826,7 +2826,7 @@ pub unsafe extern "C" fn dc_array_search_id(
// Returns 1 if location belongs to the track of the user,
// 0 if location was reported independently.
#[no_mangle]
pub unsafe fn dc_array_is_independent(
pub unsafe extern "C" fn dc_array_is_independent(
array: *const dc_array_t,
index: libc::size_t,
) -> libc::c_int {
@@ -4054,16 +4054,6 @@ pub unsafe extern "C" fn dc_msg_get_saved_msg_id(msg: *const dc_msg_t) -> u32 {
})
}
#[no_mangle]
pub unsafe extern "C" fn dc_msg_force_plaintext(msg: *mut dc_msg_t) {
if msg.is_null() {
eprintln!("ignoring careless call to dc_msg_force_plaintext()");
return;
}
let ffi_msg = &mut *msg;
ffi_msg.message.force_plaintext();
}
// dc_contact_t
/// FFI struct for [dc_contact_t]
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "deltachat-jsonrpc"
version = "2.48.0-dev"
version = "2.50.0-dev"
description = "DeltaChat JSON-RPC API"
edition = "2021"
license = "MPL-2.0"
+21 -7
View File
@@ -678,7 +678,7 @@ impl CommandApi {
ChatId::new(chat_id).get_fresh_msg_cnt(&ctx).await
}
/// Gets messages to be processed by the bot and returns their IDs.
/// (deprecated) Gets messages to be processed by the bot and returns their IDs.
///
/// Only messages with database ID higher than `last_msg_id` config value
/// are returned. After processing the messages, the bot should
@@ -686,6 +686,13 @@ impl CommandApi {
/// or manually updating the value to avoid getting already
/// processed messages.
///
/// Deprecated 2026-04: This returns the message's id as soon as the first part arrives,
/// even if it is not fully downloaded yet.
/// The bot needs to wait for the message to be fully downloaded.
/// Since this is usually not the desired behavior,
/// bots should instead use the #DC_EVENT_INCOMING_MSG / [`types::events::EventType::IncomingMsg`]
/// event for getting notified about new messages.
///
/// [`markseen_msgs`]: Self::markseen_msgs
async fn get_next_msgs(&self, account_id: u32) -> Result<Vec<u32>> {
let ctx = self.get_context(account_id).await?;
@@ -698,7 +705,7 @@ impl CommandApi {
Ok(msg_ids)
}
/// Waits for messages to be processed by the bot and returns their IDs.
/// (deprecated) Waits for messages to be processed by the bot and returns their IDs.
///
/// This function is similar to [`get_next_msgs`],
/// but waits for internal new message notification before returning.
@@ -709,6 +716,13 @@ impl CommandApi {
/// To shutdown the bot, stopping I/O can be used to interrupt
/// pending or next `wait_next_msgs` call.
///
/// Deprecated 2026-04: This returns the message's id as soon as the first part arrives,
/// even if it is not fully downloaded yet.
/// The bot needs to wait for the message to be fully downloaded.
/// Since this is usually not the desired behavior,
/// bots should instead use the #DC_EVENT_INCOMING_MSG / [`types::events::EventType::IncomingMsg`]
/// event for getting notified about new messages.
///
/// [`get_next_msgs`]: Self::get_next_msgs
async fn wait_next_msgs(&self, account_id: u32) -> Result<Vec<u32>> {
let ctx = self.get_context(account_id).await?;
@@ -2363,12 +2377,12 @@ impl CommandApi {
Ok(message_id.to_u32())
}
/// Send a reaction to message.
/// Sends a reaction to message.
///
/// Reaction is a string of emojis separated by spaces. Reaction to a
/// single message can be sent multiple times. The last reaction
/// received overrides all previously received reactions. It is
/// possible to remove all reactions by sending an empty string.
/// A reaction is a string that represents an emoji.
/// You can call this function again to change the emoji;
/// the last sent reaction overrides all previously sent reactions.
/// It is possible to remove the reaction by sending an empty string.
async fn send_reaction(
&self,
account_id: u32,
+10 -2
View File
@@ -33,6 +33,12 @@ pub struct EnteredLoginParam {
/// Imap server port.
pub imap_port: Option<u16>,
/// IMAP server folder.
///
/// Defaults to "INBOX" if not set.
/// Should not be an empty string.
pub imap_folder: Option<String>,
/// Imap socket security.
pub imap_security: Option<Socket>,
@@ -85,6 +91,7 @@ impl From<dc::EnteredLoginParam> for EnteredLoginParam {
password: param.imap.password,
imap_server: param.imap.server.into_option(),
imap_port: param.imap.port.into_option(),
imap_folder: param.imap.folder.into_option(),
imap_security: imap_security.into_option(),
imap_user: param.imap.user.into_option(),
smtp_server: param.smtp.server.into_option(),
@@ -104,14 +111,15 @@ impl TryFrom<EnteredLoginParam> for dc::EnteredLoginParam {
fn try_from(param: EnteredLoginParam) -> Result<Self> {
Ok(Self {
addr: param.addr,
imap: dc::EnteredServerLoginParam {
imap: dc::EnteredImapLoginParam {
server: param.imap_server.unwrap_or_default(),
port: param.imap_port.unwrap_or_default(),
folder: param.imap_folder.unwrap_or_default(),
security: param.imap_security.unwrap_or_default().into(),
user: param.imap_user.unwrap_or_default(),
password: param.password,
},
smtp: dc::EnteredServerLoginParam {
smtp: dc::EnteredSmtpLoginParam {
server: param.smtp_server.unwrap_or_default(),
port: param.smtp_port.unwrap_or_default(),
security: param.smtp_security.unwrap_or_default().into(),
+9 -18
View File
@@ -24,6 +24,8 @@ pub struct JsonrpcReaction {
#[serde(rename = "Reactions", rename_all = "camelCase")]
pub struct JsonrpcReactions {
/// Map from a contact to it's reaction to message.
/// There is only a single reaction per contact,
/// but this contains a list of reactions for historical reasons.
reactions_by_contact: BTreeMap<u32, Vec<String>>,
/// Unique reactions and their count, sorted in descending order.
reactions: Vec<JsonrpcReaction>,
@@ -31,27 +33,16 @@ pub struct JsonrpcReactions {
impl From<Reactions> for JsonrpcReactions {
fn from(reactions: Reactions) -> Self {
let mut reactions_by_contact: BTreeMap<u32, Vec<String>> = BTreeMap::new();
for contact_id in reactions.contacts() {
let reaction = reactions.get(contact_id);
if reaction.is_empty() {
continue;
}
let emojis: Vec<String> = reaction
.emojis()
.into_iter()
.map(|emoji| emoji.to_owned())
.collect();
reactions_by_contact.insert(contact_id.to_u32(), emojis.clone());
}
let self_reactions = reactions_by_contact.get(&ContactId::SELF.to_u32());
let reactions_by_contact: BTreeMap<u32, Vec<String>> = reactions
.iter()
.map(|(key, value)| (key.to_u32(), vec![value.as_str().to_string()]))
.collect();
let self_reaction = reactions_by_contact.get(&ContactId::SELF.to_u32());
let mut reactions_v = Vec::new();
for (emoji, count) in reactions.emoji_sorted_by_frequency() {
let is_from_self = if let Some(self_reactions) = self_reactions {
self_reactions.contains(&emoji)
let is_from_self = if let Some(self_reaction) = self_reaction {
self_reaction.contains(&emoji)
} else {
false
};
@@ -37,6 +37,11 @@ pub struct WebxdcMessageInfo {
internet_access: bool,
/// Address to be used for `window.webxdc.selfAddr` in JS land.
self_addr: String,
/// Address of the peer who initially shared the webxdc in the chat.
app_sender_addr: String,
/// True if updates sent by the local user
/// will only be seen by the app sender.
can_only_send_updates_to_app_sender: bool,
/// Milliseconds to wait before calling `sendUpdate()` again since the last call.
/// Should be exposed to `window.sendUpdateInterval` in JS land.
send_update_interval: usize,
@@ -60,6 +65,8 @@ impl WebxdcMessageInfo {
request_integration: _,
internet_access,
self_addr,
app_sender_addr,
can_only_send_updates_to_app_sender,
send_update_interval,
send_update_max_size,
} = message.get_webxdc_info(context).await?;
@@ -72,6 +79,8 @@ impl WebxdcMessageInfo {
source_code_url: maybe_empty_string_to_option(source_code_url),
internet_access,
self_addr,
app_sender_addr,
can_only_send_updates_to_app_sender,
send_update_interval,
send_update_max_size,
})
+1 -1
View File
@@ -54,5 +54,5 @@
},
"type": "module",
"types": "dist/deltachat.d.ts",
"version": "2.48.0-dev"
"version": "2.50.0-dev"
}
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "deltachat-repl"
version = "2.48.0-dev"
version = "2.50.0-dev"
license = "MPL-2.0"
edition = "2021"
repository = "https://github.com/chatmail/core"
+1 -1
View File
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "deltachat-rpc-client"
version = "2.48.0-dev"
version = "2.50.0-dev"
license = "MPL-2.0"
description = "Python client for Delta Chat core JSON-RPC interface"
classifiers = [
@@ -405,7 +405,15 @@ class Account:
@futuremethod
def wait_next_messages(self) -> list[Message]:
"""Wait for new messages and return a list of them."""
"""(deprecated) Wait for new messages and return a list of them. Meant for bots.
Deprecated 2026-04: This returns the message's id as soon as the first part arrives,
even if it is not fully downloaded yet.
The bot needs to wait for the message to be fully downloaded.
Since this is usually not the desired behavior,
bots should instead use the `EventType.INCOMING_MSG`
event for getting notified about new messages.
"""
next_msg_ids = yield self._rpc.wait_next_msgs.future(self.id)
return [Message(self, msg_id) for msg_id in next_msg_ids]
@@ -25,7 +25,14 @@ class Message:
return self.account._rpc
def send_reaction(self, *reaction: str) -> "Message":
"""Send a reaction to this message."""
"""
Sends a reaction to message.
A reaction is a string that represents an emoji.
You can call this function again to change the emoji;
the last sent reaction overrides all previously sent reactions.
It is possible to remove the reaction by sending an empty string.
"""
msg_id = self._rpc.send_reaction(self.account.id, self.id, reaction)
return Message(self.account, msg_id)
+25 -61
View File
@@ -2,32 +2,13 @@ import logging
import re
import time
import pytest
from imap_tools import AND, U
from deltachat_rpc_client import Contact, EventType, Message
def test_move_works(acfactory, direct_imap):
ac1, ac2 = acfactory.get_online_accounts(2)
ac2_direct_imap = direct_imap(ac2)
ac2_direct_imap.create_folder("DeltaChat")
ac2.set_config("mvbox_move", "1")
ac2.bring_online()
chat = ac1.create_chat(ac2)
chat.send_text("message1")
# Message is moved to the movebox
ac2.wait_for_event(EventType.IMAP_MESSAGE_MOVED)
# Message is downloaded
msg = ac2.wait_for_incoming_msg().get_snapshot()
assert msg.text == "message1"
def test_reactions_for_a_reordering_move(acfactory, direct_imap):
"""When a batch of messages is moved from Inbox to DeltaChat folder with a single MOVE command,
"""When a batch of messages is moved from Inbox to another folder with a single MOVE command,
their UIDs may be reordered (e.g. Gmail is known for that) which led to that messages were
processed by receive_imf in the wrong order, and, particularly, reactions were processed before
messages they refer to and thus dropped.
@@ -37,9 +18,6 @@ def test_reactions_for_a_reordering_move(acfactory, direct_imap):
addr, password = acfactory.get_credentials()
ac2 = acfactory.get_unconfigured_account()
ac2.add_or_update_transport({"addr": addr, "password": password})
ac2_direct_imap = direct_imap(ac2)
ac2_direct_imap.create_folder("DeltaChat")
ac2.set_config("mvbox_move", "1")
assert ac2.is_configured()
ac2.bring_online()
@@ -55,11 +33,17 @@ def test_reactions_for_a_reordering_move(acfactory, direct_imap):
react_str = "\N{THUMBS UP SIGN}"
msg1.send_reaction(react_str).wait_until_delivered()
logging.info("moving messages to ac2's DeltaChat folder in the reverse order")
logging.info("moving messages to ac2's movebox folder in the reverse order")
ac2_direct_imap = direct_imap(ac2)
ac2_direct_imap.create_folder("Movebox")
ac2_direct_imap.connect()
for uid in sorted([m.uid for m in ac2_direct_imap.get_all_messages()], reverse=True):
ac2_direct_imap.conn.move(uid, "DeltaChat")
ac2_direct_imap.conn.move(uid, "Movebox")
logging.info("moving messages back")
ac2_direct_imap.select_folder("Movebox")
for uid in sorted([m.uid for m in ac2_direct_imap.get_all_messages()]):
ac2_direct_imap.conn.move(uid, "INBOX")
logging.info("receiving messages by ac2")
ac2.start_io()
@@ -72,33 +56,22 @@ def test_reactions_for_a_reordering_move(acfactory, direct_imap):
assert list(reactions.reactions_by_contact.values())[0] == [react_str]
def test_move_works_on_self_sent(acfactory, direct_imap):
ac1, ac2 = acfactory.get_online_accounts(2)
# Create and enable movebox.
ac1_direct_imap = direct_imap(ac1)
ac1_direct_imap.create_folder("DeltaChat")
ac1.set_config("mvbox_move", "1")
ac1.set_config("bcc_self", "1")
ac1.bring_online()
chat = ac1.create_chat(ac2)
chat.send_text("message1")
ac1.wait_for_event(EventType.IMAP_MESSAGE_MOVED)
chat.send_text("message2")
ac1.wait_for_event(EventType.IMAP_MESSAGE_MOVED)
chat.send_text("message3")
ac1.wait_for_event(EventType.IMAP_MESSAGE_MOVED)
def test_moved_markseen(acfactory, direct_imap):
def test_moved_markseen(acfactory, direct_imap, log):
"""Test that message already moved to DeltaChat folder is marked as seen."""
ac1, ac2 = acfactory.get_online_accounts(2)
ac1 = acfactory.get_online_account()
addr, password = acfactory.get_credentials()
ac2 = acfactory.get_unconfigured_account()
ac2.add_or_update_transport({"addr": addr, "password": password})
ac2.bring_online()
log.section("ac2: creating DeltaChat folder")
ac2_direct_imap = direct_imap(ac2)
ac2_direct_imap.create_folder("DeltaChat")
ac2.set_config("mvbox_move", "1")
ac2.set_config("delete_server_after", "0")
ac2.set_config("sync_msgs", "0") # Do not send a sync message when accepting a contact request.
ac2.add_or_update_transport({"addr": addr, "password": password, "imapFolder": "DeltaChat"})
ac2.bring_online()
ac2.stop_io()
@@ -108,6 +81,7 @@ def test_moved_markseen(acfactory, direct_imap):
idle2.wait_for_new_message()
# Emulate moving of the message to DeltaChat folder by Sieve rule.
log.section("ac2: moving message into DeltaChat folder")
ac2_direct_imap.conn.move(["*"], "DeltaChat")
ac2_direct_imap.select_folder("DeltaChat")
assert len(list(ac2_direct_imap.conn.fetch("*", mark_seen=False))) == 1
@@ -131,17 +105,11 @@ def test_moved_markseen(acfactory, direct_imap):
assert len(list(ac2_direct_imap.conn.fetch(AND(seen=True, uid=U(1, "*")), mark_seen=False))) == 1
@pytest.mark.parametrize("mvbox_move", [True, False])
def test_markseen_message_and_mdn(acfactory, direct_imap, mvbox_move):
def test_markseen_message_and_mdn(acfactory, direct_imap):
ac1, ac2 = acfactory.get_online_accounts(2)
for ac in ac1, ac2:
ac.set_config("delete_server_after", "0")
if mvbox_move:
ac_direct_imap = direct_imap(ac)
ac_direct_imap.create_folder("DeltaChat")
ac.set_config("mvbox_move", "1")
ac.bring_online()
# Do not send BCC to self, we only want to test MDN on ac1.
ac1.set_config("bcc_self", "0")
@@ -150,10 +118,7 @@ def test_markseen_message_and_mdn(acfactory, direct_imap, mvbox_move):
msg = ac2.wait_for_incoming_msg()
msg.mark_seen()
if mvbox_move:
rex = re.compile("Marked messages [0-9]+ in folder DeltaChat as seen.")
else:
rex = re.compile("Marked messages [0-9]+ in folder INBOX as seen.")
rex = re.compile("Marked messages [0-9]+ in folder INBOX as seen.")
for ac in ac1, ac2:
while True:
@@ -161,12 +126,11 @@ def test_markseen_message_and_mdn(acfactory, direct_imap, mvbox_move):
if event.kind == EventType.INFO and rex.search(event.msg):
break
folder = "mvbox" if mvbox_move else "inbox"
ac1_direct_imap = direct_imap(ac1)
ac2_direct_imap = direct_imap(ac2)
ac1_direct_imap.select_config_folder(folder)
ac2_direct_imap.select_config_folder(folder)
ac1_direct_imap.select_folder("INBOX")
ac2_direct_imap.select_folder("INBOX")
# Check that the mdn is marked as seen
assert len(list(ac1_direct_imap.conn.fetch(AND(seen=True), mark_seen=False))) == 1
@@ -9,10 +9,6 @@ def test_add_second_address(acfactory) -> None:
account = acfactory.new_configured_account()
assert len(account.list_transports()) == 1
# When the first transport is created,
# mvbox_move and only_fetch_mvbox should be disabled.
assert account.get_config("mvbox_move") == "0"
assert account.get_config("only_fetch_mvbox") == "0"
assert account.get_config("show_emails") == "2"
qr = acfactory.get_account_qr()
@@ -32,32 +28,10 @@ def test_add_second_address(acfactory) -> None:
account.delete_transport(second_addr)
assert len(account.list_transports()) == 2
# Enabling mvbox_move or only_fetch_mvbox
# is not allowed when multi-transport is enabled.
for option in ["mvbox_move", "only_fetch_mvbox"]:
with pytest.raises(JsonRpcError):
account.set_config(option, "1")
# show_emails does not matter for multi-relay, can be set to anything
account.set_config("show_emails", "0")
@pytest.mark.parametrize("key", ["mvbox_move", "only_fetch_mvbox"])
def test_no_second_transport_with_mvbox(acfactory, key) -> None:
"""Test that second transport cannot be configured if mvbox is used."""
account = acfactory.new_configured_account()
assert len(account.list_transports()) == 1
assert account.get_config("mvbox_move") == "0"
assert account.get_config("only_fetch_mvbox") == "0"
qr = acfactory.get_account_qr()
account.set_config(key, "1")
with pytest.raises(JsonRpcError):
account.add_transport_from_qr(qr)
def test_second_transport_without_classic_emails(acfactory) -> None:
"""Test that second transport can be configured if classic emails are not fetched."""
account = acfactory.new_configured_account()
@@ -147,44 +121,13 @@ def test_download_on_demand(acfactory) -> None:
assert msg.get_snapshot().download_state == dstate
@pytest.mark.parametrize("is_chatmail", ["0", "1"])
def test_mvbox_move_first_transport(acfactory, is_chatmail) -> None:
"""Test that mvbox_move is disabled by default even for non-chatmail accounts.
Disabling mvbox_move is required to be able to setup a second transport.
"""
account = acfactory.get_unconfigured_account()
account.set_config("fix_is_chatmail", "1")
account.set_config("is_chatmail", is_chatmail)
# The default value when the setting is unset is "1".
# This is not changed for compatibility with old databases
# imported from backups.
assert account.get_config("mvbox_move") == "1"
qr = acfactory.get_account_qr()
account.add_transport_from_qr(qr)
# Once the first transport is set up,
# mvbox_move is disabled.
assert account.get_config("mvbox_move") == "0"
assert account.get_config("is_chatmail") == is_chatmail
def test_reconfigure_transport(acfactory) -> None:
"""Test that reconfiguring the transport works
even if settings not supported for multi-transport
like mvbox_move are enabled."""
"""Test that reconfiguring the transport works."""
account = acfactory.get_online_account()
account.set_config("mvbox_move", "1")
[transport] = account.list_transports()
account.add_or_update_transport(transport)
# Reconfiguring the transport should not reset
# the settings as if when configuring the first transport.
assert account.get_config("mvbox_move") == "1"
def test_transport_synchronization(acfactory, log) -> None:
"""Test synchronization of transports between devices."""
@@ -1047,6 +1047,7 @@ def test_no_old_msg_is_fresh(acfactory):
assert ac1.create_chat(ac2).get_fresh_message_count() == 1
assert len(list(ac1.get_fresh_messages())) == 1
ac1_clone.wait_for_incoming_msg_event()
ac1.wait_for_event(EventType.IMAP_INBOX_IDLE)
logging.info("Send a message from ac1_clone to ac2 and check that ac1 marks the first message as 'noticed'")
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "deltachat-rpc-server"
version = "2.48.0-dev"
version = "2.50.0-dev"
description = "DeltaChat JSON-RPC server"
edition = "2021"
readme = "README.md"
+1 -1
View File
@@ -18,7 +18,7 @@ import { startDeltaChat } from "@deltachat/stdio-rpc-server";
import { C } from "@deltachat/jsonrpc-client";
async function main() {
const dc = await startDeltaChat("deltachat-data");
const dc = startDeltaChat("deltachat-data");
console.log(await dc.rpc.getSystemInfo());
dc.close()
}
+9 -2
View File
@@ -15,7 +15,7 @@ export interface SearchOptions {
*/
export function getRPCServerPath(
options?: Partial<SearchOptions>
): Promise<string>;
): string;
@@ -33,8 +33,15 @@ export interface StartOptions {
* @param directory directory for accounts folder
* @param options
*/
export function startDeltaChat(directory: string, options?: Partial<SearchOptions & StartOptions> ): Promise<DeltaChatOverJsonRpcServer>
export function startDeltaChat(directory: string, options?: Partial<SearchOptions & StartOptions> ): DeltaChatOverJsonRpcServer
export class DeltaChatOverJsonRpc extends StdioDeltaChat {
constructor(
directory: string,
options?: Partial<SearchOptions & StartOptions>
);
readonly pathToServerBinary: string;
}
export namespace FnTypes {
export type getRPCServerPath = typeof getRPCServerPath
+48 -40
View File
@@ -1,6 +1,6 @@
//@ts-check
import { spawn } from "node:child_process";
import { stat } from "node:fs/promises";
import { statSync } from "node:fs";
import os from "node:os";
import process from "node:process";
import { ENV_VAR_NAME, PATH_EXECUTABLE_NAME } from "./src/const.js";
@@ -36,7 +36,7 @@ function findRPCServerInNodeModules() {
}
/** @type {import("./index").FnTypes.getRPCServerPath} */
export async function getRPCServerPath(options = {}) {
export function getRPCServerPath(options = {}) {
const { takeVersionFromPATH, disableEnvPath } = {
takeVersionFromPATH: false,
disableEnvPath: false,
@@ -45,7 +45,7 @@ export async function getRPCServerPath(options = {}) {
// 1. check if it is set as env var
if (process.env[ENV_VAR_NAME] && !disableEnvPath) {
try {
if (!(await stat(process.env[ENV_VAR_NAME])).isFile()) {
if (!statSync(process.env[ENV_VAR_NAME]).isFile()) {
throw new Error(
`expected ${ENV_VAR_NAME} to point to the deltachat-rpc-server executable`
);
@@ -68,41 +68,49 @@ export async function getRPCServerPath(options = {}) {
import { StdioDeltaChat } from "@deltachat/jsonrpc-client";
/** @type {import("./index").FnTypes.startDeltaChat} */
export async function startDeltaChat(directory, options = {}) {
const pathToServerBinary = await getRPCServerPath(options);
const server = spawn(pathToServerBinary, {
env: {
RUST_LOG: process.env.RUST_LOG,
DC_ACCOUNTS_PATH: directory,
},
stdio: ["pipe", "pipe", options.muteStdErr ? "ignore" : "inherit"],
});
server.on("error", (err) => {
throw new Error(FAILED_TO_START_SERVER_EXECUTABLE(pathToServerBinary, err));
});
let shouldClose = false;
server.on("exit", () => {
if (shouldClose) {
return;
}
throw new Error("Server quit");
});
/** @type {import('./index').DeltaChatOverJsonRpcServer} */
//@ts-expect-error
const dc = new StdioDeltaChat(server.stdin, server.stdout, true);
dc.close = () => {
shouldClose = true;
if (!server.kill()) {
console.log("server termination failed");
}
};
//@ts-expect-error
dc.pathToServerBinary = pathToServerBinary;
return dc;
export function startDeltaChat(directory, options = {}) {
return new DeltaChatOverJsonRpc(directory, options);
}
export class DeltaChatOverJsonRpc extends StdioDeltaChat {
/**
*
* @param {string} directory
* @param {Partial<import("./index").SearchOptions & import("./index").StartOptions>} options
*/
constructor(directory, options = {}) {
const pathToServerBinary = getRPCServerPath(options);
const server = spawn(pathToServerBinary, {
env: {
RUST_LOG: process.env.RUST_LOG,
DC_ACCOUNTS_PATH: directory,
},
stdio: ["pipe", "pipe", options.muteStdErr ? "ignore" : "inherit"],
});
server.on("error", (err) => {
throw new Error(
FAILED_TO_START_SERVER_EXECUTABLE(pathToServerBinary, err)
);
});
let shouldClose = false;
server.on("exit", () => {
if (shouldClose) {
return;
}
throw new Error("Server quit");
});
super(server.stdin, server.stdout, true);
this.close = () => {
shouldClose = true;
if (!server.kill()) {
console.log("server termination failed");
}
};
this.pathToServerBinary = pathToServerBinary;
}
}
@@ -15,5 +15,5 @@
},
"type": "module",
"types": "index.d.ts",
"version": "2.48.0-dev"
"version": "2.50.0-dev"
}
+12
View File
@@ -23,7 +23,19 @@ ignore = [
# it is a transitive dependency of iroh 0.35.0
# which depends on ^0.102.
# <https://rustsec.org/advisories/RUSTSEC-2026-0049>
# <https://rustsec.org/advisories/RUSTSEC-2026-0098>
# <https://rustsec.org/advisories/RUSTSEC-2026-0099>
"RUSTSEC-2026-0049",
"RUSTSEC-2026-0098",
"RUSTSEC-2026-0099",
# rand 0.8.x
# <https://rustsec.org/advisories/RUSTSEC-2026-0097>
# We already use rand 0.9,
# version 0.8 that cannot be upgraded
# is a dependency of iroh 0.35.0 and rPGP.
# rPGP upgrade is waiting for <https://github.com/rpgp/rpgp/pull/573>
"RUSTSEC-2026-0097"
]
[bans]
+1 -1
View File
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "deltachat"
version = "2.48.0-dev"
version = "2.50.0-dev"
license = "MPL-2.0"
description = "Python bindings for the Delta Chat Core library using CFFI against the Rust-implemented libdeltachat"
readme = "README.rst"
-4
View File
@@ -254,10 +254,6 @@ class Message:
"""Quote setter."""
lib.dc_msg_set_quote(self._dc_msg, quoted_message._dc_msg)
def force_plaintext(self) -> None:
"""Force the message to be sent in plain text."""
lib.dc_msg_force_plaintext(self._dc_msg)
@property
def error(self) -> Optional[str]:
"""Error message."""
-1
View File
@@ -522,7 +522,6 @@ class ACFactory:
ac = self.get_unconfigured_account()
assert "addr" in configdict and "mail_pw" in configdict, configdict
configdict.setdefault("bcc_self", False)
configdict.setdefault("mvbox_move", False)
configdict.setdefault("sync_msgs", False)
configdict.setdefault("delete_server_after", 0)
ac.update_config(configdict)
+1
View File
@@ -288,6 +288,7 @@ def test_use_new_verified_group_after_going_online(acfactory, data, tmp_path, lp
assert open(contact.get_profile_image(), "rb").read() == open(avatar_path, "rb").read()
lp.sec("ac2_offl: sending message")
chat2.accept()
msg_out = chat2.send_text("hello")
lp.sec("ac1: receiving message")
+10 -10
View File
@@ -52,19 +52,19 @@ class TestOfflineAccountBasic:
def test_set_config_int_conversion(self, acfactory):
ac1 = acfactory.get_unconfigured_account()
ac1.set_config("mvbox_move", False)
assert ac1.get_config("mvbox_move") == "0"
ac1.set_config("mvbox_move", True)
assert ac1.get_config("mvbox_move") == "1"
ac1.set_config("mvbox_move", 0)
assert ac1.get_config("mvbox_move") == "0"
ac1.set_config("mvbox_move", 1)
assert ac1.get_config("mvbox_move") == "1"
ac1.set_config("bcc_self", False)
assert ac1.get_config("bcc_self") == "0"
ac1.set_config("bcc_self", True)
assert ac1.get_config("bcc_self") == "1"
ac1.set_config("bcc_self", 0)
assert ac1.get_config("bcc_self") == "0"
ac1.set_config("bcc_self", 1)
assert ac1.get_config("bcc_self") == "1"
def test_update_config(self, acfactory):
ac1 = acfactory.get_unconfigured_account()
ac1.update_config({"mvbox_move": False})
assert ac1.get_config("mvbox_move") == "0"
ac1.update_config({"bcc_self": True})
assert ac1.get_config("bcc_self") == "1"
def test_has_bccself(self, acfactory):
ac1 = acfactory.get_unconfigured_account()
+1 -1
View File
@@ -1 +1 @@
2026-03-24
2026-04-13
-16
View File
@@ -48,19 +48,3 @@ the build machine (ask your friendly sysadmin on #deltachat Libera Chat) to type
This will **rsync** your current checkout to the remote build machine
(no need to commit before) and then run either rust or python tests.
# coredeps Dockerfile
`coredeps/Dockerfile` specifies an image that contains all
of Delta Chat's core dependencies. It is used to
build python wheels (binary packages for Python).
You can build the docker images yourself locally
to avoid the relatively large download:
cd scripts # where all CI things are
docker build -t deltachat/coredeps coredeps
Additionally, you can install qemu and build arm64 docker image on x86\_64 machine:
apt-get install qemu binfmt-support qemu-user-static
docker build -t deltachat/coredeps-arm64 --build-arg BASEIMAGE=quay.io/pypa/manylinux2014_aarch64 coredeps
+23
View File
@@ -0,0 +1,23 @@
#!/usr/bin/env python3
#
# Script to check that current version ends with -dev.
# Meant to be run in CI to check that PRs are made against the -dev version.
# If the version is not -dev, it was forgotten to be updated
# after making a release.
from pathlib import Path
import tomllib
import sys
def main():
with Path("Cargo.toml").open("rb") as fp:
cargo_toml = tomllib.load(fp)
version = cargo_toml["package"]["version"]
if not version.endswith("-dev"):
print(f"Current version {version} does not end with -dev", file=sys.stderr)
sys.exit(1)
if __name__ == "__main__":
main()
-26
View File
@@ -1,26 +0,0 @@
# Concourse CI pipeline
`docs_wheels.yml` is a pipeline for [Concourse CI](https://concourse-ci.org/)
that builds C documentation, Python documentation, Python wheels for `x86_64`
and `aarch64` and Python source packages, and uploads them.
To setup the pipeline run
```
fly -t <your-target> set-pipeline -c docs_wheels.yml -p docs_wheels -l secret.yml
```
where `secret.yml` contains the following secrets:
```
c.delta.chat:
private_key: |
-----BEGIN OPENSSH PRIVATE KEY-----
...
-----END OPENSSH PRIVATE KEY-----
devpi:
login: dc
password: ...
```
Secrets can be read from the password manager:
```
fly -t b1 set-pipeline -c docs_wheels.yml -p docs_wheels -l <(pass show delta/b1.delta.chat/secret.yml)
```
-305
View File
@@ -1,305 +0,0 @@
resources:
- name: deltachat-core-rust
type: git
icon: github
source:
branch: main
uri: https://github.com/chatmail/core.git
- name: deltachat-core-rust-release
type: git
icon: github
source:
branch: main
uri: https://github.com/chatmail/core.git
tag_filter: "v*"
jobs:
- name: python-x86_64
plan:
- get: deltachat-core-rust
- get: deltachat-core-rust-release
trigger: true
# Build manylinux image with additional dependencies
- task: build-coredeps
privileged: true
config:
inputs:
# Building the latest, not tagged coredeps
- name: deltachat-core-rust
image_resource:
source:
repository: concourse/oci-build-task
type: registry-image
outputs:
- name: coredeps-image
path: image
params:
CONTEXT: deltachat-core-rust/scripts/coredeps
UNPACK_ROOTFS: "true"
BUILD_ARG_BASEIMAGE: quay.io/pypa/manylinux2014_x86_64
platform: linux
caches:
- path: cache
run:
path: build
# Use built image to build python wheels
- task: build-wheels
image: coredeps-image
config:
inputs:
- name: deltachat-core-rust-release
path: .
outputs:
# Binary wheels
- name: py-wheels
path: ./python/.docker-tox/wheelhouse/
platform: linux
run:
path: bash
args:
- -exc
- |
scripts/run_all.sh
# Upload x86_64 wheels and source packages
- task: upload-wheels
config:
inputs:
- name: py-wheels
image_resource:
type: registry-image
source:
repository: debian
platform: linux
run:
path: sh
args:
- -ec
- |
apt-get update -y
apt-get install -y --no-install-recommends python3-pip python3-setuptools python3-venv
python3 -m venv env
env/bin/pip install --upgrade pip
env/bin/pip install devpi
env/bin/devpi use https://m.devpi.net/dc/master
env/bin/devpi login ((devpi.login)) --password ((devpi.password))
env/bin/devpi upload py-wheels/*manylinux201*
- name: python-aarch64
plan:
- get: deltachat-core-rust
- get: deltachat-core-rust-release
trigger: true
# Build manylinux image with additional dependencies
- task: build-coredeps
privileged: true
config:
inputs:
# Building the latest, not tagged coredeps
- name: deltachat-core-rust
image_resource:
source:
repository: concourse/oci-build-task
type: registry-image
outputs:
- name: coredeps-image
path: image
params:
CONTEXT: deltachat-core-rust/scripts/coredeps
UNPACK_ROOTFS: "true"
BUILD_ARG_BASEIMAGE: quay.io/pypa/manylinux2014_aarch64
platform: linux
caches:
- path: cache
run:
path: build
# Use built image to build python wheels
- task: build-wheels
image: coredeps-image
config:
inputs:
- name: deltachat-core-rust-release
path: .
outputs:
- name: py-wheels
path: ./python/.docker-tox/wheelhouse/
platform: linux
run:
path: bash
args:
- -exc
- |
scripts/run_all.sh
# Upload aarch64 wheels
- task: upload-wheels
config:
inputs:
- name: py-wheels
image_resource:
type: registry-image
source:
repository: debian
platform: linux
run:
path: sh
args:
- -ec
- |
apt-get update -y
apt-get install -y --no-install-recommends python3-pip python3-setuptools python3-venv
python3 -m venv env
env/bin/pip install --upgrade pip
env/bin/pip install devpi
env/bin/devpi use https://m.devpi.net/dc/master
env/bin/devpi login ((devpi.login)) --password ((devpi.password))
env/bin/devpi upload py-wheels/*manylinux201*
- name: python-musl-x86_64
plan:
- get: deltachat-core-rust
- get: deltachat-core-rust-release
trigger: true
# Build manylinux image with additional dependencies
- task: build-coredeps
privileged: true
config:
inputs:
# Building the latest, not tagged coredeps
- name: deltachat-core-rust
image_resource:
source:
repository: concourse/oci-build-task
type: registry-image
outputs:
- name: coredeps-image
path: image
params:
CONTEXT: deltachat-core-rust/scripts/coredeps
UNPACK_ROOTFS: "true"
BUILD_ARG_BASEIMAGE: quay.io/pypa/musllinux_1_1_x86_64
platform: linux
caches:
- path: cache
run:
path: build
# Use built image to build python wheels
- task: build-wheels
image: coredeps-image
config:
inputs:
- name: deltachat-core-rust-release
path: .
outputs:
- name: py-wheels
path: ./python/.docker-tox/wheelhouse/
platform: linux
run:
path: bash
args:
- -exc
- |
scripts/run_all.sh
# Upload musl x86_64 wheels
- task: upload-wheels
config:
inputs:
- name: py-wheels
image_resource:
type: registry-image
source:
repository: debian
platform: linux
run:
path: sh
args:
- -ec
- |
apt-get update -y
apt-get install -y --no-install-recommends python3-pip python3-setuptools python3-venv
python3 -m venv env
env/bin/pip install --upgrade pip
env/bin/pip install devpi
env/bin/devpi use https://m.devpi.net/dc/master
env/bin/devpi login ((devpi.login)) --password ((devpi.password))
env/bin/devpi upload py-wheels/*musllinux_1_1_x86_64*
- name: python-musl-aarch64
plan:
- get: deltachat-core-rust
- get: deltachat-core-rust-release
trigger: true
# Build manylinux image with additional dependencies
- task: build-coredeps
privileged: true
config:
inputs:
# Building the latest, not tagged coredeps
- name: deltachat-core-rust
image_resource:
source:
repository: concourse/oci-build-task
type: registry-image
outputs:
- name: coredeps-image
path: image
params:
CONTEXT: deltachat-core-rust/scripts/coredeps
UNPACK_ROOTFS: "true"
BUILD_ARG_BASEIMAGE: quay.io/pypa/musllinux_1_1_aarch64
platform: linux
caches:
- path: cache
run:
path: build
# Use built image to build python wheels
- task: build-wheels
image: coredeps-image
config:
inputs:
- name: deltachat-core-rust-release
path: .
outputs:
- name: py-wheels
path: ./python/.docker-tox/wheelhouse/
platform: linux
run:
path: bash
args:
- -exc
- |
scripts/run_all.sh
# Upload musl aarch64 wheels
- task: upload-wheels
config:
inputs:
- name: py-wheels
image_resource:
type: registry-image
source:
repository: debian
platform: linux
run:
path: sh
args:
- -ec
- |
apt-get update -y
apt-get install -y --no-install-recommends python3-pip python3-setuptools python3-venv
python3 -m venv env
env/bin/pip install --upgrade pip
env/bin/pip install devpi
env/bin/devpi use https://m.devpi.net/dc/master
env/bin/devpi login ((devpi.login)) --password ((devpi.password))
env/bin/devpi upload py-wheels/*musllinux_1_1_aarch64*
-9
View File
@@ -1,9 +0,0 @@
ARG BASEIMAGE=quay.io/pypa/manylinux2014_x86_64
#ARG BASEIMAGE=quay.io/pypa/musllinux_1_1_x86_64
#ARG BASEIMAGE=quay.io/pypa/manylinux2014_aarch64
FROM $BASEIMAGE
RUN pipx install tox
COPY install-rust.sh /scripts/
RUN /scripts/install-rust.sh
RUN if command -v yum; then yum install -y perl-IPC-Cmd; fi
-20
View File
@@ -1,20 +0,0 @@
#!/usr/bin/env bash
set -euo pipefail
# Install Rust
#
# Path from https://forge.rust-lang.org/infra/other-installation-methods.html
#
# Avoid using rustup here as it depends on reading /proc/self/exe and
# has problems running under QEMU.
RUST_VERSION=1.94.0
ARCH="$(uname -m)"
test -f "/lib/libc.musl-$ARCH.so.1" && LIBC=musl || LIBC=gnu
curl "https://static.rust-lang.org/dist/rust-${RUST_VERSION}-$ARCH-unknown-linux-$LIBC.tar.gz" | tar xz
cd "rust-${RUST_VERSION}-$ARCH-unknown-linux-$LIBC"
./install.sh --prefix=/usr --components=rustc,cargo,"rust-std-$ARCH-unknown-linux-$LIBC"
rustc --version
cd ..
rm -fr "rust-${RUST_VERSION}-$ARCH-unknown-linux-$LIBC"
+17 -21
View File
@@ -10,8 +10,8 @@ use anyhow::{Context as _, Result, ensure, format_err};
use base64::Engine as _;
use futures::StreamExt;
use image::ImageReader;
use image::codecs::jpeg::JpegEncoder;
use image::{DynamicImage, GenericImage, GenericImageView, ImageFormat, Pixel, Rgba};
use image::{codecs::jpeg::JpegEncoder, metadata::Orientation};
use num_traits::FromPrimitive;
use tokio::{fs, task};
use tokio_stream::wrappers::ReadDirStream;
@@ -362,7 +362,10 @@ impl<'a> BlobObject<'a> {
return Ok(name);
}
let mut img = imgreader.decode().context("image decode failure")?;
let orientation = exif.as_ref().map(|exif| exif_orientation(exif, context));
let orientation = exif
.as_ref()
.map(|exif| exif_orientation(exif, context))
.unwrap_or(Orientation::NoTransforms);
let mut encoded = Vec::new();
if *vt == Viewtype::Sticker {
@@ -381,13 +384,7 @@ impl<'a> BlobObject<'a> {
return Ok(name);
}
}
img = match orientation {
Some(90) => img.rotate90(),
Some(180) => img.rotate180(),
Some(270) => img.rotate270(),
_ => img,
};
img.apply_orientation(orientation);
// max_wh is the maximum image width and height, i.e. the resolution-limit.
// target_wh target-resolution for resizing the image.
@@ -468,7 +465,7 @@ impl<'a> BlobObject<'a> {
));
}
target_wh = target_wh * 2 / 3;
target_wh = target_wh * 7 / 8;
} else {
info!(
context,
@@ -551,18 +548,17 @@ fn image_metadata(file: &std::fs::File) -> Result<(u64, Option<exif::Exif>)> {
Ok((len, exif))
}
fn exif_orientation(exif: &exif::Exif, context: &Context) -> i32 {
if let Some(orientation) = exif.get_field(exif::Tag::Orientation, exif::In::PRIMARY) {
// possible orientation values are described at http://sylvana.net/jpegcrop/exif_orientation.html
// we only use rotation, in practise, flipping is not used.
match orientation.value.get_uint(0) {
Some(3) => return 180,
Some(6) => return 90,
Some(8) => return 270,
other => warn!(context, "Exif orientation value ignored: {other:?}."),
}
fn exif_orientation(exif: &exif::Exif, context: &Context) -> Orientation {
if let Some(orientation) = exif.get_field(exif::Tag::Orientation, exif::In::PRIMARY)
&& let Some(val) = orientation.value.get_uint(0)
&& let Ok(val) = TryInto::<u8>::try_into(val)
{
return Orientation::from_exif(val).unwrap_or({
warn!(context, "Exif orientation value ignored: {val:?}.");
Orientation::NoTransforms
});
}
0
Orientation::NoTransforms
}
/// All files in the blobdir.
+25 -3
View File
@@ -305,7 +305,7 @@ async fn test_recode_image_2() {
has_exif: true,
original_width: 2000,
original_height: 1800,
orientation: 270,
orientation: Some(Orientation::Rotate270),
compressed_width: 1800,
compressed_height: 2000,
..Default::default()
@@ -336,6 +336,28 @@ async fn test_recode_image_2() {
assert_correct_rotation(&img_rotated);
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_recode_image_vflipped() {
let bytes = include_bytes!("../../test-data/image/rectangle200x180-vflipped.jpg");
let img_rotated = SendImageCheckMediaquality {
viewtype: Viewtype::Image,
media_quality_config: "0",
bytes,
extension: "jpg",
has_exif: true,
original_width: 200,
original_height: 180,
orientation: Some(Orientation::FlipVertical),
compressed_width: 200,
compressed_height: 180,
..Default::default()
}
.test()
.await
.unwrap();
assert_correct_rotation(&img_rotated);
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_recode_image_bad_exif() {
// `exiftool` reports for this file "Bad offset for IFD0 XResolution", still Exif must be
@@ -530,7 +552,7 @@ struct SendImageCheckMediaquality<'a> {
pub(crate) has_exif: bool,
pub(crate) original_width: u32,
pub(crate) original_height: u32,
pub(crate) orientation: i32,
pub(crate) orientation: Option<Orientation>,
pub(crate) res_viewtype: Option<Viewtype>,
pub(crate) compressed_width: u32,
pub(crate) compressed_height: u32,
@@ -546,7 +568,7 @@ impl SendImageCheckMediaquality<'_> {
let has_exif = self.has_exif;
let original_width = self.original_width;
let original_height = self.original_height;
let orientation = self.orientation;
let orientation = self.orientation.unwrap_or(Orientation::NoTransforms);
let res_viewtype = self.res_viewtype.unwrap_or(Viewtype::Image);
let compressed_width = self.compressed_width;
let compressed_height = self.compressed_height;
+128 -111
View File
@@ -1,7 +1,7 @@
//! # Chat module.
use std::cmp;
use std::collections::{BTreeSet, HashMap, HashSet};
use std::collections::{BTreeSet, HashMap};
use std::fmt;
use std::io::Cursor;
use std::marker::Sync;
@@ -477,12 +477,17 @@ impl ChatId {
/// Adds message "Messages are end-to-end encrypted".
pub(crate) async fn add_e2ee_notice(self, context: &Context, timestamp: i64) -> Result<()> {
let text = stock_str::messages_e2ee_info_msg(context);
// Sort this notice to the very beginning of the chat.
// We don't want any message to appear before this notice
// which is normally added when encrypted chat is created.
let sort_timestamp = 0;
add_info_msg_with_cmd(
context,
self,
&text,
SystemMessage::ChatE2ee,
Some(timestamp),
Some(sort_timestamp),
timestamp,
None,
None,
@@ -492,6 +497,27 @@ impl ChatId {
Ok(())
}
/// Adds info message to the beginning of the chat.
///
/// Used for messages such as
/// "Others will only see this group after you sent a first message."
pub(crate) async fn add_start_info_message(self, context: &Context, text: &str) -> Result<()> {
let sort_timestamp = 0;
add_info_msg_with_cmd(
context,
self,
text,
SystemMessage::Unknown,
Some(sort_timestamp),
time(),
None,
None,
None,
)
.await?;
Ok(())
}
/// Archives or unarchives a chat.
pub async fn set_visibility(self, context: &Context, visibility: ChatVisibility) -> Result<()> {
self.set_visibility_ex(context, Sync, visibility).await
@@ -925,6 +951,17 @@ SELECT id, rfc724_mid, pre_rfc724_mid, timestamp, ?, 1 FROM msgs WHERE chat_id=?
.unwrap_or(0))
}
/// Returns timestamp of us joining the chat if we are the member of the chat.
pub(crate) async fn join_timestamp(self, context: &Context) -> Result<Option<i64>> {
context
.sql
.query_get_value(
"SELECT add_timestamp FROM chats_contacts WHERE chat_id=? AND contact_id=?",
(self, ContactId::SELF),
)
.await
}
/// Returns timestamp of the latest message in the chat,
/// including hidden messages or a draft if there is one.
pub(crate) async fn get_timestamp(self, context: &Context) -> Result<Option<i64>> {
@@ -1151,7 +1188,6 @@ SELECT id, rfc724_mid, pre_rfc724_mid, timestamp, ?, 1 FROM msgs WHERE chat_id=?
/// prefer plaintext emails.
///
/// To get more verbose summary for a contact, including its key fingerprint, use [`Contact::get_encrinfo`].
#[expect(clippy::arithmetic_side_effects)]
pub async fn get_encryption_info(self, context: &Context) -> Result<String> {
let chat = Chat::load_from_db(context, self).await?;
if !chat.is_encrypted(context).await? {
@@ -1212,15 +1248,11 @@ SELECT id, rfc724_mid, pre_rfc724_mid, timestamp, ?, 1 FROM msgs WHERE chat_id=?
/// corresponding event in case of a system message (usually the current system time).
/// `always_sort_to_bottom` makes this adjust the returned timestamp up so that the message goes
/// to the chat bottom.
/// `received` -- whether the message is received. Otherwise being sent.
/// `incoming` -- whether the message is incoming.
pub(crate) async fn calc_sort_timestamp(
self,
context: &Context,
message_timestamp: i64,
always_sort_to_bottom: bool,
received: bool,
incoming: bool,
) -> Result<i64> {
let mut sort_timestamp = cmp::min(message_timestamp, smeared_time(context));
@@ -1240,38 +1272,6 @@ SELECT id, rfc724_mid, pre_rfc724_mid, timestamp, ?, 1 FROM msgs WHERE chat_id=?
(self, MessageState::OutDraft),
)
.await?
} else if received {
// Received messages shouldn't mingle with just sent ones and appear somewhere in the
// middle of the chat, so we go after the newest non fresh message.
//
// But if a received outgoing message is older than some seen message, better sort the
// received message purely by timestamp. We could place it just before that seen
// message, but anyway the user may not notice it.
//
// NB: Received outgoing messages may break sorting of fresh incoming ones, but this
// shouldn't happen frequently. Seen incoming messages don't really break sorting of
// fresh ones, they rather mean that older incoming messages are actually seen as well.
context
.sql
.query_row_optional(
"SELECT MAX(timestamp), MAX(IIF(state=?,timestamp_sent,0))
FROM msgs
WHERE chat_id=? AND hidden=0 AND state>?
HAVING COUNT(*) > 0",
(MessageState::InSeen, self, MessageState::InFresh),
|row| {
let ts: i64 = row.get(0)?;
let ts_sent_seen: i64 = row.get(1)?;
Ok((ts, ts_sent_seen))
},
)
.await?
.and_then(|(ts, ts_sent_seen)| {
match incoming || ts_sent_seen <= message_timestamp {
true => Some(ts),
false => None,
}
})
} else {
None
};
@@ -1282,7 +1282,16 @@ SELECT id, rfc724_mid, pre_rfc724_mid, timestamp, ?, 1 FROM msgs WHERE chat_id=?
sort_timestamp = last_msg_time;
}
Ok(sort_timestamp)
if let Some(join_timestamp) = self.join_timestamp(context).await? {
// If we are the member of the chat, don't add messages
// before the timestamp of us joining it.
// This is needed to avoid sorting "Member added"
// or automatically sent bot welcome messages
// above SecureJoin system messages.
Ok(std::cmp::max(sort_timestamp, join_timestamp))
} else {
Ok(sort_timestamp)
}
}
}
@@ -1742,7 +1751,6 @@ impl Chat {
///
/// If `update_msg_id` is set, that record is reused;
/// if `update_msg_id` is None, a new record is created.
#[expect(clippy::arithmetic_side_effects)]
async fn prepare_msg_raw(
&mut self,
context: &Context,
@@ -2618,7 +2626,7 @@ pub async fn send_msg(context: &Context, chat_id: ChatId, msg: &mut Message) ->
if msg.state != MessageState::Undefined && msg.state != MessageState::OutPreparing {
msg.param.remove(Param::GuaranteeE2ee);
msg.param.remove(Param::ForcePlaintext);
msg.update_param(context).await?;
// create_send_msg_jobs() will update `param` in the db.
}
// protect all system messages against RTLO attacks
@@ -2723,7 +2731,19 @@ async fn prepare_send_msg(
None
};
// ... then change the MessageState in the message object
if matches!(
msg.state,
MessageState::Undefined | MessageState::OutPreparing
)
// Legacy SecureJoin "v*-request" messages are unencrypted.
&& msg.param.get_cmd() != SystemMessage::SecurejoinMessage
&& chat.is_encrypted(context).await?
{
msg.param.set_int(Param::GuaranteeE2ee, 1);
if !msg.id.is_unset() {
msg.update_param(context).await?;
}
}
msg.state = MessageState::OutPending;
msg.timestamp_sort = create_smeared_timestamp(context);
@@ -2769,15 +2789,13 @@ async fn render_mime_message_and_pre_message(
let mut mimefactory_post_msg = mimefactory.clone();
mimefactory_post_msg.set_as_post_message();
let rendered_msg = mimefactory_post_msg
.render(context)
let rendered_msg = Box::pin(mimefactory_post_msg.render(context))
.await
.context("Failed to render post-message")?;
let mut mimefactory_pre_msg = mimefactory;
mimefactory_pre_msg.set_as_pre_message_for(&rendered_msg);
let rendered_pre_msg = mimefactory_pre_msg
.render(context)
let rendered_pre_msg = Box::pin(mimefactory_pre_msg.render(context))
.await
.context("pre-message failed to render")?;
@@ -2792,7 +2810,7 @@ async fn render_mime_message_and_pre_message(
Ok((Some(rendered_pre_msg), rendered_msg))
} else {
Ok((None, mimefactory.render(context).await?))
Ok((None, Box::pin(mimefactory.render(context)).await?))
}
}
@@ -2921,11 +2939,24 @@ pub(crate) async fn create_send_msg_jobs(context: &Context, msg: &mut Message) -
msg.param.remove(Param::GuaranteeE2ee);
}
msg.subject.clone_from(&rendered_msg.subject);
// Sort the message to the bottom. Employ `msgs_index7` to compute `timestamp`.
context
.sql
.execute(
"UPDATE msgs SET pre_rfc724_mid=?, subject=?, param=? WHERE id=?",
"
UPDATE msgs SET
timestamp=(
SELECT MAX(timestamp) FROM msgs WHERE
-- From `InFresh` to `OutMdnRcvd` inclusive except `OutDraft`.
state IN(10,13,16,18,20,24,26,28) AND
hidden IN(0,1) AND
chat_id=?
),
pre_rfc724_mid=?, subject=?, param=?
WHERE id=?
",
(
msg.chat_id,
&msg.pre_rfc724_mid,
&msg.subject,
msg.param.to_string(),
@@ -2990,7 +3021,6 @@ pub async fn send_text_msg(
}
/// Sends chat members a request to edit the given message's text.
#[expect(clippy::arithmetic_side_effects)]
pub async fn send_edit_request(context: &Context, msg_id: MsgId, new_text: String) -> Result<()> {
let mut original_msg = Message::load_from_db(context, msg_id).await?;
ensure!(
@@ -3622,7 +3652,7 @@ pub(crate) async fn create_group_ex(
// Add "Messages in this chat use classic email and are not encrypted." message.
stock_str::chat_unencrypted_explanation(context)
};
add_info_msg(context, chat_id, &text).await?;
chat_id.add_start_info_message(context, &text).await?;
}
if let (true, true) = (sync.into(), !grpid.is_empty()) {
let id = SyncId::Grpid(grpid);
@@ -3753,7 +3783,7 @@ pub(crate) async fn update_chat_contacts_table(
context: &Context,
timestamp: i64,
id: ChatId,
contacts: &HashSet<ContactId>,
contacts: &BTreeSet<ContactId>,
) -> Result<()> {
context
.sql
@@ -4121,61 +4151,56 @@ pub async fn remove_contact_from_chat(
delete_broadcast_secret(context, chat_id).await?;
}
if matches!(
chat.typ,
Chattype::Group | Chattype::OutBroadcast | Chattype::InBroadcast
) {
if !chat.is_self_in_chat(context).await? {
let err_msg = format!(
"Cannot remove contact {contact_id} from chat {chat_id}: self not in group."
);
context.emit_event(EventType::ErrorSelfNotInGroup(err_msg.clone()));
bail!("{err_msg}");
} else {
let mut sync = Nosync;
ensure!(
matches!(
chat.typ,
Chattype::Group | Chattype::OutBroadcast | Chattype::InBroadcast
),
"Cannot remove members from non-group chats."
);
if chat.is_promoted() && chat.typ != Chattype::OutBroadcast {
remove_from_chat_contacts_table(context, chat_id, contact_id).await?;
} else {
remove_from_chat_contacts_table_without_trace(context, chat_id, contact_id).await?;
}
if !chat.is_self_in_chat(context).await? {
let err_msg =
format!("Cannot remove contact {contact_id} from chat {chat_id}: self not in group.");
context.emit_event(EventType::ErrorSelfNotInGroup(err_msg.clone()));
bail!("{err_msg}");
}
// We do not return an error if the contact does not exist in the database.
// This allows to delete dangling references to deleted contacts
// in case of the database becoming inconsistent due to a bug.
if let Some(contact) = Contact::get_by_id_optional(context, contact_id).await? {
if chat.is_promoted() {
let addr = contact.get_addr();
let fingerprint = contact.fingerprint().map(|f| f.hex());
let mut sync = Nosync;
let res = send_member_removal_msg(
context,
&chat,
contact_id,
addr,
fingerprint.as_deref(),
)
if chat.is_promoted() && chat.typ != Chattype::OutBroadcast {
remove_from_chat_contacts_table(context, chat_id, contact_id).await?;
} else {
remove_from_chat_contacts_table_without_trace(context, chat_id, contact_id).await?;
}
// We do not return an error if the contact does not exist in the database.
// This allows to delete dangling references to deleted contacts
// in case of the database becoming inconsistent due to a bug.
if let Some(contact) = Contact::get_by_id_optional(context, contact_id).await? {
if chat.is_promoted() {
let addr = contact.get_addr();
let fingerprint = contact.fingerprint().map(|f| f.hex());
let res =
send_member_removal_msg(context, &chat, contact_id, addr, fingerprint.as_deref())
.await;
if contact_id == ContactId::SELF {
res?;
} else if let Err(e) = res {
warn!(
context,
"remove_contact_from_chat({chat_id}, {contact_id}): send_msg() failed: {e:#}."
);
}
} else {
sync = Sync;
}
}
context.emit_event(EventType::ChatModified(chat_id));
if sync.into() {
chat.sync_contacts(context).await.log_err(context).ok();
if contact_id == ContactId::SELF {
res?;
} else if let Err(e) = res {
warn!(
context,
"remove_contact_from_chat({chat_id}, {contact_id}): send_msg() failed: {e:#}."
);
}
} else {
sync = Sync;
}
} else {
bail!("Cannot remove members from non-group chats.");
}
context.emit_event(EventType::ChatModified(chat_id));
if sync.into() {
chat.sync_contacts(context).await.log_err(context).ok();
}
Ok(())
@@ -4584,7 +4609,6 @@ pub async fn save_msgs(context: &Context, msg_ids: &[MsgId]) -> Result<()> {
/// the copy contains a reference to the original message
/// as well as to the original chat in case the original message gets deleted.
/// Returns data needed to add a `SaveMessage` sync item.
#[expect(clippy::arithmetic_side_effects)]
pub(crate) async fn save_copy_in_self_talk(
context: &Context,
src_msg_id: MsgId,
@@ -4938,15 +4962,8 @@ pub(crate) async fn add_info_msg_with_cmd(
ts
} else {
let sort_to_bottom = true;
let (received, incoming) = (false, false);
chat_id
.calc_sort_timestamp(
context,
smeared_time(context),
sort_to_bottom,
received,
incoming,
)
.calc_sort_timestamp(context, smeared_time(context), sort_to_bottom)
.await?
};
@@ -5024,7 +5041,7 @@ async fn set_contacts_by_addrs(context: &Context, id: ChatId, addrs: &[String])
chat.typ == Chattype::OutBroadcast,
"{id} is not a broadcast list",
);
let mut contacts = HashSet::new();
let mut contacts = BTreeSet::new();
for addr in addrs {
let contact_addr = ContactAddress::new(addr)?;
let contact = Contact::add_or_lookup(context, "", &contact_addr, Origin::Hidden)
@@ -5032,7 +5049,7 @@ async fn set_contacts_by_addrs(context: &Context, id: ChatId, addrs: &[String])
.0;
contacts.insert(contact);
}
let contacts_old = HashSet::<ContactId>::from_iter(get_chat_contacts(context, id).await?);
let contacts_old = BTreeSet::<ContactId>::from_iter(get_chat_contacts(context, id).await?);
if contacts == contacts_old {
return Ok(());
}
+146 -13
View File
@@ -7,7 +7,7 @@ use crate::constants::{DC_GCL_ARCHIVED_ONLY, DC_GCL_NO_SPECIALS};
use crate::ephemeral::Timer;
use crate::headerdef::HeaderDef;
use crate::imex::{ImexMode, has_backup, imex};
use crate::message::{MessengerMessage, delete_msgs};
use crate::message::{Message, MessengerMessage, delete_msgs};
use crate::mimeparser::{self, MimeMessage};
use crate::receive_imf::receive_imf;
use crate::securejoin::{get_securejoin_qr, join_securejoin};
@@ -2792,7 +2792,7 @@ async fn test_broadcast_members_cant_see_each_other() -> Result<()> {
assert_eq!(parsed.decoded_data_contains("bob@example.net"), false);
let parsed_by_bob = bob.parse_msg(&vc_pubkey).await;
assert!(parsed_by_bob.decrypting_failed);
assert!(parsed_by_bob.decryption_error.is_some());
charlie.recv_msg_trash(&vc_pubkey).await;
}
@@ -2821,7 +2821,7 @@ async fn test_broadcast_members_cant_see_each_other() -> Result<()> {
assert_eq!(parsed.decoded_data_contains("bob@example.net"), false);
let parsed_by_bob = bob.parse_msg(&member_added).await;
assert!(parsed_by_bob.decrypting_failed);
assert!(parsed_by_bob.decryption_error.is_some());
let rcvd = charlie.recv_msg(&member_added).await;
assert_eq!(rcvd.param.get_cmd(), SystemMessage::MemberAddedToGroup);
@@ -2836,7 +2836,7 @@ async fn test_broadcast_members_cant_see_each_other() -> Result<()> {
assert_eq!(parsed.decoded_data_contains("bob@example.net"), false);
let parsed_by_bob = bob.parse_msg(&hi_msg).await;
assert_eq!(parsed_by_bob.decrypting_failed, false);
assert!(parsed_by_bob.decryption_error.is_none());
}
tcm.section("Alice removes Charlie. Bob must not see it.");
@@ -2853,7 +2853,7 @@ async fn test_broadcast_members_cant_see_each_other() -> Result<()> {
assert_eq!(parsed.decoded_data_contains("bob@example.net"), false);
let parsed_by_bob = bob.parse_msg(&member_removed).await;
assert!(parsed_by_bob.decrypting_failed);
assert!(parsed_by_bob.decryption_error.is_some());
let rcvd = charlie.recv_msg(&member_removed).await;
assert_eq!(rcvd.param.get_cmd(), SystemMessage::MemberRemovedFromGroup);
@@ -3768,14 +3768,7 @@ async fn test_leave_broadcast_multidevice() -> Result<()> {
// The contact should be marked as verified.
check_direct_chat_is_hidden_and_contact_is_verified(alice, bob0).await;
check_direct_chat_is_hidden_and_contact_is_verified(bob0, alice).await;
// TODO: There is a known bug in `observe_securejoin_on_other_device()`:
// When Bob joins a group or broadcast with his first device,
// then a chat with Alice will pop up on his second device.
// When it's fixed, the 2 following lines can be replaced with
// `check_direct_chat_is_hidden_and_contact_is_verified(bob1, alice).await;`
let bob1_alice_contact = bob1.add_or_lookup_contact_no_key(alice).await;
assert!(bob1_alice_contact.is_verified(bob1).await.unwrap());
check_direct_chat_is_hidden_and_contact_is_verified(bob1, alice).await;
tcm.section("Alice sends first message to broadcast.");
let sent_msg = alice.send_text(alice_chat_id, "Hello!").await;
@@ -5034,6 +5027,31 @@ async fn test_do_not_overwrite_draft() -> Result<()> {
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_outgoing_msg_after_another_from_future() -> Result<()> {
let mut tcm = TestContextManager::new();
let t = &tcm.alice().await;
let chat_id = t.get_self_chat().await.id;
// Simulate sending a message with clock set to the future.
SystemTime::shift(Duration::from_secs(3600));
let msg_id = send_text_msg(t, chat_id, "test".to_string()).await?;
SystemTime::shift_back(Duration::from_secs(3600));
let timestamp_sent: i64 = t
.sql
.query_get_value("SELECT timestamp_sent FROM msgs WHERE id=?", (msg_id,))
.await?
.unwrap();
// Let's have a check here that locally sent messages have zero `timestamp_sent`, it can be a
// useful invariant.
assert_eq!(timestamp_sent, 0);
let msg_id = send_text_msg(t, chat_id, "Fixed my clock".to_string()).await?;
assert_eq!(t.get_last_msg_in(chat_id).await.id, msg_id);
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_info_contact_id() -> Result<()> {
let mut tcm = TestContextManager::new();
@@ -6112,3 +6130,118 @@ async fn test_leftgrps() -> Result<()> {
Ok(())
}
/// Tests that if the message arrives late,
/// it can still be sorted above the last seen message.
///
/// Versions 2.47 and below always sorted incoming messages
/// after the last seen message, but with
/// the introduction of multi-relay it is possible
/// that some messages arrive only to some relays
/// and are fetched after the already arrived seen message.
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_late_message_above_seen() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
let bob = &tcm.bob().await;
let charlie = &tcm.charlie().await;
let alice_chat_id = alice
.create_group_with_members("Group", &[bob, charlie])
.await;
let alice_sent = alice.send_text(alice_chat_id, "Hello everyone!").await;
let bob_chat_id = bob.recv_msg(&alice_sent).await.chat_id;
bob_chat_id.accept(bob).await?;
let charlie_chat_id = charlie.recv_msg(&alice_sent).await.chat_id;
charlie_chat_id.accept(charlie).await?;
// Bob sends a message, but the message is delayed.
let bob_sent = bob.send_text(bob_chat_id, "Hello from Bob!").await;
SystemTime::shift(Duration::from_secs(1000));
let charlie_sent = charlie
.send_text(charlie_chat_id, "Hello from Charlie!")
.await;
// Alice immediately receives a message from Charlie and reads it.
let alice_received_from_charlie = alice.recv_msg(&charlie_sent).await;
assert_eq!(
alice_received_from_charlie.get_text(),
"Hello from Charlie!"
);
message::markseen_msgs(alice, vec![alice_received_from_charlie.id]).await?;
// Bob message arrives later, it should be above the message from Charlie.
let alice_received_from_bob = alice.recv_msg(&bob_sent).await;
assert_eq!(alice_received_from_bob.get_text(), "Hello from Bob!");
// The last message in the chat is still from Charlie.
let last_msg = alice.get_last_msg_in(alice_chat_id).await;
assert_eq!(last_msg.get_text(), "Hello from Charlie!");
Ok(())
}
/// Tests that start message for unpromoted groups sticks to the top of the chat.
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_unpromoted_group_start_message() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
let bob = &tcm.bob().await;
// Start messages are disabled for test context by default,
// but this test is specifically about start messages.
alice.set_config(Config::SkipStartMessages, None).await?;
// Shift the clock forward, so we can rewind it back later.
SystemTime::shift(Duration::from_secs(3600));
// Alice creates unpromoted group with Bob.
let chat_id = create_group(alice, "Group").await?;
let bob_id = alice.add_or_lookup_contact_id(bob).await;
add_contact_to_chat(alice, chat_id, bob_id).await?;
let [
ChatItem::Message {
msg_id: e2ee_msg_id,
},
ChatItem::Message {
msg_id: info_msg_id,
},
] = get_chat_msgs(alice, chat_id).await?[..]
else {
panic!("Expected two messages in the chat");
};
let msg = Message::load_from_db(alice, e2ee_msg_id).await?;
assert_eq!(msg.text, "Messages are end-to-end encrypted.");
let msg = Message::load_from_db(alice, info_msg_id).await?;
assert_eq!(
msg.text,
"Others will only see this group after you sent a first message."
);
// Alice rewinds the clock.
SystemTime::shift_back(Duration::from_secs(3600));
let text_msg_id = send_text_msg(alice, chat_id, "Hello".to_string()).await?;
let [
ChatItem::Message {
msg_id: e2ee_msg_id2,
},
ChatItem::Message {
msg_id: info_msg_id2,
},
ChatItem::Message {
msg_id: text_msg_id2,
},
] = get_chat_msgs(alice, chat_id).await?[..]
else {
panic!("Expected three messages in the chat");
};
assert_eq!(e2ee_msg_id2, e2ee_msg_id);
assert_eq!(info_msg_id2, info_msg_id);
assert_eq!(text_msg_id2, text_msg_id);
Ok(())
}
+5 -5
View File
@@ -142,7 +142,7 @@ impl Chatlist {
AND c.blocked!=1
AND c.id IN(SELECT chat_id FROM chats_contacts WHERE contact_id=?2 AND add_timestamp >= remove_timestamp)
GROUP BY c.id
ORDER BY c.archived=?3 DESC, IFNULL(m.timestamp,c.created_timestamp) DESC, m.id DESC;",
ORDER BY c.archived=?3 DESC, IFNULL(NULLIF(m.timestamp,0),c.created_timestamp) DESC, m.id DESC;",
(MessageState::OutDraft, query_contact_id, ChatVisibility::Pinned),
process_row,
).await?
@@ -168,7 +168,7 @@ impl Chatlist {
AND c.blocked!=1
AND c.archived=1
GROUP BY c.id
ORDER BY IFNULL(m.timestamp,c.created_timestamp) DESC, m.id DESC;",
ORDER BY IFNULL(NULLIF(m.timestamp,0),c.created_timestamp) DESC, m.id DESC;",
(MessageState::OutDraft,),
process_row,
)
@@ -204,7 +204,7 @@ impl Chatlist {
AND IFNULL(c.name_normalized,c.name) LIKE ?3
AND (NOT ?4 OR EXISTS (SELECT 1 FROM msgs m WHERE m.chat_id = c.id AND m.state == ?5 AND hidden=0))
GROUP BY c.id
ORDER BY IFNULL(m.timestamp,c.created_timestamp) DESC, m.id DESC;",
ORDER BY IFNULL(NULLIF(m.timestamp,0),c.created_timestamp) DESC, m.id DESC;",
(MessageState::OutDraft, skip_id, str_like_cmd, only_unread, MessageState::InFresh),
process_row,
)
@@ -253,7 +253,7 @@ impl Chatlist {
AND NOT c.archived=?
AND (c.type!=? OR c.id IN(SELECT chat_id FROM chats_contacts WHERE contact_id=? AND add_timestamp >= remove_timestamp))
GROUP BY c.id
ORDER BY c.id=? DESC, c.archived=? DESC, IFNULL(m.timestamp,c.created_timestamp) DESC, m.id DESC;",
ORDER BY c.id=? DESC, c.archived=? DESC, IFNULL(NULLIF(m.timestamp,0),c.created_timestamp) DESC, m.id DESC;",
(
MessageState::OutDraft, skip_id, ChatVisibility::Archived,
Chattype::Group, ContactId::SELF,
@@ -279,7 +279,7 @@ impl Chatlist {
AND (c.blocked=0 OR c.blocked=2)
AND NOT c.archived=?
GROUP BY c.id
ORDER BY c.id=0 DESC, c.archived=? DESC, IFNULL(m.timestamp,c.created_timestamp) DESC, m.id DESC;",
ORDER BY c.id=0 DESC, c.archived=? DESC, IFNULL(NULLIF(m.timestamp,0),c.created_timestamp) DESC, m.id DESC;",
(MessageState::OutDraft, skip_id, ChatVisibility::Archived, ChatVisibility::Pinned),
process_row,
).await?
+1 -40
View File
@@ -155,18 +155,6 @@ pub enum Config {
#[strum(props(default = "1"))]
MdnsEnabled,
/// True if chat messages should be moved to a separate folder. Auto-sent messages like sync
/// ones are moved there anyway.
#[strum(props(default = "1"))]
MvboxMove,
/// Watch for new messages in the "Mvbox" (aka DeltaChat folder) only.
///
/// This will not entirely disable other folders, e.g. the spam folder will also still
/// be watched for new messages.
#[strum(props(default = "0"))]
OnlyFetchMvbox,
/// Whether to show classic emails or only chat messages.
#[strum(props(default = "2"))] // also change ShowEmails.default() on changes
ShowEmails,
@@ -268,9 +256,6 @@ pub enum Config {
/// Configured folder for incoming messages.
ConfiguredInboxFolder,
/// Configured folder for chat messages.
ConfiguredMvboxFolder,
/// Unix timestamp of the last successful configuration.
ConfiguredTimestamp,
@@ -467,7 +452,6 @@ impl Config {
self,
Self::Displayname
| Self::MdnsEnabled
| Self::MvboxMove
| Self::ShowEmails
| Self::Selfavatar
| Self::Selfstatus,
@@ -476,10 +460,7 @@ impl Config {
/// Whether the config option needs an IO scheduler restart to take effect.
pub(crate) fn needs_io_restart(&self) -> bool {
matches!(
self,
Config::MvboxMove | Config::OnlyFetchMvbox | Config::ConfiguredAddr
)
matches!(self, Config::ConfiguredAddr)
}
}
@@ -594,13 +575,6 @@ impl Context {
.is_some_and(|x| x != 0))
}
/// Returns true if movebox ("DeltaChat" folder) should be watched.
pub(crate) async fn should_watch_mvbox(&self) -> Result<bool> {
Ok(self.get_config_bool(Config::MvboxMove).await?
|| self.get_config_bool(Config::OnlyFetchMvbox).await?
|| !self.get_config_bool(Config::IsChatmail).await?)
}
/// Returns true if sync messages should be sent.
pub(crate) async fn should_send_sync_msgs(&self) -> Result<bool> {
Ok(self.get_config_bool(Config::SyncMsgs).await?
@@ -682,8 +656,6 @@ impl Context {
| Config::ProxyEnabled
| Config::BccSelf
| Config::MdnsEnabled
| Config::MvboxMove
| Config::OnlyFetchMvbox
| Config::Configured
| Config::Bot
| Config::NotifyAboutWrongPw
@@ -706,11 +678,6 @@ impl Context {
pub async fn set_config(&self, key: Config, value: Option<&str>) -> Result<()> {
Self::check_config(key, value)?;
let n_transports = self.count_transports().await?;
if n_transports > 1 && matches!(key, Config::MvboxMove | Config::OnlyFetchMvbox) {
bail!("Cannot reconfigure {key} when multiple transports are configured");
}
let _pause = match key.needs_io_restart() {
true => self.scheduler.pause(self).await?,
_ => Default::default(),
@@ -789,12 +756,6 @@ impl Context {
.set_raw_config(key.as_ref(), value.map(|s| s.to_lowercase()).as_deref())
.await?;
}
Config::MvboxMove => {
self.sql.set_raw_config(key.as_ref(), value).await?;
self.sql
.set_raw_config(constants::DC_FOLDERS_CONFIGURED_KEY, None)
.await?;
}
Config::ConfiguredAddr => {
let Some(addr) = value else {
bail!("Cannot unset configured_addr");
+4 -4
View File
@@ -196,11 +196,11 @@ async fn test_sync() -> Result<()> {
sync(&alice0, &alice1).await;
assert_eq!(alice1.get_config_bool(Config::MdnsEnabled).await?, false);
for key in [Config::ShowEmails, Config::MvboxMove] {
let val = alice0.get_config_bool(key).await?;
alice0.set_config_bool(key, !val).await?;
{
let val = alice0.get_config_bool(Config::ShowEmails).await?;
alice0.set_config_bool(Config::ShowEmails, !val).await?;
sync(&alice0, &alice1).await;
assert_eq!(alice1.get_config_bool(key).await?, !val);
assert_eq!(alice1.get_config_bool(Config::ShowEmails).await?, !val);
}
// `Config::SyncMsgs` mustn't be synced.
+10 -32
View File
@@ -243,11 +243,6 @@ impl Context {
Ok((id, add_timestamp))
},
)?;
transaction.execute("DELETE FROM imap WHERE transport_id=?", (transport_id,))?;
transaction.execute(
"DELETE FROM imap_sync WHERE transport_id=?",
(transport_id,),
)?;
// Removal timestamp should not be lower than addition timestamp
// to be accepted by other devices when synced.
@@ -266,6 +261,7 @@ impl Context {
.await?;
send_sync_transports(self).await?;
self.quota.write().await.remove(&removed_transport_id);
self.restart_io_if_running().await;
Ok(())
}
@@ -320,31 +316,16 @@ impl Context {
(&param.addr,),
)
.await?
{
// Should be checked before `MvboxMove` because the latter makes no sense in presense of
// `OnlyFetchMvbox` and even grayed out in the UIs in this case.
if self.get_config(Config::OnlyFetchMvbox).await?.as_deref() != Some("0") {
bail!(
"To use additional relays, disable the legacy option \"Settings / Advanced / Only Fetch from DeltaChat Folder\"."
);
}
if self.get_config(Config::MvboxMove).await?.as_deref() != Some("0") {
bail!(
"To use additional relays, disable the legacy option \"Settings / Advanced / Move automatically to DeltaChat Folder\"."
);
}
if self
&& self
.sql
.count("SELECT COUNT(*) FROM transports", ())
.await?
>= MAX_TRANSPORT_RELAYS
{
bail!(
"You have reached the maximum number of relays ({}).",
MAX_TRANSPORT_RELAYS
)
}
{
bail!(
"You have reached the maximum number of relays ({}).",
MAX_TRANSPORT_RELAYS
)
}
let provider = match configure(self, param).await {
@@ -557,6 +538,7 @@ async fn get_configured_param(
.collect(),
imap_user: param.imap.user.clone(),
imap_password: param.imap.password.clone(),
imap_folder: Some(param.imap.folder.clone()).filter(|folder| !folder.is_empty()),
smtp: servers
.iter()
.filter_map(|params| {
@@ -649,10 +631,6 @@ async fn configure(ctx: &Context, param: &EnteredLoginParam) -> Result<Option<&'
progress!(ctx, 900);
let is_configured = ctx.is_configured().await?;
if !is_configured {
ctx.sql.set_raw_config("mvbox_move", Some("0")).await?;
ctx.sql.set_raw_config("only_fetch_mvbox", None).await?;
}
if !ctx.get_config_bool(Config::FixIsChatmail).await? {
if imap_session.is_chatmail() {
ctx.sql.set_raw_config("is_chatmail", Some("1")).await?;
@@ -816,7 +794,7 @@ pub enum Error {
mod tests {
use super::*;
use crate::config::Config;
use crate::login_param::EnteredServerLoginParam;
use crate::login_param::EnteredImapLoginParam;
use crate::test_utils::TestContext;
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
@@ -835,7 +813,7 @@ mod tests {
let entered_param = EnteredLoginParam {
addr: "alice@example.org".to_string(),
imap: EnteredServerLoginParam {
imap: EnteredImapLoginParam {
user: "alice@example.net".to_string(),
password: "foobar".to_string(),
..Default::default()
+1 -6
View File
@@ -199,7 +199,7 @@ pub const WORSE_IMAGE_BYTES: usize = 130_000;
// max. width/height and bytes of an avatar
pub(crate) const BALANCED_AVATAR_SIZE: u32 = 512;
pub(crate) const BALANCED_AVATAR_BYTES: usize = 60_000;
pub(crate) const WORSE_AVATAR_SIZE: u32 = 128;
pub(crate) const WORSE_AVATAR_SIZE: u32 = 256;
pub(crate) const WORSE_AVATAR_BYTES: usize = 20_000; // this also fits to Outlook servers don't allowing headers larger than 32k.
// max. width/height of images scaled down because of being too huge
@@ -210,11 +210,6 @@ pub const WORSE_IMAGE_SIZE: u32 = 640;
/// usage by UIs.
pub const MAX_RCVD_IMAGE_PIXELS: u32 = 50_000_000;
// Key for the folder configuration version (see below).
pub(crate) const DC_FOLDERS_CONFIGURED_KEY: &str = "folders_configured";
// this value can be increased if the folder configuration is changed and must be redone on next program start
pub(crate) const DC_FOLDERS_CONFIGURED_VERSION: i32 = 5;
// If more recipients are needed in SMTP's `RCPT TO:` header, the recipient list is split into
// chunks. This does not affect MIME's `To:` header. Can be overwritten by setting
// `max_smtp_rcpt_to` in the provider db.
+3 -2
View File
@@ -1182,7 +1182,9 @@ VALUES (?, ?, ?, ?, ?, ?)
let mut ret = Vec::new();
let flag_add_self = (listflags & constants::DC_GCL_ADD_SELF) != 0;
let flag_address = (listflags & constants::DC_GCL_ADDRESS) != 0;
let minimal_origin = if context.get_config_bool(Config::Bot).await? {
let minimal_origin = if context.get_config_bool(Config::Bot).await?
|| query.is_some_and(may_be_valid_addr)
{
Origin::Unknown
} else {
Origin::IncomingReplyTo
@@ -2066,7 +2068,6 @@ pub(crate) async fn mark_contact_id_as_verified(
Ok(())
}
#[expect(clippy::arithmetic_side_effects)]
fn cat_fingerprint(ret: &mut String, name: &str, addr: &str, fingerprint: &str) {
*ret += &format!("\n\n{name} ({addr}):\n{fingerprint}");
}
+51 -2
View File
@@ -4,6 +4,7 @@ use super::*;
use crate::chat::{Chat, get_chat_contacts, send_text_msg};
use crate::chatlist::Chatlist;
use crate::receive_imf::receive_imf;
use crate::securejoin::get_securejoin_qr;
use crate::test_utils::{self, TestContext, TestContextManager, TimeShiftFalsePositiveNote, sync};
#[test]
@@ -154,6 +155,50 @@ async fn test_get_contacts() -> Result<()> {
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_search_contacts_from_group() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
let bob = &tcm.bob().await;
let fiona = &tcm.fiona().await;
let alice_chat_id = chat::create_group(alice, "").await?;
let qr = get_securejoin_qr(alice, Some(alice_chat_id)).await?;
let bob_chat_id = tcm.exec_securejoin_qr(bob, alice, &qr).await;
tcm.exec_securejoin_qr(fiona, alice, &qr).await;
// Workaround for "member added" for fiona not sent to bob.
let gossip_period = alice.get_config_int(Config::GossipPeriod).await?;
SystemTime::shift(Duration::from_secs(gossip_period.try_into()?));
send_text_msg(alice, alice_chat_id, "hello".to_string()).await?;
let sent_msg = alice.pop_sent_msg().await;
bob.recv_msg(&sent_msg).await;
fiona.recv_msg(&sent_msg).await;
let contacts = Contact::get_all(bob, 0, None).await?;
let bob_alice_id = bob.add_or_lookup_contact_id(alice).await;
assert_eq!(contacts.len(), 1);
assert_eq!(contacts[0], bob_alice_id);
let contacts = Contact::get_all(fiona, 0, None).await?;
let fiona_alice_id = fiona.add_or_lookup_contact_id(alice).await;
assert_eq!(contacts.len(), 1);
assert_eq!(contacts[0], fiona_alice_id);
// Sending to the group adds new members to the contact list.
send_text_msg(bob, bob_chat_id, "hello".to_string()).await?;
fiona.recv_msg(&bob.pop_sent_msg().await).await;
let contacts = Contact::get_all(bob, 0, None).await?;
let bob_fiona_id = bob.add_or_lookup_contact_id(fiona).await;
assert_eq!(contacts.len(), 2);
assert_eq!(contacts[0], bob_alice_id);
assert_eq!(contacts[1], bob_fiona_id);
let contacts = Contact::get_all(fiona, 0, None).await?;
assert_eq!(contacts.len(), 1);
assert_eq!(contacts[0], fiona_alice_id);
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_is_self_addr() -> Result<()> {
let t = TestContext::new().await;
@@ -420,12 +465,16 @@ async fn test_delete() -> Result<()> {
Contact::delete(&alice, contact_id).await?;
let contact = Contact::get_by_id(&alice, contact_id).await?;
assert_eq!(contact.origin, Origin::Hidden);
// Hidden contacts are found when searching by email address
assert_eq!(
Contact::get_all(&alice, 0, Some("bob@example.net"))
.await?
.len(),
0
1
);
// Hidden contacts are not found by a non-address query
assert_eq!(Contact::get_all(&alice, 0, Some("bob")).await?.len(), 0);
// Delete chat.
chat.get_id().delete(&alice).await?;
@@ -483,7 +532,7 @@ async fn test_delete_and_recreate_contact() -> Result<()> {
Contact::get_all(&t, 0, Some("bob@example.net"))
.await?
.len(),
0
1
);
let contact_id3 = t.add_or_lookup_contact_id(&bob).await;
+32 -48
View File
@@ -20,16 +20,16 @@ use crate::constants::{self, DC_BACKGROUND_FETCH_QUOTA_CHECK_RATELIMIT, DC_VERSI
use crate::contact::{Contact, ContactId};
use crate::debug_logging::DebugLogging;
use crate::events::{Event, EventEmitter, EventType, Events};
use crate::imap::{FolderMeaning, Imap, ServerMetadata};
use crate::imap::{Imap, ServerMetadata};
use crate::key::self_fingerprint;
use crate::log::warn;
use crate::logged_debug_assert;
use crate::message::{self, MessageState, MsgId};
use crate::net::tls::TlsSessionStore;
use crate::net::tls::{SpkiHashStore, TlsSessionStore};
use crate::peer_channels::Iroh;
use crate::push::PushSubscriber;
use crate::quota::QuotaInfo;
use crate::scheduler::{ConnectivityStore, SchedulerState, convert_folder_meaning};
use crate::scheduler::{ConnectivityStore, SchedulerState};
use crate::sql::Sql;
use crate::stock_str::StockStrings;
use crate::timesmearing::SmearedTimestamp;
@@ -308,6 +308,13 @@ pub struct InnerContext {
/// TLS session resumption cache.
pub(crate) tls_session_store: TlsSessionStore,
/// Store for TLS SPKI hashes.
///
/// Used to remember public keys
/// of TLS certificates to accept them
/// even after they expire.
pub(crate) spki_hash_store: SpkiHashStore,
/// Iroh for realtime peer channels.
pub(crate) iroh: Arc<RwLock<Option<Iroh>>>,
@@ -511,6 +518,7 @@ impl Context {
push_subscriber,
push_subscribed: AtomicBool::new(false),
tls_session_store: TlsSessionStore::new(),
spki_hash_store: SpkiHashStore::new(),
iroh: Arc::new(RwLock::new(None)),
self_fingerprint: OnceLock::new(),
self_public_key: Mutex::new(None),
@@ -623,17 +631,10 @@ impl Context {
let mut session = connection.prepare(self).await?;
// Fetch IMAP folders.
// Inbox is fetched before Mvbox because fetching from Inbox
// may result in moving some messages to Mvbox.
for folder_meaning in [FolderMeaning::Inbox, FolderMeaning::Mvbox] {
if let Some((_folder_config, watch_folder)) =
convert_folder_meaning(self, folder_meaning).await?
{
connection
.fetch_move_delete(self, &mut session, &watch_folder, folder_meaning)
.await?;
}
}
let folder = connection.folder.clone();
connection
.fetch_move_delete(self, &mut session, &folder)
.await?;
// Update quota (to send warning if full) - but only check it once in a while.
// note: For now this only checks quota of primary transport,
@@ -644,7 +645,7 @@ impl Context {
DC_BACKGROUND_FETCH_QUOTA_CHECK_RATELIMIT,
)
.await
&& let Err(err) = self.update_recent_quota(&mut session).await
&& let Err(err) = self.update_recent_quota(&mut session, &folder).await
{
warn!(self, "Failed to update quota: {err:#}.");
}
@@ -884,23 +885,6 @@ impl Context {
Err(err) => format!("<key failure: {err}>"),
};
let mvbox_move = self.get_config_int(Config::MvboxMove).await?;
let only_fetch_mvbox = self.get_config_int(Config::OnlyFetchMvbox).await?;
let folders_configured = self
.sql
.get_raw_config_int(constants::DC_FOLDERS_CONFIGURED_KEY)
.await?
.unwrap_or_default();
let configured_inbox_folder = self
.get_config(Config::ConfiguredInboxFolder)
.await?
.unwrap_or_else(|| "<unset>".to_string());
let configured_mvbox_folder = self
.get_config(Config::ConfiguredMvboxFolder)
.await?
.unwrap_or_else(|| "<unset>".to_string());
let mut res = get_info();
// insert values
@@ -976,14 +960,6 @@ impl Context {
.await?
.to_string(),
);
res.insert("mvbox_move", mvbox_move.to_string());
res.insert("only_fetch_mvbox", only_fetch_mvbox.to_string());
res.insert(
constants::DC_FOLDERS_CONFIGURED_KEY,
folders_configured.to_string(),
);
res.insert("configured_inbox_folder", configured_inbox_folder);
res.insert("configured_mvbox_folder", configured_mvbox_folder);
res.insert("mdns_enabled", mdns_enabled.to_string());
res.insert("bcc_self", bcc_self.to_string());
res.insert("sync_msgs", sync_msgs.to_string());
@@ -1142,10 +1118,17 @@ ORDER BY m.timestamp DESC,m.id DESC",
Ok(list)
}
/// Returns a list of messages with database ID higher than requested.
/// (deprecated) Returns a list of messages with database ID higher than requested.
///
/// Blocked contacts and chats are excluded,
/// but self-sent messages and contact requests are included in the results.
///
/// Deprecated 2026-04: This returns the message's id as soon as the first part arrives,
/// even if it is not fully downloaded yet.
/// The bot needs to wait for the message to be fully downloaded.
/// Since this is usually not the desired behavior,
/// bots should instead use the [`EventType::IncomingMsg`]
/// event for getting notified about new messages.
pub async fn get_next_msgs(&self) -> Result<Vec<MsgId>> {
let last_msg_id = match self.get_config(Config::LastMsgId).await? {
Some(s) => MsgId::new(s.parse()?),
@@ -1194,7 +1177,7 @@ ORDER BY m.timestamp DESC,m.id DESC",
Ok(list)
}
/// Returns a list of messages with database ID higher than last marked as seen.
/// (deprecated) Returns a list of messages with database ID higher than last marked as seen.
///
/// This function is supposed to be used by bot to request messages
/// that are not processed yet.
@@ -1204,6 +1187,13 @@ ORDER BY m.timestamp DESC,m.id DESC",
/// shortly after notification or notification is manually triggered
/// to interrupt waiting.
/// Notification may be manually triggered by calling [`Self::stop_io`].
///
/// Deprecated 2026-04: This returns the message's id as soon as the first part arrives,
/// even if it is not fully downloaded yet.
/// The bot needs to wait for the message to be fully downloaded.
/// Since this is usually not the desired behavior,
/// bots should instead use the #DC_EVENT_INCOMING_MSG / [`EventType::IncomingMsg`]
/// event for getting notified about new messages.
pub async fn wait_next_msgs(&self) -> Result<Vec<MsgId>> {
self.new_msgs_notify.notified().await;
let list = self.get_next_msgs().await?;
@@ -1283,12 +1273,6 @@ ORDER BY m.timestamp DESC,m.id DESC",
Ok(list)
}
/// Returns true if given folder name is the name of the "DeltaChat" folder.
pub async fn is_mvbox(&self, folder_name: &str) -> Result<bool> {
let mvbox = self.get_config(Config::ConfiguredMvboxFolder).await?;
Ok(mvbox.as_deref() == Some(folder_name))
}
pub(crate) fn derive_blobdir(dbfile: &Path) -> PathBuf {
let mut blob_fname = OsString::new();
blob_fname.push(dbfile.file_name().unwrap_or_default());
+6 -5
View File
@@ -173,9 +173,7 @@ pub(crate) async fn download_msg(
if msg_transport_id != transport_id {
return Ok(None);
}
session
.fetch_single_msg(context, &server_folder, server_uid, rfc724_mid)
.await?;
Box::pin(session.fetch_single_msg(context, &server_folder, server_uid, rfc724_mid)).await?;
Ok(Some(()))
}
@@ -204,8 +202,11 @@ impl Session {
let mut uid_message_ids: BTreeMap<u32, String> = BTreeMap::new();
uid_message_ids.insert(uid, rfc724_mid);
let (sender, receiver) = async_channel::unbounded();
self.fetch_many_msgs(context, folder, vec![uid], &uid_message_ids, sender)
.await?;
{
let _fetch_msgs_lock_guard = context.fetch_msgs_mutex.lock().await;
self.fetch_many_msgs(context, folder, vec![uid], &uid_message_ids, sender)
.await?;
}
if receiver.recv().await.is_err() {
bail!("Failed to fetch UID {uid}");
}
+3 -7
View File
@@ -89,13 +89,9 @@ mod test_chatlist_events {
.get_matching(|evt| match evt {
EventType::ChatlistItemChanged {
chat_id: Some(ev_chat_id),
} => {
if ev_chat_id == &chat_id {
first_event_is_item.store(true, Ordering::Relaxed);
true
} else {
false
}
} if ev_chat_id == &chat_id => {
first_event_is_item.store(true, Ordering::Relaxed);
true
}
EventType::ChatlistChanged => true,
_ => false,
+17 -19
View File
@@ -86,8 +86,7 @@ impl HtmlMsgParser {
/// Function takes a raw mime-message string,
/// searches for the main-text part
/// and returns that as parser.html
#[expect(clippy::arithmetic_side_effects)]
pub async fn from_bytes<'a>(
pub fn from_bytes<'a>(
context: &Context,
rawmime: &'a [u8],
) -> Result<(Self, mailparse::ParsedMail<'a>)> {
@@ -99,14 +98,14 @@ impl HtmlMsgParser {
let parsedmail = mailparse::parse_mail(rawmime).context("Failed to parse mail")?;
parser.collect_texts_recursive(context, &parsedmail).await?;
parser.collect_texts_recursive(context, &parsedmail)?;
if parser.html.is_empty() {
if let Some(plain) = &parser.plain {
parser.html = plain.to_html();
}
} else {
parser.cid_to_data_recursive(context, &parsedmail).await?;
parser.cid_to_data_recursive(context, &parsedmail)?;
}
parser.html += &mem::take(&mut parser.msg_html);
Ok((parser, parsedmail))
@@ -120,8 +119,7 @@ impl HtmlMsgParser {
/// Usually, there is at most one plain-text and one HTML-text part,
/// multiple plain-text parts might be used for mailinglist-footers,
/// therefore we use the first one.
#[expect(clippy::arithmetic_side_effects)]
async fn collect_texts_recursive<'a>(
fn collect_texts_recursive<'a>(
&'a mut self,
context: &'a Context,
mail: &'a mailparse::ParsedMail<'a>,
@@ -129,7 +127,7 @@ impl HtmlMsgParser {
match get_mime_multipart_type(&mail.ctype) {
MimeMultipartType::Multiple => {
for cur_data in &mail.subparts {
Box::pin(self.collect_texts_recursive(context, cur_data)).await?
self.collect_texts_recursive(context, cur_data)?
}
Ok(())
}
@@ -138,7 +136,7 @@ impl HtmlMsgParser {
if raw.is_empty() {
return Ok(());
}
let (parser, mail) = Box::pin(HtmlMsgParser::from_bytes(context, &raw)).await?;
let (parser, mail) = HtmlMsgParser::from_bytes(context, &raw)?;
if !parser.html.is_empty() {
let mut text = "\r\n\r\n".to_string();
for h in mail.headers {
@@ -201,7 +199,7 @@ impl HtmlMsgParser {
/// Replace cid:-protocol by the data:-protocol where appropriate.
/// This allows the final html-file to be self-contained.
async fn cid_to_data_recursive<'a>(
fn cid_to_data_recursive<'a>(
&'a mut self,
context: &'a Context,
mail: &'a mailparse::ParsedMail<'a>,
@@ -209,7 +207,7 @@ impl HtmlMsgParser {
match get_mime_multipart_type(&mail.ctype) {
MimeMultipartType::Multiple => {
for cur_data in &mail.subparts {
Box::pin(self.cid_to_data_recursive(context, cur_data)).await?;
self.cid_to_data_recursive(context, cur_data)?;
}
Ok(())
}
@@ -271,7 +269,7 @@ impl MsgId {
let rawmime = rawmime?;
if !rawmime.is_empty() {
match HtmlMsgParser::from_bytes(context, &rawmime).await {
match HtmlMsgParser::from_bytes(context, &rawmime) {
Err(err) => {
warn!(context, "get_html: parser error: {:#}", err);
Ok(None)
@@ -300,7 +298,7 @@ mod tests {
async fn test_htmlparse_plain_unspecified() {
let t = TestContext::new().await;
let raw = include_bytes!("../test-data/message/text_plain_unspecified.eml");
let (parser, _) = HtmlMsgParser::from_bytes(&t.ctx, raw).await.unwrap();
let (parser, _) = HtmlMsgParser::from_bytes(&t.ctx, raw).unwrap();
assert_eq!(
parser.html,
r#"<!DOCTYPE html>
@@ -318,7 +316,7 @@ This message does not have Content-Type nor Subject.<br/>
async fn test_htmlparse_plain_iso88591() {
let t = TestContext::new().await;
let raw = include_bytes!("../test-data/message/text_plain_iso88591.eml");
let (parser, _) = HtmlMsgParser::from_bytes(&t.ctx, raw).await.unwrap();
let (parser, _) = HtmlMsgParser::from_bytes(&t.ctx, raw).unwrap();
assert_eq!(
parser.html,
r#"<!DOCTYPE html>
@@ -336,7 +334,7 @@ message with a non-UTF-8 encoding: äöüßÄÖÜ<br/>
async fn test_htmlparse_plain_flowed() {
let t = TestContext::new().await;
let raw = include_bytes!("../test-data/message/text_plain_flowed.eml");
let (parser, _) = HtmlMsgParser::from_bytes(&t.ctx, raw).await.unwrap();
let (parser, _) = HtmlMsgParser::from_bytes(&t.ctx, raw).unwrap();
assert!(parser.plain.unwrap().flowed);
assert_eq!(
parser.html,
@@ -358,7 +356,7 @@ and will be wrapped as usual.<br/>
async fn test_htmlparse_alt_plain() {
let t = TestContext::new().await;
let raw = include_bytes!("../test-data/message/text_alt_plain.eml");
let (parser, _) = HtmlMsgParser::from_bytes(&t.ctx, raw).await.unwrap();
let (parser, _) = HtmlMsgParser::from_bytes(&t.ctx, raw).unwrap();
assert_eq!(
parser.html,
r#"<!DOCTYPE html>
@@ -378,7 +376,7 @@ test some special html-characters as &lt; &gt; and &amp; but also &quot; and &#x
async fn test_htmlparse_html() {
let t = TestContext::new().await;
let raw = include_bytes!("../test-data/message/text_html.eml");
let (parser, _) = HtmlMsgParser::from_bytes(&t.ctx, raw).await.unwrap();
let (parser, _) = HtmlMsgParser::from_bytes(&t.ctx, raw).unwrap();
// on windows, `\r\n` linends are returned from mimeparser,
// however, rust multiline-strings use just `\n`;
@@ -396,7 +394,7 @@ test some special html-characters as &lt; &gt; and &amp; but also &quot; and &#x
async fn test_htmlparse_alt_html() {
let t = TestContext::new().await;
let raw = include_bytes!("../test-data/message/text_alt_html.eml");
let (parser, _) = HtmlMsgParser::from_bytes(&t.ctx, raw).await.unwrap();
let (parser, _) = HtmlMsgParser::from_bytes(&t.ctx, raw).unwrap();
assert_eq!(
parser.html.replace('\r', ""), // see comment in test_htmlparse_html()
r##"<html>
@@ -410,7 +408,7 @@ test some special html-characters as &lt; &gt; and &amp; but also &quot; and &#x
async fn test_htmlparse_alt_plain_html() {
let t = TestContext::new().await;
let raw = include_bytes!("../test-data/message/text_alt_plain_html.eml");
let (parser, _) = HtmlMsgParser::from_bytes(&t.ctx, raw).await.unwrap();
let (parser, _) = HtmlMsgParser::from_bytes(&t.ctx, raw).unwrap();
assert_eq!(
parser.html.replace('\r', ""), // see comment in test_htmlparse_html()
r##"<html>
@@ -434,7 +432,7 @@ test some special html-characters as &lt; &gt; and &amp; but also &quot; and &#x
assert!(test.find("data:").is_none());
// parsing converts cid: to data:
let (parser, _) = HtmlMsgParser::from_bytes(&t.ctx, raw).await.unwrap();
let (parser, _) = HtmlMsgParser::from_bytes(&t.ctx, raw).unwrap();
assert!(parser.html.contains("<html>"));
assert!(!parser.html.contains("Content-Id:"));
assert!(parser.html.contains("data:image/jpeg;base64,/9j/4AAQ"));
+56 -251
View File
@@ -27,13 +27,14 @@ use crate::calls::{
use crate::chat::{self, ChatId, ChatIdBlocked, add_device_msg};
use crate::chatlist_events;
use crate::config::Config;
use crate::constants::{self, Blocked, DC_VERSION_STR};
use crate::constants::{Blocked, DC_VERSION_STR};
use crate::contact::ContactId;
use crate::context::Context;
use crate::ensure_and_debug_assert;
use crate::events::EventType;
use crate::headerdef::{HeaderDef, HeaderDefMap};
use crate::log::{LogExt, warn};
use crate::message::{self, Message, MessageState, MessengerMessage, MsgId};
use crate::message::{self, Message, MessageState, MsgId};
use crate::mimeparser;
use crate::net::proxy::ProxyConfig;
use crate::net::session::SessionStream;
@@ -91,6 +92,9 @@ pub(crate) struct Imap {
oauth2: bool,
/// Watched folder.
pub(crate) folder: String,
authentication_failed_once: bool,
pub(crate) connectivity: ConnectivityStore,
@@ -162,7 +166,6 @@ pub enum FolderMeaning {
/// Spam folder.
Spam,
Inbox,
Mvbox,
Trash,
/// Virtual folders.
@@ -174,19 +177,6 @@ pub enum FolderMeaning {
Virtual,
}
impl FolderMeaning {
pub fn to_config(self) -> Option<Config> {
match self {
FolderMeaning::Unknown => None,
FolderMeaning::Spam => None,
FolderMeaning::Inbox => Some(Config::ConfiguredInboxFolder),
FolderMeaning::Mvbox => Some(Config::ConfiguredMvboxFolder),
FolderMeaning::Trash => None,
FolderMeaning::Virtual => None,
}
}
}
struct UidGrouper<T: Iterator<Item = (i64, u32, String)>> {
inner: Peekable<T>,
}
@@ -263,6 +253,11 @@ impl Imap {
let addr = &param.addr;
let strict_tls = param.strict_tls(proxy_config.is_some());
let oauth2 = param.oauth2;
let folder = param
.imap_folder
.clone()
.unwrap_or_else(|| "INBOX".to_string());
ensure_and_debug_assert!(!folder.is_empty(), "Watched folder name cannot be empty");
let (resync_request_sender, resync_request_receiver) = async_channel::bounded(1);
Ok(Imap {
transport_id,
@@ -273,6 +268,7 @@ impl Imap {
proxy_config,
strict_tls,
oauth2,
folder,
authentication_failed_once: false,
connectivity: Default::default(),
conn_last_try: UNIX_EPOCH,
@@ -490,7 +486,7 @@ impl Imap {
/// that folders are created and IMAP capabilities are determined.
pub(crate) async fn prepare(&mut self, context: &Context) -> Result<Session> {
let configuring = false;
let mut session = match self.connect(context, configuring).await {
let session = match self.connect(context, configuring).await {
Ok(session) => session,
Err(err) => {
self.connectivity.set_err(context, &err);
@@ -498,14 +494,6 @@ impl Imap {
}
};
let folders_configured = context
.sql
.get_raw_config_int(constants::DC_FOLDERS_CONFIGURED_KEY)
.await?;
if folders_configured.unwrap_or_default() < constants::DC_FOLDERS_CONFIGURED_VERSION {
self.configure_folders(context, &mut session).await?;
}
Ok(session)
}
@@ -518,15 +506,15 @@ impl Imap {
context: &Context,
session: &mut Session,
watch_folder: &str,
folder_meaning: FolderMeaning,
) -> Result<()> {
ensure_and_debug_assert!(!watch_folder.is_empty(), "Watched folder cannot be empty");
if !context.sql.is_open().await {
// probably shutdown
bail!("IMAP operation attempted while it is torn down");
}
let msgs_fetched = self
.fetch_new_messages(context, session, watch_folder, folder_meaning)
.fetch_new_messages(context, session, watch_folder)
.await
.context("fetch_new_messages")?;
if msgs_fetched && context.get_config_delete_device_after().await?.is_some() {
@@ -554,19 +542,9 @@ impl Imap {
context: &Context,
session: &mut Session,
folder: &str,
folder_meaning: FolderMeaning,
) -> Result<bool> {
let transport_id = session.transport_id();
if should_ignore_folder(context, folder, folder_meaning).await? {
info!(
context,
"Transport {transport_id}: Not fetching from {folder:?}."
);
session.new_mail = false;
return Ok(false);
}
let folder_exists = session
.select_with_uidvalidity(context, folder)
.await
@@ -589,9 +567,8 @@ impl Imap {
let mut read_cnt = 0;
loop {
let (n, fetch_more) = self
.fetch_new_msg_batch(context, session, folder, folder_meaning)
.await?;
let (n, fetch_more) =
Box::pin(self.fetch_new_msg_batch(context, session, folder)).await?;
read_cnt += n;
if !fetch_more {
return Ok(read_cnt > 0);
@@ -606,7 +583,6 @@ impl Imap {
context: &Context,
session: &mut Session,
folder: &str,
folder_meaning: FolderMeaning,
) -> Result<(usize, bool)> {
let transport_id = self.transport_id;
let uid_validity = get_uidvalidity(context, transport_id, folder).await?;
@@ -676,13 +652,7 @@ impl Imap {
info!(context, "Deleting locally deleted message {message_id}.");
}
let _target;
let target = if delete {
""
} else {
_target = target_folder(context, folder, folder_meaning, &headers).await?;
&_target
};
let target = if delete { "" } else { folder };
context
.sql
@@ -710,18 +680,9 @@ impl Imap {
// message, move it to the movebox and then download the second message before
// downloading the first one, if downloading from inbox before moving is allowed.
if folder == target
// Never download messages directly from the spam folder.
// If the sender is known, the message will be moved to the Inbox or Mvbox
// and then we download the message from there.
// Also see `spam_target_folder_cfg()`.
&& folder_meaning != FolderMeaning::Spam
&& prefetch_should_download(
context,
&headers,
&message_id,
fetch_response.flags(),
)
.await.context("prefetch_should_download")?
&& prefetch_should_download(context, &headers, &message_id, fetch_response.flags())
.await
.context("prefetch_should_download")?
{
if headers
.get_header_value(HeaderDef::ChatIsPostMessage)
@@ -730,10 +691,19 @@ impl Imap {
info!(context, "{message_id:?} is a post-message.");
available_post_msgs.push(message_id.clone());
if download_limit.is_none_or(|download_limit| size <= download_limit) {
download_later.push(message_id.clone());
let is_bot = context.get_config_bool(Config::Bot).await?;
if is_bot && download_limit.is_none_or(|download_limit| size <= download_limit)
{
uids_fetch.push(uid);
uid_message_ids.insert(uid, message_id);
} else {
if download_limit.is_none_or(|download_limit| size <= download_limit) {
// Download later after all the small messages are downloaded,
// so that large messages don't delay receiving small messages
download_later.push(message_id.clone());
}
largest_uid_skipped = Some(uid);
}
largest_uid_skipped = Some(uid);
} else {
info!(context, "{message_id:?} is not a post-message.");
if download_limit.is_none_or(|download_limit| size <= download_limit) {
@@ -1103,15 +1073,22 @@ impl Session {
return Ok(());
}
context
.sql
.execute(
"DELETE FROM imap_markseen WHERE id NOT IN (SELECT imap.id FROM imap)",
(),
)
.await?;
let transport_id = self.transport_id();
let rows = context
let mut rows = context
.sql
.query_map_vec(
"SELECT imap.id, uid, folder FROM imap, imap_markseen
WHERE imap.id = imap_markseen.id
AND imap.transport_id=?
AND target = folder
ORDER BY folder, uid",
AND target = folder",
(transport_id,),
|row| {
let rowid: i64 = row.get(0)?;
@@ -1122,6 +1099,16 @@ impl Session {
)
.await?;
// Number of SQL results is expected to be low as
// we usually don't have many messages to mark on IMAP at once.
// We are sorting outside of SQL to avoid SQLite constructing a query plan
// that scans the whole `imap` table. Scanning `imap_markseen` is fine
// as it should not have many items.
// If you change the SQL query, test it with `EXPLAIN QUERY PLAN`.
rows.sort_unstable_by(|(_rowid1, uid1, folder1), (_rowid2, uid2, folder2)| {
(folder1, uid1).cmp(&(folder2, uid2))
});
for (folder, rowid_set, uid_set) in UidGrouper::from(rows) {
let folder_exists = match self.select_with_uidvalidity(context, &folder).await {
Err(err) => {
@@ -1645,13 +1632,8 @@ impl Session {
// Store new encrypted device token on the server
// even if it is the same as the old one.
if let Some(encrypted_device_token) = new_encrypted_device_token {
let folder = context
.get_config(Config::ConfiguredInboxFolder)
.await?
.context("INBOX is not configured")?;
self.run_command_and_check_ok(&format_setmetadata(
&folder,
"INBOX",
&encrypted_device_token,
))
.await
@@ -1696,117 +1678,6 @@ impl Session {
}
Ok(())
}
/// Attempts to configure mvbox.
///
/// Tries to find any folder examining `folders` in the order they go.
/// This method does not use LIST command to ensure that
/// configuration works even if mailbox lookup is forbidden via Access Control List (see
/// <https://datatracker.ietf.org/doc/html/rfc4314>).
///
/// Returns first found folder name.
async fn configure_mvbox<'a>(
&mut self,
context: &Context,
folders: &[&'a str],
) -> Result<Option<&'a str>> {
// Close currently selected folder if needed.
// We are going to select folders using low-level EXAMINE operations below.
self.maybe_close_folder(context).await?;
for folder in folders {
info!(context, "Looking for MVBOX-folder \"{}\"...", &folder);
let res = self.examine(&folder).await;
if res.is_ok() {
info!(
context,
"MVBOX-folder {:?} successfully selected, using it.", &folder
);
self.close().await?;
// Before moving emails to the mvbox we need to remember its UIDVALIDITY, otherwise
// emails moved before that wouldn't be fetched but considered "old" instead.
let folder_exists = self.select_with_uidvalidity(context, folder).await?;
ensure!(folder_exists, "No MVBOX folder {:?}??", &folder);
return Ok(Some(folder));
}
}
Ok(None)
}
}
impl Imap {
pub(crate) async fn configure_folders(
&mut self,
context: &Context,
session: &mut Session,
) -> Result<()> {
let mut folders = session
.list(Some(""), Some("*"))
.await
.context("list_folders failed")?;
let mut delimiter = ".".to_string();
let mut delimiter_is_default = true;
let mut folder_configs = BTreeMap::new();
while let Some(folder) = folders.try_next().await? {
info!(context, "Scanning folder: {:?}", folder);
// Update the delimiter iff there is a different one, but only once.
if let Some(d) = folder.delimiter()
&& delimiter_is_default
&& !d.is_empty()
&& delimiter != d
{
delimiter = d.to_string();
delimiter_is_default = false;
}
let folder_meaning = get_folder_meaning_by_attrs(folder.attributes());
let folder_name_meaning = get_folder_meaning_by_name(folder.name());
if let Some(config) = folder_meaning.to_config() {
// Always takes precedence
folder_configs.insert(config, folder.name().to_string());
} else if let Some(config) = folder_name_meaning.to_config() {
// only set if none has been already set
folder_configs
.entry(config)
.or_insert_with(|| folder.name().to_string());
}
}
drop(folders);
info!(context, "Using \"{}\" as folder-delimiter.", delimiter);
let fallback_folder = format!("INBOX{delimiter}DeltaChat");
let mvbox_folder = session
.configure_mvbox(context, &["DeltaChat", &fallback_folder])
.await
.context("failed to configure mvbox")?;
context
.set_config_internal(Config::ConfiguredInboxFolder, Some("INBOX"))
.await?;
if let Some(mvbox_folder) = mvbox_folder {
info!(context, "Setting MVBOX FOLDER TO {}", &mvbox_folder);
context
.set_config_internal(Config::ConfiguredMvboxFolder, Some(mvbox_folder))
.await?;
}
for (config, name) in folder_configs {
context.set_config_internal(config, Some(&name)).await?;
}
context
.sql
.set_raw_config_int(
constants::DC_FOLDERS_CONFIGURED_KEY,
constants::DC_FOLDERS_CONFIGURED_VERSION,
)
.await?;
info!(context, "FINISHED configuring IMAP-folders.");
Ok(())
}
}
impl Session {
@@ -1940,15 +1811,7 @@ async fn spam_target_folder_cfg(
return Ok(None);
}
if needs_move_to_mvbox(context, headers).await?
// If OnlyFetchMvbox is set, we don't want to move the message to
// the inbox where we wouldn't fetch it again:
|| context.get_config_bool(Config::OnlyFetchMvbox).await?
{
Ok(Some(Config::ConfiguredMvboxFolder))
} else {
Ok(Some(Config::ConfiguredInboxFolder))
}
Ok(Some(Config::ConfiguredInboxFolder))
}
/// Returns `ConfiguredInboxFolder` or `ConfiguredMvboxFolder` if
@@ -1959,16 +1822,12 @@ pub async fn target_folder_cfg(
folder_meaning: FolderMeaning,
headers: &[mailparse::MailHeader<'_>],
) -> Result<Option<Config>> {
if context.is_mvbox(folder).await? {
if folder == "DeltaChat" {
return Ok(None);
}
if folder_meaning == FolderMeaning::Spam {
spam_target_folder_cfg(context, headers).await
} else if folder_meaning == FolderMeaning::Inbox
&& needs_move_to_mvbox(context, headers).await?
{
Ok(Some(Config::ConfiguredMvboxFolder))
} else {
Ok(None)
}
@@ -1989,27 +1848,6 @@ pub async fn target_folder(
}
}
async fn needs_move_to_mvbox(
context: &Context,
headers: &[mailparse::MailHeader<'_>],
) -> Result<bool> {
let has_chat_version = headers.get_header_value(HeaderDef::ChatVersion).is_some();
if !context.get_config_bool(Config::MvboxMove).await? {
return Ok(false);
}
if has_chat_version {
Ok(true)
} else if let Some(parent) = get_prefetch_parent_message(context, headers).await? {
match parent.is_dc_message {
MessengerMessage::No => Ok(false),
MessengerMessage::Yes | MessengerMessage::Reply => Ok(true),
}
} else {
Ok(false)
}
}
/// Try to get the folder meaning by the name of the folder only used if the server does not support XLIST.
// TODO: lots languages missing - maybe there is a list somewhere on other MUAs?
// however, if we fail to find out the sent-folder,
@@ -2340,21 +2178,6 @@ async fn get_modseq(context: &Context, transport_id: u32, folder: &str) -> Resul
.unwrap_or(0))
}
/// Whether to ignore fetching messages from a folder.
///
/// This caters for the [`Config::OnlyFetchMvbox`] setting which means mails from folders
/// not explicitly watched should not be fetched.
async fn should_ignore_folder(
context: &Context,
folder: &str,
folder_meaning: FolderMeaning,
) -> Result<bool> {
if !context.get_config_bool(Config::OnlyFetchMvbox).await? {
return Ok(false);
}
Ok(!(context.is_mvbox(folder).await? || folder_meaning == FolderMeaning::Spam))
}
/// Builds a list of sequence/uid sets. The returned sets have each no more than around 1000
/// characters because according to <https://tools.ietf.org/html/rfc2683#section-3.2.1.5>
/// command lines should not be much more than 1000 chars (servers should allow at least 8000 chars)
@@ -2414,23 +2237,5 @@ impl std::fmt::Display for UidRange {
}
}
pub(crate) async fn get_watched_folder_configs(context: &Context) -> Result<Vec<Config>> {
let mut res = vec![Config::ConfiguredInboxFolder];
if context.should_watch_mvbox().await? {
res.push(Config::ConfiguredMvboxFolder);
}
Ok(res)
}
pub(crate) async fn get_watched_folders(context: &Context) -> Result<Vec<String>> {
let mut res = Vec::new();
for folder_config in get_watched_folder_configs(context).await? {
if let Some(folder) = context.get_config(folder_config).await? {
res.push(folder);
}
}
Ok(res)
}
#[cfg(test)]
mod imap_tests;
+8
View File
@@ -220,6 +220,8 @@ impl Client {
alpn(addr.port()),
logging_stream,
&context.tls_session_store,
&context.spki_hash_store,
&context.sql,
)
.await?;
let buffered_stream = BufWriter::new(tls_stream);
@@ -282,6 +284,8 @@ impl Client {
"",
tcp_stream,
&context.tls_session_store,
&context.spki_hash_store,
&context.sql,
)
.await
.context("STARTTLS upgrade failed")?;
@@ -310,6 +314,8 @@ impl Client {
alpn(port),
proxy_stream,
&context.tls_session_store,
&context.spki_hash_store,
&context.sql,
)
.await?;
let buffered_stream = BufWriter::new(tls_stream);
@@ -373,6 +379,8 @@ impl Client {
"",
proxy_stream,
&context.tls_session_store,
&context.spki_hash_store,
&context.sql,
)
.await
.context("STARTTLS upgrade failed")?;
+1 -5
View File
@@ -120,11 +120,7 @@ impl Session {
impl Imap {
/// Idle using polling.
pub(crate) async fn fake_idle(
&mut self,
context: &Context,
watch_folder: String,
) -> Result<()> {
pub(crate) async fn fake_idle(&mut self, context: &Context, watch_folder: &str) -> Result<()> {
let fake_idle_start_time = tools::Time::now();
info!(context, "IMAP-fake-IDLEing folder={:?}", watch_folder);
+22 -58
View File
@@ -100,23 +100,16 @@ fn test_build_sequence_sets() {
async fn check_target_folder_combination(
folder: &str,
mvbox_move: bool,
chat_msg: bool,
expected_destination: &str,
accepted_chat: bool,
outgoing: bool,
) -> Result<()> {
println!(
"Testing: For folder {folder}, mvbox_move {mvbox_move}, chat_msg {chat_msg}, accepted {accepted_chat}, outgoing {outgoing}"
"Testing: For folder {folder}, chat_msg {chat_msg}, accepted {accepted_chat}, outgoing {outgoing}"
);
let t = TestContext::new_alice().await;
t.ctx
.set_config(Config::ConfiguredMvboxFolder, Some("DeltaChat"))
.await?;
t.ctx
.set_config(Config::MvboxMove, Some(if mvbox_move { "1" } else { "0" }))
.await?;
if accepted_chat {
let contact_id = Contact::create(&t.ctx, "", "bob@example.net").await?;
@@ -161,64 +154,42 @@ async fn check_target_folder_combination(
assert_eq!(
expected,
actual.as_deref(),
"For folder {folder}, mvbox_move {mvbox_move}, chat_msg {chat_msg}, accepted {accepted_chat}, outgoing {outgoing}: expected {expected:?}, got {actual:?}"
"For folder {folder}, chat_msg {chat_msg}, accepted {accepted_chat}, outgoing {outgoing}: expected {expected:?}, got {actual:?}"
);
Ok(())
}
// chat_msg means that the message was sent by Delta Chat
// The tuples are (folder, mvbox_move, chat_msg, expected_destination)
const COMBINATIONS_ACCEPTED_CHAT: &[(&str, bool, bool, &str)] = &[
("INBOX", false, false, "INBOX"),
("INBOX", false, true, "INBOX"),
("INBOX", true, false, "INBOX"),
("INBOX", true, true, "DeltaChat"),
("Spam", false, false, "INBOX"), // Move classical emails in accepted chats from Spam to Inbox, not 100% sure on this, we could also just never move non-chat-msgs
("Spam", false, true, "INBOX"),
("Spam", true, false, "INBOX"), // Move classical emails in accepted chats from Spam to Inbox, not 100% sure on this, we could also just never move non-chat-msgs
("Spam", true, true, "DeltaChat"),
// The tuples are (folder, chat_msg, expected_destination)
const COMBINATIONS_ACCEPTED_CHAT: &[(&str, bool, &str)] = &[
("INBOX", false, "INBOX"),
("INBOX", true, "INBOX"),
("Spam", false, "INBOX"), // Move classical emails in accepted chats from Spam to Inbox, not 100% sure on this, we could also just never move non-chat-msgs
("Spam", true, "INBOX"),
];
// These are the same as above, but non-chat messages in Spam stay in Spam
const COMBINATIONS_REQUEST: &[(&str, bool, bool, &str)] = &[
("INBOX", false, false, "INBOX"),
("INBOX", false, true, "INBOX"),
("INBOX", true, false, "INBOX"),
("INBOX", true, true, "DeltaChat"),
("Spam", false, false, "Spam"),
("Spam", false, true, "INBOX"),
("Spam", true, false, "Spam"),
("Spam", true, true, "DeltaChat"),
const COMBINATIONS_REQUEST: &[(&str, bool, &str)] = &[
("INBOX", false, "INBOX"),
("INBOX", true, "INBOX"),
("Spam", false, "Spam"),
("Spam", true, "INBOX"),
];
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_target_folder_incoming_accepted() -> Result<()> {
for (folder, mvbox_move, chat_msg, expected_destination) in COMBINATIONS_ACCEPTED_CHAT {
check_target_folder_combination(
folder,
*mvbox_move,
*chat_msg,
expected_destination,
true,
false,
)
.await?;
for (folder, chat_msg, expected_destination) in COMBINATIONS_ACCEPTED_CHAT {
check_target_folder_combination(folder, *chat_msg, expected_destination, true, false)
.await?;
}
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_target_folder_incoming_request() -> Result<()> {
for (folder, mvbox_move, chat_msg, expected_destination) in COMBINATIONS_REQUEST {
check_target_folder_combination(
folder,
*mvbox_move,
*chat_msg,
expected_destination,
false,
false,
)
.await?;
for (folder, chat_msg, expected_destination) in COMBINATIONS_REQUEST {
check_target_folder_combination(folder, *chat_msg, expected_destination, false, false)
.await?;
}
Ok(())
}
@@ -226,16 +197,9 @@ async fn test_target_folder_incoming_request() -> Result<()> {
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_target_folder_outgoing() -> Result<()> {
// Test outgoing emails
for (folder, mvbox_move, chat_msg, expected_destination) in COMBINATIONS_ACCEPTED_CHAT {
check_target_folder_combination(
folder,
*mvbox_move,
*chat_msg,
expected_destination,
true,
true,
)
.await?;
for (folder, chat_msg, expected_destination) in COMBINATIONS_ACCEPTED_CHAT {
check_target_folder_combination(folder, *chat_msg, expected_destination, true, true)
.await?;
}
Ok(())
}
+12
View File
@@ -1137,4 +1137,16 @@ mod tests {
Ok(())
}
/// Tests importing a backup from Delta Chat 1.30.3 for Android (core v1.86.0).
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_import_ancient_backup() -> Result<()> {
let mut tcm = TestContextManager::new();
let context = &tcm.unconfigured().await;
let backup_path = Path::new("test-data/core-1.86.0-backup.tar");
imex(context, ImexMode::ImportBackup, backup_path, None).await?;
Ok(())
}
}
+1
View File
@@ -17,6 +17,7 @@
clippy::cloned_instead_of_copied,
clippy::manual_is_variant_and
)]
#![cfg_attr(not(test), warn(clippy::large_futures))]
#![cfg_attr(not(test), warn(clippy::arithmetic_side_effects))]
#![cfg_attr(not(test), forbid(clippy::indexing_slicing))]
#![cfg_attr(not(test), forbid(clippy::string_slice))]
+44 -8
View File
@@ -56,9 +56,37 @@ pub enum EnteredCertificateChecks {
AcceptInvalidCertificates2 = 3,
}
/// Login parameters for a single server, either IMAP or SMTP
/// Login parameters for a single IMAP server.
#[derive(Debug, Default, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct EnteredServerLoginParam {
pub struct EnteredImapLoginParam {
/// Server hostname or IP address.
pub server: String,
/// Server port.
///
/// 0 if not specified.
pub port: u16,
/// Folder to watch.
///
/// If empty, user has not entered anything and it shuold expand to "INBOX" later.
pub folder: String,
/// Socket security.
pub security: Socket,
/// Username.
///
/// Empty string if not specified.
pub user: String,
/// Password.
pub password: String,
}
/// Login parameters for a single SMTP server.
#[derive(Debug, Default, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct EnteredSmtpLoginParam {
/// Server hostname or IP address.
pub server: String,
@@ -96,10 +124,10 @@ pub struct EnteredLoginParam {
pub addr: String,
/// IMAP settings.
pub imap: EnteredServerLoginParam,
pub imap: EnteredImapLoginParam,
/// SMTP settings.
pub smtp: EnteredServerLoginParam,
pub smtp: EnteredSmtpLoginParam,
/// TLS options: whether to allow invalid certificates and/or
/// invalid hostnames
@@ -111,6 +139,8 @@ pub struct EnteredLoginParam {
impl EnteredLoginParam {
/// Loads entered account settings.
///
/// This is a legacy API for loading from separate config parameters.
pub(crate) async fn load(context: &Context) -> Result<Self> {
let addr = context
.get_config(Config::Addr)
@@ -127,6 +157,10 @@ impl EnteredLoginParam {
.get_config_parsed::<u16>(Config::MailPort)
.await?
.unwrap_or_default();
// There is no way to set custom folder with this legacy API.
let mail_folder = String::new();
let mail_security = context
.get_config_parsed::<i32>(Config::MailSecurity)
.await?
@@ -185,14 +219,15 @@ impl EnteredLoginParam {
Ok(EnteredLoginParam {
addr,
imap: EnteredServerLoginParam {
imap: EnteredImapLoginParam {
server: mail_server,
port: mail_port,
folder: mail_folder,
security: mail_security,
user: mail_user,
password: mail_pw,
},
smtp: EnteredServerLoginParam {
smtp: EnteredSmtpLoginParam {
server: send_server,
port: send_port,
security: send_security,
@@ -354,14 +389,15 @@ mod tests {
let t = TestContext::new().await;
let param = EnteredLoginParam {
addr: "alice@example.org".to_string(),
imap: EnteredServerLoginParam {
imap: EnteredImapLoginParam {
server: "".to_string(),
port: 0,
folder: "".to_string(),
security: Socket::Starttls,
user: "".to_string(),
password: "foobar".to_string(),
},
smtp: EnteredServerLoginParam {
smtp: EnteredSmtpLoginParam {
server: "".to_string(),
port: 2947,
security: Socket::default(),
+10 -7
View File
@@ -1,7 +1,6 @@
//! # Messages and their identifiers.
use std::collections::BTreeSet;
use std::collections::HashSet;
use std::path::{Path, PathBuf};
use std::str;
@@ -200,7 +199,6 @@ SELECT ?1, rfc724_mid, pre_rfc724_mid, timestamp, ?, ? FROM msgs WHERE id=?1
}
/// Returns detailed message information in a multi-line text form.
#[expect(clippy::arithmetic_side_effects)]
pub async fn get_info(self, context: &Context) -> Result<String> {
let msg = Message::load_from_db(context, self).await?;
@@ -815,14 +813,17 @@ impl Message {
/// Returns the timestamp of the message for sorting.
pub fn get_sort_timestamp(&self) -> i64 {
self.timestamp_sort
if self.timestamp_sort != 0 {
self.timestamp_sort
} else {
self.timestamp_sent
}
}
/// Returns the text of the message.
///
/// Currently this includes `additional_text`, but this may change in future, when the UIs show
/// the necessary info themselves.
#[expect(clippy::arithmetic_side_effects)]
pub fn get_text(&self) -> String {
self.text.clone() + &self.additional_text
}
@@ -1315,7 +1316,7 @@ impl Message {
}
/// Force the message to be sent in plain text.
pub fn force_plaintext(&mut self) {
pub(crate) fn force_plaintext(&mut self) {
self.param.set_int(Param::ForcePlaintext, 1);
}
@@ -1389,6 +1390,8 @@ pub enum MessageState {
/// For files which need time to be prepared before they can be
/// sent, the message enters this state before
/// OutPending.
///
/// Deprecated 2024-12-07.
OutPreparing = 18,
/// Message saved as draft.
@@ -1689,7 +1692,7 @@ pub(crate) async fn delete_msg_locally(context: &Context, msg: &Message) -> Resu
pub(crate) async fn delete_msgs_locally_done(
context: &Context,
msg_ids: &[MsgId],
modified_chat_ids: HashSet<ChatId>,
modified_chat_ids: BTreeSet<ChatId>,
) -> Result<()> {
for modified_chat_id in modified_chat_ids {
context.emit_msgs_changed_without_msg_id(modified_chat_id);
@@ -1719,7 +1722,7 @@ pub async fn delete_msgs_ex(
msg_ids: &[MsgId],
delete_for_all: bool,
) -> Result<()> {
let mut modified_chat_ids = HashSet::new();
let mut modified_chat_ids = BTreeSet::new();
let mut deleted_rfc724_mid = Vec::new();
let mut res = Ok(());
+19 -15
View File
@@ -232,11 +232,7 @@ impl MimeFactory {
if chat.is_self_talk() {
to.push((from_displayname.to_string(), from_addr.to_string()));
encryption_pubkeys = if msg.param.get_bool(Param::ForcePlaintext).unwrap_or(false) {
None
} else {
Some(Vec::new())
};
encryption_pubkeys = Some(Vec::new());
} else if chat.is_mailing_list() {
let list_post = chat
.param
@@ -467,18 +463,21 @@ impl MimeFactory {
.into_iter()
.filter(|id| *id != ContactId::SELF)
.collect();
if recipient_ids.len() == 1
&& !matches!(
msg.param.get_cmd(),
SystemMessage::MemberRemovedFromGroup | SystemMessage::SecurejoinMessage
)
&& !matches!(chat.typ, Chattype::OutBroadcast | Chattype::InBroadcast)
if !matches!(
msg.param.get_cmd(),
SystemMessage::MemberRemovedFromGroup | SystemMessage::SecurejoinMessage
) && !matches!(chat.typ, Chattype::OutBroadcast | Chattype::InBroadcast)
{
let origin = match recipient_ids.len() {
1 => Origin::OutgoingTo,
// Use the same origin as ChatId::accept_ex() does for groups.
_ => Origin::IncomingTo,
};
info!(
context,
"Scale up origin of {} recipients to OutgoingTo.", chat.id
"Scale up origin of {} recipients to {origin:?}.", chat.id
);
ContactId::scaleup_origin(context, &recipient_ids, Origin::OutgoingTo).await?;
ContactId::scaleup_origin(context, &recipient_ids, origin).await?;
}
if !msg.is_system_message()
@@ -852,7 +851,13 @@ impl MimeFactory {
let rfc724_mid = match &self.loaded {
Loaded::Message { msg, .. } => match &self.pre_message_mode {
PreMessageMode::Pre { .. } => create_outgoing_rfc724_mid(),
PreMessageMode::Pre { .. } => {
if msg.pre_rfc724_mid.is_empty() {
create_outgoing_rfc724_mid()
} else {
msg.pre_rfc724_mid.clone()
}
}
_ => msg.rfc724_mid.clone(),
},
Loaded::Mdn { .. } => create_outgoing_rfc724_mid(),
@@ -1882,7 +1887,6 @@ impl MimeFactory {
}
/// Render an MDN
#[expect(clippy::arithmetic_side_effects)]
fn render_mdn(&mut self) -> Result<MimePart<'static>> {
// RFC 6522, this also requires the `report-type` parameter which is equal
// to the MIME subtype of the second body part of the multipart/report
+19 -9
View File
@@ -86,7 +86,9 @@ pub(crate) struct MimeMessage {
/// messages to this address to post them to the list.
pub list_post: Option<String>,
pub chat_disposition_notification_to: Option<SingleInfo>,
pub decrypting_failed: bool,
/// Decryption error if decryption of the message has failed.
pub decryption_error: Option<String>,
/// Valid signature fingerprint if a message is an
/// Autocrypt encrypted and signed message and corresponding intended recipient fingerprints
@@ -267,7 +269,6 @@ impl MimeMessage {
///
/// This method has some side-effects,
/// such as saving blobs and saving found public keys to the database.
#[expect(clippy::arithmetic_side_effects)]
pub(crate) async fn from_bytes(context: &Context, body: &[u8]) -> Result<Self> {
let mail = mailparse::parse_mail(body)?;
@@ -371,7 +372,7 @@ impl MimeMessage {
hop_info += "\n\n";
hop_info += &dkim_results.to_string();
let incoming = !context.is_self_addr(&from.addr).await?;
let from_is_not_self_addr = !context.is_self_addr(&from.addr).await?;
let mut aheader_values = mail.headers.get_all_values(HeaderDef::Autocrypt.into());
@@ -436,7 +437,7 @@ impl MimeMessage {
};
let mut autocrypt_header = None;
if incoming {
if from_is_not_self_addr {
// See `get_all_addresses_from_header()` for why we take the last valid header.
for val in aheader_values.iter().rev() {
autocrypt_header = match Aheader::from_str(val) {
@@ -467,7 +468,7 @@ impl MimeMessage {
None
};
let mut public_keyring = if incoming {
let mut public_keyring = if from_is_not_self_addr {
if let Some(autocrypt_header) = autocrypt_header {
vec![autocrypt_header.public_key]
} else {
@@ -652,6 +653,15 @@ impl MimeMessage {
.into_iter()
.last()
.map(|(fp, recipient_fps)| (fp, recipient_fps.into_iter().collect::<HashSet<_>>()));
let incoming = if let Some((ref sig_fp, _)) = signature {
sig_fp.hex() != key::self_fingerprint(context).await?
} else {
// rare case of getting a cleartext message
// so we determine 'incoming' flag by From-address
from_is_not_self_addr
};
let mut parser = MimeMessage {
parts: Vec::new(),
headers,
@@ -664,7 +674,7 @@ impl MimeMessage {
from,
incoming,
chat_disposition_notification_to,
decrypting_failed: mail.is_err(),
decryption_error: mail.err().map(|err| format!("{err:#}")),
// only non-empty if it was a valid autocrypt message
signature,
@@ -905,7 +915,7 @@ impl MimeMessage {
&& let Some(ref subject) = self.get_subject()
{
let mut prepend_subject = true;
if !self.decrypting_failed {
if self.decryption_error.is_none() {
let colon = subject.find(':');
if colon == Some(2)
|| colon == Some(3)
@@ -946,7 +956,7 @@ impl MimeMessage {
self.parse_attachments();
// See if an MDN is requested from the other side
if !self.decrypting_failed
if self.decryption_error.is_none()
&& !self.parts.is_empty()
&& let Some(ref dn_to) = self.chat_disposition_notification_to
{
@@ -1078,7 +1088,7 @@ impl MimeMessage {
#[cfg(test)]
/// Returns whether the decrypted data contains the given `&str`.
pub(crate) fn decoded_data_contains(&self, s: &str) -> bool {
assert!(!self.decrypting_failed);
assert!(self.decryption_error.is_none());
let decoded_str = str::from_utf8(&self.decoded_data).unwrap();
decoded_str.contains(s)
}
+5 -1
View File
@@ -12,7 +12,7 @@ use tokio_io_timeout::TimeoutStream;
use crate::context::Context;
use crate::net::session::SessionStream;
use crate::net::tls::TlsSessionStore;
use crate::net::tls::{SpkiHashStore, TlsSessionStore};
use crate::sql::Sql;
use crate::tools::time;
@@ -130,6 +130,8 @@ pub(crate) async fn connect_tls_inner(
strict_tls: bool,
alpn: &str,
tls_session_store: &TlsSessionStore,
spki_hash_store: &SpkiHashStore,
sql: &Sql,
) -> Result<impl SessionStream + 'static> {
let use_sni = true;
let tcp_stream = connect_tcp_inner(addr).await?;
@@ -141,6 +143,8 @@ pub(crate) async fn connect_tls_inner(
alpn,
tcp_stream,
tls_session_store,
spki_hash_store,
sql,
)
.await?;
Ok(tls_stream)
+4
View File
@@ -87,6 +87,8 @@ where
"",
proxy_stream,
&context.tls_session_store,
&context.spki_hash_store,
&context.sql,
)
.await?;
Box::new(tls_stream)
@@ -99,6 +101,8 @@ where
"",
tcp_stream,
&context.tls_session_store,
&context.spki_hash_store,
&context.sql,
)
.await?;
Box::new(tls_stream)
+48 -9
View File
@@ -19,7 +19,6 @@ use tokio_io_timeout::TimeoutStream;
use url::Url;
use crate::config::Config;
use crate::constants::NON_ALPHANUMERIC_WITHOUT_DOT;
use crate::context::Context;
use crate::net::connect_tcp;
use crate::net::session::SessionStream;
@@ -93,13 +92,12 @@ impl HttpConfig {
}
fn to_url(&self, scheme: &str) -> String {
let host = utf8_percent_encode(&self.host, NON_ALPHANUMERIC_WITHOUT_DOT);
if let Some((user, password)) = &self.user_password {
let user = utf8_percent_encode(user, NON_ALPHANUMERIC);
let password = utf8_percent_encode(password, NON_ALPHANUMERIC);
format!("{scheme}://{user}:{password}@{host}:{}", self.port)
format!("{scheme}://{user}:{password}@{}:{}", self.host, self.port)
} else {
format!("{scheme}://{host}:{}", self.port)
format!("{scheme}://{}:{}", self.host, self.port)
}
}
}
@@ -143,13 +141,12 @@ impl Socks5Config {
}
fn to_url(&self) -> String {
let host = utf8_percent_encode(&self.host, NON_ALPHANUMERIC_WITHOUT_DOT);
if let Some((user, password)) = &self.user_password {
let user = utf8_percent_encode(user, NON_ALPHANUMERIC);
let password = utf8_percent_encode(password, NON_ALPHANUMERIC);
format!("socks5://{user}:{password}@{host}:{}", self.port)
format!("socks5://{user}:{password}@{}:{}", self.host, self.port)
} else {
format!("socks5://{host}:{}", self.port)
format!("socks5://{}:{}", self.host, self.port)
}
}
}
@@ -174,7 +171,6 @@ pub enum ProxyConfig {
}
/// Constructs HTTP/1.1 `CONNECT` request for HTTP(S) proxy.
#[expect(clippy::arithmetic_side_effects)]
fn http_connect_request(host: &str, port: u16, auth: Option<(&str, &str)>) -> String {
// According to <https://datatracker.ietf.org/doc/html/rfc7230#section-5.4>
// clients MUST send `Host:` header in HTTP/1.1 requests,
@@ -323,7 +319,6 @@ impl ProxyConfig {
/// config into `proxy_url` if `proxy_url` is unset or empty.
///
/// Unsets `socks5_host`, `socks5_port`, `socks5_user` and `socks5_password` in any case.
#[expect(clippy::arithmetic_side_effects)]
async fn migrate_socks_config(sql: &Sql) -> Result<()> {
if sql.get_raw_config("proxy_url").await?.is_none() {
// Load legacy SOCKS5 settings.
@@ -439,6 +434,8 @@ impl ProxyConfig {
"",
tcp_stream,
&context.tls_session_store,
&context.spki_hash_store,
&context.sql,
)
.await?;
let auth = if let Some((username, password)) = &https_config.user_password {
@@ -565,6 +562,20 @@ mod tests {
user_password: None
})
);
let proxy_config = ProxyConfig::from_url("socks5://my-proxy.example.org").unwrap();
assert_eq!(
proxy_config,
ProxyConfig::Socks5(Socks5Config {
host: "my-proxy.example.org".to_string(),
port: 1080,
user_password: None
})
);
assert_eq!(
proxy_config.to_url(),
"socks5://my-proxy.example.org:1080".to_string()
);
}
#[test]
@@ -598,6 +609,20 @@ mod tests {
user_password: None
})
);
let proxy_config = ProxyConfig::from_url("http://my-proxy.example.org").unwrap();
assert_eq!(
proxy_config,
ProxyConfig::Http(HttpConfig {
host: "my-proxy.example.org".to_string(),
port: 80,
user_password: None
})
);
assert_eq!(
proxy_config.to_url(),
"http://my-proxy.example.org:80".to_string()
);
}
#[test]
@@ -631,6 +656,20 @@ mod tests {
user_password: None
})
);
let proxy_config = ProxyConfig::from_url("https://my-proxy.example.org").unwrap();
assert_eq!(
proxy_config,
ProxyConfig::Https(HttpConfig {
host: "my-proxy.example.org".to_string(),
port: 443,
user_password: None
})
);
assert_eq!(
proxy_config.to_url(),
"https://my-proxy.example.org:443".to_string()
);
}
#[test]
+45 -16
View File
@@ -6,13 +6,20 @@ use std::sync::Arc;
use anyhow::Result;
use crate::net::session::SessionStream;
use crate::sql::Sql;
use crate::tools::time;
use tokio_rustls::rustls;
use tokio_rustls::rustls::client::ClientSessionStore;
use tokio_rustls::rustls::server::ParsedCertificate;
mod danger;
use danger::NoCertificateVerification;
use danger::CustomCertificateVerifier;
mod spki;
pub use spki::SpkiHashStore;
#[expect(clippy::too_many_arguments)]
pub async fn wrap_tls<'a>(
strict_tls: bool,
hostname: &str,
@@ -21,10 +28,21 @@ pub async fn wrap_tls<'a>(
alpn: &str,
stream: impl SessionStream + 'static,
tls_session_store: &TlsSessionStore,
spki_hash_store: &SpkiHashStore,
sql: &Sql,
) -> Result<impl SessionStream + 'a> {
if strict_tls {
let tls_stream =
wrap_rustls(hostname, port, use_sni, alpn, stream, tls_session_store).await?;
let tls_stream = wrap_rustls(
hostname,
port,
use_sni,
alpn,
stream,
tls_session_store,
spki_hash_store,
sql,
)
.await?;
let boxed_stream: Box<dyn SessionStream> = Box::new(tls_stream);
Ok(boxed_stream)
} else {
@@ -94,6 +112,7 @@ impl TlsSessionStore {
}
}
#[expect(clippy::too_many_arguments)]
pub async fn wrap_rustls<'a>(
hostname: &str,
port: u16,
@@ -101,9 +120,11 @@ pub async fn wrap_rustls<'a>(
alpn: &str,
stream: impl SessionStream + 'a,
tls_session_store: &TlsSessionStore,
spki_hash_store: &SpkiHashStore,
sql: &Sql,
) -> Result<impl SessionStream + 'a> {
let mut root_cert_store = rustls::RootCertStore::empty();
root_cert_store.extend(webpki_roots::TLS_SERVER_ROOTS.iter().cloned());
let root_cert_store =
rustls::RootCertStore::from_iter(webpki_roots::TLS_SERVER_ROOTS.iter().cloned());
let mut config = rustls::ClientConfig::builder()
.with_root_certificates(root_cert_store)
@@ -127,20 +148,28 @@ pub async fn wrap_rustls<'a>(
config.resumption = resumption;
config.enable_sni = use_sni;
// Do not verify certificates for hostnames starting with `_`.
// They are used for servers with self-signed certificates, e.g. for local testing.
// Hostnames starting with `_` can have only self-signed TLS certificates or wildcard certificates.
// It is not possible to get valid non-wildcard TLS certificates because CA/Browser Forum requirements
// explicitly state that domains should start with a letter, digit or hyphen:
// https://github.com/cabforum/servercert/blob/24f38fd4765e019db8bb1a8c56bf63c7115ce0b0/docs/BR.md
if hostname.starts_with("_") {
config
.dangerous()
.set_certificate_verifier(Arc::new(NoCertificateVerification::new()));
}
config
.dangerous()
.set_certificate_verifier(Arc::new(CustomCertificateVerifier::new(
spki_hash_store.get_spki_hash(hostname, sql).await?,
)));
let tls = tokio_rustls::TlsConnector::from(Arc::new(config));
let name = tokio_rustls::rustls::pki_types::ServerName::try_from(hostname)?.to_owned();
let tls_stream = tls.connect(name, stream).await?;
// Successfully connected.
// Remember SPKI hash to accept it later if certificate expires.
let (_io, client_connection) = tls_stream.get_ref();
if let Some(end_entity) = client_connection
.peer_certificates()
.and_then(|certs| certs.first())
{
let now = time();
let parsed_certificate = ParsedCertificate::try_from(end_entity)?;
let spki = parsed_certificate.subject_public_key_info();
spki_hash_store.save_spki(hostname, &spki, sql, now).await?;
}
Ok(tls_stream)
}
+78 -11
View File
@@ -1,26 +1,93 @@
//! Dangerous TLS implementation of accepting invalid certificates for Rustls.
//! Custom TLS verification.
//!
//! We want to accept expired certificates.
use rustls::RootCertStore;
use rustls::client::{verify_server_cert_signed_by_trust_anchor, verify_server_name};
use rustls::pki_types::{CertificateDer, ServerName, UnixTime};
use rustls::server::ParsedCertificate;
use tokio_rustls::rustls;
#[derive(Debug)]
pub(super) struct NoCertificateVerification();
use crate::net::tls::spki::spki_hash;
impl NoCertificateVerification {
pub(super) fn new() -> Self {
Self()
#[derive(Debug)]
pub(super) struct CustomCertificateVerifier {
/// Root certificates.
root_cert_store: RootCertStore,
/// Expected SPKI hash as a base64 of SHA-256.
spki_hash: Option<String>,
}
impl CustomCertificateVerifier {
pub(super) fn new(spki_hash: Option<String>) -> Self {
let root_cert_store =
RootCertStore::from_iter(webpki_roots::TLS_SERVER_ROOTS.iter().cloned());
Self {
root_cert_store,
spki_hash,
}
}
}
impl rustls::client::danger::ServerCertVerifier for NoCertificateVerification {
impl rustls::client::danger::ServerCertVerifier for CustomCertificateVerifier {
fn verify_server_cert(
&self,
_end_entity: &CertificateDer<'_>,
_intermediates: &[CertificateDer<'_>],
_server_name: &ServerName<'_>,
end_entity: &CertificateDer<'_>,
intermediates: &[CertificateDer<'_>],
server_name: &ServerName<'_>,
// OCSP is a certificate revocation mechanism that is intentionally ignored.
// It is practically not used and is essentially deprecated
// in favor of Certificate Revocation Lists.
// Let's Encrypt has disabled OCSP responders in 2025:
// <https://letsencrypt.org/2025/08/06/ocsp-service-has-reached-end-of-life>.
// Theoretically checking of stapled OCSP responses could be implemented,
// but it is not interesting to implement it because it is not used
// by the servers: <https://github.com/rustls/webpki/issues/217>.
_ocsp_response: &[u8],
_now: UnixTime,
now: UnixTime,
) -> Result<rustls::client::danger::ServerCertVerified, rustls::Error> {
let parsed_certificate = ParsedCertificate::try_from(end_entity)?;
let spki = parsed_certificate.subject_public_key_info();
let provider = rustls::crypto::ring::default_provider();
if let ServerName::DnsName(dns_name) = server_name
&& dns_name.as_ref().starts_with("_")
{
// Do not verify certificates for hostnames starting with `_`.
// They are used for servers with self-signed certificates, e.g. for local testing.
// Hostnames starting with `_` can have only self-signed TLS certificates or wildcard certificates.
// It is not possible to get valid non-wildcard TLS certificates because CA/Browser Forum requirements
// explicitly state that domains should start with a letter, digit or hyphen:
// https://github.com/cabforum/servercert/blob/24f38fd4765e019db8bb1a8c56bf63c7115ce0b0/docs/BR.md
} else if let Some(hash) = &self.spki_hash
&& spki_hash(&spki) == *hash
{
// Last time we successfully connected to this hostname with TLS checks,
// SPKI had this hash.
// It does not matter if certificate has now expired.
} else {
// verify_server_cert_signed_by_trust_anchor does no revocation checking:
// <https://docs.rs/rustls/0.23.37/rustls/client/fn.verify_server_cert_signed_by_trust_anchor.html>
// We don't do it either.
verify_server_cert_signed_by_trust_anchor(
&parsed_certificate,
&self.root_cert_store,
intermediates,
now,
provider.signature_verification_algorithms.all,
)?;
}
// Verify server name unconditionally.
//
// We do this even for self-signed certificates when hostname starts with `_`
// so we don't try to connect to captive portals
// and fail on MITM certificates if they are generated once
// and reused for all hostnames.
verify_server_name(&parsed_certificate, server_name)?;
Ok(rustls::client::danger::ServerCertVerified::assertion())
}
+131
View File
@@ -0,0 +1,131 @@
//! SPKI hash storage.
//!
//! We store hashes of Subject Public Key Info from TLS certificates
//! after successful connection to allow connecting when
//! server certificate expires as long as the key is not changed.
use std::collections::BTreeMap;
use anyhow::Context as _;
use anyhow::Result;
use base64::Engine as _;
use parking_lot::RwLock;
use sha2::{Digest, Sha256};
use tokio_rustls::rustls::pki_types::SubjectPublicKeyInfoDer;
use crate::sql::Sql;
use crate::tools::time;
/// Calculates Subject Public Key Info SHA-256 hash and returns it as base64.
///
/// This is the same format as used in <https://www.rfc-editor.org/rfc/rfc7469>.
/// You can calculate the same hash for any remote host with
/// ```sh
/// openssl s_client -connect "$HOST:993" -servername "$HOST" </dev/null 2>/dev/null |
/// openssl x509 -pubkey -noout |
/// openssl pkey -pubin -outform der |
/// openssl dgst -sha256 -binary |
/// openssl enc -base64
/// ```
pub fn spki_hash(spki: &SubjectPublicKeyInfoDer) -> String {
let spki_hash = Sha256::digest(spki);
base64::engine::general_purpose::STANDARD.encode(spki_hash)
}
/// Write-through cache for SPKI hashes.
#[derive(Debug)]
pub struct SpkiHashStore {
/// Map from hostnames to base64 of SHA-256 hashes.
pub hash_store: RwLock<BTreeMap<String, String>>,
}
impl SpkiHashStore {
pub fn new() -> Self {
Self {
hash_store: RwLock::new(BTreeMap::new()),
}
}
/// Returns base64 of SPKI hash if we have previously successfully connected to given hostname.
pub async fn get_spki_hash(&self, hostname: &str, sql: &Sql) -> Result<Option<String>> {
if let Some(hash) = self.hash_store.read().get(hostname).cloned() {
return Ok(Some(hash));
}
match sql
.query_row_optional(
"SELECT spki_hash FROM tls_spki WHERE host=?",
(hostname,),
|row| {
let spki_hash: String = row.get(0)?;
Ok(spki_hash)
},
)
.await?
{
Some(hash) => {
self.hash_store
.write()
.insert(hostname.to_string(), hash.clone());
Ok(Some(hash))
}
None => Ok(None),
}
}
/// Saves SPKI hash after successful connection.
pub async fn save_spki(
&self,
hostname: &str,
spki: &SubjectPublicKeyInfoDer<'_>,
sql: &Sql,
timestamp: i64,
) -> Result<()> {
let hash = spki_hash(spki);
self.hash_store
.write()
.insert(hostname.to_string(), hash.clone());
sql.execute(
"INSERT OR REPLACE INTO tls_spki (host, spki_hash, timestamp) VALUES (?, ?, ?)",
(hostname, hash, timestamp),
)
.await?;
Ok(())
}
/// Removes stale entries from SPKI storage.
pub async fn cleanup(&self, sql: &Sql) -> Result<()> {
let now = time();
let removed_hosts = sql
.query_map_vec(
"DELETE FROM tls_spki WHERE ? > timestamp + ? RETURNING host",
(now, 30 * 24 * 60 * 60),
|row| {
let host: String = row.get(0)?;
Ok(host)
},
)
.await
.context("DELETE FROM tls_spki")?;
// Fix timestamps that happen to be in the future
// if we had clock set incorrectly when the timestamp was stored.
// Otherwise entry may take more than 30 days to expire.
sql.execute(
"UPDATE tls_spki SET timestamp = ?1 WHERE timestamp > ?1",
(now,),
)
.await?;
let mut lock = self.hash_store.write();
for host in removed_hosts {
// We may accidentally remove a host that was added
// to the cache after SQL query but before we got
// the write lock on `hash_store`.
// It is unlikely and will only result
// in additional SQL query next time.
lock.remove(&host);
}
Ok(())
}
}
-1
View File
@@ -24,7 +24,6 @@ pub struct PlainText {
impl PlainText {
/// Convert plain text to HTML.
/// The function handles quotes, links, fixed and floating text paragraphs.
#[expect(clippy::arithmetic_side_effects)]
pub fn to_html(&self) -> String {
static LINKIFY_MAIL_RE: LazyLock<regex::Regex> =
LazyLock::new(|| regex::Regex::new(r"\b([\w.\-+]+@[\w.\-]+)\b").unwrap());
+4 -5
View File
@@ -17,7 +17,7 @@ use crate::config::Config;
use crate::contact::{Contact, ContactId, Origin};
use crate::context::Context;
use crate::key::Fingerprint;
use crate::login_param::{EnteredCertificateChecks, EnteredLoginParam, EnteredServerLoginParam};
use crate::login_param::{EnteredCertificateChecks, EnteredImapLoginParam, EnteredLoginParam};
use crate::net::http::post_empty;
use crate::net::proxy::{DEFAULT_SOCKS_PORT, ProxyConfig};
use crate::token;
@@ -41,7 +41,7 @@ pub(crate) const DCBACKUP_SCHEME_PREFIX: &str = "DCBACKUP";
/// Version written to Backups and Backup-QR-Codes.
/// Imports will fail when they have a larger version.
pub(crate) const DCBACKUP_VERSION: i32 = 4;
pub(crate) const DCBACKUP_VERSION: i32 = 5;
/// Scanned QR code.
#[derive(Debug, Clone, PartialEq, Eq)]
@@ -714,7 +714,6 @@ fn decode_account(qr: &str) -> Result<Qr> {
}
/// scheme: `https://t.me/socks?server=foo&port=123` or `https://t.me/socks?server=1.2.3.4&port=123`
#[expect(clippy::arithmetic_side_effects)]
fn decode_tg_socks_proxy(_context: &Context, qr: &str) -> Result<Qr> {
let url = url::Url::parse(qr).context("Invalid t.me/socks url")?;
@@ -824,7 +823,7 @@ pub(crate) async fn login_param_from_account_qr(
let param = EnteredLoginParam {
addr,
imap: EnteredServerLoginParam {
imap: EnteredImapLoginParam {
password,
..Default::default()
},
@@ -844,7 +843,7 @@ pub(crate) async fn login_param_from_account_qr(
let param = EnteredLoginParam {
addr: email,
imap: EnteredServerLoginParam {
imap: EnteredImapLoginParam {
password,
..Default::default()
},
+6 -3
View File
@@ -5,7 +5,9 @@ use anyhow::{Context as _, Result, bail};
use deltachat_contact_tools::may_be_valid_addr;
use super::{DCLOGIN_SCHEME, Qr};
use crate::login_param::{EnteredCertificateChecks, EnteredLoginParam, EnteredServerLoginParam};
use crate::login_param::{
EnteredCertificateChecks, EnteredImapLoginParam, EnteredLoginParam, EnteredSmtpLoginParam,
};
use crate::provider::Socket;
/// Options for `dclogin:` scheme.
@@ -178,14 +180,15 @@ pub(crate) fn login_param_from_login_qr(
} => {
let param = EnteredLoginParam {
addr: addr.to_string(),
imap: EnteredServerLoginParam {
imap: EnteredImapLoginParam {
server: imap_host.unwrap_or_default(),
port: imap_port.unwrap_or_default(),
folder: "INBOX".to_string(),
security: imap_security.unwrap_or_default(),
user: imap_username.unwrap_or_default(),
password: imap_password.unwrap_or(mail_pw),
},
smtp: EnteredServerLoginParam {
smtp: EnteredSmtpLoginParam {
server: smtp_host.unwrap_or_default(),
port: smtp_port.unwrap_or_default(),
security: smtp_security.unwrap_or_default(),
+26
View File
@@ -892,6 +892,32 @@ async fn test_set_proxy_config_from_qr() -> Result<()> {
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_dont_encode_hyphen_in_proxy_hostnames() -> Result<()> {
let mut tcm = TestContextManager::new();
let t = &tcm.alice().await;
let qr_text = "socks5://my-proxy.example.org";
let qr = check_qr(t, qr_text).await?;
assert_eq!(
qr,
Qr::Proxy {
url: "socks5://my-proxy.example.org".to_string(),
host: "my-proxy.example.org".to_string(),
port: 1080,
}
);
set_config_from_qr(t, "socks5://my-proxy.example.org").await?;
assert_eq!(
t.get_config(Config::ProxyUrl).await?,
Some("socks5://my-proxy.example.org:1080".to_string())
);
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_decode_shadowsocks() -> Result<()> {
let ctx = TestContext::new().await;
-2
View File
@@ -15,7 +15,6 @@ use crate::securejoin;
use crate::stock_str::{self, backup_transfer_qr};
/// Create a QR code from any input data.
#[expect(clippy::arithmetic_side_effects)]
pub fn create_qr_svg(qrcode_content: &str) -> Result<String> {
let all_size = 512.0;
let qr_code_size = 416.0;
@@ -176,7 +175,6 @@ async fn self_info(context: &Context) -> Result<(Option<Vec<u8>>, String, String
Ok((avatar, displayname, addr, color))
}
#[expect(clippy::arithmetic_side_effects)]
fn inner_generate_secure_join_qr_code(
qrcode_description: &str,
qrcode_content: &str,
+22 -22
View File
@@ -9,7 +9,6 @@ use async_imap::types::{Quota, QuotaResource};
use crate::chat::add_device_msg_with_importance;
use crate::config::Config;
use crate::context::Context;
use crate::imap::get_watched_folders;
use crate::imap::session::Session as ImapSession;
use crate::log::warn;
use crate::message::Message;
@@ -48,26 +47,24 @@ pub struct QuotaInfo {
async fn get_unique_quota_roots_and_usage(
session: &mut ImapSession,
folders: Vec<String>,
folder: &str,
) -> Result<BTreeMap<String, Vec<QuotaResource>>> {
let mut unique_quota_roots: BTreeMap<String, Vec<QuotaResource>> = BTreeMap::new();
for folder in folders {
let (quota_roots, quotas) = &session.get_quota_root(&folder).await?;
// if there are new quota roots found in this imap folder, add them to the list
for qr_entries in quota_roots {
for quota_root_name in &qr_entries.quota_root_names {
// the quota for that quota root
let quota: Quota = quotas
.iter()
.find(|q| &q.root_name == quota_root_name)
.cloned()
.context("quota_root should have a quota")?;
// replace old quotas, because between fetching quotaroots for folders,
// messages could be received and so the usage could have been changed
*unique_quota_roots
.entry(quota_root_name.clone())
.or_default() = quota.resources;
}
let (quota_roots, quotas) = &session.get_quota_root(folder).await?;
// if there are new quota roots found in this imap folder, add them to the list
for qr_entries in quota_roots {
for quota_root_name in &qr_entries.quota_root_names {
// the quota for that quota root
let quota: Quota = quotas
.iter()
.find(|q| &q.root_name == quota_root_name)
.cloned()
.context("quota_root should have a quota")?;
// replace old quotas, because between fetching quotaroots for folders,
// messages could be received and so the usage could have been changed
*unique_quota_roots
.entry(quota_root_name.clone())
.or_default() = quota.resources;
}
}
Ok(unique_quota_roots)
@@ -122,14 +119,17 @@ impl Context {
/// As the message is added only once, the user is not spammed
/// in case for some providers the quota is always at ~100%
/// and new space is allocated as needed.
pub(crate) async fn update_recent_quota(&self, session: &mut ImapSession) -> Result<()> {
pub(crate) async fn update_recent_quota(
&self,
session: &mut ImapSession,
folder: &str,
) -> Result<()> {
let transport_id = session.transport_id();
info!(self, "Transport {transport_id}: Updating quota.");
let quota = if session.can_check_quota() {
let folders = get_watched_folders(self).await?;
get_unique_quota_roots_and_usage(session, folders).await
get_unique_quota_roots_and_usage(session, folder).await
} else {
Err(anyhow!(stock_str::not_supported_by_provider(self)))
};
+44 -102
View File
@@ -1,6 +1,6 @@
//! # Reactions.
//!
//! Reactions are short messages consisting of emojis sent in reply to
//! Reactions are short messages representing an emoji sent in reply to
//! messages. Unlike normal messages which are added to the end of the chat,
//! reactions are supposed to be displayed near the original messages.
//!
@@ -11,7 +11,7 @@
//! [XEP-0444](https://xmpp.org/extensions/xep-0444.html) section
//! "3.2 Updating reactions to a message". Received reactions override
//! all previously received reactions from the same user and it is
//! possible to remove all reactions by sending an empty string as a reaction,
//! possible to remove the reaction by sending an empty string as a reaction,
//! even though RFC 9078 requires at least one emoji to be sent.
use std::cmp::Ordering;
@@ -29,9 +29,7 @@ use crate::events::EventType;
use crate::message::{Message, MsgId, rfc724_mid_exists};
use crate::param::Param;
/// A single reaction consisting of multiple emoji sequences.
///
/// It is guaranteed to have all emojis sorted and deduplicated inside.
/// A single reaction.
#[derive(Debug, Default, Clone, Deserialize, Eq, PartialEq, Serialize)]
pub struct Reaction {
/// Canonical representation of reaction as a string of space-separated emojis.
@@ -42,11 +40,8 @@ pub struct Reaction {
// FromStr requires error type and reaction parsing never returns an
// error.
impl From<&str> for Reaction {
/// Parses a string containing a reaction.
///
/// Reaction string is separated by spaces or tabs (`WSP` in ABNF),
/// but this function accepts any ASCII whitespace, so even a CRLF at
/// the end of string is acceptable.
/// Convert a `&str` into a `Reaction`.
/// Everything after the first whitespace is ignored.
///
/// Any short enough string is accepted as a reaction to avoid the
/// complexity of validating emoji sequences as required by RFC
@@ -57,42 +52,27 @@ impl From<&str> for Reaction {
/// such as sending large numbers of large messages, and should be
/// dealt with the same way, e.g. by blocking the user.
fn from(reaction: &str) -> Self {
let mut emojis: Vec<&str> = reaction
let reaction: &str = reaction
.split_ascii_whitespace()
.next()
.filter(|&emoji| emoji.len() < 30)
.collect();
emojis.sort_unstable();
emojis.dedup();
let reaction = emojis.join(" ");
Self { reaction }
.unwrap_or("");
Self {
reaction: reaction.to_string(),
}
}
}
impl Reaction {
/// Returns true if reaction contains no emojis.
/// Returns true if reaction contains no emoji.
pub fn is_empty(&self) -> bool {
self.reaction.is_empty()
}
/// Returns a vector of emojis composing a reaction.
pub fn emojis(&self) -> Vec<&str> {
self.reaction.split(' ').collect()
}
/// Returns space-separated string of emojis
/// Returns a string representing the emoji.
pub fn as_str(&self) -> &str {
&self.reaction
}
/// Appends emojis from another reaction to this reaction.
pub fn add(&self, other: Self) -> Self {
let mut emojis: Vec<&str> = self.emojis();
emojis.append(&mut other.emojis());
emojis.sort_unstable();
emojis.dedup();
let reaction = emojis.join(" ");
Self { reaction }
}
}
/// Structure representing all reactions to a particular message.
@@ -126,12 +106,10 @@ impl Reactions {
pub fn emoji_frequencies(&self) -> BTreeMap<String, usize> {
let mut emoji_frequencies: BTreeMap<String, usize> = BTreeMap::new();
for reaction in self.reactions.values() {
for emoji in reaction.emojis() {
emoji_frequencies
.entry(emoji.to_string())
.and_modify(|x| *x += 1)
.or_insert(1);
}
emoji_frequencies
.entry(reaction.as_str().to_string())
.and_modify(|x| *x += 1)
.or_insert(1);
}
emoji_frequencies
}
@@ -152,6 +130,11 @@ impl Reactions {
});
emoji_frequencies
}
/// Returns an iterator of the contacts that reacted and their corresponding reactions.
pub fn iter(&self) -> impl Iterator<Item = (&ContactId, &Reaction)> {
self.reactions.iter()
}
}
impl fmt::Display for Reactions {
@@ -223,7 +206,7 @@ async fn set_msg_id_reaction(
/// Sends a reaction to message `msg_id`, overriding previously sent reactions.
///
/// `reaction` is a string consisting of space-separated emoji. Use
/// `reaction` is a string consisting of a single emoji. Use
/// empty string to retract a reaction.
pub async fn send_reaction(context: &Context, msg_id: MsgId, reaction: &str) -> Result<MsgId> {
let msg = Message::load_from_db(context, msg_id).await?;
@@ -251,24 +234,12 @@ pub async fn send_reaction(context: &Context, msg_id: MsgId, reaction: &str) ->
Ok(reaction_msg_id)
}
/// Adds given reaction to message `msg_id` and sends an update.
///
/// This can be used to implement advanced clients that allow reacting
/// with multiple emojis. For a simple messenger UI, you probably want
/// to use [`send_reaction()`] instead so reacting with a new emoji
/// removes previous emoji at the same time.
pub async fn add_reaction(context: &Context, msg_id: MsgId, reaction: &str) -> Result<MsgId> {
let self_reaction = get_self_reaction(context, msg_id).await?;
let reaction = self_reaction.add(Reaction::from(reaction));
send_reaction(context, msg_id, reaction.as_str()).await
}
/// Updates reaction of `contact_id` on the message with `in_reply_to`
/// Message-ID. If no such message is found in the database, reaction
/// is ignored.
///
/// `reaction` is a space-separated string of emojis. It can be empty
/// if contact wants to remove all reactions.
/// `reaction` is string representing the emoji. It can be empty
/// if contact wants to remove the reaction.
pub(crate) async fn set_msg_reaction(
context: &Context,
in_reply_to: &str,
@@ -301,26 +272,9 @@ pub(crate) async fn set_msg_reaction(
Ok(())
}
/// Get our own reaction for a given message.
async fn get_self_reaction(context: &Context, msg_id: MsgId) -> Result<Reaction> {
let reaction_str: Option<String> = context
.sql
.query_get_value(
"SELECT reaction
FROM reactions
WHERE msg_id=? AND contact_id=?",
(msg_id, ContactId::SELF),
)
.await?;
Ok(reaction_str
.as_deref()
.map(Reaction::from)
.unwrap_or_default())
}
/// Returns a structure containing all reactions to the message.
pub async fn get_msg_reactions(context: &Context, msg_id: MsgId) -> Result<Reactions> {
let reactions: BTreeMap<ContactId, Reaction> = context
let mut reactions: BTreeMap<ContactId, Reaction> = context
.sql
.query_map_collect(
"SELECT contact_id, reaction FROM reactions WHERE msg_id=?",
@@ -332,6 +286,7 @@ pub async fn get_msg_reactions(context: &Context, msg_id: MsgId) -> Result<React
},
)
.await?;
reactions.retain(|_contact, reaction| !reaction.is_empty());
Ok(Reactions { reactions })
}
@@ -406,46 +361,32 @@ mod tests {
#[test]
fn test_parse_reaction() {
// Check that basic set of emojis from RFC 9078 is supported.
assert_eq!(Reaction::from("👍").emojis(), vec!["👍"]);
assert_eq!(Reaction::from("👎").emojis(), vec!["👎"]);
assert_eq!(Reaction::from("😀").emojis(), vec!["😀"]);
assert_eq!(Reaction::from("☹").emojis(), vec!["☹"]);
assert_eq!(Reaction::from("😢").emojis(), vec!["😢"]);
assert_eq!(Reaction::from("👍").as_str(), "👍");
assert_eq!(Reaction::from("👎").as_str(), "👎");
assert_eq!(Reaction::from("😀").as_str(), "😀");
assert_eq!(Reaction::from("☹").as_str(), "☹");
assert_eq!(Reaction::from("😢").as_str(), "😢");
// Empty string can be used to remove all reactions.
assert!(Reaction::from("").is_empty());
// Short strings can be used as emojis, could be used to add
// support for custom emojis via emoji shortcodes.
assert_eq!(Reaction::from(":deltacat:").emojis(), vec![":deltacat:"]);
assert_eq!(Reaction::from(":deltacat:").as_str(), ":deltacat:");
// Check that long strings are not valid emojis.
assert!(
Reaction::from(":foobarbazquuxaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa:").is_empty()
);
// Multiple reactions separated by spaces or tabs are supported.
assert_eq!(Reaction::from("👍 ❤").emojis(), vec!["❤", "👍"]);
assert_eq!(Reaction::from("👍\t❤").emojis(), vec!["❤", "👍"]);
// Multiple reactions separated by spaces or tabs are not supported.
assert_eq!(Reaction::from("👍 ❤").as_str(), "👍");
assert_eq!(Reaction::from("👍\t❤").as_str(), "👍");
// Invalid emojis are removed, but valid emojis are retained.
assert_eq!(
Reaction::from("👍\t:foo: ❤").emojis(),
vec![":foo:", "❤", "👍"]
);
assert_eq!(Reaction::from("👍\t:foo: ❤").as_str(), ":foo: ❤ 👍");
assert_eq!(Reaction::from("👍\t:foo: ❤").as_str(), "👍");
assert_eq!(Reaction::from("👍\t:foo: ❤").as_str(), "👍");
// Duplicates are removed.
assert_eq!(Reaction::from("👍 👍").emojis(), vec!["👍"]);
}
#[test]
fn test_add_reaction() {
let reaction1 = Reaction::from("👍 😀");
let reaction2 = Reaction::from("❤");
let reaction_sum = reaction1.add(reaction2);
assert_eq!(reaction_sum.emojis(), vec!["❤", "👍", "😀"]);
assert_eq!(Reaction::from("👍 👍").as_str(), "👍");
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
@@ -511,7 +452,7 @@ Content-Disposition: reaction\n\
assert_eq!(contacts.first(), Some(&bob_id));
let bob_reaction = reactions.get(bob_id);
assert_eq!(bob_reaction.is_empty(), false);
assert_eq!(bob_reaction.emojis(), vec!["👍"]);
assert_eq!(bob_reaction.as_str(), "👍");
assert_eq!(bob_reaction.as_str(), "👍");
// Alice receives reaction to her message from Bob with a footer.
@@ -730,7 +671,7 @@ Content-Disposition: reaction\n\
let bob_id = contacts.first().unwrap();
let bob_reaction = reactions.get(*bob_id);
assert_eq!(bob_reaction.is_empty(), false);
assert_eq!(bob_reaction.emojis(), vec!["👍"]);
assert_eq!(bob_reaction.as_str(), "👍");
assert_eq!(bob_reaction.as_str(), "👍");
expect_reactions_changed_event(&alice, chat_alice.id, alice_msg.sender_msg_id, *bob_id)
.await?;
@@ -769,15 +710,16 @@ Content-Disposition: reaction\n\
);
// Alice reacts to own message.
// Trying to set multiple reactions at once is not allowed.
send_reaction(&alice, alice_msg.sender_msg_id, "👍 😀")
.await
.unwrap();
let reactions = get_msg_reactions(&alice, alice_msg.sender_msg_id).await?;
assert_eq!(reactions.to_string(), "👍2 😀1");
assert_eq!(reactions.to_string(), "👍2");
assert_eq!(
reactions.emoji_sorted_by_frequency(),
vec![("👍".to_string(), 2), ("😀".to_string(), 1)]
vec![("👍".to_string(), 2)]
);
Ok(())
+199 -117
View File
@@ -1,7 +1,7 @@
//! Internet Message Format reception pipeline.
use std::cmp;
use std::collections::{BTreeMap, BTreeSet, HashSet};
use std::collections::{BTreeMap, BTreeSet};
use std::iter;
use std::sync::LazyLock;
@@ -14,7 +14,9 @@ use mailparse::SingleInfo;
use num_traits::FromPrimitive;
use regex::Regex;
use crate::chat::{self, Chat, ChatId, ChatIdBlocked, ChatVisibility, save_broadcast_secret};
use crate::chat::{
self, Chat, ChatId, ChatIdBlocked, ChatVisibility, is_contact_in_chat, save_broadcast_secret,
};
use crate::config::Config;
use crate::constants::{self, Blocked, Chattype, DC_CHAT_ID_TRASH, EDITED_PREFIX, ShowEmails};
use crate::contact::{self, Contact, ContactId, Origin, mark_contact_id_as_verified};
@@ -45,6 +47,7 @@ use crate::securejoin::{
self, get_secure_join_step, handle_securejoin_handshake, observe_securejoin_on_other_device,
};
use crate::simplify;
use crate::smtp::msg_has_pending_smtp_job;
use crate::stats::STATISTICS_BOT_EMAIL;
use crate::stock_str;
use crate::sync::Sync::*;
@@ -582,14 +585,7 @@ pub(crate) async fn receive_imf_inner(
(rfc724_mid_orig, &self_addr),
)
.await?;
if !context
.sql
.exists(
"SELECT COUNT(*) FROM smtp WHERE rfc724_mid=?",
(rfc724_mid_orig,),
)
.await?
{
if !msg_has_pending_smtp_job(context, msg_id).await? {
msg_id.set_delivered(context).await?;
}
return Ok(None);
@@ -727,7 +723,7 @@ pub(crate) async fn receive_imf_inner(
let show_emails = ShowEmails::from_i32(context.get_config_int(Config::ShowEmails).await?)
.unwrap_or_default();
let allow_creation = if mime_parser.decrypting_failed {
let allow_creation = if mime_parser.decryption_error.is_some() {
false
} else if is_dc_message == MessengerMessage::No
&& !context.get_config_bool(Config::IsChatmail).await?
@@ -777,6 +773,7 @@ pub(crate) async fn receive_imf_inner(
chat_id,
chat_id_blocked,
is_dc_message,
is_created,
)
.await
.context("add_parts error")?
@@ -882,11 +879,23 @@ UPDATE config SET value=? WHERE keyname='configured_addr' AND value!=?1
.is_some_and(|part| part.typ == Viewtype::Webxdc)
{
can_info_msg = false;
Some(
Message::load_from_db(context, insert_msg_id)
.await
.context("Failed to load just created webxdc instance")?,
)
if mime_parser.pre_message == PreMessageMode::Post
&& let Some(msg_id) = message::rfc724_mid_exists(context, rfc724_mid_orig).await?
{
// The messsage is a post-message and pre-message exists.
// Assign status update to existing message because just received post-message will be trashed.
Some(
Message::load_from_db(context, msg_id)
.await
.context("Failed to load webxdc instance that we just checked exists")?,
)
} else {
Some(
Message::load_from_db(context, insert_msg_id)
.await
.context("Failed to load just created webxdc instance")?,
)
}
} else if let Some(field) = mime_parser.get_header(HeaderDef::InReplyTo) {
if let Some(instance) =
message::get_by_rfc724_mids(context, &parse_message_ids(field)).await?
@@ -1010,11 +1019,11 @@ UPDATE config SET value=? WHERE keyname='configured_addr' AND value!=?1
&& msg.chat_visibility == ChatVisibility::Archived;
updated_chats
.entry(msg.chat_id)
.and_modify(|ts| *ts = cmp::max(*ts, msg.timestamp_sort))
.or_insert(msg.timestamp_sort);
.and_modify(|pos| *pos = cmp::max(*pos, (msg.timestamp_sort, msg.id)))
.or_insert((msg.timestamp_sort, msg.id));
}
}
for (chat_id, timestamp_sort) in updated_chats {
for (chat_id, (timestamp_sort, msg_id)) in updated_chats {
context
.sql
.execute(
@@ -1023,12 +1032,13 @@ UPDATE msgs SET state=? WHERE
state=? AND
hidden=0 AND
chat_id=? AND
timestamp<?",
(timestamp,id)<(?,?)",
(
MessageState::InNoticed,
MessageState::InFresh,
chat_id,
timestamp_sort,
msg_id,
),
)
.await
@@ -1062,7 +1072,12 @@ UPDATE msgs SET state=? WHERE
let fresh = received_msg.state == MessageState::InFresh
&& mime_parser.is_system_message != SystemMessage::CallAccepted
&& mime_parser.is_system_message != SystemMessage::CallEnded;
let important = mime_parser.incoming && fresh && !is_old_contact_request;
let is_bot = context.get_config_bool(Config::Bot).await?;
let is_pre_message = matches!(mime_parser.pre_message, PreMessageMode::Pre { .. });
let skip_bot_notify = is_bot && is_pre_message;
let important =
mime_parser.incoming && fresh && !is_old_contact_request && !skip_bot_notify;
for msg_id in &received_msg.msg_ids {
chat_id.emit_msg_event(context, *msg_id, important);
}
@@ -1211,14 +1226,18 @@ async fn decide_chat_assignment(
{
info!(context, "Call state changed (TRASH).");
true
} else if mime_parser.decrypting_failed && !mime_parser.incoming {
} else if let Some(ref decryption_error) = mime_parser.decryption_error
&& !mime_parser.incoming
{
// Outgoing undecryptable message.
let last_time = context
.get_config_i64(Config::LastCantDecryptOutgoingMsgs)
.await?;
let now = tools::time();
let update_config = if last_time.saturating_add(24 * 60 * 60) <= now {
let txt = "⚠️ It seems you are using Delta Chat on multiple devices that cannot decrypt each other's outgoing messages. To fix this, on the older device use \"Settings / Add Second Device\" and follow the instructions.";
let txt = format!(
"⚠️ It seems you are using Delta Chat on multiple devices that cannot decrypt each other's outgoing messages. To fix this, on the older device use \"Settings / Add Second Device\" and follow the instructions. (Error: {decryption_error}, {rfc724_mid})."
);
let mut msg = Message::new_text(txt.to_string());
chat::add_device_msg(context, None, Some(&mut msg))
.await
@@ -1294,8 +1313,9 @@ async fn decide_chat_assignment(
..
} = &mime_parser.pre_message
{
let msg_id = rfc724_mid_exists(context, post_msg_rfc724_mid).await?;
if let Some(msg_id) = msg_id {
let post_msg_exists = if let Some((msg_id, not_downloaded)) =
message::rfc724_mid_exists_ex(context, post_msg_rfc724_mid, "download_state<>0").await?
{
context
.sql
.execute(
@@ -1303,8 +1323,10 @@ async fn decide_chat_assignment(
(rfc724_mid, msg_id),
)
.await?;
}
let post_msg_exists = msg_id.is_some();
!not_downloaded
} else {
false
};
info!(
context,
"Message {rfc724_mid} is a pre-message for {post_msg_rfc724_mid} (post_msg_exists:{post_msg_exists})."
@@ -1791,6 +1813,7 @@ async fn add_parts(
mut chat_id: ChatId,
mut chat_id_blocked: Blocked,
is_dc_message: MessengerMessage,
is_chat_created: bool,
) -> Result<ReceivedMsg> {
let to_id = if mime_parser.incoming {
ContactId::SELF
@@ -1835,6 +1858,16 @@ async fn add_parts(
}
}
// Sort message to the bottom if we are not in the chat
// so if we are added via QR code scan
// the message about our addition goes after all the info messages.
// Info messages are sorted by local smeared_timestamp()
// which advances quickly during SecureJoin,
// while "member added" message may have older timestamp
// corresponding to the sender clock.
// In practice inviter clock may even be slightly in the past.
let sort_to_bottom = !chat.is_self_in_chat(context).await?;
let is_location_kml = mime_parser.location_kml.is_some();
let mut group_changes = match chat.typ {
_ if chat.id.is_special() => GroupChangesInfo::default(),
@@ -1844,7 +1877,16 @@ async fn add_parts(
apply_out_broadcast_changes(context, mime_parser, &mut chat, from_id).await?
}
Chattype::Group => {
apply_group_changes(context, mime_parser, &mut chat, from_id, to_ids, past_ids).await?
apply_group_changes(
context,
mime_parser,
&mut chat,
from_id,
to_ids,
past_ids,
is_chat_created,
)
.await?
}
Chattype::InBroadcast => {
apply_in_broadcast_changes(context, mime_parser, &mut chat, from_id).await?
@@ -1885,16 +1927,8 @@ async fn add_parts(
};
let in_fresh = state == MessageState::InFresh;
let sort_to_bottom = false;
let received = true;
let sort_timestamp = chat_id
.calc_sort_timestamp(
context,
mime_parser.timestamp_sent,
sort_to_bottom,
received,
mime_parser.incoming,
)
.calc_sort_timestamp(context, mime_parser.timestamp_sent, sort_to_bottom)
.await?;
// Apply ephemeral timer changes to the chat.
@@ -1907,7 +1941,7 @@ async fn add_parts(
&& chat_id.get_ephemeral_timer(context).await? != ephemeral_timer
{
let chat_contacts =
HashSet::<ContactId>::from_iter(chat::get_chat_contacts(context, chat_id).await?);
BTreeSet::<ContactId>::from_iter(chat::get_chat_contacts(context, chat_id).await?);
let is_from_in_chat =
!chat_contacts.contains(&ContactId::SELF) || chat_contacts.contains(&from_id);
@@ -2290,7 +2324,7 @@ RETURNING id
if trash { 0 } else { ephemeral_timestamp },
if trash {
DownloadState::Done
} else if mime_parser.decrypting_failed {
} else if mime_parser.decryption_error.is_some() {
DownloadState::Undecipherable
} else if let PreMessageMode::Pre {..} = mime_parser.pre_message {
DownloadState::Available
@@ -2354,7 +2388,7 @@ RETURNING id
info!(
context,
"Message has {icnt} parts and is assigned to chat #{chat_id}."
"Message has {icnt} parts and is assigned to chat #{chat_id}, timestamp={sort_timestamp}."
);
if !chat_id.is_trash() && !hidden {
@@ -2442,7 +2476,7 @@ async fn handle_edit_delete(
// See `message::delete_msgs_ex()`, unlike edit requests, DC doesn't send unencrypted
// deletion requests, so there's no need to support them.
if part.param.get_bool(Param::GuaranteeE2ee).unwrap_or(false) {
let mut modified_chat_ids = HashSet::new();
let mut modified_chat_ids = BTreeSet::new();
let mut msg_ids = Vec::new();
let rfc724_mid_vec: Vec<&str> = rfc724_mid_list.split_whitespace().collect();
@@ -2560,7 +2594,22 @@ WHERE id=?
),
)
.await?;
context.emit_msgs_changed(original_msg.chat_id, original_msg.id);
if context.get_config_bool(Config::Bot).await? {
if original_msg.hidden {
// No need to emit an event about the changed message
} else if !original_msg.chat_id.is_trash() {
let fresh = original_msg.state == MessageState::InFresh;
let important = mime_parser.incoming && fresh;
original_msg
.chat_id
.emit_msg_event(context, original_msg.id, important);
context.new_msgs_notify.notify_one();
}
} else {
context.emit_msgs_changed(original_msg.chat_id, original_msg.id);
}
Ok(())
}
@@ -2703,7 +2752,7 @@ async fn lookup_or_create_adhoc_group(
allow_creation: bool,
create_blocked: Blocked,
) -> Result<Option<(ChatId, Blocked, bool)>> {
if mime_parser.decrypting_failed {
if mime_parser.decryption_error.is_some() {
warn!(
context,
"Not creating ad-hoc group for message that cannot be decrypted."
@@ -2854,8 +2903,7 @@ async fn create_group(
let mut chat_id = None;
let mut chat_id_blocked = Default::default();
if chat_id.is_none()
&& !mime_parser.is_mailinglist_message()
if !mime_parser.is_mailinglist_message()
&& !grpid.is_empty()
&& mime_parser.get_header(HeaderDef::ChatGroupName).is_some()
// otherwise, a pending "quit" message may pop up
@@ -2925,7 +2973,7 @@ async fn create_group(
if let Some(chat_id) = chat_id {
Ok(Some((chat_id, chat_id_blocked)))
} else if mime_parser.decrypting_failed {
} else if mime_parser.decryption_error.is_some() {
// It is possible that the message was sent to a valid,
// yet unknown group, which was rejected because
// Chat-Group-Name, which is in the encrypted part, was
@@ -3047,6 +3095,7 @@ async fn apply_group_changes(
from_id: ContactId,
to_ids: &[Option<ContactId>],
past_ids: &[Option<ContactId>],
is_chat_created: bool,
) -> Result<GroupChangesInfo> {
let from_is_key_contact = Contact::get_by_id(context, from_id).await?.is_key_contact();
ensure!(from_is_key_contact || chat.grpid.is_empty());
@@ -3059,7 +3108,7 @@ async fn apply_group_changes(
let mut better_msg = None;
let mut silent = false;
let chat_contacts =
HashSet::<ContactId>::from_iter(chat::get_chat_contacts(context, chat.id).await?);
BTreeSet::<ContactId>::from_iter(chat::get_chat_contacts(context, chat.id).await?);
let is_from_in_chat =
!chat_contacts.contains(&ContactId::SELF) || chat_contacts.contains(&from_id);
@@ -3120,24 +3169,25 @@ async fn apply_group_changes(
}
}
if is_from_in_chat {
apply_chat_name_avatar_and_description_changes(
context,
mime_parser,
from_id,
chat,
&mut send_event_chat_modified,
&mut better_msg,
)
.await?;
apply_chat_name_avatar_and_description_changes(
context,
mime_parser,
from_id,
is_from_in_chat,
chat,
&mut send_event_chat_modified,
&mut better_msg,
)
.await?;
if is_from_in_chat {
// Avoid insertion of `from_id` into a group with inappropriate encryption state.
if from_is_key_contact != chat.grpid.is_empty()
&& chat.member_list_is_stale(context).await?
{
info!(context, "Member list is stale.");
let mut new_members: HashSet<ContactId> =
HashSet::from_iter(to_ids_flat.iter().copied());
let mut new_members: BTreeSet<ContactId> =
BTreeSet::from_iter(to_ids_flat.iter().copied());
new_members.insert(ContactId::SELF);
if !from_id.is_special() {
new_members.insert(from_id);
@@ -3181,7 +3231,7 @@ async fn apply_group_changes(
)
.await?;
} else {
let mut new_members: HashSet<ContactId>;
let mut new_members: BTreeSet<ContactId>;
// True if a Delta Chat client has explicitly and really added our primary address to an
// already existing group.
let self_added =
@@ -3192,7 +3242,7 @@ async fn apply_group_changes(
false
};
if self_added {
new_members = HashSet::from_iter(to_ids_flat.iter().copied());
new_members = BTreeSet::from_iter(to_ids_flat.iter().copied());
new_members.insert(ContactId::SELF);
if !from_id.is_special() && from_is_key_contact != chat.grpid.is_empty() {
new_members.insert(from_id);
@@ -3243,7 +3293,7 @@ async fn apply_group_changes(
.await?;
}
let new_chat_contacts = HashSet::<ContactId>::from_iter(
let new_chat_contacts = BTreeSet::<ContactId>::from_iter(
chat::get_chat_contacts(context, chat.id)
.await?
.iter()
@@ -3251,27 +3301,23 @@ async fn apply_group_changes(
);
// These are for adding info messages about implicit membership changes.
let mut added_ids: HashSet<ContactId> = new_chat_contacts
let mut added_ids: BTreeSet<ContactId> = new_chat_contacts
.difference(&chat_contacts)
.copied()
.collect();
let mut removed_ids: HashSet<ContactId> = chat_contacts
let mut removed_ids: BTreeSet<ContactId> = chat_contacts
.difference(&new_chat_contacts)
.copied()
.collect();
if let Some(added_id) = added_id
&& !added_ids.remove(&added_id)
&& added_id != ContactId::SELF
{
// No-op "Member added" message. An exception is self-addition messages because they at
// least must be shown when a chat is created on our side.
info!(context, "No-op 'Member added' message (TRASH)");
better_msg = Some(String::new());
}
let id_was_already_added = if let Some(added_id) = added_id {
!added_ids.remove(&added_id)
} else {
false
};
if let Some(removed_id) = removed_id {
removed_ids.remove(&removed_id);
}
let group_changes_msgs = if !chat_contacts.contains(&ContactId::SELF)
&& new_chat_contacts.contains(&ContactId::SELF)
{
@@ -3280,6 +3326,11 @@ async fn apply_group_changes(
group_changes_msgs(context, &added_ids, &removed_ids, chat.id).await?
};
if id_was_already_added && group_changes_msgs.is_empty() && !is_chat_created {
info!(context, "No-op 'Member added' message (TRASH)");
better_msg = Some(String::new());
}
if send_event_chat_modified {
context.emit_event(EventType::ChatModified(chat.id));
chatlist_events::emit_chatlist_item_changed(context, chat.id);
@@ -3304,6 +3355,7 @@ async fn apply_chat_name_avatar_and_description_changes(
context: &Context,
mime_parser: &MimeMessage,
from_id: ContactId,
is_from_in_chat: bool,
chat: &mut Chat,
send_event_chat_modified: &mut bool,
better_msg: &mut Option<String>,
@@ -3332,7 +3384,8 @@ async fn apply_chat_name_avatar_and_description_changes(
let chat_group_name_timestamp = chat.param.get_i64(Param::GroupNameTimestamp).unwrap_or(0);
let group_name_timestamp = group_name_timestamp.unwrap_or(mime_parser.timestamp_sent);
// To provide group name consistency, compare names if timestamps are equal.
if (chat_group_name_timestamp, grpname) < (group_name_timestamp, &chat.name)
if is_from_in_chat
&& (chat_group_name_timestamp, grpname) < (group_name_timestamp, &chat.name)
&& chat
.id
.update_timestamp(context, Param::GroupNameTimestamp, group_name_timestamp)
@@ -3353,14 +3406,19 @@ async fn apply_chat_name_avatar_and_description_changes(
.get_header(HeaderDef::ChatGroupNameChanged)
.is_some()
{
let old_name = &sanitize_single_line(old_name);
better_msg.get_or_insert(
if matches!(chat.typ, Chattype::InBroadcast | Chattype::OutBroadcast) {
stock_str::msg_broadcast_name_changed(context, old_name, grpname)
} else {
stock_str::msg_grp_name(context, old_name, grpname, from_id).await
},
);
if is_from_in_chat {
let old_name = &sanitize_single_line(old_name);
better_msg.get_or_insert(
if matches!(chat.typ, Chattype::InBroadcast | Chattype::OutBroadcast) {
stock_str::msg_broadcast_name_changed(context, old_name, grpname)
} else {
stock_str::msg_grp_name(context, old_name, grpname, from_id).await
},
);
} else {
// Attempt to change group name by non-member, trash it.
*better_msg = Some(String::new());
}
}
}
@@ -3383,7 +3441,8 @@ async fn apply_chat_name_avatar_and_description_changes(
let new_timestamp = timestamp_in_header.unwrap_or(mime_parser.timestamp_sent);
// To provide consistency, compare descriptions if timestamps are equal.
if (old_timestamp, &old_description) < (new_timestamp, &new_description)
if is_from_in_chat
&& (old_timestamp, &old_description) < (new_timestamp, &new_description)
&& chat
.id
.update_timestamp(context, Param::GroupDescriptionTimestamp, new_timestamp)
@@ -3404,8 +3463,13 @@ async fn apply_chat_name_avatar_and_description_changes(
.get_header(HeaderDef::ChatGroupDescriptionChanged)
.is_some()
{
better_msg
.get_or_insert(stock_str::msg_chat_description_changed(context, from_id).await);
if is_from_in_chat {
better_msg
.get_or_insert(stock_str::msg_chat_description_changed(context, from_id).await);
} else {
// Attempt to change group description by non-member, trash it.
*better_msg = Some(String::new());
}
}
}
@@ -3415,39 +3479,46 @@ async fn apply_chat_name_avatar_and_description_changes(
&& value == "group-avatar-changed"
&& let Some(avatar_action) = &mime_parser.group_avatar
{
// this is just an explicit message containing the group-avatar,
// apart from that, the group-avatar is send along with various other messages
better_msg.get_or_insert(
if matches!(chat.typ, Chattype::InBroadcast | Chattype::OutBroadcast) {
stock_str::msg_broadcast_img_changed(context)
} else {
match avatar_action {
AvatarAction::Delete => stock_str::msg_grp_img_deleted(context, from_id).await,
AvatarAction::Change(_) => {
stock_str::msg_grp_img_changed(context, from_id).await
if is_from_in_chat {
// this is just an explicit message containing the group-avatar,
// apart from that, the group-avatar is send along with various other messages
better_msg.get_or_insert(
if matches!(chat.typ, Chattype::InBroadcast | Chattype::OutBroadcast) {
stock_str::msg_broadcast_img_changed(context)
} else {
match avatar_action {
AvatarAction::Delete => {
stock_str::msg_grp_img_deleted(context, from_id).await
}
AvatarAction::Change(_) => {
stock_str::msg_grp_img_changed(context, from_id).await
}
}
}
},
);
},
);
} else {
// Attempt to change group avatar by non-member, trash it.
*better_msg = Some(String::new());
}
}
if let Some(avatar_action) = &mime_parser.group_avatar {
info!(context, "Group-avatar change for {}.", chat.id);
if chat
if let Some(avatar_action) = &mime_parser.group_avatar
&& is_from_in_chat
&& chat
.param
.update_timestamp(Param::AvatarTimestamp, mime_parser.timestamp_sent)?
{
match avatar_action {
AvatarAction::Change(profile_image) => {
chat.param.set(Param::ProfileImage, profile_image);
}
AvatarAction::Delete => {
chat.param.remove(Param::ProfileImage);
}
};
chat.update_param(context).await?;
*send_event_chat_modified = true;
}
{
info!(context, "Group-avatar change for {}.", chat.id);
match avatar_action {
AvatarAction::Change(profile_image) => {
chat.param.set(Param::ProfileImage, profile_image);
}
AvatarAction::Delete => {
chat.param.remove(Param::ProfileImage);
}
};
chat.update_param(context).await?;
*send_event_chat_modified = true;
}
Ok(())
@@ -3457,8 +3528,8 @@ async fn apply_chat_name_avatar_and_description_changes(
#[expect(clippy::arithmetic_side_effects)]
async fn group_changes_msgs(
context: &Context,
added_ids: &HashSet<ContactId>,
removed_ids: &HashSet<ContactId>,
added_ids: &BTreeSet<ContactId>,
removed_ids: &BTreeSet<ContactId>,
chat_id: ChatId,
) -> Result<Vec<(String, SystemMessage, Option<ContactId>)>> {
let mut group_changes_msgs: Vec<(String, SystemMessage, Option<ContactId>)> = Vec::new();
@@ -3561,7 +3632,14 @@ async fn create_or_lookup_mailinglist_or_broadcast(
chattype,
&listid,
name,
create_blocked,
if chattype == Chattype::InBroadcast {
// If we joined the broadcast, we have scanned a QR code.
// Even if 1:1 chat does not exist or is in a contact request,
// create the channel as unblocked.
Blocked::Not
} else {
create_blocked
},
param,
mime_parser.timestamp_sent,
)
@@ -3747,10 +3825,12 @@ async fn apply_out_broadcast_changes(
let mut added_removed_id: Option<ContactId> = None;
if from_id == ContactId::SELF {
let is_from_in_chat = true;
apply_chat_name_avatar_and_description_changes(
context,
mime_parser,
from_id,
is_from_in_chat,
chat,
&mut send_event_chat_modified,
&mut better_msg,
@@ -3839,10 +3919,12 @@ async fn apply_in_broadcast_changes(
let mut send_event_chat_modified = false;
let mut better_msg = None;
let is_from_in_chat = is_contact_in_chat(context, chat.id, from_id).await?;
apply_chat_name_avatar_and_description_changes(
context,
mime_parser,
from_id,
is_from_in_chat,
chat,
&mut send_event_chat_modified,
&mut better_msg,
+172 -36
View File
@@ -13,9 +13,10 @@ use crate::constants::DC_GCL_FOR_FORWARDING;
use crate::contact;
use crate::imap::prefetch_should_download;
use crate::imex::{ImexMode, imex};
use crate::key;
use crate::securejoin::get_securejoin_qr;
use crate::test_utils::{
E2EE_INFO_MSGS, TestContext, TestContextManager, get_chat_msg, mark_as_verified,
E2EE_INFO_MSGS, TestContext, TestContextManager, alice_keypair, get_chat_msg, mark_as_verified,
};
use crate::tools::{SystemTime, time};
@@ -819,9 +820,12 @@ async fn load_imf_email(context: &Context, imf_raw: &[u8]) -> Message {
.set_config(Config::ShowEmails, Some("2"))
.await
.unwrap();
receive_imf(context, imf_raw, false).await.unwrap();
let chats = Chatlist::try_load(context, 0, None, None).await.unwrap();
let msg_id = chats.get_msg_id(0).unwrap().unwrap();
let received_msg = receive_imf(context, imf_raw, false)
.await
.expect("receive_imf failure")
.expect("No message received");
assert_eq!(received_msg.msg_ids.len(), 1);
let msg_id = received_msg.msg_ids[0];
Message::load_from_db(context, msg_id).await.unwrap()
}
@@ -2872,9 +2876,8 @@ async fn test_rfc1847_encapsulation() -> Result<()> {
// Alice sends a message to Bob using Thunderbird.
let raw = include_bytes!("../../test-data/message/rfc1847_encapsulation.eml");
receive_imf(bob, raw, false).await?;
let msg = bob.get_last_msg().await;
let msg = load_imf_email(bob, raw).await;
assert!(msg.get_showpadlock());
Ok(())
@@ -3082,8 +3085,8 @@ async fn test_auto_accept_for_bots() -> Result<()> {
async fn test_auto_accept_group_for_bots() -> Result<()> {
let t = TestContext::new_alice().await;
t.set_config(Config::Bot, Some("1")).await.unwrap();
receive_imf(&t, GRP_MAIL, false).await?;
let msg = t.get_last_msg().await;
let msg = load_imf_email(&t, GRP_MAIL).await;
let chat = chat::Chat::load_from_db(&t, msg.chat_id).await?;
assert!(!chat.is_contact_request());
Ok(())
@@ -3327,7 +3330,7 @@ async fn test_outgoing_undecryptable() -> Result<()> {
assert!(
dev_msg
.text
.contains("⚠️ It seems you are using Delta Chat on multiple devices that cannot decrypt each other's outgoing messages. To fix this, on the older device use \"Settings / Add Second Device\" and follow the instructions.")
.starts_with("⚠️ It seems you are using Delta Chat on multiple devices that cannot decrypt each other's outgoing messages. To fix this, on the older device use \"Settings / Add Second Device\" and follow the instructions. (Error:")
);
let raw = include_bytes!("../../test-data/message/thunderbird_encrypted_signed.eml");
@@ -3556,9 +3559,9 @@ async fn test_messed_up_message_id() -> Result<()> {
let t = TestContext::new_bob().await;
let raw = include_bytes!("../../test-data/message/messed_up_message_id.eml");
receive_imf(&t, raw, false).await?;
let msg = load_imf_email(&t, raw).await;
assert_eq!(
t.get_last_msg().await.rfc724_mid,
msg.rfc724_mid,
"0bb9ffe1-2596-d997-95b4-1fef8cc4808e@example.org"
);
@@ -3872,21 +3875,17 @@ async fn test_group_contacts_goto_bottom() -> Result<()> {
assert_eq!(Contact::get_all(bob, 0, None).await?.len(), 0);
bob_chat_id.accept(bob).await?;
let contacts = Contact::get_all(bob, 0, None).await?;
assert_eq!(contacts.len(), 2);
let bob_fiona_id = bob.add_or_lookup_contact_id(fiona).await;
// Fiona hasn't been online, so she goes after Alice.
assert_eq!(contacts.len(), 2);
assert_eq!(contacts[1], bob_fiona_id);
ChatId::create_for_contact(bob, bob_fiona_id).await?;
let bob_fiona_chat_id = ChatId::create_for_contact(bob, bob_fiona_id).await?;
let contacts = Contact::get_all(bob, 0, None).await?;
assert_eq!(contacts.len(), 2);
assert_eq!(contacts[0], bob_fiona_id);
send_text_msg(
bob,
bob_chat_id,
"Hi Alice, stay down in my contact list".to_string(),
)
.await?;
send_text_msg(bob, bob_chat_id, "Hi all".to_string()).await?;
bob.pop_sent_msg().await;
let contacts = Contact::get_all(bob, 0, None).await?;
assert_eq!(contacts[0], bob_fiona_id);
@@ -3902,6 +3901,13 @@ async fn test_group_contacts_goto_bottom() -> Result<()> {
bob.pop_sent_msg().await;
let contacts = Contact::get_all(bob, 0, None).await?;
let bob_alice_id = bob.add_or_lookup_contact_id(alice).await;
// As the group only contains Alice, the sent message promotes her in the contact list.
assert_eq!(contacts[0], bob_alice_id);
send_text_msg(bob, bob_fiona_chat_id, "Hi Fiona".to_string()).await?;
bob.pop_sent_msg().await;
let contacts = Contact::get_all(bob, 0, None).await?;
// Alice is still the 0th contact because Fiona hasn't been online.
assert_eq!(contacts[0], bob_alice_id);
Ok(())
}
@@ -4375,39 +4381,42 @@ async fn test_recreate_member_list_on_missing_add_of_self() -> Result<()> {
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_keep_member_list_if_possibly_nomember() -> Result<()> {
let alice = TestContext::new_alice().await;
let bob = TestContext::new_bob().await;
let alice_chat_id = create_group(&alice, "Group").await?;
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
let bob = &tcm.bob().await;
let alice_chat_id = create_group(alice, "Group").await?;
add_contact_to_chat(
&alice,
alice,
alice_chat_id,
alice.add_or_lookup_contact_id(&bob).await,
alice.add_or_lookup_contact_id(bob).await,
)
.await?;
send_text_msg(&alice, alice_chat_id, "populate".to_string()).await?;
send_text_msg(alice, alice_chat_id, "populate".to_string()).await?;
let bob_chat_id = bob.recv_msg(&alice.pop_sent_msg().await).await.chat_id;
let fiona = TestContext::new_fiona().await;
let fiona = &tcm.fiona().await;
add_contact_to_chat(
&alice,
alice,
alice_chat_id,
alice.add_or_lookup_contact_id(&fiona).await,
alice.add_or_lookup_contact_id(fiona).await,
)
.await?;
let fiona_chat_id = fiona.recv_msg(&alice.pop_sent_msg().await).await.chat_id;
fiona_chat_id.accept(&fiona).await?;
fiona_chat_id.accept(fiona).await?;
SystemTime::shift(Duration::from_secs(60));
chat::set_chat_name(&fiona, fiona_chat_id, "Renamed").await?;
bob.recv_msg(&fiona.pop_sent_msg().await).await;
chat::set_chat_name(fiona, fiona_chat_id, "Renamed").await?;
// Message about chat name change from non-member is trashed.
bob.recv_msg_trash(&fiona.pop_sent_msg().await).await;
// Bob missed the message adding fiona, but mustn't recreate the member list or apply the group
// name change.
assert_eq!(get_chat_contacts(&bob, bob_chat_id).await?.len(), 2);
assert!(is_contact_in_chat(&bob, bob_chat_id, ContactId::SELF).await?);
let bob_alice_contact = bob.add_or_lookup_contact_id(&alice).await;
assert!(is_contact_in_chat(&bob, bob_chat_id, bob_alice_contact).await?);
let chat = Chat::load_from_db(&bob, bob_chat_id).await?;
assert_eq!(get_chat_contacts(bob, bob_chat_id).await?.len(), 2);
assert!(is_contact_in_chat(bob, bob_chat_id, ContactId::SELF).await?);
let bob_alice_contact = bob.add_or_lookup_contact_id(alice).await;
assert!(is_contact_in_chat(bob, bob_chat_id, bob_alice_contact).await?);
let chat = Chat::load_from_db(bob, bob_chat_id).await?;
assert_eq!(chat.get_name(), "Group");
Ok(())
}
@@ -5360,6 +5369,46 @@ async fn test_outgoing_unencrypted_chat_assignment() {
assert_eq!(received.chat_id, chat.id);
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_incoming_reply_with_date_in_past() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
let msg0 = receive_imf(
alice,
b"From: bob@example.net\n\
To: alice@example.org\n\
Message-ID: <message@example.net>\n\
Date: Sun, 22 Mar 2020 22:22:22 +0000\n\
\n\
This device has an atomic clock\n",
false,
)
.await?
.unwrap();
let msg1 = receive_imf(
alice,
b"From: bob@example.net\n\
To: alice@example.org\n\
Message-ID: <message1@example.net>\n\
In-Reply-To: <message@example.net>\n\
Date: Sun, 22 Mar 2020 11:11:11 +0000\n\
\n\
And this one has a wind-up clock\n",
false,
)
.await?
.unwrap();
assert_eq!(msg1.chat_id, msg0.chat_id);
assert!(msg1.sort_timestamp >= msg0.sort_timestamp);
assert_eq!(
alice.get_last_msg_in(msg0.chat_id).await.id,
*msg1.msg_ids.last().unwrap()
);
Ok(())
}
/// Tests Bob receiving a message from Alice
/// in a new group she just created
/// with only Alice and Bob.
@@ -5559,3 +5608,90 @@ async fn test_calendar_alternative() -> Result<()> {
Ok(())
}
/// Tests that outgoing encrypted messages are detected
/// by verifying own signature, completely ignoring From address.
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_outgoing_determined_by_signature() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
let bob = &tcm.bob().await;
// alice_dev2: same key, different address.
let different_from = "very@different.from";
assert!(!alice.is_self_addr(different_from).await?);
let alice_dev2 = &tcm.unconfigured().await;
alice_dev2.configure_addr(different_from).await;
key::store_self_keypair(alice_dev2, &alice_keypair()).await?;
assert_ne!(
alice.get_config(Config::Addr).await?.unwrap(),
different_from
);
// Send message from alice_dev2 and check alice sees it as outgoing
let chat_id = alice_dev2.create_chat_id(bob).await;
let sent_msg = alice_dev2.send_text(chat_id, "hello from new device").await;
let msg = alice.recv_msg(&sent_msg).await;
assert_eq!(msg.state, MessageState::OutDelivered);
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_mark_message_as_delivered_only_after_sent_out_fully() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
let bob = &tcm.bob().await;
alice.set_config_bool(Config::BccSelf, true).await?;
let alice_chat_id = alice.create_chat_id(bob).await;
let file_bytes = include_bytes!("../../test-data/image/screenshot.gif");
let mut msg = Message::new(Viewtype::Image);
msg.set_file_from_bytes(alice, "a.jpg", file_bytes, None)?;
let msg_id = chat::send_msg(alice, alice_chat_id, &mut msg)
.await
.unwrap();
let (pre_msg_id, pre_msg_payload) = first_row_in_smtp_queue(alice).await;
assert_eq!(msg_id, pre_msg_id);
assert!(pre_msg_payload.len() < file_bytes.len());
assert_eq!(msg_id.get_state(alice).await?, MessageState::OutPending);
// Alice receives her own pre-message because of bcc_self
// This should not yet mark the message as delivered,
// because not everything was sent,
// but it does remove the pre-message from the SMTP queue
receive_imf(alice, pre_msg_payload.as_bytes(), false).await?;
assert_eq!(msg_id.get_state(alice).await?, MessageState::OutPending);
let (post_msg_id, post_msg_payload) = first_row_in_smtp_queue(alice).await;
assert_eq!(msg_id, post_msg_id);
assert!(post_msg_payload.len() > file_bytes.len());
assert_eq!(msg_id.get_state(alice).await?, MessageState::OutPending);
// Alice receives her own post-message because of bcc_self
// This should now mark the message as delivered,
// because everything was sent by now.
receive_imf(alice, post_msg_payload.as_bytes(), false).await?;
assert_eq!(msg_id.get_state(alice).await?, MessageState::OutDelivered);
Ok(())
}
/// Queries the first sent message in the SMTP queue
/// without removing it from the SMTP queue.
/// This simulates the case that a message is successfully sent out,
/// but the 'OK' answer from the server doesn't arrive,
/// so that the SMTP row stays in the database.
pub(crate) async fn first_row_in_smtp_queue(alice: &TestContext) -> (MsgId, String) {
alice
.sql
.query_row_optional("SELECT msg_id, mime FROM smtp ORDER BY id", (), |row| {
let msg_id: MsgId = row.get(0)?;
let mime: String = row.get(1)?;
Ok((msg_id, mime))
})
.await
.expect("query_row_optional failed")
.expect("No SMTP row found")
}
+26 -154
View File
@@ -17,7 +17,7 @@ use crate::context::Context;
use crate::download::{download_known_post_messages_without_pre_message, download_msgs};
use crate::ephemeral::{self, delete_expired_imap_messages};
use crate::events::EventType;
use crate::imap::{FolderMeaning, Imap, session::Session};
use crate::imap::{Imap, session::Session};
use crate::location;
use crate::log::{LogExt, warn};
use crate::smtp::{Smtp, send_smtp_messages};
@@ -211,25 +211,19 @@ impl SchedulerState {
/// Indicate that the network likely has come back.
pub(crate) async fn maybe_network(&self) {
let inner = self.inner.read().await;
let (inboxes, oboxes) = match *inner {
let inboxes = match *inner {
InnerSchedulerState::Started(ref scheduler) => {
scheduler.maybe_network();
let inboxes = scheduler
scheduler
.inboxes
.iter()
.map(|b| b.conn_state.state.connectivity.clone())
.collect::<Vec<_>>();
let oboxes = scheduler
.oboxes
.iter()
.map(|b| b.conn_state.state.connectivity.clone())
.collect::<Vec<_>>();
(inboxes, oboxes)
.collect::<Vec<_>>()
}
_ => return,
};
drop(inner);
connectivity::idle_interrupted(inboxes, oboxes);
connectivity::idle_interrupted(inboxes);
}
/// Indicate that the network likely is lost.
@@ -318,7 +312,10 @@ impl Drop for IoPausedGuard {
struct SchedBox {
/// Address at the used chatmail/email relay
addr: String,
meaning: FolderMeaning,
/// Folder name
folder: String,
conn_state: ImapConnectionState,
/// IMAP loop task handle.
@@ -330,8 +327,6 @@ struct SchedBox {
pub(crate) struct Scheduler {
/// Inboxes, one per transport.
inboxes: Vec<SchedBox>,
/// Optional boxes -- mvbox.
oboxes: Vec<SchedBox>,
smtp: SmtpConnectionState,
smtp_handle: task::JoinHandle<()>,
ephemeral_handle: task::JoinHandle<()>,
@@ -412,42 +407,12 @@ async fn inbox_loop(
.await;
}
/// Convert folder meaning
/// used internally by [fetch_idle] and [Context::background_fetch].
///
/// Returns folder configuration key and folder name
/// if such folder is configured, `Ok(None)` otherwise.
pub async fn convert_folder_meaning(
ctx: &Context,
folder_meaning: FolderMeaning,
) -> Result<Option<(Config, String)>> {
let folder_config = match folder_meaning.to_config() {
Some(c) => c,
None => {
// Such folder cannot be configured,
// e.g. a `FolderMeaning::Spam` folder.
return Ok(None);
}
};
let folder = ctx
.get_config(folder_config)
.await
.with_context(|| format!("Failed to retrieve {folder_config} folder"))?;
if let Some(watch_folder) = folder {
Ok(Some((folder_config, watch_folder)))
} else {
Ok(None)
}
}
async fn inbox_fetch_idle(ctx: &Context, imap: &mut Imap, mut session: Session) -> Result<Session> {
let transport_id = session.transport_id();
// Update quota no more than once a minute.
if ctx.quota_needs_update(session.transport_id(), 60).await
&& let Err(err) = ctx.update_recent_quota(&mut session).await
&& let Err(err) = ctx.update_recent_quota(&mut session, &imap.folder).await
{
warn!(
ctx,
@@ -494,7 +459,7 @@ async fn inbox_fetch_idle(ctx: &Context, imap: &mut Imap, mut session: Session)
.await
.context("Failed to register push token")?;
let session = fetch_idle(ctx, imap, session, FolderMeaning::Inbox).await?;
let session = fetch_idle(ctx, imap, session).await?;
Ok(session)
}
@@ -503,34 +468,19 @@ async fn inbox_fetch_idle(ctx: &Context, imap: &mut Imap, mut session: Session)
/// This function performs all IMAP operations on a single folder, selecting it if necessary and
/// handling all the errors. In case of an error, an error is returned and connection is dropped,
/// otherwise connection is returned.
async fn fetch_idle(
ctx: &Context,
connection: &mut Imap,
mut session: Session,
folder_meaning: FolderMeaning,
) -> Result<Session> {
async fn fetch_idle(ctx: &Context, connection: &mut Imap, mut session: Session) -> Result<Session> {
let transport_id = session.transport_id();
let Some((folder_config, watch_folder)) = convert_folder_meaning(ctx, folder_meaning).await?
else {
// The folder is not configured.
// For example, this happens if the server does not have Sent folder
// but watching Sent folder is enabled.
connection.connectivity.set_not_configured(ctx);
connection.idle_interrupt_receiver.recv().await.ok();
bail!("Cannot fetch folder {folder_meaning} because it is not configured");
};
let watch_folder = connection.folder.clone();
if folder_config == Config::ConfiguredInboxFolder {
session
.store_seen_flags_on_imap(ctx)
.await
.context("store_seen_flags_on_imap")?;
}
session
.store_seen_flags_on_imap(ctx)
.await
.context("store_seen_flags_on_imap")?;
// Fetch the watched folder.
connection
.fetch_move_delete(ctx, &mut session, &watch_folder, folder_meaning)
.fetch_move_delete(ctx, &mut session, &watch_folder)
.await
.context("fetch_move_delete")?;
@@ -564,7 +514,7 @@ async fn fetch_idle(
ctx,
"Transport {transport_id}: IMAP session does not support IDLE, going to fake idle."
);
connection.fake_idle(ctx, watch_folder).await?;
connection.fake_idle(ctx, &watch_folder).await?;
return Ok(session);
}
@@ -579,7 +529,7 @@ async fn fetch_idle(
ctx,
"Transport {transport_id}: IMAP IDLE is disabled, going to fake idle."
);
connection.fake_idle(ctx, watch_folder).await?;
connection.fake_idle(ctx, &watch_folder).await?;
return Ok(session);
}
@@ -595,69 +545,6 @@ async fn fetch_idle(
Ok(session)
}
/// Simplified IMAP loop to watch non-inbox folders.
async fn simple_imap_loop(
ctx: Context,
started: oneshot::Sender<()>,
inbox_handlers: ImapConnectionHandlers,
folder_meaning: FolderMeaning,
) {
use futures::future::FutureExt;
info!(ctx, "Starting simple loop for {folder_meaning}.");
let ImapConnectionHandlers {
mut connection,
stop_token,
} = inbox_handlers;
let ctx1 = ctx.clone();
let fut = async move {
let ctx = ctx1;
if let Err(()) = started.send(()) {
warn!(
ctx,
"Simple imap loop for {folder_meaning}, missing started receiver."
);
return;
}
let mut old_session: Option<Session> = None;
loop {
let session = if let Some(session) = old_session.take() {
session
} else {
info!(ctx, "Preparing new IMAP session for {folder_meaning}.");
match connection.prepare(&ctx).await {
Err(err) => {
warn!(
ctx,
"Failed to prepare {folder_meaning} connection: {err:#}."
);
continue;
}
Ok(session) => session,
}
};
match fetch_idle(&ctx, &mut connection, session, folder_meaning).await {
Err(err) => warn!(ctx, "Failed fetch_idle: {err:#}"),
Ok(session) => {
old_session = Some(session);
}
}
}
};
stop_token
.cancelled()
.map(|_| {
info!(ctx, "Shutting down IMAP loop for {folder_meaning}.");
})
.race(fut)
.await;
}
async fn smtp_loop(
ctx: Context,
started: oneshot::Sender<()>,
@@ -760,7 +647,6 @@ impl Scheduler {
let (location_interrupt_send, location_interrupt_recv) = channel::bounded(1);
let mut inboxes = Vec::new();
let mut oboxes = Vec::new();
let mut start_recvs = Vec::new();
for (transport_id, configured_login_param) in ConfiguredLoginParam::load_all(ctx).await? {
@@ -772,30 +658,17 @@ impl Scheduler {
task::spawn(inbox_loop(ctx, inbox_start_send, inbox_handlers))
};
let addr = configured_login_param.addr.clone();
let folder = configured_login_param
.imap_folder
.unwrap_or_else(|| "INBOX".to_string());
let inbox = SchedBox {
addr: addr.clone(),
meaning: FolderMeaning::Inbox,
folder,
conn_state,
handle,
};
inboxes.push(inbox);
start_recvs.push(inbox_start_recv);
if ctx.should_watch_mvbox().await? {
let (conn_state, handlers) =
ImapConnectionState::new(ctx, transport_id, configured_login_param).await?;
let (start_send, start_recv) = oneshot::channel();
let ctx = ctx.clone();
let meaning = FolderMeaning::Mvbox;
let handle = task::spawn(simple_imap_loop(ctx, start_send, handlers, meaning));
oboxes.push(SchedBox {
addr,
meaning,
conn_state,
handle,
});
start_recvs.push(start_recv);
}
}
let smtp_handle = {
@@ -822,7 +695,6 @@ impl Scheduler {
let res = Self {
inboxes,
oboxes,
smtp,
smtp_handle,
ephemeral_handle,
@@ -842,7 +714,7 @@ impl Scheduler {
}
fn boxes(&self) -> impl Iterator<Item = &SchedBox> {
self.inboxes.iter().chain(self.oboxes.iter())
self.inboxes.iter()
}
fn maybe_network(&self) {
@@ -896,7 +768,7 @@ impl Scheduler {
let timeout_duration = std::time::Duration::from_secs(30);
let tracker = TaskTracker::new();
for b in self.inboxes.into_iter().chain(self.oboxes) {
for b in self.inboxes {
let context = context.clone();
tracker.spawn(async move {
tokio::time::timeout(timeout_duration, b.handle)
+27 -85
View File
@@ -5,11 +5,10 @@ use std::{iter::once, ops::Deref, sync::Arc};
use anyhow::Result;
use humansize::{BINARY, format_size};
use crate::context::Context;
use crate::events::EventType;
use crate::imap::{FolderMeaning, get_watched_folder_configs};
use crate::quota::{QUOTA_ERROR_THRESHOLD_PERCENTAGE, QUOTA_WARN_THRESHOLD_PERCENTAGE};
use crate::stock_str;
use crate::{context::Context, log::LogExt};
use super::InnerSchedulerState;
@@ -67,40 +66,33 @@ enum DetailedConnectivity {
/// Connection is established and is idle.
Idle,
/// The folder was configured not to be watched or configured_*_folder is not set
NotConfigured,
}
impl DetailedConnectivity {
fn to_basic(&self) -> Option<Connectivity> {
fn to_basic(&self) -> Connectivity {
match self {
DetailedConnectivity::Error(_) => Some(Connectivity::NotConnected),
DetailedConnectivity::Uninitialized => Some(Connectivity::NotConnected),
DetailedConnectivity::Connecting => Some(Connectivity::Connecting),
DetailedConnectivity::Working => Some(Connectivity::Working),
DetailedConnectivity::InterruptingIdle => Some(Connectivity::Working),
DetailedConnectivity::Error(_) => Connectivity::NotConnected,
DetailedConnectivity::Uninitialized => Connectivity::NotConnected,
DetailedConnectivity::Connecting => Connectivity::Connecting,
DetailedConnectivity::Working => Connectivity::Working,
DetailedConnectivity::InterruptingIdle => Connectivity::Working,
// At this point IMAP has just connected,
// but does not know yet if there are messages to download.
// We still convert this to Working state
// so user can see "Updating..." and not "Connected"
// which is reserved for idle state.
DetailedConnectivity::Preparing => Some(Connectivity::Working),
DetailedConnectivity::Preparing => Connectivity::Working,
// Just don't return a connectivity, probably the folder is configured not to be
// watched, so we are not interested in it.
DetailedConnectivity::NotConfigured => None,
DetailedConnectivity::Idle => Some(Connectivity::Connected),
DetailedConnectivity::Idle => Connectivity::Connected,
}
}
fn to_icon(&self) -> String {
match self {
DetailedConnectivity::Error(_)
| DetailedConnectivity::Uninitialized
| DetailedConnectivity::NotConfigured => "<span class=\"red dot\"></span>".to_string(),
DetailedConnectivity::Error(_) | DetailedConnectivity::Uninitialized => {
"<span class=\"red dot\"></span>".to_string()
}
DetailedConnectivity::Connecting => "<span class=\"yellow dot\"></span>".to_string(),
DetailedConnectivity::Preparing
| DetailedConnectivity::Working
@@ -120,7 +112,6 @@ impl DetailedConnectivity {
DetailedConnectivity::InterruptingIdle | DetailedConnectivity::Idle => {
stock_str::connected(context)
}
DetailedConnectivity::NotConfigured => "Not configured".to_string(),
}
}
@@ -139,7 +130,6 @@ impl DetailedConnectivity {
DetailedConnectivity::InterruptingIdle
| DetailedConnectivity::Preparing
| DetailedConnectivity::Idle => stock_str::last_msg_sent_successfully(context),
DetailedConnectivity::NotConfigured => "Not configured".to_string(),
}
}
@@ -151,7 +141,6 @@ impl DetailedConnectivity {
DetailedConnectivity::Working => false,
DetailedConnectivity::InterruptingIdle => false,
DetailedConnectivity::Preparing => false, // Just connected, there may still be work to do.
DetailedConnectivity::NotConfigured => true,
DetailedConnectivity::Idle => true,
}
}
@@ -180,9 +169,6 @@ impl ConnectivityStore {
pub(crate) fn set_preparing(&self, context: &Context) {
self.set(context, DetailedConnectivity::Preparing);
}
pub(crate) fn set_not_configured(&self, context: &Context) {
self.set(context, DetailedConnectivity::NotConfigured);
}
pub(crate) fn set_idle(&self, context: &Context) {
self.set(context, DetailedConnectivity::Idle);
}
@@ -190,7 +176,7 @@ impl ConnectivityStore {
fn get_detailed(&self) -> DetailedConnectivity {
self.0.lock().deref().clone()
}
fn get_basic(&self) -> Option<Connectivity> {
fn get_basic(&self) -> Connectivity {
self.0.lock().to_basic()
}
fn get_all_work_done(&self) -> bool {
@@ -201,27 +187,14 @@ impl ConnectivityStore {
/// Set all folder states to InterruptingIdle in case they were `Idle` before.
/// Called during `dc_maybe_network()` to make sure that `all_work_done()`
/// returns false immediately after `dc_maybe_network()`.
pub(crate) fn idle_interrupted(inboxes: Vec<ConnectivityStore>, oboxes: Vec<ConnectivityStore>) {
pub(crate) fn idle_interrupted(inboxes: Vec<ConnectivityStore>) {
for inbox in inboxes {
let mut connectivity_lock = inbox.0.lock();
// For the inbox, we also have to set the connectivity to InterruptingIdle if it was
// NotConfigured before: If all folders are NotConfigured, dc_get_connectivity()
// returns Connected. But after dc_maybe_network(), dc_get_connectivity() must not
// return Connected until DC is completely done with fetching folders; this also
// includes scan_folders() which happens on the inbox thread.
if *connectivity_lock == DetailedConnectivity::Idle
|| *connectivity_lock == DetailedConnectivity::NotConfigured
{
*connectivity_lock = DetailedConnectivity::InterruptingIdle;
}
}
for state in oboxes {
let mut connectivity_lock = state.0.lock();
if *connectivity_lock == DetailedConnectivity::Idle {
*connectivity_lock = DetailedConnectivity::InterruptingIdle;
}
}
// No need to send ConnectivityChanged, the user-facing connectivity doesn't change because
// of what we do here.
}
@@ -234,9 +207,7 @@ pub(crate) fn maybe_network_lost(context: &Context, stores: Vec<ConnectivityStor
let mut connectivity_lock = store.0.lock();
if !matches!(
*connectivity_lock,
DetailedConnectivity::Uninitialized
| DetailedConnectivity::Error(_)
| DetailedConnectivity::NotConfigured,
DetailedConnectivity::Uninitialized | DetailedConnectivity::Error(_)
) {
*connectivity_lock = DetailedConnectivity::Error("Connection lost".to_string());
}
@@ -273,9 +244,8 @@ impl Context {
let stores = self.connectivities.lock().clone();
let mut connectivities = Vec::new();
for s in stores {
if let Some(connectivity) = s.get_basic() {
connectivities.push(connectivity);
}
let connectivity = s.get_basic();
connectivities.push(connectivity);
}
connectivities
.into_iter()
@@ -386,7 +356,7 @@ impl Context {
.map(|b| {
(
b.addr.clone(),
b.meaning,
b.folder.clone(),
b.conn_state.state.connectivity.clone(),
)
})
@@ -411,7 +381,6 @@ impl Context {
// [======67%===== ]
// =============================================================================================
let watched_folders = get_watched_folder_configs(self).await?;
let incoming_messages = stock_str::incoming_messages(self);
ret += &format!("<h3>{incoming_messages}</h3><ul>");
@@ -433,41 +402,14 @@ impl Context {
let folders = folders_states
.iter()
.filter(|(folder_addr, ..)| *folder_addr == transport_addr);
for (_addr, folder, state) in folders {
let mut folder_added = false;
if let Some(config) = folder.to_config().filter(|c| watched_folders.contains(c)) {
let f = self.get_config(config).await.log_err(self).ok().flatten();
if let Some(foldername) = f {
let detailed = &state.get_detailed();
ret += &*detailed.to_icon();
ret += " <b>";
if folder == &FolderMeaning::Inbox {
ret += &*domain_escaped;
} else {
ret += &*escaper::encode_minimal(&foldername);
}
ret += ":</b> ";
ret += &*escaper::encode_minimal(&detailed.to_string_imap(self));
ret += "<br />";
folder_added = true;
}
}
if !folder_added && folder == &FolderMeaning::Inbox {
let detailed = &state.get_detailed();
if let DetailedConnectivity::Error(_) = detailed {
// On the inbox thread, we also do some other things like scan_folders and run jobs
// so, maybe, the inbox is not watched, but something else went wrong
ret += &*detailed.to_icon();
ret += " ";
ret += &*escaper::encode_minimal(&detailed.to_string_imap(self));
ret += "<br />";
}
}
for (_addr, _folder, state) in folders {
let detailed = &state.get_detailed();
ret += &*detailed.to_icon();
ret += " <b>";
ret += &*domain_escaped;
ret += ":</b> ";
ret += &*escaper::encode_minimal(&detailed.to_string_imap(self));
ret += "<br />";
}
let Some(quota) = quota.get(&transport_id) else {
+2 -11
View File
@@ -450,10 +450,8 @@ pub(crate) async fn handle_securejoin_handshake(
) {
let mut self_found = false;
let self_fingerprint = load_self_public_key(context).await?.dc_fingerprint();
for (addr, key) in &mime_message.gossiped_keys {
if key.public_key.dc_fingerprint() == self_fingerprint
&& context.is_self_addr(addr).await?
{
for key in mime_message.gossiped_keys.values() {
if key.public_key.dc_fingerprint() == self_fingerprint {
self_found = true;
break;
}
@@ -841,13 +839,6 @@ pub(crate) async fn observe_securejoin_on_other_device(
inviter_progress(context, contact_id, chat_id, chat_type)?;
}
if matches!(step, SecureJoinStep::RequestWithAuth) {
// This actually reflects what happens on the first device (which does the secure
// join) and causes a subsequent "vg-member-added" message to create an unblocked
// verified group.
ChatId::create_for_contact_with_blocked(context, contact_id, Blocked::Not).await?;
}
if matches!(step, SecureJoinStep::MemberAdded) {
Ok(HandshakeMessage::Propagate)
} else {
+48
View File
@@ -1584,3 +1584,51 @@ async fn test_auth_token_is_synchronized() -> Result<()> {
Ok(())
}
/// Tests that Bob deduplicates messages about being added to the group.
///
/// If Alice (inviter) has multiple devices,
/// Bob may receive multiple "member added" messages.
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_deduplicate_member_added() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice1 = &tcm.alice().await;
let alice2 = &tcm.alice().await;
let bob = &tcm.bob().await;
// Enable sync messages so Alice QR code tokens
// are synchronized.
alice1.set_config_bool(Config::SyncMsgs, true).await?;
alice2.set_config_bool(Config::SyncMsgs, true).await?;
tcm.section("Alice creates a group");
let alice1_chat_id = chat::create_group(alice1, "Group").await?;
tcm.section("Alice creates group QR code and synchronizes it");
let qr = get_securejoin_qr(alice1, Some(alice1_chat_id)).await?;
sync(alice1, alice2).await;
// Import Alice's key to skip key request step.
tcm.section("Bob imports Alice's vCard");
bob.add_or_lookup_contact_id(alice1).await;
tcm.section("Bob scans the QR code");
let bob_chat_id = join_securejoin(bob, &qr).await?;
tcm.section("Both Alice's devices add Bob to chat");
let sent = bob.pop_sent_msg().await;
alice1.recv_msg_trash(&sent).await;
alice2.recv_msg_trash(&sent).await;
let sent1 = alice1.pop_sent_msg().await;
let sent2 = alice2.pop_sent_msg().await;
let bob_rcvd = bob.recv_msg(&sent1).await;
assert_eq!(bob_rcvd.chat_id, bob_chat_id);
assert_eq!(bob_rcvd.text, "Member Me added by alice@example.org.");
// Second message is a no-op, so it is trashed.
bob.recv_msg_trash(&sent2).await;
Ok(())
}
-1
View File
@@ -9,7 +9,6 @@ use crate::tools::IsNoneOrEmpty;
/// This escapes a bit more than actually needed by delta (e.g. also lines as "-- footer"),
/// but for non-delta-compatibility, that seems to be better.
/// (to be only compatible with delta, only "[\r\n|\n]-- {0,2}[\r\n|\n]" needs to be replaced)
#[expect(clippy::arithmetic_side_effects)]
pub fn escape_message_footer_marks(text: &str) -> String {
if let Some(text) = text.strip_prefix("--") {
"-\u{200B}-".to_string() + &text.replace("\n--", "\n-\u{200B}-")
+12 -6
View File
@@ -465,11 +465,7 @@ pub(crate) async fn send_msg_to_smtp(
match status {
SendResult::Retry => Err(format_err!("Retry")),
SendResult::Success => {
if !context
.sql
.exists("SELECT COUNT(*) FROM smtp WHERE msg_id=?", (msg_id,))
.await?
{
if !msg_has_pending_smtp_job(context, msg_id).await? {
msg_id.set_delivered(context).await?;
}
Ok(())
@@ -478,6 +474,16 @@ pub(crate) async fn send_msg_to_smtp(
}
}
pub(crate) async fn msg_has_pending_smtp_job(
context: &Context,
msg_id: MsgId,
) -> Result<bool, Error> {
context
.sql
.exists("SELECT COUNT(*) FROM smtp WHERE msg_id=?", (msg_id,))
.await
}
/// Attempts to send queued MDNs.
async fn send_mdns(context: &Context, connection: &mut Smtp) -> Result<()> {
loop {
@@ -574,7 +580,7 @@ async fn send_mdn_rfc724_mid(
)
.await?;
let encrypted = mimefactory.will_be_encrypted();
let rendered_msg = mimefactory.render(context).await?;
let rendered_msg = Box::pin(mimefactory.render(context)).await?;
let body = rendered_msg.message;
let mut recipients = Vec::new();
+32 -3
View File
@@ -11,11 +11,12 @@ use crate::log::warn;
use crate::net::dns::{lookup_host_with_cache, update_connect_timestamp};
use crate::net::proxy::ProxyConfig;
use crate::net::session::SessionBufStream;
use crate::net::tls::{TlsSessionStore, wrap_tls};
use crate::net::tls::{SpkiHashStore, TlsSessionStore, wrap_tls};
use crate::net::{
connect_tcp_inner, connect_tls_inner, run_connection_attempts, update_connection_history,
};
use crate::oauth2::get_oauth2_access_token;
use crate::sql::Sql;
use crate::tools::time;
use crate::transport::ConnectionCandidate;
use crate::transport::ConnectionSecurity;
@@ -111,10 +112,26 @@ async fn connection_attempt(
);
let res = match security {
ConnectionSecurity::Tls => {
connect_secure(resolved_addr, host, strict_tls, &context.tls_session_store).await
connect_secure(
resolved_addr,
host,
strict_tls,
&context.tls_session_store,
&context.spki_hash_store,
&context.sql,
)
.await
}
ConnectionSecurity::Starttls => {
connect_starttls(resolved_addr, host, strict_tls, &context.tls_session_store).await
connect_starttls(
resolved_addr,
host,
strict_tls,
&context.tls_session_store,
&context.spki_hash_store,
&context.sql,
)
.await
}
ConnectionSecurity::Plain => connect_insecure(resolved_addr).await,
};
@@ -240,6 +257,8 @@ async fn connect_secure_proxy(
alpn(port),
proxy_stream,
&context.tls_session_store,
&context.spki_hash_store,
&context.sql,
)
.await?;
let mut buffered_stream = BufStream::new(tls_stream);
@@ -273,6 +292,8 @@ async fn connect_starttls_proxy(
"",
tcp_stream,
&context.tls_session_store,
&context.spki_hash_store,
&context.sql,
)
.await
.context("STARTTLS upgrade failed")?;
@@ -299,6 +320,8 @@ async fn connect_secure(
hostname: &str,
strict_tls: bool,
tls_session_store: &TlsSessionStore,
spki_hash_store: &SpkiHashStore,
sql: &Sql,
) -> Result<Box<dyn SessionBufStream>> {
let tls_stream = connect_tls_inner(
addr,
@@ -306,6 +329,8 @@ async fn connect_secure(
strict_tls,
alpn(addr.port()),
tls_session_store,
spki_hash_store,
sql,
)
.await?;
let mut buffered_stream = BufStream::new(tls_stream);
@@ -319,6 +344,8 @@ async fn connect_starttls(
host: &str,
strict_tls: bool,
tls_session_store: &TlsSessionStore,
spki_hash_store: &SpkiHashStore,
sql: &Sql,
) -> Result<Box<dyn SessionBufStream>> {
let use_sni = false;
let tcp_stream = connect_tcp_inner(addr).await?;
@@ -336,6 +363,8 @@ async fn connect_starttls(
"",
tcp_stream,
tls_session_store,
spki_hash_store,
sql,
)
.await
.context("STARTTLS upgrade failed")?;
+26 -21
View File
@@ -9,7 +9,6 @@ use rusqlite::{Connection, OpenFlags, Row, config::DbConfig, types::ValueRef};
use tokio::sync::RwLock;
use crate::blob::BlobObject;
use crate::chat::add_device_msg;
use crate::config::Config;
use crate::constants::DC_CHAT_ID_TRASH;
use crate::context::Context;
@@ -18,13 +17,11 @@ use crate::ephemeral::start_ephemeral_timers;
use crate::imex::BLOBS_BACKUP_NAME;
use crate::location::delete_orphaned_poi_locations;
use crate::log::{LogExt, warn};
use crate::message::Message;
use crate::message::MsgId;
use crate::net::dns::prune_dns_cache;
use crate::net::http::http_cache_cleanup;
use crate::net::prune_connection_history;
use crate::param::{Param, Params};
use crate::stock_str;
use crate::tools::{SystemTime, delete_file, time};
/// Extension to [`rusqlite::ToSql`] trait
@@ -830,12 +827,6 @@ pub async fn housekeeping(context: &Context) -> Result<()> {
);
}
maybe_add_mvbox_move_deprecation_message(context)
.await
.context("maybe_add_mvbox_move_deprecation_message")
.log_err(context)
.ok();
if let Err(err) = incremental_vacuum(context).await {
warn!(context, "Failed to run incremental vacuum: {err:#}.");
}
@@ -883,6 +874,32 @@ pub async fn housekeeping(context: &Context) -> Result<()> {
.log_err(context)
.ok();
context
.spki_hash_store
.cleanup(&context.sql)
.await
.context("Failed to prune SPKI store")
.log_err(context)
.ok();
// Cleanup `imap` and `imap_sync` entries for deleted transports.
//
// Transports may be deleted directly or via sync messages,
// so it is easier to cleanup orphaned entries in a single place.
context
.sql
.execute(
"DELETE FROM imap WHERE transport_id NOT IN (SELECT transports.id FROM transports)",
(),
)
.await
.log_err(context)
.ok();
context.sql.execute(
"DELETE FROM imap_sync WHERE transport_id NOT IN (SELECT transports.id FROM transports)",
(),
).await.log_err(context).ok();
// Delete POI locations
// which don't have corresponding message.
delete_orphaned_poi_locations(context)
@@ -895,18 +912,6 @@ pub async fn housekeeping(context: &Context) -> Result<()> {
Ok(())
}
/// Adds device message about `mvbox_move` config deprecation
/// if the user has it enabled.
async fn maybe_add_mvbox_move_deprecation_message(context: &Context) -> Result<()> {
if !context.get_config_bool(Config::OnlyFetchMvbox).await?
&& context.get_config_bool(Config::MvboxMove).await?
{
let mut msg = Message::new_text(stock_str::mvbox_move_deprecation(context));
add_device_msg(context, Some("mvbox_move_deprecation"), Some(&mut msg)).await?;
}
Ok(())
}
/// Get the value of a column `idx` of the `row` as `Vec<u8>`.
pub fn row_get_vec(row: &Row, idx: usize) -> rusqlite::Result<Vec<u8>> {
row.get(idx).or_else(|err| match row.get_ref(idx)? {
+65 -52
View File
@@ -16,7 +16,6 @@ use crate::constants::ShowEmails;
use crate::context::Context;
use crate::key::DcKey;
use crate::log::warn;
use crate::message::MsgId;
use crate::provider::get_provider_info;
use crate::sql::Sql;
use crate::tools::{Time, inc_and_check, time_elapsed};
@@ -734,12 +733,6 @@ impl Sql {
Ok(())
}
async fn set_db_version_in_cache(&self, version: i32) -> Result<()> {
let mut lock = self.config_cache.write().await;
lock.insert(VERSION_CFG.to_string(), Some(format!("{version}")));
Ok(())
}
async fn execute_migration(&self, query: &str, version: i32) -> Result<()> {
self.execute_migration_transaction(
|transaction| {
@@ -1612,51 +1605,11 @@ CREATE INDEX msgs_status_updates_index2 ON msgs_status_updates (uid);
.await?;
}
if dbversion < 108 {
let version = 108;
let chunk_size = context.get_max_smtp_rcpt_to().await?;
sql.transaction(move |trans| {
Sql::set_db_version_trans(trans, version)?;
let id_max =
trans.query_row("SELECT IFNULL((SELECT MAX(id) FROM smtp), 0)", (), |row| {
let id_max: i64 = row.get(0)?;
Ok(id_max)
})?;
while let Some((id, rfc724_mid, mime, msg_id, recipients, retries)) = trans
.query_row(
"SELECT id, rfc724_mid, mime, msg_id, recipients, retries FROM smtp \
WHERE id<=? LIMIT 1",
(id_max,),
|row| {
let id: i64 = row.get(0)?;
let rfc724_mid: String = row.get(1)?;
let mime: String = row.get(2)?;
let msg_id: MsgId = row.get(3)?;
let recipients: String = row.get(4)?;
let retries: i64 = row.get(5)?;
Ok((id, rfc724_mid, mime, msg_id, recipients, retries))
},
)
.optional()?
{
trans.execute("DELETE FROM smtp WHERE id=?", (id,))?;
let recipients = recipients.split(' ').collect::<Vec<_>>();
for recipients in recipients.chunks(chunk_size) {
let recipients = recipients.join(" ");
trans.execute(
"INSERT INTO smtp (rfc724_mid, mime, msg_id, recipients, retries) \
VALUES (?, ?, ?, ?, ?)",
(&rfc724_mid, &mime, msg_id, recipients, retries),
)?;
}
}
Ok(())
})
.await
.with_context(|| format!("migration failed for version {version}"))?;
sql.set_db_version_in_cache(version).await?;
}
// Migration 108 is removed, it was using high level code
// to split SMTP queue messages into chunks with smaller number of recipients
// and started to fail later as high level code started
// expecting `transports` table that is only added in future migrations.
// Migration 108 was not changing the database schema.
if dbversion < 109 {
sql.execute_migration(
@@ -2363,6 +2316,66 @@ ALTER TABLE contacts ADD COLUMN name_normalized TEXT;
.await?;
}
inc_and_check(&mut migration_version, 150)?;
if dbversion < migration_version {
sql.execute_migration_transaction(
|transaction| {
let only_fetch_mvbox = transaction
.query_row(
"SELECT value FROM config WHERE keyname='only_fetch_mvbox'",
(),
|row| {
let value: String = row.get(0)?;
Ok(value)
},
)
.optional()?
.as_deref()
== Some("1");
if only_fetch_mvbox {
let mvbox_folder = transaction
.query_row(
"SELECT value FROM config WHERE keyname='configured_mvbox_folder'",
(),
|row| {
let value: String = row.get(0)?;
Ok(value)
},
)
.optional()?
.unwrap_or_else(|| "DeltaChat".to_string());
transaction.execute(
"UPDATE transports
SET entered_param=json_set(entered_param, '$.imap.folder', ?1),
configured_param=json_set(configured_param', '$.imap_folder', ?1)",
(mvbox_folder,),
)?;
}
Ok(())
},
migration_version,
)
.await?;
}
inc_and_check(&mut migration_version, 151)?;
if dbversion < migration_version {
sql.execute_migration(
"CREATE TABLE tls_spki (
host TEXT NOT NULL UNIQUE,
spki_hash TEXT NOT NULL, -- base64 of SPKI SHA-256 hash
timestamp INTEGER NOT NULL -- timestamp of the last time we have seen this key
) STRICT;
-- Index on host column is created implicitly because of UNIQUE constraint.
CREATE INDEX tls_spki_index_timestamp ON tls_spki (timestamp);
",
migration_version,
)
.await?;
}
let new_version = sql
.get_raw_config_int(VERSION_CFG)
.await?
+6 -10
View File
@@ -57,9 +57,9 @@ async fn test_key_contacts_migration_autocrypt() -> Result<()> {
);
assert_eq!(pgp_bob.get_verifier_id(&t).await?, None);
// Hidden address-contact can't be looked up.
// Hidden address-contact can't be looked up by name.
assert!(
Contact::get_all(&t, constants::DC_GCL_ADDRESS, Some("bob@example.net"))
Contact::get_all(&t, constants::DC_GCL_ADDRESS, Some("bob"))
.await?
.is_empty()
);
@@ -113,12 +113,8 @@ async fn test_key_contacts_migration_email2() -> Result<()> {
)?)).await?;
t.sql.run_migrations(&t).await?;
// Hidden key-contact can't be looked up.
assert!(
Contact::get_all(&t, 0, Some("bob@example.net"))
.await?
.is_empty()
);
// Hidden key-contact can't be looked up by name.
assert!(Contact::get_all(&t, 0, Some("bob")).await?.is_empty());
let pgp_bob = Contact::get_by_id(&t, ContactId::new(11001)).await?;
assert_eq!(pgp_bob.is_key_contact(), true);
assert_eq!(pgp_bob.origin, Origin::Hidden);
@@ -156,9 +152,9 @@ async fn test_key_contacts_migration_verified() -> Result<()> {
t.sql.run_migrations(&t).await?;
// Hidden address-contact can't be looked up.
// Hidden address-contact can't be looked up by name.
assert!(
Contact::get_all(&t, constants::DC_GCL_ADDRESS, Some("bob@example.net"))
Contact::get_all(&t, constants::DC_GCL_ADDRESS, Some("bob"))
.await?
.is_empty()
);
-10
View File
@@ -413,11 +413,6 @@ https://delta.chat/donate"))]
#[strum(props(fallback = "Messages in this chat use classic email and are not encrypted."))]
ChatUnencryptedExplanation = 230,
#[strum(props(
fallback = "You are using the legacy option \"Settings → Advanced → Move automatically to DeltaChat Folder\".\n\nThis option will be removed in a few weeks and you should disable it already today.\n\nIf having chat messages mixed into your inbox is a problem, see https://delta.chat/legacy-move"
))]
MvboxMoveDeprecation = 231,
#[strum(props(fallback = "Outgoing audio call"))]
OutgoingAudioCall = 232,
@@ -1225,11 +1220,6 @@ pub(crate) fn chat_unencrypted_explanation(context: &Context) -> String {
translated(context, StockMessage::ChatUnencryptedExplanation)
}
/// Stock string: `You are using the legacy option "Move automatically to DeltaChat Folder`…
pub(crate) fn mvbox_move_deprecation(context: &Context) -> String {
translated(context, StockMessage::MvboxMoveDeprecation)
}
impl Viewtype {
/// returns Localized name for message viewtype
pub fn to_locale_string(&self, context: &Context) -> String {
+2 -2
View File
@@ -19,7 +19,7 @@ use crate::token::Namespace;
use crate::tools::time;
use crate::transport::{ConfiguredLoginParamJson, sync_transports};
use crate::{message, stock_str, token};
use std::collections::HashSet;
use std::collections::BTreeSet;
/// Whether to send device sync messages. Aimed for usage in the internal API.
#[derive(Debug, PartialEq)]
@@ -383,7 +383,7 @@ impl Context {
}
async fn sync_message_deletion(&self, msgs: &Vec<String>) -> Result<()> {
let mut modified_chat_ids = HashSet::new();
let mut modified_chat_ids = BTreeSet::new();
let mut msg_ids = Vec::new();
for rfc724_mid in msgs {
if let Some(msg_id) = message::rfc724_mid_exists(self, rfc724_mid).await? {
+21 -7
View File
@@ -40,6 +40,7 @@ use crate::message::{Message, MessageState, MsgId, update_msg_state};
use crate::mimeparser::{MimeMessage, SystemMessage};
use crate::receive_imf::receive_imf;
use crate::securejoin::{get_securejoin_qr, join_securejoin};
use crate::smtp::msg_has_pending_smtp_job;
use crate::stock_str::StockStrings;
use crate::tools::time;
@@ -566,7 +567,6 @@ impl TestContext {
.unwrap();
ctx.set_config(Config::BccSelf, Some("1")).await.unwrap();
ctx.set_config(Config::SyncMsgs, Some("0")).await.unwrap();
ctx.set_config(Config::MvboxMove, Some("0")).await.unwrap();
Self {
ctx,
@@ -658,10 +658,7 @@ impl TestContext {
.execute("DELETE FROM smtp WHERE id=?;", (rowid,))
.await
.expect("failed to remove job");
if !self
.ctx
.sql
.exists("SELECT COUNT(*) FROM smtp WHERE msg_id=?", (msg_id,))
if !msg_has_pending_smtp_job(self, msg_id)
.await
.expect("Failed to check for more jobs")
{
@@ -715,7 +712,8 @@ impl TestContext {
}
pub async fn get_smtp_rows_for_msg<'a>(&'a self, msg_id: MsgId) -> Vec<SentMessage<'a>> {
self.ctx
let sent_msgs = self
.ctx
.sql
.query_map_vec(
"SELECT id, msg_id, mime, recipients FROM smtp WHERE msg_id=?",
@@ -737,7 +735,23 @@ impl TestContext {
sender_context: &self.ctx,
recipients,
})
.collect()
.collect();
self.ctx
.sql
.execute("DELETE FROM smtp WHERE msg_id=?", (msg_id,))
.await
.expect("Delete smtp jobs");
update_msg_state(&self.ctx, msg_id, MessageState::OutDelivered)
.await
.expect("Update message state");
self.sql
.execute(
"UPDATE msgs SET timestamp_sent=? WHERE id=?",
(time(), msg_id),
)
.await
.expect("Update timestamp_sent");
sent_msgs
}
/// Parses a message.
+146 -1
View File
@@ -4,16 +4,19 @@ use pretty_assertions::assert_eq;
use crate::EventType;
use crate::chat;
use crate::chat::send_msg;
use crate::config::Config;
use crate::contact;
use crate::download::{DownloadState, PRE_MSG_ATTACHMENT_SIZE_THRESHOLD, PostMsgMetadata};
use crate::message::{Message, MessageState, Viewtype, delete_msgs, markseen_msgs};
use crate::mimeparser::MimeMessage;
use crate::param::Param;
use crate::reaction::{get_msg_reactions, send_reaction};
use crate::receive_imf::receive_imf;
use crate::summary::assert_summary_texts;
use crate::test_utils::TestContextManager;
use crate::tests::pre_messages::util::{
send_large_file_message, send_large_image_message, send_large_webxdc_message,
big_webxdc_app, send_large_file_message, send_large_image_message, send_large_webxdc_message,
};
use crate::webxdc::StatusUpdateSerial;
@@ -115,7 +118,13 @@ async fn test_receive_pre_message() -> Result<()> {
let msg = bob.recv_msg(&pre_message).await;
assert_eq!(msg.download_state, DownloadState::Available);
assert_summary_texts(&msg, bob, "👤 test").await;
let chat_id = msg.chat_id;
assert!(bob.recv_msg_opt(&pre_message).await.is_none());
let msg = Message::load_from_db(bob, msg.id).await?;
assert_eq!(msg.download_state, DownloadState::Available);
assert_summary_texts(&msg, bob, "👤 test").await;
assert_eq!(msg.chat_id, chat_id);
Ok(())
}
@@ -513,6 +522,99 @@ async fn test_webxdc_update_for_not_downloaded_instance() -> Result<()> {
Ok(())
}
/// Tests receiving of a large webxdc with updates attached to the the .xdc message.
///
/// Updates are sent in a post message and should be assigned to xdc instance
/// once post message is downloaded.
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_webxdc_updates_in_post_message_after_pre_message() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
let bob = &tcm.bob().await;
let alice_chat_id = alice.create_chat_id(bob).await;
let big_webxdc_app = big_webxdc_app().await?;
let mut alice_instance = Message::new(Viewtype::Webxdc);
alice_instance.set_file_from_bytes(alice, "test.xdc", &big_webxdc_app, None)?;
alice_instance.set_text("Test".to_string());
alice_chat_id
.set_draft(alice, Some(&mut alice_instance))
.await?;
alice
.send_webxdc_status_update(alice_instance.id, r#"{"payload":42, "info":"i"}"#)
.await?;
send_msg(alice, alice_chat_id, &mut alice_instance).await?;
let post_message = alice.pop_sent_msg().await;
let pre_message = alice.pop_sent_msg().await;
let bob_instance = bob.recv_msg(&pre_message).await;
assert_eq!(bob_instance.download_state, DownloadState::Available);
bob.recv_msg_trash(&post_message).await;
let bob_instance = Message::load_from_db(bob, bob_instance.id).await?;
assert_eq!(bob_instance.download_state, DownloadState::Done);
assert_eq!(
bob.get_webxdc_status_updates(bob_instance.id, StatusUpdateSerial::new(0))
.await?,
r#"[{"payload":42,"info":"i","serial":1,"max_serial":1}]"#
);
Ok(())
}
/// Tests receiving of a large webxdc post-message with updates attached
/// to the the .xdc post-message when pre-message arrives later.
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_webxdc_updates_in_post_message_without_pre_message() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
let bob = &tcm.bob().await;
let alice_chat_id = alice.create_chat_id(bob).await;
let big_webxdc_app = big_webxdc_app().await?;
let mut alice_instance = Message::new(Viewtype::Webxdc);
alice_instance.set_file_from_bytes(alice, "test.xdc", &big_webxdc_app, None)?;
alice_instance.set_text("Test".to_string());
alice_chat_id
.set_draft(alice, Some(&mut alice_instance))
.await?;
alice
.send_webxdc_status_update(alice_instance.id, r#"{"payload":42, "info":"i"}"#)
.await?;
send_msg(alice, alice_chat_id, &mut alice_instance).await?;
let post_message = alice.pop_sent_msg().await;
let pre_message = alice.pop_sent_msg().await;
// Bob receives post-message first.
let bob_instance = bob.recv_msg(&post_message).await;
assert_eq!(bob_instance.download_state, DownloadState::Done);
assert_eq!(
bob.get_webxdc_status_updates(bob_instance.id, StatusUpdateSerial::new(0))
.await?,
r#"[{"payload":42,"info":"i","serial":1,"max_serial":1}]"#
);
// Bob may still receive pre-message later.
bob.recv_msg_trash(&pre_message).await;
let bob_instance = Message::load_from_db(bob, bob_instance.id).await?;
assert_eq!(bob_instance.download_state, DownloadState::Done);
assert_eq!(
bob.get_webxdc_status_updates(bob_instance.id, StatusUpdateSerial::new(0))
.await?,
r#"[{"payload":42,"info":"i","serial":1,"max_serial":1}]"#
);
Ok(())
}
/// Test mark seen pre-message
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_markseen_pre_msg() -> Result<()> {
@@ -701,3 +803,46 @@ async fn test_chatlist_event_on_post_msg_download() -> Result<()> {
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_bot_pre_message_notifications() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = tcm.alice().await;
let bob = tcm.bob().await;
bob.set_config_bool(Config::Bot, true).await?;
let alice_group_id = alice.create_group_with_members("test group", &[&bob]).await;
let (pre_message, post_message, _alice_msg_id) = send_large_file_message(
&alice,
alice_group_id,
Viewtype::File,
&vec![0u8; (PRE_MSG_ATTACHMENT_SIZE_THRESHOLD + 1) as usize],
)
.await?;
// Bob receives pre-message
bob.evtracker.clear_events();
receive_imf(&bob, pre_message.payload().as_bytes(), false).await?;
// Verify Bob does NOT get an IncomingMsg event for the pre-message
assert!(
bob.evtracker
.get_matching_opt(&bob, |e| matches!(e, EventType::IncomingMsg { .. }))
.await
.is_none()
);
// Bob receives post-message
receive_imf(&bob, post_message.payload().as_bytes(), false).await?;
// Verify Bob DOES get an IncomingMsg event for the complete message
bob.evtracker
.get_matching(|e| matches!(e, EventType::IncomingMsg { .. }))
.await;
let msg = bob.get_last_msg().await;
assert_eq!(msg.download_state, DownloadState::Done);
Ok(())
}
+34 -13
View File
@@ -56,26 +56,30 @@ async fn test_sending_pre_message() -> Result<()> {
.is_some()
);
let post_rfc724_mid = post_message_parsed
.headers
.get_header_value(HeaderDef::MessageId);
assert_eq!(
post_message_parsed
.headers
.get_header_value(HeaderDef::MessageId),
post_rfc724_mid,
Some(format!("<{}>", msg.rfc724_mid)),
"Post-Message should have the rfc message id of the database message"
);
let pre_rfc724_mid = pre_message_parsed
.headers
.get_header_value(HeaderDef::MessageId);
assert_ne!(
pre_message_parsed
.headers
.get_header_value(HeaderDef::MessageId),
post_message_parsed
.headers
.get_header_value(HeaderDef::MessageId),
pre_rfc724_mid, post_rfc724_mid,
"message ids of Pre-Message and Post-Message should be different"
);
assert_eq!(
pre_rfc724_mid,
Some(format!("<{}>", msg.pre_rfc724_mid)),
"Unexpected pre-message RFC 724 ID"
);
let decrypted_post_message = bob.parse_msg(post_message).await;
assert_eq!(decrypted_post_message.decrypting_failed, false);
assert!(decrypted_post_message.decryption_error.is_none());
assert_eq!(
decrypted_post_message.header_exists(HeaderDef::ChatPostMessageId),
false
@@ -86,9 +90,7 @@ async fn test_sending_pre_message() -> Result<()> {
decrypted_pre_message
.get_header(HeaderDef::ChatPostMessageId)
.map(String::from),
post_message_parsed
.headers
.get_header_value(HeaderDef::MessageId)
post_rfc724_mid,
);
assert!(
pre_message_parsed
@@ -98,6 +100,25 @@ async fn test_sending_pre_message() -> Result<()> {
"no Chat-Post-Message-ID header in unprotected headers of Pre-Message"
);
chat::resend_msgs(alice, &[msg_id]).await?;
let smtp_rows = alice.get_smtp_rows_for_msg(msg_id).await;
assert_eq!(smtp_rows.len(), 2);
let pre_message_parsed = mailparse::parse_mail(smtp_rows[0].payload.as_bytes())?;
assert_eq!(
pre_message_parsed
.headers
.get_header_value(HeaderDef::MessageId),
pre_rfc724_mid
);
let post_message_parsed = mailparse::parse_mail(smtp_rows[1].payload.as_bytes())?;
assert_eq!(
post_message_parsed
.headers
.get_header_value(HeaderDef::MessageId),
post_rfc724_mid
);
Ok(())
}
+9 -5
View File
@@ -37,10 +37,7 @@ pub async fn send_large_file_message<'a>(
Ok((pre_message.to_owned(), post_message.to_owned(), msg_id))
}
pub async fn send_large_webxdc_message<'a>(
sender: &'a TestContext,
target_chat: ChatId,
) -> Result<(SentMessage<'a>, SentMessage<'a>, MsgId)> {
pub async fn big_webxdc_app() -> Result<Vec<u8>> {
let futures_cursor = FuturesCursor::new(Vec::new());
let mut buffer = futures_cursor.compat_write();
let mut writer = ZipFileWriter::with_tokio(&mut buffer);
@@ -51,7 +48,14 @@ pub async fn send_large_webxdc_message<'a>(
)
.await?;
writer.close().await?;
let big_webxdc_app = buffer.into_inner().into_inner();
Ok(buffer.into_inner().into_inner())
}
pub async fn send_large_webxdc_message<'a>(
sender: &'a TestContext,
target_chat: ChatId,
) -> Result<(SentMessage<'a>, SentMessage<'a>, MsgId)> {
let big_webxdc_app = big_webxdc_app().await?;
send_large_file_message(sender, target_chat, Viewtype::Webxdc, &big_webxdc_app).await
}

Some files were not shown because too many files have changed in this diff Show More