Compare commits

...
Author SHA1 Message Date
iequidoo 974e32dd76 fix: Don't decrease member add/remove timestamps if they aren't far away in the future
We shouldn't decrease `add_timestamp` and `remove_timestamp` in the `chats_contacts` table normally,
even if remote changes arrive reordered. This particularly makes sense for ad-hoc groups (see
`chat::update_chat_contacts_table()` in `apply_group_changes()`) and in case if we join an encrypted
group which we were a member of before (see `chat::add_to_chat_contacts_table()` call).

Still, limit already stored timestamps in case local clock was in the future and is set back
now. But our clock may be slow, so limit stored timestamps with a remote timestamp if it's
bigger.

NB: `receive_imf::update_chats_contacts_timestamps()` already only increases timestamps, but it's
used only for handling of the "Chat-Group-Member-Timestamps" header, i.e. for encrypted groups.
2026-06-07 21:45:47 -03:00
iequidoo c91608e9f1 fix: Don't send removal message to contact that hasn't been a chat member (#8298)
I.e. don't fail `remove_contact_from_chat()` for such a contact because there may be a race
condition with a remote removal of the contact done without trace, but don't send a removal message
and sync message in this case and don't emit a `ChatModified` event.

If a contact is already a past member, we still send a removal message to the chat, this is safe and
protects from lost removal messages, so there's no need to complicate the code in this case.
2026-06-07 13:23:16 -03:00
link2xt 207c2e6e4c feat: switch to aws-lc-rs cryptography provider for Rustls
aws-lc-rs is the default provider for Rustls.
We have been using ring as the default provider
to avoid duplicate dependency
because iroh supports only ring in 0.35.0,
but with version 1.0 it will be possible
to select other crypto providers
and we can switch to using more common default.

This change also enables "tls12" feature explicitly,
otherwise if there is no other dependency
requiring it, only TLS 1.3 will be supported
and we want to keep supporting TLS 1.2.
2026-06-06 09:56:31 +00:00
dependabot[bot] c0705a8d92 chore(deps): bump taiki-e/install-action from 2.79.2 to 2.79.10
Bumps [taiki-e/install-action](https://github.com/taiki-e/install-action) from 2.79.2 to 2.79.10.
- [Release notes](https://github.com/taiki-e/install-action/releases)
- [Changelog](https://github.com/taiki-e/install-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/taiki-e/install-action/compare/213ccc1a076163c093f914550b94feb90fab916d...60ae4ce63c7aeb6e96d7f572c1ec7fafbb17ca80)

---
updated-dependencies:
- dependency-name: taiki-e/install-action
  dependency-version: 2.79.10
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-06-03 19:55:34 +00:00
dependabot[bot] 18ce5a02cc chore(deps): bump EmbarkStudios/cargo-deny-action from 2.0.18 to 2.0.19
Bumps [EmbarkStudios/cargo-deny-action](https://github.com/embarkstudios/cargo-deny-action) from 2.0.18 to 2.0.19.
- [Release notes](https://github.com/embarkstudios/cargo-deny-action/releases)
- [Commits](https://github.com/embarkstudios/cargo-deny-action/compare/6c8f9facfa5047ec02d8485b6bf52b587b7777d1...a531616d8ce3b9177443e48a1159bc945a099823)

---
updated-dependencies:
- dependency-name: EmbarkStudios/cargo-deny-action
  dependency-version: 2.0.19
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-06-03 19:55:13 +00:00
dependabot[bot] e87e269f98 chore(cargo): bump serde_json from 1.0.149 to 1.0.150
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.149 to 1.0.150.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.149...v1.0.150)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-version: 1.0.150
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-06-03 19:54:21 +00:00
dependabot[bot] 4bb557cf53 chore(cargo): bump log from 0.4.29 to 0.4.30
Bumps [log](https://github.com/rust-lang/log) from 0.4.29 to 0.4.30.
- [Release notes](https://github.com/rust-lang/log/releases)
- [Changelog](https://github.com/rust-lang/log/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/log/compare/0.4.29...0.4.30)

---
updated-dependencies:
- dependency-name: log
  dependency-version: 0.4.30
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-06-03 19:54:02 +00:00
dependabot[bot] 9b4503e3f5 chore(cargo): bump tokio from 1.52.1 to 1.52.3
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.52.1 to 1.52.3.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.52.1...tokio-1.52.3)

---
updated-dependencies:
- dependency-name: tokio
  dependency-version: 1.52.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-06-03 19:53:16 +00:00
dependabot[bot] 4428382433 chore(cargo): bump pin-project from 1.1.11 to 1.1.13
Bumps [pin-project](https://github.com/taiki-e/pin-project) from 1.1.11 to 1.1.13.
- [Release notes](https://github.com/taiki-e/pin-project/releases)
- [Changelog](https://github.com/taiki-e/pin-project/blob/main/CHANGELOG.md)
- [Commits](https://github.com/taiki-e/pin-project/compare/v1.1.11...v1.1.13)

---
updated-dependencies:
- dependency-name: pin-project
  dependency-version: 1.1.13
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-06-03 15:24:20 +00:00
Hocuri 8c56b63f21 feat: Add number_of_keys to statistics (#8297)
As of 3b29469102, all public keys except
for the currently-used one are ignored. But Delta Chat still tries to
use all _private_ keys for decryption.

This PR adds the information how many keys there are in the database. If
there is more than one, we know that the user imported their own key
back when that was possible. This info is interesting both for ourselves
and for the Cispa researchers that want to write a paper about Delta
Chat.

An alternative to this PR would be to make `key_create_timestamps`
actually contain the full list of key creation timestamps. I'm not sure
if that's worth the effort, though.

Follow-up to #8293
2026-06-03 12:36:31 +02:00
Hocuri 50e83f2072 feat: Add cryptography-related statistics (#8293)
This adds `number_of_transports`, `key_version`, `key_algorithm`,
`pubkey_size`.
2026-06-02 10:26:53 +02:00
link2xt 4a94a34c6d chore: update preloaded DNS cache 2026-06-01 11:29:25 +00:00
Hocuri bd6c9908e4 fix: Update the channel title after joining if the QR code included a wrong title (#8260)
Fix https://github.com/chatmail/core/issues/8250. Not sure why anyone
would tamper with the link, but maybe the name gets ellipsized because
it is very long, or maybe we decide to ellipsize more aggressively in
the future.

The fix is easy enough, just set the GroupNameTimestamp when creating
the channel, so that `HeaderDef::ChatGroupNameTimestamp` gets set when
sending a message and the logic in
`apply_chat_name_avatar_and_description_changes()` recognizes the
incoming name's timestamp as newer than `chat_group_name_timestamp`
(which is 0 right after joining a chat).

The test is somewhat hacky and depends on the title being the last
parameter in the invite code URL, but this is fine; if we ever change
that, then the test will fail and we need to modify it.

This also removes some wrong doc comments.

Since groups are unpromoted until sending a first message, the same fix isn't needed for groups.
2026-06-01 08:48:01 +00:00
iequidoo dd5ec9621b feat: Add IMAP folder to Context::get_info() (#8285)
Note that the default value "INBOX" isn't shown however.
2026-05-31 19:59:31 -03:00
link2xt 92d522473f refactor: remove support for building "source" packages for deltachat-rpc-server
This hack was needed to "build" Android packages
by repackaging Linux packages on-the-fly,
but PyPI has since added Android support
and we are using it directly.
2026-05-31 08:45:39 +00:00
iequidoo 049fd4f355 refactor: Remove misplaced comment
It was added in ff99e1997 "inital commit" and even there it looks misplaced.
2026-05-29 15:39:15 -03:00
link2xt 7ea637c930 chore: bump version to 2.51.0-dev 2026-05-29 18:32:58 +02:00
link2xt 3a1b0d4679 chore(release): prepare for 2.51.0 2026-05-29 18:05:43 +02:00
WofWca 7a60c79301 feat: message info: add database msg ID
We have recently removed it from the normal "message info" dialog
at least on Delta Chat Desktop, so without this one can't easily find
the ID of a particular message,
which is useful for debugging.
2026-05-29 15:33:14 +04:00
link2xt 0bc7849e8a build(nix): fix downloads from crates.io in nix builds
crates.io recently started rejecting requests to https://crates.io/api/v1/crates
made by Nix: https://github.com/rust-lang/crates.io/issues/13482
This resulted in failing to download any dependencies
from crates.io with 403 error during Rust package builds.
The problem was solved in nixpkgs by switching
to CDN URL https://static.crates.io/crates
in https://github.com/NixOS/nixpkgs/pull/524985

As of writing this on 2026-05-29
the fix is on "master" branch of nixpkgs,
but not on "nixos-unstable" or "nixpkgs-unstable"
branches yet according to https://nixpk.gs/pr-tracker.html?pr=524985
so I have switched nixpkgs to "master" branch to get the problem fixed.

naersk needs a similar fix. I opened a PR for it
https://github.com/nix-community/naersk/pull/391
but because the fix is not merged yet,
switched to my PR branch in flake.nix.

Updating nixpkgs required some minor changes,
e.g. sphinx_rtd_theme has been renamed to sphinx-rtd-theme,
pthreads in Windows builds had to be moved to buildInputs to fix
"Refusing to evaluate package 'mingw_w64-pthreads-13.0.0'
in /nix/store/f78lkqnk63pd0kf52zf2wcx35p1nnalr-source/pkgs/os-specific/windows/mingw-w64/headers.nix:35
because it is not available on the requested hostPlatform"
2026-05-29 10:16:16 +00:00
link2xt dfea8b0134 test: test encrypted messages in test_delete_expired_imap_messages 2026-05-29 08:37:21 +00:00
link2xt 376c819374 feat: immediately remove all encrypted messages from the server in single-device mode 2026-05-29 08:37:21 +00:00
link2xt 02827406f3 docs: fix reference in delete_expired_imap_messages comment
It should refer to delete_expired_messages,
not to itself.
2026-05-29 08:37:21 +00:00
link2xt 9d9f61d9eb refactor: make should_delete_all_downloaded_messages non-async
Pass bcc_self into it as a boolean.
2026-05-29 08:37:21 +00:00
link2xt a2816d7bd3 test: test bcc_self in test_delete_expired_imap_messages 2026-05-29 08:37:21 +00:00
iequidoo 9719aa1415 fix: Don't make message OutDelivered after successful resending to new broadcast member
Follow-up to 970222f. We shouldn't update the message state to `OutDelivered` if message sending
failed before and the message was re-sent successfully only to a new broadcast member.
2026-05-29 00:35:46 -03:00
Tom Niget 5733a783fb feat: follow certificate check parameter in autoconfig 2026-05-28 20:18:40 +00:00
holger krekel c26b6a017c fix: syntax error in older migration 2026-05-27 06:20:50 +00:00
dependabot[bot] bb6a478430 chore(deps): bump taiki-e/install-action from 2.78.1 to 2.79.2
Bumps [taiki-e/install-action](https://github.com/taiki-e/install-action) from 2.78.1 to 2.79.2.
- [Release notes](https://github.com/taiki-e/install-action/releases)
- [Changelog](https://github.com/taiki-e/install-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/taiki-e/install-action/compare/184183c2401be73c3bf42c2e61268aa5855379c1...213ccc1a076163c093f914550b94feb90fab916d)

---
updated-dependencies:
- dependency-name: taiki-e/install-action
  dependency-version: 2.79.2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-05-27 05:27:42 +00:00
link2xt 480248a168 refactor: remove pre_encrypt_mime_hook
It is only used in a single test
and the test is more reliable by having a blob .eml
that will not be accidentally modified
by mimefactory changes.
2026-05-26 08:10:51 +00:00
dependabot[bot] 627f98915d chore(deps): bump zizmorcore/zizmor-action from 0.5.3 to 0.5.6
Bumps [zizmorcore/zizmor-action](https://github.com/zizmorcore/zizmor-action) from 0.5.3 to 0.5.6.
- [Release notes](https://github.com/zizmorcore/zizmor-action/releases)
- [Commits](https://github.com/zizmorcore/zizmor-action/compare/b1d7e1fb5de872772f31590499237e7cce841e8e...5f14fd08f7cf1cb1609c1e344975f152c7ee938d)

---
updated-dependencies:
- dependency-name: zizmorcore/zizmor-action
  dependency-version: 0.5.6
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-05-26 05:06:42 +00:00
link2xt 24848c0265 fix: do not fail to receive post-message with status updates for deleted webxdc
receive_imf should not fail in this case,
but trash the post-message and updates.

The problem with previously used message::rfc724_mid_exists check
was that rfc724_mid_exists may return a trashed message.
2026-05-23 21:57:58 +00:00
link2xt 98123afb62 test: test IPv6 addresses in HTTP(S) proxies 2026-05-23 21:49:37 +00:00
link2xt 16b65c59bf fix: do not try to resolve proxy IPv6 addresses in square brackets
"[::1]" is not a hostname, but an IPv6 address,
and should be passed as "::1" to tokio::net functions.

Otherwise connecting to proxy addresses such as `socks5://[::1]:9150`
fails by trying to resolve IPv6 address with DNS.
2026-05-23 07:01:05 +00:00
link2xt bafb5f71ad chore: bump version to 2.51.0-dev 2026-05-22 21:41:03 +02:00
link2xt 784a6abb3b chore(release): prepare for 2.50.0 2026-05-22 20:16:32 +02:00
dependabot[bot] e91f2718dd chore(deps): bump taiki-e/install-action from 2.77.1 to 2.78.1
Bumps [taiki-e/install-action](https://github.com/taiki-e/install-action) from 2.77.1 to 2.78.1.
- [Release notes](https://github.com/taiki-e/install-action/releases)
- [Changelog](https://github.com/taiki-e/install-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/taiki-e/install-action/compare/cca35edeb1d01366c2843b68fc3ca441446d73d3...184183c2401be73c3bf42c2e61268aa5855379c1)

---
updated-dependencies:
- dependency-name: taiki-e/install-action
  dependency-version: 2.77.6
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-05-22 17:20:34 +00:00
Hocuri 961cacd795 fix: Correctly delete unneeded messages on the server; Remove "Delete Messages from Server" (delete_server_after) config option (#8240)
Fix https://github.com/chatmail/core/issues/8195
Supersedes https://github.com/chatmail/core/pull/8217

The most interesting change is in `delete_expired_imap_messages()`
because there, messages are marked for deletion on single-device
chatmail profiles.

The logic in `delete_expired_imap_messages()` was wrong before, and
pre-messages were not deleted at all. This is fixed by also querying
`pre_rfc724_mid` in addition to `rfc724_mid` from the `msgs` table.
In order not to make the SQL statement too complex, I split it into two.

WRT tests:
- `test_immediate_autodelete()` tests the auto-deletion;
`test_imap_autodelete_fully_downloaded_msg()` tests that even for a
message that is split into pre- and post-message, both messages are
deleted.
- A lot of tests test that if bcc_self is on, messages are not
auto-deleted. E.g. `test_one_account_send_bcc_setting()` and
`test_markseen_message_and_mdn()` relies on the fact that messages stay
on the server when bcc_self is on.
- Unfortunately, it is not possible to test that messages are only
deleted for chatmail servers, because we don't have any tests that use a
non-chatmail server.
2026-05-22 16:45:50 +00:00
66 changed files with 1429 additions and 918 deletions
+2 -2
View File
@@ -62,7 +62,7 @@ jobs:
with:
show-progress: false
persist-credentials: false
- uses: EmbarkStudios/cargo-deny-action@6c8f9facfa5047ec02d8485b6bf52b587b7777d1
- uses: EmbarkStudios/cargo-deny-action@a531616d8ce3b9177443e48a1159bc945a099823
with:
arguments: --workspace --all-features --locked
command: check
@@ -146,7 +146,7 @@ jobs:
cache-bin: false
- name: Install nextest
uses: taiki-e/install-action@cca35edeb1d01366c2843b68fc3ca441446d73d3
uses: taiki-e/install-action@60ae4ce63c7aeb6e96d7f572c1ec7fafbb17ca80
with:
tool: nextest
-1
View File
@@ -63,7 +63,6 @@ jobs:
- deltachat-rpc-server-armv7l-linux-wheel
- deltachat-rpc-server-i686-linux
- deltachat-rpc-server-i686-linux-wheel
- deltachat-rpc-server-source
- deltachat-rpc-server-win32
- deltachat-rpc-server-win32-wheel
- deltachat-rpc-server-win64
+1 -1
View File
@@ -23,4 +23,4 @@ jobs:
persist-credentials: false
- name: Run zizmor
uses: zizmorcore/zizmor-action@b1d7e1fb5de872772f31590499237e7cce841e8e # v0.5.3
uses: zizmorcore/zizmor-action@5f14fd08f7cf1cb1609c1e344975f152c7ee938d # v0.5.6
+194
View File
@@ -1,5 +1,197 @@
# Changelog
## [2.51.0] - 2026-05-29
### Features / Changes
- Follow certificate check parameter in autoconfig.
- Immediately remove all encrypted messages from the server in single-device mode.
### Fixes
- Fix syntax error in `only_fetch_mvbox` migration 150 resulting in failure to upgrade for `only_fetch_mvbox` users.
- Do not try to resolve proxy IPv6 addresses in square brackets.
- Do not fail to receive post-message with status updates for deleted webxdc.
- Don't make message `OutDelivered` after successful resending to new broadcast member.
### Build system
- nix: fix downloads from crates.io in nix builds.
### Documentation
- Fix reference in `delete_expired_imap_messages` comment.
### Refactor
- Remove `pre_encrypt_mime_hook`.
- Make `should_delete_all_downloaded_messages` non-async.
### Tests
- Test IPv6 addresses in HTTP(S) proxies.
- Test `bcc_self` in `test_delete_expired_imap_messages`.
- Test encrypted messages in `test_delete_expired_imap_messages`.
### Miscellaneous Tasks
- Bump version to 2.51.0-dev.
- deps: bump zizmorcore/zizmor-action from 0.5.3 to 0.5.6.
- deps: bump taiki-e/install-action from 2.78.1 to 2.79.2.
## [2.50.0] - 2026-05-22
### API-Changes
- Add JSON-RPC APIs for location streaming.
- [**breaking**] Remove unused config `smtp_certificate_checks`.
- Deprecate old server config keys that were replaced by `add_or_update_transport()`.
- [**breaking**] remove `dc_delete_all_locations`.
- [**breaking**] Remove unused `info_only` option when loading a chatlist ([#8171](https://github.com/chatmail/core/pull/8171)).
- [**breaking**] location: avoid repeating module name in function names
- [**breaking**] deltachat-rpc-client: remove deprecated `get_fresh_messages_in_arrival_order()`.
- Remove unused `set_draft_vcard()` JSON-RPC API.
- Remove mostly-unused `sign_unencrypted` config ([#8190](https://github.com/chatmail/core/pull/8190)).
### Features / Changes
- [**breaking**] Remove `mvbox_move` and `only_fetch_mvbox` configs. Transports have `folder` configuration to watch the folder other than `INBOX` as a replacement for `only_fetch_mvbox`. Non-chatmail transports no longer watch mvbox, each transport watches exactly one folder now.
- Add `force_encryption` config to ignore incoming unencrypted messages and enforce encryption for outgoing messages.
- Remove "Delete Messages from Server" (`delete_server_after`) config ([#8240](https://github.com/chatmail/core/pull/8240)).
- Remove `show_emails` config.
- Remove non-sticker heuristics and `force_sticker()`. UIs should make sure not to send images from gallery such as screenshots as stickers.
- Enable PQC (Post-Quantum Cryptography) support for OpenPGP. We do not generate PQC keys yet, this step is needed for forward compatibility.
- Resend the last 10 messages to new broadcast member ([#8151](https://github.com/chatmail/core/pull/8151)).
- Allow TLS connections with invalid certificate if the key is unchanged.
- Add `is_app_sender` and `is_broadcast` contexts for webxdc.
- Increase the resolution-limit `WORSE_AVATAR_SIZE` from 128 to 256.
- Change multiplier to 7/8 when scaling down avatars.
- Add error cause to connectivity view for IMAP errors.
- Remove the largely-unused ability to send multiple reactions to one message ([#8131](https://github.com/chatmail/core/pull/8131)).
- Don't show non-delivery-notfications in broadcast channels ([#8159](https://github.com/chatmail/core/pull/8159)).
- Adapt quota warning to automatic cleanup.
- Remove `Content-Description` and `Content-Disposition` from `multipart/encrypted` parts.
- Log all connection attempt errors instead of the first one.
- Remove workaround for old filtermail (part of chatmail relay) which expected exact number of newlines in OpenPGP messages.
- Remove key fingerprint from `Context.get_info()`.
- Mask local part of email addresses in `used_transport_settings`.
### Fixes
- Trash no-op messages about self being added to groups.
- `decide_chat_assignment`: Log correct `post_msg_exists` value.
- Don't send `Chat-Group-Name*` headers for InBroadcast-s.
- Restart io on transport deletion.
- Never remove primary transport when applying `SyncTransports` message.
- Set Param::GuaranteeE2ee before preparing message blob ([#8090](https://github.com/chatmail/core/pull/8090)).
- `fetch_single_msg()`: Lock `fetch_msgs_mutex` before fetching.
- Set dir to "auto" in body tag when converting plain-text to HTML ([#8227](https://github.com/chatmail/core/pull/8227)).
- Scale up contacts messaged in groups to `IncomingTo`.
- Do not sort prefetched messages by INTERNALDATE.
- Don't resort re-sent message to the bottom ([#8145](https://github.com/chatmail/core/pull/8145)).
- Ensure that message being sent is added to the bottom ([#8027](https://github.com/chatmail/core/pull/8027)).
- Don't receive message if a deletion request was received before ([#8143](https://github.com/chatmail/core/pull/8143)).
- Emit `MsgsChanged`, not `IncomingMsg`, for messages only having special parts ([#8157](https://github.com/chatmail/core/pull/8157)).
- Generate new pre-message `Message-ID` when forwarding.
- use correct dir converting plaintext to HTML ([#8248](https://github.com/chatmail/core/pull/8248)).
- hide connectivity HTML quota if not supported.
- Delete pre-messages on the server for single-device chatmail transports ([#8240](https://github.com/chatmail/core/pull/8240)).
### Build system
- Upgrade rustls-webpki to 0.103.12.
- Remove coredeps `Dockerfile`.
- Increase MSRV to 1.89.
### CI
- Remove Concourse CI pipelines.
- Update Rust to 1.95.0.
- Do not store Rust cache from PRs.
- Set cache-bin to "false" for `swatinem/rust-cache` action.
- Use `--locked` flag with `cargo build`.
- Upgrade `cargo-deny-action` to v2.0.17.
### Documentation
- Update `echobot_no_hooks.py` example.
- Discourage `into()`, `try_into()` and `parse()` ([#8180](https://github.com/chatmail/core/pull/8180)).
- Remove outdated comment about "quota warning" device message.
- Update README.md: Use ci-chatmail instead of nine ([#8238](https://github.com/chatmail/core/pull/8238)).
- spec: remove AEAP section.
### Performance
- Enable `clippy::large_futures` lint.
- Stop sending locations concurrently.
- Set location for all accounts in parallel.
- `is_self_addr()`: Employ the config cache to optimize for `ConfiguredAddr` passed.
### Refactor
- Get rid of `MessageState::{OutPreparing,OutMdnRcvd}` in the db.
- Make HTML parser non-async.
- Replace `HashSet` with `BTreeSet`.
- Rename `EnteredLoginParam::load()` and save() to `load_legacy()` and `save_legacy()`.
- Remove unnecessary async block in `dc_set_location`.
- Remove unused Authentication-Results parsing ([#8172](https://github.com/chatmail/core/pull/8172)).
- Remove mostly-unused function `get_secondary_self_addrs()` ([#8173](https://github.com/chatmail/core/pull/8173)).
- Use `self_fingerprint()` where it makes sense ([#8174](https://github.com/chatmail/core/pull/8174)).
- Split `is_sending_locations_to_chat()` into two functions.
- Use regular functions rather than FromStr impls ([#8178](https://github.com/chatmail/core/pull/8178)).
- Make `Fingerprint` not implement `Display` ([#8177](https://github.com/chatmail/core/pull/8177)).
- Don't temporarily extend `signatures` for signed-only messages.
- Use some more let..else.
- Remove outdated comment.
- Un-nest `handle_edit_delete`.
- Drop support for replacing partial download stubs.
### Tests
- Use `displayname` instead of `show_emails` for config cache test.
- Remove unused test data related to Authentication-Result parsing ([#8175](https://github.com/chatmail/core/pull/8175)).
- `EventTracker::get_matching_opt`: Return the first matching event, not last.
- Set email addresses explicitly for the test accounts.
- Use encrypted messages in more tests.
- Add `TestContext.allow_unencrypted()`.
- Online test for legacy Secure-Join key request.
### Miscellaneous Tasks
- cargo: bump rand from 0.9.2 to 0.9.3.
- deps: bump taiki-e/install-action from 2.64.0 to 2.74.0.
- add exception for RUSTSEC-2026-0097.
- deps: bump swatinem/rust-cache from 2.8.2 to 2.9.1.
- cargo: upgrade rand 0.8.5 to rand 0.8.6.
- deps: bump zizmorcore/zizmor-action from 0.5.2 to 0.5.3.
- deps: bump pypa/gh-action-pypi-publish from 1.13.0 to 1.14.0.
- update provider database.
- cargo: update rustls-webpki to 0.103.13.
- cargo: bump openssl from 0.10.72 to 0.10.78.
- Apply rustmft after the previous commit.
- json-rpc: deprecate `send_sticker` ([#8189](https://github.com/chatmail/core/pull/8189)).
- deps: bump cachix/install-nix-action from 31.9.1 to 31.10.5.
- deps: bump taiki-e/install-action from 2.75.10 to 2.75.19.
- update astral-tokio-tar from 0.6.0 to 0.6.1.
- add exceptions for hickory-proto 0.25.2 in deny.toml.
- cargo: bump blake3 from 1.8.3 to 1.8.5.
- deny.toml: add cpufeatures duplicate dependency exception.
- cargo: bump hyper from 1.8.1 to 1.9.0.
- cargo: bump tokio from 1.50.0 to 1.52.1.
- cargo: bump libc from 0.2.184 to 0.2.186.
- cargo: bump colorutils-rs from 0.7.6 to 0.8.0.
- cargo: bump data-encoding from 2.10.0 to 2.11.0.
- cargo: bump openssl from 0.10.78 to 0.10.79.
- deps: bump taiki-e/install-action from 2.75.19 to 2.77.1.
- deps: bump cachix/install-nix-action from 31.10.5 to 31.10.6.
- allow passing arguments to scripts/clippy.sh.
- clippy::useless-borrows-in-formatting fixes.
- update zerocopy from 0.7.32 to 0.7.35.
- upgrade astral-tokio-tar to 0.6.2 ([#8255](https://github.com/chatmail/core/pull/8255)).
- deps: bump EmbarkStudios/cargo-deny-action from 2.0.17 to 2.0.18.
- cargo: bump openssl from 0.10.79 to 0.10.80.
- deps: bump taiki-e/install-action from 2.77.1 to 2.78.1.
## [2.49.0] - 2026-04-13
### Features / Changes
@@ -8103,3 +8295,5 @@ https://github.com/chatmail/core/pulls?q=is%3Apr+is%3Aclosed
[2.47.0]: https://github.com/chatmail/core/compare/v2.46.0..v2.47.0
[2.48.0]: https://github.com/chatmail/core/compare/v2.47.0..v2.48.0
[2.49.0]: https://github.com/chatmail/core/compare/v2.48.0..v2.49.0
[2.50.0]: https://github.com/chatmail/core/compare/v2.49.0..v2.50.0
[2.51.0]: https://github.com/chatmail/core/compare/v2.50.0..v2.51.0
Generated
+88 -24
View File
@@ -391,6 +391,28 @@ version = "1.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ace50bade8e6234aa140d9a2f552bbee1db4d353f69b8217bc503490fc1a9f26"
[[package]]
name = "aws-lc-rs"
version = "1.17.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5ec2f1fc3ec205783a5da9a7e6c1509cc69dedf09a1949e412c1e18469326d00"
dependencies = [
"aws-lc-sys",
"zeroize",
]
[[package]]
name = "aws-lc-sys"
version = "0.41.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1a2f9779ce85b93ab6170dd940ad0169b5766ff848247aff13bb788b832fe3f4"
dependencies = [
"cc",
"cmake",
"dunce",
"fs_extra",
]
[[package]]
name = "backon"
version = "1.5.0"
@@ -763,10 +785,13 @@ dependencies = [
[[package]]
name = "cc"
version = "1.2.14"
version = "1.2.63"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0c3d1b2e905a3a7b00a6141adb0e4c0bb941d11caf55349d863942a1cc44e3c9"
checksum = "556e016178bb5662a08681bbe0f00f8e17631781a4dfc8c45e466e4b185ec27f"
dependencies = [
"find-msvc-tools",
"jobserver",
"libc",
"shlex",
]
@@ -920,6 +945,15 @@ dependencies = [
"digest",
]
[[package]]
name = "cmake"
version = "0.1.58"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c0f78a02292a74a88ac736019ab962ece0bc380e3f977bf72e376c5d78ff0678"
dependencies = [
"cc",
]
[[package]]
name = "cobs"
version = "0.2.3"
@@ -1316,7 +1350,7 @@ dependencies = [
[[package]]
name = "deltachat"
version = "2.50.0-dev"
version = "2.51.0-dev"
dependencies = [
"anyhow",
"astral-tokio-tar",
@@ -1425,7 +1459,7 @@ dependencies = [
[[package]]
name = "deltachat-jsonrpc"
version = "2.50.0-dev"
version = "2.51.0-dev"
dependencies = [
"anyhow",
"async-channel 2.5.0",
@@ -1446,7 +1480,7 @@ dependencies = [
[[package]]
name = "deltachat-repl"
version = "2.50.0-dev"
version = "2.51.0-dev"
dependencies = [
"anyhow",
"deltachat",
@@ -1462,7 +1496,7 @@ dependencies = [
[[package]]
name = "deltachat-rpc-server"
version = "2.50.0-dev"
version = "2.51.0-dev"
dependencies = [
"anyhow",
"deltachat",
@@ -1491,7 +1525,7 @@ dependencies = [
[[package]]
name = "deltachat_ffi"
version = "2.50.0-dev"
version = "2.51.0-dev"
dependencies = [
"anyhow",
"deltachat",
@@ -1676,7 +1710,7 @@ dependencies = [
"libc",
"option-ext",
"redox_users",
"windows-sys 0.61.1",
"windows-sys 0.59.0",
]
[[package]]
@@ -1726,6 +1760,12 @@ dependencies = [
"zeroize",
]
[[package]]
name = "dunce"
version = "1.0.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "92773504d58c093f6de2459af4af33faa518c13451eb8f2b5698ed3d36e7c813"
[[package]]
name = "dyn-clone"
version = "1.0.18"
@@ -2051,6 +2091,12 @@ dependencies = [
"windows-sys 0.59.0",
]
[[package]]
name = "find-msvc-tools"
version = "0.1.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5baebc0774151f905a1a2cc41989300b1e6fbb29aff0ceffa1064fdd3088d582"
[[package]]
name = "fixedbitset"
version = "0.5.7"
@@ -2108,6 +2154,12 @@ dependencies = [
name = "format-flowed"
version = "1.0.0"
[[package]]
name = "fs_extra"
version = "1.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "42703706b716c37f96a77aea830392ad231f44c9e9a67872fa5548707e11b11c"
[[package]]
name = "funty"
version = "2.0.0"
@@ -2681,7 +2733,7 @@ dependencies = [
"hyper",
"libc",
"pin-project-lite",
"socket2 0.6.3",
"socket2 0.5.9",
"tokio",
"tower-service",
"tracing",
@@ -3234,6 +3286,16 @@ version = "1.0.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b1a46d1a171d865aa5f83f92695765caa047a9b4cbae2cbf37dbd613a793fd4c"
[[package]]
name = "jobserver"
version = "0.1.34"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9afb3de4395d6b3e67a780b6de64b51c978ecf11cb9a462c66be7d4ca9039d33"
dependencies = [
"getrandom 0.3.3",
"libc",
]
[[package]]
name = "js-sys"
version = "0.3.77"
@@ -3374,9 +3436,9 @@ dependencies = [
[[package]]
name = "log"
version = "0.4.29"
version = "0.4.31"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5e5032e24019045c762d3c0f28f5b6b8bbf38563a65908389bf7978758920897"
checksum = "113b30b4cd05f7c06868fdb2854f66a7b9fece9a48425351cd532e810d74024f"
[[package]]
name = "loom"
@@ -3818,7 +3880,7 @@ version = "0.50.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7957b9740744892f114936ab4a57b3f487491bbeafaf8083688b16841a4240e5"
dependencies = [
"windows-sys 0.61.1",
"windows-sys 0.59.0",
]
[[package]]
@@ -4303,18 +4365,18 @@ dependencies = [
[[package]]
name = "pin-project"
version = "1.1.11"
version = "1.1.13"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f1749c7ed4bcaf4c3d0a3efc28538844fb29bcdd7d2b67b2be7e20ba861ff517"
checksum = "2466b2336ed02bcdca6b294417127b90ec92038d1d5c4fbeac971a922e0e0924"
dependencies = [
"pin-project-internal",
]
[[package]]
name = "pin-project-internal"
version = "1.1.11"
version = "1.1.13"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d9b20ed30f105399776b9c883e68e536ef602a16ae6f596d2c473591d6ad64c6"
checksum = "c96395f0a926bc13b1c17622aaddda1ecb55d49c8f1bf9777e4d877800a43f8b"
dependencies = [
"proc-macro2",
"quote",
@@ -5219,7 +5281,7 @@ dependencies = [
"errno",
"libc",
"linux-raw-sys 0.12.1",
"windows-sys 0.61.1",
"windows-sys 0.52.0",
]
[[package]]
@@ -5228,6 +5290,7 @@ version = "0.23.37"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "758025cb5fccfd3bc2fd74708fd4682be41d99e5dff73c377c0646c6012c73a4"
dependencies = [
"aws-lc-rs",
"log",
"once_cell",
"ring",
@@ -5273,6 +5336,7 @@ version = "0.103.13"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "61c429a8649f110dddef65e2a5ad240f747e85f7758a6bccc7e5777bd33f756e"
dependencies = [
"aws-lc-rs",
"ring",
"rustls-pki-types",
"untrusted",
@@ -5517,9 +5581,9 @@ dependencies = [
[[package]]
name = "serde_json"
version = "1.0.149"
version = "1.0.150"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "83fc039473c5595ace860d8c4fafa220ff474b3fc6bfdb4293327f1a37e94d86"
checksum = "e8014e44b4736ed0538adeecded0fce2a272f22dc9578a7eb6b2d9993c74cfb9"
dependencies = [
"itoa",
"memchr",
@@ -5695,9 +5759,9 @@ dependencies = [
[[package]]
name = "shlex"
version = "1.3.0"
version = "2.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0fda2ff0d084019ba4d7c6f371c95d8fd75ce3524c3cb8fb653a3023f6323e64"
checksum = "f8fadd59c855ef2080decdef8ff161eb6661b86933c9d82e5ba29dc602a55aba"
[[package]]
name = "signal-hook-registry"
@@ -6083,7 +6147,7 @@ dependencies = [
"getrandom 0.3.3",
"once_cell",
"rustix 1.1.4",
"windows-sys 0.61.1",
"windows-sys 0.52.0",
]
[[package]]
@@ -6231,9 +6295,9 @@ checksum = "1f3ccbac311fea05f86f61904b462b55fb3df8837a366dfc601a0161d0532f20"
[[package]]
name = "tokio"
version = "1.52.1"
version = "1.52.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b67dee974fe86fd92cc45b7a95fdd2f99a36a6d7b0d431a231178d3d670bbcc6"
checksum = "8fc7f01b389ac15039e4dc9531aa973a135d7a4135281b12d7c1bc79fd57fffe"
dependencies = [
"bytes",
"libc",
+2 -2
View File
@@ -1,6 +1,6 @@
[package]
name = "deltachat"
version = "2.50.0-dev"
version = "2.51.0-dev"
edition = "2024"
license = "MPL-2.0"
rust-version = "1.89"
@@ -101,7 +101,7 @@ tagger = "4.3.4"
textwrap = "0.16.2"
thiserror = { workspace = true }
tokio-io-timeout = "1.2.1"
tokio-rustls = { version = "0.26.2", default-features = false }
tokio-rustls = { version = "0.26.2", default-features = false, features = ["aws-lc-rs", "tls12"] }
tokio-stream = { version = "0.1.17", features = ["fs"] }
astral-tokio-tar = { version = "0.6.2", default-features = false }
tokio-util = { workspace = true }
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "deltachat_ffi"
version = "2.50.0-dev"
version = "2.51.0-dev"
description = "Deltachat FFI"
edition = "2018"
readme = "README.md"
+3 -8
View File
@@ -413,11 +413,6 @@ char* dc_get_blobdir (const dc_context_t* context);
* Messages in the "saved messages" chat (see dc_chat_is_self_talk()) are skipped.
* Messages are deleted whether they were seen or not, the UI should clearly point that out.
* See also dc_estimate_deletion_cnt().
* - `delete_server_after` = 0=do not delete messages from server automatically (default),
* 1=delete messages directly after receiving from server, mvbox is skipped.
* >1=seconds, after which messages are deleted automatically from the server, mvbox is used as defined.
* "Saved messages" are deleted from the server as well as emails, the UI should clearly point that out.
* See also dc_estimate_deletion_cnt().
* - `media_quality` = DC_MEDIA_QUALITY_BALANCED (0) =
* good outgoing images/videos/voice quality at reasonable sizes (default)
* DC_MEDIA_QUALITY_WORSE (1)
@@ -1462,16 +1457,16 @@ dc_chatlist_t* dc_get_similar_chatlist (dc_context_t* context, uint32_t ch
/**
* Estimate the number of messages that will be deleted
* by the dc_set_config()-options `delete_device_after` or `delete_server_after`.
* by the dc_set_config()-option `delete_device_after`.
* This is typically used to show the estimated impact to the user
* before actually enabling deletion of old messages.
*
* @memberof dc_context_t
* @param context The context object as returned from dc_context_new().
* @param from_server 1=Estimate deletion count for server, 0=Estimate deletion count for device
* @param from_server Deprecated, pass 0 here
* @param seconds Count messages older than the given number of seconds.
* @return Number of messages that are older than the given number of seconds.
* Messages in the "saved messages" folder are not counted as they will not be deleted automatically.
* Messages in the "Saved Messages" chat are not counted as they will not be deleted automatically.
*/
int dc_estimate_deletion_cnt (dc_context_t* context, int from_server, int64_t seconds);
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "deltachat-jsonrpc"
version = "2.50.0-dev"
version = "2.51.0-dev"
description = "DeltaChat JSON-RPC API"
edition = "2021"
license = "MPL-2.0"
+11 -5
View File
@@ -735,10 +735,19 @@ impl CommandApi {
Ok(msg_ids)
}
/// Estimate the number of messages that will be deleted
/// by the set_config()-options `delete_device_after` or `delete_server_after`.
/// Estimates the number of messages that will be deleted
/// by the `set_config()`-option `delete_device_after`.
///
/// This is typically used to show the estimated impact to the user
/// before actually enabling deletion of old messages.
///
/// Messages in the "Saved Messages" chat are not counted as they will not be deleted automatically.
///
/// Parameters:
/// - `from_server`: Deprecated, pass `false` here
/// - `seconds`: Count messages older than the given number of seconds.
///
/// Returns the number of messages that are older than the given number of seconds.
async fn estimate_auto_deletion_count(
&self,
account_id: u32,
@@ -1097,9 +1106,6 @@ impl CommandApi {
/// because the word "channel" already appears a lot in the code,
/// which would make it hard to grep for it.
///
/// After creation, the chat contains no recipients and is in _unpromoted_ state;
/// see [`CommandApi::create_group_chat`] for more information on the unpromoted state.
///
/// Returns the created chat's id.
async fn create_broadcast(&self, account_id: u32, chat_name: String) -> Result<u32> {
let ctx = self.get_context(account_id).await?;
+1 -1
View File
@@ -54,5 +54,5 @@
},
"type": "module",
"types": "dist/deltachat.d.ts",
"version": "2.50.0-dev"
"version": "2.51.0-dev"
}
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "deltachat-repl"
version = "2.50.0-dev"
version = "2.51.0-dev"
license = "MPL-2.0"
edition = "2021"
repository = "https://github.com/chatmail/core"
+3 -5
View File
@@ -1236,11 +1236,9 @@ pub async fn cmdline(context: Context, line: &str, chat_id: &mut ChatId) -> Resu
"estimatedeletion" => {
ensure!(!arg1.is_empty(), "Argument <seconds> missing");
let seconds = arg1.parse()?;
let device_cnt = message::estimate_deletion_cnt(&context, false, seconds).await?;
let server_cnt = message::estimate_deletion_cnt(&context, true, seconds).await?;
println!(
"estimated count of messages older than {seconds} seconds:\non device: {device_cnt}\non server: {server_cnt}"
);
let from_server = false;
let device_cnt = message::estimate_deletion_cnt(&context, from_server, seconds).await?;
println!("estimated count of messages older than {seconds} seconds: {device_cnt}");
}
"" => (),
_ => bail!("Unknown command: \"{arg0}\" type ? for help."),
+1 -1
View File
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "deltachat-rpc-client"
version = "2.50.0-dev"
version = "2.51.0-dev"
license = "MPL-2.0"
description = "Python client for Delta Chat core JSON-RPC interface"
classifiers = [
@@ -340,9 +340,6 @@ class Account:
because the word "channel" already appears a lot in the code,
which would make it hard to grep for it.
After creation, the chat contains no recipients and is in _unpromoted_ state;
see `create_group()` for more information on the unpromoted state.
Returns the created chat.
"""
return Chat(self, self._rpc.create_broadcast(self.id, name))
+13 -11
View File
@@ -14,10 +14,13 @@ def test_moved_markseen(acfactory, direct_imap, log):
ac2.add_or_update_transport({"addr": addr, "password": password})
ac2.bring_online()
# Make sure that messages are not immediately auto-deleted on the server:
ac1.set_config("bcc_self", "1")
ac2.set_config("bcc_self", "1")
log.section("ac2: creating DeltaChat folder")
ac2_direct_imap = direct_imap(ac2)
ac2_direct_imap.create_folder("DeltaChat")
ac2.set_config("delete_server_after", "0")
ac2.set_config("sync_msgs", "0") # Do not send a sync message when accepting a contact request.
ac2.add_or_update_transport({"addr": addr, "password": password, "imapFolder": "DeltaChat"})
@@ -57,11 +60,9 @@ def test_moved_markseen(acfactory, direct_imap, log):
def test_markseen_message_and_mdn(acfactory, direct_imap):
ac1, ac2 = acfactory.get_online_accounts(2)
for ac in ac1, ac2:
ac.set_config("delete_server_after", "0")
# Do not send BCC to self, we only want to test MDN on ac1.
ac1.set_config("bcc_self", "0")
# Make sure that messages are not immediately auto-deleted on the server:
ac1.set_config("bcc_self", "1")
ac2.set_config("bcc_self", "1")
acfactory.get_accepted_chat(ac1, ac2).send_text("hi")
msg = ac2.wait_for_incoming_msg()
@@ -81,17 +82,18 @@ def test_markseen_message_and_mdn(acfactory, direct_imap):
ac1_direct_imap.select_folder("INBOX")
ac2_direct_imap.select_folder("INBOX")
# Check that the mdn is marked as seen
assert len(list(ac1_direct_imap.conn.fetch(AND(seen=True), mark_seen=False))) == 1
# Check original message is marked as seen
assert len(list(ac2_direct_imap.conn.fetch(AND(seen=True), mark_seen=False))) == 1
# Check that the mdn and original message is marked as seen
assert len(list(ac1_direct_imap.conn.fetch(AND(seen=True), mark_seen=False))) == 2
assert len(list(ac2_direct_imap.conn.fetch(AND(seen=True), mark_seen=False))) == 2
def test_trash_multiple_messages(acfactory, direct_imap, log):
ac1, ac2 = acfactory.get_online_accounts(2)
ac2.stop_io()
ac2.set_config("delete_server_after", "0")
# Make sure that messages are not immediately auto-deleted on the server:
ac2.set_config("bcc_self", "1")
ac2.set_config("sync_msgs", "0")
ac2.start_io()
+4 -14
View File
@@ -4,39 +4,29 @@ from deltachat_rpc_client import EventType
from deltachat_rpc_client.const import MessageState
def test_bcc_self_delete_server_after_defaults(acfactory):
"""Test default values for bcc_self and delete_server_after."""
def test_bcc_self_is_enabled_when_setting_up_second_device(acfactory):
ac = acfactory.get_online_account()
# Initially after getting online
# the setting bcc_self is set to 0 because there is only one device
# and delete_server_after is "1", meaning immediate deletion.
assert ac.get_config("bcc_self") == "0"
assert ac.get_config("delete_server_after") == "1"
# Setup a second device.
ac_clone = ac.clone()
ac_clone.bring_online()
# Second device setup
# enables bcc_self and changes default delete_server_after.
# Second device setup enables bcc_self.
assert ac.get_config("bcc_self") == "1"
assert ac.get_config("delete_server_after") == "0"
assert ac_clone.get_config("bcc_self") == "1"
assert ac_clone.get_config("delete_server_after") == "0"
# Manually disabling bcc_self
# also restores the default for delete_server_after.
# Test manually disabling bcc_self
ac.set_config("bcc_self", "0")
assert ac.get_config("bcc_self") == "0"
assert ac.get_config("delete_server_after") == "1"
# Cloning the account again enables bcc_self
# Cloning the account again enables bcc_self again
# even though it was manually disabled.
ac_clone = ac.clone()
assert ac.get_config("bcc_self") == "1"
assert ac.get_config("delete_server_after") == "0"
def test_one_account_send_bcc_setting(acfactory, log, direct_imap):
+5 -3
View File
@@ -1240,10 +1240,12 @@ def test_leave_and_delete_group(acfactory, log):
def test_immediate_autodelete(acfactory, direct_imap, log):
"""
`bcc_self` is off by default,
so that messages are supposed to be immediately autodeleted
"""
ac1, ac2 = acfactory.get_online_accounts(2)
# "1" means delete immediately, while "0" means do not delete
ac2.set_config("delete_server_after", "1")
assert ac1.get_config("bcc_self") == "0"
log.section("ac1: create chat with ac2")
chat1 = ac1.create_chat(ac2)
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "deltachat-rpc-server"
version = "2.50.0-dev"
version = "2.51.0-dev"
description = "DeltaChat JSON-RPC server"
edition = "2021"
readme = "README.md"
@@ -15,5 +15,5 @@
},
"type": "module",
"types": "index.d.ts",
"version": "2.50.0-dev"
"version": "2.51.0-dev"
}
Generated
+43 -108
View File
@@ -3,15 +3,19 @@
"android": {
"inputs": {
"devshell": "devshell",
"flake-utils": "flake-utils",
"nixpkgs": "nixpkgs"
"flake-utils": [
"flake-utils"
],
"nixpkgs": [
"nixpkgs"
]
},
"locked": {
"lastModified": 1731356359,
"narHash": "sha256-vYqJnu6jotmWpPT4DgzHVdvNIZcKZCIUqS8QaptsZA0=",
"lastModified": 1779918845,
"narHash": "sha256-FbpOOBg15L7X6NWWmTKbSdccnH59Jq53wWmAO37d2Q8=",
"owner": "tadfisher",
"repo": "android-nixpkgs",
"rev": "c028ead7e88edb2e94cd7c90ee37593f63ae494a",
"rev": "105c093afc8c8fbeea98f8e398403f93043eba17",
"type": "github"
},
"original": {
@@ -28,11 +32,11 @@
]
},
"locked": {
"lastModified": 1728330715,
"narHash": "sha256-xRJ2nPOXb//u1jaBnDP56M7v5ldavjbtR6lfGqSvcKg=",
"lastModified": 1768818222,
"narHash": "sha256-460jc0+CZfyaO8+w8JNtlClB2n4ui1RbHfPTLkpwhU8=",
"owner": "numtide",
"repo": "devshell",
"rev": "dd6b80932022cea34a019e2bb32f6fa9e494dfef",
"rev": "255a2b1725a20d060f566e4755dbf571bbbb5f76",
"type": "github"
},
"original": {
@@ -43,15 +47,17 @@
},
"fenix": {
"inputs": {
"nixpkgs": "nixpkgs_2",
"nixpkgs": [
"nixpkgs"
],
"rust-analyzer-src": "rust-analyzer-src"
},
"locked": {
"lastModified": 1763361733,
"narHash": "sha256-ka7dpwH3HIXCyD2wl5F7cPLeRbqZoY2ullALsvxdPt8=",
"lastModified": 1779876442,
"narHash": "sha256-O25HomVNmdROO13PEQ3Ran8Hq5EsyLmVn8Gb8JvJtJE=",
"owner": "nix-community",
"repo": "fenix",
"rev": "6c8d48e3b0ae371b19ac1485744687b788e80193",
"rev": "2eff81fc84390a35e1565395ae945d9394856824",
"type": "github"
},
"original": {
@@ -65,29 +71,11 @@
"systems": "systems"
},
"locked": {
"lastModified": 1726560853,
"narHash": "sha256-X6rJYSESBVr3hBoH0WbKE5KvhPU5bloyZ2L4K60/fPQ=",
"lastModified": 1731533236,
"narHash": "sha256-l0KFg5HjrsfsO/JpG+r7fRrqm12kzFHyUHqHCVpMMbI=",
"owner": "numtide",
"repo": "flake-utils",
"rev": "c1dfcf08411b08f6b8615f7d8971a2bfa81d5e8a",
"type": "github"
},
"original": {
"owner": "numtide",
"repo": "flake-utils",
"type": "github"
}
},
"flake-utils_2": {
"inputs": {
"systems": "systems_2"
},
"locked": {
"lastModified": 1726560853,
"narHash": "sha256-X6rJYSESBVr3hBoH0WbKE5KvhPU5bloyZ2L4K60/fPQ=",
"owner": "numtide",
"repo": "flake-utils",
"rev": "c1dfcf08411b08f6b8615f7d8971a2bfa81d5e8a",
"rev": "11707dc2f618dd54ca8739b309ec4fc024de578b",
"type": "github"
},
"original": {
@@ -98,29 +86,35 @@
},
"naersk": {
"inputs": {
"nixpkgs": "nixpkgs_3"
"fenix": [
"fenix"
],
"nixpkgs": [
"nixpkgs"
]
},
"locked": {
"lastModified": 1721727458,
"narHash": "sha256-r/xppY958gmZ4oTfLiHN0ZGuQ+RSTijDblVgVLFi1mw=",
"lastModified": 1779912356,
"narHash": "sha256-yj5O6vmAj+OfhTQMiUwhmQRP0HAII3BxEI6zuY6h/5k=",
"owner": "nix-community",
"repo": "naersk",
"rev": "3fb418eaf352498f6b6c30592e3beb63df42ef11",
"rev": "33eaf5c72a67db15073322d26cd342c443556214",
"type": "github"
},
"original": {
"owner": "nix-community",
"ref": "pull/391/head",
"repo": "naersk",
"type": "github"
}
},
"nix-filter": {
"locked": {
"lastModified": 1730207686,
"narHash": "sha256-SCHiL+1f7q9TAnxpasriP6fMarWE5H43t25F5/9e28I=",
"lastModified": 1757882181,
"narHash": "sha256-+cCxYIh2UNalTz364p+QYmWHs0P+6wDhiWR4jDIKQIU=",
"owner": "numtide",
"repo": "nix-filter",
"rev": "776e68c1d014c3adde193a18db9d738458cd2ba4",
"rev": "59c44d1909c72441144b93cf0f054be7fe764de5",
"type": "github"
},
"original": {
@@ -131,60 +125,16 @@
},
"nixpkgs": {
"locked": {
"lastModified": 1731139594,
"narHash": "sha256-IigrKK3vYRpUu+HEjPL/phrfh7Ox881er1UEsZvw9Q4=",
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "76612b17c0ce71689921ca12d9ffdc9c23ce40b2",
"type": "github"
},
"original": {
"owner": "NixOS",
"ref": "nixos-unstable",
"repo": "nixpkgs",
"type": "github"
}
},
"nixpkgs_2": {
"locked": {
"lastModified": 1762977756,
"narHash": "sha256-4PqRErxfe+2toFJFgcRKZ0UI9NSIOJa+7RXVtBhy4KE=",
"lastModified": 1779931091,
"narHash": "sha256-gc8NEz7a++7OQPGvMv+zIjXCec1PO38XRXZRa3m97ew=",
"owner": "nixos",
"repo": "nixpkgs",
"rev": "c5ae371f1a6a7fd27823bc500d9390b38c05fa55",
"rev": "3052ddf0614791c1869384a868248be5607a309f",
"type": "github"
},
"original": {
"owner": "nixos",
"ref": "nixos-unstable",
"repo": "nixpkgs",
"type": "github"
}
},
"nixpkgs_3": {
"locked": {
"lastModified": 0,
"narHash": "sha256-Dqg6si5CqIzm87sp57j5nTaeBbWhHFaVyG7V6L8k3lY=",
"path": "/nix/store/zq2axpgzd5kykk1v446rkffj3bxa2m2h-source",
"type": "path"
},
"original": {
"id": "nixpkgs",
"type": "indirect"
}
},
"nixpkgs_4": {
"locked": {
"lastModified": 1747179050,
"narHash": "sha256-qhFMmDkeJX9KJwr5H32f1r7Prs7XbQWtO0h3V0a0rFY=",
"owner": "nixos",
"repo": "nixpkgs",
"rev": "adaa24fbf46737f3f1b5497bf64bae750f82942e",
"type": "github"
},
"original": {
"owner": "nixos",
"ref": "nixos-unstable",
"ref": "master",
"repo": "nixpkgs",
"type": "github"
}
@@ -193,20 +143,20 @@
"inputs": {
"android": "android",
"fenix": "fenix",
"flake-utils": "flake-utils_2",
"flake-utils": "flake-utils",
"naersk": "naersk",
"nix-filter": "nix-filter",
"nixpkgs": "nixpkgs_4"
"nixpkgs": "nixpkgs"
}
},
"rust-analyzer-src": {
"flake": false,
"locked": {
"lastModified": 1762860488,
"narHash": "sha256-rMfWMCOo/pPefM2We0iMBLi2kLBAnYoB9thi4qS7uk4=",
"lastModified": 1779827300,
"narHash": "sha256-J6pHxKoZzWCrAvOVInwBcYYWix/NWwM10Ad+i29Qc5s=",
"owner": "rust-lang",
"repo": "rust-analyzer",
"rev": "2efc80078029894eec0699f62ec8d5c1a56af763",
"rev": "c3af07ad84d68adc5e652e86f0c20009caa29014",
"type": "github"
},
"original": {
@@ -230,21 +180,6 @@
"repo": "default",
"type": "github"
}
},
"systems_2": {
"locked": {
"lastModified": 1681028828,
"narHash": "sha256-Vy1rq5AaRuLzOxct8nz4T6wlgyUR7zLU309k9mBC768=",
"owner": "nix-systems",
"repo": "default",
"rev": "da67096a3b9bf56a91d16901293e51ba5b49a27e",
"type": "github"
},
"original": {
"owner": "nix-systems",
"repo": "default",
"type": "github"
}
}
},
"root": "root",
+18 -20
View File
@@ -2,11 +2,16 @@
description = "Chatmail core";
inputs = {
fenix.url = "github:nix-community/fenix";
fenix.inputs.nixpkgs.follows = "nixpkgs";
flake-utils.url = "github:numtide/flake-utils";
naersk.url = "github:nix-community/naersk";
naersk.url = "github:nix-community/naersk/pull/391/head";
naersk.inputs.nixpkgs.follows = "nixpkgs";
naersk.inputs.fenix.follows = "fenix";
nix-filter.url = "github:numtide/nix-filter";
nixpkgs.url = "github:nixos/nixpkgs/nixos-unstable";
nixpkgs.url = "github:nixos/nixpkgs/master";
android.url = "github:tadfisher/android-nixpkgs";
android.inputs.nixpkgs.follows = "nixpkgs";
android.inputs.flake-utils.follows = "flake-utils";
};
outputs = { self, nixpkgs, flake-utils, nix-filter, naersk, fenix, android }:
flake-utils.lib.eachDefaultSystem (system:
@@ -133,6 +138,8 @@
];
depsBuildBuild = [
pkgsWin64.stdenv.cc
];
buildInputs = [
pkgsWin64.windows.pthreads
];
auditable = false; # Avoid cargo-auditable failures.
@@ -143,6 +150,8 @@
CARGO_BUILD_RUSTFLAGS = [
"-C"
"linker=${TARGET_CC}"
"-L"
"native=${pkgsWin64.windows.pthreads}/lib"
];
CC = "${pkgsWin64.stdenv.cc}/bin/${pkgsWin64.stdenv.cc.targetPrefix}cc";
@@ -180,7 +189,8 @@
};
})).overrideAttrs (oldAttr: {
configureFlags = oldAttr.configureFlags ++ [
"--disable-sjlj-exceptions --with-dwarf2"
"--disable-sjlj-exceptions"
"--with-dwarf2"
];
})
);
@@ -196,6 +206,8 @@
];
depsBuildBuild = [
winCC
];
buildInputs = [
pkgsWin32.windows.pthreads
];
auditable = false; # Avoid cargo-auditable failures.
@@ -206,6 +218,8 @@
CARGO_BUILD_RUSTFLAGS = [
"-C"
"linker=${TARGET_CC}"
"-L"
"native=${pkgsWin32.windows.pthreads}/lib"
];
CC = "${winCC}/bin/${winCC.targetPrefix}cc";
@@ -504,22 +518,6 @@
'';
};
# Source package for deltachat-rpc-server.
# Fake package that downloads Linux version,
# needed to install deltachat-rpc-server on Android with `pip`.
deltachat-rpc-server-source =
pkgs.stdenv.mkDerivation {
pname = "deltachat-rpc-server-source";
version = manifest.version;
src = pkgs.lib.cleanSource ./.;
nativeBuildInputs = [
pkgs.python3
pkgs.python3Packages.wheel
];
buildPhase = ''python3 scripts/wheel-rpc-server.py source deltachat_rpc_server-${manifest.version}.tar.gz'';
installPhase = ''mkdir -p $out; cp -av deltachat_rpc_server-${manifest.version}.tar.gz $out'';
};
deltachat-rpc-client =
pkgs.python3Packages.buildPythonPackage {
pname = "deltachat-rpc-client";
@@ -562,7 +560,7 @@
deltachat-python
deltachat-rpc-client
pkgs.python3Packages.breathe
pkgs.python3Packages.sphinx_rtd_theme
pkgs.python3Packages.sphinx-rtd-theme
];
nativeBuildInputs = [ pkgs.sphinx ];
buildPhase = ''sphinx-build -b html -a python/doc/ dist/html'';
+1 -1
View File
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "deltachat"
version = "2.50.0-dev"
version = "2.51.0-dev"
license = "MPL-2.0"
description = "Python bindings for the Delta Chat Core library using CFFI against the Rust-implemented libdeltachat"
readme = "README.rst"
-1
View File
@@ -521,7 +521,6 @@ class ACFactory:
assert "addr" in configdict and "mail_pw" in configdict, configdict
configdict.setdefault("bcc_self", False)
configdict.setdefault("sync_msgs", False)
configdict.setdefault("delete_server_after", 0)
ac.update_config(configdict)
self._acsetup._account2config[ac] = configdict
self._preconfigure_key(ac)
-67
View File
@@ -298,73 +298,6 @@ def test_use_new_verified_group_after_going_online(acfactory, data, tmp_path, lp
assert msg_in.text == msg_out.text
def test_verified_group_vs_delete_server_after(acfactory, tmp_path, lp):
"""Test for the issue #4346:
- User is added to a verified group.
- First device of the user downloads "member added" from the group.
- First device removes "member added" from the server.
- Some new messages are sent to the group.
- Second device comes online, receives these new messages.
The result is an unverified group with unverified members.
- First device re-gossips Autocrypt keys to the group.
- Now the second device has all members and group verified.
"""
ac1, ac2 = acfactory.get_online_accounts(2)
acfactory.remove_preconfigured_keys()
ac2_offl = acfactory.new_online_configuring_account(cloned_from=ac2)
for ac in [ac2, ac2_offl]:
ac.set_config("bcc_self", "1")
ac2.set_config("delete_server_after", "1")
ac2.set_config("gossip_period", "0") # Re-gossip in every message
acfactory.bring_accounts_online()
dir = tmp_path / "exportdir"
dir.mkdir()
ac2.export_self_keys(str(dir))
ac2_offl.import_self_keys(str(dir))
ac2_offl.stop_io()
lp.sec("ac1: create verified-group QR, ac2 scans and joins")
chat1 = ac1.create_group_chat("hello")
qr = chat1.get_join_qr()
lp.sec("ac2: start QR-code based join-group protocol")
chat2 = ac2.qr_join_chat(qr)
ac1._evtracker.wait_securejoin_inviter_progress(1000)
# Wait for "Member Me (<addr>) added by <addr>." message.
msg_in = ac2._evtracker.wait_next_incoming_message()
assert msg_in.is_system_message()
lp.sec("ac2: waiting for 'member added' to be deleted on the server")
ac2._evtracker.get_matching("DC_EVENT_IMAP_MESSAGE_DELETED")
lp.sec("ac1: sending 'hi' to the group")
ac2.set_config("delete_server_after", "0")
chat1.send_text("hi")
lp.sec("ac2_offl: going online, checking the 'hi' message")
ac2_offl.start_io()
msg_in = ac2_offl._evtracker.wait_next_incoming_message()
assert not msg_in.is_system_message()
assert msg_in.text == "hi"
ac2_offl_ac1_contact = msg_in.get_sender_contact()
assert ac2_offl_ac1_contact.addr == ac1.get_config("addr")
assert not ac2_offl_ac1_contact.is_verified()
chat2_offl = msg_in.chat
lp.sec("ac2: sending message re-gossiping Autocrypt keys")
chat2.send_text("hi2")
lp.sec("ac2_offl: receiving message")
ev = ac2_offl._evtracker.get_matching("DC_EVENT_INCOMING_MSG|DC_EVENT_MSGS_CHANGED")
msg_in = ac2_offl.get_message_by_id(ev.data2)
assert not msg_in.is_system_message()
assert msg_in.text == "hi2"
assert msg_in.chat == chat2_offl
assert msg_in.get_sender_contact().addr == ac2.get_config("addr")
# Until we reset verifications and then send the _verified header,
# verification is not gossiped here:
assert not ac2_offl_ac1_contact.is_verified()
def test_deleted_msgs_dont_reappear(acfactory):
ac1 = acfactory.new_online_configuring_account()
acfactory.bring_accounts_online()
+19 -1
View File
@@ -15,6 +15,9 @@ def test_basic_imap_api(acfactory, tmp_path):
ac1, ac2 = acfactory.get_online_accounts(2)
chat12 = acfactory.get_accepted_chat(ac1, ac2)
# Make sure that messages are not immediately auto-deleted on the server:
ac2.set_config("bcc_self", "1")
imap2 = ac2.direct_imap
with imap2.idle() as idle2:
@@ -162,6 +165,9 @@ def test_webxdc_message(acfactory, data, lp):
ac1, ac2 = acfactory.get_online_accounts(2)
chat = acfactory.get_accepted_chat(ac1, ac2)
# Make sure that messages are not immediately auto-deleted on the server:
ac2.set_config("bcc_self", "1")
lp.sec("ac1: prepare and send text message to ac2")
msg1 = chat.send_text("message0")
assert not msg1.is_webxdc()
@@ -362,6 +368,10 @@ def test_send_and_receive_message_markseen(acfactory, lp):
# make DC's life harder wrt to encodings
ac1.set_config("displayname", "ä name")
# Make sure that messages are not immediately auto-deleted on the server:
ac1.set_config("bcc_self", "1")
ac2.set_config("bcc_self", "1")
# clear any fresh device messages
ac1.get_device_chat().mark_noticed()
ac2.get_device_chat().mark_noticed()
@@ -506,9 +516,15 @@ def test_mdn_asymmetric(acfactory, lp):
ac1.set_config("mdns_enabled", "1")
ac2.set_config("mdns_enabled", "1")
# Make sure that the mdn is not immediately auto-deleted on the server:
ac1.set_config("bcc_self", "1")
lp.sec("sending text message from ac1 to ac2")
msg_out = chat.send_text("message1")
# Wait for the message to be marked as seen on IMAP.
ac1._evtracker.get_info_contains("Marked messages [0-9]+ in folder INBOX as seen.")
assert len(chat.get_messages()) == 1 + E2EE_INFO_MSGS
lp.sec("disable ac1 MDNs")
@@ -525,7 +541,7 @@ def test_mdn_asymmetric(acfactory, lp):
lp.sec("ac1: waiting for incoming activity")
assert len(chat.get_messages()) == 1 + E2EE_INFO_MSGS
# Wait for the message to be marked as seen on IMAP.
# Wait for the mdn to be marked as seen on IMAP.
ac1._evtracker.get_info_contains("Marked messages [0-9]+ in folder INBOX as seen.")
# MDN is received even though MDNs are already disabled
@@ -1073,6 +1089,8 @@ def test_send_receive_locations(acfactory, lp):
def test_delete_multiple_messages(acfactory, lp):
ac1, ac2 = acfactory.get_online_accounts(2)
# Make sure that messages are not immediately auto-deleted on the server:
ac2.set_config("bcc_self", "1")
chat12 = acfactory.get_accepted_chat(ac1, ac2)
lp.sec("ac1: sending seven messages")
+1 -1
View File
@@ -1 +1 @@
2026-04-13
2026-05-29
+1 -1
View File
@@ -6,7 +6,7 @@ set -euo pipefail
export TZ=UTC
# Provider database revision.
REV=ad097ee40579c884e7757de2d3bb0a51f481a32a
REV=2cba4b72f4c6e6417b83ba549aff7781be5f166c
CORE_ROOT="$PWD"
TMP="$(mktemp -d)"
+12 -96
View File
@@ -20,86 +20,6 @@ Description-Content-Type: text/markdown
"""
def build_source_package(version, filename):
with tarfile.open(filename, "w:gz") as pkg:
def pack(name, contents):
contents = contents.encode()
tar_info = tarfile.TarInfo(f"deltachat_rpc_server-{version}/{name}")
tar_info.mode = 0o644
tar_info.size = len(contents)
pkg.addfile(tar_info, BytesIO(contents))
pack("PKG-INFO", metadata_contents(version))
pack(
"pyproject.toml",
f"""[build-system]
requires = ["setuptools==68.2.2", "pip"]
build-backend = "setuptools.build_meta"
[project]
name = "deltachat-rpc-server"
version = "{version}"
[project.scripts]
deltachat-rpc-server = "deltachat_rpc_server:main"
""",
)
pack(
"setup.py",
f"""
import sys
from setuptools import setup, find_packages
from distutils.cmd import Command
from setuptools.command.install import install
from setuptools.command.build import build
import subprocess
import platform
import tempfile
from zipfile import ZipFile
from pathlib import Path
import shutil
class BuildCommand(build):
def run(self):
tmpdir = tempfile.mkdtemp()
subprocess.run(
[
sys.executable,
"-m",
"pip",
"download",
"--no-input",
"--timeout",
"1000",
"--platform",
"musllinux_1_1_" + platform.machine(),
"--only-binary=:all:",
"deltachat-rpc-server=={version}",
],
cwd=tmpdir,
)
wheel_path = next(Path(tmpdir).glob("*.whl"))
with ZipFile(wheel_path, "r") as wheel:
exe_path = wheel.extract("deltachat_rpc_server/deltachat-rpc-server", "src")
Path(exe_path).chmod(0o700)
wheel.extract("deltachat_rpc_server/__init__.py", "src")
shutil.rmtree(tmpdir)
return super().run()
setup(
cmdclass={{"build": BuildCommand}},
package_data={{"deltachat_rpc_server": ["deltachat-rpc-server"]}},
)
""",
)
pack("src/deltachat_rpc_server/__init__.py", "")
def build_wheel(version, binary, tag, windows=False):
filename = f"deltachat_rpc_server-{version}-{tag}.whl"
@@ -168,23 +88,19 @@ def main():
with Path("Cargo.toml").open("rb") as fp:
cargo_manifest = tomllib.load(fp)
version = cargo_manifest["package"]["version"]
if sys.argv[1] == "source":
filename = f"deltachat_rpc_server-{version}.tar.gz"
build_source_package(version, filename)
else:
arch = sys.argv[1]
executable = sys.argv[2]
tags = arch2tags[arch]
arch = sys.argv[1]
executable = sys.argv[2]
tags = arch2tags[arch]
if arch in ["win32", "win64"]:
build_wheel(
version,
executable,
f"py3-none-{tags}",
windows=True,
)
else:
build_wheel(version, executable, f"py3-none-{tags}")
if arch in ["win32", "win64"]:
build_wheel(
version,
executable,
f"py3-none-{tags}",
windows=True,
)
else:
build_wheel(version, executable, f"py3-none-{tags}")
main()
+53 -34
View File
@@ -32,6 +32,7 @@ use crate::debug_logging::maybe_set_logging_xdc;
use crate::download::{
DownloadState, PRE_MSG_ATTACHMENT_SIZE_THRESHOLD, PRE_MSG_SIZE_WARNING_THRESHOLD,
};
use crate::ensure_and_debug_assert_eq;
use crate::ephemeral::{Timer as EphemeralTimer, start_chat_ephemeral_timers};
use crate::events::EventType;
use crate::key::{Fingerprint, self_fingerprint};
@@ -1782,9 +1783,8 @@ impl Chat {
);
bail!("Cannot set message, contact for {} not found.", self.id);
}
} else if matches!(self.typ, Chattype::Group | Chattype::OutBroadcast)
&& self.param.get_int(Param::Unpromoted).unwrap_or_default() == 1
{
} else if self.param.get_int(Param::Unpromoted).unwrap_or_default() == 1 {
ensure_and_debug_assert_eq!(self.typ, Chattype::Group,);
msg.param.set_int(Param::AttachChatAvatarAndDescription, 1);
self.param
.remove(Param::Unpromoted)
@@ -3626,9 +3626,6 @@ pub(crate) async fn create_group_ex(
/// because the word "channel" already appears a lot in the code,
/// which would make it hard to grep for it.
///
/// After creation, the chat contains no recipients and is in _unpromoted_ state;
/// see [`create_group`] for more information on the unpromoted state.
///
/// Returns the created chat's id.
pub async fn create_broadcast(context: &Context, chat_name: String) -> Result<ChatId> {
let grpid = create_id();
@@ -3660,17 +3657,20 @@ pub(crate) async fn create_out_broadcast_ex(
|row| row.get(0),
)?;
ensure!(cnt == 0, "{cnt} chats exist with grpid {grpid}");
let mut params: Params = Params::new();
params.update_timestamp(Param::GroupNameTimestamp, time())?;
t.execute(
"INSERT INTO chats
(type, name, name_normalized, grpid, created_timestamp)
VALUES(?, ?, ?, ?, ?)",
(type, name, name_normalized, grpid, created_timestamp, param)
VALUES(?, ?, ?, ?, ?, ?)",
(
Chattype::OutBroadcast,
&chat_name,
normalize_text(&chat_name),
&grpid,
timestamp,
params.to_string(),
),
)?;
let chat_id = ChatId::new(t.last_insert_rowid().try_into()?);
@@ -3738,17 +3738,19 @@ pub(crate) async fn update_chat_contacts_table(
id: ChatId,
contacts: &BTreeSet<ContactId>,
) -> Result<()> {
// See add_to_chat_contacts_table() for reasoning.
let limit = cmp::max(time().saturating_add(TIMESTAMP_SENT_TOLERANCE), timestamp);
context
.sql
.transaction(move |transaction| {
// Bump `remove_timestamp` to at least `now`
// even for members from `contacts`.
// Bump `remove_timestamp` even for members from `contacts`.
// We add members from `contacts` back below.
transaction.execute(
"UPDATE chats_contacts
SET remove_timestamp=MAX(add_timestamp+1, ?)
"UPDATE chats_contacts SET
add_timestamp=MIN(add_timestamp, ?1),
remove_timestamp=MAX(MIN(remove_timestamp,?1), MIN(add_timestamp,?1)+1, ?)
WHERE chat_id=?",
(timestamp, id),
(limit, timestamp, id),
)?;
if !contacts.is_empty() {
@@ -3760,9 +3762,8 @@ pub(crate) async fn update_chat_contacts_table(
)?;
for contact_id in contacts {
// We bumped `add_timestamp` for existing rows above,
// so on conflict it is enough to set `add_timestamp = remove_timestamp`
// and this guarantees that `add_timestamp` is no less than `timestamp`.
// We bumped `remove_timestamp` for existing rows above,
// so on conflict it is enough to set `add_timestamp = remove_timestamp`.
statement.execute((id, contact_id, timestamp))?;
}
}
@@ -3779,17 +3780,24 @@ pub(crate) async fn add_to_chat_contacts_table(
chat_id: ChatId,
contact_ids: &[ContactId],
) -> Result<()> {
// Our clock may be slow, so limit stored timestamps with `timestamp` if it's bigger. This way
// we only cap remote timestamps if, in addition, remote changes arrive reordered or we do local
// changes. Also allow some tolerance, moreover, previous removals might lend time from the
// future.
let limit = cmp::max(time().saturating_add(TIMESTAMP_SENT_TOLERANCE), timestamp);
context
.sql
.transaction(move |transaction| {
let mut add_statement = transaction.prepare(
"INSERT INTO chats_contacts (chat_id, contact_id, add_timestamp) VALUES(?1, ?2, ?3)
ON CONFLICT (chat_id, contact_id)
DO UPDATE SET add_timestamp=MAX(remove_timestamp, ?3)",
DO UPDATE SET
remove_timestamp=MIN(remove_timestamp, ?4),
add_timestamp=MIN(MAX(add_timestamp,remove_timestamp,?3), ?4)",
)?;
for contact_id in contact_ids {
add_statement.execute((chat_id, contact_id, timestamp))?;
add_statement.execute((chat_id, contact_id, timestamp, limit))?;
}
Ok(())
})
@@ -3800,26 +3808,34 @@ pub(crate) async fn add_to_chat_contacts_table(
/// Removes a contact from the chat
/// by updating the `remove_timestamp`.
/// Returns whether the contact has been a chat member recently. If so, a removal message should be
/// sent.
pub(crate) async fn remove_from_chat_contacts_table(
context: &Context,
chat_id: ChatId,
contact_id: ContactId,
) -> Result<()> {
) -> Result<bool> {
let now = time();
context
// See add_to_chat_contacts_table() for reasoning.
let limit = now.saturating_add(TIMESTAMP_SENT_TOLERANCE);
let is_past_member = context
.sql
.execute(
"UPDATE chats_contacts
SET remove_timestamp=MAX(add_timestamp+1, ?)
"UPDATE chats_contacts SET
add_timestamp=MIN(add_timestamp, ?1),
remove_timestamp=MAX(MIN(remove_timestamp,?1), MIN(add_timestamp,?1)+1, ?)
WHERE chat_id=? AND contact_id=?",
(now, chat_id, contact_id),
(limit, now, chat_id, contact_id),
)
.await?;
Ok(())
.await?
> 0;
Ok(is_past_member)
}
/// Removes a contact from the chat
/// without leaving a trace.
/// without leaving a trace in the db.
/// Returns whether the contact was removed, even if it was a past contact. If so, a removal message
/// should be sent if the removal is issued by this device.
///
/// Note that if we call this function,
/// and then receive a message from another device
@@ -3829,17 +3845,17 @@ pub(crate) async fn remove_from_chat_contacts_table_without_trace(
context: &Context,
chat_id: ChatId,
contact_id: ContactId,
) -> Result<()> {
context
) -> Result<bool> {
let removed = context
.sql
.execute(
"DELETE FROM chats_contacts
WHERE chat_id=? AND contact_id=?",
(chat_id, contact_id),
)
.await?;
Ok(())
.await?
> 0;
Ok(removed)
}
/// Adds a contact to the chat.
@@ -4159,10 +4175,13 @@ pub async fn remove_contact_from_chat(
let mut sync = Nosync;
if chat.is_promoted() && chat.typ != Chattype::OutBroadcast {
remove_from_chat_contacts_table(context, chat_id, contact_id).await?;
let removed = if chat.is_promoted() && chat.typ != Chattype::OutBroadcast {
remove_from_chat_contacts_table(context, chat_id, contact_id).await?
} else {
remove_from_chat_contacts_table_without_trace(context, chat_id, contact_id).await?;
remove_from_chat_contacts_table_without_trace(context, chat_id, contact_id).await?
};
if !removed {
return Ok(());
}
// We do not return an error if the contact does not exist in the database.
+67 -3
View File
@@ -9,6 +9,7 @@ use crate::headerdef::HeaderDef;
use crate::imex::{ImexMode, has_backup, imex};
use crate::message::{Message, MessengerMessage, delete_msgs};
use crate::mimeparser::{self, MimeMessage};
use crate::qr::{Qr, check_qr};
use crate::receive_imf::receive_imf;
use crate::securejoin::{get_securejoin_qr, join_securejoin};
use crate::test_utils;
@@ -2799,6 +2800,30 @@ async fn test_can_send_group() -> Result<()> {
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_cant_remove_nonmember() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
let bob = &tcm.bob().await;
let charlie = &tcm.charlie().await;
let alice_broadcast_id = create_broadcast(alice, "Channel".to_string()).await?;
let qr = get_securejoin_qr(alice, Some(alice_broadcast_id))
.await
.unwrap();
tcm.exec_securejoin_qr(bob, alice, &qr).await;
let alice_charlie_id = alice.add_or_lookup_contact_id(charlie).await;
remove_contact_from_chat(alice, alice_broadcast_id, alice_charlie_id).await?;
assert!(alice.pop_sent_msg_opt(Duration::ZERO).await.is_none());
assert!(!remove_from_chat_contacts_table(alice, alice_broadcast_id, alice_charlie_id).await?);
assert!(
!remove_from_chat_contacts_table_without_trace(alice, alice_broadcast_id, alice_charlie_id)
.await?
);
Ok(())
}
/// Tests that in a broadcast channel,
/// the recipients can't see the identity of their fellow recipients.
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
@@ -2922,10 +2947,24 @@ async fn test_broadcast_change_name() -> Result<()> {
let fiona = &tcm.fiona().await;
let broadcast_id = create_broadcast(alice, "Channel".to_string()).await?;
let qr = get_securejoin_qr(alice, Some(broadcast_id)).await.unwrap();
let mut qr = get_securejoin_qr(alice, Some(broadcast_id)).await.unwrap();
// Something goes wrong with the title, e.g. maybe it gets ellipsized
// Note that the title always comes at the end for human readability
qr += "+modified+title";
{
tcm.section("Alice invites Bob to her channel");
let Qr::AskJoinBroadcast { name, .. } = check_qr(bob, &qr).await? else {
panic!();
};
assert_eq!(name, "Channel modified title");
// The channel's name gets fixed after actually joining the channel:
let bob_chat_id = tcm.exec_securejoin_qr(bob, alice, &qr).await;
let bob_chat = Chat::load_from_db(bob, bob_chat_id).await?;
assert_eq!(bob_chat.name, "Channel");
}
tcm.section("Alice invites Bob to her channel");
tcm.exec_securejoin_qr(bob, alice, &qr).await;
tcm.section("Alice invites Fiona to her channel");
tcm.exec_securejoin_qr(fiona, alice, &qr).await;
@@ -3049,6 +3088,31 @@ async fn test_broadcast_resend_to_new_member() -> Result<()> {
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_broadcast_resend_failed_msg_to_new_member() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
let bob = &tcm.bob().await;
let fiona = &tcm.fiona().await;
let alice_bc_id = create_broadcast(alice, "bc".to_string()).await?;
let qr = get_securejoin_qr(alice, Some(alice_bc_id)).await.unwrap();
tcm.exec_securejoin_qr(bob, alice, &qr).await;
let alice_msg_id = alice.send_text(alice_bc_id, "text").await.sender_msg_id;
let mut msg = Message::load_from_db(alice, alice_msg_id).await?;
message::set_msg_failed(alice, &mut msg, "error").await?;
let fiona_bc_id = tcm.exec_securejoin_qr(fiona, alice, &qr).await;
let resent_msg = alice.pop_sent_msg().await;
let fiona_msg = fiona.recv_msg(&resent_msg).await;
assert_eq!(fiona_msg.chat_id, fiona_bc_id);
assert_eq!(fiona_msg.text, "text");
assert_eq!(
alice_msg_id.get_state(alice).await?,
MessageState::OutFailed
);
Ok(())
}
/// - Alice has multiple devices
/// - Alice creates a broadcast and sends a message into it
/// - Alice's second device sees the broadcast
-41
View File
@@ -194,17 +194,6 @@ pub enum Config {
#[strum(props(default = "0"))] // also change MediaQuality.default() on changes
MediaQuality,
/// Timer in seconds after which the message is deleted from the
/// server.
///
/// 0 means messages are never deleted by Delta Chat.
///
/// Value 1 is treated as "delete at once": messages are deleted
/// immediately, without moving to DeltaChat folder.
///
/// Default is 1 for chatmail accounts without `BccSelf`, 0 otherwise.
DeleteServerAfter,
/// Timer in seconds after which the message is deleted from the
/// device.
///
@@ -463,11 +452,6 @@ pub enum Config {
/// storing the same token multiple times on the server.
EncryptedDeviceToken,
/// Enables running test hooks, e.g. see `InnerContext::pre_encrypt_mime_hook`.
/// This way is better than conditional compilation, i.e. `#[cfg(test)]`, because tests not
/// using this still run unmodified code.
TestHooks,
/// Return an error from `receive_imf_inner()`. For tests.
SimulateReceiveImfError,
@@ -565,14 +549,6 @@ impl Context {
// Default values
let val = match key {
Config::ConfiguredInboxFolder => Some("INBOX".to_string()),
Config::DeleteServerAfter => {
match !Box::pin(self.get_config_bool(Config::BccSelf)).await?
&& Box::pin(self.is_chatmail()).await?
{
true => Some("1".to_string()),
false => Some("0".to_string()),
}
}
Config::Addr => self.get_config_opt(Config::ConfiguredAddr).await?,
_ => key.get_str("default").map(|s| s.to_string()),
};
@@ -653,23 +629,6 @@ impl Context {
self.get_config_bool(Config::MdnsEnabled).await
}
/// Gets configured "delete_server_after" value.
///
/// `None` means never delete the message, `Some(0)` means delete
/// at once, `Some(x)` means delete after `x` seconds.
pub async fn get_config_delete_server_after(&self) -> Result<Option<i64>> {
let val = match self
.get_config_parsed::<i64>(Config::DeleteServerAfter)
.await?
.unwrap_or(0)
{
0 => None,
1 => Some(0),
x => Some(x),
};
Ok(val)
}
/// Gets the configured provider.
///
/// The provider is determined by the current primary transport.
-22
View File
@@ -142,28 +142,6 @@ async fn test_mdns_default_behaviour() -> Result<()> {
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_delete_server_after_default() -> Result<()> {
let t = &TestContext::new_alice().await;
// Check that the settings are displayed correctly.
assert_eq!(t.get_config(Config::BccSelf).await?, Some("1".to_string()));
assert_eq!(
t.get_config(Config::DeleteServerAfter).await?,
Some("0".to_string())
);
// Leaving emails on the server even w/o `BccSelf` is a good default at least because other
// MUAs do so even if the server doesn't save sent messages to some sentbox (like Gmail
// does).
t.set_config_bool(Config::BccSelf, false).await?;
assert_eq!(
t.get_config(Config::DeleteServerAfter).await?,
Some("0".to_string())
);
Ok(())
}
const SAVED_MESSAGES_DEDUPLICATED_FILE: &str = "969142cb84015bc135767bc2370934a.png";
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
+7
View File
@@ -680,6 +680,8 @@ async fn get_autoconfig(
param: &EnteredLoginParam,
param_domain: &str,
) -> Option<Vec<ServerParams>> {
let accept_invalid_certificates = param.certificate_checks.accept_invalid_certificates();
// Make sure to not encode `.` as `%2E` here.
// Some servers like murena.io on 2024-11-01 produce incorrect autoconfig XML
// when address is encoded.
@@ -696,6 +698,7 @@ async fn get_autoconfig(
"https://autoconfig.{param_domain}/mail/config-v1.1.xml?emailaddress={param_addr_urlencoded}"
),
&param.addr,
accept_invalid_certificates,
)
.await
{
@@ -710,6 +713,7 @@ async fn get_autoconfig(
"https://{param_domain}/.well-known/autoconfig/mail/config-v1.1.xml?emailaddress={param_addr_urlencoded}"
),
&param.addr,
accept_invalid_certificates,
)
.await
{
@@ -721,6 +725,7 @@ async fn get_autoconfig(
if let Ok(res) = outlk_autodiscover(
ctx,
format!("https://{param_domain}/autodiscover/autodiscover.xml"),
accept_invalid_certificates,
)
.await
{
@@ -731,6 +736,7 @@ async fn get_autoconfig(
if let Ok(res) = outlk_autodiscover(
ctx,
format!("https://autodiscover.{param_domain}/autodiscover/autodiscover.xml",),
accept_invalid_certificates,
)
.await
{
@@ -743,6 +749,7 @@ async fn get_autoconfig(
ctx,
&format!("https://autoconfig.thunderbird.net/v1.1/{param_domain}"),
&param.addr,
accept_invalid_certificates,
)
.await
{
+3 -2
View File
@@ -10,7 +10,7 @@ use quick_xml::events::{BytesStart, Event};
use super::{Error, ServerParams};
use crate::context::Context;
use crate::log::warn;
use crate::net::read_url;
use crate::net::read_url_with_tls;
use crate::provider::{Protocol, Socket};
#[derive(Debug)]
@@ -249,8 +249,9 @@ pub(crate) async fn moz_autoconfigure(
context: &Context,
url: &str,
addr: &str,
accept_invalid_certificates: bool,
) -> Result<Vec<ServerParams>, Error> {
let xml_raw = read_url(context, url).await?;
let xml_raw = read_url_with_tls(context, url, !accept_invalid_certificates).await?;
let res = parse_serverparams(addr, &xml_raw);
if let Err(err) = &res {
+3 -2
View File
@@ -10,7 +10,7 @@ use quick_xml::events::Event;
use super::{Error, ServerParams};
use crate::context::Context;
use crate::log::warn;
use crate::net::read_url;
use crate::net::read_url_with_tls;
use crate::provider::{Protocol, Socket};
/// Result of parsing a single `Protocol` tag.
@@ -196,10 +196,11 @@ fn protocols_to_serverparams(protocols: Vec<ProtocolTag>) -> Vec<ServerParams> {
pub(crate) async fn outlk_autodiscover(
context: &Context,
mut url: String,
accept_invalid_certificates: bool,
) -> Result<Vec<ServerParams>, Error> {
/* Follow up to 10 xml-redirects (http-redirects are followed in read_url() */
for _i in 0..10 {
let xml_raw = read_url(context, &url).await?;
let xml_raw = read_url_with_tls(context, &url, !accept_invalid_certificates).await?;
let res = parse_xml(&xml_raw);
if let Err(err) = &res {
warn!(context, "{}", err);
-18
View File
@@ -332,17 +332,6 @@ pub struct InnerContext {
/// `Connectivity` values for mailboxes, unordered. Used to compute the aggregate connectivity,
/// see [`Context::get_connectivity()`].
pub(crate) connectivities: parking_lot::Mutex<Vec<ConnectivityStore>>,
#[expect(clippy::type_complexity)]
/// Transforms the root of the cryptographic payload before encryption.
pub(crate) pre_encrypt_mime_hook: parking_lot::Mutex<
Option<
for<'a> fn(
&Context,
mail_builder::mime::MimePart<'a>,
) -> mail_builder::mime::MimePart<'a>,
>,
>,
}
/// The state of ongoing process.
@@ -522,7 +511,6 @@ impl Context {
self_fingerprint: OnceLock::new(),
self_public_key: Mutex::new(None),
connectivities: parking_lot::Mutex::new(Vec::new()),
pre_encrypt_mime_hook: None.into(),
};
let ctx = Context {
@@ -965,12 +953,6 @@ impl Context {
.await?
.to_string(),
);
res.insert(
"delete_server_after",
self.get_config_int(Config::DeleteServerAfter)
.await?
.to_string(),
);
res.insert(
"last_housekeeping",
self.get_config_int(Config::LastHousekeeping)
+12 -7
View File
@@ -6,21 +6,16 @@ use anyhow::{Result, anyhow, bail, ensure};
use deltachat_derive::{FromSql, ToSql};
use serde::{Deserialize, Serialize};
use crate::config::Config;
use crate::context::Context;
use crate::imap::session::Session;
use crate::log::warn;
use crate::message::{self, Message, MsgId, rfc724_mid_exists};
use crate::{EventType, chatlist_events};
use crate::{EventType, chatlist_events, ephemeral};
pub(crate) mod post_msg_metadata;
pub(crate) use post_msg_metadata::PostMsgMetadata;
/// If a message is downloaded only partially
/// and `delete_server_after` is set to small timeouts (eg. "at once"),
/// the user might have no chance to actually download that message.
/// `MIN_DELETE_SERVER_AFTER` increases the timeout in this case.
pub(crate) const MIN_DELETE_SERVER_AFTER: i64 = 48 * 60 * 60;
/// From this point onward outgoing messages are considered large
/// and get a Pre-Message, which announces the Post-Message.
/// This is only about sending so we can modify it any time.
@@ -174,6 +169,16 @@ pub(crate) async fn download_msg(
return Ok(None);
}
Box::pin(session.fetch_single_msg(context, &server_folder, server_uid, rfc724_mid)).await?;
let bcc_self = context.get_config_bool(Config::BccSelf).await?;
if ephemeral::should_delete_all_downloaded_messages(bcc_self, session.is_chatmail()) {
// Now that the message was downloaded, it likely needs to be deleted;
// trigger a re-check by interrupting the inbox folder.
// This is mainly needed to make the tests pass;
// on real devices, it would be fine to delete the message at the next iteration.
context.scheduler.interrupt_inbox().await;
}
Ok(Some(()))
}
+99 -40
View File
@@ -23,16 +23,15 @@
//! ## Device settings
//!
//! In addition to per-chat ephemeral message setting, each device has
//! two global user-configured settings that complement per-chat
//! settings: `delete_device_after` and `delete_server_after`. These
//! settings are not synchronized among devices and apply to all
//! a global user-configured setting that complements per-chat
//! settings, `delete_device_after`.
//! This setting is not synchronized among devices and applies to all
//! messages known to the device, including messages sent or received
//! before configuring the setting.
//!
//! `delete_device_after` configures the maximum time device is
//! storing the messages locally. `delete_server_after` configures the
//! time after which device will delete the messages it knows about
//! from the server.
//! storing the messages locally,
//! but does not delete messages from the server.
//!
//! ## How messages are deleted
//!
@@ -60,9 +59,8 @@
//!
//! Server deletion happens by updating the `imap` table based on
//! the database entries which are expired either according to their
//! ephemeral message timers or global `delete_server_after` setting.
//! ephemeral message timers.
use std::cmp::max;
use std::collections::BTreeSet;
use std::fmt;
use std::num::ParseIntError;
@@ -75,10 +73,11 @@ use serde::{Deserialize, Serialize};
use tokio::time::timeout;
use crate::chat::{ChatId, ChatIdBlocked, send_msg};
use crate::config::Config;
use crate::constants::{DC_CHAT_ID_LAST_SPECIAL, DC_CHAT_ID_TRASH};
use crate::contact::ContactId;
use crate::context::Context;
use crate::download::MIN_DELETE_SERVER_AFTER;
use crate::download::DownloadState;
use crate::events::EventType;
use crate::log::{LogExt, warn};
use crate::message::{Message, MessageState, MsgId, Viewtype};
@@ -455,11 +454,14 @@ WHERE
Ok(rows)
}
/// Deletes messages which are expired according to
/// Locally deletes messages which are expired according to
/// `delete_device_after` setting or `ephemeral_timestamp` column.
///
/// Emits relevant `MsgsChanged` and `WebxdcInstanceDeleted` events
/// if messages are deleted.
///
/// Also see [`delete_expired_imap_messages`],
/// which marks the messages for deletion on the IMAP server.
pub(crate) async fn delete_expired_messages(context: &Context, now: i64) -> Result<()> {
let rows = select_expired_messages(context, now).await?;
@@ -650,42 +652,99 @@ pub(crate) async fn ephemeral_loop(context: &Context, interrupt_receiver: Receiv
}
}
/// Schedules expired IMAP messages for deletion.
#[expect(clippy::arithmetic_side_effects)]
pub(crate) async fn delete_expired_imap_messages(context: &Context) -> Result<()> {
/// Schedules expired IMAP messages for deletion on the server.
///
/// Also see [`delete_expired_messages`],
/// which locally deletes expired messages.
pub(crate) async fn delete_expired_imap_messages(
context: &Context,
transport_id: u32,
is_chatmail: bool,
) -> Result<()> {
let now = time();
let (threshold_timestamp, threshold_timestamp_extended) =
match context.get_config_delete_server_after().await? {
None => (0, 0),
Some(delete_server_after) => (
match delete_server_after {
// Guarantee immediate deletion.
0 => i64::MAX,
_ => now - delete_server_after,
},
now - max(delete_server_after, MIN_DELETE_SERVER_AFTER),
),
};
context
.sql
.execute(
"UPDATE imap
SET target=''
WHERE rfc724_mid IN (
SELECT rfc724_mid FROM msgs
WHERE ((download_state = 0 AND timestamp < ?) OR
(download_state != 0 AND timestamp < ?) OR
(ephemeral_timestamp != 0 AND ephemeral_timestamp <= ?))
)",
(threshold_timestamp, threshold_timestamp_extended, now),
)
.await?;
let bcc_self = context.get_config_bool(Config::BccSelf).await?;
if should_delete_all_downloaded_messages(bcc_self, is_chatmail) {
// This is the only device using this relay.
// Mark all downloaded messages for deletion, because they are not needed anymore.
//
// For pre- and post-messages, `rfc724_mid` contains the post-message's Message-Id.
// The pre-message's Message-Id is in pre_rfc724_mid, if it exists.
//
// Pre-messages can be deleted even if the message wasn't fully downloaded yet,
// because it's only the post-message that hasn't been downloaded.
context
.sql
.execute(
"UPDATE imap
SET target=''
WHERE transport_id=?1
AND rfc724_mid IN (
SELECT rfc724_mid FROM msgs
WHERE ((ephemeral_timestamp!=0 AND ephemeral_timestamp<=?2) OR download_state=?3)
AND id>9
UNION
SELECT pre_rfc724_mid FROM msgs
WHERE pre_rfc724_mid!=''
AND id>9
)",
(transport_id, now, DownloadState::Done),
)
.await?;
} else if bcc_self {
// There may be other devices using this relay,
// either because there is multi-device or because this is a classical email server.
// Only delete expired ephemeral messages.
context
.sql
.execute(
"UPDATE imap
SET target=''
WHERE transport_id=?1
AND rfc724_mid IN (
SELECT rfc724_mid FROM msgs
WHERE ephemeral_timestamp!=0 AND ephemeral_timestamp<=?2 AND id>9
UNION
SELECT pre_rfc724_mid FROM msgs
WHERE pre_rfc724_mid!=''
AND ephemeral_timestamp!=0 AND ephemeral_timestamp<=?2 AND id>9
)",
(transport_id, now),
)
.await?;
} else {
// Single device.
// Delete all expired and encrypted messages.
context
.sql
.execute(
"UPDATE imap
SET target=''
WHERE transport_id=?1
AND rfc724_mid IN (
SELECT rfc724_mid FROM msgs
WHERE id>9
AND ((ephemeral_timestamp!=0 AND ephemeral_timestamp<=?2) OR
((param GLOB '*\nc=1*' OR param GLOB 'c=1*') AND download_state=?3))
UNION
SELECT pre_rfc724_mid FROM msgs
WHERE pre_rfc724_mid!=''
AND id>9
AND ((ephemeral_timestamp!=0 AND ephemeral_timestamp<=?2) OR
(param GLOB '*\nc=1*' OR param GLOB 'c=1*'))
)",
(transport_id, now, DownloadState::Done),
)
.await?;
}
Ok(())
}
pub(crate) fn should_delete_all_downloaded_messages(bcc_self: bool, is_chatmail: bool) -> bool {
!bcc_self && is_chatmail
}
/// Start ephemeral timers for seen messages if they are not started
/// yet.
///
+178 -91
View File
@@ -452,107 +452,194 @@ async fn check_msg_is_deleted(t: &TestContext, chat: &Chat, msg_id: MsgId) {
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_delete_expired_imap_messages() -> Result<()> {
let t = TestContext::new_alice().await;
const HOUR: i64 = 60 * 60;
let now = time();
let transport_id = 1;
let uidvalidity = 12345;
for (id, timestamp, ephemeral_timestamp) in &[
(900, now - 2 * HOUR, 0),
(1000, now - 23 * HOUR - MIN_DELETE_SERVER_AFTER, 0),
(1010, now - 23 * HOUR, 0),
(1020, now - 21 * HOUR, 0),
(1030, now - 19 * HOUR, 0),
(2000, now - 18 * HOUR, now - HOUR),
(2020, now - 17 * HOUR, now + HOUR),
(3000, now + HOUR, 0),
] {
let message_id = id.to_string();
t.sql
.execute(
"INSERT INTO msgs (id, rfc724_mid, timestamp, ephemeral_timestamp) VALUES (?,?,?,?);",
(id, &message_id, timestamp, ephemeral_timestamp),
)
.await?;
let transport_id: u32 = 1;
let uidvalidity = 12345u32;
t.set_config_bool(Config::BccSelf, false).await?;
async fn is_deleted(context: &Context, mid: &str) -> Result<bool> {
Ok(context
.sql
.count(
"SELECT COUNT(*) FROM imap WHERE target='' AND rfc724_mid=?",
(mid,),
)
.await?
== 1)
}
async fn reset_targets(context: &Context) {
context
.sql
.execute("UPDATE imap SET target='INBOX'", ())
.await
.unwrap();
}
// Test messages:
//
// Four messages that were not split into pre- and post- message:
// "expired@localhost" - expired ephemeral message
// "no_expire@localhost" - non-ephemeral message
// "no_expire_unencrypted@localhost" - non-ephemeral message, not encrypted
// "future@localhost" - will expire in the future, but not yet
//
// And four messages that were split into pre- and post-message.
// "expired_*@localhost" - has pre-msg, expired ephemeral message, not downloaded yet
// "no_expire_*@localhost" - has pre-msg, non-ephemeral, not downloaded yet
// "future_*@localhost" - has pre-msg, not expired yet, not downloaded yet
// "done_*@localhost" - Fully downloaded -> post- message can be deleted
//
// The tuple is (rfc724_mid, ephemeral_timestamp, download_state, pre_rfc724_mid, is_encrypted)
let msgs: [(&str, i64, DownloadState, &str, bool); 8] = [
("expired@localhost", now - 1, DownloadState::Done, "", true),
("no_expire@localhost", 0, DownloadState::Done, "", true),
(
"no_expire_unencrypted@localhost",
0,
DownloadState::Done,
"",
false,
),
// Use "now + 3600" rather than "now + 1", otherwise the test may be flaky
// if it is slow and the message expires in a second
(
"future@localhost",
now + 3600,
DownloadState::Done,
"",
true,
),
(
"expired_post@localhost",
now - 1,
DownloadState::Available,
"expired_pre@localhost",
true,
),
(
"no_expire_post@localhost",
0,
DownloadState::Available,
"no_expire_pre@localhost",
true,
),
(
"future_post@localhost",
now + 3600,
DownloadState::Available,
"future_pre@localhost",
true,
),
(
"done_post@localhost",
0,
DownloadState::Done,
"done_pre@localhost",
true,
),
];
for (rfc724_mid, ephemeral_timestamp, download_state, pre_rfc724_mid, is_encrypted) in msgs {
t.sql
.execute(
"INSERT INTO imap (transport_id, rfc724_mid, folder, uid, target, uidvalidity) VALUES (?, ?,'INBOX',?, 'INBOX', ?);",
(transport_id, &message_id, id, uidvalidity),
"INSERT INTO msgs \
(rfc724_mid, timestamp, ephemeral_timestamp, download_state, pre_rfc724_mid, param) \
VALUES (?,?,?,?,?,?)",
(
rfc724_mid,
now,
ephemeral_timestamp,
download_state,
pre_rfc724_mid,
if is_encrypted {
"c=1"
} else {
""
}
),
)
.await?;
}
let rfc724_mids: Vec<&str> = msgs
.iter()
.flat_map(|(rfc724_mid, _, _, pre_rfc724_mid, _is_encrypted)| {
[*rfc724_mid, *pre_rfc724_mid]
})
.filter(|s| !s.is_empty())
.collect();
for (i, rfc724_mid) in rfc724_mids.iter().enumerate() {
t.sql
.execute(
"INSERT INTO imap \
(transport_id, rfc724_mid, folder, uid, target, uidvalidity) \
VALUES (?,?,'INBOX',?,'INBOX',?)",
(transport_id, *rfc724_mid, (i + 1) as u32, uidvalidity),
)
.await?;
}
async fn test_marked_for_deletion(context: &Context, id: u32) -> Result<()> {
assert_eq!(
context
.sql
.count(
"SELECT COUNT(*) FROM imap WHERE target='' AND rfc724_mid=?",
(id.to_string(),),
)
.await?,
1
for (is_chatmail, other_transport, bcc_self) in [
(false, false, false),
(false, false, true),
(false, true, false),
(false, true, true),
(true, false, false),
(true, false, true),
(true, true, false),
(true, true, true),
] {
println!(
"Testing combination is_chatmail={is_chatmail}, other_transport={other_transport}, bcc_self={bcc_self}"
);
Ok(())
t.set_config_bool(Config::BccSelf, bcc_self).await?;
delete_expired_imap_messages(
&t,
if other_transport {
transport_id + 1
} else {
transport_id
},
is_chatmail,
)
.await?;
if other_transport {
// Nothing should be deleted on another transport
for rfc724_mid in &rfc724_mids {
assert_eq!(is_deleted(&t, rfc724_mid).await?, false);
}
continue;
}
assert_eq!(is_deleted(&t, "expired@localhost").await?, true);
assert_eq!(is_deleted(&t, "no_expire@localhost").await?, !bcc_self);
assert_eq!(
is_deleted(&t, "no_expire_unencrypted@localhost").await?,
is_chatmail && !bcc_self
);
assert_eq!(is_deleted(&t, "future@localhost").await?, !bcc_self);
assert_eq!(is_deleted(&t, "expired_post@localhost").await?, true);
assert_eq!(is_deleted(&t, "expired_pre@localhost").await?, true);
assert_eq!(is_deleted(&t, "no_expire_post@localhost").await?, false);
assert_eq!(is_deleted(&t, "no_expire_pre@localhost").await?, !bcc_self);
assert_eq!(is_deleted(&t, "future_post@localhost").await?, false);
assert_eq!(is_deleted(&t, "future_pre@localhost").await?, !bcc_self);
assert_eq!(is_deleted(&t, "done_pre@localhost").await?, !bcc_self);
assert_eq!(is_deleted(&t, "done_post@localhost").await?, !bcc_self);
reset_targets(&t).await;
}
async fn remove_uid(context: &Context, id: u32) -> Result<()> {
context
.sql
.execute("DELETE FROM imap WHERE rfc724_mid=?", (id.to_string(),))
.await?;
Ok(())
}
// This should mark message 2000 for deletion.
delete_expired_imap_messages(&t).await?;
test_marked_for_deletion(&t, 2000).await?;
remove_uid(&t, 2000).await?;
// No other messages are marked for deletion.
assert_eq!(
t.sql
.count("SELECT COUNT(*) FROM imap WHERE target=''", ())
.await?,
0
);
t.set_config(Config::DeleteServerAfter, Some(&*(25 * HOUR).to_string()))
.await?;
delete_expired_imap_messages(&t).await?;
test_marked_for_deletion(&t, 1000).await?;
MsgId::new(1000)
.update_download_state(&t, DownloadState::Available)
.await?;
t.sql
.execute("UPDATE imap SET target=folder WHERE rfc724_mid='1000'", ())
.await?;
delete_expired_imap_messages(&t).await?;
test_marked_for_deletion(&t, 1000).await?; // Delete downloadable anyway.
remove_uid(&t, 1000).await?;
t.set_config(Config::DeleteServerAfter, Some(&*(22 * HOUR).to_string()))
.await?;
delete_expired_imap_messages(&t).await?;
test_marked_for_deletion(&t, 1010).await?;
t.sql
.execute("UPDATE imap SET target=folder WHERE rfc724_mid='1010'", ())
.await?;
MsgId::new(1010)
.update_download_state(&t, DownloadState::Available)
.await?;
delete_expired_imap_messages(&t).await?;
// Keep downloadable for now.
assert_eq!(
t.sql
.count("SELECT COUNT(*) FROM imap WHERE target=''", ())
.await?,
0
);
t.set_config(Config::DeleteServerAfter, Some("1")).await?;
delete_expired_imap_messages(&t).await?;
test_marked_for_deletion(&t, 3000).await?;
// With BccSelf=true, non-expired messages are kept even if `is_chatmail` is true
t.set_config_bool(Config::BccSelf, true).await?;
delete_expired_imap_messages(&t, transport_id, true).await?;
assert_eq!(is_deleted(&t, "expired@localhost").await?, true);
assert_eq!(is_deleted(&t, "no_expire@localhost").await?, false);
assert_eq!(is_deleted(&t, "done_pre@localhost").await?, false);
assert_eq!(is_deleted(&t, "done_post@localhost").await?, false);
Ok(())
}
+12 -3
View File
@@ -21,9 +21,6 @@ use futures_lite::FutureExt;
use ratelimit::Ratelimit;
use url::Url;
use crate::calls::{
UnresolvedIceServer, create_fallback_ice_servers, create_ice_servers_from_metadata,
};
use crate::chat::{self, ChatId, ChatIdBlocked, add_device_msg};
use crate::chatlist_events;
use crate::config::Config;
@@ -49,6 +46,10 @@ use crate::tools::{self, create_id, duration_to_str, time};
use crate::transport::{
ConfiguredLoginParam, ConfiguredServerLoginParam, prioritize_server_login_params,
};
use crate::{
calls::{UnresolvedIceServer, create_fallback_ice_servers, create_ice_servers_from_metadata},
ephemeral::delete_expired_imap_messages,
};
pub(crate) mod capabilities;
mod client;
@@ -525,6 +526,12 @@ impl Imap {
context.scheduler.interrupt_ephemeral_task().await;
}
// Mark expired messages for deletion. Note that `delete_expired_imap_messages` is
// not well optimized and should not be called before fetching.
delete_expired_imap_messages(context, session.transport_id(), session.is_chatmail())
.await
.context("delete_expired_imap_messages")?;
session
.move_delete_messages(context, watch_folder)
.await
@@ -1379,6 +1386,8 @@ impl Session {
"Passing message UID {} to receive_imf().", request_uid
);
let res = receive_imf_inner(context, rfc724_mid, body, is_seen).await;
// If there was an error receiving the message, show a device message:
let received_msg = match res {
Err(err) => {
warn!(context, "receive_imf error: {err:#}.");
+8 -19
View File
@@ -979,21 +979,16 @@ mod tests {
context1.set_config(Config::BccSelf, None).await?;
// Check that the settings are displayed correctly.
assert_eq!(
context1.get_config(Config::DeleteServerAfter).await?,
Some("0".to_string())
);
context1.set_config_bool(Config::IsChatmail, true).await?;
assert_eq!(
context1.get_config(Config::BccSelf).await?,
Some("0".to_string())
);
assert_eq!(
context1.get_config(Config::DeleteServerAfter).await?,
Some("1".to_string())
);
context1.set_config_bool(Config::IsChatmail, true).await?;
assert_eq!(context1.get_config_bool(Config::IsMuted).await?, false);
context1.set_config_bool(Config::IsMuted, true).await?;
assert_eq!(context1.get_config_bool(Config::IsMuted).await?, true);
assert_eq!(context1.get_config_delete_server_after().await?, Some(0));
imex(context1, ImexMode::ExportBackup, backup_dir.path(), None).await?;
let _event = context1
.evtracker
@@ -1010,15 +1005,9 @@ mod tests {
assert!(context2.is_configured().await?);
assert!(context2.is_chatmail().await?);
for ctx in [context1, context2] {
assert_eq!(
ctx.get_config(Config::BccSelf).await?,
Some("1".to_string())
);
assert_eq!(
ctx.get_config(Config::DeleteServerAfter).await?,
Some("0".to_string())
);
assert_eq!(ctx.get_config_delete_server_after().await?, None);
// BccSelf should be enabled automatically when exporting a backup
assert_eq!(ctx.get_config_bool(Config::BccSelf).await?, true);
assert_eq!(ctx.get_config_bool(Config::IsMuted).await?, true);
}
Ok(())
}
+9
View File
@@ -56,6 +56,15 @@ pub enum EnteredCertificateChecks {
AcceptInvalidCertificates2 = 3,
}
impl EnteredCertificateChecks {
pub(crate) fn accept_invalid_certificates(self) -> bool {
matches!(
self,
Self::AcceptInvalidCertificates | Self::AcceptInvalidCertificates2
)
}
}
/// Login parameters for a single IMAP server.
#[derive(Debug, Default, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct EnteredImapLoginParam {
+66 -43
View File
@@ -140,8 +140,20 @@ SELECT ?1, rfc724_mid, pre_rfc724_mid, timestamp, ?, ? FROM msgs WHERE id=?1
Ok(())
}
pub(crate) async fn set_delivered(self, context: &Context) -> Result<()> {
update_msg_state(context, self, MessageState::OutDelivered).await?;
/// Returns whether the message state is updated to `OutDelivered`.
pub(crate) async fn set_delivered(self, context: &Context) -> Result<bool> {
if context
.sql
.execute(
// Only update `OutPending` i.e. if the message is (re-)sent to all chat members.
"UPDATE msgs SET state=?, error='' WHERE id=? AND state=?",
(MessageState::OutDelivered, self, MessageState::OutPending),
)
.await?
== 0
{
return Ok(false);
}
let chat_id: Option<ChatId> = context
.sql
.query_get_value("SELECT chat_id FROM msgs WHERE id=?", (self,))
@@ -153,7 +165,7 @@ SELECT ?1, rfc724_mid, pre_rfc724_mid, timestamp, ?, ? FROM msgs WHERE id=?1
if let Some(chat_id) = chat_id {
chatlist_events::emit_chatlist_item_changed(context, chat_id);
}
Ok(())
Ok(true)
}
/// Bad evil escape hatch.
@@ -312,6 +324,7 @@ SELECT ?1, rfc724_mid, pre_rfc724_mid, timestamp, ?, ? FROM msgs WHERE id=?1
if duration != 0 {
ret += &format!("Duration: {duration} ms\n",);
}
ret += &format!("\nDatabase ID: {}", msg.id);
if !msg.rfc724_mid.is_empty() {
ret += &format!("\nMessage-ID: {}", msg.rfc724_mid);
@@ -602,6 +615,31 @@ impl Message {
Ok(msg)
}
/// Loads the message with given Message-ID from the database.
///
/// Cannot return a trashed message.
pub async fn load_by_rfc724_mid_optional(
context: &Context,
rfc724_mid: &str,
) -> Result<Option<Message>> {
if let Some(msg_id) = context
.sql
.query_row_optional(
"SELECT id FROM msgs WHERE rfc724_mid=? AND chat_id != ?",
(rfc724_mid, DC_CHAT_ID_TRASH),
|row| {
let msg_id: MsgId = row.get(0)?;
Ok(msg_id)
},
)
.await?
{
Self::load_from_db_optional(context, msg_id).await
} else {
Ok(None)
}
}
/// Returns additional text which is appended to the message's text field
/// when it is loaded from the database.
/// Currently this is used to add infomation to pre-messages of what the download will be and how large it is
@@ -1389,6 +1427,9 @@ pub enum MessageState {
/// The user has pressed the "send" button but the message is not
/// yet sent and is pending in some way. Maybe we're offline (no
/// checkmark).
///
/// This state means that the message is being (re-)sent to all chat members. It shalln't be
/// used e.g. for resending only to a new broadcast member.
OutPending = 20,
/// *Unrecoverable* error (*recoverable* errors result in pending
@@ -2001,13 +2042,6 @@ pub(crate) async fn update_msg_state(
Ok(())
}
// as we do not cut inside words, this results in about 32-42 characters.
// Do not use too long subjects - we add a tag after the subject which gets truncated by the clients otherwise.
// It should also be very clear, the subject is _not_ the whole message.
// The value is also used for CC:-summaries
// Context functions to work with messages
pub(crate) async fn set_msg_failed(
context: &Context,
msg: &mut Message,
@@ -2099,63 +2133,52 @@ pub async fn get_request_msg_cnt(context: &Context) -> usize {
}
/// Estimates the number of messages that will be deleted
/// by the options `delete_device_after` or `delete_server_after`.
/// by the `set_config()`-option `delete_device_after`.
///
/// This is typically used to show the estimated impact to the user
/// before actually enabling deletion of old messages.
///
/// If `from_server` is true,
/// estimate deletion count for server,
/// otherwise estimate deletion count for device.
/// Messages in the "Saved Messages" chat are not counted as they will not be deleted automatically.
///
/// Count messages older than the given number of `seconds`.
/// Parameters:
/// - `from_server`: Deprecated, pass `false` here
/// - `seconds`: Count messages older than the given number of seconds.
///
/// Returns the number of messages that are older than the given number of seconds.
/// Messages in the "saved messages" folder are not counted as they will not be deleted automatically.
#[expect(clippy::arithmetic_side_effects)]
pub async fn estimate_deletion_cnt(
context: &Context,
from_server: bool,
seconds: i64,
) -> Result<usize> {
ensure!(
!from_server,
"The `delete_server_after` config option was removed. You need to pass `false` for `from_server`"
);
let self_chat_id = ChatIdBlocked::lookup_by_contact(context, ContactId::SELF)
.await?
.map(|c| c.id)
.unwrap_or_default();
let threshold_timestamp = time() - seconds;
let cnt = if from_server {
context
.sql
.count(
"SELECT COUNT(*)
FROM msgs m
WHERE m.id > ?
AND timestamp < ?
AND chat_id != ?
AND EXISTS (SELECT * FROM imap WHERE rfc724_mid=m.rfc724_mid);",
(DC_MSG_ID_LAST_SPECIAL, threshold_timestamp, self_chat_id),
)
.await?
} else {
context
.sql
.count(
"SELECT COUNT(*)
let cnt = context
.sql
.count(
"SELECT COUNT(*)
FROM msgs m
WHERE m.id > ?
AND timestamp < ?
AND chat_id != ?
AND chat_id != ? AND hidden = 0;",
(
DC_MSG_ID_LAST_SPECIAL,
threshold_timestamp,
self_chat_id,
DC_CHAT_ID_TRASH,
),
)
.await?
};
(
DC_MSG_ID_LAST_SPECIAL,
threshold_timestamp,
self_chat_id,
DC_CHAT_ID_TRASH,
),
)
.await?;
Ok(cnt)
}
-6
View File
@@ -1168,12 +1168,6 @@ impl MimeFactory {
_ => None,
};
if context.get_config_bool(Config::TestHooks).await?
&& let Some(hook) = &*context.pre_encrypt_mime_hook.lock()
{
message = hook(context, message);
}
let encrypted = if let Some(shared_secret) = shared_secret {
let sign = true;
encrypt_helper
+15 -23
View File
@@ -1790,35 +1790,27 @@ async fn test_time_in_future() -> Result<()> {
Ok(())
}
/// Tests receiving a message with RFC 9788 header protection and legacy display element.
///
/// Legacy display elements should not be rendered:
/// <https://www.rfc-editor.org/rfc/rfc9788.html#name-do-not-render-legacy-displa>
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_hp_legacy_display() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
let bob = &tcm.bob().await;
let mut msg = Message::new_text(
"Subject: Dinner plans\n\
\n\
Let's eat"
.to_string(),
);
msg.set_subject("Dinner plans".to_string());
let chat_id = alice.create_chat(bob).await.id;
alice.set_config_bool(Config::TestHooks, true).await?;
*alice.pre_encrypt_mime_hook.lock() = Some(|_, mut mime| {
for (h, v) in &mut mime.headers {
if h == "Content-Type"
&& let mail_builder::headers::HeaderType::ContentType(ct) = v
{
*ct = ct.clone().attribute("hp-legacy-display", "1");
}
}
mime
});
let sent_msg = alice.send_msg(chat_id, &mut msg).await;
let msg_bob = bob.recv_msg(&sent_msg).await;
let msg_id = receive_imf(
bob,
include_bytes!("../../test-data/message/hp_legacy_display.eml"),
false,
)
.await?
.unwrap()
.msg_ids[0];
let msg_bob = Message::load_from_db(bob, msg_id).await?;
assert_eq!(msg_bob.subject, "Dinner plans");
// Legacy display element is removed from the text/plain body.
assert_eq!(msg_bob.text, "Let's eat");
Ok(())
}
+1
View File
@@ -23,6 +23,7 @@ pub(crate) mod session;
pub(crate) mod tls;
use dns::lookup_host_with_cache;
pub(crate) use http::read_url_with_tls;
pub use http::{Response as HttpResponse, read_url, read_url_blob};
use tls::wrap_tls;
+50 -3
View File
@@ -231,7 +231,10 @@ static DNS_PRELOAD: LazyLock<HashMap<&'static str, Vec<IpAddr>>> = LazyLock::new
HashMap::from([
(
"imap.163.com",
vec![IpAddr::V4(Ipv4Addr::new(111, 124, 203, 45))],
vec![
IpAddr::V4(Ipv4Addr::new(111, 124, 203, 45)),
IpAddr::V4(Ipv4Addr::new(111, 124, 203, 50)),
],
),
(
"smtp.163.com",
@@ -422,12 +425,12 @@ static DNS_PRELOAD: LazyLock<HashMap<&'static str, Vec<IpAddr>>> = LazyLock::new
"nine.testrun.org",
vec![
IpAddr::V4(Ipv4Addr::new(128, 140, 126, 197)),
IpAddr::V4(Ipv4Addr::new(116, 202, 233, 236)),
IpAddr::V4(Ipv4Addr::new(216, 144, 228, 100)),
IpAddr::V6(Ipv6Addr::new(0x2a01, 0x4f8, 0x241, 0x4ce8, 0, 0, 0, 2)),
IpAddr::V4(Ipv4Addr::new(77, 42, 49, 41)),
IpAddr::V6(Ipv6Addr::new(
0x2001, 0x41d0, 0x701, 0x1100, 0, 0, 0, 0x8ab1,
)),
IpAddr::V6(Ipv6Addr::new(0x2a01, 0x4f9, 0xfff1, 0x59, 0, 0, 0, 1)),
],
),
(
@@ -697,6 +700,10 @@ static DNS_PRELOAD: LazyLock<HashMap<&'static str, Vec<IpAddr>>> = LazyLock::new
"chatmail.hackea.org",
vec![IpAddr::V4(Ipv4Addr::new(82, 165, 11, 85))],
),
(
"chat.adminforge.de",
vec![IpAddr::V4(Ipv4Addr::new(94, 130, 17, 142))],
),
(
"chika.aangat.lahat.computer",
vec![IpAddr::V4(Ipv4Addr::new(71, 19, 150, 113))],
@@ -738,6 +745,46 @@ static DNS_PRELOAD: LazyLock<HashMap<&'static str, Vec<IpAddr>>> = LazyLock::new
"danneskjold.de",
vec![IpAddr::V4(Ipv4Addr::new(46, 62, 216, 132))],
),
(
"chat.in-the.eu",
vec![IpAddr::V4(Ipv4Addr::new(78, 46, 190, 129))],
),
(
"chat.nuvon.app",
vec![IpAddr::V4(Ipv4Addr::new(178, 238, 38, 165))],
),
(
"nibblehole.com",
vec![IpAddr::V4(Ipv4Addr::new(94, 247, 42, 209))],
),
(
"chat.zashm.org",
vec![IpAddr::V4(Ipv4Addr::new(91, 245, 76, 39))],
),
(
"chat.sus.fr",
vec![IpAddr::V4(Ipv4Addr::new(152, 67, 76, 190))],
),
(
"delta.thelab.uno",
vec![IpAddr::V4(Ipv4Addr::new(146, 59, 228, 39))],
),
(
"chat.vim.wtf",
vec![IpAddr::V4(Ipv4Addr::new(116, 203, 206, 170))],
),
(
"uninterest.ing",
vec![IpAddr::V4(Ipv4Addr::new(172, 245, 70, 237))],
),
(
"sweetfern.net",
vec![IpAddr::V4(Ipv4Addr::new(178, 156, 228, 133))],
),
(
"delta.disobey.net",
vec![IpAddr::V4(Ipv4Addr::new(37, 74, 102, 44))],
),
(
"darkrun.dev",
vec![IpAddr::V4(Ipv4Addr::new(72, 11, 149, 146))],
+58 -37
View File
@@ -13,7 +13,7 @@ use crate::context::Context;
use crate::log::warn;
use crate::net::proxy::ProxyConfig;
use crate::net::session::SessionStream;
use crate::net::tls::wrap_rustls;
use crate::net::tls::wrap_tls;
use crate::tools::time;
/// User-Agent for HTTP requests if a resource usage policy requires it.
@@ -35,7 +35,16 @@ pub struct Response {
/// Retrieves the text contents of URL using HTTP GET request.
pub async fn read_url(context: &Context, url: &str) -> Result<String> {
let response = read_url_blob(context, url).await?;
read_url_with_tls(context, url, true).await
}
/// Retrieves the text contents of URL using HTTP GET request.
pub(crate) async fn read_url_with_tls(
context: &Context,
url: &str,
strict_tls: bool,
) -> Result<String> {
let response = read_url_blob_with_tls(context, url, strict_tls).await?;
let text = String::from_utf8_lossy(&response.blob);
Ok(text.to_string())
}
@@ -43,6 +52,7 @@ pub async fn read_url(context: &Context, url: &str) -> Result<String> {
async fn get_http_sender<B>(
context: &Context,
parsed_url: hyper::Uri,
strict_tls: bool,
) -> Result<hyper::client::conn::http1::SendRequest<B>>
where
B: hyper::body::Body + 'static + Send,
@@ -76,37 +86,29 @@ where
let port = parsed_url.port_u16().unwrap_or(443);
let (use_sni, load_cache) = (true, true);
if let Some(proxy_config) = proxy_config_opt {
let tcp_stream: Box<dyn SessionStream> = if let Some(proxy_config) = proxy_config_opt {
let proxy_stream = proxy_config
.connect(context, host, port, load_cache)
.await?;
let tls_stream = wrap_rustls(
host,
port,
use_sni,
"",
proxy_stream,
&context.tls_session_store,
&context.spki_hash_store,
&context.sql,
)
.await?;
Box::new(tls_stream)
Box::new(proxy_stream)
} else {
let tcp_stream = crate::net::connect_tcp(context, host, port, load_cache).await?;
let tls_stream = wrap_rustls(
host,
port,
use_sni,
"",
tcp_stream,
&context.tls_session_store,
&context.spki_hash_store,
&context.sql,
)
.await?;
Box::new(tls_stream)
}
Box::new(tcp_stream)
};
let tls_stream = wrap_tls(
strict_tls,
host,
port,
use_sni,
"",
tcp_stream,
&context.tls_session_store,
&context.spki_hash_store,
&context.sql,
)
.await?;
Box::new(tls_stream)
}
_ => bail!("Unknown URL scheme"),
};
@@ -260,7 +262,7 @@ pub(crate) async fn http_cache_cleanup(context: &Context) -> Result<()> {
/// Fetches URL and updates the cache.
///
/// URL is fetched regardless of whether there is an existing result in the cache.
async fn fetch_url(context: &Context, original_url: &str) -> Result<Response> {
async fn fetch_url(context: &Context, original_url: &str, strict_tls: bool) -> Result<Response> {
let mut url = original_url.to_string();
// Follow up to 10 http-redirects
@@ -269,7 +271,7 @@ async fn fetch_url(context: &Context, original_url: &str) -> Result<Response> {
.parse::<hyper::Uri>()
.with_context(|| format!("Failed to parse URL {url:?}"))?;
let mut sender = get_http_sender(context, parsed_url.clone()).await?;
let mut sender = get_http_sender(context, parsed_url.clone(), strict_tls).await?;
let authority = parsed_url
.authority()
.context("URL has no authority")?
@@ -339,8 +341,10 @@ async fn fetch_url(context: &Context, original_url: &str) -> Result<Response> {
mimetype,
encoding,
};
info!(context, "Inserting {original_url:?} into cache.");
http_cache_put(context, &url, &response).await?;
if strict_tls {
info!(context, "Inserting {original_url:?} into cache.");
http_cache_put(context, &url, &response).await?;
}
return Ok(response);
}
@@ -349,6 +353,23 @@ async fn fetch_url(context: &Context, original_url: &str) -> Result<Response> {
/// Retrieves the binary contents of URL using HTTP GET request.
pub async fn read_url_blob(context: &Context, url: &str) -> Result<Response> {
read_url_blob_with_tls(context, url, true).await
}
/// Retrieves the binary contents of URL using HTTP GET request.
pub(crate) async fn read_url_blob_with_tls(
context: &Context,
url: &str,
strict_tls: bool,
) -> Result<Response> {
if !strict_tls {
info!(
context,
"Fetching {url:?} without HTTP cache due to relaxed TLS."
);
return fetch_url(context, url, strict_tls).await;
}
if let Some((response, is_stale)) = http_cache_get(context, url).await? {
info!(context, "Returning {url:?} from cache.");
if is_stale {
@@ -357,7 +378,7 @@ pub async fn read_url_blob(context: &Context, url: &str) -> Result<Response> {
tokio::spawn(async move {
// Fetch URL in background to update the cache.
info!(context, "Fetching stale {url:?} in background.");
if let Err(err) = fetch_url(&context, &url).await {
if let Err(err) = fetch_url(&context, &url, true).await {
warn!(context, "Failed to revalidate {url:?}: {err:#}.");
}
});
@@ -366,7 +387,7 @@ pub async fn read_url_blob(context: &Context, url: &str) -> Result<Response> {
}
info!(context, "Not found {url:?} in cache, fetching.");
let response = fetch_url(context, url).await?;
let response = fetch_url(context, url, true).await?;
Ok(response)
}
@@ -384,7 +405,7 @@ pub(crate) async fn post_empty(context: &Context, url: &str) -> Result<(String,
bail!("POST requests to non-HTTPS URLs are not allowed");
}
let mut sender = get_http_sender(context, parsed_url.clone()).await?;
let mut sender = get_http_sender(context, parsed_url.clone(), true).await?;
let authority = parsed_url
.authority()
.context("URL has no authority")?
@@ -418,7 +439,7 @@ pub(crate) async fn post_string(context: &Context, url: &str, body: String) -> R
bail!("POST requests to non-HTTPS URLs are not allowed");
}
let mut sender = get_http_sender(context, parsed_url.clone()).await?;
let mut sender = get_http_sender(context, parsed_url.clone(), true).await?;
let authority = parsed_url
.authority()
.context("URL has no authority")?
@@ -449,7 +470,7 @@ pub(crate) async fn post_form<T: Serialize + ?Sized>(
}
let encoded_body = serde_urlencoded::to_string(form).context("Failed to encode data")?;
let mut sender = get_http_sender(context, parsed_url.clone()).await?;
let mut sender = get_http_sender(context, parsed_url.clone(), true).await?;
let authority = parsed_url
.authority()
.context("URL has no authority")?
+110 -41
View File
@@ -50,7 +50,7 @@ impl ShadowsocksConfig {
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct HttpConfig {
/// HTTP proxy host.
pub host: String,
pub host: url::Host,
/// HTTP proxy port.
pub port: u16,
@@ -63,10 +63,7 @@ pub struct HttpConfig {
impl HttpConfig {
fn from_url(url: Url) -> Result<Self> {
let host = url
.host_str()
.context("HTTP proxy URL has no host")?
.to_string();
let host = url.host().context("HTTP proxy URL has no host")?.to_owned();
let port = url
.port_or_known_default()
.context("HTTP(S) URLs are guaranteed to return Some port")?;
@@ -104,7 +101,8 @@ impl HttpConfig {
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct Socks5Config {
pub host: String,
/// Hostname or IP address.
pub host: url::Host,
pub port: u16,
pub user_password: Option<(String, String)>,
}
@@ -117,7 +115,13 @@ impl Socks5Config {
target_port: u16,
load_dns_cache: bool,
) -> Result<Socks5Stream<Pin<Box<TimeoutStream<TcpStream>>>>> {
let tcp_stream = connect_tcp(context, &self.host, self.port, load_dns_cache)
let hostname = match &self.host {
url::Host::Domain(domain) => domain.to_string(),
url::Host::Ipv4(addr) => addr.to_string(),
url::Host::Ipv6(addr) => addr.to_string(),
};
let tcp_stream = connect_tcp(context, &hostname, self.port, load_dns_cache)
.await
.context("Failed to connect to SOCKS5 proxy")?;
@@ -273,10 +277,7 @@ impl ProxyConfig {
// Because of this we do not distinguish
// between `socks5` and `socks5h`.
"socks5" => {
let host = url
.host_str()
.context("socks5 URL has no host")?
.to_string();
let host = url.host().context("socks5 URL has no host")?.to_owned();
let port = url.port().unwrap_or(DEFAULT_SOCKS_PORT);
let user_password = if let Some(password) = url.password() {
let username = percent_encoding::percent_decode_str(url.username())
@@ -402,13 +403,14 @@ impl ProxyConfig {
match self {
ProxyConfig::Http(http_config) => {
let load_cache = false;
let tcp_stream = crate::net::connect_tcp(
context,
&http_config.host,
http_config.port,
load_cache,
)
.await?;
let hostname = match &http_config.host {
url::Host::Domain(domain) => domain.to_string(),
url::Host::Ipv4(addr) => addr.to_string(),
url::Host::Ipv6(addr) => addr.to_string(),
};
let tcp_stream =
crate::net::connect_tcp(context, &hostname, http_config.port, load_cache)
.await?;
let auth = if let Some((username, password)) = &http_config.user_password {
Some((username.as_str(), password.as_str()))
} else {
@@ -419,16 +421,18 @@ impl ProxyConfig {
}
ProxyConfig::Https(https_config) => {
let load_cache = true;
let tcp_stream = crate::net::connect_tcp(
context,
&https_config.host,
https_config.port,
load_cache,
)
.await?;
let hostname = match &https_config.host {
url::Host::Domain(domain) => domain.to_string(),
url::Host::Ipv4(addr) => addr.to_string(),
url::Host::Ipv6(addr) => addr.to_string(),
};
let tcp_stream =
crate::net::connect_tcp(context, &hostname, https_config.port, load_cache)
.await?;
let use_sni = true;
let tls_stream = wrap_rustls(
&https_config.host,
&hostname,
https_config.port,
use_sni,
"",
@@ -500,6 +504,7 @@ mod tests {
use super::*;
use crate::config::Config;
use crate::test_utils::TestContext;
use std::net::{Ipv4Addr, Ipv6Addr};
#[test]
fn test_socks5_url() {
@@ -507,17 +512,35 @@ mod tests {
assert_eq!(
proxy_config,
ProxyConfig::Socks5(Socks5Config {
host: "127.0.0.1".to_string(),
// IPv4 address is parsed as a domain and not url::Host::Ipv4.
// This is expected: <https://github.com/servo/rust-url/issues/767>.
// We only need a distinction for IPv6 to remove square brackets
// before passing the address to `lookup_host()`.
host: url::Host::Domain("127.0.0.1".to_string()),
port: 9050,
user_password: None
})
);
assert_eq!(proxy_config.to_url(), "socks5://127.0.0.1:9050".to_string());
let proxy_config = ProxyConfig::from_url("socks5://[::1]:9050").unwrap();
assert_eq!(
proxy_config,
ProxyConfig::Socks5(Socks5Config {
// IPv6 address should be recognized as IPv6 address and not "[::1]" hostname.
// Otherwise we may try to resolve "[::1]" and fail to connect.
host: url::Host::Ipv6(Ipv6Addr::LOCALHOST),
port: 9050,
user_password: None
})
);
assert_eq!(proxy_config.to_url(), "socks5://[::1]:9050".to_string());
let proxy_config = ProxyConfig::from_url("socks5://foo:bar@127.0.0.1:9150").unwrap();
assert_eq!(
proxy_config,
ProxyConfig::Socks5(Socks5Config {
host: "127.0.0.1".to_string(),
host: url::Host::Domain("127.0.0.1".to_string()),
port: 9150,
user_password: Some(("foo".to_string(), "bar".to_string()))
})
@@ -527,7 +550,7 @@ mod tests {
assert_eq!(
proxy_config,
ProxyConfig::Socks5(Socks5Config {
host: "127.0.0.1".to_string(),
host: url::Host::Domain("127.0.0.1".to_string()),
port: 9150,
user_password: Some(("foo".to_string(), "bar".to_string()))
})
@@ -537,7 +560,7 @@ mod tests {
assert_eq!(
proxy_config,
ProxyConfig::Socks5(Socks5Config {
host: "127.0.0.1".to_string(),
host: url::Host::Domain("127.0.0.1".to_string()),
port: 80,
user_password: None
})
@@ -547,7 +570,7 @@ mod tests {
assert_eq!(
proxy_config,
ProxyConfig::Socks5(Socks5Config {
host: "127.0.0.1".to_string(),
host: url::Host::Domain("127.0.0.1".to_string()),
port: 1080,
user_password: None
})
@@ -557,7 +580,7 @@ mod tests {
assert_eq!(
proxy_config,
ProxyConfig::Socks5(Socks5Config {
host: "127.0.0.1".to_string(),
host: url::Host::Domain("127.0.0.1".to_string()),
port: 1080,
user_password: None
})
@@ -567,7 +590,7 @@ mod tests {
assert_eq!(
proxy_config,
ProxyConfig::Socks5(Socks5Config {
host: "my-proxy.example.org".to_string(),
host: url::Host::Domain("my-proxy.example.org".to_string()),
port: 1080,
user_password: None
})
@@ -584,37 +607,61 @@ mod tests {
assert_eq!(
proxy_config,
ProxyConfig::Http(HttpConfig {
host: "127.0.0.1".to_string(),
host: url::Host::Ipv4(Ipv4Addr::LOCALHOST),
port: 80,
user_password: None
})
);
let proxy_config = ProxyConfig::from_url("http://[::1]").unwrap();
assert_eq!(
proxy_config,
ProxyConfig::Http(HttpConfig {
host: url::Host::Ipv6(Ipv6Addr::LOCALHOST),
port: 80,
user_password: None
})
);
assert_eq!(proxy_config.to_url(), "http://[::1]:80".to_string());
let proxy_config = ProxyConfig::from_url("http://127.0.0.1:80").unwrap();
assert_eq!(
proxy_config,
ProxyConfig::Http(HttpConfig {
host: "127.0.0.1".to_string(),
host: url::Host::Ipv4(Ipv4Addr::LOCALHOST),
port: 80,
user_password: None
})
);
assert_eq!(proxy_config.to_url(), "http://127.0.0.1:80".to_string());
let proxy_config = ProxyConfig::from_url("http://[::1]:80").unwrap();
assert_eq!(
proxy_config,
ProxyConfig::Http(HttpConfig {
host: url::Host::Ipv6(Ipv6Addr::LOCALHOST),
port: 80,
user_password: None
})
);
assert_eq!(proxy_config.to_url(), "http://[::1]:80".to_string());
let proxy_config = ProxyConfig::from_url("http://127.0.0.1:443").unwrap();
assert_eq!(
proxy_config,
ProxyConfig::Http(HttpConfig {
host: "127.0.0.1".to_string(),
host: url::Host::Ipv4(Ipv4Addr::LOCALHOST),
port: 443,
user_password: None
})
);
assert_eq!(proxy_config.to_url(), "http://127.0.0.1:443".to_string());
let proxy_config = ProxyConfig::from_url("http://my-proxy.example.org").unwrap();
assert_eq!(
proxy_config,
ProxyConfig::Http(HttpConfig {
host: "my-proxy.example.org".to_string(),
host: url::Host::Domain("my-proxy.example.org".to_string()),
port: 80,
user_password: None
})
@@ -631,7 +678,7 @@ mod tests {
assert_eq!(
proxy_config,
ProxyConfig::Https(HttpConfig {
host: "127.0.0.1".to_string(),
host: url::Host::Ipv4(Ipv4Addr::LOCALHOST),
port: 443,
user_password: None
})
@@ -641,7 +688,17 @@ mod tests {
assert_eq!(
proxy_config,
ProxyConfig::Https(HttpConfig {
host: "127.0.0.1".to_string(),
host: url::Host::Ipv4(Ipv4Addr::LOCALHOST),
port: 80,
user_password: None
})
);
let proxy_config = ProxyConfig::from_url("https://[::1]:80").unwrap();
assert_eq!(
proxy_config,
ProxyConfig::Https(HttpConfig {
host: url::Host::Ipv6(Ipv6Addr::LOCALHOST),
port: 80,
user_password: None
})
@@ -651,17 +708,29 @@ mod tests {
assert_eq!(
proxy_config,
ProxyConfig::Https(HttpConfig {
host: "127.0.0.1".to_string(),
host: url::Host::Ipv4(Ipv4Addr::LOCALHOST),
port: 443,
user_password: None
})
);
assert_eq!(proxy_config.to_url(), "https://127.0.0.1:443".to_string());
let proxy_config = ProxyConfig::from_url("https://[::1]:443").unwrap();
assert_eq!(
proxy_config,
ProxyConfig::Https(HttpConfig {
host: url::Host::Ipv6(Ipv6Addr::LOCALHOST),
port: 443,
user_password: None
})
);
assert_eq!(proxy_config.to_url(), "https://[::1]:443".to_string());
let proxy_config = ProxyConfig::from_url("https://my-proxy.example.org").unwrap();
assert_eq!(
proxy_config,
ProxyConfig::Https(HttpConfig {
host: "my-proxy.example.org".to_string(),
host: url::Host::Domain("my-proxy.example.org".to_string()),
port: 443,
user_password: None
})
+6 -3
View File
@@ -126,9 +126,12 @@ pub async fn wrap_rustls<'a>(
let root_cert_store =
rustls::RootCertStore::from_iter(webpki_roots::TLS_SERVER_ROOTS.iter().cloned());
let mut config = rustls::ClientConfig::builder()
.with_root_certificates(root_cert_store)
.with_no_client_auth();
let mut config = rustls::ClientConfig::builder_with_provider(Arc::new(
rustls::crypto::aws_lc_rs::default_provider(),
))
.with_safe_default_protocol_versions()?
.with_root_certificates(root_cert_store)
.with_no_client_auth();
config.alpn_protocols = if alpn.is_empty() {
vec![]
} else {
+4 -4
View File
@@ -51,7 +51,7 @@ impl rustls::client::danger::ServerCertVerifier for CustomCertificateVerifier {
let spki = parsed_certificate.subject_public_key_info();
let provider = rustls::crypto::ring::default_provider();
let provider = rustls::crypto::aws_lc_rs::default_provider();
if let ServerName::DnsName(dns_name) = server_name
&& dns_name.as_ref().starts_with("_")
@@ -97,7 +97,7 @@ impl rustls::client::danger::ServerCertVerifier for CustomCertificateVerifier {
cert: &CertificateDer<'_>,
dss: &rustls::DigitallySignedStruct,
) -> Result<rustls::client::danger::HandshakeSignatureValid, rustls::Error> {
let provider = rustls::crypto::ring::default_provider();
let provider = rustls::crypto::aws_lc_rs::default_provider();
let supported_schemes = &provider.signature_verification_algorithms;
rustls::crypto::verify_tls12_signature(message, cert, dss, supported_schemes)
}
@@ -108,13 +108,13 @@ impl rustls::client::danger::ServerCertVerifier for CustomCertificateVerifier {
cert: &CertificateDer<'_>,
dss: &rustls::DigitallySignedStruct,
) -> Result<rustls::client::danger::HandshakeSignatureValid, rustls::Error> {
let provider = rustls::crypto::ring::default_provider();
let provider = rustls::crypto::aws_lc_rs::default_provider();
let supported_schemes = &provider.signature_verification_algorithms;
rustls::crypto::verify_tls13_signature(message, cert, dss, supported_schemes)
}
fn supported_verify_schemes(&self) -> Vec<rustls::SignatureScheme> {
let provider = rustls::crypto::ring::default_provider();
let provider = rustls::crypto::aws_lc_rs::default_provider();
provider
.signature_verification_algorithms
.supported_schemes()
+5 -11
View File
@@ -890,16 +890,10 @@ static P_NAUTA_CU: Provider = Provider {
strict_tls: false,
..ProviderOptions::new()
},
config_defaults: Some(&[
ConfigDefault {
key: Config::DeleteServerAfter,
value: "1",
},
ConfigDefault {
key: Config::MediaQuality,
value: "1",
},
]),
config_defaults: Some(&[ConfigDefault {
key: Config::MediaQuality,
value: "1",
}]),
oauth2_authorizer: None,
};
@@ -2382,4 +2376,4 @@ pub(crate) static PROVIDER_IDS: LazyLock<HashMap<&'static str, &'static Provider
});
pub static _PROVIDER_UPDATED: LazyLock<chrono::NaiveDate> =
LazyLock::new(|| chrono::NaiveDate::from_ymd_opt(2026, 4, 21).unwrap());
LazyLock::new(|| chrono::NaiveDate::from_ymd_opt(2026, 5, 6).unwrap());
+26 -26
View File
@@ -835,21 +835,16 @@ UPDATE config SET value=? WHERE keyname='configured_addr' AND value!=?1
{
can_info_msg = false;
if mime_parser.pre_message == PreMessageMode::Post
&& let Some(msg_id) = message::rfc724_mid_exists(context, rfc724_mid_orig).await?
&& let Some(msg) =
Message::load_by_rfc724_mid_optional(context, rfc724_mid_orig).await?
{
// The messsage is a post-message and pre-message exists.
// Assign status update to existing message because just received post-message will be trashed.
Some(
Message::load_from_db(context, msg_id)
.await
.context("Failed to load webxdc instance that we just checked exists")?,
)
Some(msg)
} else {
Some(
Message::load_from_db(context, insert_msg_id)
.await
.context("Failed to load just created webxdc instance")?,
)
Message::load_from_db_optional(context, insert_msg_id)
.await
.context("Failed to load just created webxdc instance")?
}
} else if let Some(field) = mime_parser.get_header(HeaderDef::InReplyTo) {
if let Some(instance) =
@@ -910,10 +905,8 @@ UPDATE config SET value=? WHERE keyname='configured_addr' AND value!=?1
}
// Get user-configured server deletion
let delete_server_after = context.get_config_delete_server_after().await?;
if !received_msg.msg_ids.is_empty() {
let target = if received_msg.needs_delete_job || delete_server_after == Some(0) {
let target = if received_msg.needs_delete_job {
Some("".to_string())
} else {
None
@@ -3797,13 +3790,17 @@ async fn apply_out_broadcast_changes(
} else if from_id == ContactId::SELF
&& let Some(removed_id) = removed_id
{
chat::remove_from_chat_contacts_table_without_trace(context, chat.id, removed_id)
.await?;
better_msg.get_or_insert(
stock_str::msg_del_member_local(context, removed_id, ContactId::SELF).await,
);
added_removed_id = Some(removed_id);
if chat::remove_from_chat_contacts_table_without_trace(context, chat.id, removed_id)
.await?
{
better_msg.get_or_insert(
stock_str::msg_del_member_local(context, removed_id, ContactId::SELF).await,
);
added_removed_id = Some(removed_id);
} else {
info!(context, "No-op broadcast member removal message (TRASH).");
better_msg = Some("".to_string());
}
}
}
@@ -3877,17 +3874,20 @@ async fn apply_in_broadcast_changes(
}
chat::delete_broadcast_secret(context, chat.id).await?;
if from_id == ContactId::SELF {
let removed =
chat::remove_from_chat_contacts_table_without_trace(context, chat.id, ContactId::SELF)
.await?;
if !removed {
info!(context, "No-op broadcast SELF-removal message (TRASH).");
better_msg = Some("".to_string());
} else if from_id == ContactId::SELF {
better_msg.get_or_insert(stock_str::msg_you_left_broadcast(context));
} else {
better_msg.get_or_insert(
stock_str::msg_del_member_local(context, ContactId::SELF, from_id).await,
);
}
chat::remove_from_chat_contacts_table_without_trace(context, chat.id, ContactId::SELF)
.await?;
send_event_chat_modified = true;
send_event_chat_modified |= removed;
} else if !chat.is_self_in_chat(context).await? {
chat::add_to_chat_contacts_table(
context,
+1 -6
View File
@@ -2033,15 +2033,10 @@ async fn test_no_smtp_job_for_self_chat() -> Result<()> {
assert!(bob.pop_sent_msg_opt(Duration::ZERO).await.is_none());
bob.set_config_bool(Config::BccSelf, true).await?;
bob.set_config(Config::DeleteServerAfter, Some("1")).await?;
let mut msg = Message::new_text("Happy birthday to me".to_string());
chat::send_msg(bob, chat_id, &mut msg).await?;
assert!(bob.pop_sent_msg_opt(Duration::ZERO).await.is_none());
bob.set_config(Config::DeleteServerAfter, None).await?;
let mut msg = Message::new_text("Happy birthday to me".to_string());
chat::send_msg(bob, chat_id, &mut msg).await?;
assert!(bob.pop_sent_msg_opt(Duration::ZERO).await.is_some());
Ok(())
}
+1 -9
View File
@@ -15,7 +15,7 @@ use crate::config::Config;
use crate::contact::{ContactId, RecentlySeenLoop};
use crate::context::Context;
use crate::download::{download_known_post_messages_without_pre_message, download_msgs};
use crate::ephemeral::{self, delete_expired_imap_messages};
use crate::ephemeral;
use crate::events::EventType;
use crate::imap::{Imap, session::Session};
use crate::location;
@@ -484,14 +484,6 @@ async fn fetch_idle(ctx: &Context, connection: &mut Imap, mut session: Session)
.await
.context("fetch_move_delete")?;
// Mark expired messages for deletion. Marked messages will be deleted from the server
// on the next iteration of `fetch_move_delete`. `delete_expired_imap_messages` is not
// called right before `fetch_move_delete` because it is not well optimized and would
// otherwise slow down message fetching.
delete_expired_imap_messages(ctx)
.await
.context("delete_expired_imap_messages")?;
download_known_post_messages_without_pre_message(ctx, &mut session).await?;
download_msgs(ctx, &mut session)
.await
+15 -19
View File
@@ -699,26 +699,22 @@ pub(crate) async fn add_self_recipients(
recipients: &mut Vec<String>,
encrypted: bool,
) -> Result<()> {
// Previous versions of Delta Chat did not send BCC self
// if DeleteServerAfter was set to immediately delete messages
// from the server. This is not the case anymore
// because BCC-self messages are also used to detect
// that message was sent if SMTP server is slow to respond
// and connection is frequently lost
// before receiving status line. NB: This is not a problem for chatmail servers, so `BccSelf`
// disabled by default is fine.
if context.get_config_delete_server_after().await? != Some(0) || !recipients.is_empty() {
// Avoid sending unencrypted messages to all transports, chatmail relays won't accept
// them. Normally the user should have a non-chatmail primary transport to send unencrypted
// messages.
if encrypted {
for addr in context.get_published_secondary_self_addrs().await? {
recipients.push(addr);
}
// Avoid sending unencrypted messages to all transports, chatmail relays won't accept
// them. Normally the user should have a non-chatmail primary transport to send unencrypted
// messages.
if encrypted {
for addr in context.get_published_secondary_self_addrs().await? {
recipients.push(addr);
}
// `from` must be the last addr, see `receive_imf_inner()` why.
let from = context.get_primary_self_addr().await?;
recipients.push(from);
}
// `from` must be the last addr
// because `receive_imf_inner()` marks the message as 'delivered'
// if it arrives to the self-server via `bcc_self`.
// This helps with marking messages as delivered
// if the server is slow and we never get an `OK` response
// before the connection times out.
let from = context.get_primary_self_addr().await?;
recipients.push(from);
Ok(())
}
+1 -1
View File
@@ -2346,7 +2346,7 @@ ALTER TABLE contacts ADD COLUMN name_normalized TEXT;
transaction.execute(
"UPDATE transports
SET entered_param=json_set(entered_param, '$.imap.folder', ?1),
configured_param=json_set(configured_param', '$.imap_folder', ?1)",
configured_param=json_set(configured_param, '$.imap_folder', ?1)",
(mvbox_folder,),
)?;
}
+21 -5
View File
@@ -18,7 +18,7 @@ use crate::config::Config;
use crate::constants::{Chattype, DC_VERSION_STR};
use crate::contact::{Contact, ContactId, Origin, import_vcard, mark_contact_id_as_verified};
use crate::context::Context;
use crate::key::load_self_public_keyring;
use crate::key::{DcKey, load_self_public_key};
use crate::log::LogExt;
use crate::message::{Message, Viewtype};
use crate::securejoin::QrInvite;
@@ -33,7 +33,14 @@ const MESSAGE_STATS_UPDATE_INTERVAL_SECONDS: i64 = 4 * 60; // 4 minutes (less th
#[derive(Serialize)]
struct Statistics {
core_version: String,
number_of_transports: usize,
key_create_timestamps: Vec<u32>,
number_of_keys: u32,
/// OpenPGP version of the key.
key_version: u8,
key_algorithm: String,
/// Size of the public key in bytes (encoded in binary, not base64).
pubkey_size: usize,
stats_id: String,
is_chatmail: bool,
contact_stats: Vec<ContactStat>,
@@ -345,11 +352,15 @@ async fn get_stats(context: &Context) -> Result<String> {
.get_config_u32(Config::StatsLastOldContactId)
.await?;
let key_create_timestamps: Vec<u32> = load_self_public_keyring(context)
let self_public_key = load_self_public_key(context).await?;
// `key_create_timestamps` is a `Vec` for historical reasons,
// support for using multiple keys is being phased out.
let key_create_timestamps: Vec<u32> = vec![self_public_key.created_at().as_secs()];
let number_of_keys: u32 = context
.sql
.query_get_value("SELECT COUNT(*) FROM keypairs", ())
.await?
.iter()
.map(|k| k.created_at().as_secs())
.collect();
.unwrap_or(0);
let sending_enabled_timestamps =
get_timestamps(context, "stats_sending_enabled_events").await?;
@@ -358,7 +369,12 @@ async fn get_stats(context: &Context) -> Result<String> {
let stats = Statistics {
core_version: DC_VERSION_STR.to_string(),
number_of_transports: context.count_transports().await?,
key_create_timestamps,
number_of_keys,
key_version: self_public_key.primary_key.version().into(),
key_algorithm: format!("{:?}", self_public_key.algorithm()),
pubkey_size: DcKey::to_bytes(&self_public_key).len(),
stats_id: stats_id(context).await?,
is_chatmail: context.is_chatmail().await?,
contact_stats: get_contact_stats(context, last_old_contact).await?,
+30
View File
@@ -595,3 +595,33 @@ async fn test_stats_enable_disable_timestamps() -> Result<()> {
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_cryptography_stats() -> Result<()> {
let alice = &TestContext::new_alice().await;
let stats = get_stats(alice).await.unwrap();
let stats: serde_json::Value = serde_json::from_str(&stats)?;
let number_of_transports: u64 = stats.get("number_of_transports").unwrap().as_u64().unwrap();
assert_eq!(number_of_transports, 1);
let key_version = stats.get("key_version").unwrap().as_u64().unwrap();
// Alice's key is v4
assert_eq!(key_version, 4);
let key_algorithm = stats.get("key_algorithm").unwrap().as_str().unwrap();
assert_eq!(key_algorithm, "EdDSALegacy");
let pubkey_size = stats.get("pubkey_size").unwrap().as_u64().unwrap();
assert_eq!(pubkey_size, 583);
crate::transport::add_pseudo_transport(alice, "alice@ten.testrun.org").await?;
let stats = get_stats(alice).await.unwrap();
let stats: serde_json::Value = serde_json::from_str(&stats)?;
let number_of_transports: u64 = stats.get("number_of_transports").unwrap().as_u64().unwrap();
assert_eq!(number_of_transports, 2);
Ok(())
}
+17 -13
View File
@@ -38,7 +38,7 @@ use crate::events::{Event, EventEmitter, EventType, Events};
use crate::key::{self, DcKey, self_fingerprint};
use crate::log::warn;
use crate::login_param::EnteredLoginParam;
use crate::message::{Message, MessageState, MsgId, update_msg_state};
use crate::message::{Message, MessageState, MsgId};
use crate::mimeparser::{MimeMessage, SystemMessage};
use crate::pgp::SeipdVersion;
use crate::receive_imf::{ReceivedMsg, receive_imf};
@@ -692,10 +692,11 @@ ORDER BY id"
if !msg_has_pending_smtp_job(self, msg_id)
.await
.expect("Failed to check for more jobs")
{
update_msg_state(&self.ctx, msg_id, MessageState::OutDelivered)
&& msg_id
.set_delivered(self)
.await
.expect("failed to update message state");
.expect("MsgId::set_delivered")
{
self.sql
.execute(
"UPDATE msgs SET timestamp_sent=? WHERE id=?",
@@ -772,16 +773,19 @@ ORDER BY id"
.execute("DELETE FROM smtp WHERE msg_id=?", (msg_id,))
.await
.expect("Delete smtp jobs");
update_msg_state(&self.ctx, msg_id, MessageState::OutDelivered)
if msg_id
.set_delivered(self)
.await
.expect("Update message state");
self.sql
.execute(
"UPDATE msgs SET timestamp_sent=? WHERE id=?",
(time(), msg_id),
)
.await
.expect("Update timestamp_sent");
.expect("MsgId::set_delivered")
{
self.sql
.execute(
"UPDATE msgs SET timestamp_sent=? WHERE id=?",
(time(), msg_id),
)
.await
.expect("Update timestamp_sent");
}
sent_msgs
}
+39
View File
@@ -565,6 +565,45 @@ async fn test_webxdc_updates_in_post_message_after_pre_message() -> Result<()> {
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_webxdc_updates_in_post_message_after_deleted_pre_message() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
let bob = &tcm.bob().await;
let alice_chat_id = alice.create_chat_id(bob).await;
let big_webxdc_app = big_webxdc_app().await?;
let mut alice_instance = Message::new(Viewtype::Webxdc);
alice_instance.set_file_from_bytes(alice, "test.xdc", &big_webxdc_app, None)?;
alice_instance.set_text("Test".to_string());
alice_chat_id
.set_draft(alice, Some(&mut alice_instance))
.await?;
alice
.send_webxdc_status_update(alice_instance.id, r#"{"payload":42, "info":"i"}"#)
.await?;
send_msg(alice, alice_chat_id, &mut alice_instance).await?;
let post_message = alice.pop_sent_msg().await;
let pre_message = alice.pop_sent_msg().await;
let bob_instance = bob.recv_msg(&pre_message).await;
assert_eq!(bob_instance.download_state, DownloadState::Available);
delete_msgs(bob, &[bob_instance.id]).await?;
bob.recv_msg_trash(&post_message).await;
// Deleted message stays trashed because of a tombstone.
assert!(
Message::load_from_db_optional(bob, bob_instance.id)
.await?
.is_none()
);
Ok(())
}
/// Tests receiving of a large webxdc post-message with updates attached
/// to the the .xdc post-message when pre-message arrives later.
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
+6 -1
View File
@@ -41,6 +41,7 @@ async fn test_parse_receive_headers_integration() {
let raw = include_bytes!("../../test-data/message/mail_with_cc.txt");
let expected = r"State: Fresh
Database ID: X
Message-ID: 2dfdbde7@example.org
Hop: From: localhost; By: hq5.merlinux.eu; Date: Sat, 14 Sep 2019 17:00:22 +0000
@@ -50,6 +51,7 @@ Hop: From: hq5.merlinux.eu; By: hq5.merlinux.eu; Date: Sat, 14 Sep 2019 17:00:25
let raw = include_bytes!("../../test-data/message/encrypted_with_received_headers.eml");
let expected = "State: Fresh, Encrypted
Database ID: X
Message-ID: Mr.adQpEwndXLH.LPDdlFVJ7wG@example.net
Hop: From: [127.0.0.1]; By: mail.example.org; Date: Mon, 27 Dec 2021 11:21:21 +0000
@@ -71,7 +73,10 @@ async fn check_parse_receive_headers_integration(raw: &[u8], expected: &str) {
// received time that depends on the test time which makes it impossible to
// compare with a static string
let capped_result = &msg_info[msg_info.find("State").unwrap()..];
assert_eq!(expected, capped_result);
assert_eq!(
expected.replace("\nDatabase ID: X", &format!("\nDatabase ID: {msg_id}")),
capped_result
);
}
#[test]
+5 -1
View File
@@ -252,7 +252,11 @@ impl fmt::Display for ConfiguredLoginParam {
write!(f, "{imap}")?;
first = false;
}
write!(f, "] smtp:[")?;
write!(f, "]")?;
if let Some(folder) = &self.imap_folder {
write!(f, " folder:{folder:?}")?;
}
write!(f, " smtp:[")?;
let mut first = true;
for smtp in &self.smtp {
if !first {
+10 -2
View File
@@ -34,7 +34,7 @@ async fn test_save_load_login_param() -> Result<()> {
},
user: "alice".to_string(),
}],
imap_folder: None,
imap_folder: Some("Folder".to_string()),
imap_user: "".to_string(),
imap_password: "foo".to_string(),
smtp: vec![ConfiguredServerLoginParam {
@@ -56,7 +56,7 @@ async fn test_save_load_login_param() -> Result<()> {
.clone()
.save_to_transports_table(&t, &EnteredLoginParam::default(), time())
.await?;
let expected_param = r#"{"addr":"alice@example.org","imap":[{"connection":{"host":"imap.example.com","port":123,"security":"Starttls"},"user":"alice"}],"imap_user":"","imap_password":"foo","smtp":[{"connection":{"host":"smtp.example.com","port":456,"security":"Tls"},"user":"alice@example.org"}],"smtp_user":"","smtp_password":"bar","provider_id":null,"certificate_checks":"Strict","oauth2":false}"#;
let expected_param = r#"{"addr":"alice@example.org","imap":[{"connection":{"host":"imap.example.com","port":123,"security":"Starttls"},"user":"alice"}],"imap_folder":"Folder","imap_user":"","imap_password":"foo","smtp":[{"connection":{"host":"smtp.example.com","port":456,"security":"Tls"},"user":"alice@example.org"}],"smtp_user":"","smtp_password":"bar","provider_id":null,"certificate_checks":"Strict","oauth2":false}"#;
assert_eq!(
t.sql
.query_get_value::<String>("SELECT configured_param FROM transports", ())
@@ -68,6 +68,14 @@ async fn test_save_load_login_param() -> Result<()> {
let (_transport_id, loaded) = ConfiguredLoginParam::load(&t).await?.unwrap();
assert_eq!(param, loaded);
let formatted = format!(" {loaded}");
assert!(formatted.contains(" ***@example.org"));
assert!(formatted.contains(" imap:[imap.example.com:123:starttls]"));
assert!(formatted.contains(" folder:\"Folder\""));
assert!(formatted.contains(" smtp:[smtp.example.com:456:tls]"));
assert!(formatted.contains(" provider:none"));
assert!(formatted.contains(" cert_strict"));
// Legacy ConfiguredImapCertificateChecks config is ignored
t.set_config(Config::ConfiguredImapCertificateChecks, Some("999"))
.await?;
+63
View File
@@ -0,0 +1,63 @@
Content-Type: multipart/encrypted; protocol="application/pgp-encrypted";
boundary="18b255ee53144064_9da6abff99cb33df_3854fea559a18923"
MIME-Version: 1.0
From: <alice@example.org>
To: "hidden-recipients": ;
Subject: [...]
Date: Fri, 22 May 2026 22:48:16 +0000
Message-ID: <3a5ae0d2-be4b-4463-8011-ab4a354ee690@localhost>
Chat-Version: 1.0
--18b255ee53144064_9da6abff99cb33df_3854fea559a18923
Content-Type: application/pgp-encrypted; charset="utf-8"
Content-Transfer-Encoding: 7bit
Version: 1
--18b255ee53144064_9da6abff99cb33df_3854fea559a18923
Content-Type: application/octet-stream; charset="utf-8"
Content-Transfer-Encoding: 7bit
-----BEGIN PGP MESSAGE-----
wUcGABIBB0BNBleU/G9GwCpizow72+5GANtiX+F7y/gI45x7MJLpFiBi+7OqwdFk
dmINorerXmnk1jR7TEppDYhjCBRWaCXHJsHARQYAAQf/Zj43ZUA3rtGS6Nq9vUcc
0L6knU0ukLc3KGTExQXu4ktiPT0RIjQ11wtSGlS75Bj7tu3syDf2/oQXi7eUh35T
IULpBQ5Pnpk6h26GX9WbRYR5zmBNnlyppqJcaYueQzPX1gvybJ4cwfOmnMdUqzFr
D9BR/YeyrBETsGKR+UfH167xgGY8d0ev8Tt4X01muiyrwLVqfoGCRUwjQFUY+ClY
9yuCrNDnu8fHY/WRTKXoDkA8Qn9/6TdJM7G41wOaJYbWn4R8WqjqoQNdUYwL2cBg
Pj1SNgaTaLMMH8YEtlRUN3JHiiqaBVIJIKBduOXnVEovf8FVXlmqX8DARLvqFCk2
LdLEjQIHAgdxw8asmmKOpatAEwFHuVwkWe/ZbRhu3D1V38BzZEuSLHzQ/nWbWWVP
GRmcZPmpusvRGX5W4zC6ybr5jePkW1dEqpgFFeoywz5REQxnl2QDPMHbWRFmC+q5
FSA/zacYGIKmU761xWc1raaosN4uzf8ujJA8H+TQT4AKu4K2nzSMFpUl05sNToGN
FdCNAvsnM9/PAraE3pju6YK+l4Woo/MLVEkrDEcLYEC/TrbNCJ6cnx/3uU+9gZyp
ntXzwOJIIQ5el2qcSQuzec931Lr4UuX4k0FUKeLS24EiYid4K59iPHgKJn6E7grb
34bxbQXkIMug0nTBKuAOTv7MFF4/rzn5deAUXpKty5zYLjqGApo4o5nnEvY9inXP
3489C77lCTLuXgkASrYhqTZcFT6ewaLNJxKRQgLB/V8BY4DrckA/If31z57EK2Ju
5kXpUfZKw/qjnHELLLSmYqx0Sd6w37d0x4LxHIjnpBEmsjWagM9fhiBJyaitNfCz
hBEUAQoO2DlqrZMwAhKxADH8WUbXOhgXcj68NM185A+/UzMbbdV0cmLKynH3tlLX
d16ZyriiRcllSuISXYLTsUwlQeA0zIyuYItzk7bGkw/2k4AmxFAFZjZPvaHOrt98
BapEkxYp8wyhXdloknJ687E5Oa4UTsMKzRTrXRiPhVNstYcYnde4fZs0cUX0JJqE
bZrR+T8biflfGWidc0D0cSd2rx6RryvykFBzvZJH4zqNaHap3w5F0miZWfrePQ8Y
ljPCb162x3SAHc5RzhHH7tq4YatWlnncSAuGkZA7XMqBgFbIRdY2z+Hv5AQLj563
OHD0Dc0pXUZYcHVdJnY4svuiekjEYLaGzjklULBsQo4KQXb8i3SufS8JUwypwAT1
BzMWYQrT6cv3IQHtF9Yys6Z3ngYs9UoCIyqPHYFZYV8Y64injpBFtiY6kd0zvm/3
5D5c0DcmI6kq0LNNNzZg5F0oMoDvyny1pI5/IGxS1H3OyBNJlNZkftMadyvDydY7
khCjWzB9mTStrmeYVqJof/NyfMWJMa0NtJrqW+730FYnld95cGzf3gnRsTYd/CgD
vXVcj5F2/fxsDUekmMMAYBB3DCBv0yCJJfqaTgQKcwngOL0+kuGVnjT8qfjf0FQR
yzImEPJFTQFe+mpRALBoMSDRpzja6ZkU/AIOTT/xy1ZkfpLU/TpNk7XS6InsIjdH
wBzfPvDUV7hHWwgILE+DTlebZVEuVBuoKTM72paMomYzskTkCGnIkTxioA1SyJxJ
eOsSlNC30Le6rcbvz+kl+H7cW2RGZuCUub/UtCMlcdjlzEVAjwXgQZgys2mAwLPx
n61I6rO+eEphXr5F/S6/0NwomR/02FY5AmqeViKC0G6/bDJOO374vU3bix28xqeB
yYYeO+ZymhjqA7tLTEjrkrCh0vo1WoKSAWbhVqM9YUzAeusqyndZ2uLbTgCEayER
MviI27DrXTDpExV9qLyELpo0z18kwoFScJ5ULXDN4lpwfsCeg3ThvBJD86V+4faT
4neKUwzasjYvuzKqSRV4j89I5Bs1g8ERaQr3VZ4BnyjKxQ91eLsMDD+LMx2QW2lB
HPUTWc4qmSAItogwZRxGUj+dASjcuGwqVhabUxWpxPsVGdOp6D3au1M3k7EuvLLa
NmnDDyNRSU8d7Mc/Tp9swfPKd2ItI4cREXTpyu53ayzSD5/7LaDtpHUASdaDLgDv
2evlUTBRk2a8+trbhE0oy8zzRdbgiNrwcegGM+y5BYCdV5Gooerk4g8pnULtKT1R
VgWMKvXvFduvQsfXW+PKChw=
=EBeW
-----END PGP MESSAGE-----
--18b255ee53144064_9da6abff99cb33df_3854fea559a18923--