Compare commits

..
Author SHA1 Message Date
Hocuri fdcfb20ab0 test: In test_qr_scan_updates_new_relay_address(), test that sending a message actually works 2026-06-22 13:20:35 +02:00
holger krekel a9d19bc020 fix: use new QR-contained addresses when alice and bob have dropped their singular relay and added a new one and then perform a securejoin 2026-06-22 04:24:03 +02:00
link2xt 2aba6aa339 refactor: make crate::pgp::symm_encrypt_message non-async 2026-06-20 00:36:05 +00:00
link2xt 93a6f35ef7 refactor: remove the ability to send messages with non-standard header protection 2026-06-20 00:09:48 +00:00
iequidoo cd38aae4d6 doc: create_group_chat_unencrypted() may lead to chat split on the first device 2026-06-18 17:36:02 -03:00
link2xt 3f371b3895 refactor: remove commented out RenderedEmail.envelope 2026-06-18 17:59:56 +00:00
Jagoda Estera Ślązak 8a1d97d925 fix: Hide synced chat if we only know its visibility (#8343)
Prevents showing "empty" chat on other devices,
when pinning a chat that did not complete securejoin yet.

Fixes: #7713

Signed-off-by: Jagoda Ślązak <jslazak@jslazak.com>
2026-06-18 11:42:28 +00:00
iequidoo 8f51c7b9d5 feat: Remove imap::Session::sync_seen_flags() (#7742) 2026-06-17 21:02:27 -03:00
iequidoo fd4bf98776 feat: Add silent group changes messages as InNoticed, not InSeen
This way they also can be processed by `markseen_msgs()` resulting in self-MDNs which improves
multi-device synchronization. MDNs to contacts won't really be sent because silent group changes
messages are self-removal messages which don't request MDNs.
2026-06-17 21:02:27 -03:00
iequidoo 81dc56877c fix: Make truncated edited messages have HTML for receivers (#8249)
Otherwise it's not possible to see the full message text if it's too long in a received edit. On the
sender side let it still be text so that the message can be edited again.
2026-06-17 20:39:54 -03:00
iequidoo 58b3f0e89b fix: Make pre-messages w/o text want MDNs (#8004)
Maybe it's even good that MDNs will be sent for pre-messages only having placeholder text with the
viewtype and size, at least this way we notify the contact that we've noticed the message. Anyway,
with adding message previews the problem will be solved for the corresponding viewtypes.
2026-06-17 00:02:07 -03:00
link2xt b860583f8d fix: request MDNs for resent channel messages 2026-06-16 14:49:49 +00:00
Hocuri d39b79f6fc feat: Implement support for populating and maintaining a list of default relays (#8341) 2026-06-16 14:02:27 +02:00
WofWca ab03fe3040 feat: add MsgReadCountChanged event
Useful for updating the read count on channel messages live.

This supersedes https://github.com/chatmail/core/pull/7979.
Unlike that MR, this one is not breaking.

Needed for https://github.com/deltachat/deltachat-desktop/issues/5220.
2026-06-16 11:38:07 +04:00
dependabot[bot] 246376259e chore(deps): bump taiki-e/install-action from 2.81.1 to 2.81.8
Bumps [taiki-e/install-action](https://github.com/taiki-e/install-action) from 2.81.1 to 2.81.8.
- [Release notes](https://github.com/taiki-e/install-action/releases)
- [Changelog](https://github.com/taiki-e/install-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/taiki-e/install-action/compare/e49978b799e49ff429d162b7a30601a569ab6538...0631aa6515c7d545823c67cfae7ef4fc7f490154)

---
updated-dependencies:
- dependency-name: taiki-e/install-action
  dependency-version: 2.81.8
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-06-16 00:11:44 +00:00
link2xt 44bdd5ef0c chore: bump version to 2.54.0-dev 2026-06-15 21:38:38 +02:00
link2xt d844f29c68 chore(release): prepare for 2.53.0 2026-06-15 21:26:11 +02:00
link2xt 07d6b89b85 chore: bump version to 2.53.0-dev
Fixup for 93bf3d6ebb
which should have bumped the version to 2.53.0-dev
but bumped to 2.52.0-dev.
2026-06-15 18:29:14 +02:00
link2xt 2bcfbe99fa fix: do not abort IMAP connection if setting the push token fails
With madmail 2.2.2 from https://github.com/themadorg/madmail/releases/download/v2.2.2/madmail-linux-amd64
using IMAP command SETMETADATA to set /private/devicetoken
returns error "A0003 BAD command not supported".

With this change connection is not aborted anymore
and the error is logged like this:
src/scheduler.rs:458: Transport 1: Failed to register push token: SETMETADATA command failed: bad response: code: None, info: Some("command not supported").

madmail 2.2.2 however fails to select the INBOX afterwards
and the following command `A0004 SELECT "INBOX"`
results in `* BAD command not supported` response.

Fixes https://github.com/chatmail/core/issues/8334
2026-06-15 13:45:26 +00:00
link2xt 3e10cf2c07 feat: do not log the recipient list for sent messages 2026-06-15 13:41:40 +00:00
link2xt c7a399e3ca refactor(flake.nix): Use hostPlatform.rust.rustcTarget instead of hardcoding it 2026-06-15 13:41:16 +00:00
iequidoo 14b9577c39 feat: MsgId::get_html: Make only one db query
Merge `message::get_mime_headers()` into `MsgId::get_html()` for that, it's unlikely that it will be
used elsewhere.
2026-06-14 03:25:03 -03:00
link2xt f961a49906 test: remove timeout from pop_sent_msg_ex()
Arbitrary timeouts often result in flaky tests,
especially if CI runners may be paused.
In this case however timeout was not used by any tests
and only slowed down the tests in cases where
no message is expected but non-zero timeout is set.
2026-06-13 23:14:13 +00:00
link2xt b98b32317c test: fixup the tests after removing timesmearing 2026-06-13 22:59:03 +00:00
iequidoo b94c177997 fix: Ignore SecureJoin messages from blocked contacts (#8295)
- Ignore "vc-request-with-auth" if the sender is blocked.
- Ignore SecureJoin messages on an observing device if the joiner is blocked. Even if it's a
  "vc-contact-confirm" message, it might be issued by another device before the joiner was blocked
  on the observing device. Still, to avoid membership inconsistency on devices, don't ignore
  "vg-member-added".
- Ignore "vc-request-pubkey" if the sender is blocked by address. We don't know sender's key yet, so
  we should handle the sender as an address contact at this point.

The sender can generate a new key and still join, but it's at least an extra work and also, in case
of a group, other members will see that there's a new unknown member and be more careful.
2026-06-13 19:35:51 -03:00
iequidoo 15059ad8d7 test: exec_securejoin_qr_multi_device(): Make inviter devices receive each other messages
Otherwise we're testing an abnormal scenario when `Config::BccSelf` ("multi-device") is disabled.
2026-06-13 19:35:51 -03:00
link2xt 293b524373 refactor: remove timesmearing 2026-06-13 15:43:47 +00:00
link2xt edb8a87cf8 test: print multiline chat descriptions with debug formatter 2026-06-13 15:43:47 +00:00
link2xt 9f2de665fc build(nix): switch to the "master" branch for naersk
PR <https://github.com/nix-community/naersk/pull/391> has been merged.
2026-06-13 14:21:31 +00:00
72374 906e99a6f7 feat: Make quality of images sent in chats more consistent
Currently, the resolution of a resized image that was sent in a chat,
depends on the aspect-ratio.

Assuming the `balanced`-quality-setting is used,
a square image, that is larger than the limits for resolution and file-size,
will be resized to 1280x1280 (1,638,400 pixels),
an image with an aspect-ratio of 16:9,
will be resized to 1280x720 (921,600 pixels),
and if the aspect-ratio is 32:9, to 1280x360 (460,800 pixels).

This change makes it so, that the number of pixels,
in images with different aspect-ratios, will be similar.
2026-06-13 10:07:56 +02:00
72374 b215bc6d16 refactor: Move the definition of the target_wh-variable
to the only part of the function where it is used.
2026-06-13 10:07:56 +02:00
holger krekel 6cd5b21a26 fix: don't send or process webxdc status updates in pre-messages
it makes no sense to send or receive status updates in pre-messages for large webxdc attachments because they can't be processed anyway.
2026-06-12 17:01:46 +02:00
dependabot[bot] 87c1fb2118 chore(deps): bump EmbarkStudios/cargo-deny-action from 2.0.19 to 2.0.20
Bumps [EmbarkStudios/cargo-deny-action](https://github.com/embarkstudios/cargo-deny-action) from 2.0.19 to 2.0.20.
- [Release notes](https://github.com/embarkstudios/cargo-deny-action/releases)
- [Commits](https://github.com/embarkstudios/cargo-deny-action/compare/a531616d8ce3b9177443e48a1159bc945a099823...bb137d7af7e4fb67e5f82a49c4fce4fad40782fe)

---
updated-dependencies:
- dependency-name: EmbarkStudios/cargo-deny-action
  dependency-version: 2.0.20
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-06-12 09:58:44 +00:00
link2xt e8b94781a5 fix: do not trash pre-messages without text but with a webxdc update
Webxdc updates go to pre-message and when the message has no text,
the whole pre-message is trashed if it has a webxdc update.
The solution is not to trash pre-message in this case so preview is displayed.
2026-06-11 15:10:14 +00:00
dependabot[bot] 07231f28ae chore(deps): bump taiki-e/install-action from 2.79.10 to 2.81.1
Bumps [taiki-e/install-action](https://github.com/taiki-e/install-action) from 2.79.10 to 2.81.1.
- [Release notes](https://github.com/taiki-e/install-action/releases)
- [Changelog](https://github.com/taiki-e/install-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/taiki-e/install-action/compare/60ae4ce63c7aeb6e96d7f572c1ec7fafbb17ca80...e49978b799e49ff429d162b7a30601a569ab6538)

---
updated-dependencies:
- dependency-name: taiki-e/install-action
  dependency-version: 2.81.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-06-11 14:31:02 +00:00
iequidoo 64f0d2352c docs(STYLE.md): Require to list columns explicitly in INSERT statements 2026-06-09 12:34:38 -03:00
link2xt 93bf3d6ebb chore: bump version to 2.52.0-dev 2026-06-09 02:18:40 +02:00
link2xt f05336f793 chore(release): prepare for 2.52.0 2026-06-09 00:40:07 +02:00
link2xt 8df028e9a8 build(nix): fix windows cross-compilation 2026-06-08 17:17:10 +00:00
link2xt 40309ce857 chore: add exception for unmaintained proc-macro-error2 to deny.toml 2026-06-08 17:16:10 +00:00
69 changed files with 935 additions and 932 deletions
+2 -2
View File
@@ -62,7 +62,7 @@ jobs:
with:
show-progress: false
persist-credentials: false
- uses: EmbarkStudios/cargo-deny-action@a531616d8ce3b9177443e48a1159bc945a099823
- uses: EmbarkStudios/cargo-deny-action@bb137d7af7e4fb67e5f82a49c4fce4fad40782fe
with:
arguments: --workspace --all-features --locked
command: check
@@ -146,7 +146,7 @@ jobs:
cache-bin: false
- name: Install nextest
uses: taiki-e/install-action@60ae4ce63c7aeb6e96d7f572c1ec7fafbb17ca80
uses: taiki-e/install-action@0631aa6515c7d545823c67cfae7ef4fc7f490154
with:
tool: nextest
+77
View File
@@ -1,5 +1,80 @@
# Changelog
## [2.53.0] - 2026-06-15
### Features / Changes
- Make quality of images sent in chats more consistent between images with different aspect ratio.
- `MsgId::get_html`: Make only one db query.
- Do not log the recipient list for sent messages.
### Fixes
- Do not trash pre-messages without text but with a webxdc update.
- Don't send or process webxdc status updates in pre-messages.
- Ignore SecureJoin messages from blocked contacts ([#8295](https://github.com/chatmail/core/pull/8295)).
- Do not abort IMAP connection if setting the push token fails.
### Documentation
- STYLE.md: Require to list columns explicitly in `INSERT` statements.
### Build system
- nix: switch to the "master" branch for naersk.
- flake.nix: Use hostPlatform.rust.rustcTarget instead of hardcoding it.
### Miscellaneous Tasks
- Bump version to 2.52.0-dev.
- deps: bump taiki-e/install-action from 2.79.10 to 2.81.1.
- deps: bump EmbarkStudios/cargo-deny-action from 2.0.19 to 2.0.20.
- Bump version to 2.53.0-dev.
### Refactor
- Move the definition of the `target_wh`-variable.
- Remove timesmearing.
### Tests
- Print multiline chat descriptions with debug formatter.
- `exec_securejoin_qr_multi_device()`: Make inviter devices receive each other messages.
- Fixup the tests after removing timesmearing.
- Remove timeout from `pop_sent_msg_ex()`.
## [2.52.0] - 2026-06-09
### Fixes
- Update the channel title after joining if the QR code included a wrong title ([#8260](https://github.com/chatmail/core/pull/8260)).
- Don't send removal message to contact that hasn't been a chat member ([#8298](https://github.com/chatmail/core/pull/8298)).
### Features / Changes
- Add cryptography-related statistics (`number_of_transports`, `key_version`, `key_algorithm`, `pubkey_size`, `number_of_keys`) ([#8293](https://github.com/chatmail/core/pull/8293), [#8297](https://github.com/chatmail/core/pull/8297)).
- Add IMAP folder to `Context::get_info()` ([#8285](https://github.com/chatmail/core/pull/8285)).
### Miscellaneous Tasks
- Update preloaded DNS cache.
- Use default aws-lc-rs cryptography provider for rustls.
- Add exception for unmaintained proc-macro-error2 to deny.toml.
- cargo: bump `pin-project` from 1.1.11 to 1.1.13.
- cargo: bump `tokio` from 1.52.1 to 1.52.3.
- cargo: bump `log` from 0.4.29 to 0.4.30.
- cargo: bump `serde_json` from 1.0.149 to 1.0.150.
- deps: bump EmbarkStudios/cargo-deny-action from 2.0.18 to 2.0.19.
- deps: bump taiki-e/install-action from 2.79.2 to 2.79.10.
### Build system
- nix: fix windows cross-compilation by adding pthreads includes.
### Refactor
- Remove support for building "source" packages for deltachat-rpc-server.
## [2.51.0] - 2026-05-29
### Features / Changes
@@ -8297,3 +8372,5 @@ https://github.com/chatmail/core/pulls?q=is%3Apr+is%3Aclosed
[2.49.0]: https://github.com/chatmail/core/compare/v2.48.0..v2.49.0
[2.50.0]: https://github.com/chatmail/core/compare/v2.49.0..v2.50.0
[2.51.0]: https://github.com/chatmail/core/compare/v2.50.0..v2.51.0
[2.52.0]: https://github.com/chatmail/core/compare/v2.51.0..v2.52.0
[2.53.0]: https://github.com/chatmail/core/compare/v2.52.0..v2.53.0
Generated
+5 -5
View File
@@ -1350,7 +1350,7 @@ dependencies = [
[[package]]
name = "deltachat"
version = "2.51.0-dev"
version = "2.54.0-dev"
dependencies = [
"anyhow",
"astral-tokio-tar",
@@ -1459,7 +1459,7 @@ dependencies = [
[[package]]
name = "deltachat-jsonrpc"
version = "2.51.0-dev"
version = "2.54.0-dev"
dependencies = [
"anyhow",
"async-channel 2.5.0",
@@ -1480,7 +1480,7 @@ dependencies = [
[[package]]
name = "deltachat-repl"
version = "2.51.0-dev"
version = "2.54.0-dev"
dependencies = [
"anyhow",
"deltachat",
@@ -1496,7 +1496,7 @@ dependencies = [
[[package]]
name = "deltachat-rpc-server"
version = "2.51.0-dev"
version = "2.54.0-dev"
dependencies = [
"anyhow",
"deltachat",
@@ -1525,7 +1525,7 @@ dependencies = [
[[package]]
name = "deltachat_ffi"
version = "2.51.0-dev"
version = "2.54.0-dev"
dependencies = [
"anyhow",
"deltachat",
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "deltachat"
version = "2.51.0-dev"
version = "2.54.0-dev"
edition = "2024"
license = "MPL-2.0"
rust-version = "1.89"
+7
View File
@@ -59,6 +59,13 @@ If column is already declared without `NOT NULL`, use `IFNULL` function to provi
Use `HAVING COUNT(*) > 0` clause
to [prevent aggregate functions such as `MIN` and `MAX` from returning `NULL`](https://stackoverflow.com/questions/66527856/aggregate-functions-max-etc-return-null-instead-of-no-rows).
List columns explicitly in `INSERT` statements:
```
INSERT OR IGNORE INTO download (rfc724_mid, msg_id) VALUES (?,0);
```
Otherwise if a new column with default value is added in a future DB version, an upgraded DB can't
be used with the old code, e.g. after transferring a DB from a device running a newer version.
Don't delete unused columns too early, but maybe after several months/releases, unused columns are
still used by older versions, so deleting them breaks downgrading the core or importing a backup in
an older version. Also don't change the column type, consider adding a new column with another name
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "deltachat_ffi"
version = "2.51.0-dev"
version = "2.54.0-dev"
description = "Deltachat FFI"
edition = "2018"
readme = "README.md"
+9
View File
@@ -6341,6 +6341,15 @@ void dc_event_unref(dc_event_t* event);
*/
#define DC_EVENT_MSG_DELETED 2016
/**
* Like @ref DC_EVENT_MSG_READ, but also fires on subsequent MDNs,
* if there are multiple receivers, i.e. in groups and channels.
*
* @param data1 (int) chat_id
* @param data2 (int) msg_id
*/
#define DC_EVENT_MSG_READ_COUNT_CHANGED 2018
/**
* Chat changed. The name or the image of a chat group was changed or members were added or removed.
+5 -1
View File
@@ -528,6 +528,7 @@ pub unsafe extern "C" fn dc_event_get_id(event: *mut dc_event_t) -> libc::c_int
EventType::MsgFailed { .. } => 2012,
EventType::MsgRead { .. } => 2015,
EventType::MsgDeleted { .. } => 2016,
EventType::MsgReadCountChanged { .. } => 2018,
EventType::ChatModified(_) => 2020,
EventType::ChatEphemeralTimerModified { .. } => 2021,
EventType::ChatDeleted { .. } => 2023,
@@ -602,6 +603,7 @@ pub unsafe extern "C" fn dc_event_get_data1_int(event: *mut dc_event_t) -> libc:
| EventType::MsgFailed { chat_id, .. }
| EventType::MsgRead { chat_id, .. }
| EventType::MsgDeleted { chat_id, .. }
| EventType::MsgReadCountChanged { chat_id, .. }
| EventType::ChatModified(chat_id)
| EventType::ChatEphemeralTimerModified { chat_id, .. }
| EventType::ChatDeleted { chat_id } => chat_id.to_u32() as libc::c_int,
@@ -688,7 +690,8 @@ pub unsafe extern "C" fn dc_event_get_data2_int(event: *mut dc_event_t) -> libc:
| EventType::MsgDelivered { msg_id, .. }
| EventType::MsgFailed { msg_id, .. }
| EventType::MsgRead { msg_id, .. }
| EventType::MsgDeleted { msg_id, .. } => msg_id.to_u32() as libc::c_int,
| EventType::MsgDeleted { msg_id, .. }
| EventType::MsgReadCountChanged { msg_id, .. } => msg_id.to_u32() as libc::c_int,
EventType::SecurejoinInviterProgress { progress, .. }
| EventType::SecurejoinJoinerProgress { progress, .. } => *progress as libc::c_int,
EventType::ChatEphemeralTimerModified { timer, .. } => timer.to_u32() as libc::c_int,
@@ -762,6 +765,7 @@ pub unsafe extern "C" fn dc_event_get_data2_str(event: *mut dc_event_t) -> *mut
| EventType::MsgFailed { .. }
| EventType::MsgRead { .. }
| EventType::MsgDeleted { .. }
| EventType::MsgReadCountChanged { .. }
| EventType::ChatModified(_)
| EventType::ContactsChanged(_)
| EventType::LocationChanged(_)
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "deltachat-jsonrpc"
version = "2.51.0-dev"
version = "2.54.0-dev"
description = "DeltaChat JSON-RPC API"
edition = "2021"
license = "MPL-2.0"
+3 -1
View File
@@ -1081,7 +1081,9 @@ impl CommandApi {
/// Create a new unencrypted group chat.
///
/// Same as [`Self::create_group_chat`], but the chat is unencrypted and can only have
/// address-contacts.
/// address-contacts. NB: Chats with similar names and the same members are merged on other
/// devices, but usually users don't create such chats and look up the existing one instead, so
/// chat split on the first device is acceptable.
async fn create_group_chat_unencrypted(&self, account_id: u32, name: String) -> Result<u32> {
let ctx = self.get_context(account_id).await?;
chat::create_group_unencrypted(&ctx, &name)
+15
View File
@@ -203,6 +203,17 @@ pub enum EventType {
msg_id: u32,
},
/// Like [`EventType::MsgRead`], but also fires on subsequent MDNs,
/// if there are multiple receivers, i.e. in groups and channels.
#[serde(rename_all = "camelCase")]
MsgReadCountChanged {
/// ID of the chat which the message belongs to.
chat_id: u32,
/// ID of the message that was read.
msg_id: u32,
},
/// A single message was deleted.
///
/// This event means that the message will no longer appear in the messagelist.
@@ -546,6 +557,10 @@ impl From<CoreEventType> for EventType {
chat_id: chat_id.to_u32(),
msg_id: msg_id.to_u32(),
},
CoreEventType::MsgReadCountChanged { chat_id, msg_id } => MsgReadCountChanged {
chat_id: chat_id.to_u32(),
msg_id: msg_id.to_u32(),
},
CoreEventType::MsgDeleted { chat_id, msg_id } => MsgDeleted {
chat_id: chat_id.to_u32(),
msg_id: msg_id.to_u32(),
+1 -1
View File
@@ -54,5 +54,5 @@
},
"type": "module",
"types": "dist/deltachat.d.ts",
"version": "2.51.0-dev"
"version": "2.54.0-dev"
}
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "deltachat-repl"
version = "2.51.0-dev"
version = "2.54.0-dev"
license = "MPL-2.0"
edition = "2021"
repository = "https://github.com/chatmail/core"
+1 -1
View File
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "deltachat-rpc-client"
version = "2.51.0-dev"
version = "2.54.0-dev"
license = "MPL-2.0"
description = "Python client for Delta Chat core JSON-RPC interface"
classifiers = [
@@ -54,6 +54,7 @@ class EventType(str, Enum):
MSG_DELIVERED = "MsgDelivered"
MSG_FAILED = "MsgFailed"
MSG_READ = "MsgRead"
MSG_READ_COUNT_CHANGED = "MsgReadCountChanged"
MSG_DELETED = "MsgDeleted"
CHAT_MODIFIED = "ChatModified"
CHAT_DELETED = "ChatDeleted"
+3 -12
View File
@@ -37,19 +37,14 @@ def test_one_account_send_bcc_setting(acfactory, log, direct_imap):
log.section("send out message without bcc to ourselves")
ac1.set_config("bcc_self", "0")
chat = ac1.create_chat(ac2)
self_addr = ac1.get_config("addr")
other_addr = ac2.get_config("addr")
msg_out = chat.send_text("message1")
assert not msg_out.get_snapshot().is_forwarded
# wait for send out (no BCC)
ev = ac1.wait_for_event(EventType.SMTP_MESSAGE_SENT)
ac1.wait_for_event(EventType.SMTP_MESSAGE_SENT)
assert ac1.get_config("bcc_self") == "0"
assert self_addr not in ev.msg
assert other_addr in ev.msg
log.section("ac1: setting bcc_self=1")
ac1.set_config("bcc_self", "1")
@@ -57,20 +52,16 @@ def test_one_account_send_bcc_setting(acfactory, log, direct_imap):
msg_out = chat.send_text("message2")
# wait for send out (BCC)
ev = ac1.wait_for_event(EventType.SMTP_MESSAGE_SENT)
ac1.wait_for_event(EventType.SMTP_MESSAGE_SENT)
assert ac1.get_config("bcc_self") == "1"
# Second client receives only second message, but not the first.
# Second client receives only the second message, but not the first.
ev_msg = ac1_clone.wait_for_event(EventType.MSGS_CHANGED)
assert ac1_clone.get_message_by_id(ev_msg.msg_id).get_snapshot().text == "Messages are end-to-end encrypted."
ev_msg = ac1_clone.wait_for_event(EventType.MSGS_CHANGED)
assert ac1_clone.get_message_by_id(ev_msg.msg_id).get_snapshot().text == msg_out.get_snapshot().text
# now make sure we are sending message to ourselves too
assert self_addr in ev.msg
assert self_addr in ev.msg
# BCC-self messages are marked as seen by the sender device.
while True:
event = ac1.wait_for_event()
@@ -704,3 +704,25 @@ def test_withdraw_securejoin_qr(acfactory):
and "Ignoring RequestWithAuth message because of invalid auth code." in event.msg
):
break
def test_qr_scan_updates_new_relay_address(acfactory):
alice, bob = acfactory.get_online_accounts(2)
bob_alice_chat = bob.secure_join(alice.get_qr_code())
alice.wait_for_securejoin_inviter_success()
bob.wait_for_securejoin_joiner_success()
for ac in [alice, bob]:
old_addr = ac.get_config("configured_addr")
ac.add_transport_from_qr(acfactory.get_account_qr())
ac.set_config("configured_addr", ac.list_transports()[1]["addr"])
ac.delete_transport(old_addr)
bob.secure_join(alice.get_qr_code())
alice.wait_for_securejoin_inviter_success()
bob.wait_for_securejoin_joiner_success()
bob_alice_chat.send_text("hi")
snapshot = alice.wait_for_incoming_msg().get_snapshot()
assert snapshot.text == "hi"
@@ -450,6 +450,35 @@ def test_reaction_seen_on_another_dev(acfactory) -> None:
assert chat_id == alice2_chat_bob.id
def test_2nd_device_events_when_msgs_are_seen(acfactory) -> None:
alice, bob = acfactory.get_online_accounts(2)
alice2 = alice.clone()
alice2.start_io()
# Get an accepted chat, otherwise alice2 won't be notified about the 2nd message.
chat_alice2 = alice2.create_chat(bob)
chat_id_alice2 = chat_alice2.get_basic_snapshot().id
chat_bob_alice = bob.create_chat(alice)
chat_bob_alice.send_text("Hello!")
msg_alice = alice.wait_for_incoming_msg()
assert alice2.wait_for_incoming_msg_event().chat_id == chat_id_alice2
chat_bob_alice.send_text("What's new?")
assert alice2.wait_for_incoming_msg_event().chat_id == chat_id_alice2
chat_alice2 = alice2.get_chat_by_id(chat_id_alice2)
assert chat_alice2.get_fresh_message_count() == 2
msg_alice.mark_seen()
assert alice2.wait_for_msgs_changed_event().chat_id == chat_id_alice2
assert chat_alice2.get_fresh_message_count() == 1
msg_id = alice.wait_for_msgs_changed_event().msg_id
msg = alice.get_message_by_id(msg_id)
msg.mark_seen()
assert alice2.wait_for_event(EventType.MSGS_NOTICED).chat_id == chat_id_alice2
assert chat_alice2.get_fresh_message_count() == 0
def test_is_bot(acfactory) -> None:
"""Test that we can recognize messages submitted by bots."""
alice, bob = acfactory.get_online_accounts(2)
@@ -1099,6 +1128,7 @@ def test_rename_group(acfactory):
bob.wait_for_event(EventType.CHATLIST_ITEM_CHANGED)
for name in ["Baz", "Foo bar", "Xyzzy"]:
time.sleep(1)
alice_group.set_name(name)
bob.wait_for_event(EventType.CHATLIST_ITEM_CHANGED)
bob.wait_for_event(EventType.CHATLIST_ITEM_CHANGED)
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "deltachat-rpc-server"
version = "2.51.0-dev"
version = "2.54.0-dev"
description = "DeltaChat JSON-RPC server"
edition = "2021"
readme = "README.md"
@@ -15,5 +15,5 @@
},
"type": "module",
"types": "index.d.ts",
"version": "2.51.0-dev"
"version": "2.54.0-dev"
}
+5
View File
@@ -18,6 +18,11 @@ ignore = [
# this should be fixed by upgrading to iroh 1.0 once it is released.
"RUSTSEC-2025-0134",
# Unmaintained proc-macro-error2
# Transitive dependency of typescript-type-def 0.5.13.
# <https://rustsec.org/advisories/RUSTSEC-2026-0173>
"RUSTSEC-2026-0173",
# rustls-webpki v0.102.8
# We cannot upgrade to >=0.103.10 because
# it is a transitive dependency of iroh 0.35.0
Generated
+3 -4
View File
@@ -94,16 +94,15 @@
]
},
"locked": {
"lastModified": 1779912356,
"narHash": "sha256-yj5O6vmAj+OfhTQMiUwhmQRP0HAII3BxEI6zuY6h/5k=",
"lastModified": 1780914171,
"narHash": "sha256-NYoa+CvsCgayY356worC9g6QoZkKJmvrj6nVdEA2Lyk=",
"owner": "nix-community",
"repo": "naersk",
"rev": "33eaf5c72a67db15073322d26cd342c443556214",
"rev": "41b6e9efb62fac9c16d5617e0456a715928a2206",
"type": "github"
},
"original": {
"owner": "nix-community",
"ref": "pull/391/head",
"repo": "naersk",
"type": "github"
}
+18 -41
View File
@@ -4,7 +4,7 @@
fenix.url = "github:nix-community/fenix";
fenix.inputs.nixpkgs.follows = "nixpkgs";
flake-utils.url = "github:numtide/flake-utils";
naersk.url = "github:nix-community/naersk/pull/391/head";
naersk.url = "github:nix-community/naersk";
naersk.inputs.nixpkgs.follows = "nixpkgs";
naersk.inputs.fenix.follows = "fenix";
nix-filter.url = "github:numtide/nix-filter";
@@ -66,37 +66,15 @@
];
};
# Map from architecture name to rust targets and nixpkgs targets.
# Map from architecture name to nixpkgs targets.
arch2targets = {
"x86_64-linux" = {
rustTarget = "x86_64-unknown-linux-musl";
crossTarget = "x86_64-unknown-linux-musl";
};
"armv7l-linux" = {
rustTarget = "armv7-unknown-linux-musleabihf";
crossTarget = "armv7l-unknown-linux-musleabihf";
};
"armv6l-linux" = {
rustTarget = "arm-unknown-linux-musleabihf";
crossTarget = "armv6l-unknown-linux-musleabihf";
};
"aarch64-linux" = {
rustTarget = "aarch64-unknown-linux-musl";
crossTarget = "aarch64-unknown-linux-musl";
};
"i686-linux" = {
rustTarget = "i686-unknown-linux-musl";
crossTarget = "i686-unknown-linux-musl";
};
"x86_64-darwin" = {
rustTarget = "x86_64-apple-darwin";
crossTarget = "x86_64-darwin";
};
"aarch64-darwin" = {
rustTarget = "aarch64-apple-darwin";
crossTarget = "aarch64-darwin";
};
"x86_64-linux" = "x86_64-unknown-linux-musl";
"armv7l-linux" = "armv7l-unknown-linux-musleabihf";
"armv6l-linux" = "armv6l-unknown-linux-musleabihf";
"aarch64-linux" = "aarch64-unknown-linux-musl";
"i686-linux" = "i686-unknown-linux-musl";
"x86_64-darwin" = "x86_64-darwin";
"aarch64-darwin" = "aarch64-darwin";
};
cargoLock = {
lockFile = ./Cargo.lock;
@@ -113,10 +91,10 @@
auditable = false; # Avoid cargo-auditable failures.
doCheck = false; # Disable test as it requires network access.
};
pkgsWin64 = pkgs.pkgsCross.mingwW64;
mkWin64RustPackage = packageName:
let
rustTarget = "x86_64-pc-windows-gnu";
pkgsWin64 = pkgs.pkgsCross.mingwW64;
rustTarget = pkgsWin64.stdenv.hostPlatform.rust.rustcTarget;
toolchainWin = fenixPkgs.combine [
fenixPkgs.stable.rustc
fenixPkgs.stable.cargo
@@ -147,6 +125,7 @@
CARGO_BUILD_TARGET = rustTarget;
TARGET_CC = "${pkgsWin64.stdenv.cc}/bin/${pkgsWin64.stdenv.cc.targetPrefix}cc";
CFLAGS_x86_64_pc_windows_gnu = "-I${pkgsWin64.windows.pthreads}/include";
CARGO_BUILD_RUSTFLAGS = [
"-C"
"linker=${TARGET_CC}"
@@ -158,12 +137,10 @@
LD = "${pkgsWin64.stdenv.cc}/bin/${pkgsWin64.stdenv.cc.targetPrefix}cc";
};
pkgsWin32 = pkgs.pkgsCross.mingw32;
mkWin32RustPackage = packageName:
let
rustTarget = "i686-pc-windows-gnu";
in
let
pkgsWin32 = pkgs.pkgsCross.mingw32;
rustTarget = pkgsWin32.stdenv.hostPlatform.rust.rustcTarget;
toolchainWin = fenixPkgs.combine [
fenixPkgs.stable.rustc
fenixPkgs.stable.cargo
@@ -203,6 +180,7 @@
src = pkgs.lib.cleanSource ./.;
nativeBuildInputs = [
pkgs.perl # Needed to build vendored OpenSSL.
pkgs.nasm # aws-lc-sys requires it
];
depsBuildBuild = [
winCC
@@ -215,6 +193,7 @@
CARGO_BUILD_TARGET = rustTarget;
TARGET_CC = "${winCC}/bin/${winCC.targetPrefix}cc";
CFLAGS_i686_pc_windows_gnu = "-I${pkgsWin32.windows.pthreads}/include";
CARGO_BUILD_RUSTFLAGS = [
"-C"
"linker=${TARGET_CC}"
@@ -228,14 +207,12 @@
mkCrossRustPackage = arch: packageName:
let
rustTarget = arch2targets."${arch}".rustTarget;
crossTarget = arch2targets."${arch}".crossTarget;
crossTarget = arch2targets."${arch}";
pkgsCross = import nixpkgs {
system = system;
crossSystem.config = crossTarget;
};
in
let
rustTarget = pkgsCross.stdenv.hostPlatform.rust.rustcTarget;
toolchain = fenixPkgs.combine [
fenixPkgs.stable.rustc
fenixPkgs.stable.cargo
+1 -1
View File
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "deltachat"
version = "2.51.0-dev"
version = "2.54.0-dev"
license = "MPL-2.0"
description = "Python bindings for the Delta Chat Core library using CFFI against the Rust-implemented libdeltachat"
readme = "README.rst"
+1 -1
View File
@@ -1 +1 @@
2026-05-29
2026-06-15
+33 -9
View File
@@ -1,6 +1,6 @@
//! # Blob directory management.
use std::cmp::max;
use std::cmp::{max, min};
use std::io::{Cursor, Seek};
use std::iter::FusedIterator;
use std::mem;
@@ -12,7 +12,7 @@ use futures::StreamExt;
use image::ImageReader;
use image::{DynamicImage, GenericImage, GenericImageView, ImageFormat, Pixel, Rgba};
use image::{codecs::jpeg::JpegEncoder, metadata::Orientation};
use num_traits::FromPrimitive;
use num_traits::{FromPrimitive, cast};
use tokio::{fs, task};
use tokio_stream::wrappers::ReadDirStream;
@@ -382,14 +382,9 @@ impl<'a> BlobObject<'a> {
}
img.apply_orientation(orientation);
// max_wh is the maximum image width and height, i.e. the resolution-limit.
// target_wh target-resolution for resizing the image.
// max_wh is the maximum image width and height, i.e. the resolution-limit,
// as set by `Config::MediaQuality`.
let exceeds_wh = img.width() > max_wh || img.height() > max_wh;
let mut target_wh = if exceeds_wh {
max_wh
} else {
max(img.width(), img.height())
};
let exceeds_max_bytes = nr_bytes > max_bytes as u64;
let jpeg_quality = 75;
@@ -428,6 +423,35 @@ impl<'a> BlobObject<'a> {
});
if do_scale {
let longest_side_len = max(img.width(), img.height());
// target_wh will be used as the target-resolution for resizing the image,
// so that the longest sides of the image match the target-resolution.
let mut target_wh = if !is_avatar {
let area_sqrt = (f64::from(img.width()) * f64::from(img.height())).sqrt();
// Limit resolution to the number of pixels that fit within max_wh * max_wh,
// so that the image-quality does not depend on the aspect-ratio.
let mut resolution_limit: u32 = cast(
(f64::from(longest_side_len) * (f64::from(max_wh) / area_sqrt)).floor(),
)
.unwrap_or(max_wh);
// Align at least one dimension of the resampled image to a multiple of 8 pixels,
// to have fewer partially used JPEG-blocks (which represent 8x8 pixels each).
if resolution_limit < longest_side_len && resolution_limit > 8 {
while !resolution_limit.is_multiple_of(8) {
resolution_limit -= 1
}
}
resolution_limit
} else {
max_wh
};
target_wh = min(target_wh, longest_side_len);
// For images in JPEG-format, 65535 pixels is the maximum resolution per dimension.
target_wh = min(target_wh, 65535);
loop {
if mem::take(&mut add_white_bg) {
self::add_white_bg(&mut img);
+6 -6
View File
@@ -406,8 +406,8 @@ async fn test_recode_image_balanced_png() {
extension: "png",
original_width: 1920,
original_height: 1080,
compressed_width: constants::WORSE_IMAGE_SIZE,
compressed_height: constants::WORSE_IMAGE_SIZE * 1080 / 1920,
compressed_width: 848,
compressed_height: 477,
..Default::default()
}
.test()
@@ -497,8 +497,8 @@ async fn test_recode_image_rgba_png_to_jpeg() {
extension: "png",
original_width: 1920,
original_height: 1080,
compressed_width: constants::WORSE_IMAGE_SIZE,
compressed_height: constants::WORSE_IMAGE_SIZE * 1080 / 1920,
compressed_width: 848,
compressed_height: 477,
..Default::default()
}
.test()
@@ -517,8 +517,8 @@ async fn test_recode_image_huge_jpg() {
has_exif: true,
original_width: 1920,
original_height: 1080,
compressed_width: constants::BALANCED_IMAGE_SIZE,
compressed_height: constants::BALANCED_IMAGE_SIZE * 1080 / 1920,
compressed_width: 1704,
compressed_height: 959,
..Default::default()
}
.test()
+55 -53
View File
@@ -50,8 +50,8 @@ use crate::stock_str;
use crate::sync::{self, Sync::*, SyncData};
use crate::tools::{
IsNoneOrEmpty, SystemTime, buf_compress, create_broadcast_secret, create_id,
create_outgoing_rfc724_mid, create_smeared_timestamp, create_smeared_timestamps, get_abs_path,
gm2local_offset, normalize_text, smeared_time, time, truncate_msg_text,
create_outgoing_rfc724_mid, get_abs_path, gm2local_offset, normalize_text, time,
truncate_msg_text,
};
use crate::webxdc::StatusUpdateSerial;
@@ -292,7 +292,7 @@ impl ChatId {
timestamp: i64,
) -> Result<Self> {
let grpname = sanitize_single_line(grpname);
let timestamp = cmp::min(timestamp, smeared_time(context));
let timestamp = cmp::min(timestamp, time());
let row_id =
context.sql.insert(
"INSERT INTO chats (type, name, name_normalized, grpid, blocked, created_timestamp, protected, param) VALUES(?, ?, ?, ?, ?, ?, 0, ?)",
@@ -1256,7 +1256,7 @@ SELECT id, rfc724_mid, pre_rfc724_mid, timestamp, ?, 1 FROM msgs WHERE chat_id=?
message_timestamp: i64,
always_sort_to_bottom: bool,
) -> Result<i64> {
let mut sort_timestamp = cmp::min(message_timestamp, smeared_time(context));
let mut sort_timestamp = cmp::min(message_timestamp, time());
let last_msg_time: Option<i64> = if always_sort_to_bottom {
// get newest message for this chat
@@ -2405,7 +2405,7 @@ impl ChatIdBlocked {
_ => (),
}
let smeared_time = create_smeared_timestamp(context);
let now = time();
let chat_id = context
.sql
@@ -2420,7 +2420,7 @@ impl ChatIdBlocked {
normalize_text(&chat_name),
params.to_string(),
create_blocked as u8,
smeared_time,
now,
),
)?;
let chat_id = ChatId::new(
@@ -2446,7 +2446,7 @@ impl ChatIdBlocked {
&& !chat.param.exists(Param::Devicetalk)
&& !chat.param.exists(Param::Selftalk)
{
chat_id.add_e2ee_notice(context, smeared_time).await?;
chat_id.add_e2ee_notice(context, now).await?;
}
Ok(Self {
@@ -2730,7 +2730,7 @@ async fn prepare_send_msg(
}
msg.state = MessageState::OutPending;
msg.timestamp_sort = create_smeared_timestamp(context);
msg.timestamp_sort = time();
prepare_msg_blob(context, msg).await?;
if !msg.hidden {
chat_id.unarchive_if_not_muted(context, msg.state).await?;
@@ -2924,7 +2924,7 @@ pub(crate) async fn create_send_msg_jobs(context: &Context, msg: &mut Message) -
);
}
let now = smeared_time(context);
let now = time();
if rendered_msg.last_added_location_id.is_some()
&& let Err(err) = location::set_kml_sent_timestamp(context, msg.chat_id, now).await
@@ -3046,7 +3046,7 @@ pub async fn send_edit_request(context: &Context, msg_id: MsgId, new_text: Strin
return Ok(());
}
save_text_edit_to_db(context, &mut original_msg, &new_text).await?;
save_text_edit_to_db(context, &mut original_msg, &new_text, &[]).await?;
let mut edit_msg = Message::new_text(EDITED_PREFIX.to_owned() + &new_text); // prefix only set for nicer display in Non-Delta-MUAs
edit_msg.set_quote(context, Some(&original_msg)).await?; // quote only set for nicer display in Non-Delta-MUAs
@@ -3065,16 +3065,20 @@ pub(crate) async fn save_text_edit_to_db(
context: &Context,
original_msg: &mut Message,
new_text: &str,
mime_headers: &[u8],
) -> Result<()> {
original_msg.param.set_int(Param::IsEdited, 1);
context
.sql
.execute(
"UPDATE msgs SET txt=?, txt_normalized=?, param=? WHERE id=?",
"
UPDATE msgs SET txt=?, txt_normalized=?, param=?, mime_headers=?, mime_modified=? WHERE id=?",
(
new_text,
normalize_text(new_text),
original_msg.param.to_string(),
mime_headers,
!mime_headers.is_empty(),
original_msg.id,
),
)
@@ -3552,6 +3556,10 @@ pub async fn create_group_unencrypted(context: &Context, name: &str) -> Result<C
/// unencrypted chats currently.
/// * `grpid` - Group ID. Iff nonempty, the chat is encrypted (with key-contacts).
/// * `name` - Chat name.
///
/// NB: Unencrypted chats with similar names and the same members are merged on other devices, but
/// usually users don't create such chats and look up the existing one instead, so chat split on the
/// first device is acceptable.
pub(crate) async fn create_group_ex(
context: &Context,
sync: sync::Sync,
@@ -3566,7 +3574,7 @@ pub(crate) async fn create_group_ex(
chat_name = "…".to_string();
}
let timestamp = create_smeared_timestamp(context);
let timestamp = time();
let row_id = context
.sql
.insert(
@@ -3649,7 +3657,7 @@ pub(crate) async fn create_out_broadcast_ex(
bail!("Invalid broadcast channel name: {chat_name}.");
}
let timestamp = create_smeared_timestamp(context);
let timestamp = time();
let trans_fn = |t: &mut rusqlite::Transaction| -> Result<ChatId> {
let cnt: u32 = t.query_row(
"SELECT COUNT(*) FROM chats WHERE grpid=?",
@@ -3738,19 +3746,17 @@ pub(crate) async fn update_chat_contacts_table(
id: ChatId,
contacts: &BTreeSet<ContactId>,
) -> Result<()> {
// See add_to_chat_contacts_table() for reasoning.
let limit = cmp::max(time().saturating_add(TIMESTAMP_SENT_TOLERANCE), timestamp);
context
.sql
.transaction(move |transaction| {
// Bump `remove_timestamp` even for members from `contacts`.
// Bump `remove_timestamp` to at least `now`
// even for members from `contacts`.
// We add members from `contacts` back below.
transaction.execute(
"UPDATE chats_contacts SET
add_timestamp=MIN(add_timestamp, ?1),
remove_timestamp=MAX(MIN(remove_timestamp,?1), MIN(add_timestamp,?1)+1, ?)
"UPDATE chats_contacts
SET remove_timestamp=MAX(add_timestamp+1, ?)
WHERE chat_id=?",
(limit, timestamp, id),
(timestamp, id),
)?;
if !contacts.is_empty() {
@@ -3762,8 +3768,9 @@ pub(crate) async fn update_chat_contacts_table(
)?;
for contact_id in contacts {
// We bumped `remove_timestamp` for existing rows above,
// so on conflict it is enough to set `add_timestamp = remove_timestamp`.
// We bumped `add_timestamp` for existing rows above,
// so on conflict it is enough to set `add_timestamp = remove_timestamp`
// and this guarantees that `add_timestamp` is no less than `timestamp`.
statement.execute((id, contact_id, timestamp))?;
}
}
@@ -3780,24 +3787,17 @@ pub(crate) async fn add_to_chat_contacts_table(
chat_id: ChatId,
contact_ids: &[ContactId],
) -> Result<()> {
// Our clock may be slow, so limit stored timestamps with `timestamp` if it's bigger. This way
// we only cap remote timestamps if, in addition, remote changes arrive reordered or we do local
// changes. Also allow some tolerance, moreover, previous removals might lend time from the
// future.
let limit = cmp::max(time().saturating_add(TIMESTAMP_SENT_TOLERANCE), timestamp);
context
.sql
.transaction(move |transaction| {
let mut add_statement = transaction.prepare(
"INSERT INTO chats_contacts (chat_id, contact_id, add_timestamp) VALUES(?1, ?2, ?3)
ON CONFLICT (chat_id, contact_id)
DO UPDATE SET
remove_timestamp=MIN(remove_timestamp, ?4),
add_timestamp=MIN(MAX(add_timestamp,remove_timestamp,?3), ?4)",
DO UPDATE SET add_timestamp=MAX(remove_timestamp, ?3)",
)?;
for contact_id in contact_ids {
add_statement.execute((chat_id, contact_id, timestamp, limit))?;
add_statement.execute((chat_id, contact_id, timestamp))?;
}
Ok(())
})
@@ -3816,16 +3816,13 @@ pub(crate) async fn remove_from_chat_contacts_table(
contact_id: ContactId,
) -> Result<bool> {
let now = time();
// See add_to_chat_contacts_table() for reasoning.
let limit = now.saturating_add(TIMESTAMP_SENT_TOLERANCE);
let is_past_member = context
.sql
.execute(
"UPDATE chats_contacts SET
add_timestamp=MIN(add_timestamp, ?1),
remove_timestamp=MAX(MIN(remove_timestamp,?1), MIN(add_timestamp,?1)+1, ?)
"UPDATE chats_contacts
SET remove_timestamp=MAX(add_timestamp+1, ?)
WHERE chat_id=? AND contact_id=?",
(limit, now, chat_id, contact_id),
(now, chat_id, contact_id),
)
.await?
> 0;
@@ -3918,11 +3915,11 @@ pub(crate) async fn add_contact_to_chat_ex(
return Ok(false);
}
if from_handshake && chat.param.get_int(Param::Unpromoted).unwrap_or_default() == 1 {
let smeared_time = smeared_time(context);
let now = time();
chat.param
.remove(Param::Unpromoted)
.set_i64(Param::GroupNameTimestamp, smeared_time)
.set_i64(Param::GroupDescriptionTimestamp, smeared_time);
.set_i64(Param::GroupNameTimestamp, now)
.set_i64(Param::GroupDescriptionTimestamp, now);
chat.update_param(context).await?;
}
if context.is_self_addr(contact.get_addr()).await? {
@@ -4488,7 +4485,6 @@ pub async fn forward_msgs(context: &Context, msg_ids: &[MsgId], chat_id: ChatId)
}
/// Forwards multiple messages to a chat in another context.
#[expect(clippy::arithmetic_side_effects)]
pub async fn forward_msgs_2ctx(
ctx_src: &Context,
msg_ids: &[MsgId],
@@ -4499,7 +4495,6 @@ pub async fn forward_msgs_2ctx(
ensure!(!chat_id.is_special(), "can not forward to special chat");
let mut created_msgs: Vec<MsgId> = Vec::new();
let mut curr_timestamp: i64;
chat_id
.unarchive_if_not_muted(ctx_dst, MessageState::Undefined)
@@ -4508,7 +4503,7 @@ pub async fn forward_msgs_2ctx(
if let Some(reason) = chat.why_cant_send(ctx_dst).await? {
bail!("cannot send to {chat_id}: {reason}");
}
curr_timestamp = create_smeared_timestamps(ctx_dst, msg_ids.len());
let now = time();
let mut msgs = Vec::with_capacity(msg_ids.len());
for id in msg_ids {
let ts: i64 = ctx_src
@@ -4573,10 +4568,9 @@ pub async fn forward_msgs_2ctx(
msg.state = MessageState::OutPending;
msg.rfc724_mid = create_outgoing_rfc724_mid();
msg.pre_rfc724_mid.clear();
msg.timestamp_sort = curr_timestamp;
msg.timestamp_sort = now;
chat.prepare_msg_raw(ctx_dst, &mut msg, None).await?;
curr_timestamp += 1;
if !create_send_msg_jobs(ctx_dst, &mut msg).await?.is_empty() {
ctx_dst.scheduler.interrupt_smtp().await;
}
@@ -4669,7 +4663,7 @@ pub(crate) async fn save_copy_in_self_talk(
} else {
MessageState::InSeen
},
create_smeared_timestamp(context),
time(),
msg.param.to_string(),
src_msg_id,
src_msg_id,
@@ -4846,7 +4840,7 @@ pub async fn add_device_msg_with_importance(
chat_id = ChatId::get_for_contact(context, ContactId::DEVICE).await?;
let rfc724_mid = create_outgoing_rfc724_mid();
let timestamp_sent = create_smeared_timestamp(context);
let timestamp_sent = time();
// makes sure, the added message is the last one,
// even if the date is wrong (useful esp. when warning about bad dates)
@@ -4993,7 +4987,7 @@ pub(crate) async fn add_info_msg_with_cmd(
} else {
let sort_to_bottom = true;
chat_id
.calc_sort_timestamp(context, smeared_time(context), sort_to_bottom)
.calc_sort_timestamp(context, time(), sort_to_bottom)
.await?
};
@@ -5156,7 +5150,7 @@ async fn set_contacts_by_fingerprints(
Ok(broadcast_contacts_added)
})
.await?;
let timestamp = smeared_time(context);
let timestamp = time();
for added_id in broadcast_contacts_added {
let msg = stock_str::msg_add_member_local(context, added_id, ContactId::UNDEFINED).await;
add_info_msg_with_cmd(
@@ -5240,9 +5234,11 @@ impl Context {
}
_ => (),
}
// Use `Request` so that even if the program crashes, the user doesn't have to look
// into the blocked contacts.
ChatIdBlocked::get_for_contact(self, contact_id, Blocked::Request)
// Newly created chat will be soon unblocked, `Blocked::Yes` here is just
// to hide it from chatlist, as at this point it is completely blank (no name etc.).
// Even if app crashes at this point, the chat will re-appear on e.g. any new
// message sent from other device.
ChatIdBlocked::get_for_contact(self, contact_id, Blocked::Yes)
.await?
.id
}
@@ -5266,7 +5262,13 @@ impl Context {
}
_ => (),
}
ChatIdBlocked::get_for_contact(self, contact_id, Blocked::Request)
// Don't show a chat on other devices until securejoin completes.
// E.g. pinning a not-synced chat on device A shouldn't display it on device B yet.
//
// A pinned chat will appear on other devices only when the first
// message is sent from the device that read the invitation - which is the same
// behavior as with un-pinned chats.
ChatIdBlocked::get_for_contact(self, contact_id, Blocked::Yes)
.await?
.id
}
+170 -28
View File
@@ -3,7 +3,9 @@ use std::sync::Arc;
use super::*;
use crate::Event;
use crate::chatlist::get_archived_cnt;
use crate::constants::{DC_GCL_ARCHIVED_ONLY, DC_GCL_NO_SPECIALS, N_MSGS_TO_NEW_BROADCAST_MEMBER};
use crate::constants::{
self, DC_GCL_ARCHIVED_ONLY, DC_GCL_NO_SPECIALS, N_MSGS_TO_NEW_BROADCAST_MEMBER,
};
use crate::ephemeral::Timer;
use crate::headerdef::HeaderDef;
use crate::imex::{ImexMode, has_backup, imex};
@@ -748,7 +750,7 @@ async fn test_leave_group() -> Result<()> {
assert_eq!(get_chat_contacts(&alice, alice_chat_id).await?.len(), 1);
assert_eq!(rcvd_leave_msg.state, MessageState::InSeen);
assert_eq!(rcvd_leave_msg.state, MessageState::InNoticed);
alice.emit_event(EventType::Test);
alice
@@ -1281,7 +1283,7 @@ async fn test_marknoticed_all_chats() -> Result<()> {
tcm.section("bob: receive messages, accept all chats and send a reply to each messsage");
while let Some(sent_msg) = alice.pop_sent_msg_opt(Duration::default()).await {
while let Some(sent_msg) = alice.pop_sent_msg_opt().await {
let bob_message = bob.recv_msg(&sent_msg).await;
let bob_chat_id = bob_message.chat_id;
bob_chat_id.accept(bob).await?;
@@ -1289,7 +1291,7 @@ async fn test_marknoticed_all_chats() -> Result<()> {
}
tcm.section("alice: receive replies from bob");
while let Some(sent_msg) = bob.pop_sent_msg_opt(Duration::default()).await {
while let Some(sent_msg) = bob.pop_sent_msg_opt().await {
alice.recv_msg(&sent_msg).await;
}
// ensure chats have unread messages
@@ -1633,6 +1635,7 @@ async fn test_set_chat_name() {
"another name",
"something different",
] {
SystemTime::shift(Duration::from_secs(1));
set_chat_name(alice, chat_id, new_name).await.unwrap();
let sent_msg = alice.pop_sent_msg().await;
let received_msg = bob.recv_msg(&sent_msg).await;
@@ -2815,7 +2818,7 @@ async fn test_cant_remove_nonmember() -> Result<()> {
let alice_charlie_id = alice.add_or_lookup_contact_id(charlie).await;
remove_contact_from_chat(alice, alice_broadcast_id, alice_charlie_id).await?;
assert!(alice.pop_sent_msg_opt(Duration::ZERO).await.is_none());
assert!(alice.pop_sent_msg_opt().await.is_none());
assert!(!remove_from_chat_contacts_table(alice, alice_broadcast_id, alice_charlie_id).await?);
assert!(
!remove_from_chat_contacts_table_without_trace(alice, alice_broadcast_id, alice_charlie_id)
@@ -2970,6 +2973,7 @@ async fn test_broadcast_change_name() -> Result<()> {
{
tcm.section("Alice changes the chat name");
SystemTime::shift(Duration::from_secs(1));
set_chat_name(alice, broadcast_id, "My great broadcast").await?;
let sent = alice.pop_sent_msg().await;
@@ -2988,6 +2992,7 @@ async fn test_broadcast_change_name() -> Result<()> {
{
tcm.section("Alice changes the chat name again, but the system message is lost somehow");
SystemTime::shift(Duration::from_secs(1));
set_chat_name(alice, broadcast_id, "Broadcast channel").await?;
let chat = Chat::load_from_db(alice, broadcast_id).await?;
@@ -3064,13 +3069,11 @@ async fn test_broadcast_resend_to_new_member() -> Result<()> {
}
for i in 0..N_MSGS_TO_NEW_BROADCAST_MEMBER {
let rev_order = false;
let resent_msg = alice
.pop_sent_msg_ex(rev_order, Duration::ZERO)
.await
.unwrap();
let resent_msg = alice.pop_sent_msg_ex(rev_order).await.unwrap();
let fiona_msg = fiona.recv_msg(&resent_msg).await;
assert_eq!(fiona_msg.chat_id, fiona_bc_id);
assert_eq!(fiona_msg.text, (i + 1).to_string());
assert_eq!(fiona_msg.param.get_bool(Param::WantsMdn).unwrap(), true);
assert!(resent_msg.recipients.contains("fiona@example.net"));
assert!(!resent_msg.recipients.contains("bob@"));
// The message is undecryptable for Bob, he mustn't be able to know yet that somebody joined
@@ -3084,7 +3087,7 @@ async fn test_broadcast_resend_to_new_member() -> Result<()> {
);
bob.recv_msg_trash(&resent_msg).await;
}
assert!(alice.pop_sent_msg_opt(Duration::ZERO).await.is_none());
assert!(alice.pop_sent_msg_opt().await.is_none());
Ok(())
}
@@ -3342,6 +3345,7 @@ async fn test_broadcasts_name_and_avatar() -> Result<()> {
assert_eq!(bob_chat.get_profile_image(bob).await?, None);
tcm.section("Change broadcast channel name, and check that receivers see it");
SystemTime::shift(Duration::from_secs(1));
set_chat_name(alice, alice_chat_id, "New Channel name").await?;
let sent = alice.pop_sent_msg().await;
let rcvd = bob.recv_msg(&sent).await;
@@ -3507,8 +3511,9 @@ async fn test_chat_description(
"",
"ä ẟ 😂",
] {
SystemTime::shift(Duration::from_secs(1));
tcm.section(&format!(
"Alice sets the chat description to '{description}'"
"Alice sets the chat description to {description:?}"
));
set_chat_description(alice, alice_chat_id, description).await?;
let sent = alice.pop_sent_msg().await;
@@ -3547,12 +3552,7 @@ async fn test_chat_description(
tcm.section("Alice calls set_chat_description() without actually changing the description");
set_chat_description(alice, alice_chat_id, "ä ẟ 😂").await?;
assert!(
alice
.pop_sent_msg_opt(Duration::from_secs(0))
.await
.is_none()
);
assert!(alice.pop_sent_msg_opt().await.is_none());
Ok(())
}
@@ -3577,12 +3577,7 @@ async fn test_setting_empty_chat_description() -> Result<()> {
let _hi = alice.send_text(alice_chat_id, "hi").await;
set_chat_description(alice, alice_chat_id, "").await?;
assert!(
alice
.pop_sent_msg_opt(Duration::from_secs(0))
.await
.is_none()
);
assert!(alice.pop_sent_msg_opt().await.is_none());
Ok(())
}
@@ -4529,7 +4524,9 @@ async fn test_get_chat_media_webxdc_order() -> Result<()> {
assert_eq!(media.first().unwrap(), &instance1_id);
assert_eq!(media.get(1).unwrap(), &instance2_id);
// add a status update for the oder instance; that resorts the list
SystemTime::shift(Duration::from_secs(1));
// add a status update for the other instance; that resorts the list
alice
.send_webxdc_status_update(instance1_id, r#"{"payload": {"foo": "bar"}}"#)
.await?;
@@ -4945,10 +4942,6 @@ async fn test_sync_broadcast_and_send_message() -> Result<()> {
vec![a2b_contact_id]
);
// alice2's smeared clock may be behind alice1's one, so we need to work around "hi" appearing
// before "You joined the channel." for bob. alice1 makes 3 more calls of
// create_smeared_timestamp() than alice2 does as of 2026-03-10.
SystemTime::shift(Duration::from_secs(3));
tcm.section("Alice's second device sends a message to the channel");
let sent_msg = alice2.send_text(a2_broadcast_id, "hi").await;
let msg = bob.recv_msg(&sent_msg).await;
@@ -5099,6 +5092,126 @@ async fn test_broadcast_contacts_are_hidden() -> Result<()> {
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_blocked_bob_cant_join_chat() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice1 = &tcm.alice().await;
let alice2 = &tcm.alice().await;
let bob = &tcm.bob().await;
for a in [alice1, alice2] {
a.set_config_bool(Config::SyncMsgs, true).await?;
}
// The observing device has Bob blocked from the early start.
let alice2_bob_id = alice2.add_or_lookup_contact_id(bob).await;
Contact::block(alice2, alice2_bob_id).await?;
let alice1_chat_id = create_group(alice1, "").await?;
sync(alice1, alice2).await;
let alice1_chat = Chat::load_from_db(alice1, alice1_chat_id).await?;
let (alice2_chat_id, _blocked) = get_chat_id_by_grpid(alice2, &alice1_chat.grpid)
.await?
.unwrap();
let qr = get_securejoin_qr(alice1, Some(alice1_chat_id)).await?;
sync(alice1, alice2).await;
tcm.exec_securejoin_qr_multi_device(bob, &[alice1, alice2], &qr)
.await;
let alice1_bob_id = alice1.add_or_lookup_contact_id(bob).await;
assert_eq!(get_chat_contacts(alice1, alice1_chat_id).await?.len(), 2);
// "vg-member-added" from alice1 adds bob for alice2 to provide membership consistency on
// devices.
assert_eq!(get_chat_contacts(alice2, alice2_chat_id).await?.len(), 2);
remove_contact_from_chat(alice1, alice1_chat_id, alice1_bob_id).await?;
bob.recv_msg(&alice1.pop_sent_msg().await).await;
tcm.exec_securejoin_qr(bob, alice1, &qr).await;
// Bob can join again if he isn't blocked.
assert_eq!(get_chat_contacts(alice1, alice1_chat_id).await?.len(), 2);
Contact::block(alice1, alice1_bob_id).await?;
remove_contact_from_chat(alice1, alice1_chat_id, alice1_bob_id).await?;
bob.recv_msg(&alice1.pop_sent_msg().await).await;
tcm.exec_securejoin_qr(bob, alice1, &qr).await;
let members = get_chat_contacts(alice1, alice1_chat_id).await?;
assert_eq!(members.len(), 1);
assert!(members.contains(&ContactId::SELF));
let past_members = get_past_chat_contacts(alice1, alice1_chat_id).await?;
assert_eq!(past_members.len(), 1);
assert!(past_members.contains(&alice1_bob_id));
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_blocked_bob_cant_create_11_chat_via_securejoin() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice1 = &tcm.alice().await;
let alice2 = &tcm.alice().await;
let bob = &tcm.bob().await;
for a in [alice1, alice2] {
a.set_config_bool(Config::SyncMsgs, true).await?;
}
// The observing device has Bob blocked.
let alice2_bob_id = alice2.add_or_lookup_contact_id(bob).await;
Contact::block(alice2, alice2_bob_id).await?;
let qr = get_securejoin_qr(alice1, None).await?;
sync(alice1, alice2).await;
let chat_cnt = get_chat_cnt(alice1).await?;
assert_eq!(get_chat_cnt(alice2).await?, chat_cnt);
tcm.exec_securejoin_qr_multi_device(bob, &[alice1, alice2], &qr)
.await;
assert_eq!(get_chat_cnt(alice1).await?, chat_cnt + 1);
assert_eq!(get_chat_cnt(alice2).await?, chat_cnt);
Ok(())
}
/// Regression test:
///
/// Pinning a chat that didn't complete securejoin on one device
/// should not cause a blank chat to be displayed on other device.
/// Such chat should be marked as blocked until the first message is sent.
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_sync_no_blank_chat() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice1 = &tcm.alice().await;
let alice2 = &tcm.alice().await;
let bob = &tcm.bob().await;
for a in [alice1, alice2] {
a.set_config_bool(Config::SyncMsgs, true).await?;
a.set_config_bool(Config::BccSelf, true).await?;
}
let qr = get_securejoin_qr(bob, None).await?;
// Scan QR on alice1 and pin the resulting chat.
let chat_id = tcm.exec_securejoin_qr(alice1, bob, &qr).await;
chat_id
.set_visibility(&alice1.ctx, ChatVisibility::Pinned)
.await?;
// alice2 should receive a sync item about changing visibility of a chat it doesn't know about.
sync(alice1, alice2).await;
// Chat created on alice2 should be blocked (hidden)
let chat = Chat::load_from_db(&alice2.ctx, chat_id).await?;
assert_eq!(chat.blocked, Blocked::Yes);
assert_eq!(chat.get_name(), "");
assert_eq!(chat.visibility, ChatVisibility::Pinned);
// First message should unblock and fill missing data on other device.
let msg = alice1.send_text(chat_id, "meow").await;
alice2.recv_msg(&msg).await;
let chat = Chat::load_from_db(&alice2.ctx, chat_id).await?;
assert_eq!(chat.blocked, Blocked::Not);
assert_eq!(chat.get_name(), "bob@example.net");
assert_eq!(chat.visibility, ChatVisibility::Pinned);
Ok(())
}
/// Tests sending JPEG image with .png extension.
///
/// This is a regression test, previously sending failed
@@ -5759,6 +5872,35 @@ async fn test_send_edit_request() -> Result<()> {
let forwarded = alice2.get_last_msg().await;
assert!(!forwarded.is_edited());
// If a message is too long after editing, it becomes an HTML message on the receiver side. On
// the sender side it's still text so that it can be edited again.
static REPEAT_TXT: &str = "this text with 42 chars is just repeated.\n";
static REPEAT_CNT: usize = constants::DC_DESIRED_TEXT_LEN / REPEAT_TXT.len() + 2;
let long_txt = REPEAT_TXT.repeat(REPEAT_CNT);
send_edit_request(alice, alice_msg.id, long_txt.clone()).await?;
let sent = alice.pop_sent_msg().await;
let test = Message::load_from_db(alice, alice_msg.id).await?;
assert!(!test.has_html());
assert_eq!(test.text, long_txt);
bob.recv_msg_opt(&sent).await;
let test = Message::load_from_db(bob, bob_msg.id).await?;
assert!(test.is_edited());
assert!(test.has_html());
let html = test.id.get_html(bob).await?.unwrap();
assert_eq!(html.matches("just repeated.<br/>").count(), REPEAT_CNT);
assert!(test.text.matches("just repeated.").count() > 0);
// Alice shortens the message back so it's not HTML for Bob anymore.
send_edit_request(alice, alice_msg.id, "Text me on Delta.Chat".to_string()).await?;
let sent = alice.pop_sent_msg().await;
let test = Message::load_from_db(alice, alice_msg.id).await?;
assert_eq!(test.text, "Text me on Delta.Chat");
bob.recv_msg_opt(&sent).await;
let test = Message::load_from_db(bob, bob_msg.id).await?;
assert!(test.is_edited());
assert!(!test.has_html());
assert_eq!(test.text, "Text me on Delta.Chat");
Ok(())
}
-6
View File
@@ -455,12 +455,6 @@ pub enum Config {
/// Return an error from `receive_imf_inner()`. For tests.
SimulateReceiveImfError,
/// Enable composing emails with Header Protection as defined in
/// <https://www.rfc-editor.org/rfc/rfc9788.html> "Header Protection for Cryptographically
/// Protected Email".
#[strum(props(default = "1"))]
StdHeaderProtectionComposing,
/// Who can call me.
///
/// The options are from the `WhoCanCallMe` enum.
+15
View File
@@ -49,6 +49,21 @@ use crate::{chat, provider};
/// see <https://github.com/chatmail/core/issues/7608>
pub(crate) const MAX_TRANSPORT_RELAYS: usize = 5;
/// Hard-coded candidates for default relays.
/// In the future, we want to use it during onboarding;
/// note that before onboarding automatically on any of these,
/// we need to ask the admins whether their relay is able to handle this.
/// For now, this is just the first 6 relays from chatmail.at/relays.
#[allow(unused)]
const DEFAULT_RELAY_CANDIDATES: &[&str] = &[
"mehl.cloud",
"mailchat.pl",
"chatmail.woodpeckersnest.space",
"chatmail.culturanerd.it",
"tarpit.fun",
"d.gaufr.es",
];
macro_rules! progress {
($context:tt, $progress:expr, $comment:expr) => {
assert!(
+3 -1
View File
@@ -1026,7 +1026,9 @@ impl Contact {
|| row_authname.is_empty());
row_id = id;
if origin >= row_origin && addr != row_addr {
let qr_with_fingerprint = !fingerprint.is_empty()
&& origin == Origin::UnhandledSecurejoinQrScan;
if (origin >= row_origin || qr_with_fingerprint) && addr != row_addr {
update_addr = true;
}
if update_name || update_authname || update_addr || origin > row_origin {
-3
View File
@@ -31,7 +31,6 @@ use crate::quota::QuotaInfo;
use crate::scheduler::{ConnectivityStore, SchedulerState};
use crate::sql::Sql;
use crate::stock_str::StockStrings;
use crate::timesmearing::SmearedTimestamp;
use crate::tools::{self, duration_to_str, time, time_elapsed};
use crate::transport::ConfiguredLoginParam;
use crate::{chatlist_events, stats};
@@ -227,7 +226,6 @@ pub struct InnerContext {
/// Blob directory path
pub(crate) blobdir: PathBuf,
pub(crate) sql: Sql,
pub(crate) smeared_timestamp: SmearedTimestamp,
/// The global "ongoing" process state.
///
/// This is a global mutex-like state for operations which should be modal in the
@@ -486,7 +484,6 @@ impl Context {
blobdir,
running_state: RwLock::new(Default::default()),
sql: Sql::new(dbfile),
smeared_timestamp: SmearedTimestamp::new(),
oauth2_mutex: Mutex::new(()),
wrong_pw_warning_mutex: Mutex::new(()),
housekeeping_mutex: Mutex::new(()),
+5 -2
View File
@@ -83,12 +83,15 @@ impl EncryptHelper {
None
};
let shared_secret = shared_secret.to_string();
let mut raw_message = Vec::new();
let cursor = Cursor::new(&mut raw_message);
mail_to_encrypt.clone().write_part(cursor).ok();
let ctext =
pgp::symm_encrypt_message(raw_message, sign_key, shared_secret, compress).await?;
let ctext = tokio::task::spawn_blocking(move || {
pgp::symm_encrypt_message(raw_message, sign_key, shared_secret, compress)
})
.await??;
Ok(ctext)
}
+6 -15
View File
@@ -11,7 +11,6 @@ use crate::message::markseen_msgs;
use crate::receive_imf::receive_imf;
use crate::test_utils;
use crate::test_utils::{TestContext, TestContextManager};
use crate::timesmearing::MAX_SECONDS_TO_LEND_FROM_FUTURE;
use crate::{
chat::{self, Chat, ChatItem, create_group, send_text_msg},
tools::IsNoneOrEmpty,
@@ -172,7 +171,7 @@ async fn test_ephemeral_unpromoted() -> Result<()> {
chat_id
.set_ephemeral_timer(&alice, Timer::Enabled { duration: 60 })
.await?;
let sent = alice.pop_sent_msg_opt(Duration::from_secs(1)).await;
let sent = alice.pop_sent_msg_opt().await;
assert!(sent.is_none());
assert_eq!(
chat_id.get_ephemeral_timer(&alice).await?,
@@ -182,13 +181,13 @@ async fn test_ephemeral_unpromoted() -> Result<()> {
// Promote the group.
send_text_msg(&alice, chat_id, "hi!".to_string()).await?;
assert!(chat_id.is_promoted(&alice).await?);
let sent = alice.pop_sent_msg_opt(Duration::from_secs(1)).await;
let sent = alice.pop_sent_msg_opt().await;
assert!(sent.is_some());
chat_id
.set_ephemeral_timer(&alice.ctx, Timer::Disabled)
.await?;
let sent = alice.pop_sent_msg_opt(Duration::from_secs(1)).await;
let sent = alice.pop_sent_msg_opt().await;
assert!(sent.is_some());
assert_eq!(chat_id.get_ephemeral_timer(&alice).await?, Timer::Disabled);
@@ -355,17 +354,9 @@ async fn test_ephemeral_delete_msgs() -> Result<()> {
let now = time();
let msg = t.send_text(bob_chat.id, "Message text").await;
check_msg_will_be_deleted(
t,
msg.sender_msg_id,
&bob_chat,
now + 1799,
// The message may appear to be sent MAX_SECONDS_TO_LEND_FROM_FUTURE later and
// therefore be deleted MAX_SECONDS_TO_LEND_FROM_FUTURE later.
time() + 1801 + MAX_SECONDS_TO_LEND_FROM_FUTURE,
)
.await
.unwrap();
check_msg_will_be_deleted(t, msg.sender_msg_id, &bob_chat, now + 1799, time() + 1801)
.await
.unwrap();
// Enable ephemeral messages with Bob -> message will be deleted after 60s.
// This tests that the message is deleted at min(ephemeral deletion time, DeleteDeviceAfter deletion time).
+11
View File
@@ -181,6 +181,17 @@ pub enum EventType {
msg_id: MsgId,
},
/// Like [`EventType::MsgRead`], but also fires on subsequent MDNs,
/// if there are multiple receivers, i.e. in groups and channels.
#[serde(rename_all = "camelCase")]
MsgReadCountChanged {
/// ID of the chat which the message belongs to.
chat_id: ChatId,
/// ID of the message that was read.
msg_id: MsgId,
},
/// A single message was deleted.
///
/// This event means that the message will no longer appear in the messagelist.
+65 -20
View File
@@ -9,7 +9,7 @@
use std::mem;
use anyhow::{Context as _, Result};
use anyhow::{Context as _, Result, ensure};
use base64::Engine as _;
use mailparse::ParsedContentType;
use mime::Mime;
@@ -17,10 +17,12 @@ use mime::Mime;
use crate::context::Context;
use crate::headerdef::{HeaderDef, HeaderDefMap};
use crate::log::warn;
use crate::message::{self, Message, MsgId};
use crate::message::{Message, MsgId};
use crate::mimeparser::parse_message_id;
use crate::param::Param::SendHtml;
use crate::param::{Param::SendHtml, Params};
use crate::plaintext::PlainText;
use crate::sql;
use crate::tools::{buf_compress, buf_decompress};
impl Message {
/// Check if the message can be retrieved as HTML.
@@ -258,28 +260,71 @@ impl MsgId {
/// NB: we do not save raw mime unconditionally in the database to save space.
/// The corresponding ffi-function is `dc_get_msg_html()`.
pub async fn get_html(self, context: &Context) -> Result<Option<String>> {
// If there are many concurrent db readers, going to the queue earlier makes sense.
let (param, rawmime) = tokio::join!(
self.get_param(context),
message::get_mime_headers(context, self)
);
if let Some(html) = param?.get(SendHtml) {
let (param, headers, compressed) = context
.sql
.query_row(
"SELECT param, mime_headers, mime_compressed FROM msgs WHERE id=?",
(self,),
|row| {
let param: String = row.get(0)?;
let param: Params = param.parse().unwrap_or_default();
let headers = sql::row_get_vec(row, 1)?;
let compressed: bool = row.get(2)?;
Ok((param, headers, compressed))
},
)
.await?;
if let Some(html) = param.get(SendHtml) {
return Ok(Some(html.to_string()));
}
let rawmime = rawmime?;
if !rawmime.is_empty() {
match HtmlMsgParser::from_bytes(context, &rawmime) {
Err(err) => {
warn!(context, "get_html: parser error: {:#}", err);
Ok(None)
let from_rawmime = |rawmime: Vec<u8>| {
if !rawmime.is_empty() {
match HtmlMsgParser::from_bytes(context, &rawmime) {
Err(err) => {
warn!(context, "get_html: parser error: {:#}", err);
Ok(None)
}
Ok((parser, _)) => Ok(Some(parser.html)),
}
Ok((parser, _)) => Ok(Some(parser.html)),
} else {
warn!(context, "get_html: no mime for {}", self);
Ok(None)
}
} else {
warn!(context, "get_html: no mime for {}", self);
Ok(None)
};
if compressed {
return from_rawmime(buf_decompress(&headers)?);
}
let headers2 = headers.clone();
let compressed = match tokio::task::block_in_place(move || buf_compress(&headers2)) {
Err(e) => {
warn!(context, "get_mime_headers: buf_compress() failed: {}", e);
return from_rawmime(headers);
}
Ok(o) => o,
};
let update = |conn: &mut rusqlite::Connection| {
match conn.execute(
"
UPDATE msgs SET mime_headers=?, mime_compressed=1
WHERE id=? AND mime_headers!='' AND mime_compressed=0",
(compressed, self),
) {
Ok(rows_updated) => ensure!(rows_updated <= 1),
Err(e) => {
warn!(context, "get_mime_headers: UPDATE failed: {}", e);
return Err(e.into());
}
}
Ok(())
};
if let Err(e) = context.sql.call_write(update).await {
warn!(
context,
"get_mime_headers: failed to update mime_headers: {}", e
);
}
from_rawmime(headers)
}
}
+3 -188
View File
@@ -6,7 +6,7 @@
use std::{
cmp::max,
cmp::min,
collections::{BTreeMap, BTreeSet, HashMap},
collections::{BTreeMap, HashMap},
iter::Peekable,
mem::take,
sync::atomic::Ordering,
@@ -21,8 +21,7 @@ use futures_lite::FutureExt;
use ratelimit::Ratelimit;
use url::Url;
use crate::chat::{self, ChatId, ChatIdBlocked, add_device_msg};
use crate::chatlist_events;
use crate::chat::{self, ChatIdBlocked, add_device_msg};
use crate::config::Config;
use crate::constants::{Blocked, DC_VERSION_STR};
use crate::contact::ContactId;
@@ -31,7 +30,7 @@ use crate::ensure_and_debug_assert;
use crate::events::EventType;
use crate::headerdef::{HeaderDef, HeaderDefMap};
use crate::log::{LogExt, warn};
use crate::message::{self, Message, MessageState, MsgId};
use crate::message::{self, Message};
use crate::mimeparser;
use crate::net::proxy::ProxyConfig;
use crate::net::session::SessionStream;
@@ -1154,114 +1153,6 @@ impl Session {
Ok(())
}
/// Synchronizes `\Seen` flags using `CONDSTORE` extension.
pub(crate) async fn sync_seen_flags(&mut self, context: &Context, folder: &str) -> Result<()> {
if !self.can_condstore() {
info!(
context,
"Server does not support CONDSTORE, skipping flag synchronization."
);
return Ok(());
}
if context.get_config_bool(Config::TeamProfile).await? {
return Ok(());
}
let folder_exists = self
.select_with_uidvalidity(context, folder)
.await
.context("Failed to select folder")?;
if !folder_exists {
return Ok(());
}
let mailbox = self
.selected_mailbox
.as_ref()
.with_context(|| format!("No mailbox selected, folder: {folder}"))?;
// Check if the mailbox supports MODSEQ.
// We are not interested in actual value of HIGHESTMODSEQ.
if mailbox.highest_modseq.is_none() {
info!(
context,
"Mailbox {} does not support mod-sequences, skipping flag synchronization.", folder
);
return Ok(());
}
let transport_id = self.transport_id();
let mut updated_chat_ids = BTreeSet::new();
let uid_validity = get_uidvalidity(context, transport_id, folder)
.await
.with_context(|| format!("failed to get UID validity for folder {folder}"))?;
let mut highest_modseq = get_modseq(context, transport_id, folder)
.await
.with_context(|| format!("failed to get MODSEQ for folder {folder}"))?;
let mut list = self
.uid_fetch("1:*", format!("(FLAGS) (CHANGEDSINCE {highest_modseq})"))
.await
.context("failed to fetch flags")?;
let mut got_unsolicited_fetch = false;
while let Some(fetch) = list
.try_next()
.await
.context("failed to get FETCH result")?
{
let uid = if let Some(uid) = fetch.uid {
uid
} else {
info!(context, "FETCH result contains no UID, skipping");
got_unsolicited_fetch = true;
continue;
};
let is_seen = fetch.flags().any(|flag| flag == Flag::Seen);
if is_seen
&& let Some(chat_id) = mark_seen_by_uid(context, transport_id, folder, uid_validity, uid)
.await
.with_context(|| {
format!("Transport {transport_id}: Failed to update seen status for msg {folder}/{uid}")
})?
{
updated_chat_ids.insert(chat_id);
}
if let Some(modseq) = fetch.modseq {
if modseq > highest_modseq {
highest_modseq = modseq;
}
} else {
warn!(context, "FETCH result contains no MODSEQ");
}
}
drop(list);
if got_unsolicited_fetch {
// We got unsolicited FETCH, which means some flags
// have been modified while our request was in progress.
// We may or may not have these new flags as a part of the response,
// so better skip next IDLE and do another round of flag synchronization.
info!(context, "Got unsolicited fetch, will skip idle");
self.new_mail = true;
}
set_modseq(context, transport_id, folder, highest_modseq)
.await
.with_context(|| format!("failed to set MODSEQ for folder {folder}"))?;
if !updated_chat_ids.is_empty() {
context.on_archived_chats_maybe_noticed();
}
for updated_chat_id in updated_chat_ids {
context.emit_event(EventType::MsgsNoticed(updated_chat_id));
chatlist_events::emit_chatlist_item_changed(context, updated_chat_id);
}
Ok(())
}
/// Fetches a list of messages by server UID.
///
/// Sends pairs of UID and info about each downloaded message to the provided channel.
@@ -2026,71 +1917,6 @@ pub(crate) async fn prefetch_should_download(
Ok(should_download)
}
/// Marks messages in `msgs` table as seen, searching for them by UID.
///
/// Returns updated chat ID if any message was marked as seen.
async fn mark_seen_by_uid(
context: &Context,
transport_id: u32,
folder: &str,
uid_validity: u32,
uid: u32,
) -> Result<Option<ChatId>> {
if let Some((msg_id, chat_id)) = context
.sql
.query_row_optional(
"SELECT id, chat_id FROM msgs
WHERE id > 9 AND rfc724_mid IN (
SELECT rfc724_mid FROM imap
WHERE transport_id=?
AND folder=?
AND uidvalidity=?
AND uid=?
LIMIT 1
)",
(transport_id, &folder, uid_validity, uid),
|row| {
let msg_id: MsgId = row.get(0)?;
let chat_id: ChatId = row.get(1)?;
Ok((msg_id, chat_id))
},
)
.await
.with_context(|| format!("failed to get msg and chat ID for IMAP message {folder}/{uid}"))?
{
let updated = context
.sql
.execute(
"UPDATE msgs SET state=?1
WHERE (state=?2 OR state=?3)
AND id=?4",
(
MessageState::InSeen,
MessageState::InFresh,
MessageState::InNoticed,
msg_id,
),
)
.await
.with_context(|| format!("failed to update msg {msg_id} state"))?
> 0;
if updated {
msg_id
.start_ephemeral_timer(context)
.await
.with_context(|| format!("failed to start ephemeral timer for message {msg_id}"))?;
Ok(Some(chat_id))
} else {
// Message state has not changed.
Ok(None)
}
} else {
// There is no message is `msgs` table matching the given UID.
Ok(None)
}
}
/// Schedule marking the message as Seen on IMAP by adding all known IMAP messages corresponding to
/// the given Message-ID to `imap_markseen` table.
pub(crate) async fn markseen_on_imap_table(context: &Context, message_id: &str) -> Result<()> {
@@ -2188,17 +2014,6 @@ pub(crate) async fn set_modseq(
Ok(())
}
async fn get_modseq(context: &Context, transport_id: u32, folder: &str) -> Result<u64> {
Ok(context
.sql
.query_get_value(
"SELECT modseq FROM imap_sync WHERE transport_id=? AND folder=?",
(transport_id, folder),
)
.await?
.unwrap_or(0))
}
/// Builds a list of sequence/uid sets. The returned sets have each no more than around 1000
/// characters because according to <https://tools.ietf.org/html/rfc2683#section-3.2.1.5>
/// command lines should not be much more than 1000 chars (servers should allow at least 8000 chars)
-4
View File
@@ -17,10 +17,6 @@ pub(crate) struct Capabilities {
/// <https://tools.ietf.org/html/rfc2087>
pub can_check_quota: bool,
/// True if the server has CONDSTORE capability as defined in
/// <https://tools.ietf.org/html/rfc7162>
pub can_condstore: bool,
/// True if the server has METADATA capability as defined in
/// <https://tools.ietf.org/html/rfc5464>
pub can_metadata: bool,
-1
View File
@@ -66,7 +66,6 @@ pub(crate) async fn determine_capabilities(
can_idle: caps.has_str("IDLE"),
can_move: caps.has_str("MOVE"),
can_check_quota: caps.has_str("QUOTA"),
can_condstore: caps.has_str("CONDSTORE"),
can_metadata: caps.has_str("METADATA"),
can_compress: caps.has_str("COMPRESS=DEFLATE"),
can_push: caps.has_str("XDELTAPUSH"),
+1
View File
@@ -1,4 +1,5 @@
use super::*;
use crate::chat::ChatId;
use crate::contact::Contact;
use crate::test_utils::TestContext;
+1 -5
View File
@@ -65,11 +65,7 @@ impl ImapSession {
self.maybe_close_folder(context).await?;
// select new folder
let res = if self.can_condstore() {
self.select_condstore(folder).await
} else {
self.select(folder).await
};
let res = self.select(folder).await;
let transport_id = self.transport_id();
-4
View File
@@ -102,10 +102,6 @@ impl Session {
self.capabilities.can_check_quota
}
pub fn can_condstore(&self) -> bool {
self.capabilities.can_condstore
}
pub fn can_metadata(&self) -> bool {
self.capabilities.can_metadata
}
-1
View File
@@ -94,7 +94,6 @@ mod smtp;
pub mod stock_str;
pub mod storage_usage;
mod sync;
mod timesmearing;
mod token;
mod transport;
mod update_helper;
+9 -60
View File
@@ -30,13 +30,12 @@ use crate::log::warn;
use crate::mimeparser::{SystemMessage, parse_message_id};
use crate::param::{Param, Params};
use crate::reaction::get_msg_reactions;
use crate::sql;
use crate::summary::Summary;
use crate::sync::SyncData;
use crate::tools::create_outgoing_rfc724_mid;
use crate::tools::{
buf_compress, buf_decompress, get_filebytes, get_filemeta, gm2local_offset, read_file,
sanitize_filename, time, timestamp_to_str,
get_filebytes, get_filemeta, gm2local_offset, read_file, sanitize_filename, time,
timestamp_to_str,
};
/// Message ID, including reserved IDs.
@@ -1624,62 +1623,6 @@ pub(crate) fn guess_msgtype_from_path_suffix(path: &Path) -> Option<(Viewtype, &
Some(info)
}
/// Get the raw mime-headers of the given message.
/// Raw headers are saved for large messages
/// that need a "Show full message..."
/// to see HTML part.
///
/// Returns an empty vector if there are no headers saved for the given message.
pub(crate) async fn get_mime_headers(context: &Context, msg_id: MsgId) -> Result<Vec<u8>> {
let (headers, compressed) = context
.sql
.query_row(
"SELECT mime_headers, mime_compressed FROM msgs WHERE id=?",
(msg_id,),
|row| {
let headers = sql::row_get_vec(row, 0)?;
let compressed: bool = row.get(1)?;
Ok((headers, compressed))
},
)
.await?;
if compressed {
return buf_decompress(&headers);
}
let headers2 = headers.clone();
let compressed = match tokio::task::block_in_place(move || buf_compress(&headers2)) {
Err(e) => {
warn!(context, "get_mime_headers: buf_compress() failed: {}", e);
return Ok(headers);
}
Ok(o) => o,
};
let update = |conn: &mut rusqlite::Connection| {
match conn.execute(
"\
UPDATE msgs SET mime_headers=?, mime_compressed=1 \
WHERE id=? AND mime_headers!='' AND mime_compressed=0",
(compressed, msg_id),
) {
Ok(rows_updated) => ensure!(rows_updated <= 1),
Err(e) => {
warn!(context, "get_mime_headers: UPDATE failed: {}", e);
return Err(e.into());
}
}
Ok(())
};
if let Err(e) = context.sql.call_write(update).await {
warn!(
context,
"get_mime_headers: failed to update mime_headers: {}", e
);
}
Ok(headers)
}
/// Delete a single message from the database, including references in other tables.
/// This may be called in batches; the final events are emitted in delete_msgs_locally_done() then.
pub(crate) async fn delete_msg_locally(context: &Context, msg: &Message) -> Result<()> {
@@ -1936,9 +1879,10 @@ pub async fn markseen_msgs(context: &Context, msg_ids: Vec<MsgId>) -> Result<()>
//
// We also don't send read receipts for contact requests.
// Read receipts will not be sent even after accepting the chat.
let wants_mdn = curr_param.get_bool(Param::WantsMdn).unwrap_or_default();
let to_id = if curr_blocked == Blocked::Not
&& !curr_hidden
&& curr_param.get_bool(Param::WantsMdn).unwrap_or_default()
&& wants_mdn
&& curr_param.get_cmd() == SystemMessage::Unknown
&& context.should_send_mdns().await?
{
@@ -1960,6 +1904,11 @@ pub async fn markseen_msgs(context: &Context, msg_ids: Vec<MsgId>) -> Result<()>
None
};
if let Some(to_id) = to_id {
info!(
context,
"Queuing MDN to {to_id} for {id} from {curr_from_id}, wants_mdn={wants_mdn}, cmd={}.",
curr_param.get_cmd()
);
context
.sql
.execute(
+47 -60
View File
@@ -35,10 +35,7 @@ use crate::peer_channels::{create_iroh_header, get_iroh_topic_for_msg};
use crate::pgp::{SeipdVersion, addresses_from_public_key, pubkey_supports_seipdv2};
use crate::simplify::escape_message_footer_marks;
use crate::stock_str;
use crate::tools::{
IsNoneOrEmpty, create_outgoing_rfc724_mid, create_smeared_timestamp, remove_subject_prefix,
time,
};
use crate::tools::{IsNoneOrEmpty, create_outgoing_rfc724_mid, remove_subject_prefix, time};
use crate::webxdc::StatusUpdateSerial;
// attachments of 25 mb brutto should work on the majority of providers
@@ -167,7 +164,6 @@ pub struct MimeFactory {
#[derive(Debug, Clone)]
pub struct RenderedEmail {
pub message: String,
// pub envelope: Envelope,
pub is_encrypted: bool,
pub last_added_location_id: Option<u32>,
@@ -224,7 +220,10 @@ impl MimeFactory {
let mut member_fingerprints = Vec::new();
let mut member_timestamps = Vec::new();
let mut recipient_ids = HashSet::new();
let mut req_mdn = false;
let req_mdn = !chat.is_self_talk()
&& !msg.is_system_message()
&& msg.param.get_int(Param::Reaction).unwrap_or_default() == 0
&& context.should_request_mdns().await?;
let encryption_pubkeys;
@@ -272,9 +271,7 @@ impl MimeFactory {
let public_key = SignedPublicKey::from_slice(&public_key_bytes)?;
let relays =
addresses_from_public_key(&public_key).unwrap_or_else(|| vec![addr.clone()]);
recipients.extend(relays);
recipients.extend(relay_addrs(&public_key, &addr));
to.push((authname, addr.clone()));
encryption_pubkeys = Some(vec![(addr, public_key)]);
@@ -354,7 +351,7 @@ impl MimeFactory {
};
if add_timestamp >= remove_timestamp {
let relays = if let Some(public_key) = public_key_opt {
let addrs = addresses_from_public_key(&public_key);
let addrs = relay_addrs(&public_key, &addr);
keys.push((addr.clone(), public_key));
addrs
} else if id != ContactId::SELF && !should_encrypt_symmetrically(&msg, &chat) {
@@ -362,10 +359,10 @@ impl MimeFactory {
if is_encrypted {
warn!(context, "Missing key for {addr}");
}
None
vec![addr.clone()]
} else {
None
}.unwrap_or_else(|| vec![addr.clone()]);
vec![addr.clone()]
};
if !recipients_contain_addr(&to, &addr) {
if id != ContactId::SELF {
@@ -394,7 +391,7 @@ impl MimeFactory {
if let Some(email_to_remove) = email_to_remove
&& email_to_remove == addr {
let relays = if let Some(public_key) = public_key_opt {
let addrs = addresses_from_public_key(&public_key);
let addrs = relay_addrs(&public_key, &addr);
keys.push((addr.clone(), public_key));
addrs
} else if id != ContactId::SELF && !should_encrypt_symmetrically(&msg, &chat) {
@@ -402,10 +399,10 @@ impl MimeFactory {
if is_encrypted {
warn!(context, "Missing key for {addr}");
}
None
vec![addr.clone()]
} else {
None
}.unwrap_or_else(|| vec![addr.clone()]);
vec![addr.clone()]
};
// This is a "member removed" message,
// we need to notify removed member
@@ -481,13 +478,6 @@ impl MimeFactory {
ContactId::scaleup_origin(context, &recipient_ids, origin).await?;
}
if !msg.is_system_message()
&& msg.param.get_int(Param::Reaction).unwrap_or_default() == 0
&& context.should_request_mdns().await?
{
req_mdn = true;
}
encryption_pubkeys = if !is_encrypted {
None
} else if should_encrypt_symmetrically(&msg, &chat) {
@@ -580,7 +570,7 @@ impl MimeFactory {
) -> Result<MimeFactory> {
let contact = Contact::get_by_id(context, from_id).await?;
let from_addr = context.get_primary_self_addr().await?;
let timestamp = create_smeared_timestamp(context);
let timestamp = time();
let addr = contact.get_addr().to_string();
let encryption_pubkeys = if from_id == ContactId::SELF {
@@ -1038,16 +1028,12 @@ impl MimeFactory {
is_securejoin_message,
);
let use_std_header_protection = context
.get_config_bool(Config::StdHeaderProtectionComposing)
.await?;
let outer_message = if let Some(encryption_pubkeys) = self.encryption_pubkeys {
let mut message = add_headers_to_encrypted_part(
message,
&unprotected_headers,
hidden_headers,
protected_headers,
use_std_header_protection,
);
// Add gossip headers in chats with multiple recipients
@@ -1244,7 +1230,6 @@ impl MimeFactory {
Ok(RenderedEmail {
message,
// envelope: Envelope::new,
is_encrypted,
last_added_location_id,
sync_ids_to_delete: self.sync_ids_to_delete,
@@ -1813,14 +1798,15 @@ impl MimeFactory {
HeaderDef::IrohGossipTopic.get_headername(),
mail_builder::headers::raw::Raw::new(topic).into(),
));
if let (Some(json), _) = context
.render_webxdc_status_update_object(
msg.id,
StatusUpdateSerial::MIN,
StatusUpdateSerial::MAX,
None,
)
.await?
if !matches!(self.pre_message_mode, PreMessageMode::Pre { .. })
&& let (Some(json), _) = context
.render_webxdc_status_update_object(
msg.id,
StatusUpdateSerial::MIN,
StatusUpdateSerial::MAX,
None,
)
.await?
{
parts.push(context.build_status_update_part(&json));
}
@@ -1949,7 +1935,6 @@ fn add_headers_to_encrypted_part(
unprotected_headers: &[(&'static str, HeaderType<'static>)],
hidden_headers: Vec<(&'static str, HeaderType<'static>)>,
protected_headers: Vec<(&'static str, HeaderType<'static>)>,
use_std_header_protection: bool,
) -> MimePart<'static> {
// Store protected headers in the inner message.
let message = protected_headers
@@ -1965,21 +1950,19 @@ fn add_headers_to_encrypted_part(
message.header(header, value)
});
if use_std_header_protection {
message = unprotected_headers
.iter()
// Structural headers shouldn't be added as "HP-Outer". They are defined in
// <https://www.rfc-editor.org/rfc/rfc9787.html#structural-header-fields>.
.filter(|(name, _)| {
!(name.eq_ignore_ascii_case("mime-version")
|| name.eq_ignore_ascii_case("content-type")
|| name.eq_ignore_ascii_case("content-transfer-encoding")
|| name.eq_ignore_ascii_case("content-disposition"))
})
.fold(message, |message, (name, value)| {
message.header(format!("HP-Outer: {name}"), value.clone())
});
}
message = unprotected_headers
.iter()
// Structural headers shouldn't be added as "HP-Outer". They are defined in
// <https://www.rfc-editor.org/rfc/rfc9787.html#structural-header-fields>.
.filter(|(name, _)| {
!(name.eq_ignore_ascii_case("mime-version")
|| name.eq_ignore_ascii_case("content-type")
|| name.eq_ignore_ascii_case("content-transfer-encoding")
|| name.eq_ignore_ascii_case("content-disposition"))
})
.fold(message, |message, (name, value)| {
message.header(format!("HP-Outer: {name}"), value.clone())
});
// Set the appropriate Content-Type for the inner message
for (h, v) in &mut message.headers {
@@ -1988,9 +1971,7 @@ fn add_headers_to_encrypted_part(
{
let mut ct_new = ct.clone();
ct_new = ct_new.attribute("protected-headers", "v1");
if use_std_header_protection {
ct_new = ct_new.attribute("hp", "cipher");
}
ct_new = ct_new.attribute("hp", "cipher");
*ct = ct_new;
break;
}
@@ -2220,6 +2201,14 @@ async fn build_avatar_file(context: &Context, path: &str) -> Result<String> {
Ok(encoded_body)
}
fn relay_addrs(public_key: &SignedPublicKey, addr: &str) -> Vec<String> {
let mut addrs = addresses_from_public_key(public_key).unwrap_or_default();
if !addrs.iter().any(|r| r == addr) {
addrs.push(addr.to_string());
}
addrs
}
fn recipients_contain_addr(recipients: &[(String, String)], addr: &str) -> bool {
let addr_lc = addr.to_lowercase();
recipients
@@ -2276,7 +2265,7 @@ pub(crate) async fn render_symm_encrypted_securejoin_message(
mail_builder::headers::text::Text::new("Secure-Join".to_string()).into(),
));
let timestamp = create_smeared_timestamp(context);
let timestamp = time();
let date = chrono::DateTime::<chrono::Utc>::from_timestamp(timestamp, 0)
.unwrap()
.to_rfc2822();
@@ -2332,13 +2321,11 @@ pub(crate) async fn render_symm_encrypted_securejoin_message(
);
let outer_message = {
let use_std_header_protection = true;
let message = add_headers_to_encrypted_part(
message,
&unprotected_headers,
hidden_headers,
protected_headers,
use_std_header_protection,
);
// Disable compression for SecureJoin to ensure
+7 -14
View File
@@ -730,21 +730,14 @@ async fn test_hp_outer_headers() -> Result<()> {
let t = &tcm.alice().await;
let chat_id = t.get_self_chat().await.id;
for std_hp_composing in [false, true] {
t.set_config_bool(Config::StdHeaderProtectionComposing, std_hp_composing)
.await?;
chat::send_text_msg(t, chat_id, "hi!".to_string()).await?;
let sent_msg = t.pop_sent_msg().await;
let msg = MimeMessage::from_bytes(t, sent_msg.payload.as_bytes()).await?;
assert_eq!(msg.header_exists(HeaderDef::HpOuter), std_hp_composing);
for hdr in ["Date", "From", "Message-ID"] {
assert_eq!(
msg.decoded_data_contains(&format!("HP-Outer: {hdr}:")),
std_hp_composing,
);
}
assert!(!msg.decoded_data_contains("HP-Outer: Content-Type"));
chat::send_text_msg(t, chat_id, "hi!".to_string()).await?;
let sent_msg = t.pop_sent_msg().await;
let msg = MimeMessage::from_bytes(t, sent_msg.payload.as_bytes()).await?;
assert!(msg.header_exists(HeaderDef::HpOuter));
for hdr in ["Date", "From", "Message-ID"] {
assert!(msg.decoded_data_contains(&format!("HP-Outer: {hdr}:")),);
}
assert!(!msg.decoded_data_contains("HP-Outer: Content-Type"));
Ok(())
}
+12 -6
View File
@@ -31,9 +31,7 @@ use crate::message::{self, Message, MsgId, Viewtype, get_vcard_summary, set_msg_
use crate::param::{Param, Params};
use crate::simplify::{SimplifiedText, simplify};
use crate::sync::SyncItems;
use crate::tools::{
get_filemeta, parse_receive_headers, smeared_time, time, truncate_msg_text, validate_id,
};
use crate::tools::{get_filemeta, parse_receive_headers, time, truncate_msg_text, validate_id};
use crate::{chatlist_events, location, tools};
/// Public key extracted from `Autocrypt-Gossip`
@@ -271,7 +269,7 @@ impl MimeMessage {
pub(crate) async fn from_bytes(context: &Context, body: &[u8]) -> Result<Self> {
let mail = mailparse::parse_mail(body)?;
let timestamp_rcvd = smeared_time(context);
let timestamp_rcvd = time();
let mut timestamp_sent =
Self::get_timestamp_sent(&mail.headers, timestamp_rcvd, timestamp_rcvd);
let hop_info = parse_receive_headers(&mail.get_headers());
@@ -924,14 +922,16 @@ impl MimeMessage {
self.parse_attachments();
// See if an MDN is requested from the other side
let mut wants_mdn = false;
if self.decryption_error.is_none()
&& !self.parts.is_empty()
&& (!self.parts.is_empty() || matches!(&self.pre_message, PreMessageMode::Pre { .. }))
&& let Some(ref dn_to) = self.chat_disposition_notification_to
{
// Check that the message is not outgoing.
let from = &self.from.addr;
if !context.is_self_addr(from).await? {
if from.to_lowercase() == dn_to.addr.to_lowercase() {
wants_mdn = true;
if let Some(part) = self.parts.last_mut() {
part.param.set_int(Param::WantsMdn, 1);
}
@@ -953,7 +953,9 @@ impl MimeMessage {
typ: Viewtype::Text,
..Default::default()
};
if wants_mdn {
part.param.set_int(Param::WantsMdn, 1);
}
if let Some(ref subject) = self.get_subject()
&& !self.has_chat_version()
&& self.webxdc_status_update.is_none()
@@ -2462,6 +2464,9 @@ async fn handle_mdn(
let Some((msg_id, chat_id, has_mdns, is_dup)) = context
.sql
.query_row_optional(
// MDN on a pre-message (message preview) references the post-message. So we can't tell
// if the pre-message or fully downloaded message was seen, but this is on purpose. For
// images this problem is going to be solved by showing thumbnails.
"SELECT
m.id AS msg_id,
c.id AS chat_id,
@@ -2509,6 +2514,7 @@ async fn handle_mdn(
// note(treefit): only matters if it is the last message in chat (but probably too expensive to check, debounce also solves it)
chatlist_events::emit_chatlist_item_changed(context, chat_id);
}
context.emit_event(EventType::MsgReadCountChanged { chat_id, msg_id });
Ok(())
}
+12 -19
View File
@@ -1,6 +1,5 @@
use mailparse::ParsedMail;
use std::mem;
use std::time::Duration;
use super::*;
use crate::{
@@ -1405,23 +1404,17 @@ async fn test_extra_imf_headers() -> Result<()> {
let t = &tcm.alice().await;
let chat_id = t.get_self_chat().await.id;
for std_hp_composing in [false, true] {
t.set_config_bool(Config::StdHeaderProtectionComposing, std_hp_composing)
.await?;
chat::send_text_msg(t, chat_id, "hi!".to_string()).await?;
let sent_msg = t.pop_sent_msg().await;
// Check removal of some nonexistent "Chat-*" header to protect the code from future
// breakages. But headers not prefixed with "Chat-" remain unless a message has standard
// Header Protection.
let payload = sent_msg.payload.replace(
"Message-ID:",
"Chat-Forty-Two: 42\r\nForty-Two: 42\r\nMessage-ID:",
);
let msg = MimeMessage::from_bytes(t, payload.as_bytes()).await?;
assert!(msg.headers.contains_key("chat-version"));
assert!(!msg.headers.contains_key("chat-forty-two"));
assert_ne!(msg.headers.contains_key("forty-two"), std_hp_composing);
}
chat::send_text_msg(t, chat_id, "hi!".to_string()).await?;
let sent_msg = t.pop_sent_msg().await;
// Check that extra headers are ignored with RFC 9788 Header Protection.
let payload = sent_msg.payload.replace(
"Message-ID:",
"Chat-Forty-Two: 42\r\nForty-Two: 42\r\nMessage-ID:",
);
let msg = MimeMessage::from_bytes(t, payload.as_bytes()).await?;
assert!(msg.headers.contains_key("chat-version"));
assert!(!msg.headers.contains_key("chat-forty-two"));
assert!(!msg.headers.contains_key("forty-two"));
Ok(())
}
@@ -1435,7 +1428,7 @@ async fn test_intended_recipient_fingerprint() -> Result<()> {
let chat_id = chat::create_group(t, "").await?;
chat::send_text_msg(t, chat_id, "hi!".to_string()).await?;
assert!(t.pop_sent_msg_opt(Duration::ZERO).await.is_none());
assert!(t.pop_sent_msg_opt().await.is_none());
for (i, member) in members.iter().enumerate() {
let contact = t.add_or_lookup_contact(member).await;
@@ -48,10 +48,9 @@ async fn test_shared_secret_decryption_ex(
let encrypted_msg = pgp::symm_encrypt_message(
plain_text.as_bytes().to_vec(),
signer_key,
secret_for_encryption,
secret_for_encryption.to_string(),
true,
)
.await?;
)?;
let boundary = "boundary123";
let rcvd_mail = format!(
+27 -30
View File
@@ -254,44 +254,41 @@ pub fn pk_validate(
/// Symmetrically encrypt the message.
/// This is used for broadcast channels and for version 2 of the Securejoin protocol.
/// `shared secret` is the secret that will be used for symmetric encryption.
pub async fn symm_encrypt_message(
pub fn symm_encrypt_message(
plain: Vec<u8>,
private_key_for_signing: Option<SignedSecretKey>,
shared_secret: &str,
shared_secret: String,
compress: bool,
) -> Result<String> {
let shared_secret = Password::from(shared_secret.to_string());
let shared_secret = Password::from(shared_secret);
tokio::task::spawn_blocking(move || {
let msg = MessageBuilder::from_bytes("", plain);
let mut rng = thread_rng();
let mut salt = [0u8; 8];
rng.fill(&mut salt[..]);
let s2k = StringToKey::Salted {
hash_alg: HashAlgorithm::default(),
salt,
};
let mut msg = msg.seipd_v2(
&mut rng,
SYMMETRIC_KEY_ALGORITHM,
AeadAlgorithm::Ocb,
ChunkSize::C8KiB,
);
msg.encrypt_with_password(&mut rng, s2k, &shared_secret)?;
let msg = MessageBuilder::from_bytes("", plain);
let mut rng = thread_rng();
let mut salt = [0u8; 8];
rng.fill(&mut salt[..]);
let s2k = StringToKey::Salted {
hash_alg: HashAlgorithm::default(),
salt,
};
let mut msg = msg.seipd_v2(
&mut rng,
SYMMETRIC_KEY_ALGORITHM,
AeadAlgorithm::Ocb,
ChunkSize::C8KiB,
);
msg.encrypt_with_password(&mut rng, s2k, &shared_secret)?;
if let Some(private_key_for_signing) = private_key_for_signing.as_deref() {
let hash_algorithm = private_key_for_signing.hash_alg();
msg.sign(private_key_for_signing, Password::empty(), hash_algorithm);
}
if compress {
msg.compression(CompressionAlgorithm::ZLIB);
}
if let Some(private_key_for_signing) = private_key_for_signing.as_deref() {
let hash_algorithm = private_key_for_signing.hash_alg();
msg.sign(private_key_for_signing, Password::empty(), hash_algorithm);
}
if compress {
msg.compression(CompressionAlgorithm::ZLIB);
}
let encoded_msg = msg.to_armored_string(&mut rng, Default::default())?;
let encoded_msg = msg.to_armored_string(&mut rng, Default::default())?;
Ok(encoded_msg)
})
.await?
Ok(encoded_msg)
}
/// Merges and minimizes OpenPGP certificates.
+10 -8
View File
@@ -826,7 +826,9 @@ UPDATE config SET value=? WHERE keyname='configured_addr' AND value!=?1
}
}
if let Some(ref status_update) = mime_parser.webxdc_status_update {
if let Some(ref status_update) = mime_parser.webxdc_status_update
&& !matches!(mime_parser.pre_message, PreMessageMode::Pre { .. })
{
let can_info_msg;
let instance = if mime_parser
.parts
@@ -1215,6 +1217,8 @@ async fn decide_chat_assignment(
// Most mailboxes have a "Drafts" folder where constantly new emails appear but we don't actually want to show them
info!(context, "Email is probably just a draft (TRASH).");
true
} else if matches!(mime_parser.pre_message, PreMessageMode::Pre { .. }) {
false
} else if mime_parser.webxdc_status_update.is_some() && mime_parser.parts.len() == 1 {
if let Some(part) = mime_parser.parts.first() {
if part.typ == Viewtype::Text && part.msg.is_empty() {
@@ -1848,14 +1852,11 @@ async fn add_parts(
let state = if !mime_parser.incoming {
MessageState::OutDelivered
} else if seen
|| !mime_parser.mdn_reports.is_empty()
|| chat_id_blocked == Blocked::Yes
|| group_changes.silent
} else if seen || !mime_parser.mdn_reports.is_empty() || chat_id_blocked == Blocked::Yes
// No check for `hidden` because only reactions are such and they should be `InFresh`.
{
MessageState::InSeen
} else if mime_parser.from.addr == STATISTICS_BOT_EMAIL {
} else if mime_parser.from.addr == STATISTICS_BOT_EMAIL || group_changes.silent {
MessageState::InNoticed
} else {
MessageState::InFresh
@@ -2079,7 +2080,7 @@ async fn add_parts(
}
}
handle_edit_delete(context, mime_parser, from_id).await?;
handle_edit_delete(context, mime_parser, from_id, &mime_headers).await?;
handle_post_message(context, mime_parser, from_id, state).await?;
if mime_parser.is_system_message == SystemMessage::CallAccepted
@@ -2349,6 +2350,7 @@ async fn handle_edit_delete(
context: &Context,
mime_parser: &MimeMessage,
from_id: ContactId,
mime_headers: &[u8],
) -> Result<()> {
if let Some(rfc724_mid) = mime_parser.get_header(HeaderDef::ChatEdit) {
let Some(original_msg_id) = rfc724_mid_exists(context, rfc724_mid).await? else {
@@ -2382,7 +2384,7 @@ async fn handle_edit_delete(
}
let new_text = part.msg.strip_prefix(EDITED_PREFIX).unwrap_or(&part.msg);
chat::save_text_edit_to_db(context, &mut original_msg, new_text).await?;
chat::save_text_edit_to_db(context, &mut original_msg, new_text, mime_headers).await?;
} else if let Some(rfc724_mid_list) = mime_parser.get_header(HeaderDef::ChatDelete)
&& let Some(part) = mime_parser.parts.first()
{
+8 -4
View File
@@ -1663,8 +1663,8 @@ async fn test_save_mime_headers_off() -> anyhow::Result<()> {
let msg = bob.recv_msg(&alice.pop_sent_msg().await).await;
assert_eq!(msg.get_text(), "hi!");
let mime = message::get_mime_headers(&bob, msg.id).await?;
assert!(mime.is_empty());
let html = msg.id.get_html(&bob).await?;
assert!(html.is_none());
Ok(())
}
@@ -2030,12 +2030,12 @@ async fn test_no_smtp_job_for_self_chat() -> Result<()> {
let chat_id = bob.get_self_chat().await.id;
let mut msg = Message::new_text("Happy birthday to me".to_string());
chat::send_msg(bob, chat_id, &mut msg).await?;
assert!(bob.pop_sent_msg_opt(Duration::ZERO).await.is_none());
assert!(bob.pop_sent_msg_opt().await.is_none());
bob.set_config_bool(Config::BccSelf, true).await?;
let mut msg = Message::new_text("Happy birthday to me".to_string());
chat::send_msg(bob, chat_id, &mut msg).await?;
assert!(bob.pop_sent_msg_opt(Duration::ZERO).await.is_some());
assert!(bob.pop_sent_msg_opt().await.is_some());
Ok(())
}
@@ -4059,6 +4059,8 @@ async fn test_delayed_removal_is_ignored() -> Result<()> {
remove_contact_from_chat(bob, bob_chat_id, bob_contact_fiona).await?;
let remove_msg = bob.pop_sent_msg().await;
SystemTime::shift(Duration::from_secs(1));
// Bob adds new members Dom and Elena, but first addition message is lost.
let dom = &tcm.dom().await;
let elena = &tcm.elena().await;
@@ -4075,6 +4077,8 @@ async fn test_delayed_removal_is_ignored() -> Result<()> {
alice.recv_msg(&add_msg).await;
assert_eq!(get_chat_contacts(alice, chat_id).await?.len(), 4);
SystemTime::shift(Duration::from_secs(1));
// Alice re-adds Fiona.
add_contact_to_chat(alice, chat_id, alice_fiona).await?;
assert_eq!(get_chat_contacts(alice, chat_id).await?.len(), 5);
+6 -12
View File
@@ -454,10 +454,12 @@ async fn inbox_fetch_idle(ctx: &Context, imap: &mut Imap, mut session: Session)
.update_metadata(ctx)
.await
.context("update_metadata")?;
session
.register_token(ctx)
.await
.context("Failed to register push token")?;
if let Err(err) = session.register_token(ctx).await {
warn!(
ctx,
"Transport {transport_id}: Failed to register push token: {err:#}."
);
}
let session = fetch_idle(ctx, imap, session).await?;
Ok(session)
@@ -489,14 +491,6 @@ async fn fetch_idle(ctx: &Context, connection: &mut Imap, mut session: Session)
.await
.context("download_msgs")?;
// Synchronize Seen flags.
session
.sync_seen_flags(ctx, &watch_folder)
.await
.context("sync_seen_flags")
.log_err(ctx)
.ok();
connection.connectivity.set_idle(ctx);
ctx.emit_event(EventType::ImapInboxIdle);
+22
View File
@@ -533,6 +533,18 @@ pub(crate) async fn handle_securejoin_handshake(
warn!(context, "Secure-join denied (bad auth).");
return Ok(HandshakeMessage::Ignore);
}
if Contact::lookup_id_by_addr_ex(
context,
&mime_message.from.addr,
Origin::Unknown,
Some(Blocked::Yes),
)
.await?
.is_some()
{
warn!(context, "Ignoring {step} message: {contact_id} is blocked.");
return Ok(HandshakeMessage::Ignore);
}
let rfc724_mid = create_outgoing_rfc724_mid();
let addr = ContactAddress::new(&mime_message.from.addr)?;
@@ -640,6 +652,10 @@ pub(crate) async fn handle_securejoin_handshake(
if time() < timestamp + VERIFICATION_TIMEOUT_SECONDS {
mark_contact_id_as_verified(context, contact_id, Some(ContactId::SELF)).await?;
}
if sender_contact.blocked {
warn!(context, "Ignoring {step} message: {contact_id} is blocked.");
return Ok(HandshakeMessage::Ignore);
}
contact_id.regossip_keys(context).await?;
// for setup-contact, make Alice's one-to-one chat with Bob visible
// (secure-join-information are shown in the group chat)
@@ -811,6 +827,12 @@ pub(crate) async fn observe_securejoin_on_other_device(
}
mark_contact_id_as_verified(context, contact_id, Some(ContactId::SELF)).await?;
if contact.blocked && step != SecureJoinStep::MemberAdded {
// Contact might be blocked after another device had issued the message. Still, to avoid
// membership inconsistency on devices, don't ignore "vg-member-added".
warn!(context, "Observing {step}: {contact_id} is blocked.");
return Ok(HandshakeMessage::Ignore);
}
if matches!(
step,
+2 -2
View File
@@ -19,7 +19,7 @@ use crate::securejoin::{
};
use crate::stock_str;
use crate::sync::Sync::*;
use crate::tools::{create_outgoing_rfc724_mid, smeared_time, time};
use crate::tools::{create_outgoing_rfc724_mid, time};
use crate::{chatlist_events, mimefactory};
/// Starts the securejoin protocol with the QR `invite`.
@@ -465,7 +465,7 @@ async fn joining_chat_id(
name,
Blocked::Not,
None,
smeared_time(context),
time(),
)
.await?
}
+2 -2
View File
@@ -950,7 +950,7 @@ async fn test_parallel_setup_contact(bob_deletes_fiona_contact: bool) -> Result<
Contact::delete(bob, bob_fiona_contact_id).await?;
bob.recv_msg_trash(&sent_fiona_vc_auth_required).await;
let sent = bob.pop_sent_msg_opt(Duration::ZERO).await;
let sent = bob.pop_sent_msg_opt().await;
assert!(sent.is_none());
} else {
bob.recv_msg_trash(&sent_fiona_vc_auth_required).await;
@@ -1235,7 +1235,7 @@ async fn test_rejoin_group() -> Result<()> {
assert_eq!(progress, 1000);
// Bob does not send any more messages by scanning the QR code.
assert!(bob.pop_sent_msg_opt(Duration::ZERO).await.is_none());
assert!(bob.pop_sent_msg_opt().await.is_none());
Ok(())
}
+13 -16
View File
@@ -40,31 +40,28 @@ impl Smtp {
}
let message_len_bytes = message.len();
let recipients_display = recipients
.iter()
.map(|x| x.as_ref())
.collect::<Vec<&str>>()
.join(",");
let envelope =
Envelope::new(self.from.clone(), recipients.to_vec()).map_err(Error::Envelope)?;
let mail = SendableEmail::new(envelope, message);
if let Some(ref mut transport) = self.transport {
transport.send(mail).await.map_err(Error::SmtpSend)?;
let info_msg =
format!("Message len={message_len_bytes} was SMTP-sent to {recipients_display}");
info!(context, "{info_msg}.");
context.emit_event(EventType::SmtpMessageSent(info_msg));
self.last_success = Some(tools::Time::now());
} else {
let Some(ref mut transport) = self.transport else {
warn!(
context,
"uh? SMTP has no transport, failed to send to {}", recipients_display
"Failed to send a message because SMTP client has no SmtpTransport."
);
return Err(Error::NoTransport);
}
};
transport.send(mail).await.map_err(Error::SmtpSend)?;
let info_msg = format!(
"Message len={message_len_bytes} was SMTP-sent to {} recipients.",
recipients.len()
);
info!(context, "{info_msg}.");
context.emit_event(EventType::SmtpMessageSent(info_msg));
self.last_success = Some(tools::Time::now());
Ok(())
}
}
+32 -36
View File
@@ -8,7 +8,7 @@ use std::ops::{Deref, DerefMut};
use std::panic;
use std::path::Path;
use std::sync::{Arc, LazyLock};
use std::time::{Duration, Instant};
use std::time::Duration;
use anyhow::Result;
use async_channel::{self as channel, Receiver, Sender};
@@ -272,6 +272,7 @@ impl TestContextManager {
/// Executes SecureJoin initiated by `joiner`
/// scanning `qr` generated by one of the `inviters` devices.
/// `inviters` devices must have the same primary address.
/// All of the `inviters` devices will get the messages and send replies.
///
/// The [`ChatId`] of the created chat is returned, for a SetupContact QR this is the 1:1
@@ -282,9 +283,11 @@ impl TestContextManager {
inviters: &[&TestContext],
qr: &str,
) -> ChatId {
assert!(joiner.pop_sent_msg_opt(Duration::ZERO).await.is_none());
assert!(joiner.pop_sent_msg_opt().await.is_none());
let inviter_addr = inviters[0].get_primary_self_addr().await.unwrap();
for inviter in inviters {
assert!(inviter.pop_sent_msg_opt(Duration::ZERO).await.is_none());
assert!(inviter.pop_sent_msg_opt().await.is_none());
assert_eq!(inviter.get_primary_self_addr().await.unwrap(), inviter_addr);
}
let chat_id = join_securejoin(&joiner.ctx, qr).await.unwrap();
@@ -292,14 +295,22 @@ impl TestContextManager {
for _ in 0..2 {
let mut something_sent = false;
let rev_order = false;
if let Some(sent) = joiner.pop_sent_msg_ex(rev_order, Duration::ZERO).await {
if let Some(sent) = joiner.pop_sent_msg_ex(rev_order).await {
for inviter in inviters {
inviter.recv_msg_opt(&sent).await;
}
something_sent = true;
}
for inviter in inviters {
if let Some(sent) = inviter.pop_sent_msg_ex(rev_order, Duration::ZERO).await {
if let Some(sent) = inviter.pop_sent_msg_ex(rev_order).await {
if sent.recipients.split(' ').any(|addr| addr == inviter_addr) {
for observer in inviters {
// `imap::prefetch_should_download()` returns false on the sender side.
if observer.get_id() != inviter.get_id() {
observer.recv_msg_opt(&sent).await;
}
}
}
joiner.recv_msg_opt(&sent).await;
something_sent = true;
}
@@ -638,22 +649,17 @@ impl TestContext {
///
/// Panics if there is no message or on any error.
pub async fn pop_sent_msg(&self) -> SentMessage<'_> {
self.pop_sent_msg_opt(Duration::from_secs(3))
self.pop_sent_msg_opt()
.await
.expect("no sent message found in jobs table")
}
pub async fn pop_sent_msg_opt(&self, timeout: Duration) -> Option<SentMessage<'_>> {
pub async fn pop_sent_msg_opt(&self) -> Option<SentMessage<'_>> {
let rev_order = true;
self.pop_sent_msg_ex(rev_order, timeout).await
self.pop_sent_msg_ex(rev_order).await
}
pub async fn pop_sent_msg_ex(
&self,
rev_order: bool,
timeout: Duration,
) -> Option<SentMessage<'_>> {
let start = Instant::now();
pub async fn pop_sent_msg_ex(&self, rev_order: bool) -> Option<SentMessage<'_>> {
let mut query = "
SELECT id, msg_id, mime, recipients
FROM smtp
@@ -662,28 +668,18 @@ ORDER BY id"
if rev_order {
query += " DESC";
}
let (rowid, msg_id, payload, recipients) = loop {
let row = self
.ctx
.sql
.query_row_optional(&query, (), |row| {
let rowid: i64 = row.get(0)?;
let msg_id: MsgId = row.get(1)?;
let mime: String = row.get(2)?;
let recipients: String = row.get(3)?;
Ok((rowid, msg_id, mime, recipients))
})
.await
.expect("query_row_optional failed");
if let Some(row) = row {
break row;
}
if start.elapsed() < timeout {
tokio::time::sleep(Duration::from_millis(100)).await;
} else {
return None;
}
};
let (rowid, msg_id, payload, recipients) = self
.ctx
.sql
.query_row_optional(&query, (), |row| {
let rowid: i64 = row.get(0)?;
let msg_id: MsgId = row.get(1)?;
let mime: String = row.get(2)?;
let recipients: String = row.get(3)?;
Ok((rowid, msg_id, mime, recipients))
})
.await
.expect("query_row_optional failed")?;
self.ctx
.sql
.execute("DELETE FROM smtp WHERE id=?;", (rowid,))
+1 -1
View File
@@ -34,7 +34,7 @@ async fn test_additional_text_on_different_viewtypes() -> Result<()> {
let (pre_message, _, _) = send_large_image_message(alice, a_group_id).await?;
let msg = bob.recv_msg(&pre_message).await;
assert_eq!(msg.text, "test".to_owned());
assert_eq!(msg.get_text(), "test [Image – 146.12 KiB]".to_owned());
assert_eq!(msg.get_text(), "test [Image – 228.45 KiB]".to_owned());
Ok(())
}
+1 -7
View File
@@ -1,5 +1,4 @@
//! Tests about forwarding and saving Pre-Messages
use std::time::Duration;
use anyhow::Result;
use pretty_assertions::assert_eq;
@@ -108,12 +107,7 @@ async fn test_receive_both() -> Result<()> {
forward_msgs(alice, &[alice_msg_id], alice_chat_id).await?;
let rev_order = false;
let msg = bob
.recv_msg(
&alice
.pop_sent_msg_ex(rev_order, Duration::ZERO)
.await
.unwrap(),
)
.recv_msg(&alice.pop_sent_msg_ex(rev_order).await.unwrap())
.await;
assert_eq!(msg.download_state(), DownloadState::Available);
assert_eq!(msg.is_forwarded(), true);
+126 -4
View File
@@ -261,6 +261,61 @@ async fn test_lost_pre_msg() -> Result<()> {
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_pre_msg_mdn_before_sending_full() -> Result<()> {
pre_msg_mdn_before_sending_full("").await
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_pre_msg_mdn_before_sending_full_with_text() -> Result<()> {
pre_msg_mdn_before_sending_full("text").await
}
async fn pre_msg_mdn_before_sending_full(text: &str) -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
let bob = &tcm.bob().await;
let alice_chat_id = alice.create_group_with_members("", &[bob]).await;
let file_bytes = include_bytes!("../../../test-data/image/screenshot.gif");
let mut msg = Message::new(Viewtype::Image);
msg.set_file_from_bytes(alice, "a.jpg", file_bytes, None)?;
msg.set_text(text.to_string());
chat::send_msg(alice, alice_chat_id, &mut msg).await?;
let rev_order = false;
let pre_msg = alice.pop_sent_msg_ex(rev_order).await.unwrap();
let alice_msg_id = msg.id;
let msg = bob.recv_msg(&pre_msg).await;
assert_eq!(msg.download_state, DownloadState::Available);
assert_eq!(msg.id.get_state(bob).await?, MessageState::InFresh);
assert_eq!(msg.text, text);
assert!(msg.param.get_bool(Param::WantsMdn).unwrap_or_default());
msg.chat_id.accept(bob).await?;
markseen_msgs(bob, vec![msg.id]).await?;
assert_eq!(msg.id.get_state(bob).await?, MessageState::InSeen);
assert_eq!(
bob.sql
.count(
"SELECT COUNT(*) FROM smtp_mdns WHERE from_id=?",
(msg.from_id,)
)
.await?,
1
);
assert_eq!(
alice_msg_id.get_state(alice).await?,
MessageState::OutPending
);
let _full_msg = alice.pop_sent_msg().await;
assert_eq!(
alice_msg_id.get_state(alice).await?,
MessageState::OutDelivered
);
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_post_msg_bad_sender() -> Result<()> {
let mut tcm = TestContextManager::new();
@@ -402,9 +457,9 @@ async fn test_receive_pre_message_image() -> Result<()> {
// test that metadata is correctly returned by methods
assert_eq!(msg.get_post_message_viewtype(), Some(Viewtype::Image));
// recoded image dimensions
assert_eq!(msg.get_filebytes(bob).await?, Some(149632));
assert_eq!(msg.get_height(), 1280);
assert_eq!(msg.get_width(), 720);
assert_eq!(msg.get_filebytes(bob).await?, Some(233935));
assert_eq!(msg.get_height(), 1704);
assert_eq!(msg.get_width(), 959);
Ok(())
}
@@ -534,6 +589,17 @@ async fn test_webxdc_updates_in_post_message_after_pre_message() -> Result<()> {
let alice_chat_id = alice.create_chat_id(bob).await;
// regression test where updates get assigned to an unrelated prior webxdc message
let mut unrelated_xdc = Message::new(Viewtype::Webxdc);
unrelated_xdc.set_file_from_bytes(
alice,
"first.xdc",
include_bytes!("../../../test-data/webxdc/minimal.xdc"),
None,
)?;
send_msg(alice, alice_chat_id, &mut unrelated_xdc).await?;
let bob_unrelated_webxdc = bob.recv_msg(&alice.pop_sent_msg().await).await;
let big_webxdc_app = big_webxdc_app().await?;
let mut alice_instance = Message::new(Viewtype::Webxdc);
@@ -552,6 +618,14 @@ async fn test_webxdc_updates_in_post_message_after_pre_message() -> Result<()> {
let bob_instance = bob.recv_msg(&pre_message).await;
assert_eq!(bob_instance.download_state, DownloadState::Available);
// don't accidentally assign updates from a pre-message to parent message
assert_eq!(
bob.get_webxdc_status_updates(bob_unrelated_webxdc.id, StatusUpdateSerial::new(0))
.await?,
"[]"
);
bob.recv_msg_trash(&post_message).await;
let bob_instance = Message::load_from_db(bob, bob_instance.id).await?;
assert_eq!(bob_instance.download_state, DownloadState::Done);
@@ -565,6 +639,51 @@ async fn test_webxdc_updates_in_post_message_after_pre_message() -> Result<()> {
Ok(())
}
/// Tests sending large webxdc without text.
///
/// This is a regression test, previously pre-message
/// was trashed when it had no text.
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_large_webxdc_without_text() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
let bob = &tcm.bob().await;
tcm.section("Bob sends large webxdc without attached text message.");
let bob_chat_id = bob.create_chat_id(alice).await;
let big_webxdc_app = big_webxdc_app().await?;
let mut bob_instance = Message::new(Viewtype::Webxdc);
bob_instance.set_file_from_bytes(bob, "test.xdc", &big_webxdc_app, None)?;
bob_chat_id.set_draft(bob, Some(&mut bob_instance)).await?;
bob.send_webxdc_status_update(bob_instance.id, r#"{"payload":42, "info":"i"}"#)
.await?;
send_msg(bob, bob_chat_id, &mut bob_instance).await?;
let post_message = bob.pop_sent_msg().await;
let pre_message = bob.pop_sent_msg().await;
tcm.section("Alice receives a pre-message");
let alice_instance = alice.recv_msg(&pre_message).await;
assert_eq!(alice_instance.download_state, DownloadState::Available);
tcm.section("Alice receives a post-message");
alice.recv_msg_trash(&post_message).await;
let alice_instance = Message::load_from_db(alice, alice_instance.id).await?;
assert_eq!(alice_instance.download_state, DownloadState::Done);
let alice_file_path = alice_instance.get_file(alice).expect("No file");
tokio::fs::try_exists(alice_file_path).await?;
assert_eq!(
alice
.get_webxdc_status_updates(alice_instance.id, StatusUpdateSerial::new(0))
.await?,
r#"[{"payload":42,"info":"i","serial":1,"max_serial":1}]"#
);
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_webxdc_updates_in_post_message_after_deleted_pre_message() -> Result<()> {
let mut tcm = TestContextManager::new();
@@ -679,7 +798,10 @@ async fn test_markseen_pre_msg() -> Result<()> {
assert_eq!(
alice
.sql
.count("SELECT COUNT(*) FROM smtp_mdns", ())
.count(
"SELECT COUNT(*) FROM smtp_mdns WHERE from_id=?",
(msg.from_id,)
)
.await?,
1
);
-194
View File
@@ -1,194 +0,0 @@
//! # Time smearing.
//!
//! As e-mails typically only use a second-based-resolution for timestamps,
//! the order of two mails sent within one second is unclear.
//! This is bad e.g. when forwarding some messages from a chat -
//! these messages will appear at the recipient easily out of order.
//!
//! We work around this issue by not sending out two mails with the same timestamp.
//! For this purpose, in short, we track the last timestamp used in `last_smeared_timestamp`
//! when another timestamp is needed in the same second, we use `last_smeared_timestamp+1`
//! after some moments without messages sent out,
//! `last_smeared_timestamp` is again in sync with the normal time.
//!
//! However, we do not do all this for the far future,
//! but at max `MAX_SECONDS_TO_LEND_FROM_FUTURE`
use std::cmp::{max, min};
use std::sync::atomic::{AtomicI64, Ordering};
pub(crate) const MAX_SECONDS_TO_LEND_FROM_FUTURE: i64 = 30;
/// Smeared timestamp generator.
#[derive(Debug)]
pub struct SmearedTimestamp {
/// Next timestamp available for allocation.
smeared_timestamp: AtomicI64,
}
impl SmearedTimestamp {
/// Creates a new smeared timestamp generator.
pub fn new() -> Self {
Self {
smeared_timestamp: AtomicI64::new(0),
}
}
/// Allocates `count` unique timestamps.
///
/// Returns the first allocated timestamp.
#[expect(clippy::arithmetic_side_effects)]
pub fn create_n(&self, now: i64, count: i64) -> i64 {
let mut prev = self.smeared_timestamp.load(Ordering::Relaxed);
loop {
// Advance the timestamp if it is in the past,
// but keep `count - 1` timestamps from the past if possible.
let t = max(prev, now - count + 1);
// Rewind the time back if there is no room
// to allocate `count` timestamps without going too far into the future.
// Not going too far into the future
// is more important than generating unique timestamps.
let first = min(t, now + MAX_SECONDS_TO_LEND_FROM_FUTURE - count + 1);
// Allocate `count` timestamps by advancing the current timestamp.
let next = first + count;
if let Err(x) = self.smeared_timestamp.compare_exchange_weak(
prev,
next,
Ordering::Relaxed,
Ordering::Relaxed,
) {
prev = x;
} else {
return first;
}
}
}
/// Creates a single timestamp.
pub fn create(&self, now: i64) -> i64 {
self.create_n(now, 1)
}
/// Returns the current smeared timestamp.
pub fn current(&self) -> i64 {
self.smeared_timestamp.load(Ordering::Relaxed)
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::test_utils::TestContext;
use crate::tools::{
SystemTime, create_smeared_timestamp, create_smeared_timestamps, smeared_time, time,
};
#[test]
fn test_smeared_timestamp() {
let smeared_timestamp = SmearedTimestamp::new();
let now = time();
assert_eq!(smeared_timestamp.current(), 0);
for i in 0..MAX_SECONDS_TO_LEND_FROM_FUTURE {
assert_eq!(smeared_timestamp.create(now), now + i);
}
assert_eq!(
smeared_timestamp.create(now),
now + MAX_SECONDS_TO_LEND_FROM_FUTURE
);
assert_eq!(
smeared_timestamp.create(now),
now + MAX_SECONDS_TO_LEND_FROM_FUTURE
);
// System time rewinds back by 1000 seconds.
let now = now - 1000;
assert_eq!(
smeared_timestamp.create(now),
now + MAX_SECONDS_TO_LEND_FROM_FUTURE
);
assert_eq!(
smeared_timestamp.create(now),
now + MAX_SECONDS_TO_LEND_FROM_FUTURE
);
assert_eq!(
smeared_timestamp.create(now + 1),
now + MAX_SECONDS_TO_LEND_FROM_FUTURE + 1
);
assert_eq!(smeared_timestamp.create(now + 100), now + 100);
assert_eq!(smeared_timestamp.create(now + 100), now + 101);
assert_eq!(smeared_timestamp.create(now + 100), now + 102);
}
#[test]
fn test_create_n_smeared_timestamps() {
let smeared_timestamp = SmearedTimestamp::new();
let now = time();
// Create a single timestamp to initialize the generator.
assert_eq!(smeared_timestamp.create(now), now);
// Wait a minute.
let now = now + 60;
// Simulate forwarding 7 messages.
let forwarded_messages = 7;
// We have not sent anything for a minute,
// so we can take the current timestamp and take 6 timestamps from the past.
assert_eq!(smeared_timestamp.create_n(now, forwarded_messages), now - 6);
assert_eq!(smeared_timestamp.current(), now + 1);
// Wait 4 seconds.
// Now we have 3 free timestamps in the past.
let now = now + 4;
assert_eq!(smeared_timestamp.current(), now - 3);
// Forward another 7 messages.
// We can only lend 3 timestamps from the past.
assert_eq!(smeared_timestamp.create_n(now, forwarded_messages), now - 3);
// We had to borrow 3 timestamps from the future
// because there were not enough timestamps in the past.
assert_eq!(smeared_timestamp.current(), now + 4);
// Forward another 32 messages.
// We cannot use more than 30 timestamps from the future,
// so we use 30 timestamps from the future,
// the current timestamp and one timestamp from the past.
assert_eq!(smeared_timestamp.create_n(now, 32), now - 1);
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_create_smeared_timestamp() {
let t = TestContext::new().await;
assert_ne!(create_smeared_timestamp(&t), create_smeared_timestamp(&t));
assert!(
create_smeared_timestamp(&t)
>= SystemTime::now()
.duration_since(SystemTime::UNIX_EPOCH)
.unwrap()
.as_secs() as i64
);
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_create_smeared_timestamps() {
let t = TestContext::new().await;
let count = MAX_SECONDS_TO_LEND_FROM_FUTURE - 1;
let start = create_smeared_timestamps(&t, count as usize);
let next = smeared_time(&t);
assert!((start + count - 1) < next);
let count = MAX_SECONDS_TO_LEND_FROM_FUTURE + 30;
let start = create_smeared_timestamps(&t, count as usize);
let next = smeared_time(&t);
assert!((start + count - 1) < next);
}
}
-23
View File
@@ -180,29 +180,6 @@ pub(crate) fn gm2local_offset() -> i64 {
i64::from(lt.offset().local_minus_utc())
}
/// Returns the current smeared timestamp,
///
/// The returned timestamp MAY NOT be unique and MUST NOT go to "Date" header.
pub(crate) fn smeared_time(context: &Context) -> i64 {
let now = time();
let ts = context.smeared_timestamp.current();
std::cmp::max(ts, now)
}
/// Returns a timestamp that is guaranteed to be unique.
pub(crate) fn create_smeared_timestamp(context: &Context) -> i64 {
let now = time();
context.smeared_timestamp.create(now)
}
// creates `count` timestamps that are guaranteed to be unique.
// the first created timestamps is returned directly,
// get the other timestamps just by adding 1..count-1
pub(crate) fn create_smeared_timestamps(context: &Context, count: usize) -> i64 {
let now = time();
context.smeared_timestamp.create_n(now, count as i64)
}
/// Returns the last release timestamp as a unix timestamp compatible for comparison with time() and
/// database times.
pub fn get_release_timestamp() -> i64 {
+2 -2
View File
@@ -46,7 +46,7 @@ use crate::mimefactory::RECOMMENDED_FILE_SIZE;
use crate::mimeparser::SystemMessage;
use crate::param::Param;
use crate::param::Params;
use crate::tools::{create_id, create_smeared_timestamp, get_abs_path};
use crate::tools::{create_id, get_abs_path, time};
/// The current API version.
/// If `min_api` in manifest.toml is set to a larger value,
@@ -558,7 +558,7 @@ impl Context {
.create_status_update_record(
&instance,
status_update,
create_smeared_timestamp(self),
time(),
send_now,
ContactId::SELF,
)
+1 -2
View File
@@ -145,7 +145,6 @@ mod tests {
use crate::message::{Message, Viewtype};
use crate::test_utils::TestContext;
use anyhow::Result;
use std::time::Duration;
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_default_integrations_are_single_device() -> Result<()> {
@@ -158,7 +157,7 @@ mod tests {
t.set_webxdc_integration(file.to_str().unwrap()).await?;
// default integrations are shipped with the apps and should not be sent over the wire
let sent = t.pop_sent_msg_opt(Duration::from_secs(1)).await;
let sent = t.pop_sent_msg_opt().await;
assert!(sent.is_none());
Ok(())
-1
View File
@@ -13,7 +13,6 @@ Filename encoding | Encoded Words ([RFC 2047][]), Encoded Word Ex
Identify server folders | IMAP LIST Extension ([RFC 6154][])
Push | IMAP IDLE ([RFC 2177][])
Quota | IMAP QUOTA extension ([RFC 2087][])
Seen status synchronization | IMAP CONDSTORE extension ([RFC 7162][])
Client/server identification | IMAP ID extension ([RFC 2971][])
Authorization | OAuth2 ([RFC 6749][])
End-to-end encryption | [Autocrypt Level 1][], OpenPGP ([RFC 4880][]), Security Multiparts for MIME ([RFC 1847][]) and [“Mixed Up” Encryption repairing](https://datatracker.ietf.org/doc/html/draft-dkg-openpgp-pgpmime-message-mangling-00)
+2 -2
View File
@@ -1,7 +1,7 @@
OutBroadcast#Chat#1001: Channel [0 member(s)]
--------------------------------------------------------------------------------
Msg#1001: info (Contact#Contact#Info): Messages are end-to-end encrypted. [NOTICED][INFO]
Msg#1008🔒: Me (Contact#Contact#Self): hi √
Msg#1006🔒: Me (Contact#Contact#Self): Member bob@example.net added. [INFO] √
Msg#1009🔒: Me (Contact#Contact#Self): You removed member bob@example.net. [INFO] √
Msg#1009🔒: Me (Contact#Contact#Self): hi √
Msg#1010🔒: Me (Contact#Contact#Self): You removed member bob@example.net. [INFO] √
--------------------------------------------------------------------------------
+2 -2
View File
@@ -2,6 +2,6 @@ OutBroadcast#Chat#1001: Channel [0 member(s)]
--------------------------------------------------------------------------------
Msg#1002: info (Contact#Contact#Info): Messages are end-to-end encrypted. [NOTICED][INFO]
Msg#1006🔒: Me (Contact#Contact#Self): Member bob@example.net added. [INFO] √
Msg#1008🔒: Me (Contact#Contact#Self): hi √
Msg#1009🔒: Me (Contact#Contact#Self): You removed member bob@example.net. [INFO] √
Msg#1009🔒: Me (Contact#Contact#Self): hi √
Msg#1010🔒: Me (Contact#Contact#Self): You removed member bob@example.net. [INFO] √
--------------------------------------------------------------------------------