Compare commits

..
Author SHA1 Message Date
iequidoo 974e32dd76 fix: Don't decrease member add/remove timestamps if they aren't far away in the future
We shouldn't decrease `add_timestamp` and `remove_timestamp` in the `chats_contacts` table normally,
even if remote changes arrive reordered. This particularly makes sense for ad-hoc groups (see
`chat::update_chat_contacts_table()` in `apply_group_changes()`) and in case if we join an encrypted
group which we were a member of before (see `chat::add_to_chat_contacts_table()` call).

Still, limit already stored timestamps in case local clock was in the future and is set back
now. But our clock may be slow, so limit stored timestamps with a remote timestamp if it's
bigger.

NB: `receive_imf::update_chats_contacts_timestamps()` already only increases timestamps, but it's
used only for handling of the "Chat-Group-Member-Timestamps" header, i.e. for encrypted groups.
2026-06-07 21:45:47 -03:00
82 changed files with 1198 additions and 1284 deletions
+2 -2
View File
@@ -62,7 +62,7 @@ jobs:
with:
show-progress: false
persist-credentials: false
- uses: EmbarkStudios/cargo-deny-action@bb137d7af7e4fb67e5f82a49c4fce4fad40782fe
- uses: EmbarkStudios/cargo-deny-action@a531616d8ce3b9177443e48a1159bc945a099823
with:
arguments: --workspace --all-features --locked
command: check
@@ -146,7 +146,7 @@ jobs:
cache-bin: false
- name: Install nextest
uses: taiki-e/install-action@15449e3094499af05d8d964a1c884208e4b8b595
uses: taiki-e/install-action@60ae4ce63c7aeb6e96d7f572c1ec7fafbb17ca80
with:
tool: nextest
-77
View File
@@ -1,80 +1,5 @@
# Changelog
## [2.53.0] - 2026-06-15
### Features / Changes
- Make quality of images sent in chats more consistent between images with different aspect ratio.
- `MsgId::get_html`: Make only one db query.
- Do not log the recipient list for sent messages.
### Fixes
- Do not trash pre-messages without text but with a webxdc update.
- Don't send or process webxdc status updates in pre-messages.
- Ignore SecureJoin messages from blocked contacts ([#8295](https://github.com/chatmail/core/pull/8295)).
- Do not abort IMAP connection if setting the push token fails.
### Documentation
- STYLE.md: Require to list columns explicitly in `INSERT` statements.
### Build system
- nix: switch to the "master" branch for naersk.
- flake.nix: Use hostPlatform.rust.rustcTarget instead of hardcoding it.
### Miscellaneous Tasks
- Bump version to 2.52.0-dev.
- deps: bump taiki-e/install-action from 2.79.10 to 2.81.1.
- deps: bump EmbarkStudios/cargo-deny-action from 2.0.19 to 2.0.20.
- Bump version to 2.53.0-dev.
### Refactor
- Move the definition of the `target_wh`-variable.
- Remove timesmearing.
### Tests
- Print multiline chat descriptions with debug formatter.
- `exec_securejoin_qr_multi_device()`: Make inviter devices receive each other messages.
- Fixup the tests after removing timesmearing.
- Remove timeout from `pop_sent_msg_ex()`.
## [2.52.0] - 2026-06-09
### Fixes
- Update the channel title after joining if the QR code included a wrong title ([#8260](https://github.com/chatmail/core/pull/8260)).
- Don't send removal message to contact that hasn't been a chat member ([#8298](https://github.com/chatmail/core/pull/8298)).
### Features / Changes
- Add cryptography-related statistics (`number_of_transports`, `key_version`, `key_algorithm`, `pubkey_size`, `number_of_keys`) ([#8293](https://github.com/chatmail/core/pull/8293), [#8297](https://github.com/chatmail/core/pull/8297)).
- Add IMAP folder to `Context::get_info()` ([#8285](https://github.com/chatmail/core/pull/8285)).
### Miscellaneous Tasks
- Update preloaded DNS cache.
- Use default aws-lc-rs cryptography provider for rustls.
- Add exception for unmaintained proc-macro-error2 to deny.toml.
- cargo: bump `pin-project` from 1.1.11 to 1.1.13.
- cargo: bump `tokio` from 1.52.1 to 1.52.3.
- cargo: bump `log` from 0.4.29 to 0.4.30.
- cargo: bump `serde_json` from 1.0.149 to 1.0.150.
- deps: bump EmbarkStudios/cargo-deny-action from 2.0.18 to 2.0.19.
- deps: bump taiki-e/install-action from 2.79.2 to 2.79.10.
### Build system
- nix: fix windows cross-compilation by adding pthreads includes.
### Refactor
- Remove support for building "source" packages for deltachat-rpc-server.
## [2.51.0] - 2026-05-29
### Features / Changes
@@ -8372,5 +8297,3 @@ https://github.com/chatmail/core/pulls?q=is%3Apr+is%3Aclosed
[2.49.0]: https://github.com/chatmail/core/compare/v2.48.0..v2.49.0
[2.50.0]: https://github.com/chatmail/core/compare/v2.49.0..v2.50.0
[2.51.0]: https://github.com/chatmail/core/compare/v2.50.0..v2.51.0
[2.52.0]: https://github.com/chatmail/core/compare/v2.51.0..v2.52.0
[2.53.0]: https://github.com/chatmail/core/compare/v2.52.0..v2.53.0
Generated
+20 -42
View File
@@ -1350,7 +1350,7 @@ dependencies = [
[[package]]
name = "deltachat"
version = "2.54.0-dev"
version = "2.51.0-dev"
dependencies = [
"anyhow",
"astral-tokio-tar",
@@ -1459,7 +1459,7 @@ dependencies = [
[[package]]
name = "deltachat-jsonrpc"
version = "2.54.0-dev"
version = "2.51.0-dev"
dependencies = [
"anyhow",
"async-channel 2.5.0",
@@ -1480,7 +1480,7 @@ dependencies = [
[[package]]
name = "deltachat-repl"
version = "2.54.0-dev"
version = "2.51.0-dev"
dependencies = [
"anyhow",
"deltachat",
@@ -1496,7 +1496,7 @@ dependencies = [
[[package]]
name = "deltachat-rpc-server"
version = "2.54.0-dev"
version = "2.51.0-dev"
dependencies = [
"anyhow",
"deltachat",
@@ -1525,7 +1525,7 @@ dependencies = [
[[package]]
name = "deltachat_ffi"
version = "2.54.0-dev"
version = "2.51.0-dev"
dependencies = [
"anyhow",
"deltachat",
@@ -1710,7 +1710,7 @@ dependencies = [
"libc",
"option-ext",
"redox_users",
"windows-sys 0.61.1",
"windows-sys 0.59.0",
]
[[package]]
@@ -2733,7 +2733,7 @@ dependencies = [
"hyper",
"libc",
"pin-project-lite",
"socket2 0.6.3",
"socket2 0.5.9",
"tokio",
"tower-service",
"tracing",
@@ -3154,7 +3154,7 @@ dependencies = [
"iroh-metrics-derive",
"itoa",
"serde",
"snafu 0.8.5",
"snafu",
"tracing",
]
@@ -3524,9 +3524,9 @@ checksum = "490cc448043f947bae3cbee9c203358d62dbee0db12107a74be5c30ccfd09771"
[[package]]
name = "memchr"
version = "2.8.2"
version = "2.7.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "88904434abc2901f197fe8cc55f0445e7ded921dba5911dad2e2b39b48e663c4"
checksum = "78ca9ab1a0babb1e7d5695e3530886289c18cf2f87ec19a575a0abdce112e3a3"
[[package]]
name = "mime"
@@ -3792,7 +3792,7 @@ dependencies = [
"netlink-proto",
"netlink-sys",
"serde",
"snafu 0.8.5",
"snafu",
"socket2 0.5.9",
"time",
"tokio",
@@ -3880,7 +3880,7 @@ version = "0.50.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7957b9740744892f114936ab4a57b3f487491bbeafaf8083688b16841a4240e5"
dependencies = [
"windows-sys 0.61.1",
"windows-sys 0.59.0",
]
[[package]]
@@ -4284,9 +4284,9 @@ dependencies = [
[[package]]
name = "pgp"
version = "0.20.0"
version = "0.19.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1cfa4743b28656065ff4c0ba09e46b357a65e8c00fc2341e89084b82f87cbdf1"
checksum = "eaffe1ec22db286599c30ae6be75b37493b558735d86c8e59ec5c38794415fe4"
dependencies = [
"aead",
"aes",
@@ -4325,7 +4325,6 @@ dependencies = [
"k256",
"log",
"md-5",
"memchr",
"ml-dsa",
"ml-kem",
"nom 8.0.0",
@@ -4337,6 +4336,7 @@ dependencies = [
"p384",
"p521",
"rand 0.8.6",
"regex",
"replace_with",
"ripemd",
"rsa",
@@ -4347,8 +4347,7 @@ dependencies = [
"signature",
"slh-dsa",
"smallvec",
"snafu 0.9.1",
"subtle",
"snafu",
"twofish",
"x25519-dalek",
"zeroize",
@@ -4587,7 +4586,7 @@ dependencies = [
"rand 0.8.6",
"serde",
"smallvec",
"snafu 0.8.5",
"snafu",
"socket2 0.5.9",
"time",
"tokio",
@@ -5282,7 +5281,7 @@ dependencies = [
"errno",
"libc",
"linux-raw-sys 0.12.1",
"windows-sys 0.61.1",
"windows-sys 0.52.0",
]
[[package]]
@@ -5850,16 +5849,7 @@ version = "0.8.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "223891c85e2a29c3fe8fb900c1fae5e69c2e42415e3177752e8718475efa5019"
dependencies = [
"snafu-derive 0.8.5",
]
[[package]]
name = "snafu"
version = "0.9.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d1a012328be2e3f5d5f6f3218147ca02588cea4cb865e876849ab6debcf36522"
dependencies = [
"snafu-derive 0.9.1",
"snafu-derive",
]
[[package]]
@@ -5874,18 +5864,6 @@ dependencies = [
"syn 2.0.117",
]
[[package]]
name = "snafu-derive"
version = "0.9.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5f103c50866b8743da9429b8a581d81a27c2d3a9c4ac7df8f8571c1dd7896eda"
dependencies = [
"heck 0.5.0",
"proc-macro2",
"quote",
"syn 2.0.117",
]
[[package]]
name = "socket2"
version = "0.5.9"
@@ -6169,7 +6147,7 @@ dependencies = [
"getrandom 0.3.3",
"once_cell",
"rustix 1.1.4",
"windows-sys 0.61.1",
"windows-sys 0.52.0",
]
[[package]]
+2 -2
View File
@@ -1,6 +1,6 @@
[package]
name = "deltachat"
version = "2.54.0-dev"
version = "2.51.0-dev"
edition = "2024"
license = "MPL-2.0"
rust-version = "1.89"
@@ -78,7 +78,7 @@ num-derive = "0.4"
num-traits = { workspace = true }
parking_lot = "0.12.4"
percent-encoding = "2.3"
pgp = { version = "0.20.0", features = ["draft-pqc"], default-features = false }
pgp = { version = "0.19.0", features = ["draft-pqc"], default-features = false }
pin-project = "1"
qrcodegen = "1.7.0"
quick-xml = { version = "0.39", features = ["escape-html"] }
-7
View File
@@ -59,13 +59,6 @@ If column is already declared without `NOT NULL`, use `IFNULL` function to provi
Use `HAVING COUNT(*) > 0` clause
to [prevent aggregate functions such as `MIN` and `MAX` from returning `NULL`](https://stackoverflow.com/questions/66527856/aggregate-functions-max-etc-return-null-instead-of-no-rows).
List columns explicitly in `INSERT` statements:
```
INSERT OR IGNORE INTO download (rfc724_mid, msg_id) VALUES (?,0);
```
Otherwise if a new column with default value is added in a future DB version, an upgraded DB can't
be used with the old code, e.g. after transferring a DB from a device running a newer version.
Don't delete unused columns too early, but maybe after several months/releases, unused columns are
still used by older versions, so deleting them breaks downgrading the core or importing a backup in
an older version. Also don't change the column type, consider adding a new column with another name
+1 -1
View File
@@ -39,7 +39,7 @@ mod vcard;
pub use vcard::{make_vcard, parse_vcard, VcardContact};
/// Valid contact address.
#[derive(Debug, Clone, PartialEq, Eq)]
#[derive(Debug, Clone)]
pub struct ContactAddress(String);
impl Deref for ContactAddress {
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "deltachat_ffi"
version = "2.54.0-dev"
version = "2.51.0-dev"
description = "Deltachat FFI"
edition = "2018"
readme = "README.md"
+1 -13
View File
@@ -462,10 +462,7 @@ char* dc_get_blobdir (const dc_context_t* context);
* - `gossip_period` = How often to gossip Autocrypt keys in chats with multiple recipients, in
* seconds. 2 days by default.
* This is not supposed to be changed by UIs and only used for testing.
* - `is_chatmail` = (deprecated) 1 if the the server is a chatmail server, 0 otherwise.
* This is deprecated, UIs should not behave differently
* for chatmail relays and classical email servers.
* Most usages in UIs can be replaced by `force_encryption`.
* - `is_chatmail` = 1 if the the server is a chatmail server, 0 otherwise.
* - `is_muted` = Whether a context is muted by the user.
* Muted contexts should not sound, vibrate or show notifications.
* In contrast to `dc_set_chat_mute_duration()`,
@@ -6344,15 +6341,6 @@ void dc_event_unref(dc_event_t* event);
*/
#define DC_EVENT_MSG_DELETED 2016
/**
* Like @ref DC_EVENT_MSG_READ, but also fires on subsequent MDNs,
* if there are multiple receivers, i.e. in groups and channels.
*
* @param data1 (int) chat_id
* @param data2 (int) msg_id
*/
#define DC_EVENT_MSG_READ_COUNT_CHANGED 2018
/**
* Chat changed. The name or the image of a chat group was changed or members were added or removed.
+1 -5
View File
@@ -528,7 +528,6 @@ pub unsafe extern "C" fn dc_event_get_id(event: *mut dc_event_t) -> libc::c_int
EventType::MsgFailed { .. } => 2012,
EventType::MsgRead { .. } => 2015,
EventType::MsgDeleted { .. } => 2016,
EventType::MsgReadCountChanged { .. } => 2018,
EventType::ChatModified(_) => 2020,
EventType::ChatEphemeralTimerModified { .. } => 2021,
EventType::ChatDeleted { .. } => 2023,
@@ -603,7 +602,6 @@ pub unsafe extern "C" fn dc_event_get_data1_int(event: *mut dc_event_t) -> libc:
| EventType::MsgFailed { chat_id, .. }
| EventType::MsgRead { chat_id, .. }
| EventType::MsgDeleted { chat_id, .. }
| EventType::MsgReadCountChanged { chat_id, .. }
| EventType::ChatModified(chat_id)
| EventType::ChatEphemeralTimerModified { chat_id, .. }
| EventType::ChatDeleted { chat_id } => chat_id.to_u32() as libc::c_int,
@@ -690,8 +688,7 @@ pub unsafe extern "C" fn dc_event_get_data2_int(event: *mut dc_event_t) -> libc:
| EventType::MsgDelivered { msg_id, .. }
| EventType::MsgFailed { msg_id, .. }
| EventType::MsgRead { msg_id, .. }
| EventType::MsgDeleted { msg_id, .. }
| EventType::MsgReadCountChanged { msg_id, .. } => msg_id.to_u32() as libc::c_int,
| EventType::MsgDeleted { msg_id, .. } => msg_id.to_u32() as libc::c_int,
EventType::SecurejoinInviterProgress { progress, .. }
| EventType::SecurejoinJoinerProgress { progress, .. } => *progress as libc::c_int,
EventType::ChatEphemeralTimerModified { timer, .. } => timer.to_u32() as libc::c_int,
@@ -765,7 +762,6 @@ pub unsafe extern "C" fn dc_event_get_data2_str(event: *mut dc_event_t) -> *mut
| EventType::MsgFailed { .. }
| EventType::MsgRead { .. }
| EventType::MsgDeleted { .. }
| EventType::MsgReadCountChanged { .. }
| EventType::ChatModified(_)
| EventType::ContactsChanged(_)
| EventType::LocationChanged(_)
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "deltachat-jsonrpc"
version = "2.54.0-dev"
version = "2.51.0-dev"
description = "DeltaChat JSON-RPC API"
edition = "2021"
license = "MPL-2.0"
+1 -3
View File
@@ -1081,9 +1081,7 @@ impl CommandApi {
/// Create a new unencrypted group chat.
///
/// Same as [`Self::create_group_chat`], but the chat is unencrypted and can only have
/// address-contacts. NB: Chats with similar names and the same members are merged on other
/// devices, but usually users don't create such chats and look up the existing one instead, so
/// chat split on the first device is acceptable.
/// address-contacts.
async fn create_group_chat_unencrypted(&self, account_id: u32, name: String) -> Result<u32> {
let ctx = self.get_context(account_id).await?;
chat::create_group_unencrypted(&ctx, &name)
-15
View File
@@ -203,17 +203,6 @@ pub enum EventType {
msg_id: u32,
},
/// Like [`EventType::MsgRead`], but also fires on subsequent MDNs,
/// if there are multiple receivers, i.e. in groups and channels.
#[serde(rename_all = "camelCase")]
MsgReadCountChanged {
/// ID of the chat which the message belongs to.
chat_id: u32,
/// ID of the message that was read.
msg_id: u32,
},
/// A single message was deleted.
///
/// This event means that the message will no longer appear in the messagelist.
@@ -557,10 +546,6 @@ impl From<CoreEventType> for EventType {
chat_id: chat_id.to_u32(),
msg_id: msg_id.to_u32(),
},
CoreEventType::MsgReadCountChanged { chat_id, msg_id } => MsgReadCountChanged {
chat_id: chat_id.to_u32(),
msg_id: msg_id.to_u32(),
},
CoreEventType::MsgDeleted { chat_id, msg_id } => MsgDeleted {
chat_id: chat_id.to_u32(),
msg_id: msg_id.to_u32(),
-3
View File
@@ -236,7 +236,6 @@ impl From<Qr> for QrObject {
invitenumber,
authcode,
is_v3,
..
} => {
let contact_id = contact_id.to_u32();
let fingerprint = fingerprint.human_readable();
@@ -256,7 +255,6 @@ impl From<Qr> for QrObject {
invitenumber,
authcode,
is_v3,
..
} => {
let contact_id = contact_id.to_u32();
let fingerprint = fingerprint.human_readable();
@@ -278,7 +276,6 @@ impl From<Qr> for QrObject {
authcode,
invitenumber,
is_v3,
..
} => {
let contact_id = contact_id.to_u32();
let fingerprint = fingerprint.human_readable();
+1 -1
View File
@@ -54,5 +54,5 @@
},
"type": "module",
"types": "dist/deltachat.d.ts",
"version": "2.54.0-dev"
"version": "2.51.0-dev"
}
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "deltachat-repl"
version = "2.54.0-dev"
version = "2.51.0-dev"
license = "MPL-2.0"
edition = "2021"
repository = "https://github.com/chatmail/core"
+1 -1
View File
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "deltachat-rpc-client"
version = "2.54.0-dev"
version = "2.51.0-dev"
license = "MPL-2.0"
description = "Python client for Delta Chat core JSON-RPC interface"
classifiers = [
@@ -54,7 +54,6 @@ class EventType(str, Enum):
MSG_DELIVERED = "MsgDelivered"
MSG_FAILED = "MsgFailed"
MSG_READ = "MsgRead"
MSG_READ_COUNT_CHANGED = "MsgReadCountChanged"
MSG_DELETED = "MsgDeleted"
CHAT_MODIFIED = "ChatModified"
CHAT_DELETED = "ChatDeleted"
@@ -251,7 +251,12 @@ def test_realtime_large_webxdc(acfactory, path_to_large_webxdc):
ac1_ac2_chat = ac1.create_chat(ac2)
ac1_webxdc_msg = ac1_ac2_chat.send_message(text="realtime check", file=path_to_large_webxdc)
# Receive pre-message.
ac2_webxdc_msg = ac2.wait_for_incoming_msg()
# Receive post-message.
ac2_webxdc_msg = ac2.wait_for_msg(EventType.MSGS_CHANGED)
ac2_webxdc_msg.send_webxdc_realtime_advertisement()
event = ac1.wait_for_event(EventType.WEBXDC_REALTIME_ADVERTISEMENT_RECEIVED)
assert event.msg_id == ac1_webxdc_msg.id
+12 -3
View File
@@ -37,14 +37,19 @@ def test_one_account_send_bcc_setting(acfactory, log, direct_imap):
log.section("send out message without bcc to ourselves")
ac1.set_config("bcc_self", "0")
chat = ac1.create_chat(ac2)
self_addr = ac1.get_config("addr")
other_addr = ac2.get_config("addr")
msg_out = chat.send_text("message1")
assert not msg_out.get_snapshot().is_forwarded
# wait for send out (no BCC)
ac1.wait_for_event(EventType.SMTP_MESSAGE_SENT)
ev = ac1.wait_for_event(EventType.SMTP_MESSAGE_SENT)
assert ac1.get_config("bcc_self") == "0"
assert self_addr not in ev.msg
assert other_addr in ev.msg
log.section("ac1: setting bcc_self=1")
ac1.set_config("bcc_self", "1")
@@ -52,16 +57,20 @@ def test_one_account_send_bcc_setting(acfactory, log, direct_imap):
msg_out = chat.send_text("message2")
# wait for send out (BCC)
ac1.wait_for_event(EventType.SMTP_MESSAGE_SENT)
ev = ac1.wait_for_event(EventType.SMTP_MESSAGE_SENT)
assert ac1.get_config("bcc_self") == "1"
# Second client receives only the second message, but not the first.
# Second client receives only second message, but not the first.
ev_msg = ac1_clone.wait_for_event(EventType.MSGS_CHANGED)
assert ac1_clone.get_message_by_id(ev_msg.msg_id).get_snapshot().text == "Messages are end-to-end encrypted."
ev_msg = ac1_clone.wait_for_event(EventType.MSGS_CHANGED)
assert ac1_clone.get_message_by_id(ev_msg.msg_id).get_snapshot().text == msg_out.get_snapshot().text
# now make sure we are sending message to ourselves too
assert self_addr in ev.msg
assert self_addr in ev.msg
# BCC-self messages are marked as seen by the sender device.
while True:
event = ac1.wait_for_event()
@@ -704,25 +704,3 @@ def test_withdraw_securejoin_qr(acfactory):
and "Ignoring RequestWithAuth message because of invalid auth code." in event.msg
):
break
def test_qr_scan_updates_new_relay_address(acfactory):
alice, bob = acfactory.get_online_accounts(2)
bob_alice_chat = bob.secure_join(alice.get_qr_code())
alice.wait_for_securejoin_inviter_success()
bob.wait_for_securejoin_joiner_success()
for ac in [alice, bob]:
old_addr = ac.get_config("configured_addr")
ac.add_transport_from_qr(acfactory.get_account_qr())
ac.set_config("configured_addr", ac.list_transports()[1]["addr"])
ac.delete_transport(old_addr)
bob.secure_join(alice.get_qr_code())
alice.wait_for_securejoin_inviter_success()
bob.wait_for_securejoin_joiner_success()
bob_alice_chat.send_text("hi")
snapshot = alice.wait_for_incoming_msg().get_snapshot()
assert snapshot.text == "hi"
+10 -54
View File
@@ -221,30 +221,6 @@ def test_account(acfactory) -> None:
alice.stop_io()
def test_mark_fresh_vs_self_mdn(acfactory) -> None:
alice, bob = acfactory.get_online_accounts(2)
bob.set_config("bcc_self", "1")
alice_contact_bob = alice.create_contact(bob)
alice_chat = alice_contact_bob.create_chat()
alice_chat.send_text("Hello!")
event = bob.wait_for_incoming_msg_event()
chat_id = event.chat_id
msg_id = event.msg_id
bob_chat = bob.get_chat_by_id(chat_id)
message = bob.get_message_by_id(msg_id)
bob_chat.accept()
bob.mark_seen_messages([message])
bob_chat.mark_fresh()
assert bob_chat.get_fresh_message_count() == 1
alice.wait_for_event(EventType.MSG_READ)
alice_chat.send_text("You've read 'Hello!'")
bob.wait_for_incoming_msg_event()
assert bob_chat.get_fresh_message_count() == 2
def test_chat(acfactory) -> None:
alice, bob = acfactory.get_online_accounts(2)
@@ -474,35 +450,6 @@ def test_reaction_seen_on_another_dev(acfactory) -> None:
assert chat_id == alice2_chat_bob.id
def test_2nd_device_events_when_msgs_are_seen(acfactory) -> None:
alice, bob = acfactory.get_online_accounts(2)
alice2 = alice.clone()
alice2.start_io()
# Get an accepted chat, otherwise alice2 won't be notified about the 2nd message.
chat_alice2 = alice2.create_chat(bob)
chat_id_alice2 = chat_alice2.get_basic_snapshot().id
chat_bob_alice = bob.create_chat(alice)
chat_bob_alice.send_text("Hello!")
msg_alice = alice.wait_for_incoming_msg()
assert alice2.wait_for_incoming_msg_event().chat_id == chat_id_alice2
chat_bob_alice.send_text("What's new?")
assert alice2.wait_for_incoming_msg_event().chat_id == chat_id_alice2
chat_alice2 = alice2.get_chat_by_id(chat_id_alice2)
assert chat_alice2.get_fresh_message_count() == 2
msg_alice.mark_seen()
assert alice2.wait_for_msgs_changed_event().chat_id == chat_id_alice2
assert chat_alice2.get_fresh_message_count() == 1
msg_id = alice.wait_for_msgs_changed_event().msg_id
msg = alice.get_message_by_id(msg_id)
msg.mark_seen()
assert alice2.wait_for_event(EventType.MSGS_NOTICED).chat_id == chat_id_alice2
assert chat_alice2.get_fresh_message_count() == 0
def test_is_bot(acfactory) -> None:
"""Test that we can recognize messages submitted by bots."""
alice, bob = acfactory.get_online_accounts(2)
@@ -926,6 +873,10 @@ def test_delete_fully_downloaded_msg(acfactory, tmp_path, direct_imap):
msg = bob.wait_for_incoming_msg()
msg_snapshot = msg.get_snapshot()
assert msg_snapshot.download_state == DownloadState.AVAILABLE
msgs_changed_event = bob.wait_for_msgs_changed_event()
assert msgs_changed_event.msg_id == msg.id
msg_snapshot = msg.get_snapshot()
assert msg_snapshot.download_state == DownloadState.DONE
bob_direct_imap = direct_imap(bob)
@@ -956,6 +907,10 @@ def test_imap_autodelete_fully_downloaded_msg(acfactory, tmp_path, direct_imap):
msg = bob.wait_for_incoming_msg()
msg_snapshot = msg.get_snapshot()
assert msg_snapshot.download_state == DownloadState.AVAILABLE
msgs_changed_event = bob.wait_for_msgs_changed_event()
assert msgs_changed_event.msg_id == msg.id
msg_snapshot = msg.get_snapshot()
assert msg_snapshot.download_state == DownloadState.DONE
bob_direct_imap = direct_imap(bob)
@@ -1144,7 +1099,6 @@ def test_rename_group(acfactory):
bob.wait_for_event(EventType.CHATLIST_ITEM_CHANGED)
for name in ["Baz", "Foo bar", "Xyzzy"]:
time.sleep(1)
alice_group.set_name(name)
bob.wait_for_event(EventType.CHATLIST_ITEM_CHANGED)
bob.wait_for_event(EventType.CHATLIST_ITEM_CHANGED)
@@ -1435,5 +1389,7 @@ def test_large_message(acfactory) -> None:
)
msg = bob.wait_for_incoming_msg()
msgs_changed_event = bob.wait_for_msgs_changed_event()
assert msg.id == msgs_changed_event.msg_id
snapshot = msg.get_snapshot()
assert snapshot.text == "Hello World, this message is bigger than 5 bytes"
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "deltachat-rpc-server"
version = "2.54.0-dev"
version = "2.51.0-dev"
description = "DeltaChat JSON-RPC server"
edition = "2021"
readme = "README.md"
@@ -15,5 +15,5 @@
},
"type": "module",
"types": "index.d.ts",
"version": "2.54.0-dev"
"version": "2.51.0-dev"
}
-7
View File
@@ -18,11 +18,6 @@ ignore = [
# this should be fixed by upgrading to iroh 1.0 once it is released.
"RUSTSEC-2025-0134",
# Unmaintained proc-macro-error2
# Transitive dependency of typescript-type-def 0.5.13.
# <https://rustsec.org/advisories/RUSTSEC-2026-0173>
"RUSTSEC-2026-0173",
# rustls-webpki v0.102.8
# We cannot upgrade to >=0.103.10 because
# it is a transitive dependency of iroh 0.35.0
@@ -83,8 +78,6 @@ skip = [
{ name = "rustix", version = "0.38.44" },
{ name = "rustls-webpki", version = "0.102.8" },
{ name = "serdect", version = "0.2.0" },
{ name = "snafu-derive", version = "0.8.5" },
{ name = "snafu", version = "0.8.5" },
{ name = "socket2", version = "0.5.9" },
{ name = "spin", version = "0.9.8" },
{ name = "strum_macros", version = "0.26.2" },
Generated
+4 -3
View File
@@ -94,15 +94,16 @@
]
},
"locked": {
"lastModified": 1780914171,
"narHash": "sha256-NYoa+CvsCgayY356worC9g6QoZkKJmvrj6nVdEA2Lyk=",
"lastModified": 1779912356,
"narHash": "sha256-yj5O6vmAj+OfhTQMiUwhmQRP0HAII3BxEI6zuY6h/5k=",
"owner": "nix-community",
"repo": "naersk",
"rev": "41b6e9efb62fac9c16d5617e0456a715928a2206",
"rev": "33eaf5c72a67db15073322d26cd342c443556214",
"type": "github"
},
"original": {
"owner": "nix-community",
"ref": "pull/391/head",
"repo": "naersk",
"type": "github"
}
+41 -18
View File
@@ -4,7 +4,7 @@
fenix.url = "github:nix-community/fenix";
fenix.inputs.nixpkgs.follows = "nixpkgs";
flake-utils.url = "github:numtide/flake-utils";
naersk.url = "github:nix-community/naersk";
naersk.url = "github:nix-community/naersk/pull/391/head";
naersk.inputs.nixpkgs.follows = "nixpkgs";
naersk.inputs.fenix.follows = "fenix";
nix-filter.url = "github:numtide/nix-filter";
@@ -66,15 +66,37 @@
];
};
# Map from architecture name to nixpkgs targets.
# Map from architecture name to rust targets and nixpkgs targets.
arch2targets = {
"x86_64-linux" = "x86_64-unknown-linux-musl";
"armv7l-linux" = "armv7l-unknown-linux-musleabihf";
"armv6l-linux" = "armv6l-unknown-linux-musleabihf";
"aarch64-linux" = "aarch64-unknown-linux-musl";
"i686-linux" = "i686-unknown-linux-musl";
"x86_64-darwin" = "x86_64-darwin";
"aarch64-darwin" = "aarch64-darwin";
"x86_64-linux" = {
rustTarget = "x86_64-unknown-linux-musl";
crossTarget = "x86_64-unknown-linux-musl";
};
"armv7l-linux" = {
rustTarget = "armv7-unknown-linux-musleabihf";
crossTarget = "armv7l-unknown-linux-musleabihf";
};
"armv6l-linux" = {
rustTarget = "arm-unknown-linux-musleabihf";
crossTarget = "armv6l-unknown-linux-musleabihf";
};
"aarch64-linux" = {
rustTarget = "aarch64-unknown-linux-musl";
crossTarget = "aarch64-unknown-linux-musl";
};
"i686-linux" = {
rustTarget = "i686-unknown-linux-musl";
crossTarget = "i686-unknown-linux-musl";
};
"x86_64-darwin" = {
rustTarget = "x86_64-apple-darwin";
crossTarget = "x86_64-darwin";
};
"aarch64-darwin" = {
rustTarget = "aarch64-apple-darwin";
crossTarget = "aarch64-darwin";
};
};
cargoLock = {
lockFile = ./Cargo.lock;
@@ -91,10 +113,10 @@
auditable = false; # Avoid cargo-auditable failures.
doCheck = false; # Disable test as it requires network access.
};
pkgsWin64 = pkgs.pkgsCross.mingwW64;
mkWin64RustPackage = packageName:
let
pkgsWin64 = pkgs.pkgsCross.mingwW64;
rustTarget = pkgsWin64.stdenv.hostPlatform.rust.rustcTarget;
rustTarget = "x86_64-pc-windows-gnu";
toolchainWin = fenixPkgs.combine [
fenixPkgs.stable.rustc
fenixPkgs.stable.cargo
@@ -125,7 +147,6 @@
CARGO_BUILD_TARGET = rustTarget;
TARGET_CC = "${pkgsWin64.stdenv.cc}/bin/${pkgsWin64.stdenv.cc.targetPrefix}cc";
CFLAGS_x86_64_pc_windows_gnu = "-I${pkgsWin64.windows.pthreads}/include";
CARGO_BUILD_RUSTFLAGS = [
"-C"
"linker=${TARGET_CC}"
@@ -137,10 +158,12 @@
LD = "${pkgsWin64.stdenv.cc}/bin/${pkgsWin64.stdenv.cc.targetPrefix}cc";
};
pkgsWin32 = pkgs.pkgsCross.mingw32;
mkWin32RustPackage = packageName:
let
pkgsWin32 = pkgs.pkgsCross.mingw32;
rustTarget = pkgsWin32.stdenv.hostPlatform.rust.rustcTarget;
rustTarget = "i686-pc-windows-gnu";
in
let
toolchainWin = fenixPkgs.combine [
fenixPkgs.stable.rustc
fenixPkgs.stable.cargo
@@ -180,7 +203,6 @@
src = pkgs.lib.cleanSource ./.;
nativeBuildInputs = [
pkgs.perl # Needed to build vendored OpenSSL.
pkgs.nasm # aws-lc-sys requires it
];
depsBuildBuild = [
winCC
@@ -193,7 +215,6 @@
CARGO_BUILD_TARGET = rustTarget;
TARGET_CC = "${winCC}/bin/${winCC.targetPrefix}cc";
CFLAGS_i686_pc_windows_gnu = "-I${pkgsWin32.windows.pthreads}/include";
CARGO_BUILD_RUSTFLAGS = [
"-C"
"linker=${TARGET_CC}"
@@ -207,12 +228,14 @@
mkCrossRustPackage = arch: packageName:
let
crossTarget = arch2targets."${arch}";
rustTarget = arch2targets."${arch}".rustTarget;
crossTarget = arch2targets."${arch}".crossTarget;
pkgsCross = import nixpkgs {
system = system;
crossSystem.config = crossTarget;
};
rustTarget = pkgsCross.stdenv.hostPlatform.rust.rustcTarget;
in
let
toolchain = fenixPkgs.combine [
fenixPkgs.stable.rustc
fenixPkgs.stable.cargo
+1 -1
View File
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "deltachat"
version = "2.54.0-dev"
version = "2.51.0-dev"
license = "MPL-2.0"
description = "Python bindings for the Delta Chat Core library using CFFI against the Rust-implemented libdeltachat"
readme = "README.rst"
+1 -1
View File
@@ -1 +1 @@
2026-06-15
2026-05-29
+9 -33
View File
@@ -1,6 +1,6 @@
//! # Blob directory management.
use std::cmp::{max, min};
use std::cmp::max;
use std::io::{Cursor, Seek};
use std::iter::FusedIterator;
use std::mem;
@@ -12,7 +12,7 @@ use futures::StreamExt;
use image::ImageReader;
use image::{DynamicImage, GenericImage, GenericImageView, ImageFormat, Pixel, Rgba};
use image::{codecs::jpeg::JpegEncoder, metadata::Orientation};
use num_traits::{FromPrimitive, cast};
use num_traits::FromPrimitive;
use tokio::{fs, task};
use tokio_stream::wrappers::ReadDirStream;
@@ -382,9 +382,14 @@ impl<'a> BlobObject<'a> {
}
img.apply_orientation(orientation);
// max_wh is the maximum image width and height, i.e. the resolution-limit,
// as set by `Config::MediaQuality`.
// max_wh is the maximum image width and height, i.e. the resolution-limit.
// target_wh target-resolution for resizing the image.
let exceeds_wh = img.width() > max_wh || img.height() > max_wh;
let mut target_wh = if exceeds_wh {
max_wh
} else {
max(img.width(), img.height())
};
let exceeds_max_bytes = nr_bytes > max_bytes as u64;
let jpeg_quality = 75;
@@ -423,35 +428,6 @@ impl<'a> BlobObject<'a> {
});
if do_scale {
let longest_side_len = max(img.width(), img.height());
// target_wh will be used as the target-resolution for resizing the image,
// so that the longest sides of the image match the target-resolution.
let mut target_wh = if !is_avatar {
let area_sqrt = (f64::from(img.width()) * f64::from(img.height())).sqrt();
// Limit resolution to the number of pixels that fit within max_wh * max_wh,
// so that the image-quality does not depend on the aspect-ratio.
let mut resolution_limit: u32 = cast(
(f64::from(longest_side_len) * (f64::from(max_wh) / area_sqrt)).floor(),
)
.unwrap_or(max_wh);
// Align at least one dimension of the resampled image to a multiple of 8 pixels,
// to have fewer partially used JPEG-blocks (which represent 8x8 pixels each).
if resolution_limit < longest_side_len && resolution_limit > 8 {
while !resolution_limit.is_multiple_of(8) {
resolution_limit -= 1
}
}
resolution_limit
} else {
max_wh
};
target_wh = min(target_wh, longest_side_len);
// For images in JPEG-format, 65535 pixels is the maximum resolution per dimension.
target_wh = min(target_wh, 65535);
loop {
if mem::take(&mut add_white_bg) {
self::add_white_bg(&mut img);
+6 -6
View File
@@ -406,8 +406,8 @@ async fn test_recode_image_balanced_png() {
extension: "png",
original_width: 1920,
original_height: 1080,
compressed_width: 848,
compressed_height: 477,
compressed_width: constants::WORSE_IMAGE_SIZE,
compressed_height: constants::WORSE_IMAGE_SIZE * 1080 / 1920,
..Default::default()
}
.test()
@@ -497,8 +497,8 @@ async fn test_recode_image_rgba_png_to_jpeg() {
extension: "png",
original_width: 1920,
original_height: 1080,
compressed_width: 848,
compressed_height: 477,
compressed_width: constants::WORSE_IMAGE_SIZE,
compressed_height: constants::WORSE_IMAGE_SIZE * 1080 / 1920,
..Default::default()
}
.test()
@@ -517,8 +517,8 @@ async fn test_recode_image_huge_jpg() {
has_exif: true,
original_width: 1920,
original_height: 1080,
compressed_width: 1704,
compressed_height: 959,
compressed_width: constants::BALANCED_IMAGE_SIZE,
compressed_height: constants::BALANCED_IMAGE_SIZE * 1080 / 1920,
..Default::default()
}
.test()
+64 -66
View File
@@ -50,8 +50,8 @@ use crate::stock_str;
use crate::sync::{self, Sync::*, SyncData};
use crate::tools::{
IsNoneOrEmpty, SystemTime, buf_compress, create_broadcast_secret, create_id,
create_outgoing_rfc724_mid, get_abs_path, gm2local_offset, normalize_text, time,
truncate_msg_text,
create_outgoing_rfc724_mid, create_smeared_timestamp, create_smeared_timestamps, get_abs_path,
gm2local_offset, normalize_text, smeared_time, time, truncate_msg_text,
};
use crate::webxdc::StatusUpdateSerial;
@@ -292,7 +292,7 @@ impl ChatId {
timestamp: i64,
) -> Result<Self> {
let grpname = sanitize_single_line(grpname);
let timestamp = cmp::min(timestamp, time());
let timestamp = cmp::min(timestamp, smeared_time(context));
let row_id =
context.sql.insert(
"INSERT INTO chats (type, name, name_normalized, grpid, blocked, created_timestamp, protected, param) VALUES(?, ?, ?, ?, ?, ?, 0, ?)",
@@ -704,7 +704,8 @@ SELECT id, rfc724_mid, pre_rfc724_mid, timestamp, ?, 1 FROM msgs WHERE chat_id=?
context
.set_config_internal(Config::LastHousekeeping, None)
.await?;
context.scheduler.interrupt_inbox().await;
context.scheduler.interrupt_smtp().await;
Ok(())
}
@@ -1255,7 +1256,7 @@ SELECT id, rfc724_mid, pre_rfc724_mid, timestamp, ?, 1 FROM msgs WHERE chat_id=?
message_timestamp: i64,
always_sort_to_bottom: bool,
) -> Result<i64> {
let mut sort_timestamp = cmp::min(message_timestamp, time());
let mut sort_timestamp = cmp::min(message_timestamp, smeared_time(context));
let last_msg_time: Option<i64> = if always_sort_to_bottom {
// get newest message for this chat
@@ -2404,7 +2405,7 @@ impl ChatIdBlocked {
_ => (),
}
let now = time();
let smeared_time = create_smeared_timestamp(context);
let chat_id = context
.sql
@@ -2419,7 +2420,7 @@ impl ChatIdBlocked {
normalize_text(&chat_name),
params.to_string(),
create_blocked as u8,
now,
smeared_time,
),
)?;
let chat_id = ChatId::new(
@@ -2445,7 +2446,7 @@ impl ChatIdBlocked {
&& !chat.param.exists(Param::Devicetalk)
&& !chat.param.exists(Param::Selftalk)
{
chat_id.add_e2ee_notice(context, now).await?;
chat_id.add_e2ee_notice(context, smeared_time).await?;
}
Ok(Self {
@@ -2729,7 +2730,7 @@ async fn prepare_send_msg(
}
msg.state = MessageState::OutPending;
msg.timestamp_sort = time();
msg.timestamp_sort = create_smeared_timestamp(context);
prepare_msg_blob(context, msg).await?;
if !msg.hidden {
chat_id.unarchive_if_not_muted(context, msg.state).await?;
@@ -2923,7 +2924,7 @@ pub(crate) async fn create_send_msg_jobs(context: &Context, msg: &mut Message) -
);
}
let now = time();
let now = smeared_time(context);
if rendered_msg.last_added_location_id.is_some()
&& let Err(err) = location::set_kml_sent_timestamp(context, msg.chat_id, now).await
@@ -2986,6 +2987,15 @@ WHERE id=?
)?;
for recipients_chunk in recipients.chunks(chunk_size) {
let recipients_chunk = recipients_chunk.join(" ");
if let Some(pre_msg) = &rendered_pre_msg {
let row_id = stmt.execute((
&pre_msg.rfc724_mid,
&recipients_chunk,
&pre_msg.message,
msg.id,
))?;
row_ids.push(row_id.try_into()?);
}
let row_id = stmt.execute((
&rendered_msg.rfc724_mid,
&recipients_chunk,
@@ -2993,16 +3003,6 @@ WHERE id=?
msg.id,
))?;
row_ids.push(row_id.try_into()?);
let Some(pre_msg) = &rendered_pre_msg else {
continue;
};
let row_id = stmt.execute((
&pre_msg.rfc724_mid,
&recipients_chunk,
&pre_msg.message,
msg.id,
))?;
row_ids.push(row_id.try_into()?);
}
Ok(row_ids)
};
@@ -3046,7 +3046,7 @@ pub async fn send_edit_request(context: &Context, msg_id: MsgId, new_text: Strin
return Ok(());
}
save_text_edit_to_db(context, &mut original_msg, &new_text, &[]).await?;
save_text_edit_to_db(context, &mut original_msg, &new_text).await?;
let mut edit_msg = Message::new_text(EDITED_PREFIX.to_owned() + &new_text); // prefix only set for nicer display in Non-Delta-MUAs
edit_msg.set_quote(context, Some(&original_msg)).await?; // quote only set for nicer display in Non-Delta-MUAs
@@ -3065,20 +3065,16 @@ pub(crate) async fn save_text_edit_to_db(
context: &Context,
original_msg: &mut Message,
new_text: &str,
mime_headers: &[u8],
) -> Result<()> {
original_msg.param.set_int(Param::IsEdited, 1);
context
.sql
.execute(
"
UPDATE msgs SET txt=?, txt_normalized=?, param=?, mime_headers=?, mime_modified=? WHERE id=?",
"UPDATE msgs SET txt=?, txt_normalized=?, param=? WHERE id=?",
(
new_text,
normalize_text(new_text),
original_msg.param.to_string(),
mime_headers,
!mime_headers.is_empty(),
original_msg.id,
),
)
@@ -3556,10 +3552,6 @@ pub async fn create_group_unencrypted(context: &Context, name: &str) -> Result<C
/// unencrypted chats currently.
/// * `grpid` - Group ID. Iff nonempty, the chat is encrypted (with key-contacts).
/// * `name` - Chat name.
///
/// NB: Unencrypted chats with similar names and the same members are merged on other devices, but
/// usually users don't create such chats and look up the existing one instead, so chat split on the
/// first device is acceptable.
pub(crate) async fn create_group_ex(
context: &Context,
sync: sync::Sync,
@@ -3574,7 +3566,7 @@ pub(crate) async fn create_group_ex(
chat_name = "…".to_string();
}
let timestamp = time();
let timestamp = create_smeared_timestamp(context);
let row_id = context
.sql
.insert(
@@ -3657,7 +3649,7 @@ pub(crate) async fn create_out_broadcast_ex(
bail!("Invalid broadcast channel name: {chat_name}.");
}
let timestamp = time();
let timestamp = create_smeared_timestamp(context);
let trans_fn = |t: &mut rusqlite::Transaction| -> Result<ChatId> {
let cnt: u32 = t.query_row(
"SELECT COUNT(*) FROM chats WHERE grpid=?",
@@ -3746,17 +3738,19 @@ pub(crate) async fn update_chat_contacts_table(
id: ChatId,
contacts: &BTreeSet<ContactId>,
) -> Result<()> {
// See add_to_chat_contacts_table() for reasoning.
let limit = cmp::max(time().saturating_add(TIMESTAMP_SENT_TOLERANCE), timestamp);
context
.sql
.transaction(move |transaction| {
// Bump `remove_timestamp` to at least `now`
// even for members from `contacts`.
// Bump `remove_timestamp` even for members from `contacts`.
// We add members from `contacts` back below.
transaction.execute(
"UPDATE chats_contacts
SET remove_timestamp=MAX(add_timestamp+1, ?)
"UPDATE chats_contacts SET
add_timestamp=MIN(add_timestamp, ?1),
remove_timestamp=MAX(MIN(remove_timestamp,?1), MIN(add_timestamp,?1)+1, ?)
WHERE chat_id=?",
(timestamp, id),
(limit, timestamp, id),
)?;
if !contacts.is_empty() {
@@ -3768,9 +3762,8 @@ pub(crate) async fn update_chat_contacts_table(
)?;
for contact_id in contacts {
// We bumped `add_timestamp` for existing rows above,
// so on conflict it is enough to set `add_timestamp = remove_timestamp`
// and this guarantees that `add_timestamp` is no less than `timestamp`.
// We bumped `remove_timestamp` for existing rows above,
// so on conflict it is enough to set `add_timestamp = remove_timestamp`.
statement.execute((id, contact_id, timestamp))?;
}
}
@@ -3787,17 +3780,24 @@ pub(crate) async fn add_to_chat_contacts_table(
chat_id: ChatId,
contact_ids: &[ContactId],
) -> Result<()> {
// Our clock may be slow, so limit stored timestamps with `timestamp` if it's bigger. This way
// we only cap remote timestamps if, in addition, remote changes arrive reordered or we do local
// changes. Also allow some tolerance, moreover, previous removals might lend time from the
// future.
let limit = cmp::max(time().saturating_add(TIMESTAMP_SENT_TOLERANCE), timestamp);
context
.sql
.transaction(move |transaction| {
let mut add_statement = transaction.prepare(
"INSERT INTO chats_contacts (chat_id, contact_id, add_timestamp) VALUES(?1, ?2, ?3)
ON CONFLICT (chat_id, contact_id)
DO UPDATE SET add_timestamp=MAX(remove_timestamp, ?3)",
DO UPDATE SET
remove_timestamp=MIN(remove_timestamp, ?4),
add_timestamp=MIN(MAX(add_timestamp,remove_timestamp,?3), ?4)",
)?;
for contact_id in contact_ids {
add_statement.execute((chat_id, contact_id, timestamp))?;
add_statement.execute((chat_id, contact_id, timestamp, limit))?;
}
Ok(())
})
@@ -3816,13 +3816,16 @@ pub(crate) async fn remove_from_chat_contacts_table(
contact_id: ContactId,
) -> Result<bool> {
let now = time();
// See add_to_chat_contacts_table() for reasoning.
let limit = now.saturating_add(TIMESTAMP_SENT_TOLERANCE);
let is_past_member = context
.sql
.execute(
"UPDATE chats_contacts
SET remove_timestamp=MAX(add_timestamp+1, ?)
"UPDATE chats_contacts SET
add_timestamp=MIN(add_timestamp, ?1),
remove_timestamp=MAX(MIN(remove_timestamp,?1), MIN(add_timestamp,?1)+1, ?)
WHERE chat_id=? AND contact_id=?",
(now, chat_id, contact_id),
(limit, now, chat_id, contact_id),
)
.await?
> 0;
@@ -3915,11 +3918,11 @@ pub(crate) async fn add_contact_to_chat_ex(
return Ok(false);
}
if from_handshake && chat.param.get_int(Param::Unpromoted).unwrap_or_default() == 1 {
let now = time();
let smeared_time = smeared_time(context);
chat.param
.remove(Param::Unpromoted)
.set_i64(Param::GroupNameTimestamp, now)
.set_i64(Param::GroupDescriptionTimestamp, now);
.set_i64(Param::GroupNameTimestamp, smeared_time)
.set_i64(Param::GroupDescriptionTimestamp, smeared_time);
chat.update_param(context).await?;
}
if context.is_self_addr(contact.get_addr()).await? {
@@ -4485,6 +4488,7 @@ pub async fn forward_msgs(context: &Context, msg_ids: &[MsgId], chat_id: ChatId)
}
/// Forwards multiple messages to a chat in another context.
#[expect(clippy::arithmetic_side_effects)]
pub async fn forward_msgs_2ctx(
ctx_src: &Context,
msg_ids: &[MsgId],
@@ -4495,6 +4499,7 @@ pub async fn forward_msgs_2ctx(
ensure!(!chat_id.is_special(), "can not forward to special chat");
let mut created_msgs: Vec<MsgId> = Vec::new();
let mut curr_timestamp: i64;
chat_id
.unarchive_if_not_muted(ctx_dst, MessageState::Undefined)
@@ -4503,7 +4508,7 @@ pub async fn forward_msgs_2ctx(
if let Some(reason) = chat.why_cant_send(ctx_dst).await? {
bail!("cannot send to {chat_id}: {reason}");
}
let now = time();
curr_timestamp = create_smeared_timestamps(ctx_dst, msg_ids.len());
let mut msgs = Vec::with_capacity(msg_ids.len());
for id in msg_ids {
let ts: i64 = ctx_src
@@ -4568,9 +4573,10 @@ pub async fn forward_msgs_2ctx(
msg.state = MessageState::OutPending;
msg.rfc724_mid = create_outgoing_rfc724_mid();
msg.pre_rfc724_mid.clear();
msg.timestamp_sort = now;
msg.timestamp_sort = curr_timestamp;
chat.prepare_msg_raw(ctx_dst, &mut msg, None).await?;
curr_timestamp += 1;
if !create_send_msg_jobs(ctx_dst, &mut msg).await?.is_empty() {
ctx_dst.scheduler.interrupt_smtp().await;
}
@@ -4663,7 +4669,7 @@ pub(crate) async fn save_copy_in_self_talk(
} else {
MessageState::InSeen
},
time(),
create_smeared_timestamp(context),
msg.param.to_string(),
src_msg_id,
src_msg_id,
@@ -4840,7 +4846,7 @@ pub async fn add_device_msg_with_importance(
chat_id = ChatId::get_for_contact(context, ContactId::DEVICE).await?;
let rfc724_mid = create_outgoing_rfc724_mid();
let timestamp_sent = time();
let timestamp_sent = create_smeared_timestamp(context);
// makes sure, the added message is the last one,
// even if the date is wrong (useful esp. when warning about bad dates)
@@ -4987,7 +4993,7 @@ pub(crate) async fn add_info_msg_with_cmd(
} else {
let sort_to_bottom = true;
chat_id
.calc_sort_timestamp(context, time(), sort_to_bottom)
.calc_sort_timestamp(context, smeared_time(context), sort_to_bottom)
.await?
};
@@ -5150,7 +5156,7 @@ async fn set_contacts_by_fingerprints(
Ok(broadcast_contacts_added)
})
.await?;
let timestamp = time();
let timestamp = smeared_time(context);
for added_id in broadcast_contacts_added {
let msg = stock_str::msg_add_member_local(context, added_id, ContactId::UNDEFINED).await;
add_info_msg_with_cmd(
@@ -5234,11 +5240,9 @@ impl Context {
}
_ => (),
}
// Newly created chat will be soon unblocked, `Blocked::Yes` here is just
// to hide it from chatlist, as at this point it is completely blank (no name etc.).
// Even if app crashes at this point, the chat will re-appear on e.g. any new
// message sent from other device.
ChatIdBlocked::get_for_contact(self, contact_id, Blocked::Yes)
// Use `Request` so that even if the program crashes, the user doesn't have to look
// into the blocked contacts.
ChatIdBlocked::get_for_contact(self, contact_id, Blocked::Request)
.await?
.id
}
@@ -5262,13 +5266,7 @@ impl Context {
}
_ => (),
}
// Don't show a chat on other devices until securejoin completes.
// E.g. pinning a not-synced chat on device A shouldn't display it on device B yet.
//
// A pinned chat will appear on other devices only when the first
// message is sent from the device that read the invitation - which is the same
// behavior as with un-pinned chats.
ChatIdBlocked::get_for_contact(self, contact_id, Blocked::Yes)
ChatIdBlocked::get_for_contact(self, contact_id, Blocked::Request)
.await?
.id
}
+28 -182
View File
@@ -3,9 +3,7 @@ use std::sync::Arc;
use super::*;
use crate::Event;
use crate::chatlist::get_archived_cnt;
use crate::constants::{
self, DC_GCL_ARCHIVED_ONLY, DC_GCL_NO_SPECIALS, N_MSGS_TO_NEW_BROADCAST_MEMBER,
};
use crate::constants::{DC_GCL_ARCHIVED_ONLY, DC_GCL_NO_SPECIALS, N_MSGS_TO_NEW_BROADCAST_MEMBER};
use crate::ephemeral::Timer;
use crate::headerdef::HeaderDef;
use crate::imex::{ImexMode, has_backup, imex};
@@ -750,7 +748,7 @@ async fn test_leave_group() -> Result<()> {
assert_eq!(get_chat_contacts(&alice, alice_chat_id).await?.len(), 1);
assert_eq!(rcvd_leave_msg.state, MessageState::InNoticed);
assert_eq!(rcvd_leave_msg.state, MessageState::InSeen);
alice.emit_event(EventType::Test);
alice
@@ -1283,7 +1281,7 @@ async fn test_marknoticed_all_chats() -> Result<()> {
tcm.section("bob: receive messages, accept all chats and send a reply to each messsage");
while let Some(sent_msg) = alice.pop_sent_msg_opt().await {
while let Some(sent_msg) = alice.pop_sent_msg_opt(Duration::default()).await {
let bob_message = bob.recv_msg(&sent_msg).await;
let bob_chat_id = bob_message.chat_id;
bob_chat_id.accept(bob).await?;
@@ -1291,7 +1289,7 @@ async fn test_marknoticed_all_chats() -> Result<()> {
}
tcm.section("alice: receive replies from bob");
while let Some(sent_msg) = bob.pop_sent_msg_opt().await {
while let Some(sent_msg) = bob.pop_sent_msg_opt(Duration::default()).await {
alice.recv_msg(&sent_msg).await;
}
// ensure chats have unread messages
@@ -1374,18 +1372,6 @@ async fn test_markfresh_chat() -> Result<()> {
assert_eq!(bob_chat_id.get_fresh_msg_cnt(bob).await?, 0);
assert_eq!(bob.get_fresh_msgs().await?.len(), 0);
// Marking a message as seen results to sending an MDN to the contact and self.
message::markseen_msgs(bob, vec![bob_msg2.id]).await?;
assert_eq!(
bob.sql
.count(
"SELECT COUNT(*) FROM smtp_mdns WHERE from_id=?",
(bob_msg2.from_id,)
)
.await?,
1
);
// bob marks the chat as fresh again, fresh count is 1 again
markfresh_chat(bob, bob_chat_id).await?;
let bob_msg1 = Message::load_from_db(bob, bob_msg1.id).await?;
@@ -1647,7 +1633,6 @@ async fn test_set_chat_name() {
"another name",
"something different",
] {
SystemTime::shift(Duration::from_secs(1));
set_chat_name(alice, chat_id, new_name).await.unwrap();
let sent_msg = alice.pop_sent_msg().await;
let received_msg = bob.recv_msg(&sent_msg).await;
@@ -2830,7 +2815,7 @@ async fn test_cant_remove_nonmember() -> Result<()> {
let alice_charlie_id = alice.add_or_lookup_contact_id(charlie).await;
remove_contact_from_chat(alice, alice_broadcast_id, alice_charlie_id).await?;
assert!(alice.pop_sent_msg_opt().await.is_none());
assert!(alice.pop_sent_msg_opt(Duration::ZERO).await.is_none());
assert!(!remove_from_chat_contacts_table(alice, alice_broadcast_id, alice_charlie_id).await?);
assert!(
!remove_from_chat_contacts_table_without_trace(alice, alice_broadcast_id, alice_charlie_id)
@@ -2985,7 +2970,6 @@ async fn test_broadcast_change_name() -> Result<()> {
{
tcm.section("Alice changes the chat name");
SystemTime::shift(Duration::from_secs(1));
set_chat_name(alice, broadcast_id, "My great broadcast").await?;
let sent = alice.pop_sent_msg().await;
@@ -3004,7 +2988,6 @@ async fn test_broadcast_change_name() -> Result<()> {
{
tcm.section("Alice changes the chat name again, but the system message is lost somehow");
SystemTime::shift(Duration::from_secs(1));
set_chat_name(alice, broadcast_id, "Broadcast channel").await?;
let chat = Chat::load_from_db(alice, broadcast_id).await?;
@@ -3081,11 +3064,13 @@ async fn test_broadcast_resend_to_new_member() -> Result<()> {
}
for i in 0..N_MSGS_TO_NEW_BROADCAST_MEMBER {
let rev_order = false;
let resent_msg = alice.pop_sent_msg_ex(rev_order).await.unwrap();
let resent_msg = alice
.pop_sent_msg_ex(rev_order, Duration::ZERO)
.await
.unwrap();
let fiona_msg = fiona.recv_msg(&resent_msg).await;
assert_eq!(fiona_msg.chat_id, fiona_bc_id);
assert_eq!(fiona_msg.text, (i + 1).to_string());
assert_eq!(fiona_msg.param.get_bool(Param::WantsMdn).unwrap(), true);
assert!(resent_msg.recipients.contains("fiona@example.net"));
assert!(!resent_msg.recipients.contains("bob@"));
// The message is undecryptable for Bob, he mustn't be able to know yet that somebody joined
@@ -3099,7 +3084,7 @@ async fn test_broadcast_resend_to_new_member() -> Result<()> {
);
bob.recv_msg_trash(&resent_msg).await;
}
assert!(alice.pop_sent_msg_opt().await.is_none());
assert!(alice.pop_sent_msg_opt(Duration::ZERO).await.is_none());
Ok(())
}
@@ -3357,7 +3342,6 @@ async fn test_broadcasts_name_and_avatar() -> Result<()> {
assert_eq!(bob_chat.get_profile_image(bob).await?, None);
tcm.section("Change broadcast channel name, and check that receivers see it");
SystemTime::shift(Duration::from_secs(1));
set_chat_name(alice, alice_chat_id, "New Channel name").await?;
let sent = alice.pop_sent_msg().await;
let rcvd = bob.recv_msg(&sent).await;
@@ -3523,9 +3507,8 @@ async fn test_chat_description(
"",
"ä ẟ 😂",
] {
SystemTime::shift(Duration::from_secs(1));
tcm.section(&format!(
"Alice sets the chat description to {description:?}"
"Alice sets the chat description to '{description}'"
));
set_chat_description(alice, alice_chat_id, description).await?;
let sent = alice.pop_sent_msg().await;
@@ -3564,7 +3547,12 @@ async fn test_chat_description(
tcm.section("Alice calls set_chat_description() without actually changing the description");
set_chat_description(alice, alice_chat_id, "ä ẟ 😂").await?;
assert!(alice.pop_sent_msg_opt().await.is_none());
assert!(
alice
.pop_sent_msg_opt(Duration::from_secs(0))
.await
.is_none()
);
Ok(())
}
@@ -3589,7 +3577,12 @@ async fn test_setting_empty_chat_description() -> Result<()> {
let _hi = alice.send_text(alice_chat_id, "hi").await;
set_chat_description(alice, alice_chat_id, "").await?;
assert!(alice.pop_sent_msg_opt().await.is_none());
assert!(
alice
.pop_sent_msg_opt(Duration::from_secs(0))
.await
.is_none()
);
Ok(())
}
@@ -4536,9 +4529,7 @@ async fn test_get_chat_media_webxdc_order() -> Result<()> {
assert_eq!(media.first().unwrap(), &instance1_id);
assert_eq!(media.get(1).unwrap(), &instance2_id);
SystemTime::shift(Duration::from_secs(1));
// add a status update for the other instance; that resorts the list
// add a status update for the oder instance; that resorts the list
alice
.send_webxdc_status_update(instance1_id, r#"{"payload": {"foo": "bar"}}"#)
.await?;
@@ -4954,6 +4945,10 @@ async fn test_sync_broadcast_and_send_message() -> Result<()> {
vec![a2b_contact_id]
);
// alice2's smeared clock may be behind alice1's one, so we need to work around "hi" appearing
// before "You joined the channel." for bob. alice1 makes 3 more calls of
// create_smeared_timestamp() than alice2 does as of 2026-03-10.
SystemTime::shift(Duration::from_secs(3));
tcm.section("Alice's second device sends a message to the channel");
let sent_msg = alice2.send_text(a2_broadcast_id, "hi").await;
let msg = bob.recv_msg(&sent_msg).await;
@@ -5104,126 +5099,6 @@ async fn test_broadcast_contacts_are_hidden() -> Result<()> {
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_blocked_bob_cant_join_chat() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice1 = &tcm.alice().await;
let alice2 = &tcm.alice().await;
let bob = &tcm.bob().await;
for a in [alice1, alice2] {
a.set_config_bool(Config::SyncMsgs, true).await?;
}
// The observing device has Bob blocked from the early start.
let alice2_bob_id = alice2.add_or_lookup_contact_id(bob).await;
Contact::block(alice2, alice2_bob_id).await?;
let alice1_chat_id = create_group(alice1, "").await?;
sync(alice1, alice2).await;
let alice1_chat = Chat::load_from_db(alice1, alice1_chat_id).await?;
let (alice2_chat_id, _blocked) = get_chat_id_by_grpid(alice2, &alice1_chat.grpid)
.await?
.unwrap();
let qr = get_securejoin_qr(alice1, Some(alice1_chat_id)).await?;
sync(alice1, alice2).await;
tcm.exec_securejoin_qr_multi_device(bob, &[alice1, alice2], &qr)
.await;
let alice1_bob_id = alice1.add_or_lookup_contact_id(bob).await;
assert_eq!(get_chat_contacts(alice1, alice1_chat_id).await?.len(), 2);
// "vg-member-added" from alice1 adds bob for alice2 to provide membership consistency on
// devices.
assert_eq!(get_chat_contacts(alice2, alice2_chat_id).await?.len(), 2);
remove_contact_from_chat(alice1, alice1_chat_id, alice1_bob_id).await?;
bob.recv_msg(&alice1.pop_sent_msg().await).await;
tcm.exec_securejoin_qr(bob, alice1, &qr).await;
// Bob can join again if he isn't blocked.
assert_eq!(get_chat_contacts(alice1, alice1_chat_id).await?.len(), 2);
Contact::block(alice1, alice1_bob_id).await?;
remove_contact_from_chat(alice1, alice1_chat_id, alice1_bob_id).await?;
bob.recv_msg(&alice1.pop_sent_msg().await).await;
tcm.exec_securejoin_qr(bob, alice1, &qr).await;
let members = get_chat_contacts(alice1, alice1_chat_id).await?;
assert_eq!(members.len(), 1);
assert!(members.contains(&ContactId::SELF));
let past_members = get_past_chat_contacts(alice1, alice1_chat_id).await?;
assert_eq!(past_members.len(), 1);
assert!(past_members.contains(&alice1_bob_id));
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_blocked_bob_cant_create_11_chat_via_securejoin() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice1 = &tcm.alice().await;
let alice2 = &tcm.alice().await;
let bob = &tcm.bob().await;
for a in [alice1, alice2] {
a.set_config_bool(Config::SyncMsgs, true).await?;
}
// The observing device has Bob blocked.
let alice2_bob_id = alice2.add_or_lookup_contact_id(bob).await;
Contact::block(alice2, alice2_bob_id).await?;
let qr = get_securejoin_qr(alice1, None).await?;
sync(alice1, alice2).await;
let chat_cnt = get_chat_cnt(alice1).await?;
assert_eq!(get_chat_cnt(alice2).await?, chat_cnt);
tcm.exec_securejoin_qr_multi_device(bob, &[alice1, alice2], &qr)
.await;
assert_eq!(get_chat_cnt(alice1).await?, chat_cnt + 1);
assert_eq!(get_chat_cnt(alice2).await?, chat_cnt);
Ok(())
}
/// Regression test:
///
/// Pinning a chat that didn't complete securejoin on one device
/// should not cause a blank chat to be displayed on other device.
/// Such chat should be marked as blocked until the first message is sent.
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_sync_no_blank_chat() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice1 = &tcm.alice().await;
let alice2 = &tcm.alice().await;
let bob = &tcm.bob().await;
for a in [alice1, alice2] {
a.set_config_bool(Config::SyncMsgs, true).await?;
a.set_config_bool(Config::BccSelf, true).await?;
}
let qr = get_securejoin_qr(bob, None).await?;
// Scan QR on alice1 and pin the resulting chat.
let chat_id = tcm.exec_securejoin_qr(alice1, bob, &qr).await;
chat_id
.set_visibility(&alice1.ctx, ChatVisibility::Pinned)
.await?;
// alice2 should receive a sync item about changing visibility of a chat it doesn't know about.
sync(alice1, alice2).await;
// Chat created on alice2 should be blocked (hidden)
let chat = Chat::load_from_db(&alice2.ctx, chat_id).await?;
assert_eq!(chat.blocked, Blocked::Yes);
assert_eq!(chat.get_name(), "");
assert_eq!(chat.visibility, ChatVisibility::Pinned);
// First message should unblock and fill missing data on other device.
let msg = alice1.send_text(chat_id, "meow").await;
alice2.recv_msg(&msg).await;
let chat = Chat::load_from_db(&alice2.ctx, chat_id).await?;
assert_eq!(chat.blocked, Blocked::Not);
assert_eq!(chat.get_name(), "bob@example.net");
assert_eq!(chat.visibility, ChatVisibility::Pinned);
Ok(())
}
/// Tests sending JPEG image with .png extension.
///
/// This is a regression test, previously sending failed
@@ -5884,35 +5759,6 @@ async fn test_send_edit_request() -> Result<()> {
let forwarded = alice2.get_last_msg().await;
assert!(!forwarded.is_edited());
// If a message is too long after editing, it becomes an HTML message on the receiver side. On
// the sender side it's still text so that it can be edited again.
static REPEAT_TXT: &str = "this text with 42 chars is just repeated.\n";
static REPEAT_CNT: usize = constants::DC_DESIRED_TEXT_LEN / REPEAT_TXT.len() + 2;
let long_txt = REPEAT_TXT.repeat(REPEAT_CNT);
send_edit_request(alice, alice_msg.id, long_txt.clone()).await?;
let sent = alice.pop_sent_msg().await;
let test = Message::load_from_db(alice, alice_msg.id).await?;
assert!(!test.has_html());
assert_eq!(test.text, long_txt);
bob.recv_msg_opt(&sent).await;
let test = Message::load_from_db(bob, bob_msg.id).await?;
assert!(test.is_edited());
assert!(test.has_html());
let html = test.id.get_html(bob).await?.unwrap();
assert_eq!(html.matches("just repeated.<br/>").count(), REPEAT_CNT);
assert!(test.text.matches("just repeated.").count() > 0);
// Alice shortens the message back so it's not HTML for Bob anymore.
send_edit_request(alice, alice_msg.id, "Text me on Delta.Chat".to_string()).await?;
let sent = alice.pop_sent_msg().await;
let test = Message::load_from_db(alice, alice_msg.id).await?;
assert_eq!(test.text, "Text me on Delta.Chat");
bob.recv_msg_opt(&sent).await;
let test = Message::load_from_db(bob, bob_msg.id).await?;
assert!(test.is_edited());
assert!(!test.has_html());
assert_eq!(test.text, "Text me on Delta.Chat");
Ok(())
}
+6 -6
View File
@@ -319,12 +319,6 @@ pub enum Config {
/// True if account is configured.
Configured,
/// Deprecated, we are trying to get rid of this global setting.
/// It is possible to configure a profile with both chatmail relays
/// and classical email servers.
///
/// Most usages in UIs can be replaced by `force_encryption`.
///
/// True if account is a chatmail account.
IsChatmail,
@@ -461,6 +455,12 @@ pub enum Config {
/// Return an error from `receive_imf_inner()`. For tests.
SimulateReceiveImfError,
/// Enable composing emails with Header Protection as defined in
/// <https://www.rfc-editor.org/rfc/rfc9788.html> "Header Protection for Cryptographically
/// Protected Email".
#[strum(props(default = "1"))]
StdHeaderProtectionComposing,
/// Who can call me.
///
/// The options are from the `WhoCanCallMe` enum.
-15
View File
@@ -49,21 +49,6 @@ use crate::{chat, provider};
/// see <https://github.com/chatmail/core/issues/7608>
pub(crate) const MAX_TRANSPORT_RELAYS: usize = 5;
/// Hard-coded candidates for default relays.
/// In the future, we want to use it during onboarding;
/// note that before onboarding automatically on any of these,
/// we need to ask the admins whether their relay is able to handle this.
/// For now, this is just the first 6 relays from chatmail.at/relays.
#[allow(unused)]
const DEFAULT_RELAY_CANDIDATES: &[&str] = &[
"mehl.cloud",
"mailchat.pl",
"chatmail.woodpeckersnest.space",
"chatmail.culturanerd.it",
"tarpit.fun",
"d.gaufr.es",
];
macro_rules! progress {
($context:tt, $progress:expr, $comment:expr) => {
assert!(
+3 -4
View File
@@ -31,6 +31,7 @@ use crate::quota::QuotaInfo;
use crate::scheduler::{ConnectivityStore, SchedulerState};
use crate::sql::Sql;
use crate::stock_str::StockStrings;
use crate::timesmearing::SmearedTimestamp;
use crate::tools::{self, duration_to_str, time, time_elapsed};
use crate::transport::ConfiguredLoginParam;
use crate::{chatlist_events, stats};
@@ -226,6 +227,7 @@ pub struct InnerContext {
/// Blob directory path
pub(crate) blobdir: PathBuf,
pub(crate) sql: Sql,
pub(crate) smeared_timestamp: SmearedTimestamp,
/// The global "ongoing" process state.
///
/// This is a global mutex-like state for operations which should be modal in the
@@ -484,6 +486,7 @@ impl Context {
blobdir,
running_state: RwLock::new(Default::default()),
sql: Sql::new(dbfile),
smeared_timestamp: SmearedTimestamp::new(),
oauth2_mutex: Mutex::new(()),
wrong_pw_warning_mutex: Mutex::new(()),
housekeeping_mutex: Mutex::new(()),
@@ -566,10 +569,6 @@ impl Context {
self.scheduler.maybe_network().await;
}
/// Deprecated, we are trying to get rid of this global setting.
/// It is possible to configure a profile with both chatmail relays
/// and classical email servers.
///
/// Returns true if an account is on a chatmail server.
pub async fn is_chatmail(&self) -> Result<bool> {
self.get_config_bool(Config::IsChatmail).await
+7 -29
View File
@@ -6,16 +6,13 @@ use std::io::Cursor;
use anyhow::{Context as _, Result, bail};
use mailparse::ParsedMail;
use pgp::composed::DecryptionOptions;
use pgp::composed::Esk;
use pgp::composed::Message;
use pgp::composed::PlainSessionKey;
use pgp::composed::SignedSecretKey;
use pgp::composed::TheRing;
use pgp::composed::decrypt_session_key_with_password;
use pgp::packet::SymKeyEncryptedSessionKey;
use pgp::types::Password;
use pgp::types::Seipdv1ReadMode;
use pgp::types::StringToKey;
use crate::chat::ChatId;
@@ -51,15 +48,6 @@ pub(crate) async fn decrypt(
};
let expected_sender_fingerprint: Option<String>;
let abort_early = true;
// Use streaming mode for SEIPDv1 decryption to save memory.
// This was the default in rPGP 0.19.0
// and requires explicitly changing the mode in rPGP 0.20.0.
// SEPIDv2 is decrypted in streaming mode in any case.
let decrypt_options =
DecryptionOptions::new().set_seipdv1_read_mode(Seipdv1ReadMode::Streaming);
let plain = if let Message::Encrypted { esk, .. } = &*msg
// We only allow one ESK for symmetrically encrypted messages
// to avoid dealing with messages that are encrypted to multiple symmetric keys
@@ -73,15 +61,9 @@ pub(crate) async fn decrypt(
expected_sender_fingerprint = fingerprint;
tokio::task::spawn_blocking(move || -> Result<Message<'_>> {
let ring = TheRing {
session_keys: vec![psk],
decrypt_options,
..Default::default()
};
let (plain, _ring_result) = msg
.decrypt_the_ring(ring, abort_early)
.context("decrypt_the_ring")?;
let plain = msg
.decrypt_with_session_key(psk)
.context("decrypt_with_session_key")?;
let plain: Message<'static> = plain.decompress()?;
Ok(plain)
@@ -93,15 +75,11 @@ pub(crate) async fn decrypt(
expected_sender_fingerprint = None;
tokio::task::spawn_blocking(move || -> Result<Message<'_>> {
let empty_pw = Password::empty();
let secret_keys: Vec<&SignedSecretKey> = secret_keys.iter().collect();
let ring = TheRing {
secret_keys,
decrypt_options,
..Default::default()
};
let (plain, _ring_result) = msg
.decrypt_the_ring(ring, abort_early)
.context("decrypt_the_ring")?;
let plain = msg
.decrypt_with_keys(vec![&empty_pw], secret_keys)
.context("decrypt_with_keys")?;
let plain: Message<'static> = plain.decompress()?;
Ok(plain)
+72 -2
View File
@@ -160,6 +160,7 @@ pub(crate) async fn download_msg(
let Some((server_uid, server_folder, msg_transport_id)) = row else {
// No IMAP record found, we don't know the UID and folder.
delete_from_available_post_msgs(context, &rfc724_mid).await?;
return Err(anyhow!(
"IMAP location for {rfc724_mid:?} post-message is unknown"
));
@@ -233,6 +234,31 @@ async fn set_state_to_failure(context: &Context, rfc724_mid: &str) -> Result<()>
Ok(())
}
async fn available_post_msgs_contains_rfc724_mid(
context: &Context,
rfc724_mid: &str,
) -> Result<bool> {
Ok(context
.sql
.query_get_value::<String>(
"SELECT rfc724_mid FROM available_post_msgs WHERE rfc724_mid=?",
(&rfc724_mid,),
)
.await?
.is_some())
}
async fn delete_from_available_post_msgs(context: &Context, rfc724_mid: &str) -> Result<()> {
context
.sql
.execute(
"DELETE FROM available_post_msgs WHERE rfc724_mid=?",
(&rfc724_mid,),
)
.await?;
Ok(())
}
async fn delete_from_downloads(context: &Context, rfc724_mid: &str) -> Result<()> {
context
.sql
@@ -260,6 +286,7 @@ pub(crate) async fn download_msgs(context: &Context, session: &mut Session) -> R
let res = download_msg(context, rfc724_mid.clone(), session).await;
if let Ok(Some(())) = res {
delete_from_downloads(context, rfc724_mid).await?;
delete_from_available_post_msgs(context, rfc724_mid).await?;
}
if let Err(err) = res {
warn!(
@@ -273,13 +300,18 @@ pub(crate) async fn download_msgs(context: &Context, session: &mut Session) -> R
"{rfc724_mid} download failed and there is no downloaded pre-message."
);
delete_from_downloads(context, rfc724_mid).await?;
} else {
} else if available_post_msgs_contains_rfc724_mid(context, rfc724_mid).await? {
warn!(
context,
"Failed to fetch {rfc724_mid}, setting DownloadState to Failure."
"{rfc724_mid} is in available_post_msgs table but we failed to fetch it,
so set the message to DownloadState::Failure - probably it was deleted on the server in the meantime"
);
set_state_to_failure(context, rfc724_mid).await?;
delete_from_downloads(context, rfc724_mid).await?;
delete_from_available_post_msgs(context, rfc724_mid).await?;
} else {
// leave the message in DownloadState::InProgress;
// it will be downloaded once it arrives.
}
}
}
@@ -287,6 +319,44 @@ pub(crate) async fn download_msgs(context: &Context, session: &mut Session) -> R
Ok(())
}
/// Downloads known post-messages without pre-messages
/// in order to guard against lost pre-messages.
pub(crate) async fn download_known_post_messages_without_pre_message(
context: &Context,
session: &mut Session,
) -> Result<()> {
let rfc724_mids = context
.sql
.query_map_vec("SELECT rfc724_mid FROM available_post_msgs", (), |row| {
let rfc724_mid: String = row.get(0)?;
Ok(rfc724_mid)
})
.await?;
for rfc724_mid in &rfc724_mids {
if msg_is_downloaded_for(context, rfc724_mid).await? {
delete_from_available_post_msgs(context, rfc724_mid).await?;
continue;
}
// Download the Post-Message unconditionally,
// because the Pre-Message got lost.
// The message may be in the wrong order,
// but at least we have it at all.
let res = download_msg(context, rfc724_mid.clone(), session).await;
if let Ok(Some(())) = res {
delete_from_available_post_msgs(context, rfc724_mid).await?;
}
if let Err(err) = res {
warn!(
context,
"download_known_post_messages_without_pre_message: Failed to download message rfc724_mid={rfc724_mid}: {:#}.",
err
);
}
}
Ok(())
}
#[cfg(test)]
mod tests {
use num_traits::FromPrimitive;
+31 -6
View File
@@ -83,20 +83,29 @@ impl EncryptHelper {
None
};
let shared_secret = shared_secret.to_string();
let mut raw_message = Vec::new();
let cursor = Cursor::new(&mut raw_message);
mail_to_encrypt.clone().write_part(cursor).ok();
let ctext = tokio::task::spawn_blocking(move || {
pgp::symm_encrypt_message(raw_message, sign_key, shared_secret, compress)
})
.await??;
let ctext =
pgp::symm_encrypt_message(raw_message, sign_key, shared_secret, compress).await?;
Ok(ctext)
}
}
/// Ensures a private key exists for the configured user.
///
/// Normally the private key is generated when the first message is
/// sent but in a few locations there are no such guarantees,
/// e.g. when exporting keys, and calling this function ensures a
/// private key will be present.
// TODO, remove this once deltachat::key::Key no longer exists.
pub async fn ensure_secret_key_exists(context: &Context) -> Result<()> {
load_self_public_key(context).await?;
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
@@ -106,7 +115,23 @@ mod tests {
use crate::message::Message;
use crate::mimeparser::SystemMessage;
use crate::receive_imf::receive_imf;
use crate::test_utils::TestContextManager;
use crate::test_utils::{TestContext, TestContextManager};
mod ensure_secret_key_exists {
use super::*;
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_prexisting() {
let t = TestContext::new_alice().await;
assert!(ensure_secret_key_exists(&t).await.is_ok());
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_not_configured() {
let t = TestContext::new().await;
assert!(ensure_secret_key_exists(&t).await.is_err());
}
}
#[test]
fn test_mailmime_parse() {
+15 -6
View File
@@ -11,6 +11,7 @@ use crate::message::markseen_msgs;
use crate::receive_imf::receive_imf;
use crate::test_utils;
use crate::test_utils::{TestContext, TestContextManager};
use crate::timesmearing::MAX_SECONDS_TO_LEND_FROM_FUTURE;
use crate::{
chat::{self, Chat, ChatItem, create_group, send_text_msg},
tools::IsNoneOrEmpty,
@@ -171,7 +172,7 @@ async fn test_ephemeral_unpromoted() -> Result<()> {
chat_id
.set_ephemeral_timer(&alice, Timer::Enabled { duration: 60 })
.await?;
let sent = alice.pop_sent_msg_opt().await;
let sent = alice.pop_sent_msg_opt(Duration::from_secs(1)).await;
assert!(sent.is_none());
assert_eq!(
chat_id.get_ephemeral_timer(&alice).await?,
@@ -181,13 +182,13 @@ async fn test_ephemeral_unpromoted() -> Result<()> {
// Promote the group.
send_text_msg(&alice, chat_id, "hi!".to_string()).await?;
assert!(chat_id.is_promoted(&alice).await?);
let sent = alice.pop_sent_msg_opt().await;
let sent = alice.pop_sent_msg_opt(Duration::from_secs(1)).await;
assert!(sent.is_some());
chat_id
.set_ephemeral_timer(&alice.ctx, Timer::Disabled)
.await?;
let sent = alice.pop_sent_msg_opt().await;
let sent = alice.pop_sent_msg_opt(Duration::from_secs(1)).await;
assert!(sent.is_some());
assert_eq!(chat_id.get_ephemeral_timer(&alice).await?, Timer::Disabled);
@@ -354,9 +355,17 @@ async fn test_ephemeral_delete_msgs() -> Result<()> {
let now = time();
let msg = t.send_text(bob_chat.id, "Message text").await;
check_msg_will_be_deleted(t, msg.sender_msg_id, &bob_chat, now + 1799, time() + 1801)
.await
.unwrap();
check_msg_will_be_deleted(
t,
msg.sender_msg_id,
&bob_chat,
now + 1799,
// The message may appear to be sent MAX_SECONDS_TO_LEND_FROM_FUTURE later and
// therefore be deleted MAX_SECONDS_TO_LEND_FROM_FUTURE later.
time() + 1801 + MAX_SECONDS_TO_LEND_FROM_FUTURE,
)
.await
.unwrap();
// Enable ephemeral messages with Bob -> message will be deleted after 60s.
// This tests that the message is deleted at min(ephemeral deletion time, DeleteDeviceAfter deletion time).
-11
View File
@@ -181,17 +181,6 @@ pub enum EventType {
msg_id: MsgId,
},
/// Like [`EventType::MsgRead`], but also fires on subsequent MDNs,
/// if there are multiple receivers, i.e. in groups and channels.
#[serde(rename_all = "camelCase")]
MsgReadCountChanged {
/// ID of the chat which the message belongs to.
chat_id: ChatId,
/// ID of the message that was read.
msg_id: MsgId,
},
/// A single message was deleted.
///
/// This event means that the message will no longer appear in the messagelist.
+19 -64
View File
@@ -9,7 +9,7 @@
use std::mem;
use anyhow::{Context as _, Result, ensure};
use anyhow::{Context as _, Result};
use base64::Engine as _;
use mailparse::ParsedContentType;
use mime::Mime;
@@ -17,12 +17,10 @@ use mime::Mime;
use crate::context::Context;
use crate::headerdef::{HeaderDef, HeaderDefMap};
use crate::log::warn;
use crate::message::{Message, MsgId};
use crate::message::{self, Message, MsgId};
use crate::mimeparser::parse_message_id;
use crate::param::{Param::SendHtml, Params};
use crate::param::Param::SendHtml;
use crate::plaintext::PlainText;
use crate::sql;
use crate::tools::{buf_compress, buf_decompress};
impl Message {
/// Check if the message can be retrieved as HTML.
@@ -260,71 +258,28 @@ impl MsgId {
/// NB: we do not save raw mime unconditionally in the database to save space.
/// The corresponding ffi-function is `dc_get_msg_html()`.
pub async fn get_html(self, context: &Context) -> Result<Option<String>> {
let (param, headers, compressed) = context
.sql
.query_row(
"SELECT param, mime_headers, mime_compressed FROM msgs WHERE id=?",
(self,),
|row| {
let param: String = row.get(0)?;
let param: Params = param.parse().unwrap_or_default();
let headers = sql::row_get_vec(row, 1)?;
let compressed: bool = row.get(2)?;
Ok((param, headers, compressed))
},
)
.await?;
if let Some(html) = param.get(SendHtml) {
// If there are many concurrent db readers, going to the queue earlier makes sense.
let (param, rawmime) = tokio::join!(
self.get_param(context),
message::get_mime_headers(context, self)
);
if let Some(html) = param?.get(SendHtml) {
return Ok(Some(html.to_string()));
}
let from_rawmime = |rawmime: Vec<u8>| {
if !rawmime.is_empty() {
match HtmlMsgParser::from_bytes(context, &rawmime) {
Err(err) => {
warn!(context, "get_html: parser error: {:#}", err);
Ok(None)
}
Ok((parser, _)) => Ok(Some(parser.html)),
}
} else {
warn!(context, "get_html: no mime for {}", self);
Ok(None)
}
};
if compressed {
return from_rawmime(buf_decompress(&headers)?);
}
let headers2 = headers.clone();
let compressed = match tokio::task::block_in_place(move || buf_compress(&headers2)) {
Err(e) => {
warn!(context, "get_mime_headers: buf_compress() failed: {}", e);
return from_rawmime(headers);
}
Ok(o) => o,
};
let update = |conn: &mut rusqlite::Connection| {
match conn.execute(
"
UPDATE msgs SET mime_headers=?, mime_compressed=1
WHERE id=? AND mime_headers!='' AND mime_compressed=0",
(compressed, self),
) {
Ok(rows_updated) => ensure!(rows_updated <= 1),
Err(e) => {
warn!(context, "get_mime_headers: UPDATE failed: {}", e);
return Err(e.into());
let rawmime = rawmime?;
if !rawmime.is_empty() {
match HtmlMsgParser::from_bytes(context, &rawmime) {
Err(err) => {
warn!(context, "get_html: parser error: {:#}", err);
Ok(None)
}
Ok((parser, _)) => Ok(Some(parser.html)),
}
Ok(())
};
if let Err(e) = context.sql.call_write(update).await {
warn!(
context,
"get_mime_headers: failed to update mime_headers: {}", e
);
} else {
warn!(context, "get_html: no mime for {}", self);
Ok(None)
}
from_rawmime(headers)
}
}
+205 -7
View File
@@ -6,7 +6,7 @@
use std::{
cmp::max,
cmp::min,
collections::{BTreeMap, HashMap},
collections::{BTreeMap, BTreeSet, HashMap},
iter::Peekable,
mem::take,
sync::atomic::Ordering,
@@ -21,7 +21,8 @@ use futures_lite::FutureExt;
use ratelimit::Ratelimit;
use url::Url;
use crate::chat::{self, ChatIdBlocked, add_device_msg};
use crate::chat::{self, ChatId, ChatIdBlocked, add_device_msg};
use crate::chatlist_events;
use crate::config::Config;
use crate::constants::{Blocked, DC_VERSION_STR};
use crate::contact::ContactId;
@@ -30,7 +31,7 @@ use crate::ensure_and_debug_assert;
use crate::events::EventType;
use crate::headerdef::{HeaderDef, HeaderDefMap};
use crate::log::{LogExt, warn};
use crate::message::{self, Message};
use crate::message::{self, Message, MessageState, MsgId};
use crate::mimeparser;
use crate::net::proxy::ProxyConfig;
use crate::net::session::SessionStream;
@@ -607,6 +608,7 @@ impl Imap {
let _fetch_msgs_lock_guard = context.fetch_msgs_mutex.lock().await;
let mut uids_fetch: Vec<u32> = Vec::new();
let mut available_post_msgs: Vec<String> = Vec::new();
let mut download_later: Vec<String> = Vec::new();
let mut uid_message_ids = BTreeMap::new();
let mut largest_uid_skipped = None;
@@ -694,6 +696,8 @@ impl Imap {
.is_some()
{
info!(context, "{message_id:?} is a post-message.");
available_post_msgs.push(message_id.clone());
let is_bot = context.get_config_bool(Config::Bot).await?;
if is_bot && download_limit.is_none_or(|download_limit| size <= download_limit)
{
@@ -789,8 +793,18 @@ impl Imap {
chat::mark_old_messages_as_noticed(context, received_msgs).await?;
if fetch_res.is_ok() {
info!(context, "download_later: {}.", download_later.len(),);
info!(
context,
"available_post_msgs: {}, download_later: {}.",
available_post_msgs.len(),
download_later.len(),
);
let trans_fn = |t: &mut rusqlite::Transaction| {
let mut stmt = t.prepare("INSERT OR IGNORE INTO available_post_msgs VALUES (?)")?;
for rfc724_mid in available_post_msgs {
stmt.execute((rfc724_mid,))
.context("INSERT OR IGNORE INTO available_post_msgs")?;
}
let mut stmt =
t.prepare("INSERT OR IGNORE INTO download (rfc724_mid, msg_id) VALUES (?,0)")?;
for rfc724_mid in download_later {
@@ -1104,7 +1118,7 @@ impl Session {
Err(err) => {
warn!(
context,
"store_seen_flags_on_imap: Transport {transport_id}: Failed to select {folder}, will retry later: {err:#}."
"store_seen_flags_on_imap: Failed to select {folder}, will retry later: {err:#}."
);
continue;
}
@@ -1115,13 +1129,13 @@ impl Session {
} else if let Err(err) = self.add_flag_finalized_with_set(&uid_set, "\\Seen").await {
warn!(
context,
"Transport {transport_id}: Cannot mark messages {uid_set} in {folder} as seen, will retry later: {err:#}."
"Cannot mark messages {uid_set} in {folder} as seen, will retry later: {err:#}."
);
continue;
} else {
info!(
context,
"Transport {transport_id}: Marked messages {uid_set} in folder {folder} as seen."
"Marked messages {} in folder {} as seen.", uid_set, folder
);
}
context
@@ -1140,6 +1154,114 @@ impl Session {
Ok(())
}
/// Synchronizes `\Seen` flags using `CONDSTORE` extension.
pub(crate) async fn sync_seen_flags(&mut self, context: &Context, folder: &str) -> Result<()> {
if !self.can_condstore() {
info!(
context,
"Server does not support CONDSTORE, skipping flag synchronization."
);
return Ok(());
}
if context.get_config_bool(Config::TeamProfile).await? {
return Ok(());
}
let folder_exists = self
.select_with_uidvalidity(context, folder)
.await
.context("Failed to select folder")?;
if !folder_exists {
return Ok(());
}
let mailbox = self
.selected_mailbox
.as_ref()
.with_context(|| format!("No mailbox selected, folder: {folder}"))?;
// Check if the mailbox supports MODSEQ.
// We are not interested in actual value of HIGHESTMODSEQ.
if mailbox.highest_modseq.is_none() {
info!(
context,
"Mailbox {} does not support mod-sequences, skipping flag synchronization.", folder
);
return Ok(());
}
let transport_id = self.transport_id();
let mut updated_chat_ids = BTreeSet::new();
let uid_validity = get_uidvalidity(context, transport_id, folder)
.await
.with_context(|| format!("failed to get UID validity for folder {folder}"))?;
let mut highest_modseq = get_modseq(context, transport_id, folder)
.await
.with_context(|| format!("failed to get MODSEQ for folder {folder}"))?;
let mut list = self
.uid_fetch("1:*", format!("(FLAGS) (CHANGEDSINCE {highest_modseq})"))
.await
.context("failed to fetch flags")?;
let mut got_unsolicited_fetch = false;
while let Some(fetch) = list
.try_next()
.await
.context("failed to get FETCH result")?
{
let uid = if let Some(uid) = fetch.uid {
uid
} else {
info!(context, "FETCH result contains no UID, skipping");
got_unsolicited_fetch = true;
continue;
};
let is_seen = fetch.flags().any(|flag| flag == Flag::Seen);
if is_seen
&& let Some(chat_id) = mark_seen_by_uid(context, transport_id, folder, uid_validity, uid)
.await
.with_context(|| {
format!("Transport {transport_id}: Failed to update seen status for msg {folder}/{uid}")
})?
{
updated_chat_ids.insert(chat_id);
}
if let Some(modseq) = fetch.modseq {
if modseq > highest_modseq {
highest_modseq = modseq;
}
} else {
warn!(context, "FETCH result contains no MODSEQ");
}
}
drop(list);
if got_unsolicited_fetch {
// We got unsolicited FETCH, which means some flags
// have been modified while our request was in progress.
// We may or may not have these new flags as a part of the response,
// so better skip next IDLE and do another round of flag synchronization.
info!(context, "Got unsolicited fetch, will skip idle");
self.new_mail = true;
}
set_modseq(context, transport_id, folder, highest_modseq)
.await
.with_context(|| format!("failed to set MODSEQ for folder {folder}"))?;
if !updated_chat_ids.is_empty() {
context.on_archived_chats_maybe_noticed();
}
for updated_chat_id in updated_chat_ids {
context.emit_event(EventType::MsgsNoticed(updated_chat_id));
chatlist_events::emit_chatlist_item_changed(context, updated_chat_id);
}
Ok(())
}
/// Fetches a list of messages by server UID.
///
/// Sends pairs of UID and info about each downloaded message to the provided channel.
@@ -1904,6 +2026,71 @@ pub(crate) async fn prefetch_should_download(
Ok(should_download)
}
/// Marks messages in `msgs` table as seen, searching for them by UID.
///
/// Returns updated chat ID if any message was marked as seen.
async fn mark_seen_by_uid(
context: &Context,
transport_id: u32,
folder: &str,
uid_validity: u32,
uid: u32,
) -> Result<Option<ChatId>> {
if let Some((msg_id, chat_id)) = context
.sql
.query_row_optional(
"SELECT id, chat_id FROM msgs
WHERE id > 9 AND rfc724_mid IN (
SELECT rfc724_mid FROM imap
WHERE transport_id=?
AND folder=?
AND uidvalidity=?
AND uid=?
LIMIT 1
)",
(transport_id, &folder, uid_validity, uid),
|row| {
let msg_id: MsgId = row.get(0)?;
let chat_id: ChatId = row.get(1)?;
Ok((msg_id, chat_id))
},
)
.await
.with_context(|| format!("failed to get msg and chat ID for IMAP message {folder}/{uid}"))?
{
let updated = context
.sql
.execute(
"UPDATE msgs SET state=?1
WHERE (state=?2 OR state=?3)
AND id=?4",
(
MessageState::InSeen,
MessageState::InFresh,
MessageState::InNoticed,
msg_id,
),
)
.await
.with_context(|| format!("failed to update msg {msg_id} state"))?
> 0;
if updated {
msg_id
.start_ephemeral_timer(context)
.await
.with_context(|| format!("failed to start ephemeral timer for message {msg_id}"))?;
Ok(Some(chat_id))
} else {
// Message state has not changed.
Ok(None)
}
} else {
// There is no message is `msgs` table matching the given UID.
Ok(None)
}
}
/// Schedule marking the message as Seen on IMAP by adding all known IMAP messages corresponding to
/// the given Message-ID to `imap_markseen` table.
pub(crate) async fn markseen_on_imap_table(context: &Context, message_id: &str) -> Result<()> {
@@ -2001,6 +2188,17 @@ pub(crate) async fn set_modseq(
Ok(())
}
async fn get_modseq(context: &Context, transport_id: u32, folder: &str) -> Result<u64> {
Ok(context
.sql
.query_get_value(
"SELECT modseq FROM imap_sync WHERE transport_id=? AND folder=?",
(transport_id, folder),
)
.await?
.unwrap_or(0))
}
/// Builds a list of sequence/uid sets. The returned sets have each no more than around 1000
/// characters because according to <https://tools.ietf.org/html/rfc2683#section-3.2.1.5>
/// command lines should not be much more than 1000 chars (servers should allow at least 8000 chars)
+4
View File
@@ -17,6 +17,10 @@ pub(crate) struct Capabilities {
/// <https://tools.ietf.org/html/rfc2087>
pub can_check_quota: bool,
/// True if the server has CONDSTORE capability as defined in
/// <https://tools.ietf.org/html/rfc7162>
pub can_condstore: bool,
/// True if the server has METADATA capability as defined in
/// <https://tools.ietf.org/html/rfc5464>
pub can_metadata: bool,
+1
View File
@@ -66,6 +66,7 @@ pub(crate) async fn determine_capabilities(
can_idle: caps.has_str("IDLE"),
can_move: caps.has_str("MOVE"),
can_check_quota: caps.has_str("QUOTA"),
can_condstore: caps.has_str("CONDSTORE"),
can_metadata: caps.has_str("METADATA"),
can_compress: caps.has_str("COMPRESS=DEFLATE"),
can_push: caps.has_str("XDELTAPUSH"),
-1
View File
@@ -1,5 +1,4 @@
use super::*;
use crate::chat::ChatId;
use crate::contact::Contact;
use crate::test_utils::TestContext;
+5 -1
View File
@@ -65,7 +65,11 @@ impl ImapSession {
self.maybe_close_folder(context).await?;
// select new folder
let res = self.select(folder).await;
let res = if self.can_condstore() {
self.select_condstore(folder).await
} else {
self.select(folder).await
};
let transport_id = self.transport_id();
+5 -1
View File
@@ -102,6 +102,10 @@ impl Session {
self.capabilities.can_check_quota
}
pub fn can_condstore(&self) -> bool {
self.capabilities.can_condstore
}
pub fn can_metadata(&self) -> bool {
self.capabilities.can_metadata
}
@@ -111,7 +115,7 @@ impl Session {
}
// Returns true if IMAP server has `XCHATMAIL` capability.
pub(crate) fn is_chatmail(&self) -> bool {
pub fn is_chatmail(&self) -> bool {
self.capabilities.is_chatmail
}
+2 -1
View File
@@ -17,6 +17,7 @@ use crate::blob::BlobDirContents;
use crate::chat::delete_and_reset_all_device_msgs;
use crate::config::Config;
use crate::context::Context;
use crate::e2ee;
use crate::events::EventType;
use crate::key::{self, DcKey, SignedSecretKey};
use crate::log::{LogExt, warn};
@@ -169,7 +170,7 @@ async fn imex_inner(
if what == ImexMode::ExportBackup || what == ImexMode::ExportSelfKeys {
// before we export anything, make sure the private key exists
key::ensure_secret_key_exists(context)
e2ee::ensure_secret_key_exists(context)
.await
.context("Cannot create private key or private key not available")?;
+2 -3
View File
@@ -38,16 +38,15 @@ use tokio::fs;
use tokio::task::JoinHandle;
use tokio_util::sync::CancellationToken;
use crate::EventType;
use crate::chat::add_device_msg;
use crate::context::Context;
use crate::imex::BlobDirContents;
use crate::key;
use crate::log::warn;
use crate::message::Message;
use crate::qr::Qr;
use crate::stock_str::backup_transfer_msg_body;
use crate::tools::{TempPathGuard, create_id, time};
use crate::{EventType, e2ee};
use super::{DBFILE_BACKUP_NAME, export_backup_stream, export_database, import_backup_stream};
@@ -113,7 +112,7 @@ impl BackupProvider {
.context("Context dir not found")?;
// before we export, make sure the private key exists
key::ensure_secret_key_exists(context)
e2ee::ensure_secret_key_exists(context)
.await
.context("Cannot create private key or private key not available")?;
-27
View File
@@ -320,17 +320,6 @@ pub(crate) async fn load_self_public_key(context: &Context) -> Result<SignedPubl
}
}
/// Ensures a private key exists for the configured user.
///
/// Normally the private key is generated when the first message is
/// sent but in a few locations there are no such guarantees,
/// e.g. when exporting keys, and calling this function ensures a
/// private key will be present.
pub async fn ensure_secret_key_exists(context: &Context) -> Result<()> {
load_self_public_key(context).await?;
Ok(())
}
/// Returns our own public keyring.
///
/// No keys are generated and at most one key is returned.
@@ -909,20 +898,4 @@ i8pcjGO+IZffvyZJVRWfVooBJmWWbPB1pueo3tx8w3+fcuzpxz+RLFKaPyqXO+dD
"0102 0408 1020 4080 FF01\n0204 0810 2040 80FF 1314"
);
}
mod ensure_secret_key_exists {
use super::*;
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_prexisting() {
let t = TestContext::new_alice().await;
assert!(ensure_secret_key_exists(&t).await.is_ok());
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_not_configured() {
let t = TestContext::new().await;
assert!(ensure_secret_key_exists(&t).await.is_err());
}
}
}
+1
View File
@@ -94,6 +94,7 @@ mod smtp;
pub mod stock_str;
pub mod storage_usage;
mod sync;
mod timesmearing;
mod token;
mod transport;
mod update_helper;
+64 -9
View File
@@ -30,12 +30,13 @@ use crate::log::warn;
use crate::mimeparser::{SystemMessage, parse_message_id};
use crate::param::{Param, Params};
use crate::reaction::get_msg_reactions;
use crate::sql;
use crate::summary::Summary;
use crate::sync::SyncData;
use crate::tools::create_outgoing_rfc724_mid;
use crate::tools::{
get_filebytes, get_filemeta, gm2local_offset, read_file, sanitize_filename, time,
timestamp_to_str,
buf_compress, buf_decompress, get_filebytes, get_filemeta, gm2local_offset, read_file,
sanitize_filename, time, timestamp_to_str,
};
/// Message ID, including reserved IDs.
@@ -1623,6 +1624,62 @@ pub(crate) fn guess_msgtype_from_path_suffix(path: &Path) -> Option<(Viewtype, &
Some(info)
}
/// Get the raw mime-headers of the given message.
/// Raw headers are saved for large messages
/// that need a "Show full message..."
/// to see HTML part.
///
/// Returns an empty vector if there are no headers saved for the given message.
pub(crate) async fn get_mime_headers(context: &Context, msg_id: MsgId) -> Result<Vec<u8>> {
let (headers, compressed) = context
.sql
.query_row(
"SELECT mime_headers, mime_compressed FROM msgs WHERE id=?",
(msg_id,),
|row| {
let headers = sql::row_get_vec(row, 0)?;
let compressed: bool = row.get(1)?;
Ok((headers, compressed))
},
)
.await?;
if compressed {
return buf_decompress(&headers);
}
let headers2 = headers.clone();
let compressed = match tokio::task::block_in_place(move || buf_compress(&headers2)) {
Err(e) => {
warn!(context, "get_mime_headers: buf_compress() failed: {}", e);
return Ok(headers);
}
Ok(o) => o,
};
let update = |conn: &mut rusqlite::Connection| {
match conn.execute(
"\
UPDATE msgs SET mime_headers=?, mime_compressed=1 \
WHERE id=? AND mime_headers!='' AND mime_compressed=0",
(compressed, msg_id),
) {
Ok(rows_updated) => ensure!(rows_updated <= 1),
Err(e) => {
warn!(context, "get_mime_headers: UPDATE failed: {}", e);
return Err(e.into());
}
}
Ok(())
};
if let Err(e) = context.sql.call_write(update).await {
warn!(
context,
"get_mime_headers: failed to update mime_headers: {}", e
);
}
Ok(headers)
}
/// Delete a single message from the database, including references in other tables.
/// This may be called in batches; the final events are emitted in delete_msgs_locally_done() then.
pub(crate) async fn delete_msg_locally(context: &Context, msg: &Message) -> Result<()> {
@@ -1723,6 +1780,10 @@ pub async fn delete_msgs_ex(
"DELETE FROM download WHERE rfc724_mid=?",
(&msg.rfc724_mid,),
)?;
trans.execute(
"DELETE FROM available_post_msgs WHERE rfc724_mid=?",
(&msg.rfc724_mid,),
)?;
Ok(())
};
if let Err(e) = context.sql.transaction(update_db).await {
@@ -1875,10 +1936,9 @@ pub async fn markseen_msgs(context: &Context, msg_ids: Vec<MsgId>) -> Result<()>
//
// We also don't send read receipts for contact requests.
// Read receipts will not be sent even after accepting the chat.
let wants_mdn = curr_param.get_bool(Param::WantsMdn).unwrap_or_default();
let to_id = if curr_blocked == Blocked::Not
&& !curr_hidden
&& wants_mdn
&& curr_param.get_bool(Param::WantsMdn).unwrap_or_default()
&& curr_param.get_cmd() == SystemMessage::Unknown
&& context.should_send_mdns().await?
{
@@ -1900,11 +1960,6 @@ pub async fn markseen_msgs(context: &Context, msg_ids: Vec<MsgId>) -> Result<()>
None
};
if let Some(to_id) = to_id {
info!(
context,
"Queuing MDN to {to_id} for {id} from {curr_from_id}, wants_mdn={wants_mdn}, cmd={}.",
curr_param.get_cmd()
);
context
.sql
.execute(
+69 -44
View File
@@ -35,7 +35,10 @@ use crate::peer_channels::{create_iroh_header, get_iroh_topic_for_msg};
use crate::pgp::{SeipdVersion, addresses_from_public_key, pubkey_supports_seipdv2};
use crate::simplify::escape_message_footer_marks;
use crate::stock_str;
use crate::tools::{IsNoneOrEmpty, create_outgoing_rfc724_mid, remove_subject_prefix, time};
use crate::tools::{
IsNoneOrEmpty, create_outgoing_rfc724_mid, create_smeared_timestamp, remove_subject_prefix,
time,
};
use crate::webxdc::StatusUpdateSerial;
// attachments of 25 mb brutto should work on the majority of providers
@@ -164,6 +167,7 @@ pub struct MimeFactory {
#[derive(Debug, Clone)]
pub struct RenderedEmail {
pub message: String,
// pub envelope: Envelope,
pub is_encrypted: bool,
pub last_added_location_id: Option<u32>,
@@ -220,10 +224,7 @@ impl MimeFactory {
let mut member_fingerprints = Vec::new();
let mut member_timestamps = Vec::new();
let mut recipient_ids = HashSet::new();
let req_mdn = !chat.is_self_talk()
&& !msg.is_system_message()
&& msg.param.get_int(Param::Reaction).unwrap_or_default() == 0
&& context.should_request_mdns().await?;
let mut req_mdn = false;
let encryption_pubkeys;
@@ -480,6 +481,13 @@ impl MimeFactory {
ContactId::scaleup_origin(context, &recipient_ids, origin).await?;
}
if !msg.is_system_message()
&& msg.param.get_int(Param::Reaction).unwrap_or_default() == 0
&& context.should_request_mdns().await?
{
req_mdn = true;
}
encryption_pubkeys = if !is_encrypted {
None
} else if should_encrypt_symmetrically(&msg, &chat) {
@@ -572,7 +580,7 @@ impl MimeFactory {
) -> Result<MimeFactory> {
let contact = Contact::get_by_id(context, from_id).await?;
let from_addr = context.get_primary_self_addr().await?;
let timestamp = time();
let timestamp = create_smeared_timestamp(context);
let addr = contact.get_addr().to_string();
let encryption_pubkeys = if from_id == ContactId::SELF {
@@ -618,7 +626,9 @@ impl MimeFactory {
fn should_skip_autocrypt(&self) -> bool {
match &self.loaded {
Loaded::Message { .. } => false,
Loaded::Message { msg, .. } => {
msg.param.get_bool(Param::SkipAutocrypt).unwrap_or_default()
}
Loaded::Mdn { .. } => true,
}
}
@@ -1028,12 +1038,16 @@ impl MimeFactory {
is_securejoin_message,
);
let use_std_header_protection = context
.get_config_bool(Config::StdHeaderProtectionComposing)
.await?;
let outer_message = if let Some(encryption_pubkeys) = self.encryption_pubkeys {
let mut message = add_headers_to_encrypted_part(
message,
&unprotected_headers,
hidden_headers,
protected_headers,
use_std_header_protection,
);
// Add gossip headers in chats with multiple recipients
@@ -1230,6 +1244,7 @@ impl MimeFactory {
Ok(RenderedEmail {
message,
// envelope: Envelope::new,
is_encrypted,
last_added_location_id,
sync_ids_to_delete: self.sync_ids_to_delete,
@@ -1702,19 +1717,23 @@ impl MimeFactory {
let footer = if is_reaction { "" } else { &self.selfstatus };
let message_text = format!(
"{}{}{}{}{}{}",
fwdhint.unwrap_or_default(),
quoted_text.unwrap_or_default(),
escape_message_footer_marks(final_text),
if !final_text.is_empty() && !footer.is_empty() {
"\r\n\r\n"
} else {
""
},
if !footer.is_empty() { "-- \r\n" } else { "" },
footer
);
let message_text = if self.pre_message_mode == PreMessageMode::Post {
"".to_string()
} else {
format!(
"{}{}{}{}{}{}",
fwdhint.unwrap_or_default(),
quoted_text.unwrap_or_default(),
escape_message_footer_marks(final_text),
if !final_text.is_empty() && !footer.is_empty() {
"\r\n\r\n"
} else {
""
},
if !footer.is_empty() { "-- \r\n" } else { "" },
footer
)
};
let mut main_part = MimePart::new("text/plain", message_text);
if is_reaction {
@@ -1794,15 +1813,14 @@ impl MimeFactory {
HeaderDef::IrohGossipTopic.get_headername(),
mail_builder::headers::raw::Raw::new(topic).into(),
));
if !matches!(self.pre_message_mode, PreMessageMode::Pre { .. })
&& let (Some(json), _) = context
.render_webxdc_status_update_object(
msg.id,
StatusUpdateSerial::MIN,
StatusUpdateSerial::MAX,
None,
)
.await?
if let (Some(json), _) = context
.render_webxdc_status_update_object(
msg.id,
StatusUpdateSerial::MIN,
StatusUpdateSerial::MAX,
None,
)
.await?
{
parts.push(context.build_status_update_part(&json));
}
@@ -1931,6 +1949,7 @@ fn add_headers_to_encrypted_part(
unprotected_headers: &[(&'static str, HeaderType<'static>)],
hidden_headers: Vec<(&'static str, HeaderType<'static>)>,
protected_headers: Vec<(&'static str, HeaderType<'static>)>,
use_std_header_protection: bool,
) -> MimePart<'static> {
// Store protected headers in the inner message.
let message = protected_headers
@@ -1946,19 +1965,21 @@ fn add_headers_to_encrypted_part(
message.header(header, value)
});
message = unprotected_headers
.iter()
// Structural headers shouldn't be added as "HP-Outer". They are defined in
// <https://www.rfc-editor.org/rfc/rfc9787.html#structural-header-fields>.
.filter(|(name, _)| {
!(name.eq_ignore_ascii_case("mime-version")
|| name.eq_ignore_ascii_case("content-type")
|| name.eq_ignore_ascii_case("content-transfer-encoding")
|| name.eq_ignore_ascii_case("content-disposition"))
})
.fold(message, |message, (name, value)| {
message.header(format!("HP-Outer: {name}"), value.clone())
});
if use_std_header_protection {
message = unprotected_headers
.iter()
// Structural headers shouldn't be added as "HP-Outer". They are defined in
// <https://www.rfc-editor.org/rfc/rfc9787.html#structural-header-fields>.
.filter(|(name, _)| {
!(name.eq_ignore_ascii_case("mime-version")
|| name.eq_ignore_ascii_case("content-type")
|| name.eq_ignore_ascii_case("content-transfer-encoding")
|| name.eq_ignore_ascii_case("content-disposition"))
})
.fold(message, |message, (name, value)| {
message.header(format!("HP-Outer: {name}"), value.clone())
});
}
// Set the appropriate Content-Type for the inner message
for (h, v) in &mut message.headers {
@@ -1967,7 +1988,9 @@ fn add_headers_to_encrypted_part(
{
let mut ct_new = ct.clone();
ct_new = ct_new.attribute("protected-headers", "v1");
ct_new = ct_new.attribute("hp", "cipher");
if use_std_header_protection {
ct_new = ct_new.attribute("hp", "cipher");
}
*ct = ct_new;
break;
}
@@ -2253,7 +2276,7 @@ pub(crate) async fn render_symm_encrypted_securejoin_message(
mail_builder::headers::text::Text::new("Secure-Join".to_string()).into(),
));
let timestamp = time();
let timestamp = create_smeared_timestamp(context);
let date = chrono::DateTime::<chrono::Utc>::from_timestamp(timestamp, 0)
.unwrap()
.to_rfc2822();
@@ -2309,11 +2332,13 @@ pub(crate) async fn render_symm_encrypted_securejoin_message(
);
let outer_message = {
let use_std_header_protection = true;
let message = add_headers_to_encrypted_part(
message,
&unprotected_headers,
hidden_headers,
protected_headers,
use_std_header_protection,
);
// Disable compression for SecureJoin to ensure
+14 -7
View File
@@ -730,14 +730,21 @@ async fn test_hp_outer_headers() -> Result<()> {
let t = &tcm.alice().await;
let chat_id = t.get_self_chat().await.id;
chat::send_text_msg(t, chat_id, "hi!".to_string()).await?;
let sent_msg = t.pop_sent_msg().await;
let msg = MimeMessage::from_bytes(t, sent_msg.payload.as_bytes()).await?;
assert!(msg.header_exists(HeaderDef::HpOuter));
for hdr in ["Date", "From", "Message-ID"] {
assert!(msg.decoded_data_contains(&format!("HP-Outer: {hdr}:")),);
for std_hp_composing in [false, true] {
t.set_config_bool(Config::StdHeaderProtectionComposing, std_hp_composing)
.await?;
chat::send_text_msg(t, chat_id, "hi!".to_string()).await?;
let sent_msg = t.pop_sent_msg().await;
let msg = MimeMessage::from_bytes(t, sent_msg.payload.as_bytes()).await?;
assert_eq!(msg.header_exists(HeaderDef::HpOuter), std_hp_composing);
for hdr in ["Date", "From", "Message-ID"] {
assert_eq!(
msg.decoded_data_contains(&format!("HP-Outer: {hdr}:")),
std_hp_composing,
);
}
assert!(!msg.decoded_data_contains("HP-Outer: Content-Type"));
}
assert!(!msg.decoded_data_contains("HP-Outer: Content-Type"));
Ok(())
}
+6 -12
View File
@@ -31,7 +31,9 @@ use crate::message::{self, Message, MsgId, Viewtype, get_vcard_summary, set_msg_
use crate::param::{Param, Params};
use crate::simplify::{SimplifiedText, simplify};
use crate::sync::SyncItems;
use crate::tools::{get_filemeta, parse_receive_headers, time, truncate_msg_text, validate_id};
use crate::tools::{
get_filemeta, parse_receive_headers, smeared_time, time, truncate_msg_text, validate_id,
};
use crate::{chatlist_events, location, tools};
/// Public key extracted from `Autocrypt-Gossip`
@@ -269,7 +271,7 @@ impl MimeMessage {
pub(crate) async fn from_bytes(context: &Context, body: &[u8]) -> Result<Self> {
let mail = mailparse::parse_mail(body)?;
let timestamp_rcvd = time();
let timestamp_rcvd = smeared_time(context);
let mut timestamp_sent =
Self::get_timestamp_sent(&mail.headers, timestamp_rcvd, timestamp_rcvd);
let hop_info = parse_receive_headers(&mail.get_headers());
@@ -922,16 +924,14 @@ impl MimeMessage {
self.parse_attachments();
// See if an MDN is requested from the other side
let mut wants_mdn = false;
if self.decryption_error.is_none()
&& (!self.parts.is_empty() || matches!(&self.pre_message, PreMessageMode::Pre { .. }))
&& !self.parts.is_empty()
&& let Some(ref dn_to) = self.chat_disposition_notification_to
{
// Check that the message is not outgoing.
let from = &self.from.addr;
if !context.is_self_addr(from).await? {
if from.to_lowercase() == dn_to.addr.to_lowercase() {
wants_mdn = true;
if let Some(part) = self.parts.last_mut() {
part.param.set_int(Param::WantsMdn, 1);
}
@@ -953,9 +953,7 @@ impl MimeMessage {
typ: Viewtype::Text,
..Default::default()
};
if wants_mdn {
part.param.set_int(Param::WantsMdn, 1);
}
if let Some(ref subject) = self.get_subject()
&& !self.has_chat_version()
&& self.webxdc_status_update.is_none()
@@ -2464,9 +2462,6 @@ async fn handle_mdn(
let Some((msg_id, chat_id, has_mdns, is_dup)) = context
.sql
.query_row_optional(
// MDN on a pre-message (message preview) references the post-message. So we can't tell
// if the pre-message or fully downloaded message was seen, but this is on purpose. For
// images this problem is going to be solved by showing thumbnails.
"SELECT
m.id AS msg_id,
c.id AS chat_id,
@@ -2514,7 +2509,6 @@ async fn handle_mdn(
// note(treefit): only matters if it is the last message in chat (but probably too expensive to check, debounce also solves it)
chatlist_events::emit_chatlist_item_changed(context, chat_id);
}
context.emit_event(EventType::MsgReadCountChanged { chat_id, msg_id });
Ok(())
}
+19 -12
View File
@@ -1,5 +1,6 @@
use mailparse::ParsedMail;
use std::mem;
use std::time::Duration;
use super::*;
use crate::{
@@ -1404,17 +1405,23 @@ async fn test_extra_imf_headers() -> Result<()> {
let t = &tcm.alice().await;
let chat_id = t.get_self_chat().await.id;
chat::send_text_msg(t, chat_id, "hi!".to_string()).await?;
let sent_msg = t.pop_sent_msg().await;
// Check that extra headers are ignored with RFC 9788 Header Protection.
let payload = sent_msg.payload.replace(
"Message-ID:",
"Chat-Forty-Two: 42\r\nForty-Two: 42\r\nMessage-ID:",
);
let msg = MimeMessage::from_bytes(t, payload.as_bytes()).await?;
assert!(msg.headers.contains_key("chat-version"));
assert!(!msg.headers.contains_key("chat-forty-two"));
assert!(!msg.headers.contains_key("forty-two"));
for std_hp_composing in [false, true] {
t.set_config_bool(Config::StdHeaderProtectionComposing, std_hp_composing)
.await?;
chat::send_text_msg(t, chat_id, "hi!".to_string()).await?;
let sent_msg = t.pop_sent_msg().await;
// Check removal of some nonexistent "Chat-*" header to protect the code from future
// breakages. But headers not prefixed with "Chat-" remain unless a message has standard
// Header Protection.
let payload = sent_msg.payload.replace(
"Message-ID:",
"Chat-Forty-Two: 42\r\nForty-Two: 42\r\nMessage-ID:",
);
let msg = MimeMessage::from_bytes(t, payload.as_bytes()).await?;
assert!(msg.headers.contains_key("chat-version"));
assert!(!msg.headers.contains_key("chat-forty-two"));
assert_ne!(msg.headers.contains_key("forty-two"), std_hp_composing);
}
Ok(())
}
@@ -1428,7 +1435,7 @@ async fn test_intended_recipient_fingerprint() -> Result<()> {
let chat_id = chat::create_group(t, "").await?;
chat::send_text_msg(t, chat_id, "hi!".to_string()).await?;
assert!(t.pop_sent_msg_opt().await.is_none());
assert!(t.pop_sent_msg_opt(Duration::ZERO).await.is_none());
for (i, member) in members.iter().enumerate() {
let contact = t.add_or_lookup_contact(member).await;
@@ -48,9 +48,10 @@ async fn test_shared_secret_decryption_ex(
let encrypted_msg = pgp::symm_encrypt_message(
plain_text.as_bytes().to_vec(),
signer_key,
secret_for_encryption.to_string(),
secret_for_encryption,
true,
)?;
)
.await?;
let boundary = "boundary123";
let rcvd_mail = format!(
+1 -2
View File
@@ -64,8 +64,7 @@ pub enum Param {
ForcePlaintext = b'u',
/// For Messages: do not include Autocrypt header.
/// Deprecated on 2026-06-20
DeprecatedSkipAutocrypt = b'o',
SkipAutocrypt = b'o',
/// For Messages
WantsMdn = b'r',
+41 -33
View File
@@ -34,7 +34,7 @@ const SYMMETRIC_KEY_ALGORITHM: SymmetricKeyAlgorithm = SymmetricKeyAlgorithm::AE
/// as [described in the Autocrypt standard](https://autocrypt.org/level1.html#openpgp-based-key-data).
pub(crate) fn create_keypair(addr: EmailAddress) -> Result<SignedSecretKey> {
let signing_key_type = PgpKeyType::Ed25519Legacy;
let encryption_key_type = PgpKeyType::ECDH(ECCCurve::Curve25519Legacy);
let encryption_key_type = PgpKeyType::ECDH(ECCCurve::Curve25519);
let user_id = format!("<{addr}>");
let key_params = SecretKeyParamsBuilder::default()
@@ -254,41 +254,44 @@ pub fn pk_validate(
/// Symmetrically encrypt the message.
/// This is used for broadcast channels and for version 2 of the Securejoin protocol.
/// `shared secret` is the secret that will be used for symmetric encryption.
pub fn symm_encrypt_message(
pub async fn symm_encrypt_message(
plain: Vec<u8>,
private_key_for_signing: Option<SignedSecretKey>,
shared_secret: String,
shared_secret: &str,
compress: bool,
) -> Result<String> {
let shared_secret = Password::from(shared_secret);
let shared_secret = Password::from(shared_secret.to_string());
let msg = MessageBuilder::from_bytes("", plain);
let mut rng = thread_rng();
let mut salt = [0u8; 8];
rng.fill(&mut salt[..]);
let s2k = StringToKey::Salted {
hash_alg: HashAlgorithm::default(),
salt,
};
let mut msg = msg.seipd_v2(
&mut rng,
SYMMETRIC_KEY_ALGORITHM,
AeadAlgorithm::Ocb,
ChunkSize::C8KiB,
);
msg.encrypt_with_password(&mut rng, s2k, &shared_secret)?;
tokio::task::spawn_blocking(move || {
let msg = MessageBuilder::from_bytes("", plain);
let mut rng = thread_rng();
let mut salt = [0u8; 8];
rng.fill(&mut salt[..]);
let s2k = StringToKey::Salted {
hash_alg: HashAlgorithm::default(),
salt,
};
let mut msg = msg.seipd_v2(
&mut rng,
SYMMETRIC_KEY_ALGORITHM,
AeadAlgorithm::Ocb,
ChunkSize::C8KiB,
);
msg.encrypt_with_password(&mut rng, s2k, &shared_secret)?;
if let Some(private_key_for_signing) = private_key_for_signing.as_deref() {
let hash_algorithm = private_key_for_signing.hash_alg();
msg.sign(private_key_for_signing, Password::empty(), hash_algorithm);
}
if compress {
msg.compression(CompressionAlgorithm::ZLIB);
}
if let Some(private_key_for_signing) = private_key_for_signing.as_deref() {
let hash_algorithm = private_key_for_signing.hash_alg();
msg.sign(private_key_for_signing, Password::empty(), hash_algorithm);
}
if compress {
msg.compression(CompressionAlgorithm::ZLIB);
}
let encoded_msg = msg.to_armored_string(&mut rng, Default::default())?;
let encoded_msg = msg.to_armored_string(&mut rng, Default::default())?;
Ok(encoded_msg)
Ok(encoded_msg)
})
.await?
}
/// Merges and minimizes OpenPGP certificates.
@@ -371,7 +374,10 @@ pub fn merge_openpgp_certificates(
.into_iter()
.chain(new_direct_signatures)
.filter(|x: &Signature| x.verify_key(&old_primary_key).is_ok())
.max_by_key(|x: &Signature| x.created());
.max_by_key(|x: &Signature|
// Converting to seconds because `Ord` is not derived for `Timestamp`:
// <https://github.com/rpgp/rpgp/issues/737>
x.created().map_or(0, |ts| ts.as_secs()));
let direct_signatures: Vec<Signature> = best_direct_key_signature.into_iter().collect();
// Select at most one User ID.
@@ -393,10 +399,12 @@ pub fn merge_openpgp_certificates(
.verify_certification(&old_primary_key, pgp::types::Tag::UserId, &id)
.is_ok()
})
.max_by_key(|signature: &Signature| signature.created());
.max_by_key(|signature: &Signature| {
signature.created().map_or(0, |ts| ts.as_secs())
});
best_user_signature.map(|signature| (id, signature))
})
.max_by_key(|(_id, signature)| signature.created())
.max_by_key(|(_id, signature)| signature.created().map_or(0, |ts| ts.as_secs()))
.map(|(id, signature)| SignedUser {
id,
signatures: vec![signature],
@@ -702,7 +710,7 @@ mod tests {
// This error message is actually not great,
// but grepping for it will lead to the correct code
test_dont_decrypt_expensive_message_ex(s2k, true, Some("decrypt_the_ring: missing key"))
test_dont_decrypt_expensive_message_ex(s2k, true, Some("decrypt_with_keys: missing key"))
.await
}
@@ -785,7 +793,7 @@ mod tests {
.await
.unwrap_err();
assert_eq!(format!("{error:#}"), "decrypt_the_ring: missing key");
assert_eq!(format!("{error:#}"), "decrypt_with_keys: missing key");
Ok(())
}
-12
View File
@@ -56,9 +56,6 @@ pub enum Qr {
/// Fingerprint of the contact key as scanned from the QR code.
fingerprint: Fingerprint,
/// The inviter's addresses.
addrs: Vec<String>,
/// Invite number.
invitenumber: String,
@@ -83,9 +80,6 @@ pub enum Qr {
/// Fingerprint of the contact key as scanned from the QR code.
fingerprint: Fingerprint,
/// The inviter's addresses.
addrs: Vec<String>,
/// Invite number.
invitenumber: String,
@@ -114,9 +108,6 @@ pub enum Qr {
/// Fingerprint of the contact's key as scanned from the QR code.
fingerprint: Fingerprint,
/// The inviter's addresses.
addrs: Vec<String>,
/// Invite number.
invitenumber: String,
/// Authentication code.
@@ -572,7 +563,6 @@ async fn decode_openpgp(context: &Context, qr: &str) -> Result<Qr> {
grpid,
contact_id,
fingerprint,
addrs: vec![addr.to_string()],
invitenumber,
authcode,
is_v3,
@@ -609,7 +599,6 @@ async fn decode_openpgp(context: &Context, qr: &str) -> Result<Qr> {
grpid,
contact_id,
fingerprint,
addrs: vec![addr.to_string()],
invitenumber,
authcode,
is_v3,
@@ -635,7 +624,6 @@ async fn decode_openpgp(context: &Context, qr: &str) -> Result<Qr> {
Ok(Qr::AskVerifyContact {
contact_id,
fingerprint,
addrs: vec![addr.to_string()],
invitenumber,
authcode,
is_v3,
+32 -47
View File
@@ -533,18 +533,20 @@ pub(crate) async fn receive_imf_inner(
"Receiving message {rfc724_mid_orig:?}, seen={seen}...",
);
let existing_msg_id = message::rfc724_mid_exists(context, rfc724_mid_orig).await?;
if let Some(msg_id) = existing_msg_id
&& !mime_parser.incoming
{
// It sometimes happens that a slow server (usually a classical email server)
// receives a message via SMTP,
// but then the connection to the server dies before it sends the OK response.
// In order to handle this case, we delete the SMTP send jobs if we receive our own message via IMAP.
//
// Now, messages with long recipient lists are split into multiple SMTP jobs.
// In this case, we only want to delete the SMTP job that was sent to self
// because this is the only chunk we can be sure was sent out.
// These checks must be done before processing of SecureJoin and other special messages.
if mime_parser.pre_message == mimeparser::PreMessageMode::Post {
// Post-Message just replaces the attachment and modifies Params, not the whole message.
// This is done in the `handle_post_message` method.
} else if let Some(msg_id) = message::rfc724_mid_exists(context, rfc724_mid_orig).await? {
info!(
context,
"Message {rfc724_mid} is already in some chat or deleted."
);
if mime_parser.incoming {
return Ok(None);
}
// For the case if we missed a successful SMTP response. Be optimistic that the message is
// delivered also.
let self_addr = context.get_primary_self_addr().await?;
context
.sql
@@ -557,16 +559,6 @@ pub(crate) async fn receive_imf_inner(
if !msg_has_pending_smtp_job(context, msg_id).await? {
msg_id.set_delivered(context).await?;
}
}
// These checks must be done before processing of SecureJoin and other special messages.
if mime_parser.pre_message == mimeparser::PreMessageMode::Post {
// Post-Message just replaces the attachment and modifies Params, not the whole message.
// This is done in the `update_from_post_msg` method.
} else if existing_msg_id.is_some() {
info!(
context,
"Message {rfc724_mid} is already in some chat or deleted."
);
return Ok(None);
}
@@ -834,9 +826,7 @@ UPDATE config SET value=? WHERE keyname='configured_addr' AND value!=?1
}
}
if let Some(ref status_update) = mime_parser.webxdc_status_update
&& !matches!(mime_parser.pre_message, PreMessageMode::Pre { .. })
{
if let Some(ref status_update) = mime_parser.webxdc_status_update {
let can_info_msg;
let instance = if mime_parser
.parts
@@ -1225,8 +1215,6 @@ async fn decide_chat_assignment(
// Most mailboxes have a "Drafts" folder where constantly new emails appear but we don't actually want to show them
info!(context, "Email is probably just a draft (TRASH).");
true
} else if matches!(mime_parser.pre_message, PreMessageMode::Pre { .. }) {
false
} else if mime_parser.webxdc_status_update.is_some() && mime_parser.parts.len() == 1 {
if let Some(part) = mime_parser.parts.first() {
if part.typ == Viewtype::Text && part.msg.is_empty() {
@@ -1860,11 +1848,14 @@ async fn add_parts(
let state = if !mime_parser.incoming {
MessageState::OutDelivered
} else if seen || !mime_parser.mdn_reports.is_empty() || chat_id_blocked == Blocked::Yes
} else if seen
|| !mime_parser.mdn_reports.is_empty()
|| chat_id_blocked == Blocked::Yes
|| group_changes.silent
// No check for `hidden` because only reactions are such and they should be `InFresh`.
{
MessageState::InSeen
} else if mime_parser.from.addr == STATISTICS_BOT_EMAIL || group_changes.silent {
} else if mime_parser.from.addr == STATISTICS_BOT_EMAIL {
MessageState::InNoticed
} else {
MessageState::InFresh
@@ -2088,8 +2079,8 @@ async fn add_parts(
}
}
handle_edit_delete(context, mime_parser, from_id, &mime_headers).await?;
update_from_post_msg(context, mime_parser, from_id, state).await?;
handle_edit_delete(context, mime_parser, from_id).await?;
handle_post_message(context, mime_parser, from_id, state).await?;
if mime_parser.is_system_message == SystemMessage::CallAccepted
|| mime_parser.is_system_message == SystemMessage::CallEnded
@@ -2283,7 +2274,8 @@ INSERT INTO msgs
// Maybe set logging xdc and add gossip topics for webxdcs.
for (part, msg_id) in mime_parser.parts.iter().zip(&created_db_entries) {
if part.typ == Viewtype::Webxdc
if mime_parser.pre_message != PreMessageMode::Post
&& part.typ == Viewtype::Webxdc
&& let Some(topic) = mime_parser.get_header(HeaderDef::IrohGossipTopic)
{
let topic = iroh_topic_from_str(topic)?;
@@ -2357,7 +2349,6 @@ async fn handle_edit_delete(
context: &Context,
mime_parser: &MimeMessage,
from_id: ContactId,
mime_headers: &[u8],
) -> Result<()> {
if let Some(rfc724_mid) = mime_parser.get_header(HeaderDef::ChatEdit) {
let Some(original_msg_id) = rfc724_mid_exists(context, rfc724_mid).await? else {
@@ -2391,7 +2382,7 @@ async fn handle_edit_delete(
}
let new_text = part.msg.strip_prefix(EDITED_PREFIX).unwrap_or(&part.msg);
chat::save_text_edit_to_db(context, &mut original_msg, new_text, mime_headers).await?;
chat::save_text_edit_to_db(context, &mut original_msg, new_text).await?;
} else if let Some(rfc724_mid_list) = mime_parser.get_header(HeaderDef::ChatDelete)
&& let Some(part) = mime_parser.parts.first()
{
@@ -2433,7 +2424,7 @@ async fn handle_edit_delete(
Ok(())
}
async fn update_from_post_msg(
async fn handle_post_message(
context: &Context,
mime_parser: &MimeMessage,
from_id: ContactId,
@@ -2451,7 +2442,7 @@ async fn update_from_post_msg(
let Some(msg_id) = message::rfc724_mid_exists(context, &rfc724_mid).await? else {
warn!(
context,
"update_from_post_msg: {rfc724_mid}: Database entry does not exist."
"handle_post_message: {rfc724_mid}: Database entry does not exist."
);
return Ok(());
};
@@ -2459,7 +2450,7 @@ async fn update_from_post_msg(
// else: message is processed like a normal message
warn!(
context,
"update_from_post_msg: {rfc724_mid}: Pre-message was not downloaded yet so treat as normal message."
"handle_post_message: {rfc724_mid}: Pre-message was not downloaded yet so treat as normal message."
);
return Ok(());
};
@@ -2470,7 +2461,7 @@ async fn update_from_post_msg(
// Do nothing if safety checks fail, the worst case is the message modifies the chat if the
// sender is a member.
if from_id != original_msg.from_id {
warn!(context, "update_from_post_msg: {rfc724_mid}: Bad sender.");
warn!(context, "handle_post_message: {rfc724_mid}: Bad sender.");
return Ok(());
}
let post_msg_showpadlock = part
@@ -2478,17 +2469,14 @@ async fn update_from_post_msg(
.get_bool(Param::GuaranteeE2ee)
.unwrap_or_default();
if !post_msg_showpadlock && original_msg.get_showpadlock() {
warn!(
context,
"update_from_post_msg: {rfc724_mid}: Not encrypted."
);
warn!(context, "handle_post_message: {rfc724_mid}: Not encrypted.");
return Ok(());
}
if !part.typ.has_file() {
warn!(
context,
"update_from_post_msg: {rfc724_mid}: First mime part's message-viewtype has no file."
"handle_post_message: {rfc724_mid}: First mime part's message-viewtype has no file."
);
return Ok(());
}
@@ -2519,10 +2507,7 @@ WHERE id=?
part.typ,
part.bytes as isize,
part.error.as_deref().unwrap_or_default(),
match mime_parser.incoming {
true => state,
false => MessageState::Undefined,
},
state,
DownloadState::Done as u32,
original_msg.id,
),
+17 -21
View File
@@ -1663,8 +1663,8 @@ async fn test_save_mime_headers_off() -> anyhow::Result<()> {
let msg = bob.recv_msg(&alice.pop_sent_msg().await).await;
assert_eq!(msg.get_text(), "hi!");
let html = msg.id.get_html(&bob).await?;
assert!(html.is_none());
let mime = message::get_mime_headers(&bob, msg.id).await?;
assert!(mime.is_empty());
Ok(())
}
@@ -2030,12 +2030,12 @@ async fn test_no_smtp_job_for_self_chat() -> Result<()> {
let chat_id = bob.get_self_chat().await.id;
let mut msg = Message::new_text("Happy birthday to me".to_string());
chat::send_msg(bob, chat_id, &mut msg).await?;
assert!(bob.pop_sent_msg_opt().await.is_none());
assert!(bob.pop_sent_msg_opt(Duration::ZERO).await.is_none());
bob.set_config_bool(Config::BccSelf, true).await?;
let mut msg = Message::new_text("Happy birthday to me".to_string());
chat::send_msg(bob, chat_id, &mut msg).await?;
assert!(bob.pop_sent_msg_opt().await.is_some());
assert!(bob.pop_sent_msg_opt(Duration::ZERO).await.is_some());
Ok(())
}
@@ -4059,8 +4059,6 @@ async fn test_delayed_removal_is_ignored() -> Result<()> {
remove_contact_from_chat(bob, bob_chat_id, bob_contact_fiona).await?;
let remove_msg = bob.pop_sent_msg().await;
SystemTime::shift(Duration::from_secs(1));
// Bob adds new members Dom and Elena, but first addition message is lost.
let dom = &tcm.dom().await;
let elena = &tcm.elena().await;
@@ -4077,8 +4075,6 @@ async fn test_delayed_removal_is_ignored() -> Result<()> {
alice.recv_msg(&add_msg).await;
assert_eq!(get_chat_contacts(alice, chat_id).await?.len(), 4);
SystemTime::shift(Duration::from_secs(1));
// Alice re-adds Fiona.
add_contact_to_chat(alice, chat_id, alice_fiona).await?;
assert_eq!(get_chat_contacts(alice, chat_id).await?.len(), 5);
@@ -5696,27 +5692,27 @@ async fn test_mark_message_as_delivered_only_after_sent_out_fully() -> Result<()
.await
.unwrap();
let (post_msg_id, post_msg_payload) = first_row_in_smtp_queue(alice).await;
assert_eq!(msg_id, post_msg_id);
assert!(post_msg_payload.len() > file_bytes.len());
assert_eq!(msg_id.get_state(alice).await?, MessageState::OutPending);
// Alice receives her own post-message because of bcc_self
// This should not yet mark the message as delivered,
// because not everything was sent,
// but it does remove the post-message from the SMTP queue.
receive_imf(alice, post_msg_payload.as_bytes(), false).await?;
assert_eq!(msg_id.get_state(alice).await?, MessageState::OutPending);
let (pre_msg_id, pre_msg_payload) = first_row_in_smtp_queue(alice).await;
assert_eq!(msg_id, pre_msg_id);
assert!(pre_msg_payload.len() < file_bytes.len());
assert_eq!(msg_id.get_state(alice).await?, MessageState::OutPending);
// Alice receives her own pre-message because of bcc_self
// This should not yet mark the message as delivered,
// because not everything was sent,
// but it does remove the pre-message from the SMTP queue
receive_imf(alice, pre_msg_payload.as_bytes(), false).await?;
assert_eq!(msg_id.get_state(alice).await?, MessageState::OutPending);
let (post_msg_id, post_msg_payload) = first_row_in_smtp_queue(alice).await;
assert_eq!(msg_id, post_msg_id);
assert!(post_msg_payload.len() > file_bytes.len());
assert_eq!(msg_id.get_state(alice).await?, MessageState::OutPending);
// Alice receives her own post-message because of bcc_self
// This should now mark the message as delivered,
// because everything was sent by now.
receive_imf(alice, pre_msg_payload.as_bytes(), false).await?;
receive_imf(alice, post_msg_payload.as_bytes(), false).await?;
assert_eq!(msg_id.get_state(alice).await?, MessageState::OutDelivered);
Ok(())
+14 -7
View File
@@ -14,7 +14,7 @@ pub(crate) use self::connectivity::ConnectivityStore;
use crate::config::Config;
use crate::contact::{ContactId, RecentlySeenLoop};
use crate::context::Context;
use crate::download::download_msgs;
use crate::download::{download_known_post_messages_without_pre_message, download_msgs};
use crate::ephemeral;
use crate::events::EventType;
use crate::imap::{Imap, session::Session};
@@ -454,12 +454,10 @@ async fn inbox_fetch_idle(ctx: &Context, imap: &mut Imap, mut session: Session)
.update_metadata(ctx)
.await
.context("update_metadata")?;
if let Err(err) = session.register_token(ctx).await {
warn!(
ctx,
"Transport {transport_id}: Failed to register push token: {err:#}."
);
}
session
.register_token(ctx)
.await
.context("Failed to register push token")?;
let session = fetch_idle(ctx, imap, session).await?;
Ok(session)
@@ -486,10 +484,19 @@ async fn fetch_idle(ctx: &Context, connection: &mut Imap, mut session: Session)
.await
.context("fetch_move_delete")?;
download_known_post_messages_without_pre_message(ctx, &mut session).await?;
download_msgs(ctx, &mut session)
.await
.context("download_msgs")?;
// Synchronize Seen flags.
session
.sync_seen_flags(ctx, &watch_folder)
.await
.context("sync_seen_flags")
.log_err(ctx)
.ok();
connection.connectivity.set_idle(ctx);
ctx.emit_event(EventType::ImapInboxIdle);
+4 -26
View File
@@ -14,9 +14,9 @@ use crate::constants::{
use crate::contact::mark_contact_id_as_verified;
use crate::contact::{Contact, ContactId, Origin};
use crate::context::Context;
use crate::e2ee::ensure_secret_key_exists;
use crate::events::EventType;
use crate::headerdef::HeaderDef;
use crate::key;
use crate::key::{DcKey, Fingerprint, load_self_public_key, self_fingerprint};
use crate::log::LogExt as _;
use crate::log::warn;
@@ -92,7 +92,7 @@ pub async fn get_securejoin_qr(context: &Context, chat: Option<ChatId>) -> Resul
==== Step 1 in "Setup verified contact" protocol ====
=======================================================*/
key::ensure_secret_key_exists(context).await.ok();
ensure_secret_key_exists(context).await.ok();
let chat = match chat {
Some(id) => {
@@ -533,18 +533,6 @@ pub(crate) async fn handle_securejoin_handshake(
warn!(context, "Secure-join denied (bad auth).");
return Ok(HandshakeMessage::Ignore);
}
if Contact::lookup_id_by_addr_ex(
context,
&mime_message.from.addr,
Origin::Unknown,
Some(Blocked::Yes),
)
.await?
.is_some()
{
warn!(context, "Ignoring {step} message: {contact_id} is blocked.");
return Ok(HandshakeMessage::Ignore);
}
let rfc724_mid = create_outgoing_rfc724_mid();
let addr = ContactAddress::new(&mime_message.from.addr)?;
@@ -652,10 +640,6 @@ pub(crate) async fn handle_securejoin_handshake(
if time() < timestamp + VERIFICATION_TIMEOUT_SECONDS {
mark_contact_id_as_verified(context, contact_id, Some(ContactId::SELF)).await?;
}
if sender_contact.blocked {
warn!(context, "Ignoring {step} message: {contact_id} is blocked.");
return Ok(HandshakeMessage::Ignore);
}
contact_id.regossip_keys(context).await?;
// for setup-contact, make Alice's one-to-one chat with Bob visible
// (secure-join-information are shown in the group chat)
@@ -741,7 +725,7 @@ pub(crate) async fn handle_securejoin_handshake(
async fn insert_into_smtp(
context: &Context,
rfc724_mid: &str,
recipients: &str,
recipient: &str,
rendered_message: String,
msg_id: MsgId,
) -> Result<(), Error> {
@@ -750,7 +734,7 @@ async fn insert_into_smtp(
.execute(
"INSERT INTO smtp (rfc724_mid, recipients, mime, msg_id)
VALUES (?1, ?2, ?3, ?4)",
(&rfc724_mid, &recipients, &rendered_message, msg_id),
(&rfc724_mid, &recipient, &rendered_message, msg_id),
)
.await?;
Ok(())
@@ -827,12 +811,6 @@ pub(crate) async fn observe_securejoin_on_other_device(
}
mark_contact_id_as_verified(context, contact_id, Some(ContactId::SELF)).await?;
if contact.blocked && step != SecureJoinStep::MemberAdded {
// Contact might be blocked after another device had issued the message. Still, to avoid
// membership inconsistency on devices, don't ignore "vg-member-added".
warn!(context, "Observing {step}: {contact_id} is blocked.");
return Ok(HandshakeMessage::Ignore);
}
if matches!(
step,
+12 -27
View File
@@ -1,27 +1,25 @@
//! Bob's side of SecureJoin handling, the joiner-side.
use anyhow::{Context as _, Result};
use pgp::composed::SignedPublicKey;
use super::HandshakeMessage;
use super::qrinvite::QrInvite;
use crate::chat::{self, ChatId, is_contact_in_chat};
use crate::constants::{Blocked, Chattype};
use crate::contact::Origin;
use crate::contact::{Contact, Origin};
use crate::context::Context;
use crate::events::EventType;
use crate::key::{DcKey as _, self_fingerprint};
use crate::key::self_fingerprint;
use crate::log::LogExt;
use crate::message::{self, Message, MsgId, Viewtype};
use crate::mimeparser::{MimeMessage, SystemMessage};
use crate::param::{Param, Params};
use crate::pgp::addresses_from_public_key;
use crate::securejoin::{
ContactId, encrypted_and_signed, insert_into_smtp, verify_sender_by_fingerprint,
};
use crate::stock_str;
use crate::sync::Sync::*;
use crate::tools::{create_outgoing_rfc724_mid, time};
use crate::tools::{create_outgoing_rfc724_mid, smeared_time, time};
use crate::{chatlist_events, mimefactory};
/// Starts the securejoin protocol with the QR `invite`.
@@ -60,28 +58,14 @@ pub(super) async fn start_protocol(context: &Context, invite: QrInvite) -> Resul
QrInvite::Broadcast { .. } => {}
}
let public_key_bytes: Option<Vec<u8>> = context
let has_key = context
.sql
.query_get_value(
"SELECT public_key FROM public_keys WHERE fingerprint=?",
.exists(
"SELECT COUNT(*) FROM public_keys WHERE fingerprint=?",
(invite.fingerprint().hex(),),
)
.await?;
let key_contains_all_invite_addrs = if let Some(public_key_bytes) = public_key_bytes {
let public_key = SignedPublicKey::from_slice(&public_key_bytes)?;
if let Some(addrs_in_key) = addresses_from_public_key(&public_key) {
invite.addrs().iter().all(|a| addrs_in_key.contains(a))
} else {
// This can happen if the inviter is using an old version of Delta Chat
// that doesn't put the relay list into the key.
// In this case, we never take the securejoin protocol shortcut, which is fine.
false
}
} else {
false
};
// Now start the protocol and initialise the state.
{
// `joining_chat_id` is `Some` if group chat
@@ -113,7 +97,7 @@ pub(super) async fn start_protocol(context: &Context, invite: QrInvite) -> Resul
progress: JoinerProgress::Succeeded.into_u16(),
});
return Ok(joining_chat_id);
} else if key_contains_all_invite_addrs
} else if has_key
&& verify_sender_by_fingerprint(context, invite.fingerprint(), invite.contact_id())
.await?
{
@@ -170,7 +154,7 @@ pub(super) async fn start_protocol(context: &Context, invite: QrInvite) -> Resul
QrInvite::Contact { .. } => {
// For setup-contact the BobState already ensured the 1:1 chat exists because it is
// used to send the handshake messages.
if !key_contains_all_invite_addrs {
if !has_key {
chat::add_info_msg_with_cmd(
context,
private_chat_id,
@@ -326,7 +310,8 @@ pub(crate) async fn send_handshake_message(
if invite.is_v3() && matches!(step, BobHandshakeMsg::Request) {
// Send a minimal symmetrically-encrypted vc-request-pubkey message
let rfc724_mid = create_outgoing_rfc724_mid();
let recipients = invite.addrs().join(" ");
let contact = Contact::get_by_id(context, invite.contact_id()).await?;
let recipient = contact.get_addr();
let alice_fp = invite.fingerprint().hex();
let auth = invite.authcode();
let shared_secret = format!("securejoin/{alice_fp}/{auth}");
@@ -342,7 +327,7 @@ pub(crate) async fn send_handshake_message(
.await?;
let msg_id = message::insert_tombstone(context, &rfc724_mid).await?;
insert_into_smtp(context, &rfc724_mid, &recipients, rendered_message, msg_id).await?;
insert_into_smtp(context, &rfc724_mid, recipient, rendered_message, msg_id).await?;
context.scheduler.interrupt_smtp().await;
} else {
let mut msg = Message {
@@ -480,7 +465,7 @@ async fn joining_chat_id(
name,
Blocked::Not,
None,
time(),
smeared_time(context),
)
.await?
}
-20
View File
@@ -18,8 +18,6 @@ pub enum QrInvite {
Contact {
contact_id: ContactId,
fingerprint: Fingerprint,
#[serde(default)]
addrs: Vec<String>,
invitenumber: String,
authcode: String,
#[serde(default)]
@@ -28,8 +26,6 @@ pub enum QrInvite {
Group {
contact_id: ContactId,
fingerprint: Fingerprint,
#[serde(default)]
addrs: Vec<String>,
name: String,
grpid: String,
invitenumber: String,
@@ -40,8 +36,6 @@ pub enum QrInvite {
Broadcast {
contact_id: ContactId,
fingerprint: Fingerprint,
#[serde(default)]
addrs: Vec<String>,
name: String,
grpid: String,
invitenumber: String,
@@ -98,14 +92,6 @@ impl QrInvite {
QrInvite::Broadcast { is_v3, .. } => is_v3,
}
}
pub(crate) fn addrs(&self) -> &Vec<String> {
match self {
QrInvite::Contact { addrs, .. } => addrs,
QrInvite::Group { addrs, .. } => addrs,
QrInvite::Broadcast { addrs, .. } => addrs,
}
}
}
impl TryFrom<Qr> for QrInvite {
@@ -116,14 +102,12 @@ impl TryFrom<Qr> for QrInvite {
Qr::AskVerifyContact {
contact_id,
fingerprint,
addrs,
invitenumber,
authcode,
is_v3,
} => Ok(QrInvite::Contact {
contact_id,
fingerprint,
addrs,
invitenumber,
authcode,
is_v3,
@@ -133,14 +117,12 @@ impl TryFrom<Qr> for QrInvite {
grpid,
contact_id,
fingerprint,
addrs,
invitenumber,
authcode,
is_v3,
} => Ok(QrInvite::Group {
contact_id,
fingerprint,
addrs,
name: grpname,
grpid,
invitenumber,
@@ -152,7 +134,6 @@ impl TryFrom<Qr> for QrInvite {
grpid,
contact_id,
fingerprint,
addrs,
authcode,
invitenumber,
is_v3,
@@ -161,7 +142,6 @@ impl TryFrom<Qr> for QrInvite {
grpid,
contact_id,
fingerprint,
addrs,
authcode,
invitenumber,
is_v3,
+2 -2
View File
@@ -950,7 +950,7 @@ async fn test_parallel_setup_contact(bob_deletes_fiona_contact: bool) -> Result<
Contact::delete(bob, bob_fiona_contact_id).await?;
bob.recv_msg_trash(&sent_fiona_vc_auth_required).await;
let sent = bob.pop_sent_msg_opt().await;
let sent = bob.pop_sent_msg_opt(Duration::ZERO).await;
assert!(sent.is_none());
} else {
bob.recv_msg_trash(&sent_fiona_vc_auth_required).await;
@@ -1235,7 +1235,7 @@ async fn test_rejoin_group() -> Result<()> {
assert_eq!(progress, 1000);
// Bob does not send any more messages by scanning the QR code.
assert!(bob.pop_sent_msg_opt().await.is_none());
assert!(bob.pop_sent_msg_opt(Duration::ZERO).await.is_none());
Ok(())
}
+3 -3
View File
@@ -379,7 +379,7 @@ pub(crate) async fn send_msg_to_smtp(
if retries > 6 {
context
.sql
.execute("DELETE FROM smtp WHERE msg_id=?", (msg_id,))
.execute("DELETE FROM smtp WHERE id=?", (rowid,))
.await
.context("Failed to remove message with exceeded retry limit from smtp table")?;
if let Some(mut msg) = Message::load_from_db_optional(context, msg_id).await? {
@@ -457,7 +457,7 @@ pub(crate) async fn send_msg_to_smtp(
}
context
.sql
.execute("DELETE FROM smtp WHERE msg_id=?", (msg_id,))
.execute("DELETE FROM smtp WHERE id=?", (rowid,))
.await?;
}
};
@@ -599,7 +599,7 @@ async fn send_mdn_rfc724_mid(
.ok()
})
.collect();
message::insert_tombstone(context, &rendered_msg.rfc724_mid).await?;
match smtp_send(context, &recipients, &body, smtp, None).await {
SendResult::Success => {
if !recipients.is_empty() {
+16 -13
View File
@@ -40,28 +40,31 @@ impl Smtp {
}
let message_len_bytes = message.len();
let recipients_display = recipients
.iter()
.map(|x| x.as_ref())
.collect::<Vec<&str>>()
.join(",");
let envelope =
Envelope::new(self.from.clone(), recipients.to_vec()).map_err(Error::Envelope)?;
let mail = SendableEmail::new(envelope, message);
let Some(ref mut transport) = self.transport else {
if let Some(ref mut transport) = self.transport {
transport.send(mail).await.map_err(Error::SmtpSend)?;
let info_msg =
format!("Message len={message_len_bytes} was SMTP-sent to {recipients_display}");
info!(context, "{info_msg}.");
context.emit_event(EventType::SmtpMessageSent(info_msg));
self.last_success = Some(tools::Time::now());
} else {
warn!(
context,
"Failed to send a message because SMTP client has no SmtpTransport."
"uh? SMTP has no transport, failed to send to {}", recipients_display
);
return Err(Error::NoTransport);
};
transport.send(mail).await.map_err(Error::SmtpSend)?;
let info_msg = format!(
"Message len={message_len_bytes} was SMTP-sent to {} recipients.",
recipients.len()
);
info!(context, "{info_msg}.");
context.emit_event(EventType::SmtpMessageSent(info_msg));
self.last_success = Some(tools::Time::now());
}
Ok(())
}
}
-22
View File
@@ -2434,28 +2434,6 @@ UPDATE msgs SET state=24 WHERE state=18; -- Change OutPreparing to OutFailed.
.await?;
}
inc_and_check(&mut migration_version, 154)?;
if dbversion < migration_version {
// Recreate imap_markseen with PRIMARY KEY and NOT NULL constraints.
// PRIMARY KEY is needed to turn
// "DELETE FROM imap_markseen_new WHERE id = ?"
// query from SCAN into SEARCH.
sql.execute_migration(
"
CREATE TABLE new_imap_markseen (
id INTEGER PRIMARY KEY NOT NULL,
FOREIGN KEY(id) REFERENCES imap(id) ON DELETE CASCADE
);
INSERT OR IGNORE INTO new_imap_markseen (id)
SELECT id FROM imap_markseen;
DROP TABLE imap_markseen;
ALTER TABLE new_imap_markseen RENAME TO imap_markseen;
",
migration_version,
)
.await?;
}
let new_version = sql
.get_raw_config_int(VERSION_CFG)
.await?
+41 -42
View File
@@ -8,7 +8,7 @@ use std::ops::{Deref, DerefMut};
use std::panic;
use std::path::Path;
use std::sync::{Arc, LazyLock};
use std::time::Duration;
use std::time::{Duration, Instant};
use anyhow::Result;
use async_channel::{self as channel, Receiver, Sender};
@@ -232,7 +232,9 @@ impl TestContextManager {
test_context.set_primary_self_addr(new_addr).await.unwrap();
// ensure_secret_key_exists() is called during configure
key::ensure_secret_key_exists(test_context).await.unwrap();
crate::e2ee::ensure_secret_key_exists(test_context)
.await
.unwrap();
assert_eq!(
test_context.get_primary_self_addr().await.unwrap(),
@@ -270,7 +272,6 @@ impl TestContextManager {
/// Executes SecureJoin initiated by `joiner`
/// scanning `qr` generated by one of the `inviters` devices.
/// `inviters` devices must have the same primary address.
/// All of the `inviters` devices will get the messages and send replies.
///
/// The [`ChatId`] of the created chat is returned, for a SetupContact QR this is the 1:1
@@ -281,11 +282,9 @@ impl TestContextManager {
inviters: &[&TestContext],
qr: &str,
) -> ChatId {
assert!(joiner.pop_sent_msg_opt().await.is_none());
let inviter_addr = inviters[0].get_primary_self_addr().await.unwrap();
assert!(joiner.pop_sent_msg_opt(Duration::ZERO).await.is_none());
for inviter in inviters {
assert!(inviter.pop_sent_msg_opt().await.is_none());
assert_eq!(inviter.get_primary_self_addr().await.unwrap(), inviter_addr);
assert!(inviter.pop_sent_msg_opt(Duration::ZERO).await.is_none());
}
let chat_id = join_securejoin(&joiner.ctx, qr).await.unwrap();
@@ -293,22 +292,14 @@ impl TestContextManager {
for _ in 0..2 {
let mut something_sent = false;
let rev_order = false;
if let Some(sent) = joiner.pop_sent_msg_ex(rev_order).await {
if let Some(sent) = joiner.pop_sent_msg_ex(rev_order, Duration::ZERO).await {
for inviter in inviters {
inviter.recv_msg_opt(&sent).await;
}
something_sent = true;
}
for inviter in inviters {
if let Some(sent) = inviter.pop_sent_msg_ex(rev_order).await {
if sent.recipients.split(' ').any(|addr| addr == inviter_addr) {
for observer in inviters {
// `imap::prefetch_should_download()` returns false on the sender side.
if observer.get_id() != inviter.get_id() {
observer.recv_msg_opt(&sent).await;
}
}
}
if let Some(sent) = inviter.pop_sent_msg_ex(rev_order, Duration::ZERO).await {
joiner.recv_msg_opt(&sent).await;
something_sent = true;
}
@@ -647,17 +638,22 @@ impl TestContext {
///
/// Panics if there is no message or on any error.
pub async fn pop_sent_msg(&self) -> SentMessage<'_> {
self.pop_sent_msg_opt()
self.pop_sent_msg_opt(Duration::from_secs(3))
.await
.expect("no sent message found in jobs table")
}
pub async fn pop_sent_msg_opt(&self) -> Option<SentMessage<'_>> {
pub async fn pop_sent_msg_opt(&self, timeout: Duration) -> Option<SentMessage<'_>> {
let rev_order = true;
self.pop_sent_msg_ex(rev_order).await
self.pop_sent_msg_ex(rev_order, timeout).await
}
pub async fn pop_sent_msg_ex(&self, rev_order: bool) -> Option<SentMessage<'_>> {
pub async fn pop_sent_msg_ex(
&self,
rev_order: bool,
timeout: Duration,
) -> Option<SentMessage<'_>> {
let start = Instant::now();
let mut query = "
SELECT id, msg_id, mime, recipients
FROM smtp
@@ -666,18 +662,28 @@ ORDER BY id"
if rev_order {
query += " DESC";
}
let (rowid, msg_id, payload, recipients) = self
.ctx
.sql
.query_row_optional(&query, (), |row| {
let rowid: i64 = row.get(0)?;
let msg_id: MsgId = row.get(1)?;
let mime: String = row.get(2)?;
let recipients: String = row.get(3)?;
Ok((rowid, msg_id, mime, recipients))
})
.await
.expect("query_row_optional failed")?;
let (rowid, msg_id, payload, recipients) = loop {
let row = self
.ctx
.sql
.query_row_optional(&query, (), |row| {
let rowid: i64 = row.get(0)?;
let msg_id: MsgId = row.get(1)?;
let mime: String = row.get(2)?;
let recipients: String = row.get(3)?;
Ok((rowid, msg_id, mime, recipients))
})
.await
.expect("query_row_optional failed");
if let Some(row) = row {
break row;
}
if start.elapsed() < timeout {
tokio::time::sleep(Duration::from_millis(100)).await;
} else {
return None;
}
};
self.ctx
.sql
.execute("DELETE FROM smtp WHERE id=?;", (rowid,))
@@ -737,14 +743,12 @@ ORDER BY id"
})
}
/// Returns `SentMessage` instances representing `smtp` rows for the given message. Returned
/// items go in reverse order for historical reasons.
pub async fn get_smtp_rows_for_msg<'a>(&'a self, msg_id: MsgId) -> Vec<SentMessage<'a>> {
let sent_msgs = self
.ctx
.sql
.query_map_vec(
"SELECT id, msg_id, mime, recipients FROM smtp WHERE msg_id=? ORDER BY id DESC",
"SELECT id, msg_id, mime, recipients FROM smtp WHERE msg_id=?",
(msg_id,),
|row| {
let _id: MsgId = row.get(0)?;
@@ -1072,14 +1076,9 @@ ORDER BY id"
/// This is not hooked up to any SMTP-IMAP pipeline, so the other account must call
/// [`TestContext::recv_msg`] with the returned [`SentMessage`] if it wants to receive
/// the message.
///
/// Removes SMTP jobs existed before and marks the corresponding messages as delivered, as
/// tracking of these jobs is probably already lost by the test code.
pub async fn send_msg(&self, chat_id: ChatId, msg: &mut Message) -> SentMessage<'_> {
while self.pop_sent_msg_opt().await.is_some() {}
let msg_id = chat::send_msg(self, chat_id, msg).await.unwrap();
let rev_order = false;
let res = self.pop_sent_msg_ex(rev_order).await.unwrap();
let res = self.pop_sent_msg().await;
assert_eq!(
res.sender_msg_id, msg_id,
"Apparently the message was not actually sent out"
+1 -1
View File
@@ -34,7 +34,7 @@ async fn test_additional_text_on_different_viewtypes() -> Result<()> {
let (pre_message, _, _) = send_large_image_message(alice, a_group_id).await?;
let msg = bob.recv_msg(&pre_message).await;
assert_eq!(msg.text, "test".to_owned());
assert_eq!(msg.get_text(), "test [Image – 228.45 KiB]".to_owned());
assert_eq!(msg.get_text(), "test [Image – 146.12 KiB]".to_owned());
Ok(())
}
+8 -2
View File
@@ -1,4 +1,5 @@
//! Tests about forwarding and saving Pre-Messages
use std::time::Duration;
use anyhow::Result;
use pretty_assertions::assert_eq;
@@ -105,9 +106,14 @@ async fn test_receive_both() -> Result<()> {
assert_eq!(msg.text, "test".to_owned());
forward_msgs(alice, &[alice_msg_id], alice_chat_id).await?;
let rev_order = true;
let rev_order = false;
let msg = bob
.recv_msg(&alice.pop_sent_msg_ex(rev_order).await.unwrap())
.recv_msg(
&alice
.pop_sent_msg_ex(rev_order, Duration::ZERO)
.await
.unwrap(),
)
.await;
assert_eq!(msg.download_state(), DownloadState::Available);
assert_eq!(msg.is_forwarded(), true);
+8 -149
View File
@@ -257,81 +257,7 @@ async fn test_lost_pre_msg() -> Result<()> {
let _pre_msg = alice.pop_sent_msg().await;
let msg = bob.recv_msg(&full_msg).await;
assert_eq!(msg.download_state, DownloadState::Done);
assert_eq!(msg.text, "populate");
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_post_msg_mdn_before_sending_pre_msg() -> Result<()> {
post_msg_mdn_before_sending_pre_msg("").await
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_post_msg_mdn_before_sending_pre_msg_with_text() -> Result<()> {
post_msg_mdn_before_sending_pre_msg("text").await
}
async fn post_msg_mdn_before_sending_pre_msg(text: &str) -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
let bob = &tcm.bob().await;
let alice_chat_id = alice.create_group_with_members("", &[bob]).await;
let file_bytes = include_bytes!("../../../test-data/image/screenshot.gif");
let mut msg = Message::new(Viewtype::Image);
msg.set_file_from_bytes(alice, "a.jpg", file_bytes, None)?;
msg.set_text(text.to_string());
chat::send_msg(alice, alice_chat_id, &mut msg).await?;
let rev_order = false;
let post_msg = alice.pop_sent_msg_ex(rev_order).await.unwrap();
let alice_msg_id = msg.id;
let msg = bob.recv_msg(&post_msg).await;
assert_eq!(msg.download_state, DownloadState::Done);
assert_eq!(msg.id.get_state(bob).await?, MessageState::InFresh);
assert_eq!(msg.text, text);
assert!(msg.param.get_bool(Param::WantsMdn).unwrap_or_default());
msg.chat_id.accept(bob).await?;
markseen_msgs(bob, vec![msg.id]).await?;
assert_eq!(msg.id.get_state(bob).await?, MessageState::InSeen);
assert_eq!(
bob.sql
.count(
"SELECT COUNT(*) FROM smtp_mdns WHERE from_id=?",
(msg.from_id,)
)
.await?,
1
);
assert_eq!(
alice_msg_id.get_state(alice).await?,
MessageState::OutPending
);
// Bob's other device receives the pre-message first.
let bob = &tcm.bob().await;
let pre_msg = alice.pop_sent_msg().await;
assert_eq!(
alice_msg_id.get_state(alice).await?,
MessageState::OutDelivered
);
let msg = bob.recv_msg(&pre_msg).await;
assert_eq!(msg.download_state, DownloadState::Available);
assert_eq!(msg.id.get_state(bob).await?, MessageState::InFresh);
assert_eq!(msg.text, text);
assert!(msg.param.get_bool(Param::WantsMdn).unwrap_or_default());
msg.chat_id.accept(bob).await?;
markseen_msgs(bob, vec![msg.id]).await?;
assert_eq!(msg.id.get_state(bob).await?, MessageState::InSeen);
assert_eq!(
bob.sql
.count(
"SELECT COUNT(*) FROM smtp_mdns WHERE from_id=?",
(msg.from_id,)
)
.await?,
1
);
assert_eq!(msg.text, "");
Ok(())
}
@@ -476,9 +402,9 @@ async fn test_receive_pre_message_image() -> Result<()> {
// test that metadata is correctly returned by methods
assert_eq!(msg.get_post_message_viewtype(), Some(Viewtype::Image));
// recoded image dimensions
assert_eq!(msg.get_filebytes(bob).await?, Some(233935));
assert_eq!(msg.get_height(), 1704);
assert_eq!(msg.get_width(), 959);
assert_eq!(msg.get_filebytes(bob).await?, Some(149632));
assert_eq!(msg.get_height(), 1280);
assert_eq!(msg.get_width(), 720);
Ok(())
}
@@ -608,17 +534,6 @@ async fn test_webxdc_updates_in_post_message_after_pre_message() -> Result<()> {
let alice_chat_id = alice.create_chat_id(bob).await;
// regression test where updates get assigned to an unrelated prior webxdc message
let mut unrelated_xdc = Message::new(Viewtype::Webxdc);
unrelated_xdc.set_file_from_bytes(
alice,
"first.xdc",
include_bytes!("../../../test-data/webxdc/minimal.xdc"),
None,
)?;
send_msg(alice, alice_chat_id, &mut unrelated_xdc).await?;
let bob_unrelated_webxdc = bob.recv_msg(&alice.pop_sent_msg().await).await;
let big_webxdc_app = big_webxdc_app().await?;
let mut alice_instance = Message::new(Viewtype::Webxdc);
@@ -632,19 +547,11 @@ async fn test_webxdc_updates_in_post_message_after_pre_message() -> Result<()> {
.await?;
send_msg(alice, alice_chat_id, &mut alice_instance).await?;
let pre_message = alice.pop_sent_msg().await;
let post_message = alice.pop_sent_msg().await;
let pre_message = alice.pop_sent_msg().await;
let bob_instance = bob.recv_msg(&pre_message).await;
assert_eq!(bob_instance.download_state, DownloadState::Available);
// don't accidentally assign updates from a pre-message to parent message
assert_eq!(
bob.get_webxdc_status_updates(bob_unrelated_webxdc.id, StatusUpdateSerial::new(0))
.await?,
"[]"
);
bob.recv_msg_trash(&post_message).await;
let bob_instance = Message::load_from_db(bob, bob_instance.id).await?;
assert_eq!(bob_instance.download_state, DownloadState::Done);
@@ -658,51 +565,6 @@ async fn test_webxdc_updates_in_post_message_after_pre_message() -> Result<()> {
Ok(())
}
/// Tests sending large webxdc without text.
///
/// This is a regression test, previously pre-message
/// was trashed when it had no text.
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_large_webxdc_without_text() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
let bob = &tcm.bob().await;
tcm.section("Bob sends large webxdc without attached text message.");
let bob_chat_id = bob.create_chat_id(alice).await;
let big_webxdc_app = big_webxdc_app().await?;
let mut bob_instance = Message::new(Viewtype::Webxdc);
bob_instance.set_file_from_bytes(bob, "test.xdc", &big_webxdc_app, None)?;
bob_chat_id.set_draft(bob, Some(&mut bob_instance)).await?;
bob.send_webxdc_status_update(bob_instance.id, r#"{"payload":42, "info":"i"}"#)
.await?;
send_msg(bob, bob_chat_id, &mut bob_instance).await?;
let pre_message = bob.pop_sent_msg().await;
let post_message = bob.pop_sent_msg().await;
tcm.section("Alice receives a pre-message");
let alice_instance = alice.recv_msg(&pre_message).await;
assert_eq!(alice_instance.download_state, DownloadState::Available);
tcm.section("Alice receives a post-message");
alice.recv_msg_trash(&post_message).await;
let alice_instance = Message::load_from_db(alice, alice_instance.id).await?;
assert_eq!(alice_instance.download_state, DownloadState::Done);
let alice_file_path = alice_instance.get_file(alice).expect("No file");
tokio::fs::try_exists(alice_file_path).await?;
assert_eq!(
alice
.get_webxdc_status_updates(alice_instance.id, StatusUpdateSerial::new(0))
.await?,
r#"[{"payload":42,"info":"i","serial":1,"max_serial":1}]"#
);
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_webxdc_updates_in_post_message_after_deleted_pre_message() -> Result<()> {
let mut tcm = TestContextManager::new();
@@ -723,8 +585,8 @@ async fn test_webxdc_updates_in_post_message_after_deleted_pre_message() -> Resu
.await?;
send_msg(alice, alice_chat_id, &mut alice_instance).await?;
let pre_message = alice.pop_sent_msg().await;
let post_message = alice.pop_sent_msg().await;
let pre_message = alice.pop_sent_msg().await;
let bob_instance = bob.recv_msg(&pre_message).await;
assert_eq!(bob_instance.download_state, DownloadState::Available);
@@ -765,8 +627,8 @@ async fn test_webxdc_updates_in_post_message_without_pre_message() -> Result<()>
.await?;
send_msg(alice, alice_chat_id, &mut alice_instance).await?;
let pre_message = alice.pop_sent_msg().await;
let post_message = alice.pop_sent_msg().await;
let pre_message = alice.pop_sent_msg().await;
// Bob receives post-message first.
let bob_instance = bob.recv_msg(&post_message).await;
@@ -817,10 +679,7 @@ async fn test_markseen_pre_msg() -> Result<()> {
assert_eq!(
alice
.sql
.count(
"SELECT COUNT(*) FROM smtp_mdns WHERE from_id=?",
(msg.from_id,)
)
.count("SELECT COUNT(*) FROM smtp_mdns", ())
.await?,
1
);
+3 -4
View File
@@ -6,9 +6,7 @@ use crate::chat::{self, Chat, add_contact_to_chat, remove_contact_from_chat, sen
use crate::config::Config;
use crate::constants::Chattype;
use crate::contact::{Contact, ContactId};
use crate::key;
use crate::key::self_fingerprint;
use crate::message;
use crate::message::{Message, Viewtype};
use crate::mimefactory::MimeFactory;
use crate::mimeparser::SystemMessage;
@@ -20,6 +18,7 @@ use crate::test_utils::{
E2EE_INFO_MSGS, TestContext, TestContextManager, get_chat_msg, mark_as_verified,
};
use crate::tools::SystemTime;
use crate::{e2ee, message};
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_verified_oneonone_chat_not_broken_by_classical() {
@@ -127,7 +126,7 @@ async fn test_create_verified_oneonone_chat() -> Result<()> {
let fiona_new = tcm.unconfigured().await;
fiona_new.configure_addr("fiona@example.net").await;
key::ensure_secret_key_exists(&fiona_new).await?;
e2ee::ensure_secret_key_exists(&fiona_new).await?;
tcm.send_recv(&fiona_new, &alice, "I have a new device")
.await;
@@ -429,7 +428,7 @@ async fn test_verify_then_verify_again() -> Result<()> {
drop(bob);
let bob_new = tcm.unconfigured().await;
bob_new.configure_addr("bob@example.net").await;
key::ensure_secret_key_exists(&bob_new).await?;
e2ee::ensure_secret_key_exists(&bob_new).await?;
tcm.execute_securejoin(&bob_new, &alice).await;
assert_verified(&alice, &bob_new).await;
+194
View File
@@ -0,0 +1,194 @@
//! # Time smearing.
//!
//! As e-mails typically only use a second-based-resolution for timestamps,
//! the order of two mails sent within one second is unclear.
//! This is bad e.g. when forwarding some messages from a chat -
//! these messages will appear at the recipient easily out of order.
//!
//! We work around this issue by not sending out two mails with the same timestamp.
//! For this purpose, in short, we track the last timestamp used in `last_smeared_timestamp`
//! when another timestamp is needed in the same second, we use `last_smeared_timestamp+1`
//! after some moments without messages sent out,
//! `last_smeared_timestamp` is again in sync with the normal time.
//!
//! However, we do not do all this for the far future,
//! but at max `MAX_SECONDS_TO_LEND_FROM_FUTURE`
use std::cmp::{max, min};
use std::sync::atomic::{AtomicI64, Ordering};
pub(crate) const MAX_SECONDS_TO_LEND_FROM_FUTURE: i64 = 30;
/// Smeared timestamp generator.
#[derive(Debug)]
pub struct SmearedTimestamp {
/// Next timestamp available for allocation.
smeared_timestamp: AtomicI64,
}
impl SmearedTimestamp {
/// Creates a new smeared timestamp generator.
pub fn new() -> Self {
Self {
smeared_timestamp: AtomicI64::new(0),
}
}
/// Allocates `count` unique timestamps.
///
/// Returns the first allocated timestamp.
#[expect(clippy::arithmetic_side_effects)]
pub fn create_n(&self, now: i64, count: i64) -> i64 {
let mut prev = self.smeared_timestamp.load(Ordering::Relaxed);
loop {
// Advance the timestamp if it is in the past,
// but keep `count - 1` timestamps from the past if possible.
let t = max(prev, now - count + 1);
// Rewind the time back if there is no room
// to allocate `count` timestamps without going too far into the future.
// Not going too far into the future
// is more important than generating unique timestamps.
let first = min(t, now + MAX_SECONDS_TO_LEND_FROM_FUTURE - count + 1);
// Allocate `count` timestamps by advancing the current timestamp.
let next = first + count;
if let Err(x) = self.smeared_timestamp.compare_exchange_weak(
prev,
next,
Ordering::Relaxed,
Ordering::Relaxed,
) {
prev = x;
} else {
return first;
}
}
}
/// Creates a single timestamp.
pub fn create(&self, now: i64) -> i64 {
self.create_n(now, 1)
}
/// Returns the current smeared timestamp.
pub fn current(&self) -> i64 {
self.smeared_timestamp.load(Ordering::Relaxed)
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::test_utils::TestContext;
use crate::tools::{
SystemTime, create_smeared_timestamp, create_smeared_timestamps, smeared_time, time,
};
#[test]
fn test_smeared_timestamp() {
let smeared_timestamp = SmearedTimestamp::new();
let now = time();
assert_eq!(smeared_timestamp.current(), 0);
for i in 0..MAX_SECONDS_TO_LEND_FROM_FUTURE {
assert_eq!(smeared_timestamp.create(now), now + i);
}
assert_eq!(
smeared_timestamp.create(now),
now + MAX_SECONDS_TO_LEND_FROM_FUTURE
);
assert_eq!(
smeared_timestamp.create(now),
now + MAX_SECONDS_TO_LEND_FROM_FUTURE
);
// System time rewinds back by 1000 seconds.
let now = now - 1000;
assert_eq!(
smeared_timestamp.create(now),
now + MAX_SECONDS_TO_LEND_FROM_FUTURE
);
assert_eq!(
smeared_timestamp.create(now),
now + MAX_SECONDS_TO_LEND_FROM_FUTURE
);
assert_eq!(
smeared_timestamp.create(now + 1),
now + MAX_SECONDS_TO_LEND_FROM_FUTURE + 1
);
assert_eq!(smeared_timestamp.create(now + 100), now + 100);
assert_eq!(smeared_timestamp.create(now + 100), now + 101);
assert_eq!(smeared_timestamp.create(now + 100), now + 102);
}
#[test]
fn test_create_n_smeared_timestamps() {
let smeared_timestamp = SmearedTimestamp::new();
let now = time();
// Create a single timestamp to initialize the generator.
assert_eq!(smeared_timestamp.create(now), now);
// Wait a minute.
let now = now + 60;
// Simulate forwarding 7 messages.
let forwarded_messages = 7;
// We have not sent anything for a minute,
// so we can take the current timestamp and take 6 timestamps from the past.
assert_eq!(smeared_timestamp.create_n(now, forwarded_messages), now - 6);
assert_eq!(smeared_timestamp.current(), now + 1);
// Wait 4 seconds.
// Now we have 3 free timestamps in the past.
let now = now + 4;
assert_eq!(smeared_timestamp.current(), now - 3);
// Forward another 7 messages.
// We can only lend 3 timestamps from the past.
assert_eq!(smeared_timestamp.create_n(now, forwarded_messages), now - 3);
// We had to borrow 3 timestamps from the future
// because there were not enough timestamps in the past.
assert_eq!(smeared_timestamp.current(), now + 4);
// Forward another 32 messages.
// We cannot use more than 30 timestamps from the future,
// so we use 30 timestamps from the future,
// the current timestamp and one timestamp from the past.
assert_eq!(smeared_timestamp.create_n(now, 32), now - 1);
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_create_smeared_timestamp() {
let t = TestContext::new().await;
assert_ne!(create_smeared_timestamp(&t), create_smeared_timestamp(&t));
assert!(
create_smeared_timestamp(&t)
>= SystemTime::now()
.duration_since(SystemTime::UNIX_EPOCH)
.unwrap()
.as_secs() as i64
);
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_create_smeared_timestamps() {
let t = TestContext::new().await;
let count = MAX_SECONDS_TO_LEND_FROM_FUTURE - 1;
let start = create_smeared_timestamps(&t, count as usize);
let next = smeared_time(&t);
assert!((start + count - 1) < next);
let count = MAX_SECONDS_TO_LEND_FROM_FUTURE + 30;
let start = create_smeared_timestamps(&t, count as usize);
let next = smeared_time(&t);
assert!((start + count - 1) < next);
}
}
+23
View File
@@ -180,6 +180,29 @@ pub(crate) fn gm2local_offset() -> i64 {
i64::from(lt.offset().local_minus_utc())
}
/// Returns the current smeared timestamp,
///
/// The returned timestamp MAY NOT be unique and MUST NOT go to "Date" header.
pub(crate) fn smeared_time(context: &Context) -> i64 {
let now = time();
let ts = context.smeared_timestamp.current();
std::cmp::max(ts, now)
}
/// Returns a timestamp that is guaranteed to be unique.
pub(crate) fn create_smeared_timestamp(context: &Context) -> i64 {
let now = time();
context.smeared_timestamp.create(now)
}
// creates `count` timestamps that are guaranteed to be unique.
// the first created timestamps is returned directly,
// get the other timestamps just by adding 1..count-1
pub(crate) fn create_smeared_timestamps(context: &Context, count: usize) -> i64 {
let now = time();
context.smeared_timestamp.create_n(now, count as i64)
}
/// Returns the last release timestamp as a unix timestamp compatible for comparison with time() and
/// database times.
pub fn get_release_timestamp() -> i64 {
+2 -2
View File
@@ -46,7 +46,7 @@ use crate::mimefactory::RECOMMENDED_FILE_SIZE;
use crate::mimeparser::SystemMessage;
use crate::param::Param;
use crate::param::Params;
use crate::tools::{create_id, get_abs_path, time};
use crate::tools::{create_id, create_smeared_timestamp, get_abs_path};
/// The current API version.
/// If `min_api` in manifest.toml is set to a larger value,
@@ -558,7 +558,7 @@ impl Context {
.create_status_update_record(
&instance,
status_update,
time(),
create_smeared_timestamp(self),
send_now,
ContactId::SELF,
)
+2 -1
View File
@@ -145,6 +145,7 @@ mod tests {
use crate::message::{Message, Viewtype};
use crate::test_utils::TestContext;
use anyhow::Result;
use std::time::Duration;
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_default_integrations_are_single_device() -> Result<()> {
@@ -157,7 +158,7 @@ mod tests {
t.set_webxdc_integration(file.to_str().unwrap()).await?;
// default integrations are shipped with the apps and should not be sent over the wire
let sent = t.pop_sent_msg_opt().await;
let sent = t.pop_sent_msg_opt(Duration::from_secs(1)).await;
assert!(sent.is_none());
Ok(())
+1
View File
@@ -13,6 +13,7 @@ Filename encoding | Encoded Words ([RFC 2047][]), Encoded Word Ex
Identify server folders | IMAP LIST Extension ([RFC 6154][])
Push | IMAP IDLE ([RFC 2177][])
Quota | IMAP QUOTA extension ([RFC 2087][])
Seen status synchronization | IMAP CONDSTORE extension ([RFC 7162][])
Client/server identification | IMAP ID extension ([RFC 2971][])
Authorization | OAuth2 ([RFC 6749][])
End-to-end encryption | [Autocrypt Level 1][], OpenPGP ([RFC 4880][]), Security Multiparts for MIME ([RFC 1847][]) and [“Mixed Up” Encryption repairing](https://datatracker.ietf.org/doc/html/draft-dkg-openpgp-pgpmime-message-mangling-00)
+2 -2
View File
@@ -1,7 +1,7 @@
OutBroadcast#Chat#1001: Channel [0 member(s)]
--------------------------------------------------------------------------------
Msg#1001: info (Contact#Contact#Info): Messages are end-to-end encrypted. [NOTICED][INFO]
Msg#1008🔒: Me (Contact#Contact#Self): hi √
Msg#1006🔒: Me (Contact#Contact#Self): Member bob@example.net added. [INFO] √
Msg#1009🔒: Me (Contact#Contact#Self): hi √
Msg#1010🔒: Me (Contact#Contact#Self): You removed member bob@example.net. [INFO] √
Msg#1009🔒: Me (Contact#Contact#Self): You removed member bob@example.net. [INFO] √
--------------------------------------------------------------------------------
+2 -2
View File
@@ -2,6 +2,6 @@ OutBroadcast#Chat#1001: Channel [0 member(s)]
--------------------------------------------------------------------------------
Msg#1002: info (Contact#Contact#Info): Messages are end-to-end encrypted. [NOTICED][INFO]
Msg#1006🔒: Me (Contact#Contact#Self): Member bob@example.net added. [INFO] √
Msg#1009🔒: Me (Contact#Contact#Self): hi √
Msg#1010🔒: Me (Contact#Contact#Self): You removed member bob@example.net. [INFO] √
Msg#1008🔒: Me (Contact#Contact#Self): hi √
Msg#1009🔒: Me (Contact#Contact#Self): You removed member bob@example.net. [INFO] √
--------------------------------------------------------------------------------