chore(release): prepare for 1.154.1

This partially restores the fix from c9cf2b7f2e
that was removed during the addition of new group consistency at de63527d94
but only for "Member added" messages.

Multiple "Member added" messages happen
when the same QR code is processed multiple times
by multiple devices.

.github/workflows/ci.yml

@@ -24,7 +24,7 @@ jobs:
     name: Lint Rust
     runs-on: ubuntu-latest
     env:
-      RUSTUP_TOOLCHAIN: 1.83.0
+      RUSTUP_TOOLCHAIN: 1.84.0
     steps:
       - uses: actions/checkout@v4
         with:
@@ -97,15 +97,15 @@ jobs:
       matrix:
         include:
           - os: ubuntu-latest
-            rust: 1.83.0
+            rust: 1.84.0
           - os: windows-latest
-            rust: 1.83.0
+            rust: 1.84.0
           - os: macos-latest
-            rust: 1.83.0
+            rust: 1.84.0
 
-          # Minimum Supported Rust Version = 1.81.0
+          # Minimum Supported Rust Version = 1.77.0
           - os: ubuntu-latest
-            rust: 1.81.0
+            rust: 1.77.0
     runs-on: ${{ matrix.os }}
     steps:
       - uses: actions/checkout@v4
@@ -152,7 +152,7 @@ jobs:
         uses: swatinem/rust-cache@v2
 
       - name: Build C library
-        run: cargo build -p deltachat_ffi --features jsonrpc
+        run: cargo build -p deltachat_ffi
 
       - name: Upload C library
         uses: actions/upload-artifact@v4
@@ -223,11 +223,11 @@ jobs:
           - os: macos-latest
             python: pypy3.10
 
-          # Minimum Supported Python Version = 3.7
+          # Minimum Supported Python Version = 3.8
           # This is the minimum version for which manylinux Python wheels are
           # built. Test it with minimum supported Rust version.
           - os: ubuntu-latest
-            python: 3.7
+            python: 3.8
 
     runs-on: ${{ matrix.os }}
     steps:
@@ -277,9 +277,9 @@ jobs:
           - os: macos-latest
             python: pypy3.10
 
-          # Minimum Supported Python Version = 3.7
+          # Minimum Supported Python Version = 3.8
           - os: ubuntu-latest
-            python: 3.7
+            python: 3.8
 
     runs-on: ${{ matrix.os }}
     steps:

CHANGELOG.md

@@ -1,5 +1,119 @@
 # Changelog
 
+## [1.154.1] - 2025-01-15
+
+### Tests
+
+- Expect trashing of no-op "member added" in non_member_cannot_modify_member_list.
+
+## [1.154.0] - 2025-01-15
+
+### Features / Changes
+
+- New group consistency algorithm.
+
+### Fixes
+
+- Migration: Set bcc_self=1 if it's unset and delete_server_after!=1 ([#6432](https://github.com/deltachat/deltachat-core-rust/pull/6432)).
+- Clear the config cache after every migration ([#6438](https://github.com/deltachat/deltachat-core-rust/pull/6438)).
+
+### Build system
+
+- Increase minimum supported Python version to 3.8.
+- [**breaking**] Remove jsonrpc feature flag.
+
+### CI
+
+- Update Rust to 1.84.0.
+
+### Miscellaneous Tasks
+
+- Beta Clippy suggestions ([#6422](https://github.com/deltachat/deltachat-core-rust/pull/6422)).
+
+### Refactor
+
+- Use let..else.
+- Add why_cant_send_ex() capable to only ignore specified conditions.
+- Remove unnecessary is_contact_in_chat check.
+- Eliminate remaining repeat_vars() calls ([#6359](https://github.com/deltachat/deltachat-core-rust/pull/6359)).
+
+### Tests
+
+- Use assert_eq! to compare chatlist length.
+
+## [1.153.0] - 2025-01-05
+
+### Features / Changes
+
+- Remove "jobs" from imap_markseen if folder doesn't exist ([#5870](https://github.com/deltachat/deltachat-core-rust/pull/5870)).
+- Delete `vg-request-with-auth` from IMAP after processing ([#6208](https://github.com/deltachat/deltachat-core-rust/pull/6208)).
+
+### API-Changes
+
+- Add `IncomingWebxdcNotify.chat_id` ([#6356](https://github.com/deltachat/deltachat-core-rust/pull/6356)).
+- rpc-client: Add INCOMING_REACTION to const.EventType ([#6349](https://github.com/deltachat/deltachat-core-rust/pull/6349)).
+
+### Documentation
+
+- Viewtype::Sticker may be changed to Image and how to disable that ([#6352](https://github.com/deltachat/deltachat-core-rust/pull/6352)).
+
+### Fixes
+
+- Never change Viewtype::Sticker to Image if file has non-image extension ([#6352](https://github.com/deltachat/deltachat-core-rust/pull/6352)).
+- Change BccSelf default to 0 for chatmail ([#6340](https://github.com/deltachat/deltachat-core-rust/pull/6340)).
+- Mark holiday notice messages as bot-generated.
+- Don't treat location-only and sync messages as bot ones ([#6357](https://github.com/deltachat/deltachat-core-rust/pull/6357)).
+- Update shadowsocks crate to 1.22.0 to avoid panic when parsing some QR codes.
+- Prefer to encrypt if E2eeEnabled even if peers have EncryptPreference::NoPreference.
+- Prioritize mailing list over self-sent messages.
+- Allow empty `To` field for self-sent messages.
+- Default `to_id` to self instead of 0.
+
+### Refactor
+
+- Remove unused parameter and return value from `build_body_file(…)` ([#6369](https://github.com/deltachat/deltachat-core-rust/pull/6369)).
+- Deprecate Param::ErroneousE2ee.
+- Add `emit_msgs_changed_without_msg_id`.
+- Add_parts: Remove excessive `is_mdn` checks.
+- Simplify `self_sent` condition.
+- Don't ignore get_for_contact errors.
+
+### Tests
+
+- Messages without recipients are assigned to self chat.
+- Message with empty To: field should have a valid to_id.
+- Fix `test_logged_ac_process_ffi_failure` flakiness.
+
+## [1.152.2] - 2024-12-24
+
+### Features / Changes
+
+- Emit ImexProgress(1) after receiving backup size.
+- `delete_msgs`: Use `transaction()` instead of `call_write()`.
+- Start ephemeral timers when the chat is noticed.
+- Start ephemeral timers when the chat is archived.
+- Revalidate HTTP cache entries once per minute maximum.
+
+### Fixes
+
+- Reduce number of `repeat_vars()` calls.
+- `sanitise_name`: Don't consider punctuation and control chars as part of file extension ([#6362](https://github.com/deltachat/deltachat-core-rust/pull/6362)).
+
+### Refactor
+
+- Remove marknoticed_chat_if_older_than().
+
+### Miscellaneous Tasks
+
+- Remove contrib/ directory.
+
+## [1.152.1] - 2024-12-17
+
+### Build system
+
+- Downgrade Rust version used to build binaries.
+- Reduce MSRV to 1.77.0.
+
 ## [1.152.0] - 2024-12-12
 
 ### API-Changes
@@ -5509,3 +5623,8 @@ https://github.com/deltachat/deltachat-core-rust/pulls?q=is%3Apr+is%3Aclosed
 [1.151.5]: https://github.com/deltachat/deltachat-core-rust/compare/v1.151.4..v1.151.5
 [1.151.6]: https://github.com/deltachat/deltachat-core-rust/compare/v1.151.5..v1.151.6
 [1.152.0]: https://github.com/deltachat/deltachat-core-rust/compare/v1.151.6..v1.152.0
+[1.152.1]: https://github.com/deltachat/deltachat-core-rust/compare/v1.152.0..v1.152.1
+[1.152.2]: https://github.com/deltachat/deltachat-core-rust/compare/v1.152.1..v1.152.2
+[1.153.0]: https://github.com/deltachat/deltachat-core-rust/compare/v1.152.2..v1.153.0
+[1.154.0]: https://github.com/deltachat/deltachat-core-rust/compare/v1.153.0..v1.154.0
+[1.154.1]: https://github.com/deltachat/deltachat-core-rust/compare/v1.154.0..v1.154.1

CMakeLists.txt

@@ -27,7 +27,7 @@ add_custom_command(
         PREFIX=${CMAKE_INSTALL_PREFIX}
         LIBDIR=${CMAKE_INSTALL_FULL_LIBDIR}
         INCLUDEDIR=${CMAKE_INSTALL_FULL_INCLUDEDIR}
-	${CARGO} build --target-dir=${CMAKE_BINARY_DIR}/target --release --features jsonrpc
+	${CARGO} build --target-dir=${CMAKE_BINARY_DIR}/target --release
 	WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}/deltachat-ffi
 )
 

Cargo.toml

@@ -1,9 +1,9 @@
 [package]
 name = "deltachat"
-version = "1.152.0"
+version = "1.154.1"
 edition = "2021"
 license = "MPL-2.0"
-rust-version = "1.81"
+rust-version = "1.77"
 repository = "https://github.com/deltachat/deltachat-core-rust"
 
 [profile.dev]
@@ -39,7 +39,7 @@ format-flowed = { path = "./format-flowed" }
 ratelimit = { path = "./deltachat-ratelimit" }
 
 anyhow = { workspace = true }
-async-broadcast = "0.7.1"
+async-broadcast = "0.7.2"
 async-channel = { workspace = true }
 async-imap = { version = "0.10.2", default-features = false, features = ["runtime-tokio", "compress"] }
 async-native-tls = { version = "0.5", default-features = false, features = ["runtime-tokio"] }
@@ -52,7 +52,7 @@ chrono = { workspace = true, features = ["alloc", "clock", "std"] }
 email = { git = "https://github.com/deltachat/rust-email", branch = "master" }
 encoded-words = { git = "https://github.com/async-email/encoded-words", branch = "master" }
 escaper = "0.1"
-fast-socks5 = "0.9"
+fast-socks5 = "0.10"
 fd-lock = "4"
 futures-lite = { workspace = true }
 futures = { workspace = true }
@@ -85,15 +85,15 @@ rand = { workspace = true }
 regex = { workspace = true }
 rusqlite = { workspace = true, features = ["sqlcipher"] }
 rust-hsluv = "0.1"
-rustls-pki-types = "1.10.0"
-rustls = { version = "0.23.19", default-features = false }
+rustls-pki-types = "1.10.1"
+rustls = { version = "0.23.20", default-features = false }
 sanitize-filename = { workspace = true }
 serde_json = { workspace = true }
 serde_urlencoded = "0.7.1"
 serde = { workspace = true, features = ["derive"] }
 sha-1 = "0.10"
 sha2 = "0.10"
-shadowsocks = { version = "1.21.0", default-features = false, features = ["aead-cipher-2022"] }
+shadowsocks = { version = "1.22.0", default-features = false, features = ["aead-cipher", "aead-cipher-2022"] }
 smallvec = "1.13.2"
 strum = "0.26"
 strum_macros = "0.26"
@@ -101,8 +101,8 @@ tagger = "4.3.4"
 textwrap = "0.16.1"
 thiserror = { workspace = true }
 tokio-io-timeout = "1.2.0"
-tokio-rustls = { version = "0.26.0", default-features = false }
-tokio-stream = { version = "0.1.16", features = ["fs"] }
+tokio-rustls = { version = "0.26.1", default-features = false }
+tokio-stream = { version = "0.1.17", features = ["fs"] }
 tokio-tar = { version = "0.3" } # TODO: integrate tokio into async-tar
 tokio-util = { workspace = true }
 tokio = { workspace = true, features = ["fs", "rt-multi-thread", "macros"] }
@@ -120,7 +120,7 @@ nu-ansi-term = { workspace = true }
 pretty_assertions = "1.4.1"
 proptest = { version = "1", default-features = false, features = ["std"] }
 tempfile = { workspace = true }
-testdir = "0.9.0"
+testdir = "0.9.3"
 tokio = { workspace = true, features = ["rt-multi-thread", "macros"] }
 
 [workspace]
@@ -169,7 +169,7 @@ harness = false
 anyhow = "1"
 async-channel = "2.3.1"
 base64 = "0.22"
-chrono = { version = "0.4.38", default-features = false }
+chrono = { version = "0.4.39", default-features = false }
 deltachat-contact-tools = { path = "deltachat-contact-tools" }
 deltachat-jsonrpc = { path = "deltachat-jsonrpc", default-features = false }
 deltachat = { path = ".", default-features = false }
@@ -189,7 +189,7 @@ serde_json = "1"
 tempfile = "3.14.0"
 thiserror = "1"
 tokio = "1"
-tokio-util = "0.7.11"
+tokio-util = "0.7.13"
 tracing-subscriber = "0.3"
 yerpc = "0.6.2"
 

contrib/proxy.py

@@ -1,80 +0,0 @@
-#!/usr/bin/env python3
-# Examples:
-#
-# Original server that doesn't use SSL:
-# ./proxy.py 8080 imap.nauta.cu 143
-# ./proxy.py 8081 smtp.nauta.cu 25
-#
-# Original server that uses SSL:
-# ./proxy.py 8080 testrun.org 993 --ssl
-# ./proxy.py 8081 testrun.org 465 --ssl
-
-from datetime import datetime
-import argparse
-import selectors
-import ssl
-import socket
-import socketserver
-
-
-class Proxy(socketserver.ThreadingTCPServer):
-    allow_reuse_address = True
-
-    def __init__(self, proxy_host, proxy_port, real_host, real_port, use_ssl):
-        self.real_host = real_host
-        self.real_port = real_port
-        self.use_ssl = use_ssl
-        super().__init__((proxy_host, proxy_port), RequestHandler)
-
-
-class RequestHandler(socketserver.BaseRequestHandler):
-
-    def handle(self):
-        print('{} - {} CONNECTED.'.format(datetime.now(), self.client_address))
-
-        total = 0
-        real_server = (self.server.real_host, self.server.real_port)
-        with socket.create_connection(real_server) as sock:
-            if self.server.use_ssl:
-                context = ssl.create_default_context()
-                sock = context.wrap_socket(
-                    sock, server_hostname=real_server[0])
-
-            forward = {self.request: sock, sock: self.request}
-
-            sel = selectors.DefaultSelector()
-            sel.register(self.request, selectors.EVENT_READ,
-                         self.client_address)
-            sel.register(sock, selectors.EVENT_READ, real_server)
-
-            active = True
-            while active:
-                events = sel.select()
-                for key, mask in events:
-                    print('\n{} - {} wrote:'.format(datetime.now(), key.data))
-                    data = key.fileobj.recv(1024)
-                    received = len(data)
-                    total += received
-                    print(data)
-                    print('{} Bytes\nTotal: {} Bytes'.format(received, total))
-                    if data:
-                        forward[key.fileobj].sendall(data)
-                    else:
-                        print('\nCLOSING CONNECTION.\n\n')
-                        forward[key.fileobj].close()
-                        key.fileobj.close()
-                        active = False
-
-
-if __name__ == '__main__':
-    p = argparse.ArgumentParser(description='Simple Python Proxy')
-    p.add_argument(
-        "proxy_port", help="the port where the proxy will listen", type=int)
-    p.add_argument('host', help="the real host")
-    p.add_argument('port', help="the port of the real host", type=int)
-    p.add_argument("--ssl", help="use ssl to connect to the real host",
-                   action="store_true")
-    args = p.parse_args()
-
-    with Proxy('', args.proxy_port, args.host, args.port, args.ssl) as proxy:
-        proxy.serve_forever()

deltachat-ffi/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "deltachat_ffi"
-version = "1.152.0"
+version = "1.154.1"
 description = "Deltachat FFI"
 edition = "2018"
 readme = "README.md"
@@ -15,7 +15,7 @@ crate-type = ["cdylib", "staticlib"]
 
 [dependencies]
 deltachat = { workspace = true, default-features = false }
-deltachat-jsonrpc = { workspace = true, optional = true }
+deltachat-jsonrpc = { workspace = true }
 libc = { workspace = true }
 human-panic = { version = "2", default-features = false }
 num-traits = { workspace = true }
@@ -30,5 +30,4 @@ yerpc = { workspace = true, features = ["anyhow_expose"] }
 [features]
 default = ["vendored"]
 vendored = ["deltachat/vendored", "deltachat-jsonrpc/vendored"]
-jsonrpc = ["dep:deltachat-jsonrpc"]
 

deltachat-ffi/deltachat.h

@@ -5393,6 +5393,8 @@ int64_t         dc_lot_get_timestamp     (const dc_lot_t* lot);
 
 /**
  * Message containing a sticker, similar to image.
+ * NB: When sending, the message viewtype may be changed to `Image` by some heuristics like checking
+ * for transparent pixels.
  * If possible, the UI should display the image without borders in a transparent way.
  * A click on a sticker will offer to install the sticker set in some future.
  */

deltachat-ffi/src/lib.rs

@@ -35,6 +35,8 @@ use deltachat::stock_str::StockMessage;
 use deltachat::webxdc::StatusUpdateSerial;
 use deltachat::*;
 use deltachat::{accounts::Accounts, log::LogExt};
+use deltachat_jsonrpc::api::CommandApi;
+use deltachat_jsonrpc::yerpc::{OutReceiver, RpcClient, RpcSession};
 use num_traits::{FromPrimitive, ToPrimitive};
 use once_cell::sync::Lazy;
 use rand::Rng;
@@ -4930,105 +4932,97 @@ pub unsafe extern "C" fn dc_accounts_get_event_emitter(
     Box::into_raw(Box::new(emitter))
 }
 
-#[cfg(feature = "jsonrpc")]
-mod jsonrpc {
-    use deltachat_jsonrpc::api::CommandApi;
-    use deltachat_jsonrpc::yerpc::{OutReceiver, RpcClient, RpcSession};
+pub struct dc_jsonrpc_instance_t {
+    receiver: OutReceiver,
+    handle: RpcSession<CommandApi>,
+}
 
-    use super::*;
-
-    pub struct dc_jsonrpc_instance_t {
-        receiver: OutReceiver,
-        handle: RpcSession<CommandApi>,
+#[no_mangle]
+pub unsafe extern "C" fn dc_jsonrpc_init(
+    account_manager: *mut dc_accounts_t,
+) -> *mut dc_jsonrpc_instance_t {
+    if account_manager.is_null() {
+        eprintln!("ignoring careless call to dc_jsonrpc_init()");
+        return ptr::null_mut();
     }
 
-    #[no_mangle]
-    pub unsafe extern "C" fn dc_jsonrpc_init(
-        account_manager: *mut dc_accounts_t,
-    ) -> *mut dc_jsonrpc_instance_t {
-        if account_manager.is_null() {
-            eprintln!("ignoring careless call to dc_jsonrpc_init()");
-            return ptr::null_mut();
-        }
+    let account_manager = &*account_manager;
+    let cmd_api = block_on(deltachat_jsonrpc::api::CommandApi::from_arc(
+        account_manager.inner.clone(),
+    ));
 
-        let account_manager = &*account_manager;
-        let cmd_api = block_on(deltachat_jsonrpc::api::CommandApi::from_arc(
-            account_manager.inner.clone(),
-        ));
+    let (request_handle, receiver) = RpcClient::new();
+    let handle = RpcSession::new(request_handle, cmd_api);
 
-        let (request_handle, receiver) = RpcClient::new();
-        let handle = RpcSession::new(request_handle, cmd_api);
+    let instance = dc_jsonrpc_instance_t { receiver, handle };
 
-        let instance = dc_jsonrpc_instance_t { receiver, handle };
+    Box::into_raw(Box::new(instance))
+}
 
-        Box::into_raw(Box::new(instance))
+#[no_mangle]
+pub unsafe extern "C" fn dc_jsonrpc_unref(jsonrpc_instance: *mut dc_jsonrpc_instance_t) {
+    if jsonrpc_instance.is_null() {
+        eprintln!("ignoring careless call to dc_jsonrpc_unref()");
+        return;
+    }
+    drop(Box::from_raw(jsonrpc_instance));
+}
+
+fn spawn_handle_jsonrpc_request(handle: RpcSession<CommandApi>, request: String) {
+    spawn(async move {
+        handle.handle_incoming(&request).await;
+    });
+}
+
+#[no_mangle]
+pub unsafe extern "C" fn dc_jsonrpc_request(
+    jsonrpc_instance: *mut dc_jsonrpc_instance_t,
+    request: *const libc::c_char,
+) {
+    if jsonrpc_instance.is_null() || request.is_null() {
+        eprintln!("ignoring careless call to dc_jsonrpc_request()");
+        return;
     }
 
-    #[no_mangle]
-    pub unsafe extern "C" fn dc_jsonrpc_unref(jsonrpc_instance: *mut dc_jsonrpc_instance_t) {
-        if jsonrpc_instance.is_null() {
-            eprintln!("ignoring careless call to dc_jsonrpc_unref()");
-            return;
-        }
-        drop(Box::from_raw(jsonrpc_instance));
+    let handle = &(*jsonrpc_instance).handle;
+    let request = to_string_lossy(request);
+    spawn_handle_jsonrpc_request(handle.clone(), request);
+}
+
+#[no_mangle]
+pub unsafe extern "C" fn dc_jsonrpc_next_response(
+    jsonrpc_instance: *mut dc_jsonrpc_instance_t,
+) -> *mut libc::c_char {
+    if jsonrpc_instance.is_null() {
+        eprintln!("ignoring careless call to dc_jsonrpc_next_response()");
+        return ptr::null_mut();
     }
+    let api = &*jsonrpc_instance;
+    block_on(api.receiver.recv())
+        .map(|result| serde_json::to_string(&result).unwrap_or_default().strdup())
+        .unwrap_or(ptr::null_mut())
+}
 
-    fn spawn_handle_jsonrpc_request(handle: RpcSession<CommandApi>, request: String) {
-        spawn(async move {
-            handle.handle_incoming(&request).await;
-        });
+#[no_mangle]
+pub unsafe extern "C" fn dc_jsonrpc_blocking_call(
+    jsonrpc_instance: *mut dc_jsonrpc_instance_t,
+    input: *const libc::c_char,
+) -> *mut libc::c_char {
+    if jsonrpc_instance.is_null() {
+        eprintln!("ignoring careless call to dc_jsonrpc_blocking_call()");
+        return ptr::null_mut();
     }
-
-    #[no_mangle]
-    pub unsafe extern "C" fn dc_jsonrpc_request(
-        jsonrpc_instance: *mut dc_jsonrpc_instance_t,
-        request: *const libc::c_char,
-    ) {
-        if jsonrpc_instance.is_null() || request.is_null() {
-            eprintln!("ignoring careless call to dc_jsonrpc_request()");
-            return;
-        }
-
-        let handle = &(*jsonrpc_instance).handle;
-        let request = to_string_lossy(request);
-        spawn_handle_jsonrpc_request(handle.clone(), request);
-    }
-
-    #[no_mangle]
-    pub unsafe extern "C" fn dc_jsonrpc_next_response(
-        jsonrpc_instance: *mut dc_jsonrpc_instance_t,
-    ) -> *mut libc::c_char {
-        if jsonrpc_instance.is_null() {
-            eprintln!("ignoring careless call to dc_jsonrpc_next_response()");
-            return ptr::null_mut();
-        }
-        let api = &*jsonrpc_instance;
-        block_on(api.receiver.recv())
-            .map(|result| serde_json::to_string(&result).unwrap_or_default().strdup())
-            .unwrap_or(ptr::null_mut())
-    }
-
-    #[no_mangle]
-    pub unsafe extern "C" fn dc_jsonrpc_blocking_call(
-        jsonrpc_instance: *mut dc_jsonrpc_instance_t,
-        input: *const libc::c_char,
-    ) -> *mut libc::c_char {
-        if jsonrpc_instance.is_null() {
-            eprintln!("ignoring careless call to dc_jsonrpc_blocking_call()");
-            return ptr::null_mut();
-        }
-        let api = &*jsonrpc_instance;
-        let input = to_string_lossy(input);
-        let res = block_on(api.handle.process_incoming(&input));
-        match res {
-            Some(message) => {
-                if let Ok(message) = serde_json::to_string(&message) {
-                    message.strdup()
-                } else {
-                    ptr::null_mut()
-                }
+    let api = &*jsonrpc_instance;
+    let input = to_string_lossy(input);
+    let res = block_on(api.handle.process_incoming(&input));
+    match res {
+        Some(message) => {
+            if let Ok(message) = serde_json::to_string(&message) {
+                message.strdup()
+            } else {
+                ptr::null_mut()
             }
-            None => ptr::null_mut(),
         }
+        None => ptr::null_mut(),
     }
 }

deltachat-jsonrpc/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "deltachat-jsonrpc"
-version = "1.152.0"
+version = "1.154.1"
 description = "DeltaChat JSON-RPC API"
 edition = "2021"
 default-run = "deltachat-jsonrpc-server"
@@ -33,7 +33,7 @@ base64 = { workspace = true }
 
 # optional dependencies
 axum = { version = "0.7", optional = true, features = ["ws"] }
-env_logger = { version = "0.11.5", optional = true }
+env_logger = { version = "0.11.6", optional = true }
 
 [dev-dependencies]
 tokio = { workspace = true, features = ["full", "rt-multi-thread"] }

deltachat-jsonrpc/src/api/types/events.rs

@@ -109,6 +109,7 @@ pub enum EventType {
     /// Incoming webxdc info or summary update, should be notified.
     #[serde(rename_all = "camelCase")]
     IncomingWebxdcNotify {
+        chat_id: u32,
         contact_id: u32,
         msg_id: u32,
         text: String,
@@ -343,11 +344,13 @@ impl From<CoreEventType> for EventType {
                 reaction: reaction.as_str().to_string(),
             },
             CoreEventType::IncomingWebxdcNotify {
+                chat_id,
                 contact_id,
                 msg_id,
                 text,
                 href,
             } => IncomingWebxdcNotify {
+                chat_id: chat_id.to_u32(),
                 contact_id: contact_id.to_u32(),
                 msg_id: msg_id.to_u32(),
                 text,

deltachat-jsonrpc/src/api/types/message.rs

@@ -273,6 +273,9 @@ pub enum MessageViewtype {
     Gif,
 
     /// Message containing a sticker, similar to image.
+    /// NB: When sending, the message viewtype may be changed to `Image` by some heuristics like
+    /// checking for transparent pixels. Use `Message::force_sticker()` to disable them.
+    ///
     /// If possible, the ui should display the image without borders in a transparent way.
     /// A click on a sticker will offer to install the sticker set in some future.
     Sticker,

deltachat-jsonrpc/typescript/package.json

@@ -58,5 +58,5 @@
   },
   "type": "module",
   "types": "dist/deltachat.d.ts",
-  "version": "1.152.0"
+  "version": "1.154.1"
 }

deltachat-repl/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "deltachat-repl"
-version = "1.152.0"
+version = "1.154.1"
 license = "MPL-2.0"
 edition = "2021"
 repository = "https://github.com/deltachat/deltachat-core-rust"
@@ -13,7 +13,7 @@ log = { workspace = true }
 nu-ansi-term = { workspace = true }
 qr2term = "0.3.3"
 rusqlite = { workspace = true }
-rustyline = "15"
+rustyline = "14"
 tokio = { workspace = true, features = ["fs", "rt-multi-thread", "macros"] }
 tracing-subscriber = { workspace = true, features = ["env-filter"] }
 

deltachat-repl/src/main.rs

@@ -22,7 +22,7 @@ use log::{error, info, warn};
 use nu_ansi_term::Color;
 use rustyline::completion::{Completer, FilenameCompleter, Pair};
 use rustyline::error::ReadlineError;
-use rustyline::highlight::{CmdKind as HighlightCmdKind, Highlighter, MatchingBracketHighlighter};
+use rustyline::highlight::{Highlighter, MatchingBracketHighlighter};
 use rustyline::hint::{Hinter, HistoryHinter};
 use rustyline::validate::Validator;
 use rustyline::{
@@ -298,8 +298,8 @@ impl Highlighter for DcHelper {
         self.highlighter.highlight(line, pos)
     }
 
-    fn highlight_char(&self, line: &str, pos: usize, kind: HighlightCmdKind) -> bool {
-        self.highlighter.highlight_char(line, pos, kind)
+    fn highlight_char(&self, line: &str, pos: usize, forced: bool) -> bool {
+        self.highlighter.highlight_char(line, pos, forced)
     }
 }
 

deltachat-rpc-client/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "deltachat-rpc-client"
-version = "1.152.0"
+version = "1.154.1"
 description = "Python client for Delta Chat core JSON-RPC interface"
 classifiers = [
     "Development Status :: 5 - Production/Stable",
@@ -13,7 +13,6 @@ classifiers = [
     "Operating System :: POSIX :: Linux",
     "Operating System :: MacOS :: MacOS X",
     "Programming Language :: Python :: 3",
-    "Programming Language :: Python :: 3.7",
     "Programming Language :: Python :: 3.8",
     "Programming Language :: Python :: 3.9",
     "Programming Language :: Python :: 3.10",
@@ -24,6 +23,7 @@ classifiers = [
     "Topic :: Communications :: Email"
 ]
 readme = "README.md"
+requires-python = ">=3.8"
 
 [tool.setuptools.package-data]
 deltachat_rpc_client = [

deltachat-rpc-client/src/deltachat_rpc_client/const.py

@@ -41,6 +41,7 @@ class EventType(str, Enum):
     REACTIONS_CHANGED = "ReactionsChanged"
     INCOMING_MSG = "IncomingMsg"
     INCOMING_MSG_BUNCH = "IncomingMsgBunch"
+    INCOMING_REACTION = "IncomingReaction"
     MSGS_NOTICED = "MsgsNoticed"
     MSG_DELIVERED = "MsgDelivered"
     MSG_FAILED = "MsgFailed"

deltachat-rpc-client/src/deltachat_rpc_client/rpc.py

@@ -131,10 +131,7 @@ class Rpc:
 
     def reader_loop(self) -> None:
         try:
-            while True:
-                line = self.process.stdout.readline()
-                if not line:  # EOF
-                    break
+            while line := self.process.stdout.readline():
                 response = json.loads(line)
                 if "id" in response:
                     response_id = response["id"]
@@ -150,10 +147,7 @@ class Rpc:
     def writer_loop(self) -> None:
         """Writer loop ensuring only a single thread writes requests."""
         try:
-            while True:
-                request = self.request_queue.get()
-                if not request:
-                    break
+            while request := self.request_queue.get():
                 data = (json.dumps(request) + "\n").encode()
                 self.process.stdin.write(data)
                 self.process.stdin.flush()

deltachat-rpc-client/tests/test_securejoin.py

@@ -73,22 +73,25 @@ def test_qr_securejoin(acfactory, protect, tmp_path):
     qr_code = alice_chat.get_qr_code()
     bob.secure_join(qr_code)
 
-    # Check that at least some of the handshake messages are deleted.
+    # Alice deletes "vg-request".
+    while True:
+        event = alice.wait_for_event()
+        if event["kind"] == "ImapMessageDeleted":
+            break
+    alice.wait_for_securejoin_inviter_success()
+    # Bob deletes "vg-auth-required", Alice deletes "vg-request-with-auth".
     for ac in [alice, bob]:
         while True:
             event = ac.wait_for_event()
             if event["kind"] == "ImapMessageDeleted":
                 break
-
-    alice.wait_for_securejoin_inviter_success()
+    bob.wait_for_securejoin_joiner_success()
 
     # Test that Alice verified Bob's profile.
     alice_contact_bob = alice.get_contact_by_addr(bob.get_config("addr"))
     alice_contact_bob_snapshot = alice_contact_bob.get_snapshot()
     assert alice_contact_bob_snapshot.is_verified
 
-    bob.wait_for_securejoin_joiner_success()
-
     snapshot = bob.get_message_by_id(bob.wait_for_incoming_msg_event().msg_id).get_snapshot()
     assert snapshot.text == "Member Me ({}) added by {}.".format(bob.get_config("addr"), alice.get_config("addr"))
     assert snapshot.chat.get_basic_snapshot().is_protected == protect

deltachat-rpc-server/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "deltachat-rpc-server"
-version = "1.152.0"
+version = "1.154.1"
 description = "DeltaChat JSON-RPC server"
 edition = "2021"
 readme = "README.md"

deltachat-rpc-server/npm-package/package.json

@@ -15,5 +15,5 @@
   },
   "type": "module",
   "types": "index.d.ts",
-  "version": "1.152.0"
+  "version": "1.154.1"
 }

deny.toml

@@ -51,8 +51,9 @@ skip = [
      { name = "regex-syntax", version = "0.6.29" },
      { name = "sync_wrapper", version = "0.1.2" },
      { name = "syn", version = "1.0.109" },
+     { name = "thiserror-impl", version = "1.0.69" },
+     { name = "thiserror", version = "1.0.69" },
      { name = "time", version = "<0.3" },
-     { name = "unicode-width", version = "0.1.11" },
      { name = "wasi", version = "<0.11" },
      { name = "windows_aarch64_gnullvm", version = "<0.52" },
      { name = "windows_aarch64_msvc", version = "<0.52" },

flake.lock

@@ -47,16 +47,17 @@
         "rust-analyzer-src": "rust-analyzer-src"
       },
       "locked": {
-        "lastModified": 1731393059,
-        "narHash": "sha256-rmzi0GHEwpzg1LGfGPO4SRD7D6QGV3UYGQxkJvn+J5U=",
+        "lastModified": 1711088506,
+        "narHash": "sha256-USdlY7Tx2oJWqFBpp10+03+h7eVhpkQ4s9t1ERjeIJE=",
         "owner": "nix-community",
         "repo": "fenix",
-        "rev": "fda8d5b59bb0dc0021ad3ba1d722f9ef6d36e4d9",
+        "rev": "85f4139f3c092cf4afd9f9906d7ed218ef262c97",
         "type": "github"
       },
       "original": {
         "owner": "nix-community",
         "repo": "fenix",
+        "rev": "85f4139f3c092cf4afd9f9906d7ed218ef262c97",
         "type": "github"
       }
     },
@@ -114,6 +115,25 @@
         "type": "github"
       }
     },
+    "new-fenix": {
+      "inputs": {
+        "nixpkgs": "nixpkgs_4",
+        "rust-analyzer-src": "rust-analyzer-src_2"
+      },
+      "locked": {
+        "lastModified": 1734417396,
+        "narHash": "sha256-32x1Z+Pz3Jv0cK9EG56cFTKXy/mZ/c+Ikxw+aVfKHp4=",
+        "owner": "nix-community",
+        "repo": "fenix",
+        "rev": "a18d41b26e998e95a598858fdb86ba22fb5da47d",
+        "type": "github"
+      },
+      "original": {
+        "owner": "nix-community",
+        "repo": "fenix",
+        "type": "github"
+      }
+    },
     "nix-filter": {
       "locked": {
         "lastModified": 1730207686,
@@ -174,6 +194,22 @@
       }
     },
     "nixpkgs_4": {
+      "locked": {
+        "lastModified": 1734119587,
+        "narHash": "sha256-AKU6qqskl0yf2+JdRdD0cfxX4b9x3KKV5RqA6wijmPM=",
+        "owner": "nixos",
+        "repo": "nixpkgs",
+        "rev": "3566ab7246670a43abd2ffa913cc62dad9cdf7d5",
+        "type": "github"
+      },
+      "original": {
+        "owner": "nixos",
+        "ref": "nixos-unstable",
+        "repo": "nixpkgs",
+        "type": "github"
+      }
+    },
+    "nixpkgs_5": {
       "locked": {
         "lastModified": 1731139594,
         "narHash": "sha256-IigrKK3vYRpUu+HEjPL/phrfh7Ox881er1UEsZvw9Q4=",
@@ -195,8 +231,9 @@
         "fenix": "fenix",
         "flake-utils": "flake-utils_2",
         "naersk": "naersk",
+        "new-fenix": "new-fenix",
         "nix-filter": "nix-filter",
-        "nixpkgs": "nixpkgs_4"
+        "nixpkgs": "nixpkgs_5"
       }
     },
     "rust-analyzer-src": {
@@ -216,6 +253,23 @@
         "type": "github"
       }
     },
+    "rust-analyzer-src_2": {
+      "flake": false,
+      "locked": {
+        "lastModified": 1734386068,
+        "narHash": "sha256-Py025JiD9lcPmldB7X1AEjq3WBTS60jZUJRtTDonmaE=",
+        "owner": "rust-lang",
+        "repo": "rust-analyzer",
+        "rev": "0a706f7d2ac093985eae317781200689cfd48b78",
+        "type": "github"
+      },
+      "original": {
+        "owner": "rust-lang",
+        "ref": "nightly",
+        "repo": "rust-analyzer",
+        "type": "github"
+      }
+    },
     "systems": {
       "locked": {
         "lastModified": 1681028828,

flake.nix

@@ -1,14 +1,19 @@
 {
   description = "Delta Chat core";
   inputs = {
-    fenix.url = "github:nix-community/fenix";
+    # Old Rust to build releases.
+    fenix.url = "github:nix-community/fenix?rev=85f4139f3c092cf4afd9f9906d7ed218ef262c97";
+
+    # New Rust for development shell.
+    new-fenix.url = "github:nix-community/fenix";
+
     flake-utils.url = "github:numtide/flake-utils";
     naersk.url = "github:nix-community/naersk";
     nix-filter.url = "github:numtide/nix-filter";
     nixpkgs.url = "github:nixos/nixpkgs/nixos-unstable";
     android.url = "github:tadfisher/android-nixpkgs";
   };
-  outputs = { self, nixpkgs, flake-utils, nix-filter, naersk, fenix, android }:
+  outputs = { self, nixpkgs, flake-utils, nix-filter, naersk, fenix, new-fenix, android }:
     flake-utils.lib.eachDefaultSystem (system:
       let
         pkgs = nixpkgs.legacyPackages.${system};
@@ -539,13 +544,13 @@
           let
             pkgs = import nixpkgs {
               system = system;
-              overlays = [ fenix.overlays.default ];
+              overlays = [ new-fenix.overlays.default ];
             };
           in
           pkgs.mkShell {
 
             buildInputs = with pkgs; [
-              (fenix.packages.${system}.complete.withComponents [
+              (new-fenix.packages.${system}.complete.withComponents [
                 "cargo"
                 "clippy"
                 "rust-src"

node/scripts/rebuild-core.js

@@ -9,7 +9,7 @@ const buildArgs = [
   'build',
   '--release',
   '--features',
-  'vendored,jsonrpc',
+  'vendored',
   '-p',
   'deltachat_ffi'
 ]

package.json

@@ -55,5 +55,5 @@
     "test:mocha": "mocha node/test/test.mjs --growl --reporter=spec --bail --exit"
   },
   "types": "node/dist/index.d.ts",
-  "version": "1.152.0"
+  "version": "1.154.1"
 }

python/doc/cffi/install.rst

@@ -52,10 +52,7 @@ python3-venv` should give you a usable python installation.
 
 First, build the core library::
 
-   cargo build --release -p deltachat_ffi --features jsonrpc
-
-`jsonrpc` feature is required even if not used by the bindings
-because `deltachat.h` includes JSON-RPC functions unconditionally.
+   cargo build --release -p deltachat_ffi
 
 Create the virtual environment and activate it::
 

python/pyproject.toml

@@ -4,10 +4,10 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "deltachat"
-version = "1.152.0"
+version = "1.154.1"
 description = "Python bindings for the Delta Chat Core library using CFFI against the Rust-implemented libdeltachat"
 readme = "README.rst"
-requires-python = ">=3.7"
+requires-python = ">=3.8"
 authors = [
     { name = "holger krekel, Floris Bruynooghe, Bjoern Petersen and contributors" },
 ]

python/tests/test_1_online.py

@@ -1253,7 +1253,10 @@ def test_no_old_msg_is_fresh(acfactory, lp):
 
 def test_prefer_encrypt(acfactory, lp):
     """Test quorum rule for encryption preference in 1:1 and group chat."""
-    ac1, ac2, ac3 = acfactory.get_online_accounts(3)
+    ac1 = acfactory.new_online_configuring_account(fix_is_chatmail=True)
+    ac2 = acfactory.new_online_configuring_account(fix_is_chatmail=True)
+    ac3 = acfactory.new_online_configuring_account(fix_is_chatmail=True)
+    acfactory.bring_accounts_online()
     ac1.set_config("e2ee_enabled", "0")
     ac2.set_config("e2ee_enabled", "1")
     ac3.set_config("e2ee_enabled", "0")
@@ -1276,7 +1279,8 @@ def test_prefer_encrypt(acfactory, lp):
     lp.sec("ac2: sending message to ac1")
     chat2 = ac2.create_chat(ac1)
     msg2 = chat2.send_text("message2")
-    assert not msg2.is_encrypted()
+    # Own preference is `Mutual` and we have the peer's key.
+    assert msg2.is_encrypted()
     ac1._evtracker.wait_next_incoming_message()
 
     lp.sec("ac1: sending message to group chat with ac2 and ac3")
@@ -1292,8 +1296,8 @@ def test_prefer_encrypt(acfactory, lp):
     ac3.set_config("e2ee_enabled", "1")
     chat3 = ac3.create_chat(ac1)
     msg4 = chat3.send_text("message4")
-    # ac1 still does not prefer encryption
-    assert not msg4.is_encrypted()
+    # Own preference is `Mutual` and we have the peer's key.
+    assert msg4.is_encrypted()
     ac1._evtracker.wait_next_incoming_message()
 
     lp.sec("ac1: sending another message to group chat with ac2 and ac3")

python/tests/test_4_lowlevel.py

@@ -212,8 +212,13 @@ def test_logged_ac_process_ffi_failure(acfactory):
             0 / 0
 
     cap = Queue()
-    ac1.log = cap.put
+
+    # Make sure the next attempt to log an event fails.
     ac1.add_account_plugin(FailPlugin())
+
+    # Start capturing events.
+    ac1.log = cap.put
+
     # cause any event eg contact added/changed
     ac1.create_contact("something@example.org")
     res = cap.get(timeout=10)

release-date.in

@@ -1 +1 @@
-2024-12-12
+2025-01-15

scripts/coredeps/install-rust.sh

@@ -7,7 +7,7 @@ set -euo pipefail
 #
 # Avoid using rustup here as it depends on reading /proc/self/exe and
 # has problems running under QEMU.
-RUST_VERSION=1.83.0
+RUST_VERSION=1.84.0
 
 ARCH="$(uname -m)"
 test -f "/lib/libc.musl-$ARCH.so.1" && LIBC=musl || LIBC=gnu

scripts/make-python-testenv.sh

@@ -11,7 +11,7 @@ set -euo pipefail
 
 export DCC_RS_TARGET=debug
 export DCC_RS_DEV="$PWD"
-cargo build -p deltachat_ffi --features jsonrpc
+cargo build -p deltachat_ffi
 
 tox -c python -e py --devenv venv
 venv/bin/pip install --upgrade pip

scripts/run-python-test.sh

@@ -12,7 +12,7 @@ export DCC_RS_DEV=`pwd`
 
 cd python
 
-cargo build -p deltachat_ffi --features jsonrpc
+cargo build -p deltachat_ffi
 
 # remove and inhibit writing PYC files 
 rm -rf tests/__pycache__

scripts/run_all.sh

@@ -8,7 +8,7 @@ set -e -x
 
 # compile core lib
 
-cargo build --release -p deltachat_ffi --features jsonrpc
+cargo build --release -p deltachat_ffi
 
 # Statically link against libdeltachat.a.
 export DCC_RS_DEV="$PWD"
@@ -31,6 +31,6 @@ unset CHATMAIL_DOMAIN
 
 # Try to build wheels for a range of interpreters, but don't fail if they are not available.
 # E.g. musllinux_1_1 does not have PyPy interpreters as of 2022-07-10
-tox --workdir "$TOXWORKDIR" -e py37,py38,py39,py310,py311,py312,py313,pypy37,pypy38,pypy39,pypy310 --skip-missing-interpreters true
+tox --workdir "$TOXWORKDIR" -e py38,py39,py310,py311,py312,py313,pypy38,pypy39,pypy310 --skip-missing-interpreters true
 
 auditwheel repair "$TOXWORKDIR"/wheelhouse/deltachat* -w "$TOXWORKDIR/wheelhouse"

src/blob.rs

@@ -279,7 +279,9 @@ impl<'a> BlobObject<'a> {
         let ext: String = name
             .chars()
             .rev()
-            .take_while(|c| !c.is_whitespace())
+            .take_while(|c| {
+                (!c.is_ascii_punctuation() || *c == '.') && !c.is_whitespace() && !c.is_control()
+            })
             .take(33)
             .collect::<Vec<_>>()
             .iter()
@@ -982,6 +984,10 @@ mod tests {
         let (stem, ext) = BlobObject::sanitise_name("a. tar.tar.gz");
         assert_eq!(stem, "a. tar");
         assert_eq!(ext, ".tar.gz");
+
+        let (stem, ext) = BlobObject::sanitise_name("Guia_uso_GNB (v0.8).pdf");
+        assert_eq!(stem, "Guia_uso_GNB (v0.8)");
+        assert_eq!(ext, ".pdf");
     }
 
     #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
@@ -1455,4 +1461,23 @@ mod tests {
         check_image_size(file_saved, width, height);
         Ok(())
     }
+
+    #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
+    async fn test_send_gif_as_sticker() -> Result<()> {
+        let bytes = include_bytes!("../test-data/image/image100x50.gif");
+        let alice = &TestContext::new_alice().await;
+        let file = alice.get_blobdir().join("file").with_extension("gif");
+        fs::write(&file, &bytes)
+            .await
+            .context("failed to write file")?;
+        let mut msg = Message::new(Viewtype::Sticker);
+        msg.set_file(file.to_str().unwrap(), None);
+        let chat = alice.get_self_chat().await;
+        let sent = alice.send_msg(chat.id, &mut msg).await;
+        let msg = Message::load_from_db(alice, sent.sender_msg_id).await?;
+        // Message::force_sticker() wasn't used, still Viewtype::Sticker is preserved because of the
+        // extension.
+        assert_eq!(msg.get_viewtype(), Viewtype::Sticker);
+        Ok(())
+    }
 }

src/chat.rs

@@ -3,6 +3,7 @@
 use std::cmp;
 use std::collections::{HashMap, HashSet};
 use std::fmt;
+use std::marker::Sync;
 use std::path::{Path, PathBuf};
 use std::str::FromStr;
 use std::time::Duration;
@@ -28,7 +29,7 @@ use crate::contact::{self, Contact, ContactId, Origin};
 use crate::context::Context;
 use crate::debug_logging::maybe_set_logging_xdc;
 use crate::download::DownloadState;
-use crate::ephemeral::Timer as EphemeralTimer;
+use crate::ephemeral::{start_chat_ephemeral_timers, Timer as EphemeralTimer};
 use crate::events::EventType;
 use crate::html::new_html_mimepart;
 use crate::location;
@@ -41,7 +42,6 @@ use crate::peerstate::Peerstate;
 use crate::receive_imf::ReceivedMsg;
 use crate::securejoin::BobState;
 use crate::smtp::send_msg_to_smtp;
-use crate::sql;
 use crate::stock_str;
 use crate::sync::{self, Sync::*, SyncData};
 use crate::tools::{
@@ -312,7 +312,7 @@ impl ChatId {
 
     /// Create a group or mailinglist raw database record with the given parameters.
     /// The function does not add SELF nor checks if the record already exists.
-    #[expect(clippy::too_many_arguments)]
+    #[allow(clippy::too_many_arguments)]
     pub(crate) async fn create_multiuser_record(
         context: &Context,
         chattype: Chattype,
@@ -364,8 +364,8 @@ impl ChatId {
             .sql
             .execute(
                 "UPDATE contacts
-                SET selfavatar_sent=?
-              WHERE id IN(SELECT contact_id FROM chats_contacts WHERE chat_id=?);",
+                 SET selfavatar_sent=?
+                 WHERE id IN(SELECT contact_id FROM chats_contacts WHERE chat_id=? AND add_timestamp >= remove_timestamp)",
                 (timestamp, self),
             )
             .await?;
@@ -688,6 +688,10 @@ impl ChatId {
             })
             .await?;
 
+        if visibility == ChatVisibility::Archived {
+            start_chat_ephemeral_timers(context, self).await?;
+        }
+
         context.emit_msgs_changed_without_ids();
         chatlist_events::emit_chatlist_changed(context);
         chatlist_events::emit_chatlist_item_changed(context, self);
@@ -743,7 +747,7 @@ impl ChatId {
                 .await?;
             if unread_cnt == 1 {
                 // Added the first unread message in the chat.
-                context.emit_msgs_changed(DC_CHAT_ID_ARCHIVED_LINK, MsgId::new(0));
+                context.emit_msgs_changed_without_msg_id(DC_CHAT_ID_ARCHIVED_LINK);
             }
             return Ok(());
         }
@@ -758,6 +762,8 @@ impl ChatId {
     /// shown.
     pub(crate) fn emit_msg_event(self, context: &Context, msg_id: MsgId, important: bool) {
         if important {
+            debug_assert!(!msg_id.is_unset());
+
             context.emit_incoming_msg(self, msg_id);
         } else {
             context.emit_msgs_changed(self, msg_id);
@@ -819,17 +825,14 @@ impl ChatId {
         };
 
         if changed {
-            context.emit_msgs_changed(
-                self,
-                if msg.is_some() {
-                    match self.get_draft_msg_id(context).await? {
-                        Some(msg_id) => msg_id,
-                        None => MsgId::new(0),
-                    }
-                } else {
-                    MsgId::new(0)
-                },
-            );
+            if msg.is_some() {
+                match self.get_draft_msg_id(context).await? {
+                    Some(msg_id) => context.emit_msgs_changed(self, msg_id),
+                    None => context.emit_msgs_changed_without_msg_id(self),
+                }
+            } else {
+                context.emit_msgs_changed_without_msg_id(self)
+            }
         }
 
         Ok(())
@@ -1077,6 +1080,8 @@ impl ChatId {
                  JOIN chats_contacts as y
                  WHERE x.contact_id > 9
                    AND y.contact_id > 9
+                   AND x.add_timestamp >= x.remove_timestamp
+                   AND y.add_timestamp >= y.remove_timestamp
                    AND x.chat_id=?
                    AND y.chat_id<>x.chat_id
                    AND y.chat_id>?
@@ -1101,6 +1106,7 @@ impl ChatId {
                 "SELECT chat_id, count(*) AS n
                  FROM chats_contacts
                  WHERE contact_id > ? AND chat_id > ?
+                 AND add_timestamp >= remove_timestamp
                  GROUP BY chat_id",
                 (ContactId::LAST_SPECIAL, DC_CHAT_ID_LAST_SPECIAL),
                 |row| {
@@ -1215,15 +1221,6 @@ impl ChatId {
         Ok(self.get_param(context).await?.exists(Param::Devicetalk))
     }
 
-    /// Returns chat member list timestamp.
-    pub(crate) async fn get_member_list_timestamp(self, context: &Context) -> Result<i64> {
-        Ok(self
-            .get_param(context)
-            .await?
-            .get_i64(Param::MemberListTimestamp)
-            .unwrap_or_default())
-    }
-
     async fn parent_query<T, F>(
         self,
         context: &Context,
@@ -1649,31 +1646,63 @@ impl Chat {
     ///
     /// Otherwise returns a reason useful for logging.
     pub(crate) async fn why_cant_send(&self, context: &Context) -> Result<Option<CantSendReason>> {
-        use CantSendReason::*;
+        self.why_cant_send_ex(context, &|_| false).await
+    }
 
+    pub(crate) async fn why_cant_send_ex(
+        &self,
+        context: &Context,
+        skip_fn: &(dyn Send + Sync + Fn(&CantSendReason) -> bool),
+    ) -> Result<Option<CantSendReason>> {
+        use CantSendReason::*;
         // NB: Don't forget to update Chatlist::try_load() when changing this function!
-        let reason = if self.id.is_special() {
-            Some(SpecialChat)
-        } else if self.is_device_talk() {
-            Some(DeviceChat)
-        } else if self.is_contact_request() {
-            Some(ContactRequest)
-        } else if self.is_protection_broken() {
-            Some(ProtectionBroken)
-        } else if self.is_mailing_list() && self.get_mailinglist_addr().is_none_or_empty() {
-            Some(ReadOnlyMailingList)
-        } else if !self.is_self_in_chat(context).await? {
-            Some(NotAMember)
-        } else if self
-            .check_securejoin_wait(context, constants::SECUREJOIN_WAIT_TIMEOUT)
-            .await?
-            > 0
+
+        if self.id.is_special() {
+            let reason = SpecialChat;
+            if !skip_fn(&reason) {
+                return Ok(Some(reason));
+            }
+        }
+        if self.is_device_talk() {
+            let reason = DeviceChat;
+            if !skip_fn(&reason) {
+                return Ok(Some(reason));
+            }
+        }
+        if self.is_contact_request() {
+            let reason = ContactRequest;
+            if !skip_fn(&reason) {
+                return Ok(Some(reason));
+            }
+        }
+        if self.is_protection_broken() {
+            let reason = ProtectionBroken;
+            if !skip_fn(&reason) {
+                return Ok(Some(reason));
+            }
+        }
+        if self.is_mailing_list() && self.get_mailinglist_addr().is_none_or_empty() {
+            let reason = ReadOnlyMailingList;
+            if !skip_fn(&reason) {
+                return Ok(Some(reason));
+            }
+        }
+
+        // Do potentially slow checks last and after calls to `skip_fn` which should be fast.
+        let reason = NotAMember;
+        if !skip_fn(&reason) && !self.is_self_in_chat(context).await? {
+            return Ok(Some(reason));
+        }
+        let reason = SecurejoinWait;
+        if !skip_fn(&reason)
+            && self
+                .check_securejoin_wait(context, constants::SECUREJOIN_WAIT_TIMEOUT)
+                .await?
+                > 0
         {
-            Some(SecurejoinWait)
-        } else {
-            None
-        };
-        Ok(reason)
+            return Ok(Some(reason));
+        }
+        Ok(None)
     }
 
     /// Returns true if can send to the chat.
@@ -1854,7 +1883,6 @@ impl Chat {
             profile_image: self
                 .get_profile_image(context)
                 .await?
-                .map(Into::into)
                 .unwrap_or_else(std::path::PathBuf::new),
             draft,
             is_muted: self.is_muted(),
@@ -1987,9 +2015,6 @@ impl Chat {
                 .ok();
         }
 
-        // reset encrypt error state eg. for forwarding
-        msg.param.remove(Param::ErroneousE2ee);
-
         let is_bot = context.get_config_bool(Config::Bot).await?;
         msg.param
             .set_optional(Param::Bot, Some("1").filter(|_| is_bot));
@@ -2232,7 +2257,7 @@ impl Chat {
                 "SELECT c.addr \
                 FROM contacts c INNER JOIN chats_contacts cc \
                 ON c.id=cc.contact_id \
-                WHERE cc.chat_id=?",
+                WHERE cc.chat_id=? AND cc.add_timestamp >= cc.remove_timestamp",
                 (self.id,),
                 |row| row.get::<_, String>(0),
                 |addrs| addrs.collect::<Result<Vec<_>, _>>().map_err(Into::into),
@@ -2571,7 +2596,6 @@ impl ChatIdBlocked {
                 },
             )
             .await
-            .map_err(Into::into)
     }
 
     /// Returns the chat for the 1:1 chat with this contact.
@@ -2688,7 +2712,10 @@ async fn prepare_msg_blob(context: &Context, msg: &mut Message) -> Result<()> {
             .with_context(|| format!("attachment missing for message of type #{}", msg.viewtype))?;
         let send_as_is = msg.viewtype == Viewtype::File;
 
-        if msg.viewtype == Viewtype::File || msg.viewtype == Viewtype::Image {
+        if msg.viewtype == Viewtype::File
+            || msg.viewtype == Viewtype::Image
+            || msg.viewtype == Viewtype::Sticker && !msg.param.exists(Param::ForceSticker)
+        {
             // Correct the type, take care not to correct already very special
             // formats as GIF or VOICE.
             //
@@ -2697,7 +2724,12 @@ async fn prepare_msg_blob(context: &Context, msg: &mut Message) -> Result<()> {
             // - from FILE/IMAGE to GIF */
             if let Some((better_type, _)) = message::guess_msgtype_from_suffix(&blob.to_abs_path())
             {
-                if better_type != Viewtype::Webxdc
+                if msg.viewtype == Viewtype::Sticker {
+                    if better_type != Viewtype::Image {
+                        // UIs don't want conversions of `Sticker` to anything other than `Image`.
+                        msg.param.set_int(Param::ForceSticker, 1);
+                    }
+                } else if better_type != Viewtype::Webxdc
                     || context
                         .ensure_sendable_webxdc_file(&blob.to_abs_path())
                         .await
@@ -2772,7 +2804,9 @@ pub async fn is_contact_in_chat(
     let exists = context
         .sql
         .exists(
-            "SELECT COUNT(*) FROM chats_contacts WHERE chat_id=? AND contact_id=?;",
+            "SELECT COUNT(*) FROM chats_contacts
+             WHERE chat_id=? AND contact_id=?
+             AND add_timestamp >= remove_timestamp",
             (chat_id, contact_id),
         )
         .await?;
@@ -2846,20 +2880,22 @@ async fn prepare_send_msg(
 ) -> Result<Vec<i64>> {
     let mut chat = Chat::load_from_db(context, chat_id).await?;
 
-    // Check if the chat can be sent to.
-    if let Some(reason) = chat.why_cant_send(context).await? {
-        if matches!(
-            reason,
-            CantSendReason::ProtectionBroken
-                | CantSendReason::ContactRequest
-                | CantSendReason::SecurejoinWait
-        ) && msg.param.get_cmd() == SystemMessage::SecurejoinMessage
-        {
-            // Send out the message, the securejoin message is supposed to repair the verification.
+    let skip_fn = |reason: &CantSendReason| match reason {
+        CantSendReason::ProtectionBroken
+        | CantSendReason::ContactRequest
+        | CantSendReason::SecurejoinWait => {
+            // Allow securejoin messages, they are supposed to repair the verification.
             // If the chat is a contact request, let the user accept it later.
-        } else {
-            bail!("cannot send to {chat_id}: {reason}");
+            msg.param.get_cmd() == SystemMessage::SecurejoinMessage
         }
+        // Allow to send "Member removed" messages so we can leave the group.
+        // Necessary checks should be made anyway before removing contact
+        // from the chat.
+        CantSendReason::NotAMember => msg.param.get_cmd() == SystemMessage::MemberRemovedFromGroup,
+        _ => false,
+    };
+    if let Some(reason) = chat.why_cant_send_ex(context, &skip_fn).await? {
+        bail!("Cannot send to {chat_id}: {reason}");
     }
 
     // Check a quote reply is not leaking data from other chats.
@@ -2931,7 +2967,8 @@ pub(crate) async fn create_send_msg_jobs(context: &Context, msg: &mut Message) -
     // because BCC-self messages are also used to detect
     // that message was sent if SMTP server is slow to respond
     // and connection is frequently lost
-    // before receiving status line.
+    // before receiving status line. NB: This is not a problem for chatmail servers, so `BccSelf`
+    // disabled by default is fine.
     //
     // `from` must be the last addr, see `receive_imf_inner()` why.
     if context.get_config_bool(Config::BccSelf).await?
@@ -2987,18 +3024,6 @@ pub(crate) async fn create_send_msg_jobs(context: &Context, msg: &mut Message) -
         msg.chat_id.set_gossiped_timestamp(context, now).await?;
     }
 
-    if msg.param.get_cmd() == SystemMessage::MemberRemovedFromGroup {
-        // Reject member list synchronisation from older messages. See also
-        // `receive_imf::apply_group_changes()`.
-        msg.chat_id
-            .update_timestamp(
-                context,
-                Param::MemberListTimestamp,
-                now.saturating_add(constants::TIMESTAMP_SENT_TOLERANCE),
-            )
-            .await?;
-    }
-
     if rendered_msg.last_added_location_id.is_some() {
         if let Err(err) = location::set_kml_sent_timestamp(context, msg.chat_id, now).await {
             error!(context, "Failed to set kml sent_timestamp: {err:#}.");
@@ -3231,19 +3256,6 @@ pub async fn get_chat_msgs_ex(
     Ok(items)
 }
 
-pub(crate) async fn marknoticed_chat_if_older_than(
-    context: &Context,
-    chat_id: ChatId,
-    timestamp: i64,
-) -> Result<()> {
-    if let Some(chat_timestamp) = chat_id.get_timestamp(context).await? {
-        if timestamp > chat_timestamp {
-            marknoticed_chat(context, chat_id).await?;
-        }
-    }
-    Ok(())
-}
-
 /// Marks all messages in the chat as noticed.
 /// If the given chat-id is the archive-link, marks all messages in all archived chats as noticed.
 pub async fn marknoticed_chat(context: &Context, chat_id: ChatId) -> Result<()> {
@@ -3255,10 +3267,10 @@ pub async fn marknoticed_chat(context: &Context, chat_id: ChatId) -> Result<()>
             .query_map(
                 "SELECT DISTINCT(m.chat_id) FROM msgs m
                     LEFT JOIN chats c ON m.chat_id=c.id
-                    WHERE m.state=10 AND m.hidden=0 AND m.chat_id>9 AND c.blocked=0 AND c.archived=1",
-                    (),
+                    WHERE m.state=10 AND m.hidden=0 AND m.chat_id>9 AND c.archived=1",
+                (),
                 |row| row.get::<_, ChatId>(0),
-                |ids| ids.collect::<Result<Vec<_>, _>>().map_err(Into::into)
+                |ids| ids.collect::<Result<Vec<_>, _>>().map_err(Into::into),
             )
             .await?;
         if chat_ids_in_archive.is_empty() {
@@ -3267,32 +3279,40 @@ pub async fn marknoticed_chat(context: &Context, chat_id: ChatId) -> Result<()>
 
         context
             .sql
-            .execute(
-                &format!(
-                    "UPDATE msgs SET state=13 WHERE state=10 AND hidden=0 AND chat_id IN ({});",
-                    sql::repeat_vars(chat_ids_in_archive.len())
-                ),
-                rusqlite::params_from_iter(&chat_ids_in_archive),
-            )
+            .transaction(|transaction| {
+                let mut stmt = transaction.prepare(
+                    "UPDATE msgs SET state=13 WHERE state=10 AND hidden=0 AND chat_id = ?",
+                )?;
+                for chat_id_in_archive in &chat_ids_in_archive {
+                    stmt.execute((chat_id_in_archive,))?;
+                }
+                Ok(())
+            })
             .await?;
+
         for chat_id_in_archive in chat_ids_in_archive {
+            start_chat_ephemeral_timers(context, chat_id_in_archive).await?;
             context.emit_event(EventType::MsgsNoticed(chat_id_in_archive));
             chatlist_events::emit_chatlist_item_changed(context, chat_id_in_archive);
         }
-    } else if context
-        .sql
-        .execute(
-            "UPDATE msgs
+    } else {
+        start_chat_ephemeral_timers(context, chat_id).await?;
+
+        if context
+            .sql
+            .execute(
+                "UPDATE msgs
             SET state=?
           WHERE state=?
             AND hidden=0
             AND chat_id=?;",
-            (MessageState::InNoticed, MessageState::InFresh, chat_id),
-        )
-        .await?
-        == 0
-    {
-        return Ok(());
+                (MessageState::InNoticed, MessageState::InFresh, chat_id),
+            )
+            .await?
+            == 0
+        {
+            return Ok(());
+        }
     }
 
     context.emit_event(EventType::MsgsNoticed(chat_id));
@@ -3364,6 +3384,7 @@ pub(crate) async fn mark_old_messages_as_noticed(
     }
 
     for c in changed_chats {
+        start_chat_ephemeral_timers(context, c).await?;
         context.emit_event(EventType::MsgsNoticed(c));
         chatlist_events::emit_chatlist_item_changed(context, c);
     }
@@ -3430,7 +3451,7 @@ pub async fn get_chat_contacts(context: &Context, chat_id: ChatId) -> Result<Vec
                FROM chats_contacts cc
                LEFT JOIN contacts c
                       ON c.id=cc.contact_id
-              WHERE cc.chat_id=?
+              WHERE cc.chat_id=? AND cc.add_timestamp >= cc.remove_timestamp
               ORDER BY c.id=1, c.last_seen DESC, c.id DESC;",
             (chat_id,),
             |row| row.get::<_, ContactId>(0),
@@ -3441,6 +3462,26 @@ pub async fn get_chat_contacts(context: &Context, chat_id: ChatId) -> Result<Vec
     Ok(list)
 }
 
+/// Returns a vector of contact IDs for given chat ID that are no longer part of the group.
+pub async fn get_past_chat_contacts(context: &Context, chat_id: ChatId) -> Result<Vec<ContactId>> {
+    let list = context
+        .sql
+        .query_map(
+            "SELECT cc.contact_id
+             FROM chats_contacts cc
+             LEFT JOIN contacts c
+                  ON c.id=cc.contact_id
+             WHERE cc.chat_id=? AND cc.add_timestamp < cc.remove_timestamp
+             ORDER BY c.id=1, c.last_seen DESC, c.id DESC",
+            (chat_id,),
+            |row| row.get::<_, ContactId>(0),
+            |ids| ids.collect::<Result<Vec<_>, _>>().map_err(Into::into),
+        )
+        .await?;
+
+    Ok(list)
+}
+
 /// Creates a group chat with a given `name`.
 pub async fn create_group_chat(
     context: &Context,
@@ -3464,9 +3505,7 @@ pub async fn create_group_chat(
         .await?;
 
     let chat_id = ChatId::new(u32::try_from(row_id)?);
-    if !is_contact_in_chat(context, chat_id, ContactId::SELF).await? {
-        add_to_chat_contacts_table(context, chat_id, &[ContactId::SELF]).await?;
-    }
+    add_to_chat_contacts_table(context, timestamp, chat_id, &[ContactId::SELF]).await?;
 
     context.emit_msgs_changed_without_ids();
     chatlist_events::emit_chatlist_changed(context);
@@ -3572,18 +3611,37 @@ pub(crate) async fn create_broadcast_list_ex(
 /// Set chat contacts in the `chats_contacts` table.
 pub(crate) async fn update_chat_contacts_table(
     context: &Context,
+    timestamp: i64,
     id: ChatId,
     contacts: &HashSet<ContactId>,
 ) -> Result<()> {
     context
         .sql
         .transaction(move |transaction| {
-            transaction.execute("DELETE FROM chats_contacts WHERE chat_id=?", (id,))?;
-            for contact_id in contacts {
-                transaction.execute(
-                    "INSERT INTO chats_contacts (chat_id, contact_id) VALUES(?, ?)",
-                    (id, contact_id),
+            // Bump `remove_timestamp` to at least `now`
+            // even for members from `contacts`.
+            // We add members from `contacts` back below.
+            transaction.execute(
+                "UPDATE chats_contacts
+                 SET remove_timestamp=MAX(add_timestamp+1, ?)
+                 WHERE chat_id=?",
+                (timestamp, id),
+            )?;
+
+            if !contacts.is_empty() {
+                let mut statement = transaction.prepare(
+                    "INSERT INTO chats_contacts (chat_id, contact_id, add_timestamp)
+                     VALUES                     (?1,      ?2,         ?3)
+                     ON CONFLICT (chat_id, contact_id)
+                     DO UPDATE SET add_timestamp=remove_timestamp",
                 )?;
+
+                for contact_id in contacts {
+                    // We bumped `add_timestamp` for existing rows above,
+                    // so on conflict it is enough to set `add_timestamp = remove_timestamp`
+                    // and this guarantees that `add_timestamp` is no less than `timestamp`.
+                    statement.execute((id, contact_id, timestamp))?;
+                }
             }
             Ok(())
         })
@@ -3594,17 +3652,21 @@ pub(crate) async fn update_chat_contacts_table(
 /// Adds contacts to the `chats_contacts` table.
 pub(crate) async fn add_to_chat_contacts_table(
     context: &Context,
+    timestamp: i64,
     chat_id: ChatId,
     contact_ids: &[ContactId],
 ) -> Result<()> {
     context
         .sql
         .transaction(move |transaction| {
+            let mut add_statement = transaction.prepare(
+                "INSERT INTO chats_contacts (chat_id, contact_id, add_timestamp) VALUES(?1, ?2, ?3)
+                 ON CONFLICT (chat_id, contact_id)
+                 DO UPDATE SET add_timestamp=MAX(remove_timestamp, ?3)",
+            )?;
+
             for contact_id in contact_ids {
-                transaction.execute(
-                    "INSERT OR IGNORE INTO chats_contacts (chat_id, contact_id) VALUES(?, ?)",
-                    (chat_id, contact_id),
-                )?;
+                add_statement.execute((chat_id, contact_id, timestamp))?;
             }
             Ok(())
         })
@@ -3613,17 +3675,21 @@ pub(crate) async fn add_to_chat_contacts_table(
     Ok(())
 }
 
-/// remove a contact from the chats_contact table
+/// Removes a contact from the chat
+/// by updating the `remove_timestamp`.
 pub(crate) async fn remove_from_chat_contacts_table(
     context: &Context,
     chat_id: ChatId,
     contact_id: ContactId,
 ) -> Result<()> {
+    let now = time();
     context
         .sql
         .execute(
-            "DELETE FROM chats_contacts WHERE chat_id=? AND contact_id=?",
-            (chat_id, contact_id),
+            "UPDATE chats_contacts
+             SET remove_timestamp=MAX(add_timestamp+1, ?)
+             WHERE chat_id=? AND contact_id=?",
+            (now, chat_id, contact_id),
         )
         .await?;
     Ok(())
@@ -3713,7 +3779,7 @@ pub(crate) async fn add_contact_to_chat_ex(
         if is_contact_in_chat(context, chat_id, contact_id).await? {
             return Ok(false);
         }
-        add_to_chat_contacts_table(context, chat_id, &[contact_id]).await?;
+        add_to_chat_contacts_table(context, time(), chat_id, &[contact_id]).await?;
     }
     if chat.typ == Chattype::Group && chat.is_promoted() {
         msg.viewtype = Viewtype::Text;
@@ -3723,10 +3789,8 @@ pub(crate) async fn add_contact_to_chat_ex(
         msg.param.set_cmd(SystemMessage::MemberAddedToGroup);
         msg.param.set(Param::Arg, contact_addr);
         msg.param.set_int(Param::Arg2, from_handshake.into());
-        if let Err(e) = send_msg(context, chat_id, &mut msg).await {
-            remove_from_chat_contacts_table(context, chat_id, contact_id).await?;
-            return Err(e);
-        }
+        send_msg(context, chat_id, &mut msg).await?;
+
         sync = Nosync;
         // TODO: Remove this compat code needed because Core <= v1.143:
         // - doesn't accept synchronization of QR code tokens for unpromoted groups, so we also send
@@ -3761,9 +3825,9 @@ pub(crate) async fn shall_attach_selfavatar(context: &Context, chat_id: ChatId)
         .sql
         .query_map(
             "SELECT c.selfavatar_sent
-           FROM chats_contacts cc
-           LEFT JOIN contacts c ON c.id=cc.contact_id
-          WHERE cc.chat_id=? AND cc.contact_id!=?;",
+             FROM chats_contacts cc
+             LEFT JOIN contacts c ON c.id=cc.contact_id
+             WHERE cc.chat_id=? AND cc.contact_id!=? AND cc.add_timestamp >= cc.remove_timestamp",
             (chat_id, ContactId::SELF),
             |row| Ok(row.get::<_, i64>(0)),
             |rows| {
@@ -3890,6 +3954,9 @@ pub async fn remove_contact_from_chat(
             bail!("{}", err_msg);
         } else {
             let mut sync = Nosync;
+
+            remove_from_chat_contacts_table(context, chat_id, contact_id).await?;
+
             // We do not return an error if the contact does not exist in the database.
             // This allows to delete dangling references to deleted contacts
             // in case of the database becoming inconsistent due to a bug.
@@ -3919,18 +3986,6 @@ pub async fn remove_contact_from_chat(
                     sync = Sync;
                 }
             }
-            // we remove the member from the chat after constructing the
-            // to-be-send message. If between send_msg() and here the
-            // process dies, the user will be able to redo the action. It's better than the other
-            // way round: you removed someone from DB but no peer or device gets to know about it
-            // and group membership is thus different on different devices. But if send_msg()
-            // failed, we still remove the member locally, otherwise it would be impossible to
-            // remove a member with missing key from a protected group.
-            // Note also that sending a message needs all recipients
-            // in order to correctly determine encryption so if we
-            // removed it first, it would complicate the
-            // check/encryption logic.
-            remove_from_chat_contacts_table(context, chat_id, contact_id).await?;
             context.emit_event(EventType::ChatModified(chat_id));
             if sync.into() {
                 chat.sync_contacts(context).await.log_err(context).ok();
@@ -4088,7 +4143,6 @@ pub async fn forward_msgs(context: &Context, msg_ids: &[MsgId], chat_id: ChatId)
     ensure!(!msg_ids.is_empty(), "empty msgs_ids: nothing to forward");
     ensure!(!chat_id.is_special(), "can not forward to special chat");
 
-    let mut created_chats: Vec<ChatId> = Vec::new();
     let mut created_msgs: Vec<MsgId> = Vec::new();
     let mut curr_timestamp: i64;
 
@@ -4100,20 +4154,17 @@ pub async fn forward_msgs(context: &Context, msg_ids: &[MsgId], chat_id: ChatId)
         bail!("cannot send to {}: {}", chat_id, reason);
     }
     curr_timestamp = create_smeared_timestamps(context, msg_ids.len());
-    let ids = context
-        .sql
-        .query_map(
-            &format!(
-                "SELECT id FROM msgs WHERE id IN({}) ORDER BY timestamp,id",
-                sql::repeat_vars(msg_ids.len())
-            ),
-            rusqlite::params_from_iter(msg_ids),
-            |row| row.get::<_, MsgId>(0),
-            |ids| ids.collect::<Result<Vec<_>, _>>().map_err(Into::into),
-        )
-        .await?;
-
-    for id in ids {
+    let mut msgs = Vec::with_capacity(msg_ids.len());
+    for id in msg_ids {
+        let ts: i64 = context
+            .sql
+            .query_get_value("SELECT timestamp FROM msgs WHERE id=?", (id,))
+            .await?
+            .context("No message {id}")?;
+        msgs.push((ts, *id));
+    }
+    msgs.sort_unstable();
+    for (_, id) in msgs {
         let src_msg_id: MsgId = id;
         let mut msg = Message::load_from_db(context, src_msg_id).await?;
         if msg.state == MessageState::OutDraft {
@@ -4150,11 +4201,10 @@ pub async fn forward_msgs(context: &Context, msg_ids: &[MsgId], chat_id: ChatId)
         if !create_send_msg_jobs(context, &mut msg).await?.is_empty() {
             context.scheduler.interrupt_smtp().await;
         }
-        created_chats.push(chat_id);
         created_msgs.push(new_msg_id);
     }
-    for (chat_id, msg_id) in created_chats.iter().zip(created_msgs.iter()) {
-        context.emit_msgs_changed(*chat_id, *msg_id);
+    for msg_id in created_msgs {
+        context.emit_msgs_changed(chat_id, msg_id);
     }
     Ok(())
 }
@@ -4444,7 +4494,7 @@ pub(crate) async fn delete_and_reset_all_device_msgs(context: &Context) -> Resul
 ///
 /// For example, it can be a message showing that a member was added to a group.
 /// Doesn't fail if the chat doesn't exist.
-#[expect(clippy::too_many_arguments)]
+#[allow(clippy::too_many_arguments)]
 pub(crate) async fn add_info_msg_with_cmd(
     context: &Context,
     chat_id: ChatId,
@@ -4550,7 +4600,7 @@ async fn set_contacts_by_addrs(context: &Context, id: ChatId, addrs: &[String])
     if contacts == contacts_old {
         return Ok(());
     }
-    update_chat_contacts_table(context, id, &contacts).await?;
+    update_chat_contacts_table(context, time(), id, &contacts).await?;
     context.emit_event(EventType::ChatModified(id));
     Ok(())
 }
@@ -4643,7 +4693,7 @@ impl Context {
     /// a noticed chat is archived. Emitting events should be cheap, a false-positive `MsgsChanged`
     /// is ok.
     pub(crate) fn on_archived_chats_maybe_noticed(&self) {
-        self.emit_msgs_changed(DC_CHAT_ID_ARCHIVED_LINK, MsgId::new(0));
+        self.emit_msgs_changed_without_msg_id(DC_CHAT_ID_ARCHIVED_LINK);
     }
 }
 
@@ -5058,11 +5108,11 @@ mod tests {
         bob.recv_msg(&alice_sent_add_msg).await;
 
         SystemTime::shift(Duration::from_secs(3600));
-        // This adds Bob because they left quite long ago.
+
+        // Alice sends a message to Bob because the message about leaving is lost.
         let alice_sent_msg = alice.send_text(alice_chat_id, "What a silence!").await;
         bob.recv_msg(&alice_sent_msg).await;
 
-        // Test that add message is rewritten.
         bob.golden_test_chat(bob_chat_id, "chat_test_parallel_member_remove")
             .await;
 
@@ -5082,9 +5132,9 @@ mod tests {
         Ok(())
     }
 
-    /// Test that if a message implicitly adds a member, both messages appear.
+    /// Test that member removal is synchronized eventually even if the message is lost.
     #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
-    async fn test_msg_with_implicit_member_add() -> Result<()> {
+    async fn test_msg_with_implicit_member_removed() -> Result<()> {
         let mut tcm = TestContextManager::new();
         let alice = tcm.alice().await;
         let bob = tcm.bob().await;
@@ -5100,22 +5150,35 @@ mod tests {
         let bob_received_msg = bob.recv_msg(&sent_msg).await;
         let bob_chat_id = bob_received_msg.get_chat_id();
         bob_chat_id.accept(&bob).await?;
+        assert_eq!(get_chat_contacts(&bob, bob_chat_id).await?.len(), 2);
 
         add_contact_to_chat(&alice, alice_chat_id, alice_fiona_contact_id).await?;
         let sent_msg = alice.pop_sent_msg().await;
         bob.recv_msg(&sent_msg).await;
+
+        // Bob removed Fiona, but the message is lost.
         remove_contact_from_chat(&bob, bob_chat_id, bob_fiona_contact_id).await?;
         bob.pop_sent_msg().await;
 
         // This doesn't add Fiona back because Bob just removed them.
         let sent_msg = alice.send_text(alice_chat_id, "Welcome, Fiona!").await;
         bob.recv_msg(&sent_msg).await;
+        assert_eq!(get_chat_contacts(&bob, bob_chat_id).await?.len(), 2);
 
+        // Even after some time Fiona is not added back.
         SystemTime::shift(Duration::from_secs(3600));
         let sent_msg = alice.send_text(alice_chat_id, "Welcome back, Fiona!").await;
         bob.recv_msg(&sent_msg).await;
-        bob.golden_test_chat(bob_chat_id, "chat_test_msg_with_implicit_member_add")
+        assert_eq!(get_chat_contacts(&bob, bob_chat_id).await?.len(), 2);
+
+        // If Bob sends a message to Alice now, Fiona is removed.
+        assert_eq!(get_chat_contacts(&alice, alice_chat_id).await?.len(), 3);
+        let sent_msg = bob
+            .send_text(alice_chat_id, "I have removed Fiona some time ago.")
             .await;
+        alice.recv_msg(&sent_msg).await;
+        assert_eq!(get_chat_contacts(&alice, alice_chat_id).await?.len(), 2);
+
         Ok(())
     }
 
@@ -5158,6 +5221,8 @@ mod tests {
         assert_eq!(a2_msg.get_info_type(), SystemMessage::MemberAddedToGroup);
         assert_eq!(get_chat_contacts(&a1, a1_chat_id).await?.len(), 2);
         assert_eq!(get_chat_contacts(&a2, a2_chat_id).await?.len(), 2);
+        assert_eq!(get_past_chat_contacts(&a1, a1_chat_id).await?.len(), 0);
+        assert_eq!(get_past_chat_contacts(&a2, a2_chat_id).await?.len(), 0);
 
         // rename the group
         set_chat_name(&a1, a1_chat_id, "bar").await?;
@@ -5190,6 +5255,8 @@ mod tests {
         );
         assert_eq!(get_chat_contacts(&a1, a1_chat_id).await?.len(), 1);
         assert_eq!(get_chat_contacts(&a2, a2_chat_id).await?.len(), 1);
+        assert_eq!(get_past_chat_contacts(&a1, a1_chat_id).await?.len(), 1);
+        assert_eq!(get_past_chat_contacts(&a2, a2_chat_id).await?.len(), 1);
 
         Ok(())
     }
@@ -5199,7 +5266,7 @@ mod tests {
         let _n = TimeShiftFalsePositiveNote;
 
         // Alice creates a group with Bob, Claire and Daisy and then removes Claire and Daisy
-        // (sleep() is needed as otherwise smeared time from Alice looks to Bob like messages from the future which are all set to "now" then)
+        // (time shift is needed as otherwise smeared time from Alice looks to Bob like messages from the future which are all set to "now" then)
         let alice = TestContext::new_alice().await;
 
         let bob_id = Contact::create(&alice, "", "bob@example.net").await?;
@@ -5214,17 +5281,17 @@ mod tests {
 
         add_contact_to_chat(&alice, alice_chat_id, claire_id).await?;
         let add2 = alice.pop_sent_msg().await;
-        tokio::time::sleep(std::time::Duration::from_millis(1100)).await;
+        SystemTime::shift(Duration::from_millis(1100));
 
         add_contact_to_chat(&alice, alice_chat_id, daisy_id).await?;
         let add3 = alice.pop_sent_msg().await;
-        tokio::time::sleep(std::time::Duration::from_millis(1100)).await;
+        SystemTime::shift(Duration::from_millis(1100));
 
         assert_eq!(get_chat_contacts(&alice, alice_chat_id).await?.len(), 4);
 
         remove_contact_from_chat(&alice, alice_chat_id, claire_id).await?;
         let remove1 = alice.pop_sent_msg().await;
-        tokio::time::sleep(std::time::Duration::from_millis(1100)).await;
+        SystemTime::shift(Duration::from_millis(1100));
 
         remove_contact_from_chat(&alice, alice_chat_id, daisy_id).await?;
         let remove2 = alice.pop_sent_msg().await;
@@ -5234,8 +5301,8 @@ mod tests {
         // Bob receives the add and deletion messages out of order
         let bob = TestContext::new_bob().await;
         bob.recv_msg(&add1).await;
-        bob.recv_msg(&add3).await;
-        let bob_chat_id = bob.recv_msg(&add2).await.chat_id;
+        let bob_chat_id = bob.recv_msg(&add3).await.chat_id;
+        bob.recv_msg_trash(&add2).await; // No-op addition message is trashed.
         assert_eq!(get_chat_contacts(&bob, bob_chat_id).await?.len(), 4);
 
         bob.recv_msg(&remove2).await;
@@ -5274,7 +5341,8 @@ mod tests {
     /// Test that group updates are robust to lost messages and eventual out of order arrival.
     #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
     async fn test_modify_chat_lost() -> Result<()> {
-        let alice = TestContext::new_alice().await;
+        let mut tcm = TestContextManager::new();
+        let alice = tcm.alice().await;
 
         let bob_id = Contact::create(&alice, "", "bob@example.net").await?;
         let claire_id = Contact::create(&alice, "", "claire@foo.de").await?;
@@ -5287,16 +5355,16 @@ mod tests {
 
         send_text_msg(&alice, alice_chat_id, "populate".to_string()).await?;
         let add = alice.pop_sent_msg().await;
-        tokio::time::sleep(std::time::Duration::from_millis(1100)).await;
+        SystemTime::shift(Duration::from_millis(1100));
 
         remove_contact_from_chat(&alice, alice_chat_id, claire_id).await?;
         let remove1 = alice.pop_sent_msg().await;
-        tokio::time::sleep(std::time::Duration::from_millis(1100)).await;
+        SystemTime::shift(Duration::from_millis(1100));
 
         remove_contact_from_chat(&alice, alice_chat_id, daisy_id).await?;
         let remove2 = alice.pop_sent_msg().await;
 
-        let bob = TestContext::new_bob().await;
+        let bob = tcm.bob().await;
 
         bob.recv_msg(&add).await;
         let bob_chat_id = bob.get_last_msg().await.chat_id;
@@ -5309,7 +5377,7 @@ mod tests {
 
         // Eventually, first removal message arrives.
         // This has no effect.
-        bob.recv_msg_trash(&remove1).await;
+        bob.recv_msg(&remove1).await;
         assert_eq!(get_chat_contacts(&bob, bob_chat_id).await?.len(), 2);
         Ok(())
     }
@@ -7711,6 +7779,121 @@ mod tests {
         self_chat.set_draft(&alice, Some(&mut msg)).await.unwrap();
         let draft2 = self_chat.get_draft(&alice).await?.unwrap();
         assert_eq!(draft1.timestamp_sort, draft2.timestamp_sort);
+
+        Ok(())
+    }
+
+    /// Test group consistency.
+    #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
+    async fn test_add_member_bug() -> Result<()> {
+        let mut tcm = TestContextManager::new();
+
+        let alice = &tcm.alice().await;
+        let bob = &tcm.bob().await;
+
+        let alice_bob_contact_id = Contact::create(alice, "Bob", "bob@example.net").await?;
+        let alice_fiona_contact_id = Contact::create(alice, "Fiona", "fiona@example.net").await?;
+
+        // Create a group.
+        let alice_chat_id =
+            create_group_chat(alice, ProtectionStatus::Unprotected, "Group chat").await?;
+        add_contact_to_chat(alice, alice_chat_id, alice_bob_contact_id).await?;
+        add_contact_to_chat(alice, alice_chat_id, alice_fiona_contact_id).await?;
+
+        // Promote the group.
+        let alice_sent_msg = alice
+            .send_text(alice_chat_id, "Hi! I created a group.")
+            .await;
+        let bob_received_msg = bob.recv_msg(&alice_sent_msg).await;
+
+        let bob_chat_id = bob_received_msg.get_chat_id();
+        bob_chat_id.accept(bob).await?;
+
+        // Alice removes Fiona from the chat.
+        remove_contact_from_chat(alice, alice_chat_id, alice_fiona_contact_id).await?;
+        let _alice_sent_add_msg = alice.pop_sent_msg().await;
+
+        SystemTime::shift(Duration::from_secs(3600));
+
+        // Bob sends a message
+        // to Alice and Fiona because he still has not received
+        // a message about Fiona being removed.
+        let bob_sent_msg = bob.send_text(bob_chat_id, "Hi Alice!").await;
+
+        // Alice receives a message.
+        // This should not add Fiona back.
+        let _alice_received_msg = alice.recv_msg(&bob_sent_msg).await;
+
+        assert_eq!(get_chat_contacts(alice, alice_chat_id).await?.len(), 2);
+
+        Ok(())
+    }
+
+    /// Test that tombstones for past members are added to chats_contacts table
+    /// even if the row did not exist before.
+    #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
+    async fn test_past_members() -> Result<()> {
+        let mut tcm = TestContextManager::new();
+
+        let alice = &tcm.alice().await;
+        let alice_fiona_contact_id = Contact::create(alice, "Fiona", "fiona@example.net").await?;
+
+        let alice_chat_id =
+            create_group_chat(alice, ProtectionStatus::Unprotected, "Group chat").await?;
+        add_contact_to_chat(alice, alice_chat_id, alice_fiona_contact_id).await?;
+        alice
+            .send_text(alice_chat_id, "Hi! I created a group.")
+            .await;
+        remove_contact_from_chat(alice, alice_chat_id, alice_fiona_contact_id).await?;
+        assert_eq!(get_past_chat_contacts(alice, alice_chat_id).await?.len(), 1);
+
+        let bob = &tcm.bob().await;
+        let bob_addr = bob.get_config(Config::Addr).await?.unwrap();
+        let alice_bob_contact_id = Contact::create(alice, "Bob", &bob_addr).await?;
+        add_contact_to_chat(alice, alice_chat_id, alice_bob_contact_id).await?;
+
+        let add_message = alice.pop_sent_msg().await;
+        let bob_add_message = bob.recv_msg(&add_message).await;
+        let bob_chat_id = bob_add_message.chat_id;
+        assert_eq!(get_chat_contacts(bob, bob_chat_id).await?.len(), 2);
+        assert_eq!(get_past_chat_contacts(bob, bob_chat_id).await?.len(), 1);
+
+        Ok(())
+    }
+
+    #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
+    async fn non_member_cannot_modify_member_list() -> Result<()> {
+        let mut tcm = TestContextManager::new();
+
+        let alice = &tcm.alice().await;
+        let bob = &tcm.bob().await;
+
+        let bob_addr = bob.get_config(Config::Addr).await?.unwrap();
+        let alice_bob_contact_id = Contact::create(alice, "Bob", &bob_addr).await?;
+
+        let alice_chat_id =
+            create_group_chat(alice, ProtectionStatus::Unprotected, "Group chat").await?;
+        add_contact_to_chat(alice, alice_chat_id, alice_bob_contact_id).await?;
+        let alice_sent_msg = alice
+            .send_text(alice_chat_id, "Hi! I created a group.")
+            .await;
+        let bob_received_msg = bob.recv_msg(&alice_sent_msg).await;
+        let bob_chat_id = bob_received_msg.get_chat_id();
+        bob_chat_id.accept(bob).await?;
+
+        let bob_fiona_contact_id = Contact::create(bob, "Fiona", "fiona@example.net").await?;
+
+        // Alice removes Bob and Bob adds Fiona at the same time.
+        remove_contact_from_chat(alice, alice_chat_id, alice_bob_contact_id).await?;
+        add_contact_to_chat(bob, bob_chat_id, bob_fiona_contact_id).await?;
+
+        let bob_sent_add_msg = bob.pop_sent_msg().await;
+
+        // Alice ignores Bob's message because Bob is not a member.
+        assert_eq!(get_chat_contacts(alice, alice_chat_id).await?.len(), 1);
+        alice.recv_msg_trash(&bob_sent_add_msg).await;
+        assert_eq!(get_chat_contacts(alice, alice_chat_id).await?.len(), 1);
+
         Ok(())
     }
 }

src/chatlist.rs

@@ -144,7 +144,7 @@ impl Chatlist {
                                   ORDER BY timestamp DESC, id DESC LIMIT 1)
                  WHERE c.id>9
                    AND c.blocked!=1
-                   AND c.id IN(SELECT chat_id FROM chats_contacts WHERE contact_id=?2)
+                   AND c.id IN(SELECT chat_id FROM chats_contacts WHERE contact_id=?2 AND add_timestamp >= remove_timestamp)
                  GROUP BY c.id
                  ORDER BY c.archived=?3 DESC, IFNULL(m.timestamp,c.created_timestamp) DESC, m.id DESC;",
                 (MessageState::OutDraft, query_contact_id, ChatVisibility::Pinned),
@@ -261,7 +261,7 @@ impl Chatlist {
                      WHERE c.id>9 AND c.id!=?
                        AND c.blocked=0
                        AND NOT c.archived=?
-                       AND (c.type!=? OR c.id IN(SELECT chat_id FROM chats_contacts WHERE contact_id=?))
+                       AND (c.type!=? OR c.id IN(SELECT chat_id FROM chats_contacts WHERE contact_id=? AND add_timestamp >= remove_timestamp))
                      GROUP BY c.id
                      ORDER BY c.id=? DESC, c.archived=? DESC, IFNULL(m.timestamp,c.created_timestamp) DESC, m.id DESC;",
                     (
@@ -550,7 +550,7 @@ mod tests {
         let chats = Chatlist::try_load(&t, 0, Some("is:unread"), None)
             .await
             .unwrap();
-        assert!(chats.len() == 1);
+        assert_eq!(chats.len(), 1);
 
         let chats = Chatlist::try_load(&t, DC_GCL_ARCHIVED_ONLY, None, None)
             .await
@@ -576,7 +576,7 @@ mod tests {
             .unwrap();
 
         let chats = Chatlist::try_load(&t, 0, None, None).await.unwrap();
-        assert!(chats.len() == 3);
+        assert_eq!(chats.len(), 3);
         assert!(!Chat::load_from_db(&t, chats.get_chat_id(0).unwrap())
             .await
             .unwrap()
@@ -585,7 +585,7 @@ mod tests {
         let chats = Chatlist::try_load(&t, DC_GCL_FOR_FORWARDING, None, None)
             .await
             .unwrap();
-        assert!(chats.len() == 2); // device chat cannot be written and is skipped on forwarding
+        assert_eq!(chats.len(), 2); // device chat cannot be written and is skipped on forwarding
         assert!(Chat::load_from_db(&t, chats.get_chat_id(0).unwrap())
             .await
             .unwrap()
@@ -597,7 +597,7 @@ mod tests {
         let chats = Chatlist::try_load(&t, DC_GCL_FOR_FORWARDING, None, None)
             .await
             .unwrap();
-        assert!(chats.len() == 1);
+        assert_eq!(chats.len(), 1);
     }
 
     #[tokio::test(flavor = "multi_thread", worker_threads = 2)]

src/config.rs

@@ -143,7 +143,7 @@ pub enum Config {
     /// Send BCC copy to self.
     ///
     /// Should be enabled for multidevice setups.
-    #[strum(props(default = "1"))]
+    /// Default is 0 for chatmail accounts before a backup export, 1 otherwise.
     BccSelf,
 
     /// True if encryption is preferred according to Autocrypt standard.
@@ -202,7 +202,7 @@ pub enum Config {
     /// Value 1 is treated as "delete at once": messages are deleted
     /// immediately, without moving to DeltaChat folder.
     ///
-    /// Default is 1 for chatmail accounts before a backup export, 0 otherwise.
+    /// Default is 1 for chatmail accounts without `BccSelf`, 0 otherwise.
     DeleteServerAfter,
 
     /// Timer in seconds after which the message is deleted from the
@@ -519,11 +519,19 @@ impl Context {
 
         // Default values
         let val = match key {
-            Config::ConfiguredInboxFolder => Some("INBOX"),
-            Config::DeleteServerAfter => match Box::pin(self.is_chatmail()).await? {
-                false => Some("0"),
-                true => Some("1"),
+            Config::BccSelf => match Box::pin(self.is_chatmail()).await? {
+                false => Some("1"),
+                true => Some("0"),
             },
+            Config::ConfiguredInboxFolder => Some("INBOX"),
+            Config::DeleteServerAfter => {
+                match !Box::pin(self.get_config_bool(Config::BccSelf)).await?
+                    && Box::pin(self.is_chatmail()).await?
+                {
+                    true => Some("1"),
+                    false => Some("0"),
+                }
+            }
             _ => key.get_str("default"),
         };
         Ok(val.map(|s| s.to_string()))
@@ -1105,6 +1113,28 @@ mod tests {
         Ok(())
     }
 
+    #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
+    async fn test_delete_server_after_default() -> Result<()> {
+        let t = &TestContext::new_alice().await;
+
+        // Check that the settings are displayed correctly.
+        assert_eq!(t.get_config(Config::BccSelf).await?, Some("1".to_string()));
+        assert_eq!(
+            t.get_config(Config::DeleteServerAfter).await?,
+            Some("0".to_string())
+        );
+
+        // Leaving emails on the server even w/o `BccSelf` is a good default at least because other
+        // MUAs do so even if the server doesn't save sent messages to some sentbox (like Gmail
+        // does).
+        t.set_config_bool(Config::BccSelf, false).await?;
+        assert_eq!(
+            t.get_config(Config::DeleteServerAfter).await?,
+            Some("0".to_string())
+        );
+        Ok(())
+    }
+
     #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
     async fn test_sync() -> Result<()> {
         let alice0 = TestContext::new_alice().await;

src/configure.rs

@@ -61,10 +61,7 @@ macro_rules! progress {
 impl Context {
     /// Checks if the context is already configured.
     pub async fn is_configured(&self) -> Result<bool> {
-        self.sql
-            .get_raw_config_bool("configured")
-            .await
-            .map_err(Into::into)
+        self.sql.get_raw_config_bool("configured").await
     }
 
     /// Configures this account with the currently set parameters.
@@ -452,8 +449,9 @@ async fn configure(ctx: &Context, param: &EnteredLoginParam) -> Result<Configure
     imap.configure_folders(ctx, &mut imap_session, create_mvbox)
         .await?;
 
+    let create = true;
     imap_session
-        .select_with_uidvalidity(ctx, "INBOX")
+        .select_with_uidvalidity(ctx, "INBOX", create)
         .await
         .context("could not read INBOX status")?;
 

src/contact.rs

@@ -1,7 +1,7 @@
 //! Contacts module
 
 use std::cmp::{min, Reverse};
-use std::collections::BinaryHeap;
+use std::collections::{BinaryHeap, HashSet};
 use std::fmt;
 use std::path::{Path, PathBuf};
 use std::time::UNIX_EPOCH;
@@ -34,7 +34,6 @@ use crate::message::MessageState;
 use crate::mimeparser::AvatarAction;
 use crate::param::{Param, Params};
 use crate::peerstate::Peerstate;
-use crate::sql::{self, params_iter};
 use crate::sync::{self, Sync::*};
 use crate::tools::{duration_to_str, get_abs_path, smeared_time, time, SystemTime};
 use crate::{chat, chatlist_events, stock_str};
@@ -114,7 +113,8 @@ impl ContactId {
                  SET gossiped_timestamp=0
                  WHERE EXISTS (SELECT 1 FROM chats_contacts
                                WHERE chats_contacts.chat_id=chats.id
-                               AND chats_contacts.contact_id=?)",
+                               AND chats_contacts.contact_id=?
+                               AND chats_contacts.add_timestamp >= chats_contacts.remove_timestamp)",
                 (self,),
             )
             .await?;
@@ -129,17 +129,14 @@ impl ContactId {
     ) -> Result<()> {
         context
             .sql
-            .execute(
-                &format!(
-                    "UPDATE contacts SET origin=? WHERE id IN ({}) AND origin<?",
-                    sql::repeat_vars(ids.len())
-                ),
-                rusqlite::params_from_iter(
-                    params_iter(&[origin])
-                        .chain(params_iter(ids))
-                        .chain(params_iter(&[origin])),
-                ),
-            )
+            .transaction(|transaction| {
+                let mut stmt = transaction
+                    .prepare("UPDATE contacts SET origin=?1 WHERE id = ?2 AND origin < ?1")?;
+                for id in ids {
+                    stmt.execute((origin, id))?;
+                }
+                Ok(())
+            })
             .await?;
         Ok(())
     }
@@ -805,6 +802,7 @@ impl Contact {
         }
 
         let mut name = sanitize_name(name);
+        #[allow(clippy::collapsible_if)]
         if origin <= Origin::OutgoingTo {
             // The user may accidentally have written to a "noreply" address with another MUA:
             if addr.contains("noreply")
@@ -1041,7 +1039,11 @@ impl Contact {
         listflags: u32,
         query: Option<&str>,
     ) -> Result<Vec<ContactId>> {
-        let self_addrs = context.get_all_self_addrs().await?;
+        let self_addrs = context
+            .get_all_self_addrs()
+            .await?
+            .into_iter()
+            .collect::<HashSet<_>>();
         let mut add_self = false;
         let mut ret = Vec::new();
         let flag_verified_only = (listflags & DC_GCL_VERIFIED_ONLY) != 0;
@@ -1056,29 +1058,32 @@ impl Contact {
             context
                 .sql
                 .query_map(
-                    &format!(
-                        "SELECT c.id FROM contacts c \
+                    "SELECT c.id, c.addr FROM contacts c
                  LEFT JOIN acpeerstates ps ON c.addr=ps.addr  \
-                 WHERE c.addr NOT IN ({})
-                 AND c.id>? \
+                 WHERE c.id>?
                  AND c.origin>=? \
                  AND c.blocked=0 \
                  AND (iif(c.name='',c.authname,c.name) LIKE ? OR c.addr LIKE ?) \
                  AND (1=? OR LENGTH(ps.verified_key_fingerprint)!=0)  \
                  ORDER BY c.last_seen DESC, c.id DESC;",
-                        sql::repeat_vars(self_addrs.len())
-                    ),
-                    rusqlite::params_from_iter(params_iter(&self_addrs).chain(params_slice![
+                    (
                         ContactId::LAST_SPECIAL,
                         minimal_origin,
-                        s3str_like_cmd,
-                        s3str_like_cmd,
-                        if flag_verified_only { 0i32 } else { 1i32 }
-                    ])),
-                    |row| row.get::<_, ContactId>(0),
-                    |ids| {
-                        for id in ids {
-                            ret.push(id?);
+                        &s3str_like_cmd,
+                        &s3str_like_cmd,
+                        if flag_verified_only { 0i32 } else { 1i32 },
+                    ),
+                    |row| {
+                        let id: ContactId = row.get(0)?;
+                        let addr: String = row.get(1)?;
+                        Ok((id, addr))
+                    },
+                    |rows| {
+                        for row in rows {
+                            let (id, addr) = row?;
+                            if !self_addrs.contains(&addr) {
+                                ret.push(id);
+                            }
                         }
                         Ok(())
                     },
@@ -1111,23 +1116,23 @@ impl Contact {
             context
                 .sql
                 .query_map(
-                    &format!(
-                        "SELECT id FROM contacts
-                 WHERE addr NOT IN ({})
-                 AND id>?
+                    "SELECT id, addr FROM contacts
+                 WHERE id>?
                  AND origin>=?
                  AND blocked=0
                  ORDER BY last_seen DESC, id DESC;",
-                        sql::repeat_vars(self_addrs.len())
-                    ),
-                    rusqlite::params_from_iter(
-                        params_iter(&self_addrs)
-                            .chain(params_slice![ContactId::LAST_SPECIAL, minimal_origin]),
-                    ),
-                    |row| row.get::<_, ContactId>(0),
-                    |ids| {
-                        for id in ids {
-                            ret.push(id?);
+                    (ContactId::LAST_SPECIAL, minimal_origin),
+                    |row| {
+                        let id: ContactId = row.get(0)?;
+                        let addr: String = row.get(1)?;
+                        Ok((id, addr))
+                    },
+                    |rows| {
+                        for row in rows {
+                            let (id, addr) = row?;
+                            if !self_addrs.contains(&addr) {
+                                ret.push(id);
+                            }
                         }
                         Ok(())
                     },

src/context.rs

@@ -643,14 +643,36 @@ impl Context {
     }
 
     /// Emits a MsgsChanged event with specified chat and message ids
+    ///
+    /// If IDs are unset, [`Self::emit_msgs_changed_without_ids`]
+    /// or [`Self::emit_msgs_changed_without_msg_id`] should be used
+    /// instead of this function.
     pub fn emit_msgs_changed(&self, chat_id: ChatId, msg_id: MsgId) {
+        debug_assert!(!chat_id.is_unset());
+        debug_assert!(!msg_id.is_unset());
+
         self.emit_event(EventType::MsgsChanged { chat_id, msg_id });
         chatlist_events::emit_chatlist_changed(self);
         chatlist_events::emit_chatlist_item_changed(self, chat_id);
     }
 
+    /// Emits a MsgsChanged event with specified chat and without message id.
+    pub fn emit_msgs_changed_without_msg_id(&self, chat_id: ChatId) {
+        debug_assert!(!chat_id.is_unset());
+
+        self.emit_event(EventType::MsgsChanged {
+            chat_id,
+            msg_id: MsgId::new(0),
+        });
+        chatlist_events::emit_chatlist_changed(self);
+        chatlist_events::emit_chatlist_item_changed(self, chat_id);
+    }
+
     /// Emits an IncomingMsg event with specified chat and message ids
     pub fn emit_incoming_msg(&self, chat_id: ChatId, msg_id: MsgId) {
+        debug_assert!(!chat_id.is_unset());
+        debug_assert!(!msg_id.is_unset());
+
         self.emit_event(EventType::IncomingMsg { chat_id, msg_id });
         chatlist_events::emit_chatlist_changed(self);
         chatlist_events::emit_chatlist_item_changed(self, chat_id);

src/download.rs

@@ -3,7 +3,7 @@
 use std::cmp::max;
 use std::collections::BTreeMap;
 
-use anyhow::{anyhow, bail, Result};
+use anyhow::{anyhow, bail, ensure, Result};
 use deltachat_derive::{FromSql, ToSql};
 use serde::{Deserialize, Serialize};
 
@@ -201,7 +201,11 @@ impl Session {
             bail!("Attempt to fetch UID 0");
         }
 
-        self.select_with_uidvalidity(context, folder).await?;
+        let create = false;
+        let folder_exists = self
+            .select_with_uidvalidity(context, folder, create)
+            .await?;
+        ensure!(folder_exists, "No folder {folder}");
 
         // we are connected, and the folder is selected
         info!(context, "Downloading message {}/{} fully...", folder, uid);

src/e2ee.rs

@@ -40,20 +40,17 @@ impl EncryptHelper {
 
     /// Determines if we can and should encrypt.
     ///
-    /// For encryption to be enabled, `e2ee_guaranteed` should be true, or strictly more than a half
-    /// of peerstates should prefer encryption. Own preference is counted equally to peer
-    /// preferences, even if message copy is not sent to self.
-    ///
     /// `e2ee_guaranteed` should be set to true for replies to encrypted messages (as required by
     /// Autocrypt Level 1, version 1.1) and for messages sent in protected groups.
     ///
     /// Returns an error if `e2ee_guaranteed` is true, but one or more keys are missing.
-    pub fn should_encrypt(
+    pub(crate) async fn should_encrypt(
         &self,
         context: &Context,
         e2ee_guaranteed: bool,
         peerstates: &[(Option<Peerstate>, String)],
     ) -> Result<bool> {
+        let is_chatmail = context.is_chatmail().await?;
         let mut prefer_encrypt_count = if self.prefer_encrypt == EncryptPreference::Mutual {
             1
         } else {
@@ -64,10 +61,15 @@ impl EncryptHelper {
                 Some(peerstate) => {
                     let prefer_encrypt = peerstate.prefer_encrypt;
                     info!(context, "Peerstate for {addr:?} is {prefer_encrypt}.");
-                    match peerstate.prefer_encrypt {
-                        EncryptPreference::NoPreference | EncryptPreference::Reset => {}
-                        EncryptPreference::Mutual => prefer_encrypt_count += 1,
-                    };
+                    if match peerstate.prefer_encrypt {
+                        EncryptPreference::NoPreference | EncryptPreference::Reset => {
+                            (peerstate.prefer_encrypt != EncryptPreference::Reset || is_chatmail)
+                                && self.prefer_encrypt == EncryptPreference::Mutual
+                        }
+                        EncryptPreference::Mutual => true,
+                    } {
+                        prefer_encrypt_count += 1;
+                    }
                 }
                 None => {
                     let msg = format!("Peerstate for {addr:?} missing, cannot encrypt");
@@ -170,9 +172,11 @@ pub async fn ensure_secret_key_exists(context: &Context) -> Result<()> {
 #[cfg(test)]
 mod tests {
     use super::*;
+    use crate::chat::send_text_msg;
     use crate::key::DcKey;
     use crate::message::{Message, Viewtype};
     use crate::param::Param;
+    use crate::receive_imf::receive_imf;
     use crate::test_utils::{bob_keypair, TestContext, TestContextManager};
 
     mod ensure_secret_key_exists {
@@ -320,29 +324,109 @@ Sent with my Delta Chat Messenger: https://delta.chat";
     }
 
     #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
-    async fn test_should_encrypt() {
+    async fn test_should_encrypt() -> Result<()> {
         let t = TestContext::new_alice().await;
+        assert!(t.get_config_bool(Config::E2eeEnabled).await?);
         let encrypt_helper = EncryptHelper::new(&t).await.unwrap();
 
-        // test with EncryptPreference::NoPreference:
-        // if e2ee_eguaranteed is unset, there is no encryption as not more than half of peers want encryption
         let ps = new_peerstates(EncryptPreference::NoPreference);
-        assert!(encrypt_helper.should_encrypt(&t, true, &ps).unwrap());
-        assert!(!encrypt_helper.should_encrypt(&t, false, &ps).unwrap());
+        assert!(encrypt_helper.should_encrypt(&t, true, &ps).await?);
+        // Own preference is `Mutual` and we have the peer's key.
+        assert!(encrypt_helper.should_encrypt(&t, false, &ps).await?);
 
-        // test with EncryptPreference::Reset
         let ps = new_peerstates(EncryptPreference::Reset);
-        assert!(encrypt_helper.should_encrypt(&t, true, &ps).unwrap());
-        assert!(!encrypt_helper.should_encrypt(&t, false, &ps).unwrap());
+        assert!(encrypt_helper.should_encrypt(&t, true, &ps).await?);
+        assert!(!encrypt_helper.should_encrypt(&t, false, &ps).await?);
 
-        // test with EncryptPreference::Mutual (self is also Mutual)
         let ps = new_peerstates(EncryptPreference::Mutual);
-        assert!(encrypt_helper.should_encrypt(&t, true, &ps).unwrap());
-        assert!(encrypt_helper.should_encrypt(&t, false, &ps).unwrap());
+        assert!(encrypt_helper.should_encrypt(&t, true, &ps).await?);
+        assert!(encrypt_helper.should_encrypt(&t, false, &ps).await?);
 
         // test with missing peerstate
         let ps = vec![(None, "bob@foo.bar".to_string())];
-        assert!(encrypt_helper.should_encrypt(&t, true, &ps).is_err());
-        assert!(!encrypt_helper.should_encrypt(&t, false, &ps).unwrap());
+        assert!(encrypt_helper.should_encrypt(&t, true, &ps).await.is_err());
+        assert!(!encrypt_helper.should_encrypt(&t, false, &ps).await?);
+        Ok(())
+    }
+
+    #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
+    async fn test_should_encrypt_e2ee_disabled() -> Result<()> {
+        let t = &TestContext::new_alice().await;
+        t.set_config_bool(Config::E2eeEnabled, false).await?;
+        let encrypt_helper = EncryptHelper::new(t).await.unwrap();
+
+        let ps = new_peerstates(EncryptPreference::NoPreference);
+        assert!(!encrypt_helper.should_encrypt(t, false, &ps).await?);
+
+        let ps = new_peerstates(EncryptPreference::Reset);
+        assert!(encrypt_helper.should_encrypt(t, true, &ps).await?);
+
+        let mut ps = new_peerstates(EncryptPreference::Mutual);
+        // Own preference is `NoPreference` and there's no majority with `Mutual`.
+        assert!(!encrypt_helper.should_encrypt(t, false, &ps).await?);
+        // Now the majority wants to encrypt. Let's encrypt, anyway there are other cases when we
+        // can't send unencrypted, e.g. protected groups.
+        ps.push(ps[0].clone());
+        assert!(encrypt_helper.should_encrypt(t, false, &ps).await?);
+
+        // Test with missing peerstate.
+        let ps = vec![(None, "bob@foo.bar".to_string())];
+        assert!(encrypt_helper.should_encrypt(t, true, &ps).await.is_err());
+        Ok(())
+    }
+
+    #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
+    async fn test_chatmail_prefers_to_encrypt() -> Result<()> {
+        let mut tcm = TestContextManager::new();
+        let alice = &tcm.alice().await;
+        let bob = &tcm.bob().await;
+        bob.set_config_bool(Config::IsChatmail, true).await?;
+
+        let bob_chat_id = tcm
+            .send_recv_accept(alice, bob, "Hello from DC")
+            .await
+            .chat_id;
+        receive_imf(
+            bob,
+            b"From: alice@example.org\n\
+            To: bob@example.net\n\
+            Message-ID: <2222@example.org>\n\
+            Date: Sun, 22 Mar 3000 22:37:58 +0000\n\
+            \n\
+            Hello from another MUA\n",
+            false,
+        )
+        .await?;
+        send_text_msg(bob, bob_chat_id, "hi".to_string()).await?;
+        let sent_msg = bob.pop_sent_msg().await;
+        let msg = Message::load_from_db(bob, sent_msg.sender_msg_id).await?;
+        assert!(msg.get_showpadlock());
+        Ok(())
+    }
+
+    #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
+    async fn test_chatmail_can_send_unencrypted() -> Result<()> {
+        let mut tcm = TestContextManager::new();
+        let bob = &tcm.bob().await;
+        bob.set_config_bool(Config::IsChatmail, true).await?;
+        let bob_chat_id = receive_imf(
+            bob,
+            b"From: alice@example.org\n\
+            To: bob@example.net\n\
+            Message-ID: <2222@example.org>\n\
+            Date: Sun, 22 Mar 3000 22:37:58 +0000\n\
+            \n\
+            Hello\n",
+            false,
+        )
+        .await?
+        .unwrap()
+        .chat_id;
+        bob_chat_id.accept(bob).await?;
+        send_text_msg(bob, bob_chat_id, "hi".to_string()).await?;
+        let sent_msg = bob.pop_sent_msg().await;
+        let msg = Message::load_from_db(bob, sent_msg.sender_msg_id).await?;
+        assert!(!msg.get_showpadlock());
+        Ok(())
     }
 }

src/ephemeral.rs

@@ -84,7 +84,6 @@ use crate::location;
 use crate::log::LogExt;
 use crate::message::{Message, MessageState, MsgId, Viewtype};
 use crate::mimeparser::SystemMessage;
-use crate::sql::{self, params_iter};
 use crate::stock_str;
 use crate::tools::{duration_to_str, time, SystemTime};
 
@@ -329,23 +328,44 @@ pub(crate) async fn start_ephemeral_timers_msgids(
     msg_ids: &[MsgId],
 ) -> Result<()> {
     let now = time();
-    let count = context
+    let should_interrupt =
+    context
+        .sql
+        .transaction(move |transaction| {
+            let mut should_interrupt = false;
+            let mut stmt =
+                transaction.prepare(
+                    "UPDATE msgs SET ephemeral_timestamp = ?1 + ephemeral_timer
+                     WHERE (ephemeral_timestamp == 0 OR ephemeral_timestamp > ?1 + ephemeral_timer) AND ephemeral_timer > 0
+                     AND id=?2")?;
+            for msg_id in msg_ids {
+                should_interrupt |= stmt.execute((now, msg_id))? > 0;
+            }
+            Ok(should_interrupt)
+        }).await?;
+    if should_interrupt {
+        context.scheduler.interrupt_ephemeral_task().await;
+    }
+    Ok(())
+}
+
+/// Starts ephemeral timer for all messages in the chat.
+///
+/// This should be called when chat is marked as noticed.
+pub(crate) async fn start_chat_ephemeral_timers(context: &Context, chat_id: ChatId) -> Result<()> {
+    let now = time();
+    let should_interrupt = context
         .sql
         .execute(
-            &format!(
-                "UPDATE msgs SET ephemeral_timestamp = ? + ephemeral_timer
-         WHERE (ephemeral_timestamp == 0 OR ephemeral_timestamp > ? + ephemeral_timer) AND ephemeral_timer > 0
-         AND id IN ({})",
-                sql::repeat_vars(msg_ids.len())
-            ),
-            rusqlite::params_from_iter(
-                std::iter::once(&now as &dyn crate::sql::ToSql)
-                    .chain(std::iter::once(&now as &dyn crate::sql::ToSql))
-                    .chain(params_iter(msg_ids)),
-            ),
+            "UPDATE msgs SET ephemeral_timestamp = ?1 + ephemeral_timer
+             WHERE chat_id = ?2
+             AND ephemeral_timer > 0
+             AND (ephemeral_timestamp == 0 OR ephemeral_timestamp > ?1 + ephemeral_timer)",
+            (now, chat_id),
         )
-        .await?;
-    if count > 0 {
+        .await?
+        > 0;
+    if should_interrupt {
         context.scheduler.interrupt_ephemeral_task().await;
     }
     Ok(())
@@ -482,7 +502,7 @@ pub(crate) async fn delete_expired_messages(context: &Context, now: i64) -> Resu
         }
 
         for modified_chat_id in modified_chat_ids {
-            context.emit_msgs_changed(modified_chat_id, MsgId::new(0));
+            context.emit_msgs_changed_without_msg_id(modified_chat_id);
         }
 
         for msg_id in webxdc_deleted {
@@ -695,7 +715,9 @@ pub(crate) async fn start_ephemeral_timers(context: &Context) -> Result<()> {
 #[cfg(test)]
 mod tests {
     use super::*;
+    use crate::chat::{marknoticed_chat, set_muted, ChatVisibility, MuteDuration};
     use crate::config::Config;
+    use crate::constants::DC_CHAT_ID_ARCHIVED_LINK;
     use crate::download::DownloadState;
     use crate::location;
     use crate::message::markseen_msgs;
@@ -1422,4 +1444,77 @@ mod tests {
 
         Ok(())
     }
+
+    /// Tests that ephemeral timer is started when the chat is noticed.
+    #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
+    async fn test_noticed_ephemeral_timer() -> Result<()> {
+        let mut tcm = TestContextManager::new();
+        let alice = &tcm.alice().await;
+        let bob = &tcm.bob().await;
+
+        let chat = alice.create_chat(bob).await;
+        let duration = 60;
+        chat.id
+            .set_ephemeral_timer(alice, Timer::Enabled { duration })
+            .await?;
+        let bob_received_message = tcm.send_recv(alice, bob, "Hello!").await;
+
+        marknoticed_chat(bob, bob_received_message.chat_id).await?;
+        SystemTime::shift(Duration::from_secs(100));
+
+        delete_expired_messages(bob, time()).await?;
+
+        assert!(Message::load_from_db_optional(bob, bob_received_message.id)
+            .await?
+            .is_none());
+        Ok(())
+    }
+
+    /// Tests that archiving the chat starts ephemeral timer.
+    #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
+    async fn test_archived_ephemeral_timer() -> Result<()> {
+        let mut tcm = TestContextManager::new();
+        let alice = &tcm.alice().await;
+        let bob = &tcm.bob().await;
+
+        let chat = alice.create_chat(bob).await;
+        let duration = 60;
+        chat.id
+            .set_ephemeral_timer(alice, Timer::Enabled { duration })
+            .await?;
+        let bob_received_message = tcm.send_recv(alice, bob, "Hello!").await;
+
+        bob_received_message
+            .chat_id
+            .set_visibility(bob, ChatVisibility::Archived)
+            .await?;
+        SystemTime::shift(Duration::from_secs(100));
+
+        delete_expired_messages(bob, time()).await?;
+
+        assert!(Message::load_from_db_optional(bob, bob_received_message.id)
+            .await?
+            .is_none());
+
+        // Bob mutes the chat so it is not unarchived.
+        set_muted(bob, bob_received_message.chat_id, MuteDuration::Forever).await?;
+
+        // Now test that for already archived chat
+        // timer is started if all archived chats are marked as noticed.
+        let bob_received_message_2 = tcm.send_recv(alice, bob, "Hello again!").await;
+        assert_eq!(bob_received_message_2.state, MessageState::InFresh);
+
+        marknoticed_chat(bob, DC_CHAT_ID_ARCHIVED_LINK).await?;
+        SystemTime::shift(Duration::from_secs(100));
+
+        delete_expired_messages(bob, time()).await?;
+
+        assert!(
+            Message::load_from_db_optional(bob, bob_received_message_2.id)
+                .await?
+                .is_none()
+        );
+
+        Ok(())
+    }
 }

src/events/payload.rs

@@ -109,6 +109,9 @@ pub enum EventType {
 
     /// A webxdc wants an info message or a changed summary to be notified.
     IncomingWebxdcNotify {
+        /// ID of the chat.
+        chat_id: ChatId,
+
         /// ID of the contact sending.
         contact_id: ContactId,
 

src/headerdef.rs

@@ -65,6 +65,15 @@ pub enum HeaderDef {
     ChatGroupMemberAdded,
     ChatContent,
 
+    /// Past members of the group.
+    ChatGroupPastMembers,
+
+    /// Space-separated timestamps of member addition
+    /// for members listed in the `To` field
+    /// followed by timestamps of member removal
+    /// for members listed in the `Chat-Group-Past-Members` field.
+    ChatGroupMemberTimestamps,
+
     /// Duration of the attached media file.
     ChatDuration,
 
@@ -73,6 +82,7 @@ pub enum HeaderDef {
 
     /// [Autocrypt](https://autocrypt.org/) header.
     Autocrypt,
+    AutocryptGossip,
     AutocryptSetupMessage,
     SecureJoin,
 

src/imap.rs

@@ -13,7 +13,7 @@ use std::{
     time::{Duration, UNIX_EPOCH},
 };
 
-use anyhow::{bail, format_err, Context as _, Result};
+use anyhow::{bail, ensure, format_err, Context as _, Result};
 use async_channel::Receiver;
 use async_imap::types::{Fetch, Flag, Name, NameAttribute, UnsolicitedResponse};
 use deltachat_contact_tools::ContactAddress;
@@ -46,7 +46,6 @@ use crate::receive_imf::{
     from_field_to_contact_id, get_prefetch_parent_message, receive_imf_inner, ReceivedMsg,
 };
 use crate::scheduler::connectivity::ConnectivityStore;
-use crate::sql;
 use crate::stock_str;
 use crate::tools::{self, create_id, duration_to_str};
 
@@ -541,10 +540,14 @@ impl Imap {
             return Ok(false);
         }
 
-        session
-            .select_with_uidvalidity(context, folder)
+        let create = false;
+        let folder_exists = session
+            .select_with_uidvalidity(context, folder, create)
             .await
             .with_context(|| format!("Failed to select folder {folder:?}"))?;
+        if !folder_exists {
+            return Ok(false);
+        }
 
         if !session.new_mail && !fetch_existing_msgs {
             info!(context, "No new emails in folder {folder:?}.");
@@ -836,45 +839,52 @@ impl Session {
         folder: &str,
         folder_meaning: FolderMeaning,
     ) -> Result<()> {
+        let uid_validity;
         // Collect pairs of UID and Message-ID.
         let mut msgs = BTreeMap::new();
 
-        self.select_with_uidvalidity(context, folder).await?;
+        let create = false;
+        let folder_exists = self
+            .select_with_uidvalidity(context, folder, create)
+            .await?;
+        if folder_exists {
+            let mut list = self
+                .uid_fetch("1:*", RFC724MID_UID)
+                .await
+                .with_context(|| format!("Can't resync folder {folder}"))?;
+            while let Some(fetch) = list.try_next().await? {
+                let headers = match get_fetch_headers(&fetch) {
+                    Ok(headers) => headers,
+                    Err(err) => {
+                        warn!(context, "Failed to parse FETCH headers: {}", err);
+                        continue;
+                    }
+                };
+                let message_id = prefetch_get_message_id(&headers);
 
-        let mut list = self
-            .uid_fetch("1:*", RFC724MID_UID)
-            .await
-            .with_context(|| format!("can't resync folder {folder}"))?;
-        while let Some(fetch) = list.try_next().await? {
-            let headers = match get_fetch_headers(&fetch) {
-                Ok(headers) => headers,
-                Err(err) => {
-                    warn!(context, "Failed to parse FETCH headers: {}", err);
-                    continue;
+                if let (Some(uid), Some(rfc724_mid)) = (fetch.uid, message_id) {
+                    msgs.insert(
+                        uid,
+                        (
+                            rfc724_mid,
+                            target_folder(context, folder, folder_meaning, &headers).await?,
+                        ),
+                    );
                 }
-            };
-            let message_id = prefetch_get_message_id(&headers);
-
-            if let (Some(uid), Some(rfc724_mid)) = (fetch.uid, message_id) {
-                msgs.insert(
-                    uid,
-                    (
-                        rfc724_mid,
-                        target_folder(context, folder, folder_meaning, &headers).await?,
-                    ),
-                );
             }
+
+            info!(
+                context,
+                "resync_folder_uids: Collected {} message IDs in {folder}.",
+                msgs.len(),
+            );
+
+            uid_validity = get_uidvalidity(context, folder).await?;
+        } else {
+            warn!(context, "resync_folder_uids: No folder {folder}.");
+            uid_validity = 0;
         }
 
-        info!(
-            context,
-            "Resync: collected {} message IDs in folder {}",
-            msgs.len(),
-            folder,
-        );
-
-        let uid_validity = get_uidvalidity(context, folder).await?;
-
         // Write collected UIDs to SQLite database.
         context
             .sql
@@ -911,15 +921,15 @@ impl Session {
             .await?;
         context
             .sql
-            .execute(
-                &format!(
-                    "DELETE FROM imap WHERE id IN ({})",
-                    sql::repeat_vars(row_ids.len())
-                ),
-                rusqlite::params_from_iter(row_ids),
-            )
+            .transaction(|transaction| {
+                let mut stmt = transaction.prepare("DELETE FROM imap WHERE id = ?")?;
+                for row_id in row_ids {
+                    stmt.execute((row_id,))?;
+                }
+                Ok(())
+            })
             .await
-            .context("cannot remove deleted messages from imap table")?;
+            .context("Cannot remove deleted messages from imap table")?;
 
         context.emit_event(EventType::ImapMessageDeleted(format!(
             "IMAP messages {uid_set} marked as deleted"
@@ -942,15 +952,15 @@ impl Session {
                     // Messages are moved or don't exist, IMAP returns OK response in both cases.
                     context
                         .sql
-                        .execute(
-                            &format!(
-                                "DELETE FROM imap WHERE id IN ({})",
-                                sql::repeat_vars(row_ids.len())
-                            ),
-                            rusqlite::params_from_iter(row_ids),
-                        )
+                        .transaction(|transaction| {
+                            let mut stmt = transaction.prepare("DELETE FROM imap WHERE id = ?")?;
+                            for row_id in row_ids {
+                                stmt.execute((row_id,))?;
+                            }
+                            Ok(())
+                        })
                         .await
-                        .context("cannot delete moved messages from imap table")?;
+                        .context("Cannot delete moved messages from imap table")?;
                     context.emit_event(EventType::ImapMessageMoved(format!(
                         "IMAP messages {set} moved to {target}"
                     )));
@@ -996,15 +1006,15 @@ impl Session {
         }
         context
             .sql
-            .execute(
-                &format!(
-                    "UPDATE imap SET target='' WHERE id IN ({})",
-                    sql::repeat_vars(row_ids.len())
-                ),
-                rusqlite::params_from_iter(row_ids),
-            )
+            .transaction(|transaction| {
+                let mut stmt = transaction.prepare("UPDATE imap SET target='' WHERE id = ?")?;
+                for row_id in row_ids {
+                    stmt.execute((row_id,))?;
+                }
+                Ok(())
+            })
             .await
-            .context("cannot plan deletion of messages")?;
+            .context("Cannot plan deletion of messages")?;
         if copy {
             context.emit_event(EventType::ImapMessageMoved(format!(
                 "IMAP messages {set} copied to {target}"
@@ -1040,7 +1050,11 @@ impl Session {
             // MOVE/DELETE operations. This does not result in multiple SELECT commands
             // being sent because `select_folder()` does nothing if the folder is already
             // selected.
-            self.select_with_uidvalidity(context, folder).await?;
+            let create = false;
+            let folder_exists = self
+                .select_with_uidvalidity(context, folder, create)
+                .await?;
+            ensure!(folder_exists, "No folder {folder}");
 
             // Empty target folder name means messages should be deleted.
             if target.is_empty() {
@@ -1134,29 +1148,40 @@ impl Session {
             .await?;
 
         for (folder, rowid_set, uid_set) in UidGrouper::from(rows) {
-            if let Err(err) = self.select_with_uidvalidity(context, &folder).await {
-                warn!(context, "store_seen_flags_on_imap: Failed to select {folder}, will retry later: {err:#}.");
+            let create = false;
+            let folder_exists = match self.select_with_uidvalidity(context, &folder, create).await {
+                Err(err) => {
+                    warn!(
+                        context,
+                        "store_seen_flags_on_imap: Failed to select {folder}, will retry later: {err:#}.");
+                    continue;
+                }
+                Ok(folder_exists) => folder_exists,
+            };
+            if !folder_exists {
+                warn!(context, "store_seen_flags_on_imap: No folder {folder}.");
             } else if let Err(err) = self.add_flag_finalized_with_set(&uid_set, "\\Seen").await {
                 warn!(
                     context,
                     "Cannot mark messages {uid_set} in {folder} as seen, will retry later: {err:#}.");
+                continue;
             } else {
                 info!(
                     context,
                     "Marked messages {} in folder {} as seen.", uid_set, folder
                 );
-                context
-                    .sql
-                    .execute(
-                        &format!(
-                            "DELETE FROM imap_markseen WHERE id IN ({})",
-                            sql::repeat_vars(rowid_set.len())
-                        ),
-                        rusqlite::params_from_iter(rowid_set),
-                    )
-                    .await
-                    .context("cannot remove messages marked as seen from imap_markseen table")?;
             }
+            context
+                .sql
+                .transaction(|transaction| {
+                    let mut stmt = transaction.prepare("DELETE FROM imap_markseen WHERE id = ?")?;
+                    for rowid in rowid_set {
+                        stmt.execute((rowid,))?;
+                    }
+                    Ok(())
+                })
+                .await
+                .context("Cannot remove messages marked as seen from imap_markseen table")?;
         }
 
         Ok(())
@@ -1172,9 +1197,14 @@ impl Session {
             return Ok(());
         }
 
-        self.select_with_uidvalidity(context, folder)
+        let create = false;
+        let folder_exists = self
+            .select_with_uidvalidity(context, folder, create)
             .await
-            .context("failed to select folder")?;
+            .context("Failed to select folder")?;
+        if !folder_exists {
+            return Ok(());
+        }
 
         let mailbox = self
             .selected_mailbox
@@ -1301,7 +1331,7 @@ impl Session {
     /// Returns the last UID fetched successfully and the info about each downloaded message.
     /// If the message is incorrect or there is a failure to write a message to the database,
     /// it is skipped and the error is logged.
-    #[expect(clippy::too_many_arguments)]
+    #[allow(clippy::too_many_arguments)]
     pub(crate) async fn fetch_many_msgs(
         &mut self,
         context: &Context,
@@ -1422,9 +1452,7 @@ impl Session {
 
                 let is_seen = fetch_response.flags().any(|flag| flag == Flag::Seen);
 
-                let rfc724_mid = if let Some(rfc724_mid) = uid_message_ids.get(&request_uid) {
-                    rfc724_mid
-                } else {
+                let Some(rfc724_mid) = uid_message_ids.get(&request_uid) else {
                     error!(
                         context,
                         "No Message-ID corresponding to UID {} passed in uid_messsage_ids.",
@@ -1561,10 +1589,8 @@ impl Session {
         };
 
         if self.can_metadata() && self.can_push() {
-            let device_token_changed = context
-                .get_config(Config::DeviceToken)
-                .await?
-                .map_or(true, |config_token| device_token != config_token);
+            let device_token_changed =
+                context.get_config(Config::DeviceToken).await?.as_ref() != Some(&device_token);
 
             if device_token_changed {
                 let folder = context
@@ -1630,7 +1656,7 @@ fn format_setmetadata(folder: &str, device_token: &str) -> String {
 impl Session {
     /// Returns success if we successfully set the flag or we otherwise
     /// think add_flag should not be retried: Disconnection during setting
-    /// the flag, or other imap-errors, returns true as well.
+    /// the flag, or other imap-errors, returns Ok as well.
     ///
     /// Returning error means that the operation can be retried.
     async fn add_flag_finalized_with_set(&mut self, uid_set: &str, flag: &str) -> Result<()> {
@@ -1677,7 +1703,11 @@ impl Session {
                 self.close().await?;
                 // Before moving emails to the mvbox we need to remember its UIDVALIDITY, otherwise
                 // emails moved before that wouldn't be fetched but considered "old" instead.
-                self.select_with_uidvalidity(context, folder).await?;
+                let create = false;
+                let folder_exists = self
+                    .select_with_uidvalidity(context, folder, create)
+                    .await?;
+                ensure!(folder_exists, "No MVBOX folder {:?}??", &folder);
                 return Ok(Some(folder));
             }
         }
@@ -1688,7 +1718,10 @@ impl Session {
         // Some servers require namespace-style folder names like "INBOX.DeltaChat", so we try all
         // the variants here.
         for folder in folders {
-            match self.select_with_uidvalidity(context, folder).await {
+            match self
+                .select_with_uidvalidity(context, folder, create_mvbox)
+                .await
+            {
                 Ok(_) => {
                     info!(context, "MVBOX-folder {} created.", folder);
                     return Ok(Some(folder));
@@ -2539,10 +2572,14 @@ async fn add_all_recipients_as_contacts(
         );
         return Ok(());
     };
-    session
-        .select_with_uidvalidity(context, &mailbox)
+    let create = false;
+    let folder_exists = session
+        .select_with_uidvalidity(context, &mailbox, create)
         .await
         .with_context(|| format!("could not select {mailbox}"))?;
+    if !folder_exists {
+        return Ok(());
+    }
 
     let recipients = session
         .get_all_recipients(context)
@@ -2689,6 +2726,7 @@ mod tests {
         }
     }
 
+    #[allow(clippy::too_many_arguments)]
     async fn check_target_folder_combination(
         folder: &str,
         mvbox_move: bool,

src/imap/idle.rs

@@ -29,7 +29,9 @@ impl Session {
     ) -> Result<Self> {
         use futures::future::FutureExt;
 
-        self.select_with_uidvalidity(context, folder).await?;
+        let create = true;
+        self.select_with_uidvalidity(context, folder, create)
+            .await?;
 
         if self.drain_unsolicited_responses(context)? {
             self.new_mail = true;

src/imap/scan_folders.rs

@@ -34,6 +34,7 @@ impl Imap {
         let watched_folders = get_watched_folders(context).await?;
 
         let mut folder_configs = BTreeMap::new();
+        let mut folder_names = Vec::new();
 
         for folder in folders {
             let folder_meaning = get_folder_meaning_by_attrs(folder.attributes());
@@ -44,6 +45,7 @@ impl Imap {
                 // already been moved and left it in the inbox.
                 continue;
             }
+            folder_names.push(folder.name().to_string());
             let folder_name_meaning = get_folder_meaning_by_name(folder.name());
 
             if let Some(config) = folder_meaning.to_config() {
@@ -91,6 +93,7 @@ impl Imap {
             }
         }
 
+        info!(context, "Found folders: {folder_names:?}.");
         last_scan.replace(tools::Time::now());
         Ok(true)
     }

src/imap/select_folder.rs

@@ -111,7 +111,8 @@ impl ImapSession {
         }
     }
 
-    /// Selects a folder and takes care of UIDVALIDITY changes.
+    /// Selects a folder optionally creating it and takes care of UIDVALIDITY changes. Returns false
+    /// iff `folder` doesn't exist.
     ///
     /// When selecting a folder for the first time, sets the uid_next to the current
     /// mailbox.uid_next so that no old emails are fetched.
@@ -123,11 +124,24 @@ impl ImapSession {
         &mut self,
         context: &Context,
         folder: &str,
-    ) -> Result<()> {
-        let newly_selected = self
-            .select_or_create_folder(context, folder)
-            .await
-            .with_context(|| format!("failed to select or create folder {folder}"))?;
+        create: bool,
+    ) -> Result<bool> {
+        let newly_selected = if create {
+            self.select_or_create_folder(context, folder)
+                .await
+                .with_context(|| format!("failed to select or create folder {folder}"))?
+        } else {
+            match self.select_folder(context, folder).await {
+                Ok(newly_selected) => newly_selected,
+                Err(err) => match err {
+                    Error::NoFolder(..) => return Ok(false),
+                    _ => {
+                        return Err(err)
+                            .with_context(|| format!("failed to select folder {folder}"))?
+                    }
+                },
+            }
+        };
         let mailbox = self
             .selected_mailbox
             .as_mut()
@@ -199,7 +213,7 @@ impl ImapSession {
                 }
             }
 
-            return Ok(());
+            return Ok(true);
         }
 
         // UIDVALIDITY is modified, reset highest seen MODSEQ.
@@ -233,7 +247,7 @@ impl ImapSession {
             old_uid_next,
             old_uid_validity,
         );
-        Ok(())
+        Ok(true)
     }
 }
 

src/imex.rs

@@ -416,7 +416,7 @@ async fn import_backup_stream_inner<R: tokio::io::AsyncRead + Unpin>(
             .context("cannot import unpacked database");
     }
     if res.is_ok() {
-        res = adjust_delete_server_after(context).await;
+        res = adjust_bcc_self(context).await;
     }
     fs::remove_file(unpacked_database)
         .await
@@ -796,7 +796,7 @@ async fn export_database(
         .to_str()
         .with_context(|| format!("path {} is not valid unicode", dest.display()))?;
 
-    adjust_delete_server_after(context).await?;
+    adjust_bcc_self(context).await?;
     context
         .sql
         .set_raw_config_int("backup_time", timestamp)
@@ -826,15 +826,14 @@ async fn export_database(
         .await
 }
 
-/// Sets `Config::DeleteServerAfter` to "never" if needed so that new messages are present on the
-/// server after a backup restoration or available for all devices in multi-device case.
-/// NB: Calling this after a backup import isn't reliable as we can crash in between, but this is a
-/// problem only for old backups, new backups already have `DeleteServerAfter` set if necessary.
-async fn adjust_delete_server_after(context: &Context) -> Result<()> {
-    if context.is_chatmail().await? && !context.config_exists(Config::DeleteServerAfter).await? {
-        context
-            .set_config(Config::DeleteServerAfter, Some("0"))
-            .await?;
+/// Sets `Config::BccSelf` (and `DeleteServerAfter` to "never" in effect) if needed so that new
+/// messages are present on the server after a backup restoration or available for all devices in
+/// multi-device case. NB: Calling this after a backup import isn't reliable as we can crash in
+/// between, but this is a problem only for old backups, new backups already have `BccSelf` set if
+/// necessary.
+async fn adjust_bcc_self(context: &Context) -> Result<()> {
+    if context.is_chatmail().await? && !context.config_exists(Config::BccSelf).await? {
+        context.set_config(Config::BccSelf, Some("1")).await?;
     }
     Ok(())
 }
@@ -1030,12 +1029,20 @@ mod tests {
 
         let context1 = &TestContext::new_alice().await;
 
-        // Check that the setting is displayed correctly.
+        // Check that the settings are displayed correctly.
+        assert_eq!(
+            context1.get_config(Config::BccSelf).await?,
+            Some("1".to_string())
+        );
         assert_eq!(
             context1.get_config(Config::DeleteServerAfter).await?,
             Some("0".to_string())
         );
         context1.set_config_bool(Config::IsChatmail, true).await?;
+        assert_eq!(
+            context1.get_config(Config::BccSelf).await?,
+            Some("0".to_string())
+        );
         assert_eq!(
             context1.get_config(Config::DeleteServerAfter).await?,
             Some("1".to_string())
@@ -1058,6 +1065,10 @@ mod tests {
         assert!(context2.is_configured().await?);
         assert!(context2.is_chatmail().await?);
         for ctx in [context1, context2] {
+            assert_eq!(
+                ctx.get_config(Config::BccSelf).await?,
+                Some("1".to_string())
+            );
             assert_eq!(
                 ctx.get_config(Config::DeleteServerAfter).await?,
                 Some("0".to_string())

src/imex/transfer.rs

@@ -178,6 +178,7 @@ impl BackupProvider {
         }
 
         info!(context, "Received valid backup authentication token.");
+        // Emit a nonzero progress so that UIs can display smth like "Transferring...".
         context.emit_event(EventType::ImexProgress(1));
 
         let blobdir = BlobDirContents::new(&context).await?;
@@ -309,6 +310,10 @@ pub async fn get_backup2(
     let mut file_size_buf = [0u8; 8];
     recv_stream.read_exact(&mut file_size_buf).await?;
     let file_size = u64::from_be_bytes(file_size_buf);
+    info!(context, "Received backup file size.");
+    // Emit a nonzero progress so that UIs can display smth like "Transferring...".
+    context.emit_event(EventType::ImexProgress(1));
+
     import_backup_stream(context, recv_stream, file_size, passphrase)
         .await
         .context("Failed to import backup from QUIC stream")?;
@@ -434,12 +439,14 @@ mod tests {
         assert!(msg.save_file(&ctx1, &path).await.is_err());
 
         // Check that both received the ImexProgress events.
-        ctx0.evtracker
-            .get_matching(|ev| matches!(ev, EventType::ImexProgress(1000)))
-            .await;
-        ctx1.evtracker
-            .get_matching(|ev| matches!(ev, EventType::ImexProgress(1000)))
-            .await;
+        for ctx in [&ctx0, &ctx1] {
+            ctx.evtracker
+                .get_matching(|ev| matches!(ev, EventType::ImexProgress(1)))
+                .await;
+            ctx.evtracker
+                .get_matching(|ev| matches!(ev, EventType::ImexProgress(1000)))
+                .await;
+        }
     }
 
     #[tokio::test(flavor = "multi_thread", worker_threads = 2)]

src/location.rs

@@ -707,9 +707,6 @@ pub(crate) async fn save(
                     ))?;
 
                     if timestamp > newest_timestamp {
-                        // okay to drop, as we use cached prepared statements
-                        drop(stmt_test);
-                        drop(stmt_insert);
                         newest_timestamp = timestamp;
                         newest_location_id = Some(u32::try_from(conn.last_insert_rowid())?);
                     }
@@ -1130,6 +1127,10 @@ Content-Disposition: attachment; filename="location.kml"
         assert_eq!(get_range(alice, None, None, 0, 0).await?.len(), 1);
         assert_eq!(get_range(bob, None, None, 0, 0).await?.len(), 1);
 
+        // Location-only messages are "auto-generated", but they mustn't make the contact a bot.
+        let contact = bob.add_or_lookup_contact(alice).await;
+        assert!(!contact.is_bot());
+
         // Day later Bob removes location.
         SystemTime::shift(Duration::from_secs(86400));
         delete_expired(alice, time()).await?;

src/message.rs

@@ -293,13 +293,7 @@ impl MsgId {
             ret += ", Location sent";
         }
 
-        let e2ee_errors = msg.param.get_int(Param::ErroneousE2ee).unwrap_or_default();
-
-        if 0 != e2ee_errors {
-            if 0 != e2ee_errors & 0x2 {
-                ret += ", Encrypted, no valid signature";
-            }
-        } else if 0 != msg.param.get_int(Param::GuaranteeE2ee).unwrap_or_default() {
+        if 0 != msg.param.get_int(Param::GuaranteeE2ee).unwrap_or_default() {
             ret += ", Encrypted";
         }
 
@@ -1117,7 +1111,9 @@ impl Message {
 
     /// Updates message state from the vCard attachment.
     pub(crate) async fn try_set_vcard(&mut self, context: &Context, path: &Path) -> Result<()> {
-        let vcard = fs::read(path).await.context("Could not read {path}")?;
+        let vcard = fs::read(path)
+            .await
+            .with_context(|| format!("Could not read {path:?}"))?;
         if let Some(summary) = get_vcard_summary(&vcard) {
             self.param.set(Param::Summary1, summary);
         } else {
@@ -1613,15 +1609,15 @@ pub async fn delete_msgs(context: &Context, msg_ids: &[MsgId]) -> Result<()> {
         modified_chat_ids.insert(msg.chat_id);
 
         let target = context.get_delete_msgs_target().await?;
-        let update_db = |conn: &mut rusqlite::Connection| {
-            conn.execute(
+        let update_db = |trans: &mut rusqlite::Transaction| {
+            trans.execute(
                 "UPDATE imap SET target=? WHERE rfc724_mid=?",
                 (target, msg.rfc724_mid),
             )?;
-            conn.execute("DELETE FROM smtp WHERE msg_id=?", (msg_id,))?;
+            trans.execute("DELETE FROM smtp WHERE msg_id=?", (msg_id,))?;
             Ok(())
         };
-        if let Err(e) = context.sql.call_write(update_db).await {
+        if let Err(e) = context.sql.transaction(update_db).await {
             error!(context, "delete_msgs: failed to update db: {e:#}.");
             res = Err(e);
             continue;
@@ -1643,7 +1639,7 @@ pub async fn delete_msgs(context: &Context, msg_ids: &[MsgId]) -> Result<()> {
     res?;
 
     for modified_chat_id in modified_chat_ids {
-        context.emit_msgs_changed(modified_chat_id, MsgId::new(0));
+        context.emit_msgs_changed_without_msg_id(modified_chat_id);
         chatlist_events::emit_chatlist_item_changed(context, modified_chat_id);
     }
 
@@ -1673,12 +1669,12 @@ pub async fn markseen_msgs(context: &Context, msg_ids: Vec<MsgId>) -> Result<()>
         .set_config_internal(Config::LastMsgId, Some(&last_msg_id.to_u32().to_string()))
         .await?;
 
-    let msgs = context
-        .sql
-        .query_map(
-            &format!(
+    let mut msgs = Vec::with_capacity(msg_ids.len());
+    for &id in &msg_ids {
+        if let Some(msg) = context
+            .sql
+            .query_row_optional(
                 "SELECT
-                    m.id AS id,
                     m.chat_id AS chat_id,
                     m.state AS state,
                     m.download_state as download_state,
@@ -1689,39 +1685,39 @@ pub async fn markseen_msgs(context: &Context, msg_ids: Vec<MsgId>) -> Result<()>
                     c.archived AS archived,
                     c.blocked AS blocked
                  FROM msgs m LEFT JOIN chats c ON c.id=m.chat_id
-                 WHERE m.id IN ({}) AND m.chat_id>9",
-                sql::repeat_vars(msg_ids.len())
-            ),
-            rusqlite::params_from_iter(&msg_ids),
-            |row| {
-                let id: MsgId = row.get("id")?;
-                let chat_id: ChatId = row.get("chat_id")?;
-                let state: MessageState = row.get("state")?;
-                let download_state: DownloadState = row.get("download_state")?;
-                let param: Params = row.get::<_, String>("param")?.parse().unwrap_or_default();
-                let from_id: ContactId = row.get("from_id")?;
-                let rfc724_mid: String = row.get("rfc724_mid")?;
-                let visibility: ChatVisibility = row.get("archived")?;
-                let blocked: Option<Blocked> = row.get("blocked")?;
-                let ephemeral_timer: EphemeralTimer = row.get("ephemeral_timer")?;
-                Ok((
-                    (
-                        id,
-                        chat_id,
-                        state,
-                        download_state,
-                        param,
-                        from_id,
-                        rfc724_mid,
-                        visibility,
-                        blocked.unwrap_or_default(),
-                    ),
-                    ephemeral_timer,
-                ))
-            },
-            |rows| rows.collect::<Result<Vec<_>, _>>().map_err(Into::into),
-        )
-        .await?;
+                 WHERE m.id=? AND m.chat_id>9",
+                (id,),
+                |row| {
+                    let chat_id: ChatId = row.get("chat_id")?;
+                    let state: MessageState = row.get("state")?;
+                    let download_state: DownloadState = row.get("download_state")?;
+                    let param: Params = row.get::<_, String>("param")?.parse().unwrap_or_default();
+                    let from_id: ContactId = row.get("from_id")?;
+                    let rfc724_mid: String = row.get("rfc724_mid")?;
+                    let visibility: ChatVisibility = row.get("archived")?;
+                    let blocked: Option<Blocked> = row.get("blocked")?;
+                    let ephemeral_timer: EphemeralTimer = row.get("ephemeral_timer")?;
+                    Ok((
+                        (
+                            id,
+                            chat_id,
+                            state,
+                            download_state,
+                            param,
+                            from_id,
+                            rfc724_mid,
+                            visibility,
+                            blocked.unwrap_or_default(),
+                        ),
+                        ephemeral_timer,
+                    ))
+                },
+            )
+            .await?
+        {
+            msgs.push(msg);
+        }
+    }
 
     if msgs
         .iter()
@@ -2094,6 +2090,9 @@ pub enum Viewtype {
     Gif = 21,
 
     /// Message containing a sticker, similar to image.
+    /// NB: When sending, the message viewtype may be changed to `Image` by some heuristics like
+    /// checking for transparent pixels. Use `Message::force_sticker()` to disable them.
+    ///
     /// If possible, the ui should display the image without borders in a transparent way.
     /// A click on a sticker will offer to install the sticker set in some future.
     Sticker = 23,
@@ -2710,6 +2709,29 @@ mod tests {
     async fn test_is_bot() -> Result<()> {
         let alice = TestContext::new_alice().await;
 
+        // Alice receives an auto-generated non-chat message.
+        //
+        // This could be a holiday notice,
+        // in which case the message should be marked as bot-generated,
+        // but the contact should not.
+        receive_imf(
+            &alice,
+            b"From: Claire <claire@example.com>\n\
+                    To: alice@example.org\n\
+                    Message-ID: <789@example.com>\n\
+                    Auto-Submitted: auto-generated\n\
+                    Date: Fri, 29 Jan 2021 21:37:55 +0000\n\
+                    \n\
+                    hello\n",
+            false,
+        )
+        .await?;
+        let msg = alice.get_last_msg().await;
+        assert_eq!(msg.get_text(), "hello".to_string());
+        assert!(msg.is_bot());
+        let contact = Contact::get_by_id(&alice, msg.from_id).await?;
+        assert!(!contact.is_bot());
+
         // Alice receives a message from Bob the bot.
         receive_imf(
             &alice,

src/mimefactory.rs

@@ -66,8 +66,36 @@ pub struct MimeFactory {
 
     selfstatus: String,
 
-    /// Vector of pairs of recipient name and address
-    recipients: Vec<(String, String)>,
+    /// Vector of actual recipient addresses.
+    ///
+    /// This is the list of addresses the message should be sent to.
+    /// It is not the same as the `To` header,
+    /// because in case of "member removed" message
+    /// removed member is in the recipient list,
+    /// but not in the `To` header.
+    /// In case of broadcast lists there are multiple recipients,
+    /// but the `To` header has no members.
+    ///
+    /// If `bcc_self` configuration is enabled,
+    /// this list will be extended with own address later,
+    /// but `MimeFactory` is not responsible for this.
+    recipients: Vec<String>,
+
+    /// Vector of pairs of recipient name and address that goes into the `To` field.
+    ///
+    /// The list of actual message recipient addresses may be different,
+    /// e.g. if members are hidden for broadcast lists.
+    to: Vec<(String, String)>,
+
+    /// Vector of pairs of past group member names and addresses.
+    past_members: Vec<(String, String)>,
+
+    /// Timestamps of the members in the same order as in the `recipients`
+    /// followed by `past_members`.
+    ///
+    /// If this is not empty, its length
+    /// should be the sum of `recipients` and `past_members` length.
+    member_timestamps: Vec<i64>,
 
     timestamp: i64,
     loaded: Loaded,
@@ -128,6 +156,7 @@ impl MimeFactory {
     pub async fn from_msg(context: &Context, msg: Message) -> Result<MimeFactory> {
         let chat = Chat::load_from_db(context, msg.chat_id).await?;
         let attach_profile_data = Self::should_attach_profile_data(&msg);
+        let undisclosed_recipients = chat.typ == Chattype::Broadcast;
 
         let from_addr = context.get_primary_self_addr().await?;
         let config_displayname = context
@@ -145,47 +174,101 @@ impl MimeFactory {
                 (name, None)
             };
 
-        let mut recipients = Vec::with_capacity(5);
+        let mut recipients = Vec::new();
+        let mut to = Vec::new();
+        let mut past_members = Vec::new();
+        let mut member_timestamps = Vec::new();
         let mut recipient_ids = HashSet::new();
         let mut req_mdn = false;
 
         if chat.is_self_talk() {
             if msg.param.get_cmd() == SystemMessage::AutocryptSetupMessage {
-                recipients.push((from_displayname.to_string(), from_addr.to_string()));
+                recipients.push(from_addr.to_string());
+                to.push((from_displayname.to_string(), from_addr.to_string()));
             }
         } else if chat.is_mailing_list() {
             let list_post = chat
                 .param
                 .get(Param::ListPost)
                 .context("Can't write to mailinglist without ListPost param")?;
-            recipients.push(("".to_string(), list_post.to_string()));
+            to.push(("".to_string(), list_post.to_string()));
+            recipients.push(list_post.to_string());
         } else {
+            let email_to_remove = if msg.param.get_cmd() == SystemMessage::MemberRemovedFromGroup {
+                msg.param.get(Param::Arg)
+            } else {
+                None
+            };
+
             context
                 .sql
                 .query_map(
-                    "SELECT c.authname, c.addr, c.id \
-                 FROM chats_contacts cc  \
-                 LEFT JOIN contacts c ON cc.contact_id=c.id  \
-                 WHERE cc.chat_id=? AND cc.contact_id>9;",
-                    (msg.chat_id,),
+                    "SELECT c.authname, c.addr, c.id, cc.add_timestamp, cc.remove_timestamp
+                     FROM chats_contacts cc
+                     LEFT JOIN contacts c ON cc.contact_id=c.id
+                     WHERE cc.chat_id=? AND cc.contact_id>9 OR (cc.contact_id=1 AND ?)",
+                    (msg.chat_id, chat.typ == Chattype::Group),
                     |row| {
                         let authname: String = row.get(0)?;
                         let addr: String = row.get(1)?;
                         let id: ContactId = row.get(2)?;
-                        Ok((authname, addr, id))
+                        let add_timestamp: i64 = row.get(3)?;
+                        let remove_timestamp: i64 = row.get(4)?;
+                        Ok((authname, addr, id, add_timestamp, remove_timestamp))
                     },
                     |rows| {
+                        let mut past_member_timestamps = Vec::new();
+
                         for row in rows {
-                            let (authname, addr, id) = row?;
-                            if !recipients_contain_addr(&recipients, &addr) {
-                                let name = match attach_profile_data {
-                                    true => authname,
-                                    false => "".to_string(),
-                                };
-                                recipients.push((name, addr));
+                            let (authname, addr, id, add_timestamp, remove_timestamp) = row?;
+                            let addr = if id == ContactId::SELF {
+                                from_addr.to_string()
+                            } else {
+                                addr
+                            };
+                            let name = match attach_profile_data {
+                                true => authname,
+                                false => "".to_string(),
+                            };
+                            if add_timestamp >= remove_timestamp {
+                                if !recipients_contain_addr(&to, &addr) {
+                                    recipients.push(addr.clone());
+                                    if !undisclosed_recipients {
+                                        to.push((name, addr));
+                                        member_timestamps.push(add_timestamp);
+                                    }
+                                }
+                                recipient_ids.insert(id);
+                            } else {
+                                // Row is a tombstone,
+                                // member is not actually part of the group.
+                                if !recipients_contain_addr(&past_members, &addr) {
+                                    if let Some(email_to_remove) = email_to_remove {
+                                        if email_to_remove == addr {
+                                            // This is a "member removed" message,
+                                            // we need to notify removed member
+                                            // that it was removed.
+                                            recipients.push(addr.clone());
+                                        }
+                                    }
+                                    if !undisclosed_recipients {
+                                        past_members.push((name, addr));
+                                        past_member_timestamps.push(remove_timestamp);
+                                    }
+                                }
                             }
-                            recipient_ids.insert(id);
                         }
+
+                        debug_assert!(member_timestamps.len() >= to.len());
+
+                        if to.len() > 1 {
+                            if let Some(position) = to.iter().position(|(_, x)| x == &from_addr) {
+                                to.remove(position);
+                                member_timestamps.remove(position);
+                            }
+                        }
+
+                        member_timestamps.extend(past_member_timestamps);
                         Ok(())
                     },
                 )
@@ -226,12 +309,19 @@ impl MimeFactory {
         };
         let attach_selfavatar = Self::should_attach_selfavatar(context, &msg).await;
 
+        debug_assert!(
+            member_timestamps.is_empty()
+                || to.len() + past_members.len() == member_timestamps.len()
+        );
         let factory = MimeFactory {
             from_addr,
             from_displayname,
             sender_displayname,
             selfstatus,
             recipients,
+            to,
+            past_members,
+            member_timestamps,
             timestamp: msg.timestamp_sort,
             loaded: Loaded::Message { msg, chat },
             in_reply_to,
@@ -259,7 +349,10 @@ impl MimeFactory {
             from_displayname: "".to_string(),
             sender_displayname: None,
             selfstatus: "".to_string(),
-            recipients: vec![("".to_string(), contact.get_addr().to_string())],
+            recipients: vec![contact.get_addr().to_string()],
+            to: vec![("".to_string(), contact.get_addr().to_string())],
+            past_members: vec![],
+            member_timestamps: vec![],
             timestamp,
             loaded: Loaded::Mdn {
                 rfc724_mid,
@@ -283,11 +376,7 @@ impl MimeFactory {
         let self_addr = context.get_primary_self_addr().await?;
 
         let mut res = Vec::new();
-        for (_, addr) in self
-            .recipients
-            .iter()
-            .filter(|(_, addr)| addr != &self_addr)
-        {
+        for addr in self.recipients.iter().filter(|&addr| *addr != self_addr) {
             res.push((Peerstate::from_addr(context, addr).await?, addr.clone()));
         }
 
@@ -475,10 +564,7 @@ impl MimeFactory {
     }
 
     pub fn recipients(&self) -> Vec<String> {
-        self.recipients
-            .iter()
-            .map(|(_, addr)| addr.clone())
-            .collect()
+        self.recipients.clone()
     }
 
     /// Consumes a `MimeFactory` and renders it into a message which is then stored in
@@ -488,46 +574,33 @@ impl MimeFactory {
 
         let from = new_address_with_name(&self.from_displayname, self.from_addr.clone());
 
-        let undisclosed_recipients = match &self.loaded {
-            Loaded::Message { chat, .. } => chat.typ == Chattype::Broadcast,
-            Loaded::Mdn { .. } => false,
-        };
-
         let mut to = Vec::new();
-        if undisclosed_recipients {
+        for (name, addr) in &self.to {
+            if name.is_empty() {
+                to.push(Address::new_mailbox(addr.clone()));
+            } else {
+                to.push(new_address_with_name(name, addr.clone()));
+            }
+        }
+
+        let mut past_members = Vec::new(); // Contents of `Chat-Group-Past-Members` header.
+        for (name, addr) in &self.past_members {
+            if name.is_empty() {
+                past_members.push(Address::new_mailbox(addr.clone()));
+            } else {
+                past_members.push(new_address_with_name(name, addr.clone()));
+            }
+        }
+
+        debug_assert!(
+            self.member_timestamps.is_empty()
+                || to.len() + past_members.len() == self.member_timestamps.len()
+        );
+        if to.is_empty() {
             to.push(Address::new_group(
                 "hidden-recipients".to_string(),
                 Vec::new(),
             ));
-        } else {
-            let email_to_remove = match &self.loaded {
-                Loaded::Message { msg, .. } => {
-                    if msg.param.get_cmd() == SystemMessage::MemberRemovedFromGroup {
-                        msg.param.get(Param::Arg)
-                    } else {
-                        None
-                    }
-                }
-                Loaded::Mdn { .. } => None,
-            };
-
-            for (name, addr) in &self.recipients {
-                if let Some(email_to_remove) = email_to_remove {
-                    if email_to_remove == addr {
-                        continue;
-                    }
-                }
-
-                if name.is_empty() {
-                    to.push(Address::new_mailbox(addr.clone()));
-                } else {
-                    to.push(new_address_with_name(name, addr.clone()));
-                }
-            }
-
-            if to.is_empty() {
-                to.push(from.clone());
-            }
         }
 
         // Start with Internet Message Format headers in the order of the standard example
@@ -540,6 +613,26 @@ impl MimeFactory {
             headers.push(Header::new_with_value("Sender".into(), vec![sender]).unwrap());
         }
         headers.push(Header::new_with_value("To".into(), to.clone()).unwrap());
+        if !past_members.is_empty() {
+            headers.push(
+                Header::new_with_value("Chat-Group-Past-Members".into(), past_members.clone())
+                    .unwrap(),
+            );
+        }
+
+        if !self.member_timestamps.is_empty() {
+            headers.push(
+                Header::new_with_value(
+                    "Chat-Group-Member-Timestamps".into(),
+                    self.member_timestamps
+                        .iter()
+                        .map(|ts| ts.to_string())
+                        .collect::<Vec<String>>()
+                        .join(" "),
+                )
+                .unwrap(),
+            );
+        }
 
         let subject_str = self.subject_str(context).await?;
         let encoded_subject = if subject_str
@@ -652,7 +745,9 @@ impl MimeFactory {
 
         let peerstates = self.peerstates_for_recipients(context).await?;
         let is_encrypted = !self.should_force_plaintext()
-            && encrypt_helper.should_encrypt(context, e2ee_guaranteed, &peerstates)?;
+            && encrypt_helper
+                .should_encrypt(context, e2ee_guaranteed, &peerstates)
+                .await?;
         let is_securejoin_message = if let Loaded::Message { msg, .. } = &self.loaded {
             msg.param.get_cmd() == SystemMessage::SecurejoinMessage
         } else {
@@ -1047,6 +1142,7 @@ impl MimeFactory {
         part.body(text)
     }
 
+    #[allow(clippy::cognitive_complexity)]
     async fn render_message(
         &mut self,
         context: &Context,
@@ -1368,7 +1464,7 @@ impl MimeFactory {
 
         // add attachment part
         if msg.viewtype.has_file() {
-            let (file_part, _) = build_body_file(context, &msg, "").await?;
+            let file_part = build_body_file(context, &msg).await?;
             parts.push(file_part);
         }
 
@@ -1508,11 +1604,7 @@ pub(crate) fn wrapped_base64_encode(buf: &[u8]) -> String {
         .join("\r\n")
 }
 
-async fn build_body_file(
-    context: &Context,
-    msg: &Message,
-    base_name: &str,
-) -> Result<(PartBuilder, String)> {
+async fn build_body_file(context: &Context, msg: &Message) -> Result<PartBuilder> {
     let blob = msg
         .param
         .get_blob(Param::File, context)
@@ -1538,17 +1630,13 @@ async fn build_body_file(
         ),
         Viewtype::Image | Viewtype::Gif => format!(
             "image_{}.{}",
-            if base_name.is_empty() {
-                chrono::Utc
-                    .timestamp_opt(msg.timestamp_sort, 0)
-                    .single()
-                    .map_or_else(
-                        || "YY-mm-dd_hh:mm:ss".to_string(),
-                        |ts| ts.format("%Y-%m-%d_%H-%M-%S").to_string(),
-                    )
-            } else {
-                base_name.to_string()
-            },
+            chrono::Utc
+                .timestamp_opt(msg.timestamp_sort, 0)
+                .single()
+                .map_or_else(
+                    || "YY-mm-dd_hh:mm:ss".to_string(),
+                    |ts| ts.format("%Y-%m-%d_%H-%M-%S").to_string(),
+                ),
             &suffix,
         ),
         Viewtype::Video => format!(
@@ -1600,7 +1688,7 @@ async fn build_body_file(
         .header(("Content-Transfer-Encoding", "base64"))
         .body(encoded_body);
 
-    Ok((mail, filename_to_send))
+    Ok(mail)
 }
 
 async fn build_avatar_file(context: &Context, path: &str) -> Result<String> {
@@ -2466,8 +2554,9 @@ mod tests {
         // Alice creates a group with Bob and Claire and then removes Bob.
         let alice = TestContext::new_alice().await;
 
+        let claire_addr = "claire@foo.de";
         let bob_id = Contact::create(&alice, "Bob", "bob@example.net").await?;
-        let claire_id = Contact::create(&alice, "Claire", "claire@foo.de").await?;
+        let claire_id = Contact::create(&alice, "Claire", claire_addr).await?;
 
         let alice_chat_id = create_group_chat(&alice, ProtectionStatus::Unprotected, "foo").await?;
         add_contact_to_chat(&alice, alice_chat_id, bob_id).await?;
@@ -2483,10 +2572,17 @@ mod tests {
             .get_first_header("To")
             .context("no To: header parsed")?;
         let to = addrparse_header(to)?;
-        let mailbox = to
-            .extract_single_info()
-            .context("to: field does not contain exactly one address")?;
-        assert_eq!(mailbox.addr, "bob@example.net");
+        for to_addr in to.iter() {
+            match to_addr {
+                mailparse::MailAddr::Single(ref info) => {
+                    // Addresses should be of existing members (Alice and Bob) and not Claire.
+                    assert_ne!(info.addr, claire_addr);
+                }
+                mailparse::MailAddr::Group(_) => {
+                    panic!("Group addresses are not expected here");
+                }
+            }
+        }
 
         Ok(())
     }

src/mimeparser.rs

@@ -35,6 +35,7 @@ use crate::param::{Param, Params};
 use crate::peerstate::Peerstate;
 use crate::simplify::{simplify, SimplifiedText};
 use crate::sync::SyncItems;
+use crate::tools::time;
 use crate::tools::{
     get_filemeta, parse_receive_headers, smeared_time, truncate_msg_text, validate_id,
 };
@@ -57,9 +58,14 @@ pub(crate) struct MimeMessage {
     /// Message headers.
     headers: HashMap<String, String>,
 
-    /// Addresses are normalized and lowercase
+    /// List of addresses from the `To` and `Cc` headers.
+    ///
+    /// Addresses are normalized and lowercase.
     pub recipients: Vec<SingleInfo>,
 
+    /// List of addresses from the `Chat-Group-Past-Members` header.
+    pub past_members: Vec<SingleInfo>,
+
     /// `From:` address.
     pub from: SingleInfo,
 
@@ -116,7 +122,15 @@ pub(crate) struct MimeMessage {
     /// Hop info for debugging.
     pub(crate) hop_info: String,
 
-    /// Whether the contact sending this should be marked as bot or non-bot.
+    /// Whether the message is auto-generated.
+    ///
+    /// If chat message (with `Chat-Version` header) is auto-generated,
+    /// the contact sending this should be marked as bot.
+    ///
+    /// If non-chat message is auto-generated,
+    /// it could be a holiday notice auto-reply,
+    /// in which case the message should be marked as bot-generated,
+    /// but the contact should not be.
     pub(crate) is_bot: Option<bool>,
 
     /// When the message was received, in secs since epoch.
@@ -224,6 +238,7 @@ impl MimeMessage {
 
         let mut headers = Default::default();
         let mut recipients = Default::default();
+        let mut past_members = Default::default();
         let mut from = Default::default();
         let mut list_post = Default::default();
         let mut chat_disposition_notification_to = None;
@@ -233,6 +248,7 @@ impl MimeMessage {
             context,
             &mut headers,
             &mut recipients,
+            &mut past_members,
             &mut from,
             &mut list_post,
             &mut chat_disposition_notification_to,
@@ -253,6 +269,7 @@ impl MimeMessage {
                         context,
                         &mut headers,
                         &mut recipients,
+                        &mut past_members,
                         &mut from,
                         &mut list_post,
                         &mut chat_disposition_notification_to,
@@ -430,6 +447,8 @@ impl MimeMessage {
                     HeaderDef::ChatGroupAvatar,
                     HeaderDef::ChatGroupMemberRemoved,
                     HeaderDef::ChatGroupMemberAdded,
+                    HeaderDef::ChatGroupMemberTimestamps,
+                    HeaderDef::ChatGroupPastMembers,
                 ] {
                     headers.remove(h.get_headername());
                 }
@@ -446,6 +465,7 @@ impl MimeMessage {
                 context,
                 &mut headers,
                 &mut recipients,
+                &mut past_members,
                 &mut inner_from,
                 &mut list_post,
                 &mut chat_disposition_notification_to,
@@ -503,6 +523,7 @@ impl MimeMessage {
             parts: Vec::new(),
             headers,
             recipients,
+            past_members,
             list_post,
             from,
             from_is_signed,
@@ -562,11 +583,14 @@ impl MimeMessage {
             },
         };
 
-        if parser.mdn_reports.is_empty() && parser.webxdc_status_update.is_none() {
-            // "Auto-Submitted" is also set by holiday-notices so we also check "chat-version".
-            let is_bot = parser.headers.get("auto-submitted")
-                == Some(&"auto-generated".to_string())
-                && parser.headers.contains_key("chat-version");
+        let is_location_only = parser.location_kml.is_some() && parser.parts.is_empty();
+        if parser.mdn_reports.is_empty()
+            && !is_location_only
+            && parser.sync_items.is_none()
+            && parser.webxdc_status_update.is_none()
+        {
+            let is_bot =
+                parser.headers.get("auto-submitted") == Some(&"auto-generated".to_string());
             parser.is_bot = Some(is_bot);
         }
         parser.maybe_remove_bad_parts();
@@ -1279,7 +1303,7 @@ impl MimeMessage {
         Ok(self.parts.len() > old_part_count)
     }
 
-    #[expect(clippy::too_many_arguments)]
+    #[allow(clippy::too_many_arguments)]
     async fn do_add_single_file_part(
         &mut self,
         context: &Context,
@@ -1519,10 +1543,12 @@ impl MimeMessage {
         }
     }
 
+    #[allow(clippy::too_many_arguments)]
     fn merge_headers(
         context: &Context,
         headers: &mut HashMap<String, String>,
         recipients: &mut Vec<SingleInfo>,
+        past_members: &mut Vec<SingleInfo>,
         from: &mut Option<SingleInfo>,
         list_post: &mut Option<String>,
         chat_disposition_notification_to: &mut Option<SingleInfo>,
@@ -1551,6 +1577,11 @@ impl MimeMessage {
         if !recipients_new.is_empty() {
             *recipients = recipients_new;
         }
+        let past_members_addresses =
+            get_all_addresses_from_header(fields, "chat-group-past-members");
+        if !past_members_addresses.is_empty() {
+            *past_members = past_members_addresses;
+        }
         let from_new = get_from(fields);
         if from_new.is_some() {
             *from = from_new;
@@ -1817,6 +1848,20 @@ impl MimeMessage {
         };
         Ok(parent_timestamp)
     }
+
+    /// Returns parsed `Chat-Group-Member-Timestamps` header contents.
+    ///
+    /// Returns `None` if there is no such header.
+    pub fn chat_group_member_timestamps(&self) -> Option<Vec<i64>> {
+        let now = time() + constants::TIMESTAMP_SENT_TOLERANCE;
+        self.get_header(HeaderDef::ChatGroupMemberTimestamps)
+            .map(|h| {
+                h.split_ascii_whitespace()
+                    .filter_map(|ts| ts.parse::<i64>().ok())
+                    .map(|ts| std::cmp::min(now, ts))
+                    .collect()
+            })
+    }
 }
 
 /// Parses `Autocrypt-Gossip` headers from the email and applies them to peerstates.

src/net/http.rs

@@ -151,7 +151,7 @@ async fn http_cache_put(context: &Context, url: &str, response: &Response) -> Re
 /// Also returns if the response is stale and should be revalidated in the background.
 async fn http_cache_get(context: &Context, url: &str) -> Result<Option<(Response, bool)>> {
     let now = time();
-    let Some((blob_name, mimetype, encoding, is_stale)) = context
+    let Some((blob_name, mimetype, encoding, stale_timestamp)) = context
         .sql
         .query_row_optional(
             "SELECT blobname, mimetype, encoding, stale
@@ -162,13 +162,14 @@ async fn http_cache_get(context: &Context, url: &str) -> Result<Option<(Response
                 let mimetype: Option<String> = Some(row.get(1)?).filter(|s: &String| !s.is_empty());
                 let encoding: Option<String> = Some(row.get(2)?).filter(|s: &String| !s.is_empty());
                 let stale_timestamp: i64 = row.get(3)?;
-                Ok((blob_name, mimetype, encoding, now > stale_timestamp))
+                Ok((blob_name, mimetype, encoding, stale_timestamp))
             },
         )
         .await?
     else {
         return Ok(None);
     };
+    let is_stale = now > stale_timestamp;
 
     let blob_object = BlobObject::from_name(context, blob_name)?;
     let blob_abs_path = blob_object.to_abs_path();
@@ -195,15 +196,16 @@ async fn http_cache_get(context: &Context, url: &str) -> Result<Option<(Response
     // Update expiration timestamp
     // to prevent deletion of the file still in use.
     //
-    // We do not update stale timestamp here
-    // as we have not revalidated the response.
-    // Stale timestamp is updated only
-    // when the URL is sucessfully fetched.
+    // If the response is stale, the caller should revalidate it in the background, so update
+    // `stale` timestamp to avoid revalidating too frequently (and have many parallel revalidation
+    // tasks) if revalidation fails or the HTTP request takes some time. The stale period >= 1 hour,
+    // so 1 more minute won't be a problem.
+    let stale_timestamp = if is_stale { now + 60 } else { stale_timestamp };
     context
         .sql
         .execute(
-            "UPDATE http_cache SET expires=? WHERE url=?",
-            (expires, url),
+            "UPDATE http_cache SET expires=?, stale=? WHERE url=?",
+            (expires, stale_timestamp, url),
         )
         .await?;
 
@@ -305,8 +307,6 @@ pub async fn read_url_blob(context: &Context, url: &str) -> Result<Response> {
                 }
             });
         }
-
-        // Return stale result.
         return Ok(response);
     }
 
@@ -495,6 +495,22 @@ mod tests {
         );
         assert_eq!(http_cache_get(t, xdc_pixel_url).await?, None);
 
+        // If we get the blob the second time quickly, it shouldn't be stale because it's supposed
+        // that we've already run a revalidation task which will update the blob soon.
+        assert_eq!(
+            http_cache_get(t, xdc_editor_url).await?,
+            Some((xdc_response.clone(), false))
+        );
+        // But if the revalidation task hasn't succeeded after some time, the blob is stale again
+        // even if we continue to get it frequently.
+        for i in (0..100).rev() {
+            SystemTime::shift(Duration::from_secs(6));
+            if let Some((_, true)) = http_cache_get(t, xdc_editor_url).await? {
+                break;
+            }
+            assert!(i > 0);
+        }
+
         // Test that if the file is accidentally removed from the blobdir,
         // there is no error when trying to load the cache entry.
         for entry in std::fs::read_dir(t.get_blobdir())? {

src/net/proxy.rs

@@ -640,6 +640,9 @@ mod tests {
     fn test_invalid_proxy_url() {
         assert!(ProxyConfig::from_url("foobar://127.0.0.1:9050").is_err());
         assert!(ProxyConfig::from_url("abc").is_err());
+
+        // This caused panic before shadowsocks 1.22.0.
+        assert!(ProxyConfig::from_url("ss://foo:bar@127.0.0.1:9999").is_err());
     }
 
     #[tokio::test(flavor = "multi_thread", worker_threads = 2)]

src/param.rs

@@ -55,6 +55,8 @@ pub enum Param {
 
     /// For Messages: decrypted with validation errors or without mutual set, if neither
     /// 'c' nor 'e' are preset, the messages is only transport encrypted.
+    ///
+    /// Deprecated on 2024-12-25.
     ErroneousE2ee = b'e',
 
     /// For Messages: force unencrypted message, a value from `ForcePlaintext` enum.
@@ -181,6 +183,8 @@ pub enum Param {
     GroupNameTimestamp = b'g',
 
     /// For Chats: timestamp of member list update.
+    ///
+    /// Deprecated 2025-01-07.
     MemberListTimestamp = b'k',
 
     /// For Webxdc Message Instances: Current document name
@@ -371,6 +375,7 @@ impl Params {
     /// Note that in the [ParamsFile::FsPath] case the blob can be
     /// created without copying if the path already refers to a valid
     /// blob.  If so a [BlobObject] will be returned.
+    #[allow(clippy::needless_lifetimes)]
     pub async fn get_blob<'a>(
         &self,
         key: Param,

src/peer_channels.rs

@@ -417,7 +417,6 @@ async fn get_iroh_gossip_peers(ctx: &Context, msg_id: MsgId) -> Result<Vec<NodeA
                     ))
                 })
                 .collect::<std::result::Result<Vec<_>, _>>()
-                .map_err(Into::into)
             },
         )
         .await

src/peerstate.rs

@@ -711,9 +711,25 @@ impl Peerstate {
                                 Origin::IncomingUnknownFrom,
                             )
                             .await?;
-                            chat::remove_from_chat_contacts_table(context, *chat_id, contact_id)
-                                .await?;
-                            chat::add_to_chat_contacts_table(context, *chat_id, &[new_contact_id])
+                            context
+                                .sql
+                                .transaction(|transaction| {
+                                    transaction.execute(
+                                        "UPDATE chats_contacts
+                                         SET remove_timestamp=MAX(add_timestamp+1, ?)
+                                         WHERE chat_id=? AND contact_id=?",
+                                        (timestamp, chat_id, contact_id),
+                                    )?;
+                                    transaction.execute(
+                                        "INSERT INTO chats_contacts
+                                         (chat_id, contact_id, add_timestamp)
+                                         VALUES (?1, ?2, ?3)
+                                         ON CONFLICT (chat_id, contact_id)
+                                         DO UPDATE SET add_timestamp=MAX(remove_timestamp, ?3)",
+                                        (chat_id, new_contact_id, timestamp),
+                                    )?;
+                                    Ok(())
+                                })
                                 .await?;
 
                             context.emit_event(EventType::ChatModified(*chat_id));

src/pgp.rs

@@ -21,9 +21,11 @@ use tokio::runtime::Handle;
 use crate::constants::KeyGenType;
 use crate::key::{DcKey, Fingerprint};
 
+#[allow(missing_docs)]
 #[cfg(test)]
 pub(crate) const HEADER_AUTOCRYPT: &str = "autocrypt-prefer-encrypt";
 
+#[allow(missing_docs)]
 pub const HEADER_SETUPCODE: &str = "passphrase-begin";
 
 /// Preferred symmetric encryption algorithm.

src/quota.rs

@@ -187,7 +187,7 @@ mod tests {
         Ok(())
     }
 
-    #[expect(clippy::assertions_on_constants)]
+    #[allow(clippy::assertions_on_constants)]
     #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
     async fn test_quota_thresholds() -> anyhow::Result<()> {
         assert!(QUOTA_ALLCLEAR_PERCENTAGE > 50);

src/receive_imf.rs

@@ -1,6 +1,7 @@
 //! Internet Message Format reception pipeline.
 
 use std::collections::HashSet;
+use std::iter;
 use std::str::FromStr;
 
 use anyhow::{Context as _, Result};
@@ -14,7 +15,7 @@ use regex::Regex;
 use crate::aheader::EncryptPreference;
 use crate::chat::{self, Chat, ChatId, ChatIdBlocked, ProtectionStatus};
 use crate::config::Config;
-use crate::constants::{self, Blocked, Chattype, ShowEmails, DC_CHAT_ID_TRASH};
+use crate::constants::{Blocked, Chattype, ShowEmails, DC_CHAT_ID_TRASH};
 use crate::contact::{Contact, ContactId, Origin};
 use crate::context::Context;
 use crate::debug_logging::maybe_set_logging_xdc_inner;
@@ -25,17 +26,16 @@ use crate::headerdef::{HeaderDef, HeaderDefMap};
 use crate::imap::{markseen_on_imap_table, GENERATED_PREFIX};
 use crate::log::LogExt;
 use crate::message::{
-    self, rfc724_mid_exists, rfc724_mid_exists_ex, Message, MessageState, MessengerMessage, MsgId,
-    Viewtype,
+    self, rfc724_mid_exists, Message, MessageState, MessengerMessage, MsgId, Viewtype,
 };
 use crate::mimeparser::{parse_message_ids, AvatarAction, MimeMessage, SystemMessage};
 use crate::param::{Param, Params};
 use crate::peer_channels::{add_gossip_peer_from_header, insert_topic_stub};
 use crate::peerstate::Peerstate;
 use crate::reaction::{set_msg_reaction, Reaction};
+use crate::rusqlite::OptionalExtension;
 use crate::securejoin::{self, handle_securejoin_handshake, observe_securejoin_on_other_device};
 use crate::simplify;
-use crate::sql::{self, params_iter};
 use crate::stock_str;
 use crate::sync::Sync::*;
 use crate::tools::{self, buf_compress, remove_subject_prefix};
@@ -158,7 +158,7 @@ async fn insert_tombstone(context: &Context, rfc724_mid: &str) -> Result<MsgId>
 /// If `is_partial_download` is set, it contains the full message size in bytes.
 /// Do not confuse that with `replace_msg_id` that will be set when the full message is loaded
 /// later.
-#[expect(clippy::too_many_arguments)]
+#[allow(clippy::too_many_arguments)]
 pub(crate) async fn receive_imf_inner(
     context: &Context,
     folder: &str,
@@ -345,6 +345,18 @@ pub(crate) async fn receive_imf_inner(
         },
     )
     .await?;
+    let past_ids = add_or_lookup_contacts_by_address_list(
+        context,
+        &mime_parser.past_members,
+        if !mime_parser.incoming {
+            Origin::OutgoingTo
+        } else if incoming_origin.is_known() {
+            Origin::IncomingTo
+        } else {
+            Origin::IncomingUnknownTo
+        },
+    )
+    .await?;
 
     update_verified_keys(context, &mut mime_parser, from_id).await?;
 
@@ -360,7 +372,7 @@ pub(crate) async fn receive_imf_inner(
             let contact = Contact::get_by_id(context, from_id).await?;
             mime_parser.peerstate = Peerstate::from_addr(context, contact.get_addr()).await?;
         } else {
-            let to_id = to_ids.first().copied().unwrap_or_default();
+            let to_id = to_ids.first().copied().unwrap_or(ContactId::SELF);
             // handshake may mark contacts as verified and must be processed before chats are created
             res = observe_securejoin_on_other_device(context, &mime_parser, to_id)
                 .await
@@ -418,6 +430,7 @@ pub(crate) async fn receive_imf_inner(
             &mut mime_parser,
             imf_raw,
             &to_ids,
+            &past_ids,
             rfc724_mid_orig,
             from_id,
             seen,
@@ -440,10 +453,10 @@ pub(crate) async fn receive_imf_inner(
     // and waste traffic.
     let chat_id = received_msg.chat_id;
     if !chat_id.is_special()
-        && mime_parser
-            .recipients
-            .iter()
-            .all(|recipient| mime_parser.gossiped_keys.contains_key(&recipient.addr))
+        && mime_parser.recipients.iter().all(|recipient| {
+            recipient.addr == mime_parser.from.addr
+                || mime_parser.gossiped_keys.contains_key(&recipient.addr)
+        })
     {
         info!(
             context,
@@ -607,7 +620,7 @@ pub(crate) async fn receive_imf_inner(
     }
 
     if let Some(replace_chat_id) = replace_chat_id {
-        context.emit_msgs_changed(replace_chat_id, MsgId::new(0));
+        context.emit_msgs_changed_without_msg_id(replace_chat_id);
     } else if !chat_id.is_trash() {
         let fresh = received_msg.state == MessageState::InFresh;
         for msg_id in &received_msg.msg_ids {
@@ -621,7 +634,11 @@ pub(crate) async fn receive_imf_inner(
         .await;
 
     if let Some(is_bot) = mime_parser.is_bot {
-        from_id.mark_bot(context, is_bot).await?;
+        // If the message is auto-generated and was generated by Delta Chat,
+        // mark the contact as a bot.
+        if mime_parser.get_header(HeaderDef::ChatVersion).is_some() {
+            from_id.mark_bot(context, is_bot).await?;
+        }
     }
 
     Ok(Some(received_msg))
@@ -679,12 +696,13 @@ pub async fn from_field_to_contact_id(
 /// Creates a `ReceivedMsg` from given parts which might consist of
 /// multiple messages (if there are multiple attachments).
 /// Every entry in `mime_parser.parts` produces a new row in the `msgs` table.
-#[expect(clippy::too_many_arguments)]
+#[allow(clippy::too_many_arguments, clippy::cognitive_complexity)]
 async fn add_parts(
     context: &Context,
     mime_parser: &mut MimeMessage,
     imf_raw: &[u8],
     to_ids: &[ContactId],
+    past_ids: &[ContactId],
     rfc724_mid: &str,
     from_id: ContactId,
     seen: bool,
@@ -774,6 +792,11 @@ async fn add_parts(
         }
     }
 
+    if chat_id.is_none() && is_mdn {
+        chat_id = Some(DC_CHAT_ID_TRASH);
+        info!(context, "Message is an MDN (TRASH).",);
+    }
+
     if mime_parser.incoming {
         to_id = ContactId::SELF;
 
@@ -785,11 +808,6 @@ async fn add_parts(
             markseen_on_imap_table(context, rfc724_mid).await.ok();
         }
 
-        if chat_id.is_none() && is_mdn {
-            chat_id = Some(DC_CHAT_ID_TRASH);
-            info!(context, "Message is an MDN (TRASH).",);
-        }
-
         let create_blocked_default = if is_bot {
             Blocked::Not
         } else {
@@ -832,6 +850,7 @@ async fn add_parts(
                         create_blocked,
                         from_id,
                         to_ids,
+                        past_ids,
                         &verified_encryption,
                         &grpid,
                     )
@@ -902,7 +921,7 @@ async fn add_parts(
                 group_chat_id,
                 from_id,
                 to_ids,
-                is_partial_download.is_some(),
+                past_ids,
                 &verified_encryption,
             )
             .await?;
@@ -942,14 +961,11 @@ async fn add_parts(
                 chat_id = Some(chat.id);
                 chat_id_blocked = chat.blocked;
             } else if allow_creation {
-                if let Ok(chat) = ChatIdBlocked::get_for_contact(context, from_id, create_blocked)
+                let chat = ChatIdBlocked::get_for_contact(context, from_id, create_blocked)
                     .await
-                    .context("Failed to get (new) chat for contact")
-                    .log_err(context)
-                {
-                    chat_id = Some(chat.id);
-                    chat_id_blocked = chat.blocked;
-                }
+                    .context("Failed to get (new) chat for contact")?;
+                chat_id = Some(chat.id);
+                chat_id_blocked = chat.blocked;
             }
 
             if let Some(chat_id) = chat_id {
@@ -973,7 +989,6 @@ async fn add_parts(
                 // the 1:1 chat accordingly.
                 let chat = match is_partial_download.is_none()
                     && mime_parser.get_header(HeaderDef::SecureJoin).is_none()
-                    && !is_mdn
                 {
                     true => Some(Chat::load_from_db(context, chat_id).await?)
                         .filter(|chat| chat.typ == Chattype::Single),
@@ -1034,9 +1049,13 @@ async fn add_parts(
         // the mail is on the IMAP server, probably it is also delivered.
         // We cannot recreate other states (read, error).
         state = MessageState::OutDelivered;
-        to_id = to_ids.first().copied().unwrap_or_default();
+        to_id = to_ids.first().copied().unwrap_or(ContactId::SELF);
 
-        let self_sent = to_ids.len() == 1 && to_ids.contains(&ContactId::SELF);
+        // Older Delta Chat versions with core <=1.152.2 only accepted
+        // self-sent messages in Saved Messages with own address in the `To` field.
+        // New Delta Chat versions may use empty `To` field
+        // with only a single `hidden-recipients` group in this case.
+        let self_sent = to_ids.len() <= 1 && to_id == ContactId::SELF;
 
         if mime_parser.sync_items.is_some() && self_sent {
             chat_id = Some(DC_CHAT_ID_TRASH);
@@ -1071,6 +1090,7 @@ async fn add_parts(
                         Blocked::Not,
                         from_id,
                         to_ids,
+                        past_ids,
                         &verified_encryption,
                         &grpid,
                     )
@@ -1142,9 +1162,8 @@ async fn add_parts(
                         chat_id = Some(id);
                         chat_id_blocked = blocked;
                     }
-                } else if let Ok(chat) =
-                    ChatIdBlocked::get_for_contact(context, to_id, Blocked::Not).await
-                {
+                } else {
+                    let chat = ChatIdBlocked::get_for_contact(context, to_id, Blocked::Not).await?;
                     chat_id = Some(chat.id);
                     chat_id_blocked = chat.blocked;
                 }
@@ -1160,7 +1179,7 @@ async fn add_parts(
             if chat_id_blocked != Blocked::Not {
                 if let Some(chat_id) = chat_id {
                     chat_id.unblock_ex(context, Nosync).await?;
-                    chat_id_blocked = Blocked::Not;
+                    // Not assigning `chat_id_blocked = Blocked::Not` to avoid unused_assignments warning.
                 }
             }
         }
@@ -1172,32 +1191,12 @@ async fn add_parts(
                 chat_id,
                 from_id,
                 to_ids,
-                is_partial_download.is_some(),
+                past_ids,
                 &verified_encryption,
             )
             .await?;
         }
 
-        if chat_id.is_none() && self_sent {
-            // from_id==to_id==ContactId::SELF - this is a self-sent messages,
-            // maybe an Autocrypt Setup Message
-            if let Ok(chat) = ChatIdBlocked::get_for_contact(context, ContactId::SELF, Blocked::Not)
-                .await
-                .context("Failed to get (new) chat for contact")
-                .log_err(context)
-            {
-                chat_id = Some(chat.id);
-                chat_id_blocked = chat.blocked;
-            }
-
-            if let Some(chat_id) = chat_id {
-                if Blocked::Not != chat_id_blocked {
-                    chat_id.unblock_ex(context, Nosync).await?;
-                    // Not assigning `chat_id_blocked = Blocked::Not` to avoid unused_assignments warning.
-                }
-            }
-        }
-
         if chat_id.is_none() {
             // Check if the message belongs to a broadcast list.
             if let Some(mailinglist_header) = mime_parser.get_mailinglist_header() {
@@ -1213,6 +1212,21 @@ async fn add_parts(
                 );
             }
         }
+
+        if chat_id.is_none() && self_sent {
+            // from_id==to_id==ContactId::SELF - this is a self-sent messages,
+            // maybe an Autocrypt Setup Message
+            let chat = ChatIdBlocked::get_for_contact(context, ContactId::SELF, Blocked::Not)
+                .await
+                .context("Failed to get (new) chat for contact")?;
+
+            chat_id = Some(chat.id);
+            // Not assigning `chat_id_blocked = chat.blocked` to avoid unused_assignments warning.
+
+            if Blocked::Not != chat.blocked {
+                chat.id.unblock_ex(context, Nosync).await?;
+            }
+        }
     }
 
     if fetching_existing_messages && mime_parser.decrypting_failed {
@@ -1232,7 +1246,7 @@ async fn add_parts(
     }
 
     let orig_chat_id = chat_id;
-    let mut chat_id = if is_mdn || is_reaction {
+    let mut chat_id = if is_reaction {
         DC_CHAT_ID_TRASH
     } else {
         chat_id.unwrap_or_else(|| {
@@ -1690,12 +1704,7 @@ RETURNING id
         "Message has {icnt} parts and is assigned to chat #{chat_id}."
     );
 
-    // new outgoing message from another device marks the chat as noticed.
-    if !mime_parser.incoming && !chat_id.is_special() {
-        chat::marknoticed_chat_if_older_than(context, chat_id, sort_timestamp).await?;
-    }
-
-    if !is_mdn {
+    if !chat_id.is_trash() {
         let mut chat = Chat::load_from_db(context, chat_id).await?;
 
         // In contrast to most other update-timestamps,
@@ -1834,7 +1843,7 @@ async fn lookup_chat_by_reply(
     Ok(Some((parent_chat.id, parent_chat.blocked)))
 }
 
-#[expect(clippy::too_many_arguments)]
+#[allow(clippy::too_many_arguments)]
 async fn lookup_chat_or_create_adhoc_group(
     context: &Context,
     mime_parser: &MimeMessage,
@@ -1878,34 +1887,44 @@ async fn lookup_chat_or_create_adhoc_group(
     if !contact_ids.contains(&from_id) {
         contact_ids.push(from_id);
     }
-    if let Some((chat_id, blocked)) = context
-        .sql
-        .query_row_optional(
-            &format!(
+    let trans_fn = |t: &mut rusqlite::Transaction| {
+        t.pragma_update(None, "query_only", "0")?;
+        t.execute(
+            "CREATE TEMP TABLE temp.contacts (
+                id INTEGER PRIMARY KEY
+            ) STRICT",
+            (),
+        )?;
+        let mut stmt = t.prepare("INSERT INTO temp.contacts(id) VALUES (?)")?;
+        for &id in &contact_ids {
+            stmt.execute((id,))?;
+        }
+        let val = t
+            .query_row(
                 "SELECT c.id, c.blocked
                 FROM chats c INNER JOIN msgs m ON c.id=m.chat_id
                 WHERE m.hidden=0 AND c.grpid='' AND c.name=?
                 AND (SELECT COUNT(*) FROM chats_contacts
-                    WHERE chat_id=c.id)=?
+                     WHERE chat_id=c.id
+                     AND add_timestamp >= remove_timestamp)=?
                 AND (SELECT COUNT(*) FROM chats_contacts
-                    WHERE chat_id=c.id
-                    AND contact_id NOT IN ({}))=0
+                     WHERE chat_id=c.id
+                     AND contact_id NOT IN (SELECT id FROM temp.contacts)
+                     AND add_timestamp >= remove_timestamp)=0
                 ORDER BY m.timestamp DESC",
-                sql::repeat_vars(contact_ids.len()),
-            ),
-            rusqlite::params_from_iter(
-                params_iter(&[&grpname])
-                    .chain(params_iter(&[contact_ids.len()]))
-                    .chain(params_iter(&contact_ids)),
-            ),
-            |row| {
-                let id: ChatId = row.get(0)?;
-                let blocked: Blocked = row.get(1)?;
-                Ok((id, blocked))
-            },
-        )
-        .await?
-    {
+                (&grpname, contact_ids.len()),
+                |row| {
+                    let id: ChatId = row.get(0)?;
+                    let blocked: Blocked = row.get(1)?;
+                    Ok((id, blocked))
+                },
+            )
+            .optional()?;
+        t.execute("DROP TABLE temp.contacts", ())?;
+        Ok(val)
+    };
+    let query_only = true;
+    if let Some((chat_id, blocked)) = context.sql.transaction_ex(query_only, trans_fn).await? {
         info!(
             context,
             "Assigning message to ad-hoc group {chat_id} with matching name and members."
@@ -1969,7 +1988,7 @@ async fn is_probably_private_reply(
 /// than two members, a new ad hoc group is created.
 ///
 /// On success the function returns the created (chat_id, chat_blocked) tuple.
-#[expect(clippy::too_many_arguments)]
+#[allow(clippy::too_many_arguments)]
 async fn create_group(
     context: &Context,
     mime_parser: &mut MimeMessage,
@@ -1977,6 +1996,7 @@ async fn create_group(
     create_blocked: Blocked,
     from_id: ContactId,
     to_ids: &[ContactId],
+    past_ids: &[ContactId],
     verified_encryption: &VerifiedEncryption,
     grpid: &str,
 ) -> Result<Option<(ChatId, Blocked)>> {
@@ -2050,14 +2070,37 @@ async fn create_group(
         chat_id_blocked = create_blocked;
 
         // Create initial member list.
-        let mut members = vec![ContactId::SELF];
-        if !from_id.is_special() {
-            members.push(from_id);
+        if let Some(mut chat_group_member_timestamps) = mime_parser.chat_group_member_timestamps() {
+            let mut new_to_ids = to_ids.to_vec();
+            if !new_to_ids.contains(&from_id) {
+                new_to_ids.insert(0, from_id);
+                chat_group_member_timestamps.insert(0, mime_parser.timestamp_sent);
+            }
+
+            update_chats_contacts_timestamps(
+                context,
+                new_chat_id,
+                None,
+                &new_to_ids,
+                past_ids,
+                &chat_group_member_timestamps,
+            )
+            .await?;
+        } else {
+            let mut members = vec![ContactId::SELF];
+            if !from_id.is_special() {
+                members.push(from_id);
+            }
+            members.extend(to_ids);
+
+            chat::add_to_chat_contacts_table(
+                context,
+                mime_parser.timestamp_sent,
+                new_chat_id,
+                &members,
+            )
+            .await?;
         }
-        members.extend(to_ids);
-        members.sort_unstable();
-        members.dedup();
-        chat::add_to_chat_contacts_table(context, new_chat_id, &members).await?;
 
         context.emit_event(EventType::ChatModified(new_chat_id));
         chatlist_events::emit_chatlist_changed(context);
@@ -2082,20 +2125,94 @@ async fn create_group(
     }
 }
 
+async fn update_chats_contacts_timestamps(
+    context: &Context,
+    chat_id: ChatId,
+    ignored_id: Option<ContactId>,
+    to_ids: &[ContactId],
+    past_ids: &[ContactId],
+    chat_group_member_timestamps: &[i64],
+) -> Result<bool> {
+    let expected_timestamps_count = to_ids.len() + past_ids.len();
+
+    if chat_group_member_timestamps.len() != expected_timestamps_count {
+        warn!(
+            context,
+            "Chat-Group-Member-Timestamps has wrong number of timestamps, got {}, expected {}.",
+            chat_group_member_timestamps.len(),
+            expected_timestamps_count
+        );
+        return Ok(false);
+    }
+
+    let mut modified = false;
+
+    context
+        .sql
+        .transaction(|transaction| {
+            let mut add_statement = transaction.prepare(
+                "INSERT INTO chats_contacts (chat_id, contact_id, add_timestamp)
+                 VALUES                     (?1,      ?2,         ?3)
+                 ON CONFLICT (chat_id, contact_id)
+                 DO
+                   UPDATE SET add_timestamp=?3
+                   WHERE ?3>add_timestamp AND ?3>=remove_timestamp",
+            )?;
+
+            for (contact_id, ts) in iter::zip(
+                to_ids.iter(),
+                chat_group_member_timestamps.iter().take(to_ids.len()),
+            ) {
+                if Some(*contact_id) != ignored_id {
+                    // It could be that member was already added,
+                    // but updated addition timestamp
+                    // is also a modification worth notifying about.
+                    modified |= add_statement.execute((chat_id, contact_id, ts))? > 0;
+                }
+            }
+
+            let mut remove_statement = transaction.prepare(
+                "INSERT INTO chats_contacts (chat_id, contact_id, remove_timestamp)
+                 VALUES                     (?1,      ?2,         ?3)
+                 ON CONFLICT (chat_id, contact_id)
+                 DO
+                   UPDATE SET remove_timestamp=?3
+                   WHERE ?3>remove_timestamp AND ?3>add_timestamp",
+            )?;
+
+            for (contact_id, ts) in iter::zip(
+                past_ids.iter(),
+                chat_group_member_timestamps.iter().skip(to_ids.len()),
+            ) {
+                // It could be that member was already removed,
+                // but updated removal timestamp
+                // is also a modification worth notifying about.
+                modified |= remove_statement.execute((chat_id, contact_id, ts))? > 0;
+            }
+
+            Ok(())
+        })
+        .await?;
+
+    Ok(modified)
+}
+
 /// Apply group member list, name, avatar and protection status changes from the MIME message.
 ///
 /// Returns `Vec` of group changes messages and, optionally, a better message to replace the
-/// original system message. If the better message is empty, the original system message should be
-/// just omitted.
+/// original system message. If the better message is empty, the original system message
+/// should be trashed.
 ///
-/// * `is_partial_download` - whether the message is not fully downloaded.
+/// * `to_ids` - contents of the `To` and `Cc` headers.
+/// * `past_ids` - contents of the `Chat-Group-Past-Members` header.
+#[allow(clippy::too_many_arguments)]
 async fn apply_group_changes(
     context: &Context,
     mime_parser: &mut MimeMessage,
     chat_id: ChatId,
     from_id: ContactId,
     to_ids: &[ContactId],
-    is_partial_download: bool,
+    past_ids: &[ContactId],
     verified_encryption: &VerifiedEncryption,
 ) -> Result<(Vec<String>, Option<String>)> {
     if chat_id.is_special() {
@@ -2124,49 +2241,6 @@ async fn apply_group_changes(
         HashSet::<ContactId>::from_iter(chat::get_chat_contacts(context, chat_id).await?);
     let is_from_in_chat =
         !chat_contacts.contains(&ContactId::SELF) || chat_contacts.contains(&from_id);
-    // Reject group membership changes from non-members and old changes.
-    let member_list_ts = match !is_partial_download && is_from_in_chat {
-        true => Some(chat_id.get_member_list_timestamp(context).await?),
-        false => None,
-    };
-    // When we remove a member locally, we shift `MemberListTimestamp` by `TIMESTAMP_SENT_TOLERANCE`
-    // into the future, so add some more tolerance here to allow remote membership changes as well.
-    let timestamp_sent_tolerance = constants::TIMESTAMP_SENT_TOLERANCE * 2;
-    let allow_member_list_changes = member_list_ts
-        .filter(|t| {
-            *t <= mime_parser
-                .timestamp_sent
-                .saturating_add(timestamp_sent_tolerance)
-        })
-        .is_some();
-    let sync_member_list = member_list_ts
-        .filter(|t| *t <= mime_parser.timestamp_sent)
-        .is_some();
-    // Whether to rebuild the member list from scratch.
-    let recreate_member_list = {
-        // Always recreate membership list if SELF has been added. The older versions of DC
-        // don't always set "In-Reply-To" to the latest message they sent, but to the latest
-        // delivered message (so it's a race), so we have this heuristic here.
-        self_added
-            || match mime_parser.get_header(HeaderDef::InReplyTo) {
-                // If we don't know the referenced message, we missed some messages.
-                // Maybe they added/removed members, so we need to recreate our member list.
-                Some(reply_to) => rfc724_mid_exists_ex(context, reply_to, "download_state=0")
-                    .await?
-                    .filter(|(_, _, downloaded)| *downloaded)
-                    .is_none(),
-                None => false,
-            }
-    } && (
-        // Don't allow the timestamp tolerance here for more reliable leaving of groups.
-        sync_member_list || {
-            info!(
-                context,
-                "Ignoring a try to recreate member list of {chat_id} by {from_id}.",
-            );
-            false
-        }
-    );
 
     if mime_parser.get_header(HeaderDef::ChatVerified).is_some() {
         if let VerifiedEncryption::NotVerified(err) = verified_encryption {
@@ -2190,44 +2264,24 @@ async fn apply_group_changes(
     if let Some(removed_addr) = mime_parser.get_header(HeaderDef::ChatGroupMemberRemoved) {
         removed_id = Contact::lookup_id_by_addr(context, removed_addr, Origin::Unknown).await?;
         if let Some(id) = removed_id {
-            if allow_member_list_changes && chat_contacts.contains(&id) {
-                better_msg = if id == from_id {
-                    Some(stock_str::msg_group_left_local(context, from_id).await)
-                } else {
-                    Some(stock_str::msg_del_member_local(context, removed_addr, from_id).await)
-                };
-            }
+            better_msg = if id == from_id {
+                Some(stock_str::msg_group_left_local(context, from_id).await)
+            } else {
+                Some(stock_str::msg_del_member_local(context, removed_addr, from_id).await)
+            };
         } else {
             warn!(context, "Removed {removed_addr:?} has no contact id.")
         }
-        better_msg.get_or_insert_with(Default::default);
-        if !allow_member_list_changes {
-            info!(
-                context,
-                "Ignoring removal of {removed_addr:?} from {chat_id}."
-            );
-        }
     } else if let Some(added_addr) = mime_parser.get_header(HeaderDef::ChatGroupMemberAdded) {
-        if allow_member_list_changes {
-            let is_new_member;
-            if let Some(contact_id) =
-                Contact::lookup_id_by_addr(context, added_addr, Origin::Unknown).await?
-            {
-                added_id = Some(contact_id);
-                is_new_member = !chat_contacts.contains(&contact_id);
-            } else {
-                warn!(context, "Added {added_addr:?} has no contact id.");
-                is_new_member = false;
-            }
-
-            if is_new_member || self_added {
-                better_msg =
-                    Some(stock_str::msg_add_member_local(context, added_addr, from_id).await);
-            }
+        if let Some(contact_id) =
+            Contact::lookup_id_by_addr(context, added_addr, Origin::Unknown).await?
+        {
+            added_id = Some(contact_id);
         } else {
-            info!(context, "Ignoring addition of {added_addr:?} to {chat_id}.");
+            warn!(context, "Added {added_addr:?} has no contact id.");
         }
-        better_msg.get_or_insert_with(Default::default);
+
+        better_msg = Some(stock_str::msg_add_member_local(context, added_addr, from_id).await);
     } else if let Some(old_name) = mime_parser
         .get_header(HeaderDef::ChatGroupNameChanged)
         .map(|s| s.trim())
@@ -2274,113 +2328,114 @@ async fn apply_group_changes(
         }
     }
 
-    if allow_member_list_changes {
-        let mut new_members = HashSet::from_iter(to_ids.iter().copied());
-        new_members.insert(ContactId::SELF);
-        if !from_id.is_special() {
-            new_members.insert(from_id);
-        }
+    // These are for adding info messages about implicit membership changes, so they are only
+    // filled when such messages are needed.
+    let mut added_ids = HashSet::<ContactId>::new();
+    let mut removed_ids = HashSet::<ContactId>::new();
 
-        // These are for adding info messages about implicit membership changes, so they are only
-        // filled when such messages are needed.
-        let mut added_ids = HashSet::<ContactId>::new();
-        let mut removed_ids = HashSet::<ContactId>::new();
-
-        if !recreate_member_list {
-            if sync_member_list {
-                added_ids = new_members.difference(&chat_contacts).copied().collect();
-            } else if let Some(added_id) = added_id {
-                added_ids.insert(added_id);
-            }
-            new_members.clone_from(&chat_contacts);
-            // Don't delete any members locally, but instead add absent ones to provide group
-            // membership consistency for all members:
-            // - Classical MUA users usually don't intend to remove users from an email thread, so
-            //   if they removed a recipient then it was probably by accident.
-            // - DC users could miss new member additions and then better to handle this in the same
-            //   way as for classical MUA messages. Moreover, if we remove a member implicitly, they
-            //   will never know that and continue to think they're still here.
-            // But it shouldn't be a big problem if somebody missed a member removal, because they
-            // will likely recreate the member list from the next received message. The problem
-            // occurs only if that "somebody" managed to reply earlier. Really, it's a problem for
-            // big groups with high message rate, but let it be for now.
-            new_members.extend(added_ids.clone());
-        }
-        if let Some(removed_id) = removed_id {
-            new_members.remove(&removed_id);
-        }
-        if recreate_member_list {
-            if self_added {
-                // ... then `better_msg` is already set.
-            } else if chat.blocked == Blocked::Request || !chat_contacts.contains(&ContactId::SELF)
-            {
-                warn!(context, "Implicit addition of SELF to chat {chat_id}.");
-                group_changes_msgs.push(
-                    stock_str::msg_add_member_local(
-                        context,
-                        &context.get_primary_self_addr().await?,
-                        ContactId::UNDEFINED,
-                    )
-                    .await,
-                );
-            } else {
-                added_ids = new_members.difference(&chat_contacts).copied().collect();
-                removed_ids = chat_contacts.difference(&new_members).copied().collect();
-            }
-        }
-
-        if let Some(added_id) = added_id {
-            added_ids.remove(&added_id);
-        }
-        if let Some(removed_id) = removed_id {
-            removed_ids.remove(&removed_id);
-        }
-        if !added_ids.is_empty() {
-            warn!(
+    if is_from_in_chat {
+        if let Some(ref chat_group_member_timestamps) = mime_parser.chat_group_member_timestamps() {
+            send_event_chat_modified |= update_chats_contacts_timestamps(
                 context,
-                "Implicit addition of {added_ids:?} to chat {chat_id}."
+                chat_id,
+                Some(from_id),
+                to_ids,
+                past_ids,
+                chat_group_member_timestamps,
+            )
+            .await?;
+            let new_chat_contacts = HashSet::<ContactId>::from_iter(
+                chat::get_chat_contacts(context, chat_id)
+                    .await?
+                    .iter()
+                    .copied(),
             );
-        }
-        if !removed_ids.is_empty() {
-            warn!(
-                context,
-                "Implicit removal of {removed_ids:?} from chat {chat_id}."
-            );
-        }
-        group_changes_msgs.reserve(added_ids.len() + removed_ids.len());
-        for contact_id in added_ids {
-            let contact = Contact::get_by_id(context, contact_id).await?;
-            group_changes_msgs.push(
-                stock_str::msg_add_member_local(context, contact.get_addr(), ContactId::UNDEFINED)
-                    .await,
-            );
-        }
-        for contact_id in removed_ids {
-            let contact = Contact::get_by_id(context, contact_id).await?;
-            group_changes_msgs.push(
-                stock_str::msg_del_member_local(context, contact.get_addr(), ContactId::UNDEFINED)
-                    .await,
-            );
-        }
-
-        if new_members != chat_contacts {
-            chat::update_chat_contacts_table(context, chat_id, &new_members).await?;
-            chat_contacts = new_members;
-            send_event_chat_modified = true;
-        }
-        if sync_member_list {
-            let mut ts = mime_parser.timestamp_sent;
-            if recreate_member_list {
-                // Reject all older membership changes. See `allow_member_list_changes` to know how
-                // this works.
-                ts += timestamp_sent_tolerance;
+            added_ids = new_chat_contacts
+                .difference(&chat_contacts)
+                .copied()
+                .collect();
+            removed_ids = chat_contacts
+                .difference(&new_chat_contacts)
+                .copied()
+                .collect();
+        } else {
+            let mut new_members = HashSet::from_iter(to_ids.iter().copied());
+            new_members.insert(ContactId::SELF);
+            if !from_id.is_special() {
+                new_members.insert(from_id);
             }
-            chat_id
-                .update_timestamp(context, Param::MemberListTimestamp, ts)
+
+            if !self_added {
+                if mime_parser.get_header(HeaderDef::ChatVersion).is_none() {
+                    // Allow non-Delta Chat MUAs to add members.
+                    added_ids = new_members.difference(&chat_contacts).copied().collect();
+                }
+
+                if let Some(added_id) = added_id {
+                    added_ids.insert(added_id);
+                }
+                new_members.clone_from(&chat_contacts);
+                // Don't delete any members locally, but instead add absent ones to provide group
+                // membership consistency for all members:
+                new_members.extend(added_ids.clone());
+            }
+            if let Some(removed_id) = removed_id {
+                new_members.remove(&removed_id);
+            }
+
+            if new_members != chat_contacts {
+                chat::update_chat_contacts_table(
+                    context,
+                    mime_parser.timestamp_sent,
+                    chat_id,
+                    &new_members,
+                )
                 .await?;
+                chat_contacts = new_members;
+                send_event_chat_modified = true;
+            }
         }
     }
 
+    if let Some(added_id) = added_id {
+        if !added_ids.remove(&added_id) && !self_added {
+            // No-op "Member added" message.
+            //
+            // Trash it.
+            better_msg = Some(String::new());
+        }
+    }
+    if let Some(removed_id) = removed_id {
+        removed_ids.remove(&removed_id);
+    }
+    if !added_ids.is_empty() {
+        warn!(
+            context,
+            "Implicit addition of {added_ids:?} to chat {chat_id}."
+        );
+    }
+    if !removed_ids.is_empty() {
+        warn!(
+            context,
+            "Implicit removal of {removed_ids:?} from chat {chat_id}."
+        );
+    }
+    group_changes_msgs.reserve(added_ids.len() + removed_ids.len());
+    for contact_id in added_ids {
+        let contact = Contact::get_by_id(context, contact_id).await?;
+        group_changes_msgs.push(
+            stock_str::msg_add_member_local(context, contact.get_addr(), ContactId::UNDEFINED)
+                .await,
+        );
+    }
+    for contact_id in removed_ids {
+        let contact = Contact::get_by_id(context, contact_id).await?;
+        group_changes_msgs.push(
+            stock_str::msg_del_member_local(context, contact.get_addr(), ContactId::UNDEFINED)
+                .await,
+        );
+    }
+
     if let Some(avatar_action) = &mime_parser.group_avatar {
         if !chat_contacts.contains(&ContactId::SELF) {
             warn!(
@@ -2487,7 +2542,13 @@ async fn create_or_lookup_mailinglist(
             )
         })?;
 
-        chat::add_to_chat_contacts_table(context, chat_id, &[ContactId::SELF]).await?;
+        chat::add_to_chat_contacts_table(
+            context,
+            mime_parser.timestamp_sent,
+            chat_id,
+            &[ContactId::SELF],
+        )
+        .await?;
         Ok(Some((chat_id, blocked)))
     } else {
         info!(context, "Creating list forbidden by caller.");
@@ -2683,7 +2744,13 @@ async fn create_adhoc_group(
         context,
         "Created ad-hoc group id={new_chat_id}, name={grpname:?}."
     );
-    chat::add_to_chat_contacts_table(context, new_chat_id, &member_ids).await?;
+    chat::add_to_chat_contacts_table(
+        context,
+        mime_parser.timestamp_sent,
+        new_chat_id,
+        &member_ids,
+    )
+    .await?;
 
     context.emit_event(EventType::ChatModified(new_chat_id));
     chatlist_events::emit_chatlist_changed(context);
@@ -2816,38 +2883,27 @@ async fn mark_recipients_as_verified(
     to_ids: Vec<ContactId>,
     mimeparser: &MimeMessage,
 ) -> Result<()> {
-    if to_ids.is_empty() {
-        return Ok(());
-    }
-
     if mimeparser.get_header(HeaderDef::ChatVerified).is_none() {
         return Ok(());
     }
-
-    let rows = context
-        .sql
-        .query_map(
-            &format!(
-                "SELECT c.addr, LENGTH(ps.verified_key_fingerprint)  FROM contacts c  \
-             LEFT JOIN acpeerstates ps ON c.addr=ps.addr  WHERE c.id IN({}) ",
-                sql::repeat_vars(to_ids.len())
-            ),
-            rusqlite::params_from_iter(&to_ids),
-            |row| {
-                let to_addr: String = row.get(0)?;
-                let is_verified: i32 = row.get(1).unwrap_or(0);
-                Ok((to_addr, is_verified != 0))
-            },
-            |rows| {
-                rows.collect::<std::result::Result<Vec<_>, _>>()
-                    .map_err(Into::into)
-            },
-        )
-        .await?;
-
     let contact = Contact::get_by_id(context, from_id).await?;
-
-    for (to_addr, is_verified) in rows {
+    for id in to_ids {
+        let Some((to_addr, is_verified)) = context
+            .sql
+            .query_row_optional(
+                "SELECT c.addr, LENGTH(ps.verified_key_fingerprint) FROM contacts c
+                LEFT JOIN acpeerstates ps ON c.addr=ps.addr WHERE c.id=?",
+                (id,),
+                |row| {
+                    let to_addr: String = row.get(0)?;
+                    let is_verified: i32 = row.get(1).unwrap_or(0);
+                    Ok((to_addr, is_verified != 0))
+                },
+            )
+            .await?
+        else {
+            continue;
+        };
         // mark gossiped keys (if any) as verified
         if let Some(gossiped_key) = mimeparser.gossiped_keys.get(&to_addr.to_lowercase()) {
             if let Some(mut peerstate) = Peerstate::from_addr(context, &to_addr).await? {
@@ -2945,14 +3001,12 @@ pub(crate) async fn get_prefetch_parent_message(
 }
 
 /// Looks up contact IDs from the database given the list of recipients.
-///
-/// Returns vector of IDs guaranteed to be unique.
 async fn add_or_lookup_contacts_by_address_list(
     context: &Context,
     address_list: &[SingleInfo],
     origin: Origin,
 ) -> Result<Vec<ContactId>> {
-    let mut contact_ids = HashSet::new();
+    let mut contact_ids = Vec::new();
     for info in address_list {
         let addr = &info.addr;
         if !may_be_valid_addr(addr) {
@@ -2963,13 +3017,13 @@ async fn add_or_lookup_contacts_by_address_list(
             let (contact_id, _) =
                 Contact::add_or_lookup(context, display_name.unwrap_or_default(), &addr, origin)
                     .await?;
-            contact_ids.insert(contact_id);
+            contact_ids.push(contact_id);
         } else {
             warn!(context, "Contact with address {:?} cannot exist.", addr);
         }
     }
 
-    Ok(contact_ids.into_iter().collect::<Vec<ContactId>>())
+    Ok(contact_ids)
 }
 
 #[cfg(test)]

src/receive_imf/tests.rs

@@ -566,6 +566,8 @@ async fn test_escaped_recipients() {
     .unwrap()
     .0;
 
+    // We test with non-chat message here
+    // because chat messages are not expected to have `Cc` header.
     receive_imf(
         &t,
         b"From: Foobar <foobar@example.com>\n\
@@ -573,8 +575,6 @@ async fn test_escaped_recipients() {
                  Cc: =?utf-8?q?=3Ch2=3E?= <carl@host.tld>\n\
                  Subject: foo\n\
                  Message-ID: <asdklfjjaweofi@example.com>\n\
-                 Chat-Version: 1.0\n\
-                 Chat-Disposition-Notification-To: <foobar@example.com>\n\
                  Date: Sun, 22 Mar 2020 22:37:57 +0000\n\
                  \n\
                  hello\n",
@@ -590,11 +590,12 @@ async fn test_escaped_recipients() {
     let msg = Message::load_from_db(&t, chats.get_msg_id(0).unwrap().unwrap())
         .await
         .unwrap();
-    assert_eq!(msg.is_dc_message, MessengerMessage::Yes);
-    assert_eq!(msg.text, "hello");
-    assert_eq!(msg.param.get_int(Param::WantsMdn).unwrap(), 1);
+    assert_eq!(msg.is_dc_message, MessengerMessage::No);
+    assert_eq!(msg.text, "foo – hello");
 }
 
+/// Tests that `Cc` header updates display name
+/// if existing contact has low enough origin.
 #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
 async fn test_cc_to_contact() {
     let t = TestContext::new_alice().await;
@@ -612,6 +613,8 @@ async fn test_cc_to_contact() {
     .unwrap()
     .0;
 
+    // We use non-chat message here
+    // because chat messages are not expected to have `Cc` header.
     receive_imf(
         &t,
         b"Received: (Postfix, from userid 1000); Mon, 4 Dec 2006 14:51:39 +0100 (CET)\n\
@@ -620,8 +623,6 @@ async fn test_cc_to_contact() {
                  Cc: Carl <carl@host.tld>\n\
                  Subject: foo\n\
                  Message-ID: <asdklfjjaweofi@example.com>\n\
-                 Chat-Version: 1.0\n\
-                 Chat-Disposition-Notification-To: <foobar@example.com>\n\
                  Date: Sun, 22 Mar 2020 22:37:57 +0000\n\
                  \n\
                  hello\n",
@@ -2200,6 +2201,30 @@ Message content",
     assert_ne!(msg.chat_id, t.get_self_chat().await.id);
 }
 
+/// Tests that message with hidden recipients is assigned to Saved Messages chat.
+#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
+async fn test_hidden_recipients_self_chat() {
+    let t = TestContext::new_alice().await;
+
+    receive_imf(
+        &t,
+        b"Subject: s
+Chat-Version: 1.0
+Message-ID: <foobar@localhost>
+To: hidden-recipients:;
+From: <alice@example.org>
+
+Message content",
+        false,
+    )
+    .await
+    .unwrap();
+
+    let msg = t.get_last_msg().await;
+    assert_eq!(msg.chat_id, t.get_self_chat().await.id);
+    assert_eq!(msg.to_id, ContactId::SELF);
+}
+
 #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
 async fn test_no_unencrypted_name_in_self_chat() -> Result<()> {
     let mut tcm = TestContextManager::new();
@@ -3309,6 +3334,7 @@ async fn test_outgoing_private_reply_multidevice() -> Result<()> {
     let group_id = chat::create_group_chat(&bob, ProtectionStatus::Unprotected, "Group").await?;
     chat::add_to_chat_contacts_table(
         &bob,
+        time(),
         group_id,
         &[
             bob.add_or_lookup_contact(&alice1).await.id,
@@ -3518,26 +3544,27 @@ async fn test_no_private_reply_to_blocked_account() -> Result<()> {
     let alice = tcm.alice().await;
     let bob = tcm.bob().await;
 
-    // =============== Bob creates a group ===============
+    tcm.section("Bob creates a group");
     let group_id = chat::create_group_chat(&bob, ProtectionStatus::Unprotected, "Group").await?;
     chat::add_to_chat_contacts_table(
         &bob,
+        time(),
         group_id,
         &[bob.add_or_lookup_contact(&alice).await.id],
     )
     .await?;
 
-    // =============== Bob sends the first message to the group ===============
+    tcm.section("Bob sends the first message to the group");
     let sent = bob.send_text(group_id, "Hello all!").await;
     alice.recv_msg(&sent).await;
 
     let chats = Chatlist::try_load(&bob, 0, None, None).await?;
     assert_eq!(chats.len(), 1);
 
-    // =============== Bob blocks Alice ================
+    tcm.section("Bob blocks Alice");
     Contact::block(&bob, bob.add_or_lookup_contact(&alice).await.id).await?;
 
-    // =============== Alice replies private to Bob ==============
+    tcm.section("Alice replies private to Bob");
     let received = alice.get_last_msg().await;
     assert_eq!(received.text, "Hello all!");
 
@@ -3551,7 +3578,7 @@ async fn test_no_private_reply_to_blocked_account() -> Result<()> {
     let sent2 = alice.send_msg(alice_bob_chat.id, &mut msg_out).await;
     bob.recv_msg(&sent2).await;
 
-    // ========= check that no contact request was created ============
+    // check that no contact request was created
     let chats = Chatlist::try_load(&bob, 0, None, None).await.unwrap();
     assert_eq!(chats.len(), 1);
     let chat_id = chats.get_chat_id(0).unwrap();
@@ -3562,7 +3589,7 @@ async fn test_no_private_reply_to_blocked_account() -> Result<()> {
     let received = bob.get_last_msg().await;
     assert_eq!(received.text, "Hello all!");
 
-    // =============== Bob unblocks Alice ================
+    tcm.section("Bob unblocks Alice");
     // test if the blocked chat is restored correctly
     Contact::unblock(&bob, bob.add_or_lookup_contact(&alice).await.id).await?;
     let chats = Chatlist::try_load(&bob, 0, None, None).await.unwrap();
@@ -4127,11 +4154,15 @@ async fn test_ignore_outdated_membership_changes() -> Result<()> {
 
     SystemTime::shift(Duration::from_secs(3600));
 
-    // Bob replies again adding Alice back.
+    // Bob replies again, even after some time this does not add Alice back.
+    //
+    // Bob cannot learn from Alice that Alice has left the group
+    // because Alice is not going to send more messages to the group.
     send_text_msg(bob, bob_chat_id, "i'm bob".to_string()).await?;
     let msg = &bob.pop_sent_msg().await;
     alice.recv_msg(msg).await;
-    assert!(is_contact_in_chat(alice, alice_chat_id, ContactId::SELF).await?);
+
+    assert!(!is_contact_in_chat(alice, alice_chat_id, ContactId::SELF).await?);
 
     Ok(())
 }
@@ -4192,7 +4223,7 @@ async fn test_dont_recreate_contacts_on_add_remove() -> Result<()> {
 }
 
 #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
-async fn test_recreate_contact_list_on_missing_messages() -> Result<()> {
+async fn test_delayed_removal_is_ignored() -> Result<()> {
     let alice = TestContext::new_alice().await;
     let bob = TestContext::new_bob().await;
     let chat_id = create_group_chat(&alice, ProtectionStatus::Unprotected, "Group").await?;
@@ -4200,6 +4231,7 @@ async fn test_recreate_contact_list_on_missing_messages() -> Result<()> {
     // create chat with three members
     add_to_chat_contacts_table(
         &alice,
+        time(),
         chat_id,
         &[
             Contact::create(&alice, "bob", "bob@example.net").await?,
@@ -4212,12 +4244,12 @@ async fn test_recreate_contact_list_on_missing_messages() -> Result<()> {
     let bob_chat_id = bob.recv_msg(&alice.pop_sent_msg().await).await.chat_id;
     bob_chat_id.accept(&bob).await?;
 
-    // bob removes a member
+    // Bob removes Fiona.
     let bob_contact_fiona = Contact::create(&bob, "fiona", "fiona@example.net").await?;
     remove_contact_from_chat(&bob, bob_chat_id, bob_contact_fiona).await?;
     let remove_msg = bob.pop_sent_msg().await;
 
-    // bob adds new members
+    // Bob adds new members "blue" and "orange", but first addition message is lost.
     let bob_blue = Contact::create(&bob, "blue", "blue@example.net").await?;
     add_contact_to_chat(&bob, bob_chat_id, bob_blue).await?;
     bob.pop_sent_msg().await;
@@ -4225,32 +4257,32 @@ async fn test_recreate_contact_list_on_missing_messages() -> Result<()> {
     add_contact_to_chat(&bob, bob_chat_id, bob_orange).await?;
     let add_msg = bob.pop_sent_msg().await;
 
-    // alice only receives the second member addition
+    // Alice only receives the second member addition,
+    // but this results in addition of both members
+    // and removal of Fiona.
     alice.recv_msg(&add_msg).await;
-
-    // since we missed messages, a new contact list should be build
     assert_eq!(get_chat_contacts(&alice, chat_id).await?.len(), 4);
 
-    // re-add fiona
+    // Alice re-adds Fiona.
     add_contact_to_chat(&alice, chat_id, alice_fiona).await?;
+    assert_eq!(get_chat_contacts(&alice, chat_id).await?.len(), 5);
 
-    // delayed removal of fiona shouldn't remove her
-    alice.recv_msg_trash(&remove_msg).await;
+    // Delayed removal of Fiona by Bob shouldn't remove her.
+    alice.recv_msg(&remove_msg).await;
     assert_eq!(get_chat_contacts(&alice, chat_id).await?.len(), 5);
 
     alice
-        .golden_test_chat(
-            chat_id,
-            "receive_imf_recreate_contact_list_on_missing_messages",
-        )
+        .golden_test_chat(chat_id, "receive_imf_delayed_removal_is_ignored")
         .await;
+
     Ok(())
 }
 
 #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
 async fn test_dont_readd_with_normal_msg() -> Result<()> {
-    let alice = TestContext::new_alice().await;
-    let bob = TestContext::new_bob().await;
+    let mut tcm = TestContextManager::new();
+    let alice = tcm.alice().await;
+    let bob = tcm.bob().await;
 
     let alice_chat_id = create_group_chat(&alice, ProtectionStatus::Unprotected, "Group").await?;
 
@@ -4265,6 +4297,7 @@ async fn test_dont_readd_with_normal_msg() -> Result<()> {
     let bob_chat_id = bob.recv_msg(&alice.pop_sent_msg().await).await.chat_id;
     bob_chat_id.accept(&bob).await?;
 
+    // Bob leaves, but Alice didn't receive Bob's leave message.
     remove_contact_from_chat(&bob, bob_chat_id, ContactId::SELF).await?;
     bob.pop_sent_msg().await;
     assert_eq!(get_chat_contacts(&bob, bob_chat_id).await?.len(), 1);
@@ -4278,12 +4311,11 @@ async fn test_dont_readd_with_normal_msg() -> Result<()> {
     .await?;
     bob.recv_msg(&alice.pop_sent_msg().await).await;
 
-    // Alice didn't receive Bob's leave message although a lot of time has
-    // passed, so Bob must re-add themselves otherwise other members would think
-    // Bob is still here while they aren't. Bob should retry to leave if they
-    // think that Alice didn't re-add them on purpose (which is possible if Alice uses a classical
-    // MUA).
-    assert!(is_contact_in_chat(&bob, bob_chat_id, ContactId::SELF).await?);
+    // Bob received a message from Alice, but this should not re-add him to the group.
+    assert!(!is_contact_in_chat(&bob, bob_chat_id, ContactId::SELF).await?);
+
+    // Bob got an update that fiora is added nevertheless.
+    assert_eq!(get_chat_contacts(&bob, bob_chat_id).await?.len(), 2);
     Ok(())
 }
 
@@ -4511,19 +4543,14 @@ async fn test_recreate_member_list_on_missing_add_of_self() -> Result<()> {
     bob.recv_msg(&alice.pop_sent_msg().await).await;
     assert!(!is_contact_in_chat(&bob, bob_chat_id, ContactId::SELF).await?);
 
-    // But if Bob left a long time ago, they must recreate the member list after missing a message.
+    // Even if some time passed, Bob must not be re-added back.
     SystemTime::shift(Duration::from_secs(3600));
     send_text_msg(&alice, alice_chat_id, "5th message".to_string()).await?;
     alice.pop_sent_msg().await;
     send_text_msg(&alice, alice_chat_id, "6th message".to_string()).await?;
     bob.recv_msg(&alice.pop_sent_msg().await).await;
-    assert!(is_contact_in_chat(&bob, bob_chat_id, ContactId::SELF).await?);
+    assert!(!is_contact_in_chat(&bob, bob_chat_id, ContactId::SELF).await?);
 
-    bob.golden_test_chat(
-        bob_chat_id,
-        "receive_imf_recreate_member_list_on_missing_add_of_self",
-    )
-    .await;
     Ok(())
 }
 
@@ -4757,13 +4784,6 @@ async fn test_partial_group_consistency() -> Result<()> {
     let contacts = get_chat_contacts(&bob, bob_chat_id).await?;
     assert_eq!(contacts.len(), 2);
 
-    // Get initial timestamp.
-    let timestamp = bob_chat_id
-        .get_param(&bob)
-        .await?
-        .get_i64(Param::MemberListTimestamp)
-        .unwrap();
-
     // Bob receives partial message.
     let msg_id = receive_imf_from_inbox(
         &bob,
@@ -4784,15 +4804,9 @@ Chat-Group-Member-Added: charlie@example.com",
     .context("no received message")?;
 
     let msg = Message::load_from_db(&bob, msg_id.msg_ids[0]).await?;
-    let timestamp2 = bob_chat_id
-        .get_param(&bob)
-        .await?
-        .get_i64(Param::MemberListTimestamp)
-        .unwrap();
 
     // Partial download does not change the member list.
     assert_eq!(msg.download_state, DownloadState::Available);
-    assert_eq!(timestamp, timestamp2);
     assert_eq!(get_chat_contacts(&bob, bob_chat_id).await?, contacts);
 
     // Alice sends normal message to bob, adding fiona.
@@ -4805,15 +4819,6 @@ Chat-Group-Member-Added: charlie@example.com",
 
     bob.recv_msg(&alice.pop_sent_msg().await).await;
 
-    let timestamp3 = bob_chat_id
-        .get_param(&bob)
-        .await?
-        .get_i64(Param::MemberListTimestamp)
-        .unwrap();
-
-    // Receiving a message after a partial download recreates the member list because we treat
-    // such messages as if we have not seen them.
-    assert_ne!(timestamp, timestamp3);
     let contacts = get_chat_contacts(&bob, bob_chat_id).await?;
     assert_eq!(contacts.len(), 3);
 
@@ -4837,15 +4842,9 @@ Chat-Group-Member-Added: charlie@example.com",
     .context("no received message")?;
 
     let msg = Message::load_from_db(&bob, msg_id.msg_ids[0]).await?;
-    let timestamp4 = bob_chat_id
-        .get_param(&bob)
-        .await?
-        .get_i64(Param::MemberListTimestamp)
-        .unwrap();
 
     // After full download, the old message should not change group state.
     assert_eq!(msg.download_state, DownloadState::Done);
-    assert_eq!(timestamp3, timestamp4);
     assert_eq!(get_chat_contacts(&bob, bob_chat_id).await?, contacts);
 
     Ok(())
@@ -4868,19 +4867,13 @@ async fn test_leave_protected_group_missing_member_key() -> Result<()> {
             ("b@b", "bob@example.net"),
         )
         .await?;
+
+    // We fail to send the message.
     assert!(remove_contact_from_chat(alice, group_id, ContactId::SELF)
         .await
         .is_err());
-    assert!(is_contact_in_chat(alice, group_id, ContactId::SELF).await?);
-    alice
-        .sql
-        .execute(
-            "UPDATE acpeerstates SET addr=? WHERE addr=?",
-            ("bob@example.net", "b@b"),
-        )
-        .await?;
-    remove_contact_from_chat(alice, group_id, ContactId::SELF).await?;
-    alice.pop_sent_msg().await;
+
+    // The contact is already removed anyway.
     assert!(!is_contact_in_chat(alice, group_id, ContactId::SELF).await?);
     Ok(())
 }
@@ -4902,12 +4895,22 @@ async fn test_protected_group_add_remove_member_missing_key() -> Result<()> {
         .await?;
 
     let fiona = &tcm.fiona().await;
+    let fiona_addr = fiona.get_config(Config::Addr).await?.unwrap();
     mark_as_verified(alice, fiona).await;
     let alice_fiona_id = alice.add_or_lookup_contact(fiona).await.id;
     assert!(add_contact_to_chat(alice, group_id, alice_fiona_id)
         .await
         .is_err());
-    assert!(!is_contact_in_chat(alice, group_id, alice_fiona_id).await?);
+    // Sending the message failed,
+    // but member is added to the chat locally already.
+    assert!(is_contact_in_chat(alice, group_id, alice_fiona_id).await?);
+    let msg = alice.get_last_msg_in(group_id).await;
+    assert!(msg.is_info());
+    assert_eq!(
+        msg.get_text(),
+        stock_str::msg_add_member_local(alice, &fiona_addr, ContactId::SELF).await
+    );
+
     // Now the chat has a message "You added member fiona@example.net. [INFO] !!" (with error) that
     // may be confusing, but if the error is displayed in UIs, it's more or less ok. This is not a
     // normal scenario anyway.

src/securejoin.rs

@@ -454,6 +454,9 @@ pub(crate) async fn handle_securejoin_handshake(
                     .await?;
                 inviter_progress(context, contact_id, 800);
                 inviter_progress(context, contact_id, 1000);
+                // IMAP-delete the message to avoid handling it by another device and adding the
+                // member twice. Another device will know the member's key from Autocrypt-Gossip.
+                Ok(HandshakeMessage::Done)
             } else {
                 // Setup verified contact.
                 secure_connection_established(
@@ -468,8 +471,8 @@ pub(crate) async fn handle_securejoin_handshake(
                     .context("failed sending vc-contact-confirm message")?;
 
                 inviter_progress(context, contact_id, 1000);
+                Ok(HandshakeMessage::Ignore) // "Done" would delete the message and break multi-device (the key from Autocrypt-header is needed)
             }
-            Ok(HandshakeMessage::Ignore) // "Done" would delete the message and break multi-device (the key from Autocrypt-header is needed)
         }
         /*=======================================================
         ====             Bob - the joiner's side             ====
@@ -1353,6 +1356,8 @@ mod tests {
         // explicit user action, the Auto-Submitted header shouldn't be present. Otherwise it would
         // be strange to have it in "member-added" messages of verified groups only.
         assert!(msg.get_header(HeaderDef::AutoSubmitted).is_none());
+        // This is a two-member group, but Alice must Autocrypt-gossip to her other devices.
+        assert!(msg.get_header(HeaderDef::AutocryptGossip).is_some());
 
         {
             // Now Alice's chat with Bob should still be hidden, the verified message should

src/securejoin/bob.rs

@@ -59,8 +59,13 @@ pub(super) async fn start_protocol(context: &Context, invite: QrInvite) -> Resul
             // only become usable once the protocol is finished.
             let group_chat_id = state.joining_chat_id(context).await?;
             if !is_contact_in_chat(context, group_chat_id, invite.contact_id()).await? {
-                chat::add_to_chat_contacts_table(context, group_chat_id, &[invite.contact_id()])
-                    .await?;
+                chat::add_to_chat_contacts_table(
+                    context,
+                    time(),
+                    group_chat_id,
+                    &[invite.contact_id()],
+                )
+                .await?;
             }
             let msg = stock_str::secure_join_started(context, invite.contact_id()).await;
             chat::add_info_msg(context, group_chat_id, &msg, time()).await?;

src/smtp.rs

@@ -21,7 +21,6 @@ use crate::mimefactory::MimeFactory;
 use crate::net::proxy::ProxyConfig;
 use crate::net::session::SessionBufStream;
 use crate::scheduler::connectivity::ConnectivityStore;
-use crate::sql;
 use crate::stock_str::unencrypted_email;
 use crate::tools::{self, time_elapsed};
 
@@ -104,7 +103,7 @@ impl Smtp {
     }
 
     /// Connect using the provided login params.
-    #[expect(clippy::too_many_arguments)]
+    #[allow(clippy::too_many_arguments)]
     pub async fn connect(
         &mut self,
         context: &Context,
@@ -585,18 +584,16 @@ async fn send_mdn_rfc724_mid(
             info!(context, "Successfully sent MDN for {rfc724_mid}.");
             context
                 .sql
-                .execute("DELETE FROM smtp_mdns WHERE rfc724_mid = ?", (rfc724_mid,))
+                .transaction(|transaction| {
+                    let mut stmt =
+                        transaction.prepare("DELETE FROM smtp_mdns WHERE rfc724_mid = ?")?;
+                    stmt.execute((rfc724_mid,))?;
+                    for additional_rfc724_mid in additional_rfc724_mids {
+                        stmt.execute((additional_rfc724_mid,))?;
+                    }
+                    Ok(())
+                })
                 .await?;
-            if !additional_rfc724_mids.is_empty() {
-                let q = format!(
-                    "DELETE FROM smtp_mdns WHERE rfc724_mid IN({})",
-                    sql::repeat_vars(additional_rfc724_mids.len())
-                );
-                context
-                    .sql
-                    .execute(&q, rusqlite::params_from_iter(additional_rfc724_mids))
-                    .await?;
-            }
             Ok(true)
         }
         SendResult::Retry => {

src/smtp/connect.rs

@@ -45,7 +45,7 @@ async fn new_smtp_transport<S: AsyncBufRead + AsyncWrite + Unpin>(
     Ok(transport)
 }
 
-#[expect(clippy::too_many_arguments)]
+#[allow(clippy::too_many_arguments)]
 pub(crate) async fn connect_and_auth(
     context: &Context,
     proxy_config: &Option<ProxyConfig>,

src/sql.rs

@@ -44,12 +44,6 @@ macro_rules! params_slice {
     };
 }
 
-pub(crate) fn params_iter(
-    iter: &[impl crate::sql::ToSql],
-) -> impl Iterator<Item = &dyn crate::sql::ToSql> {
-    iter.iter().map(|item| item as &dyn crate::sql::ToSql)
-}
-
 mod migrations;
 mod pool;
 
@@ -441,7 +435,7 @@ impl Sql {
         .await
     }
 
-    /// Execute the function inside a transaction assuming that it does write queries.
+    /// Execute the function inside a transaction assuming that it does writes.
     ///
     /// If the function returns an error, the transaction will be rolled back. If it does not return an
     /// error, the transaction will be committed.
@@ -450,7 +444,28 @@ impl Sql {
         H: Send + 'static,
         G: Send + FnOnce(&mut rusqlite::Transaction<'_>) -> Result<H>,
     {
-        self.call_write(move |conn| {
+        let query_only = false;
+        self.transaction_ex(query_only, callback).await
+    }
+
+    /// Execute the function inside a transaction.
+    ///
+    /// * `query_only` - Whether the function only executes read statements (queries) and can be run
+    ///   in parallel with other transactions. NB: Creating and modifying temporary tables are also
+    ///   allowed with `query_only`, temporary tables aren't visible in other connections, but you
+    ///   need to pass `PRAGMA query_only=0;` to SQLite before that:
+    ///     `pragma_update(None, "query_only", "0")`.
+    ///   Also temporary tables need to be dropped because the connection is returned to the pool
+    ///   then.
+    ///
+    /// If the function returns an error, the transaction will be rolled back. If it does not return
+    /// an error, the transaction will be committed.
+    pub async fn transaction_ex<G, H>(&self, query_only: bool, callback: G) -> Result<H>
+    where
+        H: Send + 'static,
+        G: Send + FnOnce(&mut rusqlite::Transaction<'_>) -> Result<H>,
+    {
+        self.call(query_only, move |conn| {
             let mut transaction = conn.transaction()?;
             let ret = callback(&mut transaction);
 
@@ -1024,16 +1039,6 @@ async fn prune_tombstones(sql: &Sql) -> Result<()> {
     Ok(())
 }
 
-/// Helper function to return comma-separated sequence of `?` chars.
-///
-/// Use this together with [`rusqlite::ParamsFromIter`] to use dynamically generated
-/// parameter lists.
-pub fn repeat_vars(count: usize) -> String {
-    let mut s = "?,".repeat(count);
-    s.pop(); // Remove trailing comma
-    s
-}
-
 #[cfg(test)]
 mod tests {
     use super::*;

src/sql/migrations.rs

@@ -1121,6 +1121,56 @@ CREATE INDEX msgs_status_updates_index2 ON msgs_status_updates (uid);
         .await?;
     }
 
+    inc_and_check(&mut migration_version, 127)?;
+    if dbversion < migration_version {
+        // This is buggy: `delete_server_after` > 1 isn't handled. Migration #129 fixes this.
+        sql.execute_migration(
+            "INSERT OR IGNORE INTO config (keyname, value)
+             SELECT 'bcc_self', '1'
+             FROM config WHERE keyname='delete_server_after' AND value='0'
+            ",
+            migration_version,
+        )
+        .await?;
+    }
+
+    inc_and_check(&mut migration_version, 128)?;
+    if dbversion < migration_version {
+        // Add the timestamps of addition and removal.
+        //
+        // If `add_timestamp >= remove_timestamp`,
+        // then the member is currently a member of the chat.
+        // Otherwise the member is a past member.
+        sql.execute_migration(
+            "ALTER TABLE chats_contacts
+             ADD COLUMN add_timestamp NOT NULL DEFAULT 0;
+             ALTER TABLE chats_contacts
+             ADD COLUMN remove_timestamp NOT NULL DEFAULT 0;
+            ",
+            migration_version,
+        )
+        .await?;
+    }
+
+    inc_and_check(&mut migration_version, 129)?;
+    if dbversion < migration_version {
+        // Existing chatmail configurations having `delete_server_after` != "delete at once" should
+        // get `bcc_self` enabled, they may be multidevice configurations:
+        // - Before migration #127, `delete_server_after` was set to 0 upon a backup export, but
+        //   then `bcc_self` is enabled instead (whose default is changed to 0 for chatmail).
+        // - The user might set `delete_server_after` to a value other than 0 or 1 when that was
+        //   possible in UIs.
+        // We don't check `is_chatmail` for simplicity.
+        sql.execute_migration(
+            "INSERT OR IGNORE INTO config (keyname, value)
+             SELECT 'bcc_self', '1'
+             FROM config WHERE keyname='delete_server_after' AND value!='1'
+            ",
+            migration_version,
+        )
+        .await?;
+    }
+
     let new_version = sql
         .get_raw_config_int(VERSION_CFG)
         .await?
@@ -1181,6 +1231,35 @@ impl Sql {
         .await
         .with_context(|| format!("execute_migration failed for version {version}"))?;
 
-        self.set_db_version_in_cache(version).await
+        self.config_cache.write().await.clear();
+
+        Ok(())
+    }
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+    use crate::config::Config;
+    use crate::test_utils::TestContext;
+
+    #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
+    async fn test_clear_config_cache() -> anyhow::Result<()> {
+        // Some migrations change the `config` table in SQL.
+        // This test checks that the config cache is invalidated in `execute_migration()`.
+
+        let t = TestContext::new().await;
+        assert_eq!(t.get_config_bool(Config::IsChatmail).await?, false);
+
+        t.sql
+            .execute_migration(
+                "INSERT INTO config (keyname, value) VALUES ('is_chatmail', '1')",
+                1000,
+            )
+            .await?;
+        assert_eq!(t.get_config_bool(Config::IsChatmail).await?, true);
+        assert_eq!(t.sql.get_raw_config_int(VERSION_CFG).await?.unwrap(), 1000);
+
+        Ok(())
     }
 }

src/sync.rs

@@ -566,6 +566,10 @@ mod tests {
         assert!(token::exists(&alice2, token::Namespace::Auth, "testtoken").await?);
         assert_eq!(Chatlist::try_load(&alice2, 0, None, None).await?.len(), 0);
 
+        // Sync messages are "auto-generated", but they mustn't make the self-contact a bot.
+        let self_contact = alice2.add_or_lookup_contact(&alice2).await;
+        assert!(!self_contact.is_bot());
+
         // the same sync message sent to bob must not be executed
         let bob = TestContext::new_bob().await;
         bob.recv_msg(&sent_msg).await;

src/test_utils.rs

@@ -41,6 +41,7 @@ use crate::pgp::KeyPair;
 use crate::receive_imf::receive_imf;
 use crate::securejoin::{get_securejoin_qr, join_securejoin};
 use crate::stock_str::StockStrings;
+use crate::tools::time;
 
 #[allow(non_upper_case_globals)]
 pub const AVATAR_900x900_BYTES: &[u8] = include_bytes!("../test-data/image/avatar900x900.png");
@@ -880,7 +881,7 @@ impl TestContext {
             let contact = self.add_or_lookup_contact(member).await;
             to_add.push(contact.id);
         }
-        add_to_chat_contacts_table(self, chat_id, &to_add)
+        add_to_chat_contacts_table(self, time(), chat_id, &to_add)
             .await
             .unwrap();
 

src/tests/verified_chats.rs

@@ -686,7 +686,7 @@ async fn test_break_protection_then_verify_again() -> Result<()> {
     alice.create_chat(&bob).await;
     assert_verified(&alice, &bob, ProtectionStatus::Protected).await;
     let chats = Chatlist::try_load(&alice, DC_GCL_FOR_FORWARDING, None, None).await?;
-    assert!(chats.len() == 1);
+    assert_eq!(chats.len(), 1);
 
     tcm.section("Bob reinstalls DC");
     drop(bob);
@@ -709,7 +709,7 @@ async fn test_break_protection_then_verify_again() -> Result<()> {
     assert_eq!(chat.is_protected(), false);
     assert_eq!(chat.is_protection_broken(), true);
     let chats = Chatlist::try_load(&alice, DC_GCL_FOR_FORWARDING, None, None).await?;
-    assert!(chats.len() == 1);
+    assert_eq!(chats.len(), 1);
 
     {
         let alice_bob_chat = alice.get_chat(&bob_new).await;

src/webxdc.rs

@@ -421,6 +421,7 @@ impl Context {
                     notify_list.get(&self_addr).or_else(|| notify_list.get("*"))
                 {
                     self.emit_event(EventType::IncomingWebxdcNotify {
+                        chat_id: instance.chat_id,
                         contact_id: from_id,
                         msg_id: notify_msg_id,
                         text: notify_text.clone(),

test-data/golden/chat_test_msg_with_implicit_member_add

@@ -1,9 +0,0 @@
-Group#Chat#10: Group chat [3 member(s)] 
---------------------------------------------------------------------------------
-Msg#10:  (Contact#Contact#11): I created a group [FRESH]
-Msg#11:  (Contact#Contact#11): Member Fiona (fiona@example.net) added by alice@example.org. [FRESH][INFO]
-Msg#12: Me (Contact#Contact#Self): You removed member Fiona (fiona@example.net). [INFO] √
-Msg#13:  (Contact#Contact#11): Welcome, Fiona! [FRESH]
-Msg#14: info (Contact#Contact#Info): Member Fiona (fiona@example.net) added. [NOTICED][INFO]
-Msg#15:  (Contact#Contact#11): Welcome back, Fiona! [FRESH]
---------------------------------------------------------------------------------

test-data/golden/chat_test_parallel_member_remove

@@ -1,8 +1,7 @@
-Group#Chat#10: Group chat [4 member(s)] 
+Group#Chat#10: Group chat [3 member(s)] 
 --------------------------------------------------------------------------------
 Msg#10:  (Contact#Contact#10): Hi! I created a group. [FRESH]
 Msg#11: Me (Contact#Contact#Self): You left the group. [INFO] √
 Msg#12:  (Contact#Contact#10): Member claire@example.net added by alice@example.org. [FRESH][INFO]
-Msg#13: info (Contact#Contact#Info): Member Me (bob@example.net) added. [NOTICED][INFO]
-Msg#14:  (Contact#Contact#10): What a silence! [FRESH]
+Msg#13:  (Contact#Contact#10): What a silence! [FRESH]
 --------------------------------------------------------------------------------

test-data/golden/receive_imf_delayed_removal_is_ignored

@@ -5,4 +5,5 @@ Msg#11: info (Contact#Contact#Info): Member blue@example.net added. [NOTICED][IN
 Msg#12: info (Contact#Contact#Info): Member fiona (fiona@example.net) removed. [NOTICED][INFO]
 Msg#13: bob (Contact#Contact#11): Member orange@example.net added by bob (bob@example.net). [FRESH][INFO]
 Msg#14: Me (Contact#Contact#Self): You added member fiona (fiona@example.net). [INFO] o
+Msg#15: bob (Contact#Contact#11): Member fiona (fiona@example.net) removed by bob (bob@example.net). [FRESH][INFO]
 --------------------------------------------------------------------------------

test-data/golden/receive_imf_recreate_member_list_on_missing_add_of_self

@@ -1,9 +0,0 @@
-Group#Chat#10: Group [2 member(s)] 
---------------------------------------------------------------------------------
-Msg#10: info (Contact#Contact#Info): Member Me (bob@example.net) added. [NOTICED][INFO]
-Msg#11:  (Contact#Contact#10): second message [FRESH]
-Msg#12🔒: Me (Contact#Contact#Self): You left the group. [INFO] √
-Msg#13:  (Contact#Contact#10): 4th message [FRESH]
-Msg#14: info (Contact#Contact#Info): Member Me (bob@example.net) added. [NOTICED][INFO]
-Msg#15:  (Contact#Contact#10): 6th message [FRESH]
---------------------------------------------------------------------------------