Adding error context for forward_msgs

feat: add more context to remove_contact_from_chat errors
refactor(remove_contact_from_chat): remove one level of indentation
2026-10-18 02:43:02 +03:00 · 2024-10-20 08:28:48 +00:00 · 2024-10-20 08:23:45 +00:00 · 2024-10-20 08:23:45 +00:00 · 2024-10-20 06:27:57 +00:00 · 2024-10-20 06:01:30 +00:00
64 changed files with 1326 additions and 748 deletions
@@ -24,7 +24,7 @@ jobs:
    name: Lint Rust
    runs-on: ubuntu-latest
    env:
-      RUSTUP_TOOLCHAIN: 1.81.0
+      RUSTUP_TOOLCHAIN: 1.82.0
    steps:
      - uses: actions/checkout@v4
        with:
@@ -95,11 +95,11 @@ jobs:
      matrix:
        include:
          - os: ubuntu-latest
-            rust: 1.81.0
+            rust: 1.82.0
          - os: windows-latest
-            rust: 1.81.0
+            rust: 1.82.0
          - os: macos-latest
-            rust: 1.81.0
+            rust: 1.82.0

          # Minimum Supported Rust Version = 1.77.0
          - os: ubuntu-latest
@@ -211,9 +211,9 @@ jobs:
        include:
          # Currently used Rust version.
          - os: ubuntu-latest
-            python: 3.12
+            python: 3.13
          - os: macos-latest
-            python: 3.12
+            python: 3.13

          # PyPy tests
          - os: ubuntu-latest
@@ -263,11 +263,11 @@ jobs:
      matrix:
        include:
          - os: ubuntu-latest
-            python: 3.12
+            python: 3.13
          - os: macos-latest
-            python: 3.12
+            python: 3.13
          - os: windows-latest
-            python: 3.12
+            python: 3.13

          # PyPy tests
          - os: ubuntu-latest
@@ -1,5 +1,65 @@
 # Changelog

+## [1.147.1] - 2024-10-13
+
+### Build system
+
+- Build Python 3.13 wheels.
+- deltachat-rpc-client: Add classifiers for all supported Python versions.
+
+### CI
+
+- Update to Python 3.13.
+
+### Documentation
+
+- CONTRIBUTING.md: Add a note on deleting/changing db columns.
+
+### Fixes
+
+- Reset quota on configured address change ([#5908](https://github.com/deltachat/deltachat-core-rust/pull/5908)).
+- Do not emit progress 1000 when configuration is cancelled.
+- Assume file extensions are 32 chars max and don't contain whitespace ([#5338](https://github.com/deltachat/deltachat-core-rust/pull/5338)).
+- Readd tokens.foreign_id column ([#6038](https://github.com/deltachat/deltachat-core-rust/pull/6038)).
+
+### Miscellaneous Tasks
+
+- cargo: Bump futures-* from 0.3.30 to 0.3.31.
+- cargo: Upgrade async_zip to 0.0.17 ([#6035](https://github.com/deltachat/deltachat-core-rust/pull/6035)).
+
+### Refactor
+
+- MsgId::update_download_state: Don't fail if the message doesn't exist anymore.
+
+## [1.147.0] - 2024-10-05
+
+### API-Changes
+
+- [**breaking**] Remove deprecated get_next_media() APIs.
+
+### Features / Changes
+
+- Reuse existing connections in background_fetch() if I/O is started.
+- MsgId::get_info(): Report original filename as well.
+- More context for the "Cannot establish guaranteed..." info message ([#6022](https://github.com/deltachat/deltachat-core-rust/pull/6022)).
+- deltachat-repl: Add `fetch` command to test `background_fetch()`.
+- deltachat-repl: Print send-backup QR code to the terminal.
+
+### Fixes
+
+- Do not attempt to reference info messages.
+- query_row_optional: Do not treat rows with NULL as missing rows.
+- Skip unconfigured folders in `background_fetch()`.
+- Break out of accept() loop if there is an error transferring backup.
+- Make it possible to cancel ongoing backup transfer.
+- Make backup reception cancellable by stopping ongoing process.
+- Smooth progress bar for backup transfer.
+- Emit progress 0 if get_backup() fails.
+
+### Documentation
+
+- CONTRIBUTING.md: Add more SQL advices.
+
 ## [1.146.0] - 2024-10-03

 ### Fixes
@@ -4946,3 +5006,5 @@ https://github.com/deltachat/deltachat-core-rust/pulls?q=is%3Apr+is%3Aclosed
 [1.144.0]: https://github.com/deltachat/deltachat-core-rust/compare/v1.143.0..v1.144.0
 [1.145.0]: https://github.com/deltachat/deltachat-core-rust/compare/v1.144.0..v1.145.0
 [1.146.0]: https://github.com/deltachat/deltachat-core-rust/compare/v1.145.0..v1.146.0
+[1.147.0]: https://github.com/deltachat/deltachat-core-rust/compare/v1.146.0..v1.147.0
+[1.147.1]: https://github.com/deltachat/deltachat-core-rust/compare/v1.147.0..v1.147.1
@@ -32,6 +32,8 @@ on the contributing page: <https://github.com/deltachat/deltachat-core-rust/cont
 We format the code using `rustfmt`. Run `cargo fmt` prior to committing the code.
 Run `scripts/clippy.sh` to check the code for common mistakes with [Clippy].

+### SQL
+
 Multi-line SQL statements should be formatted using string literals,
 for example
 ```
@@ -65,6 +67,29 @@ is prone to errors like this if space before backslash is missing:
 Literal above results in `SELECT fooFROM bar` string.
 This style also does not allow using `--` comments.

+---
+
+Declare new SQL tables with [`STRICT`](https://sqlite.org/stricttables.html) keyword
+to make SQLite check column types.
+
+Declare primary keys with [`AUTOINCREMENT`](https://www.sqlite.org/autoinc.html) keyword.
+This avoids reuse of the row IDs and can avoid dangerous bugs
+like forwarding wrong message because the message was deleted
+and another message took its row ID.
+
+Declare all new columns as `NOT NULL`
+and set the `DEFAULT` value if it is optional so the column can be skipped in `INSERT` statements.
+Dealing with `NULL` values both in SQL and in Rust is tricky and we try to avoid it.
+If column is already declared without `NOT NULL`, use `IFNULL` function to provide default value when selecting it.
+Use `HAVING COUNT(*) > 0` clause
+to [prevent aggregate functions such as `MIN` and `MAX` from returning `NULL`](https://stackoverflow.com/questions/66527856/aggregate-functions-max-etc-return-null-instead-of-no-rows).
+
+Don't delete unused columns too early, but maybe after several months/releases, unused columns are
+still used by older versions, so deleting them breaks downgrading the core or importing a backup in
+an older version. Also don't change the column type, consider adding a new column with another name
+instead. Finally, never change column semantics, this is especially dangerous because the `STRICT`
+keyword doesn't help here.
+
 ### Commit messages

 Commit messages follow the [Conventional Commits] notation.
@@ -273,12 +273,13 @@ dependencies = [

 [[package]]
 name = "async-compression"
-version = "0.3.15"
+version = "0.4.15"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "942c7cd7ae39e91bde4820d74132e9862e62c2f386c3aa90ccf55949f5bad63a"
+checksum = "e26a9844c659a2a293d239c7910b752f8487fe122c6c8bd1659bf85a6507c302"
 dependencies = [
 "flate2",
 "futures-core",
+ "futures-io",
 "memchr",
 "pin-project-lite",
 "tokio",
@@ -286,11 +287,12 @@ dependencies = [

 [[package]]
 name = "async-imap"
-version = "0.10.1"
+version = "0.10.2"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "2162818f7b394e342a6591864bfc960824ed13e3f6609fee0d19e770ebcd99e1"
+checksum = "5488cd022c3c7bc41a9b34a540d9ac0d9c5cd42fdb106a67616521b7592d5b4e"
 dependencies = [
 "async-channel 2.3.1",
+ "async-compression",
 "base64 0.21.7",
 "bytes",
 "chrono",
@@ -299,6 +301,7 @@ dependencies = [
 "log",
 "nom",
 "once_cell",
+ "pin-project",
 "pin-utils",
 "self_cell",
 "stop-token",
@@ -368,16 +371,17 @@ dependencies = [

 [[package]]
 name = "async_zip"
-version = "0.0.12"
+version = "0.0.17"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "b2105142db9c6203b9dadc83b0553394589a6cb31b1449a3b46b42f47c3434d0"
+checksum = "00b9f7252833d5ed4b00aa9604b563529dd5e11de9c23615de2dcdf91eb87b52"
 dependencies = [
 "async-compression",
 "crc32fast",
- "log",
+ "futures-lite 2.3.0",
 "pin-project",
 "thiserror",
 "tokio",
+ "tokio-util",
 ]

 [[package]]
@@ -1105,6 +1109,28 @@ version = "0.8.19"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "248e3bacc7dc6baa3b21e405ee045c3047101a49145e7e9eca583ab4c2ca5345"

+[[package]]
+name = "crossterm"
+version = "0.28.1"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "829d955a0bb380ef178a640b91779e3987da38c9aea133b20614cfed8cdea9c6"
+dependencies = [
+ "bitflags 2.6.0",
+ "crossterm_winapi",
+ "parking_lot",
+ "rustix",
+ "winapi",
+]
+
+[[package]]
+name = "crossterm_winapi"
+version = "0.9.1"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "acdd7c62a3665c7f6830a51635d9ac9b23ed385797f70a83bb8bafe9c572ab2b"
+dependencies = [
+ "winapi",
+]
+
 [[package]]
 name = "crunchy"
 version = "0.2.2"
@@ -1267,7 +1293,7 @@ dependencies = [

 [[package]]
 name = "deltachat"
-version = "1.146.0"
+version = "1.147.1"
 dependencies = [
 "anyhow",
 "async-broadcast",
@@ -1315,6 +1341,7 @@ dependencies = [
 "parking_lot",
 "percent-encoding",
 "pgp",
+ "pin-project",
 "pretty_assertions",
 "proptest",
 "qrcodegen",
@@ -1366,7 +1393,7 @@ dependencies = [

 [[package]]
 name = "deltachat-jsonrpc"
-version = "1.146.0"
+version = "1.147.1"
 dependencies = [
 "anyhow",
 "async-channel 2.3.1",
@@ -1391,13 +1418,14 @@ dependencies = [

 [[package]]
 name = "deltachat-repl"
-version = "1.146.0"
+version = "1.147.1"
 dependencies = [
 "anyhow",
 "deltachat",
 "dirs",
 "log",
 "nu-ansi-term",
+ "qr2term",
 "rusqlite",
 "rustyline",
 "tokio",
@@ -1406,7 +1434,7 @@ dependencies = [

 [[package]]
 name = "deltachat-rpc-server"
-version = "1.146.0"
+version = "1.147.1"
 dependencies = [
 "anyhow",
 "deltachat",
@@ -1435,7 +1463,7 @@ dependencies = [

 [[package]]
 name = "deltachat_ffi"
-version = "1.146.0"
+version = "1.147.1"
 dependencies = [
 "anyhow",
 "deltachat",
@@ -2195,9 +2223,9 @@ dependencies = [

 [[package]]
 name = "futures-channel"
-version = "0.3.30"
+version = "0.3.31"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "eac8f7d7865dcb88bd4373ab671c8cf4508703796caa2b1985a9ca867b3fcb78"
+checksum = "2dff15bf788c671c1934e366d07e30c1814a8ef514e1af724a602e8a2fbe1b10"
 dependencies = [
 "futures-core",
 "futures-sink",
@@ -2220,9 +2248,9 @@ dependencies = [

 [[package]]
 name = "futures-core"
-version = "0.3.30"
+version = "0.3.31"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "dfc6580bb841c5a68e9ef15c77ccc837b40a7504914d52e47b8b0e9bbda25a1d"
+checksum = "05f29059c0c2090612e8d742178b0580d2dc940c837851ad723096f87af6663e"

 [[package]]
 name = "futures-executor"
@@ -2237,9 +2265,9 @@ dependencies = [

 [[package]]
 name = "futures-io"
-version = "0.3.30"
+version = "0.3.31"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "a44623e20b9681a318efdd71c299b6b222ed6f231972bfe2f224ebad6311f0c1"
+checksum = "9e5c1b78ca4aae1ac06c48a526a655760685149f0d465d21f37abfe57ce075c6"

 [[package]]
 name = "futures-lite"
@@ -2271,9 +2299,9 @@ dependencies = [

 [[package]]
 name = "futures-macro"
-version = "0.3.30"
+version = "0.3.31"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "87750cf4b7a4c0625b1529e4c543c2182106e4dedc60a2a6455e00d212c489ac"
+checksum = "162ee34ebcb7c64a8abebc059ce0fee27c2262618d7b60ed8faf72fef13c3650"
 dependencies = [
 "proc-macro2",
 "quote",
@@ -2282,15 +2310,15 @@ dependencies = [

 [[package]]
 name = "futures-sink"
-version = "0.3.30"
+version = "0.3.31"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "9fb8e00e87438d937621c1c6269e53f536c14d3fbd6a042bb24879e57d474fb5"
+checksum = "e575fab7d1e0dcb8d0c7bcf9a63ee213816ab51902e6d244a95819acacf1d4f7"

 [[package]]
 name = "futures-task"
-version = "0.3.30"
+version = "0.3.31"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "38d84fa142264698cdce1a9f9172cf383a0c82de1bddcf3092901442c4097004"
+checksum = "f90f7dce0722e95104fcb095585910c0977252f286e354b5e3bd38902cd99988"

 [[package]]
 name = "futures-timer"
@@ -2300,9 +2328,9 @@ checksum = "f288b0a4f20f9a56b5d1da57e2227c661b7b16168e2f72365f57b63326e29b24"

 [[package]]
 name = "futures-util"
-version = "0.3.30"
+version = "0.3.31"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "3d6401deb83407ab3da39eba7e33987a73c3df0c82b4bb5813ee871c19c41d48"
+checksum = "9fa08315bb612088cc391249efdc3bc77536f16c91f6cf495e6fbe85b20a4a81"
 dependencies = [
 "futures-channel",
 "futures-core",
@@ -4452,6 +4480,22 @@ dependencies = [
 "unarray",
 ]

+[[package]]
+name = "qr2term"
+version = "0.3.3"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "6867c60b38e9747a079a19614dbb5981a53f21b9a56c265f3bfdf6011a50a957"
+dependencies = [
+ "crossterm",
+ "qrcode",
+]
+
+[[package]]
+name = "qrcode"
+version = "0.14.1"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "d68782463e408eb1e668cf6152704bd856c78c5b6417adaee3203d8f4c1fc9ec"
+
 [[package]]
 name = "qrcodegen"
 version = "1.8.0"
@@ -6073,6 +6117,7 @@ checksum = "61e7c3654c13bcd040d4a03abee2c75b1d14a37b423cf5a813ceae1cc903ec6a"
 dependencies = [
 "bytes",
 "futures-core",
+ "futures-io",
 "futures-sink",
 "futures-util",
 "hashbrown",
@@ -1,6 +1,6 @@
 [package]
 name = "deltachat"
-version = "1.146.0"
+version = "1.147.1"
 edition = "2021"
 license = "MPL-2.0"
 rust-version = "1.77"
@@ -41,10 +41,10 @@ ratelimit = { path = "./deltachat-ratelimit" }
 anyhow = { workspace = true }
 async-broadcast = "0.7.1"
 async-channel = { workspace = true }
-async-imap = { version = "0.10.1", default-features = false, features = ["runtime-tokio"] }
+async-imap = { version = "0.10.2", default-features = false, features = ["runtime-tokio", "compress"] }
 async-native-tls = { version = "0.5", default-features = false, features = ["runtime-tokio"] }
 async-smtp = { version = "0.9", default-features = false, features = ["runtime-tokio"] }
-async_zip = { version = "0.0.12", default-features = false, features = ["deflate", "fs"] }
+async_zip = { version = "0.0.17", default-features = false, features = ["deflate", "tokio-fs"] }
 base64 = { workspace = true }
 brotli = { version = "6", default-features=false, features = ["std"] }
 bytes = "1"
@@ -77,6 +77,7 @@ once_cell = { workspace = true }
 parking_lot = "0.12"
 percent-encoding = "2.3"
 pgp = { version = "0.13.2", default-features = false }
+pin-project = "1"
 qrcodegen = "1.7.0"
 quick-xml = "0.36"
 quoted_printable = "0.5"
@@ -1,6 +1,6 @@
 [package]
 name = "deltachat_ffi"
-version = "1.146.0"
+version = "1.147.1"
 description = "Deltachat FFI"
 edition = "2018"
 readme = "README.md"
@@ -1547,30 +1547,6 @@ void            dc_marknoticed_chat          (dc_context_t* context, uint32_t ch
 dc_array_t*     dc_get_chat_media            (dc_context_t* context, uint32_t chat_id, int msg_type, int msg_type2, int msg_type3);


-/**
- * Search next/previous message based on a given message and a list of types.
- * Typically used to implement the "next" and "previous" buttons
- * in a gallery or in a media player.
- *
- * @deprecated Deprecated 2023-10-03, use dc_get_chat_media() and navigate the returned array instead.
- * @memberof dc_context_t
- * @param context The context object as returned from dc_context_new().
- * @param msg_id The ID of the current message from which the next or previous message should be searched.
- * @param dir 1=get the next message, -1=get the previous one.
- * @param msg_type The message type to search for.
- *     If 0, the message type from curr_msg_id is used.
- * @param msg_type2 Alternative message type to search for. 0 to skip.
- * @param msg_type3 Alternative message type to search for. 0 to skip.
- * @return Returns the message ID that should be played next.
- *     The returned message is in the same chat as the given one
- *     and has one of the given types.
- *     Typically, this result is passed again to dc_get_next_media()
- *     later on the next swipe.
- *     If there is not next/previous message, the function returns 0.
- */
-uint32_t        dc_get_next_media            (dc_context_t* context, uint32_t msg_id, int dir, int msg_type, int msg_type2, int msg_type3);
-
-
 /**
 * Set chat visibility to pinned, archived or normal.
 *
@@ -6272,7 +6248,7 @@ void dc_event_unref(dc_event_t* event);


 /**
- * webxdc status update received.
+ * Webxdc status update received.
 * To get the received status update, use dc_get_webxdc_status_updates() with
 * `serial` set to the last known update
 * (in case of special bots, `status_update_serial` from `data2`
@@ -6307,6 +6283,15 @@ void dc_event_unref(dc_event_t* event);

 #define DC_EVENT_WEBXDC_REALTIME_DATA             2150

+/**
+ * Advertisement for ephemeral peer channel communication received.
+ * This can be used by bots to initiate peer-to-peer communication from their side.
+ * @param data1 (int) msg_id
+ * @param data2 0
+ */
+
+#define DC_EVENT_WEBXDC_REALTIME_ADVERTISEMENT    2151
+
 /**
 * Tells that the Background fetch was completed (or timed out).
 *
@@ -563,6 +563,7 @@ pub unsafe extern "C" fn dc_event_get_id(event: *mut dc_event_t) -> libc::c_int
        EventType::WebxdcStatusUpdate { .. } => 2120,
        EventType::WebxdcInstanceDeleted { .. } => 2121,
        EventType::WebxdcRealtimeData { .. } => 2150,
+        EventType::WebxdcRealtimeAdvertisementReceived { .. } => 2151,
        EventType::AccountsBackgroundFetchDone => 2200,
        EventType::ChatlistChanged => 2300,
        EventType::ChatlistItemChanged { .. } => 2301,
@@ -621,6 +622,7 @@ pub unsafe extern "C" fn dc_event_get_data1_int(event: *mut dc_event_t) -> libc:
        }
        EventType::WebxdcRealtimeData { msg_id, .. }
        | EventType::WebxdcStatusUpdate { msg_id, .. }
+        | EventType::WebxdcRealtimeAdvertisementReceived { msg_id }
        | EventType::WebxdcInstanceDeleted { msg_id, .. } => msg_id.to_u32() as libc::c_int,
        EventType::ChatlistItemChanged { chat_id } => {
            chat_id.unwrap_or_default().to_u32() as libc::c_int
@@ -666,6 +668,7 @@ pub unsafe extern "C" fn dc_event_get_data2_int(event: *mut dc_event_t) -> libc:
        | EventType::ChatlistItemChanged { .. }
        | EventType::ConfigSynced { .. }
        | EventType::ChatModified(_)
+        | EventType::WebxdcRealtimeAdvertisementReceived { .. }
        | EventType::EventChannelOverflow { .. } => 0,
        EventType::MsgsChanged { msg_id, .. }
        | EventType::ReactionsChanged { msg_id, .. }
@@ -733,6 +736,7 @@ pub unsafe extern "C" fn dc_event_get_data2_str(event: *mut dc_event_t) -> *mut
        | EventType::IncomingMsgBunch { .. }
        | EventType::ChatlistItemChanged { .. }
        | EventType::ChatlistChanged
+        | EventType::WebxdcRealtimeAdvertisementReceived { .. }
        | EventType::EventChannelOverflow { .. } => ptr::null_mut(),
        EventType::ConfigureProgress { comment, .. } => {
            if let Some(comment) = comment {
@@ -1443,48 +1447,6 @@ pub unsafe extern "C" fn dc_get_chat_media(
    })
 }

-#[no_mangle]
-#[allow(deprecated)]
-pub unsafe extern "C" fn dc_get_next_media(
-    context: *mut dc_context_t,
-    msg_id: u32,
-    dir: libc::c_int,
-    msg_type: libc::c_int,
-    or_msg_type2: libc::c_int,
-    or_msg_type3: libc::c_int,
-) -> u32 {
-    if context.is_null() {
-        eprintln!("ignoring careless call to dc_get_next_media()");
-        return 0;
-    }
-    let direction = if dir < 0 {
-        chat::Direction::Backward
-    } else {
-        chat::Direction::Forward
-    };
-
-    let ctx = &*context;
-    let msg_type = from_prim(msg_type).expect(&format!("invalid msg_type = {msg_type}"));
-    let or_msg_type2 =
-        from_prim(or_msg_type2).expect(&format!("incorrect or_msg_type2 = {or_msg_type2}"));
-    let or_msg_type3 =
-        from_prim(or_msg_type3).expect(&format!("incorrect or_msg_type3 = {or_msg_type3}"));
-
-    block_on(async move {
-        chat::get_next_media(
-            ctx,
-            MsgId::new(msg_id),
-            direction,
-            msg_type,
-            or_msg_type2,
-            or_msg_type3,
-        )
-        .await
-        .map(|msg_id| msg_id.map(|id| id.to_u32()).unwrap_or_default())
-        .unwrap_or(0)
-    })
-}
-
 #[no_mangle]
 pub unsafe extern "C" fn dc_set_chat_visibility(
    context: *mut dc_context_t,
@@ -1,6 +1,6 @@
 [package]
 name = "deltachat-jsonrpc"
-version = "1.146.0"
+version = "1.147.1"
 description = "DeltaChat JSON-RPC API"
 edition = "2021"
 default-run = "deltachat-jsonrpc-server"
@@ -1552,55 +1552,6 @@ impl CommandApi {
        Ok(media.iter().map(|msg_id| msg_id.to_u32()).collect())
    }

-    /// Search next/previous message based on a given message and a list of types.
-    /// Typically used to implement the "next" and "previous" buttons
-    /// in a gallery or in a media player.
-    ///
-    /// one combined call for getting chat::get_next_media for both directions
-    /// the manual chat::get_next_media in only one direction is not exposed by the jsonrpc yet
-    ///
-    /// Deprecated 2023-10-03, use `get_chat_media` method
-    /// and navigate the returned array instead.
-    #[allow(deprecated)]
-    async fn get_neighboring_chat_media(
-        &self,
-        account_id: u32,
-        msg_id: u32,
-        message_type: MessageViewtype,
-        or_message_type2: Option<MessageViewtype>,
-        or_message_type3: Option<MessageViewtype>,
-    ) -> Result<(Option<u32>, Option<u32>)> {
-        let ctx = self.get_context(account_id).await?;
-
-        let msg_type: Viewtype = message_type.into();
-        let msg_type2: Viewtype = or_message_type2.map(|v| v.into()).unwrap_or_default();
-        let msg_type3: Viewtype = or_message_type3.map(|v| v.into()).unwrap_or_default();
-
-        let prev = chat::get_next_media(
-            &ctx,
-            MsgId::new(msg_id),
-            chat::Direction::Backward,
-            msg_type,
-            msg_type2,
-            msg_type3,
-        )
-        .await?
-        .map(|id| id.to_u32());
-
-        let next = chat::get_next_media(
-            &ctx,
-            MsgId::new(msg_id),
-            chat::Direction::Forward,
-            msg_type,
-            msg_type2,
-            msg_type3,
-        )
-        .await?
-        .map(|id| id.to_u32());
-
-        Ok((prev, next))
-    }
-
    // ---------------------------------------------
    //                   backup
    // ---------------------------------------------
@@ -1995,9 +1946,13 @@ impl CommandApi {

    async fn send_msg(&self, account_id: u32, chat_id: u32, data: MessageData) -> Result<u32> {
        let ctx = self.get_context(account_id).await?;
-        let mut message = data.create_message(&ctx).await?;
+        let mut message = data
+            .create_message(&ctx)
+            .await
+            .context("Failed to create message")?;
        let msg_id = chat::send_msg(&ctx, ChatId::new(chat_id), &mut message)
-            .await?
+            .await
+            .context("Failed to send created message")?
            .to_u32();
        Ok(msg_id)
    }
@@ -244,6 +244,11 @@ pub enum EventType {
    #[serde(rename_all = "camelCase")]
    WebxdcRealtimeData { msg_id: u32, data: Vec<u8> },

+    /// Advertisement received over an ephemeral peer channel.
+    /// This can be used by bots to initiate peer-to-peer communication from their side.
+    #[serde(rename_all = "camelCase")]
+    WebxdcRealtimeAdvertisementReceived { msg_id: u32 },
+
    /// Inform that a message containing a webxdc instance has been deleted
    #[serde(rename_all = "camelCase")]
    WebxdcInstanceDeleted { msg_id: u32 },
@@ -373,6 +378,11 @@ impl From<CoreEventType> for EventType {
                msg_id: msg_id.to_u32(),
                data,
            },
+            CoreEventType::WebxdcRealtimeAdvertisementReceived { msg_id } => {
+                WebxdcRealtimeAdvertisementReceived {
+                    msg_id: msg_id.to_u32(),
+                }
+            }
            CoreEventType::WebxdcInstanceDeleted { msg_id } => WebxdcInstanceDeleted {
                msg_id: msg_id.to_u32(),
            },
@@ -605,16 +605,13 @@ impl MessageData {
            message.set_location(latitude, longitude);
        }
        if let Some(id) = self.quoted_message_id {
+            let quoted_message = Message::load_from_db(context, MsgId::new(id))
+                .await
+                .context("Failed to load quoted message")?;
            message
-                .set_quote(
-                    context,
-                    Some(
-                        &Message::load_from_db(context, MsgId::new(id))
-                            .await
-                            .context("message to quote could not be loaded")?,
-                    ),
-                )
-                .await?;
+                .set_quote(context, Some(&quoted_message))
+                .await
+                .context("Failed to set quote")?;
        } else if let Some(text) = self.quoted_text {
            let protect = false;
            message.set_quote_text(Some((text, protect)));
@@ -640,7 +637,7 @@ pub struct MessageInfo {
    error: Option<String>,
    rfc724_mid: String,
    server_urls: Vec<String>,
-    hop_info: Option<String>,
+    hop_info: String,
 }

 impl MessageInfo {
@@ -58,5 +58,5 @@
  },
  "type": "module",
  "types": "dist/deltachat.d.ts",
-  "version": "1.146.0"
+  "version": "1.147.1"
 }
@@ -1,6 +1,6 @@
 [package]
 name = "deltachat-repl"
-version = "1.146.0"
+version = "1.147.1"
 license = "MPL-2.0"
 edition = "2021"
 repository = "https://github.com/deltachat/deltachat-core-rust"
@@ -11,6 +11,7 @@ deltachat = { workspace = true, features = ["internals"]}
 dirs = "5"
 log = { workspace = true }
 nu-ansi-term = { workspace = true }
+qr2term = "0.3.3"
 rusqlite = { workspace = true }
 rustyline = "14"
 tokio = { workspace = true, features = ["fs", "rt-multi-thread", "macros"] }
@@ -355,6 +355,7 @@ pub async fn cmdline(context: Context, line: &str, chat_id: &mut ChatId) -> Resu
                 configure\n\
                 connect\n\
                 disconnect\n\
+                 fetch\n\
                 connectivity\n\
                 maybenetwork\n\
                 housekeeping\n\
@@ -486,8 +487,9 @@ pub async fn cmdline(context: Context, line: &str, chat_id: &mut ChatId) -> Resu
        }
        "send-backup" => {
            let provider = BackupProvider::prepare(&context).await?;
-            let qr = provider.qr();
-            println!("QR code: {}", format_backup(&qr)?);
+            let qr = format_backup(&provider.qr())?;
+            println!("QR code: {}", qr);
+            qr2term::print_qr(qr.as_str())?;
            provider.await?;
        }
        "receive-backup" => {
@@ -9,8 +9,6 @@
 extern crate deltachat;

 use std::borrow::Cow::{self, Borrowed, Owned};
-use std::io::{self, Write};
-use std::process::Command;

 use anyhow::{bail, Error};
 use deltachat::chat::ChatId;
@@ -168,7 +166,7 @@ const IMEX_COMMANDS: [&str; 13] = [
    "stop",
 ];

-const DB_COMMANDS: [&str; 10] = [
+const DB_COMMANDS: [&str; 11] = [
    "info",
    "set",
    "get",
@@ -176,6 +174,7 @@ const DB_COMMANDS: [&str; 10] = [
    "configure",
    "connect",
    "disconnect",
+    "fetch",
    "connectivity",
    "maybenetwork",
    "housekeeping",
@@ -417,6 +416,9 @@ async fn handle_cmd(
        "disconnect" => {
            ctx.stop_io().await;
        }
+        "fetch" => {
+            ctx.background_fetch().await?;
+        }
        "configure" => {
            ctx.configure().await?;
        }
@@ -446,12 +448,7 @@ async fn handle_cmd(
                    qr.replace_range(12..22, "0000000000")
                }
                println!("{qr}");
-                let output = Command::new("qrencode")
-                    .args(["-t", "ansiutf8", qr.as_str(), "-o", "-"])
-                    .output()
-                    .expect("failed to execute process");
-                io::stdout().write_all(&output.stdout).unwrap();
-                io::stderr().write_all(&output.stderr).unwrap();
+                qr2term::print_qr(qr.as_str())?;
            }
        }
        "getqrsvg" => {
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"

 [project]
 name = "deltachat-rpc-client"
-version = "1.146.0"
+version = "1.147.1"
 description = "Python client for Delta Chat core JSON-RPC interface"
 classifiers = [
    "Development Status :: 5 - Production/Stable",
@@ -13,10 +13,13 @@ classifiers = [
    "Operating System :: POSIX :: Linux",
    "Operating System :: MacOS :: MacOS X",
    "Programming Language :: Python :: 3",
+    "Programming Language :: Python :: 3.7",
    "Programming Language :: Python :: 3.8",
    "Programming Language :: Python :: 3.9",
    "Programming Language :: Python :: 3.10",
    "Programming Language :: Python :: 3.11",
+    "Programming Language :: Python :: 3.12",
+    "Programming Language :: Python :: 3.13",
    "Topic :: Communications :: Chat",
    "Topic :: Communications :: Email"
 ]
@@ -1,4 +1,5 @@
 import logging
+import time

 import pytest

@@ -44,13 +45,6 @@ def test_qr_setup_contact_svg(acfactory) -> None:

    _qr_code, svg = alice.get_qr_code_svg()

-    # Test that email address is in SVG
-    # when we have no display name.
-    # Check only the domain name, because
-    # long address may be split over multiple lines
-    # and not matched.
-    assert domain in svg
-
    alice.set_config("displayname", "Alice")

    # Test that display name is used
@@ -332,7 +326,6 @@ def test_verified_group_member_added_recovery(acfactory) -> None:

    ac3_contact_ac2 = ac3.get_contact_by_addr(ac2.get_config("addr"))
    ac3_chat.remove_contact(ac3_contact_ac2)
-    ac3_chat.add_contact(ac3_contact_ac2)

    msg_id = ac2.wait_for_incoming_msg_event().msg_id
    message = ac2.get_message_by_id(msg_id)
@@ -342,6 +335,8 @@ def test_verified_group_member_added_recovery(acfactory) -> None:
    snapshot = ac1.get_message_by_id(ac1.wait_for_incoming_msg_event().msg_id).get_snapshot()
    assert "removed" in snapshot.text

+    ac3_chat.add_contact(ac3_contact_ac2)
+
    event = ac2.wait_for_incoming_msg_event()
    msg_id = event.msg_id
    chat_id = event.chat_id
@@ -465,7 +460,10 @@ def test_qr_new_group_unblocked(acfactory):

 def test_aeap_flow_verified(acfactory):
    """Test that a new address is added to a contact when it changes its address."""
-    ac1, ac2, ac1new = acfactory.get_online_accounts(3)
+    ac1, ac2 = acfactory.get_online_accounts(2)
+
+    # ac1new is only used to get a new address.
+    ac1new = acfactory.new_preconfigured_account()

    logging.info("ac1: create verified-group QR, ac2 scans and joins")
    chat = ac1.create_group("hello", protect=True)
@@ -474,6 +472,7 @@ def test_aeap_flow_verified(acfactory):
    logging.info("ac2: start QR-code based join-group protocol")
    ac2.secure_join(qr_code)
    ac1.wait_for_securejoin_inviter_success()
+    ac2.wait_for_securejoin_joiner_success()

    logging.info("sending first message")
    msg_out = chat.send_text("old address").get_snapshot()
@@ -571,6 +570,7 @@ def test_securejoin_after_contact_resetup(acfactory) -> None:

    # ac1 waits for member added message and creates a QR code.
    snapshot = ac1.get_message_by_id(ac1.wait_for_incoming_msg_event().msg_id).get_snapshot()
+    assert snapshot.text == "Member Me ({}) added by {}.".format(ac1.get_config("addr"), ac3.get_config("addr"))
    ac1_qr_code = snapshot.chat.get_qr_code()

    # ac2 verifies ac1
@@ -585,17 +585,29 @@ def test_securejoin_after_contact_resetup(acfactory) -> None:
    # ac1 resetups the account.
    ac1 = acfactory.resetup_account(ac1)

-    # ac1 sends a message to ac2.
-    ac1_contact_ac2 = ac1.create_contact(ac2.get_config("addr"), "")
-    ac1_chat_ac2 = ac1_contact_ac2.create_chat()
-    ac1_chat_ac2.send_text("Hello!")
+    # Loop sending message from ac1 to ac2
+    # until ac2 accepts new ac1 key.
+    #
+    # This may not happen immediately because resetup of ac1
+    # rewinds "smeared timestamp" so Date: header for messages
+    # sent by new ac1 are in the past compared to the last Date:
+    # header sent by old ac1.
+    while True:
+        # ac1 sends a message to ac2.
+        ac1_contact_ac2 = ac1.create_contact(ac2.get_config("addr"), "")
+        ac1_chat_ac2 = ac1_contact_ac2.create_chat()
+        ac1_chat_ac2.send_text("Hello!")

-    # ac2 receives a message.
-    snapshot = ac2.get_message_by_id(ac2.wait_for_incoming_msg_event().msg_id).get_snapshot()
-    assert snapshot.text == "Hello!"
+        # ac2 receives a message.
+        snapshot = ac2.get_message_by_id(ac2.wait_for_incoming_msg_event().msg_id).get_snapshot()
+        assert snapshot.text == "Hello!"
+        logging.info("ac2 received Hello!")

-    # ac1 is no longer verified for ac2 as new Autocrypt key is not the same as old verified key.
-    assert not ac2_contact_ac1.get_snapshot().is_verified
+        # ac1 is no longer verified for ac2 as new Autocrypt key is not the same as old verified key.
+        logging.info("ac2 addr={}, ac1 addr={}".format(ac2.get_config("addr"), ac1.get_config("addr")))
+        if not ac2_contact_ac1.get_snapshot().is_verified:
+            break
+        time.sleep(1)

    # ac1 goes offline.
    ac1.remove()
@@ -1,6 +1,6 @@
 [package]
 name = "deltachat-rpc-server"
-version = "1.146.0"
+version = "1.147.1"
 description = "DeltaChat JSON-RPC server"
 edition = "2021"
 readme = "README.md"
@@ -15,5 +15,5 @@
  },
  "type": "module",
  "types": "index.d.ts",
-  "version": "1.146.0"
+  "version": "1.147.1"
 }
@@ -3,15 +3,15 @@
    "android": {
      "inputs": {
        "devshell": "devshell",
-        "flake-utils": "flake-utils_2",
+        "flake-utils": "flake-utils",
        "nixpkgs": "nixpkgs"
      },
      "locked": {
-        "lastModified": 1712088936,
-        "narHash": "sha256-mVjeSWQiR/t4UZ9fUawY9OEPAhY1R3meYG+0oh8DUBs=",
+        "lastModified": 1729369131,
+        "narHash": "sha256-PtfScp+nQd1PsT5rf0Qgjdbsh4Iag6R1ivYMWLizyIc=",
        "owner": "tadfisher",
        "repo": "android-nixpkgs",
-        "rev": "2d8181caef279f19c4a33dc694723f89ffc195d4",
+        "rev": "82bffbf3f06bdccf44fc62a9bd4f152ac80a55b0",
        "type": "github"
      },
      "original": {
@@ -22,18 +22,17 @@
    },
    "devshell": {
      "inputs": {
-        "flake-utils": "flake-utils",
        "nixpkgs": [
          "android",
          "nixpkgs"
        ]
      },
      "locked": {
-        "lastModified": 1711099426,
-        "narHash": "sha256-HzpgM/wc3aqpnHJJ2oDqPBkNsqWbW0WfWUO8lKu8nGk=",
+        "lastModified": 1728330715,
+        "narHash": "sha256-xRJ2nPOXb//u1jaBnDP56M7v5ldavjbtR6lfGqSvcKg=",
        "owner": "numtide",
        "repo": "devshell",
-        "rev": "2d45b54ca4a183f2fdcf4b19c895b64fbf620ee8",
+        "rev": "dd6b80932022cea34a019e2bb32f6fa9e494dfef",
        "type": "github"
      },
      "original": {
@@ -48,11 +47,11 @@
        "rust-analyzer-src": "rust-analyzer-src"
      },
      "locked": {
-        "lastModified": 1714112748,
-        "narHash": "sha256-jq6Cpf/pQH85p+uTwPPrGG8Ky/zUOTwMJ7mcqc5M4So=",
+        "lastModified": 1729375822,
+        "narHash": "sha256-bRo4xVwUhvJ4Gz+OhWMREFMdBOYSw4Yi1Apj01ebbug=",
        "owner": "nix-community",
        "repo": "fenix",
-        "rev": "3ae4b908a795b6a3824d401a0702e11a7157d7e1",
+        "rev": "2853e7d9b5c52a148a9fb824bfe4f9f433f557ab",
        "type": "github"
      },
      "original": {
@@ -66,11 +65,11 @@
        "systems": "systems"
      },
      "locked": {
-        "lastModified": 1701680307,
-        "narHash": "sha256-kAuep2h5ajznlPMD9rnQyffWG8EM/C73lejGofXvdM8=",
+        "lastModified": 1726560853,
+        "narHash": "sha256-X6rJYSESBVr3hBoH0WbKE5KvhPU5bloyZ2L4K60/fPQ=",
        "owner": "numtide",
        "repo": "flake-utils",
-        "rev": "4022d587cbbfd70fe950c1e2083a02621806a725",
+        "rev": "c1dfcf08411b08f6b8615f7d8971a2bfa81d5e8a",
        "type": "github"
      },
      "original": {
@@ -84,29 +83,11 @@
        "systems": "systems_2"
      },
      "locked": {
-        "lastModified": 1710146030,
-        "narHash": "sha256-SZ5L6eA7HJ/nmkzGG7/ISclqe6oZdOZTNoesiInkXPQ=",
+        "lastModified": 1726560853,
+        "narHash": "sha256-X6rJYSESBVr3hBoH0WbKE5KvhPU5bloyZ2L4K60/fPQ=",
        "owner": "numtide",
        "repo": "flake-utils",
-        "rev": "b1d9ab70662946ef0850d488da1c9019f3a9752a",
-        "type": "github"
-      },
-      "original": {
-        "owner": "numtide",
-        "repo": "flake-utils",
-        "type": "github"
-      }
-    },
-    "flake-utils_3": {
-      "inputs": {
-        "systems": "systems_3"
-      },
-      "locked": {
-        "lastModified": 1710146030,
-        "narHash": "sha256-SZ5L6eA7HJ/nmkzGG7/ISclqe6oZdOZTNoesiInkXPQ=",
-        "owner": "numtide",
-        "repo": "flake-utils",
-        "rev": "b1d9ab70662946ef0850d488da1c9019f3a9752a",
+        "rev": "c1dfcf08411b08f6b8615f7d8971a2bfa81d5e8a",
        "type": "github"
      },
      "original": {
@@ -120,11 +101,11 @@
        "nixpkgs": "nixpkgs_3"
      },
      "locked": {
-        "lastModified": 1713520724,
-        "narHash": "sha256-CO8MmVDmqZX2FovL75pu5BvwhW+Vugc7Q6ze7Hj8heI=",
+        "lastModified": 1721727458,
+        "narHash": "sha256-r/xppY958gmZ4oTfLiHN0ZGuQ+RSTijDblVgVLFi1mw=",
        "owner": "nix-community",
        "repo": "naersk",
-        "rev": "c5037590290c6c7dae2e42e7da1e247e54ed2d49",
+        "rev": "3fb418eaf352498f6b6c30592e3beb63df42ef11",
        "type": "github"
      },
      "original": {
@@ -150,11 +131,11 @@
    },
    "nixpkgs": {
      "locked": {
-        "lastModified": 1711703276,
-        "narHash": "sha256-iMUFArF0WCatKK6RzfUJknjem0H9m4KgorO/p3Dopkk=",
+        "lastModified": 1729256560,
+        "narHash": "sha256-/uilDXvCIEs3C9l73JTACm4quuHUsIHcns1c+cHUJwA=",
        "owner": "NixOS",
        "repo": "nixpkgs",
-        "rev": "d8fe5e6c92d0d190646fb9f1056741a229980089",
+        "rev": "4c2fcb090b1f3e5b47eaa7bd33913b574a11e0a0",
        "type": "github"
      },
      "original": {
@@ -166,11 +147,11 @@
    },
    "nixpkgs_2": {
      "locked": {
-        "lastModified": 1713895582,
-        "narHash": "sha256-cfh1hi+6muQMbi9acOlju3V1gl8BEaZBXBR9jQfQi4U=",
+        "lastModified": 1729070438,
+        "narHash": "sha256-KOTTUfPkugH52avUvXGxvWy8ibKKj4genodIYUED+Kc=",
        "owner": "nixos",
        "repo": "nixpkgs",
-        "rev": "572af610f6151fd41c212f897c71f7056e3fb518",
+        "rev": "5785b6bb5eaae44e627d541023034e1601455827",
        "type": "github"
      },
      "original": {
@@ -182,11 +163,12 @@
    },
    "nixpkgs_3": {
      "locked": {
-        "lastModified": 1711668574,
-        "narHash": "sha256-u1dfs0ASQIEr1icTVrsKwg2xToIpn7ZXxW3RHfHxshg=",
-        "path": "/nix/store/9fpv0kjq9a80isa1wkkvrdqsh9dpcn05-source",
-        "rev": "219951b495fc2eac67b1456824cc1ec1fd2ee659",
-        "type": "path"
+        "lastModified": 1729265718,
+        "narHash": "sha256-4HQI+6LsO3kpWTYuVGIzhJs1cetFcwT7quWCk/6rqeo=",
+        "owner": "NixOS",
+        "repo": "nixpkgs",
+        "rev": "ccc0c2126893dd20963580b6478d1a10a4512185",
+        "type": "github"
      },
      "original": {
        "id": "nixpkgs",
@@ -195,11 +177,11 @@
    },
    "nixpkgs_4": {
      "locked": {
-        "lastModified": 1714076141,
-        "narHash": "sha256-Drmja/f5MRHZCskS6mvzFqxEaZMeciScCTFxWVLqWEY=",
+        "lastModified": 1729256560,
+        "narHash": "sha256-/uilDXvCIEs3C9l73JTACm4quuHUsIHcns1c+cHUJwA=",
        "owner": "nixos",
        "repo": "nixpkgs",
-        "rev": "7bb2ccd8cdc44c91edba16c48d2c8f331fb3d856",
+        "rev": "4c2fcb090b1f3e5b47eaa7bd33913b574a11e0a0",
        "type": "github"
      },
      "original": {
@@ -213,7 +195,7 @@
      "inputs": {
        "android": "android",
        "fenix": "fenix",
-        "flake-utils": "flake-utils_3",
+        "flake-utils": "flake-utils_2",
        "naersk": "naersk",
        "nix-filter": "nix-filter",
        "nixpkgs": "nixpkgs_4"
@@ -222,11 +204,11 @@
    "rust-analyzer-src": {
      "flake": false,
      "locked": {
-        "lastModified": 1714031783,
-        "narHash": "sha256-xS/niQsq1CQPOe4M4jvVPO2cnXS/EIeRG5gIopUbk+Q=",
+        "lastModified": 1729255720,
+        "narHash": "sha256-yODOuZxBkS0UfqMa6nmbqNbVfIbsu0tYLbV5vZzmsqI=",
        "owner": "rust-lang",
        "repo": "rust-analyzer",
-        "rev": "56bee2ddafa6177b19c631eedc88d43366553223",
+        "rev": "72b214fbfbe6f7b95a7877b962783bd42062cc0a",
        "type": "github"
      },
      "original": {
@@ -265,21 +247,6 @@
        "repo": "default",
        "type": "github"
      }
-    },
-    "systems_3": {
-      "locked": {
-        "lastModified": 1681028828,
-        "narHash": "sha256-Vy1rq5AaRuLzOxct8nz4T6wlgyUR7zLU309k9mBC768=",
-        "owner": "nix-systems",
-        "repo": "default",
-        "rev": "da67096a3b9bf56a91d16901293e51ba5b49a27e",
-        "type": "github"
-      },
-      "original": {
-        "owner": "nix-systems",
-        "repo": "default",
-        "type": "github"
-      }
    }
  },
  "root": "root",
@@ -67,6 +67,7 @@ module.exports = {
  DC_EVENT_SMTP_MESSAGE_SENT: 103,
  DC_EVENT_WARNING: 300,
  DC_EVENT_WEBXDC_INSTANCE_DELETED: 2121,
+  DC_EVENT_WEBXDC_REALTIME_ADVERTISEMENT: 2151,
  DC_EVENT_WEBXDC_REALTIME_DATA: 2150,
  DC_EVENT_WEBXDC_STATUS_UPDATE: 2120,
  DC_GCL_ADD_ALLDONE_HINT: 4,
@@ -38,6 +38,7 @@ module.exports = {
  2120: 'DC_EVENT_WEBXDC_STATUS_UPDATE',
  2121: 'DC_EVENT_WEBXDC_INSTANCE_DELETED',
  2150: 'DC_EVENT_WEBXDC_REALTIME_DATA',
+  2151: 'DC_EVENT_WEBXDC_REALTIME_ADVERTISEMENT',
  2200: 'DC_EVENT_ACCOUNTS_BACKGROUND_FETCH_DONE',
  2300: 'DC_EVENT_CHATLIST_CHANGED',
  2301: 'DC_EVENT_CHATLIST_ITEM_CHANGED',
@@ -67,6 +67,7 @@ export enum C {
  DC_EVENT_SMTP_MESSAGE_SENT = 103,
  DC_EVENT_WARNING = 300,
  DC_EVENT_WEBXDC_INSTANCE_DELETED = 2121,
+  DC_EVENT_WEBXDC_REALTIME_ADVERTISEMENT = 2151,
  DC_EVENT_WEBXDC_REALTIME_DATA = 2150,
  DC_EVENT_WEBXDC_STATUS_UPDATE = 2120,
  DC_GCL_ADD_ALLDONE_HINT = 4,
@@ -343,6 +344,7 @@ export const EventId2EventName: { [key: number]: string } = {
  2120: 'DC_EVENT_WEBXDC_STATUS_UPDATE',
  2121: 'DC_EVENT_WEBXDC_INSTANCE_DELETED',
  2150: 'DC_EVENT_WEBXDC_REALTIME_DATA',
+  2151: 'DC_EVENT_WEBXDC_REALTIME_ADVERTISEMENT',
  2200: 'DC_EVENT_ACCOUNTS_BACKGROUND_FETCH_DONE',
  2300: 'DC_EVENT_CHATLIST_CHANGED',
  2301: 'DC_EVENT_CHATLIST_ITEM_CHANGED',
@@ -475,47 +475,6 @@ export class Context extends EventEmitter {
    return binding.dcn_get_msg_html(this.dcn_context, Number(messageId))
  }

-  getNextMediaMessage(
-    messageId: number,
-    msgType1: number,
-    msgType2: number,
-    msgType3: number
-  ) {
-    debug(
-      `getNextMediaMessage ${messageId} ${msgType1} ${msgType2} ${msgType3}`
-    )
-    return this._getNextMedia(messageId, 1, msgType1, msgType2, msgType3)
-  }
-
-  getPreviousMediaMessage(
-    messageId: number,
-    msgType1: number,
-    msgType2: number,
-    msgType3: number
-  ) {
-    debug(
-      `getPreviousMediaMessage ${messageId} ${msgType1} ${msgType2} ${msgType3}`
-    )
-    return this._getNextMedia(messageId, -1, msgType1, msgType2, msgType3)
-  }
-
-  _getNextMedia(
-    messageId: number,
-    dir: number,
-    msgType1: number,
-    msgType2: number,
-    msgType3: number
-  ): number {
-    return binding.dcn_get_next_media(
-      this.dcn_context,
-      Number(messageId),
-      dir,
-      msgType1 || 0,
-      msgType2 || 0,
-      msgType3 || 0
-    )
-  }
-
  getSecurejoinQrCode(chatId: number): string {
    debug(`getSecurejoinQrCode ${chatId}`)
    return binding.dcn_get_securejoin_qr(this.dcn_context, Number(chatId))
@@ -1053,27 +1053,6 @@ NAPI_METHOD(dcn_get_msg_html) {
  NAPI_RETURN_AND_UNREF_STRING(msg_html);
 }

-NAPI_METHOD(dcn_get_next_media) {
-  NAPI_ARGV(6);
-  NAPI_DCN_CONTEXT();
-  NAPI_ARGV_UINT32(msg_id, 1);
-  NAPI_ARGV_INT32(dir, 2);
-  NAPI_ARGV_INT32(msg_type1, 3);
-  NAPI_ARGV_INT32(msg_type2, 4);
-  NAPI_ARGV_INT32(msg_type3, 5);
-
-  //TRACE("calling..");
-  uint32_t next_id = dc_get_next_media(dcn_context->dc_context,
-                                       msg_id,
-                                       dir,
-                                       msg_type1,
-                                       msg_type2,
-                                       msg_type3);
-  //TRACE("result %d", next_id);
-
-  NAPI_RETURN_UINT32(next_id);
-}
-
 NAPI_METHOD(dcn_set_chat_visibility) {
  NAPI_ARGV(3);
  NAPI_DCN_CONTEXT();
@@ -3443,7 +3422,6 @@ NAPI_INIT() {
  NAPI_EXPORT_FUNCTION(dcn_get_msg_cnt);
  NAPI_EXPORT_FUNCTION(dcn_get_msg_info);
  NAPI_EXPORT_FUNCTION(dcn_get_msg_html);
-  NAPI_EXPORT_FUNCTION(dcn_get_next_media);
  NAPI_EXPORT_FUNCTION(dcn_set_chat_visibility);
  NAPI_EXPORT_FUNCTION(dcn_get_securejoin_qr);
  NAPI_EXPORT_FUNCTION(dcn_get_securejoin_qr_svg);
@@ -55,5 +55,5 @@
    "test:mocha": "mocha node/test/test.mjs --growl --reporter=spec --bail --exit"
  },
  "types": "node/dist/index.d.ts",
-  "version": "1.146.0"
+  "version": "1.147.1"
 }
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"

 [project]
 name = "deltachat"
-version = "1.146.0"
+version = "1.147.1"
 description = "Python bindings for the Delta Chat Core library using CFFI against the Rust-implemented libdeltachat"
 readme = "README.rst"
 requires-python = ">=3.7"
@@ -1 +1 @@
-2024-10-03
+2024-10-13
@@ -7,7 +7,7 @@ set -euo pipefail
 #
 # Avoid using rustup here as it depends on reading /proc/self/exe and
 # has problems running under QEMU.
-RUST_VERSION=1.81.0
+RUST_VERSION=1.82.0

 ARCH="$(uname -m)"
 test -f "/lib/libc.musl-$ARCH.so.1" && LIBC=musl || LIBC=gnu
@@ -31,6 +31,6 @@ unset CHATMAIL_DOMAIN

 # Try to build wheels for a range of interpreters, but don't fail if they are not available.
 # E.g. musllinux_1_1 does not have PyPy interpreters as of 2022-07-10
-tox --workdir "$TOXWORKDIR" -e py37,py38,py39,py310,py311,py312,pypy37,pypy38,pypy39,pypy310 --skip-missing-interpreters true
+tox --workdir "$TOXWORKDIR" -e py37,py38,py39,py310,py311,py312,py313,pypy37,pypy38,pypy39,pypy310 --skip-missing-interpreters true

 auditwheel repair "$TOXWORKDIR"/wheelhouse/deltachat* -w "$TOXWORKDIR/wheelhouse"
@@ -253,16 +253,16 @@ impl<'a> BlobObject<'a> {
    ///
    /// The extension part will always be lowercased.
    fn sanitise_name(name: &str) -> (String, String) {
-        let mut name = name.to_string();
+        let mut name = name;
        for part in name.rsplit('/') {
            if !part.is_empty() {
-                name = part.to_string();
+                name = part;
                break;
            }
        }
        for part in name.rsplit('\\') {
            if !part.is_empty() {
-                name = part.to_string();
+                name = part;
                break;
            }
        }
@@ -272,32 +272,39 @@ impl<'a> BlobObject<'a> {
            replacement: "",
        };

-        let clean = sanitize_filename::sanitize_with_options(name, opts);
-        // Let's take the tricky filename
+        let name = sanitize_filename::sanitize_with_options(name, opts);
+        // Let's take a tricky filename,
        // "file.with_lots_of_characters_behind_point_and_double_ending.tar.gz" as an example.
-        // Split it into "file" and "with_lots_of_characters_behind_point_and_double_ending.tar.gz":
-        let mut iter = clean.splitn(2, '.');
-
-        let stem: String = iter.next().unwrap_or_default().chars().take(64).collect();
-        // stem == "file"
-
-        let ext_chars = iter.next().unwrap_or_default().chars();
-        let ext: String = ext_chars
+        // Assume that the extension is 32 chars maximum.
+        let ext: String = name
+            .chars()
            .rev()
-            .take(32)
+            .take_while(|c| !c.is_whitespace())
+            .take(33)
            .collect::<Vec<_>>()
            .iter()
            .rev()
            .collect();
-        // ext == "d_point_and_double_ending.tar.gz"
+        // ext == "nd_point_and_double_ending.tar.gz"

-        if ext.is_empty() {
-            (stem, "".to_string())
+        // Split it into "nd_point_and_double_ending" and "tar.gz":
+        let mut iter = ext.splitn(2, '.');
+        iter.next();
+
+        let ext = iter.next().unwrap_or_default();
+        let ext = if ext.is_empty() {
+            String::new()
        } else {
-            (stem, format!(".{ext}").to_lowercase())
-            // Return ("file", ".d_point_and_double_ending.tar.gz")
-            // which is not perfect but acceptable.
-        }
+            format!(".{ext}")
+            // ".tar.gz"
+        };
+        let stem = name
+            .strip_suffix(&ext)
+            .unwrap_or_default()
+            .chars()
+            .take(64)
+            .collect();
+        (stem, ext.to_lowercase())
    }

    /// Checks whether a name is a valid blob name.
@@ -615,7 +622,7 @@ fn exif_orientation(exif: &exif::Exif, context: &Context) -> i32 {
    0
 }

-impl<'a> fmt::Display for BlobObject<'a> {
+impl fmt::Display for BlobObject<'_> {
    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
        write!(f, "$BLOBDIR/{}", self.name)
    }
@@ -666,10 +673,6 @@ impl<'a> BlobDirContents<'a> {
    pub(crate) fn iter(&self) -> BlobDirIter<'_> {
        BlobDirIter::new(self.context, self.inner.iter())
    }
-
-    pub(crate) fn len(&self) -> usize {
-        self.inner.len()
-    }
 }

 /// A iterator over all the [`BlobObject`]s in the blobdir.
@@ -967,6 +970,19 @@ mod tests {
        assert!(!stem.contains(':'));
        assert!(!stem.contains('*'));
        assert!(!stem.contains('?'));
+
+        let (stem, ext) = BlobObject::sanitise_name(
+            "file.with_lots_of_characters_behind_point_and_double_ending.tar.gz",
+        );
+        assert_eq!(
+            stem,
+            "file.with_lots_of_characters_behind_point_and_double_ending"
+        );
+        assert_eq!(ext, ".tar.gz");
+
+        let (stem, ext) = BlobObject::sanitise_name("a. tar.tar.gz");
+        assert_eq!(stem, "a. tar");
+        assert_eq!(ext, ".tar.gz");
    }

    #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
@@ -578,7 +578,7 @@ impl ChatId {
        Ok(())
    }

-    /// Sets protection and sends or adds a message.
+    /// Sets protection and adds a message.
    ///
    /// `timestamp_sort` is used as the timestamp of the added message
    /// and should be the timestamp of the change happening.
@@ -589,20 +589,16 @@ impl ChatId {
        timestamp_sort: i64,
        contact_id: Option<ContactId>,
    ) -> Result<()> {
-        match self.inner_set_protection(context, protect).await {
-            Ok(protection_status_modified) => {
-                if protection_status_modified {
-                    self.add_protection_msg(context, protect, contact_id, timestamp_sort)
-                        .await?;
-                    chatlist_events::emit_chatlist_item_changed(context, self);
-                }
-                Ok(())
-            }
-            Err(e) => {
-                error!(context, "Cannot set protection: {e:#}."); // make error user-visible
-                Err(e)
-            }
+        let protection_status_modified = self
+            .inner_set_protection(context, protect)
+            .await
+            .with_context(|| format!("Cannot set protection for {self}"))?;
+        if protection_status_modified {
+            self.add_protection_msg(context, protect, contact_id, timestamp_sort)
+                .await?;
+            chatlist_events::emit_chatlist_item_changed(context, self);
        }
+        Ok(())
    }

    /// Sets protection and sends or adds a message.
@@ -616,8 +612,9 @@ impl ChatId {
        contact_id: Option<ContactId>,
    ) -> Result<()> {
        let sort_to_bottom = true;
+        let (received, incoming) = (false, false);
        let ts = self
-            .calc_sort_timestamp(context, timestamp_sent, sort_to_bottom, false)
+            .calc_sort_timestamp(context, timestamp_sent, sort_to_bottom, received, incoming)
            .await?
            // Always sort protection messages below `SystemMessage::SecurejoinWait{,Timeout}` ones
            // in case of race conditions.
@@ -869,13 +866,14 @@ impl ChatId {
    ///
    /// Returns `true`, if message was deleted, `false` otherwise.
    async fn maybe_delete_draft(self, context: &Context) -> Result<bool> {
-        match self.get_draft_msg_id(context).await? {
-            Some(msg_id) => {
-                msg_id.delete_from_db(context).await?;
-                Ok(true)
-            }
-            None => Ok(false),
-        }
+        Ok(context
+            .sql
+            .execute(
+                "DELETE FROM msgs WHERE chat_id=? AND state=?",
+                (self, MessageState::OutDraft),
+            )
+            .await?
+            > 0)
    }

    /// Set provided message as draft message for specified chat.
@@ -947,12 +945,18 @@ impl ChatId {
            }
        }

-        // insert new draft
-        self.maybe_delete_draft(context).await?;
        let row_id = context
            .sql
-            .insert(
-                "INSERT INTO msgs (
+            .transaction(|transaction| {
+                // Delete existing draft if it exists.
+                transaction.execute(
+                    "DELETE FROM msgs WHERE chat_id=? AND state=?",
+                    (self, MessageState::OutDraft),
+                )?;
+
+                // Insert new draft.
+                transaction.execute(
+                    "INSERT INTO msgs (
                 chat_id,
                 from_id,
                 timestamp,
@@ -964,19 +968,22 @@ impl ChatId {
                 hidden,
                 mime_in_reply_to)
         VALUES (?,?,?,?,?,?,?,?,?,?);",
-                (
-                    self,
-                    ContactId::SELF,
-                    time(),
-                    msg.viewtype,
-                    MessageState::OutDraft,
-                    &msg.text,
-                    message::normalize_text(&msg.text),
-                    msg.param.to_string(),
-                    1,
-                    msg.in_reply_to.as_deref().unwrap_or_default(),
-                ),
-            )
+                    (
+                        self,
+                        ContactId::SELF,
+                        time(),
+                        msg.viewtype,
+                        MessageState::OutDraft,
+                        &msg.text,
+                        message::normalize_text(&msg.text),
+                        msg.param.to_string(),
+                        1,
+                        msg.in_reply_to.as_deref().unwrap_or_default(),
+                    ),
+                )?;
+
+                Ok(transaction.last_insert_rowid())
+            })
            .await?;
        msg.id = MsgId::new(row_id.try_into()?);
        Ok(true)
@@ -1042,7 +1049,13 @@ impl ChatId {
    pub(crate) async fn get_timestamp(self, context: &Context) -> Result<Option<i64>> {
        let timestamp = context
            .sql
-            .query_get_value("SELECT MAX(timestamp) FROM msgs WHERE chat_id=?", (self,))
+            .query_get_value(
+                "SELECT MAX(timestamp)
+                 FROM msgs
+                 WHERE chat_id=?
+                 HAVING COUNT(*) > 0",
+                (self,),
+            )
            .await?;
        Ok(timestamp)
    }
@@ -1228,6 +1241,7 @@ impl ChatId {
             AND ((state BETWEEN {} AND {}) OR (state >= {})) \
             AND NOT hidden \
             AND download_state={} \
+             AND from_id != {} \
             ORDER BY timestamp DESC, id DESC \
             LIMIT 1;",
            MessageState::InFresh as u32,
@@ -1236,6 +1250,9 @@ impl ChatId {
            // Do not reply to not fully downloaded messages. Such a message could be a group chat
            // message that we assigned to 1:1 chat.
            DownloadState::Done as u32,
+            // Do not reference info messages, they are not actually sent out
+            // and have Message-IDs unknown to other chat members.
+            ContactId::INFO.to_u32(),
        );
        sql.query_row_optional(&query, (self,), f).await
    }
@@ -1247,7 +1264,7 @@ impl ChatId {
    ) -> Result<Option<(String, String, String)>> {
        self.parent_query(
            context,
-            "rfc724_mid, mime_in_reply_to, mime_references",
+            "rfc724_mid, mime_in_reply_to, IFNULL(mime_references, '')",
            state_out_min,
            |row: &rusqlite::Row| {
                let rfc724_mid: String = row.get(0)?;
@@ -1382,12 +1399,14 @@ impl ChatId {
    /// corresponding event in case of a system message (usually the current system time).
    /// `always_sort_to_bottom` makes this ajust the returned timestamp up so that the message goes
    /// to the chat bottom.
+    /// `received` -- whether the message is received. Otherwise being sent.
    /// `incoming` -- whether the message is incoming.
    pub(crate) async fn calc_sort_timestamp(
        self,
        context: &Context,
        message_timestamp: i64,
        always_sort_to_bottom: bool,
+        received: bool,
        incoming: bool,
    ) -> Result<i64> {
        let mut sort_timestamp = cmp::min(message_timestamp, smeared_time(context));
@@ -1401,26 +1420,45 @@ impl ChatId {
            context
                .sql
                .query_get_value(
-                    "SELECT MAX(timestamp) FROM msgs WHERE chat_id=? AND state!=?",
+                    "SELECT MAX(timestamp)
+                     FROM msgs
+                     WHERE chat_id=? AND state!=?
+                     HAVING COUNT(*) > 0",
                    (self, MessageState::OutDraft),
                )
                .await?
-        } else if incoming {
-            // get newest non fresh message for this chat.
-
-            // If a user hasn't been online for some time, the Inbox is fetched first and then the
-            // Sentbox. In order for Inbox and Sent messages to be allowed to mingle, outgoing
-            // messages are purely sorted by their sent timestamp. NB: The Inbox must be fetched
-            // first otherwise Inbox messages would be always below old Sentbox messages. We could
-            // take in the query below only incoming messages, but then new incoming messages would
-            // mingle with just sent outgoing ones and apear somewhere in the middle of the chat.
+        } else if received {
+            // Received messages shouldn't mingle with just sent ones and appear somewhere in the
+            // middle of the chat, so we go after the newest non fresh message.
+            //
+            // But if a received outgoing message is older than some seen message, better sort the
+            // received message purely by timestamp. We could place it just before that seen
+            // message, but anyway the user may not notice it.
+            //
+            // NB: Received outgoing messages may break sorting of fresh incoming ones, but this
+            // shouldn't happen frequently. Seen incoming messages don't really break sorting of
+            // fresh ones, they rather mean that older incoming messages are actually seen as well.
            context
                .sql
-                .query_get_value(
-                    "SELECT MAX(timestamp) FROM msgs WHERE chat_id=? AND hidden=0 AND state>?",
-                    (self, MessageState::InFresh),
+                .query_row_optional(
+                    "SELECT MAX(timestamp), MAX(IIF(state=?,timestamp_sent,0))
+                     FROM msgs
+                     WHERE chat_id=? AND hidden=0 AND state>?
+                     HAVING COUNT(*) > 0",
+                    (MessageState::InSeen, self, MessageState::InFresh),
+                    |row| {
+                        let ts: i64 = row.get(0)?;
+                        let ts_sent_seen: i64 = row.get(1)?;
+                        Ok((ts, ts_sent_seen))
+                    },
                )
                .await?
+                .and_then(|(ts, ts_sent_seen)| {
+                    match incoming || ts_sent_seen <= message_timestamp {
+                        true => Some(ts),
+                        false => None,
+                    }
+                })
        } else {
            None
        };
@@ -2903,7 +2941,10 @@ async fn prepare_send_msg(
        );
        message::update_msg_state(context, msg.id, MessageState::OutPending).await?;
    }
-    create_send_msg_jobs(context, msg).await
+    let row_ids = create_send_msg_jobs(context, msg)
+        .await
+        .context("Failed to create send jobs")?;
+    Ok(row_ids)
 }

 /// Constructs jobs for sending a message and inserts them into the appropriate table.
@@ -3416,65 +3457,6 @@ pub async fn get_chat_media(
    Ok(list)
 }

-/// Indicates the direction over which to iterate.
-#[derive(Debug, Clone, PartialEq, Eq)]
-#[repr(i32)]
-pub enum Direction {
-    /// Search forward.
-    Forward = 1,
-
-    /// Search backward.
-    Backward = -1,
-}
-
-/// Searches next/previous message based on the given message and list of types.
-///
-/// Deprecated since 2023-10-03.
-#[deprecated(note = "use `get_chat_media` instead")]
-pub async fn get_next_media(
-    context: &Context,
-    curr_msg_id: MsgId,
-    direction: Direction,
-    msg_type: Viewtype,
-    msg_type2: Viewtype,
-    msg_type3: Viewtype,
-) -> Result<Option<MsgId>> {
-    let mut ret: Option<MsgId> = None;
-
-    if let Ok(msg) = Message::load_from_db(context, curr_msg_id).await {
-        let list: Vec<MsgId> = get_chat_media(
-            context,
-            Some(msg.chat_id),
-            if msg_type != Viewtype::Unknown {
-                msg_type
-            } else {
-                msg.viewtype
-            },
-            msg_type2,
-            msg_type3,
-        )
-        .await?;
-        for (i, msg_id) in list.iter().enumerate() {
-            if curr_msg_id == *msg_id {
-                match direction {
-                    Direction::Forward => {
-                        if i + 1 < list.len() {
-                            ret = list.get(i + 1).copied();
-                        }
-                    }
-                    Direction::Backward => {
-                        if i >= 1 {
-                            ret = list.get(i - 1).copied();
-                        }
-                    }
-                }
-                break;
-            }
-        }
-    }
-    Ok(ret)
-}
-
 /// Returns a vector of contact IDs for given chat ID.
 pub async fn get_chat_contacts(context: &Context, chat_id: ChatId) -> Result<Vec<ContactId>> {
    // Normal chats do not include SELF.  Group chats do (as it may happen that one is deleted from a
@@ -3938,63 +3920,71 @@ pub async fn remove_contact_from_chat(
    let mut msg = Message::default();

    let chat = Chat::load_from_db(context, chat_id).await?;
-    if chat.typ == Chattype::Group || chat.typ == Chattype::Broadcast {
-        if !chat.is_self_in_chat(context).await? {
-            let err_msg = format!(
-                "Cannot remove contact {contact_id} from chat {chat_id}: self not in group."
-            );
-            context.emit_event(EventType::ErrorSelfNotInGroup(err_msg.clone()));
-            bail!("{}", err_msg);
-        } else {
-            let mut sync = Nosync;
-            // We do not return an error if the contact does not exist in the database.
-            // This allows to delete dangling references to deleted contacts
-            // in case of the database becoming inconsistent due to a bug.
-            if let Some(contact) = Contact::get_by_id_optional(context, contact_id).await? {
-                if chat.typ == Chattype::Group && chat.is_promoted() {
-                    msg.viewtype = Viewtype::Text;
-                    if contact_id == ContactId::SELF {
-                        msg.text = stock_str::msg_group_left_local(context, ContactId::SELF).await;
-                    } else {
-                        msg.text = stock_str::msg_del_member_local(
-                            context,
-                            contact.get_addr(),
-                            ContactId::SELF,
-                        )
-                        .await;
-                    }
-                    msg.param.set_cmd(SystemMessage::MemberRemovedFromGroup);
-                    msg.param.set(Param::Arg, contact.get_addr().to_lowercase());
-                    let res = send_msg(context, chat_id, &mut msg).await;
-                    if contact_id == ContactId::SELF {
-                        res?;
-                        set_group_explicitly_left(context, &chat.grpid).await?;
-                    } else if let Err(e) = res {
-                        warn!(context, "remove_contact_from_chat({chat_id}, {contact_id}): send_msg() failed: {e:#}.");
-                    }
+    if chat.typ != Chattype::Group && chat.typ != Chattype::Broadcast {
+        bail!("Cannot remove members from non-group chats.");
+    }
+
+    if !chat.is_self_in_chat(context).await? {
+        let err_msg =
+            format!("Cannot remove contact {contact_id} from chat {chat_id}: self not in group.");
+        context.emit_event(EventType::ErrorSelfNotInGroup(err_msg.clone()));
+        bail!("{}", err_msg);
+    } else {
+        let mut sync = Nosync;
+        // We do not return an error if the contact does not exist in the database.
+        // This allows to delete dangling references to deleted contacts
+        // in case of the database becoming inconsistent due to a bug.
+        if let Some(contact) = Contact::get_by_id_optional(context, contact_id).await? {
+            if chat.typ == Chattype::Group && chat.is_promoted() {
+                msg.viewtype = Viewtype::Text;
+                if contact_id == ContactId::SELF {
+                    msg.text = stock_str::msg_group_left_local(context, ContactId::SELF).await;
                } else {
-                    sync = Sync;
+                    msg.text = stock_str::msg_del_member_local(
+                        context,
+                        contact.get_addr(),
+                        ContactId::SELF,
+                    )
+                    .await;
                }
-            }
-            // we remove the member from the chat after constructing the
-            // to-be-send message. If between send_msg() and here the
-            // process dies, the user will be able to redo the action. It's better than the other
-            // way round: you removed someone from DB but no peer or device gets to know about it
-            // and group membership is thus different on different devices. But if send_msg()
-            // failed, we still remove the member locally, otherwise it would be impossible to
-            // remove a member with missing key from a protected group.
-            // Note also that sending a message needs all recipients
-            // in order to correctly determine encryption so if we
-            // removed it first, it would complicate the
-            // check/encryption logic.
-            remove_from_chat_contacts_table(context, chat_id, contact_id).await?;
-            context.emit_event(EventType::ChatModified(chat_id));
-            if sync.into() {
-                chat.sync_contacts(context).await.log_err(context).ok();
+                msg.param.set_cmd(SystemMessage::MemberRemovedFromGroup);
+                msg.param.set(Param::Arg, contact.get_addr().to_lowercase());
+                let res = send_msg(context, chat_id, &mut msg)
+                    .await
+                    .context("Failed to send member removed message");
+                if contact_id == ContactId::SELF {
+                    res?;
+                    set_group_explicitly_left(context, &chat.grpid)
+                        .await
+                        .context("Failed to mark group as explicitly left")?;
+                } else if let Err(e) = res {
+                    warn!(
+                        context,
+                        "remove_contact_from_chat({chat_id}, {contact_id}): {e:#}."
+                    );
+                }
+            } else {
+                sync = Sync;
            }
        }
-    } else {
-        bail!("Cannot remove members from non-group chats.");
+        // we remove the member from the chat after constructing the
+        // to-be-send message. If between send_msg() and here the
+        // process dies, the user will be able to redo the action. It's better than the other
+        // way round: you removed someone from DB but no peer or device gets to know about it
+        // and group membership is thus different on different devices. But if send_msg()
+        // failed, we still remove the member locally, otherwise it would be impossible to
+        // remove a member with missing key from a protected group.
+        // Note also that sending a message needs all recipients
+        // in order to correctly determine encryption so if we
+        // removed it first, it would complicate the
+        // check/encryption logic.
+        remove_from_chat_contacts_table(context, chat_id, contact_id)
+            .await
+            .context("Failed to remove contact from chats_contacts table")?;
+        context.emit_event(EventType::ChatModified(chat_id));
+        if sync.into() {
+            chat.sync_contacts(context).await.log_err(context).ok();
+        }
    }

    Ok(())
@@ -4152,7 +4142,9 @@ pub async fn forward_msgs(context: &Context, msg_ids: &[MsgId], chat_id: ChatId)
    chat_id
        .unarchive_if_not_muted(context, MessageState::Undefined)
        .await?;
-    let mut chat = Chat::load_from_db(context, chat_id).await?;
+    let mut chat = Chat::load_from_db(context, chat_id)
+        .await
+        .context("Failed to load chat from the database")?;
    if let Some(reason) = chat.why_cant_send(context).await? {
        bail!("cannot send to {}: {}", chat_id, reason);
    }
@@ -4168,7 +4160,8 @@ pub async fn forward_msgs(context: &Context, msg_ids: &[MsgId], chat_id: ChatId)
            |row| row.get::<_, MsgId>(0),
            |ids| ids.collect::<Result<Vec<_>, _>>().map_err(Into::into),
        )
-        .await?;
+        .await
+        .context("Failed to sort forwarded messages by timestamp")?;

    for id in ids {
        let src_msg_id: MsgId = id;
@@ -4223,9 +4216,14 @@ pub async fn forward_msgs(context: &Context, msg_ids: &[MsgId], chat_id: ChatId)
            msg.state = MessageState::OutPending;
            new_msg_id = chat
                .prepare_msg_raw(context, &mut msg, None, curr_timestamp)
-                .await?;
+                .await
+                .context("Failed to prepare forwarded message")?;
            curr_timestamp += 1;
-            if !create_send_msg_jobs(context, &mut msg).await?.is_empty() {
+            if !create_send_msg_jobs(context, &mut msg)
+                .await
+                .context("Failed to create send jobs for the forwarded message")?
+                .is_empty()
+            {
                context.scheduler.interrupt_smtp().await;
            }
        }
@@ -4400,7 +4398,10 @@ pub async fn add_device_msg_with_importance(
        if let Some(last_msg_time) = context
            .sql
            .query_get_value(
-                "SELECT MAX(timestamp) FROM msgs WHERE chat_id=?",
+                "SELECT MAX(timestamp)
+                 FROM msgs
+                 WHERE chat_id=?
+                 HAVING COUNT(*) > 0",
                (chat_id,),
            )
            .await?
@@ -4726,6 +4727,7 @@ mod tests {
    use super::*;
    use crate::chatlist::get_archived_cnt;
    use crate::constants::{DC_GCL_ARCHIVED_ONLY, DC_GCL_NO_SPECIALS};
+    use crate::headerdef::HeaderDef;
    use crate::message::delete_msgs;
    use crate::receive_imf::receive_imf;
    use crate::test_utils::{sync, TestContext, TestContextManager};
@@ -4881,6 +4883,37 @@ mod tests {
        Ok(())
    }

+    #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
+    async fn test_only_one_draft_per_chat() -> Result<()> {
+        let t = TestContext::new_alice().await;
+        let chat_id = create_group_chat(&t, ProtectionStatus::Unprotected, "abc").await?;
+
+        let msgs: Vec<message::Message> = (1..=1000)
+            .map(|i| {
+                let mut msg = Message::new(Viewtype::Text);
+                msg.set_text(i.to_string());
+                msg
+            })
+            .collect();
+        let mut tasks = Vec::new();
+        for mut msg in msgs {
+            let ctx = t.clone();
+            let task = tokio::spawn(async move {
+                let ctx = ctx;
+                chat_id.set_draft(&ctx, Some(&mut msg)).await
+            });
+            tasks.push(task);
+        }
+        futures::future::join_all(tasks.into_iter()).await;
+
+        assert!(chat_id.get_draft(&t).await?.is_some());
+
+        chat_id.set_draft(&t, None).await?;
+        assert!(chat_id.get_draft(&t).await?.is_none());
+
+        Ok(())
+    }
+
    #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
    async fn test_change_quotes_on_reused_message_object() -> Result<()> {
        let t = TestContext::new_alice().await;
@@ -7681,4 +7714,29 @@ mod tests {

        Ok(())
    }
+
+    /// Tests that info message is ignored when constructing `In-Reply-To`.
+    #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
+    async fn test_info_not_referenced() -> Result<()> {
+        let mut tcm = TestContextManager::new();
+        let alice = &tcm.alice().await;
+        let bob = &tcm.bob().await;
+
+        let bob_received_message = tcm.send_recv_accept(alice, bob, "Hi!").await;
+        let bob_chat_id = bob_received_message.chat_id;
+        add_info_msg(bob, bob_chat_id, "Some info", create_smeared_timestamp(bob)).await?;
+
+        // Bob sends a message.
+        // This message should reference Alice's "Hi!" message and not the info message.
+        let sent = bob.send_text(bob_chat_id, "Hi hi!").await;
+        let mime_message = alice.parse_msg(&sent).await;
+
+        let in_reply_to = mime_message.get_header(HeaderDef::InReplyTo).unwrap();
+        assert_eq!(
+            in_reply_to,
+            format!("<{}>", bob_received_message.rfc724_mid)
+        );
+
+        Ok(())
+    }
 }
@@ -857,6 +857,8 @@ impl Context {
    ///
    /// This should only be used by test code and during configure.
    pub(crate) async fn set_primary_self_addr(&self, primary_new: &str) -> Result<()> {
+        self.quota.write().await.take();
+
        // add old primary address (if exists) to secondary addresses
        let mut secondary_addrs = self.get_all_self_addrs().await?;
        // never store a primary address also as a secondary
@@ -869,7 +871,7 @@ impl Context {

        self.set_config_internal(Config::ConfiguredAddr, Some(primary_new))
            .await?;
-
+        self.emit_event(EventType::ConnectivityChanged);
        Ok(())
    }

@@ -13,7 +13,7 @@ mod auto_mozilla;
 mod auto_outlook;
 pub(crate) mod server_params;

-use anyhow::{bail, ensure, Context as _, Result};
+use anyhow::{bail, ensure, format_err, Context as _, Result};
 use auto_mozilla::moz_autoconfigure;
 use auto_outlook::outlk_autodiscover;
 use deltachat_contact_tools::EmailAddress;
@@ -80,10 +80,7 @@ impl Context {

        let res = self
            .inner_configure()
-            .race(cancel_channel.recv().map(|_| {
-                progress!(self, 0);
-                Ok(())
-            }))
+            .race(cancel_channel.recv().map(|_| Err(format_err!("Cancelled"))))
            .await;

        self.free_ongoing().await;
@@ -179,7 +179,9 @@ pub const DC_DESIRED_TEXT_LEN: usize = DC_DESIRED_TEXT_LINE_LEN * DC_DESIRED_TEX
 // and may be set together with the username, password etc.
 // via dc_set_config() using the key "server_flags".

-/// Force OAuth2 authorization. This flag does not skip automatic configuration.
+/// Force OAuth2 authorization.
+///
+/// This flag does not skip automatic configuration.
 /// Before calling configure() with DC_LP_AUTH_OAUTH2 set,
 /// the user has to confirm access at the URL returned by dc_get_oauth2_url().
 pub const DC_LP_AUTH_OAUTH2: i32 = 0x2;
@@ -515,8 +515,11 @@ impl Context {
        Ok(val)
    }

-    /// Does a background fetch
-    /// pauses the scheduler and does one imap fetch, then unpauses and returns
+    /// Does a single round of fetching from IMAP and returns.
+    ///
+    /// Can be used even if I/O is currently stopped.
+    /// If I/O is currently stopped, starts a new IMAP connection
+    /// and fetches from Inbox and DeltaChat folders.
    pub async fn background_fetch(&self) -> Result<()> {
        if !(self.is_configured().await?) {
            return Ok(());
@@ -524,35 +527,63 @@ impl Context {

        let address = self.get_primary_self_addr().await?;
        let time_start = tools::Time::now();
-        info!(self, "background_fetch started fetching {address}");
+        info!(self, "background_fetch started fetching {address}.");

-        let _pause_guard = self.scheduler.pause(self.clone()).await?;
+        if self.scheduler.is_running().await {
+            self.scheduler.maybe_network().await;

-        // connection
-        let mut connection = Imap::new_configured(self, channel::bounded(1).1).await?;
-        let mut session = connection.prepare(self).await?;
+            // Wait until fetching is finished.
+            // Ideally we could wait for connectivity change events,
+            // but sleep loop is good enough.

-        // fetch imap folders
-        for folder_meaning in [FolderMeaning::Inbox, FolderMeaning::Mvbox] {
-            let (_, watch_folder) = convert_folder_meaning(self, folder_meaning).await?;
-            connection
-                .fetch_move_delete(self, &mut session, &watch_folder, folder_meaning)
-                .await?;
-        }
+            // First 100 ms sleep in chunks of 10 ms.
+            for _ in 0..10 {
+                if self.all_work_done().await {
+                    break;
+                }
+                tokio::time::sleep(std::time::Duration::from_millis(10)).await;
+            }

-        // update quota (to send warning if full) - but only check it once in a while
-        if self
-            .quota_needs_update(DC_BACKGROUND_FETCH_QUOTA_CHECK_RATELIMIT)
-            .await
-        {
-            if let Err(err) = self.update_recent_quota(&mut session).await {
-                warn!(self, "Failed to update quota: {err:#}.");
+            // If we are not finished in 100 ms, keep waking up every 100 ms.
+            while !self.all_work_done().await {
+                tokio::time::sleep(std::time::Duration::from_millis(100)).await;
+            }
+        } else {
+            // Pause the scheduler to ensure another connection does not start
+            // while we are fetching on a dedicated connection.
+            let _pause_guard = self.scheduler.pause(self.clone()).await?;
+
+            // Start a new dedicated connection.
+            let mut connection = Imap::new_configured(self, channel::bounded(1).1).await?;
+            let mut session = connection.prepare(self).await?;
+
+            // Fetch IMAP folders.
+            // Inbox is fetched before Mvbox because fetching from Inbox
+            // may result in moving some messages to Mvbox.
+            for folder_meaning in [FolderMeaning::Inbox, FolderMeaning::Mvbox] {
+                if let Some((_folder_config, watch_folder)) =
+                    convert_folder_meaning(self, folder_meaning).await?
+                {
+                    connection
+                        .fetch_move_delete(self, &mut session, &watch_folder, folder_meaning)
+                        .await?;
+                }
+            }
+
+            // Update quota (to send warning if full) - but only check it once in a while.
+            if self
+                .quota_needs_update(DC_BACKGROUND_FETCH_QUOTA_CHECK_RATELIMIT)
+                .await
+            {
+                if let Err(err) = self.update_recent_quota(&mut session).await {
+                    warn!(self, "Failed to update quota: {err:#}.");
+                }
            }
        }

        info!(
            self,
-            "background_fetch done for {address} took {:?}",
+            "background_fetch done for {address} took {:?}.",
            time_elapsed(&time_start),
        );

@@ -313,7 +313,7 @@ pub(crate) async fn get_autocrypt_peerstate(
        if let Some(ref mut peerstate) = peerstate {
            if addr_cmp(&peerstate.addr, from) {
                if allow_change {
-                    peerstate.apply_header(header, message_time);
+                    peerstate.apply_header(context, header, message_time);
                    peerstate.save_to_db(&context.sql).await?;
                } else {
                    info!(
@@ -98,19 +98,26 @@ impl MsgId {
        Ok(())
    }

+    /// Updates the message download state. Returns `Ok` if the message doesn't exist anymore.
    pub(crate) async fn update_download_state(
        self,
        context: &Context,
        download_state: DownloadState,
    ) -> Result<()> {
-        let msg = Message::load_from_db(context, self).await?;
-        context
+        if context
            .sql
            .execute(
                "UPDATE msgs SET download_state=? WHERE id=?;",
                (download_state, self),
            )
-            .await?;
+            .await?
+            == 0
+        {
+            return Ok(());
+        }
+        let Some(msg) = Message::load_from_db_optional(context, self).await? else {
+            return Ok(());
+        };
        context.emit_event(EventType::MsgsChanged {
            chat_id: msg.chat_id,
            msg_id: self,
@@ -322,11 +329,19 @@ mod tests {
            DownloadState::InProgress,
            DownloadState::Failure,
            DownloadState::Done,
+            DownloadState::Done,
        ] {
            msg_id.update_download_state(&t, *s).await?;
            let msg = Message::load_from_db(&t, msg_id).await?;
            assert_eq!(msg.download_state(), *s);
        }
+        t.sql
+            .execute("DELETE FROM msgs WHERE id=?", (msg_id,))
+            .await?;
+        // Nothing to do is ok.
+        msg_id
+            .update_download_state(&t, DownloadState::Done)
+            .await?;

        Ok(())
    }
@@ -69,7 +69,7 @@ use std::num::ParseIntError;
 use std::str::FromStr;
 use std::time::{Duration, UNIX_EPOCH};

-use anyhow::{ensure, Result};
+use anyhow::{ensure, Context as _, Result};
 use async_channel::Receiver;
 use serde::{Deserialize, Serialize};
 use tokio::time::timeout;
@@ -176,9 +176,13 @@ impl ChatId {
    pub async fn get_ephemeral_timer(self, context: &Context) -> Result<Timer> {
        let timer = context
            .sql
-            .query_get_value("SELECT ephemeral_timer FROM chats WHERE id=?;", (self,))
-            .await?;
-        Ok(timer.unwrap_or_default())
+            .query_get_value(
+                "SELECT IFNULL(ephemeral_timer, 0) FROM chats WHERE id=?",
+                (self,),
+            )
+            .await?
+            .with_context(|| format!("Chat {self} not found"))?;
+        Ok(timer)
    }

    /// Set ephemeral timer value without sending a message.
@@ -509,7 +513,8 @@ async fn next_delete_device_after_timestamp(context: &Context) -> Result<Option<
                FROM msgs
                WHERE chat_id > ?
                  AND chat_id != ?
-                  AND chat_id != ?;
+                  AND chat_id != ?
+                HAVING count(*) > 0
                "#,
                (DC_CHAT_ID_TRASH, self_chat_id, device_chat_id),
            )
@@ -533,7 +538,8 @@ async fn next_expiration_timestamp(context: &Context) -> Option<i64> {
            SELECT min(ephemeral_timestamp)
            FROM msgs
            WHERE ephemeral_timestamp != 0
-              AND chat_id != ?;
+              AND chat_id != ?
+            HAVING count(*) > 0
            "#,
            (DC_CHAT_ID_TRASH,), // Trash contains already deleted messages, skip them
        )
@@ -1410,4 +1416,14 @@ mod tests {

        Ok(())
    }
+
+    /// Tests that `.get_ephemeral_timer()` returns an error for invalid chat ID.
+    #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
+    async fn test_get_ephemeral_timer_wrong_chat_id() -> Result<()> {
+        let context = TestContext::new().await;
+        let chat_id = ChatId::new(12345);
+        assert!(chat_id.get_ephemeral_timer(&context).await.is_err());
+
+        Ok(())
+    }
 }
@@ -288,6 +288,13 @@ pub enum EventType {
        data: Vec<u8>,
    },

+    /// Advertisement received over an ephemeral peer channel.
+    /// This can be used by bots to initiate peer-to-peer communication from their side.
+    WebxdcRealtimeAdvertisementReceived {
+        /// Message ID of the webxdc instance.
+        msg_id: MsgId,
+    },
+
    /// Inform that a message containing a webxdc instance has been deleted.
    WebxdcInstanceDeleted {
        /// ID of the deleted message.
@@ -39,6 +39,7 @@ use crate::login_param::{
 use crate::message::{self, Message, MessageState, MessengerMessage, MsgId, Viewtype};
 use crate::mimeparser;
 use crate::net::proxy::ProxyConfig;
+use crate::net::session::SessionStream;
 use crate::oauth2::get_oauth2_access_token;
 use crate::receive_imf::{
    from_field_to_contact_id, get_prefetch_parent_message, receive_imf_inner, ReceivedMsg,
@@ -55,7 +56,7 @@ pub mod scan_folders;
 pub mod select_folder;
 pub(crate) mod session;

-use client::Client;
+use client::{determine_capabilities, Client};
 use mailparse::SingleInfo;
 use session::Session;

@@ -376,7 +377,23 @@ impl Imap {
            };

            match login_res {
-                Ok(session) => {
+                Ok(mut session) => {
+                    let capabilities = determine_capabilities(&mut session).await?;
+
+                    let session = if capabilities.can_compress {
+                        info!(context, "Enabling IMAP compression.");
+                        let compressed_session = session
+                            .compress(|s| {
+                                let session_stream: Box<dyn SessionStream> = Box::new(s);
+                                session_stream
+                            })
+                            .await
+                            .context("Failed to enable IMAP compression")?;
+                        Session::new(compressed_session, capabilities)
+                    } else {
+                        Session::new(session, capabilities)
+                    };
+
                    // Store server ID in the context to display in account info.
                    let mut lock = context.server_id.write().await;
                    lock.clone_from(&session.capabilities.server_id);
@@ -25,6 +25,10 @@ pub(crate) struct Capabilities {
    /// <https://tools.ietf.org/html/rfc5464>
    pub can_metadata: bool,

+    /// True if the server has COMPRESS=DEFLATE capability as defined in
+    /// <https://tools.ietf.org/html/rfc4978>
+    pub can_compress: bool,
+
    /// True if the server supports XDELTAPUSH capability.
    /// This capability means setting /private/devicetoken IMAP METADATA
    /// on the INBOX results in new mail notifications
@@ -7,7 +7,6 @@ use async_imap::Session as ImapSession;
 use tokio::io::BufWriter;

 use super::capabilities::Capabilities;
-use super::session::Session;
 use crate::context::Context;
 use crate::login_param::{ConnectionCandidate, ConnectionSecurity};
 use crate::net::dns::{lookup_host_with_cache, update_connect_timestamp};
@@ -51,7 +50,7 @@ fn alpn(port: u16) -> &'static [&'static str] {
 /// Determine server capabilities.
 ///
 /// If server supports ID capability, send our client ID.
-async fn determine_capabilities(
+pub(crate) async fn determine_capabilities(
    session: &mut ImapSession<Box<dyn SessionStream>>,
 ) -> Result<Capabilities> {
    let caps = session
@@ -69,6 +68,7 @@ async fn determine_capabilities(
        can_check_quota: caps.has_str("QUOTA"),
        can_condstore: caps.has_str("CONDSTORE"),
        can_metadata: caps.has_str("METADATA"),
+        can_compress: caps.has_str("COMPRESS=DEFLATE"),
        can_push: caps.has_str("XDELTAPUSH"),
        is_chatmail: caps.has_str("XCHATMAIL"),
        server_id,
@@ -83,28 +83,31 @@ impl Client {
        }
    }

-    pub(crate) async fn login(self, username: &str, password: &str) -> Result<Session> {
+    pub(crate) async fn login(
+        self,
+        username: &str,
+        password: &str,
+    ) -> Result<ImapSession<Box<dyn SessionStream>>> {
        let Client { inner, .. } = self;
-        let mut session = inner
+
+        let session = inner
            .login(username, password)
            .await
            .map_err(|(err, _client)| err)?;
-        let capabilities = determine_capabilities(&mut session).await?;
-        Ok(Session::new(session, capabilities))
+        Ok(session)
    }

    pub(crate) async fn authenticate(
        self,
        auth_type: &str,
        authenticator: impl async_imap::Authenticator,
-    ) -> Result<Session> {
+    ) -> Result<ImapSession<Box<dyn SessionStream>>> {
        let Client { inner, .. } = self;
-        let mut session = inner
+        let session = inner
            .authenticate(auth_type, authenticator)
            .await
            .map_err(|(err, _client)| err)?;
-        let capabilities = determine_capabilities(&mut session).await?;
-        Ok(Session::new(session, capabilities))
+        Ok(session)
    }

    async fn connection_attempt(
@@ -2,13 +2,16 @@

 use std::ffi::OsStr;
 use std::path::{Path, PathBuf};
+use std::pin::Pin;

 use ::pgp::types::KeyTrait;
 use anyhow::{bail, ensure, format_err, Context as _, Result};
 use futures::TryStreamExt;
 use futures_lite::FutureExt;
+use pin_project::pin_project;

 use tokio::fs::{self, File};
+use tokio::io::{AsyncRead, AsyncWrite, ReadBuf};
 use tokio_tar::Archive;

 use crate::blob::BlobDirContents;
@@ -212,7 +215,7 @@ async fn imex_inner(
        path.display()
    );
    ensure!(context.sql.is_open().await, "Database not opened.");
-    context.emit_event(EventType::ImexProgress(10));
+    context.emit_event(EventType::ImexProgress(1));

    if what == ImexMode::ExportBackup || what == ImexMode::ExportSelfKeys {
        // before we export anything, make sure the private key exists
@@ -294,12 +297,71 @@ pub(crate) async fn import_backup_stream<R: tokio::io::AsyncRead + Unpin>(
        .0
 }

+/// Reader that emits progress events as bytes are read from it.
+#[pin_project]
+struct ProgressReader<R> {
+    /// Wrapped reader.
+    #[pin]
+    inner: R,
+
+    /// Number of bytes successfully read from the internal reader.
+    read: usize,
+
+    /// Total size of the backup .tar file expected to be read from the reader.
+    /// Used to calculate the progress.
+    file_size: usize,
+
+    /// Last progress emitted to avoid emitting the same progress value twice.
+    last_progress: usize,
+
+    /// Context for emitting progress events.
+    context: Context,
+}
+
+impl<R> ProgressReader<R> {
+    fn new(r: R, context: Context, file_size: u64) -> Self {
+        Self {
+            inner: r,
+            read: 0,
+            file_size: file_size as usize,
+            last_progress: 1,
+            context,
+        }
+    }
+}
+
+impl<R> AsyncRead for ProgressReader<R>
+where
+    R: AsyncRead,
+{
+    fn poll_read(
+        self: Pin<&mut Self>,
+        cx: &mut std::task::Context<'_>,
+        buf: &mut ReadBuf<'_>,
+    ) -> std::task::Poll<std::io::Result<()>> {
+        let this = self.project();
+        let before = buf.filled().len();
+        let res = this.inner.poll_read(cx, buf);
+        if let std::task::Poll::Ready(Ok(())) = res {
+            *this.read = this.read.saturating_add(buf.filled().len() - before);
+
+            let progress = std::cmp::min(1000 * *this.read / *this.file_size, 999);
+            if progress > *this.last_progress {
+                this.context.emit_event(EventType::ImexProgress(progress));
+                *this.last_progress = progress;
+            }
+        }
+        res
+    }
+}
+
 async fn import_backup_stream_inner<R: tokio::io::AsyncRead + Unpin>(
    context: &Context,
    backup_file: R,
    file_size: u64,
    passphrase: String,
 ) -> (Result<()>,) {
+    let backup_file = ProgressReader::new(backup_file, context.clone(), file_size);
    let mut archive = Archive::new(backup_file);

    let mut entries = match archive.entries() {
@@ -307,29 +369,12 @@ async fn import_backup_stream_inner<R: tokio::io::AsyncRead + Unpin>(
        Err(e) => return (Err(e).context("Failed to get archive entries"),),
    };
    let mut blobs = Vec::new();
-    // We already emitted ImexProgress(10) above
-    let mut last_progress = 10;
-    const PROGRESS_MIGRATIONS: u128 = 999;
-    let mut total_size: u64 = 0;
    let mut res: Result<()> = loop {
        let mut f = match entries.try_next().await {
            Ok(Some(f)) => f,
            Ok(None) => break Ok(()),
            Err(e) => break Err(e).context("Failed to get next entry"),
        };
-        total_size += match f.header().entry_size() {
-            Ok(size) => size,
-            Err(e) => break Err(e).context("Failed to get entry size"),
-        };
-        let max = PROGRESS_MIGRATIONS - 1;
-        let progress = std::cmp::min(
-            max * u128::from(total_size) / std::cmp::max(u128::from(file_size), 1),
-            max,
-        );
-        if progress > last_progress {
-            context.emit_event(EventType::ImexProgress(progress as usize));
-            last_progress = progress;
-        }

        let path = match f.path() {
            Ok(path) => path.to_path_buf(),
@@ -379,7 +424,7 @@ async fn import_backup_stream_inner<R: tokio::io::AsyncRead + Unpin>(
        .log_err(context)
        .ok();
    if res.is_ok() {
-        context.emit_event(EventType::ImexProgress(PROGRESS_MIGRATIONS as usize));
+        context.emit_event(EventType::ImexProgress(999));
        res = context.sql.run_migrations(context).await;
    }
    if res.is_ok() {
@@ -452,7 +497,14 @@ async fn export_backup(context: &Context, dir: &Path, passphrase: String) -> Res

    let file = File::create(&temp_path).await?;
    let blobdir = BlobDirContents::new(context).await?;
-    export_backup_stream(context, &temp_db_path, blobdir, file)
+
+    let mut file_size = 0;
+    file_size += temp_db_path.metadata()?.len();
+    for blob in blobdir.iter() {
+        file_size += blob.to_abs_path().metadata()?.len()
+    }
+
+    export_backup_stream(context, &temp_db_path, blobdir, file, file_size)
        .await
        .context("Exporting backup to file failed")?;
    fs::rename(temp_path, &dest_path).await?;
@@ -460,33 +512,99 @@ async fn export_backup(context: &Context, dir: &Path, passphrase: String) -> Res
    Ok(())
 }

+/// Writer that emits progress events as bytes are written into it.
+#[pin_project]
+struct ProgressWriter<W> {
+    /// Wrapped writer.
+    #[pin]
+    inner: W,
+
+    /// Number of bytes successfully written into the internal writer.
+    written: usize,
+
+    /// Total size of the backup .tar file expected to be written into the writer.
+    /// Used to calculate the progress.
+    file_size: usize,
+
+    /// Last progress emitted to avoid emitting the same progress value twice.
+    last_progress: usize,
+
+    /// Context for emitting progress events.
+    context: Context,
+}
+
+impl<W> ProgressWriter<W> {
+    fn new(w: W, context: Context, file_size: u64) -> Self {
+        Self {
+            inner: w,
+            written: 0,
+            file_size: file_size as usize,
+            last_progress: 1,
+            context,
+        }
+    }
+}
+
+impl<W> AsyncWrite for ProgressWriter<W>
+where
+    W: AsyncWrite,
+{
+    fn poll_write(
+        self: Pin<&mut Self>,
+        cx: &mut std::task::Context<'_>,
+        buf: &[u8],
+    ) -> std::task::Poll<Result<usize, std::io::Error>> {
+        let this = self.project();
+        let res = this.inner.poll_write(cx, buf);
+        if let std::task::Poll::Ready(Ok(written)) = res {
+            *this.written = this.written.saturating_add(written);
+
+            let progress = std::cmp::min(1000 * *this.written / *this.file_size, 999);
+            if progress > *this.last_progress {
+                this.context.emit_event(EventType::ImexProgress(progress));
+                *this.last_progress = progress;
+            }
+        }
+        res
+    }
+
+    fn poll_flush(
+        self: Pin<&mut Self>,
+        cx: &mut std::task::Context<'_>,
+    ) -> std::task::Poll<Result<(), std::io::Error>> {
+        self.project().inner.poll_flush(cx)
+    }
+
+    fn poll_shutdown(
+        self: Pin<&mut Self>,
+        cx: &mut std::task::Context<'_>,
+    ) -> std::task::Poll<Result<(), std::io::Error>> {
+        self.project().inner.poll_shutdown(cx)
+    }
+}
+
 /// Exports the database and blobs into a stream.
 pub(crate) async fn export_backup_stream<'a, W>(
    context: &'a Context,
    temp_db_path: &Path,
    blobdir: BlobDirContents<'a>,
    writer: W,
+    file_size: u64,
 ) -> Result<()>
 where
    W: tokio::io::AsyncWrite + tokio::io::AsyncWriteExt + Unpin + Send + 'static,
 {
+    let writer = ProgressWriter::new(writer, context.clone(), file_size);
    let mut builder = tokio_tar::Builder::new(writer);

    builder
        .append_path_with_name(temp_db_path, DBFILE_BACKUP_NAME)
        .await?;

-    let mut last_progress = 10;
-
-    for (i, blob) in blobdir.iter().enumerate() {
+    for blob in blobdir.iter() {
        let mut file = File::open(blob.to_abs_path()).await?;
        let path_in_archive = PathBuf::from(BLOBS_BACKUP_NAME).join(blob.as_name());
        builder.append_file(path_in_archive, &mut file).await?;
-        let progress = std::cmp::min(1000 * i / blobdir.len(), 999);
-        if progress > last_progress {
-            context.emit_event(EventType::ImexProgress(progress));
-            last_progress = progress;
-        }
    }

    builder.finish().await?;
@@ -31,7 +31,8 @@ use std::pin::Pin;
 use std::sync::Arc;
 use std::task::Poll;

-use anyhow::{bail, Context as _, Result};
+use anyhow::{bail, format_err, Context as _, Result};
+use futures_lite::FutureExt;
 use iroh_net::relay::RelayMode;
 use iroh_net::Endpoint;
 use tokio::fs;
@@ -108,6 +109,7 @@ impl BackupProvider {
            .get_blobdir()
            .parent()
            .context("Context dir not found")?;
+
        let dbfile = context_dir.join(DBFILE_BACKUP_NAME);
        if fs::metadata(&dbfile).await.is_ok() {
            fs::remove_file(&dbfile).await?;
@@ -123,7 +125,6 @@ impl BackupProvider {
        export_database(context, &dbfile, passphrase, time())
            .await
            .context("Database export failed")?;
-        context.emit_event(EventType::ImexProgress(300));

        let drop_token = CancellationToken::new();
        let handle = {
@@ -177,6 +178,7 @@ impl BackupProvider {
        }

        info!(context, "Received valid backup authentication token.");
+        context.emit_event(EventType::ImexProgress(1));

        let blobdir = BlobDirContents::new(&context).await?;

@@ -188,7 +190,7 @@ impl BackupProvider {

        send_stream.write_all(&file_size.to_be_bytes()).await?;

-        export_backup_stream(&context, &dbfile, blobdir, send_stream)
+        export_backup_stream(&context, &dbfile, blobdir, send_stream, file_size)
            .await
            .context("Failed to write backup into QUIC stream")?;
        info!(context, "Finished writing backup into QUIC stream.");
@@ -231,8 +233,20 @@ impl BackupProvider {
                        let context = context.clone();
                        let auth_token = auth_token.clone();
                        let dbfile = dbfile.clone();
-                        if let Err(err) = Self::handle_connection(context.clone(), conn, auth_token, dbfile).await {
+                        if let Err(err) = Self::handle_connection(context.clone(), conn, auth_token, dbfile).race(
+                            async {
+                                cancel_token.recv().await.ok();
+                                Err(format_err!("Backup transfer cancelled"))
+                            }
+                        ).race(
+                            async {
+                                drop_token.cancelled().await;
+                                Err(format_err!("Backup provider dropped"))
+                            }
+                        ).await {
                            warn!(context, "Error while handling backup connection: {err:#}.");
+                            context.emit_event(EventType::ImexProgress(0));
+                            break;
                        } else {
                            info!(context, "Backup transfer finished successfully.");
                            break;
@@ -242,10 +256,12 @@ impl BackupProvider {
                    }
                },
                _ = cancel_token.recv() => {
+                    info!(context, "Backup transfer cancelled by the user, stopping accept loop.");
                    context.emit_event(EventType::ImexProgress(0));
                    break;
                }
                _ = drop_token.cancelled() => {
+                    info!(context, "Backup transfer cancelled by dropping the provider, stopping accept loop.");
                    context.emit_event(EventType::ImexProgress(0));
                    break;
                }
@@ -329,7 +345,20 @@ pub async fn get_backup(context: &Context, qr: Qr) -> Result<()> {
        Qr::Backup2 {
            node_addr,
            auth_token,
-        } => get_backup2(context, node_addr, auth_token).await?,
+        } => {
+            let cancel_token = context.alloc_ongoing().await?;
+            let res = get_backup2(context, node_addr, auth_token)
+                .race(async {
+                    cancel_token.recv().await.ok();
+                    Err(format_err!("Backup reception cancelled"))
+                })
+                .await;
+            if res.is_err() {
+                context.emit_event(EventType::ImexProgress(0));
+            }
+            context.free_ongoing().await;
+            res?;
+        }
        _ => bail!("QR code for backup must be of type DCBACKUP2"),
    }
    Ok(())
@@ -148,21 +148,6 @@ impl MsgId {
        Ok(())
    }

-    /// Deletes a message, corresponding MDNs and unsent SMTP messages from the database.
-    pub(crate) async fn delete_from_db(self, context: &Context) -> Result<()> {
-        context
-            .sql
-            .transaction(move |transaction| {
-                transaction.execute("DELETE FROM smtp WHERE msg_id=?", (self,))?;
-                transaction.execute("DELETE FROM msgs_mdns WHERE msg_id=?", (self,))?;
-                transaction.execute("DELETE FROM msgs_status_updates WHERE msg_id=?", (self,))?;
-                transaction.execute("DELETE FROM msgs WHERE id=?", (self,))?;
-                Ok(())
-            })
-            .await?;
-        Ok(())
-    }
-
    pub(crate) async fn set_delivered(self, context: &Context) -> Result<()> {
        update_msg_state(context, self, MessageState::OutDelivered).await?;
        let chat_id: ChatId = context
@@ -219,11 +204,13 @@ impl MsgId {
    }

    /// Returns information about hops of a message, used for message info
-    pub async fn hop_info(self, context: &Context) -> Result<Option<String>> {
-        context
+    pub async fn hop_info(self, context: &Context) -> Result<String> {
+        let hop_info = context
            .sql
-            .query_get_value("SELECT hop_info FROM msgs WHERE id=?", (self,))
-            .await
+            .query_get_value("SELECT IFNULL(hop_info, '') FROM msgs WHERE id=?", (self,))
+            .await?
+            .with_context(|| format!("Message {self} not found"))?;
+        Ok(hop_info)
    }

    /// Returns detailed message information in a multi-line text form.
@@ -328,7 +315,12 @@ impl MsgId {

        if let Some(path) = msg.get_file(context) {
            let bytes = get_filebytes(context, &path).await?;
-            ret += &format!("\nFile: {}, {} bytes\n", path.display(), bytes);
+            ret += &format!(
+                "\nFile: {}, name: {}, {} bytes\n",
+                path.display(),
+                msg.get_filename().unwrap_or_default(),
+                bytes
+            );
        }

        if msg.viewtype != Viewtype::Text {
@@ -361,7 +353,11 @@ impl MsgId {
        let hop_info = self.hop_info(context).await?;

        ret += "\n\n";
-        ret += &hop_info.unwrap_or_else(|| "No Hop Info".to_owned());
+        if hop_info.is_empty() {
+            ret += "No Hop Info";
+        } else {
+            ret += &hop_info;
+        }

        Ok(ret)
    }
@@ -1815,8 +1811,8 @@ pub(crate) async fn update_msg_state(
    context
        .sql
        .execute(
-            &format!("UPDATE msgs SET state=?1 {error_subst} WHERE id=?2 AND (?1!=?3 OR state<?3)"),
-            (state, msg_id, MessageState::OutDelivered),
+            &format!("UPDATE msgs SET state=? {error_subst} WHERE id=?"),
+            (state, msg_id),
        )
        .await?;
    Ok(())
@@ -1904,6 +1900,7 @@ pub async fn get_request_msg_cnt(context: &Context) -> usize {

 /// Estimates the number of messages that will be deleted
 /// by the options `delete_device_after` or `delete_server_after`.
+///
 /// This is typically used to show the estimated impact to the user
 /// before actually enabling deletion of old messages.
 ///
@@ -1993,7 +1990,9 @@ pub(crate) async fn rfc724_mid_exists_ex(
        .query_row_optional(
            &("SELECT id, timestamp_sent, MIN(".to_string()
                + expr
-                + ") FROM msgs WHERE rfc724_mid=? ORDER BY timestamp_sent DESC"),
+                + ") FROM msgs WHERE rfc724_mid=?
+              HAVING COUNT(*) > 0 -- Prevent MIN(expr) from returning NULL when there are no rows.
+              ORDER BY timestamp_sent DESC"),
            (rfc724_mid,),
            |row| {
                let msg_id: MsgId = row.get(0)?;
@@ -82,7 +82,10 @@ pub struct MimeFactory {
    /// as needed.
    references: String,

+    /// True if the message requests Message Disposition Notification
+    /// using `Chat-Disposition-Notification-To` header.
    req_mdn: bool,
+
    last_added_location_id: Option<u32>,

    /// If the created mime-structure contains sync-items,
@@ -201,7 +204,8 @@ impl MimeFactory {
        let (in_reply_to, references) = context
            .sql
            .query_row(
-                "SELECT mime_in_reply_to, mime_references FROM msgs WHERE id=?",
+                "SELECT mime_in_reply_to, IFNULL(mime_references, '')
+                 FROM msgs WHERE id=?",
                (msg.id,),
                |row| {
                    let in_reply_to: String = row.get(0)?;
@@ -351,7 +355,11 @@ impl MimeFactory {
                    // beside key- and member-changes, force a periodic re-gossip.
                    let gossiped_timestamp = chat.id.get_gossiped_timestamp(context).await?;
                    let gossip_period = context.get_config_i64(Config::GossipPeriod).await?;
-                    if time() >= gossiped_timestamp + gossip_period {
+                    // `gossip_period == 0` is a special case for testing,
+                    // enabling gossip in every message.
+                    // Othewise "smeared timestamps" may result in the condition
+                    // to fail even if the clock is monotonic.
+                    if gossip_period == 0 || time() >= gossiped_timestamp + gossip_period {
                        Ok(true)
                    } else {
                        Ok(false)
@@ -609,7 +617,9 @@ impl MimeFactory {
            // because replies to "Disposition-Notification-To" are weird in many cases
            // eg. are just freetext and/or do not follow any standard.
            headers.push(Header::new(
-                "Chat-Disposition-Notification-To".into(),
+                HeaderDef::ChatDispositionNotificationTo
+                    .get_headername()
+                    .to_string(),
                self.from_addr.clone(),
            ));
        }
@@ -597,9 +597,14 @@ pub(crate) async fn lookup_host_with_cache(
    load_cache: bool,
 ) -> Result<Vec<SocketAddr>> {
    let now = time();
-    let mut resolved_addrs = match lookup_host_and_update_cache(context, hostname, port, now).await
-    {
-        Ok(res) => res,
+    let resolved_addrs = match lookup_host_and_update_cache(context, hostname, port, now).await {
+        Ok(res) => {
+            if alpn.is_empty() {
+                res
+            } else {
+                sort_by_connection_timestamp(context, res, alpn, hostname).await?
+            }
+        }
        Err(err) => {
            warn!(
                context,
@@ -608,29 +613,43 @@ pub(crate) async fn lookup_host_with_cache(
            Vec::new()
        }
    };
-    if !alpn.is_empty() {
-        resolved_addrs =
-            sort_by_connection_timestamp(context, resolved_addrs, alpn, hostname).await?;
-    }

    if load_cache {
-        for addr in lookup_cache(context, hostname, port, alpn, now).await? {
-            if !resolved_addrs.contains(&addr) {
-                resolved_addrs.push(addr);
-            }
-        }
-
+        let mut cache = lookup_cache(context, hostname, port, alpn, now).await?;
        if let Some(ips) = DNS_PRELOAD.get(hostname) {
            for ip in ips {
                let addr = SocketAddr::new(*ip, port);
-                if !resolved_addrs.contains(&addr) {
-                    resolved_addrs.push(addr);
+                if !cache.contains(&addr) {
+                    cache.push(addr);
                }
            }
        }
+
+        Ok(merge_with_cache(resolved_addrs, cache))
+    } else {
+        Ok(resolved_addrs)
+    }
+}
+
+/// Merges results received from DNS with cached results.
+///
+/// At most 10 results are returned.
+fn merge_with_cache(
+    mut resolved_addrs: Vec<SocketAddr>,
+    cache: Vec<SocketAddr>,
+) -> Vec<SocketAddr> {
+    let rest = resolved_addrs.split_off(std::cmp::min(resolved_addrs.len(), 2));
+
+    for addr in cache.into_iter().chain(rest.into_iter()) {
+        if !resolved_addrs.contains(&addr) {
+            resolved_addrs.push(addr);
+            if resolved_addrs.len() >= 10 {
+                break;
+            }
+        }
    }

-    Ok(resolved_addrs)
+    resolved_addrs
 }

 #[cfg(test)]
@@ -866,4 +885,131 @@ mod tests {
            ],
        );
    }
+
+    #[test]
+    fn test_merge_with_cache() {
+        let first_addr = IpAddr::V4(Ipv4Addr::new(192, 168, 1, 1));
+        let second_addr = IpAddr::V4(Ipv4Addr::new(192, 168, 1, 2));
+
+        // If there is no cache, just return resolved addresses.
+        {
+            let resolved_addrs = vec![
+                SocketAddr::new(first_addr, 993),
+                SocketAddr::new(second_addr, 993),
+            ];
+            let cache = vec![];
+            assert_eq!(
+                merge_with_cache(resolved_addrs.clone(), cache),
+                resolved_addrs
+            );
+        }
+
+        // If cache contains address that is not in resolution results,
+        // it is inserted in the merged result.
+        {
+            let resolved_addrs = vec![SocketAddr::new(first_addr, 993)];
+            let cache = vec![SocketAddr::new(second_addr, 993)];
+            assert_eq!(
+                merge_with_cache(resolved_addrs, cache),
+                vec![
+                    SocketAddr::new(first_addr, 993),
+                    SocketAddr::new(second_addr, 993),
+                ]
+            );
+        }
+
+        // If cache contains address that is already in resolution results,
+        // it is not duplicated.
+        {
+            let resolved_addrs = vec![
+                SocketAddr::new(first_addr, 993),
+                SocketAddr::new(second_addr, 993),
+            ];
+            let cache = vec![SocketAddr::new(second_addr, 993)];
+            assert_eq!(
+                merge_with_cache(resolved_addrs, cache),
+                vec![
+                    SocketAddr::new(first_addr, 993),
+                    SocketAddr::new(second_addr, 993),
+                ]
+            );
+        }
+
+        // If DNS resolvers returns a lot of results,
+        // we should try cached results before going through all
+        // the resolver results.
+        {
+            let resolved_addrs = vec![
+                SocketAddr::new(IpAddr::V4(Ipv4Addr::new(127, 0, 0, 1)), 993),
+                SocketAddr::new(IpAddr::V4(Ipv4Addr::new(127, 0, 0, 2)), 993),
+                SocketAddr::new(IpAddr::V4(Ipv4Addr::new(127, 0, 0, 3)), 993),
+                SocketAddr::new(IpAddr::V4(Ipv4Addr::new(127, 0, 0, 4)), 993),
+                SocketAddr::new(IpAddr::V4(Ipv4Addr::new(127, 0, 0, 5)), 993),
+                SocketAddr::new(IpAddr::V4(Ipv4Addr::new(127, 0, 0, 6)), 993),
+                SocketAddr::new(IpAddr::V4(Ipv4Addr::new(127, 0, 0, 7)), 993),
+                SocketAddr::new(IpAddr::V4(Ipv4Addr::new(127, 0, 0, 8)), 993),
+                SocketAddr::new(IpAddr::V4(Ipv4Addr::new(127, 0, 0, 9)), 993),
+            ];
+            let cache = vec![SocketAddr::new(second_addr, 993)];
+            assert_eq!(
+                merge_with_cache(resolved_addrs, cache),
+                vec![
+                    SocketAddr::new(IpAddr::V4(Ipv4Addr::new(127, 0, 0, 1)), 993),
+                    SocketAddr::new(IpAddr::V4(Ipv4Addr::new(127, 0, 0, 2)), 993),
+                    SocketAddr::new(second_addr, 993),
+                    SocketAddr::new(IpAddr::V4(Ipv4Addr::new(127, 0, 0, 3)), 993),
+                    SocketAddr::new(IpAddr::V4(Ipv4Addr::new(127, 0, 0, 4)), 993),
+                    SocketAddr::new(IpAddr::V4(Ipv4Addr::new(127, 0, 0, 5)), 993),
+                    SocketAddr::new(IpAddr::V4(Ipv4Addr::new(127, 0, 0, 6)), 993),
+                    SocketAddr::new(IpAddr::V4(Ipv4Addr::new(127, 0, 0, 7)), 993),
+                    SocketAddr::new(IpAddr::V4(Ipv4Addr::new(127, 0, 0, 8)), 993),
+                    SocketAddr::new(IpAddr::V4(Ipv4Addr::new(127, 0, 0, 9)), 993),
+                ]
+            );
+        }
+
+        // Even if cache already contains all the incorrect results
+        // that resolver returns, this should not result in them being sorted to the top.
+        // Cache has known to work result returned first,
+        // so we should try it after the second result.
+        {
+            let resolved_addrs = vec![
+                SocketAddr::new(IpAddr::V4(Ipv4Addr::new(127, 0, 0, 1)), 993),
+                SocketAddr::new(IpAddr::V4(Ipv4Addr::new(127, 0, 0, 2)), 993),
+                SocketAddr::new(IpAddr::V4(Ipv4Addr::new(127, 0, 0, 3)), 993),
+                SocketAddr::new(IpAddr::V4(Ipv4Addr::new(127, 0, 0, 4)), 993),
+                SocketAddr::new(IpAddr::V4(Ipv4Addr::new(127, 0, 0, 5)), 993),
+                SocketAddr::new(IpAddr::V4(Ipv4Addr::new(127, 0, 0, 6)), 993),
+                SocketAddr::new(IpAddr::V4(Ipv4Addr::new(127, 0, 0, 7)), 993),
+                SocketAddr::new(IpAddr::V4(Ipv4Addr::new(127, 0, 0, 8)), 993),
+                SocketAddr::new(IpAddr::V4(Ipv4Addr::new(127, 0, 0, 9)), 993),
+            ];
+            let cache = vec![
+                SocketAddr::new(second_addr, 993),
+                SocketAddr::new(IpAddr::V4(Ipv4Addr::new(127, 0, 0, 9)), 993),
+                SocketAddr::new(IpAddr::V4(Ipv4Addr::new(127, 0, 0, 8)), 993),
+                SocketAddr::new(IpAddr::V4(Ipv4Addr::new(127, 0, 0, 7)), 993),
+                SocketAddr::new(IpAddr::V4(Ipv4Addr::new(127, 0, 0, 6)), 993),
+                SocketAddr::new(IpAddr::V4(Ipv4Addr::new(127, 0, 0, 5)), 993),
+                SocketAddr::new(IpAddr::V4(Ipv4Addr::new(127, 0, 0, 4)), 993),
+                SocketAddr::new(IpAddr::V4(Ipv4Addr::new(127, 0, 0, 3)), 993),
+                SocketAddr::new(IpAddr::V4(Ipv4Addr::new(127, 0, 0, 2)), 993),
+            ];
+            assert_eq!(
+                merge_with_cache(resolved_addrs, cache),
+                vec![
+                    SocketAddr::new(IpAddr::V4(Ipv4Addr::new(127, 0, 0, 1)), 993),
+                    SocketAddr::new(IpAddr::V4(Ipv4Addr::new(127, 0, 0, 2)), 993),
+                    SocketAddr::new(second_addr, 993),
+                    SocketAddr::new(IpAddr::V4(Ipv4Addr::new(127, 0, 0, 9)), 993),
+                    SocketAddr::new(IpAddr::V4(Ipv4Addr::new(127, 0, 0, 8)), 993),
+                    SocketAddr::new(IpAddr::V4(Ipv4Addr::new(127, 0, 0, 7)), 993),
+                    SocketAddr::new(IpAddr::V4(Ipv4Addr::new(127, 0, 0, 6)), 993),
+                    SocketAddr::new(IpAddr::V4(Ipv4Addr::new(127, 0, 0, 5)), 993),
+                    SocketAddr::new(IpAddr::V4(Ipv4Addr::new(127, 0, 0, 4)), 993),
+                    SocketAddr::new(IpAddr::V4(Ipv4Addr::new(127, 0, 0, 3)), 993),
+                ]
+            );
+        }
+    }
 }
@@ -53,6 +53,11 @@ impl<T: SessionStream> SessionStream for shadowsocks::ProxyClientStream<T> {
        self.get_mut().set_read_timeout(timeout)
    }
 }
+impl<T: SessionStream> SessionStream for async_imap::DeflateStream<T> {
+    fn set_read_timeout(&mut self, timeout: Option<Duration>) {
+        self.get_mut().set_read_timeout(timeout)
+    }
+}

 /// Session stream with a read buffer.
 pub(crate) trait SessionBufStream: SessionStream + AsyncBufRead {}
@@ -576,6 +576,13 @@ mod tests {
            .unwrap();

        bob.recv_msg_trash(&alice.pop_sent_msg().await).await;
+        loop {
+            let event = bob.evtracker.recv().await.unwrap();
+            if let EventType::WebxdcRealtimeAdvertisementReceived { msg_id } = event.typ {
+                assert!(msg_id == alice_webxdc.id);
+                break;
+            }
+        }
        let bob_iroh = bob.get_or_try_init_peer_channel().await.unwrap();

        // Bob adds alice to gossip peers.
@@ -343,7 +343,7 @@ impl Peerstate {
    }

    /// Updates peerstate according to the given `Autocrypt` header.
-    pub fn apply_header(&mut self, header: &Aheader, message_time: i64) {
+    pub fn apply_header(&mut self, context: &Context, header: &Aheader, message_time: i64) {
        if !addr_cmp(&self.addr, &header.addr) {
            return;
        }
@@ -362,6 +362,13 @@ impl Peerstate {
                self.public_key = Some(header.public_key.clone());
                self.recalc_fingerprint();
            }
+        } else {
+            warn!(
+                context,
+                "Ignoring outdated Autocrypt header because message_time={} < last_seen={}.",
+                message_time,
+                self.last_seen
+            );
        }
    }

@@ -766,23 +773,65 @@ pub(crate) async fn maybe_do_aeap_transition(

    // If the from addr is different from the peerstate address we know,
    // we may want to do an AEAP transition.
-    if !addr_cmp(&peerstate.addr, &mime_parser.from.addr)
-            // Check if it's a chat message; we do this to avoid
-            // some accidental transitions if someone writes from multiple
-            // addresses with an MUA.
-            && mime_parser.has_chat_version()
-            // Check if the message is encrypted and signed correctly. If it's not encrypted, it's
-            // probably from a new contact sharing the same key.
-            && !mime_parser.signatures.is_empty()
-            // Check if the From: address was also in the signed part of the email.
-            // Without this check, an attacker could replay a message from Alice
-            // to Bob. Then Bob's device would do an AEAP transition from Alice's
-            // to the attacker's address, allowing for easier phishing.
-            && mime_parser.from_is_signed
-            // DC avoids sending messages with the same timestamp, that's why `>` is here unlike in
-            // `Peerstate::apply_header()`.
-            && info.message_time > peerstate.last_seen
-    {
+    if !addr_cmp(&peerstate.addr, &mime_parser.from.addr) {
+        // Check if it's a chat message; we do this to avoid
+        // some accidental transitions if someone writes from multiple
+        // addresses with an MUA.
+        if !mime_parser.has_chat_version() {
+            info!(
+                context,
+                "Not doing AEAP from {} to {} because the message is not a chat message.",
+                &peerstate.addr,
+                &mime_parser.from.addr
+            );
+            return Ok(());
+        }
+
+        // Check if the message is encrypted and signed correctly. If it's not encrypted, it's
+        // probably from a new contact sharing the same key.
+        if mime_parser.signatures.is_empty() {
+            info!(
+                context,
+                "Not doing AEAP from {} to {} because the message is not encrypted and signed.",
+                &peerstate.addr,
+                &mime_parser.from.addr
+            );
+            return Ok(());
+        }
+
+        // Check if the From: address was also in the signed part of the email.
+        // Without this check, an attacker could replay a message from Alice
+        // to Bob. Then Bob's device would do an AEAP transition from Alice's
+        // to the attacker's address, allowing for easier phishing.
+        if !mime_parser.from_is_signed {
+            info!(
+                context,
+                "Not doing AEAP from {} to {} because From: is not signed.",
+                &peerstate.addr,
+                &mime_parser.from.addr
+            );
+            return Ok(());
+        }
+
+        // DC avoids sending messages with the same timestamp, that's why messages
+        // with equal timestamps are ignored here unlike in `Peerstate::apply_header()`.
+        if info.message_time <= peerstate.last_seen {
+            info!(
+                context,
+                "Not doing AEAP from {} to {} because {} < {}.",
+                &peerstate.addr,
+                &mime_parser.from.addr,
+                info.message_time,
+                peerstate.last_seen
+            );
+            return Ok(());
+        }
+
+        info!(
+            context,
+            "Doing AEAP transition from {} to {}.", &peerstate.addr, &mime_parser.from.addr
+        );
+
        let info = &mut mime_parser.decryption_info;
        let peerstate = info.peerstate.as_mut().context("no peerstate??")?;
        // Add info messages to chats with this (verified) contact
@@ -800,7 +849,7 @@ pub(crate) async fn maybe_do_aeap_transition(
        let header = info.autocrypt_header.as_ref().context(
            "Internal error: Tried to do an AEAP transition without an autocrypt header??",
        )?;
-        peerstate.apply_header(header, info.message_time);
+        peerstate.apply_header(context, header, info.message_time);

        peerstate
            .save_to_db_ex(&context.sql, Some(&old_addr))
@@ -979,6 +1028,8 @@ mod tests {

    #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
    async fn test_peerstate_degrade_reordering() {
+        let ctx = crate::test_utils::TestContext::new().await;
+
        let addr = "example@example.org";
        let pub_key = alice_keypair().public;
        let header = Aheader::new(addr.to_string(), pub_key, EncryptPreference::Mutual);
@@ -1003,7 +1054,7 @@ mod tests {
            fingerprint_changed: false,
        };

-        peerstate.apply_header(&header, 100);
+        peerstate.apply_header(&ctx, &header, 100);
        assert_eq!(peerstate.prefer_encrypt, EncryptPreference::Mutual);

        peerstate.degrade_encryption(300);
@@ -1011,11 +1062,11 @@ mod tests {

        // This has message time 200, while encryption was degraded at timestamp 300.
        // Because of reordering, header should not be applied.
-        peerstate.apply_header(&header, 200);
+        peerstate.apply_header(&ctx, &header, 200);
        assert_eq!(peerstate.prefer_encrypt, EncryptPreference::Reset);

        // Same header will be applied in the future.
-        peerstate.apply_header(&header, 300);
+        peerstate.apply_header(&ctx, &header, 300);
        assert_eq!(peerstate.prefer_encrypt, EncryptPreference::Mutual);
    }
 }
@@ -41,7 +41,7 @@ enum SignedPublicKeyOrSubkey<'a> {
    Subkey(&'a SignedPublicSubKey),
 }

-impl<'a> KeyTrait for SignedPublicKeyOrSubkey<'a> {
+impl KeyTrait for SignedPublicKeyOrSubkey<'_> {
    fn fingerprint(&self) -> Vec<u8> {
        match self {
            Self::Key(k) => k.fingerprint(),
@@ -64,7 +64,7 @@ impl<'a> KeyTrait for SignedPublicKeyOrSubkey<'a> {
    }
 }

-impl<'a> PublicKeyTrait for SignedPublicKeyOrSubkey<'a> {
+impl PublicKeyTrait for SignedPublicKeyOrSubkey<'_> {
    fn verify_signature(
        &self,
        hash: HashAlgorithm,
@@ -25,6 +25,7 @@ pub const QUOTA_ERROR_THRESHOLD_PERCENTAGE: u64 = 95;

 /// if quota is below this value (again),
 /// QuotaExceeding is cleared.
+///
 /// This value should be a bit below QUOTA_WARN_THRESHOLD_PERCENTAGE to
 /// avoid jittering and lots of warnings when quota is exactly at the warning threshold.
 ///
@@ -73,7 +74,7 @@ async fn get_unique_quota_roots_and_usage(

 fn get_highest_usage<'t>(
    unique_quota_roots: &'t BTreeMap<String, Vec<QuotaResource>>,
-) -> Result<(u64, &'t String, &QuotaResource)> {
+) -> Result<(u64, &'t String, &'t QuotaResource)> {
    let mut highest: Option<(u64, &'t String, &QuotaResource)> = None;
    for (name, resources) in unique_quota_roots {
        for r in resources {
@@ -197,7 +198,7 @@ mod tests {
    }

    #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
-    async fn test_quota_needs_update() {
+    async fn test_quota_needs_update() -> Result<()> {
        let mut tcm = TestContextManager::new();
        let t = &tcm.unconfigured().await;
        const TIMEOUT: u64 = 60;
@@ -214,5 +215,14 @@ mod tests {
            modified: tools::Time::now(),
        });
        assert!(!t.quota_needs_update(TIMEOUT).await);
+
+        t.evtracker.clear_events();
+        t.set_primary_self_addr("new@addr").await?;
+        assert!(t.quota.read().await.is_none());
+        t.evtracker
+            .get_matching(|evt| matches!(evt, EventType::ConnectivityChanged))
+            .await;
+        assert!(t.quota_needs_update(TIMEOUT).await);
+        Ok(())
    }
 }
@@ -1255,11 +1255,13 @@ async fn add_parts(

    let in_fresh = state == MessageState::InFresh;
    let sort_to_bottom = false;
+    let received = true;
    let sort_timestamp = chat_id
        .calc_sort_timestamp(
            context,
            mime_parser.timestamp_sent,
            sort_to_bottom,
+            received,
            mime_parser.incoming,
        )
        .await?;
@@ -1444,6 +1446,9 @@ async fn add_parts(
            Ok(node_addr) => {
                info!(context, "Adding iroh peer with address {node_addr:?}.");
                let instance_id = parent.context("Failed to get parent message")?.id;
+                context.emit_event(EventType::WebxdcRealtimeAdvertisementReceived {
+                    msg_id: instance_id,
+                });
                if let Some(topic) = get_iroh_topic_for_msg(context, instance_id).await? {
                    let node_id = node_addr.node_id;
                    let relay_server = node_addr.relay_url().map(|relay| relay.as_str());
@@ -3163,20 +3163,21 @@ Reply from different address
 }

 #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
-async fn test_long_and_duplicated_filenames() -> Result<()> {
+async fn test_weird_and_duplicated_filenames() -> Result<()> {
    let mut tcm = TestContextManager::new();
    let alice = tcm.alice().await;
    let bob = tcm.bob().await;

    for filename_sent in &[
        "foo.bar very long file name test baz.tar.gz",
-        "foobarabababababababbababababverylongfilenametestbaz.tar.gz",
+        "foo.barabababababababbababababverylongfilenametestbaz.tar.gz",
        "fooo...tar.gz",
        "foo. .tar.gz",
        "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.tar.gz",
        "a.tar.gz",
        "a.tar.gz",
        "a.a..a.a.a.a.tar.gz",
+        "a. tar.tar.gz",
    ] {
        let attachment = alice.blobdir.join(filename_sent);
        let content = format!("File content of {filename_sent}");
@@ -4810,6 +4811,36 @@ async fn test_protected_group_add_remove_member_missing_key() -> Result<()> {
    Ok(())
 }

+#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
+async fn test_older_message_from_2nd_device() -> Result<()> {
+    let mut tcm = TestContextManager::new();
+    let alice = &tcm.alice().await;
+    let chat_id = alice
+        .create_chat_with_contact("", "bob@example.net")
+        .await
+        .id;
+    alice.send_text(chat_id, "We share this account").await;
+    let received = receive_imf(
+        alice,
+        b"From: alice@example.org\n\
+          To: bob@example.net\n\
+          Message-ID: <1234-2-4@example.org>\n\
+          Date: Sat, 07 Dec 1970 19:00:26 +0000\n\
+          \n\
+          I'm Alice too\n",
+        true,
+    )
+    .await?
+    .unwrap();
+    alice
+        .golden_test_chat(
+            received.chat_id,
+            "receive_imf_older_message_from_2nd_device",
+        )
+        .await;
+    Ok(())
+}
+
 #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
 async fn test_dont_create_adhoc_group_on_member_removal() -> Result<()> {
    let mut tcm = TestContextManager::new();
@@ -433,36 +433,33 @@ async fn inbox_loop(
 }

 /// Convert folder meaning
-/// used internally by [fetch_idle] and [Context::background_fetch]
+/// used internally by [fetch_idle] and [Context::background_fetch].
+///
+/// Returns folder configuration key and folder name
+/// if such folder is configured, `Ok(None)` otherwise.
 pub async fn convert_folder_meaning(
    ctx: &Context,
    folder_meaning: FolderMeaning,
-) -> Result<(Config, String)> {
+) -> Result<Option<(Config, String)>> {
    let folder_config = match folder_meaning.to_config() {
        Some(c) => c,
        None => {
-            bail!("Bad folder meaning: {}", folder_meaning);
+            // Such folder cannot be configured,
+            // e.g. a `FolderMeaning::Spam` folder.
+            return Ok(None);
        }
    };

-    let folder = match ctx.get_config(folder_config).await {
-        Ok(folder) => folder,
-        Err(err) => {
-            bail!(
-                "Can not watch {} folder, failed to retrieve config: {:#}",
-                folder_config,
-                err
-            );
-        }
-    };
+    let folder = ctx
+        .get_config(folder_config)
+        .await
+        .with_context(|| format!("Failed to retrieve {folder_config} folder"))?;

-    let watch_folder = if let Some(watch_folder) = folder {
-        watch_folder
+    if let Some(watch_folder) = folder {
+        Ok(Some((folder_config, watch_folder)))
    } else {
-        bail!("Can not watch {} folder, not set", folder_config);
-    };
-
-    Ok((folder_config, watch_folder))
+        Ok(None)
+    }
 }

 async fn inbox_fetch_idle(ctx: &Context, imap: &mut Imap, mut session: Session) -> Result<Session> {
@@ -554,17 +551,14 @@ async fn fetch_idle(
    mut session: Session,
    folder_meaning: FolderMeaning,
 ) -> Result<Session> {
-    let (folder_config, watch_folder) = match convert_folder_meaning(ctx, folder_meaning).await {
-        Ok(meaning) => meaning,
-        Err(err) => {
-            // Warning instead of error because the folder may not be configured.
-            // For example, this happens if the server does not have Sent folder
-            // but watching Sent folder is enabled.
-            warn!(ctx, "Error converting IMAP Folder name: {err:#}.");
-            connection.connectivity.set_not_configured(ctx).await;
-            connection.idle_interrupt_receiver.recv().await.ok();
-            return Err(err);
-        }
+    let Some((folder_config, watch_folder)) = convert_folder_meaning(ctx, folder_meaning).await?
+    else {
+        // The folder is not configured.
+        // For example, this happens if the server does not have Sent folder
+        // but watching Sent folder is enabled.
+        connection.connectivity.set_not_configured(ctx).await;
+        connection.idle_interrupt_receiver.recv().await.ok();
+        bail!("Cannot fetch folder {folder_meaning} because it is not configured");
    };

    if folder_config == Config::ConfiguredInboxFolder {
@@ -696,7 +696,10 @@ async fn could_not_establish_secure_connection(
    details: &str,
 ) -> Result<()> {
    let contact = Contact::get_by_id(context, contact_id).await?;
-    let msg = stock_str::contact_not_verified(context, &contact).await;
+    let mut msg = stock_str::contact_not_verified(context, &contact).await;
+    msg += " (";
+    msg += details;
+    msg += ")";
    chat::add_info_msg(context, chat_id, &msg, time()).await?;
    warn!(
        context,
@@ -47,7 +47,7 @@ pub(super) async fn start_protocol(context: &Context, invite: QrInvite) -> Resul
        BobState::start_protocol(context, invite.clone(), chat_id).await?;
    for state in aborted_states {
        error!(context, "Aborting previously unfinished QR Join process.");
-        state.notify_aborted(context, "new QR scanned").await?;
+        state.notify_aborted(context, "New QR code scanned").await?;
        state.emit_progress(context, JoinerProgress::Error);
    }
    if matches!(stage, BobHandshakeStage::RequestWithAuthSent) {
@@ -73,8 +73,9 @@ pub(super) async fn start_protocol(context: &Context, invite: QrInvite) -> Resul
            // Calculate the sort timestamp before checking the chat protection status so that if we
            // race with its change, we don't add our message below the protection message.
            let sort_to_bottom = true;
+            let (received, incoming) = (false, false);
            let ts_sort = chat_id
-                .calc_sort_timestamp(context, 0, sort_to_bottom, false)
+                .calc_sort_timestamp(context, 0, sort_to_bottom, received, incoming)
                .await?;
            if chat_id.is_protected(context).await? == ProtectionStatus::Unprotected {
                let ts_start = time();
@@ -194,7 +195,10 @@ impl BobState {
    /// This creates an info message in the chat being joined.
    async fn notify_aborted(&self, context: &Context, why: &str) -> Result<()> {
        let contact = Contact::get_by_id(context, self.invite().contact_id()).await?;
-        let msg = stock_str::contact_not_verified(context, &contact).await;
+        let mut msg = stock_str::contact_not_verified(context, &contact).await;
+        msg += " (";
+        msg += why;
+        msg += ")";
        let chat_id = self.joining_chat_id(context).await?;
        chat::add_info_msg(context, chat_id, &msg, time()).await?;
        warn!(
@@ -558,15 +558,13 @@ impl Sql {
        self.call(move |conn| match conn.query_row(sql.as_ref(), params, f) {
            Ok(res) => Ok(Some(res)),
            Err(rusqlite::Error::QueryReturnedNoRows) => Ok(None),
-            Err(rusqlite::Error::InvalidColumnType(_, _, rusqlite::types::Type::Null)) => Ok(None),
            Err(err) => Err(err.into()),
        })
        .await
    }

    /// Executes a query which is expected to return one row and one
-    /// column. If the query does not return a value or returns SQL
-    /// `NULL`, returns `Ok(None)`.
+    /// column. If the query does not return any rows, returns `Ok(None)`.
    pub async fn query_get_value<T>(
        &self,
        query: &str,
@@ -1037,6 +1037,39 @@ CREATE INDEX msgs_status_updates_index2 ON msgs_status_updates (uid);
        .await?;
    }

+    inc_and_check(&mut migration_version, 122)?;
+    if dbversion < migration_version {
+        sql.execute_migration(
+            "ALTER TABLE tokens ADD COLUMN foreign_id INTEGER NOT NULL DEFAULT 0",
+            migration_version,
+        )
+        .await?;
+    }
+
+    inc_and_check(&mut migration_version, 123)?;
+    if dbversion < migration_version {
+        // Add FOREIGN KEY(msg_id).
+        sql.execute_migration(
+            "CREATE TABLE new_msgs_status_updates (
+                id INTEGER PRIMARY KEY AUTOINCREMENT,
+                msg_id INTEGER,
+                update_item TEXT DEFAULT '',
+                uid TEXT UNIQUE,
+                FOREIGN KEY(msg_id) REFERENCES msgs(id) ON DELETE CASCADE
+            );
+            INSERT OR IGNORE INTO new_msgs_status_updates SELECT
+                id, msg_id, update_item, uid
+            FROM msgs_status_updates;
+            DROP TABLE msgs_status_updates;
+            ALTER TABLE new_msgs_status_updates RENAME TO msgs_status_updates;
+            CREATE INDEX msgs_status_updates_index1 ON msgs_status_updates (msg_id);
+            CREATE INDEX msgs_status_updates_index2 ON msgs_status_updates (uid);
+            ",
+            migration_version,
+        )
+        .await?;
+    }
+
    let new_version = sql
        .get_raw_config_int(VERSION_CFG)
        .await?
@@ -17,7 +17,7 @@
 use std::cmp::{max, min};
 use std::sync::atomic::{AtomicI64, Ordering};

-pub(crate) const MAX_SECONDS_TO_LEND_FROM_FUTURE: i64 = 5;
+pub(crate) const MAX_SECONDS_TO_LEND_FROM_FUTURE: i64 = 30;

 /// Smeared timestamp generator.
 #[derive(Debug)]
@@ -157,11 +157,11 @@ mod tests {
        // because there were not enough timestamps in the past.
        assert_eq!(smeared_timestamp.current(), now + 4);

-        // Forward another 7 messages.
-        // We cannot use more than 5 timestamps from the future,
-        // so we use 5 timestamps from the future,
+        // Forward another 32 messages.
+        // We cannot use more than 30 timestamps from the future,
+        // so we use 30 timestamps from the future,
        // the current timestamp and one timestamp from the past.
-        assert_eq!(smeared_timestamp.create_n(now, forwarded_messages), now - 1);
+        assert_eq!(smeared_timestamp.create_n(now, 32), now - 1);
    }

    #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
@@ -23,13 +23,13 @@ use std::path::Path;

 use anyhow::{anyhow, bail, ensure, format_err, Context as _, Result};

+use async_zip::tokio::read::fs::ZipFileReader as FsZipFileReader;
 use deltachat_contact_tools::sanitize_bidi_characters;
 use deltachat_derive::FromSql;
 use lettre_email::PartBuilder;
 use rusqlite::OptionalExtension;
 use serde::{Deserialize, Serialize};
 use serde_json::Value;
-use tokio::io::AsyncReadExt;

 use crate::chat::{self, Chat};
 use crate::constants::Chattype;
@@ -195,7 +195,7 @@ fn find_zip_entry<'a>(
    name: &str,
 ) -> Option<(usize, &'a async_zip::StoredZipEntry)> {
    for (i, ent) in file.entries().iter().enumerate() {
-        if ent.entry().filename() == name {
+        if ent.filename().as_bytes() == name.as_bytes() {
            return Some((i, ent));
        }
    }
@@ -212,7 +212,7 @@ impl Context {
            return Ok(false);
        }

-        let archive = match async_zip::read::mem::ZipFileReader::new(file.to_vec()).await {
+        let archive = match async_zip::base::read::mem::ZipFileReader::new(file.to_vec()).await {
            Ok(archive) => archive,
            Err(_) => {
                info!(self, "{} cannot be opened as zip-file", &filename);
@@ -235,7 +235,7 @@ impl Context {
            bail!("{} is not a valid webxdc file", filename);
        }

-        let valid = match async_zip::read::fs::ZipFileReader::new(path).await {
+        let valid = match FsZipFileReader::new(path).await {
            Ok(archive) => {
                if find_zip_entry(archive.file(), "index.html").is_none() {
                    warn!(self, "{} misses index.html", filename);
@@ -791,27 +791,24 @@ fn parse_webxdc_manifest(bytes: &[u8]) -> Result<WebxdcManifest> {
    Ok(manifest)
 }

-async fn get_blob(archive: &async_zip::read::fs::ZipFileReader, name: &str) -> Result<Vec<u8>> {
+async fn get_blob(archive: &FsZipFileReader, name: &str) -> Result<Vec<u8>> {
    let (i, _) = find_zip_entry(archive.file(), name)
        .ok_or_else(|| anyhow!("no entry found for {}", name))?;
-    let mut reader = archive.entry(i).await?;
+    let mut reader = archive.reader_with_entry(i).await?;
    let mut buf = Vec::new();
-    reader.read_to_end(&mut buf).await?;
+    reader.read_to_end_checked(&mut buf).await?;
    Ok(buf)
 }

 impl Message {
    /// Get handle to a webxdc ZIP-archive.
    /// To check for file existence use archive.by_name(), to read a file, use get_blob(archive).
-    async fn get_webxdc_archive(
-        &self,
-        context: &Context,
-    ) -> Result<async_zip::read::fs::ZipFileReader> {
+    async fn get_webxdc_archive(&self, context: &Context) -> Result<FsZipFileReader> {
        let path = self
            .get_file(context)
            .ok_or_else(|| format_err!("No webxdc instance file."))?;
        let path_abs = get_abs_path(context, &path);
-        let archive = async_zip::read::fs::ZipFileReader::new(path_abs).await?;
+        let archive = FsZipFileReader::new(path_abs).await?;
        Ok(archive)
    }

@@ -0,0 +1,5 @@
+Single#Chat#10: bob@example.net [bob@example.net] 
+--------------------------------------------------------------------------------
+Msg#10: Me (Contact#Contact#Self): We share this account  √
+Msg#11: Me (Contact#Contact#Self): I'm Alice too  √
+--------------------------------------------------------------------------------
Author	SHA1	Message	Date
link2xt	2e030aaab9	Adding error context for `forward_msgs`	2024-10-20 08:28:48 +00:00
link2xt	deb14600e8	feat: add more context to remove_contact_from_chat errors	2024-10-20 08:23:45 +00:00
link2xt	95c252a5a6	refactor(remove_contact_from_chat): remove one level of indentation	2024-10-20 08:23:45 +00:00
Sebastian Klähn link2xt iequidoo	1b824705fd	feat: Add realtime advertisement received event (#6043 ) Co-authored-by: link2xt <link2xt@testrun.org> Co-authored-by: iequidoo <117991069+iequidoo@users.noreply.github.com>	2024-10-20 06:27:57 +00:00
link2xt	6f22ce2722	build: nix flake update	2024-10-20 06:01:30 +00:00
link2xt	5e58bf7575	feat: add more context to `send_msg` errors	2024-10-19 20:52:30 +00:00
link2xt	85d7c1f942	ci: update Rust to 1.82.0	2024-10-19 20:31:30 +00:00
iequidoo	df4fd82140	fix: ChatId::maybe_delete_draft: Don't delete message if it's not a draft anymore (#6053 ) Follow-up to `07fa9c35ee`.	2024-10-19 11:48:45 -03:00
link2xt	65b970a191	test: fix `test_securejoin_after_contact_resetup` flakiness	2024-10-18 15:36:21 +00:00
link2xt	5e13b4c736	feat: log when late Autocrypt header is ignored	2024-10-18 15:36:21 +00:00
link2xt	864833d232	fix: increase MAX_SECONDS_TO_LEND_FROM_FUTURE to 30 5 seconds is easy to exhaust by running securejoin, especially when it happens in automatic tests. This may however easily affect bots as well.	2024-10-17 22:22:14 +00:00
link2xt	3d07db6e62	feat: log the logic for (not) doing AEAP	2024-10-17 22:22:14 +00:00
link2xt	9e88764a8a	test(test_aeap_flow_verified): do not start ac1new ac1new is an account that is only used to get a new address for ac1. It should not even be started and run IMAP loop.	2024-10-17 22:21:54 +00:00
link2xt	e70b879182	test: make test_verified_group_member_added_recovery more reliable To avoid reordering, wait for "member removed" message to be received before sending "member added". The test failed at least once because email server may reorder the messages internally while delivering.	2024-10-17 15:11:10 +00:00
link2xt	00d296e1ff	test(test_aeap_flow_verified): wait for "member added" before sending messages (#6057 ) Otherwise instead of "old address" ac2 may receive "member added", resulting in this failure: ``` > assert msg_in_1.text == msg_out.text E AssertionError: assert 'Member Me (c...hat.computer.' == 'old address' E - old address E + Member Me (ci-hfpxxe@*) added by ci-8e7mkr@*. ```	2024-10-17 15:10:55 +00:00
link2xt	e07f1a8b9c	docs: fix `too_long_first_doc_paragraph` clippy lint This lint is enabled for beta and nightly Rust.	2024-10-17 14:29:03 +00:00
link2xt	02b9085147	feat: prioritize cached results if DNS resolver returns many results This ensures we do not get stuck trying DNS resolver results when we have a known to work IP address in the cache and DNS resolver returns garbage either because it is a captive portal or if it maliciously wants to get us stuck trying a long list of unresponsive IP addresses. This also limits the number of results we try to 10 overall. If there are more results, we will retry later with new resolution results.	2024-10-17 11:55:14 +00:00
link2xt	07fa9c35ee	fix: replace old draft with a new one atomically This prevents creation of multiple drafts per chat.	2024-10-17 11:52:50 +00:00
link2xt	7db7c0aab1	refactor: use `HeaderDef` constant for Chat-Disposition-Notification-To	2024-10-17 07:10:54 +00:00
link2xt	30b23df816	docs: document MimeFactory.req_mdn	2024-10-17 07:10:54 +00:00
link2xt	4efd0d1ef7	test: always gossip if gossip_period is set to 0 This fixes flakiness of `test_verified_group_vs_delete_server_after`.	2024-10-15 22:55:33 +00:00
link2xt	f14880146a	feat(deltachat-repl): built-in QR code printer Print QR codes with Rust code instead of depending on external `qrencode`.	2024-10-15 22:55:20 +00:00
link2xt	3a72188548	test(test_qr_setup_contact_svg): stop testing for no display name It is impossible to set no display name anyway in Delta Chat Android at least because we don't want email addresses in the UI. This test does not work with long domains that may get wrapped, so better remove it instead of trying to prevent wrapping of domains.	2024-10-15 17:35:38 +00:00
link2xt	351f28361d	docs: set_protection_for_timestamp_sort does not send messages It only adds info messages.	2024-10-15 09:14:23 +00:00
link2xt	c5b78741d6	refactor: fix clippy::needless_lifetimes warnings	2024-10-15 09:14:23 +00:00
link2xt	57871bbaf8	refactor(set_protection_for_timestamp_sort): do not log bubbled up errors Otherwise error may be logged twice.	2024-10-15 09:14:23 +00:00
link2xt	287256693c	refactor: fix elided_named_lifetimes warning in beta Rust	2024-10-15 09:14:23 +00:00
iequidoo	d660f55a99	feat: Sort received outgoing message down if it's fresher than all non fresh messages Received messages shouldn't mingle with just sent ones and appear somewhere in the middle of the chat, so we go after the newest non fresh message. But if a received outgoing message is older than some `InSeen` message, better sort the received message purely by timestamp (this is an heuristic in order not to break the Gmail-like case simulated by `verified_chats::test_old_message_4()`). We could place the received message just before that `InSeen` message, but anyway the user may not notice it. At least this fixes outgoing messages sorting for shared accounts where messages from other devices should be sorted the same way as incoming ones.	2024-10-14 21:22:11 -03:00
link2xt	f1ca689f99	feat: IMAP COMPRESS support	2024-10-14 14:01:22 +00:00
iequidoo	796b0d7752	refactor: update_msg_state: Don't avoid downgrading OutMdnRcvd to OutDelivered `OutMdnRcvd` is a "virtual" message state now, only old messages can have this state in the db, so `update_msg_state()` can be simplified.	2024-10-14 10:24:01 -03:00
link2xt	2ea5c86a5a	chore(release): prepare for 1.147.1	2024-10-13 18:40:33 +00:00
iequidoo	50b250cf78	docs(CONTRIBUTING.md): Add a note on deleting/changing db columns	2024-10-13 15:34:15 -03:00
iequidoo	3c03370589	fix: Readd tokens.foreign_id column (#6038 ) Otherwise backups exported from the current core and imported in versions < 1.144.0 have QR codes not working. The breaking change which removed the column is `5a6efdff44`.	2024-10-13 15:34:15 -03:00
iequidoo	8f41aed917	fix: Assume file extensions are 32 chars max and don't contain whitespace (#5338 ) Before file extensions were also limited to 32 chars, but extra chars in the beginning were just cut off, e.g. "file.with_lots_of_characters_behind_point_and_double_ending.tar.gz" was considered to have an extension "d_point_and_double_ending.tar.gz". Better to take only "tar.gz" then. Also don't include whitespace-containing parts in extensions. File extensions generally don't contain whitespaces.	2024-10-11 11:04:41 -03:00
link2xt	19be12a25d	chore(cargo): upgrade async_zip to 0.0.17 (#6035 )	2024-10-11 00:17:40 +00:00
link2xt	6a121b87eb	fix: do not emit progress 1000 when configuration is cancelled There is already code below that emits progress 0 or 1000 depending on whether configuration succeeded or failed. Before this change cancelling resulted in progress 0 emitted, immediately followed by progress 1000.	2024-10-10 10:34:58 +00:00
link2xt	420c0ed9b0	build(deltachat-rpc-client): add classifiers for all supported Python versions	2024-10-10 07:18:06 +00:00
link2xt	e05bb03db6	build: build Python 3.13 wheels	2024-10-10 07:18:06 +00:00
link2xt	73fcb97eef	ci: update to Python 3.13	2024-10-10 07:18:06 +00:00
iequidoo	8acf391ffe	refactor: MsgId::update_download_state: Don't fail if the message doesn't exist anymore If a race happens and the message disappears, there's just nothing to do and no sense to fail. Follow-up to `22e5bf8571`.	2024-10-08 12:31:41 -03:00
iequidoo	aacea2de25	fix: Reset quota on configured address change (#5908 )	2024-10-07 18:04:53 -03:00
iequidoo	b713e8cd94	chore(cargo): bump futures-* from 0.3.30 to 0.3.31 futures-util 0.3.30 is yanked.	2024-10-07 15:33:09 -03:00
link2xt	b7be0b7bf6	chore(release): prepare for 1.147.0	2024-10-05 18:04:17 +00:00
link2xt	2cb8b53256	fix: emit progress 0 if get_backup() fails	2024-10-05 17:58:24 +00:00
link2xt	a592a470cf	fix: make backup reception cancellable by stopping ongoing process This is already documented in JSON-RPC API, but in fact ongoing process was not allocated.	2024-10-05 17:58:24 +00:00
link2xt	c4d07ab99e	fix: smooth progress bar for backup transfer Before this change progress bar only started when database is already transferred. Database is usually the largest file in the whole transfer, so the transfer appears to be stuck for the sender. With this change progress bar starts for backup export as soon as connection is received and counts bytes transferred over the connection using AsyncWrite wrapper. Similarly for backup import, AsyncRead wrapper counts the bytes received and emits progress events.	2024-10-05 17:58:24 +00:00
link2xt	eddd5a0d25	fix: make it possible to cancel ongoing backup transfer	2024-10-05 17:58:24 +00:00
link2xt	0f43d5d8f4	fix: break out of accept() loop if there is an error transferring backup	2024-10-05 17:58:24 +00:00
link2xt	2e6d3aebae	docs(CONTRIBUTING.md): add more SQL advices	2024-10-05 13:09:49 +00:00
link2xt	650995dc41	feat(deltachat-repl): print send-backup QR code to the terminal	2024-10-04 22:53:30 +00:00
link2xt	283a1f1653	fix: skip unconfigured folders in `background_fetch()` Otherwise `background_fetch()` fails on unconfigured Mvbox, which is typical for chatmail accounts, and does not get to checking QUOTA ever.	2024-10-04 21:54:42 +00:00
link2xt	d33909a054	feat: reuse existing connections in background_fetch() if I/O is started	2024-10-04 21:54:42 +00:00
link2xt	129be3aa27	feat(deltachat-repl): add `fetch` command to test `background_fetch()`	2024-10-04 15:52:59 +00:00
link2xt	8a88479d8f	fix(query_row_optional): do not treat rows with NULL as missing rows Instead of treating NULL type error as absence of the row, handle NULL values with SQL. Previously we sometimes accidentally treated a single column being NULL as the lack of the whole row.	2024-10-04 14:43:06 +00:00
Hocuri	5711f2fe3a	feat: More context for the "Cannot establish guaranteed..." info message (#6022 ) The "Cannot establish guaranteed end-to-end encryption with ..." info message can have lots of causes, and it happened twice to us now that it took us some time to figure out which one it is. So, include some more detail in the info message by simply adding the non-translated error message in parantheses. If we want to put in some more effort for nicer error messages, we could: - Introduce one new translated string "Cannot establish guaranteed end-to-end encryption with …. Cause: %2$s" or similar (and remove the old stock string) - And/Or: Introduce new translated strings for all the possible errors - And/Or: Maybe reword it in order to account better for the case that the chat already is marked as g-e2ee, or use a different wording (because if the chat is marked as g-e2ee then it might be nice to notify the user that something may have gone wrong, but it's still working, just that maybe the other side doesn't have us verified now) ![Screenshot_20241003-222245](https://github.com/user-attachments/assets/c064c82e-01ac-4bac-ab11-3c9ac9db5298)	2024-10-04 13:51:06 +02:00
link2xt	46922d4d9d	fix: do not attempt to reference info messages Info messages are added at the beginning of unpromoted group chats ("Others will only see this group after you sent a first message."), may be created by WebXDC etc. They are not sent outside and have local Message-ID that is not known to other recipients so they should be skipped when constructing In-Reply-To and References.	2024-10-03 21:49:58 +00:00
link2xt	75fe4e106a	api!: remove deprecated get_next_media() APIs	2024-10-03 21:04:03 +00:00
iequidoo	7c60ac863e	feat: MsgId::get_info(): Report original filename as well	2024-10-03 15:49:03 -03:00
@@ -1 +1 @@
 -10-03
 -10-13