api: add dc_replace_webxdc()

api: make it possible to import secret key from a file
Previously it was required that a directory path is provided to the import API. Now it is possible to point directly to the .asc file containing a secret key. This allows UI to present a file selection dialog to the user and let select the file directly. Selecting a directory is still supported for backwards compatibility.
2026-04-13 03:30:29 +03:00 · 2023-09-22 10:44:30 +00:00 · 2023-09-20 16:18:45 +00:00 · 2023-09-20 16:18:45 +00:00 · 2023-09-18 14:55:56 +00:00 · 2023-09-18 14:55:56 +00:00
71 changed files with 3974 additions and 1352 deletions
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -15,6 +15,7 @@ on:
  push:
    branches:
      - master
+      - stable

 env:
  RUSTFLAGS: -Dwarnings
@@ -24,7 +25,7 @@ jobs:
    name: Lint Rust
    runs-on: ubuntu-latest
    env:
-      RUSTUP_TOOLCHAIN: 1.70.0
+      RUSTUP_TOOLCHAIN: 1.72.0
    steps:
      - uses: actions/checkout@v3
      - name: Install rustfmt and clippy
--- a/.github/workflows/node-package.yml
+++ b/.github/workflows/node-package.yml
@@ -67,7 +67,9 @@ jobs:

    # Build Linux prebuilds inside a container with old glibc for backwards compatibility.
    # Debian 10 contained glibc 2.28 at the time of the writing (2023-06-04): https://packages.debian.org/buster/libc6
-    container: debian:10
+    # Ubuntu 18.04 is at the End of Standard Support since June 2023, but it contains glibc 2.27,
+    # so we are using it to support Ubuntu 18.04 setups that are still not upgraded.
+    container: ubuntu:18.04
    steps:
      # Working directory is owned by 1001:1001 by default.
      # Change it to our user.
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -1,5 +1,104 @@
 # Changelog

+## [1.122.0] - 2023-09-12
+
+### API-Changes
+
+- jsonrpc: Return only chat IDs for similar chats.
+
+### Fixes
+
+- Reopen all connections on database passpharse change.
+- Do not block new group chats if 1:1 chat is blocked.
+- Improve group membership consistency algorithm ([#3782](https://github.com/deltachat/deltachat-core-rust/pull/3782))([#4624](https://github.com/deltachat/deltachat-core-rust/pull/4624)).
+- Forbid membership changes from possible non-members ([#3782](https://github.com/deltachat/deltachat-core-rust/pull/3782)).
+- `ChatId::parent_query()`: Don't filter out OutPending and OutFailed messages.
+
+### Build system
+
+- Update to OpenSSL 3.0.
+- Bump webpki from 0.22.0 to 0.22.1.
+- python: Add link to Mastodon into projects.urls.
+
+### Features / Changes
+
+- Add RSA-4096 key generation support.
+
+### Refactor
+
+- pgp: Add constants for encryption algorithm and hash.
+
+## [1.121.0] - 2023-09-06
+
+### API-Changes
+
+- Add `dc_context_change_passphrase()`.
+- Add `Message.set_file_from_bytes()` API.
+- Add experimental API to get similar chats.
+
+### Build system
+
+- Build node packages on Ubuntu 18.04 instead of Debian 10.
+  This reduces the requirement for glibc version from 2.28 to 2.27.
+
+### Fixes
+
+- Allow membership changes by a MUA if we're not in the group ([#4624](https://github.com/deltachat/deltachat-core-rust/pull/4624)).
+- Save mime headers for messages not signed with a known key ([#4557](https://github.com/deltachat/deltachat-core-rust/pull/4557)).
+- Return from `dc_get_chatlist(DC_GCL_FOR_FORWARDING)` only chats where we can send ([#4616](https://github.com/deltachat/deltachat-core-rust/pull/4616)).
+- Do not allow dots at the end of email addresses.
+- deltachat-rpc-client: Remove `aiodns` optional dependency from required dependencies.
+  `aiodns` depends on `pycares` which [fails to install in Termux](https://github.com/saghul/aiodns/issues/98).
+
+## [1.120.0] - 2023-08-28
+
+### API-Changes
+
+- jsonrpc: Add `resend_messages`.
+
+### Fixes
+
+- Update async-imap to 0.9.1 to fix memory leak.
+- Delete messages from SMTP queue only on user demand ([#4579](https://github.com/deltachat/deltachat-core-rust/pull/4579)).
+- Do not send images without transparency as stickers ([#4611](https://github.com/deltachat/deltachat-core-rust/pull/4611)).
+- `prepare_msg_blob()`: do not use the image if it has Exif metadata but the image cannot be recoded.
+
+### Refactor
+
+- Hide accounts.rs constants from public API.
+- Hide pgp module from public API.
+
+### Build system
+
+- Update to Zig 0.11.0.
+- Update to Rust 1.72.0.
+
+### CI
+
+- Run on push to stable branch.
+
+### Miscellaneous Tasks
+
+- python: Fix lint errors.
+- python: Fix `ruff` 0.0.286 warnings.
+- Fix beta clippy warnings.
+
+## [1.119.1] - 2023-08-06
+
+Bugfix release attempting to fix the [iOS build error](https://github.com/deltachat/deltachat-core-rust/issues/4610).
+
+### Features / Changes
+
+- Guess message viewtype from "application/octet-stream" attachment extension ([#4378](https://github.com/deltachat/deltachat-core-rust/pull/4378)).
+
+### Fixes
+
+- Update `xattr` from 1.0.0 to 1.0.1 to fix UnsupportedPlatformError import.
+
+### Tests
+
+- webxdc: Ensure unknown WebXDC update properties do not result in an error.
+
 ## [1.119.0] - 2023-08-03

 ### Fixes
@@ -2712,3 +2811,8 @@ https://github.com/deltachat/deltachat-core-rust/pulls?q=is%3Apr+is%3Aclosed
 [1.116.0]: https://github.com/deltachat/deltachat-core-rust/compare/v1.115.0...v1.116.0
 [1.117.0]: https://github.com/deltachat/deltachat-core-rust/compare/v1.116.0...v1.117.0
 [1.118.0]: https://github.com/deltachat/deltachat-core-rust/compare/v1.117.0...v1.118.0
+[1.119.0]: https://github.com/deltachat/deltachat-core-rust/compare/v1.118.0...v1.119.0
+[1.119.1]: https://github.com/deltachat/deltachat-core-rust/compare/v1.119.0...v1.119.1
+[1.120.0]: https://github.com/deltachat/deltachat-core-rust/compare/v1.119.1...v1.120.0
+[1.121.0]: https://github.com/deltachat/deltachat-core-rust/compare/v1.120.0...v1.121.0
+[1.122.0]: https://github.com/deltachat/deltachat-core-rust/compare/v1.121.0...v1.122.0
--- a/Cargo.lock
+++ b/Cargo.lock
@@ -221,13 +221,13 @@ dependencies = [

 [[package]]
 name = "async-imap"
-version = "0.9.0"
+version = "0.9.1"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "da93622739d458dd9a6abc1abf0e38e81965a5824a3b37f9500437c82a8bb572"
+checksum = "b538b767cbf9c162a6c5795d4b932bd2c20ba10b5a91a94d2b2b6886c1dce6a8"
 dependencies = [
 "async-channel",
 "base64 0.21.2",
- "byte-pool",
+ "bytes",
 "chrono",
 "futures",
 "imap-proto",
@@ -544,16 +544,6 @@ version = "3.13.0"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "a3e2c3daef883ecc1b5d58c15adae93470a91d425f3532ba1695849656af3fc1"

-[[package]]
-name = "byte-pool"
-version = "0.2.4"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "c2f1b21189f50b5625efa6227cf45e9d4cfdc2e73582df2b879e9689e78a7158"
-dependencies = [
- "crossbeam-queue",
- "stable_deref_trait",
-]
-
 [[package]]
 name = "bytemuck"
 version = "1.13.1"
@@ -924,16 +914,6 @@ dependencies = [
 "scopeguard",
 ]

-[[package]]
-name = "crossbeam-queue"
-version = "0.3.8"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "d1cfb3ea8a53f37c40dea2c7bedcbd88bdfae54f5e2175d6ecaff1c988353add"
-dependencies = [
- "cfg-if",
- "crossbeam-utils",
-]
-
 [[package]]
 name = "crossbeam-utils"
 version = "0.8.16"
@@ -1123,7 +1103,7 @@ dependencies = [

 [[package]]
 name = "deltachat"
-version = "1.119.0"
+version = "1.122.0"
 dependencies = [
 "ansi_term",
 "anyhow",
@@ -1199,7 +1179,7 @@ dependencies = [

 [[package]]
 name = "deltachat-jsonrpc"
-version = "1.119.0"
+version = "1.122.0"
 dependencies = [
 "anyhow",
 "async-channel",
@@ -1223,7 +1203,7 @@ dependencies = [

 [[package]]
 name = "deltachat-repl"
-version = "1.119.0"
+version = "1.122.0"
 dependencies = [
 "ansi_term",
 "anyhow",
@@ -1238,7 +1218,7 @@ dependencies = [

 [[package]]
 name = "deltachat-rpc-server"
-version = "1.119.0"
+version = "1.122.0"
 dependencies = [
 "anyhow",
 "deltachat",
@@ -1263,7 +1243,7 @@ dependencies = [

 [[package]]
 name = "deltachat_ffi"
-version = "1.119.0"
+version = "1.122.0"
 dependencies = [
 "anyhow",
 "deltachat",
@@ -3098,11 +3078,11 @@ checksum = "624a8340c38c1b80fd549087862da4ba43e08858af025b236e509b6649fc13d5"

 [[package]]
 name = "openssl"
-version = "0.10.55"
+version = "0.10.57"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "345df152bc43501c5eb9e4654ff05f794effb78d4efe3d53abc158baddc0703d"
+checksum = "bac25ee399abb46215765b1cb35bc0212377e58a061560d8b29b024fd0430e7c"
 dependencies = [
- "bitflags 1.3.2",
+ "bitflags 2.3.3",
 "cfg-if",
 "foreign-types",
 "libc",
@@ -3130,18 +3110,18 @@ checksum = "ff011a302c396a5197692431fc1948019154afc178baf7d8e37367442a4601cf"

 [[package]]
 name = "openssl-src"
-version = "111.26.0+1.1.1u"
+version = "300.1.3+3.1.2"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "efc62c9f12b22b8f5208c23a7200a442b2e5999f8bdf80233852122b5a4f6f37"
+checksum = "cd2c101a165fff9935e34def4669595ab1c7847943c42be86e21503e482be107"
 dependencies = [
 "cc",
 ]

 [[package]]
 name = "openssl-sys"
-version = "0.9.90"
+version = "0.9.93"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "374533b0e45f3a7ced10fcaeccca020e66656bc03dac384f852e4e5a7a8104a6"
+checksum = "db4d56a4c0478783083cfafcc42493dd4a981d41669da64b4572a2a089b51b1d"
 dependencies = [
 "cc",
 "libc",
@@ -4596,12 +4576,6 @@ dependencies = [
 "zeroize",
 ]

-[[package]]
-name = "stable_deref_trait"
-version = "1.2.0"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "a8f112729512f8e442d81f95a8a7ddf2b7c6b8a1a6f509a95864142b30cab2d3"
-
 [[package]]
 name = "static_assertions"
 version = "1.1.0"
@@ -5474,9 +5448,9 @@ dependencies = [

 [[package]]
 name = "webpki"
-version = "0.22.0"
+version = "0.22.1"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "f095d78192e208183081cc07bc5515ef55216397af48b873e5edcd72637fa1bd"
+checksum = "f0e74f82d49d545ad128049b7e88f6576df2da6b02e9ce565c6f533be576957e"
 dependencies = [
 "ring",
 "untrusted",
@@ -5778,9 +5752,9 @@ dependencies = [

 [[package]]
 name = "xattr"
-version = "1.0.0"
+version = "1.0.1"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "ea263437ca03c1522846a4ddafbca2542d0ad5ed9b784909d4b27b76f62bc34a"
+checksum = "f4686009f71ff3e5c4dbcf1a282d0a44db3f021ba69350cd42086b3e5f1c6985"
 dependencies = [
 "libc",
 ]
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -1,6 +1,6 @@
 [package]
 name = "deltachat"
-version = "1.119.0"
+version = "1.122.0"
 edition = "2021"
 license = "MPL-2.0"
 rust-version = "1.65"
@@ -36,7 +36,7 @@ ratelimit = { path = "./deltachat-ratelimit" }

 anyhow = "1"
 async-channel = "1.8.0"
-async-imap = { version = "0.9.0", default-features = false, features = ["runtime-tokio"] }
+async-imap = { version = "0.9.1", default-features = false, features = ["runtime-tokio"] }
 async-native-tls = { version = "0.5", default-features = false, features = ["runtime-tokio"] }
 async-smtp = { version = "0.9", default-features = false, features = ["runtime-tokio"] }
 async_zip = { version = "0.0.12", default-features = false, features = ["deflate", "fs"] }
--- a/deltachat-ffi/Cargo.toml
+++ b/deltachat-ffi/Cargo.toml
@@ -1,6 +1,6 @@
 [package]
 name = "deltachat_ffi"
-version = "1.119.0"
+version = "1.122.0"
 description = "Deltachat FFI"
 edition = "2018"
 readme = "README.md"
--- a/deltachat-ffi/deltachat.h
+++ b/deltachat-ffi/deltachat.h
@@ -301,6 +301,19 @@ dc_context_t*   dc_context_new_closed        (const char* dbfile);
 int             dc_context_open              (dc_context_t *context, const char* passphrase);


+/**
+ * Changes the passphrase on the open database.
+ * Existing database must already be encrypted and the passphrase cannot be NULL or empty.
+ * It is impossible to encrypt unencrypted database with this method and vice versa.
+ *
+ * @memberof dc_context_t
+ * @param context The context object.
+ * @param passphrase The new passphrase.
+ * @return 1 on success, 0 on error.
+ */
+int             dc_context_change_passphrase (dc_context_t* context, const char* passphrase);
+
+
 /**
 * Returns 1 if database is open.
 *
@@ -430,7 +443,9 @@ char*           dc_get_blobdir               (const dc_context_t* context);
 *                    DC_KEY_GEN_RSA2048 (1)=
 *                    generate RSA 2048 keypair
 *                    DC_KEY_GEN_ED25519 (2)=
- *                    generate Ed25519 keypair
+ *                    generate Curve25519 keypair
+ *                    DC_KEY_GEN_RSA4096 (3)=
+ *                    generate RSA 4096 keypair
 * - `save_mime_headers` = 1=save mime headers
 *                    and make dc_get_mime_headers() work for subsequent calls,
 *                    0=do not save mime headers (default)
@@ -1160,6 +1175,24 @@ int dc_send_webxdc_status_update (dc_context_t* context, uint32_t msg_id, const
 */
 char* dc_get_webxdc_status_updates (dc_context_t* context, uint32_t msg_id, uint32_t serial);

+
+/**
+ * Replaces webxdc app with a new version.
+ *
+ * On the JavaScript side this API could be used like this:
+ * ```
+ * window.webxdc.replaceWebxdc(blob);
+ * ```
+ *
+ * @memberof dc_context_t
+ * @param context The context object.
+ * @param msg_id The ID of the WebXDC message to be replaced.
+ * @param blob New blob to replace WebXDC with.
+ * @param n Blob size.
+ */
+void dc_replace_webxdc(dc_context_t* context, uint32_t msg_id, uint8_t *blob, size_t n);
+
+
 /**
 * Save a draft for a chat in the database.
 *
@@ -1321,6 +1354,20 @@ int             dc_get_msg_cnt               (dc_context_t* context, uint32_t ch
 int             dc_get_fresh_msg_cnt         (dc_context_t* context, uint32_t chat_id);


+/**
+ * Returns a list of similar chats.
+ *
+ * @warning This is an experimental API which may change or be removed in the future.
+ *
+ * @memberof dc_context_t
+ * @param context The context object as returned from dc_context_new().
+ * @param chat_id The ID of the chat for which to find similar chats.
+ * @return The list of similar chats.
+ *     On errors, NULL is returned.
+ *     Must be freed using dc_chatlist_unref() when no longer used.
+ */
+dc_chatlist_t*     dc_get_similar_chatlist   (dc_context_t* context, uint32_t chat_id);
+

 /**
 * Estimate the number of messages that will be deleted
@@ -2255,6 +2302,7 @@ dc_contact_t*   dc_get_contact               (dc_context_t* context, uint32_t co
 *
 * - **DC_IMEX_IMPORT_SELF_KEYS** (2) - Import private keys found in the directory given as `param1`.
 *   The last imported key is made the default keys unless its name contains the string `legacy`. Public keys are not imported.
+ *   If `param1` is a filename, import the private key from the file and make it the default.
 *
 * While dc_imex() returns immediately, the started job may take a while,
 * you can stop it using dc_stop_ongoing_process(). During execution of the job,
@@ -6266,6 +6314,7 @@ void dc_event_unref(dc_event_t* event);
 #define DC_KEY_GEN_DEFAULT 0
 #define DC_KEY_GEN_RSA2048 1
 #define DC_KEY_GEN_ED25519 2
+#define DC_KEY_GEN_RSA4096 3


 /**
--- a/deltachat-ffi/src/lib.rs
+++ b/deltachat-ffi/src/lib.rs
@@ -36,7 +36,7 @@ use deltachat::qr_code_generator::{generate_backup_qr, get_securejoin_qr_svg};
 use deltachat::reaction::{get_msg_reactions, send_reaction, Reactions};
 use deltachat::stock_str::StockMessage;
 use deltachat::stock_str::StockStrings;
-use deltachat::webxdc::StatusUpdateSerial;
+use deltachat::webxdc::{replace_webxdc, StatusUpdateSerial};
 use deltachat::*;
 use deltachat::{accounts::Accounts, log::LogExt};
 use num_traits::{FromPrimitive, ToPrimitive};
@@ -167,6 +167,24 @@ pub unsafe extern "C" fn dc_context_open(
        .unwrap_or(0)
 }

+#[no_mangle]
+pub unsafe extern "C" fn dc_context_change_passphrase(
+    context: *mut dc_context_t,
+    passphrase: *const libc::c_char,
+) -> libc::c_int {
+    if context.is_null() {
+        eprintln!("ignoring careless call to dc_context_change_passphrase()");
+        return 0;
+    }
+
+    let ctx = &*context;
+    let passphrase = to_string_lossy(passphrase);
+    block_on(ctx.change_passphrase(passphrase))
+        .context("dc_context_change_passphrase() failed")
+        .log_err(ctx)
+        .is_ok() as libc::c_int
+}
+
 #[no_mangle]
 pub unsafe extern "C" fn dc_context_is_open(context: *mut dc_context_t) -> libc::c_int {
    if context.is_null() {
@@ -1079,6 +1097,32 @@ pub unsafe extern "C" fn dc_get_webxdc_status_updates(
    .strdup()
 }

+#[no_mangle]
+pub unsafe extern "C" fn dc_replace_webxdc(
+    context: *mut dc_context_t,
+    msg_id: u32,
+    blob: *const u8,
+    n: libc::size_t,
+) {
+    if context.is_null() {
+        eprintln!("ignoring careless call to dc_replace_webxdc()");
+        return;
+    }
+
+    let msg_id = MsgId::new(msg_id);
+    let blob_slice = std::slice::from_raw_parts(blob, n);
+
+    let ctx = &*context;
+
+    block_on(async move {
+        replace_webxdc(ctx, msg_id, blob_slice)
+            .await
+            .context("Failed to replace WebXDC")
+            .log_err(ctx)
+            .ok();
+    })
+}
+
 #[no_mangle]
 pub unsafe extern "C" fn dc_set_draft(
    context: *mut dc_context_t,
@@ -1242,6 +1286,30 @@ pub unsafe extern "C" fn dc_get_fresh_msg_cnt(
    })
 }

+#[no_mangle]
+pub unsafe extern "C" fn dc_get_similar_chatlist(
+    context: *mut dc_context_t,
+    chat_id: u32,
+) -> *mut dc_chatlist_t {
+    if context.is_null() {
+        eprintln!("ignoring careless call to dc_get_similar_chatlist()");
+        return ptr::null_mut();
+    }
+    let ctx = &*context;
+
+    let chat_id = ChatId::new(chat_id);
+    match block_on(chat_id.get_similar_chatlist(ctx))
+        .context("failed to get similar chatlist")
+        .log_err(ctx)
+    {
+        Ok(list) => {
+            let ffi_list = ChatlistWrapper { context, list };
+            Box::into_raw(Box::new(ffi_list))
+        }
+        Err(_) => ptr::null_mut(),
+    }
+}
+
 #[no_mangle]
 pub unsafe extern "C" fn dc_estimate_deletion_cnt(
    context: *mut dc_context_t,
@@ -1497,14 +1565,10 @@ pub unsafe extern "C" fn dc_delete_chat(context: *mut dc_context_t, chat_id: u32
    }
    let ctx = &*context;

-    block_on(async move {
-        ChatId::new(chat_id)
-            .delete(ctx)
-            .await
-            .context("Failed chat delete")
-            .log_err(ctx)
-            .ok();
-    })
+    block_on(ChatId::new(chat_id).delete(ctx))
+        .context("Failed chat delete")
+        .log_err(ctx)
+        .ok();
 }

 #[no_mangle]
--- a/deltachat-jsonrpc/Cargo.toml
+++ b/deltachat-jsonrpc/Cargo.toml
@@ -1,6 +1,6 @@
 [package]
 name = "deltachat-jsonrpc"
-version = "1.119.0"
+version = "1.122.0"
 description = "DeltaChat JSON-RPC API"
 edition = "2021"
 default-run = "deltachat-jsonrpc-server"
--- a/deltachat-jsonrpc/src/api/mod.rs
+++ b/deltachat-jsonrpc/src/api/mod.rs
@@ -566,6 +566,21 @@ impl CommandApi {
        Ok(l)
    }

+    /// Returns chats similar to the given one.
+    ///
+    /// Experimental API, subject to change without notice.
+    async fn get_similar_chat_ids(&self, account_id: u32, chat_id: u32) -> Result<Vec<u32>> {
+        let ctx = self.get_context(account_id).await?;
+        let chat_id = ChatId::new(chat_id);
+        let list = chat_id
+            .get_similar_chat_ids(&ctx)
+            .await?
+            .into_iter()
+            .map(|(chat_id, _metric)| chat_id.to_u32())
+            .collect();
+        Ok(list)
+    }
+
    async fn get_chatlist_items_by_entries(
        &self,
        account_id: u32,
@@ -1709,6 +1724,20 @@ impl CommandApi {
        forward_msgs(&ctx, &message_ids, ChatId::new(chat_id)).await
    }

+    /// Resend messages and make information available for newly added chat members.
+    /// Resending sends out the original message, however, recipients and webxdc-status may differ.
+    /// Clients that already have the original message can still ignore the resent message as
+    /// they have tracked the state by dedicated updates.
+    ///
+    /// Some messages cannot be resent, eg. info-messages, drafts, already pending messages or messages that are not sent by SELF.
+    ///
+    /// message_ids all message IDs that should be resend. All messages must belong to the same chat.
+    async fn resend_messages(&self, account_id: u32, message_ids: Vec<u32>) -> Result<()> {
+        let ctx = self.get_context(account_id).await?;
+        let message_ids: Vec<MsgId> = message_ids.into_iter().map(MsgId::new).collect();
+        chat::resend_msgs(&ctx, &message_ids).await
+    }
+
    async fn send_sticker(
        &self,
        account_id: u32,
--- a/deltachat-jsonrpc/src/api/types/chat_list.rs
+++ b/deltachat-jsonrpc/src/api/types/chat_list.rs
@@ -8,15 +8,12 @@ use deltachat::{
    chatlist::Chatlist,
 };
 use num_traits::cast::ToPrimitive;
-use serde::{Deserialize, Serialize};
+use serde::Serialize;
 use typescript_type_def::TypeDef;

 use super::color_int_to_hex_string;
 use super::message::MessageViewtype;

-#[derive(Deserialize, Serialize, TypeDef, schemars::JsonSchema)]
-pub struct ChatListEntry(pub u32, pub u32);
-
 #[derive(Serialize, TypeDef, schemars::JsonSchema)]
 #[serde(tag = "type")]
 pub enum ChatListItemFetchResult {
--- a/deltachat-jsonrpc/src/api/types/message.rs
+++ b/deltachat-jsonrpc/src/api/types/message.rs
@@ -318,6 +318,7 @@ pub enum DownloadState {
    Done,
    Available,
    Failure,
+    Undecipherable,
    InProgress,
 }

@@ -327,6 +328,7 @@ impl From<download::DownloadState> for DownloadState {
            download::DownloadState::Done => DownloadState::Done,
            download::DownloadState::Available => DownloadState::Available,
            download::DownloadState::Failure => DownloadState::Failure,
+            download::DownloadState::Undecipherable => DownloadState::Undecipherable,
            download::DownloadState::InProgress => DownloadState::InProgress,
        }
    }
--- a/deltachat-jsonrpc/typescript/package.json
+++ b/deltachat-jsonrpc/typescript/package.json
@@ -55,5 +55,5 @@
  },
  "type": "module",
  "types": "dist/deltachat.d.ts",
-  "version": "1.119.0"
+  "version": "1.122.0"
 }
--- a/deltachat-repl/Cargo.toml
+++ b/deltachat-repl/Cargo.toml
@@ -1,6 +1,6 @@
 [package]
 name = "deltachat-repl"
-version = "1.119.0"
+version = "1.122.0"
 license = "MPL-2.0"
 edition = "2021"

--- a/deltachat-repl/src/cmdline.rs
+++ b/deltachat-repl/src/cmdline.rs
@@ -187,6 +187,7 @@ async fn log_msg(context: &Context, prefix: impl AsRef<str>, msg: &Message) {
        DownloadState::Available => " [⬇ Download available]",
        DownloadState::InProgress => " [⬇ Download in progress...]️",
        DownloadState::Failure => " [⬇ Download failed]",
+        DownloadState::Undecipherable => " [⬇ Decryption failed]",
    };

    let temp2 = timestamp_to_str(msg.get_timestamp());
@@ -805,15 +806,30 @@ pub async fn cmdline(context: Context, line: &str, chat_id: &mut ChatId) -> Resu
        }
        "chatinfo" => {
            ensure!(sel_chat.is_some(), "No chat selected.");
+            let sel_chat_id = sel_chat.as_ref().unwrap().get_id();

-            let contacts =
-                chat::get_chat_contacts(&context, sel_chat.as_ref().unwrap().get_id()).await?;
+            let contacts = chat::get_chat_contacts(&context, sel_chat_id).await?;
            println!("Memberlist:");

            log_contactlist(&context, &contacts).await?;
+            println!("{} contacts", contacts.len());
+
+            let similar_chats = sel_chat_id.get_similar_chat_ids(&context).await?;
+            if !similar_chats.is_empty() {
+                println!("Similar chats: ");
+                for (similar_chat_id, metric) in similar_chats {
+                    let similar_chat = Chat::load_from_db(&context, similar_chat_id).await?;
+                    println!(
+                        "{} (#{}) {:.1}",
+                        similar_chat.name,
+                        similar_chat_id,
+                        100.0 * metric
+                    );
+                }
+            }
+
            println!(
-                "{} contacts\nLocation streaming: {}",
-                contacts.len(),
+                "Location streaming: {}",
                location::is_sending_locations_to_chat(
                    &context,
                    Some(sel_chat.as_ref().unwrap().get_id())
--- a/deltachat-rpc-client/pyproject.toml
+++ b/deltachat-rpc-client/pyproject.toml
@@ -6,8 +6,7 @@ build-backend = "setuptools.build_meta"
 name = "deltachat-rpc-client"
 description = "Python client for Delta Chat core JSON-RPC interface"
 dependencies = [
-    "aiohttp",
-    "aiodns"
+    "aiohttp"
 ]
 classifiers = [
    "Development Status :: 5 - Production/Stable",
--- a/deltachat-rpc-client/src/deltachat_rpc_client/rpc.py
+++ b/deltachat-rpc-client/src/deltachat_rpc_client/rpc.py
@@ -89,16 +89,14 @@ class Rpc:
        return await queue.get()

    def __getattr__(self, attr: str):
-        async def method(*args, **kwargs) -> Any:
+        async def method(*args) -> Any:
            self.id += 1
            request_id = self.id

-            assert not (args and kwargs), "Mixing positional and keyword arguments"
-
            request = {
                "jsonrpc": "2.0",
                "method": attr,
-                "params": kwargs or args,
+                "params": args,
                "id": self.id,
            }
            data = (json.dumps(request) + "\n").encode()
--- a/deltachat-rpc-server/Cargo.toml
+++ b/deltachat-rpc-server/Cargo.toml
@@ -1,6 +1,6 @@
 [package]
 name = "deltachat-rpc-server"
-version = "1.119.0"
+version = "1.122.0"
 description = "DeltaChat JSON-RPC server"
 edition = "2021"
 readme = "README.md"
--- a/deny.toml
+++ b/deny.toml
@@ -2,6 +2,7 @@
 unmaintained = "allow"
 ignore = [
    "RUSTSEC-2020-0071",
+    "RUSTSEC-2022-0093",
 ]

 [bans]
--- a/node/constants.js
+++ b/node/constants.js
@@ -90,6 +90,7 @@ module.exports = {
  DC_KEY_GEN_DEFAULT: 0,
  DC_KEY_GEN_ED25519: 2,
  DC_KEY_GEN_RSA2048: 1,
+  DC_KEY_GEN_RSA4096: 3,
  DC_LP_AUTH_NORMAL: 4,
  DC_LP_AUTH_OAUTH2: 2,
  DC_MEDIA_QUALITY_BALANCED: 0,
--- a/node/lib/constants.ts
+++ b/node/lib/constants.ts
@@ -90,6 +90,7 @@ export enum C {
  DC_KEY_GEN_DEFAULT = 0,
  DC_KEY_GEN_ED25519 = 2,
  DC_KEY_GEN_RSA2048 = 1,
+  DC_KEY_GEN_RSA4096 = 3,
  DC_LP_AUTH_NORMAL = 4,
  DC_LP_AUTH_OAUTH2 = 2,
  DC_MEDIA_QUALITY_BALANCED = 0,
--- a/package.json
+++ b/package.json
@@ -60,5 +60,5 @@
    "test:mocha": "mocha -r esm node/test/test.js --growl --reporter=spec --bail --exit"
  },
  "types": "node/dist/index.d.ts",
-  "version": "1.119.0"
+  "version": "1.122.0"
 }
--- a/python/pyproject.toml
+++ b/python/pyproject.toml
@@ -34,6 +34,7 @@ dynamic = [
 "Home" = "https://github.com/deltachat/deltachat-core-rust/"
 "Bug Tracker" = "https://github.com/deltachat/deltachat-core-rust/issues"
 "Documentation" = "https://py.delta.chat/"
+"Mastodon" = "https://chaos.social/@delta"

 [project.entry-points.pytest11]
 "deltachat.testplugin" = "deltachat.testplugin"
--- a/python/src/deltachat/account.py
+++ b/python/src/deltachat/account.py
@@ -427,7 +427,7 @@ class Account:

        assert dc_chatlist != ffi.NULL
        chatlist = []
-        for i in range(0, lib.dc_chatlist_get_cnt(dc_chatlist)):
+        for i in range(lib.dc_chatlist_get_cnt(dc_chatlist)):
            chat_id = lib.dc_chatlist_get_chat_id(dc_chatlist, i)
            chatlist.append(Chat(self, chat_id))
        return chatlist
--- a/python/src/deltachat/cutil.py
+++ b/python/src/deltachat/cutil.py
@@ -15,7 +15,7 @@ def as_dc_charpointer(obj):


 def iter_array(dc_array_t, constructor: Callable[[int], T]) -> Generator[T, None, None]:
-    for i in range(0, lib.dc_array_get_cnt(dc_array_t)):
+    for i in range(lib.dc_array_get_cnt(dc_array_t)):
        yield constructor(lib.dc_array_get_id(dc_array_t, i))


--- a/python/src/deltachat/message.py
+++ b/python/src/deltachat/message.py
@@ -510,8 +510,7 @@ def get_viewtype_code_from_name(view_type_name):
    if code is not None:
        return code
    raise ValueError(
-        "message typecode not found for {!r}, "
-        "available {!r}".format(view_type_name, list(_view_type_mapping.keys())),
+        f"message typecode not found for {view_type_name!r}, available {list(_view_type_mapping.keys())!r}",
    )


--- a/python/tests/bench_test_setup.py
+++ b/python/tests/bench_test_setup.py
@@ -15,6 +15,6 @@ class TestEmpty:
    def test_prepare_setup_measurings(self, acfactory):
        acfactory.get_online_accounts(BENCH_NUM)

-    @pytest.mark.parametrize("num", range(0, BENCH_NUM + 1))
+    @pytest.mark.parametrize("num", range(BENCH_NUM + 1))
    def test_setup_online_accounts(self, acfactory, num):
        acfactory.get_online_accounts(num)
--- a/python/tests/test_1_online.py
+++ b/python/tests/test_1_online.py
@@ -1679,6 +1679,36 @@ def test_qr_join_chat(acfactory, lp):
    ac1._evtracker.wait_securejoin_inviter_progress(1000)


+def test_qr_new_group_unblocked(acfactory, lp):
+    """Regression test for a bug intoduced in core v1.113.0.
+    ac2 scans a verified group QR code created by ac1.
+    This results in creation of a blocked 1:1 chat with ac1 on ac2,
+    but ac1 contact is not blocked on ac2.
+    Then ac1 creates a group, adds ac2 there and promotes it by sending a message.
+    ac2 should receive a message and create a contact request for the group.
+    Due to a bug previously ac2 created a blocked group.
+    """
+
+    ac1, ac2 = acfactory.get_online_accounts(2)
+    ac1_chat = ac1.create_group_chat("Group for joining", verified=True)
+    qr = ac1_chat.get_join_qr()
+    ac2.qr_join_chat(qr)
+
+    ac1._evtracker.wait_securejoin_inviter_progress(1000)
+
+    ac1_new_chat = ac1.create_group_chat("Another group")
+    ac1_new_chat.add_contact(ac2)
+    ac1_new_chat.send_text("Hello!")
+
+    # Receive "Member added" message.
+    ac2._evtracker.wait_next_incoming_message()
+
+    # Receive "Hello!" message.
+    ac2_msg = ac2._evtracker.wait_next_incoming_message()
+    assert ac2_msg.text == "Hello!"
+    assert ac2_msg.chat.is_contact_request()
+
+
 def test_qr_email_capitalization(acfactory, lp):
    """Regression test for a bug
    that resulted in failure to propagate verification via gossip in a verified group
--- a/release-date.in
+++ b/release-date.in
@@ -1 +1 @@
-2023-08-03
+2023-09-12
--- a/scripts/coredeps/install-rust.sh
+++ b/scripts/coredeps/install-rust.sh
@@ -7,7 +7,7 @@ set -euo pipefail
 #
 # Avoid using rustup here as it depends on reading /proc/self/exe and
 # has problems running under QEMU.
-RUST_VERSION=1.68.0
+RUST_VERSION=1.72.0

 ARCH="$(uname -m)"
 test -f "/lib/libc.musl-$ARCH.so.1" && LIBC=musl || LIBC=gnu
--- a/scripts/create-provider-data-rs.py
+++ b/scripts/create-provider-data-rs.py
@@ -44,7 +44,7 @@ def file2url(f):

 def process_opt(data):
    if not "opt" in data:
-        return "Default::default()"
+        return "ProviderOptions::new()"
    opt = "ProviderOptions {\n"
    opt_data = data.get("opt", "")
    for key in opt_data:
@@ -54,7 +54,7 @@ def process_opt(data):
        if value in {"True", "False"}:
            value = value.lower()
        opt += "        " + key + ": " + value + ",\n"
-    opt += "        ..Default::default()\n"
+    opt += "        ..ProviderOptions::new()\n"
    opt += "    }"
    return opt

@@ -62,7 +62,7 @@ def process_opt(data):
 def process_config_defaults(data):
    if not "config_defaults" in data:
        return "None"
-    defaults = "Some(vec![\n"
+    defaults = "Some(&[\n"
    config_defaults = data.get("config_defaults", "")
    for key in config_defaults:
        value = str(config_defaults[key])
@@ -96,11 +96,11 @@ def process_data(data, file):
            raise TypeError("domain used twice: " + domain)
        domains_set.add(domain)

-        domains += '    ("' + domain + '", &*' + file2varname(file) + "),\n"
+        domains += '    ("' + domain + '", &' + file2varname(file) + "),\n"
        comment += domain + ", "

    ids = ""
-    ids += '    ("' + file2id(file) + '", &*' + file2varname(file) + "),\n"
+    ids += '    ("' + file2id(file) + '", &' + file2varname(file) + "),\n"

    server = ""
    has_imap = False
@@ -155,18 +155,18 @@ def process_data(data, file):
        provider += (
            "static "
            + file2varname(file)
-            + ": Lazy<Provider> = Lazy::new(|| Provider {\n"
+            + ": Provider = Provider {\n"
        )
        provider += '    id: "' + file2id(file) + '",\n'
        provider += "    status: Status::" + status.capitalize() + ",\n"
        provider += '    before_login_hint: "' + before_login_hint + '",\n'
        provider += '    after_login_hint: "' + after_login_hint + '",\n'
        provider += '    overview_page: "' + file2url(file) + '",\n'
-        provider += "    server: vec![\n" + server + "    ],\n"
+        provider += "    server: &[\n" + server + "    ],\n"
        provider += "    opt: " + opt + ",\n"
        provider += "    config_defaults: " + config_defaults + ",\n"
        provider += "    oauth2_authorizer: " + oauth2 + ",\n"
-        provider += "});\n\n"
+        provider += "};\n\n"
    else:
        raise TypeError("SMTP and IMAP must be specified together or left out both")

--- a/scripts/zig-musl-check.sh
+++ b/scripts/zig-musl-check.sh
@@ -9,9 +9,9 @@ set -e
 unset RUSTFLAGS

 # Pin Rust version to avoid uncontrolled changes in the compiler and linker flags.
-export RUSTUP_TOOLCHAIN=1.70.0
+export RUSTUP_TOOLCHAIN=1.72.0

-ZIG_VERSION=0.11.0-dev.2213+515e1c93e
+ZIG_VERSION=0.11.0

 # Download Zig
 rm -fr "$ZIG_VERSION" "zig-linux-x86_64-$ZIG_VERSION.tar.xz"
--- a/scripts/zig-rpc-server.sh
+++ b/scripts/zig-rpc-server.sh
@@ -8,9 +8,9 @@ set -e
 unset RUSTFLAGS

 # Pin Rust version to avoid uncontrolled changes in the compiler and linker flags.
-export RUSTUP_TOOLCHAIN=1.70.0
+export RUSTUP_TOOLCHAIN=1.72.0

-ZIG_VERSION=0.11.0-dev.2213+515e1c93e
+ZIG_VERSION=0.11.0

 # Download Zig
 rm -fr "$ZIG_VERSION" "zig-linux-x86_64-$ZIG_VERSION.tar.xz"
--- a/src/accounts.rs
+++ b/src/accounts.rs
@@ -296,10 +296,10 @@ impl Accounts {
 }

 /// Configuration file name.
-pub const CONFIG_NAME: &str = "accounts.toml";
+const CONFIG_NAME: &str = "accounts.toml";

 /// Database file name.
-pub const DB_NAME: &str = "dc.db";
+const DB_NAME: &str = "dc.db";

 /// Account manager configuration file.
 #[derive(Debug, Clone, PartialEq)]
--- a/src/blob.rs
+++ b/src/blob.rs
@@ -9,7 +9,7 @@ use std::path::{Path, PathBuf};

 use anyhow::{format_err, Context as _, Result};
 use futures::StreamExt;
-use image::{DynamicImage, ImageFormat, ImageOutputFormat};
+use image::{DynamicImage, GenericImageView, ImageFormat, ImageOutputFormat};
 use num_traits::FromPrimitive;
 use tokio::io::AsyncWriteExt;
 use tokio::{fs, io};
@@ -323,18 +323,35 @@ impl<'a> BlobObject<'a> {
                MediaQuality::Worse => constants::WORSE_AVATAR_SIZE,
            };

+        let maybe_sticker = &mut false;
        let strict_limits = true;
        // max_bytes is 20_000 bytes: Outlook servers don't allow headers larger than 32k.
        // 32 / 4 * 3 = 24k if you account for base64 encoding. To be safe, we reduced this to 20k.
-        if let Some(new_name) =
-            self.recode_to_size(context, blob_abs, img_wh, 20_000, strict_limits)?
-        {
+        if let Some(new_name) = self.recode_to_size(
+            context,
+            blob_abs,
+            maybe_sticker,
+            img_wh,
+            20_000,
+            strict_limits,
+        )? {
            self.name = new_name;
        }
        Ok(())
    }

-    pub async fn recode_to_image_size(&mut self, context: &Context) -> Result<()> {
+    /// Recodes an image pointed by a [BlobObject] so that it fits into limits on the image width,
+    /// height and file size specified by the config.
+    ///
+    /// On some platforms images are passed to the core as [`crate::message::Viewtype::Sticker`] in
+    /// which case `maybe_sticker` flag should be set. We recheck if an image is a true sticker
+    /// assuming that it must have at least one fully transparent corner, otherwise this flag is
+    /// reset.
+    pub async fn recode_to_image_size(
+        &mut self,
+        context: &Context,
+        maybe_sticker: &mut bool,
+    ) -> Result<()> {
        let blob_abs = self.to_abs_path();
        let (img_wh, max_bytes) =
            match MediaQuality::from_i32(context.get_config_int(Config::MediaQuality).await?)
@@ -347,9 +364,14 @@ impl<'a> BlobObject<'a> {
                MediaQuality::Worse => (constants::WORSE_IMAGE_SIZE, constants::WORSE_IMAGE_BYTES),
            };
        let strict_limits = false;
-        if let Some(new_name) =
-            self.recode_to_size(context, blob_abs, img_wh, max_bytes, strict_limits)?
-        {
+        if let Some(new_name) = self.recode_to_size(
+            context,
+            blob_abs,
+            maybe_sticker,
+            img_wh,
+            max_bytes,
+            strict_limits,
+        )? {
            self.name = new_name;
        }
        Ok(())
@@ -358,20 +380,37 @@ impl<'a> BlobObject<'a> {
    /// If `!strict_limits`, then if `max_bytes` is exceeded, reduce the image to `img_wh` and just
    /// proceed with the result.
    fn recode_to_size(
-        &self,
+        &mut self,
        context: &Context,
        mut blob_abs: PathBuf,
+        maybe_sticker: &mut bool,
        mut img_wh: u32,
        max_bytes: usize,
        strict_limits: bool,
    ) -> Result<Option<String>> {
-        tokio::task::block_in_place(move || {
-            let mut img = image::open(&blob_abs).context("image decode failure")?;
+        let mut no_exif = false;
+        let no_exif_ref = &mut no_exif;
+        let res = tokio::task::block_in_place(move || {
            let (nr_bytes, exif) = self.metadata()?;
+            *no_exif_ref = exif.is_none();
+            let mut img = image::open(&blob_abs).context("image decode failure")?;
            let orientation = exif.as_ref().map(|exif| exif_orientation(exif, context));
            let mut encoded = Vec::new();
            let mut changed_name = None;

+            if *maybe_sticker {
+                let x_max = img.width().saturating_sub(1);
+                let y_max = img.height().saturating_sub(1);
+                *maybe_sticker = img.in_bounds(x_max, y_max)
+                    && (img.get_pixel(0, 0).0[3] == 0
+                        || img.get_pixel(x_max, 0).0[3] == 0
+                        || img.get_pixel(0, y_max).0[3] == 0
+                        || img.get_pixel(x_max, y_max).0[3] == 0);
+            }
+            if *maybe_sticker && exif.is_none() {
+                return Ok(None);
+            }
+
            img = match orientation {
                Some(90) => img.rotate90(),
                Some(180) => img.rotate180(),
@@ -469,7 +508,21 @@ impl<'a> BlobObject<'a> {
            }

            Ok(changed_name)
-        })
+        });
+        match res {
+            Ok(_) => res,
+            Err(err) => {
+                if !strict_limits && no_exif {
+                    warn!(
+                        context,
+                        "Cannot recode image, using original data: {err:#}.",
+                    );
+                    Ok(None)
+                } else {
+                    Err(err)
+                }
+            }
+        }
    }

    /// Returns image file size and Exif.
@@ -860,10 +913,18 @@ mod tests {
            file.metadata().await.unwrap().len()
        }

-        let blob = BlobObject::new_from_path(&t, &avatar_blob).await.unwrap();
+        let mut blob = BlobObject::new_from_path(&t, &avatar_blob).await.unwrap();
+        let maybe_sticker = &mut false;
        let strict_limits = true;
-        blob.recode_to_size(&t, blob.to_abs_path(), 1000, 3000, strict_limits)
-            .unwrap();
+        blob.recode_to_size(
+            &t,
+            blob.to_abs_path(),
+            maybe_sticker,
+            1000,
+            3000,
+            strict_limits,
+        )
+        .unwrap();
        assert!(file_size(&avatar_blob).await <= 3000);
        assert!(file_size(&avatar_blob).await > 2000);
        tokio::task::block_in_place(move || {
@@ -923,6 +984,7 @@ mod tests {
    async fn test_recode_image_1() {
        let bytes = include_bytes!("../test-data/image/avatar1000x1000.jpg");
        send_image_check_mediaquality(
+            Viewtype::Image,
            Some("0"),
            bytes,
            "jpg",
@@ -936,6 +998,7 @@ mod tests {
        .await
        .unwrap();
        send_image_check_mediaquality(
+            Viewtype::Image,
            Some("1"),
            bytes,
            "jpg",
@@ -955,6 +1018,7 @@ mod tests {
        // The "-rotated" files are rotated by 270 degrees using the Exif metadata
        let bytes = include_bytes!("../test-data/image/rectangle2000x1800-rotated.jpg");
        let img_rotated = send_image_check_mediaquality(
+            Viewtype::Image,
            Some("0"),
            bytes,
            "jpg",
@@ -974,6 +1038,7 @@ mod tests {
        let bytes = buf.into_inner();

        let img_rotated = send_image_check_mediaquality(
+            Viewtype::Image,
            Some("1"),
            &bytes,
            "jpg",
@@ -994,6 +1059,7 @@ mod tests {
        let bytes = include_bytes!("../test-data/image/screenshot.png");

        send_image_check_mediaquality(
+            Viewtype::Image,
            Some("0"),
            bytes,
            "png",
@@ -1008,6 +1074,7 @@ mod tests {
        .unwrap();

        send_image_check_mediaquality(
+            Viewtype::Image,
            Some("1"),
            bytes,
            "png",
@@ -1020,12 +1087,29 @@ mod tests {
        )
        .await
        .unwrap();
+
+        // This will be sent as Image, see [`BlobObject::maybe_sticker`] for explanation.
+        send_image_check_mediaquality(
+            Viewtype::Sticker,
+            Some("0"),
+            bytes,
+            "png",
+            false, // no Exif
+            1920,
+            1080,
+            0,
+            1920,
+            1080,
+        )
+        .await
+        .unwrap();
    }

    #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
    async fn test_recode_image_huge_jpg() {
        let bytes = include_bytes!("../test-data/image/screenshot.jpg");
        send_image_check_mediaquality(
+            Viewtype::Image,
            Some("0"),
            bytes,
            "jpg",
@@ -1059,6 +1143,7 @@ mod tests {

    #[allow(clippy::too_many_arguments)]
    async fn send_image_check_mediaquality(
+        viewtype: Viewtype,
        media_quality_config: Option<&str>,
        bytes: &[u8],
        extension: &str,
@@ -1090,7 +1175,7 @@ mod tests {
            assert!(exif.is_none());
        }

-        let mut msg = Message::new(Viewtype::Image);
+        let mut msg = Message::new(viewtype);
        msg.set_file(file.to_str().unwrap(), None);
        let chat = alice.create_chat(&bob).await;
        let sent = alice.send_msg(chat.id, &mut msg).await;
@@ -1104,6 +1189,7 @@ mod tests {
        );

        let bob_msg = bob.recv_msg(&sent).await;
+        assert_eq!(bob_msg.get_viewtype(), Viewtype::Image);
        assert_eq!(bob_msg.get_width() as u32, compressed_width);
        assert_eq!(bob_msg.get_height() as u32, compressed_height);
        let file = bob_msg.get_file(&bob).unwrap();
--- a/src/chat.rs
+++ b/src/chat.rs
@@ -13,6 +13,7 @@ use serde::{Deserialize, Serialize};

 use crate::aheader::EncryptPreference;
 use crate::blob::BlobObject;
+use crate::chatlist::Chatlist;
 use crate::color::str_to_color;
 use crate::config::Config;
 use crate::constants::{
@@ -22,6 +23,7 @@ use crate::constants::{
 use crate::contact::{Contact, ContactId, Origin, VerifiedStatus};
 use crate::context::Context;
 use crate::debug_logging::maybe_set_logging_xdc;
+use crate::download::DownloadState;
 use crate::ephemeral::Timer as EphemeralTimer;
 use crate::events::EventType;
 use crate::html::new_html_mimepart;
@@ -871,6 +873,133 @@ impl ChatId {
        Ok(count)
    }

+    /// Returns timestamp of the latest message in the chat.
+    pub(crate) async fn get_timestamp(self, context: &Context) -> Result<Option<i64>> {
+        let timestamp = context
+            .sql
+            .query_get_value("SELECT MAX(timestamp) FROM msgs WHERE chat_id=?", (self,))
+            .await?;
+        Ok(timestamp)
+    }
+
+    /// Returns a list of active similar chat IDs sorted by similarity metric.
+    ///
+    /// Jaccard similarity coefficient is used to estimate similarity of chat member sets.
+    ///
+    /// Chat is considered active if something was posted there within the last 42 days.
+    pub async fn get_similar_chat_ids(self, context: &Context) -> Result<Vec<(ChatId, f64)>> {
+        // Count number of common members in this and other chats.
+        let intersection: Vec<(ChatId, f64)> = context
+            .sql
+            .query_map(
+                "SELECT y.chat_id, SUM(x.contact_id = y.contact_id)
+                 FROM chats_contacts as x
+                 JOIN chats_contacts as y
+                 WHERE x.contact_id > 9
+                   AND y.contact_id > 9
+                   AND x.chat_id=?
+                   AND y.chat_id<>x.chat_id
+                 GROUP BY y.chat_id",
+                (self,),
+                |row| {
+                    let chat_id: ChatId = row.get(0)?;
+                    let intersection: f64 = row.get(1)?;
+                    Ok((chat_id, intersection))
+                },
+                |rows| {
+                    rows.collect::<std::result::Result<Vec<_>, _>>()
+                        .map_err(Into::into)
+                },
+            )
+            .await
+            .context("failed to calculate member set intersections")?;
+
+        let chat_size: HashMap<ChatId, f64> = context
+            .sql
+            .query_map(
+                "SELECT chat_id, count(*) AS n
+                 FROM chats_contacts where contact_id > 9
+                 GROUP BY chat_id",
+                (),
+                |row| {
+                    let chat_id: ChatId = row.get(0)?;
+                    let size: f64 = row.get(1)?;
+                    Ok((chat_id, size))
+                },
+                |rows| {
+                    rows.collect::<std::result::Result<HashMap<ChatId, f64>, _>>()
+                        .map_err(Into::into)
+                },
+            )
+            .await
+            .context("failed to count chat member sizes")?;
+
+        let our_chat_size = chat_size.get(&self).copied().unwrap_or_default();
+        let mut chats_with_metrics = Vec::new();
+        for (chat_id, intersection_size) in intersection {
+            if intersection_size > 0.0 {
+                let other_chat_size = chat_size.get(&chat_id).copied().unwrap_or_default();
+                let union_size = our_chat_size + other_chat_size - intersection_size;
+                let metric = intersection_size / union_size;
+                chats_with_metrics.push((chat_id, metric))
+            }
+        }
+        chats_with_metrics.sort_unstable_by(|(chat_id1, metric1), (chat_id2, metric2)| {
+            metric2
+                .partial_cmp(metric1)
+                .unwrap_or(chat_id2.cmp(chat_id1))
+        });
+
+        // Select up to five similar active chats.
+        let mut res = Vec::new();
+        let now = time();
+        for (chat_id, metric) in chats_with_metrics {
+            if let Some(chat_timestamp) = chat_id.get_timestamp(context).await? {
+                if now > chat_timestamp + 42 * 24 * 3600 {
+                    // Chat was inactive for 42 days, skip.
+                    continue;
+                }
+            }
+
+            if metric < 0.1 {
+                // Chat is unrelated.
+                break;
+            }
+
+            let chat = Chat::load_from_db(context, chat_id).await?;
+            if chat.typ != Chattype::Group {
+                continue;
+            }
+
+            match chat.visibility {
+                ChatVisibility::Normal | ChatVisibility::Pinned => {}
+                ChatVisibility::Archived => continue,
+            }
+
+            res.push((chat_id, metric));
+            if res.len() >= 5 {
+                break;
+            }
+        }
+
+        Ok(res)
+    }
+
+    /// Returns similar chats as a [`Chatlist`].
+    ///
+    /// [`Chatlist`]: crate::chatlist::Chatlist
+    pub async fn get_similar_chatlist(self, context: &Context) -> Result<Chatlist> {
+        let chat_ids: Vec<ChatId> = self
+            .get_similar_chat_ids(context)
+            .await
+            .context("failed to get similar chat IDs")?
+            .into_iter()
+            .map(|(chat_id, _metric)| chat_id)
+            .collect();
+        let chatlist = Chatlist::from_chat_ids(context, &chat_ids).await?;
+        Ok(chatlist)
+    }
+
    pub(crate) async fn get_param(self, context: &Context) -> Result<Params> {
        let res: Option<String> = context
            .sql
@@ -910,11 +1039,14 @@ impl ChatId {
        T: Send + 'static,
    {
        let sql = &context.sql;
+        // Do not reply to not fully downloaded messages. Such a message could be a group chat
+        // message that we assigned to 1:1 chat.
        let query = format!(
            "SELECT {fields} \
-             FROM msgs WHERE chat_id=? AND state NOT IN (?, ?, ?, ?) AND NOT hidden \
+             FROM msgs WHERE chat_id=? AND state NOT IN (?, ?) AND NOT hidden AND download_state={} \
             ORDER BY timestamp DESC, id DESC \
-             LIMIT 1;"
+             LIMIT 1;",
+             DownloadState::Done as u32,
        );
        let row = sql
            .query_row_optional(
@@ -923,8 +1055,11 @@ impl ChatId {
                    self,
                    MessageState::OutPreparing,
                    MessageState::OutDraft,
-                    MessageState::OutPending,
-                    MessageState::OutFailed,
+                    // We don't filter `OutPending` and `OutFailed` messages because the new message
+                    // for which `parent_query()` is done may assume that it will be received in a
+                    // context affected by those messages, e.g. they could add new members to a
+                    // group and the new message will contain them in "To:". Anyway recipients must
+                    // be prepared to orphaned references.
                ),
                f,
            )
@@ -936,34 +1071,17 @@ impl ChatId {
        self,
        context: &Context,
    ) -> Result<Option<(String, String, String)>> {
-        if let Some((rfc724_mid, mime_in_reply_to, mime_references, error)) = self
-            .parent_query(
-                context,
-                "rfc724_mid, mime_in_reply_to, mime_references, error",
-                |row: &rusqlite::Row| {
-                    let rfc724_mid: String = row.get(0)?;
-                    let mime_in_reply_to: String = row.get(1)?;
-                    let mime_references: String = row.get(2)?;
-                    let error: String = row.get(3)?;
-                    Ok((rfc724_mid, mime_in_reply_to, mime_references, error))
-                },
-            )
-            .await?
-        {
-            if !error.is_empty() {
-                // Do not reply to error messages.
-                //
-                // An error message could be a group chat message that we failed to decrypt and
-                // assigned to 1:1 chat. A reply to it will show up as a reply to group message
-                // on the other side. To avoid such situations, it is better not to reply to
-                // error messages at all.
-                Ok(None)
-            } else {
-                Ok(Some((rfc724_mid, mime_in_reply_to, mime_references)))
-            }
-        } else {
-            Ok(None)
-        }
+        self.parent_query(
+            context,
+            "rfc724_mid, mime_in_reply_to, mime_references",
+            |row: &rusqlite::Row| {
+                let rfc724_mid: String = row.get(0)?;
+                let mime_in_reply_to: String = row.get(1)?;
+                let mime_references: String = row.get(2)?;
+                Ok((rfc724_mid, mime_in_reply_to, mime_references))
+            },
+        )
+        .await
    }

    /// Returns multi-line text summary of encryption preferences of all chat contacts.
@@ -1241,13 +1359,14 @@ impl Chat {
    pub(crate) async fn why_cant_send(&self, context: &Context) -> Result<Option<CantSendReason>> {
        use CantSendReason::*;

+        // NB: Don't forget to update Chatlist::try_load() when changing this function!
        let reason = if self.id.is_special() {
            Some(SpecialChat)
        } else if self.is_device_talk() {
            Some(DeviceChat)
        } else if self.is_contact_request() {
            Some(ContactRequest)
-        } else if self.is_mailing_list() && self.param.get(Param::ListPost).is_none_or_empty() {
+        } else if self.is_mailing_list() && self.get_mailinglist_addr().is_none_or_empty() {
            Some(ReadOnlyMailingList)
        } else if !self.is_self_in_chat(context).await? {
            Some(NotAMember)
@@ -1310,11 +1429,11 @@ impl Chat {
    pub async fn get_profile_image(&self, context: &Context) -> Result<Option<PathBuf>> {
        if let Some(image_rel) = self.param.get(Param::ProfileImage) {
            if !image_rel.is_empty() {
-                return Ok(Some(get_abs_path(context, image_rel)));
+                return Ok(Some(get_abs_path(context, Path::new(&image_rel))));
            }
        } else if self.id.is_archived_link() {
            if let Ok(image_rel) = get_archive_icon(context).await {
-                return Ok(Some(get_abs_path(context, image_rel)));
+                return Ok(Some(get_abs_path(context, Path::new(&image_rel))));
            }
        } else if self.typ == Chattype::Single {
            let contacts = get_chat_contacts(context, self.id).await?;
@@ -1325,7 +1444,7 @@ impl Chat {
            }
        } else if self.typ == Chattype::Broadcast {
            if let Ok(image_rel) = get_broadcast_icon(context).await {
-                return Ok(Some(get_abs_path(context, image_rel)));
+                return Ok(Some(get_abs_path(context, Path::new(&image_rel))));
            }
        }
        Ok(None)
@@ -1584,6 +1703,11 @@ impl Chat {
            None
        };

+        msg.chat_id = self.id;
+        msg.from_id = ContactId::SELF;
+        msg.rfc724_mid = new_rfc724_mid;
+        msg.timestamp_sort = timestamp;
+
        // add message to the database
        if let Some(update_msg_id) = update_msg_id {
            context
@@ -1597,11 +1721,11 @@ impl Chat {
                         ephemeral_timestamp=?
                     WHERE id=?;",
                    params_slice![
-                        new_rfc724_mid,
-                        self.id,
-                        ContactId::SELF,
+                        msg.rfc724_mid,
+                        msg.chat_id,
+                        msg.from_id,
                        to_id,
-                        timestamp,
+                        msg.timestamp_sort,
                        msg.viewtype,
                        msg.state,
                        msg.text,
@@ -1646,11 +1770,11 @@ impl Chat {
                        ephemeral_timestamp)
                        VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,1,?,?,?);",
                    params_slice![
-                        new_rfc724_mid,
-                        self.id,
-                        ContactId::SELF,
+                        msg.rfc724_mid,
+                        msg.chat_id,
+                        msg.from_id,
                        to_id,
-                        timestamp,
+                        msg.timestamp_sort,
                        msg.viewtype,
                        msg.state,
                        msg.text,
@@ -2033,12 +2157,12 @@ async fn prepare_msg_blob(context: &Context, msg: &mut Message) -> Result<()> {
            .await?
            .with_context(|| format!("attachment missing for message of type #{}", msg.viewtype))?;

-        if msg.viewtype == Viewtype::Image {
-            if let Err(err) = blob.recode_to_image_size(context).await {
-                warn!(
-                    context,
-                    "Cannot recode image, using original data: {err:#}."
-                );
+        let mut maybe_sticker = msg.viewtype == Viewtype::Sticker;
+        if msg.viewtype == Viewtype::Image || maybe_sticker {
+            blob.recode_to_image_size(context, &mut maybe_sticker)
+                .await?;
+            if !maybe_sticker {
+                msg.viewtype = Viewtype::Image;
            }
        }
        msg.param.set(Param::File, blob.as_name());
@@ -2085,6 +2209,8 @@ async fn prepare_msg_blob(context: &Context, msg: &mut Message) -> Result<()> {
            }
        }

+        msg.try_calc_and_set_dimensions(context).await?;
+
        info!(
            context,
            "Attaching \"{}\" for message type #{}.",
@@ -2252,7 +2378,7 @@ async fn prepare_send_msg(
        );
        message::update_msg_state(context, msg.id, MessageState::OutPending).await?;
    }
-    let row_id = create_send_msg_job(context, msg.id).await?;
+    let row_id = create_send_msg_job(context, msg).await?;
    Ok(row_id)
 }

@@ -2262,13 +2388,10 @@ async fn prepare_send_msg(
 /// group with only self and no BCC-to-self configured.
 ///
 /// The caller has to interrupt SMTP loop or otherwise process a new row.
-async fn create_send_msg_job(context: &Context, msg_id: MsgId) -> Result<Option<i64>> {
-    let mut msg = Message::load_from_db(context, msg_id).await?;
-    msg.try_calc_and_set_dimensions(context)
-        .await
-        .context("failed to calculate media dimensions")?;
-
-    /* create message */
+pub(crate) async fn create_send_msg_job(
+    context: &Context,
+    msg: &mut Message,
+) -> Result<Option<i64>> {
    let needs_encryption = msg.param.get_bool(Param::GuaranteeE2ee).unwrap_or_default();

    let attach_selfavatar = match shall_attach_selfavatar(context, msg.chat_id).await {
@@ -2279,7 +2402,7 @@ async fn create_send_msg_job(context: &Context, msg_id: MsgId) -> Result<Option<
        }
    };

-    let mimefactory = MimeFactory::from_msg(context, &msg, attach_selfavatar).await?;
+    let mimefactory = MimeFactory::from_msg(context, msg, attach_selfavatar).await?;

    let mut recipients = mimefactory.recipients();

@@ -2301,16 +2424,17 @@ async fn create_send_msg_job(context: &Context, msg_id: MsgId) -> Result<Option<
        // may happen eg. for groups with only SELF and bcc_self disabled
        info!(
            context,
-            "Message {msg_id} has no recipient, skipping smtp-send."
+            "Message {} has no recipient, skipping smtp-send.", msg.id
        );
-        msg_id.set_delivered(context).await?;
+        msg.id.set_delivered(context).await?;
+        msg.state = MessageState::OutDelivered;
        return Ok(None);
    }

    let rendered_msg = match mimefactory.render(context).await {
        Ok(res) => Ok(res),
        Err(err) => {
-            message::set_msg_failed(context, msg_id, &err.to_string()).await;
+            message::set_msg_failed(context, msg, &err.to_string()).await?;
            Err(err)
        }
    }?;
@@ -2319,13 +2443,13 @@ async fn create_send_msg_job(context: &Context, msg_id: MsgId) -> Result<Option<
        /* unrecoverable */
        message::set_msg_failed(
            context,
-            msg_id,
+            msg,
            "End-to-end-encryption unavailable unexpectedly.",
        )
-        .await;
+        .await?;
        bail!(
            "e2e encryption unavailable {} - {:?}",
-            msg_id,
+            msg.id,
            needs_encryption
        );
    }
@@ -2381,7 +2505,7 @@ async fn create_send_msg_job(context: &Context, msg_id: MsgId) -> Result<Option<
                &rendered_msg.rfc724_mid,
                recipients,
                &rendered_msg.message,
-                msg_id,
+                msg.id,
            ),
        )
        .await?;
@@ -2574,14 +2698,7 @@ pub(crate) async fn marknoticed_chat_if_older_than(
    chat_id: ChatId,
    timestamp: i64,
 ) -> Result<()> {
-    if let Some(chat_timestamp) = context
-        .sql
-        .query_get_value(
-            "SELECT MAX(timestamp) FROM msgs WHERE chat_id=?",
-            (chat_id,),
-        )
-        .await?
-    {
+    if let Some(chat_timestamp) = chat_id.get_timestamp(context).await? {
        if timestamp > chat_timestamp {
            marknoticed_chat(context, chat_id).await?;
        }
@@ -3371,89 +3488,86 @@ pub async fn forward_msgs(context: &Context, msg_ids: &[MsgId], chat_id: ChatId)
    chat_id
        .unarchive_if_not_muted(context, MessageState::Undefined)
        .await?;
-    if let Ok(mut chat) = Chat::load_from_db(context, chat_id).await {
-        if let Some(reason) = chat.why_cant_send(context).await? {
-            bail!("cannot send to {}: {}", chat_id, reason);
+    let mut chat = Chat::load_from_db(context, chat_id).await?;
+    if let Some(reason) = chat.why_cant_send(context).await? {
+        bail!("cannot send to {}: {}", chat_id, reason);
+    }
+    curr_timestamp = create_smeared_timestamps(context, msg_ids.len());
+    let ids = context
+        .sql
+        .query_map(
+            &format!(
+                "SELECT id FROM msgs WHERE id IN({}) ORDER BY timestamp,id",
+                sql::repeat_vars(msg_ids.len())
+            ),
+            rusqlite::params_from_iter(msg_ids),
+            |row| row.get::<_, MsgId>(0),
+            |ids| ids.collect::<Result<Vec<_>, _>>().map_err(Into::into),
+        )
+        .await?;
+
+    for id in ids {
+        let src_msg_id: MsgId = id;
+        let mut msg = Message::load_from_db(context, src_msg_id).await?;
+        if msg.state == MessageState::OutDraft {
+            bail!("cannot forward drafts.");
        }
-        curr_timestamp = create_smeared_timestamps(context, msg_ids.len());
-        let ids = context
-            .sql
-            .query_map(
-                &format!(
-                    "SELECT id FROM msgs WHERE id IN({}) ORDER BY timestamp,id",
-                    sql::repeat_vars(msg_ids.len())
-                ),
-                rusqlite::params_from_iter(msg_ids),
-                |row| row.get::<_, MsgId>(0),
-                |ids| ids.collect::<Result<Vec<_>, _>>().map_err(Into::into),
-            )
-            .await?;

-        for id in ids {
-            let src_msg_id: MsgId = id;
-            let mut msg = Message::load_from_db(context, src_msg_id).await?;
-            if msg.state == MessageState::OutDraft {
-                bail!("cannot forward drafts.");
-            }
+        let original_param = msg.param.clone();

-            let original_param = msg.param.clone();
+        // we tested a sort of broadcast
+        // by not marking own forwarded messages as such,
+        // however, this turned out to be to confusing and unclear.

-            // we tested a sort of broadcast
-            // by not marking own forwarded messages as such,
-            // however, this turned out to be to confusing and unclear.
+        if msg.get_viewtype() != Viewtype::Sticker {
+            msg.param
+                .set_int(Param::Forwarded, src_msg_id.to_u32() as i32);
+        }

-            if msg.get_viewtype() != Viewtype::Sticker {
-                msg.param
-                    .set_int(Param::Forwarded, src_msg_id.to_u32() as i32);
-            }
+        msg.param.remove(Param::GuaranteeE2ee);
+        msg.param.remove(Param::ForcePlaintext);
+        msg.param.remove(Param::Cmd);
+        msg.param.remove(Param::OverrideSenderDisplayname);
+        msg.param.remove(Param::WebxdcSummary);
+        msg.param.remove(Param::WebxdcSummaryTimestamp);
+        msg.in_reply_to = None;

-            msg.param.remove(Param::GuaranteeE2ee);
-            msg.param.remove(Param::ForcePlaintext);
-            msg.param.remove(Param::Cmd);
-            msg.param.remove(Param::OverrideSenderDisplayname);
-            msg.param.remove(Param::WebxdcSummary);
-            msg.param.remove(Param::WebxdcSummaryTimestamp);
-            msg.in_reply_to = None;
+        // do not leak data as group names; a default subject is generated by mimefactory
+        msg.subject = "".to_string();

-            // do not leak data as group names; a default subject is generated by mimefactory
-            msg.subject = "".to_string();
+        let new_msg_id: MsgId;
+        if msg.state == MessageState::OutPreparing {
+            new_msg_id = chat
+                .prepare_msg_raw(context, &mut msg, None, curr_timestamp)
+                .await?;
+            curr_timestamp += 1;
+            msg.param = original_param;
+            msg.id = src_msg_id;

-            let new_msg_id: MsgId;
-            if msg.state == MessageState::OutPreparing {
-                new_msg_id = chat
-                    .prepare_msg_raw(context, &mut msg, None, curr_timestamp)
-                    .await?;
-                curr_timestamp += 1;
-                let save_param = msg.param.clone();
-                msg.param = original_param;
-                msg.id = src_msg_id;
-
-                if let Some(old_fwd) = msg.param.get(Param::PrepForwards) {
-                    let new_fwd = format!("{} {}", old_fwd, new_msg_id.to_u32());
-                    msg.param.set(Param::PrepForwards, new_fwd);
-                } else {
-                    msg.param
-                        .set(Param::PrepForwards, new_msg_id.to_u32().to_string());
-                }
-
-                msg.update_param(context).await?;
-                msg.param = save_param;
+            if let Some(old_fwd) = msg.param.get(Param::PrepForwards) {
+                let new_fwd = format!("{} {}", old_fwd, new_msg_id.to_u32());
+                msg.param.set(Param::PrepForwards, new_fwd);
            } else {
-                msg.state = MessageState::OutPending;
-                new_msg_id = chat
-                    .prepare_msg_raw(context, &mut msg, None, curr_timestamp)
-                    .await?;
-                curr_timestamp += 1;
-                if create_send_msg_job(context, new_msg_id).await?.is_some() {
-                    context
-                        .scheduler
-                        .interrupt_smtp(InterruptInfo::new(false))
-                        .await;
-                }
+                msg.param
+                    .set(Param::PrepForwards, new_msg_id.to_u32().to_string());
+            }
+
+            msg.update_param(context).await?;
+        } else {
+            msg.state = MessageState::OutPending;
+            new_msg_id = chat
+                .prepare_msg_raw(context, &mut msg, None, curr_timestamp)
+                .await?;
+            curr_timestamp += 1;
+            if create_send_msg_job(context, &mut msg).await?.is_some() {
+                context
+                    .scheduler
+                    .interrupt_smtp(InterruptInfo::new(false))
+                    .await;
            }
-            created_chats.push(chat_id);
-            created_msgs.push(new_msg_id);
        }
+        created_chats.push(chat_id);
+        created_msgs.push(new_msg_id);
    }
    for (chat_id, msg_id) in created_chats.iter().zip(created_msgs.iter()) {
        context.emit_msgs_changed(*chat_id, *msg_id);
@@ -3485,29 +3599,31 @@ pub async fn resend_msgs(context: &Context, msg_ids: &[MsgId]) -> Result<()> {
        msgs.push(msg)
    }

-    if let Some(chat_id) = chat_id {
-        let chat = Chat::load_from_db(context, chat_id).await?;
-        for mut msg in msgs {
-            if msg.get_showpadlock() && !chat.is_protected() {
-                msg.param.remove(Param::GuaranteeE2ee);
-                msg.update_param(context).await?;
-            }
-            match msg.get_state() {
-                MessageState::OutFailed | MessageState::OutDelivered | MessageState::OutMdnRcvd => {
-                    message::update_msg_state(context, msg.id, MessageState::OutPending).await?
-                }
-                _ => bail!("unexpected message state"),
-            }
-            context.emit_event(EventType::MsgsChanged {
-                chat_id: msg.chat_id,
-                msg_id: msg.id,
-            });
-            if create_send_msg_job(context, msg.id).await?.is_some() {
-                context
-                    .scheduler
-                    .interrupt_smtp(InterruptInfo::new(false))
-                    .await;
+    let Some(chat_id) = chat_id else {
+        return Ok(());
+    };
+
+    let chat = Chat::load_from_db(context, chat_id).await?;
+    for mut msg in msgs {
+        if msg.get_showpadlock() && !chat.is_protected() {
+            msg.param.remove(Param::GuaranteeE2ee);
+            msg.update_param(context).await?;
+        }
+        match msg.get_state() {
+            MessageState::OutFailed | MessageState::OutDelivered | MessageState::OutMdnRcvd => {
+                message::update_msg_state(context, msg.id, MessageState::OutPending).await?
            }
+            _ => bail!("unexpected message state"),
+        }
+        context.emit_event(EventType::MsgsChanged {
+            chat_id: msg.chat_id,
+            msg_id: msg.id,
+        });
+        if create_send_msg_job(context, &mut msg).await?.is_some() {
+            context
+                .scheduler
+                .interrupt_smtp(InterruptInfo::new(false))
+                .await;
        }
    }
    Ok(())
@@ -3577,7 +3693,6 @@ pub async fn add_device_msg_with_importance(
        chat_id = ChatId::get_for_contact(context, ContactId::DEVICE).await?;

        let rfc724_mid = create_outgoing_rfc724_mid(None, "@device");
-        msg.try_calc_and_set_dimensions(context).await.ok();
        prepare_msg_blob(context, msg).await?;

        let timestamp_sent = create_smeared_timestamp(context);
@@ -5510,7 +5625,13 @@ mod tests {
        Ok(())
    }

-    async fn test_sticker(filename: &str, bytes: &[u8], w: i32, h: i32) -> Result<()> {
+    async fn test_sticker(
+        filename: &str,
+        bytes: &[u8],
+        res_viewtype: Viewtype,
+        w: i32,
+        h: i32,
+    ) -> Result<()> {
        let alice = TestContext::new_alice().await;
        let bob = TestContext::new_bob().await;
        let alice_chat = alice.create_chat(&bob).await;
@@ -5524,12 +5645,19 @@ mod tests {

        let sent_msg = alice.send_msg(alice_chat.id, &mut msg).await;
        let mime = sent_msg.payload();
-        assert_eq!(mime.match_indices("Chat-Content: sticker").count(), 1);
+        if res_viewtype == Viewtype::Sticker {
+            assert_eq!(mime.match_indices("Chat-Content: sticker").count(), 1);
+        }

        let msg = bob.recv_msg(&sent_msg).await;
        assert_eq!(msg.chat_id, bob_chat.id);
-        assert_eq!(msg.get_viewtype(), Viewtype::Sticker);
-        assert_eq!(msg.get_filename().unwrap(), filename);
+        assert_eq!(msg.get_viewtype(), res_viewtype);
+        let msg_filename = msg.get_filename().unwrap();
+        match res_viewtype {
+            Viewtype::Sticker => assert_eq!(msg_filename, filename),
+            Viewtype::Image => assert!(msg_filename.starts_with("image_")),
+            _ => panic!("Not implemented"),
+        }
        assert_eq!(msg.get_width(), w);
        assert_eq!(msg.get_height(), h);
        assert!(msg.get_filebytes(&bob).await?.unwrap() > 250);
@@ -5541,9 +5669,10 @@ mod tests {
    async fn test_sticker_png() -> Result<()> {
        test_sticker(
            "sticker.png",
-            include_bytes!("../test-data/image/avatar64x64.png"),
-            64,
-            64,
+            include_bytes!("../test-data/image/logo.png"),
+            Viewtype::Sticker,
+            135,
+            135,
        )
        .await
    }
@@ -5553,6 +5682,7 @@ mod tests {
        test_sticker(
            "sticker.jpg",
            include_bytes!("../test-data/image/avatar1000x1000.jpg"),
+            Viewtype::Image,
            1000,
            1000,
        )
@@ -5563,9 +5693,10 @@ mod tests {
    async fn test_sticker_gif() -> Result<()> {
        test_sticker(
            "sticker.gif",
-            include_bytes!("../test-data/image/image100x50.gif"),
-            100,
-            50,
+            include_bytes!("../test-data/image/logo.gif"),
+            Viewtype::Sticker,
+            135,
+            135,
        )
        .await
    }
@@ -5579,8 +5710,8 @@ mod tests {
        let bob_chat = bob.create_chat(&alice).await;

        // create sticker
-        let file_name = "sticker.jpg";
-        let bytes = include_bytes!("../test-data/image/avatar1000x1000.jpg");
+        let file_name = "sticker.png";
+        let bytes = include_bytes!("../test-data/image/logo.png");
        let file = alice.get_blobdir().join(file_name);
        tokio::fs::write(&file, bytes).await?;
        let mut msg = Message::new(Viewtype::Sticker);
@@ -6117,7 +6248,7 @@ mod tests {
            chat_id1,
            Viewtype::Sticker,
            "b.png",
-            include_bytes!("../test-data/image/avatar64x64.png"),
+            include_bytes!("../test-data/image/logo.png"),
        )
        .await?;
        let second_image_msg_id = send_media(
--- a/src/chatlist.rs
+++ b/src/chatlist.rs
@@ -10,8 +10,10 @@ use crate::constants::{
 use crate::contact::{Contact, ContactId};
 use crate::context::Context;
 use crate::message::{Message, MessageState, MsgId};
+use crate::param::{Param, Params};
 use crate::stock_str;
 use crate::summary::Summary;
+use crate::tools::IsNoneOrEmpty;

 /// An object representing a single chatlist in memory.
 ///
@@ -204,34 +206,84 @@ impl Chatlist {
                )
                .await?
        } else {
-            //  show normal chatlist
-            let sort_id_up = if flag_for_forwarding {
-                ChatId::lookup_by_contact(context, ContactId::SELF)
+            let mut ids = if flag_for_forwarding {
+                let sort_id_up = ChatId::lookup_by_contact(context, ContactId::SELF)
                    .await?
-                    .unwrap_or_default()
+                    .unwrap_or_default();
+                let process_row = |row: &rusqlite::Row| {
+                    let chat_id: ChatId = row.get(0)?;
+                    let typ: Chattype = row.get(1)?;
+                    let param: Params = row.get::<_, String>(2)?.parse().unwrap_or_default();
+                    let msg_id: Option<MsgId> = row.get(3)?;
+                    Ok((chat_id, typ, param, msg_id))
+                };
+                let process_rows = |rows: rusqlite::MappedRows<_>| {
+                    rows.filter_map(|row: std::result::Result<(_, _, Params, _), _>| match row {
+                        Ok((chat_id, typ, param, msg_id)) => {
+                            if typ == Chattype::Mailinglist
+                                && param.get(Param::ListPost).is_none_or_empty()
+                            {
+                                None
+                            } else {
+                                Some(Ok((chat_id, msg_id)))
+                            }
+                        }
+                        Err(e) => Some(Err(e)),
+                    })
+                    .collect::<std::result::Result<Vec<_>, _>>()
+                    .map_err(Into::into)
+                };
+                // Return ProtectionBroken chats also, as that may happen to a verified chat at any
+                // time. It may be confusing if a chat that is normally in the list disappears
+                // suddenly. The UI need to deal with that case anyway.
+                context.sql.query_map(
+                    "SELECT c.id, c.type, c.param, m.id
+                     FROM chats c
+                     LEFT JOIN msgs m
+                            ON c.id=m.chat_id
+                           AND m.id=(
+                                   SELECT id
+                                     FROM msgs
+                                    WHERE chat_id=c.id
+                                      AND (hidden=0 OR state=?)
+                                      ORDER BY timestamp DESC, id DESC LIMIT 1)
+                     WHERE c.id>9 AND c.id!=?
+                       AND c.blocked=0
+                       AND NOT c.archived=?
+                       AND (c.type!=? OR c.id IN(SELECT chat_id FROM chats_contacts WHERE contact_id=?))
+                     GROUP BY c.id
+                     ORDER BY c.id=? DESC, c.archived=? DESC, IFNULL(m.timestamp,c.created_timestamp) DESC, m.id DESC;",
+                    (
+                        MessageState::OutDraft, skip_id, ChatVisibility::Archived,
+                        Chattype::Group, ContactId::SELF,
+                        sort_id_up, ChatVisibility::Pinned,
+                    ),
+                    process_row,
+                    process_rows,
+                ).await?
            } else {
-                ChatId::new(0)
+                //  show normal chatlist
+                context.sql.query_map(
+                    "SELECT c.id, m.id
+                     FROM chats c
+                     LEFT JOIN msgs m
+                            ON c.id=m.chat_id
+                           AND m.id=(
+                                   SELECT id
+                                     FROM msgs
+                                    WHERE chat_id=c.id
+                                      AND (hidden=0 OR state=?)
+                                      ORDER BY timestamp DESC, id DESC LIMIT 1)
+                     WHERE c.id>9 AND c.id!=?
+                       AND (c.blocked=0 OR c.blocked=2)
+                       AND NOT c.archived=?
+                     GROUP BY c.id
+                     ORDER BY c.id=0 DESC, c.archived=? DESC, IFNULL(m.timestamp,c.created_timestamp) DESC, m.id DESC;",
+                    (MessageState::OutDraft, skip_id, ChatVisibility::Archived, ChatVisibility::Pinned),
+                    process_row,
+                    process_rows,
+                ).await?
            };
-            let mut ids = context.sql.query_map(
-                "SELECT c.id, m.id
-                 FROM chats c
-                 LEFT JOIN msgs m
-                        ON c.id=m.chat_id
-                       AND m.id=(
-                               SELECT id
-                                 FROM msgs
-                                WHERE chat_id=c.id
-                                  AND (hidden=0 OR state=?1)
-                                  ORDER BY timestamp DESC, id DESC LIMIT 1)
-                 WHERE c.id>9 AND c.id!=?2
-                   AND (c.blocked=0 OR (c.blocked=2 AND NOT ?3))
-                   AND NOT c.archived=?4
-                 GROUP BY c.id
-                 ORDER BY c.id=?5 DESC, c.archived=?6 DESC, IFNULL(m.timestamp,c.created_timestamp) DESC, m.id DESC;",
-                (MessageState::OutDraft, skip_id, flag_for_forwarding, ChatVisibility::Archived, sort_id_up, ChatVisibility::Pinned),
-                process_row,
-                process_rows,
-            ).await?;
            if !flag_no_specials && get_archived_cnt(context).await? > 0 {
                if ids.is_empty() && flag_add_alldone_hint {
                    ids.push((DC_CHAT_ID_ALLDONE_HINT, None));
@@ -244,6 +296,27 @@ impl Chatlist {
        Ok(Chatlist { ids })
    }

+    /// Converts list of chat IDs to a chatlist.
+    pub(crate) async fn from_chat_ids(context: &Context, chat_ids: &[ChatId]) -> Result<Self> {
+        let mut ids = Vec::new();
+        for &chat_id in chat_ids {
+            let msg_id: Option<MsgId> = context
+                .sql
+                .query_get_value(
+                    "SELECT id
+                   FROM msgs
+                  WHERE chat_id=?1
+                    AND (hidden=0 OR state=?2)
+                  ORDER BY timestamp DESC, id DESC LIMIT 1",
+                    (chat_id, MessageState::OutDraft),
+                )
+                .await
+                .with_context(|| format!("failed to get msg ID for chat {}", chat_id))?;
+            ids.push((chat_id, msg_id));
+        }
+        Ok(Chatlist { ids })
+    }
+
    /// Find out the number of chats.
    pub fn len(&self) -> usize {
        self.ids.len()
@@ -388,7 +461,9 @@ pub async fn get_last_message_for_chat(
 #[cfg(test)]
 mod tests {
    use super::*;
-    use crate::chat::{create_group_chat, get_chat_contacts, ProtectionStatus};
+    use crate::chat::{
+        create_group_chat, get_chat_contacts, remove_contact_from_chat, ProtectionStatus,
+    };
    use crate::message::Viewtype;
    use crate::receive_imf::receive_imf;
    use crate::stock_str::StockMessage;
@@ -473,6 +548,14 @@ mod tests {
            .await
            .unwrap()
            .is_self_talk());
+
+        remove_contact_from_chat(&t, chats.get_chat_id(1).unwrap(), ContactId::SELF)
+            .await
+            .unwrap();
+        let chats = Chatlist::try_load(&t, DC_GCL_FOR_FORWARDING, None, None)
+            .await
+            .unwrap();
+        assert!(chats.len() == 1);
    }

    #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
--- a/src/config.rs
+++ b/src/config.rs
@@ -1,6 +1,7 @@
 //! # Key-value configuration management.

 use std::env;
+use std::path::Path;
 use std::str::FromStr;

 use anyhow::{ensure, Context as _, Result};
@@ -329,7 +330,11 @@ impl Context {
        let value = match key {
            Config::Selfavatar => {
                let rel_path = self.sql.get_raw_config(key.as_ref()).await?;
-                rel_path.map(|p| get_abs_path(self, p).to_string_lossy().into_owned())
+                rel_path.map(|p| {
+                    get_abs_path(self, Path::new(&p))
+                        .to_string_lossy()
+                        .into_owned()
+                })
            }
            Config::SysVersion => Some((*DC_VERSION_STR).clone()),
            Config::SysMsgsizeMaxRecommended => Some(format!("{RECOMMENDED_FILE_SIZE}")),
--- a/src/configure.rs
+++ b/src/configure.rs
@@ -130,8 +130,8 @@ async fn on_configure_completed(
    old_addr: Option<String>,
 ) -> Result<()> {
    if let Some(provider) = param.provider {
-        if let Some(config_defaults) = &provider.config_defaults {
-            for def in config_defaults.iter() {
+        if let Some(config_defaults) = provider.config_defaults {
+            for def in config_defaults {
                if !context.config_exists(def.key).await? {
                    info!(context, "apply config_defaults {}={}", def.key, def.value);
                    context.set_config(def.key, Some(def.value)).await?;
@@ -656,7 +656,7 @@ async fn try_smtp_one_param(
        })
    } else {
        info!(context, "success: {}", inf);
-        smtp.disconnect().await;
+        smtp.disconnect();
        Ok(())
    }
 }
--- a/src/configure/auto_mozilla.rs
+++ b/src/configure/auto_mozilla.rs
@@ -234,7 +234,7 @@ fn parse_serverparams(in_emailaddr: &str, xml_raw: &str) -> Result<Vec<ServerPar
    let res = moz_ac
        .incoming_servers
        .into_iter()
-        .chain(moz_ac.outgoing_servers.into_iter())
+        .chain(moz_ac.outgoing_servers)
        .filter_map(|server| {
            let protocol = match server.typ.as_ref() {
                "imap" => Some(Protocol::Imap),
--- a/src/constants.rs
+++ b/src/constants.rs
@@ -62,8 +62,15 @@ pub enum MediaQuality {
 pub enum KeyGenType {
    #[default]
    Default = 0,
+
+    /// 2048-bit RSA.
    Rsa2048 = 1,
+
+    /// [Ed25519](https://ed25519.cr.yp.to/) signature and X25519 encryption.
    Ed25519 = 2,
+
+    /// 4096-bit RSA.
+    Rsa4096 = 3,
 }

 /// Video chat URL type.
@@ -231,6 +238,7 @@ mod tests {
        assert_eq!(KeyGenType::Default, KeyGenType::from_i32(0).unwrap());
        assert_eq!(KeyGenType::Rsa2048, KeyGenType::from_i32(1).unwrap());
        assert_eq!(KeyGenType::Ed25519, KeyGenType::from_i32(2).unwrap());
+        assert_eq!(KeyGenType::Rsa4096, KeyGenType::from_i32(3).unwrap());
    }

    #[test]
--- a/src/contact.rs
+++ b/src/contact.rs
@@ -5,7 +5,7 @@ use std::collections::BinaryHeap;
 use std::convert::{TryFrom, TryInto};
 use std::fmt;
 use std::ops::Deref;
-use std::path::PathBuf;
+use std::path::{Path, PathBuf};
 use std::time::{SystemTime, UNIX_EPOCH};

 use anyhow::{bail, ensure, Context as _, Result};
@@ -1186,7 +1186,7 @@ impl Contact {
            }
        } else if let Some(image_rel) = self.param.get(Param::ProfileImage) {
            if !image_rel.is_empty() {
-                return Ok(Some(get_abs_path(context, image_rel)));
+                return Ok(Some(get_abs_path(context, Path::new(image_rel))));
            }
        }
        Ok(None)
@@ -1732,7 +1732,7 @@ mod tests {
        assert_eq!(may_be_valid_addr("dd.tt"), false);
        assert_eq!(may_be_valid_addr("tt.dd@uu"), true);
        assert_eq!(may_be_valid_addr("u@d"), true);
-        assert_eq!(may_be_valid_addr("u@d."), true);
+        assert_eq!(may_be_valid_addr("u@d."), false);
        assert_eq!(may_be_valid_addr("u@d.t"), true);
        assert_eq!(may_be_valid_addr("u@d.tt"), true);
        assert_eq!(may_be_valid_addr("u@.tt"), true);
@@ -1741,6 +1741,7 @@ mod tests {
        assert_eq!(may_be_valid_addr("sk <@d.tt>"), false);
        assert_eq!(may_be_valid_addr("as@sd.de>"), false);
        assert_eq!(may_be_valid_addr("ask dkl@dd.tt"), false);
+        assert_eq!(may_be_valid_addr("user@domain.tld."), false);
    }

    #[test]
--- a/src/context.rs
+++ b/src/context.rs
@@ -332,6 +332,12 @@ impl Context {
        }
    }

+    /// Changes encrypted database passphrase.
+    pub async fn change_passphrase(&self, passphrase: String) -> Result<()> {
+        self.sql.change_passphrase(passphrase).await?;
+        Ok(())
+    }
+
    /// Returns true if database is open.
    pub async fn is_open(&self) -> bool {
        self.sql.is_open().await
@@ -1453,6 +1459,35 @@ mod tests {
        Ok(())
    }

+    #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
+    async fn test_context_change_passphrase() -> Result<()> {
+        let dir = tempdir()?;
+        let dbfile = dir.path().join("db.sqlite");
+
+        let id = 1;
+        let context = Context::new_closed(&dbfile, id, Events::new(), StockStrings::new())
+            .await
+            .context("failed to create context")?;
+        assert_eq!(context.open("foo".to_string()).await?, true);
+        assert_eq!(context.is_open().await, true);
+
+        context
+            .set_config(Config::Addr, Some("alice@example.org"))
+            .await?;
+
+        context
+            .change_passphrase("bar".to_string())
+            .await
+            .context("Failed to change passphrase")?;
+
+        assert_eq!(
+            context.get_config(Config::Addr).await?.unwrap(),
+            "alice@example.org"
+        );
+
+        Ok(())
+    }
+
    #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
    async fn test_ongoing() -> Result<()> {
        let context = TestContext::new().await;
--- a/src/decrypt.rs
+++ b/src/decrypt.rs
@@ -29,14 +29,30 @@ pub fn try_decrypt(
    private_keyring: &Keyring<SignedSecretKey>,
    public_keyring_for_validate: &Keyring<SignedPublicKey>,
 ) -> Result<Option<(Vec<u8>, HashSet<Fingerprint>)>> {
-    let encrypted_data_part = match get_autocrypt_mime(mail)
-        .or_else(|| get_mixed_up_mime(mail))
-        .or_else(|| get_attachment_mime(mail))
-    {
+    let encrypted_data_part = match {
+        let mime = get_autocrypt_mime(mail);
+        if mime.is_some() {
+            info!(context, "Detected Autocrypt-mime message.");
+        }
+        mime
+    }
+    .or_else(|| {
+        let mime = get_mixed_up_mime(mail);
+        if mime.is_some() {
+            info!(context, "Detected mixed-up mime message.");
+        }
+        mime
+    })
+    .or_else(|| {
+        let mime = get_attachment_mime(mail);
+        if mime.is_some() {
+            info!(context, "Detected attached Autocrypt-mime message.");
+        }
+        mime
+    }) {
        None => return Ok(None),
        Some(res) => res,
    };
-    info!(context, "Detected Autocrypt-mime message");

    decrypt_part(
        encrypted_data_part,
@@ -403,4 +419,18 @@ mod tests {

        Ok(())
    }
+
+    #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
+    async fn test_mixed_up_mime_long() -> Result<()> {
+        // Long "mixed-up" mail as received when sending an encrypted message using Delta Chat
+        // Desktop via MS Exchange (actually made with TB though).
+        let mixed_up_mime = include_bytes!("../test-data/message/mixed-up-long.eml");
+        let bob = TestContext::new_bob().await;
+        receive_imf(&bob, mixed_up_mime, false).await?;
+        let msg = bob.get_last_msg().await;
+        assert!(!msg.get_text().is_empty());
+        assert!(msg.has_html());
+        assert!(msg.id.get_html(&bob).await?.unwrap().len() > 40000);
+        Ok(())
+    }
 }
--- a/src/download.rs
+++ b/src/download.rs
@@ -59,6 +59,9 @@ pub enum DownloadState {
    /// Failed to fully download the message.
    Failure = 20,

+    /// Undecipherable message.
+    Undecipherable = 30,
+
    /// Full download of the message is in progress.
    InProgress = 1000,
 }
@@ -80,7 +83,9 @@ impl MsgId {
    pub async fn download_full(self, context: &Context) -> Result<()> {
        let msg = Message::load_from_db(context, self).await?;
        match msg.download_state() {
-            DownloadState::Done => return Err(anyhow!("Nothing to download.")),
+            DownloadState::Done | DownloadState::Undecipherable => {
+                return Err(anyhow!("Nothing to download."))
+            }
            DownloadState::InProgress => return Err(anyhow!("Download already in progress.")),
            DownloadState::Available | DownloadState::Failure => {
                self.update_download_state(context, DownloadState::InProgress)
--- a/src/ephemeral.rs
+++ b/src/ephemeral.rs
@@ -1062,14 +1062,14 @@ mod tests {
        delete_expired_messages(t, not_deleted_at).await?;

        let loaded = Message::load_from_db(t, msg_id).await?;
-        assert_eq!(loaded.text, "Message text");
+        assert!(!loaded.text.is_empty());
        assert_eq!(loaded.chat_id, chat.id);

        assert!(next_expiration < deleted_at);
        delete_expired_messages(t, deleted_at).await?;
        t.evtracker
            .get_matching(|evt| {
-                if let EventType::MsgsChanged {
+                if let EventType::MsgDeleted {
                    msg_id: event_msg_id,
                    ..
                } = evt
@@ -1082,7 +1082,6 @@ mod tests {
            .await;

        let loaded = Message::load_from_db(t, msg_id).await?;
-        assert_eq!(loaded.text, "");
        assert_eq!(loaded.chat_id, DC_CHAT_ID_TRASH);

        // Check that the msg was deleted locally.
@@ -1300,4 +1299,32 @@ mod tests {

        Ok(())
    }
+
+    // Tests that if we are offline for a time longer than the ephemeral timer duration, the message
+    // is deleted from the chat but is still in the "smtp" table, i.e. will be sent upon a
+    // successful reconnection.
+    #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
+    async fn test_ephemeral_msg_offline() -> Result<()> {
+        let alice = TestContext::new_alice().await;
+        let chat = alice
+            .create_chat_with_contact("Bob", "bob@example.org")
+            .await;
+        let duration = 60;
+        chat.id
+            .set_ephemeral_timer(&alice, Timer::Enabled { duration })
+            .await?;
+        let mut msg = Message::new(Viewtype::Text);
+        msg.set_text("hi".to_string());
+        assert!(chat::send_msg_sync(&alice, chat.id, &mut msg)
+            .await
+            .is_err());
+        let stmt = "SELECT COUNT(*) FROM smtp WHERE msg_id=?";
+        assert!(alice.sql.exists(stmt, (msg.id,)).await?);
+        let now = time();
+        check_msg_will_be_deleted(&alice, msg.id, &chat, now, now + i64::from(duration) + 1)
+            .await?;
+        assert!(alice.sql.exists(stmt, (msg.id,)).await?);
+
+        Ok(())
+    }
 }
--- a/src/html.rs
+++ b/src/html.rs
@@ -291,7 +291,7 @@ mod tests {
        let parser = HtmlMsgParser::from_bytes(&t.ctx, raw).await.unwrap();
        assert_eq!(
            parser.html,
-            r##"<!DOCTYPE html>
+            r#"<!DOCTYPE html>
 <html><head>
 <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
 <meta name="color-scheme" content="light dark" />
@@ -299,7 +299,7 @@ mod tests {
 This message does not have Content-Type nor Subject.<br/>
 <br/>
 </body></html>
-"##
+"#
        );
    }

@@ -310,7 +310,7 @@ This message does not have Content-Type nor Subject.<br/>
        let parser = HtmlMsgParser::from_bytes(&t.ctx, raw).await.unwrap();
        assert_eq!(
            parser.html,
-            r##"<!DOCTYPE html>
+            r#"<!DOCTYPE html>
 <html><head>
 <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
 <meta name="color-scheme" content="light dark" />
@@ -318,7 +318,7 @@ This message does not have Content-Type nor Subject.<br/>
 message with a non-UTF-8 encoding: äöüßÄÖÜ<br/>
 <br/>
 </body></html>
-"##
+"#
        );
    }

@@ -330,7 +330,7 @@ message with a non-UTF-8 encoding: äöüßÄÖÜ<br/>
        assert!(parser.plain.unwrap().flowed);
        assert_eq!(
            parser.html,
-            r##"<!DOCTYPE html>
+            r#"<!DOCTYPE html>
 <html><head>
 <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
 <meta name="color-scheme" content="light dark" />
@@ -341,7 +341,7 @@ This line does not end with a space<br/>
 and will be wrapped as usual.<br/>
 <br/>
 </body></html>
-"##
+"#
        );
    }

@@ -352,7 +352,7 @@ and will be wrapped as usual.<br/>
        let parser = HtmlMsgParser::from_bytes(&t.ctx, raw).await.unwrap();
        assert_eq!(
            parser.html,
-            r##"<!DOCTYPE html>
+            r#"<!DOCTYPE html>
 <html><head>
 <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
 <meta name="color-scheme" content="light dark" />
@@ -363,7 +363,7 @@ test some special html-characters as &lt; &gt; and &amp; but also &quot; and &#x
 <br/>
 <br/>
 </body></html>
-"##
+"#
        );
    }

--- a/src/imex.rs
+++ b/src/imex.rs
@@ -586,63 +586,74 @@ async fn export_backup_inner(
    Ok(())
 }

-/*******************************************************************************
- * Classic key import
- ******************************************************************************/
-async fn import_self_keys(context: &Context, dir: &Path) -> Result<()> {
-    /* hint: even if we switch to import Autocrypt Setup Files, we should leave the possibility to import
-    plain ASC keys, at least keys without a password, if we do not want to implement a password entry function.
-    Importing ASC keys is useful to use keys in Delta Chat used by any other non-Autocrypt-PGP implementation.
+/// Imports secret key from a file.
+async fn import_secret_key(context: &Context, path: &Path, set_default: bool) -> Result<()> {
+    let buf = read_file(context, &path).await?;
+    let armored = std::string::String::from_utf8_lossy(&buf);
+    set_self_key(context, &armored, set_default, false).await?;
+    Ok(())
+}
+
+/// Imports secret keys from the provided file or directory.
+///
+/// If provided path is a file, ASCII-armored secret key is read from the file
+/// and set as the default key.
+///
+/// If provided path is a directory, all files with .asc extension
+/// containing secret keys are imported and the last successfully
+/// imported which does not contain "legacy" in its filename
+/// is set as the default.
+async fn import_self_keys(context: &Context, path: &Path) -> Result<()> {
+    let attr = tokio::fs::metadata(path).await?;
+
+    if attr.is_file() {
+        info!(
+            context,
+            "Importing secret key from {} as the default key.",
+            path.display()
+        );
+        let set_default = true;
+        import_secret_key(context, path, set_default).await?;
+        return Ok(());
+    }

-    Maybe we should make the "default" key handlong also a little bit smarter
-    (currently, the last imported key is the standard key unless it contains the string "legacy" in its name) */
-    let mut set_default: bool;
    let mut imported_cnt = 0;

-    let dir_name = dir.to_string_lossy();
-    let mut dir_handle = tokio::fs::read_dir(&dir).await?;
+    let mut dir_handle = tokio::fs::read_dir(&path).await?;
    while let Ok(Some(entry)) = dir_handle.next_entry().await {
        let entry_fn = entry.file_name();
        let name_f = entry_fn.to_string_lossy();
-        let path_plus_name = dir.join(&entry_fn);
-        match get_filesuffix_lc(&name_f) {
-            Some(suffix) => {
-                if suffix != "asc" {
-                    continue;
-                }
-                set_default = if name_f.contains("legacy") {
-                    info!(context, "found legacy key '{}'", path_plus_name.display());
-                    false
-                } else {
-                    true
-                }
-            }
-            None => {
+        let path_plus_name = path.join(&entry_fn);
+        if let Some(suffix) = get_filesuffix_lc(&name_f) {
+            if suffix != "asc" {
                continue;
            }
-        }
+        } else {
+            continue;
+        };
+        let set_default = !name_f.contains("legacy");
        info!(
            context,
-            "considering key file: {}",
+            "Considering key file: {}.",
            path_plus_name.display()
        );

-        match read_file(context, &path_plus_name).await {
-            Ok(buf) => {
-                let armored = std::string::String::from_utf8_lossy(&buf);
-                if let Err(err) = set_self_key(context, &armored, set_default, false).await {
-                    info!(context, "set_self_key: {}", err);
-                    continue;
-                }
-            }
-            Err(_) => continue,
+        if let Err(err) = import_secret_key(context, &path_plus_name, set_default).await {
+            warn!(
+                context,
+                "Failed to import secret key from {}: {:#}.",
+                path_plus_name.display(),
+                err
+            );
+            continue;
        }
+
        imported_cnt += 1;
    }
    ensure!(
        imported_cnt > 0,
-        "No private keys found in \"{}\".",
-        dir_name
+        "No private keys found in {}.",
+        path.display()
    );
    Ok(())
 }
@@ -673,7 +684,8 @@ async fn export_self_keys(context: &Context, dir: &Path) -> Result<()> {
        .await?;

    for (id, public_key, private_key, is_default) in keys {
-        let id = Some(id).filter(|_| is_default != 0);
+        let id = Some(id).filter(|_| is_default == 0);
+
        if let Ok(key) = public_key {
            if let Err(err) = export_key_to_asc_file(context, dir, id, &key).await {
                error!(context, "Failed to export public key: {:#}.", err);
@@ -864,14 +876,35 @@ mod tests {

    #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
    async fn test_export_and_import_key() {
+        let export_dir = tempfile::tempdir().unwrap();
+
        let context = TestContext::new_alice().await;
-        let blobdir = context.ctx.get_blobdir();
-        if let Err(err) = imex(&context.ctx, ImexMode::ExportSelfKeys, blobdir, None).await {
+        if let Err(err) = imex(
+            &context.ctx,
+            ImexMode::ExportSelfKeys,
+            export_dir.path(),
+            None,
+        )
+        .await
+        {
            panic!("got error on export: {err:#}");
        }

        let context2 = TestContext::new_alice().await;
-        if let Err(err) = imex(&context2.ctx, ImexMode::ImportSelfKeys, blobdir, None).await {
+        if let Err(err) = imex(
+            &context2.ctx,
+            ImexMode::ImportSelfKeys,
+            export_dir.path(),
+            None,
+        )
+        .await
+        {
+            panic!("got error on import: {err:#}");
+        }
+
+        let keyfile = export_dir.path().join("private-key-default.asc");
+        let context3 = TestContext::new_alice().await;
+        if let Err(err) = imex(&context3.ctx, ImexMode::ImportSelfKeys, &keyfile, None).await {
            panic!("got error on import: {err:#}");
        }
    }
--- a/src/job.rs
+++ b/src/job.rs
@@ -155,8 +155,8 @@ impl<'a> Connection<'a> {
 pub(crate) async fn perform_job(context: &Context, mut connection: Connection<'_>, mut job: Job) {
    info!(context, "Job {} started...", &job);

-    let try_res = match perform_job_action(context, &mut job, &mut connection, 0).await {
-        Status::RetryNow => perform_job_action(context, &mut job, &mut connection, 1).await,
+    let try_res = match perform_job_action(context, &job, &mut connection, 0).await {
+        Status::RetryNow => perform_job_action(context, &job, &mut connection, 1).await,
        x => x,
    };

@@ -205,7 +205,7 @@ pub(crate) async fn perform_job(context: &Context, mut connection: Connection<'_

 async fn perform_job_action(
    context: &Context,
-    job: &mut Job,
+    job: &Job,
    connection: &mut Connection<'_>,
    tries: u32,
 ) -> Status {
--- a/src/key.rs
+++ b/src/key.rs
@@ -260,7 +260,7 @@ pub(crate) async fn load_keypair(
    })
 }

-/// Use of a [KeyPair] for encryption or decryption.
+/// Use of a key pair for encryption or decryption.
 ///
 /// This is used by [store_self_keypair] to know what kind of key is
 /// being saved.
--- a/src/keyring.rs
+++ b/src/keyring.rs
@@ -44,10 +44,6 @@ where
        self.keys.push(key);
    }

-    pub fn len(&self) -> usize {
-        self.keys.len()
-    }
-
    pub fn is_empty(&self) -> bool {
        self.keys.is_empty()
    }
--- a/src/lib.rs
+++ b/src/lib.rs
@@ -79,7 +79,7 @@ pub mod mimeparser;
 pub mod oauth2;
 mod param;
 pub mod peerstate;
-pub mod pgp;
+mod pgp;
 pub mod provider;
 pub mod qr;
 pub mod qr_code_generator;
--- a/src/message.rs
+++ b/src/message.rs
@@ -7,6 +7,7 @@ use anyhow::{ensure, format_err, Context as _, Result};
 use deltachat_derive::{FromSql, ToSql};
 use serde::{Deserialize, Serialize};

+use crate::blob::BlobObject;
 use crate::chat::{Chat, ChatId};
 use crate::config::Config;
 use crate::constants::{
@@ -582,14 +583,22 @@ impl Message {
                if (self.viewtype == Viewtype::Image || self.viewtype == Viewtype::Gif)
                    && !self.param.exists(Param::Width)
                {
-                    self.param.set_int(Param::Width, 0);
-                    self.param.set_int(Param::Height, 0);
+                    let buf = read_file(context, &path_and_filename).await?;

-                    if let Ok(buf) = read_file(context, path_and_filename).await {
-                        if let Ok((width, height)) = get_filemeta(&buf) {
+                    match get_filemeta(&buf) {
+                        Ok((width, height)) => {
                            self.param.set_int(Param::Width, width as i32);
                            self.param.set_int(Param::Height, height as i32);
                        }
+                        Err(err) => {
+                            self.param.set_int(Param::Width, 0);
+                            self.param.set_int(Param::Height, 0);
+                            warn!(
+                                context,
+                                "Failed to get width and height for {}: {err:#}.",
+                                path_and_filename.display()
+                            );
+                        }
                    }

                    if !self.id.is_unset() {
@@ -980,19 +989,28 @@ impl Message {
            }
        }
        self.param.set(Param::File, file);
-        if let Some(filemime) = filemime {
-            self.param.set(Param::MimeType, filemime);
-        }
+        self.param.set_optional(Param::MimeType, filemime);
+    }
+
+    /// Creates a new blob and sets it as a file associated with a message.
+    pub async fn set_file_from_bytes(
+        &mut self,
+        context: &Context,
+        suggested_name: &str,
+        data: &[u8],
+        filemime: Option<&str>,
+    ) -> Result<()> {
+        let blob = BlobObject::create(context, suggested_name, data).await?;
+        self.param.set(Param::File, blob.as_name());
+        self.param.set_optional(Param::MimeType, filemime);
+        Ok(())
    }

    /// Set different sender name for a message.
    /// This overrides the name set by the `set_config()`-option `displayname`.
    pub fn set_override_sender_name(&mut self, name: Option<String>) {
-        if let Some(name) = name {
-            self.param.set(Param::OverrideSenderDisplayname, name);
-        } else {
-            self.param.remove(Param::OverrideSenderDisplayname);
-        }
+        self.param
+            .set_optional(Param::OverrideSenderDisplayname, name);
    }

    /// Sets the dimensions of associated image or video file.
@@ -1429,6 +1447,7 @@ pub async fn get_mime_headers(context: &Context, msg_id: MsgId) -> Result<Vec<u8
 /// and scheduling for deletion on IMAP.
 pub async fn delete_msgs(context: &Context, msg_ids: &[MsgId]) -> Result<()> {
    let mut modified_chat_ids = BTreeSet::new();
+    let mut res = Ok(());

    for &msg_id in msg_ids {
        let msg = Message::load_from_db(context, msg_id).await?;
@@ -1452,13 +1471,19 @@ pub async fn delete_msgs(context: &Context, msg_ids: &[MsgId]) -> Result<()> {
        modified_chat_ids.insert(msg.chat_id);

        let target = context.get_delete_msgs_target().await?;
-        context
-            .sql
-            .execute(
+        let update_db = |conn: &mut rusqlite::Connection| {
+            conn.execute(
                "UPDATE imap SET target=? WHERE rfc724_mid=?",
                (target, msg.rfc724_mid),
-            )
-            .await?;
+            )?;
+            conn.execute("DELETE FROM smtp WHERE msg_id=?", (msg_id,))?;
+            Ok(())
+        };
+        if let Err(e) = context.sql.call_write(update_db).await {
+            error!(context, "delete_msgs: failed to update db: {e:#}.");
+            res = Err(e);
+            continue;
+        }

        let logging_xdc_id = context
            .debug_logging
@@ -1473,6 +1498,7 @@ pub async fn delete_msgs(context: &Context, msg_ids: &[MsgId]) -> Result<()> {
            }
        }
    }
+    res?;

    for modified_chat_id in modified_chat_ids {
        context.emit_msgs_changed(modified_chat_id, MsgId::new(0));
@@ -1641,53 +1667,36 @@ pub(crate) async fn update_msg_state(

 // Context functions to work with messages

-/// Returns true if given message ID exists in the database and is not trashed.
-pub(crate) async fn exists(context: &Context, msg_id: MsgId) -> Result<bool> {
-    if msg_id.is_special() {
-        return Ok(false);
+pub(crate) async fn set_msg_failed(
+    context: &Context,
+    msg: &mut Message,
+    error: &str,
+) -> Result<()> {
+    if msg.state.can_fail() {
+        msg.state = MessageState::OutFailed;
+        warn!(context, "{} failed: {}", msg.id, error);
+    } else {
+        warn!(
+            context,
+            "{} seems to have failed ({}), but state is {}", msg.id, error, msg.state
+        )
    }
+    msg.error = Some(error.to_string());

-    let chat_id: Option<ChatId> = context
+    context
        .sql
-        .query_get_value("SELECT chat_id FROM msgs WHERE id=?;", (msg_id,))
+        .execute(
+            "UPDATE msgs SET state=?, error=? WHERE id=?;",
+            (msg.state, error, msg.id),
+        )
        .await?;

-    if let Some(chat_id) = chat_id {
-        Ok(!chat_id.is_trash())
-    } else {
-        Ok(false)
-    }
-}
+    context.emit_event(EventType::MsgFailed {
+        chat_id: msg.chat_id,
+        msg_id: msg.id,
+    });

-pub(crate) async fn set_msg_failed(context: &Context, msg_id: MsgId, error: &str) {
-    if let Ok(mut msg) = Message::load_from_db(context, msg_id).await {
-        if msg.state.can_fail() {
-            msg.state = MessageState::OutFailed;
-            warn!(context, "{} failed: {}", msg_id, error);
-        } else {
-            warn!(
-                context,
-                "{} seems to have failed ({}), but state is {}", msg_id, error, msg.state
-            )
-        }
-
-        match context
-            .sql
-            .execute(
-                "UPDATE msgs SET state=?, error=? WHERE id=?;",
-                (msg.state, error, msg_id),
-            )
-            .await
-        {
-            Ok(_) => context.emit_event(EventType::MsgFailed {
-                chat_id: msg.chat_id,
-                msg_id,
-            }),
-            Err(e) => {
-                warn!(context, "{:?}", e);
-            }
-        }
-    }
+    Ok(())
 }

 /// The number of messages assigned to unblocked chats
@@ -2294,7 +2303,7 @@ mod tests {
        update_msg_state(&alice, alice_msg.id, MessageState::OutMdnRcvd).await?;
        assert_state(&alice, alice_msg.id, MessageState::OutMdnRcvd).await;

-        set_msg_failed(&alice, alice_msg.id, "badly failed").await;
+        set_msg_failed(&alice, &mut alice_msg, "badly failed").await?;
        assert_state(&alice, alice_msg.id, MessageState::OutFailed).await;

        // check incoming message states on receiver side
@@ -2430,4 +2439,23 @@ def hello():

        Ok(())
    }
+
+    #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
+    async fn test_delete_msgs_offline() -> Result<()> {
+        let alice = TestContext::new_alice().await;
+        let chat = alice
+            .create_chat_with_contact("Bob", "bob@example.org")
+            .await;
+        let mut msg = Message::new(Viewtype::Text);
+        msg.set_text("hi".to_string());
+        assert!(chat::send_msg_sync(&alice, chat.id, &mut msg)
+            .await
+            .is_err());
+        let stmt = "SELECT COUNT(*) FROM smtp WHERE msg_id=?";
+        assert!(alice.sql.exists(stmt, (msg.id,)).await?);
+        delete_msgs(&alice, &[msg.id]).await?;
+        assert!(!alice.sql.exists(stmt, (msg.id,)).await?);
+
+        Ok(())
+    }
 }
--- a/src/mimeparser.rs
+++ b/src/mimeparser.rs
@@ -2,6 +2,7 @@

 use std::collections::{HashMap, HashSet};
 use std::future::Future;
+use std::path::Path;
 use std::pin::Pin;
 use std::str;

@@ -11,7 +12,6 @@ use deltachat_derive::{FromSql, ToSql};
 use format_flowed::unformat_flowed;
 use lettre_email::mime::{self, Mime};
 use mailparse::{addrparse_header, DispositionType, MailHeader, MailHeaderMap, SingleInfo};
-use once_cell::sync::Lazy;

 use crate::aheader::{Aheader, EncryptPreference};
 use crate::blob::BlobObject;
@@ -29,7 +29,9 @@ use crate::events::EventType;
 use crate::headerdef::{HeaderDef, HeaderDefMap};
 use crate::key::{DcKey, Fingerprint, SignedPublicKey, SignedSecretKey};
 use crate::keyring::Keyring;
-use crate::message::{self, set_msg_failed, update_msg_state, MessageState, MsgId, Viewtype};
+use crate::message::{
+    self, set_msg_failed, update_msg_state, Message, MessageState, MsgId, Viewtype,
+};
 use crate::param::{Param, Params};
 use crate::peerstate::Peerstate;
 use crate::simplify::{simplify, SimplifiedText};
@@ -436,6 +438,8 @@ impl MimeMessage {
                        typ: Viewtype::Text,
                        msg_raw: Some(txt.clone()),
                        msg: txt,
+                        // Don't change the error prefix for now,
+                        // receive_imf.rs:lookup_chat_by_reply() checks it.
                        error: Some(format!("Decrypting failed: {err:#}")),
                        ..Default::default()
                    };
@@ -861,7 +865,7 @@ impl MimeMessage {
        is_related: bool,
    ) -> Result<bool> {
        let mut any_part_added = false;
-        let mimetype = get_mime_type(mail)?.0;
+        let mimetype = get_mime_type(mail, &get_attachment_filename(context, mail)?)?.0;
        match (mimetype.type_(), mimetype.subtype().as_str()) {
            /* Most times, multipart/alternative contains true alternatives
            as text/plain and text/html.  If we find a multipart/mixed
@@ -869,9 +873,9 @@ impl MimeMessage {
            apple mail: "plaintext" as an alternative to "html+PDF attachment") */
            (mime::MULTIPART, "alternative") => {
                for cur_data in &mail.subparts {
-                    if get_mime_type(cur_data)?.0 == "multipart/mixed"
-                        || get_mime_type(cur_data)?.0 == "multipart/related"
-                    {
+                    let mime_type =
+                        get_mime_type(cur_data, &get_attachment_filename(context, cur_data)?)?.0;
+                    if mime_type == "multipart/mixed" || mime_type == "multipart/related" {
                        any_part_added = self
                            .parse_mime_recursive(context, cur_data, is_related)
                            .await?;
@@ -881,7 +885,11 @@ impl MimeMessage {
                if !any_part_added {
                    /* search for text/plain and add this */
                    for cur_data in &mail.subparts {
-                        if get_mime_type(cur_data)?.0.type_() == mime::TEXT {
+                        if get_mime_type(cur_data, &get_attachment_filename(context, cur_data)?)?
+                            .0
+                            .type_()
+                            == mime::TEXT
+                        {
                            any_part_added = self
                                .parse_mime_recursive(context, cur_data, is_related)
                                .await?;
@@ -1007,10 +1015,9 @@ impl MimeMessage {
        is_related: bool,
    ) -> Result<bool> {
        // return true if a part was added
-        let (mime_type, msg_type) = get_mime_type(mail)?;
-        let raw_mime = mail.ctype.mimetype.to_lowercase();
-
        let filename = get_attachment_filename(context, mail)?;
+        let (mime_type, msg_type) = get_mime_type(mail, &filename)?;
+        let raw_mime = mail.ctype.mimetype.to_lowercase();

        let old_part_count = self.parts.len();

@@ -1613,25 +1620,21 @@ impl MimeMessage {
            false
        };
        if maybe_ndn && self.delivery_report.is_none() {
-            static RE: Lazy<regex::Regex> =
-                Lazy::new(|| regex::Regex::new(r"Message-ID:(.*)").unwrap());
-            for captures in self
+            for original_message_id in self
                .parts
                .iter()
                .filter_map(|part| part.msg_raw.as_ref())
                .flat_map(|part| part.lines())
-                .filter_map(|line| RE.captures(line))
+                .filter_map(|line| line.split_once("Message-ID:"))
+                .filter_map(|(_, message_id)| parse_message_id(message_id).ok())
            {
-                if let Ok(original_message_id) = parse_message_id(&captures[1]) {
-                    if let Ok(Some(_)) =
-                        message::rfc724_mid_exists(context, &original_message_id).await
-                    {
-                        self.delivery_report = Some(DeliveryReport {
-                            rfc724_mid: original_message_id,
-                            failed_recipient: None,
-                            failure: true,
-                        })
-                    }
+                if let Ok(Some(_)) = message::rfc724_mid_exists(context, &original_message_id).await
+                {
+                    self.delivery_report = Some(DeliveryReport {
+                        rfc724_mid: original_message_id,
+                        failed_recipient: None,
+                        failure: true,
+                    })
                }
            }
        }
@@ -1865,7 +1868,10 @@ pub struct Part {
 }

 /// return mimetype and viewtype for a parsed mail
-fn get_mime_type(mail: &mailparse::ParsedMail<'_>) -> Result<(Mime, Viewtype)> {
+fn get_mime_type(
+    mail: &mailparse::ParsedMail<'_>,
+    filename: &Option<String>,
+) -> Result<(Mime, Viewtype)> {
    let mimetype = mail.ctype.mimetype.parse::<Mime>()?;

    let viewtype = match mimetype.type_() {
@@ -1901,7 +1907,16 @@ fn get_mime_type(mail: &mailparse::ParsedMail<'_>) -> Result<(Mime, Viewtype)> {
                Viewtype::Unknown
            }
        }
-        mime::APPLICATION => Viewtype::File,
+        mime::APPLICATION => match mimetype.subtype() {
+            mime::OCTET_STREAM => match filename {
+                Some(filename) => match message::guess_msgtype_from_suffix(Path::new(&filename)) {
+                    Some((viewtype, _)) => viewtype,
+                    None => Viewtype::File,
+                },
+                None => Viewtype::File,
+            },
+            _ => Viewtype::File,
+        },
        _ => Viewtype::Unknown,
    };

@@ -2145,7 +2160,8 @@ async fn handle_ndn(
    let mut first = true;
    for msg in msgs {
        let (msg_id, chat_id, chat_type) = msg?;
-        set_msg_failed(context, msg_id, &error).await;
+        let mut message = Message::load_from_db(context, msg_id).await?;
+        set_msg_failed(context, &mut message, &error).await?;
        if first {
            // Add only one info msg for all failed messages
            ndn_maybe_add_info_msg(context, failed, chat_id, chat_type).await?;
@@ -3091,7 +3107,7 @@ CWt6wx7fiLp0qS9RrX75g6Gqw7nfCs6EcBERcIPt7DTe8VStJwf3LWqVwxl4gQl46yhfoqwEO+I=
    #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
    async fn parse_outlook_html_embedded_image() {
        let context = TestContext::new_alice().await;
-        let raw = br##"From: Anonymous <anonymous@example.org>
+        let raw = br#"From: Anonymous <anonymous@example.org>
 To: Anonymous <anonymous@example.org>
 Subject: Delta Chat is great stuff!
 Date: Tue, 5 May 2020 01:23:45 +0000
@@ -3146,7 +3162,7 @@ K+TuvC7qOah0WLFhcsXWn2+dDV1bXuAeC769TkqkpHhdXfUHnVgK3Pv7u3rVPT5AMeFUGxRB2dP4
 CWt6wx7fiLp0qS9RrX75g6Gqw7nfCs6EcBERcIPt7DTe8VStJwf3LWqVwxl4gQl46yhfoqwEO+I=

 ------=_NextPart_000_0003_01D622B3.CA753E60--
-"##;
+"#;

        let message = MimeMessage::from_bytes(&context.ctx, &raw[..], None)
            .await
@@ -3521,7 +3537,7 @@ On 2020-10-25, Bob wrote:

        // A message with a long Message-ID.
        // Long message-IDs are generated by Mailjet.
-        let raw = br###"Date: Thu, 28 Jan 2021 00:26:57 +0000
+        let raw = br"Date: Thu, 28 Jan 2021 00:26:57 +0000
 Chat-Version: 1.0\n\
 Message-ID: <ABCDEFGH.1234567_AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA@mailjet.com>
 To: Bob <bob@example.org>
@@ -3529,11 +3545,11 @@ From: Alice <alice@example.org>
 Subject: ...

 Some quote.
-"###;
+";
        receive_imf(&t, raw, false).await?;

        // Delta Chat generates In-Reply-To with a starting tab when Message-ID is too long.
-        let raw = br###"In-Reply-To:
+        let raw = br"In-Reply-To:
 	<ABCDEFGH.1234567_AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA@mailjet.com>
 Date: Thu, 28 Jan 2021 00:26:57 +0000
 Chat-Version: 1.0\n\
@@ -3545,7 +3561,7 @@ Subject: ...
 > Some quote.

 Some reply
-"###;
+";

        receive_imf(&t, raw, false).await?;

@@ -3563,7 +3579,7 @@ Some reply
        let alice = TestContext::new_alice().await;
        let bob = TestContext::new_bob().await;

-        let raw = br###"Date: Thu, 28 Jan 2021 00:26:57 +0000
+        let raw = br"Date: Thu, 28 Jan 2021 00:26:57 +0000
 Chat-Version: 1.0\n\
 Message-ID: <foobarbaz@example.org>
 To: Bob <bob@example.org>
@@ -3572,7 +3588,7 @@ Subject: subject
 Chat-Disposition-Notification-To: alice@example.org

 Message.
-"###;
+";

        // Bob receives message.
        receive_imf(&bob, raw, false).await?;
@@ -3756,4 +3772,22 @@ Content-Disposition: reaction\n\

        Ok(())
    }
+
+    #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
+    async fn test_jpeg_as_application_octet_stream() -> Result<()> {
+        let context = TestContext::new_alice().await;
+        let raw = include_bytes!("../test-data/message/jpeg-as-application-octet-stream.eml");
+
+        let msg = MimeMessage::from_bytes(&context.ctx, &raw[..], None)
+            .await
+            .unwrap();
+        assert_eq!(msg.parts.len(), 1);
+        assert_eq!(msg.parts[0].typ, Viewtype::Image);
+
+        receive_imf(&context, &raw[..], false).await?;
+        let msg = context.get_last_msg().await;
+        assert_eq!(msg.get_viewtype(), Viewtype::Image);
+
+        Ok(())
+    }
 }
--- a/src/param.rs
+++ b/src/param.rs
@@ -281,6 +281,16 @@ impl Params {
        self
    }

+    /// Sets the given key from an optional value.
+    /// Removes the key if the value is `None`.
+    pub fn set_optional(&mut self, key: Param, value: Option<impl ToString>) -> &mut Self {
+        if let Some(value) = value {
+            self.set(key, value)
+        } else {
+            self.remove(key)
+        }
+    }
+
    /// Check if there are any values in this.
    pub fn is_empty(&self) -> bool {
        self.inner.is_empty()
--- a/src/pgp.rs
+++ b/src/pgp.rs
@@ -24,11 +24,18 @@ use crate::keyring::Keyring;
 use crate::tools::EmailAddress;

 #[allow(missing_docs)]
-pub const HEADER_AUTOCRYPT: &str = "autocrypt-prefer-encrypt";
+#[cfg(test)]
+pub(crate) const HEADER_AUTOCRYPT: &str = "autocrypt-prefer-encrypt";

 #[allow(missing_docs)]
 pub const HEADER_SETUPCODE: &str = "passphrase-begin";

+/// Preferred symmetric encryption algorithm.
+const SYMMETRIC_KEY_ALGORITHM: SymmetricKeyAlgorithm = SymmetricKeyAlgorithm::AES128;
+
+/// Preferred cryptographic hash.
+const HASH_ALGORITHM: HashAlgorithm = HashAlgorithm::SHA2_256;
+
 /// A wrapper for rPGP public key types
 #[derive(Debug)]
 enum SignedPublicKeyOrSubkey<'a> {
@@ -135,6 +142,7 @@ pub struct KeyPair {
 pub(crate) fn create_keypair(addr: EmailAddress, keygen_type: KeyGenType) -> Result<KeyPair> {
    let (secret_key_type, public_key_type) = match keygen_type {
        KeyGenType::Rsa2048 => (PgpKeyType::Rsa(2048), PgpKeyType::Rsa(2048)),
+        KeyGenType::Rsa4096 => (PgpKeyType::Rsa(4096), PgpKeyType::Rsa(4096)),
        KeyGenType::Ed25519 | KeyGenType::Default => (PgpKeyType::EdDSA, PgpKeyType::ECDH),
    };

@@ -248,11 +256,13 @@ pub async fn pk_encrypt(
            // TODO: measure time
            let encrypted_msg = if let Some(ref skey) = private_key_for_signing {
                lit_msg
-                    .sign(skey, || "".into(), Default::default())
+                    .sign(skey, || "".into(), HASH_ALGORITHM)
                    .and_then(|msg| msg.compress(CompressionAlgorithm::ZLIB))
-                    .and_then(|msg| msg.encrypt_to_keys(&mut rng, Default::default(), &pkeys_refs))
+                    .and_then(|msg| {
+                        msg.encrypt_to_keys(&mut rng, SYMMETRIC_KEY_ALGORITHM, &pkeys_refs)
+                    })
            } else {
-                lit_msg.encrypt_to_keys(&mut rng, Default::default(), &pkeys_refs)
+                lit_msg.encrypt_to_keys(&mut rng, SYMMETRIC_KEY_ALGORITHM, &pkeys_refs)
            };

            let msg = encrypted_msg?;
@@ -271,7 +281,7 @@ pub fn pk_calc_signature(
    let msg = Message::new_literal_bytes("", plain).sign(
        private_key_for_signing,
        || "".into(),
-        Default::default(),
+        HASH_ALGORITHM,
    )?;
    let signature = msg.into_signature().to_armored_string(None)?;
    Ok(signature)
@@ -368,7 +378,7 @@ pub async fn symm_encrypt(passphrase: &str, plain: &[u8]) -> Result<String> {
        let mut rng = thread_rng();
        let s2k = StringToKey::new_default(&mut rng);
        let msg =
-            lit_msg.encrypt_with_password(&mut rng, s2k, Default::default(), || passphrase)?;
+            lit_msg.encrypt_with_password(&mut rng, s2k, SYMMETRIC_KEY_ALGORITHM, || passphrase)?;

        let encoded_msg = msg.to_armored_string(None)?;

--- a/src/plaintext.rs
+++ b/src/plaintext.rs
@@ -26,10 +26,10 @@ impl PlainText {
    /// The function handles quotes, links, fixed and floating text paragraphs.
    pub fn to_html(&self) -> String {
        static LINKIFY_MAIL_RE: Lazy<regex::Regex> =
-            Lazy::new(|| regex::Regex::new(r#"\b([\w.\-+]+@[\w.\-]+)\b"#).unwrap());
+            Lazy::new(|| regex::Regex::new(r"\b([\w.\-+]+@[\w.\-]+)\b").unwrap());

        static LINKIFY_URL_RE: Lazy<regex::Regex> = Lazy::new(|| {
-            regex::Regex::new(r#"\b((http|https|ftp|ftps):[\w.,:;$/@!?&%\-~=#+]+)"#).unwrap()
+            regex::Regex::new(r"\b((http|https|ftp|ftps):[\w.,:;$/@!?&%\-~=#+]+)").unwrap()
        });

        let lines = split_lines(&self.text);
@@ -127,7 +127,7 @@ http://link-at-start-of-line.org
        .to_html();
        assert_eq!(
            html,
-            r##"<!DOCTYPE html>
+            r#"<!DOCTYPE html>
 <html><head>
 <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
 <meta name="color-scheme" content="light dark" />
@@ -138,7 +138,7 @@ line with <a href="https://link-mid-of-line.org">https://link-mid-of-line.org</a
 <a href="http://link-at-start-of-line.org">http://link-at-start-of-line.org</a><br/>
 <br/>
 </body></html>
-"##
+"#
        );
    }

--- a/src/provider.rs
+++ b/src/provider.rs
@@ -123,10 +123,10 @@ pub struct Provider {
    pub overview_page: &'static str,

    /// List of provider servers.
-    pub server: Vec<Server>,
+    pub server: &'static [Server],

    /// Default configuration values to set when provider is configured.
-    pub config_defaults: Option<Vec<ConfigDefault>>,
+    pub config_defaults: Option<&'static [ConfigDefault]>,

    /// Type of OAuth 2 authorization if provider supports it.
    pub oauth2_authorizer: Option<Oauth2Authorizer>,
@@ -149,8 +149,8 @@ pub struct ProviderOptions {
    pub delete_to_trash: bool,
 }

-impl Default for ProviderOptions {
-    fn default() -> Self {
+impl ProviderOptions {
+    const fn new() -> Self {
        Self {
            strict_tls: true,
            max_smtp_rcpt_to: None,
@@ -223,7 +223,7 @@ pub async fn get_provider_by_mx(context: &Context, domain: &str) -> Option<&'sta
        }

        if let Ok(mx_domains) = resolver.mx_lookup(fqdn).await {
-            for (provider_domain, provider) in PROVIDER_DATA.iter() {
+            for (provider_domain, provider) in &*PROVIDER_DATA {
                if provider.id != "gmail" {
                    // MX lookup is limited to Gmail for security reasons
                    continue;
--- a/src/provider/data.rs
+++ b/src/provider/data.rs
--- a/src/quota.rs
+++ b/src/quota.rs
@@ -71,7 +71,7 @@ async fn get_unique_quota_roots_and_usage(
                // messages could be received and so the usage could have been changed
                *unique_quota_roots
                    .entry(quota_root_name.clone())
-                    .or_insert_with(Vec::new) = quota.resources;
+                    .or_default() = quota.resources;
            }
        }
    }
--- a/src/receive_imf.rs
+++ b/src/receive_imf.rs
@@ -546,19 +546,30 @@ async fn add_parts(
        // signals whether the current user is a bot
        let is_bot = context.get_config_bool(Config::Bot).await?;

-        let create_blocked = match test_normal_chat {
-            Some(ChatIdBlocked {
-                id: _,
-                blocked: Blocked::Request,
-            }) if is_bot => Blocked::Not,
-            Some(ChatIdBlocked { id: _, blocked }) => blocked,
-            None => {
-                if is_bot {
-                    Blocked::Not
-                } else {
-                    Blocked::Request
+        let create_blocked_default = if is_bot {
+            Blocked::Not
+        } else {
+            Blocked::Request
+        };
+        let create_blocked = if let Some(ChatIdBlocked { id: _, blocked }) = test_normal_chat {
+            match blocked {
+                Blocked::Request => create_blocked_default,
+                Blocked::Not => Blocked::Not,
+                Blocked::Yes => {
+                    if Contact::is_blocked_load(context, from_id).await? {
+                        // User has blocked the contact.
+                        // Block the group contact created as well.
+                        Blocked::Yes
+                    } else {
+                        // 1:1 chat is blocked, but the contact is not.
+                        // This happens when 1:1 chat is hidden
+                        // during scanning of a group invitation code.
+                        Blocked::Request
+                    }
                }
            }
+        } else {
+            create_blocked_default
        };

        if chat_id.is_none() {
@@ -1068,7 +1079,7 @@ async fn add_parts(
    let mut save_mime_modified = mime_parser.is_mime_modified;

    let mime_headers = if save_mime_headers || save_mime_modified {
-        let headers = if mime_parser.was_encrypted() && !mime_parser.decoded_data.is_empty() {
+        let headers = if !mime_parser.decoded_data.is_empty() {
            mime_parser.decoded_data.clone()
        } else {
            imf_raw.to_vec()
@@ -1216,6 +1227,8 @@ RETURNING id
                    ephemeral_timestamp,
                    if is_partial_download.is_some() {
                        DownloadState::Available
+                    } else if mime_parser.decrypting_failed {
+                        DownloadState::Undecipherable
                    } else {
                        DownloadState::Done
                    },
@@ -1398,11 +1411,18 @@ async fn lookup_chat_by_reply(
    if let Some(parent) = parent {
        let parent_chat = Chat::load_from_db(context, parent.chat_id).await?;

-        if parent.error.is_some() {
-            // If the parent msg is undecipherable, then it may have been assigned to the wrong chat
-            // (undecipherable group msgs often get assigned to the 1:1 chat with the sender).
-            // We don't have any way of finding out whether a msg is undecipherable, so we check for
-            // error.is_some() instead.
+        if parent.download_state != DownloadState::Done
+            // TODO (2023-09-12): Added for backward compatibility with versions that did not have
+            // `DownloadState::Undecipherable`. Remove eventually with the comment in
+            // `MimeMessage::from_bytes()`.
+            || parent
+                .error
+                .as_ref()
+                .filter(|e| e.starts_with("Decrypting failed:"))
+                .is_some()
+        {
+            // If the parent msg is not fully downloaded or undecipherable, it may have been
+            // assigned to the wrong chat (they often get assigned to the 1:1 chat with the sender).
            return Ok(None);
        }

@@ -1654,37 +1674,37 @@ async fn apply_group_changes(
            false
        };

-    // Whether to allow any changes to the member list at all.
-    let allow_member_list_changes =
-        if chat::is_contact_in_chat(context, chat_id, ContactId::SELF).await? || self_added {
-            // Reject old group changes.
-            chat_id
-                .update_timestamp(context, Param::MemberListTimestamp, sent_timestamp)
-                .await?
-        } else {
-            // Member list changes are not allowed if we're not in the group
-            // and are not explicitly added.
-            // This message comes from a Delta Chat that restored an old backup
-            // or the message is a MUA reply to an old message.
-            false
-        };
+    let is_from_in_chat = !chat::is_contact_in_chat(context, chat_id, ContactId::SELF).await?
+        || chat::is_contact_in_chat(context, chat_id, from_id).await?;
+
+    // Reject group membership changes from non-members and old changes.
+    let allow_member_list_changes = is_from_in_chat
+        && chat_id
+            .update_timestamp(context, Param::MemberListTimestamp, sent_timestamp)
+            .await?;

    // Whether to rebuild the member list from scratch.
-    let recreate_member_list = if allow_member_list_changes {
+    let recreate_member_list = {
        // Recreate member list if the message comes from a MUA as these messages do _not_ set add/remove headers.
-        // Always recreate membership list if self has been added.
-        if !mime_parser.has_chat_version() || self_added {
-            true
-        } else {
-            match mime_parser.get_header(HeaderDef::InReplyTo) {
+        !mime_parser.has_chat_version()
+            // Always recreate membership list if SELF has been added. The older versions of DC
+            // don't always set "In-Reply-To" to the latest message they sent, but to the latest
+            // delivered message (so it's a race), so we have this heuristic here.
+            || self_added
+            || match mime_parser.get_header(HeaderDef::InReplyTo) {
                // If we don't know the referenced message, we missed some messages.
                // Maybe they added/removed members, so we need to recreate our member list.
                Some(reply_to) => rfc724_mid_exists(context, reply_to).await?.is_none(),
                None => false,
            }
+    } && {
+        if !allow_member_list_changes {
+            info!(
+                context,
+                "Ignoring a try to recreate member list of {chat_id} by {from_id}.",
+            );
        }
-    } else {
-        false
+        allow_member_list_changes
    };

    if let Some(removed_addr) = mime_parser.get_header(HeaderDef::ChatGroupMemberRemoved) {
@@ -1791,43 +1811,35 @@ async fn apply_group_changes(

    // Recreate the member list.
    if recreate_member_list {
-        if !chat::is_contact_in_chat(context, chat_id, from_id).await? {
-            warn!(
-                context,
-                "Contact {from_id} attempts to modify group chat {chat_id} member list without being a member."
-            );
-        } else {
-            // Only delete old contacts if the sender is not a classical MUA user:
-            // Classical MUA users usually don't intend to remove users from an email
-            // thread, so if they removed a recipient then it was probably by accident.
-            if mime_parser.has_chat_version() {
-                context
-                    .sql
-                    .execute("DELETE FROM chats_contacts WHERE chat_id=?;", (chat_id,))
-                    .await?;
-            }
-
-            let mut members_to_add = HashSet::new();
-            members_to_add.extend(to_ids);
-            members_to_add.insert(ContactId::SELF);
-
-            if !from_id.is_special() {
-                members_to_add.insert(from_id);
-            }
-
-            if let Some(removed_id) = removed_id {
-                members_to_add.remove(&removed_id);
-            }
-
-            info!(
-                context,
-                "Recreating chat {chat_id} with members {members_to_add:?}."
-            );
-
-            chat::add_to_chat_contacts_table(context, chat_id, &Vec::from_iter(members_to_add))
+        // Only delete old contacts if the sender is not a classical MUA user:
+        // Classical MUA users usually don't intend to remove users from an email
+        // thread, so if they removed a recipient then it was probably by accident.
+        if mime_parser.has_chat_version() {
+            context
+                .sql
+                .execute("DELETE FROM chats_contacts WHERE chat_id=?;", (chat_id,))
                .await?;
-            send_event_chat_modified = true;
        }
+
+        let mut members_to_add = HashSet::new();
+        members_to_add.extend(to_ids);
+        members_to_add.insert(ContactId::SELF);
+
+        if !from_id.is_special() {
+            members_to_add.insert(from_id);
+        }
+
+        if let Some(removed_id) = removed_id {
+            members_to_add.remove(&removed_id);
+        }
+
+        info!(
+            context,
+            "Recreating chat {chat_id} with members {members_to_add:?}."
+        );
+
+        chat::add_to_chat_contacts_table(context, chat_id, &Vec::from_iter(members_to_add)).await?;
+        send_event_chat_modified = true;
    }

    if let Some(avatar_action) = &mime_parser.group_avatar {
@@ -2345,7 +2357,7 @@ async fn add_or_lookup_contacts_by_address_list(
    origin: Origin,
 ) -> Result<Vec<ContactId>> {
    let mut contact_ids = HashSet::new();
-    for info in address_list.iter() {
+    for info in address_list {
        let addr = &info.addr;
        if !may_be_valid_addr(addr) {
            continue;
--- a/src/receive_imf/tests.rs
+++ b/src/receive_imf/tests.rs
@@ -8,7 +8,8 @@ use crate::chat::{
 };
 use crate::chat::{get_chat_msgs, ChatItem, ChatVisibility};
 use crate::chatlist::Chatlist;
-use crate::constants::DC_GCL_NO_SPECIALS;
+use crate::config::Config;
+use crate::constants::{DC_GCL_FOR_FORWARDING, DC_GCL_NO_SPECIALS};
 use crate::imap::prefetch_should_download;
 use crate::message::Message;
 use crate::test_utils::{get_chat_msg, TestContext, TestContextManager};
@@ -793,6 +794,8 @@ async fn test_github_mailing_list() -> Result<()> {

    let chats = Chatlist::try_load(&t.ctx, 0, None, None).await?;
    assert_eq!(chats.len(), 1);
+    let chats = Chatlist::try_load(&t.ctx, DC_GCL_FOR_FORWARDING, None, None).await?;
+    assert_eq!(chats.len(), 0);
    let contacts = Contact::get_all(&t.ctx, 0, None).await?;
    assert_eq!(contacts.len(), 0); // mailing list recipients and senders do not count as "known contacts"

@@ -3560,3 +3563,119 @@ async fn test_mua_can_add() -> Result<()> {
    );
    Ok(())
 }
+
+#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
+async fn test_mua_can_readd() -> Result<()> {
+    let alice = TestContext::new_alice().await;
+
+    // Alice creates chat with 3 contacts.
+    let msg = receive_imf(
+        &alice,
+        b"Subject: =?utf-8?q?Message_from_alice=40example=2Eorg?=\r\n\
+            From: alice@example.org\r\n\
+            To: <bob@example.net>, <claire@example.org>, <fiona@example.org> \r\n\
+            Date: Mon, 12 Dec 2022 14:30:39 +0000\r\n\
+            Message-ID: <Mr.alices_original_mail@example.org>\r\n\
+            Chat-Version: 1.0\r\n\
+            \r\n\
+            Hi!\r\n",
+        false,
+    )
+    .await?
+    .unwrap();
+    let alice_chat = Chat::load_from_db(&alice, msg.chat_id).await?;
+    assert_eq!(alice_chat.typ, Chattype::Group);
+    assert!(is_contact_in_chat(&alice, alice_chat.id, ContactId::SELF).await?);
+
+    // And leaves it.
+    remove_contact_from_chat(&alice, alice_chat.id, ContactId::SELF).await?;
+    let alice_chat = Chat::load_from_db(&alice, alice_chat.id).await?;
+    assert!(!is_contact_in_chat(&alice, alice_chat.id, ContactId::SELF).await?);
+
+    // Bob uses a classical MUA to answer, adding Alice back.
+    receive_imf(
+        &alice,
+        b"Subject: Re: Message from alice\r\n\
+            From: <bob@example.net>\r\n\
+            To: <alice@example.org>, <claire@example.org>, <fiona@example.org>\r\n\
+            Date: Mon, 12 Dec 2022 14:32:39 +0000\r\n\
+            Message-ID: <bobs_answer_to_two_recipients@example.net>\r\n\
+            In-Reply-To: <Mr.alices_original_mail@example.org>\r\n\
+            \r\n\
+            Hi back!\r\n",
+        false,
+    )
+    .await?
+    .unwrap();
+
+    let alice_chat = Chat::load_from_db(&alice, alice_chat.id).await?;
+    assert!(is_contact_in_chat(&alice, alice_chat.id, ContactId::SELF).await?);
+    Ok(())
+}
+
+#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
+async fn test_recreate_member_list_on_missing_add_of_self() -> Result<()> {
+    let alice = TestContext::new_alice().await;
+    let bob = TestContext::new_bob().await;
+    let alice_chat_id = create_group_chat(&alice, ProtectionStatus::Unprotected, "Group").await?;
+    add_contact_to_chat(
+        &alice,
+        alice_chat_id,
+        Contact::create(&alice, "bob", &bob.get_config(Config::Addr).await?.unwrap()).await?,
+    )
+    .await?;
+    send_text_msg(&alice, alice_chat_id, "populate".to_string()).await?;
+    alice.pop_sent_msg().await;
+    remove_contact_from_chat(&alice, alice_chat_id, ContactId::SELF).await?;
+    let bob_chat_id = bob.recv_msg(&alice.pop_sent_msg().await).await.chat_id;
+
+    // Bob missed the message adding them, but must recreate the member list.
+    assert_eq!(get_chat_contacts(&bob, bob_chat_id).await?.len(), 1);
+    assert!(is_contact_in_chat(&bob, bob_chat_id, ContactId::SELF).await?);
+    Ok(())
+}
+
+#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
+async fn test_keep_member_list_if_possibly_nomember() -> Result<()> {
+    let alice = TestContext::new_alice().await;
+    let bob = TestContext::new_bob().await;
+    let alice_chat_id = create_group_chat(&alice, ProtectionStatus::Unprotected, "Group").await?;
+    add_contact_to_chat(
+        &alice,
+        alice_chat_id,
+        Contact::create(&alice, "bob", &bob.get_config(Config::Addr).await?.unwrap()).await?,
+    )
+    .await?;
+    send_text_msg(&alice, alice_chat_id, "populate".to_string()).await?;
+    let bob_chat_id = bob.recv_msg(&alice.pop_sent_msg().await).await.chat_id;
+
+    let fiona = TestContext::new_fiona().await;
+    add_contact_to_chat(
+        &alice,
+        alice_chat_id,
+        Contact::create(
+            &alice,
+            "fiona",
+            &fiona.get_config(Config::Addr).await?.unwrap(),
+        )
+        .await?,
+    )
+    .await?;
+    let fiona_chat_id = fiona.recv_msg(&alice.pop_sent_msg().await).await.chat_id;
+    fiona_chat_id.accept(&fiona).await?;
+
+    send_text_msg(&fiona, fiona_chat_id, "hi".to_string()).await?;
+    bob.recv_msg(&fiona.pop_sent_msg().await).await;
+
+    // Bob missed the message adding fiona, but mustn't recreate the member list.
+    assert_eq!(get_chat_contacts(&bob, bob_chat_id).await?.len(), 2);
+    assert!(is_contact_in_chat(&bob, bob_chat_id, ContactId::SELF).await?);
+    let bob_alice_contact = Contact::create(
+        &bob,
+        "alice",
+        &alice.get_config(Config::Addr).await?.unwrap(),
+    )
+    .await?;
+    assert!(is_contact_in_chat(&bob, bob_chat_id, bob_alice_contact).await?);
+    Ok(())
+}
--- a/src/smtp.rs
+++ b/src/smtp.rs
@@ -55,7 +55,7 @@ impl Smtp {
    }

    /// Disconnect the SMTP transport and drop it entirely.
-    pub async fn disconnect(&mut self) {
+    pub fn disconnect(&mut self) {
        if let Some(mut transport) = self.transport.take() {
            // Closing connection with a QUIT command may take some time, especially if it's a
            // stale connection and an attempt to send the command times out. Send a command in a
@@ -88,7 +88,7 @@ impl Smtp {
    pub async fn connect_configured(&mut self, context: &Context) -> Result<()> {
        if self.has_maybe_stale_connection() {
            info!(context, "Closing stale connection");
-            self.disconnect().await;
+            self.disconnect();
        }

        if self.is_connected() {
@@ -465,13 +465,13 @@ pub(crate) async fn smtp_send(

            // this clears last_success info
            info!(context, "Failed to send message over SMTP, disconnecting");
-            smtp.disconnect().await;
+            smtp.disconnect();

            res
        }
        Err(crate::smtp::send::Error::Envelope(err)) => {
            // Local error, job is invalid, do not retry.
-            smtp.disconnect().await;
+            smtp.disconnect();
            warn!(context, "SMTP job is invalid: {}", err);
            SendResult::Failure(err)
        }
@@ -483,7 +483,7 @@ pub(crate) async fn smtp_send(
        }
        Err(crate::smtp::send::Error::Other(err)) => {
            // Local error, job is invalid, do not retry.
-            smtp.disconnect().await;
+            smtp.disconnect();
            warn!(context, "unable to load job: {}", err);
            SendResult::Failure(err)
        }
@@ -492,7 +492,20 @@ pub(crate) async fn smtp_send(

    if let SendResult::Failure(err) = &status {
        // We couldn't send the message, so mark it as failed
-        message::set_msg_failed(context, msg_id, &err.to_string()).await;
+        match Message::load_from_db(context, msg_id).await {
+            Ok(mut msg) => {
+                if let Err(err) = message::set_msg_failed(context, &mut msg, &err.to_string()).await
+                {
+                    error!(context, "Failed to mark {msg_id} as failed: {err:#}.");
+                }
+            }
+            Err(err) => {
+                error!(
+                    context,
+                    "Failed to load {msg_id} to mark it as failed: {err:#}."
+                );
+            }
+        }
    }
    status
 }
@@ -539,7 +552,8 @@ pub(crate) async fn send_msg_to_smtp(
        )
        .await?;
    if retries > 6 {
-        message::set_msg_failed(context, msg_id, "Number of retries exceeded the limit.").await;
+        let mut msg = Message::load_from_db(context, msg_id).await?;
+        message::set_msg_failed(context, &mut msg, "Number of retries exceeded the limit.").await?;
        context
            .sql
            .execute("DELETE FROM smtp WHERE id=?", (rowid,))
@@ -565,24 +579,6 @@ pub(crate) async fn send_msg_to_smtp(
        )
        .collect::<Vec<_>>();

-    // If there is a msg-id and it does not exist in the db, cancel sending. this happens if
-    // delete_msgs() was called before the generated mime was sent out.
-    if !message::exists(context, msg_id)
-        .await
-        .with_context(|| format!("failed to check message {msg_id} existence"))?
-    {
-        info!(
-            context,
-            "Sending of message {msg_id} (entry {rowid}) was cancelled by the user."
-        );
-        context
-            .sql
-            .execute("DELETE FROM smtp WHERE id=?", (rowid,))
-            .await
-            .context("failed to remove cancelled message from smtp table")?;
-        return Ok(());
-    }
-
    let status = smtp_send(context, &recipients_list, body.as_str(), smtp, msg_id).await;

    match status {
--- a/src/sql.rs
+++ b/src/sql.rs
@@ -304,6 +304,25 @@ impl Sql {
        }
    }

+    /// Changes the passphrase of encrypted database.
+    ///
+    /// The database must already be encrypted and the passphrase cannot be empty.
+    /// It is impossible to turn encrypted database into unencrypted
+    /// and vice versa this way, use import/export for this.
+    pub async fn change_passphrase(&self, passphrase: String) -> Result<()> {
+        let mut lock = self.pool.write().await;
+
+        let pool = lock.take().context("SQL connection pool is not open")?;
+        let conn = pool.get().await?;
+        conn.pragma_update(None, "rekey", passphrase.clone())
+            .context("failed to set PRAGMA rekey")?;
+        drop(pool);
+
+        *lock = Some(Self::new_pool(&self.dbfile, passphrase.to_string())?);
+
+        Ok(())
+    }
+
    /// Locks the write transactions mutex in order to make sure that there never are
    /// multiple write transactions at once.
    ///
@@ -1246,6 +1265,66 @@ mod tests {
        sql.open(&t, "foo".to_string())
            .await
            .context("failed to open the database second time")?;
+
+        Ok(())
+    }
+
+    #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
+    async fn test_sql_change_passphrase() -> Result<()> {
+        use tempfile::tempdir;
+
+        // The context is used only for logging.
+        let t = TestContext::new().await;
+
+        // Create a separate empty database for testing.
+        let dir = tempdir()?;
+        let dbfile = dir.path().join("testdb.sqlite");
+        let sql = Sql::new(dbfile.clone());
+
+        sql.open(&t, "foo".to_string())
+            .await
+            .context("failed to open the database first time")?;
+        sql.close().await;
+
+        // Change the passphrase from "foo" to "bar".
+        let sql = Sql::new(dbfile.clone());
+        sql.open(&t, "foo".to_string())
+            .await
+            .context("failed to open the database second time")?;
+        sql.change_passphrase("bar".to_string())
+            .await
+            .context("failed to change passphrase")?;
+
+        // Test that at least two connections are still working.
+        // This ensures that not only the connection which changed the password is working,
+        // but other connections as well.
+        {
+            let lock = sql.pool.read().await;
+            let pool = lock.as_ref().unwrap();
+            let conn1 = pool.get().await?;
+            let conn2 = pool.get().await?;
+            conn1
+                .query_row("SELECT count(*) FROM sqlite_master", [], |_row| Ok(()))
+                .unwrap();
+            conn2
+                .query_row("SELECT count(*) FROM sqlite_master", [], |_row| Ok(()))
+                .unwrap();
+        }
+
+        sql.close().await;
+
+        let sql = Sql::new(dbfile);
+
+        // Test that old passphrase is not working.
+        assert!(sql.open(&t, "foo".to_string()).await.is_err());
+
+        // Open the database with the new passphrase.
+        sql.check_passphrase("bar".to_string()).await?;
+        sql.open(&t, "bar".to_string())
+            .await
+            .context("failed to open the database third time")?;
+        sql.close().await;
+
        Ok(())
    }
 }
--- a/src/sql/migrations.rs
+++ b/src/sql/migrations.rs
@@ -516,7 +516,8 @@ DO UPDATE SET rfc724_mid=excluded.rfc724_mid,
 id INTEGER PRIMARY KEY AUTOINCREMENT,
 msg_id INTEGER,
 update_item TEXT DEFAULT '',
-update_item_read INTEGER DEFAULT 0);
+update_item_read INTEGER DEFAULT 0 -- XXX unused
+);
 CREATE INDEX msgs_status_updates_index1 ON msgs_status_updates (msg_id);"#,
            84,
        )
--- a/src/tools.rs
+++ b/src/tools.rs
@@ -323,17 +323,16 @@ pub fn get_filemeta(buf: &[u8]) -> Result<(u32, u32)> {
 ///
 /// If `path` starts with "$BLOBDIR", replaces it with the blobdir path.
 /// Otherwise, returns path as is.
-pub(crate) fn get_abs_path(context: &Context, path: impl AsRef<Path>) -> PathBuf {
-    let p: &Path = path.as_ref();
-    if let Ok(p) = p.strip_prefix("$BLOBDIR") {
+pub(crate) fn get_abs_path(context: &Context, path: &Path) -> PathBuf {
+    if let Ok(p) = path.strip_prefix("$BLOBDIR") {
        context.get_blobdir().join(p)
    } else {
-        p.into()
+        path.into()
    }
 }

 pub(crate) async fn get_filebytes(context: &Context, path: impl AsRef<Path>) -> Result<u64> {
-    let path_abs = get_abs_path(context, &path);
+    let path_abs = get_abs_path(context, path.as_ref());
    let meta = fs::metadata(&path_abs).await?;
    Ok(meta.len())
 }
@@ -377,7 +376,7 @@ pub(crate) async fn create_folder(
    context: &Context,
    path: impl AsRef<Path>,
 ) -> Result<(), io::Error> {
-    let path_abs = get_abs_path(context, &path);
+    let path_abs = get_abs_path(context, path.as_ref());
    if !path_abs.exists() {
        match fs::create_dir_all(path_abs).await {
            Ok(_) => Ok(()),
@@ -402,7 +401,7 @@ pub(crate) async fn write_file(
    path: impl AsRef<Path>,
    buf: &[u8],
 ) -> Result<(), io::Error> {
-    let path_abs = get_abs_path(context, &path);
+    let path_abs = get_abs_path(context, path.as_ref());
    fs::write(&path_abs, buf).await.map_err(|err| {
        warn!(
            context,
@@ -417,7 +416,7 @@ pub(crate) async fn write_file(

 /// Reads the file and returns its context as a byte vector.
 pub async fn read_file(context: &Context, path: impl AsRef<Path>) -> Result<Vec<u8>> {
-    let path_abs = get_abs_path(context, &path);
+    let path_abs = get_abs_path(context, path.as_ref());

    match fs::read(&path_abs).await {
        Ok(bytes) => Ok(bytes),
@@ -434,7 +433,7 @@ pub async fn read_file(context: &Context, path: impl AsRef<Path>) -> Result<Vec<
 }

 pub async fn open_file(context: &Context, path: impl AsRef<Path>) -> Result<fs::File> {
-    let path_abs = get_abs_path(context, &path);
+    let path_abs = get_abs_path(context, path.as_ref());

    match fs::File::open(&path_abs).await {
        Ok(bytes) => Ok(bytes),
@@ -450,12 +449,8 @@ pub async fn open_file(context: &Context, path: impl AsRef<Path>) -> Result<fs::
    }
 }

-pub fn open_file_std<P: AsRef<std::path::Path>>(
-    context: &Context,
-    path: P,
-) -> Result<std::fs::File> {
-    let p: PathBuf = path.as_ref().into();
-    let path_abs = get_abs_path(context, p);
+pub fn open_file_std(context: &Context, path: impl AsRef<Path>) -> Result<std::fs::File> {
+    let path_abs = get_abs_path(context, path.as_ref());

    match std::fs::File::open(path_abs) {
        Ok(bytes) => Ok(bytes),
@@ -540,6 +535,9 @@ impl EmailAddress {
                if domain.is_empty() {
                    bail!("missing domain after '@' in {:?}", input);
                }
+                if domain.ends_with('.') {
+                    bail!("Domain {domain:?} should not contain the dot in the end");
+                }
                Ok(EmailAddress {
                    local: (*local).to_string(),
                    domain: (*domain).to_string(),
@@ -1001,7 +999,7 @@ DKIM Results: Passed=true, Works=true, Allow_Keychange=true";
        assert_eq!(EmailAddress::new("dd.tt").is_ok(), false);
        assert!(EmailAddress::new("tt.dd@uu").is_ok());
        assert!(EmailAddress::new("u@d").is_ok());
-        assert!(EmailAddress::new("u@d.").is_ok());
+        assert!(EmailAddress::new("u@d.").is_err());
        assert!(EmailAddress::new("u@d.t").is_ok());
        assert_eq!(
            EmailAddress::new("u@d.tt").unwrap(),
--- a/src/webxdc.rs
+++ b/src/webxdc.rs
@@ -1,4 +1,18 @@
 //! # Handle webxdc messages.
+//!
+//! Internally status updates are stored in the `msgs_status_updates` SQL table.
+//! `msgs_status_updates` contains the following columns:
+//! - `id` - status update serial number
+//! - `msg_id` - ID of the message in the `msgs` table
+//! - `update_item` - JSON representation of the status update
+//!
+//! Status updates are scheduled for sending by adding a record
+//! to `smtp_status_updates_table` SQL table.
+//! `smtp_status_updates` contains the following columns:
+//! - `msg_id` - ID of the message in the `msgs` table
+//! - `first_serial` - serial number of the first status update to send
+//! - `last_serial` - serial number of the last status update to send
+//! - `descr` - text to send along with the updates

 use std::convert::TryFrom;
 use std::path::Path;
@@ -12,6 +26,7 @@ use serde::{Deserialize, Serialize};
 use serde_json::Value;
 use tokio::io::AsyncReadExt;

+use crate::blob::BlobObject;
 use crate::chat::Chat;
 use crate::constants::Chattype;
 use crate::contact::ContactId;
@@ -665,6 +680,10 @@ impl Context {
    ///
    /// Example: `{"updates": [{"payload":"any update data"},
    ///                        {"payload":"another update data"}]}`
+    ///
+    /// `range` is an optional range of status update serials to send.
+    /// If it is `None`, all updates are sent.
+    /// This is used when a message is resent using [`crate::chat::resend_msgs`].
    pub(crate) async fn render_webxdc_status_update_object(
        &self,
        instance_msg_id: MsgId,
@@ -707,7 +726,7 @@ fn parse_webxdc_manifest(bytes: &[u8]) -> Result<WebxdcManifest> {
    Ok(manifest)
 }

-async fn get_blob(archive: &mut async_zip::read::fs::ZipFileReader, name: &str) -> Result<Vec<u8>> {
+async fn get_blob(archive: &async_zip::read::fs::ZipFileReader, name: &str) -> Result<Vec<u8>> {
    let (i, _) = find_zip_entry(archive.file(), name)
        .ok_or_else(|| anyhow!("no entry found for {}", name))?;
    let mut reader = archive.entry(i).await?;
@@ -750,10 +769,10 @@ impl Message {
            name
        };

-        let mut archive = self.get_webxdc_archive(context).await?;
+        let archive = self.get_webxdc_archive(context).await?;

        if name == "index.html" {
-            if let Ok(bytes) = get_blob(&mut archive, "manifest.toml").await {
+            if let Ok(bytes) = get_blob(&archive, "manifest.toml").await {
                if let Ok(manifest) = parse_webxdc_manifest(&bytes) {
                    if let Some(min_api) = manifest.min_api {
                        if min_api > WEBXDC_API_VERSION {
@@ -766,15 +785,15 @@ impl Message {
            }
        }

-        get_blob(&mut archive, name).await
+        get_blob(&archive, name).await
    }

    /// Return info from manifest.toml or from fallbacks.
    pub async fn get_webxdc_info(&self, context: &Context) -> Result<WebxdcInfo> {
        ensure!(self.viewtype == Viewtype::Webxdc, "No webxdc instance.");
-        let mut archive = self.get_webxdc_archive(context).await?;
+        let archive = self.get_webxdc_archive(context).await?;

-        let mut manifest = get_blob(&mut archive, "manifest.toml")
+        let mut manifest = get_blob(&archive, "manifest.toml")
            .await
            .map(|bytes| parse_webxdc_manifest(&bytes).unwrap_or_default())
            .unwrap_or_default();
@@ -827,6 +846,35 @@ impl Message {
    }
 }

+/// Replaces WebXDC blob of existing message.
+///
+/// This API is supposed to be called from within a WebXDC to replace itself
+/// e.g. with an updated or persistently reconfigured version.
+pub async fn replace_webxdc(context: &Context, msg_id: MsgId, data: &[u8]) -> Result<()> {
+    let mut msg = Message::load_from_db(context, msg_id).await?;
+
+    ensure!(
+        msg.get_viewtype() == Viewtype::Webxdc,
+        "Message {msg_id} is not a WebXDC instance"
+    );
+
+    let blob = BlobObject::create(
+        context,
+        &msg.get_filename()
+            .context("Cannot get filename of exising WebXDC instance")?,
+        data,
+    )
+    .await
+    .context("Failed to create WebXDC replacement blob")?;
+
+    let mut param = msg.param.clone();
+    param.set(Param::File, blob.as_name());
+    msg.param = param;
+    msg.update_param(context).await?;
+
+    Ok(())
+}
+
 #[cfg(test)]
 mod tests {
    use serde_json::json;
@@ -895,10 +943,8 @@ mod tests {
    }

    async fn create_webxdc_instance(t: &TestContext, name: &str, bytes: &[u8]) -> Result<Message> {
-        let file = t.get_blobdir().join(name);
-        tokio::fs::write(&file, bytes).await?;
        let mut instance = Message::new(Viewtype::File);
-        instance.set_file(file.to_str().unwrap(), None);
+        instance.set_file_from_bytes(t, name, bytes, None).await?;
        Ok(instance)
    }

@@ -926,10 +972,10 @@ mod tests {
        assert_eq!(instance.chat_id, chat_id);

        // sending using bad extension is not working, even when setting Viewtype to webxdc
-        let file = t.get_blobdir().join("index.html");
-        tokio::fs::write(&file, b"<html>ola!</html>").await?;
        let mut instance = Message::new(Viewtype::Webxdc);
-        instance.set_file(file.to_str().unwrap(), None);
+        instance
+            .set_file_from_bytes(&t, "index.html", b"<html>ola!</html>", None)
+            .await?;
        assert!(send_msg(&t, chat_id, &mut instance).await.is_err());

        Ok(())
@@ -953,14 +999,15 @@ mod tests {
        assert_eq!(test.viewtype, Viewtype::File);

        // sending invalid .xdc as Viewtype::Webxdc should fail already on sending
-        let file = t.get_blobdir().join("invalid2.xdc");
-        tokio::fs::write(
-            &file,
-            include_bytes!("../test-data/webxdc/invalid-no-zip-but-7z.xdc"),
-        )
-        .await?;
        let mut instance = Message::new(Viewtype::Webxdc);
-        instance.set_file(file.to_str().unwrap(), None);
+        instance
+            .set_file_from_bytes(
+                &t,
+                "invalid2.xdc",
+                include_bytes!("../test-data/webxdc/invalid-no-zip-but-7z.xdc"),
+                None,
+            )
+            .await?;
        assert!(send_msg(&t, chat_id, &mut instance).await.is_err());

        Ok(())
@@ -2568,4 +2615,100 @@ sth_for_the = "future""#
        );
        Ok(())
    }
+
+    /// Tests extensibility of WebXDC updates.
+    ///
+    /// If an update sent by WebXDC contains unknown properties,
+    /// such as `aNewUnknownProperty` or a reserved property
+    /// like `serial` or `max_serial`,
+    /// they are silently dropped and are not sent over the wire.
+    ///
+    /// This ensures new WebXDC can try to send new properties
+    /// added in later revisions of the WebXDC API
+    /// and this will not result in a failure to send the whole update.
+    #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
+    async fn test_send_webxdc_status_update_extensibility() -> Result<()> {
+        let alice = TestContext::new_alice().await;
+        let bob = TestContext::new_bob().await;
+        let alice_chat = alice.create_chat(&bob).await;
+        let alice_instance = send_webxdc_instance(&alice, alice_chat.id).await?;
+
+        let bob_instance = bob.recv_msg(&alice.pop_sent_msg().await).await;
+
+        alice
+            .send_webxdc_status_update(
+                alice_instance.id,
+                r#"{"payload":"p","info":"i","aNewUnknownProperty":"x","max_serial":123}"#,
+                "Some description",
+            )
+            .await?;
+        alice.flush_status_updates().await?;
+        bob.recv_msg(&alice.pop_sent_msg().await).await;
+
+        assert_eq!(
+            bob.get_webxdc_status_updates(bob_instance.id, StatusUpdateSerial(0))
+                .await?,
+            r#"[{"payload":"p","info":"i","serial":1,"max_serial":1}]"#
+        );
+
+        Ok(())
+    }
+
+    /// Tests replacing WebXDC with a newer version.
+    ///
+    /// Updates should be preserved after upgrading.
+    #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
+    async fn test_replace_webxdc() -> Result<()> {
+        let alice = TestContext::new_alice().await;
+        let bob = TestContext::new_bob().await;
+
+        // Alice sends WebXDC instance.
+        let alice_chat = alice.create_chat(&bob).await;
+        let mut alice_instance = create_webxdc_instance(
+            &alice,
+            "minimal.xdc",
+            include_bytes!("../test-data/webxdc/minimal.xdc"),
+        )
+        .await?;
+        alice_instance.set_text("user added text".to_string());
+        send_msg(&alice, alice_chat.id, &mut alice_instance).await?;
+        let alice_instance = alice.get_last_msg().await;
+        assert_eq!(alice_instance.get_text(), "user added text");
+
+        // Bob receives that instance.
+        let alice_sent_instance = alice.pop_sent_msg().await;
+        let bob_received_instance = bob.recv_msg(&alice_sent_instance).await;
+        assert_eq!(bob_received_instance.get_text(), "user added text");
+
+        // Alice sends WebXDC update.
+        alice
+            .send_webxdc_status_update(alice_instance.id, r#"{"payload": 1}"#, "Alice update")
+            .await?;
+        alice.flush_status_updates().await?;
+        let alice_sent_update = alice.pop_sent_msg().await;
+        bob.recv_msg(&alice_sent_update).await;
+        assert_eq!(
+            bob.get_webxdc_status_updates(bob_received_instance.id, StatusUpdateSerial(0))
+                .await?,
+            r#"[{"payload":1,"serial":1,"max_serial":1}]"#
+        );
+
+        // Bob replaces WebXDC.
+        replace_webxdc(
+            &bob,
+            bob_received_instance.id,
+            include_bytes!("../test-data/webxdc/with-minimal-manifest.xdc"),
+        )
+        .await
+        .context("Failed to replace WebXDC")?;
+
+        // Updates are not modified.
+        assert_eq!(
+            bob.get_webxdc_status_updates(bob_received_instance.id, StatusUpdateSerial(0))
+                .await?,
+            r#"[{"payload":1,"serial":1,"max_serial":1}]"#
+        );
+
+        Ok(())
+    }
 }
--- a/test-data/image/logo.gif
+++ b/test-data/image/logo.gif
--- a/test-data/image/logo.png
+++ b/test-data/image/logo.png
--- a/test-data/message/jpeg-as-application-octet-stream.eml
+++ b/test-data/message/jpeg-as-application-octet-stream.eml
@@ -0,0 +1,76 @@
+X-Mozilla-Status: 0801
+X-Mozilla-Status2: 10000000
+Content-Type: multipart/mixed; boundary="------------L1v4sF5IlAZ0HirXymXElgpK"
+Message-ID: <1e3b3bb0-f34f-71e2-6b86-bce80bef2c6f@example.org>
+Date: Thu, 3 Aug 2023 13:31:01 -0300
+MIME-Version: 1.0
+User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
+ Thunderbird/102.13.0
+Content-Language: en-US
+To: bob@example.net
+From: Alice <alice@example.org>
+X-Identity-Key: id3
+Fcc: imap://alice%40example.org@in.example.org/Sent
+
+This is a multi-part message in MIME format.
+--------------L1v4sF5IlAZ0HirXymXElgpK
+Content-Type: text/plain; charset=UTF-8; format=flowed
+Content-Transfer-Encoding: 7bit
+
+
+--------------L1v4sF5IlAZ0HirXymXElgpK
+Content-Type: application/octet-stream; name="rusty_deltachat_logo.jpeg"
+Content-Disposition: attachment; filename="rusty_deltachat_logo.jpeg"
+Content-Transfer-Encoding: base64
+
+/9j/4AAQSkZJRgABAQIAzADMAAD/2wBDAP//////////////////////////////////////
+////////////////////////////////////////////////2wBDAf//////////////////
+////////////////////////////////////////////////////////////////////wAAR
+CAEAAQADASIAAhEBAxEB/8QAHwAAAQUBAQEBAQEAAAAAAAAAAAECAwQFBgcICQoL/8QAtRAA
+AgEDAwIEAwUFBAQAAAF9AQIDAAQRBRIhMUEGE1FhByJxFDKBkaEII0KxwRVS0fAkM2JyggkK
+FhcYGRolJicoKSo0NTY3ODk6Q0RFRkdISUpTVFVWV1hZWmNkZWZnaGlqc3R1dnd4eXqDhIWG
+h4iJipKTlJWWl5iZmqKjpKWmp6ipqrKztLW2t7i5usLDxMXGx8jJytLT1NXW19jZ2uHi4+Tl
+5ufo6erx8vP09fb3+Pn6/8QAHwEAAwEBAQEBAQEBAQAAAAAAAAECAwQFBgcICQoL/8QAtREA
+AgECBAQDBAcFBAQAAQJ3AAECAxEEBSExBhJBUQdhcRMiMoEIFEKRobHBCSMzUvAVYnLRChYk
+NOEl8RcYGRomJygpKjU2Nzg5OkNERUZHSElKU1RVVldYWVpjZGVmZ2hpanN0dXZ3eHl6goOE
+hYaHiImKkpOUlZaXmJmaoqOkpaanqKmqsrO0tba3uLm6wsPExcbHyMnK0tPU1dbX2Nna4uPk
+5ebn6Onq8vP09fb3+Pn6/9oADAMBAAIRAxEAPwCSiiigAooooAKKKKACiiigAooprHAzQA6i
+mhgfY0McA0AAYGnVX6U/efagA3Hdnt6VL15qvS5OMZ4oAUt82R26f596lByM1BSgkcA0APZ+
+eO3X3qSq9PV8DBoAl6UVCzZ+lPQ8fSgB9FFICD0oAWiiigAooooAKKKKACiiigAooooAKKKK
+ACiiigApCcDNLTH6UAKGB9j6UjkYx3qKigAooooAKKKKACiiigAooooAKKKKAClBI6UlFACk
+k9akTofrUVKCR0oAnoqEuT7fSnp0P1oAfRRRQAUUUUAFFFFABRRRQAUUUUAFIwyDTd496QuM
+cUAMBI6GkJJ60UUAFFFFABRSqMmpgAOlAEYQnrxTtg96fSEgdTQAmxfT+dLtX0FJvX1o3r6/
+zoAXavoKTYvp/OlBB6GloAZsHvTSh7HNS0UAQEEdRSVYqNk7j8qAI6KKKACnq2ODTKKAJS47
+CnDkA1AOtWKACiiigAooooAKKKKACimscDOM03zPb9f/AK1ADXGD9eabSk5OTSUAFFFFABRR
+RQBMowB7806kXoPpQeh+hoAjZ+w/OmUUUAFFFFABTw/r+dMooAsdaKhDEfT0qUEHkUALRRRQ
+BG69x+NR1O33T9KgoAKKKKACnBiBim0UASKSTyeKkpiYwfWn0AFFFFABRRTWbb2zQA4jPFV6
+eX46Y/GmUAFFFFABRRRQAUUUUASIe35VJVepVbPB6/zoAay45HT+VMqxTCgPTg0ARUUpBHWk
+oAKKKKAClBI6UlFAEocd+KXevr+hqGigBzNn6U2iigAooooAKKKKAAEjpTgWJxk02pExk+tA
+ElFFFABSEZGKCQOtNLjtzQBF0ooooAKKKKACiiigAooooAKKKKAJFfsfz/xqSq9OViPcUATY
+z1qMp6flTwQelLQBX6UVOQD1qMoR05/nQAyiiigAooooAKKKKACiiigAooooAKAcdKKKAHBm
+tTUxAMZ70+gBCARioSCDzU+cdahY5PHagBtFFFABRRRQAUUUUAFFFFABRRRQAUUdelPCHvx
+QA0EjpUqsD9fSkCD3NOCgdqAFooooAaVB+vrURUj/Gp6KAK9FSMncfl/hUdABRRRQAUUUUAF
+FFFABRRRQA4MR0p6tnjHNRgZOKmCgUARsp69R/KmVYqA9Tj1oASiiigAooooAKKKKACiilAJ
+4FACVIE9fypwUD6+tOoAQADpS0UhIHJoAWkLAdTUZcnpwKZQBIXHYUB+eelR0UAWKKiRux/C
+paACmMueR1/nT6KAK9FSsueR1/nUVABRRRQAUUUUAFFFFACg4OalDA+31qGpFTufyoAazE8d
+B6U2pyAetQsMEigBKKKKACiiigAooooABzxU4GBTEHf8qkoAKKKKADpUDHJ/lT3Pb8TUdABR
+RRQAUUUUAFTKcj3FQ05Tg/pQBNRRRQAVE4wc9j/OpaQjIIoAgooooAKKKKACiiigBRwQanqv
+T03Z9vegB7Nj61DUxUGmMuKAGUUUUAFFFFABRRSr1H1oAmAwAKWiigAooooAgY5JpKKKACii
+igAooooAKKKKAJwcgGlpifdp9ABRRRQBCwwx/Om09+o+lMoAKKKKACiiigCRB1qSoAcHNTA5
+GaAFqJ2zwPxqWo9h9aAI6KUjBwaSgAooooAKcv3hTaUdR9aAJ6KKKACkPQ/Q0tFAFeiiigAo
+oooAKKKKACiiigCVOh+v9BT6an3adQAUUUUARP1H0plOc5Y/lTaACiiigApVGSBSUdOaAJ8D
+0FLSA5GaWgAoooPTjrQBE5yfpTKfsb2ppBBwaAEooooAKKKKAJ1OQDS1Ehwcev8AOpaACiii
+gCAjBP1pKe45z60ygAooooAKKKKACiinIMn2FAEoGABS0UUAFITgZpaidsnHYUAMooooAKKK
+KACpti+n6moalVs8Hr/OgBwGOBS0UUAFFFFABUb9hUlRlCSTkUAR0U8oQM9aZQAUUUUAFSq2
+eD1/nUVFAFiimK2eD1/nT6AEIyMVCQQcGp6QgHrQBBRTyh7c03afQ0AJRS7T6GnBD34oAaAS
+cCpgMDFAAHSloAKKKjZ+w/P/AAoAGbsPxqOiigAooooAKKKKAFUZOKlCgf41EDg5FSBwevFA
+D6KKKACiiigAooooAQnAJqCpyMjFIEUds/WgCGinuoHI/KmUAFFFFABTw5HXn+dMooAnBB6G
+lqvTgxHegCaiot59BS+Z7frQBJRUfme1JvPsKAJaaXA96iJJ6mkoAcWJ+npTaKKACiiigAoo
+ooAKKKKACpVXHJ61FUocYGetAD6KQEHoaWgAooooAKKKKACiiigBj9B9aaEJ68VLRQBAQR1p
+KlfoPrTApNADaKUgjrSUAFFFFABRRRQAUUUUAFFFFABRRSkEdRQAlFKBkgVKUGOOKAIaKXBz
+ipto9BQBBRQRg4p+w4z39KAGUUUUASouOe5/lT6QHIzS0AFFFFABRRRQAUUUUAFFFFACEA9a
+WiigBr/d+lRqufpU1FAEbJgZFR1YqNU557dKAG7W9KbVioip3YHegBoBPQUlTgYGKay55HX+
+dADFXdSshHPWnqMD3p1AESdfwqWkAA6d6WgBgQA5H5U+iigBMDOe9LRRQAm0ZzS0UUARFTu9
+jzmpNoxjFLRQAgGOlLRRQB//2Q==
+
+--------------L1v4sF5IlAZ0HirXymXElgpK--
--- a/test-data/message/mixed-up-long.eml
+++ b/test-data/message/mixed-up-long.eml
Author	SHA1	Message	Date
link2xt	8b06d31190	api: add dc_replace_webxdc()	2023-09-22 10:44:30 +00:00
link2xt	e72d527d88	api: make it possible to import secret key from a file Previously it was required that a directory path is provided to the import API. Now it is possible to point directly to the .asc file containing a secret key. This allows UI to present a file selection dialog to the user and let select the file directly. Selecting a directory is still supported for backwards compatibility.	2023-09-20 16:18:45 +00:00
link2xt	f5c36043f6	fix(imex): use "default" in the filename of the default key Previously the logic was inverted, default key was exported with a number and all other keys were exported into "default".	2023-09-20 16:18:45 +00:00
iequidoo	b227ff87dc	fix: lookup_chat_by_reply(): Skip undecipherable parent messages created by older versions (#4676 ) It's just a dirty hack checking the error prefix, but as undecipherable messages are remembered in the db now, the hack may be removed soon.	2023-09-18 14:55:56 +00:00
iequidoo	676f311f97	fix: lookup_chat_by_reply(): Skip not fully downloaded and undecipherable messages (#4676 ) Such a message may be assigned to a wrong chat (e.g. undecipherable group msgs often get assigned to the 1:1 chat with the sender). Add `DownloadState::Undecipherable` so that messages referencing undecipherable ones don't go to that wrong chat too. Also do not reply to not fully downloaded messages. Before `Message.error` was checked for that purpose, but a message can be error for many reasons.	2023-09-18 14:55:56 +00:00
link2xt	f02299c06c	chore(release): prepare for 1.122.0	2023-09-12 17:33:22 +00:00
link2xt	ed781af52c	chore(cargo): update to OpenSSL 3.0 OpenSSL 1.1.1 has reached End of Life: https://www.openssl.org/blog/blog/2023/09/11/eol-111/	2023-09-12 17:11:58 +00:00
link2xt	67043177a9	fix: reopen all connections on database passpharse change Previously only one connection, the one used to change the key, was working after passphrase change. With this fix the whole pool of connections is recreated on passphrase change, so there is no need to reopen the database manually.	2023-09-12 16:34:26 +00:00
link2xt	49cc5fb673	feat: add RSA-4096 key generation support	2023-09-12 12:33:34 +00:00
link2xt	68c95dee17	refactor(pgp): add constants for encryption algorithm and hash These constants are current defaults in `pgp` crate, this change would prevent accidental change due to rPGP upgrade and make it easier to change in a single place.	2023-09-12 11:12:59 +02:00
iequidoo	9bd7ab7280	fix: apply_group_changes(): Forbid membership changes from possible non-members (#3782 ) It can be not good for membership consistency if we missed a message adding a member, but improves security because nobody can add themselves to a group from now on.	2023-09-12 00:30:02 -03:00
link2xt	7a359f6318	build(python): add link to mastodon into projects.urls Such links are displayed on PyPI with mastodon icon.	2023-09-11 04:47:55 +00:00
link2xt	38b31aa88d	fix: do not block new group chats if 1:1 chat is blocked 1:1 chat may be blocked while the contact is not if 1:1 chat was created as a result of scanning a verified group join QR code with the contact as the inviter. In this case 1:1 chat is blocked to hide it while the contact is unblocked.	2023-09-10 21:44:56 +00:00
iequidoo	e12e026bd8	fix: Switch to original Hocuri's group membership consistency algo (#3782 )(#4624 ) - If we don't know the parent (=In-Reply-To) message, then completely recreate the group member list (i.e. use the member list of the incoming message) (because we assume that we missed some messages & have a wrong group state). - If the message has a "Chat-Group-Member-Removed: member@example.com" header, then remove this member. - If the message has a "Chat-Group-Member-Added: member@example.com" header, then add this member. That means: - Remove checks for the presense of `ContactId::SELF` in the group. Thus all recipients of a message take the same decision about group membership changes, no matter if they are in the group currently. This fixes a situation when a recipient thinks it's not a member because it missed a message about its addition before. NOTE: But always recreate membership list if SELF has been added. The older versions of DC don't always set "In-Reply-To" to the latest message they sent, but to the latest delivered message (so it's a race), so we need this heuristic currently. - Recreate the group member list if we don't know the parent (=In-Reply-To) message, even if the sender isn't in the group as per our view, because we missed some messages and our view may be stale.	2023-09-09 16:22:13 -03:00
iequidoo	212fbc125c	fix: ChatId::parent_query(): Don't filter out OutPending and OutFailed messages The new message for which `parent_query()` is done may assume that it will be received in a context affected by those messages, e.g. they could add new members to a group and the new message will contain them in "To:". Anyway recipients must be prepared to orphaned references.	2023-09-09 16:22:13 -03:00
link2xt	2939de013b	api(jsonrpc): return only chat IDs for similar chats This is already the way `get_chatlist_entries` works. `get_similar_chatlist_entries` is renamed into `get_similar_chat_ids` because return values are not entries anymore.	2023-09-08 18:47:31 +00:00
link2xt	0562e23ee0	chore(cargo): bump webpki from 0.22.0 to 0.22.1	2023-09-08 07:00:18 +00:00
link2xt	a8551510cd	chore(release): prepare for 1.121.0	2023-09-06 20:53:00 +00:00
link2xt	087f6edd0c	refactor: make set_msg_failed accept &mut Message instead of MsgId	2023-09-06 20:33:37 +00:00
link2xt	d6b7ee04a0	docs: document `range` argument of `render_webxdc_status_update_object()`	2023-09-06 20:33:37 +00:00
link2xt	d5c5ff8b3f	refactor: accept &mut Message in create_send_msg_job()	2023-09-06 20:33:37 +00:00
link2xt	dc4396a699	fix: update passed by reference Message in prepare_msg_raw()	2023-09-06 20:33:37 +00:00
link2xt	a74b00c3f9	refactor: move try_calc_and_set_dimensions() call to prepare_msg_blob()	2023-09-06 20:33:37 +00:00
link2xt	2fdb9f8b7e	fix: do not ignore errors in `try_calc_and_set_dimensions`	2023-09-06 20:33:37 +00:00
link2xt	80fac3f1b8	refactor: use let-else in resend_msgs()	2023-09-06 20:33:37 +00:00
link2xt	17a6c88cc7	api: add Message.set_file_from_bytes() API	2023-09-06 20:33:37 +00:00
link2xt	1ba69dbb9b	fix: reset MIME type if passed to set_file value is `None`	2023-09-06 20:33:37 +00:00
link2xt	ab1c7ebbe2	refactor: remove unnecessary local variable `save_param`	2023-09-06 20:33:37 +00:00
link2xt	ee715da078	fix: do not ignore chat loading errors in forward_msgs()	2023-09-06 20:33:37 +00:00
link2xt	27e177dc05	refactor: resultify set_msg_failed()	2023-09-06 20:33:37 +00:00
link2xt	7aac4bfc83	docs: document WebXDC-related SQL tables	2023-09-06 20:33:37 +00:00
link2xt	7b24f9b7a4	docs: comment that msgs_status_updates.update_item_read column is unused	2023-09-06 20:33:37 +00:00
link2xt	b36b902eeb	build: build node packages on Ubuntu 18.04 Debian 10 has glibc 2.28, so packages built with it do not work on Ubuntu 18.04.	2023-09-06 17:23:04 +00:00
link2xt	30024abb6c	feat: add API to get similar chats	2023-09-05 16:47:19 +00:00
link2xt	1d9702e9e7	refactor: add ChatId::get_timestamp()	2023-09-05 16:47:19 +00:00
link2xt	ee2eae63d6	fix: do not allow dots at the end of email addresses	2023-09-05 13:14:37 +00:00
link2xt	cd477936b5	api: add dc_context_change_passphrase()	2023-09-05 12:41:31 +00:00
link2xt	dbe9d7e34e	refactor(provider): create provider database entries at compile time According to cargo-llvm-lines it reduces the number of lines in the crate by 0.7%.	2023-09-05 00:03:34 +00:00
link2xt	49f143e0d5	refactor: make get_abs_path non-generic Generic functions compile into multiple implementations, increasing number of lines for LLVM to process, code size and compilation times.	2023-09-04 00:43:24 +00:00
link2xt	9d7bdf369d	refactor: use slices instead of vectors in provider database	2023-09-03 20:19:58 +00:00
adbenitez	a270db1d87	remove aiodns optional dependency from required dependencies	2023-09-03 00:32:08 +00:00
iequidoo	7c7cd9cc80	fix: Return from dc_get_chatlist(DC_GCL_FOR_FORWARDING) only chats where we can send (#4616 ) I.e. exclude from the list the following chats as well: - Read-only mailing lists. - Chats we're not a member of. But as for ProtectionBroken chats, we return them, as that may happen to a verified chat at any time. It may be confusing if a chat that is normally in the list disappears suddenly. The UI need to deal with that case anyway. (cherry picked from commit `83ef25e7de`)	2023-09-01 12:28:41 -03:00
iequidoo	47d465e6e4	fix: Save mime headers for messages not signed with a known key (#4557 ) If a message is unsigned or signed with an unknown key, `MimeMessage::was_encrypted()` returns false. So, it mustn't be checked when deciding whether to look into `MimeMessage::decoded_data`. Looking through git history one can see that it's just a wrong check left in the code for historical reasons.	2023-08-31 14:07:12 -03:00
link2xt	03d3e0578f	refactor(deltachat-rpc-client): drop support for keyword arguments All Rust methods have positional arguments and it is not going to change.	2023-08-31 00:46:04 +00:00
iequidoo	440a442f30	fix: Allow membership changes by a MUA if we're not in the group (#4624 ) Other MUAs don't set add/remove headers, so the only way for them to re-add us to the group is to add us to To/CC/wherever. Previously it worked only for other members that are still in the group so that they properly handled our re-addition, but we didn't.	2023-08-29 22:19:15 -03:00
link2xt	1da52d7d1d	refactor: use split_once() instead of regexp in `heuristically_parse_ndn` Reduce dependency on the huge `regex` crate.	2023-08-29 18:41:07 +00:00
link2xt	4d74f625d3	chore(release): prepare for 1.120.0	2023-08-28 11:54:38 +00:00
link2xt	0a94fbc735	fix: update async-imap to 0.9.1 to fix memory leak	2023-08-28 09:04:28 +00:00
link2xt	9ef34890fa	chore: fix beta clippy warnings	2023-08-28 04:09:52 +00:00
iequidoo	3e07f2c173	fix: prepare_msg_blob(): If cannot recode image, don't use it if it has Exif We mustn't send images with Exif as it can leak metadata such as location, camera model, etc.	2023-08-27 23:16:19 -03:00
iequidoo	ee28298d7f	fix: W/a sending images sent as stickers on some platforms (#4611 ) Check if a sticker has at least one fully transparent corner and otherwise change the Sticker type to Image. This would fix both Android and iOS at the same time and prevent similar bug on future platforms that may get this bug like Ubuntu Touch.	2023-08-27 23:16:19 -03:00
link2xt	62aed13880	refactor: hide pgp module from public API	2023-08-27 22:03:00 +00:00
link2xt	bffe934acc	refactor: hide accounts.rs constants from public API	2023-08-27 22:03:00 +00:00
link2xt	87ffcaf03e	build: update to Rust 1.72.0	2023-08-25 23:04:47 +00:00
link2xt	2635146328	build: update to Zig 0.11.0	2023-08-25 21:20:11 +00:00
link2xt	d727d85f6d	chore(python): fix `ruff` 0.0.286 warnings	2023-08-25 20:57:44 +00:00
link2xt	81a7af10c7	chore(python): fix lint errors	2023-08-25 01:14:24 +00:00
link2xt	4a6e94f8ab	build(deny): ignore RUSTSEC-2023-0052	2023-08-25 01:04:02 +00:00
link2xt	146fe50e20	build(cargo-deny): ignore RUSTSEC-2022-0093 It is an API issue that can only be fixed in rPGP and iroh upstream.	2023-08-25 01:03:56 +00:00
link2xt	9bf2850fb1	ci: run on push to stable branch	2023-08-25 00:58:14 +00:00
iequidoo	ba2c36548e	fix: Delete messages from SMTP queue only on user demand (#4579 ) I.e. from delete_msgs(). Otherwise messages must not be deleted from there, e.g. if a message is ephemeral, but a network outage lasts longer than the ephemeral message timer, the message still must be sent upon a successful reconnection.	2023-08-24 23:24:24 +00:00
Simon Laux	d07c743cdc	api(jsonrpc): add resend_messages	2023-08-24 22:04:47 +00:00
link2xt	d70c1d48b5	chore(release) prepare for 1.119.1	2023-08-06 16:49:06 +00:00
link2xt	a8e0cb9b5a	fix: update `xattr` from 1.0.0 to 1.0.1 to fix UnsupportedPlatformError import See the folowing PRs on the `xattr` repository: - https://github.com/Stebalien/xattr/pull/34 - https://github.com/Stebalien/xattr/pull/38	2023-08-06 16:45:58 +00:00
link2xt	6ea9a8988b	test(webxdc): ensure unknown WebXDC update properties do not result in an error	2023-08-06 16:27:29 +00:00
iequidoo	45e35b3571	feat: Guess message viewtype from "application/octet-stream" attachment extension (#4378 )	2023-08-05 16:34:06 -03:00
@@ -1 +1 @@
 -08-03
 -09-12