happy clippy

we need to let the provider know that we want to support iroh collections

.github/workflows/ci.yml

@@ -15,7 +15,6 @@ on:
   push:
     branches:
       - master
-      - stable
 
 env:
   RUSTFLAGS: -Dwarnings
@@ -25,7 +24,7 @@ jobs:
     name: Lint Rust
     runs-on: ubuntu-latest
     env:
-      RUSTUP_TOOLCHAIN: 1.72.0
+      RUSTUP_TOOLCHAIN: 1.71.0
     steps:
       - uses: actions/checkout@v3
       - name: Install rustfmt and clippy
@@ -81,19 +80,15 @@ jobs:
       matrix:
         include:
           - os: ubuntu-latest
-            rust: 1.68.2
+            rust: 1.71.0
           - os: windows-latest
-            rust: 1.68.2
+            rust: 1.71.0
           - os: macos-latest
-            rust: 1.68.2
+            rust: 1.71.0
 
-          # Minimum Supported Rust Version = 1.65.0
-          #
-          # Minimum Supported Python Version = 3.7
-          # This is the minimum version for which manylinux Python wheels are
-          # built.
+          # Minimum Supported Rust Version = 1.66.0
           - os: ubuntu-latest
-            rust: 1.65.0
+            rust: 1.66.0
     runs-on: ${{ matrix.os }}
     steps:
       - uses: actions/checkout@v3

.github/workflows/node-package.yml

@@ -67,9 +67,7 @@ jobs:
 
     # Build Linux prebuilds inside a container with old glibc for backwards compatibility.
     # Debian 10 contained glibc 2.28 at the time of the writing (2023-06-04): https://packages.debian.org/buster/libc6
-    # Ubuntu 18.04 is at the End of Standard Support since June 2023, but it contains glibc 2.27,
-    # so we are using it to support Ubuntu 18.04 setups that are still not upgraded.
-    container: ubuntu:18.04
+    container: debian:10
     steps:
       # Working directory is owned by 1001:1001 by default.
       # Change it to our user.

CHANGELOG.md

@@ -1,144 +1,5 @@
 # Changelog
 
-## [1.122.0] - 2023-09-12
-
-### API-Changes
-
-- jsonrpc: Return only chat IDs for similar chats.
-
-### Fixes
-
-- Reopen all connections on database passpharse change.
-- Do not block new group chats if 1:1 chat is blocked.
-- Improve group membership consistency algorithm ([#3782](https://github.com/deltachat/deltachat-core-rust/pull/3782))([#4624](https://github.com/deltachat/deltachat-core-rust/pull/4624)).
-- Forbid membership changes from possible non-members ([#3782](https://github.com/deltachat/deltachat-core-rust/pull/3782)).
-- `ChatId::parent_query()`: Don't filter out OutPending and OutFailed messages.
-
-### Build system
-
-- Update to OpenSSL 3.0.
-- Bump webpki from 0.22.0 to 0.22.1.
-- python: Add link to Mastodon into projects.urls.
-
-### Features / Changes
-
-- Add RSA-4096 key generation support.
-
-### Refactor
-
-- pgp: Add constants for encryption algorithm and hash.
-
-## [1.121.0] - 2023-09-06
-
-### API-Changes
-
-- Add `dc_context_change_passphrase()`.
-- Add `Message.set_file_from_bytes()` API.
-- Add experimental API to get similar chats.
-
-### Build system
-
-- Build node packages on Ubuntu 18.04 instead of Debian 10.
-  This reduces the requirement for glibc version from 2.28 to 2.27.
-
-### Fixes
-
-- Allow membership changes by a MUA if we're not in the group ([#4624](https://github.com/deltachat/deltachat-core-rust/pull/4624)).
-- Save mime headers for messages not signed with a known key ([#4557](https://github.com/deltachat/deltachat-core-rust/pull/4557)).
-- Return from `dc_get_chatlist(DC_GCL_FOR_FORWARDING)` only chats where we can send ([#4616](https://github.com/deltachat/deltachat-core-rust/pull/4616)).
-- Do not allow dots at the end of email addresses.
-- deltachat-rpc-client: Remove `aiodns` optional dependency from required dependencies.
-  `aiodns` depends on `pycares` which [fails to install in Termux](https://github.com/saghul/aiodns/issues/98).
-
-## [1.120.0] - 2023-08-28
-
-### API-Changes
-
-- jsonrpc: Add `resend_messages`.
-
-### Fixes
-
-- Update async-imap to 0.9.1 to fix memory leak.
-- Delete messages from SMTP queue only on user demand ([#4579](https://github.com/deltachat/deltachat-core-rust/pull/4579)).
-- Do not send images without transparency as stickers ([#4611](https://github.com/deltachat/deltachat-core-rust/pull/4611)).
-- `prepare_msg_blob()`: do not use the image if it has Exif metadata but the image cannot be recoded.
-
-### Refactor
-
-- Hide accounts.rs constants from public API.
-- Hide pgp module from public API.
-
-### Build system
-
-- Update to Zig 0.11.0.
-- Update to Rust 1.72.0.
-
-### CI
-
-- Run on push to stable branch.
-
-### Miscellaneous Tasks
-
-- python: Fix lint errors.
-- python: Fix `ruff` 0.0.286 warnings.
-- Fix beta clippy warnings.
-
-## [1.119.1] - 2023-08-06
-
-Bugfix release attempting to fix the [iOS build error](https://github.com/deltachat/deltachat-core-rust/issues/4610).
-
-### Features / Changes
-
-- Guess message viewtype from "application/octet-stream" attachment extension ([#4378](https://github.com/deltachat/deltachat-core-rust/pull/4378)).
-
-### Fixes
-
-- Update `xattr` from 1.0.0 to 1.0.1 to fix UnsupportedPlatformError import.
-
-### Tests
-
-- webxdc: Ensure unknown WebXDC update properties do not result in an error.
-
-## [1.119.0] - 2023-08-03
-
-### Fixes
-
-- imap: Avoid IMAP move loops when DeltaChat folder is aliased.
-- imap: Do not resync IMAP after initial configuration.
-
-- webxdc: Accept WebXDC updates in mailing lists.
-- webxdc: Base64-encode WebXDC updates to prevent corruption of large unencrypted WebXDC updates.
-- webxdc: Delete old webxdc status updates during housekeeping.
-
-- Return valid MsgId from `receive_imf()` when the message is replaced.
-- Emit MsgsChanged event with correct chat id for replaced messages.
-
-- deltachat-rpc-server: Update tokio-tar to fix backup import.
-
-### Features / Changes
-
-- deltachat-rpc-client: Add `MSG_DELETED` constant.
-- Make `dc_msg_get_filename()` return the original attachment filename ([#4309](https://github.com/deltachat/deltachat-core-rust/pull/4309)).
-
-### API-Changes
-
-- deltachat-rpc-client: Add `Account.{import,export}_backup` methods.
-- deltachat-jsonrpc: Make `MessageObject.text` non-optional.
-
-### Documentation
-
-- Update default value for `show_emails` in `dc_set_config()` documentation.
-
-### Refactor
-
-- Improve IMAP logs.
-
-### Tests
-
-- Add basic import/export test for async python.
-- Add `test_webxdc_download_on_demand`.
-- Add tests for deletion of webxdc status-updates.
-
 ## [1.118.0] - 2023-07-07
 
 ### API-Changes
@@ -2811,8 +2672,3 @@ https://github.com/deltachat/deltachat-core-rust/pulls?q=is%3Apr+is%3Aclosed
 [1.116.0]: https://github.com/deltachat/deltachat-core-rust/compare/v1.115.0...v1.116.0
 [1.117.0]: https://github.com/deltachat/deltachat-core-rust/compare/v1.116.0...v1.117.0
 [1.118.0]: https://github.com/deltachat/deltachat-core-rust/compare/v1.117.0...v1.118.0
-[1.119.0]: https://github.com/deltachat/deltachat-core-rust/compare/v1.118.0...v1.119.0
-[1.119.1]: https://github.com/deltachat/deltachat-core-rust/compare/v1.119.0...v1.119.1
-[1.120.0]: https://github.com/deltachat/deltachat-core-rust/compare/v1.119.1...v1.120.0
-[1.121.0]: https://github.com/deltachat/deltachat-core-rust/compare/v1.120.0...v1.121.0
-[1.122.0]: https://github.com/deltachat/deltachat-core-rust/compare/v1.121.0...v1.122.0

CONTRIBUTING.md

@@ -76,6 +76,29 @@ If you have multiple changes in one PR, create multiple conventional commits, an
 [Conventional Commits]: https://www.conventionalcommits.org/
 [git-cliff]: https://git-cliff.org/
 
+### Errors
+
+Delta Chat core mostly uses [`anyhow`](https://docs.rs/anyhow/) errors.
+When using [`Context`](https://docs.rs/anyhow/latest/anyhow/trait.Context.html),
+capitalize it but do not add a full stop as the contexts will be separated by `:`.
+For example:
+```
+.with_context(|| format!("Unable to trash message {msg_id}"))
+```
+
+### Logging
+
+For logging, use `info!`, `warn!` and `error!` macros.
+Log messages should be capitalized and have a full stop in the end. For example:
+```
+info!(context, "Ignoring addition of {added_addr:?} to {chat_id}.");
+```
+
+Format anyhow errors with `{:#}` to print all the contexts like this:
+```
+error!(context, "Failed to set selfavatar timestamp: {err:#}.");
+```
+
 ### Reviewing
 
 Once a PR has an approval and passes CI, it can be merged.

Cargo.toml

@@ -1,9 +1,9 @@
 [package]
 name = "deltachat"
-version = "1.122.0"
+version = "1.118.0"
 edition = "2021"
 license = "MPL-2.0"
-rust-version = "1.65"
+rust-version = "1.66"
 
 [profile.dev]
 debug = 0
@@ -25,10 +25,6 @@ panic = 'abort'
 opt-level = "z"
 codegen-units = 1
 
-[patch.crates-io]
-quinn-udp = { git = "https://github.com/quinn-rs/quinn", branch="main" }
-quinn-proto = { git = "https://github.com/quinn-rs/quinn", branch="main" }
-
 [dependencies]
 deltachat_derive = { path = "./deltachat_derive" }
 format-flowed = { path = "./format-flowed" }
@@ -36,13 +32,14 @@ ratelimit = { path = "./deltachat-ratelimit" }
 
 anyhow = "1"
 async-channel = "1.8.0"
-async-imap = { version = "0.9.1", default-features = false, features = ["runtime-tokio"] }
+async-imap = { version = "0.9.0", default-features = false, features = ["runtime-tokio"] }
 async-native-tls = { version = "0.5", default-features = false, features = ["runtime-tokio"] }
 async-smtp = { version = "0.9", default-features = false, features = ["runtime-tokio"] }
 async_zip = { version = "0.0.12", default-features = false, features = ["deflate", "fs"] }
 backtrace = "0.3"
 base64 = "0.21"
 brotli = { version = "3.3", default-features=false, features = ["std"] }
+bytes = "1"
 chrono = { version = "0.4", default-features=false, features = ["clock", "std"] }
 email = { git = "https://github.com/deltachat/rust-email", branch = "master" }
 encoded-words = { git = "https://github.com/async-email/encoded-words", branch = "master" }
@@ -53,7 +50,8 @@ futures-lite = "1.13.0"
 hex = "0.4.0"
 humansize = "2"
 image = { version = "0.24.6", default-features=false, features = ["gif", "jpeg", "ico", "png", "pnm", "webp", "bmp"] }
-iroh = { version = "0.4.1", default-features = false }
+iroh = { version = "0.5.1", default-features = false, features = ["iroh-collection", "flat-db"] }
+iroh-io = "0.2.1"
 kamadak-exif = "0.5"
 lettre_email = { git = "https://github.com/deltachat/lettre", branch = "master" }
 libc = "0.2"
@@ -65,7 +63,7 @@ num-traits = "0.2"
 once_cell = "1.18.0"
 percent-encoding = "2.3"
 parking_lot = "0.12"
-pgp = { version = "0.10", default-features = false }
+pgp = { version = "0.10.2", default-features = false }
 pretty_env_logger = { version = "0.5", optional = true }
 qrcodegen = "1.7.0"
 quick-xml = "0.29"
@@ -118,11 +116,6 @@ members = [
   "format-flowed",
 ]
 
-[[example]]
-name = "simple"
-path = "examples/simple.rs"
-
-
 [[bench]]
 name = "create_account"
 harness = false

deltachat-ffi/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "deltachat_ffi"
-version = "1.122.0"
+version = "1.118.0"
 description = "Deltachat FFI"
 edition = "2018"
 readme = "README.md"

deltachat-ffi/Doxyfile

@@ -846,7 +846,7 @@ EXCLUDE_PATTERNS       =
 # exclude all test directories use the pattern */test/*
 
 ######################################################
-EXCLUDE_SYMBOLS        = dc_aheader_t dc_apeerstate_t dc_e2ee_helper_t dc_imap_t dc_job*_t dc_key_t dc_keyring_t dc_loginparam_t dc_mime*_t
+EXCLUDE_SYMBOLS        = dc_aheader_t dc_apeerstate_t dc_e2ee_helper_t dc_imap_t dc_job*_t dc_key_t dc_loginparam_t dc_mime*_t
 EXCLUDE_SYMBOLS       += dc_saxparser_t dc_simplify_t dc_smtp_t dc_sqlite3_t dc_strbuilder_t dc_param_t dc_hash_t dc_hashelem_t
 EXCLUDE_SYMBOLS       += _dc_* jsmn*
 ######################################################

deltachat-ffi/deltachat.h

@@ -301,19 +301,6 @@ dc_context_t*   dc_context_new_closed        (const char* dbfile);
 int             dc_context_open              (dc_context_t *context, const char* passphrase);
 
 
-/**
- * Changes the passphrase on the open database.
- * Existing database must already be encrypted and the passphrase cannot be NULL or empty.
- * It is impossible to encrypt unencrypted database with this method and vice versa.
- *
- * @memberof dc_context_t
- * @param context The context object.
- * @param passphrase The new passphrase.
- * @return 1 on success, 0 on error.
- */
-int             dc_context_change_passphrase (dc_context_t* context, const char* passphrase);
-
-
 /**
  * Returns 1 if database is open.
  *
@@ -443,9 +430,7 @@ char*           dc_get_blobdir               (const dc_context_t* context);
  *                    DC_KEY_GEN_RSA2048 (1)=
  *                    generate RSA 2048 keypair
  *                    DC_KEY_GEN_ED25519 (2)=
- *                    generate Curve25519 keypair
- *                    DC_KEY_GEN_RSA4096 (3)=
- *                    generate RSA 4096 keypair
+ *                    generate Ed25519 keypair
  * - `save_mime_headers` = 1=save mime headers
  *                    and make dc_get_mime_headers() work for subsequent calls,
  *                    0=do not save mime headers (default)
@@ -500,6 +485,13 @@ char*           dc_get_blobdir               (const dc_context_t* context);
  *                    to not mess up with non-delivery-reports or read-receipts.
  *                    0=no limit (default).
  *                    Changes affect future messages only.
+ * - `verified_one_on_one_chats` = Feature flag for verified 1:1 chats; the UI should set it
+ *                    to 1 if it supports verified 1:1 chats.
+ *                    Regardless of this setting, `dc_chat_is_protected()` returns true while the key is verified,
+ *                    and when the key changes, an info message is posted into the chat.
+ *                    0=Nothing else happens when the key changes.
+ *                    1=After the key changed, `dc_chat_can_send()` returns false and `dc_chat_is_protection_broken()` returns true
+ *                    until `dc_accept_chat()` is called.
  * - `ui.*`         = All keys prefixed by `ui.` can be used by the user-interfaces for system-specific purposes.
  *                    The prefix should be followed by the system and maybe subsystem,
  *                    e.g. `ui.desktop.foo`, `ui.desktop.linux.bar`, `ui.android.foo`, `ui.dc40.bar`, `ui.bot.simplebot.baz`.
@@ -1175,24 +1167,6 @@ int dc_send_webxdc_status_update (dc_context_t* context, uint32_t msg_id, const
  */
 char* dc_get_webxdc_status_updates (dc_context_t* context, uint32_t msg_id, uint32_t serial);
 
-
-/**
- * Replaces webxdc app with a new version.
- *
- * On the JavaScript side this API could be used like this:
- * ```
- * window.webxdc.replaceWebxdc(blob);
- * ```
- *
- * @memberof dc_context_t
- * @param context The context object.
- * @param msg_id The ID of the WebXDC message to be replaced.
- * @param blob New blob to replace WebXDC with.
- * @param n Blob size.
- */
-void dc_replace_webxdc(dc_context_t* context, uint32_t msg_id, uint8_t *blob, size_t n);
-
-
 /**
  * Save a draft for a chat in the database.
  *
@@ -1354,20 +1328,6 @@ int             dc_get_msg_cnt               (dc_context_t* context, uint32_t ch
 int             dc_get_fresh_msg_cnt         (dc_context_t* context, uint32_t chat_id);
 
 
-/**
- * Returns a list of similar chats.
- *
- * @warning This is an experimental API which may change or be removed in the future.
- *
- * @memberof dc_context_t
- * @param context The context object as returned from dc_context_new().
- * @param chat_id The ID of the chat for which to find similar chats.
- * @return The list of similar chats.
- *     On errors, NULL is returned.
- *     Must be freed using dc_chatlist_unref() when no longer used.
- */
-dc_chatlist_t*     dc_get_similar_chatlist   (dc_context_t* context, uint32_t chat_id);
-
 
 /**
  * Estimate the number of messages that will be deleted
@@ -1517,24 +1477,6 @@ dc_array_t*     dc_get_chat_media            (dc_context_t* context, uint32_t ch
 uint32_t        dc_get_next_media            (dc_context_t* context, uint32_t msg_id, int dir, int msg_type, int msg_type2, int msg_type3);
 
 
-/**
- * Enable or disable protection against active attacks.
- * To enable protection, it is needed that all members are verified;
- * if this condition is met, end-to-end-encryption is always enabled
- * and only the verified keys are used.
- *
- * Sends out #DC_EVENT_CHAT_MODIFIED on changes
- * and #DC_EVENT_MSGS_CHANGED if a status message was sent.
- *
- * @memberof dc_context_t
- * @param context The context object as returned from dc_context_new().
- * @param chat_id The ID of the chat to change the protection for.
- * @param protect 1=protect chat, 0=unprotect chat
- * @return 1=success, 0=error, e.g. some members may be unverified
- */
-int             dc_set_chat_protection       (dc_context_t* context, uint32_t chat_id, int protect);
-
-
 /**
  * Set chat visibility to pinned, archived or normal.
  *
@@ -2302,7 +2244,6 @@ dc_contact_t*   dc_get_contact               (dc_context_t* context, uint32_t co
  *
  * - **DC_IMEX_IMPORT_SELF_KEYS** (2) - Import private keys found in the directory given as `param1`.
  *   The last imported key is made the default keys unless its name contains the string `legacy`. Public keys are not imported.
- *   If `param1` is a filename, import the private key from the file and make it the default.
  *
  * While dc_imex() returns immediately, the started job may take a while,
  * you can stop it using dc_stop_ongoing_process(). During execution of the job,
@@ -3760,7 +3701,6 @@ int             dc_chat_can_send              (const dc_chat_t* chat);
  * Check if a chat is protected.
  * Protected chats contain only verified members and encryption is always enabled.
  * Protected chats are created using dc_create_group_chat() by setting the 'protect' parameter to 1.
- * The status can be changed using dc_set_chat_protection().
  *
  * @memberof dc_chat_t
  * @param chat The chat object.
@@ -3769,6 +3709,26 @@ int             dc_chat_can_send              (const dc_chat_t* chat);
 int             dc_chat_is_protected         (const dc_chat_t* chat);
 
 
+/**
+ * Checks if the chat was protected, and then an incoming message broke this protection.
+ *
+ * This function is only useful if the UI enabled the `verified_one_on_one_chats` feature flag,
+ * otherwise it will return false for all chats.
+ *
+ * 1:1 chats are automatically set as protected when a contact is verified.
+ * When a message comes in that is not encrypted / signed correctly,
+ * the chat is automatically set as unprotected again.
+ * dc_chat_is_protection_broken() will return true until dc_accept_chat() is called.
+ *
+ * The UI should let the user confirm that this is OK with a message like
+ * `Bob sent a message from another device. Tap to learn more` and then call dc_accept_chat().
+ * @memberof dc_chat_t
+ * @param chat The chat object.
+ * @return 1=chat protection broken, 0=otherwise.
+ */
+int             dc_chat_is_protection_broken (const dc_chat_t* chat);
+
+
 /**
  * Check if locations are sent to the chat
  * at the time the object was created using dc_get_chat().
@@ -4026,17 +3986,16 @@ char*           dc_msg_get_text               (const dc_msg_t* msg);
  */
 char*           dc_msg_get_subject            (const dc_msg_t* msg);
 
-
 /**
- * Find out full path of the file associated with a message.
+ * Find out full path, file name and extension of the file associated with a
+ * message.
  *
  * Typically files are associated with images, videos, audios, documents.
  * Plain text messages do not have a file.
- * File name may be mangled. To obtain the original attachment filename use dc_msg_get_filename().
  *
  * @memberof dc_msg_t
  * @param msg The message object.
- * @return The full path (with file name and extension) of the file associated with the message.
+ * @return The full path, the file name, and the extension of the file associated with the message.
  *     If there is no file associated with the message, an empty string is returned.
  *     NULL is never returned and the returned value must be released using dc_str_unref().
  */
@@ -4044,13 +4003,14 @@ char*           dc_msg_get_file               (const dc_msg_t* msg);
 
 
 /**
- * Get an original attachment filename, with extension but without the path. To get the full path,
- * use dc_msg_get_file().
+ * Get a base file name without the path. The base file name includes the extension; the path
+ * is not returned. To get the full path, use dc_msg_get_file().
  *
  * @memberof dc_msg_t
  * @param msg The message object.
- * @return The attachment filename. If there is no file associated with the message, an empty string
- *     is returned. The returned value must be released using dc_str_unref().
+ * @return The base file name plus the extension without part. If there is no file
+ *     associated with the message, an empty string is returned. The returned
+ *     value must be released using dc_str_unref().
  */
 char*           dc_msg_get_filename           (const dc_msg_t* msg);
 
@@ -4363,7 +4323,7 @@ int             dc_msg_is_forwarded           (const dc_msg_t* msg);
  * Check if the message is an informational message, created by the
  * device or by another users. Such messages are not "typed" by the user but
  * created due to other actions,
- * e.g. dc_set_chat_name(), dc_set_chat_profile_image(), dc_set_chat_protection()
+ * e.g. dc_set_chat_name(), dc_set_chat_profile_image(),
  * or dc_add_contact_to_chat().
  *
  * These messages are typically shown in the center of the chat view,
@@ -5057,7 +5017,12 @@ int             dc_contact_is_verified       (dc_contact_t* contact);
 /**
  * Return the address that verified a contact
  *
- * The UI may use this in addition to a checkmark showing the verification status
+ * The UI may use this in addition to a checkmark showing the verification status.
+ * In case of verification chains,
+ * the last contact in the chain is shown.
+ * This is because of privacy reasons, but also as it would not help the user
+ * to see a unknown name here - where one can mostly always ask the shown name
+ * as it is directly known.
  *
  * @memberof dc_contact_t
  * @param contact The contact object.
@@ -6314,7 +6279,6 @@ void dc_event_unref(dc_event_t* event);
 #define DC_KEY_GEN_DEFAULT 0
 #define DC_KEY_GEN_RSA2048 1
 #define DC_KEY_GEN_ED25519 2
-#define DC_KEY_GEN_RSA4096 3
 
 
 /**
@@ -6798,15 +6762,6 @@ void dc_event_unref(dc_event_t* event);
 /// Used in error strings.
 #define DC_STR_ERROR_NO_NETWORK           87
 
-/// "Chat protection enabled."
-///
-
-/// @deprecated Deprecated, replaced by DC_STR_MSG_YOU_ENABLED_PROTECTION and DC_STR_MSG_PROTECTION_ENABLED_BY.
-#define DC_STR_PROTECTION_ENABLED         88
-
-/// @deprecated Deprecated, replaced by DC_STR_MSG_YOU_DISABLED_PROTECTION and DC_STR_MSG_PROTECTION_DISABLED_BY.
-#define DC_STR_PROTECTION_DISABLED        89
-
 /// "Reply"
 ///
 /// Used in summaries.
@@ -7251,26 +7206,6 @@ void dc_event_unref(dc_event_t* event);
 /// `%2$s` will be replaced by name and address of the contact.
 #define DC_STR_EPHEMERAL_TIMER_WEEKS_BY_OTHER 157
 
-/// "You enabled chat protection."
-///
-/// Used in status messages.
-#define DC_STR_PROTECTION_ENABLED_BY_YOU 158
-
-/// "Chat protection enabled by %1$s."
-///
-/// `%1$s` will be replaced by name and address of the contact.
-///
-/// Used in status messages.
-#define DC_STR_PROTECTION_ENABLED_BY_OTHER 159
-
-/// "You disabled chat protection."
-#define DC_STR_PROTECTION_DISABLED_BY_YOU 160
-
-/// "Chat protection disabled by %1$s."
-///
-/// `%1$s` will be replaced by name and address of the contact.
-#define DC_STR_PROTECTION_DISABLED_BY_OTHER 161
-
 /// "Scan to set up second device for %1$s"
 ///
 /// `%1$s` will be replaced by name and address of the account.
@@ -7281,6 +7216,16 @@ void dc_event_unref(dc_event_t* event);
 /// Used as a device message after a successful backup transfer.
 #define DC_STR_BACKUP_TRANSFER_MSG_BODY 163
 
+/// "Messages are guaranteed to be end-to-end encrypted from now on."
+///
+/// Used in info messages.
+#define DC_STR_CHAT_PROTECTION_ENABLED 170
+
+/// "%1$s sent a message from another device."
+///
+/// Used in info messages.
+#define DC_STR_CHAT_PROTECTION_DISABLED 171
+
 /**
  * @}
  */

deltachat-ffi/src/lib.rs

@@ -36,7 +36,7 @@ use deltachat::qr_code_generator::{generate_backup_qr, get_securejoin_qr_svg};
 use deltachat::reaction::{get_msg_reactions, send_reaction, Reactions};
 use deltachat::stock_str::StockMessage;
 use deltachat::stock_str::StockStrings;
-use deltachat::webxdc::{replace_webxdc, StatusUpdateSerial};
+use deltachat::webxdc::StatusUpdateSerial;
 use deltachat::*;
 use deltachat::{accounts::Accounts, log::LogExt};
 use num_traits::{FromPrimitive, ToPrimitive};
@@ -167,24 +167,6 @@ pub unsafe extern "C" fn dc_context_open(
         .unwrap_or(0)
 }
 
-#[no_mangle]
-pub unsafe extern "C" fn dc_context_change_passphrase(
-    context: *mut dc_context_t,
-    passphrase: *const libc::c_char,
-) -> libc::c_int {
-    if context.is_null() {
-        eprintln!("ignoring careless call to dc_context_change_passphrase()");
-        return 0;
-    }
-
-    let ctx = &*context;
-    let passphrase = to_string_lossy(passphrase);
-    block_on(ctx.change_passphrase(passphrase))
-        .context("dc_context_change_passphrase() failed")
-        .log_err(ctx)
-        .is_ok() as libc::c_int
-}
-
 #[no_mangle]
 pub unsafe extern "C" fn dc_context_is_open(context: *mut dc_context_t) -> libc::c_int {
     if context.is_null() {
@@ -1097,32 +1079,6 @@ pub unsafe extern "C" fn dc_get_webxdc_status_updates(
     .strdup()
 }
 
-#[no_mangle]
-pub unsafe extern "C" fn dc_replace_webxdc(
-    context: *mut dc_context_t,
-    msg_id: u32,
-    blob: *const u8,
-    n: libc::size_t,
-) {
-    if context.is_null() {
-        eprintln!("ignoring careless call to dc_replace_webxdc()");
-        return;
-    }
-
-    let msg_id = MsgId::new(msg_id);
-    let blob_slice = std::slice::from_raw_parts(blob, n);
-
-    let ctx = &*context;
-
-    block_on(async move {
-        replace_webxdc(ctx, msg_id, blob_slice)
-            .await
-            .context("Failed to replace WebXDC")
-            .log_err(ctx)
-            .ok();
-    })
-}
-
 #[no_mangle]
 pub unsafe extern "C" fn dc_set_draft(
     context: *mut dc_context_t,
@@ -1286,30 +1242,6 @@ pub unsafe extern "C" fn dc_get_fresh_msg_cnt(
     })
 }
 
-#[no_mangle]
-pub unsafe extern "C" fn dc_get_similar_chatlist(
-    context: *mut dc_context_t,
-    chat_id: u32,
-) -> *mut dc_chatlist_t {
-    if context.is_null() {
-        eprintln!("ignoring careless call to dc_get_similar_chatlist()");
-        return ptr::null_mut();
-    }
-    let ctx = &*context;
-
-    let chat_id = ChatId::new(chat_id);
-    match block_on(chat_id.get_similar_chatlist(ctx))
-        .context("failed to get similar chatlist")
-        .log_err(ctx)
-    {
-        Ok(list) => {
-            let ffi_list = ChatlistWrapper { context, list };
-            Box::into_raw(Box::new(ffi_list))
-        }
-        Err(_) => ptr::null_mut(),
-    }
-}
-
 #[no_mangle]
 pub unsafe extern "C" fn dc_estimate_deletion_cnt(
     context: *mut dc_context_t,
@@ -1497,32 +1429,6 @@ pub unsafe extern "C" fn dc_get_next_media(
     })
 }
 
-#[no_mangle]
-pub unsafe extern "C" fn dc_set_chat_protection(
-    context: *mut dc_context_t,
-    chat_id: u32,
-    protect: libc::c_int,
-) -> libc::c_int {
-    if context.is_null() {
-        eprintln!("ignoring careless call to dc_set_chat_protection()");
-        return 0;
-    }
-    let ctx = &*context;
-    let protect = if let Some(s) = ProtectionStatus::from_i32(protect) {
-        s
-    } else {
-        warn!(ctx, "bad protect-value for dc_set_chat_protection()");
-        return 0;
-    };
-
-    block_on(async move {
-        match ChatId::new(chat_id).set_protection(ctx, protect).await {
-            Ok(()) => 1,
-            Err(_) => 0,
-        }
-    })
-}
-
 #[no_mangle]
 pub unsafe extern "C" fn dc_set_chat_visibility(
     context: *mut dc_context_t,
@@ -1565,10 +1471,14 @@ pub unsafe extern "C" fn dc_delete_chat(context: *mut dc_context_t, chat_id: u32
     }
     let ctx = &*context;
 
-    block_on(ChatId::new(chat_id).delete(ctx))
-        .context("Failed chat delete")
-        .log_err(ctx)
-        .ok();
+    block_on(async move {
+        ChatId::new(chat_id)
+            .delete(ctx)
+            .await
+            .context("Failed chat delete")
+            .log_err(ctx)
+            .ok();
+    })
 }
 
 #[no_mangle]
@@ -3148,6 +3058,16 @@ pub unsafe extern "C" fn dc_chat_is_protected(chat: *mut dc_chat_t) -> libc::c_i
     ffi_chat.chat.is_protected() as libc::c_int
 }
 
+#[no_mangle]
+pub unsafe extern "C" fn dc_chat_is_protection_broken(chat: *mut dc_chat_t) -> libc::c_int {
+    if chat.is_null() {
+        eprintln!("ignoring careless call to dc_chat_is_protection_broken()");
+        return 0;
+    }
+    let ffi_chat = &*chat;
+    ffi_chat.chat.is_protection_broken() as libc::c_int
+}
+
 #[no_mangle]
 pub unsafe extern "C" fn dc_chat_is_sending_locations(chat: *mut dc_chat_t) -> libc::c_int {
     if chat.is_null() {

deltachat-jsonrpc/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "deltachat-jsonrpc"
-version = "1.122.0"
+version = "1.118.0"
 description = "DeltaChat JSON-RPC API"
 edition = "2021"
 default-run = "deltachat-jsonrpc-server"

deltachat-jsonrpc/src/api/mod.rs

@@ -566,21 +566,6 @@ impl CommandApi {
         Ok(l)
     }
 
-    /// Returns chats similar to the given one.
-    ///
-    /// Experimental API, subject to change without notice.
-    async fn get_similar_chat_ids(&self, account_id: u32, chat_id: u32) -> Result<Vec<u32>> {
-        let ctx = self.get_context(account_id).await?;
-        let chat_id = ChatId::new(chat_id);
-        let list = chat_id
-            .get_similar_chat_ids(&ctx)
-            .await?
-            .into_iter()
-            .map(|(chat_id, _metric)| chat_id.to_u32())
-            .collect();
-        Ok(list)
-    }
-
     async fn get_chatlist_items_by_entries(
         &self,
         account_id: u32,

deltachat-jsonrpc/src/api/types/chat_list.rs

@@ -8,12 +8,15 @@ use deltachat::{
     chatlist::Chatlist,
 };
 use num_traits::cast::ToPrimitive;
-use serde::Serialize;
+use serde::{Deserialize, Serialize};
 use typescript_type_def::TypeDef;
 
 use super::color_int_to_hex_string;
 use super::message::MessageViewtype;
 
+#[derive(Deserialize, Serialize, TypeDef, schemars::JsonSchema)]
+pub struct ChatListEntry(pub u32, pub u32);
+
 #[derive(Serialize, TypeDef, schemars::JsonSchema)]
 #[serde(tag = "type")]
 pub enum ChatListItemFetchResult {

deltachat-jsonrpc/src/api/types/message.rs

@@ -318,7 +318,6 @@ pub enum DownloadState {
     Done,
     Available,
     Failure,
-    Undecipherable,
     InProgress,
 }
 
@@ -328,7 +327,6 @@ impl From<download::DownloadState> for DownloadState {
             download::DownloadState::Done => DownloadState::Done,
             download::DownloadState::Available => DownloadState::Available,
             download::DownloadState::Failure => DownloadState::Failure,
-            download::DownloadState::Undecipherable => DownloadState::Undecipherable,
             download::DownloadState::InProgress => DownloadState::InProgress,
         }
     }

deltachat-jsonrpc/typescript/package.json

@@ -55,5 +55,5 @@
   },
   "type": "module",
   "types": "dist/deltachat.d.ts",
-  "version": "1.122.0"
+  "version": "1.118.0"
 }

deltachat-repl/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "deltachat-repl"
-version = "1.122.0"
+version = "1.118.0"
 license = "MPL-2.0"
 edition = "2021"
 

deltachat-repl/src/cmdline.rs

@@ -18,6 +18,7 @@ use deltachat::imex::*;
 use deltachat::location;
 use deltachat::log::LogExt;
 use deltachat::message::{self, Message, MessageState, MsgId, Viewtype};
+use deltachat::mimeparser::SystemMessage;
 use deltachat::peerstate::*;
 use deltachat::qr::*;
 use deltachat::reaction::send_reaction;
@@ -187,7 +188,6 @@ async fn log_msg(context: &Context, prefix: impl AsRef<str>, msg: &Message) {
         DownloadState::Available => " [⬇ Download available]",
         DownloadState::InProgress => " [⬇ Download in progress...]️",
         DownloadState::Failure => " [⬇ Download failed]",
-        DownloadState::Undecipherable => " [⬇ Decryption failed]",
     };
 
     let temp2 = timestamp_to_str(msg.get_timestamp());
@@ -211,7 +211,17 @@ async fn log_msg(context: &Context, prefix: impl AsRef<str>, msg: &Message) {
         } else {
             "[FRESH]"
         },
-        if msg.is_info() { "[INFO]" } else { "" },
+        if msg.is_info() {
+            if msg.get_info_type() == SystemMessage::ChatProtectionEnabled {
+                "[INFO 🛡️]"
+            } else if msg.get_info_type() == SystemMessage::ChatProtectionDisabled {
+                "[INFO 🛡️❌]"
+            } else {
+                "[INFO]"
+            }
+        } else {
+            ""
+        },
         if msg.get_viewtype() == Viewtype::VideochatInvitation {
             format!(
                 "[VIDEOCHAT-INVITATION: {}, type={}]",
@@ -396,8 +406,6 @@ pub async fn cmdline(context: Context, line: &str, chat_id: &mut ChatId) -> Resu
                  unpin <chat-id>\n\
                  mute <chat-id> [<seconds>]\n\
                  unmute <chat-id>\n\
-                 protect <chat-id>\n\
-                 unprotect <chat-id>\n\
                  delchat <chat-id>\n\
                  accept <chat-id>\n\
                  decline <chat-id>\n\
@@ -806,30 +814,15 @@ pub async fn cmdline(context: Context, line: &str, chat_id: &mut ChatId) -> Resu
         }
         "chatinfo" => {
             ensure!(sel_chat.is_some(), "No chat selected.");
-            let sel_chat_id = sel_chat.as_ref().unwrap().get_id();
 
-            let contacts = chat::get_chat_contacts(&context, sel_chat_id).await?;
+            let contacts =
+                chat::get_chat_contacts(&context, sel_chat.as_ref().unwrap().get_id()).await?;
             println!("Memberlist:");
 
             log_contactlist(&context, &contacts).await?;
-            println!("{} contacts", contacts.len());
-
-            let similar_chats = sel_chat_id.get_similar_chat_ids(&context).await?;
-            if !similar_chats.is_empty() {
-                println!("Similar chats: ");
-                for (similar_chat_id, metric) in similar_chats {
-                    let similar_chat = Chat::load_from_db(&context, similar_chat_id).await?;
-                    println!(
-                        "{} (#{}) {:.1}",
-                        similar_chat.name,
-                        similar_chat_id,
-                        100.0 * metric
-                    );
-                }
-            }
-
             println!(
-                "Location streaming: {}",
+                "{} contacts\nLocation streaming: {}",
+                contacts.len(),
                 location::is_sending_locations_to_chat(
                     &context,
                     Some(sel_chat.as_ref().unwrap().get_id())
@@ -1072,20 +1065,6 @@ pub async fn cmdline(context: Context, line: &str, chat_id: &mut ChatId) -> Resu
             };
             chat::set_muted(&context, chat_id, duration).await?;
         }
-        "protect" | "unprotect" => {
-            ensure!(!arg1.is_empty(), "Argument <chat-id> missing.");
-            let chat_id = ChatId::new(arg1.parse()?);
-            chat_id
-                .set_protection(
-                    &context,
-                    match arg0 {
-                        "protect" => ProtectionStatus::Protected,
-                        "unprotect" => ProtectionStatus::Unprotected,
-                        _ => unreachable!("arg0={:?}", arg0),
-                    },
-                )
-                .await?;
-        }
         "delchat" => {
             ensure!(!arg1.is_empty(), "Argument <chat-id> missing.");
             let chat_id = ChatId::new(arg1.parse()?);

deltachat-rpc-client/pyproject.toml

@@ -6,7 +6,8 @@ build-backend = "setuptools.build_meta"
 name = "deltachat-rpc-client"
 description = "Python client for Delta Chat core JSON-RPC interface"
 dependencies = [
-    "aiohttp"
+    "aiohttp",
+    "aiodns"
 ]
 classifiers = [
     "Development Status :: 5 - Production/Stable",

deltachat-rpc-client/src/deltachat_rpc_client/rpc.py

@@ -89,14 +89,16 @@ class Rpc:
         return await queue.get()
 
     def __getattr__(self, attr: str):
-        async def method(*args) -> Any:
+        async def method(*args, **kwargs) -> Any:
             self.id += 1
             request_id = self.id
 
+            assert not (args and kwargs), "Mixing positional and keyword arguments"
+
             request = {
                 "jsonrpc": "2.0",
                 "method": attr,
-                "params": args,
+                "params": kwargs or args,
                 "id": self.id,
             }
             data = (json.dumps(request) + "\n").encode()

deltachat-rpc-server/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "deltachat-rpc-server"
-version = "1.122.0"
+version = "1.118.0"
 description = "DeltaChat JSON-RPC server"
 edition = "2021"
 readme = "README.md"

deny.toml

@@ -2,7 +2,6 @@
 unmaintained = "allow"
 ignore = [
     "RUSTSEC-2020-0071",
-    "RUSTSEC-2022-0093",
 ]
 
 [bans]
@@ -12,53 +11,31 @@ ignore = [
 # Please keep this list alphabetically sorted.
 skip = [
      { name = "ahash", version = "0.7.6" },
-     { name = "base16ct", version = "0.1.1" },
      { name = "base64", version = "<0.21" },
      { name = "bitflags", version = "1.3.2" },
-     { name = "block-buffer", version = "<0.10" },
-     { name = "convert_case", version = "0.4.0" },
-     { name = "curve25519-dalek", version = "3.2.0" },
-     { name = "darling_core", version = "<0.14" },
-     { name = "darling_macro", version = "<0.14" },
-     { name = "darling", version = "<0.14" },
-     { name = "der", version = "0.6.1" },
-     { name = "digest", version = "<0.10" },
-     { name = "ed25519-dalek", version = "1.0.1" },
-     { name = "ed25519", version = "1.5.3" },
+     { name = "darling", version = "<0.14.4" },
+     { name = "darling_core", version = "<0.14.4" },
+     { name = "darling_macro", version = "<0.14.4" },
+     { name = "fastrand", version = "<2.0.0" },
      { name = "getrandom", version = "<0.2" },
      { name = "hashbrown", version = "<0.14.0" },
      { name = "idna", version = "<0.3" },
      { name = "indexmap", version = "<2.0.0" },
-     { name = "linux-raw-sys", version = "0.3.8" },
-     { name = "num-derive", version = "0.3.3" },
-     { name = "pem-rfc7468", version = "0.6.0" },
-     { name = "pkcs8", version = "0.9.0" },
      { name = "quick-error", version = "<2.0" },
      { name = "rand_chacha", version = "<0.3" },
      { name = "rand_core", version = "<0.6" },
      { name = "rand", version = "<0.8" },
      { name = "redox_syscall", version = "0.2.16" },
+     { name = "regex-automata", version = "<0.3.3" },
      { name = "regex-syntax", version = "0.6.29" },
-     { name = "rustix", version = "0.37.21" },
-     { name = "sec1", version = "0.3.0" },
-     { name = "sha2", version = "<0.10" },
-     { name = "signature", version = "1.6.4" },
+     { name = "rustls-webpki", version = "<0.101.1" },
      { name = "socket2", version = "0.4.9" },
+     { name = "sha1", version = "<0.10.5" },
      { name = "spin", version = "<0.9.6" },
-     { name = "spki", version = "0.6.0" },
      { name = "syn", version = "1.0.109" },
      { name = "time", version = "<0.3" },
      { name = "wasi", version = "<0.11" },
-     { name = "windows_aarch64_gnullvm", version = "<0.48" },
-     { name = "windows_aarch64_msvc", version = "<0.48" },
-     { name = "windows_i686_gnu", version = "<0.48" },
-     { name = "windows_i686_msvc", version = "<0.48" },
-     { name = "windows-sys", version = "<0.48" },
-     { name = "windows-targets", version = "<0.48" },
-     { name = "windows_x86_64_gnullvm", version = "<0.48" },
-     { name = "windows", version = "0.32.0" },
-     { name = "windows_x86_64_gnu", version = "<0.48" },
-     { name = "windows_x86_64_msvc", version = "<0.48" },
+     { name = "webpki-roots", version = "<0.23.1" },
      { name = "winreg", version = "0.10.1" },
 ]
 
@@ -91,5 +68,4 @@ license-files = [
 github = [
        "async-email",
        "deltachat",
-       "quinn-rs",
 ]

examples/simple.rs

@@ -1,100 +0,0 @@
-use deltachat::chat::{self, ChatId};
-use deltachat::chatlist::*;
-use deltachat::config;
-use deltachat::contact::*;
-use deltachat::context::*;
-use deltachat::message::Message;
-use deltachat::stock_str::StockStrings;
-use deltachat::{EventType, Events};
-use tempfile::tempdir;
-
-fn cb(event: EventType) {
-    match event {
-        EventType::ConfigureProgress { progress, .. } => {
-            log::info!("progress: {}", progress);
-        }
-        EventType::Info(msg) => {
-            log::info!("{}", msg);
-        }
-        EventType::Warning(msg) => {
-            log::warn!("{}", msg);
-        }
-        EventType::Error(msg) => {
-            log::error!("{}", msg);
-        }
-        event => {
-            log::info!("{:?}", event);
-        }
-    }
-}
-
-/// Run with `RUST_LOG=simple=info cargo run --release --example simple -- email pw`.
-#[tokio::main]
-async fn main() {
-    pretty_env_logger::try_init_timed().ok();
-
-    let dir = tempdir().unwrap();
-    let dbfile = dir.path().join("db.sqlite");
-    log::info!("creating database {:?}", dbfile);
-    let ctx = Context::new(&dbfile, 0, Events::new(), StockStrings::new())
-        .await
-        .expect("Failed to create context");
-    let info = ctx.get_info().await;
-    log::info!("info: {:#?}", info);
-
-    let events = ctx.get_event_emitter();
-    let events_spawn = tokio::task::spawn(async move {
-        while let Some(event) = events.recv().await {
-            cb(event.typ);
-        }
-    });
-
-    log::info!("configuring");
-    let args = std::env::args().collect::<Vec<String>>();
-    assert_eq!(args.len(), 3, "requires email password");
-    let email = args[1].clone();
-    let pw = args[2].clone();
-    ctx.set_config(config::Config::Addr, Some(&email))
-        .await
-        .unwrap();
-    ctx.set_config(config::Config::MailPw, Some(&pw))
-        .await
-        .unwrap();
-
-    ctx.configure().await.unwrap();
-
-    log::info!("------ RUN ------");
-    ctx.start_io().await;
-    log::info!("--- SENDING A MESSAGE ---");
-
-    let contact_id = Contact::create(&ctx, "dignifiedquire", "dignifiedquire@gmail.com")
-        .await
-        .unwrap();
-    let chat_id = ChatId::create_for_contact(&ctx, contact_id).await.unwrap();
-
-    for i in 0..1 {
-        log::info!("sending message {}", i);
-        chat::send_text_msg(&ctx, chat_id, format!("Hi, here is my {i}nth message!"))
-            .await
-            .unwrap();
-    }
-
-    // wait for the message to be sent out
-    tokio::time::sleep(std::time::Duration::from_secs(1)).await;
-
-    log::info!("fetching chats..");
-    let chats = Chatlist::try_load(&ctx, 0, None, None).await.unwrap();
-
-    for i in 0..chats.len() {
-        let msg = Message::load_from_db(&ctx, chats.get_msg_id(i).unwrap().unwrap())
-            .await
-            .unwrap();
-        log::info!("[{}] msg: {:?}", i, msg);
-    }
-
-    log::info!("stopping");
-    ctx.stop_io().await;
-    log::info!("closing");
-    drop(ctx);
-    events_spawn.await.unwrap();
-}

node/constants.js

@@ -90,7 +90,6 @@ module.exports = {
   DC_KEY_GEN_DEFAULT: 0,
   DC_KEY_GEN_ED25519: 2,
   DC_KEY_GEN_RSA2048: 1,
-  DC_KEY_GEN_RSA4096: 3,
   DC_LP_AUTH_NORMAL: 4,
   DC_LP_AUTH_OAUTH2: 2,
   DC_MEDIA_QUALITY_BALANCED: 0,
@@ -159,6 +158,8 @@ module.exports = {
   DC_STR_BROADCAST_LIST: 115,
   DC_STR_CANNOT_LOGIN: 60,
   DC_STR_CANTDECRYPT_MSG_BODY: 29,
+  DC_STR_CHAT_PROTECTION_DISABLED: 171,
+  DC_STR_CHAT_PROTECTION_ENABLED: 170,
   DC_STR_CONFIGURATION_FAILED: 84,
   DC_STR_CONNECTED: 107,
   DC_STR_CONNTECTING: 108,
@@ -244,12 +245,6 @@ module.exports = {
   DC_STR_OUTGOING_MESSAGES: 104,
   DC_STR_PARTIAL_DOWNLOAD_MSG_BODY: 99,
   DC_STR_PART_OF_TOTAL_USED: 116,
-  DC_STR_PROTECTION_DISABLED: 89,
-  DC_STR_PROTECTION_DISABLED_BY_OTHER: 161,
-  DC_STR_PROTECTION_DISABLED_BY_YOU: 160,
-  DC_STR_PROTECTION_ENABLED: 88,
-  DC_STR_PROTECTION_ENABLED_BY_OTHER: 159,
-  DC_STR_PROTECTION_ENABLED_BY_YOU: 158,
   DC_STR_QUOTA_EXCEEDING_MSG_BODY: 98,
   DC_STR_READRCPT: 31,
   DC_STR_READRCPT_MAILBODY: 32,

node/lib/constants.ts

@@ -90,7 +90,6 @@ export enum C {
   DC_KEY_GEN_DEFAULT = 0,
   DC_KEY_GEN_ED25519 = 2,
   DC_KEY_GEN_RSA2048 = 1,
-  DC_KEY_GEN_RSA4096 = 3,
   DC_LP_AUTH_NORMAL = 4,
   DC_LP_AUTH_OAUTH2 = 2,
   DC_MEDIA_QUALITY_BALANCED = 0,
@@ -159,6 +158,8 @@ export enum C {
   DC_STR_BROADCAST_LIST = 115,
   DC_STR_CANNOT_LOGIN = 60,
   DC_STR_CANTDECRYPT_MSG_BODY = 29,
+  DC_STR_CHAT_PROTECTION_DISABLED = 171,
+  DC_STR_CHAT_PROTECTION_ENABLED = 170,
   DC_STR_CONFIGURATION_FAILED = 84,
   DC_STR_CONNECTED = 107,
   DC_STR_CONNTECTING = 108,
@@ -244,12 +245,6 @@ export enum C {
   DC_STR_OUTGOING_MESSAGES = 104,
   DC_STR_PARTIAL_DOWNLOAD_MSG_BODY = 99,
   DC_STR_PART_OF_TOTAL_USED = 116,
-  DC_STR_PROTECTION_DISABLED = 89,
-  DC_STR_PROTECTION_DISABLED_BY_OTHER = 161,
-  DC_STR_PROTECTION_DISABLED_BY_YOU = 160,
-  DC_STR_PROTECTION_ENABLED = 88,
-  DC_STR_PROTECTION_ENABLED_BY_OTHER = 159,
-  DC_STR_PROTECTION_ENABLED_BY_YOU = 158,
   DC_STR_QUOTA_EXCEEDING_MSG_BODY = 98,
   DC_STR_READRCPT = 31,
   DC_STR_READRCPT_MAILBODY = 32,

node/lib/context.ts

@@ -699,23 +699,6 @@ export class Context extends EventEmitter {
     )
   }
 
-  /**
-   *
-   * @param chatId
-   * @param protect
-   * @returns success boolean
-   */
-  setChatProtection(chatId: number, protect: boolean) {
-    debug(`setChatProtection ${chatId} ${protect}`)
-    return Boolean(
-      binding.dcn_set_chat_protection(
-        this.dcn_context,
-        Number(chatId),
-        protect ? 1 : 0
-      )
-    )
-  }
-
   getChatEphemeralTimer(chatId: number): number {
     debug(`getChatEphemeralTimer ${chatId}`)
     return binding.dcn_get_chat_ephemeral_timer(

node/src/module.c

@@ -1399,18 +1399,6 @@ NAPI_METHOD(dcn_set_chat_name) {
   NAPI_RETURN_INT32(result);
 }
 
-NAPI_METHOD(dcn_set_chat_protection) {
-  NAPI_ARGV(3);
-  NAPI_DCN_CONTEXT();
-  NAPI_ARGV_UINT32(chat_id, 1);
-  NAPI_ARGV_INT32(protect, 1);
-
-  int result = dc_set_chat_protection(dcn_context->dc_context,
-                                      chat_id,
-                                      protect);
-  NAPI_RETURN_INT32(result);
-}
-
 NAPI_METHOD(dcn_get_chat_ephemeral_timer) {
   NAPI_ARGV(2);
   NAPI_DCN_CONTEXT();
@@ -3491,7 +3479,6 @@ NAPI_INIT() {
   NAPI_EXPORT_FUNCTION(dcn_send_msg);
   NAPI_EXPORT_FUNCTION(dcn_send_videochat_invitation);
   NAPI_EXPORT_FUNCTION(dcn_set_chat_name);
-  NAPI_EXPORT_FUNCTION(dcn_set_chat_protection);
   NAPI_EXPORT_FUNCTION(dcn_get_chat_ephemeral_timer);
   NAPI_EXPORT_FUNCTION(dcn_set_chat_ephemeral_timer);
   NAPI_EXPORT_FUNCTION(dcn_set_chat_profile_image);

node/test/test.js

@@ -446,8 +446,7 @@ describe('Offline Tests with unconfigured account', function () {
     context.setChatProfileImage(chatId, imagePath)
     const blobPath = context.getChat(chatId).getProfileImage()
     expect(blobPath.startsWith(blobs)).to.be.true
-    expect(blobPath.includes('image')).to.be.true
-    expect(blobPath.endsWith('.jpeg')).to.be.true
+    expect(blobPath.endsWith(image)).to.be.true
 
     context.setChatProfileImage(chatId, null)
     expect(context.getChat(chatId).getProfileImage()).to.be.equal(

package.json

@@ -60,5 +60,5 @@
     "test:mocha": "mocha -r esm node/test/test.js --growl --reporter=spec --bail --exit"
   },
   "types": "node/dist/index.d.ts",
-  "version": "1.122.0"
+  "version": "1.118.0"
 }

python/pyproject.toml

@@ -34,7 +34,6 @@ dynamic = [
 "Home" = "https://github.com/deltachat/deltachat-core-rust/"
 "Bug Tracker" = "https://github.com/deltachat/deltachat-core-rust/issues"
 "Documentation" = "https://py.delta.chat/"
-"Mastodon" = "https://chaos.social/@delta"
 
 [project.entry-points.pytest11]
 "deltachat.testplugin" = "deltachat.testplugin"

python/src/deltachat/account.py

@@ -427,7 +427,7 @@ class Account:
 
         assert dc_chatlist != ffi.NULL
         chatlist = []
-        for i in range(lib.dc_chatlist_get_cnt(dc_chatlist)):
+        for i in range(0, lib.dc_chatlist_get_cnt(dc_chatlist)):
             chat_id = lib.dc_chatlist_get_chat_id(dc_chatlist, i)
             chatlist.append(Chat(self, chat_id))
         return chatlist

python/src/deltachat/cutil.py

@@ -15,7 +15,7 @@ def as_dc_charpointer(obj):
 
 
 def iter_array(dc_array_t, constructor: Callable[[int], T]) -> Generator[T, None, None]:
-    for i in range(lib.dc_array_get_cnt(dc_array_t)):
+    for i in range(0, lib.dc_array_get_cnt(dc_array_t)):
         yield constructor(lib.dc_array_get_id(dc_array_t, i))
 
 

python/src/deltachat/message.py

@@ -510,7 +510,8 @@ def get_viewtype_code_from_name(view_type_name):
     if code is not None:
         return code
     raise ValueError(
-        f"message typecode not found for {view_type_name!r}, available {list(_view_type_mapping.keys())!r}",
+        "message typecode not found for {!r}, "
+        "available {!r}".format(view_type_name, list(_view_type_mapping.keys())),
     )
 
 

python/tests/bench_test_setup.py

@@ -15,6 +15,6 @@ class TestEmpty:
     def test_prepare_setup_measurings(self, acfactory):
         acfactory.get_online_accounts(BENCH_NUM)
 
-    @pytest.mark.parametrize("num", range(BENCH_NUM + 1))
+    @pytest.mark.parametrize("num", range(0, BENCH_NUM + 1))
     def test_setup_online_accounts(self, acfactory, num):
         acfactory.get_online_accounts(num)

python/tests/test_1_online.py

@@ -162,9 +162,8 @@ def test_send_file_twice_unicode_filename_mangling(tmp_path, acfactory, lp):
     ac1, ac2 = acfactory.get_online_accounts(2)
     chat = acfactory.get_accepted_chat(ac1, ac2)
 
-    basename = "somedäüta"
-    ext = ".html.zip"
-    p = tmp_path / (basename + ext)
+    basename = "somedäüta.html.zip"
+    p = tmp_path / basename
     p.write_text("some data")
 
     def send_and_receive_message():
@@ -182,14 +181,12 @@ def test_send_file_twice_unicode_filename_mangling(tmp_path, acfactory, lp):
     msg = send_and_receive_message()
     assert msg.text == "withfile"
     assert open(msg.filename).read() == "some data"
-    msg.filename.index(basename)
-    assert msg.filename.endswith(ext)
+    assert msg.filename.endswith(basename)
 
     msg2 = send_and_receive_message()
     assert msg2.text == "withfile"
     assert open(msg2.filename).read() == "some data"
-    msg2.filename.index(basename)
-    assert msg2.filename.endswith(ext)
+    assert msg2.filename.endswith("html.zip")
     assert msg.filename != msg2.filename
 
 
@@ -197,11 +194,10 @@ def test_send_file_html_attachment(tmp_path, acfactory, lp):
     ac1, ac2 = acfactory.get_online_accounts(2)
     chat = acfactory.get_accepted_chat(ac1, ac2)
 
-    basename = "test"
-    ext = ".html"
+    basename = "test.html"
     content = "<html><body>text</body>data"
 
-    p = tmp_path / (basename + ext)
+    p = tmp_path / basename
     # write wrong html to see if core tries to parse it
     # (it shouldn't as it's a file attachment)
     p.write_text(content)
@@ -215,8 +211,7 @@ def test_send_file_html_attachment(tmp_path, acfactory, lp):
     msg = ac2.get_message_by_id(ev.data2)
 
     assert open(msg.filename).read() == content
-    msg.filename.index(basename)
-    assert msg.filename.endswith(ext)
+    assert msg.filename.endswith(basename)
 
 
 def test_html_message(acfactory, lp):
@@ -329,27 +324,6 @@ def test_webxdc_message(acfactory, data, lp):
     assert len(list(ac2.direct_imap.conn.fetch(AND(seen=True)))) == 1
 
 
-def test_webxdc_huge_update(acfactory, data, lp):
-    ac1, ac2 = acfactory.get_online_accounts(2)
-    chat = ac1.create_chat(ac2)
-
-    msg1 = Message.new_empty(ac1, "webxdc")
-    msg1.set_text("message1")
-    msg1.set_file(data.get_path("webxdc/minimal.xdc"))
-    msg1 = chat.send_msg(msg1)
-    assert msg1.is_webxdc()
-    assert msg1.filename
-
-    msg2 = ac2._evtracker.wait_next_incoming_message()
-    assert msg2.is_webxdc()
-
-    payload = "A" * 1000
-    assert msg1.send_status_update({"payload": payload}, "some test data")
-    ac2._evtracker.get_matching("DC_EVENT_WEBXDC_STATUS_UPDATE")
-    update = msg2.get_status_updates()[0]
-    assert update["payload"] == payload
-
-
 def test_webxdc_download_on_demand(acfactory, data, lp):
     ac1, ac2 = acfactory.get_online_accounts(2)
     acfactory.introduce_each_other([ac1, ac2])
@@ -376,11 +350,6 @@ def test_webxdc_download_on_demand(acfactory, data, lp):
     ac2._evtracker.get_matching("DC_EVENT_WEBXDC_STATUS_UPDATE")
     assert msg2.get_status_updates()
 
-    # Get a event notifying that the message disappeared from the chat.
-    msgs_changed_event = ac2._evtracker.get_matching("DC_EVENT_MSGS_CHANGED")
-    assert msgs_changed_event.data1 == msg2.chat.id
-    assert msgs_changed_event.data2 == 0
-
 
 def test_mvbox_sentbox_threads(acfactory, lp):
     lp.sec("ac1: start with mvbox thread")
@@ -1679,36 +1648,6 @@ def test_qr_join_chat(acfactory, lp):
     ac1._evtracker.wait_securejoin_inviter_progress(1000)
 
 
-def test_qr_new_group_unblocked(acfactory, lp):
-    """Regression test for a bug intoduced in core v1.113.0.
-    ac2 scans a verified group QR code created by ac1.
-    This results in creation of a blocked 1:1 chat with ac1 on ac2,
-    but ac1 contact is not blocked on ac2.
-    Then ac1 creates a group, adds ac2 there and promotes it by sending a message.
-    ac2 should receive a message and create a contact request for the group.
-    Due to a bug previously ac2 created a blocked group.
-    """
-
-    ac1, ac2 = acfactory.get_online_accounts(2)
-    ac1_chat = ac1.create_group_chat("Group for joining", verified=True)
-    qr = ac1_chat.get_join_qr()
-    ac2.qr_join_chat(qr)
-
-    ac1._evtracker.wait_securejoin_inviter_progress(1000)
-
-    ac1_new_chat = ac1.create_group_chat("Another group")
-    ac1_new_chat.add_contact(ac2)
-    ac1_new_chat.send_text("Hello!")
-
-    # Receive "Member added" message.
-    ac2._evtracker.wait_next_incoming_message()
-
-    # Receive "Hello!" message.
-    ac2_msg = ac2._evtracker.wait_next_incoming_message()
-    assert ac2_msg.text == "Hello!"
-    assert ac2_msg.chat.is_contact_request()
-
-
 def test_qr_email_capitalization(acfactory, lp):
     """Regression test for a bug
     that resulted in failure to propagate verification via gossip in a verified group

python/tests/test_2_increation.py

@@ -49,9 +49,10 @@ class TestOnlineInCreation:
         assert str(tmp_path) != ac1.get_blobdir()
         src = tmp_path / "file.txt"
         src.write_text("hello there\n")
-        msg = chat.send_file(str(src))
-        assert msg.filename.startswith(os.path.join(ac1.get_blobdir(), "file"))
-        assert msg.filename.endswith(".txt")
+        chat.send_file(str(src))
+
+        blob_src = os.path.join(ac1.get_blobdir(), "file.txt")
+        assert os.path.exists(blob_src), "file.txt not copied to blobdir"
 
     def test_forward_increation(self, acfactory, data, lp):
         ac1, ac2 = acfactory.get_online_accounts(2)

release-date.in

@@ -1 +1 @@
-2023-09-12
+2023-07-07

scripts/coredeps/install-rust.sh

@@ -7,7 +7,7 @@ set -euo pipefail
 #
 # Avoid using rustup here as it depends on reading /proc/self/exe and
 # has problems running under QEMU.
-RUST_VERSION=1.72.0
+RUST_VERSION=1.68.0
 
 ARCH="$(uname -m)"
 test -f "/lib/libc.musl-$ARCH.so.1" && LIBC=musl || LIBC=gnu

scripts/create-provider-data-rs.py

@@ -44,7 +44,7 @@ def file2url(f):
 
 def process_opt(data):
     if not "opt" in data:
-        return "ProviderOptions::new()"
+        return "Default::default()"
     opt = "ProviderOptions {\n"
     opt_data = data.get("opt", "")
     for key in opt_data:
@@ -54,7 +54,7 @@ def process_opt(data):
         if value in {"True", "False"}:
             value = value.lower()
         opt += "        " + key + ": " + value + ",\n"
-    opt += "        ..ProviderOptions::new()\n"
+    opt += "        ..Default::default()\n"
     opt += "    }"
     return opt
 
@@ -62,7 +62,7 @@ def process_opt(data):
 def process_config_defaults(data):
     if not "config_defaults" in data:
         return "None"
-    defaults = "Some(&[\n"
+    defaults = "Some(vec![\n"
     config_defaults = data.get("config_defaults", "")
     for key in config_defaults:
         value = str(config_defaults[key])
@@ -96,11 +96,11 @@ def process_data(data, file):
             raise TypeError("domain used twice: " + domain)
         domains_set.add(domain)
 
-        domains += '    ("' + domain + '", &' + file2varname(file) + "),\n"
+        domains += '    ("' + domain + '", &*' + file2varname(file) + "),\n"
         comment += domain + ", "
 
     ids = ""
-    ids += '    ("' + file2id(file) + '", &' + file2varname(file) + "),\n"
+    ids += '    ("' + file2id(file) + '", &*' + file2varname(file) + "),\n"
 
     server = ""
     has_imap = False
@@ -155,18 +155,18 @@ def process_data(data, file):
         provider += (
             "static "
             + file2varname(file)
-            + ": Provider = Provider {\n"
+            + ": Lazy<Provider> = Lazy::new(|| Provider {\n"
         )
         provider += '    id: "' + file2id(file) + '",\n'
         provider += "    status: Status::" + status.capitalize() + ",\n"
         provider += '    before_login_hint: "' + before_login_hint + '",\n'
         provider += '    after_login_hint: "' + after_login_hint + '",\n'
         provider += '    overview_page: "' + file2url(file) + '",\n'
-        provider += "    server: &[\n" + server + "    ],\n"
+        provider += "    server: vec![\n" + server + "    ],\n"
         provider += "    opt: " + opt + ",\n"
         provider += "    config_defaults: " + config_defaults + ",\n"
         provider += "    oauth2_authorizer: " + oauth2 + ",\n"
-        provider += "};\n\n"
+        provider += "});\n\n"
     else:
         raise TypeError("SMTP and IMAP must be specified together or left out both")
 

scripts/zig-musl-check.sh

@@ -9,9 +9,9 @@ set -e
 unset RUSTFLAGS
 
 # Pin Rust version to avoid uncontrolled changes in the compiler and linker flags.
-export RUSTUP_TOOLCHAIN=1.72.0
+export RUSTUP_TOOLCHAIN=1.71.0
 
-ZIG_VERSION=0.11.0
+ZIG_VERSION=0.11.0-dev.2213+515e1c93e
 
 # Download Zig
 rm -fr "$ZIG_VERSION" "zig-linux-x86_64-$ZIG_VERSION.tar.xz"

scripts/zig-rpc-server.sh

@@ -8,9 +8,9 @@ set -e
 unset RUSTFLAGS
 
 # Pin Rust version to avoid uncontrolled changes in the compiler and linker flags.
-export RUSTUP_TOOLCHAIN=1.72.0
+export RUSTUP_TOOLCHAIN=1.71.0
 
-ZIG_VERSION=0.11.0
+ZIG_VERSION=0.11.0-dev.2213+515e1c93e
 
 # Download Zig
 rm -fr "$ZIG_VERSION" "zig-linux-x86_64-$ZIG_VERSION.tar.xz"

src/accounts.rs

@@ -296,10 +296,10 @@ impl Accounts {
 }
 
 /// Configuration file name.
-const CONFIG_NAME: &str = "accounts.toml";
+pub const CONFIG_NAME: &str = "accounts.toml";
 
 /// Database file name.
-const DB_NAME: &str = "dc.db";
+pub const DB_NAME: &str = "dc.db";
 
 /// Account manager configuration file.
 #[derive(Debug, Clone, PartialEq)]

src/blob.rs

@@ -9,7 +9,7 @@ use std::path::{Path, PathBuf};
 
 use anyhow::{format_err, Context as _, Result};
 use futures::StreamExt;
-use image::{DynamicImage, GenericImageView, ImageFormat, ImageOutputFormat};
+use image::{DynamicImage, ImageFormat, ImageOutputFormat};
 use num_traits::FromPrimitive;
 use tokio::io::AsyncWriteExt;
 use tokio::{fs, io};
@@ -323,35 +323,18 @@ impl<'a> BlobObject<'a> {
                 MediaQuality::Worse => constants::WORSE_AVATAR_SIZE,
             };
 
-        let maybe_sticker = &mut false;
         let strict_limits = true;
         // max_bytes is 20_000 bytes: Outlook servers don't allow headers larger than 32k.
         // 32 / 4 * 3 = 24k if you account for base64 encoding. To be safe, we reduced this to 20k.
-        if let Some(new_name) = self.recode_to_size(
-            context,
-            blob_abs,
-            maybe_sticker,
-            img_wh,
-            20_000,
-            strict_limits,
-        )? {
+        if let Some(new_name) =
+            self.recode_to_size(context, blob_abs, img_wh, 20_000, strict_limits)?
+        {
             self.name = new_name;
         }
         Ok(())
     }
 
-    /// Recodes an image pointed by a [BlobObject] so that it fits into limits on the image width,
-    /// height and file size specified by the config.
-    ///
-    /// On some platforms images are passed to the core as [`crate::message::Viewtype::Sticker`] in
-    /// which case `maybe_sticker` flag should be set. We recheck if an image is a true sticker
-    /// assuming that it must have at least one fully transparent corner, otherwise this flag is
-    /// reset.
-    pub async fn recode_to_image_size(
-        &mut self,
-        context: &Context,
-        maybe_sticker: &mut bool,
-    ) -> Result<()> {
+    pub async fn recode_to_image_size(&mut self, context: &Context) -> Result<()> {
         let blob_abs = self.to_abs_path();
         let (img_wh, max_bytes) =
             match MediaQuality::from_i32(context.get_config_int(Config::MediaQuality).await?)
@@ -364,14 +347,9 @@ impl<'a> BlobObject<'a> {
                 MediaQuality::Worse => (constants::WORSE_IMAGE_SIZE, constants::WORSE_IMAGE_BYTES),
             };
         let strict_limits = false;
-        if let Some(new_name) = self.recode_to_size(
-            context,
-            blob_abs,
-            maybe_sticker,
-            img_wh,
-            max_bytes,
-            strict_limits,
-        )? {
+        if let Some(new_name) =
+            self.recode_to_size(context, blob_abs, img_wh, max_bytes, strict_limits)?
+        {
             self.name = new_name;
         }
         Ok(())
@@ -380,37 +358,20 @@ impl<'a> BlobObject<'a> {
     /// If `!strict_limits`, then if `max_bytes` is exceeded, reduce the image to `img_wh` and just
     /// proceed with the result.
     fn recode_to_size(
-        &mut self,
+        &self,
         context: &Context,
         mut blob_abs: PathBuf,
-        maybe_sticker: &mut bool,
         mut img_wh: u32,
         max_bytes: usize,
         strict_limits: bool,
     ) -> Result<Option<String>> {
-        let mut no_exif = false;
-        let no_exif_ref = &mut no_exif;
-        let res = tokio::task::block_in_place(move || {
-            let (nr_bytes, exif) = self.metadata()?;
-            *no_exif_ref = exif.is_none();
+        tokio::task::block_in_place(move || {
             let mut img = image::open(&blob_abs).context("image decode failure")?;
+            let (nr_bytes, exif) = self.metadata()?;
             let orientation = exif.as_ref().map(|exif| exif_orientation(exif, context));
             let mut encoded = Vec::new();
             let mut changed_name = None;
 
-            if *maybe_sticker {
-                let x_max = img.width().saturating_sub(1);
-                let y_max = img.height().saturating_sub(1);
-                *maybe_sticker = img.in_bounds(x_max, y_max)
-                    && (img.get_pixel(0, 0).0[3] == 0
-                        || img.get_pixel(x_max, 0).0[3] == 0
-                        || img.get_pixel(0, y_max).0[3] == 0
-                        || img.get_pixel(x_max, y_max).0[3] == 0);
-            }
-            if *maybe_sticker && exif.is_none() {
-                return Ok(None);
-            }
-
             img = match orientation {
                 Some(90) => img.rotate90(),
                 Some(180) => img.rotate180(),
@@ -508,21 +469,7 @@ impl<'a> BlobObject<'a> {
             }
 
             Ok(changed_name)
-        });
-        match res {
-            Ok(_) => res,
-            Err(err) => {
-                if !strict_limits && no_exif {
-                    warn!(
-                        context,
-                        "Cannot recode image, using original data: {err:#}.",
-                    );
-                    Ok(None)
-                } else {
-                    Err(err)
-                }
-            }
-        }
+        })
     }
 
     /// Returns image file size and Exif.
@@ -913,18 +860,10 @@ mod tests {
             file.metadata().await.unwrap().len()
         }
 
-        let mut blob = BlobObject::new_from_path(&t, &avatar_blob).await.unwrap();
-        let maybe_sticker = &mut false;
+        let blob = BlobObject::new_from_path(&t, &avatar_blob).await.unwrap();
         let strict_limits = true;
-        blob.recode_to_size(
-            &t,
-            blob.to_abs_path(),
-            maybe_sticker,
-            1000,
-            3000,
-            strict_limits,
-        )
-        .unwrap();
+        blob.recode_to_size(&t, blob.to_abs_path(), 1000, 3000, strict_limits)
+            .unwrap();
         assert!(file_size(&avatar_blob).await <= 3000);
         assert!(file_size(&avatar_blob).await > 2000);
         tokio::task::block_in_place(move || {
@@ -984,7 +923,6 @@ mod tests {
     async fn test_recode_image_1() {
         let bytes = include_bytes!("../test-data/image/avatar1000x1000.jpg");
         send_image_check_mediaquality(
-            Viewtype::Image,
             Some("0"),
             bytes,
             "jpg",
@@ -998,7 +936,6 @@ mod tests {
         .await
         .unwrap();
         send_image_check_mediaquality(
-            Viewtype::Image,
             Some("1"),
             bytes,
             "jpg",
@@ -1018,7 +955,6 @@ mod tests {
         // The "-rotated" files are rotated by 270 degrees using the Exif metadata
         let bytes = include_bytes!("../test-data/image/rectangle2000x1800-rotated.jpg");
         let img_rotated = send_image_check_mediaquality(
-            Viewtype::Image,
             Some("0"),
             bytes,
             "jpg",
@@ -1038,7 +974,6 @@ mod tests {
         let bytes = buf.into_inner();
 
         let img_rotated = send_image_check_mediaquality(
-            Viewtype::Image,
             Some("1"),
             &bytes,
             "jpg",
@@ -1059,7 +994,6 @@ mod tests {
         let bytes = include_bytes!("../test-data/image/screenshot.png");
 
         send_image_check_mediaquality(
-            Viewtype::Image,
             Some("0"),
             bytes,
             "png",
@@ -1074,7 +1008,6 @@ mod tests {
         .unwrap();
 
         send_image_check_mediaquality(
-            Viewtype::Image,
             Some("1"),
             bytes,
             "png",
@@ -1087,29 +1020,12 @@ mod tests {
         )
         .await
         .unwrap();
-
-        // This will be sent as Image, see [`BlobObject::maybe_sticker`] for explanation.
-        send_image_check_mediaquality(
-            Viewtype::Sticker,
-            Some("0"),
-            bytes,
-            "png",
-            false, // no Exif
-            1920,
-            1080,
-            0,
-            1920,
-            1080,
-        )
-        .await
-        .unwrap();
     }
 
     #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
     async fn test_recode_image_huge_jpg() {
         let bytes = include_bytes!("../test-data/image/screenshot.jpg");
         send_image_check_mediaquality(
-            Viewtype::Image,
             Some("0"),
             bytes,
             "jpg",
@@ -1143,7 +1059,6 @@ mod tests {
 
     #[allow(clippy::too_many_arguments)]
     async fn send_image_check_mediaquality(
-        viewtype: Viewtype,
         media_quality_config: Option<&str>,
         bytes: &[u8],
         extension: &str,
@@ -1175,7 +1090,7 @@ mod tests {
             assert!(exif.is_none());
         }
 
-        let mut msg = Message::new(viewtype);
+        let mut msg = Message::new(Viewtype::Image);
         msg.set_file(file.to_str().unwrap(), None);
         let chat = alice.create_chat(&bob).await;
         let sent = alice.send_msg(chat.id, &mut msg).await;
@@ -1189,7 +1104,6 @@ mod tests {
         );
 
         let bob_msg = bob.recv_msg(&sent).await;
-        assert_eq!(bob_msg.get_viewtype(), Viewtype::Image);
         assert_eq!(bob_msg.get_width() as u32, compressed_width);
         assert_eq!(bob_msg.get_height() as u32, compressed_height);
         let file = bob_msg.get_file(&bob).unwrap();

src/chatlist.rs

@@ -10,10 +10,8 @@ use crate::constants::{
 use crate::contact::{Contact, ContactId};
 use crate::context::Context;
 use crate::message::{Message, MessageState, MsgId};
-use crate::param::{Param, Params};
 use crate::stock_str;
 use crate::summary::Summary;
-use crate::tools::IsNoneOrEmpty;
 
 /// An object representing a single chatlist in memory.
 ///
@@ -206,84 +204,34 @@ impl Chatlist {
                 )
                 .await?
         } else {
-            let mut ids = if flag_for_forwarding {
-                let sort_id_up = ChatId::lookup_by_contact(context, ContactId::SELF)
+            //  show normal chatlist
+            let sort_id_up = if flag_for_forwarding {
+                ChatId::lookup_by_contact(context, ContactId::SELF)
                     .await?
-                    .unwrap_or_default();
-                let process_row = |row: &rusqlite::Row| {
-                    let chat_id: ChatId = row.get(0)?;
-                    let typ: Chattype = row.get(1)?;
-                    let param: Params = row.get::<_, String>(2)?.parse().unwrap_or_default();
-                    let msg_id: Option<MsgId> = row.get(3)?;
-                    Ok((chat_id, typ, param, msg_id))
-                };
-                let process_rows = |rows: rusqlite::MappedRows<_>| {
-                    rows.filter_map(|row: std::result::Result<(_, _, Params, _), _>| match row {
-                        Ok((chat_id, typ, param, msg_id)) => {
-                            if typ == Chattype::Mailinglist
-                                && param.get(Param::ListPost).is_none_or_empty()
-                            {
-                                None
-                            } else {
-                                Some(Ok((chat_id, msg_id)))
-                            }
-                        }
-                        Err(e) => Some(Err(e)),
-                    })
-                    .collect::<std::result::Result<Vec<_>, _>>()
-                    .map_err(Into::into)
-                };
-                // Return ProtectionBroken chats also, as that may happen to a verified chat at any
-                // time. It may be confusing if a chat that is normally in the list disappears
-                // suddenly. The UI need to deal with that case anyway.
-                context.sql.query_map(
-                    "SELECT c.id, c.type, c.param, m.id
-                     FROM chats c
-                     LEFT JOIN msgs m
-                            ON c.id=m.chat_id
-                           AND m.id=(
-                                   SELECT id
-                                     FROM msgs
-                                    WHERE chat_id=c.id
-                                      AND (hidden=0 OR state=?)
-                                      ORDER BY timestamp DESC, id DESC LIMIT 1)
-                     WHERE c.id>9 AND c.id!=?
-                       AND c.blocked=0
-                       AND NOT c.archived=?
-                       AND (c.type!=? OR c.id IN(SELECT chat_id FROM chats_contacts WHERE contact_id=?))
-                     GROUP BY c.id
-                     ORDER BY c.id=? DESC, c.archived=? DESC, IFNULL(m.timestamp,c.created_timestamp) DESC, m.id DESC;",
-                    (
-                        MessageState::OutDraft, skip_id, ChatVisibility::Archived,
-                        Chattype::Group, ContactId::SELF,
-                        sort_id_up, ChatVisibility::Pinned,
-                    ),
-                    process_row,
-                    process_rows,
-                ).await?
+                    .unwrap_or_default()
             } else {
-                //  show normal chatlist
-                context.sql.query_map(
-                    "SELECT c.id, m.id
-                     FROM chats c
-                     LEFT JOIN msgs m
-                            ON c.id=m.chat_id
-                           AND m.id=(
-                                   SELECT id
-                                     FROM msgs
-                                    WHERE chat_id=c.id
-                                      AND (hidden=0 OR state=?)
-                                      ORDER BY timestamp DESC, id DESC LIMIT 1)
-                     WHERE c.id>9 AND c.id!=?
-                       AND (c.blocked=0 OR c.blocked=2)
-                       AND NOT c.archived=?
-                     GROUP BY c.id
-                     ORDER BY c.id=0 DESC, c.archived=? DESC, IFNULL(m.timestamp,c.created_timestamp) DESC, m.id DESC;",
-                    (MessageState::OutDraft, skip_id, ChatVisibility::Archived, ChatVisibility::Pinned),
-                    process_row,
-                    process_rows,
-                ).await?
+                ChatId::new(0)
             };
+            let mut ids = context.sql.query_map(
+                "SELECT c.id, m.id
+                 FROM chats c
+                 LEFT JOIN msgs m
+                        ON c.id=m.chat_id
+                       AND m.id=(
+                               SELECT id
+                                 FROM msgs
+                                WHERE chat_id=c.id
+                                  AND (hidden=0 OR state=?1)
+                                  ORDER BY timestamp DESC, id DESC LIMIT 1)
+                 WHERE c.id>9 AND c.id!=?2
+                   AND (c.blocked=0 OR (c.blocked=2 AND NOT ?3))
+                   AND NOT c.archived=?4
+                 GROUP BY c.id
+                 ORDER BY c.id=?5 DESC, c.archived=?6 DESC, IFNULL(m.timestamp,c.created_timestamp) DESC, m.id DESC;",
+                (MessageState::OutDraft, skip_id, flag_for_forwarding, ChatVisibility::Archived, sort_id_up, ChatVisibility::Pinned),
+                process_row,
+                process_rows,
+            ).await?;
             if !flag_no_specials && get_archived_cnt(context).await? > 0 {
                 if ids.is_empty() && flag_add_alldone_hint {
                     ids.push((DC_CHAT_ID_ALLDONE_HINT, None));
@@ -296,27 +244,6 @@ impl Chatlist {
         Ok(Chatlist { ids })
     }
 
-    /// Converts list of chat IDs to a chatlist.
-    pub(crate) async fn from_chat_ids(context: &Context, chat_ids: &[ChatId]) -> Result<Self> {
-        let mut ids = Vec::new();
-        for &chat_id in chat_ids {
-            let msg_id: Option<MsgId> = context
-                .sql
-                .query_get_value(
-                    "SELECT id
-                   FROM msgs
-                  WHERE chat_id=?1
-                    AND (hidden=0 OR state=?2)
-                  ORDER BY timestamp DESC, id DESC LIMIT 1",
-                    (chat_id, MessageState::OutDraft),
-                )
-                .await
-                .with_context(|| format!("failed to get msg ID for chat {}", chat_id))?;
-            ids.push((chat_id, msg_id));
-        }
-        Ok(Chatlist { ids })
-    }
-
     /// Find out the number of chats.
     pub fn len(&self) -> usize {
         self.ids.len()
@@ -461,9 +388,7 @@ pub async fn get_last_message_for_chat(
 #[cfg(test)]
 mod tests {
     use super::*;
-    use crate::chat::{
-        create_group_chat, get_chat_contacts, remove_contact_from_chat, ProtectionStatus,
-    };
+    use crate::chat::{create_group_chat, get_chat_contacts, ProtectionStatus};
     use crate::message::Viewtype;
     use crate::receive_imf::receive_imf;
     use crate::stock_str::StockMessage;
@@ -548,14 +473,6 @@ mod tests {
             .await
             .unwrap()
             .is_self_talk());
-
-        remove_contact_from_chat(&t, chats.get_chat_id(1).unwrap(), ContactId::SELF)
-            .await
-            .unwrap();
-        let chats = Chatlist::try_load(&t, DC_GCL_FOR_FORWARDING, None, None)
-            .await
-            .unwrap();
-        assert!(chats.len() == 1);
     }
 
     #[tokio::test(flavor = "multi_thread", worker_threads = 2)]

src/config.rs

@@ -1,7 +1,6 @@
 //! # Key-value configuration management.
 
 use std::env;
-use std::path::Path;
 use std::str::FromStr;
 
 use anyhow::{ensure, Context as _, Result};
@@ -312,6 +311,16 @@ pub enum Config {
 
     /// Last message processed by the bot.
     LastMsgId,
+
+    /// Feature flag for verified 1:1 chats; the UI should set it
+    /// to 1 if it supports verified 1:1 chats.
+    /// Regardless of this setting, `chat.is_protected()` returns true while the key is verified,
+    /// and when the key changes, an info message is posted into the chat.
+    /// 0=Nothing else happens when the key changes.
+    /// 1=After the key changed, `can_send()` returns false and `is_protection_broken()` returns true
+    /// until `chat_id.accept()` is called.
+    #[strum(props(default = "0"))]
+    VerifiedOneOnOneChats,
 }
 
 impl Context {
@@ -330,11 +339,7 @@ impl Context {
         let value = match key {
             Config::Selfavatar => {
                 let rel_path = self.sql.get_raw_config(key.as_ref()).await?;
-                rel_path.map(|p| {
-                    get_abs_path(self, Path::new(&p))
-                        .to_string_lossy()
-                        .into_owned()
-                })
+                rel_path.map(|p| get_abs_path(self, p).to_string_lossy().into_owned())
             }
             Config::SysVersion => Some((*DC_VERSION_STR).clone()),
             Config::SysMsgsizeMaxRecommended => Some(format!("{RECOMMENDED_FILE_SIZE}")),

src/configure.rs

@@ -130,7 +130,7 @@ async fn on_configure_completed(
     old_addr: Option<String>,
 ) -> Result<()> {
     if let Some(provider) = param.provider {
-        if let Some(config_defaults) = provider.config_defaults {
+        if let Some(config_defaults) = &provider.config_defaults {
             for def in config_defaults {
                 if !context.config_exists(def.key).await? {
                     info!(context, "apply config_defaults {}={}", def.key, def.value);

src/constants.rs

@@ -62,15 +62,8 @@ pub enum MediaQuality {
 pub enum KeyGenType {
     #[default]
     Default = 0,
-
-    /// 2048-bit RSA.
     Rsa2048 = 1,
-
-    /// [Ed25519](https://ed25519.cr.yp.to/) signature and X25519 encryption.
     Ed25519 = 2,
-
-    /// 4096-bit RSA.
-    Rsa4096 = 3,
 }
 
 /// Video chat URL type.
@@ -238,7 +231,6 @@ mod tests {
         assert_eq!(KeyGenType::Default, KeyGenType::from_i32(0).unwrap());
         assert_eq!(KeyGenType::Rsa2048, KeyGenType::from_i32(1).unwrap());
         assert_eq!(KeyGenType::Ed25519, KeyGenType::from_i32(2).unwrap());
-        assert_eq!(KeyGenType::Rsa4096, KeyGenType::from_i32(3).unwrap());
     }
 
     #[test]

src/contact.rs

@@ -5,7 +5,7 @@ use std::collections::BinaryHeap;
 use std::convert::{TryFrom, TryInto};
 use std::fmt;
 use std::ops::Deref;
-use std::path::{Path, PathBuf};
+use std::path::PathBuf;
 use std::time::{SystemTime, UNIX_EPOCH};
 
 use anyhow::{bail, ensure, Context as _, Result};
@@ -1186,7 +1186,7 @@ impl Contact {
             }
         } else if let Some(image_rel) = self.param.get(Param::ProfileImage) {
             if !image_rel.is_empty() {
-                return Ok(Some(get_abs_path(context, Path::new(image_rel))));
+                return Ok(Some(get_abs_path(context, image_rel)));
             }
         }
         Ok(None)
@@ -1213,28 +1213,13 @@ impl Contact {
     /// The UI may draw a checkbox or something like that beside verified contacts.
     ///
     pub async fn is_verified(&self, context: &Context) -> Result<VerifiedStatus> {
-        self.is_verified_ex(context, None).await
-    }
-
-    /// Same as `Contact::is_verified` but allows speeding up things
-    /// by adding the peerstate belonging to the contact.
-    /// If you do not have the peerstate available, it is loaded automatically.
-    pub async fn is_verified_ex(
-        &self,
-        context: &Context,
-        peerstate: Option<&Peerstate>,
-    ) -> Result<VerifiedStatus> {
         // We're always sort of secured-verified as we could verify the key on this device any time with the key
         // on this device
         if self.id == ContactId::SELF {
             return Ok(VerifiedStatus::BidirectVerified);
         }
 
-        if let Some(peerstate) = peerstate {
-            if peerstate.verified_key.is_some() {
-                return Ok(VerifiedStatus::BidirectVerified);
-            }
-        } else if let Some(peerstate) = Peerstate::from_addr(context, &self.addr).await? {
+        if let Some(peerstate) = Peerstate::from_addr(context, &self.addr).await? {
             if peerstate.verified_key.is_some() {
                 return Ok(VerifiedStatus::BidirectVerified);
             }
@@ -1732,7 +1717,7 @@ mod tests {
         assert_eq!(may_be_valid_addr("dd.tt"), false);
         assert_eq!(may_be_valid_addr("tt.dd@uu"), true);
         assert_eq!(may_be_valid_addr("u@d"), true);
-        assert_eq!(may_be_valid_addr("u@d."), false);
+        assert_eq!(may_be_valid_addr("u@d."), true);
         assert_eq!(may_be_valid_addr("u@d.t"), true);
         assert_eq!(may_be_valid_addr("u@d.tt"), true);
         assert_eq!(may_be_valid_addr("u@.tt"), true);
@@ -1741,7 +1726,6 @@ mod tests {
         assert_eq!(may_be_valid_addr("sk <@d.tt>"), false);
         assert_eq!(may_be_valid_addr("as@sd.de>"), false);
         assert_eq!(may_be_valid_addr("ask dkl@dd.tt"), false);
-        assert_eq!(may_be_valid_addr("user@domain.tld."), false);
     }
 
     #[test]

src/context.rs

@@ -332,12 +332,6 @@ impl Context {
         }
     }
 
-    /// Changes encrypted database passphrase.
-    pub async fn change_passphrase(&self, passphrase: String) -> Result<()> {
-        self.sql.change_passphrase(passphrase).await?;
-        Ok(())
-    }
-
     /// Returns true if database is open.
     pub async fn is_open(&self) -> bool {
         self.sql.is_open().await
@@ -761,6 +755,12 @@ impl Context {
             "last_msg_id",
             self.get_config_int(Config::LastMsgId).await?.to_string(),
         );
+        res.insert(
+            "verified_one_on_one_chats",
+            self.get_config_bool(Config::VerifiedOneOnOneChats)
+                .await?
+                .to_string(),
+        );
 
         let elapsed = self.creation_time.elapsed();
         res.insert("uptime", duration_to_str(elapsed.unwrap_or_default()));
@@ -1459,35 +1459,6 @@ mod tests {
         Ok(())
     }
 
-    #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
-    async fn test_context_change_passphrase() -> Result<()> {
-        let dir = tempdir()?;
-        let dbfile = dir.path().join("db.sqlite");
-
-        let id = 1;
-        let context = Context::new_closed(&dbfile, id, Events::new(), StockStrings::new())
-            .await
-            .context("failed to create context")?;
-        assert_eq!(context.open("foo".to_string()).await?, true);
-        assert_eq!(context.is_open().await, true);
-
-        context
-            .set_config(Config::Addr, Some("alice@example.org"))
-            .await?;
-
-        context
-            .change_passphrase("bar".to_string())
-            .await
-            .context("Failed to change passphrase")?;
-
-        assert_eq!(
-            context.get_config(Config::Addr).await?.unwrap(),
-            "alice@example.org"
-        );
-
-        Ok(())
-    }
-
     #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
     async fn test_ongoing() -> Result<()> {
         let context = TestContext::new().await;

src/decrypt.rs

@@ -13,7 +13,6 @@ use crate::contact::addr_cmp;
 use crate::context::Context;
 use crate::headerdef::{HeaderDef, HeaderDefMap};
 use crate::key::{DcKey, Fingerprint, SignedPublicKey, SignedSecretKey};
-use crate::keyring::Keyring;
 use crate::peerstate::Peerstate;
 use crate::pgp;
 
@@ -26,33 +25,17 @@ use crate::pgp;
 pub fn try_decrypt(
     context: &Context,
     mail: &ParsedMail<'_>,
-    private_keyring: &Keyring<SignedSecretKey>,
-    public_keyring_for_validate: &Keyring<SignedPublicKey>,
+    private_keyring: &[SignedSecretKey],
+    public_keyring_for_validate: &[SignedPublicKey],
 ) -> Result<Option<(Vec<u8>, HashSet<Fingerprint>)>> {
-    let encrypted_data_part = match {
-        let mime = get_autocrypt_mime(mail);
-        if mime.is_some() {
-            info!(context, "Detected Autocrypt-mime message.");
-        }
-        mime
-    }
-    .or_else(|| {
-        let mime = get_mixed_up_mime(mail);
-        if mime.is_some() {
-            info!(context, "Detected mixed-up mime message.");
-        }
-        mime
-    })
-    .or_else(|| {
-        let mime = get_attachment_mime(mail);
-        if mime.is_some() {
-            info!(context, "Detected attached Autocrypt-mime message.");
-        }
-        mime
-    }) {
+    let encrypted_data_part = match get_autocrypt_mime(mail)
+        .or_else(|| get_mixed_up_mime(mail))
+        .or_else(|| get_attachment_mime(mail))
+    {
         None => return Ok(None),
         Some(res) => res,
     };
+    info!(context, "Detected Autocrypt-mime message");
 
     decrypt_part(
         encrypted_data_part,
@@ -227,8 +210,8 @@ fn get_autocrypt_mime<'a, 'b>(mail: &'a ParsedMail<'b>) -> Option<&'a ParsedMail
 /// Returns Ok(None) if nothing encrypted was found.
 fn decrypt_part(
     mail: &ParsedMail<'_>,
-    private_keyring: &Keyring<SignedSecretKey>,
-    public_keyring_for_validate: &Keyring<SignedPublicKey>,
+    private_keyring: &[SignedSecretKey],
+    public_keyring_for_validate: &[SignedPublicKey],
 ) -> Result<Option<(Vec<u8>, HashSet<Fingerprint>)>> {
     let data = mail.get_body_raw()?;
 
@@ -263,7 +246,7 @@ fn has_decrypted_pgp_armor(input: &[u8]) -> bool {
 /// Returns None if the message is not Multipart/Signed or doesn't contain necessary parts.
 pub(crate) fn validate_detached_signature<'a, 'b>(
     mail: &'a ParsedMail<'b>,
-    public_keyring_for_validate: &Keyring<SignedPublicKey>,
+    public_keyring_for_validate: &[SignedPublicKey],
 ) -> Option<(&'a ParsedMail<'b>, HashSet<Fingerprint>)> {
     if mail.ctype.mimetype != "multipart/signed" {
         return None;
@@ -283,13 +266,13 @@ pub(crate) fn validate_detached_signature<'a, 'b>(
     }
 }
 
-pub(crate) fn keyring_from_peerstate(peerstate: Option<&Peerstate>) -> Keyring<SignedPublicKey> {
-    let mut public_keyring_for_validate: Keyring<SignedPublicKey> = Keyring::new();
+pub(crate) fn keyring_from_peerstate(peerstate: Option<&Peerstate>) -> Vec<SignedPublicKey> {
+    let mut public_keyring_for_validate = Vec::new();
     if let Some(peerstate) = peerstate {
         if let Some(key) = &peerstate.public_key {
-            public_keyring_for_validate.add(key.clone());
+            public_keyring_for_validate.push(key.clone());
         } else if let Some(key) = &peerstate.gossip_key {
-            public_keyring_for_validate.add(key.clone());
+            public_keyring_for_validate.push(key.clone());
         }
     }
     public_keyring_for_validate
@@ -419,18 +402,4 @@ mod tests {
 
         Ok(())
     }
-
-    #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
-    async fn test_mixed_up_mime_long() -> Result<()> {
-        // Long "mixed-up" mail as received when sending an encrypted message using Delta Chat
-        // Desktop via MS Exchange (actually made with TB though).
-        let mixed_up_mime = include_bytes!("../test-data/message/mixed-up-long.eml");
-        let bob = TestContext::new_bob().await;
-        receive_imf(&bob, mixed_up_mime, false).await?;
-        let msg = bob.get_last_msg().await;
-        assert!(!msg.get_text().is_empty());
-        assert!(msg.has_html());
-        assert!(msg.id.get_html(&bob).await?.unwrap().len() > 40000);
-        Ok(())
-    }
 }

src/download.rs

@@ -59,9 +59,6 @@ pub enum DownloadState {
     /// Failed to fully download the message.
     Failure = 20,
 
-    /// Undecipherable message.
-    Undecipherable = 30,
-
     /// Full download of the message is in progress.
     InProgress = 1000,
 }
@@ -83,9 +80,7 @@ impl MsgId {
     pub async fn download_full(self, context: &Context) -> Result<()> {
         let msg = Message::load_from_db(context, self).await?;
         match msg.download_state() {
-            DownloadState::Done | DownloadState::Undecipherable => {
-                return Err(anyhow!("Nothing to download."))
-            }
+            DownloadState::Done => return Err(anyhow!("Nothing to download.")),
             DownloadState::InProgress => return Err(anyhow!("Download already in progress.")),
             DownloadState::Available | DownloadState::Failure => {
                 self.update_download_state(context, DownloadState::InProgress)

src/e2ee.rs

@@ -7,7 +7,6 @@ use crate::aheader::{Aheader, EncryptPreference};
 use crate::config::Config;
 use crate::context::Context;
 use crate::key::{DcKey, SignedPublicKey, SignedSecretKey};
-use crate::keyring::Keyring;
 use crate::peerstate::{Peerstate, PeerstateVerifiedStatus};
 use crate::pgp;
 
@@ -104,7 +103,7 @@ impl EncryptHelper {
         mail_to_encrypt: lettre_email::PartBuilder,
         peerstates: Vec<(Option<Peerstate>, &str)>,
     ) -> Result<String> {
-        let mut keyring: Keyring<SignedPublicKey> = Keyring::new();
+        let mut keyring: Vec<SignedPublicKey> = Vec::new();
 
         for (peerstate, addr) in peerstates
             .into_iter()
@@ -113,9 +112,9 @@ impl EncryptHelper {
             let key = peerstate
                 .take_key(min_verified)
                 .with_context(|| format!("proper enc-key for {addr} missing, cannot encrypt"))?;
-            keyring.add(key);
+            keyring.push(key);
         }
-        keyring.add(self.public_key.clone());
+        keyring.push(self.public_key.clone());
         let sign_key = SignedSecretKey::load_self(context).await?;
 
         let raw_message = mail_to_encrypt.build().as_string().into_bytes();

src/ephemeral.rs

@@ -545,7 +545,7 @@ async fn next_expiration_timestamp(context: &Context) -> Option<i64> {
 
     ephemeral_timestamp
         .into_iter()
-        .chain(delete_device_after_timestamp.into_iter())
+        .chain(delete_device_after_timestamp)
         .min()
 }
 
@@ -1062,14 +1062,14 @@ mod tests {
         delete_expired_messages(t, not_deleted_at).await?;
 
         let loaded = Message::load_from_db(t, msg_id).await?;
-        assert!(!loaded.text.is_empty());
+        assert_eq!(loaded.text, "Message text");
         assert_eq!(loaded.chat_id, chat.id);
 
         assert!(next_expiration < deleted_at);
         delete_expired_messages(t, deleted_at).await?;
         t.evtracker
             .get_matching(|evt| {
-                if let EventType::MsgDeleted {
+                if let EventType::MsgsChanged {
                     msg_id: event_msg_id,
                     ..
                 } = evt
@@ -1082,6 +1082,7 @@ mod tests {
             .await;
 
         let loaded = Message::load_from_db(t, msg_id).await?;
+        assert_eq!(loaded.text, "");
         assert_eq!(loaded.chat_id, DC_CHAT_ID_TRASH);
 
         // Check that the msg was deleted locally.
@@ -1299,32 +1300,4 @@ mod tests {
 
         Ok(())
     }
-
-    // Tests that if we are offline for a time longer than the ephemeral timer duration, the message
-    // is deleted from the chat but is still in the "smtp" table, i.e. will be sent upon a
-    // successful reconnection.
-    #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
-    async fn test_ephemeral_msg_offline() -> Result<()> {
-        let alice = TestContext::new_alice().await;
-        let chat = alice
-            .create_chat_with_contact("Bob", "bob@example.org")
-            .await;
-        let duration = 60;
-        chat.id
-            .set_ephemeral_timer(&alice, Timer::Enabled { duration })
-            .await?;
-        let mut msg = Message::new(Viewtype::Text);
-        msg.set_text("hi".to_string());
-        assert!(chat::send_msg_sync(&alice, chat.id, &mut msg)
-            .await
-            .is_err());
-        let stmt = "SELECT COUNT(*) FROM smtp WHERE msg_id=?";
-        assert!(alice.sql.exists(stmt, (msg.id,)).await?);
-        let now = time();
-        check_msg_will_be_deleted(&alice, msg.id, &chat, now, now + i64::from(duration) + 1)
-            .await?;
-        assert!(alice.sql.exists(stmt, (msg.id,)).await?);
-
-        Ok(())
-    }
 }

src/imex.rs

@@ -586,74 +586,63 @@ async fn export_backup_inner(
     Ok(())
 }
 
-/// Imports secret key from a file.
-async fn import_secret_key(context: &Context, path: &Path, set_default: bool) -> Result<()> {
-    let buf = read_file(context, &path).await?;
-    let armored = std::string::String::from_utf8_lossy(&buf);
-    set_self_key(context, &armored, set_default, false).await?;
-    Ok(())
-}
-
-/// Imports secret keys from the provided file or directory.
-///
-/// If provided path is a file, ASCII-armored secret key is read from the file
-/// and set as the default key.
-///
-/// If provided path is a directory, all files with .asc extension
-/// containing secret keys are imported and the last successfully
-/// imported which does not contain "legacy" in its filename
-/// is set as the default.
-async fn import_self_keys(context: &Context, path: &Path) -> Result<()> {
-    let attr = tokio::fs::metadata(path).await?;
-
-    if attr.is_file() {
-        info!(
-            context,
-            "Importing secret key from {} as the default key.",
-            path.display()
-        );
-        let set_default = true;
-        import_secret_key(context, path, set_default).await?;
-        return Ok(());
-    }
+/*******************************************************************************
+ * Classic key import
+ ******************************************************************************/
+async fn import_self_keys(context: &Context, dir: &Path) -> Result<()> {
+    /* hint: even if we switch to import Autocrypt Setup Files, we should leave the possibility to import
+    plain ASC keys, at least keys without a password, if we do not want to implement a password entry function.
+    Importing ASC keys is useful to use keys in Delta Chat used by any other non-Autocrypt-PGP implementation.
 
+    Maybe we should make the "default" key handlong also a little bit smarter
+    (currently, the last imported key is the standard key unless it contains the string "legacy" in its name) */
+    let mut set_default: bool;
     let mut imported_cnt = 0;
 
-    let mut dir_handle = tokio::fs::read_dir(&path).await?;
+    let dir_name = dir.to_string_lossy();
+    let mut dir_handle = tokio::fs::read_dir(&dir).await?;
     while let Ok(Some(entry)) = dir_handle.next_entry().await {
         let entry_fn = entry.file_name();
         let name_f = entry_fn.to_string_lossy();
-        let path_plus_name = path.join(&entry_fn);
-        if let Some(suffix) = get_filesuffix_lc(&name_f) {
-            if suffix != "asc" {
+        let path_plus_name = dir.join(&entry_fn);
+        match get_filesuffix_lc(&name_f) {
+            Some(suffix) => {
+                if suffix != "asc" {
+                    continue;
+                }
+                set_default = if name_f.contains("legacy") {
+                    info!(context, "found legacy key '{}'", path_plus_name.display());
+                    false
+                } else {
+                    true
+                }
+            }
+            None => {
                 continue;
             }
-        } else {
-            continue;
-        };
-        let set_default = !name_f.contains("legacy");
+        }
         info!(
             context,
-            "Considering key file: {}.",
+            "considering key file: {}",
             path_plus_name.display()
         );
 
-        if let Err(err) = import_secret_key(context, &path_plus_name, set_default).await {
-            warn!(
-                context,
-                "Failed to import secret key from {}: {:#}.",
-                path_plus_name.display(),
-                err
-            );
-            continue;
+        match read_file(context, &path_plus_name).await {
+            Ok(buf) => {
+                let armored = std::string::String::from_utf8_lossy(&buf);
+                if let Err(err) = set_self_key(context, &armored, set_default, false).await {
+                    info!(context, "set_self_key: {}", err);
+                    continue;
+                }
+            }
+            Err(_) => continue,
         }
-
         imported_cnt += 1;
     }
     ensure!(
         imported_cnt > 0,
-        "No private keys found in {}.",
-        path.display()
+        "No private keys found in \"{}\".",
+        dir_name
     );
     Ok(())
 }
@@ -684,8 +673,7 @@ async fn export_self_keys(context: &Context, dir: &Path) -> Result<()> {
         .await?;
 
     for (id, public_key, private_key, is_default) in keys {
-        let id = Some(id).filter(|_| is_default == 0);
-
+        let id = Some(id).filter(|_| is_default != 0);
         if let Ok(key) = public_key {
             if let Err(err) = export_key_to_asc_file(context, dir, id, &key).await {
                 error!(context, "Failed to export public key: {:#}.", err);
@@ -876,35 +864,14 @@ mod tests {
 
     #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
     async fn test_export_and_import_key() {
-        let export_dir = tempfile::tempdir().unwrap();
-
         let context = TestContext::new_alice().await;
-        if let Err(err) = imex(
-            &context.ctx,
-            ImexMode::ExportSelfKeys,
-            export_dir.path(),
-            None,
-        )
-        .await
-        {
+        let blobdir = context.ctx.get_blobdir();
+        if let Err(err) = imex(&context.ctx, ImexMode::ExportSelfKeys, blobdir, None).await {
             panic!("got error on export: {err:#}");
         }
 
         let context2 = TestContext::new_alice().await;
-        if let Err(err) = imex(
-            &context2.ctx,
-            ImexMode::ImportSelfKeys,
-            export_dir.path(),
-            None,
-        )
-        .await
-        {
-            panic!("got error on import: {err:#}");
-        }
-
-        let keyfile = export_dir.path().join("private-key-default.asc");
-        let context3 = TestContext::new_alice().await;
-        if let Err(err) = imex(&context3.ctx, ImexMode::ImportSelfKeys, &keyfile, None).await {
+        if let Err(err) = imex(&context2.ctx, ImexMode::ImportSelfKeys, blobdir, None).await {
             panic!("got error on import: {err:#}");
         }
     }

src/imex/transfer.rs

@@ -27,21 +27,31 @@ use std::net::Ipv4Addr;
 use std::ops::Deref;
 use std::path::{Path, PathBuf};
 use std::pin::Pin;
+use std::sync::atomic::{AtomicU64, Ordering};
+use std::sync::Arc;
 use std::task::Poll;
 
 use anyhow::{anyhow, bail, ensure, format_err, Context as _, Result};
 use async_channel::Receiver;
+use bytes::Bytes;
+use futures::FutureExt;
 use futures_lite::StreamExt;
-use iroh::blobs::Collection;
-use iroh::get::DataStream;
-use iroh::progress::ProgressEmitter;
-use iroh::protocol::AuthToken;
-use iroh::provider::{DataSource, Event, Provider, Ticket};
-use iroh::Hash;
-use tokio::fs::{self, File};
-use tokio::io::{self, AsyncWriteExt, BufWriter};
-use tokio::sync::broadcast::error::RecvError;
-use tokio::sync::{broadcast, Mutex};
+use iroh::bytes::get::{fsm, Stats};
+use iroh::bytes::protocol::{AnyGetRequest, GetRequest, RequestToken};
+use iroh::bytes::provider::Event as ProviderEvent;
+use iroh::bytes::util::runtime;
+use iroh::collection::{Collection, IrohCollectionParser};
+use iroh::database::flat::DataSource;
+use iroh::dial::Ticket;
+use iroh::net::tls::Keypair;
+use iroh::node::{Event, Node as IrohNode, StaticTokenAuthHandler};
+use iroh::util::progress::ProgressEmitter;
+use iroh_io::AsyncSliceWriter;
+use tokio::fs;
+use tokio::sync::{
+    broadcast::{self, error::RecvError},
+    Mutex,
+};
 use tokio::task::{JoinHandle, JoinSet};
 use tokio_stream::wrappers::ReadDirStream;
 use tokio_util::sync::CancellationToken;
@@ -56,7 +66,7 @@ use crate::{e2ee, EventType};
 
 use super::{export_database, DBFILE_BACKUP_NAME};
 
-const MAX_CONCURRENT_DIALS: u8 = 16;
+type Node = IrohNode<iroh::database::flat::Database>;
 
 /// Provide or send a backup of this device.
 ///
@@ -154,9 +164,9 @@ impl BackupProvider {
     /// Creates the provider task.
     ///
     /// Having this as a function makes it easier to cancel it when needed.
-    async fn prepare_inner(context: &Context, dbfile: &Path) -> Result<(Provider, Ticket)> {
+    async fn prepare_inner(context: &Context, dbfile: &Path) -> Result<(Node, Ticket)> {
         // Generate the token up front: we also use it to encrypt the database.
-        let token = AuthToken::generate();
+        let token = RequestToken::generate();
         context.emit_event(SendProgress::Started.into());
         export_database(context, dbfile, token.to_string())
             .await
@@ -176,19 +186,24 @@ impl BackupProvider {
         }
 
         // Start listening.
-        let (db, hash) = iroh::provider::create_collection(files).await?;
+        let (db, hash) = iroh::database::flat::create_collection(files).await?;
         context.emit_event(SendProgress::CollectionCreated.into());
-        let provider = Provider::builder(db)
+        let auth_token_handler = StaticTokenAuthHandler::new(Some(token.clone()));
+        let rt = runtime::Handle::from_currrent(1)?;
+        let provider = Node::builder(db)
             .bind_addr((Ipv4Addr::UNSPECIFIED, 0).into())
-            .auth_token(token)
-            .spawn()?;
+            .custom_auth_handler(Arc::new(auth_token_handler))
+            .collection_parser(IrohCollectionParser)
+            .runtime(&rt)
+            .spawn()
+            .await?;
         context.emit_event(SendProgress::ProviderListening.into());
         info!(context, "Waiting for remote to connect");
-        let ticket = provider.ticket(hash)?;
+        let ticket = provider.ticket(hash).await?.with_token(Some(token));
         Ok((provider, ticket))
     }
 
-    /// Supervises the iroh [`Provider`], terminating it when needed.
+    /// Supervises the iroh [`Node`], terminating it when needed.
     ///
     /// This will watch the provider and terminate it when:
     ///
@@ -200,67 +215,80 @@ impl BackupProvider {
     /// we must cancel this operation.
     async fn watch_provider(
         context: &Context,
-        mut provider: Provider,
+        mut provider: Node,
         cancel_token: Receiver<()>,
         drop_token: CancellationToken,
     ) -> Result<()> {
-        let mut events = provider.subscribe();
-        let mut total_size = 0;
-        let mut current_size = 0;
+        let total_size = Arc::new(AtomicU64::new(0));
+        let current_size = Arc::new(AtomicU64::new(0));
+        let (transfer_done, mut transfer_done_r) = tokio::sync::mpsc::channel(1);
+
+        let ctx = context.clone();
+        provider
+            .subscribe(move |event| {
+                let total_size = total_size.clone();
+                let current_size = current_size.clone();
+                let transfer_done = transfer_done.clone();
+                let context = ctx.clone();
+                async move {
+                    match event {
+                        Event::ByteProvide(event) => match event {
+                            ProviderEvent::ClientConnected { .. } => {
+                                context.emit_event(SendProgress::ClientConnected.into());
+                            }
+                            ProviderEvent::GetRequestReceived { .. } => {}
+                            ProviderEvent::TransferCollectionStarted {
+                                total_blobs_size, ..
+                            } => {
+                                total_size
+                                    .store(total_blobs_size.unwrap_or_default(), Ordering::Relaxed);
+                                context.emit_event(
+                                    SendProgress::TransferInProgress {
+                                        current_size: current_size.load(Ordering::Relaxed),
+                                        total_size: total_size.load(Ordering::Relaxed),
+                                    }
+                                    .into(),
+                                );
+                            }
+                            ProviderEvent::TransferBlobCompleted { size, .. } => {
+                                current_size.fetch_add(size, Ordering::Relaxed);
+                                context.emit_event(
+                                    SendProgress::TransferInProgress {
+                                        current_size: current_size.load(Ordering::Relaxed),
+                                        total_size: total_size.load(Ordering::Relaxed),
+                                    }
+                                    .into(),
+                                );
+                            }
+                            ProviderEvent::TransferCollectionCompleted { .. } => {
+                                let total_size = total_size.load(Ordering::Relaxed);
+                                context.emit_event(
+                                    SendProgress::TransferInProgress {
+                                        current_size: total_size,
+                                        total_size,
+                                    }
+                                    .into(),
+                                );
+                                transfer_done.send(()).await.ok();
+                            }
+                            ProviderEvent::TransferAborted { .. } => {
+                                transfer_done.send(()).await.ok();
+                            }
+                            ProviderEvent::CollectionAdded { .. } => {}
+                            ProviderEvent::CustomGetRequestReceived { .. } => {}
+                        },
+                    }
+                }
+                .boxed()
+            })
+            .await?;
+
         let res = loop {
             tokio::select! {
                 biased;
                 res = &mut provider => {
                     break res.context("BackupProvider failed");
-                },
-                maybe_event = events.recv() => {
-                    match maybe_event {
-                        Ok(event) => {
-                            match event {
-                                Event::ClientConnected { ..} => {
-                                    context.emit_event(SendProgress::ClientConnected.into());
-                                }
-                                Event::RequestReceived { .. } => {
-                                }
-                                Event::TransferCollectionStarted { total_blobs_size, .. } => {
-                                    total_size = total_blobs_size;
-                                    context.emit_event(SendProgress::TransferInProgress {
-                                        current_size,
-                                        total_size,
-                                    }.into());
-                                }
-                                Event::TransferBlobCompleted { size, .. } => {
-                                    current_size += size;
-                                    context.emit_event(SendProgress::TransferInProgress {
-                                        current_size,
-                                        total_size,
-                                    }.into());
-                                }
-                                Event::TransferCollectionCompleted { .. } => {
-                                    context.emit_event(SendProgress::TransferInProgress {
-                                        current_size: total_size,
-                                        total_size
-                                    }.into());
-                                    provider.shutdown();
-                                }
-                                Event::TransferAborted { .. } => {
-                                    provider.shutdown();
-                                    break Err(anyhow!("BackupProvider transfer aborted"));
-                                }
-                            }
-                        }
-                        Err(broadcast::error::RecvError::Closed) => {
-                            // We should never see this, provider.join() should complete
-                            // first.
-                        }
-                        Err(broadcast::error::RecvError::Lagged(_)) => {
-                            // We really shouldn't be lagging, if we did we may have missed
-                            // a completion event.
-                            provider.shutdown();
-                            break Err(anyhow!("Missed events from BackupProvider"));
-                        }
-                    }
-                },
+                }
                 _ = cancel_token.recv() => {
                     provider.shutdown();
                     break Err(anyhow!("BackupProvider cancelled"));
@@ -269,6 +297,10 @@ impl BackupProvider {
                     provider.shutdown();
                     break Err(anyhow!("BackupProvider dropped"));
                 }
+                _ = transfer_done_r.recv() => {
+                    provider.shutdown();
+                    break Ok(());
+                }
             }
         };
         match &res {
@@ -382,7 +414,7 @@ impl From<SendProgress> for EventType {
 /// This is a long running operation which will only when completed.
 ///
 /// Using [`Qr`] as argument is a bit odd as it only accepts one specific variant of it.  It
-/// does avoid having [`iroh::provider::Ticket`] in the primary API however, without
+/// does avoid having [`iroh::dial::Ticket`] in the primary API however, without
 /// having to revert to untyped bytes.
 pub async fn get_backup(context: &Context, qr: Qr) -> Result<()> {
     ensure!(
@@ -441,30 +473,11 @@ async fn get_backup_inner(context: &Context, qr: Qr) -> Result<()> {
 async fn transfer_from_provider(context: &Context, ticket: &Ticket) -> Result<()> {
     let progress = ProgressEmitter::new(0, ReceiveProgress::max_blob_progress());
     spawn_progress_proxy(context.clone(), progress.subscribe());
-    let on_connected = || {
-        context.emit_event(ReceiveProgress::Connected.into());
-        async { Ok(()) }
-    };
-    let on_collection = |collection: &Collection| {
-        context.emit_event(ReceiveProgress::CollectionReceived.into());
-        progress.set_total(collection.total_blobs_size());
-        async { Ok(()) }
-    };
+
     let jobs = Mutex::new(JoinSet::default());
-    let on_blob =
-        |hash, reader, name| on_blob(context, &progress, &jobs, ticket, hash, reader, name);
 
     // Perform the transfer.
-    let keylog = false; // Do not enable rustls SSLKEYLOGFILE env var functionality
-    let stats = iroh::get::run_ticket(
-        ticket,
-        keylog,
-        MAX_CONCURRENT_DIALS,
-        on_connected,
-        on_collection,
-        on_blob,
-    )
-    .await?;
+    let stats = run_get_request(context, &progress, &jobs, ticket.clone()).await?;
 
     let mut jobs = jobs.lock().await;
     while let Some(job) = jobs.join_next().await {
@@ -479,19 +492,78 @@ async fn transfer_from_provider(context: &Context, ticket: &Ticket) -> Result<()
     Ok(())
 }
 
+/// Run the get request
+async fn run_get_request(
+    context: &Context,
+    progress: &ProgressEmitter,
+    jobs: &Mutex<JoinSet<()>>,
+    ticket: Ticket,
+) -> anyhow::Result<Stats> {
+    // DERP usage for NAT traversal and relay are currently disabled.
+    let derp_map = None;
+
+    let opts = ticket.as_get_options(Keypair::generate(), derp_map);
+    let request =
+        AnyGetRequest::Get(GetRequest::all(ticket.hash())).with_token(ticket.token().cloned());
+    let connection = iroh::dial::dial(opts).await?;
+    let initial = fsm::start(connection, request);
+
+    let connected = initial.next().await?;
+    context.emit_event(ReceiveProgress::Connected.into());
+
+    let rt = runtime::Handle::from_currrent(1)?;
+
+    // we assume that the request includes the entire collection
+    let (mut next, _root, collection) = {
+        let fsm::ConnectedNext::StartRoot(sc) = connected.next().await? else {
+            bail!("request did not include collection");
+        };
+
+        let (done, data) = sc.next().concatenate_into_vec().await?;
+        let data = Bytes::from(data);
+        let collection = Collection::from_bytes(&data)?;
+
+        context.emit_event(ReceiveProgress::CollectionReceived.into());
+        progress.set_total(collection.total_blobs_size());
+
+        (done.next(), data, collection)
+    };
+
+    // download all the children
+    let mut blobs = collection.blobs().iter();
+    let finishing = loop {
+        let start = match next {
+            fsm::EndBlobNext::MoreChildren(start) => start,
+            fsm::EndBlobNext::Closing(finishing) => break finishing,
+        };
+
+        // get the hash of the next blob, or finish if there are no more
+        let Some(blob) = blobs.next() else {
+            break start.finish();
+        };
+
+        let start = start.next(blob.hash);
+        let done = on_blob(context, &rt, jobs, &ticket, start, &blob.name).await?;
+
+        next = done.next();
+    };
+    let stats = finishing.next().await?;
+
+    Ok(stats)
+}
+
 /// Get callback when a blob is received from the provider.
 ///
 /// This writes the blobs to the blobdir.  If the blob is the database it will import it to
 /// the database of the current [`Context`].
 async fn on_blob(
     context: &Context,
-    progress: &ProgressEmitter,
+    rt: &runtime::Handle,
     jobs: &Mutex<JoinSet<()>>,
     ticket: &Ticket,
-    _hash: Hash,
-    mut reader: DataStream,
-    name: String,
-) -> Result<DataStream> {
+    state: fsm::AtBlobHeader,
+    name: &str,
+) -> Result<fsm::AtEndBlob> {
     ensure!(!name.is_empty(), "Received a nameless blob");
     let path = if name.starts_with("db/") {
         let context_dir = context
@@ -510,15 +582,27 @@ async fn on_blob(
         context.get_blobdir().join(blobname)
     };
 
-    let mut wrapped_reader = progress.wrap_async_read(&mut reader);
-    let file = File::create(&path).await?;
-    let mut file = BufWriter::with_capacity(128 * 1024, file);
-    io::copy(&mut wrapped_reader, &mut file).await?;
-    file.flush().await?;
+    // `iroh_io` io needs to be done on a local spawn
+    let file_path = path.clone();
+    let done = rt
+        .local_pool()
+        .spawn_pinned(move || {
+            let file_path = file_path.clone();
+            Box::pin(async move {
+                let mut file =
+                    iroh_io::File::create(move || std::fs::File::create(&file_path)).await?;
+                // TODO: ProgressEmitter doesn't support writers :(
+                // let mut wrapped_file = progress.wrap_async_write(&mut file);
+                let done = state.write_all(&mut file).await?;
+                file.sync().await?;
+                anyhow::Ok(done)
+            })
+        })
+        .await??;
 
     if name.starts_with("db/") {
         let context = context.clone();
-        let token = ticket.token().to_string();
+        let token = ticket.token().map(|t| t.to_string()).unwrap_or_default();
         jobs.lock().await.spawn(async move {
             if let Err(err) = context.sql.import(&path, token).await {
                 error!(context, "cannot import database: {:#?}", err);
@@ -533,7 +617,8 @@ async fn on_blob(
             }
         });
     }
-    Ok(reader)
+
+    Ok(done)
 }
 
 /// Spawns a task proxying progress events.
@@ -650,9 +735,7 @@ mod tests {
             _ => panic!("wrong chat item"),
         };
         let msg = Message::load_from_db(&ctx1, *msgid).await.unwrap();
-
         let path = msg.get_file(&ctx1).unwrap();
-        assert_eq!(path.with_file_name("hello.txt"), path);
         let text = fs::read_to_string(&path).await.unwrap();
         assert_eq!(text, "i am attachment");
 

src/job.rs

@@ -155,8 +155,8 @@ impl<'a> Connection<'a> {
 pub(crate) async fn perform_job(context: &Context, mut connection: Connection<'_>, mut job: Job) {
     info!(context, "Job {} started...", &job);
 
-    let try_res = match perform_job_action(context, &job, &mut connection, 0).await {
-        Status::RetryNow => perform_job_action(context, &job, &mut connection, 1).await,
+    let try_res = match perform_job_action(context, &mut job, &mut connection, 0).await {
+        Status::RetryNow => perform_job_action(context, &mut job, &mut connection, 1).await,
         x => x,
     };
 
@@ -205,7 +205,7 @@ pub(crate) async fn perform_job(context: &Context, mut connection: Connection<'_
 
 async fn perform_job_action(
     context: &Context,
-    job: &Job,
+    job: &mut Job,
     connection: &mut Connection<'_>,
     tries: u32,
 ) -> Status {

src/key.rs

@@ -260,7 +260,7 @@ pub(crate) async fn load_keypair(
     })
 }
 
-/// Use of a key pair for encryption or decryption.
+/// Use of a [KeyPair] for encryption or decryption.
 ///
 /// This is used by [store_self_keypair] to know what kind of key is
 /// being saved.

src/keyring.rs

@@ -1,87 +0,0 @@
-//! Keyring to perform rpgp operations with.
-
-use anyhow::Result;
-
-use crate::context::Context;
-use crate::key::DcKey;
-
-/// An in-memory keyring.
-///
-/// Instances are usually constructed just for the rpgp operation and
-/// short-lived.
-#[derive(Clone, Debug, Default)]
-pub struct Keyring<T>
-where
-    T: DcKey,
-{
-    keys: Vec<T>,
-}
-
-impl<T> Keyring<T>
-where
-    T: DcKey,
-{
-    /// New empty keyring.
-    pub fn new() -> Keyring<T> {
-        Keyring { keys: Vec::new() }
-    }
-
-    /// Create a new keyring with the the user's secret key loaded.
-    pub async fn new_self(context: &Context) -> Result<Keyring<T>> {
-        let mut keyring: Keyring<T> = Keyring::new();
-        keyring.load_self(context).await?;
-        Ok(keyring)
-    }
-
-    /// Load the user's key into the keyring.
-    pub async fn load_self(&mut self, context: &Context) -> Result<()> {
-        self.add(T::load_self(context).await?);
-        Ok(())
-    }
-
-    /// Add a key to the keyring.
-    pub fn add(&mut self, key: T) {
-        self.keys.push(key);
-    }
-
-    pub fn is_empty(&self) -> bool {
-        self.keys.is_empty()
-    }
-
-    /// A vector with reference to all the keys in the keyring.
-    pub fn keys(&self) -> &[T] {
-        &self.keys
-    }
-}
-
-#[cfg(test)]
-mod tests {
-    use super::*;
-    use crate::key::{SignedPublicKey, SignedSecretKey};
-    use crate::test_utils::{alice_keypair, TestContext};
-
-    #[test]
-    fn test_keyring_add_keys() {
-        let alice = alice_keypair();
-        let mut pub_ring: Keyring<SignedPublicKey> = Keyring::new();
-        pub_ring.add(alice.public.clone());
-        assert_eq!(pub_ring.keys(), [alice.public]);
-
-        let mut sec_ring: Keyring<SignedSecretKey> = Keyring::new();
-        sec_ring.add(alice.secret.clone());
-        assert_eq!(sec_ring.keys(), [alice.secret]);
-    }
-
-    #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
-    async fn test_keyring_load_self() {
-        // new_self() implies load_self()
-        let t = TestContext::new_alice().await;
-        let alice = alice_keypair();
-
-        let pub_ring: Keyring<SignedPublicKey> = Keyring::new_self(&t).await.unwrap();
-        assert_eq!(pub_ring.keys(), [alice.public]);
-
-        let sec_ring: Keyring<SignedSecretKey> = Keyring::new_self(&t).await.unwrap();
-        assert_eq!(sec_ring.keys(), [alice.secret]);
-    }
-}

src/lib.rs

@@ -8,7 +8,6 @@
     missing_debug_implementations,
     missing_docs,
     clippy::all,
-    clippy::indexing_slicing,
     clippy::wildcard_imports,
     clippy::needless_borrow,
     clippy::cast_lossless,
@@ -17,6 +16,7 @@
     clippy::explicit_into_iter_loop,
     clippy::cloned_instead_of_copied
 )]
+#![cfg_attr(not(test), warn(clippy::indexing_slicing))]
 #![allow(
     clippy::match_bool,
     clippy::mixed_read_write_in_expression,
@@ -70,7 +70,6 @@ mod scheduler;
 #[macro_use]
 mod job;
 pub mod key;
-mod keyring;
 pub mod location;
 mod login_param;
 pub mod message;
@@ -79,7 +78,7 @@ pub mod mimeparser;
 pub mod oauth2;
 mod param;
 pub mod peerstate;
-mod pgp;
+pub mod pgp;
 pub mod provider;
 pub mod qr;
 pub mod qr_code_generator;

src/location.rs

@@ -735,7 +735,7 @@ async fn maybe_send_locations(context: &Context) -> Result<Option<u64>> {
 
             next_event = next_event
                 .into_iter()
-                .chain(u64::try_from(locations_send_until - now).into_iter())
+                .chain(u64::try_from(locations_send_until - now))
                 .min();
 
             if has_locations {
@@ -759,7 +759,7 @@ async fn maybe_send_locations(context: &Context) -> Result<Option<u64>> {
                     );
                     next_event = next_event
                         .into_iter()
-                        .chain(u64::try_from(locations_last_sent + 61 - now).into_iter())
+                        .chain(u64::try_from(locations_last_sent + 61 - now))
                         .min();
                 }
             } else {

src/message.rs

@@ -7,7 +7,6 @@ use anyhow::{ensure, format_err, Context as _, Result};
 use deltachat_derive::{FromSql, ToSql};
 use serde::{Deserialize, Serialize};
 
-use crate::blob::BlobObject;
 use crate::chat::{Chat, ChatId};
 use crate::config::Config;
 use crate::constants::{
@@ -583,22 +582,14 @@ impl Message {
                 if (self.viewtype == Viewtype::Image || self.viewtype == Viewtype::Gif)
                     && !self.param.exists(Param::Width)
                 {
-                    let buf = read_file(context, &path_and_filename).await?;
+                    self.param.set_int(Param::Width, 0);
+                    self.param.set_int(Param::Height, 0);
 
-                    match get_filemeta(&buf) {
-                        Ok((width, height)) => {
+                    if let Ok(buf) = read_file(context, path_and_filename).await {
+                        if let Ok((width, height)) = get_filemeta(&buf) {
                             self.param.set_int(Param::Width, width as i32);
                             self.param.set_int(Param::Height, height as i32);
                         }
-                        Err(err) => {
-                            self.param.set_int(Param::Width, 0);
-                            self.param.set_int(Param::Height, 0);
-                            warn!(
-                                context,
-                                "Failed to get width and height for {}: {err:#}.",
-                                path_and_filename.display()
-                            );
-                        }
                     }
 
                     if !self.id.is_unset() {
@@ -697,13 +688,11 @@ impl Message {
         &self.subject
     }
 
-    /// Returns original filename (as shown in chat).
+    /// Returns base file name without the path.
+    /// The base file name includes the extension.
     ///
     /// To get the full path, use [`Self::get_file()`].
     pub fn get_filename(&self) -> Option<String> {
-        if let Some(name) = self.param.get(Param::Filename) {
-            return Some(name.to_string());
-        }
         self.param
             .get(Param::File)
             .and_then(|file| Path::new(file).file_name())
@@ -983,34 +972,20 @@ impl Message {
     /// the file will only be used when the message is prepared
     /// for sending.
     pub fn set_file(&mut self, file: impl ToString, filemime: Option<&str>) {
-        if let Some(name) = Path::new(&file.to_string()).file_name() {
-            if let Some(name) = name.to_str() {
-                self.param.set(Param::Filename, name);
-            }
-        }
         self.param.set(Param::File, file);
-        self.param.set_optional(Param::MimeType, filemime);
-    }
-
-    /// Creates a new blob and sets it as a file associated with a message.
-    pub async fn set_file_from_bytes(
-        &mut self,
-        context: &Context,
-        suggested_name: &str,
-        data: &[u8],
-        filemime: Option<&str>,
-    ) -> Result<()> {
-        let blob = BlobObject::create(context, suggested_name, data).await?;
-        self.param.set(Param::File, blob.as_name());
-        self.param.set_optional(Param::MimeType, filemime);
-        Ok(())
+        if let Some(filemime) = filemime {
+            self.param.set(Param::MimeType, filemime);
+        }
     }
 
     /// Set different sender name for a message.
     /// This overrides the name set by the `set_config()`-option `displayname`.
     pub fn set_override_sender_name(&mut self, name: Option<String>) {
-        self.param
-            .set_optional(Param::OverrideSenderDisplayname, name);
+        if let Some(name) = name {
+            self.param.set(Param::OverrideSenderDisplayname, name);
+        } else {
+            self.param.remove(Param::OverrideSenderDisplayname);
+        }
     }
 
     /// Sets the dimensions of associated image or video file.
@@ -1447,7 +1422,6 @@ pub async fn get_mime_headers(context: &Context, msg_id: MsgId) -> Result<Vec<u8
 /// and scheduling for deletion on IMAP.
 pub async fn delete_msgs(context: &Context, msg_ids: &[MsgId]) -> Result<()> {
     let mut modified_chat_ids = BTreeSet::new();
-    let mut res = Ok(());
 
     for &msg_id in msg_ids {
         let msg = Message::load_from_db(context, msg_id).await?;
@@ -1471,19 +1445,13 @@ pub async fn delete_msgs(context: &Context, msg_ids: &[MsgId]) -> Result<()> {
         modified_chat_ids.insert(msg.chat_id);
 
         let target = context.get_delete_msgs_target().await?;
-        let update_db = |conn: &mut rusqlite::Connection| {
-            conn.execute(
+        context
+            .sql
+            .execute(
                 "UPDATE imap SET target=? WHERE rfc724_mid=?",
                 (target, msg.rfc724_mid),
-            )?;
-            conn.execute("DELETE FROM smtp WHERE msg_id=?", (msg_id,))?;
-            Ok(())
-        };
-        if let Err(e) = context.sql.call_write(update_db).await {
-            error!(context, "delete_msgs: failed to update db: {e:#}.");
-            res = Err(e);
-            continue;
-        }
+            )
+            .await?;
 
         let logging_xdc_id = context
             .debug_logging
@@ -1498,7 +1466,6 @@ pub async fn delete_msgs(context: &Context, msg_ids: &[MsgId]) -> Result<()> {
             }
         }
     }
-    res?;
 
     for modified_chat_id in modified_chat_ids {
         context.emit_msgs_changed(modified_chat_id, MsgId::new(0));
@@ -1667,36 +1634,53 @@ pub(crate) async fn update_msg_state(
 
 // Context functions to work with messages
 
-pub(crate) async fn set_msg_failed(
-    context: &Context,
-    msg: &mut Message,
-    error: &str,
-) -> Result<()> {
-    if msg.state.can_fail() {
-        msg.state = MessageState::OutFailed;
-        warn!(context, "{} failed: {}", msg.id, error);
-    } else {
-        warn!(
-            context,
-            "{} seems to have failed ({}), but state is {}", msg.id, error, msg.state
-        )
+/// Returns true if given message ID exists in the database and is not trashed.
+pub(crate) async fn exists(context: &Context, msg_id: MsgId) -> Result<bool> {
+    if msg_id.is_special() {
+        return Ok(false);
     }
-    msg.error = Some(error.to_string());
 
-    context
+    let chat_id: Option<ChatId> = context
         .sql
-        .execute(
-            "UPDATE msgs SET state=?, error=? WHERE id=?;",
-            (msg.state, error, msg.id),
-        )
+        .query_get_value("SELECT chat_id FROM msgs WHERE id=?;", (msg_id,))
         .await?;
 
-    context.emit_event(EventType::MsgFailed {
-        chat_id: msg.chat_id,
-        msg_id: msg.id,
-    });
+    if let Some(chat_id) = chat_id {
+        Ok(!chat_id.is_trash())
+    } else {
+        Ok(false)
+    }
+}
 
-    Ok(())
+pub(crate) async fn set_msg_failed(context: &Context, msg_id: MsgId, error: &str) {
+    if let Ok(mut msg) = Message::load_from_db(context, msg_id).await {
+        if msg.state.can_fail() {
+            msg.state = MessageState::OutFailed;
+            warn!(context, "{} failed: {}", msg_id, error);
+        } else {
+            warn!(
+                context,
+                "{} seems to have failed ({}), but state is {}", msg_id, error, msg.state
+            )
+        }
+
+        match context
+            .sql
+            .execute(
+                "UPDATE msgs SET state=?, error=? WHERE id=?;",
+                (msg.state, error, msg_id),
+            )
+            .await
+        {
+            Ok(_) => context.emit_event(EventType::MsgFailed {
+                chat_id: msg.chat_id,
+                msg_id,
+            }),
+            Err(e) => {
+                warn!(context, "{:?}", e);
+            }
+        }
+    }
 }
 
 /// The number of messages assigned to unblocked chats
@@ -2303,7 +2287,7 @@ mod tests {
         update_msg_state(&alice, alice_msg.id, MessageState::OutMdnRcvd).await?;
         assert_state(&alice, alice_msg.id, MessageState::OutMdnRcvd).await;
 
-        set_msg_failed(&alice, &mut alice_msg, "badly failed").await?;
+        set_msg_failed(&alice, alice_msg.id, "badly failed").await;
         assert_state(&alice, alice_msg.id, MessageState::OutFailed).await;
 
         // check incoming message states on receiver side
@@ -2439,23 +2423,4 @@ def hello():
 
         Ok(())
     }
-
-    #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
-    async fn test_delete_msgs_offline() -> Result<()> {
-        let alice = TestContext::new_alice().await;
-        let chat = alice
-            .create_chat_with_contact("Bob", "bob@example.org")
-            .await;
-        let mut msg = Message::new(Viewtype::Text);
-        msg.set_text("hi".to_string());
-        assert!(chat::send_msg_sync(&alice, chat.id, &mut msg)
-            .await
-            .is_err());
-        let stmt = "SELECT COUNT(*) FROM smtp WHERE msg_id=?";
-        assert!(alice.sql.exists(stmt, (msg.id,)).await?);
-        delete_msgs(&alice, &[msg.id]).await?;
-        assert!(!alice.sql.exists(stmt, (msg.id,)).await?);
-
-        Ok(())
-    }
 }

src/mimefactory.rs

@@ -896,7 +896,17 @@ impl<'a> MimeFactory<'a> {
         let mut placeholdertext = None;
         let mut meta_part = None;
 
-        if chat.is_protected() {
+        let send_verified_headers = match chat.typ {
+            Chattype::Undefined => bail!("Undefined chat type"),
+            // In single chats, the protection status isn't necessarily the same for both sides,
+            // so we don't send the Chat-Verified header:
+            Chattype::Single => false,
+            Chattype::Group => true,
+            // Mailinglists and broadcast lists can actually never be verified:
+            Chattype::Mailinglist => false,
+            Chattype::Broadcast => false,
+        };
+        if chat.is_protected() && send_verified_headers {
             headers
                 .protected
                 .push(Header::new("Chat-Verified".to_string(), "1".to_string()));
@@ -1368,7 +1378,7 @@ impl<'a> MimeFactory<'a> {
 ///
 /// This line length limit is an
 /// [RFC5322 requirement](https://tools.ietf.org/html/rfc5322#section-2.1.1).
-pub(crate) fn wrapped_base64_encode(buf: &[u8]) -> String {
+fn wrapped_base64_encode(buf: &[u8]) -> String {
     let base64 = base64::engine::general_purpose::STANDARD.encode(buf);
     let mut chars = base64.chars();
     std::iter::repeat_with(|| chars.by_ref().take(78).collect::<String>())
@@ -1386,7 +1396,7 @@ async fn build_body_file(
         .param
         .get_blob(Param::File, context, true)
         .await?
-        .context("msg has no file")?;
+        .context("msg has no filename")?;
     let suffix = blob.suffix().unwrap_or("dat");
 
     // Get file name to use for sending.  For privacy purposes, we do
@@ -1431,11 +1441,7 @@ async fn build_body_file(
                 ),
             &suffix
         ),
-        _ => msg
-            .param
-            .get(Param::Filename)
-            .unwrap_or_else(|| blob.as_file_name())
-            .to_string(),
+        _ => blob.as_file_name().to_string(),
     };
 
     /* check mimetype */

src/mimeparser.rs

@@ -2,7 +2,6 @@
 
 use std::collections::{HashMap, HashSet};
 use std::future::Future;
-use std::path::Path;
 use std::pin::Pin;
 use std::str;
 
@@ -12,6 +11,7 @@ use deltachat_derive::{FromSql, ToSql};
 use format_flowed::unformat_flowed;
 use lettre_email::mime::{self, Mime};
 use mailparse::{addrparse_header, DispositionType, MailHeader, MailHeaderMap, SingleInfo};
+use once_cell::sync::Lazy;
 
 use crate::aheader::{Aheader, EncryptPreference};
 use crate::blob::BlobObject;
@@ -28,10 +28,7 @@ use crate::dehtml::dehtml;
 use crate::events::EventType;
 use crate::headerdef::{HeaderDef, HeaderDefMap};
 use crate::key::{DcKey, Fingerprint, SignedPublicKey, SignedSecretKey};
-use crate::keyring::Keyring;
-use crate::message::{
-    self, set_msg_failed, update_msg_state, Message, MessageState, MsgId, Viewtype,
-};
+use crate::message::{self, set_msg_failed, update_msg_state, MessageState, MsgId, Viewtype};
 use crate::param::{Param, Params};
 use crate::peerstate::Peerstate;
 use crate::simplify::{simplify, SimplifiedText};
@@ -278,9 +275,10 @@ impl MimeMessage {
         headers.remove("chat-verified");
 
         let from = from.context("No from in message")?;
-        let private_keyring: Keyring<SignedSecretKey> = Keyring::new_self(context)
+        let private_keyring = vec![SignedSecretKey::load_self(context)
             .await
-            .context("failed to get own keyring")?;
+            .context("Failed to get own key")?];
+
         let mut decryption_info =
             prepare_decryption(context, &mail, &from.addr, message_time).await?;
 
@@ -438,8 +436,6 @@ impl MimeMessage {
                         typ: Viewtype::Text,
                         msg_raw: Some(txt.clone()),
                         msg: txt,
-                        // Don't change the error prefix for now,
-                        // receive_imf.rs:lookup_chat_by_reply() checks it.
                         error: Some(format!("Decrypting failed: {err:#}")),
                         ..Default::default()
                     };
@@ -865,7 +861,7 @@ impl MimeMessage {
         is_related: bool,
     ) -> Result<bool> {
         let mut any_part_added = false;
-        let mimetype = get_mime_type(mail, &get_attachment_filename(context, mail)?)?.0;
+        let mimetype = get_mime_type(mail)?.0;
         match (mimetype.type_(), mimetype.subtype().as_str()) {
             /* Most times, multipart/alternative contains true alternatives
             as text/plain and text/html.  If we find a multipart/mixed
@@ -873,9 +869,9 @@ impl MimeMessage {
             apple mail: "plaintext" as an alternative to "html+PDF attachment") */
             (mime::MULTIPART, "alternative") => {
                 for cur_data in &mail.subparts {
-                    let mime_type =
-                        get_mime_type(cur_data, &get_attachment_filename(context, cur_data)?)?.0;
-                    if mime_type == "multipart/mixed" || mime_type == "multipart/related" {
+                    if get_mime_type(cur_data)?.0 == "multipart/mixed"
+                        || get_mime_type(cur_data)?.0 == "multipart/related"
+                    {
                         any_part_added = self
                             .parse_mime_recursive(context, cur_data, is_related)
                             .await?;
@@ -885,11 +881,7 @@ impl MimeMessage {
                 if !any_part_added {
                     /* search for text/plain and add this */
                     for cur_data in &mail.subparts {
-                        if get_mime_type(cur_data, &get_attachment_filename(context, cur_data)?)?
-                            .0
-                            .type_()
-                            == mime::TEXT
-                        {
+                        if get_mime_type(cur_data)?.0.type_() == mime::TEXT {
                             any_part_added = self
                                 .parse_mime_recursive(context, cur_data, is_related)
                                 .await?;
@@ -1015,10 +1007,11 @@ impl MimeMessage {
         is_related: bool,
     ) -> Result<bool> {
         // return true if a part was added
-        let filename = get_attachment_filename(context, mail)?;
-        let (mime_type, msg_type) = get_mime_type(mail, &filename)?;
+        let (mime_type, msg_type) = get_mime_type(mail)?;
         let raw_mime = mail.ctype.mimetype.to_lowercase();
 
+        let filename = get_attachment_filename(context, mail)?;
+
         let old_part_count = self.parts.len();
 
         match filename {
@@ -1275,7 +1268,6 @@ impl MimeMessage {
         part.mimetype = Some(mime_type);
         part.bytes = decoded_data.len();
         part.param.set(Param::File, blob.as_name());
-        part.param.set(Param::Filename, filename);
         part.param.set(Param::MimeType, raw_mime);
         part.is_related = is_related;
 
@@ -1620,21 +1612,25 @@ impl MimeMessage {
             false
         };
         if maybe_ndn && self.delivery_report.is_none() {
-            for original_message_id in self
+            static RE: Lazy<regex::Regex> =
+                Lazy::new(|| regex::Regex::new(r"Message-ID:(.*)").unwrap());
+            for captures in self
                 .parts
                 .iter()
                 .filter_map(|part| part.msg_raw.as_ref())
                 .flat_map(|part| part.lines())
-                .filter_map(|line| line.split_once("Message-ID:"))
-                .filter_map(|(_, message_id)| parse_message_id(message_id).ok())
+                .filter_map(|line| RE.captures(line))
             {
-                if let Ok(Some(_)) = message::rfc724_mid_exists(context, &original_message_id).await
-                {
-                    self.delivery_report = Some(DeliveryReport {
-                        rfc724_mid: original_message_id,
-                        failed_recipient: None,
-                        failure: true,
-                    })
+                if let Ok(original_message_id) = parse_message_id(&captures[1]) {
+                    if let Ok(Some(_)) =
+                        message::rfc724_mid_exists(context, &original_message_id).await
+                    {
+                        self.delivery_report = Some(DeliveryReport {
+                            rfc724_mid: original_message_id,
+                            failed_recipient: None,
+                            failure: true,
+                        })
+                    }
                 }
             }
         }
@@ -1868,10 +1864,7 @@ pub struct Part {
 }
 
 /// return mimetype and viewtype for a parsed mail
-fn get_mime_type(
-    mail: &mailparse::ParsedMail<'_>,
-    filename: &Option<String>,
-) -> Result<(Mime, Viewtype)> {
+fn get_mime_type(mail: &mailparse::ParsedMail<'_>) -> Result<(Mime, Viewtype)> {
     let mimetype = mail.ctype.mimetype.parse::<Mime>()?;
 
     let viewtype = match mimetype.type_() {
@@ -1899,7 +1892,7 @@ fn get_mime_type(
             } else {
                 // Enacapsulated messages, see <https://www.w3.org/Protocols/rfc1341/7_3_Message.html>
                 // Also used as part "message/disposition-notification" of "multipart/report", which, however, will
-                // be handled separatedly.
+                // be handled separately.
                 // I've not seen any messages using this, so we do not attach these parts (maybe they're used to attach replies,
                 // which are unwanted at all).
                 // For now, we skip these parts at all; if desired, we could return DcMimeType::File/DC_MSG_File
@@ -1907,16 +1900,7 @@ fn get_mime_type(
                 Viewtype::Unknown
             }
         }
-        mime::APPLICATION => match mimetype.subtype() {
-            mime::OCTET_STREAM => match filename {
-                Some(filename) => match message::guess_msgtype_from_suffix(Path::new(&filename)) {
-                    Some((viewtype, _)) => viewtype,
-                    None => Viewtype::File,
-                },
-                None => Viewtype::File,
-            },
-            _ => Viewtype::File,
-        },
+        mime::APPLICATION => Viewtype::File,
         _ => Viewtype::Unknown,
     };
 
@@ -2160,8 +2144,7 @@ async fn handle_ndn(
     let mut first = true;
     for msg in msgs {
         let (msg_id, chat_id, chat_type) = msg?;
-        let mut message = Message::load_from_db(context, msg_id).await?;
-        set_msg_failed(context, &mut message, &error).await?;
+        set_msg_failed(context, msg_id, &error).await;
         if first {
             // Add only one info msg for all failed messages
             ndn_maybe_add_info_msg(context, failed, chat_id, chat_type).await?;
@@ -3772,22 +3755,4 @@ Content-Disposition: reaction\n\
 
         Ok(())
     }
-
-    #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
-    async fn test_jpeg_as_application_octet_stream() -> Result<()> {
-        let context = TestContext::new_alice().await;
-        let raw = include_bytes!("../test-data/message/jpeg-as-application-octet-stream.eml");
-
-        let msg = MimeMessage::from_bytes(&context.ctx, &raw[..], None)
-            .await
-            .unwrap();
-        assert_eq!(msg.parts.len(), 1);
-        assert_eq!(msg.parts[0].typ, Viewtype::Image);
-
-        receive_imf(&context, &raw[..], false).await?;
-        let msg = context.get_last_msg().await;
-        assert_eq!(msg.get_viewtype(), Viewtype::Image);
-
-        Ok(())
-    }
 }

src/param.rs

@@ -21,9 +21,6 @@ pub enum Param {
     /// For messages and jobs
     File = b'f',
 
-    /// For messages: original filename (as shown in chat)
-    Filename = b'v',
-
     /// For messages: This name should be shown instead of contact.get_display_name()
     /// (used if this is a mailinglist
     /// or explicitly set using set_override_sender_name(), eg. by bots)
@@ -281,16 +278,6 @@ impl Params {
         self
     }
 
-    /// Sets the given key from an optional value.
-    /// Removes the key if the value is `None`.
-    pub fn set_optional(&mut self, key: Param, value: Option<impl ToString>) -> &mut Self {
-        if let Some(value) = value {
-            self.set(key, value)
-        } else {
-            self.remove(key)
-        }
-    }
-
     /// Check if there are any values in this.
     pub fn is_empty(&self) -> bool {
         self.inner.is_empty()
@@ -541,7 +528,7 @@ mod tests {
 
         fs::write(fname, b"boo").await.unwrap();
         let blob = p.get_blob(Param::File, &t, true).await.unwrap().unwrap();
-        assert!(blob.as_file_name().starts_with("foo"));
+        assert_eq!(blob, BlobObject::from_name(&t, "foo".to_string()).unwrap());
 
         // Blob in blobdir, expect blob.
         let bar_path = t.get_blobdir().join("bar");

src/peerstate.rs

@@ -392,6 +392,31 @@ impl Peerstate {
         }
     }
 
+    /// Returns a reference to the contact's public key fingerprint.
+    ///
+    /// Similar to [`Self::peek_key`], but returns the fingerprint instead of the key.
+    fn peek_key_fingerprint(&self, min_verified: PeerstateVerifiedStatus) -> Option<&Fingerprint> {
+        match min_verified {
+            PeerstateVerifiedStatus::BidirectVerified => self.verified_key_fingerprint.as_ref(),
+            PeerstateVerifiedStatus::Unverified => self
+                .public_key_fingerprint
+                .as_ref()
+                .or(self.gossip_key_fingerprint.as_ref()),
+        }
+    }
+
+    /// Returns true if the key used for opportunistic encryption in the 1:1 chat
+    /// is the same as the verified key.
+    ///
+    /// Note that verified groups always use the verified key no matter if the
+    /// opportunistic key matches or not.
+    pub(crate) fn is_using_verified_key(&self) -> bool {
+        let verified = self.peek_key_fingerprint(PeerstateVerifiedStatus::BidirectVerified);
+
+        verified.is_some()
+            && verified == self.peek_key_fingerprint(PeerstateVerifiedStatus::Unverified)
+    }
+
     /// Set this peerstate to verified
     /// Make sure to call `self.save_to_db` to save these changes
     /// Params:

src/pgp.rs

@@ -20,22 +20,14 @@ use tokio::runtime::Handle;
 
 use crate::constants::KeyGenType;
 use crate::key::{DcKey, Fingerprint};
-use crate::keyring::Keyring;
 use crate::tools::EmailAddress;
 
 #[allow(missing_docs)]
-#[cfg(test)]
-pub(crate) const HEADER_AUTOCRYPT: &str = "autocrypt-prefer-encrypt";
+pub const HEADER_AUTOCRYPT: &str = "autocrypt-prefer-encrypt";
 
 #[allow(missing_docs)]
 pub const HEADER_SETUPCODE: &str = "passphrase-begin";
 
-/// Preferred symmetric encryption algorithm.
-const SYMMETRIC_KEY_ALGORITHM: SymmetricKeyAlgorithm = SymmetricKeyAlgorithm::AES128;
-
-/// Preferred cryptographic hash.
-const HASH_ALGORITHM: HashAlgorithm = HashAlgorithm::SHA2_256;
-
 /// A wrapper for rPGP public key types
 #[derive(Debug)]
 enum SignedPublicKeyOrSubkey<'a> {
@@ -142,7 +134,6 @@ pub struct KeyPair {
 pub(crate) fn create_keypair(addr: EmailAddress, keygen_type: KeyGenType) -> Result<KeyPair> {
     let (secret_key_type, public_key_type) = match keygen_type {
         KeyGenType::Rsa2048 => (PgpKeyType::Rsa(2048), PgpKeyType::Rsa(2048)),
-        KeyGenType::Rsa4096 => (PgpKeyType::Rsa(4096), PgpKeyType::Rsa(4096)),
         KeyGenType::Ed25519 | KeyGenType::Default => (PgpKeyType::EdDSA, PgpKeyType::ECDH),
     };
 
@@ -237,7 +228,7 @@ fn select_pk_for_encryption(key: &SignedPublicKey) -> Option<SignedPublicKeyOrSu
 /// and signs it using `private_key_for_signing`.
 pub async fn pk_encrypt(
     plain: &[u8],
-    public_keys_for_encryption: Keyring<SignedPublicKey>,
+    public_keys_for_encryption: Vec<SignedPublicKey>,
     private_key_for_signing: Option<SignedSecretKey>,
 ) -> Result<String> {
     let lit_msg = Message::new_literal_bytes("", plain);
@@ -245,7 +236,6 @@ pub async fn pk_encrypt(
     Handle::current()
         .spawn_blocking(move || {
             let pkeys: Vec<SignedPublicKeyOrSubkey> = public_keys_for_encryption
-                .keys()
                 .iter()
                 .filter_map(select_pk_for_encryption)
                 .collect();
@@ -256,13 +246,11 @@ pub async fn pk_encrypt(
             // TODO: measure time
             let encrypted_msg = if let Some(ref skey) = private_key_for_signing {
                 lit_msg
-                    .sign(skey, || "".into(), HASH_ALGORITHM)
+                    .sign(skey, || "".into(), Default::default())
                     .and_then(|msg| msg.compress(CompressionAlgorithm::ZLIB))
-                    .and_then(|msg| {
-                        msg.encrypt_to_keys(&mut rng, SYMMETRIC_KEY_ALGORITHM, &pkeys_refs)
-                    })
+                    .and_then(|msg| msg.encrypt_to_keys(&mut rng, Default::default(), &pkeys_refs))
             } else {
-                lit_msg.encrypt_to_keys(&mut rng, SYMMETRIC_KEY_ALGORITHM, &pkeys_refs)
+                lit_msg.encrypt_to_keys(&mut rng, Default::default(), &pkeys_refs)
             };
 
             let msg = encrypted_msg?;
@@ -281,7 +269,7 @@ pub fn pk_calc_signature(
     let msg = Message::new_literal_bytes("", plain).sign(
         private_key_for_signing,
         || "".into(),
-        HASH_ALGORITHM,
+        Default::default(),
     )?;
     let signature = msg.into_signature().to_armored_string(None)?;
     Ok(signature)
@@ -298,15 +286,15 @@ pub fn pk_calc_signature(
 #[allow(clippy::implicit_hasher)]
 pub fn pk_decrypt(
     ctext: Vec<u8>,
-    private_keys_for_decryption: &Keyring<SignedSecretKey>,
-    public_keys_for_validation: &Keyring<SignedPublicKey>,
+    private_keys_for_decryption: &[SignedSecretKey],
+    public_keys_for_validation: &[SignedPublicKey],
 ) -> Result<(Vec<u8>, HashSet<Fingerprint>)> {
     let mut ret_signature_fingerprints: HashSet<Fingerprint> = Default::default();
 
     let cursor = Cursor::new(ctext);
     let (msg, _) = Message::from_armor_single(cursor)?;
 
-    let skeys: Vec<&SignedSecretKey> = private_keys_for_decryption.keys().iter().collect();
+    let skeys: Vec<&SignedSecretKey> = private_keys_for_decryption.iter().collect();
 
     let (decryptor, _) = msg.decrypt(|| "".into(), &skeys[..])?;
     let msgs = decryptor.collect::<pgp::errors::Result<Vec<_>>>()?;
@@ -321,20 +309,13 @@ pub fn pk_decrypt(
             None => bail!("The decrypted message is empty"),
         };
 
-        if !public_keys_for_validation.is_empty() {
-            let pkeys = public_keys_for_validation.keys();
-
-            let mut fingerprints: Vec<Fingerprint> = Vec::new();
-            if let signed_msg @ pgp::composed::Message::Signed { .. } = msg {
-                for pkey in pkeys {
-                    if signed_msg.verify(&pkey.primary_key).is_ok() {
-                        let fp = DcKey::fingerprint(pkey);
-                        fingerprints.push(fp);
-                    }
+        if let signed_msg @ pgp::composed::Message::Signed { .. } = msg {
+            for pkey in public_keys_for_validation {
+                if signed_msg.verify(&pkey.primary_key).is_ok() {
+                    let fp = DcKey::fingerprint(pkey);
+                    ret_signature_fingerprints.insert(fp);
                 }
             }
-
-            ret_signature_fingerprints.extend(fingerprints);
         }
         Ok((content, ret_signature_fingerprints))
     } else {
@@ -346,12 +327,11 @@ pub fn pk_decrypt(
 pub fn pk_validate(
     content: &[u8],
     signature: &[u8],
-    public_keys_for_validation: &Keyring<SignedPublicKey>,
+    public_keys_for_validation: &[SignedPublicKey],
 ) -> Result<HashSet<Fingerprint>> {
     let mut ret: HashSet<Fingerprint> = Default::default();
 
     let standalone_signature = StandaloneSignature::from_armor_single(Cursor::new(signature))?.0;
-    let pkeys = public_keys_for_validation.keys();
 
     // Remove trailing CRLF before the delimiter.
     // According to RFC 3156 it is considered to be part of the MIME delimiter for the purpose of
@@ -360,7 +340,7 @@ pub fn pk_validate(
         .get(..content.len().saturating_sub(2))
         .context("index is out of range")?;
 
-    for pkey in pkeys {
+    for pkey in public_keys_for_validation {
         if standalone_signature.verify(pkey, content).is_ok() {
             let fp = DcKey::fingerprint(pkey);
             ret.insert(fp);
@@ -378,7 +358,7 @@ pub async fn symm_encrypt(passphrase: &str, plain: &[u8]) -> Result<String> {
         let mut rng = thread_rng();
         let s2k = StringToKey::new_default(&mut rng);
         let msg =
-            lit_msg.encrypt_with_password(&mut rng, s2k, SYMMETRIC_KEY_ALGORITHM, || passphrase)?;
+            lit_msg.encrypt_with_password(&mut rng, s2k, Default::default(), || passphrase)?;
 
         let encoded_msg = msg.to_armored_string(None)?;
 
@@ -495,9 +475,7 @@ mod tests {
     async fn ctext_signed() -> &'static String {
         CTEXT_SIGNED
             .get_or_init(|| async {
-                let mut keyring = Keyring::new();
-                keyring.add(KEYS.alice_public.clone());
-                keyring.add(KEYS.bob_public.clone());
+                let keyring = vec![KEYS.alice_public.clone(), KEYS.bob_public.clone()];
 
                 pk_encrypt(CLEARTEXT, keyring, Some(KEYS.alice_secret.clone()))
                     .await
@@ -510,9 +488,7 @@ mod tests {
     async fn ctext_unsigned() -> &'static String {
         CTEXT_UNSIGNED
             .get_or_init(|| async {
-                let mut keyring = Keyring::new();
-                keyring.add(KEYS.alice_public.clone());
-                keyring.add(KEYS.bob_public.clone());
+                let keyring = vec![KEYS.alice_public.clone(), KEYS.bob_public.clone()];
                 pk_encrypt(CLEARTEXT, keyring, None).await.unwrap()
             })
             .await
@@ -537,10 +513,8 @@ mod tests {
     #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
     async fn test_decrypt_singed() {
         // Check decrypting as Alice
-        let mut decrypt_keyring: Keyring<SignedSecretKey> = Keyring::new();
-        decrypt_keyring.add(KEYS.alice_secret.clone());
-        let mut sig_check_keyring: Keyring<SignedPublicKey> = Keyring::new();
-        sig_check_keyring.add(KEYS.alice_public.clone());
+        let decrypt_keyring = vec![KEYS.alice_secret.clone()];
+        let sig_check_keyring = vec![KEYS.alice_public.clone()];
         let (plain, valid_signatures) = pk_decrypt(
             ctext_signed().await.as_bytes().to_vec(),
             &decrypt_keyring,
@@ -551,10 +525,8 @@ mod tests {
         assert_eq!(valid_signatures.len(), 1);
 
         // Check decrypting as Bob
-        let mut decrypt_keyring = Keyring::new();
-        decrypt_keyring.add(KEYS.bob_secret.clone());
-        let mut sig_check_keyring = Keyring::new();
-        sig_check_keyring.add(KEYS.alice_public.clone());
+        let decrypt_keyring = vec![KEYS.bob_secret.clone()];
+        let sig_check_keyring = vec![KEYS.alice_public.clone()];
         let (plain, valid_signatures) = pk_decrypt(
             ctext_signed().await.as_bytes().to_vec(),
             &decrypt_keyring,
@@ -567,15 +539,9 @@ mod tests {
 
     #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
     async fn test_decrypt_no_sig_check() {
-        let mut keyring = Keyring::new();
-        keyring.add(KEYS.alice_secret.clone());
-        let empty_keyring = Keyring::new();
-        let (plain, valid_signatures) = pk_decrypt(
-            ctext_signed().await.as_bytes().to_vec(),
-            &keyring,
-            &empty_keyring,
-        )
-        .unwrap();
+        let keyring = vec![KEYS.alice_secret.clone()];
+        let (plain, valid_signatures) =
+            pk_decrypt(ctext_signed().await.as_bytes().to_vec(), &keyring, &[]).unwrap();
         assert_eq!(plain, CLEARTEXT);
         assert_eq!(valid_signatures.len(), 0);
     }
@@ -583,10 +549,8 @@ mod tests {
     #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
     async fn test_decrypt_signed_no_key() {
         // The validation does not have the public key of the signer.
-        let mut decrypt_keyring = Keyring::new();
-        decrypt_keyring.add(KEYS.bob_secret.clone());
-        let mut sig_check_keyring = Keyring::new();
-        sig_check_keyring.add(KEYS.bob_public.clone());
+        let decrypt_keyring = vec![KEYS.bob_secret.clone()];
+        let sig_check_keyring = vec![KEYS.bob_public.clone()];
         let (plain, valid_signatures) = pk_decrypt(
             ctext_signed().await.as_bytes().to_vec(),
             &decrypt_keyring,
@@ -599,13 +563,11 @@ mod tests {
 
     #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
     async fn test_decrypt_unsigned() {
-        let mut decrypt_keyring = Keyring::new();
-        decrypt_keyring.add(KEYS.bob_secret.clone());
-        let sig_check_keyring = Keyring::new();
+        let decrypt_keyring = vec![KEYS.bob_secret.clone()];
         let (plain, valid_signatures) = pk_decrypt(
             ctext_unsigned().await.as_bytes().to_vec(),
             &decrypt_keyring,
-            &sig_check_keyring,
+            &[],
         )
         .unwrap();
         assert_eq!(plain, CLEARTEXT);

src/provider.rs

@@ -123,10 +123,10 @@ pub struct Provider {
     pub overview_page: &'static str,
 
     /// List of provider servers.
-    pub server: &'static [Server],
+    pub server: Vec<Server>,
 
     /// Default configuration values to set when provider is configured.
-    pub config_defaults: Option<&'static [ConfigDefault]>,
+    pub config_defaults: Option<Vec<ConfigDefault>>,
 
     /// Type of OAuth 2 authorization if provider supports it.
     pub oauth2_authorizer: Option<Oauth2Authorizer>,
@@ -149,8 +149,8 @@ pub struct ProviderOptions {
     pub delete_to_trash: bool,
 }
 
-impl ProviderOptions {
-    const fn new() -> Self {
+impl Default for ProviderOptions {
+    fn default() -> Self {
         Self {
             strict_tls: true,
             max_smtp_rcpt_to: None,

src/qr.rs

@@ -5,6 +5,7 @@ use std::collections::BTreeMap;
 
 use anyhow::{anyhow, bail, ensure, Context as _, Result};
 pub use dclogin_scheme::LoginOptions;
+use iroh::dial::Ticket;
 use once_cell::sync::Lazy;
 use percent_encoding::percent_decode_str;
 use serde::Deserialize;
@@ -113,7 +114,7 @@ pub enum Qr {
         /// information to connect to and authenticate a backup provider.
         ///
         /// The format is somewhat opaque, but `sendme` can deserialise this.
-        ticket: iroh::provider::Ticket,
+        ticket: Ticket,
     },
 
     /// Ask the user if they want to use the given service for video chats.
@@ -496,12 +497,12 @@ fn decode_webrtc_instance(_context: &Context, qr: &str) -> Result<Qr> {
 /// Decodes a [`DCBACKUP_SCHEME`] QR code.
 ///
 /// The format of this scheme is `DCBACKUP:<encoded ticket>`.  The encoding is the
-/// [`iroh::provider::Ticket`]'s `Display` impl.
+/// [`iroh::dial::Ticket`]'s `Display` impl.
 fn decode_backup(qr: &str) -> Result<Qr> {
     let payload = qr
         .strip_prefix(DCBACKUP_SCHEME)
         .ok_or_else(|| anyhow!("invalid DCBACKUP scheme"))?;
-    let ticket: iroh::provider::Ticket = payload.parse().context("invalid DCBACKUP payload")?;
+    let ticket: Ticket = payload.parse().context("invalid DCBACKUP payload")?;
     Ok(Qr::Backup { ticket })
 }
 

src/quota.rs

@@ -71,7 +71,7 @@ async fn get_unique_quota_roots_and_usage(
                 // messages could be received and so the usage could have been changed
                 *unique_quota_roots
                     .entry(quota_root_name.clone())
-                    .or_default() = quota.resources;
+                    .or_insert_with(Vec::new) = quota.resources;
             }
         }
     }

src/receive_imf.rs

@@ -4,7 +4,7 @@ use std::cmp::min;
 use std::collections::HashSet;
 use std::convert::TryFrom;
 
-use anyhow::{bail, ensure, Context as _, Result};
+use anyhow::{Context as _, Result};
 use mailparse::{parse_mail, SingleInfo};
 use num_traits::FromPrimitive;
 use once_cell::sync::Lazy;
@@ -14,7 +14,7 @@ use crate::chat::{self, Chat, ChatId, ChatIdBlocked, ProtectionStatus};
 use crate::config::Config;
 use crate::constants::{Blocked, Chattype, ShowEmails, DC_CHAT_ID_TRASH};
 use crate::contact::{
-    may_be_valid_addr, normalize_name, Contact, ContactAddress, ContactId, Origin, VerifiedStatus,
+    may_be_valid_addr, normalize_name, Contact, ContactAddress, ContactId, Origin,
 };
 use crate::context::Context;
 use crate::debug_logging::maybe_set_logging_xdc_inner;
@@ -143,7 +143,7 @@ pub(crate) async fn receive_imf_inner(
 
     // check, if the mail is already in our database.
     // make sure, this check is done eg. before securejoin-processing.
-    let (replace_partial_download, replace_chat_id) =
+    let replace_partial_download =
         if let Some(old_msg_id) = message::rfc724_mid_exists(context, rfc724_mid).await? {
             let msg = Message::load_from_db(context, old_msg_id).await?;
             if msg.download_state() != DownloadState::Done && is_partial_download.is_none() {
@@ -152,14 +152,14 @@ pub(crate) async fn receive_imf_inner(
                     context,
                     "Message already partly in DB, replacing by full message."
                 );
-                (Some(old_msg_id), Some(msg.chat_id))
+                Some(old_msg_id)
             } else {
                 // the message was probably moved around.
                 info!(context, "Message already in DB, doing nothing.");
                 return Ok(None);
             }
         } else {
-            (None, None)
+            None
         };
 
     let prevent_rename =
@@ -347,8 +347,8 @@ pub(crate) async fn receive_imf_inner(
         }
     }
 
-    if let Some(replace_chat_id) = replace_chat_id {
-        context.emit_msgs_changed(replace_chat_id, MsgId::new(0));
+    if replace_partial_download.is_some() {
+        context.emit_msgs_changed(chat_id, MsgId::new(0));
     } else if !chat_id.is_trash() {
         let fresh = received_msg.state == MessageState::InFresh;
         for msg_id in &received_msg.msg_ids {
@@ -546,30 +546,19 @@ async fn add_parts(
         // signals whether the current user is a bot
         let is_bot = context.get_config_bool(Config::Bot).await?;
 
-        let create_blocked_default = if is_bot {
-            Blocked::Not
-        } else {
-            Blocked::Request
-        };
-        let create_blocked = if let Some(ChatIdBlocked { id: _, blocked }) = test_normal_chat {
-            match blocked {
-                Blocked::Request => create_blocked_default,
-                Blocked::Not => Blocked::Not,
-                Blocked::Yes => {
-                    if Contact::is_blocked_load(context, from_id).await? {
-                        // User has blocked the contact.
-                        // Block the group contact created as well.
-                        Blocked::Yes
-                    } else {
-                        // 1:1 chat is blocked, but the contact is not.
-                        // This happens when 1:1 chat is hidden
-                        // during scanning of a group invitation code.
-                        Blocked::Request
-                    }
+        let create_blocked = match test_normal_chat {
+            Some(ChatIdBlocked {
+                id: _,
+                blocked: Blocked::Request,
+            }) if is_bot => Blocked::Not,
+            Some(ChatIdBlocked { id: _, blocked }) => blocked,
+            None => {
+                if is_bot {
+                    Blocked::Not
+                } else {
+                    Blocked::Request
                 }
             }
-        } else {
-            create_blocked_default
         };
 
         if chat_id.is_none() {
@@ -605,12 +594,17 @@ async fn add_parts(
 
         // In lookup_chat_by_reply() and create_or_lookup_group(), it can happen that the message is put into a chat
         // but the From-address is not a member of this chat.
-        if let Some(chat_id) = chat_id {
-            if !chat::is_contact_in_chat(context, chat_id, from_id).await? {
-                let chat = Chat::load_from_db(context, chat_id).await?;
+        if let Some(group_chat_id) = chat_id {
+            if !chat::is_contact_in_chat(context, group_chat_id, from_id).await? {
+                let chat = Chat::load_from_db(context, group_chat_id).await?;
                 if chat.is_protected() {
-                    let s = stock_str::unknown_sender_for_chat(context).await;
-                    mime_parser.repl_msg_by_error(&s);
+                    if chat.typ == Chattype::Single {
+                        // Just assign the message to the 1:1 chat with the actual sender instead.
+                        chat_id = None;
+                    } else {
+                        let s = stock_str::unknown_sender_for_chat(context).await;
+                        mime_parser.repl_msg_by_error(&s);
+                    }
                 } else {
                     // In non-protected chats, just mark the sender as overridden. Therefore, the UI will prepend `~`
                     // to the sender's name, indicating to the user that he/she is not part of the group.
@@ -626,7 +620,7 @@ async fn add_parts(
                 context,
                 mime_parser,
                 sent_timestamp,
-                chat_id,
+                group_chat_id,
                 from_id,
                 to_ids,
             )
@@ -729,6 +723,48 @@ async fn add_parts(
                         );
                     }
                 }
+
+                // The next block checks if the message was sent with verified encryption
+                // and sets the protection of the 1:1 chat accordingly.
+                if is_partial_download.is_none()
+                    && mime_parser.get_header(HeaderDef::SecureJoin).is_none()
+                    && !is_mdn
+                {
+                    let mut new_protection = match has_verified_encryption(
+                        context,
+                        mime_parser,
+                        from_id,
+                        to_ids,
+                        Chattype::Single,
+                    )
+                    .await?
+                    {
+                        VerifiedEncryption::Verified => ProtectionStatus::Protected,
+                        VerifiedEncryption::NotVerified(_) => ProtectionStatus::Unprotected,
+                    };
+
+                    let chat = Chat::load_from_db(context, chat_id).await?;
+
+                    if chat.protected != new_protection {
+                        if new_protection == ProtectionStatus::Unprotected
+                            && context
+                                .get_config_bool(Config::VerifiedOneOnOneChats)
+                                .await?
+                        {
+                            new_protection = ProtectionStatus::ProtectionBroken;
+                        }
+
+                        // The message itself will be sorted under the device message since the device
+                        // message is `MessageState::InNoticed`, which means that all following
+                        // messages are sorted under it.
+                        let sort_timestamp =
+                            calc_sort_timestamp(context, sent_timestamp, chat_id, true, incoming)
+                                .await?;
+                        chat_id
+                            .set_protection(context, new_protection, sort_timestamp, Some(from_id))
+                            .await?;
+                    }
+                }
             }
         }
 
@@ -916,7 +952,8 @@ async fn add_parts(
     };
 
     let in_fresh = state == MessageState::InFresh;
-    let sort_timestamp = calc_sort_timestamp(context, sent_timestamp, chat_id, in_fresh).await?;
+    let sort_timestamp =
+        calc_sort_timestamp(context, sent_timestamp, chat_id, false, incoming).await?;
 
     // Apply ephemeral timer changes to the chat.
     //
@@ -995,42 +1032,14 @@ async fn add_parts(
     // if a chat is protected and the message is fully downloaded, check additional properties
     if !chat_id.is_special() && is_partial_download.is_none() {
         let chat = Chat::load_from_db(context, chat_id).await?;
-        let new_status = match mime_parser.is_system_message {
-            SystemMessage::ChatProtectionEnabled => Some(ProtectionStatus::Protected),
-            SystemMessage::ChatProtectionDisabled => Some(ProtectionStatus::Unprotected),
-            _ => None,
-        };
 
-        if chat.is_protected() || new_status.is_some() {
-            if let Err(err) = check_verified_properties(context, mime_parser, from_id, to_ids).await
+        if chat.is_protected() {
+            if let VerifiedEncryption::NotVerified(err) =
+                has_verified_encryption(context, mime_parser, from_id, to_ids, chat.typ).await?
             {
                 warn!(context, "Verification problem: {err:#}.");
                 let s = format!("{err}. See 'Info' for more details");
                 mime_parser.repl_msg_by_error(&s);
-            } else {
-                // change chat protection only when verification check passes
-                if let Some(new_status) = new_status {
-                    if chat_id
-                        .update_timestamp(
-                            context,
-                            Param::ProtectionSettingsTimestamp,
-                            sent_timestamp,
-                        )
-                        .await?
-                    {
-                        if let Err(e) = chat_id.inner_set_protection(context, new_status).await {
-                            chat::add_info_msg(
-                                context,
-                                chat_id,
-                                &format!("Cannot set protection: {e}"),
-                                sort_timestamp,
-                            )
-                            .await?;
-                            // do not return an error as this would result in retrying the message
-                        }
-                    }
-                    better_msg = Some(context.stock_protection_msg(new_status, from_id).await);
-                }
             }
         }
     }
@@ -1079,7 +1088,7 @@ async fn add_parts(
     let mut save_mime_modified = mime_parser.is_mime_modified;
 
     let mime_headers = if save_mime_headers || save_mime_modified {
-        let headers = if !mime_parser.decoded_data.is_empty() {
+        let headers = if mime_parser.was_encrypted() && !mime_parser.decoded_data.is_empty() {
             mime_parser.decoded_data.clone()
         } else {
             imf_raw.to_vec()
@@ -1227,8 +1236,6 @@ RETURNING id
                     ephemeral_timestamp,
                     if is_partial_download.is_some() {
                         DownloadState::Available
-                    } else if mime_parser.decrypting_failed {
-                        DownloadState::Undecipherable
                     } else {
                         DownloadState::Done
                     },
@@ -1374,25 +1381,41 @@ async fn calc_sort_timestamp(
     context: &Context,
     message_timestamp: i64,
     chat_id: ChatId,
-    is_fresh_msg: bool,
+    always_sort_to_bottom: bool,
+    incoming: bool,
 ) -> Result<i64> {
     let mut sort_timestamp = message_timestamp;
 
-    // get newest non fresh message for this chat
-    // update sort_timestamp if less than that
-    if is_fresh_msg {
-        let last_msg_time: Option<i64> = context
+    let last_msg_time: Option<i64> = if always_sort_to_bottom {
+        // get newest message for this chat
+        context
             .sql
             .query_get_value(
-                "SELECT MAX(timestamp) FROM msgs WHERE chat_id=? AND state>?",
-                (chat_id, MessageState::InFresh),
+                "SELECT MAX(timestamp) FROM msgs WHERE chat_id=?",
+                (chat_id,),
             )
-            .await?;
+            .await?
+    } else if incoming {
+        // get newest incoming non fresh message for this chat.
 
-        if let Some(last_msg_time) = last_msg_time {
-            if last_msg_time > sort_timestamp {
-                sort_timestamp = last_msg_time;
-            }
+        // If a user hasn't been online for some time, the Inbox is
+        // fetched first and then the Sentbox. In order for Inbox
+        // and Sent messages to be allowed to mingle,
+        // outgoing messages are purely sorted by their sent timestamp.
+        context
+            .sql
+            .query_get_value(
+                "SELECT MAX(timestamp) FROM msgs WHERE chat_id=? AND state>? AND from_id!=?",
+                (chat_id, MessageState::InFresh, ContactId::SELF),
+            )
+            .await?
+    } else {
+        None
+    };
+
+    if let Some(last_msg_time) = last_msg_time {
+        if last_msg_time > sort_timestamp {
+            sort_timestamp = last_msg_time;
         }
     }
 
@@ -1411,18 +1434,11 @@ async fn lookup_chat_by_reply(
     if let Some(parent) = parent {
         let parent_chat = Chat::load_from_db(context, parent.chat_id).await?;
 
-        if parent.download_state != DownloadState::Done
-            // TODO (2023-09-12): Added for backward compatibility with versions that did not have
-            // `DownloadState::Undecipherable`. Remove eventually with the comment in
-            // `MimeMessage::from_bytes()`.
-            || parent
-                .error
-                .as_ref()
-                .filter(|e| e.starts_with("Decrypting failed:"))
-                .is_some()
-        {
-            // If the parent msg is not fully downloaded or undecipherable, it may have been
-            // assigned to the wrong chat (they often get assigned to the 1:1 chat with the sender).
+        if parent.error.is_some() {
+            // If the parent msg is undecipherable, then it may have been assigned to the wrong chat
+            // (undecipherable group msgs often get assigned to the 1:1 chat with the sender).
+            // We don't have any way of finding out whether a msg is undecipherable, so we check for
+            // error.is_some() instead.
             return Ok(None);
         }
 
@@ -1547,7 +1563,9 @@ async fn create_or_lookup_group(
     }
 
     let create_protected = if mime_parser.get_header(HeaderDef::ChatVerified).is_some() {
-        if let Err(err) = check_verified_properties(context, mime_parser, from_id, to_ids).await {
+        if let VerifiedEncryption::NotVerified(err) =
+            has_verified_encryption(context, mime_parser, from_id, to_ids, Chattype::Group).await?
+        {
             warn!(context, "Verification problem: {err:#}.");
             let s = format!("{err}. See 'Info' for more details");
             mime_parser.repl_msg_by_error(&s);
@@ -1674,37 +1692,37 @@ async fn apply_group_changes(
             false
         };
 
-    let is_from_in_chat = !chat::is_contact_in_chat(context, chat_id, ContactId::SELF).await?
-        || chat::is_contact_in_chat(context, chat_id, from_id).await?;
-
-    // Reject group membership changes from non-members and old changes.
-    let allow_member_list_changes = is_from_in_chat
-        && chat_id
-            .update_timestamp(context, Param::MemberListTimestamp, sent_timestamp)
-            .await?;
+    // Whether to allow any changes to the member list at all.
+    let allow_member_list_changes =
+        if chat::is_contact_in_chat(context, chat_id, ContactId::SELF).await? || self_added {
+            // Reject old group changes.
+            chat_id
+                .update_timestamp(context, Param::MemberListTimestamp, sent_timestamp)
+                .await?
+        } else {
+            // Member list changes are not allowed if we're not in the group
+            // and are not explicitly added.
+            // This message comes from a Delta Chat that restored an old backup
+            // or the message is a MUA reply to an old message.
+            false
+        };
 
     // Whether to rebuild the member list from scratch.
-    let recreate_member_list = {
+    let recreate_member_list = if allow_member_list_changes {
         // Recreate member list if the message comes from a MUA as these messages do _not_ set add/remove headers.
-        !mime_parser.has_chat_version()
-            // Always recreate membership list if SELF has been added. The older versions of DC
-            // don't always set "In-Reply-To" to the latest message they sent, but to the latest
-            // delivered message (so it's a race), so we have this heuristic here.
-            || self_added
-            || match mime_parser.get_header(HeaderDef::InReplyTo) {
+        // Always recreate membership list if self has been added.
+        if !mime_parser.has_chat_version() || self_added {
+            true
+        } else {
+            match mime_parser.get_header(HeaderDef::InReplyTo) {
                 // If we don't know the referenced message, we missed some messages.
                 // Maybe they added/removed members, so we need to recreate our member list.
                 Some(reply_to) => rfc724_mid_exists(context, reply_to).await?.is_none(),
                 None => false,
             }
-    } && {
-        if !allow_member_list_changes {
-            info!(
-                context,
-                "Ignoring a try to recreate member list of {chat_id} by {from_id}.",
-            );
         }
-        allow_member_list_changes
+    } else {
+        false
     };
 
     if let Some(removed_addr) = mime_parser.get_header(HeaderDef::ChatGroupMemberRemoved) {
@@ -1795,51 +1813,45 @@ async fn apply_group_changes(
         }
     }
 
-    if mime_parser.get_header(HeaderDef::ChatVerified).is_some() {
-        if let Err(err) = check_verified_properties(context, mime_parser, from_id, to_ids).await {
-            warn!(context, "Verification problem: {err:#}.");
-            let s = format!("{err}. See 'Info' for more details");
-            mime_parser.repl_msg_by_error(&s);
-        }
-
-        if !chat.is_protected() {
-            chat_id
-                .inner_set_protection(context, ProtectionStatus::Protected)
-                .await?;
-        }
-    }
-
     // Recreate the member list.
     if recreate_member_list {
-        // Only delete old contacts if the sender is not a classical MUA user:
-        // Classical MUA users usually don't intend to remove users from an email
-        // thread, so if they removed a recipient then it was probably by accident.
-        if mime_parser.has_chat_version() {
-            context
-                .sql
-                .execute("DELETE FROM chats_contacts WHERE chat_id=?;", (chat_id,))
+        if !chat::is_contact_in_chat(context, chat_id, from_id).await? {
+            warn!(
+                context,
+                "Contact {from_id} attempts to modify group chat {chat_id} member list without being a member."
+            );
+        } else {
+            // Only delete old contacts if the sender is not a classical MUA user:
+            // Classical MUA users usually don't intend to remove users from an email
+            // thread, so if they removed a recipient then it was probably by accident.
+            if mime_parser.has_chat_version() {
+                context
+                    .sql
+                    .execute("DELETE FROM chats_contacts WHERE chat_id=?;", (chat_id,))
+                    .await?;
+            }
+
+            let mut members_to_add = HashSet::new();
+            members_to_add.extend(to_ids);
+            members_to_add.insert(ContactId::SELF);
+
+            if !from_id.is_special() {
+                members_to_add.insert(from_id);
+            }
+
+            if let Some(removed_id) = removed_id {
+                members_to_add.remove(&removed_id);
+            }
+
+            info!(
+                context,
+                "Recreating chat {chat_id} with members {members_to_add:?}."
+            );
+
+            chat::add_to_chat_contacts_table(context, chat_id, &Vec::from_iter(members_to_add))
                 .await?;
+            send_event_chat_modified = true;
         }
-
-        let mut members_to_add = HashSet::new();
-        members_to_add.extend(to_ids);
-        members_to_add.insert(ContactId::SELF);
-
-        if !from_id.is_special() {
-            members_to_add.insert(from_id);
-        }
-
-        if let Some(removed_id) = removed_id {
-            members_to_add.remove(&removed_id);
-        }
-
-        info!(
-            context,
-            "Recreating chat {chat_id} with members {members_to_add:?}."
-        );
-
-        chat::add_to_chat_contacts_table(context, chat_id, &Vec::from_iter(members_to_add)).await?;
-        send_event_chat_modified = true;
     }
 
     if let Some(avatar_action) = &mime_parser.group_avatar {
@@ -2137,49 +2149,53 @@ async fn create_adhoc_group(
     Ok(Some(new_chat_id))
 }
 
-async fn check_verified_properties(
+enum VerifiedEncryption {
+    Verified,
+    NotVerified(String), // The string contains the reason why it's not verified
+}
+
+/// Checks whether the message is allowed to appear in a protected chat.
+///
+/// This means that it is encrypted, signed with a verified key,
+/// and if it's a group, all the recipients are verified.
+async fn has_verified_encryption(
     context: &Context,
     mimeparser: &MimeMessage,
     from_id: ContactId,
     to_ids: &[ContactId],
-) -> Result<()> {
-    let contact = Contact::get_by_id(context, from_id).await?;
+    chat_type: Chattype,
+) -> Result<VerifiedEncryption> {
+    use VerifiedEncryption::*;
 
-    ensure!(mimeparser.was_encrypted(), "This message is not encrypted");
-
-    if mimeparser.get_header(HeaderDef::ChatVerified).is_none() {
-        // we do not fail here currently, this would exclude (a) non-deltas
-        // and (b) deltas with different protection views across multiple devices.
-        // for group creation or protection enabled/disabled, however, Chat-Verified is respected.
-        warn!(
-            context,
-            "{} did not mark message as protected.",
-            contact.get_addr()
-        );
+    if from_id == ContactId::SELF && chat_type == Chattype::Single {
+        // For outgoing emails in the 1:1 chat, we have an exception that
+        // they are allowed to be unencrypted:
+        // 1. They can't be an attack (they are outgoing, not incoming)
+        // 2. Probably the unencryptedness is just a temporary state, after all
+        //    the user obviously still uses DC
+        //    -> Showing info messages everytime would be a lot of noise
+        // 3. The info messages that are shown to the user ("Your chat partner
+        //    likely reinstalled DC" or similar) would be wrong.
+        return Ok(Verified);
     }
 
+    if !mimeparser.was_encrypted() {
+        return Ok(NotVerified("This message is not encrypted".to_string()));
+    };
+
     // ensure, the contact is verified
     // and the message is signed with a verified key of the sender.
     // this check is skipped for SELF as there is no proper SELF-peerstate
     // and results in group-splits otherwise.
     if from_id != ContactId::SELF {
-        let peerstate = Peerstate::from_addr(context, contact.get_addr()).await?;
+        let Some(peerstate) = &mimeparser.decryption_info.peerstate else {
+            return Ok(NotVerified("No peerstate, the contact isn't verified".to_string()));
+        };
 
-        if peerstate.is_none()
-            || contact.is_verified_ex(context, peerstate.as_ref()).await?
-                != VerifiedStatus::BidirectVerified
-        {
-            bail!(
-                "Sender of this message is not verified: {}",
-                contact.get_addr()
-            );
-        }
-
-        if let Some(peerstate) = peerstate {
-            ensure!(
-                peerstate.has_verified_key(&mimeparser.signatures),
-                "The message was sent with non-verified encryption"
-            );
+        if !peerstate.has_verified_key(&mimeparser.signatures) {
+            return Ok(NotVerified(
+                "The message was sent with non-verified encryption".to_string(),
+            ));
         }
     }
 
@@ -2191,7 +2207,7 @@ async fn check_verified_properties(
         .collect::<Vec<ContactId>>();
 
     if to_ids.is_empty() {
-        return Ok(());
+        return Ok(Verified);
     }
 
     let rows = context
@@ -2215,10 +2231,12 @@ async fn check_verified_properties(
         )
         .await?;
 
+    let contact = Contact::get_by_id(context, from_id).await?;
+
     for (to_addr, mut is_verified) in rows {
         info!(
             context,
-            "check_verified_properties: {:?} self={:?}.",
+            "has_verified_encryption: {:?} self={:?}.",
             to_addr,
             context.is_self_addr(&to_addr).await
         );
@@ -2252,13 +2270,13 @@ async fn check_verified_properties(
             }
         }
         if !is_verified {
-            bail!(
+            return Ok(NotVerified(format!(
                 "{} is not a member of this protected chat",
-                to_addr.to_string()
-            );
+                to_addr
+            )));
         }
     }
-    Ok(())
+    Ok(Verified)
 }
 
 /// Returns the last message referenced from `References` header if it is in the database.

src/receive_imf/tests.rs

@@ -8,8 +8,7 @@ use crate::chat::{
 };
 use crate::chat::{get_chat_msgs, ChatItem, ChatVisibility};
 use crate::chatlist::Chatlist;
-use crate::config::Config;
-use crate::constants::{DC_GCL_FOR_FORWARDING, DC_GCL_NO_SPECIALS};
+use crate::constants::DC_GCL_NO_SPECIALS;
 use crate::imap::prefetch_should_download;
 use crate::message::Message;
 use crate::test_utils::{get_chat_msg, TestContext, TestContextManager};
@@ -794,8 +793,6 @@ async fn test_github_mailing_list() -> Result<()> {
 
     let chats = Chatlist::try_load(&t.ctx, 0, None, None).await?;
     assert_eq!(chats.len(), 1);
-    let chats = Chatlist::try_load(&t.ctx, DC_GCL_FOR_FORWARDING, None, None).await?;
-    assert_eq!(chats.len(), 0);
     let contacts = Contact::get_all(&t.ctx, 0, None).await?;
     assert_eq!(contacts.len(), 0); // mailing list recipients and senders do not count as "known contacts"
 
@@ -1459,9 +1456,7 @@ async fn test_pdf_filename_simple() {
     .await;
     assert_eq!(msg.viewtype, Viewtype::File);
     assert_eq!(msg.text, "mail body");
-    let file_path = msg.param.get(Param::File).unwrap();
-    assert!(file_path.starts_with("$BLOBDIR/simple"));
-    assert!(file_path.ends_with(".pdf"));
+    assert_eq!(msg.param.get(Param::File).unwrap(), "$BLOBDIR/simple.pdf");
 }
 
 #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
@@ -1475,9 +1470,10 @@ async fn test_pdf_filename_continuation() {
     .await;
     assert_eq!(msg.viewtype, Viewtype::File);
     assert_eq!(msg.text, "mail body");
-    let file_path = msg.param.get(Param::File).unwrap();
-    assert!(file_path.starts_with("$BLOBDIR/test pdf äöüß"));
-    assert!(file_path.ends_with(".pdf"));
+    assert_eq!(
+        msg.param.get(Param::File).unwrap(),
+        "$BLOBDIR/test pdf äöüß.pdf"
+    );
 }
 
 /// HTML-images may come with many embedded images, eg. tiny icons, corners for formatting,
@@ -2801,7 +2797,7 @@ Reply from different address
 }
 
 #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
-async fn test_long_and_duplicated_filenames() -> Result<()> {
+async fn test_long_filenames() -> Result<()> {
     let mut tcm = TestContextManager::new();
     let alice = tcm.alice().await;
     let bob = tcm.bob().await;
@@ -2813,7 +2809,6 @@ async fn test_long_and_duplicated_filenames() -> Result<()> {
         "foo. .tar.gz",
         "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.tar.gz",
         "a.tar.gz",
-        "a.tar.gz",
         "a.a..a.a.a.a.tar.gz",
     ] {
         let attachment = alice.blobdir.join(filename_sent);
@@ -2828,19 +2823,22 @@ async fn test_long_and_duplicated_filenames() -> Result<()> {
 
         let msg_bob = bob.recv_msg(&sent).await;
 
-        async fn check_message(msg: &Message, t: &TestContext, filename: &str, content: &str) {
+        async fn check_message(msg: &Message, t: &TestContext, content: &str) {
             assert_eq!(msg.get_viewtype(), Viewtype::File);
             let resulting_filename = msg.get_filename().unwrap();
-            assert_eq!(resulting_filename, filename);
             let path = msg.get_file(t).unwrap();
+            assert!(
+                resulting_filename.ends_with(".tar.gz"),
+                "{resulting_filename:?} doesn't end with .tar.gz, path: {path:?}"
+            );
             assert!(
                 path.to_str().unwrap().ends_with(".tar.gz"),
                 "path {path:?} doesn't end with .tar.gz"
             );
             assert_eq!(fs::read_to_string(path).await.unwrap(), content);
         }
-        check_message(&msg_alice, &alice, filename_sent, &content).await;
-        check_message(&msg_bob, &bob, filename_sent, &content).await;
+        check_message(&msg_alice, &alice, &content).await;
+        check_message(&msg_bob, &bob, &content).await;
     }
 
     Ok(())
@@ -3563,119 +3561,3 @@ async fn test_mua_can_add() -> Result<()> {
     );
     Ok(())
 }
-
-#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
-async fn test_mua_can_readd() -> Result<()> {
-    let alice = TestContext::new_alice().await;
-
-    // Alice creates chat with 3 contacts.
-    let msg = receive_imf(
-        &alice,
-        b"Subject: =?utf-8?q?Message_from_alice=40example=2Eorg?=\r\n\
-            From: alice@example.org\r\n\
-            To: <bob@example.net>, <claire@example.org>, <fiona@example.org> \r\n\
-            Date: Mon, 12 Dec 2022 14:30:39 +0000\r\n\
-            Message-ID: <Mr.alices_original_mail@example.org>\r\n\
-            Chat-Version: 1.0\r\n\
-            \r\n\
-            Hi!\r\n",
-        false,
-    )
-    .await?
-    .unwrap();
-    let alice_chat = Chat::load_from_db(&alice, msg.chat_id).await?;
-    assert_eq!(alice_chat.typ, Chattype::Group);
-    assert!(is_contact_in_chat(&alice, alice_chat.id, ContactId::SELF).await?);
-
-    // And leaves it.
-    remove_contact_from_chat(&alice, alice_chat.id, ContactId::SELF).await?;
-    let alice_chat = Chat::load_from_db(&alice, alice_chat.id).await?;
-    assert!(!is_contact_in_chat(&alice, alice_chat.id, ContactId::SELF).await?);
-
-    // Bob uses a classical MUA to answer, adding Alice back.
-    receive_imf(
-        &alice,
-        b"Subject: Re: Message from alice\r\n\
-            From: <bob@example.net>\r\n\
-            To: <alice@example.org>, <claire@example.org>, <fiona@example.org>\r\n\
-            Date: Mon, 12 Dec 2022 14:32:39 +0000\r\n\
-            Message-ID: <bobs_answer_to_two_recipients@example.net>\r\n\
-            In-Reply-To: <Mr.alices_original_mail@example.org>\r\n\
-            \r\n\
-            Hi back!\r\n",
-        false,
-    )
-    .await?
-    .unwrap();
-
-    let alice_chat = Chat::load_from_db(&alice, alice_chat.id).await?;
-    assert!(is_contact_in_chat(&alice, alice_chat.id, ContactId::SELF).await?);
-    Ok(())
-}
-
-#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
-async fn test_recreate_member_list_on_missing_add_of_self() -> Result<()> {
-    let alice = TestContext::new_alice().await;
-    let bob = TestContext::new_bob().await;
-    let alice_chat_id = create_group_chat(&alice, ProtectionStatus::Unprotected, "Group").await?;
-    add_contact_to_chat(
-        &alice,
-        alice_chat_id,
-        Contact::create(&alice, "bob", &bob.get_config(Config::Addr).await?.unwrap()).await?,
-    )
-    .await?;
-    send_text_msg(&alice, alice_chat_id, "populate".to_string()).await?;
-    alice.pop_sent_msg().await;
-    remove_contact_from_chat(&alice, alice_chat_id, ContactId::SELF).await?;
-    let bob_chat_id = bob.recv_msg(&alice.pop_sent_msg().await).await.chat_id;
-
-    // Bob missed the message adding them, but must recreate the member list.
-    assert_eq!(get_chat_contacts(&bob, bob_chat_id).await?.len(), 1);
-    assert!(is_contact_in_chat(&bob, bob_chat_id, ContactId::SELF).await?);
-    Ok(())
-}
-
-#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
-async fn test_keep_member_list_if_possibly_nomember() -> Result<()> {
-    let alice = TestContext::new_alice().await;
-    let bob = TestContext::new_bob().await;
-    let alice_chat_id = create_group_chat(&alice, ProtectionStatus::Unprotected, "Group").await?;
-    add_contact_to_chat(
-        &alice,
-        alice_chat_id,
-        Contact::create(&alice, "bob", &bob.get_config(Config::Addr).await?.unwrap()).await?,
-    )
-    .await?;
-    send_text_msg(&alice, alice_chat_id, "populate".to_string()).await?;
-    let bob_chat_id = bob.recv_msg(&alice.pop_sent_msg().await).await.chat_id;
-
-    let fiona = TestContext::new_fiona().await;
-    add_contact_to_chat(
-        &alice,
-        alice_chat_id,
-        Contact::create(
-            &alice,
-            "fiona",
-            &fiona.get_config(Config::Addr).await?.unwrap(),
-        )
-        .await?,
-    )
-    .await?;
-    let fiona_chat_id = fiona.recv_msg(&alice.pop_sent_msg().await).await.chat_id;
-    fiona_chat_id.accept(&fiona).await?;
-
-    send_text_msg(&fiona, fiona_chat_id, "hi".to_string()).await?;
-    bob.recv_msg(&fiona.pop_sent_msg().await).await;
-
-    // Bob missed the message adding fiona, but mustn't recreate the member list.
-    assert_eq!(get_chat_contacts(&bob, bob_chat_id).await?.len(), 2);
-    assert!(is_contact_in_chat(&bob, bob_chat_id, ContactId::SELF).await?);
-    let bob_alice_contact = Contact::create(
-        &bob,
-        "alice",
-        &alice.get_config(Config::Addr).await?.unwrap(),
-    )
-    .await?;
-    assert!(is_contact_in_chat(&bob, bob_chat_id, bob_alice_contact).await?);
-    Ok(())
-}

src/scheduler.rs

@@ -864,7 +864,7 @@ impl Scheduler {
 
         // Actually shutdown tasks.
         let timeout_duration = std::time::Duration::from_secs(30);
-        for b in once(self.inbox).chain(self.oboxes.into_iter()) {
+        for b in once(self.inbox).chain(self.oboxes) {
             tokio::time::timeout(timeout_duration, b.handle)
                 .await
                 .log_err(context)

src/securejoin.rs

@@ -6,7 +6,7 @@ use anyhow::{bail, Context as _, Error, Result};
 use percent_encoding::{utf8_percent_encode, AsciiSet, NON_ALPHANUMERIC};
 
 use crate::aheader::EncryptPreference;
-use crate::chat::{self, Chat, ChatId, ChatIdBlocked};
+use crate::chat::{self, Chat, ChatId, ChatIdBlocked, ProtectionStatus};
 use crate::config::Config;
 use crate::constants::Blocked;
 use crate::contact::{Contact, ContactId, Origin, VerifiedStatus};
@@ -701,6 +701,14 @@ async fn secure_connection_established(
     let contact = Contact::get_by_id(context, contact_id).await?;
     let msg = stock_str::contact_verified(context, &contact).await;
     chat::add_info_msg(context, chat_id, &msg, time()).await?;
+    chat_id
+        .set_protection(
+            context,
+            ProtectionStatus::Protected,
+            time(),
+            Some(contact_id),
+        )
+        .await?;
     context.emit_event(EventType::ChatModified(chat_id));
     Ok(())
 }
@@ -783,6 +791,8 @@ mod tests {
     use crate::contact::VerifiedStatus;
     use crate::peerstate::Peerstate;
     use crate::receive_imf::receive_imf;
+    use crate::stock_str::chat_protection_enabled;
+    use crate::test_utils::get_chat_msg;
     use crate::test_utils::{TestContext, TestContextManager};
     use crate::tools::EmailAddress;
 
@@ -921,7 +931,7 @@ mod tests {
         // Check Alice got the verified message in her 1:1 chat.
         {
             let chat = alice.create_chat(&bob).await;
-            let msg_id = chat::get_chat_msgs(&alice.ctx, chat.get_id())
+            let msg_ids: Vec<_> = chat::get_chat_msgs(&alice.ctx, chat.get_id())
                 .await
                 .unwrap()
                 .into_iter()
@@ -929,11 +939,17 @@ mod tests {
                     chat::ChatItem::Message { msg_id } => Some(msg_id),
                     _ => None,
                 })
-                .max()
-                .expect("No messages in Alice's 1:1 chat");
-            let msg = Message::load_from_db(&alice.ctx, msg_id).await.unwrap();
-            assert!(msg.is_info());
-            assert!(msg.get_text().contains("bob@example.net verified"));
+                .collect();
+            assert_eq!(msg_ids.len(), 2);
+
+            let msg0 = Message::load_from_db(&alice.ctx, msg_ids[0]).await.unwrap();
+            assert!(msg0.is_info());
+            assert!(msg0.get_text().contains("bob@example.net verified"));
+
+            let msg1 = Message::load_from_db(&alice.ctx, msg_ids[1]).await.unwrap();
+            assert!(msg1.is_info());
+            let expected_text = chat_protection_enabled(&alice).await;
+            assert_eq!(msg1.get_text(), expected_text);
         }
 
         // Check Alice sent the right message to Bob.
@@ -969,7 +985,7 @@ mod tests {
         // Check Bob got the verified message in his 1:1 chat.
         {
             let chat = bob.create_chat(&alice).await;
-            let msg_id = chat::get_chat_msgs(&bob.ctx, chat.get_id())
+            let msg_ids: Vec<_> = chat::get_chat_msgs(&bob.ctx, chat.get_id())
                 .await
                 .unwrap()
                 .into_iter()
@@ -977,11 +993,16 @@ mod tests {
                     chat::ChatItem::Message { msg_id } => Some(msg_id),
                     _ => None,
                 })
-                .max()
-                .expect("No messages in Bob's 1:1 chat");
-            let msg = Message::load_from_db(&bob.ctx, msg_id).await.unwrap();
-            assert!(msg.is_info());
-            assert!(msg.get_text().contains("alice@example.org verified"));
+                .collect();
+
+            let msg0 = Message::load_from_db(&bob.ctx, msg_ids[0]).await.unwrap();
+            assert!(msg0.is_info());
+            assert!(msg0.get_text().contains("alice@example.org verified"));
+
+            let msg1 = Message::load_from_db(&bob.ctx, msg_ids[1]).await.unwrap();
+            assert!(msg1.is_info());
+            let expected_text = chat_protection_enabled(&bob).await;
+            assert_eq!(msg1.get_text(), expected_text);
         }
 
         // Check Bob sent the final message
@@ -1278,17 +1299,11 @@ mod tests {
                 Blocked::Yes,
                 "Alice's 1:1 chat with Bob is not hidden"
             );
-            let msg_id = chat::get_chat_msgs(&alice.ctx, alice_chatid)
-                .await
-                .unwrap()
-                .into_iter()
-                .filter_map(|item| match item {
-                    chat::ChatItem::Message { msg_id } => Some(msg_id),
-                    _ => None,
-                })
-                .min()
-                .expect("No messages in Alice's group chat");
-            let msg = Message::load_from_db(&alice.ctx, msg_id).await.unwrap();
+            // There should be 3 messages in the chat:
+            // - The ChatProtectionEnabled message
+            // - bob@example.net verified
+            // - You added member bob@example.net
+            let msg = get_chat_msg(&alice, alice_chatid, 1, 3).await;
             assert!(msg.is_info());
             assert!(msg.get_text().contains("bob@example.net verified"));
         }

src/securejoin/bob.rs

@@ -222,6 +222,14 @@ impl BobState {
         let msg = stock_str::contact_verified(context, &contact).await;
         let chat_id = self.joining_chat_id(context).await?;
         chat::add_info_msg(context, chat_id, &msg, time()).await?;
+        chat_id
+            .set_protection(
+                context,
+                ProtectionStatus::Protected,
+                time(),
+                Some(contact.id),
+            )
+            .await?;
         context.emit_event(EventType::ChatModified(chat_id));
         Ok(())
     }

src/smtp.rs

@@ -492,20 +492,7 @@ pub(crate) async fn smtp_send(
 
     if let SendResult::Failure(err) = &status {
         // We couldn't send the message, so mark it as failed
-        match Message::load_from_db(context, msg_id).await {
-            Ok(mut msg) => {
-                if let Err(err) = message::set_msg_failed(context, &mut msg, &err.to_string()).await
-                {
-                    error!(context, "Failed to mark {msg_id} as failed: {err:#}.");
-                }
-            }
-            Err(err) => {
-                error!(
-                    context,
-                    "Failed to load {msg_id} to mark it as failed: {err:#}."
-                );
-            }
-        }
+        message::set_msg_failed(context, msg_id, &err.to_string()).await;
     }
     status
 }
@@ -552,8 +539,7 @@ pub(crate) async fn send_msg_to_smtp(
         )
         .await?;
     if retries > 6 {
-        let mut msg = Message::load_from_db(context, msg_id).await?;
-        message::set_msg_failed(context, &mut msg, "Number of retries exceeded the limit.").await?;
+        message::set_msg_failed(context, msg_id, "Number of retries exceeded the limit.").await;
         context
             .sql
             .execute("DELETE FROM smtp WHERE id=?", (rowid,))
@@ -579,6 +565,24 @@ pub(crate) async fn send_msg_to_smtp(
         )
         .collect::<Vec<_>>();
 
+    // If there is a msg-id and it does not exist in the db, cancel sending. this happens if
+    // delete_msgs() was called before the generated mime was sent out.
+    if !message::exists(context, msg_id)
+        .await
+        .with_context(|| format!("failed to check message {msg_id} existence"))?
+    {
+        info!(
+            context,
+            "Sending of message {msg_id} (entry {rowid}) was cancelled by the user."
+        );
+        context
+            .sql
+            .execute("DELETE FROM smtp WHERE id=?", (rowid,))
+            .await
+            .context("failed to remove cancelled message from smtp table")?;
+        return Ok(());
+    }
+
     let status = smtp_send(context, &recipients_list, body.as_str(), smtp, msg_id).await;
 
     match status {

src/sql.rs

@@ -304,25 +304,6 @@ impl Sql {
         }
     }
 
-    /// Changes the passphrase of encrypted database.
-    ///
-    /// The database must already be encrypted and the passphrase cannot be empty.
-    /// It is impossible to turn encrypted database into unencrypted
-    /// and vice versa this way, use import/export for this.
-    pub async fn change_passphrase(&self, passphrase: String) -> Result<()> {
-        let mut lock = self.pool.write().await;
-
-        let pool = lock.take().context("SQL connection pool is not open")?;
-        let conn = pool.get().await?;
-        conn.pragma_update(None, "rekey", passphrase.clone())
-            .context("failed to set PRAGMA rekey")?;
-        drop(pool);
-
-        *lock = Some(Self::new_pool(&self.dbfile, passphrase.to_string())?);
-
-        Ok(())
-    }
-
     /// Locks the write transactions mutex in order to make sure that there never are
     /// multiple write transactions at once.
     ///
@@ -1265,66 +1246,6 @@ mod tests {
         sql.open(&t, "foo".to_string())
             .await
             .context("failed to open the database second time")?;
-
-        Ok(())
-    }
-
-    #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
-    async fn test_sql_change_passphrase() -> Result<()> {
-        use tempfile::tempdir;
-
-        // The context is used only for logging.
-        let t = TestContext::new().await;
-
-        // Create a separate empty database for testing.
-        let dir = tempdir()?;
-        let dbfile = dir.path().join("testdb.sqlite");
-        let sql = Sql::new(dbfile.clone());
-
-        sql.open(&t, "foo".to_string())
-            .await
-            .context("failed to open the database first time")?;
-        sql.close().await;
-
-        // Change the passphrase from "foo" to "bar".
-        let sql = Sql::new(dbfile.clone());
-        sql.open(&t, "foo".to_string())
-            .await
-            .context("failed to open the database second time")?;
-        sql.change_passphrase("bar".to_string())
-            .await
-            .context("failed to change passphrase")?;
-
-        // Test that at least two connections are still working.
-        // This ensures that not only the connection which changed the password is working,
-        // but other connections as well.
-        {
-            let lock = sql.pool.read().await;
-            let pool = lock.as_ref().unwrap();
-            let conn1 = pool.get().await?;
-            let conn2 = pool.get().await?;
-            conn1
-                .query_row("SELECT count(*) FROM sqlite_master", [], |_row| Ok(()))
-                .unwrap();
-            conn2
-                .query_row("SELECT count(*) FROM sqlite_master", [], |_row| Ok(()))
-                .unwrap();
-        }
-
-        sql.close().await;
-
-        let sql = Sql::new(dbfile);
-
-        // Test that old passphrase is not working.
-        assert!(sql.open(&t, "foo".to_string()).await.is_err());
-
-        // Open the database with the new passphrase.
-        sql.check_passphrase("bar".to_string()).await?;
-        sql.open(&t, "bar".to_string())
-            .await
-            .context("failed to open the database third time")?;
-        sql.close().await;
-
         Ok(())
     }
 }

src/sql/migrations.rs

@@ -516,8 +516,7 @@ DO UPDATE SET rfc724_mid=excluded.rfc724_mid,
 id INTEGER PRIMARY KEY AUTOINCREMENT,
 msg_id INTEGER,
 update_item TEXT DEFAULT '',
-update_item_read INTEGER DEFAULT 0 -- XXX unused
-);
+update_item_read INTEGER DEFAULT 0);
 CREATE INDEX msgs_status_updates_index1 ON msgs_status_updates (msg_id);"#,
             84,
         )

src/stock_str.rs

@@ -393,18 +393,6 @@ pub enum StockMessage {
     #[strum(props(fallback = "Message deletion timer is set to %1$s weeks by %2$s."))]
     MsgEphemeralTimerWeeksBy = 157,
 
-    #[strum(props(fallback = "You enabled chat protection."))]
-    YouEnabledProtection = 158,
-
-    #[strum(props(fallback = "Chat protection enabled by %1$s."))]
-    ProtectionEnabledBy = 159,
-
-    #[strum(props(fallback = "You disabled chat protection."))]
-    YouDisabledProtection = 160,
-
-    #[strum(props(fallback = "Chat protection disabled by %1$s."))]
-    ProtectionDisabledBy = 161,
-
     #[strum(props(fallback = "Scan to set up second device for %1$s"))]
     BackupTransferQr = 162,
 
@@ -419,6 +407,12 @@ pub enum StockMessage {
 
     #[strum(props(fallback = "I left the group."))]
     MsgILeftGroup = 166,
+
+    #[strum(props(fallback = "Messages are guaranteed to be end-to-end encrypted from now on."))]
+    ChatProtectionEnabled = 170,
+
+    #[strum(props(fallback = "%1$s sent a message from another device."))]
+    ChatProtectionDisabled = 171,
 }
 
 impl StockMessage {
@@ -515,13 +509,21 @@ trait StockStringMods: AsRef<str> + Sized {
 }
 
 impl ContactId {
-    /// Get contact name for stock string.
-    async fn get_stock_name(self, context: &Context) -> String {
+    /// Get contact name and address for stock string, e.g. `Bob (bob@example.net)`
+    async fn get_stock_name_n_addr(self, context: &Context) -> String {
         Contact::get_by_id(context, self)
             .await
             .map(|contact| contact.get_name_n_addr())
             .unwrap_or_else(|_| self.to_string())
     }
+
+    /// Get contact name, e.g. `Bob`, or `bob@exmple.net` if no name is set.
+    async fn get_stock_name(self, context: &Context) -> String {
+        Contact::get_by_id(context, self)
+            .await
+            .map(|contact| contact.get_display_name().to_string())
+            .unwrap_or_else(|_| self.to_string())
+    }
 }
 
 impl StockStringMods for String {}
@@ -583,7 +585,7 @@ pub(crate) async fn msg_grp_name(
             .await
             .replace1(from_group)
             .replace2(to_group)
-            .replace3(&by_contact.get_stock_name(context).await)
+            .replace3(&by_contact.get_stock_name_n_addr(context).await)
     }
 }
 
@@ -593,7 +595,7 @@ pub(crate) async fn msg_grp_img_changed(context: &Context, by_contact: ContactId
     } else {
         translated(context, StockMessage::MsgGrpImgChangedBy)
             .await
-            .replace1(&by_contact.get_stock_name(context).await)
+            .replace1(&by_contact.get_stock_name_n_addr(context).await)
     }
 }
 
@@ -640,7 +642,7 @@ pub(crate) async fn msg_add_member_local(
         translated(context, StockMessage::MsgAddMemberBy)
             .await
             .replace1(whom)
-            .replace2(&by_contact.get_stock_name(context).await)
+            .replace2(&by_contact.get_stock_name_n_addr(context).await)
     }
 }
 
@@ -687,7 +689,7 @@ pub(crate) async fn msg_del_member_local(
         translated(context, StockMessage::MsgDelMemberBy)
             .await
             .replace1(whom)
-            .replace2(&by_contact.get_stock_name(context).await)
+            .replace2(&by_contact.get_stock_name_n_addr(context).await)
     }
 }
 
@@ -703,7 +705,7 @@ pub(crate) async fn msg_group_left_local(context: &Context, by_contact: ContactI
     } else {
         translated(context, StockMessage::MsgGroupLeftBy)
             .await
-            .replace1(&by_contact.get_stock_name(context).await)
+            .replace1(&by_contact.get_stock_name_n_addr(context).await)
     }
 }
 
@@ -756,7 +758,7 @@ pub(crate) async fn msg_grp_img_deleted(context: &Context, by_contact: ContactId
     } else {
         translated(context, StockMessage::MsgGrpImgDeletedBy)
             .await
-            .replace1(&by_contact.get_stock_name(context).await)
+            .replace1(&by_contact.get_stock_name_n_addr(context).await)
     }
 }
 
@@ -782,13 +784,9 @@ pub(crate) async fn secure_join_started(
 
 /// Stock string: `%1$s replied, waiting for being added to the group…`.
 pub(crate) async fn secure_join_replies(context: &Context, contact_id: ContactId) -> String {
-    if let Ok(contact) = Contact::get_by_id(context, contact_id).await {
-        translated(context, StockMessage::SecureJoinReplies)
-            .await
-            .replace1(contact.get_display_name())
-    } else {
-        format!("secure_join_replies: unknown contact {contact_id}")
-    }
+    translated(context, StockMessage::SecureJoinReplies)
+        .await
+        .replace1(&contact_id.get_stock_name(context).await)
 }
 
 /// Stock string: `Scan to chat with %1$s`.
@@ -881,7 +879,7 @@ pub(crate) async fn msg_location_enabled_by(context: &Context, contact: ContactI
     } else {
         translated(context, StockMessage::MsgLocationEnabledBy)
             .await
-            .replace1(&contact.get_stock_name(context).await)
+            .replace1(&contact.get_stock_name_n_addr(context).await)
     }
 }
 
@@ -950,7 +948,7 @@ pub(crate) async fn msg_ephemeral_timer_disabled(
     } else {
         translated(context, StockMessage::MsgEphemeralTimerDisabledBy)
             .await
-            .replace1(&by_contact.get_stock_name(context).await)
+            .replace1(&by_contact.get_stock_name_n_addr(context).await)
     }
 }
 
@@ -968,7 +966,7 @@ pub(crate) async fn msg_ephemeral_timer_enabled(
         translated(context, StockMessage::MsgEphemeralTimerEnabledBy)
             .await
             .replace1(timer)
-            .replace2(&by_contact.get_stock_name(context).await)
+            .replace2(&by_contact.get_stock_name_n_addr(context).await)
     }
 }
 
@@ -979,7 +977,7 @@ pub(crate) async fn msg_ephemeral_timer_minute(context: &Context, by_contact: Co
     } else {
         translated(context, StockMessage::MsgEphemeralTimerMinuteBy)
             .await
-            .replace1(&by_contact.get_stock_name(context).await)
+            .replace1(&by_contact.get_stock_name_n_addr(context).await)
     }
 }
 
@@ -990,7 +988,7 @@ pub(crate) async fn msg_ephemeral_timer_hour(context: &Context, by_contact: Cont
     } else {
         translated(context, StockMessage::MsgEphemeralTimerHourBy)
             .await
-            .replace1(&by_contact.get_stock_name(context).await)
+            .replace1(&by_contact.get_stock_name_n_addr(context).await)
     }
 }
 
@@ -1001,7 +999,7 @@ pub(crate) async fn msg_ephemeral_timer_day(context: &Context, by_contact: Conta
     } else {
         translated(context, StockMessage::MsgEphemeralTimerDayBy)
             .await
-            .replace1(&by_contact.get_stock_name(context).await)
+            .replace1(&by_contact.get_stock_name_n_addr(context).await)
     }
 }
 
@@ -1012,7 +1010,7 @@ pub(crate) async fn msg_ephemeral_timer_week(context: &Context, by_contact: Cont
     } else {
         translated(context, StockMessage::MsgEphemeralTimerWeekBy)
             .await
-            .replace1(&by_contact.get_stock_name(context).await)
+            .replace1(&by_contact.get_stock_name_n_addr(context).await)
     }
 }
 
@@ -1053,26 +1051,16 @@ pub(crate) async fn error_no_network(context: &Context) -> String {
     translated(context, StockMessage::ErrorNoNetwork).await
 }
 
-/// Stock string: `Chat protection enabled.`.
-pub(crate) async fn protection_enabled(context: &Context, by_contact: ContactId) -> String {
-    if by_contact == ContactId::SELF {
-        translated(context, StockMessage::YouEnabledProtection).await
-    } else {
-        translated(context, StockMessage::ProtectionEnabledBy)
-            .await
-            .replace1(&by_contact.get_stock_name(context).await)
-    }
+/// Stock string: `Messages are guaranteed to be end-to-end encrypted from now on.`
+pub(crate) async fn chat_protection_enabled(context: &Context) -> String {
+    translated(context, StockMessage::ChatProtectionEnabled).await
 }
 
-/// Stock string: `Chat protection disabled.`.
-pub(crate) async fn protection_disabled(context: &Context, by_contact: ContactId) -> String {
-    if by_contact == ContactId::SELF {
-        translated(context, StockMessage::YouDisabledProtection).await
-    } else {
-        translated(context, StockMessage::ProtectionDisabledBy)
-            .await
-            .replace1(&by_contact.get_stock_name(context).await)
-    }
+/// Stock string: `%1$s sent a message from another device.`
+pub(crate) async fn chat_protection_disabled(context: &Context, contact_id: ContactId) -> String {
+    translated(context, StockMessage::ChatProtectionDisabled)
+        .await
+        .replace1(&contact_id.get_stock_name(context).await)
 }
 
 /// Stock string: `Reply`.
@@ -1104,7 +1092,7 @@ pub(crate) async fn msg_ephemeral_timer_minutes(
         translated(context, StockMessage::MsgEphemeralTimerMinutesBy)
             .await
             .replace1(minutes)
-            .replace2(&by_contact.get_stock_name(context).await)
+            .replace2(&by_contact.get_stock_name_n_addr(context).await)
     }
 }
 
@@ -1122,7 +1110,7 @@ pub(crate) async fn msg_ephemeral_timer_hours(
         translated(context, StockMessage::MsgEphemeralTimerHoursBy)
             .await
             .replace1(hours)
-            .replace2(&by_contact.get_stock_name(context).await)
+            .replace2(&by_contact.get_stock_name_n_addr(context).await)
     }
 }
 
@@ -1140,7 +1128,7 @@ pub(crate) async fn msg_ephemeral_timer_days(
         translated(context, StockMessage::MsgEphemeralTimerDaysBy)
             .await
             .replace1(days)
-            .replace2(&by_contact.get_stock_name(context).await)
+            .replace2(&by_contact.get_stock_name_n_addr(context).await)
     }
 }
 
@@ -1158,7 +1146,7 @@ pub(crate) async fn msg_ephemeral_timer_weeks(
         translated(context, StockMessage::MsgEphemeralTimerWeeksBy)
             .await
             .replace1(weeks)
-            .replace2(&by_contact.get_stock_name(context).await)
+            .replace2(&by_contact.get_stock_name_n_addr(context).await)
     }
 }
 
@@ -1332,11 +1320,19 @@ impl Context {
     pub(crate) async fn stock_protection_msg(
         &self,
         protect: ProtectionStatus,
-        from_id: ContactId,
+        contact_id: Option<ContactId>,
     ) -> String {
         match protect {
-            ProtectionStatus::Unprotected => protection_enabled(self, from_id).await,
-            ProtectionStatus::Protected => protection_disabled(self, from_id).await,
+            ProtectionStatus::Unprotected | ProtectionStatus::ProtectionBroken => {
+                if let Some(contact_id) = contact_id {
+                    chat_protection_disabled(self, contact_id).await
+                } else {
+                    // In a group chat, it's not possible to downgrade verification.
+                    // In a 1:1 chat, the `contact_id` always has to be provided.
+                    "[Error] No contact_id given".to_string()
+                }
+            }
+            ProtectionStatus::Protected => chat_protection_enabled(self).await,
         }
     }
 

src/summary.rs

@@ -9,6 +9,7 @@ use crate::contact::{Contact, ContactId};
 use crate::context::Context;
 use crate::message::{Message, MessageState, Viewtype};
 use crate::mimeparser::SystemMessage;
+use crate::param::Param;
 use crate::stock_str;
 use crate::tools::truncate;
 
@@ -132,8 +133,14 @@ impl Message {
                     append_text = false;
                     stock_str::ac_setup_msg_subject(context).await
                 } else {
-                    let file_name = self
-                        .get_filename()
+                    let file_name: String = self
+                        .param
+                        .get_path(Param::File, context)
+                        .unwrap_or(None)
+                        .and_then(|path| {
+                            path.file_name()
+                                .map(|fname| fname.to_string_lossy().into_owned())
+                        })
                         .unwrap_or_else(|| String::from("ErrFileName"));
                     let label = if self.viewtype == Viewtype::Audio {
                         stock_str::audio(context).await
@@ -193,7 +200,6 @@ impl Message {
 #[cfg(test)]
 mod tests {
     use super::*;
-    use crate::param::Param;
     use crate::test_utils as test;
 
     #[tokio::test(flavor = "multi_thread", worker_threads = 2)]

src/test_utils.rs

@@ -31,11 +31,14 @@ use crate::constants::Chattype;
 use crate::constants::{DC_GCL_NO_SPECIALS, DC_MSG_ID_DAYMARKER};
 use crate::contact::{Contact, ContactAddress, ContactId, Modifier, Origin};
 use crate::context::Context;
+use crate::e2ee::EncryptHelper;
 use crate::events::{Event, EventType, Events};
 use crate::key::{self, DcKey, KeyPair, KeyPairUse};
 use crate::message::{update_msg_state, Message, MessageState, MsgId, Viewtype};
-use crate::mimeparser::MimeMessage;
+use crate::mimeparser::{MimeMessage, SystemMessage};
+use crate::peerstate::Peerstate;
 use crate::receive_imf::receive_imf;
+use crate::securejoin::{get_securejoin_qr, join_securejoin};
 use crate::stock_str::StockStrings;
 use crate::tools::EmailAddress;
 
@@ -108,9 +111,15 @@ impl TestContextManager {
     /// - Let one TestContext send a message
     /// - Let the other TestContext receive it and accept the chat
     /// - Assert that the message arrived
-    pub async fn send_recv_accept(&self, from: &TestContext, to: &TestContext, msg: &str) {
+    pub async fn send_recv_accept(
+        &self,
+        from: &TestContext,
+        to: &TestContext,
+        msg: &str,
+    ) -> Message {
         let received_msg = self.send_recv(from, to, msg).await;
         received_msg.chat_id.accept(to).await.unwrap();
+        received_msg
     }
 
     /// - Let one TestContext send a message
@@ -152,6 +161,27 @@ impl TestContextManager {
             new_addr
         );
     }
+
+    pub async fn execute_securejoin(&self, scanner: &TestContext, scanned: &TestContext) {
+        self.section(&format!(
+            "{} scans {}'s QR code",
+            scanner.name(),
+            scanned.name()
+        ));
+
+        let qr = get_securejoin_qr(&scanned.ctx, None).await.unwrap();
+        join_securejoin(&scanner.ctx, &qr).await.unwrap();
+
+        loop {
+            if let Some(sent) = scanner.pop_sent_msg_opt(Duration::ZERO).await {
+                scanned.recv_msg(&sent).await;
+            } else if let Some(sent) = scanned.pop_sent_msg_opt(Duration::ZERO).await {
+                scanner.recv_msg(&sent).await;
+            } else {
+                break;
+            }
+        }
+    }
 }
 
 #[derive(Debug, Clone, Default)]
@@ -636,7 +666,7 @@ impl TestContext {
         // We're using `unwrap_or_default()` here so that if the file doesn't exist,
         // it can be created using `write` below.
         let expected = fs::read(&filename).await.unwrap_or_default();
-        let expected = String::from_utf8(expected).unwrap();
+        let expected = String::from_utf8(expected).unwrap().replace("\r\n", "\n");
         if (std::env::var("UPDATE_GOLDEN_TESTS") == Ok("1".to_string())) && actual != expected {
             fs::write(&filename, &actual)
                 .await
@@ -1008,6 +1038,26 @@ fn print_logevent(logevent: &LogEvent) {
     }
 }
 
+/// Saves the other account's public key as verified.
+pub(crate) async fn mark_as_verified(this: &TestContext, other: &TestContext) {
+    let mut peerstate = Peerstate::from_header(
+        &EncryptHelper::new(other).await.unwrap().get_aheader(),
+        // We have to give 0 as the time, not the current time:
+        // The time is going to be saved in peerstate.last_seen.
+        // The code in `peerstate.rs` then compares `if message_time > self.last_seen`,
+        // and many similar checks in peerstate.rs, and doesn't allow changes otherwise.
+        // Giving the current time would mean that message_time == peerstate.last_seen,
+        // so changes would not be allowed.
+        // This might lead to flaky tests.
+        0,
+    );
+
+    peerstate.verified_key = peerstate.public_key.clone();
+    peerstate.verified_key_fingerprint = peerstate.public_key_fingerprint.clone();
+
+    peerstate.save_to_db(&this.sql).await.unwrap();
+}
+
 /// Pretty-print an event to stdout
 ///
 /// Done during tests this is captured by `cargo test` and associated with the test itself.
@@ -1114,7 +1164,17 @@ async fn write_msg(context: &Context, prefix: &str, msg: &Message, buf: &mut Str
         } else {
             "[FRESH]"
         },
-        if msg.is_info() { "[INFO]" } else { "" },
+        if msg.is_info() {
+            if msg.get_info_type() == SystemMessage::ChatProtectionEnabled {
+                "[INFO 🛡️]"
+            } else if msg.get_info_type() == SystemMessage::ChatProtectionDisabled {
+                "[INFO 🛡️❌]"
+            } else {
+                "[INFO]"
+            }
+        } else {
+            ""
+        },
         if msg.get_viewtype() == Viewtype::VideochatInvitation {
             format!(
                 "[VIDEOCHAT-INVITATION: {}, type={}]",

src/tests.rs

@@ -1 +1,2 @@
 mod aeap;
+mod verified_chats;

src/tests/aeap.rs

@@ -8,10 +8,10 @@ use crate::contact;
 use crate::contact::Contact;
 use crate::contact::ContactId;
 use crate::message::Message;
-use crate::peerstate;
 use crate::peerstate::Peerstate;
 use crate::receive_imf::receive_imf;
 use crate::stock_str;
+use crate::test_utils::mark_as_verified;
 use crate::test_utils::TestContext;
 use crate::test_utils::TestContextManager;
 
@@ -327,19 +327,6 @@ async fn check_no_transition_done(groups: &[ChatId], old_alice_addr: &str, bob:
     }
 }
 
-async fn mark_as_verified(this: &TestContext, other: &TestContext) {
-    let other_addr = other.get_primary_self_addr().await.unwrap();
-    let mut peerstate = peerstate::Peerstate::from_addr(this, &other_addr)
-        .await
-        .unwrap()
-        .unwrap();
-
-    peerstate.verified_key = peerstate.public_key.clone();
-    peerstate.verified_key_fingerprint = peerstate.public_key_fingerprint.clone();
-
-    peerstate.save_to_db(&this.sql).await.unwrap();
-}
-
 async fn get_last_info_msg(t: &TestContext, chat_id: ChatId) -> Option<Message> {
     let msgs = chat::get_chat_msgs_ex(
         &t.ctx,

src/tests/verified_chats.rs

@@ -0,0 +1,714 @@
+use anyhow::Result;
+use pretty_assertions::assert_eq;
+
+use crate::chat::{Chat, ProtectionStatus};
+use crate::config::Config;
+use crate::contact::VerifiedStatus;
+use crate::contact::{Contact, Origin};
+use crate::message::{Message, Viewtype};
+use crate::mimefactory::MimeFactory;
+use crate::mimeparser::SystemMessage;
+use crate::receive_imf::receive_imf;
+use crate::stock_str;
+use crate::test_utils::{get_chat_msg, mark_as_verified, TestContext, TestContextManager};
+use crate::{e2ee, message};
+
+#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
+async fn test_verified_oneonone_chat_broken_by_classical() {
+    check_verified_oneonone_chat(true).await;
+}
+
+#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
+async fn test_verified_oneonone_chat_broken_by_device_change() {
+    check_verified_oneonone_chat(false).await;
+}
+
+async fn check_verified_oneonone_chat(broken_by_classical_email: bool) {
+    let mut tcm = TestContextManager::new();
+    let alice = tcm.alice().await;
+    let bob = tcm.bob().await;
+    enable_verified_oneonone_chats(&[&alice, &bob]).await;
+
+    tcm.execute_securejoin(&alice, &bob).await;
+
+    assert_verified(&alice, &bob, ProtectionStatus::Protected).await;
+    assert_verified(&bob, &alice, ProtectionStatus::Protected).await;
+
+    if broken_by_classical_email {
+        tcm.section("Bob uses a classical MUA to send a message to Alice");
+        receive_imf(
+            &alice,
+            b"Subject: Re: Message from alice\r\n\
+          From: <bob@example.net>\r\n\
+          To: <alice@example.org>\r\n\
+          Date: Mon, 12 Dec 2022 14:33:39 +0000\r\n\
+          Message-ID: <abcd@example.net>\r\n\
+          \r\n\
+          Heyho!\r\n",
+            false,
+        )
+        .await
+        .unwrap()
+        .unwrap();
+    } else {
+        tcm.section("Bob sets up another Delta Chat device");
+        let bob2 = TestContext::new().await;
+        enable_verified_oneonone_chats(&[&bob2]).await;
+        bob2.set_name("bob2");
+        bob2.configure_addr("bob@example.net").await;
+
+        tcm.send_recv(&bob2, &alice, "Using another device now")
+            .await;
+    }
+
+    // Bob's contact is still verified, but the chat isn't marked as protected anymore
+    assert_verified(&alice, &bob, ProtectionStatus::ProtectionBroken).await;
+
+    tcm.section("Bob sends another message from DC");
+    tcm.send_recv(&bob, &alice, "Using DC again").await;
+
+    let contact = alice.add_or_lookup_contact(&bob).await;
+    assert_eq!(
+        contact.is_verified(&alice.ctx).await.unwrap(),
+        VerifiedStatus::BidirectVerified
+    );
+
+    // Bob's chat is marked as verified again
+    assert_verified(&alice, &bob, ProtectionStatus::Protected).await;
+}
+
+#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
+async fn test_create_verified_oneonone_chat() -> Result<()> {
+    let mut tcm = TestContextManager::new();
+    let alice = tcm.alice().await;
+    let bob = tcm.bob().await;
+    let fiona = tcm.fiona().await;
+    enable_verified_oneonone_chats(&[&alice, &bob, &fiona]).await;
+
+    tcm.execute_securejoin(&alice, &bob).await;
+    tcm.execute_securejoin(&bob, &fiona).await;
+    assert_verified(&alice, &bob, ProtectionStatus::Protected).await;
+    assert_verified(&bob, &alice, ProtectionStatus::Protected).await;
+    assert_verified(&bob, &fiona, ProtectionStatus::Protected).await;
+    assert_verified(&fiona, &bob, ProtectionStatus::Protected).await;
+
+    let group_id = bob
+        .create_group_with_members(
+            ProtectionStatus::Protected,
+            "Group with everyone",
+            &[&alice, &fiona],
+        )
+        .await;
+    assert_eq!(
+        get_chat_msg(&bob, group_id, 0, 1).await.get_info_type(),
+        SystemMessage::ChatProtectionEnabled
+    );
+
+    {
+        let sent = bob.send_text(group_id, "Heyho").await;
+        alice.recv_msg(&sent).await;
+
+        let msg = fiona.recv_msg(&sent).await;
+        assert_eq!(
+            get_chat_msg(&fiona, msg.chat_id, 0, 2)
+                .await
+                .get_info_type(),
+            SystemMessage::ChatProtectionEnabled
+        );
+    }
+
+    // Alice and Fiona should now be verified because of gossip
+    let alice_fiona_contact = alice.add_or_lookup_contact(&fiona).await;
+    assert_eq!(
+        alice_fiona_contact.is_verified(&alice).await.unwrap(),
+        VerifiedStatus::BidirectVerified
+    );
+
+    // As soon as Alice creates a chat with Fiona, it should directly be protected
+    {
+        let chat = alice.create_chat(&fiona).await;
+        assert!(chat.is_protected());
+
+        let msg = alice.get_last_msg().await;
+        let expected_text = stock_str::chat_protection_enabled(&alice).await;
+        assert_eq!(msg.text, expected_text);
+    }
+
+    // Fiona should also see the chat as protected
+    {
+        let rcvd = tcm.send_recv(&alice, &fiona, "Hi Fiona").await;
+        let alice_fiona_id = rcvd.chat_id;
+        let chat = Chat::load_from_db(&fiona, alice_fiona_id).await?;
+        assert!(chat.is_protected());
+
+        let msg0 = get_chat_msg(&fiona, chat.id, 0, 2).await;
+        let expected_text = stock_str::chat_protection_enabled(&fiona).await;
+        assert_eq!(msg0.text, expected_text);
+    }
+
+    tcm.section("Fiona reinstalls DC");
+    drop(fiona);
+
+    let fiona_new = tcm.unconfigured().await;
+    enable_verified_oneonone_chats(&[&fiona_new]).await;
+    fiona_new.configure_addr("fiona@example.net").await;
+    e2ee::ensure_secret_key_exists(&fiona_new).await?;
+
+    tcm.send_recv(&fiona_new, &alice, "I have a new device")
+        .await;
+
+    // The chat should be and stay unprotected
+    {
+        let chat = alice.get_chat(&fiona_new).await.unwrap();
+        assert!(!chat.is_protected());
+        assert!(chat.is_protection_broken());
+
+        // After recreating the chat, it should still be unprotected
+        chat.id.delete(&alice).await?;
+
+        let chat = alice.create_chat(&fiona_new).await;
+        assert!(!chat.is_protected());
+        assert!(!chat.is_protection_broken());
+    }
+
+    Ok(())
+}
+
+#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
+async fn test_create_unverified_oneonone_chat() -> Result<()> {
+    let mut tcm = TestContextManager::new();
+    let alice = tcm.alice().await;
+    let bob = tcm.bob().await;
+    enable_verified_oneonone_chats(&[&alice, &bob]).await;
+
+    // A chat with an unknown contact should be created unprotected
+    let chat = alice.create_chat(&bob).await;
+    assert!(!chat.is_protected());
+    assert!(!chat.is_protection_broken());
+
+    receive_imf(
+        &alice,
+        b"From: Bob <bob@example.net>\n\
+          To: alice@example.org\n\
+          Message-ID: <1234-2@example.org>\n\
+          \n\
+          hello\n",
+        false,
+    )
+    .await?;
+
+    chat.id.delete(&alice).await.unwrap();
+    // Now Bob is a known contact, new chats should still be created unprotected
+    let chat = alice.create_chat(&bob).await;
+    assert!(!chat.is_protected());
+    assert!(!chat.is_protection_broken());
+
+    tcm.send_recv(&bob, &alice, "hi").await;
+    chat.id.delete(&alice).await.unwrap();
+    // Now we have a public key, new chats should still be created unprotected
+    let chat = alice.create_chat(&bob).await;
+    assert!(!chat.is_protected());
+    assert!(!chat.is_protection_broken());
+
+    Ok(())
+}
+
+#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
+async fn test_degrade_verified_oneonone_chat() -> Result<()> {
+    let mut tcm = TestContextManager::new();
+    let alice = tcm.alice().await;
+    let bob = tcm.bob().await;
+    enable_verified_oneonone_chats(&[&alice, &bob]).await;
+
+    mark_as_verified(&alice, &bob).await;
+
+    let alice_chat = alice.create_chat(&bob).await;
+    assert!(alice_chat.is_protected());
+
+    receive_imf(
+        &alice,
+        b"From: Bob <bob@example.net>\n\
+          To: alice@example.org\n\
+          Message-ID: <1234-2@example.org>\n\
+          \n\
+          hello\n",
+        false,
+    )
+    .await?;
+
+    let contact_id = Contact::lookup_id_by_addr(&alice, "bob@example.net", Origin::Hidden)
+        .await?
+        .unwrap();
+
+    let msg0 = get_chat_msg(&alice, alice_chat.id, 0, 3).await;
+    let enabled = stock_str::chat_protection_enabled(&alice).await;
+    assert_eq!(msg0.text, enabled);
+    assert_eq!(msg0.param.get_cmd(), SystemMessage::ChatProtectionEnabled);
+
+    let msg1 = get_chat_msg(&alice, alice_chat.id, 1, 3).await;
+    let disabled = stock_str::chat_protection_disabled(&alice, contact_id).await;
+    assert_eq!(msg1.text, disabled);
+    assert_eq!(msg1.param.get_cmd(), SystemMessage::ChatProtectionDisabled);
+
+    let msg2 = get_chat_msg(&alice, alice_chat.id, 2, 3).await;
+    assert_eq!(msg2.text, "hello".to_string());
+    assert!(!msg2.is_system_message());
+
+    Ok(())
+}
+
+#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
+async fn test_verified_oneonone_chat_enable_disable() -> Result<()> {
+    let mut tcm = TestContextManager::new();
+    let alice = tcm.alice().await;
+    let bob = tcm.bob().await;
+    enable_verified_oneonone_chats(&[&alice, &bob]).await;
+
+    // Alice & Bob verify each other
+    mark_as_verified(&alice, &bob).await;
+    mark_as_verified(&bob, &alice).await;
+
+    let chat = alice.create_chat(&bob).await;
+    assert!(chat.is_protected());
+
+    for alice_accepts_breakage in [true, false] {
+        // Bob uses Thunderbird to send a message
+        receive_imf(
+            &alice,
+            format!(
+                "From: Bob <bob@example.net>\n\
+              To: alice@example.org\n\
+              Message-ID: <1234-2{alice_accepts_breakage}@example.org>\n\
+              \n\
+              Message from Thunderbird\n"
+            )
+            .as_bytes(),
+            false,
+        )
+        .await?;
+
+        let chat = alice.get_chat(&bob).await.unwrap();
+        assert!(!chat.is_protected());
+        assert!(chat.is_protection_broken());
+
+        if alice_accepts_breakage {
+            tcm.section("Alice clicks 'Accept' on the input-bar-dialog");
+            chat.id.accept(&alice).await?;
+            let chat = alice.get_chat(&bob).await.unwrap();
+            assert!(!chat.is_protected());
+            assert!(!chat.is_protection_broken());
+        }
+
+        // Bob sends a message from DC again
+        tcm.send_recv(&bob, &alice, "Hello from DC").await;
+        let chat = alice.get_chat(&bob).await.unwrap();
+        assert!(chat.is_protected());
+        assert!(!chat.is_protection_broken());
+    }
+
+    alice
+        .golden_test_chat(chat.id, "test_verified_oneonone_chat_enable_disable")
+        .await;
+
+    Ok(())
+}
+
+/// Messages with old timestamps are difficult for verified chats:
+/// - They must not be sorted over a protection-changed info message.
+///   That's what `test_old_message_2` tests
+/// - If they change the protection, then they must not be sorted over existing other messages,
+///   because then the protection-changed info message would also be above these existing messages.
+///   That's what `test_old_message_3` tests.
+///
+/// `test_old_message_1` tests the case where both the old and the new message
+/// change verification
+#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
+async fn test_old_message_1() -> Result<()> {
+    let mut tcm = TestContextManager::new();
+    let alice = tcm.alice().await;
+    let bob = tcm.bob().await;
+    enable_verified_oneonone_chats(&[&alice, &bob]).await;
+
+    mark_as_verified(&alice, &bob).await;
+
+    let chat = alice.create_chat(&bob).await; // This creates a protection-changed info message
+    assert!(chat.is_protected());
+
+    // This creates protection-changed info message #2;
+    // even though the date is old, info message and email must be sorted below the original info message.
+    receive_imf(
+        &alice,
+        b"From: Bob <bob@example.net>\n\
+          To: alice@example.org\n\
+          Message-ID: <1234-2-3@example.org>\n\
+          Date: Sat, 07 Dec 2019 19:00:27 +0000\n\
+          \n\
+          Message from Thunderbird\n",
+        true,
+    )
+    .await?;
+
+    alice.golden_test_chat(chat.id, "test_old_message_1").await;
+
+    Ok(())
+}
+
+#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
+async fn test_old_message_2() -> Result<()> {
+    let mut tcm = TestContextManager::new();
+    let alice = tcm.alice().await;
+    let bob = tcm.bob().await;
+    enable_verified_oneonone_chats(&[&alice, &bob]).await;
+
+    mark_as_verified(&alice, &bob).await;
+
+    // This creates protection-changed info message #1:
+    let chat = alice.create_chat(&bob).await;
+    assert!(chat.is_protected());
+    let protection_msg = alice.get_last_msg().await;
+    assert_eq!(
+        protection_msg.param.get_cmd(),
+        SystemMessage::ChatProtectionEnabled
+    );
+
+    // This creates protection-changed info message #2.
+    let first_email = receive_imf(
+        &alice,
+        b"From: Bob <bob@example.net>\n\
+          To: alice@example.org\n\
+          Message-ID: <1234-2-3@example.org>\n\
+          Date: Sun, 08 Dec 2019 19:00:27 +0000\n\
+          \n\
+          Somewhat old message\n",
+        false,
+    )
+    .await?
+    .unwrap();
+
+    // Both messages will get the same timestamp as the protection-changed
+    // message, so this one will be sorted under the previous one
+    // even though it has an older timestamp.
+    let second_email = receive_imf(
+        &alice,
+        b"From: Bob <bob@example.net>\n\
+          To: alice@example.org\n\
+          Message-ID: <2319-2-3@example.org>\n\
+          Date: Sat, 07 Dec 2019 19:00:27 +0000\n\
+          \n\
+          Even older message, that must NOT be shown before the info message\n",
+        true,
+    )
+    .await?
+    .unwrap();
+
+    assert_eq!(first_email.sort_timestamp, second_email.sort_timestamp);
+    assert_eq!(first_email.sort_timestamp, protection_msg.timestamp_sort);
+
+    alice.golden_test_chat(chat.id, "test_old_message_2").await;
+
+    Ok(())
+}
+
+#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
+async fn test_old_message_3() -> Result<()> {
+    let mut tcm = TestContextManager::new();
+    let alice = tcm.alice().await;
+    let bob = tcm.bob().await;
+    enable_verified_oneonone_chats(&[&alice, &bob]).await;
+
+    mark_as_verified(&alice, &bob).await;
+    mark_as_verified(&bob, &alice).await;
+
+    tcm.send_recv_accept(&bob, &alice, "Heyho from my verified device!")
+        .await;
+
+    // This unverified message must not be sorted over the message sent in the previous line:
+    receive_imf(
+        &alice,
+        b"From: Bob <bob@example.net>\n\
+          To: alice@example.org\n\
+          Message-ID: <1234-2-3@example.org>\n\
+          Date: Sat, 07 Dec 2019 19:00:27 +0000\n\
+          \n\
+          Old, unverified message\n",
+        true,
+    )
+    .await?;
+
+    alice
+        .golden_test_chat(alice.get_chat(&bob).await.unwrap().id, "test_old_message_3")
+        .await;
+
+    Ok(())
+}
+
+/// Alice is offline for some time.
+/// When she comes online, first her inbox is synced and then her sentbox.
+/// This test tests that the messages are still in the right order.
+#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
+async fn test_old_message_4() -> Result<()> {
+    let alice = TestContext::new_alice().await;
+    let msg_incoming = receive_imf(
+        &alice,
+        b"From: Bob <bob@example.net>\n\
+          To: alice@example.org\n\
+          Message-ID: <1234-2-3@example.org>\n\
+          Date: Sun, 08 Dec 2019 19:00:27 +0000\n\
+          \n\
+          Thanks, Alice!\n",
+        true,
+    )
+    .await?
+    .unwrap();
+
+    let msg_sent = receive_imf(
+        &alice,
+        b"From: alice@example.org\n\
+          To: Bob <bob@example.net>\n\
+          Message-ID: <1234-2-4@example.org>\n\
+          Date: Sat, 07 Dec 2019 19:00:27 +0000\n\
+          \n\
+          Happy birthday, Bob!\n",
+        true,
+    )
+    .await?
+    .unwrap();
+
+    // The "Happy birthday" message should be shown first, and then the "Thanks" message
+    assert!(msg_sent.sort_timestamp < msg_incoming.sort_timestamp);
+
+    Ok(())
+}
+
+#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
+async fn test_mdn_doesnt_disable_verification() -> Result<()> {
+    let mut tcm = TestContextManager::new();
+    let alice = tcm.alice().await;
+    let bob = tcm.bob().await;
+    enable_verified_oneonone_chats(&[&alice, &bob]).await;
+    bob.set_config_bool(Config::MdnsEnabled, true).await?;
+
+    // Alice & Bob verify each other
+    mark_as_verified(&alice, &bob).await;
+    mark_as_verified(&bob, &alice).await;
+
+    let rcvd = tcm.send_recv_accept(&alice, &bob, "Heyho").await;
+    message::markseen_msgs(&bob, vec![rcvd.id]).await?;
+
+    let mimefactory = MimeFactory::from_mdn(&bob, &rcvd, vec![]).await?;
+    let rendered_msg = mimefactory.render(&bob).await?;
+    let body = rendered_msg.message;
+    receive_imf(&alice, body.as_bytes(), false).await.unwrap();
+
+    assert_verified(&alice, &bob, ProtectionStatus::Protected).await;
+
+    Ok(())
+}
+
+#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
+async fn test_outgoing_mua_msg() -> Result<()> {
+    let mut tcm = TestContextManager::new();
+    let alice = tcm.alice().await;
+    let bob = tcm.bob().await;
+    enable_verified_oneonone_chats(&[&alice, &bob]).await;
+
+    mark_as_verified(&alice, &bob).await;
+    mark_as_verified(&bob, &alice).await;
+
+    tcm.send_recv_accept(&bob, &alice, "Heyho from DC").await;
+    assert_verified(&alice, &bob, ProtectionStatus::Protected).await;
+
+    let sent = receive_imf(
+        &alice,
+        b"From: alice@example.org\n\
+          To: bob@example.net\n\
+          \n\
+          One classical MUA message",
+        false,
+    )
+    .await?
+    .unwrap();
+    tcm.send_recv_accept(&alice, &bob, "Sending with DC again")
+        .await;
+
+    alice
+        .golden_test_chat(sent.chat_id, "test_outgoing_mua_msg")
+        .await;
+
+    Ok(())
+}
+
+/// If Bob answers unencrypted from another address with a classical MUA,
+/// the message is under some circumstances still assigned to the original
+/// chat (see lookup_chat_by_reply()); this is meant to make aliases
+/// work nicely.
+/// However, if the original chat is verified, the unencrypted message
+/// must NOT be assigned to it (it would be replaced by an error
+/// message in the verified chat, so, this would just be a usability issue,
+/// not a security issue).
+#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
+async fn test_reply() -> Result<()> {
+    for verified in [false, true] {
+        let mut tcm = TestContextManager::new();
+        let alice = tcm.alice().await;
+        let bob = tcm.bob().await;
+        enable_verified_oneonone_chats(&[&alice, &bob]).await;
+
+        if verified {
+            mark_as_verified(&alice, &bob).await;
+            mark_as_verified(&bob, &alice).await;
+        }
+
+        tcm.send_recv_accept(&bob, &alice, "Heyho from DC").await;
+        let encrypted_msg = tcm.send_recv_accept(&alice, &bob, "Heyho back").await;
+
+        let unencrypted_msg = receive_imf(
+            &alice,
+            format!(
+                "From: bob@someotherdomain.org\n\
+                 To: some-alias-forwarding-to-alice@example.org\n\
+                 In-Reply-To: {}\n\
+                 \n\
+                 Weird reply",
+                encrypted_msg.rfc724_mid
+            )
+            .as_bytes(),
+            false,
+        )
+        .await?
+        .unwrap();
+
+        let unencrypted_msg = Message::load_from_db(&alice, unencrypted_msg.msg_ids[0]).await?;
+        assert_eq!(unencrypted_msg.text, "Weird reply");
+
+        if verified {
+            assert_ne!(unencrypted_msg.chat_id, encrypted_msg.chat_id);
+        } else {
+            assert_eq!(unencrypted_msg.chat_id, encrypted_msg.chat_id);
+        }
+    }
+
+    Ok(())
+}
+
+/// Regression test for the following bug:
+///
+/// - Scan your chat partner's QR Code
+/// - They change devices
+/// - They send you a message
+/// - Without accepting the encryption downgrade, scan your chat partner's QR Code again
+///
+/// -> The re-verification fails.
+#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
+async fn test_break_protection_then_verify_again() -> Result<()> {
+    let mut tcm = TestContextManager::new();
+    let alice = tcm.alice().await;
+    let bob = tcm.bob().await;
+    enable_verified_oneonone_chats(&[&alice, &bob]).await;
+
+    // Cave: Bob can't write a message to Alice here.
+    // If he did, alice would increase his peerstate's last_seen timestamp.
+    // Then, after Bob reinstalls DC, alice's `if message_time > last_seen*`
+    // checks would return false (there are many checks of this form in peerstate.rs).
+    // Therefore, during the securejoin, Alice wouldn't accept the new key
+    // and reject the securejoin.
+
+    mark_as_verified(&alice, &bob).await;
+    mark_as_verified(&bob, &alice).await;
+
+    alice.create_chat(&bob).await;
+    assert_verified(&alice, &bob, ProtectionStatus::Protected).await;
+
+    tcm.section("Bob reinstalls DC");
+    drop(bob);
+    let bob_new = tcm.unconfigured().await;
+    enable_verified_oneonone_chats(&[&bob_new]).await;
+    bob_new.configure_addr("bob@example.net").await;
+    e2ee::ensure_secret_key_exists(&bob_new).await?;
+
+    tcm.send_recv(&bob_new, &alice, "I have a new device").await;
+    assert_verified(&alice, &bob_new, ProtectionStatus::ProtectionBroken).await;
+
+    {
+        let alice_bob_chat = alice.get_chat(&bob_new).await.unwrap();
+        assert!(!alice_bob_chat.can_send(&alice).await?);
+
+        // Alice's UI should still be able to save a draft, which Alice started to type right when she got Bob's message:
+        let mut msg = Message::new(Viewtype::Text);
+        msg.set_text("Draftttt".to_string());
+        alice_bob_chat.id.set_draft(&alice, Some(&mut msg)).await?;
+        assert_eq!(
+            alice_bob_chat.id.get_draft(&alice).await?.unwrap().text,
+            "Draftttt"
+        );
+    }
+
+    tcm.execute_securejoin(&alice, &bob_new).await;
+    assert_verified(&alice, &bob_new, ProtectionStatus::Protected).await;
+
+    Ok(())
+}
+
+/// Regression test:
+/// - Verify a contact
+/// - The contact stops using DC and sends a message from a classical MUA instead
+/// - Delete the 1:1 chat
+/// - Create a 1:1 chat
+/// - Check that the created chat is not marked as protected
+#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
+async fn test_create_oneonone_chat_with_former_verified_contact() -> Result<()> {
+    let mut tcm = TestContextManager::new();
+    let alice = tcm.alice().await;
+    let bob = tcm.bob().await;
+    enable_verified_oneonone_chats(&[&alice]).await;
+
+    mark_as_verified(&alice, &bob).await;
+
+    receive_imf(
+        &alice,
+        b"Subject: Message from bob\r\n\
+          From: <bob@example.net>\r\n\
+          To: <alice@example.org>\r\n\
+          Date: Mon, 12 Dec 2022 14:33:39 +0000\r\n\
+          Message-ID: <abcd@example.net>\r\n\
+          \r\n\
+          Heyho!\r\n",
+        false,
+    )
+    .await
+    .unwrap()
+    .unwrap();
+
+    alice.create_chat(&bob).await;
+
+    assert_verified(&alice, &bob, ProtectionStatus::Unprotected).await;
+
+    Ok(())
+}
+
+// ============== Helper Functions ==============
+
+async fn assert_verified(this: &TestContext, other: &TestContext, protected: ProtectionStatus) {
+    let contact = this.add_or_lookup_contact(other).await;
+    assert_eq!(
+        contact.is_verified(this).await.unwrap(),
+        VerifiedStatus::BidirectVerified
+    );
+
+    let chat = this.get_chat(other).await.unwrap();
+    let (expect_protected, expect_broken) = match protected {
+        ProtectionStatus::Unprotected => (false, false),
+        ProtectionStatus::Protected => (true, false),
+        ProtectionStatus::ProtectionBroken => (false, true),
+    };
+    assert_eq!(chat.is_protected(), expect_protected);
+    assert_eq!(chat.is_protection_broken(), expect_broken);
+}
+
+async fn enable_verified_oneonone_chats(test_contexts: &[&TestContext]) {
+    for t in test_contexts {
+        t.set_config_bool(Config::VerifiedOneOnOneChats, true)
+            .await
+            .unwrap()
+    }
+}

src/tools.rs

@@ -323,16 +323,17 @@ pub fn get_filemeta(buf: &[u8]) -> Result<(u32, u32)> {
 ///
 /// If `path` starts with "$BLOBDIR", replaces it with the blobdir path.
 /// Otherwise, returns path as is.
-pub(crate) fn get_abs_path(context: &Context, path: &Path) -> PathBuf {
-    if let Ok(p) = path.strip_prefix("$BLOBDIR") {
+pub(crate) fn get_abs_path(context: &Context, path: impl AsRef<Path>) -> PathBuf {
+    let p: &Path = path.as_ref();
+    if let Ok(p) = p.strip_prefix("$BLOBDIR") {
         context.get_blobdir().join(p)
     } else {
-        path.into()
+        p.into()
     }
 }
 
 pub(crate) async fn get_filebytes(context: &Context, path: impl AsRef<Path>) -> Result<u64> {
-    let path_abs = get_abs_path(context, path.as_ref());
+    let path_abs = get_abs_path(context, &path);
     let meta = fs::metadata(&path_abs).await?;
     Ok(meta.len())
 }
@@ -376,7 +377,7 @@ pub(crate) async fn create_folder(
     context: &Context,
     path: impl AsRef<Path>,
 ) -> Result<(), io::Error> {
-    let path_abs = get_abs_path(context, path.as_ref());
+    let path_abs = get_abs_path(context, &path);
     if !path_abs.exists() {
         match fs::create_dir_all(path_abs).await {
             Ok(_) => Ok(()),
@@ -401,7 +402,7 @@ pub(crate) async fn write_file(
     path: impl AsRef<Path>,
     buf: &[u8],
 ) -> Result<(), io::Error> {
-    let path_abs = get_abs_path(context, path.as_ref());
+    let path_abs = get_abs_path(context, &path);
     fs::write(&path_abs, buf).await.map_err(|err| {
         warn!(
             context,
@@ -416,7 +417,7 @@ pub(crate) async fn write_file(
 
 /// Reads the file and returns its context as a byte vector.
 pub async fn read_file(context: &Context, path: impl AsRef<Path>) -> Result<Vec<u8>> {
-    let path_abs = get_abs_path(context, path.as_ref());
+    let path_abs = get_abs_path(context, &path);
 
     match fs::read(&path_abs).await {
         Ok(bytes) => Ok(bytes),
@@ -433,7 +434,7 @@ pub async fn read_file(context: &Context, path: impl AsRef<Path>) -> Result<Vec<
 }
 
 pub async fn open_file(context: &Context, path: impl AsRef<Path>) -> Result<fs::File> {
-    let path_abs = get_abs_path(context, path.as_ref());
+    let path_abs = get_abs_path(context, &path);
 
     match fs::File::open(&path_abs).await {
         Ok(bytes) => Ok(bytes),
@@ -449,8 +450,12 @@ pub async fn open_file(context: &Context, path: impl AsRef<Path>) -> Result<fs::
     }
 }
 
-pub fn open_file_std(context: &Context, path: impl AsRef<Path>) -> Result<std::fs::File> {
-    let path_abs = get_abs_path(context, path.as_ref());
+pub fn open_file_std<P: AsRef<std::path::Path>>(
+    context: &Context,
+    path: P,
+) -> Result<std::fs::File> {
+    let p: PathBuf = path.as_ref().into();
+    let path_abs = get_abs_path(context, p);
 
     match std::fs::File::open(path_abs) {
         Ok(bytes) => Ok(bytes),
@@ -535,9 +540,6 @@ impl EmailAddress {
                 if domain.is_empty() {
                     bail!("missing domain after '@' in {:?}", input);
                 }
-                if domain.ends_with('.') {
-                    bail!("Domain {domain:?} should not contain the dot in the end");
-                }
                 Ok(EmailAddress {
                     local: (*local).to_string(),
                     domain: (*domain).to_string(),
@@ -700,7 +702,7 @@ pub(crate) fn buf_decompress(buf: &[u8]) -> Result<Vec<u8>> {
 }
 
 const RTLO_CHARACTERS: [char; 5] = ['\u{202A}', '\u{202B}', '\u{202C}', '\u{202D}', '\u{202E}'];
-/// This method strips all occurances of the RTLO Unicode character.
+/// This method strips all occurrences of the RTLO Unicode character.
 /// [Why is this needed](https://github.com/deltachat/deltachat-core-rust/issues/3479)?
 pub(crate) fn strip_rtlo_characters(input_str: &str) -> String {
     input_str.replace(|char| RTLO_CHARACTERS.contains(&char), "")
@@ -999,7 +1001,7 @@ DKIM Results: Passed=true, Works=true, Allow_Keychange=true";
         assert_eq!(EmailAddress::new("dd.tt").is_ok(), false);
         assert!(EmailAddress::new("tt.dd@uu").is_ok());
         assert!(EmailAddress::new("u@d").is_ok());
-        assert!(EmailAddress::new("u@d.").is_err());
+        assert!(EmailAddress::new("u@d.").is_ok());
         assert!(EmailAddress::new("u@d.t").is_ok());
         assert_eq!(
             EmailAddress::new("u@d.tt").unwrap(),

src/webxdc.rs

@@ -1,18 +1,4 @@
 //! # Handle webxdc messages.
-//!
-//! Internally status updates are stored in the `msgs_status_updates` SQL table.
-//! `msgs_status_updates` contains the following columns:
-//! - `id` - status update serial number
-//! - `msg_id` - ID of the message in the `msgs` table
-//! - `update_item` - JSON representation of the status update
-//!
-//! Status updates are scheduled for sending by adding a record
-//! to `smtp_status_updates_table` SQL table.
-//! `smtp_status_updates` contains the following columns:
-//! - `msg_id` - ID of the message in the `msgs` table
-//! - `first_serial` - serial number of the first status update to send
-//! - `last_serial` - serial number of the last status update to send
-//! - `descr` - text to send along with the updates
 
 use std::convert::TryFrom;
 use std::path::Path;
@@ -26,14 +12,12 @@ use serde::{Deserialize, Serialize};
 use serde_json::Value;
 use tokio::io::AsyncReadExt;
 
-use crate::blob::BlobObject;
 use crate::chat::Chat;
 use crate::constants::Chattype;
 use crate::contact::ContactId;
 use crate::context::Context;
 use crate::download::DownloadState;
 use crate::message::{Message, MessageState, MsgId, Viewtype};
-use crate::mimefactory::wrapped_base64_encode;
 use crate::mimeparser::SystemMessage;
 use crate::param::Param;
 use crate::param::Params;
@@ -551,16 +535,13 @@ impl Context {
     }
 
     pub(crate) fn build_status_update_part(&self, json: &str) -> PartBuilder {
-        let encoded_body = wrapped_base64_encode(json.as_bytes());
-
         PartBuilder::new()
             .content_type(&"application/json".parse::<mime::Mime>().unwrap())
             .header((
                 "Content-Disposition",
                 "attachment; filename=\"status-update.json\"",
             ))
-            .header(("Content-Transfer-Encoding", "base64"))
-            .body(encoded_body)
+            .body(json)
     }
 
     /// Receives status updates from receive_imf to the database
@@ -580,6 +561,7 @@ impl Context {
         json: &str,
     ) -> Result<()> {
         let msg = Message::load_from_db(self, msg_id).await?;
+        let chat_id = msg.chat_id;
         let (timestamp, mut instance, can_info_msg) = if msg.viewtype == Viewtype::Webxdc {
             (msg.timestamp_sort, msg, false)
         } else if let Some(parent) = msg.parent(self).await? {
@@ -593,16 +575,17 @@ impl Context {
         } else {
             bail!("receive_status_update: status message has no parent.")
         };
-        let chat_id = instance.chat_id;
 
-        if from_id != ContactId::SELF && !chat::is_contact_in_chat(self, chat_id, from_id).await? {
+        if from_id != ContactId::SELF
+            && !chat::is_contact_in_chat(self, instance.chat_id, from_id).await?
+        {
             let chat_type: Chattype = self
                 .sql
                 .query_get_value("SELECT type FROM chats WHERE id=?", (chat_id,))
                 .await?
                 .with_context(|| format!("Chat type for chat {chat_id} not found"))?;
             if chat_type != Chattype::Mailinglist {
-                bail!("receive_status_update: status sender {from_id} is not a member of chat {chat_id}")
+                bail!("receive_status_update: status sender not chat member.")
             }
         }
 
@@ -680,10 +663,6 @@ impl Context {
     ///
     /// Example: `{"updates": [{"payload":"any update data"},
     ///                        {"payload":"another update data"}]}`
-    ///
-    /// `range` is an optional range of status update serials to send.
-    /// If it is `None`, all updates are sent.
-    /// This is used when a message is resent using [`crate::chat::resend_msgs`].
     pub(crate) async fn render_webxdc_status_update_object(
         &self,
         instance_msg_id: MsgId,
@@ -726,7 +705,7 @@ fn parse_webxdc_manifest(bytes: &[u8]) -> Result<WebxdcManifest> {
     Ok(manifest)
 }
 
-async fn get_blob(archive: &async_zip::read::fs::ZipFileReader, name: &str) -> Result<Vec<u8>> {
+async fn get_blob(archive: &mut async_zip::read::fs::ZipFileReader, name: &str) -> Result<Vec<u8>> {
     let (i, _) = find_zip_entry(archive.file(), name)
         .ok_or_else(|| anyhow!("no entry found for {}", name))?;
     let mut reader = archive.entry(i).await?;
@@ -769,10 +748,10 @@ impl Message {
             name
         };
 
-        let archive = self.get_webxdc_archive(context).await?;
+        let mut archive = self.get_webxdc_archive(context).await?;
 
         if name == "index.html" {
-            if let Ok(bytes) = get_blob(&archive, "manifest.toml").await {
+            if let Ok(bytes) = get_blob(&mut archive, "manifest.toml").await {
                 if let Ok(manifest) = parse_webxdc_manifest(&bytes) {
                     if let Some(min_api) = manifest.min_api {
                         if min_api > WEBXDC_API_VERSION {
@@ -785,15 +764,15 @@ impl Message {
             }
         }
 
-        get_blob(&archive, name).await
+        get_blob(&mut archive, name).await
     }
 
     /// Return info from manifest.toml or from fallbacks.
     pub async fn get_webxdc_info(&self, context: &Context) -> Result<WebxdcInfo> {
         ensure!(self.viewtype == Viewtype::Webxdc, "No webxdc instance.");
-        let archive = self.get_webxdc_archive(context).await?;
+        let mut archive = self.get_webxdc_archive(context).await?;
 
-        let mut manifest = get_blob(&archive, "manifest.toml")
+        let mut manifest = get_blob(&mut archive, "manifest.toml")
             .await
             .map(|bytes| parse_webxdc_manifest(&bytes).unwrap_or_default())
             .unwrap_or_default();
@@ -846,35 +825,6 @@ impl Message {
     }
 }
 
-/// Replaces WebXDC blob of existing message.
-///
-/// This API is supposed to be called from within a WebXDC to replace itself
-/// e.g. with an updated or persistently reconfigured version.
-pub async fn replace_webxdc(context: &Context, msg_id: MsgId, data: &[u8]) -> Result<()> {
-    let mut msg = Message::load_from_db(context, msg_id).await?;
-
-    ensure!(
-        msg.get_viewtype() == Viewtype::Webxdc,
-        "Message {msg_id} is not a WebXDC instance"
-    );
-
-    let blob = BlobObject::create(
-        context,
-        &msg.get_filename()
-            .context("Cannot get filename of exising WebXDC instance")?,
-        data,
-    )
-    .await
-    .context("Failed to create WebXDC replacement blob")?;
-
-    let mut param = msg.param.clone();
-    param.set(Param::File, blob.as_name());
-    msg.param = param;
-    msg.update_param(context).await?;
-
-    Ok(())
-}
-
 #[cfg(test)]
 mod tests {
     use serde_json::json;
@@ -943,8 +893,10 @@ mod tests {
     }
 
     async fn create_webxdc_instance(t: &TestContext, name: &str, bytes: &[u8]) -> Result<Message> {
+        let file = t.get_blobdir().join(name);
+        tokio::fs::write(&file, bytes).await?;
         let mut instance = Message::new(Viewtype::File);
-        instance.set_file_from_bytes(t, name, bytes, None).await?;
+        instance.set_file(file.to_str().unwrap(), None);
         Ok(instance)
     }
 
@@ -972,10 +924,10 @@ mod tests {
         assert_eq!(instance.chat_id, chat_id);
 
         // sending using bad extension is not working, even when setting Viewtype to webxdc
+        let file = t.get_blobdir().join("index.html");
+        tokio::fs::write(&file, b"<html>ola!</html>").await?;
         let mut instance = Message::new(Viewtype::Webxdc);
-        instance
-            .set_file_from_bytes(&t, "index.html", b"<html>ola!</html>", None)
-            .await?;
+        instance.set_file(file.to_str().unwrap(), None);
         assert!(send_msg(&t, chat_id, &mut instance).await.is_err());
 
         Ok(())
@@ -999,15 +951,14 @@ mod tests {
         assert_eq!(test.viewtype, Viewtype::File);
 
         // sending invalid .xdc as Viewtype::Webxdc should fail already on sending
+        let file = t.get_blobdir().join("invalid2.xdc");
+        tokio::fs::write(
+            &file,
+            include_bytes!("../test-data/webxdc/invalid-no-zip-but-7z.xdc"),
+        )
+        .await?;
         let mut instance = Message::new(Viewtype::Webxdc);
-        instance
-            .set_file_from_bytes(
-                &t,
-                "invalid2.xdc",
-                include_bytes!("../test-data/webxdc/invalid-no-zip-but-7z.xdc"),
-                None,
-            )
-            .await?;
+        instance.set_file(file.to_str().unwrap(), None);
         assert!(send_msg(&t, chat_id, &mut instance).await.is_err());
 
         Ok(())
@@ -1135,7 +1086,7 @@ mod tests {
         .await?;
         let instance = t.get_last_msg().await;
         assert_eq!(instance.viewtype, Viewtype::Webxdc);
-        assert_eq!(instance.get_filename().unwrap(), "minimal.xdc");
+        assert_eq!(instance.get_filename(), Some("minimal.xdc".to_string()));
 
         receive_imf(
             &t,
@@ -1145,7 +1096,7 @@ mod tests {
         .await?;
         let instance = t.get_last_msg().await;
         assert_eq!(instance.viewtype, Viewtype::File); // we require the correct extension, only a mime type is not sufficient
-        assert_eq!(instance.get_filename().unwrap(), "index.html");
+        assert_eq!(instance.get_filename(), Some("index.html".to_string()));
 
         Ok(())
     }
@@ -1833,9 +1784,10 @@ mod tests {
         // bob receives the instance together with the initial updates in a single message
         let bob_instance = bob.recv_msg(&sent1).await;
         assert_eq!(bob_instance.viewtype, Viewtype::Webxdc);
-        assert_eq!(bob_instance.get_filename().unwrap(), "minimal.xdc");
+        assert_eq!(bob_instance.get_filename(), Some("minimal.xdc".to_string()));
         assert!(sent1.payload().contains("Content-Type: application/json"));
         assert!(sent1.payload().contains("status-update.json"));
+        assert!(sent1.payload().contains(r#""payload":{"foo":"bar"}"#));
         assert_eq!(
             bob.get_webxdc_status_updates(bob_instance.id, StatusUpdateSerial(0))
                 .await?,
@@ -2615,100 +2567,4 @@ sth_for_the = "future""#
         );
         Ok(())
     }
-
-    /// Tests extensibility of WebXDC updates.
-    ///
-    /// If an update sent by WebXDC contains unknown properties,
-    /// such as `aNewUnknownProperty` or a reserved property
-    /// like `serial` or `max_serial`,
-    /// they are silently dropped and are not sent over the wire.
-    ///
-    /// This ensures new WebXDC can try to send new properties
-    /// added in later revisions of the WebXDC API
-    /// and this will not result in a failure to send the whole update.
-    #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
-    async fn test_send_webxdc_status_update_extensibility() -> Result<()> {
-        let alice = TestContext::new_alice().await;
-        let bob = TestContext::new_bob().await;
-        let alice_chat = alice.create_chat(&bob).await;
-        let alice_instance = send_webxdc_instance(&alice, alice_chat.id).await?;
-
-        let bob_instance = bob.recv_msg(&alice.pop_sent_msg().await).await;
-
-        alice
-            .send_webxdc_status_update(
-                alice_instance.id,
-                r#"{"payload":"p","info":"i","aNewUnknownProperty":"x","max_serial":123}"#,
-                "Some description",
-            )
-            .await?;
-        alice.flush_status_updates().await?;
-        bob.recv_msg(&alice.pop_sent_msg().await).await;
-
-        assert_eq!(
-            bob.get_webxdc_status_updates(bob_instance.id, StatusUpdateSerial(0))
-                .await?,
-            r#"[{"payload":"p","info":"i","serial":1,"max_serial":1}]"#
-        );
-
-        Ok(())
-    }
-
-    /// Tests replacing WebXDC with a newer version.
-    ///
-    /// Updates should be preserved after upgrading.
-    #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
-    async fn test_replace_webxdc() -> Result<()> {
-        let alice = TestContext::new_alice().await;
-        let bob = TestContext::new_bob().await;
-
-        // Alice sends WebXDC instance.
-        let alice_chat = alice.create_chat(&bob).await;
-        let mut alice_instance = create_webxdc_instance(
-            &alice,
-            "minimal.xdc",
-            include_bytes!("../test-data/webxdc/minimal.xdc"),
-        )
-        .await?;
-        alice_instance.set_text("user added text".to_string());
-        send_msg(&alice, alice_chat.id, &mut alice_instance).await?;
-        let alice_instance = alice.get_last_msg().await;
-        assert_eq!(alice_instance.get_text(), "user added text");
-
-        // Bob receives that instance.
-        let alice_sent_instance = alice.pop_sent_msg().await;
-        let bob_received_instance = bob.recv_msg(&alice_sent_instance).await;
-        assert_eq!(bob_received_instance.get_text(), "user added text");
-
-        // Alice sends WebXDC update.
-        alice
-            .send_webxdc_status_update(alice_instance.id, r#"{"payload": 1}"#, "Alice update")
-            .await?;
-        alice.flush_status_updates().await?;
-        let alice_sent_update = alice.pop_sent_msg().await;
-        bob.recv_msg(&alice_sent_update).await;
-        assert_eq!(
-            bob.get_webxdc_status_updates(bob_received_instance.id, StatusUpdateSerial(0))
-                .await?,
-            r#"[{"payload":1,"serial":1,"max_serial":1}]"#
-        );
-
-        // Bob replaces WebXDC.
-        replace_webxdc(
-            &bob,
-            bob_received_instance.id,
-            include_bytes!("../test-data/webxdc/with-minimal-manifest.xdc"),
-        )
-        .await
-        .context("Failed to replace WebXDC")?;
-
-        // Updates are not modified.
-        assert_eq!(
-            bob.get_webxdc_status_updates(bob_received_instance.id, StatusUpdateSerial(0))
-                .await?,
-            r#"[{"payload":1,"serial":1,"max_serial":1}]"#
-        );
-
-        Ok(())
-    }
 }

test-data/golden/test_old_message_1

@@ -0,0 +1,6 @@
+Single#Chat#10: Bob [bob@example.net] 
+--------------------------------------------------------------------------------
+Msg#10: info (Contact#Contact#Info): Messages are guaranteed to be end-to-end encrypted from now on. [NOTICED][INFO 🛡️]
+Msg#11: info (Contact#Contact#Info): Bob sent a message from another device. [NOTICED][INFO 🛡️❌]
+Msg#12:  (Contact#Contact#10): Message from Thunderbird [SEEN]
+--------------------------------------------------------------------------------

test-data/golden/test_old_message_2

@@ -0,0 +1,7 @@
+Single#Chat#10: Bob [bob@example.net] 
+--------------------------------------------------------------------------------
+Msg#10: info (Contact#Contact#Info): Messages are guaranteed to be end-to-end encrypted from now on. [NOTICED][INFO 🛡️]
+Msg#11: info (Contact#Contact#Info): Bob sent a message from another device. [NOTICED][INFO 🛡️❌]
+Msg#12:  (Contact#Contact#10): Somewhat old message [FRESH]
+Msg#13:  (Contact#Contact#10): Even older message, that must NOT be shown before the info message [SEEN]
+--------------------------------------------------------------------------------

test-data/golden/test_old_message_3

@@ -0,0 +1,7 @@
+Single#Chat#10: Bob [bob@example.net] 
+--------------------------------------------------------------------------------
+Msg#10: info (Contact#Contact#Info): Messages are guaranteed to be end-to-end encrypted from now on. [NOTICED][INFO 🛡️]
+Msg#11🔒:  (Contact#Contact#10): Heyho from my verified device! [FRESH]
+Msg#12: info (Contact#Contact#Info): Bob sent a message from another device. [NOTICED][INFO 🛡️❌]
+Msg#13:  (Contact#Contact#10): Old, unverified message [SEEN]
+--------------------------------------------------------------------------------

test-data/golden/test_outgoing_mua_msg

@@ -0,0 +1,7 @@
+Single#Chat#10: bob@example.net [bob@example.net] 🛡️
+--------------------------------------------------------------------------------
+Msg#10: info (Contact#Contact#Info): Messages are guaranteed to be end-to-end encrypted from now on. [NOTICED][INFO 🛡️]
+Msg#11🔒:  (Contact#Contact#10): Heyho from DC [FRESH]
+Msg#12: Me (Contact#Contact#Self): One classical MUA message  √
+Msg#13🔒: Me (Contact#Contact#Self): Sending with DC again  √
+--------------------------------------------------------------------------------

test-data/golden/test_verified_oneonone_chat_enable_disable

@@ -0,0 +1,12 @@
+Single#Chat#10: Bob [bob@example.net] 🛡️
+--------------------------------------------------------------------------------
+Msg#10: info (Contact#Contact#Info): Messages are guaranteed to be end-to-end encrypted from now on. [NOTICED][INFO 🛡️]
+Msg#11: info (Contact#Contact#Info): Bob sent a message from another device. [NOTICED][INFO 🛡️❌]
+Msg#12:  (Contact#Contact#10): Message from Thunderbird [FRESH]
+Msg#13: info (Contact#Contact#Info): Messages are guaranteed to be end-to-end encrypted from now on. [NOTICED][INFO 🛡️]
+Msg#14🔒:  (Contact#Contact#10): Hello from DC [FRESH]
+Msg#15: info (Contact#Contact#Info): Bob sent a message from another device. [NOTICED][INFO 🛡️❌]
+Msg#16:  (Contact#Contact#10): Message from Thunderbird [FRESH]
+Msg#17: info (Contact#Contact#Info): Messages are guaranteed to be end-to-end encrypted from now on. [NOTICED][INFO 🛡️]
+Msg#18🔒:  (Contact#Contact#10): Hello from DC [FRESH]
+--------------------------------------------------------------------------------

test-data/message/jpeg-as-application-octet-stream.eml

@@ -1,76 +0,0 @@
-X-Mozilla-Status: 0801
-X-Mozilla-Status2: 10000000
-Content-Type: multipart/mixed; boundary="------------L1v4sF5IlAZ0HirXymXElgpK"
-Message-ID: <1e3b3bb0-f34f-71e2-6b86-bce80bef2c6f@example.org>
-Date: Thu, 3 Aug 2023 13:31:01 -0300
-MIME-Version: 1.0
-User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
- Thunderbird/102.13.0
-Content-Language: en-US
-To: bob@example.net
-From: Alice <alice@example.org>
-X-Identity-Key: id3
-Fcc: imap://alice%40example.org@in.example.org/Sent
-
-This is a multi-part message in MIME format.
---------------L1v4sF5IlAZ0HirXymXElgpK
-Content-Type: text/plain; charset=UTF-8; format=flowed
-Content-Transfer-Encoding: 7bit
-
-
---------------L1v4sF5IlAZ0HirXymXElgpK
-Content-Type: application/octet-stream; name="rusty_deltachat_logo.jpeg"
-Content-Disposition: attachment; filename="rusty_deltachat_logo.jpeg"
-Content-Transfer-Encoding: base64
-
-/9j/4AAQSkZJRgABAQIAzADMAAD/2wBDAP//////////////////////////////////////
-////////////////////////////////////////////////2wBDAf//////////////////
-////////////////////////////////////////////////////////////////////wAAR
-CAEAAQADASIAAhEBAxEB/8QAHwAAAQUBAQEBAQEAAAAAAAAAAAECAwQFBgcICQoL/8QAtRAA
-AgEDAwIEAwUFBAQAAAF9AQIDAAQRBRIhMUEGE1FhByJxFDKBkaEII0KxwRVS0fAkM2JyggkK
-FhcYGRolJicoKSo0NTY3ODk6Q0RFRkdISUpTVFVWV1hZWmNkZWZnaGlqc3R1dnd4eXqDhIWG
-h4iJipKTlJWWl5iZmqKjpKWmp6ipqrKztLW2t7i5usLDxMXGx8jJytLT1NXW19jZ2uHi4+Tl
-5ufo6erx8vP09fb3+Pn6/8QAHwEAAwEBAQEBAQEBAQAAAAAAAAECAwQFBgcICQoL/8QAtREA
-AgECBAQDBAcFBAQAAQJ3AAECAxEEBSExBhJBUQdhcRMiMoEIFEKRobHBCSMzUvAVYnLRChYk
-NOEl8RcYGRomJygpKjU2Nzg5OkNERUZHSElKU1RVVldYWVpjZGVmZ2hpanN0dXZ3eHl6goOE
-hYaHiImKkpOUlZaXmJmaoqOkpaanqKmqsrO0tba3uLm6wsPExcbHyMnK0tPU1dbX2Nna4uPk
-5ebn6Onq8vP09fb3+Pn6/9oADAMBAAIRAxEAPwCSiiigAooooAKKKKACiiigAooprHAzQA6i
-mhgfY0McA0AAYGnVX6U/efagA3Hdnt6VL15qvS5OMZ4oAUt82R26f596lByM1BSgkcA0APZ+
-eO3X3qSq9PV8DBoAl6UVCzZ+lPQ8fSgB9FFICD0oAWiiigAooooAKKKKACiiigAooooAKKKK
-ACiiigApCcDNLTH6UAKGB9j6UjkYx3qKigAooooAKKKKACiiigAooooAKKKKAClBI6UlFACk
-k9akTofrUVKCR0oAnoqEuT7fSnp0P1oAfRRRQAUUUUAFFFFABRRRQAUUUUAFIwyDTd496QuM
-cUAMBI6GkJJ60UUAFFFFABRSqMmpgAOlAEYQnrxTtg96fSEgdTQAmxfT+dLtX0FJvX1o3r6/
-zoAXavoKTYvp/OlBB6GloAZsHvTSh7HNS0UAQEEdRSVYqNk7j8qAI6KKKACnq2ODTKKAJS47
-CnDkA1AOtWKACiiigAooooAKKKKACimscDOM03zPb9f/AK1ADXGD9eabSk5OTSUAFFFFABRR
-RQBMowB7806kXoPpQeh+hoAjZ+w/OmUUUAFFFFABTw/r+dMooAsdaKhDEfT0qUEHkUALRRRQ
-BG69x+NR1O33T9KgoAKKKKACnBiBim0UASKSTyeKkpiYwfWn0AFFFFABRRTWbb2zQA4jPFV6
-eX46Y/GmUAFFFFABRRRQAUUUUASIe35VJVepVbPB6/zoAay45HT+VMqxTCgPTg0ARUUpBHWk
-oAKKKKAClBI6UlFAEocd+KXevr+hqGigBzNn6U2iigAooooAKKKKAAEjpTgWJxk02pExk+tA
-ElFFFABSEZGKCQOtNLjtzQBF0ooooAKKKKACiiigAooooAKKKKAJFfsfz/xqSq9OViPcUATY
-z1qMp6flTwQelLQBX6UVOQD1qMoR05/nQAyiiigAooooAKKKKACiiigAooooAKAcdKKKAHBm
-+tTUxAMZ70+gBCARioSCDzU+cdahY5PHagBtFFFABRRRQAUUUUAFFFFABRRRQAUUdelPCHvx
-QA0EjpUqsD9fSkCD3NOCgdqAFooooAaVB+vrURUj/Gp6KAK9FSMncfl/hUdABRRRQAUUUUAF
-FFFABRRRQA4MR0p6tnjHNRgZOKmCgUARsp69R/KmVYqA9Tj1oASiiigAooooAKKKKACiilAJ
-4FACVIE9fypwUD6+tOoAQADpS0UhIHJoAWkLAdTUZcnpwKZQBIXHYUB+eelR0UAWKKiRux/C
-paACmMueR1/nT6KAK9FSsueR1/nUVABRRRQAUUUUAFFFFACg4OalDA+31qGpFTufyoAazE8d
-B6U2pyAetQsMEigBKKKKACiiigAooooABzxU4GBTEHf8qkoAKKKKADpUDHJ/lT3Pb8TUdABR
-RRQAUUUUAFTKcj3FQ05Tg/pQBNRRRQAVE4wc9j/OpaQjIIoAgooooAKKKKACiiigBRwQanqv
-T03Z9vegB7Nj61DUxUGmMuKAGUUUUAFFFFABRRSr1H1oAmAwAKWiigAooooAgY5JpKKKACii
-igAooooAKKKKAJwcgGlpifdp9ABRRRQBCwwx/Om09+o+lMoAKKKKACiiigCRB1qSoAcHNTA5
-GaAFqJ2zwPxqWo9h9aAI6KUjBwaSgAooooAKcv3hTaUdR9aAJ6KKKACkPQ/Q0tFAFeiiigAo
-oooAKKKKACiiigCVOh+v9BT6an3adQAUUUUARP1H0plOc5Y/lTaACiiigApVGSBSUdOaAJ8D
-0FLSA5GaWgAoooPTjrQBE5yfpTKfsb2ppBBwaAEooooAKKKKAJ1OQDS1Ehwcev8AOpaACiii
-gCAjBP1pKe45z60ygAooooAKKKKACiinIMn2FAEoGABS0UUAFITgZpaidsnHYUAMooooAKKK
-KACpti+n6moalVs8Hr/OgBwGOBS0UUAFFFFABUb9hUlRlCSTkUAR0U8oQM9aZQAUUUUAFSq2
-eD1/nUVFAFiimK2eD1/nT6AEIyMVCQQcGp6QgHrQBBRTyh7c03afQ0AJRS7T6GnBD34oAaAS
-cCpgMDFAAHSloAKKKjZ+w/P/AAoAGbsPxqOiigAooooAKKKKAFUZOKlCgf41EDg5FSBwevFA
-D6KKKACiiigAooooAQnAJqCpyMjFIEUds/WgCGinuoHI/KmUAFFFFABTw5HXn+dMooAnBB6G
-lqvTgxHegCaiot59BS+Z7frQBJRUfme1JvPsKAJaaXA96iJJ6mkoAcWJ+npTaKKACiiigAoo
-ooAKKKKACpVXHJ61FUocYGetAD6KQEHoaWgAooooAKKKKACiiigBj9B9aaEJ68VLRQBAQR1p
-KlfoPrTApNADaKUgjrSUAFFFFABRRRQAUUUUAFFFFABRRSkEdRQAlFKBkgVKUGOOKAIaKXBz
-ipto9BQBBRQRg4p+w4z39KAGUUUUASouOe5/lT6QHIzS0AFFFFABRRRQAUUUUAFFFFACEA9a
-WiigBr/d+lRqufpU1FAEbJgZFR1YqNU557dKAG7W9KbVioip3YHegBoBPQUlTgYGKay55HX+
-dADFXdSshHPWnqMD3p1AESdfwqWkAA6d6WgBgQA5H5U+iigBMDOe9LRRQAm0ZzS0UUARFTu9
-jzmpNoxjFLRQAgGOlLRRQB//2Q==
-
---------------L1v4sF5IlAZ0HirXymXElgpK--