add integration test

webxdcs can now have multiple gossip channels and decide where to send the message to.

.github/workflows/ci.yml

@@ -82,9 +82,9 @@ jobs:
           - os: macos-latest
             rust: 1.75.0
 
-          # Minimum Supported Rust Version = 1.70.0
+          # Minimum Supported Rust Version = 1.72.0
           - os: ubuntu-latest
-            rust: 1.70.0
+            rust: 1.72.0
     runs-on: ${{ matrix.os }}
     steps:
       - uses: actions/checkout@v3

CHANGELOG.md

@@ -1,5 +1,83 @@
 # Changelog
 
+## [1.133.1] - 2024-01-21
+
+### API-Changes
+
+- Add `is_bot` to cffi and jsonrpc ([#5197](https://github.com/deltachat/deltachat-core-rust/pull/5197)).
+
+### Features / Changes
+
+- Add system message when provider does not allow unencrypted messages ([#5195](https://github.com/deltachat/deltachat-core-rust/pull/5195)).
+
+### Fixes
+
+- `Chat::send_msg`: Remove encryption-related params from already sent message. This allows to send received encrypted `dc_msg_t` object to unencrypted chat, e.g. in a Python bot.
+- Set message download state to Failure on IMAP errors. This avoids partially downloaded messages getting stuck in "Downloading..." state without actually being in a download queue.
+- BCC-to-self even if server deletion is set to "at once". This is a workaround for SMTP servers which do not return response in time, BCC-self works as a confirmation that message was sent out successfully and does not need more retries.
+- node: Run tests with native ESM modules instead of `esm` ([#5194](https://github.com/deltachat/deltachat-core-rust/pull/5194)).
+- Use Quoted-Printable MIME encoding for the text part ([#3986](https://github.com/deltachat/deltachat-core-rust/pull/3986)).
+
+### Tests
+
+- python: Add `get_protected_chat` to testplugin.py.
+
+## [1.133.0] - 2024-01-14
+
+### Features / Changes
+
+- Securejoin protocol implementation refinements
+  - Track forward and backward verification separately ([#5089](https://github.com/deltachat/deltachat-core-rust/pull/5089)) to avoid inconsistent states.
+  - Mark 1:1 chat as verified for Bob early. 1:1 chat with Alice is verified as soon as Alice's key is verified rather than at the end of the protocol.
+- Put Message-ID into hidden headers and take it from there on receiver ([#4798](https://github.com/deltachat/deltachat-core-rust/pull/4798)). This works around servers which generate their own Message-ID and overwrite the one generated by Delta Chat.
+- deltachat-repl: Enable INFO logging by default and add timestamps.
+- Add `ConfigSynced` (`DC_EVENT_CONFIG_SYNCED`) event which is emitted when configuration is changed via synchronization message or synchronization message for configuration is sent. UI may refresh elments based on the configuration key which is a part of the event.
+- Sync contact creation/rename across devices ([#5163](https://github.com/deltachat/deltachat-core-rust/pull/5163)).
+- Encrypt MDNs ([#5175](https://github.com/deltachat/deltachat-core-rust/pull/5175)).
+- Only try to configure non-strict TLS checks if explicitly set ([#5181](https://github.com/deltachat/deltachat-core-rust/pull/5181)).
+
+### Build system
+
+- Use released version of iroh 0.4.2 for "setup second device" feature.
+
+### CI
+
+- Update to Rust 1.75.0.
+- Downgrade `chai` from 4.4.0 to 4.3.10.
+
+### Documentation
+
+- Add a link <https://www.ietf.org/archive/id/draft-bucksch-autoconfig-00.html> to autoconfig RFC draft.
+- Update securejoin link in `standards.md` from <https://countermitm.readthedocs.io/> to <https://securejoin.readthedocs.io>.
+- Restore "Constants" page in Doxygen >=1.9.8
+
+### Fixes
+
+- imap: Limit the rate of LOGIN attempts rather than connection attempts. This is to avoid having to wait for rate limiter right after switching from a bad or offline network to a working network while still guarding against reconnection loop.
+- Do not ignore `peerstate.save_to_db()` errors.
+- securejoin: Mark 1:1s as protected regardless of the Config::VerifiedOneOnOneChats.
+- Delete received outgoing messages from SMTP queue ([#5115](https://github.com/deltachat/deltachat-core-rust/pull/5115)).
+- imap: Fail fast on `LIST` errors to avoid busy loop when connection is lost.
+- Split SMTP jobs already in `chat::create_send_msg_jobs()` ([#5115](https://github.com/deltachat/deltachat-core-rust/pull/5115)).
+- Do not remove contents from unencrypted [Schleuder](https://schleuder.org/) mailing lists messages.
+- Reset message error when scheduling resending ([#5119](https://github.com/deltachat/deltachat-core-rust/pull/5119)).
+- Emit events more reliably when starting and stopping I/O ([#5101](https://github.com/deltachat/deltachat-core-rust/pull/5101)).
+- Fix timestamp of chat protection info message for correct message ordering after restoring a backup ([#5088](https://github.com/deltachat/deltachat-core-rust/pull/5088)).
+
+### Refactor
+
+- sql: Recreate `config` table with UNIQUE constraint.
+- sql: Recreate `keypairs` table to remove unused `addr` and `created` fields and move `is_default` flag to `config` table.
+- Send `Secure-Join-Fingerprint` only in `*-request-with-auth`.
+
+### Tests
+
+- Test joining non-protected group.
+- Test that read receipts don't degrade encryption.
+- Test that changing default private key breaks backward verification.
+- Test recovery from lost vc-contact-confirm.
+- Use `wait_for_incoming_msg_event()` more.
+
 ## [1.132.1] - 2023-12-12
 
 ### Features / Changes
@@ -3357,3 +3435,5 @@ https://github.com/deltachat/deltachat-core-rust/pulls?q=is%3Apr+is%3Aclosed
 [1.131.9]: https://github.com/deltachat/deltachat-core-rust/compare/v1.131.8...v1.131.9
 [1.132.0]: https://github.com/deltachat/deltachat-core-rust/compare/v1.131.9...v1.132.0
 [1.132.1]: https://github.com/deltachat/deltachat-core-rust/compare/v1.132.0...v1.132.1
+[1.133.0]: https://github.com/deltachat/deltachat-core-rust/compare/v1.132.1...v1.133.0
+[1.133.1]: https://github.com/deltachat/deltachat-core-rust/compare/v1.133.0...v1.133.1

Cargo.toml

@@ -1,9 +1,9 @@
 [package]
 name = "deltachat"
-version = "1.132.1"
+version = "1.133.1"
 edition = "2021"
 license = "MPL-2.0"
-rust-version = "1.70"
+rust-version = "1.72"
 
 [profile.dev]
 debug = 0
@@ -32,6 +32,7 @@ strip = true
 
 [patch.crates-io]
 imap-proto = { git = "https://github.com/djc/tokio-imap.git", rev = "01ff256a7e42a9f7d2732706f8b71a16ce93427e" }
+"iroh-blake3" = { git = "https://github.com/n0-computer/iroh-blake3.git", branch = "master" }
 
 [dependencies]
 deltachat_derive = { path = "./deltachat_derive" }
@@ -40,14 +41,26 @@ ratelimit = { path = "./deltachat-ratelimit" }
 
 anyhow = "1"
 async-channel = "2.0.0"
-async-imap = { version = "0.9.5", default-features = false, features = ["runtime-tokio"] }
-async-native-tls = { version = "0.5", default-features = false, features = ["runtime-tokio"] }
-async-smtp = { version = "0.9", default-features = false, features = ["runtime-tokio"] }
-async_zip = { version = "0.0.12", default-features = false, features = ["deflate", "fs"] }
+async-imap = { version = "0.9.5", default-features = false, features = [
+  "runtime-tokio",
+] }
+async-native-tls = { version = "0.5", default-features = false, features = [
+  "runtime-tokio",
+] }
+async-smtp = { version = "0.9", default-features = false, features = [
+  "runtime-tokio",
+] }
+async_zip = { version = "0.0.12", default-features = false, features = [
+  "deflate",
+  "fs",
+] }
 backtrace = "0.3"
 base64 = "0.21"
-brotli = { version = "3.4", default-features=false, features = ["std"] }
-chrono = { version = "0.4", default-features=false, features = ["clock", "std"] }
+brotli = { version = "3.4", default-features = false, features = ["std"] }
+chrono = { version = "0.4", default-features = false, features = [
+  "clock",
+  "std",
+] }
 email = { git = "https://github.com/deltachat/rust-email", branch = "master" }
 encoded-words = { git = "https://github.com/async-email/encoded-words", branch = "master" }
 escaper = "0.1"
@@ -58,8 +71,22 @@ futures-lite = "2.0.0"
 hex = "0.4.0"
 hickory-resolver = "0.24"
 humansize = "2"
-image = { version = "0.24.7", default-features=false, features = ["gif", "jpeg", "ico", "png", "pnm", "webp", "bmp"] }
-iroh = { version = "0.4.2", default-features = false }
+image = { version = "0.24.7", default-features = false, features = [
+  "gif",
+  "jpeg",
+  "ico",
+  "png",
+  "pnm",
+  "webp",
+  "bmp",
+] }
+iroh = { git = "https://github.com/deltachat/iroh", branch = "0.4-update-quic", default-features = false }
+iroh-net = { git = "https://github.com/n0-computer/iroh", branch = "main" }
+iroh-base = { git = "https://github.com/n0-computer/iroh", branch = "main" }
+iroh-gossip = { git = "https://github.com/n0-computer/iroh", branch = "main", features = [
+  "net",
+] }
+quinn = "0.10"
 kamadak-exif = "0.5"
 lettre_email = { git = "https://github.com/deltachat/lettre", branch = "master" }
 libc = "0.2"
@@ -76,6 +103,7 @@ pin-project = "1"
 pretty_env_logger = { version = "0.5", optional = true }
 qrcodegen = "1.7.0"
 quick-xml = "0.31"
+quoted_printable = "0.5"
 rand = "0.8"
 regex = "1.9"
 reqwest = { version = "0.11.23", features = ["json"] }
@@ -92,7 +120,12 @@ strum_macros = "0.25"
 tagger = "4.3.4"
 textwrap = "0.16.0"
 thiserror = "1"
-tokio = { version = "1", features = ["fs", "rt-multi-thread", "macros"] }
+tokio = { version = "1", features = [
+  "fs",
+  "rt-multi-thread",
+  "macros",
+  "time",
+] }
 tokio-io-timeout = "1.2.0"
 tokio-stream = { version = "0.1.14", features = ["fs"] }
 tokio-tar = { version = "0.3" } # TODO: integrate tokio into async-tar
@@ -103,7 +136,9 @@ uuid = { version = "1", features = ["serde", "v4"] }
 
 [dev-dependencies]
 ansi_term = "0.12.0"
-anyhow = { version = "1", features = ["backtrace"] } # Enable `backtrace` feature in tests.
+anyhow = { version = "1", features = [
+  "backtrace",
+] } # Enable `backtrace` feature in tests.
 criterion = { version = "0.5.1", features = ["async_tokio"] }
 futures-lite = "2.0.0"
 log = "0.4"
@@ -111,7 +146,11 @@ pretty_env_logger = "0.5"
 proptest = { version = "1", default-features = false, features = ["std"] }
 tempfile = "3"
 testdir = "0.9.0"
-tokio = { version = "1", features = ["parking_lot", "rt-multi-thread", "macros"] }
+tokio = { version = "1", features = [
+  "parking_lot",
+  "rt-multi-thread",
+  "macros",
+] }
 pretty_assertions = "1.3.0"
 
 [workspace]
@@ -159,5 +198,5 @@ internals = []
 vendored = [
   "async-native-tls/vendored",
   "rusqlite/bundled-sqlcipher-vendored-openssl",
-  "reqwest/native-tls-vendored"
+  "reqwest/native-tls-vendored",
 ]

deltachat-ffi/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "deltachat_ffi"
-version = "1.132.1"
+version = "1.133.1"
 description = "Deltachat FFI"
 edition = "2018"
 readme = "README.md"

deltachat-ffi/DoxygenLayout.xml

@@ -9,7 +9,7 @@
       <tab type="hierarchy" visible="no" title="" intro=""/>
       <tab type="classmembers" visible="no" title="" intro=""/>
     </tab>
-    <tab type="modules" visible="yes" title="Constants" intro="Here is a list of constants:"/>
+    <tab type="topics" visible="yes" title="Constants" intro="Here is a list of constants:"/>
     <tab type="pages" visible="yes" title="" intro=""/>
     <tab type="namespaces" visible="yes" title="">
       <tab type="namespacelist" visible="yes" title="" intro=""/>

deltachat-ffi/deltachat.h

@@ -4397,6 +4397,9 @@ int             dc_msg_is_info                (const dc_msg_t* msg);
  * Currently, the following types are defined:
  * - DC_INFO_PROTECTION_ENABLED (11) - Info-message for "Chat is now protected"
  * - DC_INFO_PROTECTION_DISABLED (12) - Info-message for "Chat is no longer protected"
+ * - DC_INFO_INVALID_UNENCRYPTED_MAIL (13) - Info-message for "Provider requires end-to-end encryption which is not setup yet",
+ *   the UI should change the corresponding string using #DC_STR_INVALID_UNENCRYPTED_MAIL
+ *   and also offer a way to fix the encryption, eg. by a button offering a QR scan
  *
  * Even when you display an icon,
  * you should still display the text of the informational message using dc_msg_get_text()
@@ -4423,6 +4426,7 @@ int             dc_msg_get_info_type          (const dc_msg_t* msg);
 #define         DC_INFO_EPHEMERAL_TIMER_CHANGED   10
 #define         DC_INFO_PROTECTION_ENABLED        11
 #define         DC_INFO_PROTECTION_DISABLED       12
+#define         DC_INFO_INVALID_UNENCRYPTED_MAIL  13
 #define         DC_INFO_WEBXDC_INFO_MESSAGE       32
 
 /**
@@ -5068,6 +5072,15 @@ int             dc_contact_is_blocked        (const dc_contact_t* contact);
  */
 int             dc_contact_is_verified       (dc_contact_t* contact);
 
+/**
+ * Returns whether contact is a bot.
+ *
+ * @memberof dc_contact_t
+ * @param contact The contact object.
+ * @return 0 if the contact is not a bot, 1 otherwise.
+ */
+int             dc_contact_is_bot            (dc_contact_t* contact);
+
 
 /**
  * Return the contact ID that verified a contact.
@@ -6206,6 +6219,18 @@ void dc_event_unref(dc_event_t* event);
 #define DC_EVENT_SELFAVATAR_CHANGED               2110
 
 
+/**
+ * A multi-device synced config value changed. Maybe the app needs to refresh smth. For uniformity
+ * this is emitted on the source device too. The value isn't reported, otherwise it would be logged
+ * which might not be good for privacy. You can get the new value with
+ * `dc_get_config(context, data2)`.
+ *
+ * @param data1 0
+ * @param data2 (char*) Configuration key.
+ */
+#define DC_EVENT_CONFIG_SYNCED                    2111
+
+
 /**
  * webxdc status update received.
  * To get the received status update, use dc_get_webxdc_status_updates() with
@@ -6230,22 +6255,6 @@ void dc_event_unref(dc_event_t* event);
 
 #define DC_EVENT_WEBXDC_INSTANCE_DELETED          2121
 
-/**
- * Inform UI that Order (and content as in chat ids) of the chatlist changed.
- *
- * Sometimes this is emitted together with `DC_EVENT_UI_CHATLIST_ITEM_CHANGED` such as on `DC_EVENT_INCOMING_MSG`.
- */
-
-#define DC_EVENT_UI_CHATLIST_CHANGED          2200
-
-/**
- * Inform UI that all or a single chat list item changed and needs to be rerendered
- * If `chat_id` is set to 0, then all currently visible chats need to be rerendered, and all not-visible items need to be cleared from cache if the UI has a cache.
- * 
- * @param data1 (int) chat_id chat id of chatlist item to be rerendered, if chat_id = 0 all (cached & visible) items need to be rerendered
- */
-
-#define DC_EVENT_UI_CHATLIST_ITEM_CHANGED     2201
 
 /**
  * @}
@@ -7014,6 +7023,8 @@ void dc_event_unref(dc_event_t* event);
 /// "You added member %1$s."
 ///
 /// Used in status messages.
+///
+/// `%1$s` will be replaced by the added member's name.
 #define DC_STR_ADD_MEMBER_BY_YOU 128
 
 /// "Member %1$s added by %2$s."
@@ -7235,6 +7246,21 @@ void dc_event_unref(dc_event_t* event);
 /// Used as the first info messages in newly created groups.
 #define DC_STR_NEW_GROUP_SEND_FIRST_MESSAGE 172
 
+/// "Member %1$s added."
+///
+/// Used as info messages.
+///
+/// `%1$s` will be replaced by the added member's name.
+#define DC_STR_MESSAGE_ADD_MEMBER 173
+
+/// "Your email provider %1$s requires end-to-end encryption which is not setup yet."
+///
+/// Used as info messages when a message cannot be sent because it cannot be encrypted.
+///
+/// `%1$s` will be replaced by the provider's domain.
+#define DC_STR_INVALID_UNENCRYPTED_MAIL 174
+
+
 /**
  * @}
  */

deltachat-ffi/src/lib.rs

@@ -556,10 +556,9 @@ pub unsafe extern "C" fn dc_event_get_id(event: *mut dc_event_t) -> libc::c_int
         EventType::SecurejoinJoinerProgress { .. } => 2061,
         EventType::ConnectivityChanged => 2100,
         EventType::SelfavatarChanged => 2110,
+        EventType::ConfigSynced { .. } => 2111,
         EventType::WebxdcStatusUpdate { .. } => 2120,
         EventType::WebxdcInstanceDeleted { .. } => 2121,
-        EventType::UIChatListChanged => 2200,
-        EventType::UIChatListItemChanged { .. } => 2201,
     }
 }
 
@@ -585,9 +584,9 @@ pub unsafe extern "C" fn dc_event_get_data1_int(event: *mut dc_event_t) -> libc:
         | EventType::Error(_)
         | EventType::ConnectivityChanged
         | EventType::SelfavatarChanged
+        | EventType::ConfigSynced { .. }
         | EventType::IncomingMsgBunch { .. }
-        | EventType::ErrorSelfNotInGroup(_)
-        | EventType::UIChatListChanged => 0,
+        | EventType::ErrorSelfNotInGroup(_) => 0,
         EventType::MsgsChanged { chat_id, .. }
         | EventType::ReactionsChanged { chat_id, .. }
         | EventType::IncomingMsg { chat_id, .. }
@@ -612,9 +611,6 @@ pub unsafe extern "C" fn dc_event_get_data1_int(event: *mut dc_event_t) -> libc:
         }
         EventType::WebxdcStatusUpdate { msg_id, .. } => msg_id.to_u32() as libc::c_int,
         EventType::WebxdcInstanceDeleted { msg_id, .. } => msg_id.to_u32() as libc::c_int,
-        EventType::UIChatListItemChanged { chat_id } => {
-            chat_id.unwrap_or_default().to_u32() as libc::c_int
-        }
     }
 }
 
@@ -650,8 +646,7 @@ pub unsafe extern "C" fn dc_event_get_data2_int(event: *mut dc_event_t) -> libc:
         | EventType::WebxdcInstanceDeleted { .. }
         | EventType::IncomingMsgBunch { .. }
         | EventType::SelfavatarChanged
-        | EventType::UIChatListChanged
-        | EventType::UIChatListItemChanged { .. } => 0,
+        | EventType::ConfigSynced { .. } => 0,
         EventType::ChatModified(_) => 0,
         EventType::MsgsChanged { msg_id, .. }
         | EventType::ReactionsChanged { msg_id, .. }
@@ -713,9 +708,7 @@ pub unsafe extern "C" fn dc_event_get_data2_str(event: *mut dc_event_t) -> *mut
         | EventType::SelfavatarChanged
         | EventType::WebxdcStatusUpdate { .. }
         | EventType::WebxdcInstanceDeleted { .. }
-        | EventType::ChatEphemeralTimerModified { .. }
-        | EventType::UIChatListItemChanged { .. }
-        | EventType::UIChatListChanged => ptr::null_mut(),
+        | EventType::ChatEphemeralTimerModified { .. } => ptr::null_mut(),
         EventType::ConfigureProgress { comment, .. } => {
             if let Some(comment) = comment {
                 comment.to_c_string().unwrap_or_default().into_raw()
@@ -732,6 +725,10 @@ pub unsafe extern "C" fn dc_event_get_data2_str(event: *mut dc_event_t) -> *mut
             .to_c_string()
             .unwrap_or_default()
             .into_raw(),
+        EventType::ConfigSynced { key } => {
+            let data2 = key.to_string().to_c_string().unwrap_or_default();
+            data2.into_raw()
+        }
     }
 }
 
@@ -4135,6 +4132,15 @@ pub unsafe extern "C" fn dc_contact_is_verified(contact: *mut dc_contact_t) -> l
     }
 }
 
+#[no_mangle]
+pub unsafe extern "C" fn dc_contact_is_bot(contact: *mut dc_contact_t) -> libc::c_int {
+    if contact.is_null() {
+        eprintln!("ignoring careless call to dc_contact_is_bot()");
+        return 0;
+    }
+    (*contact).contact.is_bot() as libc::c_int
+}
+
 #[no_mangle]
 pub unsafe extern "C" fn dc_contact_get_verifier_id(contact: *mut dc_contact_t) -> u32 {
     if contact.is_null() {

deltachat-jsonrpc/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "deltachat-jsonrpc"
-version = "1.132.1"
+version = "1.133.1"
 description = "DeltaChat JSON-RPC API"
 edition = "2021"
 default-run = "deltachat-jsonrpc-server"

deltachat-jsonrpc/src/api.rs

@@ -15,7 +15,6 @@ use deltachat::constants::DC_MSG_ID_DAYMARKER;
 use deltachat::contact::{may_be_valid_addr, Contact, ContactId, Origin};
 use deltachat::context::get_info;
 use deltachat::ephemeral::Timer;
-use deltachat::imex;
 use deltachat::location;
 use deltachat::message::get_msg_read_receipts;
 use deltachat::message::{
@@ -28,6 +27,7 @@ use deltachat::reaction::{get_msg_reactions, send_reaction};
 use deltachat::securejoin;
 use deltachat::stock_str::StockMessage;
 use deltachat::webxdc::StatusUpdateSerial;
+use deltachat::{imex, webxdc};
 use sanitize_filename::is_sanitized;
 use tokio::fs;
 use tokio::sync::{watch, Mutex, RwLock};
@@ -1675,6 +1675,16 @@ impl CommandApi {
             .await
     }
 
+    async fn join_gossip_topic(
+        &self,
+        account_id: u32,
+        instance_msg_id: u32,
+        topic: String,
+    ) -> Result<()> {
+        let ctx = self.get_context(account_id).await?;
+        webxdc::join_gossip_topic(&ctx, MsgId::new(instance_msg_id), &topic).await
+    }
+
     async fn get_webxdc_status_updates(
         &self,
         account_id: u32,

deltachat-jsonrpc/src/api/types/contact.rs

@@ -45,6 +45,9 @@ pub struct ContactObject {
     /// the contact's last seen timestamp
     last_seen: i64,
     was_seen_recently: bool,
+
+    /// If the contact is a bot.
+    is_bot: bool,
 }
 
 impl ContactObject {
@@ -80,6 +83,7 @@ impl ContactObject {
             verifier_id,
             last_seen: contact.last_seen(),
             was_seen_recently: contact.was_seen_recently(),
+            is_bot: contact.is_bot(),
         })
     }
 }

deltachat-jsonrpc/src/api/types/events.rs

@@ -28,55 +28,37 @@ pub enum EventType {
     ///
     /// This event should *not* be reported to the end-user using a popup or something like
     /// that.
-    Info {
-        msg: String,
-    },
+    Info { msg: String },
 
     /// Emitted when SMTP connection is established and login was successful.
-    SmtpConnected {
-        msg: String,
-    },
+    SmtpConnected { msg: String },
 
     /// Emitted when IMAP connection is established and login was successful.
-    ImapConnected {
-        msg: String,
-    },
+    ImapConnected { msg: String },
 
     /// Emitted when a message was successfully sent to the SMTP server.
-    SmtpMessageSent {
-        msg: String,
-    },
+    SmtpMessageSent { msg: String },
 
     /// Emitted when an IMAP message has been marked as deleted
-    ImapMessageDeleted {
-        msg: String,
-    },
+    ImapMessageDeleted { msg: String },
 
     /// Emitted when an IMAP message has been moved
-    ImapMessageMoved {
-        msg: String,
-    },
+    ImapMessageMoved { msg: String },
 
     /// Emitted before going into IDLE on the Inbox folder.
     ImapInboxIdle,
 
     /// Emitted when an new file in the $BLOBDIR was created
-    NewBlobFile {
-        file: String,
-    },
+    NewBlobFile { file: String },
 
     /// Emitted when an file in the $BLOBDIR was deleted
-    DeletedBlobFile {
-        file: String,
-    },
+    DeletedBlobFile { file: String },
 
     /// The library-user should write a warning string to the log.
     ///
     /// This event should *not* be reported to the end-user using a popup or something like
     /// that.
-    Warning {
-        msg: String,
-    },
+    Warning { msg: String },
 
     /// The library-user should report an error to the end-user.
     ///
@@ -88,18 +70,14 @@ pub enum EventType {
     /// it might be better to delay showing these events until the function has really
     /// failed (returned false). It should be sufficient to report only the *last* error
     /// in a messasge box then.
-    Error {
-        msg: String,
-    },
+    Error { msg: String },
 
     /// An action cannot be performed because the user is not in the group.
     /// Reported eg. after a call to
     /// setChatName(), setChatProfileImage(),
     /// addContactToChat(), removeContactFromChat(),
     /// and messages sending functions.
-    ErrorSelfNotInGroup {
-        msg: String,
-    },
+    ErrorSelfNotInGroup { msg: String },
 
     /// Messages or chats changed.  One or more messages or chats changed for various
     /// reasons in the database:
@@ -110,10 +88,7 @@ pub enum EventType {
     /// `chatId` is set if only a single chat is affected by the changes, otherwise 0.
     /// `msgId` is set if only a single message is affected by the changes, otherwise 0.
     #[serde(rename_all = "camelCase")]
-    MsgsChanged {
-        chat_id: u32,
-        msg_id: u32,
-    },
+    MsgsChanged { chat_id: u32, msg_id: u32 },
 
     /// Reactions for the message changed.
     #[serde(rename_all = "camelCase")]
@@ -128,10 +103,7 @@ pub enum EventType {
     ///
     /// There is no extra #DC_EVENT_MSGS_CHANGED event send together with this event.
     #[serde(rename_all = "camelCase")]
-    IncomingMsg {
-        chat_id: u32,
-        msg_id: u32,
-    },
+    IncomingMsg { chat_id: u32, msg_id: u32 },
 
     /// Downloading a bunch of messages just finished. This is an experimental
     /// event to allow the UI to only show one notification per message bunch,
@@ -139,47 +111,31 @@ pub enum EventType {
     ///
     /// msg_ids contains the message ids.
     #[serde(rename_all = "camelCase")]
-    IncomingMsgBunch {
-        msg_ids: Vec<u32>,
-    },
+    IncomingMsgBunch { msg_ids: Vec<u32> },
 
     /// Messages were seen or noticed.
     /// chat id is always set.
     #[serde(rename_all = "camelCase")]
-    MsgsNoticed {
-        chat_id: u32,
-    },
+    MsgsNoticed { chat_id: u32 },
 
     /// A single message is sent successfully. State changed from  DC_STATE_OUT_PENDING to
     /// DC_STATE_OUT_DELIVERED, see `Message.state`.
     #[serde(rename_all = "camelCase")]
-    MsgDelivered {
-        chat_id: u32,
-        msg_id: u32,
-    },
+    MsgDelivered { chat_id: u32, msg_id: u32 },
 
     /// A single message could not be sent. State changed from DC_STATE_OUT_PENDING or DC_STATE_OUT_DELIVERED to
     /// DC_STATE_OUT_FAILED, see `Message.state`.
     #[serde(rename_all = "camelCase")]
-    MsgFailed {
-        chat_id: u32,
-        msg_id: u32,
-    },
+    MsgFailed { chat_id: u32, msg_id: u32 },
 
     /// A single message is read by the receiver. State changed from DC_STATE_OUT_DELIVERED to
     /// DC_STATE_OUT_MDN_RCVD, see `Message.state`.
     #[serde(rename_all = "camelCase")]
-    MsgRead {
-        chat_id: u32,
-        msg_id: u32,
-    },
+    MsgRead { chat_id: u32, msg_id: u32 },
 
     /// A single message is deleted.
     #[serde(rename_all = "camelCase")]
-    MsgDeleted {
-        chat_id: u32,
-        msg_id: u32,
-    },
+    MsgDeleted { chat_id: u32, msg_id: u32 },
 
     /// Chat changed.  The name or the image of a chat group was changed or members were added or removed.
     /// Or the verify state of a chat has changed.
@@ -189,24 +145,17 @@ pub enum EventType {
     /// This event does not include ephemeral timer modification, which
     /// is a separate event.
     #[serde(rename_all = "camelCase")]
-    ChatModified {
-        chat_id: u32,
-    },
+    ChatModified { chat_id: u32 },
 
     /// Chat ephemeral timer changed.
     #[serde(rename_all = "camelCase")]
-    ChatEphemeralTimerModified {
-        chat_id: u32,
-        timer: u32,
-    },
+    ChatEphemeralTimerModified { chat_id: u32, timer: u32 },
 
     /// Contact(s) created, renamed, blocked or deleted.
     ///
     /// @param data1 (int) If set, this is the contact_id of an added contact that should be selected.
     #[serde(rename_all = "camelCase")]
-    ContactsChanged {
-        contact_id: Option<u32>,
-    },
+    ContactsChanged { contact_id: Option<u32> },
 
     /// Location of one or more contact has changed.
     ///
@@ -214,9 +163,7 @@ pub enum EventType {
     ///     If the locations of several contacts have been changed,
     ///     this parameter is set to `None`.
     #[serde(rename_all = "camelCase")]
-    LocationChanged {
-        contact_id: Option<u32>,
-    },
+    LocationChanged { contact_id: Option<u32> },
 
     /// Inform about the configuration progress started by configure().
     ConfigureProgress {
@@ -234,9 +181,7 @@ pub enum EventType {
     /// @param data1 (usize) 0=error, 1-999=progress in permille, 1000=success and done
     /// @param data2 0
     #[serde(rename_all = "camelCase")]
-    ImexProgress {
-        progress: usize,
-    },
+    ImexProgress { progress: usize },
 
     /// A file has been exported. A file has been written by imex().
     /// This event may be sent multiple times by a single call to imex().
@@ -246,9 +191,7 @@ pub enum EventType {
     ///
     /// @param data2 0
     #[serde(rename_all = "camelCase")]
-    ImexFileWritten {
-        path: String,
-    },
+    ImexFileWritten { path: String },
 
     /// Progress information of a secure-join handshake from the view of the inviter
     /// (Alice, the person who shows the QR code).
@@ -263,10 +206,7 @@ pub enum EventType {
     ///     800=vg-member-added-received received, shown as "bob@addr securely joined GROUP", only sent for the verified-group-protocol.
     ///     1000=Protocol finished for this contact.
     #[serde(rename_all = "camelCase")]
-    SecurejoinInviterProgress {
-        contact_id: u32,
-        progress: usize,
-    },
+    SecurejoinInviterProgress { contact_id: u32, progress: usize },
 
     /// Progress information of a secure-join handshake from the view of the joiner
     /// (Bob, the person who scans the QR code).
@@ -277,10 +217,7 @@ pub enum EventType {
     ///     400=vg-/vc-request-with-auth sent, typically shown as "alice@addr verified, introducing myself."
     ///     (Bob has verified alice and waits until Alice does the same for him)
     #[serde(rename_all = "camelCase")]
-    SecurejoinJoinerProgress {
-        contact_id: u32,
-        progress: usize,
-    },
+    SecurejoinJoinerProgress { contact_id: u32, progress: usize },
 
     /// The connectivity to the server changed.
     /// This means that you should refresh the connectivity view
@@ -288,8 +225,17 @@ pub enum EventType {
     /// getConnectivityHtml() for details.
     ConnectivityChanged,
 
+    /// Deprecated by `ConfigSynced`.
     SelfavatarChanged,
 
+    /// A multi-device synced config value changed. Maybe the app needs to refresh smth. For
+    /// uniformity this is emitted on the source device too. The value isn't here, otherwise it
+    /// would be logged which might not be good for privacy.
+    ConfigSynced {
+        /// Configuration key.
+        key: String,
+    },
+
     #[serde(rename_all = "camelCase")]
     WebxdcStatusUpdate {
         msg_id: u32,
@@ -298,21 +244,7 @@ pub enum EventType {
 
     /// Inform that a message containing a webxdc instance has been deleted
     #[serde(rename_all = "camelCase")]
-    WebxdcInstanceDeleted {
-        msg_id: u32,
-    },
-
-    /// Inform UI that Order (and content as in chat ids) of the chatlist changed.
-    ///
-    /// Sometimes this is emitted together with `UIChatListItemChanged` such as on IncomingMessage.
-    UIChatListChanged,
-
-    /// Inform UI that a single chat list item changed and needs to be rerendered
-    /// If `chat_id` is set to None, then all currently visible chats need to be rerendered, and all not-visible items need to be cleared from cache if the UI has a cache.
-    #[serde(rename_all = "camelCase")]
-    UIChatListItemChanged {
-        chat_id: Option<u32>,
-    },
+    WebxdcInstanceDeleted { msg_id: u32 },
 }
 
 impl From<CoreEventType> for EventType {
@@ -408,6 +340,9 @@ impl From<CoreEventType> for EventType {
             },
             CoreEventType::ConnectivityChanged => ConnectivityChanged,
             CoreEventType::SelfavatarChanged => SelfavatarChanged,
+            CoreEventType::ConfigSynced { key } => ConfigSynced {
+                key: key.to_string(),
+            },
             CoreEventType::WebxdcStatusUpdate {
                 msg_id,
                 status_update_serial,
@@ -418,10 +353,6 @@ impl From<CoreEventType> for EventType {
             CoreEventType::WebxdcInstanceDeleted { msg_id } => WebxdcInstanceDeleted {
                 msg_id: msg_id.to_u32(),
             },
-            CoreEventType::UIChatListItemChanged { chat_id } => UIChatListItemChanged {
-                chat_id: chat_id.map(|id| id.to_u32()),
-            },
-            CoreEventType::UIChatListChanged => UIChatListChanged,
         }
     }
 }

deltachat-jsonrpc/src/api/types/message.rs

@@ -345,6 +345,7 @@ pub enum SystemMessageType {
     SecurejoinMessage,
     LocationStreamingEnabled,
     LocationOnly,
+    InvalidUnencryptedMail,
 
     /// Chat ephemeral message timer is changed.
     EphemeralTimerChanged,
@@ -385,6 +386,7 @@ impl From<deltachat::mimeparser::SystemMessage> for SystemMessageType {
             SystemMessage::MultiDeviceSync => SystemMessageType::MultiDeviceSync,
             SystemMessage::WebxdcStatusUpdate => SystemMessageType::WebxdcStatusUpdate,
             SystemMessage::WebxdcInfoMessage => SystemMessageType::WebxdcInfoMessage,
+            SystemMessage::InvalidUnencryptedMail => SystemMessageType::InvalidUnencryptedMail,
         }
     }
 }

deltachat-jsonrpc/typescript/package.json

@@ -53,5 +53,5 @@
   },
   "type": "module",
   "types": "dist/deltachat.d.ts",
-  "version": "1.132.1"
+  "version": "1.133.1"
 }

deltachat-repl/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "deltachat-repl"
-version = "1.132.1"
+version = "1.133.1"
 license = "MPL-2.0"
 edition = "2021"
 
@@ -12,7 +12,7 @@ dirs = "5"
 log = "0.4.20"
 pretty_env_logger = "0.5"
 rusqlite = "0.30"
-rustyline = "12"
+rustyline = "13"
 tokio = { version = "1", features = ["fs", "rt-multi-thread", "macros"] }
 
 [features]

deltachat-repl/src/main.rs

@@ -299,8 +299,8 @@ impl Highlighter for DcHelper {
         self.highlighter.highlight(line, pos)
     }
 
-    fn highlight_char(&self, line: &str, pos: usize) -> bool {
-        self.highlighter.highlight_char(line, pos)
+    fn highlight_char(&self, line: &str, pos: usize, forced: bool) -> bool {
+        self.highlighter.highlight_char(line, pos, forced)
     }
 }
 

deltachat-rpc-client/tests/test_securejoin.py

@@ -175,7 +175,7 @@ def test_verified_group_recovery(acfactory) -> None:
     logging.info("ac2 joins verified group")
     qr_code, _svg = chat.get_qr_code()
     ac2.secure_join(qr_code)
-    ac1.wait_for_securejoin_inviter_success()
+    ac2.wait_for_securejoin_joiner_success()
 
     # ac1 has ac2 directly verified.
     ac1_contact_ac2 = ac1.get_contact_by_addr(ac2.get_config("addr"))
@@ -183,7 +183,8 @@ def test_verified_group_recovery(acfactory) -> None:
 
     logging.info("ac3 joins verified group")
     ac3_chat = ac3.secure_join(qr_code)
-    ac1.wait_for_securejoin_inviter_success()
+    ac3.wait_for_securejoin_joiner_success()
+    ac3.wait_for_incoming_msg_event()  # Member added
 
     logging.info("ac2 logs in on a new device")
     ac2 = acfactory.resetup_account(ac2)
@@ -191,8 +192,7 @@ def test_verified_group_recovery(acfactory) -> None:
     logging.info("ac2 reverifies with ac3")
     qr_code, _svg = ac3.get_qr_code()
     ac2.secure_join(qr_code)
-
-    ac3.wait_for_securejoin_inviter_success()
+    ac2.wait_for_securejoin_joiner_success()
 
     logging.info("ac3 sends a message to the group")
     assert len(ac3_chat.get_contacts()) == 3
@@ -239,7 +239,7 @@ def test_verified_group_member_added_recovery(acfactory) -> None:
     logging.info("ac2 joins verified group")
     qr_code, _svg = chat.get_qr_code()
     ac2.secure_join(qr_code)
-    ac1.wait_for_securejoin_inviter_success()
+    ac2.wait_for_securejoin_joiner_success()
 
     # ac1 has ac2 directly verified.
     ac1_contact_ac2 = ac1.get_contact_by_addr(ac2.get_config("addr"))
@@ -247,7 +247,8 @@ def test_verified_group_member_added_recovery(acfactory) -> None:
 
     logging.info("ac3 joins verified group")
     ac3_chat = ac3.secure_join(qr_code)
-    ac1.wait_for_securejoin_inviter_success()
+    ac3.wait_for_securejoin_joiner_success()
+    ac3.wait_for_incoming_msg_event()  # Member added
 
     logging.info("ac2 logs in on a new device")
     ac2 = acfactory.resetup_account(ac2)
@@ -255,8 +256,7 @@ def test_verified_group_member_added_recovery(acfactory) -> None:
     logging.info("ac2 reverifies with ac3")
     qr_code, _svg = ac3.get_qr_code()
     ac2.secure_join(qr_code)
-
-    ac3.wait_for_securejoin_inviter_success()
+    ac2.wait_for_securejoin_joiner_success()
 
     logging.info("ac3 sends a message to the group")
     assert len(ac3_chat.get_contacts()) == 3

deltachat-rpc-server/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "deltachat-rpc-server"
-version = "1.132.1"
+version = "1.133.1"
 description = "DeltaChat JSON-RPC server"
 edition = "2021"
 readme = "README.md"

deny.toml

@@ -18,57 +18,58 @@ ignore = [
 # when upgrading.
 # Please keep this list alphabetically sorted.
 skip = [
-     { name = "async-channel", version = "1.9.0" },
-     { name = "base16ct", version = "0.1.1" },
-     { name = "base64", version = "<0.21" },
-     { name = "bitflags", version = "1.3.2" },
-     { name = "block-buffer", version = "<0.10" },
-     { name = "convert_case", version = "0.4.0" },
-     { name = "curve25519-dalek", version = "3.2.0" },
-     { name = "darling_core", version = "<0.14" },
-     { name = "darling_macro", version = "<0.14" },
-     { name = "darling", version = "<0.14" },
-     { name = "der", version = "0.6.1" },
-     { name = "digest", version = "<0.10" },
-     { name = "ed25519-dalek", version = "1.0.1" },
-     { name = "ed25519", version = "1.5.3" },
-     { name = "event-listener", version = "2.5.3" },
-     { name = "fd-lock", version = "3.0.13" },
-     { name = "getrandom", version = "<0.2" },
-     { name = "h2", version = "0.3.22" },
-     { name = "http-body", version = "0.4.5" },
-     { name = "http", version = "0.2.11" },
-     { name = "hyper", version = "0.14.27" },
-     { name = "idna", version = "0.4.0" },
-     { name = "pem-rfc7468", version = "0.6.0" },
-     { name = "pkcs8", version = "0.9.0" },
-     { name = "quick-error", version = "<2.0" },
-     { name = "rand_chacha", version = "<0.3" },
-     { name = "rand_core", version = "<0.6" },
-     { name = "rand", version = "<0.8" },
-     { name = "redox_syscall", version = "0.3.5" },
-     { name = "regex-automata", version = "0.1.10" },
-     { name = "regex-syntax", version = "0.6.29" },
-     { name = "ring", version = "0.16.20" },
-     { name = "sec1", version = "0.3.0" },
-     { name = "sha2", version = "<0.10" },
-     { name = "signature", version = "1.6.4" },
-     { name = "spin", version = "<0.9.6" },
-     { name = "spki", version = "0.6.0" },
-     { name = "syn", version = "1.0.109" },
-     { name = "time", version = "<0.3" },
-     { name = "untrusted", version = "0.7.1" },
-     { name = "wasi", version = "<0.11" },
-     { name = "windows_aarch64_gnullvm", version = "<0.52" },
-     { name = "windows_aarch64_msvc", version = "<0.52" },
-     { name = "windows_i686_gnu", version = "<0.52" },
-     { name = "windows_i686_msvc", version = "<0.52" },
-     { name = "windows-sys", version = "<0.52" },
-     { name = "windows-targets", version = "<0.52" },
-     { name = "windows", version = "0.32.0" },
-     { name = "windows_x86_64_gnullvm", version = "<0.52" },
-     { name = "windows_x86_64_gnu", version = "<0.52" },
-     { name = "windows_x86_64_msvc", version = "<0.52" },
+    { name = "async-channel", version = "1.9.0" },
+    { name = "base16ct", version = "0.1.1" },
+    { name = "base64", version = "<0.21" },
+    { name = "bitflags", version = "1.3.2" },
+    { name = "block-buffer", version = "<0.10" },
+    { name = "convert_case", version = "0.4.0" },
+    { name = "curve25519-dalek", version = "3.2.0" },
+    { name = "darling_core", version = "<0.14" },
+    { name = "darling_macro", version = "<0.14" },
+    { name = "darling", version = "<0.14" },
+    { name = "der", version = "0.6.1" },
+    { name = "digest", version = "<0.10" },
+    { name = "ed25519-dalek", version = "1.0.1" },
+    { name = "ed25519", version = "1.5.3" },
+    { name = "event-listener", version = "2.5.3" },
+    { name = "fd-lock", version = "3.0.13" },
+    { name = "getrandom", version = "<0.2" },
+    { name = "h2", version = "0.3.22" },
+    { name = "http-body", version = "0.4.5" },
+    { name = "http", version = "0.2.11" },
+    { name = "hyper", version = "0.14.27" },
+    { name = "idna", version = "0.4.0" },
+    { name = "pem-rfc7468", version = "0.6.0" },
+    { name = "pkcs8", version = "0.9.0" },
+    { name = "quick-error", version = "<2.0" },
+    { name = "rand_chacha", version = "<0.3" },
+    { name = "rand_core", version = "<0.6" },
+    { name = "rand", version = "<0.8" },
+    { name = "redox_syscall", version = "0.3.5" },
+    { name = "regex-automata", version = "0.1.10" },
+    { name = "regex-syntax", version = "0.6.29" },
+    { name = "ring", version = "0.16.20" },
+    { name = "sec1", version = "0.3.0" },
+    { name = "sha2", version = "<0.10" },
+    { name = "signature", version = "1.6.4" },
+    { name = "socket2", version = "0.4.9" },
+    { name = "spin", version = "<0.9.6" },
+    { name = "spki", version = "0.6.0" },
+    { name = "syn", version = "1.0.109" },
+    { name = "time", version = "<0.3" },
+    { name = "untrusted", version = "0.7.1" },
+    { name = "wasi", version = "<0.11" },
+    { name = "windows_aarch64_gnullvm", version = "<0.52" },
+    { name = "windows_aarch64_msvc", version = "<0.52" },
+    { name = "windows_i686_gnu", version = "<0.52" },
+    { name = "windows_i686_msvc", version = "<0.52" },
+    { name = "windows-sys", version = "<0.52" },
+    { name = "windows-targets", version = "<0.52" },
+    { name = "windows", version = "0.32.0" },
+    { name = "windows_x86_64_gnullvm", version = "<0.52" },
+    { name = "windows_x86_64_gnu", version = "<0.52" },
+    { name = "windows_x86_64_msvc", version = "<0.52" },
 ]
 
 
@@ -78,7 +79,7 @@ allow = [
     "Apache-2.0",
     "BSD-2-Clause",
     "BSD-3-Clause",
-    "BSL-1.0", # Boost Software License 1.0
+    "BSL-1.0",          # Boost Software License 1.0
     "CC0-1.0",
     "ISC",
     "MIT",
@@ -91,14 +92,13 @@ allow = [
 [[licenses.clarify]]
 name = "ring"
 expression = "MIT AND ISC AND OpenSSL"
-license-files = [
-      { path = "LICENSE", hash = 0xbd0eed23 },
-]
+license-files = [{ path = "LICENSE", hash = 0xbd0eed23 }]
 
 [sources.allow-org]
 # Organisations which we allow git sources from.
 github = [
-       "async-email",
-       "deltachat",
-       "djc",
+    "async-email",
+    "deltachat",
+    "djc",
+    "n0-computer", # iroh
 ]

fuzz/Cargo.lock

@@ -190,11 +190,11 @@ dependencies = [
 
 [[package]]
 name = "async-imap"
-version = "0.9.1"
+version = "0.9.5"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "b538b767cbf9c162a6c5795d4b932bd2c20ba10b5a91a94d2b2b6886c1dce6a8"
+checksum = "a9d69fc1499878158750f644c4eb46aff55bb9d32d77e3dc4aecf8308d5c3ba6"
 dependencies = [
- "async-channel 1.8.0",
+ "async-channel 2.1.1",
  "base64 0.21.0",
  "bytes",
  "chrono",
@@ -922,7 +922,7 @@ dependencies = [
 
 [[package]]
 name = "deltachat"
-version = "1.131.9"
+version = "1.133.0"
 dependencies = [
  "anyhow",
  "async-channel 2.1.1",
@@ -963,6 +963,7 @@ dependencies = [
  "pin-project",
  "qrcodegen",
  "quick-xml",
+ "quoted_printable 0.5.0",
  "rand 0.8.5",
  "ratelimit",
  "regex",
@@ -1859,9 +1860,9 @@ dependencies = [
 
 [[package]]
 name = "h2"
-version = "0.3.17"
+version = "0.3.24"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "66b91535aa35fea1523ad1b86cb6b53c28e0ae566ba4a460f4457e936cad7c6f"
+checksum = "bb2c4422095b67ee78da96fbb51a4cc413b3b25883c7717ff7ca1ab31022c9c9"
 dependencies = [
  "bytes",
  "fnv",
@@ -1869,7 +1870,7 @@ dependencies = [
  "futures-sink",
  "futures-util",
  "http",
- "indexmap 1.9.2",
+ "indexmap",
  "slab",
  "tokio",
  "tokio-util",
@@ -2146,16 +2147,6 @@ dependencies = [
  "nom",
 ]
 
-[[package]]
-name = "indexmap"
-version = "1.9.2"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "1885e79c1fc4b10f0e172c475f458b7f7b93061064d98c3293e98c5ba0c8b399"
-dependencies = [
- "autocfg",
- "hashbrown 0.12.3",
-]
-
 [[package]]
 name = "indexmap"
 version = "2.1.0"
@@ -2215,7 +2206,8 @@ checksum = "11b0d96e660696543b251e58030cf9787df56da39dab19ad60eae7353040917e"
 [[package]]
 name = "iroh"
 version = "0.4.2"
-source = "git+https://github.com/n0-computer/iroh?branch=maint-0.4#9881b7886235035a1124e4371f7a4cd59379e51b"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "85075391dcb8491a4939266334b28601052d418b37d20b33c58ffb5776adc912"
 dependencies = [
  "abao",
  "anyhow",
@@ -2420,7 +2412,7 @@ checksum = "8cae768a50835557749599277fc59f7c728118724eb34185e8feb633ef266a32"
 dependencies = [
  "charset",
  "data-encoding",
- "quoted_printable",
+ "quoted_printable 0.4.6",
 ]
 
 [[package]]
@@ -2431,7 +2423,7 @@ checksum = "6b56570f5f8c0047260d1c8b5b331f62eb9c660b9dd4071a8c46f8c7d3f280aa"
 dependencies = [
  "charset",
  "data-encoding",
- "quoted_printable",
+ "quoted_printable 0.4.6",
 ]
 
 [[package]]
@@ -3258,6 +3250,12 @@ version = "0.4.6"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "20f14e071918cbeefc5edc986a7aa92c425dae244e003a35e1cdddb5ca39b5cb"
 
+[[package]]
+name = "quoted_printable"
+version = "0.5.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "79ec282e887b434b68c18fe5c121d38e72a5cf35119b59e54ec5b992ea9c8eb0"
+
 [[package]]
 name = "rand"
 version = "0.7.3"
@@ -3411,9 +3409,9 @@ checksum = "dbb5fb1acd8a1a18b3dd5be62d25485eb770e05afb408a9627d14d451bae12da"
 
 [[package]]
 name = "reqwest"
-version = "0.11.20"
+version = "0.11.23"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "3e9ad3fe7488d7e34558a2033d45a0c90b72d97b4f80705666fea71472e2e6a1"
+checksum = "37b1ae8d9ac08420c66222fb9096fc5de435c3c48542bc5336c51892cffafb41"
 dependencies = [
  "base64 0.21.0",
  "bytes",
@@ -3436,6 +3434,7 @@ dependencies = [
  "serde",
  "serde_json",
  "serde_urlencoded",
+ "system-configuration",
  "tokio",
  "tokio-native-tls",
  "tower-service",
@@ -4156,9 +4155,9 @@ dependencies = [
 
 [[package]]
 name = "system-configuration"
-version = "0.5.0"
+version = "0.5.1"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "d75182f12f490e953596550b65ee31bda7c8e043d9386174b353bda50838c3fd"
+checksum = "ba3a3adc5c275d719af8cb4272ea1c4a6d668a777f37e115f6d11ddbc1c8e0e7"
 dependencies = [
  "bitflags 1.3.2",
  "core-foundation",
@@ -4429,7 +4428,7 @@ version = "0.21.0"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "d34d383cd00a163b4a5b85053df514d45bc330f6de7737edfe0a93311d1eaa03"
 dependencies = [
- "indexmap 2.1.0",
+ "indexmap",
  "serde",
  "serde_spanned",
  "toml_datetime",

node/constants.js

@@ -32,6 +32,7 @@ module.exports = {
   DC_EVENT_CHAT_EPHEMERAL_TIMER_MODIFIED: 2021,
   DC_EVENT_CHAT_MODIFIED: 2020,
   DC_EVENT_CONFIGURE_PROGRESS: 2041,
+  DC_EVENT_CONFIG_SYNCED: 2111,
   DC_EVENT_CONNECTIVITY_CHANGED: 2100,
   DC_EVENT_CONTACTS_CHANGED: 2030,
   DC_EVENT_DELETED_BLOB_FILE: 151,
@@ -60,8 +61,6 @@ module.exports = {
   DC_EVENT_SELFAVATAR_CHANGED: 2110,
   DC_EVENT_SMTP_CONNECTED: 101,
   DC_EVENT_SMTP_MESSAGE_SENT: 103,
-  DC_EVENT_UI_CHATLIST_CHANGED: 2200,
-  DC_EVENT_UI_CHATLIST_ITEM_CHANGED: 2201,
   DC_EVENT_WARNING: 300,
   DC_EVENT_WEBXDC_INSTANCE_DELETED: 2121,
   DC_EVENT_WEBXDC_STATUS_UPDATE: 2120,
@@ -81,6 +80,7 @@ module.exports = {
   DC_INFO_EPHEMERAL_TIMER_CHANGED: 10,
   DC_INFO_GROUP_IMAGE_CHANGED: 3,
   DC_INFO_GROUP_NAME_CHANGED: 2,
+  DC_INFO_INVALID_UNENCRYPTED_MAIL: 13,
   DC_INFO_LOCATIONSTREAMING_ENABLED: 8,
   DC_INFO_LOCATION_ONLY: 9,
   DC_INFO_MEMBER_ADDED_TO_GROUP: 4,
@@ -227,11 +227,13 @@ module.exports = {
   DC_STR_GROUP_NAME_CHANGED_BY_YOU: 124,
   DC_STR_IMAGE: 9,
   DC_STR_INCOMING_MESSAGES: 103,
+  DC_STR_INVALID_UNENCRYPTED_MAIL: 174,
   DC_STR_LAST_MSG_SENT_SUCCESSFULLY: 111,
   DC_STR_LOCATION: 66,
   DC_STR_LOCATION_ENABLED_BY_OTHER: 137,
   DC_STR_LOCATION_ENABLED_BY_YOU: 136,
   DC_STR_MESSAGES: 114,
+  DC_STR_MESSAGE_ADD_MEMBER: 173,
   DC_STR_MSGACTIONBYME: 63,
   DC_STR_MSGACTIONBYUSER: 62,
   DC_STR_MSGADDMEMBER: 17,

node/events.js

@@ -34,8 +34,7 @@ module.exports = {
   2061: 'DC_EVENT_SECUREJOIN_JOINER_PROGRESS',
   2100: 'DC_EVENT_CONNECTIVITY_CHANGED',
   2110: 'DC_EVENT_SELFAVATAR_CHANGED',
+  2111: 'DC_EVENT_CONFIG_SYNCED',
   2120: 'DC_EVENT_WEBXDC_STATUS_UPDATE',
-  2121: 'DC_EVENT_WEBXDC_INSTANCE_DELETED',
-  2200: 'DC_EVENT_UI_CHATLIST_CHANGED',
-  2201: 'DC_EVENT_UI_CHATLIST_ITEM_CHANGED'
+  2121: 'DC_EVENT_WEBXDC_INSTANCE_DELETED'
 }

node/lib/constants.ts

@@ -32,6 +32,7 @@ export enum C {
   DC_EVENT_CHAT_EPHEMERAL_TIMER_MODIFIED = 2021,
   DC_EVENT_CHAT_MODIFIED = 2020,
   DC_EVENT_CONFIGURE_PROGRESS = 2041,
+  DC_EVENT_CONFIG_SYNCED = 2111,
   DC_EVENT_CONNECTIVITY_CHANGED = 2100,
   DC_EVENT_CONTACTS_CHANGED = 2030,
   DC_EVENT_DELETED_BLOB_FILE = 151,
@@ -60,8 +61,6 @@ export enum C {
   DC_EVENT_SELFAVATAR_CHANGED = 2110,
   DC_EVENT_SMTP_CONNECTED = 101,
   DC_EVENT_SMTP_MESSAGE_SENT = 103,
-  DC_EVENT_UI_CHATLIST_CHANGED = 2200,
-  DC_EVENT_UI_CHATLIST_ITEM_CHANGED = 2201,
   DC_EVENT_WARNING = 300,
   DC_EVENT_WEBXDC_INSTANCE_DELETED = 2121,
   DC_EVENT_WEBXDC_STATUS_UPDATE = 2120,
@@ -81,6 +80,7 @@ export enum C {
   DC_INFO_EPHEMERAL_TIMER_CHANGED = 10,
   DC_INFO_GROUP_IMAGE_CHANGED = 3,
   DC_INFO_GROUP_NAME_CHANGED = 2,
+  DC_INFO_INVALID_UNENCRYPTED_MAIL = 13,
   DC_INFO_LOCATIONSTREAMING_ENABLED = 8,
   DC_INFO_LOCATION_ONLY = 9,
   DC_INFO_MEMBER_ADDED_TO_GROUP = 4,
@@ -227,11 +227,13 @@ export enum C {
   DC_STR_GROUP_NAME_CHANGED_BY_YOU = 124,
   DC_STR_IMAGE = 9,
   DC_STR_INCOMING_MESSAGES = 103,
+  DC_STR_INVALID_UNENCRYPTED_MAIL = 174,
   DC_STR_LAST_MSG_SENT_SUCCESSFULLY = 111,
   DC_STR_LOCATION = 66,
   DC_STR_LOCATION_ENABLED_BY_OTHER = 137,
   DC_STR_LOCATION_ENABLED_BY_YOU = 136,
   DC_STR_MESSAGES = 114,
+  DC_STR_MESSAGE_ADD_MEMBER = 173,
   DC_STR_MSGACTIONBYME = 63,
   DC_STR_MSGACTIONBYUSER = 62,
   DC_STR_MSGADDMEMBER = 17,
@@ -321,8 +323,7 @@ export const EventId2EventName: { [key: number]: string } = {
   2061: 'DC_EVENT_SECUREJOIN_JOINER_PROGRESS',
   2100: 'DC_EVENT_CONNECTIVITY_CHANGED',
   2110: 'DC_EVENT_SELFAVATAR_CHANGED',
+  2111: 'DC_EVENT_CONFIG_SYNCED',
   2120: 'DC_EVENT_WEBXDC_STATUS_UPDATE',
   2121: 'DC_EVENT_WEBXDC_INSTANCE_DELETED',
-  2200: 'DC_EVENT_UI_CHATLIST_CHANGED',
-  2201: 'DC_EVENT_UI_CHATLIST_ITEM_CHANGED',
 }

node/lib/deltachat.ts

@@ -178,7 +178,7 @@ export class AccountManager extends EventEmitter {
   static newTemporary() {
     let directory = null
     while (true) {
-      const randomString = Math.random().toString(36).substr(2, 5)
+      const randomString = Math.random().toString(36).substring(2, 5)
       directory = join(tmpdir(), 'deltachat-' + randomString)
       if (!existsSync(directory)) break
     }

node/test/test.mjs

@@ -1,13 +1,17 @@
 // @ts-check
-import DeltaChat from '../dist'
+import { DeltaChat } from '../dist/index.js'
 
 import { deepStrictEqual, strictEqual } from 'assert'
 import chai, { expect } from 'chai'
 import chaiAsPromised from 'chai-as-promised'
-import { EventId2EventName, C } from '../dist/constants'
+import { EventId2EventName, C } from '../dist/constants.js'
 import { join } from 'path'
 import { statSync } from 'fs'
-import { Context } from '../dist/context'
+import { Context } from '../dist/context.js'
+import {fileURLToPath} from 'url';
+
+const __dirname = fileURLToPath(new URL('.', import.meta.url));
+
 chai.use(chaiAsPromised)
 chai.config.truncateThreshold = 0 // Do not truncate assertion errors.
 

package.json

@@ -10,7 +10,6 @@
     "@types/node": "^20.8.10",
     "chai": "~4.3.10",
     "chai-as-promised": "^7.1.1",
-    "esm": "^3.2.25",
     "mocha": "^8.2.1",
     "node-gyp": "^10.0.0",
     "prebuildify": "^5.0.1",
@@ -53,8 +52,8 @@
     "prebuildify": "cd node && prebuildify -t 18.0.0 --napi --strip --postinstall \"node scripts/postinstall.js --prebuild\"",
     "test": "npm run test:lint && npm run test:mocha",
     "test:lint": "npm run lint",
-    "test:mocha": "mocha -r esm node/test/test.js --growl --reporter=spec --bail --exit"
+    "test:mocha": "mocha node/test/test.mjs --growl --reporter=spec --bail --exit"
   },
   "types": "node/dist/index.d.ts",
-  "version": "1.132.1"
+  "version": "1.133.1"
 }

python/src/deltachat/testplugin.py

@@ -10,6 +10,7 @@ import time
 import weakref
 import random
 from queue import Queue
+from threading import Event
 from typing import Callable, Dict, List, Optional, Set
 
 import pytest
@@ -590,6 +591,27 @@ class ACFactory:
         ac2.create_chat(ac1)
         return ac1.create_chat(ac2)
 
+    def get_protected_chat(self, ac1: Account, ac2: Account):
+        class SetupPlugin:
+            def __init__(self) -> None:
+                self.member_added = Event()
+
+            @account_hookimpl
+            def ac_member_added(self, chat: deltachat.Chat, contact, actor, message):
+                self.member_added.set()
+
+        setupplugin = SetupPlugin()
+        ac1.add_account_plugin(setupplugin)
+        chat = ac1.create_group_chat("Protected Group", verified=True)
+        qr = chat.get_join_qr()
+        ac2.qr_join_chat(qr)
+        setupplugin.member_added.wait()
+        msg = ac2.wait_next_incoming_message()
+        assert msg.text == "Messages are guaranteed to be end-to-end encrypted from now on."
+        msg = ac2.wait_next_incoming_message()
+        assert "Member Me " in msg.text and " added by " in msg.text
+        return chat
+
     def introduce_each_other(self, accounts, sending=True):
         to_wait = []
         for i, acc in enumerate(accounts):

python/tests/test_1_online.py

@@ -498,6 +498,26 @@ def test_forward_messages(acfactory, lp):
     assert not chat3.get_messages()
 
 
+def test_forward_encrypted_to_unencrypted(acfactory, lp):
+    ac1, ac2, ac3 = acfactory.get_online_accounts(3)
+    chat = acfactory.get_protected_chat(ac1, ac2)
+
+    lp.sec("ac1: send encrypted message to ac2")
+    txt = "This should be encrypted"
+    chat.send_text(txt)
+    msg = ac2.wait_next_incoming_message()
+    assert msg.text == txt
+    assert msg.is_encrypted()
+
+    lp.sec("ac2: forward message to ac3 unencrypted")
+    unencrypted_chat = ac2.create_chat(ac3)
+    msg_id = msg.id
+    msg2 = unencrypted_chat.send_msg(msg)
+    assert msg2 == msg
+    assert msg.id != msg_id
+    assert not msg.is_encrypted()
+
+
 def test_forward_own_message(acfactory, lp):
     ac1, ac2 = acfactory.get_online_accounts(2)
     chat = acfactory.get_accepted_chat(ac1, ac2)

release-date.in

@@ -1 +1 @@
-2023-12-12
+2024-01-21

scripts/set_core_version.py

@@ -102,7 +102,7 @@ def main():
                 found = True
         if not found:
             raise SystemExit(
-                f"{changelog_name} contains no entry for version: {newversion}"
+                f"CHANGELOG.md contains no entry for version: {newversion}"
             )
 
     for toml_filename in toml_list:

src/chat.rs

@@ -46,7 +46,6 @@ use crate::tools::{
     create_smeared_timestamps, get_abs_path, gm2local_offset, improve_single_line_input,
     smeared_time, strip_rtlo_characters, time, IsNoneOrEmpty,
 };
-use crate::ui_events;
 use crate::webxdc::WEBXDC_SUFFIX;
 
 /// An chat item, such as a message or a marker.
@@ -310,8 +309,6 @@ impl ChatId {
             }
         };
         context.emit_msgs_changed_without_ids();
-        ui_events::emit_chatlist_changed(context);
-        ui_events::emit_chatlist_item_changed(context, chat_id);
         Ok(chat_id)
     }
 
@@ -428,7 +425,6 @@ impl ChatId {
                 }
             }
         }
-        ui_events::emit_chatlist_changed(context);
 
         if sync.into() {
             // NB: For a 1:1 chat this currently triggers `Contact::block()` on other devices.
@@ -451,8 +447,6 @@ impl ChatId {
     pub(crate) async fn unblock_ex(self, context: &Context, sync: sync::Sync) -> Result<()> {
         self.set_blocked(context, Blocked::Not).await?;
 
-        ui_events::emit_chatlist_changed(context);
-
         if sync.into() {
             let chat = Chat::load_from_db(context, self).await?;
             // TODO: For a 1:1 chat this currently triggers `Contact::unblock()` on other devices.
@@ -463,7 +457,6 @@ impl ChatId {
                 .log_err(context)
                 .ok();
         }
-
         Ok(())
     }
 
@@ -507,7 +500,6 @@ impl ChatId {
 
         if self.set_blocked(context, Blocked::Not).await? {
             context.emit_event(EventType::ChatModified(self));
-            ui_events::emit_chatlist_item_changed(context, self);
         }
 
         if sync.into() {
@@ -550,7 +542,6 @@ impl ChatId {
             .await?;
 
         context.emit_event(EventType::ChatModified(self));
-        ui_events::emit_chatlist_item_changed(context, self);
 
         // make sure, the receivers will get all keys
         self.reset_gossiped_timestamp(context).await?;
@@ -599,7 +590,6 @@ impl ChatId {
                 if protection_status_modified {
                     self.add_protection_msg(context, protect, contact_id, timestamp_sort)
                         .await?;
-                    ui_events::emit_chatlist_item_changed(context, self);
                 }
                 Ok(())
             }
@@ -686,8 +676,6 @@ impl ChatId {
             .await?;
 
         context.emit_msgs_changed_without_ids();
-        ui_events::emit_chatlist_changed(context);
-        ui_events::emit_chatlist_item_changed(context, self);
 
         if sync.into() {
             let chat = Chat::load_from_db(context, self).await?;
@@ -794,7 +782,6 @@ impl ChatId {
             .await?;
 
         context.emit_msgs_changed_without_ids();
-        ui_events::emit_chatlist_changed(context);
 
         context.set_config(Config::LastHousekeeping, None).await?;
         context.scheduler.interrupt_inbox().await;
@@ -804,7 +791,6 @@ impl ChatId {
             msg.text = stock_str::self_deleted_msg_body(context).await;
             add_device_msg(context, None, Some(&mut msg)).await?;
         }
-        ui_events::emit_chatlist_changed(context);
 
         Ok(())
     }
@@ -2671,6 +2657,11 @@ pub async fn send_msg(context: &Context, chat_id: ChatId, msg: &mut Message) ->
         return send_msg_inner(context, chat_id, msg).await;
     }
 
+    if msg.state != MessageState::Undefined && msg.state != MessageState::OutPreparing {
+        msg.param.remove(Param::GuaranteeE2ee);
+        msg.param.remove(Param::ForcePlaintext);
+        msg.update_param(context).await?;
+    }
     send_msg_inner(context, chat_id, msg).await
 }
 
@@ -2760,10 +2751,18 @@ pub(crate) async fn create_send_msg_jobs(context: &Context, msg: &mut Message) -
     let from = context.get_primary_self_addr().await?;
     let lowercase_from = from.to_lowercase();
 
-    // Send BCC to self if it is enabled and we are not going to
-    // delete it immediately. `from` must be the last addr, see `receive_imf_inner()` why.
+    // Send BCC to self if it is enabled.
+    //
+    // Previous versions of Delta Chat did not send BCC self
+    // if DeleteServerAfter was set to immediately delete messages
+    // from the server. This is not the case anymore
+    // because BCC-self messages are also used to detect
+    // that message was sent if SMTP server is slow to respond
+    // and connection is frequently lost
+    // before receiving status line.
+    //
+    // `from` must be the last addr, see `receive_imf_inner()` why.
     if context.get_config_bool(Config::BccSelf).await?
-        && context.get_config_delete_server_after().await? != Some(0)
         && !recipients
             .iter()
             .any(|x| x.to_lowercase() == lowercase_from)
@@ -3094,9 +3093,7 @@ pub async fn marknoticed_chat(context: &Context, chat_id: ChatId) -> Result<()>
             .await?;
         for chat_id_in_archive in chat_ids_in_archive {
             context.emit_event(EventType::MsgsNoticed(chat_id_in_archive));
-            ui_events::emit_chatlist_item_changed(context, chat_id_in_archive);
         }
-        ui_events::emit_chatlist_item_changed(context, DC_CHAT_ID_ARCHIVED_LINK);
     } else {
         let exists = context
             .sql
@@ -3123,7 +3120,6 @@ pub async fn marknoticed_chat(context: &Context, chat_id: ChatId) -> Result<()>
     }
 
     context.emit_event(EventType::MsgsNoticed(chat_id));
-    ui_events::emit_chatlist_item_changed(context, chat_id);
 
     Ok(())
 }
@@ -3191,7 +3187,6 @@ pub(crate) async fn mark_old_messages_as_noticed(
 
     for c in changed_chats {
         context.emit_event(EventType::MsgsNoticed(c));
-        ui_events::emit_chatlist_item_changed(context, c);
     }
 
     Ok(())
@@ -3354,8 +3349,6 @@ pub async fn create_group_chat(
     }
 
     context.emit_msgs_changed_without_ids();
-    ui_events::emit_chatlist_changed(context);
-    ui_events::emit_chatlist_item_changed(context, chat_id);
 
     if protect == ProtectionStatus::Protected {
         chat_id
@@ -3443,14 +3436,11 @@ pub(crate) async fn create_broadcast_list_ex(
     let chat_id = ChatId::new(u32::try_from(row_id)?);
 
     context.emit_msgs_changed_without_ids();
-    ui_events::emit_chatlist_changed(context);
-
     if sync.into() {
         let id = SyncId::Grpid(grpid);
         let action = SyncAction::CreateBroadcast(chat_name);
         self::sync(context, id, action).await.log_err(context).ok();
     }
-
     Ok(chat_id)
 }
 
@@ -3721,7 +3711,6 @@ pub(crate) async fn set_muted_ex(
         .await
         .context(format!("Failed to set mute duration for {chat_id}"))?;
     context.emit_event(EventType::ChatModified(chat_id));
-    ui_events::emit_chatlist_item_changed(context, chat_id);
     if sync.into() {
         let chat = Chat::load_from_db(context, chat_id).await?;
         chat.sync(context, SyncAction::SetMuted(duration))
@@ -3882,7 +3871,6 @@ async fn rename_ex(
                 sync = Nosync;
             }
             context.emit_event(EventType::ChatModified(chat_id));
-            ui_events::emit_chatlist_item_changed(context, chat_id);
             success = true;
         }
     }
@@ -3943,7 +3931,6 @@ pub async fn set_chat_profile_image(
         context.emit_msgs_changed(chat_id, msg.id);
     }
     context.emit_event(EventType::ChatModified(chat_id));
-    ui_events::emit_chatlist_item_changed(context, chat_id);
     Ok(())
 }
 
@@ -4089,8 +4076,6 @@ pub async fn resend_msgs(context: &Context, msg_ids: &[MsgId]) -> Result<()> {
             chat_id: msg.chat_id,
             msg_id: msg.id,
         });
-        // note(treefit): only matters if it is the last message in chat (but probably to expensive to check, debounce also solves it)
-        ui_events::emit_chatlist_item_changed(context, msg.chat_id);
         if !create_send_msg_jobs(context, &mut msg).await?.is_empty() {
             context.scheduler.interrupt_smtp().await;
         }

src/config.rs

@@ -7,7 +7,7 @@ use std::str::FromStr;
 use anyhow::{ensure, Context as _, Result};
 use serde::{Deserialize, Serialize};
 use strum::{EnumProperty, IntoEnumIterator};
-use strum_macros::{AsRefStr, Display, EnumIter, EnumProperty, EnumString};
+use strum_macros::{AsRefStr, Display, EnumIter, EnumString};
 
 use crate::blob::BlobObject;
 use crate::constants::DC_VERSION_STR;
@@ -347,6 +347,9 @@ pub enum Config {
     /// Row ID of the key in the `keypairs` table
     /// used for signatures, encryption to self and included in `Autocrypt` header.
     KeyId,
+
+    /// Iroh secret key.
+    IrohSecretKey,
 }
 
 impl Config {
@@ -564,6 +567,7 @@ impl Context {
         if sync != Sync {
             return Ok(());
         }
+        self.emit_event(EventType::ConfigSynced { key });
         let Some(val) = value else {
             return Ok(());
         };
@@ -866,10 +870,43 @@ mod tests {
         // Reset to default. Test that it's not synced because defaults may differ across client
         // versions.
         alice0.set_config(Config::MdnsEnabled, None).await?;
-        assert!(alice0.get_config_bool(Config::MdnsEnabled).await?);
+        assert_eq!(alice0.get_config_bool(Config::MdnsEnabled).await?, true);
+        alice0
+            .evtracker
+            .get_matching(|e| {
+                matches!(
+                    e,
+                    EventType::ConfigSynced {
+                        key: Config::MdnsEnabled
+                    }
+                )
+            })
+            .await;
         alice0.set_config_bool(Config::MdnsEnabled, false).await?;
+        alice0
+            .evtracker
+            .get_matching(|e| {
+                matches!(
+                    e,
+                    EventType::ConfigSynced {
+                        key: Config::MdnsEnabled
+                    }
+                )
+            })
+            .await;
         sync(&alice0, &alice1).await;
-        assert!(!alice1.get_config_bool(Config::MdnsEnabled).await?);
+        assert_eq!(alice1.get_config_bool(Config::MdnsEnabled).await?, false);
+        alice1
+            .evtracker
+            .get_matching(|e| {
+                matches!(
+                    e,
+                    EventType::ConfigSynced {
+                        key: Config::MdnsEnabled
+                    }
+                )
+            })
+            .await;
 
         let show_emails = alice0.get_config_bool(Config::ShowEmails).await?;
         alice0

src/configure/server_params.rs

@@ -137,20 +137,11 @@ impl ServerParams {
     }
 
     fn expand_strict_tls(self) -> Vec<ServerParams> {
-        if self.strict_tls.is_none() {
-            vec![
-                Self {
-                    strict_tls: Some(true), // Strict.
-                    ..self.clone()
-                },
-                Self {
-                    strict_tls: None, // Automatic.
-                    ..self
-                },
-            ]
-        } else {
-            vec![self]
-        }
+        vec![Self {
+            // Strict if not set by the user or provider database.
+            strict_tls: Some(self.strict_tls.unwrap_or(true)),
+            ..self
+        }]
     }
 }
 
@@ -162,31 +153,10 @@ pub(crate) fn expand_param_vector(
     domain: &str,
 ) -> Vec<ServerParams> {
     v.into_iter()
-        .map(|params| {
-            if params.socket == Socket::Plain {
-                ServerParams {
-                    // Avoid expanding plaintext configuration into configuration with and without
-                    // `strict_tls` if `strict_tls` is set to `None` as `strict_tls` is not used for
-                    // plaintext connections. Always setting it to "enabled", just in case.
-                    strict_tls: Some(true),
-                    ..params
-                }
-            } else {
-                params
-            }
-        })
         // The order of expansion is important.
         //
         // Ports are expanded the last, so they are changed the first.  Username is only changed if
         // default value (address with domain) didn't work for all available hosts and ports.
-        //
-        // Strict TLS must be expanded first, so we try all configurations with strict TLS first
-        // and only then try again without strict TLS. Otherwise we may lock to wrong hostname
-        // without strict TLS when another hostname with strict TLS is available.  For example, if
-        // both smtp.example.net and mail.example.net are running an SMTP server, but both use a
-        // certificate that is only valid for mail.example.net, we want to skip smtp.example.net
-        // and use mail.example.net with strict TLS instead of using smtp.example.net without
-        // strict TLS.
         .flat_map(|params| params.expand_strict_tls().into_iter())
         .flat_map(|params| params.expand_usernames(addr).into_iter())
         .flat_map(|params| params.expand_hostnames(domain).into_iter())
@@ -257,22 +227,6 @@ mod tests {
                     username: "foobar".to_string(),
                     strict_tls: Some(true)
                 },
-                ServerParams {
-                    protocol: Protocol::Smtp,
-                    hostname: "example.net".to_string(),
-                    port: 123,
-                    socket: Socket::Ssl,
-                    username: "foobar".to_string(),
-                    strict_tls: None,
-                },
-                ServerParams {
-                    protocol: Protocol::Smtp,
-                    hostname: "example.net".to_string(),
-                    port: 123,
-                    socket: Socket::Starttls,
-                    username: "foobar".to_string(),
-                    strict_tls: None
-                }
             ],
         );
 
@@ -284,7 +238,7 @@ mod tests {
                 port: 123,
                 socket: Socket::Plain,
                 username: "foobar".to_string(),
-                strict_tls: None,
+                strict_tls: Some(true),
             }],
             "foobar@example.net",
             "example.net",

src/constants.rs

@@ -5,7 +5,6 @@
 use deltachat_derive::{FromSql, ToSql};
 use once_cell::sync::Lazy;
 use serde::{Deserialize, Serialize};
-use tokio::time::Duration;
 
 use crate::chat::ChatId;
 
@@ -215,11 +214,6 @@ pub(crate) const DC_FOLDERS_CONFIGURED_VERSION: i32 = 4;
 // `max_smtp_rcpt_to` in the provider db.
 pub(crate) const DEFAULT_MAX_SMTP_RCPT_TO: usize = 50;
 
-
-/// How often UI events should be sent out / How much they should be debounced.
-/// Defines the tick rate/delay of the debounce loop for UI events in milliseconds.
-pub(crate) const UI_EVENTS_TICK_RATE: Duration = Duration::from_millis(50); // 50ms which means 20 fps
-
 #[cfg(test)]
 mod tests {
     use num_traits::FromPrimitive;

src/contact.rs

@@ -38,7 +38,7 @@ use crate::tools::{
     duration_to_str, get_abs_path, improve_single_line_input, strip_rtlo_characters, time,
     EmailAddress,
 };
-use crate::{chat, stock_str, ui_events};
+use crate::{chat, stock_str};
 
 /// Time during which a contact is considered as seen recently.
 const SEEN_RECENTLY_SECONDS: i64 = 600;
@@ -761,7 +761,6 @@ impl Contact {
                             if count > 0 {
                                 // Chat name updated
                                 context.emit_event(EventType::ChatModified(chat_id));
-                                ui_events::emit_chatlist_items_changed_for_contact(context, contact_id);
                             }
                         }
                     }
@@ -798,31 +797,7 @@ impl Contact {
             Ok(row_id)
         }).await?;
 
-        let contact_id = ContactId::new(row_id);
-
-        Ok((contact_id, sth_modified))
-    }
-
-    /// Get all chats the contact is part of
-    pub async fn get_chats_with_contact(
-        context: &Context,
-        contact_id: &ContactId,
-    ) -> Result<Vec<ChatId>> {
-        context
-            .sql
-            .query_map(
-                "SELECT chat_id FROM chats_contacts WHERE contact_id=?",
-                (contact_id,),
-                |row| {
-                    let chat_id: ChatId = row.get(0)?;
-                    Ok(chat_id)
-                },
-                |rows| {
-                    rows.collect::<std::result::Result<Vec<_>, _>>()
-                        .map_err(Into::into)
-                },
-            )
-            .await
+        Ok((ContactId::new(row_id), sth_modified))
     }
 
     /// Add a number of contacts.
@@ -1553,7 +1528,6 @@ WHERE type=? AND id IN (
         }
     }
 
-    ui_events::emit_chatlist_changed(context);
     Ok(())
 }
 
@@ -1602,7 +1576,6 @@ pub(crate) async fn set_profile_image(
     if changed {
         contact.update_param(context).await?;
         context.emit_event(EventType::ContactsChanged(Some(contact_id)));
-        ui_events::emit_chatlist_item_changed_for_contacts_dm_chat(context, contact_id);
     }
     Ok(())
 }
@@ -1809,10 +1782,6 @@ impl RecentlySeenLoop {
                         // Timeout, notify about contact.
                         if let Some(contact_id) = contact_id {
                             context.emit_event(EventType::ContactsChanged(Some(*contact_id)));
-                            ui_events::emit_chatlist_item_changed_for_contacts_dm_chat(
-                                &context,
-                                *contact_id,
-                            );
                             unseen_queue.pop();
                         }
                     }
@@ -1842,10 +1811,6 @@ impl RecentlySeenLoop {
                 // Event is already in the past.
                 if let Some(contact_id) = contact_id {
                     context.emit_event(EventType::ContactsChanged(Some(*contact_id)));
-                    ui_events::emit_chatlist_item_changed_for_contacts_dm_chat(
-                        &context,
-                        *contact_id,
-                    );
                 }
                 unseen_queue.pop();
             }

src/context.rs

@@ -1,6 +1,5 @@
 //! Context module.
 
-use std::borrow::BorrowMut;
 use std::collections::{BTreeMap, HashMap};
 use std::ffi::OsString;
 use std::ops::Deref;
@@ -11,6 +10,8 @@ use std::time::{Duration, Instant, SystemTime};
 
 use anyhow::{bail, ensure, Context as _, Result};
 use async_channel::{self as channel, Receiver, Sender};
+use iroh_gossip::net::Gossip;
+use iroh_net::MagicEndpoint;
 use ratelimit::Ratelimit;
 use tokio::sync::{Mutex, Notify, RwLock};
 
@@ -29,7 +30,6 @@ use crate::sql::Sql;
 use crate::stock_str::StockStrings;
 use crate::timesmearing::SmearedTimestamp;
 use crate::tools::{duration_to_str, time};
-use crate::ui_events::{self, UIEvents};
 
 /// Builder for the [`Context`].
 ///
@@ -205,7 +205,6 @@ pub struct InnerContext {
     pub(crate) wrong_pw_warning_mutex: Mutex<()>,
     pub(crate) translated_stockstrings: StockStrings,
     pub(crate) events: Events,
-    pub(crate) ui_events: Mutex<UIEvents>,
 
     pub(crate) scheduler: SchedulerState,
     pub(crate) ratelimit: RwLock<Ratelimit>,
@@ -247,6 +246,12 @@ pub struct InnerContext {
     /// Standard RwLock instead of [`tokio::sync::RwLock`] is used
     /// because the lock is used from synchronous [`Context::emit_event`].
     pub(crate) debug_logging: std::sync::RwLock<Option<DebugLogging>>,
+
+    /// [MagicEndpoint] needed for iroh peer channels.
+    pub(crate) endpoint: Mutex<Option<MagicEndpoint>>,
+
+    /// [Gossip] needed for iroh peer channels.
+    pub(crate) gossip: Mutex<Option<Gossip>>,
 }
 
 /// The state of ongoing process.
@@ -370,8 +375,6 @@ impl Context {
         // without starting I/O.
         new_msgs_notify.notify_one();
 
-        let (ui_events, ui_events_receiver) = UIEvents::new();
-
         let inner = InnerContext {
             id,
             blobdir,
@@ -383,7 +386,6 @@ impl Context {
             wrong_pw_warning_mutex: Mutex::new(()),
             translated_stockstrings: stockstrings,
             events,
-            ui_events: Mutex::new(ui_events),
             scheduler: SchedulerState::new(),
             ratelimit: RwLock::new(Ratelimit::new(Duration::new(60, 0), 6.0)), // Allow at least 1 message every 10 seconds + a burst of 6.
             quota: RwLock::new(None),
@@ -394,14 +396,14 @@ impl Context {
             last_full_folder_scan: Mutex::new(None),
             last_error: std::sync::RwLock::new("".to_string()),
             debug_logging: std::sync::RwLock::new(None),
+            endpoint: Mutex::new(None),
+            gossip: Mutex::new(None),
         };
 
         let ctx = Context {
             inner: Arc::new(inner),
         };
 
-        ctx.inner.ui_events.blocking_lock().start(&ctx, ui_events_receiver);
-
         Ok(ctx)
     }
 
@@ -425,11 +427,17 @@ impl Context {
                 *lock = Ratelimit::new(Duration::new(3, 0), 3.0);
             }
         }
+
+        if let Err(e) = self.create_gossip().await {
+            warn!(self, "{e}");
+        }
         self.scheduler.start(self.clone()).await;
     }
 
     /// Stops the IO scheduler.
     pub async fn stop_io(&self) {
+        self.endpoint.lock().await.take();
+        self.gossip.lock().await.take();
         self.scheduler.stop(self).await;
     }
 
@@ -441,6 +449,9 @@ impl Context {
 
     /// Indicate that the network likely has come back.
     pub async fn maybe_network(&self) {
+        if let Some(ref mut endpoint) = *self.endpoint.lock().await {
+            endpoint.network_change().await;
+        }
         self.scheduler.maybe_network().await;
     }
 
@@ -493,15 +504,11 @@ impl Context {
     /// Emits a MsgsChanged event with specified chat and message ids
     pub fn emit_msgs_changed(&self, chat_id: ChatId, msg_id: MsgId) {
         self.emit_event(EventType::MsgsChanged { chat_id, msg_id });
-        ui_events::emit_chatlist_changed(self);
-        ui_events::emit_chatlist_item_changed(self, chat_id);
     }
 
     /// Emits an IncomingMsg event with specified chat and message ids
     pub fn emit_incoming_msg(&self, chat_id: ChatId, msg_id: MsgId) {
         self.emit_event(EventType::IncomingMsg { chat_id, msg_id });
-        ui_events::emit_chatlist_changed(self);
-        ui_events::emit_chatlist_item_changed(self, chat_id);
     }
 
     /// Returns a receiver for emitted events.
@@ -1331,6 +1338,7 @@ mod tests {
             "socks5_user",
             "socks5_password",
             "key_id",
+            "iroh_secret_key",
         ];
         let t = TestContext::new().await;
         let info = t.get_info().await.unwrap();
@@ -1617,4 +1625,15 @@ mod tests {
 
         Ok(())
     }
+
+    #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
+    async fn test_keypair_saving() -> Result<()> {
+        let alice = TestContext::new_alice().await;
+
+        let key = alice.get_or_generate_iroh_keypair().await?;
+        let loaded_key = alice.get_or_generate_iroh_keypair().await?;
+        assert_eq!(key.to_bytes(), loaded_key.to_bytes());
+
+        Ok(())
+    }
 }

src/debug_logging.rs

@@ -63,6 +63,7 @@ pub async fn debug_logging_loop(context: &Context, events: Receiver<DebugEventLo
                     summary: None,
                     document: None,
                     uid: None,
+                    gossip_topic: None,
                 },
             )
             .await
@@ -72,12 +73,10 @@ pub async fn debug_logging_loop(context: &Context, events: Receiver<DebugEventLo
             }
             Ok(serial) => {
                 if let Some(serial) = serial {
-                    if !matches!(event, EventType::WebxdcStatusUpdate { .. }) {
-                        context.emit_event(EventType::WebxdcStatusUpdate {
-                            msg_id,
-                            status_update_serial: serial,
-                        });
-                    }
+                    context.emit_event(EventType::WebxdcStatusUpdate {
+                        msg_id,
+                        status_update_serial: serial,
+                    });
                 } else {
                     // This should not happen as the update has no `uid`.
                     error!(context, "Debug logging update is not created.");

src/download.rs

@@ -13,7 +13,7 @@ use crate::imap::{Imap, ImapActionResult};
 use crate::message::{Message, MsgId, Viewtype};
 use crate::mimeparser::{MimeMessage, Part};
 use crate::tools::time;
-use crate::{stock_str, ui_events, EventType};
+use crate::{stock_str, EventType};
 
 /// Download limits should not be used below `MIN_DOWNLOAD_LIMIT`.
 ///
@@ -115,7 +115,6 @@ impl MsgId {
             chat_id: msg.chat_id,
             msg_id: self,
         });
-        ui_events::emit_chatlist_item_changed(context, msg.chat_id);
         Ok(())
     }
 }
@@ -147,29 +146,19 @@ pub(crate) async fn download_msg(context: &Context, msg_id: MsgId, imap: &mut Im
         )
         .await?;
 
-    if let Some((server_uid, server_folder)) = row {
-        match imap
-            .fetch_single_msg(context, &server_folder, server_uid, msg.rfc724_mid.clone())
-            .await
-        {
-            ImapActionResult::RetryLater | ImapActionResult::Failed => {
-                msg.id
-                    .update_download_state(context, DownloadState::Failure)
-                    .await?;
-                Err(anyhow!("Call download_full() again to try over."))
-            }
-            ImapActionResult::Success => {
-                // update_download_state() not needed as receive_imf() already
-                // set the state and emitted the event.
-                Ok(())
-            }
-        }
-    } else {
+    let Some((server_uid, server_folder)) = row else {
         // No IMAP record found, we don't know the UID and folder.
-        msg.id
-            .update_download_state(context, DownloadState::Failure)
-            .await?;
-        Err(anyhow!("Call download_full() again to try over."))
+        return Err(anyhow!("Call download_full() again to try over."));
+    };
+
+    match imap
+        .fetch_single_msg(context, &server_folder, server_uid, msg.rfc724_mid.clone())
+        .await
+    {
+        ImapActionResult::RetryLater | ImapActionResult::Failed => {
+            Err(anyhow!("Call download_full() again to try over."))
+        }
+        ImapActionResult::Success => Ok(()),
     }
 }
 

src/e2ee.rs

@@ -52,7 +52,7 @@ impl EncryptHelper {
         &self,
         context: &Context,
         e2ee_guaranteed: bool,
-        peerstates: &[(Option<Peerstate>, &str)],
+        peerstates: &[(Option<Peerstate>, String)],
     ) -> Result<bool> {
         let mut prefer_encrypt_count = if self.prefer_encrypt == EncryptPreference::Mutual {
             1
@@ -94,7 +94,7 @@ impl EncryptHelper {
         context: &Context,
         verified: bool,
         mail_to_encrypt: lettre_email::PartBuilder,
-        peerstates: Vec<(Option<Peerstate>, &str)>,
+        peerstates: Vec<(Option<Peerstate>, String)>,
     ) -> Result<String> {
         let mut keyring: Vec<SignedPublicKey> = Vec::new();
 
@@ -117,7 +117,7 @@ impl EncryptHelper {
         // Encrypt to secondary verified keys
         // if we also encrypt to the introducer ("verifier") of the key.
         if verified {
-            for (peerstate, _addr) in peerstates {
+            for (peerstate, _addr) in &peerstates {
                 if let Some(peerstate) = peerstate {
                     if let (Some(key), Some(verifier)) = (
                         peerstate.secondary_verified_key.as_ref(),
@@ -293,7 +293,7 @@ Sent with my Delta Chat Messenger: https://delta.chat";
         Ok(())
     }
 
-    fn new_peerstates(prefer_encrypt: EncryptPreference) -> Vec<(Option<Peerstate>, &'static str)> {
+    fn new_peerstates(prefer_encrypt: EncryptPreference) -> Vec<(Option<Peerstate>, String)> {
         let addr = "bob@foo.bar";
         let pub_key = bob_keypair().public;
         let peerstate = Peerstate {
@@ -315,7 +315,7 @@ Sent with my Delta Chat Messenger: https://delta.chat";
             backward_verified_key_id: None,
             fingerprint_changed: false,
         };
-        vec![(Some(peerstate), addr)]
+        vec![(Some(peerstate), addr.to_string())]
     }
 
     #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
@@ -340,7 +340,7 @@ Sent with my Delta Chat Messenger: https://delta.chat";
         assert!(encrypt_helper.should_encrypt(&t, false, &ps).unwrap());
 
         // test with missing peerstate
-        let ps = vec![(None, "bob@foo.bar")];
+        let ps = vec![(None, "bob@foo.bar".to_string())];
         assert!(encrypt_helper.should_encrypt(&t, true, &ps).is_err());
         assert!(!encrypt_helper.should_encrypt(&t, false, &ps).unwrap());
     }

src/events.rs

@@ -4,7 +4,6 @@ use async_channel::{self as channel, Receiver, Sender, TrySendError};
 use pin_project::pin_project;
 
 mod payload;
-pub(crate) mod ui_events;
 
 pub use self::payload::EventType;
 

src/events/payload.rs

@@ -4,6 +4,7 @@ use serde::{Deserialize, Serialize};
 use std::path::PathBuf;
 
 use crate::chat::ChatId;
+use crate::config::Config;
 use crate::contact::ContactId;
 use crate::ephemeral::Timer as EphemeralTimer;
 use crate::message::MsgId;
@@ -261,8 +262,17 @@ pub enum EventType {
     ConnectivityChanged,
 
     /// The user's avatar changed.
+    /// Deprecated by `ConfigSynced`.
     SelfavatarChanged,
 
+    /// A multi-device synced config value changed. Maybe the app needs to refresh smth. For
+    /// uniformity this is emitted on the source device too. The value isn't here, otherwise it
+    /// would be logged which might not be good for privacy.
+    ConfigSynced {
+        /// Configuration key.
+        key: Config,
+    },
+
     /// Webxdc status update received.
     WebxdcStatusUpdate {
         /// Message ID.
@@ -277,16 +287,4 @@ pub enum EventType {
         /// ID of the deleted message.
         msg_id: MsgId,
     },
-
-    /// Inform UI that Order (and content as in chat ids) of the chatlist changed.
-    ///
-    /// Sometimes this is emitted together with `UIChatListItemChanged` such as on IncomingMessage.
-    UIChatListChanged,
-
-    /// Inform UI that a single chat list item changed and needs to be rerendered
-    /// If `chat_id` is set to None, then all currently visible chats need to be rerendered, and all not-visible items need to be cleared from cache if the UI has a cache.
-    UIChatListItemChanged {
-        /// ID of the changed chat
-        chat_id: Option<ChatId>,
-    },
 }

src/events/ui_events.rs

@@ -1,253 +0,0 @@
-use crate::{
-    chat::{ChatId, ChatIdBlocked},
-    constants::UI_EVENTS_TICK_RATE,
-    contact::{Contact, ContactId},
-    context::Context,
-    EventType,
-};
-use async_channel::{self as channel, Receiver, Sender};
-use tokio::{
-    task,
-    time::{sleep_until, Instant},
-};
-
-/// order or content of chatlist changes (chat ids, not the actual chatlist item)
-pub(crate) fn emit_chatlist_changed(context: &Context) {
-    context
-        .ui_events
-        .blocking_lock()
-        .send_chat_list_event(context, InternalUIEvent::ChatListChanged)
-}
-
-/// Chatlist item of a specific chat changed
-pub(crate) fn emit_chatlist_item_changed(context: &Context, chat_id: ChatId) {
-    context
-        .ui_events
-        .blocking_lock()
-        .send_chat_list_event(context, InternalUIEvent::ChatListItemChanged(chat_id))
-}
-
-#[allow(unused)]
-/// Used when you don't know which chatlist items changed, this reloads all cached chatlist items in the UI
-/// note(treefit): This is not used right now, but I know there will be a point where someone wants it
-pub(crate) fn emit_unknown_chatlist_items_changed(context: &Context) {
-    context
-        .ui_events
-        .blocking_lock()
-        .send_chat_list_event(context, InternalUIEvent::UnknownChatListItemsChanged)
-}
-
-/// update event for dm chat of contact
-/// used when recently seen changes and when profile image changes
-pub(crate) fn emit_chatlist_item_changed_for_contacts_dm_chat(
-    context: &Context,
-    contact_id: ContactId,
-) {
-    context
-        .ui_events
-        .blocking_lock()
-        .send_chat_list_event(context, InternalUIEvent::ContactDMChatChanged(contact_id))
-}
-
-/// update dm for chats that have the contact
-/// used when contact changes their name or did AEAP for example
-pub(crate) fn emit_chatlist_items_changed_for_contact(context: &Context, contact_id: ContactId) {
-    context
-        .ui_events
-        .blocking_lock()
-        .send_chat_list_event(context, InternalUIEvent::ContactChatsChanged(contact_id));
-}
-
-#[derive(Debug)]
-pub(crate) enum InternalUIEvent {
-    ChatListChanged,
-    ChatListItemChanged(ChatId),
-    UnknownChatListItemsChanged,
-    ContactDMChatChanged(ContactId),
-    ContactChatsChanged(ContactId),
-}
-
-struct EventLoopTickState {
-    chat_list_changed: bool,
-    has_unknown_items: bool,
-    chat_ids: Vec<ChatId>,
-    contact_ids_dm: Vec<ContactId>,
-    contact_ids_chats: Vec<ContactId>,
-}
-
-impl EventLoopTickState {
-    fn new(capacity: usize) -> Self {
-        Self {
-            chat_list_changed: false,
-            has_unknown_items: false,
-            chat_ids: Vec::with_capacity(capacity),
-            contact_ids_dm: Vec::with_capacity(capacity),
-            contact_ids_chats: Vec::with_capacity(capacity),
-        }
-    }
-
-    fn apply_internal_ui_event(&mut self, event: InternalUIEvent) {
-        match event {
-            InternalUIEvent::ChatListChanged => {
-                self.chat_list_changed = true;
-            }
-            InternalUIEvent::ChatListItemChanged(chat_id) => {
-                self.chat_ids.push(chat_id);
-            }
-            InternalUIEvent::UnknownChatListItemsChanged => {
-                self.has_unknown_items = true;
-            }
-            InternalUIEvent::ContactDMChatChanged(contact_id) => {
-                self.contact_ids_dm.push(contact_id);
-            }
-            InternalUIEvent::ContactChatsChanged(contact_id) => {
-                self.contact_ids_chats.push(contact_id);
-            }
-        }
-    }
-
-    async fn emit_chatlist_ui_events(&mut self, context: &Context) {
-        if self.chat_list_changed {
-            context.emit_event(EventType::UIChatListChanged);
-        }
-        if self.has_unknown_items {
-            context.emit_event(EventType::UIChatListItemChanged { chat_id: None });
-            return; // since this refreshes everything no further events are needed
-        }
-
-        for contact_id in self
-            .contact_ids_dm
-            .iter()
-            .filter(|contact| !self.contact_ids_chats.contains(contact))
-            .collect::<Vec<&ContactId>>()
-        {
-            if let Ok(Some(chat_id)) = ChatIdBlocked::lookup_by_contact(context, *contact_id).await
-            {
-                self.chat_ids.push(chat_id.id)
-            }
-        }
-
-        // note:(treefit): could make sense to only update chats where the last message is from the contact, but the db query for that is more expensive
-        for contact_id in &self.contact_ids_chats {
-            match Contact::get_chats_with_contact(context, contact_id).await {
-                Ok(contacts_chat_ids) => {
-                    self.chat_ids.extend(contacts_chat_ids);
-                }
-                Err(err) => {
-                    warn!(
-                        context,
-                        "Error while getting chats for contact {} in chatlist events loop: {}",
-                        contact_id,
-                        err
-                    );
-                }
-            }
-        }
-
-        self.chat_ids.sort();
-        self.chat_ids.dedup();
-
-        // TODO change event so it accepts a list of chat ids to get rid of this loop? wouldn't work with cffi unless we give it out as json
-        for chat_id in &self.chat_ids {
-            context.emit_event(EventType::UIChatListItemChanged {
-                chat_id: Some(*chat_id),
-            })
-        }
-    }
-}
-
-/// Debounces UI events
-#[derive(Debug)]
-pub(crate) struct UIEvents {
-    task_handle: Option<task::JoinHandle<()>>,
-    chatlist_event_queue: Sender<InternalUIEvent>,
-}
-
-impl UIEvents {
-    pub(crate) fn new() -> (Self, Receiver<InternalUIEvent>) {
-        let (chatlist_event_queue, chatlist_event_queue_recv) = channel::unbounded();
-        (
-            Self {
-                task_handle: None,
-                chatlist_event_queue,
-            },
-            chatlist_event_queue_recv,
-        )
-    }
-
-    pub(crate) fn start(
-        &mut self,
-        context: &Context,
-        chatlist_event_queue_recv: Receiver<InternalUIEvent>,
-    ) {
-        if let Some(handle) = self.task_handle {
-            handle.abort()
-        }
-        self.task_handle = Some(task::spawn(Self::run_task(
-            context,
-            chatlist_event_queue_recv,
-        )))
-    }
-
-    async fn run_task(context: &Context, chatlist_event_queue: Receiver<InternalUIEvent>) {
-        loop {
-            match chatlist_event_queue.recv().await {
-                Ok(chatlist_event) => {
-                    let backlog_len = chatlist_event_queue.len();
-                    let mut tick_state = EventLoopTickState::new(backlog_len);
-
-                    tick_state.apply_internal_ui_event(chatlist_event);
-                    // get all events from the queue
-                    while let Ok(event) = chatlist_event_queue.try_recv() {
-                        tick_state.apply_internal_ui_event(event);
-                    }
-
-                    tick_state.emit_chatlist_ui_events(context).await;
-
-                    // cooldown
-                    sleep_until(Instant::now() + UI_EVENTS_TICK_RATE).await;
-                }
-                Err(err) => {
-                    warn!(
-                        context,
-                        "Error receiving an interruption in ui chatlist events loop: {}", err
-                    );
-                    // Maybe the sender side is closed, so terminate the loop to avoid looping indefinitely.
-                    return;
-                }
-            }
-        }
-    }
-
-    pub(crate) fn send_chat_list_event(&self, context: &Context, event: InternalUIEvent) {
-        // todo check if ui events are enabled?
-        if let Err(error) = self.chatlist_event_queue.try_send(event) {
-            warn!(
-                context,
-                "Error receiving an interruption in ui chatlist events loop: {}", error
-            );
-        }
-    }
-}
-
-impl Drop for UIEvents {
-    fn drop(&mut self) {
-        if let Some(handle) = &self.task_handle {
-            handle.abort()
-        }
-    }
-}
-
-#[cfg(test)]
-mod test {
-
-    // todo tests:
-
-    // send ui events though the UIEventsLoop
-
-    // check that UIEventsLoop really ratelimits the events
-
-    // check that has_unknown_items does not send out any ids before or afterwards
-
-    // if we should make it possible to disable via config then test that as well
-}

src/headerdef.rs

@@ -88,6 +88,9 @@ pub enum HeaderDef {
     /// See <https://datatracker.ietf.org/doc/html/rfc8601>
     AuthenticationResults,
 
+    /// Public key to join gossip network.
+    IrohPublicGossip,
+
     #[cfg(test)]
     TestHeader,
 }

src/imap.rs

@@ -42,7 +42,6 @@ use crate::socks::Socks5Config;
 use crate::sql;
 use crate::stock_str;
 use crate::tools::{create_id, duration_to_str};
-use crate::ui_events;
 
 pub(crate) mod capabilities;
 mod client;
@@ -1320,7 +1319,6 @@ impl Imap {
             .with_context(|| format!("failed to set MODSEQ for folder {folder}"))?;
         for updated_chat_id in updated_chat_ids {
             context.emit_event(EventType::MsgsNoticed(updated_chat_id));
-            ui_events::emit_chatlist_item_changed(context, updated_chat_id);
         }
 
         Ok(())

src/lib.rs

@@ -105,6 +105,7 @@ pub mod receive_imf;
 pub mod tools;
 
 pub mod accounts;
+pub mod peer_channels;
 pub mod reaction;
 
 /// If set IMAP/incoming and SMTP/outgoing MIME messages will be printed.

src/location.rs

@@ -14,8 +14,8 @@ use crate::context::Context;
 use crate::events::EventType;
 use crate::message::{Message, MsgId, Viewtype};
 use crate::mimeparser::SystemMessage;
+use crate::stock_str;
 use crate::tools::{duration_to_str, time};
-use crate::{stock_str, ui_events};
 
 /// Location record.
 #[derive(Debug, Clone, Default)]
@@ -290,7 +290,6 @@ pub async fn send_locations_to_chat(
         chat::add_info_msg(context, chat_id, &stock_str, now).await?;
     }
     context.emit_event(EventType::ChatModified(chat_id));
-    ui_events::emit_chatlist_item_changed(context, chat_id);
     if 0 != seconds {
         context.scheduler.interrupt_location().await;
     }
@@ -788,7 +787,6 @@ async fn maybe_send_locations(context: &Context) -> Result<Option<u64>> {
             let stock_str = stock_str::msg_location_disabled(context).await;
             chat::add_info_msg(context, chat_id, &stock_str, now).await?;
             context.emit_event(EventType::ChatModified(chat_id));
-            ui_events::emit_chatlist_item_changed(context, chat_id);
         }
     }
 

src/login_param.rs

@@ -14,8 +14,22 @@ use crate::socks::Socks5Config;
 #[repr(u32)]
 #[strum(serialize_all = "snake_case")]
 pub enum CertificateChecks {
-    /// Same as AcceptInvalidCertificates unless overridden by
-    /// `strict_tls` setting in provider database.
+    /// Same as AcceptInvalidCertificates if stored in the database
+    /// as `configured_{imap,smtp}_certificate_checks`.
+    ///
+    /// Previous Delta Chat versions stored this in `configured_*`
+    /// if Automatic configuration
+    /// was selected, configuration with strict TLS checks failed
+    /// and configuration without strict TLS checks succeeded.
+    ///
+    /// Currently Delta Chat stores only
+    /// `Strict` or `AcceptInvalidCertificates` variants
+    /// in `configured_*` settings.
+    ///
+    /// `Automatic` in `{imap,smtp}_certificate_checks`
+    /// means that provider database setting should be taken.
+    /// If there is no provider database setting for certificate checks,
+    /// `Automatic` is the same as `Strict`.
     Automatic = 0,
 
     Strict = 1,

src/message.rs

@@ -30,7 +30,6 @@ use crate::tools::{
     buf_compress, buf_decompress, get_filebytes, get_filemeta, gm2local_offset, read_file, time,
     timestamp_to_str, truncate,
 };
-use crate::ui_events;
 
 /// Message ID, including reserved IDs.
 ///
@@ -139,7 +138,6 @@ WHERE id=?;
             chat_id,
             msg_id: self,
         });
-        ui_events::emit_chatlist_item_changed(context, chat_id);
         Ok(())
     }
 
@@ -1520,12 +1518,9 @@ pub async fn delete_msgs(context: &Context, msg_ids: &[MsgId]) -> Result<()> {
 
     for modified_chat_id in modified_chat_ids {
         context.emit_msgs_changed(modified_chat_id, MsgId::new(0));
-        ui_events::emit_chatlist_item_changed(context, modified_chat_id);
     }
 
     if !msg_ids.is_empty() {
-        context.emit_msgs_changed_without_ids();
-        ui_events::emit_chatlist_changed(context);
         // Run housekeeping to delete unused blobs.
         context.set_config(Config::LastHousekeeping, None).await?;
     }
@@ -1658,7 +1653,6 @@ pub async fn markseen_msgs(context: &Context, msg_ids: Vec<MsgId>) -> Result<()>
 
     for updated_chat_id in updated_chat_ids {
         context.emit_event(EventType::MsgsNoticed(updated_chat_id));
-        ui_events::emit_chatlist_item_changed(context, updated_chat_id);
     }
 
     Ok(())
@@ -1719,7 +1713,6 @@ pub(crate) async fn set_msg_failed(
         chat_id: msg.chat_id,
         msg_id: msg.id,
     });
-    ui_events::emit_chatlist_item_changed(context, msg.chat_id);
 
     Ok(())
 }

src/mimefactory.rs

@@ -18,6 +18,7 @@ use crate::contact::Contact;
 use crate::context::Context;
 use crate::e2ee::EncryptHelper;
 use crate::ephemeral::Timer as EphemeralTimer;
+use crate::headerdef::HeaderDef;
 use crate::html::new_html_mimepart;
 use crate::location;
 use crate::message::{self, Message, MsgId, Viewtype};
@@ -277,7 +278,7 @@ impl<'a> MimeFactory<'a> {
     async fn peerstates_for_recipients(
         &self,
         context: &Context,
-    ) -> Result<Vec<(Option<Peerstate>, &str)>> {
+    ) -> Result<Vec<(Option<Peerstate>, String)>> {
         let self_addr = context.get_primary_self_addr().await?;
 
         let mut res = Vec::new();
@@ -286,7 +287,7 @@ impl<'a> MimeFactory<'a> {
             .iter()
             .filter(|(_, addr)| addr != &self_addr)
         {
-            res.push((Peerstate::from_addr(context, addr).await?, addr.as_str()));
+            res.push((Peerstate::from_addr(context, addr).await?, addr.clone()));
         }
 
         Ok(res)
@@ -351,7 +352,7 @@ impl<'a> MimeFactory<'a> {
                         .unwrap_or_default()
                 }
             }
-            Loaded::Mdn { .. } => true,
+            Loaded::Mdn { .. } => false,
         }
     }
 
@@ -917,6 +918,16 @@ impl<'a> MimeFactory<'a> {
         Ok(Some(part))
     }
 
+    fn add_message_text(&self, part: PartBuilder, mut text: String) -> PartBuilder {
+        // This is needed to protect from ESPs (such as gmx.at) doing their own Quoted-Printable
+        // encoding and thus breaking messages and signatures. It's unlikely that the reader uses a
+        // MUA not supporting Quoted-Printable encoding. And RFC 2646 "4.6" also recommends it for
+        // encrypted messages.
+        let part = part.header(("Content-Transfer-Encoding", "quoted-printable"));
+        text = quoted_printable::encode_to_str(text);
+        part.body(text)
+    }
+
     #[allow(clippy::cognitive_complexity)]
     async fn render_message(
         &mut self,
@@ -1214,13 +1225,11 @@ impl<'a> MimeFactory<'a> {
             footer
         );
 
-        // Message is sent as text/plain, with charset = utf-8
-        let mut main_part = PartBuilder::new()
-            .header((
-                "Content-Type".to_string(),
-                "text/plain; charset=utf-8; format=flowed; delsp=no".to_string(),
-            ))
-            .body(message_text);
+        let mut main_part = PartBuilder::new().header((
+            "Content-Type",
+            "text/plain; charset=utf-8; format=flowed; delsp=no",
+        ));
+        main_part = self.add_message_text(main_part, message_text);
 
         if is_reaction {
             main_part = main_part.header(("Content-Disposition", "reaction"));
@@ -1266,6 +1275,7 @@ impl<'a> MimeFactory<'a> {
             }
         }
 
+        println!("hiiiii");
         // we do not piggyback sync-files to other self-sent-messages
         // to not risk files becoming too larger and being skipped by download-on-demand.
         if command == SystemMessage::MultiDeviceSync && self.is_e2ee_guaranteed() {
@@ -1275,6 +1285,16 @@ impl<'a> MimeFactory<'a> {
             self.sync_ids_to_delete = Some(ids.to_string());
         } else if command == SystemMessage::WebxdcStatusUpdate {
             let json = self.msg.param.get(Param::Arg).unwrap_or_default();
+            if json.find("gossip_topic").is_some() {
+                if let Some(ref endpoint) = *context.endpoint.lock().await {
+                    // Add iroh NodeAddr to headers so peers can connect to us.
+                    let node_addr = endpoint.my_addr().await.unwrap();
+                    headers.protected.push(Header::new(
+                        HeaderDef::IrohPublicGossip.get_headername().to_string(),
+                        serde_json::to_string(&node_addr)?,
+                    ));
+                }
+            }
             parts.push(context.build_status_update_part(json));
         } else if self.msg.viewtype == Viewtype::Webxdc {
             if let Some(json) = context
@@ -1347,15 +1367,12 @@ impl<'a> MimeFactory<'a> {
         };
         let p2 = stock_str::read_rcpt_mail_body(context, &p1).await;
         let message_text = format!("{}\r\n", format_flowed(&p2));
-        message = message.child(
-            PartBuilder::new()
-                .header((
-                    "Content-Type".to_string(),
-                    "text/plain; charset=utf-8; format=flowed; delsp=no".to_string(),
-                ))
-                .body(message_text)
-                .build(),
-        );
+        let text_part = PartBuilder::new().header((
+            "Content-Type".to_string(),
+            "text/plain; charset=utf-8; format=flowed; delsp=no".to_string(),
+        ));
+        let text_part = self.add_message_text(text_part, message_text);
+        message = message.child(text_part.build());
 
         // second body part: machine-readable, always REQUIRED by RFC 6522
         let message_text2 = format!(
@@ -1571,6 +1588,7 @@ mod tests {
     use crate::mimeparser::MimeMessage;
     use crate::receive_imf::receive_imf;
     use crate::test_utils::{get_chat_msg, TestContext, TestContextManager};
+
     #[test]
     fn test_render_email_address() {
         let display_name = "ä space";
@@ -1826,6 +1844,37 @@ mod tests {
         assert_eq!("Re: Hello, Bob", mf.subject_str(&t).await.unwrap());
     }
 
+    #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
+    async fn test_mdn_create_encrypted() -> Result<()> {
+        let mut tcm = TestContextManager::new();
+        let alice = tcm.alice().await;
+        let bob = tcm.bob().await;
+        bob.set_config_bool(Config::MdnsEnabled, true).await?;
+
+        let mut msg = Message::new(Viewtype::Text);
+        msg.param.set_int(Param::SkipAutocrypt, 1);
+        let chat_alice = alice.create_chat(&bob).await.id;
+        let sent = alice.send_msg(chat_alice, &mut msg).await;
+
+        let rcvd = bob.recv_msg(&sent).await;
+        message::markseen_msgs(&bob, vec![rcvd.id]).await?;
+        let mimefactory = MimeFactory::from_mdn(&bob, &rcvd, vec![]).await?;
+        let rendered_msg = mimefactory.render(&bob).await?;
+
+        assert!(!rendered_msg.is_encrypted);
+
+        let rcvd = tcm.send_recv(&alice, &bob, "Heyho").await;
+        message::markseen_msgs(&bob, vec![rcvd.id]).await?;
+
+        let mimefactory = MimeFactory::from_mdn(&bob, &rcvd, vec![]).await?;
+        let rendered_msg = mimefactory.render(&bob).await?;
+
+        // When encrypted, the MDN should be encrypted as well
+        assert!(rendered_msg.is_encrypted);
+
+        Ok(())
+    }
+
     #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
     async fn test_subject_in_group() -> Result<()> {
         async fn send_msg_get_subject(
@@ -2166,6 +2215,7 @@ mod tests {
         assert_eq!(inner.match_indices("Message-ID:").count(), 1);
         assert_eq!(inner.match_indices("Chat-User-Avatar:").count(), 1);
         assert_eq!(inner.match_indices("Subject:").count(), 0);
+        assert_eq!(inner.match_indices("quoted-printable").count(), 1);
 
         assert_eq!(body.match_indices("this is the text!").count(), 1);
 
@@ -2186,6 +2236,7 @@ mod tests {
         assert_eq!(inner.match_indices("Message-ID:").count(), 1);
         assert_eq!(inner.match_indices("Chat-User-Avatar:").count(), 0);
         assert_eq!(inner.match_indices("Subject:").count(), 0);
+        assert_eq!(inner.match_indices("quoted-printable").count(), 1);
 
         assert_eq!(body.match_indices("this is the text!").count(), 1);
 
@@ -2242,6 +2293,7 @@ mod tests {
         assert_eq!(part.match_indices("Message-ID:").count(), 1);
         assert_eq!(part.match_indices("Chat-User-Avatar:").count(), 1);
         assert_eq!(part.match_indices("Subject:").count(), 0);
+        assert_eq!(part.match_indices("quoted-printable").count(), 1);
 
         let body = payload.next().unwrap();
         assert_eq!(body.match_indices("this is the text!").count(), 1);
@@ -2289,6 +2341,7 @@ mod tests {
         assert_eq!(part.match_indices("Message-ID:").count(), 1);
         assert_eq!(part.match_indices("Chat-User-Avatar:").count(), 0);
         assert_eq!(part.match_indices("Subject:").count(), 0);
+        assert_eq!(part.match_indices("quoted-printable").count(), 1);
 
         let body = payload.next().unwrap();
         assert_eq!(body.match_indices("this is the text!").count(), 1);

src/mimeparser.rs

@@ -35,12 +35,13 @@ use crate::message::{
 use crate::param::{Param, Params};
 use crate::peerstate::Peerstate;
 use crate::simplify::{simplify, SimplifiedText};
+use crate::stock_str;
 use crate::sync::SyncItems;
 use crate::tools::{
     create_smeared_timestamp, get_filemeta, parse_receive_headers, smeared_time,
     strip_rtlo_characters, truncate_by_lines,
 };
-use crate::{location, stock_str, tools, ui_events};
+use crate::{location, tools};
 
 /// A parsed MIME message.
 ///
@@ -175,6 +176,10 @@ pub enum SystemMessage {
     /// "%1$s sent a message from another device."
     ChatProtectionDisabled = 12,
 
+    /// Message can't be sent because of `Invalid unencrypted mail to <>`
+    /// which is sent by chatmail servers.
+    InvalidUnencryptedMail = 13,
+
     /// Self-sent-message that contains only json used for multi-device-sync;
     /// if possible, we attach that to other messages as for locations.
     MultiDeviceSync = 20,
@@ -2118,8 +2123,6 @@ async fn handle_mdn(
     {
         update_msg_state(context, msg_id, MessageState::OutMdnRcvd).await?;
         context.emit_event(EventType::MsgRead { chat_id, msg_id });
-        // note(treefit): only matters if it is the last message in chat (but probably to expensive to check, debounce also solves it)
-        ui_events::emit_chatlist_item_changed(context, chat_id);
     }
     Ok(())
 }

src/peer_channels.rs

@@ -0,0 +1,413 @@
+//! Peer channels for webxdc updates using iroh.
+
+use anyhow::{anyhow, Context as _, Result};
+use image::EncodableLayout;
+use iroh_base::base32;
+use iroh_gossip::net::{Gossip, GOSSIP_ALPN};
+use iroh_gossip::proto::{Event as IrohEvent, TopicId};
+use iroh_net::magic_endpoint::accept_conn;
+use iroh_net::NodeId;
+use iroh_net::{derp::DerpMode, key::SecretKey, MagicEndpoint};
+
+use crate::config::Config;
+use crate::contact::ContactId;
+use crate::context::Context;
+use crate::message::{Message, MsgId};
+use crate::tools::time;
+use crate::webxdc::StatusUpdateItem;
+
+impl Context {
+    /// Create magic endpoint and gossip for the context.
+    pub async fn create_gossip(&self) -> Result<()> {
+        let secret_key: SecretKey = self.get_or_generate_iroh_keypair().await?;
+        println!("> our secret key: {}", base32::fmt(secret_key.to_bytes()));
+
+        if self.endpoint.lock().await.is_some() {
+            warn!(
+                self,
+                "Tried to create endpoint even though there is already one."
+            );
+            return Ok(());
+        }
+
+        // build magic endpoint
+        let endpoint = MagicEndpoint::builder()
+            .secret_key(secret_key)
+            .alpns(vec![GOSSIP_ALPN.to_vec()])
+            .derp_mode(DerpMode::Default)
+            .bind(0)
+            .await?;
+
+        // create gossip
+        let my_addr = endpoint.my_addr().await?;
+        let gossip = Gossip::from_endpoint(endpoint.clone(), Default::default(), &my_addr.info);
+
+        // spawn endpoint loop that forwards incoming connections to the gossiper
+        let context = self.clone();
+        tokio::spawn(endpoint_loop(context, endpoint.clone(), gossip.clone()));
+
+        *self.gossip.lock().await = Some(gossip);
+        *self.endpoint.lock().await = Some(endpoint);
+        Ok(())
+    }
+
+    /// Join a topic and create the subscriber loop for it.
+    pub async fn join_and_subscribe_topic(&self, topic: TopicId, msg_id: MsgId) -> Result<()> {
+        info!(&self, "Joining topic {}.", topic.to_string());
+
+        let Some(ref gossip) = *self.gossip.lock().await else {
+            warn!(
+                self,
+                "Not joining topic {topic} because there is no gossip."
+            );
+            return Ok(());
+        };
+
+        // restore old peers from db, if any
+        let peers = self.get_peers_for_topic(topic).await?;
+        if peers.len() == 0 {
+            // TODO: When there's no peers we will never be able to join the gossip?
+            warn!(self, "joining gossip with zero peers");
+        } else {
+            info!(self, "joining gossip with peers: {peers:?}");
+            info!(
+                self,
+                "{:?}",
+                self.endpoint
+                    .lock()
+                    .await
+                    .as_ref()
+                    .unwrap()
+                    .my_addr()
+                    .await?
+            );
+        }
+
+        // TODO: add timeout as the returned future might be pending forever
+        let connect_future = gossip.join(topic, peers).await?;
+
+        tokio::spawn(connect_future);
+        tokio::spawn(subscribe_loop(self.clone(), gossip.clone(), topic, msg_id));
+
+        Ok(())
+    }
+
+    /// Get list of [NodeId]s for one topic.
+    /// This is used to rejoin a gossip group when reopening the xdc.
+    /// Only [NodeId] is needed because the magic endpoint caches region and derp server for [NodeId]s.
+    pub async fn get_peers_for_topic(&self, topic: TopicId) -> Result<Vec<NodeId>> {
+        self.sql
+            .query_map(
+                "SELECT public_key FROM iroh_gossip_peers WHERE topic = ?",
+                (topic.as_bytes(),),
+                |row| {
+                    let data = row.get::<_, Vec<u8>>(0)?;
+                    Ok(data)
+                },
+                |g| {
+                    g.map(|data| {
+                        Ok::<NodeId, anyhow::Error>(NodeId::from_bytes(
+                            &data?
+                                .try_into()
+                                .map_err(|_| anyhow!("Can't convert sql data to [u8; 32]"))?,
+                        )?)
+                    })
+                    .collect::<std::result::Result<Vec<_>, _>>()
+                    .map_err(Into::into)
+                },
+            )
+            .await
+    }
+
+    /// Cache a peers [NodeId] for one topic.
+    pub async fn add_peer_for_topic(
+        &self,
+        msg_id: MsgId,
+        topic: TopicId,
+        peer: NodeId,
+    ) -> Result<()> {
+        self.sql
+            .execute(
+                "INSERT INTO iroh_gossip_peers (msg_id, public_key, topic) VALUES (?, ?, ?)",
+                (msg_id, peer.as_bytes(), topic.as_bytes()),
+            )
+            .await?;
+        Ok(())
+    }
+
+    /// Remove one cached peer from a topic.
+    pub async fn delete_peer_for_topic(&self, topic: TopicId, peer: NodeId) -> Result<()> {
+        self.sql
+            .execute(
+                "DELETE FROM iroh_gossip_peers WHERE public_key = ? topic = ?",
+                (peer.as_bytes(), topic.as_bytes()),
+            )
+            .await?;
+        Ok(())
+    }
+
+    /// Get the iroh gossip secret key from the database or generate a new one and persist it.
+    pub async fn get_or_generate_iroh_keypair(&self) -> Result<SecretKey> {
+        match self.get_config_parsed(Config::IrohSecretKey).await? {
+            Some(key) => Ok(key),
+            None => {
+                let key = SecretKey::generate();
+                self.set_config(Config::IrohSecretKey, Some(&key.to_string()))
+                    .await?;
+                Ok(key)
+            }
+        }
+    }
+}
+
+async fn endpoint_loop(context: Context, endpoint: MagicEndpoint, gossip: Gossip) {
+    while let Some(conn) = endpoint.accept().await {
+        info!(context, "accepting connection with {:?}", conn);
+        let gossip = gossip.clone();
+        let context = context.clone();
+        tokio::spawn(async move {
+            if let Err(err) = handle_connection(&context, conn, gossip).await {
+                warn!(context, "iroh connection error: {err}");
+            }
+        });
+    }
+}
+
+async fn handle_connection(
+    context: &Context,
+    conn: quinn::Connecting,
+    gossip: Gossip,
+) -> anyhow::Result<()> {
+    let (peer_id, alpn, conn) = accept_conn(conn).await?;
+    match alpn.as_bytes() {
+        GOSSIP_ALPN => gossip
+            .handle_connection(conn)
+            .await
+            .context(format!("Connection to {peer_id} with ALPN {alpn} failed"))?,
+        _ => info!(
+            context,
+            "Ignoring connection from {peer_id}: unsupported ALPN protocol"
+        ),
+    }
+    Ok(())
+}
+
+async fn subscribe_loop(
+    context: Context,
+    gossip: Gossip,
+    topic: TopicId,
+    msg_id: MsgId,
+) -> Result<()> {
+    let mut stream = gossip.subscribe(topic).await?;
+    loop {
+        let event = stream.recv().await?;
+        match event {
+            IrohEvent::NeighborUp(node) => {
+                info!(context, "NeighborUp: {:?}", node);
+                context.add_peer_for_topic(msg_id, topic, node).await?;
+            }
+            IrohEvent::NeighborDown(node) => {
+                info!(context, "NeighborDown: {:?}", node);
+                context.delete_peer_for_topic(topic, node).await?;
+            }
+            IrohEvent::Received(event) => {
+                info!(context, "Received: {:?}", event);
+                let payload = String::from_utf8_lossy(event.content.as_bytes());
+                let mut instance = Message::load_from_db(&context, msg_id).await?;
+                let update: StatusUpdateItem = serde_json::from_str(&payload)?;
+                context
+                    .create_status_update_record(
+                        &mut instance,
+                        update,
+                        time(),
+                        false,
+                        ContactId::SELF,
+                    )
+                    .await?;
+            }
+        };
+    }
+}
+
+#[cfg(test)]
+mod tests {
+    use std::{os::unix::thread, str::FromStr, time::Duration};
+
+    use tokio::time::timeout;
+
+    use crate::{
+        chat::send_msg,
+        message::Viewtype,
+        test_utils::TestContextManager,
+        webxdc::{join_gossip_topic, StatusUpdateSerial},
+        EventType,
+    };
+
+    use super::*;
+
+    #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
+    async fn test_can_connect() {
+        let mut tcm = TestContextManager::new();
+        let alice = &mut tcm.alice().await;
+        let bob = &mut tcm.bob().await;
+        alice.ctx.start_io().await;
+        bob.ctx.start_io().await;
+
+        // Alice sends webxdc to bob
+        let alice_chat = alice.create_chat(bob).await;
+        let mut instance = Message::new(Viewtype::File);
+        instance
+            .set_file_from_bytes(
+                alice,
+                "minimal.xdc",
+                include_bytes!("../test-data/webxdc/minimal.xdc"),
+                None,
+            )
+            .await
+            .unwrap();
+
+        send_msg(alice, alice_chat.id, &mut instance).await.unwrap();
+
+        let alice_instance = alice.get_last_msg().await;
+        assert_eq!(alice_instance.get_viewtype(), Viewtype::Webxdc);
+
+        let webxdc = alice.pop_sent_msg().await;
+        let bob_webdxc = bob.recv_msg(&webxdc).await;
+        bob_webdxc.chat_id.accept(bob).await.unwrap();
+
+        assert_eq!(bob_webdxc.get_viewtype(), Viewtype::Webxdc);
+
+        // Alice sends webxdc update with gossip.
+        // This produces an SMTP message that contains the topic and a header with alices' node id
+        alice
+            .send_webxdc_status_update_struct(
+                alice_instance.id,
+                StatusUpdateItem {
+                    payload: "test".to_string().into(),
+                    gossip_topic: Some("aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa".to_string()),
+                    ..Default::default()
+                },
+                "",
+            )
+            .await
+            .unwrap();
+
+        alice.flush_status_updates().await.unwrap();
+        bob.recv_msg(&alice.pop_sent_msg().await).await;
+
+        let status = bob
+            .get_webxdc_status_updates(bob_webdxc.id, StatusUpdateSerial::new(0))
+            .await
+            .unwrap();
+        let status_update_items: Vec<StatusUpdateItem> = serde_json::from_str(&status).unwrap();
+        let topic = status_update_items[0].gossip_topic.as_ref().unwrap();
+        assert_eq!(topic, "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa");
+
+        let topic_id = TopicId::from_str(&iroh_base::base32::fmt(topic)).unwrap();
+        let topics = bob.get_peers_for_topic(topic_id).await.unwrap();
+        assert_eq!(
+            topics,
+            vec![alice.endpoint.lock().await.as_ref().unwrap().node_id()]
+        );
+
+        let mut stream = alice
+            .ctx
+            .gossip
+            .lock()
+            .await
+            .as_ref()
+            .unwrap()
+            .subscribe(topic_id)
+            .await
+            .unwrap();
+
+        // Bob joins topic
+        join_gossip_topic(bob, bob_webdxc.id, topic).await.unwrap();
+
+        let event = timeout(Duration::from_secs(5), stream.recv())
+            .await
+            .unwrap()
+            .unwrap();
+        match event {
+            IrohEvent::NeighborUp(node) => {
+                assert_eq!(node, bob.endpoint.lock().await.as_ref().unwrap().node_id());
+            }
+            _ => panic!("Expected NeighborUp event"),
+        }
+
+        // Bob sends webxdc update with gossip.
+        bob.send_webxdc_status_update_struct(
+            bob_webdxc.id,
+            StatusUpdateItem {
+                payload: "bob -> alice".to_string().into(),
+                gossip_topic: Some("aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa".to_string()),
+                ..Default::default()
+            },
+            "",
+        )
+        .await
+        .unwrap();
+
+        alice.evtracker.try_recv().unwrap();
+        while let Ok(event) = alice.evtracker.try_recv() {
+            if let EventType::WebxdcStatusUpdate {
+                msg_id,
+                status_update_serial,
+            } = event.typ
+            {
+                let status_update = alice
+                    .get_status_update(msg_id, status_update_serial)
+                    .await
+                    .unwrap();
+                let status_update_item: StatusUpdateItem =
+                    serde_json::from_str(&status_update).unwrap();
+                println!("{:?}", status_update_item.payload.to_string());
+                if status_update_item
+                    .payload
+                    .to_string()
+                    .contains("bob -> alice")
+                {
+                    break;
+                }
+            }
+        }
+
+        // Alice sends webxdc update with gossip.
+        alice
+            .send_webxdc_status_update_struct(
+                bob_webdxc.id,
+                StatusUpdateItem {
+                    payload: "alice -> bob".to_string().into(),
+                    gossip_topic: Some("aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa".to_string()),
+                    ..Default::default()
+                },
+                "",
+            )
+            .await
+            .unwrap();
+
+        while let Ok(event) = bob.evtracker.try_recv() {
+            if let EventType::WebxdcStatusUpdate {
+                msg_id,
+                status_update_serial,
+            } = event.typ
+            {
+                let status_update = alice
+                    .get_status_update(msg_id, status_update_serial)
+                    .await
+                    .unwrap();
+
+                let status_update_item: StatusUpdateItem =
+                    serde_json::from_str(&status_update).unwrap();
+
+                if status_update_item
+                    .payload
+                    .to_string()
+                    .contains("alice -> bob")
+                {
+                    break;
+                }
+            }
+        }
+    }
+}

src/peerstate.rs

@@ -15,7 +15,7 @@ use crate::key::{DcKey, Fingerprint, SignedPublicKey};
 use crate::message::Message;
 use crate::mimeparser::SystemMessage;
 use crate::sql::Sql;
-use crate::{stock_str, ui_events};
+use crate::stock_str;
 
 /// Type of the public key stored inside the peerstate.
 #[derive(Debug)]
@@ -695,9 +695,6 @@ impl Peerstate {
             .await?;
         }
 
-        ui_events::emit_chatlist_changed(context);
-        // update the chats the contact is part of
-        ui_events::emit_chatlist_items_changed_for_contact(context, contact_id);
         Ok(())
     }
 

src/receive_imf.rs

@@ -4,6 +4,7 @@ use std::collections::HashSet;
 use std::convert::TryFrom;
 
 use anyhow::{Context as _, Result};
+use iroh_net::NodeAddr;
 use mailparse::{parse_mail, SingleInfo};
 use num_traits::FromPrimitive;
 use once_cell::sync::Lazy;
@@ -23,6 +24,7 @@ use crate::ephemeral::{stock_ephemeral_timer_changed, Timer as EphemeralTimer};
 use crate::events::EventType;
 use crate::headerdef::{HeaderDef, HeaderDefMap};
 use crate::imap::{markseen_on_imap_table, GENERATED_PREFIX};
+use crate::location;
 use crate::log::LogExt;
 use crate::message::{
     self, rfc724_mid_exists, rfc724_mid_exists_and, Message, MessageState, MessengerMessage, MsgId,
@@ -39,7 +41,6 @@ use crate::stock_str;
 use crate::sync::Sync::*;
 use crate::tools::{buf_compress, extract_grpid_from_rfc724_mid, strip_rtlo_characters};
 use crate::{contact, imap};
-use crate::{location, ui_events};
 
 /// This is the struct that is returned after receiving one email (aka MIME message).
 ///
@@ -433,11 +434,49 @@ pub(crate) async fn receive_imf_inner(
     }
 
     if let Some(ref status_update) = mime_parser.webxdc_status_update {
-        if let Err(err) = context
+        match context
             .receive_status_update(from_id, insert_msg_id, status_update)
             .await
         {
-            warn!(context, "receive_imf cannot update status: {err:#}.");
+            // join advertised gossip topics
+            Ok((topics, instance_id)) => {
+                warn!(context, "Joining topics: {:#?}", topics);
+                if let Some(node_addr) = mime_parser.get_header(HeaderDef::IrohPublicGossip) {
+                    match serde_json::from_str::<NodeAddr>(node_addr)
+                        .context("Failed to parse node address")
+                    {
+                        Ok(node_addr) => {
+                            context
+                                .endpoint
+                                .lock()
+                                .await
+                                .as_ref()
+                                .context("Failed to get magic endpoint")?
+                                .add_node_addr(node_addr.clone())
+                                .context("Failed to add node address")?;
+
+                            let node_id = node_addr.node_id;
+                            for topic in topics {
+                                println!("Adding peer: {:?}", node_id);
+                                context
+                                    .add_peer_for_topic(instance_id, topic, node_id)
+                                    .await?;
+
+                                println!(
+                                    "New peer topics: {:?}",
+                                    context.get_peers_for_topic(topic).await?
+                                );
+                            }
+                        }
+                        Err(err) => {
+                            warn!(context, "couldn't parse NodeAddr: {err}");
+                        }
+                    }
+                } else {
+                    error!(context, "No IrohPublicGossip header found");
+                }
+            }
+            Err(err) => warn!(context, "receive_imf cannot update status: {err:#}."),
         }
     }
 
@@ -733,7 +772,7 @@ async fn add_parts(
             create_blocked_default
         };
 
-        if chat_id.is_none() {
+        if chat_id.is_none() && !is_mdn {
             // try to create a group
 
             if let Some((new_chat_id, new_chat_id_blocked)) = create_or_lookup_group(
@@ -1778,8 +1817,6 @@ async fn create_or_lookup_group(
         chat::add_to_chat_contacts_table(context, new_chat_id, &members).await?;
 
         context.emit_event(EventType::ChatModified(new_chat_id));
-        ui_events::emit_chatlist_changed(context);
-        ui_events::emit_chatlist_item_changed(context, new_chat_id);
     }
 
     if let Some(chat_id) = chat_id {
@@ -2071,7 +2108,6 @@ async fn apply_group_changes(
 
     if send_event_chat_modified {
         context.emit_event(EventType::ChatModified(chat_id));
-        ui_events::emit_chatlist_item_changed(context, chat_id);
     }
     Ok((group_changes_msgs, better_msg))
 }
@@ -2371,8 +2407,6 @@ async fn create_adhoc_group(
     chat::add_to_chat_contacts_table(context, new_chat_id, member_ids).await?;
 
     context.emit_event(EventType::ChatModified(new_chat_id));
-    ui_events::emit_chatlist_changed(context);
-    ui_events::emit_chatlist_item_changed(context, new_chat_id);
 
     Ok(Some(new_chat_id))
 }

src/receive_imf/tests.rs

@@ -310,6 +310,56 @@ async fn test_read_receipt_and_unarchive() -> Result<()> {
     Ok(())
 }
 
+#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
+async fn test_mdn_and_alias() -> Result<()> {
+    let mut tcm = TestContextManager::new();
+    let alice = tcm.alice().await;
+    let bob = tcm.bob().await;
+    let alice_chat = alice.create_chat(&bob).await;
+    let sent = alice.send_text(alice_chat.id, "alice -> bob").await;
+    let msg_id = sent.sender_msg_id;
+    receive_imf(
+        &alice,
+        format!(
+            "Received: (Postfix, from userid 1000); Mon, 4 Dec 2006 14:51:39 +0100 (CET)\n\
+             From: bob@example.net\n\
+             To: alicechat@example.org\n\
+             Subject: message opened\n\
+             Date: Sun, 22 Mar 2020 23:37:57 +0000\n\
+             Chat-Version: 1.0\n\
+             Message-ID: <aranudiaerudiaduiaertd@example.com>\n\
+             Content-Type: multipart/report; report-type=disposition-notification; boundary=\"SNIPP\"\n\
+             \n\
+             \n\
+             --SNIPP\n\
+             Content-Type: text/plain; charset=utf-8\n\
+             \n\
+             Read receipts do not guarantee sth. was read.\n\
+             \n\
+             \n\
+             --SNIPP\n\
+             Content-Type: message/disposition-notification\n\
+             \n\
+             Reporting-UA: Delta Chat 1.28.0\n\
+             Original-Recipient: rfc822;bob@example.com\n\
+             Final-Recipient: rfc822;bob@example.com\n\
+             Original-Message-ID: <{msg_id}>\n\
+             Disposition: manual-action/MDN-sent-automatically; displayed\n\
+             \n\
+             \n\
+             --SNIPP--",
+        )
+        .as_bytes(),
+        false,
+    )
+    .await?;
+
+    let chats = Chatlist::try_load(&alice, 0, None, None).await?;
+    assert_eq!(chats.len(), 1);
+
+    Ok(())
+}
+
 #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
 async fn test_no_from() {
     // if there is no from given, from_id stays 0 which is just fine. These messages

src/scheduler.rs

@@ -15,7 +15,7 @@ use self::connectivity::ConnectivityStore;
 use crate::config::Config;
 use crate::contact::{ContactId, RecentlySeenLoop};
 use crate::context::Context;
-use crate::download::download_msg;
+use crate::download::{download_msg, DownloadState};
 use crate::ephemeral::{self, delete_expired_imap_messages};
 use crate::events::EventType;
 use crate::imap::{FolderMeaning, Imap};
@@ -70,8 +70,11 @@ impl SchedulerState {
         context.new_msgs_notify.notify_one();
 
         let ctx = context.clone();
-        match Scheduler::start(context).await {
-            Ok(scheduler) => *inner = InnerSchedulerState::Started(scheduler),
+        match Scheduler::start(&context).await {
+            Ok(scheduler) => {
+                *inner = InnerSchedulerState::Started(scheduler);
+                context.emit_event(EventType::ConnectivityChanged);
+            }
             Err(err) => error!(&ctx, "Failed to start IO: {:#}", err),
         }
     }
@@ -116,6 +119,7 @@ impl SchedulerState {
             debug_logging.loop_handle.abort();
         }
         let prev_state = std::mem::replace(&mut *inner, new_state);
+        context.emit_event(EventType::ConnectivityChanged);
         match prev_state {
             InnerSchedulerState::Started(scheduler) => scheduler.stop(context).await,
             InnerSchedulerState::Stopped | InnerSchedulerState::Paused { .. } => (),
@@ -346,6 +350,16 @@ async fn download_msgs(context: &Context, imap: &mut Imap) -> Result<()> {
     for msg_id in msg_ids {
         if let Err(err) = download_msg(context, msg_id, imap).await {
             warn!(context, "Failed to download message {msg_id}: {:#}.", err);
+
+            // Update download state to failure
+            // so it can be retried.
+            //
+            // On success update_download_state() is not needed
+            // as receive_imf() already
+            // set the state and emitted the event.
+            msg_id
+                .update_download_state(context, DownloadState::Failure)
+                .await?;
         }
         context
             .sql
@@ -772,7 +786,7 @@ async fn smtp_loop(
 
 impl Scheduler {
     /// Start the scheduler.
-    pub async fn start(ctx: Context) -> Result<Self> {
+    pub async fn start(ctx: &Context) -> Result<Self> {
         let (smtp, smtp_handlers) = SmtpConnectionState::new();
 
         let (smtp_start_send, smtp_start_recv) = oneshot::channel();
@@ -782,7 +796,7 @@ impl Scheduler {
         let mut oboxes = Vec::new();
         let mut start_recvs = Vec::new();
 
-        let (conn_state, inbox_handlers) = ImapConnectionState::new(&ctx).await?;
+        let (conn_state, inbox_handlers) = ImapConnectionState::new(ctx).await?;
         let (inbox_start_send, inbox_start_recv) = oneshot::channel();
         let handle = {
             let ctx = ctx.clone();
@@ -803,7 +817,7 @@ impl Scheduler {
             ),
         ] {
             if should_watch? {
-                let (conn_state, handlers) = ImapConnectionState::new(&ctx).await?;
+                let (conn_state, handlers) = ImapConnectionState::new(ctx).await?;
                 let (start_send, start_recv) = oneshot::channel();
                 let ctx = ctx.clone();
                 let handle = task::spawn(simple_imap_loop(ctx, start_send, handlers, meaning));

src/securejoin.rs

@@ -25,7 +25,6 @@ use crate::stock_str;
 use crate::sync::Sync::*;
 use crate::token;
 use crate::tools::time;
-use crate::ui_events;
 
 mod bob;
 mod bobstate;
@@ -684,7 +683,6 @@ async fn secure_connection_established(
         )
         .await?;
     context.emit_event(EventType::ChatModified(chat_id));
-    ui_events::emit_chatlist_item_changed(context, chat_id);
     Ok(())
 }
 

src/smtp.rs

@@ -10,6 +10,7 @@ use async_smtp::{self as smtp, EmailAddress, SmtpTransport};
 use tokio::io::BufStream;
 use tokio::task;
 
+use crate::chat::{add_info_msg_with_cmd, ChatId};
 use crate::config::Config;
 use crate::contact::{Contact, ContactId};
 use crate::context::Context;
@@ -26,6 +27,7 @@ use crate::provider::Socket;
 use crate::scheduler::connectivity::ConnectivityStore;
 use crate::socks::Socks5Config;
 use crate::sql;
+use crate::stock_str::unencrypted_email;
 
 /// SMTP connection, write and read timeout.
 const SMTP_TIMEOUT: Duration = Duration::from_secs(60);
@@ -584,7 +586,46 @@ pub(crate) async fn send_msg_to_smtp(
 
     match status {
         SendResult::Retry => {}
-        SendResult::Success | SendResult::Failure(_) => {
+        SendResult::Success => {
+            context
+                .sql
+                .execute("DELETE FROM smtp WHERE id=?", (rowid,))
+                .await?;
+        }
+        SendResult::Failure(ref err) => {
+            if err.to_string().contains("Invalid unencrypted mail") {
+                let res = context
+                    .sql
+                    .query_row_optional(
+                        "SELECT chat_id, timestamp FROM msgs WHERE id=?;",
+                        (msg_id,),
+                        |row| Ok((row.get::<_, ChatId>(0)?, row.get::<_, i64>(1)?)),
+                    )
+                    .await?;
+
+                if let Some((chat_id, timestamp_sort)) = res {
+                    let addr = context.get_config(Config::ConfiguredAddr).await?;
+                    let text = unencrypted_email(
+                        context,
+                        addr.unwrap_or_default()
+                            .split('@')
+                            .nth(1)
+                            .unwrap_or_default(),
+                    )
+                    .await;
+                    add_info_msg_with_cmd(
+                        context,
+                        chat_id,
+                        &text,
+                        crate::mimeparser::SystemMessage::InvalidUnencryptedMail,
+                        timestamp_sort,
+                        None,
+                        None,
+                        None,
+                    )
+                    .await?;
+                };
+            }
             context
                 .sql
                 .execute("DELETE FROM smtp WHERE id=?", (rowid,))
@@ -618,7 +659,7 @@ async fn send_mdns(context: &Context, connection: &mut Smtp) -> Result<()> {
 
         let more_mdns = send_mdn(context, connection).await?;
         if !more_mdns {
-            // No more MDNs to send.
+            // No more MDNs to send or one of them failed.
             return Ok(());
         }
     }
@@ -752,7 +793,7 @@ async fn send_mdn_msg_id(
     }
 }
 
-/// Tries to send a single MDN. Returns false if there are no MDNs to send.
+/// Tries to send a single MDN. Returns true if more MDNs should be sent.
 async fn send_mdn(context: &Context, smtp: &mut Smtp) -> Result<bool> {
     let mdns_enabled = context.get_config_bool(Config::MdnsEnabled).await?;
     if !mdns_enabled {

src/sql/migrations.rs

@@ -900,6 +900,14 @@ CREATE INDEX msgs_status_updates_index2 ON msgs_status_updates (uid);
         .await?;
     }
 
+    if dbversion < 110 {
+        sql.execute_migration(
+            "CREATE TABLE iroh_gossip_peers (msg_id TEXT not NULL, topic TEXT NOT NULL, public_key TEXT NOT NULL)",
+            110,
+        )
+        .await?;
+    }
+
     let new_version = sql
         .get_raw_config_int(VERSION_CFG)
         .await?

src/stock_str.rs

@@ -419,6 +419,11 @@ pub enum StockMessage {
 
     #[strum(props(fallback = "Member %1$s added."))]
     MsgAddMember = 173,
+
+    #[strum(props(
+        fallback = "⚠️ Your email provider %1$s requires end-to-end encryption which is not setup yet."
+    ))]
+    InvalidUnencryptedMail = 174,
 }
 
 impl StockMessage {
@@ -1285,6 +1290,13 @@ pub(crate) async fn aeap_addr_changed(
         .replace3(new_addr)
 }
 
+/// Stock string: `⚠️ Your email provider %1$s requires end-to-end encryption which is not setup yet. Tap to learn more.`.
+pub(crate) async fn unencrypted_email(context: &Context, provider: &str) -> String {
+    translated(context, StockMessage::InvalidUnencryptedMail)
+        .await
+        .replace1(provider)
+}
+
 pub(crate) async fn aeap_explanation_and_link(
     context: &Context,
     old_addr: &str,

src/webxdc.rs

@@ -16,10 +16,11 @@
 //! - `descr` - text to send along with the updates
 
 use std::path::Path;
+use std::str::FromStr;
 
 use anyhow::{anyhow, bail, ensure, format_err, Context as _, Result};
-
 use deltachat_derive::FromSql;
+use iroh_gossip::proto::TopicId;
 use lettre_email::mime;
 use lettre_email::PartBuilder;
 use serde::{Deserialize, Serialize};
@@ -154,7 +155,7 @@ struct StatusUpdates {
 }
 
 /// Update items as sent on the wire and as stored in the database.
-#[derive(Debug, Serialize, Deserialize, Default)]
+#[derive(Debug, Serialize, Deserialize, Default, Clone)]
 pub struct StatusUpdateItem {
     /// The playload of the status update.
     pub payload: Value,
@@ -181,6 +182,12 @@ pub struct StatusUpdateItem {
     /// If there is no ID, message is always considered to be unique.
     #[serde(skip_serializing_if = "Option::is_none")]
     pub uid: Option<String>,
+
+    /// If this update should only be gossiped and which topic to use.
+    /// Gossiped Updates will only be received by the other side if they
+    /// are currently online and part of the gossip topic.
+    #[serde(default)]
+    pub gossip_topic: Option<String>,
 }
 
 /// Update items as passed to the UIs.
@@ -305,7 +312,7 @@ impl Context {
 
     /// Takes an update-json as `{payload: PAYLOAD}`
     /// writes it to the database and handles events, info-messages, document name and summary.
-    async fn create_status_update_record(
+    pub(crate) async fn create_status_update_record(
         &self,
         instance: &mut Message,
         status_update_item: StatusUpdateItem,
@@ -396,7 +403,6 @@ impl Context {
         instance_id: &MsgId,
         status_update_item: &StatusUpdateItem,
     ) -> Result<Option<StatusUpdateSerial>> {
-        let _lock = self.sql.write_lock().await;
         let uid = status_update_item.uid.as_deref();
         let Some(rowid) = self
             .sql
@@ -490,6 +496,47 @@ impl Context {
             MessageState::Undefined | MessageState::OutPreparing | MessageState::OutDraft
         );
 
+        let mut ephemeral = status_update.gossip_topic.is_some();
+        if send_now {
+            if let Some(ref topic) = status_update.gossip_topic {
+                let topic = TopicId::from_str(&iroh_base::base32::fmt(
+                    topic.get(0..32).context("Can't get 32 bytes from topic")?,
+                ))?;
+
+                let topic_exists = self
+                    .sql
+                    .query_row_optional(
+                        "SELECT 1 FROM iroh_gossip_peers WHERE topic=?",
+                        (topic.as_bytes(),),
+                        |_| Ok(()),
+                    )
+                    .await
+                    .context("Failed to check if gossip topic exists")?
+                    .is_some();
+
+                if !topic_exists {
+                    info!(
+                        self,
+                        "Gossip topic {topic} does not exist, sending over smtp",
+                    );
+                    self.join_and_subscribe_topic(topic, instance_msg_id)
+                        .await
+                        .context("Failed to join and subscribe to gossip topic")?;
+                    ephemeral = false;
+                } else {
+                    if let Some(ref gossip) = *self.gossip.lock().await {
+                        println!(
+                            "sending to topic {topic} with peers: {:?}",
+                            self.get_peers_for_topic(topic).await?
+                        );
+                        gossip
+                            .broadcast(topic, serde_json::to_string(&status_update)?.into())
+                            .await?;
+                    }
+                }
+            }
+        }
+
         status_update.uid = Some(create_id());
         let status_update_serial: StatusUpdateSerial = self
             .create_status_update_record(
@@ -499,11 +546,10 @@ impl Context {
                 send_now,
                 ContactId::SELF,
             )
-            .await
-            .context("Failed to create status update")?
-            .context("Duplicate status update UID was generated")?;
+            .await?
+            .context("Failed to create status update")?;
 
-        if send_now {
+        if send_now && !ephemeral {
             self.sql.insert(
                 "INSERT INTO smtp_status_updates (msg_id, first_serial, last_serial, descr) VALUES(?, ?, ?, ?)
                  ON CONFLICT(msg_id)
@@ -595,12 +641,14 @@ impl Context {
     ///
     /// `json` is an array containing one or more update items as created by send_webxdc_status_update(),
     /// the array is parsed using serde, the single payloads are used as is.
+    ///
+    /// Returns: List of topics that have been advertised in the updates and the [MsgId] of the instance.
     pub(crate) async fn receive_status_update(
         &self,
         from_id: ContactId,
         msg_id: MsgId,
         json: &str,
-    ) -> Result<()> {
+    ) -> Result<(Vec<TopicId>, MsgId)> {
         let msg = Message::load_from_db(self, msg_id).await?;
         let (timestamp, mut instance, can_info_msg) = if msg.viewtype == Viewtype::Webxdc {
             (msg.timestamp_sort, msg, false)
@@ -629,7 +677,15 @@ impl Context {
         }
 
         let updates: StatusUpdates = serde_json::from_str(json)?;
+        let mut topics = Vec::new();
         for update_item in updates.updates {
+            if let Some(ref topic) = update_item.gossip_topic {
+                let topic = TopicId::from_str(&iroh_base::base32::fmt(
+                    topic.get(0..32).context("Can't get 32 bytes from topic")?,
+                ))?;
+                topics.push(topic);
+            }
+
             self.create_status_update_record(
                 &mut instance,
                 update_item,
@@ -640,7 +696,7 @@ impl Context {
             .await?;
         }
 
-        Ok(())
+        Ok((topics, instance.id))
     }
 
     /// Returns status updates as an JSON-array, ready to be consumed by a webxdc.
@@ -871,9 +927,18 @@ impl Message {
     }
 }
 
+/// Join a gossip topic and subscribe to it.
+pub async fn join_gossip_topic(ctx: &Context, msg_id: MsgId, topic: &str) -> Result<()> {
+    let topic = TopicId::from_str(&iroh_base::base32::fmt(
+        topic.get(0..32).context("Can't get 32 bytes from topic")?,
+    ))?;
+    ctx.join_and_subscribe_topic(topic, msg_id).await
+}
+
 #[cfg(test)]
 mod tests {
     use serde_json::json;
+    use std::time::Duration;
 
     use super::*;
     use crate::chat::{
@@ -1340,11 +1405,10 @@ mod tests {
 
         // set_draft(None) deletes the message without the need to simulate network
         chat_id.set_draft(&t, None).await?;
-        assert_eq!(
-            t.get_webxdc_status_updates(instance.id, StatusUpdateSerial(0))
-                .await?,
-            "[]".to_string()
-        );
+        assert!(t
+            .get_webxdc_status_updates(instance.id, StatusUpdateSerial(0))
+            .await
+            .is_err());
         assert_eq!(
             t.sql
                 .count("SELECT COUNT(*) FROM msgs_status_updates;", ())
@@ -1376,6 +1440,7 @@ mod tests {
                     document: None,
                     summary: None,
                     uid: Some("iecie2Ze".to_string()),
+                    gossip_topic: None,
                 },
                 1640178619,
                 true,
@@ -1400,6 +1465,7 @@ mod tests {
                     document: None,
                     summary: None,
                     uid: Some("iecie2Ze".to_string()),
+                    gossip_topic: None,
                 },
                 1640178619,
                 true,
@@ -1433,6 +1499,7 @@ mod tests {
                     document: None,
                     summary: None,
                     uid: None,
+                    gossip_topic: None,
                 },
                 1640178619,
                 true,
@@ -1452,6 +1519,7 @@ mod tests {
                 document: None,
                 summary: None,
                 uid: None,
+                gossip_topic: None,
             },
             1640178619,
             true,
@@ -1677,6 +1745,43 @@ mod tests {
         Ok(())
     }
 
+    #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
+    async fn test_send_ephemeral_webxdc_status_update() -> Result<()> {
+        let alice = TestContext::new_alice().await;
+        alice.set_config_bool(Config::BccSelf, true).await?;
+        let bob = TestContext::new_bob().await;
+
+        // Alice sends an webxdc instance and a status update
+        let alice_chat = alice.create_chat(&bob).await;
+        let alice_instance = send_webxdc_instance(&alice, alice_chat.id).await?;
+        alice
+            .send_webxdc_status_update(
+                alice_instance.id,
+                r#"{"payload" : {"foo":"bar"}}"#,
+                "descr text",
+            )
+            .await?;
+        alice.flush_status_updates().await?;
+        // Not setting ephemeral should prepare a message
+        alice.pop_sent_msg().await;
+
+        alice
+            .send_webxdc_status_update(
+                alice_instance.id,
+                r#"{"payload" : {"foo":"bar"}, "ephemeral": true}"#,
+                "descr text",
+            )
+            .await?;
+        alice.flush_status_updates().await?;
+
+        // Setting ephemeral should noot prepare a message
+        assert!(&alice
+            .pop_sent_msg_opt(Duration::from_secs(1))
+            .await
+            .is_none());
+        Ok(())
+    }
+
     #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
     async fn test_render_webxdc_status_update_object() -> Result<()> {
         let t = TestContext::new_alice().await;

standards.md

@@ -17,7 +17,7 @@ Seen status synchronization      | IMAP CONDSTORE extension ([RFC 7162][])
 Client/server identification     | IMAP ID extension ([RFC 2971][])
 Authorization                    | OAuth2 ([RFC 6749][])
 End-to-end encryption            | [Autocrypt Level 1][], OpenPGP ([RFC 4880][]), Security Multiparts for MIME ([RFC 1847][]) and [“Mixed Up” Encryption repairing](https://tools.ietf.org/id/draft-dkg-openpgp-pgpmime-message-mangling-00.html)
-Detect/prevent active attacks    | [countermitm][] protocols
+Detect/prevent active attacks    | [securejoin][] protocols
 Compare public keys              | [openpgp4fpr][] URI Scheme
 Header encryption                | [Protected Headers for Cryptographic E-mail](https://datatracker.ietf.org/doc/draft-autocrypt-lamps-protected-headers/)
 Configuration assistance         | [Autoconfigure](https://web.archive.org/web/20210402044801/https://developer.mozilla.org/en-US/docs/Mozilla/Thunderbird/Autoconfiguration) and [Autodiscover][]
@@ -29,7 +29,7 @@ Return receipts                  | Message Disposition Notification (MDN, [RFC 8
 Locations                        | KML ([Open Geospatial Consortium](http://www.opengeospatial.org/standards/kml/), [Google Dev](https://developers.google.com/kml/))
 
 [Autocrypt Level 1]: https://autocrypt.org/level1.html
-[countermitm]: https://countermitm.readthedocs.io/en/latest/
+[securejoin]: https://securejoin.readthedocs.io/en/latest/
 [openpgp4fpr]: https://metacode.biz/openpgp/openpgp4fpr
 [Autodiscover]: https://learn.microsoft.com/en-us/exchange/autodiscover-service-for-exchange-2013
 [XEP-0392]: https://xmpp.org/extensions/xep-0392.html