Release 1.89.0

Wheels are now published to PyPI, recommend it instead of devpi. We
build the wheels only for releases anyway.

Suggest using tox instead of listing all the pytest dependencies to
avoid keeping them up to date in the readme.

We no longer publish `docker/coredeps` images, they cannot be
pulled from a container registry anymore.

Troubleshooting section is outdated, because vsyscall emulation is
only needed for manylinux2010 images, not manylinux2014 or
musllinux_1_1 images.

Mention Podman as an alternative to Docker.

Link to https://py.delta.chat/ instead of only examples.

Remove note about wheels for Mac and Windows. Nobody requests them
anyway.

CHANGELOG.md

@@ -2,6 +2,26 @@
 
 ## Unreleased
 
+### Changes
+
+### Fixes
+
+
+## 1.89.0
+
+### Changes
+
+- (AEAP) When one of your contacts changed their address, they are
+  only replaced in the chat where you got a message from them
+  for now #3491
+
+### Fixes
+- replace musl libc name resolution errors with a better message #3485
+- handle updates for not yet downloaded webxdc instances #3487
+
+
+## 1.88.0
+
 ### Changes
 - Implemented "Automatic e-mail address Porting" (AEAP). You can
   configure a new address in DC now, and when receivers get messages
@@ -12,6 +32,7 @@
 - configure DeltaChat folder by selecting it, so it is configured even if not LISTed #3371
 - build PyPy wheels #6683
 - improve default error if NDN does not provide an error #3456
+- increase ratelimit from 3 to 6 messages per 60 seconds #3481
 
 ### Fixes
 - mailing list: remove square-brackets only for first name #3452

Cargo.lock

@@ -752,7 +752,7 @@ dependencies = [
 
 [[package]]
 name = "deltachat"
-version = "1.87.0"
+version = "1.89.0"
 dependencies = [
  "ansi_term",
  "anyhow",
@@ -832,7 +832,7 @@ dependencies = [
 
 [[package]]
 name = "deltachat_ffi"
-version = "1.87.0"
+version = "1.89.0"
 dependencies = [
  "anyhow",
  "deltachat",

Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "deltachat"
-version = "1.87.0"
+version = "1.89.0"
 authors = ["Delta Chat Developers (ML) <delta@codespeak.net>"]
 edition = "2021"
 license = "MPL-2.0"

deltachat-ffi/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "deltachat_ffi"
-version = "1.87.0"
+version = "1.89.0"
 description = "Deltachat FFI"
 authors = ["Delta Chat Developers (ML) <delta@codespeak.net>"]
 edition = "2018"

node/README.md

@@ -234,7 +234,7 @@ We have the following scripts for building, testing and coverage:
 The following steps are needed to make a release:
 
 1. Wait until `pack-module` github action is completed
-2. Run `npm publish https://download.delta.chat/node/deltachat-node-v1.x.x.tar.gz` to publish it to npm. You probably need write rights to npm.
+2. Run `npm publish https://download.delta.chat/node/deltachat-node-1.x.x.tar.gz` to publish it to npm. You probably need write rights to npm.
 
 ## License
 

package.json

@@ -60,5 +60,5 @@
     "test:mocha": "mocha -r esm node/test/test.js --growl --reporter=spec --bail"
   },
   "types": "node/dist/index.d.ts",
-  "version": "1.87.0"
+  "version": "1.89.0"
 }

python/README.rst

@@ -1,84 +1,77 @@
 =========================
-deltachat python bindings
+DeltaChat Python bindings
 =========================
 
-This package provides bindings to the deltachat-core_ Rust -library
-which implements IMAP/SMTP/MIME/PGP e-mail standards and offers
+This package provides `Python bindings`_ to the `deltachat-core library`_
+which implements IMAP/SMTP/MIME/OpenPGP e-mail standards and offers
 a low-level Chat/Contact/Message API to user interfaces and bots.
 
+.. _`deltachat-core library`: https://github.com/deltachat/deltachat-core-rust
+.. _`Python bindings`: https://py.delta.chat/
 
 Installing pre-built packages (Linux-only)
-========================================================
+==========================================
 
 If you have a Linux system you may try to install the ``deltachat`` binary "wheel" packages
 without any "build-from-source" steps.
-Otherwise you need to `compile the Delta Chat bindings yourself <#sourceinstall>`_.
+Otherwise you need to `compile the Delta Chat bindings yourself`__.
+
+__ sourceinstall_
 
 We recommend to first `install virtualenv <https://virtualenv.pypa.io/en/stable/installation.html>`_,
 then create a fresh Python virtual environment and activate it in your shell::
 
-        virtualenv venv  # or: python -m venv
-        source venv/bin/activate
+        virtualenv env  # or: python -m venv
+        source env/bin/activate
 
 Afterwards, invoking ``python`` or ``pip install`` only
-modifies files in your ``venv`` directory and leaves
+modifies files in your ``env`` directory and leaves
 your system installation alone.
 
-For Linux, we automatically build wheels for all github PR branches
-and push them to a python package index. To install the latest
-github ``master`` branch::
+For Linux we build wheels for all releases and push them to a python package
+index. To install the latest release::
 
-    pip install --pre -i https://m.devpi.net/dc/master deltachat
+    pip install deltachat
 
 To verify it worked::
 
     python -c "import deltachat"
 
-.. note::
-
-    If you can help to automate the building of wheels for Mac or Windows,
-    that'd be much appreciated! please then get
-    `in contact with us <https://delta.chat/en/contribute>`_.
-
-
 Running tests
 =============
 
-After successful binding installation you can install a few more
-Python packages before running the tests::
+Recommended way to run tests is using `tox <https://tox.wiki>`_.
+After successful binding installation you can install tox
+and run the tests::
 
-    python -m pip install pytest pytest-xdist pytest-timeout pytest-rerunfailures requests
-    pytest -v tests
+    pip install tox
+    tox -e py3
 
 This will run all "offline" tests and skip all functional
 end-to-end tests that require accounts on real e-mail servers.
 
 .. _livetests:
 
-running "live" tests with temporary accounts
----------------------------------------------
+Running "live" tests with temporary accounts
+--------------------------------------------
 
-If you want to run live functional tests you can set ``DCC_NEW_TMP_EMAIL`` to a URL that creates e-mail accounts.  Most developers use https://testrun.org URLS created and managed by [mailadm](https://mailadm.readthedocs.io/en/latest/).
+If you want to run live functional tests you can set ``DCC_NEW_TMP_EMAIL`` to a URL that creates e-mail accounts.  Most developers use https://testrun.org URLs created and managed by `mailadm <https://mailadm.readthedocs.io/>`_.
 
-Please feel free to contact us through a github issue or by e-mail and we'll send you a URL that you can then use for functional tests like this:
+Please feel free to contact us through a github issue or by e-mail and we'll send you a URL that you can then use for functional tests like this::
 
     export DCC_NEW_TMP_EMAIL=<URL you got from us>
 
 With this account-creation setting, pytest runs create ephemeral e-mail accounts on the http://testrun.org server.  These accounts exists only for one hour and then are removed completely.
-One hour is enough to invoke pytest and run all offline and online tests:
+One hour is enough to invoke pytest and run all offline and online tests::
 
-    pytest
-
-    # or if you have installed pytest-xdist for parallel test execution
-    pytest -n6
+    tox -e py3
 
 Each test run creates new accounts.
 
-
 .. _sourceinstall:
 
-Installing bindings from source (Updated: July 2020)
-=========================================================
+Installing bindings from source
+===============================
 
 Install Rust and Cargo first.
 The easiest is probably to use `rustup <https://rustup.rs/>`_.
@@ -97,74 +90,42 @@ E.g. on Debian-based systems `apt install python3 python3-pip
 python3-venv` should give you a usable python installation.
 
 Ensure you are in the deltachat-core-rust/python directory, create the
-virtual environment and activate it in your shell::
+virtual environment with dependencies using tox
+and activate it in your shell::
 
    cd python
-   python3 -m venv venv  # or: virtualenv venv
-   source venv/bin/activate
+   tox --devenv env
+   source env/bin/activate
 
 You should now be able to build the python bindings using the supplied script::
 
-   python install_python_bindings.py
+   python3 install_python_bindings.py
 
 The core compilation and bindings building might take a while,
 depending on the speed of your machine.
-The bindings will be installed in release mode but with debug symbols.
-The release mode is currently necessary because some tests generate RSA keys
-which is prohibitively slow in non-release mode.
-
-
-Code examples
-=============
-
-You may look at `examples <https://py.delta.chat/examples.html>`_.
-
-
-.. _`deltachat-core-rust github repository`: https://github.com/deltachat/deltachat-core-rust
-.. _`deltachat-core`: https://github.com/deltachat/deltachat-core-rust
-
 
 Building manylinux based wheels
-====================================
+===============================
 
 Building portable manylinux wheels which come with libdeltachat.so
-can be done with docker-tooling.
+can be done with Docker_ or Podman_.
 
-using docker pull / premade images
-------------------------------------
+.. _Docker: https://www.docker.com/
+.. _Podman: https://podman.io/
 
-We publish a build environment under the ``deltachat/coredeps`` tag so
-that you can pull it from the ``hub.docker.com`` site's "deltachat"
-organization::
+If you want to build your own wheels, build container image first::
 
-    $ docker pull deltachat/coredeps
+   $ cd deltachat-core-rust # cd to deltachat-core-rust working tree
+   $ docker build -t deltachat/coredeps scripts/coredeps
 
-This docker image can be used to run tests and build Python wheels for all interpreters::
-
-    $ docker run -e DCC_NEW_TMP_EMAIL \
-       --rm -it -v \$(pwd):/mnt -w /mnt \
-       deltachat/coredeps scripts/run_all.sh
-
-
-Optionally build your own docker image
---------------------------------------
-
-If you want to build your own custom docker image you can do this::
-
-   $ cd deltachat-core # cd to deltachat-core checkout directory
-   $ docker build -t deltachat/coredeps scripts/docker_coredeps
-
-This will use the ``scripts/docker_coredeps/Dockerfile`` to build
-up docker image called ``deltachat/coredeps``.  You can afterwards
+This will use the ``scripts/coredeps/Dockerfile`` to build
+container image called ``deltachat/coredeps``.  You can afterwards
 find it with::
 
    $ docker images
 
+This docker image can be used to run tests and build Python wheels for all interpreters::
 
-Troubleshooting
----------------
-
-On more recent systems running the docker image may crash.  You can
-fix this by adding ``vsyscall=emulate`` to the Linux kernel boot
-arguments commandline.  E.g. on Debian you'd add this to
-``GRUB_CMDLINE_LINUX_DEFAULT`` in ``/etc/default/grub``.
+    $ docker run -e DCC_NEW_TMP_EMAIL \
+       --rm -it -v $(pwd):/mnt -w /mnt \
+       deltachat/coredeps scripts/run_all.sh

python/tests/test_1_online.py

@@ -1854,7 +1854,7 @@ def test_configure_error_msgs_invalid_server(acfactory):
     # Can't connect so it probably should say something about "internet"
     # again, should not repeat itself
     # If this fails then probably `e.msg.to_lowercase().contains("could not resolve")`
-    # in configure/mod.rs returned false because the error message was changed
+    # in configure.rs returned false because the error message was changed
     # (i.e. did not contain "could not resolve" anymore)
     assert (ev.data2.count("internet") + ev.data2.count("network")) == 1
     # Should mention that it can't connect:

python/tox.ini

@@ -11,7 +11,6 @@ commands =
     pytest -n6 --extra-info --reruns 2 --reruns-delay 5 -v -rsXx --ignored --strict-tls {posargs: tests examples}
     python tests/package_wheels.py {toxworkdir}/wheelhouse
 passenv = 
-    TRAVIS 
     DCC_RS_DEV
     DCC_RS_TARGET
     DCC_NEW_TMP_EMAIL 

scripts/README.md

@@ -14,11 +14,7 @@ and an own build machine.
 - `remote_tests_rust.sh` rsyncs to the build machine and runs
   `run-rust-test.sh` remotely on the build machine. 
 
-- `doxygen/Dockerfile` specifies an image that contains
-  the doxygen tool which is used by `run-doxygen.sh` 
-  to generate C-docs which are then uploaded 
-  via `ci_upload.sh` to `https://c.delta.chat/_unofficial_unreleased_docs/<BRANCH>`
-  (and the master branch is linked to https://c.delta.chat proper). 
+- `run-doxygen.sh` generates C-docs which are then uploaded to https://c.delta.chat/
 
 
 ## Triggering runs on the build machine locally (fast!)

scripts/concourse/docs_wheels.yml

@@ -29,18 +29,19 @@ jobs:
         - name: c-docs
       image_resource:
         source:
-          repository: hrektts/doxygen
+          repository: alpine
         type: registry-image
       platform: linux
       run:
-        path: bash
+        path: sh
         args:
-        - -exc
+        - -ec
         - |
+          apk add --no-cache doxygen git
           cd deltachat-core-rust
-          bash scripts/run-doxygen.sh
+          scripts/run-doxygen.sh
           cd ..
-          cp -av deltachat-core-rust/deltachat-ffi/{html,xml} c-docs/
+          cp -av deltachat-core-rust/deltachat-ffi/html deltachat-core-rust/deltachat-ffi/xml c-docs/
 
   - task: upload-c-docs
     config:

scripts/docker-doxygen/Dockerfile

@@ -1,5 +0,0 @@
-FROM debian:stable
-
-# this is tagged as deltachat/doxygen 
-
-RUN apt-get update && apt-get install -y doxygen 

scripts/run-doxygen.sh

@@ -1,6 +1,5 @@
-#!/usr/bin/env bash
-
-set -ex
+#!/bin/sh
+set -e
 
 cd deltachat-ffi 
 PROJECT_NUMBER=$(git log -1 --format="%h (%cd)") doxygen

src/configure.rs

@@ -642,6 +642,9 @@ async fn nicer_configuration_error(context: &Context, errors: Vec<ConfigurationE
                 .to_lowercase()
                 .contains("temporary failure in name resolution")
             || e.msg.to_lowercase().contains("name or service not known")
+            || e.msg
+                .to_lowercase()
+                .contains("failed to lookup address information")
     }) {
         return stock_str::error_no_network(context).await;
     }

src/context.rs

@@ -193,7 +193,7 @@ impl Context {
             translated_stockstrings: RwLock::new(HashMap::new()),
             events,
             scheduler: RwLock::new(None),
-            ratelimit: RwLock::new(Ratelimit::new(Duration::new(60, 0), 3.0)), // Allow to send 3 messages immediately, no more than once every 20 seconds.
+            ratelimit: RwLock::new(Ratelimit::new(Duration::new(60, 0), 6.0)), // Allow to send 6 messages immediately, no more than once every 10 seconds.
             quota: RwLock::new(None),
             server_id: RwLock::new(None),
             creation_time: std::time::SystemTime::now(),
@@ -594,7 +594,7 @@ impl Context {
 
         let list = if let Some(chat_id) = chat_id {
             do_query(
-                "SELECT m.id AS id, m.timestamp AS timestamp
+                "SELECT m.id AS id
                  FROM msgs m
                  LEFT JOIN contacts ct
                         ON m.from_id=ct.id
@@ -618,7 +618,7 @@ impl Context {
             // According to some tests, this limit speeds up eg. 2 character searches by factor 10.
             // The limit is documented and UI may add a hint when getting 1000 results.
             do_query(
-                "SELECT m.id AS id, m.timestamp AS timestamp
+                "SELECT m.id AS id
                  FROM msgs m
                  LEFT JOIN contacts ct
                         ON m.from_id=ct.id

src/download.rs

@@ -11,7 +11,7 @@ use crate::imap::{Imap, ImapActionResult};
 use crate::job::{self, Action, Job, Status};
 use crate::message::{Message, MsgId, Viewtype};
 use crate::mimeparser::{MimeMessage, Part};
-use crate::param::Params;
+use crate::param::{Param, Params};
 use crate::tools::time;
 use crate::{job_try, stock_str, EventType};
 use std::cmp::max;
@@ -69,6 +69,42 @@ impl Context {
             Ok(Some(max(MIN_DOWNLOAD_LIMIT, download_limit as u32)))
         }
     }
+
+    // Merges the two messages to `placeholder_msg_id`;
+    // `full_msg_id` is no longer used afterwards.
+    pub(crate) async fn merge_messages(
+        &self,
+        full_msg_id: MsgId,
+        placeholder_msg_id: MsgId,
+    ) -> Result<()> {
+        let placeholder = Message::load_from_db(self, placeholder_msg_id).await?;
+        self.sql
+            .transaction(move |transaction| {
+                transaction
+                    .execute("DELETE FROM msgs WHERE id=?;", paramsv![placeholder_msg_id])?;
+                transaction.execute(
+                    "UPDATE msgs SET id=? WHERE id=?",
+                    paramsv![placeholder_msg_id, full_msg_id],
+                )?;
+                Ok(())
+            })
+            .await?;
+        let mut full = Message::load_from_db(self, placeholder_msg_id).await?;
+
+        for key in [
+            Param::WebxdcSummary,
+            Param::WebxdcSummaryTimestamp,
+            Param::WebxdcDocument,
+            Param::WebxdcDocumentTimestamp,
+        ] {
+            if let Some(value) = placeholder.param.get(key) {
+                full.param.set(key, value);
+            }
+        }
+        full.update_param(self).await?;
+
+        Ok(())
+    }
 }
 
 impl MsgId {
@@ -256,7 +292,7 @@ impl MimeMessage {
 mod tests {
     use num_traits::FromPrimitive;
 
-    use crate::chat::send_msg;
+    use crate::chat::{get_chat_msgs, send_msg};
     use crate::ephemeral::Timer;
     use crate::message::Viewtype;
     use crate::receive_imf::receive_imf_inner;
@@ -410,4 +446,119 @@ mod tests {
 
         Ok(())
     }
+
+    #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
+    async fn test_status_update_expands_to_nothing() -> Result<()> {
+        let alice = TestContext::new_alice().await;
+        let bob = TestContext::new_bob().await;
+        let chat_id = alice.create_chat(&bob).await.id;
+
+        let file = alice.get_blobdir().join("minimal.xdc");
+        tokio::fs::write(&file, include_bytes!("../test-data/webxdc/minimal.xdc")).await?;
+        let mut instance = Message::new(Viewtype::File);
+        instance.set_file(file.to_str().unwrap(), None);
+        let _sent1 = alice.send_msg(chat_id, &mut instance).await;
+
+        alice
+            .send_webxdc_status_update(instance.id, r#"{"payload":7}"#, "d")
+            .await?;
+        alice.flush_status_updates().await?;
+        let sent2 = alice.pop_sent_msg().await;
+        let sent2_rfc742_mid = Message::load_from_db(&alice, sent2.sender_msg_id)
+            .await?
+            .rfc724_mid;
+
+        // not downloading the status update results in an placeholder
+        receive_imf_inner(
+            &bob,
+            &sent2_rfc742_mid,
+            sent2.payload().as_bytes(),
+            false,
+            Some(sent2.payload().len() as u32),
+            false,
+        )
+        .await?;
+        let msg = bob.get_last_msg().await;
+        let chat_id = msg.chat_id;
+        assert_eq!(get_chat_msgs(&bob, chat_id, 0).await?.len(), 1);
+        assert_eq!(msg.download_state(), DownloadState::Available);
+
+        // downloading the status update afterwards expands to nothing and moves the placeholder to trash-chat
+        // (usually status updates are too small for not being downloaded directly)
+        receive_imf_inner(
+            &bob,
+            &sent2_rfc742_mid,
+            sent2.payload().as_bytes(),
+            false,
+            None,
+            false,
+        )
+        .await?;
+        assert_eq!(get_chat_msgs(&bob, chat_id, 0).await?.len(), 0);
+        assert!(Message::load_from_db(&bob, msg.id)
+            .await?
+            .chat_id
+            .is_trash());
+
+        Ok(())
+    }
+
+    #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
+    async fn test_mdn_expands_to_nothing() -> Result<()> {
+        let bob = TestContext::new_bob().await;
+        let raw = b"Subject: Message opened\n\
+            Date: Mon, 10 Jan 2020 00:00:00 +0000\n\
+            Chat-Version: 1.0\n\
+            Message-ID: <bar@example.org>\n\
+            To: Alice <alice@example.org>\n\
+            From: Bob <bob@example.org>\n\
+            Content-Type: multipart/report; report-type=disposition-notification;\n\t\
+            boundary=\"kJBbU58X1xeWNHgBtTbMk80M5qnV4N\"\n\
+            \n\
+            \n\
+            --kJBbU58X1xeWNHgBtTbMk80M5qnV4N\n\
+            Content-Type: text/plain; charset=utf-8\n\
+            \n\
+            bla\n\
+            \n\
+            \n\
+            --kJBbU58X1xeWNHgBtTbMk80M5qnV4N\n\
+            Content-Type: message/disposition-notification\n\
+            \n\
+            Reporting-UA: Delta Chat 1.88.0\n\
+            Original-Recipient: rfc822;bob@example.org\n\
+            Final-Recipient: rfc822;bob@example.org\n\
+            Original-Message-ID: <foo@example.org>\n\
+            Disposition: manual-action/MDN-sent-automatically; displayed\n\
+            \n\
+            \n\
+            --kJBbU58X1xeWNHgBtTbMk80M5qnV4N--\n\
+            ";
+
+        // not downloading the mdn results in an placeholder
+        receive_imf_inner(
+            &bob,
+            "bar@example.org",
+            raw,
+            false,
+            Some(raw.len() as u32),
+            false,
+        )
+        .await?;
+        let msg = bob.get_last_msg().await;
+        let chat_id = msg.chat_id;
+        assert_eq!(get_chat_msgs(&bob, chat_id, 0).await?.len(), 1);
+        assert_eq!(msg.download_state(), DownloadState::Available);
+
+        // downloading the mdn afterwards expands to nothing and deletes the placeholder directly
+        // (usually mdn are too small for not being downloaded directly)
+        receive_imf_inner(&bob, "bar@example.org", raw, false, None, false).await?;
+        assert_eq!(get_chat_msgs(&bob, chat_id, 0).await?.len(), 0);
+        assert!(Message::load_from_db(&bob, msg.id)
+            .await?
+            .chat_id
+            .is_trash());
+
+        Ok(())
+    }
 }

src/peerstate.rs

@@ -4,12 +4,13 @@ use std::collections::HashSet;
 use std::fmt;
 
 use crate::aheader::{Aheader, EncryptPreference};
-use crate::chat::{self, is_contact_in_chat, Chat};
+use crate::chat::{self, is_contact_in_chat, Chat, ChatId};
 use crate::chatlist::Chatlist;
 use crate::constants::Chattype;
 use crate::contact::{addr_cmp, Contact, Origin};
 use crate::context::Context;
 use crate::events::EventType;
+use crate::headerdef::HeaderDef;
 use crate::key::{DcKey, Fingerprint, SignedPublicKey};
 use crate::message::Message;
 use crate::mimeparser::SystemMessage;
@@ -521,16 +522,6 @@ impl Peerstate {
                 PeerstateChange::FingerprintChange => {
                     stock_str::contact_setup_changed(context, self.addr.clone()).await
                 }
-                PeerstateChange::Aeap(new_addr) => {
-                    let old_contact = Contact::load_from_db(context, contact_id).await?;
-                    stock_str::aeap_addr_changed(
-                        context,
-                        old_contact.get_display_name(),
-                        &self.addr,
-                        new_addr,
-                    )
-                    .await
-                }
             };
             let timestamp_sort = if let Some(msg_id) = msg_id {
                 let lastmsg = Message::load_from_db(context, *msg_id).await?;
@@ -556,22 +547,6 @@ impl Peerstate {
                 None,
             )
             .await?;
-
-            if let PeerstateChange::Aeap(new_addr) = &change {
-                let chat = Chat::load_from_db(context, *chat_id).await?;
-                if chat.typ == Chattype::Group || chat.typ == Chattype::Broadcast {
-                    chat::remove_from_chat_contacts_table(context, *chat_id, contact_id).await?;
-
-                    let (new_contact_id, _) =
-                        Contact::add_or_lookup(context, "", new_addr, Origin::IncomingUnknownFrom)
-                            .await?;
-                    if !is_contact_in_chat(context, *chat_id, new_contact_id).await? {
-                        chat::add_to_chat_contacts_table(context, *chat_id, new_contact_id).await?;
-                    }
-
-                    context.emit_event(EventType::ChatModified(*chat_id));
-                }
-            }
         }
 
         Ok(())
@@ -621,15 +596,63 @@ pub async fn maybe_do_aeap_transition(
                 && mime_parser.from_is_signed
                 && info.message_time > peerstate.last_seen
             {
-                // Add an info messages to all chats with this contact
-                //
-                peerstate
-                    .handle_setup_change(
-                        context,
-                        info.message_time,
-                        PeerstateChange::Aeap(info.from.clone()),
+                // Add an info messages to the chat
+                let contact_id = context
+                    .sql
+                    .query_get_value(
+                        "SELECT id FROM contacts WHERE addr=? COLLATE NOCASE;",
+                        paramsv![peerstate.addr],
                     )
-                    .await?;
+                    .await?
+                    .with_context(|| {
+                        format!("contact with addr {:?} not found", &peerstate.addr)
+                    })?;
+
+                let old_contact = Contact::load_from_db(context, contact_id).await?;
+                let msg = stock_str::aeap_addr_changed(
+                    context,
+                    old_contact.get_display_name(),
+                    &peerstate.addr,
+                    &from.addr,
+                )
+                .await;
+
+                let grpid = match mime_parser.get_header(HeaderDef::ChatGroupId) {
+                    Some(h) => h,
+                    None => return Ok(()),
+                };
+                let (chat_id, protected, _blocked) =
+                    match chat::get_chat_id_by_grpid(context, grpid).await? {
+                        Some(s) => s,
+                        None => return Ok(()),
+                    };
+
+                if protected && !peerstate.has_verified_key(&mime_parser.signatures) {
+                    return Ok(());
+                }
+
+                chat::add_info_msg(context, chat_id, &msg, info.message_time).await?;
+
+                let (new_contact_id, _) =
+                    Contact::add_or_lookup(context, "", &from.addr, Origin::IncomingUnknownFrom)
+                        .await?;
+
+                let chat = Chat::load_from_db(context, chat_id).await?;
+                if chat.typ == Chattype::Group || chat.typ == Chattype::Broadcast {
+                    chat::remove_from_chat_contacts_table(context, chat_id, contact_id).await?;
+
+                    if !is_contact_in_chat(context, chat_id, new_contact_id).await? {
+                        chat::add_to_chat_contacts_table(context, chat_id, new_contact_id).await?;
+                    }
+
+                    context.emit_event(EventType::ChatModified(chat_id));
+                }
+
+                // Create a chat with the new address with the same blocked-level as the old chat
+                let new_chat_id =
+                    ChatId::create_for_contact_with_blocked(context, new_contact_id, chat.blocked)
+                        .await?;
+                chat::add_info_msg(context, new_chat_id, &msg, info.message_time).await?;
 
                 peerstate.addr = info.from.clone();
                 let header = info.autocrypt_header.as_ref().context(
@@ -655,9 +678,9 @@ enum PeerstateChange {
     /// The contact's public key fingerprint changed, likely because
     /// the contact uses a new device and didn't transfer their key.
     FingerprintChange,
-    /// The contact changed their address to the given new address
-    /// (Automatic Email Address Porting).
-    Aeap(String),
+    // /// The contact changed their address to the given new address
+    // /// (Automatic Email Address Porting).
+    // Aeap(String), (currently unused)
 }
 
 /// Removes duplicate peerstates from `acpeerstates` database table.

src/provider/data.rs

@@ -9,20 +9,32 @@ use std::collections::HashMap;
 use once_cell::sync::Lazy;
 
 // 163.md: 163.com
-static P_163: Lazy<Provider> = Lazy::new(|| {
-    Provider {
+static P_163: Lazy<Provider> = Lazy::new(|| Provider {
     id: "163",
-    status: Status::Broken,
-    before_login_hint: "163 Mail does not work since it forces the email clients to connect with an IMAP ID, which is currently not the case of Delta Chat.",
+    status: Status::Ok,
+    before_login_hint: "",
     after_login_hint: "",
     overview_page: "https://providers.delta.chat/163",
     server: vec![
+        Server {
+            protocol: Imap,
+            socket: Ssl,
+            hostname: "imap.163.com",
+            port: 993,
+            username_pattern: Email,
+        },
+        Server {
+            protocol: Smtp,
+            socket: Ssl,
+            hostname: "smtp.163.com",
+            port: 465,
+            username_pattern: Email,
+        },
     ],
     config_defaults: None,
     strict_tls: true,
     max_smtp_rcpt_to: None,
     oauth2_authorizer: None,
-}
 });
 
 // aktivix.org.md: aktivix.org
@@ -1879,4 +1891,4 @@ pub(crate) static PROVIDER_IDS: Lazy<HashMap<&'static str, &'static Provider>> =
 });
 
 pub static PROVIDER_UPDATED: Lazy<chrono::NaiveDate> =
-    Lazy::new(|| chrono::NaiveDate::from_ymd(2022, 6, 4));
+    Lazy::new(|| chrono::NaiveDate::from_ymd(2022, 7, 5));

src/receive_imf.rs

@@ -130,15 +130,14 @@ pub(crate) async fn receive_imf_inner(
                     context,
                     "Message already partly in DB, replacing by full message."
                 );
-                old_msg_id.delete_from_db(context).await?;
-                true
+                Some(old_msg_id)
             } else {
                 // the message was probably moved around.
                 info!(context, "Message already in DB, doing nothing.");
                 return Ok(None);
             }
         } else {
-            false
+            None
         };
 
     // the function returns the number of created messages in the database
@@ -189,8 +188,9 @@ pub(crate) async fn receive_imf_inner(
         sent_timestamp,
         rcvd_timestamp,
         from_id,
-        seen || replace_partial_download,
+        seen || replace_partial_download.is_some(),
         is_partial_download,
+        replace_partial_download,
         fetching_existing_messages,
         prevent_rename,
     )
@@ -322,7 +322,7 @@ pub(crate) async fn receive_imf_inner(
         }
     }
 
-    if replace_partial_download {
+    if replace_partial_download.is_some() {
         context.emit_msgs_changed(chat_id, MsgId::new(0));
     } else if !chat_id.is_trash() {
         let fresh = received_msg.state == MessageState::InFresh;
@@ -401,6 +401,7 @@ async fn add_parts(
     from_id: ContactId,
     seen: bool,
     is_partial_download: Option<u32>,
+    replace_msg_id: Option<MsgId>,
     fetching_existing_messages: bool,
     prevent_rename: bool,
 ) -> Result<ReceivedMsg> {
@@ -1145,6 +1146,17 @@ INSERT INTO msgs
     }
     drop(conn);
 
+    if let Some(replace_msg_id) = replace_msg_id {
+        if let Some(created_msg_id) = created_db_entries.pop() {
+            context
+                .merge_messages(created_msg_id, replace_msg_id)
+                .await?;
+            created_db_entries.push(replace_msg_id);
+        } else {
+            replace_msg_id.delete_from_db(context).await?;
+        }
+    }
+
     chat_id.unarchive_if_not_muted(context).await?;
 
     info!(

src/tests/aeap.rs

@@ -69,24 +69,24 @@ Message w/out In-Reply-To
 }
 
 enum ChatForTransition {
-    OneToOne,
+    // OneToOne,
     GroupChat,
     VerifiedGroup,
 }
 use ChatForTransition::*;
 
-#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
-async fn test_aeap_transition_0() {
-    check_aeap_transition(OneToOne, false, false).await;
-}
+// #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
+// async fn test_aeap_transition_0() {
+//     check_aeap_transition(OneToOne, false, false).await;
+// }
 #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
 async fn test_aeap_transition_1() {
     check_aeap_transition(GroupChat, false, false).await;
 }
-#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
-async fn test_aeap_transition_0_verified() {
-    check_aeap_transition(OneToOne, true, false).await;
-}
+// #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
+// async fn test_aeap_transition_0_verified() {
+//     check_aeap_transition(OneToOne, true, false).await;
+// }
 #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
 async fn test_aeap_transition_1_verified() {
     check_aeap_transition(GroupChat, true, false).await;
@@ -96,18 +96,18 @@ async fn test_aeap_transition_2_verified() {
     check_aeap_transition(VerifiedGroup, true, false).await;
 }
 
-#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
-async fn test_aeap_transition_0_bob_knew_new_addr() {
-    check_aeap_transition(OneToOne, false, true).await;
-}
+// #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
+// async fn test_aeap_transition_0_bob_knew_new_addr() {
+//     check_aeap_transition(OneToOne, false, true).await;
+// }
 #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
 async fn test_aeap_transition_1_bob_knew_new_addr() {
     check_aeap_transition(GroupChat, false, true).await;
 }
-#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
-async fn test_aeap_transition_0_verified_bob_knew_new_addr() {
-    check_aeap_transition(OneToOne, true, true).await;
-}
+// #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
+// async fn test_aeap_transition_0_verified_bob_knew_new_addr() {
+//     check_aeap_transition(OneToOne, true, true).await;
+// }
 #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
 async fn test_aeap_transition_1_verified_bob_knew_new_addr() {
     check_aeap_transition(GroupChat, true, true).await;
@@ -206,7 +206,7 @@ async fn check_aeap_transition(
     tcm.section("Alice sends another message to Bob, this time from her new addr");
     // No matter which chat Alice sends to, the transition should be done in all groups
     let chat_to_send = match chat_for_transition {
-        OneToOne => alice.create_chat(&bob).await.id,
+        // OneToOne => alice.create_chat(&bob).await.id,
         GroupChat => group1_alice,
         VerifiedGroup => group3_alice.expect("No verified group"),
     };
@@ -219,7 +219,8 @@ async fn check_aeap_transition(
 
     tcm.section("Check that the AEAP transition worked");
     check_that_transition_worked(
-        &groups,
+        // The transition is only done in the chat where the message was sent for now:
+        &[recvd.chat_id],
         &alice,
         "alice@example.org",
         ALICE_NEW_ADDR,
@@ -246,7 +247,8 @@ async fn check_aeap_transition(
     assert_eq!(recvd.text.unwrap(), "Hello from my old addr!");
 
     check_that_transition_worked(
-        &groups,
+        // The transition is only done in the chat where the message was sent for now:
+        &[recvd.chat_id],
         &alice,
         // Note that "alice@example.org" and ALICE_NEW_ADDR are switched now:
         ALICE_NEW_ADDR,

src/webxdc.rs

@@ -14,6 +14,7 @@ use tokio::io::AsyncReadExt;
 use crate::chat::Chat;
 use crate::contact::ContactId;
 use crate::context::Context;
+use crate::download::DownloadState;
 use crate::message::{Message, MessageState, MsgId, Viewtype};
 use crate::mimeparser::SystemMessage;
 use crate::param::Param;
@@ -330,10 +331,12 @@ impl Context {
 
         let status_update_serial = StatusUpdateSerial(u32::try_from(rowid)?);
 
-        self.emit_event(EventType::WebxdcStatusUpdate {
-            msg_id: instance.id,
-            status_update_serial,
-        });
+        if instance.viewtype == Viewtype::Webxdc {
+            self.emit_event(EventType::WebxdcStatusUpdate {
+                msg_id: instance.id,
+                status_update_serial,
+            });
+        }
 
         Ok(status_update_serial)
     }
@@ -475,6 +478,8 @@ impl Context {
         } else if let Some(parent) = msg.parent(self).await? {
             if parent.viewtype == Viewtype::Webxdc {
                 (msg.timestamp_sort, parent, true)
+            } else if parent.download_state() != DownloadState::Done {
+                (msg.timestamp_sort, parent, false)
             } else {
                 bail!("receive_status_update: message is not the child of a webxdc message.")
             }
@@ -730,7 +735,7 @@ mod tests {
     use crate::chatlist::Chatlist;
     use crate::config::Config;
     use crate::contact::Contact;
-    use crate::receive_imf::receive_imf;
+    use crate::receive_imf::{receive_imf, receive_imf_inner};
     use crate::test_utils::TestContext;
 
     use super::*;
@@ -1038,6 +1043,71 @@ mod tests {
         Ok(())
     }
 
+    #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
+    async fn test_webxdc_update_for_not_downloaded_instance() -> Result<()> {
+        // Alice sends a larger instance and an update
+        let alice = TestContext::new_alice().await;
+        let bob = TestContext::new_bob().await;
+        let chat = alice.create_chat(&bob).await;
+        bob.set_config(Config::DownloadLimit, Some("40000")).await?;
+        let mut alice_instance = create_webxdc_instance(
+            &alice,
+            "chess.xdc",
+            include_bytes!("../test-data/webxdc/chess.xdc"),
+        )
+        .await?;
+        let sent1 = alice.send_msg(chat.id, &mut alice_instance).await;
+        let alice_instance = Message::load_from_db(&alice, sent1.sender_msg_id).await?;
+        alice
+            .send_webxdc_status_update(
+                alice_instance.id,
+                r#"{"payload": 7, "summary":"sum", "document":"doc"}"#,
+                "bla",
+            )
+            .await?;
+        alice.flush_status_updates().await?;
+        let sent2 = alice.pop_sent_msg().await;
+
+        // Bob does not download instance but already receives update
+        receive_imf_inner(
+            &bob,
+            &alice_instance.rfc724_mid,
+            sent1.payload().as_bytes(),
+            false,
+            Some(70790),
+            false,
+        )
+        .await?;
+        let bob_instance = bob.get_last_msg().await;
+        bob_instance.chat_id.accept(&bob).await?;
+        bob.recv_msg(&sent2).await;
+        assert_eq!(bob_instance.download_state, DownloadState::Available);
+
+        // Bob downloads instance, updates should be assigned correctly
+        receive_imf_inner(
+            &bob,
+            &alice_instance.rfc724_mid,
+            sent1.payload().as_bytes(),
+            false,
+            None,
+            false,
+        )
+        .await?;
+        let bob_instance = bob.get_last_msg().await;
+        assert_eq!(bob_instance.viewtype, Viewtype::Webxdc);
+        assert_eq!(bob_instance.download_state, DownloadState::Done);
+        assert_eq!(
+            bob.get_webxdc_status_updates(bob_instance.id, StatusUpdateSerial(0))
+                .await?,
+            r#"[{"payload":7,"document":"doc","summary":"sum","serial":1,"max_serial":1}]"#
+        );
+        let info = bob_instance.get_webxdc_info(&bob).await?;
+        assert_eq!(info.document, "doc");
+        assert_eq!(info.summary, "sum");
+
+        Ok(())
+    }
+
     #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
     async fn test_delete_webxdc_instance() -> Result<()> {
         let t = TestContext::new_alice().await;