fix a few more places that used format! with sql-queries -- although they are all probably not critical

Revert "Enable clippy::unreadable_literal error"
This reverts commit b424ef9ebb.
2026-04-02 13:32:11 +03:00 · 2019-12-14 18:28:59 +01:00 · 2019-12-14 15:55:33 +00:00 · 2019-12-14 15:20:58 +00:00 · 2019-12-14 15:20:58 +00:00 · 2019-12-14 15:20:58 +00:00
68 changed files with 3120 additions and 1891 deletions
--- a/.circleci/config.yml
+++ b/.circleci/config.yml
@@ -36,12 +36,6 @@ jobs:
    executor: default
    steps:
      - checkout
-      - run:
-          name: Update submodules
-          command: git submodule update --init --recursive
-      - run:
-          name: Calculate dependencies
-          command: cargo generate-lockfile
      - restore_cache:
          keys:
            - cargo-v3-{{ checksum "rust-toolchain" }}-{{ checksum "Cargo.toml" }}-{{ checksum "Cargo.lock" }}-{{ arch }}
@@ -49,7 +43,6 @@ jobs:
      - run: rustup default $(cat rust-toolchain)
      - run: rustup component add --toolchain $(cat rust-toolchain) rustfmt
      - run: rustup component add --toolchain $(cat rust-toolchain) clippy-preview
-      - run: cargo update
      - run: cargo fetch
      - run: rustc +stable --version
      - run: rustc +$(cat rust-toolchain) --version
@@ -91,7 +84,6 @@ jobs:
            curl https://sh.rustup.rs -sSf | sh -s -- -y
      - run: rustup install $(cat rust-toolchain)
      - run: rustup default $(cat rust-toolchain)
-      - run: cargo update
      - run: cargo fetch
      - run:
          name: Test
@@ -187,7 +179,7 @@ jobs:
      - *restore-cache
      - run:
          name: Run cargo clippy
-          command: cargo clippy --all
+          command: cargo clippy 


 workflows:
@@ -195,7 +187,7 @@ workflows:

  test:
    jobs:
-      - cargo_fetch
+      # - cargo_fetch

      - remote_tests_rust

@@ -205,12 +197,12 @@ workflows:
      #     requires:
      #       - build_test_docs_wheel
      #       - build_doxygen
-      - rustfmt:
-          requires:
-            - cargo_fetch
-      - clippy:
-          requires:
-           - cargo_fetch
+      # - rustfmt:
+      #     requires:
+      #       - cargo_fetch
+      # - clippy:
+      #     requires:
+      #      - cargo_fetch

      - build_doxygen

--- a/.github/workflows/code-quality.yml
+++ b/.github/workflows/code-quality.yml
@@ -0,0 +1,47 @@
+on: push
+name: Code Quality
+jobs:
+
+  check:
+    name: Check
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v1
+      - uses: actions-rs/toolchain@v1
+        with:
+          profile: minimal
+          toolchain: nightly-2019-11-06
+          override: true
+      - uses: actions-rs/cargo@v1
+        with:
+          command: check
+
+  fmt:
+    name: Rustfmt
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v1
+      - uses: actions-rs/toolchain@v1
+        with:
+          profile: minimal
+          toolchain: nightly-2019-11-06
+          override: true
+      - run: rustup component add rustfmt
+      - uses: actions-rs/cargo@v1
+        with:
+          command: fmt
+          args: --all -- --check
+
+  run_clippy:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v1
+      - uses: actions-rs/toolchain@v1
+        with:
+            toolchain: nightly-2019-11-06
+            components: clippy
+            override: true
+      - uses: actions-rs/clippy-check@v1
+        with:
+          token: ${{ secrets.GITHUB_TOKEN }}
+          args: --all-features
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -1,6 +1,86 @@
 # Changelog 

-## 1.0.0-beta.10 (pending)
+## 1.0.0-beta.16
+
+- alleviate login problems with providers which only
+  support RSA1024 keys by switching back from Rustls 
+  to native-tls, by using the new async-email/async-native-tls 
+  crate from @dignifiedquire. thanks @link2xt. 
+
+- introduce per-contact profile images to send out 
+  own profile image heuristically, and fix sending
+  out of profile images in "in-prepare" groups. 
+  this also extends the Chat-spec that is maintained
+  in core to specify Chat-Group-Image and Chat-Group-Avatar
+  headers. thanks @r10s and @hpk42.
+
+- fix merging of protected headers from the encrypted
+  to the unencrypted parts, now not happening recursively
+  anymore.  thanks @hpk and @r10s
+
+- fix/optimize autocrypt gossip headers to only get 
+  sent when there are more than 2 people in a chat. 
+  thanks @link2xt
+
+- fix displayname to use the authenticated name 
+  when available (displayname as coming from contacts 
+  themselves). thanks @simon-laux
+
+- introduce preliminary support for offline autoconfig 
+  for nauta provider. thanks @hpk42 @r10s
+
+## 1.0.0-beta.15
+
+- fix #994 attachment appeared doubled in chats (and where actually
+  downloaded after smtp-send). @hpk42
+
+## 1.0.0-beta.14
+
+- fix packaging issue with our rust-email fork, now we are tracking
+  master again there. hpk42
+
+## 1.0.0-beta.13
+
+- fix #976 -- unicode-issues in display-name of email addresses. @hpk42
+
+- fix #985 group add/remove member bugs resulting in broken groups.  @hpk42
+
+- fix hanging IMAP connections -- we now detect with a 15second timeout
+  if we cannot terminate the IDLE IMAP protocol. @hpk42 @link2xt
+
+- fix incoming multipart/mixed containing html, to show up as
+  attachments again.  Fixes usage for simplebot which sends html
+  files for users to interact with the bot. @adbenitez @hpk42 
+
+- refinements to internal autocrypt-handling code, do not send
+  prefer-encrypt=nopreference as it is the default if no attribute
+  is present.  @linkxt 
+
+- simplify, modularize and rustify several parts 
+  of dc-core (general WIP). @link2xt @flub @hpk42 @r10s
+
+- use async-email/async-smtp to handle SMTP connections, might
+  fix connection/reconnection issues. @link2xt 
+
+- more tests and refinements for dealing with blobstorage @flub @hpk42 
+
+- use a dedicated build-server for CI testing of core PRs
+
+
+## 1.0.0-beta.12
+
+- fix python bindings to use core for copying attachments to blobdir
+  and fix core to actually do it. @hpk42
+
+## 1.0.0-beta.11
+
+- trigger reconnect more often on imap error states.  Should fix an 
+  issue observed when trying to empty a folder.  @hpk42
+
+- un-split qr tests: we fixed qr-securejoin protocol flakyness 
+  last weeks. @hpk42
+
+## 1.0.0-beta.10

 - fix grpid-determination from in-reply-to and references headers. @hpk42

@@ -10,6 +90,11 @@

 - remove last unsafe code from dc_receive_imf :) @hpk42

+- add experimental new dc_chat_get_info_json FFI/API so that desktop devs
+  can play with using it. @jikstra
+
+- fix encoding of subjects and attachment-filenames @hpk42
+  @dignifiedquire . 

 ## 1.0.0-beta.9

--- a/Cargo.lock
+++ b/Cargo.lock
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -1,6 +1,6 @@
 [package]
 name = "deltachat"
-version = "1.0.0-beta.9"
+version = "1.0.0-beta.16"
 authors = ["Delta Chat Developers (ML) <delta@codespeak.net>"]
 edition = "2018"
 license = "MPL"
@@ -9,18 +9,19 @@ license = "MPL"
 deltachat_derive = { path = "./deltachat_derive" }

 libc = "0.2.51"
-pgp = { git = "https://github.com/rpgp/rpgp", branch = "master", default-features = false }
+pgp = { version = "0.4.0", default-features = false }
 hex = "0.4.0"
 sha2 = "0.8.0"
-rand = "0.6.5"
-smallvec = "0.6.9"
-reqwest = { version = "0.9.15", default-features = false, features = ["rustls-tls"] }
-num-derive = "0.2.5"
+rand = "0.7.0"
+smallvec = "1.0.0"
+reqwest = { version = "0.9.15" }
+num-derive = "0.3.0"
 num-traits = "0.2.6"
-lettre = { git = "https://github.com/deltachat/lettre", branch = "feat/mail" }
-lettre_email = { git = "https://github.com/deltachat/lettre", branch = "feat/mail" }
-async-imap = { git = "https://github.com/async-email/async-imap", branch="master" }
-async-tls = "0.6"
+async-smtp = { git = "https://github.com/async-email/async-smtp", branch = "master" }
+email = { git = "https://github.com/deltachat/rust-email", branch = "master" }
+lettre_email = { git = "https://github.com/deltachat/lettre", branch = "native_tls" }
+async-imap = { git = "https://github.com/async-email/async-imap", branch="native_tls", default-features = false, features = ["tls_native"] }
+async-native-tls = "0.1.1"
 async-std = { version = "1.0", features = ["unstable"] }
 base64 = "0.11"
 charset = "0.1"
@@ -30,6 +31,7 @@ serde_json = "1.0"
 chrono = "0.4.6"
 failure = "0.1.5"
 failure_derive = "0.1.5"
+indexmap = "1.3.0"
 # TODO: make optional
 rustyline = "4.1.0"
 lazy_static = "1.4.0"
@@ -44,17 +46,15 @@ backtrace = "0.3.33"
 byteorder = "1.3.1"
 itertools = "0.8.0"
 image-meta = "0.1.0"
-quick-xml = "0.15.0"
+quick-xml = "0.17.1"
 escaper = "0.1.0"
 bitflags = "1.1.0"
 debug_stub_derive = "0.3.0"
 sanitize-filename = "0.2.1"
 stop-token = { version = "0.1.1", features = ["unstable"] }
-rustls = "0.16.0"
-webpki-roots = "0.18.0"
-webpki = "0.21.0"
 mailparse = "0.10.1"
 encoded-words = { git = "https://github.com/async-email/encoded-words", branch="master" }
+native-tls = "0.2.3"

 [dev-dependencies]
 tempfile = "3.0"
@@ -79,6 +79,6 @@ path = "examples/repl/main.rs"

 [features]
 default = ["nightly", "ringbuf"]
-vendored = []
+vendored = ["native-tls/vendored", "reqwest/default-tls-vendored"]
 nightly = ["pgp/nightly"]
 ringbuf = ["pgp/ringbuf"]
--- a/appveyor.yml
+++ b/appveyor.yml
@@ -8,7 +8,6 @@ install:
  - set PATH=%PATH%;%USERPROFILE%\.cargo\bin
  - rustc -vV
  - cargo -vV
-  - cargo update

 build: false

--- a/assets/icon-device.png
+++ b/assets/icon-device.png
--- a/assets/icon-device.svg
+++ b/assets/icon-device.svg
@@ -7,42 +7,42 @@
   xmlns="http://www.w3.org/2000/svg"
   xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
   xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
-   inkscape:version="0.92.4 5da689c313, 2019-01-14"
-   sodipodi:docname="icon-device.svg"
-   id="svg4344"
-   preserveAspectRatio="xMidYMid meet"
-   viewBox="0 0 45 45"
-   height="60"
-   width="60"
-   version="1.0"
-   inkscape:export-filename="/home/kerle/Workspace/deltachat-core-rust/assets/icon-device.png"
+   inkscape:export-ydpi="409.60001"
   inkscape:export-xdpi="409.60001"
-   inkscape:export-ydpi="409.60001">
+   inkscape:export-filename="/Users/bpetersen/projects/deltachat-core-rust/assets/icon-device.png"
+   version="1.0"
+   width="60"
+   height="60"
+   viewBox="0 0 45 45"
+   preserveAspectRatio="xMidYMid meet"
+   id="svg4344"
+   sodipodi:docname="icon-device.svg"
+   inkscape:version="1.0beta1 (32d4812, 2019-09-19)">
  <defs
     id="defs4348" />
  <sodipodi:namedview
-     inkscape:current-layer="svg4344"
-     inkscape:window-maximized="0"
-     inkscape:window-y="0"
-     inkscape:window-x="0"
-     inkscape:cy="28.471578"
-     inkscape:cx="28.03926"
-     inkscape:zoom="5.6"
-     units="px"
-     showgrid="false"
-     id="namedview4346"
-     inkscape:window-height="1019"
-     inkscape:window-width="1248"
-     inkscape:pageshadow="2"
-     inkscape:pageopacity="0"
-     guidetolerance="10"
-     gridtolerance="10"
-     objecttolerance="10"
-     borderopacity="1"
-     inkscape:document-rotation="0"
-     bordercolor="#666666"
+     inkscape:snap-global="false"
     pagecolor="#ffffff"
-     inkscape:snap-global="false" />
+     bordercolor="#666666"
+     inkscape:document-rotation="0"
+     borderopacity="1"
+     objecttolerance="10"
+     gridtolerance="10"
+     guidetolerance="10"
+     inkscape:pageopacity="0"
+     inkscape:pageshadow="2"
+     inkscape:window-width="1600"
+     inkscape:window-height="1035"
+     id="namedview4346"
+     showgrid="false"
+     units="px"
+     inkscape:zoom="3.959798"
+     inkscape:cx="28.322498"
+     inkscape:cy="24.898474"
+     inkscape:window-x="45"
+     inkscape:window-y="23"
+     inkscape:window-maximized="0"
+     inkscape:current-layer="svg4344" />
  <metadata
     id="metadata4336">
 Created by potrace 1.15, written by Peter Selinger 2001-2017
@@ -56,35 +56,28 @@ Created by potrace 1.15, written by Peter Selinger 2001-2017
  </cc:Work>
 </rdf:RDF>
 </metadata>
-  <circle
-     style="opacity:1;fill:#ffffff;fill-opacity:1;stroke-width:0.731071"
-     id="path859"
-     cx="22.5"
-     cy="22.5"
-     r="22.5" />
-  <circle
-     r="22.5"
-     cy="22.5"
-     cx="22.5"
-     id="path4917"
-     style="fill:#364e54;fill-opacity:0.67681497;stroke-width:0.58855402"
-     inkscape:export-xdpi="409"
-     inkscape:export-ydpi="409" />
+  <rect
+     y="-4.4408921e-16"
+     x="0"
+     height="45"
+     width="45"
+     id="rect860"
+     style="opacity:1;fill:#76868b;fill-opacity:1;stroke-width:0.819271" />
  <g
-     id="g4342"
-     transform="matrix(0.00255113,0,0,-0.00255113,5.586152,38.200477)"
-     style="fill:#ffffff;fill-opacity:1"
+     fill="#000000"
     stroke="none"
-     fill="#000000">
+     style="fill:#ffffff;fill-opacity:1"
+     transform="matrix(0.00255113,0,0,-0.00255113,5.586152,38.200477)"
+     id="g4342">
    <path
-       inkscape:connector-curvature="0"
-       id="path4338"
+       style="fill:#ffffff;fill-opacity:1"
       d="m 8175,12765 c -703,-114 -1248,-608 -1387,-1258 -17,-82 -21,-136 -22,-277 0,-202 15,-307 70,-470 149,-446 499,-733 1009,-828 142,-26 465,-23 619,6 691,131 1201,609 1328,1244 31,158 31,417 0,565 -114,533 -482,889 -1038,1004 -133,27 -448,35 -579,14 z"
-       style="fill:#ffffff;fill-opacity:1" />
+       id="path4338"
+       inkscape:connector-curvature="0" />
    <path
-       inkscape:connector-curvature="0"
-       id="path4340"
+       style="fill:#ffffff;fill-opacity:1"
       d="m 7070,9203 c -212,-20 -275,-27 -397,-48 -691,-117 -1400,-444 -2038,-940 -182,-142 -328,-270 -585,-517 -595,-571 -911,-974 -927,-1181 -6,-76 11,-120 69,-184 75,-80 159,-108 245,-79 109,37 263,181 632,595 539,606 774,826 1035,969 135,75 231,105 341,106 82,1 94,-2 138,-27 116,-68 161,-209 122,-376 -9,-36 -349,-868 -757,-1850 -407,-982 -785,-1892 -838,-2021 -287,-694 -513,-1389 -615,-1889 -70,-342 -90,-683 -52,-874 88,-440 381,-703 882,-792 124,-23 401,-30 562,-16 783,69 1674,461 2561,1125 796,596 1492,1354 1607,1751 43,146 -33,308 -168,360 -61,23 -100,15 -173,-36 -105,-74 -202,-170 -539,-529 -515,-551 -762,-783 -982,-927 -251,-164 -437,-186 -543,-65 -56,64 -74,131 -67,247 13,179 91,434 249,815 135,324 1588,4102 1646,4280 106,325 151,561 159,826 9,281 -22,463 -112,652 -58,122 -114,199 -211,292 -245,233 -582,343 -1044,338 -91,-1 -181,-3 -200,-5 z"
-       style="fill:#ffffff;fill-opacity:1" />
+       id="path4340"
+       inkscape:connector-curvature="0" />
  </g>
 </svg>
--- a/assets/icon-saved-messages.png
+++ b/assets/icon-saved-messages.png
--- a/assets/icon-saved-messages.svg
+++ b/assets/icon-saved-messages.svg
@@ -7,42 +7,42 @@
   xmlns="http://www.w3.org/2000/svg"
   xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
   xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
-   inkscape:version="0.92.4 5da689c313, 2019-01-14"
-   sodipodi:docname="icon-saved-messages.svg"
-   id="svg4344"
-   preserveAspectRatio="xMidYMid meet"
-   viewBox="0 0 45 45"
-   height="60"
-   width="60"
-   version="1.0"
-   inkscape:export-filename="/home/kerle/test-icon.png"
+   inkscape:export-ydpi="409.60001"
   inkscape:export-xdpi="409.60001"
-   inkscape:export-ydpi="409.60001">
+   inkscape:export-filename="/home/kerle/test-icon.png"
+   version="1.0"
+   width="60"
+   height="60"
+   viewBox="0 0 45 45"
+   preserveAspectRatio="xMidYMid meet"
+   id="svg4344"
+   sodipodi:docname="icon-saved-messages.svg"
+   inkscape:version="1.0beta1 (32d4812, 2019-09-19)">
  <defs
     id="defs4348" />
  <sodipodi:namedview
-     inkscape:lockguides="false"
-     inkscape:current-layer="svg4344"
-     inkscape:window-maximized="0"
-     inkscape:window-y="130"
-     inkscape:window-x="89"
-     inkscape:cy="43.28624"
-     inkscape:cx="-21.066324"
-     inkscape:zoom="4"
-     units="px"
-     showgrid="false"
-     id="namedview4346"
-     inkscape:window-height="855"
-     inkscape:window-width="1395"
-     inkscape:pageshadow="2"
-     inkscape:pageopacity="0"
-     guidetolerance="10"
-     gridtolerance="10"
-     objecttolerance="10"
-     borderopacity="1"
-     inkscape:document-rotation="0"
+     pagecolor="#ffffff"
     bordercolor="#666666"
-     pagecolor="#ffffff" />
+     inkscape:document-rotation="0"
+     borderopacity="1"
+     objecttolerance="10"
+     gridtolerance="10"
+     guidetolerance="10"
+     inkscape:pageopacity="0"
+     inkscape:pageshadow="2"
+     inkscape:window-width="1395"
+     inkscape:window-height="855"
+     id="namedview4346"
+     showgrid="false"
+     units="px"
+     inkscape:zoom="4"
+     inkscape:cx="29.308676"
+     inkscape:cy="49.03624"
+     inkscape:window-x="89"
+     inkscape:window-y="108"
+     inkscape:window-maximized="0"
+     inkscape:current-layer="svg4344"
+     inkscape:lockguides="false" />
  <metadata
     id="metadata4336">
 Created by potrace 1.15, written by Peter Selinger 2001-2017
@@ -56,21 +56,16 @@ Created by potrace 1.15, written by Peter Selinger 2001-2017
  </cc:Work>
 </rdf:RDF>
 </metadata>
-  <circle
-     style="opacity:1;fill:#ffffff;fill-opacity:1;stroke-width:0.731071"
-     id="path859"
-     cx="22.5"
-     cy="22.5"
-     r="22.5" />
-  <circle
-     r="22.5"
-     cy="22.50293"
-     cx="22.5"
-     id="path4917"
-     style="fill:#87aade;fill-opacity:1;stroke-width:0.58855402" />
+  <rect
+     y="0"
+     x="0"
+     height="45"
+     width="45"
+     id="rect1420"
+     style="fill:#87aade;fill-opacity:1;stroke:none;stroke-width:0.968078" />
  <path
-     inkscape:connector-curvature="0"
-     d="M 13.5,7.5 V 39 h 0.08654 L 22.533801,29.370239 31.482419,39 h 0.01758 V 7.5 Z m 9.004056,4.108698 1.879508,4.876388 5.039514,0.359779 -3.879358,3.363728 1.227764,5.095749 -4.276893,-2.796643 -4.280949,2.788618 1.237229,-5.093073 -3.873949,-3.371754 5.040866,-0.350417 z"
+     id="rect846"
     style="fill:#ffffff;stroke-width:0.58409804"
-     id="rect846" />
+     d="M 13.5,7.5 V 39 h 0.08654 L 22.533801,29.370239 31.482419,39 h 0.01758 V 7.5 Z m 9.004056,4.108698 1.879508,4.876388 5.039514,0.359779 -3.879358,3.363728 1.227764,5.095749 -4.276893,-2.796643 -4.280949,2.788618 1.237229,-5.093073 -3.873949,-3.371754 5.040866,-0.350417 z"
+     inkscape:connector-curvature="0" />
 </svg>
--- a/ci_scripts/remote_tests_python.sh
+++ b/ci_scripts/remote_tests_python.sh
@@ -2,7 +2,7 @@

 export BRANCH=${CIRCLE_BRANCH:?branch to build}
 export REPONAME=${CIRCLE_PROJECT_REPONAME:?repository name}
-export SSHTARGET=ci@b1.delta.chat
+export SSHTARGET=${SSHTARGET-ci@b1.delta.chat}

 # we construct the BUILDDIR such that we can easily share the
 # CARGO_TARGET_DIR between runs ("..")
--- a/ci_scripts/remote_tests_rust.sh
+++ b/ci_scripts/remote_tests_rust.sh
@@ -2,7 +2,7 @@

 export BRANCH=${CIRCLE_BRANCH:?branch to build}
 export REPONAME=${CIRCLE_PROJECT_REPONAME:?repository name}
-export SSHTARGET=ci@b1.delta.chat
+export SSHTARGET=${SSHTARGET-ci@b1.delta.chat}

 # we construct the BUILDDIR such that we can easily share the
 # CARGO_TARGET_DIR between runs ("..")
--- a/deltachat-ffi/Cargo.toml
+++ b/deltachat-ffi/Cargo.toml
@@ -1,6 +1,6 @@
 [package]
 name = "deltachat_ffi"
-version = "1.0.0-beta.9"
+version = "1.0.0-beta.16"
 description = "Deltachat FFI"
 authors = ["Delta Chat Developers (ML) <delta@codespeak.net>"]
 edition = "2018"
--- a/deltachat-ffi/deltachat.h
+++ b/deltachat-ffi/deltachat.h
@@ -994,11 +994,20 @@ uint32_t        dc_get_chat_id_by_contact_id (dc_context_t* context, uint32_t co
 *
 * Example:
 * ~~~
+ * char* blobdir = dc_get_blobdir(context);
+ * char* file_to_send = mprintf("%s/%s", blobdir, "send.mp4")
+ *
 * dc_msg_t* msg = dc_msg_new(context, DC_MSG_VIDEO);
- * dc_msg_set_file(msg, "/file/to/send.mp4", NULL);
+ * dc_msg_set_file(msg, file_to_send, NULL);
 * dc_prepare_msg(context, chat_id, msg);
- * // ... after /file/to/send.mp4 is ready:
+ *
+ * // ... create the file ...
+ *
 * dc_send_msg(context, chat_id, msg);
+ *
+ * dc_msg_unref(msg);
+ * free(file_to_send);
+ * dc_str_unref(file_to_send);
 * ~~~
 *
 * @memberof dc_context_t
@@ -1024,8 +1033,11 @@ uint32_t        dc_prepare_msg               (dc_context_t* context, uint32_t ch
 * Example:
 * ~~~
 * dc_msg_t* msg = dc_msg_new(context, DC_MSG_IMAGE);
+ *
 * dc_msg_set_file(msg, "/file/to/send.jpg", NULL);
 * dc_send_msg(context, chat_id, msg);
+ *
+ * dc_msg_unref(msg);
 * ~~~
 *
 * @memberof dc_context_t
@@ -1134,6 +1146,9 @@ void            dc_set_draft                 (dc_context_t* context, uint32_t ch
 *     // add a changelog
 *     dc_add_device_msg(context, "update-123", changelog_msg);
 * }
+ *
+ * dc_msg_unref(changelog_msg);
+ * dc_msg_unref(welome_msg);
 * ~~~
 */
 uint32_t        dc_add_device_msg            (dc_context_t* context, const char* label, dc_msg_t* msg);
@@ -1947,7 +1962,7 @@ void            dc_imex                      (dc_context_t* context, int what, c
 *         }
 *         while (!configure_succeeded())
 *     }
- *     free(file);
+ *     dc_str_unref(file);
 * }
 * ~~~
 *
@@ -2602,6 +2617,25 @@ dc_lot_t*        dc_chatlist_get_summary     (const dc_chatlist_t* chatlist, siz
 dc_context_t*    dc_chatlist_get_context     (dc_chatlist_t* chatlist);


+/**
+ * Get info summary for a chat, in json format. 
+ *
+ * The returned json string has the following key/values: 
+ *
+ * id: chat id 
+ * name: chat/group name 
+ * color: color of this chat 
+ * last-message-from: who sent the last message 
+ * last-message-text: message (truncated) 
+ * last-message-state: DC_STATE* constant
+ * last-message-date: 
+ * avatar-path: path-to-blobfile 
+ * is_verified: yes/no
+
+ * @return a utf8-encoded json string containing all requested info. Must be freed using dc_str_unref().  NULL is never returned.
+ */
+char*            dc_chat_get_info_json       (dc_context_t* context, size_t chat_id);
+
 /**
 * @class dc_chat_t
 *
@@ -3611,11 +3645,15 @@ int             dc_contact_is_blocked        (const dc_contact_t* contact);


 /**
- * Same as dc_contact_is_verified() but allows speeding up things
- * by adding the peerstate belonging to the contact.
- * If you do not have the peerstate available, it is loaded automatically.
+ * Check if a contact was verified. E.g. by a secure-join QR code scan
+ * and if the key has not changed since this verification.
 *
- * @private @memberof dc_context_t
+ * The UI may draw a checkbox or something like that beside verified contacts.
+ *
+ * @memberof dc_contact_t
+ * @param contact The contact object.
+ * @return 0: contact is not verified.
+ *    2: SELF and contact have verified their fingerprints in both directions; in the UI typically checkmarks are shown.
 */
 int             dc_contact_is_verified       (dc_contact_t* contact);

@@ -4016,11 +4054,6 @@ int64_t          dc_lot_get_timestamp     (const dc_lot_t* lot);
 */
 #define DC_CERTCK_STRICT 1

-/**
- * Accept invalid hostnames, but not invalid certificates.
- */
-#define DC_CERTCK_ACCEPT_INVALID_HOSTNAMES 2
-
 /**
 * Accept invalid certificates, including self-signed ones
 * or having incorrect hostname.
--- a/deltachat-ffi/src/dc_array.rs
+++ b/deltachat-ffi/src/dc_array.rs
--- a/deltachat-ffi/src/lib.rs
+++ b/deltachat-ffi/src/lib.rs
@@ -29,6 +29,8 @@ use deltachat::message::MsgId;
 use deltachat::stock::StockMessage;
 use deltachat::*;

+mod dc_array;
+
 mod string;
 use self::string::*;

@@ -2377,6 +2379,27 @@ pub unsafe extern "C" fn dc_chat_is_sending_locations(chat: *mut dc_chat_t) -> l
    ffi_chat.chat.is_sending_locations() as libc::c_int
 }

+#[no_mangle]
+pub unsafe extern "C" fn dc_chat_get_info_json(
+    context: *mut dc_context_t,
+    chat_id: u32,
+) -> *mut libc::c_char {
+    if context.is_null() {
+        eprintln!("ignoring careless call to dc_chat_get_info_json()");
+        return "".strdup();
+    }
+    let ffi_context = &*context;
+    ffi_context
+        .with_inner(|ctx| match chat::get_info_json(ctx, chat_id) {
+            Ok(s) => s.strdup(),
+            Err(err) => {
+                error!(ctx, "get_info_json({}) returned: {}", chat_id, err);
+                return "".strdup();
+            }
+        })
+        .unwrap_or_else(|_| "".strdup())
+}
+
 // dc_msg_t

 /// FFI struct for [dc_msg_t]
--- a/examples/repl/cmdline.rs
+++ b/examples/repl/cmdline.rs
@@ -94,7 +94,9 @@ pub fn dc_reset_tables(context: &Context, bits: i32) -> i32 {
 fn dc_poke_eml_file(context: &Context, filename: impl AsRef<Path>) -> Result<(), Error> {
    let data = dc_read_file(context, filename)?;

-    dc_receive_imf(context, &data, "import", 0, 0);
+    if let Err(err) = dc_receive_imf(context, &data, "import", 0, 0) {
+        println!("dc_receive_imf errored: {:?}", err);
+    }
    Ok(())
 }

@@ -936,7 +938,14 @@ pub fn dc_cmdline(context: &Context, line: &str) -> Result<(), failure::Error> {
            let contact = Contact::get_by_id(context, contact_id)?;
            let name_n_addr = contact.get_name_n_addr();

-            let mut res = format!("Contact info for: {}:\n\n", name_n_addr);
+            let mut res = format!(
+                "Contact info for: {}:\nIcon: {}\n",
+                name_n_addr,
+                match contact.get_profile_image(context) {
+                    Some(image) => image.to_str().unwrap().to_string(),
+                    None => "NoIcon".to_string(),
+                }
+            );

            res += &Contact::get_encrinfo(context, contact_id)?;

--- a/python/src/deltachat/account.py
+++ b/python/src/deltachat/account.py
@@ -3,6 +3,7 @@
 from __future__ import print_function
 import atexit
 import threading
+import os
 import re
 import time
 from array import array
@@ -94,9 +95,12 @@ class Account(object):
        """
        self._check_config_key(name)
        name = name.encode("utf8")
-        value = value.encode("utf8")
        if name == b"addr" and self.is_configured():
            raise ValueError("can not change 'addr' after account is configured.")
+        if value is not None:
+            value = value.encode("utf8")
+        else:
+            value = ffi.NULL
        lib.dc_set_config(self._dc_context, name, value)

    def get_config(self, name):
@@ -132,6 +136,18 @@ class Account(object):
        """
        return lib.dc_is_configured(self._dc_context)

+    def set_avatar(self, img_path):
+        """Set self avatar.
+
+        :raises ValueError: if profile image could not be set
+        :returns: None
+        """
+        if img_path is None:
+            self.set_config("selfavatar", None)
+        else:
+            assert os.path.exists(img_path), img_path
+            self.set_config("selfavatar", img_path)
+
    def check_is_configured(self):
        """ Raise ValueError if this account is not configured. """
        if not self.is_configured():
--- a/python/src/deltachat/chat.py
+++ b/python/src/deltachat/chat.py
@@ -2,6 +2,7 @@

 import mimetypes
 import calendar
+import json
 from datetime import datetime
 import os
 from .cutil import as_dc_charpointer, from_dc_charpointer, iter_array
@@ -108,6 +109,30 @@ class Chat(object):

    # ------  chat messaging API ------------------------------

+    def send_msg(self, msg):
+        """send a message by using a ready Message object.
+
+        :param msg: a :class:`deltachat.message.Message` instance
+           previously returned by
+           e.g. :meth:`deltachat.message.Message.new_empty` or
+           :meth:`prepare_file`.
+        :raises ValueError: if message can not be sent.
+
+        :returns: a :class:`deltachat.message.Message` instance as
+           sent out.  This is the same object as was passed in, which
+           has been modified with the new state of the core.
+        """
+        if msg.is_out_preparing():
+            assert msg.id != 0
+            # get a fresh copy of dc_msg, the core needs it
+            msg = Message.from_db(self.account, msg.id)
+        sent_id = lib.dc_send_msg(self._dc_context, self.id, msg._dc_msg)
+        if sent_id == 0:
+            raise ValueError("message could not be sent")
+        # modify message in place to avoid bad state for the caller
+        msg._dc_msg = Message.from_db(self.account, sent_id)._dc_msg
+        return msg
+
    def send_text(self, text):
        """ send a text message and return the resulting Message instance.

@@ -129,9 +154,12 @@ class Chat(object):
        :raises ValueError: if message can not be send/chat does not exist.
        :returns: the resulting :class:`deltachat.message.Message` instance
        """
-        msg = self.prepare_message_file(path=path, mime_type=mime_type)
-        self.send_prepared(msg)
-        return msg
+        msg = Message.new_empty(self.account, view_type="file")
+        msg.set_file(path, mime_type)
+        sent_id = lib.dc_send_msg(self._dc_context, self.id, msg._dc_msg)
+        if sent_id == 0:
+            raise ValueError("message could not be sent")
+        return Message.from_db(self.account, sent_id)

    def send_image(self, path):
        """ send an image message and return the resulting Message instance.
@@ -141,9 +169,12 @@ class Chat(object):
        :returns: the resulting :class:`deltachat.message.Message` instance
        """
        mime_type = mimetypes.guess_type(path)[0]
-        msg = self.prepare_message_file(path=path, mime_type=mime_type, view_type="image")
-        self.send_prepared(msg)
-        return msg
+        msg = Message.new_empty(self.account, view_type="image")
+        msg.set_file(path, mime_type)
+        sent_id = lib.dc_send_msg(self._dc_context, self.id, msg._dc_msg)
+        if sent_id == 0:
+            raise ValueError("message could not be sent")
+        return Message.from_db(self.account, sent_id)

    def prepare_message(self, msg):
        """ create a new prepared message.
@@ -242,6 +273,12 @@ class Chat(object):
        """
        return lib.dc_marknoticed_chat(self._dc_context, self.id)

+    def get_summary(self):
+        """ return dictionary with summary information. """
+        dc_res = lib.dc_chat_get_info_json(self._dc_context, self.id)
+        s = from_dc_charpointer(dc_res)
+        return json.loads(s)
+
    # ------  group management API ------------------------------

    def add_contact(self, contact):
@@ -324,6 +361,18 @@ class Chat(object):
            return None
        return from_dc_charpointer(dc_res)

+    def get_color(self):
+        """return the color of the chat.
+        :returns: color as 0x00rrggbb
+        """
+        return lib.dc_chat_get_color(self._dc_chat)
+
+    def get_subtitle(self):
+        """return the subtitle of the chat
+        :returns: the subtitle
+        """
+        return from_dc_charpointer(lib.dc_chat_get_subtitle(self._dc_chat))
+
    # ------  location streaming API ------------------------------

    def is_sending_locations(self):
@@ -332,6 +381,12 @@ class Chat(object):
        """
        return lib.dc_is_sending_locations_to_chat(self._dc_context, self.id)

+    def is_archived(self):
+        """return True if this chat is archived.
+        :returns: True if archived.
+        """
+        return lib.dc_chat_get_archived(self._dc_chat)
+
    def enable_sending_locations(self, seconds):
        """enable sending locations for this chat.

--- a/python/src/deltachat/const.py
+++ b/python/src/deltachat/const.py
@@ -68,7 +68,6 @@ DC_LP_SMTP_SOCKET_SSL = 0x20000
 DC_LP_SMTP_SOCKET_PLAIN = 0x40000
 DC_CERTCK_AUTO = 0
 DC_CERTCK_STRICT = 1
-DC_CERTCK_ACCEPT_INVALID_HOSTNAMES = 2
 DC_CERTCK_ACCEPT_INVALID_CERTIFICATES = 3
 DC_EMPTY_MVBOX = 0x01
 DC_EMPTY_INBOX = 0x02
--- a/python/src/deltachat/contact.py
+++ b/python/src/deltachat/contact.py
@@ -47,3 +47,13 @@ class Contact(object):
    def is_verified(self):
        """ Return True if the contact is verified. """
        return lib.dc_contact_is_verified(self._dc_contact)
+
+    def get_profile_image(self):
+        """Get contact profile image.
+
+        :returns: path to profile image, None if no profile image exists.
+        """
+        dc_res = lib.dc_contact_get_profile_image(self._dc_contact)
+        if dc_res == ffi.NULL:
+            return None
+        return from_dc_charpointer(dc_res)
--- a/python/src/deltachat/message.py
+++ b/python/src/deltachat/message.py
@@ -1,7 +1,6 @@
 """ The Message object. """

 import os
-import shutil
 from . import props
 from .cutil import from_dc_charpointer, as_dc_charpointer
 from .capi import lib, ffi
@@ -70,19 +69,6 @@ class Message(object):
        mtype = ffi.NULL if mime_type is None else as_dc_charpointer(mime_type)
        if not os.path.exists(path):
            raise ValueError("path does not exist: {!r}".format(path))
-        blobdir = self.account.get_blobdir()
-        if not path.startswith(blobdir):
-            for i in range(50):
-                ext = "" if i == 0 else "-" + str(i)
-                dest = os.path.join(blobdir, os.path.basename(path) + ext)
-                if os.path.exists(dest):
-                    continue
-                shutil.copyfile(path, dest)
-                break
-            else:
-                raise ValueError("could not create blobdir-path for {}".format(path))
-            path = dest
-        assert path.startswith(blobdir), path
        lib.dc_msg_set_file(self._dc_msg, as_dc_charpointer(path), mtype)

    @props.with_doc
@@ -188,7 +174,7 @@ class Message(object):
    @property
    def _msgstate(self):
        if self.id == 0:
-            dc_msg = self.message._dc_msg
+            dc_msg = self._dc_msg
        else:
            # load message from db to get a fresh/current state
            dc_msg = ffi.gc(
--- a/python/tests/conftest.py
+++ b/python/tests/conftest.py
@@ -85,16 +85,21 @@ class SessionLiveConfigFromFile:
 class SessionLiveConfigFromURL:
    def __init__(self, url, create_token):
        self.configlist = []
-        for i in range(2):
-            res = requests.post(url, json={"token_create_user": int(create_token)})
+        self.url = url
+        self.create_token = create_token
+
+    def get(self, index):
+        try:
+            return self.configlist[index]
+        except IndexError:
+            assert index == len(self.configlist), index
+            res = requests.post(self.url, json={"token_create_user": int(self.create_token)})
            if res.status_code != 200:
                pytest.skip("creating newtmpuser failed {!r}".format(res))
            d = res.json()
            config = dict(addr=d["email"], mail_pw=d["password"])
            self.configlist.append(config)
-
-    def get(self, index):
-        return self.configlist[index]
+            return config

    def exists(self):
        return bool(self.configlist)
--- a/python/tests/test_account.py
+++ b/python/tests/test_account.py
@@ -155,6 +155,18 @@ class TestOfflineChat:
        chat.set_name("title2")
        assert chat.get_name() == "title2"

+        d = chat.get_summary()
+        print(d)
+        assert d["id"] == chat.id
+        assert d["type"] == chat.get_type()
+        assert d["name"] == chat.get_name()
+        assert d["archived"] == chat.is_archived()
+        # assert d["param"] == chat.param
+        assert d["color"] == chat.get_color()
+        assert d["profile_image"] == "" if chat.get_profile_image() is None else chat.get_profile_image()
+        assert d["subtitle"] == chat.get_subtitle()
+        assert d["draft"] == "" if chat.get_draft() is None else chat.get_draft()
+
    def test_group_chat_creation_with_translation(self, ac1):
        ac1.set_stock_translation(const.DC_STR_NEWGROUPDRAFT, "xyz %1$s")
        ac1._evlogger.consume_events()
@@ -370,6 +382,23 @@ class TestOfflineChat:
        assert not res.is_ask_verifygroup()
        assert res.contact_id == 10

+    def test_group_chat_many_members_add_remove(self, ac1, lp):
+        lp.sec("ac1: creating group chat with 10 other members")
+        chat = ac1.create_group_chat(name="title1")
+        contacts = []
+        for i in range(10):
+            contact = ac1.create_contact("some{}@example.org".format(i))
+            contacts.append(contact)
+            chat.add_contact(contact)
+
+        num_contacts = len(chat.get_contacts())
+        assert num_contacts == 11
+
+        lp.sec("ac1: removing two contacts and checking things are right")
+        chat.remove_contact(contacts[9])
+        chat.remove_contact(contacts[3])
+        assert len(chat.get_contacts()) == 9
+

 class TestOnlineAccount:
    def get_chat(self, ac1, ac2, both_created=False):
@@ -430,30 +459,60 @@ class TestOnlineAccount:
        assert self_addr not in ev[2]
        ev = ac1._evlogger.get_matching("DC_EVENT_DELETED_BLOB_FILE")

-    def test_prepare_file_with_unicode(self, acfactory, lp):
+    def test_send_file_twice_unicode_filename_mangling(self, tmpdir, acfactory, lp):
        ac1, ac2 = acfactory.get_two_online_accounts()
        chat = self.get_chat(ac1, ac2)

-        lp.sec("ac1: prepare and send attachment + text to ac2")
-        blobdir = ac1.get_blobdir()
-        basename = "somedäüta.txt"
-        p = os.path.join(blobdir, basename)
+        basename = "somedäüta.html.zip"
+        p = os.path.join(tmpdir.strpath, basename)
        with open(p, "w") as f:
            f.write("some data")
-        msg = Message.new_empty(ac1, "file")
-        msg.set_text("hello ä world")
-        msg.set_file(p)
-        message = chat.prepare_message(msg)
-        assert message.is_out_preparing()
-        assert message.text == "hello ä world"
-        chat.send_prepared(message)
+
+        def send_and_receive_message():
+            lp.sec("ac1: prepare and send attachment + text to ac2")
+            msg1 = Message.new_empty(ac1, "file")
+            msg1.set_text("withfile")
+            msg1.set_file(p)
+            chat.send_msg(msg1)
+
+            lp.sec("ac2: receive message")
+            ev = ac2._evlogger.get_matching("DC_EVENT_INCOMING_MSG|DC_EVENT_MSGS_CHANGED")
+            assert ev[2] > const.DC_CHAT_ID_LAST_SPECIAL
+            return ac2.get_message_by_id(ev[2])
+
+        msg = send_and_receive_message()
+        assert msg.text == "withfile"
+        assert open(msg.filename).read() == "some data"
+        assert msg.filename.endswith(basename)
+
+        msg2 = send_and_receive_message()
+        assert msg2.text == "withfile"
+        assert open(msg2.filename).read() == "some data"
+        assert msg2.filename.endswith("html.zip")
+        assert msg.filename != msg2.filename
+
+    def test_send_file_html_attachment(self, tmpdir, acfactory, lp):
+        ac1, ac2 = acfactory.get_two_online_accounts()
+        chat = self.get_chat(ac1, ac2)
+
+        basename = "test.html"
+        content = "<html><body>text</body>data"
+
+        p = os.path.join(tmpdir.strpath, basename)
+        with open(p, "w") as f:
+            # write wrong html to see if core tries to parse it
+            # (it shouldn't as it's a file attachment)
+            f.write(content)
+
+        lp.sec("ac1: prepare and send attachment + text to ac2")
+        chat.send_file(p, mime_type="text/html")

        lp.sec("ac2: receive message")
        ev = ac2._evlogger.get_matching("DC_EVENT_INCOMING_MSG|DC_EVENT_MSGS_CHANGED")
        assert ev[2] > const.DC_CHAT_ID_LAST_SPECIAL
-        msg = ac2.get_message_by_id(ev[1])
-        assert msg.text == "hello ä world"
-        assert open(msg.filename).read() == "some data"
+        msg = ac2.get_message_by_id(ev[2])
+
+        assert open(msg.filename).read() == content
        assert msg.filename.endswith(basename)

    def test_mvbox_sentbox_threads(self, acfactory, lp):
@@ -573,6 +632,9 @@ class TestOnlineAccount:
    def test_send_and_receive_message_markseen(self, acfactory, lp):
        ac1, ac2 = acfactory.get_two_online_accounts()

+        # make DC's life harder wrt to encodings
+        ac1.set_config("displayname", "ä name")
+
        lp.sec("ac1: create chat with ac2")
        chat = self.get_chat(ac1, ac2)

@@ -590,6 +652,7 @@ class TestOnlineAccount:
        msg_in = ac2.get_message_by_id(msg_out.id)
        assert msg_in.text == "message1"
        assert not msg_in.is_forwarded()
+        assert msg_in.get_sender_contact().display_name == ac1.get_config("displayname")

        lp.sec("check the message arrived in contact-requets/deaddrop")
        chat2 = msg_in.chat
@@ -652,6 +715,12 @@ class TestOnlineAccount:
        assert msg_back.text == "message-back"
        assert msg_back.is_encrypted()

+        # Test that we do not gossip peer keys in 1-to-1 chat,
+        # as it makes no sense to gossip to peers their own keys.
+        # Gossip is only sent in encrypted messages,
+        # and we sent encrypted msg_back right above.
+        assert chat2b.get_summary()["gossiped_timestamp"] == 0
+
        lp.sec("create group chat with two members, one of which has no encrypt state")
        chat = ac1.create_group_chat("encryption test")
        chat.add_contact(ac1.create_contact(ac2.get_config("addr")))
@@ -913,7 +982,54 @@ class TestOnlineAccount:
        assert msg.text == "world"
        assert msg.is_encrypted()

-    def test_set_get_profile_image(self, acfactory, data, lp):
+    def test_set_get_contact_avatar(self, acfactory, data, lp):
+        lp.sec("configuring ac1 and ac2")
+        ac1, ac2 = acfactory.get_two_online_accounts()
+
+        lp.sec("ac1: set own profile image")
+        p = data.get_path("d.png")
+        ac1.set_avatar(p)
+
+        lp.sec("ac1: create 1:1 chat with ac2")
+        chat = self.get_chat(ac1, ac2, both_created=True)
+
+        msg = chat.send_text("hi -- do you see my brand new avatar?")
+        assert not msg.is_encrypted()
+
+        lp.sec("ac2: wait for receiving message and avatar from ac1")
+        msg1 = ac2.wait_next_incoming_message()
+        assert not msg1.chat.is_deaddrop()
+        received_path = msg1.get_sender_contact().get_profile_image()
+        assert open(received_path, "rb").read() == open(p, "rb").read()
+
+        lp.sec("ac2: set own profile image")
+        p = data.get_path("d.png")
+        ac2.set_avatar(p)
+
+        lp.sec("ac2: send back message")
+        m = msg1.chat.send_text("yes, i received your avatar -- how do you like mine?")
+        assert m.is_encrypted()
+
+        lp.sec("ac1: wait for receiving message and avatar from ac2")
+        msg2 = ac1.wait_next_incoming_message()
+        received_path = msg2.get_sender_contact().get_profile_image()
+        assert received_path is not None, "did not get avatar through encrypted message"
+        assert open(received_path, "rb").read() == open(p, "rb").read()
+
+        ac2._evlogger.consume_events()
+        ac1._evlogger.consume_events()
+
+        # XXX not sure if the following is correct / possible. you may remove it
+        lp.sec("ac1: delete profile image from chat, and send message to ac2")
+        ac1.set_avatar(None)
+        m = msg2.chat.send_text("i don't like my avatar anymore and removed it")
+        assert m.is_encrypted()
+
+        lp.sec("ac2: wait for message along with avatar deletion of ac1")
+        msg3 = ac2.wait_next_incoming_message()
+        assert msg3.get_sender_contact().get_profile_image() is None
+
+    def test_set_get_group_image(self, acfactory, data, lp):
        ac1, ac2 = acfactory.get_two_online_accounts()

        lp.sec("create unpromoted group chat")
@@ -1015,6 +1131,52 @@ class TestOnlineAccount:
        assert not locations3


+class TestGroupStressTests:
+    def test_group_many_members_add_leave_remove(self, acfactory, lp):
+        lp.sec("creating and configuring five accounts")
+        ac1 = acfactory.get_online_configuring_account()
+        accounts = [acfactory.get_online_configuring_account() for i in range(3)]
+        wait_configuration_progress(ac1, 1000)
+        for acc in accounts:
+            wait_configuration_progress(acc, 1000)
+
+        lp.sec("ac1: creating group chat with 3 other members")
+        chat = ac1.create_group_chat("title1")
+        contacts = []
+        chars = list("äöüsr")
+        for acc in accounts:
+            contact = ac1.create_contact(acc.get_config("addr"), name=chars.pop())
+            contacts.append(contact)
+            chat.add_contact(contact)
+            # make sure the other side accepts our messages
+            c1 = acc.create_contact(ac1.get_config("addr"), "ä member")
+            acc.create_chat_by_contact(c1)
+
+        assert not chat.is_promoted()
+
+        lp.sec("ac1: send mesage to new group chat")
+        chat.send_text("hello")
+        assert chat.is_promoted()
+
+        num_contacts = len(chat.get_contacts())
+        assert num_contacts == 3 + 1
+
+        lp.sec("ac2: checking that the chat arrived correctly")
+        ac2 = accounts[0]
+        msg = ac2.wait_next_incoming_message()
+        assert msg.text == "hello"
+        print("chat is", msg.chat)
+        assert len(msg.chat.get_contacts()) == 4
+
+        lp.sec("ac1: removing one contacts and checking things are right")
+        to_remove = msg.chat.get_contacts()[-1]
+        msg.chat.remove_contact(to_remove)
+
+        sysmsg = ac1.wait_next_incoming_message()
+        assert to_remove.addr in sysmsg.text
+        assert len(sysmsg.chat.get_contacts()) == 3
+
+
 class TestOnlineConfigureFails:
    def test_invalid_password(self, acfactory):
        ac1, configdict = acfactory.get_online_config()
--- a/python/tests/test_increation.py
+++ b/python/tests/test_increation.py
@@ -1,10 +1,49 @@
 from __future__ import print_function
+
+import os.path
+import shutil
+
+import pytest
 from filecmp import cmp
-from deltachat import const
+
 from conftest import wait_configuration_progress, wait_msgs_changed
+from deltachat import const


 class TestOnlineInCreation:
+    def test_increation_not_blobdir(self, tmpdir, acfactory, lp):
+        ac1 = acfactory.get_online_configuring_account()
+        ac2 = acfactory.get_online_configuring_account()
+        wait_configuration_progress(ac1, 1000)
+        wait_configuration_progress(ac2, 1000)
+
+        c2 = ac1.create_contact(email=ac2.get_config("addr"))
+        chat = ac1.create_chat_by_contact(c2)
+
+        lp.sec("Creating in-creation file outside of blobdir")
+        assert tmpdir.strpath != ac1.get_blobdir()
+        src = tmpdir.join('file.txt').ensure(file=1)
+        with pytest.raises(Exception):
+            chat.prepare_message_file(src.strpath)
+
+    def test_no_increation_copies_to_blobdir(self, tmpdir, acfactory, lp):
+        ac1 = acfactory.get_online_configuring_account()
+        ac2 = acfactory.get_online_configuring_account()
+        wait_configuration_progress(ac1, 1000)
+        wait_configuration_progress(ac2, 1000)
+
+        c2 = ac1.create_contact(email=ac2.get_config("addr"))
+        chat = ac1.create_chat_by_contact(c2)
+
+        lp.sec("Creating file outside of blobdir")
+        assert tmpdir.strpath != ac1.get_blobdir()
+        src = tmpdir.join('file.txt')
+        src.write("hello there\n")
+        chat.send_file(src.strpath)
+
+        blob_src = os.path.join(ac1.get_blobdir(), 'file.txt')
+        assert os.path.exists(blob_src), "file.txt not copied to blobdir"
+
    def test_forward_increation(self, acfactory, data, lp):
        ac1 = acfactory.get_online_configuring_account()
        ac2 = acfactory.get_online_configuring_account()
@@ -17,7 +56,10 @@ class TestOnlineInCreation:
        wait_msgs_changed(ac1, 0, 0)  # why no chat id?

        lp.sec("create a message with a file in creation")
-        path = data.get_path("d.png")
+        orig = data.get_path("d.png")
+        path = os.path.join(ac1.get_blobdir(), 'd.png')
+        with open(path, "x") as fp:
+            fp.write("preparing")
        prepared_original = chat.prepare_message_file(path)
        assert prepared_original.is_out_preparing()
        wait_msgs_changed(ac1, chat.id, prepared_original.id)
@@ -38,6 +80,7 @@ class TestOnlineInCreation:

        lp.sec("finish creating the file and send it")
        assert prepared_original.is_out_preparing()
+        shutil.copyfile(orig, path)
        chat.send_prepared(prepared_original)
        assert prepared_original.is_out_pending() or prepared_original.is_out_delivered()
        wait_msgs_changed(ac1, chat.id, prepared_original.id)
@@ -59,11 +102,11 @@ class TestOnlineInCreation:
        ev1 = ac2._evlogger.get_matching("DC_EVENT_MSGS_CHANGED")
        assert ev1[1] > const.DC_CHAT_ID_LAST_SPECIAL
        received_original = ac2.get_message_by_id(ev1[2])
-        assert cmp(received_original.filename, path, False)
+        assert cmp(received_original.filename, orig, shallow=False)

        lp.sec("wait2 for original or forwarded messages to arrive")
        ev2 = ac2._evlogger.get_matching("DC_EVENT_MSGS_CHANGED")
        assert ev2[1] > const.DC_CHAT_ID_LAST_SPECIAL
        assert ev2[1] != ev1[1]
        received_copy = ac2.get_message_by_id(ev2[2])
-        assert cmp(received_copy.filename, path, False)
+        assert cmp(received_copy.filename, orig, shallow=False)
--- a/python/tox.ini
+++ b/python/tox.ini
@@ -7,11 +7,7 @@ envlist =

 [testenv]
 commands = 
-    # (some qr tests are pretty heavy in terms of send/received
-    # messages and async-imap's likely has concurrency problems, 
-    # eg https://github.com/async-email/async-imap/issues/4 )
-    pytest -n6 --reruns 3 --reruns-delay 5 -v -rsXx -k "not qr" {posargs:tests} 
-    pytest -n6 --reruns 5 --reruns-delay 5 -v -rsXx -k "qr" {posargs:tests} 
+    pytest -n6 --reruns 2 --reruns-delay 5 -v -rsXx {posargs:tests} 
    # python tests/package_wheels.py {toxworkdir}/wheelhouse
 passenv = 
    TRAVIS 
@@ -36,7 +32,7 @@ commands =

 [testenv:lint]
 skipsdist = True
-usedevelop = True
+skip_install = True
 deps =
    flake8
    # pygments required by rst-lint
@@ -69,8 +65,8 @@ commands =


 [pytest]
-addopts = -v -rs 
-python_files = tests/test_*.py 
+addopts = -v -ra
+python_files = tests/test_*.py
 norecursedirs = .tox 
 xfail_strict=true
 timeout = 60 
--- a/set_core_version.py
+++ b/set_core_version.py
@@ -55,7 +55,8 @@ if __name__ == "__main__":
    replace_toml_version("Cargo.toml", newversion)
    replace_toml_version("deltachat-ffi/Cargo.toml", newversion)

-    subprocess.call(["cargo", "update", "-p", "deltachat"])
+    subprocess.call(["git", "add", "-u"])
+    # subprocess.call(["cargo", "update", "-p", "deltachat"])

    print("after commit make sure to: ")
    print("")
--- a/spec.md
+++ b/spec.md
@@ -1,6 +1,6 @@
 # Chat-over-Email specification

-Version 0.19.0
+Version 0.20.0

 This document describes how emails can be used
 to implement typical messenger functions
@@ -248,11 +248,11 @@ and the message SHOULD appear as a message or action from the sender.
 A group MAY have a group-image.
 To change or set the group-image,
 the messenger MUST attach an image file to a message
-and MUST add the header `Chat-Group-Image`
+and MUST add the header `Chat-Group-Avatar`
 with the value set to the image name.

 To remove the group-image,
-the messenger MUST add the header `Chat-Group-Image: 0`.
+the messenger MUST add the header `Chat-Group-Avatar: 0`.

 The messenger SHOULD send an explicit mail for each group image change.
 The body of the message SHOULD contain
@@ -265,7 +265,7 @@ and the message SHOULD appear as a message or action from the sender.
    Chat-Version: 1.0
    Chat-Group-ID: 12345uvwxyZ
    Chat-Group-Name: Our Group
-    Chat-Group-Image: image.jpg
+    Chat-Group-Avatar: image.jpg
    Message-ID: Gr.12345uvwxyZ.0005@domain
    Subject: Chat: Our Group: Hello, ...
    Content-Type: multipart/mixed; boundary="==break=="
@@ -283,25 +283,25 @@ and the message SHOULD appear as a message or action from the sender.

 The image format SHOULD be image/jpeg or image/png.
 To save data, it is RECOMMENDED
-to add a `Chat-Group-Image` only on image changes.
+to add a `Chat-Group-Avatar` only on image changes.


 # Set profile image

-A user MAY have a profile-image that MAY be spread to his contacts.
+A user MAY have a profile-image that MAY be spread to their contacts.
 To change or set the profile-image,
 the messenger MUST attach an image file to a message
-and MUST add the header `Chat-Profile-Image`
+and MUST add the header `Chat-User-Avatar`
 with the value set to the image name.

 To remove the profile-image,
-the messenger MUST add the header `Chat-Profile-Image: 0`.
+the messenger MUST add the header `Chat-User-Avatar: 0`.

 To spread the image,
 the messenger MAY send the profile image
 together with the next mail to a given contact
 (to do this only once,
-the messenger has to keep a `profile_image_update_state` somewhere).
+the messenger has to keep a `user_avatar_update_state` somewhere).
 Alternatively, the messenger MAY send an explicit mail
 for each profile-image change to all contacts using a compatible messenger.
 The messenger SHOULD NOT send an explicit mail to normal MUAs.
@@ -309,7 +309,7 @@ The messenger SHOULD NOT send an explicit mail to normal MUAs.
    From: sender@domain
    To: rcpt@domain
    Chat-Version: 1.0
-    Chat-Profile-Image: photo.jpg
+    Chat-User-Avatar: photo.jpg
    Subject: Chat: Hello, ...
    Content-Type: multipart/mixed; boundary="==break=="

@@ -325,10 +325,10 @@ The messenger SHOULD NOT send an explicit mail to normal MUAs.
    --==break==--

 The image format SHOULD be image/jpeg or image/png.
-Note that `Chat-Profile-Image` may appear together with all other headers,
-eg. there may be a `Chat-Profile-Image` and a `Chat-Group-Image` header
+Note that `Chat-User-Avatar` may appear together with all other headers,
+eg. there may be a `Chat-User-Avatar` and a `Chat-Group-Avatar` header
 in the same message.
-To save data, it is RECOMMENDED to add a `Chat-Profile-Image` header
+To save data, it is RECOMMENDED to add a `Chat-User-Avatar` header
 only on image changes.


--- a/src/aheader.rs
+++ b/src/aheader.rs
@@ -95,16 +95,25 @@ impl Aheader {

 impl fmt::Display for Aheader {
    fn fmt(&self, fmt: &mut fmt::Formatter) -> fmt::Result {
-        // TODO replace 78 with enum /rtn
-        // adds a whitespace every 78 characters, this allows libEtPan to
-        // wrap the lines according to RFC 5322
+        write!(fmt, "addr={};", self.addr)?;
+        if self.prefer_encrypt == EncryptPreference::Mutual {
+            write!(fmt, " prefer-encrypt=mutual;")?;
+        }
+
+        // adds a whitespace every 78 characters, this allows
+        // email crate to wrap the lines according to RFC 5322
        // (which may insert a linebreak before every whitespace)
-        let keydata = self.public_key.to_base64(78);
-        write!(
-            fmt,
-            "addr={}; prefer-encrypt={}; keydata={}",
-            self.addr, self.prefer_encrypt, keydata
-        )
+        let keydata = self.public_key.to_base64().chars().enumerate().fold(
+            String::new(),
+            |mut res, (i, c)| {
+                if i % 78 == 78 - "keydata=".len() {
+                    res.push(' ')
+                }
+                res.push(c);
+                res
+            },
+        );
+        write!(fmt, " keydata={}", keydata)
    }
 }

@@ -150,13 +159,10 @@ impl str::FromStr for Aheader {
            }
        };

-        let prefer_encrypt = match attributes
+        let prefer_encrypt = attributes
            .remove("prefer-encrypt")
            .and_then(|raw| raw.parse().ok())
-        {
-            Some(pref) => pref,
-            None => EncryptPreference::NoPreference,
-        };
+            .unwrap_or_default();

        // Autocrypt-Level0: unknown attributes starting with an underscore can be safely ignored
        // Autocrypt-Level0: unknown attribute, treat the header as invalid
@@ -176,15 +182,13 @@ impl str::FromStr for Aheader {
 mod tests {
    use super::*;

-    fn rawkey() -> String {
-        "xsBNBFzG3j0BCAC6iNhT8zydvCXi8LI/gFnkadMbfmSE/rTJskRRra/utGbLyDta/yTrJgWL7O3y/g4HdDW/dN2z26Y6W13IMzx9gLInn1KQZChtqWAcr/ReUucXcymwcfg1mdkBGk3TSLeLihN6CJx8Wsv8ig+kgAzte4f5rqEEAJVQ9WZHuti7UiYs6oRzqTo06CRe9owVXxzdMf0VDQtf7ZFm9dpzKKbhH7Lu8880iiotQ9/yRCkDGp9fNThsrLdZiK6OIAcIBAqi2rI89aS1dAmnRbktQieCx5izzyYkR1KvVL3gTTllHOzfKVEC2asmtWu2e4se/+O4WMIS1eGrn7GeWVb0Vwc5ABEBAAHNETxhQEBiLmV4YW1wbGUuZGU+wsCJBBABCAAzAhkBBQJcxt5FAhsDBAsJCAcGFQgJCgsCAxYCARYhBI4xxYKBgH3ANh5cufaKrc9mtiMLAAoJEPaKrc9mtiML938H/18F+3Wf9/JaAy/8hCO1v4S2PVBhxaKCokaNFtkfaMRne2l087LscCFPiFNyb4mv6Z3YeK8Xpxlp2sI0ecvdiqLUOGfnxS6tQrj+83EjtIrZ/hXOk1h121QFWH9Zg2VNHtODXjAgdLDC0NWUrclR0ZOqEDQHeo0ibTILdokVfXFN25wakPmGaYJP2y729cb1ve7RzvIvwn+Dddfxo3ao72rBfLi7l4NQ4S0KsY4cw+/6l5bRCKYCP77wZtvCwUvfVVosLdT43agtSiBI49+ayqvZ8OCvSJa61i+v81brTiEy9GBod4eAp45Ibsuemkw+gon4ZOvUXHTjwFB+h63MrozOwE0EXMbePQEIAL/vauf1zK8JgCu3V+G+SOX0iWw5xUlCPX+ERpBbWfwu3uAqn4wYXD3JDE/fVAF668xiV4eTPtlSUd5h0mn+G7uXMMOtkb+20SoEt50f8zw8TrL9t+ZsV11GKZWJpCar5AhXWsn6EEi8I2hLL5vn55ZZmHuGgN4jjmkRl3ToKCLhaXwTBjCJem7N5EH7F75wErEITa55v4Lb4Nfca7vnvtYrI1OA446xa8gHra0SINelTD09/JM/Fw4sWVPBaRZmJK/Tnu79N23No9XBUubmFPv1pNexZsQclicnTpt/BEWhiun7d6lfGB63K1aoHRTR1pcrWvBuALuuz0gqar2zlI0AEQEAAcLAdgQYAQgAIAUCXMbeRQIbDBYhBI4xxYKBgH3ANh5cufaKrc9mtiMLAAoJEPaKrc9mtiMLKSEIAIyLCRO2OyZ0IYRvRPpMn4p7E+7Pfcz/0mSkOy+1hshgJnqivXurm8zwGrwdMqeV4eslKR9H1RUdWGUQJNbtwmmjrt5DHpIhYHl5t3FpCBaGbV20Omo00Q38lBl9MtrmZkZw+ktEk6X+0xCKssMF+2MADkSOIufbR5HrDVB89VZOHCO9DeXvCUUAw2hyJiL/LHmLzJ40zYoTmb+F//f0k0j+tRdbkefyRoCmwG7YGiT+2hnCdgcezswnzah5J3ZKlrg7jOGo1LxtbvNUzxNBbC6S/aNgwm6qxo7xegRhmEl5uZ16zwyj4qz+xkjGy25Of5mWfUDoNw7OT7sjUbHOOMc=".into()
-    }
+    const RAWKEY: &str = "xsBNBFzG3j0BCAC6iNhT8zydvCXi8LI/gFnkadMbfmSE/rTJskRRra/utGbLyDta/yTrJgWL7O3y/g4HdDW/dN2z26Y6W13IMzx9gLInn1KQZChtqWAcr/ReUucXcymwcfg1mdkBGk3TSLeLihN6CJx8Wsv8ig+kgAzte4f5rqEEAJVQ9WZHuti7UiYs6oRzqTo06CRe9owVXxzdMf0VDQtf7ZFm9dpzKKbhH7Lu8880iiotQ9/yRCkDGp9fNThsrLdZiK6OIAcIBAqi2rI89aS1dAmnRbktQieCx5izzyYkR1KvVL3gTTllHOzfKVEC2asmtWu2e4se/+O4WMIS1eGrn7GeWVb0Vwc5ABEBAAHNETxhQEBiLmV4YW1wbGUuZGU+wsCJBBABCAAzAhkBBQJcxt5FAhsDBAsJCAcGFQgJCgsCAxYCARYhBI4xxYKBgH3ANh5cufaKrc9mtiMLAAoJEPaKrc9mtiML938H/18F+3Wf9/JaAy/8hCO1v4S2PVBhxaKCokaNFtkfaMRne2l087LscCFPiFNyb4mv6Z3YeK8Xpxlp2sI0ecvdiqLUOGfnxS6tQrj+83EjtIrZ/hXOk1h121QFWH9Zg2VNHtODXjAgdLDC0NWUrclR0ZOqEDQHeo0ibTILdokVfXFN25wakPmGaYJP2y729cb1ve7RzvIvwn+Dddfxo3ao72rBfLi7l4NQ4S0KsY4cw+/6l5bRCKYCP77wZtvCwUvfVVosLdT43agtSiBI49+ayqvZ8OCvSJa61i+v81brTiEy9GBod4eAp45Ibsuemkw+gon4ZOvUXHTjwFB+h63MrozOwE0EXMbePQEIAL/vauf1zK8JgCu3V+G+SOX0iWw5xUlCPX+ERpBbWfwu3uAqn4wYXD3JDE/fVAF668xiV4eTPtlSUd5h0mn+G7uXMMOtkb+20SoEt50f8zw8TrL9t+ZsV11GKZWJpCar5AhXWsn6EEi8I2hLL5vn55ZZmHuGgN4jjmkRl3ToKCLhaXwTBjCJem7N5EH7F75wErEITa55v4Lb4Nfca7vnvtYrI1OA446xa8gHra0SINelTD09/JM/Fw4sWVPBaRZmJK/Tnu79N23No9XBUubmFPv1pNexZsQclicnTpt/BEWhiun7d6lfGB63K1aoHRTR1pcrWvBuALuuz0gqar2zlI0AEQEAAcLAdgQYAQgAIAUCXMbeRQIbDBYhBI4xxYKBgH3ANh5cufaKrc9mtiMLAAoJEPaKrc9mtiMLKSEIAIyLCRO2OyZ0IYRvRPpMn4p7E+7Pfcz/0mSkOy+1hshgJnqivXurm8zwGrwdMqeV4eslKR9H1RUdWGUQJNbtwmmjrt5DHpIhYHl5t3FpCBaGbV20Omo00Q38lBl9MtrmZkZw+ktEk6X+0xCKssMF+2MADkSOIufbR5HrDVB89VZOHCO9DeXvCUUAw2hyJiL/LHmLzJ40zYoTmb+F//f0k0j+tRdbkefyRoCmwG7YGiT+2hnCdgcezswnzah5J3ZKlrg7jOGo1LxtbvNUzxNBbC6S/aNgwm6qxo7xegRhmEl5uZ16zwyj4qz+xkjGy25Of5mWfUDoNw7OT7sjUbHOOMc=";

    #[test]
    fn test_from_str() {
        let h: Aheader = format!(
            "addr=me@mail.com; prefer-encrypt=mutual; keydata={}",
-            rawkey()
+            RAWKEY
        )
        .parse()
        .expect("failed to parse");
@@ -193,9 +197,22 @@ mod tests {
        assert_eq!(h.prefer_encrypt, EncryptPreference::Mutual);
    }

+    // EncryptPreference::Reset is an internal value, parser should never return it
+    #[test]
+    fn test_from_str_reset() {
+        let raw = format!(
+            "addr=reset@example.com; prefer-encrypt=reset; keydata={}",
+            RAWKEY
+        );
+        let h: Aheader = raw.parse().expect("failed to parse");
+
+        assert_eq!(h.addr, "reset@example.com");
+        assert_eq!(h.prefer_encrypt, EncryptPreference::NoPreference);
+    }
+
    #[test]
    fn test_from_str_non_critical() {
-        let raw = format!("addr=me@mail.com; _foo=one; _bar=two; keydata={}", rawkey());
+        let raw = format!("addr=me@mail.com; _foo=one; _bar=two; keydata={}", RAWKEY);
        let h: Aheader = raw.parse().expect("failed to parse");

        assert_eq!(h.addr, "me@mail.com");
@@ -206,33 +223,57 @@ mod tests {
    fn test_from_str_superflous_critical() {
        let raw = format!(
            "addr=me@mail.com; _foo=one; _bar=two; other=me; keydata={}",
-            rawkey()
+            RAWKEY
        );
        assert!(raw.parse::<Aheader>().is_err());
    }

    #[test]
    fn test_good_headers() {
-        let fixed_header = "addr=a@b.example.org; prefer-encrypt=mutual; keydata=xsBNBFzG3j0BCAC6iNhT8zydvCXi8LI/gFnkadMbfmSE/rTJskRRra/utGbLyDta/yTrJgWL7O3y/g 4HdDW/dN2z26Y6W13IMzx9gLInn1KQZChtqWAcr/ReUucXcymwcfg1mdkBGk3TSLeLihN6CJx8Wsv8 ig+kgAzte4f5rqEEAJVQ9WZHuti7UiYs6oRzqTo06CRe9owVXxzdMf0VDQtf7ZFm9dpzKKbhH7Lu88 80iiotQ9/yRCkDGp9fNThsrLdZiK6OIAcIBAqi2rI89aS1dAmnRbktQieCx5izzyYkR1KvVL3gTTll HOzfKVEC2asmtWu2e4se/+O4WMIS1eGrn7GeWVb0Vwc5ABEBAAHNETxhQEBiLmV4YW1wbGUuZGU+ws CJBBABCAAzAhkBBQJcxt5FAhsDBAsJCAcGFQgJCgsCAxYCARYhBI4xxYKBgH3ANh5cufaKrc9mtiML AAoJEPaKrc9mtiML938H/18F+3Wf9/JaAy/8hCO1v4S2PVBhxaKCokaNFtkfaMRne2l087LscCFPiF Nyb4mv6Z3YeK8Xpxlp2sI0ecvdiqLUOGfnxS6tQrj+83EjtIrZ/hXOk1h121QFWH9Zg2VNHtODXjAg dLDC0NWUrclR0ZOqEDQHeo0ibTILdokVfXFN25wakPmGaYJP2y729cb1ve7RzvIvwn+Dddfxo3ao72 rBfLi7l4NQ4S0KsY4cw+/6l5bRCKYCP77wZtvCwUvfVVosLdT43agtSiBI49+ayqvZ8OCvSJa61i+v 81brTiEy9GBod4eAp45Ibsuemkw+gon4ZOvUXHTjwFB+h63MrozOwE0EXMbePQEIAL/vauf1zK8JgC u3V+G+SOX0iWw5xUlCPX+ERpBbWfwu3uAqn4wYXD3JDE/fVAF668xiV4eTPtlSUd5h0mn+G7uXMMOt kb+20SoEt50f8zw8TrL9t+ZsV11GKZWJpCar5AhXWsn6EEi8I2hLL5vn55ZZmHuGgN4jjmkRl3ToKC LhaXwTBjCJem7N5EH7F75wErEITa55v4Lb4Nfca7vnvtYrI1OA446xa8gHra0SINelTD09/JM/Fw4s WVPBaRZmJK/Tnu79N23No9XBUubmFPv1pNexZsQclicnTpt/BEWhiun7d6lfGB63K1aoHRTR1pcrWv BuALuuz0gqar2zlI0AEQEAAcLAdgQYAQgAIAUCXMbeRQIbDBYhBI4xxYKBgH3ANh5cufaKrc9mtiML AAoJEPaKrc9mtiMLKSEIAIyLCRO2OyZ0IYRvRPpMn4p7E+7Pfcz/0mSkOy+1hshgJnqivXurm8zwGr wdMqeV4eslKR9H1RUdWGUQJNbtwmmjrt5DHpIhYHl5t3FpCBaGbV20Omo00Q38lBl9MtrmZkZw+ktE k6X+0xCKssMF+2MADkSOIufbR5HrDVB89VZOHCO9DeXvCUUAw2hyJiL/LHmLzJ40zYoTmb+F//f0k0 j+tRdbkefyRoCmwG7YGiT+2hnCdgcezswnzah5J3ZKlrg7jOGo1LxtbvNUzxNBbC6S/aNgwm6qxo7x egRhmEl5uZ16zwyj4qz+xkjGy25Of5mWfUDoNw7OT7sjUbHOOMc=";
+        let fixed_header = concat!(
+            "addr=a@b.example.org; prefer-encrypt=mutual; ",
+            "keydata=xsBNBFzG3j0BCAC6iNhT8zydvCXi8LI/gFnkadMbfmSE/rTJskRRra/utGbLyDta/yTrJg",
+            " WL7O3y/g4HdDW/dN2z26Y6W13IMzx9gLInn1KQZChtqWAcr/ReUucXcymwcfg1mdkBGk3TSLeLihN6",
+            " CJx8Wsv8ig+kgAzte4f5rqEEAJVQ9WZHuti7UiYs6oRzqTo06CRe9owVXxzdMf0VDQtf7ZFm9dpzKK",
+            " bhH7Lu8880iiotQ9/yRCkDGp9fNThsrLdZiK6OIAcIBAqi2rI89aS1dAmnRbktQieCx5izzyYkR1Kv",
+            " VL3gTTllHOzfKVEC2asmtWu2e4se/+O4WMIS1eGrn7GeWVb0Vwc5ABEBAAHNETxhQEBiLmV4YW1wbG",
+            " UuZGU+wsCJBBABCAAzAhkBBQJcxt5FAhsDBAsJCAcGFQgJCgsCAxYCARYhBI4xxYKBgH3ANh5cufaK",
+            " rc9mtiMLAAoJEPaKrc9mtiML938H/18F+3Wf9/JaAy/8hCO1v4S2PVBhxaKCokaNFtkfaMRne2l087",
+            " LscCFPiFNyb4mv6Z3YeK8Xpxlp2sI0ecvdiqLUOGfnxS6tQrj+83EjtIrZ/hXOk1h121QFWH9Zg2VN",
+            " HtODXjAgdLDC0NWUrclR0ZOqEDQHeo0ibTILdokVfXFN25wakPmGaYJP2y729cb1ve7RzvIvwn+Ddd",
+            " fxo3ao72rBfLi7l4NQ4S0KsY4cw+/6l5bRCKYCP77wZtvCwUvfVVosLdT43agtSiBI49+ayqvZ8OCv",
+            " SJa61i+v81brTiEy9GBod4eAp45Ibsuemkw+gon4ZOvUXHTjwFB+h63MrozOwE0EXMbePQEIAL/vau",
+            " f1zK8JgCu3V+G+SOX0iWw5xUlCPX+ERpBbWfwu3uAqn4wYXD3JDE/fVAF668xiV4eTPtlSUd5h0mn+",
+            " G7uXMMOtkb+20SoEt50f8zw8TrL9t+ZsV11GKZWJpCar5AhXWsn6EEi8I2hLL5vn55ZZmHuGgN4jjm",
+            " kRl3ToKCLhaXwTBjCJem7N5EH7F75wErEITa55v4Lb4Nfca7vnvtYrI1OA446xa8gHra0SINelTD09",
+            " /JM/Fw4sWVPBaRZmJK/Tnu79N23No9XBUubmFPv1pNexZsQclicnTpt/BEWhiun7d6lfGB63K1aoHR",
+            " TR1pcrWvBuALuuz0gqar2zlI0AEQEAAcLAdgQYAQgAIAUCXMbeRQIbDBYhBI4xxYKBgH3ANh5cufaK",
+            " rc9mtiMLAAoJEPaKrc9mtiMLKSEIAIyLCRO2OyZ0IYRvRPpMn4p7E+7Pfcz/0mSkOy+1hshgJnqivX",
+            " urm8zwGrwdMqeV4eslKR9H1RUdWGUQJNbtwmmjrt5DHpIhYHl5t3FpCBaGbV20Omo00Q38lBl9Mtrm",
+            " ZkZw+ktEk6X+0xCKssMF+2MADkSOIufbR5HrDVB89VZOHCO9DeXvCUUAw2hyJiL/LHmLzJ40zYoTmb",
+            " +F//f0k0j+tRdbkefyRoCmwG7YGiT+2hnCdgcezswnzah5J3ZKlrg7jOGo1LxtbvNUzxNBbC6S/aNg",
+            " wm6qxo7xegRhmEl5uZ16zwyj4qz+xkjGy25Of5mWfUDoNw7OT7sjUbHOOMc="
+        );

        let ah = Aheader::from_str(fixed_header).expect("failed to parse");
        assert_eq!(ah.addr, "a@b.example.org");
        assert_eq!(ah.prefer_encrypt, EncryptPreference::Mutual);
+        assert_eq!(format!("{}", ah), fixed_header);

        let rendered = ah.to_string();
        assert_eq!(rendered, fixed_header);

-        let ah = Aheader::from_str(&format!(" _foo; __FOO=BAR ;;; addr = a@b.example.org ;\r\n   prefer-encrypt = mutual ; keydata = {}", rawkey())).expect("failed to parse");
+        let ah = Aheader::from_str(&format!(" _foo; __FOO=BAR ;;; addr = a@b.example.org ;\r\n   prefer-encrypt = mutual ; keydata = {}", RAWKEY)).expect("failed to parse");
        assert_eq!(ah.addr, "a@b.example.org");
        assert_eq!(ah.prefer_encrypt, EncryptPreference::Mutual);

        Aheader::from_str(&format!(
            "addr=a@b.example.org; prefer-encrypt=ignoreUnknownValues; keydata={}",
-            rawkey()
+            RAWKEY
        ))
        .expect("failed to parse");

-        Aheader::from_str(&format!("addr=a@b.example.org; keydata={}", rawkey()))
+        Aheader::from_str(&format!("addr=a@b.example.org; keydata={}", RAWKEY))
            .expect("failed to parse");
    }

@@ -244,4 +285,30 @@ mod tests {
        assert!(Aheader::from_str(" ;;").is_err());
        assert!(Aheader::from_str("addr=a@t.de; unknwon=1; keydata=jau").is_err());
    }
+
+    #[test]
+    fn test_display_aheader() {
+        assert!(format!(
+            "{}",
+            Aheader::new(
+                "test@example.com".to_string(),
+                Key::from_base64(RAWKEY, KeyType::Public).unwrap(),
+                EncryptPreference::Mutual
+            )
+        )
+        .contains("prefer-encrypt=mutual;"));
+
+        // According to Autocrypt Level 1 specification,
+        // only "prefer-encrypt=mutual;" can be used.
+        // If the setting is nopreference, the whole attribute is omitted.
+        assert!(!format!(
+            "{}",
+            Aheader::new(
+                "test@example.com".to_string(),
+                Key::from_base64(RAWKEY, KeyType::Public).unwrap(),
+                EncryptPreference::NoPreference
+            )
+        )
+        .contains("prefer-encrypt"));
+    }
 }
--- a/src/blob.rs
+++ b/src/blob.rs
@@ -159,7 +159,7 @@ impl<'a> BlobObject<'a> {
    /// This merely delegates to the [BlobObject::create_and_copy] and
    /// the [BlobObject::from_path] methods.  See those for possible
    /// errors.
-    pub fn create_from_path(
+    pub fn new_from_path(
        context: &Context,
        src: impl AsRef<Path>,
    ) -> std::result::Result<BlobObject, BlobError> {
@@ -559,14 +559,14 @@ mod tests {

        let src_ext = t.dir.path().join("external");
        fs::write(&src_ext, b"boo").unwrap();
-        let blob = BlobObject::create_from_path(&t.ctx, &src_ext).unwrap();
+        let blob = BlobObject::new_from_path(&t.ctx, &src_ext).unwrap();
        assert_eq!(blob.as_name(), "$BLOBDIR/external");
        let data = fs::read(blob.to_abs_path()).unwrap();
        assert_eq!(data, b"boo");

        let src_int = t.ctx.get_blobdir().join("internal");
        fs::write(&src_int, b"boo").unwrap();
-        let blob = BlobObject::create_from_path(&t.ctx, &src_int).unwrap();
+        let blob = BlobObject::new_from_path(&t.ctx, &src_int).unwrap();
        assert_eq!(blob.as_name(), "$BLOBDIR/internal");
        let data = fs::read(blob.to_abs_path()).unwrap();
        assert_eq!(data, b"boo");
@@ -576,7 +576,7 @@ mod tests {
        let t = dummy_context();
        let src_ext = t.dir.path().join("autocrypt-setup-message-4137848473.html");
        fs::write(&src_ext, b"boo").unwrap();
-        let blob = BlobObject::create_from_path(&t.ctx, &src_ext).unwrap();
+        let blob = BlobObject::new_from_path(&t.ctx, &src_ext).unwrap();
        assert_eq!(
            blob.as_name(),
            "$BLOBDIR/autocrypt-setup-message-4137848473.html"
--- a/src/chat.rs
+++ b/src/chat.rs
@@ -4,6 +4,7 @@ use std::path::{Path, PathBuf};

 use itertools::Itertools;
 use num_traits::FromPrimitive;
+use serde_json::json;

 use crate::blob::{BlobError, BlobObject};
 use crate::chatlist::*;
@@ -34,7 +35,6 @@ pub struct Chat {
    pub grpid: String,
    blocked: Blocked,
    pub param: Params,
-    pub gossiped_timestamp: i64,
    is_sending_locations: bool,
 }

@@ -43,7 +43,7 @@ impl Chat {
    pub fn load_from_db(context: &Context, chat_id: u32) -> Result<Self, Error> {
        let res = context.sql.query_row(
            "SELECT c.id,c.type,c.name, c.grpid,c.param,c.archived, \
-             c.blocked, c.gossiped_timestamp, c.locations_send_until  \
+             c.blocked, c.locations_send_until  \
             FROM chats c WHERE c.id=?;",
            params![chat_id as i32],
            |row| {
@@ -55,8 +55,7 @@ impl Chat {
                    param: row.get::<_, String>(4)?.parse().unwrap_or_default(),
                    archived: row.get(5)?,
                    blocked: row.get::<_, Option<_>>(6)?.unwrap_or_default(),
-                    gossiped_timestamp: row.get(7)?,
-                    is_sending_locations: row.get(8)?,
+                    is_sending_locations: row.get(7)?,
                };

                Ok(c)
@@ -215,6 +214,10 @@ impl Chat {
        None
    }

+    pub fn get_gossiped_timestamp(&self, context: &Context) -> i64 {
+        get_gossiped_timestamp(context, self.id)
+    }
+
    pub fn get_color(&self, context: &Context) -> u32 {
        let mut color = 0;

@@ -309,10 +312,10 @@ impl Chat {
                        self.id
                    );
                }
-            } else if self.typ == Chattype::Group
-                || self.typ == Chattype::VerifiedGroup
-                    && self.param.get_int(Param::Unpromoted).unwrap_or_default() == 1
+            } else if (self.typ == Chattype::Group || self.typ == Chattype::VerifiedGroup)
+                && self.param.get_int(Param::Unpromoted).unwrap_or_default() == 1
            {
+                msg.param.set_int(Param::AttachGroupImage, 1);
                self.param.remove(Param::Unpromoted);
                self.update_param(context)?;
            }
@@ -592,12 +595,6 @@ pub fn set_blocking(context: &Context, chat_id: u32, new_blocking: Blocked) -> b
    .is_ok()
 }

-fn copy_device_icon_to_blobs(context: &Context) -> Result<String, Error> {
-    let icon = include_bytes!("../assets/icon-device.png");
-    let blob = BlobObject::create(context, "icon-device.png".to_string(), icon)?;
-    Ok(blob.as_name().to_string())
-}
-
 pub fn update_saved_messages_icon(context: &Context) -> Result<(), Error> {
    // if there is no saved-messages chat, there is nothing to update. this is no error.
    if let Ok((chat_id, _)) = lookup_by_contact_id(context, DC_CONTACT_ID_SELF) {
@@ -612,6 +609,24 @@ pub fn update_saved_messages_icon(context: &Context) -> Result<(), Error> {
    Ok(())
 }

+pub fn update_device_icon(context: &Context) -> Result<(), Error> {
+    // if there is no device-chat, there is nothing to update. this is no error.
+    if let Ok((chat_id, _)) = lookup_by_contact_id(context, DC_CONTACT_ID_DEVICE) {
+        let icon = include_bytes!("../assets/icon-device.png");
+        let blob = BlobObject::create(context, "icon-device.png".to_string(), icon)?;
+        let icon = blob.as_name().to_string();
+
+        let mut chat = Chat::load_from_db(context, chat_id)?;
+        chat.param.set(Param::ProfileImage, &icon);
+        chat.update_param(context)?;
+
+        let mut contact = Contact::load_from_db(context, DC_CONTACT_ID_DEVICE)?;
+        contact.param.set(Param::ProfileImage, icon);
+        contact.update_param(context)?;
+    }
+    Ok(())
+}
+
 pub fn create_or_lookup_by_contact_id(
    context: &Context,
    contact_id: u32,
@@ -637,10 +652,7 @@ pub fn create_or_lookup_by_contact_id(
            chat_name,
            match contact_id {
                DC_CONTACT_ID_SELF => "K=1".to_string(), // K = Param::Selftalk
-                DC_CONTACT_ID_DEVICE => {
-                    let icon = copy_device_icon_to_blobs(context)?;
-                    format!("D=1\ni={}", icon) // D = Param::Devicetalk, i = Param::ProfileImage
-                },
+                DC_CONTACT_ID_DEVICE => "D=1".to_string(), // D = Param::Devicetalk
                _ => "".to_string()
            },
            create_blocked as u8,
@@ -664,6 +676,8 @@ pub fn create_or_lookup_by_contact_id(

    if contact_id == DC_CONTACT_ID_SELF {
        update_saved_messages_icon(context)?;
+    } else if contact_id == DC_CONTACT_ID_DEVICE {
+        update_device_icon(context)?;
    }

    Ok((chat_id, create_blocked))
@@ -729,6 +743,7 @@ fn prepare_msg_blob(context: &Context, msg: &mut Message) -> Result<(), Error> {
            .get_blob(Param::File, context, !msg.is_increation())?
            .ok_or_else(|| format_err!("Attachment missing for message of type #{}", msg.type_0))?;
        msg.param.set(Param::File, blob.as_name());
+
        if msg.type_0 == Viewtype::File || msg.type_0 == Viewtype::Image {
            // Correct the type, take care not to correct already very special
            // formats as GIF or VOICE.
@@ -1452,7 +1467,7 @@ pub(crate) fn add_contact_to_chat_ex(
    let contact = Contact::get_by_id(context, contact_id)?;
    let mut msg = Message::default();

-    reset_gossiped_timestamp(context, chat_id);
+    reset_gossiped_timestamp(context, chat_id)?;

    /*this also makes sure, not contacts are added to special or normal chats*/
    let mut chat = Chat::load_from_db(context, chat_id)?;
@@ -1550,12 +1565,28 @@ fn real_group_exists(context: &Context, chat_id: u32) -> bool {
        .unwrap_or_default()
 }

-pub fn reset_gossiped_timestamp(context: &Context, chat_id: u32) {
-    set_gossiped_timestamp(context, chat_id, 0);
+pub fn reset_gossiped_timestamp(context: &Context, chat_id: u32) -> crate::sql::Result<()> {
+    set_gossiped_timestamp(context, chat_id, 0)
 }

-// Should return Result
-pub fn set_gossiped_timestamp(context: &Context, chat_id: u32, timestamp: i64) {
+/// Get timestamp of the last gossip sent in the chat.
+/// Zero return value means that gossip was never sent.
+pub fn get_gossiped_timestamp(context: &Context, chat_id: u32) -> i64 {
+    context
+        .sql
+        .query_get_value::<_, i64>(
+            context,
+            "SELECT gossiped_timestamp FROM chats WHERE id=?;",
+            params![chat_id as i32],
+        )
+        .unwrap_or_default()
+}
+
+pub fn set_gossiped_timestamp(
+    context: &Context,
+    chat_id: u32,
+    timestamp: i64,
+) -> crate::sql::Result<()> {
    if 0 != chat_id {
        info!(
            context,
@@ -1568,7 +1599,6 @@ pub fn set_gossiped_timestamp(context: &Context, chat_id: u32, timestamp: i64) {
            "UPDATE chats SET gossiped_timestamp=? WHERE id=?;",
            params![timestamp, chat_id as i32],
        )
-        .ok();
    } else {
        info!(
            context,
@@ -1580,10 +1610,58 @@ pub fn set_gossiped_timestamp(context: &Context, chat_id: u32, timestamp: i64) {
            "UPDATE chats SET gossiped_timestamp=?;",
            params![timestamp],
        )
-        .ok();
    }
 }

+pub fn shall_attach_selfavatar(context: &Context, chat_id: u32) -> Result<bool, Error> {
+    // versions before 12/2019 already allowed to set selfavatar, however, it was never sent to others.
+    // to avoid sending out previously set selfavatars unexpectedly we added this additional check.
+    // it can be removed after some time.
+    if !context
+        .sql
+        .get_raw_config_bool(context, "attach_selfavatar")
+    {
+        return Ok(false);
+    }
+
+    let timestamp_some_days_ago = time() - DC_RESEND_USER_AVATAR_DAYS * 24 * 60 * 60;
+    let needs_attach = context.sql.query_map(
+        "SELECT c.selfavatar_sent
+           FROM chats_contacts cc
+           LEFT JOIN contacts c ON c.id=cc.contact_id
+          WHERE cc.chat_id=? AND cc.contact_id!=?;",
+        params![chat_id, DC_CONTACT_ID_SELF],
+        |row| Ok(row.get::<_, i64>(0)),
+        |rows| {
+            let mut needs_attach = false;
+            for row in rows {
+                if let Ok(selfavatar_sent) = row {
+                    let selfavatar_sent = selfavatar_sent?;
+                    if selfavatar_sent < timestamp_some_days_ago {
+                        needs_attach = true;
+                    }
+                }
+            }
+            Ok(needs_attach)
+        },
+    )?;
+    Ok(needs_attach)
+}
+
+pub fn set_selfavatar_timestamp(
+    context: &Context,
+    chat_id: u32,
+    timestamp: i64,
+) -> Result<(), Error> {
+    context.sql.execute(
+        "UPDATE contacts
+            SET selfavatar_sent=?
+          WHERE id IN(SELECT contact_id FROM chats_contacts WHERE chat_id=?);",
+        params![timestamp, chat_id],
+    )?;
+    Ok(())
+}
+
 pub fn remove_contact_from_chat(
    context: &Context,
    chat_id: u32,
@@ -1837,11 +1915,8 @@ pub fn forward_msgs(context: &Context, msg_ids: &[MsgId], chat_id: u32) -> Resul
        ensure!(chat.can_send(), "cannot send to chat #{}", chat_id);
        curr_timestamp = dc_create_smeared_timestamps(context, msg_ids.len());
        let ids = context.sql.query_map(
-            format!(
-                "SELECT id FROM msgs WHERE id IN({}) ORDER BY timestamp,id",
-                msg_ids.iter().map(|_| "?").join(",")
-            ),
-            msg_ids,
+            "SELECT id FROM msgs WHERE id IN({}) ORDER BY timestamp,id",
+            params![msg_ids.iter().map(|_| "?").join(",")],
            |row| row.get::<_, MsgId>(0),
            |ids| ids.collect::<Result<Vec<_>, _>>().map_err(Into::into),
        )?;
@@ -1903,6 +1978,54 @@ pub fn forward_msgs(context: &Context, msg_ids: &[MsgId], chat_id: u32) -> Resul
    Ok(())
 }

+pub fn get_info_json(context: &Context, chat_id: u32) -> Result<String, Error> {
+    let chat = Chat::load_from_db(context, chat_id).unwrap();
+
+    // ToDo:
+    // - [x] id
+    // - [x] type
+    // - [x] name
+    // - [x] archived
+    // - [x] color
+    // - [x] profileImage
+    // - [x] subtitle
+    // - [x] draft,
+    // - [ ] deaddrop,
+    // - [ ] summary,
+    // - [ ] lastUpdated,
+    // - [ ] freshMessageCounter,
+    // - [ ] email
+
+    let profile_image = match chat.get_profile_image(context) {
+        Some(path) => path.into_os_string().into_string().unwrap(),
+        None => "".to_string(),
+    };
+
+    let draft = match get_draft(context, chat_id) {
+        Ok(message) => match message {
+            Some(m) => m.text.unwrap_or_else(|| "".to_string()),
+            None => "".to_string(),
+        },
+        Err(_) => "".to_string(),
+    };
+
+    let s = json!({
+         "id": chat.id,
+         "type": chat.typ as u32,
+         "name": chat.name,
+         "archived": chat.archived,
+         "param": chat.param.to_string(),
+         "gossiped_timestamp": chat.get_gossiped_timestamp(context),
+         "is_sending_locations": chat.is_sending_locations,
+         "color": chat.get_color(context),
+         "profile_image": profile_image,
+         "subtitle": chat.get_subtitle(context),
+         "draft": draft
+    });
+
+    Ok(s.to_string())
+}
+
 pub fn get_chat_contact_cnt(context: &Context, chat_id: u32) -> usize {
    context
        .sql
@@ -2353,4 +2476,42 @@ mod tests {
            "bar"
        );
    }
+
+    #[test]
+    fn test_create_same_chat_twice() {
+        let context = dummy_context();
+        let contact1 = Contact::create(&context.ctx, "bob", "bob@mail.de").unwrap();
+        assert_ne!(contact1, 0);
+
+        let chat_id = create_by_contact_id(&context.ctx, contact1).unwrap();
+        assert!(
+            chat_id > DC_CHAT_ID_LAST_SPECIAL,
+            "chat_id too small {}",
+            chat_id
+        );
+        let chat = Chat::load_from_db(&context.ctx, chat_id).unwrap();
+
+        let chat2_id = create_by_contact_id(&context.ctx, contact1).unwrap();
+        assert_eq!(chat2_id, chat_id);
+        let chat2 = Chat::load_from_db(&context.ctx, chat2_id).unwrap();
+
+        assert_eq!(chat2.name, chat.name);
+    }
+
+    #[test]
+    fn test_shall_attach_selfavatar() {
+        let t = dummy_context();
+        let chat_id = create_group_chat(&t.ctx, VerifiedStatus::Unverified, "foo").unwrap();
+        assert!(!shall_attach_selfavatar(&t.ctx, chat_id).unwrap());
+
+        let (contact_id, _) =
+            Contact::add_or_lookup(&t.ctx, "", "foo@bar.org", Origin::IncomingUnknownTo).unwrap();
+        add_contact_to_chat(&t.ctx, chat_id, contact_id);
+        assert!(!shall_attach_selfavatar(&t.ctx, chat_id).unwrap());
+        t.ctx.set_config(Config::Selfavatar, None).unwrap(); // setting to None also forces re-sending
+        assert!(shall_attach_selfavatar(&t.ctx, chat_id).unwrap());
+
+        assert!(set_selfavatar_timestamp(&t.ctx, chat_id, time()).is_ok());
+        assert!(!shall_attach_selfavatar(&t.ctx, chat_id).unwrap());
+    }
 }
--- a/src/config.rs
+++ b/src/config.rs
@@ -9,6 +9,7 @@ use crate::context::Context;
 use crate::dc_tools::*;
 use crate::job::*;
 use crate::stock::StockMessage;
+use rusqlite::NO_PARAMS;

 /// The available configuration keys.
 #[derive(
@@ -127,9 +128,18 @@ impl Context {
    /// If `None` is passed as a value the value is cleared and set to the default if there is one.
    pub fn set_config(&self, key: Config, value: Option<&str>) -> crate::sql::Result<()> {
        match key {
-            Config::Selfavatar if value.is_some() => {
-                let blob = BlobObject::create_from_path(&self, value.unwrap())?;
-                self.sql.set_raw_config(self, key, Some(blob.as_name()))
+            Config::Selfavatar => {
+                self.sql
+                    .execute("UPDATE contacts SET selfavatar_sent=0;", NO_PARAMS)?;
+                self.sql
+                    .set_raw_config_bool(self, "attach_selfavatar", true)?;
+                match value {
+                    Some(value) => {
+                        let blob = BlobObject::new_from_path(&self, value)?;
+                        self.sql.set_raw_config(self, key, Some(blob.as_name()))
+                    }
+                    None => self.sql.set_raw_config(self, key, None),
+                }
            }
            Config::InboxWatch => {
                let ret = self.sql.set_raw_config(self, key, value);
--- a/src/configure/auto_mozilla.rs
+++ b/src/configure/auto_mozilla.rs
@@ -1,8 +1,6 @@
 //! # Thunderbird's Autoconfiguration implementation
 //!
 //! Documentation: https://developer.mozilla.org/en-US/docs/Mozilla/Thunderbird/Autoconfiguration */
-use failure::Fail;
-
 use quick_xml;
 use quick_xml::events::{BytesEnd, BytesStart, BytesText};

--- a/src/configure/auto_outlook.rs
+++ b/src/configure/auto_outlook.rs
@@ -1,7 +1,5 @@
 //! Outlook's Autodiscover

-use failure::Fail;
-
 use quick_xml;
 use quick_xml::events::BytesEnd;

--- a/src/configure/mod.rs
+++ b/src/configure/mod.rs
@@ -12,7 +12,7 @@ use crate::context::Context;
 use crate::dc_tools::*;
 use crate::e2ee;
 use crate::job::*;
-use crate::login_param::LoginParam;
+use crate::login_param::{CertificateChecks, LoginParam};
 use crate::oauth2::*;
 use crate::param::Params;

@@ -92,9 +92,11 @@ pub fn JobConfigureImap(context: &Context) {
    let mut param_domain = "undefined.undefined".to_owned();
    let mut param_addr_urlencoded: String =
        "Internal Error: this value should never be used".to_owned();
-    let mut keep_flags = std::i32::MAX;
+    let mut keep_flags = 0;
+
+    const STEP_12_USE_AUTOCONFIG: u8 = 12;
+    const STEP_13_AFTER_AUTOCONFIG: u8 = 13;

-    const STEP_3_INDEX: u8 = 13;
    let mut step_counter: u8 = 0;
    while !context.shall_stop_ongoing() {
        step_counter += 1;
@@ -110,7 +112,7 @@ pub fn JobConfigureImap(context: &Context) {
            }
            // Step 1: Load the parameters and check email-address and password
            2 => {
-                if 0 != param.server_flags & 0x2 {
+                if 0 != param.server_flags & DC_LP_AUTH_OAUTH2 {
                    // the used oauth2 addr may differ, check this.
                    // if dc_get_oauth2_addr() is not available in the oauth2 implementation,
                    // just use the given one.
@@ -145,6 +147,7 @@ pub fn JobConfigureImap(context: &Context) {
            // Step 2: Autoconfig
            4 => {
                progress!(context, 200);
+
                if param.mail_server.is_empty()
                            && param.mail_port == 0
                            /*&&param.mail_user.is_empty() -- the user can enter a loginname which is used by autoconfig then */
@@ -152,12 +155,18 @@ pub fn JobConfigureImap(context: &Context) {
                            && param.send_port == 0
                            && param.send_user.is_empty()
                            /*&&param.send_pw.is_empty() -- the password cannot be auto-configured and is no criterion for autoconfig or not */
-                            && param.server_flags & !0x2 == 0
+                            && (param.server_flags & !DC_LP_AUTH_OAUTH2) == 0
                {
-                    keep_flags = param.server_flags & 0x2;
+                    // no advanced parameters entered by the user: query provider-database or do Autoconfig
+                    keep_flags = param.server_flags & DC_LP_AUTH_OAUTH2;
+                    if let Some(new_param) = get_offline_autoconfig(context, &param) {
+                        // got parameters from our provider-database, skip Autoconfig, preserve the OAuth2 setting
+                        param_autoconfig = Some(new_param);
+                        step_counter = STEP_12_USE_AUTOCONFIG - 1; // minus one as step_counter is increased on next loop
+                    }
                } else {
-                    // Autoconfig is not needed so skip it.
-                    step_counter = STEP_3_INDEX - 1;
+                    // advanced parameters entered by the user: skip Autoconfig
+                    step_counter = STEP_13_AFTER_AUTOCONFIG - 1; // minus one as step_counter is increased on next loop
                }
                true
            }
@@ -242,8 +251,10 @@ pub fn JobConfigureImap(context: &Context) {
                }
                true
            }
-            /* C.  Do we have any result? */
-            12 => {
+            /* C.  Do we have any autoconfig result?
+               If you change the match-number here, also update STEP_12_COPY_AUTOCONFIG above
+            */
+            STEP_12_USE_AUTOCONFIG => {
                progress!(context, 500);
                if let Some(ref cfg) = param_autoconfig {
                    info!(context, "Got autoconfig: {}", &cfg);
@@ -256,15 +267,15 @@ pub fn JobConfigureImap(context: &Context) {
                    param.send_port = cfg.send_port;
                    param.send_user = cfg.send_user.clone();
                    param.server_flags = cfg.server_flags;
-                    /* although param_autoconfig's data are no longer needed from, it is important to keep the object as
-                    we may enter "deep guessing" if we could not read a configuration */
+                    /* although param_autoconfig's data are no longer needed from,
+                    it is used to later to prevent trying variations of port/server/logins */
                }
                param.server_flags |= keep_flags;
                true
            }
            // Step 3: Fill missing fields with defaults
-            13 => {
-                // if you move this, don't forget to update STEP_3_INDEX, too
+            // If you change the match-number here, also update STEP_13_AFTER_AUTOCONFIG above
+            STEP_13_AFTER_AUTOCONFIG => {
                if param.mail_server.is_empty() {
                    param.mail_server = format!("imap.{}", param_domain,)
                }
@@ -430,6 +441,42 @@ pub fn JobConfigureImap(context: &Context) {
    progress!(context, if success { 1000 } else { 0 });
 }

+fn get_offline_autoconfig(context: &Context, param: &LoginParam) -> Option<LoginParam> {
+    // XXX we don't have https://github.com/deltachat/provider-db  APIs
+    // integrated yet but we'll already add nauta as a first use case, also
+    // showing what we need from provider-db in the future.
+    info!(
+        context,
+        "checking internal provider-info for offline autoconfig"
+    );
+
+    if param.addr.ends_with("@nauta.cu") {
+        let mut p = LoginParam::new();
+
+        p.addr = param.addr.clone();
+        p.mail_server = "imap.nauta.cu".to_string();
+        p.mail_user = param.addr.clone();
+        p.mail_pw = param.mail_pw.clone();
+        p.mail_port = 143;
+        p.imap_certificate_checks = CertificateChecks::AcceptInvalidCertificates;
+
+        p.send_server = "smtp.nauta.cu".to_string();
+        p.send_user = param.addr.clone();
+        p.send_pw = param.mail_pw.clone();
+        p.send_port = 25;
+        p.smtp_certificate_checks = CertificateChecks::AcceptInvalidCertificates;
+        p.server_flags = DC_LP_AUTH_NORMAL as i32
+            | DC_LP_IMAP_SOCKET_STARTTLS as i32
+            | DC_LP_SMTP_SOCKET_STARTTLS as i32;
+
+        info!(context, "found offline autoconfig: {}", p);
+        Some(p)
+    } else {
+        info!(context, "no offline autoconfig found");
+        None
+    }
+}
+
 fn try_imap_connections(
    context: &Context,
    mut param: &mut LoginParam,
@@ -485,8 +532,12 @@ fn try_imap_connection(

 fn try_imap_one_param(context: &Context, param: &LoginParam) -> Option<bool> {
    let inf = format!(
-        "imap: {}@{}:{} flags=0x{:x}",
-        param.mail_user, param.mail_server, param.mail_port, param.server_flags
+        "imap: {}@{}:{} flags=0x{:x} certificate_checks={}",
+        param.mail_user,
+        param.mail_server,
+        param.mail_port,
+        param.server_flags,
+        param.imap_certificate_checks
    );
    info!(context, "Trying: {}", inf);
    if context
@@ -567,6 +618,7 @@ fn try_smtp_one_param(context: &Context, param: &LoginParam) -> Option<bool> {
 #[cfg(test)]
 mod tests {

+    use super::*;
    use crate::config::*;
    use crate::configure::JobConfigureImap;
    use crate::test_utils::*;
@@ -580,4 +632,19 @@ mod tests {
        t.ctx.set_config(Config::MailPw, Some("123456")).unwrap();
        JobConfigureImap(&t.ctx);
    }
+
+    #[test]
+    fn test_get_offline_autoconfig() {
+        let context = dummy_context().ctx;
+
+        let mut params = LoginParam::new();
+        params.addr = "someone123@example.org".to_string();
+        assert!(get_offline_autoconfig(&context, &params).is_none());
+
+        let mut params = LoginParam::new();
+        params.addr = "someone123@nauta.cu".to_string();
+        let found_params = get_offline_autoconfig(&context, &params).unwrap();
+        assert_eq!(found_params.mail_server, "imap.nauta.cu".to_string());
+        assert_eq!(found_params.send_server, "smtp.nauta.cu".to_string());
+    }
 }
--- a/src/configure/read_url.rs
+++ b/src/configure/read_url.rs
@@ -1,5 +1,4 @@
 use crate::context::Context;
-use failure::Fail;

 #[derive(Debug, Fail)]
 pub enum Error {
--- a/src/constants.rs
+++ b/src/constants.rs
@@ -58,6 +58,9 @@ pub const DC_GCM_ADDDAYMARKER: u32 = 0x01;
 pub const DC_GCL_VERIFIED_ONLY: usize = 0x01;
 pub const DC_GCL_ADD_SELF: usize = 0x02;

+// unchanged user avatars are resent to the recipients every some days
+pub const DC_RESEND_USER_AVATAR_DAYS: i64 = 14;
+
 // values for DC_PARAM_FORCE_PLAINTEXT
 pub(crate) const DC_FP_NO_AUTOCRYPT_HEADER: i32 = 2;
 pub(crate) const DC_FP_ADD_AUTOCRYPT_HEADER: i32 = 1;
@@ -119,6 +122,10 @@ pub const DC_CONTACT_ID_INFO: u32 = 2;
 pub const DC_CONTACT_ID_DEVICE: u32 = 5;
 pub const DC_CONTACT_ID_LAST_SPECIAL: u32 = 9;

+// decorative address that is used for DC_CONTACT_ID_DEVICE
+// when an api that returns an email is called.
+pub const DC_CONTACT_ID_DEVICE_ADDR: &str = "device@localhost";
+
 // Flags for empty server job

 pub const DC_EMPTY_MVBOX: u32 = 0x01;
--- a/src/contact.rs
+++ b/src/contact.rs
@@ -10,11 +10,13 @@ use crate::constants::*;
 use crate::context::Context;
 use crate::dc_tools::*;
 use crate::e2ee;
-use crate::error::Result;
+use crate::error::{Error, Result};
 use crate::events::Event;
 use crate::key::*;
 use crate::login_param::LoginParam;
 use crate::message::{MessageState, MsgId};
+use crate::mimeparser::AvatarAction;
+use crate::param::*;
 use crate::peerstate::*;
 use crate::sql;
 use crate::stock::StockMessage;
@@ -58,6 +60,8 @@ pub struct Contact {
    blocked: bool,
    /// The origin/source of the contact.
    pub origin: Origin,
+    /// Parameters as Param::ProfileImage
+    pub param: Params,
 }

 /// Possible origins of a contact.
@@ -139,32 +143,10 @@ pub enum VerifiedStatus {

 impl Contact {
    pub fn load_from_db(context: &Context, contact_id: u32) -> crate::sql::Result<Self> {
-        if contact_id == DC_CONTACT_ID_SELF {
-            let contact = Contact {
-                id: contact_id,
-                name: context.stock_str(StockMessage::SelfMsg).into(),
-                authname: "".into(),
-                addr: context
-                    .get_config(Config::ConfiguredAddr)
-                    .unwrap_or_default(),
-                blocked: false,
-                origin: Origin::Unknown,
-            };
-            return Ok(contact);
-        } else if contact_id == DC_CONTACT_ID_DEVICE {
-            let contact = Contact {
-                id: contact_id,
-                name: context.stock_str(StockMessage::DeviceMessages).into(),
-                authname: "".into(),
-                addr: "device@localhost".into(),
-                blocked: false,
-                origin: Origin::Unknown,
-            };
-            return Ok(contact);
-        }
-
-        context.sql.query_row(
-            "SELECT c.name, c.addr, c.origin, c.blocked, c.authname  FROM contacts c  WHERE c.id=?;",
+        let mut res = context.sql.query_row(
+            "SELECT c.name, c.addr, c.origin, c.blocked, c.authname, c.param
+               FROM contacts c
+              WHERE c.id=?;",
            params![contact_id as i32],
            |row| {
                let contact = Self {
@@ -174,10 +156,21 @@ impl Contact {
                    addr: row.get::<_, String>(1)?,
                    blocked: row.get::<_, Option<i32>>(3)?.unwrap_or_default() != 0,
                    origin: row.get(2)?,
+                    param: row.get::<_, String>(5)?.parse().unwrap_or_default(),
                };
                Ok(contact)
-            }
-        )
+            },
+        )?;
+        if contact_id == DC_CONTACT_ID_SELF {
+            res.name = context.stock_str(StockMessage::SelfMsg).to_string();
+            res.addr = context
+                .get_config(Config::ConfiguredAddr)
+                .unwrap_or_default();
+        } else if contact_id == DC_CONTACT_ID_DEVICE {
+            res.name = context.stock_str(StockMessage::DeviceMessages).to_string();
+            res.addr = DC_CONTACT_ID_DEVICE_ADDR.to_string();
+        }
+        Ok(res)
    }

    /// Returns `true` if this contact is blocked.
@@ -709,6 +702,16 @@ impl Contact {
        Ok(Contact::load_from_db(context, contact_id)?)
    }

+    pub fn update_param(&mut self, context: &Context) -> Result<()> {
+        sql::execute(
+            context,
+            &context.sql,
+            "UPDATE contacts SET param=? WHERE id=?",
+            params![self.param.to_string(), self.id as i32],
+        )?;
+        Ok(())
+    }
+
    /// Get the ID of the contact.
    pub fn get_id(&self) -> u32 {
        self.id
@@ -742,6 +745,9 @@ impl Contact {
        if !self.name.is_empty() {
            return &self.name;
        }
+        if !self.authname.is_empty() {
+            return &self.authname;
+        }
        &self.addr
    }

@@ -777,8 +783,11 @@ impl Contact {
            if let Some(p) = context.get_config(Config::Selfavatar) {
                return Some(PathBuf::from(p));
            }
+        } else if let Some(image_rel) = self.param.get(Param::ProfileImage) {
+            if !image_rel.is_empty() {
+                return Some(dc_get_abs_path(context, image_rel));
+            }
        }
-        // TODO: else get image_abs from contact param
        None
    }

@@ -861,14 +870,14 @@ impl Contact {
            .unwrap_or_default() as usize
    }

-    pub fn get_origin_by_id(context: &Context, contact_id: u32, ret_blocked: &mut i32) -> Origin {
+    pub fn get_origin_by_id(context: &Context, contact_id: u32, ret_blocked: &mut bool) -> Origin {
        let mut ret = Origin::Unknown;
-        *ret_blocked = 0;
+        *ret_blocked = false;

        if let Ok(contact) = Contact::load_from_db(context, contact_id) {
            /* we could optimize this by loading only the needed fields */
            if contact.blocked {
-                *ret_blocked = 1;
+                *ret_blocked = true;
            } else {
                ret = contact.origin;
            }
@@ -957,6 +966,32 @@ fn set_block_contact(context: &Context, contact_id: u32, new_blocking: bool) {
    }
 }

+pub fn set_profile_image(
+    context: &Context,
+    contact_id: u32,
+    profile_image: AvatarAction,
+) -> Result<()> {
+    // the given profile image is expected to be already in the blob directory
+    // as profile images can be set only by receiving messages, this should be always the case, however.
+    let mut contact = Contact::load_from_db(context, contact_id)?;
+    let changed = match profile_image {
+        AvatarAction::Change(profile_image) => {
+            contact.param.set(Param::ProfileImage, profile_image);
+            true
+        }
+        AvatarAction::Delete => {
+            contact.param.remove(Param::ProfileImage);
+            true
+        }
+        AvatarAction::None => false,
+    };
+    if changed {
+        contact.update_param(context)?;
+        context.call_cb(Event::ContactsChanged(Some(contact_id)));
+    }
+    Ok(())
+}
+
 /// Normalize a name.
 ///
 /// - Remove quotes (come from some bad MUA implementations)
@@ -1021,6 +1056,18 @@ fn cat_fingerprint(
    }
 }

+impl Context {
+    /// determine whether the specified addr maps to the/a self addr
+    pub fn is_self_addr(&self, addr: &str) -> Result<bool> {
+        let self_addr = match self.get_config(Config::ConfiguredAddr) {
+            Some(s) => s,
+            None => return Err(Error::NotConfigured),
+        };
+
+        Ok(addr_cmp(self_addr, addr))
+    }
+}
+
 pub fn addr_cmp(addr1: impl AsRef<str>, addr2: impl AsRef<str>) -> bool {
    let norm1 = addr_normalize(addr1.as_ref()).to_lowercase();
    let norm2 = addr_normalize(addr2.as_ref()).to_lowercase();
@@ -1028,15 +1075,6 @@ pub fn addr_cmp(addr1: impl AsRef<str>, addr2: impl AsRef<str>) -> bool {
    norm1 == norm2
 }

-pub fn addr_equals_self(context: &Context, addr: impl AsRef<str>) -> bool {
-    if !addr.as_ref().is_empty() {
-        if let Some(self_addr) = context.get_config(Config::ConfiguredAddr) {
-            return addr_cmp(addr, self_addr);
-        }
-    }
-    false
-}
-
 fn split_address_book(book: &str) -> Vec<(&str, &str)> {
    book.lines()
        .chunks(2)
@@ -1120,6 +1158,18 @@ mod tests {
        assert_eq!(contacts.len(), 0);
    }

+    #[test]
+    fn test_is_self_addr() -> Result<()> {
+        let t = test_context(None);
+        assert!(t.ctx.is_self_addr("me@me.org").is_err());
+
+        let addr = configure_alice_keypair(&t.ctx);
+        assert_eq!(t.ctx.is_self_addr("me@me.org")?, false);
+        assert_eq!(t.ctx.is_self_addr(&addr)?, true);
+
+        Ok(())
+    }
+
    #[test]
    fn test_add_or_lookup() {
        // add some contacts, this also tests add_address_book()
--- a/src/context.rs
+++ b/src/context.rs
@@ -298,6 +298,11 @@ impl Context {
        res.insert("database_version", dbversion.to_string());
        res.insert("blobdir", self.get_blobdir().display().to_string());
        res.insert("display_name", displayname.unwrap_or_else(|| unset.into()));
+        res.insert(
+            "selfavatar",
+            self.get_config(Config::Selfavatar)
+                .unwrap_or_else(|| "<unset>".to_string()),
+        );
        res.insert("is_configured", is_configured.to_string());
        res.insert("entered_account_settings", l.to_string());
        res.insert("used_account_settings", l2.to_string());
--- a/src/dc_receive_imf.rs
+++ b/src/dc_receive_imf.rs
--- a/src/dc_simplify.rs
+++ b/src/dc_simplify.rs
@@ -49,7 +49,7 @@ impl Simplify {
    /**
     * Simplify Plain Text
     */
-    #[allow(non_snake_case, clippy::mut_range_bound)]
+    #[allow(non_snake_case, clippy::mut_range_bound, clippy::needless_range_loop)]
    fn simplify_plain_text(&mut self, buf_terminated: &str, is_msgrmsg: bool) -> String {
        /* This function ...
        ... removes all text after the line `-- ` (footer mark)
--- a/src/dc_tools.rs
+++ b/src/dc_tools.rs
@@ -182,22 +182,6 @@ fn encode_66bits_as_base64(v1: u32, v2: u32, fill: u32) -> String {
    String::from_utf8(wrapped_writer).unwrap()
 }

-pub(crate) fn dc_create_incoming_rfc724_mid(
-    message_timestamp: i64,
-    contact_id_from: u32,
-    contact_ids_to: &[u32],
-) -> Option<String> {
-    /* create a deterministic rfc724_mid from input such that
-    repeatedly calling it with the same input results in the same Message-id */
-
-    let largest_id_to = contact_ids_to.iter().max().copied().unwrap_or_default();
-    let result = format!(
-        "{}-{}-{}@stub",
-        message_timestamp, contact_id_from, largest_id_to
-    );
-    Some(result)
-}
-
 /// Function generates a Message-ID that can be used for a new outgoing message.
 /// - this function is called for all outgoing messages.
 /// - the message ID should be globally unique
@@ -278,11 +262,9 @@ pub fn dc_derive_safe_stem_ext(filename: &str) -> (String, String) {
 // the returned suffix is lower-case
 #[allow(non_snake_case)]
 pub fn dc_get_filesuffix_lc(path_filename: impl AsRef<str>) -> Option<String> {
-    if let Some(p) = Path::new(path_filename.as_ref()).extension() {
-        Some(p.to_string_lossy().to_lowercase())
-    } else {
-        None
-    }
+    Path::new(path_filename.as_ref())
+        .extension()
+        .map(|p| p.to_string_lossy().to_lowercase())
 }

 /// Returns the `(width, height)` of the given image buffer.
@@ -782,14 +764,6 @@ mod tests {
        }
    }

-    #[test]
-    fn test_dc_create_incoming_rfc724_mid() {
-        let res = dc_create_incoming_rfc724_mid(123, 45, &[6, 7]);
-        assert_eq!(res, Some("123-45-7@stub".into()));
-        let res = dc_create_incoming_rfc724_mid(123, 45, &[]);
-        assert_eq!(res, Some("123-45-0@stub".into()));
-    }
-
    #[test]
    fn test_file_get_safe_basename() {
        assert_eq!(get_safe_basename("12312/hello"), "hello");
--- a/src/e2ee.rs
+++ b/src/e2ee.rs
@@ -247,13 +247,13 @@ fn decrypt_if_autocrypt_message<'a>(
    // Errors are returned for failures related to decryption of AC-messages.

    let encrypted_data_part = match wrapmime::get_autocrypt_mime(mail) {
-        Err(err) => {
-            // not a proper autocrypt message, abort and ignore
-            info!(context, "Not an autocrypt message: {:?}", err);
+        Err(_) => {
+            // not an autocrypt mime message, abort and ignore
            return Ok(None);
        }
        Ok(res) => res,
    };
+    info!(context, "Detected Autocrypt-mime message");

    decrypt_part(
        context,
--- a/src/error.rs
+++ b/src/error.rs
@@ -1,6 +1,5 @@
 //! # Error handling

-use failure::Fail;
 use lettre_email::mime;

 #[derive(Debug, Fail)]
@@ -29,12 +28,14 @@ pub enum Error {
    InvalidMsgId,
    #[fail(display = "Watch folder not found {:?}", _0)]
    WatchFolderNotFound(String),
-    #[fail(display = "Inalid Email: {:?}", _0)]
+    #[fail(display = "Invalid Email: {:?}", _0)]
    MailParseError(#[cause] mailparse::MailParseError),
    #[fail(display = "Building invalid Email: {:?}", _0)]
    LettreError(#[cause] lettre_email::error::Error),
    #[fail(display = "FromStr error: {:?}", _0)]
    FromStr(#[cause] mime::FromStrError),
+    #[fail(display = "Not Configured")]
+    NotConfigured,
 }

 pub type Result<T> = std::result::Result<T, Error>;
--- a/src/headerdef.rs
+++ b/src/headerdef.rs
@@ -0,0 +1,58 @@
+#[derive(Debug, Display, Clone, PartialEq, Eq, EnumVariantNames)]
+#[strum(serialize_all = "kebab_case")]
+#[allow(dead_code)]
+pub enum HeaderDef {
+    MessageId,
+    Subject,
+    Date,
+    From_,
+    To,
+    Cc,
+    Disposition,
+    OriginalMessageId,
+    ListId,
+    References,
+    InReplyTo,
+    Precedence,
+    ChatVersion,
+    ChatGroupId,
+    ChatGroupName,
+    ChatGroupNameChanged,
+    ChatVerified,
+    ChatGroupImage, // deprecated
+    ChatGroupAvatar,
+    ChatUserAvatar,
+    ChatVoiceMessage,
+    ChatGroupMemberRemoved,
+    ChatGroupMemberAdded,
+    ChatContent,
+    ChatDuration,
+    ChatDispositionNotificationTo,
+    AutocryptSetupMessage,
+    SecureJoin,
+    SecureJoinGroup,
+    SecureJoinFingerprint,
+    SecureJoinInvitenumber,
+    SecureJoinAuth,
+    _TestHeader,
+}
+
+impl HeaderDef {
+    /// Returns the corresponding Event id.
+    pub fn get_headername(&self) -> String {
+        self.to_string()
+    }
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    #[test]
+    /// Test that kebab_case serialization works as expected
+    fn kebab_test() {
+        assert_eq!(HeaderDef::From_.to_string(), "from");
+
+        assert_eq!(HeaderDef::_TestHeader.to_string(), "test-header");
+    }
+}
--- a/src/imap/idle.rs
+++ b/src/imap/idle.rs
@@ -0,0 +1,273 @@
+use super::Imap;
+
+use async_imap::extensions::idle::IdleResponse;
+use async_std::prelude::*;
+use async_std::task;
+use std::sync::atomic::Ordering;
+use std::time::{Duration, SystemTime};
+
+use crate::context::Context;
+use crate::imap_client::*;
+
+use super::select_folder;
+
+type Result<T> = std::result::Result<T, Error>;
+
+#[derive(Debug, Fail)]
+pub enum Error {
+    #[fail(display = "IMAP IDLE protocol failed to init/complete")]
+    IdleProtocolFailed(#[cause] async_imap::error::Error),
+
+    #[fail(display = "IMAP IDLE protocol timed out")]
+    IdleTimeout(#[cause] async_std::future::TimeoutError),
+
+    #[fail(display = "IMAP server does not have IDLE capability")]
+    IdleAbilityMissing,
+
+    #[fail(display = "IMAP select folder error")]
+    SelectFolderError(#[cause] select_folder::Error),
+
+    #[fail(display = "IMAP error")]
+    ImapError(#[cause] async_imap::error::Error),
+
+    #[fail(display = "Setup handle error")]
+    SetupHandleError(#[cause] super::Error),
+}
+
+impl From<select_folder::Error> for Error {
+    fn from(err: select_folder::Error) -> Error {
+        Error::SelectFolderError(err)
+    }
+}
+
+impl Imap {
+    pub fn can_idle(&self) -> bool {
+        task::block_on(async move { self.config.read().await.can_idle })
+    }
+
+    pub fn idle(&self, context: &Context, watch_folder: Option<String>) -> Result<()> {
+        task::block_on(async move {
+            if !self.can_idle() {
+                return Err(Error::IdleAbilityMissing);
+            }
+
+            self.setup_handle_if_needed(context)
+                .await
+                .map_err(Error::SetupHandleError)?;
+
+            self.select_folder(context, watch_folder.clone()).await?;
+
+            let session = self.session.lock().await.take();
+            let timeout = Duration::from_secs(23 * 60);
+            if let Some(session) = session {
+                match session.idle() {
+                    // BEWARE: If you change the Secure branch you
+                    // typically also need to change the Insecure branch.
+                    IdleHandle::Secure(mut handle) => {
+                        if let Err(err) = handle.init().await {
+                            return Err(Error::IdleProtocolFailed(err));
+                        }
+
+                        let (idle_wait, interrupt) = handle.wait_with_timeout(timeout);
+                        *self.interrupt.lock().await = Some(interrupt);
+
+                        if self.skip_next_idle_wait.load(Ordering::SeqCst) {
+                            // interrupt_idle has happened before we
+                            // provided self.interrupt
+                            self.skip_next_idle_wait.store(false, Ordering::SeqCst);
+                            std::mem::drop(idle_wait);
+                            info!(context, "Idle wait was skipped");
+                        } else {
+                            info!(context, "Idle entering wait-on-remote state");
+                            match idle_wait.await {
+                                IdleResponse::NewData(_) => {
+                                    info!(context, "Idle has NewData");
+                                }
+                                // TODO: idle_wait does not distinguish manual interrupts
+                                // from Timeouts if we would know it's a Timeout we could bail
+                                // directly and reconnect .
+                                IdleResponse::Timeout => {
+                                    info!(context, "Idle-wait timeout or interruption");
+                                }
+                                IdleResponse::ManualInterrupt => {
+                                    info!(context, "Idle wait was interrupted");
+                                }
+                            }
+                        }
+                        // if we can't properly terminate the idle
+                        // protocol let's break the connection.
+                        let res =
+                            async_std::future::timeout(Duration::from_secs(15), handle.done())
+                                .await
+                                .map_err(|err| {
+                                    self.trigger_reconnect();
+                                    Error::IdleTimeout(err)
+                                })?;
+
+                        match res {
+                            Ok(session) => {
+                                *self.session.lock().await = Some(Session::Secure(session));
+                            }
+                            Err(err) => {
+                                // if we cannot terminate IDLE it probably
+                                // means that we waited long (with idle_wait)
+                                // but the network went away/changed
+                                self.trigger_reconnect();
+                                return Err(Error::IdleProtocolFailed(err));
+                            }
+                        }
+                    }
+                    IdleHandle::Insecure(mut handle) => {
+                        if let Err(err) = handle.init().await {
+                            return Err(Error::IdleProtocolFailed(err));
+                        }
+
+                        let (idle_wait, interrupt) = handle.wait_with_timeout(timeout);
+                        *self.interrupt.lock().await = Some(interrupt);
+
+                        if self.skip_next_idle_wait.load(Ordering::SeqCst) {
+                            // interrupt_idle has happened before we
+                            // provided self.interrupt
+                            self.skip_next_idle_wait.store(false, Ordering::SeqCst);
+                            std::mem::drop(idle_wait);
+                            info!(context, "Idle wait was skipped");
+                        } else {
+                            info!(context, "Idle entering wait-on-remote state");
+                            match idle_wait.await {
+                                IdleResponse::NewData(_) => {
+                                    info!(context, "Idle has NewData");
+                                }
+                                // TODO: idle_wait does not distinguish manual interrupts
+                                // from Timeouts if we would know it's a Timeout we could bail
+                                // directly and reconnect .
+                                IdleResponse::Timeout => {
+                                    info!(context, "Idle-wait timeout or interruption");
+                                }
+                                IdleResponse::ManualInterrupt => {
+                                    info!(context, "Idle wait was interrupted");
+                                }
+                            }
+                        }
+                        // if we can't properly terminate the idle
+                        // protocol let's break the connection.
+                        let res =
+                            async_std::future::timeout(Duration::from_secs(15), handle.done())
+                                .await
+                                .map_err(|err| {
+                                    self.trigger_reconnect();
+                                    Error::IdleTimeout(err)
+                                })?;
+
+                        match res {
+                            Ok(session) => {
+                                *self.session.lock().await = Some(Session::Insecure(session));
+                            }
+                            Err(err) => {
+                                // if we cannot terminate IDLE it probably
+                                // means that we waited long (with idle_wait)
+                                // but the network went away/changed
+                                self.trigger_reconnect();
+                                return Err(Error::IdleProtocolFailed(err));
+                            }
+                        }
+                    }
+                }
+            }
+
+            Ok(())
+        })
+    }
+
+    pub(crate) fn fake_idle(&self, context: &Context, watch_folder: Option<String>) {
+        // Idle using polling. This is also needed if we're not yet configured -
+        // in this case, we're waiting for a configure job (and an interrupt).
+        task::block_on(async move {
+            let fake_idle_start_time = SystemTime::now();
+
+            info!(context, "IMAP-fake-IDLEing...");
+
+            let interrupt = stop_token::StopSource::new();
+
+            // check every minute if there are new messages
+            // TODO: grow sleep durations / make them more flexible
+            let interval = async_std::stream::interval(Duration::from_secs(60));
+            let mut interrupt_interval = interrupt.stop_token().stop_stream(interval);
+            *self.interrupt.lock().await = Some(interrupt);
+            if self.skip_next_idle_wait.load(Ordering::SeqCst) {
+                // interrupt_idle has happened before we
+                // provided self.interrupt
+                self.skip_next_idle_wait.store(false, Ordering::SeqCst);
+                info!(context, "fake-idle wait was skipped");
+            } else {
+                // loop until we are interrupted or if we fetched something
+                while let Some(_) = interrupt_interval.next().await {
+                    // try to connect with proper login params
+                    // (setup_handle_if_needed might not know about them if we
+                    // never successfully connected)
+                    if let Err(err) = self.connect_configured(context) {
+                        warn!(context, "fake_idle: could not connect: {}", err);
+                        continue;
+                    }
+                    if self.config.read().await.can_idle {
+                        // we only fake-idled because network was gone during IDLE, probably
+                        break;
+                    }
+                    info!(context, "fake_idle is connected");
+                    // we are connected, let's see if fetching messages results
+                    // in anything.  If so, we behave as if IDLE had data but
+                    // will have already fetched the messages so perform_*_fetch
+                    // will not find any new.
+
+                    if let Some(ref watch_folder) = watch_folder {
+                        match self.fetch_new_messages(context, watch_folder).await {
+                            Ok(res) => {
+                                info!(context, "fetch_new_messages returned {:?}", res);
+                                if res {
+                                    break;
+                                }
+                            }
+                            Err(err) => {
+                                error!(context, "could not fetch from folder: {}", err);
+                                self.trigger_reconnect()
+                            }
+                        }
+                    }
+                }
+            }
+            self.interrupt.lock().await.take();
+
+            info!(
+                context,
+                "IMAP-fake-IDLE done after {:.4}s",
+                SystemTime::now()
+                    .duration_since(fake_idle_start_time)
+                    .unwrap()
+                    .as_millis() as f64
+                    / 1000.,
+            );
+        })
+    }
+
+    pub fn interrupt_idle(&self, context: &Context) {
+        task::block_on(async move {
+            let mut interrupt: Option<stop_token::StopSource> = self.interrupt.lock().await.take();
+            if interrupt.is_none() {
+                // idle wait is not running, signal it needs to skip
+                self.skip_next_idle_wait.store(true, Ordering::SeqCst);
+
+                // meanwhile idle-wait may have produced the StopSource
+                interrupt = self.interrupt.lock().await.take();
+            }
+            // let's manually drop the StopSource
+            if interrupt.is_some() {
+                // the imap thread provided us a stop token but might
+                // not have entered idle_wait yet, give it some time
+                // for that to happen. XXX handle this without extra wait
+                // https://github.com/deltachat/deltachat-core-rust/issues/925
+                std::thread::sleep(Duration::from_millis(200));
+                info!(context, "low-level: dropping stop-source to interrupt idle");
+                std::mem::drop(interrupt)
+            }
+        });
+    }
+}
--- a/src/imap/mod.rs
+++ b/src/imap/mod.rs
@@ -4,14 +4,11 @@
 //! to implement connect, fetch, delete functionality with standard IMAP servers.

 use std::sync::atomic::{AtomicBool, Ordering};
-use std::time::{Duration, SystemTime};

 use async_imap::{
    error::Result as ImapResult,
-    extensions::idle::IdleResponse,
-    types::{Fetch, Flag, Mailbox, Name, NameAttribute},
+    types::{Capability, Fetch, Flag, Mailbox, Name, NameAttribute},
 };
-use async_std::prelude::*;
 use async_std::sync::{Mutex, RwLock};
 use async_std::task;

@@ -28,6 +25,7 @@ use crate::param::Params;
 use crate::stock::StockMessage;
 use crate::wrapmime;

+mod idle;
 pub mod select_folder;

 const DC_IMAP_SEEN: usize = 0x0001;
@@ -51,22 +49,16 @@ pub enum Error {
    #[fail(display = "IMAP Could not login as {}", _0)]
    LoginFailed(String),

-    #[fail(display = "IMAP Could not fetch {}", _0)]
+    #[fail(display = "IMAP Could not fetch")]
    FetchFailed(#[cause] async_imap::error::Error),

-    #[fail(display = "IMAP IDLE protocol failed to init/complete")]
-    IdleProtocolFailed(#[cause] async_imap::error::Error),
-
-    #[fail(display = "IMAP server does not have IDLE capability")]
-    IdleAbilityMissing,
-
    #[fail(display = "IMAP operation attempted while it is torn down")]
    InTeardown,

    #[fail(display = "IMAP operation attempted while it is torn down")]
    SqlError(#[cause] crate::sql::Error),

-    #[fail(display = "IMAP got error from elsewhere: {:?}", _0)]
+    #[fail(display = "IMAP got error from elsewhere")]
    WrappedError(#[cause] crate::error::Error),

    #[fail(display = "IMAP select folder error")]
@@ -201,112 +193,107 @@ impl Imap {
        self.should_reconnect.store(true, Ordering::Relaxed)
    }

-    fn setup_handle_if_needed(&self, context: &Context) -> Result<()> {
-        task::block_on(async move {
-            if self.config.read().await.imap_server.is_empty() {
-                return Err(Error::InTeardown);
-            }
+    async fn setup_handle_if_needed(&self, context: &Context) -> Result<()> {
+        if self.config.read().await.imap_server.is_empty() {
+            return Err(Error::InTeardown);
+        }

-            if self.should_reconnect() {
-                self.unsetup_handle(context).await;
-                self.should_reconnect.store(false, Ordering::Relaxed);
-            } else if self.is_connected().await {
-                return Ok(());
-            }
+        if self.should_reconnect() {
+            self.unsetup_handle(context).await;
+            self.should_reconnect.store(false, Ordering::Relaxed);
+        } else if self.is_connected().await {
+            return Ok(());
+        }

-            let server_flags = self.config.read().await.server_flags as i32;
+        let server_flags = self.config.read().await.server_flags as i32;

-            let connection_res: ImapResult<Client> =
-                if (server_flags & (DC_LP_IMAP_SOCKET_STARTTLS | DC_LP_IMAP_SOCKET_PLAIN)) != 0 {
-                    let config = self.config.read().await;
-                    let imap_server: &str = config.imap_server.as_ref();
-                    let imap_port = config.imap_port;
+        let connection_res: ImapResult<Client> =
+            if (server_flags & (DC_LP_IMAP_SOCKET_STARTTLS | DC_LP_IMAP_SOCKET_PLAIN)) != 0 {
+                let config = self.config.read().await;
+                let imap_server: &str = config.imap_server.as_ref();
+                let imap_port = config.imap_port;

-                    match Client::connect_insecure((imap_server, imap_port)).await {
-                        Ok(client) => {
-                            if (server_flags & DC_LP_IMAP_SOCKET_STARTTLS) != 0 {
-                                client.secure(imap_server, config.certificate_checks).await
-                            } else {
-                                Ok(client)
-                            }
-                        }
-                        Err(err) => Err(err),
-                    }
-                } else {
-                    let config = self.config.read().await;
-                    let imap_server: &str = config.imap_server.as_ref();
-                    let imap_port = config.imap_port;
-
-                    Client::connect_secure(
-                        (imap_server, imap_port),
-                        imap_server,
-                        config.certificate_checks,
-                    )
-                    .await
-                };
-
-            let login_res = match connection_res {
-                Ok(client) => {
-                    let config = self.config.read().await;
-                    let imap_user: &str = config.imap_user.as_ref();
-                    let imap_pw: &str = config.imap_pw.as_ref();
-
-                    if (server_flags & DC_LP_AUTH_OAUTH2) != 0 {
-                        let addr: &str = config.addr.as_ref();
-
-                        if let Some(token) =
-                            dc_get_oauth2_access_token(context, addr, imap_pw, true)
-                        {
-                            let auth = OAuth2 {
-                                user: imap_user.into(),
-                                access_token: token,
-                            };
-                            client.authenticate("XOAUTH2", &auth).await
+                match Client::connect_insecure((imap_server, imap_port)).await {
+                    Ok(client) => {
+                        if (server_flags & DC_LP_IMAP_SOCKET_STARTTLS) != 0 {
+                            client.secure(imap_server, config.certificate_checks).await
                        } else {
-                            return Err(Error::OauthError);
+                            Ok(client)
                        }
-                    } else {
-                        client.login(imap_user, imap_pw).await
                    }
+                    Err(err) => Err(err),
                }
-                Err(err) => {
-                    let message = {
-                        let config = self.config.read().await;
-                        let imap_server: &str = config.imap_server.as_ref();
-                        let imap_port = config.imap_port;
-                        context.stock_string_repl_str2(
-                            StockMessage::ServerResponse,
-                            format!("{}:{}", imap_server, imap_port),
-                            err.to_string(),
-                        )
-                    };
-                    // IMAP connection failures are reported to users
-                    emit_event!(context, Event::ErrorNetwork(message));
-                    return Err(Error::ConnectionFailed(err.to_string()));
-                }
+            } else {
+                let config = self.config.read().await;
+                let imap_server: &str = config.imap_server.as_ref();
+                let imap_port = config.imap_port;
+
+                Client::connect_secure(
+                    (imap_server, imap_port),
+                    imap_server,
+                    config.certificate_checks,
+                )
+                .await
            };

-            self.should_reconnect.store(false, Ordering::Relaxed);
+        let login_res = match connection_res {
+            Ok(client) => {
+                let config = self.config.read().await;
+                let imap_user: &str = config.imap_user.as_ref();
+                let imap_pw: &str = config.imap_pw.as_ref();

-            match login_res {
-                Ok(session) => {
-                    *self.session.lock().await = Some(session);
-                    Ok(())
-                }
-                Err((err, _)) => {
-                    let imap_user = self.config.read().await.imap_user.to_owned();
-                    let message =
-                        context.stock_string_repl_str(StockMessage::CannotLogin, &imap_user);
+                if (server_flags & DC_LP_AUTH_OAUTH2) != 0 {
+                    let addr: &str = config.addr.as_ref();

-                    emit_event!(
-                        context,
-                        Event::ErrorNetwork(format!("{} ({})", message, err))
-                    );
-                    self.trigger_reconnect();
-                    Err(Error::LoginFailed(format!("cannot login as {}", imap_user)))
+                    if let Some(token) = dc_get_oauth2_access_token(context, addr, imap_pw, true) {
+                        let auth = OAuth2 {
+                            user: imap_user.into(),
+                            access_token: token,
+                        };
+                        client.authenticate("XOAUTH2", &auth).await
+                    } else {
+                        return Err(Error::OauthError);
+                    }
+                } else {
+                    client.login(imap_user, imap_pw).await
                }
            }
-        })
+            Err(err) => {
+                let message = {
+                    let config = self.config.read().await;
+                    let imap_server: &str = config.imap_server.as_ref();
+                    let imap_port = config.imap_port;
+                    context.stock_string_repl_str2(
+                        StockMessage::ServerResponse,
+                        format!("{}:{}", imap_server, imap_port),
+                        err.to_string(),
+                    )
+                };
+                // IMAP connection failures are reported to users
+                emit_event!(context, Event::ErrorNetwork(message));
+                return Err(Error::ConnectionFailed(err.to_string()));
+            }
+        };
+
+        self.should_reconnect.store(false, Ordering::Relaxed);
+
+        match login_res {
+            Ok(session) => {
+                *self.session.lock().await = Some(session);
+                Ok(())
+            }
+            Err((err, _)) => {
+                let imap_user = self.config.read().await.imap_user.to_owned();
+                let message = context.stock_string_repl_str(StockMessage::CannotLogin, &imap_user);
+
+                emit_event!(
+                    context,
+                    Event::ErrorNetwork(format!("{} ({})", message, err))
+                );
+                self.trigger_reconnect();
+                Err(Error::LoginFailed(format!("cannot login as {}", imap_user)))
+            }
+        }
    }

    async fn unsetup_handle(&self, context: &Context) {
@@ -342,9 +329,7 @@ impl Imap {

    /// Connects to imap account using already-configured parameters.
    pub fn connect_configured(&self, context: &Context) -> Result<()> {
-        if async_std::task::block_on(async move {
-            self.is_connected().await && !self.should_reconnect()
-        }) {
+        if async_std::task::block_on(self.is_connected()) && !self.should_reconnect() {
            return Ok(());
        }
        if !context.sql.get_raw_config_bool(context, "configured") {
@@ -387,7 +372,7 @@ impl Imap {
                config.server_flags = server_flags;
            }

-            if let Err(err) = self.setup_handle_if_needed(context) {
+            if let Err(err) = self.setup_handle_if_needed(context).await {
                warn!(context, "failed to setup imap handle: {}", err);
                self.free_connect_params().await;
                return false;
@@ -402,9 +387,14 @@ impl Imap {
                        } else {
                            let can_idle = caps.has_str("IDLE");
                            let has_xlist = caps.has_str("XLIST");
-                            let caps_list = caps
-                                .iter()
-                                .fold(String::new(), |s, c| s + &format!(" {:?}", c));
+                            let caps_list = caps.iter().fold(String::new(), |s, c| {
+                                if let Capability::Atom(x) = c {
+                                    s + &format!(" {}", x)
+                                } else {
+                                    s + &format!(" {:?}", c)
+                                }
+                            });
+
                            self.config.write().await.can_idle = can_idle;
                            self.config.write().await.has_xlist = has_xlist;
                            *self.connected.lock().await = true;
@@ -443,20 +433,17 @@ impl Imap {
        });
    }

-    pub fn fetch(&self, context: &Context, watch_folder: &str) -> Result<()> {
-        task::block_on(async move {
-            if !context.sql.is_open() {
-                // probably shutdown
-                return Err(Error::InTeardown);
-            }
-            while self
-                .fetch_from_single_folder(context, &watch_folder)
-                .await?
-            {
-                // We fetch until no more new messages are there.
-            }
-            Ok(())
-        })
+    pub async fn fetch(&self, context: &Context, watch_folder: &str) -> Result<()> {
+        if !context.sql.is_open() {
+            // probably shutdown
+            return Err(Error::InTeardown);
+        }
+        self.setup_handle_if_needed(context).await?;
+
+        while self.fetch_new_messages(context, &watch_folder).await? {
+            // We fetch until no more new messages are there.
+        }
+        Ok(())
    }

    fn get_config_last_seen_uid<S: AsRef<str>>(&self, context: &Context, folder: S) -> (u32, u32) {
@@ -560,7 +547,7 @@ impl Imap {
        })
    }

-    async fn fetch_from_single_folder<S: AsRef<str>>(
+    async fn fetch_new_messages<S: AsRef<str>>(
        &self,
        context: &Context,
        folder: S,
@@ -593,9 +580,11 @@ impl Imap {
        for msg in &list {
            let cur_uid = msg.uid.unwrap_or_default();
            if cur_uid <= last_seen_uid {
-                warn!(
+                // seems that at least dovecot sends the last available UID
+                // even if we asked for higher UID+N:*
+                info!(
                    context,
-                    "unexpected uid {}, last seen was {}", cur_uid, last_seen_uid
+                    "fetch_new_messages: ignoring uid {}, last seen was {}", cur_uid, last_seen_uid
                );
                continue;
            }
@@ -727,217 +716,23 @@ impl Imap {

            if !is_deleted && msg.body().is_some() {
                let body = msg.body().unwrap_or_default();
-                dc_receive_imf(context, &body, folder.as_ref(), server_uid, flags as u32);
+                if let Err(err) =
+                    dc_receive_imf(context, &body, folder.as_ref(), server_uid, flags as u32)
+                {
+                    warn!(
+                        context,
+                        "dc_receive_imf failed for imap-message {}/{}: {:?}",
+                        folder.as_ref(),
+                        server_uid,
+                        err
+                    );
+                }
            }
        }

        1
    }

-    pub fn idle(&self, context: &Context, watch_folder: Option<String>) -> Result<()> {
-        task::block_on(async move {
-            if !self.config.read().await.can_idle {
-                return Err(Error::IdleAbilityMissing);
-            }
-
-            self.setup_handle_if_needed(context)?;
-
-            self.select_folder(context, watch_folder.clone()).await?;
-
-            let session = self.session.lock().await.take();
-            let timeout = Duration::from_secs(23 * 60);
-            if let Some(session) = session {
-                match session.idle() {
-                    // BEWARE: If you change the Secure branch you
-                    // typically also need to change the Insecure branch.
-                    IdleHandle::Secure(mut handle) => {
-                        if let Err(err) = handle.init().await {
-                            return Err(Error::IdleProtocolFailed(err));
-                        }
-
-                        let (idle_wait, interrupt) = handle.wait_with_timeout(timeout);
-                        *self.interrupt.lock().await = Some(interrupt);
-
-                        if self.skip_next_idle_wait.load(Ordering::SeqCst) {
-                            // interrupt_idle has happened before we
-                            // provided self.interrupt
-                            self.skip_next_idle_wait.store(false, Ordering::SeqCst);
-                            std::mem::drop(idle_wait);
-                            info!(context, "Idle wait was skipped");
-                        } else {
-                            info!(context, "Idle entering wait-on-remote state");
-                            match idle_wait.await {
-                                IdleResponse::NewData(_) => {
-                                    info!(context, "Idle has NewData");
-                                }
-                                // TODO: idle_wait does not distinguish manual interrupts
-                                // from Timeouts if we would know it's a Timeout we could bail
-                                // directly and reconnect .
-                                IdleResponse::Timeout => {
-                                    info!(context, "Idle-wait timeout or interruption");
-                                }
-                                IdleResponse::ManualInterrupt => {
-                                    info!(context, "Idle wait was interrupted");
-                                }
-                            }
-                        }
-                        match handle.done().await {
-                            Ok(session) => {
-                                *self.session.lock().await = Some(Session::Secure(session));
-                            }
-                            Err(err) => {
-                                // if we cannot terminate IDLE it probably
-                                // means that we waited long (with idle_wait)
-                                // but the network went away/changed
-                                self.trigger_reconnect();
-                                return Err(Error::IdleProtocolFailed(err));
-                            }
-                        }
-                    }
-                    IdleHandle::Insecure(mut handle) => {
-                        if let Err(err) = handle.init().await {
-                            return Err(Error::IdleProtocolFailed(err));
-                        }
-
-                        let (idle_wait, interrupt) = handle.wait_with_timeout(timeout);
-                        *self.interrupt.lock().await = Some(interrupt);
-
-                        if self.skip_next_idle_wait.load(Ordering::SeqCst) {
-                            // interrupt_idle has happened before we
-                            // provided self.interrupt
-                            self.skip_next_idle_wait.store(false, Ordering::SeqCst);
-                            std::mem::drop(idle_wait);
-                            info!(context, "Idle wait was skipped");
-                        } else {
-                            info!(context, "Idle entering wait-on-remote state");
-                            match idle_wait.await {
-                                IdleResponse::NewData(_) => {
-                                    info!(context, "Idle has NewData");
-                                }
-                                // TODO: idle_wait does not distinguish manual interrupts
-                                // from Timeouts if we would know it's a Timeout we could bail
-                                // directly and reconnect .
-                                IdleResponse::Timeout => {
-                                    info!(context, "Idle-wait timeout or interruption");
-                                }
-                                IdleResponse::ManualInterrupt => {
-                                    info!(context, "Idle wait was interrupted");
-                                }
-                            }
-                        }
-                        match handle.done().await {
-                            Ok(session) => {
-                                *self.session.lock().await = Some(Session::Insecure(session));
-                            }
-                            Err(err) => {
-                                // if we cannot terminate IDLE it probably
-                                // means that we waited long (with idle_wait)
-                                // but the network went away/changed
-                                self.trigger_reconnect();
-                                return Err(Error::IdleProtocolFailed(err));
-                            }
-                        }
-                    }
-                }
-            }
-
-            Ok(())
-        })
-    }
-
-    pub(crate) fn fake_idle(&self, context: &Context, watch_folder: Option<String>) {
-        // Idle using polling. This is also needed if we're not yet configured -
-        // in this case, we're waiting for a configure job (and an interrupt).
-        task::block_on(async move {
-            let fake_idle_start_time = SystemTime::now();
-
-            info!(context, "IMAP-fake-IDLEing...");
-
-            let interrupt = stop_token::StopSource::new();
-
-            // check every minute if there are new messages
-            // TODO: grow sleep durations / make them more flexible
-            let interval = async_std::stream::interval(Duration::from_secs(60));
-            let mut interrupt_interval = interrupt.stop_token().stop_stream(interval);
-            *self.interrupt.lock().await = Some(interrupt);
-            if self.skip_next_idle_wait.load(Ordering::SeqCst) {
-                // interrupt_idle has happened before we
-                // provided self.interrupt
-                self.skip_next_idle_wait.store(false, Ordering::SeqCst);
-                info!(context, "fake-idle wait was skipped");
-            } else {
-                // loop until we are interrupted or if we fetched something
-                while let Some(_) = interrupt_interval.next().await {
-                    // try to connect with proper login params
-                    // (setup_handle_if_needed might not know about them if we
-                    // never successfully connected)
-                    if let Err(err) = self.connect_configured(context) {
-                        warn!(context, "fake_idle: could not connect: {}", err);
-                        continue;
-                    }
-                    if self.config.read().await.can_idle {
-                        // we only fake-idled because network was gone during IDLE, probably
-                        break;
-                    }
-                    info!(context, "fake_idle is connected");
-                    // we are connected, let's see if fetching messages results
-                    // in anything.  If so, we behave as if IDLE had data but
-                    // will have already fetched the messages so perform_*_fetch
-                    // will not find any new.
-
-                    if let Some(ref watch_folder) = watch_folder {
-                        match self.fetch_from_single_folder(context, watch_folder).await {
-                            Ok(res) => {
-                                info!(context, "fetch_from_single_folder returned {:?}", res);
-                                if res {
-                                    break;
-                                }
-                            }
-                            Err(err) => {
-                                error!(context, "could not fetch from folder: {}", err);
-                                self.trigger_reconnect()
-                            }
-                        }
-                    }
-                }
-            }
-            self.interrupt.lock().await.take();
-
-            info!(
-                context,
-                "IMAP-fake-IDLE done after {:.4}s",
-                SystemTime::now()
-                    .duration_since(fake_idle_start_time)
-                    .unwrap()
-                    .as_millis() as f64
-                    / 1000.,
-            );
-        })
-    }
-
-    pub fn interrupt_idle(&self, context: &Context) {
-        task::block_on(async move {
-            let mut interrupt: Option<stop_token::StopSource> = self.interrupt.lock().await.take();
-            if interrupt.is_none() {
-                // idle wait is not running, signal it needs to skip
-                self.skip_next_idle_wait.store(true, Ordering::SeqCst);
-
-                // meanwhile idle-wait may have produced the StopSource
-                interrupt = self.interrupt.lock().await.take();
-            }
-            // let's manually drop the StopSource
-            if interrupt.is_some() {
-                // the imap thread provided us a stop token but might
-                // not have entered idle_wait yet, give it some time
-                // for that to happen. XXX handle this without extra wait
-                // https://github.com/deltachat/deltachat-core-rust/issues/925
-                std::thread::sleep(Duration::from_millis(200));
-                info!(context, "low-level: dropping stop-source to interrupt idle");
-                std::mem::drop(interrupt)
-            }
-        });
-    }
-
    pub fn mv(
        &self,
        context: &Context,
@@ -1322,13 +1117,17 @@ impl Imap {
        task::block_on(async move {
            info!(context, "emptying folder {}", folder);

+            // we want to report all error to the user
+            // (no retry should be attempted)
            if folder.is_empty() {
                error!(context, "cannot perform empty, folder not set");
                return;
            }
+            if let Err(err) = self.setup_handle_if_needed(context).await {
+                error!(context, "could not setup imap connection: {:?}", err);
+                return;
+            }
            if let Err(err) = self.select_folder(context, Some(&folder)).await {
-                // we want to report all error to the user
-                // (no retry should be attempted)
                error!(
                    context,
                    "Could not select {} for expunging: {:?}", folder, err
--- a/src/imap/select_folder.rs
+++ b/src/imap/select_folder.rs
@@ -34,6 +34,7 @@ impl Imap {
            let mut cfg = self.config.write().await;
            cfg.selected_folder = None;
            cfg.selected_folder_needs_expunge = false;
+            self.trigger_reconnect();
            return Err(Error::NoSession);
        }

@@ -61,6 +62,7 @@ impl Imap {
                            info!(context, "close/expunge succeeded");
                        }
                        Err(err) => {
+                            self.trigger_reconnect();
                            return Err(Error::CloseExpungeFailed(err));
                        }
                    }
--- a/src/imap_client.rs
+++ b/src/imap_client.rs
@@ -4,12 +4,11 @@ use async_imap::{
    types::{Capabilities, Fetch, Mailbox, Name},
    Client as ImapClient, Session as ImapSession,
 };
+use async_native_tls::TlsStream;
 use async_std::net::{self, TcpStream};
 use async_std::prelude::*;
-use async_std::sync::Arc;
-use async_tls::client::TlsStream;

-use crate::login_param::{dc_build_tls_config, CertificateChecks};
+use crate::login_param::{dc_build_tls, CertificateChecks};

 #[derive(Debug)]
 pub(crate) enum Client {
@@ -36,9 +35,9 @@ impl Client {
        certificate_checks: CertificateChecks,
    ) -> ImapResult<Self> {
        let stream = TcpStream::connect(addr).await?;
-        let tls_config = dc_build_tls_config(certificate_checks);
-        let tls_connector: async_tls::TlsConnector = Arc::new(tls_config).into();
-        let tls_stream = tls_connector.connect(domain.as_ref(), stream)?.await?;
+        let tls = dc_build_tls(certificate_checks)?;
+        let tls_connector: async_native_tls::TlsConnector = tls.into();
+        let tls_stream = tls_connector.connect(domain.as_ref(), stream).await?;
        let mut client = ImapClient::new(tls_stream);
        if std::env::var(crate::DCC_IMAP_DEBUG).is_ok() {
            client.debug = true;
@@ -74,10 +73,10 @@ impl Client {
    ) -> ImapResult<Client> {
        match self {
            Client::Insecure(client) => {
-                let tls_config = dc_build_tls_config(certificate_checks);
-                let tls: async_tls::TlsConnector = Arc::new(tls_config).into();
+                let tls = dc_build_tls(certificate_checks)?;
+                let tls_stream = tls.into();

-                let client_sec = client.secure(domain, &tls).await?;
+                let client_sec = client.secure(domain, &tls_stream).await?;

                Ok(Client::Secure(client_sec))
            }
--- a/src/job.rs
+++ b/src/job.rs
@@ -8,6 +8,8 @@ use std::time::Duration;
 use deltachat_derive::{FromSql, ToSql};
 use rand::{thread_rng, Rng};

+use async_std::task;
+
 use crate::blob::BlobObject;
 use crate::chat;
 use crate::config::Config;
@@ -165,13 +167,15 @@ impl Job {
                if let Some(recipients) = self.param.get(Param::Recipients) {
                    let recipients_list = recipients
                        .split('\x1e')
-                        .filter_map(|addr| match lettre::EmailAddress::new(addr.to_string()) {
-                            Ok(addr) => Some(addr),
-                            Err(err) => {
-                                warn!(context, "invalid recipient: {} {:?}", addr, err);
-                                None
-                            }
-                        })
+                        .filter_map(
+                            |addr| match async_smtp::EmailAddress::new(addr.to_string()) {
+                                Ok(addr) => Some(addr),
+                                Err(err) => {
+                                    warn!(context, "invalid recipient: {} {:?}", addr, err);
+                                    None
+                                }
+                            },
+                        )
                        .collect::<Vec<_>>();

                    /* if there is a msg-id and it does not exist in the db, cancel sending.
@@ -196,11 +200,11 @@ impl Job {
                        info!(context, "smtp-sending out mime message:");
                        println!("{}", String::from_utf8_lossy(&body));
                    }
-                    match smtp.send(context, recipients_list, body, self.job_id) {
+                    match task::block_on(smtp.send(context, recipients_list, body, self.job_id)) {
                        Err(crate::smtp::send::Error::SendError(err)) => {
                            // Remote error, retry later.
-                            smtp.disconnect();
                            info!(context, "SMTP failed to send: {}", err);
+                            smtp.disconnect();
                            self.try_again_later(TryAgain::AtOnce, Some(err.to_string()));
                        }
                        Err(crate::smtp::send::Error::EnvelopeError(err)) => {
@@ -398,31 +402,37 @@ pub fn job_kill_action(context: &Context, action: Action) -> bool {
 pub fn perform_inbox_fetch(context: &Context) {
    let use_network = context.get_config_bool(Config::InboxWatch);

-    context
-        .inbox_thread
-        .write()
-        .unwrap()
-        .fetch(context, use_network);
+    task::block_on(
+        context
+            .inbox_thread
+            .write()
+            .unwrap()
+            .fetch(context, use_network),
+    );
 }

 pub fn perform_mvbox_fetch(context: &Context) {
    let use_network = context.get_config_bool(Config::MvboxWatch);

-    context
-        .mvbox_thread
-        .write()
-        .unwrap()
-        .fetch(context, use_network);
+    task::block_on(
+        context
+            .mvbox_thread
+            .write()
+            .unwrap()
+            .fetch(context, use_network),
+    );
 }

 pub fn perform_sentbox_fetch(context: &Context) {
    let use_network = context.get_config_bool(Config::SentboxWatch);

-    context
-        .sentbox_thread
-        .write()
-        .unwrap()
-        .fetch(context, use_network);
+    task::block_on(
+        context
+            .sentbox_thread
+            .write()
+            .unwrap()
+            .fetch(context, use_network),
+    );
 }

 pub fn perform_inbox_idle(context: &Context) {
@@ -623,7 +633,15 @@ pub fn job_send_msg(context: &Context, msg_id: MsgId) -> Result<(), Error> {
    /* create message */
    let needs_encryption = msg.param.get_int(Param::GuaranteeE2ee).unwrap_or_default();

-    let mimefactory = MimeFactory::from_msg(context, &msg)?;
+    let attach_selfavatar = match chat::shall_attach_selfavatar(context, msg.chat_id) {
+        Ok(attach_selfavatar) => attach_selfavatar,
+        Err(err) => {
+            warn!(context, "job: cannot get selfavatar-state: {}", err);
+            false
+        }
+    };
+
+    let mimefactory = MimeFactory::from_msg(context, &msg, attach_selfavatar)?;
    let mut rendered_msg = mimefactory.render().map_err(|err| {
        message::set_msg_failed(context, msg_id, Some(err.to_string()));
        err
@@ -660,8 +678,9 @@ pub fn job_send_msg(context: &Context, msg_id: MsgId) -> Result<(), Error> {
    }

    if rendered_msg.is_gossiped {
-        chat::set_gossiped_timestamp(context, msg.chat_id, time());
+        chat::set_gossiped_timestamp(context, msg.chat_id, time())?;
    }
+
    if 0 != rendered_msg.last_added_location_id {
        if let Err(err) = location::set_kml_sent_timestamp(context, msg.chat_id, time()) {
            error!(context, "Failed to set kml sent_timestamp: {:?}", err);
@@ -674,6 +693,13 @@ pub fn job_send_msg(context: &Context, msg_id: MsgId) -> Result<(), Error> {
            }
        }
    }
+
+    if attach_selfavatar {
+        if let Err(err) = chat::set_selfavatar_timestamp(context, msg.chat_id, time()) {
+            error!(context, "Failed to set selfavatar timestamp: {:?}", err);
+        }
+    }
+
    if rendered_msg.is_encrypted && needs_encryption == 0 {
        msg.param.set_int(Param::GuaranteeE2ee, 1);
        msg.save_param_to_disk(context);
--- a/src/job_thread.rs
+++ b/src/job_thread.rs
@@ -73,7 +73,7 @@ impl JobThread {
        info!(context, "Interrupting {}-IDLE... finished", self.name);
    }

-    pub fn fetch(&mut self, context: &Context, use_network: bool) {
+    pub async fn fetch(&mut self, context: &Context, use_network: bool) {
        {
            let &(ref lock, _) = &*self.state.clone();
            let mut state = lock.lock().unwrap();
@@ -86,10 +86,10 @@ impl JobThread {
        }

        if use_network {
-            if let Err(err) = self.connect_and_fetch(context) {
+            if let Err(err) = self.connect_and_fetch(context).await {
                warn!(context, "connect+fetch failed: {}, reconnect & retry", err);
                self.imap.trigger_reconnect();
-                if let Err(err) = self.connect_and_fetch(context) {
+                if let Err(err) = self.connect_and_fetch(context).await {
                    warn!(context, "connect+fetch failed: {}", err);
                }
            }
@@ -97,14 +97,18 @@ impl JobThread {
        self.state.0.lock().unwrap().using_handle = false;
    }

-    fn connect_and_fetch(&mut self, context: &Context) -> Result<()> {
+    async fn connect_and_fetch(&mut self, context: &Context) -> Result<()> {
        let prefix = format!("{}-fetch", self.name);
        match self.imap.connect_configured(context) {
            Ok(()) => {
                if let Some(watch_folder) = self.get_watch_folder(context) {
                    let start = std::time::Instant::now();
                    info!(context, "{} started...", prefix);
-                    let res = self.imap.fetch(context, &watch_folder).map_err(Into::into);
+                    let res = self
+                        .imap
+                        .fetch(context, &watch_folder)
+                        .await
+                        .map_err(Into::into);
                    let elapsed = start.elapsed().as_millis();
                    info!(context, "{} done in {:.3} ms.", prefix, elapsed);

@@ -170,20 +174,19 @@ impl JobThread {
        let prefix = format!("{}-IDLE", self.name);
        let do_fake_idle = match self.imap.connect_configured(context) {
            Ok(()) => {
-                info!(context, "{} started...", prefix);
-                let watch_folder = self.get_watch_folder(context);
-                let res = self.imap.idle(context, watch_folder);
-                info!(context, "{} ended...", prefix);
-                match res {
-                    Ok(()) => false,
-                    Err(crate::imap::Error::IdleAbilityMissing) => true, // we have to do fake_idle
-                    Err(err) => {
+                if !self.imap.can_idle() {
+                    true // we have to do fake_idle
+                } else {
+                    let watch_folder = self.get_watch_folder(context);
+                    info!(context, "{} started...", prefix);
+                    let res = self.imap.idle(context, watch_folder);
+                    info!(context, "{} ended...", prefix);
+                    if let Err(err) = res {
                        warn!(context, "{} failed: {} -> reconnecting", prefix, err);
                        // something is borked, let's start afresh on the next occassion
                        self.imap.disconnect(context);
-
-                        false
                    }
+                    false
                }
            }
            Err(err) => {
--- a/src/key.rs
+++ b/src/key.rs
@@ -178,20 +178,9 @@ impl Key {
        }
    }

-    pub fn to_base64(&self, break_every: usize) -> String {
+    pub fn to_base64(&self) -> String {
        let buf = self.to_bytes();
-
-        let encoded = base64::encode(&buf);
-        encoded
-            .chars()
-            .enumerate()
-            .fold(String::new(), |mut res, (i, c)| {
-                if i > 0 && i % break_every == 0 {
-                    res.push(' ')
-                }
-                res.push(c);
-                res
-            })
+        base64::encode(&buf)
    }

    pub fn to_armored_string(
--- a/src/lib.rs
+++ b/src/lib.rs
@@ -3,16 +3,11 @@
 #![allow(
    clippy::type_complexity,
    clippy::cognitive_complexity,
-    clippy::too_many_arguments,
-    clippy::block_in_if_condition_stmt,
-    clippy::large_enum_variant
-)]
-#![allow(
-    clippy::unreadable_literal,
-    clippy::needless_range_loop,
-    clippy::match_bool
+    clippy::too_many_arguments
 )]
+#![allow(clippy::unreadable_literal, clippy::match_bool)]
 #![feature(ptr_wrapping_offset_from)]
+#![feature(drain_filter)]

 #[macro_use]
 extern crate failure_derive;
@@ -33,6 +28,8 @@ mod log;
 #[macro_use]
 pub mod error;

+pub mod headerdef;
+
 pub(crate) mod events;
 pub use events::*;

@@ -73,7 +70,6 @@ mod token;
 mod wrapmime;
 mod dehtml;

-pub mod dc_array;
 pub mod dc_receive_imf;
 mod dc_simplify;
 pub mod dc_tools;
--- a/src/login_param.rs
+++ b/src/login_param.rs
@@ -4,11 +4,6 @@ use std::borrow::Cow;
 use std::fmt;

 use crate::context::Context;
-use crate::error::Error;
-use async_std::sync::Arc;
-use rustls;
-use webpki;
-use webpki_roots;

 #[derive(Copy, Clone, Debug, Display, FromPrimitive)]
 #[repr(i32)]
@@ -16,7 +11,11 @@ use webpki_roots;
 pub enum CertificateChecks {
    Automatic = 0,
    Strict = 1,
-    AcceptInvalidHostnames = 2,
+
+    /// Same as AcceptInvalidCertificates
+    /// Previously known as AcceptInvalidHostnames, now deprecated.
+    AcceptInvalidCertificates2 = 2,
+
    AcceptInvalidCertificates = 3,
 }

@@ -130,7 +129,7 @@ impl LoginParam {
        &self,
        context: &Context,
        prefix: impl AsRef<str>,
-    ) -> Result<(), Error> {
+    ) -> crate::sql::Result<()> {
        let prefix = prefix.as_ref();
        let sql = &context.sql;

@@ -259,49 +258,25 @@ fn get_readable_flags(flags: i32) -> String {
    res
 }

-pub struct NoCertificateVerification {}
-
-impl rustls::ServerCertVerifier for NoCertificateVerification {
-    fn verify_server_cert(
-        &self,
-        _roots: &rustls::RootCertStore,
-        _presented_certs: &[rustls::Certificate],
-        _dns_name: webpki::DNSNameRef<'_>,
-        _ocsp: &[u8],
-    ) -> Result<rustls::ServerCertVerified, rustls::TLSError> {
-        Ok(rustls::ServerCertVerified::assertion())
-    }
-}
-
-pub fn dc_build_tls_config(certificate_checks: CertificateChecks) -> rustls::ClientConfig {
-    let mut config = rustls::ClientConfig::new();
-    config
-        .root_store
-        .add_server_trust_anchors(&webpki_roots::TLS_SERVER_ROOTS);
-
+pub fn dc_build_tls(
+    certificate_checks: CertificateChecks,
+) -> Result<native_tls::TlsConnector, native_tls::Error> {
+    let mut tls_builder = native_tls::TlsConnector::builder();
    match certificate_checks {
-        CertificateChecks::Strict => {}
        CertificateChecks::Automatic => {
            // Same as AcceptInvalidCertificates for now.
            // TODO: use provider database when it becomes available
-            config
-                .dangerous()
-                .set_certificate_verifier(Arc::new(NoCertificateVerification {}));
-        }
-        CertificateChecks::AcceptInvalidCertificates => {
-            // TODO: only accept invalid certs
-            config
-                .dangerous()
-                .set_certificate_verifier(Arc::new(NoCertificateVerification {}));
-        }
-        CertificateChecks::AcceptInvalidHostnames => {
-            // TODO: only accept invalid hostnames
-            config
-                .dangerous()
-                .set_certificate_verifier(Arc::new(NoCertificateVerification {}));
+            tls_builder
+                .danger_accept_invalid_hostnames(true)
+                .danger_accept_invalid_certs(true)
        }
+        CertificateChecks::Strict => &mut tls_builder,
+        CertificateChecks::AcceptInvalidCertificates
+        | CertificateChecks::AcceptInvalidCertificates2 => tls_builder
+            .danger_accept_invalid_hostnames(true)
+            .danger_accept_invalid_certs(true),
    }
-    config
+    .build()
 }

 #[cfg(test)]
@@ -313,8 +288,8 @@ mod tests {
        use std::string::ToString;

        assert_eq!(
-            "accept_invalid_hostnames".to_string(),
-            CertificateChecks::AcceptInvalidHostnames.to_string()
+            "accept_invalid_certificates".to_string(),
+            CertificateChecks::AcceptInvalidCertificates.to_string()
        );
    }
 }
--- a/src/mimefactory.rs
+++ b/src/mimefactory.rs
@@ -1,6 +1,7 @@
 use chrono::TimeZone;
 use lettre_email::{mime, Address, Header, MimeMultipartType, PartBuilder};

+use crate::blob::BlobObject;
 use crate::chat::{self, Chat};
 use crate::config::Config;
 use crate::constants::*;
@@ -41,6 +42,7 @@ pub struct MimeFactory<'a, 'b> {
    pub req_mdn: bool,
    pub context: &'a Context,
    last_added_location_id: u32,
+    attach_selfavatar: bool,
 }

 /// Result of rendering a message, ready to be submitted to a send job.
@@ -62,7 +64,11 @@ pub struct RenderedEmail {
 }

 impl<'a, 'b> MimeFactory<'a, 'b> {
-    pub fn from_msg(context: &'a Context, msg: &'b Message) -> Result<MimeFactory<'a, 'b>, Error> {
+    pub fn from_msg(
+        context: &'a Context,
+        msg: &'b Message,
+        add_selfavatar: bool,
+    ) -> Result<MimeFactory<'a, 'b>, Error> {
        let chat = Chat::load_from_db(context, msg.chat_id)?;

        let mut factory = MimeFactory {
@@ -84,6 +90,7 @@ impl<'a, 'b> MimeFactory<'a, 'b> {
            references: String::default(),
            req_mdn: false,
            last_added_location_id: 0,
+            attach_selfavatar: add_selfavatar,
            context,
        };

@@ -152,7 +159,10 @@ impl<'a, 'b> MimeFactory<'a, 'b> {
                let in_reply_to: String = row.get(0)?;
                let references: String = row.get(1)?;

-                Ok((in_reply_to, references))
+                Ok((
+                    render_rfc724_mid_list(&in_reply_to),
+                    render_rfc724_mid_list(&references),
+                ))
            },
        );

@@ -207,6 +217,7 @@ impl<'a, 'b> MimeFactory<'a, 'b> {
            references: String::default(),
            req_mdn: false,
            last_added_location_id: 0,
+            attach_selfavatar: false,
        })
    }

@@ -293,9 +304,8 @@ impl<'a, 'b> MimeFactory<'a, 'b> {
            Loaded::Message => {
                let chat = self.chat.as_ref().unwrap();
                // beside key- and member-changes, force re-gossip every 48 hours
-                if chat.gossiped_timestamp == 0
-                    || (chat.gossiped_timestamp + (2 * 24 * 60 * 60)) > time()
-                {
+                let gossiped_timestamp = chat.get_gossiped_timestamp(self.context);
+                if gossiped_timestamp == 0 || (gossiped_timestamp + (2 * 24 * 60 * 60)) > time() {
                    return true;
                }

@@ -321,6 +331,15 @@ impl<'a, 'b> MimeFactory<'a, 'b> {
                    _ => {}
                }

+                if self
+                    .msg
+                    .param
+                    .get_bool(Param::AttachGroupImage)
+                    .unwrap_or_default()
+                {
+                    return chat.param.get(Param::ProfileImage).map(Into::into);
+                }
+
                None
            }
            Loaded::MDN => None,
@@ -382,7 +401,7 @@ impl<'a, 'b> MimeFactory<'a, 'b> {
        let mut unprotected_headers: Vec<Header> = Vec::new();

        let from = Address::new_mailbox_with_name(
-            encode_words(&self.from_displayname),
+            self.from_displayname.to_string(),
            self.from_addr.clone(),
        );

@@ -394,7 +413,7 @@ impl<'a, 'b> MimeFactory<'a, 'b> {
                to.push(Address::new_mailbox(addr.clone()));
            } else {
                to.push(Address::new_mailbox_with_name(
-                    encode_words(name),
+                    name.to_string(),
                    addr.clone(),
                ));
            }
@@ -443,7 +462,6 @@ impl<'a, 'b> MimeFactory<'a, 'b> {
        }

        let min_verified = self.min_verified();
-        let do_gossip = self.should_do_gossip();
        let grpimage = self.grpimage();
        let force_plaintext = self.should_force_plaintext();
        let subject_str = self.subject_str();
@@ -477,19 +495,31 @@ impl<'a, 'b> MimeFactory<'a, 'b> {
            Loaded::MDN => dc_create_outgoing_rfc724_mid(None, &self.from_addr),
        };

-        protected_headers.push(Header::new("Message-ID".into(), rfc724_mid.clone()));
+        // we could also store the message-id in the protected headers
+        // which would probably help to survive providers like
+        // Outlook.com or hotmail which mangle the Message-ID.
+        // but they also strip the Autocrypt header so we probably
+        // never get a chance to tunnel our protected headers in a
+        // cryptographic payload.
+        unprotected_headers.push(Header::new(
+            "Message-ID".into(),
+            render_rfc724_mid(&rfc724_mid),
+        ));

        unprotected_headers.push(Header::new_with_value("To".into(), to).unwrap());
        unprotected_headers.push(Header::new_with_value("From".into(), vec![from]).unwrap());

+        let mut is_gossiped = false;
+
        let outer_message = if is_encrypted {
-            // Add gossip headers
-            if do_gossip {
+            // Add gossip headers in chats with multiple recipients
+            if peerstates.len() > 1 && self.should_do_gossip() {
                for peerstate in peerstates.iter().filter_map(|(state, _)| state.as_ref()) {
                    if peerstate.peek_key(min_verified).is_some() {
                        if let Some(header) = peerstate.render_gossip_header(min_verified) {
                            message =
                                message.header(Header::new("Autocrypt-Gossip".into(), header));
+                            is_gossiped = true;
                        }
                    }
                }
@@ -525,6 +555,12 @@ impl<'a, 'b> MimeFactory<'a, 'b> {
                outer_message = outer_message.header(header);
            }

+            if std::env::var(crate::DCC_MIME_DEBUG).is_ok() {
+                info!(self.context, "mimefactory: outgoing message mime:");
+                let raw_message = message.clone().build().as_string();
+                println!("{}", raw_message);
+            }
+
            let encrypted =
                encrypt_helper.encrypt(self.context, min_verified, message, &peerstates)?;

@@ -564,8 +600,6 @@ impl<'a, 'b> MimeFactory<'a, 'b> {
            message
        };

-        let is_gossiped = is_encrypted && do_gossip && !peerstates.is_empty();
-
        let MimeFactory {
            recipients_addr,
            from_addr,
@@ -602,6 +636,7 @@ impl<'a, 'b> MimeFactory<'a, 'b> {
        let command = self.msg.param.get_cmd();
        let mut placeholdertext = None;
        let mut meta_part = None;
+        let mut add_compatibility_header = false;

        if chat.typ == Chattype::VerifiedGroup {
            protected_headers.push(Header::new("Chat-Verified".to_string(), "1".to_string()));
@@ -642,6 +677,7 @@ impl<'a, 'b> MimeFactory<'a, 'b> {
                            "vg-member-added".to_string(),
                        ));
                    }
+                    add_compatibility_header = true;
                }
                SystemMessage::GroupNameChanged => {
                    let value_to_add = self.msg.param.get(Param::Arg).unwrap_or_default();
@@ -652,10 +688,17 @@ impl<'a, 'b> MimeFactory<'a, 'b> {
                    ));
                }
                SystemMessage::GroupImageChanged => {
+                    protected_headers.push(Header::new(
+                        "Chat-Content".to_string(),
+                        "group-avatar-changed".to_string(),
+                    ));
                    if grpimage.is_none() {
-                        protected_headers
-                            .push(Header::new("Chat-Group-Image".to_string(), "0".to_string()));
+                        protected_headers.push(Header::new(
+                            "Chat-Group-Avatar".to_string(),
+                            "0".to_string(),
+                        ));
                    }
+                    add_compatibility_header = true;
                }
                _ => {}
            }
@@ -723,7 +766,18 @@ impl<'a, 'b> MimeFactory<'a, 'b> {

            let (mail, filename_as_sent) = build_body_file(context, &meta, "group-image")?;
            meta_part = Some(mail);
-            protected_headers.push(Header::new("Chat-Group-Image".into(), filename_as_sent));
+            protected_headers.push(Header::new(
+                "Chat-Group-Avatar".into(),
+                filename_as_sent.clone(),
+            ));
+
+            // add the old group-image headers for versions <=0.973 resp. <=beta.15 (december 2019)
+            // image deletion is not supported in the compatibility layer.
+            // this can be removed some time after releasing 1.0,
+            // grep for #DeprecatedAvatar to get the place where compatibility parsing takes place.
+            if add_compatibility_header {
+                protected_headers.push(Header::new("Chat-Group-Image".into(), filename_as_sent));
+            }
        }

        if self.msg.type_0 == Viewtype::Sticker {
@@ -855,6 +909,19 @@ impl<'a, 'b> MimeFactory<'a, 'b> {
            }
        }

+        if self.attach_selfavatar {
+            match context.get_config(Config::Selfavatar) {
+                Some(path) => match build_selfavatar_file(context, path) {
+                    Ok((part, filename)) => {
+                        parts.push(part);
+                        protected_headers.push(Header::new("Chat-User-Avatar".into(), filename))
+                    }
+                    Err(err) => warn!(context, "mimefactory: cannot attach selfavatar: {}", err),
+                },
+                None => protected_headers.push(Header::new("Chat-User-Avatar".into(), "0".into())),
+            }
+        }
+
        // Single part, render as regular message.
        if parts.len() == 1 {
            return Ok(parts.pop().unwrap());
@@ -1006,6 +1073,31 @@ fn build_body_file(
    Ok((mail, filename_to_send))
 }

+fn build_selfavatar_file(context: &Context, path: String) -> Result<(PartBuilder, String), Error> {
+    let blob = BlobObject::from_path(context, path)?;
+    let filename_to_send = match blob.suffix() {
+        Some(suffix) => format!("avatar.{}", suffix),
+        None => "avatar".to_string(),
+    };
+    let mimetype = match message::guess_msgtype_from_suffix(blob.as_rel_path()) {
+        Some(res) => res.1.parse()?,
+        None => mime::APPLICATION_OCTET_STREAM,
+    };
+    let body = std::fs::read(blob.to_abs_path())?;
+    let encoded_body = base64::encode(&body);
+
+    let part = PartBuilder::new()
+        .content_type(&mimetype)
+        .header((
+            "Content-Disposition",
+            format!("attachment; filename=\"{}\"", &filename_to_send),
+        ))
+        .header(("Content-Transfer-Encoding", "base64"))
+        .body(encoded_body);
+
+    Ok((part, filename_to_send))
+}
+
 pub(crate) fn vec_contains_lowercase(vec: &[String], part: &str) -> bool {
    let partlc = part.to_lowercase();
    for cur in vec.iter() {
@@ -1026,6 +1118,25 @@ fn is_file_size_okay(context: &Context, msg: &Message) -> bool {
    }
 }

+fn render_rfc724_mid(rfc724_mid: &str) -> String {
+    let rfc724_mid = rfc724_mid.trim().to_string();
+
+    if rfc724_mid.chars().nth(0).unwrap_or_default() == '<' {
+        rfc724_mid
+    } else {
+        format!("<{}>", rfc724_mid)
+    }
+}
+
+fn render_rfc724_mid_list(mid_list: &str) -> String {
+    mid_list
+        .trim()
+        .split_ascii_whitespace()
+        .map(render_rfc724_mid)
+        .collect::<Vec<String>>()
+        .join(" ")
+}
+
 /* ******************************************************************************
 * Encode/decode header words, RFC 2047
 ******************************************************************************/
@@ -1045,3 +1156,51 @@ pub fn needs_encoding(to_check: impl AsRef<str>) -> bool {
        !c.is_ascii_alphanumeric() && c != '-' && c != '_' && c != '.' && c != '~' && c != '%'
    })
 }
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    #[test]
+    fn test_render_email_address() {
+        let display_name = "ä space";
+        let addr = "x@y.org";
+
+        assert!(!display_name.is_ascii());
+
+        let s = format!(
+            "{}",
+            Address::new_mailbox_with_name(display_name.to_string(), addr.to_string())
+        );
+
+        println!("{}", s);
+
+        assert_eq!(s, "=?utf-8?q?=C3=A4_space?= <x@y.org>");
+    }
+
+    #[test]
+    fn test_render_rfc724_mid() {
+        assert_eq!(
+            render_rfc724_mid("kqjwle123@qlwe"),
+            "<kqjwle123@qlwe>".to_string()
+        );
+        assert_eq!(
+            render_rfc724_mid("  kqjwle123@qlwe "),
+            "<kqjwle123@qlwe>".to_string()
+        );
+        assert_eq!(
+            render_rfc724_mid("<kqjwle123@qlwe>"),
+            "<kqjwle123@qlwe>".to_string()
+        );
+    }
+
+    #[test]
+    fn test_render_rc724_mid_list() {
+        assert_eq!(render_rfc724_mid_list("123@q "), "<123@q>".to_string());
+        assert_eq!(render_rfc724_mid_list(" 123@q "), "<123@q>".to_string());
+        assert_eq!(
+            render_rfc724_mid_list("123@q 456@d "),
+            "<123@q> <456@d>".to_string()
+        );
+    }
+}
--- a/src/mimeparser.rs
+++ b/src/mimeparser.rs
@@ -2,7 +2,7 @@ use std::collections::{HashMap, HashSet};

 use deltachat_derive::{FromSql, ToSql};
 use lettre_email::mime::{self, Mime};
-use mailparse::MailHeaderMap;
+use mailparse::{DispositionType, MailHeaderMap};

 use crate::aheader::Aheader;
 use crate::blob::BlobObject;
@@ -14,6 +14,7 @@ use crate::dc_simplify::*;
 use crate::dc_tools::*;
 use crate::e2ee;
 use crate::error::Result;
+use crate::headerdef::HeaderDef;
 use crate::job::{job_add, Action};
 use crate::location;
 use crate::message;
@@ -22,27 +23,44 @@ use crate::param::*;
 use crate::peerstate::Peerstate;
 use crate::securejoin::handle_degrade_event;
 use crate::stock::StockMessage;
+use crate::{bail, ensure};

 #[derive(Debug)]
 pub struct MimeParser<'a> {
    pub context: &'a Context,
    pub parts: Vec<Part>,
-    pub header: HashMap<String, String>,
-    pub subject: Option<String>,
-    pub is_send_by_messenger: bool,
+    header: HashMap<String, String>,
    pub decrypting_failed: bool,
-    pub encrypted: bool,
    pub signatures: HashSet<String>,
    pub gossipped_addr: HashSet<String>,
    pub is_forwarded: bool,
    pub is_system_message: SystemMessage,
    pub location_kml: Option<location::Kml>,
    pub message_kml: Option<location::Kml>,
+    pub user_avatar: AvatarAction,
+    pub group_avatar: AvatarAction,
    reports: Vec<Report>,
    mdns_enabled: bool,
    parsed_protected_headers: bool,
 }

+#[derive(Debug, PartialEq)]
+pub enum AvatarAction {
+    None,
+    Delete,
+    Change(String),
+}
+
+impl AvatarAction {
+    pub fn is_change(&self) -> bool {
+        match self {
+            AvatarAction::None => false,
+            AvatarAction::Delete => false,
+            AvatarAction::Change(_) => true,
+        }
+    }
+}
+
 #[derive(Debug, Display, Clone, Copy, PartialEq, Eq, FromPrimitive, ToPrimitive, ToSql, FromSql)]
 #[repr(i32)]
 pub enum SystemMessage {
@@ -73,10 +91,9 @@ impl<'a> MimeParser<'a> {
        let mut parser = MimeParser {
            parts: Vec::new(),
            header: Default::default(),
-            subject: None,
-            is_send_by_messenger: false,
            decrypting_failed: false,
-            encrypted: false,
+
+            // only non-empty if it was a valid autocrypt message
            signatures: Default::default(),
            gossipped_addr: Default::default(),
            is_forwarded: false,
@@ -85,6 +102,8 @@ impl<'a> MimeParser<'a> {
            is_system_message: SystemMessage::Unknown,
            location_kml: None,
            message_kml: None,
+            user_avatar: AvatarAction::None,
+            group_avatar: AvatarAction::None,
            mdns_enabled,
            parsed_protected_headers: false,
        };
@@ -95,7 +114,8 @@ impl<'a> MimeParser<'a> {
            .and_then(|v| mailparse::dateparse(&v).ok())
            .unwrap_or_default();

-        parser.hash_header(&mail.headers);
+        // init known headers with what mailparse provided us
+        parser.merge_headers(&mail.headers);

        // Memory location for a possible decrypted message.
        let mail_raw;
@@ -103,12 +123,15 @@ impl<'a> MimeParser<'a> {
        let mail = match e2ee::try_decrypt(parser.context, &mail, message_time) {
            Ok((raw, signatures)) => {
                // Valid autocrypt message, encrypted
-                parser.encrypted = raw.is_some();
                parser.signatures = signatures;

                if let Some(raw) = raw {
                    mail_raw = raw;
                    let decrypted_mail = mailparse::parse_mail(&mail_raw)?;
+                    if std::env::var(crate::DCC_MIME_DEBUG).is_ok() {
+                        info!(context, "decrypted message mime-body:");
+                        println!("{}", String::from_utf8_lossy(&mail_raw));
+                    }

                    // Handle any gossip headers if the mail was encrypted.  See section
                    // "3.6 Key Gossip" of https://autocrypt.org/autocrypt-spec-1.1.0.pdf
@@ -117,6 +140,10 @@ impl<'a> MimeParser<'a> {
                    parser.gossipped_addr =
                        update_gossip_peerstates(context, message_time, &mail, gossip_headers)?;

+                    // let known protected headers from the decrypted
+                    // part override the unencrypted top-level
+                    parser.merge_headers(&decrypted_mail.headers);
+
                    decrypted_mail
                } else {
                    // Message was not encrypted
@@ -144,61 +171,35 @@ impl<'a> MimeParser<'a> {
    }

    fn parse_headers(&mut self) -> Result<()> {
-        if let Some(field) = self.lookup_field("Subject") {
-            self.subject = Some(field.clone());
-        }
-
-        if self.lookup_field("Chat-Version").is_some() {
-            self.is_send_by_messenger = true
-        }
-
-        if self.lookup_field("Autocrypt-Setup-Message").is_some() {
-            let has_setup_file = self.parts.iter().any(|p| {
-                p.mimetype.is_some() && p.mimetype.as_ref().unwrap().as_ref() == MIME_AC_SETUP_FILE
+        if self.get(HeaderDef::AutocryptSetupMessage).is_some() {
+            self.parts.drain_filter(|part| {
+                part.mimetype.is_some()
+                    && part.mimetype.as_ref().unwrap().as_ref() != MIME_AC_SETUP_FILE
            });
-
-            if has_setup_file {
+            if self.parts.len() == 1 {
                self.is_system_message = SystemMessage::AutocryptSetupMessage;
-
-                // TODO: replace the following code with this
-                // once drain_filter stabilizes.
-                //
-                // See https://doc.rust-lang.org/std/vec/struct.Vec.html#method.drain_filter
-                // and https://github.com/rust-lang/rust/issues/43244
-                //
-                // mimeparser
-                //    .parts
-                //    .drain_filter(|part| part.int_mimetype != 111)
-                //    .for_each(|part| dc_mimepart_unref(part));
-
-                let mut i = 0;
-                while i != self.parts.len() {
-                    let mimetype = &self.parts[i].mimetype;
-                    if mimetype.is_none()
-                        || mimetype.as_ref().unwrap().as_ref() != MIME_AC_SETUP_FILE
-                    {
-                        self.parts.remove(i);
-                    } else {
-                        i += 1;
-                    }
-                }
+            } else {
+                warn!(self.context, "could not determine ASM mime-part");
            }
-        } else if let Some(value) = self.lookup_field("Chat-Content") {
+        } else if let Some(value) = self.get(HeaderDef::ChatContent) {
            if value == "location-streaming-enabled" {
                self.is_system_message = SystemMessage::LocationStreamingEnabled;
            }
        }
-        if self.lookup_field("Chat-Group-Image").is_some() && !self.parts.is_empty() {
-            let textpart = &self.parts[0];
-            if textpart.typ == Viewtype::Text && self.parts.len() >= 2 {
-                let imgpart = &mut self.parts[1];
-                if imgpart.typ == Viewtype::Image {
-                    imgpart.is_meta = true;
-                }
-            }
+
+        if let Some(header_value) = self.get(HeaderDef::ChatGroupAvatar).cloned() {
+            self.group_avatar = self.avatar_action_from_header(header_value);
+        } else if let Some(header_value) = self.get(HeaderDef::ChatGroupImage).cloned() {
+            // parse the old group-image headers for versions <=0.973 resp. <=beta.15 (december 2019)
+            // grep for #DeprecatedAvatar to get the place where a compatibility header is generated.
+            self.group_avatar = self.avatar_action_from_header(header_value);
        }

-        if self.is_send_by_messenger && self.parts.len() == 2 {
+        if let Some(header_value) = self.get(HeaderDef::ChatUserAvatar).cloned() {
+            self.user_avatar = self.avatar_action_from_header(header_value);
+        }
+
+        if self.has_chat_version() && self.parts.len() == 2 {
            let need_drop = {
                let textpart = &self.parts[0];
                let filepart = &self.parts[1];
@@ -226,13 +227,13 @@ impl<'a> MimeParser<'a> {
                std::mem::replace(&mut self.parts[0], filepart);
            }
        }
-        if let Some(ref subject) = self.subject {
+        if let Some(ref subject) = self.get_subject() {
            let mut prepend_subject = 1i32;
            if !self.decrypting_failed {
                let colon = subject.find(':');
                if colon == Some(2)
                    || colon == Some(3)
-                    || self.is_send_by_messenger
+                    || self.has_chat_version()
                    || subject.contains("Chat:")
                {
                    prepend_subject = 0i32
@@ -263,13 +264,13 @@ impl<'a> MimeParser<'a> {
        }
        if self.parts.len() == 1 {
            if self.parts[0].typ == Viewtype::Audio
-                && self.lookup_field("Chat-Voice-Message").is_some()
+                && self.get(HeaderDef::ChatVoiceMessage).is_some()
            {
                let part_mut = &mut self.parts[0];
                part_mut.typ = Viewtype::Voice;
            }
            if self.parts[0].typ == Viewtype::Image {
-                if let Some(value) = self.lookup_field("Chat-Content") {
+                if let Some(value) = self.get(HeaderDef::ChatContent) {
                    if value == "sticker" {
                        let part_mut = &mut self.parts[0];
                        part_mut.typ = Viewtype::Sticker;
@@ -281,7 +282,7 @@ impl<'a> MimeParser<'a> {
                || part.typ == Viewtype::Voice
                || part.typ == Viewtype::Video
            {
-                if let Some(field_0) = self.lookup_field("Chat-Duration") {
+                if let Some(field_0) = self.get(HeaderDef::ChatDuration) {
                    let duration_ms = field_0.parse().unwrap_or_default();
                    if duration_ms > 0 && duration_ms < 24 * 60 * 60 * 1000 {
                        let part_mut = &mut self.parts[0];
@@ -291,12 +292,12 @@ impl<'a> MimeParser<'a> {
            }
        }
        if !self.decrypting_failed {
-            if let Some(dn_field) = self.lookup_field("Chat-Disposition-Notification-To") {
+            if let Some(dn_field) = self.get(HeaderDef::ChatDispositionNotificationTo) {
                if self.get_last_nonmeta().is_some() {
                    let addrs = mailparse::addrparse(&dn_field).unwrap();

                    if let Some(dn_to_addr) = addrs.first() {
-                        if let Some(from_field) = self.lookup_field("From") {
+                        if let Some(from_field) = self.get(HeaderDef::From_) {
                            let from_addrs = mailparse::addrparse(&from_field).unwrap();

                            if let Some(from_addr) = from_addrs.first() {
@@ -317,8 +318,8 @@ impl<'a> MimeParser<'a> {
            let mut part = Part::default();
            part.typ = Viewtype::Text;

-            if let Some(ref subject) = self.subject {
-                if !self.is_send_by_messenger {
+            if let Some(ref subject) = self.get_subject() {
+                if !self.has_chat_version() {
                    part.msg = subject.to_string();
                }
            }
@@ -329,6 +330,29 @@ impl<'a> MimeParser<'a> {
        Ok(())
    }

+    fn avatar_action_from_header(&mut self, header_value: String) -> AvatarAction {
+        if header_value == "0" {
+            return AvatarAction::Delete;
+        } else {
+            let mut i = 0;
+            while i != self.parts.len() {
+                let part = &mut self.parts[i];
+                if let Some(part_filename) = &part.org_filename {
+                    if part_filename == &header_value {
+                        if let Some(blob) = part.param.get(Param::File) {
+                            let res = AvatarAction::Change(blob.to_string());
+                            self.parts.remove(i);
+                            return res;
+                        }
+                        break;
+                    }
+                }
+                i += 1;
+            }
+        }
+        AvatarAction::None
+    }
+
    pub fn get_last_nonmeta(&self) -> Option<&Part> {
        self.parts.iter().rev().find(|part| !part.is_meta)
    }
@@ -337,8 +361,31 @@ impl<'a> MimeParser<'a> {
        self.parts.iter_mut().rev().find(|part| !part.is_meta)
    }

-    pub fn lookup_field(&self, field_name: &str) -> Option<&String> {
-        self.header.get(&field_name.to_lowercase())
+    pub fn was_encrypted(&self) -> bool {
+        !self.signatures.is_empty()
+    }
+
+    pub(crate) fn has_chat_version(&self) -> bool {
+        self.header.contains_key("chat-version")
+    }
+
+    pub(crate) fn has_headers(&self) -> bool {
+        !self.header.is_empty()
+    }
+
+    pub(crate) fn get_subject(&self) -> Option<String> {
+        if let Some(s) = self.get(HeaderDef::Subject) {
+            if s.is_empty() {
+                return None;
+            }
+            Some(s.to_string())
+        } else {
+            None
+        }
+    }
+
+    pub fn get(&self, headerdef: HeaderDef) -> Option<&String> {
+        self.header.get(&headerdef.get_headername())
    }

    fn parse_mime_recursive(&mut self, mail: &mailparse::ParsedMail<'_>) -> Result<bool> {
@@ -351,12 +398,7 @@ impl<'a> MimeParser<'a> {
                return Ok(false);
            }

-            if self.parsed_protected_headers {
-                warn!(self.context, "Ignoring nested protected headers");
-            } else {
-                self.hash_header(&mail.headers);
-                self.parsed_protected_headers = true;
-            }
+            warn!(self.context, "Ignoring nested protected headers");
        }

        enum MimeS {
@@ -442,6 +484,9 @@ impl<'a> MimeParser<'a> {
                }
            }
            (mime::MULTIPART, "encrypted") => {
+                // we currently do not try to decrypt non-autocrypt messages
+                // at all. If we see an encrypted part, we set
+                // decrypting_failed.
                let msg_body = self.context.stock_str(StockMessage::CantDecryptMsgBody);
                let txt = format!("[{}]", msg_body);

@@ -487,41 +532,10 @@ impl<'a> MimeParser<'a> {
                }
            }
            _ => {
-                // Add all parts (in fact,
-                // AddSinglePartIfKnown() later check if the parts are really supported)
-                // HACK: the following lines are a hack for clients who use
-                // multipart/mixed instead of multipart/alternative for
-                // combined text/html messages (eg. Stock Android "Mail" does so).
-                // So, if we detect such a message below, we skip the Html
-                // part.  However, not sure, if there are useful situations to use
-                // plain+html in multipart/mixed - if so, we should disable the hack.
-                let mut skip_part = -1;
-                let mut html_part = -1;
-                let mut plain_cnt = 0;
-                let mut html_cnt = 0;
-
-                for (i, cur_data) in mail.subparts.iter().enumerate() {
-                    match get_mime_type(cur_data)?.0.type_() {
-                        mime::TEXT => {
-                            plain_cnt += 1;
-                        }
-                        mime::HTML => {
-                            html_part = i as isize;
-                            html_cnt += 1;
-                        }
-                        _ => {}
-                    }
-                }
-                if plain_cnt == 1 && html_cnt == 1 {
-                    warn!(
-                        self.context,
-                        "HACK: multipart/mixed message found with Plain and HTML, we\'ll skip the HTML part as this seems to be unwanted."
-                    );
-                    skip_part = html_part;
-                }
-
-                for (i, cur_data) in mail.subparts.iter().enumerate() {
-                    if i as isize != skip_part && self.parse_mime_recursive(cur_data)? {
+                // Add all parts (in fact, AddSinglePartIfKnown() later check if
+                // the parts are really supported)
+                for cur_data in mail.subparts.iter() {
+                    if self.parse_mime_recursive(cur_data)? {
                        any_part_added = true;
                    }
                }
@@ -536,84 +550,56 @@ impl<'a> MimeParser<'a> {
        let (mime_type, msg_type) = get_mime_type(mail)?;
        let raw_mime = mail.ctype.mimetype.to_lowercase();

+        let filename = get_attachment_filename(mail);
+
        let old_part_count = self.parts.len();

-        // regard `Content-Transfer-Encoding:`
-        match mime_type.type_() {
-            mime::TEXT | mime::HTML => {
-                let decoded_data = match mail.get_body() {
-                    Ok(decoded_data) => decoded_data,
-                    Err(err) => {
-                        warn!(self.context, "Invalid body parsed {:?}", err);
-                        // Note that it's not always an error - might be no data
-                        return Ok(false);
-                    }
-                };
-
-                // check header directly as is_send_by_messenger is not yet set up
-                let is_msgrmsg = self.lookup_field("Chat-Version").is_some();
-
-                let mut simplifier = Simplify::new();
-                let simplified_txt = if decoded_data.is_empty() {
-                    "".into()
-                } else {
-                    let is_html = mime_type == mime::TEXT_HTML;
-                    simplifier.simplify(&decoded_data, is_html, is_msgrmsg)
-                };
-
-                if !simplified_txt.is_empty() {
-                    let mut part = Part::default();
-                    part.typ = Viewtype::Text;
-                    part.mimetype = Some(mime_type);
-                    part.msg = simplified_txt;
-                    part.msg_raw = Some(decoded_data);
-                    self.do_add_single_part(part);
+        if let Ok(filename) = filename {
+            self.do_add_single_file_part(
+                msg_type,
+                mime_type,
+                &raw_mime,
+                &mail.get_body_raw()?,
+                &filename,
+            );
+        } else {
+            match mime_type.type_() {
+                mime::IMAGE | mime::AUDIO | mime::VIDEO | mime::APPLICATION => {
+                    bail!("missing attachment");
                }
+                mime::TEXT | mime::HTML => {
+                    let decoded_data = match mail.get_body() {
+                        Ok(decoded_data) => decoded_data,
+                        Err(err) => {
+                            warn!(self.context, "Invalid body parsed {:?}", err);
+                            // Note that it's not always an error - might be no data
+                            return Ok(false);
+                        }
+                    };

-                if simplifier.is_forwarded {
-                    self.is_forwarded = true;
-                }
-            }
-            mime::IMAGE | mime::AUDIO | mime::VIDEO | mime::APPLICATION => {
-                // try to get file name from
-                //    `Content-Disposition: ... filename*=...`
-                // or `Content-Disposition: ... filename*0*=... filename*1*=... filename*2*=...`
-                // or `Content-Disposition: ... filename=...`
-
-                let ct = mail.get_content_disposition()?;
-                let mut desired_filename = ct
-                    .params
-                    .iter()
-                    .filter(|(key, _value)| key.starts_with("filename"))
-                    .fold(String::new(), |mut acc, (_key, value)| {
-                        acc += value;
-                        acc
-                    });
-
-                if desired_filename.is_empty() {
-                    if let Some(param) = ct.params.get("name") {
-                        // might be a wrongly encoded filename
-                        desired_filename = param.to_string();
-                    }
-                }
-
-                // if there is still no filename, guess one
-                if desired_filename.is_empty() {
-                    if let Some(subtype) = mail.ctype.mimetype.split('/').nth(1) {
-                        desired_filename = format!("file.{}", subtype,);
+                    let mut simplifier = Simplify::new();
+                    let simplified_txt = if decoded_data.is_empty() {
+                        "".into()
                    } else {
-                        return Ok(false);
+                        let is_html = mime_type == mime::TEXT_HTML;
+                        simplifier.simplify(&decoded_data, is_html, self.has_chat_version())
+                    };
+
+                    if !simplified_txt.is_empty() {
+                        let mut part = Part::default();
+                        part.typ = Viewtype::Text;
+                        part.mimetype = Some(mime_type);
+                        part.msg = simplified_txt;
+                        part.msg_raw = Some(decoded_data);
+                        self.do_add_single_part(part);
+                    }
+
+                    if simplifier.is_forwarded {
+                        self.is_forwarded = true;
                    }
                }
-                self.do_add_single_file_part(
-                    msg_type,
-                    mime_type,
-                    &raw_mime,
-                    &mail.get_body_raw()?,
-                    &desired_filename,
-                );
+                _ => {}
            }
-            _ => {}
        }

        // add object? (we do not add all objects, eg. signatures etc. are ignored)
@@ -662,6 +648,7 @@ impl<'a> MimeParser<'a> {
                return;
            }
        };
+        info!(self.context, "added blobfile: {:?}", blob.as_name());

        /* create and register Mime part referencing the new Blob object */
        let mut part = Part::default();
@@ -673,6 +660,7 @@ impl<'a> MimeParser<'a> {
        }

        part.typ = msg_type;
+        part.org_filename = Some(filename.to_string());
        part.mimetype = Some(mime_type);
        part.bytes = decoded_data.len();
        part.param.set(Param::File, blob.as_name());
@@ -682,50 +670,24 @@ impl<'a> MimeParser<'a> {
    }

    fn do_add_single_part(&mut self, mut part: Part) {
-        if self.encrypted {
-            if !self.signatures.is_empty() {
-                part.param.set_int(Param::GuaranteeE2ee, 1);
-            } else {
-                // XXX if the message was encrypted but not signed
-                // it's not neccessarily an error we need to signal.
-                // we could just treat it as if it was not encrypted.
-                part.param.set_int(Param::ErroneousE2ee, 0x2);
-            }
+        if self.was_encrypted() {
+            part.param.set_int(Param::GuaranteeE2ee, 1);
        }
        self.parts.push(part);
    }

    pub fn is_mailinglist_message(&self) -> bool {
-        if self.lookup_field("List-Id").is_some() {
+        if self.get(HeaderDef::ListId).is_some() {
            return true;
        }

-        if let Some(precedence) = self.lookup_field("Precedence") {
+        if let Some(precedence) = self.get(HeaderDef::Precedence) {
            precedence == "list" || precedence == "bulk"
        } else {
            false
        }
    }

-    pub fn sender_equals_recipient(&self) -> bool {
-        /* get From: and check there is exactly one sender */
-        if let Some(field) = self.lookup_field("From") {
-            if let Ok(addrs) = mailparse::addrparse(field) {
-                if addrs.len() != 1 {
-                    return false;
-                }
-                if let mailparse::MailAddr::Single(ref info) = addrs[0] {
-                    let from_addr_norm = addr_normalize(&info.addr);
-                    let recipients = get_recipients(self.header.iter());
-                    if recipients.len() == 1 && recipients.contains(from_addr_norm) {
-                        return true;
-                    }
-                }
-            }
-        }
-        false
-    }
-
    pub fn repl_msg_by_error(&mut self, error_msg: impl AsRef<str>) {
        if self.parts.is_empty() {
            return;
@@ -740,14 +702,14 @@ impl<'a> MimeParser<'a> {
    }

    pub fn get_rfc724_mid(&self) -> Option<String> {
-        // get Message-ID from header
-        if let Some(field) = self.lookup_field("Message-ID") {
-            return parse_message_id(field);
+        if let Some(msgid) = self.get(HeaderDef::MessageId) {
+            parse_message_id(msgid)
+        } else {
+            None
        }
-        None
    }

-    fn hash_header(&mut self, fields: &[mailparse::MailHeader<'_>]) {
+    fn merge_headers(&mut self, fields: &[mailparse::MailHeader<'_>]) {
        for field in fields {
            if let Ok(key) = field.get_key() {
                // lowercasing all headers is technically not correct, but makes things work better
@@ -774,9 +736,10 @@ impl<'a> MimeParser<'a> {
        let (report_fields, _) = mailparse::parse_headers(&report_body)?;

        // must be present
-        if let Some(_disposition) = report_fields.get_first_value("Disposition").ok().flatten() {
+        let disp = HeaderDef::Disposition.get_headername();
+        if let Some(_disposition) = report_fields.get_first_value(&disp).ok().flatten() {
            if let Some(original_message_id) = report_fields
-                .get_first_value("Original-Message-ID")
+                .get_first_value(&HeaderDef::OriginalMessageId.get_headername())
                .ok()
                .flatten()
                .and_then(|v| parse_message_id(&v))
@@ -812,11 +775,11 @@ impl<'a> MimeParser<'a> {
                mdn_consumed = true;
            }

-            if self.is_send_by_messenger || mdn_consumed {
+            if self.has_chat_version() || mdn_consumed {
                let mut param = Params::new();
                param.set(Param::ServerFolder, server_folder.as_ref());
                param.set_int(Param::ServerUid, server_uid as i32);
-                if self.is_send_by_messenger && self.context.get_config_bool(Config::MvboxMove) {
+                if self.has_chat_version() && self.context.get_config_bool(Config::MvboxMove) {
                    param.set_int(Param::AlsoMove, 1);
                }
                job_add(self.context, Action::MarkseenMdnOnImap, 0, param, 0);
@@ -911,14 +874,16 @@ pub struct Part {
    pub msg_raw: Option<String>,
    pub bytes: usize,
    pub param: Params,
+    org_filename: Option<String>,
 }

+/// return mimetype and viewtype for a parsed mail
 fn get_mime_type(mail: &mailparse::ParsedMail<'_>) -> Result<(Mime, Viewtype)> {
    let mimetype = mail.ctype.mimetype.parse::<Mime>()?;

    let viewtype = match mimetype.type_() {
        mime::TEXT => {
-            if !mailmime_is_attachment_disposition(mail) {
+            if !is_attachment_disposition(mail) {
                match mimetype.subtype() {
                    mime::PLAIN | mime::HTML => Viewtype::Text,
                    _ => Viewtype::File,
@@ -952,14 +917,59 @@ fn get_mime_type(mail: &mailparse::ParsedMail<'_>) -> Result<(Mime, Viewtype)> {
    Ok((mimetype, viewtype))
 }

-fn mailmime_is_attachment_disposition(mail: &mailparse::ParsedMail<'_>) -> bool {
-    if let Some(ct) = mail.ctype.params.get("Content-Disposition") {
-        return ct.to_lowercase().starts_with("attachment");
+fn is_attachment_disposition(mail: &mailparse::ParsedMail<'_>) -> bool {
+    if let Ok(ct) = mail.get_content_disposition() {
+        return ct.disposition == DispositionType::Attachment
+            && ct
+                .params
+                .iter()
+                .any(|(key, _value)| key.starts_with("filename"));
    }

    false
 }

+fn get_attachment_filename(mail: &mailparse::ParsedMail) -> Result<String> {
+    // try to get file name from
+    //    `Content-Disposition: ... filename*=...`
+    // or `Content-Disposition: ... filename*0*=... filename*1*=... filename*2*=...`
+    // or `Content-Disposition: ... filename=...`
+
+    let ct = mail.get_content_disposition()?;
+    ensure!(
+        ct.disposition == DispositionType::Attachment,
+        "disposition not an attachment: {:?}",
+        ct.disposition
+    );
+
+    let mut desired_filename = ct
+        .params
+        .iter()
+        .filter(|(key, _value)| key.starts_with("filename"))
+        .fold(String::new(), |mut acc, (_key, value)| {
+            acc += value;
+            acc
+        });
+
+    if desired_filename.is_empty() {
+        if let Some(param) = ct.params.get("name") {
+            // might be a wrongly encoded filename
+            desired_filename = param.to_string();
+        }
+    }
+
+    // if there is still no filename, guess one
+    if desired_filename.is_empty() {
+        if let Some(subtype) = mail.ctype.mimetype.split('/').nth(1) {
+            desired_filename = format!("file.{}", subtype,);
+        } else {
+            bail!("could not determine filename: {:?}", ct.disposition);
+        }
+    }
+
+    Ok(desired_filename)
+}
+
 // returned addresses are normalized.
 fn get_recipients<S: AsRef<str>, T: Iterator<Item = (S, S)>>(headers: T) -> HashSet<String> {
    let mut recipients: HashSet<String> = Default::default();
@@ -1018,7 +1028,7 @@ mod tests {
        let raw = include_bytes!("../test-data/message/issue_523.txt");
        let mimeparser = MimeParser::from_bytes(&context.ctx, &raw[..]).unwrap();

-        assert_eq!(mimeparser.subject, None);
+        assert_eq!(mimeparser.get_subject(), None);
        assert_eq!(mimeparser.parts.len(), 1);
    }

@@ -1064,6 +1074,29 @@ mod tests {
        assert_eq!(recipients.len(), 2);
    }

+    #[test]
+    fn test_is_attachment() {
+        let raw = include_bytes!("../test-data/message/mail_with_cc.txt");
+        let mail = mailparse::parse_mail(raw).unwrap();
+        assert!(!is_attachment_disposition(&mail));
+
+        let raw = include_bytes!("../test-data/message/mail_attach_txt.eml");
+        let mail = mailparse::parse_mail(raw).unwrap();
+        assert!(!is_attachment_disposition(&mail));
+        assert!(!is_attachment_disposition(&mail.subparts[0]));
+        assert!(is_attachment_disposition(&mail.subparts[1]));
+    }
+
+    #[test]
+    fn test_get_attachment_filename() {
+        let raw = include_bytes!("../test-data/message/html_attach.eml");
+        let mail = mailparse::parse_mail(raw).unwrap();
+        assert!(get_attachment_filename(&mail).is_err());
+        assert!(get_attachment_filename(&mail.subparts[0]).is_err());
+        let filename = get_attachment_filename(&mail.subparts[1]).unwrap();
+        assert_eq!(filename, "test.html")
+    }
+
    #[test]
    fn test_mailparse_content_type() {
        let ctype =
@@ -1083,14 +1116,14 @@ mod tests {
        let raw = b"From: hello\n\
                    Content-Type: multipart/mixed; boundary=\"==break==\";\n\
                    Subject: outer-subject\n\
-                    X-Special-A: special-a\n\
-                    Foo: Bar\nChat-Version: 0.0\n\
+                    Secure-Join-Group: no\n\
+                    Test-Header: Bar\nChat-Version: 0.0\n\
                    \n\
                    --==break==\n\
                    Content-Type: text/plain; protected-headers=\"v1\";\n\
                    Subject: inner-subject\n\
-                    X-Special-B: special-b\n\
-                    Foo: Xy\n\
+                    SecureBar-Join-Group: yes\n\
+                    Test-Header: Xy\n\
                    Chat-Version: 1.0\n\
                    \n\
                    test1\n\
@@ -1098,18 +1131,65 @@ mod tests {
                    --==break==--\n\
                    \n\
                    \x00";
+
        let mimeparser = MimeParser::from_bytes(&context.ctx, &raw[..]).unwrap();

-        assert_eq!(mimeparser.subject, Some("inner-subject".into()));
+        // non-overwritten headers do not bubble up
+        let of = mimeparser.get(HeaderDef::SecureJoinGroup).unwrap();
+        assert_eq!(of, "no");

-        let of = mimeparser.lookup_field("X-Special-A").unwrap();
-        assert_eq!(of, "special-a");
-
-        let of = mimeparser.lookup_field("Foo").unwrap();
+        // unknown headers do not bubble upwards
+        let of = mimeparser.get(HeaderDef::_TestHeader).unwrap();
        assert_eq!(of, "Bar");

-        let of = mimeparser.lookup_field("Chat-Version").unwrap();
-        assert_eq!(of, "1.0");
+        // the following fields would bubble up
+        // if the test would really use encryption for the protected part
+        // however, as this is not the case, the outer things stay valid
+        assert_eq!(mimeparser.get_subject(), Some("outer-subject".into()));
+
+        let of = mimeparser.get(HeaderDef::ChatVersion).unwrap();
+        assert_eq!(of, "0.0");
        assert_eq!(mimeparser.parts.len(), 1);
    }
+
+    #[test]
+    fn test_mimeparser_with_avatars() {
+        let t = dummy_context();
+
+        let raw = include_bytes!("../test-data/message/mail_attach_txt.eml");
+        let mimeparser = MimeParser::from_bytes(&t.ctx, &raw[..]).unwrap();
+        assert_eq!(mimeparser.user_avatar, AvatarAction::None);
+        assert_eq!(mimeparser.group_avatar, AvatarAction::None);
+
+        let raw = include_bytes!("../test-data/message/mail_with_user_avatar.eml");
+        let mimeparser = MimeParser::from_bytes(&t.ctx, &raw[..]).unwrap();
+        assert_eq!(mimeparser.parts.len(), 1);
+        assert_eq!(mimeparser.parts[0].typ, Viewtype::Text);
+        assert!(mimeparser.user_avatar.is_change());
+        assert_eq!(mimeparser.group_avatar, AvatarAction::None);
+
+        let raw = include_bytes!("../test-data/message/mail_with_user_avatar_deleted.eml");
+        let mimeparser = MimeParser::from_bytes(&t.ctx, &raw[..]).unwrap();
+        assert_eq!(mimeparser.parts.len(), 1);
+        assert_eq!(mimeparser.parts[0].typ, Viewtype::Text);
+        assert_eq!(mimeparser.user_avatar, AvatarAction::Delete);
+        assert_eq!(mimeparser.group_avatar, AvatarAction::None);
+
+        let raw = include_bytes!("../test-data/message/mail_with_user_and_group_avatars.eml");
+        let mimeparser = MimeParser::from_bytes(&t.ctx, &raw[..]).unwrap();
+        assert_eq!(mimeparser.parts.len(), 1);
+        assert_eq!(mimeparser.parts[0].typ, Viewtype::Text);
+        assert!(mimeparser.user_avatar.is_change());
+        assert!(mimeparser.group_avatar.is_change());
+
+        // if the Chat-User-Avatar header is missing, the avatar become a normal attachment
+        let raw = include_bytes!("../test-data/message/mail_with_user_and_group_avatars.eml");
+        let raw = String::from_utf8_lossy(raw).to_string();
+        let raw = raw.replace("Chat-User-Avatar:", "Xhat-Xser-Xvatar:");
+        let mimeparser = MimeParser::from_bytes(&t.ctx, raw.as_bytes()).unwrap();
+        assert_eq!(mimeparser.parts.len(), 1);
+        assert_eq!(mimeparser.parts[0].typ, Viewtype::Image);
+        assert_eq!(mimeparser.user_avatar, AvatarAction::None);
+        assert!(mimeparser.group_avatar.is_change());
+    }
 }
--- a/src/param.rs
+++ b/src/param.rs
@@ -48,6 +48,10 @@ pub enum Param {
    Arg4 = b'H',
    /// For Messages
    Error = b'L',
+
+    /// For Messages
+    AttachGroupImage = b'A',
+
    /// For Messages: space-separated list of messaged IDs of forwarded copies.
    ///
    /// This is used when a [crate::message::Message] is in the
@@ -192,6 +196,11 @@ impl Params {
        self.get(key).and_then(|s| s.parse().ok())
    }

+    /// Get the given parameter and parse as `bool`.
+    pub fn get_bool(&self, key: Param) -> Option<bool> {
+        self.get_int(key).map(|v| v != 0)
+    }
+
    /// Get the parameter behind `Param::Cmd` interpreted as `SystemMessage`.
    pub fn get_cmd(&self) -> SystemMessage {
        self.get_int(Param::Cmd)
@@ -250,7 +259,7 @@ impl Params {
        let file = ParamsFile::from_param(context, val)?;
        let blob = match file {
            ParamsFile::FsPath(path) => match create {
-                true => BlobObject::create_from_path(context, path)?,
+                true => BlobObject::new_from_path(context, path)?,
                false => BlobObject::from_path(context, path)?,
            },
            ParamsFile::Blob(blob) => blob,
--- a/src/peerstate.rs
+++ b/src/peerstate.rs
@@ -417,7 +417,7 @@ impl<'a> Peerstate<'a> {
                    &self.addr,
                ],
            )?;
-            reset_gossiped_timestamp(self.context, 0);
+            reset_gossiped_timestamp(self.context, 0)?;
        } else if self.to_save == Some(ToSave::Timestamps) {
            sql::execute(
                self.context,
--- a/src/securejoin.rs
+++ b/src/securejoin.rs
@@ -11,6 +11,7 @@ use crate::context::Context;
 use crate::e2ee::*;
 use crate::error::Error;
 use crate::events::Event;
+use crate::headerdef::HeaderDef;
 use crate::key::*;
 use crate::lot::LotState;
 use crate::message::Message;
@@ -350,7 +351,7 @@ pub(crate) fn handle_securejoin_handshake(
        "handle_securejoin_handshake(): called with special contact id"
    );
    let step = mimeparser
-        .lookup_field("Secure-Join")
+        .get(HeaderDef::SecureJoin)
        .ok_or_else(|| format_err!("This message is not a Secure-Join message"))?;

    info!(
@@ -378,7 +379,7 @@ pub(crate) fn handle_securejoin_handshake(
            // it just ensures, we have Bobs key now. If we do _not_ have the key because eg. MitM has removed it,
            // send_message() will fail with the error "End-to-end-encryption unavailable unexpectedly.", so, there is no additional check needed here.
            // verify that the `Secure-Join-Invitenumber:`-header matches invitenumber written to the QR code
-            let invitenumber = match mimeparser.lookup_field("Secure-Join-Invitenumber") {
+            let invitenumber = match mimeparser.get(HeaderDef::SecureJoinInvitenumber) {
                Some(n) => n,
                None => {
                    warn!(context, "Secure-join denied (invitenumber missing).",);
@@ -422,7 +423,7 @@ pub(crate) fn handle_securejoin_handshake(
                could_not_establish_secure_connection(
                    context,
                    contact_chat_id,
-                    if mimeparser.encrypted {
+                    if mimeparser.was_encrypted() {
                        "No valid signature."
                    } else {
                        "Not encrypted."
@@ -467,7 +468,7 @@ pub(crate) fn handle_securejoin_handshake(
            ====  Step 6 in "Out-of-band verified groups" protocol  ====
            ============================================================ */
            // verify that Secure-Join-Fingerprint:-header matches the fingerprint of Bob
-            let fingerprint = match mimeparser.lookup_field("Secure-Join-Fingerprint") {
+            let fingerprint = match mimeparser.get(HeaderDef::SecureJoinFingerprint) {
                Some(fp) => fp,
                None => {
                    could_not_establish_secure_connection(
@@ -496,7 +497,7 @@ pub(crate) fn handle_securejoin_handshake(
            }
            info!(context, "Fingerprint verified.",);
            // verify that the `Secure-Join-Auth:`-header matches the secret written to the QR code
-            let auth_0 = match mimeparser.lookup_field("Secure-Join-Auth") {
+            let auth_0 = match mimeparser.get(HeaderDef::SecureJoinAuth) {
                Some(auth) => auth,
                None => {
                    could_not_establish_secure_connection(
@@ -526,7 +527,7 @@ pub(crate) fn handle_securejoin_handshake(
            inviter_progress!(context, contact_id, 600);
            if join_vg {
                let field_grpid = mimeparser
-                    .lookup_field("Secure-Join-Group")
+                    .get(HeaderDef::SecureJoinGroup)
                    .map(|s| s.as_str())
                    .unwrap_or_else(|| "");
                let (group_chat_id, _, _) = chat::get_chat_id_by_grpid(context, field_grpid);
@@ -600,10 +601,10 @@ pub(crate) fn handle_securejoin_handshake(
            Contact::scaleup_origin_by_id(context, contact_id, Origin::SecurejoinJoined);
            emit_event!(context, Event::ContactsChanged(None));
            let cg_member_added = mimeparser
-                .lookup_field("Chat-Group-Member-Added")
+                .get(HeaderDef::ChatGroupMemberAdded)
                .map(|s| s.as_str())
                .unwrap_or_else(|| "");
-            if join_vg && !addr_equals_self(context, cg_member_added) {
+            if join_vg && !context.is_self_addr(cg_member_added)? {
                info!(context, "Message belongs to a different handshake (scaled up contact anyway to allow creation of group).");
                return Ok(ret);
            }
@@ -635,7 +636,7 @@ pub(crate) fn handle_securejoin_handshake(
                inviter_progress!(context, contact_id, 800);
                inviter_progress!(context, contact_id, 1000);
                let field_grpid = mimeparser
-                    .lookup_field("Secure-Join-Group")
+                    .get(HeaderDef::SecureJoinGroup)
                    .map(|s| s.as_str())
                    .unwrap_or_else(|| "");
                let (group_chat_id, _, _) = chat::get_chat_id_by_grpid(context, &field_grpid);
@@ -717,7 +718,7 @@ fn mark_peer_as_verified(context: &Context, fingerprint: impl AsRef<str>) -> Res
 ******************************************************************************/

 fn encrypted_and_signed(mimeparser: &MimeParser, expected_fingerprint: impl AsRef<str>) -> bool {
-    if !mimeparser.encrypted {
+    if !mimeparser.was_encrypted() {
        warn!(mimeparser.context, "Message not encrypted.",);
        false
    } else if mimeparser.signatures.is_empty() {
--- a/src/smtp/mod.rs
+++ b/src/smtp/mod.rs
@@ -2,15 +2,15 @@

 pub mod send;

-use lettre::smtp::client::net::*;
-use lettre::*;
+use async_smtp::smtp::client::net::*;
+use async_smtp::*;

-use failure::Fail;
+use async_std::task;

 use crate::constants::*;
 use crate::context::Context;
 use crate::events::Event;
-use crate::login_param::{dc_build_tls_config, LoginParam};
+use crate::login_param::{dc_build_tls, LoginParam};
 use crate::oauth2::*;

 #[derive(Debug, Fail)]
@@ -21,14 +21,22 @@ pub enum Error {
    InvalidLoginAddress {
        address: String,
        #[cause]
-        error: lettre::error::Error,
+        error: async_smtp::error::Error,
    },
    #[fail(display = "SMTP failed to connect: {:?}", _0)]
-    ConnectionFailure(#[cause] lettre::smtp::error::Error),
+    ConnectionFailure(#[cause] async_smtp::smtp::error::Error),
    #[fail(display = "SMTP: failed to setup connection {:?}", _0)]
-    ConnectionSetupFailure(#[cause] lettre::smtp::error::Error),
+    ConnectionSetupFailure(#[cause] async_smtp::smtp::error::Error),
    #[fail(display = "SMTP: oauth2 error {:?}", _0)]
    Oauth2Error { address: String },
+    #[fail(display = "TLS error")]
+    Tls(#[cause] native_tls::Error),
+}
+
+impl From<native_tls::Error> for Error {
+    fn from(err: native_tls::Error) -> Error {
+        Error::Tls(err)
+    }
 }

 pub type Result<T> = std::result::Result<T, Error>;
@@ -36,8 +44,7 @@ pub type Result<T> = std::result::Result<T, Error>;
 #[derive(Default, DebugStub)]
 pub struct Smtp {
    #[debug_stub(some = "SmtpTransport")]
-    transport: Option<lettre::smtp::SmtpTransport>,
-    transport_connected: bool,
+    transport: Option<async_smtp::smtp::SmtpTransport>,
    /// Email address we are sending from.
    from: Option<EmailAddress>,
 }
@@ -50,16 +57,12 @@ impl Smtp {

    /// Disconnect the SMTP transport and drop it entirely.
    pub fn disconnect(&mut self) {
-        if self.transport.is_none() || !self.transport_connected {
-            return;
+        if let Some(ref mut transport) = self.transport.take() {
+            transport.close();
        }
-
-        let mut transport = self.transport.take().unwrap();
-        transport.close();
-        self.transport_connected = false;
    }

-    /// Check if a connection already exists.
+    /// check whether we are connected
    pub fn is_connected(&self) -> bool {
        self.transport.is_some()
    }
@@ -86,7 +89,7 @@ impl Smtp {
        let domain = &lp.send_server;
        let port = lp.send_port as u16;

-        let tls_config = dc_build_tls_config(lp.smtp_certificate_checks);
+        let tls_config = dc_build_tls(lp.smtp_certificate_checks)?.into();
        let tls_parameters = ClientTlsParameters::new(domain.to_string(), tls_config);

        let (creds, mechanism) = if 0 != lp.server_flags & (DC_LP_AUTH_OAUTH2 as i32) {
@@ -101,21 +104,21 @@ impl Smtp {
            }
            let user = &lp.send_user;
            (
-                lettre::smtp::authentication::Credentials::new(
+                async_smtp::smtp::authentication::Credentials::new(
                    user.to_string(),
                    access_token.unwrap_or_default(),
                ),
-                vec![lettre::smtp::authentication::Mechanism::Xoauth2],
+                vec![async_smtp::smtp::authentication::Mechanism::Xoauth2],
            )
        } else {
            // plain
            let user = lp.send_user.clone();
            let pw = lp.send_pw.clone();
            (
-                lettre::smtp::authentication::Credentials::new(user, pw),
+                async_smtp::smtp::authentication::Credentials::new(user, pw),
                vec![
-                    lettre::smtp::authentication::Mechanism::Plain,
-                    lettre::smtp::authentication::Mechanism::Login,
+                    async_smtp::smtp::authentication::Mechanism::Plain,
+                    async_smtp::smtp::authentication::Mechanism::Login,
                ],
            )
        };
@@ -123,24 +126,26 @@ impl Smtp {
        let security = if 0
            != lp.server_flags & (DC_LP_SMTP_SOCKET_STARTTLS | DC_LP_SMTP_SOCKET_PLAIN) as i32
        {
-            lettre::smtp::ClientSecurity::Opportunistic(tls_parameters)
+            async_smtp::smtp::ClientSecurity::Opportunistic(tls_parameters)
        } else {
-            lettre::smtp::ClientSecurity::Wrapper(tls_parameters)
+            async_smtp::smtp::ClientSecurity::Wrapper(tls_parameters)
        };

-        let client = lettre::smtp::SmtpClient::new((domain.as_str(), port), security)
-            .map_err(Error::ConnectionSetupFailure)?;
+        let client = task::block_on(async_smtp::smtp::SmtpClient::with_security(
+            (domain.as_str(), port),
+            security,
+        ))
+        .map_err(Error::ConnectionSetupFailure)?;

        let client = client
            .smtp_utf8(true)
            .credentials(creds)
            .authentication_mechanism(mechanism)
-            .connection_reuse(lettre::smtp::ConnectionReuseParameters::ReuseUnlimited);
-        let mut trans = client.transport();
-        trans.connect().map_err(Error::ConnectionFailure)?;
+            .connection_reuse(async_smtp::smtp::ConnectionReuseParameters::ReuseUnlimited);
+        let mut trans = client.into_transport();
+        task::block_on(trans.connect()).map_err(Error::ConnectionFailure)?;

        self.transport = Some(trans);
-        self.transport_connected = true;
        context.call_cb(Event::SmtpConnected(format!(
            "SMTP-LOGIN as {} ok",
            lp.send_user,
--- a/src/smtp/send.rs
+++ b/src/smtp/send.rs
@@ -1,7 +1,7 @@
 //! # SMTP message sending

 use super::Smtp;
-use lettre::*;
+use async_smtp::*;

 use crate::context::Context;
 use crate::events::Event;
@@ -11,9 +11,9 @@ pub type Result<T> = std::result::Result<T, Error>;
 #[derive(Debug, Fail)]
 pub enum Error {
    #[fail(display = "Envelope error: {}", _0)]
-    EnvelopeError(#[cause] lettre::error::Error),
+    EnvelopeError(#[cause] async_smtp::error::Error),
    #[fail(display = "Send error: {}", _0)]
-    SendError(#[cause] lettre::smtp::error::Error),
+    SendError(#[cause] async_smtp::smtp::error::Error),
    #[fail(display = "SMTP has no transport")]
    NoTransport,
 }
@@ -21,7 +21,7 @@ pub enum Error {
 impl Smtp {
    /// Send a prepared mail to recipients.
    /// On successful send out Ok() is returned.
-    pub fn send(
+    pub async fn send(
        &mut self,
        context: &Context,
        recipients: Vec<EmailAddress>,
@@ -36,22 +36,21 @@ impl Smtp {
            .collect::<Vec<String>>()
            .join(",");

+        let envelope =
+            Envelope::new(self.from.clone(), recipients).map_err(Error::EnvelopeError)?;
+        let mail = SendableEmail::new(
+            envelope,
+            format!("{}", job_id), // only used for internal logging
+            message,
+        );
        if let Some(ref mut transport) = self.transport {
-            let envelope =
-                Envelope::new(self.from.clone(), recipients).map_err(Error::EnvelopeError)?;
-            let mail = SendableEmail::new(
-                envelope,
-                format!("{}", job_id), // only used for internal logging
-                message,
-            );
-
-            transport.send(mail).map_err(Error::SendError)?;
+            transport.send(mail).await.map_err(Error::SendError)?;

            context.call_cb(Event::SmtpMessageSent(format!(
                "Message len={} was smtp-sent to {}",
                message_len, recipients_display
            )));
-            self.transport_connected = true;
+
            Ok(())
        } else {
            warn!(
--- a/src/sql.rs
+++ b/src/sql.rs
@@ -1,7 +1,5 @@
 //! # SQLite wrapper

-use failure::Fail;
-
 use std::collections::HashSet;
 use std::sync::{Arc, RwLock};
 use std::time::Duration;
@@ -9,7 +7,7 @@ use std::time::Duration;
 use rusqlite::{Connection, OpenFlags, Statement, NO_PARAMS};
 use thread_local_object::ThreadLocal;

-use crate::chat::update_saved_messages_icon;
+use crate::chat::{update_device_icon, update_saved_messages_icon};
 use crate::constants::ShowEmails;
 use crate::context::Context;
 use crate::dc_tools::*;
@@ -850,7 +848,6 @@ fn open(
            if exists_before_update && sql.get_raw_config_int(context, "bcc_self").is_none() {
                sql.set_raw_config_int(context, "bcc_self", 1)?;
            }
-            update_icons = true;
            sql.set_raw_config_int(context, "dbversion", 59)?;
        }
        if dbversion < 60 {
@@ -861,6 +858,15 @@ fn open(
            )?;
            sql.set_raw_config_int(context, "dbversion", 60)?;
        }
+        if dbversion < 61 {
+            info!(context, "[migration] v61");
+            sql.execute(
+                "ALTER TABLE contacts ADD COLUMN selfavatar_sent INTEGER DEFAULT 0;",
+                NO_PARAMS,
+            )?;
+            update_icons = true;
+            sql.set_raw_config_int(context, "dbversion", 61)?;
+        }

        // (2) updates that require high-level objects
        // (the structure is complete now and all objects are usable)
@@ -886,6 +892,7 @@ fn open(
        }
        if update_icons {
            update_saved_messages_icon(context)?;
+            update_device_icon(context)?;
        }
    }

--- a/test-data/message/html_attach.eml
+++ b/test-data/message/html_attach.eml
@@ -0,0 +1,25 @@
+Chat-Disposition-Notification-To: tmp_6272287793210918@testrun.org
+Subject: =?utf-8?q?Chat=3A_File_=E2=80=93_test=2Ehtml?=
+Message-ID: Mr.XA6y3og8-az.WGbH9_dNcQx@testrun.org
+Date: Sat, 07 Dec 2019 19:00:27 +0000
+X-Mailer: Delta Chat Core 1.0.0-beta.12/DcFFI
+Chat-Version: 1.0
+To: <tmp_5890965001269692@testrun.org>
+From: "=?utf-8?q??=" <tmp_6272287793210918@testrun.org>
+Content-Type: multipart/mixed; boundary="mwkNRwaJw1M5n2xcr2ODfAqvTjcj9Z"
+
+
+--mwkNRwaJw1M5n2xcr2ODfAqvTjcj9Z
+Content-Type: text/plain; charset=utf-8
+
+-- 
+Sent with my Delta Chat Messenger: https://delta.chat
+
+--mwkNRwaJw1M5n2xcr2ODfAqvTjcj9Z
+Content-Type: text/html
+Content-Disposition: attachment; filename="test.html"
+Content-Transfer-Encoding: base64
+
+PGh0bWw+PGJvZHk+dGV4dDwvYm9keT5kYXRh
+
+--mwkNRwaJw1M5n2xcr2ODfAqvTjcj9Z--
--- a/test-data/message/mail_attach_txt.eml
+++ b/test-data/message/mail_attach_txt.eml
@@ -0,0 +1,40 @@
+From holger@merlinux.eu Sat Dec  7 11:53:58 2019
+Return-Path: <holger@merlinux.eu>
+X-Original-To: holger+test@merlinux.eu
+Delivered-To: holger+test@merlinux.eu
+Received: from [127.0.0.1] (localhost [127.0.0.1])
+	(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
+	(No client certificate requested)
+	by mail.merlinux.eu (Postfix) with ESMTPSA id 61825100531;
+	Sat,  7 Dec 2019 10:53:58 +0000 (UTC)
+DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=merlinux.eu;
+	s=default; t=1575716038;
+	bh=xhaPssQzVOHRafcciTQqDnZ1Zi4GMwsmg9pHLH3i8P8=;
+	h=Date:From:To:Subject;
+	b=U4HxGDZ8RwLwRPFtIvRsb+x5BiyICnbbY2ZOGlZdLt12MuDTfiYi/phHiQUC402EY
+	 GXb8dYgYr5+0PDiPBa7dyt2VQLC/h9QRfOA82tb1vpJYC+KksSAH0nYQqJvs7XrqCN
+	 i95/jwZnsWrV7w72+xsrO5qPujIE68TmM5I9Cyec=
+Received: by beto.merlinux.eu (Postfix, from userid 1000)
+	id 229D3820070; Sat,  7 Dec 2019 11:53:58 +0100 (CET)
+Date: Sat, 7 Dec 2019 11:53:57 +0100
+From: holger krekel <holger@merlinux.eu>
+To: holger+test@merlinux.eu
+Subject: hello
+Message-ID: <20191207105357.GA6266@beto>
+MIME-Version: 1.0
+Content-Type: multipart/mixed; boundary="YiEDa0DAkWCtVeE4"
+Content-Disposition: inline
+
+--YiEDa0DAkWCtVeE4
+Content-Type: text/plain; charset=us-ascii
+Content-Disposition: inline
+
+siehe anhang
+
+--YiEDa0DAkWCtVeE4
+Content-Type: text/plain; charset=us-ascii
+Content-Disposition: attachment; filename="x.txt"
+
+hello
+
+--YiEDa0DAkWCtVeE4--
--- a/test-data/message/mail_with_user_and_group_avatars.eml
+++ b/test-data/message/mail_with_user_and_group_avatars.eml
@@ -0,0 +1,55 @@
+Chat-Group-ID: WVnDtF5azch
+Chat-Group-Name: =?utf-8?q?testgr1?=
+Chat-Group-Avatar: group-image.png
+Chat-User-Avatar: avatar.png
+Subject: =?utf-8?q?Chat=3A_testgr1=3A_hi!_?=
+Date: Thu, 12 Dec 2019 17:24:03 +0000
+X-Mailer: Delta Chat Core 1.0.0-beta.15/CLI
+Chat-Version: 1.0
+Message-ID: <Gr.WVnDtF5azch.c6vUZfnnXYx@testrun.org>
+To: <bpetersen@b44t.com>
+From: =?utf-8?q??= <tunis4@testrun.org>
+Content-Type: multipart/mixed; boundary="LV8nfXkpyyn39fsVyoB1b29PKDMeb5"
+
+
+--LV8nfXkpyyn39fsVyoB1b29PKDMeb5
+Content-Type: text/plain; charset=utf-8
+
+hi! 
+
+-- 
+Sent with my Delta Chat Messenger: https://delta.chat
+
+--LV8nfXkpyyn39fsVyoB1b29PKDMeb5
+Content-Type: image/png
+Content-Disposition: attachment; filename="group-image.png"
+Content-Transfer-Encoding: base64
+
+iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAIAAACQkWg2AAABhGlDQ1BJQ0MgcHJvZmlsZQAAKJF9kT
+1Iw0AYht+mSqVUHCwi4pChOlkQFXHUKhShQqgVWnUwufQPmjQkKS6OgmvBwZ/FqoOLs64OroIg+APi
+4uqk6CIlfpcUWsR4x3EP733vy913gNCoMM3qGgc03TbTyYSYza2KoVdEEMYATUFmljEnSSn4jq97BP
+h+F+dZ/nV/jl41bzEgIBLPMsO0iTeIpzdtg/M+cZSVZJX4nHjMpAsSP3Jd8fiNc9FlgWdGzUx6njhK
+LBY7WOlgVjI14inimKrplC9kPVY5b3HWKjXWuid/YSSvryxzndYwkljEEiSIUFBDGRXYiNOuk2IhTe
+cJH/+Q65fIpZCrDEaOBVShQXb94H/wu7dWYXLCS4okgO4Xx/kYAUK7QLPuON/HjtM8AYLPwJXe9lcb
+wMwn6fW2FjsC+raBi+u2puwBlzvA4JMhm7IrBWkJhQLwfkbflAP6b4Hwmte31jlOH4AM9Sp1AxwcAq
+NFyl73eXdPZ9/+rWn17wcR7HKATfSiTAAAAAlwSFlzAAAuIwAALiMBeKU/dgAAAAd0SU1FB+MMChYX
+Fh+1IOwAAAAZdEVYdENvbW1lbnQAQ3JlYXRlZCB3aXRoIEdJTVBXgQ4XAAAAGElEQVQoz2P858hAEm
+BiYBjVMKphuGoAAAO8AV+n297RAAAAAElFTkSuQmCC
+
+--LV8nfXkpyyn39fsVyoB1b29PKDMeb5
+Content-Type: image/png
+Content-Disposition: attachment; filename="avatar.png"
+Content-Transfer-Encoding: base64
+
+iVBORw0KGgoAAAANSUhEUgAAABAAAAAQAQMAAAAlPW0iAAABhGlDQ1BJQ0MgcHJvZmlsZQAAKJF9kT
+1Iw0AcxV9TpUUqIhYRcchQnSyIFXHUKhShQqgVWnUwufQLmjQkKS6OgmvBwY/FqoOLs64OroIg+AHi
+4uqk6CIl/q8ptIjx4Lgf7+497t4BQr3MNKtrAtB020wl4mImuyoGXhFCEIOIoV9mljEnSUl4jq97+P
+h6F+VZ3uf+HL1qzmKATySeZYZpE28QT2/aBud94jAryirxOfG4SRckfuS64vIb50KTBZ4ZNtOpeeIw
+sVjoYKWDWdHUiKeII6qmU76QcVnlvMVZK1dZ6578haGcvrLMdZojSGARS5AgQkEVJZRhI0qrToqFFO
+3HPfzDTb9ELoVcJTByLKACDXLTD/4Hv7u18rFJNykUB7pfHOdjFAjsAo2a43wfO07jBPA/A1d621+p
+AzOfpNfaWuQI6NsGLq7bmrIHXO4AQ0+GbMpNyU9TyOeB9zP6piwwcAv0rLm9tfZx+gCkqavkDXBwCI
+wVKHvd493Bzt7+PdPq7wd6nHKqMKZUTAAAAANQTFRF/sYAhYATyAAAAAlwSFlzAAAuIwAALiMBeKU/
+dgAAAAd0SU1FB+MMCBY0D29+N8YAAAAZdEVYdENvbW1lbnQAQ3JlYXRlZCB3aXRoIEdJTVBXgQ4XAA
+AADElEQVQI12NgIA0AAAAwAAHHqoWOAAAAAElFTkSuQmCC
+
+--LV8nfXkpyyn39fsVyoB1b29PKDMeb5--
--- a/test-data/message/mail_with_user_avatar.eml
+++ b/test-data/message/mail_with_user_avatar.eml
@@ -0,0 +1,37 @@
+Chat-User-Avatar: avatar.png
+Subject: =?utf-8?q?Chat=3A_this_is_a_message_with_a_=2E=2E=2E?=
+Message-ID: Mr.wOBwZNbBTVt.NZpmQDwWoNk@example.org
+In-Reply-To: Mr.ETXqza5-WpB.zDEYOLECxAw@example.org
+Date: Sun, 08 Dec 2019 23:12:55 +0000
+X-Mailer: Delta Chat Core 1.0.0-beta.12/CLI
+Chat-Version: 1.0
+To: <tunis3@example.org>
+From: "=?utf-8?q??=" <tunis4@example.org>
+Content-Type: multipart/mixed; boundary="luTiGu6GBoVLCvTkzVtmZmwsmhkNMw"
+
+
+--luTiGu6GBoVLCvTkzVtmZmwsmhkNMw
+Content-Type: text/plain; charset=utf-8
+
+this is a message with a profile-image attached
+
+-- 
+Sent with my Delta Chat Messenger: https://delta.chat
+
+--luTiGu6GBoVLCvTkzVtmZmwsmhkNMw
+Content-Type: image/png
+Content-Disposition: attachment; filename="avatar.png"
+Content-Transfer-Encoding: base64
+
+iVBORw0KGgoAAAANSUhEUgAAABAAAAAQAQMAAAAlPW0iAAABhGlDQ1BJQ0MgcHJvZmlsZQAAKJF9kT
+1Iw0AcxV9TpUUqIhYRcchQnSyIFXHUKhShQqgVWnUwufQLmjQkKS6OgmvBwY/FqoOLs64OroIg+AHi
+4uqk6CIl/q8ptIjx4Lgf7+497t4BQr3MNKtrAtB020wl4mImuyoGXhFCEIOIoV9mljEnSUl4jq97+P
+h6F+VZ3uf+HL1qzmKATySeZYZpE28QT2/aBud94jAryirxOfG4SRckfuS64vIb50KTBZ4ZNtOpeeIw
+sVjoYKWDWdHUiKeII6qmU76QcVnlvMVZK1dZ6578haGcvrLMdZojSGARS5AgQkEVJZRhI0qrToqFFO
+3HPfzDTb9ELoVcJTByLKACDXLTD/4Hv7u18rFJNykUB7pfHOdjFAjsAo2a43wfO07jBPA/A1d621+p
+AzOfpNfaWuQI6NsGLq7bmrIHXO4AQ0+GbMpNyU9TyOeB9zP6piwwcAv0rLm9tfZx+gCkqavkDXBwCI
+wVKHvd493Bzt7+PdPq7wd6nHKqMKZUTAAAAANQTFRF/sYAhYATyAAAAAlwSFlzAAAuIwAALiMBeKU/
+dgAAAAd0SU1FB+MMCBY0D29+N8YAAAAZdEVYdENvbW1lbnQAQ3JlYXRlZCB3aXRoIEdJTVBXgQ4XAA
+AADElEQVQI12NgIA0AAAAwAAHHqoWOAAAAAElFTkSuQmCC
+
+--luTiGu6GBoVLCvTkzVtmZmwsmhkNMw--
--- a/test-data/message/mail_with_user_avatar_deleted.eml
+++ b/test-data/message/mail_with_user_avatar_deleted.eml
@@ -0,0 +1,14 @@
+Content-Type: text/plain; charset=utf-8
+Chat-User-Avatar: 0
+Subject: =?utf-8?q?Chat=3A_profile_image_deleted?=
+Message-ID: Mr.tsgoJgn-cBf.0TkFWKJzeSp@example.org
+Date: Sun, 08 Dec 2019 23:28:30 +0000
+X-Mailer: Delta Chat Core 1.0.0-beta.12/CLI
+Chat-Version: 1.0
+To: <tunis3@example>
+From: "=?utf-8?q??=" <tunis4@example.org>
+
+profile image deleted
+
+-- 
+Sent with my Delta Chat Messenger: https://delta.chat
--- a/tests/stress.rs
+++ b/tests/stress.rs
@@ -2,9 +2,7 @@

 use std::collections::HashSet;

-use deltachat::chat::{self, Chat};
 use deltachat::config;
-use deltachat::contact::*;
 use deltachat::context::*;
 use deltachat::keyring::*;
 use deltachat::pgp;
@@ -227,20 +225,3 @@ fn test_stress_tests() {
    let context = create_test_context();
    stress_functions(&context.ctx);
 }
-
-#[test]
-fn test_chat() {
-    let context = create_test_context();
-    let contact1 = Contact::create(&context.ctx, "bob", "bob@mail.de").unwrap();
-    assert_ne!(contact1, 0);
-
-    let chat_id = chat::create_by_contact_id(&context.ctx, contact1).unwrap();
-    assert!(chat_id > 9, "chat_id too small {}", chat_id);
-    let chat = Chat::load_from_db(&context.ctx, chat_id).unwrap();
-
-    let chat2_id = chat::create_by_contact_id(&context.ctx, contact1).unwrap();
-    assert_eq!(chat2_id, chat_id);
-    let chat2 = Chat::load_from_db(&context.ctx, chat2_id).unwrap();
-
-    assert_eq!(chat2.name, chat.name);
-}
Author	SHA1	Message	Date
holger krekel	ed790879cd	fix a few more places that used format! with sql-queries -- although they are all probably not critical	2019-12-14 18:28:59 +01:00
Alexander Krotov	558466d506	Revert "Enable clippy::unreadable_literal error" This reverts commit `b424ef9ebb`.	2019-12-14 15:55:33 +00:00
Alexander Krotov	b424ef9ebb	Enable clippy::unreadable_literal error	2019-12-14 15:20:58 +00:00
Alexander Krotov	69f5fd86a4	Allow clippy::needless_range_loop only in simplify_plain_text	2019-12-14 15:20:58 +00:00
Alexander Krotov	f9ba9ae912	Enable clippy::large_enum_variant error	2019-12-14 15:20:58 +00:00
Alexander Krotov	b33fec6236	Enable clippy::block_in_if_condition_stmt error	2019-12-14 15:20:58 +00:00
Alexander Krotov	70e12485aa	Switch to native_tls version of deltachat/lettre fork This commit completely removes rusttls dependency	2019-12-14 16:12:13 +01:00
Alexander Krotov	b2b7674b59	Update Cargo.lock	2019-12-14 16:12:13 +01:00
holger krekel	8fa175f36d	changelog and version up to 1.0.0-beta.16	2019-12-14 00:22:09 +01:00
B. Petersen	95fbc904d1	go to correct step so that param_autoconfig is actually used	2019-12-14 00:12:14 +01:00
B. Petersen	ce37a8dda2	use provider-database only if the user did not enter a server/port/whatever manually; this allows the user to override erroneous choices	2019-12-14 00:12:14 +01:00
holger krekel	e0499c9552	add preliminary support for offline-autoconfigure for nauta	2019-12-14 00:12:14 +01:00
Simon Laux	4825f2510f	Merge pull request #1016 from deltachat/fix_displayname fix displayname to use authname when availible	2019-12-13 13:21:03 +01:00
Alexander Krotov	7a12134795	Test that no gossip is sent in 1-to-1 chat	2019-12-13 12:51:44 +01:00
Alexander Krotov	66adfa074b	Do not cache gossiped_timestamp	2019-12-13 12:51:44 +01:00
Alexander Krotov	9d201eb9c6	Add get_gossiped_timestamp	2019-12-13 12:51:44 +01:00
Alexander Krotov	789fc0a7e0	Resultify set_gossiped_timestamp	2019-12-13 12:51:44 +01:00
Alexander Krotov	9e309132f8	Only add gossip headers if chat has more than one peer	2019-12-13 12:51:44 +01:00
Alexander Krotov	88923173c2	Set is_gossiped to true when Autocrypt-Gossip header is added	2019-12-13 12:51:44 +01:00
Simon Laux	7b55863418	switch priority order	2019-12-13 11:58:21 +01:00
Simon Laux	38d5fad4e5	fix displayname to use authname when availible	2019-12-13 11:58:08 +01:00
holger krekel	4d1357568b	use async-smtp master where @link2xt's branch has been merged	2019-12-13 11:03:46 +01:00
Alexander Krotov	ccc190f991	Switch to native_tls	2019-12-13 11:03:46 +01:00
B. Petersen	0a9f880fb4	remove another superfluous return value	2019-12-13 01:17:48 +01:00
holger krekel	b8faf54f0b	remove superflous expression values	2019-12-13 01:17:48 +01:00
B. Petersen	888507f7ba	test group- and user-avatars in the same mail	2019-12-13 01:17:48 +01:00
B. Petersen	29866092de	adapt the protected-header test; protected-headers really require encryption now and parts of the tests are know to fail now. however, protected-headers are also tested in python implicitly in several places	2019-12-13 01:17:48 +01:00
B. Petersen	36fc2c83f2	return selfavatar from dc_get_info()	2019-12-13 01:17:48 +01:00
holger krekel	5e777b3c51	fix test also for avatar deletion, use that drain filter while we in this code area :)	2019-12-13 01:17:48 +01:00
holger krekel	5690c48863	fix merging of protected headers	2019-12-13 01:17:48 +01:00
holger krekel	8ab3363097	add a test that runs until an avatar is send encrypted	2019-12-13 01:17:48 +01:00
B. Petersen	f6861ca5f5	target comments from @flub and @dignifiedquire	2019-12-13 01:17:48 +01:00
B. Petersen	3bb58be2b5	add missing delete header	2019-12-13 01:17:48 +01:00
B. Petersen	409c96e571	add compatiblity headers	2019-12-13 01:17:48 +01:00
B. Petersen	d681fa6cba	use Chat-User-Avatar and Chat-Group-Avatar we change the name on the wire as the old Chat-Group-Image header could not be used on random mails, it was the marker for a "Changed" message, if we would keep this names, things will fail for exising installations as messages are dropped and a "Group image changed" message is shown instead.	2019-12-13 01:17:48 +01:00
B. Petersen	7c3d8356c4	always handle incoming group-avatars	2019-12-13 01:17:48 +01:00
B. Petersen	a8842da50a	&& has a higher precedence than \|\| - fix a logical error when resetting Unpromoted	2019-12-13 01:17:48 +01:00
B. Petersen	ff29b84146	share group image on group creation, fixes #602	2019-12-13 01:17:48 +01:00
B. Petersen	ab12a4eb39	move ImageAction parsing to a reusable function	2019-12-13 01:17:48 +01:00
B. Petersen	c3fd0889e2	use tri-state ImageAction instead of Option<Option>	2019-12-13 01:17:48 +01:00
B. Petersen	c62532a665	adapt to new HeaderDef api	2019-12-13 01:17:48 +01:00
B. Petersen	ca63d6ba1c	handle incoming profile-images	2019-12-13 01:17:48 +01:00
B. Petersen	a1f496b019	let mimeparser handle profile-images	2019-12-13 01:17:48 +01:00
B. Petersen	da421438cd	do not send selfavatar on existing installation without resetting	2019-12-13 01:17:48 +01:00
B. Petersen	7f723ef2bf	attach selfavatar	2019-12-13 01:17:48 +01:00
B. Petersen	3cf39dace0	print profile-image in repl tool	2019-12-13 01:17:48 +01:00
B. Petersen	021ad4f12c	set correct icon for DC_CONTACT_ID_DEVICE	2019-12-13 01:17:48 +01:00
B. Petersen	36edf447e7	load param incl. user-profile-image from db	2019-12-13 01:17:48 +01:00
B. Petersen	ea2273aef4	reset selfavatar_sent column when the selfavatar is updated	2019-12-13 01:17:48 +01:00
B. Petersen	251aa22c4c	add column for selfavatar_sent timestamp	2019-12-13 01:17:48 +01:00
Alexander Krotov	541710147a	Copy correct dc_contact_is_verified documentation from C core instead of dc_contact_is_verified_ex documentation.	2019-12-11 17:54:28 +01:00
dignifiedquire	a197ca4cf7	feat: update dependencies, including rand@0.7 and rpgp@0.4	2019-12-11 15:33:17 +01:00
Simon Laux	775c36bb65	wip move some ci tasks to gh-actions (#998 ) * add-action-rs-clippy-action just to try this action out * add --all argument * rm --all argument again * add cargo check and fmt * disable circle ci counterparts(fetch, clippy, fmt)	2019-12-10 23:26:42 +01:00
holger krekel	1953b95c57	commit missing Cargo.lock	2019-12-10 21:42:52 +01:00
holger krekel	542a33952f	prepare 1.0.0-beta.15	2019-12-10 21:35:20 +01:00
holger krekel	1819712667	make sure we send out <> around message-id's in references and in-reply-to	2019-12-10 21:25:36 +01:00
holger krekel	69a596fdff	fix #994 -- avoid double-attachment messages in local chats	2019-12-10 21:25:36 +01:00
björn petersen	4f88b93495	Merge pull request #1003 from deltachat/flub-spec-their Use 3rd person singular	2019-12-10 19:10:59 +01:00
Floris Bruynooghe	a388052e2a	Use 3rd person singular	2019-12-10 19:07:22 +01:00
holger krekel	8b81ea3b6f	actually update to 1.0.0-beta.14	2019-12-10 15:39:09 +01:00
holger krekel	b913de5928	prepare a 1.0.0-beta.14 with fixed deps	2019-12-10 14:42:55 +01:00
holger krekel	4e551cde66	update tags and rust-email dep	2019-12-10 14:17:16 +01:00
holger krekel	b681cbd47f	prepare 1.0.0-beta.13	2019-12-10 14:02:21 +01:00
holger krekel	551253b4e0	use async-smtp with hostname fixed to version 0.1.5	2019-12-10 14:00:43 +01:00
Alexander Krotov	4ad9166b5a	Move one chat test from stress.rs to chat module	2019-12-10 13:54:05 +01:00
holger krekel	8487255c33	prepare changelog for 1.0.0-beta.13	2019-12-10 13:52:57 +01:00
Alexander Krotov	2792d4ea1e	Stop using create::error in login_param.rs	2019-12-10 12:39:58 +01:00
björn petersen	95180a850f	Merge pull request #993 from deltachat/group_add_remove Fix various issues around group add/remove	2019-12-10 12:14:52 +01:00
holger krekel	56ee7a0abd	try to remove updates to lock file	2019-12-10 02:33:13 +01:00
holger krekel	d0a04be825	remove hack for decoding, and add a test that encoding/decoding works now	2019-12-10 01:35:09 +01:00
holger krekel	2cbf287998	don't cargo-update on CI jobs but use the Cargo.lock file we manually maintain/update	2019-12-10 00:40:18 +01:00
holger krekel	054cf98754	try to work around mailparser not decoding rfc2047 displaynames this pulls in changes in our fork of rust-email to also correctly generate rfc2047 encoding	2019-12-10 00:26:46 +01:00
holger krekel	a95fbfe271	add a test	2019-12-09 22:28:29 +01:00
holger krekel	17ce02a87c	add some comment and remove some code after quick a/v with @r10s	2019-12-09 21:45:01 +01:00
holger krekel	4dc5e0378f	fix final problem, tests pass now	2019-12-09 20:25:46 +01:00
holger krekel	c33797ff84	remove reverse "check_self" return	2019-12-09 19:41:58 +01:00
holger krekel	f242b40d0a	nicer print of imap capabilities	2019-12-09 18:36:26 +01:00
holger krekel	5f916f5a9c	- create and use a ContactIds type as an ordered set instead of "Vec<u32>". - recreate the group list more carefully, fixes #985 - resultify a few functions in the dc_receive pipeline - don't quote displaynames in email-addresses, use utf8, preliminrarily addresses #976	2019-12-09 16:29:24 +01:00
holger krekel	6edb525540	snap	2019-12-09 14:16:46 +01:00
holger krekel	0c04d5b2ab	add a failing test that creates 4 accounts and a group chat with remove/add see also #985	2019-12-09 14:16:46 +01:00
Alexander Krotov	301852fd87	Add test for kebab_case serialization of From header	2019-12-09 15:43:06 +03:00
holger krekel	0e8df7d633	use automatic serialization, thanks @link2xt for the tip	2019-12-09 15:43:06 +03:00
holger krekel	e4155e0e16	remove lookup_field in favor of get(HeaderDef::...) with all headers defined in headerdef.rs	2019-12-09 15:43:06 +03:00
Floris Bruynooghe	2c4dbe6e68	Re-work some in-creation file handling This effectively reverts https://github.com/deltachat/deltachat-core-rust/pull/964 for chat.rs, which in that PR was thought to fix something. So maybe something is still broken? But after improving tests the previous code seems to be correct. - Update Python bindings to not always use dc_prepare_msg path when sending messages with attachements. When using dc_prepare_msg the blobs need to be created in the blobdir since they will not get copied and many tests where not doing this. - Add a test that ensures that calling dc_prepare_msg with a file not in the blobdir fails. - Add a test that ensures that calling dc_send_msg directly with a file not in the blobdir copies the file to the blobdir. This test cheats a little by knowing what the filename in the blobdir will be which is implementation-dependent and thus a bit brittle. But for now it proves correct behaviour so let's go with this. - Improve the test_forward_increation test to ensure that the in-creation file only has it's final state before calling dc_send_msg. This checks the correct file data is sent out and not the preparing data, this fails with the chat.rs changes in #964 (reverted here to make this work again). Also fix the test to actually create the in-creation file in the blobdir. - Fix test_send_file_twice_unicode_filename_mangling to not use in-creation. It was not creating it's files in the blobdir and that is an error when using in-creation and it didn't seem it was trying to test something about the in-creation logic (which is tested in test_increation.py already). - Fix Message._msgtate code which presumably was not used before? - Rename `BlobObject::create_from_path` to `BlobObject::new_from_path`. All the `BlobObject::create*` calls now always create new files which is much more consistent. APIs should do what is obious.	2019-12-09 10:30:57 +01:00
Alexander Krotov	a99b96e36e	Use bool for from_id_blocked flag	2019-12-09 09:54:46 +01:00
Alexander Krotov	0889467c7b	Use bool for hidden flag	2019-12-09 09:54:46 +01:00
Alexander Krotov	d141e228de	Use bool for incoming flag	2019-12-09 09:54:46 +01:00
Alexander Krotov	9b15c42801	Use bool for allow_creation flag	2019-12-09 09:54:46 +01:00
Alexander Krotov	a781b631e1	Return bool from is_reply_to_{known_message,messenger_message}	2019-12-09 09:54:46 +01:00
Alexander Krotov	08af5c8e09	Replace some integers with bools in dc_receive_imf.rs	2019-12-09 09:54:46 +01:00
holger krekel	93e8cca02f	remove redundant state from smtp	2019-12-08 23:04:08 +01:00
Alexander Krotov	a8e9a1fbe5	Make Smtp.send async	2019-12-08 21:00:03 +01:00
Alexander Krotov	54eb30f3db	Switch from lettre to async-smtp	2019-12-08 21:00:03 +01:00
Alexander Krotov	c08a1adc9b	Deprecate AcceptInvalidHostnames option Rustls does not offer a documented way to accept valid certificates with invalid hostnames. Implementation of certificate verification in Rustls does not have a public API and reimplementing it is error-prone.	2019-12-08 20:54:04 +01:00
Alexander Krotov	cd951ad396	Improve IMAP idle timeout error handling Return proper cause and avoid matching on nested Result	2019-12-08 20:50:27 +01:00
holger krekel	33793d878b	address @link2xt comment on done/async	2019-12-08 01:13:00 +01:00
holger krekel	357955015d	[wip] initial detection of idle done() not working	2019-12-08 01:13:00 +01:00
holger krekel	d6d94adab0	remove encrypted attr from mimeparser.	2019-12-08 00:47:46 +01:00
holger krekel	099cc9f727	remove subject attribute	2019-12-08 00:47:46 +01:00
holger krekel	61f8d6f171	remove is_send_by_messenger attr from mimeparser	2019-12-08 00:47:46 +01:00
Alexander Krotov	a7af4685f1	Log IMAP certificate checks configuration	2019-12-08 00:31:22 +01:00
Alexander Krotov	2cebed4f77	Move dc_array.rs to deltachat-ffi package	2019-12-08 00:30:43 +01:00
holger krekel	3f2a371599	run lint without installing the package	2019-12-07 22:56:54 +01:00
holger krekel	a3ca3d9179	allow to use a different buildhost :)	2019-12-07 22:56:54 +01:00
holger krekel	86ace1a4af	- test and fix receiving text/html attachment in multipart/mixed situations They are now preserved as attachment, instead of diving into parsing-html and simplifying. - adapt mime-debugging	2019-12-07 22:56:54 +01:00
holger krekel	1abdf62045	passes test but needs cleanup	2019-12-07 22:56:54 +01:00
holger krekel	9e2a96675d	try fix incoming text/html in multipart/mixed	2019-12-07 22:56:54 +01:00
Alexander Krotov	f32876708a	Update outdated comment MIME wrapping is now done by the email crate, which implements RFC5322	2019-12-07 21:35:42 +01:00
Alexander Krotov	84e0e7e020	aheader.rs: do not generate lines longer than 78 characters	2019-12-07 21:35:42 +01:00
Alexander Krotov	38cddff6e9	Move keydata= wrapping from key.rs to aheader.rs	2019-12-07 21:35:42 +01:00
Alexander Krotov	f1aba8115b	Use Option<EncryptPreference>.unwrap_or_default in aheader.rs	2019-12-07 07:58:53 +01:00
B. Petersen	0db5ff55fd	unshape icons, used indexed colors	2019-12-06 20:10:19 +01:00
B. Petersen	7421b0de6c	adapt the ffi-examples to reality, free stuff	2019-12-06 20:08:57 +01:00
Alexander Krotov	6c275c30a7	Do not include "prefer-encrypt=nopreference;" in Autocrypt header Specification https://autocrypt.org/level1.html#the-autocrypt-header says that prefer-encrypt attribute can only occur with the value `mutual`.	2019-12-06 20:01:34 +03:00
Alexander Krotov	91f1d793e9	Test that internal "prefer-encrypt=reset" value is not parsed from header This is a test for bugfix `e8f2f7b24e`	2019-12-06 14:23:45 +01:00
Alexander Krotov	cddfd04a7f	Turn rawkey from function into a constant	2019-12-06 14:23:45 +01:00
Alexander Krotov	7fa94c33bf	cargo fmt	2019-12-06 10:35:02 +01:00
Alexander Krotov	30dd20dc7b	Make Imap.fetch() async	2019-12-06 10:35:02 +01:00
Alexander Krotov	339c0d3dc7	Remove unnecessary use of failure::Fail	2019-12-06 10:34:34 +01:00
Alexander Krotov	2304d63bb3	Simplify dc_get_filesuffix_lc	2019-12-06 09:45:24 +01:00
holger krekel	36014f9fe5	add next changelog entry	2019-12-06 09:00:23 +01:00
Alexander Krotov	351383dfa4	Split IMAP idle functions into imap::idle submodule Also introduce Imap.can_idle() to avoid having to match on IdleAbilityMissing error	2019-12-06 08:59:22 +01:00
holger krekel	defad94f5a	bump to beta12	2019-12-06 00:55:24 +01:00
holger krekel	a0517478e3	adding auto-copy-blob logic when preparing a message	2019-12-06 00:27:22 +01:00
holger krekel	7f4e68f21c	failing test for sending out a file twice	2019-12-06 00:27:22 +01:00
holger krekel	c0a425c26d	bump to beta11	2019-12-05 21:53:03 +01:00
holger krekel	e756859b16	don't split qr tests out anymore now	2019-12-05 21:50:17 +01:00
holger krekel	174bc017ad	make setup_handle_if_needed async, call it ahead of select_folder and fetch_new_messages (renamed from fetch_from_single_folder), and be more eager triggering reconnect on error conditions	2019-12-05 21:47:32 +01:00
holger krekel	4a9fb0212f	complete changelog and bump version	2019-12-05 19:32:16 +01:00
holger krekel	212848409f	use encoded-words crate, which friedel ported from python	2019-12-05 19:29:12 +01:00
holger krekel	e45ee0eb81	try fix filename encoding bug -- fails in one test	2019-12-05 19:29:12 +01:00
dignifiedquire	3b8e37de58	switch to quoted-printable, which is already used by mailparse	2019-12-05 19:29:12 +01:00
holger krekel	c2e8cc9bd6	use rfc2047 crate from @valodim and remove dc_strencode.rs completely	2019-12-05 19:29:12 +01:00
holger krekel	ec81d29580	fix multi-line subject encoding and introduce MIME debugging env var	2019-12-05 19:29:12 +01:00
jikstra	c4de0f3b17	Apply remaining requested changes	2019-12-05 18:57:19 +01:00
jikstra	965d41990e	change return to "" for errors	2019-12-05 18:57:19 +01:00
holger krekel	d96dba336b	Update deltachat-ffi/deltachat.h fix doc	2019-12-05 18:57:19 +01:00
jikstra	a7e1b4653e	Apply requested changes	2019-12-05 18:57:19 +01:00
jikstra	e38b42bc21	Add api docu	2019-12-05 18:57:19 +01:00
jikstra	36bd502292	cargo fmt	2019-12-05 18:57:19 +01:00
jikstra	594bf3dfc8	fixup! Move to json_serde, add tests and implement missing python api	2019-12-05 18:57:19 +01:00
jikstra	6d30ccfc63	Move to json_serde, add tests and implement missing python api	2019-12-05 18:57:19 +01:00
jikstra	1b79f513a3	Implement more json key/value pairs	2019-12-05 18:57:19 +01:00
holger krekel	74825a0f57	working example	2019-12-05 18:57:19 +01:00
holger krekel	4a23d12df2	add a ffi-definiton for a new get-chat summary function that returns json	2019-12-05 18:57:19 +01:00