Compare commits

..

1 Commits

Author SHA1 Message Date
link2xt
bd81ecdb5d Add option to force E2EE encryption preference
Enabling this option ignores Autocrypt recommendation taking others
encryption preferences into account and overrides it with our own
encryption preference when possible.

This is similar to user always manually enabling/disabling encryption
manually in a classic Autocrypt-capable MUA UI whenever the control is
not disabled.

The goal is to allow encrypting responses to MUAs which can send
Autocrypt header but don't support setting encryption preference, such
as Thunderbird 91.
2021-11-28 16:32:18 +00:00
137 changed files with 6422 additions and 12525 deletions

26
.github/mergeable.yml vendored
View File

@@ -1,26 +0,0 @@
version: 2
mergeable:
- when: pull_request.*
name: "Changelog check"
validate:
- do: or
validate:
- do: description
must_include:
regex: '#skip-changelog'
- do: and
validate:
- do: dependent
changed:
file: 'src/**'
required: ['CHANGELOG.md']
- do: dependent
changed:
file: 'deltachat-ffi/**'
required: ['CHANGELOG.md']
fail:
- do: checks
status: 'action_required'
payload:
title: Changlog might need an update
summary: "Check if CHANGELOG.md needs an update or add #skip-changelog to the PR description."

View File

@@ -8,9 +8,6 @@ on:
- staging
- trying
env:
RUSTFLAGS: -Dwarnings
jobs:
fmt:
@@ -24,8 +21,6 @@ jobs:
toolchain: stable
override: true
- run: rustup component add rustfmt
- name: Cache rust cargo artifacts
uses: swatinem/rust-cache@v1
- uses: actions-rs/cargo@v1
with:
command: fmt
@@ -40,8 +35,6 @@ jobs:
toolchain: stable
components: clippy
override: true
- name: Cache rust cargo artifacts
uses: swatinem/rust-cache@v1
- uses: actions-rs/clippy-check@v1
with:
token: ${{ secrets.GITHUB_TOKEN }}
@@ -77,19 +70,19 @@ jobs:
include:
# Currently used Rust version, same as in `rust-toolchain` file.
- os: ubuntu-latest
rust: 1.59.0
rust: 1.54.0
python: 3.9
- os: windows-latest
rust: 1.59.0
rust: 1.54.0
python: false # Python bindings compilation on Windows is not supported.
# Minimum Supported Rust Version = 1.56.0
# Minimum Supported Rust Version = 1.51.0
#
# Minimum Supported Python Version = 3.7
# This is the minimum version for which manylinux Python wheels are
# built.
- os: ubuntu-latest
rust: 1.56.0
rust: 1.51.0
python: 3.7
runs-on: ${{ matrix.os }}
steps:
@@ -101,14 +94,31 @@ jobs:
toolchain: ${{ matrix.rust }}
override: true
- name: Cache rust cargo artifacts
uses: swatinem/rust-cache@v1
- name: Cache cargo registry
uses: actions/cache@v2
with:
path: ~/.cargo/registry
key: ${{ matrix.os }}-${{ matrix.rust }}-cargo-registry-${{ hashFiles('**/Cargo.toml') }}
- name: Cache cargo index
uses: actions/cache@v2
with:
path: ~/.cargo/git
key: ${{ matrix.os }}-${{ matrix.rust }}-cargo-index-${{ hashFiles('**/Cargo.toml') }}
- name: Cache cargo build
uses: actions/cache@v2
with:
path: target
key: ${{ matrix.os }}-${{ matrix.rust }}-cargo-build-target-${{ hashFiles('**/Cargo.toml') }}
- name: check
uses: actions-rs/cargo@v1
env:
RUSTFLAGS: -D warnings
with:
command: check
args: --all --bins --examples --tests --features repl --benches
args: --all --bins --examples --tests --features repl
- name: tests
uses: actions-rs/cargo@v1

View File

@@ -1,28 +0,0 @@
name: Build & Deploy Documentation on rs.delta.chat
on:
push:
branches:
- master
- docs-gh-action
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v1
- uses: actions-rs/toolchain@v1
- name: Build the documentation with cargo
run: |
cargo doc --package deltachat --no-deps
- name: Upload to rs.delta.chat
uses: up9cloud/action-rsync@v1.3
env:
USER: ${{ secrets.USERNAME }}
KEY: ${{ secrets.KEY }}
HOST: "delta.chat"
SOURCE: "target/doc"
TARGET: "/var/www/html/rs/"

View File

@@ -1,28 +0,0 @@
name: Build & Deploy Documentation on cffi.delta.chat
on:
push:
branches:
- master
- docs-gh-action
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v1
- uses: actions-rs/toolchain@v1
- name: Build the documentation with cargo
run: |
cargo doc --package deltachat_ffi --no-deps
- name: Upload to cffi.delta.chat
uses: up9cloud/action-rsync@v1.3
env:
USER: ${{ secrets.USERNAME }}
KEY: ${{ secrets.KEY }}
HOST: "delta.chat"
SOURCE: "target/doc"
TARGET: "/var/www/html/cffi/"

View File

@@ -1,162 +1,5 @@
# Changelog
## Unreleased
### API changes
- change semantics of `dc_get_webxdc_status_updates()` second parameter
and remove update-id from `DC_EVENT_WEBXDC_STATUS_UPDATE` #3081
### Fixes
- Hopefully fix a bug where outgoing messages appear twice with Amazon SES #3077
- do not delete messages without Message-IDs as duplicates #3095
- Assign replies from a different email address to the correct chat #3119
- start ephemeral timer when seen status is synchronized via IMAP #3122
- do not delete duplicate messages on IMAP immediately to accidentally deleting
the last copy #3138
### Changes
- add more SMTP logging #3093
- place common headers like `From:` before the large `Autocrypt:` header #3079
- keep track of securejoin joiner status in database to survive restarts #2920
- remove never used `SentboxMove` option #3111
- improve speed by caching config values #3131 #3145
- optimize `markseen_msgs` #3141
- automatically accept chats with outgoing messages #3143
- `dc_receive_imf` refactorings #3154 #3156
- add index to speedup deletion of expired ephemeral messages #3155
### Fixes
- Fix a bug where sometimes the file extension of a long filename containing a dot was cropped #3098
## 1.76.0
### Changes
- move messages in batches #3058
- delete messages in batches #3060
- python: remove arbitrary timeouts from tests #3059
- refactorings #3026
### Fixes
- avoid archived, fresh chats #3053
- Also resync UIDs in folders that are not configured #2289
- treat "NO" IMAP response to MOVE and COPY commands as an error #3058
- Fix a bug where messages in the Spam folder created contact requests #3015
- Fix a bug where drafts disappeared after some days #3067
- Parse MS Exchange read receipts and mark the original message as read #3075
- do not retry message sending infinitely in case of permanent SMTP failure #3070
- set message state to failed when retry limit is exceeded #3072
## 1.75.0
### Changes
- optimize `delete_expired_imap_messages()` #3047
## 1.74.0
### Fixes
- avoid reconnection loop when message without Message-ID is marked as seen #3044
## 1.73.0
### API changes
- added `only_fetch_mvbox` config #3028
### Changes
- don't watch Sent folder by default #3025
- use webxdc app name in chatlist/quotes/replies etc. #3027
- make it possible to cancel message sending by removing the message #3034,
this was previosuly removed in 1.71.0 #2939
- synchronize Seen flags only on watched folders to speed up
folder scanning #3041
- remove direct dependency on `byteorder` crate #3031
- refactorings #3023 #3013
- update provider database #3043
- improve documentation #3017 #3018 #3021
### Fixes
- fix splitting off text from webxdc messages #3032
- call slow `delete_expired_imap_messages()` less often #3037
- make synchronization of Seen status more robust in case unsolicited FETCH
result without UID is returned #3022
- fetch Inbox before scanning folders to ensure iOS does
not kill the app before it gets to fetch the Inbox in background #3040
## 1.72.0
### Fixes
- run migrations on backup import #3006
## 1.71.0
### API Changes
- added APIs to handle database passwords: `dc_context_new_closed()`, `dc_context_open()`,
`dc_context_is_open()` and `dc_accounts_add_closed_account()` #2956 #2972
- use second parameter of `dc_imex` to provide backup passphrase #2980
- added `DC_MSG_WEBXDC`, `dc_send_webxdc_status_update()`,
`dc_get_webxdc_status_updates()`, `dc_msg_get_webxdc_blob()`, `dc_msg_get_webxdc_info()`
and `DC_EVENT_WEBXDC_STATUS_UPDATE` #2826 #2971 #2975 #2977 #2979 #2993 #2994 #2998 #3001 #3003
- added `dc_msg_get_parent()` #2984
- added `dc_msg_force_plaintext()` API for bots #2847
- allow removing quotes on drafts `dc_msg_set_quote(msg, NULL)` #2950
- removed `mvbox_watch` option; watching is enabled when `mvbox_move` is enabled #2906
- removed `inbox_watch` option #2922
- deprecated `os_name` in `dc_context_new()`, pass `NULL` or an empty string #2956
### Changes
- start making it possible to write to mailing lists #2736
- add `hop_info` to `dc_get_info()` #2751 #2914 #2923
- add information about whether the database is encrypted or not to `dc_get_info()` #3000
- selfstatus now defaults to empty #2951 #2960
- validate detached cryptographic signatures as used eg. by Thunderbird #2865
- do not change the draft's `msg_id` on updates and sending #2887
- add `imap` table to keep track of message UIDs #2909 #2938
- replace `SendMsgToSmtp` jobs which stored outgoing messages in blobdir with `smtp` SQL table #2939 #2996
- sql: enable `auto_vacuum=INCREMENTAL` #2931
- sql: build rusqlite with sqlcipher #2934
- synchronize Seen status across devices #2942
- `dc_preconfigure_keypair` now takes ascii armored keys instead of base64 #2862
- put removed member in Bcc instead of To in the message about removal #2864
- improve group updates #2889
- re-write the blob filename creation loop #2981
- update provider database (11 Jan 2022) #2959
- python: allow timeout for internal configure tracker API #2967
- python: remove API deprecated in Python 3.10 #2907
- refactorings #2932 #2957 #2947
- improve tests #2863 #2866 #2881 #2908 #2918 #2901 #2973
- improve documentation #2880 #2886 #2895
- improve ci #2919 #2926 #2969 #2999
### Fixes
- fix leaving groups #2929
- fix unread count #2861
- make `add_parts()` not early-exit #2879
- recognize MS Exchange read receipts as read receipts #2890
- create parent directory if creating a new file fails #2978
- save "configured" flag later #2974
- improve log #2928
- `dc_receive_imf`: don't fail on invalid address in the To field #2940
## 1.70.0
### Fixes
- fix: do not abort Param parsing on unknown keys #2856
- fix: execute `Chat-Group-Member-Removed:` even when arriving disordered #2857
## 1.69.0
### Fixes
- fix group-related system messages in multi-device setups #2848
- fix "Google Workspace" (former "G Suite") issues related to bad resolvers #2852
## 1.68.0
### Fixes

965
Cargo.lock generated

File diff suppressed because it is too large Load Diff

View File

@@ -1,10 +1,10 @@
[package]
name = "deltachat"
version = "1.76.0"
version = "1.68.0"
authors = ["Delta Chat Developers (ML) <delta@codespeak.net>"]
edition = "2021"
edition = "2018"
license = "MPL-2.0"
rust-version = "1.56"
resolver = "2"
[profile.dev]
debug = 0
@@ -20,13 +20,14 @@ anyhow = "1"
async-imap = { git = "https://github.com/async-email/async-imap" }
async-native-tls = { version = "0.3" }
async-smtp = { git = "https://github.com/async-email/async-smtp", branch="master", features = ["socks5"] }
async-std-resolver = "0.21"
async-std = { version = "1" }
async-std-resolver = "0.20"
async-std = { version = "1", features = ["unstable"] }
async-tar = { version = "0.4", default-features=false }
async-trait = "0.1"
backtrace = "0.3"
base64 = "0.13"
bitflags = "1.3"
byteorder = "1.3"
chrono = "0.4"
dirs = { version = "4", optional=true }
email = { git = "https://github.com/deltachat/rust-email", branch = "master" }
@@ -34,37 +35,37 @@ encoded-words = { git = "https://github.com/async-email/encoded-words", branch="
escaper = "0.1"
futures = "0.3"
hex = "0.4.0"
image = { version = "0.24.1", default-features=false, features = ["gif", "jpeg", "ico", "png", "pnm", "webp", "bmp"] }
image = { version = "0.23.5", default-features=false, features = ["gif", "jpeg", "ico", "png", "pnm", "webp", "bmp"] }
kamadak-exif = "0.5"
lettre_email = { git = "https://github.com/deltachat/lettre", branch = "master" }
libc = "0.2"
log = {version = "0.4.16", optional = true }
log = {version = "0.4.8", optional = true }
mailparse = "0.13"
native-tls = "0.2"
num_cpus = "1.13"
num-derive = "0.3"
num-traits = "0.2"
once_cell = "1.10.0"
once_cell = "1.8.0"
percent-encoding = "2.0"
pgp = { version = "0.7", default-features = false }
pretty_env_logger = { version = "0.4", optional = true }
quick-xml = "0.22"
r2d2 = "0.8"
r2d2_sqlite = "0.20"
r2d2_sqlite = "0.19"
rand = "0.7"
regex = "1.5"
rusqlite = { version = "0.27", features = ["sqlcipher"] }
rusqlite = "0.26"
rust-hsluv = "0.1"
rustyline = { version = "9", optional = true }
rustyline = { version = "9.0", optional = true }
sanitize-filename = "0.3"
serde_json = "1.0"
serde = { version = "1.0", features = ["derive"] }
sha-1 = "0.10"
sha2 = "0.10"
sha-1 = "0.9"
sha2 = "0.9"
smallvec = "1"
stop-token = "0.7"
strum = "0.24"
strum_macros = "0.24"
stop-token = "0.6"
strum = "0.23"
strum_macros = "0.23"
surf = { version = "2.3", default-features = false, features = ["h1-client"] }
thiserror = "1"
toml = "0.5"
@@ -73,14 +74,13 @@ uuid = { version = "0.8", features = ["serde", "v4"] }
fast-socks5 = "0.4"
humansize = "1"
qrcodegen = "1.7.0"
tagger = "4.3.3"
textwrap = "0.15.0"
zip = { version = "0.6.0", default-features = false, features = ["deflate"] }
tagger = "3.2.1"
textwrap = "0.14.2"
[dev-dependencies]
ansi_term = "0.12.0"
async-std = { version = "1", features = ["unstable", "attributes"] }
criterion = { version = "0.3.4", features = ["async_std"] }
criterion = "0.3"
futures-lite = "1.12"
log = "0.4"
pretty_env_logger = "0.4"
@@ -116,17 +116,9 @@ harness = false
name = "search_msgs"
harness = false
[[bench]]
name = "receive_emails"
harness = false
[[bench]]
name = "get_chat_msgs"
harness = false
[features]
default = ["vendored"]
internals = []
repl = ["internals", "rustyline", "log", "pretty_env_logger", "ansi_term", "dirs"]
vendored = ["async-native-tls/vendored", "async-smtp/native-tls-vendored", "rusqlite/bundled-sqlcipher-vendored-openssl"]
vendored = ["async-native-tls/vendored", "async-smtp/native-tls-vendored", "rusqlite/bundled"]
nightly = ["pgp/nightly"]

View File

@@ -12,8 +12,6 @@ To download and install the official compiler for the Rust programming language,
$ curl https://sh.rustup.rs -sSf | sh
```
> On Windows, you may need to also install **Perl** to be able to compile deltachat-core.
## Using the CLI client
Compile and run Delta Chat Core command line utility, using `cargo`:
@@ -127,11 +125,11 @@ $ cargo test -- --ignored
Language bindings are available for:
- **C** \[[📂 source](./deltachat-ffi) | [📚 docs](https://c.delta.chat)\]
- **Node.js** \[[📂 source](https://github.com/deltachat/deltachat-node) | [📦 npm](https://www.npmjs.com/package/deltachat-node) | [📚 docs](https://js.delta.chat)\]
- **Python** \[[📂 source](./python) | [📦 pypi](https://pypi.org/project/deltachat) | [📚 docs](https://py.delta.chat)\]
- **Go** \[[📂 source](https://github.com/deltachat/go-deltachat/)\]
- **Free Pascal** \[[📂 source](https://github.com/deltachat/deltachat-fp/)\]
- [C](https://c.delta.chat)
- [Node.js](https://www.npmjs.com/package/deltachat-node)
- [Python](https://py.delta.chat)
- [Go](https://github.com/deltachat/go-deltachat/)
- [Free Pascal](https://github.com/deltachat/deltachat-fp/)
- **Java** and **Swift** (contained in the Android/iOS repos)
The following "frontend" projects make use of the Rust-library

Binary file not shown.

Before

Width:  |  Height:  |  Size: 6.9 KiB

View File

@@ -1,181 +0,0 @@
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<svg
xmlns:dc="http://purl.org/dc/elements/1.1/"
xmlns:cc="http://creativecommons.org/ns#"
xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
xmlns:svg="http://www.w3.org/2000/svg"
xmlns="http://www.w3.org/2000/svg"
xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
width="80mm"
height="297mm"
viewBox="0 0 80 297"
version="1.1"
id="svg71"
inkscape:version="1.0.2 (e86c8708, 2021-01-15)"
sodipodi:docname="icon-webxdc.svg"
inkscape:export-filename="C:\Users\user\OneDrive - BFW-Leipzig\Documents\LogoDC\finalohnerand.png"
inkscape:export-xdpi="96"
inkscape:export-ydpi="96">
<metadata
id="metadata856">
<rdf:RDF>
<cc:Work
rdf:about="">
<dc:format>image/svg+xml</dc:format>
<dc:type
rdf:resource="http://purl.org/dc/dcmitype/StillImage" />
<dc:title></dc:title>
</cc:Work>
</rdf:RDF>
</metadata>
<sodipodi:namedview
id="namedview73"
pagecolor="#767676"
bordercolor="#666666"
borderopacity="1.0"
inkscape:pageshadow="2"
inkscape:pageopacity="0"
inkscape:pagecheckerboard="true"
inkscape:document-units="mm"
showgrid="false"
showborder="false"
inkscape:snap-bbox="true"
inkscape:bbox-paths="true"
inkscape:bbox-nodes="true"
inkscape:snap-bbox-edge-midpoints="true"
inkscape:snap-bbox-midpoints="true"
inkscape:object-paths="true"
inkscape:snap-intersection-paths="true"
inkscape:zoom="1.4142136"
inkscape:cx="-90.271136"
inkscape:cy="-1233.1209"
inkscape:window-width="1864"
inkscape:window-height="1027"
inkscape:window-x="56"
inkscape:window-y="25"
inkscape:window-maximized="1"
inkscape:current-layer="layer1"
inkscape:snap-global="false"
showguides="false"
inkscape:guide-bbox="true"
inkscape:document-rotation="0"
units="px">
<sodipodi:guide
position="-154.76097,641.11689"
orientation="0,-1"
id="guide21118" />
<sodipodi:guide
position="-60.286487,633.36619"
orientation="0,-1"
id="guide21120" />
</sodipodi:namedview>
<defs
id="defs68">
<linearGradient
id="linearGradient4375">
<stop
style="stop-color:#364e59;stop-opacity:1;"
offset="0"
id="stop4377" />
<stop
style="stop-color:#364e59;stop-opacity:0;"
offset="1"
id="stop4379" />
</linearGradient>
</defs>
<g
inkscape:label="Ebene 1"
inkscape:groupmode="layer"
id="layer1">
<rect
style="fill:#1a1a1a;stroke:#000000;stroke-width:0.167903"
id="rect880"
width="79.8321"
height="79.8321"
x="-64.03286"
y="-375.9097"
ry="0" />
<path
inkscape:connector-curvature="0"
id="path3799-2"
d="m -24.089585,-372.59579 c -19.986026,0.24336 -36.196903,16.666 -36.196903,36.67011 0,20.00409 16.210877,36.03233 36.196903,35.78912 19.0024236,-0.076 14.5340713,-10.6146 35.538854,-0.85693 -11.50627538,-17.97454 0.390097,-20.36737 0.658079,-35.81316 0,-20.00411 -16.2108788,-36.03235 -36.196911,-35.78914 z"
style="fill:#364e59;fill-opacity:1;stroke:none;stroke-width:1.93355"
sodipodi:nodetypes="sscccs" />
<path
style="fill:none;stroke:#000000;stroke-width:0.264583px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"
d="M -54.193871,-325.26419 Z"
id="path3846" />
<path
style="fill:none;stroke:#000000;stroke-width:0.264583px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"
d="M -49.397951,-326.67773 Z"
id="path3848" />
<path
style="fill:none;stroke:#000000;stroke-width:0.264583px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"
d="m -49.397951,-326.67773 v 0 0"
id="path3850" />
<path
style="fill:none;stroke:#000000;stroke-width:0.01;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1"
d="m -51.35133,-325.0334 -7.964067,5.98895 z"
id="path3965" />
<path
inkscape:connector-curvature="0"
id="path11037"
d="m -24.089585,-372.19891 c -19.986026,0.24156 -36.196903,16.54352 -36.196903,36.40062 0,7.86524 2.543315,15.1113 6.857155,20.97971 6.577146,8.94734 11.123515,9.77363 11.123515,9.77363 1.343237,1.78324 10.270932,4.3223 10.270932,4.3223 l 16.791727,-70.86654 -0.468369,-0.33457 c 0.458597,0.26445 0.428277,-0.27515 -8.378035,-0.27515 z"
style="fill:#7cc5cc;fill-opacity:1;stroke:none;stroke-width:1.92643"
sodipodi:nodetypes="sssccccss" />
<path
style="fill:none;stroke:#000000;stroke-width:0.264583px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"
d="M -49.944239,-310.69957 Z"
id="path13674" />
<g
id="g15178"
transform="matrix(0.79975737,0,0,0.79975737,53.088959,-63.716396)">
<rect
style="fill:#364e59;fill-opacity:1;stroke-width:0.01;stroke-miterlimit:4;stroke-dasharray:none"
id="rect15072"
width="29.897917"
height="6.8791666"
x="-334.4964"
y="-154.51025"
transform="rotate(45)" />
<rect
style="fill:#364e59;fill-opacity:1;stroke-width:0.01;stroke-miterlimit:4;stroke-dasharray:none"
id="rect15072-5"
width="29.897917"
height="6.8791666"
x="147.63107"
y="-334.4964"
transform="rotate(-45)"
inkscape:transform-center-x="-0.74835017"
inkscape:transform-center-y="0.37417525" />
</g>
<g
id="g22468"
transform="translate(3.3033974)">
<g
id="g15178-0"
transform="matrix(-0.79975737,0,0,0.79975737,-103.11028,-63.716404)"
style="fill:#7cc5cc;fill-opacity:1">
<rect
style="fill:#7cc5cc;fill-opacity:1;stroke-width:0.01;stroke-miterlimit:4;stroke-dasharray:none"
id="rect15072-2"
width="29.897917"
height="6.8791666"
x="-334.4964"
y="-154.51025"
transform="rotate(45)" />
<rect
style="fill:#7cc5cc;fill-opacity:1;stroke-width:0.01;stroke-miterlimit:4;stroke-dasharray:none"
id="rect15072-5-5"
width="29.897917"
height="6.8791666"
x="147.63107"
y="-334.4964"
transform="rotate(-45)"
inkscape:transform-center-x="-0.74835017"
inkscape:transform-center-y="0.37417525" />
</g>
</g>
</g>
</svg>

Before

Width:  |  Height:  |  Size: 6.5 KiB

View File

@@ -8,7 +8,9 @@ async fn address_book_benchmark(n: u32, read_count: u32) {
let dir = tempdir().unwrap();
let dbfile = dir.path().join("db.sqlite");
let id = 100;
let context = Context::new(dbfile.into(), id).await.unwrap();
let context = Context::new("FakeOS".into(), dbfile.into(), id)
.await
.unwrap();
let book = (0..n)
.map(|i| format!("Name {}\naddr{}@example.org\n", i, i))

View File

@@ -8,7 +8,7 @@ async fn create_accounts(n: u32) {
let dir = tempdir().unwrap();
let p: PathBuf = dir.path().join("accounts").into();
let mut accounts = Accounts::new(p.clone()).await.unwrap();
let accounts = Accounts::new("my_os".into(), p.clone()).await.unwrap();
for expected_id in 2..n {
let id = accounts.add_account().await.unwrap();

View File

@@ -1,38 +0,0 @@
use async_std::path::Path;
use criterion::async_executor::AsyncStdExecutor;
use criterion::{black_box, criterion_group, criterion_main, Criterion};
use deltachat::chat::{self, ChatId};
use deltachat::chatlist::Chatlist;
use deltachat::context::Context;
async fn get_chat_msgs_benchmark(dbfile: &Path, chats: &[ChatId]) {
let id = 100;
let context = Context::new(dbfile.into(), id).await.unwrap();
for c in chats.iter().take(10) {
black_box(chat::get_chat_msgs(&context, *c, 0, None).await.ok());
}
}
fn criterion_benchmark(c: &mut Criterion) {
// To enable this benchmark, set `DELTACHAT_BENCHMARK_DATABASE` to some large database with many
// messages, such as your primary account.
if let Ok(path) = std::env::var("DELTACHAT_BENCHMARK_DATABASE") {
let chats: Vec<_> = async_std::task::block_on(async {
let context = Context::new((&path).into(), 100).await.unwrap();
let chatlist = Chatlist::try_load(&context, 0, None, None).await.unwrap();
let len = chatlist.len();
(0..len).map(|i| chatlist.get_chat_id(i).unwrap()).collect()
});
c.bench_function("Load all chats", |b| {
b.to_async(AsyncStdExecutor)
.iter(|| get_chat_msgs_benchmark(black_box(&path.as_ref()), black_box(&chats)))
});
}
}
criterion_group!(benches, criterion_benchmark);
criterion_main!(benches);

View File

@@ -1,84 +0,0 @@
use async_std::{path::PathBuf, task::block_on};
use criterion::{
async_executor::AsyncStdExecutor, black_box, criterion_group, criterion_main, BatchSize,
Criterion,
};
use deltachat::{
config::Config,
context::Context,
dc_receive_imf::dc_receive_imf,
imex::{imex, ImexMode},
};
use tempfile::tempdir;
async fn recv_all_emails(context: Context) -> Context {
for i in 0..100 {
let imf_raw = format!(
"Subject: Benchmark
Message-ID: Mr.OssSYnOFkhR.{i}@testrun.org
Date: Sat, 07 Dec 2019 19:00:27 +0000
To: alice@example.com
From: sender@testrun.org
Chat-Version: 1.0
Chat-Disposition-Notification-To: sender@testrun.org
Chat-User-Avatar: 0
In-Reply-To: Mr.OssSYnOFkhR.{i_dec}@testrun.org
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed; delsp=no
Hello {i}",
i = i,
i_dec = i - 1,
);
dc_receive_imf(&context, black_box(imf_raw.as_bytes()), "INBOX", false)
.await
.unwrap();
}
context
}
async fn create_context() -> Context {
let dir = tempdir().unwrap();
let dbfile = dir.path().join("db.sqlite");
let id = 100;
let context = Context::new(dbfile.into(), id).await.unwrap();
let backup: PathBuf = std::env::current_dir()
.unwrap()
.join("delta-chat-backup.tar")
.into();
if backup.exists().await {
println!("Importing backup");
imex(&context, ImexMode::ImportBackup, &backup, None)
.await
.unwrap();
}
let addr = "alice@example.com";
context.set_config(Config::Addr, Some(addr)).await.unwrap();
context
.set_config(Config::ConfiguredAddr, Some(addr))
.await
.unwrap();
context
.set_config(Config::Configured, Some("1"))
.await
.unwrap();
context
}
fn criterion_benchmark(c: &mut Criterion) {
let mut group = c.benchmark_group("Receive messages");
group.bench_function("Receive 100 simple text msgs", |b| {
b.to_async(AsyncStdExecutor).iter_batched(
|| block_on(create_context()),
|context| recv_all_emails(black_box(context)),
BatchSize::LargeInput,
);
});
group.finish();
}
criterion_group!(benches, criterion_benchmark);
criterion_main!(benches);

View File

@@ -6,7 +6,9 @@ use std::path::Path;
async fn search_benchmark(path: impl AsRef<Path>) {
let dbfile = path.as_ref();
let id = 100;
let context = Context::new(dbfile.into(), id).await.unwrap();
let context = Context::new("FakeOS".into(), dbfile.into(), id)
.await
.unwrap();
for _ in 0..10u32 {
context.search_msgs(None, "hello").await.unwrap();

View File

@@ -1,6 +1,6 @@
[package]
name = "deltachat_ffi"
version = "1.76.0"
version = "1.68.0"
description = "Deltachat FFI"
authors = ["Delta Chat Developers (ML) <delta@codespeak.net>"]
edition = "2018"

View File

@@ -1,9 +1,5 @@
# Delta Chat C Interface
## Installation
see `Installing libdeltachat system wide` in [../README.md](../README.md)
## Documentation
To generate the C Interface documentation,

File diff suppressed because it is too large Load Diff

View File

@@ -27,17 +27,15 @@ use async_std::sync::RwLock;
use async_std::task::{block_on, spawn};
use deltachat::qr_code_generator::get_securejoin_qr_svg;
use num_traits::{FromPrimitive, ToPrimitive};
use rand::Rng;
use deltachat::chat::{ChatId, ChatVisibility, MuteDuration, ProtectionStatus};
use deltachat::constants::DC_MSG_ID_LAST_SPECIAL;
use deltachat::contact::{Contact, ContactId, Origin};
use deltachat::contact::{Contact, Origin};
use deltachat::context::Context;
use deltachat::ephemeral::Timer as EphemeralTimer;
use deltachat::key::DcKey;
use deltachat::message::MsgId;
use deltachat::stock_str::StockMessage;
use deltachat::webxdc::StatusUpdateSerial;
use deltachat::*;
use deltachat::{accounts::Accounts, log::LogExt};
@@ -65,7 +63,7 @@ pub type dc_context_t = Context;
#[no_mangle]
pub unsafe extern "C" fn dc_context_new(
_os_name: *const libc::c_char,
os_name: *const libc::c_char,
dbfile: *const libc::c_char,
blobdir: *const libc::c_char,
) -> *mut dc_context_t {
@@ -76,10 +74,21 @@ pub unsafe extern "C" fn dc_context_new(
return ptr::null_mut();
}
let os_name = if os_name.is_null() {
String::from("DcFFI")
} else {
to_string_lossy(os_name)
};
let ctx = if blobdir.is_null() || *blobdir == 0 {
use rand::Rng;
// generate random ID as this functionality is not yet available on the C-api.
let id = rand::thread_rng().gen();
block_on(Context::new(as_path(dbfile).to_path_buf().into(), id))
block_on(Context::new(
os_name,
as_path(dbfile).to_path_buf().into(),
id,
))
} else {
eprintln!("blobdir can not be defined explicitly anymore");
return ptr::null_mut();
@@ -87,63 +96,12 @@ pub unsafe extern "C" fn dc_context_new(
match ctx {
Ok(ctx) => Box::into_raw(Box::new(ctx)),
Err(err) => {
eprintln!("failed to create context: {:#}", err);
eprintln!("failed to create context: {}", err);
ptr::null_mut()
}
}
}
#[no_mangle]
pub unsafe extern "C" fn dc_context_new_closed(dbfile: *const libc::c_char) -> *mut dc_context_t {
setup_panic!();
if dbfile.is_null() {
eprintln!("ignoring careless call to dc_context_new_closed()");
return ptr::null_mut();
}
let id = rand::thread_rng().gen();
match block_on(Context::new_closed(
as_path(dbfile).to_path_buf().into(),
id,
)) {
Ok(context) => Box::into_raw(Box::new(context)),
Err(err) => {
eprintln!("failed to create context: {:#}", err);
ptr::null_mut()
}
}
}
#[no_mangle]
pub unsafe extern "C" fn dc_context_open(
context: *mut dc_context_t,
passphrase: *const libc::c_char,
) -> libc::c_int {
if context.is_null() {
eprintln!("ignoring careless call to dc_context_open()");
return 0;
}
let ctx = &*context;
let passphrase = to_string_lossy(passphrase);
block_on(ctx.open(passphrase))
.log_err(ctx, "dc_context_open() failed")
.map(|b| b as libc::c_int)
.unwrap_or(0)
}
#[no_mangle]
pub unsafe extern "C" fn dc_context_is_open(context: *mut dc_context_t) -> libc::c_int {
if context.is_null() {
eprintln!("ignoring careless call to dc_context_is_open()");
return 0;
}
let ctx = &*context;
block_on(ctx.is_open()) as libc::c_int
}
/// Release the context structure.
///
/// This function releases the memory of the `dc_context_t` structure.
@@ -235,7 +193,7 @@ pub unsafe extern "C" fn dc_get_config(
.unwrap_or_default()
.strdup(),
Err(_) => {
warn!(ctx, "dc_get_config(): invalid key '{}'", &key);
warn!(ctx, "dc_get_config(): invalid key");
"".strdup()
}
}
@@ -493,17 +451,14 @@ pub unsafe extern "C" fn dc_event_get_data1_int(event: *mut dc_event_t) -> libc:
| EventType::ChatEphemeralTimerModified { chat_id, .. } => chat_id.to_u32() as libc::c_int,
EventType::ContactsChanged(id) | EventType::LocationChanged(id) => {
let id = id.unwrap_or_default();
id.to_u32() as libc::c_int
id as libc::c_int
}
EventType::ConfigureProgress { progress, .. } | EventType::ImexProgress(progress) => {
*progress as libc::c_int
}
EventType::ImexFileWritten(_) => 0,
EventType::SecurejoinInviterProgress { contact_id, .. }
| EventType::SecurejoinJoinerProgress { contact_id, .. } => {
contact_id.to_u32() as libc::c_int
}
EventType::WebxdcStatusUpdate(msg_id) => msg_id.to_u32() as libc::c_int,
| EventType::SecurejoinJoinerProgress { contact_id, .. } => *contact_id as libc::c_int,
}
}
@@ -536,7 +491,6 @@ pub unsafe extern "C" fn dc_event_get_data2_int(event: *mut dc_event_t) -> libc:
| EventType::MsgsNoticed(_)
| EventType::ConnectivityChanged
| EventType::SelfavatarChanged
| EventType::WebxdcStatusUpdate(_)
| EventType::ChatModified(_) => 0,
EventType::MsgsChanged { msg_id, .. }
| EventType::IncomingMsg { msg_id, .. }
@@ -587,7 +541,6 @@ pub unsafe extern "C" fn dc_event_get_data2_str(event: *mut dc_event_t) -> *mut
| EventType::SecurejoinJoinerProgress { .. }
| EventType::ConnectivityChanged
| EventType::SelfavatarChanged
| EventType::WebxdcStatusUpdate(_)
| EventType::ChatEphemeralTimerModified { .. } => ptr::null_mut(),
EventType::ConfigureProgress { comment, .. } => {
if let Some(comment) = comment {
@@ -689,8 +642,8 @@ pub unsafe extern "C" fn dc_preconfigure_keypair(
let ctx = &*context;
block_on(async move {
let addr = dc_tools::EmailAddress::new(&to_string_lossy(addr))?;
let public = key::SignedPublicKey::from_asc(&to_string_lossy(public_data))?.0;
let secret = key::SignedSecretKey::from_asc(&to_string_lossy(secret_data))?.0;
let public = key::SignedPublicKey::from_base64(&to_string_lossy(public_data))?;
let secret = key::SignedSecretKey::from_base64(&to_string_lossy(secret_data))?;
let keypair = key::KeyPair {
addr,
public,
@@ -717,11 +670,7 @@ pub unsafe extern "C" fn dc_get_chatlist(
let ctx = &*context;
let qs = to_opt_string_lossy(query_str);
let qi = if query_id == 0 {
None
} else {
Some(ContactId::new(query_id))
};
let qi = if query_id == 0 { None } else { Some(query_id) };
block_on(async move {
match chatlist::Chatlist::try_load(ctx, flags as usize, qs.as_deref(), qi)
@@ -749,7 +698,7 @@ pub unsafe extern "C" fn dc_create_chat_by_contact_id(
let ctx = &*context;
block_on(async move {
ChatId::create_for_contact(ctx, ContactId::new(contact_id))
ChatId::create_for_contact(ctx, contact_id)
.await
.log_err(ctx, "Failed to create chat from contact_id")
.map(|id| id.to_u32())
@@ -769,7 +718,7 @@ pub unsafe extern "C" fn dc_get_chat_id_by_contact_id(
let ctx = &*context;
block_on(async move {
ChatId::lookup_by_contact(ctx, ContactId::new(contact_id))
ChatId::lookup_by_contact(ctx, contact_id)
.await
.log_err(ctx, "Failed to get chat for contact_id")
.unwrap_or_default() // unwraps the Result
@@ -881,48 +830,6 @@ pub unsafe extern "C" fn dc_send_videochat_invitation(
})
}
#[no_mangle]
pub unsafe extern "C" fn dc_send_webxdc_status_update(
context: *mut dc_context_t,
msg_id: u32,
json: *const libc::c_char,
descr: *const libc::c_char,
) -> libc::c_int {
if context.is_null() {
eprintln!("ignoring careless call to dc_send_webxdc_status_update()");
return 0;
}
let ctx = &*context;
block_on(ctx.send_webxdc_status_update(
MsgId::new(msg_id),
&to_string_lossy(json),
&to_string_lossy(descr),
))
.log_err(ctx, "Failed to send webxdc update")
.is_ok() as libc::c_int
}
#[no_mangle]
pub unsafe extern "C" fn dc_get_webxdc_status_updates(
context: *mut dc_context_t,
msg_id: u32,
last_known_serial: u32,
) -> *mut libc::c_char {
if context.is_null() {
eprintln!("ignoring careless call to dc_get_webxdc_status_updates()");
return "".strdup();
}
let ctx = &*context;
block_on(ctx.get_webxdc_status_updates(
MsgId::new(msg_id),
StatusUpdateSerial::new(last_known_serial),
))
.unwrap_or_else(|_| "".to_string())
.strdup()
}
#[no_mangle]
pub unsafe extern "C" fn dc_set_draft(
context: *mut dc_context_t,
@@ -1346,10 +1253,7 @@ pub unsafe extern "C" fn dc_get_chat_contacts(
let arr = dc_array_t::from(
chat::get_chat_contacts(ctx, ChatId::new(chat_id))
.await
.unwrap_or_log_default(ctx, "Failed get_chat_contacts")
.iter()
.map(|id| id.to_u32())
.collect::<Vec<u32>>(),
.unwrap_or_log_default(ctx, "Failed get_chat_contacts"),
);
Box::into_raw(Box::new(arr))
})
@@ -1459,7 +1363,7 @@ pub unsafe extern "C" fn dc_is_contact_in_chat(
block_on(chat::is_contact_in_chat(
ctx,
ChatId::new(chat_id),
ContactId::new(contact_id),
contact_id,
))
.log_err(ctx, "is_contact_in_chat failed")
.unwrap_or_default() as libc::c_int
@@ -1480,7 +1384,7 @@ pub unsafe extern "C" fn dc_add_contact_to_chat(
block_on(chat::add_contact_to_chat(
ctx,
ChatId::new(chat_id),
ContactId::new(contact_id),
contact_id,
))
.log_err(ctx, "Failed to add contact")
.is_ok() as libc::c_int
@@ -1501,7 +1405,7 @@ pub unsafe extern "C" fn dc_remove_contact_from_chat(
block_on(chat::remove_contact_from_chat(
ctx,
ChatId::new(chat_id),
ContactId::new(contact_id),
contact_id,
))
.log_err(ctx, "Failed to remove contact")
.is_ok() as libc::c_int
@@ -1833,11 +1737,10 @@ pub unsafe extern "C" fn dc_lookup_contact_id_by_addr(
let ctx = &*context;
block_on(async move {
Contact::lookup_id_by_addr(ctx, &to_string_lossy(addr), Origin::IncomingReplyTo)
Contact::lookup_id_by_addr(ctx, to_string_lossy(addr), Origin::IncomingReplyTo)
.await
.unwrap_or_log_default(ctx, "failed to lookup id")
.map(|id| id.to_u32())
.unwrap_or_default()
.unwrap_or(0)
})
}
@@ -1857,7 +1760,6 @@ pub unsafe extern "C" fn dc_create_contact(
block_on(async move {
Contact::create(ctx, &name, &to_string_lossy(addr))
.await
.map(|id| id.to_u32())
.unwrap_or(0)
})
}
@@ -1896,9 +1798,7 @@ pub unsafe extern "C" fn dc_get_contacts(
block_on(async move {
match Contact::get_all(ctx, flags, query).await {
Ok(contacts) => Box::into_raw(Box::new(dc_array_t::from(
contacts.iter().map(|id| id.to_u32()).collect::<Vec<u32>>(),
))),
Ok(contacts) => Box::into_raw(Box::new(dc_array_t::from(contacts))),
Err(_) => ptr::null_mut(),
}
})
@@ -1935,10 +1835,7 @@ pub unsafe extern "C" fn dc_get_blocked_contacts(
Contact::get_all_blocked(ctx)
.await
.log_err(ctx, "Can't get blocked contacts")
.unwrap_or_default()
.iter()
.map(|id| id.to_u32())
.collect::<Vec<u32>>(),
.unwrap_or_default(),
)))
})
}
@@ -1949,18 +1846,18 @@ pub unsafe extern "C" fn dc_block_contact(
contact_id: u32,
block: libc::c_int,
) {
if context.is_null() || contact_id <= constants::DC_CONTACT_ID_LAST_SPECIAL.to_u32() {
if context.is_null() || contact_id <= constants::DC_CONTACT_ID_LAST_SPECIAL as u32 {
eprintln!("ignoring careless call to dc_block_contact()");
return;
}
let ctx = &*context;
block_on(async move {
if block == 0 {
Contact::unblock(ctx, ContactId::new(contact_id))
Contact::unblock(ctx, contact_id)
.await
.ok_or_log_msg(ctx, "Can't unblock contact");
} else {
Contact::block(ctx, ContactId::new(contact_id))
Contact::block(ctx, contact_id)
.await
.ok_or_log_msg(ctx, "Can't block contact");
}
@@ -1979,7 +1876,7 @@ pub unsafe extern "C" fn dc_get_contact_encrinfo(
let ctx = &*context;
block_on(async move {
Contact::get_encrinfo(ctx, ContactId::new(contact_id))
Contact::get_encrinfo(ctx, contact_id)
.await
.map(|s| s.strdup())
.unwrap_or_else(|e| {
@@ -1994,14 +1891,14 @@ pub unsafe extern "C" fn dc_delete_contact(
context: *mut dc_context_t,
contact_id: u32,
) -> libc::c_int {
if context.is_null() || contact_id <= constants::DC_CONTACT_ID_LAST_SPECIAL.to_u32() {
if context.is_null() || contact_id <= constants::DC_CONTACT_ID_LAST_SPECIAL as u32 {
eprintln!("ignoring careless call to dc_delete_contact()");
return 0;
}
let ctx = &*context;
block_on(async move {
match Contact::delete(ctx, ContactId::new(contact_id)).await {
match Contact::delete(ctx, contact_id).await {
Ok(_) => 1,
Err(_) => 0,
}
@@ -2020,7 +1917,7 @@ pub unsafe extern "C" fn dc_get_contact(
let ctx = &*context;
block_on(async move {
Contact::get_by_id(ctx, ContactId::new(contact_id))
Contact::get_by_id(ctx, contact_id)
.await
.map(|contact| Box::into_raw(Box::new(ContactWrapper { context, contact })))
.unwrap_or_else(|_| ptr::null_mut())
@@ -2032,7 +1929,7 @@ pub unsafe extern "C" fn dc_imex(
context: *mut dc_context_t,
what_raw: libc::c_int,
param1: *const libc::c_char,
param2: *const libc::c_char,
_param2: *const libc::c_char,
) {
if context.is_null() {
eprintln!("ignoring careless call to dc_imex()");
@@ -2045,13 +1942,12 @@ pub unsafe extern "C" fn dc_imex(
return;
}
};
let passphrase = to_opt_string_lossy(param2);
let ctx = &*context;
if let Some(param1) = to_opt_string_lossy(param1) {
spawn(async move {
imex::imex(ctx, what, param1.as_ref(), passphrase)
imex::imex(ctx, what, param1.as_ref())
.await
.log_err(ctx, "IMEX failed")
});
@@ -2443,7 +2339,7 @@ pub unsafe extern "C" fn dc_array_get_contact_id(
return 0;
}
(*array).get_location(index).contact_id.to_u32()
(*array).get_location(index).contact_id
}
#[no_mangle]
pub unsafe extern "C" fn dc_array_get_msg_id(
@@ -2557,14 +2453,7 @@ pub unsafe extern "C" fn dc_chatlist_get_chat_id(
return 0;
}
let ffi_list = &*chatlist;
let ctx = &*ffi_list.context;
match ffi_list.list.get_chat_id(index as usize) {
Ok(chat_id) => chat_id.to_u32(),
Err(err) => {
warn!(ctx, "get_chat_id failed: {}", err);
0
}
}
ffi_list.list.get_chat_id(index as usize).to_u32()
}
#[no_mangle]
@@ -2959,7 +2848,7 @@ pub unsafe extern "C" fn dc_msg_get_from_id(msg: *mut dc_msg_t) -> u32 {
return 0;
}
let ffi_msg = &*msg;
ffi_msg.message.get_from_id().to_u32()
ffi_msg.message.get_from_id()
}
#[no_mangle]
@@ -3081,61 +2970,6 @@ pub unsafe extern "C" fn dc_msg_get_filename(msg: *mut dc_msg_t) -> *mut libc::c
ffi_msg.message.get_filename().unwrap_or_default().strdup()
}
#[no_mangle]
pub unsafe extern "C" fn dc_msg_get_webxdc_blob(
msg: *mut dc_msg_t,
filename: *const libc::c_char,
ret_bytes: *mut libc::size_t,
) -> *mut libc::c_char {
if msg.is_null() || filename.is_null() || ret_bytes.is_null() {
eprintln!("ignoring careless call to dc_msg_get_webxdc_blob()");
return ptr::null_mut();
}
let ffi_msg = &*msg;
let ctx = &*ffi_msg.context;
let blob = block_on(async move {
ffi_msg
.message
.get_webxdc_blob(ctx, &to_string_lossy(filename))
.await
});
match blob {
Ok(blob) => {
*ret_bytes = blob.len();
let ptr = libc::malloc(*ret_bytes);
libc::memcpy(ptr, blob.as_ptr() as *mut libc::c_void, *ret_bytes);
ptr as *mut libc::c_char
}
Err(err) => {
eprintln!("failed read blob from archive: {}", err);
ptr::null_mut()
}
}
}
#[no_mangle]
pub unsafe extern "C" fn dc_msg_get_webxdc_info(msg: *mut dc_msg_t) -> *mut libc::c_char {
if msg.is_null() {
eprintln!("ignoring careless call to dc_msg_get_webxdc_info()");
return "".strdup();
}
let ffi_msg = &*msg;
let ctx = &*ffi_msg.context;
block_on(async move {
let info = match ffi_msg.message.get_webxdc_info(ctx).await {
Ok(info) => info,
Err(err) => {
error!(ctx, "dc_msg_get_webxdc_info() failed to get info: {}", err);
return "".strdup();
}
};
serde_json::to_string(&info)
.unwrap_or_log_default(ctx, "dc_msg_get_webxdc_info() failed to serialise to json")
.strdup()
})
}
#[no_mangle]
pub unsafe extern "C" fn dc_msg_get_filemime(msg: *mut dc_msg_t) -> *mut libc::c_char {
if msg.is_null() {
@@ -3548,21 +3382,17 @@ pub unsafe extern "C" fn dc_msg_set_quote(msg: *mut dc_msg_t, quote: *const dc_m
return;
}
let ffi_msg = &mut *msg;
let quote_msg = if quote.is_null() {
None
} else {
let ffi_quote = &*quote;
if ffi_msg.context != ffi_quote.context {
eprintln!("ignoring attempt to quote message from a different context");
return;
}
Some(&ffi_quote.message)
};
let ffi_quote = &*quote;
if ffi_msg.context != ffi_quote.context {
eprintln!("ignoring attempt to quote message from a different context");
return;
}
block_on(async move {
ffi_msg
.message
.set_quote(&*ffi_msg.context, quote_msg)
.set_quote(&*ffi_msg.context, &ffi_quote.message)
.await
.log_err(&*ffi_msg.context, "failed to set quote")
.ok();
@@ -3605,39 +3435,6 @@ pub unsafe extern "C" fn dc_msg_get_quoted_msg(msg: *const dc_msg_t) -> *mut dc_
}
}
#[no_mangle]
pub unsafe extern "C" fn dc_msg_get_parent(msg: *const dc_msg_t) -> *mut dc_msg_t {
if msg.is_null() {
eprintln!("ignoring careless call to dc_msg_get_parent()");
return ptr::null_mut();
}
let ffi_msg: &MessageWrapper = &*msg;
let context = &*ffi_msg.context;
let res = block_on(async move {
ffi_msg
.message
.parent(context)
.await
.log_err(context, "failed to get parent message")
.unwrap_or(None)
});
match res {
Some(message) => Box::into_raw(Box::new(MessageWrapper { context, message })),
None => ptr::null_mut(),
}
}
#[no_mangle]
pub unsafe extern "C" fn dc_msg_force_plaintext(msg: *mut dc_msg_t) {
if msg.is_null() {
eprintln!("ignoring careless call to dc_msg_force_plaintext()");
return;
}
let ffi_msg = &mut *msg;
ffi_msg.message.force_plaintext();
}
// dc_contact_t
/// FFI struct for [dc_contact_t]
@@ -3670,7 +3467,7 @@ pub unsafe extern "C" fn dc_contact_get_id(contact: *mut dc_contact_t) -> u32 {
return 0;
}
let ffi_contact = &*contact;
ffi_contact.contact.get_id().to_u32()
ffi_contact.contact.get_id()
}
#[no_mangle]
@@ -3956,11 +3753,7 @@ pub unsafe extern "C" fn dc_provider_new_from_email(
match socks5_enabled {
Ok(socks5_enabled) => {
match block_on(provider::get_provider_info(
ctx,
addr.as_str(),
socks5_enabled,
)) {
match block_on(provider::get_provider_info(addr.as_str(), socks5_enabled)) {
Some(provider) => provider,
None => ptr::null_mut(),
}
@@ -4042,7 +3835,7 @@ pub type dc_accounts_t = AccountsWrapper;
#[no_mangle]
pub unsafe extern "C" fn dc_accounts_new(
_os_name: *const libc::c_char,
os_name: *const libc::c_char,
dbfile: *const libc::c_char,
) -> *mut dc_accounts_t {
setup_panic!();
@@ -4052,7 +3845,13 @@ pub unsafe extern "C" fn dc_accounts_new(
return ptr::null_mut();
}
let accs = block_on(Accounts::new(as_path(dbfile).to_path_buf().into()));
let os_name = if os_name.is_null() {
String::from("DcFFI")
} else {
to_string_lossy(os_name)
};
let accs = block_on(Accounts::new(os_name, as_path(dbfile).to_path_buf().into()));
match accs {
Ok(accs) => Box::into_raw(Box::new(AccountsWrapper::new(accs))),
@@ -4157,30 +3956,6 @@ pub unsafe extern "C" fn dc_accounts_add_account(accounts: *mut dc_accounts_t) -
})
}
#[no_mangle]
pub unsafe extern "C" fn dc_accounts_add_closed_account(accounts: *mut dc_accounts_t) -> u32 {
if accounts.is_null() {
eprintln!("ignoring careless call to dc_accounts_add_closed_account()");
return 0;
}
let accounts = &mut *accounts;
block_on(async move {
let mut accounts = accounts.write().await;
match accounts.add_closed_account().await {
Ok(id) => id,
Err(err) => {
accounts.emit_event(EventType::Error(format!(
"Failed to add account: {:#}",
err
)));
0
}
}
})
}
#[no_mangle]
pub unsafe extern "C" fn dc_accounts_remove_account(
accounts: *mut dc_accounts_t,

View File

@@ -111,19 +111,19 @@ impl Lot {
match self {
Self::Summary(_) => Default::default(),
Self::Qr(qr) => match qr {
Qr::AskVerifyContact { contact_id, .. } => contact_id.to_u32(),
Qr::AskVerifyContact { contact_id, .. } => *contact_id,
Qr::AskVerifyGroup { .. } => Default::default(),
Qr::FprOk { contact_id } => contact_id.to_u32(),
Qr::FprMismatch { contact_id } => contact_id.unwrap_or_default().to_u32(),
Qr::FprOk { contact_id } => *contact_id,
Qr::FprMismatch { contact_id } => contact_id.unwrap_or_default(),
Qr::FprWithoutAddr { .. } => Default::default(),
Qr::Account { .. } => Default::default(),
Qr::WebrtcInstance { .. } => Default::default(),
Qr::Addr { contact_id } => contact_id.to_u32(),
Qr::Addr { contact_id } => *contact_id,
Qr::Url { .. } => Default::default(),
Qr::Text { .. } => Default::default(),
Qr::WithdrawVerifyContact { contact_id, .. } => contact_id.to_u32(),
Qr::WithdrawVerifyContact { contact_id, .. } => *contact_id,
Qr::WithdrawVerifyGroup { .. } => Default::default(),
Qr::ReviveVerifyContact { contact_id, .. } => contact_id.to_u32(),
Qr::ReviveVerifyContact { contact_id, .. } => *contact_id,
Qr::ReviveVerifyGroup { .. } => Default::default(),
},
Self::Error(_) => Default::default(),

View File

@@ -1,181 +0,0 @@
# Webxdc Developer Reference
## Webxdc File Format
- a **Webxdc app** is a **ZIP-file** with the extension `.xdc`
- the ZIP-file must use the default compression methods as of RFC 1950,
this is "Deflate" or "Store"
- the ZIP-file must contain at least the file `index.html`
- if the Webxdc app is started, `index.html` is opened in a restricted webview
that allow accessing resources only from the ZIP-file
## Webxdc API
There are some additional APIs available once `webxdc.js` is included
(the file will be provided by the concrete implementations,
no need to add `webxdc.js` to your ZIP-file):
```html
<script src="webxdc.js"></script>
```
### sendUpdate()
```js
window.webxdc.sendUpdate(update, descr);
```
Webxdc apps are usually shared in a chat and run independently on each peer.
To get a shared state, the peers use `sendUpdate()` to send updates to each other.
- `update`: an object with the following properties:
- `update.payload`: any javascript primitive, array or object.
- `update.info`: optional, short, informational message that will be added to the chat,
eg. "Alice voted" or "Bob scored 123 in MyGame";
usually only one line of text is shown,
use this option sparingly to not spam the chat.
- `update.summary`: optional, short text, shown beside app icon;
it is recommended to use some aggregated value, eg. "8 votes", "Highscore: 123"
- `descr`: short, human-readable description what this update is about.
this is shown eg. as a fallback text in an email program.
All peers, including the sending one,
will receive the update by the callback given to `setUpdateListener()`.
There are situations where the user cannot send messages to a chat,
eg. if the webxdc instance comes as a contact request or if the user has left a group.
In these cases, you can still call `sendUpdate()`,
however, the update won't be sent to other peers
and you won't get the update by `setUpdateListener()`.
### setUpdateListener()
```js
window.webxdc.setUpdateListener((update) => {}, serial);
```
With `setUpdateListener()` you define a callback that receives the updates
sent by `sendUpdate()`. The callback is called for updates sent by you or other peers.
The `serial` specifies the last serial that you know about (defaults to 0).
Each `update` which is passed to the callback comes with the following properties:
- `update.payload`: equals the payload given to `sendUpdate()`
- `update.serial`: the serial number of this update.
Serials are larger `0` and newer serials have higher numbers.
There may be gaps in the serials
and it is not guaranteed that the next serial is exactly incremented by one.
- `update.max_serial`: the maximum serial currently known.
If `max_serial` equals `serial` this update is the last update (until new network messages arrive).
- `update.info`: optional, short, informational message (see `send_update`)
- `update.summary`: optional, short text, shown beside app icon (see `send_update`)
### selfAddr
```js
window.webxdc.selfAddr
```
Property with the peer's own address.
This is esp. useful if you want to differ between different peers -
just send the address along with the payload,
and, if needed, compare the payload addresses against selfAddr() later on.
### selfName
```js
window.webxdc.selfName
```
Property with the peer's own name.
This is name chosen by the user in their settings,
if there is nothing set, that defaults to the peer's address.
## manifest.toml
If the ZIP-file contains a `manifest.toml` in its root directory,
some basic information are read and used from there.
the `manifest.toml` has the following format
```toml
name = "My App Name"
```
- **name** - The name of the app.
If no name is set or if there is no manifest, the filename is used as the app name.
## App Icon
If the ZIP-root contains an `icon.png` or `icon.jpg`,
these files are used as the icon for the app.
The icon should be a square at reasonable width/height;
round corners etc. will be added by the implementations as needed.
If no icon is set, a default icon will be used.
## Webxdc Examples
The following example shows an input field and every input is show on all peers.
```html
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8"/>
<script src="webxdc.js"></script>
</head>
<body>
<input id="input" type="text"/>
<a href="" onclick="sendMsg(); return false;">Send</a>
<p id="output"></p>
<script>
function sendMsg() {
msg = document.getElementById("input").value;
window.webxdc.sendUpdate({payload: msg}, 'Someone typed "'+msg+'".');
}
function receiveUpdate(update) {
document.getElementById('output').innerHTML += update.payload + "<br>";
}
window.webxdc.setUpdateListener(receiveUpdate, 0);
</script>
</body>
</html>
```
[Webxdc Development Tool](https://github.com/deltachat/webxdc-dev)
offers an **Webxdc Simulator** that can be used in many browsers without any installation needed.
You can also use that repository as a template for your own app -
just clone and start adapting things to your need.
### Advanced Examples
- [2048](https://github.com/adbenitez/2048.xdc)
- [Draw](https://github.com/adbenitez/draw.xdc)
- [Poll](https://github.com/r10s/webxdc-poll/)
- [Tic Tac Toe](https://github.com/Simon-Laux/tictactoe.xdc)
- Even more with [Topic #webxdc on Github](https://github.com/topics/webxdc)
## Closing Remarks
- older devices might not have the newest js features in their webview,
you may want to transpile your code down to an older js version eg. with https://babeljs.io
- there are tons of ideas for enhancements of the API and the file format,
eg. in the future, we will may define icon- and manifest-files,
allow to aggregate the state or add metadata.

View File

@@ -17,7 +17,7 @@ use deltachat::download::DownloadState;
use deltachat::imex::*;
use deltachat::location;
use deltachat::log::LogExt;
use deltachat::message::{self, Message, MessageState, MsgId, Viewtype};
use deltachat::message::{self, Message, MessageState, MsgId};
use deltachat::peerstate::*;
use deltachat::qr::*;
use deltachat::sql;
@@ -84,7 +84,6 @@ async fn reset_tables(context: &Context, bits: i32) {
)
.await
.unwrap();
context.sql().config_cache().write().await.clear();
context
.sql()
.execute("DELETE FROM leftgrps;", paramsv![])
@@ -102,7 +101,7 @@ async fn reset_tables(context: &Context, bits: i32) {
async fn poke_eml_file(context: &Context, filename: impl AsRef<Path>) -> Result<()> {
let data = dc_read_file(context, filename).await?;
if let Err(err) = dc_receive_imf(context, &data, "import", false).await {
if let Err(err) = dc_receive_imf(context, &data, "import", 0, false).await {
println!("dc_receive_imf errored: {:?}", err);
}
Ok(())
@@ -210,7 +209,7 @@ async fn log_msg(context: &Context, prefix: impl AsRef<str>, msg: &Message) {
contact_id,
msgtext.unwrap_or_default(),
if msg.has_html() { "[HAS-HTML]" } else { "" },
if msg.get_from_id() == DC_CONTACT_ID_SELF {
if msg.get_from_id() == 1 {
""
} else if msg.get_state() == MessageState::InSeen {
"[SEEN]"
@@ -268,8 +267,9 @@ async fn log_msglist(context: &Context, msglist: &[MsgId]) -> Result<()> {
Ok(())
}
async fn log_contactlist(context: &Context, contacts: &[ContactId]) -> Result<()> {
async fn log_contactlist(context: &Context, contacts: &[u32]) -> Result<()> {
for contact_id in contacts {
let line;
let mut line2 = "".to_string();
let contact = Contact::get_by_id(context, *contact_id).await?;
let name = contact.get_display_name();
@@ -284,20 +284,24 @@ async fn log_contactlist(context: &Context, contacts: &[ContactId]) -> Result<()
} else {
""
};
let line = format!(
line = format!(
"{}{} <{}>",
if !name.is_empty() {
name
&name
} else {
"<name unset>"
},
verified_str,
if !addr.is_empty() { addr } else { "addr unset" }
if !addr.is_empty() {
&addr
} else {
"addr unset"
}
);
let peerstate = Peerstate::from_addr(context, addr)
let peerstate = Peerstate::from_addr(context, &addr)
.await
.expect("peerstate error");
if peerstate.is_some() && *contact_id != DC_CONTACT_ID_SELF {
if peerstate.is_some() && *contact_id != 1 {
line2 = format!(
", prefer-encrypt={}",
peerstate.as_ref().unwrap().prefer_encrypt
@@ -383,7 +387,6 @@ pub async fn cmdline(context: Context, line: &str, chat_id: &mut ChatId) -> Resu
sendfile <file> [<text>]\n\
sendhtml <file for html-part> [<text for plain-part>]\n\
sendsyncmsg\n\
sendupdate <msg-id> <json status update>\n\
videochat\n\
draft [<text>]\n\
devicemsg <text>\n\
@@ -426,6 +429,7 @@ pub async fn cmdline(context: Context, line: &str, chat_id: &mut ChatId) -> Resu
joinqr <qr-content>\n\
setqr <qr-content>\n\
providerinfo <addr>\n\
event <event-id to test>\n\
fileinfo <file>\n\
estimatedeletion <seconds>\n\
clear -- clear screen\n\
@@ -467,32 +471,20 @@ pub async fn cmdline(context: Context, line: &str, chat_id: &mut ChatId) -> Resu
}
"export-backup" => {
let dir = dirs::home_dir().unwrap_or_default();
imex(
&context,
ImexMode::ExportBackup,
dir.as_ref(),
Some(arg2.to_string()),
)
.await?;
imex(&context, ImexMode::ExportBackup, dir.as_ref()).await?;
println!("Exported to {}.", dir.to_string_lossy());
}
"import-backup" => {
ensure!(!arg1.is_empty(), "Argument <backup-file> missing.");
imex(
&context,
ImexMode::ImportBackup,
arg1.as_ref(),
Some(arg2.to_string()),
)
.await?;
imex(&context, ImexMode::ImportBackup, arg1.as_ref()).await?;
}
"export-keys" => {
let dir = dirs::home_dir().unwrap_or_default();
imex(&context, ImexMode::ExportSelfKeys, dir.as_ref(), None).await?;
imex(&context, ImexMode::ExportSelfKeys, dir.as_ref()).await?;
println!("Exported to {}.", dir.to_string_lossy());
}
"import-keys" => {
imex(&context, ImexMode::ImportSelfKeys, arg1.as_ref(), None).await?;
imex(&context, ImexMode::ImportSelfKeys, arg1.as_ref()).await?;
}
"export-setup" => {
let setup_code = create_setup_code(&context);
@@ -571,7 +563,7 @@ pub async fn cmdline(context: Context, line: &str, chat_id: &mut ChatId) -> Resu
);
for i in (0..cnt).rev() {
let chat = Chat::load_from_db(&context, chatlist.get_chat_id(i)?).await?;
let chat = Chat::load_from_db(&context, chatlist.get_chat_id(i)).await?;
println!(
"{}#{}: {} [{} fresh] {}{}{}{}",
chat_prefix(&chat),
@@ -714,7 +706,7 @@ pub async fn cmdline(context: Context, line: &str, chat_id: &mut ChatId) -> Resu
}
"createchat" => {
ensure!(!arg1.is_empty(), "Argument <contact-id> missing.");
let contact_id = ContactId::new(arg1.parse()?);
let contact_id: u32 = arg1.parse()?;
let chat_id = ChatId::create_for_contact(&context, contact_id).await?;
println!("Single#{} created successfully.", chat_id,);
@@ -742,7 +734,7 @@ pub async fn cmdline(context: Context, line: &str, chat_id: &mut ChatId) -> Resu
ensure!(sel_chat.is_some(), "No chat selected");
ensure!(!arg1.is_empty(), "Argument <contact-id> missing.");
let contact_id_0 = ContactId::new(arg1.parse()?);
let contact_id_0: u32 = arg1.parse()?;
chat::add_contact_to_chat(&context, sel_chat.as_ref().unwrap().get_id(), contact_id_0)
.await?;
println!("Contact added to chat.");
@@ -750,7 +742,7 @@ pub async fn cmdline(context: Context, line: &str, chat_id: &mut ChatId) -> Resu
"removemember" => {
ensure!(sel_chat.is_some(), "No chat selected.");
ensure!(!arg1.is_empty(), "Argument <contact-id> missing.");
let contact_id_1 = ContactId::new(arg1.parse()?);
let contact_id_1: u32 = arg1.parse()?;
chat::remove_contact_from_chat(
&context,
sel_chat.as_ref().unwrap().get_id(),
@@ -766,7 +758,7 @@ pub async fn cmdline(context: Context, line: &str, chat_id: &mut ChatId) -> Resu
chat::set_chat_name(
&context,
sel_chat.as_ref().unwrap().get_id(),
format!("{} {}", arg1, arg2).trim(),
&format!("{} {}", arg1, arg2).trim(),
)
.await?;
@@ -915,16 +907,6 @@ pub async fn cmdline(context: Context, line: &str, chat_id: &mut ChatId) -> Resu
Some(msg_id) => println!("sync message sent as {}.", msg_id),
None => println!("sync message not needed."),
},
"sendupdate" => {
ensure!(
!arg1.is_empty() && !arg2.is_empty(),
"Arguments <msg-id> <json status update> expected"
);
let msg_id = MsgId::new(arg1.parse()?);
context
.send_webxdc_status_update(msg_id, arg2, "this is a webxdc status update")
.await?;
}
"videochat" => {
ensure!(sel_chat.is_some(), "No chat selected.");
chat::send_videochat_invitation(&context, sel_chat.as_ref().unwrap().get_id()).await?;
@@ -1134,7 +1116,7 @@ pub async fn cmdline(context: Context, line: &str, chat_id: &mut ChatId) -> Resu
"contactinfo" => {
ensure!(!arg1.is_empty(), "Argument <contact-id> missing.");
let contact_id = ContactId::new(arg1.parse()?);
let contact_id: u32 = arg1.parse()?;
let contact = Contact::get_by_id(&context, contact_id).await?;
let name_n_addr = contact.get_name_n_addr();
@@ -1160,7 +1142,7 @@ pub async fn cmdline(context: Context, line: &str, chat_id: &mut ChatId) -> Resu
if 0 != i {
res += ", ";
}
let chat = Chat::load_from_db(&context, chatlist.get_chat_id(i)?).await?;
let chat = Chat::load_from_db(&context, chatlist.get_chat_id(i)).await?;
res += &format!("{}#{}", chat_prefix(&chat), chat.get_id());
}
}
@@ -1169,16 +1151,16 @@ pub async fn cmdline(context: Context, line: &str, chat_id: &mut ChatId) -> Resu
}
"delcontact" => {
ensure!(!arg1.is_empty(), "Argument <contact-id> missing.");
Contact::delete(&context, ContactId::new(arg1.parse()?)).await?;
Contact::delete(&context, arg1.parse()?).await?;
}
"block" => {
ensure!(!arg1.is_empty(), "Argument <contact-id> missing.");
let contact_id = ContactId::new(arg1.parse()?);
let contact_id = arg1.parse()?;
Contact::block(&context, contact_id).await?;
}
"unblock" => {
ensure!(!arg1.is_empty(), "Argument <contact-id> missing.");
let contact_id = ContactId::new(arg1.parse()?);
let contact_id = arg1.parse()?;
Contact::unblock(&context, contact_id).await?;
}
"listblocked" => {
@@ -1203,7 +1185,7 @@ pub async fn cmdline(context: Context, line: &str, chat_id: &mut ChatId) -> Resu
let socks5_enabled = context
.get_config_bool(config::Config::Socks5Enabled)
.await?;
match provider::get_provider_info(&context, arg1, socks5_enabled).await {
match provider::get_provider_info(arg1, socks5_enabled).await {
Some(info) => {
println!("Information for provider belonging to {}:", arg1);
println!("status: {}", info.status as u32);
@@ -1219,6 +1201,17 @@ pub async fn cmdline(context: Context, line: &str, chat_id: &mut ChatId) -> Resu
}
}
}
// TODO: implement this again, unclear how to match this through though, without writing a parser.
// "event" => {
// ensure!(!arg1.is_empty(), "Argument <id> missing.");
// let event = arg1.parse()?;
// let event = EventType::from_u32(event).ok_or(format_err!("EventType::from_u32({})", event))?;
// let r = context.emit_event(event, 0 as libc::uintptr_t, 0 as libc::uintptr_t);
// println!(
// "Sending event {:?}({}), received value {}.",
// event, event as usize, r,
// );
// }
"fileinfo" => {
ensure!(!arg1.is_empty(), "Argument <file> missing.");

View File

@@ -169,7 +169,7 @@ const DB_COMMANDS: [&str; 10] = [
"housekeeping",
];
const CHAT_COMMANDS: [&str; 36] = [
const CHAT_COMMANDS: [&str; 35] = [
"listchats",
"listarchived",
"chat",
@@ -191,7 +191,6 @@ const CHAT_COMMANDS: [&str; 36] = [
"sendfile",
"sendhtml",
"sendsyncmsg",
"sendupdate",
"videochat",
"draft",
"listmedia",
@@ -227,12 +226,13 @@ const CONTACT_COMMANDS: [&str; 9] = [
"unblock",
"listblocked",
];
const MISC_COMMANDS: [&str; 11] = [
const MISC_COMMANDS: [&str; 12] = [
"getqr",
"getqrsvg",
"getbadqr",
"checkqr",
"joinqr",
"event",
"fileinfo",
"clear",
"exit",
@@ -297,7 +297,7 @@ async fn start(args: Vec<String>) -> Result<(), Error> {
println!("Error: Bad arguments, expected [db-name].");
bail!("No db-name specified");
}
let context = Context::new(Path::new(&args[1]).to_path_buf(), 0).await?;
let context = Context::new("CLI".into(), Path::new(&args[1]).to_path_buf(), 0).await?;
let events = context.get_event_emitter();
async_std::task::spawn(async move {
@@ -415,7 +415,7 @@ async fn handle_cmd(
}
"getqr" | "getbadqr" => {
ctx.start_io().await;
let group = arg1.parse::<u32>().ok().map(ChatId::new);
let group = arg1.parse::<u32>().ok().map(|id| ChatId::new(id));
let mut qr = dc_get_securejoin_qr(&ctx, group).await?;
if !qr.is_empty() {
if arg0 == "getbadqr" && qr.len() > 40 {
@@ -432,7 +432,7 @@ async fn handle_cmd(
}
"getqrsvg" => {
ctx.start_io().await;
let group = arg1.parse::<u32>().ok().map(ChatId::new);
let group = arg1.parse::<u32>().ok().map(|id| ChatId::new(id));
let file = dirs::home_dir().unwrap_or_default().join("qr.svg");
match get_securejoin_qr_svg(&ctx, group).await {
Ok(svg) => {

View File

@@ -36,7 +36,7 @@ async fn main() {
let dir = tempdir().unwrap();
let dbfile = dir.path().join("db.sqlite");
log::info!("creating database {:?}", dbfile);
let ctx = Context::new(dbfile.into(), 0)
let ctx = Context::new("FakeOs".into(), dbfile.into(), 0)
.await
.expect("Failed to create context");
let info = ctx.get_info().await;

View File

@@ -75,6 +75,7 @@ def run_cmdline(argv=None, account_plugins=None):
ac.set_config("addr", args.email)
ac.set_config("mail_pw", args.password)
ac.set_config("mvbox_move", "0")
ac.set_config("mvbox_watch", "0")
ac.set_config("sentbox_watch", "0")
ac.set_config("bot", "1")
configtracker = ac.configure()

View File

@@ -179,12 +179,6 @@ class Account(object):
"""
return True if lib.dc_is_configured(self._dc_context) else False
def is_open(self) -> bool:
"""Determine if account is open
:returns True if account is open."""
return True if lib.dc_context_is_open(self._dc_context) else False
def set_avatar(self, img_path: Optional[str]) -> None:
"""Set self avatar.
@@ -409,10 +403,7 @@ class Account(object):
"""
arr = array("i")
for msg in messages:
if isinstance(msg, Message):
arr.append(msg.id)
else:
arr.append(msg)
arr.append(getattr(msg, "id", msg))
msg_ids = ffi.cast("uint32_t*", ffi.from_buffer(arr))
lib.dc_markseen_msgs(self._dc_context, msg_ids, len(messages))

View File

@@ -224,7 +224,9 @@ class DirectImap:
""" (blocking) wait for next idle message from server. """
assert self._idling
self.account.log("imap-direct: calling idle_check")
res = self.conn.idle_check()
res = self.conn.idle_check(timeout=30)
if len(res) == 0:
raise TimeoutError
if terminate:
self.idle_done()
self.account.log("imap-direct: idle_check returned {!r}".format(res))

View File

@@ -40,7 +40,7 @@ class FFIEventLogger:
@account_hookimpl
def ac_log_line(self, message):
t = threading.current_thread()
t = threading.currentThread()
tname = getattr(t, "name", t)
if tname == "MainThread":
tname = "MAIN"
@@ -193,7 +193,7 @@ class EventThread(threading.Thread):
def __init__(self, account) -> None:
self.account = account
super(EventThread, self).__init__(name="events")
self.daemon = True
self.setDaemon(True)
self._marked_for_shutdown = False
self.start()

View File

@@ -225,10 +225,6 @@ class Message(object):
"""Quote setter"""
lib.dc_msg_set_quote(self._dc_msg, quoted_message._dc_msg)
def force_plaintext(self) -> None:
"""Force the message to be sent in plain text."""
lib.dc_msg_force_plaintext(self._dc_msg)
def get_mime_headers(self):
""" return mime-header object for an incoming message.

View File

@@ -241,6 +241,7 @@ def acfactory(pytestconfig, tmpdir, request, session_liveconfig, data):
def make_account(self, path, logid, quiet=False):
ac = Account(path, logging=self._logging)
ac._evtracker = ac.add_account_plugin(FFIEventTracker(ac))
ac._evtracker.set_timeout(30)
ac.addr = ac.get_self_contact().addr
ac.set_config("displayname", logid)
if not quiet:
@@ -302,20 +303,21 @@ def acfactory(pytestconfig, tmpdir, request, session_liveconfig, data):
self._preconfigure_key(ac, configdict['addr'])
return ac, dict(configdict)
def get_online_configuring_account(self, sentbox=False, move=False,
def get_online_configuring_account(self, mvbox=False, sentbox=False, move=False,
pre_generated_key=True, quiet=False, config={}):
ac, configdict = self.get_online_config(
pre_generated_key=pre_generated_key, quiet=quiet)
configdict.update(config)
configdict["mvbox_watch"] = str(int(mvbox))
configdict["mvbox_move"] = str(int(move))
configdict["sentbox_watch"] = str(int(sentbox))
ac.update_config(configdict)
ac._configtracker = ac.configure()
return ac
def get_one_online_account(self, pre_generated_key=True, move=False):
def get_one_online_account(self, pre_generated_key=True, mvbox=False, move=False):
ac1 = self.get_online_configuring_account(
pre_generated_key=pre_generated_key, move=move)
pre_generated_key=pre_generated_key, mvbox=mvbox, move=move)
self.wait_configure_and_start_io([ac1])
return ac1
@@ -334,7 +336,7 @@ def acfactory(pytestconfig, tmpdir, request, session_liveconfig, data):
return accounts
def clone_online_account(self, account, pre_generated_key=True):
""" Clones addr, mail_pw, mvbox_move, sentbox_watch and the
""" Clones addr, mail_pw, mvbox_watch, mvbox_move, sentbox_watch and the
direct_imap object of an online account. This simulates the user setting
up a new device without importing a backup.
@@ -349,6 +351,7 @@ def acfactory(pytestconfig, tmpdir, request, session_liveconfig, data):
ac.update_config(dict(
addr=account.get_config("addr"),
mail_pw=account.get_config("mail_pw"),
mvbox_watch=account.get_config("mvbox_watch"),
mvbox_move=account.get_config("mvbox_move"),
sentbox_watch=account.get_config("sentbox_watch"),
))
@@ -464,7 +467,7 @@ class BotProcess:
# the (unicode) lines available for readers through a queue.
self.stdout_queue = queue.Queue()
self.stdout_thread = t = threading.Thread(target=self._run_stdout_thread, name="bot-stdout-thread")
t.daemon = True
t.setDaemon(True)
t.start()
def _run_stdout_thread(self) -> None:
@@ -482,7 +485,7 @@ class BotProcess:
def kill(self) -> None:
self.popen.kill()
def wait(self, timeout=None) -> None:
def wait(self, timeout=30) -> None:
self.popen.wait(timeout=timeout)
def fnmatch_lines(self, pattern_lines):
@@ -491,7 +494,7 @@ class BotProcess:
print("+++FNMATCH:", next_pattern)
ignored = []
while 1:
line = self.stdout_queue.get()
line = self.stdout_queue.get(timeout=15)
if line is None:
if ignored:
print("BOT stdout terminated after these lines")

View File

@@ -90,11 +90,11 @@ class ConfigureTracker:
if data1 is None or evdata == data1:
break
def wait_finish(self, timeout=None):
def wait_finish(self):
""" wait until configure is completed.
Raise Exception if Configure failed
"""
if not self._configure_events.get(timeout=timeout):
if not self._configure_events.get():
content = "\n".join(map(str, self._ffi_events))
raise ConfigureFailed(content)

View File

@@ -41,8 +41,8 @@ class TestOfflineAccountBasic:
def test_wrong_db(self, tmpdir):
p = tmpdir.join("hello.db")
p.write("123")
account = Account(p.strpath)
assert not account.is_open()
with pytest.raises(ValueError):
Account(p.strpath)
def test_os_name(self, tmpdir):
p = tmpdir.join("hello.db")
@@ -57,7 +57,7 @@ class TestOfflineAccountBasic:
alice_public = data.read_path("key/alice-public.asc")
alice_secret = data.read_path("key/alice-secret.asc")
assert alice_public and alice_secret
ac._preconfigure_keypair("alice@example.org", alice_public, alice_secret)
ac._preconfigure_keypair("alice@example.com", alice_public, alice_secret)
def test_getinfo(self, acfactory):
ac1 = acfactory.get_unconfigured_account()
@@ -652,6 +652,8 @@ class TestOnlineAccount:
pre_generated_key=False,
config={"key_gen_type": str(const.DC_KEY_GEN_ED25519)}
)
# rsa key gen can be slow especially on CI, adjust timeout
ac1._evtracker.set_timeout(240)
acfactory.wait_configure_and_start_io()
chat = acfactory.get_accepted_chat(ac1, ac2)
@@ -735,6 +737,7 @@ class TestOnlineAccount:
# make sure we are not sending message to ourselves
assert self_addr not in ev.data2
assert other_addr in ev.data2
ev = ac1._evtracker.get_matching("DC_EVENT_DELETED_BLOB_FILE")
lp.sec("ac1: setting bcc_self=1")
ac1.set_config("bcc_self", "1")
@@ -750,6 +753,7 @@ class TestOnlineAccount:
# now make sure we are sending message to ourselves too
assert self_addr in ev.data2
assert other_addr in ev.data2
ev = ac1._evtracker.get_matching("DC_EVENT_DELETED_BLOB_FILE")
assert ac1.direct_imap.idle_wait_for_seen()
# Second client receives only second message, but not the first
@@ -856,7 +860,7 @@ class TestOnlineAccount:
def test_mvbox_sentbox_threads(self, acfactory, lp):
lp.sec("ac1: start with mvbox thread")
ac1 = acfactory.get_online_configuring_account(move=True, sentbox=True)
ac1 = acfactory.get_online_configuring_account(mvbox=True, move=True, sentbox=True)
lp.sec("ac2: start without mvbox/sentbox threads")
ac2 = acfactory.get_online_configuring_account()
@@ -870,31 +874,27 @@ class TestOnlineAccount:
def test_move_works(self, acfactory):
ac1 = acfactory.get_online_configuring_account()
ac2 = acfactory.get_online_configuring_account(move=True)
ac2 = acfactory.get_online_configuring_account(mvbox=True, move=True)
acfactory.wait_configure_and_start_io()
chat = acfactory.get_accepted_chat(ac1, ac2)
chat.send_text("message1")
# Message is moved to the movebox
ac2._evtracker.get_matching("DC_EVENT_IMAP_MESSAGE_MOVED")
# Message is downloaded
ev = ac2._evtracker.get_matching("DC_EVENT_INCOMING_MSG")
assert ev.data2 > const.DC_CHAT_ID_LAST_SPECIAL
ac2._evtracker.get_matching("DC_EVENT_IMAP_MESSAGE_MOVED")
def test_move_works_on_self_sent(self, acfactory):
ac1 = acfactory.get_online_configuring_account(move=True)
ac1 = acfactory.get_online_configuring_account(mvbox=True, move=True)
ac2 = acfactory.get_online_configuring_account()
acfactory.wait_configure_and_start_io()
ac1.set_config("bcc_self", "1")
chat = acfactory.get_accepted_chat(ac1, ac2)
chat.send_text("message1")
ac1._evtracker.get_matching("DC_EVENT_IMAP_MESSAGE_MOVED")
chat.send_text("message2")
ac1._evtracker.get_matching("DC_EVENT_IMAP_MESSAGE_MOVED")
chat.send_text("message3")
ac1._evtracker.get_matching("DC_EVENT_IMAP_MESSAGE_MOVED")
ac1._evtracker.get_matching("DC_EVENT_IMAP_MESSAGE_MOVED")
ac1._evtracker.get_matching("DC_EVENT_IMAP_MESSAGE_MOVED")
def test_forward_messages(self, acfactory, lp):
ac1, ac2 = acfactory.get_two_online_accounts()
@@ -953,7 +953,7 @@ class TestOnlineAccount:
assert msg_in.is_forwarded()
def test_send_self_message(self, acfactory, lp):
ac1 = acfactory.get_one_online_account(move=True)
ac1 = acfactory.get_one_online_account(mvbox=True, move=True)
lp.sec("ac1: create self chat")
chat = ac1.get_self_contact().create_chat()
chat.send_text("hello")
@@ -983,8 +983,7 @@ class TestOnlineAccount:
assert msg2 in chat2.get_messages()
assert chat2.is_contact_request()
assert chat2.count_fresh_messages() == 1
# Like it or not, this assert is flaky
# assert msg2.time_received >= msg1.time_sent
assert msg2.time_received >= msg1.time_sent
lp.sec("create new chat with contact and verify it's proper")
chat2b = msg2.create_chat()
@@ -1035,94 +1034,31 @@ class TestOnlineAccount:
def test_moved_markseen(self, acfactory, lp):
"""Test that message already moved to DeltaChat folder is marked as seen."""
ac1 = acfactory.get_online_configuring_account()
ac2 = acfactory.get_online_configuring_account(move=True)
ac1 = acfactory.get_online_configuring_account(mvbox=True, config={"inbox_watch": "0"})
ac2 = acfactory.get_online_configuring_account()
acfactory.wait_configure_and_start_io([ac1, ac2])
ac1.set_config("bcc_self", "1")
ac2.stop_io()
ac2.direct_imap.idle_start()
ac1.direct_imap.idle_start()
ac1.create_chat(ac2).send_text("Hello!")
# Wait for the message to arrive.
ac2.direct_imap.idle_check(terminate=True)
ac1.direct_imap.idle_check(terminate=True)
ac1.stop_io()
# Emulate moving of the message to DeltaChat folder by Sieve rule.
# mailcow server contains this rule by default.
ac2.direct_imap.conn.move(["*"], "DeltaChat")
ac1.direct_imap.conn.move(["*"], "DeltaChat")
ac2.direct_imap.select_folder("DeltaChat")
ac2.direct_imap.idle_start()
ac2.start_io()
msg = ac2._evtracker.wait_next_incoming_message()
ac1.direct_imap.select_folder("DeltaChat")
ac1.direct_imap.idle_start()
ac1.start_io()
ac1.direct_imap.idle_wait_for_seen()
ac1.direct_imap.idle_done()
# Accept the contact request.
msg.chat.accept()
ac2.mark_seen_messages([msg])
ac2.direct_imap.idle_wait_for_seen()
ac2.direct_imap.idle_done()
fetch = list(ac2.direct_imap.conn.fetch("*", b'FLAGS').values())
fetch = list(ac1.direct_imap.conn.fetch("*", b'FLAGS').values())
flags = fetch[-1][b'FLAGS']
is_seen = b'\\Seen' in flags
assert is_seen
def test_multidevice_sync_seen(self, acfactory, lp):
"""Test that message marked as seen on one device is marked as seen on another."""
ac1 = acfactory.get_online_configuring_account()
ac2 = acfactory.get_online_configuring_account()
ac1_clone = acfactory.clone_online_account(ac1)
acfactory.wait_configure_and_start_io()
ac1.set_config("bcc_self", "1")
ac1_clone.set_config("bcc_self", "1")
ac1_chat = ac1.create_chat(ac2)
ac1_clone_chat = ac1_clone.create_chat(ac2)
ac2_chat = ac2.create_chat(ac1)
lp.sec("Send a message from ac2 to ac1 and check that it's 'fresh'")
ac2_chat.send_text("Hi")
ac1_message = ac1._evtracker.wait_next_incoming_message()
ac1_clone_message = ac1_clone._evtracker.wait_next_incoming_message()
assert ac1_chat.count_fresh_messages() == 1
assert ac1_clone_chat.count_fresh_messages() == 1
assert ac1_message.is_in_fresh
assert ac1_clone_message.is_in_fresh
lp.sec("ac1 marks message as seen on the first device")
ac1.mark_seen_messages([ac1_message])
assert ac1_message.is_in_seen
lp.sec("ac1 clone detects that message is marked as seen")
ev = ac1_clone._evtracker.get_matching("DC_EVENT_MSGS_NOTICED")
assert ev.data1 == ac1_clone_chat.id
assert ac1_clone_message.is_in_seen
lp.sec("Send an ephemeral message from ac2 to ac1")
ac2_chat.set_ephemeral_timer(60)
ac1._evtracker.get_matching("DC_EVENT_CHAT_EPHEMERAL_TIMER_MODIFIED")
ac1._evtracker.wait_next_incoming_message()
ac1_clone._evtracker.get_matching("DC_EVENT_CHAT_EPHEMERAL_TIMER_MODIFIED")
ac1_clone._evtracker.wait_next_incoming_message()
ac2_chat.send_text("Foobar")
ac1_message = ac1._evtracker.wait_next_incoming_message()
ac1_clone_message = ac1_clone._evtracker.wait_next_incoming_message()
assert "Ephemeral timer: 60\n" in ac1_message.get_message_info()
assert "Expires: " not in ac1_clone_message.get_message_info()
assert "Ephemeral timer: 60\n" in ac1_message.get_message_info()
assert "Expires: " not in ac1_clone_message.get_message_info()
ac1.mark_seen_messages([ac1_message])
assert ac1_message.is_in_seen
assert "Expires: " in ac1_message.get_message_info()
ev = ac1_clone._evtracker.get_matching("DC_EVENT_MSGS_NOTICED")
assert ev.data1 == ac1_clone_chat.id
assert ac1_clone_message.is_in_seen
# Test that the timer is started on the second device after synchronizing the seen status.
assert "Expires: " in ac1_clone_message.get_message_info()
def test_message_override_sender_name(self, acfactory, lp):
ac1, ac2 = acfactory.get_two_online_accounts()
chat = acfactory.get_accepted_chat(ac1, ac2)
@@ -1159,8 +1095,8 @@ class TestOnlineAccount:
def test_markseen_message_and_mdn(self, acfactory, mvbox_move):
# Please only change this test if you are very sure that it will still catch the issues it catches now.
# We had so many problems with markseen, if in doubt, rather create another test, it can't harm.
ac1 = acfactory.get_online_configuring_account(move=mvbox_move)
ac2 = acfactory.get_online_configuring_account(move=mvbox_move)
ac1 = acfactory.get_online_configuring_account(move=mvbox_move, mvbox=mvbox_move)
ac2 = acfactory.get_online_configuring_account(move=mvbox_move, mvbox=mvbox_move)
acfactory.wait_configure_and_start_io()
# Do not send BCC to self, we only want to test MDN on ac1.
@@ -1206,7 +1142,7 @@ class TestOnlineAccount:
assert not msg_reply1.chat.is_group()
assert msg_reply1.chat.id == private_chat1.id
def test_mdn_asymmetric(self, acfactory, lp):
def test_mdn_asymetric(self, acfactory, lp):
ac1, ac2 = acfactory.get_two_online_accounts(move=True)
lp.sec("ac1: create chat with ac2")
@@ -1230,9 +1166,6 @@ class TestOnlineAccount:
assert len(msg.chat.get_messages()) == 1
ac1.direct_imap.select_config_folder("mvbox")
ac1.direct_imap.idle_start()
lp.sec("ac2: mark incoming message as seen")
ac2.mark_seen_messages([msg])
@@ -1242,9 +1175,6 @@ class TestOnlineAccount:
assert len(chat.get_messages()) == 1
# Wait for the message to be marked as seen on IMAP.
assert ac1.direct_imap.idle_wait_for_seen()
# MDN is received even though MDNs are already disabled
assert msg_out.is_out_mdn_received()
@@ -1422,21 +1352,18 @@ class TestOnlineAccount:
assert not device_chat.can_send()
assert device_chat.get_draft() is None
def test_dont_show_emails(self, acfactory, lp):
def test_dont_show_emails_in_draft_folder(self, acfactory, lp):
"""Most mailboxes have a "Drafts" folder where constantly new emails appear but we don't actually want to show them.
So: If it's outgoing AND there is no Received header AND it's not in the sentbox, then ignore the email.
If the draft email is sent out later (i.e. moved to "Sent"), it must be shown.
Also, test that unknown emails in the Spam folder are not shown."""
If the draft email is sent out later (i.e. moved to "Sent"), it must be shown."""
ac1 = acfactory.get_online_configuring_account()
ac1.set_config("show_emails", "2")
ac1.create_contact("alice@example.org").create_chat()
ac1.create_contact("alice@example.com").create_chat()
acfactory.wait_configure(ac1)
ac1.direct_imap.create_folder("Drafts")
ac1.direct_imap.create_folder("Sent")
ac1.direct_imap.create_folder("Spam")
acfactory.wait_configure_and_start_io()
# Wait until each folder was selected once and we are IDLEing again:
@@ -1446,7 +1373,7 @@ class TestOnlineAccount:
ac1.direct_imap.append("Drafts", """
From: ac1 <{}>
Subject: subj
To: alice@example.org
To: alice@example.com
Message-ID: <aepiors@example.org>
Content-Type: text/plain; charset=utf-8
@@ -1455,21 +1382,12 @@ class TestOnlineAccount:
ac1.direct_imap.append("Sent", """
From: ac1 <{}>
Subject: subj
To: alice@example.org
To: alice@example.com
Message-ID: <hsabaeni@example.org>
Content-Type: text/plain; charset=utf-8
message in Sent
""".format(ac1.get_config("configured_addr")))
ac1.direct_imap.append("Spam", """
From: unknown.address@junk.org
Subject: subj
To: {}
Message-ID: <spam.message@junk.org>
Content-Type: text/plain; charset=utf-8
Unknown message in Spam
""".format(ac1.get_config("configured_addr")))
ac1.set_config("scan_all_folders_debounce_secs", "0")
lp.sec("All prepared, now let DC find the message")
@@ -1483,10 +1401,6 @@ class TestOnlineAccount:
assert msg.text == "subj message in Sent"
assert len(msg.chat.get_messages()) == 1
assert not any("unknown.address" in c.get_name() for c in ac1.get_chats())
ac1.direct_imap.select_folder("Spam")
assert ac1.direct_imap.get_uid_by_message_id("spam.message@junk.org")
ac1.stop_io()
lp.sec("'Send out' the draft, i.e. move it to the Sent folder, and wait for DC to display it this time")
ac1.direct_imap.select_folder("Drafts")
@@ -1499,35 +1413,6 @@ class TestOnlineAccount:
assert msg2.text == "subj message in Drafts that is moved to Sent later"
assert len(msg.chat.get_messages()) == 2
def test_no_old_msg_is_fresh(self, acfactory, lp):
ac1 = acfactory.get_online_configuring_account()
ac2 = acfactory.get_online_configuring_account()
ac1_clone = acfactory.clone_online_account(ac1)
acfactory.wait_configure_and_start_io()
ac1.set_config("e2ee_enabled", "0")
ac1_clone.set_config("e2ee_enabled", "0")
ac2.set_config("e2ee_enabled", "0")
ac1_clone.set_config("bcc_self", "1")
ac1.create_chat(ac2)
ac1_clone.create_chat(ac2)
lp.sec("Send a first message from ac2 to ac1 and check that it's 'fresh'")
first_msg_id = ac2.create_chat(ac1).send_text("Hi")
ac1._evtracker.wait_next_incoming_message()
assert ac1.create_chat(ac2).count_fresh_messages() == 1
assert len(list(ac1.get_fresh_messages())) == 1
lp.sec("Send a message from ac1_clone to ac2 and check that ac1 marks the first message as 'noticed'")
ac1_clone.create_chat(ac2).send_text("Hi back")
ev = ac1._evtracker.get_matching("DC_EVENT_MSGS_NOTICED")
assert ev.data1 == first_msg_id.chat.id
assert ac1.create_chat(ac2).count_fresh_messages() == 0
assert len(list(ac1.get_fresh_messages())) == 0
def test_prefer_encrypt(self, acfactory, lp):
"""Test quorum rule for encryption preference in 1:1 and group chat."""
ac1, ac2, ac3 = acfactory.get_many_online_accounts(3)
@@ -1872,6 +1757,7 @@ class TestOnlineAccount:
lp.sec("trigger ac setup message and return setupcode")
assert ac1.get_info()["fingerprint"] != ac2.get_info()["fingerprint"]
setup_code = ac1.initiate_key_transfer()
ac2._evtracker.set_timeout(30)
ev = ac2._evtracker.get_matching("DC_EVENT_INCOMING_MSG|DC_EVENT_MSGS_CHANGED")
msg = ac2.get_message_by_id(ev.data2)
assert msg.is_setup_message()
@@ -1888,6 +1774,7 @@ class TestOnlineAccount:
def test_ac_setup_message_twice(self, acfactory, lp):
ac1 = acfactory.get_online_configuring_account()
ac2 = acfactory.clone_online_account(ac1)
ac2._evtracker.set_timeout(30)
acfactory.wait_configure_and_start_io()
lp.sec("trigger ac setup message but ignore")
@@ -2061,7 +1948,7 @@ class TestOnlineAccount:
lp.sec("ac1: send a message to group chat to promote the group")
chat.send_text("afterwards promoted")
ev = in_list.get()
ev = in_list.get(timeout=10)
assert ev.action == "chat-modified"
assert chat.is_promoted()
assert sorted(x.addr for x in chat.get_contacts()) == \
@@ -2071,29 +1958,29 @@ class TestOnlineAccount:
# note that if the above create_chat() would not
# happen we would not receive a proper member_added event
contact2 = chat.add_contact("devnull@testrun.org")
ev = in_list.get()
ev = in_list.get(timeout=10)
assert ev.action == "chat-modified"
ev = in_list.get()
ev = in_list.get(timeout=10)
assert ev.action == "chat-modified"
ev = in_list.get()
ev = in_list.get(timeout=10)
assert ev.action == "added"
assert ev.message.get_sender_contact().addr == ac1_addr
assert ev.contact.addr == "devnull@testrun.org"
lp.sec("ac1: remove address2")
chat.remove_contact(contact2)
ev = in_list.get()
ev = in_list.get(timeout=10)
assert ev.action == "chat-modified"
ev = in_list.get()
ev = in_list.get(timeout=10)
assert ev.action == "removed"
assert ev.contact.addr == contact2.addr
assert ev.message.get_sender_contact().addr == ac1_addr
lp.sec("ac1: remove ac2 contact from chat")
chat.remove_contact(ac2)
ev = in_list.get()
ev = in_list.get(timeout=10)
assert ev.action == "chat-modified"
ev = in_list.get()
ev = in_list.get(timeout=10)
assert ev.action == "removed"
assert ev.message.get_sender_contact().addr == ac1_addr
@@ -2190,8 +2077,10 @@ class TestOnlineAccount:
assert msg_back.chat == chat
assert chat.get_profile_image() is None
def test_connectivity(self, acfactory, lp):
@pytest.mark.parametrize("inbox_watch", ["0", "1"])
def test_connectivity(self, acfactory, lp, inbox_watch):
ac1, ac2 = acfactory.get_two_online_accounts()
ac1.set_config("inbox_watch", inbox_watch)
ac1.set_config("scan_all_folders_debounce_secs", "0")
ac1._evtracker.wait_for_connectivity(const.DC_CONNECTIVITY_CONNECTED)
@@ -2431,7 +2320,7 @@ class TestOnlineAccount:
def test_immediate_autodelete(self, acfactory, lp):
ac1 = acfactory.get_online_configuring_account()
ac2 = acfactory.get_online_configuring_account(move=False, sentbox=False)
ac2 = acfactory.get_online_configuring_account(mvbox=False, move=False, sentbox=False)
# "1" means delete immediately, while "0" means do not delete
ac2.set_config("delete_server_after", "1")
@@ -2540,7 +2429,8 @@ class TestOnlineAccount:
lp.sec("ac2: deleting all messages except third")
assert len(to_delete) == len(texts) - 1
ac2.delete_messages(to_delete)
ac2._evtracker.get_matching("DC_EVENT_IMAP_MESSAGE_DELETED")
for msg in to_delete:
ac2._evtracker.get_matching("DC_EVENT_IMAP_MESSAGE_DELETED")
ac2._evtracker.get_info_contains("close/expunge succeeded")
@@ -2691,39 +2581,38 @@ class TestOnlineAccount:
assert received_reply.quoted_text == "hello"
assert received_reply.quote.id == out_msg.id
@pytest.mark.parametrize("folder,move,expected_destination,", [
("xyz", False, "xyz"), # Test that emails are recognized in a random folder but not moved
("xyz", True, "DeltaChat"), # ...emails are found in a random folder and moved to DeltaChat
("Spam", False, "INBOX"), # ...emails are moved from the spam folder to the Inbox
@pytest.mark.parametrize("folder,move,expected_destination,inbox_watch,", [
("xyz", False, "xyz", "1"), # Test that emails are recognized in a random folder but not moved
("xyz", True, "DeltaChat", "1"), # ...emails are found in a random folder and moved to DeltaChat
("Spam", False, "INBOX", "1"), # ...emails are moved from the spam folder to the Inbox
("INBOX", False, "INBOX", "0"), # ...emails are found in the `Inbox` folder even if `inbox_watch` is "0"
])
# Testrun.org does not support the CREATE-SPECIAL-USE capability, which means that we can't create a folder with
# the "\Junk" flag (see https://tools.ietf.org/html/rfc6154). So, we can't test spam folder detection by flag.
def test_scan_folders(self, acfactory, lp, folder, move, expected_destination):
def test_scan_folders(self, acfactory, lp, folder, move, expected_destination, inbox_watch):
"""Delta Chat periodically scans all folders for new messages to make sure we don't miss any."""
variant = folder + "-" + str(move) + "-" + expected_destination
lp.sec("Testing variant " + variant)
ac1 = acfactory.get_online_configuring_account(move=move)
ac2 = acfactory.get_online_configuring_account()
ac1.set_config("inbox_watch", inbox_watch)
acfactory.wait_configure(ac1)
ac1.direct_imap.create_folder(folder)
acfactory.wait_configure_and_start_io()
# Wait until each folder was selected once and we are IDLEing:
ac1._evtracker.get_info_contains("INBOX: Idle entering wait-on-remote state")
if inbox_watch == "1":
ac1._evtracker.get_info_contains("INBOX: Idle entering wait-on-remote state")
else:
ac1._evtracker.get_info_contains("IMAP-fake-IDLE: no folder, waiting for interrupt")
ac1.stop_io()
# Send a message to ac1 and move it to the mvbox:
ac1.direct_imap.select_config_folder("inbox")
ac1.direct_imap.idle_start()
acfactory.get_accepted_chat(ac2, ac1).send_text("hello")
while True:
if len(ac1.direct_imap.idle_check(terminate=True)) > 1:
# If length is 1, it's [(b'OK', b'Still here')]
# Could happen on very slow network.
#
# More is usually [(1, b'EXISTS'), (1, b'RECENT')]
break
ac1.direct_imap.idle_check(terminate=True)
ac1.direct_imap.conn.move(["*"], folder) # "*" means "biggest UID in mailbox"
lp.sec("Everything prepared, now see if DeltaChat finds the message (" + variant + ")")
@@ -2732,7 +2621,11 @@ class TestOnlineAccount:
msg = ac1._evtracker.wait_next_incoming_message()
assert msg.text == "hello"
# The message has been downloaded, which means it has reached its destination.
# Wait until the message was moved (if at all) and we are IDLEing again:
if inbox_watch == "1":
ac1._evtracker.get_info_contains("INBOX: Idle entering wait-on-remote state")
else:
ac1._evtracker.get_info_contains("IMAP-fake-IDLE: no folder, waiting for interrupt")
ac1.direct_imap.select_folder(expected_destination)
assert len(ac1.direct_imap.get_all_messages()) == 1
if folder != expected_destination:
@@ -2755,7 +2648,8 @@ class TestOnlineAccount:
if mvbox_move:
assert ac.get_config("configured_mvbox_folder")
ac1 = acfactory.get_online_configuring_account(move=mvbox_move)
ac1 = acfactory.get_online_configuring_account(mvbox=mvbox_move, move=mvbox_move)
ac1.set_config("sentbox_move", "1")
ac2 = acfactory.get_online_configuring_account()
acfactory.wait_configure(ac1)
@@ -2775,7 +2669,7 @@ class TestOnlineAccount:
if mvbox_move:
ac1.direct_imap.select_config_folder("mvbox")
else:
ac1.direct_imap.select_folder("INBOX")
ac1.direct_imap.select_config_folder("sentbox")
ac1.direct_imap.idle_start()
lp.sec("send out message with bcc to ourselves")
@@ -2854,7 +2748,7 @@ class TestOnlineAccount:
def test_delete_deltachat_folder(self, acfactory):
"""Test that DeltaChat folder is recreated if user deletes it manually."""
ac1 = acfactory.get_online_configuring_account(move=True)
ac1 = acfactory.get_online_configuring_account(mvbox=True)
ac2 = acfactory.get_online_configuring_account()
acfactory.wait_configure(ac1)
@@ -2982,8 +2876,7 @@ class TestGroupStressTests:
lp.sec("ac2: check that ac3 is removed")
msg = ac2._evtracker.wait_next_incoming_message()
assert chat.num_contacts() == 2
assert msg.chat.num_contacts() == 2
assert msg.chat.num_contacts() == chat.num_contacts()
acfactory.dump_imap_summary(sys.stdout)

View File

@@ -36,8 +36,7 @@ def test_wrong_db(tmpdir):
# write an invalid database file
p.write("x123" * 10)
context = lib.dc_context_new(ffi.NULL, p.strpath.encode("ascii"), ffi.NULL)
assert not lib.dc_context_is_open(context)
assert ffi.NULL == lib.dc_context_new(ffi.NULL, p.strpath.encode("ascii"), ffi.NULL)
def test_empty_blobdir(tmpdir):

View File

@@ -78,8 +78,8 @@ commands =
addopts = -v -ra --strict-markers
norecursedirs = .tox
xfail_strict=true
timeout = 150
timeout_func_only = True
timeout = 90
timeout_method = thread
markers =
ignored: ignore this test in default test runs, use --ignored to run.

View File

@@ -1 +1 @@
1.59.0
1.54.0

View File

@@ -8,7 +8,7 @@ set -e -x
#
# Avoid using rustup here as it depends on reading /proc/self/exe and
# has problems running under QEMU.
RUST_VERSION=1.59.0
RUST_VERSION=1.54.0
curl "https://static.rust-lang.org/dist/rust-${RUST_VERSION}-$(uname -m)-unknown-linux-gnu.tar.gz" | tar xz
cd "rust-${RUST_VERSION}-$(uname -m)-unknown-linux-gnu"

View File

@@ -8,7 +8,7 @@ set -e -x
#
# Avoid using rustup here as it depends on reading /proc/self/exe and
# has problems running under QEMU.
RUST_VERSION=1.59.0
RUST_VERSION=1.54.0
curl "https://static.rust-lang.org/dist/rust-${RUST_VERSION}-$(uname -m)-unknown-linux-gnu.tar.gz" | tar xz
cd "rust-${RUST_VERSION}-$(uname -m)-unknown-linux-gnu"

View File

@@ -29,21 +29,21 @@ pub struct Accounts {
impl Accounts {
/// Loads or creates an accounts folder at the given `dir`.
pub async fn new(dir: PathBuf) -> Result<Self> {
pub async fn new(os_name: String, dir: PathBuf) -> Result<Self> {
if !dir.exists().await {
Accounts::create(&dir).await?;
Accounts::create(os_name, &dir).await?;
}
Accounts::open(dir).await
}
/// Creates a new default structure.
pub async fn create(dir: &PathBuf) -> Result<()> {
pub async fn create(os_name: String, dir: &PathBuf) -> Result<()> {
fs::create_dir_all(dir)
.await
.context("failed to create folder")?;
Config::new(dir).await?;
Config::new(os_name.clone(), dir).await?;
Ok(())
}
@@ -56,13 +56,8 @@ impl Accounts {
let config_file = dir.join(CONFIG_NAME);
ensure!(config_file.exists().await, "accounts.toml does not exist");
let config = Config::from_file(config_file)
.await
.context("failed to load accounts config")?;
let accounts = config
.load_accounts()
.await
.context("failed to load accounts")?;
let config = Config::from_file(config_file).await?;
let accounts = config.load_accounts().await?;
let emitter = EventEmitter::new();
@@ -71,9 +66,7 @@ impl Accounts {
emitter.sender.send(events.get_emitter()).await?;
for account in accounts.values() {
emitter.add_account(account).await.with_context(|| {
format!("failed to add account {} to event emitter", account.id)
})?;
emitter.add_account(account).await?;
}
Ok(Self {
@@ -111,24 +104,12 @@ impl Accounts {
Ok(())
}
/// Add a new account and opens it.
///
/// Returns account ID.
/// Add a new account.
pub async fn add_account(&mut self) -> Result<u32> {
let os_name = self.config.os_name().await;
let account_config = self.config.new_account(&self.dir).await?;
let ctx = Context::new(account_config.dbfile().into(), account_config.id).await?;
self.emitter.add_account(&ctx).await?;
self.accounts.insert(account_config.id, ctx);
Ok(account_config.id)
}
/// Adds a new closed account.
pub async fn add_closed_account(&mut self) -> Result<u32> {
let account_config = self.config.new_account(&self.dir).await?;
let ctx = Context::new_closed(account_config.dbfile().into(), account_config.id).await?;
let ctx = Context::new(os_name, account_config.dbfile().into(), account_config.id).await?;
self.emitter.add_account(&ctx).await?;
self.accounts.insert(account_config.id, ctx);
@@ -202,7 +183,13 @@ impl Accounts {
match res {
Ok(_) => {
let ctx = Context::new(new_dbfile, account_config.id).await?;
let ctx = Context::with_blobdir(
self.config.os_name().await,
new_dbfile,
new_blobdir,
account_config.id,
)
.await?;
self.emitter.add_account(&ctx).await?;
self.accounts.insert(account_config.id, ctx);
Ok(account_config.id)
@@ -362,6 +349,7 @@ pub struct Config {
/// This is serialized into TOML.
#[derive(Serialize, Deserialize, Debug, Clone, PartialEq)]
struct InnerConfig {
pub os_name: String,
/// The currently selected account.
pub selected_account: u32,
pub next_id: u32,
@@ -369,8 +357,9 @@ struct InnerConfig {
}
impl Config {
pub async fn new(dir: &PathBuf) -> Result<Self> {
pub async fn new(os_name: String, dir: &PathBuf) -> Result<Self> {
let inner = InnerConfig {
os_name,
accounts: Vec::new(),
selected_account: 0,
next_id: 1,
@@ -385,6 +374,10 @@ impl Config {
Ok(cfg)
}
pub async fn os_name(&self) -> String {
self.inner.os_name.clone()
}
/// Sync the inmemory representation to disk.
async fn sync(&self) -> Result<()> {
fs::write(&self.file, toml::to_string_pretty(&self.inner)?)
@@ -403,15 +396,12 @@ impl Config {
pub async fn load_accounts(&self) -> Result<BTreeMap<u32, Context>> {
let mut accounts = BTreeMap::new();
for account_config in &self.inner.accounts {
let ctx = Context::new(account_config.dbfile().into(), account_config.id)
.await
.with_context(|| {
format!(
"failed to create context from file {:?}",
account_config.dbfile()
)
})?;
let ctx = Context::new(
self.inner.os_name.clone(),
account_config.dbfile().into(),
account_config.id,
)
.await?;
accounts.insert(account_config.id, ctx);
}
@@ -436,13 +426,8 @@ impl Config {
self.sync().await?;
self.select_account(id)
.await
.context("failed to select just added account")?;
let cfg = self
.get_account(id)
.await
.context("failed to get just added account")?;
self.select_account(id).await.expect("just added");
let cfg = self.get_account(id).await.expect("just added");
Ok(cfg)
}
@@ -513,7 +498,7 @@ mod tests {
let dir = tempfile::tempdir().unwrap();
let p: PathBuf = dir.path().join("accounts1").into();
let mut accounts1 = Accounts::new(p.clone()).await.unwrap();
let mut accounts1 = Accounts::new("my_os".into(), p.clone()).await.unwrap();
accounts1.add_account().await.unwrap();
let accounts2 = Accounts::open(p).await.unwrap();
@@ -531,7 +516,7 @@ mod tests {
let dir = tempfile::tempdir().unwrap();
let p: PathBuf = dir.path().join("accounts").into();
let mut accounts = Accounts::new(p.clone()).await.unwrap();
let mut accounts = Accounts::new("my_os".into(), p.clone()).await.unwrap();
assert_eq!(accounts.accounts.len(), 0);
assert_eq!(accounts.config.get_selected_account().await, 0);
@@ -558,7 +543,7 @@ mod tests {
let dir = tempfile::tempdir()?;
let p: PathBuf = dir.path().join("accounts").into();
let mut accounts = Accounts::new(p.clone()).await?;
let mut accounts = Accounts::new("my_os".into(), p.clone()).await?;
assert!(accounts.get_selected_account().await.is_none());
assert_eq!(accounts.config.get_selected_account().await, 0);
@@ -579,12 +564,14 @@ mod tests {
let dir = tempfile::tempdir().unwrap();
let p: PathBuf = dir.path().join("accounts").into();
let mut accounts = Accounts::new(p.clone()).await.unwrap();
let mut accounts = Accounts::new("my_os".into(), p.clone()).await.unwrap();
assert_eq!(accounts.accounts.len(), 0);
assert_eq!(accounts.config.get_selected_account().await, 0);
let extern_dbfile: PathBuf = dir.path().join("other").into();
let ctx = Context::new(extern_dbfile.clone(), 0).await.unwrap();
let ctx = Context::new("my_os".into(), extern_dbfile.clone(), 0)
.await
.unwrap();
ctx.set_config(crate::config::Config::Addr, Some("me@mail.com"))
.await
.unwrap();
@@ -614,7 +601,7 @@ mod tests {
let dir = tempfile::tempdir().unwrap();
let p: PathBuf = dir.path().join("accounts").into();
let mut accounts = Accounts::new(p.clone()).await.unwrap();
let mut accounts = Accounts::new("my_os".into(), p.clone()).await.unwrap();
for expected_id in 1..10 {
let id = accounts.add_account().await.unwrap();
@@ -634,7 +621,7 @@ mod tests {
let dummy_accounts = 10;
let (id0, id1, id2) = {
let mut accounts = Accounts::new(p.clone()).await?;
let mut accounts = Accounts::new("my_os".into(), p.clone()).await?;
accounts.add_account().await?;
let ids = accounts.get_all().await;
assert_eq!(ids.len(), 1);
@@ -669,7 +656,7 @@ mod tests {
assert!(id2 > id1 + dummy_accounts);
let (id0_reopened, id1_reopened, id2_reopened) = {
let accounts = Accounts::new(p.clone()).await?;
let accounts = Accounts::new("my_os".into(), p.clone()).await?;
let ctx = accounts.get_selected_account().await.unwrap();
assert_eq!(
ctx.get_config(crate::config::Config::Addr).await?,
@@ -714,7 +701,7 @@ mod tests {
let dir = tempfile::tempdir().unwrap();
let p: PathBuf = dir.path().join("accounts").into();
let accounts = Accounts::new(p.clone()).await?;
let accounts = Accounts::new("my_os".into(), p.clone()).await?;
// Make sure there are no accounts.
assert_eq!(accounts.accounts.len(), 0);
@@ -734,49 +721,4 @@ mod tests {
Ok(())
}
#[async_std::test]
async fn test_encrypted_account() -> Result<()> {
let dir = tempfile::tempdir().context("failed to create tempdir")?;
let p: PathBuf = dir.path().join("accounts").into();
let mut accounts = Accounts::new(p.clone())
.await
.context("failed to create accounts manager")?;
assert_eq!(accounts.accounts.len(), 0);
let account_id = accounts
.add_closed_account()
.await
.context("failed to add closed account")?;
let account = accounts
.get_selected_account()
.await
.context("failed to get account")?;
assert_eq!(account.id, account_id);
let passphrase_set_success = account
.open("foobar".to_string())
.await
.context("failed to set passphrase")?;
assert!(passphrase_set_success);
drop(accounts);
let accounts = Accounts::new(p.clone())
.await
.context("failed to create second accounts manager")?;
let account = accounts
.get_selected_account()
.await
.context("failed to get account")?;
assert_eq!(account.is_open().await, false);
// Try wrong passphrase.
assert_eq!(account.open("barfoo".to_string()).await?, false);
assert_eq!(account.open("".to_string()).await?, false);
assert_eq!(account.open("foobar".to_string()).await?, true);
assert_eq!(account.is_open().await, true);
Ok(())
}
}

View File

@@ -2,7 +2,7 @@
//!
//! Parse and create [Autocrypt-headers](https://autocrypt.org/en/latest/level1.html#the-autocrypt-header).
use anyhow::{bail, Context as _, Error, Result};
use anyhow::{bail, format_err, Error, Result};
use std::collections::BTreeMap;
use std::str::FromStr;
use std::{fmt, str};
@@ -139,14 +139,15 @@ impl str::FromStr for Aheader {
};
let public_key: SignedPublicKey = attributes
.remove("keydata")
.context("keydata attribute is not found")
.ok_or_else(|| format_err!("keydata attribute is not found"))
.and_then(|raw| {
SignedPublicKey::from_base64(&raw).context("autocrypt key cannot be decoded")
SignedPublicKey::from_base64(&raw)
.map_err(|_| format_err!("Autocrypt key cannot be decoded"))
})
.and_then(|key| {
key.verify()
.and(Ok(key))
.context("autocrypt key cannot be verified")
.map_err(|_| format_err!("Autocrypt key cannot be verified"))
})?;
let prefer_encrypt = attributes

View File

@@ -3,26 +3,28 @@
use core::cmp::max;
use std::ffi::OsStr;
use std::fmt;
use std::io::Cursor;
use async_std::path::{Path, PathBuf};
use async_std::prelude::*;
use async_std::{fs, io};
use anyhow::{format_err, Context as _, Error};
use image::{DynamicImage, ImageFormat};
use anyhow::format_err;
use anyhow::Context as _;
use anyhow::Error;
use image::DynamicImage;
use image::GenericImageView;
use image::ImageFormat;
use num_traits::FromPrimitive;
use thiserror::Error;
use crate::config::Config;
use crate::constants::{
MediaQuality, BALANCED_AVATAR_SIZE, BALANCED_IMAGE_SIZE, WORSE_AVATAR_SIZE, WORSE_IMAGE_SIZE,
MediaQuality, Viewtype, BALANCED_AVATAR_SIZE, BALANCED_IMAGE_SIZE, WORSE_AVATAR_SIZE,
WORSE_IMAGE_SIZE,
};
use crate::context::Context;
use crate::events::EventType;
use crate::log::LogExt;
use crate::message;
use crate::message::Viewtype;
/// Represents a file in the blob directory.
///
@@ -61,7 +63,7 @@ impl<'a> BlobObject<'a> {
) -> std::result::Result<BlobObject<'a>, BlobError> {
let blobdir = context.get_blobdir();
let (stem, ext) = BlobObject::sanitise_name(suggested_name);
let (name, mut file) = BlobObject::create_new_file(context, blobdir, &stem, &ext).await?;
let (name, mut file) = BlobObject::create_new_file(blobdir, &stem, &ext).await?;
file.write_all(data)
.await
.map_err(|err| BlobError::WriteFailure {
@@ -85,16 +87,13 @@ impl<'a> BlobObject<'a> {
// Creates a new file, returning a tuple of the name and the handle.
async fn create_new_file(
context: &Context,
dir: &Path,
stem: &str,
ext: &str,
) -> Result<(String, fs::File), BlobError> {
const MAX_ATTEMPT: u32 = 16;
let mut attempt = 0;
let max_attempt = 15;
let mut name = format!("{}{}", stem, ext);
loop {
attempt += 1;
for attempt in 0..max_attempt {
let path = dir.join(&name);
match fs::OpenOptions::new()
.create_new(true)
@@ -104,20 +103,24 @@ impl<'a> BlobObject<'a> {
{
Ok(file) => return Ok((name, file)),
Err(err) => {
if attempt >= MAX_ATTEMPT {
if attempt == max_attempt {
return Err(BlobError::CreateFailure {
blobdir: dir.to_path_buf(),
blobname: name,
cause: err,
});
} else if attempt == 1 && !dir.exists().await {
fs::create_dir_all(dir).await.ok_or_log(context);
} else {
name = format!("{}-{}{}", stem, rand::random::<u32>(), ext);
}
}
}
}
// This is supposed to be unreachable, but the compiler doesn't know.
Err(BlobError::CreateFailure {
blobdir: dir.to_path_buf(),
blobname: name,
cause: std::io::Error::new(std::io::ErrorKind::Other, "supposedly unreachable"),
})
}
/// Creates a new blob object with unique name by copying an existing file.
@@ -146,7 +149,7 @@ impl<'a> BlobObject<'a> {
})?;
let (stem, ext) = BlobObject::sanitise_name(&src.to_string_lossy());
let (name, mut dst_file) =
BlobObject::create_new_file(context, context.get_blobdir(), &stem, &ext).await?;
BlobObject::create_new_file(context.get_blobdir(), &stem, &ext).await?;
let name_for_err = name.clone();
if let Err(err) = io::copy(&mut src_file, &mut dst_file).await {
{
@@ -289,7 +292,7 @@ impl<'a> BlobObject<'a> {
/// Returns the filename of the blob.
pub fn as_file_name(&self) -> &str {
self.name.rsplit('/').next().unwrap()
self.name.rsplitn(2, '/').next().unwrap()
}
/// The path relative in the blob directory.
@@ -302,7 +305,7 @@ impl<'a> BlobObject<'a> {
/// If a blob's filename has an extension, it is always guaranteed
/// to be lowercase.
pub fn suffix(&self) -> Option<&str> {
let ext = self.name.rsplit('.').next();
let ext = self.name.rsplitn(2, '.').next();
if ext == Some(&self.name) {
None
} else {
@@ -345,30 +348,13 @@ impl<'a> BlobObject<'a> {
};
let clean = sanitize_filename::sanitize_with_options(name, opts);
// Let's take the tricky filename
// "file.with_lots_of_characters_behind_point_and_double_ending.tar.gz" as an example.
// Split it into "file" and "with_lots_of_characters_behind_point_and_double_ending.tar.gz":
let mut iter = clean.splitn(2, '.');
let stem: String = iter.next().unwrap_or_default().chars().take(64).collect();
// stem == "file"
let ext_chars = iter.next().unwrap_or_default().chars();
let ext: String = ext_chars
.rev()
.take(32)
.collect::<Vec<_>>()
.iter()
.rev()
.collect();
// ext == "d_point_and_double_ending.tar.gz"
let ext: String = iter.next().unwrap_or_default().chars().take(32).collect();
if ext.is_empty() {
(stem, "".to_string())
} else {
(stem, format!(".{}", ext).to_lowercase())
// Return ("file", ".d_point_and_double_ending.tar.gz")
// which is not perfect but acceptable.
}
}
@@ -463,8 +449,7 @@ impl<'a> BlobObject<'a> {
fn encode_img(img: &DynamicImage, encoded: &mut Vec<u8>) -> anyhow::Result<()> {
encoded.clear();
let mut buf = Cursor::new(encoded);
img.write_to(&mut buf, image::ImageFormat::Jpeg)?;
img.write_to(encoded, image::ImageFormat::Jpeg)?;
Ok(())
}
fn encoded_img_exceeds_bytes(
@@ -634,15 +619,14 @@ pub enum BlobError {
mod tests {
use fs::File;
use anyhow::Result;
use image::{GenericImageView, Pixel};
use crate::chat::{self, create_group_chat, ProtectionStatus};
use crate::message::Message;
use crate::test_utils::{self, TestContext};
use super::*;
use crate::{
message::Message,
test_utils::{self, TestContext},
};
use image::Pixel;
#[async_std::test]
async fn test_create() {
let t = TestContext::new().await;
@@ -973,11 +957,10 @@ mod tests {
.unwrap();
assert_correct_rotation(&img_rotated);
let mut buf = Cursor::new(vec![]);
let mut bytes = vec![];
img_rotated
.write_to(&mut buf, image::ImageFormat::Jpeg)
.write_to(&mut bytes, image::ImageFormat::Jpeg)
.unwrap();
let bytes = buf.into_inner();
let img_rotated = send_image_check_mediaquality(
Some("0"),
&bytes,
@@ -1083,38 +1066,4 @@ mod tests {
assert_eq!(img.height() as u32, compressed_height);
Ok(img)
}
#[async_std::test]
async fn test_increation_in_blobdir() -> Result<()> {
let t = TestContext::new_alice().await;
let chat_id = create_group_chat(&t, ProtectionStatus::Unprotected, "abc").await?;
let file = t.get_blobdir().join("anyfile.dat");
File::create(&file).await?.write_all("bla".as_ref()).await?;
let mut msg = Message::new(Viewtype::File);
msg.set_file(file.to_str().unwrap(), None);
let prepared_id = chat::prepare_msg(&t, chat_id, &mut msg).await?;
assert_eq!(prepared_id, msg.id);
assert!(msg.is_increation());
let msg = Message::load_from_db(&t, prepared_id).await?;
assert!(msg.is_increation());
Ok(())
}
#[async_std::test]
async fn test_increation_not_blobdir() -> Result<()> {
let t = TestContext::new_alice().await;
let chat_id = create_group_chat(&t, ProtectionStatus::Unprotected, "abc").await?;
assert_ne!(t.get_blobdir().to_str(), t.dir.path().to_str());
let file = t.dir.path().join("anyfile.dat");
File::create(&file).await?.write_all("bla".as_ref()).await?;
let mut msg = Message::new(Viewtype::File);
msg.set_file(file.to_str().unwrap(), None);
assert!(chat::prepare_msg(&t, chat_id, &mut msg).await.is_err());
Ok(())
}
}

File diff suppressed because it is too large Load Diff

View File

@@ -1,6 +1,6 @@
//! # Chat list module.
use anyhow::{ensure, Context as _, Result};
use anyhow::{bail, ensure, Result};
use crate::chat::{update_special_chat_names, Chat, ChatId, ChatVisibility};
use crate::constants::{
@@ -8,7 +8,7 @@ use crate::constants::{
DC_CONTACT_ID_SELF, DC_CONTACT_ID_UNDEFINED, DC_GCL_ADD_ALLDONE_HINT, DC_GCL_ARCHIVED_ONLY,
DC_GCL_FOR_FORWARDING, DC_GCL_NO_SPECIALS,
};
use crate::contact::{Contact, ContactId};
use crate::contact::Contact;
use crate::context::Context;
use crate::ephemeral::delete_expired_messages;
use crate::message::{Message, MessageState, MsgId};
@@ -85,7 +85,7 @@ impl Chatlist {
context: &Context,
listflags: usize,
query: Option<&str>,
query_contact_id: Option<ContactId>,
query_contact_id: Option<u32>,
) -> Result<Self> {
let flag_archived_only = 0 != listflags & DC_GCL_ARCHIVED_ONLY;
let flag_for_forwarding = 0 != listflags & DC_GCL_FOR_FORWARDING;
@@ -147,7 +147,7 @@ impl Chatlist {
AND c.id IN(SELECT chat_id FROM chats_contacts WHERE contact_id=?2)
GROUP BY c.id
ORDER BY c.archived=?3 DESC, IFNULL(m.timestamp,c.created_timestamp) DESC, m.id DESC;",
paramsv![MessageState::OutDraft, query_contact_id, ChatVisibility::Pinned],
paramsv![MessageState::OutDraft, query_contact_id as i32, ChatVisibility::Pinned],
process_row,
process_rows,
).await?
@@ -271,23 +271,21 @@ impl Chatlist {
/// Get a single chat ID of a chatlist.
///
/// To get the message object from the message ID, use dc_get_chat().
pub fn get_chat_id(&self, index: usize) -> Result<ChatId> {
let (chat_id, _msg_id) = self
.ids
.get(index)
.context("chatlist index is out of range")?;
Ok(*chat_id)
pub fn get_chat_id(&self, index: usize) -> ChatId {
match self.ids.get(index) {
Some((chat_id, _msg_id)) => *chat_id,
None => ChatId::new(0),
}
}
/// Get a single message ID of a chatlist.
///
/// To get the message object from the message ID, use dc_get_msg().
pub fn get_msg_id(&self, index: usize) -> Result<Option<MsgId>> {
let (_chat_id, msg_id) = self
.ids
.get(index)
.context("chatlist index is out of range")?;
Ok(*msg_id)
match self.ids.get(index) {
Some((_chat_id, msg_id)) => Ok(*msg_id),
None => bail!("Chatlist index out of range"),
}
}
/// Returns a summary for a given chatlist index.
@@ -301,10 +299,11 @@ impl Chatlist {
// This is because we may want to display drafts here or stuff as
// "is typing".
// Also, sth. as "No messages" would not work if the summary comes from a message.
let (chat_id, lastmsg_id) = self
.ids
.get(index)
.context("chatlist index is out of range")?;
let (chat_id, lastmsg_id) = match self.ids.get(index) {
Some(ids) => ids,
None => bail!("Chatlist index out of range"),
};
Chatlist::get_summary2(context, *chat_id, *lastmsg_id, chat).await
}
@@ -375,8 +374,8 @@ mod tests {
use super::*;
use crate::chat::{create_group_chat, get_chat_contacts, ProtectionStatus};
use crate::constants::Viewtype;
use crate::dc_receive_imf::dc_receive_imf;
use crate::message::Viewtype;
use crate::stock_str::StockMessage;
use crate::test_utils::TestContext;
@@ -396,9 +395,9 @@ mod tests {
// check that the chatlist starts with the most recent message
let chats = Chatlist::try_load(&t, 0, None, None).await.unwrap();
assert_eq!(chats.len(), 3);
assert_eq!(chats.get_chat_id(0).unwrap(), chat_id3);
assert_eq!(chats.get_chat_id(1).unwrap(), chat_id2);
assert_eq!(chats.get_chat_id(2).unwrap(), chat_id1);
assert_eq!(chats.get_chat_id(0), chat_id3);
assert_eq!(chats.get_chat_id(1), chat_id2);
assert_eq!(chats.get_chat_id(2), chat_id1);
// New drafts are sorted to the top
// We have to set a draft on the other two messages, too, as
@@ -415,7 +414,7 @@ mod tests {
}
let chats = Chatlist::try_load(&t, 0, None, None).await.unwrap();
assert_eq!(chats.get_chat_id(0).unwrap(), chat_id2);
assert_eq!(chats.get_chat_id(0), chat_id2);
// check chatlist query and archive functionality
let chats = Chatlist::try_load(&t, 0, Some("b"), None).await.unwrap();
@@ -446,7 +445,7 @@ mod tests {
let chats = Chatlist::try_load(&t, 0, None, None).await.unwrap();
assert!(chats.len() == 3);
assert!(!Chat::load_from_db(&t, chats.get_chat_id(0).unwrap())
assert!(!Chat::load_from_db(&t, chats.get_chat_id(0))
.await
.unwrap()
.is_self_talk());
@@ -455,7 +454,7 @@ mod tests {
.await
.unwrap();
assert!(chats.len() == 2); // device chat cannot be written and is skipped on forwarding
assert!(Chat::load_from_db(&t, chats.get_chat_id(0).unwrap())
assert!(Chat::load_from_db(&t, chats.get_chat_id(0))
.await
.unwrap()
.is_self_talk());
@@ -500,7 +499,7 @@ mod tests {
dc_receive_imf(
&t,
b"From: Bob Authname <bob@example.org>\n\
To: alice@example.org\n\
To: alice@example.com\n\
Subject: foo\n\
Message-ID: <msg1234@example.org>\n\
Chat-Version: 1.0\n\
@@ -508,6 +507,7 @@ mod tests {
\n\
hello foo\n",
"INBOX",
1,
false,
)
.await?;
@@ -528,7 +528,7 @@ mod tests {
// check, the one-to-one-chat can be found using chatlist search query
let chats = Chatlist::try_load(&t, 0, Some("bob authname"), None).await?;
assert_eq!(chats.len(), 1);
assert_eq!(chats.get_chat_id(0).unwrap(), chat_id);
assert_eq!(chats.get_chat_id(0), chat_id);
// change the name of the contact; this also changes the name of the one-to-one-chat
let test_id = Contact::create(&t, "Bob Nickname", "bob@example.org").await?;
@@ -561,7 +561,7 @@ mod tests {
dc_receive_imf(
&t,
b"From: bob@example.org\n\
To: alice@example.org\n\
To: alice@example.com\n\
Subject: foo\n\
Message-ID: <msg5678@example.org>\n\
Chat-Version: 1.0\n\
@@ -569,6 +569,7 @@ mod tests {
\n\
hello foo\n",
"INBOX",
1,
false,
)
.await?;
@@ -584,7 +585,7 @@ mod tests {
// check, the one-to-one-chat can be found using chatlist search query
let chats = Chatlist::try_load(&t, 0, Some("bob@example.org"), None).await?;
assert_eq!(chats.len(), 1);
assert_eq!(chats.get_chat_id(0)?, chat_id);
assert_eq!(chats.get_chat_id(0), chat_id);
// change the name of the contact; this also changes the name of the one-to-one-chat
let test_id = Contact::create(&t, "Bob Nickname", "bob@example.org").await?;
@@ -595,7 +596,7 @@ mod tests {
assert_eq!(chats.len(), 0); // email-addresses are searchable in contacts, not in chats
let chats = Chatlist::try_load(&t, 0, Some("Bob Nickname"), None).await?;
assert_eq!(chats.len(), 1);
assert_eq!(chats.get_chat_id(0)?, chat_id);
assert_eq!(chats.get_chat_id(0), chat_id);
// revert name change, this again changes the name of the one-to-one-chat to the email-address
let test_id = Contact::create(&t, "", "bob@example.org").await?;

View File

@@ -10,9 +10,11 @@ use crate::constants::DC_VERSION_STR;
use crate::context::Context;
use crate::dc_tools::{dc_get_abs_path, improve_single_line_input};
use crate::events::EventType;
use crate::job;
use crate::message::MsgId;
use crate::mimefactory::RECOMMENDED_FILE_SIZE;
use crate::provider::{get_provider_by_id, Provider};
use crate::stock_str;
/// The available configuration keys.
#[derive(
@@ -62,21 +64,30 @@ pub enum Config {
#[strum(props(default = "1"))]
E2eeEnabled,
/// Ignore Autocrypt recommendation for message encryption if possible.
///
/// The only expection is when recommendation is "disable", i.e. encryption is not possible
/// because some recipient has no OpenPGP key.
#[strum(props(default = "0"))]
E2eeForce,
#[strum(props(default = "1"))]
MdnsEnabled,
#[strum(props(default = "0"))]
#[strum(props(default = "1"))]
InboxWatch,
#[strum(props(default = "1"))]
SentboxWatch,
#[strum(props(default = "1"))]
MvboxWatch,
#[strum(props(default = "1"))]
MvboxMove,
/// Watch for new messages in the "Mvbox" (aka DeltaChat folder) only.
///
/// This will not entirely disable other folders, e.g. the spam folder will also still
/// be watched for new messages.
#[strum(props(default = "0"))]
OnlyFetchMvbox,
SentboxMove, // If `MvboxMove` is true, this config is ignored. Currently only used in tests.
#[strum(props(default = "0"))] // also change ShowEmails.default() on changes
ShowEmails,
@@ -202,6 +213,7 @@ impl Context {
// Default values
match key {
Config::Selfstatus => Ok(Some(stock_str::status_line(self).await)),
Config::ConfiguredInboxFolder => Ok(Some("INBOX".to_owned())),
_ => Ok(key.get_str("default").map(|s| s.to_string())),
}
@@ -229,11 +241,6 @@ impl Context {
Ok(self.get_config_int(key).await? != 0)
}
pub(crate) async fn should_watch_mvbox(&self) -> Result<bool> {
Ok(self.get_config_bool(Config::MvboxMove).await?
|| self.get_config_bool(Config::OnlyFetchMvbox).await?)
}
/// Gets configured "delete_server_after" value.
///
/// `None` means never delete the message, `Some(0)` means delete
@@ -242,7 +249,7 @@ impl Context {
match self.get_config_int(Config::DeleteServerAfter).await? {
0 => Ok(None),
1 => Ok(Some(0)),
x => Ok(Some(i64::from(x))),
x => Ok(Some(x as i64)),
}
}
@@ -264,7 +271,7 @@ impl Context {
pub async fn get_config_delete_device_after(&self) -> Result<Option<i64>> {
match self.get_config_int(Config::DeleteDeviceAfter).await? {
0 => Ok(None),
x => Ok(Some(i64::from(x))),
x => Ok(Some(x as i64)),
}
}
@@ -290,29 +297,55 @@ impl Context {
}
}
self.emit_event(EventType::SelfavatarChanged);
Ok(())
}
Config::Selfstatus => {
let def = stock_str::status_line(self).await;
let val = if value.is_none() || value.unwrap() == def {
None
} else {
value
};
self.sql.set_raw_config(key, val).await?;
Ok(())
}
Config::DeleteDeviceAfter => {
let ret = self.sql.set_raw_config(key, value).await;
let ret = self
.sql
.set_raw_config(key, value)
.await
.map_err(Into::into);
// Force chatlist reload to delete old messages immediately.
self.emit_event(EventType::MsgsChanged {
msg_id: MsgId::new(0),
chat_id: ChatId::new(0),
});
ret?
ret
}
Config::Displayname => {
let value = value.map(improve_single_line_input);
self.sql.set_raw_config(key, value.as_deref()).await?;
Ok(())
}
Config::DeleteServerAfter => {
let ret = self
.sql
.set_raw_config(key, value)
.await
.map_err(Into::into);
job::schedule_resync(self).await?;
ret
}
_ => {
self.sql.set_raw_config(key, value).await?;
Ok(())
}
}
Ok(())
}
pub async fn set_config_bool(&self, key: Config, value: bool) -> Result<()> {
self.set_config(key, if value { Some("1") } else { Some("0") })
self.set_config(key, if value { Some("1") } else { None })
.await?;
Ok(())
}
@@ -404,17 +437,4 @@ mod tests {
Ok(())
}
/// Regression test for https://github.com/deltachat/deltachat-core-rust/issues/3012
#[async_std::test]
async fn test_set_config_bool() -> Result<()> {
let t = TestContext::new().await;
// We need some config that defaults to true
let c = Config::E2eeEnabled;
assert_eq!(t.get_config_bool(c).await?, true);
t.set_config_bool(c, false).await?;
assert_eq!(t.get_config_bool(c).await?, false);
Ok(())
}
}

View File

@@ -11,20 +11,21 @@ use async_std::task;
use job::Action;
use percent_encoding::{utf8_percent_encode, NON_ALPHANUMERIC};
use crate::config::Config;
use crate::constants::{DC_LP_AUTH_FLAGS, DC_LP_AUTH_NORMAL, DC_LP_AUTH_OAUTH2};
use crate::context::Context;
use crate::dc_tools::{time, EmailAddress};
use crate::dc_tools::EmailAddress;
use crate::imap::Imap;
use crate::job;
use crate::login_param::{CertificateChecks, LoginParam, ServerLoginParam, Socks5Config};
use crate::message::{Message, Viewtype};
use crate::message::Message;
use crate::oauth2::dc_get_oauth2_addr;
use crate::param::Params;
use crate::provider::{Protocol, Socket, UsernamePattern};
use crate::smtp::Smtp;
use crate::stock_str;
use crate::{chat, e2ee, provider};
use crate::{config::Config, dc_tools::time};
use crate::{
constants::{Viewtype, DC_LP_AUTH_FLAGS, DC_LP_AUTH_NORMAL, DC_LP_AUTH_OAUTH2},
job,
};
use crate::{context::Context, param::Params};
use auto_mozilla::moz_autoconfigure;
use auto_outlook::outlk_autodiscover;
@@ -220,9 +221,7 @@ async fn configure(ctx: &Context, param: &mut LoginParam) -> Result<()> {
"checking internal provider-info for offline autoconfig"
);
if let Some(provider) =
provider::get_provider_info(ctx, &param_domain, socks5_enabled).await
{
if let Some(provider) = provider::get_provider_info(&param_domain, socks5_enabled).await {
param.provider = Some(provider);
match provider.status {
provider::Status::Ok | provider::Status::Preparation => {
@@ -442,7 +441,8 @@ async fn configure(ctx: &Context, param: &mut LoginParam) -> Result<()> {
progress!(ctx, 900);
let create_mvbox = ctx.should_watch_mvbox().await?;
let create_mvbox = ctx.get_config_bool(Config::MvboxWatch).await?
|| ctx.get_config_bool(Config::MvboxMove).await?;
imap.configure_folders(ctx, create_mvbox).await?;
@@ -453,8 +453,11 @@ async fn configure(ctx: &Context, param: &mut LoginParam) -> Result<()> {
drop(imap);
progress!(ctx, 910);
// configuration success - write back the configured parameters with the
// "configured_" prefix; also write the "configured"-flag */
// the trailing underscore is correct
param.save_to_database(ctx, "configured_").await?;
ctx.sql.set_raw_config_bool("configured", true).await?;
ctx.set_config(Config::ConfiguredTimestamp, Some(&time().to_string()))
.await?;
@@ -472,8 +475,6 @@ async fn configure(ctx: &Context, param: &mut LoginParam) -> Result<()> {
progress!(ctx, 940);
update_device_chats_handle.await?;
ctx.sql.set_raw_config_bool("configured", true).await?;
Ok(())
}

View File

@@ -4,7 +4,6 @@ use once_cell::sync::Lazy;
use serde::{Deserialize, Serialize};
use crate::chat::ChatId;
use crate::contact::ContactId;
pub static DC_VERSION_STR: Lazy<String> = Lazy::new(|| env!("CARGO_PKG_VERSION").to_string());
@@ -180,11 +179,11 @@ pub const DC_ELLIPSIS: &str = "[...]";
/// `char`s), not Unicode Grapheme Clusters.
pub const DC_DESIRED_TEXT_LEN: usize = 5000;
pub const DC_CONTACT_ID_UNDEFINED: ContactId = ContactId::new(0);
pub const DC_CONTACT_ID_SELF: ContactId = ContactId::new(1);
pub const DC_CONTACT_ID_INFO: ContactId = ContactId::new(2);
pub const DC_CONTACT_ID_DEVICE: ContactId = ContactId::new(5);
pub const DC_CONTACT_ID_LAST_SPECIAL: ContactId = ContactId::new(9);
pub const DC_CONTACT_ID_UNDEFINED: u32 = 0;
pub const DC_CONTACT_ID_SELF: u32 = 1;
pub const DC_CONTACT_ID_INFO: u32 = 2;
pub const DC_CONTACT_ID_DEVICE: u32 = 5;
pub const DC_CONTACT_ID_LAST_SPECIAL: u32 = 9;
// decorative address that is used for DC_CONTACT_ID_DEVICE
// when an api that returns an email is called.
@@ -231,6 +230,79 @@ pub const DC_FOLDERS_CONFIGURED_VERSION: i32 = 3;
// can be overwritten by the setting `max_smtp_rcpt_to` in provider-db.
pub const DEFAULT_MAX_SMTP_RCPT_TO: usize = 50;
#[derive(
Debug,
Display,
Clone,
Copy,
PartialEq,
Eq,
FromPrimitive,
ToPrimitive,
FromSql,
ToSql,
Serialize,
Deserialize,
)]
#[repr(u32)]
pub enum Viewtype {
Unknown = 0,
/// Text message.
/// The text of the message is set using dc_msg_set_text()
/// and retrieved with dc_msg_get_text().
Text = 10,
/// Image message.
/// If the image is an animated GIF, the type DC_MSG_GIF should be used.
/// File, width and height are set via dc_msg_set_file(), dc_msg_set_dimension
/// and retrieved via dc_msg_set_file(), dc_msg_set_dimension().
Image = 20,
/// Animated GIF message.
/// File, width and height are set via dc_msg_set_file(), dc_msg_set_dimension()
/// and retrieved via dc_msg_get_file(), dc_msg_get_width(), dc_msg_get_height().
Gif = 21,
/// Message containing a sticker, similar to image.
/// If possible, the ui should display the image without borders in a transparent way.
/// A click on a sticker will offer to install the sticker set in some future.
Sticker = 23,
/// Message containing an Audio file.
/// File and duration are set via dc_msg_set_file(), dc_msg_set_duration()
/// and retrieved via dc_msg_get_file(), dc_msg_get_duration().
Audio = 40,
/// A voice message that was directly recorded by the user.
/// For all other audio messages, the type #DC_MSG_AUDIO should be used.
/// File and duration are set via dc_msg_set_file(), dc_msg_set_duration()
/// and retrieved via dc_msg_get_file(), dc_msg_get_duration()
Voice = 41,
/// Video messages.
/// File, width, height and durarion
/// are set via dc_msg_set_file(), dc_msg_set_dimension(), dc_msg_set_duration()
/// and retrieved via
/// dc_msg_get_file(), dc_msg_get_width(),
/// dc_msg_get_height(), dc_msg_get_duration().
Video = 50,
/// Message containing any file, eg. a PDF.
/// The file is set via dc_msg_set_file()
/// and retrieved via dc_msg_get_file().
File = 60,
/// Message is an invitation to a videochat.
VideochatInvitation = 70,
}
impl Default for Viewtype {
fn default() -> Self {
Viewtype::Unknown
}
}
pub const DC_JOB_DELETE_MSG_ON_IMAP: i32 = 110;
#[derive(Debug, Clone, Copy, PartialEq, Eq, FromPrimitive, ToPrimitive)]
@@ -242,9 +314,32 @@ pub enum KeyType {
#[cfg(test)]
mod tests {
use super::*;
use num_traits::FromPrimitive;
use super::*;
#[test]
fn test_derive_display_works_as_expected() {
assert_eq!(format!("{}", Viewtype::Audio), "Audio");
}
#[test]
fn test_viewtype_values() {
// values may be written to disk and must not change
assert_eq!(Viewtype::Unknown, Viewtype::default());
assert_eq!(Viewtype::Unknown, Viewtype::from_i32(0).unwrap());
assert_eq!(Viewtype::Text, Viewtype::from_i32(10).unwrap());
assert_eq!(Viewtype::Image, Viewtype::from_i32(20).unwrap());
assert_eq!(Viewtype::Gif, Viewtype::from_i32(21).unwrap());
assert_eq!(Viewtype::Sticker, Viewtype::from_i32(23).unwrap());
assert_eq!(Viewtype::Audio, Viewtype::from_i32(40).unwrap());
assert_eq!(Viewtype::Voice, Viewtype::from_i32(41).unwrap());
assert_eq!(Viewtype::Video, Viewtype::from_i32(50).unwrap());
assert_eq!(Viewtype::File, Viewtype::from_i32(60).unwrap());
assert_eq!(
Viewtype::VideochatInvitation,
Viewtype::from_i32(70).unwrap()
);
}
#[test]
fn test_chattype_values() {

View File

@@ -1,14 +1,12 @@
//! Contacts module
use std::convert::{TryFrom, TryInto};
use std::fmt;
use anyhow::{bail, ensure, Context as _, Result};
use async_std::path::PathBuf;
use deltachat_derive::{FromSql, ToSql};
use once_cell::sync::Lazy;
use regex::Regex;
use serde::{Deserialize, Serialize};
use crate::aheader::EncryptPreference;
use crate::chat::ChatId;
@@ -29,66 +27,6 @@ use crate::param::{Param, Params};
use crate::peerstate::{Peerstate, PeerstateVerifiedStatus};
use crate::{chat, stock_str};
/// Contact ID, including reserved IDs.
///
/// Some contact IDs are reserved to identify special contacts. This
/// type can represent both the special as well as normal contacts.
#[derive(
Debug, Copy, Clone, Default, PartialEq, Eq, Hash, PartialOrd, Ord, Serialize, Deserialize,
)]
pub struct ContactId(u32);
impl ContactId {
/// Creates a new [`ContactId`].
pub const fn new(id: u32) -> ContactId {
ContactId(id)
}
/// Bad evil escape hatch, do not use.
pub const fn to_u32(&self) -> u32 {
self.0
}
}
impl fmt::Display for ContactId {
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
write!(f, "{}", self.0)
// if *self == DC_CONTACT_ID_UNDEFINED {
// write!(f, "Contact#Undefined")
// } else if *self == DC_CONTACT_ID_SELF {
// write!(f, "Contact#Self")
// } else if *self == DC_CONTACT_ID_INFO {
// write!(f, "Contact#Info")
// } else if *self == DC_CONTACT_ID_DEVICE {
// write!(f, "Contact#Device")
// } else if *self <= DC_CONTACT_ID_LAST_SPECIAL {
// write!(f, "Contact#Special{}", self.0)
// } else {
// write!(f, "Contact#{}", self.0)
// }
}
}
/// Allow converting [`ContactId`] to an SQLite type.
impl rusqlite::types::ToSql for ContactId {
fn to_sql(&self) -> rusqlite::Result<rusqlite::types::ToSqlOutput> {
let val = rusqlite::types::Value::Integer(i64::from(self.0));
let out = rusqlite::types::ToSqlOutput::Owned(val);
Ok(out)
}
}
/// Allow converting an SQLite integer directly into [`ContactId`].
impl rusqlite::types::FromSql for ContactId {
fn column_result(value: rusqlite::types::ValueRef) -> rusqlite::types::FromSqlResult<Self> {
i64::column_result(value).and_then(|val| {
val.try_into()
.map(ContactId::new)
.map_err(|_| rusqlite::types::FromSqlError::OutOfRange(val))
})
}
}
/// An object representing a single contact in memory.
///
/// The contact object is not updated.
@@ -110,7 +48,7 @@ pub struct Contact {
/// `dc_set_config` using "addr".
///
/// Normal contact IDs are larger than these special ones (larger than DC_CONTACT_ID_LAST_SPECIAL).
pub id: ContactId,
pub id: u32,
/// Contact name. It is recommended to use `Contact::get_name`,
/// `Contact::get_display_name` or `Contact::get_name_n_addr` to access this field.
@@ -245,7 +183,7 @@ impl Default for VerifiedStatus {
}
impl Contact {
pub async fn load_from_db(context: &Context, contact_id: ContactId) -> Result<Self> {
pub async fn load_from_db(context: &Context, contact_id: u32) -> Result<Self> {
let mut contact = context
.sql
.query_row(
@@ -253,7 +191,7 @@ impl Contact {
c.authname, c.param, c.status
FROM contacts c
WHERE c.id=?;",
paramsv![contact_id],
paramsv![contact_id as i32],
|row| {
let name: String = row.get(0)?;
let addr: String = row.get(1)?;
@@ -307,18 +245,18 @@ impl Contact {
}
/// Check if a contact is blocked.
pub async fn is_blocked_load(context: &Context, id: ContactId) -> Result<bool> {
pub async fn is_blocked_load(context: &Context, id: u32) -> Result<bool> {
let blocked = Self::load_from_db(context, id).await?.blocked;
Ok(blocked)
}
/// Block the given contact.
pub async fn block(context: &Context, id: ContactId) -> Result<()> {
pub async fn block(context: &Context, id: u32) -> Result<()> {
set_block_contact(context, id, true).await
}
/// Unblock the given contact.
pub async fn unblock(context: &Context, id: ContactId) -> Result<()> {
pub async fn unblock(context: &Context, id: u32) -> Result<()> {
set_block_contact(context, id, false).await
}
@@ -331,7 +269,7 @@ impl Contact {
/// a bunch of addresses.
///
/// May result in a `#DC_EVENT_CONTACTS_CHANGED` event.
pub async fn create(context: &Context, name: &str, addr: &str) -> Result<ContactId> {
pub async fn create(context: &Context, name: &str, addr: &str) -> Result<u32> {
let name = improve_single_line_input(name);
ensure!(!addr.is_empty(), "Cannot create contact with empty address");
@@ -354,12 +292,12 @@ impl Contact {
}
/// Mark messages from a contact as noticed.
pub async fn mark_noticed(context: &Context, id: ContactId) -> Result<()> {
pub async fn mark_noticed(context: &Context, id: u32) -> Result<()> {
context
.sql
.execute(
"UPDATE msgs SET state=? WHERE from_id=? AND state=?;",
paramsv![MessageState::InNoticed, id, MessageState::InFresh],
paramsv![MessageState::InNoticed, id as i32, MessageState::InFresh],
)
.await?;
Ok(())
@@ -373,17 +311,17 @@ impl Contact {
/// use `dc_may_be_valid_addr()`.
pub async fn lookup_id_by_addr(
context: &Context,
addr: &str,
addr: impl AsRef<str>,
min_origin: Origin,
) -> Result<Option<ContactId>> {
if addr.is_empty() {
) -> Result<Option<u32>> {
if addr.as_ref().is_empty() {
bail!("lookup_id_by_addr: empty address");
}
let addr_normalized = addr_normalize(addr);
let addr_normalized = addr_normalize(addr.as_ref());
if let Some(addr_self) = context.get_config(Config::ConfiguredAddr).await? {
if addr_cmp(addr_normalized, &addr_self) {
if addr_cmp(addr_normalized, addr_self) {
return Ok(Some(DC_CONTACT_ID_SELF));
}
}
@@ -395,7 +333,7 @@ impl Contact {
AND id>?2 AND origin>=?3 AND blocked=0;",
paramsv![
addr_normalized,
DC_CONTACT_ID_LAST_SPECIAL,
DC_CONTACT_ID_LAST_SPECIAL as i32,
min_origin as u32,
],
)
@@ -433,7 +371,7 @@ impl Contact {
name: &str,
addr: &str,
mut origin: Origin,
) -> Result<(ContactId, Modifier)> {
) -> Result<(u32, Modifier)> {
let mut sth_modified = Modifier::None;
ensure!(!addr.is_empty(), "Can not add_or_lookup empty address");
@@ -445,7 +383,7 @@ impl Contact {
.await?
.unwrap_or_default();
if addr_cmp(&addr, &addr_self) {
if addr_cmp(&addr, addr_self) {
return Ok((DC_CONTACT_ID_SELF, sth_modified));
}
@@ -562,7 +500,7 @@ impl Contact {
paramsv![Chattype::Single, isize::try_from(row_id)?]
).await?;
if let Some(chat_id) = chat_id {
let contact = Contact::get_by_id(context, ContactId::new(row_id)).await?;
let contact = Contact::get_by_id(context, row_id as u32).await?;
let chat_name = contact.get_display_name();
match context
.sql
@@ -619,7 +557,7 @@ impl Contact {
}
}
Ok((ContactId::new(row_id), sth_modified))
Ok((row_id, sth_modified))
}
/// Add a number of contacts.
@@ -644,7 +582,7 @@ impl Contact {
for (name, addr) in split_address_book(addr_book).into_iter() {
let (name, addr) = sanitize_name_and_addr(name, addr);
let name = normalize_name(&name);
let name = normalize_name(name);
match Contact::add_or_lookup(context, &name, &addr, Origin::AddressBook).await {
Err(err) => {
warn!(
@@ -679,7 +617,7 @@ impl Contact {
context: &Context,
listflags: u32,
query: Option<impl AsRef<str>>,
) -> Result<Vec<ContactId>> {
) -> Result<Vec<u32>> {
let self_addr = context
.get_config(Config::ConfiguredAddr)
.await?
@@ -706,16 +644,16 @@ impl Contact {
ORDER BY LOWER(iif(c.name='',c.authname,c.name)||c.addr),c.id;",
paramsv![
self_addr,
DC_CONTACT_ID_LAST_SPECIAL,
DC_CONTACT_ID_LAST_SPECIAL as i32,
Origin::IncomingReplyTo,
s3str_like_cmd,
s3str_like_cmd,
if flag_verified_only { 0i32 } else { 1i32 },
],
|row| row.get::<_, ContactId>(0),
|row| row.get::<_, i32>(0),
|ids| {
for id in ids {
ret.push(id?);
ret.push(id? as u32);
}
Ok(())
},
@@ -752,13 +690,13 @@ impl Contact {
ORDER BY LOWER(iif(name='',authname,name)||addr),id;",
paramsv![
self_addr,
DC_CONTACT_ID_LAST_SPECIAL,
DC_CONTACT_ID_LAST_SPECIAL as i32,
Origin::IncomingReplyTo
],
|row| row.get::<_, ContactId>(0),
|row| row.get::<_, i32>(0),
|ids| {
for id in ids {
ret.push(id?);
ret.push(id? as u32);
}
Ok(())
},
@@ -829,7 +767,7 @@ impl Contact {
}
/// Get blocked contacts.
pub async fn get_all_blocked(context: &Context) -> Result<Vec<ContactId>> {
pub async fn get_all_blocked(context: &Context) -> Result<Vec<u32>> {
Contact::update_blocked_mailinglist_contacts(context)
.await
.context("cannot update blocked mailinglist contacts")?;
@@ -838,8 +776,8 @@ impl Contact {
.sql
.query_map(
"SELECT id FROM contacts WHERE id>? AND blocked!=0 ORDER BY LOWER(iif(name='',authname,name)||addr),id;",
paramsv![DC_CONTACT_ID_LAST_SPECIAL],
|row| row.get::<_, ContactId>(0),
paramsv![DC_CONTACT_ID_LAST_SPECIAL as i32],
|row| row.get::<_, u32>(0),
|ids| {
ids.collect::<std::result::Result<Vec<_>, _>>()
.map_err(Into::into)
@@ -854,7 +792,7 @@ impl Contact {
/// This function returns a string explaining the encryption state
/// of the contact and if the connection is encrypted the
/// fingerprints of the keys involved.
pub async fn get_encrinfo(context: &Context, contact_id: ContactId) -> Result<String> {
pub async fn get_encrinfo(context: &Context, contact_id: u32) -> Result<String> {
ensure!(
contact_id > DC_CONTACT_ID_LAST_SPECIAL,
"Can not provide encryption info for special contact"
@@ -923,7 +861,7 @@ impl Contact {
/// possible as the contact is in use. In this case, the contact can be blocked.
///
/// May result in a `#DC_EVENT_CONTACTS_CHANGED` event.
pub async fn delete(context: &Context, contact_id: ContactId) -> Result<()> {
pub async fn delete(context: &Context, contact_id: u32) -> Result<()> {
ensure!(
contact_id > DC_CONTACT_ID_LAST_SPECIAL,
"Can not delete special contact"
@@ -933,14 +871,17 @@ impl Contact {
.sql
.count(
"SELECT COUNT(*) FROM chats_contacts WHERE contact_id=?;",
paramsv![contact_id],
paramsv![contact_id as i32],
)
.await?;
if count_chats == 0 {
match context
.sql
.execute("DELETE FROM contacts WHERE id=?;", paramsv![contact_id])
.execute(
"DELETE FROM contacts WHERE id=?;",
paramsv![contact_id as i32],
)
.await
{
Ok(_) => {
@@ -966,7 +907,7 @@ impl Contact {
/// For contact DC_CONTACT_ID_SELF (1), the function returns sth.
/// like "Me" in the selected language and the email address
/// defined by dc_set_config().
pub async fn get_by_id(context: &Context, contact_id: ContactId) -> Result<Contact> {
pub async fn get_by_id(context: &Context, contact_id: u32) -> Result<Contact> {
let contact = Contact::load_from_db(context, contact_id).await?;
Ok(contact)
@@ -978,7 +919,7 @@ impl Contact {
.sql
.execute(
"UPDATE contacts SET param=? WHERE id=?",
paramsv![self.param.to_string(), self.id],
paramsv![self.param.to_string(), self.id as i32],
)
.await?;
Ok(())
@@ -990,14 +931,14 @@ impl Contact {
.sql
.execute(
"UPDATE contacts SET status=? WHERE id=?",
paramsv![self.status, self.id],
paramsv![self.status, self.id as i32],
)
.await?;
Ok(())
}
/// Get the ID of the contact.
pub fn get_id(&self) -> ContactId {
pub fn get_id(&self) -> u32 {
self.id
}
@@ -1124,7 +1065,7 @@ impl Contact {
pub async fn addr_equals_contact(
context: &Context,
addr: &str,
contact_id: ContactId,
contact_id: u32,
) -> Result<bool> {
if addr.is_empty() {
return Ok(false);
@@ -1150,13 +1091,13 @@ impl Contact {
.sql
.count(
"SELECT COUNT(*) FROM contacts WHERE id>?;",
paramsv![DC_CONTACT_ID_LAST_SPECIAL],
paramsv![DC_CONTACT_ID_LAST_SPECIAL as i32],
)
.await?;
Ok(count)
}
pub async fn real_exists_by_id(context: &Context, contact_id: ContactId) -> Result<bool> {
pub async fn real_exists_by_id(context: &Context, contact_id: u32) -> Result<bool> {
if contact_id <= DC_CONTACT_ID_LAST_SPECIAL {
return Ok(false);
}
@@ -1165,7 +1106,7 @@ impl Contact {
.sql
.exists(
"SELECT COUNT(*) FROM contacts WHERE id=?;",
paramsv![contact_id],
paramsv![contact_id as i32],
)
.await?;
Ok(exists)
@@ -1173,14 +1114,14 @@ impl Contact {
pub async fn scaleup_origin_by_id(
context: &Context,
contact_id: ContactId,
contact_id: u32,
origin: Origin,
) -> Result<()> {
context
.sql
.execute(
"UPDATE contacts SET origin=? WHERE id=? AND origin<?;",
paramsv![origin, contact_id, origin],
paramsv![origin, contact_id as i32, origin],
)
.await?;
Ok(())
@@ -1224,11 +1165,7 @@ fn sanitize_name_and_addr(name: &str, addr: &str) -> (String, String) {
}
}
async fn set_block_contact(
context: &Context,
contact_id: ContactId,
new_blocking: bool,
) -> Result<()> {
async fn set_block_contact(context: &Context, contact_id: u32, new_blocking: bool) -> Result<()> {
ensure!(
contact_id > DC_CONTACT_ID_LAST_SPECIAL,
"Can't block special contact {}",
@@ -1242,7 +1179,7 @@ async fn set_block_contact(
.sql
.execute(
"UPDATE contacts SET blocked=? WHERE id=?;",
paramsv![i32::from(new_blocking), contact_id],
paramsv![new_blocking as i32, contact_id as i32],
)
.await?;
@@ -1273,8 +1210,7 @@ WHERE type=? AND id IN (
// also unblock mailinglist
// if the contact is a mailinglist address explicitly created to allow unblocking
if !new_blocking && contact.origin == Origin::MailinglistAddress {
if let Some((chat_id, _, _)) =
chat::get_chat_id_by_grpid(context, &contact.addr).await?
if let Some((chat_id, _, _)) = chat::get_chat_id_by_grpid(context, contact.addr).await?
{
chat_id.unblock(context).await?;
}
@@ -1293,7 +1229,7 @@ WHERE type=? AND id IN (
/// this typically happens if we see message with our own profile image, sent from another device.
pub(crate) async fn set_profile_image(
context: &Context,
contact_id: ContactId,
contact_id: u32,
profile_image: &AvatarAction,
was_encrypted: bool,
) -> Result<()> {
@@ -1340,7 +1276,7 @@ pub(crate) async fn set_profile_image(
/// between Delta Chat devices.
pub(crate) async fn set_status(
context: &Context,
contact_id: ContactId,
contact_id: u32,
status: String,
encrypted: bool,
has_chat_version: bool,
@@ -1366,7 +1302,7 @@ pub(crate) async fn set_status(
/// Updates last seen timestamp of the contact if it is earlier than the given `timestamp`.
pub(crate) async fn update_last_seen(
context: &Context,
contact_id: ContactId,
contact_id: u32,
timestamp: i64,
) -> Result<()> {
ensure!(
@@ -1390,8 +1326,8 @@ pub(crate) async fn update_last_seen(
/// - Trims the resulting string
///
/// Typically, this function is not needed as it is called implicitly by `Contact::add_address_book`.
pub fn normalize_name(full_name: &str) -> String {
let full_name = full_name.trim();
pub fn normalize_name(full_name: impl AsRef<str>) -> String {
let full_name = full_name.as_ref().trim();
if full_name.is_empty() {
return full_name.into();
}
@@ -1435,16 +1371,16 @@ impl Context {
/// determine whether the specified addr maps to the/a self addr
pub async fn is_self_addr(&self, addr: &str) -> Result<bool> {
if let Some(self_addr) = self.get_config(Config::ConfiguredAddr).await? {
Ok(addr_cmp(&self_addr, addr))
Ok(addr_cmp(self_addr, addr))
} else {
Ok(false)
}
}
}
pub fn addr_cmp(addr1: &str, addr2: &str) -> bool {
let norm1 = addr_normalize(addr1).to_lowercase();
let norm2 = addr_normalize(addr2).to_lowercase();
pub fn addr_cmp(addr1: impl AsRef<str>, addr2: impl AsRef<str>) -> bool {
let norm1 = addr_normalize(addr1.as_ref()).to_lowercase();
let norm2 = addr_normalize(addr2.as_ref()).to_lowercase();
norm1 == norm2
}
@@ -1537,7 +1473,7 @@ mod tests {
Origin::IncomingReplyTo,
)
.await?;
assert_ne!(id, ContactId::new(0));
assert_ne!(id, 0);
let contact = Contact::load_from_db(&context.ctx, id).await.unwrap();
assert_eq!(contact.get_name(), "");
@@ -1589,9 +1525,9 @@ mod tests {
let t = TestContext::new().await;
assert_eq!(t.is_self_addr("me@me.org").await?, false);
t.configure_addr("you@you.net").await;
let addr = t.configure_alice().await;
assert_eq!(t.is_self_addr("me@me.org").await?, false);
assert_eq!(t.is_self_addr("you@you.net").await?, true);
assert_eq!(t.is_self_addr(&addr).await?, true);
Ok(())
}
@@ -1692,7 +1628,7 @@ mod tests {
// check SELF
let contact = Contact::load_from_db(&t, DC_CONTACT_ID_SELF).await.unwrap();
assert_eq!(DC_CONTACT_ID_SELF, ContactId::new(1));
assert_eq!(DC_CONTACT_ID_SELF, 1);
assert_eq!(contact.get_name(), stock_str::self_msg(&t).await);
assert_eq!(contact.get_addr(), ""); // we're not configured
assert!(!contact.is_blocked());
@@ -1954,7 +1890,7 @@ mod tests {
let alice = TestContext::new_alice().await;
let id = Contact::lookup_id_by_addr(&alice.ctx, "alice@example.org", Origin::Unknown)
let id = Contact::lookup_id_by_addr(&alice.ctx, "alice@example.com", Origin::Unknown)
.await
.unwrap();
assert_eq!(id, Some(DC_CONTACT_ID_SELF));
@@ -1979,7 +1915,7 @@ mod tests {
let bob = TestContext::new_bob().await;
let chat_alice = bob
.create_chat_with_contact("Alice", "alice@example.org")
.create_chat_with_contact("Alice", "alice@example.com")
.await;
send_text_msg(&bob, chat_alice.id, "Hello".to_string()).await?;
let msg = bob.pop_sent_msg().await;
@@ -1991,7 +1927,7 @@ mod tests {
"End-to-end encryption preferred.
Fingerprints:
alice@example.org:
alice@example.com:
2E6F A2CB 23B5 32D7 2863
4B58 64B0 8F61 A9ED 9443
@@ -2042,7 +1978,7 @@ CCCB 5AA9 F6E1 141C 9431
// Bob replies.
let chat = bob
.create_chat_with_contact("Alice", "alice@example.org")
.create_chat_with_contact("Alice", "alice@example.com")
.await;
send_text_msg(&bob, chat.id, "Reply".to_string()).await?;
@@ -2093,12 +2029,12 @@ CCCB 5AA9 F6E1 141C 9431
alice1
.evtracker
.get_matching(|e| matches!(e, EventType::SelfavatarChanged))
.get_matching(|e| e == EventType::SelfavatarChanged)
.await;
// Bob sends a message so that Alice can encrypt to him.
let chat = bob
.create_chat_with_contact("Alice", "alice@example.org")
.create_chat_with_contact("Alice", "alice@example.com")
.await;
send_text_msg(&bob, chat.id, "Reply".to_string()).await?;
@@ -2123,7 +2059,7 @@ CCCB 5AA9 F6E1 141C 9431
assert!(alice2.get_config(Config::Selfavatar).await?.is_some());
alice2
.evtracker
.get_matching(|e| matches!(e, EventType::SelfavatarChanged))
.get_matching(|e| e == EventType::SelfavatarChanged)
.await;
Ok(())
@@ -2141,14 +2077,14 @@ CCCB 5AA9 F6E1 141C 9431
let mime = br#"Subject: Hello
Message-ID: message@example.net
To: Alice <alice@example.org>
To: Alice <alice@example.com>
From: Bob <bob@example.net>
Content-Type: text/plain; charset=utf-8; format=flowed; delsp=no
Chat-Version: 1.0
Date: Sun, 22 Mar 2020 22:37:55 +0000
Hi."#;
dc_receive_imf(&alice, mime, "Inbox", false).await?;
dc_receive_imf(&alice, mime, "Inbox", 1, false).await?;
let msg = alice.get_last_msg().await;
let timestamp = msg.get_timestamp();

View File

@@ -24,6 +24,7 @@ use crate::login_param::LoginParam;
use crate::message::{self, MessageState, MsgId};
use crate::quota::QuotaInfo;
use crate::scheduler::Scheduler;
use crate::securejoin::Bob;
use crate::sql::Sql;
#[derive(Clone, Debug)]
@@ -41,9 +42,13 @@ impl Deref for Context {
#[derive(Debug)]
pub struct InnerContext {
/// Database file path
pub(crate) dbfile: PathBuf,
/// Blob directory path
pub(crate) blobdir: PathBuf,
pub(crate) sql: Sql,
pub(crate) os_name: Option<String>,
pub(crate) bob: Bob,
pub(crate) last_smeared_timestamp: RwLock<i64>,
pub(crate) running_state: RwLock<RunningState>,
/// Mutex to avoid generating the key for the user more than once.
@@ -80,7 +85,7 @@ pub struct InnerContext {
#[derive(Debug)]
pub struct RunningState {
ongoing_running: bool,
pub ongoing_running: bool,
shall_stop_ongoing: bool,
cancel_sender: Option<Sender<()>>,
}
@@ -102,19 +107,10 @@ pub fn get_info() -> BTreeMap<&'static str, String> {
}
impl Context {
/// Creates new context and opens the database.
pub async fn new(dbfile: PathBuf, id: u32) -> Result<Context> {
let context = Self::new_closed(dbfile, id).await?;
/// Creates new context.
pub async fn new(os_name: String, dbfile: PathBuf, id: u32) -> Result<Context> {
// pretty_env_logger::try_init_timed().ok();
// Open the database if is not encrypted.
if context.check_passphrase("".to_string()).await? {
context.sql.open(&context, "".to_string()).await?;
}
Ok(context)
}
/// Creates new context without opening the database.
pub async fn new_closed(dbfile: PathBuf, id: u32) -> Result<Context> {
let mut blob_fname = OsString::new();
blob_fname.push(dbfile.file_name().unwrap_or_default());
blob_fname.push("-blobs");
@@ -122,38 +118,11 @@ impl Context {
if !blobdir.exists().await {
async_std::fs::create_dir_all(&blobdir).await?;
}
let context = Context::with_blobdir(dbfile, blobdir, id).await?;
Ok(context)
}
/// Opens the database with the given passphrase.
///
/// Returns true if passphrase is correct, false is passphrase is not correct. Fails on other
/// errors.
pub async fn open(&self, passphrase: String) -> Result<bool> {
if self.sql.check_passphrase(passphrase.clone()).await? {
self.sql.open(self, passphrase).await?;
Ok(true)
} else {
Ok(false)
}
}
/// Returns true if database is open.
pub async fn is_open(&self) -> bool {
self.sql.is_open().await
}
/// Tests the database passphrase.
///
/// Returns true if passphrase is correct.
///
/// Fails if database is already open.
pub(crate) async fn check_passphrase(&self, passphrase: String) -> Result<bool> {
self.sql.check_passphrase(passphrase).await
Context::with_blobdir(os_name, dbfile, blobdir, id).await
}
pub(crate) async fn with_blobdir(
os_name: String,
dbfile: PathBuf,
blobdir: PathBuf,
id: u32,
@@ -167,8 +136,11 @@ impl Context {
let inner = InnerContext {
id,
blobdir,
dbfile,
os_name: Some(os_name),
running_state: RwLock::new(Default::default()),
sql: Sql::new(dbfile),
sql: Sql::new(),
bob: Default::default(),
last_smeared_timestamp: RwLock::new(0),
generating_key_mutex: Mutex::new(()),
oauth2_mutex: Mutex::new(()),
@@ -186,6 +158,7 @@ impl Context {
let ctx = Context {
inner: Arc::new(inner),
};
ctx.sql.open(&ctx, &ctx.dbfile, false).await?;
Ok(ctx)
}
@@ -223,7 +196,7 @@ impl Context {
/// Returns database file path.
pub fn get_dbfile(&self) -> &Path {
self.sql.dbfile.as_path()
self.dbfile.as_path()
}
/// Returns blob directory path.
@@ -254,7 +227,7 @@ impl Context {
// Ongoing process allocation/free/check
pub(crate) async fn alloc_ongoing(&self) -> Result<Receiver<()>> {
pub async fn alloc_ongoing(&self) -> Result<Receiver<()>> {
if self.has_ongoing().await {
bail!("There is already another ongoing process running.");
}
@@ -270,7 +243,7 @@ impl Context {
Ok(receiver)
}
pub(crate) async fn free_ongoing(&self) {
pub async fn free_ongoing(&self) {
let s_a = &self.running_state;
let mut s = s_a.write().await;
@@ -279,7 +252,7 @@ impl Context {
s.cancel_sender.take();
}
pub(crate) async fn has_ongoing(&self) -> bool {
pub async fn has_ongoing(&self) -> bool {
let s_a = &self.running_state;
let s = s_a.read().await;
@@ -304,7 +277,7 @@ impl Context {
};
}
pub(crate) async fn shall_stop_ongoing(&self) -> bool {
pub async fn shall_stop_ongoing(&self) -> bool {
self.running_state.read().await.shall_stop_ongoing
}
@@ -334,6 +307,7 @@ impl Context {
.await?
.unwrap_or_else(|| "unknown".to_string());
let e2ee_enabled = self.get_config_int(Config::E2eeEnabled).await?;
let e2ee_force = self.get_config_int(Config::E2eeForce).await?;
let mdns_enabled = self.get_config_int(Config::MdnsEnabled).await?;
let bcc_self = self.get_config_int(Config::BccSelf).await?;
let send_sync_msgs = self.get_config_int(Config::SendSyncMsgs).await?;
@@ -352,9 +326,11 @@ impl Context {
Err(err) => format!("<key failure: {}>", err),
};
let inbox_watch = self.get_config_int(Config::InboxWatch).await?;
let sentbox_watch = self.get_config_int(Config::SentboxWatch).await?;
let mvbox_watch = self.get_config_int(Config::MvboxWatch).await?;
let mvbox_move = self.get_config_int(Config::MvboxMove).await?;
let only_fetch_mvbox = self.get_config_int(Config::OnlyFetchMvbox).await?;
let sentbox_move = self.get_config_int(Config::SentboxMove).await?;
let folders_configured = self
.sql
.get_raw_config_int("folders_configured")
@@ -380,13 +356,6 @@ impl Context {
res.insert("number_of_contacts", contacts.to_string());
res.insert("database_dir", self.get_dbfile().display().to_string());
res.insert("database_version", dbversion.to_string());
res.insert(
"database_encrypted",
self.sql
.is_encrypted()
.await
.map_or_else(|| "closed".to_string(), |b| b.to_string()),
);
res.insert("journal_mode", journal_mode);
res.insert("blobdir", self.get_blobdir().display().to_string());
res.insert("display_name", displayname.unwrap_or_else(|| unset.into()));
@@ -416,14 +385,17 @@ impl Context {
.await?
.to_string(),
);
res.insert("inbox_watch", inbox_watch.to_string());
res.insert("sentbox_watch", sentbox_watch.to_string());
res.insert("mvbox_watch", mvbox_watch.to_string());
res.insert("mvbox_move", mvbox_move.to_string());
res.insert("only_fetch_mvbox", only_fetch_mvbox.to_string());
res.insert("sentbox_move", sentbox_move.to_string());
res.insert("folders_configured", folders_configured.to_string());
res.insert("configured_sentbox_folder", configured_sentbox_folder);
res.insert("configured_mvbox_folder", configured_mvbox_folder);
res.insert("mdns_enabled", mdns_enabled.to_string());
res.insert("e2ee_enabled", e2ee_enabled.to_string());
res.insert("e2ee_force", e2ee_force.to_string());
res.insert(
"key_gen_type",
self.get_config_int(Config::KeyGenType).await?.to_string(),
@@ -611,14 +583,14 @@ impl Context {
Ok(spam.as_deref() == Some(folder_name))
}
pub(crate) fn derive_blobdir(dbfile: &PathBuf) -> PathBuf {
pub fn derive_blobdir(dbfile: &PathBuf) -> PathBuf {
let mut blob_fname = OsString::new();
blob_fname.push(dbfile.file_name().unwrap_or_default());
blob_fname.push("-blobs");
dbfile.with_file_name(blob_fname)
}
pub(crate) fn derive_walfile(dbfile: &PathBuf) -> PathBuf {
pub fn derive_walfile(dbfile: &PathBuf) -> PathBuf {
let mut wal_fname = OsString::new();
wal_fname.push(dbfile.file_name().unwrap_or_default());
wal_fname.push("-wal");
@@ -670,26 +642,21 @@ mod tests {
use crate::chat::{
get_chat_contacts, get_chat_msgs, send_msg, set_muted, Chat, ChatId, MuteDuration,
};
use crate::constants::DC_CONTACT_ID_SELF;
use crate::constants::{Viewtype, DC_CONTACT_ID_SELF};
use crate::dc_receive_imf::dc_receive_imf;
use crate::dc_tools::dc_create_outgoing_rfc724_mid;
use crate::message::{Message, Viewtype};
use crate::message::Message;
use crate::test_utils::TestContext;
use anyhow::Context as _;
use std::time::Duration;
use strum::IntoEnumIterator;
use tempfile::tempdir;
#[async_std::test]
async fn test_wrong_db() -> Result<()> {
let tmp = tempfile::tempdir()?;
async fn test_wrong_db() {
let tmp = tempfile::tempdir().unwrap();
let dbfile = tmp.path().join("db.sqlite");
std::fs::write(&dbfile, b"123")?;
let res = Context::new(dbfile.into(), 1).await?;
// Broken database is indistinguishable from encrypted one.
assert_eq!(res.is_open().await, false);
Ok(())
std::fs::write(&dbfile, b"123").unwrap();
let res = Context::new("FakeOs".into(), dbfile.into(), 1).await;
assert!(res.is_err());
}
#[async_std::test]
@@ -706,7 +673,7 @@ mod tests {
.unwrap();
let msg = format!(
"From: {}\n\
To: alice@example.org\n\
To: alice@example.com\n\
Message-ID: <{}>\n\
Chat-Version: 1.0\n\
Date: Sun, 22 Mar 2020 22:37:57 +0000\n\
@@ -716,7 +683,7 @@ mod tests {
dc_create_outgoing_rfc724_mid(None, contact.get_addr())
);
println!("{}", msg);
dc_receive_imf(t, msg.as_bytes(), "INBOX", false)
dc_receive_imf(t, msg.as_bytes(), "INBOX", 1, false)
.await
.unwrap();
}
@@ -839,7 +806,9 @@ mod tests {
async fn test_blobdir_exists() {
let tmp = tempfile::tempdir().unwrap();
let dbfile = tmp.path().join("db.sqlite");
Context::new(dbfile.into(), 1).await.unwrap();
Context::new("FakeOS".into(), dbfile.into(), 1)
.await
.unwrap();
let blobdir = tmp.path().join("db.sqlite-blobs");
assert!(blobdir.is_dir());
}
@@ -850,7 +819,7 @@ mod tests {
let dbfile = tmp.path().join("db.sqlite");
let blobdir = tmp.path().join("db.sqlite-blobs");
std::fs::write(&blobdir, b"123").unwrap();
let res = Context::new(dbfile.into(), 1).await;
let res = Context::new("FakeOS".into(), dbfile.into(), 1).await;
assert!(res.is_err());
}
@@ -860,7 +829,9 @@ mod tests {
let subdir = tmp.path().join("subdir");
let dbfile = subdir.join("db.sqlite");
let dbfile2 = dbfile.clone();
Context::new(dbfile.into(), 1).await.unwrap();
Context::new("FakeOS".into(), dbfile.into(), 1)
.await
.unwrap();
assert!(subdir.is_dir());
assert!(dbfile2.is_file());
}
@@ -870,7 +841,7 @@ mod tests {
let tmp = tempfile::tempdir().unwrap();
let dbfile = tmp.path().join("db.sqlite");
let blobdir = PathBuf::new();
let res = Context::with_blobdir(dbfile.into(), blobdir, 1).await;
let res = Context::with_blobdir("FakeOS".into(), dbfile.into(), blobdir, 1).await;
assert!(res.is_err());
}
@@ -879,7 +850,7 @@ mod tests {
let tmp = tempfile::tempdir().unwrap();
let dbfile = tmp.path().join("db.sqlite");
let blobdir = tmp.path().join("blobs");
let res = Context::with_blobdir(dbfile.into(), blobdir.into(), 1).await;
let res = Context::with_blobdir("FakeOS".into(), dbfile.into(), blobdir.into(), 1).await;
assert!(res.is_err());
}
@@ -1041,29 +1012,4 @@ mod tests {
Ok(())
}
#[async_std::test]
async fn test_check_passphrase() -> Result<()> {
let dir = tempdir()?;
let dbfile = dir.path().join("db.sqlite");
let id = 1;
let context = Context::new_closed(dbfile.clone().into(), id)
.await
.context("failed to create context")?;
assert_eq!(context.open("foo".to_string()).await?, true);
assert_eq!(context.is_open().await, true);
drop(context);
let id = 2;
let context = Context::new(dbfile.into(), id)
.await
.context("failed to create context")?;
assert_eq!(context.is_open().await, false);
assert_eq!(context.check_passphrase("bar".to_string()).await?, false);
assert_eq!(context.open("false".to_string()).await?, false);
assert_eq!(context.open("foo".to_string()).await?, true);
Ok(())
}
}

File diff suppressed because it is too large Load Diff

View File

@@ -5,7 +5,6 @@ use core::cmp::{max, min};
use std::borrow::Cow;
use std::fmt;
use std::io::Cursor;
use std::str::from_utf8;
use std::str::FromStr;
use std::time::{Duration, SystemTime};
@@ -13,18 +12,15 @@ use async_std::path::{Path, PathBuf};
use async_std::prelude::*;
use async_std::{fs, io};
use anyhow::Error;
use anyhow::{bail, Error};
use chrono::{Local, TimeZone};
use mailparse::dateparse;
use mailparse::headers::Headers;
use mailparse::MailHeaderMap;
use rand::{thread_rng, Rng};
use crate::chat::{add_device_msg, add_device_msg_with_importance};
use crate::constants::{DC_ELLIPSIS, DC_OUTDATED_WARNING_DAYS};
use crate::constants::{Viewtype, DC_ELLIPSIS, DC_OUTDATED_WARNING_DAYS};
use crate::context::Context;
use crate::events::EventType;
use crate::message::{Message, Viewtype};
use crate::message::Message;
use crate::provider::get_provider_update_timestamp;
use crate::stock_str;
@@ -71,7 +67,7 @@ pub(crate) fn dc_gm2local_offset() -> i64 {
/* returns the offset that must be _added_ to an UTC/GMT-time to create the localtime.
the function may return negative values. */
let lt = Local::now();
i64::from(lt.offset().local_minus_utc())
lt.offset().local_minus_utc() as i64
}
// timesmearing
@@ -195,26 +191,44 @@ async fn maybe_warn_on_outdated(context: &Context, now: i64, approx_compile_time
}
/* Message-ID tools */
/// Generate an ID. The generated ID should be as short and as unique as possible:
/// - short, because it may also used as part of Message-ID headers or in QR codes
/// - unique as two IDs generated on two devices should not be the same. However, collisions are not world-wide but only by the few contacts.
/// IDs generated by this function are 66 bit wide and are returned as 11 base64 characters.
///
/// Additional information when used as a message-id or group-id:
/// - for OUTGOING messages this ID is written to the header as `Chat-Group-ID:` and is added to the message ID as Gr.<grpid>.<random>@<random>
/// - for INCOMING messages, the ID is taken from the Chat-Group-ID-header or from the Message-ID in the In-Reply-To: or References:-Header
/// - the group-id should be a string with the characters [a-zA-Z0-9\-_]
pub(crate) fn dc_create_id() -> String {
// ThreadRng implements CryptoRng trait and is supposed to be cryptographically secure.
/* generate an id. the generated ID should be as short and as unique as possible:
- short, because it may also used as part of Message-ID headers or in QR codes
- unique as two IDs generated on two devices should not be the same. However, collisions are not world-wide but only by the few contacts.
IDs generated by this function are 66 bit wide and are returned as 11 base64 characters.
If possible, RNG of OpenSSL is used.
Additional information when used as a message-id or group-id:
- for OUTGOING messages this ID is written to the header as `Chat-Group-ID:` and is added to the message ID as Gr.<grpid>.<random>@<random>
- for INCOMING messages, the ID is taken from the Chat-Group-ID-header or from the Message-ID in the In-Reply-To: or References:-Header
- the group-id should be a string with the characters [a-zA-Z0-9\-_] */
let mut rng = thread_rng();
let buf: [u32; 3] = [rng.gen(), rng.gen(), rng.gen()];
// Generate 72 random bits.
let mut arr = [0u8; 9];
rng.fill(&mut arr[..]);
encode_66bits_as_base64(buf[0usize], buf[1usize], buf[2usize])
}
// Take 11 base64 characters containing 66 random bits.
base64::encode(&arr).chars().take(11).collect()
/// Encode 66 bits as a base64 string.
/// This is useful for ID generating with short strings as we save 5 character
/// in each id compared to 64 bit hex encoding. For a typical group ID, these
/// are 10 characters (grpid+msgid):
/// hex: 64 bit, 4 bits/character, length = 64/4 = 16 characters
/// base64: 64 bit, 6 bits/character, length = 64/6 = 11 characters (plus 2 additional bits)
/// Only the lower 2 bits of `fill` are used.
fn encode_66bits_as_base64(v1: u32, v2: u32, fill: u32) -> String {
use byteorder::{BigEndian, WriteBytesExt};
let mut wrapped_writer = Vec::new();
{
let mut enc = base64::write::EncoderWriter::new(&mut wrapped_writer, base64::URL_SAFE);
enc.write_u32::<BigEndian>(v1).unwrap();
enc.write_u32::<BigEndian>(v2).unwrap();
enc.write_u8(((fill & 0x3) as u8) << 6).unwrap();
enc.finish().unwrap();
}
assert_eq!(wrapped_writer.pop(), Some(b'A')); // Remove last "A"
String::from_utf8(wrapped_writer).unwrap()
}
/// Function generates a Message-ID that can be used for a new outgoing message.
@@ -336,6 +350,63 @@ pub async fn dc_delete_files_in_dir(context: &Context, path: impl AsRef<Path>) {
}
}
pub(crate) async fn dc_copy_file(
context: &Context,
src_path: impl AsRef<Path>,
dest_path: impl AsRef<Path>,
) -> bool {
let src_abs = dc_get_abs_path(context, &src_path);
let mut src_file = match fs::File::open(&src_abs).await {
Ok(file) => file,
Err(err) => {
warn!(
context,
"failed to open for read '{}': {}",
src_abs.display(),
err
);
return false;
}
};
let dest_abs = dc_get_abs_path(context, &dest_path);
let mut dest_file = match fs::OpenOptions::new()
.create_new(true)
.write(true)
.open(&dest_abs)
.await
{
Ok(file) => file,
Err(err) => {
warn!(
context,
"failed to open for write '{}': {}",
dest_abs.display(),
err
);
return false;
}
};
match io::copy(&mut src_file, &mut dest_file).await {
Ok(_) => true,
Err(err) => {
error!(
context,
"Cannot copy \"{}\" to \"{}\": {}",
src_abs.display(),
dest_abs.display(),
err
);
{
// Attempt to remove the failed file, swallow errors resulting from that.
fs::remove_file(dest_abs).await.ok();
}
false
}
}
}
pub(crate) async fn dc_create_folder(
context: &Context,
path: impl AsRef<Path>,
@@ -433,6 +504,33 @@ pub fn dc_open_file_std<P: AsRef<std::path::Path>>(
}
}
/// Returns Ok((temp_path, dest_path)) on success. The backup can then be written to temp_path. If the backup succeeded,
/// it can be renamed to dest_path. This guarantees that the backup is complete.
pub(crate) async fn get_next_backup_path(
folder: impl AsRef<Path>,
backup_time: i64,
) -> Result<(PathBuf, PathBuf), Error> {
let folder = PathBuf::from(folder.as_ref());
let stem = chrono::NaiveDateTime::from_timestamp(backup_time, 0)
// Don't change this file name format, in has_backup() we use string comparison to determine which backup is newer:
.format("delta-chat-backup-%Y-%m-%d")
.to_string();
// 64 backup files per day should be enough for everyone
for i in 0..64 {
let mut tempfile = folder.clone();
tempfile.push(format!("{}-{:02}.tar.part", stem, i));
let mut destfile = folder.clone();
destfile.push(format!("{}-{:02}.tar", stem, i));
if !tempfile.exists().await && !destfile.exists().await {
return Ok((tempfile, destfile));
}
}
bail!("could not create backup file, disk full?");
}
pub(crate) fn time() -> i64 {
SystemTime::now()
.duration_since(SystemTime::UNIX_EPOCH)
@@ -534,8 +632,8 @@ impl rusqlite::types::ToSql for EmailAddress {
/// Makes sure that a user input that is not supposed to contain newlines does not contain newlines.
pub(crate) fn improve_single_line_input(input: &str) -> String {
input
.replace('\n', " ")
.replace('\r', " ")
.replace("\n", " ")
.replace("\r", " ")
.trim()
.to_string()
}
@@ -572,144 +670,13 @@ pub fn remove_subject_prefix(last_subject: &str) -> String {
.to_string()
}
// Types and methods to create hop-info for message-info
fn extract_address_from_receive_header<'a>(header: &'a str, start: &str) -> Option<&'a str> {
let header_len = header.len();
header.find(start).and_then(|mut begin| {
begin += start.len();
let end = header
.get(begin..)?
.find(|c: char| c.is_whitespace())
.unwrap_or(header_len);
header.get(begin..begin + end)
})
}
pub(crate) fn parse_receive_header(header: &str) -> String {
let header = header.replace(&['\r', '\n'][..], "");
let mut hop_info = String::from("Hop: ");
if let Some(from) = extract_address_from_receive_header(&header, "from ") {
hop_info += &format!("From: {}; ", from.trim());
}
if let Some(by) = extract_address_from_receive_header(&header, "by ") {
hop_info += &format!("By: {}; ", by.trim());
}
if let Ok(date) = dateparse(&header) {
// In tests, use the UTC timezone so that the test is reproducible
#[cfg(test)]
let date_obj = chrono::Utc.timestamp(date, 0);
#[cfg(not(test))]
let date_obj = Local.timestamp(date, 0);
hop_info += &format!("Date: {}", date_obj.to_rfc2822());
};
hop_info
}
/// parses "receive"-headers
pub(crate) fn parse_receive_headers(headers: &Headers) -> String {
headers
.get_all_headers("Received")
.iter()
.rev()
.filter_map(|header_map_item| from_utf8(header_map_item.get_value_raw()).ok())
.map(parse_receive_header)
.collect::<Vec<_>>()
.join("\n")
}
#[cfg(test)]
mod tests {
#![allow(clippy::indexing_slicing)]
use super::*;
use crate::{
config::Config, dc_receive_imf::dc_receive_imf, message::get_msg_info,
test_utils::TestContext,
};
#[test]
fn test_parse_receive_headers() {
// Test `parse_receive_headers()` with some more-or-less random emails from the test-data
let raw = include_bytes!("../test-data/message/mail_with_cc.txt");
let expected =
"Hop: From: localhost; By: hq5.merlinux.eu; Date: Sat, 14 Sep 2019 17:00:22 +0000\n\
Hop: From: hq5.merlinux.eu; By: hq5.merlinux.eu; Date: Sat, 14 Sep 2019 17:00:25 +0000";
check_parse_receive_headers(raw, expected);
let raw = include_bytes!("../test-data/message/wrong-html.eml");
let expected =
"Hop: From: oxbsltgw18.schlund.de; By: mrelayeu.kundenserver.de; Date: Thu, 06 Aug 2020 16:40:31 +0000\n\
Hop: From: mout.kundenserver.de; By: dd37930.kasserver.com; Date: Thu, 06 Aug 2020 16:40:32 +0000";
check_parse_receive_headers(raw, expected);
let raw = include_bytes!("../test-data/message/posteo_ndn.eml");
let expected =
"Hop: By: mout01.posteo.de; Date: Tue, 09 Jun 2020 18:44:22 +0000\n\
Hop: From: mout01.posteo.de; By: mx04.posteo.de; Date: Tue, 09 Jun 2020 18:44:22 +0000\n\
Hop: From: mx04.posteo.de; By: mailin06.posteo.de; Date: Tue, 09 Jun 2020 18:44:23 +0000\n\
Hop: From: mailin06.posteo.de; By: proxy02.posteo.de; Date: Tue, 09 Jun 2020 18:44:23 +0000\n\
Hop: From: proxy02.posteo.de; By: proxy02.posteo.name; Date: Tue, 09 Jun 2020 18:44:23 +0000\n\
Hop: From: proxy02.posteo.name; By: dovecot03.posteo.local; Date: Tue, 09 Jun 2020 18:44:24 +0000";
check_parse_receive_headers(raw, expected);
}
fn check_parse_receive_headers(raw: &[u8], expected: &str) {
let mail = mailparse::parse_mail(raw).unwrap();
let hop_info = parse_receive_headers(&mail.get_headers());
assert_eq!(hop_info, expected)
}
#[async_std::test]
async fn test_parse_receive_headers_integration() {
let raw = include_bytes!("../test-data/message/mail_with_cc.txt");
let expected = r"State: Fresh
hi
Message-ID: 2dfdbde7@example.org
Hop: From: localhost; By: hq5.merlinux.eu; Date: Sat, 14 Sep 2019 17:00:22 +0000
Hop: From: hq5.merlinux.eu; By: hq5.merlinux.eu; Date: Sat, 14 Sep 2019 17:00:25 +0000";
check_parse_receive_headers_integration(raw, expected).await;
let raw = include_bytes!("../test-data/message/encrypted_with_received_headers.eml");
let expected = "State: Fresh, Encrypted
Re: Message from alice@example.org
hi back\r\n\
\r\n\
-- \r\n\
Sent with my Delta Chat Messenger: https://delta.chat
Message-ID: Mr.adQpEwndXLH.LPDdlFVJ7wG@example.net
Hop: From: [127.0.0.1]; By: mail.example.org; Date: Mon, 27 Dec 2021 11:21:21 +0000
Hop: From: mout.example.org; By: hq5.example.org; Date: Mon, 27 Dec 2021 11:21:22 +0000
Hop: From: hq5.example.org; By: hq5.example.org; Date: Mon, 27 Dec 2021 11:21:22 +0000";
check_parse_receive_headers_integration(raw, expected).await;
}
async fn check_parse_receive_headers_integration(raw: &[u8], expected: &str) {
let t = TestContext::new_alice().await;
t.set_config(Config::ShowEmails, Some("2")).await.unwrap();
dc_receive_imf(&t, raw, "INBOX", false).await.unwrap();
let msg = t.get_last_msg().await;
let msg_info = get_msg_info(&t, msg.id).await.unwrap();
// Ignore the first rows of the msg_info because they contain a
// received time that depends on the test time which makes it impossible to
// compare with a static string
let capped_result = &msg_info[msg_info.find("State").unwrap()..];
assert_eq!(expected, capped_result);
}
use crate::test_utils::TestContext;
#[test]
fn test_rust_ftoa() {
@@ -762,6 +729,26 @@ Hop: From: hq5.example.org; By: hq5.example.org; Date: Mon, 27 Dec 2021 11:21:22
assert_eq!(buf.len(), 11);
}
#[test]
fn test_encode_66bits_as_base64() {
assert_eq!(
encode_66bits_as_base64(0x01234567, 0x89abcdef, 0),
"ASNFZ4mrze8"
);
assert_eq!(
encode_66bits_as_base64(0x01234567, 0x89abcdef, 1),
"ASNFZ4mrze9"
);
assert_eq!(
encode_66bits_as_base64(0x01234567, 0x89abcdef, 2),
"ASNFZ4mrze-"
);
assert_eq!(
encode_66bits_as_base64(0x01234567, 0x89abcdef, 3),
"ASNFZ4mrze_"
);
}
#[test]
fn test_dc_extract_grpid_from_rfc724_mid() {
// Should return None if we pass invalid mid
@@ -903,7 +890,20 @@ Hop: From: hq5.example.org; By: hq5.example.org; Date: Mon, 27 Dec 2021 11:21:22
assert!(dc_file_exist!(context, &abs_path).await);
assert!(dc_copy_file(context, "$BLOBDIR/foobar", "$BLOBDIR/dada").await);
// attempting to copy a second time should fail
assert!(!dc_copy_file(context, "$BLOBDIR/foobar", "$BLOBDIR/dada").await);
assert_eq!(dc_get_filebytes(context, "$BLOBDIR/dada").await, 7);
let buf = dc_read_file(context, "$BLOBDIR/dada").await.unwrap();
assert_eq!(buf.len(), 7);
assert_eq!(&buf, b"content");
assert!(dc_delete_file(context, "$BLOBDIR/foobar").await);
assert!(dc_delete_file(context, "$BLOBDIR/dada").await);
assert!(dc_create_folder(context, "$BLOBDIR/foobar-folder")
.await
.is_ok());
@@ -1029,7 +1029,7 @@ Hop: From: hq5.example.org; By: hq5.example.org; Date: Mon, 27 Dec 2021 11:21:22
maybe_warn_on_bad_time(&t, timestamp_past, get_provider_update_timestamp()).await;
let chats = Chatlist::try_load(&t, 0, None, None).await.unwrap();
assert_eq!(chats.len(), 1);
let device_chat_id = chats.get_chat_id(0).unwrap();
let device_chat_id = chats.get_chat_id(0);
let msgs = chat::get_chat_msgs(&t, device_chat_id, 0, None)
.await
.unwrap();
@@ -1067,7 +1067,7 @@ Hop: From: hq5.example.org; By: hq5.example.org; Date: Mon, 27 Dec 2021 11:21:22
.await;
let chats = Chatlist::try_load(&t, 0, None, None).await.unwrap();
assert_eq!(chats.len(), 1);
assert_eq!(device_chat_id, chats.get_chat_id(0).unwrap());
assert_eq!(device_chat_id, chats.get_chat_id(0));
let msgs = chat::get_chat_msgs(&t, device_chat_id, 0, None)
.await
.unwrap();
@@ -1099,7 +1099,7 @@ Hop: From: hq5.example.org; By: hq5.example.org; Date: Mon, 27 Dec 2021 11:21:22
.await;
let chats = Chatlist::try_load(&t, 0, None, None).await.unwrap();
assert_eq!(chats.len(), 1);
let device_chat_id = chats.get_chat_id(0).unwrap();
let device_chat_id = chats.get_chat_id(0);
let msgs = chat::get_chat_msgs(&t, device_chat_id, 0, None)
.await
.unwrap();
@@ -1121,7 +1121,7 @@ Hop: From: hq5.example.org; By: hq5.example.org; Date: Mon, 27 Dec 2021 11:21:22
.await;
let chats = Chatlist::try_load(&t, 0, None, None).await.unwrap();
assert_eq!(chats.len(), 1);
let device_chat_id = chats.get_chat_id(0).unwrap();
let device_chat_id = chats.get_chat_id(0);
let msgs = chat::get_chat_msgs(&t, device_chat_id, 0, None)
.await
.unwrap();
@@ -1138,7 +1138,7 @@ Hop: From: hq5.example.org; By: hq5.example.org; Date: Mon, 27 Dec 2021 11:21:22
.await;
let chats = Chatlist::try_load(&t, 0, None, None).await.unwrap();
assert_eq!(chats.len(), 1);
let device_chat_id = chats.get_chat_id(0).unwrap();
let device_chat_id = chats.get_chat_id(0);
let msgs = chat::get_chat_msgs(&t, device_chat_id, 0, None)
.await
.unwrap();

View File

@@ -36,9 +36,9 @@ impl Dehtml {
""
}
}
fn append_prefix(&self, line_end: &str) -> String {
fn append_prefix(&self, line_end: impl AsRef<str>) -> String {
// line_end is e.g. "\n\n". We add "> " if necessary.
line_end.to_string() + self.line_prefix()
line_end.as_ref().to_owned() + self.line_prefix()
}
fn get_add_text(&self) -> AddText {
if self.divs_since_quote_div > 0 && self.divs_since_quoted_content_div == 0 {

View File

@@ -3,14 +3,14 @@
use anyhow::{anyhow, Result};
use deltachat_derive::{FromSql, ToSql};
use serde::{Deserialize, Serialize};
use std::collections::BTreeMap;
use crate::config::Config;
use crate::constants::Viewtype;
use crate::context::Context;
use crate::dc_tools::time;
use crate::imap::{Imap, ImapActionResult};
use crate::job::{self, Action, Job, Status};
use crate::message::{Message, MsgId, Viewtype};
use crate::message::{Message, MsgId};
use crate::mimeparser::{MimeMessage, Part};
use crate::param::Params;
use crate::{job_try, stock_str, EventType};
@@ -129,48 +129,24 @@ impl Job {
}
let msg = job_try!(Message::load_from_db(context, MsgId::new(self.foreign_id)).await);
let row = job_try!(
context
.sql
.query_row_optional(
"SELECT uid, folder FROM imap WHERE rfc724_mid=? AND target!=''",
paramsv![msg.rfc724_mid],
|row| {
let server_uid: u32 = row.get(0)?;
let server_folder: String = row.get(1)?;
Ok((server_uid, server_folder))
}
)
.await
);
if let Some((server_uid, server_folder)) = row {
match imap
.fetch_single_msg(context, &server_folder, server_uid, msg.rfc724_mid.clone())
.await
{
ImapActionResult::RetryLater | ImapActionResult::Failed => {
job_try!(
msg.id
.update_download_state(context, DownloadState::Failure)
.await
);
Status::Finished(Err(anyhow!("Call download_full() again to try over.")))
}
ImapActionResult::Success => {
// update_download_state() not needed as receive_imf() already
// set the state and emitted the event.
Status::Finished(Ok(()))
}
let server_folder = msg.server_folder.unwrap_or_default();
match imap
.fetch_single_msg(context, &server_folder, msg.server_uid)
.await
{
ImapActionResult::RetryLater | ImapActionResult::Failed => {
job_try!(
msg.id
.update_download_state(context, DownloadState::Failure)
.await
);
Status::Finished(Err(anyhow!("Call download_full() again to try over.")))
}
ImapActionResult::Success | ImapActionResult::AlreadyDone => {
// update_download_state() not needed as receive_imf() already
// set the state and emitted the event.
Status::Finished(Ok(()))
}
} else {
// No IMAP record found, we don't know the UID and folder.
job_try!(
msg.id
.update_download_state(context, DownloadState::Failure)
.await
);
Status::Finished(Err(anyhow!("Call download_full() again to try over.")))
}
}
}
@@ -185,7 +161,6 @@ impl Imap {
context: &Context,
folder: &str,
uid: u32,
rfc724_mid: String,
) -> ImapActionResult {
if let Some(imapresult) = self
.prepare_imap_operation_on_msg(context, folder, uid)
@@ -197,20 +172,14 @@ impl Imap {
// we are connected, and the folder is selected
info!(context, "Downloading message {}/{} fully...", folder, uid);
let mut uid_message_ids: BTreeMap<u32, String> = BTreeMap::new();
uid_message_ids.insert(uid, rfc724_mid);
let (last_uid, _received) = match self
.fetch_many_msgs(context, folder, vec![uid], &uid_message_ids, false, false)
.await
{
Ok(res) => res,
Err(_) => return ImapActionResult::Failed,
};
if last_uid.is_none() {
ImapActionResult::Failed
} else {
ImapActionResult::Success
let (_, error_cnt) = self
.fetch_many_msgs(context, folder, vec![uid], false, false)
.await;
if error_cnt > 0 {
return ImapActionResult::Failed;
}
ImapActionResult::Success
}
}
@@ -254,15 +223,13 @@ impl MimeMessage {
#[cfg(test)]
mod tests {
use num_traits::FromPrimitive;
use super::*;
use crate::chat::send_msg;
use crate::constants::Viewtype;
use crate::dc_receive_imf::dc_receive_imf_inner;
use crate::ephemeral::Timer;
use crate::message::Viewtype;
use crate::test_utils::TestContext;
use super::*;
use num_traits::FromPrimitive;
#[test]
fn test_downloadstate_values() {
@@ -344,9 +311,9 @@ mod tests {
dc_receive_imf_inner(
&t,
"Mr.12345678901@example.com",
header.as_bytes(),
"INBOX",
1,
false,
Some(100000),
false,
@@ -362,9 +329,9 @@ mod tests {
dc_receive_imf_inner(
&t,
"Mr.12345678901@example.com",
format!("{}\n\n100k text...", header).as_bytes(),
"INBOX",
1,
false,
None,
false,
@@ -392,15 +359,15 @@ mod tests {
// download message from bob partially, this must not change the ephemeral timer
dc_receive_imf_inner(
&t,
"first@example.org",
b"From: Bob <bob@example.org>\n\
To: Alice <alice@example.org>\n\
To: Alice <alice@example.com>\n\
Chat-Version: 1.0\n\
Subject: subject\n\
Message-ID: <first@example.org>\n\
Date: Sun, 14 Nov 2021 00:10:00 +0000\
Content-Type: text/plain",
"INBOX",
1,
false,
Some(100000),
false,

View File

@@ -2,7 +2,7 @@
use std::collections::HashSet;
use anyhow::{bail, format_err, Context as _, Result};
use anyhow::{bail, ensure, format_err, Result};
use mailparse::ParsedMail;
use num_traits::FromPrimitive;
@@ -19,6 +19,7 @@ use crate::pgp;
#[derive(Debug)]
pub struct EncryptHelper {
pub prefer_encrypt: EncryptPreference,
force_preference: bool,
pub addr: String,
pub public_key: SignedPublicKey,
}
@@ -28,6 +29,7 @@ impl EncryptHelper {
let prefer_encrypt =
EncryptPreference::from_i32(context.get_config_int(Config::E2eeEnabled).await?)
.unwrap_or_default();
let force_preference = context.get_config_bool(Config::E2eeForce).await?;
let addr = match context.get_config(Config::ConfiguredAddr).await? {
None => {
bail!("addr not configured!");
@@ -39,6 +41,7 @@ impl EncryptHelper {
Ok(EncryptHelper {
prefer_encrypt,
force_preference,
addr,
public_key,
})
@@ -100,11 +103,17 @@ impl EncryptHelper {
}
}
// Count number of recipients, including self.
// This does not depend on whether we send a copy to self or not.
let recipients_count = peerstates.len() + 1;
let want_encrypt = if self.force_preference {
// Ignore preferences of others.
self.prefer_encrypt == EncryptPreference::Mutual
} else {
// Count number of recipients, including self.
// This does not depend on whether we send a copy to self or not.
let recipients_count = peerstates.len() + 1;
2 * prefer_encrypt_count > recipients_count
};
Ok(e2ee_guaranteed || 2 * prefer_encrypt_count > recipients_count)
Ok(e2ee_guaranteed || want_encrypt)
}
/// Tries to encrypt the passed in `mail`.
@@ -121,9 +130,9 @@ impl EncryptHelper {
.into_iter()
.filter_map(|(state, addr)| state.map(|s| (s, addr)))
{
let key = peerstate
.take_key(min_verified)
.with_context(|| format!("proper enc-key for {} missing, cannot encrypt", addr))?;
let key = peerstate.take_key(min_verified).ok_or_else(|| {
format_err!("proper enc-key for {} missing, cannot encrypt", addr)
})?;
keyring.add(key);
}
keyring.add(self.public_key.clone());
@@ -179,6 +188,7 @@ pub async fn try_decrypt(
// Possibly perform decryption
let private_keyring: Keyring<SignedSecretKey> = Keyring::new_self(context).await?;
let mut public_keyring_for_validate: Keyring<SignedPublicKey> = Keyring::new();
let mut signatures = HashSet::default();
if let Some(ref mut peerstate) = peerstate {
peerstate
@@ -191,17 +201,14 @@ pub async fn try_decrypt(
}
}
let (out_mail, signatures) = match decrypt_if_autocrypt_message(
let out_mail = decrypt_if_autocrypt_message(
context,
mail,
private_keyring,
public_keyring_for_validate,
&mut signatures,
)
.await?
{
Some((out_mail, signatures)) => (Some(out_mail), signatures),
None => (None, Default::default()),
};
.await?;
if let Some(mut peerstate) = peerstate {
// If message is not encrypted and it is not a read receipt, degrade encryption.
@@ -277,7 +284,8 @@ async fn decrypt_if_autocrypt_message(
mail: &ParsedMail<'_>,
private_keyring: Keyring<SignedSecretKey>,
public_keyring_for_validate: Keyring<SignedPublicKey>,
) -> Result<Option<(Vec<u8>, HashSet<Fingerprint>)>> {
ret_valid_signatures: &mut HashSet<Fingerprint>,
) -> Result<Option<Vec<u8>>> {
let encrypted_data_part = match get_autocrypt_mime(mail).or_else(|| get_mixed_up_mime(mail)) {
None => {
// not an autocrypt mime message, abort and ignore
@@ -291,60 +299,36 @@ async fn decrypt_if_autocrypt_message(
encrypted_data_part,
private_keyring,
public_keyring_for_validate,
ret_valid_signatures,
)
.await
}
/// Validates signatures of Multipart/Signed message part, as defined in RFC 1847.
///
/// Returns `None` if the part is not a Multipart/Signed part, otherwise retruns the set of key
/// fingerprints for which there is a valid signature.
async fn validate_detached_signature(
mail: &ParsedMail<'_>,
public_keyring_for_validate: &Keyring<SignedPublicKey>,
) -> Result<Option<(Vec<u8>, HashSet<Fingerprint>)>> {
if mail.ctype.mimetype != "multipart/signed" {
return Ok(None);
}
if let [first_part, second_part] = &mail.subparts[..] {
// First part is the content, second part is the signature.
let content = first_part.raw_bytes;
let signature = second_part.get_body_raw()?;
let ret_valid_signatures =
pgp::pk_validate(content, &signature, public_keyring_for_validate).await?;
Ok(Some((content.to_vec(), ret_valid_signatures)))
} else {
Ok(None)
}
}
/// Returns Ok(None) if nothing encrypted was found.
async fn decrypt_part(
mail: &ParsedMail<'_>,
private_keyring: Keyring<SignedSecretKey>,
public_keyring_for_validate: Keyring<SignedPublicKey>,
) -> Result<Option<(Vec<u8>, HashSet<Fingerprint>)>> {
ret_valid_signatures: &mut HashSet<Fingerprint>,
) -> Result<Option<Vec<u8>>> {
let data = mail.get_body_raw()?;
if has_decrypted_pgp_armor(&data) {
let (plain, ret_valid_signatures) =
pgp::pk_decrypt(data, private_keyring, &public_keyring_for_validate).await?;
// we should only have one decryption happening
ensure!(ret_valid_signatures.is_empty(), "corrupt signatures");
// Check for detached signatures.
// If decrypted part is a multipart/signed, then there is a detached signature.
let decrypted_part = mailparse::parse_mail(&plain)?;
if let Some((content, valid_detached_signatures)) =
validate_detached_signature(&decrypted_part, &public_keyring_for_validate).await?
{
return Ok(Some((content, valid_detached_signatures)));
} else {
// If the message was wrongly or not signed, still return the plain text.
// The caller has to check the signatures then.
let plain = pgp::pk_decrypt(
data,
private_keyring,
public_keyring_for_validate,
Some(ret_valid_signatures),
)
.await?;
return Ok(Some((plain, ret_valid_signatures)));
}
// If the message was wrongly or not signed, still return the plain text.
// The caller has to check the signatures then.
return Ok(Some(plain));
}
Ok(None)
@@ -390,34 +374,36 @@ pub async fn ensure_secret_key_exists(context: &Context) -> Result<String> {
let self_addr = context
.get_config(Config::ConfiguredAddr)
.await?
.context(concat!(
"Failed to get self address, ",
"cannot ensure secret key if not configured."
))?;
.ok_or_else(|| {
format_err!(concat!(
"Failed to get self address, ",
"cannot ensure secret key if not configured."
))
})?;
SignedPublicKey::load_self(context).await?;
Ok(self_addr)
}
#[cfg(test)]
mod tests {
use super::*;
use crate::chat;
use crate::message::{Message, Viewtype};
use crate::constants::Viewtype;
use crate::dc_receive_imf::dc_receive_imf;
use crate::message::Message;
use crate::param::Param;
use crate::peerstate::ToSave;
use crate::test_utils::{bob_keypair, TestContext};
use super::*;
mod ensure_secret_key_exists {
use super::*;
#[async_std::test]
async fn test_prexisting() {
let t = TestContext::new_alice().await;
assert_eq!(
ensure_secret_key_exists(&t).await.unwrap(),
"alice@example.org"
);
let t = TestContext::new().await;
let test_addr = t.configure_alice().await;
assert_eq!(ensure_secret_key_exists(&t).await.unwrap(), test_addr);
}
#[async_std::test]
@@ -488,7 +474,7 @@ Sent with my Delta Chat Messenger: https://delta.chat";
assert!(!msg.was_encrypted());
// Parsing a message is enough to update peerstate
let peerstate_alice = Peerstate::from_addr(&bob.ctx, "alice@example.org")
let peerstate_alice = Peerstate::from_addr(&bob.ctx, "alice@example.com")
.await?
.expect("no peerstate found in the database");
assert_eq!(peerstate_alice.prefer_encrypt, EncryptPreference::Mutual);
@@ -519,28 +505,28 @@ Sent with my Delta Chat Messenger: https://delta.chat";
let msg = bob.parse_msg(&sent).await;
assert!(msg.was_encrypted());
let peerstate_alice = Peerstate::from_addr(&bob.ctx, "alice@example.org")
let peerstate_alice = Peerstate::from_addr(&bob.ctx, "alice@example.com")
.await?
.expect("no peerstate found in the database");
assert_eq!(peerstate_alice.prefer_encrypt, EncryptPreference::Mutual);
// Alice sends plaintext message with Autocrypt header.
let mut msg = Message::new(Viewtype::Text);
msg.force_plaintext();
msg.param.set_int(Param::ForcePlaintext, 1);
chat::prepare_msg(&alice.ctx, chat_alice, &mut msg).await?;
chat::send_msg(&alice.ctx, chat_alice, &mut msg).await?;
let sent = alice.pop_sent_msg().await;
let msg = bob.parse_msg(&sent).await;
assert!(!msg.was_encrypted());
let peerstate_alice = Peerstate::from_addr(&bob.ctx, "alice@example.org")
let peerstate_alice = Peerstate::from_addr(&bob.ctx, "alice@example.com")
.await?
.expect("no peerstate found in the database");
assert_eq!(peerstate_alice.prefer_encrypt, EncryptPreference::Mutual);
// Alice sends plaintext message without Autocrypt header.
let mut msg = Message::new(Viewtype::Text);
msg.force_plaintext();
msg.param.set_int(Param::ForcePlaintext, 1);
msg.param.set_int(Param::SkipAutocrypt, 1);
chat::prepare_msg(&alice.ctx, chat_alice, &mut msg).await?;
chat::send_msg(&alice.ctx, chat_alice, &mut msg).await?;
@@ -548,7 +534,7 @@ Sent with my Delta Chat Messenger: https://delta.chat";
let msg = bob.parse_msg(&sent).await;
assert!(!msg.was_encrypted());
let peerstate_alice = Peerstate::from_addr(&bob.ctx, "alice@example.org")
let peerstate_alice = Peerstate::from_addr(&bob.ctx, "alice@example.com")
.await?
.expect("no peerstate found in the database");
assert_eq!(peerstate_alice.prefer_encrypt, EncryptPreference::Reset);
@@ -626,4 +612,68 @@ Sent with my Delta Chat Messenger: https://delta.chat";
Ok(())
}
#[async_std::test]
async fn test_e2ee_force() -> Result<()> {
let alice = TestContext::new_alice().await;
let bob = TestContext::new_bob().await;
let alice_chat = alice.create_chat(&bob).await;
let bob_chat = bob.create_chat(&alice).await;
alice.set_config(Config::ShowEmails, Some("2")).await?;
bob.set_config(Config::ShowEmails, Some("2")).await?;
// Alice does not prefer encryption.
alice.set_config(Config::E2eeEnabled, Some("0")).await?;
bob.set_config(Config::E2eeEnabled, Some("1")).await?;
// Alice sends her key to Bob.
let sent_msg = alice.send_text(alice_chat.id, "Hi Bob").await;
bob.recv_msg(&sent_msg).await;
let received_msg = bob.get_last_msg().await;
assert!(!received_msg.get_showpadlock());
// Bob should not encrypt, because Alice does not prefer encryption.
let sent_msg = bob
.send_text(bob_chat.id, "This should not be encrypted")
.await;
alice.recv_msg(&sent_msg).await;
let received_msg = alice.get_last_msg().await;
assert!(!received_msg.get_showpadlock());
// Bob ignores Alice's preference for no encryption.
bob.set_config(Config::E2eeForce, Some("1")).await?;
let sent_msg = bob.send_text(bob_chat.id, "This should be encrypted").await;
alice.recv_msg(&sent_msg).await;
let received_msg = alice.get_last_msg().await;
assert!(received_msg.get_showpadlock());
// Alice switches to MUA without Autocrypt support.
dc_receive_imf(
&bob,
br#"Subject: Hello from MUA
Message-ID: foobar@example.com
To: Bob <bob@example.net>
From: Alice <alice@example.com>
Content-Type: text/plain; charset=utf-8
Date: Sun, 14 Mar 2500 00:00:00 +0000
Hello from MUA."#,
"INBOX",
100,
false,
)
.await?;
// Bob can't encrypt now because Alice has no key.
let sent_msg = bob
.send_text(bob_chat.id, "This should not be encrypted again")
.await;
alice.recv_msg(&sent_msg).await;
let received_msg = alice.get_last_msg().await;
assert!(!received_msg.get_showpadlock());
Ok(())
}
}

View File

@@ -52,7 +52,7 @@
//! `MsgsChanged` event is emitted when a message deletion is due, to
//! make UI reload displayed messages and cause actual deletion.
//!
//! Server deletion happens by updating the `imap` table based on
//! Server deletion happens by generating IMAP deletion jobs based on
//! the database entries which are expired either according to their
//! ephemeral message timers or global `delete_server_after` setting.
@@ -67,16 +67,15 @@ use serde::{Deserialize, Serialize};
use crate::chat::{send_msg, ChatId};
use crate::constants::{
DC_CHAT_ID_LAST_SPECIAL, DC_CHAT_ID_TRASH, DC_CONTACT_ID_DEVICE, DC_CONTACT_ID_SELF,
Viewtype, DC_CHAT_ID_LAST_SPECIAL, DC_CHAT_ID_TRASH, DC_CONTACT_ID_DEVICE, DC_CONTACT_ID_SELF,
};
use crate::contact::ContactId;
use crate::context::Context;
use crate::dc_tools::time;
use crate::download::MIN_DELETE_SERVER_AFTER;
use crate::events::EventType;
use crate::message::{Message, MessageState, MsgId, Viewtype};
use crate::job;
use crate::message::{Message, MessageState, MsgId};
use crate::mimeparser::SystemMessage;
use crate::sql;
use crate::stock_str;
use std::cmp::max;
@@ -214,7 +213,7 @@ impl ChatId {
pub(crate) async fn stock_ephemeral_timer_changed(
context: &Context,
timer: Timer,
from_id: ContactId,
from_id: u32,
) -> String {
match timer {
Timer::Disabled => stock_str::msg_ephemeral_timer_disabled(context, from_id).await,
@@ -264,7 +263,7 @@ pub(crate) async fn stock_ephemeral_timer_changed(
impl MsgId {
/// Returns ephemeral message timer value for the message.
pub(crate) async fn ephemeral_timer(self, context: &Context) -> Result<Timer> {
pub(crate) async fn ephemeral_timer(self, context: &Context) -> anyhow::Result<Timer> {
let res = match context
.sql
.query_get_value(
@@ -280,7 +279,7 @@ impl MsgId {
}
/// Starts ephemeral message timer for the message if it is not started yet.
pub(crate) async fn start_ephemeral_timer(self, context: &Context) -> Result<()> {
pub(crate) async fn start_ephemeral_timer(self, context: &Context) -> anyhow::Result<()> {
if let Timer::Enabled { duration } = self.ephemeral_timer(context).await? {
let ephemeral_timestamp = time().saturating_add(duration.into());
@@ -299,37 +298,6 @@ impl MsgId {
}
}
pub(crate) async fn start_ephemeral_timers_msgids(
context: &Context,
msg_ids: &[MsgId],
) -> Result<()> {
let msg_ids: Vec<&dyn crate::ToSql> = msg_ids
.iter()
.map(|msg_id| msg_id as &dyn crate::ToSql)
.collect();
let now = time();
let count = context
.sql
.execute(
format!(
"UPDATE msgs SET ephemeral_timestamp = ? + ephemeral_timer
WHERE (ephemeral_timestamp == 0 OR ephemeral_timestamp > ? + ephemeral_timer) AND ephemeral_timer > 0
AND id IN ({})",
sql::repeat_vars(msg_ids.len())?
),
rusqlite::params_from_iter(
std::iter::once(&now as &dyn crate::ToSql)
.chain(std::iter::once(&now as &dyn crate::ToSql))
.chain(msg_ids),
),
)
.await?;
if count > 0 {
schedule_ephemeral_task(context).await;
}
Ok(())
}
/// Deletes messages which are expired according to
/// `delete_device_after` setting or `ephemeral_timestamp` column.
///
@@ -466,8 +434,11 @@ pub async fn schedule_ephemeral_task(context: &Context) {
}
}
/// Schedules expired IMAP messages for deletion.
pub(crate) async fn delete_expired_imap_messages(context: &Context) -> Result<()> {
/// Returns ID of any expired message that should be deleted from the server.
///
/// It looks up the trash chat too, to find messages that are already
/// deleted locally, but not deleted on the server.
pub(crate) async fn load_imap_deletion_msgid(context: &Context) -> anyhow::Result<Option<MsgId>> {
let now = time();
let (threshold_timestamp, threshold_timestamp_extended) =
@@ -481,20 +452,27 @@ pub(crate) async fn delete_expired_imap_messages(context: &Context) -> Result<()
context
.sql
.execute(
"UPDATE imap
SET target=''
WHERE rfc724_mid IN (
SELECT rfc724_mid FROM msgs
WHERE ((download_state = 0 AND timestamp < ?) OR
(download_state != 0 AND timestamp < ?) OR
(ephemeral_timestamp != 0 AND ephemeral_timestamp <= ?))
)",
paramsv![threshold_timestamp, threshold_timestamp_extended, now],
.query_row_optional(
"SELECT id FROM msgs \
WHERE ( \
((download_state = 0 AND timestamp < ?) OR (download_state != 0 AND timestamp < ?)) \
OR (ephemeral_timestamp != 0 AND ephemeral_timestamp <= ?) \
) \
AND server_uid != 0 \
AND NOT id IN (SELECT foreign_id FROM jobs WHERE action = ?)
LIMIT 1",
paramsv![
threshold_timestamp,
threshold_timestamp_extended,
now,
job::Action::DeleteMsgOnImap
],
|row| {
let msg_id: MsgId = row.get(0)?;
Ok(msg_id)
},
)
.await?;
Ok(())
.await
}
/// Start ephemeral timers for seen messages if they are not started
@@ -529,6 +507,9 @@ pub(crate) async fn start_ephemeral_timers(context: &Context) -> Result<()> {
#[cfg(test)]
mod tests {
use crate::param::Params;
use async_std::task::sleep;
use super::*;
use crate::config::Config;
use crate::dc_receive_imf::dc_receive_imf;
@@ -744,7 +725,7 @@ mod tests {
/// Test that Alice replying to the chat without a timer at the same time as Bob enables the
/// timer does not result in disabling the timer on the Bob's side.
#[async_std::test]
async fn test_ephemeral_timer_rollback() -> Result<()> {
async fn test_ephemeral_timer_rollback() -> anyhow::Result<()> {
let alice = TestContext::new_alice().await;
let bob = TestContext::new_bob().await;
@@ -818,14 +799,14 @@ mod tests {
}
#[async_std::test]
async fn test_ephemeral_delete_msgs() -> Result<()> {
async fn test_ephemeral_delete_msgs() {
let t = TestContext::new_alice().await;
let chat = t.get_self_chat().await;
t.send_text(chat.id, "Saved message, which we delete manually")
.await;
let msg = t.get_last_msg_in(chat.id).await;
msg.id.delete_from_db(&t).await?;
msg.id.delete_from_db(&t).await.unwrap();
check_msg_was_deleted(&t, &chat, msg.id).await;
chat.id
@@ -836,12 +817,36 @@ mod tests {
.send_text(chat.id, "Saved message, disappearing after 1s")
.await;
async_std::task::sleep(Duration::from_millis(1100)).await;
sleep(Duration::from_millis(1100)).await;
// Check that the msg was deleted locally.
// Check checks that the msg was deleted locally
check_msg_was_deleted(&t, &chat, msg.sender_msg_id).await;
Ok(())
// Check that the msg will be deleted on the server
// First of all, set a server_uid so that DC thinks that it's actually possible to delete
t.sql
.execute(
"UPDATE msgs SET server_uid=1 WHERE id=?",
paramsv![msg.sender_msg_id],
)
.await
.unwrap();
let job = job::load_imap_deletion_job(&t).await.unwrap();
assert_eq!(
job,
Some(job::Job::new(
job::Action::DeleteMsgOnImap,
msg.sender_msg_id.to_u32(),
Params::new(),
0,
))
);
// Let's assume that executing the job fails on first try and the job is saved to the db
job.unwrap().save(&t).await.unwrap();
// Make sure that we don't get yet another job when loading from db
let job2 = job::load_imap_deletion_job(&t).await.unwrap();
assert_eq!(job2, None);
}
async fn check_msg_was_deleted(t: &TestContext, chat: &Chat, msg_id: MsgId) {
@@ -856,8 +861,8 @@ mod tests {
// Check that if there is a message left, the text and metadata are gone
if let Ok(msg) = Message::load_from_db(t, msg_id).await {
assert_eq!(msg.from_id, ContactId::new(0));
assert_eq!(msg.to_id, ContactId::new(0));
assert_eq!(msg.from_id, 0);
assert_eq!(msg.to_id, 0);
assert!(msg.text.is_none_or_empty(), "{:?}", msg.text);
let rawtxt: Option<String> = t
.sql
@@ -869,7 +874,7 @@ mod tests {
}
#[async_std::test]
async fn test_delete_expired_imap_messages() -> Result<()> {
async fn test_load_imap_deletion_msgid() -> Result<()> {
let t = TestContext::new_alice().await;
const HOUR: i64 = 60 * 60;
let now = time();
@@ -882,98 +887,42 @@ mod tests {
(2000, now - 18 * HOUR, now - HOUR),
(2020, now - 17 * HOUR, now + HOUR),
] {
let message_id = id.to_string();
t.sql
.execute(
"INSERT INTO msgs (id, rfc724_mid, timestamp, ephemeral_timestamp) VALUES (?,?,?,?);",
paramsv![id, message_id, timestamp, ephemeral_timestamp],
)
.await?;
t.sql
.execute(
"INSERT INTO imap (rfc724_mid, folder, uid, target) VALUES (?,'INBOX',?, 'INBOX');",
paramsv![message_id, id],
)
.await?;
}
async fn test_marked_for_deletion(context: &Context, id: u32) -> Result<()> {
assert_eq!(
context
.sql
.count(
"SELECT COUNT(*) FROM imap WHERE target='' AND rfc724_mid=?",
paramsv![id.to_string()],
)
.await?,
1
);
Ok(())
}
async fn remove_uid(context: &Context, id: u32) -> Result<()> {
context
.sql
.execute(
"DELETE FROM imap WHERE rfc724_mid=?",
paramsv![id.to_string()],
"INSERT INTO msgs (id, server_uid, timestamp, ephemeral_timestamp) VALUES (?,?,?,?);",
paramsv![id, id, timestamp, ephemeral_timestamp],
)
.await?;
Ok(())
}
// This should mark message 2000 for deletion.
delete_expired_imap_messages(&t).await?;
test_marked_for_deletion(&t, 2000).await?;
remove_uid(&t, 2000).await?;
// No other messages are marked for deletion.
assert_eq!(
t.sql
.count("SELECT COUNT(*) FROM imap WHERE target=''", paramsv![],)
.await?,
0
);
assert_eq!(load_imap_deletion_msgid(&t).await?, Some(MsgId::new(2000)));
MsgId::new(2000).delete_from_db(&t).await?;
assert_eq!(load_imap_deletion_msgid(&t).await?, None);
t.set_config(Config::DeleteServerAfter, Some(&*(25 * HOUR).to_string()))
.await?;
delete_expired_imap_messages(&t).await?;
test_marked_for_deletion(&t, 1000).await?;
assert_eq!(load_imap_deletion_msgid(&t).await?, Some(MsgId::new(1000)));
MsgId::new(1000)
.update_download_state(&t, DownloadState::Available)
.await?;
t.sql
.execute(
"UPDATE imap SET target=folder WHERE rfc724_mid='1000'",
paramsv![],
)
.await?;
delete_expired_imap_messages(&t).await?;
test_marked_for_deletion(&t, 1000).await?; // Delete downloadable anyway.
remove_uid(&t, 1000).await?;
assert_eq!(load_imap_deletion_msgid(&t).await?, Some(MsgId::new(1000))); // delete downloadable anyway
MsgId::new(1000).delete_from_db(&t).await?;
assert_eq!(load_imap_deletion_msgid(&t).await?, None);
t.set_config(Config::DeleteServerAfter, Some(&*(22 * HOUR).to_string()))
.await?;
delete_expired_imap_messages(&t).await?;
test_marked_for_deletion(&t, 1010).await?;
t.sql
.execute(
"UPDATE imap SET target=folder WHERE rfc724_mid='1010'",
paramsv![],
)
.await?;
assert_eq!(load_imap_deletion_msgid(&t).await?, Some(MsgId::new(1010)));
MsgId::new(1010)
.update_download_state(&t, DownloadState::Available)
.await?;
delete_expired_imap_messages(&t).await?;
// Keep downloadable for now.
assert_eq!(
t.sql
.count("SELECT COUNT(*) FROM imap WHERE target=''", paramsv![],)
.await?,
0
);
assert_eq!(load_imap_deletion_msgid(&t).await?, None); // keep downloadable for now
MsgId::new(1010).delete_from_db(&t).await?;
assert_eq!(load_imap_deletion_msgid(&t).await?, None);
Ok(())
}
@@ -987,7 +936,7 @@ mod tests {
dc_receive_imf(
&alice,
b"From: Bob <bob@example.com>\n\
To: Alice <alice@example.org>\n\
To: Alice <alice@example.com>\n\
Chat-Version: 1.0\n\
Subject: Subject\n\
Message-ID: <first@example.com>\n\
@@ -995,6 +944,7 @@ mod tests {
\n\
hello\n",
"INBOX",
1,
false,
)
.await?;
@@ -1007,7 +957,7 @@ mod tests {
dc_receive_imf(
&alice,
b"From: Bob <bob@example.com>\n\
To: Alice <alice@example.org>\n\
To: Alice <alice@example.com>\n\
Chat-Version: 1.0\n\
Subject: Subject\n\
Message-ID: <second@example.com>\n\
@@ -1016,6 +966,7 @@ mod tests {
\n\
second message\n",
"INBOX",
2,
false,
)
.await?;
@@ -1043,7 +994,7 @@ mod tests {
dc_receive_imf(
&alice,
b"From: Bob <bob@example.com>\n\
To: Alice <alice@example.org>\n\
To: Alice <alice@example.com>\n\
Chat-Version: 1.0\n\
Subject: Subject\n\
Message-ID: <third@example.com>\n\
@@ -1053,6 +1004,7 @@ mod tests {
\n\
> hello\n",
"INBOX",
3,
false,
)
.await?;

View File

@@ -7,7 +7,6 @@ use async_std::path::PathBuf;
use strum::EnumProperty;
use crate::chat::ChatId;
use crate::contact::ContactId;
use crate::ephemeral::Timer as EphemeralTimer;
use crate::message::MsgId;
@@ -253,7 +252,7 @@ pub enum EventType {
///
/// @param data1 (int) If set, this is the contact_id of an added contact that should be selected.
#[strum(props(id = "2030"))]
ContactsChanged(Option<ContactId>),
ContactsChanged(Option<u32>),
/// Location of one or more contact has changed.
///
@@ -261,7 +260,7 @@ pub enum EventType {
/// If the locations of several contacts have been changed,
/// eg. after calling dc_delete_all_locations(), this parameter is set to `None`.
#[strum(props(id = "2035"))]
LocationChanged(Option<ContactId>),
LocationChanged(Option<u32>),
/// Inform about the configuration progress started by configure().
#[strum(props(id = "2041"))]
@@ -305,10 +304,7 @@ pub enum EventType {
/// 800=vg-member-added-received received, shown as "bob@addr securely joined GROUP", only sent for the verified-group-protocol.
/// 1000=Protocol finished for this contact.
#[strum(props(id = "2060"))]
SecurejoinInviterProgress {
contact_id: ContactId,
progress: usize,
},
SecurejoinInviterProgress { contact_id: u32, progress: usize },
/// Progress information of a secure-join handshake from the view of the joiner
/// (Bob, the person who scans the QR code).
@@ -319,10 +315,7 @@ pub enum EventType {
/// 400=vg-/vc-request-with-auth sent, typically shown as "alice@addr verified, introducing myself."
/// (Bob has verified alice and waits until Alice does the same for him)
#[strum(props(id = "2061"))]
SecurejoinJoinerProgress {
contact_id: ContactId,
progress: usize,
},
SecurejoinJoinerProgress { contact_id: u32, progress: usize },
/// The connectivity to the server changed.
/// This means that you should refresh the connectivity view
@@ -333,7 +326,4 @@ pub enum EventType {
#[strum(props(id = "2110"))]
SelfavatarChanged,
#[strum(props(id = "2120"))]
WebxdcStatusUpdate(MsgId),
}

View File

@@ -33,7 +33,6 @@ pub enum HeaderDef {
XMozillaDraftInfo,
ListId,
ListPost,
References,
InReplyTo,
Precedence,

View File

@@ -279,9 +279,9 @@ mod tests {
use crate::chat;
use crate::chat::forward_msgs;
use crate::config::Config;
use crate::constants::DC_CONTACT_ID_SELF;
use crate::constants::{Viewtype, DC_CONTACT_ID_SELF};
use crate::dc_receive_imf::dc_receive_imf;
use crate::message::{MessengerMessage, Viewtype};
use crate::message::MessengerMessage;
use crate::test_utils::TestContext;
#[async_std::test]
@@ -365,7 +365,7 @@ test some special html-characters as &lt; &gt; and &amp; but also &quot; and &#x
// however, rust multiline-strings use just `\n`;
// therefore, we just remove `\r` before comparison.
assert_eq!(
parser.html.replace('\r', ""),
parser.html.replace("\r", ""),
r##"
<html>
<p>mime-modified <b>set</b>; simplify is always regarded as lossy.</p>
@@ -379,7 +379,7 @@ test some special html-characters as &lt; &gt; and &amp; but also &quot; and &#x
let raw = include_bytes!("../test-data/message/text_alt_html.eml");
let parser = HtmlMsgParser::from_bytes(&t.ctx, raw).await.unwrap();
assert_eq!(
parser.html.replace('\r', ""), // see comment in test_htmlparse_html()
parser.html.replace("\r", ""), // see comment in test_htmlparse_html()
r##"<html>
<p>mime-modified <b>set</b>; simplify is always regarded as lossy.</p>
</html>
@@ -394,7 +394,7 @@ test some special html-characters as &lt; &gt; and &amp; but also &quot; and &#x
let raw = include_bytes!("../test-data/message/text_alt_plain_html.eml");
let parser = HtmlMsgParser::from_bytes(&t.ctx, raw).await.unwrap();
assert_eq!(
parser.html.replace('\r', ""), // see comment in test_htmlparse_html()
parser.html.replace("\r", ""), // see comment in test_htmlparse_html()
r##"<html>
<p>
this is <b>html</b>
@@ -440,7 +440,9 @@ test some special html-characters as &lt; &gt; and &amp; but also &quot; and &#x
.create_chat_with_contact("", "sender@testrun.org")
.await;
let raw = include_bytes!("../test-data/message/text_alt_plain_html.eml");
dc_receive_imf(&alice, raw, "INBOX", false).await.unwrap();
dc_receive_imf(&alice, raw, "INBOX", 1, false)
.await
.unwrap();
let msg = alice.get_last_msg_in(chat.get_id()).await;
assert_ne!(msg.get_from_id(), DC_CONTACT_ID_SELF);
assert_eq!(msg.is_dc_message, MessengerMessage::No);
@@ -466,7 +468,7 @@ test some special html-characters as &lt; &gt; and &amp; but also &quot; and &#x
// bob: check that bob also got the html-part of the forwarded message
let bob = TestContext::new_bob().await;
let chat = bob.create_chat_with_contact("", "alice@example.org").await;
let chat = bob.create_chat_with_contact("", "alice@example.com").await;
bob.recv_msg(&alice.pop_sent_msg().await).await;
let msg = bob.get_last_msg_in(chat.get_id()).await;
assert_ne!(msg.get_from_id(), DC_CONTACT_ID_SELF);
@@ -489,7 +491,9 @@ test some special html-characters as &lt; &gt; and &amp; but also &quot; and &#x
.create_chat_with_contact("", "sender@testrun.org")
.await;
let raw = include_bytes!("../test-data/message/text_alt_plain_html.eml");
dc_receive_imf(&alice, raw, "INBOX", false).await.unwrap();
dc_receive_imf(&alice, raw, "INBOX", 1, false)
.await
.unwrap();
let msg = alice.get_last_msg_in(chat.get_id()).await;
// forward the message to saved-messages,
@@ -556,6 +560,7 @@ test some special html-characters as &lt; &gt; and &amp; but also &quot; and &#x
&t,
include_bytes!("../test-data/message/cp1252-html.eml"),
"INBOX",
0,
false,
)
.await?;

File diff suppressed because it is too large Load Diff

View File

@@ -1,6 +1,6 @@
use super::Imap;
use anyhow::{bail, Context as _, Result};
use anyhow::{bail, format_err, Result};
use async_imap::extensions::idle::IdleResponse;
use async_std::prelude::*;
use std::time::{Duration, SystemTime};
@@ -31,7 +31,7 @@ impl Imap {
let timeout = Duration::from_secs(23 * 60);
let mut info = Default::default();
if self.server_sent_unsolicited_exists(context)? {
if self.server_sent_unsolicited_exists(context) {
return Ok(info);
}
@@ -90,8 +90,8 @@ impl Imap {
let session = handle
.done()
.timeout(Duration::from_secs(15))
.await?
.context("IMAP IDLE protocol timed out")?;
.await
.map_err(|err| format_err!("IMAP IDLE protocol timed out: {}", err))??;
self.session = Some(Session { inner: session });
} else {
warn!(context, "Attempted to idle without a session");
@@ -150,18 +150,19 @@ impl Imap {
}
if self.config.can_idle {
// we only fake-idled because network was gone during IDLE, probably
break InterruptInfo::new(false);
break InterruptInfo::new(false, None);
}
info!(context, "fake_idle is connected");
// we are connected, let's see if fetching messages results
// in anything. If so, we behave as if IDLE had data but
// will have already fetched the messages so perform_*_fetch
// will not find any new.
match self.fetch_new_messages(context, &watch_folder, false).await {
Ok(res) => {
info!(context, "fetch_new_messages returned {:?}", res);
if res {
break InterruptInfo::new(false);
break InterruptInfo::new(false, None);
}
}
Err(err) => {

View File

@@ -2,18 +2,15 @@ use std::{collections::BTreeMap, time::Instant};
use anyhow::{Context as _, Result};
use crate::config::Config;
use crate::imap::Imap;
use crate::log::LogExt;
use crate::{config::Config, log::LogExt};
use crate::{context::Context, imap::FolderMeaning};
use async_std::stream::StreamExt;
use async_std::prelude::*;
use super::{get_folder_meaning, get_folder_meaning_by_name};
impl Imap {
/// Returns true if folders were scanned, false if scanning was postponed.
pub(crate) async fn scan_folders(&mut self, context: &Context) -> Result<bool> {
pub async fn scan_folders(&mut self, context: &Context) -> Result<()> {
// First of all, debounce to once per minute:
let mut last_scan = context.last_full_folder_scan.lock().await;
if let Some(last_scan) = *last_scan {
@@ -23,18 +20,28 @@ impl Imap {
.await?;
if elapsed_secs < debounce_secs {
return Ok(false);
return Ok(());
}
}
info!(context, "Starting full folder scan");
self.prepare(context).await?;
let folders = self.list_folders(context).await?;
let watched_folders = get_watched_folders(context).await?;
let session = self.session.as_mut();
let session = session.context("scan_folders(): IMAP No Connection established")?;
let folders: Vec<_> = session.list(Some(""), Some("*")).await?.collect().await;
let watched_folders = get_watched_folders(context).await;
let mut folder_configs = BTreeMap::new();
for folder in folders {
let folder = match folder {
Ok(f) => f,
Err(e) => {
warn!(context, "Can't get folder: {}", e);
continue;
}
};
// Gmail labels are not folders and should be skipped. For example,
// emails appear in the inbox and under "All Mail" as soon as it is
// received. The code used to wrongly conclude that the email had
@@ -64,15 +71,15 @@ impl Imap {
// Don't scan folders that are watched anyway
if !watched_folders.contains(&folder.name().to_string()) && !is_drafts {
// Drain leftover unsolicited EXISTS messages
self.server_sent_unsolicited_exists(context)?;
self.server_sent_unsolicited_exists(context);
loop {
self.fetch_move_delete(context, folder.name())
self.fetch_new_messages(context, folder.name(), false)
.await
.ok_or_log_msg(context, "Can't fetch new msgs in scanned folder");
// If the server sent an unsocicited EXISTS during the fetch, we need to fetch again
if !self.server_sent_unsolicited_exists(context)? {
if !self.server_sent_unsolicited_exists(context) {
break;
}
}
@@ -91,41 +98,23 @@ impl Imap {
}
last_scan.replace(Instant::now());
Ok(true)
}
/// Returns the names of all folders on the IMAP server.
pub async fn list_folders(
self: &mut Imap,
context: &Context,
) -> Result<Vec<async_imap::types::Name>> {
let session = self.session.as_mut();
let session = session.context("No IMAP connection")?;
let list = session
.list(Some(""), Some("*"))
.await?
.filter_map(|f| f.ok_or_log_msg(context, "list_folders() can't get folder"));
Ok(list.collect().await)
Ok(())
}
}
pub(crate) async fn get_watched_folder_configs(context: &Context) -> Result<Vec<Config>> {
let mut res = vec![Config::ConfiguredInboxFolder];
if context.get_config_bool(Config::SentboxWatch).await? {
res.push(Config::ConfiguredSentboxFolder);
}
if context.should_watch_mvbox().await? {
res.push(Config::ConfiguredMvboxFolder);
}
Ok(res)
}
pub(crate) async fn get_watched_folders(context: &Context) -> Result<Vec<String>> {
pub(crate) async fn get_watched_folders(context: &Context) -> Vec<String> {
let mut res = Vec::new();
for folder_config in get_watched_folder_configs(context).await? {
if let Some(folder) = context.get_config(folder_config).await? {
res.push(folder);
let folder_watched_configured = &[
(Config::SentboxWatch, Config::ConfiguredSentboxFolder),
(Config::MvboxWatch, Config::ConfiguredMvboxFolder),
(Config::InboxWatch, Config::ConfiguredInboxFolder),
];
for (watched, configured) in folder_watched_configured {
if context.get_config_bool(*watched).await.unwrap_or_default() {
if let Ok(Some(folder)) = context.get_config(*configured).await {
res.push(folder);
}
}
}
Ok(res)
res
}

View File

@@ -93,11 +93,7 @@ impl Imap {
// select new folder
if let Some(folder) = folder {
if let Some(ref mut session) = &mut self.session {
let res = if self.config.can_condstore {
session.select_condstore(folder).await
} else {
session.select(folder).await
};
let res = session.select(folder).await;
// <https://tools.ietf.org/html/rfc3501#section-6.3.1>
// says that if the server reports select failure we are in

View File

@@ -16,21 +16,21 @@ use rand::{thread_rng, Rng};
use crate::blob::BlobObject;
use crate::chat::{self, delete_and_reset_all_device_msgs, ChatId};
use crate::config::Config;
use crate::constants::DC_CONTACT_ID_SELF;
use crate::constants::{Viewtype, DC_CONTACT_ID_SELF};
use crate::context::Context;
use crate::dc_tools::{
dc_create_folder, dc_delete_file, dc_delete_files_in_dir, dc_get_filesuffix_lc,
dc_open_file_std, dc_read_file, dc_write_file, time, EmailAddress,
dc_copy_file, dc_create_folder, dc_delete_file, dc_delete_files_in_dir, dc_get_filesuffix_lc,
dc_open_file_std, dc_read_file, dc_write_file, get_next_backup_path, time, EmailAddress,
};
use crate::e2ee;
use crate::events::EventType;
use crate::key::{self, DcKey, DcSecretKey, SignedPublicKey, SignedSecretKey};
use crate::log::LogExt;
use crate::message::{Message, MsgId, Viewtype};
use crate::message::{Message, MsgId};
use crate::mimeparser::SystemMessage;
use crate::param::Param;
use crate::pgp;
use crate::sql;
use crate::sql::{self, Sql};
use crate::stock_str;
// Name of the database file in the backup.
@@ -41,24 +41,24 @@ const BLOBS_BACKUP_NAME: &str = "blobs_backup";
#[repr(u32)]
pub enum ImexMode {
/// Export all private keys and all public keys of the user to the
/// directory given as `path`. The default key is written to the files `public-key-default.asc`
/// directory given as `param1`. The default key is written to the files `public-key-default.asc`
/// and `private-key-default.asc`, if there are more keys, they are written to files as
/// `public-key-<id>.asc` and `private-key-<id>.asc`
ExportSelfKeys = 1,
/// Import private keys found in the directory given as `path`.
/// Import private keys found in the directory given as `param1`.
/// The last imported key is made the default keys unless its name contains the string `legacy`.
/// Public keys are not imported.
ImportSelfKeys = 2,
/// Export a backup to the directory given as `path` with the given `passphrase`.
/// Export a backup to the directory given as `param1`.
/// The backup contains all contacts, chats, images and other data and device independent settings.
/// The backup does not contain device dependent settings as ringtones or LED notification settings.
/// The name of the backup is typically `delta-chat-<day>.tar`, if more than one backup is create on a day,
/// the format is `delta-chat-<day>-<number>.tar`
ExportBackup = 11,
/// `path` is the file (not: directory) to import. The file is normally
/// `param1` is the file (not: directory) to import. The file is normally
/// created by DC_IMEX_EXPORT_BACKUP and detected by dc_imex_has_backup(). Importing a backup
/// is only possible as long as the context is not configured or used in another way.
ImportBackup = 12,
@@ -78,16 +78,11 @@ pub enum ImexMode {
///
/// Only one import-/export-progress can run at the same time.
/// To cancel an import-/export-progress, drop the future returned by this function.
pub async fn imex(
context: &Context,
what: ImexMode,
path: &Path,
passphrase: Option<String>,
) -> Result<()> {
pub async fn imex(context: &Context, what: ImexMode, param1: &Path) -> Result<()> {
let cancel = context.alloc_ongoing().await?;
let res = async {
let success = imex_inner(context, what, path, passphrase).await;
let success = imex_inner(context, what, param1).await;
match success {
Ok(()) => {
info!(context, "IMEX successfully completed");
@@ -120,10 +115,15 @@ async fn cleanup_aborted_imex(context: &Context, what: ImexMode) {
dc_delete_file(context, context.get_dbfile()).await;
dc_delete_files_in_dir(context, context.get_blobdir()).await;
}
if what == ImexMode::ExportBackup || what == ImexMode::ImportBackup {
if let Err(e) = context.sql.open(context, context.get_dbfile(), false).await {
warn!(context, "Re-opening db after imex failed: {}", e);
}
}
}
/// Returns the filename of the backup found (otherwise an error)
pub async fn has_backup(_context: &Context, dir_name: &Path) -> Result<String> {
pub async fn has_backup(context: &Context, dir_name: &Path) -> Result<String> {
let mut dir_iter = async_std::fs::read_dir(dir_name).await?;
let mut newest_backup_name = "".to_string();
let mut newest_backup_path: Option<PathBuf> = None;
@@ -145,6 +145,59 @@ pub async fn has_backup(_context: &Context, dir_name: &Path) -> Result<String> {
}
}
match newest_backup_path {
Some(path) => Ok(path.to_string_lossy().into_owned()),
None => has_backup_old(context, dir_name).await,
// When we decide to remove support for .bak backups, we can replace this with `None => bail!("no backup found in {}", dir_name.display()),`.
}
}
/// Returns the filename of the backup found (otherwise an error)
pub async fn has_backup_old(context: &Context, dir_name: &Path) -> Result<String> {
let mut dir_iter = async_std::fs::read_dir(dir_name).await?;
let mut newest_backup_time = 0;
let mut newest_backup_name = "".to_string();
let mut newest_backup_path: Option<PathBuf> = None;
while let Some(dirent) = dir_iter.next().await {
if let Ok(dirent) = dirent {
let path = dirent.path();
let name = dirent.file_name();
let name = name.to_string_lossy();
if name.starts_with("delta-chat") && name.ends_with(".bak") {
let sql = Sql::new();
match sql.open(context, &path, true).await {
Ok(_) => {
let curr_backup_time = sql
.get_raw_config_int("backup_time")
.await?
.unwrap_or_default();
if curr_backup_time > newest_backup_time {
newest_backup_path = Some(path);
newest_backup_time = curr_backup_time;
}
info!(context, "backup_time of {} is {}", name, curr_backup_time);
sql.close().await;
}
Err(e) => {
warn!(
context,
"Found backup file {} which could not be opened: {}", name, e
);
// On some Android devices we can't open sql files that are not in our private directory
// (see <https://github.com/deltachat/deltachat-android/issues/1768>). So, compare names
// to still find the newest backup.
let name: String = name.into();
if newest_backup_time == 0
&& (newest_backup_name.is_empty() || name > newest_backup_name)
{
newest_backup_path = Some(path);
newest_backup_name = name;
}
}
}
}
}
}
match newest_backup_path {
Some(path) => Ok(path.to_string_lossy().into_owned()),
None => bail!("no backup found in {}", dir_name.display()),
@@ -165,6 +218,7 @@ pub async fn initiate_key_transfer(context: &Context) -> Result<String> {
}
async fn do_initiate_key_transfer(context: &Context) -> Result<String> {
let mut msg: Message;
let setup_code = create_setup_code(context);
/* this may require a keypair to be created. this may take a second ... */
let setup_file_content = render_setup_file(context, &setup_code).await?;
@@ -177,16 +231,14 @@ async fn do_initiate_key_transfer(context: &Context) -> Result<String> {
.await?;
let chat_id = ChatId::create_for_contact(context, DC_CONTACT_ID_SELF).await?;
let mut msg = Message {
viewtype: Viewtype::File,
..Default::default()
};
msg = Message::default();
msg.viewtype = Viewtype::File;
msg.param.set(Param::File, setup_file_blob.as_name());
msg.subject = stock_str::ac_setup_msg_subject(context).await;
msg.param
.set(Param::MimeType, "application/autocrypt-setup");
msg.param.set_cmd(SystemMessage::AutocryptSetupMessage);
msg.force_plaintext();
msg.param.set_int(Param::ForcePlaintext, 1);
msg.param.set_int(Param::SkipAutocrypt, 1);
let msg_id = chat::send_msg(context, chat_id, &mut msg).await?;
@@ -237,7 +289,7 @@ pub async fn render_setup_file(context: &Context, passphrase: &str) -> Result<St
let msg_subj = stock_str::ac_setup_msg_subject(context).await;
let msg_body = stock_str::ac_setup_msg_body(context).await;
let msg_body_html = msg_body.replace('\r', "").replace('\n', "<br>");
let msg_body_html = msg_body.replace("\r", "").replace("\n", "<br>");
Ok(format!(
concat!(
"<!DOCTYPE html>\r\n",
@@ -397,12 +449,7 @@ fn normalize_setup_code(s: &str) -> String {
out
}
async fn imex_inner(
context: &Context,
what: ImexMode,
path: &Path,
passphrase: Option<String>,
) -> Result<()> {
async fn imex_inner(context: &Context, what: ImexMode, path: &Path) -> Result<()> {
info!(context, "Import/export dir: {}", path.display());
ensure!(context.sql.is_open().await, "Database not opened.");
context.emit_event(EventType::ImexProgress(10));
@@ -420,27 +467,26 @@ async fn imex_inner(
ImexMode::ExportSelfKeys => export_self_keys(context, path).await,
ImexMode::ImportSelfKeys => import_self_keys(context, path).await,
ImexMode::ExportBackup => {
export_backup(context, path, passphrase.unwrap_or_default()).await
}
ImexMode::ImportBackup => {
import_backup(context, path, passphrase.unwrap_or_default()).await?;
context.sql.run_migrations(context).await
}
ImexMode::ExportBackup => export_backup(context, path).await,
// import_backup() will call import_backup_old() if this is an old backup.
ImexMode::ImportBackup => import_backup(context, path).await,
}
}
/// Imports backup into the currently open database.
///
/// The contents of the currently open database will be lost.
///
/// `passphrase` is the passphrase used to open backup database. If backup is unencrypted, pass
/// empty string here.
async fn import_backup(
context: &Context,
backup_to_import: &Path,
passphrase: String,
) -> Result<()> {
/// Import Backup
async fn import_backup(context: &Context, backup_to_import: &Path) -> Result<()> {
if backup_to_import.to_string_lossy().ends_with(".bak") {
// Backwards compability
return import_backup_old(context, backup_to_import).await;
}
info!(
context,
"Import \"{}\" to \"{}\".",
backup_to_import.display(),
context.get_dbfile().display()
);
ensure!(
!context.is_configured().await?,
"Cannot import backups to accounts in use."
@@ -449,19 +495,15 @@ async fn import_backup(
!context.scheduler.read().await.is_running(),
"cannot import backup, IO already running"
);
context.sql.close().await;
dc_delete_file(context, context.get_dbfile()).await;
ensure!(
!context.get_dbfile().exists().await,
"Cannot delete old database."
);
let backup_file = File::open(backup_to_import).await?;
let file_size = backup_file.metadata().await?.len();
info!(
context,
"Import \"{}\" ({} bytes) to \"{}\".",
backup_to_import.display(),
file_size,
context.get_dbfile().display()
);
context.sql.config_cache.write().await.clear();
let archive = Archive::new(backup_file);
let mut entries = archive.entries()?;
@@ -480,15 +522,11 @@ async fn import_backup(
if f.path()?.file_name() == Some(OsStr::new(DBFILE_BACKUP_NAME)) {
// async_tar can't unpack to a specified file name, so we just unpack to the blobdir and then move the unpacked file.
f.unpack_in(context.get_blobdir()).await?;
let unpacked_database = context.get_blobdir().join(DBFILE_BACKUP_NAME);
context
.sql
.import(&unpacked_database, passphrase.clone())
.await
.context("cannot import unpacked database")?;
fs::remove_file(unpacked_database)
.await
.context("cannot remove unpacked database")?;
fs::rename(
context.get_blobdir().join(DBFILE_BACKUP_NAME),
context.get_dbfile(),
)
.await?;
} else {
// async_tar will unpack to blobdir/BLOBS_BACKUP_NAME, so we move the file afterwards.
f.unpack_in(context.get_blobdir()).await?;
@@ -503,52 +541,136 @@ async fn import_backup(
}
}
context
.sql
.open(context, context.get_dbfile(), false)
.await
.context("Could not re-open db")?;
delete_and_reset_all_device_msgs(context).await?;
Ok(())
}
async fn import_backup_old(context: &Context, backup_to_import: &Path) -> Result<()> {
info!(
context,
"Import \"{}\" to \"{}\".",
backup_to_import.display(),
context.get_dbfile().display()
);
ensure!(
!context.is_configured().await?,
"Cannot import backups to accounts in use."
);
ensure!(
!context.scheduler.read().await.is_running(),
"cannot import backup, IO already running"
);
context.sql.close().await;
dc_delete_file(context, context.get_dbfile()).await;
ensure!(
!context.get_dbfile().exists().await,
"Cannot delete old database."
);
ensure!(
dc_copy_file(context, backup_to_import, context.get_dbfile()).await,
"could not copy file"
);
/* error already logged */
/* re-open copied database file */
context
.sql
.open(context, context.get_dbfile(), false)
.await
.context("Could not re-open db")?;
delete_and_reset_all_device_msgs(context).await?;
let total_files_cnt = context
.sql
.count("SELECT COUNT(*) FROM backup_blobs;", paramsv![])
.await?;
info!(
context,
"***IMPORT-in-progress: total_files_cnt={:?}", total_files_cnt,
);
// Load IDs only for now, without the file contents, to avoid
// consuming too much memory.
let file_ids = context
.sql
.query_map(
"SELECT id FROM backup_blobs ORDER BY id",
paramsv![],
|row| row.get(0),
|ids| {
ids.collect::<std::result::Result<Vec<i64>, _>>()
.map_err(Into::into)
},
)
.await?;
let mut all_files_extracted = true;
for (processed_files_cnt, file_id) in file_ids.into_iter().enumerate() {
// Load a single blob into memory
let (file_name, file_blob) = context
.sql
.query_row(
"SELECT file_name, file_content FROM backup_blobs WHERE id = ?",
paramsv![file_id],
|row| {
let file_name: String = row.get(0)?;
let file_blob: Vec<u8> = row.get(1)?;
Ok((file_name, file_blob))
},
)
.await?;
if context.shall_stop_ongoing().await {
all_files_extracted = false;
break;
}
let mut permille = processed_files_cnt * 1000 / total_files_cnt;
if permille < 10 {
permille = 10
}
if permille > 990 {
permille = 990
}
context.emit_event(EventType::ImexProgress(permille));
if file_blob.is_empty() {
continue;
}
let path_filename = context.get_blobdir().join(file_name);
dc_write_file(context, &path_filename, &file_blob).await?;
}
if all_files_extracted {
// only delete backup_blobs if all files were successfully extracted
context
.sql
.execute("DROP TABLE backup_blobs;", paramsv![])
.await?;
context.sql.execute("VACUUM;", paramsv![]).await.ok();
Ok(())
} else {
bail!("received stop signal");
}
}
/*******************************************************************************
* Export backup
******************************************************************************/
/// Returns Ok((temp_db_path, temp_path, dest_path)) on success. Unencrypted database can be
/// written to temp_db_path. The backup can then be written to temp_path. If the backup succeeded,
/// it can be renamed to dest_path. This guarantees that the backup is complete.
async fn get_next_backup_path(
folder: &Path,
backup_time: i64,
) -> Result<(PathBuf, PathBuf, PathBuf)> {
let folder = PathBuf::from(folder);
let stem = chrono::NaiveDateTime::from_timestamp(backup_time, 0)
// Don't change this file name format, in has_backup() we use string comparison to determine which backup is newer:
.format("delta-chat-backup-%Y-%m-%d")
.to_string();
// 64 backup files per day should be enough for everyone
for i in 0..64 {
let mut tempdbfile = folder.clone();
tempdbfile.push(format!("{}-{:02}.db", stem, i));
let mut tempfile = folder.clone();
tempfile.push(format!("{}-{:02}.tar.part", stem, i));
let mut destfile = folder.clone();
destfile.push(format!("{}-{:02}.tar", stem, i));
if !tempdbfile.exists().await && !tempfile.exists().await && !destfile.exists().await {
return Ok((tempdbfile, tempfile, destfile));
}
}
bail!("could not create backup file, disk full?");
}
async fn export_backup(context: &Context, dir: &Path, passphrase: String) -> Result<()> {
#[allow(unused)]
async fn export_backup(context: &Context, dir: &Path) -> Result<()> {
// get a fine backup file name (the name includes the date so that multiple backup instances are possible)
let now = time();
let (temp_db_path, temp_path, dest_path) = get_next_backup_path(dir, now).await?;
let _d1 = DeleteOnDrop(temp_db_path.clone());
let _d2 = DeleteOnDrop(temp_path.clone());
let (temp_path, dest_path) = get_next_backup_path(dir, now).await?;
let _d = DeleteOnDrop(temp_path.clone());
context
.sql
@@ -560,14 +682,16 @@ async fn export_backup(context: &Context, dir: &Path, passphrase: String) -> Res
.sql
.execute("VACUUM;", paramsv![])
.await
.map_err(|e| warn!(context, "Vacuum failed, exporting anyway {}", e))
.ok();
.map_err(|e| warn!(context, "Vacuum failed, exporting anyway {}", e));
ensure!(
!context.scheduler.read().await.is_running(),
"cannot export backup, IO already running"
);
// we close the database during the export
context.sql.close().await;
info!(
context,
"Backup '{}' to '{}'.",
@@ -575,13 +699,10 @@ async fn export_backup(context: &Context, dir: &Path, passphrase: String) -> Res
dest_path.display(),
);
context
.sql
.export(&temp_db_path, passphrase)
.await
.with_context(|| format!("failed to backup plaintext database to {:?}", temp_db_path))?;
let res = export_backup_inner(context, &temp_path).await;
let res = export_backup_inner(context, &temp_db_path, &temp_path).await;
// we re-open the database after export is finished
context.sql.open(context, context.get_dbfile(), false).await;
match &res {
Ok(_) => {
@@ -600,21 +721,18 @@ impl Drop for DeleteOnDrop {
fn drop(&mut self) {
let file = self.0.clone();
// Not using dc_delete_file() here because it would send a DeletedBlobFile event
async_std::task::block_on(fs::remove_file(file)).ok();
async_std::task::block_on(async move { fs::remove_file(file).await.ok() });
}
}
async fn export_backup_inner(
context: &Context,
temp_db_path: &Path,
temp_path: &Path,
) -> Result<()> {
async fn export_backup_inner(context: &Context, temp_path: &PathBuf) -> Result<()> {
let file = File::create(temp_path).await?;
let mut builder = async_tar::Builder::new(file);
// append_path_with_name() wants the source path as the first argument, append_dir_all() wants it as the second argument.
builder
.append_path_with_name(temp_db_path, DBFILE_BACKUP_NAME)
.append_path_with_name(context.get_dbfile(), DBFILE_BACKUP_NAME)
.await?;
let read_dir: Vec<_> = fs::read_dir(context.get_blobdir()).await?.collect().await;
@@ -818,7 +936,9 @@ mod tests {
#[async_std::test]
async fn test_render_setup_file() {
let t = TestContext::new_alice().await;
let t = TestContext::new().await;
t.configure_alice().await;
let msg = render_setup_file(&t, "hello").await.unwrap();
println!("{}", &msg);
// Check some substrings, indicating things got substituted.
@@ -835,10 +955,11 @@ mod tests {
#[async_std::test]
async fn test_render_setup_file_newline_replace() {
let t = TestContext::new_alice().await;
let t = TestContext::new().await;
t.set_stock_translation(StockMessage::AcSetupMsgBody, "hello\r\nthere".to_string())
.await
.unwrap();
t.configure_alice().await;
let msg = render_setup_file(&t, "pw").await.unwrap();
println!("{}", &msg);
assert!(msg.contains("<p>hello<br>there</p>"));
@@ -891,66 +1012,20 @@ mod tests {
#[async_std::test]
async fn test_export_and_import_key() {
let context = TestContext::new_alice().await;
let context = TestContext::new().await;
context.configure_alice().await;
let blobdir = context.ctx.get_blobdir();
if let Err(err) = imex(&context.ctx, ImexMode::ExportSelfKeys, blobdir, None).await {
if let Err(err) = imex(&context.ctx, ImexMode::ExportSelfKeys, blobdir).await {
panic!("got error on export: {:?}", err);
}
let context2 = TestContext::new_alice().await;
if let Err(err) = imex(&context2.ctx, ImexMode::ImportSelfKeys, blobdir, None).await {
let context2 = TestContext::new().await;
context2.configure_alice().await;
if let Err(err) = imex(&context2.ctx, ImexMode::ImportSelfKeys, blobdir).await {
panic!("got error on import: {:?}", err);
}
}
#[async_std::test]
async fn test_export_and_import_backup() -> Result<()> {
let backup_dir = tempfile::tempdir().unwrap();
let context1 = TestContext::new_alice().await;
assert!(context1.is_configured().await?);
let context2 = TestContext::new().await;
assert!(!context2.is_configured().await?);
assert!(has_backup(&context2, backup_dir.path().as_ref())
.await
.is_err());
// export from context1
assert!(imex(
&context1,
ImexMode::ExportBackup,
backup_dir.path().as_ref(),
None,
)
.await
.is_ok());
let _event = context1
.evtracker
.get_matching(|evt| matches!(evt, EventType::ImexProgress(1000)))
.await;
// import to context2
let backup = has_backup(&context2, backup_dir.path().as_ref()).await?;
assert!(
imex(&context2, ImexMode::ImportBackup, backup.as_ref(), None)
.await
.is_ok()
);
let _event = context2
.evtracker
.get_matching(|evt| matches!(evt, EventType::ImexProgress(1000)))
.await;
assert!(context2.is_configured().await?);
assert_eq!(
context2.get_config(Config::Addr).await?,
Some("alice@example.org".to_string())
);
Ok(())
}
#[test]
fn test_normalize_setup_code() {
let norm = normalize_setup_code("123422343234423452346234723482349234");

View File

@@ -3,24 +3,29 @@
//! This module implements a job queue maintained in the SQLite database
//! and job types.
use std::fmt;
use std::future::Future;
use anyhow::{bail, format_err, Context as _, Error, Result};
use anyhow::{bail, ensure, format_err, Context as _, Error, Result};
use async_smtp::smtp::response::{Category, Code, Detail};
use deltachat_derive::{FromSql, ToSql};
use rand::{thread_rng, Rng};
use crate::blob::BlobObject;
use crate::chat::{self, ChatId};
use crate::config::Config;
use crate::contact::{normalize_name, Contact, ContactId, Modifier, Origin};
use crate::contact::{normalize_name, Contact, Modifier, Origin};
use crate::context::Context;
use crate::dc_tools::time;
use crate::dc_tools::{dc_delete_file, dc_read_file, time};
use crate::ephemeral::load_imap_deletion_msgid;
use crate::events::EventType;
use crate::imap::{Imap, ImapActionResult};
use crate::location;
use crate::log::LogExt;
use crate::message::{Message, MsgId};
use crate::message::{self, Message, MessageState, MsgId};
use crate::mimefactory::MimeFactory;
use crate::param::{Param, Params};
use crate::scheduler::InterruptInfo;
use crate::smtp::{smtp_send, Smtp};
use crate::smtp::Smtp;
use crate::sql;
// results in ~3 weeks for the last backoff timespan
@@ -91,6 +96,11 @@ pub enum Action {
// this is user initiated so it should have a fairly high priority
UpdateRecentQuota = 140,
// Moving message is prioritized lower than deletion so we don't
// bother moving message if it is already scheduled for deletion.
MoveMsg = 200,
DeleteMsgOnImap = 210,
// This job will download partially downloaded messages completely
// and is added when download_full() is called.
// Most messages are downloaded automatically on fetch
@@ -105,6 +115,7 @@ pub enum Action {
MaybeSendLocations = 5005, // low priority ...
MaybeSendLocationsEnded = 5007,
SendMdn = 5010,
SendMsgToSmtp = 5901, // ... high priority
}
impl Default for Action {
@@ -122,14 +133,17 @@ impl From<Action> for Thread {
Housekeeping => Thread::Imap,
FetchExistingMsgs => Thread::Imap,
DeleteMsgOnImap => Thread::Imap,
ResyncFolders => Thread::Imap,
MarkseenMsgOnImap => Thread::Imap,
MoveMsg => Thread::Imap,
UpdateRecentQuota => Thread::Imap,
DownloadMsg => Thread::Imap,
MaybeSendLocations => Thread::Smtp,
MaybeSendLocationsEnded => Thread::Smtp,
SendMdn => Thread::Smtp,
SendMsgToSmtp => Thread::Smtp,
}
}
}
@@ -197,7 +211,7 @@ impl Job {
"UPDATE jobs SET desired_timestamp=?, tries=?, param=? WHERE id=?;",
paramsv![
self.desired_timestamp,
i64::from(self.tries),
self.tries as i64,
self.param.to_string(),
self.job_id as i32,
],
@@ -220,11 +234,222 @@ impl Job {
Ok(())
}
async fn smtp_send<F, Fut>(
&mut self,
context: &Context,
recipients: Vec<async_smtp::EmailAddress>,
message: Vec<u8>,
job_id: u32,
smtp: &mut Smtp,
success_cb: F,
) -> Status
where
F: FnOnce() -> Fut,
Fut: Future<Output = Result<()>>,
{
// hold the smtp lock during sending of a job and
// its ok/error response processing. Note that if a message
// was sent we need to mark it in the database ASAP as we
// otherwise might send it twice.
if std::env::var(crate::DCC_MIME_DEBUG).is_ok() {
info!(context, "smtp-sending out mime message:");
println!("{}", String::from_utf8_lossy(&message));
}
smtp.connectivity.set_working(context).await;
let send_result = smtp.send(context, recipients, message, job_id).await;
smtp.last_send_error = send_result.as_ref().err().map(|e| e.to_string());
let status = match send_result {
Err(crate::smtp::send::Error::SmtpSend(err)) => {
// Remote error, retry later.
warn!(context, "SMTP failed to send: {:?}", &err);
let res = match err {
async_smtp::smtp::error::Error::Permanent(ref response) => {
// Workaround for incorrectly configured servers returning permanent errors
// instead of temporary ones.
let maybe_transient = match response.code {
// Sometimes servers send a permanent error when actually it is a temporary error
// For documentation see <https://tools.ietf.org/html/rfc3463>
Code {
category: Category::MailSystem,
detail: Detail::Zero,
..
} => {
// Ignore status code 5.5.0, see <https://support.delta.chat/t/every-other-message-gets-stuck/877/2>
// Maybe incorrectly configured Postfix milter with "reject" instead of "tempfail", which returns
// "550 5.5.0 Service unavailable" instead of "451 4.7.1 Service unavailable - try again later".
//
// Other enhanced status codes, such as Postfix
// "550 5.1.1 <foobar@example.org>: Recipient address rejected: User unknown in local recipient table"
// are not ignored.
response.first_word() == Some(&"5.5.0".to_string())
}
_ => false,
};
if maybe_transient {
Status::RetryLater
} else {
// If we do not retry, add an info message to the chat.
// Yandex error "554 5.7.1 [2] Message rejected under suspicion of SPAM; https://ya.cc/..."
// should definitely go here, because user has to open the link to
// resume message sending.
Status::Finished(Err(format_err!("Permanent SMTP error: {}", err)))
}
}
async_smtp::smtp::error::Error::Transient(ref response) => {
// We got a transient 4xx response from SMTP server.
// Give some time until the server-side error maybe goes away.
if let Some(first_word) = response.first_word() {
if first_word.ends_with(".1.1")
|| first_word.ends_with(".1.2")
|| first_word.ends_with(".1.3")
{
// Sometimes we receive transient errors that should be permanent.
// Any extended smtp status codes like x.1.1, x.1.2 or x.1.3 that we
// receive as a transient error are misconfigurations of the smtp server.
// See <https://tools.ietf.org/html/rfc3463#section-3.2>
info!(context, "Smtp-job #{} Received extended status code {} for a transient error. This looks like a misconfigured smtp server, let's fail immediatly", self.job_id, first_word);
Status::Finished(Err(format_err!("Permanent SMTP error: {}", err)))
} else {
Status::RetryLater
}
} else {
Status::RetryLater
}
}
_ => {
if smtp.has_maybe_stale_connection().await {
info!(context, "stale connection? immediately reconnecting");
Status::RetryNow
} else {
Status::RetryLater
}
}
};
// this clears last_success info
smtp.disconnect().await;
res
}
Err(crate::smtp::send::Error::Envelope(err)) => {
// Local error, job is invalid, do not retry.
smtp.disconnect().await;
warn!(context, "SMTP job is invalid: {}", err);
Status::Finished(Err(err.into()))
}
Err(crate::smtp::send::Error::NoTransport) => {
// Should never happen.
// It does not even make sense to disconnect here.
error!(context, "SMTP job failed because SMTP has no transport");
Status::Finished(Err(format_err!("SMTP has not transport")))
}
Err(crate::smtp::send::Error::Other(err)) => {
// Local error, job is invalid, do not retry.
smtp.disconnect().await;
warn!(context, "unable to load job: {}", err);
Status::Finished(Err(err))
}
Ok(()) => {
job_try!(success_cb().await);
Status::Finished(Ok(()))
}
};
if let Status::Finished(Err(err)) = &status {
// We couldn't send the message, so mark it as failed
let msg_id = MsgId::new(self.foreign_id);
message::set_msg_failed(context, msg_id, Some(err.to_string())).await;
}
status
}
pub(crate) async fn send_msg_to_smtp(&mut self, context: &Context, smtp: &mut Smtp) -> Status {
// SMTP server, if not yet done
if let Err(err) = smtp.connect_configured(context).await {
warn!(context, "SMTP connection failure: {:?}", err);
smtp.last_send_error = Some(format!("SMTP connection failure: {:#}", err));
return Status::RetryLater;
}
let filename = job_try!(job_try!(self
.param
.get_path(Param::File, context)
.map_err(|_| format_err!("Can't get filename")))
.ok_or_else(|| format_err!("Can't get filename")));
let body = job_try!(dc_read_file(context, &filename).await);
let recipients = job_try!(self.param.get(Param::Recipients).ok_or_else(|| {
warn!(context, "Missing recipients for job {}", self.job_id);
format_err!("Missing recipients")
}));
let recipients_list = recipients
.split('\x1e')
.filter_map(
|addr| match async_smtp::EmailAddress::new(addr.to_string()) {
Ok(addr) => Some(addr),
Err(err) => {
warn!(context, "invalid recipient: {} {:?}", addr, err);
None
}
},
)
.collect::<Vec<_>>();
/* if there is a msg-id and it does not exist in the db, cancel sending.
this happends if dc_delete_msgs() was called
before the generated mime was sent out */
if 0 != self.foreign_id {
match message::exists(context, MsgId::new(self.foreign_id)).await {
Ok(exists) => {
if !exists {
return Status::Finished(Err(format_err!(
"Not sending Message {} as it was deleted",
self.foreign_id
)));
}
}
Err(err) => {
warn!(context, "failed to check message existence: {:?}", err);
smtp.last_send_error =
Some(format!("failed to check message existence: {:#}", err));
return Status::RetryLater;
}
}
};
let foreign_id = self.foreign_id;
self.smtp_send(context, recipients_list, body, self.job_id, smtp, || {
async move {
// smtp success, update db ASAP, then delete smtp file
if 0 != foreign_id {
set_delivered(context, MsgId::new(foreign_id)).await?;
}
// now also delete the generated file
dc_delete_file(context, filename).await;
// finally, create another send-job if there are items to be synced.
// triggering sync-job after msg-send-job guarantees, the recipient has grpid etc.
// once the sync message arrives.
// if there are no items to sync, this function returns fast.
context.send_sync_msg().await?;
Ok(())
}
})
.await
}
/// Get `SendMdn` jobs with foreign_id equal to `contact_id` excluding the `job_id` job.
async fn get_additional_mdn_jobs(
&self,
context: &Context,
contact_id: ContactId,
contact_id: u32,
) -> Result<(Vec<u32>, Vec<String>)> {
// Extract message IDs from job parameters
let res: Vec<(u32, MsgId)> = context
@@ -271,7 +496,7 @@ impl Job {
return Status::Finished(Err(format_err!("MDNs are disabled")));
}
let contact_id = ContactId::new(self.foreign_id);
let contact_id = self.foreign_id;
let contact = job_try!(Contact::load_from_db(context, contact_id).await);
if contact.is_blocked() {
return Status::Finished(Err(format_err!("Contact is blocked")));
@@ -318,13 +543,157 @@ impl Job {
return Status::RetryLater;
}
let status = smtp_send(context, &recipients, &body, smtp, msg_id, 0).await;
if matches!(status, Status::Finished(Ok(_))) {
// Remove additional SendMdn jobs we have aggregated into this one.
job_try!(kill_ids(context, &additional_job_ids).await);
self.smtp_send(context, recipients, body, self.job_id, smtp, || {
async move {
// Remove additional SendMdn jobs we have aggregated into this one.
kill_ids(context, &additional_job_ids).await?;
Ok(())
}
})
.await
}
async fn move_msg(&mut self, context: &Context, imap: &mut Imap) -> Status {
if let Err(err) = imap.prepare(context).await {
warn!(context, "could not connect: {:?}", err);
return Status::RetryLater;
}
status
let msg = job_try!(Message::load_from_db(context, MsgId::new(self.foreign_id)).await);
let server_folder = &job_try!(msg
.server_folder
.context("Can't move message out of folder if we don't know the current folder"));
let move_res = msg.id.needs_move(context, server_folder).await;
let dest_folder = match move_res {
Err(e) => {
warn!(context, "could not load dest folder: {}", e);
return Status::RetryLater;
}
Ok(None) => {
warn!(
context,
"msg {} does not need to be moved from {}", msg.id, server_folder
);
return Status::Finished(Ok(()));
}
Ok(Some(config)) => match context.get_config(config).await {
Ok(folder) => folder,
Err(err) => {
warn!(context, "failed to load config: {}", err);
return Status::RetryLater;
}
},
};
if let Some(dest_folder) = dest_folder {
match imap
.mv(context, server_folder, msg.server_uid, &dest_folder)
.await
{
ImapActionResult::RetryLater => Status::RetryLater,
ImapActionResult::Success => {
// Rust-Imap provides no target uid on mv, so just set it to 0, update again when precheck_imf() is called for the moved message
message::update_server_uid(context, &msg.rfc724_mid, &dest_folder, 0).await;
Status::Finished(Ok(()))
}
ImapActionResult::Failed => {
Status::Finished(Err(format_err!("IMAP action failed")))
}
ImapActionResult::AlreadyDone => Status::Finished(Ok(())),
}
} else {
Status::Finished(Err(format_err!("No mvbox folder configured")))
}
}
/// Deletes a message on the server.
///
/// `foreign_id` is a MsgId.
///
/// If the message is in the trash chat or hidden, this job
/// removes database record, otherwise it only clears the
/// `server_uid` column. If there are no more records pointing to
/// the same message on the server, the job actually removes the
/// message on the server.
async fn delete_msg_on_imap(&mut self, context: &Context, imap: &mut Imap) -> Status {
if let Err(err) = imap.prepare(context).await {
warn!(context, "could not connect: {:?}", err);
return Status::RetryLater;
}
let msg = job_try!(Message::load_from_db(context, MsgId::new(self.foreign_id)).await);
if !msg.rfc724_mid.is_empty() {
let cnt = message::rfc724_mid_cnt(context, &msg.rfc724_mid).await;
info!(
context,
"Running delete job for message {} which has {} entries in the database",
&msg.rfc724_mid,
cnt
);
if cnt > 1 {
info!(
context,
"The message is deleted from the server when all parts are deleted.",
);
} else if cnt == 0 {
warn!(
context,
"The message {} has no UID on the server to delete", &msg.rfc724_mid
);
} else {
/* if this is the last existing part of the message,
we delete the message from the server */
let mid = msg.rfc724_mid;
let server_folder = msg.server_folder.as_ref().unwrap();
let res = if msg.server_uid == 0 {
// Message is already deleted on IMAP server.
ImapActionResult::AlreadyDone
} else {
imap.delete_msg(context, &mid, server_folder, msg.server_uid)
.await
};
match res {
ImapActionResult::AlreadyDone | ImapActionResult::Success => {}
ImapActionResult::RetryLater | ImapActionResult::Failed => {
// If job has failed, for example due to some
// IMAP bug, we postpone it instead of failing
// immediately. This will prevent adding it
// immediately again if user has enabled
// automatic message deletion. Without this,
// we might waste a lot of traffic constantly
// retrying message deletion.
return Status::RetryLater;
}
}
}
if msg.chat_id.is_trash() || msg.hidden {
// Messages are stored in trash chat only to keep
// their server UID and Message-ID. Once message is
// deleted from the server, database record can be
// removed as well.
//
// Hidden messages are similar to trashed, but are
// related to some chat. We also delete their
// database records.
job_try!(msg.id.delete_from_db(context).await)
} else {
// Remove server UID from the database record.
//
// We have either just removed the message from the
// server, in which case UID is not valid anymore, or
// we have more refernces to the same server UID, so
// we remove UID to reduce the number of messages
// pointing to the corresponding UID. Once the counter
// reaches zero, we will remove the message.
job_try!(msg.id.unlink(context).await);
}
Status::Finished(Ok(()))
} else {
/* eg. device messages have no Message-ID */
Status::Finished(Ok(()))
}
}
/// Read the recipients from old emails sent by the user and add them as contacts.
@@ -365,38 +734,37 @@ impl Job {
Status::Finished(Ok(()))
}
/// Synchronizes UIDs for all folders.
/// Synchronizes UIDs for sentbox, inbox and mvbox, in this order.
///
/// If a copy of the message is present in multiple folders, mvbox
/// is preferred to inbox, which is in turn preferred to
/// sentbox. This is because in the database it is impossible to
/// store multiple UIDs for one message, so we prefer to
/// automatically delete messages in the folders managed by Delta
/// Chat in contrast to the Sent folder, which is normally managed
/// by the user via webmail or another email client.
async fn resync_folders(&mut self, context: &Context, imap: &mut Imap) -> Status {
if let Err(err) = imap.prepare(context).await {
warn!(context, "could not connect: {:?}", err);
return Status::RetryLater;
}
let all_folders = match imap.list_folders(context).await {
Ok(v) => v,
Err(e) => {
warn!(context, "Listing folders for resync failed: {:#}", e);
return Status::RetryLater;
}
};
let mut any_failed = false;
for folder in all_folders {
if let Err(e) = imap
.resync_folder_uids(context, folder.name().to_string())
.await
{
warn!(context, "{:#}", e);
any_failed = true;
}
let sentbox_folder = job_try!(context.get_config(Config::ConfiguredSentboxFolder).await);
if let Some(sentbox_folder) = sentbox_folder {
job_try!(imap.resync_folder_uids(context, sentbox_folder).await);
}
if any_failed {
Status::RetryLater
} else {
Status::Finished(Ok(()))
let inbox_folder = job_try!(context.get_config(Config::ConfiguredInboxFolder).await);
if let Some(inbox_folder) = inbox_folder {
job_try!(imap.resync_folder_uids(context, inbox_folder).await);
}
let mvbox_folder = job_try!(context.get_config(Config::ConfiguredMvboxFolder).await);
if let Some(mvbox_folder) = mvbox_folder {
job_try!(imap.resync_folder_uids(context, mvbox_folder).await);
}
Status::Finished(Ok(()))
}
async fn markseen_msg_on_imap(&mut self, context: &Context, imap: &mut Imap) -> Status {
@@ -406,58 +774,55 @@ impl Job {
}
let msg = job_try!(Message::load_from_db(context, MsgId::new(self.foreign_id)).await);
let row = job_try!(
context
.sql
.query_row_optional(
"SELECT uid, folder FROM imap
WHERE rfc724_mid=? AND folder=target
ORDER BY uid ASC
LIMIT 1",
paramsv![msg.rfc724_mid],
|row| {
let uid: u32 = row.get(0)?;
let folder: String = row.get(1)?;
Ok((uid, folder))
}
)
.await
);
if let Some((server_uid, server_folder)) = row {
let result = imap.set_seen(context, &server_folder, server_uid).await;
match result {
ImapActionResult::RetryLater => return Status::RetryLater,
ImapActionResult::Success | ImapActionResult::Failed => {}
}
let folder = msg.server_folder.as_ref().unwrap();
let result = if msg.server_uid == 0 {
// The message is moved or deleted by us.
//
// Do not call set_seen with zero UID, as it will return
// ImapActionResult::RetryLater, but we do not want to
// retry. If the message was moved, we will create another
// job to mark the message as seen later. If it was
// deleted, there is nothing to do.
info!(context, "Can't mark message as seen: No UID");
ImapActionResult::Failed
} else {
info!(
context,
"Can't mark the message {} as seen on IMAP because there is no known UID",
msg.rfc724_mid
);
}
imap.set_seen(context, folder, msg.server_uid).await
};
// XXX we send MDN even in case of failure to mark the messages as seen, e.g. if it was
// already deleted on the server by another device. The job will not be retried so locally
// there is no risk of double-sending MDNs.
//
// Read receipts for system messages are never sent. These messages have no place to
// display received read receipt anyway. And since their text is locally generated,
// quoting them is dangerous as it may contain contact names. E.g., for original message
// "Group left by me", a read receipt will quote "Group left by <name>", and the name can
// be a display name stored in address book rather than the name sent in the From field by
// the user.
if msg.param.get_bool(Param::WantsMdn).unwrap_or_default() && !msg.is_system_message() {
let mdns_enabled = job_try!(context.get_config_bool(Config::MdnsEnabled).await);
if mdns_enabled {
if let Err(err) = send_mdn(context, &msg).await {
warn!(context, "could not send out mdn for {}: {}", msg.id, err);
return Status::Finished(Err(err));
match result {
ImapActionResult::RetryLater => Status::RetryLater,
ImapActionResult::AlreadyDone => Status::Finished(Ok(())),
ImapActionResult::Success | ImapActionResult::Failed => {
// XXX the message might just have been moved
// we want to send out an MDN anyway
// The job will not be retried so locally
// there is no risk of double-sending MDNs.
//
// Read receipts for system messages are never
// sent. These messages have no place to display
// received read receipt anyway. And since their text
// is locally generated, quoting them is dangerous as
// it may contain contact names. E.g., for original
// message "Group left by me", a read receipt will
// quote "Group left by <name>", and the name can be a
// display name stored in address book rather than
// the name sent in the From field by the user.
if msg.param.get_bool(Param::WantsMdn).unwrap_or_default()
&& !msg.is_system_message()
{
let mdns_enabled = job_try!(context.get_config_bool(Config::MdnsEnabled).await);
if mdns_enabled {
if let Err(err) = send_mdn(context, &msg).await {
warn!(context, "could not send out mdn for {}: {}", msg.id, err);
return Status::Finished(Err(err));
}
}
}
Status::Finished(Ok(()))
}
}
Status::Finished(Ok(()))
}
}
@@ -472,12 +837,9 @@ pub async fn kill_action(context: &Context, action: Action) -> Result<()> {
/// Remove jobs with specified IDs.
async fn kill_ids(context: &Context, job_ids: &[u32]) -> Result<()> {
if job_ids.is_empty() {
return Ok(());
}
let q = format!(
"DELETE FROM jobs WHERE id IN({})",
sql::repeat_vars(job_ids.len())?
job_ids.iter().map(|_| "?").collect::<Vec<&str>>().join(",")
);
context
.sql
@@ -497,6 +859,17 @@ pub async fn action_exists(context: &Context, action: Action) -> Result<bool> {
Ok(exists)
}
async fn set_delivered(context: &Context, msg_id: MsgId) -> Result<()> {
message::update_msg_state(context, msg_id, MessageState::OutDelivered).await;
let chat_id: ChatId = context
.sql
.query_get_value("SELECT chat_id FROM msgs WHERE id=?", paramsv![msg_id])
.await?
.unwrap_or_default();
context.emit_event(EventType::MsgDelivered { chat_id, msg_id });
Ok(())
}
async fn add_all_recipients_as_contacts(context: &Context, imap: &mut Imap, folder: Config) {
let mailbox = if let Ok(Some(m)) = context.get_config(folder).await {
m
@@ -515,7 +888,7 @@ async fn add_all_recipients_as_contacts(context: &Context, imap: &mut Imap, fold
let display_name_normalized = contact
.display_name
.as_ref()
.map(|s| normalize_name(s))
.map(normalize_name)
.unwrap_or_default();
match Contact::add_or_lookup(
@@ -542,11 +915,144 @@ async fn add_all_recipients_as_contacts(context: &Context, imap: &mut Imap, fold
};
}
/// Constructs a job for sending a message.
///
/// Returns `None` if no messages need to be sent out.
///
/// In order to be processed, must be `add`ded.
pub async fn send_msg_job(context: &Context, msg_id: MsgId) -> Result<Option<Job>> {
let mut msg = Message::load_from_db(context, msg_id).await?;
msg.try_calc_and_set_dimensions(context).await.ok();
/* create message */
let needs_encryption = msg.param.get_bool(Param::GuaranteeE2ee).unwrap_or_default();
let attach_selfavatar = match chat::shall_attach_selfavatar(context, msg.chat_id).await {
Ok(attach_selfavatar) => attach_selfavatar,
Err(err) => {
warn!(context, "job: cannot get selfavatar-state: {}", err);
false
}
};
let mimefactory = MimeFactory::from_msg(context, &msg, attach_selfavatar).await?;
let mut recipients = mimefactory.recipients();
let from = context
.get_config(Config::ConfiguredAddr)
.await?
.unwrap_or_default();
let lowercase_from = from.to_lowercase();
// Send BCC to self if it is enabled and we are not going to
// delete it immediately.
if context.get_config_bool(Config::BccSelf).await?
&& context.get_config_delete_server_after().await? != Some(0)
&& !recipients
.iter()
.any(|x| x.to_lowercase() == lowercase_from)
{
recipients.push(from);
}
if recipients.is_empty() {
// may happen eg. for groups with only SELF and bcc_self disabled
info!(
context,
"message {} has no recipient, skipping smtp-send", msg_id
);
set_delivered(context, msg_id).await?;
return Ok(None);
}
let rendered_msg = match mimefactory.render(context).await {
Ok(res) => Ok(res),
Err(err) => {
message::set_msg_failed(context, msg_id, Some(err.to_string())).await;
Err(err)
}
}?;
if needs_encryption && !rendered_msg.is_encrypted {
/* unrecoverable */
message::set_msg_failed(
context,
msg_id,
Some("End-to-end-encryption unavailable unexpectedly."),
)
.await;
bail!(
"e2e encryption unavailable {} - {:?}",
msg_id,
needs_encryption
);
}
if rendered_msg.is_gossiped {
msg.chat_id.set_gossiped_timestamp(context, time()).await?;
}
if 0 != rendered_msg.last_added_location_id {
if let Err(err) = location::set_kml_sent_timestamp(context, msg.chat_id, time()).await {
error!(context, "Failed to set kml sent_timestamp: {:?}", err);
}
if !msg.hidden {
if let Err(err) =
location::set_msg_location_id(context, msg.id, rendered_msg.last_added_location_id)
.await
{
error!(context, "Failed to set msg_location_id: {:?}", err);
}
}
}
if let Some(sync_ids) = rendered_msg.sync_ids_to_delete {
if let Err(err) = context.delete_sync_ids(sync_ids).await {
error!(context, "Failed to delete sync ids: {:?}", err);
}
}
if attach_selfavatar {
if let Err(err) = msg.chat_id.set_selfavatar_timestamp(context, time()).await {
error!(context, "Failed to set selfavatar timestamp: {:?}", err);
}
}
if rendered_msg.is_encrypted && !needs_encryption {
msg.param.set_int(Param::GuaranteeE2ee, 1);
msg.update_param(context).await;
}
ensure!(!recipients.is_empty(), "no recipients for smtp job set");
let mut param = Params::new();
let bytes = &rendered_msg.message;
let blob = BlobObject::create(context, &rendered_msg.rfc724_mid, bytes).await?;
let recipients = recipients.join("\x1e");
param.set(Param::File, blob.as_name());
param.set(Param::Recipients, &recipients);
msg.subject = rendered_msg.subject.clone();
msg.update_subject(context).await;
let job = create(Action::SendMsgToSmtp, msg_id.to_u32(), param, 0)?;
Ok(Some(job))
}
pub(crate) enum Connection<'a> {
Inbox(&'a mut Imap),
Smtp(&'a mut Smtp),
}
pub(crate) async fn load_imap_deletion_job(context: &Context) -> Result<Option<Job>> {
let res = load_imap_deletion_msgid(context)
.await?
.map(|msg_id| Job::new(Action::DeleteMsgOnImap, msg_id.to_u32(), Params::new(), 0));
Ok(res)
}
impl<'a> fmt::Display for Connection<'a> {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
match self {
@@ -649,13 +1155,16 @@ async fn perform_job_action(
let try_res = match job.action {
Action::Unknown => Status::Finished(Err(format_err!("Unknown job id found"))),
Action::SendMsgToSmtp => job.send_msg_to_smtp(context, connection.smtp()).await,
Action::SendMdn => job.send_mdn(context, connection.smtp()).await,
Action::MaybeSendLocations => location::job_maybe_send_locations(context, job).await,
Action::MaybeSendLocationsEnded => {
location::job_maybe_send_locations_ended(context, job).await
}
Action::DeleteMsgOnImap => job.delete_msg_on_imap(context, connection.inbox()).await,
Action::ResyncFolders => job.resync_folders(context, connection.inbox()).await,
Action::MarkseenMsgOnImap => job.markseen_msg_on_imap(context, connection.inbox()).await,
Action::MoveMsg => job.move_msg(context, connection.inbox()).await,
Action::FetchExistingMsgs => job.fetch_existing_msgs(context, connection.inbox()).await,
Action::Housekeeping => {
sql::housekeeping(context).await.ok_or_log(context);
@@ -688,7 +1197,7 @@ fn get_backoff_time_offset(tries: u32, action: Action) -> i64 {
if seconds < 1 {
seconds = 1;
}
i64::from(seconds)
seconds as i64
}
}
}
@@ -697,11 +1206,7 @@ async fn send_mdn(context: &Context, msg: &Message) -> Result<()> {
let mut param = Params::new();
param.set(Param::MsgId, msg.id.to_u32().to_string());
add(
context,
Job::new(Action::SendMdn, msg.from_id.to_u32(), param, 0),
)
.await?;
add(context, Job::new(Action::SendMdn, msg.from_id, param, 0)).await?;
Ok(())
}
@@ -716,6 +1221,16 @@ pub(crate) async fn schedule_resync(context: &Context) -> Result<()> {
Ok(())
}
/// Creates a job.
pub fn create(action: Action, foreign_id: u32, param: Params, delay_seconds: i64) -> Result<Job> {
ensure!(
action != Action::Unknown,
"Invalid action passed to job_add"
);
Ok(Job::new(action, foreign_id, param, delay_seconds))
}
/// Adds a job to the database, scheduling it.
pub async fn add(context: &Context, job: Job) -> Result<()> {
let action = job.action;
@@ -726,17 +1241,26 @@ pub async fn add(context: &Context, job: Job) -> Result<()> {
match action {
Action::Unknown => unreachable!(),
Action::Housekeeping
| Action::DeleteMsgOnImap
| Action::ResyncFolders
| Action::MarkseenMsgOnImap
| Action::FetchExistingMsgs
| Action::MoveMsg
| Action::UpdateRecentQuota
| Action::DownloadMsg => {
info!(context, "interrupt: imap");
context.interrupt_inbox(InterruptInfo::new(false)).await;
context
.interrupt_inbox(InterruptInfo::new(false, None))
.await;
}
Action::MaybeSendLocations | Action::MaybeSendLocationsEnded | Action::SendMdn => {
Action::MaybeSendLocations
| Action::MaybeSendLocationsEnded
| Action::SendMdn
| Action::SendMsgToSmtp => {
info!(context, "interrupt: smtp");
context.interrupt_smtp(InterruptInfo::new(false)).await;
context
.interrupt_smtp(InterruptInfo::new(false, None))
.await;
}
}
}
@@ -771,9 +1295,20 @@ pub(crate) async fn load_next(
let query;
let params;
let t = time();
let m;
let thread_i = thread as i64;
if !info.probe_network {
if let Some(msg_id) = info.msg_id {
query = r#"
SELECT id, action, foreign_id, param, added_timestamp, desired_timestamp, tries
FROM jobs
WHERE thread=? AND foreign_id=?
ORDER BY action DESC, added_timestamp
LIMIT 1;
"#;
m = msg_id;
params = paramsv![thread_i, m];
} else if !info.probe_network {
// processing for first-try and after backoff-timeouts:
// process jobs in the order they were added.
query = r#"
@@ -843,6 +1378,12 @@ LIMIT 1;
}
Thread::Imap => {
if let Some(job) = job {
if job.action < Action::DeleteMsgOnImap {
Ok(load_imap_deletion_job(context).await?.or(Some(job)))
} else {
Ok(Some(job))
}
} else if let Some(job) = load_imap_deletion_job(context).await? {
Ok(Some(job))
} else {
Ok(load_housekeeping_job(context).await?)
@@ -868,12 +1409,8 @@ mod tests {
VALUES (?, ?, ?, ?, ?, ?);",
paramsv![
now,
Thread::from(Action::DownloadMsg),
if valid {
Action::DownloadMsg as i32
} else {
-1
},
Thread::from(Action::MoveMsg),
if valid { Action::MoveMsg as i32 } else { -1 },
foreign_id,
Params::new().to_string(),
now
@@ -892,8 +1429,8 @@ mod tests {
insert_job(&t, 1, false).await; // This can not be loaded into Job struct.
let jobs = load_next(
&t,
Thread::from(Action::DownloadMsg),
&InterruptInfo::new(false),
Thread::from(Action::MoveMsg),
&InterruptInfo::new(false, None),
)
.await?;
// The housekeeping job should be loaded as we didn't run housekeeping in the last day:
@@ -902,8 +1439,8 @@ mod tests {
insert_job(&t, 1, true).await;
let jobs = load_next(
&t,
Thread::from(Action::DownloadMsg),
&InterruptInfo::new(false),
Thread::from(Action::MoveMsg),
&InterruptInfo::new(false, None),
)
.await?;
assert!(jobs.is_some());
@@ -918,8 +1455,8 @@ mod tests {
let jobs = load_next(
&t,
Thread::from(Action::DownloadMsg),
&InterruptInfo::new(false),
Thread::from(Action::MoveMsg),
&InterruptInfo::new(false, None),
)
.await?;
assert!(jobs.is_some());

View File

@@ -4,7 +4,7 @@ use std::collections::BTreeMap;
use std::fmt;
use std::io::Cursor;
use anyhow::{format_err, Context as _, Result};
use anyhow::{format_err, Result};
use async_trait::async_trait;
use num_traits::FromPrimitive;
use pgp::composed::Deserializable;
@@ -50,7 +50,8 @@ pub trait DcKey: Serialize + Deserializable + KeyTrait + Clone {
/// the ASCII-armored representation.
fn from_asc(data: &str) -> Result<(Self::KeyType, BTreeMap<String, String>)> {
let bytes = data.as_bytes();
Self::KeyType::from_armor_single(Cursor::new(bytes)).context("rPGP error")
Self::KeyType::from_armor_single(Cursor::new(bytes))
.map_err(|err| format_err!("rPGP error: {}", err))
}
/// Load the users' default key from the database.
@@ -201,7 +202,7 @@ async fn generate_keypair(context: &Context) -> Result<KeyPair> {
let addr = context
.get_config(Config::ConfiguredAddr)
.await?
.context("no address configured")?;
.ok_or_else(|| format_err!("No address configured"))?;
let addr = EmailAddress::new(&addr)?;
let _guard = context.generating_key_mutex.lock().await;
@@ -288,17 +289,17 @@ pub async fn store_self_keypair(
paramsv![public_key, secret_key],
)
.await
.context("failed to remove old use of key")?;
.map_err(|err| err.context("failed to remove old use of key"))?;
if default == KeyPairUse::Default {
context
.sql
.execute("UPDATE keypairs SET is_default=0;", paramsv![])
.await
.context("failed to clear default")?;
.map_err(|err| err.context("failed to clear default"))?;
}
let is_default = match default {
KeyPairUse::Default => i32::from(true),
KeyPairUse::ReadOnly => i32::from(false),
KeyPairUse::Default => true as i32,
KeyPairUse::ReadOnly => false as i32,
};
let addr = keypair.addr.to_string();
@@ -312,13 +313,13 @@ pub async fn store_self_keypair(
paramsv![addr, is_default, public_key, secret_key, t],
)
.await
.context("failed to insert keypair")?;
.map_err(|err| err.context("failed to insert keypair"))?;
Ok(())
}
/// A key fingerprint
#[derive(Clone, Eq, PartialEq, Hash, serde::Serialize, serde::Deserialize)]
#[derive(Clone, Eq, PartialEq, Hash)]
pub struct Fingerprint(Vec<u8>);
impl Fingerprint {
@@ -509,7 +510,8 @@ i8pcjGO+IZffvyZJVRWfVooBJmWWbPB1pueo3tx8w3+fcuzpxz+RLFKaPyqXO+dD
#[async_std::test]
async fn test_load_self_existing() {
let alice = alice_keypair();
let t = TestContext::new_alice().await;
let t = TestContext::new().await;
t.configure_alice().await;
let pubkey = SignedPublicKey::load_self(&t).await.unwrap();
assert_eq!(alice.public, pubkey);
let seckey = SignedSecretKey::load_self(&t).await.unwrap();
@@ -519,7 +521,7 @@ i8pcjGO+IZffvyZJVRWfVooBJmWWbPB1pueo3tx8w3+fcuzpxz+RLFKaPyqXO+dD
#[async_std::test]
async fn test_load_self_generate_public() {
let t = TestContext::new().await;
t.set_config(Config::ConfiguredAddr, Some("alice@example.org"))
t.set_config(Config::ConfiguredAddr, Some("alice@example.com"))
.await
.unwrap();
let key = SignedPublicKey::load_self(&t).await;
@@ -529,7 +531,7 @@ i8pcjGO+IZffvyZJVRWfVooBJmWWbPB1pueo3tx8w3+fcuzpxz+RLFKaPyqXO+dD
#[async_std::test]
async fn test_load_self_generate_secret() {
let t = TestContext::new().await;
t.set_config(Config::ConfiguredAddr, Some("alice@example.org"))
t.set_config(Config::ConfiguredAddr, Some("alice@example.com"))
.await
.unwrap();
let key = SignedSecretKey::load_self(&t).await;
@@ -541,7 +543,7 @@ i8pcjGO+IZffvyZJVRWfVooBJmWWbPB1pueo3tx8w3+fcuzpxz+RLFKaPyqXO+dD
use std::thread;
let t = TestContext::new().await;
t.set_config(Config::ConfiguredAddr, Some("alice@example.org"))
t.set_config(Config::ConfiguredAddr, Some("alice@example.com"))
.await
.unwrap();
let thr0 = {
@@ -587,6 +589,27 @@ i8pcjGO+IZffvyZJVRWfVooBJmWWbPB1pueo3tx8w3+fcuzpxz+RLFKaPyqXO+dD
assert_eq!(nrows().await, 1);
}
// Convenient way to create a new key if you need one, run with
// `cargo test key::tests::gen_key`.
// #[test]
// fn gen_key() {
// let name = "fiona";
// let keypair = crate::pgp::create_keypair(
// EmailAddress::new(&format!("{}@example.net", name)).unwrap(),
// )
// .unwrap();
// std::fs::write(
// format!("test-data/key/{}-public.asc", name),
// keypair.public.to_base64(),
// )
// .unwrap();
// std::fs::write(
// format!("test-data/key/{}-secret.asc", name),
// keypair.secret.to_base64(),
// )
// .unwrap();
// }
#[test]
fn test_fingerprint_from_str() {
let res = Fingerprint::new(vec![

View File

@@ -79,7 +79,8 @@ mod tests {
#[async_std::test]
async fn test_keyring_load_self() {
// new_self() implies load_self()
let t = TestContext::new_alice().await;
let t = TestContext::new().await;
t.configure_alice().await;
let alice = alice_keypair();
let pub_ring: Keyring<SignedPublicKey> = Keyring::new_self(&t).await.unwrap();

View File

@@ -7,14 +7,12 @@
clippy::all,
clippy::indexing_slicing,
clippy::wildcard_imports,
clippy::needless_borrow,
clippy::cast_lossless
clippy::needless_borrow
)]
#![allow(
clippy::match_bool,
clippy::eval_order_dependence,
clippy::bool_assert_comparison,
clippy::manual_split_once
clippy::bool_assert_comparison
)]
#[macro_use]
@@ -88,7 +86,6 @@ pub mod stock_str;
mod sync;
mod token;
mod update_helper;
pub mod webxdc;
#[macro_use]
mod dehtml;
mod color;

View File

@@ -7,13 +7,12 @@ use quick_xml::events::{BytesEnd, BytesStart, BytesText};
use crate::chat::{self, ChatId};
use crate::config::Config;
use crate::constants::DC_CONTACT_ID_SELF;
use crate::contact::ContactId;
use crate::constants::{Viewtype, DC_CONTACT_ID_SELF};
use crate::context::Context;
use crate::dc_tools::time;
use crate::events::EventType;
use crate::job::{self, Job};
use crate::message::{Message, MsgId, Viewtype};
use crate::message::{Message, MsgId};
use crate::mimeparser::SystemMessage;
use crate::param::Params;
use crate::stock_str;
@@ -26,7 +25,7 @@ pub struct Location {
pub longitude: f64,
pub accuracy: f64,
pub timestamp: i64,
pub contact_id: ContactId,
pub contact_id: u32,
pub msg_id: u32,
pub chat_id: ChatId,
pub marker: Option<String>,
@@ -102,10 +101,10 @@ impl Kml {
let val = event.unescape_and_decode(reader).unwrap_or_default();
let val = val
.replace('\n', "")
.replace('\r', "")
.replace('\t', "")
.replace(' ', "");
.replace("\n", "")
.replace("\r", "")
.replace("\t", "")
.replace(" ", "");
if self.tag.contains(KmlTag::WHEN) && val.len() >= 19 {
// YYYY-MM-DDTHH:MM:SSZ
@@ -224,7 +223,7 @@ pub async fn send_locations_to_chat(
.unwrap_or_default();
} else if 0 == seconds && is_sending_locations_before {
let stock_str = stock_str::msg_location_disabled(context).await;
chat::add_info_msg(context, chat_id, &stock_str, now).await?;
chat::add_info_msg(context, chat_id, stock_str, now).await?;
}
context.emit_event(EventType::ChatModified(chat_id));
if 0 != seconds {
@@ -559,7 +558,7 @@ pub async fn set_msg_location_id(context: &Context, msg_id: MsgId, location_id:
pub(crate) async fn save(
context: &Context,
chat_id: ChatId,
contact_id: ContactId,
contact_id: u32,
locations: &[Location],
independent: bool,
) -> Result<Option<u32>> {
@@ -586,12 +585,12 @@ pub(crate) async fn save(
conn.prepare_cached("SELECT id FROM locations WHERE timestamp=? AND from_id=?")?;
let mut stmt_insert = conn.prepare_cached(stmt_insert)?;
let exists = stmt_test.exists(paramsv![timestamp, contact_id])?;
let exists = stmt_test.exists(paramsv![timestamp, contact_id as i32])?;
if independent || !exists {
stmt_insert.execute(paramsv![
timestamp,
contact_id,
contact_id as i32,
chat_id,
latitude,
longitude,
@@ -748,7 +747,7 @@ pub(crate) async fn job_maybe_send_locations_ended(
);
let stock_str = stock_str::msg_location_disabled(context).await;
job_try!(chat::add_info_msg(context, chat_id, &stock_str, now).await);
job_try!(chat::add_info_msg(context, chat_id, stock_str, now).await);
context.emit_event(EventType::ChatModified(chat_id));
}
}

View File

@@ -253,8 +253,7 @@ impl LoginParam {
sql.set_raw_config(key, Some(&self.imap.server)).await?;
let key = format!("{}mail_port", prefix);
sql.set_raw_config_int(key, i32::from(self.imap.port))
.await?;
sql.set_raw_config_int(key, self.imap.port as i32).await?;
let key = format!("{}mail_user", prefix);
sql.set_raw_config(key, Some(&self.imap.user)).await?;
@@ -274,8 +273,7 @@ impl LoginParam {
sql.set_raw_config(key, Some(&self.smtp.server)).await?;
let key = format!("{}send_port", prefix);
sql.set_raw_config_int(key, i32::from(self.smtp.port))
.await?;
sql.set_raw_config_int(key, self.smtp.port as i32).await?;
let key = format!("{}send_user", prefix);
sql.set_raw_config(key, Some(&self.smtp.user)).await?;
@@ -415,7 +413,7 @@ mod tests {
let t = TestContext::new().await;
let param = LoginParam {
addr: "alice@example.org".to_string(),
addr: "alice@example.com".to_string(),
imap: ServerLoginParam {
server: "imap.example.com".to_string(),
user: "alice".to_string(),
@@ -426,7 +424,7 @@ mod tests {
},
smtp: ServerLoginParam {
server: "smtp.example.com".to_string(),
user: "alice@example.org".to_string(),
user: "alice@example.com".to_string(),
password: "bar".to_string(),
port: 456,
security: Socket::Ssl,

File diff suppressed because it is too large Load Diff

View File

@@ -2,14 +2,14 @@
use std::convert::TryInto;
use anyhow::{bail, ensure, Context as _, Result};
use anyhow::{bail, ensure, format_err, Result};
use chrono::TimeZone;
use lettre_email::{mime, Address, Header, MimeMultipartType, PartBuilder};
use crate::blob::BlobObject;
use crate::chat::Chat;
use crate::chat::{self, Chat};
use crate::config::Config;
use crate::constants::{Chattype, DC_FROM_HANDSHAKE};
use crate::constants::{Chattype, Viewtype, DC_FROM_HANDSHAKE};
use crate::contact::Contact;
use crate::context::{get_version_str, Context};
use crate::dc_tools::IsNoneOrEmpty;
@@ -22,7 +22,7 @@ use crate::ephemeral::Timer as EphemeralTimer;
use crate::format_flowed::{format_flowed, format_flowed_quote};
use crate::html::new_html_mimepart;
use crate::location;
use crate::message::{self, Message, MsgId, Viewtype};
use crate::message::{self, Message, MsgId};
use crate::mimeparser::SystemMessage;
use crate::param::Param;
use crate::peerstate::{Peerstate, PeerstateVerifiedStatus};
@@ -81,7 +81,7 @@ pub struct MimeFactory<'a> {
/// Result of rendering a message, ready to be submitted to a send job.
#[derive(Debug, Clone)]
pub struct RenderedEmail {
pub message: String,
pub message: Vec<u8>,
// pub envelope: Envelope,
pub is_encrypted: bool,
pub is_gossiped: bool,
@@ -154,12 +154,6 @@ impl<'a> MimeFactory<'a> {
if chat.is_self_talk() {
recipients.push((from_displayname.to_string(), from_addr.to_string()));
} else if chat.is_mailing_list() {
let list_post = chat
.param
.get(Param::ListPost)
.context("Can't write to mailinglist without ListPost param")?;
recipients.push(("".to_string(), list_post.to_string()));
} else {
context
.sql
@@ -207,6 +201,7 @@ impl<'a> MimeFactory<'a> {
)
.await?;
let default_str = stock_str::status_line(context).await;
let factory = MimeFactory {
from_addr,
from_displayname,
@@ -214,7 +209,7 @@ impl<'a> MimeFactory<'a> {
selfstatus: context
.get_config(Config::Selfstatus)
.await?
.unwrap_or_default(),
.unwrap_or(default_str),
recipients,
timestamp: msg.timestamp_sort,
loaded: Loaded::Message { chat },
@@ -245,10 +240,11 @@ impl<'a> MimeFactory<'a> {
.get_config(Config::Displayname)
.await?
.unwrap_or_default();
let default_str = stock_str::status_line(context).await;
let selfstatus = context
.get_config(Config::Selfstatus)
.await?
.unwrap_or_default();
.unwrap_or(default_str);
let timestamp = dc_create_smeared_timestamp(context).await;
let res = MimeFactory::<'a> {
@@ -281,7 +277,7 @@ impl<'a> MimeFactory<'a> {
let self_addr = context
.get_config(Config::ConfiguredAddr)
.await?
.context("not configured")?;
.ok_or_else(|| format_err!("Not configured"))?;
let mut res = Vec::new();
for (_, addr) in self
@@ -462,121 +458,49 @@ impl<'a> MimeFactory<'a> {
self.from_addr.clone(),
);
let undisclosed_recipients = match &self.loaded {
Loaded::Message { chat } => chat.typ == Chattype::Broadcast,
Loaded::Mdn { .. } => false,
};
let mut to = Vec::new();
if undisclosed_recipients {
to.push(Address::new_group(
"hidden-recipients".to_string(),
Vec::new(),
));
} else {
let email_to_remove =
if self.msg.param.get_cmd() == SystemMessage::MemberRemovedFromGroup {
self.msg.param.get(Param::Arg)
} else {
None
};
for (name, addr) in self.recipients.iter() {
if let Some(email_to_remove) = email_to_remove {
if email_to_remove == addr {
continue;
}
}
if name.is_empty() {
to.push(Address::new_mailbox(addr.clone()));
} else {
to.push(Address::new_mailbox_with_name(
name.to_string(),
addr.clone(),
));
}
}
if to.is_empty() {
to.push(from.clone());
for (name, addr) in self.recipients.iter() {
if name.is_empty() {
to.push(Address::new_mailbox(addr.clone()));
} else {
to.push(Address::new_mailbox_with_name(
name.to_string(),
addr.clone(),
));
}
}
// Start with Internet Message Format headers in the order of the standard example
// <https://datatracker.ietf.org/doc/html/rfc5322#appendix-A.1.1>.
headers
.unprotected
.push(Header::new_with_value("From".into(), vec![from]).unwrap());
if let Some(sender_displayname) = &self.sender_displayname {
let sender =
Address::new_mailbox_with_name(sender_displayname.clone(), self.from_addr.clone());
headers
.unprotected
.push(Header::new_with_value("Sender".into(), vec![sender]).unwrap());
}
headers
.unprotected
.push(Header::new_with_value("To".into(), to).unwrap());
let subject_str = self.subject_str(context).await?;
let encoded_subject = if subject_str
.chars()
.all(|c| c.is_ascii_alphanumeric() || c == ' ')
// We do not use needs_encoding() here because needs_encoding() returns true if the string contains a space
// but we do not want to encode all subjects just because they contain a space.
{
subject_str.clone()
} else {
encode_words(&subject_str)
};
headers
.protected
.push(Header::new("Subject".into(), encoded_subject));
let date = chrono::Utc
.from_local_datetime(&chrono::NaiveDateTime::from_timestamp(self.timestamp, 0))
.unwrap()
.to_rfc2822();
headers.unprotected.push(Header::new("Date".into(), date));
let rfc724_mid = match self.loaded {
Loaded::Message { .. } => self.msg.rfc724_mid.clone(),
Loaded::Mdn { .. } => dc_create_outgoing_rfc724_mid(None, &self.from_addr),
};
let rfc724_mid_headervalue = render_rfc724_mid(&rfc724_mid);
// Amazon's SMTP servers change the `Message-ID`, just as Outlook's SMTP servers do.
// Outlook's servers add an `X-Microsoft-Original-Message-ID` header with the original `Message-ID`,
// and when downloading messages we look for this header in order to correctly identify
// messages.
// Amazon's servers do not add such a header, so we just add it ourselves.
if let Some(server) = context.get_config(Config::ConfiguredSendServer).await? {
if server.ends_with(".amazonaws.com") {
headers.unprotected.push(Header::new(
"X-Microsoft-Original-Message-ID".into(),
rfc724_mid_headervalue.clone(),
))
}
if to.is_empty() {
to.push(from.clone());
}
headers
.unprotected
.push(Header::new("Message-ID".into(), rfc724_mid_headervalue));
.push(Header::new("MIME-Version".into(), "1.0".into()));
// Reply headers as in <https://datatracker.ietf.org/doc/html/rfc5322#appendix-A.2>.
if !self.in_reply_to.is_empty() {
headers
.unprotected
.push(Header::new("In-Reply-To".into(), self.in_reply_to.clone()));
}
if !self.references.is_empty() {
headers
.unprotected
.push(Header::new("References".into(), self.references.clone()));
}
// Automatic Response headers <https://www.rfc-editor.org/rfc/rfc3834>
if !self.in_reply_to.is_empty() {
headers
.unprotected
.push(Header::new("In-Reply-To".into(), self.in_reply_to.clone()));
}
let date = chrono::Utc
.from_local_datetime(&chrono::NaiveDateTime::from_timestamp(self.timestamp, 0))
.unwrap()
.to_rfc2822();
headers.unprotected.push(Header::new("Date".into(), date));
headers
.unprotected
.push(Header::new("Chat-Version".to_string(), "1.0".to_string()));
if let Loaded::Mdn { .. } = self.loaded {
headers.unprotected.push(Header::new(
"Auto-Submitted".to_string(),
@@ -589,11 +513,6 @@ impl<'a> MimeFactory<'a> {
));
}
// Non-standard headers.
headers
.unprotected
.push(Header::new("Chat-Version".to_string(), "1.0".to_string()));
if self.req_mdn {
// we use "Chat-Disposition-Notification-To"
// because replies to "Disposition-Notification-To" are weird in many cases
@@ -608,9 +527,21 @@ impl<'a> MimeFactory<'a> {
let grpimage = self.grpimage();
let force_plaintext = self.should_force_plaintext();
let skip_autocrypt = self.should_skip_autocrypt();
let subject_str = self.subject_str(context).await?;
let e2ee_guaranteed = self.is_e2ee_guaranteed();
let encrypt_helper = EncryptHelper::new(context).await?;
let encoded_subject = if subject_str
.chars()
.all(|c| c.is_ascii_alphanumeric() || c == ' ')
// We do not use needs_encoding() here because needs_encoding() returns true if the string contains a space
// but we do not want to encode all subjects just because they contain a space.
{
subject_str.clone()
} else {
encode_words(&subject_str)
};
if !skip_autocrypt {
// unless determined otherwise we add the Autocrypt header
let aheader = encrypt_helper.get_aheader().to_string();
@@ -619,6 +550,15 @@ impl<'a> MimeFactory<'a> {
.push(Header::new("Autocrypt".into(), aheader));
}
headers
.protected
.push(Header::new("Subject".into(), encoded_subject));
let rfc724_mid = match self.loaded {
Loaded::Message { .. } => self.msg.rfc724_mid.clone(),
Loaded::Mdn { .. } => dc_create_outgoing_rfc724_mid(None, &self.from_addr),
};
let ephemeral_timer = self.msg.chat_id.get_ephemeral_timer(context).await?;
if let EphemeralTimer::Enabled { duration } = ephemeral_timer {
headers.protected.push(Header::new(
@@ -627,11 +567,36 @@ impl<'a> MimeFactory<'a> {
));
}
// MIME header <https://datatracker.ietf.org/doc/html/rfc2045>.
// Content-Type
headers.unprotected.push(Header::new(
"Message-ID".into(),
render_rfc724_mid(&rfc724_mid),
));
let undisclosed_recipients = match &self.loaded {
Loaded::Message { chat } => chat.typ == Chattype::Broadcast,
Loaded::Mdn { .. } => false,
};
if undisclosed_recipients {
headers
.unprotected
.push(Header::new("To".into(), "hidden-recipients: ;".to_string()));
} else {
headers
.unprotected
.push(Header::new_with_value("To".into(), to).unwrap());
}
headers
.unprotected
.push(Header::new("MIME-Version".into(), "1.0".into()));
.push(Header::new_with_value("From".into(), vec![from]).unwrap());
if let Some(sender_displayname) = &self.sender_displayname {
let sender =
Address::new_mailbox_with_name(sender_displayname.clone(), self.from_addr.clone());
headers
.unprotected
.push(Header::new_with_value("Sender".into(), vec![sender]).unwrap());
}
let mut is_gossiped = false;
@@ -658,11 +623,6 @@ impl<'a> MimeFactory<'a> {
"Content-Type".to_string(),
"multipart/report; report-type=multi-device-sync".to_string(),
))
} else if self.msg.param.get_cmd() == SystemMessage::WebxdcStatusUpdate {
PartBuilder::new().header((
"Content-Type".to_string(),
"multipart/report; report-type=status-update".to_string(),
))
} else {
PartBuilder::new().message_type(MimeMultipartType::Mixed)
};
@@ -787,7 +747,7 @@ impl<'a> MimeFactory<'a> {
} = self;
Ok(RenderedEmail {
message: outer_message.build().as_string(),
message: outer_message.build().as_string().into_bytes(),
// envelope: Envelope::new,
is_encrypted,
is_gossiped,
@@ -937,9 +897,7 @@ impl<'a> MimeFactory<'a> {
"ephemeral-timer-changed".to_string(),
));
}
SystemMessage::LocationOnly
| SystemMessage::MultiDeviceSync
| SystemMessage::WebxdcStatusUpdate => {
SystemMessage::LocationOnly | SystemMessage::MultiDeviceSync => {
// This should prevent automatic replies,
// such as non-delivery reports.
//
@@ -1140,7 +1098,7 @@ impl<'a> MimeFactory<'a> {
}
// add attachment part
if self.msg.viewtype.has_file() {
if chat::msgtype_has_file(self.msg.viewtype) {
if !is_file_size_okay(context, self.msg).await? {
bail!(
"Message exceeds the recommended {} MB.",
@@ -1176,16 +1134,6 @@ impl<'a> MimeFactory<'a> {
let ids = self.msg.param.get(Param::Arg2).unwrap_or_default();
parts.push(context.build_sync_part(json.to_string()).await);
self.sync_ids_to_delete = Some(ids.to_string());
} else if command == SystemMessage::WebxdcStatusUpdate {
let json = self.msg.param.get(Param::Arg).unwrap_or_default();
parts.push(context.build_status_update_part(json).await);
} else if self.msg.viewtype == Viewtype::Webxdc {
if let Some(json) = context
.render_webxdc_status_update_object(self.msg.id, None)
.await?
{
parts.push(context.build_status_update_part(&json).await);
}
}
if self.attach_selfavatar {
@@ -1317,7 +1265,7 @@ async fn build_body_file(
.param
.get_blob(Param::File, context, true)
.await?
.context("msg has no filename")?;
.ok_or_else(|| format_err!("msg has no filename"))?;
let suffix = blob.suffix().unwrap_or("dat");
// Get file name to use for sending. For privacy purposes, we do
@@ -1345,7 +1293,8 @@ async fn build_body_file(
"video_{}.{}",
chrono::Utc
.timestamp(msg.timestamp_sort, 0)
.format("%Y-%m-%d_%H-%M-%S"),
.format("%Y-%m-%d_%H-%M-%S")
.to_string(),
&suffix
),
_ => blob.as_file_name().to_string(),
@@ -1452,22 +1401,18 @@ fn maybe_encode_words(words: &str) -> String {
#[cfg(test)]
mod tests {
use async_std::fs::File;
use super::*;
use async_std::prelude::*;
use mailparse::{addrparse_header, MailHeaderMap};
use crate::chat::ChatId;
use crate::chat::{
self, add_contact_to_chat, create_group_chat, remove_contact_from_chat, send_text_msg,
ProtectionStatus,
};
use crate::chatlist::Chatlist;
use crate::contact::Origin;
use crate::dc_receive_imf::dc_receive_imf;
use crate::mimeparser::MimeMessage;
use crate::test_utils::{get_chat_msg, TestContext};
use super::*;
use async_std::fs::File;
#[test]
fn test_render_email_address() {
let display_name = "ä space";
@@ -1564,7 +1509,7 @@ mod tests {
msg_to_subject_str(
b"Received: (Postfix, from userid 1000); Mon, 4 Dec 2006 14:51:39 +0100 (CET)\n\
From: Bob <bob@example.com>\n\
To: alice@example.org\n\
To: alice@example.com\n\
Subject: Antw: Chat: hello\n\
Message-ID: <2222@example.com>\n\
Date: Sun, 22 Mar 2020 22:37:56 +0000\n\
@@ -1579,7 +1524,7 @@ mod tests {
msg_to_subject_str(
b"Received: (Postfix, from userid 1000); Mon, 4 Dec 2006 14:51:39 +0100 (CET)\n\
From: Bob <bob@example.com>\n\
To: alice@example.org\n\
To: alice@example.com\n\
Subject: Infos: 42\n\
Message-ID: <2222@example.com>\n\
Date: Sun, 22 Mar 2020 22:37:56 +0000\n\
@@ -1598,7 +1543,7 @@ mod tests {
msg_to_subject_str(
b"Received: (Postfix, from userid 1000); Mon, 4 Dec 2006 14:51:39 +0100 (CET)\n\
From: bob@example.com\n\
To: alice@example.org\n\
To: alice@example.com\n\
Subject: Chat: hello\n\
Chat-Version: 1.0\n\
Message-ID: <2223@example.com>\n\
@@ -1616,7 +1561,7 @@ mod tests {
// 3. Send the first message to a new contact
let t = TestContext::new_alice().await;
assert_eq!(first_subject_str(t).await, "Message from alice@example.org");
assert_eq!(first_subject_str(t).await, "Message from alice@example.com");
let t = TestContext::new_alice().await;
t.set_config(Config::Displayname, Some("Alice"))
@@ -1631,7 +1576,7 @@ mod tests {
msg_to_subject_str(
"Received: (Postfix, from userid 1000); Mon, 4 Dec 2006 14:51:39 +0100 (CET)\n\
From: bob@example.com\n\
To: alice@example.org\n\
To: alice@example.com\n\
Subject: äääää\n\
Chat-Version: 1.0\n\
Message-ID: <2893@example.com>\n\
@@ -1645,7 +1590,7 @@ mod tests {
msg_to_subject_str(
"Received: (Postfix, from userid 1000); Mon, 4 Dec 2006 14:51:39 +0100 (CET)\n\
From: bob@example.com\n\
To: alice@example.org\n\
To: alice@example.com\n\
Subject: aäääää\n\
Chat-Version: 1.0\n\
Message-ID: <2893@example.com>\n\
@@ -1664,7 +1609,7 @@ mod tests {
dc_receive_imf(
&t,
b"Received: (Postfix, from userid 1000); Mon, 4 Dec 2006 14:51:39 +0100 (CET)\n\
From: alice@example.org\n\
From: alice@example.com\n\
To: bob@example.com\n\
Subject: Hello, Bob\n\
Chat-Version: 1.0\n\
@@ -1673,6 +1618,7 @@ mod tests {
\n\
hello\n",
"INBOX",
1,
false,
)
.await
@@ -1680,7 +1626,7 @@ mod tests {
let new_msg = incoming_msg_to_reply_msg(
b"Received: (Postfix, from userid 1000); Mon, 4 Dec 2006 14:51:39 +0100 (CET)\n\
From: bob@example.com\n\
To: alice@example.org\n\
To: alice@example.com\n\
Subject: message opened\n\
Date: Sun, 22 Mar 2020 23:37:57 +0000\n\
Chat-Version: 1.0\n\
@@ -1718,7 +1664,7 @@ mod tests {
let mut new_msg = Message::new(Viewtype::Text);
new_msg.set_text(Some("Hi".to_string()));
if let Some(q) = quote {
new_msg.set_quote(t, Some(q)).await?;
new_msg.set_quote(t, q).await?;
}
let sent = t.send_msg(group_id, &mut new_msg).await;
get_subject(t, sent).await
@@ -1755,7 +1701,7 @@ mod tests {
format!(
"Received: (Postfix, from userid 1000); Mon, 4 Dec 2006 14:51:39 +0100 (CET)\n\
From: bob@example.com\n\
To: alice@example.org\n\
To: alice@example.com\n\
Subject: Different subject\n\
In-Reply-To: {}\n\
Message-ID: <2893@example.com>\n\
@@ -1766,6 +1712,7 @@ mod tests {
)
.as_bytes(),
"INBOX",
5,
false,
)
.await?;
@@ -1809,7 +1756,7 @@ mod tests {
mf.subject_str(&t).await.unwrap()
}
// In `imf_raw`, From has to be bob@example.com, To has to be alice@example.org
// In `imf_raw`, From has to be bob@example.com, To has to be alice@example.com
async fn msg_to_subject_str(imf_raw: &[u8]) -> String {
let subject_str = msg_to_subject_str_inner(imf_raw, false, false, false).await;
@@ -1870,13 +1817,14 @@ mod tests {
&t,
b"Received: (Postfix, from userid 1000); Mon, 4 Dec 2006 14:51:39 +0100 (CET)\n\
From: Bob <bob@example.com>\n\
To: alice@example.org\n\
To: alice@example.com\n\
Subject: Some other, completely unrelated subject\n\
Message-ID: <3cl4@example.com>\n\
Date: Sun, 22 Mar 2020 22:37:56 +0000\n\
\n\
Some other, completely unrelated content\n",
"INBOX",
2,
false,
)
.await
@@ -1887,7 +1835,7 @@ mod tests {
}
if reply {
new_msg.set_quote(&t, Some(&incoming_msg)).await.unwrap();
new_msg.set_quote(&t, &incoming_msg).await.unwrap();
}
let mf = MimeFactory::from_msg(&t, &new_msg, false).await.unwrap();
@@ -1901,13 +1849,13 @@ mod tests {
.await
.unwrap();
dc_receive_imf(context, imf_raw, "INBOX", false)
dc_receive_imf(context, imf_raw, "INBOX", 1, false)
.await
.unwrap();
let chats = Chatlist::try_load(context, 0, None, None).await.unwrap();
let chat_id = chats.get_chat_id(0).unwrap();
let chat_id = chats.get_chat_id(0);
chat_id.accept(context).await.unwrap();
let mut new_msg = Message::new(Viewtype::Text);
@@ -1929,7 +1877,7 @@ mod tests {
let msg = incoming_msg_to_reply_msg(
b"Received: (Postfix, from userid 1000); Mon, 4 Dec 2006 14:51:39 +0100 (CET)\n\
From: Charlie <charlie@example.com>\n\
To: alice@example.org\n\
To: alice@example.com\n\
Subject: Chat: hello\n\
Chat-Version: 1.0\n\
Message-ID: <2223@example.com>\n\
@@ -1947,7 +1895,7 @@ mod tests {
let rendered_msg = mimefactory.render(context).await.unwrap();
let mail = mailparse::parse_mail(rendered_msg.message.as_bytes()).unwrap();
let mail = mailparse::parse_mail(&rendered_msg.message).unwrap();
assert_eq!(
mail.headers
.iter()
@@ -1957,7 +1905,7 @@ mod tests {
"1.0"
);
let _mime_msg = MimeMessage::from_bytes(context, rendered_msg.message.as_bytes())
let _mime_msg = MimeMessage::from_bytes(context, &rendered_msg.message)
.await
.unwrap();
}
@@ -2031,9 +1979,8 @@ mod tests {
let mut msg = Message::new(Viewtype::Text);
msg.set_text(Some("this is the text!".to_string()));
let sent_msg = t.send_msg(chat.id, &mut msg).await;
let mut payload = sent_msg.payload().splitn(3, "\r\n\r\n");
let payload = t.send_msg(chat.id, &mut msg).await.payload();
let mut payload = payload.splitn(3, "\r\n\r\n");
let outer = payload.next().unwrap();
let inner = payload.next().unwrap();
let body = payload.next().unwrap();
@@ -2051,8 +1998,8 @@ mod tests {
// if another message is sent, that one must not contain the avatar
// and no artificial multipart/mixed nesting
let sent_msg = t.send_msg(chat.id, &mut msg).await;
let mut payload = sent_msg.payload().splitn(2, "\r\n\r\n");
let payload = t.send_msg(chat.id, &mut msg).await.payload();
let mut payload = payload.splitn(2, "\r\n\r\n");
let outer = payload.next().unwrap();
let body = payload.next().unwrap();
@@ -2069,58 +2016,4 @@ mod tests {
Ok(())
}
/// Test that removed member address does not go into the `To:` field.
#[async_std::test]
async fn test_remove_member_bcc() -> Result<()> {
// Alice creates a group with Bob and Claire and then removes Bob.
let alice = TestContext::new_alice().await;
let bob_id = Contact::create(&alice, "Bob", "bob@example.net").await?;
let claire_id = Contact::create(&alice, "Claire", "claire@foo.de").await?;
let alice_chat_id = create_group_chat(&alice, ProtectionStatus::Unprotected, "foo").await?;
add_contact_to_chat(&alice, alice_chat_id, bob_id).await?;
add_contact_to_chat(&alice, alice_chat_id, claire_id).await?;
send_text_msg(&alice, alice_chat_id, "Creating a group".to_string()).await?;
remove_contact_from_chat(&alice, alice_chat_id, claire_id).await?;
let remove = alice.pop_sent_msg().await;
let remove_payload = remove.payload();
let parsed = mailparse::parse_mail(remove_payload.as_bytes())?;
let to = parsed
.headers
.get_first_header("To")
.context("no To: header parsed")?;
let to = addrparse_header(to)?;
let mailbox = to
.extract_single_info()
.context("to: field does not contain exactly one address")?;
assert_eq!(mailbox.addr, "bob@example.net");
Ok(())
}
/// Tests that standard IMF header "From:" comes before non-standard "Autocrypt:" header.
#[async_std::test]
async fn test_from_before_autocrypt() -> Result<()> {
// create chat with bob
let t = TestContext::new_alice().await;
let chat = t.create_chat_with_contact("bob", "bob@example.org").await;
// send message to bob: that should get multipart/mixed because of the avatar moved to inner header;
// make sure, `Subject:` stays in the outer header (imf header)
let mut msg = Message::new(Viewtype::Text);
msg.set_text(Some("this is the text!".to_string()));
let sent_msg = t.send_msg(chat.id, &mut msg).await;
let payload = sent_msg.payload();
assert_eq!(payload.match_indices("Autocrypt:").count(), 1);
assert_eq!(payload.match_indices("From:").count(), 1);
assert!(payload.match_indices("From:").next() < payload.match_indices("Autocrypt:").next());
Ok(())
}
}

View File

@@ -2,7 +2,6 @@
use std::collections::{HashMap, HashSet};
use std::future::Future;
use std::io::Cursor;
use std::pin::Pin;
use anyhow::{bail, Result};
@@ -13,10 +12,10 @@ use once_cell::sync::Lazy;
use crate::aheader::Aheader;
use crate::blob::BlobObject;
use crate::constants::{DC_DESIRED_TEXT_LEN, DC_ELLIPSIS};
use crate::contact::{addr_normalize, ContactId};
use crate::constants::{Viewtype, DC_DESIRED_TEXT_LEN, DC_ELLIPSIS};
use crate::contact::addr_normalize;
use crate::context::Context;
use crate::dc_tools::{dc_get_filemeta, dc_truncate, parse_receive_headers};
use crate::dc_tools::{dc_get_filemeta, dc_truncate};
use crate::dehtml::dehtml;
use crate::e2ee;
use crate::events::EventType;
@@ -24,7 +23,7 @@ use crate::format_flowed::unformat_flowed;
use crate::headerdef::{HeaderDef, HeaderDefMap};
use crate::key::Fingerprint;
use crate::location;
use crate::message::{self, Viewtype};
use crate::message;
use crate::param::{Param, Params};
use crate::peerstate::Peerstate;
use crate::simplify::simplify;
@@ -48,7 +47,6 @@ pub struct MimeMessage {
/// Addresses are normalized and lowercased:
pub recipients: Vec<SingleInfo>,
pub from: Vec<SingleInfo>,
pub list_post: Option<String>,
pub chat_disposition_notification_to: Option<SingleInfo>,
pub decrypting_failed: bool,
@@ -67,7 +65,6 @@ pub struct MimeMessage {
pub location_kml: Option<location::Kml>,
pub message_kml: Option<location::Kml>,
pub(crate) sync_items: Option<SyncItems>,
pub(crate) webxdc_status_update: Option<String>,
pub(crate) user_avatar: Option<AvatarAction>,
pub(crate) group_avatar: Option<AvatarAction>,
pub(crate) mdn_reports: Vec<Report>,
@@ -85,8 +82,6 @@ pub struct MimeMessage {
/// This is non-empty only if the message was actually encrypted. It is used
/// for e.g. late-parsing HTML.
pub decoded_data: Vec<u8>,
pub(crate) hop_info: String,
}
#[derive(Debug, PartialEq)]
@@ -137,10 +132,6 @@ pub enum SystemMessage {
/// Self-sent-message that contains only json used for multi-device-sync;
/// if possible, we attach that to other messages as for locations.
MultiDeviceSync = 20,
// Sync message that contains a json payload
// sent to the other webxdc instances
WebxdcStatusUpdate = 30,
}
impl Default for SystemMessage {
@@ -172,12 +163,10 @@ impl MimeMessage {
.get_header_value(HeaderDef::Date)
.and_then(|v| mailparse::dateparse(&v).ok())
.unwrap_or_default();
let hop_info = parse_receive_headers(&mail.get_headers());
let mut headers = Default::default();
let mut recipients = Default::default();
let mut from = Default::default();
let mut list_post = Default::default();
let mut chat_disposition_notification_to = None;
// Parse IMF headers.
@@ -186,7 +175,6 @@ impl MimeMessage {
&mut headers,
&mut recipients,
&mut from,
&mut list_post,
&mut chat_disposition_notification_to,
&mail.headers,
);
@@ -260,7 +248,6 @@ impl MimeMessage {
&mut headers,
&mut recipients,
&mut throwaway_from,
&mut list_post,
&mut chat_disposition_notification_to,
&decrypted_mail.headers,
);
@@ -288,7 +275,6 @@ impl MimeMessage {
parts: Vec::new(),
header: headers,
recipients,
list_post,
from,
chat_disposition_notification_to,
decrypting_failed: false,
@@ -302,14 +288,12 @@ impl MimeMessage {
location_kml: None,
message_kml: None,
sync_items: None,
webxdc_status_update: None,
user_avatar: None,
group_avatar: None,
failure_report: None,
footer: None,
is_mime_modified: false,
decoded_data: Vec::new(),
hop_info,
};
match partial {
@@ -369,16 +353,6 @@ impl MimeMessage {
} else if value == "protection-disabled" {
self.is_system_message = SystemMessage::ChatProtectionDisabled;
}
} else if self.get_header(HeaderDef::ChatGroupMemberRemoved).is_some() {
self.is_system_message = SystemMessage::MemberRemovedFromGroup;
} else if self.get_header(HeaderDef::ChatGroupMemberAdded).is_some() {
self.is_system_message = SystemMessage::MemberAddedToGroup;
} else if self.get_header(HeaderDef::ChatGroupNameChanged).is_some() {
self.is_system_message = SystemMessage::GroupNameChanged;
} else if let Some(value) = self.get_header(HeaderDef::ChatContent) {
if value == "group-avatar-changed" {
self.is_system_message = SystemMessage::GroupImageChanged;
}
}
}
@@ -414,18 +388,16 @@ impl MimeMessage {
#[allow(clippy::indexing_slicing)]
fn squash_attachment_parts(&mut self) {
if let [textpart, filepart] = &self.parts[..] {
let need_drop = textpart.typ == Viewtype::Text
&& match filepart.typ {
Viewtype::Image
| Viewtype::Gif
| Viewtype::Sticker
| Viewtype::Audio
| Viewtype::Voice
| Viewtype::Video
| Viewtype::File
| Viewtype::Webxdc => true,
Viewtype::Unknown | Viewtype::Text | Viewtype::VideochatInvitation => false,
};
let need_drop = {
textpart.typ == Viewtype::Text
&& (filepart.typ == Viewtype::Image
|| filepart.typ == Viewtype::Gif
|| filepart.typ == Viewtype::Sticker
|| filepart.typ == Viewtype::Audio
|| filepart.typ == Viewtype::Voice
|| filepart.typ == Viewtype::Video
|| filepart.typ == Viewtype::File)
};
if need_drop {
let mut filepart = self.parts.swap_remove(1);
@@ -557,7 +529,7 @@ impl MimeMessage {
};
if let Some(ref subject) = self.get_subject() {
if !self.has_chat_version() && self.webxdc_status_update.is_none() {
if !self.has_chat_version() {
part.msg = subject.to_string();
}
}
@@ -856,12 +828,6 @@ impl MimeMessage {
.await?;
}
}
Some("status-update") => {
if let Some(second) = mail.subparts.get(1) {
self.add_single_part_if_known(context, second, is_related)
.await?;
}
}
Some(_) => {
if let Some(first) = mail.subparts.get(0) {
any_part_added = self
@@ -1031,14 +997,8 @@ impl MimeMessage {
if decoded_data.is_empty() {
return;
}
let reader = Cursor::new(decoded_data);
let msg_type = if context
.is_webxdc_file(filename, reader)
.await
.unwrap_or(false)
{
Viewtype::Webxdc
} else if filename.ends_with(".kml") {
// treat location/message kml file attachments specially
if filename.ends_with(".kml") {
// XXX what if somebody sends eg an "location-highlights.kml"
// attachment unrelated to location streaming?
if filename.starts_with("location") || filename.starts_with("message") {
@@ -1054,7 +1014,6 @@ impl MimeMessage {
}
return;
}
msg_type
} else if filename == "multi-device-sync.json" {
let serialized = String::from_utf8_lossy(decoded_data)
.parse()
@@ -1067,15 +1026,7 @@ impl MimeMessage {
})
.ok();
return;
} else if filename == "status-update.json" {
let serialized = String::from_utf8_lossy(decoded_data)
.parse()
.unwrap_or_default();
self.webxdc_status_update = Some(serialized);
return;
} else {
msg_type
};
}
/* we have a regular file attachment,
write decoded data to new blob object */
@@ -1141,16 +1092,16 @@ impl MimeMessage {
}
}
pub fn repl_msg_by_error(&mut self, error_msg: &str) {
pub fn repl_msg_by_error(&mut self, error_msg: impl AsRef<str>) {
self.is_system_message = SystemMessage::Unknown;
if let Some(part) = self.parts.first_mut() {
part.typ = Viewtype::Text;
part.msg = format!("[{}]", error_msg);
part.msg = format!("[{}]", error_msg.as_ref());
self.parts.truncate(1);
}
}
pub(crate) fn get_rfc724_mid(&self) -> Option<String> {
pub fn get_rfc724_mid(&self) -> Option<String> {
self.get_header(HeaderDef::XMicrosoftOriginalMessageId)
.or_else(|| self.get_header(HeaderDef::MessageId))
.and_then(|msgid| parse_message_id(msgid).ok())
@@ -1161,7 +1112,6 @@ impl MimeMessage {
headers: &mut HashMap<String, String>,
recipients: &mut Vec<SingleInfo>,
from: &mut Vec<SingleInfo>,
list_post: &mut Option<String>,
chat_disposition_notification_to: &mut Option<SingleInfo>,
fields: &[mailparse::MailHeader<'_>],
) {
@@ -1192,10 +1142,6 @@ impl MimeMessage {
if !from_new.is_empty() {
*from = from_new;
}
let list_post_new = get_list_post(fields);
if list_post_new.is_some() {
*list_post = list_post_new;
}
}
fn process_report(
@@ -1213,24 +1159,23 @@ impl MimeMessage {
// must be present
if let Some(_disposition) = report_fields.get_header_value(HeaderDef::Disposition) {
let original_message_id = report_fields
if let Some(original_message_id) = report_fields
.get_header_value(HeaderDef::OriginalMessageId)
// MS Exchange doesn't add an Original-Message-Id header. Instead, they put
// the original message id into the In-Reply-To header:
.or_else(|| report.headers.get_header_value(HeaderDef::InReplyTo))
.and_then(|v| parse_message_id(&v).ok());
let additional_message_ids = report_fields
.get_header_value(HeaderDef::AdditionalMessageIds)
.map_or_else(Vec::new, |v| {
v.split(' ')
.filter_map(|s| parse_message_id(s).ok())
.collect()
});
.and_then(|v| parse_message_id(&v).ok())
{
let additional_message_ids = report_fields
.get_header_value(HeaderDef::AdditionalMessageIds)
.map_or_else(Vec::new, |v| {
v.split(' ')
.filter_map(|s| parse_message_id(s).ok())
.collect()
});
return Ok(Some(Report {
original_message_id,
additional_message_ids,
}));
return Ok(Some(Report {
original_message_id,
additional_message_ids,
}));
}
}
warn!(
context,
@@ -1381,15 +1326,13 @@ impl MimeMessage {
pub async fn handle_reports(
&self,
context: &Context,
from_id: ContactId,
from_id: u32,
sent_timestamp: i64,
parts: &[Part],
) {
for report in &self.mdn_reports {
for original_message_id in report
.original_message_id
.iter()
.chain(&report.additional_message_ids)
for original_message_id in
std::iter::once(&report.original_message_id).chain(&report.additional_message_ids)
{
match message::handle_mdn(context, from_id, original_message_id, sent_timestamp)
.await
@@ -1494,10 +1437,7 @@ async fn update_gossip_peerstates(
#[derive(Debug)]
pub(crate) struct Report {
/// Original-Message-ID header
///
/// It MUST be present if the original message has a Message-ID according to RFC 8098.
/// In case we can't find it (shouldn't happen), this is None.
original_message_id: Option<String>,
original_message_id: String,
/// Additional-Message-IDs
additional_message_ids: Vec<String>,
}
@@ -1687,14 +1627,6 @@ pub(crate) fn get_from(headers: &[MailHeader]) -> Vec<SingleInfo> {
get_all_addresses_from_header(headers, |header_key| header_key == "from")
}
/// Returned addresses are normalized and lowercased.
pub(crate) fn get_list_post(headers: &[MailHeader]) -> Option<String> {
get_all_addresses_from_header(headers, |header_key| header_key == "list-post")
.into_iter()
.next()
.map(|s| s.addr)
}
fn get_all_addresses_from_header<F>(headers: &[MailHeader], pred: F) -> Vec<SingleInfo>
where
F: Fn(String) -> bool,
@@ -2414,7 +2346,7 @@ Additional-Message-IDs: <foo@example.com> <foo@example.net>\n\
assert_eq!(message.mdn_reports.len(), 1);
assert_eq!(
message.mdn_reports[0].original_message_id,
Some("foo@example.org".to_string())
"foo@example.org"
);
assert_eq!(
&message.mdn_reports[0].additional_message_ids,
@@ -2916,6 +2848,7 @@ On 2020-10-25, Bob wrote:
&t.ctx,
include_bytes!("../test-data/message/subj_with_multimedia_msg.eml"),
"INBOX",
1,
false,
)
.await
@@ -3064,7 +2997,7 @@ Subject: ...
Some quote.
"###;
dc_receive_imf(&t, raw, "INBOX", false).await?;
dc_receive_imf(&t, raw, "INBOX", 1, false).await?;
// Delta Chat generates In-Reply-To with a starting tab when Message-ID is too long.
let raw = br###"In-Reply-To:
@@ -3081,7 +3014,7 @@ Subject: ...
Some reply
"###;
dc_receive_imf(&t, raw, "INBOX", false).await?;
dc_receive_imf(&t, raw, "INBOX", 2, false).await?;
let msg = t.get_last_msg().await;
assert_eq!(msg.get_text().unwrap(), "Some reply");
@@ -3101,21 +3034,21 @@ Some reply
Chat-Version: 1.0\n\
Message-ID: <foobarbaz@example.org>
To: Bob <bob@example.org>
From: Alice <alice@example.org>
From: Alice <alice@example.com>
Subject: subject
Chat-Disposition-Notification-To: alice@example.org
Chat-Disposition-Notification-To: alice@example.com
Message.
"###;
// Bob receives message.
dc_receive_imf(&bob, raw, "INBOX", false).await?;
dc_receive_imf(&bob, raw, "INBOX", 1, false).await?;
let msg = bob.get_last_msg().await;
// Message is incoming.
assert!(msg.param.get_bool(Param::WantsMdn).unwrap());
// Alice receives copy-to-self.
dc_receive_imf(&alice, raw, "INBOX", false).await?;
dc_receive_imf(&alice, raw, "INBOX", 1, false).await?;
let msg = alice.get_last_msg().await;
// Message is outgoing, don't send read receipt to self.
assert!(msg.param.get_bool(Param::WantsMdn).is_none());
@@ -3131,17 +3064,18 @@ Message.
dc_receive_imf(
&alice,
"Received: (Postfix, from userid 1000); Mon, 4 Dec 2006 14:51:39 +0100 (CET)\n\
From: alice@example.org\n\
From: alice@example.com\n\
To: bob@example.net\n\
Subject: foo\n\
Message-ID: first@example.com\n\
Chat-Version: 1.0\n\
Chat-Disposition-Notification-To: alice@example.org\n\
Chat-Disposition-Notification-To: alice@example.com\n\
Date: Sun, 22 Mar 2020 22:37:57 +0000\n\
\n\
hello\n"
.as_bytes(),
"INBOX",
1,
false,
)
.await?;
@@ -3153,8 +3087,8 @@ Message.
dc_receive_imf(
&alice,
"Received: (Postfix, from userid 1000); Mon, 4 Dec 2006 14:51:39 +0100 (CET)\n\
From: alice@example.org\n\
To: alice@example.org\n\
From: alice@example.com\n\
To: alice@example.com\n\
Subject: message opened\n\
Date: Sun, 22 Mar 2020 23:37:57 +0000\n\
Chat-Version: 1.0\n\
@@ -3180,6 +3114,7 @@ Message.
--SNIPP--"
.as_bytes(),
"INBOX",
2,
false,
)
.await?;
@@ -3190,40 +3125,4 @@ Message.
Ok(())
}
/// Test parsing of MDN sent by MS Exchange.
///
/// It does not have required Original-Message-ID field, so it is useless, but we want to
/// recognize it as MDN nevertheless to avoid displaying it in the chat as normal message.
#[async_std::test]
async fn test_ms_exchange_mdn() -> Result<()> {
let t = TestContext::new_alice().await;
t.set_config(Config::ShowEmails, Some("2")).await?;
let original =
include_bytes!("../test-data/message/ms_exchange_report_original_message.eml");
dc_receive_imf(&t, original, "INBOX", false).await?;
let original_msg_id = t.get_last_msg().await.id;
// 1. Test mimeparser directly
let mdn =
include_bytes!("../test-data/message/ms_exchange_report_disposition_notification.eml");
let mimeparser = MimeMessage::from_bytes(&t.ctx, mdn).await?;
assert_eq!(mimeparser.mdn_reports.len(), 1);
assert_eq!(
mimeparser.mdn_reports[0].original_message_id.as_deref(),
Some("d5904dc344eeb5deaf9bb44603f0c716@posteo.de")
);
assert!(mimeparser.mdn_reports[0].additional_message_ids.is_empty());
// 2. Test that marking the original msg as read works
dc_receive_imf(&t, mdn, "INBOX", false).await?;
assert_eq!(
original_msg_id.get_state(&t).await?,
MessageState::OutMdnRcvd
);
Ok(())
}
}

View File

@@ -41,7 +41,6 @@ struct Oauth2 {
/// OAuth 2 Access Token Response
#[derive(Debug, Deserialize)]
#[allow(dead_code)]
struct Response {
// Should always be there according to: <https://www.oauth.com/oauth2-servers/access-tokens/access-token-response/>
// but previous code handled its abscense.
@@ -59,7 +58,7 @@ pub async fn dc_get_oauth2_url(
redirect_uri: &str,
) -> Result<Option<String>> {
let socks5_enabled = context.get_config_bool(Config::Socks5Enabled).await?;
if let Some(oauth2) = Oauth2::from_address(context, addr, socks5_enabled).await {
if let Some(oauth2) = Oauth2::from_address(addr, socks5_enabled).await {
context
.sql
.set_raw_config("oauth2_pending_redirect_uri", Some(redirect_uri))
@@ -80,7 +79,7 @@ pub async fn dc_get_oauth2_access_token(
regenerate: bool,
) -> Result<Option<String>> {
let socks5_enabled = context.get_config_bool(Config::Socks5Enabled).await?;
if let Some(oauth2) = Oauth2::from_address(context, addr, socks5_enabled).await {
if let Some(oauth2) = Oauth2::from_address(addr, socks5_enabled).await {
let lock = context.oauth2_mutex.lock().await;
// read generated token
@@ -226,7 +225,7 @@ pub async fn dc_get_oauth2_addr(
code: &str,
) -> Result<Option<String>> {
let socks5_enabled = context.get_config_bool(Config::Socks5Enabled).await?;
let oauth2 = match Oauth2::from_address(context, addr, socks5_enabled).await {
let oauth2 = match Oauth2::from_address(addr, socks5_enabled).await {
Some(o) => o,
None => return Ok(None),
};
@@ -254,13 +253,13 @@ pub async fn dc_get_oauth2_addr(
}
impl Oauth2 {
async fn from_address(context: &Context, addr: &str, skip_mx: bool) -> Option<Self> {
async fn from_address(addr: &str, skip_mx: bool) -> Option<Self> {
let addr_normalized = normalize_addr(addr);
if let Some(domain) = addr_normalized
.find('@')
.map(|index| addr_normalized.split_at(index + 1).1)
{
if let Some(oauth2_authorizer) = provider::get_provider_info(context, domain, skip_mx)
if let Some(oauth2_authorizer) = provider::get_provider_info(domain, skip_mx)
.await
.and_then(|provider| provider.oauth2_authorizer.as_ref())
{
@@ -357,39 +356,32 @@ mod tests {
#[async_std::test]
async fn test_oauth_from_address() {
let t = TestContext::new().await;
assert_eq!(
Oauth2::from_address(&t, "hello@gmail.com", false).await,
Oauth2::from_address("hello@gmail.com", false).await,
Some(OAUTH2_GMAIL)
);
assert_eq!(
Oauth2::from_address(&t, "hello@googlemail.com", false).await,
Oauth2::from_address("hello@googlemail.com", false).await,
Some(OAUTH2_GMAIL)
);
assert_eq!(
Oauth2::from_address(&t, "hello@yandex.com", false).await,
Oauth2::from_address("hello@yandex.com", false).await,
Some(OAUTH2_YANDEX)
);
assert_eq!(
Oauth2::from_address(&t, "hello@yandex.ru", false).await,
Oauth2::from_address("hello@yandex.ru", false).await,
Some(OAUTH2_YANDEX)
);
assert_eq!(Oauth2::from_address(&t, "hello@web.de", false).await, None);
assert_eq!(Oauth2::from_address("hello@web.de", false).await, None);
}
#[async_std::test]
async fn test_oauth_from_mx() {
// youtube staff seems to use "google workspace with oauth2", figures this out by MX lookup
let t = TestContext::new().await;
assert_eq!(
Oauth2::from_address(&t, "hello@youtube.com", false).await,
Oauth2::from_address("hello@google.com", false).await,
Some(OAUTH2_GMAIL)
);
// without MX lookup, we would not know as youtube.com is not in our provider-db
assert_eq!(
Oauth2::from_address(&t, "hello@youtube.com", true).await,
None
);
}
#[async_std::test]

View File

@@ -110,8 +110,8 @@ pub enum Param {
/// For Jobs
AlsoMove = b'M',
/// For MDN-sending job
MsgId = b'I',
/// For Jobs: space-separated list of message recipients
Recipients = b'R',
/// For Groups
///
@@ -136,17 +136,8 @@ pub enum Param {
/// For Chats
Devicetalk = b'D',
/// For Chats: If this is a mailing list chat, contains the List-Post address.
/// None if there simply is no `List-Post` header in the mailing list.
/// Some("") if the mailing list is using multiple different List-Post headers.
///
/// The List-Post address is the email address where the user can write to in order to
/// post something to the mailing list.
ListPost = b'p',
/// For Contacts: If this is the List-Post address of a mailing list, contains
/// the List-Id of the mailing list (which is also used as the group id of the chat).
ListId = b's',
/// For MDN-sending job
MsgId = b'I',
/// For Contacts: timestamp of status (aka signature or footer) update.
StatusTimestamp = b'j',
@@ -168,12 +159,6 @@ pub enum Param {
/// For Chats: timestamp of protection settings update.
ProtectionSettingsTimestamp = b'L',
/// For Webxdc Message Instances: Current summary
WebxdcSummary = b'N',
/// For Webxdc Message Instances: timestamp of summary update.
WebxdcSummaryTimestamp = b'Q',
}
/// An object for handling key=value parameter lists.
@@ -206,11 +191,6 @@ impl fmt::Display for Params {
impl str::FromStr for Params {
type Err = Error;
/// Parse a raw string to Param.
///
/// Silently ignore unknown keys:
/// they may come from a downgrade (when a shortly new version adds a key)
/// or from an upgrade (when a key is dropped but was used in the past)
fn from_str(s: &str) -> std::result::Result<Self, Self::Err> {
let mut inner = BTreeMap::new();
let mut lines = s.lines().peekable();
@@ -230,6 +210,8 @@ impl str::FromStr for Params {
if let Some(key) = key.as_bytes().first().and_then(|key| Param::from_u8(*key)) {
inner.insert(key, value);
} else {
bail!("Unknown key: {}", key);
}
} else {
bail!("Not a key-value pair: {:?}", line);
@@ -432,12 +414,10 @@ impl<'a> ParamsFile<'a> {
mod tests {
use super::*;
use anyhow::Result;
use async_std::fs;
use async_std::path::Path;
use crate::test_utils::TestContext;
use std::str::FromStr;
#[test]
fn test_dc_param() {
@@ -540,14 +520,4 @@ mod tests {
assert!(p.get_path(Param::File, &t).unwrap().is_none());
assert!(p.get_blob(Param::File, &t, false).await.unwrap().is_none());
}
#[async_std::test]
async fn test_params_unknown_key() -> Result<()> {
// 'Z' is used as a key that is known to be unused; these keys should be ignored silently by definition.
let p = Params::from_str("w=12\nZ=13\nh=14")?;
assert_eq!(p.len(), 2);
assert_eq!(p.get(Param::Width), Some("12"));
assert_eq!(p.get(Param::Height), Some("14"));
Ok(())
}
}

View File

@@ -277,7 +277,7 @@ impl Peerstate {
let msg = stock_str::contact_setup_changed(context, self.addr.clone()).await;
chat::add_info_msg(context, chat_id, &msg, timestamp).await?;
chat::add_info_msg(context, chat_id, msg, timestamp).await?;
context.emit_event(EventType::ChatModified(chat_id));
} else {
bail!("contact with peerstate.addr {:?} not found", &self.addr);
@@ -379,9 +379,10 @@ impl Peerstate {
pub fn peek_key(&self, min_verified: PeerstateVerifiedStatus) -> Option<&SignedPublicKey> {
match min_verified {
PeerstateVerifiedStatus::BidirectVerified => self.verified_key.as_ref(),
PeerstateVerifiedStatus::Unverified => {
self.public_key.as_ref().or(self.gossip_key.as_ref())
}
PeerstateVerifiedStatus::Unverified => self
.public_key
.as_ref()
.or_else(|| self.gossip_key.as_ref()),
}
}

View File

@@ -4,11 +4,11 @@ use std::collections::{BTreeMap, HashSet};
use std::io;
use std::io::Cursor;
use anyhow::{bail, ensure, format_err, Context as _, Result};
use anyhow::{bail, ensure, format_err, Result};
use pgp::armor::BlockType;
use pgp::composed::{
Deserializable, KeyType as PgpKeyType, Message, SecretKeyParamsBuilder, SignedPublicKey,
SignedPublicSubKey, SignedSecretKey, StandaloneSignature, SubkeyParamsBuilder,
SignedPublicSubKey, SignedSecretKey, SubkeyParamsBuilder,
};
use pgp::crypto::{HashAlgorithm, SymmetricKeyAlgorithm};
use pgp::types::{
@@ -248,7 +248,7 @@ pub async fn pk_encrypt(
let pkeys: Vec<SignedPublicKeyOrSubkey> = public_keys_for_encryption
.keys()
.iter()
.filter_map(select_pk_for_encryption)
.filter_map(|key| select_pk_for_encryption(key))
.collect();
let pkeys_refs: Vec<&SignedPublicKeyOrSubkey> = pkeys.iter().collect();
@@ -277,17 +277,16 @@ pub async fn pk_encrypt(
/// Receiver private keys are provided in
/// `private_keys_for_decryption`.
///
/// Returns decrypted message and fingerprints
/// If `ret_signature_fingerprints` is not `None`, stores fingerprints
/// of all keys from the `public_keys_for_validation` keyring that
/// have valid signatures there.
#[allow(clippy::implicit_hasher)]
pub async fn pk_decrypt(
ctext: Vec<u8>,
private_keys_for_decryption: Keyring<SignedSecretKey>,
public_keys_for_validation: &Keyring<SignedPublicKey>,
) -> Result<(Vec<u8>, HashSet<Fingerprint>)> {
let mut ret_signature_fingerprints: HashSet<Fingerprint> = Default::default();
public_keys_for_validation: Keyring<SignedPublicKey>,
ret_signature_fingerprints: Option<&mut HashSet<Fingerprint>>,
) -> Result<Vec<u8>> {
let msgs = async_std::task::spawn_blocking(move || {
let cursor = Cursor::new(ctext);
let (msg, _) = Message::from_armor_single(cursor)?;
@@ -309,54 +308,33 @@ pub async fn pk_decrypt(
None => bail!("The decrypted message is empty"),
};
if !public_keys_for_validation.is_empty() {
let pkeys = public_keys_for_validation.keys();
if let Some(ret_signature_fingerprints) = ret_signature_fingerprints {
if !public_keys_for_validation.is_empty() {
let fingerprints = async_std::task::spawn_blocking(move || {
let pkeys = public_keys_for_validation.keys();
let mut fingerprints: Vec<Fingerprint> = Vec::new();
if let signed_msg @ pgp::composed::Message::Signed { .. } = msg {
for pkey in pkeys {
if signed_msg.verify(&pkey.primary_key).is_ok() {
let fp = DcKey::fingerprint(pkey);
fingerprints.push(fp);
let mut fingerprints: Vec<Fingerprint> = Vec::new();
if let signed_msg @ pgp::composed::Message::Signed { .. } = msg {
for pkey in pkeys {
if signed_msg.verify(&pkey.primary_key).is_ok() {
let fp = DcKey::fingerprint(pkey);
fingerprints.push(fp);
}
}
}
}
}
fingerprints
})
.await;
ret_signature_fingerprints.extend(fingerprints);
ret_signature_fingerprints.extend(fingerprints);
}
}
Ok((content, ret_signature_fingerprints))
Ok(content)
} else {
bail!("No valid messages found");
}
}
/// Validates detached signature.
pub async fn pk_validate(
content: &[u8],
signature: &[u8],
public_keys_for_validation: &Keyring<SignedPublicKey>,
) -> Result<HashSet<Fingerprint>> {
let mut ret: HashSet<Fingerprint> = Default::default();
let standalone_signature = StandaloneSignature::from_armor_single(Cursor::new(signature))?.0;
let pkeys = public_keys_for_validation.keys();
// Remove trailing CRLF before the delimiter.
// According to RFC 3156 it is considered to be part of the MIME delimiter for the purpose of
// OpenPGP signature calculation.
let content = content
.get(..content.len().saturating_sub(2))
.context("index is out of range")?;
for pkey in pkeys {
if standalone_signature.verify(pkey, content).is_ok() {
let fp = DcKey::fingerprint(pkey);
ret.insert(fp);
}
}
Ok(ret)
}
/// Symmetric encryption.
pub async fn symm_encrypt(passphrase: &str, plain: &[u8]) -> Result<String> {
let lit_msg = Message::new_literal_bytes("", plain);
@@ -514,12 +492,15 @@ mod tests {
decrypt_keyring.add(KEYS.alice_secret.clone());
let mut sig_check_keyring: Keyring<SignedPublicKey> = Keyring::new();
sig_check_keyring.add(KEYS.alice_public.clone());
let (plain, valid_signatures) = pk_decrypt(
let mut valid_signatures: HashSet<Fingerprint> = Default::default();
let plain = pk_decrypt(
CTEXT_SIGNED.as_bytes().to_vec(),
decrypt_keyring,
&sig_check_keyring,
sig_check_keyring,
Some(&mut valid_signatures),
)
.await
.map_err(|err| println!("{:?}", err))
.unwrap();
assert_eq!(plain, CLEARTEXT);
assert_eq!(valid_signatures.len(), 1);
@@ -529,12 +510,15 @@ mod tests {
decrypt_keyring.add(KEYS.bob_secret.clone());
let mut sig_check_keyring = Keyring::new();
sig_check_keyring.add(KEYS.alice_public.clone());
let (plain, valid_signatures) = pk_decrypt(
let mut valid_signatures: HashSet<Fingerprint> = Default::default();
let plain = pk_decrypt(
CTEXT_SIGNED.as_bytes().to_vec(),
decrypt_keyring,
&sig_check_keyring,
sig_check_keyring,
Some(&mut valid_signatures),
)
.await
.map_err(|err| println!("{:?}", err))
.unwrap();
assert_eq!(plain, CLEARTEXT);
assert_eq!(valid_signatures.len(), 1);
@@ -545,10 +529,15 @@ mod tests {
let mut keyring = Keyring::new();
keyring.add(KEYS.alice_secret.clone());
let empty_keyring = Keyring::new();
let (plain, valid_signatures) =
pk_decrypt(CTEXT_SIGNED.as_bytes().to_vec(), keyring, &empty_keyring)
.await
.unwrap();
let mut valid_signatures: HashSet<Fingerprint> = Default::default();
let plain = pk_decrypt(
CTEXT_SIGNED.as_bytes().to_vec(),
keyring,
empty_keyring,
Some(&mut valid_signatures),
)
.await
.unwrap();
assert_eq!(plain, CLEARTEXT);
assert_eq!(valid_signatures.len(), 0);
}
@@ -560,10 +549,12 @@ mod tests {
decrypt_keyring.add(KEYS.bob_secret.clone());
let mut sig_check_keyring = Keyring::new();
sig_check_keyring.add(KEYS.bob_public.clone());
let (plain, valid_signatures) = pk_decrypt(
let mut valid_signatures: HashSet<Fingerprint> = Default::default();
let plain = pk_decrypt(
CTEXT_SIGNED.as_bytes().to_vec(),
decrypt_keyring,
&sig_check_keyring,
sig_check_keyring,
Some(&mut valid_signatures),
)
.await
.unwrap();
@@ -576,14 +567,34 @@ mod tests {
let mut decrypt_keyring = Keyring::new();
decrypt_keyring.add(KEYS.bob_secret.clone());
let sig_check_keyring = Keyring::new();
let (plain, valid_signatures) = pk_decrypt(
let mut valid_signatures: HashSet<Fingerprint> = Default::default();
let plain = pk_decrypt(
CTEXT_UNSIGNED.as_bytes().to_vec(),
decrypt_keyring,
&sig_check_keyring,
sig_check_keyring,
Some(&mut valid_signatures),
)
.await
.unwrap();
assert_eq!(plain, CLEARTEXT);
assert_eq!(valid_signatures.len(), 0);
}
#[async_std::test]
async fn test_decrypt_signed_no_sigret() {
// Check decrypting signed cyphertext without providing the HashSet for signatures.
let mut decrypt_keyring = Keyring::new();
decrypt_keyring.add(KEYS.bob_secret.clone());
let mut sig_check_keyring = Keyring::new();
sig_check_keyring.add(KEYS.alice_public.clone());
let plain = pk_decrypt(
CTEXT_SIGNED.as_bytes().to_vec(),
decrypt_keyring,
sig_check_keyring,
None,
)
.await
.unwrap();
assert_eq!(plain, CLEARTEXT);
}
}

View File

@@ -41,7 +41,7 @@ impl PlainText {
// as <http://example.org> cannot be handled correctly
// (they would become &lt;http://example.org&gt; where the trailing &gt; would become a valid url part).
// to avoid double encoding, we escape our html-entities by \r that must not be used in the string elsewhere.
let line = line.to_string().replace('\r', "");
let line = line.to_string().replace("\r", "");
let mut line = LINKIFY_MAIL_RE
.replace_all(&*line, "\rLTa href=\rQUOTmailto:$1\rQUOT\rGT$1\rLT/a\rGT")

View File

@@ -3,10 +3,8 @@
mod data;
use crate::config::Config;
use crate::context::Context;
use crate::provider::data::{PROVIDER_DATA, PROVIDER_IDS, PROVIDER_UPDATED};
use anyhow::Result;
use async_std_resolver::{config, resolver, resolver_from_system_conf, AsyncStdResolver};
use async_std_resolver::resolver_from_system_conf;
use chrono::{NaiveDateTime, NaiveTime};
#[derive(Debug, Display, Copy, Clone, PartialEq, FromPrimitive, ToPrimitive)]
@@ -83,23 +81,6 @@ pub struct Provider {
pub oauth2_authorizer: Option<Oauth2Authorizer>,
}
/// Get resolver to query MX records.
///
/// We first try resolver_from_system_conf() which reads the system's resolver from `/etc/resolv.conf`.
/// This does not work at least on some Androids, therefore we use use ResolverConfig::default()
/// which default eg. to google's 8.8.8.8 or 8.8.4.4 as a fallback.
async fn get_resolver() -> Result<AsyncStdResolver> {
if let Ok(resolver) = resolver_from_system_conf().await {
return Ok(resolver);
}
let resolver = resolver(
config::ResolverConfig::default(),
config::ResolverOpts::default(),
)
.await?;
Ok(resolver)
}
/// Returns provider for the given domain.
///
/// This function looks up domain in offline database first. If not
@@ -108,19 +89,15 @@ async fn get_resolver() -> Result<AsyncStdResolver> {
///
/// For compatibility, email address can be passed to this function
/// instead of the domain.
pub async fn get_provider_info(
context: &Context,
domain: &str,
skip_mx: bool,
) -> Option<&'static Provider> {
let domain = domain.rsplit('@').next()?;
pub async fn get_provider_info(domain: &str, skip_mx: bool) -> Option<&'static Provider> {
let domain = domain.rsplitn(2, '@').next()?;
if let Some(provider) = get_provider_by_domain(domain) {
return Some(provider);
}
if !skip_mx {
if let Some(provider) = get_provider_by_mx(context, domain).await {
if let Some(provider) = get_provider_by_mx(domain).await {
return Some(provider);
}
}
@@ -140,8 +117,8 @@ pub fn get_provider_by_domain(domain: &str) -> Option<&'static Provider> {
/// Finds a provider based on MX record for the given domain.
///
/// For security reasons, only Gmail can be configured this way.
pub async fn get_provider_by_mx(context: &Context, domain: &str) -> Option<&'static Provider> {
if let Ok(resolver) = get_resolver().await {
pub async fn get_provider_by_mx(domain: &str) -> Option<&'static Provider> {
if let Ok(resolver) = resolver_from_system_conf().await {
let mut fqdn: String = domain.to_string();
if !fqdn.ends_with('.') {
fqdn.push('.');
@@ -166,8 +143,6 @@ pub async fn get_provider_by_mx(context: &Context, domain: &str) -> Option<&'sta
}
}
}
} else {
warn!(context, "cannot get a resolver to check MX records.");
}
None
@@ -194,7 +169,6 @@ mod tests {
use super::*;
use crate::dc_tools::time;
use crate::test_utils::TestContext;
use chrono::NaiveDate;
#[test]
@@ -244,13 +218,12 @@ mod tests {
#[async_std::test]
async fn test_get_provider_info() {
let t = TestContext::new().await;
assert!(get_provider_info(&t, "", false).await.is_none());
assert!(get_provider_info(&t, "google.com", false).await.unwrap().id == "gmail");
assert!(get_provider_info("", false).await.is_none());
assert!(get_provider_info("google.com", false).await.unwrap().id == "gmail");
// get_provider_info() accepts email addresses for backwards compatibility
assert!(
get_provider_info(&t, "example@google.com", false)
get_provider_info("example@google.com", false)
.await
.unwrap()
.id
@@ -269,10 +242,4 @@ mod tests {
assert!(get_provider_update_timestamp() <= time());
assert!(get_provider_update_timestamp() > timestamp_past);
}
#[async_std::test]
async fn test_get_resolver() -> Result<()> {
assert!(get_resolver().await.is_ok());
Ok(())
}
}

View File

@@ -282,23 +282,61 @@ static P_DISROOT: Lazy<Provider> = Lazy::new(|| Provider {
before_login_hint: "",
after_login_hint: "",
overview_page: "https://providers.delta.chat/disroot",
server: vec![],
config_defaults: None,
strict_tls: true,
max_smtp_rcpt_to: None,
oauth2_authorizer: None,
});
// dubby.org.md: dubby.org
static P_DUBBY_ORG: Lazy<Provider> = Lazy::new(|| Provider {
id: "dubby.org",
status: Status::Ok,
before_login_hint: "",
after_login_hint: "",
overview_page: "https://providers.delta.chat/dubby-org",
server: vec![
Server {
protocol: Imap,
socket: Ssl,
hostname: "disroot.org",
hostname: "dubby.org",
port: 993,
username_pattern: Email,
},
Server {
protocol: Smtp,
socket: Starttls,
hostname: "disroot.org",
hostname: "dubby.org",
port: 587,
username_pattern: Email,
},
Server {
protocol: Smtp,
socket: Ssl,
hostname: "dubby.org",
port: 465,
username_pattern: Email,
},
],
config_defaults: None,
config_defaults: Some(vec![
ConfigDefault {
key: Config::BccSelf,
value: "1",
},
ConfigDefault {
key: Config::SentboxWatch,
value: "0",
},
ConfigDefault {
key: Config::MvboxWatch,
value: "0",
},
ConfigDefault {
key: Config::MvboxMove,
value: "0",
},
]),
strict_tls: true,
max_smtp_rcpt_to: None,
oauth2_authorizer: None,
@@ -366,7 +404,7 @@ static P_EXAMPLE_COM: Lazy<Provider> = Lazy::new(|| {
}
});
// fastmail.md: 123mail.org, 150mail.com, 150ml.com, 16mail.com, 2-mail.com, 4email.net, 50mail.com, airpost.net, allmail.net, bestmail.us, cluemail.com, elitemail.org, emailcorner.net, emailengine.net, emailengine.org, emailgroups.net, emailplus.org, emailuser.net, eml.cc, f-m.fm, fast-email.com, fast-mail.org, fastem.com, fastemail.us, fastemailer.com, fastest.cc, fastimap.com, fastmail.cn, fastmail.co.uk, fastmail.com, fastmail.com.au, fastmail.de, fastmail.es, fastmail.fm, fastmail.fr, fastmail.im, fastmail.in, fastmail.jp, fastmail.mx, fastmail.net, fastmail.nl, fastmail.org, fastmail.se, fastmail.to, fastmail.tw, fastmail.uk, fastmail.us, fastmailbox.net, fastmessaging.com, fea.st, fmail.co.uk, fmailbox.com, fmgirl.com, fmguy.com, ftml.net, h-mail.us, hailmail.net, imap-mail.com, imap.cc, imapmail.org, inoutbox.com, internet-e-mail.com, internet-mail.org, internetemails.net, internetmailing.net, jetemail.net, justemail.net, letterboxes.org, mail-central.com, mail-page.com, mailandftp.com, mailas.com, mailbolt.com, mailc.net, mailcan.com, mailforce.net, mailftp.com, mailhaven.com, mailingaddress.org, mailite.com, mailmight.com, mailnew.com, mailsent.net, mailservice.ms, mailup.net, mailworks.org, ml1.net, mm.st, myfastmail.com, mymacmail.com, nospammail.net, ownmail.net, petml.com, postinbox.com, postpro.net, proinbox.com, promessage.com, realemail.net, reallyfast.biz, reallyfast.info, rushpost.com, sent.as, sent.at, sent.com, speedpost.net, speedymail.org, ssl-mail.com, swift-mail.com, the-fastest.net, the-quickest.com, theinternetemail.com, veryfast.biz, veryspeedy.net, warpmail.net, xsmail.com, yepmail.net, your-mail.com
// fastmail.md: fastmail.com
static P_FASTMAIL: Lazy<Provider> = Lazy::new(|| Provider {
id: "fastmail",
status: Status::Preparation,
@@ -389,6 +427,13 @@ static P_FASTMAIL: Lazy<Provider> = Lazy::new(|| Provider {
port: 465,
username_pattern: Email,
},
Server {
protocol: Smtp,
socket: Starttls,
hostname: "smtp.fastmail.com",
port: 587,
username_pattern: Email,
},
],
config_defaults: None,
strict_tls: true,
@@ -430,6 +475,10 @@ static P_FIVE_CHAT: Lazy<Provider> = Lazy::new(|| Provider {
key: Config::SentboxWatch,
value: "0",
},
ConfigDefault {
key: Config::MvboxWatch,
value: "0",
},
ConfigDefault {
key: Config::MvboxMove,
value: "0",
@@ -619,35 +668,6 @@ static P_ICLOUD: Lazy<Provider> = Lazy::new(|| Provider {
oauth2_authorizer: None,
});
// infomaniak.com.md: ik.me
static P_INFOMANIAK_COM: Lazy<Provider> = Lazy::new(|| Provider {
id: "infomaniak.com",
status: Status::Ok,
before_login_hint: "",
after_login_hint: "",
overview_page: "https://providers.delta.chat/infomaniak-com",
server: vec![
Server {
protocol: Imap,
socket: Ssl,
hostname: "mail.infomaniak.com",
port: 993,
username_pattern: Email,
},
Server {
protocol: Smtp,
socket: Ssl,
hostname: "mail.infomaniak.com",
port: 465,
username_pattern: Email,
},
],
config_defaults: None,
strict_tls: true,
max_smtp_rcpt_to: Some(10),
oauth2_authorizer: None,
});
// kolst.com.md: kolst.com
static P_KOLST_COM: Lazy<Provider> = Lazy::new(|| Provider {
id: "kolst.com",
@@ -676,41 +696,12 @@ static P_KONTENT_COM: Lazy<Provider> = Lazy::new(|| Provider {
oauth2_authorizer: None,
});
// mail.de.md: mail.de
static P_MAIL_DE: Lazy<Provider> = Lazy::new(|| Provider {
id: "mail.de",
status: Status::Ok,
before_login_hint: "",
after_login_hint: "",
overview_page: "https://providers.delta.chat/mail-de",
server: vec![
Server {
protocol: Imap,
socket: Ssl,
hostname: "imap.mail.de",
port: 993,
username_pattern: Email,
},
Server {
protocol: Smtp,
socket: Ssl,
hostname: "smtp.mail.de",
port: 465,
username_pattern: Email,
},
],
config_defaults: None,
strict_tls: true,
max_smtp_rcpt_to: None,
oauth2_authorizer: None,
});
// mail.ru.md: mail.ru, inbox.ru, internet.ru, bk.ru, list.ru
static P_MAIL_RU: Lazy<Provider> = Lazy::new(|| {
Provider {
id: "mail.ru",
status: Status::Preparation,
before_login_hint: "Вам необходимо сгенерировать \"пароль для внешнего приложения\" в веб-интерфейсе mail.ru, чтобы mail.ru работал с Delta Chat.",
status: Status::Ok,
before_login_hint: "Не рекомендуется использовать mail.ru, потому что он разряжает вашу батарею быстрее, чем другие провайдеры.",
after_login_hint: "",
overview_page: "https://providers.delta.chat/mail-ru",
server: vec![
@@ -760,22 +751,7 @@ static P_MAILBOX_ORG: Lazy<Provider> = Lazy::new(|| Provider {
before_login_hint: "",
after_login_hint: "",
overview_page: "https://providers.delta.chat/mailbox-org",
server: vec![
Server {
protocol: Imap,
socket: Ssl,
hostname: "imap.mailbox.org",
port: 993,
username_pattern: Email,
},
Server {
protocol: Smtp,
socket: Ssl,
hostname: "smtp.mailbox.org",
port: 465,
username_pattern: Email,
},
],
server: vec![],
config_defaults: None,
strict_tls: true,
max_smtp_rcpt_to: None,
@@ -847,6 +823,10 @@ static P_NAUTA_CU: Lazy<Provider> = Lazy::new(|| Provider {
key: Config::SentboxWatch,
value: "0",
},
ConfigDefault {
key: Config::MvboxWatch,
value: "0",
},
ConfigDefault {
key: Config::MvboxMove,
value: "0",
@@ -898,7 +878,7 @@ static P_NAVER: Lazy<Provider> = Lazy::new(|| Provider {
oauth2_authorizer: None,
});
// outlook.com.md: hotmail.com, outlook.com, office365.com, outlook.com.tr, live.com, outlook.de
// outlook.com.md: hotmail.com, outlook.com, office365.com, outlook.com.tr, live.com
static P_OUTLOOK_COM: Lazy<Provider> = Lazy::new(|| Provider {
id: "outlook.com",
status: Status::Ok,
@@ -927,7 +907,7 @@ static P_OUTLOOK_COM: Lazy<Provider> = Lazy::new(|| Provider {
oauth2_authorizer: None,
});
// posteo.md: posteo.de, posteo.af, posteo.at, posteo.be, posteo.ca, posteo.ch, posteo.cl, posteo.co, posteo.co.uk, posteo.com.br, posteo.cr, posteo.cz, posteo.dk, posteo.ee, posteo.es, posteo.eu, posteo.fi, posteo.gl, posteo.gr, posteo.hn, posteo.hr, posteo.hu, posteo.ie, posteo.in, posteo.is, posteo.it, posteo.jp, posteo.la, posteo.li, posteo.lt, posteo.lu, posteo.me, posteo.mx, posteo.my, posteo.net, posteo.nl, posteo.no, posteo.nz, posteo.org, posteo.pe, posteo.pl, posteo.pm, posteo.pt, posteo.ro, posteo.ru, posteo.se, posteo.sg, posteo.si, posteo.tn, posteo.uk, posteo.us
// posteo.md: posteo.de, posteo.af, posteo.at, posteo.be, posteo.ch, posteo.cl, posteo.co, posteo.co.uk, posteo.com.br, posteo.cr, posteo.cz, posteo.dk, posteo.ee, posteo.es, posteo.eu, posteo.fi, posteo.gl, posteo.gr, posteo.hn, posteo.hr, posteo.hu, posteo.ie, posteo.in, posteo.is, posteo.it, posteo.jp, posteo.la, posteo.li, posteo.lt, posteo.lu, posteo.me, posteo.mx, posteo.my, posteo.net, posteo.nl, posteo.no, posteo.nz, posteo.org, posteo.pe, posteo.pl, posteo.pm, posteo.pt, posteo.ro, posteo.ru, posteo.se, posteo.sg, posteo.si, posteo.tn, posteo.uk, posteo.us
static P_POSTEO: Lazy<Provider> = Lazy::new(|| Provider {
id: "posteo",
status: Status::Ok,
@@ -956,7 +936,7 @@ static P_POSTEO: Lazy<Provider> = Lazy::new(|| Provider {
oauth2_authorizer: None,
});
// protonmail.md: protonmail.com, protonmail.ch, pm.me
// protonmail.md: protonmail.com, protonmail.ch
static P_PROTONMAIL: Lazy<Provider> = Lazy::new(|| {
Provider {
id: "protonmail",
@@ -999,22 +979,7 @@ static P_RISEUP_NET: Lazy<Provider> = Lazy::new(|| Provider {
before_login_hint: "",
after_login_hint: "",
overview_page: "https://providers.delta.chat/riseup-net",
server: vec![
Server {
protocol: Imap,
socket: Ssl,
hostname: "mail.riseup.net",
port: 993,
username_pattern: Email,
},
Server {
protocol: Smtp,
socket: Ssl,
hostname: "mail.riseup.net",
port: 465,
username_pattern: Email,
},
],
server: vec![],
config_defaults: None,
strict_tls: true,
max_smtp_rcpt_to: None,
@@ -1071,22 +1036,7 @@ static P_SYSTEMLI_ORG: Lazy<Provider> = Lazy::new(|| Provider {
before_login_hint: "",
after_login_hint: "",
overview_page: "https://providers.delta.chat/systemli-org",
server: vec![
Server {
protocol: Imap,
socket: Ssl,
hostname: "mail.systemli.org",
port: 993,
username_pattern: Email,
},
Server {
protocol: Smtp,
socket: Ssl,
hostname: "mail.systemli.org",
port: 465,
username_pattern: Email,
},
],
server: vec![],
config_defaults: None,
strict_tls: true,
max_smtp_rcpt_to: None,
@@ -1151,6 +1101,10 @@ static P_TESTRUN: Lazy<Provider> = Lazy::new(|| Provider {
key: Config::SentboxWatch,
value: "0",
},
ConfigDefault {
key: Config::MvboxWatch,
value: "0",
},
ConfigDefault {
key: Config::MvboxMove,
value: "0",
@@ -1483,128 +1437,13 @@ pub(crate) static PROVIDER_DATA: Lazy<HashMap<&'static str, &'static Provider>>
("comcast.net", &*P_COMCAST),
("dismail.de", &*P_DISMAIL_DE),
("disroot.org", &*P_DISROOT),
("dubby.org", &*P_DUBBY_ORG),
("e.email", &*P_E_EMAIL),
("espiv.net", &*P_ESPIV_NET),
("example.com", &*P_EXAMPLE_COM),
("example.org", &*P_EXAMPLE_COM),
("example.net", &*P_EXAMPLE_COM),
("123mail.org", &*P_FASTMAIL),
("150mail.com", &*P_FASTMAIL),
("150ml.com", &*P_FASTMAIL),
("16mail.com", &*P_FASTMAIL),
("2-mail.com", &*P_FASTMAIL),
("4email.net", &*P_FASTMAIL),
("50mail.com", &*P_FASTMAIL),
("airpost.net", &*P_FASTMAIL),
("allmail.net", &*P_FASTMAIL),
("bestmail.us", &*P_FASTMAIL),
("cluemail.com", &*P_FASTMAIL),
("elitemail.org", &*P_FASTMAIL),
("emailcorner.net", &*P_FASTMAIL),
("emailengine.net", &*P_FASTMAIL),
("emailengine.org", &*P_FASTMAIL),
("emailgroups.net", &*P_FASTMAIL),
("emailplus.org", &*P_FASTMAIL),
("emailuser.net", &*P_FASTMAIL),
("eml.cc", &*P_FASTMAIL),
("f-m.fm", &*P_FASTMAIL),
("fast-email.com", &*P_FASTMAIL),
("fast-mail.org", &*P_FASTMAIL),
("fastem.com", &*P_FASTMAIL),
("fastemail.us", &*P_FASTMAIL),
("fastemailer.com", &*P_FASTMAIL),
("fastest.cc", &*P_FASTMAIL),
("fastimap.com", &*P_FASTMAIL),
("fastmail.cn", &*P_FASTMAIL),
("fastmail.co.uk", &*P_FASTMAIL),
("fastmail.com", &*P_FASTMAIL),
("fastmail.com.au", &*P_FASTMAIL),
("fastmail.de", &*P_FASTMAIL),
("fastmail.es", &*P_FASTMAIL),
("fastmail.fm", &*P_FASTMAIL),
("fastmail.fr", &*P_FASTMAIL),
("fastmail.im", &*P_FASTMAIL),
("fastmail.in", &*P_FASTMAIL),
("fastmail.jp", &*P_FASTMAIL),
("fastmail.mx", &*P_FASTMAIL),
("fastmail.net", &*P_FASTMAIL),
("fastmail.nl", &*P_FASTMAIL),
("fastmail.org", &*P_FASTMAIL),
("fastmail.se", &*P_FASTMAIL),
("fastmail.to", &*P_FASTMAIL),
("fastmail.tw", &*P_FASTMAIL),
("fastmail.uk", &*P_FASTMAIL),
("fastmail.us", &*P_FASTMAIL),
("fastmailbox.net", &*P_FASTMAIL),
("fastmessaging.com", &*P_FASTMAIL),
("fea.st", &*P_FASTMAIL),
("fmail.co.uk", &*P_FASTMAIL),
("fmailbox.com", &*P_FASTMAIL),
("fmgirl.com", &*P_FASTMAIL),
("fmguy.com", &*P_FASTMAIL),
("ftml.net", &*P_FASTMAIL),
("h-mail.us", &*P_FASTMAIL),
("hailmail.net", &*P_FASTMAIL),
("imap-mail.com", &*P_FASTMAIL),
("imap.cc", &*P_FASTMAIL),
("imapmail.org", &*P_FASTMAIL),
("inoutbox.com", &*P_FASTMAIL),
("internet-e-mail.com", &*P_FASTMAIL),
("internet-mail.org", &*P_FASTMAIL),
("internetemails.net", &*P_FASTMAIL),
("internetmailing.net", &*P_FASTMAIL),
("jetemail.net", &*P_FASTMAIL),
("justemail.net", &*P_FASTMAIL),
("letterboxes.org", &*P_FASTMAIL),
("mail-central.com", &*P_FASTMAIL),
("mail-page.com", &*P_FASTMAIL),
("mailandftp.com", &*P_FASTMAIL),
("mailas.com", &*P_FASTMAIL),
("mailbolt.com", &*P_FASTMAIL),
("mailc.net", &*P_FASTMAIL),
("mailcan.com", &*P_FASTMAIL),
("mailforce.net", &*P_FASTMAIL),
("mailftp.com", &*P_FASTMAIL),
("mailhaven.com", &*P_FASTMAIL),
("mailingaddress.org", &*P_FASTMAIL),
("mailite.com", &*P_FASTMAIL),
("mailmight.com", &*P_FASTMAIL),
("mailnew.com", &*P_FASTMAIL),
("mailsent.net", &*P_FASTMAIL),
("mailservice.ms", &*P_FASTMAIL),
("mailup.net", &*P_FASTMAIL),
("mailworks.org", &*P_FASTMAIL),
("ml1.net", &*P_FASTMAIL),
("mm.st", &*P_FASTMAIL),
("myfastmail.com", &*P_FASTMAIL),
("mymacmail.com", &*P_FASTMAIL),
("nospammail.net", &*P_FASTMAIL),
("ownmail.net", &*P_FASTMAIL),
("petml.com", &*P_FASTMAIL),
("postinbox.com", &*P_FASTMAIL),
("postpro.net", &*P_FASTMAIL),
("proinbox.com", &*P_FASTMAIL),
("promessage.com", &*P_FASTMAIL),
("realemail.net", &*P_FASTMAIL),
("reallyfast.biz", &*P_FASTMAIL),
("reallyfast.info", &*P_FASTMAIL),
("rushpost.com", &*P_FASTMAIL),
("sent.as", &*P_FASTMAIL),
("sent.at", &*P_FASTMAIL),
("sent.com", &*P_FASTMAIL),
("speedpost.net", &*P_FASTMAIL),
("speedymail.org", &*P_FASTMAIL),
("ssl-mail.com", &*P_FASTMAIL),
("swift-mail.com", &*P_FASTMAIL),
("the-fastest.net", &*P_FASTMAIL),
("the-quickest.com", &*P_FASTMAIL),
("theinternetemail.com", &*P_FASTMAIL),
("veryfast.biz", &*P_FASTMAIL),
("veryspeedy.net", &*P_FASTMAIL),
("warpmail.net", &*P_FASTMAIL),
("xsmail.com", &*P_FASTMAIL),
("yepmail.net", &*P_FASTMAIL),
("your-mail.com", &*P_FASTMAIL),
("firemail.at", &*P_FIREMAIL_DE),
("firemail.de", &*P_FIREMAIL_DE),
("five.chat", &*P_FIVE_CHAT),
@@ -1628,10 +1467,8 @@ pub(crate) static PROVIDER_DATA: Lazy<HashMap<&'static str, &'static Provider>>
("icloud.com", &*P_ICLOUD),
("me.com", &*P_ICLOUD),
("mac.com", &*P_ICLOUD),
("ik.me", &*P_INFOMANIAK_COM),
("kolst.com", &*P_KOLST_COM),
("kontent.com", &*P_KONTENT_COM),
("mail.de", &*P_MAIL_DE),
("mail.ru", &*P_MAIL_RU),
("inbox.ru", &*P_MAIL_RU),
("internet.ru", &*P_MAIL_RU),
@@ -1648,12 +1485,10 @@ pub(crate) static PROVIDER_DATA: Lazy<HashMap<&'static str, &'static Provider>>
("office365.com", &*P_OUTLOOK_COM),
("outlook.com.tr", &*P_OUTLOOK_COM),
("live.com", &*P_OUTLOOK_COM),
("outlook.de", &*P_OUTLOOK_COM),
("posteo.de", &*P_POSTEO),
("posteo.af", &*P_POSTEO),
("posteo.at", &*P_POSTEO),
("posteo.be", &*P_POSTEO),
("posteo.ca", &*P_POSTEO),
("posteo.ch", &*P_POSTEO),
("posteo.cl", &*P_POSTEO),
("posteo.co", &*P_POSTEO),
@@ -1702,7 +1537,6 @@ pub(crate) static PROVIDER_DATA: Lazy<HashMap<&'static str, &'static Provider>>
("posteo.us", &*P_POSTEO),
("protonmail.com", &*P_PROTONMAIL),
("protonmail.ch", &*P_PROTONMAIL),
("pm.me", &*P_PROTONMAIL),
("qq.com", &*P_QQ),
("foxmail.com", &*P_QQ),
("riseup.net", &*P_RISEUP_NET),
@@ -1799,6 +1633,7 @@ pub(crate) static PROVIDER_IDS: Lazy<HashMap<&'static str, &'static Provider>> =
("comcast", &*P_COMCAST),
("dismail.de", &*P_DISMAIL_DE),
("disroot", &*P_DISROOT),
("dubby.org", &*P_DUBBY_ORG),
("e.email", &*P_E_EMAIL),
("espiv.net", &*P_ESPIV_NET),
("example.com", &*P_EXAMPLE_COM),
@@ -1813,10 +1648,8 @@ pub(crate) static PROVIDER_IDS: Lazy<HashMap<&'static str, &'static Provider>> =
("i.ua", &*P_I_UA),
("i3.net", &*P_I3_NET),
("icloud", &*P_ICLOUD),
("infomaniak.com", &*P_INFOMANIAK_COM),
("kolst.com", &*P_KOLST_COM),
("kontent.com", &*P_KONTENT_COM),
("mail.de", &*P_MAIL_DE),
("mail.ru", &*P_MAIL_RU),
("mail2tor", &*P_MAIL2TOR),
("mailbox.org", &*P_MAILBOX_ORG),
@@ -1853,4 +1686,4 @@ pub(crate) static PROVIDER_IDS: Lazy<HashMap<&'static str, &'static Provider>> =
});
pub static PROVIDER_UPDATED: Lazy<chrono::NaiveDate> =
Lazy::new(|| chrono::NaiveDate::from_ymd(2022, 1, 31));
Lazy::new(|| chrono::NaiveDate::from_ymd(2021, 9, 29));

View File

@@ -9,7 +9,7 @@ use std::collections::BTreeMap;
use crate::chat::{self, get_chat_id_by_grpid, ChatIdBlocked};
use crate::config::Config;
use crate::constants::Blocked;
use crate::contact::{addr_normalize, may_be_valid_addr, Contact, ContactId, Origin};
use crate::contact::{addr_normalize, may_be_valid_addr, Contact, Origin};
use crate::context::Context;
use crate::dc_tools::time;
use crate::key::Fingerprint;
@@ -30,7 +30,7 @@ const HTTPS_SCHEME: &str = "https://";
#[derive(Debug, Clone, PartialEq)]
pub enum Qr {
AskVerifyContact {
contact_id: ContactId,
contact_id: u32,
fingerprint: Fingerprint,
invitenumber: String,
authcode: String,
@@ -38,16 +38,16 @@ pub enum Qr {
AskVerifyGroup {
grpname: String,
grpid: String,
contact_id: ContactId,
contact_id: u32,
fingerprint: Fingerprint,
invitenumber: String,
authcode: String,
},
FprOk {
contact_id: ContactId,
contact_id: u32,
},
FprMismatch {
contact_id: Option<ContactId>,
contact_id: Option<u32>,
},
FprWithoutAddr {
fingerprint: String,
@@ -60,7 +60,7 @@ pub enum Qr {
instance_pattern: String,
},
Addr {
contact_id: ContactId,
contact_id: u32,
},
Url {
url: String,
@@ -69,7 +69,7 @@ pub enum Qr {
text: String,
},
WithdrawVerifyContact {
contact_id: ContactId,
contact_id: u32,
fingerprint: Fingerprint,
invitenumber: String,
authcode: String,
@@ -77,13 +77,13 @@ pub enum Qr {
WithdrawVerifyGroup {
grpname: String,
grpid: String,
contact_id: ContactId,
contact_id: u32,
fingerprint: Fingerprint,
invitenumber: String,
authcode: String,
},
ReviveVerifyContact {
contact_id: ContactId,
contact_id: u32,
fingerprint: Fingerprint,
invitenumber: String,
authcode: String,
@@ -91,7 +91,7 @@ pub enum Qr {
ReviveVerifyGroup {
grpname: String,
grpid: String,
contact_id: ContactId,
contact_id: u32,
fingerprint: Fingerprint,
invitenumber: String,
authcode: String,
@@ -173,7 +173,7 @@ async fn decode_openpgp(context: &Context, qr: &str) -> Result<Qr> {
};
let name = if let Some(encoded_name) = param.get("n") {
let encoded_name = encoded_name.replace('+', "%20"); // sometimes spaces are encoded as `+`
let encoded_name = encoded_name.replace("+", "%20"); // sometimes spaces are encoded as `+`
match percent_decode_str(&encoded_name).decode_utf8() {
Ok(name) => name.to_string(),
Err(err) => bail!("Invalid name: {}", err),
@@ -188,7 +188,7 @@ async fn decode_openpgp(context: &Context, qr: &str) -> Result<Qr> {
let grpname = if grpid.is_some() {
if let Some(encoded_name) = param.get("g") {
let encoded_name = encoded_name.replace('+', "%20"); // sometimes spaces are encoded as `+`
let encoded_name = encoded_name.replace("+", "%20"); // sometimes spaces are encoded as `+`
match percent_decode_str(&encoded_name).decode_utf8() {
Ok(name) => Some(name.to_string()),
Err(err) => bail!("Invalid group name: {}", err),
@@ -282,7 +282,7 @@ async fn decode_openpgp(context: &Context, qr: &str) -> Result<Qr> {
chat::add_info_msg(
context,
chat.id,
&format!("{} verified.", peerstate.addr),
format!("{} verified.", peerstate.addr),
time(),
)
.await?;
@@ -304,14 +304,14 @@ async fn decode_openpgp(context: &Context, qr: &str) -> Result<Qr> {
fn decode_account(qr: &str) -> Result<Qr> {
let payload = qr
.get(DCACCOUNT_SCHEME.len()..)
.context("invalid DCACCOUNT payload")?;
.ok_or_else(|| format_err!("Invalid DCACCOUNT payload"))?;
let url =
url::Url::parse(payload).with_context(|| format!("Invalid account URL: {:?}", payload))?;
if url.scheme() == "http" || url.scheme() == "https" {
Ok(Qr::Account {
domain: url
.host_str()
.context("can't extract WebRTC instance domain")?
.ok_or_else(|| format_err!("Can't extract WebRTC instance domain"))?
.to_string(),
})
} else {
@@ -323,7 +323,7 @@ fn decode_account(qr: &str) -> Result<Qr> {
fn decode_webrtc_instance(_context: &Context, qr: &str) -> Result<Qr> {
let payload = qr
.get(DCWEBRTC_SCHEME.len()..)
.context("invalid DCWEBRTC payload")?;
.ok_or_else(|| format_err!("Invalid DCWEBRTC payload"))?;
let (_type, url) = Message::parse_webrtc_instance(payload);
let url =
@@ -333,7 +333,7 @@ fn decode_webrtc_instance(_context: &Context, qr: &str) -> Result<Qr> {
Ok(Qr::WebrtcInstance {
domain: url
.host_str()
.context("can't extract WebRTC instance domain")?
.ok_or_else(|| format_err!("Can't extract WebRTC instance domain"))?
.to_string(),
instance_pattern: payload.to_string(),
})
@@ -424,7 +424,7 @@ pub async fn set_config_from_qr(context: &Context, qr: &str) -> Result<()> {
grpid,
..
} => {
let chat_id = get_chat_id_by_grpid(context, &grpid)
let chat_id = get_chat_id_by_grpid(context, grpid)
.await?
.map(|(chat_id, _protected, _blocked)| chat_id);
token::save(
@@ -711,7 +711,7 @@ mod tests {
..
} = qr
{
assert_ne!(contact_id, ContactId::new(0));
assert_ne!(contact_id, 0);
assert_eq!(grpname, "test ? test !");
} else {
bail!("Wrong QR code type");
@@ -729,7 +729,7 @@ mod tests {
..
} = qr
{
assert_ne!(contact_id, ContactId::new(0));
assert_ne!(contact_id, 0);
assert_eq!(grpname, "test ? test !");
let contact = Contact::get_by_id(&ctx.ctx, contact_id).await?;
@@ -751,7 +751,7 @@ mod tests {
).await?;
if let Qr::AskVerifyContact { contact_id, .. } = qr {
assert_ne!(contact_id, ContactId::new(0));
assert_ne!(contact_id, 0);
} else {
bail!("Wrong QR code type");
}
@@ -792,12 +792,12 @@ mod tests {
async fn test_decode_openpgp_fingerprint() -> Result<()> {
let ctx = TestContext::new().await;
let alice_contact_id = Contact::create(&ctx, "Alice", "alice@example.org")
let alice_contact_id = Contact::create(&ctx, "Alice", "alice@example.com")
.await
.context("failed to create contact")?;
let pub_key = alice_keypair().public;
let peerstate = Peerstate {
addr: "alice@example.org".to_string(),
addr: "alice@example.com".to_string(),
last_seen: 1,
last_seen_autocrypt: 1,
prefer_encrypt: EncryptPreference::Mutual,
@@ -818,7 +818,7 @@ mod tests {
let qr = check_qr(
&ctx.ctx,
"OPENPGP4FPR:1234567890123456789012345678901234567890#a=alice@example.org",
"OPENPGP4FPR:1234567890123456789012345678901234567890#a=alice@example.com",
)
.await?;
if let Qr::FprMismatch { contact_id, .. } = qr {
@@ -829,7 +829,7 @@ mod tests {
let qr = check_qr(
&ctx.ctx,
&format!("OPENPGP4FPR:{}#a=alice@example.org", pub_key.fingerprint()),
&format!("OPENPGP4FPR:{}#a=alice@example.com", pub_key.fingerprint()),
)
.await?;
if let Qr::FprOk { contact_id, .. } = qr {

View File

@@ -89,23 +89,21 @@ fn inner_generate_secure_join_qr_code(
let mut w = tagger::new(&mut svg);
w.elem("svg", |d| {
d.attr("xmlns", "http://www.w3.org/2000/svg")?;
d.attr("viewBox", format_args!("0 0 {} {}", width, height))?;
Ok(())
})?
d.attr("xmlns", "http://www.w3.org/2000/svg")
.attr("viewBox", format_args!("0 0 {} {}", width, height));
})
.build(|w| {
// White Background apears like a card
w.single("rect", |d| {
d.attr("x", card_border_size)?;
d.attr("y", card_border_size)?;
d.attr("rx", card_roundness)?;
d.attr("stroke", "#c6c6c6")?;
d.attr("stroke-width", card_border_size)?;
d.attr("width", width - (card_border_size * 2.0))?;
d.attr("height", height - (card_border_size * 2.0))?;
d.attr("style", "fill:#f2f2f2")?;
Ok(())
})?;
d.attr("x", card_border_size)
.attr("y", card_border_size)
.attr("rx", card_roundness)
.attr("stroke", "#c6c6c6")
.attr("stroke-width", card_border_size)
.attr("width", width - (card_border_size * 2.0))
.attr("height", height - (card_border_size * 2.0))
.attr("style", "fill:#f2f2f2");
});
// Qrcode
w.elem("g", |d| {
d.attr(
@@ -115,12 +113,12 @@ fn inner_generate_secure_join_qr_code(
(width - qr_code_size) / 2.0,
((height - qr_code_size) / 2.0) - qr_translate_up
),
)
);
// If the qr code should be in the wrong place,
// we could also translate and scale the points in the path already,
// but that would make the resulting svg way bigger in size and might bring up rounding issues,
// so better avoid doing it manually if possible
})?
})
.build(|w| {
w.single("path", |d| {
let mut path_data = String::with_capacity(0);
@@ -134,16 +132,16 @@ fn inner_generate_secure_join_qr_code(
}
}
d.attr("style", "fill:#000000")?;
d.attr("d", path_data)?;
d.attr("transform", format!("scale({})", scale))
})
})?;
d.attr("style", "fill:#000000")
.attr("d", path_data)
.attr("transform", format!("scale({})", scale));
});
});
// Text
const BIG_TEXT_CHARS_PER_LINE: usize = 32;
const SMALL_TEXT_CHARS_PER_LINE: usize = 38;
let chars_per_line = if qrcode_description.len() > SMALL_TEXT_CHARS_PER_LINE * 2 {
let chars_per_line = if qrcode_description.len() > SMALL_TEXT_CHARS_PER_LINE*2 {
SMALL_TEXT_CHARS_PER_LINE
} else {
BIG_TEXT_CHARS_PER_LINE
@@ -154,27 +152,27 @@ fn inner_generate_secure_join_qr_code(
} else {
(19.0, -10.0)
};
for (count, line) in lines.split('\n').enumerate() {
for (count, line) in lines.split('\n').enumerate()
{
w.elem("text", |d| {
d.attr(
"y",
(count as f32 * (text_font_size * 1.2)) + text_y_pos + text_y_shift,
)?;
d.attr("x", width / 2.0)?;
d.attr("text-anchor", "middle")?;
d.attr(
"style",
format!(
"font-family:sans-serif;\
d.attr("y", (count as f32 * (text_font_size * 1.2)) + text_y_pos + text_y_shift)
.attr("x", width / 2.0)
.attr("text-anchor", "middle")
.attr(
"style",
format!(
"font-family:sans-serif;\
font-weight:bold;\
font-size:{}px;\
fill:#000000;\
stroke:none",
text_font_size
),
)
})?
.build(|w| w.put_raw(line))?;
text_font_size
),
);
})
.build(|w| {
w.put_raw(line);
});
}
// contact avatar in middle of qrcode
const LOGO_SIZE: f32 = 94.4;
@@ -185,64 +183,68 @@ fn inner_generate_secure_join_qr_code(
((height - qr_code_size) / 2.0) - qr_translate_up + logo_position_in_qr;
w.single("circle", |d| {
d.attr("cx", logo_position_x + HALF_LOGO_SIZE)?;
d.attr("cy", logo_position_y + HALF_LOGO_SIZE)?;
d.attr("r", HALF_LOGO_SIZE + avatar_border_size)?;
d.attr("style", "fill:#f2f2f2")
})?;
d.attr("cx", logo_position_x + HALF_LOGO_SIZE)
.attr("cy", logo_position_y + HALF_LOGO_SIZE)
.attr("r", HALF_LOGO_SIZE + avatar_border_size)
.attr("style", "fill:#f2f2f2");
});
if let Some(img) = avatar {
w.elem("defs", tagger::no_attr())?.build(|w| {
w.elem("clipPath", |d| d.attr("id", "avatar-cut"))?
.build(|w| {
w.single("circle", |d| {
d.attr("cx", logo_position_x + HALF_LOGO_SIZE)?;
d.attr("cy", logo_position_y + HALF_LOGO_SIZE)?;
d.attr("r", HALF_LOGO_SIZE)
})
})
})?;
w.elem("defs", |_| {}).build(|w| {
w.elem("clipPath", |d| {
d.attr("id", "avatar-cut");
})
.build(|w| {
w.single("circle", |d| {
d.attr("cx", logo_position_x + HALF_LOGO_SIZE)
.attr("cy", logo_position_y + HALF_LOGO_SIZE)
.attr("r", HALF_LOGO_SIZE);
});
});
});
w.single("image", |d| {
d.attr("x", logo_position_x)?;
d.attr("y", logo_position_y)?;
d.attr("width", HALF_LOGO_SIZE * 2.0)?;
d.attr("height", HALF_LOGO_SIZE * 2.0)?;
d.attr("preserveAspectRatio", "none")?;
d.attr("clip-path", "url(#avatar-cut)")?;
d.attr(
"href", /*might need xlink:href instead if it doesn't work on older devices?*/
format!("data:image/jpeg;base64,{}", base64::encode(img)),
)
})?;
d.attr("x", logo_position_x)
.attr("y", logo_position_y)
.attr("width", HALF_LOGO_SIZE * 2.0)
.attr("height", HALF_LOGO_SIZE * 2.0)
.attr("preserveAspectRatio", "none")
.attr("clip-path", "url(#avatar-cut)")
.attr(
"href" /*might need xlink:href instead if it doesn't work on older devices?*/,
format!("data:image/jpeg;base64,{}", base64::encode(img)),
);
});
} else {
w.single("circle", |d| {
d.attr("cx", logo_position_x + HALF_LOGO_SIZE)?;
d.attr("cy", logo_position_y + HALF_LOGO_SIZE)?;
d.attr("r", HALF_LOGO_SIZE)?;
d.attr("style", format!("fill:{}", &color))
})?;
d.attr("cx", logo_position_x + HALF_LOGO_SIZE)
.attr("cy", logo_position_y + HALF_LOGO_SIZE)
.attr("r", HALF_LOGO_SIZE)
.attr("style", format!("fill:{}", &color));
});
let avatar_font_size = LOGO_SIZE * 0.65;
let font_offset = avatar_font_size * 0.1;
w.elem("text", |d| {
d.attr("y", logo_position_y + HALF_LOGO_SIZE + font_offset)?;
d.attr("x", logo_position_x + HALF_LOGO_SIZE)?;
d.attr("text-anchor", "middle")?;
d.attr("dominant-baseline", "central")?;
d.attr("alignment-baseline", "middle")?;
d.attr(
"style",
format!(
"font-family:sans-serif;\
d.attr("y", logo_position_y + HALF_LOGO_SIZE + font_offset)
.attr("x", logo_position_x + HALF_LOGO_SIZE)
.attr("text-anchor", "middle")
.attr("dominant-baseline", "central")
.attr("alignment-baseline", "middle")
.attr(
"style",
format!(
"font-family:sans-serif;\
font-weight:400;\
font-size:{}px;\
fill:#ffffff;",
avatar_font_size
),
)
})?
.build(|w| w.put_raw(avatar_letter.to_uppercase()))?;
avatar_font_size
),
);
})
.build(|w| {
w.put_raw(avatar_letter.to_uppercase());
});
}
// Footer logo
@@ -256,10 +258,12 @@ fn inner_generate_secure_join_qr_code(
(width - FOOTER_WIDTH) / 2.0,
height - logo_offset - FOOTER_HEIGHT - text_y_shift
),
)
})?
.build(|w| w.put_raw(include_str!("../assets/qrcode_logo_footer.svg")))
})?;
);
})
.build(|w| {
w.put_raw(include_str!("../assets/qrcode_logo_footer.svg"));
});
});
Ok(svg)
}

View File

@@ -1,17 +1,18 @@
//! # Support for IMAP QUOTA extension.
use anyhow::{anyhow, Context as _, Result};
use anyhow::{anyhow, Result};
use async_imap::types::{Quota, QuotaResource};
use std::collections::BTreeMap;
use crate::chat::add_device_msg_with_importance;
use crate::config::Config;
use crate::constants::Viewtype;
use crate::context::Context;
use crate::dc_tools::time;
use crate::imap::scan_folders::get_watched_folders;
use crate::imap::Imap;
use crate::job::{Action, Status};
use crate::message::{Message, Viewtype};
use crate::message::Message;
use crate::param::Params;
use crate::{job, stock_str, EventType};
@@ -63,7 +64,7 @@ async fn get_unique_quota_roots_and_usage(
.iter()
.find(|q| &q.root_name == quota_root_name)
.cloned()
.context("quota_root should have a quota")?;
.ok_or_else(|| anyhow!("quota_root should have a quota"))?;
// replace old quotas, because between fetching quotaroots for folders,
// messages could be recieved and so the usage could have been changed
*unique_quota_roots
@@ -95,7 +96,7 @@ fn get_highest_usage<'t>(
}
}
highest.context("no quota_resource found, this is unexpected")
highest.ok_or_else(|| anyhow!("no quota_resource found, this is unexpected"))
}
/// Checks if a quota warning is needed.
@@ -136,7 +137,7 @@ impl Context {
}
let quota = if imap.can_check_quota() {
let folders = get_watched_folders(self).await?;
let folders = get_watched_folders(self).await;
get_unique_quota_roots_and_usage(folders, imap).await
} else {
Err(anyhow!(stock_str::not_supported_by_provider(self).await))

View File

@@ -1,4 +1,4 @@
use anyhow::{bail, Context as _, Result};
use anyhow::{bail, Result};
use async_std::prelude::*;
use async_std::{
channel::{self, Receiver, Sender},
@@ -8,11 +8,10 @@ use async_std::{
use crate::config::Config;
use crate::context::Context;
use crate::dc_tools::maybe_add_time_based_warnings;
use crate::ephemeral::delete_expired_imap_messages;
use crate::imap::Imap;
use crate::job::{self, Thread};
use crate::log::LogExt;
use crate::smtp::{send_smtp_messages, Smtp};
use crate::message::MsgId;
use crate::smtp::Smtp;
use self::connectivity::ConnectivityStore;
@@ -83,15 +82,11 @@ async fn inbox_loop(ctx: Context, started: Sender<()>, inbox_handlers: ImapConne
let mut jobs_loaded = 0;
let mut info = InterruptInfo::default();
loop {
let job = match job::load_next(&ctx, Thread::Imap, &info).await {
Err(err) => {
error!(ctx, "Failed loading job from the database: {:#}.", err);
None
}
Ok(job) => job,
};
match job {
match job::load_next(&ctx, Thread::Imap, &info)
.await
.ok()
.flatten()
{
Some(job) if jobs_loaded <= 20 => {
jobs_loaded += 1;
job::perform_job(&ctx, job::Connection::Inbox(&mut connection), job).await;
@@ -100,15 +95,41 @@ async fn inbox_loop(ctx: Context, started: Sender<()>, inbox_handlers: ImapConne
Some(job) => {
// Let the fetch run, but return back to the job afterwards.
jobs_loaded = 0;
info!(ctx, "postponing imap-job {} to run fetch...", job);
fetch(&ctx, &mut connection).await;
if ctx
.get_config_bool(Config::InboxWatch)
.await
.unwrap_or_default()
{
info!(ctx, "postponing imap-job {} to run fetch...", job);
fetch(&ctx, &mut connection).await;
}
}
None => {
jobs_loaded = 0;
// Expunge folder if needed, e.g. if some jobs have
// deleted messages on the server.
if let Err(err) = connection.maybe_close_folder(&ctx).await {
warn!(ctx, "failed to close folder: {:?}", err);
}
maybe_add_time_based_warnings(&ctx).await;
info = fetch_idle(&ctx, &mut connection, Config::ConfiguredInboxFolder).await;
info = if ctx
.get_config_bool(Config::InboxWatch)
.await
.unwrap_or_default()
{
fetch_idle(&ctx, &mut connection, Config::ConfiguredInboxFolder).await
} else {
if let Err(err) = connection.scan_folders(&ctx).await {
warn!(ctx, "{}", err);
connection.connectivity.set_err(&ctx, err).await;
} else {
connection.connectivity.set_not_configured(&ctx).await;
}
connection.fake_idle(&ctx, None).await
};
}
}
}
@@ -136,7 +157,7 @@ async fn fetch(ctx: &Context, connection: &mut Imap) {
}
// fetch
if let Err(err) = connection.fetch_move_delete(ctx, &watch_folder).await {
if let Err(err) = connection.fetch(ctx, &watch_folder).await {
connection.trigger_reconnect(ctx).await;
warn!(ctx, "{:#}", err);
}
@@ -162,57 +183,22 @@ async fn fetch_idle(ctx: &Context, connection: &mut Imap, folder: Config) -> Int
return connection.fake_idle(ctx, Some(watch_folder)).await;
}
// Mark expired messages for deletion.
if let Err(err) = delete_expired_imap_messages(ctx)
.await
.context("delete_expired_imap_messages failed")
{
warn!(ctx, "{:#}", err);
}
// Fetch the watched folder.
if let Err(err) = connection.fetch_move_delete(ctx, &watch_folder).await {
// fetch
if let Err(err) = connection.fetch(ctx, &watch_folder).await {
connection.trigger_reconnect(ctx).await;
warn!(ctx, "{:#}", err);
return InterruptInfo::new(false);
return InterruptInfo::new(false, None);
}
// Scan additional folders only after finishing fetching the watched folder.
//
// On iOS the application has strictly limited time to work in background, so we may not
// be able to scan all folders before time is up if there are many of them.
if folder == Config::ConfiguredInboxFolder {
// Only scan on the Inbox thread in order to prevent parallel scans, which might lead to duplicate messages
match connection.scan_folders(ctx).await {
Err(err) => {
// Don't reconnect, if there is a problem with the connection we will realize this when IDLEing
// but maybe just one folder can't be selected or something
warn!(ctx, "{}", err);
}
Ok(true) => {
// Fetch the watched folder again in case scanning other folder moved messages
// there.
//
// In most cases this will select the watched folder and return because there are
// no new messages. We want to select the watched folder anyway before going IDLE
// there, so this does not take additional protocol round-trip.
if let Err(err) = connection.fetch_move_delete(ctx, &watch_folder).await {
connection.trigger_reconnect(ctx).await;
warn!(ctx, "{:#}", err);
return InterruptInfo::new(false);
}
}
Ok(false) => {}
if let Err(err) = connection.scan_folders(ctx).await {
// Don't reconnect, if there is a problem with the connection we will realize this when IDLEing
// but maybe just one folder can't be selected or something
warn!(ctx, "{}", err);
}
}
// Synchronize Seen flags.
connection
.sync_seen_flags(ctx, &watch_folder)
.await
.context("sync_seen_flags")
.ok_or_log(ctx);
connection.connectivity.set_connected(ctx).await;
// idle
@@ -222,7 +208,7 @@ async fn fetch_idle(ctx: &Context, connection: &mut Imap, folder: Config) -> Int
Err(err) => {
connection.trigger_reconnect(ctx).await;
warn!(ctx, "{}", err);
InterruptInfo::new(false)
InterruptInfo::new(false, None)
}
}
} else {
@@ -307,25 +293,17 @@ async fn smtp_loop(ctx: Context, started: Sender<()>, smtp_handlers: SmtpConnect
let mut interrupt_info = Default::default();
loop {
let job = match job::load_next(&ctx, Thread::Smtp, &interrupt_info).await {
Err(err) => {
error!(ctx, "Failed loading job from the database: {:#}.", err);
None
}
Ok(job) => job,
};
match job {
match job::load_next(&ctx, Thread::Smtp, &interrupt_info)
.await
.ok()
.flatten()
{
Some(job) => {
info!(ctx, "executing smtp job");
job::perform_job(&ctx, job::Connection::Smtp(&mut connection), job).await;
interrupt_info = Default::default();
}
None => {
if let Err(err) = send_smtp_messages(&ctx, &mut connection).await {
warn!(ctx, "send_smtp_messages failed: {:#}", err);
}
// Fake Idle
info!(ctx, "smtp fake idle - started");
match &connection.last_send_error {
@@ -375,7 +353,7 @@ impl Scheduler {
}))
};
if ctx.should_watch_mvbox().await? {
if ctx.get_config_bool(Config::MvboxWatch).await? {
let ctx = ctx.clone();
mvbox_handle = Some(task::spawn(async move {
simple_imap_loop(
@@ -459,10 +437,10 @@ impl Scheduler {
return;
}
self.interrupt_inbox(InterruptInfo::new(true))
.join(self.interrupt_mvbox(InterruptInfo::new(true)))
.join(self.interrupt_sentbox(InterruptInfo::new(true)))
.join(self.interrupt_smtp(InterruptInfo::new(true)))
self.interrupt_inbox(InterruptInfo::new(true, None))
.join(self.interrupt_mvbox(InterruptInfo::new(true, None)))
.join(self.interrupt_sentbox(InterruptInfo::new(true, None)))
.join(self.interrupt_smtp(InterruptInfo::new(true, None)))
.await;
}
@@ -471,10 +449,10 @@ impl Scheduler {
return;
}
self.interrupt_inbox(InterruptInfo::new(false))
.join(self.interrupt_mvbox(InterruptInfo::new(false)))
.join(self.interrupt_sentbox(InterruptInfo::new(false)))
.join(self.interrupt_smtp(InterruptInfo::new(false)))
self.interrupt_inbox(InterruptInfo::new(false, None))
.join(self.interrupt_mvbox(InterruptInfo::new(false, None)))
.join(self.interrupt_sentbox(InterruptInfo::new(false, None)))
.join(self.interrupt_smtp(InterruptInfo::new(false, None)))
.await;
}
@@ -704,10 +682,14 @@ struct ImapConnectionHandlers {
#[derive(Default, Debug)]
pub struct InterruptInfo {
pub probe_network: bool,
pub msg_id: Option<MsgId>,
}
impl InterruptInfo {
pub fn new(probe_network: bool) -> Self {
Self { probe_network }
pub fn new(probe_network: bool, msg_id: Option<MsgId>) -> Self {
Self {
probe_network,
msg_id,
}
}
}

View File

@@ -5,7 +5,6 @@ use async_std::sync::{Mutex, RwLockReadGuard};
use crate::dc_tools::time;
use crate::events::EventType;
use crate::imap::scan_folders::get_watched_folder_configs;
use crate::quota::{
QUOTA_ERROR_THRESHOLD_PERCENTAGE, QUOTA_MAX_AGE_SECONDS, QUOTA_WARN_THRESHOLD_PERCENTAGE,
};
@@ -363,14 +362,17 @@ impl Context {
[
(
Config::ConfiguredInboxFolder,
Config::InboxWatch,
inbox.state.connectivity.clone(),
),
(
Config::ConfiguredMvboxFolder,
Config::MvboxWatch,
mvbox.state.connectivity.clone(),
),
(
Config::ConfiguredSentboxFolder,
Config::SentboxWatch,
sentbox.state.connectivity.clone(),
),
],
@@ -389,12 +391,12 @@ impl Context {
// - "Sent": Connected
// =============================================================================================
let watched_folders = get_watched_folder_configs(self).await?;
ret += &format!("<h3>{}</h3><ul>", stock_str::incoming_messages(self).await);
for (folder, state) in &folders_states {
let mut folder_added = false;
for (folder, watch, state) in &folders_states {
let w = self.get_config(*watch).await.ok_or_log(self);
if watched_folders.contains(folder) {
let mut folder_added = false;
if w.flatten() == Some("1".to_string()) {
let f = self.get_config(*folder).await.ok_or_log(self).flatten();
if let Some(foldername) = f {

View File

@@ -3,20 +3,21 @@
use std::convert::TryFrom;
use anyhow::{bail, Context as _, Error, Result};
use async_std::sync::Mutex;
use percent_encoding::{utf8_percent_encode, AsciiSet, NON_ALPHANUMERIC};
use crate::aheader::EncryptPreference;
use crate::chat::{self, Chat, ChatId, ChatIdBlocked};
use crate::chat::{self, is_contact_in_chat, Chat, ChatId, ChatIdBlocked, ProtectionStatus};
use crate::config::Config;
use crate::constants::{Blocked, DC_CONTACT_ID_LAST_SPECIAL};
use crate::contact::{Contact, ContactId, Origin, VerifiedStatus};
use crate::constants::{Blocked, Chattype, Viewtype, DC_CONTACT_ID_LAST_SPECIAL};
use crate::contact::{Contact, Origin, VerifiedStatus};
use crate::context::Context;
use crate::dc_tools::time;
use crate::e2ee::ensure_secret_key_exists;
use crate::events::EventType;
use crate::headerdef::HeaderDef;
use crate::key::{DcKey, Fingerprint, SignedPublicKey};
use crate::message::{Message, Viewtype};
use crate::message::Message;
use crate::mimeparser::{MimeMessage, SystemMessage};
use crate::param::Param;
use crate::peerstate::{Peerstate, PeerstateKeyType, PeerstateVerifiedStatus, ToSave};
@@ -24,16 +25,28 @@ use crate::qr::check_qr;
use crate::stock_str;
use crate::token;
mod bob;
mod bobstate;
mod qrinvite;
use crate::token::Namespace;
use bobstate::BobState;
use bobstate::{BobHandshakeStage, BobState, BobStateHandle};
use qrinvite::QrInvite;
pub const NON_ALPHANUMERIC_WITHOUT_DOT: &AsciiSet = &NON_ALPHANUMERIC.remove(b'.');
macro_rules! joiner_progress {
($context:tt, $contact_id:expr, $progress:expr) => {
assert!(
$progress >= 0 && $progress <= 1000,
"value in range 0..1000 expected with: 0=error, 1..999=progress, 1000=success"
);
$context.emit_event($crate::events::EventType::SecurejoinJoinerProgress {
contact_id: $contact_id,
progress: $progress,
});
};
}
macro_rules! inviter_progress {
($context:tt, $contact_id:expr, $progress:expr) => {
assert!(
@@ -47,6 +60,100 @@ macro_rules! inviter_progress {
};
}
/// State for setup-contact/secure-join protocol joiner's side, aka Bob's side.
///
/// The setup-contact protocol needs to carry state for both the inviter (Alice) and the
/// joiner/invitee (Bob). For Alice this state is minimal and in the `tokens` table in the
/// database. For Bob this state is only carried live on the [`Context`] in this struct.
#[derive(Debug, Default)]
pub(crate) struct Bob {
inner: Mutex<Option<BobState>>,
}
/// Return value for [`Bob::start_protocol`].
///
/// This indicates which protocol variant was started and provides the required information
/// about it.
enum StartedProtocolVariant {
/// The setup-contact protocol, to verify a contact.
SetupContact,
/// The secure-join protocol, to join a group.
SecureJoin {
group_id: String,
group_name: String,
},
}
impl Bob {
/// Starts the securejoin protocol with the QR `invite`.
///
/// This will try to start the securejoin protocol for the given QR `invite`. If it
/// succeeded the protocol state will be tracked in `self`.
///
/// This function takes care of starting the "ongoing" mechanism if required and
/// handling errors while starting the protocol.
///
/// # Returns
///
/// If the started protocol is joining a group the returned struct contains information
/// about the group and ongoing process.
async fn start_protocol(
&self,
context: &Context,
invite: QrInvite,
) -> Result<StartedProtocolVariant, JoinError> {
let mut guard = self.inner.lock().await;
if guard.is_some() {
warn!(context, "The new securejoin will replace the ongoing one.");
*guard = None;
}
let variant = match invite {
QrInvite::Group {
ref grpid,
ref name,
..
} => StartedProtocolVariant::SecureJoin {
group_id: grpid.clone(),
group_name: name.clone(),
},
_ => StartedProtocolVariant::SetupContact,
};
match BobState::start_protocol(context, invite).await {
Ok((state, stage)) => {
if matches!(stage, BobHandshakeStage::RequestWithAuthSent) {
joiner_progress!(context, state.invite().contact_id(), 400);
}
*guard = Some(state);
Ok(variant)
}
Err(err) => {
if let StartedProtocolVariant::SecureJoin { .. } = variant {
context.free_ongoing().await;
}
Err(err)
}
}
}
/// Returns a handle to the [`BobState`] of the handshake.
///
/// If there currently isn't a handshake running this will return `None`. Otherwise
/// this will return a handle to the current [`BobState`]. This handle allows
/// processing an incoming message and allows terminating the handshake.
///
/// The handle contains an exclusive lock, which is held until the handle is dropped.
/// This guarantees all state and state changes are correct and allows safely
/// terminating the handshake without worrying about concurrency.
async fn state(&self, context: &Context) -> Option<BobStateHandle<'_>> {
let guard = self.inner.lock().await;
let ret = BobStateHandle::from_guard(guard);
if ret.is_none() {
info!(context, "No active BobState found for securejoin handshake");
}
ret
}
}
/// Generates a Secure Join QR code.
///
/// With `group` set to `None` this generates a setup-contact QR code, with `group` set to a
@@ -173,7 +280,7 @@ pub enum JoinError {
pub async fn dc_join_securejoin(context: &Context, qr: &str) -> Result<ChatId, JoinError> {
securejoin(context, qr).await.map_err(|err| {
warn!(context, "Fatal joiner error: {:#}", err);
// The user just scanned this QR code so has context on what failed.
// This is a modal operation, the user has context on what failed.
error!(context, "QR process failed");
err
})
@@ -190,7 +297,46 @@ async fn securejoin(context: &Context, qr: &str) -> Result<ChatId, JoinError> {
let invite = QrInvite::try_from(qr_scan)?;
bob::start_protocol(context, invite).await
match context.bob.start_protocol(context, invite.clone()).await? {
StartedProtocolVariant::SetupContact => {
// for a one-to-one-chat, the chat is already known, return the chat-id,
// the verification runs in background
let chat_id = ChatId::create_for_contact(context, invite.contact_id())
.await
.map_err(JoinError::UnknownContact)?;
Ok(chat_id)
}
StartedProtocolVariant::SecureJoin {
group_id,
group_name,
} => {
// for a group-join, also create the chat soon and let the verification run in background.
// however, the group will become usable only when the protocol has finished.
let contact_id = invite.contact_id();
let chat_id = if let Some((chat_id, _protected, _blocked)) =
chat::get_chat_id_by_grpid(context, &group_id).await?
{
chat_id.unblock(context).await?;
chat_id
} else {
ChatId::create_multiuser_record(
context,
Chattype::Group,
group_id,
group_name,
Blocked::Not,
ProtectionStatus::Unprotected, // protection is added later as needed
)
.await?
};
if !is_contact_in_chat(context, chat_id, contact_id).await? {
chat::add_to_chat_contacts_table(context, chat_id, contact_id).await?;
}
let msg = stock_str::secure_join_started(context, contact_id).await;
chat::add_info_msg(context, chat_id, msg, time()).await?;
Ok(chat_id)
}
}
}
/// Error when failing to send a protocol handshake message.
@@ -205,7 +351,7 @@ pub struct SendMsgError(#[from] anyhow::Error);
/// Bob's handshake messages are sent in `BobState::send_handshake_message()`.
async fn send_alice_handshake_msg(
context: &Context,
contact_id: ContactId,
contact_id: u32,
step: &str,
fingerprint: Option<Fingerprint>,
) -> Result<(), SendMsgError> {
@@ -233,7 +379,7 @@ async fn send_alice_handshake_msg(
}
/// Get an unblocked chat that can be used for info messages.
async fn info_chat_id(context: &Context, contact_id: ContactId) -> Result<ChatId> {
async fn info_chat_id(context: &Context, contact_id: u32) -> Result<ChatId> {
let chat_id_blocked = ChatIdBlocked::get_for_contact(context, contact_id, Blocked::Not).await?;
Ok(chat_id_blocked.id)
}
@@ -241,7 +387,7 @@ async fn info_chat_id(context: &Context, contact_id: ContactId) -> Result<ChatId
async fn fingerprint_equals_sender(
context: &Context,
fingerprint: &Fingerprint,
contact_id: ContactId,
contact_id: u32,
) -> Result<bool, Error> {
let contact = Contact::load_from_db(context, contact_id).await?;
let peerstate = match Peerstate::from_addr(context, contact.get_addr()).await {
@@ -295,8 +441,9 @@ pub(crate) enum HandshakeMessage {
/// Handle incoming secure-join handshake.
///
/// This function will update the securejoin state in the database as the protocol
/// progresses.
/// This function will update the securejoin state in [`InnerContext::bob`] and also
/// terminate the ongoing process using [`Context::stop_ongoing`] as required by the
/// protocol.
///
/// A message which results in [`Err`] will be hidden from the user but not deleted, it may
/// be a valid message for something else we are not aware off. E.g. it could be part of a
@@ -304,11 +451,13 @@ pub(crate) enum HandshakeMessage {
///
/// When `handle_securejoin_handshake()` is called, the message is not yet filed in the
/// database; this is done by `receive_imf()` later on as needed.
///
/// [`InnerContext::bob`]: crate::context::InnerContext::bob
#[allow(clippy::indexing_slicing)]
pub(crate) async fn handle_securejoin_handshake(
context: &Context,
mime_message: &MimeMessage,
contact_id: ContactId,
contact_id: u32,
) -> Result<HandshakeMessage> {
if contact_id <= DC_CONTACT_ID_LAST_SPECIAL {
return Err(Error::msg("Can not be called with special contact ID"));
@@ -371,7 +520,38 @@ pub(crate) async fn handle_securejoin_handshake(
==== Bob - the joiner's side =====
==== Step 4 in "Setup verified contact" protocol =====
========================================================*/
bob::handle_auth_required(context, mime_message).await
match context.bob.state(context).await {
Some(mut bobstate) => match bobstate.handle_message(context, mime_message).await {
Some(BobHandshakeStage::Terminated(why)) => {
could_not_establish_secure_connection(
context,
contact_id,
bobstate.chat_id(context).await?,
why,
)
.await?;
Ok(HandshakeMessage::Done)
}
Some(_stage) => {
if join_vg {
// the message reads "Alice replied, waiting for being added to the group…";
// show it only on secure-join and not on setup-contact therefore.
let msg = stock_str::secure_join_replies(context, contact_id).await;
chat::add_info_msg(
context,
bobstate.chat_id(context).await?,
msg,
time(),
)
.await?;
}
joiner_progress!(context, bobstate.invite().contact_id(), 400);
Ok(HandshakeMessage::Done)
}
None => Ok(HandshakeMessage::Ignore),
},
None => Ok(HandshakeMessage::Ignore),
}
}
"vg-request-with-auth" | "vc-request-with-auth" => {
/*==========================================================
@@ -502,14 +682,44 @@ pub(crate) async fn handle_securejoin_handshake(
==== Bob - the joiner's side ====
==== Step 7 in "Setup verified contact" protocol ====
=======================================================*/
match BobState::from_db(&context.sql).await? {
Some(bobstate) => {
bob::handle_contact_confirm(context, bobstate, mime_message).await
}
None => match join_vg {
true => Ok(HandshakeMessage::Propagate),
false => Ok(HandshakeMessage::Ignore),
info!(context, "matched vc-contact-confirm step");
let retval = if join_vg {
HandshakeMessage::Propagate
} else {
HandshakeMessage::Ignore
};
match context.bob.state(context).await {
Some(mut bobstate) => match bobstate.handle_message(context, mime_message).await {
Some(BobHandshakeStage::Terminated(why)) => {
could_not_establish_secure_connection(
context,
contact_id,
bobstate.chat_id(context).await?,
why,
)
.await?;
Ok(HandshakeMessage::Done)
}
Some(BobHandshakeStage::Completed) => {
// Can only be BobHandshakeStage::Completed
secure_connection_established(
context,
contact_id,
bobstate.chat_id(context).await?,
)
.await?;
Ok(retval)
}
Some(_) => {
warn!(
context,
"Impossible state returned from handling handshake message"
);
Ok(retval)
}
None => Ok(retval),
},
None => Ok(retval),
}
}
"vg-member-added-received" | "vc-contact-confirm-received" => {
@@ -531,7 +741,7 @@ pub(crate) async fn handle_securejoin_handshake(
.get_header(HeaderDef::SecureJoinGroup)
.map(|s| s.as_str())
.unwrap_or_else(|| "");
if let Err(err) = chat::get_chat_id_by_grpid(context, field_grpid).await {
if let Err(err) = chat::get_chat_id_by_grpid(context, &field_grpid).await {
warn!(context, "Failed to lookup chat_id from grpid: {}", err);
return Err(
err.context(format!("Chat for group {} not found", &field_grpid))
@@ -571,7 +781,7 @@ pub(crate) async fn handle_securejoin_handshake(
pub(crate) async fn observe_securejoin_on_other_device(
context: &Context,
mime_message: &MimeMessage,
contact_id: ContactId,
contact_id: u32,
) -> Result<HandshakeMessage> {
if contact_id <= DC_CONTACT_ID_LAST_SPECIAL {
return Err(Error::msg("Can not be called with special contact ID"));
@@ -636,24 +846,32 @@ pub(crate) async fn observe_securejoin_on_other_device(
async fn secure_connection_established(
context: &Context,
contact_id: ContactId,
contact_id: u32,
chat_id: ChatId,
) -> Result<(), Error> {
let contact = Contact::get_by_id(context, contact_id).await?;
let msg = stock_str::contact_verified(context, &contact).await;
chat::add_info_msg(context, chat_id, &msg, time()).await?;
let msg = stock_str::contact_verified(context, contact.get_name_n_addr()).await;
chat::add_info_msg(context, chat_id, msg, time()).await?;
context.emit_event(EventType::ChatModified(chat_id));
Ok(())
}
async fn could_not_establish_secure_connection(
context: &Context,
contact_id: ContactId,
contact_id: u32,
chat_id: ChatId,
details: &str,
) -> Result<(), Error> {
let contact = Contact::get_by_id(context, contact_id).await?;
let msg = stock_str::contact_not_verified(context, &contact).await;
let contact = Contact::get_by_id(context, contact_id).await;
let msg = stock_str::contact_not_verified(
context,
if let Ok(ref contact) = contact {
contact.get_addr()
} else {
"?"
},
)
.await;
chat::add_info_msg(context, chat_id, &msg, time()).await?;
warn!(
context,
@@ -718,43 +936,46 @@ fn encrypted_and_signed(
mod tests {
use super::*;
use async_std::prelude::*;
use crate::chat;
use crate::chat::ProtectionStatus;
use crate::chatlist::Chatlist;
use crate::constants::Chattype;
use crate::events::Event;
use crate::peerstate::Peerstate;
use crate::test_utils::{TestContext, TestContextManager};
use crate::test_utils::TestContext;
use std::time::Duration;
#[async_std::test]
async fn test_setup_contact() {
let mut tcm = TestContextManager::new().await;
let alice = tcm.alice().await;
let bob = tcm.bob().await;
assert_eq!(
Chatlist::try_load(&alice, 0, None, None)
.await
.unwrap()
.len(),
0
);
assert_eq!(
Chatlist::try_load(&bob, 0, None, None).await.unwrap().len(),
0
);
async fn test_setup_contact() -> Result<()> {
let alice = TestContext::new_alice().await;
let bob = TestContext::new_bob().await;
assert_eq!(Chatlist::try_load(&alice, 0, None, None).await?.len(), 0);
assert_eq!(Chatlist::try_load(&bob, 0, None, None).await?.len(), 0);
// Setup JoinerProgress sinks.
let (joiner_progress_tx, joiner_progress_rx) = async_std::channel::bounded(100);
bob.add_event_sink(move |event: Event| {
let joiner_progress_tx = joiner_progress_tx.clone();
async move {
if let EventType::SecurejoinJoinerProgress { .. } = event.typ {
joiner_progress_tx.try_send(event).unwrap();
}
}
})
.await;
// Step 1: Generate QR-code, ChatId(0) indicates setup-contact
let qr = dc_get_securejoin_qr(&alice.ctx, None).await.unwrap();
let qr = dc_get_securejoin_qr(&alice.ctx, None).await?;
// Step 2: Bob scans QR-code, sends vc-request
dc_join_securejoin(&bob.ctx, &qr).await.unwrap();
assert_eq!(
Chatlist::try_load(&bob, 0, None, None).await.unwrap().len(),
1
);
dc_join_securejoin(&bob.ctx, &qr).await?;
assert_eq!(Chatlist::try_load(&bob, 0, None, None).await?.len(), 1);
let sent = bob.pop_sent_msg().await;
assert!(!bob.ctx.has_ongoing().await);
assert_eq!(sent.recipient(), "alice@example.org".parse().unwrap());
assert_eq!(sent.recipient(), "alice@example.com".parse().unwrap());
let msg = alice.parse_msg(&sent).await;
assert!(!msg.was_encrypted());
assert_eq!(msg.get_header(HeaderDef::SecureJoin).unwrap(), "vc-request");
@@ -762,13 +983,7 @@ mod tests {
// Step 3: Alice receives vc-request, sends vc-auth-required
alice.recv_msg(&sent).await;
assert_eq!(
Chatlist::try_load(&alice, 0, None, None)
.await
.unwrap()
.len(),
1
);
assert_eq!(Chatlist::try_load(&alice, 0, None, None).await?.len(), 1);
let sent = alice.pop_sent_msg().await;
let msg = bob.parse_msg(&sent).await;
@@ -782,24 +997,28 @@ mod tests {
bob.recv_msg(&sent).await;
// Check Bob emitted the JoinerProgress event.
let event = bob
.evtracker
.get_matching(|evt| matches!(evt, EventType::SecurejoinJoinerProgress { .. }))
.await;
match event {
EventType::SecurejoinJoinerProgress {
contact_id,
progress,
} => {
let alice_contact_id =
Contact::lookup_id_by_addr(&bob.ctx, "alice@example.org", Origin::Unknown)
.await
.expect("Error looking up contact")
.expect("Contact not found");
assert_eq!(contact_id, alice_contact_id);
assert_eq!(progress, 400);
{
let evt = joiner_progress_rx
.recv()
.timeout(Duration::from_secs(10))
.await
.expect("timeout waiting for JoinerProgress event")
.expect("missing JoinerProgress event");
match evt.typ {
EventType::SecurejoinJoinerProgress {
contact_id,
progress,
} => {
let alice_contact_id =
Contact::lookup_id_by_addr(&bob.ctx, "alice@example.com", Origin::Unknown)
.await
.expect("Error looking up contact")
.expect("Contact not found");
assert_eq!(contact_id, alice_contact_id);
assert_eq!(progress, 400);
}
_ => panic!("Wrong event type"),
}
_ => unreachable!(),
}
// Check Bob sent the right message.
@@ -830,30 +1049,21 @@ mod tests {
.await
.unwrap();
assert_eq!(
contact_bob.is_verified(&alice.ctx).await.unwrap(),
contact_bob.is_verified(&alice.ctx).await?,
VerifiedStatus::Unverified
);
// Step 5+6: Alice receives vc-request-with-auth, sends vc-contact-confirm
alice.recv_msg(&sent).await;
assert_eq!(
contact_bob.is_verified(&alice.ctx).await.unwrap(),
contact_bob.is_verified(&alice.ctx).await?,
VerifiedStatus::BidirectVerified
);
// exactly one one-to-one chat should be visible for both now
// (check this before calling alice.create_chat() explicitly below)
assert_eq!(
Chatlist::try_load(&alice, 0, None, None)
.await
.unwrap()
.len(),
1
);
assert_eq!(
Chatlist::try_load(&bob, 0, None, None).await.unwrap().len(),
1
);
assert_eq!(Chatlist::try_load(&alice, 0, None, None).await?.len(), 1);
assert_eq!(Chatlist::try_load(&bob, 0, None, None).await?.len(), 1);
// Check Alice got the verified message in her 1:1 chat.
{
@@ -885,7 +1095,7 @@ mod tests {
// Bob should not yet have Alice verified
let contact_alice_id =
Contact::lookup_id_by_addr(&bob.ctx, "alice@example.org", Origin::Unknown)
Contact::lookup_id_by_addr(&bob.ctx, "alice@example.com", Origin::Unknown)
.await
.expect("Error looking up contact")
.expect("Contact not found");
@@ -893,14 +1103,14 @@ mod tests {
.await
.unwrap();
assert_eq!(
contact_bob.is_verified(&bob.ctx).await.unwrap(),
contact_bob.is_verified(&bob.ctx).await?,
VerifiedStatus::Unverified
);
// Step 7: Bob receives vc-contact-confirm, sends vc-contact-confirm-received
bob.recv_msg(&sent).await;
assert_eq!(
contact_alice.is_verified(&bob.ctx).await.unwrap(),
contact_alice.is_verified(&bob.ctx).await?,
VerifiedStatus::BidirectVerified
);
@@ -920,7 +1130,7 @@ mod tests {
let msg = Message::load_from_db(&bob.ctx, msg_id).await.unwrap();
assert!(msg.is_info());
let text = msg.get_text().unwrap();
assert!(text.contains("alice@example.org verified"));
assert!(text.contains("alice@example.com verified"));
}
// Check Bob sent the final message
@@ -931,6 +1141,7 @@ mod tests {
msg.get_header(HeaderDef::SecureJoin).unwrap(),
"vc-contact-confirm-received"
);
Ok(())
}
#[async_std::test]
@@ -942,14 +1153,25 @@ mod tests {
#[async_std::test]
async fn test_setup_contact_bob_knows_alice() -> Result<()> {
let mut tcm = TestContextManager::new().await;
let alice = tcm.alice().await;
let bob = tcm.bob().await;
let alice = TestContext::new_alice().await;
let bob = TestContext::new_bob().await;
// Setup JoinerProgress sinks.
let (joiner_progress_tx, joiner_progress_rx) = async_std::channel::bounded(100);
bob.add_event_sink(move |event: Event| {
let joiner_progress_tx = joiner_progress_tx.clone();
async move {
if let EventType::SecurejoinJoinerProgress { .. } = event.typ {
joiner_progress_tx.try_send(event).unwrap();
}
}
})
.await;
// Ensure Bob knows Alice_FP
let alice_pubkey = SignedPublicKey::load_self(&alice.ctx).await?;
let peerstate = Peerstate {
addr: "alice@example.org".into(),
addr: "alice@example.com".into(),
last_seen: 10,
last_seen_autocrypt: 10,
prefer_encrypt: EncryptPreference::Mutual,
@@ -972,25 +1194,30 @@ mod tests {
dc_join_securejoin(&bob.ctx, &qr).await.unwrap();
// Check Bob emitted the JoinerProgress event.
let event = bob
.evtracker
.get_matching(|evt| matches!(evt, EventType::SecurejoinJoinerProgress { .. }))
.await;
match event {
EventType::SecurejoinJoinerProgress {
contact_id,
progress,
} => {
let alice_contact_id =
Contact::lookup_id_by_addr(&bob.ctx, "alice@example.org", Origin::Unknown)
.await
.expect("Error looking up contact")
.expect("Contact not found");
assert_eq!(contact_id, alice_contact_id);
assert_eq!(progress, 400);
{
let evt = joiner_progress_rx
.recv()
.timeout(Duration::from_secs(10))
.await
.expect("timeout waiting for JoinerProgress event")
.expect("missing JoinerProgress event");
match evt.typ {
EventType::SecurejoinJoinerProgress {
contact_id,
progress,
} => {
let alice_contact_id =
Contact::lookup_id_by_addr(&bob.ctx, "alice@example.com", Origin::Unknown)
.await
.expect("Error looking up contact")
.expect("Contact not found");
assert_eq!(contact_id, alice_contact_id);
assert_eq!(progress, 400);
}
_ => panic!("Wrong event type"),
}
_ => unreachable!(),
}
assert!(!bob.ctx.has_ongoing().await);
// Check Bob sent the right handshake message.
let sent = bob.pop_sent_msg().await;
@@ -1038,7 +1265,7 @@ mod tests {
// Bob should not yet have Alice verified
let contact_alice_id =
Contact::lookup_id_by_addr(&bob.ctx, "alice@example.org", Origin::Unknown)
Contact::lookup_id_by_addr(&bob.ctx, "alice@example.com", Origin::Unknown)
.await
.expect("Error looking up contact")
.expect("Contact not found");
@@ -1067,9 +1294,8 @@ mod tests {
#[async_std::test]
async fn test_setup_contact_concurrent_calls() -> Result<()> {
let mut tcm = TestContextManager::new().await;
let alice = tcm.alice().await;
let bob = tcm.bob().await;
let alice = TestContext::new_alice().await;
let bob = TestContext::new_bob().await;
// do a scan that is not working as claire is never responding
let qr_stale = "OPENPGP4FPR:1234567890123456789012345678901234567890#a=claire%40foo.de&n=&i=12345678901&s=23456789012";
@@ -1091,26 +1317,35 @@ mod tests {
.pop_sent_msg()
.await
.payload()
.contains("alice@example.org"));
.contains("alice@example.com"));
Ok(())
}
#[async_std::test]
async fn test_secure_join() -> Result<()> {
let mut tcm = TestContextManager::new().await;
let alice = tcm.alice().await;
let bob = tcm.bob().await;
// We start with empty chatlists.
let alice = TestContext::new_alice().await;
let bob = TestContext::new_bob().await;
assert_eq!(Chatlist::try_load(&alice, 0, None, None).await?.len(), 0);
assert_eq!(Chatlist::try_load(&bob, 0, None, None).await?.len(), 0);
let alice_chatid =
// Setup JoinerProgress sinks.
let (joiner_progress_tx, joiner_progress_rx) = async_std::channel::bounded(100);
bob.add_event_sink(move |event: Event| {
let joiner_progress_tx = joiner_progress_tx.clone();
async move {
if let EventType::SecurejoinJoinerProgress { .. } = event.typ {
joiner_progress_tx.try_send(event).unwrap();
}
}
})
.await;
let chatid =
chat::create_group_chat(&alice.ctx, ProtectionStatus::Protected, "the chat").await?;
// Step 1: Generate QR-code, secure-join implied by chatid
let qr = dc_get_securejoin_qr(&alice.ctx, Some(alice_chatid))
let qr = dc_get_securejoin_qr(&alice.ctx, Some(chatid))
.await
.unwrap();
@@ -1119,7 +1354,7 @@ mod tests {
assert_eq!(Chatlist::try_load(&bob, 0, None, None).await?.len(), 1);
let sent = bob.pop_sent_msg().await;
assert_eq!(sent.recipient(), "alice@example.org".parse().unwrap());
assert_eq!(sent.recipient(), "alice@example.com".parse().unwrap());
let msg = alice.parse_msg(&sent).await;
assert!(!msg.was_encrypted());
assert_eq!(msg.get_header(HeaderDef::SecureJoin).unwrap(), "vg-request");
@@ -1141,24 +1376,28 @@ mod tests {
let sent = bob.pop_sent_msg().await;
// Check Bob emitted the JoinerProgress event.
let event = bob
.evtracker
.get_matching(|evt| matches!(evt, EventType::SecurejoinJoinerProgress { .. }))
.await;
match event {
EventType::SecurejoinJoinerProgress {
contact_id,
progress,
} => {
let alice_contact_id =
Contact::lookup_id_by_addr(&bob.ctx, "alice@example.org", Origin::Unknown)
.await
.expect("Error looking up contact")
.expect("Contact not found");
assert_eq!(contact_id, alice_contact_id);
assert_eq!(progress, 400);
{
let evt = joiner_progress_rx
.recv()
.timeout(Duration::from_secs(10))
.await
.expect("timeout waiting for JoinerProgress event")
.expect("missing JoinerProgress event");
match evt.typ {
EventType::SecurejoinJoinerProgress {
contact_id,
progress,
} => {
let alice_contact_id =
Contact::lookup_id_by_addr(&bob.ctx, "alice@example.com", Origin::Unknown)
.await
.expect("Error looking up contact")
.expect("Contact not found");
assert_eq!(contact_id, alice_contact_id);
assert_eq!(progress, 400);
}
_ => panic!("Wrong event type"),
}
_ => unreachable!(),
}
// Check Bob sent the right handshake message.
@@ -1201,38 +1440,9 @@ mod tests {
"vg-member-added"
);
{
// Now Alice's chat with Bob should still be hidden, the verified message should
// appear in the group chat.
let chat = alice
.get_chat(&bob)
.await
.expect("Alice has no 1:1 chat with bob");
assert_eq!(
chat.blocked,
Blocked::Yes,
"Alice's 1:1 chat with Bob is not hidden"
);
let msg_id = chat::get_chat_msgs(&alice.ctx, alice_chatid, 0x1, None)
.await
.unwrap()
.into_iter()
.filter_map(|item| match item {
chat::ChatItem::Message { msg_id } => Some(msg_id),
_ => None,
})
.min()
.expect("No messages in Alice's group chat");
let msg = Message::load_from_db(&alice.ctx, msg_id).await.unwrap();
assert!(msg.is_info());
let text = msg.get_text().unwrap();
assert!(text.contains("bob@example.net verified"));
}
// Bob should not yet have Alice verified
let contact_alice_id =
Contact::lookup_id_by_addr(&bob.ctx, "alice@example.org", Origin::Unknown)
Contact::lookup_id_by_addr(&bob.ctx, "alice@example.com", Origin::Unknown)
.await
.expect("Error looking up contact")
.expect("Contact not found");
@@ -1244,53 +1454,10 @@ mod tests {
// Step 7: Bob receives vg-member-added, sends vg-member-added-received
bob.recv_msg(&sent).await;
{
// Bob has Alice verified, message shows up in the group chat.
assert_eq!(
contact_alice.is_verified(&bob.ctx).await?,
VerifiedStatus::BidirectVerified
);
let chat = bob
.get_chat(&alice)
.await
.expect("Bob has no 1:1 chat with Alice");
assert_eq!(
chat.blocked,
Blocked::Yes,
"Bob's 1:1 chat with Alice is not hidden"
);
for item in chat::get_chat_msgs(&bob.ctx, bob_chatid, 0x1, None)
.await
.unwrap()
{
if let chat::ChatItem::Message { msg_id } = item {
let msg = Message::load_from_db(&bob.ctx, msg_id).await.unwrap();
let text = msg.get_text().unwrap();
println!("msg {} text: {}", msg_id, text);
}
}
let mut msg_iter = chat::get_chat_msgs(&bob.ctx, bob_chatid, 0x1, None)
.await
.unwrap()
.into_iter();
loop {
match msg_iter.next() {
Some(chat::ChatItem::Message { msg_id }) => {
let msg = Message::load_from_db(&bob.ctx, msg_id).await.unwrap();
let text = msg.get_text().unwrap();
match text.contains("alice@example.org verified") {
true => {
assert!(msg.is_info());
break;
}
false => continue,
}
}
Some(_) => continue,
None => panic!("Verified message not found in Bob's group chat"),
}
}
}
assert_eq!(
contact_alice.is_verified(&bob.ctx).await?,
VerifiedStatus::BidirectVerified
);
let sent = bob.pop_sent_msg().await;
let msg = alice.parse_msg(&sent).await;

View File

@@ -1,257 +0,0 @@
//! Bob's side of SecureJoin handling.
//!
//! This are some helper functions around [`BobState`] which augment the state changes with
//! the required user interactions.
use anyhow::Result;
use crate::chat::{is_contact_in_chat, ChatId, ProtectionStatus};
use crate::constants::{Blocked, Chattype};
use crate::contact::Contact;
use crate::context::Context;
use crate::dc_tools::time;
use crate::events::EventType;
use crate::mimeparser::MimeMessage;
use crate::{chat, stock_str};
use super::bobstate::{BobHandshakeStage, BobState};
use super::qrinvite::QrInvite;
use super::{HandshakeMessage, JoinError};
/// Starts the securejoin protocol with the QR `invite`.
///
/// This will try to start the securejoin protocol for the given QR `invite`. If it
/// succeeded the protocol state will be tracked in `self`.
///
/// This function takes care of handling multiple concurrent joins and handling errors while
/// starting the protocol.
///
/// # Returns
///
/// The [`ChatId`] of the created chat is returned, for a SetupContact QR this is the 1:1
/// chat with Alice, for a SecureJoin QR this is the group chat.
pub(super) async fn start_protocol(
context: &Context,
invite: QrInvite,
) -> Result<ChatId, JoinError> {
// A 1:1 chat is needed to send messages to Alice. When joining a group this chat is
// hidden, if a user starts sending messages in it it will be unhidden in
// dc_receive_imf.
let hidden = match invite {
QrInvite::Contact { .. } => Blocked::Not,
QrInvite::Group { .. } => Blocked::Yes,
};
let chat_id = ChatId::create_for_contact_with_blocked(context, invite.contact_id(), hidden)
.await
.map_err(JoinError::UnknownContact)?;
// Now start the protocol and initialise the state
let (state, stage, aborted_states) =
BobState::start_protocol(context, invite.clone(), chat_id).await?;
for state in aborted_states {
error!(context, "Aborting previously unfinished QR Join process.");
state.notify_aborted(context, "new QR scanned").await?;
state.emit_progress(context, JoinerProgress::Error);
}
if matches!(stage, BobHandshakeStage::RequestWithAuthSent) {
state.emit_progress(context, JoinerProgress::RequestWithAuthSent);
}
match invite {
QrInvite::Group { .. } => {
// For a secure-join we need to create the group and add the contact. The group will
// only become usable once the protocol is finished.
// TODO: how does this group become usable?
let group_chat_id = state.joining_chat_id(context).await?;
if !is_contact_in_chat(context, group_chat_id, invite.contact_id()).await? {
chat::add_to_chat_contacts_table(context, group_chat_id, invite.contact_id())
.await?;
}
let msg = stock_str::secure_join_started(context, invite.contact_id()).await;
chat::add_info_msg(context, group_chat_id, &msg, time()).await?;
Ok(group_chat_id)
}
QrInvite::Contact { .. } => {
// For setup-contact the BobState already ensured the 1:1 chat exists because it
// uses it to send the handshake messages.
Ok(state.alice_chat())
}
}
}
/// Handles `vc-auth-required` and `vg-auth-required` handshake messages.
///
/// # Bob - the joiner's side
/// ## Step 4 in the "Setup Contact protocol"
pub(super) async fn handle_auth_required(
context: &Context,
message: &MimeMessage,
) -> Result<HandshakeMessage> {
match BobState::from_db(&context.sql).await? {
Some(mut bobstate) => match bobstate.handle_message(context, message).await? {
Some(BobHandshakeStage::Terminated(why)) => {
bobstate.notify_aborted(context, why).await?;
Ok(HandshakeMessage::Done)
}
Some(_stage) => {
if bobstate.is_join_group() {
// The message reads "Alice replied, waiting to be added to the group…",
// so only show it on secure-join and not on setup-contact.
let contact_id = bobstate.invite().contact_id();
let msg = stock_str::secure_join_replies(context, contact_id).await;
let chat_id = bobstate.joining_chat_id(context).await?;
chat::add_info_msg(context, chat_id, &msg, time()).await?;
}
bobstate.emit_progress(context, JoinerProgress::RequestWithAuthSent);
Ok(HandshakeMessage::Done)
}
None => Ok(HandshakeMessage::Ignore),
},
None => Ok(HandshakeMessage::Ignore),
}
}
/// Handles `vc-contact-confirm` and `vg-member-added` handshake messages.
///
/// # Bob - the joiner's side
/// ## Step 4 in the "Setup Contact protocol"
pub(super) async fn handle_contact_confirm(
context: &Context,
mut bobstate: BobState,
message: &MimeMessage,
) -> Result<HandshakeMessage> {
let retval = if bobstate.is_join_group() {
HandshakeMessage::Propagate
} else {
HandshakeMessage::Ignore
};
match bobstate.handle_message(context, message).await? {
Some(BobHandshakeStage::Terminated(why)) => {
bobstate.notify_aborted(context, why).await?;
Ok(HandshakeMessage::Done)
}
Some(BobHandshakeStage::Completed) => {
// Note this goes to the 1:1 chat, as when joining a group we implicitly also
// verify both contacts (this could be a bug/security issue, see
// e.g. https://github.com/deltachat/deltachat-core-rust/issues/1177).
bobstate.notify_peer_verified(context).await?;
Ok(retval)
}
Some(_) => {
warn!(
context,
"Impossible state returned from handling handshake message"
);
Ok(retval)
}
None => Ok(retval),
}
}
/// Private implementations for user interactions about this [`BobState`].
impl BobState {
fn is_join_group(&self) -> bool {
match self.invite() {
QrInvite::Contact { .. } => false,
QrInvite::Group { .. } => true,
}
}
fn emit_progress(&self, context: &Context, progress: JoinerProgress) {
let contact_id = self.invite().contact_id();
context.emit_event(EventType::SecurejoinJoinerProgress {
contact_id,
progress: progress.into(),
});
}
/// Returns the [`ChatId`] of the chat being joined.
///
/// This is the chat in which you want to notify the user as well.
///
/// When joining a group this is the [`ChatId`] of the group chat, when verifying a
/// contact this is the [`ChatId`] of the 1:1 chat. The 1:1 chat is assumed to exist
/// because a [`BobState`] can not exist without, the group chat will be created if it
/// does not yet exist.
async fn joining_chat_id(&self, context: &Context) -> Result<ChatId> {
match self.invite() {
QrInvite::Contact { .. } => Ok(self.alice_chat()),
QrInvite::Group {
ref grpid,
ref name,
..
} => {
let group_chat_id = match chat::get_chat_id_by_grpid(context, grpid).await? {
Some((chat_id, _protected, _blocked)) => {
chat_id.unblock(context).await?;
chat_id
}
None => {
ChatId::create_multiuser_record(
context,
Chattype::Group,
grpid,
name,
Blocked::Not,
ProtectionStatus::Unprotected, // protection is added later as needed
None,
)
.await?
}
};
Ok(group_chat_id)
}
}
}
/// Notifies the user that the SecureJoin was aborted.
///
/// This creates an info message in the chat being joined.
async fn notify_aborted(&self, context: &Context, why: &str) -> Result<()> {
let contact = Contact::get_by_id(context, self.invite().contact_id()).await?;
let msg = stock_str::contact_not_verified(context, &contact).await;
let chat_id = self.joining_chat_id(context).await?;
chat::add_info_msg(context, chat_id, &msg, time()).await?;
warn!(
context,
"StockMessage::ContactNotVerified posted to joining chat ({})", why
);
Ok(())
}
/// Notifies the user that the SecureJoin peer is verified.
///
/// This creates an info message in the chat being joined.
async fn notify_peer_verified(&self, context: &Context) -> Result<()> {
let contact = Contact::get_by_id(context, self.invite().contact_id()).await?;
let msg = stock_str::contact_verified(context, &contact).await;
let chat_id = self.joining_chat_id(context).await?;
chat::add_info_msg(context, chat_id, &msg, time()).await?;
context.emit_event(EventType::ChatModified(chat_id));
Ok(())
}
}
/// Progress updates for [`EventType::SecurejoinJoinerProgress`].
///
/// This has an `From<JoinerProgress> for usize` impl yielding numbers between 0 and a 1000
/// which can be shown as a progress bar.
enum JoinerProgress {
/// An error occurred.
Error,
/// vg-vc-request-with-auth sent.
///
/// Typically shows as "alice@addr verified, introducing myself."
RequestWithAuthSent,
// /// Completed securejoin.
// Succeeded,
}
impl From<JoinerProgress> for usize {
fn from(progress: JoinerProgress) -> Self {
match progress {
JoinerProgress::Error => 0,
JoinerProgress::RequestWithAuthSent => 400,
// JoinerProgress::Succeeded => 1000,
}
}
}

View File

@@ -5,21 +5,22 @@
//! provides all the information to its driver so it can perform the correct interactions.
//!
//! The [`BobState`] is only directly used to initially create it when starting the
//! protocol.
//! protocol. Afterwards it must be stored in a mutex and the [`BobStateHandle`] should be
//! used to work with the state.
use anyhow::{Error, Result};
use rusqlite::Connection;
use anyhow::{bail, Error, Result};
use async_std::sync::MutexGuard;
use crate::chat::{self, ChatId};
use crate::constants::{Blocked, Viewtype};
use crate::contact::{Contact, Origin};
use crate::context::Context;
use crate::events::EventType;
use crate::headerdef::HeaderDef;
use crate::key::{DcKey, SignedPublicKey};
use crate::message::{Message, Viewtype};
use crate::message::Message;
use crate::mimeparser::{MimeMessage, SystemMessage};
use crate::param::Param;
use crate::sql::Sql;
use super::qrinvite::QrInvite;
use super::{
@@ -45,14 +46,123 @@ pub enum BobHandshakeStage {
Terminated(&'static str),
}
/// The securejoin state kept while Bob is joining.
/// A handle to work with the [`BobState`] of Bob's securejoin protocol.
///
/// This is stored in the database and loaded from there using [`BobState::from_db`]. To
/// create a new one use [`BobState::start_protocol`].
/// This handle can only be created for when an underlying [`BobState`] exists. It keeps
/// open a lock which guarantees unique access to the state and this struct must be dropped
/// to return the lock.
pub struct BobStateHandle<'a> {
guard: MutexGuard<'a, Option<BobState>>,
bobstate: BobState,
clear_state_on_drop: bool,
}
impl<'a> BobStateHandle<'a> {
/// Creates a new instance, upholding the guarantee that [`BobState`] must exist.
pub fn from_guard(mut guard: MutexGuard<'a, Option<BobState>>) -> Option<Self> {
guard.take().map(|bobstate| Self {
guard,
bobstate,
clear_state_on_drop: false,
})
}
/// Returns the [`ChatId`] of the group chat to join or the 1:1 chat with Alice.
pub async fn chat_id(&self, context: &Context) -> Result<ChatId> {
match self.bobstate.invite {
QrInvite::Group { ref grpid, .. } => {
if let Some((chat_id, _, _)) = chat::get_chat_id_by_grpid(context, &grpid).await? {
Ok(chat_id)
} else {
bail!("chat not found")
}
}
QrInvite::Contact { .. } => Ok(self.bobstate.chat_id),
}
}
/// Returns a reference to the [`QrInvite`] of the joiner process.
pub fn invite(&self) -> &QrInvite {
&self.bobstate.invite
}
/// Handles the given message for the securejoin handshake for Bob.
///
/// This proxies to [`BobState::handle_message`] and makes sure to clear the state when
/// the protocol state is terminal. It returns `Some` if the message successfully
/// advanced the state of the protocol state machine, `None` otherwise.
pub async fn handle_message(
&mut self,
context: &Context,
mime_message: &MimeMessage,
) -> Option<BobHandshakeStage> {
info!(context, "Handling securejoin message for BobStateHandle");
match self.bobstate.handle_message(context, mime_message).await {
Ok(Some(stage)) => {
if matches!(
stage,
BobHandshakeStage::Completed | BobHandshakeStage::Terminated(_)
) {
self.finish_protocol(context).await;
}
Some(stage)
}
Ok(None) => None,
Err(err) => {
warn!(
context,
"Error handling handshake message, aborting handshake: {}", err
);
self.finish_protocol(context).await;
None
}
}
}
/// Marks the bob handshake as finished.
///
/// This will clear the state on [`InnerContext::bob`] once this handle is dropped,
/// allowing a new handshake to be started from [`Bob`].
///
/// Note that the state is only cleared on Drop since otherwise the invariant that the
/// state is always consistent is violated. However the "ongoing" process is released
/// here a little bit earlier as this requires access to the Context, which we do not
/// have on Drop (Drop can not run asynchronous code). Stopping the "ongoing" process
/// will release [`securejoin`](super::securejoin) which in turn will finally free the
/// ongoing process using [`Context::free_ongoing`].
///
/// [`InnerContext::bob`]: crate::context::InnerContext::bob
/// [`Bob`]: super::Bob
async fn finish_protocol(&mut self, context: &Context) {
info!(context, "Finishing securejoin handshake protocol for Bob");
self.clear_state_on_drop = true;
if let QrInvite::Group { .. } = self.bobstate.invite {
context.stop_ongoing().await;
}
}
}
impl<'a> Drop for BobStateHandle<'a> {
fn drop(&mut self) {
if self.clear_state_on_drop {
// The Option should already be empty because we take it out in the ctor,
// however the typesystem doesn't guarantee this so do it again anyway.
self.guard.take();
} else {
// Make sure to put back the BobState into the Option of the Mutex, it was taken
// out by the constructor.
self.guard.replace(self.bobstate.clone());
}
}
}
/// The securejoin state kept in-memory while Bob is joining.
///
/// This purposefully has nothing optional, the state is always fully valid. However once a
/// terminal state is reached in [`BobState::next`] the entry in the database will already
/// have been deleted.
/// This is currently stored in [`Bob`] which is stored on the [`Context`], thus Bob can
/// only run one securejoin joiner protocol at a time.
///
/// This purposefully has nothing optional, the state is always fully valid. See
/// [`Bob::state`] to get access to this state.
///
/// # Conducting the securejoin handshake
///
@@ -67,8 +177,6 @@ pub enum BobHandshakeStage {
/// [`Bob::state`]: super::Bob::state
#[derive(Debug, Clone)]
pub struct BobState {
/// Database primary key.
id: i64,
/// The QR Invite code.
invite: QrInvite,
/// The next expected message from Alice.
@@ -80,120 +188,39 @@ pub struct BobState {
impl BobState {
/// Starts the securejoin protocol and creates a new [`BobState`].
///
/// The `chat_id` needs to be the ID of the 1:1 chat with Alice, this chat will be used
/// to exchange the SecureJoin handshake messages as well as for showing error messages.
///
/// # Bob - the joiner's side
/// ## Step 2 in the "Setup Contact protocol", section 2.1 of countermitm 0.10.0
///
/// This currently aborts any other securejoin process if any did not yet complete. The
/// ChatIds of the relevant 1:1 chat of any aborted handshakes are returned so that you
/// can report the aboreted handshake in the chat. (Yes, there can only ever be one
/// ChatId in that Vec, the database doesn't care though.)
pub async fn start_protocol(
context: &Context,
invite: QrInvite,
chat_id: ChatId,
) -> Result<(Self, BobHandshakeStage, Vec<Self>), JoinError> {
let (stage, next) =
if fingerprint_equals_sender(context, invite.fingerprint(), invite.contact_id()).await?
{
// The scanned fingerprint matches Alice's key, we can proceed to step 4b.
info!(context, "Taking securejoin protocol shortcut");
send_handshake_message(context, &invite, chat_id, BobHandshakeMsg::RequestWithAuth)
.await?;
(
BobHandshakeStage::RequestWithAuthSent,
SecureJoinStep::ContactConfirm,
)
} else {
send_handshake_message(context, &invite, chat_id, BobHandshakeMsg::Request).await?;
(BobHandshakeStage::RequestSent, SecureJoinStep::AuthRequired)
) -> Result<(Self, BobHandshakeStage), JoinError> {
let chat_id =
ChatId::create_for_contact_with_blocked(context, invite.contact_id(), Blocked::Yes)
.await
.map_err(JoinError::UnknownContact)?;
if fingerprint_equals_sender(context, invite.fingerprint(), invite.contact_id()).await? {
// The scanned fingerprint matches Alice's key, we can proceed to step 4b.
info!(context, "Taking securejoin protocol shortcut");
let state = Self {
invite,
next: SecureJoinStep::ContactConfirm,
chat_id,
};
let (id, aborted_states) =
Self::insert_new_db_entry(context, next, invite.clone(), chat_id).await?;
let state = Self {
id,
invite,
next,
chat_id,
};
Ok((state, stage, aborted_states))
}
/// Inserts a new entry in the bobstate table, deleting all previous entries.
///
/// Returns the ID of the newly inserted entry and all the aborted states.
async fn insert_new_db_entry(
context: &Context,
next: SecureJoinStep,
invite: QrInvite,
chat_id: ChatId,
) -> Result<(i64, Vec<Self>)> {
context
.sql
.transaction(move |transaction| {
// We need to start a write transaction right away, so that we have the
// database locked and no one else can write to this table while we read the
// rows that we will delete. So start with a dummy UPDATE.
transaction.execute(
r#"UPDATE bobstate SET next_step=?;"#,
params![SecureJoinStep::Terminated],
)?;
let mut stmt = transaction.prepare("SELECT id FROM bobstate;")?;
let mut aborted = Vec::new();
for id in stmt.query_map(params![], |row| row.get::<_, i64>(0))? {
let id = id?;
let state = BobState::from_db_id(transaction, id)?;
aborted.push(state);
}
// Finally delete everything and insert new row.
transaction.execute("DELETE FROM bobstate;", params![])?;
transaction.execute(
"INSERT INTO bobstate (invite, next_step, chat_id) VALUES (?, ?, ?);",
params![invite, next, chat_id],
)?;
let id = transaction.last_insert_rowid();
Ok((id, aborted))
})
.await
}
/// Load [`BobState`] from the database.
pub async fn from_db(sql: &Sql) -> Result<Option<Self>> {
// Because of how Self::start_protocol() updates the database we are currently
// guaranteed to only have one row.
sql.query_row_optional(
"SELECT id, invite, next_step, chat_id FROM bobstate;",
paramsv![],
|row| {
let s = BobState {
id: row.get(0)?,
invite: row.get(1)?,
next: row.get(2)?,
chat_id: row.get(3)?,
};
Ok(s)
},
)
.await
}
fn from_db_id(connection: &Connection, id: i64) -> rusqlite::Result<Self> {
connection.query_row(
"SELECT invite, next_step, chat_id FROM bobstate WHERE id=?;",
params![id],
|row| {
let s = BobState {
id,
invite: row.get(0)?,
next: row.get(1)?,
chat_id: row.get(2)?,
};
Ok(s)
},
)
state
.send_handshake_message(context, BobHandshakeMsg::RequestWithAuth)
.await?;
Ok((state, BobHandshakeStage::RequestWithAuthSent))
} else {
let state = Self {
invite,
next: SecureJoinStep::AuthRequired,
chat_id,
};
state
.send_handshake_message(context, BobHandshakeMsg::Request)
.await?;
Ok((state, BobHandshakeStage::RequestSent))
}
}
/// Returns the [`QrInvite`] used to create this [`BobState`].
@@ -201,45 +228,20 @@ impl BobState {
&self.invite
}
/// Returns the [`ChatId`] of the 1:1 chat with the inviter (Alice).
pub fn alice_chat(&self) -> ChatId {
self.chat_id
}
/// Updates the [`BobState::next`] field in memory and the database.
///
/// If the next state is a terminal state it will remove this [`BobState`] from the
/// database.
///
/// If a user scanned a new QR code after this [`BobState`] was loaded this update will
/// fail currently because starting a new joiner process currently kills any previously
/// running processes. This is a limitation which will go away in the future.
async fn update_next(&mut self, sql: &Sql, next: SecureJoinStep) -> Result<()> {
// TODO: write test verifying how this would fail.
match next {
SecureJoinStep::AuthRequired | SecureJoinStep::ContactConfirm => {
sql.execute(
"UPDATE bobstate SET next_step=? WHERE id=?;",
paramsv![next, self.id],
)
.await?;
}
SecureJoinStep::Terminated | SecureJoinStep::Completed => {
sql.execute("DELETE FROM bobstate WHERE id=?;", paramsv!(self.id))
.await?;
}
}
self.next = next;
Ok(())
}
/// Handles the given message for the securejoin handshake for Bob.
///
/// If the message was not used for this handshake `None` is returned, otherwise the new
/// stage is returned. Once [`BobHandshakeStage::Completed`] or
/// [`BobHandshakeStage::Terminated`] are reached this [`BobState`] should be destroyed,
/// further calling it will just result in the messages being unused by this handshake.
pub async fn handle_message(
///
/// # Errors
///
/// Under normal operation this should never return an error, regardless of what kind of
/// message it is called with. Any errors therefore should be treated as fatal internal
/// errors and this entire [`BobState`] should be thrown away as the state machine can
/// no longer be considered consistent.
async fn handle_message(
&mut self,
context: &Context,
mime_message: &MimeMessage,
@@ -302,20 +304,17 @@ impl BobState {
} else {
"Required encryption missing"
};
self.update_next(&context.sql, SecureJoinStep::Terminated)
.await?;
self.next = SecureJoinStep::Terminated;
return Ok(Some(BobHandshakeStage::Terminated(reason)));
}
if !fingerprint_equals_sender(context, self.invite.fingerprint(), self.invite.contact_id())
.await?
{
self.update_next(&context.sql, SecureJoinStep::Terminated)
.await?;
self.next = SecureJoinStep::Terminated;
return Ok(Some(BobHandshakeStage::Terminated("Fingerprint mismatch")));
}
info!(context, "Fingerprint verified.",);
self.update_next(&context.sql, SecureJoinStep::ContactConfirm)
.await?;
self.next = SecureJoinStep::ContactConfirm;
self.send_handshake_message(context, BobHandshakeMsg::RequestWithAuth)
.await?;
Ok(Some(BobHandshakeStage::RequestWithAuthSent))
@@ -363,8 +362,7 @@ impl BobState {
if vg_expect_encrypted
&& !encrypted_and_signed(context, mime_message, Some(self.invite.fingerprint()))
{
self.update_next(&context.sql, SecureJoinStep::Terminated)
.await?;
self.next = SecureJoinStep::Terminated;
return Ok(Some(BobHandshakeStage::Terminated(
"Contact confirm message not encrypted",
)));
@@ -396,8 +394,7 @@ impl BobState {
// This is not an error affecting the protocol outcome.
.ok();
self.update_next(&context.sql, SecureJoinStep::Completed)
.await?;
self.next = SecureJoinStep::Completed;
Ok(Some(BobHandshakeStage::Completed))
}
@@ -409,60 +406,48 @@ impl BobState {
context: &Context,
step: BobHandshakeMsg,
) -> Result<(), SendMsgError> {
send_handshake_message(context, &self.invite, self.chat_id, step).await
let mut msg = Message {
viewtype: Viewtype::Text,
text: Some(step.body_text(&self.invite)),
hidden: true,
..Default::default()
};
msg.param.set_cmd(SystemMessage::SecurejoinMessage);
// Sends the step in Secure-Join header.
msg.param
.set(Param::Arg, step.securejoin_header(&self.invite));
match step {
BobHandshakeMsg::Request => {
// Sends the Secure-Join-Invitenumber header in mimefactory.rs.
msg.param.set(Param::Arg2, self.invite.invitenumber());
msg.param.set_int(Param::ForcePlaintext, 1);
}
BobHandshakeMsg::RequestWithAuth => {
// Sends the Secure-Join-Auth header in mimefactory.rs.
msg.param.set(Param::Arg2, self.invite.authcode());
msg.param.set_int(Param::GuaranteeE2ee, 1);
}
BobHandshakeMsg::ContactConfirmReceived => {
msg.param.set_int(Param::GuaranteeE2ee, 1);
}
};
// Sends our own fingerprint in the Secure-Join-Fingerprint header.
let bob_fp = SignedPublicKey::load_self(context).await?.fingerprint();
msg.param.set(Param::Arg3, bob_fp.hex());
// Sends the grpid in the Secure-Join-Group header.
if let QrInvite::Group { ref grpid, .. } = self.invite {
msg.param.set(Param::Arg4, grpid);
}
chat::send_msg(context, self.chat_id, &mut msg).await?;
Ok(())
}
}
/// Sends the requested handshake message to Alice.
///
/// Same as [`BobState::send_handshake_message`] but this variation allows us to send this
/// message before we create the state in [`BobState::start_protocol`].
async fn send_handshake_message(
context: &Context,
invite: &QrInvite,
chat_id: ChatId,
step: BobHandshakeMsg,
) -> Result<(), SendMsgError> {
let mut msg = Message {
viewtype: Viewtype::Text,
text: Some(step.body_text(invite)),
hidden: true,
..Default::default()
};
msg.param.set_cmd(SystemMessage::SecurejoinMessage);
// Sends the step in Secure-Join header.
msg.param.set(Param::Arg, step.securejoin_header(invite));
match step {
BobHandshakeMsg::Request => {
// Sends the Secure-Join-Invitenumber header in mimefactory.rs.
msg.param.set(Param::Arg2, invite.invitenumber());
msg.force_plaintext();
}
BobHandshakeMsg::RequestWithAuth => {
// Sends the Secure-Join-Auth header in mimefactory.rs.
msg.param.set(Param::Arg2, invite.authcode());
msg.param.set_int(Param::GuaranteeE2ee, 1);
}
BobHandshakeMsg::ContactConfirmReceived => {
msg.param.set_int(Param::GuaranteeE2ee, 1);
}
};
// Sends our own fingerprint in the Secure-Join-Fingerprint header.
let bob_fp = SignedPublicKey::load_self(context).await?.fingerprint();
msg.param.set(Param::Arg3, bob_fp.hex());
// Sends the grpid in the Secure-Join-Group header.
if let QrInvite::Group { ref grpid, .. } = invite {
msg.param.set(Param::Arg4, grpid);
}
chat::send_msg(context, chat_id, &mut msg).await?;
Ok(())
}
/// Identifies the SecureJoin handshake messages Bob can send.
enum BobHandshakeMsg {
/// vc-request or vg-request
@@ -507,8 +492,8 @@ impl BobHandshakeMsg {
}
/// The next message expected by [`BobState`] in the setup-contact/secure-join protocol.
#[derive(Debug, Clone, Copy, PartialEq)]
pub enum SecureJoinStep {
#[derive(Debug, Clone, PartialEq)]
enum SecureJoinStep {
/// Expecting the auth-required message.
///
/// This corresponds to the `vc-auth-required` or `vg-auth-required` message of step 3d.
@@ -548,29 +533,3 @@ impl SecureJoinStep {
}
}
}
impl rusqlite::types::ToSql for SecureJoinStep {
fn to_sql(&self) -> rusqlite::Result<rusqlite::types::ToSqlOutput<'_>> {
let num = match &self {
SecureJoinStep::AuthRequired => 0,
SecureJoinStep::ContactConfirm => 1,
SecureJoinStep::Terminated => 2,
SecureJoinStep::Completed => 3,
};
let val = rusqlite::types::Value::Integer(num);
let out = rusqlite::types::ToSqlOutput::Owned(val);
Ok(out)
}
}
impl rusqlite::types::FromSql for SecureJoinStep {
fn column_result(value: rusqlite::types::ValueRef<'_>) -> rusqlite::types::FromSqlResult<Self> {
i64::column_result(value).and_then(|val| match val {
0 => Ok(SecureJoinStep::AuthRequired),
1 => Ok(SecureJoinStep::ContactConfirm),
2 => Ok(SecureJoinStep::Terminated),
3 => Ok(SecureJoinStep::Completed),
_ => Err(rusqlite::types::FromSqlError::OutOfRange(val)),
})
}
}

View File

@@ -8,23 +8,22 @@ use std::convert::TryFrom;
use anyhow::{bail, Error, Result};
use crate::contact::ContactId;
use crate::key::Fingerprint;
use crate::qr::Qr;
/// Represents the data from a QR-code scan.
///
/// There are methods to conveniently access fields present in both variants.
#[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]
#[derive(Debug, Clone)]
pub enum QrInvite {
Contact {
contact_id: ContactId,
contact_id: u32,
fingerprint: Fingerprint,
invitenumber: String,
authcode: String,
},
Group {
contact_id: ContactId,
contact_id: u32,
fingerprint: Fingerprint,
name: String,
grpid: String,
@@ -38,7 +37,7 @@ impl QrInvite {
///
/// The actual QR-code contains a URL-encoded email address, but upon scanning this is
/// translated to a contact ID.
pub fn contact_id(&self) -> ContactId {
pub fn contact_id(&self) -> u32 {
match self {
Self::Contact { contact_id, .. } | Self::Group { contact_id, .. } => *contact_id,
}
@@ -101,22 +100,3 @@ impl TryFrom<Qr> for QrInvite {
}
}
}
impl rusqlite::types::ToSql for QrInvite {
fn to_sql(&self) -> rusqlite::Result<rusqlite::types::ToSqlOutput<'_>> {
let json = serde_json::to_string(self)
.map_err(|err| rusqlite::Error::ToSqlConversionFailure(Box::new(err)))?;
let val = rusqlite::types::Value::Text(json);
let out = rusqlite::types::ToSqlOutput::Owned(val);
Ok(out)
}
}
impl rusqlite::types::FromSql for QrInvite {
fn column_result(value: rusqlite::types::ValueRef<'_>) -> rusqlite::types::FromSqlResult<Self> {
String::column_result(value).and_then(|val| {
serde_json::from_str(&val)
.map_err(|err| rusqlite::types::FromSqlError::Other(Box::new(err)))
})
}
}

View File

@@ -240,7 +240,7 @@ fn render_message(lines: &[&str], is_cut_at_end: bool) -> String {
ret += " [...]";
}
// redo escaping done by escape_message_footer_marks()
ret.replace('\u{200B}', "")
ret.replace("\u{200B}", "")
}
/// Returns true if the line contains only whitespace.

View File

@@ -4,18 +4,14 @@ pub mod send;
use std::time::{Duration, SystemTime};
use anyhow::{bail, format_err, Context as _, Result};
use async_smtp::smtp::client::net::ClientTlsParameters;
use async_smtp::smtp::response::{Category, Code, Detail};
use async_smtp::{smtp, EmailAddress, ServerAddress};
use async_smtp::{error, smtp, EmailAddress, ServerAddress};
use crate::constants::DC_LP_AUTH_OAUTH2;
use crate::events::EventType;
use crate::job::Status;
use crate::login_param::{
dc_build_tls, CertificateChecks, LoginParam, ServerLoginParam, Socks5Config,
};
use crate::message::{self, MsgId};
use crate::oauth2::dc_get_oauth2_access_token;
use crate::provider::Socket;
use crate::{context::Context, scheduler::connectivity::ConnectivityStore};
@@ -23,6 +19,28 @@ use crate::{context::Context, scheduler::connectivity::ConnectivityStore};
/// SMTP write and read timeout in seconds.
const SMTP_TIMEOUT: u64 = 30;
#[derive(Debug, thiserror::Error)]
pub enum Error {
#[error("Bad parameters")]
BadParameters,
#[error("Invalid login address {address}: {error}")]
InvalidLoginAddress {
address: String,
#[source]
error: error::Error,
},
#[error("SMTP failed to connect: {0}")]
ConnectionFailure(#[source] smtp::error::Error),
#[error("SMTP oauth2 error {address}")]
Oauth2 { address: String },
#[error("TLS error {0}")]
Tls(#[from] async_native_tls::Error),
#[error("{0}")]
Other(#[from] anyhow::Error),
}
pub type Result<T> = std::result::Result<T, Error>;
#[derive(Default)]
pub(crate) struct Smtp {
transport: Option<smtp::SmtpTransport>,
@@ -113,11 +131,14 @@ impl Smtp {
}
if lp.server.is_empty() || lp.port == 0 {
bail!("bad connection parameters");
return Err(Error::BadParameters);
}
let from = EmailAddress::new(addr.to_string())
.with_context(|| format!("invalid login address {}", addr))?;
let from =
EmailAddress::new(addr.to_string()).map_err(|err| Error::InvalidLoginAddress {
address: addr.to_string(),
error: err,
})?;
self.from = Some(from);
@@ -138,7 +159,9 @@ impl Smtp {
let send_pw = &lp.password;
let access_token = dc_get_oauth2_access_token(context, addr, send_pw, false).await?;
if access_token.is_none() {
bail!("SMTP OAuth 2 error {}", addr);
return Err(Error::Oauth2 {
address: addr.to_string(),
});
}
let user = &lp.user;
(
@@ -182,7 +205,9 @@ impl Smtp {
}
let mut trans = client.into_transport();
trans.connect().await.context("SMTP failed to connect")?;
if let Err(err) = trans.connect().await {
return Err(Error::ConnectionFailure(err));
}
self.transport = Some(trans);
self.last_success = Some(SystemTime::now());
@@ -195,324 +220,3 @@ impl Smtp {
Ok(())
}
}
/// Tries to send a message.
///
/// Returns Status::Finished if sending the message should not be retried anymore,
/// Status::RetryLater if sending should be postponed and Status::RetryNow if it is suspected that
/// temporary failure is caused by stale connection, in which case a second attempt to send the
/// same message may be done immediately.
pub(crate) async fn smtp_send(
context: &Context,
recipients: &[async_smtp::EmailAddress],
message: &str,
smtp: &mut Smtp,
msg_id: MsgId,
rowid: i64,
) -> Status {
if std::env::var(crate::DCC_MIME_DEBUG).is_ok() {
info!(context, "smtp-sending out mime message:");
println!("{}", message);
}
smtp.connectivity.set_working(context).await;
let send_result = smtp
.send(context, recipients, message.as_bytes(), rowid)
.await;
smtp.last_send_error = send_result.as_ref().err().map(|e| e.to_string());
let status = match send_result {
Err(crate::smtp::send::Error::SmtpSend(err)) => {
// Remote error, retry later.
info!(context, "SMTP failed to send: {:?}", &err);
let res = match err {
async_smtp::smtp::error::Error::Permanent(ref response) => {
// Workaround for incorrectly configured servers returning permanent errors
// instead of temporary ones.
let maybe_transient = match response.code {
// Sometimes servers send a permanent error when actually it is a temporary error
// For documentation see <https://tools.ietf.org/html/rfc3463>
Code {
category: Category::MailSystem,
detail: Detail::Zero,
..
} => {
// Ignore status code 5.5.0, see <https://support.delta.chat/t/every-other-message-gets-stuck/877/2>
// Maybe incorrectly configured Postfix milter with "reject" instead of "tempfail", which returns
// "550 5.5.0 Service unavailable" instead of "451 4.7.1 Service unavailable - try again later".
//
// Other enhanced status codes, such as Postfix
// "550 5.1.1 <foobar@example.org>: Recipient address rejected: User unknown in local recipient table"
// are not ignored.
response.first_word() == Some("5.5.0")
}
_ => false,
};
if maybe_transient {
info!(context, "Permanent error that is likely to actually be transient, postponing retry for later");
Status::RetryLater
} else {
info!(context, "Permanent error, message sending failed");
// If we do not retry, add an info message to the chat.
// Yandex error "554 5.7.1 [2] Message rejected under suspicion of SPAM; https://ya.cc/..."
// should definitely go here, because user has to open the link to
// resume message sending.
Status::Finished(Err(format_err!("Permanent SMTP error: {}", err)))
}
}
async_smtp::smtp::error::Error::Transient(ref response) => {
// We got a transient 4xx response from SMTP server.
// Give some time until the server-side error maybe goes away.
if let Some(first_word) = response.first_word() {
if first_word.ends_with(".1.1")
|| first_word.ends_with(".1.2")
|| first_word.ends_with(".1.3")
{
// Sometimes we receive transient errors that should be permanent.
// Any extended smtp status codes like x.1.1, x.1.2 or x.1.3 that we
// receive as a transient error are misconfigurations of the smtp server.
// See <https://tools.ietf.org/html/rfc3463#section-3.2>
info!(context, "Received extended status code {} for a transient error. This looks like a misconfigured smtp server, let's fail immediatly", first_word);
Status::Finished(Err(format_err!("Permanent SMTP error: {}", err)))
} else {
info!(
context,
"Transient error with status code {}, postponing retry for later",
first_word
);
Status::RetryLater
}
} else {
info!(
context,
"Transient error without status code, postponing retry for later"
);
Status::RetryLater
}
}
_ => {
info!(
context,
"Message sending failed without error returned by the server"
);
if smtp.has_maybe_stale_connection().await {
info!(context, "Connection is probably stale, retry immediately");
Status::RetryNow
} else {
info!(context, "Connection is not stale, retry later");
Status::RetryLater
}
}
};
// this clears last_success info
info!(context, "Failed to send message over SMTP, disconnecting");
smtp.disconnect().await;
res
}
Err(crate::smtp::send::Error::Envelope(err)) => {
// Local error, job is invalid, do not retry.
smtp.disconnect().await;
warn!(context, "SMTP job is invalid: {}", err);
Status::Finished(Err(err.into()))
}
Err(crate::smtp::send::Error::NoTransport) => {
// Should never happen.
// It does not even make sense to disconnect here.
error!(context, "SMTP job failed because SMTP has no transport");
Status::Finished(Err(format_err!("SMTP has not transport")))
}
Err(crate::smtp::send::Error::Other(err)) => {
// Local error, job is invalid, do not retry.
smtp.disconnect().await;
warn!(context, "unable to load job: {}", err);
Status::Finished(Err(err))
}
Ok(()) => Status::Finished(Ok(())),
};
if let Status::Finished(Err(err)) = &status {
// We couldn't send the message, so mark it as failed
message::set_msg_failed(context, msg_id, Some(err.to_string())).await;
}
status
}
/// Sends message identified by `smtp` table rowid over SMTP connection.
///
/// Removes row if the message should not be retried, otherwise increments retry count.
pub(crate) async fn send_msg_to_smtp(
context: &Context,
smtp: &mut Smtp,
rowid: i64,
) -> anyhow::Result<()> {
if let Err(err) = smtp
.connect_configured(context)
.await
.context("SMTP connection failure")
{
smtp.last_send_error = Some(format!("{:#}", err));
return Err(err);
}
// Increase retry count as soon as we have an SMTP connection. This ensures that the message is
// eventually removed from the queue by exceeding retry limit even in case of an error that
// keeps happening early in the message sending code, e.g. failure to read the message from the
// database.
context
.sql
.execute(
"UPDATE smtp SET retries=retries+1 WHERE id=?",
paramsv![rowid],
)
.await
.context("failed to update retries count")?;
let (body, recipients, msg_id, retries) = context
.sql
.query_row(
"SELECT mime, recipients, msg_id, retries FROM smtp WHERE id=?",
paramsv![rowid],
|row| {
let mime: String = row.get(0)?;
let recipients: String = row.get(1)?;
let msg_id: MsgId = row.get(2)?;
let retries: i64 = row.get(3)?;
Ok((mime, recipients, msg_id, retries))
},
)
.await?;
if retries > 6 {
message::set_msg_failed(
context,
msg_id,
Some("Number of retries exceeded the limit."),
)
.await;
context
.sql
.execute("DELETE FROM smtp WHERE id=?", paramsv![rowid])
.await
.context("failed to remove message with exceeded retry limit from smtp table")?;
bail!("Number of retries exceeded the limit");
}
info!(
context,
"Retry number {} to send message {} over SMTP", retries, msg_id
);
let recipients_list = recipients
.split(' ')
.filter_map(
|addr| match async_smtp::EmailAddress::new(addr.to_string()) {
Ok(addr) => Some(addr),
Err(err) => {
warn!(context, "invalid recipient: {} {:?}", addr, err);
None
}
},
)
.collect::<Vec<_>>();
// If there is a msg-id and it does not exist in the db, cancel sending. this happens if
// dc_delete_msgs() was called before the generated mime was sent out.
if !message::exists(context, msg_id)
.await
.with_context(|| format!("failed to check message {} existence", msg_id))?
{
info!(
context,
"Sending of message {} was cancelled by the user.", msg_id
);
return Ok(());
}
let status = match smtp_send(
context,
&recipients_list,
body.as_str(),
smtp,
msg_id,
rowid,
)
.await
{
Status::RetryNow => {
// Do a single retry immediately without increasing retry counter in case of stale
// connection.
info!(context, "Doing immediate retry to send message.");
// smtp_send just closed stale SMTP connection, reconnect and try again.
if let Err(err) = smtp
.connect_configured(context)
.await
.context("failed to reopen stale SMTP connection")
{
smtp.last_send_error = Some(format!("{:#}", err));
return Err(err);
}
smtp_send(
context,
&recipients_list,
body.as_str(),
smtp,
msg_id,
rowid,
)
.await
}
status => status,
};
match status {
Status::Finished(res) => {
context
.sql
.execute("DELETE FROM smtp WHERE id=?", paramsv![rowid])
.await?;
if res.is_ok() {
msg_id.set_delivered(context).await?;
}
res
}
Status::RetryNow | Status::RetryLater => Err(format_err!("Retry")),
}
}
/// Tries to send all messages currently in `smtp` table.
///
/// Logs and ignores SMTP errors to ensure that a single SMTP message constantly failing to be sent
/// does not block other messages in the queue from being sent.
pub(crate) async fn send_smtp_messages(
context: &Context,
connection: &mut Smtp,
) -> anyhow::Result<()> {
context.send_sync_msg().await?; // Add sync message to the end of the queue if needed.
let rowids = context
.sql
.query_map(
"SELECT id FROM smtp ORDER BY id ASC",
paramsv![],
|row| {
let rowid: i64 = row.get(0)?;
Ok(rowid)
},
|rowids| {
rowids
.collect::<std::result::Result<Vec<_>, _>>()
.map_err(Into::into)
},
)
.await?;
for rowid in rowids {
if let Err(err) = send_msg_to_smtp(context, connection, rowid).await {
info!(context, "Failed to send message over SMTP: {:#}.", err);
}
}
Ok(())
}

View File

@@ -28,9 +28,9 @@ impl Smtp {
pub async fn send(
&mut self,
context: &Context,
recipients: &[EmailAddress],
message: &[u8],
rowid: i64,
recipients: Vec<EmailAddress>,
message: Vec<u8>,
job_id: u32,
) -> Result<()> {
let message_len_bytes = message.len();
@@ -41,7 +41,7 @@ impl Smtp {
}
}
for recipients_chunk in recipients.chunks(chunk_size) {
for recipients_chunk in recipients.chunks(chunk_size).into_iter() {
let recipients_display = recipients_chunk
.iter()
.map(|x| x.as_ref())
@@ -52,8 +52,8 @@ impl Smtp {
.map_err(Error::Envelope)?;
let mail = SendableEmail::new(
envelope,
rowid.to_string(), // only used for internal logging
message,
format!("{}", job_id), // only used for internal logging
&message,
);
if let Some(ref mut transport) = self.transport {

View File

@@ -3,23 +3,22 @@
use async_std::path::Path;
use async_std::sync::RwLock;
use std::collections::{HashMap, HashSet};
use std::collections::HashSet;
use std::convert::TryFrom;
use std::time::Duration;
use anyhow::{bail, Context as _, Result};
use async_std::path::PathBuf;
use anyhow::{bail, format_err, Context as _, Result};
use async_std::prelude::*;
use rusqlite::{config::DbConfig, Connection, OpenFlags};
use rusqlite::OpenFlags;
use crate::blob::BlobObject;
use crate::chat::{add_device_msg, update_device_icon, update_saved_messages_icon};
use crate::config::Config;
use crate::constants::DC_CHAT_ID_TRASH;
use crate::constants::{Viewtype, DC_CHAT_ID_TRASH};
use crate::context::Context;
use crate::dc_tools::{dc_delete_file, time};
use crate::ephemeral::start_ephemeral_timers;
use crate::message::{Message, Viewtype};
use crate::message::Message;
use crate::param::{Param, Params};
use crate::peerstate::{deduplicate_peerstates, Peerstate};
use crate::stock_str;
@@ -39,53 +38,20 @@ mod migrations;
/// A wrapper around the underlying Sqlite3 object.
#[derive(Debug)]
pub struct Sql {
/// Database file path
pub(crate) dbfile: PathBuf,
pool: RwLock<Option<r2d2::Pool<r2d2_sqlite::SqliteConnectionManager>>>,
}
/// None if the database is not open, true if it is open with passphrase and false if it is
/// open without a passphrase.
is_encrypted: RwLock<Option<bool>>,
pub(crate) config_cache: RwLock<HashMap<String, Option<String>>>,
impl Default for Sql {
fn default() -> Self {
Self {
pool: RwLock::new(None),
}
}
}
impl Sql {
pub fn new(dbfile: PathBuf) -> Sql {
Self {
dbfile,
pool: Default::default(),
is_encrypted: Default::default(),
config_cache: Default::default(),
}
}
/// Tests SQLCipher passphrase.
///
/// Returns true if passphrase is correct, i.e. the database is new or can be unlocked with
/// this passphrase, and false if the database is already encrypted with another passphrase or
/// corrupted.
///
/// Fails if database is already open.
pub async fn check_passphrase(&self, passphrase: String) -> Result<bool> {
if self.is_open().await {
bail!("Database is already opened.");
}
// Hold the lock to prevent other thread from opening the database.
let _lock = self.pool.write().await;
// Test that the key is correct using a single connection.
let connection = Connection::open(&self.dbfile)?;
connection
.pragma_update(None, "key", &passphrase)
.context("failed to set PRAGMA key")?;
let key_is_correct = connection
.query_row("SELECT count(*) FROM sqlite_master", [], |_row| Ok(()))
.is_ok();
Ok(key_is_correct)
pub fn new() -> Sql {
Self::default()
}
/// Checks if there is currently a connection to the underlying Sqlite database.
@@ -93,98 +59,37 @@ impl Sql {
self.pool.read().await.is_some()
}
/// Returns true if the database is encrypted.
///
/// If database is not open, returns `None`.
pub(crate) async fn is_encrypted(&self) -> Option<bool> {
*self.is_encrypted.read().await
}
/// Closes all underlying Sqlite connections.
async fn close(&self) {
pub async fn close(&self) {
let _ = self.pool.write().await.take();
// drop closes the connection
}
/// Exports the database to a separate file with the given passphrase.
///
/// Set passphrase to empty string to export the database unencrypted.
pub(crate) async fn export(&self, path: &Path, passphrase: String) -> Result<()> {
let path_str = path
.to_str()
.with_context(|| format!("path {:?} is not valid unicode", path))?;
let conn = self.get_conn().await?;
conn.execute(
"ATTACH DATABASE ? AS backup KEY ?",
paramsv![path_str, passphrase],
)
.context("failed to attach backup database")?;
let res = conn
.query_row("SELECT sqlcipher_export('backup')", [], |_row| Ok(()))
.context("failed to export to attached backup database");
conn.execute("DETACH DATABASE backup", [])
.context("failed to detach backup database")?;
res?;
Ok(())
}
/// Imports the database from a separate file with the given passphrase.
pub(crate) async fn import(&self, path: &Path, passphrase: String) -> Result<()> {
let path_str = path
.to_str()
.with_context(|| format!("path {:?} is not valid unicode", path))?;
let conn = self.get_conn().await?;
// Reset the database without reopening it. We don't want to reopen the database because we
// don't have main database passphrase at this point.
// See <https://sqlite.org/c3ref/c_dbconfig_enable_fkey.html> for documentation.
// Without resetting import may fail due to existing tables.
conn.set_db_config(DbConfig::SQLITE_DBCONFIG_RESET_DATABASE, true)
.context("failed to set SQLITE_DBCONFIG_RESET_DATABASE")?;
conn.execute("VACUUM", [])
.context("failed to vacuum the database")?;
conn.set_db_config(DbConfig::SQLITE_DBCONFIG_RESET_DATABASE, false)
.context("failed to unset SQLITE_DBCONFIG_RESET_DATABASE")?;
conn.execute(
"ATTACH DATABASE ? AS backup KEY ?",
paramsv![path_str, passphrase],
)
.context("failed to attach backup database")?;
let res = conn
.query_row("SELECT sqlcipher_export('main', 'backup')", [], |_row| {
Ok(())
})
.context("failed to import from attached backup database");
conn.execute("DETACH DATABASE backup", [])
.context("failed to detach backup database")?;
res?;
Ok(())
}
fn new_pool(
pub fn new_pool(
dbfile: &Path,
passphrase: String,
) -> Result<r2d2::Pool<r2d2_sqlite::SqliteConnectionManager>> {
readonly: bool,
) -> anyhow::Result<r2d2::Pool<r2d2_sqlite::SqliteConnectionManager>> {
let mut open_flags = OpenFlags::SQLITE_OPEN_NO_MUTEX;
open_flags.insert(OpenFlags::SQLITE_OPEN_READ_WRITE);
open_flags.insert(OpenFlags::SQLITE_OPEN_CREATE);
if readonly {
open_flags.insert(OpenFlags::SQLITE_OPEN_READ_ONLY);
} else {
open_flags.insert(OpenFlags::SQLITE_OPEN_READ_WRITE);
open_flags.insert(OpenFlags::SQLITE_OPEN_CREATE);
}
// this actually creates min_idle database handles just now.
// therefore, with_init() must not try to modify the database as otherwise
// we easily get busy-errors (eg. table-creation, journal_mode etc. should be done on only one handle)
let mgr = r2d2_sqlite::SqliteConnectionManager::file(dbfile)
.with_flags(open_flags)
.with_init(move |c| {
.with_init(|c| {
c.execute_batch(&format!(
"PRAGMA cipher_memory_security = OFF; -- Too slow on Android
PRAGMA secure_delete=on;
"PRAGMA secure_delete=on;
PRAGMA busy_timeout = {};
PRAGMA temp_store=memory; -- Avoid SQLITE_IOERR_GETTEMPPATH errors on Android
",
Duration::from_secs(10).as_millis()
))?;
c.pragma_update(None, "key", passphrase.clone())?;
Ok(())
});
@@ -197,125 +102,105 @@ impl Sql {
Ok(pool)
}
async fn try_open(&self, context: &Context, dbfile: &Path, passphrase: String) -> Result<()> {
*self.pool.write().await = Some(Self::new_pool(dbfile, passphrase.to_string())?);
{
let conn = self.get_conn().await?;
// Try to enable auto_vacuum. This will only be
// applied if the database is new or after successful
// VACUUM, which usually happens before backup export.
// When auto_vacuum is INCREMENTAL, it is possible to
// use PRAGMA incremental_vacuum to return unused
// database pages to the filesystem.
conn.pragma_update(None, "auto_vacuum", &"INCREMENTAL".to_string())?;
// journal_mode is persisted, it is sufficient to change it only for one handle.
conn.pragma_update(None, "journal_mode", &"WAL".to_string())?;
// Default synchronous=FULL is much slower. NORMAL is sufficient for WAL mode.
conn.pragma_update(None, "synchronous", &"NORMAL".to_string())?;
}
self.run_migrations(context).await?;
Ok(())
}
pub async fn run_migrations(&self, context: &Context) -> Result<()> {
// (1) update low-level database structure.
// this should be done before updates that use high-level objects that
// rely themselves on the low-level structure.
let (recalc_fingerprints, update_icons, disable_server_delete, recode_avatar) =
migrations::run(context, self)
.await
.context("failed to run migrations")?;
// (2) updates that require high-level objects
// the structure is complete now and all objects are usable
if recalc_fingerprints {
info!(context, "[migration] recalc fingerprints");
let addrs = self
.query_map(
"SELECT addr FROM acpeerstates;",
paramsv![],
|row| row.get::<_, String>(0),
|addrs| {
addrs
.collect::<std::result::Result<Vec<_>, _>>()
.map_err(Into::into)
},
)
.await?;
for addr in &addrs {
if let Some(ref mut peerstate) = Peerstate::from_addr(context, addr).await? {
peerstate.recalc_fingerprint();
peerstate.save_to_db(self, false).await?;
}
}
}
if update_icons {
update_saved_messages_icon(context).await?;
update_device_icon(context).await?;
}
if disable_server_delete {
// We now always watch all folders and delete messages there if delete_server is enabled.
// So, for people who have delete_server enabled, disable it and add a hint to the devicechat:
if context.get_config_delete_server_after().await?.is_some() {
let mut msg = Message::new(Viewtype::Text);
msg.text = Some(stock_str::delete_server_turned_off(context).await);
add_device_msg(context, None, Some(&mut msg)).await?;
context
.set_config(Config::DeleteServerAfter, Some("0"))
.await?;
}
}
if recode_avatar {
if let Some(avatar) = context.get_config(Config::Selfavatar).await? {
let mut blob = BlobObject::new_from_path(context, avatar.as_ref()).await?;
match blob.recode_to_avatar_size(context).await {
Ok(()) => {
context
.set_config(Config::Selfavatar, Some(&avatar))
.await?
}
Err(e) => {
warn!(context, "Migrations can't recode avatar, removing. {:#}", e);
context.set_config(Config::Selfavatar, None).await?
}
}
}
}
Ok(())
}
/// Opens the provided database and runs any necessary migrations.
/// If a database is already open, this will return an error.
pub async fn open(&self, context: &Context, passphrase: String) -> Result<()> {
pub async fn open(
&self,
context: &Context,
dbfile: &Path,
readonly: bool,
) -> anyhow::Result<()> {
if self.is_open().await {
error!(
context,
"Cannot open, database \"{:?}\" already opened.", self.dbfile,
"Cannot open, database \"{:?}\" already opened.", dbfile,
);
bail!("SQL database is already opened.");
}
let passphrase_nonempty = !passphrase.is_empty();
if let Err(err) = self.try_open(context, &self.dbfile, passphrase).await {
self.close().await;
Err(err)
} else {
info!(context, "Opened database {:?}.", self.dbfile);
*self.is_encrypted.write().await = Some(passphrase_nonempty);
Ok(())
*self.pool.write().await = Some(Self::new_pool(dbfile, readonly)?);
if !readonly {
{
let conn = self.get_conn().await?;
// journal_mode is persisted, it is sufficient to change it only for one handle.
conn.pragma_update(None, "journal_mode", &"WAL".to_string())?;
// Default synchronous=FULL is much slower. NORMAL is sufficient for WAL mode.
conn.pragma_update(None, "synchronous", &"NORMAL".to_string())?;
}
// (1) update low-level database structure.
// this should be done before updates that use high-level objects that
// rely themselves on the low-level structure.
let (recalc_fingerprints, update_icons, disable_server_delete, recode_avatar) =
migrations::run(context, self).await?;
// (2) updates that require high-level objects
// the structure is complete now and all objects are usable
if recalc_fingerprints {
info!(context, "[migration] recalc fingerprints");
let addrs = self
.query_map(
"SELECT addr FROM acpeerstates;",
paramsv![],
|row| row.get::<_, String>(0),
|addrs| {
addrs
.collect::<std::result::Result<Vec<_>, _>>()
.map_err(Into::into)
},
)
.await?;
for addr in &addrs {
if let Some(ref mut peerstate) = Peerstate::from_addr(context, addr).await? {
peerstate.recalc_fingerprint();
peerstate.save_to_db(self, false).await?;
}
}
}
if update_icons {
update_saved_messages_icon(context).await?;
update_device_icon(context).await?;
}
if disable_server_delete {
// We now always watch all folders and delete messages there if delete_server is enabled.
// So, for people who have delete_server enabled, disable it and add a hint to the devicechat:
if context.get_config_delete_server_after().await?.is_some() {
let mut msg = Message::new(Viewtype::Text);
msg.text = Some(stock_str::delete_server_turned_off(context).await);
add_device_msg(context, None, Some(&mut msg)).await?;
context
.set_config(Config::DeleteServerAfter, Some("0"))
.await?;
}
}
if recode_avatar {
if let Some(avatar) = context.get_config(Config::Selfavatar).await? {
let mut blob = BlobObject::new_from_path(context, avatar.as_ref()).await?;
match blob.recode_to_avatar_size(context).await {
Ok(()) => {
context
.set_config(Config::Selfavatar, Some(&avatar))
.await?
}
Err(e) => {
warn!(context, "Migrations can't recode avatar, removing. {:#}", e);
context.set_config(Config::Selfavatar, None).await?
}
}
}
}
}
info!(context, "Opened database {:?}.", dbfile);
Ok(())
}
/// Execute the given query, returning the number of affected rows.
@@ -334,10 +219,10 @@ impl Sql {
&self,
query: impl AsRef<str>,
params: impl rusqlite::Params,
) -> Result<i64> {
) -> anyhow::Result<usize> {
let conn = self.get_conn().await?;
conn.execute(query.as_ref(), params)?;
Ok(conn.last_insert_rowid())
Ok(usize::try_from(conn.last_insert_rowid())?)
}
/// Prepares and executes the statement and maps a function over the resulting rows.
@@ -366,7 +251,9 @@ impl Sql {
&self,
) -> Result<r2d2::PooledConnection<r2d2_sqlite::SqliteConnectionManager>> {
let lock = self.pool.read().await;
let pool = lock.as_ref().context("no SQL connection")?;
let pool = lock
.as_ref()
.ok_or_else(|| format_err!("No SQL connection"))?;
let conn = pool.get()?;
Ok(conn)
@@ -500,8 +387,6 @@ impl Sql {
/// will already have been logged.
pub async fn set_raw_config(&self, key: impl AsRef<str>, value: Option<&str>) -> Result<()> {
let key = key.as_ref();
let mut lock = self.config_cache.write().await;
if let Some(value) = value {
let exists = self
.exists(
@@ -527,23 +412,12 @@ impl Sql {
self.execute("DELETE FROM config WHERE keyname=?;", paramsv![key])
.await?;
}
lock.insert(key.to_string(), value.map(|s| s.to_string()));
drop(lock);
Ok(())
}
/// Get configuration options from the database.
pub async fn get_raw_config(&self, key: impl AsRef<str>) -> Result<Option<String>> {
let lock = self.config_cache.read().await;
let cached = lock.get(key.as_ref()).cloned();
drop(lock);
if let Some(c) = cached {
return Ok(c);
}
let mut lock = self.config_cache.write().await;
let value = self
.query_get_value(
"SELECT value FROM config WHERE keyname=?;",
@@ -551,8 +425,6 @@ impl Sql {
)
.await
.context(format!("failed to fetch raw config: {}", key.as_ref()))?;
lock.insert(key.as_ref().to_string(), value.clone());
drop(lock);
Ok(value)
}
@@ -591,11 +463,6 @@ impl Sql {
.await
.map(|s| s.and_then(|r| r.parse().ok()))
}
#[cfg(feature = "internals")]
pub fn config_cache(&self) -> &RwLock<HashMap<String, Option<String>>> {
&self.config_cache
}
}
pub async fn housekeeping(context: &Context) -> Result<()> {
@@ -737,16 +604,6 @@ pub async fn housekeeping(context: &Context) -> Result<()> {
context.schedule_quota_update().await?;
// Try to clear the freelist to free some space on the disk. This
// only works if auto_vacuum is enabled.
if let Err(err) = context
.sql
.execute("PRAGMA incremental_vacuum", paramsv![])
.await
{
warn!(context, "Failed to run incremental vacuum: {}", err);
}
if let Err(e) = context
.set_config(Config::LastHousekeeping, Some(&time().to_string()))
.await
@@ -809,37 +666,21 @@ async fn maybe_add_from_param(
/// have a server UID.
async fn prune_tombstones(sql: &Sql) -> Result<()> {
sql.execute(
"DELETE FROM msgs
WHERE chat_id=?
AND NOT EXISTS (
SELECT * FROM imap WHERE msgs.rfc724_mid=rfc724_mid AND target!=''
)",
"DELETE FROM msgs \
WHERE (chat_id = ? OR hidden) \
AND server_uid = 0",
paramsv![DC_CHAT_ID_TRASH],
)
.await?;
Ok(())
}
/// Helper function to return comma-separated sequence of `?` chars.
///
/// Use this together with [`rusqlite::ParamsFromIter`] to use dynamically generated
/// parameter lists.
pub fn repeat_vars(count: usize) -> Result<String> {
if count == 0 {
bail!("Must have at least one repeat variable");
}
let mut s = "?,".repeat(count);
s.pop(); // Remove trailing comma
Ok(s)
}
#[cfg(test)]
mod tests {
use async_std::channel;
mod test {
use async_std::fs::File;
use crate::config::Config;
use crate::{test_utils::TestContext, EventType};
use crate::{test_utils::TestContext, Event, EventType};
use super::*;
@@ -884,22 +725,6 @@ mod tests {
assert!(!t.ctx.sql.col_exists("foobar", "foobar").await.unwrap());
}
/// Tests that auto_vacuum is enabled for new databases.
#[async_std::test]
async fn test_auto_vacuum() -> Result<()> {
let t = TestContext::new().await;
let conn = t.sql.get_conn().await?;
let auto_vacuum = conn.pragma_query_value(None, "auto_vacuum", |row| {
let auto_vacuum: i32 = row.get(0)?;
Ok(auto_vacuum)
})?;
// auto_vacuum=2 is the same as auto_vacuum=INCREMENTAL
assert_eq!(auto_vacuum, 2);
Ok(())
}
#[async_std::test]
async fn test_housekeeping_db_closed() {
let t = TestContext::new().await;
@@ -916,20 +741,7 @@ mod tests {
.await
.unwrap();
let (event_sink, event_source) = channel::unbounded();
t.add_event_sender(event_sink).await;
let a = t.get_config(Config::Selfavatar).await.unwrap().unwrap();
assert_eq!(avatar_bytes, &async_std::fs::read(&a).await.unwrap()[..]);
t.sql.close().await;
housekeeping(&t).await.unwrap_err(); // housekeeping should fail as the db is closed
t.sql.open(&t, "".to_string()).await.unwrap();
let a = t.get_config(Config::Selfavatar).await.unwrap().unwrap();
assert_eq!(avatar_bytes, &async_std::fs::read(&a).await.unwrap()[..]);
while let Ok(event) = event_source.try_recv() {
t.add_event_sink(move |event: Event| async move {
match event.typ {
EventType::Info(s) => assert!(
!s.contains("Keeping new unreferenced file"),
@@ -939,24 +751,18 @@ mod tests {
EventType::Error(s) => panic!("{}", s),
_ => {}
}
}
}
})
.await;
/// Regression test for a bug where housekeeping deleted drafts since their
/// `hidden` flag is set.
#[async_std::test]
async fn test_housekeeping_dont_delete_drafts() {
let t = TestContext::new_alice().await;
let a = t.get_config(Config::Selfavatar).await.unwrap().unwrap();
assert_eq!(avatar_bytes, &async_std::fs::read(&a).await.unwrap()[..]);
let chat = t.create_chat_with_contact("bob", "bob@example.com").await;
let mut new_draft = Message::new(Viewtype::Text);
new_draft.set_text(Some("This is my draft".to_string()));
chat.id.set_draft(&t, Some(&mut new_draft)).await.unwrap();
t.sql.close().await;
housekeeping(&t).await.unwrap_err(); // housekeeping should fail as the db is closed
t.sql.open(&t, t.get_dbfile(), false).await.unwrap();
housekeeping(&t).await.unwrap();
let loaded_draft = chat.id.get_draft(&t).await.unwrap();
assert_eq!(loaded_draft.unwrap().text.unwrap(), "This is my draft");
let a = t.get_config(Config::Selfavatar).await.unwrap().unwrap();
assert_eq!(avatar_bytes, &async_std::fs::read(&a).await.unwrap()[..]);
}
/// Regression test.
@@ -981,14 +787,14 @@ mod tests {
// Create a separate empty database for testing.
let dir = tempdir()?;
let dbfile = dir.path().join("testdb.sqlite");
let sql = Sql::new(dbfile.into());
let sql = Sql::new();
// Create database with all the tables.
sql.open(&t, "".to_string()).await.unwrap();
sql.open(&t, dbfile.as_ref(), false).await.unwrap();
sql.close().await;
// Reopen the database
sql.open(&t, "".to_string()).await?;
sql.open(&t, dbfile.as_ref(), false).await?;
sql.execute(
"INSERT INTO config (keyname, value) VALUES (?, ?);",
paramsv!("foo", "bar"),
@@ -1018,59 +824,20 @@ mod tests {
assert!(!disable_server_delete);
assert!(!recode_avatar);
info!(&t, "test_migration_flags: XXX END MARKER");
info!(&t, "test_migration_flags: XXX");
loop {
let evt = t
.evtracker
.get_matching(|evt| matches!(evt, EventType::Info(_)))
.await;
match evt {
EventType::Info(msg) => {
assert!(
!msg.contains("[migration]"),
"Migrations were run twice, you probably forgot to update the db version"
);
if msg.contains("test_migration_flags: XXX END MARKER") {
break;
}
if let EventType::Info(info) = t.evtracker.recv().await.unwrap() {
assert!(
!info.contains("[migration]"),
"Migrations were run twice, you probably forgot to update the db version"
);
if info.contains("test_migration_flags: XXX") {
break;
}
_ => unreachable!(),
}
}
Ok(())
}
#[async_std::test]
async fn test_check_passphrase() -> Result<()> {
use tempfile::tempdir;
// The context is used only for logging.
let t = TestContext::new().await;
// Create a separate empty database for testing.
let dir = tempdir()?;
let dbfile = dir.path().join("testdb.sqlite");
let sql = Sql::new(dbfile.clone().into());
sql.check_passphrase("foo".to_string()).await?;
sql.open(&t, "foo".to_string())
.await
.context("failed to open the database first time")?;
sql.close().await;
// Reopen the database
let sql = Sql::new(dbfile.into());
// Test that we can't open encrypted database without a passphrase.
assert!(sql.open(&t, "".to_string()).await.is_err());
// Now open the database with passpharse, it should succeed.
sql.check_passphrase("foo".to_string()).await?;
sql.open(&t, "foo".to_string())
.await
.context("failed to open the database second time")?;
Ok(())
}
}

View File

@@ -1,6 +1,6 @@
//! Migrations module.
use anyhow::{Context as _, Result};
use anyhow::Result;
use crate::config::Config;
use crate::constants::ShowEmails;
@@ -19,11 +19,7 @@ pub async fn run(context: &Context, sql: &Sql) -> Result<(bool, bool, bool, bool
let mut exists_before_update = false;
let mut dbversion_before_update = DBVERSION;
if !sql
.table_exists("config")
.await
.context("failed to check if config table exists")?
{
if !sql.table_exists("config").await? {
info!(context, "First time init: creating tables",);
sql.transaction(move |transaction| {
transaction.execute_batch(TABLES)?;
@@ -36,13 +32,6 @@ pub async fn run(context: &Context, sql: &Sql) -> Result<(bool, bool, bool, bool
Ok(())
})
.await?;
let mut lock = context.sql.config_cache.write().await;
lock.insert(
VERSION_CFG.to_string(),
Some(format!("{}", dbversion_before_update)),
);
drop(lock);
} else {
exists_before_update = true;
dbversion_before_update = sql
@@ -508,106 +497,6 @@ item TEXT DEFAULT '');"#,
)
.await?;
}
if dbversion < 81 {
info!(context, "[migration] v81");
sql.execute_migration("ALTER TABLE msgs ADD COLUMN hop_info TEXT;", 81)
.await?;
}
if dbversion < 82 {
info!(context, "[migration] v82");
sql.execute_migration(
r#"CREATE TABLE imap (
id INTEGER PRIMARY KEY AUTOINCREMENT,
rfc724_mid TEXT DEFAULT '', -- Message-ID header
folder TEXT DEFAULT '', -- IMAP folder
target TEXT DEFAULT '', -- Destination folder, empty to delete.
uid INTEGER DEFAULT 0, -- UID
uidvalidity INTEGER DEFAULT 0,
UNIQUE (folder, uid, uidvalidity)
);
CREATE INDEX imap_folder ON imap(folder);
CREATE INDEX imap_messageid ON imap(rfc724_mid);
INSERT INTO imap
(rfc724_mid, folder, target, uid, uidvalidity)
SELECT
rfc724_mid,
server_folder AS folder,
server_folder AS target,
server_uid AS uid,
(SELECT uidvalidity FROM imap_sync WHERE folder=server_folder) AS uidvalidity
FROM msgs
WHERE server_uid>0
ON CONFLICT (folder, uid, uidvalidity)
DO UPDATE SET rfc724_mid=excluded.rfc724_mid,
target=excluded.target;
"#,
82,
)
.await?;
}
if dbversion < 83 {
info!(context, "[migration] v83");
sql.execute_migration(
"ALTER TABLE imap_sync
ADD COLUMN modseq -- Highest modification sequence
INTEGER DEFAULT 0",
83,
)
.await?;
}
if dbversion < 84 {
info!(context, "[migration] v84");
sql.execute_migration(
r#"CREATE TABLE msgs_status_updates (
id INTEGER PRIMARY KEY AUTOINCREMENT,
msg_id INTEGER,
update_item TEXT DEFAULT '',
update_item_read INTEGER DEFAULT 0);
CREATE INDEX msgs_status_updates_index1 ON msgs_status_updates (msg_id);"#,
84,
)
.await?;
}
if dbversion < 85 {
info!(context, "[migration] v85");
sql.execute_migration(
r#"CREATE TABLE smtp (
id INTEGER PRIMARY KEY,
rfc724_mid TEXT NOT NULL, -- Message-ID
mime TEXT NOT NULL, -- SMTP payload
msg_id INTEGER NOT NULL, -- ID of the message in `msgs` table
recipients TEXT NOT NULL, -- List of recipients separated by space
retries INTEGER NOT NULL DEFAULT 0 -- Number of failed attempts to send the messsage
);
CREATE INDEX smtp_messageid ON imap(rfc724_mid);
"#,
85,
)
.await?;
}
if dbversion < 86 {
info!(context, "[migration] v86");
sql.execute_migration(
r#"CREATE TABLE bobstate (
id INTEGER PRIMARY KEY AUTOINCREMENT,
invite TEXT NOT NULL,
next_step INTEGER NOT NULL,
chat_id INTEGER NOT NULL
);"#,
86,
)
.await?;
}
if dbversion < 87 {
info!(context, "[migration] v87");
// the index is used to speed up delete_expired_messages()
sql.execute_migration(
"CREATE INDEX IF NOT EXISTS msgs_index8 ON msgs (ephemeral_timestamp);",
87,
)
.await?;
}
Ok((
recalc_fingerprints,
@@ -635,12 +524,7 @@ impl Sql {
Ok(())
})
.await
.with_context(|| format!("execute_migration failed for version {}", version))?;
let mut lock = self.config_cache.write().await;
lock.insert(VERSION_CFG.to_string(), Some(format!("{}", version)));
drop(lock);
.await?;
Ok(())
}

View File

@@ -168,14 +168,6 @@ CREATE TABLE tokens (
timestamp INTEGER DEFAULT 0
);
-- The currently running securejoin protocols, joiner-side.
-- CREATE TABLE bobstate (
-- id INTEGER PRIMARY KEY AUTOINCREMENT,
-- invite TEXT NOT NULL,
-- next_step INTEGER NOT NULL,
-- chat_id INTEGER NOT NULL
-- );
CREATE TABLE locations (
id INTEGER PRIMARY KEY AUTOINCREMENT,
latitude REAL DEFAULT 0.0,

View File

@@ -10,11 +10,11 @@ use strum_macros::EnumProperty;
use crate::blob::BlobObject;
use crate::chat::{self, Chat, ChatId, ProtectionStatus};
use crate::config::Config;
use crate::constants::DC_CONTACT_ID_SELF;
use crate::contact::{Contact, ContactId, Origin};
use crate::constants::{Viewtype, DC_CONTACT_ID_SELF};
use crate::contact::{Contact, Origin};
use crate::context::Context;
use crate::dc_tools::dc_timestamp_to_str;
use crate::message::{Message, Viewtype};
use crate::message::Message;
use crate::param::Param;
use humansize::{file_size_opts, FileSize};
@@ -53,6 +53,9 @@ pub enum StockMessage {
#[strum(props(fallback = "File"))]
File = 12,
#[strum(props(fallback = "Sent with my Delta Chat Messenger: https://delta.chat"))]
StatusLine = 13,
#[strum(props(fallback = "Group name changed from \"%1$s\" to \"%2$s\"."))]
MsgGrpName = 15,
@@ -389,7 +392,7 @@ trait StockStringMods: AsRef<str> + Sized {
fn action_by_contact<'a>(
self,
context: &'a Context,
contact_id: ContactId,
contact_id: u32,
) -> Pin<Box<dyn Future<Output = String> + Send + 'a>>
where
Self: Send + 'a,
@@ -452,12 +455,17 @@ pub(crate) async fn file(context: &Context) -> String {
translated(context, StockMessage::File).await
}
/// Stock string: `Sent with my Delta Chat Messenger: https://delta.chat`.
pub(crate) async fn status_line(context: &Context) -> String {
translated(context, StockMessage::StatusLine).await
}
/// Stock string: `Group name changed from "%1$s" to "%2$s".`.
pub(crate) async fn msg_grp_name(
context: &Context,
from_group: impl AsRef<str>,
to_group: impl AsRef<str>,
by_contact: ContactId,
by_contact: u32,
) -> String {
translated(context, StockMessage::MsgGrpName)
.await
@@ -468,7 +476,7 @@ pub(crate) async fn msg_grp_name(
}
/// Stock string: `Group image changed.`.
pub(crate) async fn msg_grp_img_changed(context: &Context, by_contact: ContactId) -> String {
pub(crate) async fn msg_grp_img_changed(context: &Context, by_contact: u32) -> String {
translated(context, StockMessage::MsgGrpImgChanged)
.await
.action_by_contact(context, by_contact)
@@ -482,7 +490,7 @@ pub(crate) async fn msg_grp_img_changed(context: &Context, by_contact: ContactId
pub(crate) async fn msg_add_member(
context: &Context,
added_member_addr: impl AsRef<str>,
by_contact: ContactId,
by_contact: u32,
) -> String {
let addr = added_member_addr.as_ref();
let who = match Contact::lookup_id_by_addr(context, addr, Origin::Unknown).await {
@@ -506,7 +514,7 @@ pub(crate) async fn msg_add_member(
pub(crate) async fn msg_del_member(
context: &Context,
removed_member_addr: impl AsRef<str>,
by_contact: ContactId,
by_contact: u32,
) -> String {
let addr = removed_member_addr.as_ref();
let who = match Contact::lookup_id_by_addr(context, addr, Origin::Unknown).await {
@@ -524,7 +532,7 @@ pub(crate) async fn msg_del_member(
}
/// Stock string: `Group left.`.
pub(crate) async fn msg_group_left(context: &Context, by_contact: ContactId) -> String {
pub(crate) async fn msg_group_left(context: &Context, by_contact: u32) -> String {
translated(context, StockMessage::MsgGroupLeft)
.await
.action_by_contact(context, by_contact)
@@ -574,7 +582,7 @@ pub(crate) async fn read_rcpt_mail_body(context: &Context, message: impl AsRef<s
}
/// Stock string: `Group image deleted.`.
pub(crate) async fn msg_grp_img_deleted(context: &Context, by_contact: ContactId) -> String {
pub(crate) async fn msg_grp_img_deleted(context: &Context, by_contact: u32) -> String {
translated(context, StockMessage::MsgGrpImgDeleted)
.await
.action_by_contact(context, by_contact)
@@ -587,10 +595,7 @@ pub(crate) async fn e2e_preferred(context: &Context) -> String {
}
/// Stock string: `%1$s invited you to join this group. Waiting for the device of %2$s to reply…`.
pub(crate) async fn secure_join_started(
context: &Context,
inviter_contact_id: ContactId,
) -> String {
pub(crate) async fn secure_join_started(context: &Context, inviter_contact_id: u32) -> String {
if let Ok(contact) = Contact::get_by_id(context, inviter_contact_id).await {
translated(context, StockMessage::SecureJoinStarted)
.await
@@ -605,7 +610,7 @@ pub(crate) async fn secure_join_started(
}
/// Stock string: `%1$s replied, waiting for being added to the group…`.
pub(crate) async fn secure_join_replies(context: &Context, contact_id: ContactId) -> String {
pub(crate) async fn secure_join_replies(context: &Context, contact_id: u32) -> String {
if let Ok(contact) = Contact::get_by_id(context, contact_id).await {
translated(context, StockMessage::SecureJoinReplies)
.await
@@ -639,19 +644,20 @@ pub(crate) async fn secure_join_group_qr_description(context: &Context, chat: &C
}
/// Stock string: `%1$s verified.`.
pub(crate) async fn contact_verified(context: &Context, contact: &Contact) -> String {
let addr = contact.get_name_n_addr();
pub(crate) async fn contact_verified(context: &Context, contact_addr: impl AsRef<str>) -> String {
translated(context, StockMessage::ContactVerified)
.await
.replace1(addr)
.replace1(contact_addr)
}
/// Stock string: `Cannot verify %1$s`.
pub(crate) async fn contact_not_verified(context: &Context, contact: &Contact) -> String {
let addr = contact.get_name_n_addr();
pub(crate) async fn contact_not_verified(
context: &Context,
contact_addr: impl AsRef<str>,
) -> String {
translated(context, StockMessage::ContactNotVerified)
.await
.replace1(addr)
.replace1(contact_addr)
}
/// Stock string: `Changed setup for %1$s`.
@@ -721,7 +727,7 @@ pub(crate) async fn msg_location_enabled(context: &Context) -> String {
}
/// Stock string: `Location streaming enabled by ...`.
pub(crate) async fn msg_location_enabled_by(context: &Context, contact: ContactId) -> String {
pub(crate) async fn msg_location_enabled_by(context: &Context, contact: u32) -> String {
translated(context, StockMessage::MsgLocationEnabled)
.await
.action_by_contact(context, contact)
@@ -787,10 +793,7 @@ pub(crate) async fn failed_sending_to(context: &Context, name: impl AsRef<str>)
}
/// Stock string: `Message deletion timer is disabled.`.
pub(crate) async fn msg_ephemeral_timer_disabled(
context: &Context,
by_contact: ContactId,
) -> String {
pub(crate) async fn msg_ephemeral_timer_disabled(context: &Context, by_contact: u32) -> String {
translated(context, StockMessage::MsgEphemeralTimerDisabled)
.await
.action_by_contact(context, by_contact)
@@ -801,7 +804,7 @@ pub(crate) async fn msg_ephemeral_timer_disabled(
pub(crate) async fn msg_ephemeral_timer_enabled(
context: &Context,
timer: impl AsRef<str>,
by_contact: ContactId,
by_contact: u32,
) -> String {
translated(context, StockMessage::MsgEphemeralTimerEnabled)
.await
@@ -811,7 +814,7 @@ pub(crate) async fn msg_ephemeral_timer_enabled(
}
/// Stock string: `Message deletion timer is set to 1 minute.`.
pub(crate) async fn msg_ephemeral_timer_minute(context: &Context, by_contact: ContactId) -> String {
pub(crate) async fn msg_ephemeral_timer_minute(context: &Context, by_contact: u32) -> String {
translated(context, StockMessage::MsgEphemeralTimerMinute)
.await
.action_by_contact(context, by_contact)
@@ -819,7 +822,7 @@ pub(crate) async fn msg_ephemeral_timer_minute(context: &Context, by_contact: Co
}
/// Stock string: `Message deletion timer is set to 1 hour.`.
pub(crate) async fn msg_ephemeral_timer_hour(context: &Context, by_contact: ContactId) -> String {
pub(crate) async fn msg_ephemeral_timer_hour(context: &Context, by_contact: u32) -> String {
translated(context, StockMessage::MsgEphemeralTimerHour)
.await
.action_by_contact(context, by_contact)
@@ -827,7 +830,7 @@ pub(crate) async fn msg_ephemeral_timer_hour(context: &Context, by_contact: Cont
}
/// Stock string: `Message deletion timer is set to 1 day.`.
pub(crate) async fn msg_ephemeral_timer_day(context: &Context, by_contact: ContactId) -> String {
pub(crate) async fn msg_ephemeral_timer_day(context: &Context, by_contact: u32) -> String {
translated(context, StockMessage::MsgEphemeralTimerDay)
.await
.action_by_contact(context, by_contact)
@@ -835,7 +838,7 @@ pub(crate) async fn msg_ephemeral_timer_day(context: &Context, by_contact: Conta
}
/// Stock string: `Message deletion timer is set to 1 week.`.
pub(crate) async fn msg_ephemeral_timer_week(context: &Context, by_contact: ContactId) -> String {
pub(crate) async fn msg_ephemeral_timer_week(context: &Context, by_contact: u32) -> String {
translated(context, StockMessage::MsgEphemeralTimerWeek)
.await
.action_by_contact(context, by_contact)
@@ -880,7 +883,7 @@ pub(crate) async fn error_no_network(context: &Context) -> String {
}
/// Stock string: `Chat protection enabled.`.
pub(crate) async fn protection_enabled(context: &Context, by_contact: ContactId) -> String {
pub(crate) async fn protection_enabled(context: &Context, by_contact: u32) -> String {
translated(context, StockMessage::ProtectionEnabled)
.await
.action_by_contact(context, by_contact)
@@ -888,7 +891,7 @@ pub(crate) async fn protection_enabled(context: &Context, by_contact: ContactId)
}
/// Stock string: `Chat protection disabled.`.
pub(crate) async fn protection_disabled(context: &Context, by_contact: ContactId) -> String {
pub(crate) async fn protection_disabled(context: &Context, by_contact: u32) -> String {
translated(context, StockMessage::ProtectionDisabled)
.await
.action_by_contact(context, by_contact)
@@ -914,7 +917,7 @@ pub(crate) async fn delete_server_turned_off(context: &Context) -> String {
pub(crate) async fn msg_ephemeral_timer_minutes(
context: &Context,
minutes: impl AsRef<str>,
by_contact: ContactId,
by_contact: u32,
) -> String {
translated(context, StockMessage::MsgEphemeralTimerMinutes)
.await
@@ -927,7 +930,7 @@ pub(crate) async fn msg_ephemeral_timer_minutes(
pub(crate) async fn msg_ephemeral_timer_hours(
context: &Context,
hours: impl AsRef<str>,
by_contact: ContactId,
by_contact: u32,
) -> String {
translated(context, StockMessage::MsgEphemeralTimerHours)
.await
@@ -940,7 +943,7 @@ pub(crate) async fn msg_ephemeral_timer_hours(
pub(crate) async fn msg_ephemeral_timer_days(
context: &Context,
days: impl AsRef<str>,
by_contact: ContactId,
by_contact: u32,
) -> String {
translated(context, StockMessage::MsgEphemeralTimerDays)
.await
@@ -953,7 +956,7 @@ pub(crate) async fn msg_ephemeral_timer_days(
pub(crate) async fn msg_ephemeral_timer_weeks(
context: &Context,
weeks: impl AsRef<str>,
by_contact: ContactId,
by_contact: u32,
) -> String {
translated(context, StockMessage::MsgEphemeralTimerWeeks)
.await
@@ -1110,7 +1113,7 @@ impl Context {
pub(crate) async fn stock_protection_msg(
&self,
protect: ProtectionStatus,
from_id: ContactId,
from_id: u32,
) -> String {
match protect {
ProtectionStatus::Unprotected => protection_enabled(self, from_id).await,
@@ -1202,15 +1205,8 @@ mod tests {
#[async_std::test]
async fn test_stock_string_repl_str() {
let t = TestContext::new().await;
let contact_id = Contact::create(&t.ctx, "Someone", "someone@example.org")
.await
.unwrap();
let contact = Contact::load_from_db(&t.ctx, contact_id).await.unwrap();
// uses %1$s substitution
assert_eq!(
contact_verified(&t, &contact).await,
"Someone (someone@example.org) verified."
);
assert_eq!(contact_verified(&t, "Foo").await, "Foo verified.");
// We have no string using %1$d to test...
}
@@ -1233,20 +1229,20 @@ mod tests {
async fn test_stock_system_msg_add_member_by_me() {
let t = TestContext::new().await;
assert_eq!(
msg_add_member(&t, "alice@example.org", DC_CONTACT_ID_SELF).await,
"Member alice@example.org added by me."
msg_add_member(&t, "alice@example.com", DC_CONTACT_ID_SELF).await,
"Member alice@example.com added by me."
)
}
#[async_std::test]
async fn test_stock_system_msg_add_member_by_me_with_displayname() {
let t = TestContext::new().await;
Contact::create(&t, "Alice", "alice@example.org")
Contact::create(&t, "Alice", "alice@example.com")
.await
.expect("failed to create contact");
assert_eq!(
msg_add_member(&t, "alice@example.org", DC_CONTACT_ID_SELF).await,
"Member Alice (alice@example.org) added by me."
msg_add_member(&t, "alice@example.com", DC_CONTACT_ID_SELF).await,
"Member Alice (alice@example.com) added by me."
);
}
@@ -1254,7 +1250,7 @@ mod tests {
async fn test_stock_system_msg_add_member_by_other_with_displayname() {
let t = TestContext::new().await;
let contact_id = {
Contact::create(&t, "Alice", "alice@example.org")
Contact::create(&t, "Alice", "alice@example.com")
.await
.expect("Failed to create contact Alice");
Contact::create(&t, "Bob", "bob@example.com")
@@ -1262,8 +1258,8 @@ mod tests {
.expect("failed to create bob")
};
assert_eq!(
msg_add_member(&t, "alice@example.org", contact_id,).await,
"Member Alice (alice@example.org) added by Bob (bob@example.com)."
msg_add_member(&t, "alice@example.com", contact_id,).await,
"Member Alice (alice@example.com) added by Bob (bob@example.com)."
);
}
@@ -1292,13 +1288,11 @@ mod tests {
let chats = Chatlist::try_load(&t, 0, None, None).await.unwrap();
assert_eq!(chats.len(), 2);
let chat0 = Chat::load_from_db(&t, chats.get_chat_id(0).unwrap())
.await
.unwrap();
let chat0 = Chat::load_from_db(&t, chats.get_chat_id(0)).await.unwrap();
let (self_talk_id, device_chat_id) = if chat0.is_self_talk() {
(chats.get_chat_id(0).unwrap(), chats.get_chat_id(1).unwrap())
(chats.get_chat_id(0), chats.get_chat_id(1))
} else {
(chats.get_chat_id(1).unwrap(), chats.get_chat_id(0).unwrap())
(chats.get_chat_id(1), chats.get_chat_id(0))
};
// delete self-talk first; this adds a message to device-chat about how self-talk can be restored

View File

@@ -1,11 +1,11 @@
//! # Message summary for chatlist.
use crate::chat::Chat;
use crate::constants::{Chattype, DC_CONTACT_ID_SELF};
use crate::constants::{Chattype, Viewtype, DC_CONTACT_ID_SELF};
use crate::contact::Contact;
use crate::context::Context;
use crate::dc_tools::dc_truncate;
use crate::message::{Message, MessageState, Viewtype};
use crate::message::{Message, MessageState};
use crate::mimeparser::SystemMessage;
use crate::param::Param;
use crate::stock_str;
@@ -137,14 +137,7 @@ impl Message {
append_text = false;
stock_str::videochat_invitation(context).await
}
Viewtype::Webxdc => {
append_text = true;
self.get_webxdc_info(context)
.await
.map(|info| info.name)
.unwrap_or_else(|_| "ErrWebxdcName".to_string())
}
Viewtype::Text | Viewtype::Unknown => {
_ => {
if self.param.get_cmd() != SystemMessage::LocationOnly {
"".to_string()
} else {

View File

@@ -2,10 +2,10 @@
use crate::chat::{Chat, ChatId};
use crate::config::Config;
use crate::constants::{Blocked, DC_CONTACT_ID_SELF};
use crate::constants::{Blocked, Viewtype, DC_CONTACT_ID_SELF};
use crate::context::Context;
use crate::dc_tools::time;
use crate::message::{Message, MsgId, Viewtype};
use crate::message::{Message, MsgId};
use crate::mimeparser::SystemMessage;
use crate::param::Param;
use crate::sync::SyncData::{AddQrToken, DeleteQrToken};

View File

@@ -2,161 +2,76 @@
//!
//! This private module is only compiled for test runs.
use std::collections::BTreeMap;
use std::ops::Deref;
use std::panic;
use std::thread;
use std::str::FromStr;
use std::time::{Duration, Instant};
use std::{collections::BTreeMap, panic};
use std::{fmt, thread};
use ansi_term::Color;
use async_std::channel::{self, Receiver, Sender};
use async_std::prelude::*;
use async_std::channel::Receiver;
use async_std::path::PathBuf;
use async_std::sync::{Arc, RwLock};
use async_std::task;
use async_std::{channel, pin::Pin};
use async_std::{future::Future, task};
use chat::ChatItem;
use once_cell::sync::Lazy;
use rand::Rng;
use tempfile::{tempdir, TempDir};
use crate::chat::{self, Chat, ChatId};
use crate::chatlist::Chatlist;
use crate::config::Config;
use crate::constants::Chattype;
use crate::constants::{DC_CONTACT_ID_SELF, DC_MSG_ID_DAYMARKER, DC_MSG_ID_MARKER1};
use crate::contact::{Contact, Modifier, Origin};
use crate::constants::{Viewtype, DC_CONTACT_ID_SELF, DC_MSG_ID_DAYMARKER, DC_MSG_ID_MARKER1};
use crate::contact::{Contact, Origin};
use crate::context::Context;
use crate::dc_receive_imf::dc_receive_imf;
use crate::dc_tools::EmailAddress;
use crate::events::{Event, EventType};
use crate::key::{self, DcKey, KeyPair, KeyPairUse};
use crate::message::{update_msg_state, Message, MessageState, MsgId, Viewtype};
use crate::job::Action;
use crate::key::{self, DcKey};
use crate::message::{update_msg_state, Message, MessageState, MsgId};
use crate::mimeparser::MimeMessage;
use crate::param::{Param, Params};
#[allow(non_upper_case_globals)]
pub const AVATAR_900x900_BYTES: &[u8] = include_bytes!("../test-data/image/avatar900x900.png");
type EventSink =
dyn Fn(Event) -> Pin<Box<dyn Future<Output = ()> + Send + 'static>> + Send + Sync + 'static;
/// Map of [`Context::id`] to names for [`TestContext`]s.
static CONTEXT_NAMES: Lazy<std::sync::RwLock<BTreeMap<u32, String>>> =
Lazy::new(|| std::sync::RwLock::new(BTreeMap::new()));
pub struct TestContextManager {
log_tx: Sender<Event>,
_log_sink: LogSink,
}
impl TestContextManager {
pub async fn new() -> Self {
let (log_tx, _log_sink) = LogSink::create();
Self { log_tx, _log_sink }
}
pub async fn alice(&mut self) -> TestContext {
TestContext::builder()
.configure_alice()
.with_log_sink(self.log_tx.clone())
.build()
.await
}
pub async fn bob(&mut self) -> TestContext {
TestContext::builder()
.configure_bob()
.with_log_sink(self.log_tx.clone())
.build()
.await
}
}
#[derive(Debug, Clone, Default)]
pub struct TestContextBuilder {
key_pair: Option<KeyPair>,
log_sink: Option<Sender<Event>>,
}
impl TestContextBuilder {
/// Configures as alice@example.org with fixed secret key.
///
/// This is a shortcut for `.with_key_pair(alice_keypair()).
pub fn configure_alice(self) -> Self {
self.with_key_pair(alice_keypair())
}
/// Configures as bob@example.net with fixed secret key.
///
/// This is a shortcut for `.with_key_pair(bob_keypair()).
pub fn configure_bob(self) -> Self {
self.with_key_pair(bob_keypair())
}
/// Configures the new [`TestContext`] with the provided [`KeyPair`].
///
/// This will extract the email address from the key and configure the context with the
/// given identity.
pub fn with_key_pair(mut self, key_pair: KeyPair) -> Self {
self.key_pair = Some(key_pair);
self
}
/// Attaches a [`LogSink`] to this [`TestContext`].
///
/// This is useful when using multiple [`TestContext`] instances in one test: it allows
/// using a single [`LogSink`] for both contexts. This shows the log messages in
/// sequence as they occurred rather than all messages from each context in a single
/// block.
pub fn with_log_sink(mut self, sink: Sender<Event>) -> Self {
self.log_sink = Some(sink);
self
}
/// Builds the [`TestContext`].
pub async fn build(self) -> TestContext {
let name = self.key_pair.as_ref().map(|key| key.addr.local.clone());
let test_context = TestContext::new_internal(name, self.log_sink).await;
if let Some(key_pair) = self.key_pair {
test_context
.configure_addr(&key_pair.addr.to_string())
.await;
key::store_self_keypair(&test_context, &key_pair, KeyPairUse::Default)
.await
.expect("Failed to save key");
}
test_context
}
}
/// A Context and temporary directory.
///
/// The temporary directory can be used to store the SQLite database,
/// see e.g. [test_context] which does this.
#[derive(Debug)]
pub struct TestContext {
pub(crate) struct TestContext {
pub ctx: Context,
pub dir: TempDir,
pub evtracker: EventTracker,
/// Channels which should receive events from this context.
event_senders: Arc<RwLock<Vec<Sender<Event>>>>,
pub evtracker: EvTracker,
/// Counter for fake IMAP UIDs in [recv_msg], for private use in that function only.
recv_idx: RwLock<u32>,
/// Functions to call for events received.
event_sinks: Arc<RwLock<Vec<Box<EventSink>>>>,
/// Receives panics from sinks ("sink" means "event handler" here)
poison_receiver: Receiver<String>,
/// Reference to implicit [`LogSink`] so it is dropped together with the context.
///
/// Only used if no explicit `log_sender` is passed into [`TestContext::new_internal`]
/// (which is assumed to be the sending end of a [`LogSink`]).
///
/// This is a convenience in case only a single [`TestContext`] is used to avoid dealing
/// with [`LogSink`]. Never read, thus "dead code", since the only purpose is to
/// control when Drop is invoked.
#[allow(dead_code)]
log_sink: Option<LogSink>,
poison_receiver: channel::Receiver<String>,
}
impl fmt::Debug for TestContext {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
f.debug_struct("TestContext")
.field("ctx", &self.ctx)
.field("dir", &self.dir)
.field("recv_idx", &self.recv_idx)
.field("event_sinks", &String::from("Vec<EventSink>"))
.finish()
}
}
impl TestContext {
/// Returns the builder to have more control over creating the context.
pub fn builder() -> TestContextBuilder {
TestContextBuilder::default()
}
/// Creates a new [`TestContext`].
///
/// The [Context] will be created and have an SQLite database named "db.sqlite" in the
@@ -165,33 +80,18 @@ impl TestContext {
///
/// [Context]: crate::context::Context
pub async fn new() -> Self {
Self::new_internal(None, None).await
Self::new_named(None).await
}
/// Creates a new configured [`TestContext`].
///
/// This is a shortcut which automatically calls [`TestContext::configure_alice`] after
/// creating the context.
pub async fn new_alice() -> Self {
Self::builder().configure_alice().build().await
/// Creates a new [`TestContext`] with a set name used in event logging.
pub async fn with_name(name: impl Into<String>) -> Self {
Self::new_named(Some(name.into())).await
}
/// Creates a new configured [`TestContext`].
///
/// This is a shortcut which configures bob@example.net with a fixed key.
pub async fn new_bob() -> Self {
Self::builder().configure_bob().build().await
}
async fn new_named(name: Option<String>) -> Self {
use rand::Rng;
pretty_env_logger::try_init().ok();
/// Internal constructor.
///
/// `name` is used to identify this context in e.g. log output. This is useful mostly
/// when you have multiple [`TestContext`]s in a test.
///
/// `log_sender` is assumed to be the sender for a [`LogSink`]. If not supplied a new
/// [`LogSink`] will be created so that events are logged to this test when the
/// [`TestContext`] is dropped.
async fn new_internal(name: Option<String>, log_sender: Option<Sender<Event>>) -> Self {
let dir = tempdir().unwrap();
let dbfile = dir.path().join("db.sqlite");
let id = rand::thread_rng().gen();
@@ -199,26 +99,18 @@ impl TestContext {
let mut context_names = CONTEXT_NAMES.write().unwrap();
context_names.insert(id, name);
}
let ctx = Context::new(dbfile.into(), id)
let ctx = Context::new("FakeOS".into(), dbfile.into(), id)
.await
.expect("failed to create context");
let events = ctx.get_event_emitter();
let (log_sender, log_sink) = match log_sender {
Some(sender) => (sender, None),
None => {
let (sender, sink) = LogSink::create();
(sender, Some(sink))
}
};
let (evtracker_sender, evtracker_receiver) = channel::unbounded();
let event_senders = Arc::new(RwLock::new(vec![log_sender, evtracker_sender]));
let senders = Arc::clone(&event_senders);
let event_sinks: Arc<RwLock<Vec<Box<EventSink>>>> = Arc::new(RwLock::new(Vec::new()));
let sinks = Arc::clone(&event_sinks);
let (poison_sender, poison_receiver) = channel::bounded(1);
let (evtracker_sender, evtracker_receiver) = channel::unbounded();
task::spawn(async move {
async_std::task::spawn(async move {
// Make sure that the test fails if there is a panic on this thread here
// (but not if there is a panic on another thread)
let looptask_id = task::current().id();
@@ -234,26 +126,50 @@ impl TestContext {
while let Some(event) = events.recv().await {
{
let sinks = senders.read().await;
for sender in sinks.iter() {
// Don't block because someone wanted to use a oneshot receiver, use
// an unbounded channel if you want all events.
sender.try_send(event.clone()).ok();
log::debug!("{:?}", event);
let sinks = sinks.read().await;
for sink in sinks.iter() {
sink(event.clone()).await;
}
}
receive_event(&event);
evtracker_sender.send(event.typ).await.ok();
}
});
Self {
ctx,
dir,
evtracker: EventTracker(evtracker_receiver),
event_senders,
evtracker: EvTracker(evtracker_receiver),
recv_idx: RwLock::new(0),
event_sinks,
poison_receiver,
log_sink,
}
}
/// Creates a new configured [`TestContext`].
///
/// This is a shortcut which automatically calls [`TestContext::configure_alice`] after
/// creating the context.
pub async fn new_alice() -> Self {
let t = Self::with_name("alice").await;
t.configure_alice().await;
t
}
/// Creates a new configured [`TestContext`].
///
/// This is a shortcut which configures bob@example.net with a fixed key.
pub async fn new_bob() -> Self {
let t = Self::with_name("bob").await;
let keypair = bob_keypair();
t.configure_addr(&keypair.addr.to_string()).await;
key::store_self_keypair(&t, &keypair, key::KeyPairUse::Default)
.await
.expect("Failed to save Bob's key");
t
}
/// Sets a name for this [`TestContext`] if one isn't yet set.
///
/// This will show up in events logged in the test output.
@@ -264,12 +180,32 @@ impl TestContext {
.or_insert_with(|| name.into());
}
/// Adds a new [`Event`]s sender.
/// Add a new callback which will receive events.
///
/// Once added, all events emitted by this context will be sent to this channel. This
/// is useful if you need to wait for events or make assertions on them.
pub async fn add_event_sender(&self, sink: Sender<Event>) {
self.event_senders.write().await.push(sink)
/// The test context runs an async task receiving all events from the [`Context`], which
/// are logged to stdout. This allows you to register additional callbacks which will
/// receive all events in case your tests need to watch for a specific event.
pub async fn add_event_sink<F, R>(&self, sink: F)
where
// Aka `F: EventSink` but type aliases are not allowed.
F: Fn(Event) -> R + Send + Sync + 'static,
R: Future<Output = ()> + Send + 'static,
{
let mut sinks = self.event_sinks.write().await;
sinks.push(Box::new(move |evt| Box::pin(sink(evt))));
}
/// Configure with alice@example.com.
///
/// The context will be fake-configured as the alice user, with a pre-generated secret
/// key. The email address of the user is returned as a string.
pub async fn configure_alice(&self) -> String {
let keypair = alice_keypair();
self.configure_addr(&keypair.addr.to_string()).await;
key::store_self_keypair(&self.ctx, &keypair, key::KeyPairUse::Default)
.await
.expect("Failed to save Alice's key");
keypair.addr.to_string()
}
/// Configure as a given email address.
@@ -299,27 +235,27 @@ impl TestContext {
/// Panics if there is no message or on any error.
pub async fn pop_sent_msg(&self) -> SentMessage {
let start = Instant::now();
let (rowid, msg_id, payload, recipients) = loop {
let (rowid, foreign_id, raw_params) = loop {
let row = self
.ctx
.sql
.query_row_optional(
.query_row(
r#"
SELECT id, msg_id, mime, recipients
FROM smtp
ORDER BY id DESC"#,
paramsv![],
SELECT id, foreign_id, param
FROM jobs
WHERE action=?
ORDER BY desired_timestamp DESC;
"#,
paramsv![Action::SendMsgToSmtp],
|row| {
let rowid: i64 = row.get(0)?;
let msg_id: MsgId = row.get(1)?;
let mime: String = row.get(2)?;
let recipients: String = row.get(3)?;
Ok((rowid, msg_id, mime, recipients))
let id: u32 = row.get(0)?;
let foreign_id: u32 = row.get(1)?;
let param: String = row.get(2)?;
Ok((id, foreign_id, param))
},
)
.await
.expect("query_row_optional failed");
if let Some(row) = row {
.await;
if let Ok(row) = row {
break row;
}
if start.elapsed() < Duration::from_secs(3) {
@@ -328,18 +264,24 @@ impl TestContext {
panic!("no sent message found in jobs table");
}
};
let id = MsgId::new(foreign_id);
let params = Params::from_str(&raw_params).unwrap();
let blob_path = params
.get_blob(Param::File, &self.ctx, false)
.await
.expect("failed to parse blob from param")
.expect("no Param::File found in Params")
.to_abs_path();
self.ctx
.sql
.execute("DELETE FROM jobs WHERE id=?;", paramsv![rowid])
.await
.expect("failed to remove job");
update_msg_state(&self.ctx, msg_id, MessageState::OutDelivered)
.await
.expect("failed to update message state");
update_msg_state(&self.ctx, id, MessageState::OutDelivered).await;
SentMessage {
payload,
sender_msg_id: msg_id,
recipients,
params,
blob_path,
sender_msg_id: id,
}
}
@@ -360,11 +302,13 @@ impl TestContext {
///
/// Receives a message using the `dc_receive_imf()` pipeline.
pub async fn recv_msg(&self, msg: &SentMessage) {
let mut idx = self.recv_idx.write().await;
*idx += 1;
let received_msg =
"Received: (Postfix, from userid 1000); Mon, 4 Dec 2006 14:51:39 +0100 (CET)\n"
.to_owned()
+ msg.payload();
dc_receive_imf(&self.ctx, received_msg.as_bytes(), "INBOX", false)
+ &msg.payload();
dc_receive_imf(&self.ctx, received_msg.as_bytes(), "INBOX", *idx, false)
.await
.unwrap();
}
@@ -399,56 +343,31 @@ impl TestContext {
.expect("failed to load msg")
}
/// Returns the [`Contact`] for the other [`TestContext`], creating it if necessary.
pub async fn add_or_lookup_contact(&self, other: &TestContext) -> Contact {
let name = other
.ctx
.get_config(Config::Displayname)
.await
.unwrap_or_default()
.unwrap_or_default();
let addr = other
.ctx
.get_config(Config::ConfiguredAddr)
.await
.unwrap()
.unwrap();
// MailinglistAddress is the lowest allowed origin, we'd prefer to not modify the
// origin when creating this contact.
let (contact_id, modified) =
Contact::add_or_lookup(self, &name, &addr, Origin::MailinglistAddress)
.await
.unwrap();
match modified {
Modifier::None => (),
Modifier::Modified => warn!(&self.ctx, "Contact {} modified by TestContext", &addr),
Modifier::Created => warn!(&self.ctx, "Contact {} created by TestContext", &addr),
}
Contact::load_from_db(&self.ctx, contact_id).await.unwrap()
}
/// Returns 1:1 [`Chat`] with another account, if it exists.
///
/// This first creates a contact using the configured details on the other account, then
/// creates a 1:1 chat with this contact.
pub async fn get_chat(&self, other: &TestContext) -> Option<Chat> {
let contact = self.add_or_lookup_contact(other).await;
match ChatId::lookup_by_contact(&self.ctx, contact.id)
.await
.unwrap()
{
Some(id) => Some(Chat::load_from_db(&self.ctx, id).await.unwrap()),
None => None,
}
}
/// Creates or returns an existing 1:1 [`Chat`] with another account.
///
/// This first creates a contact using the configured details on the other account, then
/// creates a 1:1 chat with this contact.
pub async fn create_chat(&self, other: &TestContext) -> Chat {
let contact = self.add_or_lookup_contact(other).await;
let chat_id = ChatId::create_for_contact(self, contact.id).await.unwrap();
let (contact_id, _modified) = Contact::add_or_lookup(
self,
&other
.ctx
.get_config(Config::Displayname)
.await
.unwrap_or_default()
.unwrap_or_default(),
&other
.ctx
.get_config(Config::ConfiguredAddr)
.await
.unwrap()
.unwrap(),
Origin::ManuallyCreated,
)
.await
.unwrap();
let chat_id = ChatId::create_for_contact(self, contact_id).await.unwrap();
Chat::load_from_db(self, chat_id).await.unwrap()
}
@@ -595,46 +514,14 @@ impl Drop for TestContext {
}
}
/// A receiver of [`Event`]s which will log the events to the captured test stdout.
///
/// Tests redirect the stdout of the test thread and capture this, showing the captured
/// stdout if the test fails. This means printing log messages must be done on the thread
/// of the test itself and not from a spawned task.
///
/// This sink achieves this by printing the events, in the order received, at the time it is
/// dropped. Thus to use you must only make sure this sink is dropped in the test itself.
///
/// To use this create an instance using [`LogSink::create`] and then use the
/// [`TestContextBuilder::with_log_sink`].
#[derive(Debug)]
pub struct LogSink {
events: Receiver<Event>,
}
impl LogSink {
/// Creates a new [`LogSink`] and returns the attached event sink.
pub fn create() -> (Sender<Event>, Self) {
let (tx, rx) = channel::unbounded();
(tx, Self { events: rx })
}
}
impl Drop for LogSink {
fn drop(&mut self) {
while let Ok(event) = self.events.try_recv() {
print_event(&event);
}
}
}
/// A raw message as it was scheduled to be sent.
///
/// This is a raw message, probably in the shape DC was planning to send it but not having
/// passed through a SMTP-IMAP pipeline.
#[derive(Debug, Clone)]
pub struct SentMessage {
payload: String,
recipients: String,
params: Params,
blob_path: PathBuf,
pub sender_msg_id: MsgId,
}
@@ -643,34 +530,33 @@ impl SentMessage {
///
/// If there are multiple recipients this is just a random one, so is not very useful.
pub fn recipient(&self) -> EmailAddress {
let rcpt = self
.recipients
.split(' ')
.next()
.expect("no recipient found");
let raw = self
.params
.get(Param::Recipients)
.expect("no recipients in params");
let rcpt = raw.split(' ').next().expect("no recipient found");
rcpt.parse().expect("failed to parse email address")
}
/// The raw message payload.
pub fn payload(&self) -> &str {
&self.payload
pub fn payload(&self) -> String {
std::fs::read_to_string(&self.blob_path).unwrap()
}
}
/// Load a pre-generated keypair for alice@example.org from disk.
/// Load a pre-generated keypair for alice@example.com from disk.
///
/// This saves CPU cycles by avoiding having to generate a key.
///
/// The keypair was created using the crate::key::tests::gen_key test.
pub fn alice_keypair() -> KeyPair {
let addr = EmailAddress::new("alice@example.org").unwrap();
let public = key::SignedPublicKey::from_asc(include_str!("../test-data/key/alice-public.asc"))
.unwrap()
.0;
let secret = key::SignedSecretKey::from_asc(include_str!("../test-data/key/alice-secret.asc"))
.unwrap()
.0;
pub fn alice_keypair() -> key::KeyPair {
let addr = EmailAddress::new("alice@example.com").unwrap();
let public =
key::SignedPublicKey::from_base64(include_str!("../test-data/key/alice-public.asc"))
.unwrap();
let secret =
key::SignedSecretKey::from_base64(include_str!("../test-data/key/alice-secret.asc"))
.unwrap();
key::KeyPair {
addr,
public,
@@ -681,14 +567,12 @@ pub fn alice_keypair() -> KeyPair {
/// Load a pre-generated keypair for bob@example.net from disk.
///
/// Like [alice_keypair] but a different key and identity.
pub fn bob_keypair() -> KeyPair {
pub fn bob_keypair() -> key::KeyPair {
let addr = EmailAddress::new("bob@example.net").unwrap();
let public = key::SignedPublicKey::from_asc(include_str!("../test-data/key/bob-public.asc"))
.unwrap()
.0;
let secret = key::SignedSecretKey::from_asc(include_str!("../test-data/key/bob-secret.asc"))
.unwrap()
.0;
let public =
key::SignedPublicKey::from_base64(include_str!("../test-data/key/bob-public.asc")).unwrap();
let secret =
key::SignedSecretKey::from_base64(include_str!("../test-data/key/bob-secret.asc")).unwrap();
key::KeyPair {
addr,
public,
@@ -696,43 +580,40 @@ pub fn bob_keypair() -> KeyPair {
}
}
/// Utility to help wait for and retrieve events.
///
/// This buffers the events in order they are emitted. This allows consuming events in
/// order while looking for the right events using the provided methods.
///
/// The methods only return [`EventType`] rather than the full [`Event`] since it can only
/// be attached to a single [`TestContext`] and therefore the context is already known as
/// you will be accessing it as [`TestContext::evtracker`].
#[derive(Debug)]
pub struct EventTracker(Receiver<Event>);
pub struct EvTracker(Receiver<EventType>);
impl EventTracker {
/// Consumes emitted events returning the first matching one.
///
/// If no matching events are ready this will wait for new events to arrive and time out
/// after 10 seconds.
pub async fn get_matching<F: Fn(&EventType) -> bool>(&self, event_matcher: F) -> EventType {
async move {
loop {
let event = self.0.recv().await.unwrap();
if event_matcher(&event.typ) {
return event.typ;
impl EvTracker {
pub async fn get_info_contains(&self, s: &str) -> EventType {
loop {
let event = self.0.recv().await.unwrap();
if let EventType::Info(i) = &event {
if i.contains(s) {
return event;
}
}
}
.timeout(Duration::from_secs(10))
.await
.expect("timeout waiting for event match")
}
/// Consumes events looking for an [`EventType::Info`] with substring matching.
pub async fn get_info_contains(&self, s: &str) -> EventType {
self.get_matching(|evt| match evt {
EventType::Info(ref msg) => msg.contains(s),
_ => false,
})
.await
pub async fn get_matching<F: Fn(EventType) -> bool>(&self, event_matcher: F) -> EventType {
const TIMEOUT: Duration = Duration::from_secs(20);
loop {
let event = async_std::future::timeout(TIMEOUT, self.recv())
.await
.unwrap()
.unwrap();
if event_matcher(event.clone()) {
return event;
}
}
}
}
impl Deref for EvTracker {
type Target = Receiver<EventType>;
fn deref(&self) -> &Self::Target {
&self.0
}
}
@@ -759,7 +640,7 @@ pub(crate) async fn get_chat_msg(
/// Pretty-print an event to stdout
///
/// Done during tests this is captured by `cargo test` and associated with the test itself.
fn print_event(event: &Event) {
fn receive_event(event: &Event) {
let green = Color::Green.normal();
let yellow = Color::Yellow.normal();
let red = Color::Red.normal();
@@ -851,7 +732,7 @@ async fn log_msg(context: &Context, prefix: impl AsRef<str>, msg: &Message) {
&contact_name,
contact_id,
msgtext.unwrap_or_default(),
if msg.get_from_id() == DC_CONTACT_ID_SELF {
if msg.get_from_id() == 1u32 {
""
} else if msg.get_state() == MessageState::InSeen {
"[SEEN]"
@@ -878,36 +759,3 @@ async fn log_msg(context: &Context, prefix: impl AsRef<str>, msg: &Message) {
statestr,
);
}
#[cfg(test)]
mod tests {
use super::*;
// The following three tests demonstrate, when made to fail, the log output being
// directed to the correct test output.
#[async_std::test]
async fn test_with_alice() {
let alice = TestContext::builder().configure_alice().build().await;
alice.ctx.emit_event(EventType::Info("hello".into()));
// panic!("Alice fails");
}
#[async_std::test]
async fn test_with_bob() {
let bob = TestContext::builder().configure_bob().build().await;
bob.ctx.emit_event(EventType::Info("there".into()));
// panic!("Bob fails");
}
#[async_std::test]
async fn test_with_both() {
let mut tcm = TestContextManager::new().await;
let alice = tcm.alice().await;
let bob = tcm.bob().await;
alice.ctx.emit_event(EventType::Info("hello".into()));
bob.ctx.emit_event(EventType::Info("there".into()));
// panic!("Both fail");
}
}

View File

@@ -1,7 +1,7 @@
//! # Functions to update timestamps.
use crate::chat::{Chat, ChatId};
use crate::contact::{Contact, ContactId};
use crate::contact::Contact;
use crate::context::Context;
use crate::param::{Param, Params};
use anyhow::Result;
@@ -12,7 +12,7 @@ impl Context {
/// (if we have a ContactId type at some point, the function should go there)
pub(crate) async fn update_contacts_timestamp(
&self,
contact_id: ContactId,
contact_id: u32,
scope: Param,
new_timestamp: i64,
) -> Result<bool> {
@@ -92,7 +92,7 @@ mod tests {
dc_receive_imf(
&t,
b"From: Bob Authname <bob@example.org>\n\
To: alice@example.org\n\
To: alice@example.com\n\
Subject: updated subject\n\
Message-ID: <msg2@example.org>\n\
Chat-Version: 1.0\n\
@@ -100,13 +100,14 @@ mod tests {
\n\
second message\n",
"INBOX",
1,
false,
)
.await?;
dc_receive_imf(
&t,
b"From: Bob Authname <bob@example.org>\n\
To: alice@example.org\n\
To: alice@example.com\n\
Subject: original subject\n\
Message-ID: <msg1@example.org>\n\
Chat-Version: 1.0\n\
@@ -114,6 +115,7 @@ mod tests {
\n\
first message\n",
"INBOX",
2,
false,
)
.await?;
@@ -135,7 +137,7 @@ mod tests {
dc_receive_imf(
&t,
b"From: Bob Authname <bob@example.org>\n\
To: alice@example.org\n\
To: alice@example.com\n\
Message-ID: <msg1@example.org>\n\
Chat-Version: 1.0\n\
Chat-Group-ID: abcde\n\
@@ -144,6 +146,7 @@ mod tests {
\n\
first message\n",
"INBOX",
1,
false,
)
.await?;
@@ -154,7 +157,7 @@ mod tests {
dc_receive_imf(
&t,
b"From: Bob Authname <bob@example.org>\n\
To: alice@example.org\n\
To: alice@example.com\n\
Message-ID: <msg3@example.org>\n\
Chat-Version: 1.0\n\
Chat-Group-ID: abcde\n\
@@ -164,13 +167,14 @@ mod tests {
\n\
third message\n",
"INBOX",
2,
false,
)
.await?;
dc_receive_imf(
&t,
b"From: Bob Authname <bob@example.org>\n\
To: alice@example.org\n\
To: alice@example.com\n\
Message-ID: <msg2@example.org>\n\
Chat-Version: 1.0\n\
Chat-Group-ID: abcde\n\
@@ -180,6 +184,7 @@ mod tests {
\n\
second message\n",
"INBOX",
3,
false,
)
.await?;

File diff suppressed because it is too large Load Diff

View File

@@ -12,11 +12,9 @@ Filename encoding | Encoded Words ([RFC 2047](https://tools.ietf.
Identify server folders | IMAP LIST Extension ([RFC 6154](https://tools.ietf.org/html/rfc6154))
Push | IMAP IDLE ([RFC 2177](https://tools.ietf.org/html/rfc2177))
Quota | IMAP QUOTA extension ([RFC 2087](https://tools.ietf.org/html/rfc2087))
Seen status synchronization | IMAP CONDSTORE extension ([RFC 7162](https://tools.ietf.org/html/rfc7162))
Authorization | OAuth2 ([RFC 6749](https://tools.ietf.org/html/rfc6749))
End-to-end encryption | [Autocrypt Level 1](https://autocrypt.org/level1.html), OpenPGP ([RFC 4880](https://tools.ietf.org/html/rfc4880)), Security Multiparts for MIME ([RFC 1847](https://tools.ietf.org/html/rfc1847)) and [“Mixed Up” Encryption repairing](https://tools.ietf.org/id/draft-dkg-openpgp-pgpmime-message-mangling-00.html)
Header encryption | [Protected Headers for Cryptographic E-mail](https://datatracker.ietf.org/doc/draft-autocrypt-lamps-protected-headers/)
Configuration assistance | [Autoconfigure](https://web.archive.org/web/20210402044801/https://developer.mozilla.org/en-US/docs/Mozilla/Thunderbird/Autoconfiguration) and [Autodiscover](https://technet.microsoft.com/library/bb124251(v=exchg.150).aspx)
Configuration assistance | [Autoconfigure](https://developer.mozilla.org/en-US/docs/Mozilla/Thunderbird/Autoconfiguration) and [Autodiscover](https://technet.microsoft.com/library/bb124251(v=exchg.150).aspx)
Messenger functions | [Chat-over-Email](https://github.com/deltachat/deltachat-core-rust/blob/master/spec.md#chat-mail-specification)
Detect mailing list | List-Id ([RFC 2919](https://tools.ietf.org/html/rfc2919)) and Precedence ([RFC 3834](https://tools.ietf.org/html/rfc3834))
User and chat colors | [XEP-0392](https://xmpp.org/extensions/xep-0392.html): Consistent Color Generation

View File

@@ -1,13 +1 @@
-----BEGIN PGP PUBLIC KEY BLOCK-----
mDMEXlh13RYJKwYBBAHaRw8BAQdAzfVIAleCXMJrq8VeLlEVof6ITCviMktKjmcB
KAu4m5C0GUFsaWNlIDxhbGljZUBleGFtcGxlLm9yZz6IkAQTFggAOBYhBC5vossj
tTLXKGNLWGSwj2Gp7ZRDBQJeWHXdAhsDBQsJCAcCBhUKCQgLAgQWAgMBAh4BAheA
AAoJEGSwj2Gp7ZRDE3oA/i4MCyDMTsjWqDZoQwX/A/GoTO2/V0wKPhjJJy/8m2pM
APkBjOnGOtx2SZpQvJGTa9h804RY6iDrRuI8A/8tEEXAA7g4BF5Ydd0SCisGAQQB
l1UBBQEBB0AG7cjWy2SFAU8KnltlubVW67rFiyfp01JrRe6Xqy22HQMBCAeIeAQY
FggAIBYhBC5vossjtTLXKGNLWGSwj2Gp7ZRDBQJeWHXdAhsMAAoJEGSwj2Gp7ZRD
Lo8BAObE8GnsGVwKzNqCvHeWgJsqhjS3C6gvSlV3tEm9XmF6AQDXucIyVfoBwoyM
h2h6cSn/ATn5QJb35pgo+ivp3jsMAg==
=t/Qq
-----END PGP PUBLIC KEY BLOCK-----
mDMEXlh13RYJKwYBBAHaRw8BAQdAzfVIAleCXMJrq8VeLlEVof6ITCviMktKjmcBKAu4m5C0GUFsaWNlIDxhbGljZUBleGFtcGxlLm9yZz6IkAQTFggAOBYhBC5vossjtTLXKGNLWGSwj2Gp7ZRDBQJeWHXdAhsDBQsJCAcCBhUKCQgLAgQWAgMBAh4BAheAAAoJEGSwj2Gp7ZRDE3oA/i4MCyDMTsjWqDZoQwX/A/GoTO2/V0wKPhjJJy/8m2pMAPkBjOnGOtx2SZpQvJGTa9h804RY6iDrRuI8A/8tEEXAA7g4BF5Ydd0SCisGAQQBl1UBBQEBB0AG7cjWy2SFAU8KnltlubVW67rFiyfp01JrRe6Xqy22HQMBCAeIeAQYFggAIBYhBC5vossjtTLXKGNLWGSwj2Gp7ZRDBQJeWHXdAhsMAAoJEGSwj2Gp7ZRDLo8BAObE8GnsGVwKzNqCvHeWgJsqhjS3C6gvSlV3tEm9XmF6AQDXucIyVfoBwoyMh2h6cSn/ATn5QJb35pgo+ivp3jsMAg==

Some files were not shown because too many files have changed in this diff Show More