Compare commits

..

69 Commits

Author SHA1 Message Date
link2xt
ea6905dc57 Try to use windows-2019 2024-05-11 17:30:19 +00:00
link2xt
7cf7604442 ci: run Rust tests on windows-2022 instead of windows-latest 2024-05-11 16:57:15 +00:00
bjoern
14aaab05b0 limit quote replies (#5543)
this PR checks if the quotes are used in a reasonable way:

- quoted messages should be send to the same chat
- or to one-to-one chats

if the quote's chat ID is not the same than the sending chat _and_ the
sending chat is not a one-to-one chat, sending is aborted.

usually, the UIs does not allow quoting in other ways, so, this check is
only a "last defence line" to avoid leaking data in case the UI has
bugs, as recently in
https://github.com/deltachat/deltachat-android/issues/3032

@iequidoo @link2xt @adbenitez i am not 100% sure about this PR, maybe
i've overseen a reasonable usecase where other quotes make sense

---------

Co-authored-by: link2xt <link2xt@testrun.org>
2024-05-10 16:43:49 +02:00
link2xt
72c09feb64 feat: do not add location markers to messages with non-POI location 2024-05-10 14:04:16 +00:00
Nico de Haen
8a4dff2212 Add webxdc internal integration commands in jsonrpc (#5541)
Adds 
_setWebxdcIntegration_ 
_initWebxdcIntegration_
2024-05-08 07:53:04 +02:00
dependabot[bot]
022f836d35 chore(cargo): bump schemars from 0.8.16 to 0.8.19
Bumps [schemars](https://github.com/GREsau/schemars) from 0.8.16 to 0.8.19.
- [Release notes](https://github.com/GREsau/schemars/releases)
- [Changelog](https://github.com/GREsau/schemars/blob/master/CHANGELOG.md)
- [Commits](https://github.com/GREsau/schemars/compare/v0.8.16...v0.8.19)

---
updated-dependencies:
- dependency-name: schemars
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-05-08 03:42:51 +00:00
link2xt
636ab4a9e5 fix: delete non-POI locations after delete_device_after, not immediately 2024-05-06 18:32:04 +00:00
link2xt
2bddefa1ab refactor: remove allow_keychange 2024-05-05 21:23:32 +00:00
dependabot[bot]
7d67100a3c Merge pull request #5523 from deltachat/dependabot/cargo/mailparse-0.15.0 2024-05-05 21:23:10 +00:00
dependabot[bot]
1043916411 chore(cargo): bump mailparse from 0.14.1 to 0.15.0
Bumps [mailparse](https://github.com/staktrace/mailparse) from 0.14.1 to 0.15.0.
- [Commits](https://github.com/staktrace/mailparse/compare/v0.14.1...v0.15.0)

---
updated-dependencies:
- dependency-name: mailparse
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-05-05 20:59:52 +00:00
dependabot[bot]
f4e58e90ae chore(cargo): bump syn from 2.0.57 to 2.0.60
Bumps [syn](https://github.com/dtolnay/syn) from 2.0.57 to 2.0.60.
- [Release notes](https://github.com/dtolnay/syn/releases)
- [Commits](https://github.com/dtolnay/syn/compare/2.0.57...2.0.60)

---
updated-dependencies:
- dependency-name: syn
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-05-05 13:50:07 +00:00
link2xt
e4f10b32dd chore(cargo): bump imap-proto from 0.16.4 to 0.16.5 2024-05-04 22:41:30 +00:00
dependabot[bot]
e9431888a6 Merge pull request #5522 from deltachat/dependabot/cargo/chrono-0.4.38 2024-05-04 22:40:17 +00:00
dependabot[bot]
1649073c0f chore(cargo): bump anyhow from 1.0.81 to 1.0.82
Bumps [anyhow](https://github.com/dtolnay/anyhow) from 1.0.81 to 1.0.82.
- [Release notes](https://github.com/dtolnay/anyhow/releases)
- [Commits](https://github.com/dtolnay/anyhow/compare/1.0.81...1.0.82)

---
updated-dependencies:
- dependency-name: anyhow
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-05-04 03:23:17 -03:00
dependabot[bot]
b2cf18d8b3 chore(cargo): bump chrono from 0.4.37 to 0.4.38
Bumps [chrono](https://github.com/chronotope/chrono) from 0.4.37 to 0.4.38.
- [Release notes](https://github.com/chronotope/chrono/releases)
- [Changelog](https://github.com/chronotope/chrono/blob/main/CHANGELOG.md)
- [Commits](https://github.com/chronotope/chrono/compare/v0.4.37...v0.4.38)

---
updated-dependencies:
- dependency-name: chrono
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-05-04 02:53:42 +00:00
iequidoo
2eceb4be29 feat(contact-tools): Add make_vcard()
Add a function making a vCard containing given `VcardContact`s.
2024-05-03 23:52:51 -03:00
iequidoo
ae7ff17ba2 feat(contact-tools): Support parsing profile images from "PHOTO:data:image/jpeg;base64,..." 2024-05-03 23:52:51 -03:00
Hocuri
026f678452 feat: Parsing vCards for contacts sharing (#5482)
Co-authored-by: iequidoo <dgreshilov@gmail.com>
2024-05-03 22:44:09 -03:00
link2xt
add8c0680f ci: update Rust to 1.78.0 2024-05-03 02:11:07 +00:00
dependabot[bot]
aee2b81c06 chore(cargo): bump thiserror from 1.0.58 to 1.0.59
Bumps [thiserror](https://github.com/dtolnay/thiserror) from 1.0.58 to 1.0.59.
- [Release notes](https://github.com/dtolnay/thiserror/releases)
- [Commits](https://github.com/dtolnay/thiserror/compare/1.0.58...1.0.59)

---
updated-dependencies:
- dependency-name: thiserror
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-05-02 20:35:31 -03:00
dependabot[bot]
3624aad1b5 chore(cargo): bump async-channel from 2.2.0 to 2.2.1
Bumps [async-channel](https://github.com/smol-rs/async-channel) from 2.2.0 to 2.2.1.
- [Release notes](https://github.com/smol-rs/async-channel/releases)
- [Changelog](https://github.com/smol-rs/async-channel/blob/master/CHANGELOG.md)
- [Commits](https://github.com/smol-rs/async-channel/compare/v2.2.0...v2.2.1)

---
updated-dependencies:
- dependency-name: async-channel
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-05-02 20:24:27 -03:00
link2xt
299d994d4b ci: replace black with ruff format
We use `ruff` anyway, so it is one less dependency.
2024-05-02 14:44:11 +02:00
dependabot[bot]
5e0f5ec390 chore(cargo): bump serde from 1.0.197 to 1.0.200
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.197 to 1.0.200.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.197...v1.0.200)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-05-02 04:27:42 -03:00
dependabot[bot]
c318ca5d1a chore(cargo): bump base64 from 0.22.0 to 0.22.1
Bumps [base64](https://github.com/marshallpierce/rust-base64) from 0.22.0 to 0.22.1.
- [Changelog](https://github.com/marshallpierce/rust-base64/blob/master/RELEASE-NOTES.md)
- [Commits](https://github.com/marshallpierce/rust-base64/compare/v0.22.0...v0.22.1)

---
updated-dependencies:
- dependency-name: base64
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-05-02 04:18:18 -03:00
dependabot[bot]
38a2e07194 Merge pull request #5515 from deltachat/dependabot/cargo/quote-1.0.36 2024-05-02 06:22:56 +00:00
dependabot[bot]
1ff6740938 chore(cargo): bump quote from 1.0.35 to 1.0.36
Bumps [quote](https://github.com/dtolnay/quote) from 1.0.35 to 1.0.36.
- [Release notes](https://github.com/dtolnay/quote/releases)
- [Commits](https://github.com/dtolnay/quote/compare/1.0.35...1.0.36)

---
updated-dependencies:
- dependency-name: quote
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-05-01 21:52:09 +00:00
Christian Hagenest
402d5bed85 rm npm install 2024-04-30 21:10:36 +02:00
missytake
57bc046381 ci: use rsync instead of 3rd party github action 2024-04-30 21:10:36 +02:00
Christian Hagenest
0617236eb0 rm leading slash 2024-04-30 21:10:36 +02:00
Christian Hagenest
8c5ffe0237 upload-docs npm run => npm run build 2024-04-30 21:10:36 +02:00
Christian Hagenest
39f977c1e6 add : to upload-docs.yml 2024-04-30 21:10:36 +02:00
Christian Hagenest
ec03614cae add npm install to upload-docs.yml 2024-04-30 21:10:36 +02:00
Christian Hagenest
ea0b063c19 test ts build on branch 2024-04-30 21:10:36 +02:00
Christian Hagenest
98d7a93909 correct folder for js.jsonrpc docs 2024-04-30 21:10:36 +02:00
Christian Hagenest
49bf8414ed rm unneeded rust install from ts docs ci 2024-04-30 21:10:36 +02:00
Christian Hagenest
1e7dbea351 Implement jsonrpc-docs build in github action 2024-04-30 21:10:36 +02:00
Christian Hagenest
0412244646 Revert "WIP: build ts docs with ci + nix"
This reverts commit 00d486e58f.
2024-04-30 21:10:36 +02:00
Christian Hagenest
bbd854d7bc Revert "push docs to delta.chat instead of codespeak"
This reverts commit c04188ce56.
2024-04-30 21:10:36 +02:00
Christian Hagenest
ba2bb517f7 Revert "WIP npm2nix, error: no source/nix"
This reverts commit 83bfba53de.
2024-04-30 21:10:36 +02:00
Christian Hagenest
0ae831eca0 WIP npm2nix, error: no source/nix 2024-04-30 21:10:36 +02:00
Christian Hagenest
ab494ae786 push docs to delta.chat instead of codespeak 2024-04-30 21:10:36 +02:00
Christian Hagenest
8a58ae8a3a WIP: build ts docs with ci + nix 2024-04-30 21:10:36 +02:00
link2xt
cf84255e99 test: test that POIs are deleted when ephemeral message expires 2024-04-29 22:35:59 +00:00
link2xt
462bd63065 docs: add location module documentation 2024-04-29 22:35:59 +00:00
link2xt
6bfbf6547b feat: delete orphaned POI locations during housekeeping 2024-04-29 22:35:59 +00:00
link2xt
13802bab42 fix: delete POI location when disappearing message expires 2024-04-29 22:35:59 +00:00
link2xt
adb2e4ea32 refactor: move delete_poi_location to location module and document it 2024-04-29 22:35:59 +00:00
link2xt
421a7b277d docs: remove Doxygen markup from Message.has_location() 2024-04-29 22:35:59 +00:00
link2xt
14d8139883 feat: delete expired path locations in ephemeral loop 2024-04-29 22:35:59 +00:00
link2xt
062905924c docs: fix references in Message.set_location() documentation 2024-04-29 22:35:59 +00:00
iequidoo
20d79970a2 fix: Correct message viewtype before recoding image blob (#5496)
Otherwise, e.g. if a message is a large GIF, but its viewtype is set to `Image` by the app, this GIF
will be recoded to JPEG to reduce its size. GIFs and other special viewtypes must be always detected
and sent as is.
2024-04-27 23:44:50 -03:00
link2xt
f49588e64e fix: interrupt location loop when new location is stored
Otherwise location-only messages
that should be sent every 60 seconds
are never sent because location loop
waits until the end of location streaming
and is only interrupted by location streaming
ending in other chats or being enabled in other chats.
2024-04-27 20:55:05 +00:00
link2xt
496a8e3810 test: test that member is added even if "Member added" is lost
This is similar to `test_modify_chat_disordered`,
but tests that recovery works in the simplest case
where next message is not modifying group membership.
2024-04-27 07:51:08 +00:00
link2xt
94dc65c1a2 refactor(python): fix ruff 0.4.2 warnings 2024-04-25 20:45:54 +00:00
link2xt
4fe7fa3148 fix: never prepend subject to message text when bot receives it 2024-04-25 12:15:54 +00:00
link2xt
4cf923ccb9 fix: do not fail to send encrypted quotes to unencrypted chats
Replace quote text with "..." instead.
2024-04-25 09:00:49 +00:00
iequidoo
56b86adf18 api: Add dc_msg_save_file() which saves file copy at the provided path (#4309)
... and fails if file already exists. The UI should open the file saving dialog, defaulting to
Downloads and original filename, when asked to save the file. After confirmation it should call
dc_msg_save_file().
2024-04-24 16:38:25 -03:00
iequidoo
cfccee2ad4 fix: Message::set_file_from_bytes(): Set Param::Filename 2024-04-24 16:38:25 -03:00
Hocuri
37d92e3fa5 test: Explain test_was_seen_recently false-positive and give workaround instructions (#5474)
Until the issue is fixed, keep others from running into the same issue.
2024-04-24 14:55:15 +00:00
link2xt
a1ee2b463f chore(release): prepare for 1.137.4 2024-04-24 11:10:25 +00:00
link2xt
8df3b1bb1b fix: use only CRLF in Autocrypt Setup Message 2024-04-24 01:50:28 +00:00
iequidoo
22f240dd4d feat: Add progressive backoff for failing IMAP connection attempts (#5443)
This way we avoid an immediate retry if the network is not yet ready exhausting the ratelimiter's
quota of two connection attempts. Also notify the ratelimiter only after a successful connection so
that it only limits the server load, but not connection attempts.
2024-04-23 22:00:47 -03:00
iequidoo
ae10ed5c40 refactor: Imap: remove RwLock from ratelimit 2024-04-23 22:00:47 -03:00
link2xt
aff6bf9402 fix: convert images to RGB8 (without alpha) before encoding into JPEG
Otherwise an error
"The encoder or decoder for Jpeg does not support the color type `Rgba8`"
is returned if image has an alpha channel.

This is caused by the recent change of JPEG encoder
in `image` crate: <https://github.com/image-rs/image/issues/2211>
2024-04-23 23:37:58 +00:00
link2xt
43fc55e542 test: test recoding RGBA image 2024-04-23 23:37:58 +00:00
link2xt
7ea05cb8a0 test: add screenshot-rgba.png
Created by `convert -alpha deactivate screenshot.png screenshot-rbga.png`.
2024-04-23 23:37:58 +00:00
link2xt
d036ad5853 fix: do not fail if Autocrypt Setup Message has no encryption preference
According to Autocrypt specification
Autocrypt Setup Message SHOULD
contain Autocrypt-Prefer-Encrypt header,
but K-9 6.802 does not include it.
2024-04-23 22:16:54 +00:00
link2xt
e9280b8413 refactor: group use at the top of the test modules 2024-04-23 21:07:50 +00:00
link2xt
2108a8ba94 fix(node): undefine NAPI_EXPERIMENTAL
This fixes build with new clang
which treats -Wincompatible-function-pointer-types as an error.

Related upstream issue: <https://github.com/nodejs/node/issues/52229>
2024-04-23 11:34:03 +00:00
52 changed files with 1367 additions and 578 deletions

View File

@@ -24,7 +24,7 @@ jobs:
name: Lint Rust
runs-on: ubuntu-latest
env:
RUSTUP_TOOLCHAIN: 1.77.1
RUSTUP_TOOLCHAIN: 1.78.0
steps:
- uses: actions/checkout@v4
with:
@@ -83,11 +83,11 @@ jobs:
matrix:
include:
- os: ubuntu-latest
rust: 1.77.1
- os: windows-latest
rust: 1.77.1
rust: 1.78.0
- os: windows-2019
rust: 1.78.0
- os: macos-latest
rust: 1.77.1
rust: 1.78.0
# Minimum Supported Rust Version = 1.77.0
- os: ubuntu-latest

View File

@@ -21,17 +21,14 @@ jobs:
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ubuntu-latest, macos-14, windows-latest]
node: ["18", "20"]
os: [ubuntu-latest, macos-latest, windows-latest]
steps:
- uses: actions/checkout@v4
with:
show-progress: false
- uses: actions/setup-node@v4
with:
node-version: ${{ matrix.node }}
- if: matrix.os == 'macos-14'
run: sudo xcode-select --switch /Applications/Xcode_15.3.app/Contents/Developer
node-version: "18"
- name: System info
run: |
rustc -vV

View File

@@ -4,6 +4,7 @@ on:
push:
branches:
- main
- build_jsonrpc_docs_ci
jobs:
build-rs:
@@ -17,13 +18,11 @@ jobs:
run: |
cargo doc --package deltachat --no-deps --document-private-items
- name: Upload to rs.delta.chat
uses: up9cloud/action-rsync@v1.3
env:
USER: ${{ secrets.USERNAME }}
KEY: ${{ secrets.KEY }}
HOST: "delta.chat"
SOURCE: "target/doc"
TARGET: "/var/www/html/rs/"
run: |
mkdir -p "$HOME/.ssh"
echo "${{ secrets.KEY }}" > "$HOME/.ssh/key"
chmod 600 "$HOME/.ssh/key"
rsync -avzh -e "ssh -i $HOME/.ssh/key -o StrictHostKeyChecking=no" $GITHUB_WORKSPACE/target/doc "${{ secrets.USERNAME }}@rs.delta.chat:/var/www/html/rs/"
build-python:
runs-on: ubuntu-latest
@@ -62,3 +61,31 @@ jobs:
echo "${{ secrets.CODESPEAK_KEY }}" > "$HOME/.ssh/key"
chmod 600 "$HOME/.ssh/key"
rsync -avzh -e "ssh -i $HOME/.ssh/key -o StrictHostKeyChecking=no" $GITHUB_WORKSPACE/result/html/ "delta@c.delta.chat:/home/delta/build-c/master"
build-ts:
runs-on: ubuntu-latest
defaults:
run:
working-directory: ./deltachat-jsonrpc/typescript
steps:
- uses: actions/checkout@v4
with:
show-progress: false
fetch-depth: 0 # Fetch history to calculate VCS version number.
- name: Use Node.js
uses: actions/setup-node@v2
with:
node-version: '18'
- name: npm install
run: npm install
- name: npm run build
run: npm run build
- name: Run docs script
run: npm run docs
- name: Upload to js.jsonrpc.delta.chat
run: |
mkdir -p "$HOME/.ssh"
echo "${{ secrets.KEY }}" > "$HOME/.ssh/key"
chmod 600 "$HOME/.ssh/key"
rsync -avzh -e "ssh -i $HOME/.ssh/key -o StrictHostKeyChecking=no" $GITHUB_WORKSPACE/deltachat-jsonrpc/typescript/docs/ "${{ secrets.USERNAME }}@js.jsonrpc.delta.chat:/var/www/html/js-jsonrpc/"

View File

@@ -1,5 +1,48 @@
# Changelog
## [1.137.4] - 2024-04-24
### API-Changes
- [**breaking**] Remove `Stream` implementation for `EventEmitter`.
- Experimental Webxdc Integration API, Maps Integration ([#5461](https://github.com/deltachat/deltachat-core-rust/pull/5461)).
### Features / Changes
- Add progressive backoff for failing IMAP connection attempts ([#5443](https://github.com/deltachat/deltachat-core-rust/pull/5443)).
- Replace event channel with broadcast channel.
- Mark contact request messages as seen on IMAP.
### Fixes
- Convert images to RGB8 (without alpha) before encoding into JPEG to fix sending of large RGBA images.
- Don't set `is_bot` for webxdc status updates ([#5445](https://github.com/deltachat/deltachat-core-rust/pull/5445)).
- Do not fail if Autocrypt Setup Message has no encryption preference to fix key transfer from K-9 Mail to Delta Chat.
- Use only CRLF in Autocrypt Setup Message.
- python: Use cached message object if `dc_get_msg()` returns `NULL`.
- python: `Message::is_outgoing`: Don't reload message from db.
- python: `_map_ffi_event`: Always check if `get_message_by_id()` returned None.
- node: Undefine `NAPI_EXPERIMENTAL` to fix build with new clang.
### Build system
- nix: Add `imap-tools` as `deltachat-rpc-client` dependency.
- nix: Add `./deltachat-contact-tools` to sources.
- nix: Update nix flake.
- deps: Update rustls to 0.21.11.
### Documentation
- Update references to SecureJoin protocols.
- Fix broken references in documentation comments.
### Refactor
- imap: remove `RwLock` from `ratelimit`.
- deltachat-ffi: Remove unused `ResultNullableExt`.
- Remove duplicate clippy exceptions.
- Group `use` at the top of the test modules.
## [1.137.3] - 2024-04-16
### API-Changes
@@ -3939,3 +3982,4 @@ https://github.com/deltachat/deltachat-core-rust/pulls?q=is%3Apr+is%3Aclosed
[1.137.1]: https://github.com/deltachat/deltachat-core-rust/compare/v1.137.0...v1.137.1
[1.137.2]: https://github.com/deltachat/deltachat-core-rust/compare/v1.137.1...v1.137.2
[1.137.3]: https://github.com/deltachat/deltachat-core-rust/compare/v1.137.2...v1.137.3
[1.137.4]: https://github.com/deltachat/deltachat-core-rust/compare/v1.137.3...v1.137.4

135
Cargo.lock generated
View File

@@ -163,9 +163,9 @@ dependencies = [
[[package]]
name = "anyhow"
version = "1.0.81"
version = "1.0.82"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0952808a6c2afd1aa8947271f3a60f1a6763c7b912d210184c5149b5cf147247"
checksum = "f538837af36e6f6a9be0faa67f9a314f8119e4e4b5867c6ab40ed60360142519"
dependencies = [
"backtrace",
]
@@ -252,9 +252,9 @@ dependencies = [
[[package]]
name = "async-channel"
version = "2.2.0"
version = "2.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f28243a43d821d11341ab73c80bed182dc015c514b951616cf79bd4af39af0c3"
checksum = "136d4d23bcc79e27423727b36823d86233aad06dfea531837b038394d11e9928"
dependencies = [
"concurrent-queue",
"event-listener 5.2.0",
@@ -282,7 +282,7 @@ version = "0.9.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "98892ebee4c05fc66757e600a7466f0d9bfcde338f645d64add323789f26cb36"
dependencies = [
"async-channel 2.2.0",
"async-channel 2.2.1",
"base64 0.21.7",
"bytes",
"chrono",
@@ -344,7 +344,7 @@ checksum = "c980ee35e870bd1a4d2c8294d4c04d0499e67bca1e4b5cefcc693c2fa00caea9"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.57",
"syn 2.0.60",
]
[[package]]
@@ -478,9 +478,9 @@ checksum = "9d297deb1925b89f2ccc13d7635fa0714f12c87adce1c75356b39ca9b7178567"
[[package]]
name = "base64"
version = "0.22.0"
version = "0.22.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9475866fec1451be56a3c2400fd081ff546538961565ccb5b7142cbd22bc7a51"
checksum = "72b3254f16251a8381aa12e40e3c4d2f0199f8c6508fbecb9d91f575e0fbb8c6"
[[package]]
name = "base64ct"
@@ -724,9 +724,9 @@ dependencies = [
[[package]]
name = "chrono"
version = "0.4.37"
version = "0.4.38"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8a0d04d43504c61aa6c7531f1871dd0d418d91130162063b789da00fd7057a5e"
checksum = "a21f936df1771bf62b77f047b726c4625ff2e8aa607c01ec06e5a05bd8463401"
dependencies = [
"android-tzdata",
"iana-time-zone",
@@ -1059,7 +1059,7 @@ checksum = "f46882e17999c6cc590af592290432be3bce0428cb0d5f8b6715e4dc7b383eb3"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.57",
"syn 2.0.60",
]
[[package]]
@@ -1134,9 +1134,9 @@ dependencies = [
[[package]]
name = "data-encoding"
version = "2.5.0"
version = "2.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7e962a19be5cfc3f3bf6dd8f61eb50107f356ad6270fbb3ed41476571db78be5"
checksum = "e8566979429cf69b49a5c740c60791108e86440e8be149bbea4fe54d2c32d6e2"
[[package]]
name = "default-net"
@@ -1157,18 +1157,18 @@ dependencies = [
[[package]]
name = "deltachat"
version = "1.137.3"
version = "1.137.4"
dependencies = [
"ansi_term",
"anyhow",
"async-broadcast",
"async-channel 2.2.0",
"async-channel 2.2.1",
"async-imap",
"async-native-tls",
"async-smtp",
"async_zip",
"backtrace",
"base64 0.22.0",
"base64 0.22.1",
"brotli",
"chrono",
"criterion",
@@ -1239,9 +1239,10 @@ dependencies = [
[[package]]
name = "deltachat-contact-tools"
version = "0.1.0"
version = "0.0.0"
dependencies = [
"anyhow",
"chrono",
"once_cell",
"regex",
"rusqlite",
@@ -1249,12 +1250,12 @@ dependencies = [
[[package]]
name = "deltachat-jsonrpc"
version = "1.137.3"
version = "1.137.4"
dependencies = [
"anyhow",
"async-channel 2.2.0",
"async-channel 2.2.1",
"axum",
"base64 0.22.0",
"base64 0.22.1",
"deltachat",
"env_logger 0.11.3",
"futures",
@@ -1273,7 +1274,7 @@ dependencies = [
[[package]]
name = "deltachat-repl"
version = "1.137.3"
version = "1.137.4"
dependencies = [
"ansi_term",
"anyhow",
@@ -1288,7 +1289,7 @@ dependencies = [
[[package]]
name = "deltachat-rpc-server"
version = "1.137.3"
version = "1.137.4"
dependencies = [
"anyhow",
"deltachat",
@@ -1312,12 +1313,12 @@ name = "deltachat_derive"
version = "2.0.0"
dependencies = [
"quote",
"syn 2.0.57",
"syn 2.0.60",
]
[[package]]
name = "deltachat_ffi"
version = "1.137.3"
version = "1.137.4"
dependencies = [
"anyhow",
"deltachat",
@@ -1521,7 +1522,7 @@ checksum = "487585f4d0c6655fe74905e2504d8ad6908e4db67f744eb140876906c2f3175d"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.57",
"syn 2.0.60",
]
[[package]]
@@ -1830,7 +1831,7 @@ dependencies = [
"heck",
"proc-macro2",
"quote",
"syn 2.0.57",
"syn 2.0.60",
]
[[package]]
@@ -1843,7 +1844,7 @@ dependencies = [
"num-traits",
"proc-macro2",
"quote",
"syn 2.0.57",
"syn 2.0.60",
]
[[package]]
@@ -2164,7 +2165,7 @@ checksum = "87750cf4b7a4c0625b1529e4c543c2182106e4dedc60a2a6455e00d212c489ac"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.57",
"syn 2.0.60",
]
[[package]]
@@ -2638,9 +2639,9 @@ dependencies = [
[[package]]
name = "imap-proto"
version = "0.16.4"
version = "0.16.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "22e70cd66882c8cb1c9802096ba75212822153c51478dc61621e1a22f6c92361"
checksum = "de555d9526462b6f9ece826a26fb7c67eca9a0245bd9ff84fa91972a5d5d8856"
dependencies = [
"nom",
]
@@ -2906,9 +2907,9 @@ dependencies = [
[[package]]
name = "mailparse"
version = "0.14.1"
version = "0.15.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2d096594926cab442e054e047eb8c1402f7d5b2272573b97ba68aa40629f9757"
checksum = "3da03d5980411a724e8aaf7b61a7b5e386ec55a7fb49ee3d0ff79efc7e5e7c7e"
dependencies = [
"charset",
"data-encoding",
@@ -3166,7 +3167,7 @@ checksum = "ed3955f1a9c7c0c15e092f9c887db08b1fc683305fdf6eb6684f22555355e202"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.57",
"syn 2.0.60",
]
[[package]]
@@ -3227,7 +3228,7 @@ dependencies = [
"proc-macro-crate",
"proc-macro2",
"quote",
"syn 2.0.57",
"syn 2.0.60",
]
[[package]]
@@ -3289,7 +3290,7 @@ checksum = "a948666b637a0f465e8564c73e89d4dde00d72d4d473cc972f390fc3dcee7d9c"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.57",
"syn 2.0.60",
]
[[package]]
@@ -3530,7 +3531,7 @@ checksum = "2f38a4412a78282e09a2cf38d195ea5420d15ba0602cb375210efbc877243965"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.57",
"syn 2.0.60",
]
[[package]]
@@ -3747,9 +3748,9 @@ dependencies = [
[[package]]
name = "proc-macro2"
version = "1.0.78"
version = "1.0.81"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e2422ad645d89c99f8f3e6b88a9fdeca7fabeac836b1002371c4367c8f984aae"
checksum = "3d1597b0c024618f09a9c3b8655b7e430397a36d23fdafec26d6965e9eec3eba"
dependencies = [
"unicode-ident",
]
@@ -3860,9 +3861,9 @@ dependencies = [
[[package]]
name = "quote"
version = "1.0.35"
version = "1.0.36"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "291ec9ab5efd934aaf503a6466c5d5251535d108ee747472c3977cc5acc868ef"
checksum = "0fa76aaf39101c457836aec0ce2316dbdc3ab723cdda1c6bd4e6ad4208acaca7"
dependencies = [
"proc-macro2",
]
@@ -4394,9 +4395,9 @@ dependencies = [
[[package]]
name = "schemars"
version = "0.8.16"
version = "0.8.19"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "45a28f4c49489add4ce10783f7911893516f15afe45d015608d41faca6bc4d29"
checksum = "fc6e7ed6919cb46507fb01ff1654309219f62b4d603822501b0b80d42f6f21ef"
dependencies = [
"dyn-clone",
"schemars_derive",
@@ -4406,14 +4407,14 @@ dependencies = [
[[package]]
name = "schemars_derive"
version = "0.8.16"
version = "0.8.19"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c767fd6fa65d9ccf9cf026122c1b555f2ef9a4f0cea69da4d7dbc3e258d30967"
checksum = "185f2b7aa7e02d418e453790dde16890256bbd2bcd04b7dc5348811052b53f49"
dependencies = [
"proc-macro2",
"quote",
"serde_derive_internals",
"syn 1.0.109",
"syn 2.0.60",
]
[[package]]
@@ -4500,9 +4501,9 @@ dependencies = [
[[package]]
name = "serde"
version = "1.0.197"
version = "1.0.200"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3fb1c873e1b9b056a4dc4c0c198b24c3ffa059243875552b2bd0933b1aee4ce2"
checksum = "ddc6f9cc94d67c0e21aaf7eda3a010fd3af78ebf6e096aa6e2e13c79749cce4f"
dependencies = [
"serde_derive",
]
@@ -4527,24 +4528,24 @@ dependencies = [
[[package]]
name = "serde_derive"
version = "1.0.197"
version = "1.0.200"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7eb0b34b42edc17f6b7cac84a52a1c5f0e1bb2227e997ca9011ea3dd34e8610b"
checksum = "856f046b9400cee3c8c94ed572ecdb752444c24528c035cd35882aad6f492bcb"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.57",
"syn 2.0.60",
]
[[package]]
name = "serde_derive_internals"
version = "0.26.0"
version = "0.29.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "85bf8229e7920a9f636479437026331ce11aa132b4dde37d121944a44d6e5f3c"
checksum = "330f01ce65a3a5fe59a60c82f3c9a024b573b8a6e875bd233fe5f934e71d54e3"
dependencies = [
"proc-macro2",
"quote",
"syn 1.0.109",
"syn 2.0.60",
]
[[package]]
@@ -4818,7 +4819,7 @@ dependencies = [
"proc-macro2",
"quote",
"rustversion",
"syn 2.0.57",
"syn 2.0.60",
]
[[package]]
@@ -4840,9 +4841,9 @@ dependencies = [
[[package]]
name = "syn"
version = "2.0.57"
version = "2.0.60"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "11a6ae1e52eb25aab8f3fb9fca13be982a373b8f1157ca14b897a825ba4a2d35"
checksum = "909518bc7b1c9b779f1bbf07f2929d35af9f0f37e47c6e9ef7f9dddc1e1821f3"
dependencies = [
"proc-macro2",
"quote",
@@ -4962,22 +4963,22 @@ dependencies = [
[[package]]
name = "thiserror"
version = "1.0.58"
version = "1.0.59"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "03468839009160513471e86a034bb2c5c0e4baae3b43f79ffc55c4a5427b3297"
checksum = "f0126ad08bff79f29fc3ae6a55cc72352056dfff61e3ff8bb7129476d44b23aa"
dependencies = [
"thiserror-impl",
]
[[package]]
name = "thiserror-impl"
version = "1.0.58"
version = "1.0.59"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c61f3ba182994efc43764a46c018c347bc492c79f024e705f46567b418f6d4f7"
checksum = "d1cd413b5d558b4c5bf3680e324a6fa5014e7b7c067a51e69dbdf47eb7148b66"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.57",
"syn 2.0.60",
]
[[package]]
@@ -5094,7 +5095,7 @@ checksum = "5b8a1e28f2deaa14e508979454cb3a223b10b938b45af148bc0986de36f1923b"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.57",
"syn 2.0.60",
]
[[package]]
@@ -5267,7 +5268,7 @@ checksum = "34704c8d6ebcbc939824180af020566b01a7c01f80641264eba0999f6c2b6be7"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.57",
"syn 2.0.60",
]
[[package]]
@@ -5561,7 +5562,7 @@ dependencies = [
"once_cell",
"proc-macro2",
"quote",
"syn 2.0.57",
"syn 2.0.60",
"wasm-bindgen-shared",
]
@@ -5595,7 +5596,7 @@ checksum = "e94f17b526d0a461a191c78ea52bbce64071ed5c04c9ffe424dcb38f74171bb7"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.57",
"syn 2.0.60",
"wasm-bindgen-backend",
"wasm-bindgen-shared",
]
@@ -5991,7 +5992,7 @@ checksum = "9ce1b18ccd8e73a9321186f97e46f9f04b778851177567b1975109d26a08d2a6"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.57",
"syn 2.0.60",
]
[[package]]
@@ -6011,7 +6012,7 @@ checksum = "ce36e65b0d2999d2aafac989fb249189a141aee1f53c612c1f37d72631959f69"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.57",
"syn 2.0.60",
]
[[package]]

View File

@@ -1,6 +1,6 @@
[package]
name = "deltachat"
version = "1.137.3"
version = "1.137.4"
edition = "2021"
license = "MPL-2.0"
rust-version = "1.77"
@@ -40,7 +40,7 @@ ratelimit = { path = "./deltachat-ratelimit" }
anyhow = { workspace = true }
async-broadcast = "0.7.0"
async-channel = "2.0.0"
async-channel = "2.2.1"
async-imap = { version = "0.9.7", default-features = false, features = ["runtime-tokio"] }
async-native-tls = { version = "0.5", default-features = false, features = ["runtime-tokio"] }
async-smtp = { version = "0.9", default-features = false, features = ["runtime-tokio"] }
@@ -48,7 +48,7 @@ async_zip = { version = "0.0.12", default-features = false, features = ["deflate
backtrace = "0.3"
base64 = "0.22"
brotli = { version = "5", default-features=false, features = ["std"] }
chrono = { version = "0.4.37", default-features=false, features = ["clock", "std"] }
chrono = { workspace = true }
email = { git = "https://github.com/deltachat/rust-email", branch = "master" }
encoded-words = { git = "https://github.com/async-email/encoded-words", branch = "master" }
escaper = "0.1"
@@ -64,7 +64,7 @@ iroh = { version = "0.4.2", default-features = false }
kamadak-exif = "0.5.3"
lettre_email = { git = "https://github.com/deltachat/lettre", branch = "master" }
libc = "0.2"
mailparse = "0.14"
mailparse = "0.15"
mime = "0.3.17"
num_cpus = "1.16"
num-derive = "0.4"
@@ -168,7 +168,8 @@ harness = false
anyhow = "1"
once_cell = "1.18.0"
regex = "1.10"
rusqlite = { version = "0.31" }
rusqlite = "0.31"
chrono = { version = "0.4.38", default-features=false, features = ["alloc", "clock", "std"] }
[features]
default = ["vendored"]

View File

@@ -1,10 +1,9 @@
[package]
name = "deltachat-contact-tools"
version = "0.1.0"
version = "0.0.0" # No semver-stable versioning
edition = "2021"
description = "Contact-related tools, like parsing vcards and sanitizing name and address"
description = "Contact-related tools, like parsing vcards and sanitizing name and address. Meant for internal use in the deltachat crate."
license = "MPL-2.0"
# TODO maybe it should be called "deltachat-text-utils" or similar?
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
@@ -13,6 +12,7 @@ anyhow = { workspace = true }
once_cell = { workspace = true }
regex = { workspace = true }
rusqlite = { workspace = true } # Needed in order to `impl rusqlite::types::ToSql for EmailAddress`. Could easily be put behind a feature.
chrono = { workspace = true }
[dev-dependencies]
anyhow = { workspace = true, features = ["backtrace"] } # Enable `backtrace` feature in tests.

View File

@@ -29,10 +29,181 @@ use std::fmt;
use std::ops::Deref;
use anyhow::bail;
use anyhow::Context as _;
use anyhow::Result;
use chrono::{DateTime, NaiveDateTime};
use once_cell::sync::Lazy;
use regex::Regex;
// TODOs to clean up:
// - Check if sanitizing is done correctly everywhere
// - Apply lints everywhere (https://doc.rust-lang.org/cargo/reference/workspaces.html#the-lints-table)
#[derive(Debug)]
/// A Contact, as represented in a VCard.
pub struct VcardContact {
/// The email address, vcard property `email`
pub addr: String,
/// The contact's display name, vcard property `fn`
pub display_name: String,
/// The contact's public PGP key in Base64, vcard property `key`
pub key: Option<String>,
/// The contact's profile image (=avatar) in Base64, vcard property `photo`
pub profile_image: Option<String>,
/// The timestamp when the vcard was created / last updated, vcard property `rev`
pub timestamp: Result<u64>,
}
/// Returns a vCard containing given contacts.
///
/// Calling [`parse_vcard()`] on the returned result is a reverse operation.
pub fn make_vcard(contacts: &[VcardContact]) -> String {
fn format_timestamp(c: &VcardContact) -> Option<String> {
let timestamp = *c.timestamp.as_ref().ok()?;
let timestamp: i64 = timestamp.try_into().ok()?;
let datetime = DateTime::from_timestamp(timestamp, 0)?;
Some(datetime.format("%Y%m%dT%H%M%SZ").to_string())
}
let mut res = "".to_string();
for c in contacts {
let addr = &c.addr;
let display_name = match c.display_name.is_empty() {
false => &c.display_name,
true => &c.addr,
};
res += &format!(
"BEGIN:VCARD\n\
VERSION:4.0\n\
EMAIL:{addr}\n\
FN:{display_name}\n"
);
if let Some(key) = &c.key {
res += &format!("KEY:data:application/pgp-keys;base64,{key}\n");
}
if let Some(profile_image) = &c.profile_image {
res += &format!("PHOTO:data:image/jpeg;base64,{profile_image}\n");
}
if let Some(timestamp) = format_timestamp(c) {
res += &format!("REV:{timestamp}\n");
}
res += "END:VCARD\n";
}
res
}
/// Parses `VcardContact`s from a given `&str`.
pub fn parse_vcard(vcard: &str) -> Result<Vec<VcardContact>> {
fn remove_prefix<'a>(s: &'a str, prefix: &str) -> Option<&'a str> {
let start_of_s = s.get(..prefix.len())?;
if start_of_s.eq_ignore_ascii_case(prefix) {
s.get(prefix.len()..)
} else {
None
}
}
fn vcard_property<'a>(s: &'a str, property: &str) -> Option<&'a str> {
let remainder = remove_prefix(s, property)?;
// If `s` is `EMAIL;TYPE=work:alice@example.com` and `property` is `EMAIL`,
// then `remainder` is now `;TYPE=work:alice@example.com`
// TODO this doesn't handle the case where there are quotes around a colon
let (params, value) = remainder.split_once(':')?;
// In the example from above, `params` is now `;TYPE=work`
// and `value` is now `alice@example.com`
if params
.chars()
.next()
.filter(|c| !c.is_ascii_punctuation() || *c == '_')
.is_some()
{
// `s` started with `property`, but the next character after it was not punctuation,
// so this line's property is actually something else
return None;
}
Some(value)
}
fn parse_datetime(datetime: &str) -> Result<u64> {
// According to https://www.rfc-editor.org/rfc/rfc6350#section-4.3.5, the timestamp
// is in ISO.8601.2004 format. DateTime::parse_from_rfc3339() apparently parses
// ISO.8601, but fails to parse any of the examples given.
// So, instead just parse using a format string.
// Parses 19961022T140000Z, 19961022T140000-05, or 19961022T140000-0500.
let timestamp = match DateTime::parse_from_str(datetime, "%Y%m%dT%H%M%S%#z") {
Ok(datetime) => datetime.timestamp(),
// Parses 19961022T140000.
Err(e) => match NaiveDateTime::parse_from_str(datetime, "%Y%m%dT%H%M%S") {
Ok(datetime) => datetime
.and_local_timezone(chrono::offset::Local)
.single()
.context("Could not apply local timezone to parsed date and time")?
.timestamp(),
Err(_) => return Err(e.into()),
},
};
Ok(timestamp.try_into()?)
}
let mut lines = vcard.lines().peekable();
let mut contacts = Vec::new();
while lines.peek().is_some() {
// Skip to the start of the vcard:
for line in lines.by_ref() {
if line.eq_ignore_ascii_case("BEGIN:VCARD") {
break;
}
}
let mut display_name = None;
let mut addr = None;
let mut key = None;
let mut photo = None;
let mut datetime = None;
for line in lines.by_ref() {
if let Some(email) = vcard_property(line, "email") {
addr.get_or_insert(email);
} else if let Some(name) = vcard_property(line, "fn") {
display_name.get_or_insert(name);
} else if let Some(k) = remove_prefix(line, "KEY;PGP;ENCODING=BASE64:")
.or_else(|| remove_prefix(line, "KEY;TYPE=PGP;ENCODING=b:"))
.or_else(|| remove_prefix(line, "KEY:data:application/pgp-keys;base64,"))
{
key.get_or_insert(k);
} else if let Some(p) = remove_prefix(line, "PHOTO;JPEG;ENCODING=BASE64:")
.or_else(|| remove_prefix(line, "PHOTO;TYPE=JPEG;ENCODING=b:"))
.or_else(|| remove_prefix(line, "PHOTO;ENCODING=BASE64;TYPE=JPEG:"))
.or_else(|| remove_prefix(line, "PHOTO:data:image/jpeg;base64,"))
{
photo.get_or_insert(p);
} else if let Some(rev) = vcard_property(line, "rev") {
datetime.get_or_insert(rev);
} else if line.eq_ignore_ascii_case("END:VCARD") {
break;
}
}
let (display_name, addr) =
sanitize_name_and_addr(display_name.unwrap_or(""), addr.unwrap_or(""));
contacts.push(VcardContact {
display_name,
addr,
key: key.map(|s| s.to_string()),
profile_image: photo.map(|s| s.to_string()),
timestamp: datetime
.context("No timestamp in vcard")
.and_then(parse_datetime),
});
}
Ok(contacts)
}
/// Valid contact address.
#[derive(Debug, Clone)]
pub struct ContactAddress(String);
@@ -81,14 +252,10 @@ impl rusqlite::types::ToSql for ContactAddress {
/// Make the name and address
pub fn sanitize_name_and_addr(name: &str, addr: &str) -> (String, String) {
static ADDR_WITH_NAME_REGEX: Lazy<Regex> = Lazy::new(|| Regex::new("(.*)<(.*)>").unwrap());
if let Some(captures) = ADDR_WITH_NAME_REGEX.captures(addr.as_ref()) {
let (name, addr) = if let Some(captures) = ADDR_WITH_NAME_REGEX.captures(addr.as_ref()) {
(
if name.is_empty() {
strip_rtlo_characters(
&captures
.get(1)
.map_or("".to_string(), |m| normalize_name(m.as_str())),
)
strip_rtlo_characters(captures.get(1).map_or("", |m| m.as_str()))
} else {
strip_rtlo_characters(name)
},
@@ -97,8 +264,21 @@ pub fn sanitize_name_and_addr(name: &str, addr: &str) -> (String, String) {
.map_or("".to_string(), |m| m.as_str().to_string()),
)
} else {
(strip_rtlo_characters(name), addr.to_string())
(
strip_rtlo_characters(&normalize_name(name)),
addr.to_string(),
)
};
let mut name = normalize_name(&name);
// If the 'display name' is just the address, remove it:
// Otherwise, the contact would sometimes be shown as "alice@example.com (alice@example.com)" (see `get_name_n_addr()`).
// If the display name is empty, DC will just show the address when it needs a display name.
if name == addr {
name = "".to_string();
}
(name, addr)
}
/// Normalize a name.
@@ -230,8 +410,105 @@ impl rusqlite::types::ToSql for EmailAddress {
#[cfg(test)]
mod tests {
use chrono::TimeZone;
use super::*;
#[test]
fn test_vcard_thunderbird() {
let contacts = parse_vcard(
"BEGIN:VCARD
VERSION:4.0
FN:'Alice Mueller'
EMAIL;PREF=1:alice.mueller@posteo.de
UID:a8083264-ca47-4be7-98a8-8ec3db1447ca
END:VCARD
BEGIN:VCARD
VERSION:4.0
FN:'bobzzz@freenet.de'
EMAIL;PREF=1:bobzzz@freenet.de
UID:cac4fef4-6351-4854-bbe4-9b6df857eaed
END:VCARD
",
)
.unwrap();
assert_eq!(contacts[0].addr, "alice.mueller@posteo.de".to_string());
assert_eq!(contacts[0].display_name, "Alice Mueller".to_string());
assert_eq!(contacts[0].key, None);
assert_eq!(contacts[0].profile_image, None);
assert!(contacts[0].timestamp.is_err());
assert_eq!(contacts[1].addr, "bobzzz@freenet.de".to_string());
assert_eq!(contacts[1].display_name, "".to_string());
assert_eq!(contacts[1].key, None);
assert_eq!(contacts[1].profile_image, None);
assert!(contacts[1].timestamp.is_err());
assert_eq!(contacts.len(), 2);
}
#[test]
fn test_vcard_simple_example() {
let contacts = parse_vcard(
"BEGIN:VCARD
VERSION:4.0
FN:Alice Wonderland
N:Wonderland;Alice;;;Ms.
GENDER:W
EMAIL;TYPE=work:alice@example.com
KEY;TYPE=PGP;ENCODING=b:[base64-data]
REV:20240418T184242Z
END:VCARD",
)
.unwrap();
assert_eq!(contacts[0].addr, "alice@example.com".to_string());
assert_eq!(contacts[0].display_name, "Alice Wonderland".to_string());
assert_eq!(contacts[0].key, Some("[base64-data]".to_string()));
assert_eq!(contacts[0].profile_image, None);
assert_eq!(*contacts[0].timestamp.as_ref().unwrap(), 1713465762);
assert_eq!(contacts.len(), 1);
}
#[test]
fn test_make_and_parse_vcard() {
let contacts = [
VcardContact {
addr: "alice@example.org".to_string(),
display_name: "Alice Wonderland".to_string(),
key: Some("[base64-data]".to_string()),
profile_image: Some("image in Base64".to_string()),
timestamp: Ok(1713465762),
},
VcardContact {
addr: "bob@example.com".to_string(),
display_name: "".to_string(),
key: None,
profile_image: None,
timestamp: Ok(0),
},
];
for len in 0..=contacts.len() {
let contacts = &contacts[0..len];
let vcard = make_vcard(contacts);
let parsed = parse_vcard(&vcard).unwrap();
assert_eq!(parsed.len(), contacts.len());
for i in 0..parsed.len() {
assert_eq!(parsed[i].addr, contacts[i].addr);
assert_eq!(parsed[i].display_name, contacts[i].display_name);
assert_eq!(parsed[i].key, contacts[i].key);
assert_eq!(parsed[i].profile_image, contacts[i].profile_image);
assert_eq!(
parsed[i].timestamp.as_ref().unwrap(),
contacts[i].timestamp.as_ref().unwrap()
);
}
}
}
#[test]
fn test_contact_address() -> Result<()> {
let alice_addr = "alice@example.org";
@@ -277,4 +554,62 @@ mod tests {
assert!(EmailAddress::new("u@tt").is_ok());
assert_eq!(EmailAddress::new("@d.tt").is_ok(), false);
}
#[test]
fn test_vcard_android() {
let contacts = parse_vcard(
"BEGIN:VCARD
VERSION:2.1
N:;Bob;;;
FN:Bob
TEL;CELL:+1-234-567-890
EMAIL;HOME:bob@example.org
END:VCARD
BEGIN:VCARD
VERSION:2.1
N:;Alice;;;
FN:Alice
EMAIL;HOME:alice@example.org
END:VCARD
",
)
.unwrap();
assert_eq!(contacts[0].addr, "bob@example.org".to_string());
assert_eq!(contacts[0].display_name, "Bob".to_string());
assert_eq!(contacts[0].key, None);
assert_eq!(contacts[0].profile_image, None);
assert_eq!(contacts[1].addr, "alice@example.org".to_string());
assert_eq!(contacts[1].display_name, "Alice".to_string());
assert_eq!(contacts[1].key, None);
assert_eq!(contacts[1].profile_image, None);
assert_eq!(contacts.len(), 2);
}
#[test]
fn test_vcard_local_datetime() {
let contacts = parse_vcard(
"BEGIN:VCARD\n\
VERSION:4.0\n\
FN:Alice Wonderland\n\
EMAIL;TYPE=work:alice@example.org\n\
REV:20240418T184242\n\
END:VCARD",
)
.unwrap();
assert_eq!(contacts.len(), 1);
assert_eq!(contacts[0].addr, "alice@example.org".to_string());
assert_eq!(contacts[0].display_name, "Alice Wonderland".to_string());
assert_eq!(
*contacts[0].timestamp.as_ref().unwrap(),
chrono::offset::Local
.with_ymd_and_hms(2024, 4, 18, 18, 42, 42)
.unwrap()
.timestamp()
.try_into()
.unwrap()
);
}
}

View File

@@ -1,6 +1,6 @@
[package]
name = "deltachat_ffi"
version = "1.137.3"
version = "1.137.4"
description = "Deltachat FFI"
edition = "2018"
readme = "README.md"

View File

@@ -4105,6 +4105,19 @@ char* dc_msg_get_subject (const dc_msg_t* msg);
char* dc_msg_get_file (const dc_msg_t* msg);
/**
* Save file copy at the user-provided path.
*
* Fails if file already exists at the provided path.
*
* @memberof dc_msg_t
* @param msg The message object.
* @param path Destination file path with filename and extension.
* @return 0 on failure, 1 on success.
*/
int dc_msg_save_file (const dc_msg_t* msg, const char* path);
/**
* Get an original attachment filename, with extension but without the path. To get the full path,
* use dc_msg_get_file().
@@ -4377,9 +4390,9 @@ int dc_msg_has_deviating_timestamp(const dc_msg_t* msg);
/**
* Check if a message has a location bound to it.
* These messages are also returned by dc_get_locations()
* and the UI may decide to display a special icon beside such messages,
* Check if a message has a POI location bound to it.
* These locations are also returned by dc_get_locations()
* The UI may decide to display a special icon beside such messages.
*
* @memberof dc_msg_t
* @param msg The message object.

View File

@@ -3368,6 +3368,34 @@ pub unsafe extern "C" fn dc_msg_get_file(msg: *mut dc_msg_t) -> *mut libc::c_cha
.unwrap_or_else(|| "".strdup())
}
#[no_mangle]
pub unsafe extern "C" fn dc_msg_save_file(
msg: *mut dc_msg_t,
path: *const libc::c_char,
) -> libc::c_int {
if msg.is_null() || path.is_null() {
eprintln!("ignoring careless call to dc_msg_save_file()");
return 0;
}
let ffi_msg = &*msg;
let ctx = &*ffi_msg.context;
let path = to_string_lossy(path);
let r = block_on(
ffi_msg
.message
.save_file(ctx, &std::path::PathBuf::from(path)),
);
match r {
Ok(()) => 1,
Err(_) => {
r.context("Failed to save file from message")
.log_err(ctx)
.unwrap_or_default();
0
}
}
}
#[no_mangle]
pub unsafe extern "C" fn dc_msg_get_filename(msg: *mut dc_msg_t) -> *mut libc::c_char {
if msg.is_null() {

View File

@@ -1,6 +1,6 @@
[package]
name = "deltachat-jsonrpc"
version = "1.137.3"
version = "1.137.4"
description = "DeltaChat JSON-RPC API"
edition = "2021"
default-run = "deltachat-jsonrpc-server"
@@ -16,11 +16,11 @@ required-features = ["webserver"]
anyhow = "1"
deltachat = { path = ".." }
num-traits = "0.2"
schemars = "0.8.13"
schemars = "0.8.19"
serde = { version = "1.0", features = ["derive"] }
tempfile = "3.10.1"
log = "0.4"
async-channel = { version = "2.0.0" }
async-channel = { version = "2.2.1" }
futures = { version = "0.3.30" }
serde_json = "1"
yerpc = { version = "0.5.2", features = ["anyhow_expose", "openrpc"] }

View File

@@ -1,4 +1,5 @@
use std::collections::BTreeMap;
use std::path::Path;
use std::sync::Arc;
use std::time::Duration;
use std::{collections::HashMap, str::FromStr};
@@ -1770,6 +1771,29 @@ impl CommandApi {
Ok(general_purpose::STANDARD_NO_PAD.encode(blob))
}
/// Sets Webxdc file as integration.
/// `file` is the .xdc to use as Webxdc integration.
async fn set_webxdc_integration(&self, account_id: u32, file_path: String) -> Result<()> {
let ctx = self.get_context(account_id).await?;
ctx.set_webxdc_integration(&file_path).await
}
/// Returns Webxdc instance used for optional integrations.
/// UI can open the Webxdc as usual.
/// Returns `None` if there is no integration; the caller can add one using `set_webxdc_integration` then.
/// `integrate_for` is the chat to get the integration for.
async fn init_webxdc_integration(
&self,
account_id: u32,
chat_id: Option<u32>,
) -> Result<Option<u32>> {
let ctx = self.get_context(account_id).await?;
Ok(ctx
.init_webxdc_integration(chat_id.map(ChatId::new))
.await?
.map(|msg_id| msg_id.to_u32()))
}
/// Makes an HTTP GET request and returns a response.
///
/// `url` is the HTTP or HTTPS URL.
@@ -1878,6 +1902,15 @@ impl CommandApi {
Ok(can_send)
}
/// Saves a file copy at the user-provided path.
///
/// Fails if file already exists at the provided path.
async fn save_msg_file(&self, account_id: u32, msg_id: u32, path: String) -> Result<()> {
let ctx = self.get_context(account_id).await?;
let message = Message::load_from_db(&ctx, MsgId::new(msg_id)).await?;
message.save_file(&ctx, Path::new(&path)).await
}
// ---------------------------------------------
// functions for the composer
// the composer is the message input field
@@ -1950,19 +1983,21 @@ impl CommandApi {
);
let destination_path = account_folder.join("stickers").join(collection);
fs::create_dir_all(&destination_path).await?;
let file = message.get_file(&ctx).context("no file")?;
fs::copy(
&file,
destination_path.join(format!(
"{}.{}",
msg_id,
file.extension()
.unwrap_or_default()
.to_str()
.unwrap_or_default()
)),
)
.await?;
let file = message.get_filename().context("no file?")?;
message
.save_file(
&ctx,
&destination_path.join(format!(
"{}.{}",
msg_id,
Path::new(&file)
.extension()
.unwrap_or_default()
.to_str()
.unwrap_or_default()
)),
)
.await?;
Ok(())
}

View File

@@ -35,6 +35,10 @@ pub struct MessageObject {
parent_id: Option<u32>,
text: String,
/// Check if a message has a POI location bound to it.
/// These locations are also returned by `get_locations` method.
/// The UI may decide to display a special icon beside such messages.
has_location: bool,
has_html: bool,
view_type: MessageViewtype,

View File

@@ -53,5 +53,5 @@
},
"type": "module",
"types": "dist/deltachat.d.ts",
"version": "1.137.3"
"version": "1.137.4"
}

View File

@@ -1,6 +1,6 @@
[package]
name = "deltachat-repl"
version = "1.137.3"
version = "1.137.4"
license = "MPL-2.0"
edition = "2021"
repository = "https://github.com/deltachat/deltachat-core-rust"

View File

@@ -4,6 +4,7 @@
it will echo back any text send to it, it also will print to console all Delta Chat core events.
Pass --help to the CLI to see available options.
"""
from deltachat_rpc_client import events, run_bot_cli
hooks = events.HookCollection()

View File

@@ -3,6 +3,7 @@
it will echo back any message that has non-empty text and also supports the /help command.
"""
import logging
import sys
from threading import Thread

View File

@@ -2,6 +2,7 @@
"""
Example echo bot without using hooks
"""
import logging
import sys

View File

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "deltachat-rpc-client"
version = "1.137.3"
version = "1.137.4"
description = "Python client for Delta Chat core JSON-RPC interface"
classifiers = [
"Development Status :: 5 - Production/Stable",
@@ -37,7 +37,7 @@ deltachat_rpc_client = [
line-length = 120
[tool.ruff]
select = [
lint.select = [
"E", "W", # pycodestyle
"F", # Pyflakes
"N", # pep8-naming

View File

@@ -22,9 +22,8 @@ skipsdist = True
skip_install = True
deps =
ruff
black
commands =
black --quiet --check --diff src/ examples/ tests/
ruff format --quiet --diff src/ examples/ tests/
ruff check src/ examples/ tests/
[pytest]

View File

@@ -1,6 +1,6 @@
[package]
name = "deltachat-rpc-server"
version = "1.137.3"
version = "1.137.4"
description = "DeltaChat JSON-RPC server"
edition = "2021"
readme = "README.md"

View File

@@ -55,5 +55,5 @@
"test:mocha": "mocha node/test/test.mjs --growl --reporter=spec --bail --exit"
},
"types": "node/dist/index.d.ts",
"version": "1.137.3"
"version": "1.137.4"
}

View File

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "deltachat"
version = "1.137.3"
version = "1.137.4"
description = "Python bindings for the Delta Chat Core library using CFFI against the Rust-implemented libdeltachat"
readme = "README.rst"
requires-python = ">=3.7"
@@ -46,7 +46,7 @@ deltachat = [
line-length = 120
[tool.ruff]
select = ["E", "F", "W", "YTT", "C4", "ISC", "ICN", "TID", "DTZ", "PLC", "PLE", "PLW", "PIE", "COM", "UP004", "UP010", "UP031", "UP032", "ANN204"]
lint.select = ["E", "F", "W", "YTT", "C4", "ISC", "ICN", "TID", "DTZ", "PLC", "PLE", "PLW", "PIE", "COM", "UP004", "UP010", "UP031", "UP032", "ANN204"]
line-length = 120
[tool.isort]

View File

@@ -18,14 +18,14 @@ def test_db_busy_error(acfactory):
# make a number of accounts
accounts = acfactory.get_many_online_accounts(3)
log("created %s accounts" % len(accounts))
log(f"created {len(accounts)} accounts")
# put a bigfile into each account
for acc in accounts:
acc.bigfile = os.path.join(acc.get_blobdir(), "bigfile")
with open(acc.bigfile, "wb") as f:
f.write(b"01234567890" * 1000_000)
log("created %s bigfiles" % len(accounts))
log(f"created {len(accounts)} bigfiles")
contact_addrs = [acc.get_self_contact().addr for acc in accounts]
chat = accounts[0].create_group_chat("stress-group")

View File

@@ -1343,7 +1343,6 @@ def test_quote_encrypted(acfactory, lp):
for quoted_msg in msg1, msg3:
# Save the draft with a quote.
# It should be encrypted if quoted message is encrypted.
msg_draft = Message.new_empty(ac1, "text")
msg_draft.set_text("message reply")
msg_draft.quote = quoted_msg
@@ -1357,10 +1356,14 @@ def test_quote_encrypted(acfactory, lp):
chat.set_draft(None)
assert chat.get_draft() is None
# Quote should be replaced with "..." if quoted message is encrypted.
msg_in = ac2._evtracker.wait_next_incoming_message()
assert msg_in.text == "message reply"
assert msg_in.quoted_text == quoted_msg.text
assert msg_in.is_encrypted() == quoted_msg.is_encrypted()
assert not msg_in.is_encrypted()
if quoted_msg.is_encrypted():
assert msg_in.quoted_text == "..."
else:
assert msg_in.quoted_text == quoted_msg.text
def test_quote_attachment(tmp_path, acfactory, lp):
@@ -2033,14 +2036,15 @@ def test_send_receive_locations(acfactory, lp):
assert chat1.is_sending_locations()
ac1._evtracker.get_matching("DC_EVENT_SMTP_MESSAGE_SENT")
# Wait for "enabled location streaming" message.
ac2._evtracker.wait_next_incoming_message()
# First location is sent immediately as a location-only message.
ac1.set_location(latitude=2.0, longitude=3.0, accuracy=0.5)
ac1._evtracker.get_matching("DC_EVENT_LOCATION_CHANGED")
chat1.send_text("🍞")
ac1._evtracker.get_matching("DC_EVENT_SMTP_MESSAGE_SENT")
lp.sec("ac2: wait for incoming location message")
# currently core emits location changed before event_incoming message
ac2._evtracker.get_matching("DC_EVENT_LOCATION_CHANGED")
locations = chat2.get_locations()
@@ -2049,7 +2053,7 @@ def test_send_receive_locations(acfactory, lp):
assert locations[0].longitude == 3.0
assert locations[0].accuracy == 0.5
assert locations[0].timestamp > now
assert locations[0].marker == "🍞"
assert locations[0].marker is None
contact = ac2.create_contact(ac1)
locations2 = chat2.get_locations(contact=contact)

View File

@@ -41,12 +41,11 @@ skipsdist = True
skip_install = True
deps =
ruff
black
# pygments required by rst-lint
pygments
restructuredtext_lint
commands =
black --quiet --check --diff setup.py src/deltachat examples/ tests/
ruff format --quiet --diff setup.py src/deltachat examples/ tests/
ruff check src/deltachat tests/ examples/
rst-lint --encoding 'utf-8' README.rst

View File

@@ -1 +1 @@
2024-04-16
2024-04-24

View File

@@ -7,7 +7,7 @@ set -euo pipefail
#
# Avoid using rustup here as it depends on reading /proc/self/exe and
# has problems running under QEMU.
RUST_VERSION=1.77.1
RUST_VERSION=1.78.0
ARCH="$(uname -m)"
test -f "/lib/libc.musl-$ARCH.so.1" && LIBC=musl || LIBC=gnu

View File

@@ -14,7 +14,6 @@ use once_cell::sync::Lazy;
use crate::config::Config;
use crate::context::Context;
use crate::headerdef::HeaderDef;
use crate::tools::time;
/// `authres` is short for the Authentication-Results header, defined in
/// <https://datatracker.ietf.org/doc/html/rfc8601>, which contains info
@@ -29,45 +28,28 @@ pub(crate) async fn handle_authres(
context: &Context,
mail: &ParsedMail<'_>,
from: &str,
message_time: i64,
) -> Result<DkimResults> {
let from_domain = match EmailAddress::new(from) {
Ok(email) => email.domain,
Err(e) => {
// This email is invalid, but don't return an error, we still want to
// add a stub to the database so that it's not downloaded again
return Err(anyhow::format_err!("invalid email {}: {:#}", from, e));
}
};
let authres = parse_authres_headers(&mail.get_headers(), &from_domain);
update_authservid_candidates(context, &authres).await?;
compute_dkim_results(context, authres, &from_domain, message_time).await
compute_dkim_results(context, authres).await
}
#[derive(Debug)]
pub(crate) struct DkimResults {
/// Whether DKIM passed for this particular e-mail.
pub dkim_passed: bool,
/// Whether DKIM is known to work for e-mails coming from the sender's domain,
/// i.e. whether we expect DKIM to work.
pub dkim_should_work: bool,
/// Whether changing the public Autocrypt key should be allowed.
/// This is false if we expected DKIM to work (dkim_works=true),
/// but it failed now (dkim_passed=false).
pub allow_keychange: bool,
}
impl fmt::Display for DkimResults {
fn fmt(&self, fmt: &mut fmt::Formatter) -> fmt::Result {
write!(
fmt,
"DKIM Results: Passed={}, Works={}, Allow_Keychange={}",
self.dkim_passed, self.dkim_should_work, self.allow_keychange
)?;
if !self.allow_keychange {
write!(fmt, " KEYCHANGES NOT ALLOWED!!!!")?;
}
write!(fmt, "DKIM Results: Passed={}", self.dkim_passed)?;
Ok(())
}
}
@@ -218,10 +200,6 @@ async fn update_authservid_candidates(
context
.set_config_internal(Config::AuthservIdCandidates, Some(&new_config))
.await?;
// Updating the authservid candidates may mean that we now consider
// emails as "failed" which "passed" previously, so we need to
// reset our expectation which DKIMs work.
clear_dkim_works(context).await?
}
Ok(())
}
@@ -238,8 +216,6 @@ async fn update_authservid_candidates(
async fn compute_dkim_results(
context: &Context,
mut authres: ParsedAuthresHeaders,
from_domain: &str,
message_time: i64,
) -> Result<DkimResults> {
let mut dkim_passed = false;
@@ -272,71 +248,7 @@ async fn compute_dkim_results(
}
}
let last_working_timestamp = dkim_works_timestamp(context, from_domain).await?;
let mut dkim_should_work = dkim_should_work(last_working_timestamp)?;
if message_time > last_working_timestamp && dkim_passed {
set_dkim_works_timestamp(context, from_domain, message_time).await?;
dkim_should_work = true;
}
Ok(DkimResults {
dkim_passed,
dkim_should_work,
allow_keychange: dkim_passed || !dkim_should_work,
})
}
/// Whether DKIM in emails from this domain should be considered to work.
fn dkim_should_work(last_working_timestamp: i64) -> Result<bool> {
// When we get an email with valid DKIM-Authentication-Results,
// then we assume that DKIM works for 30 days from this time on.
let should_work_until = last_working_timestamp + 3600 * 24 * 30;
let dkim_ever_worked = last_working_timestamp > 0;
// We're using time() here and not the time when the message
// claims to have been sent (passed around as `message_time`)
// because otherwise an attacker could just put a time way
// in the future into the `Date` header and then we would
// assume that DKIM doesn't have to be valid anymore.
let dkim_should_work_now = should_work_until > time();
Ok(dkim_ever_worked && dkim_should_work_now)
}
async fn dkim_works_timestamp(context: &Context, from_domain: &str) -> Result<i64, anyhow::Error> {
let last_working_timestamp: i64 = context
.sql
.query_get_value(
"SELECT dkim_works FROM sending_domains WHERE domain=?",
(from_domain,),
)
.await?
.unwrap_or(0);
Ok(last_working_timestamp)
}
async fn set_dkim_works_timestamp(
context: &Context,
from_domain: &str,
timestamp: i64,
) -> Result<()> {
context
.sql
.execute(
"INSERT INTO sending_domains (domain, dkim_works) VALUES (?,?)
ON CONFLICT(domain) DO UPDATE SET dkim_works=excluded.dkim_works",
(from_domain, timestamp),
)
.await?;
Ok(())
}
async fn clear_dkim_works(context: &Context) -> Result<()> {
context
.sql
.execute("DELETE FROM sending_domains", ())
.await?;
Ok(())
Ok(DkimResults { dkim_passed })
}
fn parse_authservid_candidates_config(config: &Option<String>) -> BTreeSet<&str> {
@@ -349,19 +261,12 @@ fn parse_authservid_candidates_config(config: &Option<String>) -> BTreeSet<&str>
#[cfg(test)]
mod tests {
#![allow(clippy::indexing_slicing)]
use std::time::Duration;
use tokio::fs;
use tokio::io::AsyncReadExt;
use super::*;
use crate::aheader::EncryptPreference;
use crate::e2ee;
use crate::mimeparser;
use crate::peerstate::Peerstate;
use crate::securejoin::get_securejoin_qr;
use crate::securejoin::join_securejoin;
use crate::test_utils;
use crate::test_utils::TestContext;
use crate::test_utils::TestContextManager;
use crate::tools;
@@ -574,33 +479,8 @@ Authentication-Results: box.hispanilandia.net; spf=pass smtp.mailfrom=adbenitez@
let mail = mailparse::parse_mail(&bytes)?;
let from = &mimeparser::get_from(&mail.headers).unwrap().addr;
let res = handle_authres(&t, &mail, from, time()).await?;
assert!(res.allow_keychange);
}
for entry in &dir {
let mut file = fs::File::open(entry.path()).await?;
bytes.clear();
file.read_to_end(&mut bytes).await.unwrap();
let mail = mailparse::parse_mail(&bytes)?;
let from = &mimeparser::get_from(&mail.headers).unwrap().addr;
let res = handle_authres(&t, &mail, from, time()).await?;
if !res.allow_keychange {
println!(
"!!!!!! FAILURE Receiving {:?}, keychange is not allowed !!!!!!",
entry.path()
);
test_failed = true;
}
let res = handle_authres(&t, &mail, from).await?;
let from_domain = EmailAddress::new(from).unwrap().domain;
assert_eq!(
res.dkim_should_work,
dkim_should_work(dkim_works_timestamp(&t, &from_domain).await?)?
);
assert_eq!(res.dkim_passed, res.dkim_should_work);
// delta.blinzeln.de and gmx.de have invalid DKIM, so the DKIM check should fail
let expected_result = (from_domain != "delta.blinzeln.de") && (from_domain != "gmx.de")
@@ -613,9 +493,8 @@ Authentication-Results: box.hispanilandia.net; spf=pass smtp.mailfrom=adbenitez@
if res.dkim_passed != expected_result {
if authres_parsing_works {
println!(
"!!!!!! FAILURE Receiving {:?}, order {:#?} wrong result: !!!!!!",
"!!!!!! FAILURE Receiving {:?} wrong result: !!!!!!",
entry.path(),
dir.iter().map(|e| e.file_name()).collect::<Vec<_>>()
);
test_failed = true;
}
@@ -638,116 +517,7 @@ Authentication-Results: box.hispanilandia.net; spf=pass smtp.mailfrom=adbenitez@
let bytes = b"From: invalid@from.com
Authentication-Results: dkim=";
let mail = mailparse::parse_mail(bytes).unwrap();
handle_authres(&t, &mail, "invalid@rom.com", time())
.await
.unwrap();
}
#[ignore = "Disallowing keychanges is disabled for now"]
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_handle_authres_fails() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = tcm.alice().await;
let bob = tcm.bob().await;
// Bob sends Alice a message, so she gets his key
tcm.send_recv_accept(&bob, &alice, "Hi").await;
// We don't need bob anymore, let's make sure it's not accidentally used
drop(bob);
// Assume Alice receives an email from bob@example.net with
// correct DKIM -> `set_dkim_works()` was called
set_dkim_works_timestamp(&alice, "example.net", time()).await?;
// And Alice knows her server's authserv-id
alice
.set_config(Config::AuthservIdCandidates, Some("example.org"))
.await?;
tcm.section("An attacker, bob2, sends a from-forged email to Alice!");
// Sleep to make sure key reset is ignored because of DKIM failure
// and not because reordering is suspected.
tokio::time::sleep(std::time::Duration::from_millis(1100)).await;
let bob2 = tcm.unconfigured().await;
bob2.configure_addr("bob@example.net").await;
e2ee::ensure_secret_key_exists(&bob2).await?;
let chat = bob2.create_chat(&alice).await;
let mut sent = bob2
.send_text(chat.id, "Please send me lots of money")
.await;
sent.payload
.insert_str(0, "Authentication-Results: example.org; dkim=fail\n");
let received = alice.recv_msg(&sent).await;
// Assert that the error tells the user about the problem
assert!(received.error.unwrap().contains("DKIM failed"));
let bob_state = Peerstate::from_addr(&alice, "bob@example.net")
.await?
.unwrap();
// Encryption preference is still mutual.
assert_eq!(bob_state.prefer_encrypt, EncryptPreference::Mutual);
// Also check that the keypair was not changed
assert_eq!(
bob_state.public_key.unwrap(),
test_utils::bob_keypair().public
);
// Since Alice didn't change the key, Bob can't read her message
let received = tcm
.try_send_recv(&alice, &bob2, "My credit card number is 1234")
.await;
assert!(!received.text.contains("1234"));
assert!(received.error.is_some());
tcm.section("Turns out bob2 wasn't an attacker at all, Bob just has a new phone and DKIM just stopped working.");
tcm.section("To fix the key problems, Bob scans Alice's QR code.");
let qr = get_securejoin_qr(&alice.ctx, None).await.unwrap();
join_securejoin(&bob2.ctx, &qr).await.unwrap();
loop {
if let Some(mut sent) = bob2.pop_sent_msg_opt(Duration::ZERO).await {
sent.payload
.insert_str(0, "Authentication-Results: example.org; dkim=fail\n");
alice.recv_msg(&sent).await;
} else if let Some(sent) = alice.pop_sent_msg_opt(Duration::ZERO).await {
bob2.recv_msg(&sent).await;
} else {
break;
}
}
// Unfortunately, securejoin currently doesn't work with authres-checking,
// so these checks would fail:
// let contact_bob = alice.add_or_lookup_contact(&bob2).await;
// assert_eq!(
// contact_bob.is_verified(&alice.ctx).await.unwrap(),
// VerifiedStatus::BidirectVerified
// );
// let contact_alice = bob2.add_or_lookup_contact(&alice).await;
// assert_eq!(
// contact_alice.is_verified(&bob2.ctx).await.unwrap(),
// VerifiedStatus::BidirectVerified
// );
// // Bob can read Alice's messages again
// let received = tcm
// .try_send_recv(&alice, &bob2, "Can you read this again?")
// .await;
// assert_eq!(received.text.as_ref().unwrap(), "Can you read this again?");
// assert!(received.error.is_none());
Ok(())
handle_authres(&t, &mail, "invalid@rom.com").await.unwrap();
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
@@ -796,10 +566,7 @@ Authentication-Results: dkim=";
let alice = tcm.alice().await;
let bob = tcm.bob().await;
// Assume Bob received an email from something@example.net with
// correct DKIM -> `set_dkim_works()` was called
set_dkim_works_timestamp(&bob, "example.org", time()).await?;
// And Bob knows his server's authserv-id
// Bob knows his server's authserv-id
bob.set_config(Config::AuthservIdCandidates, Some("example.net"))
.await?;
@@ -821,15 +588,13 @@ Authentication-Results: dkim=";
.insert_str(0, "Authentication-Results: example.net; dkim=fail\n");
let rcvd = bob.recv_msg(&sent).await;
// Disallowing keychanges is disabled for now:
// assert!(rcvd.error.unwrap().contains("DKIM failed"));
// The message info should contain a warning:
assert!(rcvd
.id
.get_info(&bob)
.await
.unwrap()
.contains("KEYCHANGES NOT ALLOWED"));
.contains("DKIM Results: Passed=false"));
Ok(())
}

View File

@@ -698,7 +698,10 @@ fn encode_img(
ImageOutputFormat::Png => img.write_to(&mut buf, ImageFormat::Png)?,
ImageOutputFormat::Jpeg { quality } => {
let encoder = JpegEncoder::new_with_quality(&mut buf, quality);
img.write_with_encoder(encoder)?;
// Convert image into RGB8 to avoid the error
// "The encoder or decoder for Jpeg does not support the color type Rgba8"
// (<https://github.com/image-rs/image/issues/2211>).
img.clone().into_rgb8().write_with_encoder(encoder)?;
}
}
Ok(())
@@ -1205,6 +1208,28 @@ mod tests {
.unwrap();
}
/// Tests that RGBA PNG can be recoded into JPEG
/// by dropping alpha channel.
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_recode_image_rgba_png_to_jpeg() {
let bytes = include_bytes!("../test-data/image/screenshot-rgba.png");
send_image_check_mediaquality(
Viewtype::Image,
Some("1"),
bytes,
"png",
false, // no Exif
1920,
1080,
0,
constants::WORSE_IMAGE_SIZE,
constants::WORSE_IMAGE_SIZE * 1080 / 1920,
)
.await
.unwrap();
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_recode_image_huge_jpg() {
let bytes = include_bytes!("../test-data/image/screenshot.jpg");
@@ -1282,26 +1307,65 @@ mod tests {
let alice_msg = alice.get_last_msg().await;
assert_eq!(alice_msg.get_width() as u32, compressed_width);
assert_eq!(alice_msg.get_height() as u32, compressed_height);
check_image_size(
alice_msg.get_file(&alice).unwrap(),
compressed_width,
compressed_height,
);
let file_saved = alice
.get_blobdir()
.join("saved-".to_string() + &alice_msg.get_filename().unwrap());
alice_msg.save_file(&alice, &file_saved).await?;
check_image_size(file_saved, compressed_width, compressed_height);
let bob_msg = bob.recv_msg(&sent).await;
assert_eq!(bob_msg.get_viewtype(), Viewtype::Image);
assert_eq!(bob_msg.get_width() as u32, compressed_width);
assert_eq!(bob_msg.get_height() as u32, compressed_height);
let file = bob_msg.get_file(&bob).unwrap();
let file_saved = bob
.get_blobdir()
.join("saved-".to_string() + &bob_msg.get_filename().unwrap());
bob_msg.save_file(&bob, &file_saved).await?;
let blob = BlobObject::new_from_path(&bob, &file).await?;
let blob = BlobObject::new_from_path(&bob, &file_saved).await?;
let (_, exif) = blob.metadata()?;
assert!(exif.is_none());
let img = check_image_size(file, compressed_width, compressed_height);
let img = check_image_size(file_saved, compressed_width, compressed_height);
Ok(img)
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_send_big_gif_as_image() -> Result<()> {
let bytes = include_bytes!("../test-data/image/screenshot.gif");
let (width, height) = (1920u32, 1080u32);
let alice = TestContext::new_alice().await;
let bob = TestContext::new_bob().await;
alice
.set_config(
Config::MediaQuality,
Some(&(MediaQuality::Worse as i32).to_string()),
)
.await?;
let file = alice.get_blobdir().join("file").with_extension("gif");
fs::write(&file, &bytes)
.await
.context("failed to write file")?;
let mut msg = Message::new(Viewtype::Image);
msg.set_file(file.to_str().unwrap(), None);
let chat = alice.create_chat(&bob).await;
let sent = alice.send_msg(chat.id, &mut msg).await;
let bob_msg = bob.recv_msg(&sent).await;
// DC must detect the image as GIF and send it w/o reencoding.
assert_eq!(bob_msg.get_viewtype(), Viewtype::Gif);
assert_eq!(bob_msg.get_width() as u32, width);
assert_eq!(bob_msg.get_height() as u32, height);
let file_saved = bob
.get_blobdir()
.join("saved-".to_string() + &bob_msg.get_filename().unwrap());
bob_msg.save_file(&bob, &file_saved).await?;
let blob = BlobObject::new_from_path(&bob, &file_saved).await?;
let (file_size, _) = blob.metadata()?;
assert_eq!(file_size, bytes.len() as u64);
check_image_size(file_saved, width, height);
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_increation_in_blobdir() -> Result<()> {
let t = TestContext::new_alice().await;

View File

@@ -2505,6 +2505,30 @@ async fn prepare_msg_blob(context: &Context, msg: &mut Message) -> Result<()> {
.await?
.with_context(|| format!("attachment missing for message of type #{}", msg.viewtype))?;
if msg.viewtype == Viewtype::File || msg.viewtype == Viewtype::Image {
// Correct the type, take care not to correct already very special
// formats as GIF or VOICE.
//
// Typical conversions:
// - from FILE to AUDIO/VIDEO/IMAGE
// - from FILE/IMAGE to GIF */
if let Some((better_type, _)) = message::guess_msgtype_from_suffix(&blob.to_abs_path())
{
if better_type != Viewtype::Webxdc
|| context
.ensure_sendable_webxdc_file(&blob.to_abs_path())
.await
.is_ok()
{
msg.viewtype = better_type;
}
}
} else if msg.viewtype == Viewtype::Webxdc {
context
.ensure_sendable_webxdc_file(&blob.to_abs_path())
.await?;
}
let mut maybe_sticker = msg.viewtype == Viewtype::Sticker;
if msg.viewtype == Viewtype::Image
|| maybe_sticker && !msg.param.exists(Param::ForceSticker)
@@ -2526,34 +2550,6 @@ async fn prepare_msg_blob(context: &Context, msg: &mut Message) -> Result<()> {
.set(Param::Filename, stem.to_string() + "." + blob_ext);
}
if msg.viewtype == Viewtype::File || msg.viewtype == Viewtype::Image {
// Correct the type, take care not to correct already very special
// formats as GIF or VOICE.
//
// Typical conversions:
// - from FILE to AUDIO/VIDEO/IMAGE
// - from FILE/IMAGE to GIF */
if let Some((better_type, better_mime)) =
message::guess_msgtype_from_suffix(&blob.to_abs_path())
{
if better_type != Viewtype::Webxdc
|| context
.ensure_sendable_webxdc_file(&blob.to_abs_path())
.await
.is_ok()
{
msg.viewtype = better_type;
if !msg.param.exists(Param::MimeType) {
msg.param.set(Param::MimeType, better_mime);
}
}
}
} else if msg.viewtype == Viewtype::Webxdc {
context
.ensure_sendable_webxdc_file(&blob.to_abs_path())
.await?;
}
if !msg.param.exists(Param::MimeType) {
if let Some((_, mime)) = message::guess_msgtype_from_suffix(&blob.to_abs_path()) {
msg.param.set(Param::MimeType, mime);
@@ -2597,6 +2593,18 @@ async fn prepare_msg_common(
}
}
// Check a quote reply is not leaking data from other chats.
// This is meant as a last line of defence, the UI should check that before as well.
// (We allow Chattype::Single in general for "Reply Privately";
// checking for exact contact_id will produce false positives when ppl just left the group)
if chat.typ != Chattype::Single && !context.get_config_bool(Config::Bot).await? {
if let Some(quoted_message) = msg.quoted_message(context).await? {
if quoted_message.chat_id != chat_id {
bail!("Bad quote reply");
}
}
}
// check current MessageState for drafts (to keep msg_id) ...
let update_msg_id = if msg.state == MessageState::OutDraft {
msg.hidden = false;
@@ -2839,17 +2847,10 @@ pub(crate) async fn create_send_msg_jobs(context: &Context, msg: &mut Message) -
.await?;
}
if let Some(last_added_location_id) = rendered_msg.last_added_location_id {
if rendered_msg.last_added_location_id.is_some() {
if let Err(err) = location::set_kml_sent_timestamp(context, msg.chat_id, now).await {
error!(context, "Failed to set kml sent_timestamp: {err:#}.");
}
if !msg.hidden {
if let Err(err) =
location::set_msg_location_id(context, msg.id, last_added_location_id).await
{
error!(context, "Failed to set msg_location_id: {err:#}.");
}
}
}
if let Some(sync_ids) = rendered_msg.sync_ids_to_delete {
@@ -4724,6 +4725,59 @@ mod tests {
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_quote_replies() -> Result<()> {
let alice = TestContext::new_alice().await;
let bob = TestContext::new_bob().await;
let grp_chat_id = create_group_chat(&alice, ProtectionStatus::Unprotected, "grp").await?;
let grp_msg_id = send_text_msg(&alice, grp_chat_id, "bar".to_string()).await?;
let grp_msg = Message::load_from_db(&alice, grp_msg_id).await?;
let one2one_chat_id = alice.create_chat(&bob).await.id;
let one2one_msg_id = send_text_msg(&alice, one2one_chat_id, "foo".to_string()).await?;
let one2one_msg = Message::load_from_db(&alice, one2one_msg_id).await?;
// quoting messages in same chat is okay
let mut msg = Message::new(Viewtype::Text);
msg.set_text("baz".to_string());
msg.set_quote(&alice, Some(&grp_msg)).await?;
let result = send_msg(&alice, grp_chat_id, &mut msg).await;
assert!(result.is_ok());
let mut msg = Message::new(Viewtype::Text);
msg.set_text("baz".to_string());
msg.set_quote(&alice, Some(&one2one_msg)).await?;
let result = send_msg(&alice, one2one_chat_id, &mut msg).await;
assert!(result.is_ok());
let one2one_quote_reply_msg_id = result.unwrap();
// quoting messages from groups to one-to-ones is okay ("reply privately")
let mut msg = Message::new(Viewtype::Text);
msg.set_text("baz".to_string());
msg.set_quote(&alice, Some(&grp_msg)).await?;
let result = send_msg(&alice, one2one_chat_id, &mut msg).await;
assert!(result.is_ok());
// quoting messages from one-to-one chats in groups is an error; usually this is also not allowed by UI at all ...
let mut msg = Message::new(Viewtype::Text);
msg.set_text("baz".to_string());
msg.set_quote(&alice, Some(&one2one_msg)).await?;
let result = send_msg(&alice, grp_chat_id, &mut msg).await;
assert!(result.is_err());
// ... but forwarding messages with quotes is allowed
let result = forward_msgs(&alice, &[one2one_quote_reply_msg_id], grp_chat_id).await;
assert!(result.is_ok());
// ... and bots are not restricted
alice.set_config(Config::Bot, Some("1")).await?;
let result = send_msg(&alice, grp_chat_id, &mut msg).await;
assert!(result.is_ok());
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_add_contact_to_chat_ex_add_self() {
// Adding self to a contact should succeed, even though it's pointless.
@@ -5038,6 +5092,32 @@ mod tests {
Ok(())
}
/// Tests that if member added message is completely lost,
/// member is eventually added.
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_lost_member_added() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
let bob = &tcm.bob().await;
let alice_chat_id = alice
.create_group_with_members(ProtectionStatus::Unprotected, "Group", &[bob])
.await;
let alice_sent = alice.send_text(alice_chat_id, "Hi!").await;
let bob_chat_id = bob.recv_msg(&alice_sent).await.chat_id;
assert_eq!(get_chat_contacts(bob, bob_chat_id).await?.len(), 2);
// Attempt to add member, but message is lost.
let claire_id = Contact::create(alice, "", "claire@foo.de").await?;
add_contact_to_chat(alice, alice_chat_id, claire_id).await?;
alice.pop_sent_msg().await;
let alice_sent = alice.send_text(alice_chat_id, "Hi again!").await;
bob.recv_msg(&alice_sent).await;
assert_eq!(get_chat_contacts(bob, bob_chat_id).await?.len(), 3);
Ok(())
}
/// Test that group updates are robust to lost messages and eventual out of order arrival.
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_modify_chat_lost() -> Result<()> {

View File

@@ -2708,7 +2708,19 @@ Hi."#;
bob.recv_msg(&sent_msg).await;
let contact = Contact::get_by_id(&bob, *contacts.first().unwrap()).await?;
assert!(contact.was_seen_recently());
let green = ansi_term::Color::Green.normal();
assert!(
contact.was_seen_recently(),
"{}",
green.paint(
"\nNOTE: This test failure is probably a false-positive, caused by tests running in parallel.
The issue is that `SystemTime::shift()` (a utility function for tests) changes the time for all threads doing tests, and not only for the running test.
Until the false-positive is fixed:
- Use `cargo test -- --test-threads 1` instead of `cargo test`
- Or use `cargo nextest run` (install with `cargo install cargo-nextest --locked`)\n"
)
);
let self_contact = Contact::get_by_id(&bob, ContactId::SELF).await?;
assert!(!self_contact.was_seen_recently());

View File

@@ -57,11 +57,7 @@ pub(crate) async fn prepare_decryption(
autocrypt_header: None,
peerstate: None,
message_time,
dkim_results: DkimResults {
dkim_passed: false,
dkim_should_work: false,
allow_keychange: true,
},
dkim_results: DkimResults { dkim_passed: false },
});
}
@@ -86,15 +82,13 @@ pub(crate) async fn prepare_decryption(
None
};
let dkim_results = handle_authres(context, mail, from, message_time).await?;
let dkim_results = handle_authres(context, mail, from).await?;
let allow_aeap = get_encrypted_mime(mail).is_some();
let peerstate = get_autocrypt_peerstate(
context,
from,
autocrypt_header.as_ref(),
message_time,
// Disallowing keychanges is disabled for now:
true, // dkim_results.allow_keychange,
allow_aeap,
)
.await?;
@@ -287,19 +281,15 @@ pub(crate) fn keyring_from_peerstate(peerstate: Option<&Peerstate>) -> Vec<Signe
/// If we already know this fingerprint from another contact's peerstate, return that
/// peerstate in order to make AEAP work, but don't save it into the db yet.
///
/// The param `allow_change` is used to prevent the autocrypt key from being changed
/// if we suspect that the message may be forged and have a spoofed sender identity.
///
/// Returns updated peerstate.
pub(crate) async fn get_autocrypt_peerstate(
context: &Context,
from: &str,
autocrypt_header: Option<&Aheader>,
message_time: i64,
allow_change: bool,
allow_aeap: bool,
) -> Result<Option<Peerstate>> {
let allow_change = allow_change && !context.is_self_addr(from).await?;
let allow_change = !context.is_self_addr(from).await?;
let mut peerstate;
// Apply Autocrypt header

View File

@@ -80,6 +80,7 @@ use crate::contact::ContactId;
use crate::context::Context;
use crate::download::MIN_DELETE_SERVER_AFTER;
use crate::events::EventType;
use crate::location;
use crate::log::LogExt;
use crate::message::{Message, MessageState, MsgId, Viewtype};
use crate::mimeparser::SystemMessage;
@@ -348,16 +349,16 @@ pub(crate) async fn start_ephemeral_timers_msgids(
/// Selects messages which are expired according to
/// `delete_device_after` setting or `ephemeral_timestamp` column.
///
/// For each message a row ID, chat id and viewtype is returned.
/// For each message a row ID, chat id, viewtype and location ID is returned.
async fn select_expired_messages(
context: &Context,
now: i64,
) -> Result<Vec<(MsgId, ChatId, Viewtype)>> {
) -> Result<Vec<(MsgId, ChatId, Viewtype, u32)>> {
let mut rows = context
.sql
.query_map(
r#"
SELECT id, chat_id, type
SELECT id, chat_id, type, location_id
FROM msgs
WHERE
ephemeral_timestamp != 0
@@ -369,7 +370,8 @@ WHERE
let id: MsgId = row.get("id")?;
let chat_id: ChatId = row.get("chat_id")?;
let viewtype: Viewtype = row.get("type")?;
Ok((id, chat_id, viewtype))
let location_id: u32 = row.get("location_id")?;
Ok((id, chat_id, viewtype, location_id))
},
|rows| rows.collect::<Result<Vec<_>, _>>().map_err(Into::into),
)
@@ -389,7 +391,7 @@ WHERE
.sql
.query_map(
r#"
SELECT id, chat_id, type
SELECT id, chat_id, type, location_id
FROM msgs
WHERE
timestamp < ?1
@@ -408,7 +410,8 @@ WHERE
let id: MsgId = row.get("id")?;
let chat_id: ChatId = row.get("chat_id")?;
let viewtype: Viewtype = row.get("type")?;
Ok((id, chat_id, viewtype))
let location_id: u32 = row.get("location_id")?;
Ok((id, chat_id, viewtype, location_id))
},
|rows| rows.collect::<Result<Vec<_>, _>>().map_err(Into::into),
)
@@ -439,7 +442,7 @@ pub(crate) async fn delete_expired_messages(context: &Context, now: i64) -> Resu
// If you change which information is removed here, also change MsgId::trash() and
// which information receive_imf::add_parts() still adds to the db if the chat_id is TRASH
for (msg_id, chat_id, viewtype) in rows {
for (msg_id, chat_id, viewtype, location_id) in rows {
transaction.execute(
"UPDATE msgs
SET chat_id=?, txt='', subject='', txt_raw='',
@@ -448,6 +451,13 @@ pub(crate) async fn delete_expired_messages(context: &Context, now: i64) -> Resu
(DC_CHAT_ID_TRASH, msg_id),
)?;
if location_id > 0 {
transaction.execute(
"DELETE FROM locations WHERE independent=1 AND id=?",
(location_id,),
)?;
}
msgs_changed.push((chat_id, msg_id));
if viewtype == Viewtype::Webxdc {
webxdc_deleted.push(msg_id)
@@ -592,6 +602,11 @@ pub(crate) async fn ephemeral_loop(context: &Context, interrupt_receiver: Receiv
.await
.log_err(context)
.ok();
location::delete_expired(context, time())
.await
.log_err(context)
.ok();
}
}
@@ -671,8 +686,10 @@ mod tests {
use super::*;
use crate::config::Config;
use crate::download::DownloadState;
use crate::location;
use crate::message::markseen_msgs;
use crate::receive_imf::receive_imf;
use crate::test_utils::TestContext;
use crate::test_utils::{TestContext, TestContextManager};
use crate::timesmearing::MAX_SECONDS_TO_LEND_FROM_FUTURE;
use crate::{
chat::{self, create_group_chat, send_text_msg, Chat, ChatItem, ProtectionStatus},
@@ -1349,4 +1366,44 @@ mod tests {
Ok(())
}
/// Tests that POI location is deleted when ephemeral message expires.
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_ephemeral_poi_location() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
let bob = &tcm.bob().await;
let chat = alice.create_chat(bob).await;
let duration = 60;
chat.id
.set_ephemeral_timer(alice, Timer::Enabled { duration })
.await?;
let sent = alice.pop_sent_msg().await;
bob.recv_msg(&sent).await;
let mut poi_msg = Message::new(Viewtype::Text);
poi_msg.text = "Here".to_string();
poi_msg.set_location(10.0, 20.0);
let alice_sent_message = alice.send_msg(chat.id, &mut poi_msg).await;
let bob_received_message = bob.recv_msg(&alice_sent_message).await;
markseen_msgs(bob, vec![bob_received_message.id]).await?;
for account in [alice, bob] {
let locations = location::get_range(account, None, None, 0, 0).await?;
assert_eq!(locations.len(), 1);
}
SystemTime::shift(Duration::from_secs(100));
for account in [alice, bob] {
delete_expired_messages(account, time()).await?;
let locations = location::get_range(account, None, None, 0, 0).await?;
assert_eq!(locations.len(), 0);
}
Ok(())
}
}

View File

@@ -5,11 +5,12 @@
use std::{
cmp::max,
cmp::min,
collections::{BTreeMap, BTreeSet, HashMap},
iter::Peekable,
mem::take,
sync::atomic::Ordering,
time::Duration,
time::{Duration, UNIX_EPOCH},
};
use anyhow::{bail, format_err, Context as _, Result};
@@ -19,8 +20,8 @@ use deltachat_contact_tools::{normalize_name, ContactAddress};
use futures::{FutureExt as _, StreamExt, TryStreamExt};
use futures_lite::FutureExt;
use num_traits::FromPrimitive;
use rand::Rng;
use ratelimit::Ratelimit;
use tokio::sync::RwLock;
use crate::chat::{self, ChatId, ChatIdBlocked};
use crate::chatlist_events;
@@ -42,7 +43,7 @@ use crate::scheduler::connectivity::ConnectivityStore;
use crate::socks::Socks5Config;
use crate::sql;
use crate::stock_str;
use crate::tools::{create_id, duration_to_str};
use crate::tools::{self, create_id, duration_to_str};
pub(crate) mod capabilities;
mod client;
@@ -82,15 +83,17 @@ pub(crate) struct Imap {
pub(crate) connectivity: ConnectivityStore,
/// Rate limit for IMAP connection attempts.
conn_last_try: tools::Time,
conn_backoff_ms: u64,
/// Rate limit for successful IMAP connections.
///
/// This rate limit prevents busy loop
/// in case the server refuses connections
/// This rate limit prevents busy loop in case the server refuses logins
/// or in case connection gets dropped over and over due to IMAP bug,
/// e.g. the server returning invalid response to SELECT command
/// immediately after logging in or returning an error in response to LOGIN command
/// due to internal server error.
ratelimit: RwLock<Ratelimit>,
ratelimit: Ratelimit,
}
#[derive(Debug)]
@@ -248,8 +251,10 @@ impl Imap {
strict_tls,
login_failed_once: false,
connectivity: Default::default(),
conn_last_try: UNIX_EPOCH,
conn_backoff_ms: 0,
// 1 connection per minute + a burst of 2.
ratelimit: RwLock::new(Ratelimit::new(Duration::new(120, 0), 2.0)),
ratelimit: Ratelimit::new(Duration::new(120, 0), 2.0),
};
Ok(imap)
@@ -293,7 +298,15 @@ impl Imap {
bail!("IMAP operation attempted while it is torn down");
}
let ratelimit_duration = self.ratelimit.read().await.until_can_send();
let now = tools::Time::now();
let until_can_send = max(
min(self.conn_last_try, now)
.checked_add(Duration::from_millis(self.conn_backoff_ms))
.unwrap_or(now),
now,
)
.duration_since(now)?;
let ratelimit_duration = max(until_can_send, self.ratelimit.until_can_send());
if !ratelimit_duration.is_zero() {
warn!(
context,
@@ -316,7 +329,16 @@ impl Imap {
info!(context, "Connecting to IMAP server");
self.connectivity.set_connecting(context).await;
self.ratelimit.write().await.send();
self.conn_last_try = tools::Time::now();
const BACKOFF_MIN_MS: u64 = 2000;
const BACKOFF_MAX_MS: u64 = 80_000;
self.conn_backoff_ms = min(self.conn_backoff_ms, BACKOFF_MAX_MS / 2);
self.conn_backoff_ms = self.conn_backoff_ms.saturating_add(
rand::thread_rng().gen_range((self.conn_backoff_ms / 2)..=self.conn_backoff_ms),
);
self.conn_backoff_ms = max(BACKOFF_MIN_MS, self.conn_backoff_ms);
let connection_res: Result<Client> =
if self.lp.security == Socket::Starttls || self.lp.security == Socket::Plain {
let imap_server: &str = self.lp.server.as_ref();
@@ -364,6 +386,8 @@ impl Imap {
}
};
let client = connection_res?;
self.conn_backoff_ms = BACKOFF_MIN_MS;
self.ratelimit.send();
let imap_user: &str = self.lp.user.as_ref();
let imap_pw: &str = self.lp.password.as_ref();

View File

@@ -193,7 +193,9 @@ pub async fn render_setup_file(context: &Context, passphrase: &str) -> Result<St
true => Some(("Autocrypt-Prefer-Encrypt", "mutual")),
};
let private_key_asc = private_key.to_asc(ac_headers);
let encr = pgp::symm_encrypt(passphrase, private_key_asc.as_bytes()).await?;
let encr = pgp::symm_encrypt(passphrase, private_key_asc.as_bytes())
.await?
.replace('\n', "\r\n");
let replacement = format!(
concat!(
@@ -284,7 +286,7 @@ pub async fn continue_key_transfer(
let file = open_file_std(context, filename)?;
let sc = normalize_setup_code(setup_code);
let armored_key = decrypt_setup_file(&sc, file).await?;
set_self_key(context, &armored_key, true, true).await?;
set_self_key(context, &armored_key, true).await?;
maybe_add_bcc_self_device_msg(context).await?;
Ok(())
@@ -293,35 +295,32 @@ pub async fn continue_key_transfer(
}
}
async fn set_self_key(
context: &Context,
armored: &str,
set_default: bool,
prefer_encrypt_required: bool,
) -> Result<()> {
async fn set_self_key(context: &Context, armored: &str, set_default: bool) -> Result<()> {
// try hard to only modify key-state
let (private_key, header) = SignedSecretKey::from_asc(armored)?;
let public_key = private_key.split_public_key()?;
let preferencrypt = header.get("Autocrypt-Prefer-Encrypt");
match preferencrypt.map(|s| s.as_str()) {
Some(headerval) => {
let e2ee_enabled = match headerval {
"nopreference" => 0,
"mutual" => 1,
_ => {
bail!("invalid Autocrypt-Prefer-Encrypt header: {:?}", header);
}
};
context
.sql
.set_raw_config_int("e2ee_enabled", e2ee_enabled)
.await?;
}
None => {
if prefer_encrypt_required {
bail!("missing Autocrypt-Prefer-Encrypt header");
if let Some(preferencrypt) = header.get("Autocrypt-Prefer-Encrypt") {
let e2ee_enabled = match preferencrypt.as_str() {
"nopreference" => 0,
"mutual" => 1,
_ => {
bail!("invalid Autocrypt-Prefer-Encrypt header: {:?}", header);
}
}
};
context
.sql
.set_raw_config_int("e2ee_enabled", e2ee_enabled)
.await?;
} else {
// `Autocrypt-Prefer-Encrypt` is not included
// in keys exported to file.
//
// `Autocrypt-Prefer-Encrypt` also SHOULD be sent
// in Autocrypt Setup Message according to Autocrypt specification,
// but K-9 6.802 does not include this header.
//
// We keep current setting in this case.
info!(context, "No Autocrypt-Prefer-Encrypt header.");
};
let self_addr = context.get_primary_self_addr().await?;
@@ -604,7 +603,7 @@ async fn export_backup_inner(
async fn import_secret_key(context: &Context, path: &Path, set_default: bool) -> Result<()> {
let buf = read_file(context, &path).await?;
let armored = std::string::String::from_utf8_lossy(&buf);
set_self_key(context, &armored, set_default, false).await?;
set_self_key(context, &armored, set_default).await?;
Ok(())
}
@@ -825,6 +824,7 @@ mod tests {
use super::*;
use crate::pgp::{split_armored_data, HEADER_AUTOCRYPT, HEADER_SETUPCODE};
use crate::receive_imf::receive_imf;
use crate::stock_str::StockMessage;
use crate::test_utils::{alice_keypair, TestContext, TestContextManager};
@@ -834,15 +834,17 @@ mod tests {
let msg = render_setup_file(&t, "hello").await.unwrap();
println!("{}", &msg);
// Check some substrings, indicating things got substituted.
// In particular note the mixing of `\r\n` and `\n` depending
// on who generated the strings.
assert!(msg.contains("<title>Autocrypt Setup Message</title"));
assert!(msg.contains("<h1>Autocrypt Setup Message</h1>"));
assert!(msg.contains("<p>This is the Autocrypt Setup Message used to"));
assert!(msg.contains("-----BEGIN PGP MESSAGE-----\r\n"));
assert!(msg.contains("Passphrase-Format: numeric9x4\r\n"));
assert!(msg.contains("Passphrase-Begin: he\n"));
assert!(msg.contains("-----END PGP MESSAGE-----\n"));
assert!(msg.contains("Passphrase-Begin: he\r\n"));
assert!(msg.contains("-----END PGP MESSAGE-----\r\n"));
for line in msg.rsplit_terminator('\n') {
assert!(line.ends_with('\r'));
}
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
@@ -1193,4 +1195,22 @@ mod tests {
Ok(())
}
/// Tests reception of Autocrypt Setup Message from K-9 6.802.
///
/// Unlike Autocrypt Setup Message sent by Delta Chat,
/// this message does not contain `Autocrypt-Prefer-Encrypt` header.
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_key_transfer_k_9() -> Result<()> {
let t = &TestContext::new().await;
t.configure_addr("autocrypt@nine.testrun.org").await;
let raw = include_bytes!("../test-data/message/k-9-autocrypt-setup-message.eml");
let received = receive_imf(t, raw, false).await?.unwrap();
let setup_code = "0655-9868-8252-5455-4232-5158-1237-5333-2638";
continue_key_transfer(t, *received.msg_ids.last().unwrap(), setup_code).await?;
Ok(())
}
}

View File

@@ -656,6 +656,12 @@ mod tests {
let text = fs::read_to_string(&path).await.unwrap();
assert_eq!(text, "i am attachment");
let path = path.with_file_name("saved.txt");
msg.save_file(&ctx1, &path).await.unwrap();
let text = fs::read_to_string(&path).await.unwrap();
assert_eq!(text, "i am attachment");
assert!(msg.save_file(&ctx1, &path).await.is_err());
// Check that both received the ImexProgress events.
ctx0.evtracker
.get_matching(|ev| matches!(ev, EventType::ImexProgress(1000)))

View File

@@ -1,4 +1,14 @@
//! Location handling.
//!
//! Delta Chat handles two kind of locations.
//!
//! There are two kinds of locations:
//! - Independent locations, also known as Points of Interest (POI).
//! - Path locations.
//!
//! Locations are sent as KML attachments.
//! Independent locations are sent in `message.kml` attachments
//! and path locations are sent in `location.kml` attachments.
use std::time::Duration;
@@ -8,6 +18,7 @@ use quick_xml::events::{BytesEnd, BytesStart, BytesText};
use tokio::time::timeout;
use crate::chat::{self, ChatId};
use crate::constants::DC_CHAT_ID_TRASH;
use crate::contact::ContactId;
use crate::context::Context;
use crate::events::EventType;
@@ -350,6 +361,7 @@ pub async fn set(context: &Context, latitude: f64, longitude: f64, accuracy: f64
)
.await?;
let mut stored_location = false;
for chat_id in chats {
context.sql.execute(
"INSERT INTO locations \
@@ -362,6 +374,7 @@ pub async fn set(context: &Context, latitude: f64, longitude: f64, accuracy: f64
chat_id,
ContactId::SELF,
)).await.context("Failed to store location")?;
stored_location = true;
info!(context, "Stored location for chat {chat_id}.");
continue_streaming = true;
@@ -369,6 +382,10 @@ pub async fn set(context: &Context, latitude: f64, longitude: f64, accuracy: f64
if continue_streaming {
context.emit_location_changed(Some(ContactId::SELF)).await?;
};
if stored_location {
// Interrupt location loop so it may send a location-only message.
context.scheduler.interrupt_location().await;
}
Ok(continue_streaming)
}
@@ -461,6 +478,58 @@ pub async fn delete_all(context: &Context) -> Result<()> {
Ok(())
}
/// Deletes expired locations.
///
/// Only path locations are deleted.
/// POIs should be deleted when corresponding message is deleted.
pub(crate) async fn delete_expired(context: &Context, now: i64) -> Result<()> {
let Some(delete_device_after) = context.get_config_delete_device_after().await? else {
return Ok(());
};
let threshold_timestamp = now.saturating_sub(delete_device_after);
let deleted = context
.sql
.execute(
"DELETE FROM locations WHERE independent=0 AND timestamp < ?",
(threshold_timestamp,),
)
.await?
> 0;
if deleted {
info!(context, "Deleted {deleted} expired locations.");
context.emit_location_changed(None).await?;
}
Ok(())
}
/// Deletes location if it is an independent location.
///
/// This function is used when a message is deleted
/// that has a corresponding `location_id`.
pub(crate) async fn delete_poi_location(context: &Context, location_id: u32) -> Result<()> {
context
.sql
.execute(
"DELETE FROM locations WHERE independent = 1 AND id=?",
(location_id as i32,),
)
.await?;
Ok(())
}
/// Deletes POI locations that don't have corresponding message anymore.
pub(crate) async fn delete_orphaned_poi_locations(context: &Context) -> Result<()> {
context.sql.execute("
DELETE FROM locations
WHERE independent=1 AND id NOT IN
(SELECT location_id from MSGS LEFT JOIN locations
ON locations.id=location_id
WHERE location_id>0 -- This check makes the query faster by not looking for locations with ID 0 that don't exist.
AND msgs.chat_id != ?)", (DC_CHAT_ID_TRASH,)).await?;
Ok(())
}
/// Returns `location.kml` contents.
pub async fn get_kml(context: &Context, chat_id: ChatId) -> Result<Option<(String, u32)>> {
let mut last_added_location_id = 0;
@@ -815,8 +884,10 @@ mod tests {
#![allow(clippy::indexing_slicing)]
use super::*;
use crate::config::Config;
use crate::receive_imf::receive_imf;
use crate::test_utils::TestContext;
use crate::test_utils::{TestContext, TestContextManager};
use crate::tools::SystemTime;
#[test]
fn test_kml_parse() {
@@ -966,6 +1037,8 @@ Content-Disposition: attachment; filename="location.kml"
let mut msg = Message::new(Viewtype::Image);
msg.set_file(file.to_str().unwrap(), None);
let sent = alice.send_msg(alice_chat.id, &mut msg).await;
let alice_msg = Message::load_from_db(&alice, sent.sender_msg_id).await?;
assert_eq!(alice_msg.has_location(), false);
let msg = bob.recv_msg_opt(&sent).await.unwrap();
assert!(msg.chat_id == bob_chat_id);
@@ -974,6 +1047,60 @@ Content-Disposition: attachment; filename="location.kml"
let bob_msg = Message::load_from_db(&bob, *msg.msg_ids.first().unwrap()).await?;
assert_eq!(bob_msg.chat_id, bob_chat_id);
assert_eq!(bob_msg.viewtype, Viewtype::Image);
assert_eq!(bob_msg.has_location(), false);
let bob_locations = get_range(&bob, None, None, 0, 0).await?;
assert_eq!(bob_locations.len(), 1);
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_delete_expired_locations() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
let bob = &tcm.bob().await;
// Alice enables deletion of messages from device after 1 week.
alice
.set_config(Config::DeleteDeviceAfter, Some("604800"))
.await?;
// Bob enables deletion of messages from device after 1 day.
bob.set_config(Config::DeleteDeviceAfter, Some("86400"))
.await?;
let alice_chat = alice.create_chat(bob).await;
// Alice enables location streaming.
// Bob receives a message saying that Alice enabled location streaming.
send_locations_to_chat(alice, alice_chat.id, 60).await?;
bob.recv_msg(&alice.pop_sent_msg().await).await;
// Alice gets new location from GPS.
assert_eq!(set(alice, 10.0, 20.0, 1.0).await?, true);
assert_eq!(get_range(alice, None, None, 0, 0).await?.len(), 1);
// 10 seconds later location sending stream manages to send location.
SystemTime::shift(Duration::from_secs(10));
delete_expired(alice, time()).await?;
maybe_send_locations(alice).await?;
bob.recv_msg_opt(&alice.pop_sent_msg().await).await;
assert_eq!(get_range(alice, None, None, 0, 0).await?.len(), 1);
assert_eq!(get_range(bob, None, None, 0, 0).await?.len(), 1);
// Day later Bob removes location.
SystemTime::shift(Duration::from_secs(86400));
delete_expired(alice, time()).await?;
delete_expired(bob, time()).await?;
assert_eq!(get_range(alice, None, None, 0, 0).await?.len(), 1);
assert_eq!(get_range(bob, None, None, 0, 0).await?.len(), 0);
// Week late Alice removes location.
SystemTime::shift(Duration::from_secs(604800));
delete_expired(alice, time()).await?;
delete_expired(bob, time()).await?;
assert_eq!(get_range(alice, None, None, 0, 0).await?.len(), 0);
assert_eq!(get_range(bob, None, None, 0, 0).await?.len(), 0);
Ok(())
}

View File

@@ -6,6 +6,7 @@ use std::path::{Path, PathBuf};
use anyhow::{ensure, format_err, Context as _, Result};
use deltachat_derive::{FromSql, ToSql};
use serde::{Deserialize, Serialize};
use tokio::{fs, io};
use crate::blob::BlobObject;
use crate::chat::{Chat, ChatId};
@@ -21,6 +22,7 @@ use crate::download::DownloadState;
use crate::ephemeral::{start_ephemeral_timers_msgids, Timer as EphemeralTimer};
use crate::events::EventType;
use crate::imap::markseen_on_imap_table;
use crate::location::delete_poi_location;
use crate::mimeparser::{parse_message_id, SystemMessage};
use crate::param::{Param, Params};
use crate::pgp::split_armored_data;
@@ -605,6 +607,19 @@ impl Message {
self.param.get_path(Param::File, context).unwrap_or(None)
}
/// Save file copy at the user-provided path.
pub async fn save_file(&self, context: &Context, path: &Path) -> Result<()> {
let path_src = self.get_file(context).context("No file")?;
let mut src = fs::OpenOptions::new().read(true).open(path_src).await?;
let mut dst = fs::OpenOptions::new()
.write(true)
.create_new(true)
.open(path)
.await?;
io::copy(&mut src, &mut dst).await?;
Ok(())
}
/// If message is an image or gif, set Param::Width and Param::Height
pub(crate) async fn try_calc_and_set_dimensions(&mut self, context: &Context) -> Result<()> {
if self.viewtype.has_file() {
@@ -640,13 +655,11 @@ impl Message {
Ok(())
}
/// Check if a message has a location bound to it.
/// These messages are also returned by get_locations()
/// and the UI may decide to display a special icon beside such messages,
/// Check if a message has a POI location bound to it.
/// These locations are also returned by [`location::get_range()`].
/// The UI may decide to display a special icon beside such messages.
///
/// @memberof Message
/// @param msg The message object.
/// @return 1=Message has location bound to it, 0=No location bound to message.
/// [`location::get_range()`]: crate::location::get_range
pub fn has_location(&self) -> bool {
self.location_id != 0
}
@@ -656,13 +669,17 @@ impl Message {
/// at a position different from the self-location.
/// You should not call this function
/// if you want to bind the current self-location to a message;
/// this is done by set_location() and send_locations_to_chat().
/// this is done by [`location::set()`] and [`send_locations_to_chat()`].
///
/// Typically results in the event #DC_EVENT_LOCATION_CHANGED with
/// contact_id set to ContactId::SELF.
/// Typically results in the event [`LocationChanged`] with
/// `contact_id` set to [`ContactId::SELF`].
///
/// @param latitude North-south position of the location.
/// @param longitude East-west position of the location.
/// `latitude` is the North-south position of the location.
/// `longitutde` is the East-west position of the location.
///
/// [`location::set()`]: crate::location::set
/// [`send_locations_to_chat()`]: crate::location::send_locations_to_chat
/// [`LocationChanged`]: crate::events::EventType::LocationChanged
pub fn set_location(&mut self, latitude: f64, longitude: f64) {
if latitude == 0.0 && longitude == 0.0 {
return;
@@ -1043,6 +1060,7 @@ impl Message {
filemime: Option<&str>,
) -> Result<()> {
let blob = BlobObject::create(context, suggested_name, data).await?;
self.param.set(Param::Filename, suggested_name);
self.param.set(Param::File, blob.as_name());
self.param.set_optional(Param::MimeType, filemime);
Ok(())
@@ -1112,7 +1130,7 @@ impl Message {
.get_bool(Param::GuaranteeE2ee)
.unwrap_or_default()
{
self.param.set(Param::GuaranteeE2ee, "1");
self.param.set(Param::ProtectQuote, "1");
}
let text = quote.get_text();
@@ -1556,17 +1574,6 @@ pub async fn delete_msgs(context: &Context, msg_ids: &[MsgId]) -> Result<()> {
Ok(())
}
async fn delete_poi_location(context: &Context, location_id: u32) -> Result<()> {
context
.sql
.execute(
"DELETE FROM locations WHERE independent = 1 AND id=?;",
(location_id as i32,),
)
.await?;
Ok(())
}
/// Marks requested messages as seen.
pub async fn markseen_msgs(context: &Context, msg_ids: Vec<MsgId>) -> Result<()> {
if msg_ids.is_empty() {
@@ -2000,8 +2007,11 @@ mod tests {
use num_traits::FromPrimitive;
use super::*;
use crate::chat::{self, marknoticed_chat, send_text_msg, ChatItem};
use crate::chat::{
self, add_contact_to_chat, marknoticed_chat, send_text_msg, ChatItem, ProtectionStatus,
};
use crate::chatlist::Chatlist;
use crate::config::Config;
use crate::reaction::send_reaction;
use crate::receive_imf::receive_imf;
use crate::test_utils as test;
@@ -2025,8 +2035,6 @@ mod tests {
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_prepare_message_and_send() {
use crate::config::Config;
let d = test::TestContext::new().await;
let ctx = &d.ctx;
@@ -2165,8 +2173,6 @@ mod tests {
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_quote() {
use crate::config::Config;
let d = test::TestContext::new().await;
let ctx = &d.ctx;
@@ -2199,6 +2205,42 @@ mod tests {
assert_eq!(quoted_msg.get_text(), msg2.quoted_text().unwrap());
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_unencrypted_quote_encrypted_message() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
let bob = &tcm.bob().await;
let alice_group = alice
.create_group_with_members(ProtectionStatus::Unprotected, "Group chat", &[bob])
.await;
let sent = alice.send_text(alice_group, "Hi! I created a group").await;
let bob_received_message = bob.recv_msg(&sent).await;
let bob_group = bob_received_message.chat_id;
bob_group.accept(bob).await?;
let sent = bob.send_text(bob_group, "Encrypted message").await;
let alice_received_message = alice.recv_msg(&sent).await;
assert!(alice_received_message.get_showpadlock());
// Alice adds contact without key so chat becomes unencrypted.
let alice_flubby_contact_id =
Contact::create(alice, "Flubby", "flubby@example.org").await?;
add_contact_to_chat(alice, alice_group, alice_flubby_contact_id).await?;
// Alice quotes encrypted message in unencrypted chat.
let mut msg = Message::new(Viewtype::Text);
msg.set_quote(alice, Some(&alice_received_message)).await?;
chat::send_msg(alice, alice_group, &mut msg).await?;
let bob_received_message = bob.recv_msg(&alice.pop_sent_msg().await).await;
assert_eq!(bob_received_message.quoted_text().unwrap(), "...");
assert_eq!(bob_received_message.get_showpadlock(), false);
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_get_chat_id() {
// Alice receives a message that pops up as a contact request

View File

@@ -669,19 +669,19 @@ impl<'a> MimeFactory<'a> {
let mut is_gossiped = false;
let (main_part, parts) = match self.loaded {
Loaded::Message { .. } => {
self.render_message(context, &mut headers, &grpimage)
.await?
}
Loaded::Mdn { .. } => (self.render_mdn(context).await?, Vec::new()),
};
let peerstates = self.peerstates_for_recipients(context).await?;
let should_encrypt =
encrypt_helper.should_encrypt(context, e2ee_guaranteed, &peerstates)?;
let is_encrypted = should_encrypt && !force_plaintext;
let (main_part, parts) = match self.loaded {
Loaded::Message { .. } => {
self.render_message(context, &mut headers, &grpimage, is_encrypted)
.await?
}
Loaded::Mdn { .. } => (self.render_mdn(context).await?, Vec::new()),
};
let message = if parts.is_empty() {
// Single part, render as regular message.
main_part
@@ -960,6 +960,7 @@ impl<'a> MimeFactory<'a> {
context: &Context,
headers: &mut MessageHeaders,
grpimage: &Option<String>,
is_encrypted: bool,
) -> Result<(PartBuilder, Vec<PartBuilder>)> {
let chat = match &self.loaded {
Loaded::Message { chat } => chat,
@@ -1221,6 +1222,16 @@ impl<'a> MimeFactory<'a> {
.msg
.quoted_text()
.map(|quote| format_flowed_quote(&quote) + "\r\n\r\n");
if !is_encrypted
&& self
.msg
.param
.get_bool(Param::ProtectQuote)
.unwrap_or_default()
{
// Message is not encrypted but quotes encrypted message.
quoted_text = Some("> ...\r\n\r\n".to_string());
}
if quoted_text.is_none() && final_text.starts_with('>') {
// Insert empty line to avoid receiver treating user-sent quote as topquote inserted by
// Delta Chat.

View File

@@ -415,8 +415,6 @@ impl MimeMessage {
if let (Some(peerstate), Ok(mail)) = (&mut decryption_info.peerstate, mail) {
if timestamp_sent > peerstate.last_seen_autocrypt
&& mail.ctype.mimetype != "multipart/report"
// Disallowing keychanges is disabled for now:
// && decryption_info.dkim_results.allow_keychange
{
peerstate.degrade_encryption(timestamp_sent);
}
@@ -506,13 +504,6 @@ impl MimeMessage {
parser.heuristically_parse_ndn(context).await;
parser.parse_headers(context).await?;
// Disallowing keychanges is disabled for now
// if !decryption_info.dkim_results.allow_keychange {
// for part in parser.parts.iter_mut() {
// part.error = Some("Seems like DKIM failed, this either is an attack or (more likely) a bug in Authentication-Results checking. Please tell us about this at https://support.delta.chat.".to_string());
// }
// }
if parser.is_mime_modified {
parser.decoded_data = mail_raw;
}
@@ -662,32 +653,34 @@ impl MimeMessage {
self.squash_attachment_parts();
}
if let Some(ref subject) = self.get_subject() {
let mut prepend_subject = true;
if !self.decrypting_failed {
let colon = subject.find(':');
if colon == Some(2)
|| colon == Some(3)
|| self.has_chat_version()
|| subject.contains("Chat:")
{
prepend_subject = false
if !context.get_config_bool(Config::Bot).await? {
if let Some(ref subject) = self.get_subject() {
let mut prepend_subject = true;
if !self.decrypting_failed {
let colon = subject.find(':');
if colon == Some(2)
|| colon == Some(3)
|| self.has_chat_version()
|| subject.contains("Chat:")
{
prepend_subject = false
}
}
}
// For mailing lists, always add the subject because sometimes there are different topics
// and otherwise it might be hard to keep track:
if self.is_mailinglist_message() && !self.has_chat_version() {
prepend_subject = true;
}
// For mailing lists, always add the subject because sometimes there are different topics
// and otherwise it might be hard to keep track:
if self.is_mailinglist_message() && !self.has_chat_version() {
prepend_subject = true;
}
if prepend_subject && !subject.is_empty() {
let part_with_text = self
.parts
.iter_mut()
.find(|part| !part.msg.is_empty() && !part.is_reaction);
if let Some(part) = part_with_text {
part.msg = format!("{} {}", subject, part.msg);
if prepend_subject && !subject.is_empty() {
let part_with_text = self
.parts
.iter_mut()
.find(|part| !part.msg.is_empty() && !part.is_reaction);
if let Some(part) = part_with_text {
part.msg = format!("{} {}", subject, part.msg);
}
}
}
}
@@ -3933,4 +3926,31 @@ Content-Disposition: reaction\n\
Ok(())
}
/// Tests that subject is not prepended to the message
/// when bot receives it.
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_bot_no_subject() {
let context = TestContext::new().await;
context.set_config(Config::Bot, Some("1")).await.unwrap();
let raw = br#"Message-ID: <foobar@example.org>
From: foo <foo@example.org>
Subject: Some subject
To: bar@example.org
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
/help
"#;
let message = MimeMessage::from_bytes(&context.ctx, &raw[..], None)
.await
.unwrap();
assert_eq!(message.get_subject(), Some("Some subject".to_string()));
assert_eq!(message.parts.len(), 1);
assert_eq!(message.parts[0].typ, Viewtype::Text);
// Not "Some subject /help"
assert_eq!(message.parts[0].msg, "/help");
}
}

View File

@@ -48,6 +48,11 @@ pub enum Param {
/// For Messages: message is encrypted, outgoing: guarantee E2EE or the message is not send
GuaranteeE2ee = b'c',
/// For Messages: quoted message is encrypted.
///
/// If this message is sent unencrypted, quote text should be replaced.
ProtectQuote = b'0',
/// For Messages: decrypted with validation errors or without mutual set, if neither
/// 'c' nor 'e' are preset, the messages is only transport encrypted.
ErroneousE2ee = b'e',

View File

@@ -1669,11 +1669,10 @@ async fn save_locations(
if let Some(addr) = &location_kml.addr {
let contact = Contact::get_by_id(context, from_id).await?;
if contact.get_addr().to_lowercase() == addr.to_lowercase() {
if let Some(newest_location_id) =
location::save(context, chat_id, from_id, &location_kml.locations, false)
.await?
if location::save(context, chat_id, from_id, &location_kml.locations, false)
.await?
.is_some()
{
location::set_msg_location_id(context, msg_id, newest_location_id).await?;
send_event = true;
}
} else {

View File

@@ -2994,11 +2994,15 @@ async fn test_long_and_duplicated_filenames() -> Result<()> {
let resulting_filename = msg.get_filename().unwrap();
assert_eq!(resulting_filename, filename);
let path = msg.get_file(t).unwrap();
let path2 = path.with_file_name("saved.txt");
msg.save_file(t, &path2).await.unwrap();
assert!(
path.to_str().unwrap().ends_with(".tar.gz"),
"path {path:?} doesn't end with .tar.gz"
);
assert_eq!(fs::read_to_string(path).await.unwrap(), content);
assert_eq!(fs::read_to_string(&path).await.unwrap(), content);
assert_eq!(fs::read_to_string(&path2).await.unwrap(), content);
fs::remove_file(path2).await.unwrap();
}
check_message(&msg_alice, &alice, filename_sent, &content).await;
check_message(&msg_bob, &bob, filename_sent, &content).await;

View File

@@ -15,6 +15,7 @@ use crate::context::Context;
use crate::debug_logging::set_debug_logging_xdc;
use crate::ephemeral::start_ephemeral_timers;
use crate::imex::BLOBS_BACKUP_NAME;
use crate::location::delete_orphaned_poi_locations;
use crate::log::LogExt;
use crate::message::{Message, MsgId, Viewtype};
use crate::param::{Param, Params};
@@ -779,6 +780,14 @@ pub async fn housekeeping(context: &Context) -> Result<()> {
.log_err(context)
.ok();
// Delete POI locations
// which don't have corresponding message.
delete_orphaned_poi_locations(context)
.await
.context("Failed to delete orphaned POI locations")
.log_err(context)
.ok();
info!(context, "Housekeeping done.");
Ok(())
}

View File

@@ -613,6 +613,7 @@ CREATE INDEX smtp_messageid ON imap(rfc724_mid);
).await?;
}
if dbversion < 93 {
// `sending_domains` is now unused, but was not removed for backwards compatibility.
sql.execute_migration(
"CREATE TABLE sending_domains(domain TEXT PRIMARY KEY, dkim_works INTEGER DEFAULT 0);",
93,

View File

@@ -683,7 +683,12 @@ pub(crate) fn buf_decompress(buf: &[u8]) -> Result<Vec<u8>> {
mod tests {
#![allow(clippy::indexing_slicing)]
use chrono::NaiveDate;
use proptest::prelude::*;
use super::*;
use crate::chatlist::Chatlist;
use crate::{chat, test_utils};
use crate::{receive_imf::receive_imf, test_utils::TestContext};
#[test]
@@ -730,7 +735,7 @@ Message-ID: 2dfdbde7@example.org
Hop: From: localhost; By: hq5.merlinux.eu; Date: Sat, 14 Sep 2019 17:00:22 +0000
Hop: From: hq5.merlinux.eu; By: hq5.merlinux.eu; Date: Sat, 14 Sep 2019 17:00:25 +0000
DKIM Results: Passed=true, Works=true, Allow_Keychange=true";
DKIM Results: Passed=true";
check_parse_receive_headers_integration(raw, expected).await;
let raw = include_bytes!("../test-data/message/encrypted_with_received_headers.eml");
@@ -749,7 +754,7 @@ Hop: From: [127.0.0.1]; By: mail.example.org; Date: Mon, 27 Dec 2021 11:21:21 +0
Hop: From: mout.example.org; By: hq5.example.org; Date: Mon, 27 Dec 2021 11:21:22 +0000
Hop: From: hq5.example.org; By: hq5.example.org; Date: Mon, 27 Dec 2021 11:21:22 +0000
DKIM Results: Passed=true, Works=true, Allow_Keychange=true";
DKIM Results: Passed=true";
check_parse_receive_headers_integration(raw, expected).await;
}
@@ -961,12 +966,6 @@ DKIM Results: Passed=true, Works=true, Allow_Keychange=true";
assert!(extract_grpid_from_rfc724_mid(mid.as_str()).is_none());
}
use chrono::NaiveDate;
use proptest::prelude::*;
use crate::chatlist::Chatlist;
use crate::{chat, test_utils};
proptest! {
#[test]
fn test_truncate(

View File

@@ -862,6 +862,7 @@ impl Message {
mod tests {
use std::time::Duration;
use regex::Regex;
use serde_json::json;
use super::*;
@@ -1711,8 +1712,6 @@ mod tests {
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_render_webxdc_status_update_object_range() -> Result<()> {
use regex::Regex;
let t = TestContext::new_alice().await;
let chat_id = create_group_chat(&t, ProtectionStatus::Unprotected, "a chat").await?;
let instance = send_webxdc_instance(&t, chat_id).await?;

Binary file not shown.

After

Width:  |  Height:  |  Size: 392 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 202 KiB

View File

@@ -0,0 +1,65 @@
Return-Path: <autocrypt@nine.testrun.org>
Delivered-To: autocrypt@nine.testrun.org
Received: from nine.testrun.org
by nine with LMTP
id wNinAKX2J2YWDwEAPdT8mA
(envelope-from <autocrypt@nine.testrun.org>)
for <autocrypt@nine.testrun.org>; Tue, 23 Apr 2024 19:57:57 +0200
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=nine.testrun.org;
s=opendkim; t=1713895076;
bh=yuHuHSbYX5hE/xr8aU2fy/SlqfTL7XjfV2m1eEePTz4=;
h=Subject:Date:From:To:From;
b=ZbVNpJ8zjHmgrCqiRnqzENcR/PwR/G182hL18U5bp5CZmkyWcuhQU0EkhkJpCCv1n
8bZ9WlOT0cmzBHpWU43t7HufuUM56NwwuVqEuz2agpVzQV8zKIPhthrBzbYIeR4Prg
1DgwWr8EhotoV6yPgzxi9sMyO3l4spJeaREisB5MPOIdKeIxtRPLR+Woo5hQWNTFoh
ZQtCcY7w5vxXGhBMVPXOjbrrzOCsE5gGB5QYSAR8Bv3ZdJn/mHvIRCEJG5hJGSxXjQ
fD0UGJ5m5RVrF0tWnZ7U5tpoRD/UVV1+Us9Woq733R97ZchpoE4hNpMG9zYW90z4QU
kBajbsH81Nm0A==
MIME-Version: 1.0
Content-Type: multipart/mixed;
boundary=----X2OJUZLGILKJEHMTO29ZMST9701ZDH
Content-Transfer-Encoding: 7bit
Subject: Autocrypt Setup Message
Autocrypt-Setup-Message: v1
Date: Tue, 23 Apr 2024 19:57:57 +0200
From: autocrypt@nine.testrun.org
To: autocrypt@nine.testrun.org
Message-Id: <20240423175756.F19EB17C214A@nine.testrun.org>
------X2OJUZLGILKJEHMTO29ZMST9701ZDH
Content-Type: text/plain;
charset=utf-8
Content-Transfer-Encoding: quoted-printable
This message contains all information to transfer your Autocrypt settings a=
long with your secret key securely from your original device=2E To set up y=
our new device for Autocrypt, please follow the instructions that should be=
presented by your new device=2E You can keep this message and use it as a =
backup for your secret key=2E If you want to do this, you should write down=
the password and store it securely=2E
------X2OJUZLGILKJEHMTO29ZMST9701ZDH
Content-Transfer-Encoding: 7bit
Content-Type: application/autocrypt-setup
Content-Disposition: attachment; filename="autocrypt-setup-message"
-----BEGIN PGP MESSAGE-----
Passphrase-Format: numeric9x4
Passphrase-Begin: 06
ww0ECQMCAhlJ+TRwb2Fg0sGXAUc+92rmg4k57Sd4D3O/SPQNzShbVdlKsoFzyH+B
YhimOr/8C5ZHyg/WjRGlk4pD+t57WfVdE7LYnv8qsK86h2kffZAGlj+B9Lh9+qbV
KgJLpHUKg7ZGa/9aMq7KuFoNSNTbcHtzJ/Ml9GVe+opimER87mpFCjmaEHCcCp0a
ZeS5VU8gTV7AKuPW40BBipyEmKpUvE/ZWfz3KSI4RZyIwM8v8kXBMojT4WLqWm93
JoEKUyeh+3JKMvsfyRbmHXrHprG9f2e8PLvNkAiie68YJniFnwA8nmNSnPv9S9rf
7oUHtnTDKJ4FIpmfPgj1v/KIWWW9KaZWHi7K5mFUCTb4pBoCRIGaFh+JzbSlNL9i
fz7HIiN95bFJ4xXXL4gcU9wO5//npkVDUncaeHhUy1VBLu0NFYvze+s+eAIesqec
X3x++U9d+Slbpa1G2Z5Knj50mBY+k9aNwVMZGu50hzhPvdwesqmbr+GTSh0O1bxI
gw/cDq5s58Ewze3WvYaLxJz/RcwOCGSV8k21FM4WTnEahs4yfLbzNuusYvvciU6l
w0eZC+vEmh+bINSSRX/mcvkQcIkkCsqvfWyxdSNIBCwmR86oalWnxZniBLbbbZHD
0KAsv0w7t00Y715gyyFWyiEiT5Lyl4TA+cUIHKmmpKOaVubz50UD1z5rqT7joJ7G
KRmWtQW8MScgcmK7+tyavLQOxwe8i8i9JkUy+d9jhj17XZil/If26Q3V3epqCXq3
FdvEvvNGJF0DyJ4YAe9QMBumf22sMmX/XVock9/k0pB46mciMhPL3VA=
=LYx9
-----END PGP MESSAGE-----
------X2OJUZLGILKJEHMTO29ZMST9701ZDH--