Compare commits

..

207 Commits

Author SHA1 Message Date
B. Petersen
78d304443a chore(release): prepare for 1.134.0 2024-01-31 14:52:40 +00:00
link2xt
d6c24eb9f6 Make Accounts::background_fetch() not return Result 2024-01-31 14:04:03 +01:00
link2xt
f7fd1ef2bf Emit DC_EVENT_ACCOUNTS_BACKGROUND_FETCH_DONE even on timeout
Otherwise if there is a timeout,
UI will wait for DC_EVENT_ACCOUNTS_BACKGROUND_FETCH_DONE forever.
2024-01-31 14:04:03 +01:00
link2xt
af7bf5bd2b s/forgeting/forgetting/ 2024-01-31 14:04:03 +01:00
link2xt
ea666f1098 Hide background_fetch_without_timeout from public API 2024-01-31 14:04:03 +01:00
Simon Laux
5bb80f94c7 Apply suggestions from code review
Co-authored-by: bjoern <r10s@b44t.com>
2024-01-31 14:04:03 +01:00
link2xt
2f29c56a36 fix: do not log error if watched folder is not configured
This may happen if Sent folder does not exist
but configuration option to watch it is enabled.
2024-01-31 14:04:03 +01:00
Simon Laux
de86b8a96e rename event also in core 2024-01-31 14:04:03 +01:00
Simon Laux
060c9c8aa1 cargo fmt 2024-01-31 14:04:03 +01:00
Simon Laux
727428a965 rename event and mention event in method documentation 2024-01-31 14:04:03 +01:00
Simon Laux
df455bbcf5 BackgroundFetchCompletedForAllAccounts event 2024-01-31 14:04:03 +01:00
Simon Laux
946eea4c9e add rate limit for quota check in background fetch (12h for now) 2024-01-31 14:04:03 +01:00
Simon Laux
5cbc87369e rename cffi function 2024-01-31 14:04:03 +01:00
Simon Laux
5cdd5e0564 Apply suggestions from code review
Co-authored-by: link2xt <link2xt@testrun.org>
2024-01-31 14:04:03 +01:00
Simon Laux
f493d6bb40 don't hold write lock in cffi (this blocked events) 2024-01-31 14:04:03 +01:00
Simon Laux
8e073b9c3e log time that the function took 2024-01-31 14:04:03 +01:00
Simon Laux
ea2a692d18 fix iOS build issue 2024-01-31 14:04:03 +01:00
Simon Laux
1b7c5be9c5 Test server has no sentbox folder 2024-01-31 14:04:03 +01:00
Simon Laux
f7903df805 api: cffi: add dc_accounts_background_fetch_with_timeout 2024-01-31 14:04:03 +01:00
Simon Laux
d2c61dc90e api: jsonrpc: add background_fetch_for_all_accounts 2024-01-31 14:04:03 +01:00
Simon Laux
7b68098785 feat: add background fetch method 2024-01-31 14:04:03 +01:00
Simon Laux
48f2ea717e refactor: move convert folder meaning logic in own method
also unify the error handling for the cases where it can go wrong.
2024-01-31 14:04:03 +01:00
link2xt
cb3f03fd39 feat: add support for IMAP METADATA 2024-01-31 04:16:04 +00:00
iequidoo
06f1fe18d6 fix: Delete resent messages on receiver side (#5155)
If a Delta Chat message has the Message-ID already existing in the db, but a greater "Date", it's a
resent message that can be deleted. Messages having the same "Date" mustn't be deleted because they
can be already seen messages moved back to INBOX. Also don't delete messages having lesser "Date" to
avoid deleting both messages in a multi-device setting.
2024-01-30 22:01:45 -03:00
iequidoo
1dbf924c6a feat: chat::resend_msgs: Guarantee strictly increasing time in the Date header
Use `create_smeared_timestamp()` for this. This allows to dedup messages on the receiver -- if it
sees the same Message-ID, but a different timestamp, then it's a resent message that can be deleted.
2024-01-30 22:01:45 -03:00
link2xt
3f6814f421 chore: remove unnecessary exception from deny.toml 2024-01-30 20:27:07 +00:00
link2xt
782828ac4f chore(deps): update imap-proto 2024-01-30 20:12:07 +00:00
iequidoo
bd3759d55e test: test_import_export_online_all: Send the message to the existing address to avoid errors (#5220) 2024-01-29 20:34:03 -03:00
iequidoo
672993e69e feat: qr::check_qr(): Accept i.delta.chat invite links (#5217)
Accepts invite links with `https://i.delta.chat[/]#FINGERPRINT&a=ADDR[&OPTIONAL_PARAMS]`
scheme. Only `i.delta.chat` domain is supported now not to intersect with the common HTTPS scheme.
2024-01-28 20:33:56 -03:00
iequidoo
987bdaf237 ci: Add/remove necessary newlines to fix Python lint 2024-01-26 14:46:46 -03:00
iequidoo
7cf382a3b8 fix: Treat only "Auto-Submitted: auto-generated" messages as bot-sent (#5213)
"Auto-Submitted: auto-replied" messages mustn't be considered as sent by either bots or non-bots,
e.g. MDNs have this header value and it's the same for bots and non-bots.
2024-01-26 13:03:19 -03:00
Simon Laux
19dce9ddfa api!: jsonrpc: device message api now requires Option<MessageData> instead of String for the message (#5211)
api: jsonrpc: device message api now setting empty device messages to
block adding a message for a specific label

required for https://github.com/deltachat/deltachat-desktop/pull/3639

---------

Co-authored-by: iequidoo <117991069+iequidoo@users.noreply.github.com>
2024-01-25 11:26:11 +01:00
Simon Laux
0afc0dd65a fix: add tolerance for macOS and iOS changing # to %23
fixes #1969

Bug description:
macOS and iOS sometimes replace the # with %23 (uri encode it), we should be able to parse this wrong format too, see issue https://github.com/deltachat/deltachat-core-rust/issues/1969 for more info
2024-01-24 23:58:49 +01:00
link2xt
73d612a07d feat: do not drop unknown report attachments
In particular TLSRPT reports
contain files that may be interesting for admins.
Currently Delta Chat drops the attachment
so message appears as a text message without actual payload.
2024-01-24 16:13:20 +00:00
link2xt
3b1529ef81 chore(release): prepare for 1.133.2 2024-01-24 01:57:45 +00:00
link2xt
15187c0adb fix: downgrade OpenSSL from 3.2.0 to 3.1.4 2024-01-24 01:48:23 +00:00
Sebastian Klähn
c5f31c3d03 fix: No new chats for MDNs with alias (#5196) (#5199)
close #5196
2024-01-22 16:51:37 +01:00
link2xt
2c17e78347 chore(release): prepare for 1.133.1 2024-01-21 04:18:13 +00:00
Sebastian Klähn
4ee646ce0b feat(api): Add is_bot to cffi and jsonrpc (#5197)
@adbenitez wants this feature on Deltalab to display a bot tag. 
Other UIs might also want to adopt this feature :)

---------
Co-authored-by: link2xt <link2xt@testrun.org>
2024-01-20 15:00:10 +00:00
B. Petersen
1f7b4a74fa add missing 'unencrypted message' defines
in #5161, it was forgotten to adapt deltachat.h;
moreover, this PR tweaks some other minor things
2024-01-20 15:00:23 +01:00
Sebastian Klähn
4bc90701cc feat: Add system message when provider does not allow unencrypted messages (#5161) (#5195)
close #5161

![Screenshot from 2024-01-19
19-56-09](https://github.com/deltachat/deltachat-core-rust/assets/39526136/27ecdd9b-1739-410b-bb26-80d5bdbbc39a)

---------

Co-authored-by: bjoern <r10s@b44t.com>
2024-01-20 11:47:23 +00:00
dependabot[bot]
490deb9347 chore(deps): bump h2 from 0.3.17 to 0.3.24 in /fuzz
Bumps [h2](https://github.com/hyperium/h2) from 0.3.17 to 0.3.24.
- [Release notes](https://github.com/hyperium/h2/releases)
- [Changelog](https://github.com/hyperium/h2/blob/v0.3.24/CHANGELOG.md)
- [Commits](https://github.com/hyperium/h2/compare/v0.3.17...v0.3.24)

---
updated-dependencies:
- dependency-name: h2
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-01-19 23:58:52 +00:00
Sebastian Klähn
28d9484a13 fix(node): run tests with native ESM modules instead of esm (#5194)
close #5156

---------

Co-authored-by: Septias <scoreplayer2000@gmail.comclear>
2024-01-19 18:09:19 +01:00
link2xt
e67e684ee0 test: wait for joiner success in test_verified_group_[member_added]_recovery
If we wait for inviter success,
vg-member-added message may be still in flight
and reach ac2 after device resetup.

Making ac2 wait for joining the group ensures that old
device receives vg-member-added message
and new device will not receive it and fail to decrypt.

Other instances of wait_for_securejoin_inviter_success()
in the same tests are also replaced for reliability.
2024-01-18 17:00:21 +00:00
link2xt
6cfe3e6a97 chore(deps): update h2 0.4.0 -> 0.4.2 2024-01-18 13:40:38 +00:00
Sebastian Klähn
99ac524905 chore(deps): update h2 from 0.3.22 -> 0.3.24 2024-01-18 14:21:58 +01:00
link2xt
2faf7fdb78 fix: BCC-to-self even if server deletion is set to "at once" 2024-01-18 10:20:01 +00:00
link2xt
6a8ea8a083 fix: set message download state to Failure on IMAP errors
Previously the message was removed from `download` table,
but message bubble was stuck in InProgress state.

Now download state is updated by the caller,
so it cannot be accidentally skipped.
2024-01-18 10:18:57 +00:00
link2xt
e0e56cd831 chore: update quoted_printable to 0.5
And update mailparse to 0.14.1 so there is no duplicate dependency.
2024-01-18 09:35:05 +00:00
missytake
bbc6febb72 test: no timeout in SetupPlugin 2024-01-17 14:20:29 +01:00
missytake
7f7f42d721 test: Ensure that member is added before yielding chat 2024-01-17 14:20:29 +01:00
iequidoo
589236c27b fix: chat::send_msg: Remove encryption-related params from already sent message
This allows to send existing messages (incoming and outgoing) taken from encrypted chats, to
unencrypted ones. `Param::ForcePlaintext` is removed as well -- if a message can be sent encrypted
this time, nothing bad with this.
2024-01-17 14:20:29 +01:00
iequidoo
c16c5e0802 test: Bring test_forward_encrypted_to_unencrypted into line with current API
Currently `Chat.send_msg()` modifies the source message and returns another message object
equivalent to the source one. That's how it works in the core and thus in Python bindings too.
2024-01-17 14:20:29 +01:00
missytake
36cab40ac1 test: add get_protected_chat to testplugin.py 2024-01-17 14:20:29 +01:00
missytake
4186d78305 test: add python test for message forwarding from encrypted to unencrypted chat 2024-01-17 14:20:29 +01:00
iequidoo
06cccb77f8 feat: Use Quoted-Printable for the text part (#3986)
This is needed to protect from ESPs (such as gmx.at) doing their own Quoted-Printable encoding and
thus breaking messages and signatures. It's unlikely that the reader uses a MUA not supporting
Quoted-Printable encoding. And RFC 2646 "4.6" also recommends it for encrypted messages.
2024-01-16 23:46:24 -03:00
link2xt
1895f4c556 chore(release): prepare for 1.133.0 2024-01-15 22:55:26 +00:00
link2xt
849a873e61 feat: only try to configure non-strict TLS checks if explicitly set
Trying non-strict TLS checks is not necessary
for most servers with proper TLS setup,
but doubles the time needed to fail configuration
when the server is not responding, e.g.
when all connection attempts time out.

There is also a risk of accidentally
configuring non-strict TLS checks in a rare case
that strict TLS check configuration spuriously failed,
e.g. on a bad network.

If the server has a known broken TLS setup,
it can still be added to the provider database
or configured with non-strict TLS check manually.
User can also configure another email provider,
such as chatmail servers, instead of using the server
with invalid TLS hostname.

This change does not affect exising setups.
2024-01-15 22:54:31 +00:00
link2xt
b5c0372c99 docs: restore "Constants" page in Doxygen >=1.9.8
deltachat.h uses `@defgroup` commands to create topics
for groups of constants. Prior to Doxygen 1.9.8
defining a group created a "module"
and all constants were visible from the modules.html page.
In Doxygen 1.9.8 "modules" were renamed into "topics"
as C++20 modules have taken their place,
so Delta Chat documentation does not have modules
in Doxygen sense anymore.

The change is to replace "modules.html" with "topics.html"
in the DoxygenLayout.xml.

See <https://www.doxygen.nl/manual/grouping.html> for
Doxygen documentation about groups and their relation to topics.
2024-01-14 12:17:54 +00:00
link2xt
1ba9b69849 chore: npm run build:core:constants 2024-01-13 22:51:24 +00:00
holger krekel
6345a4f5b3 fix link for securejoin 2024-01-13 12:50:16 +01:00
Sebastian Klähn
382fc75b1e Add more docs (#5174)
Add some docs to smtp functions
2024-01-12 11:14:05 +01:00
Sebastian Klähn
92fc9ea971 feat: Encrypt MDNs #5168 (#5175)
This PR stops MDNs from being forced to be sent unencrypted. 
If no encryption is possible (by `should_encrypt`), the fix #5152 still
applies.

close #5168
2024-01-12 10:54:54 +01:00
Sebastian Klähn
de7ac2a240 fix: emit events more reliable when starting and stopping io #5097 (#5101)
Send `EventType::ConnectivityChanged` when using the context methods
`start_io` and `stop_io`.

close #5097

---------

Co-authored-by: Septias <scoreplayer2000@gmail.comclear>
2024-01-12 09:45:34 +01:00
link2xt
7b0e5adaee chore(deps): update rustyline from 12 to 13 2024-01-12 02:45:53 +00:00
iequidoo
406b59501b chore: deltachat-jsonrpc/src/api/types/events.rs: Apply rustfmt 2024-01-11 21:53:26 -03:00
iequidoo
d5da2bed75 feat: Add ConfigSynced event
Add an event for a case if a multi-device synced config value changed. Maybe the app needs to
refresh smth on such an event. For uniformity it is emitted on the source device too. The value is
omitted, otherwise it would be logged which might not be good for privacy.
2024-01-11 21:53:26 -03:00
iequidoo
924d5b9377 feat: Sync contact creation/rename across devices (#5163)
Use `chat::SyncAction::Rename` for that. Anyway 1:1-s can't be renamed and a separate sync action
would only complicate the code.
2024-01-10 16:46:54 -03:00
iequidoo
bb47299ee4 fix: contact::set_blocked(): Don't fail on sync errors, just log them
Multi-device synchronisation is not critical and should not fail the local operation, in other
places sync errors are already ignored.
2024-01-10 16:46:54 -03:00
link2xt
20065d3daa docs: add a NOTE comment about KeyId backward verification race 2024-01-09 21:46:37 +00:00
link2xt
ccb267beab refactor: rename notify_peer_verified() into set_peer_verified()
It was named notify_peer_verified()
because it added info message,
but this is no longer true since
https://github.com/deltachat/deltachat-core-rust/pull/4998
(commit c6ea4e389a)
is merged.
2024-01-09 21:46:37 +00:00
link2xt
32bcb59601 refactor: do not emit ChatModified event in notify_peer_verified()
The chat is not modified at least since
c6ea4e389a
(PR https://github.com/deltachat/deltachat-core-rust/pull/4998),
even the info message is not posted there.
2024-01-09 21:46:37 +00:00
link2xt
c708c44f0a feat: mark 1:1 chat as verified for Bob early
Mark 1:1 chat as verified as soon as Alice is forward-verified
so Bob can already start sending Chat-Verified headers.
This way Alice and Bob can scan each other's QR codes
and even if all Secure-Join headers are dropped from the network,
still get forward verifications via QR-code scans
and backward verifications via Chat-Verified messages in 1:1 chat.
2024-01-09 21:46:37 +00:00
link2xt
9415a71f9d refactor: rename fingerprint_equals_sender to verify_sender_by_fingerprint 2024-01-09 21:46:37 +00:00
link2xt
1fd42f2c53 test: test recovery from lost vc-contact-confirm 2024-01-09 21:46:37 +00:00
link2xt
1e52502ab3 refactor: send Secure-Join-Fingerprint only in *-request-with-auth 2024-01-09 21:46:37 +00:00
link2xt
a144d7e4f3 test: test that changing default private key breaks backward verification 2024-01-09 21:46:37 +00:00
link2xt
e855b79f9c feat: add backward_verified_key_id column to acpeerstates 2024-01-09 21:46:37 +00:00
link2xt
2f8a8f9f50 ci: update to Rust 1.75.0 and fix clippy 2024-01-08 20:01:40 +00:00
link2xt
b9a58bf625 docs: add a link to autoconfig RFC draft
This will hopefully replace deleted Mozilla documentation page
in the future.
2024-01-07 22:55:16 +00:00
iequidoo
c8075e53d2 fix: Reset message error when scheduling resending (#5119)
Before, while a message is in OutPending state after resending is requested, the user still sees the
red marker with error and it is confusing, so the user don't know the sending state of the message.
2024-01-07 15:02:31 -03:00
iequidoo
ff54cf24a1 fix: message::update_msg_state(): Reset error if message is delivered (#5119) 2024-01-05 20:43:55 -03:00
link2xt
af0833e821 ci: downgrade chai from 4.4.0 to 4.3.10
4.4.0 fails with a syntax error in CI currently.
2024-01-05 23:36:57 +00:00
link2xt
da11542322 fix: do not remove contents from Schleuder ML messages
Before this fix actual contents of the message
reposted by Schleuder is considered a mailing list footer and removed,
not visible even in the "Show Full Message..." view.

With this change there will be two message bubbles,
one for header and one for the contents,
but it is still better than losing the contents completely.

Attempting to parse header part is out of scope for this change.
2024-01-05 15:42:56 +00:00
link2xt
3bcdd1770a test: test that read receipts don't degrade encryption
This is broken since 44227d7b86
mimeparser only recognizes read receipts
by the Content-Type being "multipart/report".
If multipart/report is hidden inside multipart/mixed
and the message is not encrypted,
it degrades encryption.
2024-01-05 15:34:48 +00:00
link2xt
4dc596e646 fix(mimefactory): do not wrap MDNs into multipart/mixed part 2024-01-05 15:34:48 +00:00
link2xt
2e69210825 refactor: use wait_for_incoming_msg_event() more 2024-01-05 15:34:48 +00:00
iequidoo
625887d249 fix: Split SMTP jobs already in chat::create_send_msg_jobs() (#5115)
a27e84ad89 "fix: Delete received outgoing messages from SMTP queue"
can break sending messages sent as several SMTP messages because they have a lot of recipients:
`pub(crate) const DEFAULT_MAX_SMTP_RCPT_TO: usize = 50;`

We should not cancel sending if it is such a message and we received BCC-self because it does not
mean the other part was sent successfully. For this, split such messages into separate jobs in the
`smtp` table so that only a job containing BCC-self is canceled from `receive_imf_inner()`. Although
this doesn't solve the initial problem with timed-out SMTP requests for such messages completely,
this enables fine-grained SMTP retries so we don't need to resend all SMTP messages if only some of
them failed to be sent.
2024-01-05 01:53:41 -03:00
link2xt
b7c34b7794 chore: remove minor version from serde_json spec 2024-01-04 16:06:33 +00:00
link2xt
941cf38a3e chore(deps): cargo update 2024-01-04 16:03:28 +00:00
dependabot[bot]
7f61896ec8 Merge pull request #5136 from deltachat/dependabot/cargo/futures-0.3.30 2024-01-03 17:19:18 +00:00
dependabot[bot]
b14b49cbf0 Merge pull request #5146 from deltachat/dependabot/cargo/anyhow-1.0.79 2024-01-03 17:17:21 +00:00
dependabot[bot]
6de3510a5d Merge pull request #5131 from deltachat/dependabot/cargo/reqwest-0.11.23 2024-01-03 04:53:00 +00:00
dependabot[bot]
dea519095c chore(cargo): bump futures from 0.3.29 to 0.3.30
Bumps [futures](https://github.com/rust-lang/futures-rs) from 0.3.29 to 0.3.30.
- [Release notes](https://github.com/rust-lang/futures-rs/releases)
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.29...0.3.30)

---
updated-dependencies:
- dependency-name: futures
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-01-03 04:48:48 +00:00
dependabot[bot]
3f8ca0cee9 Merge pull request #5133 from deltachat/dependabot/cargo/tempfile-3.9.0 2024-01-03 04:47:42 +00:00
dependabot[bot]
1b998da57a chore(cargo): bump anyhow from 1.0.75 to 1.0.79
Bumps [anyhow](https://github.com/dtolnay/anyhow) from 1.0.75 to 1.0.79.
- [Release notes](https://github.com/dtolnay/anyhow/releases)
- [Commits](https://github.com/dtolnay/anyhow/compare/1.0.75...1.0.79)

---
updated-dependencies:
- dependency-name: anyhow
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-01-03 04:46:17 +00:00
dependabot[bot]
772747d42d Merge pull request #5135 from deltachat/dependabot/cargo/syn-2.0.43 2024-01-03 04:43:06 +00:00
dependabot[bot]
3998258afb Merge pull request #5141 from deltachat/dependabot/cargo/quote-1.0.34 2024-01-03 04:40:31 +00:00
dependabot[bot]
4e86de98c4 chore(cargo): bump quote from 1.0.33 to 1.0.34
Bumps [quote](https://github.com/dtolnay/quote) from 1.0.33 to 1.0.34.
- [Release notes](https://github.com/dtolnay/quote/releases)
- [Commits](https://github.com/dtolnay/quote/compare/1.0.33...1.0.34)

---
updated-dependencies:
- dependency-name: quote
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-01-01 21:33:28 +00:00
dependabot[bot]
2a497989e9 chore(cargo): bump syn from 2.0.41 to 2.0.43
Bumps [syn](https://github.com/dtolnay/syn) from 2.0.41 to 2.0.43.
- [Release notes](https://github.com/dtolnay/syn/releases)
- [Commits](https://github.com/dtolnay/syn/compare/2.0.41...2.0.43)

---
updated-dependencies:
- dependency-name: syn
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-01-01 21:32:04 +00:00
dependabot[bot]
361b19e455 chore(cargo): bump tempfile from 3.8.1 to 3.9.0
Bumps [tempfile](https://github.com/Stebalien/tempfile) from 3.8.1 to 3.9.0.
- [Changelog](https://github.com/Stebalien/tempfile/blob/master/CHANGELOG.md)
- [Commits](https://github.com/Stebalien/tempfile/compare/v3.8.1...v3.9.0)

---
updated-dependencies:
- dependency-name: tempfile
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-01-01 21:31:38 +00:00
dependabot[bot]
c036b26ae5 chore(cargo): bump reqwest from 0.11.22 to 0.11.23
Bumps [reqwest](https://github.com/seanmonstar/reqwest) from 0.11.22 to 0.11.23.
- [Release notes](https://github.com/seanmonstar/reqwest/releases)
- [Changelog](https://github.com/seanmonstar/reqwest/blob/master/CHANGELOG.md)
- [Commits](https://github.com/seanmonstar/reqwest/compare/v0.11.22...v0.11.23)

---
updated-dependencies:
- dependency-name: reqwest
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-01-01 21:31:13 +00:00
link2xt
dcf6ffef12 fix(imap): fail fast on LIST errors
async-imap returns infinite stream of errors
in case of EOF or timeout on the input stream,
so attempting to skip and log errors results in busy loop
similar to this:

   2023-12-22T13:07:35.751Z core/event WARNING "" 2 "/__w/deltachat-core-rust/deltachat-core-rust/src/imap/scan_folders.rs:112: list_folders() can't get folder: io: timed out: timed out"
   2023-12-22T13:07:35.751Z core/event WARNING "" 2 "/__w/deltachat-core-rust/deltachat-core-rust/src/imap/scan_folders.rs:112: list_folders() can't get folder: io: timed out: timed out"
   2023-12-22T13:07:35.751Z core/event WARNING "" 2 "/__w/deltachat-core-rust/deltachat-core-rust/src/imap/scan_folders.rs:112: list_folders() can't get folder: io: timed out: timed out"
   2023-12-22T13:07:35.751Z core/event WARNING "" 2 "/__w/deltachat-core-rust/deltachat-core-rust/src/imap/scan_folders.rs:112: list_folders() can't get folder: io: timed out: timed out"
   2023-12-22T13:07:35.752Z core/event WARNING "" 2 "/__w/deltachat-core-rust/deltachat-core-rust/src/imap/scan_folders.rs:112: list_folders() can't get folder: io: timed out: timed out"
   2023-12-22T13:07:35.752Z core/event WARNING "" 2 "/__w/deltachat-core-rust/deltachat-core-rust/src/imap/scan_folders.rs:112: list_folders() can't get folder: io: timed out: timed out"
   2023-12-22T13:07:35.752Z core/event WARNING "" 2 "/__w/deltachat-core-rust/deltachat-core-rust/src/imap/scan_folders.rs:112: list_folders() can't get folder: io: timed out: timed out"
   2023-12-22T13:07:35.752Z core/event WARNING "" 2 "/__w/deltachat-core-rust/deltachat-core-rust/src/imap/scan_folders.rs:112: list_folders() can't get folder: io: timed out: timed out"
   2023-12-22T13:07:35.753Z core/event WARNING "" 2 "/__w/deltachat-core-rust/deltachat-core-rust/src/imap/scan_folders.rs:112: list_folders() can't get folder: io: timed out: timed out"
   2023-12-22T13:07:35.754Z core/event WARNING "" 2 "/__w/deltachat-core-rust/deltachat-core-rust/src/imap/scan_folders.rs:112: list_folders() can't get folder: io: timed out: timed out"
   2023-12-22T13:07:35.754Z core/event WARNING "" 2 "/__w/deltachat-core-rust/deltachat-core-rust/src/imap/scan_folders.rs:112: list_folders() can't get folder: io: timed out: timed out"

To avoid busy loop, fail fast on first error
and bubble it up instead of trying to recover.
2023-12-28 15:20:15 +00:00
iequidoo
865ede39fe fix: Properly escape target in receive_imf_inner()
The bug was made in 44227d7b86. Sql::execute() with placeholders must
be used to escape strings, one never should escape them manually as strings themselves can contain
escape symbols. Thanks to @link2xt for noticing.
2023-12-22 17:42:25 -03:00
iequidoo
a27e84ad89 fix: Delete received outgoing messages from SMTP queue (#5115)
Some SMTP servers are running slow before-queue filters, most commonly Postfix with `rspamd` filter
which is implemented as a [before-queue Milter](https://www.postfix.org/MILTER_README.html). Some of
`rspamd` plugin filters are slow on large mails.

We previously had problems with timing out during waiting for SMTP response:
https://github.com/deltachat/deltachat-core-rust/issues/1383. This is largely fixed by
https://github.com/async-email/async-smtp/pull/29 and currently we have 60-second timeout just for
reading a response but apparently it is not sufficient -- maybe connection gets killed by NAT while
we are waiting for response or `rspamd` takes more than 60 seconds for large messages.

As a result a message is resent multiple times and eventually fails with "too many retries" while
multiple BCC-self messages are received.

We should remove the message from the SMTP queue as soon as we receive it via IMAP as it is clear
the message was sent even if we did not manage to get actual SMTP server response.
2023-12-21 16:57:23 -03:00
iequidoo
b83bd26325 refactor: Don't pass seen=true to add_parts() when replacing existing message
First of all, it's just downloaded and hasn't been seen yet by the user. Also this changes nothing
as `msgs.state` isn't changed when replacing a message anyway.
2023-12-21 16:14:52 -03:00
iequidoo
44227d7b86 fix: Put Message-ID into hidden headers and take it from there on receiver (#4798)
Put a copy of Message-ID into hidden headers and prefer it over the one in the IMF header section
that servers mess up with.

This also reverts "Set X-Microsoft-Original-Message-ID on outgoing emails for amazonaws (#3077)".
2023-12-21 16:14:52 -03:00
iequidoo
6bcf022523 refactor: receive_imf_inner: Rename replace_partial_download to replace_msg_id
It's more consistent with `replace_chat_id` and the same as the corresponding parameter name of
`add_parts()`.
2023-12-21 16:14:52 -03:00
link2xt
ccec26ffa7 fix(imap): limit the rate of LOGIN attempts rather than connection attempts
As ratelimit was introduced to avoid reconnecting immediately after disconnecting
in case of bugs in IMAP protocol handling,
connection attempts should only be counted when IMAP is actually used,
i.e. when the first command (LOGIN) is sent.
2023-12-21 08:07:34 +00:00
link2xt
83e159e42f refactor: better log at the start of imex_inner() 2023-12-18 21:03:09 +00:00
link2xt
cbabd4219e refactor: improve error message when non-verified contact is added to protected chat 2023-12-18 21:03:09 +00:00
link2xt
548afe3153 refactor: derive Debug, PartialEq and Eq for VerifiedEncryption 2023-12-18 21:03:09 +00:00
link2xt
35c5f42b35 refactor: use let-else in *-request-with-auth handler 2023-12-18 21:03:09 +00:00
link2xt
b9ff8b1d6c refactor: flatten peerstate::maybe_do_aeap_transition 2023-12-18 21:03:09 +00:00
link2xt
bb6a20dc11 test: test joining non-protected group 2023-12-18 21:03:09 +00:00
link2xt
e97955f5a0 refactor: flatten handle_auth_require() with let..else 2023-12-18 21:03:09 +00:00
iequidoo
35bd56ffea fix: Securejoin: Mark 1:1s as protected regardless of the Config::VerifiedOneOnOneChats
As per the comment in `receive_imf.rs`, `chat.protected` must be maintained regardless of the
`Config::VerifiedOneOnOneChats`. The only thing that mustn't be done if `VerifiedOneOnOneChats` is
unset (i.e. for non-supporting UIs) is marking chats as "protection broken" because this needs
showing the corresponding dialog to a user.
2023-12-18 16:32:09 -03:00
link2xt
78affb766e fix: do not ignore peerstate.save_to_db() errors 2023-12-18 11:57:16 +00:00
link2xt
9b1704e3b2 feat(deltachat-repl): enable INFO logging by default and add timestamps 2023-12-17 15:10:19 +00:00
link2xt
55cdbdc085 refactor(sql): recreate keypairs table
Removed unused `addr` and `created` field.
`is_default` boolean flag is moved into `config` row
pointing to the current default key.
2023-12-17 14:13:54 +00:00
link2xt
58620988d7 refactor(sql): recreate config table with UNIQUE constraint 2023-12-17 14:13:54 +00:00
link2xt
467f313091 chore: cargo update 2023-12-17 13:31:20 +00:00
dependabot[bot]
091578573a chore(cargo): bump zerocopy from 0.7.29 to 0.7.31
Bumps [zerocopy](https://github.com/google/zerocopy) from 0.7.29 to 0.7.31.
- [Release notes](https://github.com/google/zerocopy/releases)
- [Changelog](https://github.com/google/zerocopy/blob/main/CHANGELOG.md)
- [Commits](https://github.com/google/zerocopy/compare/v0.7.29...v0.7.31)

---
updated-dependencies:
- dependency-name: zerocopy
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-12-15 16:17:23 -03:00
iequidoo
62c1237024 refactor: Move calc_{protection_msg_,}sort_timestamp() to impl ChatId 2023-12-13 20:29:47 -03:00
iequidoo
8d41d02397 fix: calc_sort_timestamp: Skip messages that mustn't affect sorting of a new message (#5088)
Drafts mustn't affect sorting of any other messages, they aren't even displayed in the chat
window. Also hidden messages mustn't affect sorting of usual messages. But let hidden messages sort
together with protection messages because hidden messages also can be or not be verified, so let's
preserve this information -- even it's not useful currently, it can be useful in the future
versions.
2023-12-13 20:29:47 -03:00
iequidoo
fce3f80654 fix: Always pass the correct sort timestamp to ChatId::set_protection() (#5088)
Before in some places it was correctly calculated by passing the "sent" timestamp to
`calc_sort_timestamp()`, but in other places just the system time was used. In some complex
scenarios like #5088 (restoration of a backup made before a contact verification) it led to wrong
sort timestamps of protection messages and also messages following by them.

But to reduce number of args passed to functions needing to calculate the sort timestamp, add
message timestamps to `struct MimeMessage` which is anyway passed everywhere.
2023-12-13 20:29:47 -03:00
link2xt
2a0a51bea0 chore: remove n0-computer from deny.toml 2023-12-12 19:57:24 +00:00
link2xt
91d94d5920 build: use released version of iroh 0.4.2 2023-12-12 14:56:38 +00:00
link2xt
c59f21230d chore(release): prepare for 1.132.1 2023-12-12 02:58:29 +00:00
link2xt
828cc1fbd1 fix(connectivity): return false from all_work_done() immediately after connecting
We do not want all_work_done() to return true immediately
after calling start_io(), but only when connection goes idle.

"Connected" state is set immediately after connecting to the server,
but it does not mean there is nothing to do.

This change make all_work_done() return false
from the Connected state and introduces a new Idle
connectivity state that is only set before connection
actually goes idle. For idle state all_work_done() returns true.

From the user point of view both old Connected state
and new Idle state look the same.
2023-12-12 02:19:51 +00:00
link2xt
57f4958fc6 test(deltachat-rpc-client): test Account.{import,export}_self_keys 2023-12-11 06:43:10 +00:00
link2xt
3aeb57b4df api(deltachat-jsonrpc-client): add Account.{import,export}_self_keys 2023-12-11 06:43:10 +00:00
link2xt
1b85614db9 fix: renew IDLE timeout on keepalives and reduce it to 5 minutes
This change depends on async-imap update that resets the timeout
every time an `* OK Still here` is received.

Reducing timeout allows to detect lost connections
not later than 6 minutes
because Delta Chat will attempt to finish IDLE with DONE
after 5 minutes without keepalives
and will either get TCP RST directly
or, worst case, wait another minute for TCP socket read timeout.
2023-12-11 06:32:13 +00:00
link2xt
57ecf49eb1 chore: fix typo ("Bot" instead of "Bob") 2023-12-11 04:08:45 +00:00
iequidoo
f279b0d1e5 feat: Sync user actions for ad-hoc groups across devices (#5065)
Ad-hoc groups don't have grpid-s that can be used to identify them across devices and thus wasn't
synced until now.

The same problem already exists for assigning messages to ad-hoc groups and this assignment is done
by `get_parent_message()` and `lookup_chat_by_reply()`. Let's reuse this logic for the
synchronisation, it works well enough and this way we have less surprises than if we try to
implement grpids for ad-hoc groups. I.e. add an `Msgids` variant to `chat::SyncId` analogous to the
"References" header in messages and put two following Message-IDs to a sync message:
- The latest message A having `DownloadState::Done` and the state to be one of `InFresh, InNoticed,
  InSeen, OutDelivered, OutMdnRcvd`.
- The message that A references in `In-Reply-To`.

This way the logic is almost the same to what we have in `Chat::prepare_msg_raw()` (the difference
is that we don't use the oldest Message-ID) and it's easier to reuse the existing code.

NOTE: If a chat has only an OutPending message f.e., the synchronisation wouldn't work, but trying
to work in such a corner case has no significant value and isn't worth complicating the code.
2023-12-09 01:41:07 -03:00
iequidoo
32071297e6 feat: Add "From:" to protected headers for signed-only messages 2023-12-09 00:57:35 -03:00
link2xt
1d98c38ff3 ci: update to Rust 1.74.1 2023-12-08 23:52:36 +00:00
link2xt
c09e0e2b65 refactor: move AEAP and peerstate save from mimeparser to receive_imf()
Ideally mimeparser should be functional
and have no side effects such as modifying a peerstate in the database.
2023-12-08 23:32:03 +00:00
link2xt
0c8f967391 test: refine test_encrypted_no_autocrypt()
- Use TestContextManager
- Actually run receive_imf rather than only mimeparser on "received" messages
- Check that received message parts actually have a padlock
2023-12-08 23:32:03 +00:00
link2xt
aca34379e0 fix: add padlock to empty part if the whole message is empty
parse_mime_recursive() skips empty text parts,
so there may be no parts as the result of parsing.
In this case an empty part is added.
However, because it is added with parts.push()
rather than add_single_part(),
it is added without a padlock even if the message is encrypted.
`do_add_single_part()` adds padlock (GuaranteeE2EE param)
and should be used to add parts instead.
2023-12-07 03:38:20 +00:00
link2xt
1edd7045be chore(release): prepare for 1.132.0 2023-12-06 17:52:46 +00:00
B. Petersen
c784c499c2 fix: do not check lock_task on iOS before syncing
`lock_task` is anyways always `None` on iOS
to avoid lock files held open and cause 0xdead10cc crashes.
2023-12-06 18:17:49 +01:00
link2xt
36c751bcc3 chore: cargo update 2023-12-06 17:02:22 +00:00
link2xt
8a14a84bec test: check that ac2 gets a verified chat in test_securejoin_after_contact_resetup 2023-12-06 16:44:18 +00:00
link2xt
b00703cec2 fix: protect groups even if some members are not verified 2023-12-06 16:44:18 +00:00
link2xt
05e783564f refactor: log if the group is created as protected or not 2023-12-06 16:44:18 +00:00
link2xt
330fb02486 test: add test_securejoin_after_contact_resetup test
This test reproduces a bug preventing joining the group with a QR code
if the group already has a contact with inconsistent key state,
which means both Autocrypt and verified key exist,
but don't match.
This can happen when an invite QR code created by this contact
is scanned as scanning an invite code creates unprotected group
with the inviter for info messages.
If securejoin process never finishes because the inviter is offline,
group remains in this unprotected state with added inviter.

Normally the group becomes verified when a "Member added" (vg-member-added)
message is received in the chat.
However, current code checks that all members
of the chat are verified
(have a green checkmark, use verified key in 1:1 chat)
before marking the group as verified and fails otherwise.
2023-12-06 16:44:18 +00:00
link2xt
1447ab8dac refactor: clean up the logs and reduce noise
- Remove "Detected Autocrypt-mime message" logs printed for every incoming Autocrypt message.
- Print only a single line at the beginning of receive_imf with both the Message-ID and seen flag.
- Print Securejoin step only once, inside handle_securejoin_handshake or observe_securejoin_on_other_device.
- Do not log "Not creating ad-hoc group" every time ad-hoc group is not created, log when it is created instead.
- Log ID of the chat where Autocrypt-Gossip for all members is received.
- Do not print "Secure-join requested." for {vg,vc}-request, we already log the step.
- Remove ">>>>>>>>>>>>>>>>>>>>>>>>>" noise from securejoin logs.
2023-12-06 01:56:35 +00:00
link2xt
d574ee4edb chore: update zerocopy from 0.7.28 to 0.7.29
0.7.28 is yanked.
2023-12-05 19:25:27 +00:00
link2xt
814fe953a9 chore(cargo): update filetime 2023-12-05 16:45:24 +00:00
link2xt
280f13b8cf fix: do not lock accounts.toml on iOS
This results in 0xdead10cc crashes on suspend.
iOS itself ensures that multiple instances of Delta Chat are not running.
2023-12-04 21:51:17 +00:00
link2xt
a96b44a482 fix: do not mark recipients as verified if there is no Chat-Verified header 2023-12-04 15:34:09 +00:00
link2xt
4286d248e9 feat: increase TCP timeouts from 30 to 60 seconds
GitHub Action tests sometimes fail with TCP connection
timeouts, especially for macOS.
2023-12-04 12:50:07 +00:00
dependabot[bot]
116537019b chore(deps): bump self_cell from 1.0.1 to 1.0.2 in /fuzz
Bumps [self_cell](https://github.com/Voultapher/self_cell) from 1.0.1 to 1.0.2.
- [Release notes](https://github.com/Voultapher/self_cell/releases)
- [Commits](https://github.com/Voultapher/self_cell/compare/v1.0.1...v1.0.2)

---
updated-dependencies:
- dependency-name: self_cell
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-12-03 16:05:27 -03:00
iequidoo
8b37b8c1fd fix: Don't sort message creating a protected group over a protection message (#4963)
Otherwise it looks like the message creating a protected group is not verified. For this, use
`sent_timestamp` of the received message as an upper limit of the sort timestamp (`msgs.timestamp`)
of the protection message. As the protection message is added to the chat earlier, this way its
timestamp is always less or eq than the received message's timestamp.
2023-12-03 15:10:54 -03:00
iequidoo
63b4339ca0 test: Message order in a just created protected group on the second device (#4963)
Test that on the second device of a protected group creator the first message is
`SystemMessage::ChatProtectionEnabled` and the second one is the message populating the group.
2023-12-03 15:10:54 -03:00
link2xt
fdd239f61f fix: narrow the scope of verification exception to 1:1 chats
Allowing outgoing unencrypted messages in groups with 2 members
breaks the test
`python/tests/test_0_complex_or_slow.py::test_verified_group_vs_delete_server_after`
2023-12-03 15:46:56 +00:00
link2xt
5ca5d95c5e refactor: call has_verified_encryption() in a single place
This centralizes all Securejoin/verification checks and updates in one
place right before add_parts() even before we assign the message to
the chat, so we can decouple chat logic from verification logic.
2023-12-03 15:46:56 +00:00
link2xt
3fcad50924 refactor: move to_ids.is_empty() check into mark_recepients_as_verified() 2023-12-03 15:46:56 +00:00
link2xt
8e40540d24 refactor: add debug assertion where we expect a 1:1 chat 2023-12-03 15:46:56 +00:00
link2xt
04d22bb84d refactor: remove chattype argument from has_verified_encryption() 2023-12-03 15:46:56 +00:00
link2xt
5415f1bfa1 docs: has_verified_encryption() does not check that all members are verified 2023-12-03 15:46:56 +00:00
link2xt
ff3bf4791a chore: update dependencies 2023-12-03 00:43:21 +00:00
dependabot[bot]
eebea216cb chore(cargo): bump testdir from 0.8.1 to 0.9.0
Bumps [testdir](https://github.com/flub/testdir) from 0.8.1 to 0.9.0.
- [Changelog](https://github.com/flub/testdir/blob/main/CHANGELOG.md)
- [Commits](https://github.com/flub/testdir/compare/v0.8.1...v0.9.0)

---
updated-dependencies:
- dependency-name: testdir
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-12-02 15:50:51 +00:00
link2xt
fbcd7f46b8 chore(release): prepare for 1.131.9 2023-12-02 01:18:34 +00:00
iequidoo
846278b18e feat: chat::rename_ex: Don't send sync message if usual message is sent
It's not necessary and in other places like add_contact_to_chat_ex() sync messages are also sent
only if there are no system messages sent like MemberAddedToGroup.
2023-12-01 21:41:58 -03:00
iequidoo
2f2b1e18bf test: Split test_sync_alter_chat() into smaller tests 2023-12-01 21:41:58 -03:00
iequidoo
073c250fa4 refactor: Add test_utils::sync()
Add a function that pops a sync message from one Alice's device and receives it on another.
2023-12-01 21:41:58 -03:00
iequidoo
1f336f89a6 feat: Sync Config::Displayname across devices (#4893)
We already synchronise status/footer when we see a self-sent message with a Chat-Version
header. Would be nice to do the same for display name.

But let's do it the same way as for `Config::{MdnsEnabled,ShowEmails}`. Otherwise, if we sync the
display name using the "From" header, smth like `Param::StatusTimestamp` is needed then to reject
outdated display names. Also this timestamp needs to be updated when `Config::Displayname` is set
locally. Also this wouldn't work if system time isn't synchronised on devices. Also using multiple
approaches to sync different config values would lead to more code and bugs while having almost no
value -- using "From" only saves some bytes and allows to sync some things w/o the synchronisation
itself to be enabled. But the latter also can be a downside -- if it's usual synchonisation, you can
(potentially) disable it and share the same email account across people in some organisation
allowing them to have different display names. With using "From" for synchronisation such a
capability definitely requires a new config option.
2023-12-01 21:41:58 -03:00
iequidoo
a47fec7f6c feat: Sync Config::{MdnsEnabled,ShowEmails} across devices (#4954)
Motivation: Syncing these options will improve UX in very most cases and should be done. Other
candidates are less clear or are advanced options, we can reconsider that at some point later.

Approach:
- Sync options one-by-one when the corresponding option is set (even if to the same value).
- Don't sync when an option is reset to a default as defaults may differ across client versions.
- Check on both sides that the option should be synced so that if there are different client
  versions, the synchronisation of the option is either done or not done in both directions.
  Moreover, receivers of a config value need to check if a key can be synced because some settings
  (e.g. Avatar path) could otherwise lead to exfiltration of files from a receiver's device if we
  assume an attacker to have control of a device in a multi-device setting or if multiple users are
  sharing an account.
- Don't sync `SyncMsgs` itself.
2023-12-01 21:41:58 -03:00
iequidoo
084434d3b4 feat: receive_imf_inner: Add missing initialisation of ReceivedMsg::from_is_signed 2023-12-01 21:41:58 -03:00
iequidoo
ebfbc11973 feat: Don't affect MimeMessage with "From" and secured headers from encrypted unsigned messages
If a message is encrypted, but unsigned:
- Don't set `MimeMessage::from_is_signed`.
- Remove "secure-join-fingerprint" and "chat-verified" headers from `MimeMessage`.
- Minor: Preserve "Subject" from the unencrypted top level if there's no "Subject" in the encrypted
  part, this message is displayed w/o a padlock anyway.

Apparently it didn't lead to any vulnerabilities because there are checks for
`MimeMessage::signatures.is_empty()` in all necessary places, but still the code looked dangerous,
especially because `from_is_singed` var name didn't correspond to its actual value (it was rather
`from_is_encrypted_maybe_signed`).
2023-12-01 19:06:11 -03:00
link2xt
9cc9579b2d feat: remove receiver limit on .xdc size
If we have downloaded the file anyway,
might as well allow to open it.
2023-12-01 15:53:38 +00:00
link2xt
7beccd9dbc refactor: better error context in send_webxdc_status_update_struct() 2023-12-01 15:19:23 +00:00
link2xt
0e195bc7a2 fix: lock the database when INSERTing a webxdc update
`query_row_optional` does not hold the write lock
and may fail with "database is locked" error
or cause the other task such as SMTP loop to fail.
2023-12-01 15:19:23 +00:00
link2xt
f89efd5fce test: test inserting lots of webxdc updates
Currently this leads to
DEBUG    root:rpc.py:136 account_id=1 got an event {'kind': 'Warning', 'msg': 'src/scheduler.rs:711: send_smtp_messages failed: failed to send message: failed to update retries count: database is locked: Error code 5: The database file is locked'}
and
FAILED tests/test_webxdc.py::test_webxdc_insert_lots_of_updates - deltachat_rpc_client.rpc.JsonRpcError: {'code': -1, 'message': 'database is locked\n\nCaused by:\n    Error code 5: The database file is locked'}
2023-12-01 15:19:23 +00:00
link2xt
48d278fca9 chore: update dependencies 2023-12-01 02:41:48 +00:00
link2xt
c84effdaa1 refactor: add more error context to send_webxdc_status_update()
This is a follow-up to b9fa05c3bb
2023-12-01 02:32:21 +00:00
link2xt
e9601ef138 test: make Result-returning tests produce a line number
Without this change
when the test returns a `Result`, `cargo test` does not show
the line number.

To make the tests as easy to debug as the panicking tests,
enable `backtrace` feature on `anyhow` and add debug information
to add source line numbers to backtraces.

Now running `RUST_BACKTRACE=1 cargo test` produces backtraces
with the line numbers. For example:

Error: near ",": syntax error in SELECT COUNT(,,*) FROM msgs_status_updates; at offset 13

Caused by:
    Error code 1: SQL error or missing database

Stack backtrace:
   0: <core::result::Result<T,F> as core::ops::try_trait::FromResidual<core::result::Result<core::convert::Infallible,E>>>::from_residual
             at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library/core/src/result.rs:1962:27
   1: deltachat::sql::Sql::query_row::{{closure}}::{{closure}}
             at ./src/sql.rs:466:23
   2: deltachat::sql::Sql::call::{{closure}}::{{closure}}
             at ./src/sql.rs:379:55
   3: tokio::runtime::context::runtime_mt::exit_runtime
             at /home/user/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.34.0/src/runtime/context/runtime_mt.rs:35:5
   4: tokio::runtime::scheduler::multi_thread::worker::block_in_place
             at /home/user/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.34.0/src/runtime/scheduler/multi_thread/worker.rs:438:9
   5: tokio::runtime::scheduler::block_in_place::block_in_place
             at /home/user/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.34.0/src/runtime/scheduler/block_in_place.rs:20:5
   6: tokio::task::blocking::block_in_place
             at /home/user/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.34.0/src/task/blocking.rs:78:9
   7: deltachat::sql::Sql::call::{{closure}}
             at ./src/sql.rs:379:19
   8: deltachat::sql::Sql::query_row::{{closure}}
             at ./src/sql.rs:469:10
   9: deltachat::sql::Sql::count::{{closure}}
             at ./src/sql.rs:443:76
  10: deltachat::webxdc::tests::change_logging_webxdc::{{closure}}
             at ./src/webxdc.rs:2644:18
  11: <core::pin::Pin<P> as core::future::future::Future>::poll
             at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library/core/src/future/future.rs:125:9
  12: tokio::runtime::park::CachedParkThread::block_on::{{closure}}
             at /home/user/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.34.0/src/runtime/park.rs:282:63
  13: tokio::runtime::coop::with_budget
             at /home/user/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.34.0/src/runtime/coop.rs:107:5
      tokio::runtime::coop::budget
             at /home/user/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.34.0/src/runtime/coop.rs:73:5
      tokio::runtime::park::CachedParkThread::block_on
             at /home/user/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.34.0/src/runtime/park.rs:282:31
  14: tokio::runtime::context::blocking::BlockingRegionGuard::block_on
             at /home/user/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.34.0/src/runtime/context/blocking.rs:66:9
...

Line 10 of the backtrace contains the line number in the test (2644).
2023-11-30 22:27:38 +00:00
iequidoo
44c5cd5526 feat: Ratelimit IMAP connections (#4940)
Limit the number of IMAP connections to 1 per minute regardless of the reason of reconnection, but
allow one immediate retry. This is more reliable than ratelimiting only in error conditions because
ratelimiting can't be skipped by mistake. Anyway connections shouldn't be frequent in normal
operation mode.
2023-11-30 19:22:01 -03:00
link2xt
1c9662a8f2 refactor: rename min_verified into verified 2023-11-30 12:04:03 +00:00
link2xt
5d08b2ce33 refactor: remove unused PeerstateVerifiedStatus 2023-11-30 12:04:03 +00:00
link2xt
bb9d7d7ef3 feat: send Chat-Verified headers in 1:1 chats
Chat-Verified is going to be useful to upgrade one-way verification
to bidirectional verification.
2023-11-30 12:04:03 +00:00
link2xt
766bb5c8aa refactor: factor securejoin processing out of add_parts 2023-11-30 12:04:03 +00:00
link2xt
84144659cf refactor: remove {vc-contact-confirm,vg-member-added}-received steps 2023-11-30 12:04:03 +00:00
link2xt
1394137436 refactor: make min_verified a boolean
We either need a securejoin or autocrypt key,
there are no intermediate states.
2023-11-30 12:04:03 +00:00
link2xt
998614b923 api: make Contact.is_verified() return bool 2023-11-30 12:04:03 +00:00
B. Petersen
5b346397b8 api: deprecate CFFI APIs dc_send_reaction(), dc_get_msg_reactions(), dc_reactions_get_contacts(), dc_reactions_get_by_contact_id(), dc_reactions_unref and dc_reactions_t
this is now done with jsonrpc via
`dc_jsonrpc_request()` or `dc_jsonrpc_blocking_call()`
using the methods `send_reaction` and `get_message_reactions`
2023-11-29 11:29:29 +01:00
B. Petersen
1f99269002 api: remove dc_get_http_response(), dc_http_response_get_mimetype(), dc_http_response_get_encoding(), dc_http_response_get_blob(), dc_http_response_get_size(), dc_http_response_unref() and dc_http_response_t from cffi
this is now done with jsonrpc via
`dc_jsonrpc_request()` or `dc_jsonrpc_blocking_call()`
using the method `get_http_response`
2023-11-29 11:29:29 +01:00
iequidoo
160cbe8125 fix: Use keyring with all private keys when decrypting a message (#5046)
Before a keyring with the only default key was used, i.e. the key used for signing and encrypting to
self.
2023-11-29 02:20:19 -03:00
link2xt
b9fa05c3bb refactor: improve logging of send_webxdc_status_update errors
send_webxdc_status_update JSON-RPC call
and corresponding Rust call sometimes fail in CI with
---
database is locked

Caused by:
    Error code 5: The database file is locked
---

Adding more context to send_webxdc_status_update() errors
to better localize the error origin.
2023-11-28 22:48:55 +00:00
link2xt
4287a4d3ad refactor: factor out insert_tombstone 2023-11-28 21:57:41 +00:00
link2xt
37d2aafb26 fix: return correct MsgId for malformed message tombstone
.execute() returns the number of affected rows,
in this case it is always 1 and MsgId(1) is returned
instead of the actual tombstone row ID.
2023-11-28 21:57:41 +00:00
link2xt
4332170691 ci: add exception for RUSTSEC-2023-0071 to cargo-deny config
See
<https://rustsec.org/advisories/RUSTSEC-2023-0071>
and discussion at
<https://github.com/RustCrypto/RSA/issues/19>
for details.
2023-11-28 16:26:11 +00:00
link2xt
9a7c0f4737 chore: update OpenSSL 2023-11-28 15:05:34 +00:00
link2xt
9e7e172a7b build: switch from fork of iroh to iroh 0.4.2 pre-release 2023-11-28 02:59:42 +00:00
link2xt
71fbaf572a chore(release): prepare for 1.131.8 2023-11-28 00:01:17 +00:00
link2xt
2ab29e5bfa fix: allow IMAP servers not returning UIDNEXT on SELECT and STATUS 2023-11-27 23:50:43 +00:00
link2xt
85f8f910b9 chore: update wasm-bindgen from 0.2.88 to 0.2.89
0.2.88 is yanked: https://github.com/rustwasm/wasm-bindgen/issues/3685
2023-11-27 21:41:23 +00:00
link2xt
b779d08d7f test: check that receive_status_update has forward compatibility
This ensures old version of Delta Chat will be fine with a new "uid" field.
2023-11-27 13:49:41 +00:00
link2xt
3b5634f14b fix: do not emit events about webxdc update events logged into debug log webxdc 2023-11-27 13:49:41 +00:00
link2xt
f91ba357cf feat(webxdc): add unique IDs to status updates sent outside
This allows for deduplication
if status updates are sent over multiple transports.
2023-11-27 13:49:41 +00:00
Hocuri
616faff96b fix: Use the correct securejoin strings used in the UI, remove old TODO (#5047) 2023-11-26 15:54:11 +01:00
link2xt
5e6869403e chore(release): prepare for 1.131.7 2023-11-24 18:26:25 +00:00
link2xt
7ff7d82959 Revert "fix: check UIDNEXT with a STATUS command before going IDLE"
This reverts commit 2e50abedaa.

STATUS is broken on mail.163.com.
It returns `STATUS "INBOX" ()` reply
when `STATUS "INBOX" (UIDNEXT)` is requested.
2023-11-24 18:19:02 +00:00
link2xt
9b751c1865 chore(release): prepare for 1.131.6 2023-11-21 21:05:29 +00:00
link2xt
d1d31096e0 fix: fail fast if IMAP FETCH cannot be parsed
Otherwise when connection is lost IMAP may get into infinite loop
trying to parse remaining bytes:
11-21 18:00:48.442 14858 12946 W DeltaChat: src/imap.rs:1457: Failed to process IMAP FETCH result: io: bytes remaining in stream.
11-21 18:00:48.442 14858 12946 W DeltaChat: src/imap.rs:1457: Failed to process IMAP FETCH result: io: bytes remaining in stream.
11-21 18:00:48.442 14858 12946 W DeltaChat: src/imap.rs:1457: Failed to process IMAP FETCH result: io: bytes remaining in stream.
11-21 18:00:48.442 14858 12946 W DeltaChat: src/imap.rs:1457: Failed to process IMAP FETCH result: io: bytes remaining in stream.
11-21 18:00:48.442 14858 12946 W DeltaChat: src/imap.rs:1457: Failed to process IMAP FETCH result: io: bytes remaining in stream.
11-21 18:00:48.442 14858 12946 W DeltaChat: src/imap.rs:1457: Failed to process IMAP FETCH result: io: bytes remaining in stream.

Returning an error bubbles it up to `fetch_idle()`
which will call `trigger_reconnect()` and drop the connection.
2023-11-21 20:59:16 +00:00
97 changed files with 5309 additions and 2739 deletions

View File

@@ -24,7 +24,7 @@ jobs:
name: Lint Rust
runs-on: ubuntu-latest
env:
RUSTUP_TOOLCHAIN: 1.74.0
RUSTUP_TOOLCHAIN: 1.75.0
steps:
- uses: actions/checkout@v3
- name: Install rustfmt and clippy
@@ -76,11 +76,11 @@ jobs:
matrix:
include:
- os: ubuntu-latest
rust: 1.74.0
rust: 1.75.0
- os: windows-latest
rust: 1.74.0
rust: 1.75.0
- os: macos-latest
rust: 1.74.0
rust: 1.75.0
# Minimum Supported Rust Version = 1.70.0
- os: ubuntu-latest

View File

@@ -1,5 +1,219 @@
# Changelog
## [1.134.0] - 2024-01-31
### API-Changes
- [**breaking**] JSON-RPC: device message api now requires `Option<MessageData>` instead of `String` for the message ([#5211](https://github.com/deltachat/deltachat-core-rust/pull/5211)).
- CFFI: add `dc_accounts_background_fetch` and event `DC_EVENT_ACCOUNTS_BACKGROUND_FETCH_DONE`.
- JSON-RPC: add `accounts_background_fetch`.
### Features / Changes
- `Qr::check_qr()`: Accept i.delta.chat invite links ([#5217](https://github.com/deltachat/deltachat-core-rust/pull/5217)).
- Add support for IMAP METADATA, fetching `/shared/comment` and `/shared/admin` and displaying it in account info.
### Fixes
- Add tolerance for macOS and iOS changing `#` to `%23`.
- Do not drop unknown report attachments, such as TLS reports.
- Treat only "Auto-Submitted: auto-generated" messages as bot-sent ([#5213](https://github.com/deltachat/deltachat-core-rust/pull/5213)).
- `Chat::resend_msgs`: Guarantee strictly increasing time in the `Date` header.
- Delete resent messages on receiver side ([#5155](https://github.com/deltachat/deltachat-core-rust/pull/5155)).
- Fix iOS build issue.
### CI
- Add/remove necessary newlines to fix Python lint.
### Tests
- `test_import_export_online_all`: Send the message to the existing address to avoid errors ([#5220](https://github.com/deltachat/deltachat-core-rust/pull/5220)).
## [1.133.2] - 2024-01-24
### Fixes
- Downgrade OpenSSL from 3.2.0 to 3.1.4 ([#5206](https://github.com/deltachat/deltachat-core-rust/issues/5206))
- No new chats for MDNs with alias ([#5196](https://github.com/deltachat/deltachat-core-rust/issues/5196)) ([#5199](https://github.com/deltachat/deltachat-core-rust/pull/5199)).
## [1.133.1] - 2024-01-21
### API-Changes
- Add `is_bot` to cffi and jsonrpc ([#5197](https://github.com/deltachat/deltachat-core-rust/pull/5197)).
### Features / Changes
- Add system message when provider does not allow unencrypted messages ([#5195](https://github.com/deltachat/deltachat-core-rust/pull/5195)).
### Fixes
- `Chat::send_msg`: Remove encryption-related params from already sent message. This allows to send received encrypted `dc_msg_t` object to unencrypted chat, e.g. in a Python bot.
- Set message download state to Failure on IMAP errors. This avoids partially downloaded messages getting stuck in "Downloading..." state without actually being in a download queue.
- BCC-to-self even if server deletion is set to "at once". This is a workaround for SMTP servers which do not return response in time, BCC-self works as a confirmation that message was sent out successfully and does not need more retries.
- node: Run tests with native ESM modules instead of `esm` ([#5194](https://github.com/deltachat/deltachat-core-rust/pull/5194)).
- Use Quoted-Printable MIME encoding for the text part ([#3986](https://github.com/deltachat/deltachat-core-rust/pull/3986)).
### Tests
- python: Add `get_protected_chat` to testplugin.py.
## [1.133.0] - 2024-01-14
### Features / Changes
- Securejoin protocol implementation refinements
- Track forward and backward verification separately ([#5089](https://github.com/deltachat/deltachat-core-rust/pull/5089)) to avoid inconsistent states.
- Mark 1:1 chat as verified for Bob early. 1:1 chat with Alice is verified as soon as Alice's key is verified rather than at the end of the protocol.
- Put Message-ID into hidden headers and take it from there on receiver ([#4798](https://github.com/deltachat/deltachat-core-rust/pull/4798)). This works around servers which generate their own Message-ID and overwrite the one generated by Delta Chat.
- deltachat-repl: Enable INFO logging by default and add timestamps.
- Add `ConfigSynced` (`DC_EVENT_CONFIG_SYNCED`) event which is emitted when configuration is changed via synchronization message or synchronization message for configuration is sent. UI may refresh elments based on the configuration key which is a part of the event.
- Sync contact creation/rename across devices ([#5163](https://github.com/deltachat/deltachat-core-rust/pull/5163)).
- Encrypt MDNs ([#5175](https://github.com/deltachat/deltachat-core-rust/pull/5175)).
- Only try to configure non-strict TLS checks if explicitly set ([#5181](https://github.com/deltachat/deltachat-core-rust/pull/5181)).
### Build system
- Use released version of iroh 0.4.2 for "setup second device" feature.
### CI
- Update to Rust 1.75.0.
- Downgrade `chai` from 4.4.0 to 4.3.10.
### Documentation
- Add a link <https://www.ietf.org/archive/id/draft-bucksch-autoconfig-00.html> to autoconfig RFC draft.
- Update securejoin link in `standards.md` from <https://countermitm.readthedocs.io/> to <https://securejoin.readthedocs.io>.
- Restore "Constants" page in Doxygen >=1.9.8
### Fixes
- imap: Limit the rate of LOGIN attempts rather than connection attempts. This is to avoid having to wait for rate limiter right after switching from a bad or offline network to a working network while still guarding against reconnection loop.
- Do not ignore `peerstate.save_to_db()` errors.
- securejoin: Mark 1:1s as protected regardless of the Config::VerifiedOneOnOneChats.
- Delete received outgoing messages from SMTP queue ([#5115](https://github.com/deltachat/deltachat-core-rust/pull/5115)).
- imap: Fail fast on `LIST` errors to avoid busy loop when connection is lost.
- Split SMTP jobs already in `chat::create_send_msg_jobs()` ([#5115](https://github.com/deltachat/deltachat-core-rust/pull/5115)).
- Do not remove contents from unencrypted [Schleuder](https://schleuder.org/) mailing lists messages.
- Reset message error when scheduling resending ([#5119](https://github.com/deltachat/deltachat-core-rust/pull/5119)).
- Emit events more reliably when starting and stopping I/O ([#5101](https://github.com/deltachat/deltachat-core-rust/pull/5101)).
- Fix timestamp of chat protection info message for correct message ordering after restoring a backup ([#5088](https://github.com/deltachat/deltachat-core-rust/pull/5088)).
### Refactor
- sql: Recreate `config` table with UNIQUE constraint.
- sql: Recreate `keypairs` table to remove unused `addr` and `created` fields and move `is_default` flag to `config` table.
- Send `Secure-Join-Fingerprint` only in `*-request-with-auth`.
### Tests
- Test joining non-protected group.
- Test that read receipts don't degrade encryption.
- Test that changing default private key breaks backward verification.
- Test recovery from lost vc-contact-confirm.
- Use `wait_for_incoming_msg_event()` more.
## [1.132.1] - 2023-12-12
### Features / Changes
- Add "From:" to protected headers for signed-only messages.
- Sync user actions for ad-hoc groups across devices ([#5065](https://github.com/deltachat/deltachat-core-rust/pull/5065)).
### Fixes
- Add padlock to empty part if the whole message is empty.
- Renew IDLE timeout on keepalives and reduce it to 5 minutes.
- connectivity: Return false from `all_work_done()` immediately after connecting (iOS notification fix).
### API-Changes
- deltachat-jsonrpc-client: add `Account.{import,export}_self_keys`.
### CI
- Update to Rust 1.74.1.
## [1.132.0] - 2023-12-06
### Features / Changes
- Increase TCP timeouts from 30 to 60 seconds.
### Fixes
- Don't sort message creating a protected group over a protection message ([#4963](https://github.com/deltachat/deltachat-core-rust/pull/4963)).
- Do not lock accounts.toml on iOS.
- Protect groups even if some members are not verified and add `test_securejoin_after_contact_resetup` regression test.
## [1.131.9] - 2023-12-02
### API-Changes
- Remove `dc_get_http_response()`, `dc_http_response_get_mimetype()`, `dc_http_response_get_encoding()`, `dc_http_response_get_blob()`, `dc_http_response_get_size()`, `dc_http_response_unref()` and `dc_http_response_t` from cffi.
- Deprecate CFFI APIs `dc_send_reaction()`, `dc_get_msg_reactions()`, `dc_reactions_get_contacts()`, `dc_reactions_get_by_contact_id()`, `dc_reactions_unref` and `dc_reactions_t`.
- Make `Contact.is_verified()` return bool.
### Build system
- Switch from fork of iroh to iroh 0.4.2 pre-release.
### Features / Changes
- Send `Chat-Verified` headers in 1:1 chats.
- Ratelimit IMAP connections ([#4940](https://github.com/deltachat/deltachat-core-rust/pull/4940)).
- Remove receiver limit on `.xdc` size.
- Don't affect MimeMessage with "From" and secured headers from encrypted unsigned messages.
- Sync `Config::{MdnsEnabled,ShowEmails}` across devices ([#4954](https://github.com/deltachat/deltachat-core-rust/pull/4954)).
- Sync `Config::Displayname` across devices ([#4893](https://github.com/deltachat/deltachat-core-rust/pull/4893)).
- `Chat::rename_ex`: Don't send sync message if usual message is sent.
### Fixes
- Lock the database when INSERTing a webxdc update, avoid "Database is locked" errors.
- Use keyring with all private keys when decrypting a message ([#5046](https://github.com/deltachat/deltachat-core-rust/pull/5046)).
### Tests
- Make Result-returning tests produce a line number.
- Add `test_utils::sync()`.
- Test inserting lots of webxdc updates.
- Split `test_sync_alter_chat()` into smaller tests.
## [1.131.8] - 2023-11-27
### Features / Changes
- webxdc: Add unique IDs to status updates sent outside and deduplicate based on IDs.
### Fixes
- Allow IMAP servers not returning UIDNEXT on SELECT and STATUS such as mail.163.com.
- Use the correct securejoin strings used in the UI, remove old TODO ([#5047](https://github.com/deltachat/deltachat-core-rust/pull/5047)).
- Do not emit events about webxdc update events logged into debug log webxdc.
### Tests
- Check that `receive_status_update` has forward compatibility and unique webxdc IDs will be ignored by previous Delta Chat versions.
## [1.131.7] - 2023-11-24
### Fixes
- Revert "fix: check UIDNEXT with a STATUS command before going IDLE". This attempts to fix mail.163.com which has broken STATUS command.
## [1.131.6] - 2023-11-21
### Fixes
- Fail fast if IMAP FETCH cannot be parsed instead of getting stuck in infinite loop.
### Documentation
- Generate deltachat-rpc-client documentation and publish it to <https://py.delta.chat>.
## [1.131.5] - 2023-11-20
### API-Changes
@@ -3252,3 +3466,13 @@ https://github.com/deltachat/deltachat-core-rust/pulls?q=is%3Apr+is%3Aclosed
[1.131.3]: https://github.com/deltachat/deltachat-core-rust/compare/v1.131.2...v1.131.3
[1.131.4]: https://github.com/deltachat/deltachat-core-rust/compare/v1.131.3...v1.131.4
[1.131.5]: https://github.com/deltachat/deltachat-core-rust/compare/v1.131.4...v1.131.5
[1.131.6]: https://github.com/deltachat/deltachat-core-rust/compare/v1.131.5...v1.131.6
[1.131.7]: https://github.com/deltachat/deltachat-core-rust/compare/v1.131.6...v1.131.7
[1.131.8]: https://github.com/deltachat/deltachat-core-rust/compare/v1.131.7...v1.131.8
[1.131.9]: https://github.com/deltachat/deltachat-core-rust/compare/v1.131.8...v1.131.9
[1.132.0]: https://github.com/deltachat/deltachat-core-rust/compare/v1.131.9...v1.132.0
[1.132.1]: https://github.com/deltachat/deltachat-core-rust/compare/v1.132.0...v1.132.1
[1.133.0]: https://github.com/deltachat/deltachat-core-rust/compare/v1.132.1...v1.133.0
[1.133.1]: https://github.com/deltachat/deltachat-core-rust/compare/v1.133.0...v1.133.1
[1.133.2]: https://github.com/deltachat/deltachat-core-rust/compare/v1.133.1...v1.133.2
[1.134.0]: https://github.com/deltachat/deltachat-core-rust/compare/v1.133.2...v1.134.0

View File

@@ -91,6 +91,12 @@ All errors should be handled in one of these ways:
- With `.log_err().ok()`.
- Bubbled up with `?`.
`backtrace` feature is enabled for `anyhow` crate
and `debug = 1` option is set in the test profile.
This allows to run `RUST_BACKTRACE=1 cargo test`
and get a backtrace with line numbers in resultified tests
which return `anyhow::Result`.
### Logging
For logging, use `info!`, `warn!` and `error!` macros.

864
Cargo.lock generated

File diff suppressed because it is too large Load Diff

View File

@@ -1,6 +1,6 @@
[package]
name = "deltachat"
version = "1.131.5"
version = "1.134.0"
edition = "2021"
license = "MPL-2.0"
rust-version = "1.70"
@@ -11,6 +11,10 @@ panic = 'abort'
opt-level = 1
[profile.test]
# Make anyhow `backtrace` feature useful.
# With `debug = 0` there are no line numbers in the backtrace
# produced with RUST_BACKTRACE=1.
debug = 1
opt-level = 0
# Always optimize dependencies.
@@ -33,7 +37,7 @@ ratelimit = { path = "./deltachat-ratelimit" }
anyhow = "1"
async-channel = "2.0.0"
async-imap = { version = "0.9.1", default-features = false, features = ["runtime-tokio"] }
async-imap = { version = "0.9.7", default-features = false, features = ["runtime-tokio"] }
async-native-tls = { version = "0.5", default-features = false, features = ["runtime-tokio"] }
async-smtp = { version = "0.9", default-features = false, features = ["runtime-tokio"] }
async_zip = { version = "0.0.12", default-features = false, features = ["deflate", "fs"] }
@@ -44,15 +48,15 @@ chrono = { version = "0.4", default-features=false, features = ["clock", "std"]
email = { git = "https://github.com/deltachat/rust-email", branch = "master" }
encoded-words = { git = "https://github.com/async-email/encoded-words", branch = "master" }
escaper = "0.1"
fast-socks5 = "0.8"
fd-lock = "3.0.11"
fast-socks5 = "0.9"
fd-lock = "4"
futures = "0.3"
futures-lite = "2.0.0"
hex = "0.4.0"
hickory-resolver = "0.24"
humansize = "2"
image = { version = "0.24.7", default-features=false, features = ["gif", "jpeg", "ico", "png", "pnm", "webp", "bmp"] }
iroh = { git = "https://github.com/deltachat/iroh", branch = "0.4-update-quic", default-features = false }
iroh = { version = "0.4.2", default-features = false }
kamadak-exif = "0.5"
lettre_email = { git = "https://github.com/deltachat/lettre", branch = "master" }
libc = "0.2"
@@ -69,13 +73,14 @@ pin-project = "1"
pretty_env_logger = { version = "0.5", optional = true }
qrcodegen = "1.7.0"
quick-xml = "0.31"
quoted_printable = "0.5"
rand = "0.8"
regex = "1.9"
reqwest = { version = "0.11.20", features = ["json"] }
reqwest = { version = "0.11.23", features = ["json"] }
rusqlite = { version = "0.30", features = ["sqlcipher"] }
rust-hsluv = "0.1"
sanitize-filename = "0.5"
serde_json = "1.0"
serde_json = "1"
serde = { version = "1.0", features = ["derive"] }
sha-1 = "0.10"
sha2 = "0.10"
@@ -94,15 +99,24 @@ toml = "0.8"
url = "2"
uuid = { version = "1", features = ["serde", "v4"] }
# Pin OpenSSL to 3.1 releases.
# OpenSSL 3.2 has a regression tracked at <https://github.com/openssl/openssl/issues/23376>
# which results in broken `deltachat-rpc-server` binaries when cross-compiled using Zig toolchain.
# See <https://github.com/deltachat/deltachat-core-rust/issues/5206> for Delta Chat issue.
# According to <https://www.openssl.org/policies/releasestrat.html>
# 3.1 branch will be supported until 2025-03-14.
openssl-src = "~300.1"
[dev-dependencies]
ansi_term = "0.12.0"
anyhow = { version = "1", features = ["backtrace"] } # Enable `backtrace` feature in tests.
criterion = { version = "0.5.1", features = ["async_tokio"] }
futures-lite = "2.0.0"
log = "0.4"
pretty_env_logger = "0.5"
proptest = { version = "1", default-features = false, features = ["std"] }
tempfile = "3"
testdir = "0.8.0"
testdir = "0.9.0"
tokio = { version = "1", features = ["parking_lot", "rt-multi-thread", "macros"] }
pretty_assertions = "1.3.0"

View File

@@ -27,7 +27,7 @@ $ curl https://sh.rustup.rs -sSf | sh
Compile and run Delta Chat Core command line utility, using `cargo`:
```
$ RUST_LOG=deltachat_repl=info cargo run -p deltachat-repl -- ~/deltachat-db
$ cargo run -p deltachat-repl -- ~/deltachat-db
```
where ~/deltachat-db is the database file. Delta Chat will create it if it does not exist.
@@ -121,7 +121,7 @@ $ cargo build -p deltachat_ffi --release
- `DCC_MIME_DEBUG`: if set outgoing and incoming message will be printed
- `RUST_LOG=deltachat_repl=info,async_imap=trace,async_smtp=trace`: enable IMAP and
- `RUST_LOG=async_imap=trace,async_smtp=trace`: enable IMAP and
SMTP tracing in addition to info messages.
### Expensive tests

View File

@@ -1,6 +1,6 @@
[package]
name = "deltachat_ffi"
version = "1.131.5"
version = "1.134.0"
description = "Deltachat FFI"
edition = "2018"
readme = "README.md"

View File

@@ -9,7 +9,7 @@
<tab type="hierarchy" visible="no" title="" intro=""/>
<tab type="classmembers" visible="no" title="" intro=""/>
</tab>
<tab type="modules" visible="yes" title="Constants" intro="Here is a list of constants:"/>
<tab type="topics" visible="yes" title="Constants" intro="Here is a list of constants:"/>
<tab type="pages" visible="yes" title="" intro=""/>
<tab type="namespaces" visible="yes" title="">
<tab type="namespacelist" visible="yes" title="" intro=""/>

View File

@@ -25,7 +25,6 @@ typedef struct _dc_event dc_event_t;
typedef struct _dc_event_emitter dc_event_emitter_t;
typedef struct _dc_jsonrpc_instance dc_jsonrpc_instance_t;
typedef struct _dc_backup_provider dc_backup_provider_t;
typedef struct _dc_http_response dc_http_response_t;
// Alias for backwards compatibility, use dc_event_emitter_t instead.
typedef struct _dc_event_emitter dc_accounts_event_emitter_t;
@@ -1112,6 +1111,7 @@ uint32_t dc_send_videochat_invitation (dc_context_t* context, uint32_t chat_id);
* received overrides all previously received reactions. It is
* possible to remove all reactions by sending an empty string.
*
* @deprecated 2023-11-27, use jsonrpc method `send_reaction` instead
* @memberof dc_context_t
* @param context The context object.
* @param msg_id ID of the message you react to.
@@ -1124,6 +1124,7 @@ uint32_t dc_send_reaction (dc_context_t* context, uint32_t msg_id, char *reactio
/**
* Get a structure with reactions to the message.
*
* @deprecated 2023-11-27, use jsonrpc method `get_message_reactions` instead
* @memberof dc_context_t
* @param context The context object.
* @param msg_id The message ID to get reactions for.
@@ -3150,6 +3151,23 @@ void dc_accounts_maybe_network (dc_accounts_t* accounts);
void dc_accounts_maybe_network_lost (dc_accounts_t* accounts);
/**
* Perform a background fetch for all accounts in parallel with a timeout.
* Pauses the scheduler, fetches messages from imap and then resumes the scheduler.
*
* dc_accounts_background_fetch() was created for the iOS Background fetch.
*
* The `DC_EVENT_ACCOUNTS_BACKGROUND_FETCH_DONE` event is emitted at the end
* even in case of timeout, unless the function fails and returns 0.
* Process all events until you get this one and you can safely return to the background
* without forgetting to create notifications caused by timing race conditions.
*
* @memberof dc_accounts_t
* @param timeout The timeout in seconds
* @return Return 1 if DC_EVENT_ACCOUNTS_BACKGROUND_FETCH_DONE was emitted and 0 otherwise.
*/
int dc_accounts_background_fetch (dc_accounts_t* accounts, uint64_t timeout);
/**
* Create the event emitter that is used to receive events.
*
@@ -4396,6 +4414,9 @@ int dc_msg_is_info (const dc_msg_t* msg);
* Currently, the following types are defined:
* - DC_INFO_PROTECTION_ENABLED (11) - Info-message for "Chat is now protected"
* - DC_INFO_PROTECTION_DISABLED (12) - Info-message for "Chat is no longer protected"
* - DC_INFO_INVALID_UNENCRYPTED_MAIL (13) - Info-message for "Provider requires end-to-end encryption which is not setup yet",
* the UI should change the corresponding string using #DC_STR_INVALID_UNENCRYPTED_MAIL
* and also offer a way to fix the encryption, eg. by a button offering a QR scan
*
* Even when you display an icon,
* you should still display the text of the informational message using dc_msg_get_text()
@@ -4422,6 +4443,7 @@ int dc_msg_get_info_type (const dc_msg_t* msg);
#define DC_INFO_EPHEMERAL_TIMER_CHANGED 10
#define DC_INFO_PROTECTION_ENABLED 11
#define DC_INFO_PROTECTION_DISABLED 12
#define DC_INFO_INVALID_UNENCRYPTED_MAIL 13
#define DC_INFO_WEBXDC_INFO_MESSAGE 32
/**
@@ -5067,6 +5089,15 @@ int dc_contact_is_blocked (const dc_contact_t* contact);
*/
int dc_contact_is_verified (dc_contact_t* contact);
/**
* Returns whether contact is a bot.
*
* @memberof dc_contact_t
* @param contact The contact object.
* @return 0 if the contact is not a bot, 1 otherwise.
*/
int dc_contact_is_bot (dc_contact_t* contact);
/**
* Return the contact ID that verified a contact.
@@ -5185,72 +5216,6 @@ int dc_provider_get_status (const dc_provider_t* prov
void dc_provider_unref (dc_provider_t* provider);
/**
* Return an HTTP(S) GET response.
* This function can be used to download remote content for HTML emails.
*
* @memberof dc_context_t
* @param context The context object to take proxy settings from.
* @param url HTTP or HTTPS URL.
* @return The response must be released using dc_http_response_unref() after usage.
* NULL is returned on errors.
*/
dc_http_response_t* dc_get_http_response (const dc_context_t* context, const char* url);
/**
* @class dc_http_response_t
*
* An object containing an HTTP(S) GET response.
* Created by dc_get_http_response().
*/
/**
* Returns HTTP response MIME type as a string, e.g. "text/plain" or "text/html".
*
* @memberof dc_http_response_t
* @param response HTTP response as returned by dc_get_http_response().
* @return The string which must be released using dc_str_unref() after usage. May be NULL.
*/
char* dc_http_response_get_mimetype (const dc_http_response_t* response);
/**
* Returns HTTP response encoding, e.g. "utf-8".
*
* @memberof dc_http_response_t
* @param response HTTP response as returned by dc_get_http_response().
* @return The string which must be released using dc_str_unref() after usage. May be NULL.
*/
char* dc_http_response_get_encoding (const dc_http_response_t* response);
/**
* Returns HTTP response contents.
*
* @memberof dc_http_response_t
* @param response HTTP response as returned by dc_get_http_response().
* @return The blob which must be released using dc_str_unref() after usage. NULL is never returned.
*/
uint8_t* dc_http_response_get_blob (const dc_http_response_t* response);
/**
* Returns HTTP response content size.
*
* @memberof dc_http_response_t
* @param response HTTP response as returned by dc_get_http_response().
* @return The blob size.
*/
size_t dc_http_response_get_size (const dc_http_response_t* response);
/**
* Free an HTTP response object.
*
* @memberof dc_http_response_t
* @param response HTTP response as returned by dc_get_http_response().
*/
void dc_http_response_unref (const dc_http_response_t* response);
/**
* @class dc_lot_t
*
@@ -5350,6 +5315,7 @@ int64_t dc_lot_get_timestamp (const dc_lot_t* lot);
/**
* @class dc_reactions_t
* @deprecated 2023-11-27, use jsonrpc method `get_message_reactions` instead
*
* An object representing all reactions for a single message.
*/
@@ -5357,6 +5323,7 @@ int64_t dc_lot_get_timestamp (const dc_lot_t* lot);
/**
* Returns array of contacts which reacted to the given message.
*
* @deprecated 2023-11-27, use jsonrpc method `get_message_reactions` instead
* @memberof dc_reactions_t
* @param reactions The object containing message reactions.
* @return array of contact IDs. Use dc_array_get_cnt() to get array length and
@@ -5368,6 +5335,7 @@ dc_array_t* dc_reactions_get_contacts(dc_reactions_t* reactions);
/**
* Returns a string containing space-separated reactions of a single contact.
*
* @deprecated 2023-11-27, use jsonrpc method `get_message_reactions` instead
* @memberof dc_reactions_t
* @param reactions The object containing message reactions.
* @param contact_id ID of the contact.
@@ -5383,6 +5351,7 @@ char* dc_reactions_get_by_contact_id(dc_reactions_t* reactions, uint32
*
* Reactions objects are created by dc_get_msg_reactions().
*
* @deprecated 2023-11-27
* @memberof dc_reactions_t
* @param reactions The object to free.
* If NULL is given, nothing is done.
@@ -6267,6 +6236,18 @@ void dc_event_unref(dc_event_t* event);
#define DC_EVENT_SELFAVATAR_CHANGED 2110
/**
* A multi-device synced config value changed. Maybe the app needs to refresh smth. For uniformity
* this is emitted on the source device too. The value isn't reported, otherwise it would be logged
* which might not be good for privacy. You can get the new value with
* `dc_get_config(context, data2)`.
*
* @param data1 0
* @param data2 (char*) Configuration key.
*/
#define DC_EVENT_CONFIG_SYNCED 2111
/**
* webxdc status update received.
* To get the received status update, use dc_get_webxdc_status_updates() with
@@ -6291,6 +6272,16 @@ void dc_event_unref(dc_event_t* event);
#define DC_EVENT_WEBXDC_INSTANCE_DELETED 2121
/**
* Tells that the Background fetch was completed (or timed out).
*
* This event acts as a marker, when you reach this event you can be sure
* that all events emitted during the background fetch were processed.
*
* This event is only emitted by the account manager
*/
#define DC_EVENT_ACCOUNTS_BACKGROUND_FETCH_DONE 2200
/**
* @}
@@ -6621,7 +6612,7 @@ void dc_event_unref(dc_event_t* event);
/// - %1$s will be replaced by the name of the verified contact
#define DC_STR_CONTACT_VERIFIED 35
/// "Cannot verify %1$s."
/// "Cannot establish guaranteed end-to-end encryption with %1$s."
///
/// Used in status messages.
/// - %1$s will be replaced by the name of the contact that cannot be verified
@@ -7059,6 +7050,8 @@ void dc_event_unref(dc_event_t* event);
/// "You added member %1$s."
///
/// Used in status messages.
///
/// `%1$s` will be replaced by the added member's name.
#define DC_STR_ADD_MEMBER_BY_YOU 128
/// "Member %1$s added by %2$s."
@@ -7280,6 +7273,21 @@ void dc_event_unref(dc_event_t* event);
/// Used as the first info messages in newly created groups.
#define DC_STR_NEW_GROUP_SEND_FIRST_MESSAGE 172
/// "Member %1$s added."
///
/// Used as info messages.
///
/// `%1$s` will be replaced by the added member's name.
#define DC_STR_MESSAGE_ADD_MEMBER 173
/// "Your email provider %1$s requires end-to-end encryption which is not setup yet."
///
/// Used as info messages when a message cannot be sent because it cannot be encrypted.
///
/// `%1$s` will be replaced by the provider's domain.
#define DC_STR_INVALID_UNENCRYPTED_MAIL 174
/**
* @}
*/

View File

@@ -31,7 +31,6 @@ use deltachat::ephemeral::Timer as EphemeralTimer;
use deltachat::imex::BackupProvider;
use deltachat::key::preconfigure_keypair;
use deltachat::message::MsgId;
use deltachat::net::read_url_blob;
use deltachat::qr_code_generator::{generate_backup_qr, get_securejoin_qr_svg};
use deltachat::reaction::{get_msg_reactions, send_reaction, Reactions};
use deltachat::stock_str::StockMessage;
@@ -557,8 +556,10 @@ pub unsafe extern "C" fn dc_event_get_id(event: *mut dc_event_t) -> libc::c_int
EventType::SecurejoinJoinerProgress { .. } => 2061,
EventType::ConnectivityChanged => 2100,
EventType::SelfavatarChanged => 2110,
EventType::ConfigSynced { .. } => 2111,
EventType::WebxdcStatusUpdate { .. } => 2120,
EventType::WebxdcInstanceDeleted { .. } => 2121,
EventType::AccountsBackgroundFetchDone => 2200,
}
}
@@ -584,8 +585,10 @@ pub unsafe extern "C" fn dc_event_get_data1_int(event: *mut dc_event_t) -> libc:
| EventType::Error(_)
| EventType::ConnectivityChanged
| EventType::SelfavatarChanged
| EventType::ConfigSynced { .. }
| EventType::IncomingMsgBunch { .. }
| EventType::ErrorSelfNotInGroup(_) => 0,
| EventType::ErrorSelfNotInGroup(_)
| EventType::AccountsBackgroundFetchDone => 0,
EventType::MsgsChanged { chat_id, .. }
| EventType::ReactionsChanged { chat_id, .. }
| EventType::IncomingMsg { chat_id, .. }
@@ -644,7 +647,9 @@ pub unsafe extern "C" fn dc_event_get_data2_int(event: *mut dc_event_t) -> libc:
| EventType::ConnectivityChanged
| EventType::WebxdcInstanceDeleted { .. }
| EventType::IncomingMsgBunch { .. }
| EventType::SelfavatarChanged => 0,
| EventType::SelfavatarChanged
| EventType::AccountsBackgroundFetchDone
| EventType::ConfigSynced { .. } => 0,
EventType::ChatModified(_) => 0,
EventType::MsgsChanged { msg_id, .. }
| EventType::ReactionsChanged { msg_id, .. }
@@ -706,6 +711,7 @@ pub unsafe extern "C" fn dc_event_get_data2_str(event: *mut dc_event_t) -> *mut
| EventType::SelfavatarChanged
| EventType::WebxdcStatusUpdate { .. }
| EventType::WebxdcInstanceDeleted { .. }
| EventType::AccountsBackgroundFetchDone
| EventType::ChatEphemeralTimerModified { .. } => ptr::null_mut(),
EventType::ConfigureProgress { comment, .. } => {
if let Some(comment) = comment {
@@ -723,6 +729,10 @@ pub unsafe extern "C" fn dc_event_get_data2_str(event: *mut dc_event_t) -> *mut
.to_c_string()
.unwrap_or_default()
.into_raw(),
EventType::ConfigSynced { key } => {
let data2 = key.to_string().to_c_string().unwrap_or_default();
data2.into_raw()
}
}
}
@@ -4113,10 +4123,26 @@ pub unsafe extern "C" fn dc_contact_is_verified(contact: *mut dc_contact_t) -> l
let ffi_contact = &*contact;
let ctx = &*ffi_contact.context;
block_on(ffi_contact.contact.is_verified(ctx))
if block_on(ffi_contact.contact.is_verified(ctx))
.context("is_verified failed")
.log_err(ctx)
.unwrap_or_default() as libc::c_int
.unwrap_or_default()
{
// Return value is essentially a boolean,
// but we return 2 for true for backwards compatibility.
2
} else {
0
}
}
#[no_mangle]
pub unsafe extern "C" fn dc_contact_is_bot(contact: *mut dc_contact_t) -> libc::c_int {
if contact.is_null() {
eprintln!("ignoring careless call to dc_contact_is_bot()");
return 0;
}
(*contact).contact.is_bot() as libc::c_int
}
#[no_mangle]
@@ -4590,96 +4616,6 @@ pub unsafe extern "C" fn dc_provider_unref(provider: *mut dc_provider_t) {
// this may change once we start localizing string.
}
// dc_http_response_t
pub type dc_http_response_t = net::HttpResponse;
#[no_mangle]
pub unsafe extern "C" fn dc_get_http_response(
context: *const dc_context_t,
url: *const libc::c_char,
) -> *mut dc_http_response_t {
if context.is_null() || url.is_null() {
eprintln!("ignoring careless call to dc_get_http_response()");
return ptr::null_mut();
}
let context = &*context;
let url = to_string_lossy(url);
if let Ok(response) = block_on(read_url_blob(context, &url))
.context("read_url_blob")
.log_err(context)
{
Box::into_raw(Box::new(response))
} else {
ptr::null_mut()
}
}
#[no_mangle]
pub unsafe extern "C" fn dc_http_response_get_mimetype(
response: *const dc_http_response_t,
) -> *mut libc::c_char {
if response.is_null() {
eprintln!("ignoring careless call to dc_http_response_get_mimetype()");
return ptr::null_mut();
}
let response = &*response;
response.mimetype.strdup()
}
#[no_mangle]
pub unsafe extern "C" fn dc_http_response_get_encoding(
response: *const dc_http_response_t,
) -> *mut libc::c_char {
if response.is_null() {
eprintln!("ignoring careless call to dc_http_response_get_encoding()");
return ptr::null_mut();
}
let response = &*response;
response.encoding.strdup()
}
#[no_mangle]
pub unsafe extern "C" fn dc_http_response_get_blob(
response: *const dc_http_response_t,
) -> *mut libc::c_char {
if response.is_null() {
eprintln!("ignoring careless call to dc_http_response_get_blob()");
return ptr::null_mut();
}
let response = &*response;
let blob_len = response.blob.len();
let ptr = libc::malloc(blob_len);
libc::memcpy(ptr, response.blob.as_ptr() as *mut libc::c_void, blob_len);
ptr as *mut libc::c_char
}
#[no_mangle]
pub unsafe extern "C" fn dc_http_response_get_size(
response: *const dc_http_response_t,
) -> libc::size_t {
if response.is_null() {
eprintln!("ignoring careless call to dc_http_response_get_size()");
return 0;
}
let response = &*response;
response.blob.len()
}
#[no_mangle]
pub unsafe extern "C" fn dc_http_response_unref(response: *mut dc_http_response_t) {
if response.is_null() {
eprintln!("ignoring careless call to dc_http_response_unref()");
return;
}
drop(Box::from_raw(response));
}
// -- Accounts
/// Reader-writer lock wrapper for accounts manager to guarantee thread safety when using
@@ -4966,6 +4902,26 @@ pub unsafe extern "C" fn dc_accounts_maybe_network_lost(accounts: *mut dc_accoun
block_on(async move { accounts.write().await.maybe_network_lost().await });
}
#[no_mangle]
pub unsafe extern "C" fn dc_accounts_background_fetch(
accounts: *mut dc_accounts_t,
timeout_in_seconds: u64,
) -> libc::c_int {
if accounts.is_null() || timeout_in_seconds <= 2 {
eprintln!("ignoring careless call to dc_accounts_background_fetch()");
return 0;
}
let accounts = &*accounts;
block_on(async move {
let accounts = accounts.read().await;
accounts
.background_fetch(Duration::from_secs(timeout_in_seconds))
.await;
});
1
}
#[no_mangle]
pub unsafe extern "C" fn dc_accounts_get_event_emitter(
accounts: *mut dc_accounts_t,

View File

@@ -1,6 +1,6 @@
[package]
name = "deltachat-jsonrpc"
version = "1.131.5"
version = "1.134.0"
description = "DeltaChat JSON-RPC API"
edition = "2021"
default-run = "deltachat-jsonrpc-server"
@@ -17,11 +17,11 @@ deltachat = { path = ".." }
num-traits = "0.2"
schemars = "0.8.13"
serde = { version = "1.0", features = ["derive"] }
tempfile = "3.8.0"
tempfile = "3.9.0"
log = "0.4"
async-channel = { version = "2.0.0" }
futures = { version = "0.3.28" }
serde_json = "1.0.105"
futures = { version = "0.3.30" }
serde_json = "1"
yerpc = { version = "0.5.2", features = ["anyhow_expose", "openrpc"] }
typescript-type-def = { version = "0.5.8", features = ["json_value"] }
tokio = { version = "1.33.0" }
@@ -30,7 +30,7 @@ walkdir = "2.3.3"
base64 = "0.21"
# optional dependencies
axum = { version = "0.6.20", optional = true, features = ["ws"] }
axum = { version = "0.7", optional = true, features = ["ws"] }
env_logger = { version = "0.10.0", optional = true }
[dev-dependencies]

View File

@@ -231,6 +231,20 @@ impl CommandApi {
Ok(())
}
/// Performs a background fetch for all accounts in parallel with a timeout.
///
/// The `AccountsBackgroundFetchDone` event is emitted at the end even in case of timeout.
/// Process all events until you get this one and you can safely return to the background
/// without forgetting to create notifications caused by timing race conditions.
async fn accounts_background_fetch(&self, timeout_in_seconds: f64) -> Result<()> {
self.accounts
.write()
.await
.background_fetch(std::time::Duration::from_secs_f64(timeout_in_seconds))
.await;
Ok(())
}
// ---------------------------------------------
// Methods that work on individual accounts
// ---------------------------------------------
@@ -896,19 +910,35 @@ impl CommandApi {
.to_u32())
}
// for now only text messages, because we only used text messages in desktop thusfar
/// Add a message to the device-chat.
/// Device-messages usually contain update information
/// and some hints that are added during the program runs, multi-device etc.
/// The device-message may be defined by a label;
/// if a message with the same label was added or skipped before,
/// the message is not added again, even if the message was deleted in between.
/// If needed, the device-chat is created before.
///
/// Sends the `MsgsChanged` event on success.
///
/// Setting msg to None will prevent the device message with this label from being added in the future.
async fn add_device_message(
&self,
account_id: u32,
label: String,
text: String,
) -> Result<u32> {
msg: Option<MessageData>,
) -> Result<Option<u32>> {
let ctx = self.get_context(account_id).await?;
let mut msg = Message::new(Viewtype::Text);
msg.set_text(text);
let message_id =
deltachat::chat::add_device_msg(&ctx, Some(&label), Some(&mut msg)).await?;
Ok(message_id.to_u32())
if let Some(msg) = msg {
let mut message = msg.create_message(&ctx).await?;
let message_id =
deltachat::chat::add_device_msg(&ctx, Some(&label), Some(&mut message)).await?;
if !message_id.is_unset() {
return Ok(Some(message_id.to_u32()));
}
} else {
deltachat::chat::add_device_msg(&ctx, Some(&label), None).await?;
}
Ok(None)
}
/// Mark all messages in a chat as _noticed_.
@@ -1808,38 +1838,7 @@ impl CommandApi {
async fn send_msg(&self, account_id: u32, chat_id: u32, data: MessageData) -> Result<u32> {
let ctx = self.get_context(account_id).await?;
let mut message = Message::new(if let Some(viewtype) = data.viewtype {
viewtype.into()
} else if data.file.is_some() {
Viewtype::File
} else {
Viewtype::Text
});
message.set_text(data.text.unwrap_or_default());
if data.html.is_some() {
message.set_html(data.html);
}
if data.override_sender_name.is_some() {
message.set_override_sender_name(data.override_sender_name);
}
if let Some(file) = data.file {
message.set_file(file, None);
}
if let Some((latitude, longitude)) = data.location {
message.set_location(latitude, longitude);
}
if let Some(id) = data.quoted_message_id {
message
.set_quote(
&ctx,
Some(
&Message::load_from_db(&ctx, MsgId::new(id))
.await
.context("message to quote could not be loaded")?,
),
)
.await?;
}
let mut message = data.create_message(&ctx).await?;
let msg_id = chat::send_msg(&ctx, ChatId::new(chat_id), &mut message)
.await?
.to_u32();

View File

@@ -85,7 +85,7 @@ impl FullChat {
let can_send = chat.can_send(context).await?;
let was_seen_recently = if chat.get_type() == Chattype::Single {
match contact_ids.get(0) {
match contact_ids.first() {
Some(contact) => Contact::get_by_id(context, *contact)
.await
.context("failed to load contact for was_seen_recently")?

View File

@@ -102,7 +102,7 @@ pub(crate) async fn get_chat_list_item_by_id(
let self_in_group = chat_contacts.contains(&ContactId::SELF);
let (dm_chat_contact, was_seen_recently) = if chat.get_type() == Chattype::Single {
let contact = chat_contacts.get(0);
let contact = chat_contacts.first();
let was_seen_recently = match contact {
Some(contact) => Contact::get_by_id(ctx, *contact)
.await

View File

@@ -1,5 +1,4 @@
use anyhow::Result;
use deltachat::contact::VerifiedStatus;
use deltachat::context::Context;
use serde::Serialize;
use typescript_type_def::TypeDef;
@@ -46,6 +45,9 @@ pub struct ContactObject {
/// the contact's last seen timestamp
last_seen: i64,
was_seen_recently: bool,
/// If the contact is a bot.
is_bot: bool,
}
impl ContactObject {
@@ -57,7 +59,7 @@ impl ContactObject {
Some(path_buf) => path_buf.to_str().map(|s| s.to_owned()),
None => None,
};
let is_verified = contact.is_verified(context).await? == VerifiedStatus::BidirectVerified;
let is_verified = contact.is_verified(context).await?;
let is_profile_verified = contact.is_profile_verified(context).await?;
let verifier_id = contact
@@ -81,6 +83,7 @@ impl ContactObject {
verifier_id,
last_seen: contact.last_seen(),
was_seen_recently: contact.was_seen_recently(),
is_bot: contact.is_bot(),
})
}
}

View File

@@ -28,55 +28,37 @@ pub enum EventType {
///
/// This event should *not* be reported to the end-user using a popup or something like
/// that.
Info {
msg: String,
},
Info { msg: String },
/// Emitted when SMTP connection is established and login was successful.
SmtpConnected {
msg: String,
},
SmtpConnected { msg: String },
/// Emitted when IMAP connection is established and login was successful.
ImapConnected {
msg: String,
},
ImapConnected { msg: String },
/// Emitted when a message was successfully sent to the SMTP server.
SmtpMessageSent {
msg: String,
},
SmtpMessageSent { msg: String },
/// Emitted when an IMAP message has been marked as deleted
ImapMessageDeleted {
msg: String,
},
ImapMessageDeleted { msg: String },
/// Emitted when an IMAP message has been moved
ImapMessageMoved {
msg: String,
},
ImapMessageMoved { msg: String },
/// Emitted before going into IDLE on the Inbox folder.
ImapInboxIdle,
/// Emitted when an new file in the $BLOBDIR was created
NewBlobFile {
file: String,
},
NewBlobFile { file: String },
/// Emitted when an file in the $BLOBDIR was deleted
DeletedBlobFile {
file: String,
},
DeletedBlobFile { file: String },
/// The library-user should write a warning string to the log.
///
/// This event should *not* be reported to the end-user using a popup or something like
/// that.
Warning {
msg: String,
},
Warning { msg: String },
/// The library-user should report an error to the end-user.
///
@@ -88,18 +70,14 @@ pub enum EventType {
/// it might be better to delay showing these events until the function has really
/// failed (returned false). It should be sufficient to report only the *last* error
/// in a messasge box then.
Error {
msg: String,
},
Error { msg: String },
/// An action cannot be performed because the user is not in the group.
/// Reported eg. after a call to
/// setChatName(), setChatProfileImage(),
/// addContactToChat(), removeContactFromChat(),
/// and messages sending functions.
ErrorSelfNotInGroup {
msg: String,
},
ErrorSelfNotInGroup { msg: String },
/// Messages or chats changed. One or more messages or chats changed for various
/// reasons in the database:
@@ -110,10 +88,7 @@ pub enum EventType {
/// `chatId` is set if only a single chat is affected by the changes, otherwise 0.
/// `msgId` is set if only a single message is affected by the changes, otherwise 0.
#[serde(rename_all = "camelCase")]
MsgsChanged {
chat_id: u32,
msg_id: u32,
},
MsgsChanged { chat_id: u32, msg_id: u32 },
/// Reactions for the message changed.
#[serde(rename_all = "camelCase")]
@@ -128,10 +103,7 @@ pub enum EventType {
///
/// There is no extra #DC_EVENT_MSGS_CHANGED event send together with this event.
#[serde(rename_all = "camelCase")]
IncomingMsg {
chat_id: u32,
msg_id: u32,
},
IncomingMsg { chat_id: u32, msg_id: u32 },
/// Downloading a bunch of messages just finished. This is an experimental
/// event to allow the UI to only show one notification per message bunch,
@@ -139,47 +111,31 @@ pub enum EventType {
///
/// msg_ids contains the message ids.
#[serde(rename_all = "camelCase")]
IncomingMsgBunch {
msg_ids: Vec<u32>,
},
IncomingMsgBunch { msg_ids: Vec<u32> },
/// Messages were seen or noticed.
/// chat id is always set.
#[serde(rename_all = "camelCase")]
MsgsNoticed {
chat_id: u32,
},
MsgsNoticed { chat_id: u32 },
/// A single message is sent successfully. State changed from DC_STATE_OUT_PENDING to
/// DC_STATE_OUT_DELIVERED, see `Message.state`.
#[serde(rename_all = "camelCase")]
MsgDelivered {
chat_id: u32,
msg_id: u32,
},
MsgDelivered { chat_id: u32, msg_id: u32 },
/// A single message could not be sent. State changed from DC_STATE_OUT_PENDING or DC_STATE_OUT_DELIVERED to
/// DC_STATE_OUT_FAILED, see `Message.state`.
#[serde(rename_all = "camelCase")]
MsgFailed {
chat_id: u32,
msg_id: u32,
},
MsgFailed { chat_id: u32, msg_id: u32 },
/// A single message is read by the receiver. State changed from DC_STATE_OUT_DELIVERED to
/// DC_STATE_OUT_MDN_RCVD, see `Message.state`.
#[serde(rename_all = "camelCase")]
MsgRead {
chat_id: u32,
msg_id: u32,
},
MsgRead { chat_id: u32, msg_id: u32 },
/// A single message is deleted.
#[serde(rename_all = "camelCase")]
MsgDeleted {
chat_id: u32,
msg_id: u32,
},
MsgDeleted { chat_id: u32, msg_id: u32 },
/// Chat changed. The name or the image of a chat group was changed or members were added or removed.
/// Or the verify state of a chat has changed.
@@ -189,24 +145,17 @@ pub enum EventType {
/// This event does not include ephemeral timer modification, which
/// is a separate event.
#[serde(rename_all = "camelCase")]
ChatModified {
chat_id: u32,
},
ChatModified { chat_id: u32 },
/// Chat ephemeral timer changed.
#[serde(rename_all = "camelCase")]
ChatEphemeralTimerModified {
chat_id: u32,
timer: u32,
},
ChatEphemeralTimerModified { chat_id: u32, timer: u32 },
/// Contact(s) created, renamed, blocked or deleted.
///
/// @param data1 (int) If set, this is the contact_id of an added contact that should be selected.
#[serde(rename_all = "camelCase")]
ContactsChanged {
contact_id: Option<u32>,
},
ContactsChanged { contact_id: Option<u32> },
/// Location of one or more contact has changed.
///
@@ -214,9 +163,7 @@ pub enum EventType {
/// If the locations of several contacts have been changed,
/// this parameter is set to `None`.
#[serde(rename_all = "camelCase")]
LocationChanged {
contact_id: Option<u32>,
},
LocationChanged { contact_id: Option<u32> },
/// Inform about the configuration progress started by configure().
ConfigureProgress {
@@ -234,9 +181,7 @@ pub enum EventType {
/// @param data1 (usize) 0=error, 1-999=progress in permille, 1000=success and done
/// @param data2 0
#[serde(rename_all = "camelCase")]
ImexProgress {
progress: usize,
},
ImexProgress { progress: usize },
/// A file has been exported. A file has been written by imex().
/// This event may be sent multiple times by a single call to imex().
@@ -246,9 +191,7 @@ pub enum EventType {
///
/// @param data2 0
#[serde(rename_all = "camelCase")]
ImexFileWritten {
path: String,
},
ImexFileWritten { path: String },
/// Progress information of a secure-join handshake from the view of the inviter
/// (Alice, the person who shows the QR code).
@@ -263,10 +206,7 @@ pub enum EventType {
/// 800=vg-member-added-received received, shown as "bob@addr securely joined GROUP", only sent for the verified-group-protocol.
/// 1000=Protocol finished for this contact.
#[serde(rename_all = "camelCase")]
SecurejoinInviterProgress {
contact_id: u32,
progress: usize,
},
SecurejoinInviterProgress { contact_id: u32, progress: usize },
/// Progress information of a secure-join handshake from the view of the joiner
/// (Bob, the person who scans the QR code).
@@ -277,10 +217,7 @@ pub enum EventType {
/// 400=vg-/vc-request-with-auth sent, typically shown as "alice@addr verified, introducing myself."
/// (Bob has verified alice and waits until Alice does the same for him)
#[serde(rename_all = "camelCase")]
SecurejoinJoinerProgress {
contact_id: u32,
progress: usize,
},
SecurejoinJoinerProgress { contact_id: u32, progress: usize },
/// The connectivity to the server changed.
/// This means that you should refresh the connectivity view
@@ -288,8 +225,17 @@ pub enum EventType {
/// getConnectivityHtml() for details.
ConnectivityChanged,
/// Deprecated by `ConfigSynced`.
SelfavatarChanged,
/// A multi-device synced config value changed. Maybe the app needs to refresh smth. For
/// uniformity this is emitted on the source device too. The value isn't here, otherwise it
/// would be logged which might not be good for privacy.
ConfigSynced {
/// Configuration key.
key: String,
},
#[serde(rename_all = "camelCase")]
WebxdcStatusUpdate {
msg_id: u32,
@@ -298,9 +244,14 @@ pub enum EventType {
/// Inform that a message containing a webxdc instance has been deleted
#[serde(rename_all = "camelCase")]
WebxdcInstanceDeleted {
msg_id: u32,
},
WebxdcInstanceDeleted { msg_id: u32 },
/// Tells that the Background fetch was completed (or timed out).
/// This event acts as a marker, when you reach this event you can be sure
/// that all events emitted during the background fetch were processed.
///
/// This event is only emitted by the account manager
AccountsBackgroundFetchDone,
}
impl From<CoreEventType> for EventType {
@@ -396,6 +347,9 @@ impl From<CoreEventType> for EventType {
},
CoreEventType::ConnectivityChanged => ConnectivityChanged,
CoreEventType::SelfavatarChanged => SelfavatarChanged,
CoreEventType::ConfigSynced { key } => ConfigSynced {
key: key.to_string(),
},
CoreEventType::WebxdcStatusUpdate {
msg_id,
status_update_serial,
@@ -406,6 +360,7 @@ impl From<CoreEventType> for EventType {
CoreEventType::WebxdcInstanceDeleted { msg_id } => WebxdcInstanceDeleted {
msg_id: msg_id.to_u32(),
},
CoreEventType::AccountsBackgroundFetchDone => AccountsBackgroundFetchDone,
}
}
}

View File

@@ -345,6 +345,7 @@ pub enum SystemMessageType {
SecurejoinMessage,
LocationStreamingEnabled,
LocationOnly,
InvalidUnencryptedMail,
/// Chat ephemeral message timer is changed.
EphemeralTimerChanged,
@@ -385,6 +386,7 @@ impl From<deltachat::mimeparser::SystemMessage> for SystemMessageType {
SystemMessage::MultiDeviceSync => SystemMessageType::MultiDeviceSync,
SystemMessage::WebxdcStatusUpdate => SystemMessageType::WebxdcStatusUpdate,
SystemMessage::WebxdcInfoMessage => SystemMessageType::WebxdcInfoMessage,
SystemMessage::InvalidUnencryptedMail => SystemMessageType::InvalidUnencryptedMail,
}
}
}
@@ -546,6 +548,44 @@ pub struct MessageData {
pub quoted_message_id: Option<u32>,
}
impl MessageData {
pub(crate) async fn create_message(self, context: &Context) -> Result<Message> {
let mut message = Message::new(if let Some(viewtype) = self.viewtype {
viewtype.into()
} else if self.file.is_some() {
Viewtype::File
} else {
Viewtype::Text
});
message.set_text(self.text.unwrap_or_default());
if self.html.is_some() {
message.set_html(self.html);
}
if self.override_sender_name.is_some() {
message.set_override_sender_name(self.override_sender_name);
}
if let Some(file) = self.file {
message.set_file(file, None);
}
if let Some((latitude, longitude)) = self.location {
message.set_location(latitude, longitude);
}
if let Some(id) = self.quoted_message_id {
message
.set_quote(
context,
Some(
&Message::load_from_db(context, MsgId::new(id))
.await
.context("message to quote could not be loaded")?,
),
)
.await?;
}
Ok(message)
}
}
#[derive(Serialize, TypeDef, schemars::JsonSchema)]
#[serde(rename_all = "camelCase")]
pub struct MessageReadReceipt {

View File

@@ -33,10 +33,8 @@ async fn main() -> Result<(), std::io::Error> {
let addr = SocketAddr::from(([127, 0, 0, 1], port));
log::info!("JSON-RPC WebSocket server listening on {}", addr);
axum::Server::bind(&addr)
.serve(app.into_make_service())
.await
.unwrap();
let listener = tokio::net::TcpListener::bind(addr).await.unwrap();
axum::serve(listener, app).await.unwrap();
Ok(())
}

View File

@@ -53,5 +53,5 @@
},
"type": "module",
"types": "dist/deltachat.d.ts",
"version": "1.131.5"
"version": "1.134.0"
}

View File

@@ -1,6 +1,6 @@
[package]
name = "deltachat-repl"
version = "1.131.5"
version = "1.134.0"
license = "MPL-2.0"
edition = "2021"
@@ -12,7 +12,7 @@ dirs = "5"
log = "0.4.20"
pretty_env_logger = "0.5"
rusqlite = "0.30"
rustyline = "12"
rustyline = "13"
tokio = { version = "1", features = ["fs", "rt-multi-thread", "macros"] }
[features]

View File

@@ -284,13 +284,8 @@ async fn log_contactlist(context: &Context, contacts: &[ContactId]) -> Result<()
let contact = Contact::get_by_id(context, *contact_id).await?;
let name = contact.get_display_name();
let addr = contact.get_addr();
let verified_state = contact.is_verified(context).await?;
let verified_str = if VerifiedStatus::Unverified != verified_state {
if verified_state == VerifiedStatus::BidirectVerified {
" √√"
} else {
""
}
let verified_str = if contact.is_verified(context).await? {
""
} else {
""
};

View File

@@ -299,8 +299,8 @@ impl Highlighter for DcHelper {
self.highlighter.highlight(line, pos)
}
fn highlight_char(&self, line: &str, pos: usize) -> bool {
self.highlighter.highlight_char(line, pos)
fn highlight_char(&self, line: &str, pos: usize, forced: bool) -> bool {
self.highlighter.highlight_char(line, pos, forced)
}
}
@@ -481,7 +481,10 @@ async fn handle_cmd(
#[tokio::main]
async fn main() -> Result<(), Error> {
let _ = pretty_env_logger::try_init();
pretty_env_logger::formatted_timed_builder()
.parse_default_env()
.filter_module("deltachat_repl", log::LevelFilter::Info)
.init();
let args = std::env::args().collect();
start(args).await?;

View File

@@ -1,4 +1,5 @@
"""Delta Chat JSON-RPC high-level API"""
from ._utils import AttrDict, run_bot_cli, run_client_cli
from .account import Account
from .chat import Chat

View File

@@ -300,3 +300,13 @@ class Account:
def import_backup(self, path, passphrase: str = "") -> None:
"""Import backup."""
self._rpc.import_backup(self.id, str(path), passphrase)
def export_self_keys(self, path) -> None:
"""Export keys."""
passphrase = "" # Setting passphrase is currently not supported.
self._rpc.export_self_keys(self.id, str(path), passphrase)
def import_self_keys(self, path) -> None:
"""Import keys."""
passphrase = "" # Importing passphrase-protected keys is currently not supported.
self._rpc.import_self_keys(self.id, str(path), passphrase)

View File

@@ -1,4 +1,5 @@
"""Event loop implementations offering high level event handling/hooking."""
import logging
from typing import (
TYPE_CHECKING,

View File

@@ -1,4 +1,5 @@
"""High-level classes for event processing and filtering."""
import re
from abc import ABC, abstractmethod
from typing import TYPE_CHECKING, Callable, Iterable, Iterator, Optional, Set, Tuple, Union

View File

@@ -1,9 +1,10 @@
import logging
import pytest
from deltachat_rpc_client import Chat, SpecialContactId
def test_qr_setup_contact(acfactory) -> None:
def test_qr_setup_contact(acfactory, tmp_path) -> None:
alice, bob = acfactory.get_online_accounts(2)
qr_code, _svg = alice.get_qr_code()
@@ -23,13 +24,26 @@ def test_qr_setup_contact(acfactory) -> None:
bob_contact_alice_snapshot = bob_contact_alice.get_snapshot()
assert bob_contact_alice_snapshot.is_verified
# Test that if Bob changes the key, backwards verification is lost.
logging.info("Bob 2 is created")
bob2 = acfactory.new_configured_account()
bob2.export_self_keys(tmp_path)
def test_qr_securejoin(acfactory):
logging.info("Bob imports a key")
bob.import_self_keys(tmp_path / "private-key-default.asc")
assert bob.get_config("key_id") == "2"
bob_contact_alice_snapshot = bob_contact_alice.get_snapshot()
assert not bob_contact_alice_snapshot.is_verified
@pytest.mark.parametrize("protect", [True, False])
def test_qr_securejoin(acfactory, protect):
alice, bob = acfactory.get_online_accounts(2)
logging.info("Alice creates a verified group")
alice_chat = alice.create_group("Verified group", protect=True)
assert alice_chat.get_basic_snapshot().is_protected
alice_chat = alice.create_group("Verified group", protect=protect)
assert alice_chat.get_basic_snapshot().is_protected == protect
logging.info("Bob joins verified group")
qr_code, _svg = alice_chat.get_qr_code()
@@ -53,7 +67,7 @@ def test_qr_securejoin(acfactory):
snapshot = bob.get_message_by_id(bob.wait_for_incoming_msg_event().msg_id).get_snapshot()
assert snapshot.text == "Member Me ({}) added by {}.".format(bob.get_config("addr"), alice.get_config("addr"))
assert snapshot.chat.get_basic_snapshot().is_protected
assert snapshot.chat.get_basic_snapshot().is_protected == protect
# Test that Bob verified Alice's profile.
bob_contact_alice = bob.get_contact_by_addr(alice.get_config("addr"))
@@ -161,7 +175,7 @@ def test_verified_group_recovery(acfactory) -> None:
logging.info("ac2 joins verified group")
qr_code, _svg = chat.get_qr_code()
ac2.secure_join(qr_code)
ac1.wait_for_securejoin_inviter_success()
ac2.wait_for_securejoin_joiner_success()
# ac1 has ac2 directly verified.
ac1_contact_ac2 = ac1.get_contact_by_addr(ac2.get_config("addr"))
@@ -169,7 +183,8 @@ def test_verified_group_recovery(acfactory) -> None:
logging.info("ac3 joins verified group")
ac3_chat = ac3.secure_join(qr_code)
ac1.wait_for_securejoin_inviter_success()
ac3.wait_for_securejoin_joiner_success()
ac3.wait_for_incoming_msg_event() # Member added
logging.info("ac2 logs in on a new device")
ac2 = acfactory.resetup_account(ac2)
@@ -177,8 +192,7 @@ def test_verified_group_recovery(acfactory) -> None:
logging.info("ac2 reverifies with ac3")
qr_code, _svg = ac3.get_qr_code()
ac2.secure_join(qr_code)
ac3.wait_for_securejoin_inviter_success()
ac2.wait_for_securejoin_joiner_success()
logging.info("ac3 sends a message to the group")
assert len(ac3_chat.get_contacts()) == 3
@@ -225,7 +239,7 @@ def test_verified_group_member_added_recovery(acfactory) -> None:
logging.info("ac2 joins verified group")
qr_code, _svg = chat.get_qr_code()
ac2.secure_join(qr_code)
ac1.wait_for_securejoin_inviter_success()
ac2.wait_for_securejoin_joiner_success()
# ac1 has ac2 directly verified.
ac1_contact_ac2 = ac1.get_contact_by_addr(ac2.get_config("addr"))
@@ -233,7 +247,8 @@ def test_verified_group_member_added_recovery(acfactory) -> None:
logging.info("ac3 joins verified group")
ac3_chat = ac3.secure_join(qr_code)
ac1.wait_for_securejoin_inviter_success()
ac3.wait_for_securejoin_joiner_success()
ac3.wait_for_incoming_msg_event() # Member added
logging.info("ac2 logs in on a new device")
ac2 = acfactory.resetup_account(ac2)
@@ -241,8 +256,7 @@ def test_verified_group_member_added_recovery(acfactory) -> None:
logging.info("ac2 reverifies with ac3")
qr_code, _svg = ac3.get_qr_code()
ac2.secure_join(qr_code)
ac3.wait_for_securejoin_inviter_success()
ac2.wait_for_securejoin_joiner_success()
logging.info("ac3 sends a message to the group")
assert len(ac3_chat.get_contacts()) == 3
@@ -429,3 +443,124 @@ def test_aeap_flow_verified(acfactory):
assert ac1new.get_config("addr") in [
contact.get_snapshot().address for contact in msg_in_2_snapshot.chat.get_contacts()
]
def test_gossip_verification(acfactory) -> None:
alice, bob, carol = acfactory.get_online_accounts(3)
# Bob verifies Alice.
qr_code, _svg = alice.get_qr_code()
bob.secure_join(qr_code)
bob.wait_for_securejoin_joiner_success()
# Bob verifies Carol.
qr_code, _svg = carol.get_qr_code()
bob.secure_join(qr_code)
bob.wait_for_securejoin_joiner_success()
bob_contact_alice = bob.create_contact(alice.get_config("addr"), "Alice")
bob_contact_carol = bob.create_contact(carol.get_config("addr"), "Carol")
carol_contact_alice = carol.create_contact(alice.get_config("addr"), "Alice")
logging.info("Bob creates an Autocrypt group")
bob_group_chat = bob.create_group("Autocrypt Group")
assert not bob_group_chat.get_basic_snapshot().is_protected
bob_group_chat.add_contact(bob_contact_alice)
bob_group_chat.add_contact(bob_contact_carol)
bob_group_chat.send_message(text="Hello Autocrypt group")
snapshot = carol.get_message_by_id(carol.wait_for_incoming_msg_event().msg_id).get_snapshot()
assert snapshot.text == "Hello Autocrypt group"
assert snapshot.show_padlock
# Autocrypt group does not propagate verification.
carol_contact_alice_snapshot = carol_contact_alice.get_snapshot()
assert not carol_contact_alice_snapshot.is_verified
logging.info("Bob creates a Securejoin group")
bob_group_chat = bob.create_group("Securejoin Group", protect=True)
assert bob_group_chat.get_basic_snapshot().is_protected
bob_group_chat.add_contact(bob_contact_alice)
bob_group_chat.add_contact(bob_contact_carol)
bob_group_chat.send_message(text="Hello Securejoin group")
snapshot = carol.get_message_by_id(carol.wait_for_incoming_msg_event().msg_id).get_snapshot()
assert snapshot.text == "Hello Securejoin group"
assert snapshot.show_padlock
# Securejoin propagates verification.
carol_contact_alice_snapshot = carol_contact_alice.get_snapshot()
assert carol_contact_alice_snapshot.is_verified
def test_securejoin_after_contact_resetup(acfactory) -> None:
"""
Regression test for a bug that prevented joining verified group with a QR code
if the group is already created and contains
a contact with inconsistent (Autocrypt and verified keys exist but don't match) key state.
"""
ac1, ac2, ac3 = acfactory.get_online_accounts(3)
# ac3 creates protected group with ac1.
ac3_chat = ac3.create_group("Verified group", protect=True)
# ac1 joins ac3 group.
ac3_qr_code, _svg = ac3_chat.get_qr_code()
ac1.secure_join(ac3_qr_code)
ac1.wait_for_securejoin_joiner_success()
# ac1 waits for member added message and creates a QR code.
snapshot = ac1.get_message_by_id(ac1.wait_for_incoming_msg_event().msg_id).get_snapshot()
ac1_qr_code, _svg = snapshot.chat.get_qr_code()
# ac2 verifies ac1
qr_code, _svg = ac1.get_qr_code()
ac2.secure_join(qr_code)
ac2.wait_for_securejoin_joiner_success()
# ac1 is verified for ac2.
ac2_contact_ac1 = ac2.create_contact(ac1.get_config("addr"), "")
assert ac2_contact_ac1.get_snapshot().is_verified
# ac1 resetups the account.
ac1 = acfactory.resetup_account(ac1)
# ac1 sends a message to ac2.
ac1_contact_ac2 = ac1.create_contact(ac2.get_config("addr"), "")
ac1_chat_ac2 = ac1_contact_ac2.create_chat()
ac1_chat_ac2.send_text("Hello!")
# ac2 receives a message.
snapshot = ac2.get_message_by_id(ac2.wait_for_incoming_msg_event().msg_id).get_snapshot()
assert snapshot.text == "Hello!"
# ac1 is no longer verified for ac2 as new Autocrypt key is not the same as old verified key.
assert not ac2_contact_ac1.get_snapshot().is_verified
# ac1 goes offline.
ac1.remove()
# Scanning a QR code results in creating an unprotected group with an inviter.
# In this case inviter is ac1 which has an inconsistent key state.
# Normally inviter becomes verified as a result of Securejoin protocol
# and then the group chat becomes verified when "Member added" is received,
# but in this case ac1 is offline and this Securejoin process will never finish.
logging.info("ac2 scans ac1 QR code, this is not expected to finish")
ac2.secure_join(ac1_qr_code)
logging.info("ac2 scans ac3 QR code")
ac2.secure_join(ac3_qr_code)
logging.info("ac2 waits for joiner success")
ac2.wait_for_securejoin_joiner_success()
# Wait for member added.
logging.info("ac2 waits for member added message")
snapshot = ac2.get_message_by_id(ac2.wait_for_incoming_msg_event().msg_id).get_snapshot()
assert snapshot.is_info
ac2_chat = snapshot.chat
assert ac2_chat.get_basic_snapshot().is_protected
assert len(ac2_chat.get_contacts()) == 3
# ac1 is still "not verified" for ac2 due to inconsistent state.
assert not ac2_contact_ac1.get_snapshot().is_verified

View File

@@ -140,12 +140,9 @@ def test_chat(acfactory) -> None:
alice_chat_bob = alice_contact_bob.create_chat()
alice_chat_bob.send_text("Hello!")
while True:
event = bob.wait_for_event()
if event.kind == EventType.INCOMING_MSG:
chat_id = event.chat_id
msg_id = event.msg_id
break
event = bob.wait_for_incoming_msg_event()
chat_id = event.chat_id
msg_id = event.msg_id
message = bob.get_message_by_id(msg_id)
snapshot = message.get_snapshot()
assert snapshot.chat_id == chat_id
@@ -224,12 +221,9 @@ def test_message(acfactory) -> None:
alice_chat_bob = alice_contact_bob.create_chat()
alice_chat_bob.send_text("Hello!")
while True:
event = bob.wait_for_event()
if event.kind == EventType.INCOMING_MSG:
chat_id = event.chat_id
msg_id = event.msg_id
break
event = bob.wait_for_incoming_msg_event()
chat_id = event.chat_id
msg_id = event.msg_id
message = bob.get_message_by_id(msg_id)
snapshot = message.get_snapshot()
@@ -331,7 +325,7 @@ def test_wait_next_messages(acfactory) -> None:
next_messages_task = executor.submit(bot.wait_next_messages)
bot_addr = bot.get_config("addr")
alice_contact_bot = alice.create_contact(bot_addr, "Bob")
alice_contact_bot = alice.create_contact(bot_addr, "Bot")
alice_chat_bot = alice_contact_bot.create_chat()
alice_chat_bot.send_text("Hello!")
@@ -341,7 +335,7 @@ def test_wait_next_messages(acfactory) -> None:
assert snapshot.text == "Hello!"
def test_import_export(acfactory, tmp_path) -> None:
def test_import_export_backup(acfactory, tmp_path) -> None:
alice = acfactory.new_configured_account()
alice.export_backup(tmp_path)
@@ -352,6 +346,31 @@ def test_import_export(acfactory, tmp_path) -> None:
assert alice2.manager.get_system_info()
def test_import_export_keys(acfactory, tmp_path) -> None:
alice, bob = acfactory.get_online_accounts(2)
bob_addr = bob.get_config("addr")
alice_contact_bob = alice.create_contact(bob_addr, "Bob")
alice_chat_bob = alice_contact_bob.create_chat()
alice_chat_bob.send_text("Hello Bob!")
snapshot = bob.get_message_by_id(bob.wait_for_incoming_msg_event().msg_id).get_snapshot()
assert snapshot.text == "Hello Bob!"
# Alice resetups account, but keeps the key.
alice_keys_path = tmp_path / "alice_keys"
alice_keys_path.mkdir()
alice.export_self_keys(alice_keys_path)
alice = acfactory.resetup_account(alice)
alice.import_self_keys(alice_keys_path)
snapshot.chat.accept()
snapshot.chat.send_text("Hello Alice!")
snapshot = alice.get_message_by_id(alice.wait_for_incoming_msg_event().msg_id).get_snapshot()
assert snapshot.text == "Hello Alice!"
assert snapshot.show_padlock
def test_openrpc_command_line() -> None:
"""Test that "deltachat-rpc-server --openrpc" command returns an OpenRPC specification."""
out = subprocess.run(["deltachat-rpc-server", "--openrpc"], capture_output=True, check=True).stdout
@@ -377,3 +396,46 @@ def test_provider_info(rpc) -> None:
rpc.set_config(account_id, "socks5_enabled", "1")
provider_info = rpc.get_provider_info(account_id, "github.com")
assert provider_info is None
def test_mdn_doesnt_break_autocrypt(acfactory) -> None:
alice, bob = acfactory.get_online_accounts(2)
bob_addr = bob.get_config("addr")
alice_contact_bob = alice.create_contact(bob_addr, "Bob")
# Bob creates chat manually so chat with Alice is accepted.
alice_chat_bob = alice_contact_bob.create_chat()
# Alice sends a message to Bob.
alice_chat_bob.send_text("Hello Bob!")
event = bob.wait_for_incoming_msg_event()
msg_id = event.msg_id
message = bob.get_message_by_id(msg_id)
snapshot = message.get_snapshot()
# Bob sends a message to Alice.
bob_chat_alice = snapshot.chat
bob_chat_alice.accept()
bob_chat_alice.send_text("Hello Alice!")
event = alice.wait_for_incoming_msg_event()
msg_id = event.msg_id
message = alice.get_message_by_id(msg_id)
snapshot = message.get_snapshot()
assert snapshot.show_padlock
# Alice reads Bob's message.
message.mark_seen()
while True:
event = bob.wait_for_event()
if event.kind == EventType.MSG_READ:
break
# Bob sends a message to Alice, it should also be encrypted.
bob_chat_alice.send_text("Hi Alice!")
event = alice.wait_for_incoming_msg_event()
msg_id = event.msg_id
message = alice.get_message_by_id(msg_id)
snapshot = message.get_snapshot()
assert snapshot.show_padlock

View File

@@ -43,3 +43,15 @@ def test_webxdc(acfactory) -> None:
assert status_updates == [
{"payload": "Second update", "serial": 2, "max_serial": 2},
]
def test_webxdc_insert_lots_of_updates(acfactory) -> None:
alice, bob = acfactory.get_online_accounts(2)
bob_addr = bob.get_config("addr")
alice_contact_bob = alice.create_contact(bob_addr, "Bob")
alice_chat_bob = alice_contact_bob.create_chat()
message = alice_chat_bob.send_message(text="Let's play chess!", file="../test-data/webxdc/chess.xdc")
for i in range(2000):
message.send_webxdc_status_update({"payload": str(i)}, "description")

View File

@@ -1,6 +1,6 @@
[package]
name = "deltachat-rpc-server"
version = "1.131.5"
version = "1.134.0"
description = "DeltaChat JSON-RPC server"
edition = "2021"
readme = "README.md"
@@ -17,7 +17,7 @@ anyhow = "1"
env_logger = { version = "0.10.0" }
futures-lite = "2.0.0"
log = "0.4"
serde_json = "1.0.105"
serde_json = "1"
serde = { version = "1.0", features = ["derive"] }
tokio = { version = "1.33.0", features = ["io-std"] }
tokio-util = "0.7.9"

View File

@@ -3,6 +3,13 @@ unmaintained = "allow"
ignore = [
"RUSTSEC-2020-0071",
"RUSTSEC-2022-0093",
# Timing attack on RSA.
# Delta Chat does not use RSA for new keys
# and this requires precise measurement of the decryption time by the attacker.
# There is no fix at the time of writing this (2023-11-28).
# <https://rustsec.org/advisories/RUSTSEC-2023-0071>
"RUSTSEC-2023-0071",
]
[bans]
@@ -27,6 +34,11 @@ skip = [
{ name = "ed25519", version = "1.5.3" },
{ name = "event-listener", version = "2.5.3" },
{ name = "getrandom", version = "<0.2" },
{ name = "h2", version = "0.3.22" },
{ name = "http-body", version = "0.4.5" },
{ name = "http", version = "0.2.11" },
{ name = "hyper", version = "0.14.27" },
{ name = "idna", version = "0.4.0" },
{ name = "pem-rfc7468", version = "0.6.0" },
{ name = "pkcs8", version = "0.9.0" },
{ name = "quick-error", version = "<2.0" },
@@ -40,19 +52,22 @@ skip = [
{ name = "sec1", version = "0.3.0" },
{ name = "sha2", version = "<0.10" },
{ name = "signature", version = "1.6.4" },
{ name = "socket2", version = "0.4.9" },
{ name = "spin", version = "<0.9.6" },
{ name = "spki", version = "0.6.0" },
{ name = "syn", version = "1.0.109" },
{ name = "time", version = "<0.3" },
{ name = "untrusted", version = "0.7.1" },
{ name = "wasi", version = "<0.11" },
{ name = "windows_aarch64_msvc", version = "<0.48" },
{ name = "windows_i686_gnu", version = "<0.48" },
{ name = "windows_i686_msvc", version = "<0.48" },
{ name = "windows_aarch64_gnullvm", version = "<0.52" },
{ name = "windows_aarch64_msvc", version = "<0.52" },
{ name = "windows_i686_gnu", version = "<0.52" },
{ name = "windows_i686_msvc", version = "<0.52" },
{ name = "windows-sys", version = "<0.52" },
{ name = "windows-targets", version = "<0.52" },
{ name = "windows", version = "0.32.0" },
{ name = "windows_x86_64_gnu", version = "<0.48" },
{ name = "windows_x86_64_msvc", version = "<0.48" },
{ name = "windows_x86_64_gnullvm", version = "<0.52" },
{ name = "windows_x86_64_gnu", version = "<0.52" },
{ name = "windows_x86_64_msvc", version = "<0.52" },
]

505
fuzz/Cargo.lock generated

File diff suppressed because it is too large Load Diff

View File

@@ -29,9 +29,11 @@ module.exports = {
DC_DOWNLOAD_FAILURE: 20,
DC_DOWNLOAD_IN_PROGRESS: 1000,
DC_DOWNLOAD_UNDECIPHERABLE: 30,
DC_EVENT_ACCOUNTS_BACKGROUND_FETCH_DONE: 2200,
DC_EVENT_CHAT_EPHEMERAL_TIMER_MODIFIED: 2021,
DC_EVENT_CHAT_MODIFIED: 2020,
DC_EVENT_CONFIGURE_PROGRESS: 2041,
DC_EVENT_CONFIG_SYNCED: 2111,
DC_EVENT_CONNECTIVITY_CHANGED: 2100,
DC_EVENT_CONTACTS_CHANGED: 2030,
DC_EVENT_DELETED_BLOB_FILE: 151,
@@ -79,6 +81,7 @@ module.exports = {
DC_INFO_EPHEMERAL_TIMER_CHANGED: 10,
DC_INFO_GROUP_IMAGE_CHANGED: 3,
DC_INFO_GROUP_NAME_CHANGED: 2,
DC_INFO_INVALID_UNENCRYPTED_MAIL: 13,
DC_INFO_LOCATIONSTREAMING_ENABLED: 8,
DC_INFO_LOCATION_ONLY: 9,
DC_INFO_MEMBER_ADDED_TO_GROUP: 4,
@@ -225,11 +228,13 @@ module.exports = {
DC_STR_GROUP_NAME_CHANGED_BY_YOU: 124,
DC_STR_IMAGE: 9,
DC_STR_INCOMING_MESSAGES: 103,
DC_STR_INVALID_UNENCRYPTED_MAIL: 174,
DC_STR_LAST_MSG_SENT_SUCCESSFULLY: 111,
DC_STR_LOCATION: 66,
DC_STR_LOCATION_ENABLED_BY_OTHER: 137,
DC_STR_LOCATION_ENABLED_BY_YOU: 136,
DC_STR_MESSAGES: 114,
DC_STR_MESSAGE_ADD_MEMBER: 173,
DC_STR_MSGACTIONBYME: 63,
DC_STR_MSGACTIONBYUSER: 62,
DC_STR_MSGADDMEMBER: 17,

View File

@@ -34,6 +34,8 @@ module.exports = {
2061: 'DC_EVENT_SECUREJOIN_JOINER_PROGRESS',
2100: 'DC_EVENT_CONNECTIVITY_CHANGED',
2110: 'DC_EVENT_SELFAVATAR_CHANGED',
2111: 'DC_EVENT_CONFIG_SYNCED',
2120: 'DC_EVENT_WEBXDC_STATUS_UPDATE',
2121: 'DC_EVENT_WEBXDC_INSTANCE_DELETED'
2121: 'DC_EVENT_WEBXDC_INSTANCE_DELETED',
2200: 'DC_EVENT_ACCOUNTS_BACKGROUND_FETCH_DONE'
}

View File

@@ -29,9 +29,11 @@ export enum C {
DC_DOWNLOAD_FAILURE = 20,
DC_DOWNLOAD_IN_PROGRESS = 1000,
DC_DOWNLOAD_UNDECIPHERABLE = 30,
DC_EVENT_ACCOUNTS_BACKGROUND_FETCH_DONE = 2200,
DC_EVENT_CHAT_EPHEMERAL_TIMER_MODIFIED = 2021,
DC_EVENT_CHAT_MODIFIED = 2020,
DC_EVENT_CONFIGURE_PROGRESS = 2041,
DC_EVENT_CONFIG_SYNCED = 2111,
DC_EVENT_CONNECTIVITY_CHANGED = 2100,
DC_EVENT_CONTACTS_CHANGED = 2030,
DC_EVENT_DELETED_BLOB_FILE = 151,
@@ -79,6 +81,7 @@ export enum C {
DC_INFO_EPHEMERAL_TIMER_CHANGED = 10,
DC_INFO_GROUP_IMAGE_CHANGED = 3,
DC_INFO_GROUP_NAME_CHANGED = 2,
DC_INFO_INVALID_UNENCRYPTED_MAIL = 13,
DC_INFO_LOCATIONSTREAMING_ENABLED = 8,
DC_INFO_LOCATION_ONLY = 9,
DC_INFO_MEMBER_ADDED_TO_GROUP = 4,
@@ -225,11 +228,13 @@ export enum C {
DC_STR_GROUP_NAME_CHANGED_BY_YOU = 124,
DC_STR_IMAGE = 9,
DC_STR_INCOMING_MESSAGES = 103,
DC_STR_INVALID_UNENCRYPTED_MAIL = 174,
DC_STR_LAST_MSG_SENT_SUCCESSFULLY = 111,
DC_STR_LOCATION = 66,
DC_STR_LOCATION_ENABLED_BY_OTHER = 137,
DC_STR_LOCATION_ENABLED_BY_YOU = 136,
DC_STR_MESSAGES = 114,
DC_STR_MESSAGE_ADD_MEMBER = 173,
DC_STR_MSGACTIONBYME = 63,
DC_STR_MSGACTIONBYUSER = 62,
DC_STR_MSGADDMEMBER = 17,
@@ -319,6 +324,8 @@ export const EventId2EventName: { [key: number]: string } = {
2061: 'DC_EVENT_SECUREJOIN_JOINER_PROGRESS',
2100: 'DC_EVENT_CONNECTIVITY_CHANGED',
2110: 'DC_EVENT_SELFAVATAR_CHANGED',
2111: 'DC_EVENT_CONFIG_SYNCED',
2120: 'DC_EVENT_WEBXDC_STATUS_UPDATE',
2121: 'DC_EVENT_WEBXDC_INSTANCE_DELETED',
2200: 'DC_EVENT_ACCOUNTS_BACKGROUND_FETCH_DONE',
}

View File

@@ -178,7 +178,7 @@ export class AccountManager extends EventEmitter {
static newTemporary() {
let directory = null
while (true) {
const randomString = Math.random().toString(36).substr(2, 5)
const randomString = Math.random().toString(36).substring(2, 5)
directory = join(tmpdir(), 'deltachat-' + randomString)
if (!existsSync(directory)) break
}

View File

@@ -1,13 +1,17 @@
// @ts-check
import DeltaChat from '../dist'
import { DeltaChat } from '../dist/index.js'
import { deepStrictEqual, strictEqual } from 'assert'
import chai, { expect } from 'chai'
import chaiAsPromised from 'chai-as-promised'
import { EventId2EventName, C } from '../dist/constants'
import { EventId2EventName, C } from '../dist/constants.js'
import { join } from 'path'
import { statSync } from 'fs'
import { Context } from '../dist/context'
import { Context } from '../dist/context.js'
import {fileURLToPath} from 'url';
const __dirname = fileURLToPath(new URL('.', import.meta.url));
chai.use(chaiAsPromised)
chai.config.truncateThreshold = 0 // Do not truncate assertion errors.

View File

@@ -8,9 +8,8 @@
"devDependencies": {
"@types/debug": "^4.1.7",
"@types/node": "^20.8.10",
"chai": "^4.2.0",
"chai": "~4.3.10",
"chai-as-promised": "^7.1.1",
"esm": "^3.2.25",
"mocha": "^8.2.1",
"node-gyp": "^10.0.0",
"prebuildify": "^5.0.1",
@@ -53,8 +52,8 @@
"prebuildify": "cd node && prebuildify -t 18.0.0 --napi --strip --postinstall \"node scripts/postinstall.js --prebuild\"",
"test": "npm run test:lint && npm run test:mocha",
"test:lint": "npm run lint",
"test:mocha": "mocha -r esm node/test/test.js --growl --reporter=spec --bail --exit"
"test:mocha": "mocha node/test/test.mjs --growl --reporter=spec --bail --exit"
},
"types": "node/dist/index.d.ts",
"version": "1.131.5"
"version": "1.134.0"
}

View File

@@ -1,6 +1,5 @@
"""Account class implementation."""
import os
from array import array
from contextlib import contextmanager
@@ -478,6 +477,16 @@ class Account:
msg_ids = [msg.id for msg in messages]
lib.dc_forward_msgs(self._dc_context, msg_ids, len(msg_ids), chat.id)
def resend_messages(self, messages: List[Message]) -> None:
"""Resend list of messages.
:param messages: list of :class:`deltachat.message.Message` object.
:returns: None
"""
msg_ids = [msg.id for msg in messages]
if lib.dc_resend_msgs(self._dc_context, msg_ids, len(msg_ids)) != 1:
raise ValueError(f"could not resend messages {msg_ids}")
def delete_messages(self, messages: List[Message]) -> None:
"""delete messages (local and remote).

View File

@@ -10,6 +10,7 @@ import time
import weakref
import random
from queue import Queue
from threading import Event
from typing import Callable, Dict, List, Optional, Set
import pytest
@@ -590,6 +591,27 @@ class ACFactory:
ac2.create_chat(ac1)
return ac1.create_chat(ac2)
def get_protected_chat(self, ac1: Account, ac2: Account):
class SetupPlugin:
def __init__(self) -> None:
self.member_added = Event()
@account_hookimpl
def ac_member_added(self, chat: deltachat.Chat, contact, actor, message):
self.member_added.set()
setupplugin = SetupPlugin()
ac1.add_account_plugin(setupplugin)
chat = ac1.create_group_chat("Protected Group", verified=True)
qr = chat.get_join_qr()
ac2.qr_join_chat(qr)
setupplugin.member_added.wait()
msg = ac2.wait_next_incoming_message()
assert msg.text == "Messages are guaranteed to be end-to-end encrypted from now on."
msg = ac2.wait_next_incoming_message()
assert "Member Me " in msg.text and " added by " in msg.text
return chat
def introduce_each_other(self, accounts, sending=True):
to_wait = []
for i, acc in enumerate(accounts):

View File

@@ -498,6 +498,26 @@ def test_forward_messages(acfactory, lp):
assert not chat3.get_messages()
def test_forward_encrypted_to_unencrypted(acfactory, lp):
ac1, ac2, ac3 = acfactory.get_online_accounts(3)
chat = acfactory.get_protected_chat(ac1, ac2)
lp.sec("ac1: send encrypted message to ac2")
txt = "This should be encrypted"
chat.send_text(txt)
msg = ac2.wait_next_incoming_message()
assert msg.text == txt
assert msg.is_encrypted()
lp.sec("ac2: forward message to ac3 unencrypted")
unencrypted_chat = ac2.create_chat(ac3)
msg_id = msg.id
msg2 = unencrypted_chat.send_msg(msg)
assert msg2 == msg
assert msg.id != msg_id
assert not msg.is_encrypted()
def test_forward_own_message(acfactory, lp):
ac1, ac2 = acfactory.get_online_accounts(2)
chat = acfactory.get_accepted_chat(ac1, ac2)
@@ -523,6 +543,27 @@ def test_forward_own_message(acfactory, lp):
assert msg_in.is_forwarded()
def test_resend_message(acfactory, lp):
ac1, ac2 = acfactory.get_online_accounts(2)
chat1 = ac1.create_chat(ac2)
lp.sec("ac1: send message to ac2")
chat1.send_text("message")
lp.sec("ac2: receive message")
msg_in = ac2._evtracker.wait_next_incoming_message()
assert msg_in.text == "message"
chat2 = msg_in.chat
chat2_msg_cnt = len(chat2.get_messages())
lp.sec("ac1: resend message")
ac1.resend_messages([msg_in])
lp.sec("ac2: check that message is deleted")
ac2._evtracker.get_matching("DC_EVENT_IMAP_MESSAGE_DELETED")
assert len(chat2.get_messages()) == chat2_msg_cnt
def test_long_group_name(acfactory, lp):
"""See bug https://github.com/deltachat/deltachat-core-rust/issues/3650 "Space added before long
group names after MIME serialization/deserialization".
@@ -1531,10 +1572,11 @@ def test_reactions_for_a_reordering_move(acfactory, lp):
def test_import_export_online_all(acfactory, tmp_path, data, lp):
(ac1,) = acfactory.get_online_accounts(1)
(ac1, some1) = acfactory.get_online_accounts(2)
lp.sec("create some chat content")
chat1 = ac1.create_contact("some1@example.org", name="some1").create_chat()
some1_addr = some1.get_config("addr")
chat1 = ac1.create_contact(some1_addr, name="some1").create_chat()
chat1.send_text("msg1")
assert len(ac1.get_contacts(query="some1")) == 1
@@ -1551,7 +1593,7 @@ def test_import_export_online_all(acfactory, tmp_path, data, lp):
contacts = ac.get_contacts(query="some1")
assert len(contacts) == 1
contact2 = contacts[0]
assert contact2.addr == "some1@example.org"
assert contact2.addr == some1_addr
chat2 = contact2.create_chat()
messages = chat2.get_messages()
assert len(messages) == 3
@@ -1979,6 +2021,32 @@ def test_connectivity(acfactory, lp):
ac1._evtracker.wait_for_connectivity(dc.const.DC_CONNECTIVITY_NOT_CONNECTED)
def test_all_work_done(acfactory, lp):
"""
Tests that calling start_io() immediately followed by maybe_network()
and then waiting for all_work_done() reliably fetches the messages
delivered while account was offline.
In other words, connectivity should not change to a state
where all_work_done() returns true until IMAP connection goes idle.
"""
ac1, ac2 = acfactory.get_online_accounts(2)
ac1.stop_io()
ac1._evtracker.wait_for_connectivity(dc.const.DC_CONNECTIVITY_NOT_CONNECTED)
ac1.direct_imap.select_config_folder("inbox")
with ac1.direct_imap.idle() as idle1:
ac2.create_chat(ac1).send_text("Hi")
idle1.wait_for_new_message()
ac1.start_io()
ac1.maybe_network()
ac1._evtracker.wait_for_all_work_done()
msgs = ac1.create_chat(ac2).get_messages()
assert len(msgs) == 1
assert msgs[0].text == "Hi"
def test_fetch_deleted_msg(acfactory, lp):
"""This is a regression test: Messages with \\Deleted flag were downloaded again and again,
hundreds of times, because uid_next was not updated.

View File

@@ -156,6 +156,8 @@ def test_markseen_invalid_message_ids(acfactory):
chat = contact1.create_chat()
chat.send_text("one message")
ac1._evtracker.get_matching("DC_EVENT_MSGS_CHANGED")
# Skip configuration-related warnings, but not errors.
ac1._evtracker.ensure_event_not_queued("DC_EVENT_ERROR")
msg_ids = [9]
lib.dc_markseen_msgs(ac1._dc_context, msg_ids, len(msg_ids))
ac1._evtracker.ensure_event_not_queued("DC_EVENT_WARNING|DC_EVENT_ERROR")

View File

@@ -1 +1 @@
2023-11-20
2024-01-31

View File

@@ -102,7 +102,7 @@ def main():
found = True
if not found:
raise SystemExit(
f"{changelog_name} contains no entry for version: {newversion}"
f"CHANGELOG.md contains no entry for version: {newversion}"
)
for toml_filename in toml_list:

View File

@@ -5,14 +5,18 @@ use std::future::Future;
use std::path::{Path, PathBuf};
use anyhow::{ensure, Context as _, Result};
use futures::future::join_all;
use serde::{Deserialize, Serialize};
use tokio::fs;
use tokio::io::AsyncWriteExt;
use tokio::sync::oneshot;
use tokio::task::JoinHandle;
use tokio::time::{sleep, Duration};
use uuid::Uuid;
#[cfg(not(target_os = "ios"))]
use tokio::sync::oneshot;
#[cfg(not(target_os = "ios"))]
use tokio::time::{sleep, Duration};
use crate::context::Context;
use crate::events::{Event, EventEmitter, EventType, Events};
use crate::stock_str::StockStrings;
@@ -288,6 +292,42 @@ impl Accounts {
}
}
/// Performs a background fetch for all accounts in parallel.
///
/// This is an auxiliary function and not part of public API.
/// Use [Accounts::background_fetch] instead.
async fn background_fetch_without_timeout(&self) {
async fn background_fetch_and_log_error(account: Context) {
if let Err(error) = account.background_fetch().await {
warn!(account, "{error:#}");
}
}
join_all(
self.accounts
.values()
.cloned()
.map(background_fetch_and_log_error),
)
.await;
}
/// Performs a background fetch for all accounts in parallel with a timeout.
///
/// The `AccountsBackgroundFetchDone` event is emitted at the end,
/// process all events until you get this one and you can safely return to the background
/// without forgetting to create notifications caused by timing race conditions.
pub async fn background_fetch(&self, timeout: std::time::Duration) {
if let Err(_err) =
tokio::time::timeout(timeout, self.background_fetch_without_timeout()).await
{
self.emit_event(EventType::Warning(
"Background fetch timed out.".to_string(),
));
}
self.emit_event(EventType::AccountsBackgroundFetchDone);
}
/// Emits a single event.
pub fn emit_event(&self, event: EventType) {
self.events.emit(Event { id: 0, typ: event })
@@ -303,6 +343,7 @@ impl Accounts {
const CONFIG_NAME: &str = "accounts.toml";
/// Lockfile name.
#[cfg(not(target_os = "ios"))]
const LOCKFILE_NAME: &str = "accounts.lock";
/// Database file name.
@@ -338,22 +379,16 @@ impl Drop for Config {
}
impl Config {
/// Creates a new Config for `file`, but doesn't open/sync it.
async fn new_nosync(file: PathBuf, lock: bool) -> Result<Self> {
let dir = file.parent().context("Cannot get config file directory")?;
let inner = InnerConfig {
accounts: Vec::new(),
selected_account: 0,
next_id: 1,
};
if !lock {
let cfg = Self {
file,
inner,
lock_task: None,
};
return Ok(cfg);
}
#[cfg(target_os = "ios")]
async fn create_lock_task(_dir: PathBuf) -> Result<Option<JoinHandle<anyhow::Result<()>>>> {
// Do not lock accounts.toml on iOS.
// This results in 0xdead10cc crashes on suspend.
// iOS itself ensures that multiple instances of Delta Chat are not running.
Ok(None)
}
#[cfg(not(target_os = "ios"))]
async fn create_lock_task(dir: PathBuf) -> Result<Option<JoinHandle<anyhow::Result<()>>>> {
let lockfile = dir.join(LOCKFILE_NAME);
let mut lock = fd_lock::RwLock::new(fs::File::create(lockfile).await?);
let (locked_tx, locked_rx) = oneshot::channel();
@@ -384,12 +419,32 @@ impl Config {
rx.await?;
Ok(())
});
locked_rx.await?;
Ok(Some(lock_task))
}
/// Creates a new Config for `file`, but doesn't open/sync it.
async fn new_nosync(file: PathBuf, lock: bool) -> Result<Self> {
let dir = file.parent().context("Cannot get config file directory")?;
let inner = InnerConfig {
accounts: Vec::new(),
selected_account: 0,
next_id: 1,
};
if !lock {
let cfg = Self {
file,
inner,
lock_task: None,
};
return Ok(cfg);
}
let lock_task = Self::create_lock_task(dir.to_path_buf()).await?;
let cfg = Self {
file,
inner,
lock_task: Some(lock_task),
lock_task,
};
locked_rx.await?;
Ok(cfg)
}
@@ -406,11 +461,13 @@ impl Config {
/// Takes a mutable reference because the saved file is a part of the `Config` state. This
/// protects from parallel calls resulting to a wrong file contents.
async fn sync(&mut self) -> Result<()> {
#[cfg(not(target_os = "ios"))]
ensure!(!self
.lock_task
.as_ref()
.context("Config is read-only")?
.is_finished());
let tmp_path = self.file.with_extension("toml.tmp");
let mut file = fs::File::create(&tmp_path)
.await
@@ -526,8 +583,12 @@ impl Config {
}
if self.inner.selected_account == id {
// reset selected account
self.inner.selected_account =
self.inner.accounts.get(0).map(|e| e.id).unwrap_or_default();
self.inner.selected_account = self
.inner
.accounts
.first()
.map(|e| e.id)
.unwrap_or_default();
}
}

File diff suppressed because it is too large Load Diff

View File

@@ -5,6 +5,7 @@ use std::path::Path;
use std::str::FromStr;
use anyhow::{ensure, Context as _, Result};
use serde::{Deserialize, Serialize};
use strum::{EnumProperty, IntoEnumIterator};
use strum_macros::{AsRefStr, Display, EnumIter, EnumProperty, EnumString};
@@ -13,9 +14,11 @@ use crate::constants::DC_VERSION_STR;
use crate::contact::addr_cmp;
use crate::context::Context;
use crate::events::EventType;
use crate::log::LogExt;
use crate::mimefactory::RECOMMENDED_FILE_SIZE;
use crate::provider::{get_provider_by_id, Provider};
use crate::tools::{get_abs_path, improve_single_line_input, EmailAddress};
use crate::sync::{self, Sync::*, SyncData};
use crate::tools::{get_abs_path, improve_single_line_input};
/// The available configuration keys.
#[derive(
@@ -31,6 +34,8 @@ use crate::tools::{get_abs_path, improve_single_line_input, EmailAddress};
EnumProperty,
PartialOrd,
Ord,
Serialize,
Deserialize,
)]
#[strum(serialize_all = "snake_case")]
pub enum Config {
@@ -338,6 +343,28 @@ pub enum Config {
/// until `chat_id.accept()` is called.
#[strum(props(default = "0"))]
VerifiedOneOnOneChats,
/// Row ID of the key in the `keypairs` table
/// used for signatures, encryption to self and included in `Autocrypt` header.
KeyId,
}
impl Config {
/// Whether the config option is synced across devices.
///
/// This must be checked on both sides so that if there are different client versions, the
/// synchronisation of a particular option is either done or not done in both directions.
/// Moreover, receivers of a config value need to check if a key can be synced because some
/// settings (e.g. Avatar path) could otherwise lead to exfiltration of files from a receiver's
/// device if we assume an attacker to have control of a device in a multi-device setting or if
/// multiple users are sharing an account. Another example is `Self::SyncMsgs` itself which
/// mustn't be controlled by other devices.
pub(crate) fn is_synced(&self) -> bool {
matches!(
self,
Self::Displayname | Self::MdnsEnabled | Self::ShowEmails
)
}
}
impl Context {
@@ -460,6 +487,16 @@ impl Context {
/// Set the given config key.
/// If `None` is passed as a value the value is cleared and set to the default if there is one.
pub async fn set_config(&self, key: Config, value: Option<&str>) -> Result<()> {
self.set_config_ex(key.is_synced().into(), key, value).await
}
pub(crate) async fn set_config_ex(
&self,
sync: sync::Sync,
key: Config,
mut value: Option<&str>,
) -> Result<()> {
let better_value;
match key {
Config::Selfavatar => {
self.sql
@@ -486,10 +523,11 @@ impl Context {
ret?
}
Config::Displayname => {
let value = value.map(improve_single_line_input);
self.sql
.set_raw_config(key.as_ref(), value.as_deref())
.await?;
if let Some(v) = value {
better_value = improve_single_line_input(v);
value = Some(&better_value);
}
self.sql.set_raw_config(key.as_ref(), value).await?;
}
Config::Socks5Enabled
| Config::BccSelf
@@ -522,6 +560,24 @@ impl Context {
self.sql.set_raw_config(key.as_ref(), value).await?;
}
}
if sync != Sync {
return Ok(());
}
self.emit_event(EventType::ConfigSynced { key });
let Some(val) = value else {
return Ok(());
};
let val = val.to_string();
if self
.add_sync_item(SyncData::Config { key, val })
.await
.log_err(self)
.is_err()
{
return Ok(());
}
self.send_sync_msg().await.log_err(self).ok();
Ok(())
}
@@ -576,8 +632,6 @@ impl Context {
///
/// This should only be used by test code and during configure.
pub(crate) async fn set_primary_self_addr(&self, primary_new: &str) -> Result<()> {
let old_addr = self.get_config(Config::ConfiguredAddr).await?;
// add old primary address (if exists) to secondary addresses
let mut secondary_addrs = self.get_all_self_addrs().await?;
// never store a primary address also as a secondary
@@ -591,17 +645,6 @@ impl Context {
self.set_config(Config::ConfiguredAddr, Some(primary_new))
.await?;
if let Some(old_addr) = old_addr {
let old_addr = EmailAddress::new(&old_addr)?;
let old_keypair = crate::key::load_keypair(self, &old_addr).await?;
if let Some(mut old_keypair) = old_keypair {
old_keypair.addr = EmailAddress::new(primary_new)?;
crate::key::store_self_keypair(self, &old_keypair, crate::key::KeyPairUse::Default)
.await?;
}
}
Ok(())
}
@@ -653,7 +696,7 @@ mod tests {
use super::*;
use crate::constants;
use crate::test_utils::TestContext;
use crate::test_utils::{sync, TestContext};
#[test]
fn test_to_string() {
@@ -797,4 +840,98 @@ mod tests {
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_sync() -> Result<()> {
let alice0 = TestContext::new_alice().await;
let alice1 = TestContext::new_alice().await;
for a in [&alice0, &alice1] {
a.set_config_bool(Config::SyncMsgs, true).await?;
}
let mdns_enabled = alice0.get_config_bool(Config::MdnsEnabled).await?;
// Alice1 has a different config value.
alice1
.set_config_bool(Config::MdnsEnabled, !mdns_enabled)
.await?;
// This changes nothing, but still sends a sync message.
alice0
.set_config_bool(Config::MdnsEnabled, mdns_enabled)
.await?;
sync(&alice0, &alice1).await;
assert_eq!(
alice1.get_config_bool(Config::MdnsEnabled).await?,
mdns_enabled
);
// Reset to default. Test that it's not synced because defaults may differ across client
// versions.
alice0.set_config(Config::MdnsEnabled, None).await?;
assert_eq!(alice0.get_config_bool(Config::MdnsEnabled).await?, true);
alice0
.evtracker
.get_matching(|e| {
matches!(
e,
EventType::ConfigSynced {
key: Config::MdnsEnabled
}
)
})
.await;
alice0.set_config_bool(Config::MdnsEnabled, false).await?;
alice0
.evtracker
.get_matching(|e| {
matches!(
e,
EventType::ConfigSynced {
key: Config::MdnsEnabled
}
)
})
.await;
sync(&alice0, &alice1).await;
assert_eq!(alice1.get_config_bool(Config::MdnsEnabled).await?, false);
alice1
.evtracker
.get_matching(|e| {
matches!(
e,
EventType::ConfigSynced {
key: Config::MdnsEnabled
}
)
})
.await;
let show_emails = alice0.get_config_bool(Config::ShowEmails).await?;
alice0
.set_config_bool(Config::ShowEmails, !show_emails)
.await?;
sync(&alice0, &alice1).await;
assert_eq!(
alice1.get_config_bool(Config::ShowEmails).await?,
!show_emails
);
// `Config::SyncMsgs` mustn't be synced.
alice0.set_config_bool(Config::SyncMsgs, false).await?;
alice0.set_config_bool(Config::SyncMsgs, true).await?;
alice0.set_config_bool(Config::MdnsEnabled, true).await?;
sync(&alice0, &alice1).await;
assert!(alice1.get_config_bool(Config::MdnsEnabled).await?);
// Usual sync scenario.
alice0
.set_config(Config::Displayname, Some("Alice Sync"))
.await?;
sync(&alice0, &alice1).await;
assert_eq!(
alice1.get_config(Config::Displayname).await?,
Some("Alice Sync".to_string())
);
Ok(())
}
}

View File

@@ -34,6 +34,7 @@ use crate::provider::{Protocol, Socket, UsernamePattern};
use crate::smtp::Smtp;
use crate::socks::Socks5Config;
use crate::stock_str;
use crate::sync::Sync::*;
use crate::tools::{time, EmailAddress};
use crate::{chat, e2ee, provider};
@@ -132,7 +133,9 @@ async fn on_configure_completed(
for def in config_defaults {
if !context.config_exists(def.key).await? {
info!(context, "apply config_defaults {}={}", def.key, def.value);
context.set_config(def.key, Some(def.value)).await?;
context
.set_config_ex(Nosync, def.key, Some(def.value))
.await?;
} else {
info!(
context,

View File

@@ -1,6 +1,7 @@
//! # Thunderbird's Autoconfiguration implementation
//!
//! Documentation: <https://web.archive.org/web/20210624004729/https://developer.mozilla.org/en-US/docs/Mozilla/Thunderbird/Autoconfiguration>
//! RFC draft: <https://www.ietf.org/archive/id/draft-bucksch-autoconfig-00.html>
//! Archived original documentation: <https://web.archive.org/web/20210624004729/https://developer.mozilla.org/en-US/docs/Mozilla/Thunderbird/Autoconfiguration>
use std::io::BufRead;
use std::str::FromStr;

View File

@@ -137,20 +137,11 @@ impl ServerParams {
}
fn expand_strict_tls(self) -> Vec<ServerParams> {
if self.strict_tls.is_none() {
vec![
Self {
strict_tls: Some(true), // Strict.
..self.clone()
},
Self {
strict_tls: None, // Automatic.
..self
},
]
} else {
vec![self]
}
vec![Self {
// Strict if not set by the user or provider database.
strict_tls: Some(self.strict_tls.unwrap_or(true)),
..self
}]
}
}
@@ -162,31 +153,10 @@ pub(crate) fn expand_param_vector(
domain: &str,
) -> Vec<ServerParams> {
v.into_iter()
.map(|params| {
if params.socket == Socket::Plain {
ServerParams {
// Avoid expanding plaintext configuration into configuration with and without
// `strict_tls` if `strict_tls` is set to `None` as `strict_tls` is not used for
// plaintext connections. Always setting it to "enabled", just in case.
strict_tls: Some(true),
..params
}
} else {
params
}
})
// The order of expansion is important.
//
// Ports are expanded the last, so they are changed the first. Username is only changed if
// default value (address with domain) didn't work for all available hosts and ports.
//
// Strict TLS must be expanded first, so we try all configurations with strict TLS first
// and only then try again without strict TLS. Otherwise we may lock to wrong hostname
// without strict TLS when another hostname with strict TLS is available. For example, if
// both smtp.example.net and mail.example.net are running an SMTP server, but both use a
// certificate that is only valid for mail.example.net, we want to skip smtp.example.net
// and use mail.example.net with strict TLS instead of using smtp.example.net without
// strict TLS.
.flat_map(|params| params.expand_strict_tls().into_iter())
.flat_map(|params| params.expand_usernames(addr).into_iter())
.flat_map(|params| params.expand_hostnames(domain).into_iter())
@@ -257,22 +227,6 @@ mod tests {
username: "foobar".to_string(),
strict_tls: Some(true)
},
ServerParams {
protocol: Protocol::Smtp,
hostname: "example.net".to_string(),
port: 123,
socket: Socket::Ssl,
username: "foobar".to_string(),
strict_tls: None,
},
ServerParams {
protocol: Protocol::Smtp,
hostname: "example.net".to_string(),
port: 123,
socket: Socket::Starttls,
username: "foobar".to_string(),
strict_tls: None
}
],
);
@@ -284,7 +238,7 @@ mod tests {
port: 123,
socket: Socket::Plain,
username: "foobar".to_string(),
strict_tls: None,
strict_tls: Some(true),
}],
"foobar@example.net",
"example.net",

View File

@@ -209,6 +209,14 @@ pub const WORSE_IMAGE_SIZE: u32 = 640;
// this value can be increased if the folder configuration is changed and must be redone on next program start
pub(crate) const DC_FOLDERS_CONFIGURED_VERSION: i32 = 4;
// If more recipients are needed in SMTP's `RCPT TO:` header, the recipient list is split into
// chunks. This does not affect MIME's `To:` header. Can be overwritten by setting
// `max_smtp_rcpt_to` in the provider db.
pub(crate) const DEFAULT_MAX_SMTP_RCPT_TO: usize = 50;
/// How far the last quota check needs to be in the past to be checked by the background function (in seconds).
pub(crate) const DC_BACKGROUND_FETCH_QUOTA_CHECK_RATELIMIT: i64 = 12 * 60 * 60; // 12 hours
#[cfg(test)]
mod tests {
use num_traits::FromPrimitive;

View File

@@ -26,13 +26,14 @@ use crate::constants::{Blocked, Chattype, DC_GCL_ADD_SELF, DC_GCL_VERIFIED_ONLY}
use crate::context::Context;
use crate::events::EventType;
use crate::key::{load_self_public_key, DcKey};
use crate::log::LogExt;
use crate::login_param::LoginParam;
use crate::message::MessageState;
use crate::mimeparser::AvatarAction;
use crate::param::{Param, Params};
use crate::peerstate::{Peerstate, PeerstateVerifiedStatus};
use crate::peerstate::Peerstate;
use crate::sql::{self, params_iter};
use crate::sync::{self, Sync::*, SyncData};
use crate::sync::{self, Sync::*};
use crate::tools::{
duration_to_str, get_abs_path, improve_single_line_input, strip_rtlo_characters, time,
EmailAddress,
@@ -348,24 +349,6 @@ pub(crate) enum Modifier {
Created,
}
/// Verification status of the contact.
#[derive(Debug, PartialEq, Eq, Clone, Copy, FromPrimitive)]
#[repr(u8)]
pub enum VerifiedStatus {
/// Contact is not verified.
Unverified = 0,
/// SELF has verified the fingerprint of a contact. Currently unused.
Verified = 1,
/// SELF and contact have verified their fingerprints in both directions; in the UI typically checkmarks are shown.
BidirectVerified = 2,
}
impl Default for VerifiedStatus {
fn default() -> Self {
Self::Unverified
}
}
impl Contact {
/// Loads a single contact object from the database.
///
@@ -494,6 +477,15 @@ impl Contact {
///
/// May result in a `#DC_EVENT_CONTACTS_CHANGED` event.
pub async fn create(context: &Context, name: &str, addr: &str) -> Result<ContactId> {
Self::create_ex(context, Sync, name, addr).await
}
pub(crate) async fn create_ex(
context: &Context,
sync: sync::Sync,
name: &str,
addr: &str,
) -> Result<ContactId> {
let name = improve_single_line_input(name);
let (name, addr) = sanitize_name_and_addr(&name, addr);
@@ -514,6 +506,16 @@ impl Contact {
set_blocked(context, Nosync, contact_id, false).await?;
}
if sync.into() {
chat::sync(
context,
chat::SyncId::ContactAddr(addr.to_string()),
chat::SyncAction::Rename(name.to_string()),
)
.await
.log_err(context)
.ok();
}
Ok(contact_id)
}
@@ -790,7 +792,7 @@ impl Contact {
sth_modified = Modifier::Created;
row_id = u32::try_from(transaction.last_insert_rowid())?;
info!(context, "added contact id={} addr={}", row_id, &addr);
info!(context, "Added contact id={row_id} addr={addr}.");
}
Ok(row_id)
}).await?;
@@ -1055,11 +1057,9 @@ impl Contact {
let loginparam = LoginParam::load_configured_params(context).await?;
let peerstate = Peerstate::from_addr(context, &contact.addr).await?;
if let Some(peerstate) = peerstate.filter(|peerstate| {
peerstate
.peek_key(PeerstateVerifiedStatus::Unverified)
.is_some()
}) {
if let Some(peerstate) =
peerstate.filter(|peerstate| peerstate.peek_key(false).is_some())
{
let stock_message = match peerstate.prefer_encrypt {
EncryptPreference::Mutual => stock_str::e2e_preferred(context).await,
EncryptPreference::NoPreference => stock_str::e2e_available(context).await,
@@ -1074,11 +1074,11 @@ impl Contact {
.fingerprint()
.to_string();
let fingerprint_other_verified = peerstate
.peek_key(PeerstateVerifiedStatus::BidirectVerified)
.peek_key(true)
.map(|k| k.fingerprint().to_string())
.unwrap_or_default();
let fingerprint_other_unverified = peerstate
.peek_key(PeerstateVerifiedStatus::Unverified)
.peek_key(false)
.map(|k| k.fingerprint().to_string())
.unwrap_or_default();
if loginparam.addr < peerstate.addr {
@@ -1281,20 +1281,37 @@ impl Contact {
/// otherwise use is_chat_protected().
/// Use [Self::get_verifier_id] to display the verifier contact
/// in the info section of the contact profile.
pub async fn is_verified(&self, context: &Context) -> Result<VerifiedStatus> {
pub async fn is_verified(&self, context: &Context) -> Result<bool> {
// We're always sort of secured-verified as we could verify the key on this device any time with the key
// on this device
if self.id == ContactId::SELF {
return Ok(VerifiedStatus::BidirectVerified);
return Ok(true);
}
if let Some(peerstate) = Peerstate::from_addr(context, &self.addr).await? {
if peerstate.is_using_verified_key() {
return Ok(VerifiedStatus::BidirectVerified);
}
let Some(peerstate) = Peerstate::from_addr(context, &self.addr).await? else {
return Ok(false);
};
let forward_verified = peerstate.is_using_verified_key();
let backward_verified = peerstate.is_backward_verified(context).await?;
Ok(forward_verified && backward_verified)
}
/// Returns true if we have a verified key for the contact
/// and it is the same as Autocrypt key.
/// This is enough to send messages to the contact in verified chat
/// and verify received messages, but not enough to display green checkmark
/// or add the contact to verified groups.
pub async fn is_forward_verified(&self, context: &Context) -> Result<bool> {
if self.id == ContactId::SELF {
return Ok(true);
}
Ok(VerifiedStatus::Unverified)
let Some(peerstate) = Peerstate::from_addr(context, &self.addr).await? else {
return Ok(false);
};
Ok(peerstate.is_using_verified_key())
}
/// Returns the `ContactId` that verified the contact.
@@ -1349,7 +1366,7 @@ impl Contact {
Ok(chat_id.is_protected(context).await? == ProtectionStatus::Protected)
} else {
// 1:1 chat does not exist.
Ok(self.is_verified(context).await? == VerifiedStatus::BidirectVerified)
Ok(self.is_verified(context).await?)
}
}
@@ -1500,13 +1517,14 @@ WHERE type=? AND id IN (
true => chat::SyncAction::Block,
false => chat::SyncAction::Unblock,
};
context
.add_sync_item(SyncData::AlterChat {
id: chat::SyncId::ContactAddr(contact.addr.clone()),
action,
})
.await?;
context.send_sync_msg().await?;
chat::sync(
context,
chat::SyncId::ContactAddr(contact.addr.clone()),
action,
)
.await
.log_err(context)
.ok();
}
}
@@ -1905,12 +1923,12 @@ mod tests {
// Search by name.
let contacts = Contact::get_all(&context.ctx, 0, Some("bob")).await?;
assert_eq!(contacts.len(), 1);
assert_eq!(contacts.get(0), Some(&id));
assert_eq!(contacts.first(), Some(&id));
// Search by address.
let contacts = Contact::get_all(&context.ctx, 0, Some("user")).await?;
assert_eq!(contacts.len(), 1);
assert_eq!(contacts.get(0), Some(&id));
assert_eq!(contacts.first(), Some(&id));
let contacts = Contact::get_all(&context.ctx, 0, Some("alice")).await?;
assert_eq!(contacts.len(), 0);
@@ -1937,7 +1955,7 @@ mod tests {
// Search by display name (same as manually set name).
let contacts = Contact::get_all(&context.ctx, 0, Some("someone")).await?;
assert_eq!(contacts.len(), 1);
assert_eq!(contacts.get(0), Some(&id));
assert_eq!(contacts.first(), Some(&id));
Ok(())
}
@@ -2809,4 +2827,33 @@ Hi."#;
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_sync_create() -> Result<()> {
let alice0 = &TestContext::new_alice().await;
let alice1 = &TestContext::new_alice().await;
for a in [alice0, alice1] {
a.set_config_bool(Config::SyncMsgs, true).await?;
}
Contact::create(alice0, "Bob", "bob@example.net").await?;
test_utils::sync(alice0, alice1).await;
let a1b_contact_id =
Contact::lookup_id_by_addr(alice1, "bob@example.net", Origin::ManuallyCreated)
.await?
.unwrap();
let a1b_contact = Contact::get_by_id(alice1, a1b_contact_id).await?;
assert_eq!(a1b_contact.name, "Bob");
Contact::create(alice0, "Bob Renamed", "bob@example.net").await?;
test_utils::sync(alice0, alice1).await;
let id = Contact::lookup_id_by_addr(alice1, "bob@example.net", Origin::ManuallyCreated)
.await?
.unwrap();
assert_eq!(id, a1b_contact_id);
let a1b_contact = Contact::get_by_id(alice1, a1b_contact_id).await?;
assert_eq!(a1b_contact.name, "Bob Renamed");
Ok(())
}
}

View File

@@ -15,15 +15,16 @@ use tokio::sync::{Mutex, Notify, RwLock};
use crate::chat::{get_chat_cnt, ChatId};
use crate::config::Config;
use crate::constants::DC_VERSION_STR;
use crate::constants::{DC_BACKGROUND_FETCH_QUOTA_CHECK_RATELIMIT, DC_VERSION_STR};
use crate::contact::Contact;
use crate::debug_logging::DebugLogging;
use crate::events::{Event, EventEmitter, EventType, Events};
use crate::imap::{FolderMeaning, Imap, ServerMetadata};
use crate::key::{load_self_public_key, DcKey as _};
use crate::login_param::LoginParam;
use crate::message::{self, MessageState, MsgId};
use crate::quota::QuotaInfo;
use crate::scheduler::SchedulerState;
use crate::scheduler::{convert_folder_meaning, SchedulerState};
use crate::sql::Sql;
use crate::stock_str::StockStrings;
use crate::timesmearing::SmearedTimestamp;
@@ -224,6 +225,9 @@ pub struct InnerContext {
/// <https://datatracker.ietf.org/doc/html/rfc2971>
pub(crate) server_id: RwLock<Option<HashMap<String, String>>>,
/// IMAP METADATA.
pub(crate) metadata: RwLock<Option<ServerMetadata>>,
pub(crate) last_full_folder_scan: Mutex<Option<Instant>>,
/// ID for this `Context` in the current process.
@@ -384,6 +388,7 @@ impl Context {
resync_request: AtomicBool::new(false),
new_msgs_notify,
server_id: RwLock::new(None),
metadata: RwLock::new(None),
creation_time: std::time::SystemTime::now(),
last_full_folder_scan: Mutex::new(None),
last_error: std::sync::RwLock::new("".to_string()),
@@ -436,6 +441,55 @@ impl Context {
self.scheduler.maybe_network().await;
}
/// Does a background fetch
/// pauses the scheduler and does one imap fetch, then unpauses and returns
pub async fn background_fetch(&self) -> Result<()> {
if !(self.is_configured().await?) {
return Ok(());
}
let address = self.get_primary_self_addr().await?;
let time_start = std::time::SystemTime::now();
info!(self, "background_fetch started fetching {address}");
let _pause_guard = self.scheduler.pause(self.clone()).await?;
// connection
let mut connection = Imap::new_configured(self, channel::bounded(1).1).await?;
connection.prepare(self).await?;
// fetch imap folders
for folder_meaning in [FolderMeaning::Inbox, FolderMeaning::Mvbox] {
let (_, watch_folder) = convert_folder_meaning(self, folder_meaning).await?;
connection
.fetch_move_delete(self, &watch_folder, folder_meaning)
.await?;
}
// update quota (to send warning if full) - but only check it once in a while
let quota_needs_update = {
let quota = self.quota.read().await;
quota
.as_ref()
.filter(|quota| quota.modified + DC_BACKGROUND_FETCH_QUOTA_CHECK_RATELIMIT > time())
.is_none()
};
if quota_needs_update {
if let Err(err) = self.update_recent_quota(&mut connection).await {
warn!(self, "Failed to update quota: {err:#}.");
}
}
info!(
self,
"background_fetch done for {address} took {:?}",
time_start.elapsed().unwrap_or_default()
);
Ok(())
}
pub(crate) async fn schedule_resync(&self) -> Result<()> {
self.resync_request.store(true, Ordering::Relaxed);
self.scheduler.interrupt_inbox().await;
@@ -669,6 +723,16 @@ impl Context {
res.insert("imap_server_id", format!("{server_id:?}"));
}
if let Some(metadata) = &*self.metadata.read().await {
if let Some(comment) = &metadata.comment {
res.insert("imap_server_comment", format!("{comment:?}"));
}
if let Some(admin) = &metadata.admin {
res.insert("imap_server_admin", format!("{admin:?}"));
}
}
res.insert("secondary_addrs", secondary_addrs);
res.insert(
"fetch_existing_msgs",
@@ -1318,6 +1382,7 @@ mod tests {
"socks5_port",
"socks5_user",
"socks5_password",
"key_id",
];
let t = TestContext::new().await;
let info = t.get_info().await.unwrap();
@@ -1369,7 +1434,7 @@ mod tests {
assert_eq!(res.len(), 2);
// Message added later is returned first.
assert_eq!(res.get(0), Some(&msg2.id));
assert_eq!(res.first(), Some(&msg2.id));
assert_eq!(res.get(1), Some(&msg1.id));
// Global search with longer text does not find any message.
@@ -1586,7 +1651,7 @@ mod tests {
let bob_next_msg_ids = bob.get_next_msgs().await?;
assert_eq!(bob_next_msg_ids.len(), 1);
assert_eq!(bob_next_msg_ids.get(0), Some(&received_msg.id));
assert_eq!(bob_next_msg_ids.first(), Some(&received_msg.id));
bob.set_config_u32(Config::LastMsgId, received_msg.id.to_u32())
.await?;
@@ -1595,7 +1660,7 @@ mod tests {
// Next messages include self-sent messages.
let alice_next_msg_ids = alice.get_next_msgs().await?;
assert_eq!(alice_next_msg_ids.len(), 1);
assert_eq!(alice_next_msg_ids.get(0), Some(&sent_msg.sender_msg_id));
assert_eq!(alice_next_msg_ids.first(), Some(&sent_msg.sender_msg_id));
alice
.set_config_u32(Config::LastMsgId, sent_msg.sender_msg_id.to_u32())

View File

@@ -54,7 +54,7 @@ pub async fn debug_logging_loop(context: &Context, events: Receiver<DebugEventLo
match context
.write_status_update_inner(
&msg_id,
StatusUpdateItem {
&StatusUpdateItem {
payload: json!({
"event": event,
"time": time,
@@ -62,6 +62,7 @@ pub async fn debug_logging_loop(context: &Context, events: Receiver<DebugEventLo
info: None,
summary: None,
document: None,
uid: None,
},
)
.await
@@ -70,10 +71,17 @@ pub async fn debug_logging_loop(context: &Context, events: Receiver<DebugEventLo
eprintln!("Can't log event to webxdc status update: {err:#}");
}
Ok(serial) => {
context.emit_event(EventType::WebxdcStatusUpdate {
msg_id,
status_update_serial: serial,
});
if let Some(serial) = serial {
if !matches!(event, EventType::WebxdcStatusUpdate { .. }) {
context.emit_event(EventType::WebxdcStatusUpdate {
msg_id,
status_update_serial: serial,
});
}
} else {
// This should not happen as the update has no `uid`.
error!(context, "Debug logging update is not created.");
};
}
}
}

View File

@@ -23,32 +23,14 @@ use crate::pgp;
///
/// If the message is wrongly signed, HashSet will be empty.
pub fn try_decrypt(
context: &Context,
mail: &ParsedMail<'_>,
private_keyring: &[SignedSecretKey],
public_keyring_for_validate: &[SignedPublicKey],
) -> Result<Option<(Vec<u8>, HashSet<Fingerprint>)>> {
let encrypted_data_part = match {
let mime = get_autocrypt_mime(mail);
if mime.is_some() {
info!(context, "Detected Autocrypt-mime message.");
}
mime
}
.or_else(|| {
let mime = get_mixed_up_mime(mail);
if mime.is_some() {
info!(context, "Detected mixed-up mime message.");
}
mime
})
.or_else(|| {
let mime = get_attachment_mime(mail);
if mime.is_some() {
info!(context, "Detected attached Autocrypt-mime message.");
}
mime
}) {
let encrypted_data_part = match get_autocrypt_mime(mail)
.or_else(|| get_mixed_up_mime(mail))
.or_else(|| get_attachment_mime(mail))
{
None => return Ok(None),
Some(res) => res,
};

View File

@@ -136,39 +136,36 @@ pub(crate) async fn download_msg(context: &Context, msg_id: MsgId, imap: &mut Im
let row = context
.sql
.query_row_optional(
"SELECT uid, folder FROM imap WHERE rfc724_mid=? AND target!=''",
"SELECT uid, folder, uidvalidity FROM imap WHERE rfc724_mid=? AND target!=''",
(&msg.rfc724_mid,),
|row| {
let server_uid: u32 = row.get(0)?;
let server_folder: String = row.get(1)?;
Ok((server_uid, server_folder))
let uidvalidity: u32 = row.get(2)?;
Ok((server_uid, server_folder, uidvalidity))
},
)
.await?;
if let Some((server_uid, server_folder)) = row {
match imap
.fetch_single_msg(context, &server_folder, server_uid, msg.rfc724_mid.clone())
.await
{
ImapActionResult::RetryLater | ImapActionResult::Failed => {
msg.id
.update_download_state(context, DownloadState::Failure)
.await?;
Err(anyhow!("Call download_full() again to try over."))
}
ImapActionResult::Success => {
// update_download_state() not needed as receive_imf() already
// set the state and emitted the event.
Ok(())
}
}
} else {
let Some((server_uid, server_folder, uidvalidity)) = row else {
// No IMAP record found, we don't know the UID and folder.
msg.id
.update_download_state(context, DownloadState::Failure)
.await?;
Err(anyhow!("Call download_full() again to try over."))
return Err(anyhow!("Call download_full() again to try over."));
};
match imap
.fetch_single_msg(
context,
&server_folder,
uidvalidity,
server_uid,
msg.rfc724_mid.clone(),
)
.await
{
ImapActionResult::RetryLater | ImapActionResult::Failed => {
Err(anyhow!("Call download_full() again to try over."))
}
ImapActionResult::Success => Ok(()),
}
}
@@ -181,6 +178,7 @@ impl Imap {
&mut self,
context: &Context,
folder: &str,
uidvalidity: u32,
uid: u32,
rfc724_mid: String,
) -> ImapActionResult {
@@ -197,7 +195,15 @@ impl Imap {
let mut uid_message_ids: BTreeMap<u32, String> = BTreeMap::new();
uid_message_ids.insert(uid, rfc724_mid);
let (last_uid, _received) = match self
.fetch_many_msgs(context, folder, vec![uid], &uid_message_ids, false, false)
.fetch_many_msgs(
context,
folder,
uidvalidity,
vec![uid],
&uid_message_ids,
false,
false,
)
.await
{
Ok(res) => res,
@@ -257,7 +263,7 @@ mod tests {
use crate::chat::{get_chat_msgs, send_msg};
use crate::ephemeral::Timer;
use crate::message::Viewtype;
use crate::receive_imf::receive_imf_inner;
use crate::receive_imf::receive_imf_from_inbox;
use crate::test_utils::TestContext;
#[test]
@@ -338,7 +344,7 @@ mod tests {
Date: Sun, 22 Mar 2020 22:37:57 +0000\
Content-Type: text/plain";
receive_imf_inner(
receive_imf_from_inbox(
&t,
"Mr.12345678901@example.com",
header.as_bytes(),
@@ -354,7 +360,7 @@ mod tests {
.get_text()
.contains(&stock_str::partial_download_msg_body(&t, 100000).await));
receive_imf_inner(
receive_imf_from_inbox(
&t,
"Mr.12345678901@example.com",
format!("{header}\n\n100k text...").as_bytes(),
@@ -383,7 +389,7 @@ mod tests {
.await?;
// download message from bob partially, this must not change the ephemeral timer
receive_imf_inner(
receive_imf_from_inbox(
&t,
"first@example.org",
b"From: Bob <bob@example.org>\n\
@@ -426,7 +432,7 @@ mod tests {
let sent2_rfc724_mid = sent2.load_from_db().await.rfc724_mid;
// not downloading the status update results in an placeholder
receive_imf_inner(
receive_imf_from_inbox(
&bob,
&sent2_rfc724_mid,
sent2.payload().as_bytes(),
@@ -442,7 +448,7 @@ mod tests {
// downloading the status update afterwards expands to nothing and moves the placeholder to trash-chat
// (usually status updates are too small for not being downloaded directly)
receive_imf_inner(
receive_imf_from_inbox(
&bob,
&sent2_rfc724_mid,
sent2.payload().as_bytes(),
@@ -493,7 +499,7 @@ mod tests {
";
// not downloading the mdn results in an placeholder
receive_imf_inner(
receive_imf_from_inbox(
&bob,
"bar@example.org",
raw,
@@ -509,7 +515,7 @@ mod tests {
// downloading the mdn afterwards expands to nothing and deletes the placeholder directly
// (usually mdn are too small for not being downloaded directly)
receive_imf_inner(&bob, "bar@example.org", raw, false, None, false).await?;
receive_imf_from_inbox(&bob, "bar@example.org", raw, false, None, false).await?;
assert_eq!(get_chat_msgs(&bob, chat_id).await?.len(), 0);
assert!(Message::load_from_db(&bob, msg.id)
.await?

View File

@@ -7,7 +7,7 @@ use crate::aheader::{Aheader, EncryptPreference};
use crate::config::Config;
use crate::context::Context;
use crate::key::{load_self_public_key, load_self_secret_key, SignedPublicKey};
use crate::peerstate::{Peerstate, PeerstateVerifiedStatus};
use crate::peerstate::Peerstate;
use crate::pgp;
#[derive(Debug)]
@@ -52,7 +52,7 @@ impl EncryptHelper {
&self,
context: &Context,
e2ee_guaranteed: bool,
peerstates: &[(Option<Peerstate>, &str)],
peerstates: &[(Option<Peerstate>, String)],
) -> Result<bool> {
let mut prefer_encrypt_count = if self.prefer_encrypt == EncryptPreference::Mutual {
1
@@ -62,21 +62,19 @@ impl EncryptHelper {
for (peerstate, addr) in peerstates {
match peerstate {
Some(peerstate) => {
info!(
context,
"peerstate for {:?} is {}", addr, peerstate.prefer_encrypt
);
let prefer_encrypt = peerstate.prefer_encrypt;
info!(context, "Peerstate for {addr:?} is {prefer_encrypt}.");
match peerstate.prefer_encrypt {
EncryptPreference::NoPreference | EncryptPreference::Reset => {}
EncryptPreference::Mutual => prefer_encrypt_count += 1,
};
}
None => {
let msg = format!("peerstate for {addr:?} missing, cannot encrypt");
let msg = format!("Peerstate for {addr:?} missing, cannot encrypt");
if e2ee_guaranteed {
return Err(format_err!("{}", msg));
return Err(format_err!("{msg}"));
} else {
info!(context, "{}", msg);
info!(context, "{msg}.");
return Ok(false);
}
}
@@ -94,9 +92,9 @@ impl EncryptHelper {
pub async fn encrypt(
self,
context: &Context,
min_verified: PeerstateVerifiedStatus,
verified: bool,
mail_to_encrypt: lettre_email::PartBuilder,
peerstates: Vec<(Option<Peerstate>, &str)>,
peerstates: Vec<(Option<Peerstate>, String)>,
) -> Result<String> {
let mut keyring: Vec<SignedPublicKey> = Vec::new();
@@ -107,7 +105,7 @@ impl EncryptHelper {
.filter_map(|(state, addr)| state.clone().map(|s| (s, addr)))
{
let key = peerstate
.take_key(min_verified)
.take_key(verified)
.with_context(|| format!("proper enc-key for {addr} missing, cannot encrypt"))?;
keyring.push(key);
verifier_addresses.push(addr);
@@ -118,8 +116,8 @@ impl EncryptHelper {
// Encrypt to secondary verified keys
// if we also encrypt to the introducer ("verifier") of the key.
if min_verified == PeerstateVerifiedStatus::BidirectVerified {
for (peerstate, _addr) in peerstates {
if verified {
for (peerstate, _addr) in &peerstates {
if let Some(peerstate) = peerstate {
if let (Some(key), Some(verifier)) = (
peerstate.secondary_verified_key.as_ref(),
@@ -171,11 +169,10 @@ pub async fn ensure_secret_key_exists(context: &Context) -> Result<()> {
#[cfg(test)]
mod tests {
use super::*;
use crate::chat;
use crate::key::DcKey;
use crate::message::{Message, Viewtype};
use crate::param::Param;
use crate::test_utils::{bob_keypair, TestContext};
use crate::test_utils::{bob_keypair, TestContext, TestContextManager};
mod ensure_secret_key_exists {
use super::*;
@@ -219,37 +216,35 @@ Sent with my Delta Chat Messenger: https://delta.chat";
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_encrypted_no_autocrypt() -> anyhow::Result<()> {
let alice = TestContext::new_alice().await;
let bob = TestContext::new_bob().await;
let mut tcm = TestContextManager::new();
let alice = tcm.alice().await;
let bob = tcm.bob().await;
let chat_alice = alice.create_chat(&bob).await.id;
let chat_bob = bob.create_chat(&alice).await.id;
// Alice sends unencrypted message to Bob
let mut msg = Message::new(Viewtype::Text);
chat::prepare_msg(&alice.ctx, chat_alice, &mut msg).await?;
chat::send_msg(&alice.ctx, chat_alice, &mut msg).await?;
let sent = alice.pop_sent_msg().await;
let sent = alice.send_msg(chat_alice, &mut msg).await;
// Bob receives unencrypted message from Alice
let msg = bob.parse_msg(&sent).await;
assert!(!msg.was_encrypted());
let msg = bob.recv_msg(&sent).await;
assert!(!msg.get_showpadlock());
// Parsing a message is enough to update peerstate
let peerstate_alice = Peerstate::from_addr(&bob.ctx, "alice@example.org")
.await?
.expect("no peerstate found in the database");
assert_eq!(peerstate_alice.prefer_encrypt, EncryptPreference::Mutual);
// Bob sends encrypted message to Alice
// Bob sends empty encrypted message to Alice
let mut msg = Message::new(Viewtype::Text);
chat::prepare_msg(&bob.ctx, chat_bob, &mut msg).await?;
chat::send_msg(&bob.ctx, chat_bob, &mut msg).await?;
let sent = bob.pop_sent_msg().await;
let sent = bob.send_msg(chat_bob, &mut msg).await;
// Alice receives encrypted message from Bob
let msg = alice.parse_msg(&sent).await;
assert!(msg.was_encrypted());
// Alice receives an empty encrypted message from Bob.
// This is also a regression test for previously existing bug
// that resulted in no padlock on encrypted empty messages.
let msg = alice.recv_msg(&sent).await;
assert!(msg.get_showpadlock());
let peerstate_bob = Peerstate::from_addr(&alice.ctx, "bob@example.net")
.await?
@@ -261,12 +256,10 @@ Sent with my Delta Chat Messenger: https://delta.chat";
// Alice sends encrypted message without Autocrypt header.
let mut msg = Message::new(Viewtype::Text);
msg.param.set_int(Param::SkipAutocrypt, 1);
chat::prepare_msg(&alice.ctx, chat_alice, &mut msg).await?;
chat::send_msg(&alice.ctx, chat_alice, &mut msg).await?;
let sent = alice.pop_sent_msg().await;
let sent = alice.send_msg(chat_alice, &mut msg).await;
let msg = bob.parse_msg(&sent).await;
assert!(msg.was_encrypted());
let msg = bob.recv_msg(&sent).await;
assert!(msg.get_showpadlock());
let peerstate_alice = Peerstate::from_addr(&bob.ctx, "alice@example.org")
.await?
.expect("no peerstate found in the database");
@@ -275,12 +268,10 @@ Sent with my Delta Chat Messenger: https://delta.chat";
// Alice sends plaintext message with Autocrypt header.
let mut msg = Message::new(Viewtype::Text);
msg.force_plaintext();
chat::prepare_msg(&alice.ctx, chat_alice, &mut msg).await?;
chat::send_msg(&alice.ctx, chat_alice, &mut msg).await?;
let sent = alice.pop_sent_msg().await;
let sent = alice.send_msg(chat_alice, &mut msg).await;
let msg = bob.parse_msg(&sent).await;
assert!(!msg.was_encrypted());
let msg = bob.recv_msg(&sent).await;
assert!(!msg.get_showpadlock());
let peerstate_alice = Peerstate::from_addr(&bob.ctx, "alice@example.org")
.await?
.expect("no peerstate found in the database");
@@ -290,12 +281,10 @@ Sent with my Delta Chat Messenger: https://delta.chat";
let mut msg = Message::new(Viewtype::Text);
msg.force_plaintext();
msg.param.set_int(Param::SkipAutocrypt, 1);
chat::prepare_msg(&alice.ctx, chat_alice, &mut msg).await?;
chat::send_msg(&alice.ctx, chat_alice, &mut msg).await?;
let sent = alice.pop_sent_msg().await;
let sent = alice.send_msg(chat_alice, &mut msg).await;
let msg = bob.parse_msg(&sent).await;
assert!(!msg.was_encrypted());
let msg = bob.recv_msg(&sent).await;
assert!(!msg.get_showpadlock());
let peerstate_alice = Peerstate::from_addr(&bob.ctx, "alice@example.org")
.await?
.expect("no peerstate found in the database");
@@ -304,7 +293,7 @@ Sent with my Delta Chat Messenger: https://delta.chat";
Ok(())
}
fn new_peerstates(prefer_encrypt: EncryptPreference) -> Vec<(Option<Peerstate>, &'static str)> {
fn new_peerstates(prefer_encrypt: EncryptPreference) -> Vec<(Option<Peerstate>, String)> {
let addr = "bob@foo.bar";
let pub_key = bob_keypair().public;
let peerstate = Peerstate {
@@ -323,9 +312,10 @@ Sent with my Delta Chat Messenger: https://delta.chat";
secondary_verified_key: None,
secondary_verified_key_fingerprint: None,
secondary_verifier: None,
backward_verified_key_id: None,
fingerprint_changed: false,
};
vec![(Some(peerstate), addr)]
vec![(Some(peerstate), addr.to_string())]
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
@@ -350,7 +340,7 @@ Sent with my Delta Chat Messenger: https://delta.chat";
assert!(encrypt_helper.should_encrypt(&t, false, &ps).unwrap());
// test with missing peerstate
let ps = vec![(None, "bob@foo.bar")];
let ps = vec![(None, "bob@foo.bar".to_string())];
assert!(encrypt_helper.should_encrypt(&t, true, &ps).is_err());
assert!(!encrypt_helper.should_encrypt(&t, false, &ps).unwrap());
}

View File

@@ -4,6 +4,7 @@ use serde::{Deserialize, Serialize};
use std::path::PathBuf;
use crate::chat::ChatId;
use crate::config::Config;
use crate::contact::ContactId;
use crate::ephemeral::Timer as EphemeralTimer;
use crate::message::MsgId;
@@ -261,8 +262,17 @@ pub enum EventType {
ConnectivityChanged,
/// The user's avatar changed.
/// Deprecated by `ConfigSynced`.
SelfavatarChanged,
/// A multi-device synced config value changed. Maybe the app needs to refresh smth. For
/// uniformity this is emitted on the source device too. The value isn't here, otherwise it
/// would be logged which might not be good for privacy.
ConfigSynced {
/// Configuration key.
key: Config,
},
/// Webxdc status update received.
WebxdcStatusUpdate {
/// Message ID.
@@ -277,4 +287,11 @@ pub enum EventType {
/// ID of the deleted message.
msg_id: MsgId,
},
/// Tells that the Background fetch was completed (or timed out).
/// This event acts as a marker, when you reach this event you can be sure
/// that all events emitted during the background fetch were processed.
///
/// This event is only emitted by the account manager
AccountsBackgroundFetchDone,
}

View File

@@ -38,6 +38,9 @@ pub enum HeaderDef {
/// Mailing list ID defined in [RFC 2919](https://tools.ietf.org/html/rfc2919).
ListId,
ListPost,
/// List-Help header defined in [RFC 2369](https://datatracker.ietf.org/doc/html/rfc2369).
ListHelp,
References,
/// In-Reply-To header containing Message-ID of the parent message.

View File

@@ -9,6 +9,7 @@ use std::{
collections::{BTreeMap, BTreeSet, HashMap},
iter::Peekable,
mem::take,
time::Duration,
};
use anyhow::{bail, format_err, Context as _, Result};
@@ -16,6 +17,8 @@ use async_channel::Receiver;
use async_imap::types::{Fetch, Flag, Name, NameAttribute, UnsolicitedResponse};
use futures::{StreamExt, TryStreamExt};
use num_traits::FromPrimitive;
use ratelimit::Ratelimit;
use tokio::sync::RwLock;
use crate::chat::{self, ChatId, ChatIdBlocked};
use crate::config::Config;
@@ -38,7 +41,7 @@ use crate::scheduler::connectivity::ConnectivityStore;
use crate::socks::Socks5Config;
use crate::sql;
use crate::stock_str;
use crate::tools::create_id;
use crate::tools::{create_id, duration_to_str};
pub(crate) mod capabilities;
mod client;
@@ -70,6 +73,7 @@ pub enum ImapActionResult {
/// not necessarily sent by Delta Chat.
const PREFETCH_FLAGS: &str = "(UID INTERNALDATE RFC822.SIZE BODY.PEEK[HEADER.FIELDS (\
MESSAGE-ID \
DATE \
X-MICROSOFT-ORIGINAL-MESSAGE-ID \
FROM \
IN-REPLY-TO REFERENCES \
@@ -91,6 +95,21 @@ pub struct Imap {
login_failed_once: bool,
pub(crate) connectivity: ConnectivityStore,
/// Rate limit for IMAP connection usage attempts.
///
/// Rate limit is checked before connecting
/// and updated right before login attempt.
/// It does not limit the number of connection attempts
/// if the network is bad as only successful connections are counted.
///
/// Main purpose of this rate limit is
/// to prevent busy loop in case
/// connection gets dropped over and over due to IMAP bug,
/// e.g. the server returning invalid response to SELECT command
/// immediately after logging in or returning an error in response to LOGIN command
/// due to internal server error.
ratelimit: RwLock<Ratelimit>,
}
#[derive(Debug)]
@@ -99,6 +118,17 @@ struct OAuth2 {
access_token: String,
}
#[derive(Debug)]
pub(crate) struct ServerMetadata {
/// IMAP METADATA `/shared/comment` as defined in
/// <https://www.rfc-editor.org/rfc/rfc5464#section-6.2.1>.
pub comment: Option<String>,
/// IMAP METADATA `/shared/admin` as defined in
/// <https://www.rfc-editor.org/rfc/rfc5464#section-6.2.2>.
pub admin: Option<String>,
}
impl async_imap::Authenticator for OAuth2 {
type Response = String;
@@ -252,6 +282,8 @@ impl Imap {
session: None,
login_failed_once: false,
connectivity: Default::default(),
// 1 login per minute + a burst of 2.
ratelimit: RwLock::new(Ratelimit::new(Duration::new(120, 0), 2.0)),
};
Ok(imap)
@@ -301,7 +333,20 @@ impl Imap {
self.connectivity.set_connecting(context).await;
let oauth2 = self.config.lp.oauth2;
// Check rate limit before trying to connect
// to avoid connecting and not using the connection
// in case we are currently ratelimited.
// Otherwise connection may become unusable due to NAT forgetting about it
// before we attempt to actually login.
let ratelimit_duration = self.ratelimit.read().await.until_can_send();
if !ratelimit_duration.is_zero() {
warn!(
context,
"IMAP got rate limited, waiting for {} until can connect",
duration_to_str(ratelimit_duration),
);
tokio::time::sleep(ratelimit_duration).await;
}
info!(context, "Connecting to IMAP server");
let connection_res: Result<Client> = if self.config.lp.security == Socket::Starttls
@@ -353,11 +398,13 @@ impl Imap {
Client::connect_secure(context, imap_server, imap_port, config.strict_tls).await
}
};
let client = connection_res?;
self.ratelimit.write().await.send();
let config = &self.config;
let imap_user: &str = config.lp.user.as_ref();
let imap_pw: &str = config.lp.password.as_ref();
let oauth2 = self.config.lp.oauth2;
let login_res = if oauth2 {
info!(context, "Logging into IMAP server with OAuth 2");
@@ -572,9 +619,6 @@ impl Imap {
/// When selecting a folder for the first time, sets the uid_next to the current
/// mailbox.uid_next so that no old emails are fetched.
///
/// Makes sure that UIDNEXT is known for `selected_mailbox`
/// and errors out if UIDNEXT cannot be determined.
///
/// Returns Result<new_emails> (i.e. whether new emails arrived),
/// if in doubt, returns new_emails=true so emails are fetched.
pub(crate) async fn select_with_uidvalidity(
@@ -592,11 +636,18 @@ impl Imap {
.as_mut()
.with_context(|| format!("No mailbox selected, folder: {folder}"))?;
let old_uid_validity = get_uidvalidity(context, folder)
.await
.with_context(|| format!("failed to get old UID validity for folder {folder}"))?;
let old_uid_next = get_uid_next(context, folder)
.await
.with_context(|| format!("failed to get old UID NEXT for folder {folder}"))?;
let new_uid_validity = mailbox
.uid_validity
.with_context(|| format!("No UIDVALIDITY for folder {folder}"))?;
let new_uid_next = if let Some(uid_next) = mailbox.uid_next {
uid_next
Some(uid_next)
} else {
warn!(
context,
@@ -621,18 +672,15 @@ impl Imap {
.await
.with_context(|| format!("STATUS (UIDNEXT) error for {folder:?}"))?;
status
.uid_next
.with_context(|| format!("STATUS {folder} (UIDNEXT) did not return UIDNEXT"))?
if status.uid_next.is_none() {
// This happens with mail.163.com as of 2023-11-26.
// It does not return UIDNEXT on SELECT and returns invalid
// `* STATUS "INBOX" ()` response on explicit request for UIDNEXT.
warn!(context, "STATUS {folder} (UIDNEXT) did not return UIDNEXT.");
}
status.uid_next
};
mailbox.uid_next = Some(new_uid_next);
let old_uid_validity = get_uidvalidity(context, folder)
.await
.with_context(|| format!("failed to get old UID validity for folder {folder}"))?;
let old_uid_next = get_uid_next(context, folder)
.await
.with_context(|| format!("failed to get old UID NEXT for folder {folder}"))?;
mailbox.uid_next = new_uid_next;
if new_uid_validity == old_uid_validity {
let new_emails = if newly_selected == NewlySelected::No {
@@ -641,7 +689,7 @@ impl Imap {
// the caller tries to fetch new messages (we could of course run a SELECT command now, but trying to fetch
// new messages is only one command, just as a SELECT command)
true
} else {
} else if let Some(new_uid_next) = new_uid_next {
if new_uid_next < old_uid_next {
warn!(
context,
@@ -651,7 +699,11 @@ impl Imap {
context.schedule_resync().await?;
}
new_uid_next != old_uid_next // If UIDNEXT changed, there are new emails
} else {
// We have no UIDNEXT and if in doubt, return true.
true
};
return Ok(new_emails);
}
@@ -660,6 +712,7 @@ impl Imap {
// ============== uid_validity has changed or is being set the first time. ==============
let new_uid_next = new_uid_next.unwrap_or_default();
set_uid_next(context, folder, new_uid_next).await?;
set_uidvalidity(context, folder, new_uid_validity).await?;
@@ -728,6 +781,7 @@ impl Imap {
let mut uids_fetch = Vec::<(_, bool /* partially? */)>::with_capacity(msgs.len() + 1);
let mut uid_message_ids = BTreeMap::new();
let mut largest_uid_skipped = None;
let delete_target = context.get_delete_msgs_target().await?;
// Store the info about IMAP messages in the database.
for (uid, ref fetch_response) in msgs {
@@ -753,8 +807,24 @@ impl Imap {
// Such move to the same folder results in the messages
// getting a new UID, so the messages will be detected as new
// in the `INBOX.DeltaChat` folder again.
let _target;
let target = if let Some(message_id) = &message_id {
if context
let is_dup = if let Some((_, ts_sent_old)) =
message::rfc724_mid_exists(context, message_id).await?
{
let is_chat_msg = headers.get_header_value(HeaderDef::ChatVersion).is_some();
let ts_sent = headers
.get_header_value(HeaderDef::Date)
.and_then(|v| mailparse::dateparse(&v).ok())
.unwrap_or_default();
is_dup_msg(is_chat_msg, ts_sent, ts_sent_old)
} else {
false
};
if is_dup {
info!(context, "Deleting duplicate message {message_id}.");
&delete_target
} else if context
.sql
.exists(
"SELECT COUNT (*) FROM imap WHERE rfc724_mid=?",
@@ -766,9 +836,10 @@ impl Imap {
context,
"Not moving the message {} that we have seen before.", &message_id
);
folder.to_string()
folder
} else {
target_folder(context, folder, folder_meaning, &headers).await?
_target = target_folder(context, folder, folder_meaning, &headers).await?;
&_target
}
} else {
// Do not move the messages without Message-ID.
@@ -778,7 +849,7 @@ impl Imap {
context,
"Not moving the message that does not have a Message-ID."
);
folder.to_string()
folder
};
// Generate a fake Message-ID to identify the message in the database
@@ -793,7 +864,7 @@ impl Imap {
ON CONFLICT(folder, uid, uidvalidity)
DO UPDATE SET rfc724_mid=excluded.rfc724_mid,
target=excluded.target",
(&message_id, &folder, uid, uid_validity, &target),
(&message_id, &folder, uid, uid_validity, target),
)
.await?;
@@ -846,6 +917,7 @@ impl Imap {
.fetch_many_msgs(
context,
folder,
uid_validity,
uids_fetch_in_batch.split_off(0),
&uid_message_ids,
fetch_partially,
@@ -876,11 +948,7 @@ impl Imap {
.as_ref()
.with_context(|| format!("Expected {folder:?} to be selected"))?
.uid_next
.with_context(|| {
format!(
"Expected UIDNEXT to be determined for {folder:?} by select_with_uidvalidity"
)
})?;
.unwrap_or_default();
let new_uid_next = max(
max(largest_uid_fetched, largest_uid_skipped.unwrap_or(0)) + 1,
mailbox_uid_next,
@@ -957,7 +1025,7 @@ impl Imap {
self.prepare(context).await?;
let all_folders = self
.list_folders(context)
.list_folders()
.await
.context("listing folders for resync")?;
for folder in all_folders {
@@ -1394,10 +1462,12 @@ impl Imap {
/// Returns the last UID fetched successfully and the info about each downloaded message.
/// If the message is incorrect or there is a failure to write a message to the database,
/// it is skipped and the error is logged.
#[allow(clippy::too_many_arguments)]
pub(crate) async fn fetch_many_msgs(
&mut self,
context: &Context,
folder: &str,
uidvalidity: u32,
request_uids: Vec<u32>,
uid_message_ids: &BTreeMap<u32, String>,
fetch_partially: bool,
@@ -1451,13 +1521,8 @@ impl Imap {
break;
};
let next_fetch_response = match next_fetch_response {
Ok(next_fetch_response) => next_fetch_response,
Err(err) => {
warn!(context, "Failed to process IMAP FETCH result: {}.", err);
continue;
}
};
let next_fetch_response =
next_fetch_response.context("Failed to process IMAP FETCH result")?;
if let Some(next_uid) = next_fetch_response.uid {
if next_uid == request_uid {
@@ -1536,6 +1601,9 @@ impl Imap {
);
match receive_imf_inner(
context,
folder,
uidvalidity,
request_uid,
rfc724_mid,
body,
is_seen,
@@ -1579,6 +1647,50 @@ impl Imap {
Ok((last_uid, received_msgs))
}
/// Retrieves server metadata if it is supported.
///
/// We get [`/shared/comment`](https://www.rfc-editor.org/rfc/rfc5464#section-6.2.1)
/// and [`/shared/admin`](https://www.rfc-editor.org/rfc/rfc5464#section-6.2.2)
/// metadata.
pub(crate) async fn fetch_metadata(&mut self, context: &Context) -> Result<()> {
let session = self.session.as_mut().context("no session")?;
if !session.can_metadata() {
return Ok(());
}
let mut lock = context.metadata.write().await;
if (*lock).is_some() {
return Ok(());
}
info!(
context,
"Server supports metadata, retrieving server comment and admin contact."
);
let mut comment = None;
let mut admin = None;
let mailbox = "";
let options = "";
let metadata = session
.get_metadata(mailbox, options, "(/shared/comment /shared/admin)")
.await?;
for m in metadata {
match m.entry.as_ref() {
"/shared/comment" => {
comment = m.value;
}
"/shared/admin" => {
admin = m.value;
}
_ => {}
}
}
*lock = Some(ServerMetadata { comment, admin });
Ok(())
}
}
impl Session {
@@ -2215,6 +2327,15 @@ pub(crate) async fn prefetch_should_download(
Ok(should_download)
}
/// Returns whether a message is a duplicate (resent message).
pub(crate) fn is_dup_msg(is_chat_msg: bool, ts_sent: i64, ts_sent_old: i64) -> bool {
// If the existing message has timestamp_sent == 0, that means we don't know its actual sent
// timestamp, so don't delete the new message. E.g. outgoing messages have zero timestamp_sent
// because they are stored to the db before sending. Also consider as duplicates only messages
// with greater timestamp to avoid deleting both messages in a multi-device setting.
is_chat_msg && ts_sent_old != 0 && ts_sent > ts_sent_old
}
/// Marks messages in `msgs` table as seen, searching for them by UID.
///
/// Returns updated chat ID if any message was marked as seen.

View File

@@ -21,6 +21,10 @@ pub(crate) struct Capabilities {
/// <https://tools.ietf.org/html/rfc7162>
pub can_condstore: bool,
/// True if the server has METADATA capability as defined in
/// <https://tools.ietf.org/html/rfc5464>
pub can_metadata: bool,
/// Server ID if the server supports ID capability.
pub server_id: Option<HashMap<String, String>>,
}

View File

@@ -17,8 +17,8 @@ use crate::net::tls::wrap_tls;
use crate::socks::Socks5Config;
use fast_socks5::client::Socks5Stream;
/// IMAP write and read timeout.
pub(crate) const IMAP_TIMEOUT: Duration = Duration::from_secs(30);
/// IMAP connection, write and read timeout.
pub(crate) const IMAP_TIMEOUT: Duration = Duration::from_secs(60);
#[derive(Debug)]
pub(crate) struct Client {
@@ -59,6 +59,7 @@ async fn determine_capabilities(
can_move: caps.has_str("MOVE"),
can_check_quota: caps.has_str("QUOTA"),
can_condstore: caps.has_str("CONDSTORE"),
can_metadata: caps.has_str("METADATA"),
server_id,
};
Ok(capabilities)

View File

@@ -9,10 +9,16 @@ use super::session::Session;
use super::Imap;
use crate::config::Config;
use crate::context::Context;
use crate::imap::{client::IMAP_TIMEOUT, get_uid_next, FolderMeaning};
use crate::imap::{client::IMAP_TIMEOUT, FolderMeaning};
use crate::log::LogExt;
const IDLE_TIMEOUT: Duration = Duration::from_secs(23 * 60);
/// Timeout after which IDLE is finished
/// if there are no responses from the server.
///
/// If `* OK Still here` keepalives are sent more frequently
/// than this duration, timeout should never be triggered.
/// For example, Dovecot sends keepalives every 2 minutes by default.
const IDLE_TIMEOUT: Duration = Duration::from_secs(5 * 60);
impl Session {
pub async fn idle(
@@ -29,29 +35,6 @@ impl Session {
return Ok(self);
}
// Despite checking for unsolicited EXISTS above,
// we may have missed EXISTS if the message was
// received when the folder was not selected.
let status = self
.status(folder, "(UIDNEXT)")
.await
.with_context(|| format!("STATUS (UIDNEXT) error for {folder:?}"))?;
if let Some(uid_next) = status.uid_next {
let expected_uid_next = get_uid_next(context, folder)
.await
.with_context(|| format!("failed to get old UID NEXT for folder {folder}"))?;
if uid_next > expected_uid_next {
info!(
context,
"Skipping IDLE on {folder:?} because UIDNEXT {uid_next}>{expected_uid_next} indicates there are new messages."
);
return Ok(self);
}
} else {
warn!(context, "STATUS {folder} (UIDNEXT) did not return UIDNEXT");
// Go to IDLE anyway if STATUS is broken.
}
if let Ok(()) = idle_interrupt_receiver.try_recv() {
info!(context, "skip idle, got interrupt");
return Ok(self);

View File

@@ -1,7 +1,7 @@
use std::{collections::BTreeMap, time::Instant};
use anyhow::{Context as _, Result};
use futures::stream::StreamExt;
use futures::TryStreamExt;
use super::{get_folder_meaning_by_attrs, get_folder_meaning_by_name};
use crate::config::Config;
@@ -27,7 +27,7 @@ impl Imap {
info!(context, "Starting full folder scan");
self.prepare(context).await?;
let folders = self.list_folders(context).await?;
let folders = self.list_folders().await?;
let watched_folders = get_watched_folders(context).await?;
let mut folder_configs = BTreeMap::new();
@@ -98,21 +98,15 @@ impl Imap {
}
/// Returns the names of all folders on the IMAP server.
pub async fn list_folders(
self: &mut Imap,
context: &Context,
) -> Result<Vec<async_imap::types::Name>> {
pub async fn list_folders(self: &mut Imap) -> Result<Vec<async_imap::types::Name>> {
let session = self.session.as_mut();
let session = session.context("No IMAP connection")?;
let list = session
.list(Some(""), Some("*"))
.await?
.filter_map(|f| async {
f.context("list_folders() can't get folder")
.log_err(context)
.ok()
});
Ok(list.collect().await)
.try_collect()
.await?;
Ok(list)
}
}

View File

@@ -64,4 +64,8 @@ impl Session {
pub fn can_condstore(&self) -> bool {
self.capabilities.can_condstore
}
pub fn can_metadata(&self) -> bool {
self.capabilities.can_metadata
}
}

View File

@@ -375,7 +375,15 @@ async fn imex_inner(
path: &Path,
passphrase: Option<String>,
) -> Result<()> {
info!(context, "Import/export dir: {}", path.display());
info!(
context,
"{} path: {}",
match what {
ImexMode::ExportSelfKeys | ImexMode::ExportBackup => "Export",
ImexMode::ImportSelfKeys | ImexMode::ImportBackup => "Import",
},
path.display()
);
ensure!(context.sql.is_open().await, "Database not opened.");
context.emit_event(EventType::ImexProgress(10));
@@ -670,7 +678,7 @@ async fn export_self_keys(context: &Context, dir: &Path) -> Result<()> {
let keys = context
.sql
.query_map(
"SELECT id, public_key, private_key, is_default FROM keypairs;",
"SELECT id, public_key, private_key, id=(SELECT value FROM config WHERE keyname='key_id') FROM keypairs;",
(),
|row| {
let id = row.get(0)?;
@@ -808,6 +816,7 @@ mod tests {
use tokio::task;
use super::*;
use crate::key;
use crate::pgp::{split_armored_data, HEADER_AUTOCRYPT, HEADER_SETUPCODE};
use crate::stock_str::StockMessage;
use crate::test_utils::{alice_keypair, TestContext};
@@ -920,6 +929,37 @@ mod tests {
}
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_import_second_key() -> Result<()> {
let alice = &TestContext::new_alice().await;
let chat = alice.create_chat(alice).await;
let sent = alice.send_text(chat.id, "Encrypted with old key").await;
let export_dir = tempfile::tempdir().unwrap();
let alice = &TestContext::new().await;
alice.configure_addr("alice@example.org").await;
imex(alice, ImexMode::ExportSelfKeys, export_dir.path(), None).await?;
let alice = &TestContext::new_alice().await;
let old_key = key::load_self_secret_key(alice).await?;
imex(alice, ImexMode::ImportSelfKeys, export_dir.path(), None).await?;
let new_key = key::load_self_secret_key(alice).await?;
assert_ne!(new_key, old_key);
assert_eq!(
key::load_self_secret_keyring(alice).await?,
vec![new_key, old_key]
);
let msg = alice.recv_msg(&sent).await;
assert!(msg.get_showpadlock());
assert_eq!(msg.chat_id, alice.get_self_chat().await.id);
assert_eq!(msg.get_text(), "Encrypted with old key");
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_export_and_import_backup() -> Result<()> {
for set_verified_oneonone_chats in [true, false] {

View File

@@ -638,7 +638,7 @@ mod tests {
let self_chat = ctx1.get_self_chat().await;
let msgs = get_chat_msgs(&ctx1, self_chat.id).await.unwrap();
assert_eq!(msgs.len(), 2);
let msgid = match msgs.get(0).unwrap() {
let msgid = match msgs.first().unwrap() {
ChatItem::Message { msg_id } => msg_id,
_ => panic!("wrong chat item"),
};

View File

@@ -16,8 +16,9 @@ use tokio::runtime::Handle;
use crate::config::Config;
use crate::constants::KeyGenType;
use crate::context::Context;
use crate::log::LogExt;
use crate::pgp::KeyPair;
use crate::tools::{time, EmailAddress};
use crate::tools::EmailAddress;
/// Convenience trait for working with keys.
///
@@ -81,10 +82,9 @@ pub(crate) async fn load_self_public_key(context: &Context) -> Result<SignedPubl
match context
.sql
.query_row_optional(
r#"SELECT public_key
FROM keypairs
WHERE addr=(SELECT value FROM config WHERE keyname="configured_addr")
AND is_default=1"#,
"SELECT public_key
FROM keypairs
WHERE id=(SELECT value FROM config WHERE keyname='key_id')",
(),
|row| {
let bytes: Vec<u8> = row.get(0)?;
@@ -105,10 +105,9 @@ pub(crate) async fn load_self_secret_key(context: &Context) -> Result<SignedSecr
match context
.sql
.query_row_optional(
r#"SELECT private_key
FROM keypairs
WHERE addr=(SELECT value FROM config WHERE keyname="configured_addr")
AND is_default=1"#,
"SELECT private_key
FROM keypairs
WHERE id=(SELECT value FROM config WHERE keyname='key_id')",
(),
|row| {
let bytes: Vec<u8> = row.get(0)?;
@@ -125,6 +124,24 @@ pub(crate) async fn load_self_secret_key(context: &Context) -> Result<SignedSecr
}
}
pub(crate) async fn load_self_secret_keyring(context: &Context) -> Result<Vec<SignedSecretKey>> {
let keys = context
.sql
.query_map(
r#"SELECT private_key
FROM keypairs
ORDER BY id=(SELECT value FROM config WHERE keyname='key_id') DESC"#,
(),
|row| row.get::<_, Vec<u8>>(0),
|keys| keys.collect::<Result<Vec<_>, _>>().map_err(Into::into),
)
.await?
.into_iter()
.filter_map(|bytes| SignedSecretKey::from_slice(&bytes).log_err(context).ok())
.collect();
Ok(keys)
}
impl DcKey for SignedPublicKey {
fn to_asc(&self, header: Option<(&str, &str)>) -> String {
// Not using .to_armored_string() to make clear *why* it is
@@ -213,13 +230,10 @@ pub(crate) async fn load_keypair(
let res = context
.sql
.query_row_optional(
r#"
SELECT public_key, private_key
FROM keypairs
WHERE addr=?1
AND is_default=1;
"#,
(addr,),
"SELECT public_key, private_key
FROM keypairs
WHERE id=(SELECT value FROM config WHERE keyname='key_id')",
(),
|row| {
let pub_bytes: Vec<u8> = row.get(0)?;
let sec_bytes: Vec<u8> = row.get(1)?;
@@ -268,42 +282,44 @@ pub async fn store_self_keypair(
keypair: &KeyPair,
default: KeyPairUse,
) -> Result<()> {
context
let mut config_cache_lock = context.sql.config_cache.write().await;
let new_key_id = context
.sql
.transaction(|transaction| {
let public_key = DcKey::to_bytes(&keypair.public);
let secret_key = DcKey::to_bytes(&keypair.secret);
transaction
.execute(
"DELETE FROM keypairs WHERE public_key=? OR private_key=?;",
(&public_key, &secret_key),
)
.context("failed to remove old use of key")?;
if default == KeyPairUse::Default {
transaction
.execute("UPDATE keypairs SET is_default=0;", ())
.context("failed to clear default")?;
}
let is_default = match default {
KeyPairUse::Default => i32::from(true),
KeyPairUse::ReadOnly => i32::from(false),
KeyPairUse::Default => true,
KeyPairUse::ReadOnly => false,
};
let addr = keypair.addr.to_string();
let t = time();
transaction
.execute(
"INSERT INTO keypairs (addr, is_default, public_key, private_key, created)
VALUES (?,?,?,?,?);",
(addr, is_default, &public_key, &secret_key, t),
"INSERT OR REPLACE INTO keypairs (public_key, private_key)
VALUES (?,?)",
(&public_key, &secret_key),
)
.context("failed to insert keypair")?;
.context("Failed to insert keypair")?;
Ok(())
if is_default {
let new_key_id = transaction.last_insert_rowid();
transaction.execute(
"INSERT OR REPLACE INTO config (keyname, value) VALUES ('key_id', ?)",
(new_key_id,),
)?;
Ok(Some(new_key_id))
} else {
Ok(None)
}
})
.await?;
if let Some(new_key_id) = new_key_id {
// Update config cache if transaction succeeded and changed current default key.
config_cache_lock.insert("key_id".to_string(), Some(new_key_id.to_string()));
}
Ok(())
}

View File

@@ -954,7 +954,7 @@ Content-Disposition: attachment; filename="location.kml"
assert!(msg.chat_id == bob_chat_id);
assert_eq!(msg.msg_ids.len(), 1);
let bob_msg = Message::load_from_db(&bob, *msg.msg_ids.get(0).unwrap()).await?;
let bob_msg = Message::load_from_db(&bob, *msg.msg_ids.first().unwrap()).await?;
assert_eq!(bob_msg.chat_id, bob_chat_id);
assert_eq!(bob_msg.viewtype, Viewtype::Image);

View File

@@ -14,8 +14,22 @@ use crate::socks::Socks5Config;
#[repr(u32)]
#[strum(serialize_all = "snake_case")]
pub enum CertificateChecks {
/// Same as AcceptInvalidCertificates unless overridden by
/// `strict_tls` setting in provider database.
/// Same as AcceptInvalidCertificates if stored in the database
/// as `configured_{imap,smtp}_certificate_checks`.
///
/// Previous Delta Chat versions stored this in `configured_*`
/// if Automatic configuration
/// was selected, configuration with strict TLS checks failed
/// and configuration without strict TLS checks succeeded.
///
/// Currently Delta Chat stores only
/// `Strict` or `AcceptInvalidCertificates` variants
/// in `configured_*` settings.
///
/// `Automatic` in `{imap,smtp}_certificate_checks`
/// means that provider database setting should be taken.
/// If there is no provider database setting for certificate checks,
/// `Automatic` is the same as `Strict`.
Automatic = 0,
Strict = 1,

View File

@@ -757,7 +757,7 @@ impl Message {
self.param.get_int(Param::GuaranteeE2ee).unwrap_or_default() != 0
}
/// Returns true if message is Auto-Submitted.
/// Returns true if message is auto-generated.
pub fn is_bot(&self) -> bool {
self.param.get_bool(Param::Bot).unwrap_or_default()
}
@@ -1130,7 +1130,7 @@ impl Message {
/// `References` header is not taken into account.
pub async fn parent(&self, context: &Context) -> Result<Option<Message>> {
if let Some(in_reply_to) = &self.in_reply_to {
if let Some(msg_id) = rfc724_mid_exists(context, in_reply_to).await? {
if let Some((msg_id, _ts_sent)) = rfc724_mid_exists(context, in_reply_to).await? {
let msg = Message::load_from_db(context, msg_id).await?;
return if msg.chat_id.is_trash() {
// If message is already moved to trash chat, pretend it does not exist.
@@ -1663,9 +1663,17 @@ pub(crate) async fn update_msg_state(
msg_id: MsgId,
state: MessageState,
) -> Result<()> {
ensure!(state != MessageState::OutFailed, "use set_msg_failed()!");
let error_subst = match state >= MessageState::OutPending {
true => ", error=''",
false => "",
};
context
.sql
.execute("UPDATE msgs SET state=? WHERE id=?;", (state, msg_id))
.execute(
&format!("UPDATE msgs SET state=?1 {error_subst} WHERE id=?2 AND (?1!=?3 OR state<?3)"),
(state, msg_id, MessageState::OutDelivered),
)
.await?;
Ok(())
}
@@ -1808,18 +1816,23 @@ pub async fn estimate_deletion_cnt(
Ok(cnt)
}
/// See [`rfc724_mid_exists_and()`].
pub(crate) async fn rfc724_mid_exists(
context: &Context,
rfc724_mid: &str,
) -> Result<Option<MsgId>> {
) -> Result<Option<(MsgId, i64)>> {
rfc724_mid_exists_and(context, rfc724_mid, "1").await
}
/// Returns [MsgId] and "sent" timestamp of the message with given `rfc724_mid` (Message-ID header)
/// if it exists in the db.
///
/// @param cond SQL subexpression for filtering messages.
pub(crate) async fn rfc724_mid_exists_and(
context: &Context,
rfc724_mid: &str,
cond: &str,
) -> Result<Option<MsgId>> {
) -> Result<Option<(MsgId, i64)>> {
let rfc724_mid = rfc724_mid.trim_start_matches('<').trim_end_matches('>');
if rfc724_mid.is_empty() {
warn!(context, "Empty rfc724_mid passed to rfc724_mid_exists");
@@ -1829,12 +1842,13 @@ pub(crate) async fn rfc724_mid_exists_and(
let res = context
.sql
.query_row_optional(
&("SELECT id FROM msgs WHERE rfc724_mid=? AND ".to_string() + cond),
&("SELECT id, timestamp_sent FROM msgs WHERE rfc724_mid=? AND ".to_string() + cond),
(rfc724_mid,),
|row| {
let msg_id: MsgId = row.get(0)?;
let timestamp_sent: i64 = row.get(1)?;
Ok(msg_id)
Ok((msg_id, timestamp_sent))
},
)
.await?;
@@ -1842,6 +1856,24 @@ pub(crate) async fn rfc724_mid_exists_and(
Ok(res)
}
/// Given a list of Message-IDs, returns the latest message found in the database.
///
/// Only messages that are not in the trash chat are considered.
pub(crate) async fn get_latest_by_rfc724_mids(
context: &Context,
mids: &[String],
) -> Result<Option<Message>> {
for id in mids.iter().rev() {
if let Some((msg_id, _)) = rfc724_mid_exists(context, id).await? {
let msg = Message::load_from_db(context, msg_id).await?;
if msg.chat_id != DC_CHAT_ID_TRASH {
return Ok(Some(msg));
}
}
}
Ok(None)
}
/// How a message is primarily displayed.
#[derive(
Debug,

View File

@@ -1,5 +1,6 @@
//! # MIME message production.
use std::collections::HashSet;
use std::convert::TryInto;
use anyhow::{bail, ensure, Context as _, Result};
@@ -22,7 +23,7 @@ use crate::location;
use crate::message::{self, Message, MsgId, Viewtype};
use crate::mimeparser::SystemMessage;
use crate::param::Param;
use crate::peerstate::{Peerstate, PeerstateVerifiedStatus};
use crate::peerstate::Peerstate;
use crate::simplify::escape_message_footer_marks;
use crate::stock_str;
use crate::tools::IsNoneOrEmpty;
@@ -123,7 +124,8 @@ struct MessageHeaders {
/// Headers that MUST NOT go into IMF header section.
///
/// These are large headers which may hit the header section size limit on the server, such as
/// Chat-User-Avatar with a base64-encoded image inside.
/// Chat-User-Avatar with a base64-encoded image inside. Also there are headers duplicated here
/// that servers mess up with in the IMF header section, like Message-ID.
pub hidden: Vec<Header>,
}
@@ -275,7 +277,7 @@ impl<'a> MimeFactory<'a> {
async fn peerstates_for_recipients(
&self,
context: &Context,
) -> Result<Vec<(Option<Peerstate>, &str)>> {
) -> Result<Vec<(Option<Peerstate>, String)>> {
let self_addr = context.get_primary_self_addr().await?;
let mut res = Vec::new();
@@ -284,7 +286,7 @@ impl<'a> MimeFactory<'a> {
.iter()
.filter(|(_, addr)| addr != &self_addr)
{
res.push((Peerstate::from_addr(context, addr).await?, addr.as_str()));
res.push((Peerstate::from_addr(context, addr).await?, addr.clone()));
}
Ok(res)
@@ -312,7 +314,7 @@ impl<'a> MimeFactory<'a> {
}
}
fn min_verified(&self) -> PeerstateVerifiedStatus {
fn verified(&self) -> bool {
match &self.loaded {
Loaded::Message { chat } => {
if chat.is_protected() {
@@ -321,15 +323,15 @@ impl<'a> MimeFactory<'a> {
// In order to do this, it is necessary that they can be sent
// to a key that is not yet verified.
// This has to work independently of whether the chat is protected right now.
PeerstateVerifiedStatus::Unverified
false
} else {
PeerstateVerifiedStatus::BidirectVerified
true
}
} else {
PeerstateVerifiedStatus::Unverified
false
}
}
Loaded::Mdn { .. } => PeerstateVerifiedStatus::Unverified,
Loaded::Mdn { .. } => false,
}
}
@@ -349,7 +351,7 @@ impl<'a> MimeFactory<'a> {
.unwrap_or_default()
}
}
Loaded::Mdn { .. } => true,
Loaded::Mdn { .. } => false,
}
}
@@ -517,6 +519,7 @@ impl<'a> MimeFactory<'a> {
// <https://datatracker.ietf.org/doc/html/rfc5322#appendix-A.1.1>.
let from_header = Header::new_with_value("From".into(), vec![from]).unwrap();
headers.unprotected.push(from_header.clone());
headers.protected.push(from_header);
if let Some(sender_displayname) = &self.sender_displayname {
let sender =
@@ -558,24 +561,9 @@ impl<'a> MimeFactory<'a> {
Loaded::Mdn { .. } => create_outgoing_rfc724_mid(None, &self.from_addr),
};
let rfc724_mid_headervalue = render_rfc724_mid(&rfc724_mid);
// Amazon's SMTP servers change the `Message-ID`, just as Outlook's SMTP servers do.
// Outlook's servers add an `X-Microsoft-Original-Message-ID` header with the original `Message-ID`,
// and when downloading messages we look for this header in order to correctly identify
// messages.
// Amazon's servers do not add such a header, so we just add it ourselves.
if let Some(server) = context.get_config(Config::ConfiguredSendServer).await? {
if server.ends_with(".amazonaws.com") {
headers.unprotected.push(Header::new(
"X-Microsoft-Original-Message-ID".into(),
rfc724_mid_headervalue.clone(),
))
}
}
headers
.unprotected
.push(Header::new("Message-ID".into(), rfc724_mid_headervalue));
let rfc724_mid_header = Header::new("Message-ID".into(), rfc724_mid_headervalue);
headers.unprotected.push(rfc724_mid_header.clone());
headers.hidden.push(rfc724_mid_header);
// Reply headers as in <https://datatracker.ietf.org/doc/html/rfc5322#appendix-A.2>.
if !self.in_reply_to.is_empty() {
@@ -627,7 +615,7 @@ impl<'a> MimeFactory<'a> {
));
}
let min_verified = self.min_verified();
let verified = self.verified();
let grpimage = self.grpimage();
let force_plaintext = self.should_force_plaintext();
let skip_autocrypt = self.should_skip_autocrypt();
@@ -704,8 +692,6 @@ impl<'a> MimeFactory<'a> {
)
};
let outer_message = if is_encrypted {
headers.protected.push(from_header);
// Store protected headers in the inner message.
let message = headers
.protected
@@ -723,7 +709,7 @@ impl<'a> MimeFactory<'a> {
&& self.should_do_gossip(context).await?
{
for peerstate in peerstates.iter().filter_map(|(state, _)| state.as_ref()) {
if let Some(header) = peerstate.render_gossip_header(min_verified) {
if let Some(header) = peerstate.render_gossip_header(verified) {
message = message.header(Header::new("Autocrypt-Gossip".into(), header));
is_gossiped = true;
}
@@ -756,7 +742,7 @@ impl<'a> MimeFactory<'a> {
}
let encrypted = encrypt_helper
.encrypt(context, min_verified, message, peerstates)
.encrypt(context, verified, message, peerstates)
.await?;
outer_message
@@ -782,30 +768,53 @@ impl<'a> MimeFactory<'a> {
.build(),
)
.header(("Subject".to_string(), "...".to_string()))
} else {
let message = if headers.hidden.is_empty() {
message
} else {
// Store hidden headers in the inner unencrypted message.
let message = headers
.hidden
.into_iter()
.fold(message, |message, header| message.header(header));
} else if matches!(self.loaded, Loaded::Mdn { .. }) {
// Never add outer multipart/mixed wrapper to MDN
// as multipart/report Content-Type is used to recognize MDNs
// by Delta Chat receiver and Chatmail servers
// allowing them to be unencrypted and not contain Autocrypt header
// without resetting Autocrypt encryption or triggering Chatmail filter
// that normally only allows encrypted mails.
PartBuilder::new()
.message_type(MimeMultipartType::Mixed)
.child(message.build())
};
// Hidden headers are dropped.
// Store protected headers in the outer message.
let message = headers
.protected
.iter()
.fold(message, |message, header| message.header(header.clone()));
let protected: HashSet<Header> = HashSet::from_iter(headers.protected.into_iter());
for h in headers.unprotected.split_off(0) {
if !protected.contains(&h) {
headers.unprotected.push(h);
}
}
message
} else {
// Store hidden headers in the inner unencrypted message.
let message = headers
.hidden
.into_iter()
.fold(message, |message, header| message.header(header));
let message = PartBuilder::new()
.message_type(MimeMultipartType::Mixed)
.child(message.build());
if self.should_skip_autocrypt()
|| !context.get_config_bool(Config::SignUnencrypted).await?
{
// Store protected headers in the outer message.
let message = headers
.protected
.iter()
.fold(message, |message, header| message.header(header.clone()));
if skip_autocrypt || !context.get_config_bool(Config::SignUnencrypted).await? {
let protected: HashSet<Header> = HashSet::from_iter(headers.protected.into_iter());
for h in headers.unprotected.split_off(0) {
if !protected.contains(&h) {
headers.unprotected.push(h);
}
}
message
} else {
let message = message.header(get_content_type_directives_header());
@@ -908,6 +917,16 @@ impl<'a> MimeFactory<'a> {
Ok(Some(part))
}
fn add_message_text(&self, part: PartBuilder, mut text: String) -> PartBuilder {
// This is needed to protect from ESPs (such as gmx.at) doing their own Quoted-Printable
// encoding and thus breaking messages and signatures. It's unlikely that the reader uses a
// MUA not supporting Quoted-Printable encoding. And RFC 2646 "4.6" also recommends it for
// encrypted messages.
let part = part.header(("Content-Transfer-Encoding", "quoted-printable"));
text = quoted_printable::encode_to_str(text);
part.body(text)
}
#[allow(clippy::cognitive_complexity)]
async fn render_message(
&mut self,
@@ -924,9 +943,7 @@ impl<'a> MimeFactory<'a> {
let mut meta_part = None;
let send_verified_headers = match chat.typ {
// In single chats, the protection status isn't necessarily the same for both sides,
// so we don't send the Chat-Verified header:
Chattype::Single => false,
Chattype::Single => true,
Chattype::Group => true,
// Mailinglists and broadcast lists can actually never be verified:
Chattype::Mailinglist => false,
@@ -990,24 +1007,12 @@ impl<'a> MimeFactory<'a> {
{
info!(
context,
"sending secure-join message \'{}\' >>>>>>>>>>>>>>>>>>>>>>>>>",
"vg-member-added",
"Sending secure-join message {:?}.", "vg-member-added",
);
headers.protected.push(Header::new(
"Secure-Join".to_string(),
"vg-member-added".to_string(),
));
// FIXME: Old clients require Secure-Join-Fingerprint header. Remove this
// eventually.
let fingerprint = Peerstate::from_addr(context, email_to_add)
.await?
.context("No peerstate found in db")?
.public_key_fingerprint
.context("No public key fingerprint in db for the member to add")?;
headers.protected.push(Header::new(
"Secure-Join-Fingerprint".into(),
fingerprint.hex(),
));
}
}
SystemMessage::GroupNameChanged => {
@@ -1073,10 +1078,7 @@ impl<'a> MimeFactory<'a> {
let msg = &self.msg;
let step = msg.param.get(Param::Arg).unwrap_or_default();
if !step.is_empty() {
info!(
context,
"sending secure-join message \'{}\' >>>>>>>>>>>>>>>>>>>>>>>>>", step,
);
info!(context, "Sending secure-join message {step:?}.");
headers
.protected
.push(Header::new("Secure-Join".into(), step.into()));
@@ -1222,13 +1224,11 @@ impl<'a> MimeFactory<'a> {
footer
);
// Message is sent as text/plain, with charset = utf-8
let mut main_part = PartBuilder::new()
.header((
"Content-Type".to_string(),
"text/plain; charset=utf-8; format=flowed; delsp=no".to_string(),
))
.body(message_text);
let mut main_part = PartBuilder::new().header((
"Content-Type",
"text/plain; charset=utf-8; format=flowed; delsp=no",
));
main_part = self.add_message_text(main_part, message_text);
if is_reaction {
main_part = main_part.header(("Content-Disposition", "reaction"));
@@ -1355,15 +1355,12 @@ impl<'a> MimeFactory<'a> {
};
let p2 = stock_str::read_rcpt_mail_body(context, &p1).await;
let message_text = format!("{}\r\n", format_flowed(&p2));
message = message.child(
PartBuilder::new()
.header((
"Content-Type".to_string(),
"text/plain; charset=utf-8; format=flowed; delsp=no".to_string(),
))
.body(message_text)
.build(),
);
let text_part = PartBuilder::new().header((
"Content-Type".to_string(),
"text/plain; charset=utf-8; format=flowed; delsp=no".to_string(),
));
let text_part = self.add_message_text(text_part, message_text);
message = message.child(text_part.build());
// second body part: machine-readable, always REQUIRED by RFC 6522
let message_text2 = format!(
@@ -1579,6 +1576,7 @@ mod tests {
use crate::mimeparser::MimeMessage;
use crate::receive_imf::receive_imf;
use crate::test_utils::{get_chat_msg, TestContext, TestContextManager};
#[test]
fn test_render_email_address() {
let display_name = "ä space";
@@ -1834,6 +1832,37 @@ mod tests {
assert_eq!("Re: Hello, Bob", mf.subject_str(&t).await.unwrap());
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_mdn_create_encrypted() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = tcm.alice().await;
let bob = tcm.bob().await;
bob.set_config_bool(Config::MdnsEnabled, true).await?;
let mut msg = Message::new(Viewtype::Text);
msg.param.set_int(Param::SkipAutocrypt, 1);
let chat_alice = alice.create_chat(&bob).await.id;
let sent = alice.send_msg(chat_alice, &mut msg).await;
let rcvd = bob.recv_msg(&sent).await;
message::markseen_msgs(&bob, vec![rcvd.id]).await?;
let mimefactory = MimeFactory::from_mdn(&bob, &rcvd, vec![]).await?;
let rendered_msg = mimefactory.render(&bob).await?;
assert!(!rendered_msg.is_encrypted);
let rcvd = tcm.send_recv(&alice, &bob, "Heyho").await;
message::markseen_msgs(&bob, vec![rcvd.id]).await?;
let mimefactory = MimeFactory::from_mdn(&bob, &rcvd, vec![]).await?;
let rendered_msg = mimefactory.render(&bob).await?;
// When encrypted, the MDN should be encrypted as well
assert!(rendered_msg.is_encrypted);
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_subject_in_group() -> Result<()> {
async fn send_msg_get_subject(
@@ -2165,33 +2194,39 @@ mod tests {
let body = payload.next().unwrap();
assert_eq!(outer.match_indices("multipart/mixed").count(), 1);
assert_eq!(outer.match_indices("Message-ID:").count(), 1);
assert_eq!(outer.match_indices("Subject:").count(), 1);
assert_eq!(outer.match_indices("Autocrypt:").count(), 1);
assert_eq!(outer.match_indices("Chat-User-Avatar:").count(), 0);
assert_eq!(inner.match_indices("text/plain").count(), 1);
assert_eq!(inner.match_indices("Message-ID:").count(), 1);
assert_eq!(inner.match_indices("Chat-User-Avatar:").count(), 1);
assert_eq!(inner.match_indices("Subject:").count(), 0);
assert_eq!(inner.match_indices("quoted-printable").count(), 1);
assert_eq!(body.match_indices("this is the text!").count(), 1);
// if another message is sent, that one must not contain the avatar
// and no artificial multipart/mixed nesting
let sent_msg = t.send_msg(chat.id, &mut msg).await;
let mut payload = sent_msg.payload().splitn(2, "\r\n\r\n");
let mut payload = sent_msg.payload().splitn(3, "\r\n\r\n");
let outer = payload.next().unwrap();
let inner = payload.next().unwrap();
let body = payload.next().unwrap();
assert_eq!(outer.match_indices("text/plain").count(), 1);
assert_eq!(outer.match_indices("multipart/mixed").count(), 1);
assert_eq!(outer.match_indices("Message-ID:").count(), 1);
assert_eq!(outer.match_indices("Subject:").count(), 1);
assert_eq!(outer.match_indices("Autocrypt:").count(), 1);
assert_eq!(outer.match_indices("multipart/mixed").count(), 0);
assert_eq!(outer.match_indices("Chat-User-Avatar:").count(), 0);
assert_eq!(inner.match_indices("text/plain").count(), 1);
assert_eq!(inner.match_indices("Message-ID:").count(), 1);
assert_eq!(inner.match_indices("Chat-User-Avatar:").count(), 0);
assert_eq!(inner.match_indices("Subject:").count(), 0);
assert_eq!(inner.match_indices("quoted-printable").count(), 1);
assert_eq!(body.match_indices("this is the text!").count(), 1);
assert_eq!(body.match_indices("text/plain").count(), 0);
assert_eq!(body.match_indices("Chat-User-Avatar:").count(), 0);
assert_eq!(body.match_indices("Subject:").count(), 0);
Ok(())
}
@@ -2222,6 +2257,8 @@ mod tests {
let part = payload.next().unwrap();
assert_eq!(part.match_indices("multipart/signed").count(), 1);
assert_eq!(part.match_indices("From:").count(), 1);
assert_eq!(part.match_indices("Message-ID:").count(), 1);
assert_eq!(part.match_indices("Subject:").count(), 0);
assert_eq!(part.match_indices("Autocrypt:").count(), 1);
assert_eq!(part.match_indices("Chat-User-Avatar:").count(), 0);
@@ -2232,14 +2269,19 @@ mod tests {
.count(),
1
);
assert_eq!(part.match_indices("From:").count(), 1);
assert_eq!(part.match_indices("Message-ID:").count(), 0);
assert_eq!(part.match_indices("Subject:").count(), 1);
assert_eq!(part.match_indices("Autocrypt:").count(), 0);
assert_eq!(part.match_indices("Chat-User-Avatar:").count(), 0);
let part = payload.next().unwrap();
assert_eq!(part.match_indices("text/plain").count(), 1);
assert_eq!(part.match_indices("From:").count(), 0);
assert_eq!(part.match_indices("Message-ID:").count(), 1);
assert_eq!(part.match_indices("Chat-User-Avatar:").count(), 1);
assert_eq!(part.match_indices("Subject:").count(), 0);
assert_eq!(part.match_indices("quoted-printable").count(), 1);
let body = payload.next().unwrap();
assert_eq!(body.match_indices("this is the text!").count(), 1);
@@ -2258,28 +2300,39 @@ mod tests {
.is_some());
// if another message is sent, that one must not contain the avatar
// and no artificial multipart/mixed nesting
let sent_msg = t.send_msg(chat.id, &mut msg).await;
let mut payload = sent_msg.payload().splitn(3, "\r\n\r\n");
let mut payload = sent_msg.payload().splitn(4, "\r\n\r\n");
let part = payload.next().unwrap();
assert_eq!(part.match_indices("multipart/signed").count(), 1);
assert_eq!(part.match_indices("From:").count(), 1);
assert_eq!(part.match_indices("Message-ID:").count(), 1);
assert_eq!(part.match_indices("Subject:").count(), 0);
assert_eq!(part.match_indices("Autocrypt:").count(), 1);
assert_eq!(part.match_indices("Chat-User-Avatar:").count(), 0);
let part = payload.next().unwrap();
assert_eq!(part.match_indices("text/plain").count(), 1);
assert_eq!(
part.match_indices("multipart/mixed; protected-headers=\"v1\"")
.count(),
1
);
assert_eq!(part.match_indices("From:").count(), 1);
assert_eq!(part.match_indices("Message-ID:").count(), 0);
assert_eq!(part.match_indices("Subject:").count(), 1);
assert_eq!(part.match_indices("Autocrypt:").count(), 0);
assert_eq!(part.match_indices("multipart/mixed").count(), 0);
assert_eq!(part.match_indices("Chat-User-Avatar:").count(), 0);
let part = payload.next().unwrap();
assert_eq!(part.match_indices("text/plain").count(), 1);
assert_eq!(body.match_indices("From:").count(), 0);
assert_eq!(part.match_indices("Message-ID:").count(), 1);
assert_eq!(part.match_indices("Chat-User-Avatar:").count(), 0);
assert_eq!(part.match_indices("Subject:").count(), 0);
assert_eq!(part.match_indices("quoted-printable").count(), 1);
let body = payload.next().unwrap();
assert_eq!(body.match_indices("this is the text!").count(), 1);
assert_eq!(body.match_indices("text/plain").count(), 0);
assert_eq!(body.match_indices("Chat-User-Avatar:").count(), 0);
assert_eq!(body.match_indices("Subject:").count(), 0);
bob.recv_msg(&sent_msg).await;
let alice_contact = Contact::get_by_id(&bob.ctx, alice_id).await.unwrap();

View File

@@ -1,5 +1,6 @@
//! # MIME message parsing module.
use std::cmp::min;
use std::collections::{HashMap, HashSet};
use std::future::Future;
use std::path::Path;
@@ -27,7 +28,7 @@ use crate::decrypt::{
use crate::dehtml::dehtml;
use crate::events::EventType;
use crate::headerdef::{HeaderDef, HeaderDefMap};
use crate::key::{load_self_secret_key, DcKey, Fingerprint, SignedPublicKey};
use crate::key::{load_self_secret_keyring, DcKey, Fingerprint, SignedPublicKey};
use crate::message::{
self, set_msg_failed, update_msg_state, Message, MessageState, MsgId, Viewtype,
};
@@ -37,8 +38,8 @@ use crate::simplify::{simplify, SimplifiedText};
use crate::stock_str;
use crate::sync::SyncItems;
use crate::tools::{
create_smeared_timestamp, get_filemeta, parse_receive_headers, strip_rtlo_characters,
truncate_by_lines,
create_smeared_timestamp, get_filemeta, parse_receive_headers, smeared_time,
strip_rtlo_characters, truncate_by_lines,
};
use crate::{location, tools};
@@ -116,8 +117,14 @@ pub(crate) struct MimeMessage {
/// Hop info for debugging.
pub(crate) hop_info: String,
/// Whether the contact sending this should be marked as bot.
pub(crate) is_bot: bool,
/// Whether the contact sending this should be marked as bot or non-bot.
pub(crate) is_bot: Option<bool>,
/// When the message was received, in secs since epoch.
pub(crate) timestamp_rcvd: i64,
/// Sender timestamp in secs since epoch. Allowed to be in the future due to unsynchronized
/// clocks, but not too much.
pub(crate) timestamp_sent: i64,
}
#[derive(Debug, PartialEq)]
@@ -169,6 +176,10 @@ pub enum SystemMessage {
/// "%1$s sent a message from another device."
ChatProtectionDisabled = 12,
/// Message can't be sent because of `Invalid unencrypted mail to <>`
/// which is sent by chatmail servers.
InvalidUnencryptedMail = 13,
/// Self-sent-message that contains only json used for multi-device-sync;
/// if possible, we attach that to other messages as for locations.
MultiDeviceSync = 20,
@@ -259,14 +270,12 @@ impl MimeMessage {
}
}
// remove headers that are allowed _only_ in the encrypted part
headers.remove("secure-join-fingerprint");
headers.remove("chat-verified");
// Remove headers that are allowed _only_ in the encrypted+signed part. It's ok to leave
// them in signed-only emails, but has no value currently.
Self::remove_secured_headers(&mut headers);
let from = from.context("No from in message")?;
let private_keyring = vec![load_self_secret_key(context)
.await
.context("Failed to get own key")?];
let private_keyring = load_self_secret_keyring(context).await?;
let mut decryption_info =
prepare_decryption(context, &mail, &from.addr, message_time).await?;
@@ -280,7 +289,7 @@ impl MimeMessage {
let public_keyring = keyring_from_peerstate(decryption_info.peerstate.as_ref());
let (mail, mut signatures, encrypted) = match tokio::task::block_in_place(|| {
try_decrypt(context, &mail, &private_keyring, &public_keyring)
try_decrypt(&mail, &private_keyring, &public_keyring)
}) {
Ok(Some((raw, signatures))) => {
mail_raw = raw;
@@ -307,10 +316,11 @@ impl MimeMessage {
content
});
if let (Ok(mail), true) = (mail, encrypted) {
// Handle any gossip headers if the mail was encrypted. See section
// "3.6 Key Gossip" of <https://autocrypt.org/autocrypt-spec-1.1.0.pdf>
// but only if the mail was correctly signed:
if !signatures.is_empty() {
// Handle any gossip headers if the mail was encrypted. See section
// "3.6 Key Gossip" of <https://autocrypt.org/autocrypt-spec-1.1.0.pdf>
// but only if the mail was correctly signed. Probably it's ok to not require
// encryption here, but let's follow the standard.
let gossip_headers = mail.headers.get_all_values("Autocrypt-Gossip");
gossiped_addr = update_gossip_peerstates(
context,
@@ -320,6 +330,9 @@ impl MimeMessage {
gossip_headers,
)
.await?;
// Remove unsigned subject from messages displayed with padlock.
// See <https://github.com/deltachat/deltachat-core-rust/issues/1790>.
headers.remove("subject");
}
// let known protected headers from the decrypted
@@ -327,24 +340,20 @@ impl MimeMessage {
// Signature was checked for original From, so we
// do not allow overriding it.
let mut signed_from = None;
// We do not want to allow unencrypted subject in encrypted emails because the
// user might falsely think that the subject is safe.
// See <https://github.com/deltachat/deltachat-core-rust/issues/1790>.
headers.remove("subject");
let mut inner_from = None;
MimeMessage::merge_headers(
context,
&mut headers,
&mut recipients,
&mut signed_from,
&mut inner_from,
&mut list_post,
&mut chat_disposition_notification_to,
&mail.headers,
);
if let Some(signed_from) = signed_from {
if addr_cmp(&signed_from.addr, &from.addr) {
if let (Some(inner_from), true) = (inner_from, !signatures.is_empty()) {
if addr_cmp(&inner_from.addr, &from.addr) {
from_is_signed = true;
} else {
// There is a From: header in the encrypted &
@@ -362,6 +371,8 @@ impl MimeMessage {
}
}
if signatures.is_empty() {
Self::remove_secured_headers(&mut headers);
// If it is not a read receipt, degrade encryption.
if let (Some(peerstate), Ok(mail)) = (&mut decryption_info.peerstate, mail) {
if message_time > peerstate.last_seen_autocrypt
@@ -383,8 +394,11 @@ impl MimeMessage {
}
}
// Auto-submitted is also set by holiday-notices so we also check `chat-version`
let is_bot = headers.contains_key("auto-submitted") && headers.contains_key("chat-version");
let timestamp_rcvd = smeared_time(context);
let timestamp_sent = headers
.get(HeaderDef::Date.get_headername())
.and_then(|value| mailparse::dateparse(value).ok())
.map_or(timestamp_rcvd, |value| min(value, timestamp_rcvd + 60));
let mut parser = MimeMessage {
parts: Vec::new(),
@@ -414,7 +428,9 @@ impl MimeMessage {
is_mime_modified: false,
decoded_data: Vec::new(),
hop_info,
is_bot,
is_bot: None,
timestamp_rcvd,
timestamp_sent,
};
match partial {
@@ -445,6 +461,13 @@ impl MimeMessage {
},
};
if parser.mdn_reports.is_empty() {
// "Auto-Submitted" is also set by holiday-notices so we also check "chat-version".
let is_bot = parser.headers.get("auto-submitted")
== Some(&"auto-generated".to_string())
&& parser.headers.contains_key("chat-version");
parser.is_bot = Some(is_bot);
}
parser.maybe_remove_bad_parts();
parser.maybe_remove_inline_mailinglist_footer();
parser.heuristically_parse_ndn(context).await;
@@ -461,20 +484,6 @@ impl MimeMessage {
parser.decoded_data = mail_raw;
}
crate::peerstate::maybe_do_aeap_transition(context, &mut parser).await?;
if let Some(peerstate) = &parser.decryption_info.peerstate {
peerstate
.handle_fingerprint_change(context, message_time)
.await?;
// When peerstate is set to Mutual, it's saved immediately to not lose that fact in case
// of an error. Otherwise we don't save peerstate until get here to reduce the number of
// calls to save_to_db() and not to degrade encryption if a mail wasn't parsed
// successfully.
if peerstate.prefer_encrypt != EncryptPreference::Mutual {
peerstate.save_to_db(&context.sql).await?;
}
}
Ok(parser)
}
@@ -694,10 +703,10 @@ impl MimeMessage {
}
}
self.parts.push(part);
self.do_add_single_part(part);
}
if self.headers.contains_key("auto-submitted") {
if self.is_bot == Some(true) {
for part in &mut self.parts {
part.param.set(Param::Bot, "1");
}
@@ -913,7 +922,7 @@ impl MimeMessage {
skip the rest. (see
<https://k9mail.app/2016/11/24/OpenPGP-Considerations-Part-I.html>
for background information why we use encrypted+signed) */
if let Some(first) = mail.subparts.get(0) {
if let Some(first) = mail.subparts.first() {
any_part_added = self
.parse_mime_recursive(context, first, is_related)
.await?;
@@ -969,10 +978,13 @@ impl MimeMessage {
}
}
Some(_) => {
if let Some(first) = mail.subparts.get(0) {
any_part_added = self
.parse_mime_recursive(context, first, is_related)
.await?;
for cur_data in &mail.subparts {
if self
.parse_mime_recursive(context, cur_data, is_related)
.await?
{
any_part_added = true;
}
}
}
}
@@ -1364,6 +1376,15 @@ impl MimeMessage {
self.get_mailinglist_header().is_some()
}
/// Detects Schleuder mailing list by List-Help header.
pub(crate) fn is_schleuder_message(&self) -> bool {
if let Some(list_help) = self.get_header(HeaderDef::ListHelp) {
list_help == "<https://schleuder.org/>"
} else {
false
}
}
pub fn replace_msg_by_error(&mut self, error_msg: &str) {
self.is_system_message = SystemMessage::Unknown;
if let Some(part) = self.parts.first_mut() {
@@ -1379,6 +1400,11 @@ impl MimeMessage {
.and_then(|msgid| parse_message_id(msgid).ok())
}
fn remove_secured_headers(headers: &mut HashMap<String, String>) {
headers.remove("secure-join-fingerprint");
headers.remove("chat-verified");
}
fn merge_headers(
context: &Context,
headers: &mut HashMap<String, String>,
@@ -1587,8 +1613,12 @@ impl MimeMessage {
/// eg. when the user-edited-content is html.
/// As these footers would appear as repeated, separate text-bubbles,
/// we remove them.
///
/// We make an exception for Schleuder mailing lists
/// because they typically create messages with two text parts,
/// one for headers and one for the actual contents.
fn maybe_remove_inline_mailinglist_footer(&mut self) {
if self.is_mailinglist_message() {
if self.is_mailinglist_message() && !self.is_schleuder_message() {
let text_part_cnt = self
.parts
.iter()
@@ -1639,13 +1669,7 @@ impl MimeMessage {
/// Handle reports
/// (MDNs = Message Disposition Notification, the message was read
/// and NDNs = Non delivery notification, the message could not be delivered)
pub async fn handle_reports(
&self,
context: &Context,
from_id: ContactId,
sent_timestamp: i64,
parts: &[Part],
) {
pub async fn handle_reports(&self, context: &Context, from_id: ContactId, parts: &[Part]) {
for report in &self.mdn_reports {
for original_message_id in report
.original_message_id
@@ -1653,7 +1677,7 @@ impl MimeMessage {
.chain(&report.additional_message_ids)
{
if let Err(err) =
handle_mdn(context, from_id, original_message_id, sent_timestamp).await
handle_mdn(context, from_id, original_message_id, self.timestamp_sent).await
{
warn!(context, "Could not handle MDN: {err:#}.");
}
@@ -2720,6 +2744,7 @@ Chat-Version: 1.0\n\
Message-ID: <bar@example.org>\n\
To: Alice <alice@example.org>\n\
From: Bob <bob@example.org>\n\
Auto-Submitted: auto-replied\n\
Content-Type: multipart/report; report-type=disposition-notification;\n\t\
boundary=\"kJBbU58X1xeWNHgBtTbMk80M5qnV4N\"\n\
\n\
@@ -2755,6 +2780,7 @@ Disposition: manual-action/MDN-sent-automatically; displayed\n\
assert_eq!(message.parts.len(), 1);
assert_eq!(message.mdn_reports.len(), 1);
assert_eq!(message.is_bot, None);
}
/// Test parsing multiple MDNs combined in a single message.
@@ -3786,4 +3812,40 @@ Content-Disposition: reaction\n\
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_schleuder() -> Result<()> {
let context = TestContext::new_alice().await;
let raw = include_bytes!("../test-data/message/schleuder.eml");
let msg = MimeMessage::from_bytes(&context.ctx, &raw[..], None)
.await
.unwrap();
assert_eq!(msg.parts.len(), 2);
// Header part.
assert_eq!(msg.parts[0].typ, Viewtype::Text);
// Actual contents part.
assert_eq!(msg.parts[1].typ, Viewtype::Text);
assert_eq!(msg.parts[1].msg, "hello,\nbye");
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_tlsrpt() -> Result<()> {
let context = TestContext::new_alice().await;
let raw = include_bytes!("../test-data/message/tlsrpt.eml");
let msg = MimeMessage::from_bytes(&context.ctx, &raw[..], None)
.await
.unwrap();
assert_eq!(msg.parts.len(), 1);
assert_eq!(msg.parts[0].typ, Viewtype::File);
assert_eq!(msg.parts[0].msg, "Report Domain: nine.testrun.org Submitter: google.com Report-ID: <2024.01.20T00.00.00Z+nine.testrun.org@google.com> This is an aggregate TLS report from google.com");
Ok(())
}
}

View File

@@ -65,7 +65,7 @@ pub enum Param {
/// For Messages: the message is a reaction.
Reaction = b'x',
/// For Messages: a message with Auto-Submitted header ("bot").
/// For Messages: a message with "Auto-Submitted: auto-generated" header ("bot").
Bot = b'b',
/// For Messages: unset or 0=not forwarded,
@@ -84,7 +84,7 @@ pub enum Param {
/// For Messages
Arg2 = b'F',
/// For Messages
/// `Secure-Join-Fingerprint` header for `{vc,vg}-request-with-auth` messages.
Arg3 = b'G',
/// For Messages

View File

@@ -6,6 +6,7 @@ use num_traits::FromPrimitive;
use crate::aheader::{Aheader, EncryptPreference};
use crate::chat::{self, Chat};
use crate::chatlist::Chatlist;
use crate::config::Config;
use crate::constants::Chattype;
use crate::contact::{addr_cmp, Contact, ContactAddress, Origin};
use crate::context::Context;
@@ -26,17 +27,6 @@ pub enum PeerstateKeyType {
PublicKey,
}
/// Verification status of the contact peerstate.
#[derive(Debug, PartialEq, Eq, Clone, Copy, FromPrimitive)]
#[repr(u8)]
pub enum PeerstateVerifiedStatus {
/// Peerstate is not verified.
Unverified = 0,
//Verified = 1, // not used
/// Peerstate is verified and we assume that the contact has verified our peerstate.
BidirectVerified = 2,
}
/// Peerstate represents the state of an Autocrypt peer.
#[derive(Debug, PartialEq, Eq, Clone)]
pub struct Peerstate {
@@ -94,6 +84,10 @@ pub struct Peerstate {
/// The address that introduced secondary verified key.
pub secondary_verifier: Option<String>,
/// Row ID of the key in the `keypairs` table
/// that we think the peer knows as verified.
pub backward_verified_key_id: Option<i64>,
/// True if it was detected
/// that the fingerprint of the key used in chats with
/// opportunistic encryption was changed after Peerstate creation.
@@ -119,6 +113,7 @@ impl Peerstate {
secondary_verified_key: None,
secondary_verified_key_fingerprint: None,
secondary_verifier: None,
backward_verified_key_id: None,
fingerprint_changed: false,
}
}
@@ -148,6 +143,7 @@ impl Peerstate {
secondary_verified_key: None,
secondary_verified_key_fingerprint: None,
secondary_verifier: None,
backward_verified_key_id: None,
fingerprint_changed: false,
}
}
@@ -159,7 +155,8 @@ impl Peerstate {
verified_key, verified_key_fingerprint, \
verifier, \
secondary_verified_key, secondary_verified_key_fingerprint, \
secondary_verifier \
secondary_verifier, \
backward_verified_key_id \
FROM acpeerstates \
WHERE addr=? COLLATE NOCASE LIMIT 1;";
Self::from_stmt(context, query, (addr,)).await
@@ -175,7 +172,8 @@ impl Peerstate {
verified_key, verified_key_fingerprint, \
verifier, \
secondary_verified_key, secondary_verified_key_fingerprint, \
secondary_verifier \
secondary_verifier, \
backward_verified_key_id \
FROM acpeerstates \
WHERE public_key_fingerprint=? \
OR gossip_key_fingerprint=? \
@@ -198,7 +196,8 @@ impl Peerstate {
verified_key, verified_key_fingerprint, \
verifier, \
secondary_verified_key, secondary_verified_key_fingerprint, \
secondary_verifier \
secondary_verifier, \
backward_verified_key_id \
FROM acpeerstates \
WHERE verified_key_fingerprint=? \
OR addr=? COLLATE NOCASE \
@@ -266,6 +265,7 @@ impl Peerstate {
let secondary_verifier: Option<String> = row.get("secondary_verifier")?;
secondary_verifier.filter(|s| !s.is_empty())
},
backward_verified_key_id: row.get("backward_verified_key_id")?,
fingerprint_changed: false,
};
@@ -373,8 +373,8 @@ impl Peerstate {
}
/// Returns the contents of the `Autocrypt-Gossip` header for outgoing messages.
pub fn render_gossip_header(&self, min_verified: PeerstateVerifiedStatus) -> Option<String> {
if let Some(key) = self.peek_key(min_verified) {
pub fn render_gossip_header(&self, verified: bool) -> Option<String> {
if let Some(key) = self.peek_key(verified) {
let header = Aheader::new(
self.addr.clone(),
key.clone(), // TODO: avoid cloning
@@ -397,43 +397,41 @@ impl Peerstate {
/// Converts the peerstate into the contact public key.
///
/// Similar to [`Self::peek_key`], but consumes the peerstate and returns owned key.
pub fn take_key(mut self, min_verified: PeerstateVerifiedStatus) -> Option<SignedPublicKey> {
match min_verified {
PeerstateVerifiedStatus::BidirectVerified => self.verified_key.take(),
PeerstateVerifiedStatus::Unverified => {
self.public_key.take().or_else(|| self.gossip_key.take())
}
pub fn take_key(mut self, verified: bool) -> Option<SignedPublicKey> {
if verified {
self.verified_key.take()
} else {
self.public_key.take().or_else(|| self.gossip_key.take())
}
}
/// Returns a reference to the contact public key.
///
/// `min_verified` determines the minimum required verification status of the key.
/// `verified` determines the required verification status of the key.
/// If verified key is requested, returns the verified key,
/// otherwise returns the Autocrypt key.
///
/// Returned key is suitable for sending in `Autocrypt-Gossip` header.
///
/// Returns `None` if there is no suitable public key.
pub fn peek_key(&self, min_verified: PeerstateVerifiedStatus) -> Option<&SignedPublicKey> {
match min_verified {
PeerstateVerifiedStatus::BidirectVerified => self.verified_key.as_ref(),
PeerstateVerifiedStatus::Unverified => {
self.public_key.as_ref().or(self.gossip_key.as_ref())
}
pub fn peek_key(&self, verified: bool) -> Option<&SignedPublicKey> {
if verified {
self.verified_key.as_ref()
} else {
self.public_key.as_ref().or(self.gossip_key.as_ref())
}
}
/// Returns a reference to the contact's public key fingerprint.
///
/// Similar to [`Self::peek_key`], but returns the fingerprint instead of the key.
fn peek_key_fingerprint(&self, min_verified: PeerstateVerifiedStatus) -> Option<&Fingerprint> {
match min_verified {
PeerstateVerifiedStatus::BidirectVerified => self.verified_key_fingerprint.as_ref(),
PeerstateVerifiedStatus::Unverified => self
.public_key_fingerprint
fn peek_key_fingerprint(&self, verified: bool) -> Option<&Fingerprint> {
if verified {
self.verified_key_fingerprint.as_ref()
} else {
self.public_key_fingerprint
.as_ref()
.or(self.gossip_key_fingerprint.as_ref()),
.or(self.gossip_key_fingerprint.as_ref())
}
}
@@ -443,10 +441,20 @@ impl Peerstate {
/// Note that verified groups always use the verified key no matter if the
/// opportunistic key matches or not.
pub(crate) fn is_using_verified_key(&self) -> bool {
let verified = self.peek_key_fingerprint(PeerstateVerifiedStatus::BidirectVerified);
let verified = self.peek_key_fingerprint(true);
verified.is_some()
&& verified == self.peek_key_fingerprint(PeerstateVerifiedStatus::Unverified)
verified.is_some() && verified == self.peek_key_fingerprint(false)
}
pub(crate) async fn is_backward_verified(&self, context: &Context) -> Result<bool> {
let Some(backward_verified_key_id) = self.backward_verified_key_id else {
return Ok(false);
};
let self_key_id = context.get_config_i64(Config::KeyId).await?;
let backward_verified = backward_verified_key_id == self_key_id;
Ok(backward_verified)
}
/// Set this peerstate to verified
@@ -524,8 +532,9 @@ impl Peerstate {
secondary_verified_key,
secondary_verified_key_fingerprint,
secondary_verifier,
backward_verified_key_id,
addr)
VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)
VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)
ON CONFLICT (addr)
DO UPDATE SET
last_seen = excluded.last_seen,
@@ -541,7 +550,8 @@ impl Peerstate {
verifier = excluded.verifier,
secondary_verified_key = excluded.secondary_verified_key,
secondary_verified_key_fingerprint = excluded.secondary_verified_key_fingerprint,
secondary_verifier = excluded.secondary_verifier",
secondary_verifier = excluded.secondary_verifier,
backward_verified_key_id = excluded.backward_verified_key_id",
(
self.last_seen,
self.last_seen_autocrypt,
@@ -559,6 +569,7 @@ impl Peerstate {
.as_ref()
.map(|fp| fp.hex()),
self.secondary_verifier.as_deref().unwrap_or(""),
self.backward_verified_key_id,
&self.addr,
),
)
@@ -710,46 +721,46 @@ pub(crate) async fn maybe_do_aeap_transition(
mime_parser: &mut crate::mimeparser::MimeMessage,
) -> Result<()> {
let info = &mime_parser.decryption_info;
if let Some(peerstate) = &info.peerstate {
// If the from addr is different from the peerstate address we know,
// we may want to do an AEAP transition.
if !addr_cmp(&peerstate.addr, &mime_parser.from.addr)
// Check if it's a chat message; we do this to avoid
// some accidental transitions if someone writes from multiple
// addresses with an MUA.
&& mime_parser.has_chat_version()
// Check if the message is signed correctly.
// If it's not signed correctly, the whole autocrypt header will be mostly
// ignored anyway and the message shown as not encrypted, so we don't
// have to handle this case.
&& !mime_parser.signatures.is_empty()
// Check if the From: address was also in the signed part of the email.
// Without this check, an attacker could replay a message from Alice
// to Bob. Then Bob's device would do an AEAP transition from Alice's
// to the attacker's address, allowing for easier phishing.
&& mime_parser.from_is_signed
&& info.message_time > peerstate.last_seen
{
let info = &mut mime_parser.decryption_info;
let peerstate = info.peerstate.as_mut().context("no peerstate??")?;
// Add info messages to chats with this (verified) contact
//
peerstate
.handle_setup_change(
context,
info.message_time,
PeerstateChange::Aeap(info.from.clone()),
)
.await?;
let Some(peerstate) = &info.peerstate else {
return Ok(());
};
peerstate.addr = info.from.clone();
let header = info.autocrypt_header.as_ref().context(
"Internal error: Tried to do an AEAP transition without an autocrypt header??",
)?;
peerstate.apply_header(header, info.message_time);
// If the from addr is different from the peerstate address we know,
// we may want to do an AEAP transition.
if !addr_cmp(&peerstate.addr, &mime_parser.from.addr)
// Check if it's a chat message; we do this to avoid
// some accidental transitions if someone writes from multiple
// addresses with an MUA.
&& mime_parser.has_chat_version()
// Check if the message is signed correctly.
// Although checking `from_is_signed` below is sufficient, let's play it safe.
&& !mime_parser.signatures.is_empty()
// Check if the From: address was also in the signed part of the email.
// Without this check, an attacker could replay a message from Alice
// to Bob. Then Bob's device would do an AEAP transition from Alice's
// to the attacker's address, allowing for easier phishing.
&& mime_parser.from_is_signed
&& info.message_time > peerstate.last_seen
{
let info = &mut mime_parser.decryption_info;
let peerstate = info.peerstate.as_mut().context("no peerstate??")?;
// Add info messages to chats with this (verified) contact
//
peerstate
.handle_setup_change(
context,
info.message_time,
PeerstateChange::Aeap(info.from.clone()),
)
.await?;
peerstate.save_to_db(&context.sql).await?;
}
peerstate.addr = info.from.clone();
let header = info.autocrypt_header.as_ref().context(
"Internal error: Tried to do an AEAP transition without an autocrypt header??",
)?;
peerstate.apply_header(header, info.message_time);
peerstate.save_to_db(&context.sql).await?;
}
Ok(())
@@ -820,6 +831,7 @@ mod tests {
secondary_verified_key: None,
secondary_verified_key_fingerprint: None,
secondary_verifier: None,
backward_verified_key_id: None,
fingerprint_changed: false,
};
@@ -863,6 +875,7 @@ mod tests {
secondary_verified_key: None,
secondary_verified_key_fingerprint: None,
secondary_verifier: None,
backward_verified_key_id: None,
fingerprint_changed: false,
};
@@ -899,6 +912,7 @@ mod tests {
secondary_verified_key: None,
secondary_verified_key_fingerprint: None,
secondary_verifier: None,
backward_verified_key_id: None,
fingerprint_changed: false,
};
@@ -965,6 +979,7 @@ mod tests {
secondary_verified_key: None,
secondary_verified_key_fingerprint: None,
secondary_verifier: None,
backward_verified_key_id: None,
fingerprint_changed: false,
};

View File

@@ -25,6 +25,8 @@ use crate::socks::Socks5Config;
use crate::token;
const OPENPGP4FPR_SCHEME: &str = "OPENPGP4FPR:"; // yes: uppercase
const IDELTACHAT_SCHEME: &str = "https://i.delta.chat/#";
const IDELTACHAT_NOSLASH_SCHEME: &str = "https://i.delta.chat#";
const DCACCOUNT_SCHEME: &str = "DCACCOUNT:";
pub(super) const DCLOGIN_SCHEME: &str = "DCLOGIN:";
const DCWEBRTC_SCHEME: &str = "DCWEBRTC:";
@@ -253,6 +255,10 @@ pub async fn check_qr(context: &Context, qr: &str) -> Result<Qr> {
decode_openpgp(context, qr)
.await
.context("failed to decode OPENPGP4FPR QR code")?
} else if qr.starts_with(IDELTACHAT_SCHEME) {
decode_ideltachat(context, IDELTACHAT_SCHEME, qr).await?
} else if qr.starts_with(IDELTACHAT_NOSLASH_SCHEME) {
decode_ideltachat(context, IDELTACHAT_NOSLASH_SCHEME, qr).await?
} else if starts_with_ignore_case(qr, DCACCOUNT_SCHEME) {
decode_account(qr)?
} else if starts_with_ignore_case(qr, DCLOGIN_SCHEME) {
@@ -301,11 +307,12 @@ pub fn format_backup(qr: &Qr) -> Result<String> {
async fn decode_openpgp(context: &Context, qr: &str) -> Result<Qr> {
let payload = &qr[OPENPGP4FPR_SCHEME.len()..];
let (fingerprint, fragment) = match payload.find('#').map(|offset| {
let (fp, rest) = payload.split_at(offset);
// need to remove the # from the fragment
(fp, &rest[1..])
}) {
// macOS and iOS sometimes replace the # with %23 (uri encode it), we should be able to parse this wrong format too.
// see issue https://github.com/deltachat/deltachat-core-rust/issues/1969 for more info
let (fingerprint, fragment) = match payload
.split_once('#')
.or_else(|| payload.split_once("%23"))
{
Some(pair) => pair,
None => (payload, ""),
};
@@ -453,6 +460,15 @@ async fn decode_openpgp(context: &Context, qr: &str) -> Result<Qr> {
}
}
/// scheme: `https://i.delta.chat[/]#FINGERPRINT&a=ADDR[&OPTIONAL_PARAMS]`
async fn decode_ideltachat(context: &Context, prefix: &str, qr: &str) -> Result<Qr> {
let qr = qr.replacen(prefix, OPENPGP4FPR_SCHEME, 1);
let qr = qr.replacen('&', "#", 1);
decode_openpgp(context, &qr)
.await
.context("failed to decode {prefix} QR code")
}
/// scheme: `DCACCOUNT:https://example.org/new_email?t=1w_7wDjgjelxeX884x96v3`
fn decode_account(qr: &str) -> Result<Qr> {
let payload = qr
@@ -943,6 +959,40 @@ mod tests {
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_decode_ideltachat_link() -> Result<()> {
let ctx = TestContext::new().await;
let qr = check_qr(
&ctx.ctx,
"https://i.delta.chat/#79252762C34C5096AF57958F4FC3D21A81B0F0A7&a=cli%40deltachat.de&g=test%20%3F+test%20%21&x=h-0oKQf2CDK&i=9JEXlxAqGM0&s=0V7LzL9cxRL"
).await?;
assert!(matches!(qr, Qr::AskVerifyGroup { .. }));
let qr = check_qr(
&ctx.ctx,
"https://i.delta.chat#79252762C34C5096AF57958F4FC3D21A81B0F0A7&a=cli%40deltachat.de&g=test%20%3F+test%20%21&x=h-0oKQf2CDK&i=9JEXlxAqGM0&s=0V7LzL9cxRL"
).await?;
assert!(matches!(qr, Qr::AskVerifyGroup { .. }));
Ok(())
}
// macOS and iOS sometimes replace the # with %23 (uri encode it), we should be able to parse this wrong format too.
// see issue https://github.com/deltachat/deltachat-core-rust/issues/1969 for more info
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_decode_openpgp_tolerance_for_issue_1969() -> Result<()> {
let ctx = TestContext::new().await;
let qr = check_qr(
&ctx.ctx,
"OPENPGP4FPR:79252762C34C5096AF57958F4FC3D21A81B0F0A7%23a=cli%40deltachat.de&g=test%20%3F+test%20%21&x=h-0oKQf2CDK&i=9JEXlxAqGM0&s=0V7LzL9cxRL"
).await?;
assert!(matches!(qr, Qr::AskVerifyGroup { .. }));
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_decode_openpgp_group() -> Result<()> {
let ctx = TestContext::new().await;
@@ -1057,6 +1107,7 @@ mod tests {
secondary_verified_key: None,
secondary_verified_key_fingerprint: None,
secondary_verifier: None,
backward_verified_key_id: None,
fingerprint_changed: false,
};
assert!(

View File

@@ -252,7 +252,7 @@ pub(crate) async fn set_msg_reaction(
contact_id: ContactId,
reaction: Reaction,
) -> Result<()> {
if let Some(msg_id) = rfc724_mid_exists(context, in_reply_to).await? {
if let Some((msg_id, _)) = rfc724_mid_exists(context, in_reply_to).await? {
set_msg_id_reaction(context, msg_id, chat_id, contact_id, reaction).await
} else {
info!(
@@ -316,7 +316,7 @@ mod tests {
use crate::contact::{Contact, ContactAddress, Origin};
use crate::download::DownloadState;
use crate::message::MessageState;
use crate::receive_imf::{receive_imf, receive_imf_inner};
use crate::receive_imf::{receive_imf, receive_imf_from_inbox};
use crate::test_utils::TestContext;
use crate::test_utils::TestContextManager;
@@ -425,7 +425,7 @@ Content-Disposition: reaction\n\
let contacts = reactions.contacts();
assert_eq!(contacts.len(), 1);
assert_eq!(contacts.get(0), Some(&bob_id));
assert_eq!(contacts.first(), Some(&bob_id));
let bob_reaction = reactions.get(bob_id);
assert_eq!(bob_reaction.is_empty(), false);
assert_eq!(bob_reaction.emojis(), vec!["👍"]);
@@ -526,7 +526,7 @@ Here's my footer -- bob@example.net"
assert_eq!(reactions.to_string(), "👍1");
let contacts = reactions.contacts();
assert_eq!(contacts.len(), 1);
let bob_id = contacts.get(0).unwrap();
let bob_id = contacts.first().unwrap();
let bob_reaction = reactions.get(*bob_id);
assert_eq!(bob_reaction.is_empty(), false);
assert_eq!(bob_reaction.emojis(), vec!["👍"]);
@@ -568,7 +568,7 @@ Here's my footer -- bob@example.net"
let msg_full = format!("{msg_header}\n\n100k text...");
// Alice downloads message from Bob partially.
let alice_received_message = receive_imf_inner(
let alice_received_message = receive_imf_from_inbox(
&alice,
"first@example.org",
msg_header.as_bytes(),
@@ -578,13 +578,13 @@ Here's my footer -- bob@example.net"
)
.await?
.unwrap();
let alice_msg_id = *alice_received_message.msg_ids.get(0).unwrap();
let alice_msg_id = *alice_received_message.msg_ids.first().unwrap();
// Bob downloads own message on the other device.
let bob_received_message = receive_imf(&bob, msg_full.as_bytes(), false)
.await?
.unwrap();
let bob_msg_id = *bob_received_message.msg_ids.get(0).unwrap();
let bob_msg_id = *bob_received_message.msg_ids.first().unwrap();
// Bob reacts to own message.
send_reaction(&bob, bob_msg_id, "👍").await.unwrap();
@@ -599,7 +599,7 @@ Here's my footer -- bob@example.net"
assert_eq!(msg.download_state(), DownloadState::Available);
// Alice downloads full message.
receive_imf_inner(
receive_imf_from_inbox(
&alice,
"first@example.org",
msg_full.as_bytes(),

File diff suppressed because it is too large Load Diff

View File

@@ -2,11 +2,11 @@ use tokio::fs;
use super::*;
use crate::aheader::EncryptPreference;
use crate::chat::{self, get_chat_msgs, ChatItem, ChatVisibility};
use crate::chat::{
add_contact_to_chat, add_to_chat_contacts_table, create_group_chat, get_chat_contacts,
is_contact_in_chat, remove_contact_from_chat, send_text_msg,
};
use crate::chat::{get_chat_msgs, ChatItem, ChatVisibility};
use crate::chatlist::Chatlist;
use crate::config::Config;
use crate::constants::{DC_GCL_FOR_FORWARDING, DC_GCL_NO_SPECIALS};
@@ -310,6 +310,56 @@ async fn test_read_receipt_and_unarchive() -> Result<()> {
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_mdn_and_alias() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = tcm.alice().await;
let bob = tcm.bob().await;
let alice_chat = alice.create_chat(&bob).await;
let sent = alice.send_text(alice_chat.id, "alice -> bob").await;
let msg_id = sent.sender_msg_id;
receive_imf(
&alice,
format!(
"Received: (Postfix, from userid 1000); Mon, 4 Dec 2006 14:51:39 +0100 (CET)\n\
From: bob@example.net\n\
To: alicechat@example.org\n\
Subject: message opened\n\
Date: Sun, 22 Mar 2020 23:37:57 +0000\n\
Chat-Version: 1.0\n\
Message-ID: <aranudiaerudiaduiaertd@example.com>\n\
Content-Type: multipart/report; report-type=disposition-notification; boundary=\"SNIPP\"\n\
\n\
\n\
--SNIPP\n\
Content-Type: text/plain; charset=utf-8\n\
\n\
Read receipts do not guarantee sth. was read.\n\
\n\
\n\
--SNIPP\n\
Content-Type: message/disposition-notification\n\
\n\
Reporting-UA: Delta Chat 1.28.0\n\
Original-Recipient: rfc822;bob@example.com\n\
Final-Recipient: rfc822;bob@example.com\n\
Original-Message-ID: <{msg_id}>\n\
Disposition: manual-action/MDN-sent-automatically; displayed\n\
\n\
\n\
--SNIPP--",
)
.as_bytes(),
false,
)
.await?;
let chats = Chatlist::try_load(&alice, 0, None, None).await?;
assert_eq!(chats.len(), 1);
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_no_from() {
// if there is no from given, from_id stays 0 which is just fine. These messages
@@ -322,7 +372,7 @@ async fn test_no_from() {
let chats = Chatlist::try_load(&t, 0, None, None).await.unwrap();
assert!(chats.get_msg_id(0).is_err());
receive_imf(
let received = receive_imf(
context,
b"Received: (Postfix, from userid 1000); Mon, 4 Dec 2006 14:51:39 +0100 (CET)\n\
To: bob@example.com\n\
@@ -335,8 +385,13 @@ async fn test_no_from() {
false,
)
.await
.unwrap()
.unwrap();
// Check that tombstone MsgId is returned.
assert_eq!(received.msg_ids.len(), 1);
assert!(!received.msg_ids[0].is_special());
let chats = Chatlist::try_load(&t, 0, None, None).await.unwrap();
// Check that the message is not shown to the user:
assert!(chats.is_empty());
@@ -623,7 +678,7 @@ async fn test_parse_ndn(
rfc724_mid_outgoing: &str,
raw_ndn: &[u8],
error_msg: Option<&str>,
) {
) -> (TestContext, MsgId) {
let t = TestContext::new().await;
t.configure_addr(self_addr).await;
@@ -670,6 +725,40 @@ async fn test_parse_ndn(
);
assert_eq!(msg.error(), error_msg.map(|error| error.to_string()));
(t, msg_id)
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_resend_after_ndn() -> Result<()> {
let (t, msg_id) = test_parse_ndn(
"alice@testrun.org",
"hcksocnsofoejx@five.chat",
"Mr.A7pTA5IgrUA.q4bP41vAJOp@testrun.org",
include_bytes!("../../test-data/message/testrun_ndn.eml"),
Some("Undelivered Mail Returned to Sender This is the mail system at host hq5.merlinux.eu.\n\nI\'m sorry to have to inform you that your message could not\nbe delivered to one or more recipients. It\'s attached below.\n\nFor further assistance, please send mail to postmaster.\n\nIf you do so, please include this problem report. You can\ndelete your own text from the attached returned message.\n\n The mail system\n\n<hcksocnsofoejx@five.chat>: host mail.five.chat[195.62.125.103] said: 550 5.1.1\n <hcksocnsofoejx@five.chat>: Recipient address rejected: User unknown in\n virtual mailbox table (in reply to RCPT TO command)"),
)
.await;
chat::resend_msgs(&t, &[msg_id]).await?;
let msg = Message::load_from_db(&t, msg_id).await?;
assert_eq!(msg.state, MessageState::OutPending);
assert_eq!(msg.error(), None);
// Alice receives a BCC-self copy of their message.
receive_imf(
&t,
"To: hcksocnsofoejx@five.chat\n\
From: alice@testrun.org\n\
Date: Today, 2 January 2024 00:00:00 -300\n\
Message-ID: Mr.A7pTA5IgrUA.q4bP41vAJOp@testrun.org\n\
\n\
hi"
.as_bytes(),
false,
)
.await?;
let msg = t.get_last_msg().await;
assert_eq!(msg.state, MessageState::OutDelivered);
assert_eq!(msg.error(), None);
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
@@ -1793,7 +1882,7 @@ async fn create_test_alias(chat_request: bool, group_request: bool) -> (TestCont
.await
.unwrap();
let msg_id = rfc724_mid_exists(&claire, "non-dc-1@example.org")
let (msg_id, _) = rfc724_mid_exists(&claire, "non-dc-1@example.org")
.await
.unwrap()
.unwrap();
@@ -2595,6 +2684,36 @@ async fn test_read_receipts_dont_create_chats() -> Result<()> {
Ok(())
}
/// Test that read receipts don't unmark contacts as bots.
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_read_receipts_dont_unmark_bots() -> Result<()> {
let alice = &TestContext::new_alice().await;
let bob = &TestContext::new_bob().await;
let ab_contact = alice.add_or_lookup_contact(bob).await;
ab_contact.id.mark_bot(alice, true).await?;
let alice_chat = alice.create_chat(bob).await;
// Alice sends and Bob receives a message.
bob.recv_msg(&alice.send_text(alice_chat.id, "Message").await)
.await;
let received_msg = bob.get_last_msg().await;
// Bob sends a read receipt.
let mdn_mimefactory =
crate::mimefactory::MimeFactory::from_mdn(bob, &received_msg, vec![]).await?;
let rendered_mdn = mdn_mimefactory.render(bob).await?;
let mdn_body = rendered_mdn.message;
// Alice receives the read receipt.
receive_imf(alice, mdn_body.as_bytes(), false).await?;
let msg = alice.get_last_msg_in(alice_chat.id).await;
assert_eq!(msg.state, MessageState::OutMdnRcvd);
let ab_contact = alice.add_or_lookup_contact(bob).await;
assert!(ab_contact.is_bot());
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_gmx_forwarded_msg() -> Result<()> {
let t = TestContext::new_alice().await;
@@ -3105,7 +3224,8 @@ async fn test_thunderbird_autocrypt() -> Result<()> {
let t = TestContext::new_bob().await;
let raw = include_bytes!("../../test-data/message/thunderbird_with_autocrypt.eml");
receive_imf(&t, raw, false).await?;
let received_msg = receive_imf(&t, raw, false).await?.unwrap();
assert!(received_msg.from_is_signed);
let peerstate = Peerstate::from_addr(&t, "alice@example.org")
.await?
@@ -3186,7 +3306,8 @@ async fn test_thunderbird_unsigned() -> Result<()> {
// Alice receives an unsigned message from Bob.
let raw = include_bytes!("../../test-data/message/thunderbird_encrypted_unsigned.eml");
receive_imf(&alice, raw, false).await?;
let received_msg = receive_imf(&alice, raw, false).await?.unwrap();
assert!(!received_msg.from_is_signed);
let msg = alice.get_last_msg().await;
assert!(!msg.get_showpadlock());
@@ -3195,6 +3316,43 @@ async fn test_thunderbird_unsigned() -> Result<()> {
Ok(())
}
/// Bob receives an encrypted unsigned message with only an unencrypted Subject.
///
/// Test that the message is displayed without any errors,
/// but also without a padlock, but with the Subject.
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_thunderbird_unsigned_with_unencrypted_subject() -> Result<()> {
let bob = TestContext::new_bob().await;
let raw = include_bytes!(
"../../test-data/message/thunderbird_encrypted_unsigned_with_unencrypted_subject.eml"
);
receive_imf(&bob, raw, false).await?;
let msg = bob.get_last_msg().await;
assert!(!msg.get_showpadlock());
assert!(msg.error().is_none());
assert_eq!(msg.get_subject(), "Hello!");
Ok(())
}
/// Tests that DC takes the correct Message-ID from the encrypted message part, not the unencrypted
/// one messed up by the server.
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_messed_up_message_id() -> Result<()> {
let t = TestContext::new_bob().await;
let raw = include_bytes!("../../test-data/message/messed_up_message_id.eml");
receive_imf(&t, raw, false).await?;
assert_eq!(
t.get_last_msg().await.rfc724_mid,
"0bb9ffe1-2596-d997-95b4-1fef8cc4808e@example.org"
);
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_mua_user_adds_member() -> Result<()> {
let t = TestContext::new_alice().await;
@@ -3917,7 +4075,7 @@ async fn test_partial_group_consistency() -> Result<()> {
.unwrap();
// Bob receives partial message.
let msg_id = receive_imf_inner(
let msg_id = receive_imf_from_inbox(
&bob,
"first@example.org",
b"From: Alice <alice@example.org>\n\
@@ -3970,7 +4128,7 @@ Chat-Group-Member-Added: charlie@example.com",
assert_eq!(contacts.len(), 3);
// Bob fully reives the partial message.
let msg_id = receive_imf_inner(
let msg_id = receive_imf_from_inbox(
&bob,
"first@example.org",
b"From: Alice <alice@example.org>\n\

View File

@@ -15,7 +15,7 @@ use self::connectivity::ConnectivityStore;
use crate::config::Config;
use crate::contact::{ContactId, RecentlySeenLoop};
use crate::context::Context;
use crate::download::download_msg;
use crate::download::{download_msg, DownloadState};
use crate::ephemeral::{self, delete_expired_imap_messages};
use crate::events::EventType;
use crate::imap::{FolderMeaning, Imap};
@@ -70,8 +70,11 @@ impl SchedulerState {
context.new_msgs_notify.notify_one();
let ctx = context.clone();
match Scheduler::start(context).await {
Ok(scheduler) => *inner = InnerSchedulerState::Started(scheduler),
match Scheduler::start(&context).await {
Ok(scheduler) => {
*inner = InnerSchedulerState::Started(scheduler);
context.emit_event(EventType::ConnectivityChanged);
}
Err(err) => error!(&ctx, "Failed to start IO: {:#}", err),
}
}
@@ -116,6 +119,7 @@ impl SchedulerState {
debug_logging.loop_handle.abort();
}
let prev_state = std::mem::replace(&mut *inner, new_state);
context.emit_event(EventType::ConnectivityChanged);
match prev_state {
InnerSchedulerState::Started(scheduler) => scheduler.stop(context).await,
InnerSchedulerState::Stopped | InnerSchedulerState::Paused { .. } => (),
@@ -346,6 +350,16 @@ async fn download_msgs(context: &Context, imap: &mut Imap) -> Result<()> {
for msg_id in msg_ids {
if let Err(err) = download_msg(context, msg_id, imap).await {
warn!(context, "Failed to download message {msg_id}: {:#}.", err);
// Update download state to failure
// so it can be retried.
//
// On success update_download_state() is not needed
// as receive_imf() already
// set the state and emitted the event.
msg_id
.update_download_state(context, DownloadState::Failure)
.await?;
}
context
.sql
@@ -446,6 +460,10 @@ async fn inbox_loop(
warn!(ctx, "Failed to download messages: {:#}", err);
}
if let Err(err) = connection.fetch_metadata(&ctx).await {
warn!(ctx, "Failed to fetch metadata: {err:#}.");
}
fetch_idle(&ctx, &mut connection, FolderMeaning::Inbox).await;
}
};
@@ -459,6 +477,39 @@ async fn inbox_loop(
.await;
}
/// Convert folder meaning
/// used internally by [fetch_idle] and [Context::background_fetch]
pub async fn convert_folder_meaning(
ctx: &Context,
folder_meaning: FolderMeaning,
) -> Result<(Config, String)> {
let folder_config = match folder_meaning.to_config() {
Some(c) => c,
None => {
bail!("Bad folder meaning: {}", folder_meaning);
}
};
let folder = match ctx.get_config(folder_config).await {
Ok(folder) => folder,
Err(err) => {
bail!(
"Can not watch {} folder, failed to retrieve config: {:#}",
folder_config,
err
);
}
};
let watch_folder = if let Some(watch_folder) = folder {
watch_folder
} else {
bail!("Can not watch {} folder, not set", folder_config);
};
Ok((folder_config, watch_folder))
}
/// Implement a single iteration of IMAP loop.
///
/// This function performs all IMAP operations on a single folder, selecting it if necessary and
@@ -466,40 +517,20 @@ async fn inbox_loop(
/// critical operation fails such as fetching new messages fails, connection is reset via
/// `trigger_reconnect`, so a fresh one can be opened.
async fn fetch_idle(ctx: &Context, connection: &mut Imap, folder_meaning: FolderMeaning) {
let folder_config = match folder_meaning.to_config() {
Some(c) => c,
None => {
error!(ctx, "Bad folder meaning: {}", folder_meaning);
let (folder_config, watch_folder) = match convert_folder_meaning(ctx, folder_meaning).await {
Ok(meaning) => meaning,
Err(error) => {
// Warning instead of error because the folder may not be configured.
// For example, this happens if the server does not have Sent folder
// but watching Sent folder is enabled.
warn!(ctx, "Error converting IMAP Folder name: {:?}", error);
connection.connectivity.set_not_configured(ctx).await;
connection
.fake_idle(ctx, None, FolderMeaning::Unknown)
.await;
return;
}
};
let folder = match ctx.get_config(folder_config).await {
Ok(folder) => folder,
Err(err) => {
warn!(
ctx,
"Can not watch {} folder, failed to retrieve config: {:#}", folder_config, err
);
connection
.fake_idle(ctx, None, FolderMeaning::Unknown)
.await;
return;
}
};
let watch_folder = if let Some(watch_folder) = folder {
watch_folder
} else {
connection.connectivity.set_not_configured(ctx).await;
info!(ctx, "Can not watch {} folder, not set", folder_config);
connection
.fake_idle(ctx, None, FolderMeaning::Unknown)
.await;
return;
};
// connect and fake idle if unable to connect
if let Err(err) = connection
@@ -590,7 +621,7 @@ async fn fetch_idle(ctx: &Context, connection: &mut Imap, folder_meaning: Folder
.log_err(ctx)
.ok();
connection.connectivity.set_connected(ctx).await;
connection.connectivity.set_idle(ctx).await;
ctx.emit_event(EventType::ImapInboxIdle);
let Some(session) = connection.session.take() else {
@@ -727,7 +758,7 @@ async fn smtp_loop(
// Fake Idle
info!(ctx, "smtp fake idle - started");
match &connection.last_send_error {
None => connection.connectivity.set_connected(&ctx).await,
None => connection.connectivity.set_idle(&ctx).await,
Some(err) => connection.connectivity.set_err(&ctx, err).await,
}
@@ -772,7 +803,7 @@ async fn smtp_loop(
impl Scheduler {
/// Start the scheduler.
pub async fn start(ctx: Context) -> Result<Self> {
pub async fn start(ctx: &Context) -> Result<Self> {
let (smtp, smtp_handlers) = SmtpConnectionState::new();
let (smtp_start_send, smtp_start_recv) = oneshot::channel();
@@ -782,7 +813,7 @@ impl Scheduler {
let mut oboxes = Vec::new();
let mut start_recvs = Vec::new();
let (conn_state, inbox_handlers) = ImapConnectionState::new(&ctx).await?;
let (conn_state, inbox_handlers) = ImapConnectionState::new(ctx).await?;
let (inbox_start_send, inbox_start_recv) = oneshot::channel();
let handle = {
let ctx = ctx.clone();
@@ -803,7 +834,7 @@ impl Scheduler {
),
] {
if should_watch? {
let (conn_state, handlers) = ImapConnectionState::new(&ctx).await?;
let (conn_state, handlers) = ImapConnectionState::new(ctx).await?;
let (start_send, start_recv) = oneshot::channel();
let ctx = ctx.clone();
let handle = task::spawn(simple_imap_loop(ctx, start_send, handlers, meaning));

View File

@@ -33,10 +33,19 @@ enum DetailedConnectivity {
#[default]
Uninitialized,
Connecting,
Working,
InterruptingIdle,
/// Connection is just established, but there may be work to do.
Connected,
/// There is actual work to do, e.g. there are messages in SMTP queue
/// or we detected a message that should be downloaded.
Working,
InterruptingIdle,
/// Connection is established and is idle.
Idle,
/// The folder was configured not to be watched or configured_*_folder is not set
NotConfigured,
}
@@ -54,6 +63,8 @@ impl DetailedConnectivity {
// Just don't return a connectivity, probably the folder is configured not to be
// watched or there is e.g. no "Sent" folder, so we are not interested in it
DetailedConnectivity::NotConfigured => None,
DetailedConnectivity::Idle => Some(Connectivity::Connected),
}
}
@@ -65,7 +76,8 @@ impl DetailedConnectivity {
DetailedConnectivity::Connecting => "<span class=\"yellow dot\"></span>".to_string(),
DetailedConnectivity::Working
| DetailedConnectivity::InterruptingIdle
| DetailedConnectivity::Connected => "<span class=\"green dot\"></span>".to_string(),
| DetailedConnectivity::Connected
| DetailedConnectivity::Idle => "<span class=\"green dot\"></span>".to_string(),
}
}
@@ -75,9 +87,9 @@ impl DetailedConnectivity {
DetailedConnectivity::Uninitialized => "Not started".to_string(),
DetailedConnectivity::Connecting => stock_str::connecting(context).await,
DetailedConnectivity::Working => stock_str::updating(context).await,
DetailedConnectivity::InterruptingIdle | DetailedConnectivity::Connected => {
stock_str::connected(context).await
}
DetailedConnectivity::InterruptingIdle
| DetailedConnectivity::Connected
| DetailedConnectivity::Idle => stock_str::connected(context).await,
DetailedConnectivity::NotConfigured => "Not configured".to_string(),
}
}
@@ -94,9 +106,9 @@ impl DetailedConnectivity {
// We don't know any more than that the last message was sent successfully;
// since sending the last message, connectivity could have changed, which we don't notice
// until another message is sent
DetailedConnectivity::InterruptingIdle | DetailedConnectivity::Connected => {
stock_str::last_msg_sent_successfully(context).await
}
DetailedConnectivity::InterruptingIdle
| DetailedConnectivity::Connected
| DetailedConnectivity::Idle => stock_str::last_msg_sent_successfully(context).await,
DetailedConnectivity::NotConfigured => "Not configured".to_string(),
}
}
@@ -108,8 +120,9 @@ impl DetailedConnectivity {
DetailedConnectivity::Connecting => false,
DetailedConnectivity::Working => false,
DetailedConnectivity::InterruptingIdle => false,
DetailedConnectivity::Connected => true,
DetailedConnectivity::Connected => false, // Just connected, there may still be work to do.
DetailedConnectivity::NotConfigured => true,
DetailedConnectivity::Idle => true,
}
}
}
@@ -141,6 +154,9 @@ impl ConnectivityStore {
pub(crate) async fn set_not_configured(&self, context: &Context) {
self.set(context, DetailedConnectivity::NotConfigured).await;
}
pub(crate) async fn set_idle(&self, context: &Context) {
self.set(context, DetailedConnectivity::Idle).await;
}
async fn get_detailed(&self) -> DetailedConnectivity {
self.0.lock().await.deref().clone()
@@ -164,6 +180,7 @@ pub(crate) async fn idle_interrupted(inbox: ConnectivityStore, oboxes: Vec<Conne
// return Connected until DC is completely done with fetching folders; this also
// includes scan_folders() which happens on the inbox thread.
if *connectivity_lock == DetailedConnectivity::Connected
|| *connectivity_lock == DetailedConnectivity::Idle
|| *connectivity_lock == DetailedConnectivity::NotConfigured
{
*connectivity_lock = DetailedConnectivity::InterruptingIdle;
@@ -172,7 +189,9 @@ pub(crate) async fn idle_interrupted(inbox: ConnectivityStore, oboxes: Vec<Conne
for state in oboxes {
let mut connectivity_lock = state.0.lock().await;
if *connectivity_lock == DetailedConnectivity::Connected {
if *connectivity_lock == DetailedConnectivity::Connected
|| *connectivity_lock == DetailedConnectivity::Idle
{
*connectivity_lock = DetailedConnectivity::InterruptingIdle;
}
}

View File

@@ -20,6 +20,7 @@ use crate::mimeparser::{MimeMessage, SystemMessage};
use crate::param::Param;
use crate::peerstate::{Peerstate, PeerstateKeyType};
use crate::qr::check_qr;
use crate::securejoin::bob::JoinerProgress;
use crate::stock_str;
use crate::sync::Sync::*;
use crate::token;
@@ -173,7 +174,6 @@ async fn send_alice_handshake_msg(
context: &Context,
contact_id: ContactId,
step: &str,
fingerprint: Option<Fingerprint>,
) -> Result<()> {
let mut msg = Message {
viewtype: Viewtype::Text,
@@ -183,9 +183,6 @@ async fn send_alice_handshake_msg(
};
msg.param.set_cmd(SystemMessage::SecurejoinMessage);
msg.param.set(Param::Arg, step);
if let Some(fp) = fingerprint {
msg.param.set(Param::Arg3, fp.hex());
}
msg.param.set_int(Param::GuaranteeE2ee, 1);
chat::send_msg(
context,
@@ -204,7 +201,9 @@ async fn info_chat_id(context: &Context, contact_id: ContactId) -> Result<ChatId
Ok(chat_id_blocked.id)
}
async fn fingerprint_equals_sender(
/// Checks fingerprint and marks the contact as forward verified
/// if fingerprint matches.
async fn verify_sender_by_fingerprint(
context: &Context,
fingerprint: &Fingerprint,
contact_id: ContactId,
@@ -223,13 +222,17 @@ async fn fingerprint_equals_sender(
}
};
if let Some(peerstate) = peerstate {
if let Some(mut peerstate) = peerstate {
if peerstate
.public_key_fingerprint
.as_ref()
.filter(|&fp| fp == fingerprint)
.is_some()
{
let verifier = contact.get_addr().to_owned();
peerstate.set_verified(PeerstateKeyType::PublicKey, fingerprint.clone(), verifier)?;
peerstate.prefer_encrypt = EncryptPreference::Mutual;
peerstate.save_to_db(&context.sql).await?;
return Ok(true);
}
}
@@ -243,6 +246,7 @@ async fn fingerprint_equals_sender(
/// next with this incoming setup-contact/secure-join handshake message.
///
/// [`receive_imf`]: crate::receive_imf::receive_imf
#[derive(Debug, PartialEq, Eq)]
pub(crate) enum HandshakeMessage {
/// The message has been fully handled and should be removed/delete.
///
@@ -286,10 +290,7 @@ pub(crate) async fn handle_securejoin_handshake(
.get_header(HeaderDef::SecureJoin)
.context("Not a Secure-Join message")?;
info!(
context,
">>>>>>>>>>>>>>>>>>>>>>>>> secure-join message \'{}\' received", step,
);
info!(context, "Received secure-join message {step:?}.");
let join_vg = step.starts_with("vg-");
@@ -315,7 +316,6 @@ pub(crate) async fn handle_securejoin_handshake(
warn!(context, "Secure-join denied (bad invitenumber).");
return Ok(HandshakeMessage::Ignore);
}
info!(context, "Secure-join requested.",);
inviter_progress(context, contact_id, 300);
@@ -330,7 +330,6 @@ pub(crate) async fn handle_securejoin_handshake(
context,
contact_id,
&format!("{}-auth-required", &step[..2]),
None,
)
.await
.context("failed sending auth-required handshake message")?;
@@ -375,7 +374,7 @@ pub(crate) async fn handle_securejoin_handshake(
.await?;
return Ok(HandshakeMessage::Ignore);
}
if !fingerprint_equals_sender(context, &fingerprint, contact_id).await? {
if !verify_sender_by_fingerprint(context, &fingerprint, contact_id).await? {
could_not_establish_secure_connection(
context,
contact_id,
@@ -387,20 +386,17 @@ pub(crate) async fn handle_securejoin_handshake(
}
info!(context, "Fingerprint verified.",);
// verify that the `Secure-Join-Auth:`-header matches the secret written to the QR code
let auth_0 = match mime_message.get_header(HeaderDef::SecureJoinAuth) {
Some(auth) => auth,
None => {
could_not_establish_secure_connection(
context,
contact_id,
info_chat_id(context, contact_id).await?,
"Auth not provided.",
)
.await?;
return Ok(HandshakeMessage::Ignore);
}
let Some(auth) = mime_message.get_header(HeaderDef::SecureJoinAuth) else {
could_not_establish_secure_connection(
context,
contact_id,
info_chat_id(context, contact_id).await?,
"Auth not provided.",
)
.await?;
return Ok(HandshakeMessage::Ignore);
};
if !token::exists(context, token::Namespace::Auth, auth_0).await {
if !token::exists(context, token::Namespace::Auth, auth).await {
could_not_establish_secure_connection(
context,
contact_id,
@@ -414,8 +410,14 @@ pub(crate) async fn handle_securejoin_handshake(
.await?
.get_addr()
.to_owned();
let fingerprint_found =
mark_peer_as_verified(context, fingerprint.clone(), contact_addr).await?;
let backward_verified = true;
let fingerprint_found = mark_peer_as_verified(
context,
fingerprint.clone(),
contact_addr,
backward_verified,
)
.await?;
if !fingerprint_found {
could_not_establish_secure_connection(
context,
@@ -444,7 +446,13 @@ pub(crate) async fn handle_securejoin_handshake(
};
match chat::get_chat_id_by_grpid(context, field_grpid).await? {
Some((group_chat_id, _, _)) => {
secure_connection_established(context, contact_id, group_chat_id).await?;
secure_connection_established(
context,
contact_id,
group_chat_id,
mime_message.timestamp_sent,
)
.await?;
chat::add_contact_to_chat_ex(
context,
Nosync,
@@ -464,16 +472,12 @@ pub(crate) async fn handle_securejoin_handshake(
context,
contact_id,
info_chat_id(context, contact_id).await?,
mime_message.timestamp_sent,
)
.await?;
send_alice_handshake_msg(
context,
contact_id,
"vc-contact-confirm",
Some(fingerprint),
)
.await
.context("failed sending vc-contact-confirm message")?;
send_alice_handshake_msg(context, contact_id, "vc-contact-confirm")
.await
.context("failed sending vc-contact-confirm message")?;
inviter_progress(context, contact_id, 1000);
}
@@ -483,11 +487,18 @@ pub(crate) async fn handle_securejoin_handshake(
==== Bob - the joiner's side ====
==== Step 7 in "Setup verified contact" protocol ====
=======================================================*/
"vc-contact-confirm" => match BobState::from_db(&context.sql).await? {
Some(bobstate) => bob::handle_contact_confirm(context, bobstate, mime_message).await,
None => Ok(HandshakeMessage::Ignore),
},
"vc-contact-confirm" => {
if let Some(mut bobstate) = BobState::from_db(&context.sql).await? {
if !bobstate.is_msg_expected(context, step.as_str()) {
warn!(context, "Unexpected vc-contact-confirm.");
return Ok(HandshakeMessage::Ignore);
}
bobstate.step_contact_confirm(context).await?;
bobstate.emit_progress(context, JoinerProgress::Succeeded);
}
Ok(HandshakeMessage::Ignore)
}
"vg-member-added" => {
let Some(member_added) = mime_message
.get_header(HeaderDef::ChatGroupMemberAdded)
@@ -495,32 +506,32 @@ pub(crate) async fn handle_securejoin_handshake(
else {
warn!(
context,
"vg-member-added without Chat-Group-Member-Added header"
"vg-member-added without Chat-Group-Member-Added header."
);
return Ok(HandshakeMessage::Propagate);
};
if !context.is_self_addr(member_added).await? {
info!(
context,
"Member {member_added} added by unrelated SecureJoin process"
"Member {member_added} added by unrelated SecureJoin process."
);
return Ok(HandshakeMessage::Propagate);
}
match BobState::from_db(&context.sql).await? {
Some(bobstate) => {
bob::handle_contact_confirm(context, bobstate, mime_message).await
if let Some(mut bobstate) = BobState::from_db(&context.sql).await? {
if !bobstate.is_msg_expected(context, step.as_str()) {
warn!(context, "Unexpected vg-member-added.");
return Ok(HandshakeMessage::Propagate);
}
None => Ok(HandshakeMessage::Propagate),
bobstate.step_contact_confirm(context).await?;
bobstate.emit_progress(context, JoinerProgress::Succeeded);
}
Ok(HandshakeMessage::Propagate)
}
"vg-member-added-received" | "vc-contact-confirm-received" => {
/*==========================================================
==== Alice - the inviter side ====
==== Step 8 in "Out-of-band verified groups" protocol ====
==========================================================*/
Ok(HandshakeMessage::Done) // "Done" deletes the message
// Deprecated steps, delete them immediately.
Ok(HandshakeMessage::Done)
}
_ => {
warn!(context, "invalid step: {}", step);
@@ -529,23 +540,25 @@ pub(crate) async fn handle_securejoin_handshake(
}
}
/// observe_securejoin_on_other_device() must be called when a self-sent securejoin message is seen.
/// Observe self-sent Securejoin message.
///
/// in a multi-device-setup, there may be other devices that "see" the handshake messages.
/// if the seen messages seen are self-sent messages encrypted+signed correctly with our key,
/// we can make some conclusions of it:
/// In a multi-device-setup, there may be other devices that "see" the handshake messages.
/// If we see self-sent messages encrypted+signed correctly with our key,
/// we can make some conclusions of it.
///
/// - if we see the self-sent-message vg-member-added/vc-contact-confirm,
/// we know that we're an inviter-observer.
/// The inviting device has marked a peer as verified on vg-request-with-auth/vc-request-with-auth
/// before sending vg-member-added/vc-contact-confirm - so, if we observe vg-member-added/vc-contact-confirm,
/// we can mark the peer as verified as well.
/// If we see self-sent {vc,vg}-request-with-auth,
/// we know that we are Bob (joiner-observer)
/// that just marked peer (Alice) as forward-verified
/// either after receiving {vc,vg}-auth-required
/// or immediately after scanning the QR-code
/// if the key was already known.
///
/// - if we see the self-sent-message vg-member-added-received
/// we know that we're an joiner-observer.
/// the joining device has marked the peer as verified on vg-member-added/vc-contact-confirm
/// before sending vg-member-added-received - so, if we observe vg-member-added-received,
/// we can mark the peer as verified as well.
/// If we see self-sent vc-contact-confirm or vg-member-added message,
/// we know that we are Alice (inviter-observer)
/// that just marked peer (Bob) as forward (and backward)-verified
/// in response to correct vc-request-with-auth message.
///
/// In both cases we can mark the peer as forward-verified.
pub(crate) async fn observe_securejoin_on_other_device(
context: &Context,
mime_message: &MimeMessage,
@@ -557,126 +570,98 @@ pub(crate) async fn observe_securejoin_on_other_device(
let step = mime_message
.get_header(HeaderDef::SecureJoin)
.context("Not a Secure-Join message")?;
info!(context, "observing secure-join message \'{}\'", step);
info!(context, "Observing secure-join message {step:?}.");
match step.as_str() {
"vg-request-with-auth"
| "vc-request-with-auth"
| "vg-member-added"
| "vc-contact-confirm"
| "vg-member-added-received"
| "vc-contact-confirm-received" => {
if !encrypted_and_signed(
context,
mime_message,
get_self_fingerprint(context).await.as_ref(),
) {
could_not_establish_secure_connection(
context,
contact_id,
info_chat_id(context, contact_id).await?,
"Message not encrypted correctly.",
)
.await?;
return Ok(HandshakeMessage::Ignore);
}
let addr = Contact::get_by_id(context, contact_id)
.await?
.get_addr()
.to_lowercase();
if mime_message.gossiped_addr.contains(&addr) {
let mut peerstate = match Peerstate::from_addr(context, &addr).await? {
Some(p) => p,
None => {
could_not_establish_secure_connection(
context,
contact_id,
info_chat_id(context, contact_id).await?,
&format!("No peerstate in db for '{}' at step {}", &addr, step),
)
.await?;
return Ok(HandshakeMessage::Ignore);
}
};
let fingerprint = match peerstate.gossip_key_fingerprint.clone() {
Some(fp) => fp,
None => {
could_not_establish_secure_connection(
context,
contact_id,
info_chat_id(context, contact_id).await?,
&format!(
"No gossip key fingerprint in db for '{}' at step {}",
&addr, step,
),
)
.await?;
return Ok(HandshakeMessage::Ignore);
}
};
peerstate.set_verified(PeerstateKeyType::GossipKey, fingerprint, addr)?;
peerstate.prefer_encrypt = EncryptPreference::Mutual;
peerstate.save_to_db(&context.sql).await.unwrap_or_default();
if !matches!(
step.as_str(),
"vg-request-with-auth" | "vc-request-with-auth" | "vg-member-added" | "vc-contact-confirm"
) {
return Ok(HandshakeMessage::Ignore);
};
ChatId::set_protection_for_contact(context, contact_id).await?;
} else if let Some(fingerprint) =
mime_message.get_header(HeaderDef::SecureJoinFingerprint)
{
// FIXME: Old versions of DC send this header instead of gossips. Remove this
// eventually.
let fingerprint = fingerprint.parse()?;
let fingerprint_found = mark_peer_as_verified(
context,
fingerprint,
Contact::get_by_id(context, contact_id)
.await?
.get_addr()
.to_owned(),
)
.await?;
if !fingerprint_found {
could_not_establish_secure_connection(
context,
contact_id,
info_chat_id(context, contact_id).await?,
format!("Fingerprint mismatch on observing {step}.").as_ref(),
)
.await?;
return Ok(HandshakeMessage::Ignore);
}
} else {
could_not_establish_secure_connection(
context,
contact_id,
info_chat_id(context, contact_id).await?,
&format!(
"No gossip header for '{}' at step {}, please update Delta Chat on all \
if !encrypted_and_signed(
context,
mime_message,
get_self_fingerprint(context).await.as_ref(),
) {
could_not_establish_secure_connection(
context,
contact_id,
info_chat_id(context, contact_id).await?,
"Message not encrypted correctly.",
)
.await?;
return Ok(HandshakeMessage::Ignore);
}
let addr = Contact::get_by_id(context, contact_id)
.await?
.get_addr()
.to_lowercase();
if !mime_message.gossiped_addr.contains(&addr) {
could_not_establish_secure_connection(
context,
contact_id,
info_chat_id(context, contact_id).await?,
&format!(
"No gossip header for '{}' at step {}, please update Delta Chat on all \
your devices.",
&addr, step,
),
)
.await?;
return Ok(HandshakeMessage::Ignore);
}
if step.as_str() == "vg-member-added" {
inviter_progress(context, contact_id, 800);
}
if step.as_str() == "vg-member-added" || step.as_str() == "vc-contact-confirm" {
inviter_progress(context, contact_id, 1000);
}
if step.as_str() == "vg-request-with-auth" || step.as_str() == "vc-request-with-auth" {
// This actually reflects what happens on the first device (which does the secure
// join) and causes a subsequent "vg-member-added" message to create an unblocked
// verified group.
ChatId::create_for_contact_with_blocked(context, contact_id, Blocked::Not).await?;
}
Ok(if step.as_str() == "vg-member-added" {
HandshakeMessage::Propagate
} else {
HandshakeMessage::Ignore
})
}
_ => Ok(HandshakeMessage::Ignore),
&addr, step,
),
)
.await?;
return Ok(HandshakeMessage::Ignore);
}
let Some(mut peerstate) = Peerstate::from_addr(context, &addr).await? else {
could_not_establish_secure_connection(
context,
contact_id,
info_chat_id(context, contact_id).await?,
&format!("No peerstate in db for '{}' at step {}", &addr, step),
)
.await?;
return Ok(HandshakeMessage::Ignore);
};
let Some(fingerprint) = peerstate.gossip_key_fingerprint.clone() else {
could_not_establish_secure_connection(
context,
contact_id,
info_chat_id(context, contact_id).await?,
&format!(
"No gossip key fingerprint in db for '{}' at step {}",
&addr, step,
),
)
.await?;
return Ok(HandshakeMessage::Ignore);
};
peerstate.set_verified(PeerstateKeyType::GossipKey, fingerprint, addr)?;
peerstate.prefer_encrypt = EncryptPreference::Mutual;
peerstate.save_to_db(&context.sql).await?;
ChatId::set_protection_for_contact(context, contact_id, mime_message.timestamp_sent).await?;
if step.as_str() == "vg-member-added" {
inviter_progress(context, contact_id, 800);
}
if step.as_str() == "vg-member-added" || step.as_str() == "vc-contact-confirm" {
inviter_progress(context, contact_id, 1000);
}
if step.as_str() == "vg-request-with-auth" || step.as_str() == "vc-request-with-auth" {
// This actually reflects what happens on the first device (which does the secure
// join) and causes a subsequent "vg-member-added" message to create an unblocked
// verified group.
ChatId::create_for_contact_with_blocked(context, contact_id, Blocked::Not).await?;
}
if step.as_str() == "vg-member-added" {
Ok(HandshakeMessage::Propagate)
} else {
Ok(HandshakeMessage::Ignore)
}
}
@@ -684,23 +669,19 @@ async fn secure_connection_established(
context: &Context,
contact_id: ContactId,
chat_id: ChatId,
timestamp: i64,
) -> Result<()> {
if context
.get_config_bool(Config::VerifiedOneOnOneChats)
let private_chat_id = ChatIdBlocked::get_for_contact(context, contact_id, Blocked::Yes)
.await?
{
let private_chat_id = ChatIdBlocked::get_for_contact(context, contact_id, Blocked::Yes)
.await?
.id;
private_chat_id
.set_protection(
context,
ProtectionStatus::Protected,
time(),
Some(contact_id),
)
.await?;
}
.id;
private_chat_id
.set_protection(
context,
ProtectionStatus::Protected,
timestamp,
Some(contact_id),
)
.await?;
context.emit_event(EventType::ChatModified(chat_id));
Ok(())
}
@@ -728,13 +709,18 @@ async fn mark_peer_as_verified(
context: &Context,
fingerprint: Fingerprint,
verifier: String,
backward_verified: bool,
) -> Result<bool> {
let Some(ref mut peerstate) = Peerstate::from_fingerprint(context, &fingerprint).await? else {
return Ok(false);
};
peerstate.set_verified(PeerstateKeyType::PublicKey, fingerprint, verifier)?;
peerstate.prefer_encrypt = EncryptPreference::Mutual;
peerstate.save_to_db(&context.sql).await.unwrap_or_default();
if backward_verified {
peerstate.backward_verified_key_id =
Some(context.get_config_i64(Config::KeyId).await?).filter(|&id| id > 0);
}
peerstate.save_to_db(&context.sql).await?;
Ok(true)
}
@@ -774,7 +760,6 @@ mod tests {
use crate::chatlist::Chatlist;
use crate::constants::Chattype;
use crate::contact::ContactAddress;
use crate::contact::VerifiedStatus;
use crate::peerstate::Peerstate;
use crate::receive_imf::receive_imf;
use crate::stock_str::chat_protection_enabled;
@@ -893,17 +878,11 @@ mod tests {
let contact_bob = Contact::get_by_id(&alice.ctx, contact_bob_id)
.await
.unwrap();
assert_eq!(
contact_bob.is_verified(&alice.ctx).await.unwrap(),
VerifiedStatus::Unverified
);
assert_eq!(contact_bob.is_verified(&alice.ctx).await.unwrap(), false);
// Step 5+6: Alice receives vc-request-with-auth, sends vc-contact-confirm
alice.recv_msg(&sent).await;
assert_eq!(
contact_bob.is_verified(&alice.ctx).await.unwrap(),
VerifiedStatus::BidirectVerified
);
assert_eq!(contact_bob.is_verified(&alice.ctx).await.unwrap(), true);
// exactly one one-to-one chat should be visible for both now
// (check this before calling alice.create_chat() explicitly below)
@@ -946,35 +925,18 @@ mod tests {
let contact_alice = Contact::get_by_id(&bob.ctx, contact_alice_id)
.await
.unwrap();
assert_eq!(
contact_bob.is_verified(&bob.ctx).await.unwrap(),
VerifiedStatus::Unverified
);
assert_eq!(contact_bob.is_verified(&bob.ctx).await.unwrap(), false);
// Step 7: Bob receives vc-contact-confirm, sends vc-contact-confirm-received
// Step 7: Bob receives vc-contact-confirm
bob.recv_msg(&sent).await;
assert_eq!(
contact_alice.is_verified(&bob.ctx).await.unwrap(),
VerifiedStatus::BidirectVerified
);
assert_eq!(contact_alice.is_verified(&bob.ctx).await.unwrap(), true);
// Check Bob got the verified message in his 1:1 chat.
{
let chat = bob.create_chat(&alice).await;
let msg = get_chat_msg(&bob, chat.get_id(), 0, 1).await;
assert!(msg.is_info());
let expected_text = chat_protection_enabled(&bob).await;
assert_eq!(msg.get_text(), expected_text);
}
// Check Bob sent the final message
let sent = bob.pop_sent_msg().await;
let msg = alice.parse_msg(&sent).await;
assert!(msg.was_encrypted());
assert_eq!(
msg.get_header(HeaderDef::SecureJoin).unwrap(),
"vc-contact-confirm-received"
);
let chat = bob.create_chat(&alice).await;
let msg = get_chat_msg(&bob, chat.get_id(), 0, 1).await;
assert!(msg.is_info());
let expected_text = chat_protection_enabled(&bob).await;
assert_eq!(msg.get_text(), expected_text);
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
@@ -1008,6 +970,7 @@ mod tests {
secondary_verified_key: None,
secondary_verified_key_fingerprint: None,
secondary_verifier: None,
backward_verified_key_id: None,
fingerprint_changed: false,
};
peerstate.save_to_db(&bob.ctx.sql).await?;
@@ -1063,17 +1026,11 @@ mod tests {
)
.await?;
let contact_bob = Contact::get_by_id(&alice.ctx, contact_bob_id).await?;
assert_eq!(
contact_bob.is_verified(&alice.ctx).await?,
VerifiedStatus::Unverified
);
assert_eq!(contact_bob.is_verified(&alice.ctx).await?, false);
// Step 5+6: Alice receives vc-request-with-auth, sends vc-contact-confirm
alice.recv_msg(&sent).await;
assert_eq!(
contact_bob.is_verified(&alice.ctx).await?,
VerifiedStatus::BidirectVerified
);
assert_eq!(contact_bob.is_verified(&alice.ctx).await?, true);
let sent = alice.pop_sent_msg().await;
let msg = bob.parse_msg(&sent).await;
@@ -1090,25 +1047,12 @@ mod tests {
.expect("Error looking up contact")
.expect("Contact not found");
let contact_alice = Contact::get_by_id(&bob.ctx, contact_alice_id).await?;
assert_eq!(
contact_bob.is_verified(&bob.ctx).await?,
VerifiedStatus::Unverified
);
assert_eq!(contact_bob.is_verified(&bob.ctx).await?, false);
// Step 7: Bob receives vc-contact-confirm, sends vc-contact-confirm-received
// Step 7: Bob receives vc-contact-confirm
bob.recv_msg(&sent).await;
assert_eq!(
contact_alice.is_verified(&bob.ctx).await?,
VerifiedStatus::BidirectVerified
);
assert_eq!(contact_alice.is_verified(&bob.ctx).await?, true);
let sent = bob.pop_sent_msg().await;
let msg = alice.parse_msg(&sent).await;
assert!(msg.was_encrypted());
assert_eq!(
msg.get_header(HeaderDef::SecureJoin).unwrap(),
"vc-contact-confirm-received"
);
Ok(())
}
@@ -1231,17 +1175,11 @@ mod tests {
.await?
.expect("Contact not found");
let contact_bob = Contact::get_by_id(&alice.ctx, contact_bob_id).await?;
assert_eq!(
contact_bob.is_verified(&alice.ctx).await?,
VerifiedStatus::Unverified
);
assert_eq!(contact_bob.is_verified(&alice.ctx).await?, false);
// Step 5+6: Alice receives vg-request-with-auth, sends vg-member-added
alice.recv_msg(&sent).await;
assert_eq!(
contact_bob.is_verified(&alice.ctx).await?,
VerifiedStatus::BidirectVerified
);
assert_eq!(contact_bob.is_verified(&alice.ctx).await?, true);
let sent = alice.pop_sent_msg().await;
let msg = bob.parse_msg(&sent).await;
@@ -1277,19 +1215,13 @@ mod tests {
.expect("Error looking up contact")
.expect("Contact not found");
let contact_alice = Contact::get_by_id(&bob.ctx, contact_alice_id).await?;
assert_eq!(
contact_bob.is_verified(&bob.ctx).await?,
VerifiedStatus::Unverified
);
assert_eq!(contact_bob.is_verified(&bob.ctx).await?, false);
// Step 7: Bob receives vg-member-added, sends vg-member-added-received
// Step 7: Bob receives vg-member-added
bob.recv_msg(&sent).await;
{
// Bob has Alice verified, message shows up in the group chat.
assert_eq!(
contact_alice.is_verified(&bob.ctx).await?,
VerifiedStatus::BidirectVerified
);
assert_eq!(contact_alice.is_verified(&bob.ctx).await?, true);
let chat = bob.get_chat(&alice).await;
assert_eq!(
chat.blocked,
@@ -1305,14 +1237,6 @@ mod tests {
}
}
let sent = bob.pop_sent_msg().await;
let msg = alice.parse_msg(&sent).await;
assert!(msg.was_encrypted());
assert_eq!(
msg.get_header(HeaderDef::SecureJoin).unwrap(),
"vg-member-added-received"
);
let bob_chat = Chat::load_from_db(&bob.ctx, bob_chatid).await?;
assert!(bob_chat.is_protected());
assert!(bob_chat.typ == Chattype::Group);
@@ -1380,4 +1304,70 @@ First thread."#;
Ok(())
}
/// Tests that Bob gets Alice as verified
/// if `vc-contact-confirm` is lost but Alice then sends
/// a message to Bob in a verified 1:1 chat with a `Chat-Verified` header.
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_lost_contact_confirm() {
let mut tcm = TestContextManager::new();
let alice = tcm.alice().await;
let bob = tcm.bob().await;
alice
.set_config(Config::VerifiedOneOnOneChats, Some("1"))
.await
.unwrap();
bob.set_config(Config::VerifiedOneOnOneChats, Some("1"))
.await
.unwrap();
let qr = get_securejoin_qr(&alice.ctx, None).await.unwrap();
join_securejoin(&bob.ctx, &qr).await.unwrap();
// vc-request
let sent = bob.pop_sent_msg().await;
alice.recv_msg(&sent).await;
// vc-auth-required
let sent = alice.pop_sent_msg().await;
bob.recv_msg(&sent).await;
// vc-request-with-auth
let sent = bob.pop_sent_msg().await;
alice.recv_msg(&sent).await;
// Alice has Bob verified now.
let contact_bob_id =
Contact::lookup_id_by_addr(&alice.ctx, "bob@example.net", Origin::Unknown)
.await
.expect("Error looking up contact")
.expect("Contact not found");
let contact_bob = Contact::get_by_id(&alice.ctx, contact_bob_id)
.await
.unwrap();
assert_eq!(contact_bob.is_verified(&alice.ctx).await.unwrap(), true);
// Alice sends vc-contact-confirm, but it gets lost.
let _sent_vc_contact_confirm = alice.pop_sent_msg().await;
// Bob should not yet have Alice verified
let contact_alice_id =
Contact::lookup_id_by_addr(&bob, "alice@example.org", Origin::Unknown)
.await
.expect("Error looking up contact")
.expect("Contact not found");
let contact_alice = Contact::get_by_id(&bob, contact_alice_id).await.unwrap();
assert_eq!(contact_alice.is_verified(&bob).await.unwrap(), false);
// Alice sends a text message to Bob.
let received_hello = tcm.send_recv(&alice, &bob, "Hello!").await;
let chat_id = received_hello.chat_id;
let chat = Chat::load_from_db(&bob, chat_id).await.unwrap();
assert_eq!(chat.is_protected(), true);
// Received text message in a verified 1:1 chat results in backward verification
// and Bob now marks alice as verified.
let contact_alice = Contact::get_by_id(&bob, contact_alice_id).await.unwrap();
assert_eq!(contact_alice.is_verified(&bob).await.unwrap(), true);
}
}

View File

@@ -9,14 +9,13 @@ use super::bobstate::{BobHandshakeStage, BobState};
use super::qrinvite::QrInvite;
use super::HandshakeMessage;
use crate::chat::{is_contact_in_chat, ChatId, ProtectionStatus};
use crate::config::Config;
use crate::constants::{Blocked, Chattype};
use crate::contact::Contact;
use crate::context::Context;
use crate::events::EventType;
use crate::mimeparser::MimeMessage;
use crate::sync::Sync::*;
use crate::tools::time;
use crate::tools::{create_smeared_timestamp, time};
use crate::{chat, stock_str};
/// Starts the securejoin protocol with the QR `invite`.
@@ -58,7 +57,6 @@ pub(super) async fn start_protocol(context: &Context, invite: QrInvite) -> Resul
QrInvite::Group { .. } => {
// For a secure-join we need to create the group and add the contact. The group will
// only become usable once the protocol is finished.
// TODO: how does this group become usable?
let group_chat_id = state.joining_chat_id(context).await?;
if !is_contact_in_chat(context, group_chat_id, invite.contact_id()).await? {
chat::add_to_chat_contacts_table(context, group_chat_id, &[invite.contact_id()])
@@ -84,65 +82,31 @@ pub(super) async fn handle_auth_required(
context: &Context,
message: &MimeMessage,
) -> Result<HandshakeMessage> {
match BobState::from_db(&context.sql).await? {
Some(mut bobstate) => match bobstate.handle_message(context, message).await? {
Some(BobHandshakeStage::Terminated(why)) => {
bobstate.notify_aborted(context, why).await?;
Ok(HandshakeMessage::Done)
}
Some(_stage) => {
if bobstate.is_join_group() {
// The message reads "Alice replied, waiting to be added to the group…",
// so only show it on secure-join and not on setup-contact.
let contact_id = bobstate.invite().contact_id();
let msg = stock_str::secure_join_replies(context, contact_id).await;
let chat_id = bobstate.joining_chat_id(context).await?;
chat::add_info_msg(context, chat_id, &msg, time()).await?;
}
bobstate.emit_progress(context, JoinerProgress::RequestWithAuthSent);
Ok(HandshakeMessage::Done)
}
None => Ok(HandshakeMessage::Ignore),
},
None => Ok(HandshakeMessage::Ignore),
}
}
/// Handles `vc-contact-confirm` and `vg-member-added` handshake messages.
///
/// # Bob - the joiner's side
/// ## Step 7 in the "Setup Contact protocol"
pub(super) async fn handle_contact_confirm(
context: &Context,
mut bobstate: BobState,
message: &MimeMessage,
) -> Result<HandshakeMessage> {
let retval = if bobstate.is_join_group() {
HandshakeMessage::Propagate
} else {
HandshakeMessage::Ignore
let Some(mut bobstate) = BobState::from_db(&context.sql).await? else {
return Ok(HandshakeMessage::Ignore);
};
match bobstate.handle_message(context, message).await? {
match bobstate.handle_auth_required(context, message).await? {
Some(BobHandshakeStage::Terminated(why)) => {
bobstate.notify_aborted(context, why).await?;
Ok(HandshakeMessage::Done)
}
Some(BobHandshakeStage::Completed) => {
// Note this goes to the 1:1 chat, as when joining a group we implicitly also
// verify both contacts (this could be a bug/security issue, see
// e.g. https://github.com/deltachat/deltachat-core-rust/issues/1177).
bobstate.notify_peer_verified(context).await?;
bobstate.emit_progress(context, JoinerProgress::Succeeded);
Ok(retval)
Some(_stage) => {
if bobstate.is_join_group() {
// The message reads "Alice replied, waiting to be added to the group…",
// so only show it on secure-join and not on setup-contact.
let contact_id = bobstate.invite().contact_id();
let msg = stock_str::secure_join_replies(context, contact_id).await;
let chat_id = bobstate.joining_chat_id(context).await?;
chat::add_info_msg(context, chat_id, &msg, time()).await?;
}
bobstate
.set_peer_verified(context, message.timestamp_sent)
.await?;
bobstate.emit_progress(context, JoinerProgress::RequestWithAuthSent);
Ok(HandshakeMessage::Done)
}
Some(_) => {
warn!(
context,
"Impossible state returned from handling handshake message"
);
Ok(retval)
}
None => Ok(retval),
None => Ok(HandshakeMessage::Ignore),
}
}
@@ -155,7 +119,7 @@ impl BobState {
}
}
fn emit_progress(&self, context: &Context, progress: JoinerProgress) {
pub(crate) fn emit_progress(&self, context: &Context, progress: JoinerProgress) {
let contact_id = self.invite().contact_id();
context.emit_event(EventType::SecurejoinJoinerProgress {
contact_id,
@@ -193,6 +157,7 @@ impl BobState {
Blocked::Not,
ProtectionStatus::Unprotected, // protection is added later as needed
None,
create_smeared_timestamp(context),
)
.await?
}
@@ -217,28 +182,17 @@ impl BobState {
Ok(())
}
/// Notifies the user that the SecureJoin peer is verified.
///
/// This creates an info message in the chat being joined.
async fn notify_peer_verified(&self, context: &Context) -> Result<()> {
/// Turns 1:1 chat with SecureJoin peer into protected chat.
pub(crate) async fn set_peer_verified(&self, context: &Context, timestamp: i64) -> Result<()> {
let contact = Contact::get_by_id(context, self.invite().contact_id()).await?;
let chat_id = self.joining_chat_id(context).await?;
if context
.get_config_bool(Config::VerifiedOneOnOneChats)
.await?
{
self.alice_chat()
.set_protection(
context,
ProtectionStatus::Protected,
time(),
Some(contact.id),
)
.await?;
}
context.emit_event(EventType::ChatModified(chat_id));
self.alice_chat()
.set_protection(
context,
ProtectionStatus::Protected,
timestamp,
Some(contact.id),
)
.await?;
Ok(())
}
}
@@ -247,7 +201,7 @@ impl BobState {
///
/// This has an `From<JoinerProgress> for usize` impl yielding numbers between 0 and a 1000
/// which can be shown as a progress bar.
enum JoinerProgress {
pub(crate) enum JoinerProgress {
/// An error occurred.
Error,
/// vg-vc-request-with-auth sent.

View File

@@ -11,8 +11,9 @@ use anyhow::Result;
use rusqlite::Connection;
use super::qrinvite::QrInvite;
use super::{encrypted_and_signed, fingerprint_equals_sender, mark_peer_as_verified};
use super::{encrypted_and_signed, verify_sender_by_fingerprint};
use crate::chat::{self, ChatId};
use crate::config::Config;
use crate::contact::{Contact, Origin};
use crate::context::Context;
use crate::events::EventType;
@@ -21,7 +22,9 @@ use crate::key::{load_self_public_key, DcKey};
use crate::message::{Message, Viewtype};
use crate::mimeparser::{MimeMessage, SystemMessage};
use crate::param::Param;
use crate::securejoin::Peerstate;
use crate::sql::Sql;
use crate::tools::time;
/// The stage of the [`BobState`] securejoin handshake protocol state machine.
///
@@ -30,14 +33,9 @@ use crate::sql::Sql;
#[derive(Clone, Copy, Debug, Display)]
pub enum BobHandshakeStage {
/// Step 2 completed: (vc|vg)-request message sent.
///
/// Note that this is only ever returned by [`BobState::start_protocol`] and never by
/// [`BobState::handle_message`].
RequestSent,
/// Step 4 completed: (vc|vg)-request-with-auth message sent.
RequestWithAuthSent,
/// The protocol completed successfully.
Completed,
/// The protocol prematurely terminated with given reason.
Terminated(&'static str),
}
@@ -92,21 +90,26 @@ impl BobState {
invite: QrInvite,
chat_id: ChatId,
) -> Result<(Self, BobHandshakeStage, Vec<Self>)> {
let (stage, next) =
if fingerprint_equals_sender(context, invite.fingerprint(), invite.contact_id()).await?
{
// The scanned fingerprint matches Alice's key, we can proceed to step 4b.
info!(context, "Taking securejoin protocol shortcut");
send_handshake_message(context, &invite, chat_id, BobHandshakeMsg::RequestWithAuth)
.await?;
(
BobHandshakeStage::RequestWithAuthSent,
SecureJoinStep::ContactConfirm,
)
} else {
send_handshake_message(context, &invite, chat_id, BobHandshakeMsg::Request).await?;
(BobHandshakeStage::RequestSent, SecureJoinStep::AuthRequired)
};
let peer_verified =
verify_sender_by_fingerprint(context, invite.fingerprint(), invite.contact_id())
.await?;
let (stage, next);
if peer_verified {
// The scanned fingerprint matches Alice's key, we can proceed to step 4b.
info!(context, "Taking securejoin protocol shortcut");
send_handshake_message(context, &invite, chat_id, BobHandshakeMsg::RequestWithAuth)
.await?;
stage = BobHandshakeStage::RequestWithAuthSent;
next = SecureJoinStep::ContactConfirm;
} else {
send_handshake_message(context, &invite, chat_id, BobHandshakeMsg::Request).await?;
stage = BobHandshakeStage::RequestSent;
next = SecureJoinStep::AuthRequired;
};
let (id, aborted_states) =
Self::insert_new_db_entry(context, next, invite.clone(), chat_id).await?;
let state = Self {
@@ -115,6 +118,12 @@ impl BobState {
next,
chat_id,
};
if peer_verified {
// Mark 1:1 chat as verified already.
state.set_peer_verified(context, time()).await?;
}
Ok((state, stage, aborted_states))
}
@@ -230,13 +239,13 @@ impl BobState {
Ok(())
}
/// Handles the given message for the securejoin handshake for Bob.
/// Handles {vc,vg}-auth-required message of the securejoin handshake for Bob.
///
/// If the message was not used for this handshake `None` is returned, otherwise the new
/// stage is returned. Once [`BobHandshakeStage::Completed`] or
/// [`BobHandshakeStage::Terminated`] are reached this [`BobState`] should be destroyed,
/// stage is returned. Once [`BobHandshakeStage::Terminated`] is reached this
/// [`BobState`] should be destroyed,
/// further calling it will just result in the messages being unused by this handshake.
pub(crate) async fn handle_message(
pub(crate) async fn handle_auth_required(
&mut self,
context: &Context,
mime_message: &MimeMessage,
@@ -256,42 +265,10 @@ impl BobState {
info!(context, "{} message out of sync for BobState", step);
return Ok(None);
}
match step.as_str() {
"vg-auth-required" | "vc-auth-required" => {
self.step_auth_required(context, mime_message).await
}
"vg-member-added" | "vc-contact-confirm" => {
self.step_contact_confirm(context, mime_message).await
}
_ => {
warn!(context, "Invalid step for BobState: {}", step);
Ok(None)
}
}
}
/// Returns `true` if the message is expected according to the protocol.
fn is_msg_expected(&self, context: &Context, step: &str) -> bool {
let variant_matches = match self.invite {
QrInvite::Contact { .. } => step.starts_with("vc-"),
QrInvite::Group { .. } => step.starts_with("vg-"),
};
let step_matches = self.next.matches(context, step);
variant_matches && step_matches
}
/// Handles a *vc-auth-required* or *vg-auth-required* message.
///
/// # Bob - the joiner's side
/// ## Step 4 in the "Setup Contact protocol", section 2.1 of countermitm 0.10.0
async fn step_auth_required(
&mut self,
context: &Context,
mime_message: &MimeMessage,
) -> Result<Option<BobHandshakeStage>> {
info!(
context,
"Bob Step 4 - handling vc-auth-require/vg-auth-required message"
"Bob Step 4 - handling {{vc,vg}}-auth-required message."
);
if !encrypted_and_signed(context, mime_message, Some(self.invite.fingerprint())) {
let reason = if mime_message.was_encrypted() {
@@ -303,14 +280,19 @@ impl BobState {
.await?;
return Ok(Some(BobHandshakeStage::Terminated(reason)));
}
if !fingerprint_equals_sender(context, self.invite.fingerprint(), self.invite.contact_id())
.await?
if !verify_sender_by_fingerprint(
context,
self.invite.fingerprint(),
self.invite.contact_id(),
)
.await?
{
self.update_next(&context.sql, SecureJoinStep::Terminated)
.await?;
return Ok(Some(BobHandshakeStage::Terminated("Fingerprint mismatch")));
}
info!(context, "Fingerprint verified.",);
self.update_next(&context.sql, SecureJoinStep::ContactConfirm)
.await?;
self.send_handshake_message(context, BobHandshakeMsg::RequestWithAuth)
@@ -318,47 +300,39 @@ impl BobState {
Ok(Some(BobHandshakeStage::RequestWithAuthSent))
}
/// Returns `true` if the message is expected according to the protocol.
pub(crate) fn is_msg_expected(&self, context: &Context, step: &str) -> bool {
let variant_matches = match self.invite {
QrInvite::Contact { .. } => step.starts_with("vc-"),
QrInvite::Group { .. } => step.starts_with("vg-"),
};
let step_matches = self.next.matches(context, step);
variant_matches && step_matches
}
/// Handles a *vc-contact-confirm* or *vg-member-added* message.
///
/// # Bob - the joiner's side
/// ## Step 7 in the "Setup Contact protocol", section 2.1 of countermitm 0.10.0
///
/// This deviates from the protocol by also sending a confirmation message in response
/// to the *vc-contact-confirm* message. This has no specific value to the protocol and
/// is only done out of symmetry with *vg-member-added* handling.
async fn step_contact_confirm(
&mut self,
context: &Context,
mime_message: &MimeMessage,
) -> Result<Option<BobHandshakeStage>> {
info!(
context,
"Bob Step 7 - handling vc-contact-confirm/vg-member-added message"
);
mark_peer_as_verified(
context,
self.invite.fingerprint().clone(),
mime_message.from.addr.to_string(),
)
.await?;
pub(crate) async fn step_contact_confirm(&mut self, context: &Context) -> Result<()> {
let fingerprint = self.invite.fingerprint();
let Some(ref mut peerstate) = Peerstate::from_fingerprint(context, fingerprint).await?
else {
return Ok(());
};
// Mark peer as backward verified.
peerstate.backward_verified_key_id =
Some(context.get_config_i64(Config::KeyId).await?).filter(|&id| id > 0);
peerstate.save_to_db(&context.sql).await?;
Contact::scaleup_origin_by_id(context, self.invite.contact_id(), Origin::SecurejoinJoined)
.await?;
context.emit_event(EventType::ContactsChanged(None));
self.send_handshake_message(context, BobHandshakeMsg::ContactConfirmReceived)
.await
.map_err(|_| {
warn!(
context,
"Failed to send vc-contact-confirm-received/vg-member-added-received"
);
})
// This is not an error affecting the protocol outcome.
.ok();
self.update_next(&context.sql, SecureJoinStep::Completed)
.await?;
Ok(Some(BobHandshakeStage::Completed))
Ok(())
}
/// Sends the requested handshake message to Alice.
@@ -400,16 +374,13 @@ async fn send_handshake_message(
// Sends the Secure-Join-Auth header in mimefactory.rs.
msg.param.set(Param::Arg2, invite.authcode());
msg.param.set_int(Param::GuaranteeE2ee, 1);
}
BobHandshakeMsg::ContactConfirmReceived => {
msg.param.set_int(Param::GuaranteeE2ee, 1);
// Sends our own fingerprint in the Secure-Join-Fingerprint header.
let bob_fp = load_self_public_key(context).await?.fingerprint();
msg.param.set(Param::Arg3, bob_fp.hex());
}
};
// Sends our own fingerprint in the Secure-Join-Fingerprint header.
let bob_fp = load_self_public_key(context).await?.fingerprint();
msg.param.set(Param::Arg3, bob_fp.hex());
// Sends the grpid in the Secure-Join-Group header.
if let QrInvite::Group { ref grpid, .. } = invite {
msg.param.set(Param::Arg4, grpid);
@@ -425,8 +396,6 @@ enum BobHandshakeMsg {
Request,
/// vc-request-with-auth or vg-request-with-auth
RequestWithAuth,
/// vc-contact-confirm-received or vg-member-added-received
ContactConfirmReceived,
}
impl BobHandshakeMsg {
@@ -454,10 +423,6 @@ impl BobHandshakeMsg {
QrInvite::Contact { .. } => "vc-request-with-auth",
QrInvite::Group { .. } => "vg-request-with-auth",
},
Self::ContactConfirmReceived => match invite {
QrInvite::Contact { .. } => "vc-contact-confirm-received",
QrInvite::Group { .. } => "vg-member-added-received",
},
}
}
}

View File

@@ -10,6 +10,7 @@ use async_smtp::{self as smtp, EmailAddress, SmtpTransport};
use tokio::io::BufStream;
use tokio::task;
use crate::chat::{add_info_msg_with_cmd, ChatId};
use crate::config::Config;
use crate::contact::{Contact, ContactId};
use crate::context::Context;
@@ -26,9 +27,10 @@ use crate::provider::Socket;
use crate::scheduler::connectivity::ConnectivityStore;
use crate::socks::Socks5Config;
use crate::sql;
use crate::stock_str::unencrypted_email;
/// SMTP write and read timeout.
const SMTP_TIMEOUT: Duration = Duration::from_secs(30);
/// SMTP connection, write and read timeout.
const SMTP_TIMEOUT: Duration = Duration::from_secs(60);
#[derive(Default)]
pub(crate) struct Smtp {
@@ -584,7 +586,46 @@ pub(crate) async fn send_msg_to_smtp(
match status {
SendResult::Retry => {}
SendResult::Success | SendResult::Failure(_) => {
SendResult::Success => {
context
.sql
.execute("DELETE FROM smtp WHERE id=?", (rowid,))
.await?;
}
SendResult::Failure(ref err) => {
if err.to_string().contains("Invalid unencrypted mail") {
let res = context
.sql
.query_row_optional(
"SELECT chat_id, timestamp FROM msgs WHERE id=?;",
(msg_id,),
|row| Ok((row.get::<_, ChatId>(0)?, row.get::<_, i64>(1)?)),
)
.await?;
if let Some((chat_id, timestamp_sort)) = res {
let addr = context.get_config(Config::ConfiguredAddr).await?;
let text = unencrypted_email(
context,
addr.unwrap_or_default()
.split('@')
.nth(1)
.unwrap_or_default(),
)
.await;
add_info_msg_with_cmd(
context,
chat_id,
&text,
crate::mimeparser::SystemMessage::InvalidUnencryptedMail,
timestamp_sort,
None,
None,
None,
)
.await?;
};
}
context
.sql
.execute("DELETE FROM smtp WHERE id=?", (rowid,))
@@ -595,7 +636,13 @@ pub(crate) async fn send_msg_to_smtp(
match status {
SendResult::Retry => Err(format_err!("Retry")),
SendResult::Success => {
msg_id.set_delivered(context).await?;
if !context
.sql
.exists("SELECT COUNT(*) FROM smtp WHERE msg_id=?", (msg_id,))
.await?
{
msg_id.set_delivered(context).await?;
}
Ok(())
}
SendResult::Failure(err) => Err(format_err!("{}", err)),
@@ -612,7 +659,7 @@ async fn send_mdns(context: &Context, connection: &mut Smtp) -> Result<()> {
let more_mdns = send_mdn(context, connection).await?;
if !more_mdns {
// No more MDNs to send.
// No more MDNs to send or one of them failed.
return Ok(());
}
}
@@ -746,7 +793,7 @@ async fn send_mdn_msg_id(
}
}
/// Tries to send a single MDN. Returns false if there are no MDNs to send.
/// Tries to send a single MDN. Returns true if more MDNs should be sent.
async fn send_mdn(context: &Context, smtp: &mut Smtp) -> Result<bool> {
let mdns_enabled = context.get_config_bool(Config::MdnsEnabled).await?;
if !mdns_enabled {

View File

@@ -9,11 +9,6 @@ use crate::events::EventType;
pub type Result<T> = std::result::Result<T, Error>;
// if more recipients are needed in SMTP's `RCPT TO:` header, recipient-list is split to chunks.
// this does not affect MIME'e `To:` header.
// can be overwritten by the setting `max_smtp_rcpt_to` in provider-db.
pub(crate) const DEFAULT_MAX_SMTP_RCPT_TO: usize = 50;
#[derive(Debug, thiserror::Error)]
pub enum Error {
#[error("Envelope error: {}", _0)]
@@ -43,40 +38,30 @@ impl Smtp {
}
let message_len_bytes = message.len();
let recipients_display = recipients
.iter()
.map(|x| x.as_ref())
.collect::<Vec<&str>>()
.join(",");
let chunk_size = context
.get_configured_provider()
.await?
.and_then(|provider| provider.opt.max_smtp_rcpt_to)
.map_or(DEFAULT_MAX_SMTP_RCPT_TO, usize::from);
let envelope =
Envelope::new(self.from.clone(), recipients.to_vec()).map_err(Error::Envelope)?;
let mail = SendableEmail::new(envelope, message);
for recipients_chunk in recipients.chunks(chunk_size) {
let recipients_display = recipients_chunk
.iter()
.map(|x| x.as_ref())
.collect::<Vec<&str>>()
.join(",");
if let Some(ref mut transport) = self.transport {
transport.send(mail).await.map_err(Error::SmtpSend)?;
let envelope = Envelope::new(self.from.clone(), recipients_chunk.to_vec())
.map_err(Error::Envelope)?;
let mail = SendableEmail::new(envelope, message);
if let Some(ref mut transport) = self.transport {
transport.send(mail).await.map_err(Error::SmtpSend)?;
let info_msg = format!(
"Message len={message_len_bytes} was SMTP-sent to {recipients_display}"
);
info!(context, "{info_msg}.");
context.emit_event(EventType::SmtpMessageSent(info_msg));
self.last_success = Some(std::time::SystemTime::now());
} else {
warn!(
context,
"uh? SMTP has no transport, failed to send to {}", recipients_display
);
return Err(Error::NoTransport);
}
let info_msg =
format!("Message len={message_len_bytes} was SMTP-sent to {recipients_display}");
info!(context, "{info_msg}.");
context.emit_event(EventType::SmtpMessageSent(info_msg));
self.last_success = Some(std::time::SystemTime::now());
} else {
warn!(
context,
"uh? SMTP has no transport, failed to send to {}", recipients_display
);
return Err(Error::NoTransport);
}
Ok(())
}

View File

@@ -572,22 +572,13 @@ impl Sql {
pub async fn set_raw_config(&self, key: &str, value: Option<&str>) -> Result<()> {
let mut lock = self.config_cache.write().await;
if let Some(value) = value {
let exists = self
.exists("SELECT COUNT(*) FROM config WHERE keyname=?;", (key,))
.await?;
if exists {
self.execute("UPDATE config SET value=? WHERE keyname=?;", (value, key))
.await?;
} else {
self.execute(
"INSERT INTO config (keyname, value) VALUES (?, ?);",
(key, value),
)
.await?;
}
self.execute(
"INSERT OR REPLACE INTO config (keyname, value) VALUES (?, ?)",
(key, value),
)
.await?;
} else {
self.execute("DELETE FROM config WHERE keyname=?;", (key,))
self.execute("DELETE FROM config WHERE keyname=?", (key,))
.await?;
}
lock.insert(key.to_string(), value.map(|s| s.to_string()));
@@ -608,7 +599,7 @@ impl Sql {
let mut lock = self.config_cache.write().await;
let value = self
.query_get_value("SELECT value FROM config WHERE keyname=?;", (key,))
.query_get_value("SELECT value FROM config WHERE keyname=?", (key,))
.await
.context(format!("failed to fetch raw config: {key}"))?;
lock.insert(key.to_string(), value.clone());

View File

@@ -1,11 +1,13 @@
//! Migrations module.
use anyhow::{Context as _, Result};
use rusqlite::OptionalExtension;
use crate::config::Config;
use crate::constants::ShowEmails;
use crate::constants::{self, ShowEmails};
use crate::context::Context;
use crate::imap;
use crate::message::MsgId;
use crate::provider::get_provider_by_domain;
use crate::sql::Sql;
use crate::tools::EmailAddress;
@@ -763,6 +765,141 @@ CREATE INDEX smtp_messageid ON imap(rfc724_mid);
.await?;
}
if dbversion < 105 {
// Create UNIQUE uid column and drop unused update_item_read column.
sql.execute_migration(
r#"CREATE TABLE new_msgs_status_updates (
id INTEGER PRIMARY KEY AUTOINCREMENT,
msg_id INTEGER,
update_item TEXT DEFAULT '',
uid TEXT UNIQUE
);
INSERT OR IGNORE INTO new_msgs_status_updates SELECT
id, msg_id, update_item, NULL
FROM msgs_status_updates;
DROP TABLE msgs_status_updates;
ALTER TABLE new_msgs_status_updates RENAME TO msgs_status_updates;
CREATE INDEX msgs_status_updates_index1 ON msgs_status_updates (msg_id);
CREATE INDEX msgs_status_updates_index2 ON msgs_status_updates (uid);
"#,
105,
)
.await?;
}
if dbversion < 106 {
// Recreate `config` table with UNIQUE constraint on `keyname`.
sql.execute_migration(
"CREATE TABLE new_config (
id INTEGER PRIMARY KEY,
keyname TEXT UNIQUE,
value TEXT NOT NULL
);
INSERT OR IGNORE INTO new_config SELECT
id, keyname, value
FROM config;
DROP TABLE config;
ALTER TABLE new_config RENAME TO config;
CREATE INDEX config_index1 ON config (keyname);",
106,
)
.await?;
}
if dbversion < 107 {
sql.execute_migration(
"CREATE TABLE new_keypairs (
id INTEGER PRIMARY KEY AUTOINCREMENT,
private_key UNIQUE NOT NULL,
public_key UNIQUE NOT NULL
);
INSERT OR IGNORE INTO new_keypairs SELECT id, private_key, public_key FROM keypairs;
INSERT OR IGNORE
INTO config (keyname, value)
VALUES
('key_id', (SELECT id FROM new_keypairs
WHERE private_key=
(SELECT private_key FROM keypairs
WHERE addr=(SELECT value FROM config WHERE keyname='configured_addr')
AND is_default=1)));
-- We do not drop the old `keypairs` table for now,
-- but move it to `old_keypairs`. We can remove it later
-- in next migrations. This may be needed for recovery
-- in case something is wrong with the migration.
ALTER TABLE keypairs RENAME TO old_keypairs;
ALTER TABLE new_keypairs RENAME TO keypairs;
",
107,
)
.await?;
}
if dbversion < 108 {
let version = 108;
let chunk_size = context
.get_configured_provider()
.await?
.and_then(|provider| provider.opt.max_smtp_rcpt_to)
.map_or(constants::DEFAULT_MAX_SMTP_RCPT_TO, usize::from);
sql.transaction(move |trans| {
Sql::set_db_version_trans(trans, version)?;
let id_max =
trans.query_row("SELECT IFNULL((SELECT MAX(id) FROM smtp), 0)", (), |row| {
let id_max: i64 = row.get(0)?;
Ok(id_max)
})?;
while let Some((id, rfc724_mid, mime, msg_id, recipients, retries)) = trans
.query_row(
"SELECT id, rfc724_mid, mime, msg_id, recipients, retries FROM smtp \
WHERE id<=? LIMIT 1",
(id_max,),
|row| {
let id: i64 = row.get(0)?;
let rfc724_mid: String = row.get(1)?;
let mime: String = row.get(2)?;
let msg_id: MsgId = row.get(3)?;
let recipients: String = row.get(4)?;
let retries: i64 = row.get(5)?;
Ok((id, rfc724_mid, mime, msg_id, recipients, retries))
},
)
.optional()?
{
trans.execute("DELETE FROM smtp WHERE id=?", (id,))?;
let recipients = recipients.split(' ').collect::<Vec<_>>();
for recipients in recipients.chunks(chunk_size) {
let recipients = recipients.join(" ");
trans.execute(
"INSERT INTO smtp (rfc724_mid, mime, msg_id, recipients, retries) \
VALUES (?, ?, ?, ?, ?)",
(&rfc724_mid, &mime, msg_id, recipients, retries),
)?;
}
}
Ok(())
})
.await
.with_context(|| format!("migration failed for version {version}"))?;
sql.set_db_version_in_cache(version).await?;
}
if dbversion < 109 {
sql.execute_migration(
r#"ALTER TABLE acpeerstates
ADD COLUMN backward_verified_key_id -- What we think the contact has as our verified key
INTEGER;
UPDATE acpeerstates
SET backward_verified_key_id=(SELECT value FROM config WHERE keyname='key_id')
WHERE verified_key IS NOT NULL
"#,
109,
)
.await?;
}
let new_version = sql
.get_raw_config_int(VERSION_CFG)
.await?
@@ -802,6 +939,12 @@ impl Sql {
Ok(())
}
async fn set_db_version_in_cache(&self, version: i32) -> Result<()> {
let mut lock = self.config_cache.write().await;
lock.insert(VERSION_CFG.to_string(), Some(format!("{version}")));
Ok(())
}
async fn execute_migration(&self, query: &str, version: i32) -> Result<()> {
self.transaction(move |transaction| {
Self::set_db_version_trans(transaction, version)?;
@@ -812,10 +955,6 @@ impl Sql {
.await
.with_context(|| format!("execute_migration failed for version {version}"))?;
let mut lock = self.config_cache.write().await;
lock.insert(VERSION_CFG.to_string(), Some(format!("{version}")));
drop(lock);
Ok(())
self.set_db_version_in_cache(version).await
}
}

View File

@@ -92,7 +92,7 @@ pub enum StockMessage {
#[strum(props(fallback = "%1$s verified."))]
ContactVerified = 35,
#[strum(props(fallback = "Cannot verify %1$s"))]
#[strum(props(fallback = "Cannot establish guaranteed end-to-end encryption with %1$s"))]
ContactNotVerified = 36,
#[strum(props(fallback = "Changed setup for %1$s"))]
@@ -419,6 +419,11 @@ pub enum StockMessage {
#[strum(props(fallback = "Member %1$s added."))]
MsgAddMember = 173,
#[strum(props(
fallback = "⚠️ Your email provider %1$s requires end-to-end encryption which is not setup yet."
))]
InvalidUnencryptedMail = 174,
}
impl StockMessage {
@@ -832,7 +837,7 @@ pub(crate) async fn contact_verified(context: &Context, contact: &Contact) -> St
.replace1(addr)
}
/// Stock string: `Cannot verify %1$s`.
/// Stock string: `Cannot establish guaranteed end-to-end encryption with %1$s`.
pub(crate) async fn contact_not_verified(context: &Context, contact: &Contact) -> String {
let addr = &contact.get_name_n_addr();
translated(context, StockMessage::ContactNotVerified)
@@ -1285,6 +1290,13 @@ pub(crate) async fn aeap_addr_changed(
.replace3(new_addr)
}
/// Stock string: `⚠️ Your email provider %1$s requires end-to-end encryption which is not setup yet. Tap to learn more.`.
pub(crate) async fn unencrypted_email(context: &Context, provider: &str) -> String {
translated(context, StockMessage::InvalidUnencryptedMail)
.await
.replace1(provider)
}
pub(crate) async fn aeap_explanation_and_link(
context: &Context,
old_addr: &str,

View File

@@ -20,7 +20,7 @@ use crate::tools::time;
use crate::{stock_str, token};
/// Whether to send device sync messages. Aimed for usage in the internal API.
#[derive(Debug)]
#[derive(Debug, PartialEq)]
pub(crate) enum Sync {
Nosync,
Sync,
@@ -35,6 +35,15 @@ impl From<Sync> for bool {
}
}
impl From<bool> for Sync {
fn from(sync: bool) -> Sync {
match sync {
false => Sync::Nosync,
true => Sync::Sync,
}
}
}
#[derive(Debug, Serialize, Deserialize)]
pub(crate) struct QrTokenData {
pub(crate) invitenumber: String,
@@ -50,6 +59,10 @@ pub(crate) enum SyncData {
id: chat::SyncId,
action: chat::SyncAction,
},
Config {
key: Config,
val: String,
},
}
#[derive(Debug, Serialize, Deserialize)]
@@ -263,6 +276,10 @@ impl Context {
AddQrToken(token) => self.add_qr_token(token).await,
DeleteQrToken(token) => self.delete_qr_token(token).await,
AlterChat { id, action } => self.sync_alter_chat(id, action).await,
SyncData::Config { key, val } => match key.is_synced() {
true => self.set_config_ex(Sync::Nosync, *key, Some(val)).await,
false => Ok(()),
},
},
SyncDataOrUnknown::Unknown(data) => {
warn!(self, "Ignored unknown sync item: {data}.");
@@ -430,7 +447,7 @@ mod tests {
)?;
assert_eq!(sync_items.items.len(), 1);
let SyncDataOrUnknown::SyncData(AlterChat { id, action }) =
&sync_items.items.get(0).unwrap().data
&sync_items.items.first().unwrap().data
else {
bail!("bad item");
};
@@ -474,7 +491,7 @@ mod tests {
assert_eq!(sync_items.items.len(), 1);
if let SyncDataOrUnknown::SyncData(AddQrToken(token)) =
&sync_items.items.get(0).unwrap().data
&sync_items.items.first().unwrap().data
{
assert_eq!(token.invitenumber, "in");
assert_eq!(token.auth, "yip");

View File

@@ -1047,7 +1047,8 @@ fn print_logevent(logevent: &LogEvent) {
}
}
/// Saves the other account's public key as verified.
/// Saves the other account's public key as verified
/// and peerstate as backwards verified.
pub(crate) async fn mark_as_verified(this: &TestContext, other: &TestContext) {
let mut peerstate = Peerstate::from_header(
&EncryptHelper::new(other).await.unwrap().get_aheader(),
@@ -1063,10 +1064,18 @@ pub(crate) async fn mark_as_verified(this: &TestContext, other: &TestContext) {
peerstate.verified_key = peerstate.public_key.clone();
peerstate.verified_key_fingerprint = peerstate.public_key_fingerprint.clone();
peerstate.backward_verified_key_id = Some(this.get_config_i64(Config::KeyId).await.unwrap());
peerstate.save_to_db(&this.sql).await.unwrap();
}
/// Pops a sync message from alice0 and receives it on alice1. Should be used after an action on
/// alice0's side that implies sending a sync message.
pub(crate) async fn sync(alice0: &TestContext, alice1: &TestContext) {
let sync_msg = alice0.pop_sent_msg().await;
alice1.recv_msg(&sync_msg).await;
}
/// Pretty-print an event to stdout
///
/// Done during tests this is captured by `cargo test` and associated with the test itself.

View File

@@ -1,12 +1,11 @@
use anyhow::Result;
use pretty_assertions::assert_eq;
use crate::chat::ProtectionStatus;
use crate::chat::{self, Chat, ProtectionStatus};
use crate::chatlist::Chatlist;
use crate::config::Config;
use crate::constants::DC_GCL_FOR_FORWARDING;
use crate::contact::VerifiedStatus;
use crate::contact::{Contact, Origin};
use crate::constants::{Chattype, DC_GCL_FOR_FORWARDING};
use crate::contact::{Contact, ContactId, Origin};
use crate::message::{Message, Viewtype};
use crate::mimefactory::MimeFactory;
use crate::mimeparser::SystemMessage;
@@ -70,10 +69,7 @@ async fn check_verified_oneonone_chat(broken_by_classical_email: bool) {
tcm.send_recv(&bob, &alice, "Using DC again").await;
let contact = alice.add_or_lookup_contact(&bob).await;
assert_eq!(
contact.is_verified(&alice.ctx).await.unwrap(),
VerifiedStatus::BidirectVerified
);
assert_eq!(contact.is_verified(&alice.ctx).await.unwrap(), true);
// Bob's chat is marked as verified again
assert_verified(&alice, &bob, ProtectionStatus::Protected).await;
@@ -121,10 +117,7 @@ async fn test_create_verified_oneonone_chat() -> Result<()> {
// Alice and Fiona should now be verified because of gossip
let alice_fiona_contact = alice.add_or_lookup_contact(&fiona).await;
assert_eq!(
alice_fiona_contact.is_verified(&alice).await.unwrap(),
VerifiedStatus::BidirectVerified
);
assert!(alice_fiona_contact.is_verified(&alice).await.unwrap(),);
// Alice should have a hidden protected chat with Fiona
{
@@ -684,7 +677,7 @@ async fn test_break_protection_then_verify_again() -> Result<()> {
// Bob sent a message with a new key, so he most likely doesn't have
// the old key anymore. This means that Alice's device should show
// him as unverified:
VerifiedStatus::Unverified
false
);
let chat = alice.get_chat(&bob_new).await;
assert_eq!(chat.is_protected(), false);
@@ -782,14 +775,49 @@ async fn test_create_oneonone_chat_with_former_verified_contact() -> Result<()>
Ok(())
}
/// Tests that on the second device of a protected group creator the first message is
/// `SystemMessage::ChatProtectionEnabled` and the second one is the message populating the group.
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_create_protected_grp_multidev() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
let alice1 = &tcm.alice().await;
let group_id = alice
.create_group_with_members(ProtectionStatus::Protected, "Group", &[])
.await;
assert_eq!(
get_chat_msg(alice, group_id, 0, 1).await.get_info_type(),
SystemMessage::ChatProtectionEnabled
);
let sent = alice.send_text(group_id, "Hey").await;
// This sleep is necessary to reproduce the bug when the original message is sorted over the
// "protection enabled" message so that these messages have different timestamps. The better way
// would be to adjust the system time here if we could mock the system clock for the tests.
tokio::time::sleep(std::time::Duration::from_millis(2000)).await;
let msg = alice1.recv_msg(&sent).await;
let group1 = Chat::load_from_db(alice1, msg.chat_id).await?;
assert_eq!(group1.get_type(), Chattype::Group);
assert!(group1.is_protected());
assert_eq!(
chat::get_chat_contacts(alice1, group1.id).await?,
vec![ContactId::SELF]
);
assert_eq!(
get_chat_msg(alice1, group1.id, 0, 2).await.get_info_type(),
SystemMessage::ChatProtectionEnabled
);
assert_eq!(get_chat_msg(alice1, group1.id, 1, 2).await.id, msg.id);
Ok(())
}
// ============== Helper Functions ==============
async fn assert_verified(this: &TestContext, other: &TestContext, protected: ProtectionStatus) {
let contact = this.add_or_lookup_contact(other).await;
assert_eq!(
contact.is_verified(this).await.unwrap(),
VerifiedStatus::BidirectVerified
);
assert_eq!(contact.is_verified(this).await.unwrap(), true);
let chat = this.get_chat(other).await;
let (expect_protected, expect_broken) = match protected {

View File

@@ -5,6 +5,7 @@
//! - `id` - status update serial number
//! - `msg_id` - ID of the message in the `msgs` table
//! - `update_item` - JSON representation of the status update
//! - `uid` - "id" field of the update, used for deduplication
//!
//! Status updates are scheduled for sending by adding a record
//! to `smtp_status_updates_table` SQL table.
@@ -14,7 +15,6 @@
//! - `last_serial` - serial number of the last status update to send
//! - `descr` - text to send along with the updates
use std::convert::TryFrom;
use std::path::Path;
use anyhow::{anyhow, bail, ensure, format_err, Context as _, Result};
@@ -37,6 +37,7 @@ use crate::mimefactory::wrapped_base64_encode;
use crate::mimeparser::SystemMessage;
use crate::param::Param;
use crate::param::Params;
use crate::tools::create_id;
use crate::tools::strip_rtlo_characters;
use crate::tools::{create_smeared_timestamp, get_abs_path};
@@ -62,11 +63,6 @@ const WEBXDC_DEFAULT_ICON: &str = "__webxdc__/default-icon.png";
/// it is planned to raise that limit as needed in subsequent versions.
const WEBXDC_SENDING_LIMIT: u64 = 655360;
/// Be more tolerant for .xdc sizes on receiving -
/// might be, the senders version uses already a larger limit
/// and not showing the .xdc on some devices would be even worse ux.
const WEBXDC_RECEIVING_LIMIT: u64 = 4194304;
/// Raw information read from manifest.toml
#[derive(Debug, Deserialize, Default)]
#[non_exhaustive]
@@ -178,6 +174,13 @@ pub struct StatusUpdateItem {
/// for a voting app.
#[serde(skip_serializing_if = "Option::is_none")]
pub summary: Option<String>,
/// Unique ID for deduplication.
/// This can be used if the message is sent over multiple transports.
///
/// If there is no ID, message is always considered to be unique.
#[serde(skip_serializing_if = "Option::is_none")]
pub uid: Option<String>,
}
/// Update items as passed to the UIs.
@@ -210,14 +213,6 @@ impl Context {
return Ok(false);
}
if file.len() as u64 > WEBXDC_RECEIVING_LIMIT {
info!(
self,
"{} exceeds receiving limit of {} bytes", &filename, WEBXDC_RECEIVING_LIMIT
);
return Ok(false);
}
let archive = match async_zip::read::mem::ZipFileReader::new(file.to_vec()).await {
Ok(archive) => archive,
Err(_) => {
@@ -317,7 +312,14 @@ impl Context {
timestamp: i64,
can_info_msg: bool,
from_id: ContactId,
) -> Result<StatusUpdateSerial> {
) -> Result<Option<StatusUpdateSerial>> {
let Some(status_update_serial) = self
.write_status_update_inner(&instance.id, &status_update_item)
.await?
else {
return Ok(None);
};
if can_info_msg {
if let Some(ref info) = status_update_item.info {
if let Some(info_msg_id) =
@@ -376,10 +378,6 @@ impl Context {
self.emit_msgs_changed(instance.chat_id, instance.id);
}
let status_update_serial = self
.write_status_update_inner(&instance.id, status_update_item)
.await?;
if instance.viewtype == Viewtype::Webxdc {
self.emit_event(EventType::WebxdcStatusUpdate {
msg_id: instance.id,
@@ -387,23 +385,43 @@ impl Context {
});
}
Ok(status_update_serial)
Ok(Some(status_update_serial))
}
/// Inserts a status update item into `msgs_status_updates` table.
///
/// Returns serial ID of the status update if a new item is inserted.
pub(crate) async fn write_status_update_inner(
&self,
instance_id: &MsgId,
status_update_item: StatusUpdateItem,
) -> Result<StatusUpdateSerial> {
let rowid = self
status_update_item: &StatusUpdateItem,
) -> Result<Option<StatusUpdateSerial>> {
let _lock = self.sql.write_lock().await;
let uid = status_update_item.uid.as_deref();
let Some(rowid) = self
.sql
.insert(
"INSERT INTO msgs_status_updates (msg_id, update_item) VALUES(?, ?);",
(instance_id, serde_json::to_string(&status_update_item)?),
.query_row_optional(
"INSERT INTO msgs_status_updates (msg_id, update_item, uid) VALUES(?, ?, ?)
ON CONFLICT (uid) DO NOTHING
RETURNING id",
(
instance_id,
serde_json::to_string(&status_update_item)?,
uid,
),
|row| {
let id: u32 = row.get(0)?;
Ok(id)
},
)
.await?;
let status_update_serial = StatusUpdateSerial(u32::try_from(rowid)?);
Ok(status_update_serial)
.await?
else {
let uid = uid.unwrap_or("-");
info!(self, "Ignoring duplicate status update with uid={uid}");
return Ok(None);
};
let status_update_serial = StatusUpdateSerial(rowid);
Ok(Some(status_update_serial))
}
/// Returns the update_item with `status_update_serial` from the webxdc with message id `msg_id`.
@@ -432,13 +450,8 @@ impl Context {
update_str: &str,
descr: &str,
) -> Result<()> {
let status_update_item: StatusUpdateItem =
if let Ok(item) = serde_json::from_str::<StatusUpdateItem>(update_str) {
item
} else {
bail!("create_status_update_record: no valid update item.");
};
let status_update_item: StatusUpdateItem = serde_json::from_str(update_str)
.with_context(|| format!("Failed to parse webxdc update item from {update_str:?}"))?;
self.send_webxdc_status_update_struct(instance_msg_id, status_update_item, descr)
.await?;
Ok(())
@@ -449,17 +462,27 @@ impl Context {
pub async fn send_webxdc_status_update_struct(
&self,
instance_msg_id: MsgId,
status_update: StatusUpdateItem,
mut status_update: StatusUpdateItem,
descr: &str,
) -> Result<()> {
let mut instance = Message::load_from_db(self, instance_msg_id).await?;
if instance.viewtype != Viewtype::Webxdc {
bail!("send_webxdc_status_update: is no webxdc message");
let mut instance = Message::load_from_db(self, instance_msg_id)
.await
.with_context(|| {
format!("Failed to load message {instance_msg_id} from the database")
})?;
let viewtype = instance.viewtype;
if viewtype != Viewtype::Webxdc {
bail!("send_webxdc_status_update: message {instance_msg_id} is not a webxdc message, but a {viewtype} message.");
}
let chat = Chat::load_from_db(self, instance.chat_id).await?;
if let Some(reason) = chat.why_cant_send(self).await? {
bail!("cannot send to {}: {}", chat.id, reason);
let chat_id = instance.chat_id;
let chat = Chat::load_from_db(self, chat_id)
.await
.with_context(|| format!("Failed to load chat {chat_id} from the database"))?;
if let Some(reason) = chat.why_cant_send(self).await.with_context(|| {
format!("Failed to check if webxdc update can be sent to chat {chat_id}")
})? {
bail!("Cannot send to {chat_id}: {reason}.");
}
let send_now = !matches!(
@@ -467,6 +490,7 @@ impl Context {
MessageState::Undefined | MessageState::OutPreparing | MessageState::OutDraft
);
status_update.uid = Some(create_id());
let status_update_serial: StatusUpdateSerial = self
.create_status_update_record(
&mut instance,
@@ -475,7 +499,9 @@ impl Context {
send_now,
ContactId::SELF,
)
.await?;
.await
.context("Failed to create status update")?
.context("Duplicate status update UID was generated")?;
if send_now {
self.sql.insert(
@@ -483,7 +509,7 @@ impl Context {
ON CONFLICT(msg_id)
DO UPDATE SET last_serial=excluded.last_serial, descr=excluded.descr",
(instance.id, status_update_serial, status_update_serial, descr),
).await?;
).await.context("Failed to insert webxdc update into SMTP queue")?;
self.scheduler.interrupt_smtp().await;
}
Ok(())
@@ -655,7 +681,10 @@ impl Context {
let (update_item_str, serial) = row;
let update_item = StatusUpdateItemAndSerial
{
item: serde_json::from_str(&update_item_str)?,
item: StatusUpdateItem {
uid: None, // Erase UIDs, apps, bots and tests don't need to know them.
..serde_json::from_str(&update_item_str)?
},
serial,
max_serial,
};
@@ -854,7 +883,7 @@ mod tests {
use crate::chatlist::Chatlist;
use crate::config::Config;
use crate::contact::Contact;
use crate::receive_imf::{receive_imf, receive_imf_inner};
use crate::receive_imf::{receive_imf, receive_imf_from_inbox};
use crate::test_utils::TestContext;
use crate::{message, sql};
@@ -863,8 +892,6 @@ mod tests {
async fn test_webxdc_file_limits() -> Result<()> {
assert!(WEBXDC_SENDING_LIMIT >= 32768);
assert!(WEBXDC_SENDING_LIMIT < 16777216);
assert!(WEBXDC_RECEIVING_LIMIT >= WEBXDC_SENDING_LIMIT * 2);
assert!(WEBXDC_RECEIVING_LIMIT < 16777216);
Ok(())
}
@@ -1189,7 +1216,7 @@ mod tests {
let sent2 = alice.pop_sent_msg().await;
// Bob does not download instance but already receives update
receive_imf_inner(
receive_imf_from_inbox(
&bob,
&alice_instance.rfc724_mid,
sent1.payload().as_bytes(),
@@ -1204,7 +1231,7 @@ mod tests {
assert_eq!(bob_instance.download_state, DownloadState::Available);
// Bob downloads instance, updates should be assigned correctly
let received_msg = receive_imf_inner(
let received_msg = receive_imf_from_inbox(
&bob,
&alice_instance.rfc724_mid,
sent1.payload().as_bytes(),
@@ -1214,7 +1241,7 @@ mod tests {
)
.await?
.unwrap();
assert_eq!(*received_msg.msg_ids.get(0).unwrap(), bob_instance.id);
assert_eq!(*received_msg.msg_ids.first().unwrap(), bob_instance.id);
let bob_instance = bob.get_last_msg().await;
assert_eq!(bob_instance.viewtype, Viewtype::Webxdc);
assert_eq!(bob_instance.download_state, DownloadState::Done);
@@ -1348,17 +1375,38 @@ mod tests {
info: None,
document: None,
summary: None,
uid: Some("iecie2Ze".to_string()),
},
1640178619,
true,
ContactId::SELF,
)
.await?
.unwrap();
assert_eq!(
t.get_webxdc_status_updates(instance.id, StatusUpdateSerial(0))
.await?,
r#"[{"payload":{"foo":"bar"},"serial":1,"max_serial":1}]"#
);
// Update with duplicate update ID is received.
// Whatever the payload is, update should be ignored just because ID is duplicate.
let update_id1_duplicate = t
.create_status_update_record(
&mut instance,
StatusUpdateItem {
payload: json!({"nothing": "this should be ignored"}),
info: None,
document: None,
summary: None,
uid: Some("iecie2Ze".to_string()),
},
1640178619,
true,
ContactId::SELF,
)
.await?;
assert_eq!(
t.get_webxdc_status_updates(instance.id, StatusUpdateSerial(0))
.await?,
r#"[{"payload":{"foo":"bar"},"serial":1,"max_serial":1}]"#
);
assert_eq!(update_id1_duplicate, None);
assert!(t
.send_webxdc_status_update(instance.id, "\n\n\n", "")
@@ -1384,15 +1432,17 @@ mod tests {
info: None,
document: None,
summary: None,
uid: None,
},
1640178619,
true,
ContactId::SELF,
)
.await?;
.await?
.unwrap();
assert_eq!(
t.get_webxdc_status_updates(instance.id, update_id1).await?,
r#"[{"payload":{"foo2":"bar2"},"serial":2,"max_serial":2}]"#
r#"[{"payload":{"foo2":"bar2"},"serial":3,"max_serial":3}]"#
);
t.create_status_update_record(
&mut instance,
@@ -1401,6 +1451,7 @@ mod tests {
info: None,
document: None,
summary: None,
uid: None,
},
1640178619,
true,
@@ -1410,9 +1461,9 @@ mod tests {
assert_eq!(
t.get_webxdc_status_updates(instance.id, StatusUpdateSerial(0))
.await?,
r#"[{"payload":{"foo":"bar"},"serial":1,"max_serial":3},
{"payload":{"foo2":"bar2"},"serial":2,"max_serial":3},
{"payload":true,"serial":3,"max_serial":3}]"#
r#"[{"payload":{"foo":"bar"},"serial":1,"max_serial":4},
{"payload":{"foo2":"bar2"},"serial":3,"max_serial":4},
{"payload":true,"serial":4,"max_serial":4}]"#
);
t.send_webxdc_status_update(
@@ -1423,8 +1474,8 @@ mod tests {
.await?;
assert_eq!(
t.get_webxdc_status_updates(instance.id, update_id2).await?,
r#"[{"payload":true,"serial":3,"max_serial":4},
{"payload":1,"serial":4,"max_serial":4}]"#
r#"[{"payload":true,"serial":4,"max_serial":5},
{"payload":1,"serial":5,"max_serial":5}]"#
);
Ok(())
@@ -1468,7 +1519,7 @@ mod tests {
t.receive_status_update(
ContactId::SELF,
instance.id,
r#"{"updates":[{"payload":{"foo":"bar"}}]}"#,
r#"{"updates":[{"payload":{"foo":"bar"}, "someTrash": "definitely TrAsH"}]}"#,
)
.await?;
assert_eq!(
@@ -1654,6 +1705,8 @@ mod tests {
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_render_webxdc_status_update_object_range() -> Result<()> {
use regex::Regex;
let t = TestContext::new_alice().await;
let chat_id = create_group_chat(&t, ProtectionStatus::Unprotected, "a chat").await?;
let instance = send_webxdc_instance(&t, chat_id).await?;
@@ -1672,7 +1725,13 @@ mod tests {
)
.await?
.unwrap();
assert_eq!(json, "{\"updates\":[{\"payload\":2},\n{\"payload\":3}]}");
let json = Regex::new(r#""uid":"[^"]*""#)
.unwrap()
.replace_all(&json, "XXX");
assert_eq!(
json,
"{\"updates\":[{\"payload\":2,XXX},\n{\"payload\":3,XXX}]}"
);
assert_eq!(
t.sql

View File

@@ -0,0 +1,93 @@
From - Thu, 24 Nov 2022 19:06:16 GMT
X-Mozilla-Status: 0001
X-Mozilla-Status2: 00800000
Message-ID: <0bb9ffe1-2596-d997-95b4-1fef8cc4808f@example.org>
Date: Thu, 24 Nov 2022 20:05:57 +0100
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
Thunderbird/102.4.2
From: Alice <alice@example.org>
To: bob@example.net
Content-Language: en-US
Autocrypt: addr=alice@example.org; keydata=
xjMEXlh13RYJKwYBBAHaRw8BAQdAzfVIAleCXMJrq8VeLlEVof6ITCviMktKjmcBKAu4m5DN
GUFsaWNlIDxhbGljZUBleGFtcGxlLm9yZz7CkAQTFggAOBYhBC5vossjtTLXKGNLWGSwj2Gp
7ZRDBQJeWHXdAhsDBQsJCAcCBhUKCQgLAgQWAgMBAh4BAheAAAoJEGSwj2Gp7ZRDE3oA/i4M
CyDMTsjWqDZoQwX/A/GoTO2/V0wKPhjJJy/8m2pMAPkBjOnGOtx2SZpQvJGTa9h804RY6iDr
RuI8A/8tEEXAA844BF5Ydd0SCisGAQQBl1UBBQEBB0AG7cjWy2SFAU8KnltlubVW67rFiyfp
01JrRe6Xqy22HQMBCAfCeAQYFggAIBYhBC5vossjtTLXKGNLWGSwj2Gp7ZRDBQJeWHXdAhsM
AAoJEGSwj2Gp7ZRDLo8BAObE8GnsGVwKzNqCvHeWgJsqhjS3C6gvSlV3tEm9XmF6AQDXucIy
VfoBwoyMh2h6cSn/ATn5QJb35pgo+ivp3jsMAg==
Subject: ...
Content-Type: multipart/encrypted;
protocol="application/pgp-encrypted";
boundary="------------EOdOT2kJUL5hgCilmIhYyVZg"
This is an OpenPGP/MIME encrypted message (RFC 4880 and 3156)
--------------EOdOT2kJUL5hgCilmIhYyVZg
Content-Type: application/pgp-encrypted
Content-Description: PGP/MIME version identification
Version: 1
--------------EOdOT2kJUL5hgCilmIhYyVZg
Content-Type: application/octet-stream; name="encrypted.asc"
Content-Description: OpenPGP encrypted message
Content-Disposition: inline; filename="encrypted.asc"
-----BEGIN PGP MESSAGE-----
wV4D5tq63hTeebASAQdA1dVUsUjGZCOIfCnYtVdmOvKs/BNovI3sG8w1IH4ymTMwAZzgwVbGS5KL
+e1VTD5mUTeVSEYe1cd3VozH4KbNJa1tBlcO0nzGwCPpsTVDMoxIwcBMA+PY3JvEjuMiAQf/d2yj
t0+GyaptwX26bgSqo6vj21W8mcWS5vXOi8wjGwRbPaKKjS4kq1xDOz04eHrE8HUPD8otcXoI8CLz
etJpRbFs0XJP4Cozbsr72dgoWhozRg/iSpBndxWOddTl7Yqo8m/fyhU5uzKZ41m2T8mha6KkKWD8
QecGdOgieYBucNBjHwWc71p9G6jTnzfy4S4GtGS2gwOSMxpwO7HxpKzsHI4POqFSQbxrl/YRwWSC
f5WqyYcerasIiR/fnOIw8lnvCeQ5rB90eGEDR70YFGt0t4rFBjfGrSPUiWYaTaC1Zvpd+t5sy7zy
FpsS2/aTkwP/UpGqmtFaD/brSouRf9hijNLI0QFTaVmSoI3BKzF8B4zwvtEbOLZjyDb+Va/fZJ3w
nYd2Q/5PPPL+pE4pWKN+jl0TZNzAaqBgvggXomgUqQ7QiksUzym+yuFKrJX0RF2awdrgjQIxjnda
Qp3UFphnFTyYUJpIU9iewjOfVxgPzv7PyuCHYwoP3kh7MJZ6bgbDmOkeFSnjEDJpdf1m9xC9LlBL
beC8scmPs6kx9GARBYSHvyPQ025gN3+XEHh4OrTxHZ91U3IlTfd2kACwOOAXEuhItSHmcNOV0K4M
nI2PH6gW8HgBkWlAPm40K4jUyo3nl1usDiI6ouvYqvW7YUc2hTtPTej1l2/mS57tTt+PFurKs555
5R9DD/xg9Nx7OuQKy5bIdlXM20UmwuZTOhRJ5kpHFRzLxaHDbSzW+orhRW4llJSevBSAH3cLOjIQ
gh87j+MxG9j0TD2K2A0rcUcxdrnflw+mxcDVaL4payeqmOa+bJyhlftTqH+vqq5DhR68rX5VW+z7
riqH3o8VbvO2y0XSpYHf1jowkfJj3vr8pynAUIv1dbylUSF5wtrHvzWOprw4bNrdtwQNRNy+JcVF
dUKeNmHaL6XOe4LUWpiI11beRyCpAG52khMCEAO3Q6+4e24cEipbu6suSOtv3OpYDZeHjwNrQIhi
rJg7i9TpMqwOeCvFWK+9UZ+P2n6h9g0/JO2+I82BFGUjVa5IvCTNOgv01GqxWY9ecdtaJjTc+dF2
OAcRoKwvmtMJlxKEEgveui3BvPA4tuNdSrcoZBrQeo0ZHWVugXPvEZnwfZMcqwwPA+a/sUbZFg0P
Pr0AR0ZHpytnQE9OXE8wEUgT8H1yofQ+5QoZdgMpeAb8zGs+RuviLxcDkb9NtXUAiQ49ooWuFP3L
K9wMlaoWFTq7R+n5JVuSEYRCHC0l0bCV1/+awalT7XltXVCupI4lWzjYs52FZGGzuHG7S50Eufad
m4CQTPVgVaVn8WW2dmpMR8Gj8WbbZdyv21wMGOWjfgT0u3oiDnddGrFOoMNnZHch6rN3FRppoh7h
0U0fi8xxU1+EhUKq+fSIxZNr2iWN2if3Pipbxi9tyK9M41Y6aVF3HWjD58/OEql3aZjJZ1bqpXcE
qsPeFoXX78+7mTDvL75olMk2s/mg4mLqAAWQvTuoiOmj+SgMIFuTtFR+4r/TIFNdamz6AQ3RcmWG
ZcdRii+V27dtMA836vlAwxXRmJyE1LCL1kvUTq+J+AVsZi3xmBLFNlKPTlxswu7vSBrP1DlYOaBq
AgA0lKnkQdeXyDk/VdbTml7ywMW1g6HkFSqKGW/IIAObmBumBcIyHE6dWEHumRQomlJssIlEFSe+
XEQ0rwedLetJXi5A0AXT1we1wvaKCEg0Pb0ZUxygwNPDrj6MmdodH7gDfyx0mW/7mEMCtIJb5MB+
TRGPEa/vqdJb8uGtNXUy9UlwMhJ3tYoT7NXY4+IlNjbDH/yleMdwtWP2H2WH8oC+ysXPYXjlT8eU
poxRfJzPMVUn5SA3cvdGXDJWdX8U91j5sf9wuoYE5RBVrrJif3D3l0FpMrlWWoGw7wtZbMC2FaeT
QvdMS5c54IoXBtBTM+/AsTAw7WEE1QSmaQGHnh6xLL5Ns8olsWeKOMlVXdO9jSDbjOGBLr7mWukW
YzLXkH3TtJPQcbVN79af3YPhaHdMYITVKIwfg+vxZlLFHWLJQnkTl+9Qi7u2gKqkNeU7Zqs4E3CR
9K4dHrJMyAZLZ2HA1XQEj0/tMnbTpAzZhj02JRcFobLXK9SQfw7dzGZwMRky8cHcBHoK14P5RIEV
hr+38HSBM6wXtge5gL6DomAACvuORQO4X9x/CTjRt/J8uN3lKK5p+wi3ULeb319CEWiCiqmC1M+C
TADUhPUhUmTinSAVkTEn+BdbH/97dVaJnvd6HtLmdSlw4xqdWUfVL9Qd7+/5L6iwlOzGLKRv97c/
gCRw+hzXyAom+5C18slSwanMuyPgIyrrFy/kp9Romk9SQr/c0CUF2am99t8G5qvVi/TiJGHyKEXD
aUYd4V7lqNlHMiiasvFHeq8blwmFr7rGEvbZzLNplc6sRUVlYhY2unRfyWsq9mqk3NDRW12Fa0J2
YxQJlnXHQhNE8EyM/zsD9jCVNwsRZJ9/e5KS+ignmu6gKIR+ItDTwRfNI+NG/YmTgENUTyuO+vQC
CUKS3PCwpP+OEC966ARl7OCMdfn1hEyiAxsZnp1RmFngR6FM+mlGgfUoWNoHvnR1/YyQ4F4dadiA
QINwuSm5faw75F1EeL8Qi+LHKuqt05Pi/V9GJ6TzIkIsEbyyJ5sKHrp4QsU4C1p7ZhPjddz8De8k
6ZdwMIeXxi27WKtsFLcr8JKOBe0imIilKdMBOPS31pc1iJe4472WbWM0aBwdEYmnz9+xfOqnjHtO
0XTMjff7pzV6Y7t/u8J/zm3JS3ykote9HNRQvhZZNeVClVWd0fYFzat5ESnTojZTwHcc/BFTPnhz
VgLyw1KEIy2r3ZyGHu1b8GSYivzl33MOK/NVBQPZUIEfdcQ5vhkAvj+Yx340IYykRFEChwioprXD
LrIbTou7TNT5fTFA+beidHFsL+OE002/LMs6C3erSUW5C/LNjAQMS7cAV2yCyjX+/2GBmmDqnC4r
Ja2x5yik+fbOUPh3kk/md1YvrodlX/JkQeoWRrrVJsX2dr3BgivPJavaN0Jz1eHyxAYKNqlrfd1T
YWEDIisWerTxAVY/rEruZ6+OqLqOtZtn+4SOajOq8KFusglaMZqoYuM+LhPZck9PlZXwRqX08Vlv
8jX5V75BFWRhFd5/LYbnQHI6ZW80Wb2sBNngLL2QJT9yXGCDJb5qCdFwGd3i655pvRJXabeyCtDD
7I2PJcYRDd4stdq07BHyHJmye6vas8mG5QUygyWyUQv78za0m4gLMrRZBgoBDcVpWJUc+cPXzzfG
7PvLZu/Y0SaD5hqTp0LBB1PFxTpzdVeJ21gzVNQ6D4XGLTtdv4K4fOEYoeKEuzGoBaUDtIqz47gd
5rwfQ3ps2slkxfbtQcdKEACKvsCwzqHlgwsxD8QNOFzXYLiiiJBX22fIRoiJeSDMKSZyuFtpykCm
7bOpybPSHv3E7EIr8sIOr9MOe/R5HSthU2IgW1L5Ynr2t9HUnCA8CenkzIQjg0h5sruxcGWCYLx7
q0f1AQs4Z7SebVbq1SCWVJNX/vc1bVjnjYfri7RX5WMmjJkuSnuIoP6a42cqJcAg7m0STB0elFAy
oO4vW9/JEmFUqLyQmWnoLJHX3IKtWa9CPvE=
=OA6b
-----END PGP MESSAGE-----
--------------EOdOT2kJUL5hgCilmIhYyVZg--

View File

@@ -0,0 +1,66 @@
Return-Path: <mailing-list-bounce@example.org>
Delivered-To: alice@testrun.org
Date: Tue, 02 Jan 2024 05:00:00 +0000
From: mailing-list@example.org
Sender: mailing-list-bounce@example.org
To: alice@testrun.org
Message-ID: <87wmss8juz.fsf@example.org>
In-Reply-To:
References:
Subject: [REPOST] Some subject
Mime-Version: 1.0
Content-Type: multipart/signed;
boundary="--==_mimepart_65938a80866e8_663a2abed9b585c064398";
micalg=pgp-sha1;
protocol="application/pgp-signature"
Content-Transfer-Encoding: 7bit
List-Id: <mailing-list.example.org>
List-Owner: <mailto:mailing-list-owner@example.org> (Use list's public
key)
List-Help: <https://schleuder.org/>
List-Post: <mailto:mailing-list@example.org>
This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
----==_mimepart_65938a80866e8_663a2abed9b585c064398
Content-Type: multipart/mixed;
boundary="--==_mimepart_65938a8086476_663a2abed9b585c0642c7";
charset=UTF-8
Content-Transfer-Encoding: 7bit
----==_mimepart_65938a8086476_663a2abed9b585c0642c7
Content-Type: text/plain;
charset=UTF-8
Content-Transfer-Encoding: 7bit
From: bob@example.org
To: mailing-list@example.org
Cc:
Date: Tue, 02 Jan 2024 05:00:00 +0000
Sig: Unsigned
Enc: Unencrypted
----==_mimepart_65938a8086476_663a2abed9b585c0642c7
Content-Type: text/plain;
charset=utf-8
Content-Transfer-Encoding: quoted-printable
hello,
bye
----==_mimepart_65938a8086476_663a2abed9b585c0642c7--
----==_mimepart_65938a80866e8_663a2abed9b585c064398
Content-Type: application/pgp-signature;
name=signature.asc
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename=signature.asc
Content-Description: OpenPGP digital signature
-----BEGIN PGP SIGNATURE-----
REDACTED
-----END PGP SIGNATURE-----
----==_mimepart_65938a80866e8_663a2abed9b585c064398--

View File

@@ -0,0 +1,91 @@
From - Sun, 19 Nov 2023 01:08:24 GMT
X-Mozilla-Status: 0800
X-Mozilla-Status2: 00000000
Message-ID: <38a2a29b-8261-403b-abb5-56b0a87d2ff4@example.org>
Date: Sat, 18 Nov 2023 22:08:23 -0300
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Content-Language: en-US
To: bob@example.net
From: Alice <alice@example.org>
Autocrypt: addr=alice@example.org; keydata=
xsDNBGOaGzQBDADCFtBNMHRDJQRkd2tNm7CJm1Yo3Y5r3qP6v0FSwP1BIHbiIf0E/jFiKZWj
1uL68J2mGUuUu+Qi4ovf1l9/QQYzg/DCaLZxlbc0LKu2LXcpUL5DPu37mdw+DKs0YvNIlc+A
RjyFUwd3rsZN3k58inf1mYzKuKU6NpbdXULbOEYwnVEwzQsrtS2JgJ+tLSYUvNJeMJXm/cDL
XKJSApAyvVVdxxteG8uWcDqWV/HcXuopXLILf3yJF0De11/7G62dHNHuhmtgRLsTN4Q372Q9
KNdYEFLHaN91jEzyD/+aHNskATxtcGhppI8OQsU3NzNgHyd8Smzx5oTyZ/6NdhYoh0pKB8yf
VAyA69t5fctQRb4+bTwL+sS9KDobQOvcyOMUSccDfUhsWMghwsMCwU4Sz9hIY6dCAfroDAiL
vYUfdNJstAqvLf04mZtMmkI7Q2BYLETEgu4KQzQHRQekmOE/3EaSiojNa4ZTVURMdJ9U+I3E
q8e6TbOY7Xa4V8krAt/F2wMAEQEAAc0ZQWxpY2UgPGFsaWNlQGV4YW1wbGUub3JnPsLBDQQT
AQgANxYhBBSrP2X8J0u9tfp2jCXwByRZ5HriBQJjmhs1BQm7+B4AAhsDBAsJCAcFFQgJCgsF
FgIDAQAACgkQJfAHJFnkeuKQ2wwAgDgiCI6bz9PjqE1GoDcy/xQdy+nnYq5pOuHGUndZ7jYK
cOqM8LDEaG7GgrFsbs9vGhTA1fyqncM41pB7SmwQ7zBVaMdtHoulEG4RPGVboDaY9tuMOL3/
GVxFbovVHyU5Lr1euryNh/0JvMITY0UHaEY1k1M7izYUMyFu8I1ODZ9Iws2trUyU3Omw/sTJ
x15zzCsK8Aq+r3JmB+Q33SFSgWr/YWH0dQVIQ0I5iLN2q14oucmLBaKc9EXdRLiu8S8lLSQl
nfISJ17GBLmH1YxmPPZ3CRHC6iEKCLR6G9wzhsTPNdK7dRCYR5wTI27RVPLBcSnCKAeTopAJ
YskyNndtv0iaNRT7YLOfhrsBAofSjuLegR04CNiqBHtYQ3LO3WKhJ7riRcQ/Ksv0wYkmj1gJ
8myMwA+ybfYrpNqO4devnCvE3Eo5gzeYbvYU2Z17n9y6HAOG9/Tm/daiGEP2ni6iwV0kqLzw
eC48R1D75T66PxX/jQooujrTph8+K3ckV/q+zsDNBGOaGzUBDADV+DGgKxvCpfVFuPGrSdRU
06dxowdKOKavO6WGMvN3g/+CFrIsjUFy4S0Soo5ARnLh23i49ZSjacXFpgtZUNV3iGOSOcSE
LldLtZk5BV9w/ATqqgu4/LVdNA9rm+o197bIeSQCRTnY/QV6FdKYxVd4NBVH9abZ7t8Tm4qC
urZj56MjPCg3fqT+Q6sjxH+nKBrs8s8iCJkYhGBgU3q5W+wrtZ56kI9mxJec62KHpyLZ0rTE
xEAeVbChUJOo11vUtJfTrDhI6lhqyr72o/A6bY1OV7WzkxtiBRl35eewQ+RDLJ4yxaNj/XTS
UxOz60xNggEfDVtfgfjBZrBbiHXqf8iKVV1ZPGm0ycvXZGYFw2zXLI2PwevhQCm+t4Ywty1h
8l019MYmGadpQgbuA4ZippuzOSzSGMQ+S4uYEzeeymR9ksxVSXn90HEzqC7LdHCcd2IO6rfu
g2fuRf258Adfuoh3s8YUlWyXjEHLXKo9SRgGMfGs7qgCOL/ReAwFPtKACvEAEQEAAcLA/AQY
AQgAJhYhBBSrP2X8J0u9tfp2jCXwByRZ5HriBQJjmhs2BQm7+B4AAhsMAAoJECXwByRZ5Hri
EOkMALtq4DVYX8RfoPdU0Dt6y+yDj1NALv5GefvHbgfuaVT8PaOP0gxCjWrnUDvvJEwP1W3j
UXYqDwKP42hiGWsnXk2hbgXbplArgP3H987x7c8bu1wIAmkJ9eLjEM++rbOD4vWbYXRwaDiH
LetFJ5tGHDAIfL48NYpz2o3XZ3/O7WdTZphsAcvgPxTC+zU7WkbUl2SQlj0/qwsoD+qe9RYT
XhVXR7q7sjcGB4TpeqzRT7YKVLoVNq+bQw2lUX4W561gAYbZvVo/XByfDCoxmkxwuMlSmajj
Wy7b9TuT38t1HArv4m/LyVuBHiikX0/MUNBeSSIiKDvTL6NdHTjnZM6ptZvdvW3+ou6ET0pK
MGDpk/1NVuMnIHJESRg/SSFV6sElgq38k9wAT2oUqLcYvYI07nHmnuciaGygkCcGt+l2PvAa
j4mkQQvMU0cNRDBybk5aKi820oGIJjT7e+5RnD2mYZQdOAbQhDVCHvrfS1I60bsHT1MHqyAa
/qMLjKwBpKEd/w==
X-Mozilla-Draft-Info: internal/draft; vcard=0; receipt=0; DSN=0; uuencode=0;
attachmentreminder=0; deliveryformat=0
X-Identity-Key: id3
Fcc: imap://alice%40example.org@in.example.org/Sent
Subject: Hello!
Content-Type: multipart/encrypted;
protocol="application/pgp-encrypted";
boundary="------------HVIlnYsnz3YPWPVlor40isrE"
This is an OpenPGP/MIME encrypted message (RFC 4880 and 3156)
--------------HVIlnYsnz3YPWPVlor40isrE
Content-Type: application/pgp-encrypted
Content-Description: PGP/MIME version identification
Version: 1
--------------HVIlnYsnz3YPWPVlor40isrE
Content-Type: application/octet-stream; name="encrypted.asc"
Content-Description: OpenPGP encrypted message
Content-Disposition: inline; filename="encrypted.asc"
-----BEGIN PGP MESSAGE-----
wcDMA/K57StIWPW6AQv/cTRuyZRuX+Qkmxcbo34Df32PL2O3LSuwChdd4pd+WRNU5r08E0eLeFGv
k/C/2iFrNhgmvsJEnDtTZVc7+qykMeXwwC1f2qT7OAvTbAF9Me1dBCQy8QAPSzOowNu8434qOPwz
fZkoB4pdkKlAhGMUBdtbWYg3EDdpBUCpAuT8l3qhlfdSTRh+da2UVh9kix3SXSmLarMZGlJikh3T
pRLAlPOtRa7k5R4rc/OCD9HUL9EHGUk4rV86+mr4Wpp1aIhdTNksfrFXvgRFx0dcU33uS74X/zme
c1AOuJK8ed3F2Zc5IYTyE+ps1jBqVH1+CM697d5FnIMx96VyB4txDBBBjyq+oh+SY+xUxvtR/EzN
mR0qfiSCwOKwGdzPWDkBoCePYnykiR4shVdVV3qjYZwBmY5IWNahyBtIB470V4wMiJXdZFMQu5XD
NO4DJ6/x2UCcUiUokAwvKTSHflAHocZJ2ICfzhlJjJSiqEttwyG0xU0ZhTdZIbwzRWzYzDgMakFO
wcBMA+PY3JvEjuMiAQf+MXJB44iQ/Ti9WDH3MOX0N+X0APf5mzeqU9MpSZ+7mAnTRVmz/iMD989J
ngxu1mv3vQBjdNokIAfBYk29qyRBULXaof+6x5VJSWoopV6t1vNd+DB2HgLkbdCJuzikapCE+QAm
gmgoknQap+3l4D1RkMys7w1awsMYK0wR1Iucjb8M2I4f5ObPSMS5211ZBJuYOf1OQt0jX1jCNTOi
Q5tbufJ6EjAvP6XOYTY3in8+p7yocBgQhXaK8NB8jdg+h6IE/NNX1W5v1tMx17WRtIQGwh4cOlB6
Fsu87eMx8x9Ew2YdpN6yvIHddy9M9k3NCROT+5rIq7+1GTy0WoI/KxNwtdLBEwE2h7VFmJ5qEWhi
lmunWBIlA71IdNqpi+9lbg/QPwCgvRow05Gv0FyKbSvDA/fN18+CLuz8RicNFRbiPgwxMLRE3lZB
jj7BEDa53fjnjBJG4lA9mUaB/ALScQwkGXqKPpeDN4Iexy8eBsZoBczJcPna7GNgSYiXxbWo2P8+
T71oSht+igsyi8gYDNwmhcsQxSWtF7f53irOKWgEpgz2hMjWi764GGBzAJ231suyV2eZXVJm2Im7
RUDu3bJNu/Q/CvxQQdcUXHEoHrTzDZ3KQPW0/oYln2WZwkRQcVVGSLVINgVTdTkI8GEuXH/4yjYF
tUq68esMQ2WOWZ7IiTNPU9T2F5kEwR2sf55XgLckj0OLJn5CkSAWBJhHdL/9er06u8ksTs6V9UEo
i+15XUAiNzsuIcydoGF9sSuO9nm62xYO8uY8yQl/z2Q7CXKefkXBSeaNcYyQKrCnLfwnuvR2WFE5
OJ0HM5i/qcWmqWSvSYzdDqUbI95q9Fgn78F/bcJ9ZTdPDkWLi//SUzcWeUWgRC57U4CFTApj74EJ
lT/gVlH/y88RNcvGGAPHHS11adLmln7CbLEF9tmRM3ou78nMm7VhXT8C/QZiTvJgG3yZbeRFlC39
=hdiO
-----END PGP MESSAGE-----
--------------HVIlnYsnz3YPWPVlor40isrE--

View File

@@ -0,0 +1,66 @@
Return-Path: <37u6sZRoKALMghkXier-lfmi-mel-kXihkmbgZZhhZeX.Vhf@smtp-tls-reporting.bounces.google.com>
Delivered-To: root@nine.testrun.org
Received: from nine.testrun.org
by nine with LMTP
id UB0TOe/urGX8mhwAPdT8mA
(envelope-from <37u6sZRoKALMghkXier-lfmi-mel-kXihkmbgZZhhZeX.Vhf@smtp-tls-reporting.bounces.google.com>)
for <root@nine.testrun.org>; Sun, 21 Jan 2024 11:16:15 +0100
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=google.com; s=20230601; t=1705832174; x=1706436974; darn=nine.testrun.org;
h=to:from:subject:message-id:tls-report-submitter:tls-report-domain
:date:mime-version:from:to:cc:subject:date:message-id:reply-to;
bh=ij33gq0ofQz4EhX9TLi6EGnOILUSDSZtwZir1iybY6o=;
b=IQ/TiJ4wMppgECWZQaC8EE3Q3ON4VjB94gp/l4uxL0mSAKo+CeKn+wh5jJooKCN3uZ
wjp9w3+fcQq/3UvvthDzlcBHA2QHXkGC4ONliODX4uaWalRkc21ODHVvx8ILGuAFeKxw
dl6+hn3Qk56FbVdRNBrAKvx7YvJjQHecrO79AoURhcCVfNtqpwPXuok4c/w6TtLhLLsu
pwcPlkAJphdD5hvXLciHjRszvIWOYu9v2G0c4bcCXBRrXhNnIPLl+SO5FAkkyxNOgVmg
EYyBJJoUZH9njyHeazbDnlMCQ5aQBndrfnk357/jSR/Fx6Nti1SvguLUOZbVV2vhh4oD
R6bA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20230601; t=1705832174; x=1706436974;
h=to:from:subject:message-id:tls-report-submitter:tls-report-domain
:date:mime-version:x-gm-message-state:from:to:cc:subject:date
:message-id:reply-to;
bh=ij33gq0ofQz4EhX9TLi6EGnOILUSDSZtwZir1iybY6o=;
b=eYcl9flKTT6YIUTx01fK86e390qJCgQmR9RSDbCGiKTTCzJ1NQn1ev7pUhzOoQL44z
w27ZOgAeiz8eKHxXMQ0DQhjyQ3anHEqej4NJPJ5+7epL8eZQ7QDs2/EQmqJNe9DP4Bd7
sjq2QyUdi2UbU9OrxBL4mRKu8PRZyR4/0cgaJJIgphziUHZBRbfEksl8Ev5XBDMBy11x
1oZZSOmkqK2ujPZZiQrdbqOxWijd4bCpBj5gWH/M9jRI/gHCiIwF+ZaxIXQBoVIhvBKK
t0tADdYcd3qjN2gxr7PO04NaABJgxLGC9YFXH+jRPKdycAvRKwZYpRXuHA2bynvYryk8
MDJg==
X-Gm-Message-State: AOJu0YyM8qPQlVoa6jtAlpPtrku3niz3QTG1nLKht5uRJsjZg5pzwktC
kh/X3YhSj2u0uIzVNqlLH0Lo/XiBUTJbicLQrpfIKD5aeVC8LPhrWBE4mJ4eZ5mYtTLSmgbu3fr
mM4hyzb7+m+pqL0bi2IZTUQh4wbDop+p2LLA99g4Ezji4hkTbMzXy07ekctK/9/bcSBS2
X-Google-Smtp-Source: AGHT+IEPBnMUwx3a4EQI3kJIaLlQcaDz6nx+VMmBsWiYDbqDBgNl26HDryxj5uANI/wyiLBwSQcwyTfmJewf/23eykdQ83fh4kERed6SIB4=
MIME-Version: 1.0
X-Received: by 2002:a05:622a:124f:b0:42a:128:c13c with SMTP id
z15-20020a05622a124f00b0042a0128c13cmr470166qtx.12.1705832174325; Sun, 21 Jan
2024 02:16:14 -0800 (PST)
Date: Sun, 21 Jan 2024 02:16:14 -0800
TLS-Report-Domain: nine.testrun.org
TLS-Report-Submitter: google.com
Message-ID: <000000000000cc154d060f720057@google.com>
Subject: Report Domain: nine.testrun.org Submitter: google.com Report-ID: <2024.01.20T00.00.00Z+nine.testrun.org@google.com>
From: noreply-smtp-tls-reporting@google.com
To: root@nine.testrun.org
Content-Type: multipart/report; boundary="000000000000cc153d060f720056"; report-type=tlsrpt
--000000000000cc153d060f720056
Content-Type: text/plain; charset="UTF-8"; format=flowed; delsp=yes
This is an aggregate TLS report from google.com
--000000000000cc153d060f720056
Content-Type: application/tlsrpt+gzip;
name="google.com!nine.testrun.org!1705708800!1705795199!001.json.gz"
Content-Disposition: attachment;
filename="google.com!nine.testrun.org!1705708800!1705795199!001.json.gz"
Content-Transfer-Encoding: base64
H4sIAAAAAAAAAHVRTWvDMAz9K8HnOrhZx6hPu5Wd21NHKcZRM0MsBVsp6Ur++5R022FrQWB9vKcn
yVdFqXEYPh0HQo0ugrJqQ9S0ULyhL9VC1Y5BJ4eNlK4qs0uspxyHGVyZaqXNUldmZ4ydbS8swPoB
qnqyz2uxvRoXyhOy86wDnkhgOXKnuc06QUeJAzavzTxM6SlK11tah/qB8BEDQsmQOfVYym7C6agN
PkBW9v16Cy7TIjdP86Wb5sucf6AXLWxRFrw6Q8pyGFtsd9vzUhCRarAFyLTJwxQPtrijGd1wdI0g
q9VyXRmjDr/Na4ouoEjeow36gzJPyv+qB7lW7mN0aR6fiV2rc+895HzqxZV3+kNPPUqHl8U35ORC
2yf4WzfjeBi/APFqfhD/AQAA
--000000000000cc153d060f720056--