Compare commits

..
Author SHA1 Message Date
link2xt 3f18a16a08 test: test rotation of encryption subkey 2024-09-16 21:15:49 +00:00
link2xt f7a705c6da refactor: use KeyPair::new() in create_keypair() 2024-09-16 20:51:16 +00:00
iequidoo f497e4dd12 docs: Why search_msgs() only looks at the first kilobytes of long messages 2024-09-16 17:14:51 -03:00
iequidoo 0a63083df7 fix: Shorten message text in locally sent messages too (#2281) 2024-09-16 17:14:51 -03:00
iequidoo 5a6efdff44 fix: Save QR code token regardless of whether the group exists (#5954)
Groups promotion to other devices and QR code tokens synchronisation are not synchronised processes,
so there are reasons why a QR code token may arrive earlier than the first group message:
- We are going to upload sync messages via IMAP while group messages are sent by SMTP.
- If sync messages go to the mvbox, they can be fetched earlier than group messages from Inbox.
2024-09-16 16:40:26 -03:00
link2xt 7efb5a269c docs(CONTRIBUTING.md): document how to format SQL statements 2024-09-16 18:11:42 +00:00
link2xt 1caf672904 test(test-data): remove public keys that can be derived from secret keys 2024-09-16 17:00:16 +00:00
link2xt 7743072411 refactor: remove addr from KeyPair 2024-09-16 17:00:16 +00:00
link2xt c461c4f02e refactor: do not store deprecated addr and is_default into keypairs 2024-09-16 17:00:16 +00:00
iequidoo 5b597f3a95 feat: Don't SMTP-send messages to self-chat if BccSelf is disabled
`chat::create_send_msg_jobs()` already handles `Config::BccSelf` as needed. The only exception is
Autocrypt setup messages. This change unifies the logic for the self-chat and groups only containing
`SELF`.
2024-09-15 23:48:06 -03:00
iequidoo b69488685f feat: Make resending OutPending messages possible (#5817)
This makes possible to schedule one more sending of the message, the existing jobs are not
cancelled. Otherwise it's complicated to implement bots that resend messages when a new member joins
the group.
2024-09-15 16:27:39 -03:00
link2xt afb01e3e90 chore: update provider database
This change removes OAuth2 for Gmail
as Delta Chat does not have a working
client ID anymore.
Tests are adjusted to test against Yandex
and MX queries for OAuth2 are always disabled
because they were only used to detect Google Workspace.
2024-09-13 17:58:25 +00:00
link2xt 7ff14dc26b feat: log unexpected message state when resending fails 2024-09-12 05:06:05 +00:00
link2xt 0c33064193 chore(release): prepare for 1.143.0 2024-09-12 01:52:14 +00:00
link2xt 61d77584e8 chore(cargo): update typescript-type-def to 0.5.12
This removes unmaintained proc-macro-error dependency.
2024-09-12 01:35:43 +00:00
link2xt 37ca9d7319 feat: shadowsocks support
This change introduces new config options
`proxy_enabled` and `proxy_url`
that replace `socks5_*`.

Tested with deltachat-repl
by starting it with
`cargo run --locked -p deltachat-repl -- deltachat-db` and running
```
> set proxy_enabled 1
> set proxy_url ss://...
> setqr dcaccount:https://chatmail.example.org/new
> configure
```
2024-09-12 00:22:09 +00:00
iequidoo 2c136f6355 refactor: get_config_bool_opt(): Return None if only default value exists
And also:
- Make it `pub(crate)`.
- Use it in `should_request_mdns()` as using `config_exists()` there isn't correct because the
  latter doesn't look at environment.
2024-09-10 18:10:59 -03:00
iequidoo 52dcc7e350 refactor: Make Context::config_exists() crate-public 2024-09-10 18:10:59 -03:00
iequidooandHocuri ff6488371c feat: Delete messages from a chatmail server immediately by default (#5805) (#5840)
I.e. treat `DeleteServerAfter == None` as "delete at once". But when a backup is exported, set
`DeleteServerAfter` to 0 so that the server decides when to delete messages, in order not to break
the multi-device case. Even if a backup is not aimed for deploying more devices, `DeleteServerAfter`
must be set to 0, otherwise the backup is half-useful because after a restoration the user wouldn't
see new messages deleted by the device after the backup was done. But if the user explicitly set
`DeleteServerAfter`, don't change it when exporting a backup. Anyway even for non-chatmail case the
app should warn the user before a backup export if they have `DeleteServerAfter` enabled.

Also do the same after a backup import. While this isn't reliable as we can crash in between, this
is a problem only for old backups, new backups already have `DeleteServerAfter` set if necessary.

---------

Co-authored-by: Hocuri <hocuri@gmx.de>
2024-09-08 16:53:56 -03:00
link2xt 0782b5abdd ci: update Rust to 1.81.0 2024-09-08 07:08:12 +00:00
link2xt 2e2ba96d75 chore(cargo-deny): silence unmaintained proc-macro-error warning 2024-09-08 06:35:01 +00:00
iequidoo 853e38e054 feat: ChatId::create_for_contact_with_blocked: Don't emit events on no op 2024-09-05 10:46:30 -03:00
iequidoo 418dfbf994 fix: Don't sync QR code token before populating the group (#5935)
Otherwise other devices don't yet know about the group and can't handle the sync message correctly.
2024-09-04 15:18:26 -03:00
dependabot[bot] 533a872118 Merge pull request #5950 from deltachat/dependabot/cargo/quinn-proto-0.11.8 2024-09-04 00:04:55 +00:00
dependabot[bot] 2ae854e8ea chore(cargo): bump quinn-proto from 0.11.3 to 0.11.8
Bumps [quinn-proto](https://github.com/quinn-rs/quinn) from 0.11.3 to 0.11.8.
- [Release notes](https://github.com/quinn-rs/quinn/releases)
- [Commits](https://github.com/quinn-rs/quinn/compare/quinn-proto-0.11.3...quinn-proto-0.11.8)

---
updated-dependencies:
- dependency-name: quinn-proto
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-09-03 20:50:28 +00:00
link2xt 3969383857 Merge tag 'v1.142.12' 2024-09-02 23:58:26 +00:00
iequidoo c257482838 fix: Display Config::MdnsEnabled as true by default (#5948) 2024-09-02 20:50:15 -03:00
link2xt 0a46e64971 fix: use default server list for providers that don't have one
There are providers in the provider database
that do not have servers specified.
For such providers default list should be tried
just like when configuring unknown providers.
2024-09-02 22:57:31 +00:00
iequidoo 845420cf17 test: Alice is (non-)bot on Bob's side after QR contact setup 2024-09-02 18:06:52 -03:00
dependabot[bot] 96ea0db88e Merge pull request #5945 from deltachat/dependabot/cargo/libc-0.2.158 2024-09-01 23:21:12 +00:00
dependabot[bot] d99c735e12 Merge pull request #5944 from deltachat/dependabot/cargo/async-imap-0.10.0 2024-09-01 22:40:49 +00:00
dependabot[bot] d48f4100e9 Merge pull request #5943 from deltachat/dependabot/cargo/bytes-1.7.1 2024-09-01 22:39:57 +00:00
dependabot[bot] 7e73d5fdac chore(cargo): bump serde_json from 1.0.122 to 1.0.127
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.122 to 1.0.127.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.122...1.0.127)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-09-01 19:27:16 -03:00
dependabot[bot] 152cdfe9bc chore(cargo): bump syn from 2.0.72 to 2.0.77
Bumps [syn](https://github.com/dtolnay/syn) from 2.0.72 to 2.0.77.
- [Release notes](https://github.com/dtolnay/syn/releases)
- [Commits](https://github.com/dtolnay/syn/compare/2.0.72...2.0.77)

---
updated-dependencies:
- dependency-name: syn
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-09-01 19:16:07 -03:00
dependabot[bot] a9eedafbcb chore(cargo): bump serde from 1.0.205 to 1.0.209
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.205 to 1.0.209.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.205...v1.0.209)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-09-01 19:10:15 -03:00
dependabot[bot] 5baf191483 chore(cargo): bump quote from 1.0.36 to 1.0.37
Bumps [quote](https://github.com/dtolnay/quote) from 1.0.36 to 1.0.37.
- [Release notes](https://github.com/dtolnay/quote/releases)
- [Commits](https://github.com/dtolnay/quote/compare/1.0.36...1.0.37)

---
updated-dependencies:
- dependency-name: quote
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-09-01 19:02:34 -03:00
dependabot[bot] 2d2e703884 chore(cargo): bump libc from 0.2.155 to 0.2.158
Bumps [libc](https://github.com/rust-lang/libc) from 0.2.155 to 0.2.158.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Changelog](https://github.com/rust-lang/libc/blob/0.2.158/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.155...0.2.158)

---
updated-dependencies:
- dependency-name: libc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-09-01 21:09:15 +00:00
dependabot[bot] 026450ddf3 chore(cargo): bump async-imap from 0.9.7 to 0.10.0
Bumps [async-imap](https://github.com/async-email/async-imap) from 0.9.7 to 0.10.0.
- [Changelog](https://github.com/async-email/async-imap/blob/main/CHANGELOG.md)
- [Commits](https://github.com/async-email/async-imap/compare/v0.9.7...v0.10.0)

---
updated-dependencies:
- dependency-name: async-imap
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-09-01 21:09:01 +00:00
dependabot[bot] 5646782d23 chore(cargo): bump bytes from 1.5.0 to 1.7.1
Bumps [bytes](https://github.com/tokio-rs/bytes) from 1.5.0 to 1.7.1.
- [Release notes](https://github.com/tokio-rs/bytes/releases)
- [Changelog](https://github.com/tokio-rs/bytes/blob/master/CHANGELOG.md)
- [Commits](https://github.com/tokio-rs/bytes/compare/v1.5.0...v1.7.1)

---
updated-dependencies:
- dependency-name: bytes
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-09-01 21:08:44 +00:00
link2xt dd1c2e836b feat(securejoin): ignore invalid *-request-with-auth messages silently 2024-09-01 16:10:41 +00:00
link2xt be73076e9e chore(cargo): replace unmaintained ansi_term with nu-ansi-term 2024-09-01 16:10:30 +00:00
link2xt 9d47be0d8a Merge tag 'v1.142.11' 2024-08-30 22:38:15 +00:00
link2xt 93e181b2da docs: document that bcc_self is enabled by default
bcc_self has been enabled by default
since core version 1.95.0
by merging
PR <https://github.com/deltachat/deltachat-core-rust/pull/3612>.

However deltachat.h documentation
still incorrectly said that bcc_self is disabled by default.
2024-08-30 19:21:01 +00:00
link2xt 3867808927 chore(cargo): reduce number of duplicate dependencies 2024-08-30 04:24:31 +00:00
link2xt c7c3b9ca90 feat: replace reqwest with hyper
This change replaces
usage of `reqwest` and `hyper-util`
with custom connection establishment code
so it is done in the same way
as for IMAP and SMTP connections.
This way we control HTTP, IMAP and SMTP
connection establishment
and schedule connection attempts
to resolved IP addresses
in the same way for all 3 protocols.
2024-08-29 23:10:17 +00:00
link2xt 54cfc21e28 Remove old iroh 0.4 2024-08-29 20:59:41 +00:00
link2xt f01514dba4 fix: start new connections independently of connection failures
With current implementation
every time connection fails
we take the next delay from `delays` iterator.
In the worst case first 4 DNS results
immediately refuse connection
and we start fifth connection attempt
with 1 year timeout,
effectively continuing all remaining
connection attempts without concurrency.

With new implementation
new connection attempts are
added to `connection_attempt_set`
independently of connection failures
and after 10 seconds
we always end up with five
parallel connection attempts
as long as there are enough IP addresses.
2024-08-29 13:24:28 +00:00
link2xt ee5723416e chore(cargo): update iroh to 0.23.0 2024-08-28 23:15:38 +00:00
link2xtandHocuri aab8ef2726 feat: parallelize IMAP and SMTP connection attempts (#5915)
Previously for each connection candidate (essentially host and port
pair) after resolving the host to a list of IPs Delta Chat iterated IP
addresses one by one. Now for IMAP and SMTP we try up to 5 IP addresses
in parallel. We start with one connection and add more connections
later. If some connection fails, e.g. we try to connect to IPv6 on IPv4
network and get "Network is unreachable" (ENETUNREACH) error, we replace
failed connection with another one immediately.

Co-authored-by: Hocuri <hocuri@gmx.de>
2024-08-28 22:00:07 +00:00
link2xt 84c1ffd7cc fix: do not allow quotes with "... wrote:" headers in chat messages 2024-08-28 16:05:03 +00:00
link2xt 273158a337 fix: add Auto-Submitted header in a single place
This ensures we don't add multiple Auto-Submitted headers
when bots send vg-request or vc-request messages.

The change fixes failing
receive_imf::tests::test_bot_accepts_another_group_after_qr_scan
test.
2024-08-27 18:31:07 +00:00
link2xt 099f0e2d18 Merge tag 'v1.142.10' 2024-08-26 18:54:27 +00:00
iequidoo af77c0c987 feat: Add "Auto-Submitted: auto-replied" header to appropriate SecureJoin messages
I.e. to all messages except "v{c,g}-request" as they sent out on a QR code scanning which is a
manual action and "vg-member-added" as formally this message is auto-submitted, but the member
addition is a result of an explicit user action. Otherwise it would be strange to have the
Auto-Submitted header in "member-added" messages of verified groups only.
2024-08-25 16:19:41 -03:00
link2xt f912bc78e6 fix(http): set I/O timeout to 1 minute rather than whole request timeout
Before the fix HTTP client
had no connection timeout,
so it only had a chance
to test one IPv6 and one IPv4
address if the first addresses timed out.
Now it can test at least 4 addresses
of each family and more if some addresses
refuse connection rather than time out.
2024-08-25 17:06:34 +00:00
link2xt 137ee9334c feat: always use preloaded DNS results
Otherwise if DNS server returns incorrect results,
we may never try preloaded DNS results.
For example, we may get our first results
from a captive portal.

To test, add `127.0.0.1 example.org`
and try to create an account.
Without this change we only try 127.0.0.1 and fail.
With this change preloaded DNS results are tried as well.
2024-08-25 15:33:18 +00:00
link2xt 36e5e964e5 Merge tag 'v1.142.9' 2024-08-24 21:43:43 +00:00
link2xt ef12a76a9e chore: update provider database 2024-08-23 13:24:07 +00:00
bjoernandiequidoo 6b3de9d7da recognize t.me proxy qr codes (#5895)
this PR adds the type DC_QR_SOCKS5_PROXY to `dc_check_qr()` for
**supporting telegram proxy QR codes**. if returned, the UI should ask
the user if they want to us the proxy and call
`dc_set_config_from_qr();` afterwards (plus maybe `dc_configure()`).

idea is to improve our proxy story, follow ups may be:

- in UI, - move proxy out of "Account & Password", as a **separate
"Proxy Activity"** (it should stay in "Advanced" for now, however, below
"Server", which might be moved up)

- allow **opening the "Proxy Activity" from the welcome screens**
three-dot-menu (that would also solve a long standing issue that
entering the email address bypasses the proxy

- show proxy usage in the "Connectivity View" and/or add an **icon** to
the main chatlist screen (beside three-dot menu) in case some proxy is
in use; tapping this icon will open the "Proxy Activity"

- the the new "Proxy Activity", add a **share / show proxy QR code**
button. that would generate invite links in the form
`https://i.delta.chat/socks#...` - so that tapping then opens the app.
support for these links need to be added to core then.

- handle a list of proxies in core, offer selection in UI. the list
could be one for all profiles and could be filled eg. by normal invite
links or other channels

---------

Co-authored-by: iequidoo <117991069+iequidoo@users.noreply.github.com>
2024-08-23 09:49:49 +02:00
link2xt 3599e4be16 fix: save custom username if user entered it 2024-08-23 05:44:28 +00:00
link2xt 8dc844e194 refactor(login_param): use Config:: constants to avoid typos in key names 2024-08-22 00:44:06 +00:00
link2xt 104c60840a test: test that alternative port 443 works 2024-08-22 00:14:57 +00:00
link2xt f2cb098148 fix: make SMTP server list readable in context info 2024-08-21 13:04:37 +00:00
link2xt 30b998eca3 Merge tag 'v1.142.8' 2024-08-21 12:48:17 +00:00
link2xt b5133fe8c8 fix: fix loading imap_certificate_checks
Fix a typo in the config name
(by using `Config::` to avoid it)
and make sure we don't panic on unknown values.

Also test that we don't panic on unknown
`configured_imap_certificate_checks` values.
2024-08-21 12:46:12 +00:00
link2xt 08ec133aac refactor: use get_configured_provider() in ConfiguredLoginParam::load() 2024-08-21 07:24:15 +00:00
link2xt 7d7391887a fix: do not ignore legacy configured_{mail,send}_user for known providers 2024-08-21 07:24:15 +00:00
link2xt e7d4ccffe2 feat: automatic reconfiguration 2024-08-19 16:36:56 +00:00
79 changed files with 3994 additions and 3325 deletions
+4 -4
View File
@@ -24,7 +24,7 @@ jobs:
name: Lint Rust
runs-on: ubuntu-latest
env:
RUSTUP_TOOLCHAIN: 1.80.1
RUSTUP_TOOLCHAIN: 1.81.0
steps:
- uses: actions/checkout@v4
with:
@@ -95,11 +95,11 @@ jobs:
matrix:
include:
- os: ubuntu-latest
rust: 1.80.1
rust: 1.81.0
- os: windows-latest
rust: 1.80.1
rust: 1.81.0
- os: macos-latest
rust: 1.80.1
rust: 1.81.0
# Minimum Supported Rust Version = 1.77.0
- os: ubuntu-latest
+50
View File
@@ -1,5 +1,54 @@
# Changelog
## [1.143.0] - 2024-09-12
### Features / Changes
- Automatic reconfiguration, e.g. switching to implicit TLS if STARTTLS port stops working.
- Always use preloaded DNS results.
- Add "Auto-Submitted: auto-replied" header to appropriate SecureJoin messages.
- Parallelize IMAP and SMTP connection attempts ([#5915](https://github.com/deltachat/deltachat-core-rust/pull/5915)).
- securejoin: Ignore invalid *-request-with-auth messages silently.
- ChatId::create_for_contact_with_blocked: Don't emit events on no op.
- Delete messages from a chatmail server immediately by default ([#5805](https://github.com/deltachat/deltachat-core-rust/pull/5805)) ([#5840](https://github.com/deltachat/deltachat-core-rust/pull/5840)).
- Shadowsocks support.
- Recognize t.me SOCKS5 proxy QR codes ([#5895](https://github.com/deltachat/deltachat-core-rust/pull/5895))
- Remove old iroh 0.4 and support for old `DCBACKUP` QR codes.
### Fixes
- http: Set I/O timeout to 1 minute rather than whole request timeout.
- Add Auto-Submitted header in a single place.
- Do not allow quotes with "... wrote:" headers in chat messages.
- Don't sync QR code token before populating the group ([#5935](https://github.com/deltachat/deltachat-core-rust/pull/5935)).
### Documentation
- Document that `bcc_self` is enabled by default.
### CI
- Update Rust to 1.81.0.
### Miscellaneous Tasks
- Update provider database.
- cargo: Update iroh to 0.23.0.
- cargo: Reduce number of duplicate dependencies.
- cargo: Replace unmaintained ansi_term with nu-ansi-term.
- Replace `reqwest` with direct usage of `hyper`.
### Refactor
- login_param: Use Config:: constants to avoid typos in key names.
- Make Context::config_exists() crate-public.
- Get_config_bool_opt(): Return None if only default value exists.
### Tests
- Test that alternative port 443 works.
- Alice is (non-)bot on Bob's side after QR contact setup.
## [1.142.12] - 2024-09-02
### Fixes
@@ -4806,3 +4855,4 @@ https://github.com/deltachat/deltachat-core-rust/pulls?q=is%3Apr+is%3Aclosed
[1.142.10]: https://github.com/deltachat/deltachat-core-rust/compare/v1.142.9..v1.142.10
[1.142.11]: https://github.com/deltachat/deltachat-core-rust/compare/v1.142.10..v1.142.11
[1.142.12]: https://github.com/deltachat/deltachat-core-rust/compare/v1.142.11..v1.142.12
[1.143.0]: https://github.com/deltachat/deltachat-core-rust/compare/v1.142.12..v1.143.0
+35
View File
@@ -32,6 +32,41 @@ on the contributing page: <https://github.com/deltachat/deltachat-core-rust/cont
We format the code using `rustfmt`. Run `cargo fmt` prior to committing the code.
Run `scripts/clippy.sh` to check the code for common mistakes with [Clippy].
Multi-line SQL statements should be formatted using string literals,
for example
```
sql.execute(
"CREATE TABLE messages (
id INTEGER PRIMARY KEY AUTOINCREMENT,
text TEXT DEFAULT '' NOT NULL -- message text
) STRICT",
)
.await?;
```
Do not use macros like [`concat!`](https://doc.rust-lang.org/std/macro.concat.html)
or [`indoc!](https://docs.rs/indoc).
Do not escape newlines like this:
```
sql.execute(
"CREATE TABLE messages ( \
id INTEGER PRIMARY KEY AUTOINCREMENT, \
text TEXT DEFAULT '' NOT NULL \
) STRICT",
)
.await?;
```
Escaping newlines
is prone to errors like this if space before backslash is missing:
```
"SELECT foo\
FROM bar"
```
Literal above results in `SELECT fooFROM bar` string.
This style also does not allow using `--` comments.
### Commit messages
Commit messages follow the [Conventional Commits] notation.
We use [git-cliff] to generate the changelog from commit messages before the release.
Generated
+458 -1168
View File
File diff suppressed because it is too large Load Diff
+14 -10
View File
@@ -1,6 +1,6 @@
[package]
name = "deltachat"
version = "1.142.12"
version = "1.143.0"
edition = "2021"
license = "MPL-2.0"
rust-version = "1.77"
@@ -41,12 +41,13 @@ ratelimit = { path = "./deltachat-ratelimit" }
anyhow = { workspace = true }
async-broadcast = "0.7.1"
async-channel = { workspace = true }
async-imap = { version = "0.9.7", default-features = false, features = ["runtime-tokio"] }
async-imap = { version = "0.10.0", default-features = false, features = ["runtime-tokio"] }
async-native-tls = { version = "0.5", default-features = false, features = ["runtime-tokio"] }
async-smtp = { version = "0.9", default-features = false, features = ["runtime-tokio"] }
async_zip = { version = "0.0.12", default-features = false, features = ["deflate", "fs"] }
base64 = { workspace = true }
brotli = { version = "6", default-features=false, features = ["std"] }
bytes = "1"
chrono = { workspace = true, features = ["alloc", "clock", "std"] }
email = { git = "https://github.com/deltachat/rust-email", branch = "master" }
encoded-words = { git = "https://github.com/async-email/encoded-words", branch = "master" }
@@ -57,11 +58,13 @@ futures = { workspace = true }
futures-lite = { workspace = true }
hex = "0.4.0"
hickory-resolver = "0.24"
http-body-util = "0.1.2"
humansize = "2"
hyper = "1"
hyper-util = "0.1.7"
image = { version = "0.25.1", default-features=false, features = ["gif", "jpeg", "ico", "png", "pnm", "webp", "bmp"] }
iroh_old = { version = "0.4.2", default-features = false, package = "iroh"}
iroh-net = { version = "0.22.0", default-features = false }
iroh-gossip = { version = "0.22.0", default-features = false, features = ["net"] }
iroh-net = { version = "0.23.0", default-features = false }
iroh-gossip = { version = "0.23.0", default-features = false, features = ["net"] }
kamadak-exif = "0.5.3"
lettre_email = { git = "https://github.com/deltachat/lettre", branch = "master" }
libc = { workspace = true }
@@ -74,18 +77,20 @@ once_cell = { workspace = true }
percent-encoding = "2.3"
parking_lot = "0.12"
pgp = { version = "0.13.2", default-features = false }
pin-project = "1"
qrcodegen = "1.7.0"
quick-xml = "0.36"
quoted_printable = "0.5"
rand = { workspace = true }
regex = { workspace = true }
reqwest = { version = "0.12.5", features = ["json"] }
rusqlite = { workspace = true, features = ["sqlcipher"] }
rust-hsluv = "0.1"
sanitize-filename = { workspace = true }
serde_json = { workspace = true }
serde_urlencoded = "0.7.1"
serde = { workspace = true, features = ["derive"] }
sha-1 = "0.10"
shadowsocks = { version = "1.20.2", default-features = false, features = ["aead-cipher-2022"] }
smallvec = "1.13.2"
strum = "0.26"
strum_macros = "0.26"
@@ -102,11 +107,11 @@ url = "2"
uuid = { version = "1", features = ["serde", "v4"] }
[dev-dependencies]
ansi_term = { workspace = true }
anyhow = { workspace = true, features = ["backtrace"] } # Enable `backtrace` feature in tests.
criterion = { version = "0.5.1", features = ["async_tokio"] }
futures-lite = { workspace = true }
log = { workspace = true }
nu-ansi-term = { workspace = true }
proptest = { version = "1", default-features = false, features = ["std"] }
tempfile = { workspace = true }
testdir = "0.9.0"
@@ -156,7 +161,6 @@ harness = false
[workspace.dependencies]
anyhow = "1"
ansi_term = "0.12.1"
async-channel = "2.3.1"
base64 = "0.22"
chrono = { version = "0.4.38", default-features = false }
@@ -167,6 +171,7 @@ futures = "0.3.30"
futures-lite = "2.3.0"
libc = "0.2"
log = "0.4"
nu-ansi-term = "0.46"
num-traits = "0.2"
once_cell = "1.18.0"
rand = "0.8"
@@ -194,8 +199,7 @@ default = ["vendored"]
internals = []
vendored = [
"async-native-tls/vendored",
"rusqlite/bundled-sqlcipher-vendored-openssl",
"reqwest/native-tls-vendored"
"rusqlite/bundled-sqlcipher-vendored-openssl"
]
[lints.rust]
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "deltachat_ffi"
version = "1.142.12"
version = "1.143.0"
description = "Deltachat FFI"
edition = "2018"
readme = "README.md"
+10 -11
View File
@@ -403,11 +403,8 @@ char* dc_get_blobdir (const dc_context_t* context);
* - `send_port` = SMTP-port, guessed if left out
* - `send_security`= SMTP-socket, one of @ref DC_SOCKET, defaults to #DC_SOCKET_AUTO
* - `server_flags` = IMAP-/SMTP-flags as a combination of @ref DC_LP flags, guessed if left out
* - `socks5_enabled` = SOCKS5 enabled
* - `socks5_host` = SOCKS5 proxy server host
* - `socks5_port` = SOCKS5 proxy server port
* - `socks5_user` = SOCKS5 proxy username
* - `socks5_password` = SOCKS5 proxy password
* - `proxy_enabled` = Proxy enabled. Disabled by default.
* - `proxy_url` = Proxy URL. May contain multiple URLs separated by newline, but only the first one is used.
* - `imap_certificate_checks` = how to check IMAP certificates, one of the @ref DC_CERTCK flags, defaults to #DC_CERTCK_AUTO (0)
* - `smtp_certificate_checks` = deprecated option, should be set to the same value as `imap_certificate_checks` but ignored by the new core
* - `displayname` = Own name to use when sending messages. MUAs are allowed to spread this way e.g. using CC, defaults to empty
@@ -422,8 +419,8 @@ char* dc_get_blobdir (const dc_context_t* context);
* - `mdns_enabled` = 0=do not send or request read receipts,
* 1=send and request read receipts
* default=send and request read receipts, only send but not reuqest if `bot` is set
* - `bcc_self` = 0=do not send a copy of outgoing messages to self (default),
* 1=send a copy of outgoing messages to self.
* - `bcc_self` = 0=do not send a copy of outgoing messages to self,
* 1=send a copy of outgoing messages to self (default).
* Sending messages to self is needed for a proper multi-account setup,
* however, on the other hand, may lead to unwanted notifications in non-delta clients.
* - `sentbox_watch`= 1=watch `Sent`-folder for changes,
@@ -867,13 +864,10 @@ void dc_maybe_network (dc_context_t* context);
*
* @memberof dc_context_t
* @param context The context as created by dc_context_new().
* @param addr The e-mail address of the user. This must match the
* configured_addr setting of the context as well as the UID of the key.
* @param public_data Ignored, actual public key is extracted from secret_data.
* @param secret_data ASCII armored secret key.
* @return 1 on success, 0 on failure.
*/
int dc_preconfigure_keypair (dc_context_t* context, const char *addr, const char *public_data, const char *secret_data);
int dc_preconfigure_keypair (dc_context_t* context, const char *secret_data);
// handle chatlists
@@ -2507,6 +2501,7 @@ void dc_stop_ongoing_process (dc_context_t* context);
#define DC_QR_BACKUP 251
#define DC_QR_BACKUP2 252
#define DC_QR_WEBRTC_INSTANCE 260 // text1=domain, text2=instance pattern
#define DC_QR_SOCKS5_PROXY 270 // text1=host, text2=port
#define DC_QR_ADDR 320 // id=contact
#define DC_QR_TEXT 330 // text1=text
#define DC_QR_URL 332 // text1=URL
@@ -2560,6 +2555,10 @@ void dc_stop_ongoing_process (dc_context_t* context);
* ask the user if they want to use the given service for video chats;
* if so, call dc_set_config_from_qr().
*
* - DC_QR_SOCKS5_PROXY with dc_lot_t::text1=host, dc_lot_t::text2=port:
* ask the user if they want to use the given proxy and overwrite the previous one, if any.
* if so, call dc_set_config_from_qr() and restart I/O.
*
* - DC_QR_ADDR with dc_lot_t::id=Contact ID:
* e-mail address scanned, optionally, a draft message could be set in
* dc_lot_t::text1 in which case dc_lot_t::text1_meaning will be DC_TEXT1_DRAFT;
+7 -13
View File
@@ -835,8 +835,6 @@ pub unsafe extern "C" fn dc_maybe_network(context: *mut dc_context_t) {
#[no_mangle]
pub unsafe extern "C" fn dc_preconfigure_keypair(
context: *mut dc_context_t,
addr: *const libc::c_char,
_public_data: *const libc::c_char,
secret_data: *const libc::c_char,
) -> i32 {
if context.is_null() {
@@ -844,9 +842,8 @@ pub unsafe extern "C" fn dc_preconfigure_keypair(
return 0;
}
let ctx = &*context;
let addr = to_string_lossy(addr);
let secret_data = to_string_lossy(secret_data);
block_on(preconfigure_keypair(ctx, &addr, &secret_data))
block_on(preconfigure_keypair(ctx, &secret_data))
.context("Failed to save keypair")
.log_err(ctx)
.is_ok() as libc::c_int
@@ -4537,19 +4534,16 @@ pub unsafe extern "C" fn dc_provider_new_from_email_with_dns(
let addr = to_string_lossy(addr);
let ctx = &*context;
let socks5_enabled = block_on(async move {
ctx.get_config_bool(config::Config::Socks5Enabled)
.await
.context("Can't get config")
.log_err(ctx)
});
let proxy_enabled = block_on(ctx.get_config_bool(config::Config::ProxyEnabled))
.context("Can't get config")
.log_err(ctx);
match socks5_enabled {
Ok(socks5_enabled) => {
match proxy_enabled {
Ok(proxy_enabled) => {
match block_on(provider::get_provider_info_by_addr(
ctx,
addr.as_str(),
socks5_enabled,
proxy_enabled,
))
.log_err(ctx)
.unwrap_or_default()
+10 -4
View File
@@ -49,9 +49,9 @@ impl Lot {
Qr::FprMismatch { .. } => None,
Qr::FprWithoutAddr { fingerprint, .. } => Some(fingerprint),
Qr::Account { domain } => Some(domain),
Qr::Backup { .. } => None,
Qr::Backup2 { .. } => None,
Qr::WebrtcInstance { domain, .. } => Some(domain),
Qr::Socks5Proxy { host, .. } => Some(host),
Qr::Addr { draft, .. } => draft.as_deref(),
Qr::Url { url } => Some(url),
Qr::Text { text } => Some(text),
@@ -68,7 +68,10 @@ impl Lot {
pub fn get_text2(&self) -> Option<Cow<str>> {
match self {
Self::Summary(summary) => Some(summary.truncated_text(160)),
Self::Qr(_) => None,
Self::Qr(qr) => match qr {
Qr::Socks5Proxy { port, .. } => Some(Cow::Owned(format!("{port}"))),
_ => None,
},
Self::Error(_) => None,
}
}
@@ -102,9 +105,9 @@ impl Lot {
Qr::FprMismatch { .. } => LotState::QrFprMismatch,
Qr::FprWithoutAddr { .. } => LotState::QrFprWithoutAddr,
Qr::Account { .. } => LotState::QrAccount,
Qr::Backup { .. } => LotState::QrBackup,
Qr::Backup2 { .. } => LotState::QrBackup2,
Qr::WebrtcInstance { .. } => LotState::QrWebrtcInstance,
Qr::Socks5Proxy { .. } => LotState::QrSocks5Proxy,
Qr::Addr { .. } => LotState::QrAddr,
Qr::Url { .. } => LotState::QrUrl,
Qr::Text { .. } => LotState::QrText,
@@ -128,9 +131,9 @@ impl Lot {
Qr::FprMismatch { contact_id } => contact_id.unwrap_or_default().to_u32(),
Qr::FprWithoutAddr { .. } => Default::default(),
Qr::Account { .. } => Default::default(),
Qr::Backup { .. } => Default::default(),
Qr::Backup2 { .. } => Default::default(),
Qr::WebrtcInstance { .. } => Default::default(),
Qr::Socks5Proxy { .. } => Default::default(),
Qr::Addr { contact_id, .. } => contact_id.to_u32(),
Qr::Url { .. } => Default::default(),
Qr::Text { .. } => Default::default(),
@@ -185,6 +188,9 @@ pub enum LotState {
/// text1=domain, text2=instance pattern
QrWebrtcInstance = 260,
/// text1=host, text2=port
QrSocks5Proxy = 270,
/// id=contact
QrAddr = 320,
+2 -2
View File
@@ -1,6 +1,6 @@
[package]
name = "deltachat-jsonrpc"
version = "1.142.12"
version = "1.143.0"
description = "DeltaChat JSON-RPC API"
edition = "2021"
default-run = "deltachat-jsonrpc-server"
@@ -25,7 +25,7 @@ async-channel = { workspace = true }
futures = { workspace = true }
serde_json = { workspace = true }
yerpc = { workspace = true, features = ["anyhow_expose", "openrpc"] }
typescript-type-def = { version = "0.5.8", features = ["json_value"] }
typescript-type-def = { version = "0.5.12", features = ["json_value"] }
tokio = { workspace = true }
sanitize-filename = { workspace = true }
walkdir = "2.5.0"
+3 -3
View File
@@ -321,12 +321,12 @@ impl CommandApi {
) -> Result<Option<ProviderInfo>> {
let ctx = self.get_context(account_id).await?;
let socks5_enabled = ctx
.get_config_bool(deltachat::config::Config::Socks5Enabled)
let proxy_enabled = ctx
.get_config_bool(deltachat::config::Config::ProxyEnabled)
.await?;
let provider_info =
get_provider_info(&ctx, email.split('@').last().unwrap_or(""), socks5_enabled).await;
get_provider_info(&ctx, email.split('@').last().unwrap_or(""), proxy_enabled).await;
Ok(ProviderInfo::from_dc_type(provider_info))
}
+17 -6
View File
@@ -32,9 +32,6 @@ pub enum QrObject {
Account {
domain: String,
},
Backup {
ticket: String,
},
Backup2 {
auth_token: String,
@@ -44,6 +41,12 @@ pub enum QrObject {
domain: String,
instance_pattern: String,
},
Socks5Proxy {
host: String,
port: u16,
user: Option<String>,
pass: Option<String>,
},
Addr {
contact_id: u32,
draft: Option<String>,
@@ -134,9 +137,6 @@ impl From<Qr> for QrObject {
}
Qr::FprWithoutAddr { fingerprint } => QrObject::FprWithoutAddr { fingerprint },
Qr::Account { domain } => QrObject::Account { domain },
Qr::Backup { ticket } => QrObject::Backup {
ticket: ticket.to_string(),
},
Qr::Backup2 {
ref node_addr,
auth_token,
@@ -152,6 +152,17 @@ impl From<Qr> for QrObject {
domain,
instance_pattern,
},
Qr::Socks5Proxy {
host,
port,
user,
pass,
} => QrObject::Socks5Proxy {
host,
port,
user,
pass,
},
Qr::Addr { contact_id, draft } => {
let contact_id = contact_id.to_u32();
QrObject::Addr { contact_id, draft }
+1 -1
View File
@@ -83,7 +83,7 @@ mod tests {
assert_eq!(result, response.to_owned());
}
{
let request = r#"{"jsonrpc":"2.0","method":"batch_set_config","id":2,"params":[1,{"addr":"","mail_user":"","mail_pw":"","mail_server":"","mail_port":"","mail_security":"","imap_certificate_checks":"","send_user":"","send_pw":"","send_server":"","send_port":"","send_security":"","smtp_certificate_checks":"","socks5_enabled":"0","socks5_host":"","socks5_port":"","socks5_user":"","socks5_password":""}]}"#;
let request = r#"{"jsonrpc":"2.0","method":"batch_set_config","id":2,"params":[1,{"addr":"","mail_user":"","mail_pw":"","mail_server":"","mail_port":"","mail_security":"","imap_certificate_checks":"","send_user":"","send_pw":"","send_server":"","send_port":"","send_security":"","smtp_certificate_checks":""}]}"#;
let response = r#"{"jsonrpc":"2.0","id":2,"result":null}"#;
session.handle_incoming(request).await;
let result = receiver.recv().await?;
+1 -1
View File
@@ -58,5 +58,5 @@
},
"type": "module",
"types": "dist/deltachat.d.ts",
"version": "1.142.12"
"version": "1.143.0"
}
+2 -2
View File
@@ -1,16 +1,16 @@
[package]
name = "deltachat-repl"
version = "1.142.12"
version = "1.143.0"
license = "MPL-2.0"
edition = "2021"
repository = "https://github.com/deltachat/deltachat-core-rust"
[dependencies]
ansi_term = { workspace = true }
anyhow = { workspace = true }
deltachat = { workspace = true, features = ["internals"]}
dirs = "5"
log = { workspace = true }
nu-ansi-term = { workspace = true }
rusqlite = { workspace = true }
rustyline = "14"
tokio = { workspace = true, features = ["fs", "rt-multi-thread", "macros"] }
+3 -3
View File
@@ -1249,10 +1249,10 @@ pub async fn cmdline(context: Context, line: &str, chat_id: &mut ChatId) -> Resu
}
"providerinfo" => {
ensure!(!arg1.is_empty(), "Argument <addr> missing.");
let socks5_enabled = context
.get_config_bool(config::Config::Socks5Enabled)
let proxy_enabled = context
.get_config_bool(config::Config::ProxyEnabled)
.await?;
match provider::get_provider_info(&context, arg1, socks5_enabled).await {
match provider::get_provider_info(&context, arg1, proxy_enabled).await {
Some(info) => {
println!("Information for provider belonging to {arg1}:");
println!("status: {}", info.status as u32);
+1 -1
View File
@@ -12,7 +12,6 @@ use std::borrow::Cow::{self, Borrowed, Owned};
use std::io::{self, Write};
use std::process::Command;
use ansi_term::Color;
use anyhow::{bail, Error};
use deltachat::chat::ChatId;
use deltachat::config;
@@ -22,6 +21,7 @@ use deltachat::qr_code_generator::get_securejoin_qr_svg;
use deltachat::securejoin::*;
use deltachat::EventType;
use log::{error, info, warn};
use nu_ansi_term::Color;
use rustyline::completion::{Completer, FilenameCompleter, Pair};
use rustyline::error::ReadlineError;
use rustyline::highlight::{Highlighter, MatchingBracketHighlighter};
+1 -1
View File
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "deltachat-rpc-client"
version = "1.142.12"
version = "1.143.0"
description = "Python client for Delta Chat core JSON-RPC interface"
classifiers = [
"Development Status :: 5 - Production/Stable",
@@ -9,18 +9,19 @@ import io
import pathlib
import ssl
from contextlib import contextmanager
from typing import TYPE_CHECKING
from imap_tools import (
AND,
Header,
MailBox,
MailBoxTls,
MailMessage,
MailMessageFlags,
errors,
)
from . import Account, const
if TYPE_CHECKING:
from . import Account
FLAGS = b"FLAGS"
FETCH = b"FETCH"
@@ -35,28 +36,15 @@ class DirectImap:
self.connect()
def connect(self):
# Assume the testing server supports TLS on port 993.
host = self.account.get_config("configured_mail_server")
port = int(self.account.get_config("configured_mail_port"))
security = int(self.account.get_config("configured_mail_security"))
port = 993
user = self.account.get_config("addr")
host = user.rsplit("@")[-1]
pw = self.account.get_config("mail_pw")
if security == const.SocketSecurity.PLAIN:
ssl_context = None
else:
ssl_context = ssl.create_default_context()
# don't check if certificate hostname doesn't match target hostname
ssl_context.check_hostname = False
# don't check if the certificate is trusted by a certificate authority
ssl_context.verify_mode = ssl.CERT_NONE
if security == const.SocketSecurity.STARTTLS:
self.conn = MailBoxTls(host, port, ssl_context=ssl_context)
elif security == const.SocketSecurity.PLAIN or security == const.SocketSecurity.SSL:
self.conn = MailBox(host, port, ssl_context=ssl_context)
self.conn = MailBox(host, port, ssl_context=ssl.create_default_context())
self.conn.login(user, pw)
self.select_folder("INBOX")
+15 -7
View File
@@ -62,7 +62,7 @@ def test_qr_setup_contact_svg(acfactory) -> None:
@pytest.mark.parametrize("protect", [True, False])
def test_qr_securejoin(acfactory, protect, tmp_path):
alice, bob = acfactory.get_online_accounts(2)
alice, bob, fiona = acfactory.get_online_accounts(3)
# Setup second device for Alice
# to test observing securejoin protocol.
@@ -71,11 +71,11 @@ def test_qr_securejoin(acfactory, protect, tmp_path):
alice2 = acfactory.get_unconfigured_account()
alice2.import_backup(files[0])
logging.info("Alice creates a verified group")
alice_chat = alice.create_group("Verified group", protect=protect)
logging.info("Alice creates a group")
alice_chat = alice.create_group("Group", protect=protect)
assert alice_chat.get_basic_snapshot().is_protected == protect
logging.info("Bob joins verified group")
logging.info("Bob joins the group")
qr_code = alice_chat.get_qr_code()
bob.secure_join(qr_code)
@@ -112,6 +112,13 @@ def test_qr_securejoin(acfactory, protect, tmp_path):
alice2_contact_bob_snapshot = alice2_contact_bob.get_snapshot()
assert alice2_contact_bob_snapshot.is_verified
# The QR code token is synced, so alice2 must be able to handle join requests.
logging.info("Fiona joins the group via alice2")
alice.stop_io()
fiona.secure_join(qr_code)
alice2.wait_for_securejoin_inviter_success()
fiona.wait_for_securejoin_joiner_success()
def test_qr_securejoin_contact_request(acfactory) -> None:
"""Alice invites Bob to a group when Bob's chat with Alice is in a contact request mode."""
@@ -650,7 +657,8 @@ def test_withdraw_securejoin_qr(acfactory):
logging.info("Bob scanned withdrawn QR code")
while True:
event = alice.wait_for_event()
if event.kind == EventType.MSGS_CHANGED and event.chat_id != 0:
if (
event.kind == EventType.WARNING
and "Ignoring vg-request-with-auth message because of invalid auth code." in event.msg
):
break
snapshot = alice.get_message_by_id(event.msg_id).get_snapshot()
assert snapshot.text == "Cannot establish guaranteed end-to-end encryption with {}".format(bob.get_config("addr"))
+21 -1
View File
@@ -83,6 +83,26 @@ def test_configure_ip(acfactory) -> None:
account.configure()
def test_configure_alternative_port(acfactory) -> None:
"""Test that configuration with alternative port 443 works."""
account = acfactory.new_preconfigured_account()
account.set_config("mail_port", "443")
account.set_config("send_port", "443")
account.configure()
def test_configure_username(acfactory) -> None:
account = acfactory.new_preconfigured_account()
addr = account.get_config("addr")
account.set_config("mail_user", addr)
account.configure()
assert account.get_config("configured_mail_user") == addr
def test_account(acfactory) -> None:
alice, bob = acfactory.get_online_accounts(2)
@@ -413,7 +433,7 @@ def test_provider_info(rpc) -> None:
assert provider_info["id"] == "gmail"
# Disable MX record resolution.
rpc.set_config(account_id, "socks5_enabled", "1")
rpc.set_config(account_id, "proxy_enabled", "1")
provider_info = rpc.get_provider_info(account_id, "github.com")
assert provider_info is None
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "deltachat-rpc-server"
version = "1.142.12"
version = "1.143.0"
description = "DeltaChat JSON-RPC server"
edition = "2021"
readme = "README.md"
@@ -15,5 +15,5 @@
},
"type": "module",
"types": "index.d.ts",
"version": "1.142.12"
"version": "1.143.0"
}
+1 -43
View File
@@ -1,7 +1,6 @@
[advisories]
ignore = [
"RUSTSEC-2020-0071",
"RUSTSEC-2022-0093",
# Timing attack on RSA.
# Delta Chat does not use RSA for new keys
@@ -10,15 +9,8 @@ ignore = [
# <https://rustsec.org/advisories/RUSTSEC-2023-0071>
"RUSTSEC-2023-0071",
# Unmaintained ansi_term
"RUSTSEC-2021-0139",
# Unmaintained encoding
"RUSTSEC-2021-0153",
# Problem in curve25519-dalek 3.2.0 used by iroh 0.4.
# curve25519-dalek 4.1.3 has the problem fixed.
"RUSTSEC-2024-0344",
]
[bans]
@@ -31,24 +23,10 @@ skip = [
{ name = "asn1-rs-impl", version = "0.1.0" },
{ name = "asn1-rs", version = "0.5.2" },
{ name = "async-channel", version = "1.9.0" },
{ name = "base16ct", version = "0.1.1" },
{ name = "base64", version = "<0.21" },
{ name = "base64", version = "0.21.7" },
{ name = "bitflags", version = "1.3.2" },
{ name = "block-buffer", version = "<0.10" },
{ name = "convert_case", version = "0.4.0" },
{ name = "curve25519-dalek", version = "3.2.0" },
{ name = "darling_core", version = "<0.14" },
{ name = "darling_macro", version = "<0.14" },
{ name = "darling", version = "<0.14" },
{ name = "der_derive", version = "0.6.1" },
{ name = "derive_more", version = "0.99.17" },
{ name = "der-parser", version = "8.2.0" },
{ name = "der", version = "0.6.1" },
{ name = "digest", version = "<0.10" },
{ name = "dlopen2", version = "0.4.1" },
{ name = "ed25519-dalek", version = "1.0.1" },
{ name = "ed25519", version = "1.5.3" },
{ name = "event-listener", version = "2.5.3" },
{ name = "event-listener", version = "4.0.3" },
{ name = "fastrand", version = "1.9.0" },
@@ -59,42 +37,24 @@ skip = [
{ name = "http", version = "0.2.12" },
{ name = "hyper", version = "0.14.28" },
{ name = "idna", version = "0.4.0" },
{ name = "netlink-packet-core", version = "0.5.0" },
{ name = "netlink-packet-route", version = "0.15.0" },
{ name = "nix", version = "0.26.4" },
{ name = "oid-registry", version = "0.6.1" },
{ name = "pem-rfc7468", version = "0.6.0" },
{ name = "pem", version = "1.1.1" },
{ name = "pkcs8", version = "0.9.0" },
{ name = "quick-error", version = "<2.0" },
{ name = "rand_chacha", version = "<0.3" },
{ name = "rand_core", version = "<0.6" },
{ name = "rand", version = "<0.8" },
{ name = "rcgen", version = "<0.12.1" },
{ name = "redox_syscall", version = "0.3.5" },
{ name = "regex-automata", version = "0.1.10" },
{ name = "regex-syntax", version = "0.6.29" },
{ name = "ring", version = "0.16.20" },
{ name = "rustls-pemfile", version = "1.0.4" },
{ name = "rustls", version = "0.21.11" },
{ name = "rustls-webpki", version = "0.101.7" },
{ name = "sec1", version = "0.3.0" },
{ name = "sha2", version = "<0.10" },
{ name = "signature", version = "1.6.4" },
{ name = "spin", version = "<0.9.6" },
{ name = "spki", version = "0.6.0" },
{ name = "ssh-encoding", version = "0.1.0" },
{ name = "ssh-key", version = "0.5.1" },
{ name = "strsim", version = "0.10.0" },
{ name = "sync_wrapper", version = "0.1.2" },
{ name = "synstructure", version = "0.12.6" },
{ name = "syn", version = "1.0.109" },
{ name = "system-configuration-sys", version = "0.5.0" },
{ name = "system-configuration", version = "0.5.1" },
{ name = "time", version = "<0.3" },
{ name = "tokio-rustls", version = "0.24.1" },
{ name = "toml_edit", version = "0.21.1" },
{ name = "untrusted", version = "0.7.1" },
{ name = "wasi", version = "<0.11" },
{ name = "webpki-roots", version ="0.25.4" },
{ name = "windows_aarch64_gnullvm", version = "<0.52" },
@@ -102,14 +62,12 @@ skip = [
{ name = "windows-core", version = "<0.54.0" },
{ name = "windows_i686_gnu", version = "<0.52" },
{ name = "windows_i686_msvc", version = "<0.52" },
{ name = "windows-sys", version = "<0.52" },
{ name = "windows-sys", version = "<0.59" },
{ name = "windows-targets", version = "<0.52" },
{ name = "windows", version = "0.32.0" },
{ name = "windows", version = "<0.54.0" },
{ name = "windows_x86_64_gnullvm", version = "<0.52" },
{ name = "windows_x86_64_gnu", version = "<0.52" },
{ name = "windows_x86_64_msvc", version = "<0.52" },
{ name = "winnow", version = "0.5.40" },
{ name = "winreg", version = "0.50.0" },
{ name = "x509-parser", version = "<0.16.0" },
]
+1
View File
@@ -136,6 +136,7 @@ module.exports = {
DC_QR_LOGIN: 520,
DC_QR_REVIVE_VERIFYCONTACT: 510,
DC_QR_REVIVE_VERIFYGROUP: 512,
DC_QR_SOCKS5_PROXY: 270,
DC_QR_TEXT: 330,
DC_QR_URL: 332,
DC_QR_WEBRTC_INSTANCE: 260,
+1
View File
@@ -136,6 +136,7 @@ export enum C {
DC_QR_LOGIN = 520,
DC_QR_REVIVE_VERIFYCONTACT = 510,
DC_QR_REVIVE_VERIFYGROUP = 512,
DC_QR_SOCKS5_PROXY = 270,
DC_QR_TEXT = 330,
DC_QR_URL = 332,
DC_QR_WEBRTC_INSTANCE = 260,
+1 -1
View File
@@ -271,7 +271,7 @@ describe('Basic offline Tests', function () {
'sync_msgs',
'sentbox_watch',
'show_emails',
'socks5_enabled',
'proxy_enabled',
'sqlite_version',
'uptime',
'used_account_settings',
+1 -1
View File
@@ -55,5 +55,5 @@
"test:mocha": "mocha node/test/test.mjs --growl --reporter=spec --bail --exit"
},
"types": "node/dist/index.d.ts",
"version": "1.142.12"
"version": "1.143.0"
}
+1 -1
View File
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "deltachat"
version = "1.142.12"
version = "1.143.0"
description = "Python bindings for the Delta Chat Core library using CFFI against the Rust-implemented libdeltachat"
readme = "README.rst"
requires-python = ">=3.7"
+1 -3
View File
@@ -194,15 +194,13 @@ class Account:
assert res != ffi.NULL, f"config value not found for: {name!r}"
return from_dc_charpointer(res)
def _preconfigure_keypair(self, addr: str, secret: str) -> None:
def _preconfigure_keypair(self, secret: str) -> None:
"""See dc_preconfigure_keypair() in deltachat.h.
In other words, you don't need this.
"""
res = lib.dc_preconfigure_keypair(
self._dc_context,
as_dc_charpointer(addr),
ffi.NULL,
as_dc_charpointer(secret),
)
if res == 0:
+7 -21
View File
@@ -8,19 +8,19 @@ import io
import pathlib
import ssl
from contextlib import contextmanager
from typing import List
from typing import List, TYPE_CHECKING
from imap_tools import (
AND,
Header,
MailBox,
MailBoxTls,
MailMessage,
MailMessageFlags,
errors,
)
from deltachat import Account, const
if TYPE_CHECKING:
from deltachat import Account
FLAGS = b"FLAGS"
FETCH = b"FETCH"
@@ -28,7 +28,7 @@ ALL = "1:*"
class DirectImap:
def __init__(self, account: Account) -> None:
def __init__(self, account: "Account") -> None:
self.account = account
self.logid = account.get_config("displayname") or id(account)
self._idling = False
@@ -36,27 +36,13 @@ class DirectImap:
def connect(self):
host = self.account.get_config("configured_mail_server")
port = int(self.account.get_config("configured_mail_port"))
security = int(self.account.get_config("configured_mail_security"))
port = 993
user = self.account.get_config("addr")
host = user.rsplit("@")[-1]
pw = self.account.get_config("mail_pw")
if security == const.DC_SOCKET_PLAIN:
ssl_context = None
else:
ssl_context = ssl.create_default_context()
# don't check if certificate hostname doesn't match target hostname
ssl_context.check_hostname = False
# don't check if the certificate is trusted by a certificate authority
ssl_context.verify_mode = ssl.CERT_NONE
if security == const.DC_SOCKET_STARTTLS:
self.conn = MailBoxTls(host, port, ssl_context=ssl_context)
elif security == const.DC_SOCKET_PLAIN or security == const.DC_SOCKET_SSL:
self.conn = MailBox(host, port, ssl_context=ssl_context)
self.conn = MailBox(host, port, ssl_context=ssl.create_default_context())
self.conn.login(user, pw)
self.select_folder("INBOX")
+6 -5
View File
@@ -462,7 +462,7 @@ class ACFactory:
def remove_preconfigured_keys(self) -> None:
self._preconfigured_keys = []
def _preconfigure_key(self, account, addr):
def _preconfigure_key(self, account):
# Only set a preconfigured key if we haven't used it yet for another account.
try:
keyname = self._preconfigured_keys.pop(0)
@@ -471,9 +471,9 @@ class ACFactory:
else:
fname_sec = self.data.read_path(f"key/{keyname}-secret.asc")
if fname_sec:
account._preconfigure_keypair(addr, fname_sec)
account._preconfigure_keypair(fname_sec)
return True
print(f"WARN: could not use preconfigured keys for {addr!r}")
print("WARN: could not use preconfigured keys")
def get_pseudo_configured_account(self, passphrase: Optional[str] = None) -> Account:
# do a pseudo-configured account
@@ -492,7 +492,7 @@ class ACFactory:
"configured": "1",
},
)
self._preconfigure_key(ac, addr)
self._preconfigure_key(ac)
self._acsetup.init_logging(ac)
return ac
@@ -525,9 +525,10 @@ class ACFactory:
configdict.setdefault("mvbox_move", False)
configdict.setdefault("sentbox_watch", False)
configdict.setdefault("sync_msgs", False)
configdict.setdefault("delete_server_after", 0)
ac.update_config(configdict)
self._acsetup._account2config[ac] = configdict
self._preconfigure_key(ac, configdict["addr"])
self._preconfigure_key(ac)
return ac
def wait_configured(self, account) -> None:
+2 -3
View File
@@ -620,7 +620,7 @@ def test_long_group_name(acfactory, lp):
def test_send_self_message(acfactory, lp):
ac1 = acfactory.new_online_configuring_account(mvbox_move=True)
ac1 = acfactory.new_online_configuring_account(mvbox_move=True, bcc_self=True)
acfactory.bring_accounts_online()
lp.sec("ac1: create self chat")
chat = ac1.get_self_contact().create_chat()
@@ -2076,12 +2076,11 @@ def test_send_receive_locations(acfactory, lp):
def test_immediate_autodelete(acfactory, lp):
ac1 = acfactory.new_online_configuring_account()
ac2 = acfactory.new_online_configuring_account()
acfactory.bring_accounts_online()
# "1" means delete immediately, while "0" means do not delete
ac2.set_config("delete_server_after", "1")
acfactory.bring_accounts_online()
lp.sec("ac1: create chat with ac2")
chat1 = ac1.create_chat(ac2)
ac2.create_chat(ac1)
+1 -1
View File
@@ -67,7 +67,7 @@ class TestOfflineAccountBasic:
ac = acfactory.get_unconfigured_account()
alice_secret = data.read_path("key/alice-secret.asc")
assert alice_secret
ac._preconfigure_keypair("alice@example.org", alice_secret)
ac._preconfigure_keypair(alice_secret)
def test_getinfo(self, acfactory):
ac1 = acfactory.get_unconfigured_account()
+1 -1
View File
@@ -1 +1 @@
2024-09-02
2024-09-12
+1 -1
View File
@@ -7,7 +7,7 @@ set -euo pipefail
#
# Avoid using rustup here as it depends on reading /proc/self/exe and
# has problems running under QEMU.
RUST_VERSION=1.80.1
RUST_VERSION=1.81.0
ARCH="$(uname -m)"
test -f "/lib/libc.musl-$ARCH.so.1" && LIBC=musl || LIBC=gnu
+1 -1
View File
@@ -6,7 +6,7 @@ set -euo pipefail
export TZ=UTC
# Provider database revision.
REV=05c1b2029da74718e4bdc3799a46e29c4f794dc7
REV=77cbf92a8565fdf1bcaba10fa93c1455c750a1e9
CORE_ROOT="$PWD"
TMP="$(mktemp -d)"
+65 -25
View File
@@ -46,8 +46,8 @@ use crate::stock_str;
use crate::sync::{self, Sync::*, SyncData};
use crate::tools::{
buf_compress, create_id, create_outgoing_rfc724_mid, create_smeared_timestamp,
create_smeared_timestamps, get_abs_path, gm2local_offset, smeared_time, time, IsNoneOrEmpty,
SystemTime,
create_smeared_timestamps, get_abs_path, gm2local_offset, smeared_time, time,
truncate_msg_text, IsNoneOrEmpty, SystemTime,
};
use crate::webxdc::StatusUpdateSerial;
@@ -279,9 +279,10 @@ impl ChatId {
) -> Result<Self> {
let chat_id = match ChatIdBlocked::lookup_by_contact(context, contact_id).await? {
Some(chat) => {
if create_blocked == Blocked::Not && chat.blocked != Blocked::Not {
chat.id.set_blocked(context, Blocked::Not).await?;
if create_blocked != Blocked::Not || chat.blocked == Blocked::Not {
return Ok(chat.id);
}
chat.id.set_blocked(context, Blocked::Not).await?;
chat.id
}
None => {
@@ -1934,11 +1935,13 @@ impl Chat {
msg.param.set_int(Param::AttachGroupImage, 1);
self.param.remove(Param::Unpromoted);
self.update_param(context).await?;
// send_sync_msg() is called (usually) a moment later at send_msg_to_smtp()
// when the group creation message is actually sent through SMTP --
// this makes sure, the other devices are aware of grpid that is used in the sync-message.
// TODO: Remove this compat code needed because Core <= v1.143:
// - doesn't accept synchronization of QR code tokens for unpromoted groups, so we also
// send them when the group is promoted.
// - doesn't sync QR code tokens for unpromoted groups and the group might be created
// before an upgrade.
context
.sync_qr_code_tokens(Some(self.id))
.sync_qr_code_tokens(Some(self.grpid.as_str()))
.await
.log_err(context)
.ok();
@@ -2071,6 +2074,8 @@ impl Chat {
msg.from_id = ContactId::SELF;
msg.rfc724_mid = new_rfc724_mid;
msg.timestamp_sort = timestamp;
let (msg_text, was_truncated) = truncate_msg_text(context, msg.text.clone()).await?;
let mime_modified = new_mime_headers.is_some() | was_truncated;
// add message to the database
if let Some(update_msg_id) = update_msg_id {
@@ -2092,14 +2097,14 @@ impl Chat {
msg.timestamp_sort,
msg.viewtype,
msg.state,
msg.text,
message::normalize_text(&msg.text),
msg_text,
message::normalize_text(&msg_text),
&msg.subject,
msg.param.to_string(),
msg.hidden,
msg.in_reply_to.as_deref().unwrap_or_default(),
new_references,
new_mime_headers.is_some(),
mime_modified,
new_mime_headers.unwrap_or_default(),
location_id as i32,
ephemeral_timer,
@@ -2143,14 +2148,14 @@ impl Chat {
msg.timestamp_sort,
msg.viewtype,
msg.state,
msg.text,
message::normalize_text(&msg.text),
msg_text,
message::normalize_text(&msg_text),
&msg.subject,
msg.param.to_string(),
msg.hidden,
msg.in_reply_to.as_deref().unwrap_or_default(),
new_references,
new_mime_headers.is_some(),
mime_modified,
new_mime_headers.unwrap_or_default(),
location_id as i32,
ephemeral_timer,
@@ -3726,17 +3731,13 @@ pub(crate) async fn add_contact_to_chat_ex(
bail!("can not add contact because the account is not part of the group/broadcast");
}
let sync_qr_code_tokens;
if from_handshake && chat.param.get_int(Param::Unpromoted).unwrap_or_default() == 1 {
chat.param.remove(Param::Unpromoted);
chat.update_param(context).await?;
if context
.sync_qr_code_tokens(Some(chat_id))
.await
.log_err(context)
.is_ok()
{
context.scheduler.interrupt_smtp().await;
}
sync_qr_code_tokens = true;
} else {
sync_qr_code_tokens = false;
}
if context.is_self_addr(contact.get_addr()).await? {
@@ -3780,6 +3781,20 @@ pub(crate) async fn add_contact_to_chat_ex(
return Err(e);
}
sync = Nosync;
// TODO: Remove this compat code needed because Core <= v1.143:
// - doesn't accept synchronization of QR code tokens for unpromoted groups, so we also send
// them when the group is promoted.
// - doesn't sync QR code tokens for unpromoted groups and the group might be created before
// an upgrade.
if sync_qr_code_tokens
&& context
.sync_qr_code_tokens(Some(chat.grpid.as_str()))
.await
.log_err(context)
.is_ok()
{
context.scheduler.interrupt_smtp().await;
}
}
context.emit_event(EventType::ChatModified(chat_id));
if sync.into() {
@@ -4254,10 +4269,14 @@ pub async fn resend_msgs(context: &Context, msg_ids: &[MsgId]) -> Result<()> {
msg.update_param(context).await?;
}
match msg.get_state() {
MessageState::OutFailed | MessageState::OutDelivered | MessageState::OutMdnRcvd => {
// `get_state()` may return an outdated `OutPending`, so update anyway.
MessageState::OutPending
| MessageState::OutFailed
| MessageState::OutDelivered
| MessageState::OutMdnRcvd => {
message::update_msg_state(context, msg.id, MessageState::OutPending).await?
}
_ => bail!("unexpected message state"),
msg_state => bail!("Unexpected message state {msg_state}"),
}
context.emit_event(EventType::MsgsChanged {
chat_id: msg.chat_id,
@@ -6840,8 +6859,29 @@ mod tests {
)
.await?;
let sent2 = alice.pop_sent_msg().await;
resend_msgs(&alice, &[sent1.sender_msg_id]).await?;
let resent_msg_id = sent1.sender_msg_id;
resend_msgs(&alice, &[resent_msg_id]).await?;
assert_eq!(
resent_msg_id.get_state(&alice).await?,
MessageState::OutPending
);
resend_msgs(&alice, &[resent_msg_id]).await?;
// Message can be re-sent multiple times.
assert_eq!(
resent_msg_id.get_state(&alice).await?,
MessageState::OutPending
);
alice.pop_sent_msg().await;
// There's still one more pending SMTP job.
assert_eq!(
resent_msg_id.get_state(&alice).await?,
MessageState::OutPending
);
let sent3 = alice.pop_sent_msg().await;
assert_eq!(
resent_msg_id.get_state(&alice).await?,
MessageState::OutDelivered
);
// Bob receives all messages
let bob = TestContext::new_bob().await;
+117 -36
View File
@@ -91,21 +91,43 @@ pub enum Config {
/// Should not be extended in the future, create new config keys instead.
ServerFlags,
/// True if proxy is enabled.
///
/// Can be used to disable proxy without erasing known URLs.
ProxyEnabled,
/// Proxy URL.
///
/// Supported URLs schemes are `socks5://` (SOCKS5) and `ss://` (Shadowsocks).
///
/// May contain multiple URLs separated by newline, in which case the first one is used.
ProxyUrl,
/// True if SOCKS5 is enabled.
///
/// Can be used to disable SOCKS5 without erasing SOCKS5 configuration.
///
/// Deprecated in favor of `ProxyEnabled`.
Socks5Enabled,
/// SOCKS5 proxy server hostname or address.
///
/// Deprecated in favor of `ProxyUrl`.
Socks5Host,
/// SOCKS5 proxy server port.
///
/// Deprecated in favor of `ProxyUrl`.
Socks5Port,
/// SOCKS5 proxy server username.
///
/// Deprecated in favor of `ProxyUrl`.
Socks5User,
/// SOCKS5 proxy server password.
///
/// Deprecated in favor of `ProxyUrl`.
Socks5Password,
/// Own name to use in the `From:` field when sending messages.
@@ -174,12 +196,12 @@ pub enum Config {
/// Timer in seconds after which the message is deleted from the
/// server.
///
/// Equals to 0 by default, which means the message is never
/// deleted.
/// 0 means messages are never deleted by Delta Chat.
///
/// Value 1 is treated as "delete at once": messages are deleted
/// immediately, without moving to DeltaChat folder.
#[strum(props(default = "0"))]
///
/// Default is 1 for chatmail accounts before a backup export, 0 otherwise.
DeleteServerAfter,
/// Timer in seconds after which the message is deleted from the
@@ -200,21 +222,32 @@ pub enum Config {
/// The primary email address. Also see `SecondaryAddrs`.
ConfiguredAddr,
/// List of configured IMAP servers as a JSON array.
ConfiguredImapServers,
/// Configured IMAP server hostname.
///
/// This is replaced by `configured_imap_servers` for new configurations.
ConfiguredMailServer,
/// Configured IMAP server port.
///
/// This is replaced by `configured_imap_servers` for new configurations.
ConfiguredMailPort,
/// Configured IMAP server security (e.g. TLS, STARTTLS).
///
/// This is replaced by `configured_imap_servers` for new configurations.
ConfiguredMailSecurity,
/// Configured IMAP server username.
///
/// This is set if user has configured username manually.
ConfiguredMailUser,
/// Configured IMAP server password.
ConfiguredMailPw,
/// Configured IMAP server port.
ConfiguredMailPort,
/// Configured IMAP server security (e.g. TLS, STARTTLS).
ConfiguredMailSecurity,
/// Configured TLS certificate checks.
/// This option is saved on successful configuration
/// and should not be modified manually.
@@ -223,18 +256,32 @@ pub enum Config {
/// but has "IMAP" in the name for backwards compatibility.
ConfiguredImapCertificateChecks,
/// List of configured SMTP servers as a JSON array.
ConfiguredSmtpServers,
/// Configured SMTP server hostname.
///
/// This is replaced by `configured_smtp_servers` for new configurations.
ConfiguredSendServer,
/// Configured SMTP server port.
///
/// This is replaced by `configured_smtp_servers` for new configurations.
ConfiguredSendPort,
/// Configured SMTP server security (e.g. TLS, STARTTLS).
///
/// This is replaced by `configured_smtp_servers` for new configurations.
ConfiguredSendSecurity,
/// Configured SMTP server username.
///
/// This is set if user has configured username manually.
ConfiguredSendUser,
/// Configured SMTP server password.
ConfiguredSendPw,
/// Configured SMTP server port.
ConfiguredSendPort,
/// Deprecated, stored for backwards compatibility.
///
/// ConfiguredImapCertificateChecks is actually used.
@@ -243,9 +290,6 @@ pub enum Config {
/// Whether OAuth 2 is used with configured provider.
ConfiguredServerFlags,
/// Configured SMTP server security (e.g. TLS, STARTTLS).
ConfiguredSendSecurity,
/// Configured folder for incoming messages.
ConfiguredInboxFolder,
@@ -416,13 +460,16 @@ impl Config {
}
impl Context {
/// Returns true if configuration value is set for the given key.
pub async fn config_exists(&self, key: Config) -> Result<bool> {
/// Returns true if configuration value is set in the db for the given key.
///
/// NB: Don't use this to check if the key is configured because this doesn't look into
/// environment. The proper use of this function is e.g. checking a key before setting it.
pub(crate) async fn config_exists(&self, key: Config) -> Result<bool> {
Ok(self.sql.get_raw_config(key.as_ref()).await?.is_some())
}
/// Get a configuration key. Returns `None` if no value is set, and no default value found.
pub async fn get_config(&self, key: Config) -> Result<Option<String>> {
/// Get a config key value. Returns `None` if no value is set.
pub(crate) async fn get_config_opt(&self, key: Config) -> Result<Option<String>> {
let env_key = format!("DELTACHAT_{}", key.as_ref().to_uppercase());
if let Ok(value) = env::var(env_key) {
return Ok(Some(value));
@@ -442,19 +489,38 @@ impl Context {
Config::SysConfigKeys => Some(get_config_keys_string()),
_ => self.sql.get_raw_config(key.as_ref()).await?,
};
Ok(value)
}
/// Get a config key value if set, or a default value. Returns `None` if no value exists.
pub async fn get_config(&self, key: Config) -> Result<Option<String>> {
let value = self.get_config_opt(key).await?;
if value.is_some() {
return Ok(value);
}
// Default values
match key {
Config::ConfiguredInboxFolder => Ok(Some("INBOX".to_owned())),
_ => Ok(key.get_str("default").map(|s| s.to_string())),
}
let val = match key {
Config::ConfiguredInboxFolder => Some("INBOX"),
Config::DeleteServerAfter => match Box::pin(self.is_chatmail()).await? {
false => Some("0"),
true => Some("1"),
},
_ => key.get_str("default"),
};
Ok(val.map(|s| s.to_string()))
}
/// Returns Some(T) if a value for the given key exists and was successfully parsed.
/// Returns Some(T) if a value for the given key is set and was successfully parsed.
/// Returns None if could not parse.
pub(crate) async fn get_config_opt_parsed<T: FromStr>(&self, key: Config) -> Result<Option<T>> {
self.get_config_opt(key)
.await
.map(|s: Option<String>| s.and_then(|s| s.parse().ok()))
}
/// Returns Some(T) if a value for the given key exists (incl. default value) and was
/// successfully parsed.
/// Returns None if could not parse.
pub async fn get_config_parsed<T: FromStr>(&self, key: Config) -> Result<Option<T>> {
self.get_config(key)
@@ -482,14 +548,21 @@ impl Context {
Ok(self.get_config_parsed(key).await?.unwrap_or_default())
}
/// Returns boolean configuration value (if any) for the given key.
pub async fn get_config_bool_opt(&self, key: Config) -> Result<Option<bool>> {
Ok(self.get_config_parsed::<i32>(key).await?.map(|x| x != 0))
/// Returns boolean configuration value (if set) for the given key.
pub(crate) async fn get_config_bool_opt(&self, key: Config) -> Result<Option<bool>> {
Ok(self
.get_config_opt_parsed::<i32>(key)
.await?
.map(|x| x != 0))
}
/// Returns boolean configuration value for the given key.
pub async fn get_config_bool(&self, key: Config) -> Result<bool> {
Ok(self.get_config_bool_opt(key).await?.unwrap_or_default())
Ok(self
.get_config_parsed::<i32>(key)
.await?
.map(|x| x != 0)
.unwrap_or_default())
}
/// Returns true if movebox ("DeltaChat" folder) should be watched.
@@ -517,9 +590,9 @@ impl Context {
/// Returns whether MDNs should be requested.
pub(crate) async fn should_request_mdns(&self) -> Result<bool> {
match self.config_exists(Config::MdnsEnabled).await? {
true => self.get_config_bool(Config::MdnsEnabled).await,
false => Ok(!self.get_config_bool(Config::Bot).await?),
match self.get_config_bool_opt(Config::MdnsEnabled).await? {
Some(val) => Ok(val),
None => Ok(!self.get_config_bool(Config::Bot).await?),
}
}
@@ -533,11 +606,16 @@ impl Context {
/// `None` means never delete the message, `Some(0)` means delete
/// at once, `Some(x)` means delete after `x` seconds.
pub async fn get_config_delete_server_after(&self) -> Result<Option<i64>> {
match self.get_config_int(Config::DeleteServerAfter).await? {
0 => Ok(None),
1 => Ok(Some(0)),
x => Ok(Some(i64::from(x))),
}
let val = match self
.get_config_parsed::<i64>(Config::DeleteServerAfter)
.await?
.unwrap_or(0)
{
0 => None,
1 => Some(0),
x => Some(x),
};
Ok(val)
}
/// Gets the configured provider, as saved in the `configured_provider` value.
@@ -582,6 +660,7 @@ impl Context {
fn check_config(key: Config, value: Option<&str>) -> Result<()> {
match key {
Config::Socks5Enabled
| Config::ProxyEnabled
| Config::BccSelf
| Config::E2eeEnabled
| Config::MdnsEnabled
@@ -982,12 +1061,14 @@ mod tests {
let t = &TestContext::new_alice().await;
assert!(t.should_request_mdns().await?);
assert!(t.should_send_mdns().await?);
assert!(t.get_config_bool_opt(Config::MdnsEnabled).await?.is_none());
// The setting should be displayed correctly.
assert!(t.get_config_bool(Config::MdnsEnabled).await?);
t.set_config_bool(Config::Bot, true).await?;
assert!(!t.should_request_mdns().await?);
assert!(t.should_send_mdns().await?);
assert!(t.get_config_bool_opt(Config::MdnsEnabled).await?.is_none());
assert!(t.get_config_bool(Config::MdnsEnabled).await?);
Ok(())
}
+195 -293
View File
@@ -11,7 +11,7 @@
mod auto_mozilla;
mod auto_outlook;
mod server_params;
pub(crate) mod server_params;
use anyhow::{bail, ensure, Context as _, Result};
use auto_mozilla::moz_autoconfigure;
@@ -25,14 +25,16 @@ use tokio::task;
use crate::config::{self, Config};
use crate::context::Context;
use crate::imap::{session::Session as ImapSession, Imap};
use crate::imap::Imap;
use crate::log::LogExt;
use crate::login_param::{CertificateChecks, LoginParam, ServerLoginParam};
use crate::login_param::{
ConfiguredCertificateChecks, ConfiguredLoginParam, ConfiguredServerLoginParam,
ConnectionCandidate, EnteredCertificateChecks, EnteredLoginParam,
};
use crate::message::{Message, Viewtype};
use crate::oauth2::get_oauth2_addr;
use crate::provider::{Protocol, Socket, UsernamePattern};
use crate::smtp::Smtp;
use crate::socks::Socks5Config;
use crate::stock_str;
use crate::sync::Sync::*;
use crate::tools::time;
@@ -110,16 +112,15 @@ impl Context {
async fn inner_configure(&self) -> Result<()> {
info!(self, "Configure ...");
let mut param = LoginParam::load_candidate_params(self).await?;
let param = EnteredLoginParam::load(self).await?;
let old_addr = self.get_config(Config::ConfiguredAddr).await?;
let success = configure(self, &mut param).await;
let configured_param_res = configure(self, &param).await;
self.set_config_internal(Config::NotifyAboutWrongPw, None)
.await?;
on_configure_completed(self, param, old_addr).await?;
on_configure_completed(self, configured_param_res?, old_addr).await?;
success?;
self.set_config_internal(Config::NotifyAboutWrongPw, Some("1"))
.await?;
Ok(())
@@ -128,7 +129,7 @@ impl Context {
async fn on_configure_completed(
context: &Context,
param: LoginParam,
param: ConfiguredLoginParam,
old_addr: Option<String>,
) -> Result<()> {
if let Some(provider) = param.provider {
@@ -178,19 +179,28 @@ async fn on_configure_completed(
Ok(())
}
async fn configure(ctx: &Context, param: &mut LoginParam) -> Result<()> {
progress!(ctx, 1);
/// Retrieves data from autoconfig and provider database
/// to transform user-entered login parameters into complete configuration.
async fn get_configured_param(
ctx: &Context,
param: &EnteredLoginParam,
) -> Result<ConfiguredLoginParam> {
ensure!(!param.addr.is_empty(), "Missing email address.");
let socks5_config = param.socks5_config.clone();
let socks5_enabled = socks5_config.is_some();
ensure!(!param.imap.password.is_empty(), "Missing (IMAP) password.");
let ctx2 = ctx.clone();
let update_device_chats_handle = task::spawn(async move { ctx2.update_device_chats().await });
// SMTP password is an "advanced" setting. If unset, use the same password as for IMAP.
let smtp_password = if param.smtp.password.is_empty() {
param.imap.password.clone()
} else {
param.smtp.password.clone()
};
// Step 1: Load the parameters and check email-address and password
let proxy_config = param.proxy_config.clone();
let proxy_enabled = proxy_config.is_some();
// OAuth is always set either for both IMAP and SMTP or not at all.
if param.imap.oauth2 {
let mut addr = param.addr.clone();
if param.oauth2 {
// the used oauth2 addr may differ, check this.
// if get_oauth2_addr() is not available in the oauth2 implementation, just use the given one.
progress!(ctx, 10);
@@ -199,7 +209,7 @@ async fn configure(ctx: &Context, param: &mut LoginParam) -> Result<()> {
.and_then(|e| e.parse().ok())
{
info!(ctx, "Authorized address is {}", oauth2_addr);
param.addr = oauth2_addr;
addr = oauth2_addr;
ctx.sql
.set_raw_config("addr", Some(param.addr.as_str()))
.await?;
@@ -211,9 +221,9 @@ async fn configure(ctx: &Context, param: &mut LoginParam) -> Result<()> {
let parsed = EmailAddress::new(&param.addr).context("Bad email-address")?;
let param_domain = parsed.domain;
// Step 2: Autoconfig
progress!(ctx, 200);
let provider;
let param_autoconfig;
if param.imap.server.is_empty()
&& param.imap.port == 0
@@ -225,77 +235,51 @@ async fn configure(ctx: &Context, param: &mut LoginParam) -> Result<()> {
&& param.smtp.user.is_empty()
{
// no advanced parameters entered by the user: query provider-database or do Autoconfig
info!(
ctx,
"checking internal provider-info for offline autoconfig"
);
if let Some(provider) =
provider::get_provider_info(ctx, &param_domain, socks5_enabled).await
{
param.provider = Some(provider);
match provider.status {
provider::Status::Ok | provider::Status::Preparation => {
if provider.server.is_empty() {
info!(ctx, "offline autoconfig found, but no servers defined");
param_autoconfig = None;
} else {
info!(ctx, "offline autoconfig found");
let servers = provider
.server
.iter()
.map(|s| ServerParams {
protocol: s.protocol,
socket: s.socket,
hostname: s.hostname.to_string(),
port: s.port,
username: match s.username_pattern {
UsernamePattern::Email => param.addr.to_string(),
UsernamePattern::Emaillocalpart => {
if let Some(at) = param.addr.find('@') {
param.addr.split_at(at).0.to_string()
} else {
param.addr.to_string()
}
}
},
})
.collect();
provider = provider::get_provider_info(ctx, &param_domain, proxy_enabled).await;
if let Some(provider) = provider {
if provider.server.is_empty() {
info!(ctx, "Offline autoconfig found, but no servers defined.");
param_autoconfig = None;
} else {
info!(ctx, "Offline autoconfig found.");
let servers = provider
.server
.iter()
.map(|s| ServerParams {
protocol: s.protocol,
socket: s.socket,
hostname: s.hostname.to_string(),
port: s.port,
username: match s.username_pattern {
UsernamePattern::Email => param.addr.to_string(),
UsernamePattern::Emaillocalpart => {
if let Some(at) = param.addr.find('@') {
param.addr.split_at(at).0.to_string()
} else {
param.addr.to_string()
}
}
},
})
.collect();
param_autoconfig = Some(servers)
}
}
provider::Status::Broken => {
info!(ctx, "offline autoconfig found, provider is broken");
param_autoconfig = None;
}
param_autoconfig = Some(servers)
}
} else {
// Try receiving autoconfig
info!(ctx, "no offline autoconfig found");
info!(ctx, "No offline autoconfig found.");
param_autoconfig = get_autoconfig(ctx, param, &param_domain).await;
}
} else {
provider = None;
param_autoconfig = None;
}
let user_strict_tls = match param.certificate_checks {
CertificateChecks::Automatic => None,
CertificateChecks::Strict => Some(true),
CertificateChecks::AcceptInvalidCertificates
| CertificateChecks::AcceptInvalidCertificates2 => Some(false),
};
let provider_strict_tls = param.provider.map(|provider| provider.opt.strict_tls);
let strict_tls = user_strict_tls.or(provider_strict_tls).unwrap_or(true);
// Do not save `CertificateChecks::Automatic` into `configured_imap_certificate_checks`.
param.certificate_checks = if strict_tls {
CertificateChecks::Strict
} else {
CertificateChecks::AcceptInvalidCertificates
};
progress!(ctx, 500);
let mut servers = param_autoconfig.unwrap_or_default();
@@ -326,107 +310,125 @@ async fn configure(ctx: &Context, param: &mut LoginParam) -> Result<()> {
let servers = expand_param_vector(servers, &param.addr, &param_domain);
let configured_login_param = ConfiguredLoginParam {
addr,
imap: servers
.iter()
.filter_map(|params| {
let Ok(security) = params.socket.try_into() else {
return None;
};
if params.protocol == Protocol::Imap {
Some(ConfiguredServerLoginParam {
connection: ConnectionCandidate {
host: params.hostname.clone(),
port: params.port,
security,
},
user: params.username.clone(),
})
} else {
None
}
})
.collect(),
imap_user: param.imap.user.clone(),
imap_password: param.imap.password.clone(),
smtp: servers
.iter()
.filter_map(|params| {
let Ok(security) = params.socket.try_into() else {
return None;
};
if params.protocol == Protocol::Smtp {
Some(ConfiguredServerLoginParam {
connection: ConnectionCandidate {
host: params.hostname.clone(),
port: params.port,
security,
},
user: params.username.clone(),
})
} else {
None
}
})
.collect(),
smtp_user: param.smtp.user.clone(),
smtp_password,
proxy_config: param.proxy_config.clone(),
provider,
certificate_checks: match param.certificate_checks {
EnteredCertificateChecks::Automatic => ConfiguredCertificateChecks::Automatic,
EnteredCertificateChecks::Strict => ConfiguredCertificateChecks::Strict,
EnteredCertificateChecks::AcceptInvalidCertificates
| EnteredCertificateChecks::AcceptInvalidCertificates2 => {
ConfiguredCertificateChecks::AcceptInvalidCertificates
}
},
oauth2: param.oauth2,
};
Ok(configured_login_param)
}
async fn configure(ctx: &Context, param: &EnteredLoginParam) -> Result<ConfiguredLoginParam> {
progress!(ctx, 1);
let ctx2 = ctx.clone();
let update_device_chats_handle = task::spawn(async move { ctx2.update_device_chats().await });
let configured_param = get_configured_param(ctx, param).await?;
let strict_tls = configured_param.strict_tls();
progress!(ctx, 550);
// Spawn SMTP configuration task
let mut smtp = Smtp::new();
// to try SMTP while connecting to IMAP.
let context_smtp = ctx.clone();
let mut smtp_param = param.smtp.clone();
let smtp_addr = param.addr.clone();
let smtp_servers: Vec<ServerParams> = servers
.iter()
.filter(|params| params.protocol == Protocol::Smtp)
.cloned()
.collect();
let smtp_param = configured_param.smtp.clone();
let smtp_password = configured_param.smtp_password.clone();
let smtp_addr = configured_param.addr.clone();
let proxy_config = configured_param.proxy_config.clone();
let smtp_config_task = task::spawn(async move {
let mut smtp_configured = false;
let mut errors = Vec::new();
for smtp_server in smtp_servers {
smtp_param.user.clone_from(&smtp_server.username);
smtp_param.server.clone_from(&smtp_server.hostname);
smtp_param.port = smtp_server.port;
smtp_param.security = smtp_server.socket;
let mut smtp = Smtp::new();
smtp.connect(
&context_smtp,
&smtp_param,
&smtp_password,
&proxy_config,
&smtp_addr,
strict_tls,
configured_param.oauth2,
)
.await?;
match try_smtp_one_param(
&context_smtp,
&smtp_param,
&socks5_config,
&smtp_addr,
strict_tls,
&mut smtp,
)
.await
{
Ok(_) => {
smtp_configured = true;
break;
}
Err(e) => errors.push(e),
}
}
if smtp_configured {
Ok(smtp_param)
} else {
Err(errors)
}
Ok::<(), anyhow::Error>(())
});
progress!(ctx, 600);
// Configure IMAP
let mut imap: Option<(Imap, ImapSession)> = None;
let imap_servers: Vec<&ServerParams> = servers
.iter()
.filter(|params| params.protocol == Protocol::Imap)
.collect();
let imap_servers_count = imap_servers.len();
let mut errors = Vec::new();
for (imap_server_index, imap_server) in imap_servers.into_iter().enumerate() {
param.imap.user.clone_from(&imap_server.username);
param.imap.server.clone_from(&imap_server.hostname);
param.imap.port = imap_server.port;
param.imap.security = imap_server.socket;
match try_imap_one_param(
ctx,
&param.imap,
&param.socks5_config,
&param.addr,
strict_tls,
)
.await
{
Ok(configured_imap) => {
imap = Some(configured_imap);
break;
}
Err(e) => errors.push(e),
}
progress!(
ctx,
600 + (800 - 600) * (1 + imap_server_index) / imap_servers_count
);
}
let (mut imap, mut imap_session) = match imap {
Some(imap) => imap,
None => bail!(nicer_configuration_error(ctx, errors).await),
let (_s, r) = async_channel::bounded(1);
let mut imap = Imap::new(
configured_param.imap.clone(),
configured_param.imap_password.clone(),
configured_param.proxy_config.clone(),
&configured_param.addr,
strict_tls,
configured_param.oauth2,
r,
);
let mut imap_session = match imap.connect(ctx).await {
Ok(session) => session,
Err(err) => bail!("{}", nicer_configuration_error(ctx, err.to_string()).await),
};
progress!(ctx, 850);
// Wait for SMTP configuration
match smtp_config_task.await.unwrap() {
Ok(smtp_param) => {
param.smtp = smtp_param;
}
Err(errors) => {
bail!(nicer_configuration_error(ctx, errors).await);
}
}
smtp_config_task.await.unwrap()?;
progress!(ctx, 900);
@@ -474,8 +476,7 @@ async fn configure(ctx: &Context, param: &mut LoginParam) -> Result<()> {
}
}
// the trailing underscore is correct
param.save_as_configured_params(ctx).await?;
configured_param.save_as_configured_params(ctx).await?;
ctx.set_config_internal(Config::ConfiguredTimestamp, Some(&time().to_string()))
.await?;
@@ -493,7 +494,7 @@ async fn configure(ctx: &Context, param: &mut LoginParam) -> Result<()> {
ctx.sql.set_raw_config_bool("configured", true).await?;
Ok(())
Ok(configured_param)
}
/// Retrieve available autoconfigurations.
@@ -502,7 +503,7 @@ async fn configure(ctx: &Context, param: &mut LoginParam) -> Result<()> {
/// B. If we have no configuration yet, search configuration in Thunderbird's central database
async fn get_autoconfig(
ctx: &Context,
param: &LoginParam,
param: &EnteredLoginParam,
param_domain: &str,
) -> Option<Vec<ServerParams>> {
let param_addr_urlencoded = utf8_percent_encode(&param.addr, NON_ALPHANUMERIC).to_string();
@@ -573,140 +574,19 @@ async fn get_autoconfig(
None
}
async fn try_imap_one_param(
context: &Context,
param: &ServerLoginParam,
socks5_config: &Option<Socks5Config>,
addr: &str,
strict_tls: bool,
) -> Result<(Imap, ImapSession), ConfigurationError> {
let inf = format!(
"imap: {}@{}:{} security={} strict_tls={} oauth2={} socks5_config={}",
param.user,
param.server,
param.port,
param.security,
strict_tls,
param.oauth2,
if let Some(socks5_config) = socks5_config {
socks5_config.to_string()
} else {
"None".to_string()
}
);
info!(context, "Trying: {}", inf);
let (_s, r) = async_channel::bounded(1);
let mut imap = match Imap::new(param, socks5_config.clone(), addr, strict_tls, r) {
Err(err) => {
info!(context, "failure: {:#}", err);
return Err(ConfigurationError {
config: inf,
msg: format!("{err:#}"),
});
}
Ok(imap) => imap,
};
match imap.connect(context).await {
Err(err) => {
info!(context, "IMAP failure: {err:#}.");
Err(ConfigurationError {
config: inf,
msg: format!("{err:#}"),
})
}
Ok(session) => {
info!(context, "IMAP success: {inf}.");
Ok((imap, session))
}
}
}
async fn try_smtp_one_param(
context: &Context,
param: &ServerLoginParam,
socks5_config: &Option<Socks5Config>,
addr: &str,
strict_tls: bool,
smtp: &mut Smtp,
) -> Result<(), ConfigurationError> {
let inf = format!(
"smtp: {}@{}:{} security={} strict_tls={} oauth2={} socks5_config={}",
param.user,
param.server,
param.port,
param.security,
strict_tls,
param.oauth2,
if let Some(socks5_config) = socks5_config {
socks5_config.to_string()
} else {
"None".to_string()
}
);
info!(context, "Trying: {}", inf);
if let Err(err) = smtp
.connect(context, param, socks5_config, addr, strict_tls)
.await
async fn nicer_configuration_error(context: &Context, e: String) -> String {
if e.to_lowercase().contains("could not resolve")
|| e.to_lowercase().contains("connection attempts")
|| e.to_lowercase()
.contains("temporary failure in name resolution")
|| e.to_lowercase().contains("name or service not known")
|| e.to_lowercase()
.contains("failed to lookup address information")
{
info!(context, "SMTP failure: {err:#}.");
Err(ConfigurationError {
config: inf,
msg: format!("{err:#}"),
})
} else {
info!(context, "SMTP success: {inf}.");
smtp.disconnect();
Ok(())
}
}
/// Failure to connect and login with email client configuration.
#[derive(Debug, thiserror::Error)]
#[error("Trying {config}…\nError: {msg}")]
pub struct ConfigurationError {
/// Tried configuration description.
config: String,
/// Error message.
msg: String,
}
async fn nicer_configuration_error(context: &Context, errors: Vec<ConfigurationError>) -> String {
let first_err = if let Some(f) = errors.first() {
f
} else {
// This means configuration failed but no errors have been captured. This should never
// happen, but if it does, the user will see classic "Error: no error".
return "no error".to_string();
};
if errors.iter().all(|e| {
e.msg.to_lowercase().contains("could not resolve")
|| e.msg.to_lowercase().contains("no dns resolution results")
|| e.msg
.to_lowercase()
.contains("temporary failure in name resolution")
|| e.msg.to_lowercase().contains("name or service not known")
|| e.msg
.to_lowercase()
.contains("failed to lookup address information")
}) {
return stock_str::error_no_network(context).await;
}
if errors.iter().all(|e| e.msg == first_err.msg) {
return first_err.msg.to_string();
}
errors
.iter()
.map(|e| e.to_string())
.collect::<Vec<String>>()
.join("\n\n")
e
}
#[derive(Debug, thiserror::Error)]
@@ -732,7 +612,9 @@ pub enum Error {
mod tests {
#![allow(clippy::indexing_slicing)]
use super::*;
use crate::config::Config;
use crate::login_param::EnteredServerLoginParam;
use crate::test_utils::TestContext;
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
@@ -744,4 +626,24 @@ mod tests {
t.set_config(Config::MailPw, Some("123456")).await.unwrap();
assert!(t.configure().await.is_err());
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_get_configured_param() -> Result<()> {
let t = &TestContext::new().await;
let entered_param = EnteredLoginParam {
addr: "alice@example.org".to_string(),
imap: EnteredServerLoginParam {
user: "alice@example.net".to_string(),
password: "foobar".to_string(),
..Default::default()
},
..Default::default()
};
let configured_param = get_configured_param(t, &entered_param).await?;
assert_eq!(configured_param.imap_user, "alice@example.net");
assert_eq!(configured_param.smtp_user, "");
Ok(())
}
}
+8 -6
View File
@@ -30,7 +30,6 @@ use crate::context::Context;
use crate::events::EventType;
use crate::key::{load_self_public_key, DcKey, SignedPublicKey};
use crate::log::LogExt;
use crate::login_param::LoginParam;
use crate::message::MessageState;
use crate::mimeparser::AvatarAction;
use crate::param::{Param, Params};
@@ -1191,7 +1190,10 @@ impl Contact {
);
let contact = Contact::get_by_id(context, contact_id).await?;
let loginparam = LoginParam::load_configured_params(context).await?;
let addr = context
.get_config(Config::ConfiguredAddr)
.await?
.unwrap_or_default();
let peerstate = Peerstate::from_addr(context, &contact.addr).await?;
let Some(peerstate) = peerstate.filter(|peerstate| peerstate.peek_key(false).is_some())
@@ -1220,8 +1222,8 @@ impl Contact {
.peek_key(false)
.map(|k| k.fingerprint().to_string())
.unwrap_or_default();
if loginparam.addr < peerstate.addr {
cat_fingerprint(&mut ret, &loginparam.addr, &fingerprint_self, "");
if addr < peerstate.addr {
cat_fingerprint(&mut ret, &addr, &fingerprint_self, "");
cat_fingerprint(
&mut ret,
&peerstate.addr,
@@ -1235,7 +1237,7 @@ impl Contact {
&fingerprint_other_verified,
&fingerprint_other_unverified,
);
cat_fingerprint(&mut ret, &loginparam.addr, &fingerprint_self, "");
cat_fingerprint(&mut ret, &addr, &fingerprint_self, "");
}
Ok(ret)
@@ -2888,7 +2890,7 @@ Hi."#;
bob.recv_msg(&sent_msg).await;
let contact = Contact::get_by_id(&bob, *contacts.first().unwrap()).await?;
let green = ansi_term::Color::Green.normal();
let green = nu_ansi_term::Color::Green.normal();
assert!(
contact.was_seen_recently(),
"{}",
+16 -6
View File
@@ -27,7 +27,7 @@ use crate::download::DownloadState;
use crate::events::{Event, EventEmitter, EventType, Events};
use crate::imap::{FolderMeaning, Imap, ServerMetadata};
use crate::key::{load_self_public_key, load_self_secret_key, DcKey as _};
use crate::login_param::LoginParam;
use crate::login_param::{ConfiguredLoginParam, EnteredLoginParam};
use crate::message::{self, Message, MessageState, MsgId, Viewtype};
use crate::param::{Param, Params};
use crate::peer_channels::Iroh;
@@ -715,8 +715,10 @@ impl Context {
/// Returns information about the context as key-value pairs.
pub async fn get_info(&self) -> Result<BTreeMap<&'static str, String>> {
let unset = "0";
let l = LoginParam::load_candidate_params_unchecked(self).await?;
let l2 = LoginParam::load_configured_params(self).await?;
let l = EnteredLoginParam::load(self).await?;
let l2 = ConfiguredLoginParam::load(self)
.await?
.map_or_else(|| "Not configured".to_string(), |param| param.to_string());
let secondary_addrs = self.get_secondary_self_addrs().await?.join(", ");
let displayname = self.get_config(Config::Displayname).await?;
let chats = get_chat_cnt(self).await?;
@@ -724,7 +726,7 @@ impl Context {
let request_msgs = message::get_request_msg_cnt(self).await;
let contacts = Contact::get_real_cnt(self).await?;
let is_configured = self.get_config_int(Config::Configured).await?;
let socks5_enabled = self.get_config_int(Config::Socks5Enabled).await?;
let proxy_enabled = self.get_config_int(Config::ProxyEnabled).await?;
let dbversion = self
.sql
.get_raw_config_int("dbversion")
@@ -805,9 +807,9 @@ impl Context {
.unwrap_or_else(|| "<unset>".to_string()),
);
res.insert("is_configured", is_configured.to_string());
res.insert("socks5_enabled", socks5_enabled.to_string());
res.insert("proxy_enabled", proxy_enabled.to_string());
res.insert("entered_account_settings", l.to_string());
res.insert("used_account_settings", l2.to_string());
res.insert("used_account_settings", l2);
if let Some(server_id) = &*self.server_id.read().await {
res.insert("imap_server_id", format!("{server_id:?}"));
@@ -1265,6 +1267,12 @@ impl Context {
///
/// If `chat_id` is provided this searches only for messages in this chat, if `chat_id`
/// is `None` this searches messages from all chats.
///
/// NB: Wrt the search in long messages which are shown truncated with the "Show Full Message…"
/// button, we only look at the first several kilobytes. Let's not fix this -- one can send a
/// dictionary in the message that matches any reasonable search request, but the user won't see
/// the match because they should tap on "Show Full Message…" for that. Probably such messages
/// would only clutter search results.
pub async fn search_msgs(&self, chat_id: Option<ChatId>, query: &str) -> Result<Vec<MsgId>> {
let real_query = query.trim().to_lowercase();
if real_query.is_empty() {
@@ -1691,6 +1699,8 @@ mod tests {
"server_flags",
"skip_start_messages",
"smtp_certificate_checks",
"proxy_url", // May contain passwords, don't leak it to the logs.
"socks5_enabled", // SOCKS5 options are deprecated.
"socks5_host",
"socks5_port",
"socks5_user",
+122 -104
View File
@@ -32,15 +32,17 @@ use crate::contact::{Contact, ContactId, Modifier, Origin};
use crate::context::Context;
use crate::events::EventType;
use crate::headerdef::{HeaderDef, HeaderDefMap};
use crate::login_param::{LoginParam, ServerLoginParam};
use crate::login_param::{
prioritize_server_login_params, ConfiguredLoginParam, ConfiguredServerLoginParam,
};
use crate::message::{self, Message, MessageState, MessengerMessage, MsgId, Viewtype};
use crate::mimeparser;
use crate::net::proxy::ProxyConfig;
use crate::oauth2::get_oauth2_access_token;
use crate::receive_imf::{
from_field_to_contact_id, get_prefetch_parent_message, receive_imf_inner, ReceivedMsg,
};
use crate::scheduler::connectivity::ConnectivityStore;
use crate::socks::Socks5Config;
use crate::sql;
use crate::stock_str;
use crate::tools::{self, create_id, duration_to_str};
@@ -73,12 +75,18 @@ pub(crate) struct Imap {
addr: String,
/// Login parameters.
lp: ServerLoginParam,
lp: Vec<ConfiguredServerLoginParam>,
/// Password.
password: String,
/// Proxy configuration.
proxy_config: Option<ProxyConfig>,
/// SOCKS 5 configuration.
socks5_config: Option<Socks5Config>,
strict_tls: bool,
oauth2: bool,
login_failed_once: bool,
pub(crate) connectivity: ConnectivityStore,
@@ -228,31 +236,29 @@ impl Imap {
///
/// `addr` is used to renew token if OAuth2 authentication is used.
pub fn new(
lp: &ServerLoginParam,
socks5_config: Option<Socks5Config>,
lp: Vec<ConfiguredServerLoginParam>,
password: String,
proxy_config: Option<ProxyConfig>,
addr: &str,
strict_tls: bool,
oauth2: bool,
idle_interrupt_receiver: Receiver<()>,
) -> Result<Self> {
if lp.server.is_empty() || lp.user.is_empty() || lp.password.is_empty() {
bail!("Incomplete IMAP connection parameters");
}
let imap = Imap {
) -> Self {
Imap {
idle_interrupt_receiver,
addr: addr.to_string(),
lp: lp.clone(),
socks5_config,
lp,
password,
proxy_config,
strict_tls,
oauth2,
login_failed_once: false,
connectivity: Default::default(),
conn_last_try: UNIX_EPOCH,
conn_backoff_ms: 0,
// 1 connection per minute + a burst of 2.
ratelimit: Ratelimit::new(Duration::new(120, 0), 2.0),
};
Ok(imap)
}
}
/// Creates new disconnected IMAP client using configured parameters.
@@ -260,18 +266,18 @@ impl Imap {
context: &Context,
idle_interrupt_receiver: Receiver<()>,
) -> Result<Self> {
if !context.is_configured().await? {
bail!("IMAP Connect without configured params");
}
let param = LoginParam::load_configured_params(context).await?;
let param = ConfiguredLoginParam::load(context)
.await?
.context("Not configured")?;
let imap = Self::new(
&param.imap,
param.socks5_config.clone(),
param.imap.clone(),
param.imap_password.clone(),
param.proxy_config.clone(),
&param.addr,
param.strict_tls(),
param.oauth2,
idle_interrupt_receiver,
)?;
);
Ok(imap)
}
@@ -283,10 +289,6 @@ impl Imap {
/// instead if you are going to actually use connection rather than trying connection
/// parameters.
pub(crate) async fn connect(&mut self, context: &Context) -> Result<Session> {
if self.lp.server.is_empty() {
bail!("IMAP operation attempted while it is torn down");
}
let now = tools::Time::now();
let until_can_send = max(
min(self.conn_last_try, now)
@@ -328,91 +330,107 @@ impl Imap {
);
self.conn_backoff_ms = max(BACKOFF_MIN_MS, self.conn_backoff_ms);
let connection_res = Client::connect(
context,
self.lp.server.as_ref(),
self.lp.port,
self.strict_tls,
self.socks5_config.clone(),
self.lp.security,
)
.await;
let client = connection_res?;
self.conn_backoff_ms = BACKOFF_MIN_MS;
self.ratelimit.send();
let imap_user: &str = self.lp.user.as_ref();
let imap_pw: &str = self.lp.password.as_ref();
let oauth2 = self.lp.oauth2;
let login_res = if oauth2 {
info!(context, "Logging into IMAP server with OAuth 2");
let addr: &str = self.addr.as_ref();
let token = get_oauth2_access_token(context, addr, imap_pw, true)
.await?
.context("IMAP could not get OAUTH token")?;
let auth = OAuth2 {
user: imap_user.into(),
access_token: token,
let login_params = prioritize_server_login_params(&context.sql, &self.lp, "imap").await?;
let mut first_error = None;
for lp in login_params {
info!(context, "IMAP trying to connect to {}.", &lp.connection);
let connection_candidate = lp.connection.clone();
let client = match Client::connect(
context,
self.proxy_config.clone(),
self.strict_tls,
connection_candidate,
)
.await
{
Ok(client) => client,
Err(err) => {
warn!(context, "IMAP failed to connect: {err:#}.");
first_error.get_or_insert(err);
continue;
}
};
client.authenticate("XOAUTH2", auth).await
} else {
info!(context, "Logging into IMAP server with LOGIN");
client.login(imap_user, imap_pw).await
};
match login_res {
Ok(session) => {
// Store server ID in the context to display in account info.
let mut lock = context.server_id.write().await;
lock.clone_from(&session.capabilities.server_id);
self.conn_backoff_ms = BACKOFF_MIN_MS;
self.ratelimit.send();
self.login_failed_once = false;
context.emit_event(EventType::ImapConnected(format!(
"IMAP-LOGIN as {}",
self.lp.user
)));
self.connectivity.set_connected(context).await;
info!(context, "Successfully logged into IMAP server");
Ok(session)
}
let imap_user: &str = lp.user.as_ref();
let imap_pw: &str = &self.password;
Err(err) => {
let imap_user = self.lp.user.to_owned();
let message = stock_str::cannot_login(context, &imap_user).await;
let login_res = if self.oauth2 {
info!(context, "Logging into IMAP server with OAuth 2.");
let addr: &str = self.addr.as_ref();
warn!(context, "{} ({:#})", message, err);
let token = get_oauth2_access_token(context, addr, imap_pw, true)
.await?
.context("IMAP could not get OAUTH token")?;
let auth = OAuth2 {
user: imap_user.into(),
access_token: token,
};
client.authenticate("XOAUTH2", auth).await
} else {
info!(context, "Logging into IMAP server with LOGIN.");
client.login(imap_user, imap_pw).await
};
let lock = context.wrong_pw_warning_mutex.lock().await;
if self.login_failed_once
&& err.to_string().to_lowercase().contains("authentication")
&& context.get_config_bool(Config::NotifyAboutWrongPw).await?
{
if let Err(e) = context
.set_config_internal(Config::NotifyAboutWrongPw, None)
.await
{
warn!(context, "{:#}", e);
}
drop(lock);
match login_res {
Ok(session) => {
// Store server ID in the context to display in account info.
let mut lock = context.server_id.write().await;
lock.clone_from(&session.capabilities.server_id);
let mut msg = Message::new(Viewtype::Text);
msg.text.clone_from(&message);
if let Err(e) =
chat::add_device_msg_with_importance(context, None, Some(&mut msg), true)
.await
{
warn!(context, "{:#}", e);
}
} else {
self.login_failed_once = true;
self.login_failed_once = false;
context.emit_event(EventType::ImapConnected(format!(
"IMAP-LOGIN as {}",
lp.user
)));
self.connectivity.set_connected(context).await;
info!(context, "Successfully logged into IMAP server");
return Ok(session);
}
Err(format_err!("{}\n\n{:#}", message, err))
Err(err) => {
let imap_user = lp.user.to_owned();
let message = stock_str::cannot_login(context, &imap_user).await;
let err_str = err.to_string();
warn!(context, "IMAP failed to login: {err:#}.");
first_error.get_or_insert(format_err!("{message} ({err:#})"));
let lock = context.wrong_pw_warning_mutex.lock().await;
if self.login_failed_once
&& err_str.to_lowercase().contains("authentication")
&& context.get_config_bool(Config::NotifyAboutWrongPw).await?
{
if let Err(e) = context
.set_config_internal(Config::NotifyAboutWrongPw, None)
.await
{
warn!(context, "{e:#}.");
}
drop(lock);
let mut msg = Message::new(Viewtype::Text);
msg.text.clone_from(&message);
if let Err(e) = chat::add_device_msg_with_importance(
context,
None,
Some(&mut msg),
true,
)
.await
{
warn!(context, "Failed to add device message: {e:#}.");
}
} else {
self.login_failed_once = true;
}
}
}
}
Err(first_error.unwrap_or_else(|| format_err!("No IMAP connection candidates provided")))
}
/// Prepare for IMAP operation.
+96 -65
View File
@@ -1,22 +1,22 @@
use std::net::SocketAddr;
use std::ops::{Deref, DerefMut};
use anyhow::{bail, format_err, Context as _, Result};
use anyhow::{Context as _, Result};
use async_imap::Client as ImapClient;
use async_imap::Session as ImapSession;
use fast_socks5::client::Socks5Stream;
use tokio::io::BufWriter;
use super::capabilities::Capabilities;
use super::session::Session;
use crate::context::Context;
use crate::login_param::{ConnectionCandidate, ConnectionSecurity};
use crate::net::dns::{lookup_host_with_cache, update_connect_timestamp};
use crate::net::proxy::ProxyConfig;
use crate::net::session::SessionStream;
use crate::net::tls::wrap_tls;
use crate::net::update_connection_history;
use crate::net::{connect_tcp_inner, connect_tls_inner};
use crate::provider::Socket;
use crate::socks::Socks5Config;
use crate::net::{
connect_tcp_inner, connect_tls_inner, run_connection_attempts, update_connection_history,
};
use crate::tools::time;
#[derive(Debug)]
@@ -107,62 +107,93 @@ impl Client {
Ok(Session::new(session, capabilities))
}
async fn connection_attempt(
context: Context,
host: String,
security: ConnectionSecurity,
resolved_addr: SocketAddr,
strict_tls: bool,
) -> Result<Self> {
let context = &context;
let host = &host;
info!(
context,
"Attempting IMAP connection to {host} ({resolved_addr})."
);
let res = match security {
ConnectionSecurity::Tls => {
Client::connect_secure(resolved_addr, host, strict_tls).await
}
ConnectionSecurity::Starttls => {
Client::connect_starttls(resolved_addr, host, strict_tls).await
}
ConnectionSecurity::Plain => Client::connect_insecure(resolved_addr).await,
};
match res {
Ok(client) => {
let ip_addr = resolved_addr.ip().to_string();
let port = resolved_addr.port();
let save_cache = match security {
ConnectionSecurity::Tls | ConnectionSecurity::Starttls => strict_tls,
ConnectionSecurity::Plain => false,
};
if save_cache {
update_connect_timestamp(context, host, &ip_addr).await?;
}
update_connection_history(context, "imap", host, port, &ip_addr, time()).await?;
Ok(client)
}
Err(err) => {
warn!(
context,
"Failed to connect to {host} ({resolved_addr}): {err:#}."
);
Err(err)
}
}
}
pub async fn connect(
context: &Context,
host: &str,
port: u16,
proxy_config: Option<ProxyConfig>,
strict_tls: bool,
socks5_config: Option<Socks5Config>,
security: Socket,
candidate: ConnectionCandidate,
) -> Result<Self> {
if let Some(socks5_config) = socks5_config {
let host = &candidate.host;
let port = candidate.port;
let security = candidate.security;
if let Some(proxy_config) = proxy_config {
let client = match security {
Socket::Automatic => bail!("IMAP port security is not configured"),
Socket::Ssl => {
Client::connect_secure_socks5(context, host, port, strict_tls, socks5_config)
ConnectionSecurity::Tls => {
Client::connect_secure_proxy(context, host, port, strict_tls, proxy_config)
.await?
}
Socket::Starttls => {
Client::connect_starttls_socks5(context, host, port, socks5_config, strict_tls)
ConnectionSecurity::Starttls => {
Client::connect_starttls_proxy(context, host, port, proxy_config, strict_tls)
.await?
}
Socket::Plain => {
Client::connect_insecure_socks5(context, host, port, socks5_config).await?
ConnectionSecurity::Plain => {
Client::connect_insecure_proxy(context, host, port, proxy_config).await?
}
};
Ok(client)
} else {
let mut first_error = None;
let load_cache =
strict_tls && (security == Socket::Ssl || security == Socket::Starttls);
for resolved_addr in
lookup_host_with_cache(context, host, port, "imap", load_cache).await?
{
let res = match security {
Socket::Automatic => bail!("IMAP port security is not configured"),
Socket::Ssl => Client::connect_secure(resolved_addr, host, strict_tls).await,
Socket::Starttls => {
Client::connect_starttls(resolved_addr, host, strict_tls).await
}
Socket::Plain => Client::connect_insecure(resolved_addr).await,
};
match res {
Ok(client) => {
let ip_addr = resolved_addr.ip().to_string();
if load_cache {
update_connect_timestamp(context, host, &ip_addr).await?;
}
update_connection_history(context, "imap", host, port, &ip_addr, time())
.await?;
return Ok(client);
}
Err(err) => {
warn!(context, "Failed to connect to {resolved_addr}: {err:#}.");
first_error.get_or_insert(err);
}
}
}
Err(first_error.unwrap_or_else(|| format_err!("no DNS resolution results for {host}")))
let load_cache = match security {
ConnectionSecurity::Tls | ConnectionSecurity::Starttls => strict_tls,
ConnectionSecurity::Plain => false,
};
let connection_futures =
lookup_host_with_cache(context, host, port, "imap", load_cache)
.await?
.into_iter()
.map(|resolved_addr| {
let context = context.clone();
let host = host.to_string();
Self::connection_attempt(context, host, security, resolved_addr, strict_tls)
});
run_connection_attempts(connection_futures).await
}
}
@@ -217,17 +248,17 @@ impl Client {
Ok(client)
}
async fn connect_secure_socks5(
async fn connect_secure_proxy(
context: &Context,
domain: &str,
port: u16,
strict_tls: bool,
socks5_config: Socks5Config,
proxy_config: ProxyConfig,
) -> Result<Self> {
let socks5_stream = socks5_config
let proxy_stream = proxy_config
.connect(context, domain, port, strict_tls)
.await?;
let tls_stream = wrap_tls(strict_tls, domain, alpn(port), socks5_stream).await?;
let tls_stream = wrap_tls(strict_tls, domain, alpn(port), proxy_stream).await?;
let buffered_stream = BufWriter::new(tls_stream);
let session_stream: Box<dyn SessionStream> = Box::new(buffered_stream);
let mut client = Client::new(session_stream);
@@ -238,14 +269,14 @@ impl Client {
Ok(client)
}
async fn connect_insecure_socks5(
async fn connect_insecure_proxy(
context: &Context,
domain: &str,
port: u16,
socks5_config: Socks5Config,
proxy_config: ProxyConfig,
) -> Result<Self> {
let socks5_stream = socks5_config.connect(context, domain, port, false).await?;
let buffered_stream = BufWriter::new(socks5_stream);
let proxy_stream = proxy_config.connect(context, domain, port, false).await?;
let buffered_stream = BufWriter::new(proxy_stream);
let session_stream: Box<dyn SessionStream> = Box::new(buffered_stream);
let mut client = Client::new(session_stream);
let _greeting = client
@@ -255,20 +286,20 @@ impl Client {
Ok(client)
}
async fn connect_starttls_socks5(
async fn connect_starttls_proxy(
context: &Context,
hostname: &str,
port: u16,
socks5_config: Socks5Config,
proxy_config: ProxyConfig,
strict_tls: bool,
) -> Result<Self> {
let socks5_stream = socks5_config
let proxy_stream = proxy_config
.connect(context, hostname, port, strict_tls)
.await?;
// Run STARTTLS command and convert the client back into a stream.
let buffered_socks5_stream = BufWriter::new(socks5_stream);
let mut client = ImapClient::new(buffered_socks5_stream);
let buffered_proxy_stream = BufWriter::new(proxy_stream);
let mut client = ImapClient::new(buffered_proxy_stream);
let _greeting = client
.read_response()
.await
@@ -277,10 +308,10 @@ impl Client {
.run_command_and_check_ok("STARTTLS", None)
.await
.context("STARTTLS command failed")?;
let buffered_socks5_stream = client.into_inner();
let socks5_stream: Socks5Stream<_> = buffered_socks5_stream.into_inner();
let buffered_proxy_stream = client.into_inner();
let proxy_stream = buffered_proxy_stream.into_inner();
let tls_stream = wrap_tls(strict_tls, hostname, &[], socks5_stream)
let tls_stream = wrap_tls(strict_tls, hostname, &[], proxy_stream)
.await
.context("STARTTLS upgrade failed")?;
let buffered_stream = BufWriter::new(tls_stream);
+61 -4
View File
@@ -5,7 +5,6 @@ use std::path::{Path, PathBuf};
use ::pgp::types::KeyTrait;
use anyhow::{bail, ensure, format_err, Context as _, Result};
use deltachat_contact_tools::EmailAddress;
use futures::TryStreamExt;
use futures_lite::FutureExt;
@@ -14,6 +13,7 @@ use tokio_tar::Archive;
use crate::blob::BlobDirContents;
use crate::chat::{self, delete_and_reset_all_device_msgs};
use crate::config::Config;
use crate::context::Context;
use crate::e2ee;
use crate::events::EventType;
@@ -177,10 +177,7 @@ async fn set_self_key(context: &Context, armored: &str, set_default: bool) -> Re
info!(context, "No Autocrypt-Prefer-Encrypt header.");
};
let self_addr = context.get_primary_self_addr().await?;
let addr = EmailAddress::new(&self_addr)?;
let keypair = pgp::KeyPair {
addr,
public: public_key,
secret: private_key,
};
@@ -373,6 +370,9 @@ async fn import_backup_stream_inner<R: tokio::io::AsyncRead + Unpin>(
.await
.context("cannot import unpacked database");
}
if res.is_ok() {
res = adjust_delete_server_after(context).await;
}
fs::remove_file(unpacked_database)
.await
.context("cannot remove unpacked database")
@@ -677,6 +677,7 @@ async fn export_database(
.to_str()
.with_context(|| format!("path {} is not valid unicode", dest.display()))?;
adjust_delete_server_after(context).await?;
context
.sql
.set_raw_config_int("backup_time", timestamp)
@@ -706,6 +707,19 @@ async fn export_database(
.await
}
/// Sets `Config::DeleteServerAfter` to "never" if needed so that new messages are present on the
/// server after a backup restoration or available for all devices in multi-device case.
/// NB: Calling this after a backup import isn't reliable as we can crash in between, but this is a
/// problem only for old backups, new backups already have `DeleteServerAfter` set if necessary.
async fn adjust_delete_server_after(context: &Context) -> Result<()> {
if context.is_chatmail().await? && !context.config_exists(Config::DeleteServerAfter).await? {
context
.set_config(Config::DeleteServerAfter, Some("0"))
.await?;
}
Ok(())
}
#[cfg(test)]
mod tests {
use std::time::Duration;
@@ -891,6 +905,49 @@ mod tests {
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_export_import_chatmail_backup() -> Result<()> {
let backup_dir = tempfile::tempdir().unwrap();
let context1 = &TestContext::new_alice().await;
// Check that the setting is displayed correctly.
assert_eq!(
context1.get_config(Config::DeleteServerAfter).await?,
Some("0".to_string())
);
context1.set_config_bool(Config::IsChatmail, true).await?;
assert_eq!(
context1.get_config(Config::DeleteServerAfter).await?,
Some("1".to_string())
);
assert_eq!(context1.get_config_delete_server_after().await?, Some(0));
imex(context1, ImexMode::ExportBackup, backup_dir.path(), None).await?;
let _event = context1
.evtracker
.get_matching(|evt| matches!(evt, EventType::ImexProgress(1000)))
.await;
let context2 = &TestContext::new().await;
let backup = has_backup(context2, backup_dir.path()).await?;
imex(context2, ImexMode::ImportBackup, backup.as_ref(), None).await?;
let _event = context2
.evtracker
.get_matching(|evt| matches!(evt, EventType::ImexProgress(1000)))
.await;
assert!(context2.is_configured().await?);
assert!(context2.is_chatmail().await?);
for ctx in [context1, context2] {
assert_eq!(
ctx.get_config(Config::DeleteServerAfter).await?,
Some("0".to_string())
);
assert_eq!(ctx.get_config_delete_server_after().await?, None);
}
Ok(())
}
/// This is a regression test for
/// https://github.com/deltachat/deltachat-android/issues/2263
/// where the config cache wasn't reset properly after a backup.
+8 -229
View File
@@ -31,36 +31,24 @@ use std::pin::Pin;
use std::sync::Arc;
use std::task::Poll;
use anyhow::{anyhow, bail, ensure, format_err, Context as _, Result};
use futures_lite::StreamExt;
use anyhow::{bail, Context as _, Result};
use iroh_net::relay::RelayMode;
use iroh_net::Endpoint;
use iroh_old;
use iroh_old::blobs::Collection;
use iroh_old::get::DataStream;
use iroh_old::progress::ProgressEmitter;
use iroh_old::provider::Ticket;
use tokio::fs::{self, File};
use tokio::io::{self, AsyncWriteExt, BufWriter};
use tokio::sync::broadcast::error::RecvError;
use tokio::sync::{broadcast, Mutex};
use tokio::task::{JoinHandle, JoinSet};
use tokio_stream::wrappers::ReadDirStream;
use tokio::fs;
use tokio::task::JoinHandle;
use tokio_util::sync::CancellationToken;
use crate::chat::{add_device_msg, delete_and_reset_all_device_msgs};
use crate::chat::add_device_msg;
use crate::context::Context;
use crate::imex::BlobDirContents;
use crate::message::{Message, Viewtype};
use crate::qr::{self, Qr};
use crate::qr::Qr;
use crate::stock_str::backup_transfer_msg_body;
use crate::tools::{create_id, time, TempPathGuard};
use crate::EventType;
use super::{export_backup_stream, export_database, import_backup_stream, DBFILE_BACKUP_NAME};
const MAX_CONCURRENT_DIALS: u8 = 16;
/// ALPN protocol identifier for the backup transfer protocol.
const BACKUP_ALPN: &[u8] = b"/deltachat/backup";
@@ -279,33 +267,6 @@ impl Future for BackupProvider {
}
}
/// Retrieves backup from a legacy backup provider using iroh 0.4.
pub async fn get_legacy_backup(context: &Context, qr: Qr) -> Result<()> {
ensure!(
matches!(qr, Qr::Backup { .. }),
"QR code for backup must be of type DCBACKUP"
);
ensure!(
!context.is_configured().await?,
"Cannot import backups to accounts in use."
);
// Acquire global "ongoing" mutex.
let cancel_token = context.alloc_ongoing().await?;
let _guard = context.scheduler.pause(context.clone()).await;
info!(
context,
"Running get_backup for {}",
qr::format_backup(&qr)?
);
let res = tokio::select! {
biased;
res = get_backup_inner(context, qr) => res,
_ = cancel_token.recv() => Err(format_err!("cancelled")),
};
context.free_ongoing().await;
res
}
pub async fn get_backup2(
context: &Context,
node_addr: iroh_net::NodeAddr,
@@ -349,202 +310,20 @@ pub async fn get_backup2(
///
/// This is a long running operation which will return only when completed.
///
/// Using [`Qr`] as argument is a bit odd as it only accepts specific variants of it. It
/// does avoid having [`iroh_old::provider::Ticket`] in the primary API however, without
/// Using [`Qr`] as argument is a bit odd as it only accepts specific variant of it. It
/// does avoid having [`iroh_net::NodeAddr`] in the primary API however, without
/// having to revert to untyped bytes.
pub async fn get_backup(context: &Context, qr: Qr) -> Result<()> {
match qr {
Qr::Backup { .. } => get_legacy_backup(context, qr).await?,
Qr::Backup2 {
node_addr,
auth_token,
} => get_backup2(context, node_addr, auth_token).await?,
_ => bail!("QR code for backup must be of type DCBACKUP or DCBACKUP2"),
_ => bail!("QR code for backup must be of type DCBACKUP2"),
}
Ok(())
}
async fn get_backup_inner(context: &Context, qr: Qr) -> Result<()> {
let ticket = match qr {
Qr::Backup { ticket } => ticket,
_ => bail!("QR code for backup must be of type DCBACKUP"),
};
match transfer_from_provider(context, &ticket).await {
Ok(()) => {
context.sql.run_migrations(context).await?;
delete_and_reset_all_device_msgs(context).await?;
context.emit_event(ReceiveProgress::Completed.into());
Ok(())
}
Err(err) => {
// Clean up any blobs we already wrote.
let readdir = fs::read_dir(context.get_blobdir()).await?;
let mut readdir = ReadDirStream::new(readdir);
while let Some(dirent) = readdir.next().await {
if let Ok(dirent) = dirent {
fs::remove_file(dirent.path()).await.ok();
}
}
context.emit_event(ReceiveProgress::Failed.into());
Err(err)
}
}
}
async fn transfer_from_provider(context: &Context, ticket: &Ticket) -> Result<()> {
let progress = ProgressEmitter::new(0, ReceiveProgress::max_blob_progress());
spawn_progress_proxy(context.clone(), progress.subscribe());
let on_connected = || {
context.emit_event(ReceiveProgress::Connected.into());
async { Ok(()) }
};
let on_collection = |collection: &Collection| {
context.emit_event(ReceiveProgress::CollectionReceived.into());
progress.set_total(collection.total_blobs_size());
async { Ok(()) }
};
let jobs = Mutex::new(JoinSet::default());
let on_blob =
|hash, reader, name| on_blob(context, &progress, &jobs, ticket, hash, reader, name);
// Perform the transfer.
let keylog = false; // Do not enable rustls SSLKEYLOGFILE env var functionality
let stats = iroh_old::get::run_ticket(
ticket,
keylog,
MAX_CONCURRENT_DIALS,
on_connected,
on_collection,
on_blob,
)
.await?;
let mut jobs = jobs.lock().await;
while let Some(job) = jobs.join_next().await {
job.context("job failed")?;
}
drop(progress);
info!(
context,
"Backup transfer finished, transfer rate was {} Mbps.",
stats.mbits()
);
Ok(())
}
/// Get callback when a blob is received from the provider.
///
/// This writes the blobs to the blobdir. If the blob is the database it will import it to
/// the database of the current [`Context`].
async fn on_blob(
context: &Context,
progress: &ProgressEmitter,
jobs: &Mutex<JoinSet<()>>,
ticket: &Ticket,
_hash: iroh_old::Hash,
mut reader: DataStream,
name: String,
) -> Result<DataStream> {
ensure!(!name.is_empty(), "Received a nameless blob");
let path = if name.starts_with("db/") {
let context_dir = context
.get_blobdir()
.parent()
.ok_or_else(|| anyhow!("Context dir not found"))?;
let dbfile = context_dir.join(DBFILE_BACKUP_NAME);
if fs::metadata(&dbfile).await.is_ok() {
fs::remove_file(&dbfile).await?;
warn!(context, "Previous database export deleted");
}
dbfile
} else {
ensure!(name.starts_with("blob/"), "malformatted blob name");
let blobname = name.rsplit('/').next().context("malformatted blob name")?;
context.get_blobdir().join(blobname)
};
let mut wrapped_reader = progress.wrap_async_read(&mut reader);
let file = File::create(&path).await?;
let mut file = BufWriter::with_capacity(128 * 1024, file);
io::copy(&mut wrapped_reader, &mut file).await?;
file.flush().await?;
if name.starts_with("db/") {
let context = context.clone();
let token = ticket.token().to_string();
jobs.lock().await.spawn(async move {
if let Err(err) = context.sql.import(&path, token).await {
error!(context, "cannot import database: {:#?}", err);
}
if let Err(err) = fs::remove_file(&path).await {
error!(
context,
"failed to delete database import file '{}': {:#?}",
path.display(),
err,
);
}
});
}
Ok(reader)
}
/// Spawns a task proxying progress events.
///
/// This spawns a tokio task which receives events from the [`ProgressEmitter`] and sends
/// them to the context. The task finishes when the emitter is dropped.
///
/// This could be done directly in the emitter by making it less generic.
fn spawn_progress_proxy(context: Context, mut rx: broadcast::Receiver<u16>) {
tokio::spawn(async move {
loop {
match rx.recv().await {
Ok(step) => context.emit_event(ReceiveProgress::BlobProgress(step).into()),
Err(RecvError::Closed) => break,
Err(RecvError::Lagged(_)) => continue,
}
}
});
}
/// Create [`EventType::ImexProgress`] events using readable names.
///
/// Plus you get warnings if you don't use all variants.
#[derive(Debug)]
enum ReceiveProgress {
Connected,
CollectionReceived,
/// A value between 0 and 85 interpreted as a percentage.
///
/// Other values are already used by the other variants of this enum.
BlobProgress(u16),
Completed,
Failed,
}
impl ReceiveProgress {
/// The maximum value for [`ReceiveProgress::BlobProgress`].
///
/// This only exists to keep this magic value local in this type.
fn max_blob_progress() -> u16 {
85
}
}
impl From<ReceiveProgress> for EventType {
fn from(source: ReceiveProgress) -> Self {
let val = match source {
ReceiveProgress::Connected => 50,
ReceiveProgress::CollectionReceived => 100,
ReceiveProgress::BlobProgress(val) => 100 + 10 * val,
ReceiveProgress::Completed => 1000,
ReceiveProgress::Failed => 0,
};
EventType::ImexProgress(val.into())
}
}
#[cfg(test)]
mod tests {
use std::time::Duration;
+7 -22
View File
@@ -244,7 +244,7 @@ async fn generate_keypair(context: &Context) -> Result<KeyPair> {
let _guard = context.generating_key_mutex.lock().await;
// Check if the key appeared while we were waiting on the lock.
match load_keypair(context, &addr).await? {
match load_keypair(context).await? {
Some(key_pair) => Ok(key_pair),
None => {
let start = tools::Time::now();
@@ -266,10 +266,7 @@ async fn generate_keypair(context: &Context) -> Result<KeyPair> {
}
}
pub(crate) async fn load_keypair(
context: &Context,
addr: &EmailAddress,
) -> Result<Option<KeyPair>> {
pub(crate) async fn load_keypair(context: &Context) -> Result<Option<KeyPair>> {
let res = context
.sql
.query_row_optional(
@@ -287,7 +284,6 @@ pub(crate) async fn load_keypair(
Ok(if let Some((pub_bytes, sec_bytes)) = res {
Some(KeyPair {
addr: addr.clone(),
public: SignedPublicKey::from_slice(&pub_bytes)?,
secret: SignedSecretKey::from_slice(&sec_bytes)?,
})
@@ -337,17 +333,11 @@ pub(crate) async fn store_self_keypair(
KeyPairUse::ReadOnly => false,
};
// `addr` and `is_default` written for compatibility with older versions,
// until new cores are rolled out everywhere.
// otherwise "add second device" or "backup" may break.
// moreover, this allows downgrades to the previous version.
// writing of `addr` and `is_default` can be removed ~ 2024-08
let addr = keypair.addr.to_string();
transaction
.execute(
"INSERT OR REPLACE INTO keypairs (public_key, private_key, addr, is_default)
VALUES (?,?,?,?)",
(&public_key, &secret_key, addr, is_default),
"INSERT OR REPLACE INTO keypairs (public_key, private_key)
VALUES (?,?)",
(&public_key, &secret_key),
)
.context("Failed to insert keypair")?;
@@ -377,15 +367,10 @@ pub(crate) async fn store_self_keypair(
/// This API is used for testing purposes
/// to avoid generating the key in tests.
/// Use import/export APIs instead.
pub async fn preconfigure_keypair(context: &Context, addr: &str, secret_data: &str) -> Result<()> {
let addr = EmailAddress::new(addr)?;
pub async fn preconfigure_keypair(context: &Context, secret_data: &str) -> Result<()> {
let secret = SignedSecretKey::from_asc(secret_data)?.0;
let public = secret.split_public_key()?;
let keypair = KeyPair {
addr,
public,
secret,
};
let keypair = KeyPair { public, secret };
store_self_keypair(context, &keypair, KeyPairUse::Default).await?;
Ok(())
}
-1
View File
@@ -84,7 +84,6 @@ mod scheduler;
pub mod securejoin;
mod simplify;
mod smtp;
mod socks;
pub mod stock_str;
mod sync;
mod timesmearing;
+885 -248
View File
File diff suppressed because it is too large Load Diff
+19
View File
@@ -2357,6 +2357,25 @@ mod tests {
assert_eq!(quoted_msg.get_text(), msg2.quoted_text().unwrap());
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_no_quote() {
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
let bob = &tcm.bob().await;
tcm.send_recv_accept(alice, bob, "Hi!").await;
let msg = tcm
.send_recv(
alice,
bob,
"On 2024-08-28, Alice wrote:\n> A quote.\nNot really.",
)
.await;
assert!(msg.quoted_text().is_none());
assert!(msg.quoted_message(bob).await.unwrap().is_none());
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_unencrypted_quote_encrypted_message() -> Result<()> {
let mut tcm = TestContextManager::new();
+13 -1
View File
@@ -143,7 +143,9 @@ impl MimeFactory {
let mut req_mdn = false;
if chat.is_self_talk() {
recipients.push((from_displayname.to_string(), from_addr.to_string()));
if msg.param.get_cmd() == SystemMessage::AutocryptSetupMessage {
recipients.push((from_displayname.to_string(), from_addr.to_string()));
}
} else if chat.is_mailing_list() {
let list_post = chat
.param
@@ -579,6 +581,16 @@ impl MimeFactory {
"Auto-Submitted".to_string(),
"auto-generated".to_string(),
));
} else if let Loaded::Message { msg, .. } = &self.loaded {
if msg.param.get_cmd() == SystemMessage::SecurejoinMessage {
let step = msg.param.get(Param::Arg).unwrap_or_default();
if step != "vg-request" && step != "vc-request" {
headers.push(Header::new(
"Auto-Submitted".to_string(),
"auto-replied".to_string(),
));
}
}
}
if let Loaded::Message { chat, .. } = &self.loaded {
+18 -18
View File
@@ -17,7 +17,7 @@ use crate::aheader::{Aheader, EncryptPreference};
use crate::blob::BlobObject;
use crate::chat::{add_info_msg, ChatId};
use crate::config::Config;
use crate::constants::{self, Chattype, DC_DESIRED_TEXT_LINES, DC_DESIRED_TEXT_LINE_LEN};
use crate::constants::{self, Chattype};
use crate::contact::{Contact, ContactId, Origin};
use crate::context::Context;
use crate::decrypt::{
@@ -34,7 +34,7 @@ use crate::peerstate::Peerstate;
use crate::simplify::{simplify, SimplifiedText};
use crate::sync::SyncItems;
use crate::tools::{
create_smeared_timestamp, get_filemeta, parse_receive_headers, smeared_time, truncate_by_lines,
create_smeared_timestamp, get_filemeta, parse_receive_headers, smeared_time, truncate_msg_text,
validate_id,
};
use crate::{chatlist_events, location, stock_str, tools};
@@ -1179,22 +1179,11 @@ impl MimeMessage {
(simplified_txt, top_quote)
};
let is_bot = context.get_config_bool(Config::Bot).await?;
let simplified_txt = if is_bot {
simplified_txt
} else {
// Truncate text if it has too many lines
let (simplified_txt, was_truncated) = truncate_by_lines(
simplified_txt,
DC_DESIRED_TEXT_LINES,
DC_DESIRED_TEXT_LINE_LEN,
);
if was_truncated {
self.is_mime_modified = was_truncated;
}
simplified_txt
};
let (simplified_txt, was_truncated) =
truncate_msg_text(context, simplified_txt).await?;
if was_truncated {
self.is_mime_modified = was_truncated;
}
if !simplified_txt.is_empty() || simplified_quote.is_some() {
let mut part = Part {
@@ -3609,6 +3598,17 @@ On 2020-10-25, Bob wrote:
assert!(mimemsg.parts[0].msg.len() <= DC_DESIRED_TEXT_LEN + DC_ELLIPSIS.len());
}
{
let chat = t.get_self_chat().await;
t.send_text(chat.id, &long_txt).await;
let msg = t.get_last_msg_in(chat.id).await;
assert!(msg.has_html());
assert!(
msg.text.matches("just repeated").count() <= DC_DESIRED_TEXT_LEN / REPEAT_TXT.len()
);
assert!(msg.text.len() <= DC_DESIRED_TEXT_LEN + DC_ELLIPSIS.len());
}
t.set_config(Config::Bot, Some("1")).await?;
{
+105 -23
View File
@@ -1,4 +1,5 @@
//! # Common network utilities.
use std::future::Future;
use std::net::SocketAddr;
use std::pin::Pin;
use std::time::Duration;
@@ -6,14 +7,17 @@ use std::time::Duration;
use anyhow::{format_err, Context as _, Result};
use async_native_tls::TlsStream;
use tokio::net::TcpStream;
use tokio::task::JoinSet;
use tokio::time::timeout;
use tokio_io_timeout::TimeoutStream;
use crate::context::Context;
use crate::sql::Sql;
use crate::tools::time;
pub(crate) mod dns;
pub(crate) mod http;
pub(crate) mod proxy;
pub(crate) mod session;
pub(crate) mod tls;
@@ -64,21 +68,22 @@ pub(crate) async fn update_connection_history(
Ok(())
}
/// Returns timestamp of the most recent successful connection
/// to the host and port for given protocol.
pub(crate) async fn load_connection_timestamp(
context: &Context,
sql: &Sql,
alpn: &str,
host: &str,
port: u16,
addr: &str,
addr: Option<&str>,
) -> Result<Option<i64>> {
let timestamp = context
.sql
let timestamp = sql
.query_get_value(
"SELECT timestamp FROM connection_history
WHERE host = ?
AND port = ?
AND alpn = ?
AND addr = ?",
AND addr = IFNULL(?, addr)",
(host, port, alpn, addr),
)
.await?;
@@ -121,6 +126,96 @@ pub(crate) async fn connect_tls_inner(
Ok(tls_stream)
}
/// Runs connection attempt futures.
///
/// Accepts iterator of connection attempt futures
/// and runs them until one of them succeeds
/// or all of them fail.
///
/// If all connection attempts fail, returns the first error.
///
/// This functions starts with one connection attempt and maintains
/// up to five parallel connection attempts if connecting takes time.
pub(crate) async fn run_connection_attempts<O, I, F>(mut futures: I) -> Result<O>
where
I: Iterator<Item = F>,
F: Future<Output = Result<O>> + Send + 'static,
O: Send + 'static,
{
let mut connection_attempt_set = JoinSet::new();
// Start additional connection attempts after 300 ms, 1 s, 5 s and 10 s.
// This way we can have up to 5 parallel connection attempts at the same time.
let mut delay_set = JoinSet::new();
for delay in [
Duration::from_millis(300),
Duration::from_secs(1),
Duration::from_secs(5),
Duration::from_secs(10),
] {
delay_set.spawn(tokio::time::sleep(delay));
}
let mut first_error = None;
let res = loop {
if let Some(fut) = futures.next() {
connection_attempt_set.spawn(fut);
}
tokio::select! {
biased;
res = connection_attempt_set.join_next() => {
match res {
Some(res) => {
match res.context("Failed to join task") {
Ok(Ok(conn)) => {
// Successfully connected.
break Ok(conn);
}
Ok(Err(err)) => {
// Some connection attempt failed.
first_error.get_or_insert(err);
}
Err(err) => {
break Err(err);
}
}
}
None => {
// Out of connection attempts.
//
// Break out of the loop and return error.
break Err(
first_error.unwrap_or_else(|| format_err!("No connection attempts were made"))
);
}
}
},
_ = delay_set.join_next(), if !delay_set.is_empty() => {
// Delay expired.
//
// Don't do anything other than pushing
// another connection attempt into `connection_attempt_set`.
}
}
};
// Abort remaining connection attempts and free resources
// such as OS sockets and `Context` references
// held by connection attempt tasks.
//
// `delay_set` contains just `sleep` tasks
// so no need to await futures there,
// it is enough that futures are aborted
// when the set is dropped.
connection_attempt_set.shutdown().await;
res
}
/// If `load_cache` is true, may use cached DNS results.
/// Because the cache may be poisoned with incorrect results by networks hijacking DNS requests,
/// this option should only be used when connection is authenticated,
@@ -133,22 +228,9 @@ pub(crate) async fn connect_tcp(
port: u16,
load_cache: bool,
) -> Result<Pin<Box<TimeoutStream<TcpStream>>>> {
let mut first_error = None;
for resolved_addr in lookup_host_with_cache(context, host, port, "", load_cache).await? {
match connect_tcp_inner(resolved_addr).await {
Ok(stream) => {
return Ok(stream);
}
Err(err) => {
warn!(
context,
"Failed to connect to {}: {:#}.", resolved_addr, err
);
first_error.get_or_insert(err);
}
}
}
Err(first_error.unwrap_or_else(|| format_err!("no DNS resolution results for {host}")))
let connection_futures = lookup_host_with_cache(context, host, port, "", load_cache)
.await?
.into_iter()
.map(connect_tcp_inner);
run_connection_attempts(connection_futures).await
}
+20 -21
View File
@@ -108,6 +108,10 @@ pub(crate) async fn update_connect_timestamp(
Ok(())
}
/// Preloaded DNS results that can be used in case of DNS server failures.
///
/// See <https://support.delta.chat/t/no-dns-resolution-result/2778> and
/// <https://github.com/deltachat/deltachat-core-rust/issues/4920> for reasons.
static DNS_PRELOAD: Lazy<HashMap<&'static str, Vec<IpAddr>>> = Lazy::new(|| {
HashMap::from([
(
@@ -501,21 +505,6 @@ static DNS_PRELOAD: Lazy<HashMap<&'static str, Vec<IpAddr>>> = Lazy::new(|| {
])
});
/// Load hardcoded cache if everything else fails.
///
/// See <https://support.delta.chat/t/no-dns-resolution-result/2778> and
/// <https://github.com/deltachat/deltachat-core-rust/issues/4920> for reasons.
///
/// In the future we may pre-resolve all provider database addresses
/// and build them in.
fn load_hardcoded_cache(hostname: &str, port: u16) -> Vec<SocketAddr> {
if let Some(ips) = DNS_PRELOAD.get(hostname) {
ips.iter().map(|ip| SocketAddr::new(*ip, port)).collect()
} else {
Vec::new()
}
}
async fn lookup_cache(
context: &Context,
host: &str,
@@ -574,11 +563,16 @@ async fn sort_by_connection_timestamp(
alpn: &str,
host: &str,
) -> Result<Vec<SocketAddr>> {
let mut res: Vec<(Option<i64>, SocketAddr)> = Vec::new();
let mut res: Vec<(Option<i64>, SocketAddr)> = Vec::with_capacity(input.len());
for addr in input {
let timestamp =
load_connection_timestamp(context, alpn, host, addr.port(), &addr.ip().to_string())
.await?;
let timestamp = load_connection_timestamp(
&context.sql,
alpn,
host,
addr.port(),
Some(&addr.ip().to_string()),
)
.await?;
res.push((timestamp, addr));
}
res.sort_by_key(|(ts, _addr)| std::cmp::Reverse(*ts));
@@ -626,8 +620,13 @@ pub(crate) async fn lookup_host_with_cache(
}
}
if resolved_addrs.is_empty() {
return Ok(load_hardcoded_cache(hostname, port));
if let Some(ips) = DNS_PRELOAD.get(hostname) {
for ip in ips {
let addr = SocketAddr::new(*ip, port);
if !resolved_addrs.contains(&addr) {
resolved_addrs.push(addr);
}
}
}
}
+191 -102
View File
@@ -1,21 +1,16 @@
//! # HTTP module.
use std::sync::Arc;
use anyhow::{anyhow, Result};
use anyhow::{anyhow, bail, Context as _, Result};
use bytes::Bytes;
use http_body_util::BodyExt;
use hyper_util::rt::TokioIo;
use mime::Mime;
use once_cell::sync::Lazy;
use serde::Serialize;
use crate::context::Context;
use crate::net::lookup_host_with_cache;
use crate::socks::Socks5Config;
static LETSENCRYPT_ROOT: Lazy<reqwest::tls::Certificate> = Lazy::new(|| {
reqwest::tls::Certificate::from_der(include_bytes!(
"../../assets/root-certificates/letsencrypt/isrgrootx1.der"
))
.unwrap()
});
use crate::net::proxy::ProxyConfig;
use crate::net::session::SessionStream;
use crate::net::tls::wrap_tls;
/// HTTP(S) GET response.
#[derive(Debug)]
@@ -32,48 +27,95 @@ pub struct Response {
/// Retrieves the text contents of URL using HTTP GET request.
pub async fn read_url(context: &Context, url: &str) -> Result<String> {
Ok(read_url_inner(context, url).await?.text().await?)
let response = read_url_blob(context, url).await?;
let text = String::from_utf8_lossy(&response.blob);
Ok(text.to_string())
}
async fn get_http_sender<B>(
context: &Context,
parsed_url: hyper::Uri,
) -> Result<hyper::client::conn::http1::SendRequest<B>>
where
B: hyper::body::Body + 'static + Send,
B::Data: Send,
B::Error: Into<Box<dyn std::error::Error + Send + Sync>>,
{
let scheme = parsed_url.scheme_str().context("URL has no scheme")?;
let host = parsed_url.host().context("URL has no host")?;
let proxy_config_opt = ProxyConfig::load(context).await?;
let stream: Box<dyn SessionStream> = match scheme {
"http" => {
let port = parsed_url.port_u16().unwrap_or(80);
// It is safe to use cached IP addresses
// for HTTPS URLs, but for HTTP URLs
// better resolve from scratch each time to prevent
// cache poisoning attacks from having lasting effects.
let load_cache = false;
if let Some(proxy_config) = proxy_config_opt {
let proxy_stream = proxy_config
.connect(context, host, port, load_cache)
.await?;
Box::new(proxy_stream)
} else {
let tcp_stream = crate::net::connect_tcp(context, host, port, load_cache).await?;
Box::new(tcp_stream)
}
}
"https" => {
let port = parsed_url.port_u16().unwrap_or(443);
let load_cache = true;
let strict_tls = true;
if let Some(proxy_config) = proxy_config_opt {
let proxy_stream = proxy_config
.connect(context, host, port, load_cache)
.await?;
let tls_stream = wrap_tls(strict_tls, host, &[], proxy_stream).await?;
Box::new(tls_stream)
} else {
let tcp_stream = crate::net::connect_tcp(context, host, port, load_cache).await?;
let tls_stream = wrap_tls(strict_tls, host, &[], tcp_stream).await?;
Box::new(tls_stream)
}
}
_ => bail!("Unknown URL scheme"),
};
let io = TokioIo::new(stream);
let (sender, conn) = hyper::client::conn::http1::handshake(io).await?;
tokio::task::spawn(conn);
Ok(sender)
}
/// Retrieves the binary contents of URL using HTTP GET request.
pub async fn read_url_blob(context: &Context, url: &str) -> Result<Response> {
let response = read_url_inner(context, url).await?;
let content_type = response
.headers()
.get(reqwest::header::CONTENT_TYPE)
.and_then(|value| value.to_str().ok())
.and_then(|value| value.parse::<Mime>().ok());
let mimetype = content_type
.as_ref()
.map(|mime| mime.essence_str().to_string());
let encoding = content_type.as_ref().and_then(|mime| {
mime.get_param(mime::CHARSET)
.map(|charset| charset.as_str().to_string())
});
let blob: Vec<u8> = response.bytes().await?.into();
Ok(Response {
blob,
mimetype,
encoding,
})
}
async fn read_url_inner(context: &Context, url: &str) -> Result<reqwest::Response> {
// It is safe to use cached IP addresses
// for HTTPS URLs, but for HTTP URLs
// better resolve from scratch each time to prevent
// cache poisoning attacks from having lasting effects.
let load_cache = url.starts_with("https://");
let client = get_client(context, load_cache).await?;
let mut url = url.to_string();
// Follow up to 10 http-redirects
for _i in 0..10 {
let response = client.get(&url).send().await?;
let parsed_url = url
.parse::<hyper::Uri>()
.with_context(|| format!("Failed to parse URL {url:?}"))?;
let mut sender = get_http_sender(context, parsed_url.clone()).await?;
let authority = parsed_url
.authority()
.context("URL has no authority")?
.clone();
let req = hyper::Request::builder()
.uri(parsed_url.path())
.header(hyper::header::HOST, authority.as_str())
.body(http_body_util::Empty::<Bytes>::new())?;
let response = sender.send_request(req).await?;
if response.status().is_redirection() {
let headers = response.headers();
let header = headers
let header = response
.headers()
.get_all("location")
.iter()
.last()
@@ -84,72 +126,119 @@ async fn read_url_inner(context: &Context, url: &str) -> Result<reqwest::Respons
continue;
}
return Ok(response);
let content_type = response
.headers()
.get("content-type")
.and_then(|value| value.to_str().ok())
.and_then(|value| value.parse::<Mime>().ok());
let mimetype = content_type
.as_ref()
.map(|mime| mime.essence_str().to_string());
let encoding = content_type.as_ref().and_then(|mime| {
mime.get_param(mime::CHARSET)
.map(|charset| charset.as_str().to_string())
});
let body = response.collect().await?.to_bytes();
let blob: Vec<u8> = body.to_vec();
return Ok(Response {
blob,
mimetype,
encoding,
});
}
Err(anyhow!("Followed 10 redirections"))
}
struct CustomResolver {
context: Context,
/// Whether to return cached results or not.
/// If resolver can be used for URLs
/// without TLS, e.g. HTTP URLs from HTML email,
/// this must be false. If TLS is used
/// and certificate hostnames are checked,
/// it is safe to load cache.
load_cache: bool,
}
impl CustomResolver {
fn new(context: Context, load_cache: bool) -> Self {
Self {
context,
load_cache,
}
/// Sends an empty POST request to the URL.
///
/// Returns response text and whether request was successful or not.
///
/// Does not follow redirects.
pub(crate) async fn post_empty(context: &Context, url: &str) -> Result<(String, bool)> {
let parsed_url = url
.parse::<hyper::Uri>()
.with_context(|| format!("Failed to parse URL {url:?}"))?;
let scheme = parsed_url.scheme_str().context("URL has no scheme")?;
if scheme != "https" {
bail!("POST requests to non-HTTPS URLs are not allowed");
}
let mut sender = get_http_sender(context, parsed_url.clone()).await?;
let authority = parsed_url
.authority()
.context("URL has no authority")?
.clone();
let req = hyper::Request::post(parsed_url.path())
.header(hyper::header::HOST, authority.as_str())
.body(http_body_util::Empty::<Bytes>::new())?;
let response = sender.send_request(req).await?;
let response_status = response.status();
let body = response.collect().await?.to_bytes();
let text = String::from_utf8_lossy(&body);
let response_text = text.to_string();
Ok((response_text, response_status.is_success()))
}
impl reqwest::dns::Resolve for CustomResolver {
fn resolve(&self, hostname: reqwest::dns::Name) -> reqwest::dns::Resolving {
let context = self.context.clone();
let load_cache = self.load_cache;
Box::pin(async move {
let port = 443; // Actual port does not matter.
let socket_addrs =
lookup_host_with_cache(&context, hostname.as_str(), port, "", load_cache).await;
match socket_addrs {
Ok(socket_addrs) => {
let addrs: reqwest::dns::Addrs = Box::new(socket_addrs.into_iter());
Ok(addrs)
}
Err(err) => Err(err.into()),
}
})
/// Posts string to the given URL.
///
/// Returns true if successful HTTP response code was returned.
///
/// Does not follow redirects.
#[allow(dead_code)]
pub(crate) async fn post_string(context: &Context, url: &str, body: String) -> Result<bool> {
let parsed_url = url
.parse::<hyper::Uri>()
.with_context(|| format!("Failed to parse URL {url:?}"))?;
let scheme = parsed_url.scheme_str().context("URL has no scheme")?;
if scheme != "https" {
bail!("POST requests to non-HTTPS URLs are not allowed");
}
let mut sender = get_http_sender(context, parsed_url.clone()).await?;
let authority = parsed_url
.authority()
.context("URL has no authority")?
.clone();
let request = hyper::Request::post(parsed_url.path())
.header(hyper::header::HOST, authority.as_str())
.body(body)?;
let response = sender.send_request(request).await?;
Ok(response.status().is_success())
}
pub(crate) async fn get_client(context: &Context, load_cache: bool) -> Result<reqwest::Client> {
let socks5_config = Socks5Config::from_database(&context.sql).await?;
let resolver = Arc::new(CustomResolver::new(context.clone(), load_cache));
/// Sends a POST request with x-www-form-urlencoded data.
///
/// Does not follow redirects.
pub(crate) async fn post_form<T: Serialize + ?Sized>(
context: &Context,
url: &str,
form: &T,
) -> Result<Bytes> {
let parsed_url = url
.parse::<hyper::Uri>()
.with_context(|| format!("Failed to parse URL {url:?}"))?;
let scheme = parsed_url.scheme_str().context("URL has no scheme")?;
if scheme != "https" {
bail!("POST requests to non-HTTPS URLs are not allowed");
}
let builder = reqwest::ClientBuilder::new()
.timeout(super::TIMEOUT)
.add_root_certificate(LETSENCRYPT_ROOT.clone())
.dns_resolver(resolver);
let builder = if let Some(socks5_config) = socks5_config {
let proxy = reqwest::Proxy::all(socks5_config.to_url())?;
builder.proxy(proxy)
} else {
// Disable usage of "system" proxy configured via environment variables.
// It is enabled by default in `reqwest`, see
// <https://docs.rs/reqwest/0.11.14/reqwest/struct.ClientBuilder.html#method.no_proxy>
// for documentation.
builder.no_proxy()
};
Ok(builder.build()?)
let encoded_body = serde_urlencoded::to_string(form).context("Failed to encode data")?;
let mut sender = get_http_sender(context, parsed_url.clone()).await?;
let authority = parsed_url
.authority()
.context("URL has no authority")?
.clone();
let request = hyper::Request::post(parsed_url.path())
.header(hyper::header::HOST, authority.as_str())
.header("content-type", "application/x-www-form-urlencoded")
.body(encoded_body)?;
let response = sender.send_request(request).await?;
let bytes = response.collect().await?.to_bytes();
Ok(bytes)
}
+435
View File
@@ -0,0 +1,435 @@
//! # Proxy support.
//!
//! Delta Chat supports SOCKS5 and Shadowsocks protocols.
use std::fmt;
use std::pin::Pin;
use anyhow::{format_err, Context as _, Result};
use fast_socks5::client::Socks5Stream;
use fast_socks5::util::target_addr::ToTargetAddr;
use fast_socks5::AuthenticationMethod;
use fast_socks5::Socks5Command;
use percent_encoding::{percent_encode, NON_ALPHANUMERIC};
use pin_project::pin_project;
use tokio::io::{AsyncRead, AsyncWrite};
use tokio::net::TcpStream;
use tokio_io_timeout::TimeoutStream;
use crate::config::Config;
use crate::context::Context;
use crate::net::connect_tcp;
use crate::net::session::SessionStream;
use crate::sql::Sql;
/// Default SOCKS5 port according to [RFC 1928](https://tools.ietf.org/html/rfc1928).
pub const DEFAULT_SOCKS_PORT: u16 = 1080;
#[derive(Debug, Clone)]
pub struct ShadowsocksConfig {
pub server_config: shadowsocks::config::ServerConfig,
}
impl PartialEq for ShadowsocksConfig {
fn eq(&self, other: &Self) -> bool {
self.server_config.to_url() == other.server_config.to_url()
}
}
impl Eq for ShadowsocksConfig {}
/// Wrapper for Shadowsocks stream implementing
/// `Debug` and `SessionStream`.
///
/// Passes `AsyncRead` and `AsyncWrite` traits through.
#[pin_project]
pub(crate) struct ShadowsocksStream<S> {
#[pin]
pub(crate) stream: shadowsocks::ProxyClientStream<S>,
}
impl<S> std::fmt::Debug for ShadowsocksStream<S> {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
write!(f, "ShadowsocksStream")
}
}
impl<S> AsyncRead for ShadowsocksStream<S>
where
S: AsyncRead + AsyncWrite + Unpin,
{
fn poll_read(
self: Pin<&mut Self>,
cx: &mut std::task::Context<'_>,
buf: &mut tokio::io::ReadBuf<'_>,
) -> std::task::Poll<std::io::Result<()>> {
self.project().stream.poll_read(cx, buf)
}
}
impl<S> AsyncWrite for ShadowsocksStream<S>
where
S: AsyncRead + AsyncWrite + Unpin,
{
fn poll_write(
self: Pin<&mut Self>,
cx: &mut std::task::Context<'_>,
buf: &[u8],
) -> std::task::Poll<Result<usize, std::io::Error>> {
self.project().stream.poll_write(cx, buf)
}
fn poll_flush(
self: Pin<&mut Self>,
cx: &mut std::task::Context<'_>,
) -> std::task::Poll<Result<(), std::io::Error>> {
self.project().stream.poll_flush(cx)
}
fn poll_shutdown(
self: Pin<&mut Self>,
cx: &mut std::task::Context<'_>,
) -> std::task::Poll<Result<(), std::io::Error>> {
self.project().stream.poll_shutdown(cx)
}
}
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct Socks5Config {
pub host: String,
pub port: u16,
pub user_password: Option<(String, String)>,
}
impl Socks5Config {
async fn connect(
&self,
context: &Context,
target_host: &str,
target_port: u16,
load_dns_cache: bool,
) -> Result<Socks5Stream<Pin<Box<TimeoutStream<TcpStream>>>>> {
let tcp_stream = connect_tcp(context, &self.host, self.port, load_dns_cache)
.await
.context("Failed to connect to SOCKS5 proxy")?;
let authentication_method = if let Some((username, password)) = self.user_password.as_ref()
{
Some(AuthenticationMethod::Password {
username: username.into(),
password: password.into(),
})
} else {
None
};
let mut socks_stream =
Socks5Stream::use_stream(tcp_stream, authentication_method, Default::default()).await?;
let target_addr = (target_host, target_port).to_target_addr()?;
socks_stream
.request(Socks5Command::TCPConnect, target_addr)
.await?;
Ok(socks_stream)
}
}
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum ProxyConfig {
Socks5(Socks5Config),
Shadowsocks(ShadowsocksConfig),
}
impl ProxyConfig {
/// Creates a new proxy configuration by parsing given proxy URL.
fn from_url(url: &str) -> Result<Self> {
let url = url::Url::parse(url).context("Cannot parse proxy URL")?;
match url.scheme() {
"ss" => {
let server_config = shadowsocks::config::ServerConfig::from_url(url.as_str())?;
let shadowsocks_config = ShadowsocksConfig { server_config };
Ok(Self::Shadowsocks(shadowsocks_config))
}
// Because of `curl` convention,
// `socks5` URL scheme may be expected to resolve domain names locally
// with `socks5h` URL scheme meaning that hostnames are passed to the proxy.
// Resolving hostnames locally is not supported
// in Delta Chat when using a proxy
// to prevent DNS leaks.
// Because of this we do not distinguish
// between `socks5` and `socks5h`.
"socks5" => {
let host = url
.host_str()
.context("socks5 URL has no host")?
.to_string();
let port = url.port().unwrap_or(DEFAULT_SOCKS_PORT);
let user_password = if let Some(password) = url.password() {
let username = percent_encoding::percent_decode_str(url.username())
.decode_utf8()
.context("SOCKS5 username is not a valid UTF-8")?
.to_string();
let password = percent_encoding::percent_decode_str(password)
.decode_utf8()
.context("SOCKS5 password is not a valid UTF-8")?
.to_string();
Some((username, password))
} else {
None
};
let socks5_config = Socks5Config {
host,
port,
user_password,
};
Ok(Self::Socks5(socks5_config))
}
scheme => Err(format_err!("Unknown URL scheme {scheme:?}")),
}
}
/// Migrates legacy `socks5_host`, `socks5_port`, `socks5_user` and `socks5_password`
/// config into `proxy_url` if `proxy_url` is unset or empty.
///
/// Unsets `socks5_host`, `socks5_port`, `socks5_user` and `socks5_password` in any case.
async fn migrate_socks_config(sql: &Sql) -> Result<()> {
if sql.get_raw_config("proxy_url").await?.is_none() {
// Load legacy SOCKS5 settings.
if let Some(host) = sql
.get_raw_config("socks5_host")
.await?
.filter(|s| !s.is_empty())
{
let port: u16 = sql
.get_raw_config_int("socks5_port")
.await?
.unwrap_or(DEFAULT_SOCKS_PORT.into()) as u16;
let user = sql.get_raw_config("socks5_user").await?.unwrap_or_default();
let pass = sql
.get_raw_config("socks5_password")
.await?
.unwrap_or_default();
let mut proxy_url = "socks5://".to_string();
if !pass.is_empty() {
proxy_url += &percent_encode(user.as_bytes(), NON_ALPHANUMERIC).to_string();
proxy_url += ":";
proxy_url += &percent_encode(pass.as_bytes(), NON_ALPHANUMERIC).to_string();
proxy_url += "@";
};
proxy_url += &host;
proxy_url += ":";
proxy_url += &port.to_string();
sql.set_raw_config("proxy_url", Some(&proxy_url)).await?;
} else {
sql.set_raw_config("proxy_url", Some("")).await?;
}
let socks5_enabled = sql.get_raw_config("socks5_enabled").await?;
sql.set_raw_config("proxy_enabled", socks5_enabled.as_deref())
.await?;
}
sql.set_raw_config("socks5_enabled", None).await?;
sql.set_raw_config("socks5_host", None).await?;
sql.set_raw_config("socks5_port", None).await?;
sql.set_raw_config("socks5_user", None).await?;
sql.set_raw_config("socks5_password", None).await?;
Ok(())
}
/// Reads proxy configuration from the database.
pub async fn load(context: &Context) -> Result<Option<Self>> {
Self::migrate_socks_config(&context.sql)
.await
.context("Failed to migrate legacy SOCKS config")?;
let enabled = context.get_config_bool(Config::ProxyEnabled).await?;
if !enabled {
return Ok(None);
}
let proxy_url = context
.get_config(Config::ProxyUrl)
.await?
.unwrap_or_default();
let proxy_url = proxy_url
.split_once('\n')
.map_or(proxy_url.clone(), |(first_url, _rest)| {
first_url.to_string()
});
let proxy_config = Self::from_url(&proxy_url).context("Failed to parse proxy URL")?;
Ok(Some(proxy_config))
}
/// If `load_dns_cache` is true, loads cached DNS resolution results.
/// Use this only if the connection is going to be protected with TLS checks.
pub async fn connect(
&self,
context: &Context,
target_host: &str,
target_port: u16,
load_dns_cache: bool,
) -> Result<Box<dyn SessionStream>> {
match self {
ProxyConfig::Socks5(socks5_config) => {
let socks5_stream = socks5_config
.connect(context, target_host, target_port, load_dns_cache)
.await?;
Ok(Box::new(socks5_stream))
}
ProxyConfig::Shadowsocks(ShadowsocksConfig { server_config }) => {
let shadowsocks_context = shadowsocks::context::Context::new_shared(
shadowsocks::config::ServerType::Local,
);
let tcp_stream = {
let server_addr = server_config.addr();
let host = server_addr.host();
let port = server_addr.port();
connect_tcp(context, &host, port, load_dns_cache)
.await
.context("Failed to connect to Shadowsocks proxy")?
};
let proxy_client_stream = shadowsocks::ProxyClientStream::from_stream(
shadowsocks_context,
tcp_stream,
server_config,
(target_host.to_string(), target_port),
);
let shadowsocks_stream = ShadowsocksStream {
stream: proxy_client_stream,
};
Ok(Box::new(shadowsocks_stream))
}
}
}
}
impl fmt::Display for Socks5Config {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
write!(
f,
"host:{},port:{},user_password:{}",
self.host,
self.port,
if let Some(user_password) = self.user_password.clone() {
format!("user: {}, password: ***", user_password.0)
} else {
"user: None".to_string()
}
)
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::config::Config;
use crate::test_utils::TestContext;
#[test]
fn test_socks5_url() {
let proxy_config = ProxyConfig::from_url("socks5://127.0.0.1:9050").unwrap();
assert_eq!(
proxy_config,
ProxyConfig::Socks5(Socks5Config {
host: "127.0.0.1".to_string(),
port: 9050,
user_password: None
})
);
let proxy_config = ProxyConfig::from_url("socks5://foo:bar@127.0.0.1:9150").unwrap();
assert_eq!(
proxy_config,
ProxyConfig::Socks5(Socks5Config {
host: "127.0.0.1".to_string(),
port: 9150,
user_password: Some(("foo".to_string(), "bar".to_string()))
})
);
let proxy_config = ProxyConfig::from_url("socks5://%66oo:b%61r@127.0.0.1:9150").unwrap();
assert_eq!(
proxy_config,
ProxyConfig::Socks5(Socks5Config {
host: "127.0.0.1".to_string(),
port: 9150,
user_password: Some(("foo".to_string(), "bar".to_string()))
})
);
}
#[test]
fn test_shadowsocks_url() {
// Example URL from <https://shadowsocks.org/doc/sip002.html>.
let proxy_config =
ProxyConfig::from_url("ss://YWVzLTEyOC1nY206dGVzdA@192.168.100.1:8888#Example1")
.unwrap();
assert!(matches!(proxy_config, ProxyConfig::Shadowsocks(_)));
}
#[test]
fn test_invalid_proxy_url() {
assert!(ProxyConfig::from_url("foobar://127.0.0.1:9050").is_err());
assert!(ProxyConfig::from_url("abc").is_err());
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_socks5_migration() -> Result<()> {
let t = TestContext::new().await;
// Test that config is migrated on attempt to load even if disabled.
t.set_config(Config::Socks5Host, Some("127.0.0.1")).await?;
t.set_config(Config::Socks5Port, Some("9050")).await?;
let proxy_config = ProxyConfig::load(&t).await?;
// Even though proxy is not enabled, config should be migrated.
assert_eq!(proxy_config, None);
assert_eq!(
t.get_config(Config::ProxyUrl).await?.unwrap(),
"socks5://127.0.0.1:9050"
);
Ok(())
}
// Test SOCKS5 setting migration if proxy was never configured.
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_socks5_migration_unconfigured() -> Result<()> {
let t = TestContext::new().await;
// Try to load config to trigger migration.
assert_eq!(ProxyConfig::load(&t).await?, None);
assert_eq!(t.get_config(Config::ProxyEnabled).await?, None);
assert_eq!(
t.get_config(Config::ProxyUrl).await?.unwrap(),
String::new()
);
Ok(())
}
// Test SOCKS5 setting migration if SOCKS5 host is empty.
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_socks5_migration_empty() -> Result<()> {
let t = TestContext::new().await;
t.set_config(Config::Socks5Host, Some("")).await?;
// Try to load config to trigger migration.
assert_eq!(ProxyConfig::load(&t).await?, None);
assert_eq!(t.get_config(Config::ProxyEnabled).await?, None);
assert_eq!(
t.get_config(Config::ProxyUrl).await?.unwrap(),
String::new()
);
Ok(())
}
}
+6
View File
@@ -1,3 +1,4 @@
use crate::net::proxy::ShadowsocksStream;
use async_native_tls::TlsStream;
use fast_socks5::client::Socks5Stream;
use std::pin::Pin;
@@ -44,6 +45,11 @@ impl<T: SessionStream> SessionStream for Socks5Stream<T> {
self.get_socket_mut().set_read_timeout(timeout)
}
}
impl<T: SessionStream> SessionStream for ShadowsocksStream<T> {
fn set_read_timeout(&mut self, timeout: Option<Duration>) {
self.stream.get_mut().set_read_timeout(timeout)
}
}
/// Session stream with a read buffer.
pub(crate) trait SessionBufStream: SessionStream + AsyncBufRead {}
+41 -80
View File
@@ -2,12 +2,13 @@
use std::collections::HashMap;
use anyhow::Result;
use anyhow::{Context as _, Result};
use percent_encoding::{utf8_percent_encode, NON_ALPHANUMERIC};
use serde::Deserialize;
use crate::config::Config;
use crate::context::Context;
use crate::net::http::post_form;
use crate::net::read_url_blob;
use crate::provider;
use crate::provider::Oauth2Authorizer;
use crate::tools::time;
@@ -60,8 +61,7 @@ pub async fn get_oauth2_url(
addr: &str,
redirect_uri: &str,
) -> Result<Option<String>> {
let socks5_enabled = context.get_config_bool(Config::Socks5Enabled).await?;
if let Some(oauth2) = Oauth2::from_address(context, addr, socks5_enabled).await {
if let Some(oauth2) = Oauth2::from_address(context, addr).await {
context
.sql
.set_raw_config("oauth2_pending_redirect_uri", Some(redirect_uri))
@@ -81,8 +81,7 @@ pub(crate) async fn get_oauth2_access_token(
code: &str,
regenerate: bool,
) -> Result<Option<String>> {
let socks5_enabled = context.get_config_bool(Config::Socks5Enabled).await?;
if let Some(oauth2) = Oauth2::from_address(context, addr, socks5_enabled).await {
if let Some(oauth2) = Oauth2::from_address(context, addr).await {
let lock = context.oauth2_mutex.lock().await;
// read generated token
@@ -159,25 +158,19 @@ pub(crate) async fn get_oauth2_access_token(
// ... and POST
// All OAuth URLs are hardcoded HTTPS URLs,
// so it is safe to load DNS cache.
let load_cache = true;
let client = crate::net::http::get_client(context, load_cache).await?;
let response: Response = match client.post(post_url).form(&post_param).send().await {
Ok(resp) => match resp.json().await {
let response: Response = match post_form(context, post_url, &post_param).await {
Ok(resp) => match serde_json::from_slice(&resp) {
Ok(response) => response,
Err(err) => {
warn!(
context,
"Failed to parse OAuth2 JSON response from {}: error: {}", token_url, err
"Failed to parse OAuth2 JSON response from {token_url}: {err:#}."
);
return Ok(None);
}
},
Err(err) => {
warn!(context, "Error calling OAuth2 at {}: {:?}", token_url, err);
warn!(context, "Error calling OAuth2 at {token_url}: {err:#}.");
return Ok(None);
}
};
@@ -236,8 +229,7 @@ pub(crate) async fn get_oauth2_addr(
addr: &str,
code: &str,
) -> Result<Option<String>> {
let socks5_enabled = context.get_config_bool(Config::Socks5Enabled).await?;
let oauth2 = match Oauth2::from_address(context, addr, socks5_enabled).await {
let oauth2 = match Oauth2::from_address(context, addr).await {
Some(o) => o,
None => return Ok(None),
};
@@ -246,11 +238,20 @@ pub(crate) async fn get_oauth2_addr(
}
if let Some(access_token) = get_oauth2_access_token(context, addr, code, false).await? {
let addr_out = oauth2.get_addr(context, &access_token).await;
let addr_out = match oauth2.get_addr(context, &access_token).await {
Ok(addr) => addr,
Err(err) => {
warn!(context, "Error getting addr: {err:#}.");
None
}
};
if addr_out.is_none() {
// regenerate
if let Some(access_token) = get_oauth2_access_token(context, addr, code, true).await? {
Ok(oauth2.get_addr(context, &access_token).await)
Ok(oauth2
.get_addr(context, &access_token)
.await
.unwrap_or_default())
} else {
Ok(None)
}
@@ -263,8 +264,9 @@ pub(crate) async fn get_oauth2_addr(
}
impl Oauth2 {
async fn from_address(context: &Context, addr: &str, skip_mx: bool) -> Option<Self> {
async fn from_address(context: &Context, addr: &str) -> Option<Self> {
let addr_normalized = normalize_addr(addr);
let skip_mx = true;
if let Some(domain) = addr_normalized
.find('@')
.map(|index| addr_normalized.split_at(index + 1).1)
@@ -282,7 +284,7 @@ impl Oauth2 {
None
}
async fn get_addr(&self, context: &Context, access_token: &str) -> Option<String> {
async fn get_addr(&self, context: &Context, access_token: &str) -> Result<Option<String>> {
let userinfo_url = self.get_userinfo.unwrap_or("");
let userinfo_url = replace_in_uri(userinfo_url, "$ACCESS_TOKEN", access_token);
@@ -294,44 +296,21 @@ impl Oauth2 {
// "picture": "https://lh4.googleusercontent.com/-Gj5jh_9R0BY/AAAAAAAAAAI/AAAAAAAAAAA/IAjtjfjtjNA/photo.jpg"
// }
// All OAuth URLs are hardcoded HTTPS URLs,
// so it is safe to load DNS cache.
let load_cache = true;
let client = match crate::net::http::get_client(context, load_cache).await {
Ok(cl) => cl,
Err(err) => {
warn!(context, "failed to get HTTP client: {}", err);
return None;
}
};
let response = match client.get(userinfo_url).send().await {
Ok(response) => response,
Err(err) => {
warn!(context, "failed to get userinfo: {}", err);
return None;
}
};
let response: Result<HashMap<String, serde_json::Value>, _> = response.json().await;
let parsed = match response {
Ok(parsed) => parsed,
Err(err) => {
warn!(context, "Error getting userinfo: {}", err);
return None;
}
};
let response = read_url_blob(context, &userinfo_url).await?;
let parsed: HashMap<String, serde_json::Value> =
serde_json::from_slice(&response.blob).context("Error getting userinfo")?;
// CAVE: serde_json::Value.as_str() removes the quotes of json-strings
// but serde_json::Value.to_string() does not!
if let Some(addr) = parsed.get("email") {
if let Some(s) = addr.as_str() {
Some(s.to_string())
Ok(Some(s.to_string()))
} else {
warn!(context, "E-mail in userinfo is not a string: {}", addr);
None
Ok(None)
}
} else {
warn!(context, "E-mail missing in userinfo.");
None
Ok(None)
}
}
}
@@ -385,38 +364,20 @@ mod tests {
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_oauth_from_address() {
let t = TestContext::new().await;
assert_eq!(
Oauth2::from_address(&t, "hello@gmail.com", false).await,
Some(OAUTH2_GMAIL)
);
assert_eq!(
Oauth2::from_address(&t, "hello@googlemail.com", false).await,
Some(OAUTH2_GMAIL)
);
assert_eq!(
Oauth2::from_address(&t, "hello@yandex.com", false).await,
Some(OAUTH2_YANDEX)
);
assert_eq!(
Oauth2::from_address(&t, "hello@yandex.ru", false).await,
Some(OAUTH2_YANDEX)
);
assert_eq!(Oauth2::from_address(&t, "hello@web.de", false).await, None);
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_oauth_from_mx() {
// youtube staff seems to use "google workspace with oauth2", figures this out by MX lookup
let t = TestContext::new().await;
// Delta Chat does not have working Gmail client ID anymore.
assert_eq!(Oauth2::from_address(&t, "hello@gmail.com").await, None);
assert_eq!(Oauth2::from_address(&t, "hello@googlemail.com").await, None);
assert_eq!(
Oauth2::from_address(&t, "hello@youtube.com", false).await,
Some(OAUTH2_GMAIL)
Oauth2::from_address(&t, "hello@yandex.com").await,
Some(OAUTH2_YANDEX)
);
// without MX lookup, we would not know as youtube.com is not in our provider-db
assert_eq!(
Oauth2::from_address(&t, "hello@youtube.com", true).await,
None
Oauth2::from_address(&t, "hello@yandex.ru").await,
Some(OAUTH2_YANDEX)
);
assert_eq!(Oauth2::from_address(&t, "hello@web.de").await, None);
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
@@ -432,11 +393,11 @@ mod tests {
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_get_oauth2_url() {
let ctx = TestContext::new().await;
let addr = "dignifiedquire@gmail.com";
let addr = "example@yandex.com";
let redirect_uri = "chat.delta:/com.b44t.messenger";
let res = get_oauth2_url(&ctx.ctx, addr, redirect_uri).await.unwrap();
assert_eq!(res, Some("https://accounts.google.com/o/oauth2/auth?client_id=959970109878%2D4mvtgf6feshskf7695nfln6002mom908%2Eapps%2Egoogleusercontent%2Ecom&redirect_uri=chat%2Edelta%3A%2Fcom%2Eb44t%2Emessenger&response_type=code&scope=https%3A%2F%2Fmail.google.com%2F%20email&access_type=offline".into()));
assert_eq!(res, Some("https://oauth.yandex.com/authorize?client_id=c4d0b6735fc8420a816d7e1303469341&response_type=code&scope=mail%3Aimap_full%20mail%3Asmtp&force_confirm=true".into()));
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
+52 -18
View File
@@ -14,9 +14,7 @@ use pgp::composed::{
use pgp::crypto::ecc_curve::ECCCurve;
use pgp::crypto::hash::HashAlgorithm;
use pgp::crypto::sym::SymmetricKeyAlgorithm;
use pgp::types::{
CompressionAlgorithm, KeyTrait, Mpi, PublicKeyTrait, SecretKeyTrait, StringToKey,
};
use pgp::types::{CompressionAlgorithm, KeyTrait, Mpi, PublicKeyTrait, StringToKey};
use rand::{thread_rng, CryptoRng, Rng};
use tokio::runtime::Handle;
@@ -135,9 +133,6 @@ pub fn split_armored_data(buf: &[u8]) -> Result<(BlockType, BTreeMap<String, Str
/// keys together as they are one unit.
#[derive(Debug, Clone, Eq, PartialEq)]
pub struct KeyPair {
/// Email address.
pub addr: EmailAddress,
/// Public key.
pub public: SignedPublicKey,
@@ -145,6 +140,18 @@ pub struct KeyPair {
pub secret: SignedSecretKey,
}
impl KeyPair {
/// Creates new keypair from a secret key.
///
/// Public key is split off the secret key.
pub fn new(secret: SignedSecretKey) -> Result<Self> {
use crate::key::DcSecretKey;
let public = secret.split_public_key()?;
Ok(Self { public, secret })
}
}
/// Create a new key pair.
///
/// Both secret and public key consist of signing primary key and encryption subkey
@@ -201,19 +208,12 @@ pub(crate) fn create_keypair(addr: EmailAddress, keygen_type: KeyGenType) -> Res
.verify()
.context("invalid secret key generated")?;
let public_key = secret_key
.public_key()
.sign(&secret_key, || "".into())
.context("failed to sign public key")?;
public_key
let key_pair = KeyPair::new(secret_key)?;
key_pair
.public
.verify()
.context("invalid public key generated")?;
Ok(KeyPair {
addr,
public: public_key,
secret: secret_key,
})
Ok(key_pair)
}
/// Select public key or subkey to use for encryption.
@@ -405,7 +405,7 @@ mod tests {
use tokio::sync::OnceCell;
use super::*;
use crate::test_utils::{alice_keypair, bob_keypair};
use crate::test_utils::{alice_keypair, bob_keypair, TestContextManager};
#[test]
fn test_split_armored_data_1() {
@@ -592,4 +592,38 @@ mod tests {
assert_eq!(plain, CLEARTEXT);
assert_eq!(valid_signatures.len(), 0);
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_rotate_encryption_key() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
let bob = &tcm.bob().await;
tcm.send_recv(alice, bob, "Hi!").await;
// TODO: alice changes encryption key
let mut alice_keyring = crate::key::load_self_secret_keyring(alice).await?;
let mut alice_key = alice_keyring.pop().unwrap();
// There should be no public subkeys.
assert!(alice_key.public_subkeys.is_empty());
// There is exactly one secret subkey
// that we are going to replace.
assert_eq!(alice_key.secret_subkeys.len(), 1);
alice_key.secret_subkeys.clear();
let new_subkey = SubkeyParamsBuilder::default()
.key_type(PgpKeyType::ECDH(ECCCurve::Curve25519))
.can_encrypt(true)
.passphrase(None)
.build()
.context("Failed to build subkey parameters")?;
let new_keypair = KeyPair::new(alice_key)?;
tcm.send_recv(alice, bob, "Hi again, with a new key!").await;
Ok(())
}
}
+1 -1
View File
@@ -1,6 +1,6 @@
//! [Provider database](https://providers.delta.chat/) module.
mod data;
pub(crate) mod data;
use anyhow::Result;
use deltachat_contact_tools::EmailAddress;
+90 -21
View File
@@ -509,6 +509,8 @@ static P_FREENET_DE: Provider = Provider {
overview_page: "https://providers.delta.chat/freenet-de",
server: &[
Server { protocol: Imap, socket: Ssl, hostname: "mx.freenet.de", port: 993, username_pattern: Email },
Server { protocol: Imap, socket: Starttls, hostname: "mx.freenet.de", port: 143, username_pattern: Email },
Server { protocol: Smtp, socket: Ssl, hostname: "mx.freenet.de", port: 465, username_pattern: Email },
Server { protocol: Smtp, socket: Starttls, hostname: "mx.freenet.de", port: 587, username_pattern: Email },
],
opt: ProviderOptions::new(),
@@ -532,7 +534,7 @@ static P_GMAIL: Provider = Provider {
..ProviderOptions::new()
},
config_defaults: None,
oauth2_authorizer: Some(Oauth2Authorizer::Gmail),
oauth2_authorizer: None,
};
// gmx.net.md: gmx.net, gmx.de, gmx.at, gmx.ch, gmx.org, gmx.eu, gmx.info, gmx.biz, gmx.com
@@ -874,6 +876,20 @@ static P_MEHL_CLOUD: Provider = Provider {
after_login_hint: "",
overview_page: "https://providers.delta.chat/mehl-cloud",
server: &[
Server {
protocol: Imap,
socket: Ssl,
hostname: "mehl.cloud",
port: 443,
username_pattern: Email,
},
Server {
protocol: Smtp,
socket: Ssl,
hostname: "mehl.cloud",
port: 443,
username_pattern: Email,
},
Server {
protocol: Imap,
socket: Ssl,
@@ -924,6 +940,41 @@ static P_MEHL_STORE: Provider = Provider {
oauth2_authorizer: None,
};
// migadu.md: migadu.com
static P_MIGADU: Provider = Provider {
id: "migadu",
status: Status::Ok,
before_login_hint: "",
after_login_hint: "",
overview_page: "https://providers.delta.chat/migadu",
server: &[
Server {
protocol: Imap,
socket: Ssl,
hostname: "imap.migadu.com",
port: 993,
username_pattern: Email,
},
Server {
protocol: Smtp,
socket: Ssl,
hostname: "smtp.migadu.com",
port: 465,
username_pattern: Email,
},
Server {
protocol: Smtp,
socket: Starttls,
hostname: "smtp.migadu.com",
port: 587,
username_pattern: Email,
},
],
opt: ProviderOptions::new(),
config_defaults: None,
oauth2_authorizer: None,
};
// nauta.cu.md: nauta.cu
static P_NAUTA_CU: Provider = Provider {
id: "nauta.cu",
@@ -1009,6 +1060,20 @@ static P_NINE_TESTRUN_ORG: Provider = Provider {
after_login_hint: "",
overview_page: "https://providers.delta.chat/nine-testrun-org",
server: &[
Server {
protocol: Imap,
socket: Ssl,
hostname: "nine.testrun.org",
port: 443,
username_pattern: Email,
},
Server {
protocol: Smtp,
socket: Ssl,
hostname: "nine.testrun.org",
port: 443,
username_pattern: Email,
},
Server {
protocol: Imap,
socket: Ssl,
@@ -1037,20 +1102,6 @@ static P_NINE_TESTRUN_ORG: Provider = Provider {
port: 587,
username_pattern: Email,
},
Server {
protocol: Imap,
socket: Ssl,
hostname: "nine.testrun.org",
port: 443,
username_pattern: Email,
},
Server {
protocol: Smtp,
socket: Ssl,
hostname: "nine.testrun.org",
port: 443,
username_pattern: Email,
},
],
opt: ProviderOptions::new(),
config_defaults: Some(&[ConfigDefault {
@@ -1144,7 +1195,7 @@ static P_OUVATON_COOP: Provider = Provider {
oauth2_authorizer: None,
};
// posteo.md: posteo.de, posteo.af, posteo.at, posteo.be, posteo.ca, posteo.ch, posteo.cl, posteo.co, posteo.co.uk, posteo.com.br, posteo.cr, posteo.cz, posteo.dk, posteo.ee, posteo.es, posteo.eu, posteo.fi, posteo.gl, posteo.gr, posteo.hn, posteo.hr, posteo.hu, posteo.ie, posteo.in, posteo.is, posteo.it, posteo.jp, posteo.la, posteo.li, posteo.lt, posteo.lu, posteo.me, posteo.mx, posteo.my, posteo.net, posteo.nl, posteo.no, posteo.nz, posteo.org, posteo.pe, posteo.pl, posteo.pm, posteo.pt, posteo.ro, posteo.ru, posteo.se, posteo.sg, posteo.si, posteo.tn, posteo.uk, posteo.us
// posteo.md: posteo.de, posteo.af, posteo.at, posteo.be, posteo.ca, posteo.ch, posteo.cl, posteo.co, posteo.co.uk, posteo.com, posteo.com.br, posteo.cr, posteo.cz, posteo.dk, posteo.ee, posteo.es, posteo.eu, posteo.fi, posteo.gl, posteo.gr, posteo.hn, posteo.hr, posteo.hu, posteo.ie, posteo.in, posteo.is, posteo.it, posteo.jp, posteo.la, posteo.li, posteo.lt, posteo.lu, posteo.me, posteo.mx, posteo.my, posteo.net, posteo.nl, posteo.no, posteo.nz, posteo.org, posteo.pe, posteo.pl, posteo.pm, posteo.pt, posteo.ro, posteo.ru, posteo.se, posteo.sg, posteo.si, posteo.tn, posteo.uk, posteo.us
static P_POSTEO: Provider = Provider {
id: "posteo",
status: Status::Ok,
@@ -1518,11 +1569,26 @@ static P_TUTANOTA: Provider = Provider {
// ukr.net.md: ukr.net
static P_UKR_NET: Provider = Provider {
id: "ukr.net",
status: Status::Ok,
before_login_hint: "",
status: Status::Preparation,
before_login_hint: "You must allow IMAP access to your account before you can login.",
after_login_hint: "",
overview_page: "https://providers.delta.chat/ukr-net",
server: &[],
server: &[
Server {
protocol: Imap,
socket: Ssl,
hostname: "imap.ukr.net",
port: 993,
username_pattern: Email,
},
Server {
protocol: Smtp,
socket: Ssl,
hostname: "smtp.ukr.net",
port: 465,
username_pattern: Email,
},
],
opt: ProviderOptions::new(),
config_defaults: None,
oauth2_authorizer: None,
@@ -1804,7 +1870,7 @@ static P_ZOHO: Provider = Provider {
oauth2_authorizer: None,
};
pub(crate) static PROVIDER_DATA: [(&str, &Provider); 531] = [
pub(crate) static PROVIDER_DATA: [(&str, &Provider); 533] = [
("163.com", &P_163),
("aktivix.org", &P_AKTIVIX_ORG),
("aliyun.com", &P_ALIYUN),
@@ -2177,6 +2243,7 @@ pub(crate) static PROVIDER_DATA: [(&str, &Provider); 531] = [
("ente.quest", &P_MEHL_STORE),
("ente.cfd", &P_MEHL_STORE),
("nein.jetzt", &P_MEHL_STORE),
("migadu.com", &P_MIGADU),
("nauta.cu", &P_NAUTA_CU),
("naver.com", &P_NAVER),
("nine.testrun.org", &P_NINE_TESTRUN_ORG),
@@ -2197,6 +2264,7 @@ pub(crate) static PROVIDER_DATA: [(&str, &Provider); 531] = [
("posteo.cl", &P_POSTEO),
("posteo.co", &P_POSTEO),
("posteo.co.uk", &P_POSTEO),
("posteo.com", &P_POSTEO),
("posteo.com.br", &P_POSTEO),
("posteo.cr", &P_POSTEO),
("posteo.cz", &P_POSTEO),
@@ -2379,6 +2447,7 @@ pub(crate) static PROVIDER_IDS: Lazy<HashMap<&'static str, &'static Provider>> =
("mailo.com", &P_MAILO_COM),
("mehl.cloud", &P_MEHL_CLOUD),
("mehl.store", &P_MEHL_STORE),
("migadu", &P_MIGADU),
("nauta.cu", &P_NAUTA_CU),
("naver", &P_NAVER),
("nine.testrun.org", &P_NINE_TESTRUN_ORG),
@@ -2417,4 +2486,4 @@ pub(crate) static PROVIDER_IDS: Lazy<HashMap<&'static str, &'static Provider>> =
});
pub static _PROVIDER_UPDATED: Lazy<chrono::NaiveDate> =
Lazy::new(|| chrono::NaiveDate::from_ymd_opt(2024, 8, 14).unwrap());
Lazy::new(|| chrono::NaiveDate::from_ymd_opt(2024, 9, 13).unwrap());
+7 -10
View File
@@ -61,16 +61,13 @@ impl PushSubscriber {
return Ok(());
};
let load_cache = true;
let response = http::get_client(context, load_cache)
.await?
.post("https://notifications.delta.chat/register")
.body(format!("{{\"token\":\"{token}\"}}"))
.send()
.await?;
let response_status = response.status();
if response_status.is_success() {
if http::post_string(
context,
"https://notifications.delta.chat/register",
format!("{{\"token\":\"{token}\"}}"),
)
.await?
{
state.heartbeat_subscribed = true;
}
Ok(())
+212 -56
View File
@@ -7,11 +7,11 @@ use anyhow::{anyhow, bail, ensure, Context as _, Result};
pub use dclogin_scheme::LoginOptions;
use deltachat_contact_tools::{addr_normalize, may_be_valid_addr, ContactAddress};
use once_cell::sync::Lazy;
use percent_encoding::percent_decode_str;
use percent_encoding::{percent_decode_str, percent_encode, NON_ALPHANUMERIC};
use serde::Deserialize;
use self::dclogin_scheme::configure_from_login_qr;
use crate::chat::{get_chat_id_by_grpid, ChatIdBlocked};
use crate::chat::ChatIdBlocked;
use crate::config::Config;
use crate::constants::Blocked;
use crate::contact::{Contact, ContactId, Origin};
@@ -19,10 +19,11 @@ use crate::context::Context;
use crate::events::EventType;
use crate::key::Fingerprint;
use crate::message::Message;
use crate::net::http::post_empty;
use crate::net::proxy::DEFAULT_SOCKS_PORT;
use crate::peerstate::Peerstate;
use crate::token;
use crate::tools::validate_id;
use iroh_old as iroh;
const OPENPGP4FPR_SCHEME: &str = "OPENPGP4FPR:"; // yes: uppercase
const IDELTACHAT_SCHEME: &str = "https://i.delta.chat/#";
@@ -30,6 +31,7 @@ const IDELTACHAT_NOSLASH_SCHEME: &str = "https://i.delta.chat#";
const DCACCOUNT_SCHEME: &str = "DCACCOUNT:";
pub(super) const DCLOGIN_SCHEME: &str = "DCLOGIN:";
const DCWEBRTC_SCHEME: &str = "DCWEBRTC:";
const TG_SOCKS_SCHEME: &str = "https://t.me/socks";
const MAILTO_SCHEME: &str = "mailto:";
const MATMSG_SCHEME: &str = "MATMSG:";
const VCARD_SCHEME: &str = "BEGIN:VCARD";
@@ -37,9 +39,6 @@ const SMTP_SCHEME: &str = "SMTP:";
const HTTP_SCHEME: &str = "http://";
const HTTPS_SCHEME: &str = "https://";
/// Legacy backup transfer based on iroh 0.4.
pub(crate) const DCBACKUP_SCHEME: &str = "DCBACKUP:";
/// Backup transfer based on iroh-net.
pub(crate) const DCBACKUP2_SCHEME: &str = "DCBACKUP2:";
@@ -110,20 +109,6 @@ pub enum Qr {
domain: String,
},
/// Provides a backup that can be retrieved using legacy iroh 0.4.
///
/// This contains all the data needed to connect to a device and download a backup from
/// it to configure the receiving device with the same account.
Backup {
/// Printable version of the provider information.
///
/// This is the printable version of a `sendme` ticket, which contains all the
/// information to connect to and authenticate a backup provider.
///
/// The format is somewhat opaque, but `sendme` can deserialise this.
ticket: iroh::provider::Ticket,
},
/// Provides a backup that can be retrieved using iroh-net based backup transfer protocol.
Backup2 {
/// Iroh node address.
@@ -142,6 +127,21 @@ pub enum Qr {
instance_pattern: String,
},
/// Ask the user if they want to add or use the given SOCKS5 proxy
Socks5Proxy {
/// SOCKS5 server
host: String,
/// SOCKS5 port
port: u16,
/// SOCKS5 user
user: Option<String>,
/// SOCKS5 password
pass: Option<String>,
},
/// Contact address is scanned.
///
/// Optionally, a draft message could be provided.
@@ -277,8 +277,8 @@ pub async fn check_qr(context: &Context, qr: &str) -> Result<Qr> {
dclogin_scheme::decode_login(qr)?
} else if starts_with_ignore_case(qr, DCWEBRTC_SCHEME) {
decode_webrtc_instance(context, qr)?
} else if starts_with_ignore_case(qr, DCBACKUP_SCHEME) {
decode_backup(qr)?
} else if starts_with_ignore_case(qr, TG_SOCKS_SCHEME) {
decode_tg_socks_proxy(context, qr)?
} else if starts_with_ignore_case(qr, DCBACKUP2_SCHEME) {
decode_backup2(qr)?
} else if qr.starts_with(MAILTO_SCHEME) {
@@ -301,7 +301,7 @@ pub async fn check_qr(context: &Context, qr: &str) -> Result<Qr> {
Ok(qrcode)
}
/// Formats the text of the [`Qr::Backup`] variant.
/// Formats the text of the [`Qr::Backup2`] variant.
///
/// This is the inverse of [`check_qr`] for that variant only.
///
@@ -309,7 +309,6 @@ pub async fn check_qr(context: &Context, qr: &str) -> Result<Qr> {
/// into `FromStr`.
pub fn format_backup(qr: &Qr) -> Result<String> {
match qr {
Qr::Backup { ref ticket } => Ok(format!("{DCBACKUP_SCHEME}{ticket}")),
Qr::Backup2 {
ref node_addr,
ref auth_token,
@@ -539,16 +538,36 @@ fn decode_webrtc_instance(_context: &Context, qr: &str) -> Result<Qr> {
}
}
/// Decodes a [`DCBACKUP_SCHEME`] QR code.
///
/// The format of this scheme is `DCBACKUP:<encoded ticket>`. The encoding is the
/// [`iroh::provider::Ticket`]'s `Display` impl.
fn decode_backup(qr: &str) -> Result<Qr> {
let payload = qr
.strip_prefix(DCBACKUP_SCHEME)
.ok_or_else(|| anyhow!("invalid DCBACKUP scheme"))?;
let ticket: iroh::provider::Ticket = payload.parse().context("invalid DCBACKUP payload")?;
Ok(Qr::Backup { ticket })
/// scheme: `https://t.me/socks?server=foo&port=123` or `https://t.me/socks?server=1.2.3.4&port=123`
fn decode_tg_socks_proxy(_context: &Context, qr: &str) -> Result<Qr> {
let url = url::Url::parse(qr).context("Invalid t.me/socks url")?;
let mut host: Option<String> = None;
let mut port: u16 = DEFAULT_SOCKS_PORT;
let mut user: Option<String> = None;
let mut pass: Option<String> = None;
for (key, value) in url.query_pairs() {
if key == "server" {
host = Some(value.to_string());
} else if key == "port" {
port = value.parse().unwrap_or(DEFAULT_SOCKS_PORT);
} else if key == "user" {
user = Some(value.to_string());
} else if key == "pass" {
pass = Some(value.to_string());
}
}
if let Some(host) = host {
Ok(Qr::Socks5Proxy {
host,
port,
user,
pass,
})
} else {
bail!("Bad t.me/socks url: {:?}", url);
}
}
/// Decodes a [`DCBACKUP2_SCHEME`] QR code.
@@ -594,21 +613,8 @@ async fn set_account_from_qr(context: &Context, qr: &str) -> Result<()> {
bail!("DCACCOUNT QR codes must use HTTPS scheme");
}
// As only HTTPS is used, it is safe to load DNS cache.
let load_cache = true;
let response = crate::net::http::get_client(context, load_cache)
.await?
.post(url_str)
.send()
.await?;
let response_status = response.status();
let response_text = response
.text()
.await
.context("Cannot create account, request failed: empty response")?;
if response_status.is_success() {
let (response_text, response_success) = post_empty(context, url_str).await?;
if response_success {
let CreateAccountSuccessResponse { password, email } = serde_json::from_str(&response_text)
.with_context(|| {
format!("Cannot create account, response is malformed:\n{response_text:?}")
@@ -649,6 +655,40 @@ pub async fn set_config_from_qr(context: &Context, qr: &str) -> Result<()> {
.set_config_internal(Config::WebrtcInstance, Some(&instance_pattern))
.await?;
}
Qr::Socks5Proxy {
host,
port,
user,
pass,
} => {
let mut proxy_url = "socks5://".to_string();
if let Some(pass) = pass {
proxy_url += &percent_encode(user.unwrap_or_default().as_bytes(), NON_ALPHANUMERIC)
.to_string();
proxy_url += ":";
proxy_url += &percent_encode(pass.as_bytes(), NON_ALPHANUMERIC).to_string();
proxy_url += "@";
};
proxy_url += &host;
proxy_url += ":";
proxy_url += &port.to_string();
let old_proxy_url_value = context
.get_config(Config::ProxyUrl)
.await?
.unwrap_or_default();
let proxy_urls: Vec<&str> = std::iter::once(proxy_url.as_str())
.chain(
old_proxy_url_value
.split('\n')
.filter(|s| !s.is_empty() && *s != proxy_url),
)
.collect();
context
.set_config(Config::ProxyUrl, Some(&proxy_urls.join("\n")))
.await?;
context.set_config_bool(Config::ProxyEnabled, true).await?;
}
Qr::WithdrawVerifyContact {
invitenumber,
authcode,
@@ -687,18 +727,15 @@ pub async fn set_config_from_qr(context: &Context, qr: &str) -> Result<()> {
grpid,
..
} => {
let chat_id = get_chat_id_by_grpid(context, &grpid)
.await?
.map(|(chat_id, _protected, _blocked)| chat_id);
token::save(
context,
token::Namespace::InviteNumber,
chat_id,
Some(&grpid),
&invitenumber,
)
.await?;
token::save(context, token::Namespace::Auth, chat_id, &authcode).await?;
context.sync_qr_code_tokens(chat_id).await?;
token::save(context, token::Namespace::Auth, Some(&grpid), &authcode).await?;
context.sync_qr_code_tokens(Some(&grpid)).await?;
context.scheduler.interrupt_smtp().await;
}
Qr::Login { address, options } => {
@@ -870,6 +907,7 @@ mod tests {
use super::*;
use crate::aheader::EncryptPreference;
use crate::chat::{create_group_chat, ProtectionStatus};
use crate::config::Config;
use crate::key::DcKey;
use crate::securejoin::get_securejoin_qr;
use crate::test_utils::{alice_keypair, TestContext};
@@ -1478,6 +1516,73 @@ mod tests {
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_decode_tg_socks_proxy() -> Result<()> {
let t = TestContext::new().await;
let qr = check_qr(&t, "https://t.me/socks?server=84.53.239.95&port=4145").await?;
assert_eq!(
qr,
Qr::Socks5Proxy {
host: "84.53.239.95".to_string(),
port: 4145,
user: None,
pass: None,
}
);
let qr = check_qr(&t, "https://t.me/socks?server=foo.bar&port=123").await?;
assert_eq!(
qr,
Qr::Socks5Proxy {
host: "foo.bar".to_string(),
port: 123,
user: None,
pass: None,
}
);
let qr = check_qr(&t, "https://t.me/socks?server=foo.baz").await?;
assert_eq!(
qr,
Qr::Socks5Proxy {
host: "foo.baz".to_string(),
port: 1080,
user: None,
pass: None,
}
);
let qr = check_qr(
&t,
"https://t.me/socks?server=foo.baz&port=12345&user=ada&pass=ms%21%2F%24",
)
.await?;
assert_eq!(
qr,
Qr::Socks5Proxy {
host: "foo.baz".to_string(),
port: 12345,
user: Some("ada".to_string()),
pass: Some("ms!/$".to_string()),
}
);
// wrong domain results in Qr:Url instead of Qr::Socks5Proxy
let qr = check_qr(&t, "https://not.me/socks?noserver=84.53.239.95&port=4145").await?;
assert_eq!(
qr,
Qr::Url {
url: "https://not.me/socks?noserver=84.53.239.95&port=4145".to_string()
}
);
let qr = check_qr(&t, "https://t.me/socks?noserver=84.53.239.95&port=4145").await;
assert!(qr.is_err());
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_decode_account_bad_scheme() {
let ctx = TestContext::new().await;
@@ -1498,7 +1603,7 @@ mod tests {
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_set_config_from_qr() -> Result<()> {
async fn test_set_webrtc_instance_config_from_qr() -> Result<()> {
let ctx = TestContext::new().await;
assert!(ctx.ctx.get_config(Config::WebrtcInstance).await?.is_none());
@@ -1528,4 +1633,55 @@ mod tests {
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_set_socks5_proxy_config_from_qr() -> Result<()> {
let t = TestContext::new().await;
assert_eq!(t.get_config_bool(Config::ProxyEnabled).await?, false);
let res = set_config_from_qr(&t, "https://t.me/socks?server=foo&port=666").await;
assert!(res.is_ok());
assert_eq!(t.get_config_bool(Config::ProxyEnabled).await?, true);
assert_eq!(
t.get_config(Config::ProxyUrl).await?,
Some("socks5://foo:666".to_string())
);
// Test URL without port.
let res = set_config_from_qr(&t, "https://t.me/socks?server=1.2.3.4").await;
assert!(res.is_ok());
assert_eq!(t.get_config_bool(Config::ProxyEnabled).await?, true);
assert_eq!(
t.get_config(Config::ProxyUrl).await?,
Some("socks5://1.2.3.4:1080\nsocks5://foo:666".to_string())
);
// make sure, user&password are set when specified in the URL
// Password is an URL-encoded "x&%$X".
let res =
set_config_from_qr(&t, "https://t.me/socks?server=jau&user=Da&pass=x%26%25%24X").await;
assert!(res.is_ok());
assert_eq!(
t.get_config(Config::ProxyUrl).await?,
Some(
"socks5://Da:x%26%25%24X@jau:1080\nsocks5://1.2.3.4:1080\nsocks5://foo:666"
.to_string()
)
);
// Scanning existing proxy brings it to the top in the list.
let res = set_config_from_qr(&t, "https://t.me/socks?server=foo&port=666").await;
assert!(res.is_ok());
assert_eq!(t.get_config_bool(Config::ProxyEnabled).await?, true);
assert_eq!(
t.get_config(Config::ProxyUrl).await?,
Some(
"socks5://foo:666\nsocks5://Da:x%26%25%24X@jau:1080\nsocks5://1.2.3.4:1080"
.to_string()
)
);
Ok(())
}
}
+10 -9
View File
@@ -8,7 +8,7 @@ use num_traits::cast::ToPrimitive;
use super::{Qr, DCLOGIN_SCHEME};
use crate::config::Config;
use crate::context::Context;
use crate::login_param::CertificateChecks;
use crate::login_param::EnteredCertificateChecks;
use crate::provider::Socket;
/// Options for `dclogin:` scheme.
@@ -55,7 +55,7 @@ pub enum LoginOptions {
smtp_security: Option<Socket>,
/// Certificate checks.
certificate_checks: Option<CertificateChecks>,
certificate_checks: Option<EnteredCertificateChecks>,
},
}
@@ -72,7 +72,7 @@ pub(super) fn decode_login(qr: &str) -> Result<Qr> {
.unwrap_or(url_without_scheme);
let addr = payload
.split(|c| c == '?' || c == '/')
.split(['?', '/'])
.next()
.context("invalid DCLOGIN payload E3")?;
@@ -146,11 +146,12 @@ fn parse_socket_security(security: Option<&String>) -> Result<Option<Socket>> {
fn parse_certificate_checks(
certificate_checks: Option<&String>,
) -> Result<Option<CertificateChecks>> {
) -> Result<Option<EnteredCertificateChecks>> {
Ok(match certificate_checks.map(|s| s.as_str()) {
Some("0") => Some(CertificateChecks::Automatic),
Some("1") => Some(CertificateChecks::Strict),
Some("3") => Some(CertificateChecks::AcceptInvalidCertificates),
Some("0") => Some(EnteredCertificateChecks::Automatic),
Some("1") => Some(EnteredCertificateChecks::Strict),
Some("2") => Some(EnteredCertificateChecks::AcceptInvalidCertificates),
Some("3") => Some(EnteredCertificateChecks::AcceptInvalidCertificates2),
Some(other) => bail!("Unknown certificatecheck level: {}", other),
None => None,
})
@@ -263,7 +264,7 @@ mod test {
use anyhow::bail;
use super::{decode_login, LoginOptions};
use crate::{login_param::CertificateChecks, provider::Socket, qr::Qr};
use crate::{login_param::EnteredCertificateChecks, provider::Socket, qr::Qr};
macro_rules! login_options_just_pw {
($pw: expr) => {
@@ -386,7 +387,7 @@ mod test {
smtp_username: Some("max@host.tld".to_owned()),
smtp_password: Some("3242HS".to_owned()),
smtp_security: Some(Socket::Plain),
certificate_checks: Some(CertificateChecks::Strict),
certificate_checks: Some(EnteredCertificateChecks::Strict),
}
);
} else {
+13
View File
@@ -2107,6 +2107,19 @@ async fn test_no_unencrypted_name_in_self_chat() -> Result<()> {
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_no_smtp_job_for_self_chat() -> Result<()> {
let mut tcm = TestContextManager::new();
let bob = &tcm.bob().await;
bob.set_config_bool(Config::BccSelf, false).await?;
let chat_id = bob.get_self_chat().await.id;
let mut msg = Message::new(Viewtype::Text);
msg.text = "Happy birthday to me".to_string();
chat::send_msg(bob, chat_id, &mut msg).await?;
assert!(bob.pop_sent_msg_opt(Duration::ZERO).await.is_none());
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_outgoing_classic_mail_creates_chat() {
let alice = TestContext::new_alice().await;
+114 -91
View File
@@ -1,13 +1,13 @@
//! Implementation of [SecureJoin protocols](https://securejoin.delta.chat/).
use anyhow::{bail, Context as _, Error, Result};
use anyhow::{bail, ensure, Context as _, Error, Result};
use percent_encoding::{utf8_percent_encode, AsciiSet, NON_ALPHANUMERIC};
use crate::aheader::EncryptPreference;
use crate::chat::{self, Chat, ChatId, ChatIdBlocked, ProtectionStatus};
use crate::chat::{self, get_chat_id_by_grpid, Chat, ChatId, ChatIdBlocked, ProtectionStatus};
use crate::chatlist_events;
use crate::config::Config;
use crate::constants::Blocked;
use crate::constants::{Blocked, Chattype};
use crate::contact::{Contact, ContactId, Origin};
use crate::context::Context;
use crate::e2ee::ensure_secret_key_exists;
@@ -60,13 +60,29 @@ pub async fn get_securejoin_qr(context: &Context, group: Option<ChatId>) -> Resu
ensure_secret_key_exists(context).await.ok();
// invitenumber will be used to allow starting the handshake,
// auth will be used to verify the fingerprint
let sync_token = token::lookup(context, Namespace::InviteNumber, group)
let chat = match group {
Some(id) => {
let chat = Chat::load_from_db(context, id).await?;
ensure!(
chat.typ == Chattype::Group,
"Can't generate SecureJoin QR code for 1:1 chat {id}"
);
ensure!(
!chat.grpid.is_empty(),
"Can't generate SecureJoin QR code for ad-hoc group {id}"
);
Some(chat)
}
None => None,
};
let grpid = chat.as_ref().map(|c| c.grpid.as_str());
let sync_token = token::lookup(context, Namespace::InviteNumber, grpid)
.await?
.is_none();
let invitenumber = token::lookup_or_new(context, Namespace::InviteNumber, group).await?;
let auth = token::lookup_or_new(context, Namespace::Auth, group).await?;
// invitenumber will be used to allow starting the handshake,
// auth will be used to verify the fingerprint
let invitenumber = token::lookup_or_new(context, Namespace::InviteNumber, grpid).await?;
let auth = token::lookup_or_new(context, Namespace::Auth, grpid).await?;
let self_addr = context.get_primary_self_addr().await?;
let self_name = context
.get_config(Config::Displayname)
@@ -85,19 +101,14 @@ pub async fn get_securejoin_qr(context: &Context, group: Option<ChatId>) -> Resu
let self_name_urlencoded =
utf8_percent_encode(&self_name, NON_ALPHANUMERIC_WITHOUT_DOT).to_string();
let qr = if let Some(group) = group {
let qr = if let Some(chat) = chat {
// parameters used: a=g=x=i=s=
let chat = Chat::load_from_db(context, group).await?;
if chat.grpid.is_empty() {
bail!(
"can't generate securejoin QR code for ad-hoc group {}",
group
);
}
let group_name = chat.get_name();
let group_name_urlencoded = utf8_percent_encode(group_name, NON_ALPHANUMERIC).to_string();
if sync_token {
context.sync_qr_code_tokens(Some(chat.id)).await?;
context
.sync_qr_code_tokens(Some(chat.grpid.as_str()))
.await?;
context.scheduler.interrupt_smtp().await;
}
format!(
@@ -369,62 +380,54 @@ pub(crate) async fn handle_securejoin_handshake(
==========================================================*/
// verify that Secure-Join-Fingerprint:-header matches the fingerprint of Bob
let fingerprint: Fingerprint =
match mime_message.get_header(HeaderDef::SecureJoinFingerprint) {
Some(fp) => fp.parse()?,
None => {
could_not_establish_secure_connection(
context,
contact_id,
info_chat_id(context, contact_id).await?,
"Fingerprint not provided.",
)
.await?;
return Ok(HandshakeMessage::Ignore);
}
};
if !encrypted_and_signed(context, mime_message, Some(&fingerprint)) {
could_not_establish_secure_connection(
let Some(fp) = mime_message.get_header(HeaderDef::SecureJoinFingerprint) else {
warn!(
context,
contact_id,
info_chat_id(context, contact_id).await?,
"Auth not encrypted.",
)
.await?;
"Ignoring {step} message because fingerprint is not provided."
);
return Ok(HandshakeMessage::Ignore);
};
let fingerprint: Fingerprint = fp.parse()?;
if !encrypted_and_signed(context, mime_message, Some(&fingerprint)) {
warn!(
context,
"Ignoring {step} message because the message is not encrypted."
);
return Ok(HandshakeMessage::Ignore);
}
if !verify_sender_by_fingerprint(context, &fingerprint, contact_id).await? {
could_not_establish_secure_connection(
warn!(
context,
contact_id,
info_chat_id(context, contact_id).await?,
"Fingerprint mismatch on inviter-side.",
)
.await?;
"Ignoring {step} message because of fingerprint mismatch."
);
return Ok(HandshakeMessage::Ignore);
}
info!(context, "Fingerprint verified.",);
// verify that the `Secure-Join-Auth:`-header matches the secret written to the QR code
let Some(auth) = mime_message.get_header(HeaderDef::SecureJoinAuth) else {
could_not_establish_secure_connection(
warn!(
context,
contact_id,
info_chat_id(context, contact_id).await?,
"Auth not provided.",
)
.await?;
"Ignoring {step} message because of missing auth code."
);
return Ok(HandshakeMessage::Ignore);
};
let Some(group_chat_id) = token::auth_chat_id(context, auth).await? else {
could_not_establish_secure_connection(
let Some(grpid) = token::auth_foreign_key(context, auth).await? else {
warn!(
context,
contact_id,
info_chat_id(context, contact_id).await?,
"Auth invalid.",
)
.await?;
"Ignoring {step} message because of invalid auth code."
);
return Ok(HandshakeMessage::Ignore);
};
let group_chat_id = match grpid.as_str() {
"" => None,
id => {
let Some((chat_id, ..)) = get_chat_id_by_grpid(context, id).await? else {
warn!(context, "Ignoring {step} message: unknown grpid {id}.",);
return Ok(HandshakeMessage::Ignore);
};
Some(chat_id)
}
};
let contact_addr = Contact::get_by_id(context, contact_id)
.await?
@@ -439,13 +442,10 @@ pub(crate) async fn handle_securejoin_handshake(
)
.await?;
if !fingerprint_found {
could_not_establish_secure_connection(
warn!(
context,
contact_id,
info_chat_id(context, contact_id).await?,
"Fingerprint mismatch on inviter-side.",
)
.await?;
"Ignoring {step} message because of the failure to find matching peerstate."
);
return Ok(HandshakeMessage::Ignore);
}
contact_id.regossip_keys(context).await?;
@@ -453,7 +453,20 @@ pub(crate) async fn handle_securejoin_handshake(
info!(context, "Auth verified.",);
context.emit_event(EventType::ContactsChanged(Some(contact_id)));
inviter_progress(context, contact_id, 600);
if group_chat_id.is_unset() {
if let Some(group_chat_id) = group_chat_id {
// Join group.
secure_connection_established(
context,
contact_id,
group_chat_id,
mime_message.timestamp_sent,
)
.await?;
chat::add_contact_to_chat_ex(context, Nosync, group_chat_id, contact_id, true)
.await?;
inviter_progress(context, contact_id, 800);
inviter_progress(context, contact_id, 1000);
} else {
// Setup verified contact.
secure_connection_established(
context,
@@ -467,19 +480,6 @@ pub(crate) async fn handle_securejoin_handshake(
.context("failed sending vc-contact-confirm message")?;
inviter_progress(context, contact_id, 1000);
} else {
// Join group.
secure_connection_established(
context,
contact_id,
group_chat_id,
mime_message.timestamp_sent,
)
.await?;
chat::add_contact_to_chat_ex(context, Nosync, group_chat_id, contact_id, true)
.await?;
inviter_progress(context, contact_id, 800);
inviter_progress(context, contact_id, 1000);
}
Ok(HandshakeMessage::Ignore) // "Done" would delete the message and break multi-device (the key from Autocrypt-header is needed)
}
@@ -781,6 +781,7 @@ mod tests {
CheckProtectionTimestamp,
WrongAliceGossip,
SecurejoinWaitTimeout,
AliceIsBot,
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
@@ -803,6 +804,11 @@ mod tests {
test_setup_contact_ex(SetupContactCase::SecurejoinWaitTimeout).await
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_setup_contact_alice_is_bot() {
test_setup_contact_ex(SetupContactCase::AliceIsBot).await
}
async fn test_setup_contact_ex(case: SetupContactCase) {
let mut tcm = TestContextManager::new();
let alice = tcm.alice().await;
@@ -811,13 +817,19 @@ mod tests {
bob.set_config(Config::Displayname, Some("Bob Examplenet"))
.await
.unwrap();
alice
.set_config(Config::VerifiedOneOnOneChats, Some("1"))
.await
.unwrap();
bob.set_config(Config::VerifiedOneOnOneChats, Some("1"))
.await
.unwrap();
let alice_auto_submitted_hdr;
match case {
SetupContactCase::AliceIsBot => {
alice.set_config_bool(Config::Bot, true).await.unwrap();
alice_auto_submitted_hdr = "Auto-Submitted: auto-generated";
}
_ => alice_auto_submitted_hdr = "Auto-Submitted: auto-replied",
};
for t in [&alice, &bob] {
t.set_config_bool(Config::VerifiedOneOnOneChats, true)
.await
.unwrap();
}
assert_eq!(
Chatlist::try_load(&alice, 0, None, None)
@@ -853,6 +865,7 @@ mod tests {
assert!(!msg.was_encrypted());
assert_eq!(msg.get_header(HeaderDef::SecureJoin).unwrap(), "vc-request");
assert!(msg.get_header(HeaderDef::SecureJoinInvitenumber).is_some());
assert!(msg.get_header(HeaderDef::AutoSubmitted).is_none());
// Step 3: Alice receives vc-request, sends vc-auth-required
alice.recv_msg_trash(&sent).await;
@@ -865,6 +878,7 @@ mod tests {
);
let sent = alice.pop_sent_msg().await;
assert!(sent.payload.contains(alice_auto_submitted_hdr));
assert!(!sent.payload.contains("Alice Exampleorg"));
let msg = bob.parse_msg(&sent).await;
assert!(msg.was_encrypted());
@@ -911,6 +925,7 @@ mod tests {
// Check Bob sent the right message.
let sent = bob.pop_sent_msg().await;
assert!(sent.payload.contains("Auto-Submitted: auto-replied"));
assert!(!sent.payload.contains("Bob Examplenet"));
let mut msg = alice.parse_msg(&sent).await;
let vc_request_with_auth_ts_sent = msg
@@ -976,6 +991,7 @@ mod tests {
.await
.unwrap();
assert_eq!(contact_bob.get_authname(), "Bob Examplenet");
assert_eq!(contact_bob.is_bot(), false);
// exactly one one-to-one chat should be visible for both now
// (check this before calling alice.create_chat() explicitly below)
@@ -1015,6 +1031,7 @@ mod tests {
// Check Alice sent the right message to Bob.
let sent = alice.pop_sent_msg().await;
assert!(sent.payload.contains(alice_auto_submitted_hdr));
assert!(!sent.payload.contains("Alice Exampleorg"));
let msg = bob.parse_msg(&sent).await;
assert!(msg.was_encrypted());
@@ -1033,6 +1050,7 @@ mod tests {
.await
.unwrap();
assert_eq!(contact_alice.get_authname(), "Alice Exampleorg");
assert_eq!(contact_alice.is_bot(), case == SetupContactCase::AliceIsBot);
if case != SetupContactCase::SecurejoinWaitTimeout {
// Later we check that the timeout message isn't added to the already protected chat.
@@ -1247,6 +1265,7 @@ mod tests {
assert!(!msg.was_encrypted());
assert_eq!(msg.get_header(HeaderDef::SecureJoin).unwrap(), "vg-request");
assert!(msg.get_header(HeaderDef::SecureJoinInvitenumber).is_some());
assert!(msg.get_header(HeaderDef::AutoSubmitted).is_none());
// Old Delta Chat core sent `Secure-Join-Group` header in `vg-request`,
// but it was only used by Alice in `vg-request-with-auth`.
@@ -1260,6 +1279,7 @@ mod tests {
alice.recv_msg_trash(&sent).await;
let sent = alice.pop_sent_msg().await;
assert!(sent.payload.contains("Auto-Submitted: auto-replied"));
let msg = bob.parse_msg(&sent).await;
assert!(msg.was_encrypted());
assert_eq!(
@@ -1293,6 +1313,7 @@ mod tests {
}
// Check Bob sent the right handshake message.
assert!(sent.payload.contains("Auto-Submitted: auto-replied"));
let msg = alice.parse_msg(&sent).await;
assert!(msg.was_encrypted());
assert_eq!(
@@ -1325,6 +1346,10 @@ mod tests {
msg.get_header(HeaderDef::SecureJoin).unwrap(),
"vg-member-added"
);
// Formally this message is auto-submitted, but as the member addition is a result of an
// explicit user action, the Auto-Submitted header shouldn't be present. Otherwise it would
// be strange to have it in "member-added" messages of verified groups only.
assert!(msg.get_header(HeaderDef::AutoSubmitted).is_none());
{
// Now Alice's chat with Bob should still be hidden, the verified message should
@@ -1451,13 +1476,11 @@ First thread."#;
let mut tcm = TestContextManager::new();
let alice = tcm.alice().await;
let bob = tcm.bob().await;
alice
.set_config(Config::VerifiedOneOnOneChats, Some("1"))
.await
.unwrap();
bob.set_config(Config::VerifiedOneOnOneChats, Some("1"))
.await
.unwrap();
for t in [&alice, &bob] {
t.set_config_bool(Config::VerifiedOneOnOneChats, true)
.await
.unwrap();
}
let qr = get_securejoin_qr(&alice.ctx, None).await.unwrap();
join_securejoin(&bob.ctx, &qr).await.unwrap();
+30 -6
View File
@@ -117,7 +117,7 @@ pub(crate) fn simplify(mut input: String, is_chat_message: bool) -> SimplifiedTe
let lines = split_lines(&input);
let (lines, is_forwarded) = skip_forward_header(&lines);
let (lines, mut top_quote) = remove_top_quote(lines);
let (lines, mut top_quote) = remove_top_quote(lines, is_chat_message);
let original_lines = &lines;
let (lines, footer_lines) = remove_message_footer(lines);
let footer = footer_lines.map(|footer_lines| render_message(footer_lines, false));
@@ -210,7 +210,10 @@ fn remove_bottom_quote<'a>(lines: &'a [&str]) -> (&'a [&'a str], Option<String>)
}
#[allow(clippy::indexing_slicing)]
fn remove_top_quote<'a>(lines: &'a [&str]) -> (&'a [&'a str], Option<String>) {
fn remove_top_quote<'a>(
lines: &'a [&str],
is_chat_message: bool,
) -> (&'a [&'a str], Option<String>) {
let mut first_quoted_line = 0;
let mut last_quoted_line = None;
let mut has_quoted_headline = false;
@@ -220,7 +223,11 @@ fn remove_top_quote<'a>(lines: &'a [&str]) -> (&'a [&'a str], Option<String>) {
first_quoted_line = l;
}
last_quoted_line = Some(l)
} else if is_quoted_headline(line) && !has_quoted_headline && last_quoted_line.is_none() {
} else if !is_chat_message
&& is_quoted_headline(line)
&& !has_quoted_headline
&& last_quoted_line.is_none()
{
has_quoted_headline = true
} else {
/* non-quoting line found */
@@ -396,17 +403,34 @@ mod tests {
#[test]
fn test_remove_top_quote() {
let (lines, top_quote) = remove_top_quote(&["> first", "> second"]);
let (lines, top_quote) = remove_top_quote(&["> first", "> second"], true);
assert!(lines.is_empty());
assert_eq!(top_quote.unwrap(), "first\nsecond");
let (lines, top_quote) = remove_top_quote(&["> first", "> second", "not a quote"]);
let (lines, top_quote) = remove_top_quote(&["> first", "> second", "not a quote"], true);
assert_eq!(lines, &["not a quote"]);
assert_eq!(top_quote.unwrap(), "first\nsecond");
let (lines, top_quote) = remove_top_quote(&["not a quote", "> first", "> second"]);
let (lines, top_quote) = remove_top_quote(&["not a quote", "> first", "> second"], true);
assert_eq!(lines, &["not a quote", "> first", "> second"]);
assert!(top_quote.is_none());
let (lines, top_quote) = remove_top_quote(
&["On 2024-08-28, Bob wrote:", "> quote", "not a quote"],
false,
);
assert_eq!(lines, &["not a quote"]);
assert_eq!(top_quote.unwrap(), "quote");
let (lines, top_quote) = remove_top_quote(
&["On 2024-08-28, Bob wrote:", "> quote", "not a quote"],
true,
);
assert_eq!(
lines,
&["On 2024-08-28, Bob wrote:", "> quote", "not a quote"]
);
assert!(top_quote.is_none());
}
#[test]
+46 -66
View File
@@ -5,7 +5,7 @@ pub mod send;
use anyhow::{bail, format_err, Context as _, Error, Result};
use async_smtp::response::{Category, Code, Detail};
use async_smtp::{self as smtp, EmailAddress, SmtpTransport};
use async_smtp::{EmailAddress, SmtpTransport};
use tokio::task;
use crate::chat::{add_info_msg_with_cmd, ChatId};
@@ -13,14 +13,14 @@ use crate::config::Config;
use crate::contact::{Contact, ContactId};
use crate::context::Context;
use crate::events::EventType;
use crate::login_param::{LoginParam, ServerLoginParam};
use crate::login_param::prioritize_server_login_params;
use crate::login_param::{ConfiguredLoginParam, ConfiguredServerLoginParam};
use crate::message::Message;
use crate::message::{self, MsgId};
use crate::mimefactory::MimeFactory;
use crate::net::proxy::ProxyConfig;
use crate::net::session::SessionBufStream;
use crate::oauth2::get_oauth2_access_token;
use crate::scheduler::connectivity::ConnectivityStore;
use crate::socks::Socks5Config;
use crate::sql;
use crate::stock_str::unencrypted_email;
use crate::tools::{self, time_elapsed};
@@ -88,96 +88,76 @@ impl Smtp {
}
self.connectivity.set_connecting(context).await;
let lp = LoginParam::load_configured_params(context).await?;
let lp = ConfiguredLoginParam::load(context)
.await?
.context("Not configured")?;
self.connect(
context,
&lp.smtp,
&lp.socks5_config,
&lp.smtp_password,
&lp.proxy_config,
&lp.addr,
lp.strict_tls(),
lp.oauth2,
)
.await
}
/// Connect using the provided login params.
#[allow(clippy::too_many_arguments)]
pub async fn connect(
&mut self,
context: &Context,
lp: &ServerLoginParam,
socks5_config: &Option<Socks5Config>,
login_params: &[ConfiguredServerLoginParam],
password: &str,
proxy_config: &Option<ProxyConfig>,
addr: &str,
strict_tls: bool,
oauth2: bool,
) -> Result<()> {
if self.is_connected() {
warn!(context, "SMTP already connected.");
return Ok(());
}
if lp.server.is_empty() || lp.port == 0 {
bail!("bad connection parameters");
}
let from = EmailAddress::new(addr.to_string())
.with_context(|| format!("invalid login address {addr}"))?;
.with_context(|| format!("Invalid address {addr:?}"))?;
self.from = Some(from);
let domain = &lp.server;
let port = lp.port;
let session_stream = connect::connect_stream(
context,
domain,
port,
strict_tls,
socks5_config.clone(),
lp.security,
)
.await?;
let client = smtp::SmtpClient::new().smtp_utf8(true).without_greeting();
let mut transport = SmtpTransport::new(client, session_stream).await?;
// Authenticate.
{
let (creds, mechanism) = if lp.oauth2 {
// oauth2
let send_pw = &lp.password;
let access_token = get_oauth2_access_token(context, addr, send_pw, false).await?;
if access_token.is_none() {
bail!("SMTP OAuth 2 error {}", addr);
let login_params =
prioritize_server_login_params(&context.sql, login_params, "smtp").await?;
for lp in login_params {
info!(context, "SMTP trying to connect to {}.", &lp.connection);
let transport = match connect::connect_and_auth(
context,
proxy_config,
strict_tls,
lp.connection.clone(),
oauth2,
addr,
&lp.user,
password,
)
.await
{
Ok(transport) => transport,
Err(err) => {
warn!(context, "SMTP failed to connect: {err:#}.");
continue;
}
let user = &lp.user;
(
smtp::authentication::Credentials::new(
user.to_string(),
access_token.unwrap_or_default(),
),
vec![smtp::authentication::Mechanism::Xoauth2],
)
} else {
// plain
let user = lp.user.clone();
let pw = lp.password.clone();
(
smtp::authentication::Credentials::new(user, pw),
vec![
smtp::authentication::Mechanism::Plain,
smtp::authentication::Mechanism::Login,
],
)
};
transport.try_login(&creds, &mechanism).await?;
self.transport = Some(transport);
self.last_success = Some(tools::Time::now());
context.emit_event(EventType::SmtpConnected(format!(
"SMTP-LOGIN as {} ok",
lp.user,
)));
return Ok(());
}
self.transport = Some(transport);
self.last_success = Some(tools::Time::now());
context.emit_event(EventType::SmtpConnected(format!(
"SMTP-LOGIN as {} ok",
lp.user,
)));
Ok(())
Err(format_err!("SMTP failed to connect"))
}
}
+136 -61
View File
@@ -2,18 +2,20 @@
use std::net::SocketAddr;
use anyhow::{bail, format_err, Context as _, Result};
use anyhow::{bail, Context as _, Result};
use async_smtp::{SmtpClient, SmtpTransport};
use tokio::io::BufStream;
use crate::context::Context;
use crate::login_param::{ConnectionCandidate, ConnectionSecurity};
use crate::net::dns::{lookup_host_with_cache, update_connect_timestamp};
use crate::net::proxy::ProxyConfig;
use crate::net::session::SessionBufStream;
use crate::net::tls::wrap_tls;
use crate::net::update_connection_history;
use crate::net::{connect_tcp_inner, connect_tls_inner};
use crate::provider::Socket;
use crate::socks::Socks5Config;
use crate::net::{
connect_tcp_inner, connect_tls_inner, run_connection_attempts, update_connection_history,
};
use crate::oauth2::get_oauth2_access_token;
use crate::tools::time;
/// Converts port number to ALPN list.
@@ -26,6 +28,95 @@ fn alpn(port: u16) -> &'static [&'static str] {
}
}
#[allow(clippy::too_many_arguments)]
pub(crate) async fn connect_and_auth(
context: &Context,
proxy_config: &Option<ProxyConfig>,
strict_tls: bool,
candidate: ConnectionCandidate,
oauth2: bool,
addr: &str,
user: &str,
password: &str,
) -> Result<SmtpTransport<Box<dyn SessionBufStream>>> {
let session_stream =
connect_stream(context, proxy_config.clone(), strict_tls, candidate).await?;
let client = async_smtp::SmtpClient::new()
.smtp_utf8(true)
.without_greeting();
let mut transport = SmtpTransport::new(client, session_stream).await?;
// Authenticate.
let (creds, mechanism) = if oauth2 {
// oauth2
let access_token = get_oauth2_access_token(context, addr, password, false).await?;
if access_token.is_none() {
bail!("SMTP OAuth 2 error {}", addr);
}
(
async_smtp::authentication::Credentials::new(
user.to_string(),
access_token.unwrap_or_default(),
),
vec![async_smtp::authentication::Mechanism::Xoauth2],
)
} else {
// plain
(
async_smtp::authentication::Credentials::new(user.to_string(), password.to_string()),
vec![
async_smtp::authentication::Mechanism::Plain,
async_smtp::authentication::Mechanism::Login,
],
)
};
transport.try_login(&creds, &mechanism).await?;
Ok(transport)
}
async fn connection_attempt(
context: Context,
host: String,
security: ConnectionSecurity,
resolved_addr: SocketAddr,
strict_tls: bool,
) -> Result<Box<dyn SessionBufStream>> {
let context = &context;
let host = &host;
info!(
context,
"Attempting SMTP connection to {host} ({resolved_addr})."
);
let res = match security {
ConnectionSecurity::Tls => connect_secure(resolved_addr, host, strict_tls).await,
ConnectionSecurity::Starttls => connect_starttls(resolved_addr, host, strict_tls).await,
ConnectionSecurity::Plain => connect_insecure(resolved_addr).await,
};
match res {
Ok(stream) => {
let ip_addr = resolved_addr.ip().to_string();
let port = resolved_addr.port();
let save_cache = match security {
ConnectionSecurity::Tls | ConnectionSecurity::Starttls => strict_tls,
ConnectionSecurity::Plain => false,
};
if save_cache {
update_connect_timestamp(context, host, &ip_addr).await?;
}
update_connection_history(context, "smtp", host, port, &ip_addr, time()).await?;
Ok(stream)
}
Err(err) => {
warn!(
context,
"Failed to connect to {host} ({resolved_addr}): {err:#}."
);
Err(err)
}
}
}
/// Returns TLS, STARTTLS or plaintext connection
/// using SOCKS5 or direct connection depending on the given configuration.
///
@@ -34,59 +125,45 @@ fn alpn(port: u16) -> &'static [&'static str] {
/// does not send welcome message over TLS connection
/// after establishing it, welcome message is always ignored
/// to unify the result regardless of whether TLS or STARTTLS is used.
pub(crate) async fn connect_stream(
async fn connect_stream(
context: &Context,
host: &str,
port: u16,
proxy_config: Option<ProxyConfig>,
strict_tls: bool,
socks5_config: Option<Socks5Config>,
security: Socket,
candidate: ConnectionCandidate,
) -> Result<Box<dyn SessionBufStream>> {
if let Some(socks5_config) = socks5_config {
let host = &candidate.host;
let port = candidate.port;
let security = candidate.security;
if let Some(proxy_config) = proxy_config {
let stream = match security {
Socket::Automatic => bail!("SMTP port security is not configured"),
Socket::Ssl => {
connect_secure_socks5(context, host, port, strict_tls, socks5_config.clone())
ConnectionSecurity::Tls => {
connect_secure_proxy(context, host, port, strict_tls, proxy_config.clone()).await?
}
ConnectionSecurity::Starttls => {
connect_starttls_proxy(context, host, port, strict_tls, proxy_config.clone())
.await?
}
Socket::Starttls => {
connect_starttls_socks5(context, host, port, strict_tls, socks5_config.clone())
.await?
}
Socket::Plain => {
connect_insecure_socks5(context, host, port, socks5_config.clone()).await?
ConnectionSecurity::Plain => {
connect_insecure_proxy(context, host, port, proxy_config.clone()).await?
}
};
Ok(stream)
} else {
let mut first_error = None;
let load_cache = strict_tls && (security == Socket::Ssl || security == Socket::Starttls);
let load_cache = match security {
ConnectionSecurity::Tls | ConnectionSecurity::Starttls => strict_tls,
ConnectionSecurity::Plain => false,
};
for resolved_addr in lookup_host_with_cache(context, host, port, "smtp", load_cache).await?
{
let res = match security {
Socket::Automatic => bail!("SMTP port security is not configured"),
Socket::Ssl => connect_secure(resolved_addr, host, strict_tls).await,
Socket::Starttls => connect_starttls(resolved_addr, host, strict_tls).await,
Socket::Plain => connect_insecure(resolved_addr).await,
};
match res {
Ok(stream) => {
let ip_addr = resolved_addr.ip().to_string();
if load_cache {
update_connect_timestamp(context, host, &ip_addr).await?;
}
update_connection_history(context, "smtp", host, port, &ip_addr, time())
.await?;
return Ok(stream);
}
Err(err) => {
warn!(context, "Failed to connect to {resolved_addr}: {err:#}.");
first_error.get_or_insert(err);
}
}
}
Err(first_error.unwrap_or_else(|| format_err!("no DNS resolution results for {host}")))
let connection_futures = lookup_host_with_cache(context, host, port, "smtp", load_cache)
.await?
.into_iter()
.map(|resolved_addr| {
let context = context.clone();
let host = host.to_string();
connection_attempt(context, host, security, resolved_addr, strict_tls)
});
run_connection_attempts(connection_futures).await
}
}
@@ -114,37 +191,37 @@ async fn skip_smtp_greeting<R: tokio::io::AsyncBufReadExt + Unpin>(stream: &mut
}
}
async fn connect_secure_socks5(
async fn connect_secure_proxy(
context: &Context,
hostname: &str,
port: u16,
strict_tls: bool,
socks5_config: Socks5Config,
proxy_config: ProxyConfig,
) -> Result<Box<dyn SessionBufStream>> {
let socks5_stream = socks5_config
let proxy_stream = proxy_config
.connect(context, hostname, port, strict_tls)
.await?;
let tls_stream = wrap_tls(strict_tls, hostname, alpn(port), socks5_stream).await?;
let tls_stream = wrap_tls(strict_tls, hostname, alpn(port), proxy_stream).await?;
let mut buffered_stream = BufStream::new(tls_stream);
skip_smtp_greeting(&mut buffered_stream).await?;
let session_stream: Box<dyn SessionBufStream> = Box::new(buffered_stream);
Ok(session_stream)
}
async fn connect_starttls_socks5(
async fn connect_starttls_proxy(
context: &Context,
hostname: &str,
port: u16,
strict_tls: bool,
socks5_config: Socks5Config,
proxy_config: ProxyConfig,
) -> Result<Box<dyn SessionBufStream>> {
let socks5_stream = socks5_config
let proxy_stream = proxy_config
.connect(context, hostname, port, strict_tls)
.await?;
// Run STARTTLS command and convert the client back into a stream.
let client = SmtpClient::new().smtp_utf8(true);
let transport = SmtpTransport::new(client, BufStream::new(socks5_stream)).await?;
let transport = SmtpTransport::new(client, BufStream::new(proxy_stream)).await?;
let tcp_stream = transport.starttls().await?.into_inner();
let tls_stream = wrap_tls(strict_tls, hostname, &[], tcp_stream)
.await
@@ -154,16 +231,14 @@ async fn connect_starttls_socks5(
Ok(session_stream)
}
async fn connect_insecure_socks5(
async fn connect_insecure_proxy(
context: &Context,
hostname: &str,
port: u16,
socks5_config: Socks5Config,
proxy_config: ProxyConfig,
) -> Result<Box<dyn SessionBufStream>> {
let socks5_stream = socks5_config
.connect(context, hostname, port, false)
.await?;
let mut buffered_stream = BufStream::new(socks5_stream);
let proxy_stream = proxy_config.connect(context, hostname, port, false).await?;
let mut buffered_stream = BufStream::new(proxy_stream);
skip_smtp_greeting(&mut buffered_stream).await?;
let session_stream: Box<dyn SessionBufStream> = Box::new(buffered_stream);
Ok(session_stream)
-148
View File
@@ -1,148 +0,0 @@
//! # SOCKS5 support.
use std::fmt;
use std::pin::Pin;
use anyhow::Result;
use fast_socks5::client::{Config, Socks5Stream};
use fast_socks5::util::target_addr::ToTargetAddr;
use fast_socks5::AuthenticationMethod;
use fast_socks5::Socks5Command;
use percent_encoding::{utf8_percent_encode, NON_ALPHANUMERIC};
use tokio::net::TcpStream;
use tokio_io_timeout::TimeoutStream;
use crate::context::Context;
use crate::net::connect_tcp;
use crate::sql::Sql;
#[derive(Default, Debug, Clone, PartialEq, Eq)]
pub struct Socks5Config {
pub host: String,
pub port: u16,
pub user_password: Option<(String, String)>,
}
impl Socks5Config {
/// Reads SOCKS5 configuration from the database.
pub async fn from_database(sql: &Sql) -> Result<Option<Self>> {
let enabled = sql.get_raw_config_bool("socks5_enabled").await?;
if enabled {
let host = sql.get_raw_config("socks5_host").await?.unwrap_or_default();
let port: u16 = sql
.get_raw_config_int("socks5_port")
.await?
.unwrap_or_default() as u16;
let user = sql.get_raw_config("socks5_user").await?.unwrap_or_default();
let password = sql
.get_raw_config("socks5_password")
.await?
.unwrap_or_default();
let socks5_config = Self {
host,
port,
user_password: if !user.is_empty() {
Some((user, password))
} else {
None
},
};
Ok(Some(socks5_config))
} else {
Ok(None)
}
}
/// Converts SOCKS5 configuration into URL.
pub fn to_url(&self) -> String {
// `socks5h` means that hostname is resolved into address by the proxy
// and DNS requests should not leak.
let mut url = "socks5h://".to_string();
if let Some((username, password)) = &self.user_password {
let username_urlencoded = utf8_percent_encode(username, NON_ALPHANUMERIC).to_string();
let password_urlencoded = utf8_percent_encode(password, NON_ALPHANUMERIC).to_string();
url += &format!("{username_urlencoded}:{password_urlencoded}@");
}
url += &format!("{}:{}", self.host, self.port);
url
}
/// If `load_dns_cache` is true, loads cached DNS resolution results.
/// Use this only if the connection is going to be protected with TLS checks.
pub async fn connect(
&self,
context: &Context,
target_host: &str,
target_port: u16,
load_dns_cache: bool,
) -> Result<Socks5Stream<Pin<Box<TimeoutStream<TcpStream>>>>> {
let tcp_stream = connect_tcp(context, &self.host, self.port, load_dns_cache).await?;
let authentication_method = if let Some((username, password)) = self.user_password.as_ref()
{
Some(AuthenticationMethod::Password {
username: username.into(),
password: password.into(),
})
} else {
None
};
let mut socks_stream =
Socks5Stream::use_stream(tcp_stream, authentication_method, Config::default()).await?;
let target_addr = (target_host, target_port).to_target_addr()?;
socks_stream
.request(Socks5Command::TCPConnect, target_addr)
.await?;
Ok(socks_stream)
}
}
impl fmt::Display for Socks5Config {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
write!(
f,
"host:{},port:{},user_password:{}",
self.host,
self.port,
if let Some(user_password) = self.user_password.clone() {
format!("user: {}, password: ***", user_password.0)
} else {
"user: None".to_string()
}
)
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_socks5h_url() {
let config = Socks5Config {
host: "127.0.0.1".to_string(),
port: 9050,
user_password: None,
};
assert_eq!(config.to_url(), "socks5h://127.0.0.1:9050");
let config = Socks5Config {
host: "example.org".to_string(),
port: 1080,
user_password: Some(("root".to_string(), "toor".to_string())),
};
assert_eq!(config.to_url(), "socks5h://root:toor@example.org:1080");
let config = Socks5Config {
host: "example.org".to_string(),
port: 1080,
user_password: Some(("root".to_string(), "foo/?\\@".to_string())),
};
assert_eq!(
config.to_url(),
"socks5h://root:foo%2F%3F%5C%40@example.org:1080"
);
}
}
+20
View File
@@ -971,6 +971,26 @@ CREATE INDEX msgs_status_updates_index2 ON msgs_status_updates (uid);
.await?;
}
inc_and_check(&mut migration_version, 118)?;
if dbversion < migration_version {
sql.execute_migration(
"CREATE TABLE tokens_new (
id INTEGER PRIMARY KEY,
namespc INTEGER DEFAULT 0,
foreign_key TEXT DEFAULT '',
token TEXT DEFAULT '',
timestamp INTEGER DEFAULT 0
) STRICT;
INSERT INTO tokens_new
SELECT t.id, t.namespc, IFNULL(c.grpid, ''), t.token, t.timestamp
FROM tokens t LEFT JOIN chats c ON t.foreign_id=c.id;
DROP TABLE tokens;
ALTER TABLE tokens_new RENAME TO tokens;",
migration_version,
)
.await?;
}
let new_version = sql
.get_raw_config_int(VERSION_CFG)
.await?
+2 -2
View File
@@ -1345,12 +1345,12 @@ pub(crate) async fn new_group_send_first_message(context: &Context) -> String {
translated(context, StockMessage::NewGroupSendFirstMessage).await
}
/// Text to put in the [`Qr::Backup`] rendered SVG image.
/// Text to put in the [`Qr::Backup2`] rendered SVG image.
///
/// The default is "Scan to set up second device for <account name (account addr)>". The
/// account name and address are looked up from the context.
///
/// [`Qr::Backup`]: crate::qr::Qr::Backup
/// [`Qr::Backup2`]: crate::qr::Qr::Backup2
pub(crate) async fn backup_transfer_qr(context: &Context) -> Result<String> {
let contact = Contact::get_by_id(context, ContactId::SELF).await?;
let addr = contact.get_addr();
+62 -37
View File
@@ -4,7 +4,7 @@ use anyhow::Result;
use lettre_email::PartBuilder;
use serde::{Deserialize, Serialize};
use crate::chat::{self, Chat, ChatId};
use crate::chat::{self, ChatId};
use crate::config::Config;
use crate::constants::Blocked;
use crate::contact::ContactId;
@@ -117,36 +117,22 @@ impl Context {
Ok(())
}
/// Adds most recent qr-code tokens for a given chat to the list of items to be synced.
/// If device synchronization is disabled,
/// Adds most recent qr-code tokens for the given group or self-contact to the list of items to
/// be synced. If device synchronization is disabled,
/// no tokens exist or the chat is unpromoted, the function does nothing.
/// The caller should perform `SchedulerState::interrupt_smtp()` on its own to trigger sending.
pub(crate) async fn sync_qr_code_tokens(&self, chat_id: Option<ChatId>) -> Result<()> {
pub(crate) async fn sync_qr_code_tokens(&self, grpid: Option<&str>) -> Result<()> {
if !self.should_send_sync_msgs().await? {
return Ok(());
}
if let (Some(invitenumber), Some(auth)) = (
token::lookup(self, Namespace::InviteNumber, chat_id).await?,
token::lookup(self, Namespace::Auth, chat_id).await?,
token::lookup(self, Namespace::InviteNumber, grpid).await?,
token::lookup(self, Namespace::Auth, grpid).await?,
) {
let grpid = if let Some(chat_id) = chat_id {
let chat = Chat::load_from_db(self, chat_id).await?;
if !chat.is_promoted() {
info!(
self,
"group '{}' not yet promoted, do not sync tokens yet.", chat.grpid
);
return Ok(());
}
Some(chat.grpid)
} else {
None
};
self.add_sync_item(SyncData::AddQrToken(QrTokenData {
invitenumber,
auth,
grpid,
grpid: grpid.map(|s| s.to_string()),
}))
.await?;
}
@@ -296,21 +282,9 @@ impl Context {
}
async fn add_qr_token(&self, token: &QrTokenData) -> Result<()> {
let chat_id = if let Some(grpid) = &token.grpid {
if let Some((chat_id, _, _)) = chat::get_chat_id_by_grpid(self, grpid).await? {
Some(chat_id)
} else {
warn!(
self,
"Ignoring token for nonexistent/deleted group '{}'.", grpid
);
return Ok(());
}
} else {
None
};
token::save(self, Namespace::InviteNumber, chat_id, &token.invitenumber).await?;
token::save(self, Namespace::Auth, chat_id, &token.auth).await?;
let grpid = token.grpid.as_deref();
token::save(self, Namespace::InviteNumber, grpid, &token.invitenumber).await?;
token::save(self, Namespace::Auth, grpid, &token.auth).await?;
Ok(())
}
@@ -328,9 +302,11 @@ mod tests {
use anyhow::bail;
use super::*;
use crate::chat::{remove_contact_from_chat, Chat, ProtectionStatus};
use crate::chatlist::Chatlist;
use crate::contact::{Contact, Origin};
use crate::test_utils::{TestContext, TestContextManager};
use crate::securejoin::get_securejoin_qr;
use crate::test_utils::{self, TestContext, TestContextManager};
use crate::tools::SystemTime;
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
@@ -630,4 +606,53 @@ mod tests {
assert_eq!(msg.text, "hi");
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_unpromoted_group_qr_sync() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
alice.set_config_bool(Config::SyncMsgs, true).await?;
let alice_chatid =
chat::create_group_chat(alice, ProtectionStatus::Protected, "the chat").await?;
let qr = get_securejoin_qr(alice, Some(alice_chatid)).await?;
// alice2 syncs the QR code token.
let alice2 = &tcm.alice().await;
alice2.set_config_bool(Config::SyncMsgs, true).await?;
test_utils::sync(alice, alice2).await;
let bob = &tcm.bob().await;
tcm.exec_securejoin_qr(bob, alice, &qr).await;
let msg_id = alice.send_sync_msg().await?;
// Core <= v1.143 doesn't sync QR code tokens immediately, so current Core does that when a
// group is promoted for compatibility (because the group could be created by older Core).
// TODO: assert!(msg_id.is_none());
assert!(msg_id.is_some());
let sent = alice.pop_sent_msg().await;
let msg = alice.parse_msg(&sent).await;
let mut sync_items = msg.sync_items.unwrap().items;
assert_eq!(sync_items.len(), 1);
let data = sync_items.pop().unwrap().data;
let SyncDataOrUnknown::SyncData(AddQrToken(_)) = data else {
unreachable!();
};
// Remove Bob because alice2 doesn't have their key.
let alice_bob_id = alice.add_or_lookup_contact(bob).await.id;
remove_contact_from_chat(alice, alice_chatid, alice_bob_id).await?;
alice.pop_sent_msg().await;
let sent = alice
.send_text(alice_chatid, "Promoting group to another device")
.await;
alice2.recv_msg(&sent).await;
let fiona = &tcm.fiona().await;
tcm.exec_securejoin_qr(fiona, alice2, &qr).await;
let msg = fiona.get_last_msg().await;
assert_eq!(
msg.text,
"Member Me (fiona@example.net) added by alice@example.org."
);
Ok(())
}
}
+31 -39
View File
@@ -10,10 +10,10 @@ use std::path::Path;
use std::sync::Arc;
use std::time::{Duration, Instant};
use ansi_term::Color;
use async_channel::{self as channel, Receiver, Sender};
use chat::ChatItem;
use deltachat_contact_tools::{ContactAddress, EmailAddress};
use nu_ansi_term::Color;
use once_cell::sync::Lazy;
use pretty_assertions::assert_eq;
use rand::Rng;
@@ -256,19 +256,28 @@ impl TestContextBuilder {
/// Builds the [`TestContext`].
pub async fn build(self) -> TestContext {
let name = self.key_pair.as_ref().map(|key| key.addr.local.clone());
let test_context = TestContext::new_internal(name, self.log_sink).await;
if let Some(key_pair) = self.key_pair {
test_context
.configure_addr(&key_pair.addr.to_string())
.await;
let userid = {
let public_key = &key_pair.public;
let id_bstr = public_key.details.users.first().unwrap().id.id();
String::from_utf8(id_bstr.to_vec()).unwrap()
};
let addr = mailparse::addrparse(&userid)
.unwrap()
.extract_single_info()
.unwrap()
.addr;
let name = EmailAddress::new(&addr).unwrap().local;
let test_context = TestContext::new_internal(Some(name), self.log_sink).await;
test_context.configure_addr(&addr).await;
key::store_self_keypair(&test_context, &key_pair, KeyPairUse::Default)
.await
.expect("Failed to save key");
test_context
} else {
TestContext::new_internal(None, self.log_sink).await
}
test_context
}
}
@@ -474,9 +483,17 @@ impl TestContext {
.execute("DELETE FROM smtp WHERE id=?;", (rowid,))
.await
.expect("failed to remove job");
update_msg_state(&self.ctx, msg_id, MessageState::OutDelivered)
if !self
.ctx
.sql
.exists("SELECT COUNT(*) FROM smtp WHERE msg_id=?", (msg_id,))
.await
.expect("failed to update message state");
.expect("Failed to check for more jobs")
{
update_msg_state(&self.ctx, msg_id, MessageState::OutDelivered)
.await
.expect("failed to update message state");
}
let payload_headers = payload.split("\r\n\r\n").next().unwrap().lines();
let payload_header_names: Vec<_> = payload_headers
@@ -971,55 +988,30 @@ impl SentMessage<'_> {
///
/// The keypair was created using the crate::key::tests::gen_key test.
pub fn alice_keypair() -> KeyPair {
let addr = EmailAddress::new("alice@example.org").unwrap();
let public = key::SignedPublicKey::from_asc(include_str!("../test-data/key/alice-public.asc"))
.unwrap()
.0;
let secret = key::SignedSecretKey::from_asc(include_str!("../test-data/key/alice-secret.asc"))
.unwrap()
.0;
KeyPair {
addr,
public,
secret,
}
KeyPair::new(secret).unwrap()
}
/// Load a pre-generated keypair for bob@example.net from disk.
///
/// Like [alice_keypair] but a different key and identity.
pub fn bob_keypair() -> KeyPair {
let addr = EmailAddress::new("bob@example.net").unwrap();
let public = key::SignedPublicKey::from_asc(include_str!("../test-data/key/bob-public.asc"))
.unwrap()
.0;
let secret = key::SignedSecretKey::from_asc(include_str!("../test-data/key/bob-secret.asc"))
.unwrap()
.0;
KeyPair {
addr,
public,
secret,
}
KeyPair::new(secret).unwrap()
}
/// Load a pre-generated keypair for fiona@example.net from disk.
///
/// Like [alice_keypair] but a different key and identity.
pub fn fiona_keypair() -> KeyPair {
let addr = EmailAddress::new("fiona@example.net").unwrap();
let public = key::SignedPublicKey::from_asc(include_str!("../test-data/key/fiona-public.asc"))
.unwrap()
.0;
let secret = key::SignedSecretKey::from_asc(include_str!("../test-data/key/fiona-secret.asc"))
.unwrap()
.0;
KeyPair {
addr,
public,
secret,
}
KeyPair::new(secret).unwrap()
}
/// Utility to help wait for and retrieve events.
+29 -58
View File
@@ -7,7 +7,6 @@
use anyhow::Result;
use deltachat_derive::{FromSql, ToSql};
use crate::chat::ChatId;
use crate::context::Context;
use crate::tools::{create_id, time};
@@ -27,34 +26,22 @@ pub enum Namespace {
pub async fn save(
context: &Context,
namespace: Namespace,
foreign_id: Option<ChatId>,
foreign_key: Option<&str>,
token: &str,
) -> Result<()> {
match foreign_id {
Some(foreign_id) => context
.sql
.execute(
"INSERT INTO tokens (namespc, foreign_id, token, timestamp) VALUES (?, ?, ?, ?);",
(namespace, foreign_id, token, time()),
)
.await?,
None => {
context
.sql
.execute(
"INSERT INTO tokens (namespc, token, timestamp) VALUES (?, ?, ?);",
(namespace, token, time()),
)
.await?
}
};
context
.sql
.execute(
"INSERT INTO tokens (namespc, foreign_key, token, timestamp) VALUES (?, ?, ?, ?)",
(namespace, foreign_key.unwrap_or(""), token, time()),
)
.await?;
Ok(())
}
/// Lookup most recently created token for a namespace/chat combination.
/// Looks up most recently created token for a namespace / foreign key combination.
///
/// As there may be more than one valid token for a chat-id,
/// As there may be more than one such valid token,
/// (eg. when a qr code token is withdrawn, recreated and revived later),
/// use lookup() for qr-code creation only;
/// do not use lookup() to check for token validity.
@@ -63,43 +50,28 @@ pub async fn save(
pub async fn lookup(
context: &Context,
namespace: Namespace,
chat: Option<ChatId>,
foreign_key: Option<&str>,
) -> Result<Option<String>> {
let token = match chat {
Some(chat_id) => {
context
.sql
.query_get_value(
"SELECT token FROM tokens WHERE namespc=? AND foreign_id=? ORDER BY timestamp DESC LIMIT 1;",
(namespace, chat_id),
)
.await?
}
// foreign_id is declared as `INTEGER DEFAULT 0` in the schema.
None => {
context
.sql
.query_get_value(
"SELECT token FROM tokens WHERE namespc=? AND foreign_id=0 ORDER BY timestamp DESC LIMIT 1;",
(namespace,),
)
.await?
}
};
Ok(token)
context
.sql
.query_get_value(
"SELECT token FROM tokens WHERE namespc=? AND foreign_key=? ORDER BY timestamp DESC LIMIT 1",
(namespace, foreign_key.unwrap_or("")),
)
.await
}
pub async fn lookup_or_new(
context: &Context,
namespace: Namespace,
foreign_id: Option<ChatId>,
foreign_key: Option<&str>,
) -> Result<String> {
if let Some(token) = lookup(context, namespace, foreign_id).await? {
if let Some(token) = lookup(context, namespace, foreign_key).await? {
return Ok(token);
}
let token = create_id();
save(context, namespace, foreign_id, &token).await?;
save(context, namespace, foreign_key, &token).await?;
Ok(token)
}
@@ -114,23 +86,22 @@ pub async fn exists(context: &Context, namespace: Namespace, token: &str) -> Res
Ok(exists)
}
/// Looks up ChatId by auth token.
/// Looks up foreign key by auth token.
///
/// Returns None if auth token is not valid.
/// Returns zero/unset ChatId if the token corresponds to "setup contact" rather than group join.
pub async fn auth_chat_id(context: &Context, token: &str) -> Result<Option<ChatId>> {
let chat_id: Option<ChatId> = context
/// Returns an empty string if the token corresponds to "setup contact" rather than group join.
pub async fn auth_foreign_key(context: &Context, token: &str) -> Result<Option<String>> {
context
.sql
.query_row_optional(
"SELECT foreign_id FROM tokens WHERE namespc=? AND token=?",
"SELECT foreign_key FROM tokens WHERE namespc=? AND token=?",
(Namespace::Auth, token),
|row| {
let chat_id: ChatId = row.get(0)?;
Ok(chat_id)
let foreign_key: String = row.get(0)?;
Ok(foreign_key)
},
)
.await?;
Ok(chat_id)
.await
}
pub async fn delete(context: &Context, namespace: Namespace, token: &str) -> Result<()> {
+20 -3
View File
@@ -36,7 +36,8 @@ use tokio::{fs, io};
use url::Url;
use crate::chat::{add_device_msg, add_device_msg_with_importance};
use crate::constants::{DC_ELLIPSIS, DC_OUTDATED_WARNING_DAYS};
use crate::config::Config;
use crate::constants::{self, DC_ELLIPSIS, DC_OUTDATED_WARNING_DAYS};
use crate::context::Context;
use crate::events::EventType;
use crate::message::{Message, Viewtype};
@@ -54,7 +55,7 @@ pub(crate) fn truncate(buf: &str, approx_chars: usize) -> Cow<str> {
.map(|(n, _)| n)
.unwrap_or_default();
if let Some(index) = buf[..end_pos].rfind(|c| c == ' ' || c == '\n') {
if let Some(index) = buf[..end_pos].rfind([' ', '\n']) {
Cow::Owned(format!("{}{}", &buf[..=index], DC_ELLIPSIS))
} else {
Cow::Owned(format!("{}{}", &buf[..end_pos], DC_ELLIPSIS))
@@ -98,7 +99,7 @@ pub(crate) fn truncate_by_lines(
// Text has too many lines and needs to be truncated.
let text = {
if let Some(buffer) = buf.get(..end_pos) {
if let Some(index) = buffer.rfind(|c| c == ' ' || c == '\n') {
if let Some(index) = buffer.rfind([' ', '\n']) {
buf.get(..=index)
} else {
buf.get(..end_pos)
@@ -123,6 +124,22 @@ pub(crate) fn truncate_by_lines(
}
}
/// Shortens a message text if necessary according to the configuration. Adds "[...]" to the end of
/// the shortened text.
///
/// Returns the resulting text and a bool telling whether a truncation was done.
pub(crate) async fn truncate_msg_text(context: &Context, text: String) -> Result<(String, bool)> {
if context.get_config_bool(Config::Bot).await? {
return Ok((text, false));
}
// Truncate text if it has too many lines
Ok(truncate_by_lines(
text,
constants::DC_DESIRED_TEXT_LINES,
constants::DC_DESIRED_TEXT_LINE_LEN,
))
}
/* ******************************************************************************
* date/time tools
******************************************************************************/
-13
View File
@@ -1,13 +0,0 @@
-----BEGIN PGP PUBLIC KEY BLOCK-----
mDMEXlh13RYJKwYBBAHaRw8BAQdAzfVIAleCXMJrq8VeLlEVof6ITCviMktKjmcB
KAu4m5C0GUFsaWNlIDxhbGljZUBleGFtcGxlLm9yZz6IkAQTFggAOBYhBC5vossj
tTLXKGNLWGSwj2Gp7ZRDBQJeWHXdAhsDBQsJCAcCBhUKCQgLAgQWAgMBAh4BAheA
AAoJEGSwj2Gp7ZRDE3oA/i4MCyDMTsjWqDZoQwX/A/GoTO2/V0wKPhjJJy/8m2pM
APkBjOnGOtx2SZpQvJGTa9h804RY6iDrRuI8A/8tEEXAA7g4BF5Ydd0SCisGAQQB
l1UBBQEBB0AG7cjWy2SFAU8KnltlubVW67rFiyfp01JrRe6Xqy22HQMBCAeIeAQY
FggAIBYhBC5vossjtTLXKGNLWGSwj2Gp7ZRDBQJeWHXdAhsMAAoJEGSwj2Gp7ZRD
Lo8BAObE8GnsGVwKzNqCvHeWgJsqhjS3C6gvSlV3tEm9XmF6AQDXucIyVfoBwoyM
h2h6cSn/ATn5QJb35pgo+ivp3jsMAg==
=t/Qq
-----END PGP PUBLIC KEY BLOCK-----
-30
View File
@@ -1,30 +0,0 @@
-----BEGIN PGP PUBLIC KEY BLOCK-----
xsBNBF4wx1cBCADOwLS/xCd8iKDWUsyTfVzWby+ZGKPpamPTvdj0GFgnf0B1EBaA
5//PjAzbK5iKio6QNEmZagzJPkXPByJcAIRUm0T16tqDtCvxm+H93YEXpHi/XWOe
Jw9kohATSqUtsRO0pFJeDvPiMTmQrEmHYoWDSQBfCrowZdvnMAlbJ9JjYOngcMeT
xc0jxmPs5s17yFC+1OWu4fwWCyUM3wy1JzdKTcDWryrSkvmgFdUqJ7pJDk1HFTt+
x9tvQlK3un9BXiRwv0u0zDSuI8eDH/dRLA4UL9Pq6vmJmBame1BPsE1PA7VzeTSJ
R2ooJXMT6o2AmH8PPUfRkv3OiWuh7LM5FSpHABEBAAHNETxib2JAZXhhbXBsZS5u
ZXQ+wsCJBBABCAAzAhkBBQJeMMduAhsDBAsJCAcGFQgJCgsCAxYCARYhBMzLWqn2
4RQclDFl8dsYsYy89wSHAAoJENsYsYy89wSH9oIIALbpmicuVghM3CloiCgJhPEF
LFMaQZRDV/KCVVtBcHAhw6d42q8T50mhs+W3Va5E37DN+wcenj8CgeGPQY3kPp2c
nZruYtLhLkZ1+VEay5BQFUMb8kY21XrNTQQET8vc0L8cCLQ7RCgm1tGiFVp1nqbj
mGUdoru90ksoufWfoqVPjNrW+9eHFvY/Z7PqchCdMnbKOJiwwv4E3NDTySZ1UVZn
DztGy95Aa8OZ3cntvbq4JVi7S+N38rRPPPzpZKx+M4DUGfDAoaq7O/Xemyk1sP6C
/NgQvS8rri54PgkMgKSS4TyyEzdM+fzeNYFPXFGTbgj4p0pSueQV7/JUfYHRfe3O
wE0EXjDHVwEIAKIHgS2yI2niSCN1tqcbLvkhLrEJCVcpGxmA7asl1flwWYrGOBhN
JE2sCuZqkofqw6qrgsQ4GFgUU5xmcBCqIZ49jRu+aY38lT4WDFHSbe/mGtaIhb2Z
YK6zo9W7Y3r6ud8hbUKJTDfl9qEvJpX/Y0syMjwng8SZNTdYMWgAE4NwcgMgdU3d
MA3RT6ePJ4vKs38hmXmInLyZce+GJzmo2tpZyP8viPS7JpqojoCPB3G5h9aHeakp
1Y4XKQaExANeWCyBJEhNwtNEOVEpQ0txFYPyDrtxV5y5e79IUP418r/PHsnH6Unx
XGzB6LfVbSeEyDyKEl+w0PrlNklySomTZFUAEQEAAcLAdgQYAQgAIAUCXjDHbgIb
DBYhBMzLWqn24RQclDFl8dsYsYy89wSHAAoJENsYsYy89wSHVCIIAIH694HkQLXR
AJlXmi8K/xMVP96ywJovL/B5l4S/vk/iR4P1lYsF55A3Z2PK/iFtwAgVsppcBIPB
lqSI0GPDMvEIxj7UFOQfQzVpDes29wG8grHJEJqI/4TlRjOacxTGaJ5fIMsLXJD6
nLBuoN5Z6zm3LjqIyOx4ZGrwradPO95OMGT2Xll3YNzUqSWe33RJLqNQ5ea9I7+q
vKnW5Z9Yt5nQwOo8yD+f5fql8904B3eAyLqxgkdLmngAWmYhc7KOaKdAsx7TXBAK
soeHk51OPk59u7EbX35HWD6snl/phJdUYDXiddyYN/n2ZY9g80ycle2JfgpfrQGl
h7oJqgCjZuk=
=14Cq
-----END PGP PUBLIC KEY BLOCK-----
-30
View File
@@ -1,30 +0,0 @@
-----BEGIN PGP PUBLIC KEY BLOCK-----
xsBNBF48n2MBCAC741/YKzU76pWNijBCYC+hGSKZvBdanm4eJzi/eKPevUypdW2G
JzFFJxWjx2wIcV8cLwQvY2Z+ieJaPPwGbUr0nH2S9lmghkCCKjGxWcmrx3sQr7x9
431KYOM6+/SAZNjHYjWwcwy2QhE6J7qfC74LjTF4pv+ZRgSZDC+O70NpTWdNdwi/
0Kn9gdj5diSwJmnBxQBGp/tnZuu3XGZrgKXlGlPspRMF3Ug2WmvyBhxF8KKoosia
Xlumc5gFFuFRnoAVfp/6yehO44l9bYz0zwYWahxmLhShQVhfcH3YBg0l7hMkfC2F
p8fDF7Hr1PUOJcBY50VDN0zlV9Bi1iiPofAtABEBAAHNEzxmaW9uYUBleGFtcGxl
Lm5ldD7CwIkEEAEIADMCGQEFAl48n3wCGwMECwkIBwYVCAkKCwIDFgIBFiEEyLpQ
v0rBL6841/ZX3fyOnzx5kZUACgkQ3fyOnzx5kZUGWQgApCPQYmfa76xcscYBWg3D
vMAm0Exk/LvbN74MIqADHIgaNFUHoThTPDVAPeh5ogra/kg4QaLctivC81S2VOPI
c/4LGEFJekvythXUgSLNjhlR63Va5ObMV4UegUH9c0O8MGndkQFxiwy98D1bzNyl
3mCVOUECVZiJJbxUZwBKj4iowa7kgX/FrdjZIJrz50NMExcDuxvc+MGyDr4YQHrZ
TbCjcrJkxufklwsWme0qneuPTxwW2+TnDjHkDaYDdwEnoKRkwlXEy3Lu9ZhswlaW
TgdtClpf2geb7tCAPQ0cd2V5m6NnKMzXrnFuJXB6Xkvp32Tyuq4ebILsbCfpLV/H
zM7ATQRePJ9jAQgAzQGckmcFjCViSkwGLROhi2Gvb85ABXbkKPMg1x27LraP9YWN
Joq2GOy2vv8r8m2q+FpCR25a0NdWlbyiQpPuEWh0udJnNUm+6j5j6PSAmlRRDMhD
H4QwzJC+B+lM6NxvSRhxBBtwFAvMy0qeNhv1UCBaeQUHoFCODqJRYOywj7ZGXdQy
ttIAlC5PtVw1cV+J8/TGXNrE9DNgFGAm1BPgw/lE1OjVbF8l6NbDdi9YJoOm9mpf
fUPlUZMZh3+a5+J6F3KjYwNyi4wi3Y3Pt4avXEo+ib15XNhJcwMslc49La2T8PMn
pf3nLClxynJjYDiMuzujcbBWbfVF1383P0KikQARAQABwsB2BBgBCAAgBQJePJ98
AhsMFiEEyLpQv0rBL6841/ZX3fyOnzx5kZUACgkQ3fyOnzx5kZX1eAf9GnrA9nCd
s3OmtCUpRmVEDar29DfS79B4vBfoYXYb5kRIOxJV6yjfGYRh2IJ0CTfNYkp4AuRC
/jEHPXlVUD92Vcb0wSfwO32mMw75FRzIS7/IcwWAauWOtpao3J/tsxHWkSxfh5Zo
6vzODQY4k8eHnitxpXT0xSIjnYmVRMuqfb3NELk3PkF52+Fte4zKfBCP80HqO3Bd
VCBlQNpZiOMW+yFO/8VqLmB9442nGGhuSfCeBystI3Er0SYSDqNbg5uetBaP1MOo
mIZuuxhv3T5jD9DeEDHGOqDjPZ8dMdQYCnYmiVaMb8ECsKG5eMvS8Imss1ho2iz4
sjYdVzODl8T0zQ==
=2jSq
-----END PGP PUBLIC KEY BLOCK-----