Compare commits

...
Author SHA1 Message Date
Hocuri 4a78f4a31f . 2024-10-31 17:01:10 +01:00
Hocuri 69fbb98f3c api: Add Message::new_text() (#6123)
This adds a function to `Message`:

```rust
    pub fn new_text(text: String) -> Self {
        Message {
            viewtype: Viewtype::Text,
            text,
            ..Default::default()
        }
    }
```

I keep expecting that a function like this must exist and being
surprised that it doesn't.

Open question is whether it should be `pub` or `pub(crate)` - I made it
`pub` for now because it may be useful for others and we currently we
aren't thinking about the Rust API that much, anyway, but I can make it
`pub(crate)`, too (then it can't be used in deltachat-jsonrpc and
deltachat-repl).

I replaced some usages of Message::new(Viewtype::Text), but not all yet,
I'm going to do this in a follow-up, which will remove another around 65
LOC.
2024-10-29 16:22:52 +01:00
Hocuri c98d3818d5 fix: Show root SMTP connection failure in connectivity view (#6121)
Right now, when there is an SMTP connection error, the connectivity view
will always show "Error: SMTP connection failure: SMTP failed to
connect".

Instead, I just used the same method that is used in imap connect()
already.
2024-10-29 13:55:15 +01:00
iequidoo 10aa308501 fix: Save full text to mime_headers for long outgoing messages (#6091)
0a63083df7 (fix: Shorten message text in locally sent messages too)
sets `msgs.mime_modified` for long outgoing messages, but forgets to save full message text.
2024-10-28 12:30:29 -03:00
link2xt 146bcfe455 chore(release): prepare for 1.148.5 2024-10-27 17:03:49 +00:00
link2xt f57cdc3a2c Revert "build: nix flake update fenix"
This reverts commit aa3ef5011b.

This fixes `nix build .#deltachat-rpc-server-armeabi-v7a-android`.
2024-10-27 16:53:49 +00:00
link2xt e11fddf9aa ci: take CHATMAIL_DOMAIN from variables instead of secrets 2024-10-26 16:44:28 +00:00
link2xt f396ff4297 fix: do not lock the account manager for the whole duration of background_fetch 2024-10-26 16:38:43 +00:00
link2xt 51a1762228 fix: do not take write lock for maybe_network_lost() and set_push_device_token() 2024-10-26 16:38:43 +00:00
link2xt 69b4c0ccb4 refactor: factor out add_gossip_peer_from_header()
Also don't even add the peer to SQL if realtime is disabled.
2024-10-25 19:25:51 +00:00
iequidoo 3f1dfef0e7 feat: Auto-restore 1:1 chat protection after receiving old unverified message
I.e. add the "Messages are guaranteed to be end-to-end encrypted from now on." message and mark the
chat as protected again because no user action is required in this case. There are a couple of
problems though:
- If the program crashes earlier than the protection is restored, the chat remains
  protection-broken. But this problem already exists because `ChatId::set_protection()` is never
  retried.
- If multiple old unverified messages are received, protection messages added in between don't
  annihilate, so they clutter the chat.
2024-10-25 14:20:09 -03:00
iequidoo c0f5771140 refactor: receive_imf::add_parts: Remove excessive from_id == ContactId::SELF checks
`mime_parser.incoming` is already here for this and is checked above.
2024-10-25 14:20:09 -03:00
iequidoo 33cae2815d fix: Set Config::NotifyAboutWrongPw before saving configuration (#5896)
Let's always set `Config::NotifyAboutWrongPw` before saving configuration, better if a wrong
password notification is shown once more than not shown at all. It shouldn't be a big problem
because reconfiguration is a manual action and isn't done frequently.

Also for the same reason reset `Config::NotifyAboutWrongPw` only after a successful addition of the
appropriate device message.
2024-10-25 13:14:37 -03:00
link2xt fc2b111f5d chore(release): prepare for 1.148.4 2024-10-24 20:25:58 +00:00
link2xt 913d2c45b3 fix: do not wait for connections in maybe_add_gossip_peers()
join() method of Gossip [1]
waits for at least one connection
and this is not what we want
because it may block receive_imf()
forever if no connection arrives.

[1] https://docs.rs/iroh-gossip/0.25.0/iroh_gossip/net/struct.Gossip.html#method.join
2024-10-24 19:59:00 +00:00
link2xt e32d676a08 fix: normalize proxy URLs before saving into proxy_url 2024-10-24 16:43:10 +00:00
Simon Lauxandiequidoo 9812d5ba75 feat: jsonrpc: add private_tag to Account::Configured Object (#6107)
Co-authored-by: iequidoo <117991069+iequidoo@users.noreply.github.com>
2024-10-24 16:00:27 +00:00
link2xt bc7568e39b chore(release): prepare for 1.148.3 2024-10-24 14:08:59 +00:00
link2xt 11bf1c45d2 test: test that realtime advertisements work after chatting 2024-10-24 13:56:04 +00:00
link2xt 122c23ad4e api(deltachat-rpc-client): add EventType.WEBXDC_REALTIME_ADVERTISEMENT_RECEIVED 2024-10-24 13:56:04 +00:00
link2xt a0bde4699e fix: fix reception of realtime advertisements 2024-10-24 13:56:04 +00:00
link2xt ac01a4a771 feat: allow sending realtime messages up to 128 KB in size
Previous default value was 4 KiB.
2024-10-24 13:55:28 +00:00
link2xt 51f2a8d59e refactor: generate topic inside create_iroh_header() 2024-10-23 22:33:09 +00:00
bjoern f208c31cdf docs: fix DC_QR_PROXY docs (#6099) 2024-10-23 22:29:06 +02:00
link2xt acd7a1d17e chore(release): prepare for 1.148.2 2024-10-23 17:52:24 +00:00
link2xt db6d451c90 feat: add more logging for iroh initialization and peer addition 2024-10-23 17:48:33 +00:00
link2xt 4b3a6445fb fix: never initialize Iroh if realtime is disabled 2024-10-23 17:48:33 +00:00
link2xt aa3ef5011b build: nix flake update fenix 2024-10-23 03:31:39 +00:00
link2xt 1d3072c287 build: nix flake update nixpkgs 2024-10-23 03:19:33 +00:00
link2xt 4fb59177fa chore(release): prepare for 1.148.1 2024-10-23 02:37:22 +00:00
link2xt d841bcb41e Revert "build: nix flake update"
This reverts commit 6f22ce2722.
2024-10-23 02:07:22 +00:00
link2xt d205bc410b chore(release): prepare for 1.148.0 2024-10-23 00:27:49 +00:00
bjoern 0d573ac037 feat: add delta chat logo to QR codes (#6093)
the chosen error correction allows tolerates about 15% "erroneous
codewords", the logo is of a similar size as the old avatars,
so it should be fine.
2024-10-23 01:43:50 +02:00
link2xt a55e33fbc7 fix(sql): run PRAGMA incremental_vacuum on a write connection
Otherwise it always fails with SQLITE_READONLY:
```
WARNING src/sql.rs:769: Failed to run incremental vacuum: attempt to write a readonly database: Error code 8: Attempt to write a readonly database.
```
2024-10-22 23:23:57 +00:00
bjoern 839b0e94af api: create QR codes from any data (#6090)
this PR adds a function that can be used to create any QR code, in a raw
form.

this can be used to create add-contact as well as add-second-device QR
codes (eg. `dc_create_qr_svg(dc_get_securejoin_qr())`) - as well as for
other QR codes as proxies.

the disadvantage of the rich-formatted QR codes as created by
`dc_get_securejoin_qr_svg()` and `dc_backup_provider_get_qr_svg()` were:

- they do not look good and cannot interact with UI layout wise (but
also tapping eg. an address is not easily possible)
- esp. text really looks bad. even with
[some](https://github.com/deltachat/deltachat-android/pull/2138/commits/e5dc8fe3d82e8b72631ae5806e9c4c7eec93a37a)
[hacks](https://github.com/deltachat/deltachat-android/pull/2215) it
[stays buggy](https://github.com/deltachat/deltachat-ios/issues/2200);
the bugs mainly come from different SVG implementation, all need their
own quirks
- accessibility is probably bad as well

we thought that time, SVG is a great thing for QR codes, but apart from
basic geometrics, it is not.

so, we avoid text, this also means to avoid putting an avatar in the
middle of the QR code (we can put some generic symbol there, eg.
different ones for add-contact and add-second-device).

while this looks like a degradation, also other messengers use more raw
QR codes. also, we removed many data from the QR code anyway, eg. the
email address is no longer there. that time, sharing QR images was more
a thing, meanwhile we have invite links, that are much better for that
purpose.

in theory, we could also leave the SVG path completely and go for PNG -
which we did not that time as PNG and text looks bad, as the system font
is not easily usable :) but going for PNG would add further challenges
as passing binary data around, and also UI-implemtation-wise, that would
be a larger step. so, let's stay with SVG in a form we know is
compatible.

the old QR code functions are deprecated.
2024-10-22 21:49:45 +02:00
bjoern f2e600dc55 feat: internal profile names (#6088)
this PR allows setting a "private tag" for a profile, see
https://github.com/deltachat/deltachat-android/pull/3373 for a possible
UI.

currently, the core does not do anything with the tag (so, it could also
be a ui.-config option), however, this may change in the future - it
might bet synced, and become also otherwise useful in core. also, having
this in core is better documentation-wise, as otherwise each UI easily
does its own things :)
2024-10-22 09:43:36 +02:00
bjoernandiequidoo 61fd0d400f notify adding reactions (#6072)
this PR adds an event for reactions received for one's own messages.

this will allow UIs to add notification for these reactions.

**Screenshots** at https://github.com/deltachat/deltachat-ios/pull/2331:

---------

Co-authored-by: iequidoo <117991069+iequidoo@users.noreply.github.com>
2024-10-21 21:35:03 +02:00
link2xt 7424d06416 refactor(sql): move write mutex into connection pool 2024-10-21 19:14:37 +00:00
link2xt aa71fbe04c refactor: resultify get_self_fingerprint() 2024-10-21 13:03:58 +00:00
link2xt c5cadd9991 feat: add in-memory cache for DNS
This adds "stale-while-revalidate" in-memory cache for DNS. Instead of
calling `tokio::net::lookup_host` we use previous result of
`tokio::net::lookup_host` immediately and spawn revalidation task in the
background. This way all lookups after the first successful one return
immediately.

Most of the time results returned by resolvers are the same anyway, but
with this cache we avoid waiting 60 second timeout if DNS request is
lost. Common reason result may be different is round-robin DNS load
balancing and switching from IPv4 to IPv6 network. For round-robin DNS
we don't break load balancing but simply use a different result, and for
IPv6 we anyway likely have a result in persistent cache and can use IPv4
otherwise.

Especially frequent should be the case when you send a message over SMTP
and SMTP connection is stale (older than 60 s), so we open a new one.
With this change new connection will be set up faster as you don't need
to wait for DNS resolution, so message will be sent faster.
2024-10-21 10:46:11 +00:00
Septias c92554dc1f fix typo 2024-10-21 11:29:55 +02:00
link2xt 94c6d1dea4 fix: call update_connection_history for proxified connections 2024-10-20 18:36:37 +00:00
Hocuri d27d0ef476 chore: Silence a rust-analyzer false-positive (#6077)
rust-analyzer was showing warnings here because it is always also
building in the Test configuration, and EventType has a

```rust
#[cfg(test)]
Test,
```
variant, which was not matched.
2024-10-20 20:21:32 +02:00
Hocuri d3f75360fa fix: Resolve warning about default-features, and make it possible to disable vendoring (#6079)
On main, when running `cargo build`, the following warning is emitted:

> warning:
/home/jonathan/deltachat-android/jni/deltachat-core-rust/deltachat-ffi/Cargo.toml:
`default-features` is ignored for deltachat, since `default-features`
was not specified for `workspace.dependencies.deltachat`, this could
become a hard error in the future

This is because when referring to a workspace dependency, it's not
possible to remove features, it's only possible to add features, so that
the `vendored` feature was always enabled with no possibility to disable
it.

This PR restores the wanted behavior of enabling vendoring by default
with the possibility to disable it with "default-features = false".

It fixes `nix build .#python-docs` by not passing
`--no-default-features` when building deltachat with nix.
2024-10-20 18:33:47 +02:00
link2xtandiequidoo 06a6cc48d2 feat(sql): set PRAGMA query_only to avoid writing on read-only connections
Co-authored-by: iequidoo <dgreshilov@gmail.com>
2024-10-20 14:51:46 +00:00
iequidoo b13f2709be test: Message from old setup preserves contact verification, but breaks 1:1 protection
If a message from an old contact's setup is received, the outdated Autocrypt header isn't applied,
so the contact verification preserves. But the chat protection breaks because the old message is
sorted to the bottom as it mustn't be sorted over the protection info message (which is `InNoticed`
moreover). Would be nice to preserve the chat protection too e.g. add a "protection broken" message,
then the old message and then a new "protection enabled" message, but let's record the current
behaviour first.
2024-10-20 10:05:28 -03:00
65 changed files with 1370 additions and 400 deletions
+2 -2
View File
@@ -249,7 +249,7 @@ jobs:
- name: Run python tests
env:
CHATMAIL_DOMAIN: ${{ secrets.CHATMAIL_DOMAIN }}
CHATMAIL_DOMAIN: ${{ vars.CHATMAIL_DOMAIN }}
DCC_RS_TARGET: debug
DCC_RS_DEV: ${{ github.workspace }}
working-directory: python
@@ -314,6 +314,6 @@ jobs:
- name: Run deltachat-rpc-client tests
env:
CHATMAIL_DOMAIN: ${{ secrets.CHATMAIL_DOMAIN }}
CHATMAIL_DOMAIN: ${{ vars.CHATMAIL_DOMAIN }}
working-directory: deltachat-rpc-client
run: tox -e py
+1 -1
View File
@@ -33,7 +33,7 @@ jobs:
working-directory: deltachat-jsonrpc/typescript
run: npm run test
env:
CHATMAIL_DOMAIN: ${{ secrets.CHATMAIL_DOMAIN }}
CHATMAIL_DOMAIN: ${{ vars.CHATMAIL_DOMAIN }}
- name: make sure websocket server version still builds
working-directory: deltachat-jsonrpc
run: cargo build --bin deltachat-jsonrpc-server --features webserver
+1 -1
View File
@@ -64,5 +64,5 @@ jobs:
working-directory: node
run: npm run test
env:
CHATMAIL_DOMAIN: ${{ secrets.CHATMAIL_DOMAIN }}
CHATMAIL_DOMAIN: ${{ vars.CHATMAIL_DOMAIN }}
NODE_OPTIONS: "--force-node-api-uncaught-exceptions-policy=true"
+157
View File
@@ -1,5 +1,156 @@
# Changelog
## [1.148.5] - 2024-10-27
### Fixes
- Set Config::NotifyAboutWrongPw before saving configuration ([#5896](https://github.com/deltachat/deltachat-core-rust/pull/5896)).
- Do not take write lock for maybe_network_lost() and set_push_device_token().
- Do not lock the account manager for the whole duration of background_fetch.
### Features / Changes
- Auto-restore 1:1 chat protection after receiving old unverified message.
### CI
- Take `CHATMAIL_DOMAIN` from variables instead of secrets.
### Other
- Revert "build: nix flake update fenix" to fix `nix build .#deltachat-rpc-server-armeabi-v7a-android`.
### Refactor
- Receive_imf::add_parts: Remove excessive `from_id == ContactId::SELF` checks.
- Factor out `add_gossip_peer_from_header()`.
## [1.148.4] - 2024-10-24
### Features / Changes
- Jsonrpc: add `private_tag` to `Account::Configured` Object ([#6107](https://github.com/deltachat/deltachat-core-rust/pull/6107)).
### Fixes
- Normalize proxy URLs before saving into proxy_url.
- Do not wait for connections in maybe_add_gossip_peers().
## [1.148.3] - 2024-10-24
### Fixes
- Fix reception of realtime advertisements.
### Features / Changes
- Allow sending realtime messages up to 128 KB in size.
### API-Changes
- deltachat-rpc-client: Add EventType.WEBXDC_REALTIME_ADVERTISEMENT_RECEIVED.
### Documentation
- Fix DC_QR_PROXY docs ([#6099](https://github.com/deltachat/deltachat-core-rust/pull/6099)).
### Refactor
- Generate topic inside create_iroh_header().
### Tests
- Test that realtime advertisements work after chatting.
## [1.148.2] - 2024-10-23
### Fixes
- Never initialize Iroh if realtime is disabled.
### Features / Changes
- Add more logging for iroh initialization and peer addition.
### Build system
- `nix flake update nixpkgs`.
- `nix flake update fenix`.
## [1.148.1] - 2024-10-23
### Build system
- Revert "build: nix flake update"
This reverts commit 6f22ce2722b51773d7fbb0d89e4764f963cafd91..
## [1.148.0] - 2024-10-22
### API-Changes
- Create QR codes from any data ([#6090](https://github.com/deltachat/deltachat-core-rust/pull/6090)).
- Add delta chat logo to QR codes ([#6093](https://github.com/deltachat/deltachat-core-rust/pull/6093)).
- Add realtime advertisement received event ([#6043](https://github.com/deltachat/deltachat-core-rust/pull/6043)).
- Notify adding reactions ([#6072](https://github.com/deltachat/deltachat-core-rust/pull/6072))
- Internal profile names ([#6088](https://github.com/deltachat/deltachat-core-rust/pull/6088)).
### Features / Changes
- IMAP COMPRESS support.
- Sort received outgoing message down if it's fresher than all non fresh messages.
- Prioritize cached results if DNS resolver returns many results.
- Add in-memory cache for DNS.
- deltachat-repl: Built-in QR code printer.
- Log the logic for (not) doing AEAP.
- Log when late Autocrypt header is ignored.
- Add more context to `send_msg` errors.
### Fixes
- Replace old draft with a new one atomically.
- ChatId::maybe_delete_draft: Don't delete message if it's not a draft anymore ([#6053](https://github.com/deltachat/deltachat-core-rust/pull/6053)).
- Call update_connection_history for proxified connections.
- sql: Set PRAGMA query_only to avoid writing on read-only connections.
- sql: Run `PRAGMA incremental_vacuum` on a write connection.
- Increase MAX_SECONDS_TO_LEND_FROM_FUTURE to 30.
### Build system
- Nix flake update.
- Resolve warning about default-features, and make it possible to disable vendoring ([#6079](https://github.com/deltachat/deltachat-core-rust/pull/6079)).
- Silence a rust-analyzer false-positive ([#6077](https://github.com/deltachat/deltachat-core-rust/pull/6077)).
### CI
- Update Rust to 1.82.0.
### Documentation
- Set_protection_for_timestamp_sort does not send messages.
- Document MimeFactory.req_mdn.
- Fix `too_long_first_doc_paragraph` clippy lint.
### Refactor
- Update_msg_state: Don't avoid downgrading OutMdnRcvd to OutDelivered.
- Fix elided_named_lifetimes warning.
- set_protection_for_timestamp_sort: Do not log bubbled up errors.
- Fix clippy::needless_lifetimes warnings.
- Use `HeaderDef` constant for Chat-Disposition-Notification-To.
- Resultify get_self_fingerprint().
- sql: Move write mutex into connection pool.
### Tests
- test_qr_setup_contact_svg: Stop testing for no display name.
- Always gossip if gossip_period is set to 0.
- test_aeap_flow_verified: Wait for "member added" before sending messages ([#6057](https://github.com/deltachat/deltachat-core-rust/pull/6057)).
- Make test_verified_group_member_added_recovery more reliable.
- test_aeap_flow_verified: Do not start ac1new.
- Fix `test_securejoin_after_contact_resetup` flakiness.
- Message from old setup preserves contact verification, but breaks 1:1 protection.
## [1.147.1] - 2024-10-13
### Build system
@@ -5008,3 +5159,9 @@ https://github.com/deltachat/deltachat-core-rust/pulls?q=is%3Apr+is%3Aclosed
[1.146.0]: https://github.com/deltachat/deltachat-core-rust/compare/v1.145.0..v1.146.0
[1.147.0]: https://github.com/deltachat/deltachat-core-rust/compare/v1.146.0..v1.147.0
[1.147.1]: https://github.com/deltachat/deltachat-core-rust/compare/v1.147.0..v1.147.1
[1.148.0]: https://github.com/deltachat/deltachat-core-rust/compare/v1.147.1..v1.148.0
[1.148.1]: https://github.com/deltachat/deltachat-core-rust/compare/v1.148.0..v1.148.1
[1.148.2]: https://github.com/deltachat/deltachat-core-rust/compare/v1.148.1..v1.148.2
[1.148.3]: https://github.com/deltachat/deltachat-core-rust/compare/v1.148.2..v1.148.3
[1.148.4]: https://github.com/deltachat/deltachat-core-rust/compare/v1.148.3..v1.148.4
[1.148.5]: https://github.com/deltachat/deltachat-core-rust/compare/v1.148.4..v1.148.5
+1 -1
View File
@@ -27,7 +27,7 @@ add_custom_command(
PREFIX=${CMAKE_INSTALL_PREFIX}
LIBDIR=${CMAKE_INSTALL_FULL_LIBDIR}
INCLUDEDIR=${CMAKE_INSTALL_FULL_INCLUDEDIR}
${CARGO} build --target-dir=${CMAKE_BINARY_DIR}/target --release --no-default-features --features jsonrpc
${CARGO} build --target-dir=${CMAKE_BINARY_DIR}/target --release --features jsonrpc
WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}/deltachat-ffi
)
Generated
+5 -5
View File
@@ -1293,7 +1293,7 @@ dependencies = [
[[package]]
name = "deltachat"
version = "1.147.1"
version = "1.148.5"
dependencies = [
"anyhow",
"async-broadcast",
@@ -1393,7 +1393,7 @@ dependencies = [
[[package]]
name = "deltachat-jsonrpc"
version = "1.147.1"
version = "1.148.5"
dependencies = [
"anyhow",
"async-channel 2.3.1",
@@ -1418,7 +1418,7 @@ dependencies = [
[[package]]
name = "deltachat-repl"
version = "1.147.1"
version = "1.148.5"
dependencies = [
"anyhow",
"deltachat",
@@ -1434,7 +1434,7 @@ dependencies = [
[[package]]
name = "deltachat-rpc-server"
version = "1.147.1"
version = "1.148.5"
dependencies = [
"anyhow",
"deltachat",
@@ -1463,7 +1463,7 @@ dependencies = [
[[package]]
name = "deltachat_ffi"
version = "1.147.1"
version = "1.148.5"
dependencies = [
"anyhow",
"deltachat",
+3 -3
View File
@@ -1,6 +1,6 @@
[package]
name = "deltachat"
version = "1.147.1"
version = "1.148.5"
edition = "2021"
license = "MPL-2.0"
rust-version = "1.77"
@@ -169,8 +169,8 @@ async-channel = "2.3.1"
base64 = "0.22"
chrono = { version = "0.4.38", default-features = false }
deltachat-contact-tools = { path = "deltachat-contact-tools" }
deltachat-jsonrpc = { path = "deltachat-jsonrpc" }
deltachat = { path = "." }
deltachat-jsonrpc = { path = "deltachat-jsonrpc", default-features = false }
deltachat = { path = ".", default-features = false }
futures = "0.3.30"
futures-lite = "2.3.0"
libc = "0.2"
+12
View File
@@ -0,0 +1,12 @@
<path
style="fill:#ffffff;fill-opacity:1;stroke:none"
d="m 24.015419,1.2870249 c -12.549421,0 -22.7283936,10.1789711 -22.7283936,22.7283931 0,12.549422 10.1789726,22.728395 22.7283936,22.728395 14.337742,-0.342877 9.614352,-4.702705 23.697556,0.969161 -7.545453,-13.001555 -1.082973,-13.32964 -0.969161,-23.697556 0,-12.549422 -10.178973,-22.7283931 -22.728395,-22.7283931 z" />
<path
style="fill:#000000;fill-opacity:1;stroke:none"
d="M 23.982249,5.3106163 C 13.645822,5.4364005 5.2618355,13.92999 5.2618355,24.275753 c 0,10.345764 8.3839865,18.635301 18.7204135,18.509516 9.827724,-0.03951 7.516769,-5.489695 18.380082,-0.443187 -5.950849,-9.296115 0.201753,-10.533667 0.340336,-18.521947 0,-10.345766 -8.383989,-18.6353031 -18.720418,-18.5095187 z" />
<g
style="fill:#ffffff"
transform="scale(1.1342891,0.88160947)">
<path
d="m 21.360141,23.513382 q -1.218487,-1.364705 -3.387392,-3.265543 -2.388233,-2.095797 -3.216804,-3.289913 -0.828571,-1.218486 -0.828571,-2.6563 0,-2.144536 1.998318,-3.363022 1.998317,-1.2428565 5.215121,-1.2428565 3.216804,0 5.605037,1.0966375 2.412603,1.096638 2.412603,3.021846 0,0.92605 -0.584873,1.535293 -0.584874,0.609243 -1.364705,0.609243 -1.121008,0 -2.631931,-1.681511 -1.535292,-1.705881 -2.60756,-2.388233 -1.047898,-0.706722 -2.461343,-0.706722 -1.803359,0 -2.973106,0.804201 -1.145377,0.804201 -1.145377,2.047057 0,1.169747 0.950419,2.193275 0.950419,1.023529 4.898315,3.728568 4.215963,2.899998 5.946213,4.532769 1.75462,1.632772 2.851258,3.972265 1.096638,2.339494 1.096638,4.947055 0,4.581508 -3.241174,8.090749 -3.216804,3.484871 -7.530245,3.484871 -3.923526,0 -6.628566,-2.802519 -2.705039,-2.802518 -2.705039,-7.481506 0,-4.508399 2.973106,-7.530245 2.997477,-3.021846 7.359658,-3.655459 z m 1.072268,1.121008 q -6.994112,1.145377 -6.994112,9.601672 0,4.36218 1.730251,6.774783 1.75462,2.412603 4.069744,2.412603 2.412603,0 3.972265,-2.315124 1.559663,-2.339493 1.559663,-6.311759 0,-5.751255 -4.337811,-10.162175 z" />
</g>
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "deltachat_ffi"
version = "1.147.1"
version = "1.148.5"
description = "Deltachat FFI"
edition = "2018"
readme = "README.md"
+38 -3
View File
@@ -522,6 +522,8 @@ char* dc_get_blobdir (const dc_context_t* context);
* In contrast to `dc_set_chat_mute_duration()`,
* fresh message and badge counters are not changed by this setting,
* but should be tuned down where appropriate.
* - `private_tag` = Optional tag as "Work", "Family".
* Meant to help profile owner to differ between profiles with similar names.
* - `ui.*` = All keys prefixed by `ui.` can be used by the user-interfaces for system-specific purposes.
* The prefix should be followed by the system and maybe subsystem,
* e.g. `ui.desktop.foo`, `ui.desktop.linux.bar`, `ui.android.foo`, `ui.dc40.bar`, `ui.bot.simplebot.baz`.
@@ -2531,8 +2533,8 @@ void dc_stop_ongoing_process (dc_context_t* context);
* ask the user if they want to use the given service for video chats;
* if so, call dc_set_config_from_qr().
*
* - DC_QR_SOCKS5_PROXY with dc_lot_t::text1=host, dc_lot_t::text2=port:
* ask the user if they want to use the given proxy and overwrite the previous one, if any.
* - DC_QR_PROXY with dc_lot_t::text1=address:
* ask the user if they want to use the given proxy.
* if so, call dc_set_config_from_qr() and restart I/O.
*
* - DC_QR_ADDR with dc_lot_t::id=Contact ID:
@@ -2609,6 +2611,7 @@ char* dc_get_securejoin_qr (dc_context_t* context, uint32_t ch
* Get QR code image from the QR code text generated by dc_get_securejoin_qr().
* See dc_get_securejoin_qr() for details about the contained QR code.
*
* @deprecated 2024-10 use dc_create_qr_svg(dc_get_securejoin_qr()) instead.
* @memberof dc_context_t
* @param context The context object.
* @param chat_id group-chat-id for secure-join or 0 for setup-contact,
@@ -2789,6 +2792,22 @@ dc_array_t* dc_get_locations (dc_context_t* context, uint32_t cha
void dc_delete_all_locations (dc_context_t* context);
// misc
/**
* Create a QR code from any input data.
*
* The QR code is returned as a square SVG image.
*
* @memberof dc_context_t
* @param payload The content for the QR code.
* @return SVG image with the QR code.
* On errors, an empty string is returned.
* The returned string must be released using dc_str_unref() after usage.
*/
char* dc_create_qr_svg (const char* payload);
/**
* Get last error string.
*
@@ -2877,6 +2896,7 @@ char* dc_backup_provider_get_qr (const dc_backup_provider_t* backup_provider);
* This works like dc_backup_provider_qr() but returns the text of a rendered
* SVG image containing the QR code.
*
* @deprecated 2024-10 use dc_create_qr_svg(dc_backup_provider_get_qr()) instead.
* @memberof dc_backup_provider_t
* @param backup_provider The backup provider object as created by
* dc_backup_provider_new().
@@ -2916,7 +2936,7 @@ void dc_backup_provider_unref (dc_backup_provider_t* backup_provider);
* Gets a backup offered by a dc_backup_provider_t object on another device.
*
* This function is called on a device that scanned the QR code offered by
* dc_backup_sender_qr() or dc_backup_sender_qr_svg(). Typically this is a
* dc_backup_provider_get_qr(). Typically this is a
* different device than that which provides the backup.
*
* This call will block while the backup is being transferred and only
@@ -6031,6 +6051,21 @@ void dc_event_unref(dc_event_t* event);
#define DC_EVENT_REACTIONS_CHANGED 2001
/**
* A reaction to one's own sent message received.
* Typically, the UI will show a notification for that.
*
* In addition to this event, DC_EVENT_REACTIONS_CHANGED is emitted.
*
* @param data1 (int) contact_id ID of the contact sending this reaction.
* @param data2 (int) msg_id + (char*) reaction.
* ID of the message for which a reaction was received in dc_event_get_data2_int(),
* and the reaction as dc_event_get_data2_str().
* string must be passed to dc_str_unref() afterwards.
*/
#define DC_EVENT_INCOMING_REACTION 2002
/**
* There is a fresh message. Typically, the user will show an notification
* when receiving this message.
+41 -9
View File
@@ -30,7 +30,7 @@ use deltachat::ephemeral::Timer as EphemeralTimer;
use deltachat::imex::BackupProvider;
use deltachat::key::preconfigure_keypair;
use deltachat::message::MsgId;
use deltachat::qr_code_generator::{generate_backup_qr, get_securejoin_qr_svg};
use deltachat::qr_code_generator::{create_qr_svg, generate_backup_qr, get_securejoin_qr_svg};
use deltachat::stock_str::StockMessage;
use deltachat::webxdc::StatusUpdateSerial;
use deltachat::*;
@@ -541,6 +541,7 @@ pub unsafe extern "C" fn dc_event_get_id(event: *mut dc_event_t) -> libc::c_int
EventType::ErrorSelfNotInGroup(_) => 410,
EventType::MsgsChanged { .. } => 2000,
EventType::ReactionsChanged { .. } => 2001,
EventType::IncomingReaction { .. } => 2002,
EventType::IncomingMsg { .. } => 2005,
EventType::IncomingMsgBunch { .. } => 2006,
EventType::MsgsNoticed { .. } => 2008,
@@ -568,6 +569,9 @@ pub unsafe extern "C" fn dc_event_get_id(event: *mut dc_event_t) -> libc::c_int
EventType::ChatlistChanged => 2300,
EventType::ChatlistItemChanged { .. } => 2301,
EventType::EventChannelOverflow { .. } => 2400,
#[allow(unreachable_patterns)]
#[cfg(test)]
_ => unreachable!("This is just to silence a rust_analyzer false-positive"),
}
}
@@ -598,6 +602,7 @@ pub unsafe extern "C" fn dc_event_get_data1_int(event: *mut dc_event_t) -> libc:
| EventType::ErrorSelfNotInGroup(_)
| EventType::AccountsBackgroundFetchDone => 0,
EventType::ChatlistChanged => 0,
EventType::IncomingReaction { contact_id, .. } => contact_id.to_u32() as libc::c_int,
EventType::MsgsChanged { chat_id, .. }
| EventType::ReactionsChanged { chat_id, .. }
| EventType::IncomingMsg { chat_id, .. }
@@ -628,6 +633,9 @@ pub unsafe extern "C" fn dc_event_get_data1_int(event: *mut dc_event_t) -> libc:
chat_id.unwrap_or_default().to_u32() as libc::c_int
}
EventType::EventChannelOverflow { n } => *n as libc::c_int,
#[allow(unreachable_patterns)]
#[cfg(test)]
_ => unreachable!("This is just to silence a rust_analyzer false-positive"),
}
}
@@ -672,6 +680,7 @@ pub unsafe extern "C" fn dc_event_get_data2_int(event: *mut dc_event_t) -> libc:
| EventType::EventChannelOverflow { .. } => 0,
EventType::MsgsChanged { msg_id, .. }
| EventType::ReactionsChanged { msg_id, .. }
| EventType::IncomingReaction { msg_id, .. }
| EventType::IncomingMsg { msg_id, .. }
| EventType::MsgDelivered { msg_id, .. }
| EventType::MsgFailed { msg_id, .. }
@@ -685,6 +694,9 @@ pub unsafe extern "C" fn dc_event_get_data2_int(event: *mut dc_event_t) -> libc:
..
} => status_update_serial.to_u32() as libc::c_int,
EventType::WebxdcRealtimeData { data, .. } => data.len() as libc::c_int,
#[allow(unreachable_patterns)]
#[cfg(test)]
_ => unreachable!("This is just to silence a rust_analyzer false-positive"),
}
}
@@ -758,6 +770,14 @@ pub unsafe extern "C" fn dc_event_get_data2_str(event: *mut dc_event_t) -> *mut
libc::memcpy(ptr, data.as_ptr() as *mut libc::c_void, data.len());
ptr as *mut libc::c_char
}
EventType::IncomingReaction { reaction, .. } => reaction
.as_str()
.to_c_string()
.unwrap_or_default()
.into_raw(),
#[allow(unreachable_patterns)]
#[cfg(test)]
_ => unreachable!("This is just to silence a rust_analyzer false-positive"),
}
}
@@ -2574,6 +2594,18 @@ pub unsafe extern "C" fn dc_delete_all_locations(context: *mut dc_context_t) {
});
}
#[no_mangle]
pub unsafe extern "C" fn dc_create_qr_svg(payload: *const libc::c_char) -> *mut libc::c_char {
if payload.is_null() {
eprintln!("ignoring careless call to dc_create_qr_svg()");
return "".strdup();
}
create_qr_svg(&to_string_lossy(payload))
.unwrap_or_else(|_| "".to_string())
.strdup()
}
#[no_mangle]
pub unsafe extern "C" fn dc_get_last_error(context: *mut dc_context_t) -> *mut libc::c_char {
if context.is_null() {
@@ -4836,7 +4868,7 @@ pub unsafe extern "C" fn dc_accounts_maybe_network_lost(accounts: *mut dc_accoun
}
let accounts = &*accounts;
block_on(async move { accounts.write().await.maybe_network_lost().await });
block_on(async move { accounts.read().await.maybe_network_lost().await });
}
#[no_mangle]
@@ -4850,12 +4882,12 @@ pub unsafe extern "C" fn dc_accounts_background_fetch(
}
let accounts = &*accounts;
block_on(async move {
let accounts = accounts.read().await;
accounts
.background_fetch(Duration::from_secs(timeout_in_seconds))
.await;
});
let background_fetch_future = {
let lock = block_on(accounts.read());
lock.background_fetch(Duration::from_secs(timeout_in_seconds))
};
// At this point account manager is not locked anymore.
block_on(background_fetch_future);
1
}
@@ -4873,7 +4905,7 @@ pub unsafe extern "C" fn dc_accounts_set_push_device_token(
let token = to_string_lossy(token);
block_on(async move {
let mut accounts = accounts.write().await;
let accounts = accounts.read().await;
if let Err(err) = accounts.set_push_device_token(&token).await {
accounts.emit_event(EventType::Error(format!(
"Failed to set notify token: {err:#}."
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "deltachat-jsonrpc"
version = "1.147.1"
version = "1.148.5"
description = "DeltaChat JSON-RPC API"
edition = "2021"
default-run = "deltachat-jsonrpc-server"
+7 -7
View File
@@ -254,11 +254,12 @@ impl CommandApi {
/// Process all events until you get this one and you can safely return to the background
/// without forgetting to create notifications caused by timing race conditions.
async fn accounts_background_fetch(&self, timeout_in_seconds: f64) -> Result<()> {
self.accounts
.write()
.await
.background_fetch(std::time::Duration::from_secs_f64(timeout_in_seconds))
.await;
let future = {
let lock = self.accounts.read().await;
lock.background_fetch(std::time::Duration::from_secs_f64(timeout_in_seconds))
};
// At this point account manager is not locked anymore.
future.await;
Ok(())
}
@@ -2117,8 +2118,7 @@ impl CommandApi {
) -> Result<u32> {
let ctx = self.get_context(account_id).await?;
let mut msg = Message::new(Viewtype::Text);
msg.set_text(text);
let mut msg = Message::new_text(text);
let message_id = deltachat::chat::send_msg(&ctx, ChatId::new(chat_id), &mut msg).await?;
Ok(message_id.to_u32())
@@ -17,6 +17,9 @@ pub enum Account {
// size: u32,
profile_image: Option<String>, // TODO: This needs to be converted to work with blob http server.
color: String,
/// Optional tag as "Work", "Family".
/// Meant to help profile owner to differ between profiles with similar names.
private_tag: Option<String>,
},
#[serde(rename_all = "camelCase")]
Unconfigured { id: u32 },
@@ -31,12 +34,14 @@ impl Account {
let color = color_int_to_hex_string(
Contact::get_by_id(ctx, ContactId::SELF).await?.get_color(),
);
let private_tag = ctx.get_config(Config::PrivateTag).await?;
Ok(Account::Configured {
id,
display_name,
addr,
profile_image,
color,
private_tag,
})
} else {
Ok(Account::Unconfigured { id })
+17
View File
@@ -98,6 +98,14 @@ pub enum EventType {
contact_id: u32,
},
/// Incoming reaction, should be notified.
#[serde(rename_all = "camelCase")]
IncomingReaction {
contact_id: u32,
msg_id: u32,
reaction: String,
},
/// There is a fresh message. Typically, the user will show an notification
/// when receiving this message.
///
@@ -302,6 +310,15 @@ impl From<CoreEventType> for EventType {
msg_id: msg_id.to_u32(),
contact_id: contact_id.to_u32(),
},
CoreEventType::IncomingReaction {
contact_id,
msg_id,
reaction,
} => IncomingReaction {
contact_id: contact_id.to_u32(),
msg_id: msg_id.to_u32(),
reaction: reaction.as_str().to_string(),
},
CoreEventType::IncomingMsg { chat_id, msg_id } => IncomingMsg {
chat_id: chat_id.to_u32(),
msg_id: msg_id.to_u32(),
+1 -1
View File
@@ -58,5 +58,5 @@
},
"type": "module",
"types": "dist/deltachat.d.ts",
"version": "1.147.1"
"version": "1.148.5"
}
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "deltachat-repl"
version = "1.147.1"
version = "1.148.5"
license = "MPL-2.0"
edition = "2021"
repository = "https://github.com/deltachat/deltachat-core-rust"
+11 -4
View File
@@ -22,6 +22,7 @@ use deltachat::mimeparser::SystemMessage;
use deltachat::peer_channels::{send_webxdc_realtime_advertisement, send_webxdc_realtime_data};
use deltachat::peerstate::*;
use deltachat::qr::*;
use deltachat::qr_code_generator::create_qr_svg;
use deltachat::reaction::send_reaction;
use deltachat::receive_imf::*;
use deltachat::sql;
@@ -425,6 +426,7 @@ pub async fn cmdline(context: Context, line: &str, chat_id: &mut ChatId) -> Resu
checkqr <qr-content>\n\
joinqr <qr-content>\n\
setqr <qr-content>\n\
createqrsvg <qr-content>\n\
providerinfo <addr>\n\
fileinfo <file>\n\
estimatedeletion <seconds>\n\
@@ -1002,8 +1004,7 @@ pub async fn cmdline(context: Context, line: &str, chat_id: &mut ChatId) -> Resu
ensure!(sel_chat.is_some(), "No chat selected.");
if !arg1.is_empty() {
let mut draft = Message::new(Viewtype::Text);
draft.set_text(arg1.to_string());
let mut draft = Message::new_text(arg1.to_string());
sel_chat
.as_ref()
.unwrap()
@@ -1026,8 +1027,7 @@ pub async fn cmdline(context: Context, line: &str, chat_id: &mut ChatId) -> Resu
!arg1.is_empty(),
"Please specify text to add as device message."
);
let mut msg = Message::new(Viewtype::Text);
msg.set_text(arg1.to_string());
let mut msg = Message::new_text(arg1.to_string());
chat::add_device_msg(&context, None, Some(&mut msg)).await?;
}
"listmedia" => {
@@ -1249,6 +1249,13 @@ pub async fn cmdline(context: Context, line: &str, chat_id: &mut ChatId) -> Resu
Err(err) => println!("Cannot set config from QR code: {err:?}"),
}
}
"createqrsvg" => {
ensure!(!arg1.is_empty(), "Argument <qr-content> missing.");
let svg = create_qr_svg(arg1)?;
let file = dirs::home_dir().unwrap_or_default().join("qr.svg");
fs::write(&file, svg).await?;
println!("{file:#?} written.");
}
"providerinfo" => {
ensure!(!arg1.is_empty(), "Argument <addr> missing.");
let proxy_enabled = context
+2 -1
View File
@@ -240,12 +240,13 @@ const CONTACT_COMMANDS: [&str; 9] = [
"unblock",
"listblocked",
];
const MISC_COMMANDS: [&str; 11] = [
const MISC_COMMANDS: [&str; 12] = [
"getqr",
"getqrsvg",
"getbadqr",
"checkqr",
"joinqr",
"createqrsvg",
"fileinfo",
"clear",
"exit",
+1 -1
View File
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "deltachat-rpc-client"
version = "1.147.1"
version = "1.148.5"
description = "Python client for Delta Chat core JSON-RPC interface"
classifiers = [
"Development Status :: 5 - Production/Stable",
@@ -63,6 +63,7 @@ class EventType(str, Enum):
CHATLIST_ITEM_CHANGED = "ChatlistItemChanged"
CONFIG_SYNCED = "ConfigSynced"
WEBXDC_REALTIME_DATA = "WebxdcRealtimeData"
WEBXDC_REALTIME_ADVERTISEMENT_RECEIVED = "WebxdcRealtimeAdvertisementReceived"
class ChatId(IntEnum):
+30 -2
View File
@@ -7,6 +7,7 @@ If you want to debug iroh at rust-trace/log level set
RUST_LOG=iroh_net=trace,iroh_gossip=trace
"""
import os
import sys
import threading
import time
@@ -107,13 +108,15 @@ def test_realtime_sequentially(acfactory, path_to_webxdc):
assert snapshot.text == "ping2"
log("sending realtime data ac1 -> ac2")
ac1_webxdc_msg.send_webxdc_realtime_data(b"foo")
# Test that 128 KB of data can be sent in a single message.
data = os.urandom(128000)
ac1_webxdc_msg.send_webxdc_realtime_data(data)
log("ac2: waiting for realtime data")
while 1:
event = ac2.wait_for_event()
if event.kind == EventType.WEBXDC_REALTIME_DATA:
assert event.data == list(b"foo")
assert event.data == list(data)
break
@@ -208,3 +211,28 @@ def test_no_reordering(acfactory, path_to_webxdc):
if event.data[0] == i:
break
pytest.fail("Reordering detected")
def test_advertisement_after_chatting(acfactory, path_to_webxdc):
"""Test that realtime advertisement is assigned to the correct message after chatting."""
ac1, ac2 = acfactory.get_online_accounts(2)
ac1.set_config("webxdc_realtime_enabled", "1")
ac2.set_config("webxdc_realtime_enabled", "1")
ac1_ac2_chat = ac1.create_chat(ac2)
ac1_webxdc_msg = ac1_ac2_chat.send_message(text="WebXDC", file=path_to_webxdc)
ac2_webxdc_msg = ac2.wait_for_incoming_msg()
assert ac2_webxdc_msg.get_snapshot().text == "WebXDC"
ac1_ac2_chat.send_text("Hello!")
ac2_hello_msg = ac2.wait_for_incoming_msg()
ac2_hello_msg_snapshot = ac2_hello_msg.get_snapshot()
assert ac2_hello_msg_snapshot.text == "Hello!"
ac2_hello_msg_snapshot.chat.accept()
ac2_webxdc_msg.send_webxdc_realtime_advertisement()
while 1:
event = ac1.wait_for_event()
if event.kind == EventType.WEBXDC_REALTIME_ADVERTISEMENT_RECEIVED:
assert event.msg_id == ac1_webxdc_msg.id
break
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "deltachat-rpc-server"
version = "1.147.1"
version = "1.148.5"
description = "DeltaChat JSON-RPC server"
edition = "2021"
readme = "README.md"
@@ -15,5 +15,5 @@
},
"type": "module",
"types": "index.d.ts",
"version": "1.147.1"
"version": "1.148.5"
}
Generated
+71 -38
View File
@@ -3,15 +3,15 @@
"android": {
"inputs": {
"devshell": "devshell",
"flake-utils": "flake-utils",
"flake-utils": "flake-utils_2",
"nixpkgs": "nixpkgs"
},
"locked": {
"lastModified": 1729369131,
"narHash": "sha256-PtfScp+nQd1PsT5rf0Qgjdbsh4Iag6R1ivYMWLizyIc=",
"lastModified": 1712088936,
"narHash": "sha256-mVjeSWQiR/t4UZ9fUawY9OEPAhY1R3meYG+0oh8DUBs=",
"owner": "tadfisher",
"repo": "android-nixpkgs",
"rev": "82bffbf3f06bdccf44fc62a9bd4f152ac80a55b0",
"rev": "2d8181caef279f19c4a33dc694723f89ffc195d4",
"type": "github"
},
"original": {
@@ -22,17 +22,18 @@
},
"devshell": {
"inputs": {
"flake-utils": "flake-utils",
"nixpkgs": [
"android",
"nixpkgs"
]
},
"locked": {
"lastModified": 1728330715,
"narHash": "sha256-xRJ2nPOXb//u1jaBnDP56M7v5ldavjbtR6lfGqSvcKg=",
"lastModified": 1711099426,
"narHash": "sha256-HzpgM/wc3aqpnHJJ2oDqPBkNsqWbW0WfWUO8lKu8nGk=",
"owner": "numtide",
"repo": "devshell",
"rev": "dd6b80932022cea34a019e2bb32f6fa9e494dfef",
"rev": "2d45b54ca4a183f2fdcf4b19c895b64fbf620ee8",
"type": "github"
},
"original": {
@@ -47,11 +48,11 @@
"rust-analyzer-src": "rust-analyzer-src"
},
"locked": {
"lastModified": 1729375822,
"narHash": "sha256-bRo4xVwUhvJ4Gz+OhWMREFMdBOYSw4Yi1Apj01ebbug=",
"lastModified": 1714112748,
"narHash": "sha256-jq6Cpf/pQH85p+uTwPPrGG8Ky/zUOTwMJ7mcqc5M4So=",
"owner": "nix-community",
"repo": "fenix",
"rev": "2853e7d9b5c52a148a9fb824bfe4f9f433f557ab",
"rev": "3ae4b908a795b6a3824d401a0702e11a7157d7e1",
"type": "github"
},
"original": {
@@ -65,11 +66,11 @@
"systems": "systems"
},
"locked": {
"lastModified": 1726560853,
"narHash": "sha256-X6rJYSESBVr3hBoH0WbKE5KvhPU5bloyZ2L4K60/fPQ=",
"lastModified": 1701680307,
"narHash": "sha256-kAuep2h5ajznlPMD9rnQyffWG8EM/C73lejGofXvdM8=",
"owner": "numtide",
"repo": "flake-utils",
"rev": "c1dfcf08411b08f6b8615f7d8971a2bfa81d5e8a",
"rev": "4022d587cbbfd70fe950c1e2083a02621806a725",
"type": "github"
},
"original": {
@@ -83,11 +84,29 @@
"systems": "systems_2"
},
"locked": {
"lastModified": 1726560853,
"narHash": "sha256-X6rJYSESBVr3hBoH0WbKE5KvhPU5bloyZ2L4K60/fPQ=",
"lastModified": 1710146030,
"narHash": "sha256-SZ5L6eA7HJ/nmkzGG7/ISclqe6oZdOZTNoesiInkXPQ=",
"owner": "numtide",
"repo": "flake-utils",
"rev": "c1dfcf08411b08f6b8615f7d8971a2bfa81d5e8a",
"rev": "b1d9ab70662946ef0850d488da1c9019f3a9752a",
"type": "github"
},
"original": {
"owner": "numtide",
"repo": "flake-utils",
"type": "github"
}
},
"flake-utils_3": {
"inputs": {
"systems": "systems_3"
},
"locked": {
"lastModified": 1710146030,
"narHash": "sha256-SZ5L6eA7HJ/nmkzGG7/ISclqe6oZdOZTNoesiInkXPQ=",
"owner": "numtide",
"repo": "flake-utils",
"rev": "b1d9ab70662946ef0850d488da1c9019f3a9752a",
"type": "github"
},
"original": {
@@ -101,11 +120,11 @@
"nixpkgs": "nixpkgs_3"
},
"locked": {
"lastModified": 1721727458,
"narHash": "sha256-r/xppY958gmZ4oTfLiHN0ZGuQ+RSTijDblVgVLFi1mw=",
"lastModified": 1713520724,
"narHash": "sha256-CO8MmVDmqZX2FovL75pu5BvwhW+Vugc7Q6ze7Hj8heI=",
"owner": "nix-community",
"repo": "naersk",
"rev": "3fb418eaf352498f6b6c30592e3beb63df42ef11",
"rev": "c5037590290c6c7dae2e42e7da1e247e54ed2d49",
"type": "github"
},
"original": {
@@ -131,11 +150,11 @@
},
"nixpkgs": {
"locked": {
"lastModified": 1729256560,
"narHash": "sha256-/uilDXvCIEs3C9l73JTACm4quuHUsIHcns1c+cHUJwA=",
"lastModified": 1711703276,
"narHash": "sha256-iMUFArF0WCatKK6RzfUJknjem0H9m4KgorO/p3Dopkk=",
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "4c2fcb090b1f3e5b47eaa7bd33913b574a11e0a0",
"rev": "d8fe5e6c92d0d190646fb9f1056741a229980089",
"type": "github"
},
"original": {
@@ -147,11 +166,11 @@
},
"nixpkgs_2": {
"locked": {
"lastModified": 1729070438,
"narHash": "sha256-KOTTUfPkugH52avUvXGxvWy8ibKKj4genodIYUED+Kc=",
"lastModified": 1713895582,
"narHash": "sha256-cfh1hi+6muQMbi9acOlju3V1gl8BEaZBXBR9jQfQi4U=",
"owner": "nixos",
"repo": "nixpkgs",
"rev": "5785b6bb5eaae44e627d541023034e1601455827",
"rev": "572af610f6151fd41c212f897c71f7056e3fb518",
"type": "github"
},
"original": {
@@ -163,12 +182,11 @@
},
"nixpkgs_3": {
"locked": {
"lastModified": 1729265718,
"narHash": "sha256-4HQI+6LsO3kpWTYuVGIzhJs1cetFcwT7quWCk/6rqeo=",
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "ccc0c2126893dd20963580b6478d1a10a4512185",
"type": "github"
"lastModified": 1711668574,
"narHash": "sha256-u1dfs0ASQIEr1icTVrsKwg2xToIpn7ZXxW3RHfHxshg=",
"path": "/nix/store/9fpv0kjq9a80isa1wkkvrdqsh9dpcn05-source",
"rev": "219951b495fc2eac67b1456824cc1ec1fd2ee659",
"type": "path"
},
"original": {
"id": "nixpkgs",
@@ -177,11 +195,11 @@
},
"nixpkgs_4": {
"locked": {
"lastModified": 1729256560,
"narHash": "sha256-/uilDXvCIEs3C9l73JTACm4quuHUsIHcns1c+cHUJwA=",
"lastModified": 1729413321,
"narHash": "sha256-I4tuhRpZFa6Fu6dcH9Dlo5LlH17peT79vx1y1SpeKt0=",
"owner": "nixos",
"repo": "nixpkgs",
"rev": "4c2fcb090b1f3e5b47eaa7bd33913b574a11e0a0",
"rev": "1997e4aa514312c1af7e2bda7fad1644e778ff26",
"type": "github"
},
"original": {
@@ -195,7 +213,7 @@
"inputs": {
"android": "android",
"fenix": "fenix",
"flake-utils": "flake-utils_2",
"flake-utils": "flake-utils_3",
"naersk": "naersk",
"nix-filter": "nix-filter",
"nixpkgs": "nixpkgs_4"
@@ -204,11 +222,11 @@
"rust-analyzer-src": {
"flake": false,
"locked": {
"lastModified": 1729255720,
"narHash": "sha256-yODOuZxBkS0UfqMa6nmbqNbVfIbsu0tYLbV5vZzmsqI=",
"lastModified": 1714031783,
"narHash": "sha256-xS/niQsq1CQPOe4M4jvVPO2cnXS/EIeRG5gIopUbk+Q=",
"owner": "rust-lang",
"repo": "rust-analyzer",
"rev": "72b214fbfbe6f7b95a7877b962783bd42062cc0a",
"rev": "56bee2ddafa6177b19c631eedc88d43366553223",
"type": "github"
},
"original": {
@@ -247,6 +265,21 @@
"repo": "default",
"type": "github"
}
},
"systems_3": {
"locked": {
"lastModified": 1681028828,
"narHash": "sha256-Vy1rq5AaRuLzOxct8nz4T6wlgyUR7zLU309k9mBC768=",
"owner": "nix-systems",
"repo": "default",
"rev": "da67096a3b9bf56a91d16901293e51ba5b49a27e",
"type": "github"
},
"original": {
"owner": "nix-systems",
"repo": "default",
"type": "github"
}
}
},
"root": "root",
+1
View File
@@ -50,6 +50,7 @@ module.exports = {
DC_EVENT_IMEX_PROGRESS: 2051,
DC_EVENT_INCOMING_MSG: 2005,
DC_EVENT_INCOMING_MSG_BUNCH: 2006,
DC_EVENT_INCOMING_REACTION: 2002,
DC_EVENT_INFO: 100,
DC_EVENT_LOCATION_CHANGED: 2035,
DC_EVENT_MSGS_CHANGED: 2000,
+1
View File
@@ -16,6 +16,7 @@ module.exports = {
410: 'DC_EVENT_ERROR_SELF_NOT_IN_GROUP',
2000: 'DC_EVENT_MSGS_CHANGED',
2001: 'DC_EVENT_REACTIONS_CHANGED',
2002: 'DC_EVENT_INCOMING_REACTION',
2005: 'DC_EVENT_INCOMING_MSG',
2006: 'DC_EVENT_INCOMING_MSG_BUNCH',
2008: 'DC_EVENT_MSGS_NOTICED',
+2
View File
@@ -50,6 +50,7 @@ export enum C {
DC_EVENT_IMEX_PROGRESS = 2051,
DC_EVENT_INCOMING_MSG = 2005,
DC_EVENT_INCOMING_MSG_BUNCH = 2006,
DC_EVENT_INCOMING_REACTION = 2002,
DC_EVENT_INFO = 100,
DC_EVENT_LOCATION_CHANGED = 2035,
DC_EVENT_MSGS_CHANGED = 2000,
@@ -322,6 +323,7 @@ export const EventId2EventName: { [key: number]: string } = {
410: 'DC_EVENT_ERROR_SELF_NOT_IN_GROUP',
2000: 'DC_EVENT_MSGS_CHANGED',
2001: 'DC_EVENT_REACTIONS_CHANGED',
2002: 'DC_EVENT_INCOMING_REACTION',
2005: 'DC_EVENT_INCOMING_MSG',
2006: 'DC_EVENT_INCOMING_MSG_BUNCH',
2008: 'DC_EVENT_MSGS_NOTICED',
+1 -1
View File
@@ -55,5 +55,5 @@
"test:mocha": "mocha node/test/test.mjs --growl --reporter=spec --bail --exit"
},
"types": "node/dist/index.d.ts",
"version": "1.147.1"
"version": "1.148.5"
}
+1 -1
View File
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "deltachat"
version = "1.147.1"
version = "1.148.5"
description = "Python bindings for the Delta Chat Core library using CFFI against the Rust-implemented libdeltachat"
readme = "README.rst"
requires-python = ">=3.7"
+1 -1
View File
@@ -308,7 +308,7 @@ class Chat:
msg = as_dc_charpointer(text)
msg_id = lib.dc_send_text_msg(self.account._dc_context, self.id, msg)
if msg_id == 0:
raise ValueError("message could not be send, does chat exist?")
raise ValueError("The message could not be sent. Does the chat exist?")
return Message.from_db(self.account, msg_id)
def send_file(self, path, mime_type="application/octet-stream"):
+13
View File
@@ -2209,6 +2209,19 @@ def test_configure_error_msgs_wrong_pw(acfactory):
# Password is wrong so it definitely has to say something about "password"
assert "password" in ev.data2
ac1.stop_io()
ac1.set_config("mail_pw", "abc") # Wrong mail pw
ac1.configure()
while True:
ev = ac1._evtracker.get_matching("DC_EVENT_CONFIGURE_PROGRESS")
print(f"Configuration progress: {ev.data1}")
if ev.data1 == 0:
break
assert "password" in ev.data2
# Account will continue to work with the old password, so if it becomes wrong, a notification
# must be shown.
assert ac1.get_config("notify_about_wrong_pw") == "1"
def test_configure_error_msgs_invalid_server(acfactory):
ac2 = acfactory.get_unconfigured_account()
+1 -1
View File
@@ -1 +1 @@
2024-10-13
2024-10-27
+45 -18
View File
@@ -5,7 +5,8 @@ use std::future::Future;
use std::path::{Path, PathBuf};
use anyhow::{ensure, Context as _, Result};
use futures::future::join_all;
use futures::stream::FuturesUnordered;
use futures::StreamExt;
use serde::{Deserialize, Serialize};
use tokio::fs;
use tokio::io::AsyncWriteExt;
@@ -301,20 +302,48 @@ impl Accounts {
///
/// This is an auxiliary function and not part of public API.
/// Use [Accounts::background_fetch] instead.
async fn background_fetch_without_timeout(&self) {
async fn background_fetch_no_timeout(accounts: Vec<Context>, events: Events) {
async fn background_fetch_and_log_error(account: Context) {
if let Err(error) = account.background_fetch().await {
warn!(account, "{error:#}");
}
}
join_all(
self.accounts
.values()
.cloned()
.map(background_fetch_and_log_error),
events.emit(Event {
id: 0,
typ: EventType::Info(format!(
"Starting background fetch for {} accounts.",
accounts.len()
)),
});
let mut futures_unordered: FuturesUnordered<_> = accounts
.into_iter()
.map(background_fetch_and_log_error)
.collect();
while futures_unordered.next().await.is_some() {}
}
/// Auxiliary function for [Accounts::background_fetch].
async fn background_fetch_with_timeout(
accounts: Vec<Context>,
events: Events,
timeout: std::time::Duration,
) {
if let Err(_err) = tokio::time::timeout(
timeout,
Self::background_fetch_no_timeout(accounts, events.clone()),
)
.await;
.await
{
events.emit(Event {
id: 0,
typ: EventType::Warning("Background fetch timed out.".to_string()),
});
}
events.emit(Event {
id: 0,
typ: EventType::AccountsBackgroundFetchDone,
});
}
/// Performs a background fetch for all accounts in parallel with a timeout.
@@ -322,15 +351,13 @@ impl Accounts {
/// The `AccountsBackgroundFetchDone` event is emitted at the end,
/// process all events until you get this one and you can safely return to the background
/// without forgetting to create notifications caused by timing race conditions.
pub async fn background_fetch(&self, timeout: std::time::Duration) {
if let Err(_err) =
tokio::time::timeout(timeout, self.background_fetch_without_timeout()).await
{
self.emit_event(EventType::Warning(
"Background fetch timed out.".to_string(),
));
}
self.emit_event(EventType::AccountsBackgroundFetchDone);
///
/// Returns a future that resolves when background fetch is done,
/// but does not capture `&self`.
pub fn background_fetch(&self, timeout: std::time::Duration) -> impl Future<Output = ()> {
let accounts: Vec<Context> = self.accounts.values().cloned().collect();
let events = self.events.clone();
Self::background_fetch_with_timeout(accounts, events, timeout)
}
/// Emits a single event.
@@ -344,7 +371,7 @@ impl Accounts {
}
/// Sets notification token for Apple Push Notification service.
pub async fn set_push_device_token(&mut self, token: &str) -> Result<()> {
pub async fn set_push_device_token(&self, token: &str) -> Result<()> {
self.push_subscriber.set_device_token(token).await;
Ok(())
}
+37 -60
View File
@@ -797,8 +797,7 @@ impl ChatId {
context.scheduler.interrupt_inbox().await;
if chat.is_self_talk() {
let mut msg = Message::new(Viewtype::Text);
msg.text = stock_str::self_deleted_msg_body(context).await;
let mut msg = Message::new_text(stock_str::self_deleted_msg_body(context).await);
add_device_msg(context, None, Some(&mut msg)).await?;
}
chatlist_events::emit_chatlist_changed(context);
@@ -2092,28 +2091,31 @@ impl Chat {
EphemeralTimer::Enabled { duration } => time().saturating_add(duration.into()),
};
let (msg_text, was_truncated) = truncate_msg_text(context, msg.text.clone()).await?;
let new_mime_headers = if msg.has_html() {
let html = if msg.param.exists(Param::Forwarded) {
if msg.param.exists(Param::Forwarded) {
msg.get_id().get_html(context).await?
} else {
msg.param.get(Param::SendHtml).map(|s| s.to_string())
};
match html {
Some(html) => Some(tokio::task::block_in_place(move || {
buf_compress(new_html_mimepart(html).build().as_string().as_bytes())
})?),
None => None,
}
} else {
None
};
let new_mime_headers = new_mime_headers.or_else(|| match was_truncated {
true => Some(msg.text.clone()),
false => None,
});
let new_mime_headers = match new_mime_headers {
Some(h) => Some(tokio::task::block_in_place(move || {
buf_compress(new_html_mimepart(h).build().as_string().as_bytes())
})?),
None => None,
};
msg.chat_id = self.id;
msg.from_id = ContactId::SELF;
msg.rfc724_mid = new_rfc724_mid;
msg.timestamp_sort = timestamp;
let (msg_text, was_truncated) = truncate_msg_text(context, msg.text.clone()).await?;
let mime_modified = new_mime_headers.is_some() | was_truncated;
// add message to the database
if let Some(update_msg_id) = update_msg_id {
@@ -2142,7 +2144,7 @@ impl Chat {
msg.hidden,
msg.in_reply_to.as_deref().unwrap_or_default(),
new_references,
mime_modified,
new_mime_headers.is_some(),
new_mime_headers.unwrap_or_default(),
location_id as i32,
ephemeral_timer,
@@ -2193,7 +2195,7 @@ impl Chat {
msg.hidden,
msg.in_reply_to.as_deref().unwrap_or_default(),
new_references,
mime_modified,
new_mime_headers.is_some(),
new_mime_headers.unwrap_or_default(),
location_id as i32,
ephemeral_timer,
@@ -3103,8 +3105,7 @@ pub async fn send_text_msg(
chat_id
);
let mut msg = Message::new(Viewtype::Text);
msg.text = text_to_send;
let mut msg = Message::new_text(text_to_send);
send_msg(context, chat_id, &mut msg).await
}
@@ -4777,8 +4778,7 @@ mod tests {
async fn test_get_draft() {
let t = TestContext::new().await;
let chat_id = &t.get_self_chat().await.id;
let mut msg = Message::new(Viewtype::Text);
msg.set_text("hello".to_string());
let mut msg = Message::new_text("hello".to_string());
chat_id.set_draft(&t, Some(&mut msg)).await.unwrap();
let draft = chat_id.get_draft(&t).await.unwrap().unwrap();
@@ -4792,13 +4792,11 @@ mod tests {
let t = TestContext::new_alice().await;
let chat_id = create_group_chat(&t, ProtectionStatus::Unprotected, "abc").await?;
let mut msg = Message::new(Viewtype::Text);
msg.set_text("hi!".to_string());
let mut msg = Message::new_text("hi!".to_string());
chat_id.set_draft(&t, Some(&mut msg)).await?;
assert!(chat_id.get_draft(&t).await?.is_some());
let mut msg = Message::new(Viewtype::Text);
msg.set_text("another".to_string());
let mut msg = Message::new_text("another".to_string());
chat_id.set_draft(&t, Some(&mut msg)).await?;
assert!(chat_id.get_draft(&t).await?.is_some());
@@ -4812,8 +4810,7 @@ mod tests {
async fn test_forwarding_draft_failing() -> Result<()> {
let t = TestContext::new_alice().await;
let chat_id = &t.get_self_chat().await.id;
let mut msg = Message::new(Viewtype::Text);
msg.set_text("hello".to_string());
let mut msg = Message::new_text("hello".to_string());
chat_id.set_draft(&t, Some(&mut msg)).await?;
assert_eq!(msg.id, chat_id.get_draft(&t).await?.unwrap().id);
@@ -4826,8 +4823,7 @@ mod tests {
async fn test_draft_stable_ids() -> Result<()> {
let t = TestContext::new_alice().await;
let chat_id = &t.get_self_chat().await.id;
let mut msg = Message::new(Viewtype::Text);
msg.set_text("hello".to_string());
let mut msg = Message::new_text("hello".to_string());
assert_eq!(msg.id, MsgId::new_unset());
assert!(chat_id.get_draft_msg_id(&t).await?.is_none());
@@ -4873,11 +4869,7 @@ mod tests {
let chat_id = create_group_chat(&t, ProtectionStatus::Unprotected, "abc").await?;
let msgs: Vec<message::Message> = (1..=1000)
.map(|i| {
let mut msg = Message::new(Viewtype::Text);
msg.set_text(i.to_string());
msg
})
.map(|i| Message::new_text(i.to_string()))
.collect();
let mut tasks = Vec::new();
for mut msg in msgs {
@@ -4910,8 +4902,7 @@ mod tests {
.await?;
// save a draft
let mut draft = Message::new(Viewtype::Text);
draft.set_text("draft text".to_string());
let mut draft = Message::new_text("draft text".to_string());
chat_id.set_draft(&t, Some(&mut draft)).await?;
let test = Message::load_from_db(&t, draft.id).await?;
@@ -4964,29 +4955,25 @@ mod tests {
let one2one_msg = Message::load_from_db(&alice, one2one_msg_id).await?;
// quoting messages in same chat is okay
let mut msg = Message::new(Viewtype::Text);
msg.set_text("baz".to_string());
let mut msg = Message::new_text("baz".to_string());
msg.set_quote(&alice, Some(&grp_msg)).await?;
let result = send_msg(&alice, grp_chat_id, &mut msg).await;
assert!(result.is_ok());
let mut msg = Message::new(Viewtype::Text);
msg.set_text("baz".to_string());
let mut msg = Message::new_text("baz".to_string());
msg.set_quote(&alice, Some(&one2one_msg)).await?;
let result = send_msg(&alice, one2one_chat_id, &mut msg).await;
assert!(result.is_ok());
let one2one_quote_reply_msg_id = result.unwrap();
// quoting messages from groups to one-to-ones is okay ("reply privately")
let mut msg = Message::new(Viewtype::Text);
msg.set_text("baz".to_string());
let mut msg = Message::new_text("baz".to_string());
msg.set_quote(&alice, Some(&grp_msg)).await?;
let result = send_msg(&alice, one2one_chat_id, &mut msg).await;
assert!(result.is_ok());
// quoting messages from one-to-one chats in groups is an error; usually this is also not allowed by UI at all ...
let mut msg = Message::new(Viewtype::Text);
msg.set_text("baz".to_string());
let mut msg = Message::new_text("baz".to_string());
msg.set_quote(&alice, Some(&one2one_msg)).await?;
let result = send_msg(&alice, grp_chat_id, &mut msg).await;
assert!(result.is_err());
@@ -5478,13 +5465,11 @@ mod tests {
let t = TestContext::new().await;
// add two device-messages
let mut msg1 = Message::new(Viewtype::Text);
msg1.set_text("first message".to_string());
let mut msg1 = Message::new_text("first message".to_string());
let msg1_id = add_device_msg(&t, None, Some(&mut msg1)).await;
assert!(msg1_id.is_ok());
let mut msg2 = Message::new(Viewtype::Text);
msg2.set_text("second message".to_string());
let mut msg2 = Message::new_text("second message".to_string());
let msg2_id = add_device_msg(&t, None, Some(&mut msg2)).await;
assert!(msg2_id.is_ok());
assert_ne!(msg1_id.as_ref().unwrap(), msg2_id.as_ref().unwrap());
@@ -5513,14 +5498,12 @@ mod tests {
let t = TestContext::new().await;
// add two device-messages with the same label (second attempt is not added)
let mut msg1 = Message::new(Viewtype::Text);
msg1.text = "first message".to_string();
let mut msg1 = Message::new_text("first message".to_string());
let msg1_id = add_device_msg(&t, Some("any-label"), Some(&mut msg1)).await;
assert!(msg1_id.is_ok());
assert!(!msg1_id.as_ref().unwrap().is_unset());
let mut msg2 = Message::new(Viewtype::Text);
msg2.text = "second message".to_string();
let mut msg2 = Message::new_text("second message".to_string());
let msg2_id = add_device_msg(&t, Some("any-label"), Some(&mut msg2)).await;
assert!(msg2_id.is_ok());
assert!(msg2_id.as_ref().unwrap().is_unset());
@@ -5567,8 +5550,7 @@ mod tests {
let res = add_device_msg(&t, Some("some-label"), None).await;
assert!(res.is_ok());
let mut msg = Message::new(Viewtype::Text);
msg.set_text("message text".to_string());
let mut msg = Message::new_text("message text".to_string());
let msg_id = add_device_msg(&t, Some("some-label"), Some(&mut msg)).await;
assert!(msg_id.is_ok());
@@ -5585,8 +5567,7 @@ mod tests {
add_device_msg(&t, Some("some-label"), None).await.ok();
assert!(was_device_msg_ever_added(&t, "some-label").await.unwrap());
let mut msg = Message::new(Viewtype::Text);
msg.set_text("message text".to_string());
let mut msg = Message::new_text("message text".to_string());
add_device_msg(&t, Some("another-label"), Some(&mut msg))
.await
.ok();
@@ -5603,8 +5584,7 @@ mod tests {
async fn test_delete_device_chat() {
let t = TestContext::new().await;
let mut msg = Message::new(Viewtype::Text);
msg.set_text("message text".to_string());
let mut msg = Message::new_text("message text".to_string());
add_device_msg(&t, Some("some-label"), Some(&mut msg))
.await
.ok();
@@ -5627,8 +5607,7 @@ mod tests {
.await
.unwrap();
let mut msg = Message::new(Viewtype::Text);
msg.set_text("message text".to_string());
let mut msg = Message::new_text("message text".to_string());
assert!(send_msg(&t, device_chat_id, &mut msg).await.is_err());
assert!(prepare_msg(&t, device_chat_id, &mut msg).await.is_err());
@@ -5639,8 +5618,7 @@ mod tests {
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_delete_and_reset_all_device_msgs() {
let t = TestContext::new().await;
let mut msg = Message::new(Viewtype::Text);
msg.set_text("message text".to_string());
let mut msg = Message::new_text("message text".to_string());
let msg_id1 = add_device_msg(&t, Some("some-label"), Some(&mut msg))
.await
.unwrap();
@@ -5672,8 +5650,7 @@ mod tests {
async fn test_archive() {
// create two chats
let t = TestContext::new().await;
let mut msg = Message::new(Viewtype::Text);
msg.set_text("foo".to_string());
let mut msg = Message::new_text("foo".to_string());
let msg_id = add_device_msg(&t, None, Some(&mut msg)).await.unwrap();
let chat_id1 = message::Message::load_from_db(&t, msg_id)
.await
+4
View File
@@ -321,6 +321,10 @@ pub enum Config {
/// True if account is muted.
IsMuted,
/// Optional tag as "Work", "Family".
/// Meant to help profile owner to differ between profiles with similar names.
PrivateTag,
/// All secondary self addresses separated by spaces
/// (`addr1@example.org addr2@example.org addr3@example.org`)
SecondaryAddrs,
+4 -8
View File
@@ -111,15 +111,10 @@ impl Context {
let param = EnteredLoginParam::load(self).await?;
let old_addr = self.get_config(Config::ConfiguredAddr).await?;
let configured_param_res = configure(self, &param).await;
self.set_config_internal(Config::NotifyAboutWrongPw, None)
.await?;
on_configure_completed(self, configured_param_res?, old_addr).await?;
let configured_param = configure(self, &param).await?;
self.set_config_internal(Config::NotifyAboutWrongPw, Some("1"))
.await?;
on_configure_completed(self, configured_param, old_addr).await?;
Ok(())
}
}
@@ -417,7 +412,8 @@ async fn configure(ctx: &Context, param: &EnteredLoginParam) -> Result<Configure
configured_param.oauth2,
r,
);
let mut imap_session = match imap.connect(ctx).await {
let configuring = true;
let mut imap_session = match imap.connect(ctx, configuring).await {
Ok(session) => session,
Err(err) => bail!("{}", nicer_configuration_error(ctx, err.to_string()).await),
};
+4
View File
@@ -4,12 +4,16 @@
use deltachat_derive::{FromSql, ToSql};
use once_cell::sync::Lazy;
use percent_encoding::{AsciiSet, NON_ALPHANUMERIC};
use serde::{Deserialize, Serialize};
use crate::chat::ChatId;
pub static DC_VERSION_STR: Lazy<String> = Lazy::new(|| env!("CARGO_PKG_VERSION").to_string());
/// Set of characters to percent-encode in email addresses and names.
pub(crate) const NON_ALPHANUMERIC_WITHOUT_DOT: &AsciiSet = &NON_ALPHANUMERIC.remove(b'.');
#[derive(
Debug,
Default,
+6
View File
@@ -857,6 +857,12 @@ impl Context {
"is_muted",
self.get_config_bool(Config::IsMuted).await?.to_string(),
);
res.insert(
"private_tag",
self.get_config(Config::PrivateTag)
.await?
.unwrap_or_else(|| "<unset>".to_string()),
);
if let Some(metadata) = &*self.metadata.read().await {
if let Some(comment) = &metadata.comment {
+13
View File
@@ -8,6 +8,7 @@ use crate::config::Config;
use crate::contact::ContactId;
use crate::ephemeral::Timer as EphemeralTimer;
use crate::message::MsgId;
use crate::reaction::Reaction;
use crate::webxdc::StatusUpdateSerial;
/// Event payload.
@@ -94,6 +95,18 @@ pub enum EventType {
contact_id: ContactId,
},
/// Reactions for the message changed.
IncomingReaction {
/// ID of the contact whose reaction set is changed.
contact_id: ContactId,
/// ID of the message for which reactions were changed.
msg_id: MsgId,
/// The reaction.
reaction: Reaction,
},
/// There is a fresh message. Typically, the user will show an notification
/// when receiving this message.
///
+16 -12
View File
@@ -290,7 +290,11 @@ impl Imap {
/// Calling this function is not enough to perform IMAP operations. Use [`Imap::prepare`]
/// instead if you are going to actually use connection rather than trying connection
/// parameters.
pub(crate) async fn connect(&mut self, context: &Context) -> Result<Session> {
pub(crate) async fn connect(
&mut self,
context: &Context,
configuring: bool,
) -> Result<Session> {
let now = tools::Time::now();
let until_can_send = max(
min(self.conn_last_try, now)
@@ -416,19 +420,12 @@ impl Imap {
warn!(context, "IMAP failed to login: {err:#}.");
first_error.get_or_insert(format_err!("{message} ({err:#})"));
let lock = context.wrong_pw_warning_mutex.lock().await;
if self.login_failed_once
let _lock = context.wrong_pw_warning_mutex.lock().await;
if !configuring
&& self.login_failed_once
&& err_str.to_lowercase().contains("authentication")
&& context.get_config_bool(Config::NotifyAboutWrongPw).await?
{
if let Err(e) = context
.set_config_internal(Config::NotifyAboutWrongPw, None)
.await
{
warn!(context, "{e:#}.");
}
drop(lock);
let mut msg = Message::new(Viewtype::Text);
msg.text.clone_from(&message);
if let Err(e) = chat::add_device_msg_with_importance(
@@ -440,6 +437,12 @@ impl Imap {
.await
{
warn!(context, "Failed to add device message: {e:#}.");
} else {
context
.set_config_internal(Config::NotifyAboutWrongPw, None)
.await
.log_err(context)
.ok();
}
} else {
self.login_failed_once = true;
@@ -456,7 +459,8 @@ impl Imap {
/// Ensure that IMAP client is connected, folders are created and IMAP capabilities are
/// determined.
pub(crate) async fn prepare(&mut self, context: &Context) -> Result<Session> {
let mut session = match self.connect(context).await {
let configuring = false;
let mut session = match self.connect(context, configuring).await {
Ok(session) => session,
Err(err) => {
self.connectivity.set_err(context, &err).await;
+1
View File
@@ -180,6 +180,7 @@ impl Client {
Client::connect_insecure_proxy(context, host, port, proxy_config).await?
}
};
update_connection_history(context, "imap", host, port, host, time()).await?;
Ok(client)
} else {
let load_cache = match security {
+9
View File
@@ -491,6 +491,15 @@ impl Message {
}
}
/// Creates a new message with Viewtype::Text.
pub fn new_text(text: String) -> Self {
Message {
viewtype: Viewtype::Text,
text,
..Default::default()
}
}
/// Loads message with given ID from the database.
///
/// Returns an error if the message does not exist.
+2 -3
View File
@@ -20,6 +20,7 @@ use crate::e2ee::EncryptHelper;
use crate::ephemeral::Timer as EphemeralTimer;
use crate::headerdef::HeaderDef;
use crate::html::new_html_mimepart;
use crate::location;
use crate::message::{self, Message, MsgId, Viewtype};
use crate::mimeparser::SystemMessage;
use crate::param::Param;
@@ -32,7 +33,6 @@ use crate::tools::{
create_outgoing_rfc724_mid, create_smeared_timestamp, remove_subject_prefix, time,
};
use crate::webxdc::StatusUpdateSerial;
use crate::{location, peer_channels};
// attachments of 25 mb brutto should work on the majority of providers
// (brutto examples: web.de=50, 1&1=40, t-online.de=32, gmail=25, posteo=50, yahoo=25, all-inkl=100).
@@ -1387,8 +1387,7 @@ impl MimeFactory {
let json = msg.param.get(Param::Arg).unwrap_or_default();
parts.push(context.build_status_update_part(json));
} else if msg.viewtype == Viewtype::Webxdc {
let topic = peer_channels::create_random_topic();
headers.push(create_iroh_header(context, topic, msg.id).await?);
headers.push(create_iroh_header(context, msg.id).await?);
if let (Some(json), _) = context
.render_webxdc_status_update_object(
msg.id,
+16 -3
View File
@@ -3598,11 +3598,25 @@ On 2020-10-25, Bob wrote:
assert!(mimemsg.parts[0].msg.len() <= DC_DESIRED_TEXT_LEN + DC_ELLIPSIS.len());
}
{
for draft in [false, true] {
let chat = t.get_self_chat().await;
t.send_text(chat.id, &long_txt).await;
let mut msg = Message::new(Viewtype::Text);
msg.set_text(long_txt.clone());
if draft {
chat.id.set_draft(&t, Some(&mut msg)).await?;
}
t.send_msg(chat.id, &mut msg).await;
let msg = t.get_last_msg_in(chat.id).await;
assert!(msg.has_html());
assert_eq!(
msg.id
.get_html(&t)
.await?
.unwrap()
.matches("just repeated")
.count(),
REPEAT_CNT
);
assert!(
msg.text.matches("just repeated").count() <= DC_DESIRED_TEXT_LEN / REPEAT_TXT.len()
);
@@ -3610,7 +3624,6 @@ On 2020-10-25, Bob wrote:
}
t.set_config(Config::Bot, Some("1")).await?;
{
let mimemsg = MimeMessage::from_bytes(&t, long_txt.as_ref(), None).await?;
assert!(!mimemsg.is_mime_modified);
+8
View File
@@ -47,6 +47,14 @@ pub(crate) async fn prune_connection_history(context: &Context) -> Result<()> {
Ok(())
}
/// Update the timestamp of the last successfull connection
/// to the given `host` and `port`
/// with the given application protocol `alpn`.
///
/// `addr` is the string representation of IP address.
/// If connection is made over a proxy which does
/// its own DNS resolution,
/// `addr` should be the same as `host`.
pub(crate) async fn update_connection_history(
context: &Context,
alpn: &str,
+127 -10
View File
@@ -1,4 +1,44 @@
//! DNS resolution and cache.
//!
//! DNS cache in Delta Chat has two layers:
//! in-memory cache and persistent `dns_cache` SQL table.
//!
//! In-memory cache is using a "stale-while-revalidate" strategy.
//! If there is a cached value, it is returned immediately
//! and revalidation task is started in the background
//! to replace old cached IP addresses with new ones.
//! If there is no cached value yet,
//! lookup only finishes when `lookup_host` returns first results.
//! In-memory cache is shared between all accounts
//! and is never stored on the disk.
//! It can be thought of as an extension
//! of the system resolver.
//!
//! Persistent `dns_cache` SQL table is used to collect
//! all IP addresses ever seen for the hostname
//! together with the timestamp
//! of the last time IP address has been seen.
//! Note that this timestamp reflects the time
//! IP address was returned by the in-memory cache
//! rather than the underlying system resolver.
//! Unused entries are removed after 30 days
//! (`CACHE_TTL` constant) to avoid having
//! old non-working IP addresses in the cache indefinitely.
//!
//! When Delta Chat needs an IP address for the host,
//! it queries in-memory cache for the next result
//! and merges the list of IP addresses
//! with the list of IP addresses from persistent cache.
//! Resulting list is constructed
//! by taking the first two results from the resolver
//! followed up by persistent cache results
//! and terminated by the rest of resolver results.
//!
//! Persistent cache results are sorted
//! by the time of the most recent successful connection
//! using the result. For results that have never been
//! used for successful connection timestamp of
//! retrieving them from in-memory cache is used.
use anyhow::{Context as _, Result};
use std::collections::HashMap;
@@ -42,33 +82,110 @@ pub(crate) async fn prune_dns_cache(context: &Context) -> Result<()> {
Ok(())
}
/// Looks up the hostname and updates DNS cache
/// on success.
/// Map from hostname to IP addresses.
///
/// NOTE: sync RwLock is used, so it must not be held across `.await`
/// to avoid deadlocks.
/// See
/// <https://docs.rs/tokio/1.40.0/tokio/sync/struct.Mutex.html#which-kind-of-mutex-should-you-use>
/// and
/// <https://stackoverflow.com/questions/63712823/why-do-i-get-a-deadlock-when-using-tokio-with-a-stdsyncmutex>.
static LOOKUP_HOST_CACHE: Lazy<parking_lot::RwLock<HashMap<String, Vec<IpAddr>>>> =
Lazy::new(Default::default);
/// Wrapper for `lookup_host` that returns IP addresses.
async fn lookup_ips(host: impl tokio::net::ToSocketAddrs) -> Result<impl Iterator<Item = IpAddr>> {
Ok(lookup_host(host)
.await
.context("DNS lookup failure")?
.map(|addr| addr.ip()))
}
async fn lookup_host_with_memory_cache(
context: &Context,
hostname: &str,
port: u16,
) -> Result<Vec<IpAddr>> {
let stale_result = {
let rwlock_read_guard = LOOKUP_HOST_CACHE.read();
rwlock_read_guard.get(hostname).cloned()
};
if let Some(stale_result) = stale_result {
// Revalidate the cache in the background.
{
let context = context.clone();
let hostname = hostname.to_string();
tokio::spawn(async move {
match lookup_ips((hostname.clone(), port)).await {
Ok(res) => {
LOOKUP_HOST_CACHE.write().insert(hostname, res.collect());
}
Err(err) => {
warn!(
context,
"Failed to revalidate results for {hostname:?}: {err:#}."
);
}
}
});
}
info!(
context,
"Using memory-cached DNS resolution for {hostname}."
);
Ok(stale_result)
} else {
info!(
context,
"No memory-cached DNS resolution for {hostname} available, waiting for the resolver."
);
let res: Vec<IpAddr> = lookup_ips((hostname, port)).await?.collect();
// Insert initial result into the cache.
//
// There may already be a result from a parallel
// task stored, overwriting it is not a problem.
LOOKUP_HOST_CACHE
.write()
.insert(hostname.to_string(), res.clone());
Ok(res)
}
}
/// Looks up the hostname and updates
/// persistent DNS cache on success.
async fn lookup_host_and_update_cache(
context: &Context,
hostname: &str,
port: u16,
now: i64,
) -> Result<Vec<SocketAddr>> {
let res: Vec<SocketAddr> = timeout(super::TIMEOUT, lookup_host((hostname, port)))
.await
.context("DNS lookup timeout")?
.context("DNS lookup failure")?
.collect();
let res: Vec<IpAddr> = timeout(
super::TIMEOUT,
lookup_host_with_memory_cache(context, hostname, port),
)
.await
.context("DNS lookup timeout")?
.context("DNS lookup with memory cache failure")?;
for addr in &res {
let ip_string = addr.ip().to_string();
for ip in &res {
let ip_string = ip.to_string();
if ip_string == hostname {
// IP address resolved into itself, not interesting to cache.
continue;
}
info!(context, "Resolved {hostname}:{port} into {addr}.");
info!(context, "Resolved {hostname} into {ip}.");
// Update the cache.
update_cache(context, hostname, &ip_string, now).await?;
}
let res = res
.into_iter()
.map(|ip| SocketAddr::new(ip, port))
.collect();
Ok(res)
}
+44 -2
View File
@@ -12,13 +12,14 @@ use fast_socks5::client::Socks5Stream;
use fast_socks5::util::target_addr::ToTargetAddr;
use fast_socks5::AuthenticationMethod;
use fast_socks5::Socks5Command;
use percent_encoding::{percent_encode, NON_ALPHANUMERIC};
use percent_encoding::{percent_encode, utf8_percent_encode, NON_ALPHANUMERIC};
use tokio::io::{AsyncReadExt, AsyncWriteExt};
use tokio::net::TcpStream;
use tokio_io_timeout::TimeoutStream;
use url::Url;
use crate::config::Config;
use crate::constants::NON_ALPHANUMERIC_WITHOUT_DOT;
use crate::context::Context;
use crate::net::connect_tcp;
use crate::net::session::SessionStream;
@@ -41,6 +42,12 @@ impl PartialEq for ShadowsocksConfig {
impl Eq for ShadowsocksConfig {}
impl ShadowsocksConfig {
fn to_url(&self) -> String {
self.server_config.to_url()
}
}
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct HttpConfig {
/// HTTP proxy host.
@@ -84,6 +91,17 @@ impl HttpConfig {
};
Ok(http_config)
}
fn to_url(&self, scheme: &str) -> String {
let host = utf8_percent_encode(&self.host, NON_ALPHANUMERIC_WITHOUT_DOT);
if let Some((user, password)) = &self.user_password {
let user = utf8_percent_encode(user, NON_ALPHANUMERIC);
let password = utf8_percent_encode(password, NON_ALPHANUMERIC);
format!("{scheme}://{user}:{password}@{host}:{}", self.port)
} else {
format!("{scheme}://{host}:{}", self.port)
}
}
}
#[derive(Debug, Clone, PartialEq, Eq)]
@@ -123,6 +141,17 @@ impl Socks5Config {
Ok(socks_stream)
}
fn to_url(&self) -> String {
let host = utf8_percent_encode(&self.host, NON_ALPHANUMERIC_WITHOUT_DOT);
if let Some((user, password)) = &self.user_password {
let user = utf8_percent_encode(user, NON_ALPHANUMERIC);
let password = utf8_percent_encode(password, NON_ALPHANUMERIC);
format!("socks5://{user}:{password}@{host}:{}", self.port)
} else {
format!("socks5://{host}:{}", self.port)
}
}
}
#[derive(Debug, Clone, PartialEq, Eq)]
@@ -217,7 +246,7 @@ where
impl ProxyConfig {
/// Creates a new proxy configuration by parsing given proxy URL.
fn from_url(url: &str) -> Result<Self> {
pub(crate) fn from_url(url: &str) -> Result<Self> {
let url = Url::parse(url).context("Cannot parse proxy URL")?;
match url.scheme() {
"http" => {
@@ -272,6 +301,19 @@ impl ProxyConfig {
}
}
/// Serializes proxy config into an URL.
///
/// This function can be used to normalize proxy URL
/// by parsing it and serializing back.
pub(crate) fn to_url(&self) -> String {
match self {
Self::Http(http_config) => http_config.to_url("http"),
Self::Https(http_config) => http_config.to_url("https"),
Self::Socks5(socks5_config) => socks5_config.to_url(),
Self::Shadowsocks(shadowsocks_config) => shadowsocks_config.to_url(),
}
}
/// Migrates legacy `socks5_host`, `socks5_port`, `socks5_user` and `socks5_password`
/// config into `proxy_url` if `proxy_url` is unset or empty.
///
+70 -12
View File
@@ -23,7 +23,7 @@
//! (scoped per WebXDC app instance/message-id). The other peers can then join the gossip with `joinRealtimeChannel().setListener()`
//! and `joinRealtimeChannel().send()` just like the other peers.
use anyhow::{anyhow, Context as _, Result};
use anyhow::{anyhow, bail, Context as _, Result};
use email::Header;
use futures_lite::StreamExt;
use iroh_gossip::net::{Event, Gossip, GossipEvent, JoinOptions, GOSSIP_ALPN};
@@ -143,9 +143,10 @@ impl Iroh {
self.endpoint.add_node_addr(peer.clone())?;
}
self.gossip
.join(topic, peers.into_iter().map(|peer| peer.node_id).collect())
.await?;
self.gossip.join_with_opts(
topic,
JoinOptions::with_bootstrap(peers.into_iter().map(|peer| peer.node_id)),
);
}
Ok(())
}
@@ -232,6 +233,7 @@ impl ChannelState {
impl Context {
/// Create iroh endpoint and gossip.
async fn init_peer_channels(&self) -> Result<Iroh> {
info!(self, "Initializing peer channels.");
let secret_key = SecretKey::generate();
let public_key = secret_key.public();
@@ -258,7 +260,14 @@ impl Context {
// create gossip
let my_addr = endpoint.node_addr().await?;
let gossip = Gossip::from_endpoint(endpoint.clone(), Default::default(), &my_addr.info);
let gossip_config = iroh_gossip::proto::topic::Config {
// Allow messages up to 128 KB in size.
// We set the limit to 128 KiB to account for internal overhead,
// but only guarantee 128 KB of payload to WebXDC developers.
max_message_size: 128 * 1024,
..Default::default()
};
let gossip = Gossip::from_endpoint(endpoint.clone(), gossip_config, &my_addr.info);
// spawn endpoint loop that forwards incoming connections to the gossiper
let context = self.clone();
@@ -277,6 +286,10 @@ impl Context {
/// Get or initialize the iroh peer channel.
pub async fn get_or_try_init_peer_channel(&self) -> Result<&Iroh> {
if !self.get_config_bool(Config::WebxdcRealtimeEnabled).await? {
bail!("Attempt to get Iroh when realtime is disabled");
}
let ctx = self.clone();
self.iroh
.get_or_try_init(|| async { ctx.init_peer_channels().await })
@@ -301,6 +314,47 @@ pub(crate) async fn iroh_add_peer_for_topic(
Ok(())
}
/// Add gossip peer from `Iroh-Node-Addr` header to WebXDC message identified by `instance_id`.
pub async fn add_gossip_peer_from_header(
context: &Context,
instance_id: MsgId,
node_addr: &str,
) -> Result<()> {
if !context
.get_config_bool(Config::WebxdcRealtimeEnabled)
.await?
{
return Ok(());
}
info!(
context,
"Adding iroh peer with address {node_addr:?} to the topic of {instance_id}."
);
let node_addr =
serde_json::from_str::<NodeAddr>(node_addr).context("Failed to parse node address")?;
context.emit_event(EventType::WebxdcRealtimeAdvertisementReceived {
msg_id: instance_id,
});
let Some(topic) = get_iroh_topic_for_msg(context, instance_id).await? else {
warn!(
context,
"Could not add iroh peer because {instance_id} has no topic."
);
return Ok(());
};
let node_id = node_addr.node_id;
let relay_server = node_addr.relay_url().map(|relay| relay.as_str());
iroh_add_peer_for_topic(context, instance_id, topic, node_id, relay_server).await?;
let iroh = context.get_or_try_init_peer_channel().await?;
iroh.maybe_add_gossip_peers(topic, vec![node_addr]).await?;
Ok(())
}
/// Insert topicId into the database so that we can use it to retrieve the topic.
pub(crate) async fn insert_topic_stub(ctx: &Context, msg_id: MsgId, topic: TopicId) -> Result<()> {
ctx.sql
@@ -414,15 +468,15 @@ pub async fn leave_webxdc_realtime(ctx: &Context, msg_id: MsgId) -> Result<()> {
Ok(())
}
pub(crate) fn create_random_topic() -> TopicId {
/// Creates a new random gossip topic.
fn create_random_topic() -> TopicId {
TopicId::from_bytes(rand::random())
}
pub(crate) async fn create_iroh_header(
ctx: &Context,
topic: TopicId,
msg_id: MsgId,
) -> Result<Header> {
/// Creates `Iroh-Gossip-Header` with a new random topic
/// and stores the topic for the message.
pub(crate) async fn create_iroh_header(ctx: &Context, msg_id: MsgId) -> Result<Header> {
let topic = create_random_topic();
insert_topic_stub(ctx, msg_id, topic).await?;
Ok(Header::new(
HeaderDef::IrohGossipTopic.get_headername().to_string(),
@@ -949,6 +1003,10 @@ mod tests {
// creates iroh endpoint as side effect
leave_webxdc_realtime(alice, MsgId::new(1)).await.unwrap();
assert!(alice.ctx.iroh.get().is_none())
assert!(alice.ctx.iroh.get().is_none());
// This internal function should return error
// if accidentally called with the setting disabled.
assert!(alice.ctx.get_or_try_init_peer_channel().await.is_err());
}
}
+1 -1
View File
@@ -31,7 +31,7 @@ impl PushSubscriber {
}
/// Sets device token for Apple Push Notification service.
pub(crate) async fn set_device_token(&mut self, token: &str) {
pub(crate) async fn set_device_token(&self, token: &str) {
self.inner.write().await.device_token = Some(token.to_string());
}
+16 -1
View File
@@ -20,7 +20,7 @@ use crate::events::EventType;
use crate::key::Fingerprint;
use crate::message::Message;
use crate::net::http::post_empty;
use crate::net::proxy::DEFAULT_SOCKS_PORT;
use crate::net::proxy::{ProxyConfig, DEFAULT_SOCKS_PORT};
use crate::peerstate::Peerstate;
use crate::token;
use crate::tools::validate_id;
@@ -723,6 +723,10 @@ pub async fn set_config_from_qr(context: &Context, qr: &str) -> Result<()> {
.get_config(Config::ProxyUrl)
.await?
.unwrap_or_default();
// Normalize the URL.
let url = ProxyConfig::from_url(&url)?.to_url();
let proxy_urls: Vec<&str> = std::iter::once(url.as_str())
.chain(
old_proxy_url_value
@@ -1787,6 +1791,17 @@ mod tests {
)
);
// SOCKS5 config does not have port 1080 explicitly specified,
// but should bring `socks5://1.2.3.4:1080` to the top instead of creating another entry.
set_config_from_qr(&t, "socks5://1.2.3.4").await?;
assert_eq!(
t.get_config(Config::ProxyUrl).await?,
Some(
"socks5://1.2.3.4:1080\nss://YWVzLTEyOC1nY206dGVzdA@192.168.100.1:8888#Example1\nsocks5://foo:666\nsocks5://Da:x%26%25%24X@jau:1080"
.to_string()
)
);
Ok(())
}
+79
View File
@@ -14,6 +14,77 @@ use crate::qr::{self, Qr};
use crate::securejoin;
use crate::stock_str::{self, backup_transfer_qr};
/// Create a QR code from any input data.
pub fn create_qr_svg(qrcode_content: &str) -> Result<String> {
let all_size = 512.0;
let qr_code_size = 416.0;
let logo_size = 96.0;
let qr = QrCode::encode_text(qrcode_content, QrCodeEcc::Medium)?;
let mut svg = String::with_capacity(28000);
let mut w = tagger::new(&mut svg);
w.elem("svg", |d| {
d.attr("xmlns", "http://www.w3.org/2000/svg")?;
d.attr("viewBox", format_args!("0 0 {all_size} {all_size}"))?;
d.attr("xmlns:xlink", "http://www.w3.org/1999/xlink")?; // required for enabling xlink:href on browsers
Ok(())
})?
.build(|w| {
// background
w.single("rect", |d| {
d.attr("x", 0)?;
d.attr("y", 0)?;
d.attr("width", all_size)?;
d.attr("height", all_size)?;
d.attr("style", "fill:#ffffff")?;
Ok(())
})?;
// QR code
w.elem("g", |d| {
d.attr(
"transform",
format!(
"translate({},{})",
(all_size - qr_code_size) / 2.0,
((all_size - qr_code_size) / 2.0)
),
)
})?
.build(|w| {
w.single("path", |d| {
let mut path_data = String::with_capacity(0);
let scale = qr_code_size / qr.size() as f32;
for y in 0..qr.size() {
for x in 0..qr.size() {
if qr.get_module(x, y) {
path_data += &format!("M{x},{y}h1v1h-1z");
}
}
}
d.attr("style", "fill:#000000")?;
d.attr("d", path_data)?;
d.attr("transform", format!("scale({scale})"))
})
})?;
w.elem("g", |d| {
d.attr(
"transform",
format!(
"translate({},{}) scale(2)", // data in qr_overlay_delta.svg-part are 48 x 48, scaling by 2 results in desired logo_size of 96
(all_size - logo_size) / 2.0,
(all_size - logo_size) / 2.0
),
)
})?
.build(|w| w.put_raw_escapable(include_str!("../assets/qr_overlay_delta.svg-part")))
})?;
Ok(svg)
}
/// Returns SVG of the QR code to join the group or verify contact.
///
/// If `chat_id` is `None`, returns verification QR code.
@@ -304,6 +375,14 @@ mod tests {
use super::*;
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_create_qr_svg() -> Result<()> {
let svg = create_qr_svg("this is a test QR code \" < > &")?;
assert!(svg.contains("<svg"));
assert!(svg.contains("</svg>"));
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_svg_escaping() {
let svg = inner_generate_secure_join_qr_code(
+54 -5
View File
@@ -19,6 +19,7 @@ use std::collections::BTreeMap;
use std::fmt;
use anyhow::Result;
use serde::{Deserialize, Serialize};
use crate::chat::{send_msg, Chat, ChatId};
use crate::chatlist_events;
@@ -31,7 +32,7 @@ use crate::param::Param;
/// A single reaction consisting of multiple emoji sequences.
///
/// It is guaranteed to have all emojis sorted and deduplicated inside.
#[derive(Debug, Default, Clone)]
#[derive(Debug, Default, Clone, Deserialize, Eq, PartialEq, Serialize)]
pub struct Reaction {
/// Canonical representation of reaction as a string of space-separated emojis.
reaction: String,
@@ -173,7 +174,7 @@ async fn set_msg_id_reaction(
chat_id: ChatId,
contact_id: ContactId,
timestamp: i64,
reaction: Reaction,
reaction: &Reaction,
) -> Result<()> {
if reaction.is_empty() {
// Simply remove the record instead of setting it to empty string.
@@ -244,7 +245,7 @@ pub async fn send_reaction(context: &Context, msg_id: MsgId, reaction: &str) ->
msg.chat_id,
ContactId::SELF,
reaction_msg.timestamp_sort,
reaction,
&reaction,
)
.await?;
Ok(reaction_msg_id)
@@ -275,16 +276,28 @@ pub(crate) async fn set_msg_reaction(
contact_id: ContactId,
timestamp: i64,
reaction: Reaction,
is_incoming_fresh: bool,
) -> Result<()> {
if let Some((msg_id, _)) = rfc724_mid_exists(context, in_reply_to).await? {
set_msg_id_reaction(context, msg_id, chat_id, contact_id, timestamp, reaction).await
set_msg_id_reaction(context, msg_id, chat_id, contact_id, timestamp, &reaction).await?;
if is_incoming_fresh
&& !reaction.is_empty()
&& msg_id.get_state(context).await?.is_outgoing()
{
context.emit_event(EventType::IncomingReaction {
contact_id,
msg_id,
reaction,
});
}
} else {
info!(
context,
"Can't assign reaction to unknown message with Message-ID {}", in_reply_to
);
Ok(())
}
Ok(())
}
/// Get our own reaction for a given message.
@@ -563,6 +576,38 @@ Here's my footer -- bob@example.net"
Ok(())
}
async fn expect_incoming_reactions_event(
t: &TestContext,
expected_msg_id: MsgId,
expected_contact_id: ContactId,
expected_reaction: &str,
) -> Result<()> {
let event = t
.evtracker
.get_matching(|evt| matches!(evt, EventType::IncomingReaction { .. }))
.await;
match event {
EventType::IncomingReaction {
msg_id,
contact_id,
reaction,
} => {
assert_eq!(msg_id, expected_msg_id);
assert_eq!(contact_id, expected_contact_id);
assert_eq!(reaction, Reaction::from(expected_reaction));
}
_ => unreachable!(),
}
Ok(())
}
async fn has_incoming_reactions_event(t: &TestContext) -> bool {
t.evtracker
.get_matching_opt(t, |evt| matches!(evt, EventType::IncomingReaction { .. }))
.await
.is_some()
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_send_reaction() -> Result<()> {
let alice = TestContext::new_alice().await;
@@ -593,6 +638,7 @@ Here's my footer -- bob@example.net"
send_reaction(&bob, bob_msg.id, "👍").await.unwrap();
expect_reactions_changed_event(&bob, bob_msg.chat_id, bob_msg.id, ContactId::SELF).await?;
assert!(!has_incoming_reactions_event(&bob).await);
assert_eq!(get_chat_msgs(&bob, bob_msg.chat_id).await?.len(), 2);
let bob_reaction_msg = bob.pop_sent_msg().await;
@@ -610,6 +656,7 @@ Here's my footer -- bob@example.net"
assert_eq!(bob_reaction.as_str(), "👍");
expect_reactions_changed_event(&alice, chat_alice.id, alice_msg.sender_msg_id, *bob_id)
.await?;
expect_incoming_reactions_event(&alice, alice_msg.sender_msg_id, *bob_id, "👍").await?;
// Alice reacts to own message.
send_reaction(&alice, alice_msg.sender_msg_id, "👍 😀")
@@ -650,6 +697,7 @@ Here's my footer -- bob@example.net"
send_reaction(&bob, bob_msg1.id, "👍").await?;
let bob_send_reaction = bob.pop_sent_msg().await;
alice.recv_msg_trash(&bob_send_reaction).await;
assert!(has_incoming_reactions_event(&alice).await);
let chatlist = Chatlist::try_load(&bob, 0, None, None).await?;
let summary = chatlist.get_summary(&bob, 0, None).await?;
@@ -665,6 +713,7 @@ Here's my footer -- bob@example.net"
send_reaction(&alice, alice_msg1.sender_msg_id, "🍿").await?;
let alice_send_reaction = alice.pop_sent_msg().await;
bob.recv_msg_opt(&alice_send_reaction).await;
assert!(!has_incoming_reactions_event(&bob).await);
assert_summary(&alice, "You reacted 🍿 to \"Party?\"").await;
assert_summary(&bob, "ALICE reacted 🍿 to \"Party?\"").await;
+45 -34
View File
@@ -30,7 +30,7 @@ use crate::message::{
};
use crate::mimeparser::{parse_message_ids, AvatarAction, MimeMessage, SystemMessage};
use crate::param::{Param, Params};
use crate::peer_channels::{get_iroh_topic_for_msg, insert_topic_stub, iroh_add_peer_for_topic};
use crate::peer_channels::{add_gossip_peer_from_header, insert_topic_stub};
use crate::peerstate::Peerstate;
use crate::reaction::{set_msg_reaction, Reaction};
use crate::securejoin::{self, handle_securejoin_handshake, observe_securejoin_on_other_device};
@@ -41,7 +41,6 @@ use crate::sync::Sync::*;
use crate::tools::{self, buf_compress, remove_subject_prefix};
use crate::{chatlist_events, location};
use crate::{contact, imap};
use iroh_net::NodeAddr;
/// This is the struct that is returned after receiving one email (aka MIME message).
///
@@ -762,6 +761,7 @@ async fn add_parts(
let state: MessageState;
let mut hidden = false;
let mut needs_delete_job = false;
let mut restore_protection = false;
// if contact renaming is prevented (for mailinglists and bots),
// we use name from From:-header as override name
@@ -930,15 +930,11 @@ async fn add_parts(
if chat_id.is_none() {
// try to create a normal chat
let create_blocked = if from_id == ContactId::SELF {
Blocked::Not
} else {
let contact = Contact::get_by_id(context, from_id).await?;
match contact.is_blocked() {
true => Blocked::Yes,
false if is_bot => Blocked::Not,
false => Blocked::Request,
}
let contact = Contact::get_by_id(context, from_id).await?;
let create_blocked = match contact.is_blocked() {
true => Blocked::Yes,
false if is_bot => Blocked::Not,
false => Blocked::Request,
};
if let Some(chat) = test_normal_chat {
@@ -1010,6 +1006,13 @@ async fn add_parts(
)
.await?;
}
if let Some(peerstate) = &mime_parser.decryption_info.peerstate {
restore_protection = new_protection != ProtectionStatus::Protected
&& peerstate.prefer_encrypt == EncryptPreference::Mutual
// Check that the contact still has the Autocrypt key same as the
// verified key, see also `Peerstate::is_using_verified_key()`.
&& contact.is_verified(context).await?;
}
}
}
}
@@ -1032,8 +1035,7 @@ async fn add_parts(
state = MessageState::OutDelivered;
to_id = to_ids.first().copied().unwrap_or_default();
let self_sent =
from_id == ContactId::SELF && to_ids.len() == 1 && to_ids.contains(&ContactId::SELF);
let self_sent = to_ids.len() == 1 && to_ids.contains(&ContactId::SELF);
if mime_parser.sync_items.is_some() && self_sent {
chat_id = Some(DC_CHAT_ID_TRASH);
@@ -1442,30 +1444,28 @@ async fn add_parts(
}
if let Some(node_addr) = mime_parser.get_header(HeaderDef::IrohNodeAddr) {
match serde_json::from_str::<NodeAddr>(node_addr).context("Failed to parse node address") {
Ok(node_addr) => {
info!(context, "Adding iroh peer with address {node_addr:?}.");
let instance_id = parent.context("Failed to get parent message")?.id;
context.emit_event(EventType::WebxdcRealtimeAdvertisementReceived {
msg_id: instance_id,
});
if let Some(topic) = get_iroh_topic_for_msg(context, instance_id).await? {
let node_id = node_addr.node_id;
let relay_server = node_addr.relay_url().map(|relay| relay.as_str());
iroh_add_peer_for_topic(context, instance_id, topic, node_id, relay_server)
.await?;
let iroh = context.get_or_try_init_peer_channel().await?;
iroh.maybe_add_gossip_peers(topic, vec![node_addr]).await?;
} else {
chat_id = DC_CHAT_ID_TRASH;
match mime_parser.get_header(HeaderDef::InReplyTo) {
Some(in_reply_to) => match rfc724_mid_exists(context, in_reply_to).await? {
Some((instance_id, _ts_sent)) => {
if let Err(err) =
add_gossip_peer_from_header(context, instance_id, node_addr).await
{
warn!(context, "Failed to add iroh peer from header: {err:#}.");
}
}
None => {
warn!(
context,
"Could not add iroh peer because {instance_id} has no topic"
"Cannot add iroh peer because WebXDC instance does not exist."
);
}
chat_id = DC_CHAT_ID_TRASH;
}
Err(err) => {
warn!(context, "Couldn't parse NodeAddr: {err:#}.");
},
None => {
warn!(
context,
"Cannot add iroh peer because the message has no In-Reply-To."
);
}
}
}
@@ -1473,6 +1473,7 @@ async fn add_parts(
for part in &mime_parser.parts {
if part.is_reaction {
let reaction_str = simplify::remove_footers(part.msg.as_str());
let is_incoming_fresh = mime_parser.incoming && !seen && !fetching_existing_messages;
set_msg_reaction(
context,
mime_in_reply_to,
@@ -1480,6 +1481,7 @@ async fn add_parts(
from_id,
sort_timestamp,
Reaction::from(reaction_str.as_str()),
is_incoming_fresh,
)
.await?;
}
@@ -1712,7 +1714,16 @@ RETURNING id
// delete it.
needs_delete_job = true;
}
if restore_protection {
chat_id
.set_protection(
context,
ProtectionStatus::Protected,
mime_parser.timestamp_rcvd,
Some(from_id),
)
.await?;
}
Ok(ReceivedMsg {
chat_id,
state,
+19 -39
View File
@@ -1,13 +1,13 @@
//! Implementation of [SecureJoin protocols](https://securejoin.delta.chat/).
use anyhow::{bail, ensure, Context as _, Error, Result};
use percent_encoding::{utf8_percent_encode, AsciiSet, NON_ALPHANUMERIC};
use anyhow::{ensure, Context as _, Error, Result};
use percent_encoding::{utf8_percent_encode, NON_ALPHANUMERIC};
use crate::aheader::EncryptPreference;
use crate::chat::{self, get_chat_id_by_grpid, Chat, ChatId, ChatIdBlocked, ProtectionStatus};
use crate::chatlist_events;
use crate::config::Config;
use crate::constants::{Blocked, Chattype};
use crate::constants::{Blocked, Chattype, NON_ALPHANUMERIC_WITHOUT_DOT};
use crate::contact::{Contact, ContactId, Origin};
use crate::context::Context;
use crate::e2ee::ensure_secret_key_exists;
@@ -34,9 +34,6 @@ use qrinvite::QrInvite;
use crate::token::Namespace;
/// Set of characters to percent-encode in email addresses and names.
pub const NON_ALPHANUMERIC_WITHOUT_DOT: &AsciiSet = &NON_ALPHANUMERIC.remove(b'.');
fn inviter_progress(context: &Context, contact_id: ContactId, progress: usize) {
debug_assert!(
progress <= 1000,
@@ -89,12 +86,7 @@ pub async fn get_securejoin_qr(context: &Context, group: Option<ChatId>) -> Resu
.await?
.unwrap_or_default();
let fingerprint: Fingerprint = match get_self_fingerprint(context).await {
Some(fp) => fp,
None => {
bail!("No fingerprint, cannot generate QR code.");
}
};
let fingerprint = get_self_fingerprint(context).await?;
let self_addr_urlencoded =
utf8_percent_encode(&self_addr, NON_ALPHANUMERIC_WITHOUT_DOT).to_string();
@@ -140,14 +132,11 @@ pub async fn get_securejoin_qr(context: &Context, group: Option<ChatId>) -> Resu
Ok(qr)
}
async fn get_self_fingerprint(context: &Context) -> Option<Fingerprint> {
match load_self_public_key(context).await {
Ok(key) => Some(key.fingerprint()),
Err(_) => {
warn!(context, "get_self_fingerprint(): failed to load key");
None
}
}
async fn get_self_fingerprint(context: &Context) -> Result<Fingerprint> {
let key = load_self_public_key(context)
.await
.context("Failed to load key")?;
Ok(key.fingerprint())
}
/// Take a scanned QR-code and do the setup-contact/join-group/invite handshake.
@@ -388,7 +377,7 @@ pub(crate) async fn handle_securejoin_handshake(
return Ok(HandshakeMessage::Ignore);
};
let fingerprint: Fingerprint = fp.parse()?;
if !encrypted_and_signed(context, mime_message, Some(&fingerprint)) {
if !encrypted_and_signed(context, mime_message, &fingerprint) {
warn!(
context,
"Ignoring {step} message because the message is not encrypted."
@@ -577,11 +566,7 @@ pub(crate) async fn observe_securejoin_on_other_device(
return Ok(HandshakeMessage::Ignore);
};
if !encrypted_and_signed(
context,
mime_message,
get_self_fingerprint(context).await.as_ref(),
) {
if !encrypted_and_signed(context, mime_message, &get_self_fingerprint(context).await?) {
could_not_establish_secure_connection(
context,
contact_id,
@@ -740,24 +725,19 @@ async fn mark_peer_as_verified(
fn encrypted_and_signed(
context: &Context,
mimeparser: &MimeMessage,
expected_fingerprint: Option<&Fingerprint>,
expected_fingerprint: &Fingerprint,
) -> bool {
if !mimeparser.was_encrypted() {
warn!(context, "Message not encrypted.",);
false
} else if let Some(expected_fingerprint) = expected_fingerprint {
if !mimeparser.signatures.contains(expected_fingerprint) {
warn!(
context,
"Message does not match expected fingerprint {}.", expected_fingerprint,
);
false
} else {
true
}
} else {
warn!(context, "Fingerprint for comparison missing.");
} else if !mimeparser.signatures.contains(expected_fingerprint) {
warn!(
context,
"Message does not match expected fingerprint {}.", expected_fingerprint,
);
false
} else {
true
}
}
+1 -1
View File
@@ -270,7 +270,7 @@ impl BobState {
context,
"Bob Step 4 - handling {{vc,vg}}-auth-required message."
);
if !encrypted_and_signed(context, mime_message, Some(self.invite.fingerprint())) {
if !encrypted_and_signed(context, mime_message, self.invite.fingerprint()) {
let reason = if mime_message.was_encrypted() {
"Valid signature missing"
} else {
+3 -1
View File
@@ -126,6 +126,7 @@ impl Smtp {
let login_params =
prioritize_server_login_params(&context.sql, login_params, "smtp").await?;
let mut first_error = None;
for lp in login_params {
info!(context, "SMTP trying to connect to {}.", &lp.connection);
let transport = match connect::connect_and_auth(
@@ -143,6 +144,7 @@ impl Smtp {
Ok(transport) => transport,
Err(err) => {
warn!(context, "SMTP failed to connect and authenticate: {err:#}.");
first_error.get_or_insert(err);
continue;
}
};
@@ -157,7 +159,7 @@ impl Smtp {
return Ok(());
}
Err(format_err!("SMTP failed to connect"))
Err(first_error.unwrap_or_else(|| format_err!("No SMTP connection candidates provided")))
}
}
+1
View File
@@ -168,6 +168,7 @@ async fn connect_stream(
connect_insecure_proxy(context, host, port, proxy_config.clone()).await?
}
};
update_connection_history(context, "smtp", host, port, host, time()).await?;
Ok(stream)
} else {
let load_cache = match security {
+132 -84
View File
@@ -5,7 +5,7 @@ use std::path::{Path, PathBuf};
use anyhow::{bail, Context as _, Result};
use rusqlite::{config::DbConfig, types::ValueRef, Connection, OpenFlags, Row};
use tokio::sync::{Mutex, MutexGuard, RwLock};
use tokio::sync::RwLock;
use crate::blob::BlobObject;
use crate::chat::{self, add_device_msg, update_device_icon, update_saved_messages_icon};
@@ -60,11 +60,6 @@ pub struct Sql {
/// Database file path
pub(crate) dbfile: PathBuf,
/// Write transactions mutex.
///
/// See [`Self::write_lock`].
write_mtx: Mutex<()>,
/// SQL connection pool.
pool: RwLock<Option<Pool>>,
@@ -81,7 +76,6 @@ impl Sql {
pub fn new(dbfile: PathBuf) -> Sql {
Self {
dbfile,
write_mtx: Mutex::new(()),
pool: Default::default(),
is_encrypted: Default::default(),
config_cache: Default::default(),
@@ -147,7 +141,8 @@ impl Sql {
let mut config_cache = self.config_cache.write().await;
config_cache.clear();
self.call_write(move |conn| {
let query_only = false;
self.call(query_only, move |conn| {
// Check that backup passphrase is correct before resetting our database.
conn.execute("ATTACH DATABASE ? AS backup KEY ?", (path_str, passphrase))
.context("failed to attach backup database")?;
@@ -325,7 +320,8 @@ impl Sql {
let mut lock = self.pool.write().await;
let pool = lock.take().context("SQL connection pool is not open")?;
let conn = pool.get().await?;
let query_only = false;
let conn = pool.get(query_only).await?;
if !passphrase.is_empty() {
conn.pragma_update(None, "rekey", passphrase.clone())
.context("Failed to set PRAGMA rekey")?;
@@ -337,59 +333,20 @@ impl Sql {
Ok(())
}
/// Locks the write transactions mutex in order to make sure that there never are
/// multiple write transactions at once.
/// Allocates a connection and calls `function` with the connection.
///
/// Doing the locking ourselves instead of relying on SQLite has these reasons:
///
/// - SQLite's locking mechanism is non-async, blocking a thread
/// - SQLite's locking mechanism just sleeps in a loop, which is really inefficient
///
/// ---
///
/// More considerations on alternatives to the current approach:
///
/// We use [DEFERRED](https://www.sqlite.org/lang_transaction.html#deferred_immediate_and_exclusive_transactions) transactions.
///
/// In order to never get concurrency issues, we could make all transactions IMMEDIATE,
/// but this would mean that there can never be two simultaneous transactions.
///
/// Read transactions can simply be made DEFERRED to run in parallel w/o any drawbacks.
///
/// DEFERRED write transactions without doing the locking ourselves would have these drawbacks:
///
/// 1. As mentioned above, SQLite's locking mechanism is non-async and sleeps in a loop.
/// 2. If there are other write transactions, we block the db connection until
/// upgraded. If some reader comes then, it has to get the next, less used connection with a
/// worse per-connection page cache (SQLite allows one write and any number of reads in parallel).
/// 3. If a transaction is blocked for more than `busy_timeout`, it fails with SQLITE_BUSY.
/// 4. If upon a successful upgrade to a write transaction the db has been modified,
/// the transaction has to be rolled back and retried, which means extra work in terms of
/// CPU/battery.
///
/// The only pro of making write transactions DEFERRED w/o the external locking would be some
/// parallelism between them.
///
/// Another option would be to make write transactions IMMEDIATE, also
/// w/o the external locking. But then cons 1. - 3. above would still be valid.
pub async fn write_lock(&self) -> MutexGuard<'_, ()> {
self.write_mtx.lock().await
}
/// Allocates a connection and calls `function` with the connection. If `function` does write
/// queries,
/// - either first take a lock using `write_lock()`
/// - or use `call_write()` instead.
/// If `query_only` is true, allocates read-only connection,
/// otherwise allocates write connection.
///
/// Returns the result of the function.
async fn call<'a, F, R>(&'a self, function: F) -> Result<R>
async fn call<'a, F, R>(&'a self, query_only: bool, function: F) -> Result<R>
where
F: 'a + FnOnce(&mut Connection) -> Result<R> + Send,
R: Send + 'static,
{
let lock = self.pool.read().await;
let pool = lock.as_ref().context("no SQL connection")?;
let mut conn = pool.get().await?;
let mut conn = pool.get(query_only).await?;
let res = tokio::task::block_in_place(move || function(&mut conn))?;
Ok(res)
}
@@ -403,8 +360,8 @@ impl Sql {
F: 'a + FnOnce(&mut Connection) -> Result<R> + Send,
R: Send + 'static,
{
let _lock = self.write_lock().await;
self.call(function).await
let query_only = false;
self.call(query_only, function).await
}
/// Execute `query` assuming it is a write query, returning the number of affected rows.
@@ -444,7 +401,8 @@ impl Sql {
G: Send + FnMut(rusqlite::MappedRows<F>) -> Result<H>,
H: Send + 'static,
{
self.call(move |conn| {
let query_only = true;
self.call(query_only, move |conn| {
let mut stmt = conn.prepare(sql)?;
let res = stmt.query_map(params, f)?;
g(res)
@@ -476,7 +434,8 @@ impl Sql {
F: FnOnce(&rusqlite::Row) -> rusqlite::Result<T> + Send,
T: Send + 'static,
{
self.call(move |conn| {
let query_only = true;
self.call(query_only, move |conn| {
let res = conn.query_row(query, params, f)?;
Ok(res)
})
@@ -512,7 +471,8 @@ impl Sql {
/// Query the database if the requested table already exists.
pub async fn table_exists(&self, name: &str) -> Result<bool> {
self.call(move |conn| {
let query_only = true;
self.call(query_only, move |conn| {
let mut exists = false;
conn.pragma(None, "table_info", name.to_string(), |_row| {
// will only be executed if the info was found
@@ -527,7 +487,8 @@ impl Sql {
/// Check if a column exists in a given table.
pub async fn col_exists(&self, table_name: &str, col_name: &str) -> Result<bool> {
self.call(move |conn| {
let query_only = true;
self.call(query_only, move |conn| {
let mut exists = false;
// `PRAGMA table_info` returns one row per column,
// each row containing 0=cid, 1=name, 2=type, 3=notnull, 4=dflt_value
@@ -555,10 +516,13 @@ impl Sql {
F: Send + FnOnce(&rusqlite::Row) -> rusqlite::Result<T>,
T: Send + 'static,
{
self.call(move |conn| match conn.query_row(sql.as_ref(), params, f) {
Ok(res) => Ok(Some(res)),
Err(rusqlite::Error::QueryReturnedNoRows) => Ok(None),
Err(err) => Err(err.into()),
let query_only = true;
self.call(query_only, move |conn| {
match conn.query_row(sql.as_ref(), params, f) {
Ok(res) => Ok(Some(res)),
Err(rusqlite::Error::QueryReturnedNoRows) => Ok(None),
Err(err) => Err(err.into()),
}
})
.await
}
@@ -717,6 +681,36 @@ fn new_connection(path: &Path, passphrase: &str) -> Result<Connection> {
Ok(conn)
}
// Tries to clear the freelist to free some space on the disk.
//
// This only works if auto_vacuum is enabled.
async fn incremental_vacuum(context: &Context) -> Result<()> {
context
.sql
.call_write(move |conn| {
let mut stmt = conn
.prepare("PRAGMA incremental_vacuum")
.context("Failed to prepare incremental_vacuum statement")?;
// It is important to step the statement until it returns no more rows.
// Otherwise it will not free as many pages as it can:
// <https://stackoverflow.com/questions/53746807/sqlite-incremental-vacuum-removing-only-one-free-page>.
let mut rows = stmt
.query(())
.context("Failed to run incremental_vacuum statement")?;
let mut row_count = 0;
while let Some(_row) = rows
.next()
.context("Failed to step incremental_vacuum statement")?
{
row_count += 1;
}
info!(context, "Incremental vacuum freed {row_count} pages.");
Ok(())
})
.await
}
/// Cleanup the account to restore some storage and optimize the database.
pub async fn housekeeping(context: &Context) -> Result<()> {
// Setting `Config::LastHousekeeping` at the beginning avoids endless loops when things do not
@@ -749,24 +743,8 @@ pub async fn housekeeping(context: &Context) -> Result<()> {
);
}
// Try to clear the freelist to free some space on the disk. This
// only works if auto_vacuum is enabled.
match context
.sql
.query_row_optional("PRAGMA incremental_vacuum", (), |_row| Ok(()))
.await
{
Err(err) => {
warn!(context, "Failed to run incremental vacuum: {err:#}.");
}
Ok(Some(())) => {
// Incremental vacuum returns a zero-column result if it did anything.
info!(context, "Successfully ran incremental vacuum.");
}
Ok(None) => {
// Incremental vacuum returned `SQLITE_DONE` immediately,
// there were no pages to remove.
}
if let Err(err) = incremental_vacuum(context).await {
warn!(context, "Failed to run incremental vacuum: {err:#}.");
}
context
@@ -1092,9 +1070,10 @@ mod tests {
async fn test_auto_vacuum() -> Result<()> {
let t = TestContext::new().await;
let query_only = true;
let auto_vacuum = t
.sql
.call(|conn| {
.call(query_only, |conn| {
let auto_vacuum = conn.pragma_query_value(None, "auto_vacuum", |row| {
let auto_vacuum: i32 = row.get(0)?;
Ok(auto_vacuum)
@@ -1320,8 +1299,9 @@ mod tests {
{
let lock = sql.pool.read().await;
let pool = lock.as_ref().unwrap();
let conn1 = pool.get().await?;
let conn2 = pool.get().await?;
let query_only = true;
let conn1 = pool.get(query_only).await?;
let conn2 = pool.get(query_only).await?;
conn1
.query_row("SELECT count(*) FROM sqlite_master", [], |_row| Ok(()))
.unwrap();
@@ -1346,4 +1326,72 @@ mod tests {
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_query_only() -> Result<()> {
let t = TestContext::new().await;
// `query_row` does not acquire write lock
// and operates on read-only connection.
// Using it to `INSERT` should fail.
let res = t
.sql
.query_row(
"INSERT INTO config (keyname, value) VALUES (?, ?) RETURNING 1",
("xyz", "ijk"),
|row| {
let res: u32 = row.get(0)?;
Ok(res)
},
)
.await;
assert!(res.is_err());
// If you want to `INSERT` and get value via `RETURNING`,
// use `call_write` or `transaction`.
let res: Result<u32> = t
.sql
.call_write(|conn| {
let val = conn.query_row(
"INSERT INTO config (keyname, value) VALUES (?, ?) RETURNING 2",
("foo", "bar"),
|row| {
let res: u32 = row.get(0)?;
Ok(res)
},
)?;
Ok(val)
})
.await;
assert_eq!(res.unwrap(), 2);
let res = t
.sql
.transaction(|t| {
let val = t.query_row(
"INSERT INTO config (keyname, value) VALUES (?, ?) RETURNING 3",
("abc", "def"),
|row| {
let res: u32 = row.get(0)?;
Ok(res)
},
)?;
Ok(val)
})
.await;
assert_eq!(res.unwrap(), 3);
Ok(())
}
/// Tests that incremental_vacuum does not fail.
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_incremental_vacuum() -> Result<()> {
let t = TestContext::new().await;
incremental_vacuum(&t).await?;
Ok(())
}
}
+106 -17
View File
@@ -7,23 +7,67 @@
//! Each SQLite connection has its own page cache, so allocating recently used connections
//! improves the performance compared to, for example, organizing the pool as a queue
//! and returning the least recently used connection each time.
//!
//! Pool returns at most one write connection (with `PRAGMA query_only=0`).
//! This ensures that there never are multiple write transactions at once.
//!
//! Doing the locking ourselves instead of relying on SQLite has these reasons:
//!
//! - SQLite's locking mechanism is non-async, blocking a thread
//! - SQLite's locking mechanism just sleeps in a loop, which is really inefficient
//!
//! ---
//!
//! More considerations on alternatives to the current approach:
//!
//! We use [DEFERRED](https://www.sqlite.org/lang_transaction.html#deferred_immediate_and_exclusive_transactions) transactions.
//!
//! In order to never get concurrency issues, we could make all transactions IMMEDIATE,
//! but this would mean that there can never be two simultaneous transactions.
//!
//! Read transactions can simply be made DEFERRED to run in parallel w/o any drawbacks.
//!
//! DEFERRED write transactions without doing the locking ourselves would have these drawbacks:
//!
//! 1. As mentioned above, SQLite's locking mechanism is non-async and sleeps in a loop.
//! 2. If there are other write transactions, we block the db connection until
//! upgraded. If some reader comes then, it has to get the next, less used connection with a
//! worse per-connection page cache (SQLite allows one write and any number of reads in parallel).
//! 3. If a transaction is blocked for more than `busy_timeout`, it fails with SQLITE_BUSY.
//! 4. If upon a successful upgrade to a write transaction the db has been modified,
//! the transaction has to be rolled back and retried, which means extra work in terms of
//! CPU/battery.
//!
//! The only pro of making write transactions DEFERRED w/o the external locking would be some
//! parallelism between them.
//!
//! Another option would be to make write transactions IMMEDIATE, also
//! w/o the external locking. But then cons 1. - 3. above would still be valid.
use std::ops::{Deref, DerefMut};
use std::sync::{Arc, Weak};
use anyhow::{Context, Result};
use parking_lot::Mutex;
use rusqlite::Connection;
use tokio::sync::{OwnedSemaphorePermit, Semaphore};
use tokio::sync::{Mutex, OwnedMutexGuard, OwnedSemaphorePermit, Semaphore};
/// Inner connection pool.
#[derive(Debug)]
struct InnerPool {
/// Available connections.
connections: Mutex<Vec<Connection>>,
connections: parking_lot::Mutex<Vec<Connection>>,
/// Counts the number of available connections.
semaphore: Arc<Semaphore>,
/// Write mutex.
///
/// This mutex ensures there is at most
/// one write connection with `query_only=0`.
///
/// This mutex is locked when write connection
/// is outside the pool.
write_mutex: Arc<Mutex<()>>,
}
impl InnerPool {
@@ -35,6 +79,56 @@ impl InnerPool {
connections.push(connection);
drop(connections);
}
/// Retrieves a connection from the pool.
///
/// Sets `query_only` pragma to the provided value
/// to prevent accidentaly misuse of connection
/// for writing when reading is intended.
/// Only pass `query_only=false` if you want
/// to use the connection for writing.
pub async fn get(self: Arc<Self>, query_only: bool) -> Result<PooledConnection> {
if query_only {
let permit = self.semaphore.clone().acquire_owned().await?;
let conn = {
let mut connections = self.connections.lock();
connections
.pop()
.context("Got a permit when there are no connections in the pool")?
};
conn.pragma_update(None, "query_only", "1")?;
let conn = PooledConnection {
pool: Arc::downgrade(&self),
conn: Some(conn),
_permit: permit,
_write_mutex_guard: None,
};
Ok(conn)
} else {
// We get write guard first to avoid taking a permit
// and not using it, blocking a reader from getting a connection
// while being ourselves blocked by another wrtier.
let write_mutex_guard = Arc::clone(&self.write_mutex).lock_owned().await;
// We may still have to wait for a connection
// to be returned by some reader.
let permit = self.semaphore.clone().acquire_owned().await?;
let conn = {
let mut connections = self.connections.lock();
connections.pop().context(
"Got a permit and write lock when there are no connections in the pool",
)?
};
conn.pragma_update(None, "query_only", "0")?;
let conn = PooledConnection {
pool: Arc::downgrade(&self),
conn: Some(conn),
_permit: permit,
_write_mutex_guard: Some(write_mutex_guard),
};
Ok(conn)
}
}
}
/// Pooled connection.
@@ -47,6 +141,11 @@ pub struct PooledConnection {
/// Semaphore permit, dropped after returning the connection to the pool.
_permit: OwnedSemaphorePermit,
/// Write mutex guard.
///
/// `None` for read-only connections with `PRAGMA query_only=1`.
_write_mutex_guard: Option<OwnedMutexGuard<()>>,
}
impl Drop for PooledConnection {
@@ -86,24 +185,14 @@ impl Pool {
pub fn new(connections: Vec<Connection>) -> Self {
let semaphore = Arc::new(Semaphore::new(connections.len()));
let inner = Arc::new(InnerPool {
connections: Mutex::new(connections),
connections: parking_lot::Mutex::new(connections),
semaphore,
write_mutex: Default::default(),
});
Pool { inner }
}
/// Retrieves a connection from the pool.
pub async fn get(&self) -> Result<PooledConnection> {
let permit = self.inner.semaphore.clone().acquire_owned().await?;
let mut connections = self.inner.connections.lock();
let conn = connections
.pop()
.context("got a permit when there are no connections in the pool")?;
let conn = PooledConnection {
pool: Arc::downgrade(&self.inner),
conn: Some(conn),
_permit: permit,
};
Ok(conn)
pub async fn get(&self, query_only: bool) -> Result<PooledConnection> {
Arc::clone(&self.inner).get(query_only).await
}
}
+18
View File
@@ -1106,6 +1106,24 @@ impl EventTracker {
.expect("timeout waiting for event match")
}
/// Consumes emitted events returning the first matching one if any.
pub async fn get_matching_opt<F: Fn(&EventType) -> bool>(
&self,
ctx: &Context,
event_matcher: F,
) -> Option<EventType> {
ctx.emit_event(EventType::Test);
loop {
let event = self.recv().await.unwrap();
if event_matcher(&event.typ) {
return Some(event.typ);
}
if let EventType::Test = event.typ {
return None;
}
}
}
/// Consumes events looking for an [`EventType::Info`] with substring matching.
pub async fn get_info_contains(&self, s: &str) -> EventType {
self.get_matching(|evt| match evt {
+45
View File
@@ -297,6 +297,7 @@ async fn test_verified_oneonone_chat_enable_disable() -> Result<()> {
assert!(chat.is_protected());
for alice_accepts_breakage in [true, false] {
SystemTime::shift(std::time::Duration::from_secs(300));
// Bob uses Thunderbird to send a message
receive_imf(
&alice,
@@ -730,6 +731,50 @@ async fn test_break_protection_then_verify_again() -> Result<()> {
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_message_from_old_dc_setup() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
let bob_old = &tcm.unconfigured().await;
enable_verified_oneonone_chats(&[alice, bob_old]).await;
mark_as_verified(bob_old, alice).await;
bob_old.configure_addr("bob@example.net").await;
let chat = bob_old.create_chat(alice).await;
let sent_old = bob_old
.send_text(chat.id, "Soon i'll have a new device")
.await;
SystemTime::shift(std::time::Duration::from_secs(3600));
tcm.section("Bob reinstalls DC");
let bob = &tcm.bob().await;
enable_verified_oneonone_chats(&[bob]).await;
mark_as_verified(alice, bob).await;
mark_as_verified(bob, alice).await;
tcm.send_recv(bob, alice, "Now i have it!").await;
assert_verified(alice, bob, ProtectionStatus::Protected).await;
let msg = alice.recv_msg(&sent_old).await;
assert!(!msg.get_showpadlock());
let contact = alice.add_or_lookup_contact(bob).await;
// The outdated Bob's Autocrypt header isn't applied, so the verification preserves.
assert!(contact.is_verified(alice).await.unwrap());
let chat = alice.get_chat(bob).await;
assert!(chat.is_protected());
assert_eq!(chat.is_protection_broken(), false);
let protection_msg = alice.get_last_msg().await;
assert_eq!(
protection_msg.param.get_cmd(),
SystemMessage::ChatProtectionEnabled
);
assert!(protection_msg.timestamp_sort >= msg.timestamp_rcvd);
alice
.golden_test_chat(msg.chat_id, "verified_chats_message_from_old_dc_setup")
.await;
Ok(())
}
/// Regression test for the following bug:
///
/// - Scan your chat partner's QR Code
+1
View File
@@ -0,0 +1 @@
.
+2 -1
View File
@@ -1,7 +1,8 @@
Single#Chat#10: Bob [bob@example.net]
Single#Chat#10: Bob [bob@example.net] 🛡️
--------------------------------------------------------------------------------
Msg#10: info (Contact#Contact#Info): Messages are guaranteed to be end-to-end encrypted from now on. [NOTICED][INFO 🛡️]
Msg#11🔒: (Contact#Contact#10): Heyho from my verified device! [FRESH]
Msg#12: info (Contact#Contact#Info): Bob sent a message from another device. [NOTICED][INFO 🛡️❌]
Msg#13: (Contact#Contact#10): Old, unverified message [SEEN]
Msg#14: info (Contact#Contact#Info): Messages are guaranteed to be end-to-end encrypted from now on. [NOTICED][INFO 🛡️]
--------------------------------------------------------------------------------
@@ -0,0 +1,8 @@
Single#Chat#10: bob@example.net [bob@example.net] 🛡️
--------------------------------------------------------------------------------
Msg#10: info (Contact#Contact#Info): Messages are guaranteed to be end-to-end encrypted from now on. [NOTICED][INFO 🛡️]
Msg#11🔒: (Contact#Contact#10): Now i have it! [FRESH]
Msg#12: info (Contact#Contact#Info): bob@example.net sent a message from another device. [NOTICED][INFO 🛡️❌]
Msg#13: (Contact#Contact#10): Soon i'll have a new device [FRESH]
Msg#14: info (Contact#Contact#Info): Messages are guaranteed to be end-to-end encrypted from now on. [NOTICED][INFO 🛡️]
--------------------------------------------------------------------------------