Compare commits

...
Author SHA1 Message Date
Floris Bruynooghe 16bb02ba96 Add changelog 2023-03-20 19:55:32 +01:00
Floris Bruynooghe cabefee4c0 Make sure set_last_error is called for .log_err()
We have a common .log_err() trait method which is very prevalent and
useful in the FFI bindings.  However this was not hooked up with the
set_last_error/get_last_error mechanism, which the error! macro was
hooked up with.

This hooks correctly hooks up the two and makes sure to log all errors
around backup provider/retrieval.
2023-03-20 16:32:36 +01:00
Hocuriandbjoern 57445eedb1 More accurate maybe_add_bcc_self device message text (#4175)
* More accurate maybe_add_bcc_self device message text

* changelog

* Update src/imex.rs

Co-authored-by: bjoern <r10s@b44t.com>

* Capitalize Send Copy to Self

---------

Co-authored-by: bjoern <r10s@b44t.com>
2023-03-20 12:54:16 +01:00
link2xt a501f10756 Get rid of duplicate uuid dependency 2023-03-20 10:07:59 +00:00
link2xt 5d80d4788c Pause I/O in get_backup() 2023-03-20 10:24:59 +01:00
link2xt 0c02886005 Update human-panic, but disable color
Avoid pulling in new `anstream` dependency
2023-03-19 19:10:25 +00:00
link2xt 24856f3050 Merge branch 'flub/send-backup'
PR: <https://github.com/deltachat/deltachat-core-rust/pull/4007>
2023-03-19 15:21:59 +00:00
link2xt 8e6434068e Fix remaining cargo-deny warnings 2023-03-19 14:40:46 +00:00
link2xt 800d2b14a5 Add cargo-deny exceptions for old crates 2023-03-19 14:37:23 +00:00
B. Petersen 3a861d2f84 some doxygen fixes 2023-03-19 15:24:51 +01:00
dependabot[bot] 4ba00f7440 Merge pull request #4171 from deltachat/dependabot/cargo/axum-0.6.11 2023-03-19 13:30:54 +00:00
link2xt 40fc61da4f changelog: add link and date to the latest release 2023-03-19 12:07:55 +00:00
link2xt eb0f896d57 Use scheduler.is_running() 2023-03-19 11:23:09 +00:00
link2xt 71bb89fac1 Merge remote-tracking branch 'origin/master' into flub/send-backup 2023-03-19 11:10:07 +00:00
link2xt b89199db54 Merge branch 'flub/pause-io'
PR: <https://github.com/deltachat/deltachat-core-rust/pull/4138>
2023-03-19 11:06:01 +00:00
link2xt e39429c2e3 rustfmt 2023-03-19 10:18:49 +00:00
link2xt 17de3d3236 Remove TODOs 2023-03-19 10:17:18 +00:00
link2xt 3177f9967d Add a comment aronud IMAP loop task handle 2023-03-19 10:16:43 +00:00
link2xt 81418d8ee5 Log error on pause guard drop without resuming instead of working around
I checked that tests still pass even if error! is replaced with panic!
2023-03-19 10:13:59 +00:00
link2xt a2e7d914a0 Changelog fixup 2023-03-19 09:37:09 +00:00
Floris Bruynooghe 4bf38c0e29 clippy 2023-03-19 09:36:41 +00:00
Floris Bruynooghe 0079cd4766 Add changelog 2023-03-19 09:36:38 +00:00
Floris Bruynooghe 2c3b2b8c2d move pause to only exist on Scheduler 2023-03-19 09:36:03 +00:00
Floris Bruynooghe 52fa58a3ce No need for jsonrpc to do this manually 2023-03-19 09:36:03 +00:00
Floris Bruynooghe 32a7e5ed82 Remove requirement for stopping io for imex 2023-03-19 09:36:03 +00:00
Floris Bruynooghe 097113f01e fixup paused flag use 2023-03-19 09:36:03 +00:00
Floris Bruynooghe 1d42e4743f Allow pausing IO scheduler from inside core
To handle backups the UIs have to make sure they do stop the IO
scheduler and also don't accidentally restart it while working on it.
Since they have to call start_io from a bunch of locations this can be
a bit difficult to manage.

This introduces a mechanism for the core to pause IO for some time,
which is used by the imex function.  It interacts well with other
calls to dc_start_io() and dc_stop_io() making sure that when resumed
the scheduler will be running or not as the latest calls to them.

This was a little more invasive then hoped due to the scheduler.  The
additional abstraction of the scheduler on the context seems a nice
improvement though.
2023-03-19 09:36:03 +00:00
dependabot[bot] 5ecdea47db cargo: bump axum from 0.6.9 to 0.6.11
Bumps [axum](https://github.com/tokio-rs/axum) from 0.6.9 to 0.6.11.
- [Release notes](https://github.com/tokio-rs/axum/releases)
- [Changelog](https://github.com/tokio-rs/axum/blob/main/CHANGELOG.md)
- [Commits](https://github.com/tokio-rs/axum/compare/axum-v0.6.9...axum-v0.6.11)

---
updated-dependencies:
- dependency-name: axum
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-03-19 09:35:20 +00:00
dependabot[bot] 5b92b6355e Merge pull request #4168 from deltachat/dependabot/cargo/fuzz/libsqlite3-sys-0.25.2 2023-03-19 09:24:18 +00:00
link2xt 5eb7206b2d Format documentation comment for sync_qr_code_token_deletion 2023-03-19 00:10:45 +00:00
dependabot[bot] 0a65081db0 Bump libsqlite3-sys from 0.24.2 to 0.25.2 in /fuzz
Bumps [libsqlite3-sys](https://github.com/rusqlite/rusqlite) from 0.24.2 to 0.25.2.
- [Release notes](https://github.com/rusqlite/rusqlite/releases)
- [Changelog](https://github.com/rusqlite/rusqlite/blob/master/Changelog.md)
- [Commits](https://github.com/rusqlite/rusqlite/compare/libsqlite3-sys-0.24.2...v0.25.2)

---
updated-dependencies:
- dependency-name: libsqlite3-sys
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-03-18 19:31:46 +00:00
dignifiedquire a59e72e7d8 update iroh 2023-03-17 23:41:36 +01:00
dignifiedquire fd358617f5 feat: implement more detailed progress on sending 2023-03-17 23:37:00 +01:00
link2xt b26a351786 Switch quinn to the main branch
It has Android fixes merged
2023-03-17 22:26:42 +00:00
dignifiedquire 575b43d9a0 Merge remote-tracking branch 'origin/master' into flub/send-backup 2023-03-17 15:53:40 +01:00
link2xt 0a5542a698 Log transfer rate on success 2023-03-17 10:45:43 +00:00
dignifiedquire 518bd19e96 fix: do not block transfer on db import 2023-03-17 11:29:27 +01:00
link2xt 961e3ad7e2 Update spin 0.9.5->0.9.6 2023-03-17 00:00:37 +00:00
dignifiedquire 7a49e9401f fixup clippy & fmt 2023-03-16 17:53:27 +01:00
dignifiedquire 3701936129 Merge remote-tracking branch 'origin/master' into flub/send-backup 2023-03-16 17:50:00 +01:00
dignifiedquire c02686b56e update iroh 2023-03-16 17:38:15 +01:00
link2xt 9a7ff9d2b1 Switch quinn to ecn-einval-fallback branch. 2023-03-16 16:25:46 +00:00
link2xt 56f6d6849e Patch quinn to work on android 2023-03-15 12:45:58 +00:00
link2xt cbc18ee5a4 Log connection errors 2023-03-14 18:49:14 +00:00
dignifiedquire 14521cfc2d improve address handling 2023-03-14 15:38:28 +01:00
dignifiedquire 5e4807b7ac update patched default-net 2023-03-13 12:51:54 +01:00
link2xt 28d9bec0b4 Patch default-net 2023-03-10 17:14:01 +00:00
Floris Bruynooghe 05e50ea787 Connect to all addresses the provider has
This uses a branch directly from iroh repo again
2023-03-09 16:49:34 +01:00
Floris Bruynooghe 02afacf989 clarify docs 2023-03-09 16:12:33 +01:00
Floris Bruynooghe c7de4f66e7 Bind to 0.0.0.0 2023-03-09 15:34:15 +01:00
Floris Bruynooghe c9b8c5079b wording 2023-03-07 15:45:18 +01:00
Floris Bruynooghe eec5ae96e8 Update docs and fix string allocation
The docs say you should always unref the string and NULL is never
returned.  The implementation should follow that.
2023-03-07 15:36:33 +01:00
Floris Bruynooghe 4b94eadf5e typo 2023-03-07 14:40:51 +01:00
Floris Bruynooghe 52a1886937 naming conventions!
they're hard
2023-03-07 14:40:01 +01:00
Floris Bruynooghe 9767f51c3d update .h file too 2023-03-07 14:13:42 +01:00
Floris Bruynooghe 6674b888cc Merge branch 'master' into flub/send-backup 2023-03-07 12:52:45 +01:00
Floris Bruynooghe a5e6bd3e8e Do not require context for non-context methods
This follows the ffi style better.
2023-03-07 12:49:42 +01:00
Floris BruynoogheandHocuri b6c24932a7 Apply typos from code review
Co-authored-by: Hocuri <hocuri@gmx.de>
2023-03-07 12:23:30 +01:00
Floris Bruynooghe d73d56c399 bump testdir for windows bug workaround 2023-03-03 13:13:58 +01:00
Floris Bruynooghe 731e90f0d5 update cargo-deny 2023-03-03 12:53:43 +01:00
Floris Bruynooghe e0a6c2ef54 Merge branch 'master' into flub/send-backup 2023-03-03 12:46:05 +01:00
Floris Bruynooghe c5408e0561 Merge branch 'master' into flub/send-backup 2023-03-03 09:48:33 +01:00
Floris Bruynooghe c1a2df91ac Fix typo in blob names
This is now tested properly too.
2023-03-02 21:53:13 +01:00
Floris Bruynooghe da85c2412e fix iterator 2023-03-02 21:48:14 +01:00
Floris Bruynooghe d108f9b3e3 clippy 2023-03-02 11:47:03 +01:00
Floris Bruynooghe e3014a349c Merge branch 'master' into flub/send-backup 2023-03-02 11:35:17 +01:00
Floris Bruynooghe 9d88ef069e log some more 2023-03-02 11:21:05 +01:00
Floris Bruynooghe 155dff2813 renaming of upstream 2023-03-02 11:18:30 +01:00
Floris Bruynooghe 38d4ea8514 Use std::slice::Iter instead of manually tracking the offset 2023-03-02 11:15:02 +01:00
Floris Bruynooghe 6f24874eb8 Use a RAII guard to remove the db export 2023-03-02 10:58:39 +01:00
Floris Bruynooghe 2d20812652 some typos 2023-03-02 09:39:50 +01:00
Floris Bruynooghe 5762fbb9a7 Allow JSON-RPC to get text of QR code as well
Desktop does use this as it allows reading QR codes as text from the
clipboard as well as copying the QR text to the clipboard instead of
showing the QR code.
2023-03-01 11:27:21 +01:00
Floris Bruynooghe 0e06da22df fix symbol name 2023-02-22 18:51:17 +01:00
Floris Bruynooghe 5833a9b347 fix doc comments 2023-02-22 18:50:32 +01:00
Floris Bruynooghe 0ef8d57881 Merge branch 'master' into flub/send-backup 2023-02-22 18:15:23 +01:00
Floris Bruynooghe fc64c33368 Use released version of sendme^Wiroh
This switches to a released version.  It has been renamed from sendme
to iroh.
2023-02-22 16:05:24 +01:00
Floris Bruynooghe 1b39be8a42 Merge branch 'master' into flub/send-backup 2023-02-22 15:54:23 +01:00
Floris Bruynooghe a1e19e2c41 Merge branch 'master' into flub/send-backup 2023-02-20 17:39:52 +01:00
Floris Bruynooghe b920db12c7 Split _wait and _unref
This also removes BackupProvider::join in favour of implementing
Future directly.  I wondered about implementing a FusedFutre to make
this a little safer but it would introduce a dependency on the futures
crate in deltachat-ffi which did not exist yet, so I didn't do that.
2023-02-20 15:56:05 +01:00
Floris Bruynooghe 73b90eee3e improve docs 2023-02-20 13:10:29 +01:00
Floris Bruynooghe 4637a28bf6 doc comment 2023-02-20 13:08:43 +01:00
Floris Bruynooghe d0638c1542 typo 2023-02-20 13:05:11 +01:00
Floris Bruynooghe 788d3125a3 Do not save svg to file, just print qr text 2023-02-20 13:02:16 +01:00
Floris Bruynooghe 3c4ffc3550 Some fixes 2023-02-20 12:58:23 +01:00
Floris Bruynooghe ada858f439 Improve comments, mostly ffi. and some renames 2023-02-20 12:48:43 +01:00
Floris Bruynooghe f2570945c6 Don't reimplement qr::format_backup 2023-02-16 18:18:18 +01:00
Floris Bruynooghe 8072f78058 Do not emit ImexEvent From BlobDirIter
We no longer need that in the transfer case, that would give very
weird results.  This also means there is nothing imex-specific about
this anymore so move it to blobs.rs
2023-02-16 18:05:09 +01:00
Floris Bruynooghe 8ae0ee5a67 Merge branch 'master' into flub/send-backup 2023-02-16 17:19:31 +01:00
Floris Bruynooghe a75d2b1c80 Create a blocking call for jsonrpc 2023-02-16 17:15:54 +01:00
Floris Bruynooghe c48c2af7a1 Allow retrieval of backup QR on context
This enables being able to get the QR code without needing to have
access to the BackupProvider itself.  This is useful for the JSON-RPC
server.
2023-02-16 16:49:20 +01:00
Floris Bruynooghe 490a14c5ef Remove the need for a directory for db export
Plus on import use the context directory.  We can actually write there
just fine.
2023-02-16 16:06:41 +01:00
Floris Bruynooghe dcce6ef50b Some docs 2023-02-16 15:19:44 +01:00
Floris Bruynooghe 7cf0820d2b diff 2023-02-16 14:56:18 +01:00
Floris Bruynooghe 0bae3caaff dear CI masters: i regret every trying to be clever 2023-02-16 14:55:24 +01:00
Floris Bruynooghe bca0b256c9 goodness ci? 2023-02-16 14:52:05 +01:00
Floris Bruynooghe a53d30c459 fixed another bug, try main again 2023-02-16 14:49:48 +01:00
Floris Bruynooghe 7a9f497aa7 why can't i see this action now? 2023-02-16 14:49:04 +01:00
Floris Bruynooghe f9f9bc3efb yaml 2023-02-16 09:10:47 +01:00
Floris Bruynooghe 904990bf91 ugh, yaml syntax 2023-02-16 09:08:14 +01:00
Floris Bruynooghe b2266ffca1 make the have a valid on spec at least so gh doesn't complain too much 2023-02-16 09:06:40 +01:00
Floris Bruynooghe bb9a3d4b8e more bug hunting: disable most ci, point to branch 2023-02-16 09:00:27 +01:00
Floris Bruynooghe e565e19b42 fix msrv in sendme 2023-02-15 16:01:39 +01:00
Floris Bruynooghe 41319c85c7 patch in previous revision of sendme
main broke rust 1.63 support :'(
2023-02-15 15:12:14 +01:00
Floris Bruynooghe daf56804a5 use correct branch 2023-02-15 14:57:26 +01:00
Floris Bruynooghe 6f7a43804d Add changelog 2023-02-15 14:48:17 +01:00
Floris Bruynooghe 0ca76d36ef Merge branch 'master' into flub/send-backup 2023-02-15 14:46:57 +01:00
Floris Bruynooghe ec5789997a back to master 2023-02-15 14:45:52 +01:00
Floris Bruynooghe 7a0d61bbb0 hey 2023-02-15 13:50:39 +01:00
Floris Bruynooghe 1c2461974d better way 2023-02-14 18:29:15 +01:00
Floris Bruynooghe 2a754744fe char, not chat 2023-02-14 18:03:39 +01:00
Floris Bruynooghe b413593c43 hi 2023-02-14 17:39:58 +01:00
Floris Bruynooghe c73edd7e21 oh 2023-02-14 17:20:25 +01:00
Floris Bruynooghe a34a69d8e4 yes, ci fun 2023-02-14 17:15:52 +01:00
Floris Bruynooghe 020a9d33f6 new sendme for lower msrv 2023-02-14 15:49:21 +01:00
Floris Bruynooghe 19f6f89312 no let else :( 2023-02-14 13:29:55 +01:00
Floris Bruynooghe d56e05a11a fixup doc links 2023-02-14 13:27:15 +01:00
Floris Bruynooghe c379a4e5a7 use ProgressEmitter from sendme 2023-02-14 13:19:43 +01:00
Floris Bruynooghe 44c1efe4e4 Add jsonrpc support 2023-02-14 13:05:54 +01:00
Floris Bruynooghe ff0d675082 Make getting backup use the ongoing process 2023-02-14 12:19:40 +01:00
Floris Bruynooghe e1087b4145 translate the string for qr code 2023-02-14 12:07:02 +01:00
Floris Bruynooghe 323535584b implement ffi and use public sendme 2023-02-13 18:25:12 +01:00
Floris Bruynooghe 852adbe514 bits left over from master merge 2023-02-13 15:45:38 +01:00
Floris Bruynooghe 4c78553d90 Merge branch 'master' into flub/send-backup 2023-02-13 11:25:51 +01:00
Floris Bruynooghe a31ae5297a Add to repl example 2023-02-10 18:35:47 +01:00
Floris Bruynooghe e7792a0c65 clippy 2023-02-10 18:27:03 +01:00
Floris Bruynooghe 3c32de1859 Generate a QR code 2023-02-10 18:16:01 +01:00
Floris Bruynooghe 6a3fe3db92 fixup doc comments 2023-02-10 14:15:39 +01:00
Floris Bruynooghe ac048c154d Add progress for provider
Fix progress for getter.  Maths.  It's hard.

Add test for progress.
2023-02-10 13:54:50 +01:00
Floris Bruynooghe 3f51a8ffc2 Some more doc comments 2023-02-10 10:48:10 +01:00
Floris Bruynooghe 2129b2b7a0 Add a ton of code for receiver-side progress 2023-02-09 18:09:16 +01:00
Floris Bruynooghe 3734fc25a7 update callback to take collection by ref 2023-02-09 10:02:18 +01:00
Floris Bruynooghe 05ddc13054 Use name prefixes so the db can not be spoofed by a blob 2023-02-07 18:21:46 +01:00
Floris Bruynooghe 716504b833 do not pull in sendme cli deps 2023-02-07 17:20:35 +01:00
Floris Bruynooghe 187861c3b2 Make stuff work. With test! 2023-02-07 17:18:34 +01:00
Floris Bruynooghe 0b075ac762 Stop after a transfer happened. 2023-02-06 14:58:08 +01:00
Floris Bruynooghe a6c889ed5e Clean up files on errors 2023-02-02 18:11:12 +01:00
Floris Bruynooghe ca1533b0e4 delete device messages 2023-02-02 17:47:41 +01:00
Floris Bruynooghe 3267596a30 handle the database 2023-02-02 17:43:12 +01:00
Floris Bruynooghe 5f29b93970 Start of get support and create new module. 2023-02-02 17:15:23 +01:00
Floris Bruynooghe 2a6a21c33a handle the ongoing process correctly 2023-02-01 17:53:23 +01:00
Floris Bruynooghe 059af398eb Allow decoding the QR code 2023-02-01 17:06:07 +01:00
Floris Bruynooghe 6044e5961b Send and receive backup over network using QR code
This adds functionality to send and receive a backup over the network
using a QR code.

The sender or provider prepares the backup, sets up a server that
waits for clients.  It provides a ticket in the form of a QR code
which contains connection and authentication information.

The receiver uses the QR code to connect to the provider and fetches
backup, restoring it locally.
2023-02-01 16:45:09 +01:00
38 changed files with 3102 additions and 486 deletions
+13 -3
View File
@@ -1,6 +1,6 @@
# Changelog
## Unreleased
## [Unreleased]
### Changes
- "full message view" not needed because of footers that go to contact status #4151
@@ -8,6 +8,13 @@
- Support non-persistent configuration with DELTACHAT_* env
- Print deltachat-repl errors with causes. #4166
- Increase MSRV to 1.64. #4167
- Core takes care of stopping and re-starting IO itself where needed,
e.g. during backup creation. It is no longer needed to call
dc_stop_io(). dc_start_io() can now be called at any time without
harm. #4138
- More accurate maybe_add_bcc_self device message text #4175
- LogExt::log_err now calls Context::set_last_error so that the FFI
can use this and have consistent dc_get_last_error() reporting. #4187
### Fixes
- Fix segmentation fault if `dc_context_unref()` is called during
@@ -16,9 +23,9 @@
during handling the JSON-RPC request. #4153
- Delete expired messages using multiple SQL requests. #4158
- Do not emit "Failed to run incremental vacuum" warnings on success. #4160
- Ability to send backup over network and QR code to setup second device #4007
## 1.111.0
## [1.111.0] - 2023-03-05
### Changes
- Make smeared timestamp generation non-async. #4075
@@ -2307,3 +2314,6 @@
For a full list of changes, please see our closed Pull Requests:
https://github.com/deltachat/deltachat-core-rust/pulls?q=is%3Apr+is%3Aclosed
[unreleased]: https://github.com/deltachat/deltachat-core-rust/compare/v1.111.0...HEAD
[1.111.0]: https://github.com/deltachat/deltachat-core-rust/compare/v1.110.0...v1.111.0
Generated
+1383 -170
View File
File diff suppressed because it is too large Load Diff
+8
View File
@@ -24,6 +24,11 @@ lto = true
panic = 'abort'
opt-level = "z"
[patch.crates-io]
default-net = { git = "https://github.com/dignifiedquire/default-net.git", branch="feat-android" }
quinn-udp = { git = "https://github.com/quinn-rs/quinn", branch="main" }
quinn-proto = { git = "https://github.com/quinn-rs/quinn", branch="main" }
[dependencies]
deltachat_derive = { path = "./deltachat_derive" }
format-flowed = { path = "./format-flowed" }
@@ -48,6 +53,8 @@ futures-lite = "1.12.0"
hex = "0.4.0"
humansize = "2"
image = { version = "0.24.5", default-features=false, features = ["gif", "jpeg", "ico", "png", "pnm", "webp", "bmp"] }
# iroh = { version = "0.3.0", default-features = false }
iroh = { git = 'https://github.com/n0-computer/iroh', branch = "flub/ticket-multiple-addrs" }
kamadak-exif = "0.5"
lettre_email = { git = "https://github.com/deltachat/lettre", branch = "master" }
libc = "0.2"
@@ -95,6 +102,7 @@ log = "0.4"
pretty_env_logger = "0.4"
proptest = { version = "1", default-features = false, features = ["std"] }
tempfile = "3"
testdir = "0.7.2"
tokio = { version = "1", features = ["parking_lot", "rt-multi-thread", "macros"] }
[workspace]
+1 -1
View File
@@ -17,7 +17,7 @@ crate-type = ["cdylib", "staticlib"]
deltachat = { path = "../", default-features = false }
deltachat-jsonrpc = { path = "../deltachat-jsonrpc", optional = true }
libc = "0.2"
human-panic = "1"
human-panic = { version = "1", default-features = false }
num-traits = "0.2"
serde_json = "1.0"
tokio = { version = "1", features = ["rt-multi-thread"] }
+125 -3
View File
@@ -24,6 +24,7 @@ typedef struct _dc_provider dc_provider_t;
typedef struct _dc_event dc_event_t;
typedef struct _dc_event_emitter dc_event_emitter_t;
typedef struct _dc_jsonrpc_instance dc_jsonrpc_instance_t;
typedef struct _dc_backup_provider dc_backup_provider_t;
// Alias for backwards compatibility, use dc_event_emitter_t instead.
typedef struct _dc_event_emitter dc_accounts_event_emitter_t;
@@ -2100,8 +2101,7 @@ dc_contact_t* dc_get_contact (dc_context_t* context, uint32_t co
/**
* Import/export things.
* During backup import/export IO must not be started,
* if needed stop IO using dc_accounts_stop_io() or dc_stop_io() first.
*
* What to do is defined by the _what_ parameter which may be one of the following:
*
* - **DC_IMEX_EXPORT_BACKUP** (11) - Export a backup to the directory given as `param1`
@@ -2295,6 +2295,7 @@ void dc_stop_ongoing_process (dc_context_t* context);
#define DC_QR_FPR_MISMATCH 220 // id=contact
#define DC_QR_FPR_WITHOUT_ADDR 230 // test1=formatted fingerprint
#define DC_QR_ACCOUNT 250 // text1=domain
#define DC_QR_BACKUP 251
#define DC_QR_WEBRTC_INSTANCE 260 // text1=domain, text2=instance pattern
#define DC_QR_ADDR 320 // id=contact
#define DC_QR_TEXT 330 // text1=text
@@ -2320,7 +2321,7 @@ void dc_stop_ongoing_process (dc_context_t* context);
* ask whether to verify the contact;
* if so, start the protocol with dc_join_securejoin().
*
* - DC_QR_ASK_VERIFYGROUP withdc_lot_t::text1=Group name:
* - DC_QR_ASK_VERIFYGROUP with dc_lot_t::text1=Group name:
* ask whether to join the group;
* if so, start the protocol with dc_join_securejoin().
*
@@ -2340,6 +2341,10 @@ void dc_stop_ongoing_process (dc_context_t* context);
* ask the user if they want to create an account on the given domain,
* if so, call dc_set_config_from_qr() and then dc_configure().
*
* - DC_QR_BACKUP:
* ask the user if they want to set up a new device.
* If so, pass the qr-code to dc_receive_backup().
*
* - DC_QR_WEBRTC_INSTANCE with dc_lot_t::text1=domain:
* ask the user if they want to use the given service for video chats;
* if so, call dc_set_config_from_qr().
@@ -2630,6 +2635,123 @@ char* dc_get_last_error (dc_context_t* context);
void dc_str_unref (char* str);
/**
* @class dc_backup_provider_t
*
* Set up another device.
*/
/**
* Creates an object for sending a backup to another device.
*
* Before calling this function IO must be stopped using dc_accounts_stop_io()
* or dc_stop_io() so that no changes to the blobs or database are happening.
* IO should only be restarted once dc_backup_provider_wait() has returned.
*
* The backup is sent to through a peer-to-peer channel which is bootstrapped
* by a QR-code. The backup contains the entire state of the account
* including credentials. This can be used to setup a new device.
*
* This is a blocking call as some preparations are made like e.g. exporting
* the database. Once this function returns, the backup is being offered to
* remote devices. To wait until one device received the backup, use
* dc_backup_provider_wait(). Alternatively abort the operation using
* dc_stop_ongoing_process().
*
* During execution of the job #DC_EVENT_IMEX_PROGRESS is sent out to indicate
* state and progress.
*
* @memberof dc_backup_provider_t
* @param context The context.
* @return Opaque object for sending the backup.
* On errors, NULL is returned and dc_get_last_error() returns an error that
* should be shown to the user.
*/
dc_backup_provider_t* dc_backup_provider_new (dc_context_t* context);
/**
* Returns the QR code text that will offer the backup to other devices.
*
* The QR code contains a ticket which will validate the backup and provide
* authentication for both the provider and the recipient.
*
* The scanning device should call the scanned text to dc_check_qr(). If
* dc_check_qr() returns DC_QR_BACKUP, the backup transfer can be started using
* dc_get_backup().
*
* @memberof dc_backup_provider_t
* @param backup_provider The backup provider object as created by
* dc_backup_provider_new().
* @return The text that should be put in the QR code.
* On errors an empty string is returned, NULL is never returned.
* the returned string must be released using dc_str_unref() after usage.
*/
char* dc_backup_provider_get_qr (const dc_backup_provider_t* backup_provider);
/**
* Returns the QR code SVG image that will offer the backup to other devices.
*
* This works like dc_backup_provider_qr() but returns the text of a rendered
* SVG image containing the QR code.
*
* @memberof dc_backup_provider_t
* @param backup_provider The backup provider object as created by
* dc_backup_provider_new().
* @return The QR code rendered as SVG.
* On errors an empty string is returned, NULL is never returned.
* the returned string must be released using dc_str_unref() after usage.
*/
char* dc_backup_provider_get_qr_svg (const dc_backup_provider_t* backup_provider);
/**
* Waits for the sending to finish.
*
* This is a blocking call and should only be called once. Once this function
* returns IO can be started again using dc_accounts_start_io() or
* dc_start_io().
*
* @memberof dc_backup_provider_t
* @param backup_provider The backup provider object as created by
* dc_backup_provider_new(). If NULL is given nothing is done.
*/
void dc_backup_provider_wait (dc_backup_provider_t* backup_provider);
/**
* Frees a dc_backup_provider_t object.
*
* @memberof dc_backup_provider_t
* @param backup_provider The backup provider object as created by
* dc_backup_provider_new().
*/
void dc_backup_provider_unref (dc_backup_provider_t* backup_provider);
/**
* Gets a backup offered by a dc_backup_provider_t object on another device.
*
* This function is called on a device that scanned the QR code offered by
* dc_backup_sender_qr() or dc_backup_sender_qr_svg(). Typically this is a
* different device than that which provides the backup.
*
* This call will block while the backup is being transferred and only
* complete on success or failure. Use dc_stop_ongoing_process() to abort it
* early.
*
* During execution of the job #DC_EVENT_IMEX_PROGRESS is sent out to indicate
* state and progress. The process is finished when the event emits either 0
* or 1000, 0 means it failed and 1000 means it succeeded. These events are
* for showing progress and informational only, success and failure is also
* shown in the return code of this function.
*
* @memberof dc_context_t
* @param context The context.
* @param qr The qr code text, dc_check_qr() must have returned DC_QR_BACKUP
* on this text.
* @return 0=failure, 1=success.
*/
int dc_receive_backup (dc_context_t* context, const char* qr);
/**
* @class dc_accounts_t
*
+104 -1
View File
@@ -28,9 +28,10 @@ use deltachat::constants::DC_MSG_ID_LAST_SPECIAL;
use deltachat::contact::{Contact, ContactId, Origin};
use deltachat::context::Context;
use deltachat::ephemeral::Timer as EphemeralTimer;
use deltachat::imex::BackupProvider;
use deltachat::key::DcKey;
use deltachat::message::MsgId;
use deltachat::qr_code_generator::get_securejoin_qr_svg;
use deltachat::qr_code_generator::{generate_backup_qr, get_securejoin_qr_svg};
use deltachat::reaction::{get_msg_reactions, send_reaction, Reactions};
use deltachat::stock_str::StockMessage;
use deltachat::stock_str::StockStrings;
@@ -4142,6 +4143,108 @@ pub unsafe extern "C" fn dc_str_unref(s: *mut libc::c_char) {
libc::free(s as *mut _)
}
pub struct BackupProviderWrapper {
context: *const dc_context_t,
provider: BackupProvider,
}
pub type dc_backup_provider_t = BackupProviderWrapper;
#[no_mangle]
pub unsafe extern "C" fn dc_backup_provider_new(
context: *mut dc_context_t,
) -> *mut dc_backup_provider_t {
if context.is_null() {
eprintln!("ignoring careless call to dc_backup_provider_new()");
return ptr::null_mut();
}
let ctx = &*context;
block_on(BackupProvider::prepare(ctx))
.map(|provider| BackupProviderWrapper {
context: ctx,
provider,
})
.map(|ffi_provider| Box::into_raw(Box::new(ffi_provider)))
.log_err(ctx, "BackupProvider failed")
.unwrap_or(ptr::null_mut())
}
#[no_mangle]
pub unsafe extern "C" fn dc_backup_provider_get_qr(
provider: *const dc_backup_provider_t,
) -> *mut libc::c_char {
if provider.is_null() {
eprintln!("ignoring careless call to dc_backup_provider_qr");
return "".strdup();
}
let ffi_provider = &*provider;
let ctx = &*ffi_provider.context;
deltachat::qr::format_backup(&ffi_provider.provider.qr())
.log_err(ctx, "BackupProvider get_qr failed")
.unwrap_or_default()
.strdup()
}
#[no_mangle]
pub unsafe extern "C" fn dc_backup_provider_get_qr_svg(
provider: *const dc_backup_provider_t,
) -> *mut libc::c_char {
if provider.is_null() {
eprintln!("ignoring careless call to dc_backup_provider_qr_svg()");
return "".strdup();
}
let ffi_provider = &*provider;
let ctx = &*ffi_provider.context;
let provider = &ffi_provider.provider;
block_on(generate_backup_qr(ctx, &provider.qr()))
.log_err(ctx, "BackupProvider get_qr_svg failed")
.unwrap_or_default()
.strdup()
}
#[no_mangle]
pub unsafe extern "C" fn dc_backup_provider_wait(provider: *mut dc_backup_provider_t) {
if provider.is_null() {
eprintln!("ignoring careless call to dc_backup_provider_wait()");
return;
}
let ffi_provider = &mut *provider;
let ctx = &*ffi_provider.context;
let provider = &mut ffi_provider.provider;
block_on(provider)
.log_err(ctx, "Failed to await BackupProvider")
.ok();
}
#[no_mangle]
pub unsafe extern "C" fn dc_backup_provider_unref(provider: *mut dc_backup_provider_t) {
drop(Box::from_raw(provider));
}
#[no_mangle]
pub unsafe extern "C" fn dc_receive_backup(
context: *mut dc_context_t,
qr: *const libc::c_char,
) -> libc::c_int {
if context.is_null() {
eprintln!("ignoring careless call to dc_receive_backup()");
return 0;
}
let ctx = &*context;
let qr_text = to_string_lossy(qr);
let qr = match block_on(qr::check_qr(ctx, &qr_text)).log_err(ctx, "Invalid QR code") {
Ok(qr) => qr,
Err(_) => return 0,
};
spawn(async move {
imex::get_backup(ctx, qr)
.await
.log_err(ctx, "Get backup failed")
.ok();
});
1
}
trait ResultExt<T, E> {
/// Like `log_err()`, but:
/// - returns the default value instead of an Err value.
+7
View File
@@ -14,6 +14,8 @@ use crate::summary::{Summary, SummaryPrefix};
/// eg. by chatlist.get_summary() or dc_msg_get_summary().
///
/// *Lot* is used in the meaning *heap* here.
// The QR code grew too large. So be it.
#[allow(clippy::large_enum_variant)]
#[derive(Debug)]
pub enum Lot {
Summary(Summary),
@@ -47,6 +49,7 @@ impl Lot {
Qr::FprMismatch { .. } => None,
Qr::FprWithoutAddr { fingerprint, .. } => Some(fingerprint),
Qr::Account { domain } => Some(domain),
Qr::Backup { .. } => None,
Qr::WebrtcInstance { domain, .. } => Some(domain),
Qr::Addr { draft, .. } => draft.as_deref(),
Qr::Url { url } => Some(url),
@@ -98,6 +101,7 @@ impl Lot {
Qr::FprMismatch { .. } => LotState::QrFprMismatch,
Qr::FprWithoutAddr { .. } => LotState::QrFprWithoutAddr,
Qr::Account { .. } => LotState::QrAccount,
Qr::Backup { .. } => LotState::QrBackup,
Qr::WebrtcInstance { .. } => LotState::QrWebrtcInstance,
Qr::Addr { .. } => LotState::QrAddr,
Qr::Url { .. } => LotState::QrUrl,
@@ -122,6 +126,7 @@ impl Lot {
Qr::FprMismatch { contact_id } => contact_id.unwrap_or_default().to_u32(),
Qr::FprWithoutAddr { .. } => Default::default(),
Qr::Account { .. } => Default::default(),
Qr::Backup { .. } => Default::default(),
Qr::WebrtcInstance { .. } => Default::default(),
Qr::Addr { contact_id, .. } => contact_id.to_u32(),
Qr::Url { .. } => Default::default(),
@@ -170,6 +175,8 @@ pub enum LotState {
/// text1=domain
QrAccount = 250,
QrBackup = 251,
/// text1=domain, text2=instance pattern
QrWebrtcInstance = 260,
+1 -1
View File
@@ -29,7 +29,7 @@ walkdir = "2.3.2"
base64 = "0.21"
# optional dependencies
axum = { version = "0.6.6", optional = true, features = ["ws"] }
axum = { version = "0.6.11", optional = true, features = ["ws"] }
env_logger = { version = "0.10.0", optional = true }
[dev-dependencies]
+66 -6
View File
@@ -22,7 +22,7 @@ use deltachat::{
},
provider::get_provider_info,
qr,
qr_code_generator::get_securejoin_qr_svg,
qr_code_generator::{generate_backup_qr, get_securejoin_qr_svg},
reaction::send_reaction,
securejoin,
stock_str::StockMessage,
@@ -1325,16 +1325,13 @@ impl CommandApi {
passphrase: Option<String>,
) -> Result<()> {
let ctx = self.get_context(account_id).await?;
ctx.stop_io().await;
let result = imex::imex(
imex::imex(
&ctx,
imex::ImexMode::ExportBackup,
destination.as_ref(),
passphrase,
)
.await;
ctx.start_io().await;
result
.await
}
async fn import_backup(
@@ -1353,6 +1350,69 @@ impl CommandApi {
.await
}
/// Offers a backup for remote devices to retrieve.
///
/// Can be cancelled by stopping the ongoing process. Success or failure can be tracked
/// via the `ImexProgress` event which should either reach `1000` for success or `0` for
/// failure.
///
/// This **stops IO** while it is running.
///
/// Returns once a remote device has retrieved the backup.
async fn provide_backup(&self, account_id: u32) -> Result<()> {
let ctx = self.get_context(account_id).await?;
ctx.stop_io().await;
let provider = match imex::BackupProvider::prepare(&ctx).await {
Ok(provider) => provider,
Err(err) => {
ctx.start_io().await;
return Err(err);
}
};
let res = provider.await;
ctx.start_io().await;
res
}
/// Returns the text of the QR code for the running [`CommandApi::provide_backup`].
///
/// This QR code text can be used in [`CommandApi::get_backup`] on a second device to
/// retrieve the backup and setup this second device.
async fn get_backup_qr(&self, account_id: u32) -> Result<String> {
let ctx = self.get_context(account_id).await?;
let qr = ctx
.backup_export_qr()
.ok_or(anyhow!("no backup being exported"))?;
qr::format_backup(&qr)
}
/// Returns the rendered QR code for the running [`CommandApi::provide_backup`].
///
/// This QR code can be used in [`CommandApi::get_backup`] on a second device to
/// retrieve the backup and setup this second device.
///
/// Returns the QR code rendered as an SVG image.
async fn get_backup_qr_svg(&self, account_id: u32) -> Result<String> {
let ctx = self.get_context(account_id).await?;
let qr = ctx
.backup_export_qr()
.ok_or(anyhow!("no backup being exported"))?;
generate_backup_qr(&ctx, &qr).await
}
/// Gets a backup from a remote provider.
///
/// This retrieves the backup from a remote device over the network and imports it into
/// the current device.
///
/// Can be cancelled by stopping the ongoing process.
async fn get_backup(&self, account_id: u32, qr_text: String) -> Result<()> {
let ctx = self.get_context(account_id).await?;
let qr = qr::check_qr(&ctx, &qr_text).await?;
imex::get_backup(&ctx, qr).await?;
Ok(())
}
// ---------------------------------------------
// connectivity
// ---------------------------------------------
+6
View File
@@ -32,6 +32,9 @@ pub enum QrObject {
Account {
domain: String,
},
Backup {
ticket: String,
},
WebrtcInstance {
domain: String,
instance_pattern: String,
@@ -126,6 +129,9 @@ impl From<Qr> for QrObject {
}
Qr::FprWithoutAddr { fingerprint } => QrObject::FprWithoutAddr { fingerprint },
Qr::Account { domain } => QrObject::Account { domain },
Qr::Backup { ticket } => QrObject::Backup {
ticket: ticket.to_string(),
},
Qr::WebrtcInstance {
domain,
instance_pattern,
+13
View File
@@ -336,6 +336,8 @@ pub async fn cmdline(context: Context, line: &str, chat_id: &mut ChatId) -> Resu
has-backup\n\
export-backup\n\
import-backup <backup-file>\n\
send-backup\n\
receive-backup <qr>\n\
export-keys\n\
import-keys\n\
export-setup\n\
@@ -486,6 +488,17 @@ pub async fn cmdline(context: Context, line: &str, chat_id: &mut ChatId) -> Resu
)
.await?;
}
"send-backup" => {
let provider = BackupProvider::prepare(&context).await?;
let qr = provider.qr();
println!("QR code: {}", format_backup(&qr)?);
provider.await?;
}
"receive-backup" => {
ensure!(!arg1.is_empty(), "Argument <qr> is missing.");
let qr = check_qr(&context, arg1).await?;
deltachat::imex::get_backup(&context, qr).await?;
}
"export-keys" => {
let dir = dirs::home_dir().unwrap_or_default();
imex(&context, ImexMode::ExportSelfKeys, dir.as_ref(), None).await?;
+3 -1
View File
@@ -152,13 +152,15 @@ impl Completer for DcHelper {
}
}
const IMEX_COMMANDS: [&str; 12] = [
const IMEX_COMMANDS: [&str; 14] = [
"initiate-key-transfer",
"get-setupcodebegin",
"continue-key-transfer",
"has-backup",
"export-backup",
"import-backup",
"send-backup",
"receive-backup",
"export-keys",
"import-keys",
"export-setup",
+50 -23
View File
@@ -12,27 +12,41 @@ ignore = [
# becoming empty. Adding versions forces us to revisit this at least
# when upgrading.
skip = [
{ name = "windows-sys", version = "<0.45" },
{ name = "wasi", version = "<0.11" },
{ name = "version_check", version = "<0.9" },
{ name = "uuid", version = "<1.3" },
{ name = "sha2", version = "<0.10" },
{ name = "rand_core", version = "<0.6" },
{ name = "rand_chacha", version = "<0.3" },
{ name = "rand", version = "<0.8" },
{ name = "nom", version = "<7.1" },
{ name = "idna", version = "<0.3" },
{ name = "humantime", version = "<2.1" },
{ name = "hermit-abi", version = "<0.3" },
{ name = "getrandom", version = "<0.2" },
{ name = "quick-error", version = "<2.0" },
{ name = "env_logger", version = "<0.10" },
{ name = "digest", version = "<0.10" },
{ name = "darling_macro", version = "<0.14" },
{ name = "darling_core", version = "<0.14" },
{ name = "darling", version = "<0.14" },
{ name = "block-buffer", version = "<0.10" },
{ name = "base64", version = "<0.21" },
{ name = "block-buffer", version = "<0.10" },
{ name = "darling", version = "<0.14" },
{ name = "darling_core", version = "<0.14" },
{ name = "darling_macro", version = "<0.14" },
{ name = "digest", version = "<0.10" },
{ name = "env_logger", version = "<0.10" },
{ name = "getrandom", version = "<0.2" },
{ name = "hermit-abi", version = "<0.3" },
{ name = "humantime", version = "<2.1" },
{ name = "idna", version = "<0.3" },
{ name = "nom", version = "<7.1" },
{ name = "quick-error", version = "<2.0" },
{ name = "rand", version = "<0.8" },
{ name = "rand_chacha", version = "<0.3" },
{ name = "rand_core", version = "<0.6" },
{ name = "sha2", version = "<0.10" },
{ name = "time", version = "<0.3" },
{ name = "version_check", version = "<0.9" },
{ name = "wasi", version = "<0.11" },
{ name = "windows-sys", version = "<0.45" },
{ name = "windows_x86_64_msvc", version = "<0.42" },
{ name = "windows_x86_64_gnu", version = "<0.42" },
{ name = "windows_i686_msvc", version = "<0.42" },
{ name = "windows_i686_gnu", version = "<0.42" },
{ name = "windows_aarch64_msvc", version = "<0.42" },
{ name = "unicode-xid", version = "<0.2.4" },
{ name = "syn", version = "<1.0" },
{ name = "quote", version = "<1.0" },
{ name = "proc-macro2", version = "<1.0" },
{ name = "portable-atomic", version = "<1.0" },
{ name = "spin", version = "<0.9.6" },
{ name = "convert_case", version = "0.4.0" },
{ name = "clap_lex", version = "0.2.4" },
{ name = "clap", version = "3.2.23" },
]
@@ -42,11 +56,21 @@ allow = [
"Apache-2.0",
"BSD-2-Clause",
"BSD-3-Clause",
"CC0-1.0",
"MIT",
"BSL-1.0", # Boost Software License 1.0
"Unicode-DFS-2016",
"CC0-1.0",
"ISC",
"MIT",
"MPL-2.0",
"OpenSSL",
"Unicode-DFS-2016",
"Zlib",
]
[[licenses.clarify]]
name = "ring"
expression = "MIT AND ISC AND OpenSSL"
license-files = [
{ path = "LICENSE", hash = 0xbd0eed23 },
]
[sources.allow-org]
@@ -54,4 +78,7 @@ allow = [
github = [
"async-email",
"deltachat",
"n0-computer",
"quinn-rs",
"dignifiedquire",
]
+25 -55
View File
@@ -111,7 +111,7 @@ version = "0.6.0"
source = "git+https://github.com/async-email/async-imap?branch=master#85ff7a3d9d71a3715354fabf2fc1a8d047b5710e"
dependencies = [
"async-channel",
"async-native-tls",
"async-native-tls 0.4.0",
"base64 0.13.1",
"byte-pool",
"chrono",
@@ -139,6 +139,18 @@ dependencies = [
"url",
]
[[package]]
name = "async-native-tls"
version = "0.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9343dc5acf07e79ff82d0c37899f079db3534d99f189a1837c8e549c99405bec"
dependencies = [
"native-tls",
"thiserror",
"tokio",
"url",
]
[[package]]
name = "async-smtp"
version = "0.8.0"
@@ -696,12 +708,12 @@ checksum = "23d8666cb01533c39dde32bcbab8e227b4ed6679b2c925eba05feabea39508fb"
[[package]]
name = "deltachat"
version = "1.107.0"
version = "1.111.0"
dependencies = [
"anyhow",
"async-channel",
"async-imap",
"async-native-tls",
"async-native-tls 0.5.0",
"async-smtp",
"async_zip",
"backtrace",
@@ -723,17 +735,15 @@ dependencies = [
"lettre_email",
"libc",
"mailparse 0.14.0",
"native-tls",
"num-derive",
"num-traits",
"num_cpus",
"once_cell",
"parking_lot",
"percent-encoding",
"pgp",
"qrcodegen",
"quick-xml",
"r2d2",
"r2d2_sqlite",
"rand 0.8.5",
"ratelimit",
"regex",
@@ -1284,15 +1294,6 @@ dependencies = [
"tracing",
]
[[package]]
name = "hashbrown"
version = "0.11.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ab5ef0d4909ef3724cc8cce6ccc8572c5c817592e9285f5464f8e86f8bd3726e"
dependencies = [
"ahash",
]
[[package]]
name = "hashbrown"
version = "0.12.3"
@@ -1304,11 +1305,11 @@ dependencies = [
[[package]]
name = "hashlink"
version = "0.7.0"
version = "0.8.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7249a3129cbc1ffccd74857f81464a323a152173cdb134e0fd81bc803b29facf"
checksum = "69fe1fcf8b4278d860ad0548329f892a3631fb63f82574df68275f34cdbe0ffa"
dependencies = [
"hashbrown 0.11.2",
"hashbrown",
]
[[package]]
@@ -1506,7 +1507,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1885e79c1fc4b10f0e172c475f458b7f7b93061064d98c3293e98c5ba0c8b399"
dependencies = [
"autocfg",
"hashbrown 0.12.3",
"hashbrown",
]
[[package]]
@@ -1631,9 +1632,9 @@ checksum = "348108ab3fba42ec82ff6e9564fc4ca0247bdccdc68dd8af9764bbc79c3c8ffb"
[[package]]
name = "libsqlite3-sys"
version = "0.24.2"
version = "0.25.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "898745e570c7d0453cc1fbc4a701eb6c662ed54e8fec8b7d14be137ebeeb9d14"
checksum = "29f835d03d717946d28b1d1ed632eb6f0e24a299388ee623d0c23118d3e8a7fa"
dependencies = [
"cc",
"openssl-sys",
@@ -2241,27 +2242,6 @@ version = "0.4.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "20f14e071918cbeefc5edc986a7aa92c425dae244e003a35e1cdddb5ca39b5cb"
[[package]]
name = "r2d2"
version = "0.8.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "51de85fb3fb6524929c8a2eb85e6b6d363de4e8c48f9e2c2eac4944abc181c93"
dependencies = [
"log",
"parking_lot",
"scheduled-thread-pool",
]
[[package]]
name = "r2d2_sqlite"
version = "0.20.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6fdc8e4da70586127893be32b7adf21326a4c6b1aba907611edf467d13ffe895"
dependencies = [
"r2d2",
"rusqlite",
]
[[package]]
name = "rand"
version = "0.7.3"
@@ -2451,16 +2431,15 @@ dependencies = [
[[package]]
name = "rusqlite"
version = "0.27.0"
version = "0.28.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "85127183a999f7db96d1a976a309eebbfb6ea3b0b400ddd8340190129de6eb7a"
checksum = "01e213bc3ecb39ac32e81e51ebe31fd888a940515173e3a18a35f8c6e896422a"
dependencies = [
"bitflags",
"fallible-iterator",
"fallible-streaming-iterator",
"hashlink",
"libsqlite3-sys",
"memchr",
"smallvec",
]
@@ -2514,15 +2493,6 @@ dependencies = [
"windows-sys 0.36.1",
]
[[package]]
name = "scheduled-thread-pool"
version = "0.2.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "977a7519bff143a44f842fd07e80ad1329295bd71686457f18e496736f4bf9bf"
dependencies = [
"parking_lot",
]
[[package]]
name = "scopeguard"
version = "1.1.0"
@@ -3126,7 +3096,7 @@ version = "0.1.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c5faade31a542b8b35855fff6e8def199853b2da8da256da52f52f1316ee3137"
dependencies = [
"hashbrown 0.12.3",
"hashbrown",
"regex",
]
+2 -2
View File
@@ -271,14 +271,14 @@ impl Accounts {
/// Notifies all accounts that the network may have become available.
pub async fn maybe_network(&self) {
for account in self.accounts.values() {
account.maybe_network().await;
account.scheduler.maybe_network().await;
}
}
/// Notifies all accounts that the network connection may have been lost.
pub async fn maybe_network_lost(&self) {
for account in self.accounts.values() {
account.maybe_network_lost().await;
account.scheduler.maybe_network_lost(account).await;
}
}
+86 -2
View File
@@ -4,13 +4,16 @@ use core::cmp::max;
use std::ffi::OsStr;
use std::fmt;
use std::io::Cursor;
use std::iter::FusedIterator;
use std::path::{Path, PathBuf};
use anyhow::{format_err, Context as _, Result};
use futures::StreamExt;
use image::{DynamicImage, ImageFormat};
use num_traits::FromPrimitive;
use tokio::io::AsyncWriteExt;
use tokio::{fs, io};
use tokio_stream::wrappers::ReadDirStream;
use crate::config::Config;
use crate::constants::{
@@ -160,9 +163,9 @@ impl<'a> BlobObject<'a> {
pub fn from_path(context: &'a Context, path: &Path) -> Result<BlobObject<'a>> {
let rel_path = path
.strip_prefix(context.get_blobdir())
.context("wrong blobdir")?;
.with_context(|| format!("wrong blobdir: {}", path.display()))?;
if !BlobObject::is_acceptible_blob_name(rel_path) {
return Err(format_err!("wrong name"));
return Err(format_err!("bad blob name: {}", rel_path.display()));
}
let name = rel_path.to_str().context("wrong name")?;
BlobObject::from_name(context, name.to_string())
@@ -468,6 +471,87 @@ impl<'a> fmt::Display for BlobObject<'a> {
}
}
/// All files in the blobdir.
///
/// This exists so we can have a [`BlobDirIter`] which needs something to own the data of
/// it's `&Path`. Use [`BlobDirContents::iter`] to create the iterator.
///
/// Additionally pre-allocating this means we get a length for progress report.
pub(crate) struct BlobDirContents<'a> {
inner: Vec<PathBuf>,
context: &'a Context,
}
impl<'a> BlobDirContents<'a> {
pub(crate) async fn new(context: &'a Context) -> Result<BlobDirContents<'a>> {
let readdir = fs::read_dir(context.get_blobdir()).await?;
let inner = ReadDirStream::new(readdir)
.filter_map(|entry| async move {
match entry {
Ok(entry) => Some(entry),
Err(err) => {
error!(context, "Failed to read blob file: {err}");
None
}
}
})
.filter_map(|entry| async move {
match entry.file_type().await.ok()?.is_file() {
true => Some(entry.path()),
false => {
warn!(
context,
"Export: Found blob dir entry {} that is not a file, ignoring",
entry.path().display()
);
None
}
}
})
.collect()
.await;
Ok(Self { inner, context })
}
pub(crate) fn iter(&self) -> BlobDirIter<'_> {
BlobDirIter::new(self.context, self.inner.iter())
}
pub(crate) fn len(&self) -> usize {
self.inner.len()
}
}
/// A iterator over all the [`BlobObject`]s in the blobdir.
pub(crate) struct BlobDirIter<'a> {
iter: std::slice::Iter<'a, PathBuf>,
context: &'a Context,
}
impl<'a> BlobDirIter<'a> {
fn new(context: &'a Context, iter: std::slice::Iter<'a, PathBuf>) -> BlobDirIter<'a> {
Self { iter, context }
}
}
impl<'a> Iterator for BlobDirIter<'a> {
type Item = BlobObject<'a>;
fn next(&mut self) -> Option<Self::Item> {
for path in self.iter.by_ref() {
// In theory this can error but we'd have corrupted filenames in the blobdir, so
// silently skipping them is fine.
match BlobObject::from_path(self.context, path) {
Ok(blob) => return Some(blob),
Err(err) => warn!(self.context, "{err}"),
}
}
None
}
}
impl FusedIterator for BlobDirIter<'_> {}
fn encode_img(img: &DynamicImage, encoded: &mut Vec<u8>) -> anyhow::Result<()> {
encoded.clear();
let mut buf = Cursor::new(encoded);
+17 -5
View File
@@ -639,7 +639,10 @@ impl ChatId {
context.emit_msgs_changed_without_ids();
context.set_config(Config::LastHousekeeping, None).await?;
context.interrupt_inbox(InterruptInfo::new(false)).await;
context
.scheduler
.interrupt_inbox(InterruptInfo::new(false))
.await;
if chat.is_self_talk() {
let mut msg = Message::new(Viewtype::Text);
@@ -1667,7 +1670,7 @@ impl Chat {
maybe_set_logging_xdc(context, msg, self.id).await?;
}
context.interrupt_ephemeral_task().await;
context.scheduler.interrupt_ephemeral_task().await;
Ok(msg.id)
}
}
@@ -2201,7 +2204,10 @@ async fn send_msg_inner(context: &Context, chat_id: ChatId, msg: &mut Message) -
context.emit_event(EventType::LocationChanged(Some(ContactId::SELF)));
}
context.interrupt_smtp(InterruptInfo::new(false)).await;
context
.scheduler
.interrupt_smtp(InterruptInfo::new(false))
.await;
}
Ok(msg.id)
@@ -3433,7 +3439,10 @@ pub async fn forward_msgs(context: &Context, msg_ids: &[MsgId], chat_id: ChatId)
.await?;
curr_timestamp += 1;
if create_send_msg_job(context, new_msg_id).await?.is_some() {
context.interrupt_smtp(InterruptInfo::new(false)).await;
context
.scheduler
.interrupt_smtp(InterruptInfo::new(false))
.await;
}
}
created_chats.push(chat_id);
@@ -3488,7 +3497,10 @@ pub async fn resend_msgs(context: &Context, msg_ids: &[MsgId]) -> Result<()> {
msg_id: msg.id,
});
if create_send_msg_job(context, msg.id).await?.is_some() {
context.interrupt_smtp(InterruptInfo::new(false)).await;
context
.scheduler
.interrupt_smtp(InterruptInfo::new(false))
.await;
}
}
}
+1 -1
View File
@@ -443,7 +443,7 @@ impl Context {
Config::DeleteDeviceAfter => {
let ret = self.sql.set_raw_config(key.as_ref(), value).await;
// Interrupt ephemeral loop to delete old messages immediately.
self.interrupt_ephemeral_task().await;
self.scheduler.interrupt_ephemeral_task().await;
ret?
}
Config::Displayname => {
+4 -2
View File
@@ -59,7 +59,7 @@ impl Context {
/// Configures this account with the currently set parameters.
pub async fn configure(&self) -> Result<()> {
ensure!(
self.scheduler.read().await.is_none(),
!self.scheduler.is_running().await,
"cannot configure, already running"
);
ensure!(
@@ -469,7 +469,9 @@ async fn configure(ctx: &Context, param: &mut LoginParam) -> Result<()> {
ctx.set_config_bool(Config::FetchedExistingMsgs, false)
.await?;
ctx.interrupt_inbox(InterruptInfo::new(false)).await;
ctx.scheduler
.interrupt_inbox(InterruptInfo::new(false))
.await;
progress!(ctx, 940);
update_device_chats_handle.await??;
+4 -1
View File
@@ -1466,7 +1466,10 @@ pub(crate) async fn update_last_seen(
> 0
&& timestamp > time() - SEEN_RECENTLY_SECONDS
{
context.interrupt_recently_seen(contact_id, timestamp).await;
context
.scheduler
.interrupt_recently_seen(contact_id, timestamp)
.await;
}
Ok(())
}
+43 -30
View File
@@ -23,8 +23,9 @@ use crate::events::{Event, EventEmitter, EventType, Events};
use crate::key::{DcKey, SignedPublicKey};
use crate::login_param::LoginParam;
use crate::message::{self, MessageState, MsgId};
use crate::qr::Qr;
use crate::quota::QuotaInfo;
use crate::scheduler::Scheduler;
use crate::scheduler::SchedulerState;
use crate::sql::Sql;
use crate::stock_str::StockStrings;
use crate::timesmearing::SmearedTimestamp;
@@ -191,6 +192,10 @@ pub struct InnerContext {
pub(crate) blobdir: PathBuf,
pub(crate) sql: Sql,
pub(crate) smeared_timestamp: SmearedTimestamp,
/// The global "ongoing" process state.
///
/// This is a global mutex-like state for operations which should be modal in the
/// clients.
running_state: RwLock<RunningState>,
/// Mutex to avoid generating the key for the user more than once.
pub(crate) generating_key_mutex: Mutex<()>,
@@ -201,7 +206,7 @@ pub struct InnerContext {
pub(crate) translated_stockstrings: StockStrings,
pub(crate) events: Events,
pub(crate) scheduler: RwLock<Option<Scheduler>>,
pub(crate) scheduler: SchedulerState,
pub(crate) ratelimit: RwLock<Ratelimit>,
/// Recently loaded quota information, if any.
@@ -236,6 +241,14 @@ pub struct InnerContext {
/// If debug logging is enabled, this contains all necessary information
pub(crate) debug_logging: RwLock<Option<DebugLogging>>,
/// QR code for currently running [`BackupProvider`].
///
/// This is only available if a backup export is currently running, it will also be
/// holding the ongoing process while running.
///
/// [`BackupProvider`]: crate::imex::BackupProvider
pub(crate) export_provider: std::sync::Mutex<Option<Qr>>,
}
#[derive(Debug)]
@@ -370,7 +383,7 @@ impl Context {
wrong_pw_warning_mutex: Mutex::new(()),
translated_stockstrings: stockstrings,
events,
scheduler: RwLock::new(None),
scheduler: SchedulerState::new(),
ratelimit: RwLock::new(Ratelimit::new(Duration::new(60, 0), 6.0)), // Allow to send 6 messages immediately, no more than once every 10 seconds.
quota: RwLock::new(None),
quota_update_request: AtomicBool::new(false),
@@ -380,6 +393,7 @@ impl Context {
last_full_folder_scan: Mutex::new(None),
last_error: std::sync::RwLock::new("".to_string()),
debug_logging: RwLock::new(None),
export_provider: std::sync::Mutex::new(None),
};
let ctx = Context {
@@ -395,42 +409,23 @@ impl Context {
warn!(self, "can not start io on a context that is not configured");
return;
}
info!(self, "starting IO");
let mut lock = self.inner.scheduler.write().await;
if lock.is_none() {
match Scheduler::start(self.clone()).await {
Err(err) => error!(self, "Failed to start IO: {:#}", err),
Ok(scheduler) => *lock = Some(scheduler),
}
}
self.scheduler.start(self.clone()).await;
}
/// Stops the IO scheduler.
pub async fn stop_io(&self) {
// Sending an event wakes up event pollers (get_next_event)
// so the caller of stop_io() can arrange for proper termination.
// For this, the caller needs to instruct the event poller
// to terminate on receiving the next event and then call stop_io()
// which will emit the below event(s)
info!(self, "stopping IO");
if let Some(debug_logging) = self.debug_logging.read().await.as_ref() {
debug_logging.loop_handle.abort();
}
if let Some(scheduler) = self.inner.scheduler.write().await.take() {
scheduler.stop(self).await;
}
self.scheduler.stop(self).await;
}
/// Restarts the IO scheduler if it was running before
/// when it is not running this is an no-op
pub async fn restart_io_if_running(&self) {
info!(self, "restarting IO");
let is_running = { self.inner.scheduler.read().await.is_some() };
if is_running {
self.stop_io().await;
self.start_io().await;
}
self.scheduler.restart(self).await;
}
/// Indicate that the network likely has come back.
pub async fn maybe_network(&self) {
self.scheduler.maybe_network().await;
}
/// Returns a reference to the underlying SQL instance.
@@ -521,6 +516,13 @@ impl Context {
// Ongoing process allocation/free/check
/// Tries to acquire the global UI "ongoing" mutex.
///
/// This is for modal operations during which no other user actions are allowed. Only
/// one such operation is allowed at any given time.
///
/// The return value is a cancel token, which will release the ongoing mutex when
/// dropped.
pub(crate) async fn alloc_ongoing(&self) -> Result<Receiver<()>> {
let mut s = self.running_state.write().await;
ensure!(
@@ -566,6 +568,17 @@ impl Context {
}
}
/// Returns the QR-code of the currently running [`BackupProvider`].
///
/// [`BackupProvider`]: crate::imex::BackupProvider
pub fn backup_export_qr(&self) -> Option<Qr> {
self.export_provider
.lock()
.expect("poisoned lock")
.as_ref()
.cloned()
}
/*******************************************************************************
* UI chat/message related API
******************************************************************************/
+2 -2
View File
@@ -317,7 +317,7 @@ impl MsgId {
paramsv![ephemeral_timestamp, ephemeral_timestamp, self],
)
.await?;
context.interrupt_ephemeral_task().await;
context.scheduler.interrupt_ephemeral_task().await;
}
Ok(())
}
@@ -345,7 +345,7 @@ pub(crate) async fn start_ephemeral_timers_msgids(
)
.await?;
if count > 0 {
context.interrupt_ephemeral_task().await;
context.scheduler.interrupt_ephemeral_task().await;
}
Ok(())
}
+5 -2
View File
@@ -475,7 +475,7 @@ impl Imap {
// Note that the `Config::DeleteDeviceAfter` timer starts as soon as the messages are
// fetched while the per-chat ephemeral timers start as soon as the messages are marked
// as noticed.
context.interrupt_ephemeral_task().await;
context.scheduler.interrupt_ephemeral_task().await;
}
let session = self
@@ -2224,7 +2224,10 @@ pub(crate) async fn markseen_on_imap_table(context: &Context, message_id: &str)
paramsv![message_id],
)
.await?;
context.interrupt_inbox(InterruptInfo::new(false)).await;
context
.scheduler
.interrupt_inbox(InterruptInfo::new(false))
.await;
Ok(())
}
+71 -68
View File
@@ -5,18 +5,19 @@ use std::ffi::OsStr;
use std::path::{Path, PathBuf};
use ::pgp::types::KeyTrait;
use anyhow::{bail, ensure, format_err, Context as _, Error, Result};
use anyhow::{bail, ensure, format_err, Context as _, Result};
use futures::StreamExt;
use futures_lite::FutureExt;
use rand::{thread_rng, Rng};
use tokio::fs::{self, File};
use tokio_tar::Archive;
use crate::blob::BlobObject;
use crate::blob::{BlobDirContents, BlobObject};
use crate::chat::{self, delete_and_reset_all_device_msgs, ChatId};
use crate::config::Config;
use crate::contact::ContactId;
use crate::context::Context;
use crate::e2ee;
use crate::events::EventType;
use crate::key::{self, DcKey, DcSecretKey, SignedPublicKey, SignedSecretKey};
use crate::log::LogExt;
@@ -30,7 +31,10 @@ use crate::tools::{
create_folder, delete_file, get_filesuffix_lc, open_file_std, read_file, time, write_file,
EmailAddress,
};
use crate::{e2ee, tools};
mod transfer;
pub use transfer::{get_backup, BackupProvider};
// Name of the database file in the backup.
const DBFILE_BACKUP_NAME: &str = "dc_database_backup.sqlite";
@@ -86,13 +90,17 @@ pub async fn imex(
) -> Result<()> {
let cancel = context.alloc_ongoing().await?;
let res = imex_inner(context, what, path, passphrase)
.race(async {
cancel.recv().await.ok();
Err(format_err!("canceled"))
})
.await;
let res = {
let mut guard = context.scheduler.pause(context).await;
let res = imex_inner(context, what, path, passphrase)
.race(async {
cancel.recv().await.ok();
Err(format_err!("canceled"))
})
.await;
guard.resume().await;
res
};
context.free_ongoing().await;
if let Err(err) = res.as_ref() {
@@ -250,7 +258,7 @@ async fn maybe_add_bcc_self_device_msg(context: &Context) -> Result<()> {
msg.text = Some(
"It seems you are using multiple devices with Delta Chat. Great!\n\n\
If you also want to synchronize outgoing messages across all devices, \
go to the settings and enable \"Send copy to self\"."
go to \"Settings → Advanced\" and enable \"Send Copy to Self\"."
.to_string(),
);
chat::add_device_msg(context, Some("bcc-self-hint"), Some(&mut msg)).await?;
@@ -413,7 +421,7 @@ async fn import_backup(
"Cannot import backups to accounts in use."
);
ensure!(
context.scheduler.read().await.is_none(),
!context.scheduler.is_running().await,
"cannot import backup, IO is running"
);
@@ -516,16 +524,9 @@ async fn export_backup(context: &Context, dir: &Path, passphrase: String) -> Res
let _d1 = DeleteOnDrop(temp_db_path.clone());
let _d2 = DeleteOnDrop(temp_path.clone());
context
.sql
.set_raw_config_int("backup_time", now as i32)
.await?;
sql::housekeeping(context).await.ok_or_log(context);
ensure!(
context.scheduler.read().await.is_none(),
"cannot export backup, IO is running"
);
export_database(context, &temp_db_path, passphrase)
.await
.context("could not export database")?;
info!(
context,
@@ -534,32 +535,6 @@ async fn export_backup(context: &Context, dir: &Path, passphrase: String) -> Res
dest_path.display(),
);
let path_str = temp_db_path
.to_str()
.with_context(|| format!("path {temp_db_path:?} is not valid unicode"))?;
context
.sql
.call_write(|conn| {
if let Err(err) = conn.execute("VACUUM", params![]) {
info!(context, "Vacuum failed, exporting anyway: {:#}.", err);
}
conn.execute(
"ATTACH DATABASE ? AS backup KEY ?",
paramsv![path_str, passphrase],
)
.context("failed to attach backup database")?;
let res = conn
.query_row("SELECT sqlcipher_export('backup')", [], |_row| Ok(()))
.context("failed to export to attached backup database");
conn.execute("DETACH DATABASE backup", [])
.context("failed to detach backup database")?;
res?;
Ok::<_, Error>(())
})
.await?;
let res = export_backup_inner(context, &temp_db_path, &temp_path).await;
match &res {
@@ -597,29 +572,15 @@ async fn export_backup_inner(
.append_path_with_name(temp_db_path, DBFILE_BACKUP_NAME)
.await?;
let read_dir = tools::read_dir(context.get_blobdir()).await?;
let count = read_dir.len();
let mut written_files = 0;
let blobdir = BlobDirContents::new(context).await?;
let mut last_progress = 0;
for entry in read_dir {
let name = entry.file_name();
if !entry.file_type().await?.is_file() {
warn!(
context,
"Export: Found dir entry {} that is not a file, ignoring",
name.to_string_lossy()
);
continue;
}
let mut file = File::open(entry.path()).await?;
let path_in_archive = PathBuf::from(BLOBS_BACKUP_NAME).join(name);
builder.append_file(path_in_archive, &mut file).await?;
written_files += 1;
let progress = 1000 * written_files / count;
for (i, blob) in blobdir.iter().enumerate() {
let mut file = File::open(blob.to_abs_path()).await?;
let path_in_archive = PathBuf::from(BLOBS_BACKUP_NAME).join(blob.as_name());
builder.append_file(path_in_archive, &mut file).await?;
let progress = 1000 * i / blobdir.len();
if progress != last_progress && progress > 10 && progress < 1000 {
// We already emitted ImexProgress(10) above
context.emit_event(EventType::ImexProgress(progress));
last_progress = progress;
}
@@ -781,6 +742,48 @@ where
Ok(())
}
/// Exports the database to *dest*, encrypted using *passphrase*.
///
/// The directory of *dest* must already exist, if *dest* itself exists it will be
/// overwritten.
///
/// This also verifies that IO is not running during the export.
async fn export_database(context: &Context, dest: &Path, passphrase: String) -> Result<()> {
ensure!(
!context.scheduler.is_running().await,
"cannot export backup, IO is running"
);
let now = time().try_into().context("32-bit UNIX time overflow")?;
// TODO: Maybe introduce camino crate for UTF-8 paths where we need them.
let dest = dest
.to_str()
.with_context(|| format!("path {} is not valid unicode", dest.display()))?;
context.sql.set_raw_config_int("backup_time", now).await?;
sql::housekeeping(context).await.ok_or_log(context);
context
.sql
.call_write(|conn| {
conn.execute("VACUUM;", params![])
.map_err(|err| warn!(context, "Vacuum failed, exporting anyway {err}"))
.ok();
conn.execute(
"ATTACH DATABASE ? AS backup KEY ?",
paramsv![dest, passphrase],
)
.context("failed to attach backup database")?;
let res = conn
.query_row("SELECT sqlcipher_export('backup')", [], |_row| Ok(()))
.context("failed to export to attached backup database");
conn.execute("DETACH DATABASE backup", [])
.context("failed to detach backup database")?;
res?;
Ok(())
})
.await
}
#[cfg(test)]
mod tests {
use std::time::Duration;
+698
View File
@@ -0,0 +1,698 @@
//! Transfer a backup to an other device.
//!
//! This module provides support for using n0's iroh tool to initiate transfer of a backup
//! to another device using a QR code.
//!
//! Using the iroh terminology there are two parties to this:
//!
//! - The *Provider*, which starts a server and listens for connections.
//! - The *Getter*, which connects to the server and retrieves the data.
//!
//! Iroh is designed around the idea of verifying hashes, the downloads are verified as
//! they are retrieved. The entire transfer is initiated by requesting the data of a single
//! root hash.
//!
//! Both the provider and the getter are authenticated:
//!
//! - The provider is known by its *peer ID*.
//! - The provider needs an *authentication token* from the getter before it accepts a
//! connection.
//!
//! Both these are transferred in the QR code offered to the getter. This ensures that the
//! getter can not connect to an impersonated provider and the provider does not offer the
//! download to an impersonated getter.
use std::future::Future;
use std::net::Ipv4Addr;
use std::ops::Deref;
use std::path::{Path, PathBuf};
use std::pin::Pin;
use std::task::Poll;
use anyhow::{anyhow, bail, ensure, format_err, Context as _, Result};
use async_channel::Receiver;
use futures_lite::StreamExt;
use iroh::get::{DataStream, Options};
use iroh::progress::ProgressEmitter;
use iroh::protocol::AuthToken;
use iroh::provider::{DataSource, Event, Provider, Ticket};
use iroh::Hash;
use tokio::fs::{self, File};
use tokio::io::{self, AsyncWriteExt, BufWriter};
use tokio::sync::broadcast::error::RecvError;
use tokio::sync::{broadcast, Mutex};
use tokio::task::{JoinHandle, JoinSet};
use tokio_stream::wrappers::ReadDirStream;
use crate::blob::BlobDirContents;
use crate::chat::delete_and_reset_all_device_msgs;
use crate::context::Context;
use crate::qr::Qr;
use crate::{e2ee, EventType};
use super::{export_database, DBFILE_BACKUP_NAME};
/// Provide or send a backup of this device.
///
/// This creates a backup of the current device and starts a service which offers another
/// device to download this backup.
///
/// This does not make a full backup on disk, only the SQLite database is created on disk,
/// the blobs in the blob directory are not copied.
///
/// This starts a task which acquires the global "ongoing" mutex. If you need to stop the
/// task use the [`Context::stop_ongoing`] mechanism.
///
/// The task implements [`Future`] and awaiting it will complete once a transfer has been
/// either completed or aborted.
#[derive(Debug)]
pub struct BackupProvider {
/// The supervisor task, run by [`BackupProvider::watch_provider`].
handle: JoinHandle<Result<()>>,
/// The ticket to retrieve the backup collection.
ticket: Ticket,
}
impl BackupProvider {
/// Prepares for sending a backup to a second device.
///
/// Before calling this function all I/O must be stopped so that no changes to the blobs
/// or database are happening, this is done by calling the [`Accounts::stop_io`] or
/// [`Context::stop_io`] APIs first.
///
/// This will acquire the global "ongoing process" mutex, which can be used to cancel
/// the process.
///
/// [`Accounts::stop_io`]: crate::accounts::Accounts::stop_io
pub async fn prepare(context: &Context) -> Result<Self> {
e2ee::ensure_secret_key_exists(context)
.await
.context("Private key not available, aborting backup export")?;
// Acquire global "ongoing" mutex.
let cancel_token = context.alloc_ongoing().await?;
let context_dir = context
.get_blobdir()
.parent()
.ok_or(anyhow!("Context dir not found"))?;
let dbfile = context_dir.join(DBFILE_BACKUP_NAME);
if fs::metadata(&dbfile).await.is_ok() {
fs::remove_file(&dbfile).await?;
warn!(context, "Previous database export deleted");
}
let dbfile = TempPathGuard::new(dbfile);
let res = tokio::select! {
biased;
res = Self::prepare_inner(context, &dbfile) => {
match res {
Ok(slf) => Ok(slf),
Err(err) => {
error!(context, "Failed to set up second device setup: {:#}", err);
Err(err)
},
}
},
_ = cancel_token.recv() => Err(format_err!("cancelled")),
};
let (provider, ticket) = match res {
Ok((provider, ticket)) => (provider, ticket),
Err(err) => {
context.free_ongoing().await;
return Err(err);
}
};
let handle = tokio::spawn(Self::watch_provider(
context.clone(),
provider,
cancel_token,
dbfile,
));
let slf = Self { handle, ticket };
let qr = slf.qr();
*context.export_provider.lock().expect("poisoned lock") = Some(qr);
Ok(slf)
}
/// Creates the provider task.
///
/// Having this as a function makes it easier to cancel it when needed.
async fn prepare_inner(context: &Context, dbfile: &Path) -> Result<(Provider, Ticket)> {
// Generate the token up front: we also use it to encrypt the database.
let token = AuthToken::generate();
context.emit_event(SendProgress::Started.into());
export_database(context, dbfile, token.to_string())
.await
.context("Database export failed")?;
context.emit_event(SendProgress::DatabaseExported.into());
// Now we can be sure IO is not running.
let mut files = vec![DataSource::with_name(
dbfile.to_owned(),
format!("db/{DBFILE_BACKUP_NAME}"),
)];
let blobdir = BlobDirContents::new(context).await?;
for blob in blobdir.iter() {
let path = blob.to_abs_path();
let name = format!("blob/{}", blob.as_file_name());
files.push(DataSource::with_name(path, name));
}
// Start listening.
let (db, hash) = iroh::provider::create_collection(files).await?;
context.emit_event(SendProgress::CollectionCreated.into());
let provider = Provider::builder(db)
.bind_addr((Ipv4Addr::UNSPECIFIED, 0).into())
.auth_token(token)
.spawn()?;
context.emit_event(SendProgress::ProviderListening.into());
info!(context, "Waiting for remote to connect");
let ticket = provider.ticket(hash);
Ok((provider, ticket))
}
/// Supervises the iroh [`Provider`], terminating it when needed.
///
/// This will watch the provider and terminate it when:
///
/// - A transfer is completed, successful or unsuccessful.
/// - An event could not be observed to protect against not knowing of a completed event.
/// - The ongoing process is cancelled.
///
/// The *cancel_token* is the handle for the ongoing process mutex, when this completes
/// we must cancel this operation.
async fn watch_provider(
context: Context,
mut provider: Provider,
cancel_token: Receiver<()>,
_dbfile: TempPathGuard,
) -> Result<()> {
// _dbfile exists so we can clean up the file once it is no longer needed
let mut events = provider.subscribe();
let mut total_size = 0;
let mut current_size = 0;
let res = loop {
tokio::select! {
biased;
res = &mut provider => {
break res.context("BackupProvider failed");
},
maybe_event = events.recv() => {
match maybe_event {
Ok(event) => {
match event {
Event::ClientConnected { ..} => {
context.emit_event(SendProgress::ClientConnected.into());
}
Event::RequestReceived { .. } => {
}
Event::TransferCollectionStarted { total_blobs_size, .. } => {
total_size = total_blobs_size;
context.emit_event(SendProgress::TransferInProgress {
current_size,
total_size,
}.into());
}
Event::TransferBlobCompleted { size, .. } => {
current_size += size;
context.emit_event(SendProgress::TransferInProgress {
current_size,
total_size,
}.into());
}
Event::TransferCollectionCompleted { .. } => {
context.emit_event(SendProgress::TransferInProgress {
current_size: total_size,
total_size
}.into());
provider.shutdown();
}
Event::TransferAborted { .. } => {
provider.shutdown();
break Err(anyhow!("BackupProvider transfer aborted"));
}
}
}
Err(broadcast::error::RecvError::Closed) => {
// We should never see this, provider.join() should complete
// first.
}
Err(broadcast::error::RecvError::Lagged(_)) => {
// We really shouldn't be lagging, if we did we may have missed
// a completion event.
provider.shutdown();
break Err(anyhow!("Missed events from BackupProvider"));
}
}
},
_ = cancel_token.recv() => {
provider.shutdown();
break Err(anyhow!("BackupSender cancelled"));
},
}
};
context
.export_provider
.lock()
.expect("poisoned lock")
.take();
match &res {
Ok(_) => context.emit_event(SendProgress::Completed.into()),
Err(err) => {
error!(context, "Backup transfer failure: {err:#}");
context.emit_event(SendProgress::Failed.into())
}
}
context.free_ongoing().await;
res
}
/// Returns a QR code that allows fetching this backup.
///
/// This QR code can be passed to [`get_backup`] on a (different) device.
pub fn qr(&self) -> Qr {
Qr::Backup {
ticket: self.ticket.clone(),
}
}
}
impl Future for BackupProvider {
type Output = Result<()>;
fn poll(mut self: Pin<&mut Self>, cx: &mut std::task::Context<'_>) -> Poll<Self::Output> {
Pin::new(&mut self.handle).poll(cx)?
}
}
/// A guard which will remove the path when dropped.
///
/// It implements [`Deref`] it it can be used as a `&Path`.
#[derive(Debug)]
struct TempPathGuard {
path: PathBuf,
}
impl TempPathGuard {
fn new(path: PathBuf) -> Self {
Self { path }
}
}
impl Drop for TempPathGuard {
fn drop(&mut self) {
let path = self.path.clone();
tokio::spawn(async move {
fs::remove_file(&path).await.ok();
});
}
}
impl Deref for TempPathGuard {
type Target = Path;
fn deref(&self) -> &Self::Target {
&self.path
}
}
/// Create [`EventType::ImexProgress`] events using readable names.
///
/// Plus you get warnings if you don't use all variants.
#[derive(Debug)]
enum SendProgress {
Failed,
Started,
DatabaseExported,
CollectionCreated,
ProviderListening,
ClientConnected,
TransferInProgress { current_size: u64, total_size: u64 },
Completed,
}
impl From<SendProgress> for EventType {
fn from(source: SendProgress) -> Self {
use SendProgress::*;
let num: u16 = match source {
Failed => 0,
Started => 100,
DatabaseExported => 300,
CollectionCreated => 350,
ProviderListening => 400,
ClientConnected => 450,
TransferInProgress {
current_size,
total_size,
} => {
// the range is 450..=950
450 + ((current_size as f64 / total_size as f64) * 500.).floor() as u16
}
Completed => 1000,
};
Self::ImexProgress(num.into())
}
}
/// Contacts a backup provider and receives the backup from it.
///
/// This uses a QR code to contact another instance of deltachat which is providing a backup
/// using the [`BackupProvider`]. Once connected it will authenticate using the secrets in
/// the QR code and retrieve the backup.
///
/// This is a long running operation which will only when completed.
///
/// Using [`Qr`] as argument is a bit odd as it only accepts one specific variant of it. It
/// does avoid having [`iroh::provider::Ticket`] in the primary API however, without
/// having to revert to untyped bytes.
pub async fn get_backup(context: &Context, qr: Qr) -> Result<()> {
ensure!(
matches!(qr, Qr::Backup { .. }),
"QR code for backup must be of type DCBACKUP"
);
ensure!(
!context.is_configured().await?,
"Cannot import backups to accounts in use."
);
let mut guard = context.scheduler.pause(context).await;
// Acquire global "ongoing" mutex.
let cancel_token = context.alloc_ongoing().await?;
let res = tokio::select! {
biased;
res = get_backup_inner(context, qr) => {
context.free_ongoing().await;
res
}
_ = cancel_token.recv() => Err(format_err!("cancelled")),
};
guard.resume().await;
res
}
async fn get_backup_inner(context: &Context, qr: Qr) -> Result<()> {
let ticket = match qr {
Qr::Backup { ticket } => ticket,
_ => bail!("QR code for backup must be of type DCBACKUP"),
};
if ticket.addrs.is_empty() {
bail!("ticket is missing addresses to dial");
}
for addr in &ticket.addrs {
let opts = Options {
addr: *addr,
peer_id: Some(ticket.peer),
keylog: false,
};
info!(context, "attempting to contact {}", addr);
match transfer_from_provider(context, &ticket, opts).await {
Ok(_) => {
delete_and_reset_all_device_msgs(context).await?;
context.emit_event(ReceiveProgress::Completed.into());
return Ok(());
}
Err(TransferError::ConnectionError(err)) => {
warn!(context, "Connection error: {err:#}.");
continue;
}
Err(TransferError::Other(err)) => {
// Clean up any blobs we already wrote.
let readdir = fs::read_dir(context.get_blobdir()).await?;
let mut readdir = ReadDirStream::new(readdir);
while let Some(dirent) = readdir.next().await {
if let Ok(dirent) = dirent {
fs::remove_file(dirent.path()).await.ok();
}
}
context.emit_event(ReceiveProgress::Failed.into());
return Err(err);
}
}
}
Err(anyhow!("failed to contact provider"))
}
/// Error during a single transfer attempt.
///
/// Mostly exists to distinguish between `ConnectionError` and any other errors.
#[derive(Debug, thiserror::Error)]
enum TransferError {
#[error("connection error")]
ConnectionError(#[source] anyhow::Error),
#[error("other")]
Other(#[source] anyhow::Error),
}
async fn transfer_from_provider(
context: &Context,
ticket: &Ticket,
opts: Options,
) -> Result<(), TransferError> {
let progress = ProgressEmitter::new(0, ReceiveProgress::max_blob_progress());
spawn_progress_proxy(context.clone(), progress.subscribe());
let mut connected = false;
let on_connected = || {
context.emit_event(ReceiveProgress::Connected.into());
connected = true;
async { Ok(()) }
};
let jobs = Mutex::new(JoinSet::default());
let on_blob =
|hash, reader, name| on_blob(context, &progress, &jobs, ticket, hash, reader, name);
let res = iroh::get::run(
ticket.hash,
ticket.token,
opts,
on_connected,
|collection| {
context.emit_event(ReceiveProgress::CollectionRecieved.into());
progress.set_total(collection.total_blobs_size());
async { Ok(()) }
},
on_blob,
)
.await;
let mut jobs = jobs.lock().await;
while let Some(job) = jobs.join_next().await {
job.context("job failed").map_err(TransferError::Other)?;
}
drop(progress);
match res {
Ok(stats) => {
info!(
context,
"Backup transfer finished, transfer rate is {} Mbps.",
stats.mbits()
);
Ok(())
}
Err(err) => match connected {
true => Err(TransferError::Other(err)),
false => Err(TransferError::ConnectionError(err)),
},
}
}
/// Get callback when a blob is received from the provider.
///
/// This writes the blobs to the blobdir. If the blob is the database it will import it to
/// the database of the current [`Context`].
async fn on_blob(
context: &Context,
progress: &ProgressEmitter,
jobs: &Mutex<JoinSet<()>>,
ticket: &Ticket,
_hash: Hash,
mut reader: DataStream,
name: String,
) -> Result<DataStream> {
ensure!(!name.is_empty(), "Received a nameless blob");
let path = if name.starts_with("db/") {
let context_dir = context
.get_blobdir()
.parent()
.ok_or(anyhow!("Context dir not found"))?;
let dbfile = context_dir.join(DBFILE_BACKUP_NAME);
if fs::metadata(&dbfile).await.is_ok() {
fs::remove_file(&dbfile).await?;
warn!(context, "Previous database export deleted");
}
dbfile
} else {
ensure!(name.starts_with("blob/"), "malformatted blob name");
let blobname = name.rsplit('/').next().context("malformatted blob name")?;
context.get_blobdir().join(blobname)
};
let mut wrapped_reader = progress.wrap_async_read(&mut reader);
let file = File::create(&path).await?;
let mut file = BufWriter::with_capacity(128 * 1024, file);
io::copy(&mut wrapped_reader, &mut file).await?;
file.flush().await?;
if name.starts_with("db/") {
let context = context.clone();
let token = ticket.token.to_string();
jobs.lock().await.spawn(async move {
if let Err(err) = context.sql.import(&path, token).await {
error!(context, "cannot import database: {:#?}", err);
}
if let Err(err) = fs::remove_file(&path).await {
error!(
context,
"failed to delete database import file '{}': {:#?}",
path.display(),
err,
);
}
});
}
Ok(reader)
}
/// Spawns a task proxying progress events.
///
/// This spawns a tokio task which receives events from the [`ProgressEmitter`] and sends
/// them to the context. The task finishes when the emitter is dropped.
///
/// This could be done directly in the emitter by making it less generic.
fn spawn_progress_proxy(context: Context, mut rx: broadcast::Receiver<u16>) {
tokio::spawn(async move {
loop {
match rx.recv().await {
Ok(step) => context.emit_event(ReceiveProgress::BlobProgress(step).into()),
Err(RecvError::Closed) => break,
Err(RecvError::Lagged(_)) => continue,
}
}
});
}
/// Create [`EventType::ImexProgress`] events using readable names.
///
/// Plus you get warnings if you don't use all variants.
#[derive(Debug)]
enum ReceiveProgress {
Connected,
CollectionRecieved,
/// A value between 0 and 85 interpreted as a percentage.
///
/// Other values are already used by the other variants of this enum.
BlobProgress(u16),
Completed,
Failed,
}
impl ReceiveProgress {
/// The maximum value for [`ReceiveProgress::BlobProgress`].
///
/// This only exists to keep this magic value local in this type.
fn max_blob_progress() -> u16 {
85
}
}
impl From<ReceiveProgress> for EventType {
fn from(source: ReceiveProgress) -> Self {
let val = match source {
ReceiveProgress::Connected => 50,
ReceiveProgress::CollectionRecieved => 100,
ReceiveProgress::BlobProgress(val) => 100 + 10 * val,
ReceiveProgress::Completed => 1000,
ReceiveProgress::Failed => 0,
};
EventType::ImexProgress(val.into())
}
}
#[cfg(test)]
mod tests {
use std::time::Duration;
use crate::chat::{get_chat_msgs, send_msg, ChatItem};
use crate::message::{Message, Viewtype};
use crate::test_utils::TestContextManager;
use super::*;
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_send_receive() {
let mut tcm = TestContextManager::new();
// Create first device.
let ctx0 = tcm.alice().await;
// Write a message in the self chat
let self_chat = ctx0.get_self_chat().await;
let mut msg = Message::new(Viewtype::Text);
msg.set_text(Some("hi there".to_string()));
send_msg(&ctx0, self_chat.id, &mut msg).await.unwrap();
// Send an attachment in the self chat
let file = ctx0.get_blobdir().join("hello.txt");
fs::write(&file, "i am attachment").await.unwrap();
let mut msg = Message::new(Viewtype::File);
msg.set_file(file.to_str().unwrap(), Some("text/plain"));
send_msg(&ctx0, self_chat.id, &mut msg).await.unwrap();
// Prepare to transfer backup.
let provider = BackupProvider::prepare(&ctx0).await.unwrap();
// Set up second device.
let ctx1 = tcm.unconfigured().await;
get_backup(&ctx1, provider.qr()).await.unwrap();
// Make sure the provider finishes without an error.
tokio::time::timeout(Duration::from_secs(30), provider)
.await
.expect("timed out")
.expect("error in provider");
// Check that we have the self message.
let self_chat = ctx1.get_self_chat().await;
let msgs = get_chat_msgs(&ctx1, self_chat.id).await.unwrap();
assert_eq!(msgs.len(), 2);
let msgid = match msgs.get(0).unwrap() {
ChatItem::Message { msg_id } => msg_id,
_ => panic!("wrong chat item"),
};
let msg = Message::load_from_db(&ctx1, *msgid).await.unwrap();
let text = msg.get_text().unwrap();
assert_eq!(text, "hi there");
let msgid = match msgs.get(1).unwrap() {
ChatItem::Message { msg_id } => msg_id,
_ => panic!("wrong chat item"),
};
let msg = Message::load_from_db(&ctx1, *msgid).await.unwrap();
let path = msg.get_file(&ctx1).unwrap();
let text = fs::read_to_string(&path).await.unwrap();
assert_eq!(text, "i am attachment");
// Check that both received the ImexProgress events.
ctx0.evtracker
.get_matching(|ev| matches!(ev, EventType::ImexProgress(1000)))
.await;
ctx1.evtracker
.get_matching(|ev| matches!(ev, EventType::ImexProgress(1000)))
.await;
}
#[test]
fn test_send_progress() {
let cases = [
((0, 100), 450),
((10, 100), 500),
((50, 100), 700),
((100, 100), 950),
];
for ((current_size, total_size), progress) in cases {
let out = EventType::from(SendProgress::TransferInProgress {
current_size,
total_size,
});
assert_eq!(out, EventType::ImexProgress(progress));
}
}
}
+5 -1
View File
@@ -238,6 +238,7 @@ fn get_backoff_time_offset(tries: u32) -> i64 {
pub(crate) async fn schedule_resync(context: &Context) -> Result<()> {
context.resync_request.store(true, Ordering::Relaxed);
context
.scheduler
.interrupt_inbox(InterruptInfo {
probe_network: false,
})
@@ -250,7 +251,10 @@ pub async fn add(context: &Context, job: Job) -> Result<()> {
job.save(context).await.context("failed to save job")?;
info!(context, "interrupt: imap");
context.interrupt_inbox(InterruptInfo::new(false)).await;
context
.scheduler
.interrupt_inbox(InterruptInfo::new(false))
.await;
Ok(())
}
+1 -1
View File
@@ -267,7 +267,7 @@ pub async fn send_locations_to_chat(
}
context.emit_event(EventType::ChatModified(chat_id));
if 0 != seconds {
context.interrupt_location().await;
context.scheduler.interrupt_location().await;
}
Ok(())
}
+1
View File
@@ -147,6 +147,7 @@ impl<T, E: std::fmt::Display> LogExt<T, E> for Result<T, E> {
);
// We can't use the warn!() macro here as the file!() and line!() macros
// don't work with #[track_caller]
context.set_last_error(&full);
context.emit_event(crate::EventType::Warning(full));
};
self
+8 -2
View File
@@ -1419,7 +1419,10 @@ pub async fn delete_msgs(context: &Context, msg_ids: &[MsgId]) -> Result<()> {
}
// Interrupt Inbox loop to start message deletion and run housekeeping.
context.interrupt_inbox(InterruptInfo::new(false)).await;
context
.scheduler
.interrupt_inbox(InterruptInfo::new(false))
.await;
Ok(())
}
@@ -1531,7 +1534,10 @@ pub async fn markseen_msgs(context: &Context, msg_ids: Vec<MsgId>) -> Result<()>
)
.await
.context("failed to insert into smtp_mdns")?;
context.interrupt_smtp(InterruptInfo::new(false)).await;
context
.scheduler
.interrupt_smtp(InterruptInfo::new(false))
.await;
}
}
updated_chat_ids.insert(curr_chat_id);
+42
View File
@@ -34,6 +34,7 @@ const VCARD_SCHEME: &str = "BEGIN:VCARD";
const SMTP_SCHEME: &str = "SMTP:";
const HTTP_SCHEME: &str = "http://";
const HTTPS_SCHEME: &str = "https://";
pub(crate) const DCBACKUP_SCHEME: &str = "DCBACKUP:";
/// Scanned QR code.
#[derive(Debug, Clone, PartialEq, Eq)]
@@ -102,6 +103,20 @@ pub enum Qr {
domain: String,
},
/// Provides a backup that can be retrieve.
///
/// This contains all the data needed to connect to a device and download a backup from
/// it to configure the receiving device with the same account.
Backup {
/// Printable version of the provider information.
///
/// This is the printable version of a `sendme` ticket, which contains all the
/// information to connect to and authenticate a backup provider.
///
/// The format is somewhat opaque, but `sendme` can deserialise this.
ticket: iroh::provider::Ticket,
},
/// Ask the user if they want to use the given service for video chats.
WebrtcInstance {
/// Server domain name.
@@ -244,6 +259,8 @@ pub async fn check_qr(context: &Context, qr: &str) -> Result<Qr> {
dclogin_scheme::decode_login(qr)?
} else if starts_with_ignore_case(qr, DCWEBRTC_SCHEME) {
decode_webrtc_instance(context, qr)?
} else if starts_with_ignore_case(qr, DCBACKUP_SCHEME) {
decode_backup(qr)?
} else if qr.starts_with(MAILTO_SCHEME) {
decode_mailto(context, qr).await?
} else if qr.starts_with(SMTP_SCHEME) {
@@ -264,6 +281,19 @@ pub async fn check_qr(context: &Context, qr: &str) -> Result<Qr> {
Ok(qrcode)
}
/// Formats the text of the [`Qr::Backup`] variant.
///
/// This is the inverse of [`check_qr`] for that variant only.
///
/// TODO: Refactor this so all variants have a correct [`Display`] and transform `check_qr`
/// into `FromStr`.
pub fn format_backup(qr: &Qr) -> Result<String> {
match qr {
Qr::Backup { ref ticket } => Ok(format!("{DCBACKUP_SCHEME}{ticket}")),
_ => Err(anyhow!("Not a backup QR code")),
}
}
/// scheme: `OPENPGP4FPR:FINGERPRINT#a=ADDR&n=NAME&i=INVITENUMBER&s=AUTH`
/// or: `OPENPGP4FPR:FINGERPRINT#a=ADDR&g=GROUPNAME&x=GROUPID&i=INVITENUMBER&s=AUTH`
/// or: `OPENPGP4FPR:FINGERPRINT#a=ADDR`
@@ -471,6 +501,18 @@ fn decode_webrtc_instance(_context: &Context, qr: &str) -> Result<Qr> {
}
}
/// Decodes a [`DCBACKUP_SCHEME`] QR code.
///
/// The format of this scheme is `DCBACKUP:<encoded ticket>`. The encoding is the
/// [`iroh::provider::Ticket`]'s `Display` impl.
fn decode_backup(qr: &str) -> Result<Qr> {
let payload = qr
.strip_prefix(DCBACKUP_SCHEME)
.ok_or(anyhow!("invalid DCBACKUP scheme"))?;
let ticket: iroh::provider::Ticket = payload.parse().context("invalid DCBACKUP payload")?;
Ok(Qr::Backup { ticket })
}
#[derive(Debug, Deserialize)]
struct CreateAccountSuccessResponse {
/// Email address.
+57 -10
View File
@@ -11,7 +11,9 @@ use crate::{
config::Config,
contact::{Contact, ContactId},
context::Context,
securejoin, stock_str,
qr::{self, Qr},
securejoin,
stock_str::{self, backup_transfer_qr},
};
/// Returns SVG of the QR code to join the group or verify contact.
@@ -47,6 +49,34 @@ async fn generate_join_group_qr_code(context: &Context, chat_id: ChatId) -> Resu
}
async fn generate_verification_qr(context: &Context) -> Result<String> {
let (avatar, displayname, addr, color) = self_info(context).await?;
inner_generate_secure_join_qr_code(
&stock_str::setup_contact_qr_description(context, &displayname, &addr).await,
&securejoin::get_securejoin_qr(context, None).await?,
&color,
avatar,
displayname.chars().next().unwrap_or('#'),
)
}
/// Renders a [`Qr::Backup`] QR code as an SVG image.
pub async fn generate_backup_qr(context: &Context, qr: &Qr) -> Result<String> {
let content = qr::format_backup(qr)?;
let (avatar, displayname, _addr, color) = self_info(context).await?;
let description = backup_transfer_qr(context).await?;
inner_generate_secure_join_qr_code(
&description,
&content,
&color,
avatar,
displayname.chars().next().unwrap_or('#'),
)
}
/// Returns `(avatar, displayname, addr, color) of the configured account.
async fn self_info(context: &Context) -> Result<(Option<Vec<u8>>, String, String, String)> {
let contact = Contact::get_by_id(context, ContactId::SELF).await?;
let avatar = match contact.get_profile_image(context).await? {
@@ -59,16 +89,11 @@ async fn generate_verification_qr(context: &Context) -> Result<String> {
let displayname = match context.get_config(Config::Displayname).await? {
Some(name) => name,
None => contact.get_addr().to_owned(),
None => contact.get_addr().to_string(),
};
inner_generate_secure_join_qr_code(
&stock_str::setup_contact_qr_description(context, &displayname, contact.get_addr()).await,
&securejoin::get_securejoin_qr(context, None).await?,
&color_int_to_hex_string(contact.get_color()),
avatar,
displayname.chars().next().unwrap_or('#'),
)
let addr = contact.get_addr().to_string();
let color = color_int_to_hex_string(contact.get_color());
Ok((avatar, displayname, addr, color))
}
fn inner_generate_secure_join_qr_code(
@@ -272,6 +297,12 @@ fn inner_generate_secure_join_qr_code(
#[cfg(test)]
mod tests {
use testdir::testdir;
use crate::imex::BackupProvider;
use crate::qr::format_backup;
use crate::test_utils::TestContextManager;
use super::*;
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
@@ -286,4 +317,20 @@ mod tests {
.unwrap();
assert!(svg.contains("descr123 &quot; &lt; &gt; &amp;"))
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_generate_backup_qr() {
let dir = testdir!();
let mut tcm = TestContextManager::new();
let ctx = tcm.alice().await;
let provider = BackupProvider::prepare(&ctx).await.unwrap();
let qr = provider.qr();
println!("{}", format_backup(&qr).unwrap());
let rendered = generate_backup_qr(&ctx, &qr).await.unwrap();
tokio::fs::write(dir.join("qr.svg"), &rendered)
.await
.unwrap();
assert_eq!(rendered.get(..4), Some("<svg"));
}
}
+3 -1
View File
@@ -115,7 +115,9 @@ impl Context {
let requested = self.quota_update_request.swap(true, Ordering::Relaxed);
if !requested {
// Quota update was not requested before.
self.interrupt_inbox(InterruptInfo::new(false)).await;
self.scheduler
.interrupt_inbox(InterruptInfo::new(false))
.await;
}
Ok(())
}
+201 -50
View File
@@ -5,6 +5,7 @@ use anyhow::{bail, Context as _, Result};
use async_channel::{self as channel, Receiver, Sender};
use futures::future::try_join_all;
use futures_lite::FutureExt;
use tokio::sync::{RwLock, RwLockWriteGuard};
use tokio::task;
use self::connectivity::ConnectivityStore;
@@ -23,10 +24,210 @@ use crate::tools::{duration_to_str, maybe_add_time_based_warnings};
pub(crate) mod connectivity;
/// State of the IO scheduler, as stored on the [`Context`].
///
/// The IO scheduler can be stopped or started, but core can also pause it. After pausing
/// the IO scheduler will be restarted only if it was running before paused or
/// [`Context::start_io`] was called in the meantime while it was paused.
#[derive(Debug, Default)]
pub(crate) struct SchedulerState {
inner: RwLock<InnerSchedulerState>,
}
impl SchedulerState {
pub(crate) fn new() -> Self {
Default::default()
}
/// Whether the scheduler is currently running.
pub(crate) async fn is_running(&self) -> bool {
let inner = self.inner.read().await;
inner.scheduler.is_some()
}
/// Starts the scheduler if it is not yet started.
pub(crate) async fn start(&self, context: Context) {
let mut inner = self.inner.write().await;
inner.started = true;
if inner.scheduler.is_none() && !inner.paused {
Self::do_start(inner, context).await;
}
}
/// Starts the scheduler if it is not yet started.
async fn do_start(mut inner: RwLockWriteGuard<'_, InnerSchedulerState>, context: Context) {
info!(context, "starting IO");
let ctx = context.clone();
match Scheduler::start(context).await {
Ok(scheduler) => inner.scheduler = Some(scheduler),
Err(err) => error!(&ctx, "Failed to start IO: {:#}", err),
}
}
/// Stops the scheduler if it is currently running.
pub(crate) async fn stop(&self, context: &Context) {
let mut inner = self.inner.write().await;
inner.started = false;
Self::do_stop(inner, context).await;
}
/// Stops the scheduler if it is currently running.
async fn do_stop(mut inner: RwLockWriteGuard<'_, InnerSchedulerState>, context: &Context) {
// Sending an event wakes up event pollers (get_next_event)
// so the caller of stop_io() can arrange for proper termination.
// For this, the caller needs to instruct the event poller
// to terminate on receiving the next event and then call stop_io()
// which will emit the below event(s)
info!(context, "stopping IO");
if let Some(debug_logging) = context.debug_logging.read().await.as_ref() {
debug_logging.loop_handle.abort();
}
if let Some(scheduler) = inner.scheduler.take() {
scheduler.stop(context).await;
}
}
/// Pauses the IO scheduler.
///
/// If it is currently running the scheduler will be stopped. When
/// [`IoPausedGuard::resume`] is called the scheduler is started again.
///
/// If in the meantime [`SchedulerState::start`] or [`SchedulerState::stop`] is called
/// resume will do the right thing and restore the scheduler to the state requested by
/// the last call.
pub(crate) async fn pause<'a>(&'_ self, context: &'a Context) -> IoPausedGuard<'a> {
let mut inner = self.inner.write().await;
inner.paused = true;
Self::do_stop(inner, context).await;
IoPausedGuard {
context,
done: false,
}
}
/// Restarts the scheduler, only if it is running.
pub(crate) async fn restart(&self, context: &Context) {
info!(context, "restarting IO");
if self.is_running().await {
self.stop(context).await;
self.start(context.clone()).await;
}
}
/// Indicate that the network likely has come back.
pub(crate) async fn maybe_network(&self) {
let inner = self.inner.read().await;
let (inbox, oboxes) = match inner.scheduler {
Some(ref scheduler) => {
scheduler.maybe_network();
let inbox = scheduler.inbox.conn_state.state.connectivity.clone();
let oboxes = scheduler
.oboxes
.iter()
.map(|b| b.conn_state.state.connectivity.clone())
.collect::<Vec<_>>();
(inbox, oboxes)
}
None => return,
};
drop(inner);
connectivity::idle_interrupted(inbox, oboxes).await;
}
/// Indicate that the network likely is lost.
pub(crate) async fn maybe_network_lost(&self, context: &Context) {
let inner = self.inner.read().await;
let stores = match inner.scheduler {
Some(ref scheduler) => {
scheduler.maybe_network_lost();
scheduler
.boxes()
.map(|b| b.conn_state.state.connectivity.clone())
.collect()
}
None => return,
};
drop(inner);
connectivity::maybe_network_lost(context, stores).await;
}
pub(crate) async fn interrupt_inbox(&self, info: InterruptInfo) {
let inner = self.inner.read().await;
if let Some(ref scheduler) = inner.scheduler {
scheduler.interrupt_inbox(info);
}
}
pub(crate) async fn interrupt_smtp(&self, info: InterruptInfo) {
let inner = self.inner.read().await;
if let Some(ref scheduler) = inner.scheduler {
scheduler.interrupt_smtp(info);
}
}
pub(crate) async fn interrupt_ephemeral_task(&self) {
let inner = self.inner.read().await;
if let Some(ref scheduler) = inner.scheduler {
scheduler.interrupt_ephemeral_task();
}
}
pub(crate) async fn interrupt_location(&self) {
let inner = self.inner.read().await;
if let Some(ref scheduler) = inner.scheduler {
scheduler.interrupt_location();
}
}
pub(crate) async fn interrupt_recently_seen(&self, contact_id: ContactId, timestamp: i64) {
let inner = self.inner.read().await;
if let Some(ref scheduler) = inner.scheduler {
scheduler.interrupt_recently_seen(contact_id, timestamp);
}
}
}
#[derive(Debug, Default)]
struct InnerSchedulerState {
scheduler: Option<Scheduler>,
started: bool,
paused: bool,
}
#[derive(Debug)]
pub(crate) struct IoPausedGuard<'a> {
context: &'a Context,
done: bool,
}
impl<'a> IoPausedGuard<'a> {
pub(crate) async fn resume(&mut self) {
self.done = true;
let mut inner = self.context.scheduler.inner.write().await;
inner.paused = false;
if inner.started && inner.scheduler.is_none() {
SchedulerState::do_start(inner, self.context.clone()).await;
}
}
}
impl<'a> Drop for IoPausedGuard<'a> {
fn drop(&mut self) {
if self.done {
return;
}
// Async .resume() should be called manually due to lack of async drop.
error!(self.context, "Pause guard dropped without resuming.");
}
}
#[derive(Debug)]
struct SchedBox {
meaning: FolderMeaning,
conn_state: ImapConnectionState,
/// IMAP loop task handle.
handle: task::JoinHandle<()>,
}
@@ -46,56 +247,6 @@ pub(crate) struct Scheduler {
recently_seen_loop: RecentlySeenLoop,
}
impl Context {
/// Indicate that the network likely has come back.
pub async fn maybe_network(&self) {
let lock = self.scheduler.read().await;
if let Some(scheduler) = &*lock {
scheduler.maybe_network();
}
connectivity::idle_interrupted(lock).await;
}
/// Indicate that the network likely is lost.
pub async fn maybe_network_lost(&self) {
let lock = self.scheduler.read().await;
if let Some(scheduler) = &*lock {
scheduler.maybe_network_lost();
}
connectivity::maybe_network_lost(self, lock).await;
}
pub(crate) async fn interrupt_inbox(&self, info: InterruptInfo) {
if let Some(scheduler) = &*self.scheduler.read().await {
scheduler.interrupt_inbox(info);
}
}
pub(crate) async fn interrupt_smtp(&self, info: InterruptInfo) {
if let Some(scheduler) = &*self.scheduler.read().await {
scheduler.interrupt_smtp(info);
}
}
pub(crate) async fn interrupt_ephemeral_task(&self) {
if let Some(scheduler) = &*self.scheduler.read().await {
scheduler.interrupt_ephemeral_task();
}
}
pub(crate) async fn interrupt_location(&self) {
if let Some(scheduler) = &*self.scheduler.read().await {
scheduler.interrupt_location();
}
}
pub(crate) async fn interrupt_recently_seen(&self, contact_id: ContactId, timestamp: i64) {
if let Some(scheduler) = &*self.scheduler.read().await {
scheduler.interrupt_recently_seen(contact_id, timestamp);
}
}
}
async fn inbox_loop(ctx: Context, started: Sender<()>, inbox_handlers: ImapConnectionHandlers) {
use futures::future::FutureExt;
+13 -37
View File
@@ -3,7 +3,7 @@ use std::{iter::once, ops::Deref, sync::Arc};
use anyhow::{anyhow, Result};
use humansize::{format_size, BINARY};
use tokio::sync::{Mutex, RwLockReadGuard};
use tokio::sync::Mutex;
use crate::events::EventType;
use crate::imap::{scan_folders::get_watched_folder_configs, FolderMeaning};
@@ -12,7 +12,7 @@ use crate::quota::{
};
use crate::tools::time;
use crate::{context::Context, log::LogExt};
use crate::{scheduler::Scheduler, stock_str, tools};
use crate::{stock_str, tools};
#[derive(Debug, Clone, Copy, PartialEq, Eq, EnumProperty, PartialOrd, Ord)]
pub enum Connectivity {
@@ -156,19 +156,7 @@ impl ConnectivityStore {
/// Set all folder states to InterruptingIdle in case they were `Connected` before.
/// Called during `dc_maybe_network()` to make sure that `dc_accounts_all_work_done()`
/// returns false immediately after `dc_maybe_network()`.
pub(crate) async fn idle_interrupted(scheduler: RwLockReadGuard<'_, Option<Scheduler>>) {
let (inbox, oboxes) = match &*scheduler {
Some(Scheduler { inbox, oboxes, .. }) => (
inbox.conn_state.state.connectivity.clone(),
oboxes
.iter()
.map(|b| b.conn_state.state.connectivity.clone())
.collect::<Vec<_>>(),
),
None => return,
};
drop(scheduler);
pub(crate) async fn idle_interrupted(inbox: ConnectivityStore, oboxes: Vec<ConnectivityStore>) {
let mut connectivity_lock = inbox.0.lock().await;
// For the inbox, we also have to set the connectivity to InterruptingIdle if it was
// NotConfigured before: If all folders are NotConfigured, dc_get_connectivity()
@@ -195,19 +183,7 @@ pub(crate) async fn idle_interrupted(scheduler: RwLockReadGuard<'_, Option<Sched
/// Set the connectivity to "Not connected" after a call to dc_maybe_network_lost().
/// If we did not do this, the connectivity would stay "Connected" for quite a long time
/// after `maybe_network_lost()` was called.
pub(crate) async fn maybe_network_lost(
context: &Context,
scheduler: RwLockReadGuard<'_, Option<Scheduler>>,
) {
let stores: Vec<_> = match &*scheduler {
Some(sched) => sched
.boxes()
.map(|b| b.conn_state.state.connectivity.clone())
.collect(),
None => return,
};
drop(scheduler);
pub(crate) async fn maybe_network_lost(context: &Context, stores: Vec<ConnectivityStore>) {
for store in &stores {
let mut connectivity_lock = store.0.lock().await;
if !matches!(
@@ -249,9 +225,9 @@ impl Context {
///
/// If the connectivity changes, a DC_EVENT_CONNECTIVITY_CHANGED will be emitted.
pub async fn get_connectivity(&self) -> Connectivity {
let lock = self.scheduler.read().await;
let stores: Vec<_> = match &*lock {
Some(sched) => sched
let lock = self.scheduler.inner.read().await;
let stores: Vec<_> = match lock.scheduler {
Some(ref sched) => sched
.boxes()
.map(|b| b.conn_state.state.connectivity.clone())
.collect(),
@@ -332,9 +308,9 @@ impl Context {
// Get the states from the RwLock
// =============================================================================================
let lock = self.scheduler.read().await;
let (folders_states, smtp) = match &*lock {
Some(sched) => (
let lock = self.scheduler.inner.read().await;
let (folders_states, smtp) = match lock.scheduler {
Some(ref sched) => (
sched
.boxes()
.map(|b| (b.meaning, b.conn_state.state.connectivity.clone()))
@@ -503,9 +479,9 @@ impl Context {
/// Returns true if all background work is done.
pub async fn all_work_done(&self) -> bool {
let lock = self.scheduler.read().await;
let stores: Vec<_> = match &*lock {
Some(sched) => sched
let lock = self.scheduler.inner.read().await;
let stores: Vec<_> = match lock.scheduler {
Some(ref sched) => sched
.boxes()
.map(|b| &b.conn_state.state)
.chain(once(&sched.smtp.state))
+23 -2
View File
@@ -404,6 +404,9 @@ pub enum StockMessage {
#[strum(props(fallback = "Chat protection disabled by %1$s."))]
ProtectionDisabledBy = 161,
#[strum(props(fallback = "Scan to set up second device for %1$s"))]
BackupTransferQr = 162,
}
impl StockMessage {
@@ -741,14 +744,14 @@ pub(crate) async fn setup_contact_qr_description(
display_name: &str,
addr: &str,
) -> String {
let name = &if display_name == addr {
let name = if display_name == addr {
addr.to_owned()
} else {
format!("{display_name} ({addr})")
};
translated(context, StockMessage::SetupContactQRDescription)
.await
.replace1(name)
.replace1(&name)
}
/// Stock string: `Scan to join %1$s`.
@@ -1240,6 +1243,24 @@ pub(crate) async fn aeap_explanation_and_link(
.replace2(new_addr)
}
/// Text to put in the [`Qr::Backup`] rendered SVG image.
///
/// The default is "Scan to set up second device for <account name (account addr)>". The
/// account name and address are looked up from the context.
///
/// [`Qr::Backup`]: crate::qr::Qr::Backup
pub(crate) async fn backup_transfer_qr(context: &Context) -> Result<String> {
let contact = Contact::get_by_id(context, ContactId::SELF).await?;
let addr = contact.get_addr();
let full_name = match context.get_config(Config::Displayname).await? {
Some(name) if name != addr => format!("{name} ({addr})"),
_ => addr.to_string(),
};
Ok(translated(context, StockMessage::BackupTransferQr)
.await
.replace1(&full_name))
}
impl Context {
/// Set the stock string for the [StockMessage].
///
+2 -2
View File
@@ -109,8 +109,8 @@ impl Context {
Ok(())
}
// Add deleted qr-code token to the list of items to be synced
// so that the token also gets deleted on the other devices.
/// Adds deleted qr-code token to the list of items to be synced
/// so that the token also gets deleted on the other devices.
pub(crate) async fn sync_qr_code_token_deletion(
&self,
invitenumber: String,
+5
View File
@@ -40,6 +40,11 @@ pub const AVATAR_900x900_BYTES: &[u8] = include_bytes!("../test-data/image/avata
static CONTEXT_NAMES: Lazy<std::sync::RwLock<BTreeMap<u32, String>>> =
Lazy::new(|| std::sync::RwLock::new(BTreeMap::new()));
/// Manage multiple [`TestContext`]s in one place.
///
/// The main advantage is that the log records of the contexts will appear in the order they
/// occurred rather than grouped by context like would happen when you use separate
/// [`TestContext`]s without managing your own [`LogSink`].
pub struct TestContextManager {
log_tx: Sender<LogEvent>,
_log_sink: LogSink,
+3 -1
View File
@@ -421,7 +421,9 @@ impl Context {
DO UPDATE SET last_serial=excluded.last_serial, descr=excluded.descr",
paramsv![instance.id, status_update_serial, status_update_serial, descr],
).await?;
self.interrupt_smtp(InterruptInfo::new(false)).await;
self.scheduler
.interrupt_smtp(InterruptInfo::new(false))
.await;
}
Ok(())
}