Compare commits

...

243 Commits

Author SHA1 Message Date
link2xt
959ca06691 python: fail fast on the tests
Do not waste CI time running the rest of the tests
if CI is not going to be green anyway.
2023-03-22 12:35:27 +00:00
Floris Bruynooghe
e985588c6c ref(jsonrpc): Getting backup provider QR code now blocks (#4198)
This changes the JSON-RPC APIs to get a QR code from the backup
provider to block.  It means once you have a (blocking) call to
provide_backup() you can call get_backup_qr() or get_backup_qr_svg()
and they will block until the QR code is available.

Calling get_backup_qr() or get_backup_qr_svg() when there is no backup
provider will immediately error.
2023-03-22 12:45:38 +01:00
link2xt
7ec3a1a9a2 ci: fixup for artifact uploading in deltachat-rpc-server.yml 2023-03-21 23:17:15 +00:00
link2xt
19fa86b276 ci: remove android dependency from deltachat-rpc-server workflow 2023-03-21 22:21:05 +00:00
link2xt
c4657991c8 ci: build all deltachat-rpc-server binaries without NDK 2023-03-21 22:17:14 +00:00
link2xt
484aebdb16 smtp: disable buffering while running STARTTLS
Otherwise TLS setup fails on macOS and iOS with `errSSLClosedAbort`.
(<https://developer.apple.com/documentation/security/errsslclosedabort>)
2023-03-21 17:57:52 +00:00
Floris Bruynooghe
9c15cd5c8f Explicitly call Context::set_last_error in ffi (#4195)
This adds a result extension trait to explicitly set the last error,
which *should* be the default for the FFI.  Currently not touching all
APIs since that's potentially disruptive and we're close to a release.

The logging story is messy, as described in the doc comment.  We
should further clean this up and tidy up these APIs so it's more
obvious to people how to do the right thing.
2023-03-21 13:37:25 +01:00
Hocuri
8302d22622 Improve comment on write_lock() (#4134) 2023-03-21 11:49:14 +01:00
bjoern
034cde9289 typo: CollectionReceived (#4189) 2023-03-21 10:21:30 +01:00
link2xt
02455d8485 ci: upload deltachat-rpc-server binaries on release 2023-03-20 18:59:14 +00:00
Floris Bruynooghe
35f50a8965 feat: Pause IO for BackupProvider (#4182)
This makes the BackupProvider automatically invoke pause-io while it
is needed.

It needed to make the guard independent from the Context lifetime to
make this work.  Which is a bit sad.
2023-03-20 19:57:17 +01:00
link2xt
e04efdbd94 tox: quiet noisy message from black 2023-03-20 17:57:38 +00:00
Hocuri
57445eedb1 More accurate maybe_add_bcc_self device message text (#4175)
* More accurate maybe_add_bcc_self device message text

* changelog

* Update src/imex.rs

Co-authored-by: bjoern <r10s@b44t.com>

* Capitalize Send Copy to Self

---------

Co-authored-by: bjoern <r10s@b44t.com>
2023-03-20 12:54:16 +01:00
link2xt
a501f10756 Get rid of duplicate uuid dependency 2023-03-20 10:07:59 +00:00
link2xt
5d80d4788c Pause I/O in get_backup() 2023-03-20 10:24:59 +01:00
link2xt
0c02886005 Update human-panic, but disable color
Avoid pulling in new `anstream` dependency
2023-03-19 19:10:25 +00:00
link2xt
24856f3050 Merge branch 'flub/send-backup'
PR: <https://github.com/deltachat/deltachat-core-rust/pull/4007>
2023-03-19 15:21:59 +00:00
link2xt
8e6434068e Fix remaining cargo-deny warnings 2023-03-19 14:40:46 +00:00
link2xt
800d2b14a5 Add cargo-deny exceptions for old crates 2023-03-19 14:37:23 +00:00
B. Petersen
3a861d2f84 some doxygen fixes 2023-03-19 15:24:51 +01:00
dependabot[bot]
4ba00f7440 Merge pull request #4171 from deltachat/dependabot/cargo/axum-0.6.11 2023-03-19 13:30:54 +00:00
link2xt
40fc61da4f changelog: add link and date to the latest release 2023-03-19 12:07:55 +00:00
link2xt
eb0f896d57 Use scheduler.is_running() 2023-03-19 11:23:09 +00:00
link2xt
71bb89fac1 Merge remote-tracking branch 'origin/master' into flub/send-backup 2023-03-19 11:10:07 +00:00
link2xt
b89199db54 Merge branch 'flub/pause-io'
PR: <https://github.com/deltachat/deltachat-core-rust/pull/4138>
2023-03-19 11:06:01 +00:00
link2xt
e39429c2e3 rustfmt 2023-03-19 10:18:49 +00:00
link2xt
17de3d3236 Remove TODOs 2023-03-19 10:17:18 +00:00
link2xt
3177f9967d Add a comment aronud IMAP loop task handle 2023-03-19 10:16:43 +00:00
link2xt
81418d8ee5 Log error on pause guard drop without resuming instead of working around
I checked that tests still pass even if error! is replaced with panic!
2023-03-19 10:13:59 +00:00
link2xt
a2e7d914a0 Changelog fixup 2023-03-19 09:37:09 +00:00
Floris Bruynooghe
4bf38c0e29 clippy 2023-03-19 09:36:41 +00:00
Floris Bruynooghe
0079cd4766 Add changelog 2023-03-19 09:36:38 +00:00
Floris Bruynooghe
2c3b2b8c2d move pause to only exist on Scheduler 2023-03-19 09:36:03 +00:00
Floris Bruynooghe
52fa58a3ce No need for jsonrpc to do this manually 2023-03-19 09:36:03 +00:00
Floris Bruynooghe
32a7e5ed82 Remove requirement for stopping io for imex 2023-03-19 09:36:03 +00:00
Floris Bruynooghe
097113f01e fixup paused flag use 2023-03-19 09:36:03 +00:00
Floris Bruynooghe
1d42e4743f Allow pausing IO scheduler from inside core
To handle backups the UIs have to make sure they do stop the IO
scheduler and also don't accidentally restart it while working on it.
Since they have to call start_io from a bunch of locations this can be
a bit difficult to manage.

This introduces a mechanism for the core to pause IO for some time,
which is used by the imex function.  It interacts well with other
calls to dc_start_io() and dc_stop_io() making sure that when resumed
the scheduler will be running or not as the latest calls to them.

This was a little more invasive then hoped due to the scheduler.  The
additional abstraction of the scheduler on the context seems a nice
improvement though.
2023-03-19 09:36:03 +00:00
dependabot[bot]
5ecdea47db cargo: bump axum from 0.6.9 to 0.6.11
Bumps [axum](https://github.com/tokio-rs/axum) from 0.6.9 to 0.6.11.
- [Release notes](https://github.com/tokio-rs/axum/releases)
- [Changelog](https://github.com/tokio-rs/axum/blob/main/CHANGELOG.md)
- [Commits](https://github.com/tokio-rs/axum/compare/axum-v0.6.9...axum-v0.6.11)

---
updated-dependencies:
- dependency-name: axum
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-03-19 09:35:20 +00:00
dependabot[bot]
5b92b6355e Merge pull request #4168 from deltachat/dependabot/cargo/fuzz/libsqlite3-sys-0.25.2 2023-03-19 09:24:18 +00:00
link2xt
5eb7206b2d Format documentation comment for sync_qr_code_token_deletion 2023-03-19 00:10:45 +00:00
link2xt
a566fd6301 Upgrade async-smtp to 0.9.0
async-smtp does not implement read buffering anymore
and expects library user to implement it.

To implement read buffer, we wrap streams into BufStream
instead of BufWriter.
2023-03-18 21:26:39 +00:00
link2xt
3eadc86217 Update Rust in coredeps docker image to 1.68.0 2023-03-18 21:08:40 +00:00
dependabot[bot]
0a65081db0 Bump libsqlite3-sys from 0.24.2 to 0.25.2 in /fuzz
Bumps [libsqlite3-sys](https://github.com/rusqlite/rusqlite) from 0.24.2 to 0.25.2.
- [Release notes](https://github.com/rusqlite/rusqlite/releases)
- [Changelog](https://github.com/rusqlite/rusqlite/blob/master/Changelog.md)
- [Commits](https://github.com/rusqlite/rusqlite/compare/libsqlite3-sys-0.24.2...v0.25.2)

---
updated-dependencies:
- dependency-name: libsqlite3-sys
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-03-18 19:31:46 +00:00
link2xt
dd57854ee3 Increase Minimum Supported Rust Version to 1.64
It is required for clap_lex v0.3.2
and async_zip 0.0.11.
2023-03-18 19:30:11 +00:00
link2xt
b83b9db712 repl: print errors with causes 2023-03-18 14:37:50 +00:00
dignifiedquire
a59e72e7d8 update iroh 2023-03-17 23:41:36 +01:00
dignifiedquire
fd358617f5 feat: implement more detailed progress on sending 2023-03-17 23:37:00 +01:00
link2xt
b26a351786 Switch quinn to the main branch
It has Android fixes merged
2023-03-17 22:26:42 +00:00
link2xt
a32a3b8cca Construct HashMaps in provider database from array
This saves 450 lines according to `cargo llvm-lines --release`.
2023-03-17 18:30:37 +00:00
dignifiedquire
575b43d9a0 Merge remote-tracking branch 'origin/master' into flub/send-backup 2023-03-17 15:53:40 +01:00
Friedel Ziegelmayer
6c5654f584 fix: do not delete columns
This requires currently too much memory, crashing on larger instances
2023-03-17 15:35:08 +01:00
link2xt
0a5542a698 Log transfer rate on success 2023-03-17 10:45:43 +00:00
dignifiedquire
518bd19e96 fix: do not block transfer on db import 2023-03-17 11:29:27 +01:00
link2xt
edcc199461 Fix clippy::redundant-async-block warnings 2023-03-17 09:20:34 +00:00
link2xt
961e3ad7e2 Update spin 0.9.5->0.9.6 2023-03-17 00:00:37 +00:00
dignifiedquire
7a49e9401f fixup clippy & fmt 2023-03-16 17:53:27 +01:00
dignifiedquire
3701936129 Merge remote-tracking branch 'origin/master' into flub/send-backup 2023-03-16 17:50:00 +01:00
dignifiedquire
c02686b56e update iroh 2023-03-16 17:38:15 +01:00
link2xt
9a7ff9d2b1 Switch quinn to ecn-einval-fallback branch. 2023-03-16 16:25:46 +00:00
link2xt
f024909611 sql: replace empty paramsv![] with empty tuples 2023-03-15 22:20:40 +00:00
link2xt
8db64726ea sql: expect zero-column results from PRAGMA incremental_vacuum
The fact that `PRAGMA incremental_vacuum` may return a zero-column
SQLITE_ROW result is documented in `sqlite3_data_count()` documentation:
<https://www.sqlite.org/c3ref/data_count.html>

Previously successful auto_vacuum worked,
but resulted in a "Failed to run incremental vacuum" log.
2023-03-15 21:15:46 +00:00
link2xt
56f6d6849e Patch quinn to work on android 2023-03-15 12:45:58 +00:00
link2xt
cdd696db95 Delete expired messages using multiple SQL requests
With existing approach of constructing
the SQL query dynamically I get errors like this:
   ephemeral.rs:575: update failed: too many SQL variables: Error code 1: SQL error or missing database

In my case it is trying to delete 143658 messages.
This is the result of importing a Desktop backup
and enabling device auto-deletion on the phone.
Current SQLite limit is 32766 variables
as stated in <https://www.sqlite.org/limits.html>.
2023-03-15 12:22:27 +00:00
link2xt
cbc18ee5a4 Log connection errors 2023-03-14 18:49:14 +00:00
iequidoo
6b67f8bcfd Support non-persistent configuration with DELTACHAT_* env #3986
This way we can test some recently added config options that we don't want to expose in UI like
DeleteToTrash or SignUnencrypted. Note that persistent config options like DeleteToTrash should
remain anyway because they allow fine-grained (per-account) control. Having them matters for tests
also.
2023-03-14 12:57:06 -03:00
dignifiedquire
14521cfc2d improve address handling 2023-03-14 15:38:28 +01:00
link2xt
6fbcf67190 python: revert ruff C4 -> C40 change
It is an error in ruff 0.0.255, will be fixed in the next version:
a8c1915e2e
2023-03-14 09:37:03 +00:00
link2xt
73e7ee5c13 Build armv7 deltachat-rpc-server binaries without NDK 2023-03-13 23:09:25 +00:00
link2xt
90d2333818 python: update for latest ruff 0.0.255 2023-03-13 22:31:50 +00:00
link2xt
8d80aea5c8 Reduce the scope of unsafe blocks in FFI
Spawn tasks from safe functions to ensure there is no use-after-free.
2023-03-13 16:17:40 +00:00
link2xt
8936e9a416 Add dc_configure, dc_imex and dc_jsonrpc_request() fixes to changelog 2023-03-13 16:17:40 +00:00
link2xt
0afd3d595f Fix potential use-after-free in dc_jsonrpc_request() 2023-03-13 16:17:40 +00:00
link2xt
474faefca8 Increase dc_context_t reference count during dc_imex() 2023-03-13 16:17:40 +00:00
link2xt
30fef395b4 Increase dc_context_t reference count during dc_configure()
This fixes use-after-free in case dc_context_unref() is called
while the background process spawned by dc_configure() is still
running.

Corresponding regression test in Python can be run with
`pytest tests/test_1_online.py::test_configure_unref`.
2023-03-13 16:17:40 +00:00
link2xt
5805f99acd Test calling dc_context_unref() during dc_configure()
dc_configure() spawns a background configuration process.
It should increase the number of context references
so even if we unref the context, it is not dropped
until the end of the configuration process.

Currently running the test
with `pytest tests/test_1_online.py::test_configure_unref`
results in segmentation fault.
2023-03-13 16:17:40 +00:00
dignifiedquire
5e4807b7ac update patched default-net 2023-03-13 12:51:54 +01:00
B. Petersen
11ca698139 allow generated html being rendered in dark mode
`<meta name="color-scheme" content="light dark" />` is a hint to the browsers
that the page can be rendered in light as well as in dark mode
and that the browser can apply corresponding defaults.

as we do not add css colors on our own,
this is sufficient for letting generated html messasge being rendered
in dark mode.

cmp. https://drafts.csswg.org/css-color-adjust/#color-scheme-prop

closes #4146
2023-03-13 11:22:06 +01:00
B. Petersen
b14e59c5f3 "full message view" not needed because of footers
standard footers meanwhile go the "contact status",
so they are no longer a reason to trigger "full message view".

this was already discussed when the HTML view was introduced at #2125
however, forgotten to change when the "contact status" was added at #2218

this pr will result in a cleaner chat view
with less "Show Full Message..." buttons
and will also save some storage
(in fact, i came over that when reviewing #4129 )
2023-03-13 10:57:56 +01:00
link2xt
4d620afdb8 Enable clippy::explicit_into_iter_loop 2023-03-12 14:51:44 +00:00
link2xt
2bd7781276 Enable cloned_instead_of_copied clippy lint 2023-03-12 14:45:46 +00:00
link2xt
a432303576 Remove explicit .iter() and .iter_mut() calls in for loops 2023-03-12 14:06:42 +00:00
Asiel Díaz Benítez
b7e939e4d9 Merge pull request #4132 from deltachat/adb/rpc-server-update-readme
update deltachat-rpc-server/README.md
2023-03-12 06:10:56 -04:00
adbenitez
2151a24b7f update deltachat-rpc-server/README.md 2023-03-11 21:28:46 -05:00
link2xt
4411b883d7 Remove doxygen-dark-theme 2023-03-11 22:15:09 +00:00
B. Petersen
7d3fcd9a3c restore original Doxygen contrasts
the contrast was decreased at
https://github.com/deltachat/deltachat-core-rust/pull/4136 ,
there were also suggestions to fix it there,
but it was probably just forgotten :)

this pr increases the contrast
between code background and code font again to the level it was before
and also to what similar themes are doing.
2023-03-11 19:09:21 +01:00
link2xt
0ac61690cf Enable all features when running clippy 2023-03-11 12:57:38 +00:00
link2xt
28d9bec0b4 Patch default-net 2023-03-10 17:14:01 +00:00
link2xt
a853b8283a Upgrade to Rust 1.68.0 2023-03-10 01:30:57 +00:00
link2xt
6a394a0dc8 fixup: move changelog to unreleased section 2023-03-09 16:42:47 +00:00
Floris Bruynooghe
05e50ea787 Connect to all addresses the provider has
This uses a branch directly from iroh repo again
2023-03-09 16:49:34 +01:00
Floris Bruynooghe
02afacf989 clarify docs 2023-03-09 16:12:33 +01:00
Floris Bruynooghe
c7de4f66e7 Bind to 0.0.0.0 2023-03-09 15:34:15 +01:00
link2xt
00791157e2 Drop unused columns
We are currently using libsqlite3-sys 0.25.2,
corresponding to SQLcipher 4.5.2
and SQLite 3.39.2.

SQLite supports ALTER TABLE DROP COLUMN since version 3.35.0,
and it has received critical database corruption bugfixes in 3.35.5.
There have been no fixes to it between SQLite 3.36.0 and 3.41.0,
so it appears stable now.
2023-03-09 14:29:39 +00:00
link2xt
9e03f26992 deltachat-ffi: print causes for all errors 2023-03-09 11:36:14 +00:00
link2xt
de391155b1 Do not prepare the message explicitly in the test context
Explicit `prepare_msg` corresponding to `dc_prepare_msg`
is intended only for the case where the message is prepared
before the file is ready. It is not indented for calling
right before send_msg and requires that file path in the blobdir.

With explicit `prepare_msg` removed
all the tests still pass but there is no requirement
that the file is put into blobdir beforehand.
2023-03-09 09:37:40 +00:00
Sebastian Klähn
db28c1bdc4 Update doxygen for cffi docs (#4136)
fix styling
2023-03-08 21:35:15 +01:00
link2xt
d71bf1c4be Fix documentation on enabling REPL logs
As `repl` example was moved into `deltachat_repl` crate,
the name of the log target has changed.
2023-03-08 14:03:02 +00:00
Sebastian Klähn
ef1970b742 Also document private items for rust docs (#4137)
also document private items for rust docs
2023-03-08 14:13:07 +01:00
Asiel Díaz Benítez
4bb131e7e7 Merge pull request #4128 from deltachat/adb/add-golang-to-readme
add new golang bindings lib to README.md
2023-03-08 03:12:59 -05:00
Floris Bruynooghe
c9b8c5079b wording 2023-03-07 15:45:18 +01:00
Floris Bruynooghe
eec5ae96e8 Update docs and fix string allocation
The docs say you should always unref the string and NULL is never
returned.  The implementation should follow that.
2023-03-07 15:36:33 +01:00
Floris Bruynooghe
4b94eadf5e typo 2023-03-07 14:40:51 +01:00
Floris Bruynooghe
52a1886937 naming conventions!
they're hard
2023-03-07 14:40:01 +01:00
Floris Bruynooghe
9767f51c3d update .h file too 2023-03-07 14:13:42 +01:00
Floris Bruynooghe
6674b888cc Merge branch 'master' into flub/send-backup 2023-03-07 12:52:45 +01:00
Floris Bruynooghe
a5e6bd3e8e Do not require context for non-context methods
This follows the ffi style better.
2023-03-07 12:49:42 +01:00
Floris Bruynooghe
b6c24932a7 Apply typos from code review
Co-authored-by: Hocuri <hocuri@gmx.de>
2023-03-07 12:23:30 +01:00
link2xt
e39011a43b Release 1.111.0 2023-03-05 13:26:22 +00:00
adbenitez
8b8dcf61ef add new golang bindings lib to README.md 2023-03-04 23:14:45 -05:00
link2xt
3df5e6e9d3 Use "dep:" syntax to avoid creating dependency features
It is supported since Rust 1.60
2023-03-04 10:24:16 +00:00
link2xt
fe60b2dd2d Update scripts/anroid-rpc-server.sh
Ported changes from deltachat-android/scripts/ndk-make.sh
2023-03-04 09:35:56 +00:00
link2xt
260dbbd36f Optimize release builds and dependencies for size 2023-03-03 18:43:53 +00:00
link2xt
7e5a8714a0 Add scripts/codespell.sh and spellcheck 2023-03-03 18:40:36 +00:00
iequidoo
627cf20074 Housekeeping: delete the blobs backup dir (#4104) 2023-03-03 11:06:41 -03:00
Floris Bruynooghe
d73d56c399 bump testdir for windows bug workaround 2023-03-03 13:13:58 +01:00
Floris Bruynooghe
731e90f0d5 update cargo-deny 2023-03-03 12:53:43 +01:00
Floris Bruynooghe
e0a6c2ef54 Merge branch 'master' into flub/send-backup 2023-03-03 12:46:05 +01:00
Floris Bruynooghe
c38ae31f31 Configure cargo-deny fully
This adds more configuration to cargo-deny so that the output is not a
giant list of warnings hiding new entries.  Instead the config now
lists the things that would emit warnings.  It also adds -Dwarnings to
the CI job so that new warnings will be cleaned up: they can be added
to the config easily to fix the warnings.
2023-03-03 12:37:54 +01:00
Floris Bruynooghe
c5408e0561 Merge branch 'master' into flub/send-backup 2023-03-03 09:48:33 +01:00
Floris Bruynooghe
c1a2df91ac Fix typo in blob names
This is now tested properly too.
2023-03-02 21:53:13 +01:00
Floris Bruynooghe
da85c2412e fix iterator 2023-03-02 21:48:14 +01:00
Floris Bruynooghe
f5c334a1e4 ci: Make sure clippy script check everything
Also disable missing_docs_in_private_items as this is like 300+ errors
currently.
2023-03-02 21:33:20 +01:00
Simon Laux
3453aac27e jsonrpc: ts bindings: update .npmignore (#4119)
to remove files from the npm package that is not needed by end users.

#ignore-changelog
2023-03-02 13:57:17 +01:00
Floris Bruynooghe
d108f9b3e3 clippy 2023-03-02 11:47:03 +01:00
link2xt
9c48bf9d13 Upgrade rustyline to 11.0.0 2023-03-02 10:43:29 +00:00
Floris Bruynooghe
e3014a349c Merge branch 'master' into flub/send-backup 2023-03-02 11:35:17 +01:00
dependabot[bot]
04fa80b3bd Merge pull request #4120 from deltachat/dependabot/cargo/human-panic-1.1.1 2023-03-02 10:33:08 +00:00
dependabot[bot]
d7c8fc624a Merge pull request #4121 from deltachat/dependabot/cargo/async-native-tls-0.5.0 2023-03-02 10:32:51 +00:00
Floris Bruynooghe
9d88ef069e log some more 2023-03-02 11:21:05 +01:00
Floris Bruynooghe
155dff2813 renaming of upstream 2023-03-02 11:18:30 +01:00
Floris Bruynooghe
38d4ea8514 Use std::slice::Iter instead of manually tracking the offset 2023-03-02 11:15:02 +01:00
Floris Bruynooghe
6f24874eb8 Use a RAII guard to remove the db export 2023-03-02 10:58:39 +01:00
Floris Bruynooghe
2d20812652 some typos 2023-03-02 09:39:50 +01:00
dependabot[bot]
e866053070 cargo: bump async-native-tls from 0.4.0 to 0.5.0
Bumps [async-native-tls](https://github.com/async-email/async-native-tls) from 0.4.0 to 0.5.0.
- [Release notes](https://github.com/async-email/async-native-tls/releases)
- [Commits](https://github.com/async-email/async-native-tls/compare/v0.4.0...v0.5.0)

---
updated-dependencies:
- dependency-name: async-native-tls
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-03-02 02:32:53 +00:00
dependabot[bot]
81bacf9038 cargo: bump human-panic from 1.1.0 to 1.1.1
Bumps [human-panic](https://github.com/rust-cli/human-panic) from 1.1.0 to 1.1.1.
- [Release notes](https://github.com/rust-cli/human-panic/releases)
- [Changelog](https://github.com/rust-cli/human-panic/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-cli/human-panic/compare/v1.1.0...v1.1.1)

---
updated-dependencies:
- dependency-name: human-panic
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-03-02 02:32:35 +00:00
link2xt
4e166b1b4a Run cargo update 2023-03-01 22:42:03 +00:00
Floris Bruynooghe
5762fbb9a7 Allow JSON-RPC to get text of QR code as well
Desktop does use this as it allows reading QR codes as text from the
clipboard as well as copying the QR text to the clipboard instead of
showing the QR code.
2023-03-01 11:27:21 +01:00
link2xt
2dc04220b8 Simplify dc_jsonrpc_init 2023-02-28 22:46:24 +00:00
iequidoo
b149df1993 test_{delete,trash}_multiple_messages: Continue waiting for events if not all messages disappeared 2023-02-28 15:05:47 -03:00
iequidoo
e767d84bea Switch to DEFERRED transactions
We do not make all transactions
[IMMEDIATE](https://www.sqlite.org/lang_transaction.html#deferred_immediate_and_exclusive_transactions)
for more parallelism -- at least read transactions can be made DEFERRED to run in parallel
w/o any drawbacks. But if we make write transactions DEFERRED also w/o any external locking,
then they are upgraded from read to write ones on the first write statement. This has some
drawbacks:
- If there are other write transactions, we block the thread and the db connection until
  upgraded. Also if some reader comes then, it has to get next, less used connection with a
  worse per-connection page cache.
- If a transaction is blocked for more than busy_timeout, it fails with SQLITE_BUSY.
- Configuring busy_timeout is not the best way to manage transaction timeouts, we would
  prefer it to be integrated with Rust/tokio asyncs. Moreover, SQLite implements waiting
  using sleeps.
- If upon a successful upgrade to a write transaction the db has been modified by another
  one, the transaction has to be rolled back and retried. It is an extra work in terms of
  CPU/battery.
- Maybe minor, but we lose some fairness in servicing write transactions, i.e. we service
  them in the order of the first write statement, not in the order they come.
The only pro of making write transactions DEFERRED w/o the external locking is some
parallelism between them. Also we have an option to make write transactions IMMEDIATE, also
w/o the external locking. But then the most of cons above are still valid. Instead, if we
perform all write transactions under an async mutex, the only cons is losing some
parallelism for write transactions.
2023-02-28 15:05:47 -03:00
Simon Laux
fc019de18c jsonrpc: add get webxdc blob API getWebxdcBlob (#4070)
* jsonrpc: add get webxdc blob API `getWebxdcBlob`

* add info about path

* format
2023-02-27 17:46:13 +00:00
link2xt
c79ded1406 Fixes for get_webxdc_blob documentation 2023-02-27 14:36:57 +00:00
link2xt
d1d43e889a python: add more type annotations 2023-02-27 13:07:35 +00:00
link2xt
7bdf79dee5 python: do not shadow variables to fix pyright warnings 2023-02-27 13:07:35 +00:00
link2xt
a6b2c25d42 Move changelog to the unreleased section and add PR number 2023-02-27 08:26:34 +00:00
Asiel Díaz Benítez
210ec79872 Merge pull request #4097 from deltachat/adb/add-send-msg
add more advanced API to send a message
2023-02-27 00:32:18 -05:00
adbenitez
9f81299de0 rename DraftMessage to MessageData 2023-02-26 23:34:15 -05:00
adbenitez
f0a2ca7815 send_msg(): return only msg_id 2023-02-26 22:50:24 -05:00
link2xt
50d83ff063 Remove ResyncFolders job 2023-02-26 22:31:53 +00:00
link2xt
a2f1df052b Autoformat create-provider-data-rs.py 2023-02-26 21:27:21 +00:00
link2xt
0086232bbb Rename update-provider-database.py into create-provider-data-rs.py 2023-02-26 21:27:21 +00:00
link2xt
8e9bb8b06e Check provider database with CI
scripts/update-provider-database.sh checks out the provider database
and updates data.rs file,
making it easier to update.

CI uses this script to check that checked in data.rs
corresponds to the provider database repository.
2023-02-26 21:27:21 +00:00
link2xt
7e132b5383 Add cargo-deny config and CI 2023-02-26 20:52:53 +00:00
link2xt
8177070673 Set minimum TLS version to 1.2 2023-02-26 18:48:32 +00:00
B. Petersen
247bf5865d add countermitm and openpgp4fpr to "standard used"
noticed that this is missing
while working on https://github.com/deltachat/invite
2023-02-26 15:55:50 +01:00
Asiel Díaz Benítez
ad3c7136ec Merge branch 'master' into adb/add-send-msg 2023-02-26 05:14:12 -05:00
adbenitez
79026f93b6 add more advanced API to send a message 2023-02-26 05:13:20 -05:00
link2xt
eace8c5fee Note that 1.108.0 re-enables SMTP pipelining 2023-02-25 15:02:05 +00:00
B. Petersen
f8e86f4503 make slowest test faster
the slowest test was `test_modify_chat_disordered`;
due to several sleep(), it lasts at least 11 seconds.

however, many of the sleep() are not needed and were added
just to get things done that time.

this pr removes 7 superfluous sleeps,
making the test finish in about 3 seconds,
so that this test is no longer the bottleneck when running
`cargo test` on fast machines.
(this is not only useful to safe some seconds -
but also to notice degradations as of
https://github.com/deltachat/deltachat-core-rust/issues/4051#issuecomment-1436995166 )
2023-02-25 15:37:10 +01:00
iequidoo
d1923d68a5 Add a config option to sign all messages with Autocrypt header (#3986)
Although it does a little for security, it will help to protect from unwanted server-side
modifications and bugs. And now we have a time to test "multipart/signed" messages compatibility
with other MUAs.
2023-02-25 10:30:35 -03:00
iequidoo
89696582ad mimeparser: Handle headers from the signed part of unencrypted signed message
This makes DC compatible with "multipart/signed" messages thus allowing switching to them someday
from the current "multipart/mixed" unencrypted message format.
2023-02-25 10:30:35 -03:00
link2xt
992a6cbfc2 mimeparser: wrap try_decrypt() into block_in_place()
try_decrypt() is a CPU-bound task.
When called from async function,
it should be wrapped in tokio::task::spawn_blocking().
Using tokio::task::spawn_blocking() is difficult here
because of &mail, &private_keyring and &public_keyring borrows,
so we should at least use tokio::task::block_in_place()
to avoid blocking the executor.
2023-02-25 11:00:05 +00:00
link2xt
1450bf5483 Remove Job::delay_seconds()
It always returned 0 for added jobs.
2023-02-25 02:45:22 +00:00
link2xt
9f804c379c Remove always zero argument to Job::new() 2023-02-25 02:42:14 +00:00
link2xt
10e8bcb73e job: remove unused variable 2023-02-25 02:35:08 +00:00
link2xt
7f2ccfb168 job: remove match with a single branch 2023-02-25 02:33:47 +00:00
link2xt
3fc67de35e Stop saving and loading jobs.param column 2023-02-25 02:33:21 +00:00
link2xt
8f0d07b93c Make smeared timestamp creation non-async
Using atomic operations instead,
so create_smeared_timestamp() can be used in sync functions,
such as SQL transactions.
2023-02-25 01:10:56 +00:00
link2xt
0890b669fa Move RFC URLs in standards.md to the end
Also update Autodiscover URL
2023-02-24 23:47:51 +00:00
link2xt
c02c5ffe2c Format docs_wheels.yml with prettier 2023-02-24 22:31:04 +00:00
link2xt
064f806d90 Remove UpdateRecentQuota job 2023-02-24 17:49:08 +00:00
link2xt
b6336ce7e9 Add Unreleased changelog section 2023-02-24 17:46:45 +00:00
link2xt
aeadbb35f3 Remove empty API-Changes section 2023-02-24 17:46:35 +00:00
Floris Bruynooghe
0e06da22df fix symbol name 2023-02-22 18:51:17 +01:00
Floris Bruynooghe
5833a9b347 fix doc comments 2023-02-22 18:50:32 +01:00
Floris Bruynooghe
0ef8d57881 Merge branch 'master' into flub/send-backup 2023-02-22 18:15:23 +01:00
Floris Bruynooghe
fc64c33368 Use released version of sendme^Wiroh
This switches to a released version.  It has been renamed from sendme
to iroh.
2023-02-22 16:05:24 +01:00
Floris Bruynooghe
1b39be8a42 Merge branch 'master' into flub/send-backup 2023-02-22 15:54:23 +01:00
Floris Bruynooghe
a1e19e2c41 Merge branch 'master' into flub/send-backup 2023-02-20 17:39:52 +01:00
Floris Bruynooghe
b920db12c7 Split _wait and _unref
This also removes BackupProvider::join in favour of implementing
Future directly.  I wondered about implementing a FusedFutre to make
this a little safer but it would introduce a dependency on the futures
crate in deltachat-ffi which did not exist yet, so I didn't do that.
2023-02-20 15:56:05 +01:00
Floris Bruynooghe
73b90eee3e improve docs 2023-02-20 13:10:29 +01:00
Floris Bruynooghe
4637a28bf6 doc comment 2023-02-20 13:08:43 +01:00
Floris Bruynooghe
d0638c1542 typo 2023-02-20 13:05:11 +01:00
Floris Bruynooghe
788d3125a3 Do not save svg to file, just print qr text 2023-02-20 13:02:16 +01:00
Floris Bruynooghe
3c4ffc3550 Some fixes 2023-02-20 12:58:23 +01:00
Floris Bruynooghe
ada858f439 Improve comments, mostly ffi. and some renames 2023-02-20 12:48:43 +01:00
Floris Bruynooghe
f2570945c6 Don't reimplement qr::format_backup 2023-02-16 18:18:18 +01:00
Floris Bruynooghe
8072f78058 Do not emit ImexEvent From BlobDirIter
We no longer need that in the transfer case, that would give very
weird results.  This also means there is nothing imex-specific about
this anymore so move it to blobs.rs
2023-02-16 18:05:09 +01:00
Floris Bruynooghe
8ae0ee5a67 Merge branch 'master' into flub/send-backup 2023-02-16 17:19:31 +01:00
Floris Bruynooghe
a75d2b1c80 Create a blocking call for jsonrpc 2023-02-16 17:15:54 +01:00
Floris Bruynooghe
c48c2af7a1 Allow retrieval of backup QR on context
This enables being able to get the QR code without needing to have
access to the BackupProvider itself.  This is useful for the JSON-RPC
server.
2023-02-16 16:49:20 +01:00
Floris Bruynooghe
490a14c5ef Remove the need for a directory for db export
Plus on import use the context directory.  We can actually write there
just fine.
2023-02-16 16:06:41 +01:00
Floris Bruynooghe
dcce6ef50b Some docs 2023-02-16 15:19:44 +01:00
Floris Bruynooghe
7cf0820d2b diff 2023-02-16 14:56:18 +01:00
Floris Bruynooghe
0bae3caaff dear CI masters: i regret every trying to be clever 2023-02-16 14:55:24 +01:00
Floris Bruynooghe
bca0b256c9 goodness ci? 2023-02-16 14:52:05 +01:00
Floris Bruynooghe
a53d30c459 fixed another bug, try main again 2023-02-16 14:49:48 +01:00
Floris Bruynooghe
7a9f497aa7 why can't i see this action now? 2023-02-16 14:49:04 +01:00
Floris Bruynooghe
f9f9bc3efb yaml 2023-02-16 09:10:47 +01:00
Floris Bruynooghe
904990bf91 ugh, yaml syntax 2023-02-16 09:08:14 +01:00
Floris Bruynooghe
b2266ffca1 make the have a valid on spec at least so gh doesn't complain too much 2023-02-16 09:06:40 +01:00
Floris Bruynooghe
bb9a3d4b8e more bug hunting: disable most ci, point to branch 2023-02-16 09:00:27 +01:00
Floris Bruynooghe
e565e19b42 fix msrv in sendme 2023-02-15 16:01:39 +01:00
Floris Bruynooghe
41319c85c7 patch in previous revision of sendme
main broke rust 1.63 support :'(
2023-02-15 15:12:14 +01:00
Floris Bruynooghe
daf56804a5 use correct branch 2023-02-15 14:57:26 +01:00
Floris Bruynooghe
6f7a43804d Add changelog 2023-02-15 14:48:17 +01:00
Floris Bruynooghe
0ca76d36ef Merge branch 'master' into flub/send-backup 2023-02-15 14:46:57 +01:00
Floris Bruynooghe
ec5789997a back to master 2023-02-15 14:45:52 +01:00
Floris Bruynooghe
7a0d61bbb0 hey 2023-02-15 13:50:39 +01:00
Floris Bruynooghe
1c2461974d better way 2023-02-14 18:29:15 +01:00
Floris Bruynooghe
2a754744fe char, not chat 2023-02-14 18:03:39 +01:00
Floris Bruynooghe
b413593c43 hi 2023-02-14 17:39:58 +01:00
Floris Bruynooghe
c73edd7e21 oh 2023-02-14 17:20:25 +01:00
Floris Bruynooghe
a34a69d8e4 yes, ci fun 2023-02-14 17:15:52 +01:00
Floris Bruynooghe
020a9d33f6 new sendme for lower msrv 2023-02-14 15:49:21 +01:00
Floris Bruynooghe
19f6f89312 no let else :( 2023-02-14 13:29:55 +01:00
Floris Bruynooghe
d56e05a11a fixup doc links 2023-02-14 13:27:15 +01:00
Floris Bruynooghe
c379a4e5a7 use ProgressEmitter from sendme 2023-02-14 13:19:43 +01:00
Floris Bruynooghe
44c1efe4e4 Add jsonrpc support 2023-02-14 13:05:54 +01:00
Floris Bruynooghe
ff0d675082 Make getting backup use the ongoing process 2023-02-14 12:19:40 +01:00
Floris Bruynooghe
e1087b4145 translate the string for qr code 2023-02-14 12:07:02 +01:00
Floris Bruynooghe
323535584b implement ffi and use public sendme 2023-02-13 18:25:12 +01:00
Floris Bruynooghe
852adbe514 bits left over from master merge 2023-02-13 15:45:38 +01:00
Floris Bruynooghe
4c78553d90 Merge branch 'master' into flub/send-backup 2023-02-13 11:25:51 +01:00
Floris Bruynooghe
a31ae5297a Add to repl example 2023-02-10 18:35:47 +01:00
Floris Bruynooghe
e7792a0c65 clippy 2023-02-10 18:27:03 +01:00
Floris Bruynooghe
3c32de1859 Generate a QR code 2023-02-10 18:16:01 +01:00
Floris Bruynooghe
6a3fe3db92 fixup doc comments 2023-02-10 14:15:39 +01:00
Floris Bruynooghe
ac048c154d Add progress for provider
Fix progress for getter.  Maths.  It's hard.

Add test for progress.
2023-02-10 13:54:50 +01:00
Floris Bruynooghe
3f51a8ffc2 Some more doc comments 2023-02-10 10:48:10 +01:00
Floris Bruynooghe
2129b2b7a0 Add a ton of code for receiver-side progress 2023-02-09 18:09:16 +01:00
Floris Bruynooghe
3734fc25a7 update callback to take collection by ref 2023-02-09 10:02:18 +01:00
Floris Bruynooghe
05ddc13054 Use name prefixes so the db can not be spoofed by a blob 2023-02-07 18:21:46 +01:00
Floris Bruynooghe
716504b833 do not pull in sendme cli deps 2023-02-07 17:20:35 +01:00
Floris Bruynooghe
187861c3b2 Make stuff work. With test! 2023-02-07 17:18:34 +01:00
Floris Bruynooghe
0b075ac762 Stop after a transfer happened. 2023-02-06 14:58:08 +01:00
Floris Bruynooghe
a6c889ed5e Clean up files on errors 2023-02-02 18:11:12 +01:00
Floris Bruynooghe
ca1533b0e4 delete device messages 2023-02-02 17:47:41 +01:00
Floris Bruynooghe
3267596a30 handle the database 2023-02-02 17:43:12 +01:00
Floris Bruynooghe
5f29b93970 Start of get support and create new module. 2023-02-02 17:15:23 +01:00
Floris Bruynooghe
2a6a21c33a handle the ongoing process correctly 2023-02-01 17:53:23 +01:00
Floris Bruynooghe
059af398eb Allow decoding the QR code 2023-02-01 17:06:07 +01:00
Floris Bruynooghe
6044e5961b Send and receive backup over network using QR code
This adds functionality to send and receive a backup over the network
using a QR code.

The sender or provider prepares the backup, sets up a server that
waits for clients.  It provides a ticket in the form of a QR code
which contains connection and authentication information.

The receiver uses the QR code to connect to the provider and fetches
backup, restoring it locally.
2023-02-01 16:45:09 +01:00
131 changed files with 5681 additions and 2221 deletions

View File

@@ -20,7 +20,7 @@ jobs:
name: Rustfmt and Clippy
runs-on: ubuntu-latest
env:
RUSTUP_TOOLCHAIN: 1.67.1
RUSTUP_TOOLCHAIN: 1.68.0
steps:
- uses: actions/checkout@v3
- name: Install rustfmt and clippy
@@ -32,6 +32,25 @@ jobs:
- name: Run clippy
run: scripts/clippy.sh
cargo_deny:
name: cargo deny
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: EmbarkStudios/cargo-deny-action@v1
with:
arguments: --all-features --workspace
command: check
command-arguments: "-Dwarnings"
provider_database:
name: Check provider database
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Check provider database
run: scripts/update-provider-database.sh
docs:
name: Rust doc comments
runs-on: ubuntu-latest
@@ -53,19 +72,19 @@ jobs:
include:
# Currently used Rust version.
- os: ubuntu-latest
rust: 1.64.0
rust: 1.68.0
python: 3.9
- os: windows-latest
rust: 1.64.0
rust: 1.68.0
python: false # Python bindings compilation on Windows is not supported.
# Minimum Supported Rust Version = 1.63.0
# Minimum Supported Rust Version = 1.64.0
#
# Minimum Supported Python Version = 3.7
# This is the minimum version for which manylinux Python wheels are
# built.
- os: ubuntu-latest
rust: 1.63.0
rust: 1.64.0
python: 3.7
runs-on: ${{ matrix.os }}
steps:

View File

@@ -8,91 +8,40 @@ concurrency:
on:
workflow_dispatch:
release:
types: [published]
jobs:
# Build a version statically linked against musl libc
# to avoid problems with glibc version incompatibility.
build_static_linux:
name: Build deltachat-rpc-server for Linux
build_linux:
name: Cross-compile deltachat-rpc-server for x86_64, aarch64 and armv7 Linux
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v3
- name: Setup rust target
run: rustup target add x86_64-unknown-linux-musl
- name: Install musl-gcc
run: sudo apt install musl-tools
- name: Build
env:
RUSTFLAGS: "-C link-arg=-s"
run: cargo build --release --target x86_64-unknown-linux-musl -p deltachat-rpc-server --features vendored
- name: Upload binary
run: sh scripts/zig-rpc-server.sh
- name: Upload x86_64 binary
uses: actions/upload-artifact@v3
with:
name: deltachat-rpc-server-x86_64
path: target/x86_64-unknown-linux-musl/release/deltachat-rpc-server
if-no-files-found: error
build_aarch64_linux:
name: Cross-compile deltachat-rpc-server for aarch64 Linux
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v3
- name: Build
run: sh scripts/aarch64-unknown-linux-musl.sh
- name: Upload binary
- name: Upload aarch64 binary
uses: actions/upload-artifact@v3
with:
name: deltachat-rpc-server-aarch64
path: target/aarch64-unknown-linux-musl/release/deltachat-rpc-server
if-no-files-found: error
build_android:
name: Cross-compile deltachat-rpc-server for Android (armeabi-v7a, arm64-v8a, x86 and x86_64)
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v3
- uses: nttld/setup-ndk@v1
id: setup-ndk
with:
ndk-version: r21d
- name: Build
env:
ANDROID_NDK_ROOT: ${{ steps.setup-ndk.outputs.ndk-path }}
run: sh scripts/android-rpc-server.sh
- name: Upload binary
- name: Upload armv7 binary
uses: actions/upload-artifact@v3
with:
name: deltachat-rpc-server-android-armv7
path: target/armv7-linux-androideabi/release/deltachat-rpc-server
if-no-files-found: error
- name: Upload binary
uses: actions/upload-artifact@v3
with:
name: deltachat-rpc-server-android-aarch64
path: target/aarch64-linux-android/release/deltachat-rpc-server
if-no-files-found: error
- name: Upload binary
uses: actions/upload-artifact@v3
with:
name: deltachat-rpc-server-android-i686
path: target/i686-linux-android/release/deltachat-rpc-server
if-no-files-found: error
- name: Upload binary
uses: actions/upload-artifact@v3
with:
name: deltachat-rpc-server-android-x86_64
path: target/x86_64-linux-android/release/deltachat-rpc-server
name: deltachat-rpc-server-armv7
path: target/armv7-unknown-linux-musleabihf/release/deltachat-rpc-server
if-no-files-found: error
build_windows:
@@ -127,3 +76,51 @@ jobs:
name: deltachat-rpc-server-${{ matrix.artifact }}
path: target/${{ matrix.target}}/release/${{ matrix.path }}
if-no-files-found: error
publish:
name: Upload binaries to the release
needs: ["build_linux", "build_windows"]
permissions:
contents: write
runs-on: "ubuntu-latest"
steps:
- name: Download deltachat-rpc-server-x86_64
uses: "actions/download-artifact@v3"
with:
name: "deltachat-rpc-server-x86_64"
path: "dist/deltachat-rpc-server-x86_64"
- name: Download deltachat-rpc-server-aarch64
uses: "actions/download-artifact@v3"
with:
name: "deltachat-rpc-server-aarch64"
path: "dist/deltachat-rpc-server-aarch64"
- name: Download deltachat-rpc-server-armv7
uses: "actions/download-artifact@v3"
with:
name: "deltachat-rpc-server-armv7"
path: "dist/deltachat-rpc-server-armv7"
- name: Download deltachat-rpc-server-win32.exe
uses: "actions/download-artifact@v3"
with:
name: "deltachat-rpc-server-win32.exe"
path: "dist/deltachat-rpc-server-win32.exe"
- name: Download deltachat-rpc-server-win64.exe
uses: "actions/download-artifact@v3"
with:
name: "deltachat-rpc-server-win64.exe"
path: "dist/deltachat-rpc-server-win64.exe"
- name: List downloaded artifacts
run: ls -l dist/
- name: Upload binaries to the GitHub release
env:
GITHUB_TOKEN: "${{ secrets.GITHUB_TOKEN }}"
run: |
gh release upload ${{ github.ref_name }} \
--repo ${{ github.repository }} \
dist/deltachat-rpc-server-*

View File

@@ -22,7 +22,7 @@ jobs:
id: tag
uses: dawidd6/action-get-tag@v1
continue-on-error: true
- name: Get Pullrequest ID
- name: Get Pull Request ID
id: prepare
run: |
tag=${{ steps.tag.outputs.tag }}

View File

@@ -10,7 +10,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Get Pullrequest ID
- name: Get Pull Request ID
id: getid
run: |
export PULLREQUEST_ID=$(jq .number < $GITHUB_EVENT_PATH)

View File

@@ -78,7 +78,7 @@ jobs:
id: tag
uses: dawidd6/action-get-tag@v1
continue-on-error: true
- name: Get Pullrequest ID
- name: Get Pull Request ID
id: prepare
run: |
tag=${{ steps.tag.outputs.tag }}

View File

@@ -14,7 +14,7 @@ jobs:
- uses: actions/checkout@v3
- name: Build the documentation with cargo
run: |
cargo doc --package deltachat --no-deps
cargo doc --package deltachat --no-deps --document-private-items
- name: Upload to rs.delta.chat
uses: up9cloud/action-rsync@v1.3
env:

View File

@@ -1,5 +1,47 @@
# Changelog
## [Unreleased]
### Changes
- "full message view" not needed because of footers that go to contact status #4151
- Pick up system's light/dark mode in generated message HTML #4150
- Support non-persistent configuration with DELTACHAT_* env
- Print deltachat-repl errors with causes. #4166
- Increase MSRV to 1.64. #4167
- Core takes care of stopping and re-starting IO itself where needed,
e.g. during backup creation. It is no longer needed to call
dc_stop_io(). dc_start_io() can now be called at any time without
harm. #4138
- More accurate maybe_add_bcc_self device message text #4175
### Fixes
- Fix segmentation fault if `dc_context_unref()` is called during
background process spawned by `dc_configure()` or `dc_imex()`
or `dc_jsonrpc_instance_t` is unreferenced
during handling the JSON-RPC request. #4153
- Delete expired messages using multiple SQL requests. #4158
- Do not emit "Failed to run incremental vacuum" warnings on success. #4160
- Ability to send backup over network and QR code to setup second device #4007
- Disable buffering during STARTTLS setup. #4190
## [1.111.0] - 2023-03-05
### Changes
- Make smeared timestamp generation non-async. #4075
- Set minimum TLS version to 1.2. #4096
- Run `cargo-deny` in CI. #4101
- Check provider database with CI. #4099
- Switch to DEFERRED transactions #4100
### Fixes
- Do not block async task executor while decrypting the messages. #4079
- Housekeeping: delete the blobs backup dir #4123
### API-Changes
- jsonrpc: add more advanced API to send a message. #4097
- jsonrpc: add get webxdc blob API `getWebxdcBlob` #4070
## 1.110.0
### Changes
@@ -12,6 +54,7 @@
- Updated provider database.
- Disable DKIM-Checks again #4076
- Switch from "X.Y.Z" and "py-X.Y.Z" to "vX.Y.Z" tags. #4089
- mimeparser: handle headers from the signed part of unencrypted signed message #4013
### Fixes
- Start SQL transactions with IMMEDIATE behaviour rather than default DEFERRED one. #4063
@@ -52,6 +95,7 @@
- Prefer TLS over STARTTLS during autoconfiguration #4021
- Use SOCKS5 configuration for HTTP requests #4017
- Show non-deltachat emails by default for new installations #4019
- Re-enabled SMTP pipelining after disabling it in #4006
### Fixes
- Fix Securejoin for multiple devices on a joining side #3982
@@ -83,7 +127,7 @@
### Fixes
- Securejoin: Fix adding and handling Autocrypt-Gossip headers #3914
- fix verifier-by addr was empty string intead of None #3961
- fix verifier-by addr was empty string instead of None #3961
- Emit DC_EVENT_MSGS_CHANGED for DC_CHAT_ID_ARCHIVED_LINK when the number of archived chats with
unread messages increases #3959
- Fix Peerstate comparison #3962
@@ -184,7 +228,7 @@
- jsonrpc: Add async Python client #3734
### Fixes
- Make sure malformed messsages will never block receiving further messages anymore #3771
- Make sure malformed messages will never block receiving further messages anymore #3771
- strip leading/trailing whitespace from "Chat-Group-Name{,-Changed}:" headers content #3650
- Assume all Thunderbird users prefer encryption #3774
- refactor peerstate handling to ensure no duplicate peerstates #3776
@@ -336,7 +380,7 @@
- `importBackup()`
- `getMessageHtml()` #3671
- `miscGetStickerFolder` and `miscGetStickers` #3672
- breaking: jsonrpc: remove function `messageListGetMessageIds()`, it is replaced by `getMessageIds()` and `getMessageListItems()` the latter returns a new `MessageListItem` type, which is the now prefered way of using the message list.
- breaking: jsonrpc: remove function `messageListGetMessageIds()`, it is replaced by `getMessageIds()` and `getMessageListItems()` the latter returns a new `MessageListItem` type, which is the now preferred way of using the message list.
- jsonrpc: add type: #3641, #3645
- `MessageSearchResult`
- `Location`
@@ -362,7 +406,7 @@
- jsonrpc js client:
- Change package name from `deltachat-jsonrpc-client` to `@deltachat/jsonrpc-client`
- remove relative file dependency to it from `deltachat-node` (because it did not work anyway and broke the nix build of desktop)
- ci: add github ci action to upload it to our download server automaticaly on realease
- ci: add github ci action to upload it to our download server automatically on release
## 1.95.0
@@ -662,7 +706,7 @@
### Fixes
- node: throw error when getting context with an invalid account id
- node: throw error when instanciating a wrapper class on `null` (Context, Message, Chat, ChatList and so on)
- node: throw error when instantiating a wrapper class on `null` (Context, Message, Chat, ChatList and so on)
- use same contact-color if email address differ only in upper-/lowercase #3327
- repair encrypted mails "mixed up" by Google Workspace "Append footer" function #3315
@@ -750,7 +794,7 @@
- hopefully fix a bug where outgoing messages appear twice with Amazon SES #3077
- do not delete messages without Message-IDs as duplicates #3095
- assign replies from a different email address to the correct chat #3119
- assing outgoing private replies to the correct chat #3177
- assign outgoing private replies to the correct chat #3177
- start ephemeral timer when seen status is synchronized via IMAP #3122
- do not create empty contact requests with "setup changed" messages;
instead, send a "setup changed" message into all chats we share with the peer #3187
@@ -803,7 +847,7 @@
- don't watch Sent folder by default #3025
- use webxdc app name in chatlist/quotes/replies etc. #3027
- make it possible to cancel message sending by removing the message #3034,
this was previosuly removed in 1.71.0 #2939
this was previously removed in 1.71.0 #2939
- synchronize Seen flags only on watched folders to speed up
folder scanning #3041
- remove direct dependency on `byteorder` crate #3031
@@ -2025,7 +2069,7 @@
- #1043 avoid potential crashes in malformed From/Chat-Disposition... headers
- #1045 #1041 #1038 #1035 #1034 #1029 #1025 various cleanups and doc
improvments
improvements
## 1.0.0-beta.16
@@ -2104,7 +2148,7 @@
- trigger reconnect more often on imap error states. Should fix an
issue observed when trying to empty a folder. @hpk42
- un-split qr tests: we fixed qr-securejoin protocol flakyness
- un-split qr tests: we fixed qr-securejoin protocol flakiness
last weeks. @hpk42
## 1.0.0-beta.10
@@ -2142,7 +2186,7 @@
- fix moving self-sent messages, thanks @r10s, @flub, @hpk42
- fix flakyness/sometimes-failing verified/join-protocols,
- fix flakiness/sometimes-failing verified/join-protocols,
thanks @flub, @r10s, @hpk42
- fix reply-to-encrypted message to keep encryption
@@ -2161,7 +2205,7 @@
- fixes imap-protocol parsing bugs that lead to infinitely
repeated crashing while trying to receive messages with
a subjec that contained non-utf8. thanks @link2xt
a subject that contained non-utf8. thanks @link2xt
- fixed logic to find encryption subkey -- previously
delta chat would use the primary key for encryption
@@ -2269,3 +2313,6 @@
For a full list of changes, please see our closed Pull Requests:
https://github.com/deltachat/deltachat-core-rust/pulls?q=is%3Apr+is%3Aclosed
[unreleased]: https://github.com/deltachat/deltachat-core-rust/compare/v1.111.0...HEAD
[1.111.0]: https://github.com/deltachat/deltachat-core-rust/compare/v1.110.0...v1.111.0

2415
Cargo.lock generated

File diff suppressed because it is too large Load Diff

View File

@@ -1,9 +1,9 @@
[package]
name = "deltachat"
version = "1.110.0"
version = "1.111.0"
edition = "2021"
license = "MPL-2.0"
rust-version = "1.63"
rust-version = "1.64"
[profile.dev]
debug = 0
@@ -17,11 +17,17 @@ opt-level = 0
# This does not apply to crates in the workspace.
# <https://doc.rust-lang.org/cargo/reference/profiles.html#overrides>
[profile.dev.package."*"]
opt-level = 3
opt-level = "z"
[profile.release]
lto = true
panic = 'abort'
opt-level = "z"
[patch.crates-io]
default-net = { git = "https://github.com/dignifiedquire/default-net.git", branch="feat-android" }
quinn-udp = { git = "https://github.com/quinn-rs/quinn", branch="main" }
quinn-proto = { git = "https://github.com/quinn-rs/quinn", branch="main" }
[dependencies]
deltachat_derive = { path = "./deltachat_derive" }
@@ -31,8 +37,8 @@ ratelimit = { path = "./deltachat-ratelimit" }
anyhow = "1"
async-channel = "1.8.0"
async-imap = { git = "https://github.com/async-email/async-imap", branch = "master", default-features = false, features = ["runtime-tokio"] }
async-native-tls = { version = "0.4", default-features = false, features = ["runtime-tokio"] }
async-smtp = { version = "0.8", default-features = false, features = ["runtime-tokio"] }
async-native-tls = { version = "0.5", default-features = false, features = ["runtime-tokio"] }
async-smtp = { version = "0.9", default-features = false, features = ["runtime-tokio"] }
async_zip = { version = "0.0.9", default-features = false, features = ["deflate"] }
backtrace = "0.3"
base64 = "0.21"
@@ -47,6 +53,8 @@ futures-lite = "1.12.0"
hex = "0.4.0"
humansize = "2"
image = { version = "0.24.5", default-features=false, features = ["gif", "jpeg", "ico", "png", "pnm", "webp", "bmp"] }
# iroh = { version = "0.3.0", default-features = false }
iroh = { git = 'https://github.com/n0-computer/iroh', branch = "flub/ticket-multiple-addrs" }
kamadak-exif = "0.5"
lettre_email = { git = "https://github.com/deltachat/lettre", branch = "master" }
libc = "0.2"
@@ -94,6 +102,7 @@ log = "0.4"
pretty_env_logger = "0.4"
proptest = { version = "1", default-features = false, features = ["std"] }
tempfile = "3"
testdir = "0.7.2"
tokio = { version = "1", features = ["parking_lot", "rt-multi-thread", "macros"] }
[workspace]

View File

@@ -19,7 +19,7 @@ $ curl https://sh.rustup.rs -sSf | sh
Compile and run Delta Chat Core command line utility, using `cargo`:
```
$ RUST_LOG=repl=info cargo run -p deltachat-repl -- ~/deltachat-db
$ RUST_LOG=deltachat_repl=info cargo run -p deltachat-repl -- ~/deltachat-db
```
where ~/deltachat-db is the database file. Delta Chat will create it if it does not exist.
@@ -113,7 +113,7 @@ $ cargo build -p deltachat_ffi --release
- `DCC_MIME_DEBUG`: if set outgoing and incoming message will be printed
- `RUST_LOG=repl=info,async_imap=trace,async_smtp=trace`: enable IMAP and
- `RUST_LOG=deltachat_repl=info,async_imap=trace,async_smtp=trace`: enable IMAP and
SMTP tracing in addition to info messages.
### Expensive tests
@@ -170,7 +170,9 @@ Language bindings are available for:
- over cffi (legacy): \[[📂 source](./node) | [📦 npm](https://www.npmjs.com/package/deltachat-node) | [📚 docs](https://js.delta.chat)\]
- over jsonrpc built with napi.rs: \[[📂 source](https://github.com/deltachat/napi-jsonrpc) | [📦 npm](https://www.npmjs.com/package/@deltachat/napi-jsonrpc)\]
- **Python** \[[📂 source](./python) | [📦 pypi](https://pypi.org/project/deltachat) | [📚 docs](https://py.delta.chat)\]
- **Go**[^1] \[[📂 source](https://github.com/deltachat/go-deltachat/)\]
- **Go**
- over jsonrpc: \[[📂 source](https://github.com/deltachat/deltachat-rpc-client-go/)\]
- over cffi[^1]: \[[📂 source](https://github.com/deltachat/go-deltachat/)\]
- **Free Pascal**[^1] \[[📂 source](https://github.com/deltachat/deltachat-fp/)\]
- **Java** and **Swift** (contained in the Android/iOS repos)

View File

@@ -1,6 +1,6 @@
[package]
name = "deltachat_ffi"
version = "1.110.0"
version = "1.111.0"
description = "Deltachat FFI"
edition = "2018"
readme = "README.md"
@@ -17,7 +17,7 @@ crate-type = ["cdylib", "staticlib"]
deltachat = { path = "../", default-features = false }
deltachat-jsonrpc = { path = "../deltachat-jsonrpc", optional = true }
libc = "0.2"
human-panic = "1"
human-panic = { version = "1", default-features = false }
num-traits = "0.2"
serde_json = "1.0"
tokio = { version = "1", features = ["rt-multi-thread"] }
@@ -29,5 +29,5 @@ once_cell = "1.17.0"
[features]
default = ["vendored"]
vendored = ["deltachat/vendored"]
jsonrpc = ["deltachat-jsonrpc"]
jsonrpc = ["dep:deltachat-jsonrpc"]

View File

@@ -1,14 +1,24 @@
:root {
--accent: hsl(0 0% 85%);
}
@media (prefers-color-scheme: dark) {
:root {
--accent: hsl(0 0% 25%);
}
}
/* the code snippet frame, defaults to white which tends to get badly readable in combination with explaining text around */
div.fragment {
background-color: #e0e0e0;
background-color: var(--accent);
border: 0;
padding: 1em;
border-radius: 6px;
}
code {
background-color: #e0e0e0;
background-color: var(--accent);
padding-left: .5em;
padding-right: .5em;
border-radius: 6px;

View File

@@ -24,6 +24,7 @@ typedef struct _dc_provider dc_provider_t;
typedef struct _dc_event dc_event_t;
typedef struct _dc_event_emitter dc_event_emitter_t;
typedef struct _dc_jsonrpc_instance dc_jsonrpc_instance_t;
typedef struct _dc_backup_provider dc_backup_provider_t;
// Alias for backwards compatibility, use dc_event_emitter_t instead.
typedef struct _dc_event_emitter dc_accounts_event_emitter_t;
@@ -612,7 +613,7 @@ int dc_all_work_done (dc_context_t* context);
* While dc_configure() returns immediately,
* the started configuration-job may take a while.
*
* During configuration, #DC_EVENT_CONFIGURE_PROGRESS events are emmited;
* During configuration, #DC_EVENT_CONFIGURE_PROGRESS events are emitted;
* they indicate a successful configuration as well as errors
* and may be used to create a progress bar.
*
@@ -869,7 +870,7 @@ uint32_t dc_get_chat_id_by_contact_id (dc_context_t* context, uint32_t co
* @param context The context object as returned from dc_context_new().
* @param chat_id The chat ID to send the message to.
* @param msg The message object to send to the chat defined by the chat ID.
* On succcess, msg_id and state of the object are set up,
* On success, msg_id and state of the object are set up,
* The function does not take ownership of the object,
* so you have to free it using dc_msg_unref() as usual.
* @return The ID of the message that is being prepared.
@@ -880,7 +881,7 @@ uint32_t dc_prepare_msg (dc_context_t* context, uint32_t ch
/**
* Send a message defined by a dc_msg_t object to a chat.
*
* Sends the event #DC_EVENT_MSGS_CHANGED on succcess.
* Sends the event #DC_EVENT_MSGS_CHANGED on success.
* However, this does not imply, the message really reached the recipient -
* sending may be delayed e.g. due to network problems. However, from your
* view, you're done with the message. Sooner or later it will find its way.
@@ -909,7 +910,7 @@ uint32_t dc_prepare_msg (dc_context_t* context, uint32_t ch
* @param chat_id The chat ID to send the message to.
* If dc_prepare_msg() was called before, this parameter can be 0.
* @param msg The message object to send to the chat defined by the chat ID.
* On succcess, msg_id of the object is set up,
* On success, msg_id of the object is set up,
* The function does not take ownership of the object,
* so you have to free it using dc_msg_unref() as usual.
* @return The ID of the message that is about to be sent. 0 in case of errors.
@@ -926,7 +927,7 @@ uint32_t dc_send_msg (dc_context_t* context, uint32_t ch
* @param chat_id The chat ID to send the message to.
* If dc_prepare_msg() was called before, this parameter can be 0.
* @param msg The message object to send to the chat defined by the chat ID.
* On succcess, msg_id of the object is set up,
* On success, msg_id of the object is set up,
* The function does not take ownership of the object,
* so you have to free it using dc_msg_unref() as usual.
* @return The ID of the message that is about to be sent. 0 in case of errors.
@@ -937,7 +938,7 @@ uint32_t dc_send_msg_sync (dc_context_t* context, uint32
/**
* Send a simple text message a given chat.
*
* Sends the event #DC_EVENT_MSGS_CHANGED on succcess.
* Sends the event #DC_EVENT_MSGS_CHANGED on success.
* However, this does not imply, the message really reached the recipient -
* sending may be delayed e.g. due to network problems. However, from your
* view, you're done with the message. Sooner or later it will find its way.
@@ -1146,7 +1147,7 @@ void dc_set_draft (dc_context_t* context, uint32_t ch
* // not now and not when this code is executed again
* dc_add_device_msg(context, "update-123", NULL);
* } else {
* // welcome message was not added now, this is an oder installation,
* // welcome message was not added now, this is an older installation,
* // add a changelog
* dc_add_device_msg(context, "update-123", changelog_msg);
* }
@@ -2100,8 +2101,7 @@ dc_contact_t* dc_get_contact (dc_context_t* context, uint32_t co
/**
* Import/export things.
* During backup import/export IO must not be started,
* if needed stop IO using dc_accounts_stop_io() or dc_stop_io() first.
*
* What to do is defined by the _what_ parameter which may be one of the following:
*
* - **DC_IMEX_EXPORT_BACKUP** (11) - Export a backup to the directory given as `param1`
@@ -2295,6 +2295,7 @@ void dc_stop_ongoing_process (dc_context_t* context);
#define DC_QR_FPR_MISMATCH 220 // id=contact
#define DC_QR_FPR_WITHOUT_ADDR 230 // test1=formatted fingerprint
#define DC_QR_ACCOUNT 250 // text1=domain
#define DC_QR_BACKUP 251
#define DC_QR_WEBRTC_INSTANCE 260 // text1=domain, text2=instance pattern
#define DC_QR_ADDR 320 // id=contact
#define DC_QR_TEXT 330 // text1=text
@@ -2320,7 +2321,7 @@ void dc_stop_ongoing_process (dc_context_t* context);
* ask whether to verify the contact;
* if so, start the protocol with dc_join_securejoin().
*
* - DC_QR_ASK_VERIFYGROUP withdc_lot_t::text1=Group name:
* - DC_QR_ASK_VERIFYGROUP with dc_lot_t::text1=Group name:
* ask whether to join the group;
* if so, start the protocol with dc_join_securejoin().
*
@@ -2340,6 +2341,10 @@ void dc_stop_ongoing_process (dc_context_t* context);
* ask the user if they want to create an account on the given domain,
* if so, call dc_set_config_from_qr() and then dc_configure().
*
* - DC_QR_BACKUP:
* ask the user if they want to set up a new device.
* If so, pass the qr-code to dc_receive_backup().
*
* - DC_QR_WEBRTC_INSTANCE with dc_lot_t::text1=domain:
* ask the user if they want to use the given service for video chats;
* if so, call dc_set_config_from_qr().
@@ -2630,6 +2635,117 @@ char* dc_get_last_error (dc_context_t* context);
void dc_str_unref (char* str);
/**
* @class dc_backup_provider_t
*
* Set up another device.
*/
/**
* Creates an object for sending a backup to another device.
*
* The backup is sent to through a peer-to-peer channel which is bootstrapped
* by a QR-code. The backup contains the entire state of the account
* including credentials. This can be used to setup a new device.
*
* This is a blocking call as some preparations are made like e.g. exporting
* the database. Once this function returns, the backup is being offered to
* remote devices. To wait until one device received the backup, use
* dc_backup_provider_wait(). Alternatively abort the operation using
* dc_stop_ongoing_process().
*
* During execution of the job #DC_EVENT_IMEX_PROGRESS is sent out to indicate
* state and progress.
*
* @memberof dc_backup_provider_t
* @param context The context.
* @return Opaque object for sending the backup.
* On errors, NULL is returned and dc_get_last_error() returns an error that
* should be shown to the user.
*/
dc_backup_provider_t* dc_backup_provider_new (dc_context_t* context);
/**
* Returns the QR code text that will offer the backup to other devices.
*
* The QR code contains a ticket which will validate the backup and provide
* authentication for both the provider and the recipient.
*
* The scanning device should call the scanned text to dc_check_qr(). If
* dc_check_qr() returns DC_QR_BACKUP, the backup transfer can be started using
* dc_get_backup().
*
* @memberof dc_backup_provider_t
* @param backup_provider The backup provider object as created by
* dc_backup_provider_new().
* @return The text that should be put in the QR code.
* On errors an empty string is returned, NULL is never returned.
* the returned string must be released using dc_str_unref() after usage.
*/
char* dc_backup_provider_get_qr (const dc_backup_provider_t* backup_provider);
/**
* Returns the QR code SVG image that will offer the backup to other devices.
*
* This works like dc_backup_provider_qr() but returns the text of a rendered
* SVG image containing the QR code.
*
* @memberof dc_backup_provider_t
* @param backup_provider The backup provider object as created by
* dc_backup_provider_new().
* @return The QR code rendered as SVG.
* On errors an empty string is returned, NULL is never returned.
* the returned string must be released using dc_str_unref() after usage.
*/
char* dc_backup_provider_get_qr_svg (const dc_backup_provider_t* backup_provider);
/**
* Waits for the sending to finish.
*
* This is a blocking call and should only be called once.
*
* @memberof dc_backup_provider_t
* @param backup_provider The backup provider object as created by
* dc_backup_provider_new(). If NULL is given nothing is done.
*/
void dc_backup_provider_wait (dc_backup_provider_t* backup_provider);
/**
* Frees a dc_backup_provider_t object.
*
* @memberof dc_backup_provider_t
* @param backup_provider The backup provider object as created by
* dc_backup_provider_new().
*/
void dc_backup_provider_unref (dc_backup_provider_t* backup_provider);
/**
* Gets a backup offered by a dc_backup_provider_t object on another device.
*
* This function is called on a device that scanned the QR code offered by
* dc_backup_sender_qr() or dc_backup_sender_qr_svg(). Typically this is a
* different device than that which provides the backup.
*
* This call will block while the backup is being transferred and only
* complete on success or failure. Use dc_stop_ongoing_process() to abort it
* early.
*
* During execution of the job #DC_EVENT_IMEX_PROGRESS is sent out to indicate
* state and progress. The process is finished when the event emits either 0
* or 1000, 0 means it failed and 1000 means it succeeded. These events are
* for showing progress and informational only, success and failure is also
* shown in the return code of this function.
*
* @memberof dc_context_t
* @param context The context.
* @param qr The qr code text, dc_check_qr() must have returned DC_QR_BACKUP
* on this text.
* @return 0=failure, 1=success.
*/
int dc_receive_backup (dc_context_t* context, const char* qr);
/**
* @class dc_accounts_t
*
@@ -3184,7 +3300,7 @@ dc_lot_t* dc_chatlist_get_summary (const dc_chatlist_t* chatlist, siz
* it takes the chat ID and the message ID as returned by dc_chatlist_get_chat_id() and dc_chatlist_get_msg_id()
* as arguments. The chatlist object itself is not needed directly.
*
* This maybe useful if you convert the complete object into a different represenation
* This maybe useful if you convert the complete object into a different representation
* as done e.g. in the node-bindings.
* If you have access to the chatlist object in some way, using this function is not recommended,
* use dc_chatlist_get_summary() in this case instead.
@@ -6341,7 +6457,7 @@ void dc_event_unref(dc_event_t* event);
///
/// Used in status messages.
///
/// @deperecated 2022-09-10
/// @deprecated 2022-09-10
#define DC_STR_EPHEMERAL_MINUTE 77
/// "Message deletion timer is set to 1 hour."
@@ -6623,7 +6739,7 @@ void dc_event_unref(dc_event_t* event);
/// "You changed your email address from %1$s to %2$s.
/// If you now send a message to a group, contacts there will automatically
/// replace the old with your new address.\n\nIt's highly advised to set up
/// replace the old with your new address.\n\n It's highly advised to set up
/// your old email provider to forward all emails to your new email address.
/// Otherwise you might miss messages of contacts who did not get your new
/// address yet." + the link to the AEAP blog post

View File

@@ -1,4 +1,4 @@
#![warn(unused, clippy::all, clippy::missing_docs_in_private_items)]
#![warn(unused, clippy::all)]
#![allow(
non_camel_case_types,
non_snake_case,
@@ -28,9 +28,10 @@ use deltachat::constants::DC_MSG_ID_LAST_SPECIAL;
use deltachat::contact::{Contact, ContactId, Origin};
use deltachat::context::Context;
use deltachat::ephemeral::Timer as EphemeralTimer;
use deltachat::imex::BackupProvider;
use deltachat::key::DcKey;
use deltachat::message::MsgId;
use deltachat::qr_code_generator::get_securejoin_qr_svg;
use deltachat::qr_code_generator::{generate_backup_qr, get_securejoin_qr_svg};
use deltachat::reaction::{get_msg_reactions, send_reaction, Reactions};
use deltachat::stock_str::StockMessage;
use deltachat::stock_str::StockStrings;
@@ -292,12 +293,12 @@ pub unsafe extern "C" fn dc_set_stock_translation(
Some(id) => match ctx.set_stock_translation(id, msg).await {
Ok(()) => 1,
Err(err) => {
warn!(ctx, "set_stock_translation failed: {}", err);
warn!(ctx, "set_stock_translation failed: {err:#}");
0
}
},
None => {
warn!(ctx, "invalid stock message id {}", stock_id);
warn!(ctx, "invalid stock message id {stock_id}");
0
}
}
@@ -320,7 +321,7 @@ pub unsafe extern "C" fn dc_set_config_from_qr(
match qr::set_config_from_qr(ctx, &qr).await {
Ok(()) => 1,
Err(err) => {
error!(ctx, "Failed to create account from QR code: {}", err);
error!(ctx, "Failed to create account from QR code: {err:#}");
0
}
}
@@ -338,7 +339,7 @@ pub unsafe extern "C" fn dc_get_info(context: *const dc_context_t) -> *mut libc:
match ctx.get_info().await {
Ok(info) => render_info(info).unwrap_or_default().strdup(),
Err(err) => {
warn!(ctx, "failed to get info: {}", err);
warn!(ctx, "failed to get info: {err:#}");
"".strdup()
}
}
@@ -379,7 +380,7 @@ pub unsafe extern "C" fn dc_get_connectivity_html(
match ctx.get_connectivity_html().await {
Ok(html) => html.strdup(),
Err(err) => {
error!(ctx, "Failed to get connectivity html: {}", err);
error!(ctx, "Failed to get connectivity html: {err:#}");
"".strdup()
}
}
@@ -421,6 +422,10 @@ pub unsafe extern "C" fn dc_get_oauth2_url(
})
}
fn spawn_configure(ctx: Context) {
spawn(async move { ctx.configure().await.log_err(&ctx, "Configure failed") });
}
#[no_mangle]
pub unsafe extern "C" fn dc_configure(context: *mut dc_context_t) {
if context.is_null() {
@@ -429,8 +434,7 @@ pub unsafe extern "C" fn dc_configure(context: *mut dc_context_t) {
}
let ctx = &*context;
spawn(async move { ctx.configure().await.log_err(ctx, "Configure failed") });
spawn_configure(ctx.clone());
}
#[no_mangle]
@@ -1137,7 +1141,7 @@ pub unsafe extern "C" fn dc_get_draft(context: *mut dc_context_t, chat_id: u32)
}
Ok(None) => ptr::null_mut(),
Err(err) => {
error!(ctx, "Failed to get draft for chat #{}: {}", chat_id, err);
error!(ctx, "Failed to get draft for chat #{chat_id}: {err:#}");
ptr::null_mut()
}
}
@@ -1727,7 +1731,7 @@ pub unsafe extern "C" fn dc_get_chat_encrinfo(
.await
.map(|s| s.strdup())
.unwrap_or_else(|e| {
error!(ctx, "{}", e);
error!(ctx, "{e:#}");
ptr::null_mut()
})
})
@@ -1890,7 +1894,7 @@ pub unsafe extern "C" fn dc_resend_msgs(
let msg_ids = convert_and_prune_message_ids(msg_ids, msg_cnt);
if let Err(err) = block_on(chat::resend_msgs(ctx, &msg_ids)) {
error!(ctx, "Resending failed: {}", err);
error!(ctx, "Resending failed: {err:#}");
0
} else {
1
@@ -1931,14 +1935,11 @@ pub unsafe extern "C" fn dc_get_msg(context: *mut dc_context_t, msg_id: u32) ->
// C-core API returns empty messages, do the same
warn!(
ctx,
"dc_get_msg called with special msg_id={}, returning empty msg", msg_id
"dc_get_msg called with special msg_id={msg_id}, returning empty msg"
);
message::Message::default()
} else {
error!(
ctx,
"dc_get_msg could not retrieve msg_id {}: {}", msg_id, e
);
error!(ctx, "dc_get_msg could not retrieve msg_id {msg_id}: {e:#}");
return ptr::null_mut();
}
}
@@ -2131,7 +2132,7 @@ pub unsafe extern "C" fn dc_get_contact_encrinfo(
.await
.map(|s| s.strdup())
.unwrap_or_else(|e| {
error!(ctx, "{}", e);
error!(ctx, "{e:#}");
ptr::null_mut()
})
})
@@ -2153,7 +2154,7 @@ pub unsafe extern "C" fn dc_delete_contact(
match Contact::delete(ctx, contact_id).await {
Ok(_) => 1,
Err(err) => {
error!(ctx, "cannot delete contact: {}", err);
error!(ctx, "cannot delete contact: {err:#}");
0
}
}
@@ -2179,6 +2180,14 @@ pub unsafe extern "C" fn dc_get_contact(
})
}
fn spawn_imex(ctx: Context, what: imex::ImexMode, param1: String, passphrase: Option<String>) {
spawn(async move {
imex::imex(&ctx, what, param1.as_ref(), passphrase)
.await
.log_err(&ctx, "IMEX failed")
});
}
#[no_mangle]
pub unsafe extern "C" fn dc_imex(
context: *mut dc_context_t,
@@ -2202,11 +2211,7 @@ pub unsafe extern "C" fn dc_imex(
let ctx = &*context;
if let Some(param1) = to_opt_string_lossy(param1) {
spawn(async move {
imex::imex(ctx, what, param1.as_ref(), passphrase)
.await
.log_err(ctx, "IMEX failed")
});
spawn_imex(ctx.clone(), what, param1, passphrase);
} else {
eprintln!("dc_imex called without a valid directory");
}
@@ -2229,7 +2234,7 @@ pub unsafe extern "C" fn dc_imex_has_backup(
Err(err) => {
// do not bubble up error to the user,
// the ui will expect that the file does not exist or cannot be accessed
warn!(ctx, "dc_imex_has_backup: {}", err);
warn!(ctx, "dc_imex_has_backup: {err:#}");
ptr::null_mut()
}
}
@@ -2248,7 +2253,7 @@ pub unsafe extern "C" fn dc_initiate_key_transfer(context: *mut dc_context_t) ->
match imex::initiate_key_transfer(ctx).await {
Ok(res) => res.strdup(),
Err(err) => {
error!(ctx, "dc_initiate_key_transfer(): {}", err);
error!(ctx, "dc_initiate_key_transfer(): {err:#}");
ptr::null_mut()
}
}
@@ -2273,7 +2278,7 @@ pub unsafe extern "C" fn dc_continue_key_transfer(
{
Ok(()) => 1,
Err(err) => {
warn!(ctx, "dc_continue_key_transfer: {}", err);
warn!(ctx, "dc_continue_key_transfer: {err:#}");
0
}
}
@@ -2661,7 +2666,7 @@ pub unsafe fn dc_array_is_independent(
///
/// This is the structure behind [dc_chatlist_t] which is the opaque
/// structure representing a chatlist in the FFI API. It exists
/// because the FFI API has a refernce from the message to the
/// because the FFI API has a reference from the message to the
/// context, but the Rust API does not, so the FFI layer needs to glue
/// these together.
pub struct ChatlistWrapper {
@@ -2704,7 +2709,7 @@ pub unsafe extern "C" fn dc_chatlist_get_chat_id(
match ffi_list.list.get_chat_id(index) {
Ok(chat_id) => chat_id.to_u32(),
Err(err) => {
warn!(ctx, "get_chat_id failed: {}", err);
warn!(ctx, "get_chat_id failed: {err:#}");
0
}
}
@@ -2724,7 +2729,7 @@ pub unsafe extern "C" fn dc_chatlist_get_msg_id(
match ffi_list.list.get_msg_id(index) {
Ok(msg_id) => msg_id.map_or(0, |msg_id| msg_id.to_u32()),
Err(err) => {
warn!(ctx, "get_msg_id failed: {}", err);
warn!(ctx, "get_msg_id failed: {err:#}");
0
}
}
@@ -2805,7 +2810,7 @@ pub unsafe extern "C" fn dc_chatlist_get_context(
///
/// This is the structure behind [dc_chat_t] which is the opaque
/// structure representing a chat in the FFI API. It exists
/// because the FFI API has a refernce from the message to the
/// because the FFI API has a reference from the message to the
/// context, but the Rust API does not, so the FFI layer needs to glue
/// these together.
pub struct ChatWrapper {
@@ -2883,7 +2888,7 @@ pub unsafe extern "C" fn dc_chat_get_profile_image(chat: *mut dc_chat_t) -> *mut
Ok(Some(p)) => p.to_string_lossy().strdup(),
Ok(None) => ptr::null_mut(),
Err(err) => {
error!(ctx, "failed to get profile image: {:?}", err);
error!(ctx, "failed to get profile image: {err:#}");
ptr::null_mut()
}
}
@@ -3035,7 +3040,7 @@ pub unsafe extern "C" fn dc_chat_get_info_json(
let chat = match chat::Chat::load_from_db(ctx, ChatId::new(chat_id)).await {
Ok(chat) => chat,
Err(err) => {
error!(ctx, "dc_get_chat_info_json() failed to load chat: {}", err);
error!(ctx, "dc_get_chat_info_json() failed to load chat: {err:#}");
return "".strdup();
}
};
@@ -3044,7 +3049,7 @@ pub unsafe extern "C" fn dc_chat_get_info_json(
Err(err) => {
error!(
ctx,
"dc_get_chat_info_json() failed to get chat info: {}", err
"dc_get_chat_info_json() failed to get chat info: {err:#}"
);
return "".strdup();
}
@@ -3061,7 +3066,7 @@ pub unsafe extern "C" fn dc_chat_get_info_json(
///
/// This is the structure behind [dc_msg_t] which is the opaque
/// structure representing a message in the FFI API. It exists
/// because the FFI API has a refernce from the message to the
/// because the FFI API has a reference from the message to the
/// context, but the Rust API does not, so the FFI layer needs to glue
/// these together.
pub struct MessageWrapper {
@@ -3283,7 +3288,7 @@ pub unsafe extern "C" fn dc_msg_get_webxdc_info(msg: *mut dc_msg_t) -> *mut libc
let info = match ffi_msg.message.get_webxdc_info(ctx).await {
Ok(info) => info,
Err(err) => {
error!(ctx, "dc_msg_get_webxdc_info() failed to get info: {}", err);
error!(ctx, "dc_msg_get_webxdc_info() failed to get info: {err:#}");
return "".strdup();
}
};
@@ -3814,7 +3819,7 @@ pub unsafe extern "C" fn dc_msg_force_plaintext(msg: *mut dc_msg_t) {
///
/// This is the structure behind [dc_contact_t] which is the opaque
/// structure representing a contact in the FFI API. It exists
/// because the FFI API has a refernce from the message to the
/// because the FFI API has a reference from the message to the
/// context, but the Rust API does not, so the FFI layer needs to glue
/// these together.
pub struct ContactWrapper {
@@ -4138,6 +4143,116 @@ pub unsafe extern "C" fn dc_str_unref(s: *mut libc::c_char) {
libc::free(s as *mut _)
}
pub struct BackupProviderWrapper {
context: *const dc_context_t,
provider: BackupProvider,
}
pub type dc_backup_provider_t = BackupProviderWrapper;
#[no_mangle]
pub unsafe extern "C" fn dc_backup_provider_new(
context: *mut dc_context_t,
) -> *mut dc_backup_provider_t {
if context.is_null() {
eprintln!("ignoring careless call to dc_backup_provider_new()");
return ptr::null_mut();
}
let ctx = &*context;
block_on(BackupProvider::prepare(ctx))
.map(|provider| BackupProviderWrapper {
context: ctx,
provider,
})
.map(|ffi_provider| Box::into_raw(Box::new(ffi_provider)))
.log_err(ctx, "BackupProvider failed")
.context("BackupProvider failed")
.set_last_error(ctx)
.unwrap_or(ptr::null_mut())
}
#[no_mangle]
pub unsafe extern "C" fn dc_backup_provider_get_qr(
provider: *const dc_backup_provider_t,
) -> *mut libc::c_char {
if provider.is_null() {
eprintln!("ignoring careless call to dc_backup_provider_qr");
return "".strdup();
}
let ffi_provider = &*provider;
let ctx = &*ffi_provider.context;
deltachat::qr::format_backup(&ffi_provider.provider.qr())
.log_err(ctx, "BackupProvider get_qr failed")
.context("BackupProvider get_qr failed")
.set_last_error(ctx)
.unwrap_or_default()
.strdup()
}
#[no_mangle]
pub unsafe extern "C" fn dc_backup_provider_get_qr_svg(
provider: *const dc_backup_provider_t,
) -> *mut libc::c_char {
if provider.is_null() {
eprintln!("ignoring careless call to dc_backup_provider_qr_svg()");
return "".strdup();
}
let ffi_provider = &*provider;
let ctx = &*ffi_provider.context;
let provider = &ffi_provider.provider;
block_on(generate_backup_qr(ctx, &provider.qr()))
.log_err(ctx, "BackupProvider get_qr_svg failed")
.context("BackupProvider get_qr_svg failed")
.set_last_error(ctx)
.unwrap_or_default()
.strdup()
}
#[no_mangle]
pub unsafe extern "C" fn dc_backup_provider_wait(provider: *mut dc_backup_provider_t) {
if provider.is_null() {
eprintln!("ignoring careless call to dc_backup_provider_wait()");
return;
}
let ffi_provider = &mut *provider;
let ctx = &*ffi_provider.context;
let provider = &mut ffi_provider.provider;
block_on(provider)
.log_err(ctx, "Failed to await BackupProvider")
.context("Failed to await BackupProvider")
.set_last_error(ctx)
.ok();
}
#[no_mangle]
pub unsafe extern "C" fn dc_backup_provider_unref(provider: *mut dc_backup_provider_t) {
drop(Box::from_raw(provider));
}
#[no_mangle]
pub unsafe extern "C" fn dc_receive_backup(
context: *mut dc_context_t,
qr: *const libc::c_char,
) -> libc::c_int {
if context.is_null() {
eprintln!("ignoring careless call to dc_receive_backup()");
return 0;
}
let ctx = &*context;
let qr_text = to_string_lossy(qr);
let qr = match block_on(qr::check_qr(ctx, &qr_text)).log_err(ctx, "Invalid QR code") {
Ok(qr) => qr,
Err(_) => return 0,
};
spawn(async move {
imex::get_backup(ctx, qr)
.await
.log_err(ctx, "Get backup failed")
.ok();
});
1
}
trait ResultExt<T, E> {
/// Like `log_err()`, but:
/// - returns the default value instead of an Err value.
@@ -4151,13 +4266,63 @@ impl<T: Default, E: std::fmt::Display> ResultExt<T, E> for Result<T, E> {
match self {
Ok(t) => t,
Err(err) => {
error!(context, "{}: {}", message, err);
error!(context, "{message}: {err:#}");
Default::default()
}
}
}
}
trait ResultLastError<T, E>
where
E: std::fmt::Display,
{
/// Sets this `Err` value using [`Context::set_last_error`].
///
/// Normally each FFI-API *should* call this if it handles an error from the Rust API:
/// errors which need to be reported to users in response to an API call need to be
/// propagated up the Rust API and at the FFI boundary need to be stored into the "last
/// error" so the FFI users can retrieve an appropriate error message on failure. Often
/// you will want to combine this with a call to [`LogExt::log_err`].
///
/// Since historically calls to the `deltachat::log::error!()` macro were (and sometimes
/// still are) shown as error toasts to the user, this macro also calls
/// [`Context::set_last_error`]. It is preferable however to rely on normal error
/// propagation in Rust code however and only use this `ResultExt::set_last_error` call
/// in the FFI layer.
///
/// # Example
///
/// Fully handling an error in the FFI code looks like this currently:
///
/// ```no_compile
/// some_dc_rust_api_call_returning_result()
/// .log_err(&context, "My API call failed")
/// .context("My API call failed")
/// .set_last_error(&context)
/// .unwrap_or_default()
/// ```
///
/// As shows it is a shame the `.log_err()` call currently needs a message instead of
/// relying on an implicit call to the [`anyhow::Context`] call if needed. This stems
/// from a time before we fully embraced anyhow. Some day we'll also fix that.
///
/// [`Context::set_last_error`]: context::Context::set_last_error
fn set_last_error(self, context: &context::Context) -> Result<T, E>;
}
impl<T, E> ResultLastError<T, E> for Result<T, E>
where
E: std::fmt::Display,
{
fn set_last_error(self, context: &context::Context) -> Result<T, E> {
if let Err(ref err) = self {
context.set_last_error(&format!("{err:#}"));
}
self
}
}
trait ResultNullableExt<T> {
fn into_raw(self) -> *mut T;
}
@@ -4607,33 +4772,22 @@ mod jsonrpc {
return ptr::null_mut();
}
let cmd_api =
deltachat_jsonrpc::api::CommandApi::from_arc((*account_manager).inner.clone());
let account_manager = &*account_manager;
let events = block_on(account_manager.read()).get_event_emitter();
let cmd_api = deltachat_jsonrpc::api::CommandApi::from_arc(account_manager.inner.clone());
let (request_handle, receiver) = RpcClient::new();
let request_handle2 = request_handle.clone();
let handle = RpcSession::new(request_handle, cmd_api);
let handle = RpcSession::new(request_handle.clone(), cmd_api);
let events = block_on({
async {
let am = (*account_manager).inner.clone();
let ev = am.read().await.get_event_emitter();
drop(am);
ev
}
});
let event_thread = spawn({
async move {
while let Some(event) = events.recv().await {
let event = event_to_json_rpc_notification(event);
request_handle2
.send_notification("event", Some(event))
.await?;
}
let res: Result<(), anyhow::Error> = Ok(());
res
let event_thread = spawn(async move {
while let Some(event) = events.recv().await {
let event = event_to_json_rpc_notification(event);
request_handle
.send_notification("event", Some(event))
.await?;
}
let res: Result<(), anyhow::Error> = Ok(());
res
});
let instance = dc_jsonrpc_instance_t {
@@ -4655,6 +4809,12 @@ mod jsonrpc {
drop(Box::from_raw(jsonrpc_instance));
}
fn spawn_handle_jsonrpc_request(handle: RpcSession<CommandApi>, request: String) {
spawn(async move {
handle.handle_incoming(&request).await;
});
}
#[no_mangle]
pub unsafe extern "C" fn dc_jsonrpc_request(
jsonrpc_instance: *mut dc_jsonrpc_instance_t,
@@ -4665,12 +4825,9 @@ mod jsonrpc {
return;
}
let api = &*jsonrpc_instance;
let handle = &api.handle;
let handle = &(*jsonrpc_instance).handle;
let request = to_string_lossy(request);
spawn(async move {
handle.handle_incoming(&request).await;
});
spawn_handle_jsonrpc_request(handle.clone(), request);
}
#[no_mangle]

View File

@@ -14,6 +14,8 @@ use crate::summary::{Summary, SummaryPrefix};
/// eg. by chatlist.get_summary() or dc_msg_get_summary().
///
/// *Lot* is used in the meaning *heap* here.
// The QR code grew too large. So be it.
#[allow(clippy::large_enum_variant)]
#[derive(Debug)]
pub enum Lot {
Summary(Summary),
@@ -47,6 +49,7 @@ impl Lot {
Qr::FprMismatch { .. } => None,
Qr::FprWithoutAddr { fingerprint, .. } => Some(fingerprint),
Qr::Account { domain } => Some(domain),
Qr::Backup { .. } => None,
Qr::WebrtcInstance { domain, .. } => Some(domain),
Qr::Addr { draft, .. } => draft.as_deref(),
Qr::Url { url } => Some(url),
@@ -98,6 +101,7 @@ impl Lot {
Qr::FprMismatch { .. } => LotState::QrFprMismatch,
Qr::FprWithoutAddr { .. } => LotState::QrFprWithoutAddr,
Qr::Account { .. } => LotState::QrAccount,
Qr::Backup { .. } => LotState::QrBackup,
Qr::WebrtcInstance { .. } => LotState::QrWebrtcInstance,
Qr::Addr { .. } => LotState::QrAddr,
Qr::Url { .. } => LotState::QrUrl,
@@ -122,6 +126,7 @@ impl Lot {
Qr::FprMismatch { contact_id } => contact_id.unwrap_or_default().to_u32(),
Qr::FprWithoutAddr { .. } => Default::default(),
Qr::Account { .. } => Default::default(),
Qr::Backup { .. } => Default::default(),
Qr::WebrtcInstance { .. } => Default::default(),
Qr::Addr { contact_id, .. } => contact_id.to_u32(),
Qr::Url { .. } => Default::default(),
@@ -170,6 +175,8 @@ pub enum LotState {
/// text1=domain
QrAccount = 250,
QrBackup = 251,
/// text1=domain, text2=instance pattern
QrWebrtcInstance = 260,

View File

@@ -1,6 +1,6 @@
[package]
name = "deltachat-jsonrpc"
version = "1.110.0"
version = "1.111.0"
description = "DeltaChat JSON-RPC API"
edition = "2021"
default-run = "deltachat-jsonrpc-server"
@@ -21,14 +21,15 @@ log = "0.4"
async-channel = { version = "1.8.0" }
futures = { version = "0.3.26" }
serde_json = "1.0.91"
yerpc = { version = "^0.4.0", features = ["anyhow_expose"] }
yerpc = { version = "0.4.3", features = ["anyhow_expose"] }
typescript-type-def = { version = "0.5.5", features = ["json_value"] }
tokio = { version = "1.25.0" }
sanitize-filename = "0.4"
walkdir = "2.3.2"
base64 = "0.21"
# optional dependencies
axum = { version = "0.5.9", optional = true, features = ["ws"] }
axum = { version = "0.6.11", optional = true, features = ["ws"] }
env_logger = { version = "0.10.0", optional = true }
[dev-dependencies]
@@ -37,5 +38,5 @@ tokio = { version = "1.25.0", features = ["full", "rt-multi-thread"] }
[features]
default = ["vendored"]
webserver = ["env_logger", "axum", "tokio/full", "yerpc/support-axum"]
webserver = ["dep:env_logger", "dep:axum", "tokio/full", "yerpc/support-axum"]
vendored = ["deltachat/vendored"]

View File

@@ -35,7 +35,7 @@ The server can be configured with environment variables:
|`DC_PORT`|`20808`|port to listen on|
|`DC_ACCOUNTS_PATH`|`./accounts`|path to storage directory|
If you are targetting other architectures (like KaiOS or Android), the webserver binary can be cross-compiled easily with [rust-cross](https://github.com/cross-rs/cross):
If you are targeting other architectures (like KaiOS or Android), the webserver binary can be cross-compiled easily with [rust-cross](https://github.com/cross-rs/cross):
```sh
cross build --features=webserver --target armv7-linux-androideabi --release

View File

@@ -4,6 +4,7 @@ use std::{collections::HashMap, str::FromStr};
use anyhow::{anyhow, bail, ensure, Context, Result};
pub use deltachat::accounts::Accounts;
use deltachat::qr::Qr;
use deltachat::{
chat::{
self, add_contact_to_chat, forward_msgs, get_chat_media, get_chat_msgs, get_chat_msgs_ex,
@@ -22,14 +23,15 @@ use deltachat::{
},
provider::get_provider_info,
qr,
qr_code_generator::get_securejoin_qr_svg,
qr_code_generator::{generate_backup_qr, get_securejoin_qr_svg},
reaction::send_reaction,
securejoin,
stock_str::StockMessage,
webxdc::StatusUpdateSerial,
};
use sanitize_filename::is_sanitized;
use tokio::{fs, sync::RwLock};
use tokio::fs;
use tokio::sync::{watch, Mutex, RwLock};
use walkdir::WalkDir;
use yerpc::rpc;
@@ -41,6 +43,7 @@ use types::account::Account;
use types::chat::FullChat;
use types::chat_list::ChatListEntry;
use types::contact::ContactObject;
use types::message::MessageData;
use types::message::MessageObject;
use types::provider_info::ProviderInfo;
use types::webxdc::WebxdcMessageInfo;
@@ -56,21 +59,45 @@ use self::types::{
use crate::api::types::chat_list::{get_chat_list_item_by_id, ChatListItemFetchResult};
use crate::api::types::qr::QrObject;
#[derive(Debug)]
struct AccountState {
/// The Qr code for current [`CommandApi::provide_backup`] call.
///
/// If there currently is a call to [`CommandApi::provide_backup`] this will be
/// `Pending` or `Ready`, otherwise `NoProvider`.
backup_provider_qr: watch::Sender<ProviderQr>,
}
impl Default for AccountState {
fn default() -> Self {
let (tx, _rx) = watch::channel(ProviderQr::NoProvider);
Self {
backup_provider_qr: tx,
}
}
}
#[derive(Clone, Debug)]
pub struct CommandApi {
pub(crate) accounts: Arc<RwLock<Accounts>>,
states: Arc<Mutex<BTreeMap<u32, AccountState>>>,
}
impl CommandApi {
pub fn new(accounts: Accounts) -> Self {
CommandApi {
accounts: Arc::new(RwLock::new(accounts)),
states: Arc::new(Mutex::new(BTreeMap::new())),
}
}
#[allow(dead_code)]
pub fn from_arc(accounts: Arc<RwLock<Accounts>>) -> Self {
CommandApi { accounts }
CommandApi {
accounts,
states: Arc::new(Mutex::new(BTreeMap::new())),
}
}
async fn get_context(&self, id: u32) -> Result<deltachat::context::Context> {
@@ -82,6 +109,38 @@ impl CommandApi {
.ok_or_else(|| anyhow!("account with id {} not found", id))?;
Ok(sc)
}
async fn with_state<F, T>(&self, id: u32, with_state: F) -> T
where
F: FnOnce(&AccountState) -> T,
{
let mut states = self.states.lock().await;
let state = states.entry(id).or_insert_with(Default::default);
with_state(state)
}
async fn inner_get_backup_qr(&self, account_id: u32) -> Result<Qr> {
let mut receiver = self
.with_state(account_id, |state| state.backup_provider_qr.subscribe())
.await;
let val: ProviderQr = receiver.borrow_and_update().clone();
match val {
ProviderQr::NoProvider => bail!("No backup being provided"),
ProviderQr::Pending => loop {
if receiver.changed().await.is_err() {
bail!("No backup being provided (account state dropped)");
}
let val: ProviderQr = receiver.borrow().clone();
match val {
ProviderQr::NoProvider => bail!("No backup being provided"),
ProviderQr::Pending => continue,
ProviderQr::Ready(qr) => break Ok(qr),
};
},
ProviderQr::Ready(qr) => Ok(qr),
}
}
}
#[rpc(all_positional, ts_outdir = "typescript/generated")]
@@ -114,7 +173,13 @@ impl CommandApi {
}
async fn remove_account(&self, account_id: u32) -> Result<()> {
self.accounts.write().await.remove_account(account_id).await
self.accounts
.write()
.await
.remove_account(account_id)
.await?;
self.states.lock().await.remove(&account_id);
Ok(())
}
async fn get_all_account_ids(&self) -> Vec<u32> {
@@ -1324,16 +1389,13 @@ impl CommandApi {
passphrase: Option<String>,
) -> Result<()> {
let ctx = self.get_context(account_id).await?;
ctx.stop_io().await;
let result = imex::imex(
imex::imex(
&ctx,
imex::ImexMode::ExportBackup,
destination.as_ref(),
passphrase,
)
.await;
ctx.start_io().await;
result
.await
}
async fn import_backup(
@@ -1352,6 +1414,75 @@ impl CommandApi {
.await
}
/// Offers a backup for remote devices to retrieve.
///
/// Can be cancelled by stopping the ongoing process. Success or failure can be tracked
/// via the `ImexProgress` event which should either reach `1000` for success or `0` for
/// failure.
///
/// This **stops IO** while it is running.
///
/// Returns once a remote device has retrieved the backup, or is cancelled.
async fn provide_backup(&self, account_id: u32) -> Result<()> {
let ctx = self.get_context(account_id).await?;
self.with_state(account_id, |state| {
state.backup_provider_qr.send_replace(ProviderQr::Pending);
})
.await;
let provider = imex::BackupProvider::prepare(&ctx).await?;
self.with_state(account_id, |state| {
state
.backup_provider_qr
.send_replace(ProviderQr::Ready(provider.qr()));
})
.await;
provider.await
}
/// Returns the text of the QR code for the running [`CommandApi::provide_backup`].
///
/// This QR code text can be used in [`CommandApi::get_backup`] on a second device to
/// retrieve the backup and setup this second device.
///
/// This call will fail if there is currently no concurrent call to
/// [`CommandApi::provide_backup`]. This call may block if the QR code is not yet
/// ready.
async fn get_backup_qr(&self, account_id: u32) -> Result<String> {
let qr = self.inner_get_backup_qr(account_id).await?;
qr::format_backup(&qr)
}
/// Returns the rendered QR code for the running [`CommandApi::provide_backup`].
///
/// This QR code can be used in [`CommandApi::get_backup`] on a second device to
/// retrieve the backup and setup this second device.
///
/// This call will fail if there is currently no concurrent call to
/// [`CommandApi::provide_backup`]. This call may block if the QR code is not yet
/// ready.
///
/// Returns the QR code rendered as an SVG image.
async fn get_backup_qr_svg(&self, account_id: u32) -> Result<String> {
let ctx = self.get_context(account_id).await?;
let qr = self.inner_get_backup_qr(account_id).await?;
generate_backup_qr(&ctx, &qr).await
}
/// Gets a backup from a remote provider.
///
/// This retrieves the backup from a remote device over the network and imports it into
/// the current device.
///
/// Can be cancelled by stopping the ongoing process.
async fn get_backup(&self, account_id: u32, qr_text: String) -> Result<()> {
let ctx = self.get_context(account_id).await?;
let qr = qr::check_qr(&ctx, &qr_text).await?;
imex::get_backup(&ctx, qr).await?;
Ok(())
}
// ---------------------------------------------
// connectivity
// ---------------------------------------------
@@ -1462,6 +1593,23 @@ impl CommandApi {
WebxdcMessageInfo::get_for_message(&ctx, MsgId::new(instance_msg_id)).await
}
/// Get blob encoded as base64 from a webxdc message
///
/// path is the path of the file within webxdc archive
async fn get_webxdc_blob(
&self,
account_id: u32,
instance_msg_id: u32,
path: String,
) -> Result<String> {
let ctx = self.get_context(account_id).await?;
let message = Message::load_from_db(&ctx, MsgId::new(instance_msg_id)).await?;
let blob = message.get_webxdc_blob(&ctx, &path).await?;
use base64::{engine::general_purpose, Engine as _};
Ok(general_purpose::STANDARD_NO_PAD.encode(blob))
}
/// Forward messages to another chat.
///
/// All types of messages can be forwarded,
@@ -1511,6 +1659,48 @@ impl CommandApi {
Ok(message_id.to_u32())
}
async fn send_msg(&self, account_id: u32, chat_id: u32, data: MessageData) -> Result<u32> {
let ctx = self.get_context(account_id).await?;
let mut message = Message::new(if let Some(viewtype) = data.viewtype {
viewtype.into()
} else if data.file.is_some() {
Viewtype::File
} else {
Viewtype::Text
});
if data.text.is_some() {
message.set_text(data.text);
}
if data.html.is_some() {
message.set_html(data.html);
}
if data.override_sender_name.is_some() {
message.set_override_sender_name(data.override_sender_name);
}
if let Some(file) = data.file {
message.set_file(file, None);
}
if let Some((latitude, longitude)) = data.location {
message.set_location(latitude, longitude);
}
if let Some(id) = data.quoted_message_id {
message
.set_quote(
&ctx,
Some(
&Message::load_from_db(&ctx, MsgId::new(id))
.await
.context("message to quote could not be loaded")?,
),
)
.await?;
}
let msg_id = chat::send_msg(&ctx, ChatId::new(chat_id), &mut message)
.await?
.to_u32();
Ok(msg_id)
}
// ---------------------------------------------
// functions for the composer
// the composer is the message input field
@@ -1780,3 +1970,15 @@ async fn get_config(
.await
}
}
/// Whether a QR code for a BackupProvider is currently available.
#[allow(clippy::large_enum_variant)]
#[derive(Clone, Debug)]
enum ProviderQr {
/// There is no provider, asking for a QR is an error.
NoProvider,
/// There is a provider, the QR code is pending.
Pending,
/// There is a provider and QR code.
Ready(Qr),
}

View File

@@ -502,3 +502,15 @@ impl From<ChatItem> for JSONRPCMessageListItem {
}
}
}
#[derive(Deserialize, TypeDef)]
#[serde(rename_all = "camelCase")]
pub struct MessageData {
pub text: Option<String>,
pub html: Option<String>,
pub viewtype: Option<MessageViewtype>,
pub file: Option<String>,
pub location: Option<(f64, f64)>,
pub override_sender_name: Option<String>,
pub quoted_message_id: Option<u32>,
}

View File

@@ -8,7 +8,7 @@ use typescript_type_def::TypeDef;
pub struct ProviderInfo {
pub before_login_hint: String,
pub overview_page: String,
pub status: u32, // in reality this is an enum, but for simlicity and because it gets converted into a number anyway, we use an u32 here.
pub status: u32, // in reality this is an enum, but for simplicity and because it gets converted into a number anyway, we use an u32 here.
}
impl ProviderInfo {

View File

@@ -32,6 +32,9 @@ pub enum QrObject {
Account {
domain: String,
},
Backup {
ticket: String,
},
WebrtcInstance {
domain: String,
instance_pattern: String,
@@ -126,6 +129,9 @@ impl From<Qr> for QrObject {
}
Qr::FprWithoutAddr { fingerprint } => QrObject::FprWithoutAddr { fingerprint },
Qr::Account { domain } => QrObject::Account { domain },
Qr::Backup { ticket } => QrObject::Backup {
ticket: ticket.to_string(),
},
Qr::WebrtcInstance {
domain,
instance_pattern,

View File

@@ -4,3 +4,9 @@ docs
coverage
yarn*
package-lock.json
.prettierignore
example.html
report_api_coverage.mjs
scripts
dist/example
dist/test

View File

@@ -14,7 +14,7 @@ async function run() {
throw new Error(
"USAGE: node node-add-account.js <EMAILADDRESS> <PASSWORD>"
);
console.log(`creating acccount for ${email}`);
console.log(`creating account for ${email}`);
const id = await delta.rpc.addAccount();
console.log(`created account id ${id}`);
await delta.rpc.setConfig(id, "addr", email);

View File

@@ -55,5 +55,5 @@
},
"type": "module",
"types": "dist/deltachat.d.ts",
"version": "1.110.0"
"version": "1.111.0"
}

View File

@@ -1,5 +1,5 @@
import { readFileSync } from "fs";
// only checks for the coverge of the api functions in bindings.ts for now
// only checks for the coverage of the api functions in bindings.ts for now
const generatedFile = "typescript/generated/client.ts";
const json = JSON.parse(readFileSync("./coverage/coverage-final.json"));
const jsonCoverage =

View File

@@ -53,7 +53,7 @@ describe("basic tests", () => {
]);
});
describe("account managment", () => {
describe("account management", () => {
it("should create account", async () => {
const res = await dc.rpc.addAccount();
assert((await dc.rpc.getAllAccountIds()).length === 1);
@@ -73,7 +73,7 @@ describe("basic tests", () => {
});
});
describe("contact managment", function () {
describe("contact management", function () {
let accountId: number;
before(async () => {
accountId = await dc.rpc.addAccount();
@@ -103,7 +103,7 @@ describe("basic tests", () => {
accountId = await dc.rpc.addAccount();
});
it("set and retrive", async function () {
it("set and retrieve", async function () {
await dc.rpc.setConfig(accountId, "addr", "valid@email");
assert((await dc.rpc.getConfig(accountId, "addr")) == "valid@email");
});
@@ -115,11 +115,11 @@ describe("basic tests", () => {
await expect(dc.rpc.getConfig(accountId, "invalid_key")).to.be.eventually
.rejected;
});
it("set and retrive ui.*", async function () {
it("set and retrieve ui.*", async function () {
await dc.rpc.setConfig(accountId, "ui.chat_bg", "color:red");
assert((await dc.rpc.getConfig(accountId, "ui.chat_bg")) == "color:red");
});
it("set and retrive (batch)", async function () {
it("set and retrieve (batch)", async function () {
const config = { addr: "valid@email", mail_pw: "1234" };
await dc.rpc.batchSetConfig(accountId, config);
const retrieved = await dc.rpc.batchGetConfig(
@@ -128,7 +128,7 @@ describe("basic tests", () => {
);
expect(retrieved).to.deep.equal(config);
});
it("set and retrive ui.* (batch)", async function () {
it("set and retrieve ui.* (batch)", async function () {
const config = {
"ui.chat_bg": "color:green",
"ui.enter_key_sends": "true",
@@ -140,7 +140,7 @@ describe("basic tests", () => {
);
expect(retrieved).to.deep.equal(config);
});
it("set and retrive mixed(ui and core) (batch)", async function () {
it("set and retrieve mixed(ui and core) (batch)", async function () {
const config = {
"ui.chat_bg": "color:yellow",
"ui.enter_key_sends": "false",

View File

@@ -22,7 +22,7 @@ describe("online tests", function () {
process.exit(1);
}
console.log(
"Missing DCC_NEW_TMP_EMAIL environment variable!, skip intergration tests"
"Missing DCC_NEW_TMP_EMAIL environment variable!, skip integration tests"
);
this.skip();
}
@@ -36,7 +36,7 @@ describe("online tests", function () {
account1 = await createTempUser(process.env.DCC_NEW_TMP_EMAIL);
if (!account1 || !account1.email || !account1.password) {
console.log(
"We didn't got back an account from the api, skip intergration tests"
"We didn't got back an account from the api, skip integration tests"
);
this.skip();
}
@@ -44,7 +44,7 @@ describe("online tests", function () {
account2 = await createTempUser(process.env.DCC_NEW_TMP_EMAIL);
if (!account2 || !account2.email || !account2.password) {
console.log(
"We didn't got back an account2 from the api, skip intergration tests"
"We didn't got back an account2 from the api, skip integration tests"
);
this.skip();
}
@@ -74,7 +74,7 @@ describe("online tests", function () {
accountsConfigured = true;
});
it("send and recieve text message", async function () {
it("send and receive text message", async function () {
if (!accountsConfigured) {
this.skip();
}
@@ -106,7 +106,7 @@ describe("online tests", function () {
expect(message.text).equal("Hello");
});
it("send and recieve text message roundtrip, encrypted on answer onwards", async function () {
it("send and receive text message roundtrip, encrypted on answer onwards", async function () {
if (!accountsConfigured) {
this.skip();
}

View File

@@ -1,6 +1,7 @@
[package]
name = "deltachat-repl"
version = "1.110.0"
version = "1.111.0"
license = "MPL-2.0"
edition = "2021"
[dependencies]
@@ -11,7 +12,7 @@ dirs = "4"
log = "0.4.16"
pretty_env_logger = "0.4"
rusqlite = "0.28"
rustyline = "10"
rustyline = "11"
tokio = { version = "1", features = ["fs", "rt-multi-thread", "macros"] }
[features]

View File

@@ -29,13 +29,13 @@ use tokio::fs;
/// Reset database tables.
/// Argument is a bitmask, executing single or multiple actions in one call.
/// e.g. bitmask 7 triggers actions definded with bits 1, 2 and 4.
/// e.g. bitmask 7 triggers actions defined with bits 1, 2 and 4.
async fn reset_tables(context: &Context, bits: i32) {
println!("Resetting tables ({bits})...");
if 0 != bits & 1 {
context
.sql()
.execute("DELETE FROM jobs;", paramsv![])
.execute("DELETE FROM jobs;", ())
.await
.unwrap();
println!("(1) Jobs reset.");
@@ -43,7 +43,7 @@ async fn reset_tables(context: &Context, bits: i32) {
if 0 != bits & 2 {
context
.sql()
.execute("DELETE FROM acpeerstates;", paramsv![])
.execute("DELETE FROM acpeerstates;", ())
.await
.unwrap();
println!("(2) Peerstates reset.");
@@ -51,7 +51,7 @@ async fn reset_tables(context: &Context, bits: i32) {
if 0 != bits & 4 {
context
.sql()
.execute("DELETE FROM keypairs;", paramsv![])
.execute("DELETE FROM keypairs;", ())
.await
.unwrap();
println!("(4) Private keypairs reset.");
@@ -59,36 +59,36 @@ async fn reset_tables(context: &Context, bits: i32) {
if 0 != bits & 8 {
context
.sql()
.execute("DELETE FROM contacts WHERE id>9;", paramsv![])
.execute("DELETE FROM contacts WHERE id>9;", ())
.await
.unwrap();
context
.sql()
.execute("DELETE FROM chats WHERE id>9;", paramsv![])
.execute("DELETE FROM chats WHERE id>9;", ())
.await
.unwrap();
context
.sql()
.execute("DELETE FROM chats_contacts;", paramsv![])
.execute("DELETE FROM chats_contacts;", ())
.await
.unwrap();
context
.sql()
.execute("DELETE FROM msgs WHERE id>9;", paramsv![])
.execute("DELETE FROM msgs WHERE id>9;", ())
.await
.unwrap();
context
.sql()
.execute(
"DELETE FROM config WHERE keyname LIKE 'imap.%' OR keyname LIKE 'configured%';",
paramsv![],
(),
)
.await
.unwrap();
context.sql().config_cache().write().await.clear();
context
.sql()
.execute("DELETE FROM leftgrps;", paramsv![])
.execute("DELETE FROM leftgrps;", ())
.await
.unwrap();
println!("(8) Rest but server config reset.");
@@ -336,6 +336,8 @@ pub async fn cmdline(context: Context, line: &str, chat_id: &mut ChatId) -> Resu
has-backup\n\
export-backup\n\
import-backup <backup-file>\n\
send-backup\n\
receive-backup <qr>\n\
export-keys\n\
import-keys\n\
export-setup\n\
@@ -486,6 +488,17 @@ pub async fn cmdline(context: Context, line: &str, chat_id: &mut ChatId) -> Resu
)
.await?;
}
"send-backup" => {
let provider = BackupProvider::prepare(&context).await?;
let qr = provider.qr();
println!("QR code: {}", format_backup(&qr)?);
provider.await?;
}
"receive-backup" => {
ensure!(!arg1.is_empty(), "Argument <qr> is missing.");
let qr = check_qr(&context, arg1).await?;
deltachat::imex::get_backup(&context, qr).await?;
}
"export-keys" => {
let dir = dirs::home_dir().unwrap_or_default();
imex(&context, ImexMode::ExportSelfKeys, dir.as_ref(), None).await?;
@@ -869,7 +882,7 @@ pub async fn cmdline(context: Context, line: &str, chat_id: &mut ChatId) -> Resu
if continue_streaming {
println!("Success, streaming should be continued.");
} else {
println!("Success, streaming can be stoppped.");
println!("Success, streaming can be stopped.");
}
}
"dellocations" => {

View File

@@ -152,13 +152,15 @@ impl Completer for DcHelper {
}
}
const IMEX_COMMANDS: [&str; 12] = [
const IMEX_COMMANDS: [&str; 14] = [
"initiate-key-transfer",
"get-setupcodebegin",
"continue-key-transfer",
"has-backup",
"export-backup",
"import-backup",
"send-backup",
"receive-backup",
"export-keys",
"import-keys",
"export-setup",
@@ -350,8 +352,8 @@ async fn start(args: Vec<String>) -> Result<(), Error> {
match readline {
Ok(line) => {
// TODO: ignore "set mail_pw"
rl.add_history_entry(line.as_str());
let contine = Handle::current().block_on(async {
rl.add_history_entry(line.as_str())?;
let should_continue = Handle::current().block_on(async {
match handle_cmd(line.trim(), ctx.clone(), &mut selected_chat).await {
Ok(ExitResult::Continue) => true,
Ok(ExitResult::Exit) => {
@@ -359,13 +361,13 @@ async fn start(args: Vec<String>) -> Result<(), Error> {
false
}
Err(err) => {
println!("Error: {err}");
println!("Error: {err:#}");
true
}
}
});
if !contine {
if !should_continue {
break;
}
}
@@ -374,7 +376,7 @@ async fn start(args: Vec<String>) -> Result<(), Error> {
break;
}
Err(err) => {
println!("Error: {err}");
println!("Error: {err:#}");
break;
}
}

View File

@@ -27,7 +27,7 @@ def _to_attrdict(obj):
class AttrDict(dict):
"""Dictionary that allows accessing values usin the "dot notation" as attributes."""
"""Dictionary that allows accessing values using the "dot notation" as attributes."""
def __init__(self, *args, **kwargs) -> None:
super().__init__({_camel_to_snake(key): _to_attrdict(value) for key, value in dict(*args, **kwargs).items()})

View File

@@ -158,7 +158,7 @@ class Account:
:param contact: if a contact is specified only chats including this contact are returned.
:param archived_only: if True only archived chats are returned.
:param for_forwarding: if True the chat list is sorted with "Saved messages" at the top
and withot "Device chat" and contact requests.
and without "Device chat" and contact requests.
:param no_specials: if True archive link is not added to the list.
:param alldone_hint: if True the "all done hint" special chat will be added to the list
as needed.

View File

@@ -3,7 +3,7 @@ from dataclasses import dataclass
from typing import TYPE_CHECKING, Dict, List, Optional, Tuple, Union
from ._utils import AttrDict
from .const import ChatVisibility
from .const import ChatVisibility, ViewType
from .contact import Contact
from .message import Message
@@ -108,15 +108,27 @@ class Chat:
async def send_message(
self,
text: Optional[str] = None,
html: Optional[str] = None,
viewtype: Optional[ViewType] = None,
file: Optional[str] = None,
location: Optional[Tuple[float, float]] = None,
override_sender_name: Optional[str] = None,
quoted_msg: Optional[Union[int, Message]] = None,
) -> Message:
"""Send a message and return the resulting Message instance."""
if isinstance(quoted_msg, Message):
quoted_msg = quoted_msg.id
msg_id, _ = await self._rpc.misc_send_msg(self.account.id, self.id, text, file, location, quoted_msg)
draft = {
"text": text,
"html": html,
"viewtype": viewtype,
"file": file,
"location": location,
"overrideSenderName": override_sender_name,
"quotedMsg": quoted_msg,
}
msg_id = await self._rpc.send_msg(self.account.id, self.id, draft)
return Message(self.account, msg_id)
async def send_text(self, text: str) -> Message:

View File

@@ -6,7 +6,7 @@ envlist =
[testenv]
commands =
pytest {posargs}
pytest --exitfirst {posargs}
setenv =
# Avoid stack overflow when Rust core is built without optimizations.
RUST_MIN_STACK=8388608
@@ -25,5 +25,5 @@ deps =
ruff
black
commands =
black --check --diff src/ examples/ tests/
black --quiet --check --diff src/ examples/ tests/
ruff src/ examples/ tests/

View File

@@ -1,6 +1,6 @@
[package]
name = "deltachat-rpc-server"
version = "1.110.0"
version = "1.111.0"
description = "DeltaChat JSON-RPC server"
edition = "2021"
readme = "README.md"

View File

@@ -5,10 +5,13 @@ over standard I/O.
## Install
To install run:
To download binary pre-builds check the [releases page](https://github.com/deltachat/deltachat-core-rust/releases).
Rename the downloaded binary to `deltachat-rpc-server` and add it to your `PATH`.
To install from source run:
```sh
cargo install --path ../deltachat-rpc-server
cargo install --git https://github.com/deltachat/deltachat-core-rust/ deltachat-rpc-server
```
The `deltachat-rpc-server` executable will be installed into `$HOME/.cargo/bin` that should be available

View File

@@ -5,7 +5,7 @@ use quote::quote;
use crate::proc_macro::TokenStream;
// For now, assume (not check) that these macroses are applied to enum without
// For now, assume (not check) that these macros are applied to enum without
// data. If this assumption is violated, compiler error will point to
// generated code, which is not very user-friendly.

84
deny.toml Normal file
View File

@@ -0,0 +1,84 @@
[advisories]
unmaintained = "allow"
ignore = [
"RUSTSEC-2020-0071",
# Only affects windows if using non-default allocator (and unmaintained).
"RUSTSEC-2021-0145",
]
[bans]
# Accept some duplicate versions, ideally we work towards this list
# becoming empty. Adding versions forces us to revisit this at least
# when upgrading.
skip = [
{ name = "base64", version = "<0.21" },
{ name = "block-buffer", version = "<0.10" },
{ name = "darling", version = "<0.14" },
{ name = "darling_core", version = "<0.14" },
{ name = "darling_macro", version = "<0.14" },
{ name = "digest", version = "<0.10" },
{ name = "env_logger", version = "<0.10" },
{ name = "getrandom", version = "<0.2" },
{ name = "hermit-abi", version = "<0.3" },
{ name = "humantime", version = "<2.1" },
{ name = "idna", version = "<0.3" },
{ name = "nom", version = "<7.1" },
{ name = "quick-error", version = "<2.0" },
{ name = "rand", version = "<0.8" },
{ name = "rand_chacha", version = "<0.3" },
{ name = "rand_core", version = "<0.6" },
{ name = "sha2", version = "<0.10" },
{ name = "time", version = "<0.3" },
{ name = "version_check", version = "<0.9" },
{ name = "wasi", version = "<0.11" },
{ name = "windows-sys", version = "<0.45" },
{ name = "windows_x86_64_msvc", version = "<0.42" },
{ name = "windows_x86_64_gnu", version = "<0.42" },
{ name = "windows_i686_msvc", version = "<0.42" },
{ name = "windows_i686_gnu", version = "<0.42" },
{ name = "windows_aarch64_msvc", version = "<0.42" },
{ name = "unicode-xid", version = "<0.2.4" },
{ name = "syn", version = "<1.0" },
{ name = "quote", version = "<1.0" },
{ name = "proc-macro2", version = "<1.0" },
{ name = "portable-atomic", version = "<1.0" },
{ name = "spin", version = "<0.9.6" },
{ name = "convert_case", version = "0.4.0" },
{ name = "clap_lex", version = "0.2.4" },
{ name = "clap", version = "3.2.23" },
]
[licenses]
allow = [
"0BSD",
"Apache-2.0",
"BSD-2-Clause",
"BSD-3-Clause",
"BSL-1.0", # Boost Software License 1.0
"CC0-1.0",
"ISC",
"MIT",
"MPL-2.0",
"OpenSSL",
"Unicode-DFS-2016",
"Zlib",
]
[[licenses.clarify]]
name = "ring"
expression = "MIT AND ISC AND OpenSSL"
license-files = [
{ path = "LICENSE", hash = 0xbd0eed23 },
]
[sources.allow-org]
# Organisations which we allow git sources from.
github = [
"async-email",
"deltachat",
"n0-computer",
"quinn-rs",
"dignifiedquire",
]

View File

@@ -66,13 +66,13 @@ Note that usually a mail is signed by a key that has a UID matching the from add
#### Upsides:
- With this approach, it's easy to switch to a model where the info about the transition is encoded in the PGP key. Since the key is gossiped, the information about the transition will spread virally.
- Faster transation: If you send a message to e.g. "Delta Chat Dev", all members of the "sub-group" "delta android" will know of your transition.
- Faster transition: If you send a message to e.g. "Delta Chat Dev", all members of the "sub-group" "delta android" will know of your transition.
### Alternatives and old discussions/plans:
- Change the contact instead of rewriting the group member lists. This seems to call for more trouble since we will end up with multiple contacts having the same email address.
- If needed, we could add a header a) indicating that the sender did an address transition or b) listing all the secondary (old) addresses. For now, there is no big enough benefit to warrant introducing another header and its processing on the receiver side (including all the neccessary checks and handling of error cases). Instead, we only check for the `Chat-Version` header to prevent accidental transitions when an MUA user sends a message from another email address with the same key.
- If needed, we could add a header a) indicating that the sender did an address transition or b) listing all the secondary (old) addresses. For now, there is no big enough benefit to warrant introducing another header and its processing on the receiver side (including all the necessary checks and handling of error cases). Instead, we only check for the `Chat-Version` header to prevent accidental transitions when an MUA user sends a message from another email address with the same key.
- The condition for a transition temporarily was:
@@ -107,7 +107,7 @@ The most obvious alternative would be to create a new contact with the new addre
#### Upsides:
- With this approach, it's easier to switch to a model where the info about the transition is encoded in the PGP key. Since the key is gossiped, the information about the transition will spread virally.
- (Also, less important: Slightly faster transation: If you send a message to e.g. "Delta Chat Dev", all members of the "sub-group" "delta android" will know of your transition.)
- (Also, less important: Slightly faster transition: If you send a message to e.g. "Delta Chat Dev", all members of the "sub-group" "delta android" will know of your transition.)
- It's easier to implement (if too many problems turn up, we can still switch to another approach and didn't wast that much development time.)
[full messages](https://github.com/deltachat/deltachat-core-rust/pull/2896#discussion_r852002161)
@@ -124,4 +124,4 @@ Other
Notes during implementing
========================
- As far as I understand the code, unencrypted messages are unsigned. So, the transition only works if both sides have the other side's key.
- As far as I understand the code, unencrypted messages are unsigned. So, the transition only works if both sides have the other side's key.

View File

@@ -62,5 +62,5 @@ Notes/Questions:
and design such that we can cover all imap flags.
- It might not be neccessary to keep needs_send_mdn state in this table
- It might not be necessary to keep needs_send_mdn state in this table
if this can be decided rather when we succeed with mark_seen/mark_delete.

80
fuzz/Cargo.lock generated
View File

@@ -111,7 +111,7 @@ version = "0.6.0"
source = "git+https://github.com/async-email/async-imap?branch=master#85ff7a3d9d71a3715354fabf2fc1a8d047b5710e"
dependencies = [
"async-channel",
"async-native-tls",
"async-native-tls 0.4.0",
"base64 0.13.1",
"byte-pool",
"chrono",
@@ -139,6 +139,18 @@ dependencies = [
"url",
]
[[package]]
name = "async-native-tls"
version = "0.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9343dc5acf07e79ff82d0c37899f079db3534d99f189a1837c8e549c99405bec"
dependencies = [
"native-tls",
"thiserror",
"tokio",
"url",
]
[[package]]
name = "async-smtp"
version = "0.8.0"
@@ -696,12 +708,12 @@ checksum = "23d8666cb01533c39dde32bcbab8e227b4ed6679b2c925eba05feabea39508fb"
[[package]]
name = "deltachat"
version = "1.107.0"
version = "1.111.0"
dependencies = [
"anyhow",
"async-channel",
"async-imap",
"async-native-tls",
"async-native-tls 0.5.0",
"async-smtp",
"async_zip",
"backtrace",
@@ -723,17 +735,15 @@ dependencies = [
"lettre_email",
"libc",
"mailparse 0.14.0",
"native-tls",
"num-derive",
"num-traits",
"num_cpus",
"once_cell",
"parking_lot",
"percent-encoding",
"pgp",
"qrcodegen",
"quick-xml",
"r2d2",
"r2d2_sqlite",
"rand 0.8.5",
"ratelimit",
"regex",
@@ -1284,15 +1294,6 @@ dependencies = [
"tracing",
]
[[package]]
name = "hashbrown"
version = "0.11.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ab5ef0d4909ef3724cc8cce6ccc8572c5c817592e9285f5464f8e86f8bd3726e"
dependencies = [
"ahash",
]
[[package]]
name = "hashbrown"
version = "0.12.3"
@@ -1304,11 +1305,11 @@ dependencies = [
[[package]]
name = "hashlink"
version = "0.7.0"
version = "0.8.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7249a3129cbc1ffccd74857f81464a323a152173cdb134e0fd81bc803b29facf"
checksum = "69fe1fcf8b4278d860ad0548329f892a3631fb63f82574df68275f34cdbe0ffa"
dependencies = [
"hashbrown 0.11.2",
"hashbrown",
]
[[package]]
@@ -1506,7 +1507,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1885e79c1fc4b10f0e172c475f458b7f7b93061064d98c3293e98c5ba0c8b399"
dependencies = [
"autocfg",
"hashbrown 0.12.3",
"hashbrown",
]
[[package]]
@@ -1631,9 +1632,9 @@ checksum = "348108ab3fba42ec82ff6e9564fc4ca0247bdccdc68dd8af9764bbc79c3c8ffb"
[[package]]
name = "libsqlite3-sys"
version = "0.24.2"
version = "0.25.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "898745e570c7d0453cc1fbc4a701eb6c662ed54e8fec8b7d14be137ebeeb9d14"
checksum = "29f835d03d717946d28b1d1ed632eb6f0e24a299388ee623d0c23118d3e8a7fa"
dependencies = [
"cc",
"openssl-sys",
@@ -2241,27 +2242,6 @@ version = "0.4.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "20f14e071918cbeefc5edc986a7aa92c425dae244e003a35e1cdddb5ca39b5cb"
[[package]]
name = "r2d2"
version = "0.8.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "51de85fb3fb6524929c8a2eb85e6b6d363de4e8c48f9e2c2eac4944abc181c93"
dependencies = [
"log",
"parking_lot",
"scheduled-thread-pool",
]
[[package]]
name = "r2d2_sqlite"
version = "0.20.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6fdc8e4da70586127893be32b7adf21326a4c6b1aba907611edf467d13ffe895"
dependencies = [
"r2d2",
"rusqlite",
]
[[package]]
name = "rand"
version = "0.7.3"
@@ -2451,16 +2431,15 @@ dependencies = [
[[package]]
name = "rusqlite"
version = "0.27.0"
version = "0.28.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "85127183a999f7db96d1a976a309eebbfb6ea3b0b400ddd8340190129de6eb7a"
checksum = "01e213bc3ecb39ac32e81e51ebe31fd888a940515173e3a18a35f8c6e896422a"
dependencies = [
"bitflags",
"fallible-iterator",
"fallible-streaming-iterator",
"hashlink",
"libsqlite3-sys",
"memchr",
"smallvec",
]
@@ -2514,15 +2493,6 @@ dependencies = [
"windows-sys 0.36.1",
]
[[package]]
name = "scheduled-thread-pool"
version = "0.2.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "977a7519bff143a44f842fd07e80ad1329295bd71686457f18e496736f4bf9bf"
dependencies = [
"parking_lot",
]
[[package]]
name = "scopeguard"
version = "1.1.0"
@@ -3126,7 +3096,7 @@ version = "0.1.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c5faade31a542b8b35855fff6e8def199853b2da8da256da52f52f1316ee3137"
dependencies = [
"hashbrown 0.12.3",
"hashbrown",
"regex",
]

View File

@@ -124,7 +124,7 @@ $ npm run test
```
(when using [fnm](https://github.com/Schniz/fnm) instead of nvm, you can select the architecture)
If your node and electron are already build for arm64 you can also try bulding for arm:
If your node and electron are already build for arm64 you can also try building for arm:
```
$ fnm install 16 --arch arm64
@@ -182,7 +182,7 @@ this example can also be found in the examples folder [examples/send_message.js]
### Generating Docs
We are curently migrating to automaticaly generated documentation.
We are currently migrating to automatically generated documentation.
You can find the old documentation at [old_docs](./old_docs).
to generate the documentation, run:

View File

@@ -2,7 +2,7 @@ import { join } from 'path'
/**
* bindings are not typed yet.
* if the availible function names are required they can be found inside of `../src/module.c`
* if the available function names are required they can be found inside of `../src/module.c`
*/
export const bindings: any = require('node-gyp-build')(join(__dirname, '../'))

View File

@@ -543,7 +543,7 @@ export class Context extends EventEmitter {
/**
*
* @deprectated please use `AccountManager.getSystemInfo()` instead
* @deprecated please use `AccountManager.getSystemInfo()` instead
*/
static getSystemInfo() {
return AccountManager.getSystemInfo()
@@ -818,7 +818,7 @@ export class Context extends EventEmitter {
* Must be given in number of seconds since 00:00 hours, Jan 1, 1970 UTC.
* 0 for "all up to now".
* @return Array of locations, NULL is never returned.
* The array is sorted decending;
* The array is sorted descending;
* the first entry in the array is the location with the newest timestamp.
*
* Examples:

View File

@@ -217,7 +217,7 @@ export class AccountManager extends EventEmitter {
/** get information about the provider
*
* This function creates a temporary context to be standalone,
* if posible use `Context.getProviderFromEmail` instead. (otherwise potential proxy settings are not used)
* if possible use `Context.getProviderFromEmail` instead. (otherwise potential proxy settings are not used)
* @deprecated
*/
static getProviderFromEmail(email: string) {

View File

@@ -769,7 +769,7 @@ describe('Integration tests', function () {
this.beforeAll(async function () {
if (!process.env.DCC_NEW_TMP_EMAIL) {
console.log(
'Missing DCC_NEW_TMP_EMAIL environment variable!, skip intergration tests'
'Missing DCC_NEW_TMP_EMAIL environment variable!, skip integration tests'
)
this.skip()
}
@@ -777,7 +777,7 @@ describe('Integration tests', function () {
account = await createTempUser(process.env.DCC_NEW_TMP_EMAIL)
if (!account || !account.email || !account.password) {
console.log(
"We didn't got back an account from the api, skip intergration tests"
"We didn't got back an account from the api, skip integration tests"
)
this.skip()
}

View File

@@ -60,5 +60,5 @@
"test:mocha": "mocha -r esm node/test/test.js --growl --reporter=spec --bail --exit"
},
"types": "node/dist/index.d.ts",
"version": "1.110.0"
"version": "1.111.0"
}

View File

@@ -64,7 +64,7 @@
- use new experimental full-Rust Delta Chat core
- support Autocrypt Setup Messages
- remove synchronous events
- use CircleCI for continous integration and packaging of Linux wheels
- use CircleCI for continuous integration and packaging of Linux wheels
- use docker image for building wheels
- fix code documentation links

View File

@@ -51,7 +51,7 @@ git_describe_command = "git describe --dirty --tags --long --match v*.*"
line-length = 120
[tool.ruff]
select = ["E", "F", "W", "YTT", "C4", "ISC", "ICN", "TID", "DTZ", "PLC", "PLE", "PLW", "PIE", "COM", "UP004", "UP010", "UP031", "UP032"]
select = ["E", "F", "W", "YTT", "C4", "ISC", "ICN", "TID", "DTZ", "PLC", "PLE", "PLW", "PIE", "COM", "UP004", "UP010", "UP031", "UP032", "ANN204"]
line-length = 120
[tool.isort]

View File

@@ -121,7 +121,7 @@ class Account:
"""re-enable logging."""
self._logging = True
def __repr__(self):
def __repr__(self) -> str:
return f"<Account path={self.db_path}>"
# def __del__(self):
@@ -159,7 +159,7 @@ class Account:
"""set stock translation string.
:param id: id of stock string (const.DC_STR_*)
:param value: string to set as new transalation
:param value: string to set as new translation
:returns: None
"""
bytestring = string.encode("utf8")
@@ -284,9 +284,9 @@ class Account:
:returns: :class:`deltachat.contact.Contact` instance.
"""
(name, addr) = self.get_contact_addr_and_name(obj, name)
name = as_dc_charpointer(name)
addr = as_dc_charpointer(addr)
contact_id = lib.dc_create_contact(self._dc_context, name, addr)
name_c = as_dc_charpointer(name)
addr_c = as_dc_charpointer(addr)
contact_id = lib.dc_create_contact(self._dc_context, name_c, addr_c)
return Contact(self, contact_id)
def get_contact(self, obj) -> Optional[Contact]:
@@ -363,12 +363,12 @@ class Account:
:returns: list of :class:`deltachat.contact.Contact` objects.
"""
flags = 0
query = as_dc_charpointer(query)
query_c = as_dc_charpointer(query)
if only_verified:
flags |= const.DC_GCL_VERIFIED_ONLY
if with_self:
flags |= const.DC_GCL_ADD_SELF
dc_array = ffi.gc(lib.dc_get_contacts(self._dc_context, flags, query), lib.dc_array_unref)
dc_array = ffi.gc(lib.dc_get_contacts(self._dc_context, flags, query_c), lib.dc_array_unref)
return list(iter_array(dc_array, lambda x: Contact(self, x)))
def get_fresh_messages(self) -> Generator[Message, None, None]:
@@ -628,7 +628,7 @@ class Account:
configtracker = self.configure()
configtracker.wait_finish()
# start IO threads and configure if neccessary
# start IO threads and configure if necessary
self.start_io()
def add_account_plugin(self, plugin, name=None):
@@ -767,7 +767,7 @@ class Account:
class ScannedQRCode:
def __init__(self, dc_lot):
def __init__(self, dc_lot) -> None:
self._dc_lot = dc_lot
def is_ask_verifycontact(self):

View File

@@ -24,7 +24,7 @@ class Chat:
You obtain instances of it through :class:`deltachat.account.Account`.
"""
def __init__(self, account, id) -> None:
def __init__(self, account, id: int) -> None:
from .account import Account
assert isinstance(account, Account), repr(account)
@@ -162,8 +162,8 @@ class Chat:
:param name: as a unicode string.
:returns: True on success, False otherwise
"""
name = as_dc_charpointer(name)
return bool(lib.dc_set_chat_name(self.account._dc_context, self.id, name))
name_c = as_dc_charpointer(name)
return bool(lib.dc_set_chat_name(self.account._dc_context, self.id, name_c))
def get_color(self):
"""return the color of the chat.
@@ -532,13 +532,13 @@ class Chat:
# ------ location streaming API ------------------------------
def is_sending_locations(self):
def is_sending_locations(self) -> bool:
"""return True if this chat has location-sending enabled currently.
:returns: True if location sending is enabled.
"""
return lib.dc_is_sending_locations_to_chat(self.account._dc_context, self.id)
return bool(lib.dc_is_sending_locations_to_chat(self.account._dc_context, self.id))
def enable_sending_locations(self, seconds):
def enable_sending_locations(self, seconds) -> None:
"""enable sending locations for this chat.
all subsequent messages will carry a location with them.
@@ -572,7 +572,7 @@ class Chat:
class Location:
def __init__(self, latitude, longitude, accuracy, timestamp, marker):
def __init__(self, latitude, longitude, accuracy, timestamp, marker) -> None:
assert isinstance(timestamp, datetime)
self.latitude = latitude
self.longitude = longitude
@@ -580,5 +580,5 @@ class Location:
self.timestamp = timestamp
self.marker = marker
def __eq__(self, other):
def __eq__(self, other) -> bool:
return self.__dict__ == other.__dict__

View File

@@ -15,7 +15,7 @@ class Contact:
You obtain instances of it through :class:`deltachat.account.Account`.
"""
def __init__(self, account, id):
def __init__(self, account, id) -> None:
from .account import Account
assert isinstance(account, Account), repr(account)
@@ -27,10 +27,10 @@ class Contact:
return False
return self.account._dc_context == other.account._dc_context and self.id == other.id
def __ne__(self, other):
def __ne__(self, other) -> bool:
return not self == other
def __repr__(self):
def __repr__(self) -> str:
return f"<Contact id={self.id} addr={self.addr} dc_context={self.account._dc_context}>"
@property

View File

@@ -191,7 +191,7 @@ class DirectImap:
class IdleManager:
def __init__(self, direct_imap):
def __init__(self, direct_imap) -> None:
self.direct_imap = direct_imap
self.log = direct_imap.account.log
# fetch latest messages before starting idle so that it only

View File

@@ -25,12 +25,12 @@ def get_dc_event_name(integer, _DC_EVENTNAME_MAP={}):
class FFIEvent:
def __init__(self, name: str, data1, data2):
def __init__(self, name: str, data1, data2) -> None:
self.name = name
self.data1 = data1
self.data2 = data2
def __str__(self):
def __str__(self) -> str:
if self.name == "DC_EVENT_INFO":
return f"INFO {self.data2}"
if self.name == "DC_EVENT_WARNING":
@@ -84,7 +84,10 @@ class FFIEventLogger:
class FFIEventTracker:
def __init__(self, account, timeout=None):
account: Account
_event_queue: Queue
def __init__(self, account: Account, timeout=None) -> None:
self.account = account
self._timeout = timeout
self._event_queue = Queue()

View File

@@ -19,7 +19,7 @@ class Message:
:class:`deltachat.chat.Chat`.
"""
def __init__(self, account, dc_msg):
def __init__(self, account, dc_msg) -> None:
self.account = account
assert isinstance(self.account._dc_context, ffi.CData)
assert isinstance(dc_msg, ffi.CData)
@@ -33,7 +33,7 @@ class Message:
return False
return self.account == other.account and self.id == other.id
def __repr__(self):
def __repr__(self) -> str:
c = self.get_sender_contact()
typ = "outgoing" if self.is_outgoing() else "incoming"
return (

View File

@@ -10,14 +10,14 @@ class Reactions:
You obtain instances of it through :class:`deltachat.message.Message`.
"""
def __init__(self, account, dc_reactions):
def __init__(self, account, dc_reactions) -> None:
assert isinstance(account._dc_context, ffi.CData)
assert isinstance(dc_reactions, ffi.CData)
assert dc_reactions != ffi.NULL
self.account = account
self._dc_reactions = dc_reactions
def __repr__(self):
def __repr__(self) -> str:
return f"<Reactions dc_reactions={self._dc_reactions}>"
@classmethod

View File

@@ -9,7 +9,7 @@ import threading
import time
import weakref
from queue import Queue
from typing import Callable, List, Optional
from typing import Callable, List, Optional, Dict, Set
import pytest
import requests
@@ -60,13 +60,13 @@ def pytest_configure(config):
# Make sure we don't get garbled output because threads keep running
# collect all ever created accounts in a weakref-set (so we don't
# keep objects unneccessarily alive) and enable/disable logging
# keep objects unnecessarily alive) and enable/disable logging
# for each pytest test phase # (setup/call/teardown).
# Additionally make the acfactory use a logging/no-logging default.
class LoggingAspect:
def __init__(self):
self._accounts = weakref.WeakSet()
def __init__(self) -> None:
self._accounts: weakref.WeakSet = weakref.WeakSet()
@deltachat.global_hookimpl
def dc_account_init(self, account):
@@ -129,7 +129,7 @@ def pytest_report_header(config, startdir):
if cfg:
if "?" in cfg:
url, token = cfg.split("?", 1)
summary.append(f"Liveconfig provider: {url}?<token ommitted>")
summary.append(f"Liveconfig provider: {url}?<token omitted>")
else:
summary.append(f"Liveconfig file: {cfg}")
return summary
@@ -143,10 +143,12 @@ def testprocess(request):
class TestProcess:
"""A pytest session-scoped instance to help with managing "live" account configurations."""
def __init__(self, pytestconfig):
_addr2files: Dict[str, Dict[pathlib.Path, bytes]]
def __init__(self, pytestconfig) -> None:
self.pytestconfig = pytestconfig
self._addr2files = {}
self._configlist = []
self._configlist: List[Dict[str, str]] = []
def get_liveconfig_producer(self):
"""provide live account configs, cached on a per-test-process scope
@@ -277,10 +279,10 @@ class ACSetup:
_configured_events: Queue
def __init__(self, testprocess, init_time):
def __init__(self, testprocess, init_time) -> None:
self._configured_events = Queue()
self._account2state = {}
self._imap_cleaned = set()
self._account2state: Dict[Account, str] = {}
self._imap_cleaned: Set[str] = set()
self.testprocess = testprocess
self.init_time = init_time

View File

@@ -1,19 +1,25 @@
from queue import Queue
from threading import Event
from typing import List, TYPE_CHECKING
from .hookspec import Global, account_hookimpl
if TYPE_CHECKING:
from .events import FFIEvent
class ImexFailed(RuntimeError):
"""Exception for signalling that import/export operations failed."""
class ImexTracker:
def __init__(self):
_imex_events: Queue
def __init__(self) -> None:
self._imex_events = Queue()
@account_hookimpl
def ac_process_ffi_event(self, ffi_event):
def ac_process_ffi_event(self, ffi_event: "FFIEvent") -> None:
if ffi_event.name == "DC_EVENT_IMEX_PROGRESS":
self._imex_events.put(ffi_event.data1)
elif ffi_event.name == "DC_EVENT_IMEX_FILE_WRITTEN":
@@ -50,7 +56,13 @@ class ConfigureFailed(RuntimeError):
class ConfigureTracker:
ConfigureFailed = ConfigureFailed
def __init__(self, account):
_configure_events: Queue
_smtp_finished: Event
_imap_finished: Event
_ffi_events: List["FFIEvent"]
_progress: Queue
def __init__(self, account) -> None:
self.account = account
self._configure_events = Queue()
self._smtp_finished = Event()
@@ -60,7 +72,7 @@ class ConfigureTracker:
self._gm = Global._get_plugin_manager()
@account_hookimpl
def ac_process_ffi_event(self, ffi_event):
def ac_process_ffi_event(self, ffi_event: "FFIEvent") -> None:
self._ffi_events.append(ffi_event)
if ffi_event.name == "DC_EVENT_SMTP_CONNECTED":
self._smtp_finished.set()

View File

@@ -77,7 +77,7 @@ class ReportType:
class AutoReplier:
def __init__(self, account, log, num_send, num_bigfiles, report_func):
def __init__(self, account, log, num_send, num_bigfiles, report_func) -> None:
self.account = account
self._log = log
self.report_func = report_func
@@ -90,7 +90,7 @@ class AutoReplier:
self._thread.setDaemon(True)
self._thread.start()
def log(self, message):
def log(self, message) -> None:
self._log(f"{self.addr} {message}")
def thread_stats(self):

View File

@@ -72,6 +72,19 @@ def test_configure_canceled(acfactory):
pass
def test_configure_unref(tmpdir):
"""Test that removing the last reference to the context during ongoing configuration
does not result in use-after-free."""
from deltachat.capi import ffi, lib
path = tmpdir.mkdir("test_configure_unref").join("dc.db").strpath
dc_context = lib.dc_context_new(ffi.NULL, path.encode("utf8"), ffi.NULL)
lib.dc_set_config(dc_context, "addr".encode("utf8"), "foo@x.testrun.org".encode("utf8"))
lib.dc_set_config(dc_context, "mail_pw".encode("utf8"), "abc".encode("utf8"))
lib.dc_configure(dc_context)
lib.dc_context_unref(dc_context)
def test_export_import_self_keys(acfactory, tmpdir, lp):
ac1, ac2 = acfactory.get_online_accounts(2)
@@ -204,13 +217,13 @@ def test_html_message(acfactory, lp):
lp.sec("ac1: prepare and send text message to ac2")
msg1 = chat.send_text("message0")
assert not msg1.has_html()
assert msg1.html == ""
assert not msg1.html
lp.sec("wait for ac2 to receive message")
msg2 = ac2._evtracker.wait_next_incoming_message()
assert msg2.text == "message0"
assert not msg2.has_html()
assert msg2.html == ""
assert not msg2.html
lp.sec("ac1: prepare and send HTML+text message to ac2")
msg1 = Message.new_empty(ac1, "text")
@@ -1484,7 +1497,7 @@ def test_import_export_online_all(acfactory, tmpdir, data, lp):
def test_ac_setup_message(acfactory, lp):
# note that the receiving account needs to be configured and running
# before ther setup message is send. DC does not read old messages
# before the setup message is send. DC does not read old messages
# as of Jul2019
ac1 = acfactory.new_online_configuring_account()
ac2 = acfactory.new_online_configuring_account(cloned_from=ac1)
@@ -1604,7 +1617,7 @@ def test_add_remove_member_remote_events(acfactory, lp):
in_list = queue.Queue()
class EventHolder:
def __init__(self, **kwargs):
def __init__(self, **kwargs) -> None:
self.__dict__.update(kwargs)
class InPlugin:
@@ -1987,13 +2000,16 @@ def test_delete_multiple_messages(acfactory, lp):
lp.sec("ac2: deleting all messages except third")
assert len(to_delete) == len(texts) - 1
ac2.delete_messages(to_delete)
ac2._evtracker.get_matching("DC_EVENT_IMAP_MESSAGE_DELETED")
ac2._evtracker.get_info_contains("close/expunge succeeded")
lp.sec("ac2: test that only one message is left")
ac2.direct_imap.select_config_folder("inbox")
assert len(ac2.direct_imap.get_all_messages()) == 1
while 1:
ac2._evtracker.get_matching("DC_EVENT_IMAP_MESSAGE_DELETED")
ac2._evtracker.get_info_contains("close/expunge succeeded")
ac2.direct_imap.select_config_folder("inbox")
nr_msgs = len(ac2.direct_imap.get_all_messages())
assert nr_msgs > 0
if nr_msgs == 1:
break
def test_trash_multiple_messages(acfactory, lp):
@@ -2017,11 +2033,15 @@ def test_trash_multiple_messages(acfactory, lp):
lp.sec("ac2: deleting all messages except second")
assert len(to_delete) == len(texts) - 1
ac2.delete_messages(to_delete)
ac2._evtracker.get_matching("DC_EVENT_IMAP_MESSAGE_MOVED")
lp.sec("ac2: test that only one message is left")
ac2.direct_imap.select_config_folder("inbox")
assert len(ac2.direct_imap.get_all_messages()) == 1
while 1:
ac2._evtracker.get_matching("DC_EVENT_IMAP_MESSAGE_MOVED")
ac2.direct_imap.select_config_folder("inbox")
nr_msgs = len(ac2.direct_imap.get_all_messages())
assert nr_msgs > 0
if nr_msgs == 1:
break
def test_configure_error_msgs_wrong_pw(acfactory):
@@ -2130,7 +2150,7 @@ def test_status(acfactory):
chat12.send_text("hello")
msg = ac2._evtracker.wait_next_incoming_message()
assert msg.text == "hello"
assert msg.get_sender_contact().status == ""
assert not msg.get_sender_contact().status
def test_group_quote(acfactory, lp):

View File

@@ -295,8 +295,8 @@ class TestOfflineChat:
assert d["archived"] == chat.is_archived()
# assert d["param"] == chat.param
assert d["color"] == chat.get_color()
assert d["profile_image"] == "" if chat.get_profile_image() is None else chat.get_profile_image()
assert d["draft"] == "" if chat.get_draft() is None else chat.get_draft()
assert not d["profile_image"] if chat.get_profile_image() is None else chat.get_profile_image()
assert not d["draft"] if chat.get_draft() is None else chat.get_draft()
def test_group_chat_creation_with_translation(self, ac1):
ac1.set_stock_translation(const.DC_STR_GROUP_NAME_CHANGED_BY_YOU, "abc %1$s xyz %2$s")

View File

@@ -151,7 +151,7 @@ def test_markseen_invalid_message_ids(acfactory):
ac1 = acfactory.get_pseudo_configured_account()
contact1 = ac1.create_contact("some1@example.com", name="some1")
chat = contact1.create_chat()
chat.send_text("one messae")
chat.send_text("one message")
ac1._evtracker.get_matching("DC_EVENT_MSGS_CHANGED")
msg_ids = [9]
lib.dc_markseen_msgs(ac1._dc_context, msg_ids, len(msg_ids))

View File

@@ -8,7 +8,7 @@ envlist =
[testenv]
commands =
pytest -n6 --extra-info -v -rsXx --ignored --strict-tls {posargs: tests examples}
pytest -n6 --exitfirst --extra-info -v -rsXx --ignored --strict-tls {posargs: tests examples}
pip wheel . -w {toxworkdir}/wheelhouse --no-deps
setenv =
# Avoid stack overflow when Rust core is built without optimizations.
@@ -55,7 +55,7 @@ deps =
pygments
restructuredtext_lint
commands =
black --check --diff setup.py install_python_bindings.py src/deltachat examples/ tests/
black --quiet --check --diff setup.py install_python_bindings.py src/deltachat examples/ tests/
ruff src/deltachat tests/ examples/
rst-lint --encoding 'utf-8' README.rst

View File

@@ -20,7 +20,9 @@ and an own build machine.
- `run_all.sh` builds Python wheels
- `aarch64-unknown-linux-musl.sh` cross-compiles static `deltachat-rpc-server` for aarch64
- `zig-rpc-server.sh` compiles binaries of `deltachat-rpc-server` using Zig toolchain statically linked against musl libc.
- `android-rpc-server.sh` compiles binaries of `deltachat-rpc-server` using Android NDK.
## Triggering runs on the build machine locally (fast!)

View File

@@ -1,18 +0,0 @@
#!/bin/sh
#
# Build statically linked deltachat-rpc-server for aarch64-unknown-linux-musl.
set -x
set -e
# Download Zig
rm -fr zig-linux-x86_64-0.10.1 zig-linux-x86_64-0.10.1.tar.xz
wget https://ziglang.org/download/0.10.1/zig-linux-x86_64-0.10.1.tar.xz
tar xf zig-linux-x86_64-0.10.1.tar.xz
export PATH="$PATH:$PWD/zig-linux-x86_64-0.10.1"
cargo install cargo-zigbuild
rustup target add aarch64-unknown-linux-musl
cargo zigbuild --release --target aarch64-unknown-linux-musl -p deltachat-rpc-server --features vendored

View File

@@ -1,44 +0,0 @@
#!/bin/sh
# Build deltachat-rpc-server for Android.
set -e
test -n "$ANDROID_NDK_ROOT" || exit 1
RUSTUP_TOOLCHAIN="1.64.0"
rustup install "$RUSTUP_TOOLCHAIN"
rustup target add armv7-linux-androideabi aarch64-linux-android i686-linux-android x86_64-linux-android --toolchain "$RUSTUP_TOOLCHAIN"
KERNEL="$(uname -s | tr '[:upper:]' '[:lower:]')"
ARCH="$(uname -m)"
NDK_HOST_TAG="$KERNEL-$ARCH"
TOOLCHAIN="$ANDROID_NDK_ROOT/toolchains/llvm/prebuilt/$NDK_HOST_TAG"
export PATH="$PATH:$TOOLCHAIN/bin/"
PACKAGE="deltachat-rpc-server"
export CARGO_PROFILE_RELEASE_LTO=on
CARGO_TARGET_ARMV7_LINUX_ANDROIDEABI_LINKER="$TOOLCHAIN/bin/armv7a-linux-androideabi16-clang" \
CFLAGS=-D__ANDROID_API__=16 \
TARGET_CC=armv7a-linux-androideabi16-clang \
TARGET_AR=llvm-ar \
cargo "+$RUSTUP_TOOLCHAIN" rustc --release --target armv7-linux-androideabi -p $PACKAGE
CARGO_TARGET_AARCH64_LINUX_ANDROID_LINKER="$TOOLCHAIN/bin/aarch64-linux-android21-clang" \
CFLAGS=-D__ANDROID_API__=21 \
TARGET_CC=aarch64-linux-android21-clang \
TARGET_AR=llvm-ar \
cargo "+$RUSTUP_TOOLCHAIN" rustc --release --target aarch64-linux-android -p $PACKAGE
CARGO_TARGET_I686_LINUX_ANDROID_LINKER="$TOOLCHAIN/bin/i686-linux-android16-clang" \
CFLAGS=-D__ANDROID_API__=16 \
TARGET_CC=i686-linux-android16-clang \
TARGET_AR=llvm-ar \
cargo "+$RUSTUP_TOOLCHAIN" rustc --release --target i686-linux-android -p $PACKAGE
CARGO_TARGET_X86_64_LINUX_ANDROID_LINKER="$TOOLCHAIN/bin/x86_64-linux-android21-clang" \
CFLAGS=-D__ANDROID_API__=21 \
TARGET_CC=x86_64-linux-android21-clang \
TARGET_AR=llvm-ar \
cargo "+$RUSTUP_TOOLCHAIN" rustc --release --target x86_64-linux-android -p $PACKAGE

View File

@@ -1,3 +1,3 @@
#!/bin/sh
# Run clippy for all Rust code in the project.
cargo clippy --workspace --tests --examples --benches -- -D warnings
cargo clippy --workspace --all-targets --all-features -- -D warnings

4
scripts/codespell.sh Executable file
View File

@@ -0,0 +1,4 @@
#!/bin/sh
codespell \
--skip './test-data,./.git,node_modules,.mypy_cache,./src/provider/data.rs,.tox,site-packages,target,Cargo.lock,*.js.map,package-lock.json,./proptest-regressions' \
--ignore-words-list crate,keypair,keypairs,iif

View File

@@ -1,370 +1,370 @@
resources:
- name: deltachat-core-rust
type: git
icon: github
source:
branch: master
uri: https://github.com/deltachat/deltachat-core-rust.git
- name: deltachat-core-rust
type: git
icon: github
source:
branch: master
uri: https://github.com/deltachat/deltachat-core-rust.git
- name: deltachat-core-rust-release
type: git
icon: github
source:
branch: master
uri: https://github.com/deltachat/deltachat-core-rust.git
tag_filter: "v*"
- name: deltachat-core-rust-release
type: git
icon: github
source:
branch: master
uri: https://github.com/deltachat/deltachat-core-rust.git
tag_filter: "v*"
jobs:
- name: doxygen
plan:
- get: deltachat-core-rust
trigger: true
- name: doxygen
plan:
- get: deltachat-core-rust
trigger: true
# Build Doxygen documentation
- task: build-doxygen
config:
inputs:
- name: deltachat-core-rust
outputs:
- name: c-docs
image_resource:
source:
repository: alpine
type: registry-image
platform: linux
run:
path: sh
args:
- -ec
- |
apk add --no-cache doxygen git
cd deltachat-core-rust
scripts/run-doxygen.sh
cd ..
cp -av deltachat-core-rust/deltachat-ffi/html deltachat-core-rust/deltachat-ffi/xml c-docs/
# Build Doxygen documentation
- task: build-doxygen
config:
inputs:
- name: deltachat-core-rust
outputs:
- name: c-docs
image_resource:
source:
repository: alpine
type: registry-image
platform: linux
run:
path: sh
args:
- -ec
- |
apk add --no-cache doxygen git
cd deltachat-core-rust
scripts/run-doxygen.sh
cd ..
cp -av deltachat-core-rust/deltachat-ffi/html deltachat-core-rust/deltachat-ffi/xml c-docs/
- task: upload-c-docs
config:
inputs:
- name: c-docs
image_resource:
type: registry-image
source:
repository: alpine
platform: linux
run:
path: sh
args:
- -ec
- |
apk add --no-cache rsync openssh-client
mkdir -p ~/.ssh
chmod 700 ~/.ssh
echo "(("c.delta.chat".private_key))" > ~/.ssh/id_ed25519
chmod 600 ~/.ssh/id_ed25519
rsync -e "ssh -o StrictHostKeyChecking=no" -avz --delete c-docs/html/ delta@c.delta.chat:build-c/master
- task: upload-c-docs
config:
inputs:
- name: c-docs
image_resource:
type: registry-image
source:
repository: alpine
platform: linux
run:
path: sh
args:
- -ec
- |
apk add --no-cache rsync openssh-client
mkdir -p ~/.ssh
chmod 700 ~/.ssh
echo "(("c.delta.chat".private_key))" > ~/.ssh/id_ed25519
chmod 600 ~/.ssh/id_ed25519
rsync -e "ssh -o StrictHostKeyChecking=no" -avz --delete c-docs/html/ delta@c.delta.chat:build-c/master
- name: python-x86_64
plan:
- get: deltachat-core-rust
- get: deltachat-core-rust-release
trigger: true
- name: python-x86_64
plan:
- get: deltachat-core-rust
- get: deltachat-core-rust-release
trigger: true
# Build manylinux image with additional dependencies
- task: build-coredeps
privileged: true
config:
inputs:
# Building the latest, not tagged coredeps
- name: deltachat-core-rust
image_resource:
source:
repository: concourse/oci-build-task
type: registry-image
outputs:
- name: coredeps-image
path: image
params:
CONTEXT: deltachat-core-rust/scripts/coredeps
UNPACK_ROOTFS: "true"
BUILD_ARG_BASEIMAGE: quay.io/pypa/manylinux2014_x86_64
platform: linux
caches:
- path: cache
run:
path: build
# Build manylinux image with additional dependencies
- task: build-coredeps
privileged: true
config:
inputs:
# Building the latest, not tagged coredeps
- name: deltachat-core-rust
image_resource:
source:
repository: concourse/oci-build-task
type: registry-image
outputs:
- name: coredeps-image
path: image
params:
CONTEXT: deltachat-core-rust/scripts/coredeps
UNPACK_ROOTFS: "true"
BUILD_ARG_BASEIMAGE: quay.io/pypa/manylinux2014_x86_64
platform: linux
caches:
- path: cache
run:
path: build
# Use built image to build python wheels
- task: build-wheels
image: coredeps-image
config:
inputs:
- name: deltachat-core-rust-release
path: .
outputs:
- name: py-docs
path: ./python/doc/_build/
# Binary wheels
- name: py-wheels
path: ./python/.docker-tox/wheelhouse/
platform: linux
run:
path: bash
args:
- -exc
- |
scripts/run_all.sh
# Use built image to build python wheels
- task: build-wheels
image: coredeps-image
config:
inputs:
- name: deltachat-core-rust-release
path: .
outputs:
- name: py-docs
path: ./python/doc/_build/
# Binary wheels
- name: py-wheels
path: ./python/.docker-tox/wheelhouse/
platform: linux
run:
path: bash
args:
- -exc
- |
scripts/run_all.sh
# Upload python docs to py.delta.chat
- task: upload-py-docs
config:
inputs:
- name: py-docs
image_resource:
type: registry-image
source:
repository: alpine
platform: linux
run:
path: sh
args:
- -ec
- |
apk add --no-cache rsync openssh-client
mkdir -p ~/.ssh
chmod 700 ~/.ssh
echo "(("c.delta.chat".private_key))" > ~/.ssh/id_ed25519
chmod 600 ~/.ssh/id_ed25519
rsync -e "ssh -o StrictHostKeyChecking=no" -avz --delete py-docs/html/ delta@py.delta.chat:build/master
# Upload python docs to py.delta.chat
- task: upload-py-docs
config:
inputs:
- name: py-docs
image_resource:
type: registry-image
source:
repository: alpine
platform: linux
run:
path: sh
args:
- -ec
- |
apk add --no-cache rsync openssh-client
mkdir -p ~/.ssh
chmod 700 ~/.ssh
echo "(("c.delta.chat".private_key))" > ~/.ssh/id_ed25519
chmod 600 ~/.ssh/id_ed25519
rsync -e "ssh -o StrictHostKeyChecking=no" -avz --delete py-docs/html/ delta@py.delta.chat:build/master
# Upload x86_64 wheels and source packages
- task: upload-wheels
config:
inputs:
- name: py-wheels
image_resource:
type: registry-image
source:
repository: debian
platform: linux
run:
path: sh
args:
- -ec
- |
apt-get update -y
apt-get install -y --no-install-recommends python3-pip python3-setuptools
pip3 install devpi
devpi use https://m.devpi.net/dc/master
devpi login ((devpi.login)) --password ((devpi.password))
devpi upload py-wheels/*manylinux201*
# Upload x86_64 wheels and source packages
- task: upload-wheels
config:
inputs:
- name: py-wheels
image_resource:
type: registry-image
source:
repository: debian
platform: linux
run:
path: sh
args:
- -ec
- |
apt-get update -y
apt-get install -y --no-install-recommends python3-pip python3-setuptools
pip3 install devpi
devpi use https://m.devpi.net/dc/master
devpi login ((devpi.login)) --password ((devpi.password))
devpi upload py-wheels/*manylinux201*
- name: python-aarch64
plan:
- get: deltachat-core-rust
- get: deltachat-core-rust-release
trigger: true
- name: python-aarch64
plan:
- get: deltachat-core-rust
- get: deltachat-core-rust-release
trigger: true
# Build manylinux image with additional dependencies
- task: build-coredeps
privileged: true
config:
inputs:
# Building the latest, not tagged coredeps
- name: deltachat-core-rust
image_resource:
source:
repository: concourse/oci-build-task
type: registry-image
outputs:
- name: coredeps-image
path: image
params:
CONTEXT: deltachat-core-rust/scripts/coredeps
UNPACK_ROOTFS: "true"
BUILD_ARG_BASEIMAGE: quay.io/pypa/manylinux2014_aarch64
platform: linux
caches:
- path: cache
run:
path: build
# Build manylinux image with additional dependencies
- task: build-coredeps
privileged: true
config:
inputs:
# Building the latest, not tagged coredeps
- name: deltachat-core-rust
image_resource:
source:
repository: concourse/oci-build-task
type: registry-image
outputs:
- name: coredeps-image
path: image
params:
CONTEXT: deltachat-core-rust/scripts/coredeps
UNPACK_ROOTFS: "true"
BUILD_ARG_BASEIMAGE: quay.io/pypa/manylinux2014_aarch64
platform: linux
caches:
- path: cache
run:
path: build
# Use built image to build python wheels
- task: build-wheels
image: coredeps-image
config:
inputs:
- name: deltachat-core-rust-release
path: .
outputs:
- name: py-wheels
path: ./python/.docker-tox/wheelhouse/
platform: linux
run:
path: bash
args:
- -exc
- |
scripts/run_all.sh
# Use built image to build python wheels
- task: build-wheels
image: coredeps-image
config:
inputs:
- name: deltachat-core-rust-release
path: .
outputs:
- name: py-wheels
path: ./python/.docker-tox/wheelhouse/
platform: linux
run:
path: bash
args:
- -exc
- |
scripts/run_all.sh
# Upload aarch64 wheels
- task: upload-wheels
config:
inputs:
- name: py-wheels
image_resource:
type: registry-image
source:
repository: debian
platform: linux
run:
path: sh
args:
- -ec
- |
apt-get update -y
apt-get install -y --no-install-recommends python3-pip python3-setuptools
pip3 install devpi
devpi use https://m.devpi.net/dc/master
devpi login ((devpi.login)) --password ((devpi.password))
devpi upload py-wheels/*manylinux201*
# Upload aarch64 wheels
- task: upload-wheels
config:
inputs:
- name: py-wheels
image_resource:
type: registry-image
source:
repository: debian
platform: linux
run:
path: sh
args:
- -ec
- |
apt-get update -y
apt-get install -y --no-install-recommends python3-pip python3-setuptools
pip3 install devpi
devpi use https://m.devpi.net/dc/master
devpi login ((devpi.login)) --password ((devpi.password))
devpi upload py-wheels/*manylinux201*
- name: python-musl-x86_64
plan:
- get: deltachat-core-rust
- get: deltachat-core-rust-release
trigger: true
- name: python-musl-x86_64
plan:
- get: deltachat-core-rust
- get: deltachat-core-rust-release
trigger: true
# Build manylinux image with additional dependencies
- task: build-coredeps
privileged: true
config:
inputs:
# Building the latest, not tagged coredeps
- name: deltachat-core-rust
image_resource:
source:
repository: concourse/oci-build-task
type: registry-image
outputs:
- name: coredeps-image
path: image
params:
CONTEXT: deltachat-core-rust/scripts/coredeps
UNPACK_ROOTFS: "true"
BUILD_ARG_BASEIMAGE: quay.io/pypa/musllinux_1_1_x86_64
platform: linux
caches:
- path: cache
run:
path: build
# Build manylinux image with additional dependencies
- task: build-coredeps
privileged: true
config:
inputs:
# Building the latest, not tagged coredeps
- name: deltachat-core-rust
image_resource:
source:
repository: concourse/oci-build-task
type: registry-image
outputs:
- name: coredeps-image
path: image
params:
CONTEXT: deltachat-core-rust/scripts/coredeps
UNPACK_ROOTFS: "true"
BUILD_ARG_BASEIMAGE: quay.io/pypa/musllinux_1_1_x86_64
platform: linux
caches:
- path: cache
run:
path: build
# Use built image to build python wheels
- task: build-wheels
image: coredeps-image
config:
inputs:
- name: deltachat-core-rust-release
path: .
outputs:
- name: py-wheels
path: ./python/.docker-tox/wheelhouse/
platform: linux
run:
path: bash
args:
- -exc
- |
scripts/run_all.sh
# Use built image to build python wheels
- task: build-wheels
image: coredeps-image
config:
inputs:
- name: deltachat-core-rust-release
path: .
outputs:
- name: py-wheels
path: ./python/.docker-tox/wheelhouse/
platform: linux
run:
path: bash
args:
- -exc
- |
scripts/run_all.sh
# Upload musl x86_64 wheels
- task: upload-wheels
config:
inputs:
- name: py-wheels
image_resource:
type: registry-image
source:
repository: debian
platform: linux
run:
path: sh
args:
- -ec
- |
apt-get update -y
apt-get install -y --no-install-recommends python3-pip python3-setuptools
pip3 install devpi
devpi use https://m.devpi.net/dc/master
devpi login ((devpi.login)) --password ((devpi.password))
devpi upload py-wheels/*musllinux_1_1_x86_64*
# Upload musl x86_64 wheels
- task: upload-wheels
config:
inputs:
- name: py-wheels
image_resource:
type: registry-image
source:
repository: debian
platform: linux
run:
path: sh
args:
- -ec
- |
apt-get update -y
apt-get install -y --no-install-recommends python3-pip python3-setuptools
pip3 install devpi
devpi use https://m.devpi.net/dc/master
devpi login ((devpi.login)) --password ((devpi.password))
devpi upload py-wheels/*musllinux_1_1_x86_64*
- name: python-musl-aarch64
plan:
- get: deltachat-core-rust
- get: deltachat-core-rust-release
trigger: true
- name: python-musl-aarch64
plan:
- get: deltachat-core-rust
- get: deltachat-core-rust-release
trigger: true
# Build manylinux image with additional dependencies
- task: build-coredeps
privileged: true
config:
inputs:
# Building the latest, not tagged coredeps
- name: deltachat-core-rust
image_resource:
source:
repository: concourse/oci-build-task
type: registry-image
outputs:
- name: coredeps-image
path: image
params:
CONTEXT: deltachat-core-rust/scripts/coredeps
UNPACK_ROOTFS: "true"
BUILD_ARG_BASEIMAGE: quay.io/pypa/musllinux_1_1_aarch64
platform: linux
caches:
- path: cache
run:
path: build
# Build manylinux image with additional dependencies
- task: build-coredeps
privileged: true
config:
inputs:
# Building the latest, not tagged coredeps
- name: deltachat-core-rust
image_resource:
source:
repository: concourse/oci-build-task
type: registry-image
outputs:
- name: coredeps-image
path: image
params:
CONTEXT: deltachat-core-rust/scripts/coredeps
UNPACK_ROOTFS: "true"
BUILD_ARG_BASEIMAGE: quay.io/pypa/musllinux_1_1_aarch64
platform: linux
caches:
- path: cache
run:
path: build
# Use built image to build python wheels
- task: build-wheels
image: coredeps-image
config:
inputs:
- name: deltachat-core-rust-release
path: .
outputs:
- name: py-wheels
path: ./python/.docker-tox/wheelhouse/
platform: linux
run:
path: bash
args:
- -exc
- |
scripts/run_all.sh
# Use built image to build python wheels
- task: build-wheels
image: coredeps-image
config:
inputs:
- name: deltachat-core-rust-release
path: .
outputs:
- name: py-wheels
path: ./python/.docker-tox/wheelhouse/
platform: linux
run:
path: bash
args:
- -exc
- |
scripts/run_all.sh
# Upload musl aarch64 wheels
- task: upload-wheels
config:
inputs:
- name: py-wheels
image_resource:
type: registry-image
source:
repository: debian
platform: linux
run:
path: sh
args:
- -ec
- |
apt-get update -y
apt-get install -y --no-install-recommends python3-pip python3-setuptools
pip3 install devpi
devpi use https://m.devpi.net/dc/master
devpi login ((devpi.login)) --password ((devpi.password))
devpi upload py-wheels/*musllinux_1_1_aarch64*
# Upload musl aarch64 wheels
- task: upload-wheels
config:
inputs:
- name: py-wheels
image_resource:
type: registry-image
source:
repository: debian
platform: linux
run:
path: sh
args:
- -ec
- |
apt-get update -y
apt-get install -y --no-install-recommends python3-pip python3-setuptools
pip3 install devpi
devpi use https://m.devpi.net/dc/master
devpi login ((devpi.login)) --password ((devpi.password))
devpi upload py-wheels/*musllinux_1_1_aarch64*

View File

@@ -7,7 +7,7 @@ set -euo pipefail
#
# Avoid using rustup here as it depends on reading /proc/self/exe and
# has problems running under QEMU.
RUST_VERSION=1.64.0
RUST_VERSION=1.68.0
ARCH="$(uname -m)"
test -f "/lib/libc.musl-$ARCH.so.1" && LIBC=musl || LIBC=gnu

View File

@@ -11,15 +11,17 @@ out_domains = ""
out_ids = ""
domains_set = set()
def camel(name):
words = name.split("_")
return "".join(w.capitalize() for i, w in enumerate(words))
def cleanstr(s):
s = s.strip()
s = s.replace("\n", " ")
s = s.replace("\\", "\\\\")
s = s.replace("\"", "\\\"")
s = s.replace('"', '\\"')
return s
@@ -64,7 +66,13 @@ def process_config_defaults(data):
config_defaults = data.get("config_defaults", "")
for key in config_defaults:
value = str(config_defaults[key])
defaults += " ConfigDefault { key: Config::" + camel(key) + ", value: \"" + value + "\" },\n"
defaults += (
" ConfigDefault { key: Config::"
+ camel(key)
+ ', value: "'
+ value
+ '" },\n'
)
defaults += " ])"
return defaults
@@ -88,11 +96,11 @@ def process_data(data, file):
raise TypeError("domain used twice: " + domain)
domains_set.add(domain)
domains += " (\"" + domain + "\", &*" + file2varname(file) + "),\n"
domains += ' ("' + domain + '", &*' + file2varname(file) + "),\n"
comment += domain + ", "
ids = ""
ids += " (\"" + file2id(file) + "\", &*" + file2varname(file) + "),\n"
ids += ' ("' + file2id(file) + '", &*' + file2varname(file) + "),\n"
server = ""
has_imap = False
@@ -120,8 +128,19 @@ def process_data(data, file):
if username_pattern != "EMAIL" and username_pattern != "EMAILLOCALPART":
raise TypeError("bad username pattern")
server += (" Server { protocol: " + protocol.capitalize() + ", socket: " + socket.capitalize() + ", hostname: \""
+ hostname + "\", port: " + str(port) + ", username_pattern: " + username_pattern.capitalize() + " },\n")
server += (
" Server { protocol: "
+ protocol.capitalize()
+ ", socket: "
+ socket.capitalize()
+ ', hostname: "'
+ hostname
+ '", port: '
+ str(port)
+ ", username_pattern: "
+ username_pattern.capitalize()
+ " },\n"
)
opt = process_opt(data)
config_defaults = process_config_defaults(data)
@@ -133,12 +152,16 @@ def process_data(data, file):
before_login_hint = cleanstr(data.get("before_login_hint", ""))
after_login_hint = cleanstr(data.get("after_login_hint", ""))
if (not has_imap and not has_smtp) or (has_imap and has_smtp):
provider += "static " + file2varname(file) + ": Lazy<Provider> = Lazy::new(|| Provider {\n"
provider += " id: \"" + file2id(file) + "\",\n"
provider += (
"static "
+ file2varname(file)
+ ": Lazy<Provider> = Lazy::new(|| Provider {\n"
)
provider += ' id: "' + file2id(file) + '",\n'
provider += " status: Status::" + status.capitalize() + ",\n"
provider += " before_login_hint: \"" + before_login_hint + "\",\n"
provider += " after_login_hint: \"" + after_login_hint + "\",\n"
provider += " overview_page: \"" + file2url(file) + "\",\n"
provider += ' before_login_hint: "' + before_login_hint + '",\n'
provider += ' after_login_hint: "' + after_login_hint + '",\n'
provider += ' overview_page: "' + file2url(file) + '",\n'
provider += " server: vec![\n" + server + " ],\n"
provider += " opt: " + opt + ",\n"
provider += " config_defaults: " + config_defaults + ",\n"
@@ -148,7 +171,9 @@ def process_data(data, file):
raise TypeError("SMTP and IMAP must be specified together or left out both")
if status != "OK" and before_login_hint == "":
raise TypeError("status PREPARATION or BROKEN requires before_login_hint: " + file)
raise TypeError(
"status PREPARATION or BROKEN requires before_login_hint: " + file
)
# finally, add the provider
global out_all, out_domains, out_ids
@@ -172,7 +197,7 @@ def process_file(file):
def process_dir(dir):
print("processing directory: {}".format(dir), file=sys.stderr)
files = sorted(f for f in dir.iterdir() if f.suffix == '.md')
files = sorted(f for f in dir.iterdir() if f.suffix == ".md")
for f in files:
process_file(f)
@@ -181,28 +206,41 @@ if __name__ == "__main__":
if len(sys.argv) < 2:
raise SystemExit("usage: update.py DIR_WITH_MD_FILES > data.rs")
out_all = ("// file generated by src/provider/update.py\n\n"
"use crate::provider::Protocol::*;\n"
"use crate::provider::Socket::*;\n"
"use crate::provider::UsernamePattern::*;\n"
"use crate::provider::{\n"
" Config, ConfigDefault, Oauth2Authorizer, Provider, ProviderOptions, Server, Status,\n"
"};\n"
"use std::collections::HashMap;\n\n"
"use once_cell::sync::Lazy;\n\n")
out_all = (
"// file generated by src/provider/update.py\n\n"
"use crate::provider::Protocol::*;\n"
"use crate::provider::Socket::*;\n"
"use crate::provider::UsernamePattern::*;\n"
"use crate::provider::{\n"
" Config, ConfigDefault, Oauth2Authorizer, Provider, ProviderOptions, Server, Status,\n"
"};\n"
"use std::collections::HashMap;\n\n"
"use once_cell::sync::Lazy;\n\n"
)
process_dir(Path(sys.argv[1]))
out_all += "pub(crate) static PROVIDER_DATA: Lazy<HashMap<&'static str, &'static Provider>> = Lazy::new(|| [\n"
out_all += out_domains;
out_all += "].iter().copied().collect());\n\n"
out_all += "pub(crate) static PROVIDER_DATA: Lazy<HashMap<&'static str, &'static Provider>> = Lazy::new(|| HashMap::from([\n"
out_all += out_domains
out_all += "]));\n\n"
out_all += "pub(crate) static PROVIDER_IDS: Lazy<HashMap<&'static str, &'static Provider>> = Lazy::new(|| [\n"
out_all += out_ids;
out_all += "].iter().copied().collect());\n\n"
out_all += "pub(crate) static PROVIDER_IDS: Lazy<HashMap<&'static str, &'static Provider>> = Lazy::new(|| HashMap::from([\n"
out_all += out_ids
out_all += "]));\n\n"
now = datetime.datetime.utcnow()
out_all += "pub static PROVIDER_UPDATED: Lazy<chrono::NaiveDate> = "\
"Lazy::new(|| chrono::NaiveDate::from_ymd_opt("+str(now.year)+", "+str(now.month)+", "+str(now.day)+").unwrap());\n"
if len(sys.argv) < 3:
now = datetime.datetime.utcnow()
else:
now = datetime.datetime.fromisoformat(sys.argv[2])
out_all += (
"pub static PROVIDER_UPDATED: Lazy<chrono::NaiveDate> = "
"Lazy::new(|| chrono::NaiveDate::from_ymd_opt("
+ str(now.year)
+ ", "
+ str(now.month)
+ ", "
+ str(now.day)
+ ").unwrap());\n"
)
print(out_all)

View File

@@ -0,0 +1,22 @@
#!/usr/bin/env bash
# Updates provider database.
# Returns 1 if the database is changed, 0 otherwise.
set -euo pipefail
export TZ=UTC
# Provider database revision.
REV=3c8f7e846c915a183dc44536fb5480d1f25d7c42
CORE_ROOT="$PWD"
TMP="$(mktemp -d)"
git clone --filter=blob:none https://github.com/deltachat/provider-db.git "$TMP"
cd "$TMP"
git checkout "$REV"
DATE=$(git show -s --format=%cs)
"$CORE_ROOT"/scripts/create-provider-data-rs.py "$TMP/_providers" "$DATE" >"$CORE_ROOT/src/provider/data.rs"
rustfmt "$CORE_ROOT/src/provider/data.rs"
rm -fr "$TMP"
cd "$CORE_ROOT"
test -z "$(git status --porcelain src/provider/data.rs)"

23
scripts/zig-rpc-server.sh Executable file
View File

@@ -0,0 +1,23 @@
#!/bin/sh
#
# Build statically linked deltachat-rpc-server using cargo-zigbuild.
set -x
set -e
unset RUSTFLAGS
ZIG_VERSION=0.11.0-dev.2213+515e1c93e
# Download Zig
rm -fr "$ZIG_VERSION" "zig-linux-x86_64-$ZIG_VERSION.tar.xz"
wget "https://ziglang.org/builds/zig-linux-x86_64-$ZIG_VERSION.tar.xz"
tar xf "zig-linux-x86_64-$ZIG_VERSION.tar.xz"
export PATH="$PWD/zig-linux-x86_64-$ZIG_VERSION:$PATH"
cargo install cargo-zigbuild
for TARGET in x86_64-unknown-linux-musl aarch64-unknown-linux-musl armv7-unknown-linux-musleabihf; do
rustup target add "$TARGET"
cargo zigbuild --release --target "$TARGET" -p deltachat-rpc-server --features vendored
done

View File

@@ -339,13 +339,13 @@ only on image changes.
In older specs, the profile-image was sent as an attachment
and `Chat-User-Avatar:` specified its name.
However, it turned out that these attachements are kind of unuexpected to users,
However, it turned out that these attachments are kind of unuexpected to users,
therefore the profile-image go to the header now.
# Locations
Locations can be attachted to messages using
Locations can be attached to messages using
[standard kml-files](https://www.opengeospatial.org/standards/kml/)
with well-known names.

View File

@@ -271,14 +271,14 @@ impl Accounts {
/// Notifies all accounts that the network may have become available.
pub async fn maybe_network(&self) {
for account in self.accounts.values() {
account.maybe_network().await;
account.scheduler.maybe_network().await;
}
}
/// Notifies all accounts that the network connection may have been lost.
pub async fn maybe_network_lost(&self) {
for account in self.accounts.values() {
account.maybe_network_lost().await;
account.scheduler.maybe_network_lost(account).await;
}
}
@@ -427,7 +427,7 @@ impl Config {
Ok(cfg)
}
/// Removes an existing acccount entirely.
/// Removes an existing account entirely.
pub async fn remove_account(&mut self, id: u32) -> Result<()> {
{
if let Some(idx) = self.inner.accounts.iter().position(|e| e.id == id) {
@@ -487,7 +487,7 @@ struct AccountConfig {
}
impl AccountConfig {
/// Get the canoncial dbfile name for this configuration.
/// Get the canonical dbfile name for this configuration.
pub fn dbfile(&self, accounts_dir: &Path) -> std::path::PathBuf {
accounts_dir.join(&self.dir).join(DB_NAME)
}

View File

@@ -235,7 +235,7 @@ mod tests {
assert!(Aheader::from_str("foo").is_err());
assert!(Aheader::from_str("\n\n\n").is_err());
assert!(Aheader::from_str(" ;;").is_err());
assert!(Aheader::from_str("addr=a@t.de; unknwon=1; keydata=jau").is_err());
assert!(Aheader::from_str("addr=a@t.de; unknown=1; keydata=jau").is_err());
}
#[test]

View File

@@ -334,7 +334,7 @@ async fn set_dkim_works_timestamp(
async fn clear_dkim_works(context: &Context) -> Result<()> {
context
.sql
.execute("DELETE FROM sending_domains", paramsv![])
.execute("DELETE FROM sending_domains", ())
.await?;
Ok(())
}

View File

@@ -4,13 +4,16 @@ use core::cmp::max;
use std::ffi::OsStr;
use std::fmt;
use std::io::Cursor;
use std::iter::FusedIterator;
use std::path::{Path, PathBuf};
use anyhow::{format_err, Context as _, Result};
use futures::StreamExt;
use image::{DynamicImage, ImageFormat};
use num_traits::FromPrimitive;
use tokio::io::AsyncWriteExt;
use tokio::{fs, io};
use tokio_stream::wrappers::ReadDirStream;
use crate::config::Config;
use crate::constants::{
@@ -160,9 +163,9 @@ impl<'a> BlobObject<'a> {
pub fn from_path(context: &'a Context, path: &Path) -> Result<BlobObject<'a>> {
let rel_path = path
.strip_prefix(context.get_blobdir())
.context("wrong blobdir")?;
.with_context(|| format!("wrong blobdir: {}", path.display()))?;
if !BlobObject::is_acceptible_blob_name(rel_path) {
return Err(format_err!("wrong name"));
return Err(format_err!("bad blob name: {}", rel_path.display()));
}
let name = rel_path.to_str().context("wrong name")?;
BlobObject::from_name(context, name.to_string())
@@ -468,6 +471,87 @@ impl<'a> fmt::Display for BlobObject<'a> {
}
}
/// All files in the blobdir.
///
/// This exists so we can have a [`BlobDirIter`] which needs something to own the data of
/// it's `&Path`. Use [`BlobDirContents::iter`] to create the iterator.
///
/// Additionally pre-allocating this means we get a length for progress report.
pub(crate) struct BlobDirContents<'a> {
inner: Vec<PathBuf>,
context: &'a Context,
}
impl<'a> BlobDirContents<'a> {
pub(crate) async fn new(context: &'a Context) -> Result<BlobDirContents<'a>> {
let readdir = fs::read_dir(context.get_blobdir()).await?;
let inner = ReadDirStream::new(readdir)
.filter_map(|entry| async move {
match entry {
Ok(entry) => Some(entry),
Err(err) => {
error!(context, "Failed to read blob file: {err}");
None
}
}
})
.filter_map(|entry| async move {
match entry.file_type().await.ok()?.is_file() {
true => Some(entry.path()),
false => {
warn!(
context,
"Export: Found blob dir entry {} that is not a file, ignoring",
entry.path().display()
);
None
}
}
})
.collect()
.await;
Ok(Self { inner, context })
}
pub(crate) fn iter(&self) -> BlobDirIter<'_> {
BlobDirIter::new(self.context, self.inner.iter())
}
pub(crate) fn len(&self) -> usize {
self.inner.len()
}
}
/// A iterator over all the [`BlobObject`]s in the blobdir.
pub(crate) struct BlobDirIter<'a> {
iter: std::slice::Iter<'a, PathBuf>,
context: &'a Context,
}
impl<'a> BlobDirIter<'a> {
fn new(context: &'a Context, iter: std::slice::Iter<'a, PathBuf>) -> BlobDirIter<'a> {
Self { iter, context }
}
}
impl<'a> Iterator for BlobDirIter<'a> {
type Item = BlobObject<'a>;
fn next(&mut self) -> Option<Self::Item> {
for path in self.iter.by_ref() {
// In theory this can error but we'd have corrupted filenames in the blobdir, so
// silently skipping them is fine.
match BlobObject::from_path(self.context, path) {
Ok(blob) => return Some(blob),
Err(err) => warn!(self.context, "{err}"),
}
}
None
}
}
impl FusedIterator for BlobDirIter<'_> {}
fn encode_img(img: &DynamicImage, encoded: &mut Vec<u8>) -> anyhow::Result<()> {
encoded.clear();
let mut buf = Cursor::new(encoded);

View File

@@ -46,7 +46,7 @@ use crate::{location, sql};
pub enum ChatItem {
/// Chat message stored in the database.
Message {
/// Database ID of the messsage.
/// Database ID of the message.
msg_id: MsgId,
},
@@ -276,7 +276,7 @@ impl ChatId {
grpname,
grpid,
create_blocked,
create_smeared_timestamp(context).await,
create_smeared_timestamp(context),
create_protected,
param.unwrap_or_default(),
],
@@ -418,7 +418,7 @@ impl ChatId {
ProtectionStatus::Protected => match chat.typ {
Chattype::Single | Chattype::Group | Chattype::Broadcast => {
let contact_ids = get_chat_contacts(context, self).await?;
for contact_id in contact_ids.into_iter() {
for contact_id in contact_ids {
let contact = Contact::get_by_id(context, contact_id).await?;
if contact.is_verified(context).await? != VerifiedStatus::BidirectVerified {
bail!("{} is not verified.", contact.get_display_name());
@@ -482,7 +482,7 @@ impl ChatId {
self,
&msg_text,
cmd,
create_smeared_timestamp(context).await,
create_smeared_timestamp(context),
None,
None,
None,
@@ -639,7 +639,10 @@ impl ChatId {
context.emit_msgs_changed_without_ids();
context.set_config(Config::LastHousekeeping, None).await?;
context.interrupt_inbox(InterruptInfo::new(false)).await;
context
.scheduler
.interrupt_inbox(InterruptInfo::new(false))
.await;
if chat.is_self_talk() {
let mut msg = Message::new(Viewtype::Text);
@@ -853,7 +856,7 @@ impl ChatId {
AND c.blocked=0
AND c.archived=1
",
paramsv![],
(),
)
.await?
} else {
@@ -1206,7 +1209,10 @@ impl Chat {
}
}
Err(err) => {
error!(context, "faild to load contacts for {}: {:#}", chat.id, err);
error!(
context,
"failed to load contacts for {}: {:#}", chat.id, err
);
}
}
chat.name = chat_name;
@@ -1664,7 +1670,7 @@ impl Chat {
maybe_set_logging_xdc(context, msg, self.id).await?;
}
context.interrupt_ephemeral_task().await;
context.scheduler.interrupt_ephemeral_task().await;
Ok(msg.id)
}
}
@@ -1956,7 +1962,6 @@ impl ChatIdBlocked {
_ => (),
}
let created_timestamp = create_smeared_timestamp(context).await;
let chat_id = context
.sql
.transaction(move |transaction| {
@@ -1969,7 +1974,7 @@ impl ChatIdBlocked {
chat_name,
params.to_string(),
create_blocked as u8,
created_timestamp,
create_smeared_timestamp(context)
],
)?;
let chat_id = ChatId::new(
@@ -2117,7 +2122,7 @@ async fn prepare_msg_common(
context,
msg,
update_msg_id,
create_smeared_timestamp(context).await,
create_smeared_timestamp(context),
)
.await?;
msg.chat_id = chat_id;
@@ -2148,7 +2153,7 @@ pub async fn is_contact_in_chat(
/// Sends a message object to a chat.
///
/// Sends the event #DC_EVENT_MSGS_CHANGED on succcess.
/// Sends the event #DC_EVENT_MSGS_CHANGED on success.
/// However, this does not imply, the message really reached the recipient -
/// sending may be delayed eg. due to network problems. However, from your
/// view, you're done with the message. Sooner or later it will find its way.
@@ -2199,7 +2204,10 @@ async fn send_msg_inner(context: &Context, chat_id: ChatId, msg: &mut Message) -
context.emit_event(EventType::LocationChanged(Some(ContactId::SELF)));
}
context.interrupt_smtp(InterruptInfo::new(false)).await;
context
.scheduler
.interrupt_smtp(InterruptInfo::new(false))
.await;
}
Ok(msg.id)
@@ -2512,7 +2520,7 @@ pub async fn get_chat_msgs_ex(
context
.sql
.query_map(
// GLOB is used here instead of LIKE becase it is case-sensitive
// GLOB is used here instead of LIKE because it is case-sensitive
"SELECT m.id AS id, m.timestamp AS timestamp, m.param AS param, m.from_id AS from_id, m.to_id AS to_id
FROM msgs m
WHERE m.chat_id=?
@@ -2576,7 +2584,7 @@ pub async fn marknoticed_chat(context: &Context, chat_id: ChatId) -> Result<()>
"SELECT DISTINCT(m.chat_id) FROM msgs m
LEFT JOIN chats c ON m.chat_id=c.id
WHERE m.state=10 AND m.hidden=0 AND m.chat_id>9 AND c.blocked=0 AND c.archived=1",
paramsv![],
(),
|row| row.get::<_, ChatId>(0),
|ids| ids.collect::<Result<Vec<_>, _>>().map_err(Into::into)
)
@@ -2842,7 +2850,7 @@ pub async fn create_group_chat(
Chattype::Group,
chat_name,
grpid,
create_smeared_timestamp(context).await,
create_smeared_timestamp(context),
],
)
.await?;
@@ -2900,7 +2908,7 @@ pub async fn create_broadcast_list(context: &Context) -> Result<ChatId> {
Chattype::Broadcast,
chat_name,
grpid,
create_smeared_timestamp(context).await,
create_smeared_timestamp(context),
],
)
.await?;
@@ -3361,7 +3369,7 @@ pub async fn forward_msgs(context: &Context, msg_ids: &[MsgId], chat_id: ChatId)
if let Some(reason) = chat.why_cant_send(context).await? {
bail!("cannot send to {}: {}", chat_id, reason);
}
curr_timestamp = create_smeared_timestamps(context, msg_ids.len()).await;
curr_timestamp = create_smeared_timestamps(context, msg_ids.len());
let ids = context
.sql
.query_map(
@@ -3431,7 +3439,10 @@ pub async fn forward_msgs(context: &Context, msg_ids: &[MsgId], chat_id: ChatId)
.await?;
curr_timestamp += 1;
if create_send_msg_job(context, new_msg_id).await?.is_some() {
context.interrupt_smtp(InterruptInfo::new(false)).await;
context
.scheduler
.interrupt_smtp(InterruptInfo::new(false))
.await;
}
}
created_chats.push(chat_id);
@@ -3486,7 +3497,10 @@ pub async fn resend_msgs(context: &Context, msg_ids: &[MsgId]) -> Result<()> {
msg_id: msg.id,
});
if create_send_msg_job(context, msg.id).await?.is_some() {
context.interrupt_smtp(InterruptInfo::new(false)).await;
context
.scheduler
.interrupt_smtp(InterruptInfo::new(false))
.await;
}
}
}
@@ -3498,10 +3512,7 @@ pub(crate) async fn get_chat_cnt(context: &Context) -> Result<usize> {
// no database, no chats - this is no error (needed eg. for information)
let count = context
.sql
.count(
"SELECT COUNT(*) FROM chats WHERE id>9 AND blocked=0;",
paramsv![],
)
.count("SELECT COUNT(*) FROM chats WHERE id>9 AND blocked=0;", ())
.await?;
Ok(count)
} else {
@@ -3563,7 +3574,7 @@ pub async fn add_device_msg_with_importance(
msg.try_calc_and_set_dimensions(context).await.ok();
prepare_msg_blob(context, msg).await?;
let timestamp_sent = create_smeared_timestamp(context).await;
let timestamp_sent = create_smeared_timestamp(context);
// makes sure, the added message is the last one,
// even if the date is wrong (useful esp. when warning about bad dates)
@@ -3663,7 +3674,7 @@ pub async fn was_device_msg_ever_added(context: &Context, label: &str) -> Result
// - deletion in `msgs` with `ContactId::DEVICE` makes sure,
// no wrong information are shown in the device chat
// - deletion in `devmsglabels` makes sure,
// deleted messages are resetted and useful messages can be added again
// deleted messages are reset and useful messages can be added again
// - we reset the config-option `QuotaExceeding`
// that is used as a helper to drive the corresponding device message.
pub(crate) async fn delete_and_reset_all_device_msgs(context: &Context) -> Result<()> {
@@ -3674,17 +3685,14 @@ pub(crate) async fn delete_and_reset_all_device_msgs(context: &Context) -> Resul
paramsv![ContactId::DEVICE],
)
.await?;
context
.sql
.execute("DELETE FROM devmsglabels;", paramsv![])
.await?;
context.sql.execute("DELETE FROM devmsglabels;", ()).await?;
// Insert labels for welcome messages to avoid them being readded on reconfiguration.
context
.sql
.execute(
r#"INSERT INTO devmsglabels (label) VALUES ("core-welcome-image"), ("core-welcome")"#,
paramsv![],
(),
)
.await?;
context.set_config(Config::QuotaExceeding, None).await?;
@@ -4091,7 +4099,6 @@ mod tests {
send_text_msg(&alice, alice_chat_id, "populate".to_string()).await?;
add_contact_to_chat(&alice, alice_chat_id, bob_id).await?;
tokio::time::sleep(std::time::Duration::from_millis(1100)).await;
let add1 = alice.pop_sent_msg().await;
add_contact_to_chat(&alice, alice_chat_id, claire_id).await?;
@@ -4110,29 +4117,18 @@ mod tests {
remove_contact_from_chat(&alice, alice_chat_id, daisy_id).await?;
let remove2 = alice.pop_sent_msg().await;
tokio::time::sleep(std::time::Duration::from_millis(1100)).await;
assert_eq!(get_chat_contacts(&alice, alice_chat_id).await?.len(), 2);
// Bob receives the add and deletion messages out of order
let bob = TestContext::new_bob().await;
bob.recv_msg(&add1).await;
tokio::time::sleep(std::time::Duration::from_millis(1100)).await;
bob.recv_msg(&add3).await;
tokio::time::sleep(std::time::Duration::from_millis(1100)).await;
let bob_chat_id = bob.recv_msg(&add2).await.chat_id;
tokio::time::sleep(std::time::Duration::from_millis(1100)).await;
assert_eq!(get_chat_contacts(&bob, bob_chat_id).await?.len(), 4);
bob.recv_msg(&remove2).await;
tokio::time::sleep(std::time::Duration::from_millis(1100)).await;
bob.recv_msg(&remove1).await;
tokio::time::sleep(std::time::Duration::from_millis(1100)).await;
assert_eq!(get_chat_contacts(&bob, bob_chat_id).await?.len(), 2);
Ok(())
@@ -4444,7 +4440,7 @@ mod tests {
.unwrap();
assert!(msg_id2.is_unset());
// ... unless everything is deleted and resetted - as needed eg. on device switch
// ... unless everything is deleted and reset - as needed eg. on device switch
delete_and_reset_all_device_msgs(&t).await.unwrap();
assert!(!was_device_msg_ever_added(&t, "some-label").await.unwrap());
let msg_id3 = add_device_msg(&t, Some("some-label"), Some(&mut msg))

View File

@@ -1,5 +1,6 @@
//! # Key-value configuration management.
use std::env;
use std::str::FromStr;
use anyhow::{ensure, Context as _, Result};
@@ -246,7 +247,7 @@ pub enum Config {
Configured,
/// All secondary self addresses separated by spaces
/// (`addr1@example.org addr2@exapmle.org addr3@example.org`)
/// (`addr1@example.org addr2@example.org addr3@example.org`)
SecondaryAddrs,
/// Read-only core version string.
@@ -300,6 +301,9 @@ pub enum Config {
/// See `crate::authres::update_authservid_candidates`.
AuthservIdCandidates,
/// Make all outgoing messages with Autocrypt header "multipart/signed".
SignUnencrypted,
/// Let the core save all events to the database.
/// This value is used internally to remember the MsgId of the logging xdc
#[strum(props(default = "0"))]
@@ -314,6 +318,11 @@ impl Context {
/// Get a configuration key. Returns `None` if no value is set, and no default value found.
pub async fn get_config(&self, key: Config) -> Result<Option<String>> {
let env_key = format!("DELTACHAT_{}", key.as_ref().to_uppercase());
if let Ok(value) = env::var(env_key) {
return Ok(Some(value));
}
let value = match key {
Config::Selfavatar => {
let rel_path = self.sql.get_raw_config(key.as_ref()).await?;
@@ -415,7 +424,7 @@ impl Context {
match key {
Config::Selfavatar => {
self.sql
.execute("UPDATE contacts SET selfavatar_sent=0;", paramsv![])
.execute("UPDATE contacts SET selfavatar_sent=0;", ())
.await?;
match value {
Some(value) => {
@@ -434,7 +443,7 @@ impl Context {
Config::DeleteDeviceAfter => {
let ret = self.sql.set_raw_config(key.as_ref(), value).await;
// Interrupt ephemeral loop to delete old messages immediately.
self.interrupt_ephemeral_task().await;
self.scheduler.interrupt_ephemeral_task().await;
ret?
}
Config::Displayname => {

View File

@@ -59,7 +59,7 @@ impl Context {
/// Configures this account with the currently set parameters.
pub async fn configure(&self) -> Result<()> {
ensure!(
self.scheduler.read().await.is_none(),
!self.scheduler.is_running().await,
"cannot configure, already running"
);
ensure!(
@@ -469,7 +469,9 @@ async fn configure(ctx: &Context, param: &mut LoginParam) -> Result<()> {
ctx.set_config_bool(Config::FetchedExistingMsgs, false)
.await?;
ctx.interrupt_inbox(InterruptInfo::new(false)).await;
ctx.scheduler
.interrupt_inbox(InterruptInfo::new(false))
.await;
progress!(ctx, 940);
update_device_chats_handle.await??;

View File

@@ -4,7 +4,7 @@ use crate::provider::{Protocol, Socket};
/// Set of variable parameters to try during configuration.
///
/// Can be loaded from offline provider database, online configuraiton
/// Can be loaded from offline provider database, online configuration
/// or derived from user entered parameters.
#[derive(Debug, Clone, PartialEq)]
pub(crate) struct ServerParams {

View File

@@ -103,7 +103,7 @@ pub(crate) const DC_RESEND_USER_AVATAR_DAYS: i64 = 14;
// warn about an outdated app after a given number of days.
// as we use the "provider-db generation date" as reference (that might not be updated very often)
// and as not all system get speedy updates,
// do not use too small value that will annoy users checking for nonexistant updates.
// do not use too small value that will annoy users checking for nonexistent updates.
pub(crate) const DC_OUTDATED_WARNING_DAYS: i64 = 365;
/// messages that should be deleted get this chat_id; the messages are deleted from the working thread later then. This is also needed as rfc724_mid should be preset as long as the message is not deleted on the server (otherwise it is downloaded again)

View File

@@ -727,7 +727,7 @@ impl Contact {
pub async fn add_address_book(context: &Context, addr_book: &str) -> Result<usize> {
let mut modify_cnt = 0;
for (name, addr) in split_address_book(addr_book).into_iter() {
for (name, addr) in split_address_book(addr_book) {
let (name, addr) = sanitize_name_and_addr(name, addr);
let name = normalize_name(&name);
match ContactAddress::new(&addr) {
@@ -1466,7 +1466,10 @@ pub(crate) async fn update_last_seen(
> 0
&& timestamp > time() - SEEN_RECENTLY_SECONDS
{
context.interrupt_recently_seen(contact_id, timestamp).await;
context
.scheduler
.interrupt_recently_seen(contact_id, timestamp)
.await;
}
Ok(())
}
@@ -1552,7 +1555,7 @@ impl RecentlySeenLoop {
pub(crate) fn new(context: Context) -> Self {
let (interrupt_send, interrupt_recv) = channel::bounded(1);
let handle = task::spawn(async move { Self::run(context, interrupt_recv).await });
let handle = task::spawn(Self::run(context, interrupt_recv));
Self {
handle,
interrupt_send,

View File

@@ -4,6 +4,7 @@ use std::collections::{BTreeMap, HashMap};
use std::ffi::OsString;
use std::ops::Deref;
use std::path::{Path, PathBuf};
use std::sync::atomic::AtomicBool;
use std::sync::Arc;
use std::time::{Duration, Instant, SystemTime};
@@ -23,9 +24,10 @@ use crate::key::{DcKey, SignedPublicKey};
use crate::login_param::LoginParam;
use crate::message::{self, MessageState, MsgId};
use crate::quota::QuotaInfo;
use crate::scheduler::Scheduler;
use crate::scheduler::SchedulerState;
use crate::sql::Sql;
use crate::stock_str::StockStrings;
use crate::timesmearing::SmearedTimestamp;
use crate::tools::{duration_to_str, time};
/// Builder for the [`Context`].
@@ -112,7 +114,7 @@ impl ContextBuilder {
/// Sets the event channel for this [`Context`].
///
/// Mostly useful when using multiple [`Context`]s, this allows creating one [`Events`]
/// channel and passing it to all [`Context`]s so all events are recieved on the same
/// channel and passing it to all [`Context`]s so all events are received on the same
/// channel.
///
/// Note that the [account manager](crate::accounts::Accounts) is designed to handle the
@@ -188,24 +190,34 @@ pub struct InnerContext {
/// Blob directory path
pub(crate) blobdir: PathBuf,
pub(crate) sql: Sql,
pub(crate) last_smeared_timestamp: RwLock<i64>,
pub(crate) smeared_timestamp: SmearedTimestamp,
/// The global "ongoing" process state.
///
/// This is a global mutex-like state for operations which should be modal in the
/// clients.
running_state: RwLock<RunningState>,
/// Mutex to avoid generating the key for the user more than once.
pub(crate) generating_key_mutex: Mutex<()>,
/// Mutex to enforce only a single running oauth2 is running.
pub(crate) oauth2_mutex: Mutex<()>,
/// Mutex to prevent a race condition when a "your pw is wrong" warning is sent, resulting in multiple messeges being sent.
/// Mutex to prevent a race condition when a "your pw is wrong" warning is sent, resulting in multiple messages being sent.
pub(crate) wrong_pw_warning_mutex: Mutex<()>,
pub(crate) translated_stockstrings: StockStrings,
pub(crate) events: Events,
pub(crate) scheduler: RwLock<Option<Scheduler>>,
pub(crate) scheduler: SchedulerState,
pub(crate) ratelimit: RwLock<Ratelimit>,
/// Recently loaded quota information, if any.
/// Set to `None` if quota was never tried to load.
pub(crate) quota: RwLock<Option<QuotaInfo>>,
/// Set to true if quota update is requested.
pub(crate) quota_update_request: AtomicBool,
/// IMAP UID resync request.
pub(crate) resync_request: AtomicBool,
/// Server ID response if ID capability is supported
/// and the server returned non-NIL on the inbox connection.
/// <https://datatracker.ietf.org/doc/html/rfc2971>
@@ -226,7 +238,7 @@ pub struct InnerContext {
/// `last_error` should be used to avoid races with the event thread.
pub(crate) last_error: std::sync::RwLock<String>,
/// If debug logging is enabled, this contains all neccesary information
/// If debug logging is enabled, this contains all necessary information
pub(crate) debug_logging: RwLock<Option<DebugLogging>>,
}
@@ -234,7 +246,7 @@ pub struct InnerContext {
pub(crate) struct DebugLogging {
/// The message containing the logging xdc
pub(crate) msg_id: MsgId,
/// Handle to the background task responisble for sending
/// Handle to the background task responsible for sending
pub(crate) loop_handle: task::JoinHandle<()>,
/// Channel that log events should be send to
/// A background loop will receive and handle them
@@ -356,15 +368,17 @@ impl Context {
blobdir,
running_state: RwLock::new(Default::default()),
sql: Sql::new(dbfile),
last_smeared_timestamp: RwLock::new(0),
smeared_timestamp: SmearedTimestamp::new(),
generating_key_mutex: Mutex::new(()),
oauth2_mutex: Mutex::new(()),
wrong_pw_warning_mutex: Mutex::new(()),
translated_stockstrings: stockstrings,
events,
scheduler: RwLock::new(None),
scheduler: SchedulerState::new(),
ratelimit: RwLock::new(Ratelimit::new(Duration::new(60, 0), 6.0)), // Allow to send 6 messages immediately, no more than once every 10 seconds.
quota: RwLock::new(None),
quota_update_request: AtomicBool::new(false),
resync_request: AtomicBool::new(false),
server_id: RwLock::new(None),
creation_time: std::time::SystemTime::now(),
last_full_folder_scan: Mutex::new(None),
@@ -385,42 +399,23 @@ impl Context {
warn!(self, "can not start io on a context that is not configured");
return;
}
info!(self, "starting IO");
let mut lock = self.inner.scheduler.write().await;
if lock.is_none() {
match Scheduler::start(self.clone()).await {
Err(err) => error!(self, "Failed to start IO: {:#}", err),
Ok(scheduler) => *lock = Some(scheduler),
}
}
self.scheduler.start(self.clone()).await;
}
/// Stops the IO scheduler.
pub async fn stop_io(&self) {
// Sending an event wakes up event pollers (get_next_event)
// so the caller of stop_io() can arrange for proper termination.
// For this, the caller needs to instruct the event poller
// to terminate on receiving the next event and then call stop_io()
// which will emit the below event(s)
info!(self, "stopping IO");
if let Some(debug_logging) = self.debug_logging.read().await.as_ref() {
debug_logging.loop_handle.abort();
}
if let Some(scheduler) = self.inner.scheduler.write().await.take() {
scheduler.stop(self).await;
}
self.scheduler.stop(self).await;
}
/// Restarts the IO scheduler if it was running before
/// when it is not running this is an no-op
pub async fn restart_io_if_running(&self) {
info!(self, "restarting IO");
let is_running = { self.inner.scheduler.read().await.is_some() };
if is_running {
self.stop_io().await;
self.start_io().await;
}
self.scheduler.restart(self).await;
}
/// Indicate that the network likely has come back.
pub async fn maybe_network(&self) {
self.scheduler.maybe_network().await;
}
/// Returns a reference to the underlying SQL instance.
@@ -511,6 +506,13 @@ impl Context {
// Ongoing process allocation/free/check
/// Tries to acquire the global UI "ongoing" mutex.
///
/// This is for modal operations during which no other user actions are allowed. Only
/// one such operation is allowed at any given time.
///
/// The return value is a cancel token, which will release the ongoing mutex when
/// dropped.
pub(crate) async fn alloc_ongoing(&self) -> Result<Receiver<()>> {
let mut s = self.running_state.write().await;
ensure!(
@@ -580,7 +582,7 @@ impl Context {
.unwrap_or_default();
let journal_mode = self
.sql
.query_get_value("PRAGMA journal_mode;", paramsv![])
.query_get_value("PRAGMA journal_mode;", ())
.await?
.unwrap_or_else(|| "unknown".to_string());
let e2ee_enabled = self.get_config_int(Config::E2eeEnabled).await?;
@@ -588,14 +590,11 @@ impl Context {
let bcc_self = self.get_config_int(Config::BccSelf).await?;
let send_sync_msgs = self.get_config_int(Config::SendSyncMsgs).await?;
let prv_key_cnt = self
.sql
.count("SELECT COUNT(*) FROM keypairs;", paramsv![])
.await?;
let prv_key_cnt = self.sql.count("SELECT COUNT(*) FROM keypairs;", ()).await?;
let pub_key_cnt = self
.sql
.count("SELECT COUNT(*) FROM acpeerstates;", paramsv![])
.count("SELECT COUNT(*) FROM acpeerstates;", ())
.await?;
let fingerprint_str = match SignedPublicKey::load_self(self).await {
Ok(key) => key.fingerprint().hex(),
@@ -757,6 +756,12 @@ impl Context {
.await?
.unwrap_or_default(),
);
res.insert(
"sign_unencrypted",
self.get_config_int(Config::SignUnencrypted)
.await?
.to_string(),
);
res.insert(
"debug_logging",

View File

@@ -13,7 +13,6 @@ use crate::imap::{Imap, ImapActionResult};
use crate::job::{self, Action, Job, Status};
use crate::message::{Message, MsgId, Viewtype};
use crate::mimeparser::{MimeMessage, Part};
use crate::param::Params;
use crate::tools::time;
use crate::{job_try, stock_str, EventType};
@@ -23,7 +22,7 @@ use crate::{job_try, stock_str, EventType};
/// need to be downloaded completely to handle them correctly,
/// eg. to assign them to the correct chat.
/// As these messages are typically small,
/// they're catched by `MIN_DOWNLOAD_LIMIT`.
/// they're caught by `MIN_DOWNLOAD_LIMIT`.
const MIN_DOWNLOAD_LIMIT: u32 = 32768;
/// If a message is downloaded only partially
@@ -86,11 +85,7 @@ impl MsgId {
DownloadState::Available | DownloadState::Failure => {
self.update_download_state(context, DownloadState::InProgress)
.await?;
job::add(
context,
Job::new(Action::DownloadMsg, self.to_u32(), Params::new(), 0),
)
.await?;
job::add(context, Job::new(Action::DownloadMsg, self.to_u32())).await?;
}
}
Ok(())

View File

@@ -124,6 +124,19 @@ impl EncryptHelper {
Ok(ctext)
}
/// Signs the passed-in `mail` using the private key from `context`.
/// Returns the payload and the signature.
pub async fn sign(
self,
context: &Context,
mail: lettre_email::PartBuilder,
) -> Result<(lettre_email::MimeMessage, String)> {
let sign_key = SignedSecretKey::load_self(context).await?;
let mime_message = mail.build();
let signature = pgp::pk_calc_signature(mime_message.as_string().as_bytes(), &sign_key)?;
Ok((mime_message, signature))
}
}
/// Ensures a private key exists for the configured user.

View File

@@ -68,7 +68,7 @@ use std::num::ParseIntError;
use std::str::FromStr;
use std::time::{Duration, SystemTime, UNIX_EPOCH};
use anyhow::{ensure, Context as _, Result};
use anyhow::{ensure, Result};
use async_channel::Receiver;
use serde::{Deserialize, Serialize};
use tokio::time::timeout;
@@ -317,7 +317,7 @@ impl MsgId {
paramsv![ephemeral_timestamp, ephemeral_timestamp, self],
)
.await?;
context.interrupt_ephemeral_task().await;
context.scheduler.interrupt_ephemeral_task().await;
}
Ok(())
}
@@ -345,7 +345,7 @@ pub(crate) async fn start_ephemeral_timers_msgids(
)
.await?;
if count > 0 {
context.interrupt_ephemeral_task().await;
context.scheduler.interrupt_ephemeral_task().await;
}
Ok(())
}
@@ -433,37 +433,40 @@ pub(crate) async fn delete_expired_messages(context: &Context, now: i64) -> Resu
let rows = select_expired_messages(context, now).await?;
if !rows.is_empty() {
context
info!(context, "Attempting to delete {} messages.", rows.len());
let (msgs_changed, webxdc_deleted) = context
.sql
.execute(
.transaction(|transaction| {
let mut msgs_changed = Vec::with_capacity(rows.len());
let mut webxdc_deleted = Vec::new();
// If you change which information is removed here, also change MsgId::trash() and
// which information receive_imf::add_parts() still adds to the db if the chat_id is TRASH
&format!(
r#"
UPDATE msgs
SET
chat_id=?, txt='', subject='', txt_raw='',
mime_headers='', from_id=0, to_id=0, param=''
WHERE id IN ({})
"#,
sql::repeat_vars(rows.len())
),
rusqlite::params_from_iter(
std::iter::once(&DC_CHAT_ID_TRASH as &dyn crate::ToSql).chain(
rows.iter()
.map(|(msg_id, _chat_id, _viewtype)| msg_id as &dyn crate::ToSql),
),
),
)
.await
.context("update failed")?;
for (msg_id, chat_id, viewtype) in rows {
transaction.execute(
"UPDATE msgs
SET chat_id=?, txt='', subject='', txt_raw='',
mime_headers='', from_id=0, to_id=0, param=''
WHERE id=?",
params![DC_CHAT_ID_TRASH, msg_id],
)?;
for (msg_id, chat_id, viewtype) in rows {
msgs_changed.push((chat_id, msg_id));
if viewtype == Viewtype::Webxdc {
webxdc_deleted.push(msg_id)
}
}
Ok((msgs_changed, webxdc_deleted))
})
.await?;
for (chat_id, msg_id) in msgs_changed {
context.emit_msgs_changed(chat_id, msg_id);
}
if viewtype == Viewtype::Webxdc {
context.emit_event(EventType::WebxdcInstanceDeleted { msg_id });
}
for msg_id in webxdc_deleted {
context.emit_event(EventType::WebxdcInstanceDeleted { msg_id });
}
}
@@ -650,7 +653,7 @@ mod tests {
use crate::download::DownloadState;
use crate::receive_imf::receive_imf;
use crate::test_utils::TestContext;
use crate::tools::MAX_SECONDS_TO_LEND_FROM_FUTURE;
use crate::timesmearing::MAX_SECONDS_TO_LEND_FROM_FUTURE;
use crate::{
chat::{self, create_group_chat, send_text_msg, Chat, ChatItem, ProtectionStatus},
tools::IsNoneOrEmpty,
@@ -1170,7 +1173,7 @@ mod tests {
// No other messages are marked for deletion.
assert_eq!(
t.sql
.count("SELECT COUNT(*) FROM imap WHERE target=''", paramsv![],)
.count("SELECT COUNT(*) FROM imap WHERE target=''", ())
.await?,
0
);
@@ -1184,10 +1187,7 @@ mod tests {
.update_download_state(&t, DownloadState::Available)
.await?;
t.sql
.execute(
"UPDATE imap SET target=folder WHERE rfc724_mid='1000'",
paramsv![],
)
.execute("UPDATE imap SET target=folder WHERE rfc724_mid='1000'", ())
.await?;
delete_expired_imap_messages(&t).await?;
test_marked_for_deletion(&t, 1000).await?; // Delete downloadable anyway.
@@ -1198,10 +1198,7 @@ mod tests {
delete_expired_imap_messages(&t).await?;
test_marked_for_deletion(&t, 1010).await?;
t.sql
.execute(
"UPDATE imap SET target=folder WHERE rfc724_mid='1010'",
paramsv![],
)
.execute("UPDATE imap SET target=folder WHERE rfc724_mid='1010'", ())
.await?;
MsgId::new(1010)
@@ -1211,7 +1208,7 @@ mod tests {
// Keep downloadable for now.
assert_eq!(
t.sql
.count("SELECT COUNT(*) FROM imap WHERE target=''", paramsv![],)
.count("SELECT COUNT(*) FROM imap WHERE target=''", ())
.await?,
0
);
@@ -1273,7 +1270,7 @@ mod tests {
// protection.
//
// Previously Delta Chat fallen back to using <first@example.com> in this case and
// compared received timer value to the timer value of the <first@examle.com>. Because
// compared received timer value to the timer value of the <first@example.com>. Because
// their timer values are the same ("disabled"), Delta Chat assumed that the timer was not
// changed explicitly and the change should be ignored.
//

View File

@@ -70,7 +70,7 @@ impl Events {
pub struct EventEmitter(Receiver<Event>);
impl EventEmitter {
/// Async recv of an event. Return `None` if the `Sender` has been droped.
/// Async recv of an event. Return `None` if the `Sender` has been dropped.
pub async fn recv(&self) -> Option<Event> {
self.0.recv().await.ok()
}
@@ -89,7 +89,7 @@ impl futures::stream::Stream for EventEmitter {
/// The event emitted by a [`Context`] from an [`EventEmitter`].
///
/// Events are documented on the C/FFI API in `deltachat.h` as `DC_EVENT_*` contants. The
/// Events are documented on the C/FFI API in `deltachat.h` as `DC_EVENT_*` constants. The
/// context emits them in relation to various operations happening, a lot of these are again
/// documented in `deltachat.h`.
///

View File

@@ -5,7 +5,7 @@
/// Fuzzing target for simplify().
///
/// Calls simplify() and panics if simplify() panics.
/// Does not return any vaule to avoid exposing internal crate types.
/// Does not return any value to avoid exposing internal crate types.
#[cfg(fuzzing)]
pub fn simplify(input: String, is_chat_message: bool) {
crate::simplify::simplify(input, is_chat_message);

View File

@@ -124,7 +124,7 @@ impl HtmlMsgParser {
async move {
match get_mime_multipart_type(&mail.ctype) {
MimeMultipartType::Multiple => {
for cur_data in mail.subparts.iter() {
for cur_data in &mail.subparts {
self.collect_texts_recursive(cur_data).await?
}
Ok(())
@@ -180,7 +180,7 @@ impl HtmlMsgParser {
async move {
match get_mime_multipart_type(&mail.ctype) {
MimeMultipartType::Multiple => {
for cur_data in mail.subparts.iter() {
for cur_data in &mail.subparts {
self.cid_to_data_recursive(context, cur_data).await?;
}
Ok(())
@@ -292,7 +292,10 @@ mod tests {
assert_eq!(
parser.html,
r##"<!DOCTYPE html>
<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8" /></head><body>
<html><head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="color-scheme" content="light dark" />
</head><body>
This message does not have Content-Type nor Subject.<br/>
<br/>
</body></html>
@@ -308,7 +311,10 @@ This message does not have Content-Type nor Subject.<br/>
assert_eq!(
parser.html,
r##"<!DOCTYPE html>
<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8" /></head><body>
<html><head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="color-scheme" content="light dark" />
</head><body>
message with a non-UTF-8 encoding: äöüßÄÖÜ<br/>
<br/>
</body></html>
@@ -325,7 +331,10 @@ message with a non-UTF-8 encoding: äöüßÄÖÜ<br/>
assert_eq!(
parser.html,
r##"<!DOCTYPE html>
<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8" /></head><body>
<html><head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="color-scheme" content="light dark" />
</head><body>
This line ends with a space and will be merged with the next one due to format=flowed.<br/>
<br/>
This line does not end with a space<br/>
@@ -344,7 +353,10 @@ and will be wrapped as usual.<br/>
assert_eq!(
parser.html,
r##"<!DOCTYPE html>
<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8" /></head><body>
<html><head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="color-scheme" content="light dark" />
</head><body>
mime-modified should not be set set as there is no html and no special stuff;<br/>
although not being a delta-message.<br/>
test some special html-characters as &lt; &gt; and &amp; but also &quot; and &#x27; :)<br/>

View File

@@ -475,7 +475,7 @@ impl Imap {
// Note that the `Config::DeleteDeviceAfter` timer starts as soon as the messages are
// fetched while the per-chat ephemeral timers start as soon as the messages are marked
// as noticed.
context.interrupt_ephemeral_task().await;
context.scheduler.interrupt_ephemeral_task().await;
}
let session = self
@@ -904,6 +904,24 @@ impl Imap {
info!(context, "Done fetching existing messages.");
Ok(())
}
/// Synchronizes UIDs for all folders.
pub(crate) async fn resync_folders(&mut self, context: &Context) -> Result<()> {
self.prepare(context).await?;
let all_folders = self
.list_folders(context)
.await
.context("listing folders for resync")?;
for folder in all_folders {
let folder_meaning = get_folder_meaning(&folder);
if folder_meaning != FolderMeaning::Virtual {
self.resync_folder_uids(context, folder.name(), folder_meaning)
.await?;
}
}
Ok(())
}
}
impl Session {
@@ -1372,11 +1390,11 @@ impl Imap {
let mut uid_msgs = HashMap::with_capacity(request_uids.len());
let mut count = 0;
for &request_uid in request_uids.iter() {
for &request_uid in &request_uids {
// Check if FETCH response is already in `uid_msgs`.
let mut fetch_response = uid_msgs.remove(&request_uid);
// Try to find a requsted UID in returned FETCH responses.
// Try to find a requested UID in returned FETCH responses.
while fetch_response.is_none() {
let next_fetch_response =
if let Some(next_fetch_response) = fetch_responses.next().await {
@@ -2186,11 +2204,11 @@ async fn mark_seen_by_uid(
.with_context(|| format!("failed to start ephemeral timer for message {msg_id}"))?;
Ok(Some(chat_id))
} else {
// Message state has not chnaged.
// Message state has not changed.
Ok(None)
}
} else {
// There is no message is `msgs` table matchng the given UID.
// There is no message is `msgs` table matching the given UID.
Ok(None)
}
}
@@ -2206,7 +2224,10 @@ pub(crate) async fn markseen_on_imap_table(context: &Context, message_id: &str)
paramsv![message_id],
)
.await?;
context.interrupt_inbox(InterruptInfo::new(false)).await;
context
.scheduler
.interrupt_inbox(InterruptInfo::new(false))
.await;
Ok(())
}

View File

@@ -5,18 +5,19 @@ use std::ffi::OsStr;
use std::path::{Path, PathBuf};
use ::pgp::types::KeyTrait;
use anyhow::{bail, ensure, format_err, Context as _, Error, Result};
use anyhow::{bail, ensure, format_err, Context as _, Result};
use futures::StreamExt;
use futures_lite::FutureExt;
use rand::{thread_rng, Rng};
use tokio::fs::{self, File};
use tokio_tar::Archive;
use crate::blob::BlobObject;
use crate::blob::{BlobDirContents, BlobObject};
use crate::chat::{self, delete_and_reset_all_device_msgs, ChatId};
use crate::config::Config;
use crate::contact::ContactId;
use crate::context::Context;
use crate::e2ee;
use crate::events::EventType;
use crate::key::{self, DcKey, DcSecretKey, SignedPublicKey, SignedSecretKey};
use crate::log::LogExt;
@@ -30,11 +31,14 @@ use crate::tools::{
create_folder, delete_file, get_filesuffix_lc, open_file_std, read_file, time, write_file,
EmailAddress,
};
use crate::{e2ee, tools};
mod transfer;
pub use transfer::{get_backup, BackupProvider};
// Name of the database file in the backup.
const DBFILE_BACKUP_NAME: &str = "dc_database_backup.sqlite";
const BLOBS_BACKUP_NAME: &str = "blobs_backup";
pub(crate) const BLOBS_BACKUP_NAME: &str = "blobs_backup";
/// Import/export command.
#[derive(Debug, Display, Copy, Clone, PartialEq, Eq, FromPrimitive, ToPrimitive)]
@@ -86,13 +90,17 @@ pub async fn imex(
) -> Result<()> {
let cancel = context.alloc_ongoing().await?;
let res = imex_inner(context, what, path, passphrase)
.race(async {
cancel.recv().await.ok();
Err(format_err!("canceled"))
})
.await;
let res = {
let mut guard = context.scheduler.pause(context.clone()).await;
let res = imex_inner(context, what, path, passphrase)
.race(async {
cancel.recv().await.ok();
Err(format_err!("canceled"))
})
.await;
guard.resume().await;
res
};
context.free_ongoing().await;
if let Err(err) = res.as_ref() {
@@ -249,8 +257,8 @@ async fn maybe_add_bcc_self_device_msg(context: &Context) -> Result<()> {
// TODO: define this as a stockstring once the wording is settled.
msg.text = Some(
"It seems you are using multiple devices with Delta Chat. Great!\n\n\
If you also want to synchronize outgoing messages accross all devices, \
go to the settings and enable \"Send copy to self\"."
If you also want to synchronize outgoing messages across all devices, \
go to \"Settings → Advanced\" and enable \"Send Copy to Self\"."
.to_string(),
);
chat::add_device_msg(context, Some("bcc-self-hint"), Some(&mut msg)).await?;
@@ -413,7 +421,7 @@ async fn import_backup(
"Cannot import backups to accounts in use."
);
ensure!(
context.scheduler.read().await.is_none(),
!context.scheduler.is_running().await,
"cannot import backup, IO is running"
);
@@ -516,16 +524,9 @@ async fn export_backup(context: &Context, dir: &Path, passphrase: String) -> Res
let _d1 = DeleteOnDrop(temp_db_path.clone());
let _d2 = DeleteOnDrop(temp_path.clone());
context
.sql
.set_raw_config_int("backup_time", now as i32)
.await?;
sql::housekeeping(context).await.ok_or_log(context);
ensure!(
context.scheduler.read().await.is_none(),
"cannot export backup, IO is running"
);
export_database(context, &temp_db_path, passphrase)
.await
.context("could not export database")?;
info!(
context,
@@ -534,32 +535,6 @@ async fn export_backup(context: &Context, dir: &Path, passphrase: String) -> Res
dest_path.display(),
);
let path_str = temp_db_path
.to_str()
.with_context(|| format!("path {temp_db_path:?} is not valid unicode"))?;
context
.sql
.call(|conn| {
if let Err(err) = conn.execute("VACUUM", params![]) {
info!(context, "Vacuum failed, exporting anyway: {:#}.", err);
}
conn.execute(
"ATTACH DATABASE ? AS backup KEY ?",
paramsv![path_str, passphrase],
)
.context("failed to attach backup database")?;
let res = conn
.query_row("SELECT sqlcipher_export('backup')", [], |_row| Ok(()))
.context("failed to export to attached backup database");
conn.execute("DETACH DATABASE backup", [])
.context("failed to detach backup database")?;
res?;
Ok::<_, Error>(())
})
.await?;
let res = export_backup_inner(context, &temp_db_path, &temp_path).await;
match &res {
@@ -597,29 +572,15 @@ async fn export_backup_inner(
.append_path_with_name(temp_db_path, DBFILE_BACKUP_NAME)
.await?;
let read_dir = tools::read_dir(context.get_blobdir()).await?;
let count = read_dir.len();
let mut written_files = 0;
let blobdir = BlobDirContents::new(context).await?;
let mut last_progress = 0;
for entry in read_dir.into_iter() {
let name = entry.file_name();
if !entry.file_type().await?.is_file() {
warn!(
context,
"Export: Found dir entry {} that is not a file, ignoring",
name.to_string_lossy()
);
continue;
}
let mut file = File::open(entry.path()).await?;
let path_in_archive = PathBuf::from(BLOBS_BACKUP_NAME).join(name);
builder.append_file(path_in_archive, &mut file).await?;
written_files += 1;
let progress = 1000 * written_files / count;
for (i, blob) in blobdir.iter().enumerate() {
let mut file = File::open(blob.to_abs_path()).await?;
let path_in_archive = PathBuf::from(BLOBS_BACKUP_NAME).join(blob.as_name());
builder.append_file(path_in_archive, &mut file).await?;
let progress = 1000 * i / blobdir.len();
if progress != last_progress && progress > 10 && progress < 1000 {
// We already emitted ImexProgress(10) above
context.emit_event(EventType::ImexProgress(progress));
last_progress = progress;
}
@@ -697,7 +658,7 @@ async fn export_self_keys(context: &Context, dir: &Path) -> Result<()> {
.sql
.query_map(
"SELECT id, public_key, private_key, is_default FROM keypairs;",
paramsv![],
(),
|row| {
let id = row.get(0)?;
let public_key_blob: Vec<u8> = row.get(1)?;
@@ -781,6 +742,48 @@ where
Ok(())
}
/// Exports the database to *dest*, encrypted using *passphrase*.
///
/// The directory of *dest* must already exist, if *dest* itself exists it will be
/// overwritten.
///
/// This also verifies that IO is not running during the export.
async fn export_database(context: &Context, dest: &Path, passphrase: String) -> Result<()> {
ensure!(
!context.scheduler.is_running().await,
"cannot export backup, IO is running"
);
let now = time().try_into().context("32-bit UNIX time overflow")?;
// TODO: Maybe introduce camino crate for UTF-8 paths where we need them.
let dest = dest
.to_str()
.with_context(|| format!("path {} is not valid unicode", dest.display()))?;
context.sql.set_raw_config_int("backup_time", now).await?;
sql::housekeeping(context).await.ok_or_log(context);
context
.sql
.call_write(|conn| {
conn.execute("VACUUM;", params![])
.map_err(|err| warn!(context, "Vacuum failed, exporting anyway {err}"))
.ok();
conn.execute(
"ATTACH DATABASE ? AS backup KEY ?",
paramsv![dest, passphrase],
)
.context("failed to attach backup database")?;
let res = conn
.query_row("SELECT sqlcipher_export('backup')", [], |_row| Ok(()))
.context("failed to export to attached backup database");
conn.execute("DETACH DATABASE backup", [])
.context("failed to detach backup database")?;
res?;
Ok(())
})
.await
}
#[cfg(test)]
mod tests {
use std::time::Duration;
@@ -800,7 +803,7 @@ mod tests {
println!("{}", &msg);
// Check some substrings, indicating things got substituted.
// In particular note the mixing of `\r\n` and `\n` depending
// on who generated the stings.
// on who generated the strings.
assert!(msg.contains("<title>Autocrypt Setup Message</title"));
assert!(msg.contains("<h1>Autocrypt Setup Message</h1>"));
assert!(msg.contains("<p>This is the Autocrypt Setup Message used to"));

694
src/imex/transfer.rs Normal file
View File

@@ -0,0 +1,694 @@
//! Transfer a backup to an other device.
//!
//! This module provides support for using n0's iroh tool to initiate transfer of a backup
//! to another device using a QR code.
//!
//! Using the iroh terminology there are two parties to this:
//!
//! - The *Provider*, which starts a server and listens for connections.
//! - The *Getter*, which connects to the server and retrieves the data.
//!
//! Iroh is designed around the idea of verifying hashes, the downloads are verified as
//! they are retrieved. The entire transfer is initiated by requesting the data of a single
//! root hash.
//!
//! Both the provider and the getter are authenticated:
//!
//! - The provider is known by its *peer ID*.
//! - The provider needs an *authentication token* from the getter before it accepts a
//! connection.
//!
//! Both these are transferred in the QR code offered to the getter. This ensures that the
//! getter can not connect to an impersonated provider and the provider does not offer the
//! download to an impersonated getter.
use std::future::Future;
use std::net::Ipv4Addr;
use std::ops::Deref;
use std::path::{Path, PathBuf};
use std::pin::Pin;
use std::task::Poll;
use anyhow::{anyhow, bail, ensure, format_err, Context as _, Result};
use async_channel::Receiver;
use futures_lite::StreamExt;
use iroh::get::{DataStream, Options};
use iroh::progress::ProgressEmitter;
use iroh::protocol::AuthToken;
use iroh::provider::{DataSource, Event, Provider, Ticket};
use iroh::Hash;
use tokio::fs::{self, File};
use tokio::io::{self, AsyncWriteExt, BufWriter};
use tokio::sync::broadcast::error::RecvError;
use tokio::sync::{broadcast, Mutex};
use tokio::task::{JoinHandle, JoinSet};
use tokio_stream::wrappers::ReadDirStream;
use crate::blob::BlobDirContents;
use crate::chat::delete_and_reset_all_device_msgs;
use crate::context::Context;
use crate::qr::Qr;
use crate::{e2ee, EventType};
use super::{export_database, DBFILE_BACKUP_NAME};
/// Provide or send a backup of this device.
///
/// This creates a backup of the current device and starts a service which offers another
/// device to download this backup.
///
/// This does not make a full backup on disk, only the SQLite database is created on disk,
/// the blobs in the blob directory are not copied.
///
/// This starts a task which acquires the global "ongoing" mutex. If you need to stop the
/// task use the [`Context::stop_ongoing`] mechanism.
///
/// The task implements [`Future`] and awaiting it will complete once a transfer has been
/// either completed or aborted.
#[derive(Debug)]
pub struct BackupProvider {
/// The supervisor task, run by [`BackupProvider::watch_provider`].
handle: JoinHandle<Result<()>>,
/// The ticket to retrieve the backup collection.
ticket: Ticket,
}
impl BackupProvider {
/// Prepares for sending a backup to a second device.
///
/// Before calling this function all I/O must be stopped so that no changes to the blobs
/// or database are happening, this is done by calling the [`Accounts::stop_io`] or
/// [`Context::stop_io`] APIs first.
///
/// This will acquire the global "ongoing process" mutex, which can be used to cancel
/// the process.
///
/// [`Accounts::stop_io`]: crate::accounts::Accounts::stop_io
pub async fn prepare(context: &Context) -> Result<Self> {
e2ee::ensure_secret_key_exists(context)
.await
.context("Private key not available, aborting backup export")?;
// Acquire global "ongoing" mutex.
let cancel_token = context.alloc_ongoing().await?;
let mut paused_guard = context.scheduler.pause(context.clone()).await;
let context_dir = context
.get_blobdir()
.parent()
.ok_or(anyhow!("Context dir not found"))?;
let dbfile = context_dir.join(DBFILE_BACKUP_NAME);
if fs::metadata(&dbfile).await.is_ok() {
fs::remove_file(&dbfile).await?;
warn!(context, "Previous database export deleted");
}
let dbfile = TempPathGuard::new(dbfile);
let res = tokio::select! {
biased;
res = Self::prepare_inner(context, &dbfile) => {
match res {
Ok(slf) => Ok(slf),
Err(err) => {
error!(context, "Failed to set up second device setup: {:#}", err);
Err(err)
},
}
},
_ = cancel_token.recv() => Err(format_err!("cancelled")),
};
let (provider, ticket) = match res {
Ok((provider, ticket)) => (provider, ticket),
Err(err) => {
context.free_ongoing().await;
paused_guard.resume().await;
return Err(err);
}
};
let handle = {
let context = context.clone();
tokio::spawn(async move {
let res = Self::watch_provider(&context, provider, cancel_token).await;
context.free_ongoing().await;
paused_guard.resume().await;
drop(dbfile);
res
})
};
Ok(Self { handle, ticket })
}
/// Creates the provider task.
///
/// Having this as a function makes it easier to cancel it when needed.
async fn prepare_inner(context: &Context, dbfile: &Path) -> Result<(Provider, Ticket)> {
// Generate the token up front: we also use it to encrypt the database.
let token = AuthToken::generate();
context.emit_event(SendProgress::Started.into());
export_database(context, dbfile, token.to_string())
.await
.context("Database export failed")?;
context.emit_event(SendProgress::DatabaseExported.into());
// Now we can be sure IO is not running.
let mut files = vec![DataSource::with_name(
dbfile.to_owned(),
format!("db/{DBFILE_BACKUP_NAME}"),
)];
let blobdir = BlobDirContents::new(context).await?;
for blob in blobdir.iter() {
let path = blob.to_abs_path();
let name = format!("blob/{}", blob.as_file_name());
files.push(DataSource::with_name(path, name));
}
// Start listening.
let (db, hash) = iroh::provider::create_collection(files).await?;
context.emit_event(SendProgress::CollectionCreated.into());
let provider = Provider::builder(db)
.bind_addr((Ipv4Addr::UNSPECIFIED, 0).into())
.auth_token(token)
.spawn()?;
context.emit_event(SendProgress::ProviderListening.into());
info!(context, "Waiting for remote to connect");
let ticket = provider.ticket(hash);
Ok((provider, ticket))
}
/// Supervises the iroh [`Provider`], terminating it when needed.
///
/// This will watch the provider and terminate it when:
///
/// - A transfer is completed, successful or unsuccessful.
/// - An event could not be observed to protect against not knowing of a completed event.
/// - The ongoing process is cancelled.
///
/// The *cancel_token* is the handle for the ongoing process mutex, when this completes
/// we must cancel this operation.
async fn watch_provider(
context: &Context,
mut provider: Provider,
cancel_token: Receiver<()>,
) -> Result<()> {
// _dbfile exists so we can clean up the file once it is no longer needed
let mut events = provider.subscribe();
let mut total_size = 0;
let mut current_size = 0;
let res = loop {
tokio::select! {
biased;
res = &mut provider => {
break res.context("BackupProvider failed");
},
maybe_event = events.recv() => {
match maybe_event {
Ok(event) => {
match event {
Event::ClientConnected { ..} => {
context.emit_event(SendProgress::ClientConnected.into());
}
Event::RequestReceived { .. } => {
}
Event::TransferCollectionStarted { total_blobs_size, .. } => {
total_size = total_blobs_size;
context.emit_event(SendProgress::TransferInProgress {
current_size,
total_size,
}.into());
}
Event::TransferBlobCompleted { size, .. } => {
current_size += size;
context.emit_event(SendProgress::TransferInProgress {
current_size,
total_size,
}.into());
}
Event::TransferCollectionCompleted { .. } => {
context.emit_event(SendProgress::TransferInProgress {
current_size: total_size,
total_size
}.into());
provider.shutdown();
}
Event::TransferAborted { .. } => {
provider.shutdown();
break Err(anyhow!("BackupProvider transfer aborted"));
}
}
}
Err(broadcast::error::RecvError::Closed) => {
// We should never see this, provider.join() should complete
// first.
}
Err(broadcast::error::RecvError::Lagged(_)) => {
// We really shouldn't be lagging, if we did we may have missed
// a completion event.
provider.shutdown();
break Err(anyhow!("Missed events from BackupProvider"));
}
}
},
_ = cancel_token.recv() => {
provider.shutdown();
break Err(anyhow!("BackupSender cancelled"));
},
}
};
match &res {
Ok(_) => context.emit_event(SendProgress::Completed.into()),
Err(err) => {
error!(context, "Backup transfer failure: {err:#}");
context.emit_event(SendProgress::Failed.into())
}
}
res
}
/// Returns a QR code that allows fetching this backup.
///
/// This QR code can be passed to [`get_backup`] on a (different) device.
pub fn qr(&self) -> Qr {
Qr::Backup {
ticket: self.ticket.clone(),
}
}
}
impl Future for BackupProvider {
type Output = Result<()>;
fn poll(mut self: Pin<&mut Self>, cx: &mut std::task::Context<'_>) -> Poll<Self::Output> {
Pin::new(&mut self.handle).poll(cx)?
}
}
/// A guard which will remove the path when dropped.
///
/// It implements [`Deref`] it it can be used as a `&Path`.
#[derive(Debug)]
struct TempPathGuard {
path: PathBuf,
}
impl TempPathGuard {
fn new(path: PathBuf) -> Self {
Self { path }
}
}
impl Drop for TempPathGuard {
fn drop(&mut self) {
let path = self.path.clone();
tokio::spawn(async move {
fs::remove_file(&path).await.ok();
});
}
}
impl Deref for TempPathGuard {
type Target = Path;
fn deref(&self) -> &Self::Target {
&self.path
}
}
/// Create [`EventType::ImexProgress`] events using readable names.
///
/// Plus you get warnings if you don't use all variants.
#[derive(Debug)]
enum SendProgress {
Failed,
Started,
DatabaseExported,
CollectionCreated,
ProviderListening,
ClientConnected,
TransferInProgress { current_size: u64, total_size: u64 },
Completed,
}
impl From<SendProgress> for EventType {
fn from(source: SendProgress) -> Self {
use SendProgress::*;
let num: u16 = match source {
Failed => 0,
Started => 100,
DatabaseExported => 300,
CollectionCreated => 350,
ProviderListening => 400,
ClientConnected => 450,
TransferInProgress {
current_size,
total_size,
} => {
// the range is 450..=950
450 + ((current_size as f64 / total_size as f64) * 500.).floor() as u16
}
Completed => 1000,
};
Self::ImexProgress(num.into())
}
}
/// Contacts a backup provider and receives the backup from it.
///
/// This uses a QR code to contact another instance of deltachat which is providing a backup
/// using the [`BackupProvider`]. Once connected it will authenticate using the secrets in
/// the QR code and retrieve the backup.
///
/// This is a long running operation which will only when completed.
///
/// Using [`Qr`] as argument is a bit odd as it only accepts one specific variant of it. It
/// does avoid having [`iroh::provider::Ticket`] in the primary API however, without
/// having to revert to untyped bytes.
pub async fn get_backup(context: &Context, qr: Qr) -> Result<()> {
ensure!(
matches!(qr, Qr::Backup { .. }),
"QR code for backup must be of type DCBACKUP"
);
ensure!(
!context.is_configured().await?,
"Cannot import backups to accounts in use."
);
let mut guard = context.scheduler.pause(context.clone()).await;
// Acquire global "ongoing" mutex.
let cancel_token = context.alloc_ongoing().await?;
let res = tokio::select! {
biased;
res = get_backup_inner(context, qr) => {
context.free_ongoing().await;
res
}
_ = cancel_token.recv() => Err(format_err!("cancelled")),
};
guard.resume().await;
res
}
async fn get_backup_inner(context: &Context, qr: Qr) -> Result<()> {
let ticket = match qr {
Qr::Backup { ticket } => ticket,
_ => bail!("QR code for backup must be of type DCBACKUP"),
};
if ticket.addrs.is_empty() {
bail!("ticket is missing addresses to dial");
}
for addr in &ticket.addrs {
let opts = Options {
addr: *addr,
peer_id: Some(ticket.peer),
keylog: false,
};
info!(context, "attempting to contact {}", addr);
match transfer_from_provider(context, &ticket, opts).await {
Ok(_) => {
delete_and_reset_all_device_msgs(context).await?;
context.emit_event(ReceiveProgress::Completed.into());
return Ok(());
}
Err(TransferError::ConnectionError(err)) => {
warn!(context, "Connection error: {err:#}.");
continue;
}
Err(TransferError::Other(err)) => {
// Clean up any blobs we already wrote.
let readdir = fs::read_dir(context.get_blobdir()).await?;
let mut readdir = ReadDirStream::new(readdir);
while let Some(dirent) = readdir.next().await {
if let Ok(dirent) = dirent {
fs::remove_file(dirent.path()).await.ok();
}
}
context.emit_event(ReceiveProgress::Failed.into());
return Err(err);
}
}
}
Err(anyhow!("failed to contact provider"))
}
/// Error during a single transfer attempt.
///
/// Mostly exists to distinguish between `ConnectionError` and any other errors.
#[derive(Debug, thiserror::Error)]
enum TransferError {
#[error("connection error")]
ConnectionError(#[source] anyhow::Error),
#[error("other")]
Other(#[source] anyhow::Error),
}
async fn transfer_from_provider(
context: &Context,
ticket: &Ticket,
opts: Options,
) -> Result<(), TransferError> {
let progress = ProgressEmitter::new(0, ReceiveProgress::max_blob_progress());
spawn_progress_proxy(context.clone(), progress.subscribe());
let mut connected = false;
let on_connected = || {
context.emit_event(ReceiveProgress::Connected.into());
connected = true;
async { Ok(()) }
};
let jobs = Mutex::new(JoinSet::default());
let on_blob =
|hash, reader, name| on_blob(context, &progress, &jobs, ticket, hash, reader, name);
let res = iroh::get::run(
ticket.hash,
ticket.token,
opts,
on_connected,
|collection| {
context.emit_event(ReceiveProgress::CollectionReceived.into());
progress.set_total(collection.total_blobs_size());
async { Ok(()) }
},
on_blob,
)
.await;
let mut jobs = jobs.lock().await;
while let Some(job) = jobs.join_next().await {
job.context("job failed").map_err(TransferError::Other)?;
}
drop(progress);
match res {
Ok(stats) => {
info!(
context,
"Backup transfer finished, transfer rate is {} Mbps.",
stats.mbits()
);
Ok(())
}
Err(err) => match connected {
true => Err(TransferError::Other(err)),
false => Err(TransferError::ConnectionError(err)),
},
}
}
/// Get callback when a blob is received from the provider.
///
/// This writes the blobs to the blobdir. If the blob is the database it will import it to
/// the database of the current [`Context`].
async fn on_blob(
context: &Context,
progress: &ProgressEmitter,
jobs: &Mutex<JoinSet<()>>,
ticket: &Ticket,
_hash: Hash,
mut reader: DataStream,
name: String,
) -> Result<DataStream> {
ensure!(!name.is_empty(), "Received a nameless blob");
let path = if name.starts_with("db/") {
let context_dir = context
.get_blobdir()
.parent()
.ok_or(anyhow!("Context dir not found"))?;
let dbfile = context_dir.join(DBFILE_BACKUP_NAME);
if fs::metadata(&dbfile).await.is_ok() {
fs::remove_file(&dbfile).await?;
warn!(context, "Previous database export deleted");
}
dbfile
} else {
ensure!(name.starts_with("blob/"), "malformatted blob name");
let blobname = name.rsplit('/').next().context("malformatted blob name")?;
context.get_blobdir().join(blobname)
};
let mut wrapped_reader = progress.wrap_async_read(&mut reader);
let file = File::create(&path).await?;
let mut file = BufWriter::with_capacity(128 * 1024, file);
io::copy(&mut wrapped_reader, &mut file).await?;
file.flush().await?;
if name.starts_with("db/") {
let context = context.clone();
let token = ticket.token.to_string();
jobs.lock().await.spawn(async move {
if let Err(err) = context.sql.import(&path, token).await {
error!(context, "cannot import database: {:#?}", err);
}
if let Err(err) = fs::remove_file(&path).await {
error!(
context,
"failed to delete database import file '{}': {:#?}",
path.display(),
err,
);
}
});
}
Ok(reader)
}
/// Spawns a task proxying progress events.
///
/// This spawns a tokio task which receives events from the [`ProgressEmitter`] and sends
/// them to the context. The task finishes when the emitter is dropped.
///
/// This could be done directly in the emitter by making it less generic.
fn spawn_progress_proxy(context: Context, mut rx: broadcast::Receiver<u16>) {
tokio::spawn(async move {
loop {
match rx.recv().await {
Ok(step) => context.emit_event(ReceiveProgress::BlobProgress(step).into()),
Err(RecvError::Closed) => break,
Err(RecvError::Lagged(_)) => continue,
}
}
});
}
/// Create [`EventType::ImexProgress`] events using readable names.
///
/// Plus you get warnings if you don't use all variants.
#[derive(Debug)]
enum ReceiveProgress {
Connected,
CollectionReceived,
/// A value between 0 and 85 interpreted as a percentage.
///
/// Other values are already used by the other variants of this enum.
BlobProgress(u16),
Completed,
Failed,
}
impl ReceiveProgress {
/// The maximum value for [`ReceiveProgress::BlobProgress`].
///
/// This only exists to keep this magic value local in this type.
fn max_blob_progress() -> u16 {
85
}
}
impl From<ReceiveProgress> for EventType {
fn from(source: ReceiveProgress) -> Self {
let val = match source {
ReceiveProgress::Connected => 50,
ReceiveProgress::CollectionReceived => 100,
ReceiveProgress::BlobProgress(val) => 100 + 10 * val,
ReceiveProgress::Completed => 1000,
ReceiveProgress::Failed => 0,
};
EventType::ImexProgress(val.into())
}
}
#[cfg(test)]
mod tests {
use std::time::Duration;
use crate::chat::{get_chat_msgs, send_msg, ChatItem};
use crate::message::{Message, Viewtype};
use crate::test_utils::TestContextManager;
use super::*;
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_send_receive() {
let mut tcm = TestContextManager::new();
// Create first device.
let ctx0 = tcm.alice().await;
// Write a message in the self chat
let self_chat = ctx0.get_self_chat().await;
let mut msg = Message::new(Viewtype::Text);
msg.set_text(Some("hi there".to_string()));
send_msg(&ctx0, self_chat.id, &mut msg).await.unwrap();
// Send an attachment in the self chat
let file = ctx0.get_blobdir().join("hello.txt");
fs::write(&file, "i am attachment").await.unwrap();
let mut msg = Message::new(Viewtype::File);
msg.set_file(file.to_str().unwrap(), Some("text/plain"));
send_msg(&ctx0, self_chat.id, &mut msg).await.unwrap();
// Prepare to transfer backup.
let provider = BackupProvider::prepare(&ctx0).await.unwrap();
// Set up second device.
let ctx1 = tcm.unconfigured().await;
get_backup(&ctx1, provider.qr()).await.unwrap();
// Make sure the provider finishes without an error.
tokio::time::timeout(Duration::from_secs(30), provider)
.await
.expect("timed out")
.expect("error in provider");
// Check that we have the self message.
let self_chat = ctx1.get_self_chat().await;
let msgs = get_chat_msgs(&ctx1, self_chat.id).await.unwrap();
assert_eq!(msgs.len(), 2);
let msgid = match msgs.get(0).unwrap() {
ChatItem::Message { msg_id } => msg_id,
_ => panic!("wrong chat item"),
};
let msg = Message::load_from_db(&ctx1, *msgid).await.unwrap();
let text = msg.get_text().unwrap();
assert_eq!(text, "hi there");
let msgid = match msgs.get(1).unwrap() {
ChatItem::Message { msg_id } => msg_id,
_ => panic!("wrong chat item"),
};
let msg = Message::load_from_db(&ctx1, *msgid).await.unwrap();
let path = msg.get_file(&ctx1).unwrap();
let text = fs::read_to_string(&path).await.unwrap();
assert_eq!(text, "i am attachment");
// Check that both received the ImexProgress events.
ctx0.evtracker
.get_matching(|ev| matches!(ev, EventType::ImexProgress(1000)))
.await;
ctx1.evtracker
.get_matching(|ev| matches!(ev, EventType::ImexProgress(1000)))
.await;
}
#[test]
fn test_send_progress() {
let cases = [
((0, 100), 450),
((10, 100), 500),
((50, 100), 700),
((100, 100), 950),
];
for ((current_size, total_size), progress) in cases {
let out = EventType::from(SendProgress::TransferInProgress {
current_size,
total_size,
});
assert_eq!(out, EventType::ImexProgress(progress));
}
}
}

View File

@@ -6,14 +6,14 @@
#![allow(missing_docs)]
use std::fmt;
use std::sync::atomic::Ordering;
use anyhow::{Context as _, Result};
use deltachat_derive::{FromSql, ToSql};
use rand::{thread_rng, Rng};
use crate::context::Context;
use crate::imap::{get_folder_meaning, FolderMeaning, Imap};
use crate::param::Params;
use crate::imap::Imap;
use crate::scheduler::InterruptInfo;
use crate::tools::time;
@@ -25,7 +25,6 @@ const JOB_RETRIES: u32 = 17;
pub enum Status {
Finished(Result<()>),
RetryNow,
RetryLater,
}
#[macro_export]
@@ -58,18 +57,11 @@ macro_rules! job_try {
)]
#[repr(u32)]
pub enum Action {
// this is user initiated so it should have a fairly high priority
UpdateRecentQuota = 140,
// This job will download partially downloaded messages completely
// and is added when download_full() is called.
// Most messages are downloaded automatically on fetch
// and do not go through this job.
DownloadMsg = 250,
// UID synchronization is high-priority to make sure correct UIDs
// are used by message moving/deletion.
ResyncFolders = 300,
}
#[derive(Debug, Clone, PartialEq, Eq)]
@@ -80,7 +72,6 @@ pub struct Job {
pub desired_timestamp: i64,
pub added_timestamp: i64,
pub tries: u32,
pub param: Params,
}
impl fmt::Display for Job {
@@ -90,24 +81,19 @@ impl fmt::Display for Job {
}
impl Job {
pub fn new(action: Action, foreign_id: u32, param: Params, delay_seconds: i64) -> Self {
pub fn new(action: Action, foreign_id: u32) -> Self {
let timestamp = time();
Self {
job_id: 0,
action,
foreign_id,
desired_timestamp: timestamp + delay_seconds,
desired_timestamp: timestamp,
added_timestamp: timestamp,
tries: 0,
param,
}
}
pub fn delay_seconds(&self) -> i64 {
self.desired_timestamp - self.added_timestamp
}
/// Deletes the job from the database.
async fn delete(self, context: &Context) -> Result<()> {
if self.job_id != 0 {
@@ -130,23 +116,21 @@ impl Job {
context
.sql
.execute(
"UPDATE jobs SET desired_timestamp=?, tries=?, param=? WHERE id=?;",
"UPDATE jobs SET desired_timestamp=?, tries=? WHERE id=?;",
paramsv![
self.desired_timestamp,
i64::from(self.tries),
self.param.to_string(),
self.job_id as i32,
],
)
.await?;
} else {
context.sql.execute(
"INSERT INTO jobs (added_timestamp, action, foreign_id, param, desired_timestamp) VALUES (?,?,?,?,?);",
"INSERT INTO jobs (added_timestamp, action, foreign_id, desired_timestamp) VALUES (?,?,?,?);",
paramsv![
self.added_timestamp,
self.action,
self.foreign_id,
self.param.to_string(),
self.desired_timestamp
]
).await?;
@@ -154,63 +138,6 @@ impl Job {
Ok(())
}
/// Synchronizes UIDs for all folders.
async fn resync_folders(&mut self, context: &Context, imap: &mut Imap) -> Status {
if let Err(err) = imap.prepare(context).await {
warn!(context, "could not connect: {:#}", err);
return Status::RetryLater;
}
let all_folders = match imap.list_folders(context).await {
Ok(v) => v,
Err(e) => {
warn!(context, "Listing folders for resync failed: {:#}", e);
return Status::RetryLater;
}
};
let mut any_failed = false;
for folder in all_folders {
let folder_meaning = get_folder_meaning(&folder);
if folder_meaning == FolderMeaning::Virtual {
continue;
}
if let Err(e) = imap
.resync_folder_uids(context, folder.name(), folder_meaning)
.await
{
warn!(context, "{:#}", e);
any_failed = true;
}
}
if any_failed {
Status::RetryLater
} else {
Status::Finished(Ok(()))
}
}
}
/// Delete all pending jobs with the given action.
pub async fn kill_action(context: &Context, action: Action) -> Result<()> {
context
.sql
.execute("DELETE FROM jobs WHERE action=?;", paramsv![action])
.await?;
Ok(())
}
pub async fn action_exists(context: &Context, action: Action) -> Result<bool> {
let exists = context
.sql
.exists(
"SELECT COUNT(*) FROM jobs WHERE action=?;",
paramsv![action],
)
.await?;
Ok(exists)
}
pub(crate) enum Connection<'a> {
@@ -234,13 +161,13 @@ pub(crate) async fn perform_job(context: &Context, mut connection: Connection<'_
};
match try_res {
Status::RetryNow | Status::RetryLater => {
Status::RetryNow => {
let tries = job.tries + 1;
if tries < JOB_RETRIES {
info!(context, "increase job {} tries to {}", job, tries);
job.tries = tries;
let time_offset = get_backoff_time_offset(tries, job.action);
let time_offset = get_backoff_time_offset(tries);
job.desired_timestamp = time() + time_offset;
info!(
context,
@@ -288,11 +215,6 @@ async fn perform_job_action(
info!(context, "begin immediate try {} of job {}", tries, job);
let try_res = match job.action {
Action::ResyncFolders => job.resync_folders(context, connection.inbox()).await,
Action::UpdateRecentQuota => match context.update_recent_quota(connection.inbox()).await {
Ok(status) => status,
Err(err) => Status::Finished(Err(err)),
},
Action::DownloadMsg => job.download_msg(context, connection.inbox()).await,
};
@@ -301,50 +223,38 @@ async fn perform_job_action(
try_res
}
fn get_backoff_time_offset(tries: u32, action: Action) -> i64 {
match action {
// Just try every 10s to update the quota
// If all retries are exhausted, a new job will be created when the quota information is needed
Action::UpdateRecentQuota => 10,
_ => {
// Exponential backoff
let n = 2_i32.pow(tries - 1) * 60;
let mut rng = thread_rng();
let r: i32 = rng.gen();
let mut seconds = r % (n + 1);
if seconds < 1 {
seconds = 1;
}
i64::from(seconds)
}
fn get_backoff_time_offset(tries: u32) -> i64 {
// Exponential backoff
let n = 2_i32.pow(tries - 1) * 60;
let mut rng = thread_rng();
let r: i32 = rng.gen();
let mut seconds = r % (n + 1);
if seconds < 1 {
seconds = 1;
}
i64::from(seconds)
}
pub(crate) async fn schedule_resync(context: &Context) -> Result<()> {
kill_action(context, Action::ResyncFolders).await?;
add(
context,
Job::new(Action::ResyncFolders, 0, Params::new(), 0),
)
.await?;
context.resync_request.store(true, Ordering::Relaxed);
context
.scheduler
.interrupt_inbox(InterruptInfo {
probe_network: false,
})
.await;
Ok(())
}
/// Adds a job to the database, scheduling it.
pub async fn add(context: &Context, job: Job) -> Result<()> {
let action = job.action;
let delay_seconds = job.delay_seconds();
job.save(context).await.context("failed to save job")?;
if delay_seconds == 0 {
match action {
Action::ResyncFolders | Action::UpdateRecentQuota | Action::DownloadMsg => {
info!(context, "interrupt: imap");
context.interrupt_inbox(InterruptInfo::new(false)).await;
}
}
}
info!(context, "interrupt: imap");
context
.scheduler
.interrupt_inbox(InterruptInfo::new(false))
.await;
Ok(())
}
@@ -396,7 +306,6 @@ LIMIT 1;
desired_timestamp: row.get("desired_timestamp")?,
added_timestamp: row.get("added_timestamp")?,
tries: row.get("tries")?,
param: row.get::<_, String>("param")?.parse().unwrap_or_default(),
};
Ok(job)
@@ -436,8 +345,8 @@ mod tests {
.sql
.execute(
"INSERT INTO jobs
(added_timestamp, action, foreign_id, param, desired_timestamp)
VALUES (?, ?, ?, ?, ?);",
(added_timestamp, action, foreign_id, desired_timestamp)
VALUES (?, ?, ?, ?);",
paramsv![
now,
if valid {
@@ -446,7 +355,6 @@ mod tests {
-1
},
foreign_id,
Params::new().to_string(),
now
],
)

View File

@@ -148,7 +148,7 @@ impl DcKey for SignedSecretKey {
WHERE addr=(SELECT value FROM config WHERE keyname="configured_addr")
AND is_default=1;
"#,
paramsv![],
(),
|row| {
let bytes: Vec<u8> = row.get(0)?;
Ok(bytes)
@@ -302,7 +302,7 @@ pub async fn store_self_keypair(
.context("failed to remove old use of key")?;
if default == KeyPairUse::Default {
transaction
.execute("UPDATE keypairs SET is_default=0;", paramsv![])
.execute("UPDATE keypairs SET is_default=0;", ())
.context("failed to clear default")?;
}
let is_default = match default {
@@ -592,7 +592,7 @@ i8pcjGO+IZffvyZJVRWfVooBJmWWbPB1pueo3tx8w3+fcuzpxz+RLFKaPyqXO+dD
let nrows = || async {
ctx.sql
.count("SELECT COUNT(*) FROM keypairs;", paramsv![])
.count("SELECT COUNT(*) FROM keypairs;", ())
.await
.unwrap()
};

View File

@@ -12,7 +12,10 @@
clippy::wildcard_imports,
clippy::needless_borrow,
clippy::cast_lossless,
clippy::unused_async
clippy::unused_async,
clippy::explicit_iter_loop,
clippy::explicit_into_iter_loop,
clippy::cloned_instead_of_copied
)]
#![allow(
clippy::match_bool,
@@ -92,6 +95,7 @@ mod smtp;
mod socks;
pub mod stock_str;
mod sync;
mod timesmearing;
mod token;
mod update_helper;
pub mod webxdc;

View File

@@ -267,7 +267,7 @@ pub async fn send_locations_to_chat(
}
context.emit_event(EventType::ChatModified(chat_id));
if 0 != seconds {
context.interrupt_location().await;
context.scheduler.interrupt_location().await;
}
Ok(())
}
@@ -434,10 +434,7 @@ fn is_marker(txt: &str) -> bool {
/// Deletes all locations from the database.
pub async fn delete_all(context: &Context) -> Result<()> {
context
.sql
.execute("DELETE FROM locations;", paramsv![])
.await?;
context.sql.execute("DELETE FROM locations;", ()).await?;
context.emit_event(EventType::LocationChanged(None));
Ok(())
}
@@ -603,7 +600,7 @@ pub(crate) async fn save(
context
.sql
.call(|conn| {
.call_write(|conn| {
let mut stmt_test = conn
.prepare_cached("SELECT id FROM locations WHERE timestamp=? AND from_id=?")?;
let mut stmt_insert = conn.prepare_cached(stmt_insert)?;

View File

@@ -610,7 +610,7 @@ impl Message {
// It's a little unfortunate that the UI has to first call `dc_msg_get_override_sender_name` and then if it was `NULL`, call
// `dc_contact_get_display_name` but this was the best solution:
// - We could load a Contact struct from the db here to call `dc_get_display_name` instead of returning `None`, but then we had a db
// call everytime (and this fn is called a lot while the user is scrolling through a group), so performance would be bad
// call every time (and this fn is called a lot while the user is scrolling through a group), so performance would be bad
// - We could pass both a Contact struct and a Message struct in the FFI, but at least on Android we would need to handle raw
// C-data in the Java code (i.e. a `long` storing a C pointer)
// - We can't make a param `SenderDisplayname` for messages as sometimes the display name of a contact changes, and we want to show
@@ -1419,7 +1419,10 @@ pub async fn delete_msgs(context: &Context, msg_ids: &[MsgId]) -> Result<()> {
}
// Interrupt Inbox loop to start message deletion and run housekeeping.
context.interrupt_inbox(InterruptInfo::new(false)).await;
context
.scheduler
.interrupt_inbox(InterruptInfo::new(false))
.await;
Ok(())
}
@@ -1502,7 +1505,7 @@ pub async fn markseen_msgs(context: &Context, msg_ids: Vec<MsgId>) -> Result<()>
curr_rfc724_mid,
curr_blocked,
_curr_ephemeral_timer,
) in msgs.into_iter()
) in msgs
{
if curr_blocked == Blocked::Not
&& (curr_state == MessageState::InFresh || curr_state == MessageState::InNoticed)
@@ -1531,7 +1534,10 @@ pub async fn markseen_msgs(context: &Context, msg_ids: Vec<MsgId>) -> Result<()>
)
.await
.context("failed to insert into smtp_mdns")?;
context.interrupt_smtp(InterruptInfo::new(false)).await;
context
.scheduler
.interrupt_smtp(InterruptInfo::new(false))
.await;
}
}
updated_chat_ids.insert(curr_chat_id);
@@ -1741,7 +1747,7 @@ pub(crate) async fn handle_ndn(
};
let mut first = true;
for msg in msgs.into_iter() {
for msg in msgs {
let (msg_id, chat_id, chat_type) = msg?;
set_msg_failed(context, msg_id, &error).await;
if first {
@@ -1771,13 +1777,8 @@ async fn ndn_maybe_add_info_msg(
// Tell the user which of the recipients failed if we know that (because in
// a group, this might otherwise be unclear)
let text = stock_str::failed_sending_to(context, contact.get_display_name()).await;
chat::add_info_msg(
context,
chat_id,
&text,
create_smeared_timestamp(context).await,
)
.await?;
chat::add_info_msg(context, chat_id, &text, create_smeared_timestamp(context))
.await?;
context.emit_event(EventType::ChatModified(chat_id));
}
}
@@ -1799,7 +1800,7 @@ pub async fn get_unblocked_msg_cnt(context: &Context) -> usize {
"SELECT COUNT(*) \
FROM msgs m LEFT JOIN chats c ON c.id=m.chat_id \
WHERE m.id>9 AND m.chat_id>9 AND c.blocked=0;",
paramsv![],
(),
)
.await
{
@@ -1819,7 +1820,7 @@ pub async fn get_request_msg_cnt(context: &Context) -> usize {
"SELECT COUNT(*) \
FROM msgs m LEFT JOIN chats c ON c.id=m.chat_id \
WHERE c.blocked=2;",
paramsv![],
(),
)
.await
{

View File

@@ -250,7 +250,7 @@ impl<'a> MimeFactory<'a> {
.get_config(Config::Selfstatus)
.await?
.unwrap_or_default();
let timestamp = create_smeared_timestamp(context).await;
let timestamp = create_smeared_timestamp(context);
let res = MimeFactory::<'a> {
from_addr,
@@ -485,7 +485,7 @@ impl<'a> MimeFactory<'a> {
None
};
for (name, addr) in self.recipients.iter() {
for (name, addr) in &self.recipients {
if let Some(email_to_remove) = email_to_remove {
if email_to_remove == addr {
continue;
@@ -779,10 +779,36 @@ impl<'a> MimeFactory<'a> {
};
// Store protected headers in the outer message.
headers
let message = headers
.protected
.into_iter()
.fold(message, |message, header| message.header(header))
.fold(message, |message, header| message.header(header));
if self.should_skip_autocrypt()
|| !context.get_config_bool(Config::SignUnencrypted).await?
{
message
} else {
let (payload, signature) = encrypt_helper.sign(context, message).await?;
PartBuilder::new()
.header((
"Content-Type".to_string(),
"multipart/signed; protocol=\"application/pgp-signature\"".to_string(),
))
.child(payload)
.child(
PartBuilder::new()
.content_type(
&"application/pgp-signature; name=\"signature.asc\""
.parse::<mime::Mime>()
.unwrap(),
)
.header(("Content-Description", "OpenPGP digital signature"))
.header(("Content-Disposition", "attachment; filename=\"signature\";"))
.body(signature)
.build(),
)
}
};
// Store the unprotected headers on the outer message.
@@ -1906,7 +1932,7 @@ mod tests {
// These two combinations are different: If `message_arrives_inbetween` is true, but
// `reply` is false, the core is actually expected to use the subject of the message
// that arrived inbetween.
// that arrived in between.
assert_eq!(
"Re: Some other, completely unrelated subject",
msg_to_subject_str_inner(imf_raw, false, false, true).await
@@ -2140,6 +2166,96 @@ mod tests {
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_selfavatar_unencrypted_signed() {
// create chat with bob, set selfavatar
let t = TestContext::new_alice().await;
t.set_config(Config::SignUnencrypted, Some("1"))
.await
.unwrap();
let chat = t.create_chat_with_contact("bob", "bob@example.org").await;
let file = t.dir.path().join("avatar.png");
let bytes = include_bytes!("../test-data/image/avatar64x64.png");
tokio::fs::write(&file, bytes).await.unwrap();
t.set_config(Config::Selfavatar, Some(file.to_str().unwrap()))
.await
.unwrap();
// send message to bob: that should get multipart/mixed because of the avatar moved to inner header;
// make sure, `Subject:` stays in the outer header (imf header)
let mut msg = Message::new(Viewtype::Text);
msg.set_text(Some("this is the text!".to_string()));
let sent_msg = t.send_msg(chat.id, &mut msg).await;
let mut payload = sent_msg.payload().splitn(4, "\r\n\r\n");
let part = payload.next().unwrap();
assert_eq!(part.match_indices("multipart/signed").count(), 1);
assert_eq!(part.match_indices("Subject:").count(), 0);
assert_eq!(part.match_indices("Autocrypt:").count(), 1);
assert_eq!(part.match_indices("Chat-User-Avatar:").count(), 0);
let part = payload.next().unwrap();
assert_eq!(part.match_indices("multipart/mixed").count(), 1);
assert_eq!(part.match_indices("Subject:").count(), 1);
assert_eq!(part.match_indices("Autocrypt:").count(), 0);
assert_eq!(part.match_indices("Chat-User-Avatar:").count(), 0);
let part = payload.next().unwrap();
assert_eq!(part.match_indices("text/plain").count(), 1);
assert_eq!(part.match_indices("Chat-User-Avatar:").count(), 1);
assert_eq!(part.match_indices("Subject:").count(), 0);
let body = payload.next().unwrap();
assert_eq!(body.match_indices("this is the text!").count(), 1);
let bob = TestContext::new_bob().await;
bob.recv_msg(&sent_msg).await;
let alice_id = Contact::lookup_id_by_addr(&bob.ctx, "alice@example.org", Origin::Unknown)
.await
.unwrap()
.unwrap();
let alice_contact = Contact::load_from_db(&bob.ctx, alice_id).await.unwrap();
assert!(alice_contact
.get_profile_image(&bob.ctx)
.await
.unwrap()
.is_some());
// if another message is sent, that one must not contain the avatar
// and no artificial multipart/mixed nesting
let sent_msg = t.send_msg(chat.id, &mut msg).await;
let mut payload = sent_msg.payload().splitn(3, "\r\n\r\n");
let part = payload.next().unwrap();
assert_eq!(part.match_indices("multipart/signed").count(), 1);
assert_eq!(part.match_indices("Subject:").count(), 0);
assert_eq!(part.match_indices("Autocrypt:").count(), 1);
assert_eq!(part.match_indices("Chat-User-Avatar:").count(), 0);
let part = payload.next().unwrap();
assert_eq!(part.match_indices("text/plain").count(), 1);
assert_eq!(part.match_indices("Subject:").count(), 1);
assert_eq!(part.match_indices("Autocrypt:").count(), 0);
assert_eq!(part.match_indices("multipart/mixed").count(), 0);
assert_eq!(part.match_indices("Chat-User-Avatar:").count(), 0);
let body = payload.next().unwrap();
assert_eq!(body.match_indices("this is the text!").count(), 1);
assert_eq!(body.match_indices("text/plain").count(), 0);
assert_eq!(body.match_indices("Chat-User-Avatar:").count(), 0);
assert_eq!(body.match_indices("Subject:").count(), 0);
bob.recv_msg(&sent_msg).await;
let alice_contact = Contact::load_from_db(&bob.ctx, alice_id).await.unwrap();
assert!(alice_contact
.get_profile_image(&bob.ctx)
.await
.unwrap()
.is_some());
}
/// Test that removed member address does not go into the `To:` field.
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_remove_member_bcc() -> Result<()> {

Some files were not shown because too many files have changed in this diff Show More