Compare commits

..
Author SHA1 Message Date
Simon Laux da11148e1b fix: npm rpc: set default options for startDeltaChat
this fixes an "undefined" error
2024-05-20 18:09:55 +02:00
Simon Lauxandiequidoo dd77d32446 fix: log/print exit error of deltachat-rpc-server (#5601)
see also #5599 (this logs that error atleast, but does not fix it yet)

---------

Co-authored-by: iequidoo <117991069+iequidoo@users.noreply.github.com>
2024-05-20 10:47:36 +00:00
link2xt b32fb05ab8 fix: ignore event channel overflows
async-broadcast returns Overflowed error once
if channel overflow happened.
Public APIs such as get_next_event JSON-RPC method
are only expecting an error if the channel is closed,
so we should not propagate overflow error outside.
In particular, Delta Chat Desktop
stop receiving events completely if an error
is returned once.
If overflow happens, we should ignore it
and try again until we get an event or an error because
the channel is closed (in case of recv())
or empty (in case of try_recv()).
2024-05-20 10:44:35 +00:00
link2xt 918d87dcb6 refactor: do not try to lookup group in create_or_lookup_group() 2024-05-20 05:20:38 +00:00
link2xt 98ae05ee59 refactor: stop trying to extract chat ID from Message-IDs 2024-05-20 05:20:38 +00:00
link2xt cff5c064a6 refactor: use let..else in create_or_lookup_group() 2024-05-20 05:20:38 +00:00
link2xt e9cef4b0ba refactor(receive_imf): only call create_or_lookup_group() with allow_creation=true 2024-05-20 05:20:38 +00:00
link2xt 7f2c8ff53d refactor(receive_imf): remove unnecessary check for is_mdn
If message is an MDN, it is already assigned to the trash chat
by now, so it is enough to check if the message
is not assigned to chat yet.
2024-05-20 05:20:38 +00:00
link2xt 46d6b81058 refactor(receive_imf): do not check for ContactId::UNDEFINED
from_id should not be undefined,
we either should create a contact for it by then
or reject the message.
2024-05-20 05:20:38 +00:00
link2xt 6d59fb49aa feat: replace env_logger with tracing_subscriber
This allows to get iroh logs with
RUST_LOG=iroh_net=trace
2024-05-19 23:22:37 +00:00
Simon Laux 97602f3fd7 fix: sql syntax error in db migration 27 2024-05-19 20:15:22 +02:00
Simon Laux f17987743e fix: db migration version 59, it contained an sql syntax error 2024-05-19 20:15:22 +02:00
link2xt 5767cce178 fix(mimeparser): take the last header of multiple ones with the same name
If multiple headers with the same name are present,
we must take the last one.
This is the one that is DKIM-signed if
this header is DKIM-signed at all.

Ideally servers should prevent adding
more From, To and Cc headers by oversigning
them, but unfortunately it is not common
and OpenDKIM in its default configuration
does not oversign any headers.
2024-05-18 22:24:39 +00:00
link2xt 20a4bb1a88 api(deltachat-rpc-client): add Account.wait_for_incoming_msg() 2024-05-18 22:24:17 +00:00
link2xt 578f29f215 chore(release): prepare for 1.139.2 2024-05-18 20:58:03 +00:00
link2xt 6c9643e39e build: add repository URL to @deltachat/jsonrpc-client 2024-05-18 20:56:11 +00:00
link2xt 502ae7fd9f chore(release): prepare for 1.139.1 2024-05-18 20:40:34 +00:00
link2xt 8cb527342a ci: set --access public when publishing to npm
Otherwise it fails to upload new packages with provenance.
2024-05-18 20:39:41 +00:00
link2xt 964c943dd9 chore(release): prepare for 1.139.0 2024-05-18 19:42:54 +00:00
Simon Laux a971ad1f85 npm rpc server: fix example (#5580) 2024-05-18 19:58:44 +02:00
link2xt e66b9de922 fix: do not mark the message as seen if it has location.kml 2024-05-18 16:09:55 +00:00
adbenitezandlink2xt 5db202169b fix: save override sender displayname for outgoing messages
Co-authored-by: link2xt <link2xt@testrun.org>
2024-05-18 16:08:53 +00:00
link2xt b292b191ff test(deltachat-rpc-client): test sending vCard 2024-05-18 14:00:57 +00:00
link2xt 450ff411ec api(deltachat-rpc-client): add Chat.send_contact() 2024-05-18 14:00:57 +00:00
link2xt 8de92e54eb api(deltachat-rpc-client): add Contact.make_vcard() 2024-05-18 14:00:57 +00:00
link2xt d0844c3e62 api(deltachat-rpc-client): add ViewType.VCARD constant 2024-05-18 14:00:57 +00:00
link2xt 37d61e41ca api(deltachat-jsonrpc): return vcard contact directly in MessageObject 2024-05-17 23:37:29 +00:00
Simon Laux 0c7dad961d npm rpc: fix convert_platform.py: 32bit i32 -> ia32 (#5589) 2024-05-17 23:35:50 +02:00
36f1fc4f9d feat: ephemeral peer channels (#5346)
Co-authored-by: link2xt <link2xt@testrun.org>
Co-authored-by: iequidoo <117991069+iequidoo@users.noreply.github.com>
2024-05-17 20:13:21 +00:00
Simon Laux 517cb821fb jsonrpc: add api migrate_account and get_blob_dir (#5584)
closes #5533

adds the functions that were still missing for migration to jsonrpc (the
ones that the cffi already had, so just should be quick to review ;)
2024-05-17 21:33:59 +02:00
Simon Laux ef6c3f8476 fix: rpc npm: fix local desktop development (#5583)
typescript was complaining about missing `@deltachat/jsonrpc-client`
when it wasn't installed locally
2024-05-17 21:28:33 +02:00
link2xt f84f0d5ad9 ci: check that constants are always up-to-date 2024-05-17 19:23:27 +00:00
Simon Laux d8e98279c4 rpc npm: rename shutdown method to close and add muteStdErr option to mute the stderr output (#5588) 2024-05-17 21:14:38 +02:00
link2xt 424ac606d8 ci: publish @deltachat/jsonrpc-client directly to npm 2024-05-17 19:10:39 +00:00
Sebastian Klähn 2f35d9a013 use rust-analyzer nightly 2024-05-17 17:37:51 +00:00
Sebastian Klähn e5259176c9 nix: add git-cliff to flake 2024-05-17 17:36:02 +00:00
link2xt c370195698 chore(cargo): downgrade libc from 0.2.154 to 0.2.153
0.2.154 is yanked.
2024-05-17 13:52:19 +00:00
Simon Laux 0ba0bd3d77 fix(@deltachat/stdio-rpc-server): fix version check when deltachat-rpc-server is found in path (#5579) 2024-05-17 11:11:11 +02:00
link2xt d23a7b8523 chore(release): prepare for 1.138.5 2024-05-16 15:07:46 +00:00
link2xt 935f503bc7 chore: rebuild node constants 2024-05-16 15:01:56 +00:00
link2xt a0f0a8e021 build: add repository URL to deltachat-rpc-server packages 2024-05-16 14:59:45 +00:00
iequidoo 6290ed8752 api: Add make_vcard() (#5203)
Add a function returning a vCard containing contacts with the given ids.
2024-05-15 21:49:59 -03:00
iequidoo a38f0ba09e refactor: VcardContact: Change timestamp type to i64
- u64 only adds unnecessary conversions.
- `Contact::last_seen` is also `i64`, so make timestamps such everywhere.
2024-05-15 21:07:24 -03:00
iequidoo 191624f334 refactor(contact-tools): VcardContact: rename display_name to authname
It's actually `deltachat::contact::Contact::authname` by semantics. `display_name`/`name` shouldn't
be shared, it's the name given by the user locally.
2024-05-15 21:07:24 -03:00
Hocuri 5292a49bb1 fix: parsing vCards with avatars exported by Android's "Contacts" app 2024-05-15 21:07:24 -03:00
iequidooandHocuri 22f01a2699 api: Add Viewtype::Vcard (#5202)
Co-authored-by: Hocuri <hocuri@gmx.de>
2024-05-15 21:07:24 -03:00
95238b6e17 api(jsonrpc): Add parse_vcard() (#5202)
Add a function parsing a vCard file at the given path.

Co-authored-by: Hocuri <hocuri@gmx.de>
Co-authored-by: Asiel Díaz Benítez <asieldbenitez@gmail.com>
2024-05-15 21:07:24 -03:00
link2xt 4a738ebd19 chore(release): prepare for 1.138.4 2024-05-15 21:59:28 +00:00
link2xt d02eccd303 ci: run actions/setup-node before npm publish
Otherwise it does not try to use NODE_AUTH_TOKEN.
2024-05-15 21:57:24 +00:00
link2xt f1fa053f9f chore(release): prepare for 1.138.3 2024-05-15 20:42:34 +00:00
link2xt 38c1caf180 ci: give CI job permission to publish binaries to the release
Otherwise it fails on `gh release upload` step.
2024-05-15 20:38:59 +00:00
link2xt 97d2812644 chore(release): prepare for 1.138.2 2024-05-15 18:34:06 +00:00
link2xt 2ab713d968 ci: add npm token to publish deltachat-rpc-server packages 2024-05-15 18:08:58 +00:00
link2xt b7a25d5092 api(deltachat-rpc-client): add CONFIG_SYNCED constant 2024-05-15 17:20:06 +00:00
link2xt 8cd85fa7a4 feat: reset more settings when configuring a chatmail account 2024-05-15 06:30:17 +00:00
link2xt 7cfab9a931 test: set configuration after configure() finishes
This allows to overwrite settings changed
when XCHATMAIL capability is detected.
2024-05-15 06:27:57 +00:00
link2xt 30086038e6 chore(release): prepare for 1.138.1 2024-05-14 22:25:13 +00:00
link2xt eec1062619 feat: detect XCHATMAIL capability and expose it as is_chatmail config 2024-05-14 22:17:46 +00:00
link2xt 07ceabdf85 refactor: resultify token::lookup_or_new() 2024-05-14 20:32:23 +00:00
Simon Laux c349bf8e0c ci(deltachat-rpc-server): fix upload of npm packages to github releases (#5564) 2024-05-14 18:48:34 +00:00
link2xt 21eb4f6648 chore(cargo): bump brotli from 5.0.0 to 6.0.0 2024-05-14 17:27:33 +00:00
dependabot[bot] 10fed7d7de chore(cargo): bump human-panic from 1.2.3 to 2.0.0
Bumps [human-panic](https://github.com/rust-cli/human-panic) from 1.2.3 to 2.0.0.
- [Changelog](https://github.com/rust-cli/human-panic/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-cli/human-panic/compare/v1.2.3...v2.0.0)

---
updated-dependencies:
- dependency-name: human-panic
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-05-14 16:29:22 +00:00
dependabot[bot] b08a283fe5 chore(cargo): bump serde_json from 1.0.115 to 1.0.116
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.115 to 1.0.116.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.115...v1.0.116)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-05-14 16:12:52 +00:00
dependabot[bot] 45a2805100 chore(cargo): bump hickory-resolver from 0.24.0 to 0.24.1
Bumps [hickory-resolver](https://github.com/hickory-dns/hickory-dns) from 0.24.0 to 0.24.1.
- [Release notes](https://github.com/hickory-dns/hickory-dns/releases)
- [Changelog](https://github.com/hickory-dns/hickory-dns/blob/main/CHANGELOG.md)
- [Commits](https://github.com/hickory-dns/hickory-dns/compare/v0.24.0...v0.24.1)

---
updated-dependencies:
- dependency-name: hickory-resolver
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-05-14 16:11:48 +00:00
dependabot[bot] cc8157ecf1 chore(cargo): bump libc from 0.2.153 to 0.2.154
Bumps [libc](https://github.com/rust-lang/libc) from 0.2.153 to 0.2.154.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.153...0.2.154)

---
updated-dependencies:
- dependency-name: libc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-05-14 16:11:30 +00:00
dependabot[bot] 0c98aca5f0 chore(cargo): bump parking_lot from 0.12.1 to 0.12.2
Bumps [parking_lot](https://github.com/Amanieu/parking_lot) from 0.12.1 to 0.12.2.
- [Changelog](https://github.com/Amanieu/parking_lot/blob/master/CHANGELOG.md)
- [Commits](https://github.com/Amanieu/parking_lot/compare/0.12.1...0.12.2)

---
updated-dependencies:
- dependency-name: parking_lot
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-05-14 16:11:12 +00:00
link2xt 170e4b3530 refactor(sql): make open flags immutable 2024-05-14 14:55:04 +00:00
link2xt 5ed91e9f6e refactor: make MimeMessage.get_header() return Option<&str> 2024-05-13 17:07:58 +00:00
link2xt 2779737c56 ci: set RUSTUP_WINDOWS_PATH_ADD_BIN
This is a workaround for
<https://github.com/nextest-rs/nextest/issues/1493>
2024-05-13 14:29:33 +00:00
link2xt 0d3c0a3d8f fix: ignore parent message if message references itself
When there are no parent references,
Delta Chat inserts Message-ID into References.
Such references should be ignored
because otherwise fully downloaded message
may be assigned to the same chat as previously incorrectly assigned
partially downloaded message.

Fixes receive_imf::tests::test_create_group_with_big_msg
2024-05-13 13:29:06 +00:00
link2xt 8e38e7220b fix: always prefer Chat-Group-ID over In-Reply-To and References
Chat-Group-ID always correctly identifies the chat
message was sent to, while In-Reply-To and References
may point to a message that has itself been incorrectly
assigned to a chat.
2024-05-13 13:29:06 +00:00
link2xt acfde3cb7b fix: never treat message with Chat-Group-ID as a private reply 2024-05-13 13:29:06 +00:00
link2xt b6a461e3b7 refactor: add MimeMessage.get_chat_group_id() 2024-05-13 13:29:06 +00:00
B. Petersen 0541ecf22c chore(release): prepare for 1.138.0 2024-05-13 12:47:11 +02:00
B. Petersen 77af0a2114 update node constants 2024-05-13 12:47:11 +02:00
B. Petersen 2f679bc21a add new securejoin strings to deltachat.h 2024-05-13 12:08:36 +02:00
iequidoo 518db9a20f feat: Make one-to-one chats read-only the first seconds of a SecureJoin (#5512)
This protects Bob (the joiner) of sending unexpected-unencrypted messages during an otherwise nicely
running SecureJoin.

If things get stuck, however, we do not want to block communication -- the chat is just
opportunistic as usual, but that needs to be communicated:
1. If Bob's chat with Alice is `Unprotected` and a SecureJoin is started, then add info-message
   "Establishing guaranteed end-to-end encryption, please wait..." and let `Chat::can_send()` return
   `false`.
2. Once the info-message "Messages are guaranteed to be e2ee from now on" is added, let
   `Chat::can_send()` return `true`.
3. If after SECUREJOIN_WAIT_TIMEOUT seconds `2.` did not happen, add another info-message "Could not
   yet establish guaranteed end-to-end encryption but you may already send a message" and also let
   `Chat::can_send()` return `true`.

Both `2.` and `3.` require the event `ChatModified` being sent out so that UI pick up the change wrt
`Chat::can_send()` (this is the same way how groups become updated wrt `can_send()` changes).

SECUREJOIN_WAIT_TIMEOUT should be 10-20 seconds so that we are reasonably sure that the app remains
active and receiving also on mobile devices. If the app is killed during this time then we may need
to do step 3 for any pending Bob-join chats (right now, Bob can only join one chat at a time).
2024-05-13 12:08:36 +02:00
link2xt edf8aafbdc api!(jsonrpc): replace EphemeralTimer tag "variant" with "kind"
We use "kind" everywhere else.
2024-05-12 01:15:31 +00:00
iequidoo ab1583eef9 fix: Use ChatIdBlocked::lookup_by_contact() instead of ChatId's method when applicable
`ChatId::lookup_by_contact()` returns `None` for blocked chats, so it should be only used if we need
to filter out blocked chats, e.g. when building a chatlist.
2024-05-11 17:37:12 -03:00
iequidoo e3cb9b894b fix: Update special chats icons even if they are blocked (#5509)
E.g. the multi-device synchronisation creates the "Saved Messages" chat as blocked, in this case the
chat icon wasn't updated before and the user avatar was displayed instead.
2024-05-11 17:37:12 -03:00
Simon Lauxandlink2xt c375c03d8e stdio jsonrpc server npm package (#5332)
- [x] figgure out how to build the packages (that it installs native
optional package automatically)
- [X] Make the gluecode
- [x] expose both the lowerlevel api that desktop uses (~~send objects
and receive objects~~, getting path of rpc-server is enough)
  - [X] and the higher level api needed for bots (jsonrpc client)
  - [X] typescript types
- [x] automatically pick the right binary from npm or allow getting it
from env var, or give out an error (throw error)
- [x] find out how to dev locally (use local built core in dc desktop) -
there is the question of how to link the typescript client and the task
to add a search in the cargo target folder for a debug build or a
different way, find out some good flow that we can use and document for
dc desktop + locally built core development
- [x] build the packages in ci
- [x] fix that deltachat-rpc-server is not executable

postponed:
- [ ] publish from ci
   - [ ] add key/token to deploy to npm 

Closes #4694

## Related prs
- https://github.com/deltachat-bot/echo/pull/69
- https://github.com/deltachat/deltachat-desktop/pull/3567

---------

Co-authored-by: link2xt <link2xt@testrun.org>
2024-05-11 20:54:11 +02:00
bjoernandlink2xt 14aaab05b0 limit quote replies (#5543)
this PR checks if the quotes are used in a reasonable way:

- quoted messages should be send to the same chat
- or to one-to-one chats

if the quote's chat ID is not the same than the sending chat _and_ the
sending chat is not a one-to-one chat, sending is aborted.

usually, the UIs does not allow quoting in other ways, so, this check is
only a "last defence line" to avoid leaking data in case the UI has
bugs, as recently in
https://github.com/deltachat/deltachat-android/issues/3032

@iequidoo @link2xt @adbenitez i am not 100% sure about this PR, maybe
i've overseen a reasonable usecase where other quotes make sense

---------

Co-authored-by: link2xt <link2xt@testrun.org>
2024-05-10 16:43:49 +02:00
link2xt 72c09feb64 feat: do not add location markers to messages with non-POI location 2024-05-10 14:04:16 +00:00
Nico de Haen 8a4dff2212 Add webxdc internal integration commands in jsonrpc (#5541)
Adds 
_setWebxdcIntegration_ 
_initWebxdcIntegration_
2024-05-08 07:53:04 +02:00
dependabot[bot] 022f836d35 chore(cargo): bump schemars from 0.8.16 to 0.8.19
Bumps [schemars](https://github.com/GREsau/schemars) from 0.8.16 to 0.8.19.
- [Release notes](https://github.com/GREsau/schemars/releases)
- [Changelog](https://github.com/GREsau/schemars/blob/master/CHANGELOG.md)
- [Commits](https://github.com/GREsau/schemars/compare/v0.8.16...v0.8.19)

---
updated-dependencies:
- dependency-name: schemars
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-05-08 03:42:51 +00:00
link2xt 636ab4a9e5 fix: delete non-POI locations after delete_device_after, not immediately 2024-05-06 18:32:04 +00:00
link2xt 2bddefa1ab refactor: remove allow_keychange 2024-05-05 21:23:32 +00:00
dependabot[bot] 7d67100a3c Merge pull request #5523 from deltachat/dependabot/cargo/mailparse-0.15.0 2024-05-05 21:23:10 +00:00
dependabot[bot] 1043916411 chore(cargo): bump mailparse from 0.14.1 to 0.15.0
Bumps [mailparse](https://github.com/staktrace/mailparse) from 0.14.1 to 0.15.0.
- [Commits](https://github.com/staktrace/mailparse/compare/v0.14.1...v0.15.0)

---
updated-dependencies:
- dependency-name: mailparse
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-05-05 20:59:52 +00:00
dependabot[bot] f4e58e90ae chore(cargo): bump syn from 2.0.57 to 2.0.60
Bumps [syn](https://github.com/dtolnay/syn) from 2.0.57 to 2.0.60.
- [Release notes](https://github.com/dtolnay/syn/releases)
- [Commits](https://github.com/dtolnay/syn/compare/2.0.57...2.0.60)

---
updated-dependencies:
- dependency-name: syn
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-05-05 13:50:07 +00:00
link2xt e4f10b32dd chore(cargo): bump imap-proto from 0.16.4 to 0.16.5 2024-05-04 22:41:30 +00:00
dependabot[bot] e9431888a6 Merge pull request #5522 from deltachat/dependabot/cargo/chrono-0.4.38 2024-05-04 22:40:17 +00:00
dependabot[bot] 1649073c0f chore(cargo): bump anyhow from 1.0.81 to 1.0.82
Bumps [anyhow](https://github.com/dtolnay/anyhow) from 1.0.81 to 1.0.82.
- [Release notes](https://github.com/dtolnay/anyhow/releases)
- [Commits](https://github.com/dtolnay/anyhow/compare/1.0.81...1.0.82)

---
updated-dependencies:
- dependency-name: anyhow
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-05-04 03:23:17 -03:00
dependabot[bot] b2cf18d8b3 chore(cargo): bump chrono from 0.4.37 to 0.4.38
Bumps [chrono](https://github.com/chronotope/chrono) from 0.4.37 to 0.4.38.
- [Release notes](https://github.com/chronotope/chrono/releases)
- [Changelog](https://github.com/chronotope/chrono/blob/main/CHANGELOG.md)
- [Commits](https://github.com/chronotope/chrono/compare/v0.4.37...v0.4.38)

---
updated-dependencies:
- dependency-name: chrono
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-05-04 02:53:42 +00:00
iequidoo 2eceb4be29 feat(contact-tools): Add make_vcard()
Add a function making a vCard containing given `VcardContact`s.
2024-05-03 23:52:51 -03:00
iequidoo ae7ff17ba2 feat(contact-tools): Support parsing profile images from "PHOTO:data:image/jpeg;base64,..." 2024-05-03 23:52:51 -03:00
Hocuriandiequidoo 026f678452 feat: Parsing vCards for contacts sharing (#5482)
Co-authored-by: iequidoo <dgreshilov@gmail.com>
2024-05-03 22:44:09 -03:00
link2xt add8c0680f ci: update Rust to 1.78.0 2024-05-03 02:11:07 +00:00
dependabot[bot] aee2b81c06 chore(cargo): bump thiserror from 1.0.58 to 1.0.59
Bumps [thiserror](https://github.com/dtolnay/thiserror) from 1.0.58 to 1.0.59.
- [Release notes](https://github.com/dtolnay/thiserror/releases)
- [Commits](https://github.com/dtolnay/thiserror/compare/1.0.58...1.0.59)

---
updated-dependencies:
- dependency-name: thiserror
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-05-02 20:35:31 -03:00
dependabot[bot] 3624aad1b5 chore(cargo): bump async-channel from 2.2.0 to 2.2.1
Bumps [async-channel](https://github.com/smol-rs/async-channel) from 2.2.0 to 2.2.1.
- [Release notes](https://github.com/smol-rs/async-channel/releases)
- [Changelog](https://github.com/smol-rs/async-channel/blob/master/CHANGELOG.md)
- [Commits](https://github.com/smol-rs/async-channel/compare/v2.2.0...v2.2.1)

---
updated-dependencies:
- dependency-name: async-channel
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-05-02 20:24:27 -03:00
link2xt 299d994d4b ci: replace black with ruff format
We use `ruff` anyway, so it is one less dependency.
2024-05-02 14:44:11 +02:00
dependabot[bot] 5e0f5ec390 chore(cargo): bump serde from 1.0.197 to 1.0.200
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.197 to 1.0.200.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.197...v1.0.200)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-05-02 04:27:42 -03:00
dependabot[bot] c318ca5d1a chore(cargo): bump base64 from 0.22.0 to 0.22.1
Bumps [base64](https://github.com/marshallpierce/rust-base64) from 0.22.0 to 0.22.1.
- [Changelog](https://github.com/marshallpierce/rust-base64/blob/master/RELEASE-NOTES.md)
- [Commits](https://github.com/marshallpierce/rust-base64/compare/v0.22.0...v0.22.1)

---
updated-dependencies:
- dependency-name: base64
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-05-02 04:18:18 -03:00
dependabot[bot] 38a2e07194 Merge pull request #5515 from deltachat/dependabot/cargo/quote-1.0.36 2024-05-02 06:22:56 +00:00
dependabot[bot] 1ff6740938 chore(cargo): bump quote from 1.0.35 to 1.0.36
Bumps [quote](https://github.com/dtolnay/quote) from 1.0.35 to 1.0.36.
- [Release notes](https://github.com/dtolnay/quote/releases)
- [Commits](https://github.com/dtolnay/quote/compare/1.0.35...1.0.36)

---
updated-dependencies:
- dependency-name: quote
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-05-01 21:52:09 +00:00
Christian Hagenest 402d5bed85 rm npm install 2024-04-30 21:10:36 +02:00
missytake 57bc046381 ci: use rsync instead of 3rd party github action 2024-04-30 21:10:36 +02:00
Christian Hagenest 0617236eb0 rm leading slash 2024-04-30 21:10:36 +02:00
Christian Hagenest 8c5ffe0237 upload-docs npm run => npm run build 2024-04-30 21:10:36 +02:00
Christian Hagenest 39f977c1e6 add : to upload-docs.yml 2024-04-30 21:10:36 +02:00
Christian Hagenest ec03614cae add npm install to upload-docs.yml 2024-04-30 21:10:36 +02:00
Christian Hagenest ea0b063c19 test ts build on branch 2024-04-30 21:10:36 +02:00
Christian Hagenest 98d7a93909 correct folder for js.jsonrpc docs 2024-04-30 21:10:36 +02:00
Christian Hagenest 49bf8414ed rm unneeded rust install from ts docs ci 2024-04-30 21:10:36 +02:00
Christian Hagenest 1e7dbea351 Implement jsonrpc-docs build in github action 2024-04-30 21:10:36 +02:00
Christian Hagenest 0412244646 Revert "WIP: build ts docs with ci + nix"
This reverts commit 00d486e58f.
2024-04-30 21:10:36 +02:00
Christian Hagenest bbd854d7bc Revert "push docs to delta.chat instead of codespeak"
This reverts commit c04188ce56.
2024-04-30 21:10:36 +02:00
Christian Hagenest ba2bb517f7 Revert "WIP npm2nix, error: no source/nix"
This reverts commit 83bfba53de.
2024-04-30 21:10:36 +02:00
Christian Hagenest 0ae831eca0 WIP npm2nix, error: no source/nix 2024-04-30 21:10:36 +02:00
Christian Hagenest ab494ae786 push docs to delta.chat instead of codespeak 2024-04-30 21:10:36 +02:00
Christian Hagenest 8a58ae8a3a WIP: build ts docs with ci + nix 2024-04-30 21:10:36 +02:00
link2xt cf84255e99 test: test that POIs are deleted when ephemeral message expires 2024-04-29 22:35:59 +00:00
link2xt 462bd63065 docs: add location module documentation 2024-04-29 22:35:59 +00:00
link2xt 6bfbf6547b feat: delete orphaned POI locations during housekeeping 2024-04-29 22:35:59 +00:00
link2xt 13802bab42 fix: delete POI location when disappearing message expires 2024-04-29 22:35:59 +00:00
link2xt adb2e4ea32 refactor: move delete_poi_location to location module and document it 2024-04-29 22:35:59 +00:00
link2xt 421a7b277d docs: remove Doxygen markup from Message.has_location() 2024-04-29 22:35:59 +00:00
link2xt 14d8139883 feat: delete expired path locations in ephemeral loop 2024-04-29 22:35:59 +00:00
link2xt 062905924c docs: fix references in Message.set_location() documentation 2024-04-29 22:35:59 +00:00
iequidoo 20d79970a2 fix: Correct message viewtype before recoding image blob (#5496)
Otherwise, e.g. if a message is a large GIF, but its viewtype is set to `Image` by the app, this GIF
will be recoded to JPEG to reduce its size. GIFs and other special viewtypes must be always detected
and sent as is.
2024-04-27 23:44:50 -03:00
link2xt f49588e64e fix: interrupt location loop when new location is stored
Otherwise location-only messages
that should be sent every 60 seconds
are never sent because location loop
waits until the end of location streaming
and is only interrupted by location streaming
ending in other chats or being enabled in other chats.
2024-04-27 20:55:05 +00:00
link2xt 496a8e3810 test: test that member is added even if "Member added" is lost
This is similar to `test_modify_chat_disordered`,
but tests that recovery works in the simplest case
where next message is not modifying group membership.
2024-04-27 07:51:08 +00:00
link2xt 94dc65c1a2 refactor(python): fix ruff 0.4.2 warnings 2024-04-25 20:45:54 +00:00
link2xt 4fe7fa3148 fix: never prepend subject to message text when bot receives it 2024-04-25 12:15:54 +00:00
link2xt 4cf923ccb9 fix: do not fail to send encrypted quotes to unencrypted chats
Replace quote text with "..." instead.
2024-04-25 09:00:49 +00:00
iequidoo 56b86adf18 api: Add dc_msg_save_file() which saves file copy at the provided path (#4309)
... and fails if file already exists. The UI should open the file saving dialog, defaulting to
Downloads and original filename, when asked to save the file. After confirmation it should call
dc_msg_save_file().
2024-04-24 16:38:25 -03:00
iequidoo cfccee2ad4 fix: Message::set_file_from_bytes(): Set Param::Filename 2024-04-24 16:38:25 -03:00
Hocuri 37d92e3fa5 test: Explain test_was_seen_recently false-positive and give workaround instructions (#5474)
Until the issue is fixed, keep others from running into the same issue.
2024-04-24 14:55:15 +00:00
link2xt a1ee2b463f chore(release): prepare for 1.137.4 2024-04-24 11:10:25 +00:00
link2xt 8df3b1bb1b fix: use only CRLF in Autocrypt Setup Message 2024-04-24 01:50:28 +00:00
iequidoo 22f240dd4d feat: Add progressive backoff for failing IMAP connection attempts (#5443)
This way we avoid an immediate retry if the network is not yet ready exhausting the ratelimiter's
quota of two connection attempts. Also notify the ratelimiter only after a successful connection so
that it only limits the server load, but not connection attempts.
2024-04-23 22:00:47 -03:00
iequidoo ae10ed5c40 refactor: Imap: remove RwLock from ratelimit 2024-04-23 22:00:47 -03:00
link2xt aff6bf9402 fix: convert images to RGB8 (without alpha) before encoding into JPEG
Otherwise an error
"The encoder or decoder for Jpeg does not support the color type `Rgba8`"
is returned if image has an alpha channel.

This is caused by the recent change of JPEG encoder
in `image` crate: <https://github.com/image-rs/image/issues/2211>
2024-04-23 23:37:58 +00:00
link2xt 43fc55e542 test: test recoding RGBA image 2024-04-23 23:37:58 +00:00
link2xt 7ea05cb8a0 test: add screenshot-rgba.png
Created by `convert -alpha deactivate screenshot.png screenshot-rbga.png`.
2024-04-23 23:37:58 +00:00
link2xt d036ad5853 fix: do not fail if Autocrypt Setup Message has no encryption preference
According to Autocrypt specification
Autocrypt Setup Message SHOULD
contain Autocrypt-Prefer-Encrypt header,
but K-9 6.802 does not include it.
2024-04-23 22:16:54 +00:00
link2xt e9280b8413 refactor: group use at the top of the test modules 2024-04-23 21:07:50 +00:00
link2xt 2108a8ba94 fix(node): undefine NAPI_EXPERIMENTAL
This fixes build with new clang
which treats -Wincompatible-function-pointer-types as an error.

Related upstream issue: <https://github.com/nodejs/node/issues/52229>
2024-04-23 11:34:03 +00:00
119 changed files with 6355 additions and 1187 deletions
+19 -4
View File
@@ -24,7 +24,7 @@ jobs:
name: Lint Rust
runs-on: ubuntu-latest
env:
RUSTUP_TOOLCHAIN: 1.77.1
RUSTUP_TOOLCHAIN: 1.78.0
steps:
- uses: actions/checkout@v4
with:
@@ -40,6 +40,18 @@ jobs:
- name: Check
run: cargo check --workspace --all-targets --all-features
npm_constants:
name: Check if node constants are up to date
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
show-progress: false
- name: Rebuild constants
run: npm run build:core:constants
- name: Check that constants are not changed
run: git diff --exit-code
cargo_deny:
name: cargo deny
runs-on: ubuntu-latest
@@ -83,11 +95,11 @@ jobs:
matrix:
include:
- os: ubuntu-latest
rust: 1.77.1
rust: 1.78.0
- os: windows-latest
rust: 1.77.1
rust: 1.78.0
- os: macos-latest
rust: 1.77.1
rust: 1.78.0
# Minimum Supported Rust Version = 1.77.0
- os: ubuntu-latest
@@ -113,6 +125,9 @@ jobs:
- name: Tests
env:
RUST_BACKTRACE: 1
# Workaround for <https://github.com/nextest-rs/nextest/issues/1493>.
RUSTUP_WINDOWS_PATH_ADD_BIN: 1
run: cargo nextest run --workspace
- name: Doc-Tests
+138
View File
@@ -266,3 +266,141 @@ jobs:
- name: Publish deltachat-rpc-client to PyPI
if: github.event_name == 'release'
uses: pypa/gh-action-pypi-publish@release/v1
publish_npm_package:
name: Build & Publish npm prebuilds and deltachat-rpc-server
needs: ["build_linux", "build_windows", "build_macos"]
runs-on: "ubuntu-latest"
permissions:
id-token: write
# Needed to publish the binaries to the release.
contents: write
steps:
- uses: actions/checkout@v4
with:
show-progress: false
- uses: actions/setup-python@v5
with:
python-version: "3.11"
- name: Download Linux aarch64 binary
uses: actions/download-artifact@v4
with:
name: deltachat-rpc-server-aarch64-linux
path: deltachat-rpc-server-aarch64-linux.d
- name: Download Linux armv7l binary
uses: actions/download-artifact@v4
with:
name: deltachat-rpc-server-armv7l-linux
path: deltachat-rpc-server-armv7l-linux.d
- name: Download Linux armv6l binary
uses: actions/download-artifact@v4
with:
name: deltachat-rpc-server-armv6l-linux
path: deltachat-rpc-server-armv6l-linux.d
- name: Download Linux i686 binary
uses: actions/download-artifact@v4
with:
name: deltachat-rpc-server-i686-linux
path: deltachat-rpc-server-i686-linux.d
- name: Download Linux x86_64 binary
uses: actions/download-artifact@v4
with:
name: deltachat-rpc-server-x86_64-linux
path: deltachat-rpc-server-x86_64-linux.d
- name: Download Win32 binary
uses: actions/download-artifact@v4
with:
name: deltachat-rpc-server-win32
path: deltachat-rpc-server-win32.d
- name: Download Win64 binary
uses: actions/download-artifact@v4
with:
name: deltachat-rpc-server-win64
path: deltachat-rpc-server-win64.d
- name: Download macOS binary for x86_64
uses: actions/download-artifact@v4
with:
name: deltachat-rpc-server-x86_64-macos
path: deltachat-rpc-server-x86_64-macos.d
- name: Download macOS binary for aarch64
uses: actions/download-artifact@v4
with:
name: deltachat-rpc-server-aarch64-macos
path: deltachat-rpc-server-aarch64-macos.d
- name: Download Android binary for arm64-v8a
uses: actions/download-artifact@v4
with:
name: deltachat-rpc-server-arm64-v8a-android
path: deltachat-rpc-server-arm64-v8a-android.d
- name: Download Android binary for armeabi-v7a
uses: actions/download-artifact@v4
with:
name: deltachat-rpc-server-armeabi-v7a-android
path: deltachat-rpc-server-armeabi-v7a-android.d
- name: make npm packets for prebuilds and `@deltachat/stdio-rpc-server`
run: |
cd deltachat-rpc-server/npm-package
python --version
python scripts/pack_binary_for_platform.py aarch64-unknown-linux-musl ../../deltachat-rpc-server-aarch64-linux.d/deltachat-rpc-server
python scripts/pack_binary_for_platform.py armv7-unknown-linux-musleabihf ../../deltachat-rpc-server-armv7l-linux.d/deltachat-rpc-server
python scripts/pack_binary_for_platform.py arm-unknown-linux-musleabihf ../../deltachat-rpc-server-armv6l-linux.d/deltachat-rpc-server
python scripts/pack_binary_for_platform.py i686-unknown-linux-musl ../../deltachat-rpc-server-i686-linux.d/deltachat-rpc-server
python scripts/pack_binary_for_platform.py x86_64-unknown-linux-musl ../../deltachat-rpc-server-x86_64-linux.d/deltachat-rpc-server
python scripts/pack_binary_for_platform.py i686-pc-windows-gnu ../../deltachat-rpc-server-win32.d/deltachat-rpc-server.exe
python scripts/pack_binary_for_platform.py x86_64-pc-windows-gnu ../../deltachat-rpc-server-win64.d/deltachat-rpc-server.exe
python scripts/pack_binary_for_platform.py x86_64-apple-darwin ../../deltachat-rpc-server-x86_64-macos.d/deltachat-rpc-server
python scripts/pack_binary_for_platform.py aarch64-apple-darwin ../../deltachat-rpc-server-aarch64-macos.d/deltachat-rpc-server
python scripts/pack_binary_for_platform.py aarch64-linux-android ../../deltachat-rpc-server-arm64-v8a-android.d/deltachat-rpc-server
python scripts/pack_binary_for_platform.py armv7-linux-androideabi ../../deltachat-rpc-server-armeabi-v7a-android.d/deltachat-rpc-server
ls -lah platform_package
for platform in ./platform_package/*; do npm pack "$platform"; done
npm pack
ls -lah
- name: Upload to artifacts
uses: actions/upload-artifact@v4
with:
name: deltachat-rpc-server-npm-package
path: deltachat-rpc-server/npm-package/*.tgz
if-no-files-found: error
- name: Upload npm packets to the GitHub release
if: github.event_name == 'release'
env:
GITHUB_TOKEN: "${{ secrets.GITHUB_TOKEN }}"
run: |
gh release upload ${{ github.ref_name }} \
--repo ${{ github.repository }} \
deltachat-rpc-server/npm-package/*.tgz
# Configure Node.js for publishing.
- uses: actions/setup-node@v4
with:
node-version: 20
registry-url: "https://registry.npmjs.org"
- name: Publish npm packets for prebuilds and `@deltachat/stdio-rpc-server`
if: github.event_name == 'release'
working-directory: deltachat-rpc-server/npm-package
run: |
ls -lah platform_package
for platform in *.tgz; do npm publish --provenance "$platform" --access public; done
env:
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
@@ -1,82 +1,38 @@
name: "jsonrpc js client build"
name: "Publish @deltachat/jsonrpc-client"
on:
pull_request:
push:
tags:
- "*"
- "!py-*"
workflow_dispatch:
release:
types: [published]
jobs:
pack-module:
name: "Package @deltachat/jsonrpc-client and upload to download.delta.chat"
name: "Publish @deltachat/jsonrpc-client"
runs-on: ubuntu-20.04
permissions:
id-token: write
contents: read
steps:
- name: Install tree
run: sudo apt install tree
- uses: actions/checkout@v4
with:
show-progress: false
- uses: actions/setup-node@v4
with:
node-version: "18"
- name: Get tag
id: tag
uses: dawidd6/action-get-tag@v1
continue-on-error: true
- name: Get Pull Request ID
id: prepare
run: |
tag=${{ steps.tag.outputs.tag }}
if [ -z "$tag" ]; then
node -e "console.log('DELTACHAT_JSONRPC_TAR_GZ=deltachat-jsonrpc-client-' + '${{ github.ref }}'.split('/')[2] + '.tar.gz')" >> $GITHUB_ENV
else
echo "DELTACHAT_JSONRPC_TAR_GZ=deltachat-jsonrpc-client-${{ steps.tag.outputs.tag }}.tar.gz" >> $GITHUB_ENV
echo "No preview will be uploaded this time, but the $tag release"
fi
- name: System info
run: |
npm --version
node --version
echo $DELTACHAT_JSONRPC_TAR_GZ
node-version: 20
registry-url: "https://registry.npmjs.org"
- name: Install dependencies without running scripts
working-directory: deltachat-jsonrpc/typescript
run: npm install --ignore-scripts
- name: Package
shell: bash
working-directory: deltachat-jsonrpc/typescript
run: |
npm run build
npm pack .
ls -lah
mv $(find deltachat-jsonrpc-client-*) $DELTACHAT_JSONRPC_TAR_GZ
- name: Upload Prebuild
uses: actions/upload-artifact@v4
with:
name: deltachat-jsonrpc-client.tgz
path: deltachat-jsonrpc/typescript/${{ env.DELTACHAT_JSONRPC_TAR_GZ }}
# Upload to download.delta.chat/node/preview/
- name: Upload deltachat-jsonrpc-client preview to download.delta.chat/node/preview/
if: ${{ ! steps.tag.outputs.tag }}
id: upload-preview
shell: bash
run: |
echo -e "${{ secrets.SSH_KEY }}" >__TEMP_INPUT_KEY_FILE
chmod 600 __TEMP_INPUT_KEY_FILE
scp -o StrictHostKeyChecking=no -v -i __TEMP_INPUT_KEY_FILE -P "22" -r deltachat-jsonrpc/typescript/$DELTACHAT_JSONRPC_TAR_GZ "${{ secrets.USERNAME }}"@"download.delta.chat":"/var/www/html/download/node/preview/"
continue-on-error: true
- name: Post links to details
if: steps.upload-preview.outcome == 'success'
run: node ./node/scripts/postLinksToDetails.js
- name: Publish
working-directory: deltachat-jsonrpc/typescript
run: npm publish --provenance deltachat-jsonrpc-client-* --access public
env:
URL: preview/${{ env.DELTACHAT_JSONRPC_TAR_GZ }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
MSG_CONTEXT: Download the deltachat-jsonrpc-client.tgz
# Upload to download.delta.chat/node/
- name: Upload deltachat-jsonrpc-client build to download.delta.chat/node/
if: ${{ steps.tag.outputs.tag }}
id: upload
shell: bash
run: |
echo -e "${{ secrets.SSH_KEY }}" >__TEMP_INPUT_KEY_FILE
chmod 600 __TEMP_INPUT_KEY_FILE
scp -o StrictHostKeyChecking=no -v -i __TEMP_INPUT_KEY_FILE -P "22" -r deltachat-jsonrpc/typescript/$DELTACHAT_JSONRPC_TAR_GZ "${{ secrets.USERNAME }}"@"download.delta.chat":"/var/www/html/download/node/"
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
+2 -5
View File
@@ -21,17 +21,14 @@ jobs:
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ubuntu-latest, macos-14, windows-latest]
node: ["18", "20"]
os: [ubuntu-latest, macos-latest, windows-latest]
steps:
- uses: actions/checkout@v4
with:
show-progress: false
- uses: actions/setup-node@v4
with:
node-version: ${{ matrix.node }}
- if: matrix.os == 'macos-14'
run: sudo xcode-select --switch /Applications/Xcode_15.3.app/Contents/Developer
node-version: "18"
- name: System info
run: |
rustc -vV
+34 -7
View File
@@ -4,6 +4,7 @@ on:
push:
branches:
- main
- build_jsonrpc_docs_ci
jobs:
build-rs:
@@ -17,13 +18,11 @@ jobs:
run: |
cargo doc --package deltachat --no-deps --document-private-items
- name: Upload to rs.delta.chat
uses: up9cloud/action-rsync@v1.3
env:
USER: ${{ secrets.USERNAME }}
KEY: ${{ secrets.KEY }}
HOST: "delta.chat"
SOURCE: "target/doc"
TARGET: "/var/www/html/rs/"
run: |
mkdir -p "$HOME/.ssh"
echo "${{ secrets.KEY }}" > "$HOME/.ssh/key"
chmod 600 "$HOME/.ssh/key"
rsync -avzh -e "ssh -i $HOME/.ssh/key -o StrictHostKeyChecking=no" $GITHUB_WORKSPACE/target/doc "${{ secrets.USERNAME }}@rs.delta.chat:/var/www/html/rs/"
build-python:
runs-on: ubuntu-latest
@@ -62,3 +61,31 @@ jobs:
echo "${{ secrets.CODESPEAK_KEY }}" > "$HOME/.ssh/key"
chmod 600 "$HOME/.ssh/key"
rsync -avzh -e "ssh -i $HOME/.ssh/key -o StrictHostKeyChecking=no" $GITHUB_WORKSPACE/result/html/ "delta@c.delta.chat:/home/delta/build-c/master"
build-ts:
runs-on: ubuntu-latest
defaults:
run:
working-directory: ./deltachat-jsonrpc/typescript
steps:
- uses: actions/checkout@v4
with:
show-progress: false
fetch-depth: 0 # Fetch history to calculate VCS version number.
- name: Use Node.js
uses: actions/setup-node@v2
with:
node-version: '18'
- name: npm install
run: npm install
- name: npm run build
run: npm run build
- name: Run docs script
run: npm run docs
- name: Upload to js.jsonrpc.delta.chat
run: |
mkdir -p "$HOME/.ssh"
echo "${{ secrets.KEY }}" > "$HOME/.ssh/key"
chmod 600 "$HOME/.ssh/key"
rsync -avzh -e "ssh -i $HOME/.ssh/key -o StrictHostKeyChecking=no" $GITHUB_WORKSPACE/deltachat-jsonrpc/typescript/docs/ "${{ secrets.USERNAME }}@js.jsonrpc.delta.chat:/var/www/html/js-jsonrpc/"
+1 -1
View File
@@ -33,7 +33,7 @@ deltachat-ffi/xml
coverage/
.DS_Store
.vscode/launch.json
.vscode
python/accounts.txt
python/all-testaccounts.txt
tmp/
+283
View File
@@ -1,5 +1,278 @@
# Changelog
## [1.139.2] - 2024-05-18
### Build system
- Add repository URL to @deltachat/jsonrpc-client.
## [1.139.1] - 2024-05-18
### CI
- Set `--access public` when publishing to npm.
## [1.139.0] - 2024-05-18
### Features / Changes
- Ephemeral peer channels ([#5346](https://github.com/deltachat/deltachat-core-rust/pull/5346)).
### Fixes
- Save override sender displayname for outgoing messages.
- Do not mark the message as seen if it has `location.kml`.
- @deltachat/stdio-rpc-server: fix version check when deltachat-rpc-server is found in path ([#5579](https://github.com/deltachat/deltachat-core-rust/pull/5579)).
- @deltachat/stdio-rpc-server: fix local desktop development ([#5583](https://github.com/deltachat/deltachat-core-rust/pull/5583)).
- @deltachat/stdio-rpc-server: rename `shutdown` method to `close` and add `muteStdErr` option to mute the stderr output ([#5588](https://github.com/deltachat/deltachat-core-rust/pull/5588))
- @deltachat/stdio-rpc-server: fix `convert_platform.py`: 32bit `i32` -> `ia32` ([#5589](https://github.com/deltachat/deltachat-core-rust/pull/5589))
- @deltachat/stdio-rpc-server: fix example ([#5580](https://github.com/deltachat/deltachat-core-rust/pull/5580))
### API-Changes
- deltachat-jsonrpc: Return vcard contact directly in MessageObject.
- deltachat-jsonrpc: Add api `migrate_account` and `get_blob_dir` ([#5584](https://github.com/deltachat/deltachat-core-rust/pull/5584)).
- deltachat-rpc-client: Add ViewType.VCARD constant.
- deltachat-rpc-client: Add Contact.make_vcard().
- deltachat-rpc-client: Add Chat.send_contact().
### CI
- Publish @deltachat/jsonrpc-client directly to npm.
- Check that constants are always up-to-date.
### Build system
- nix: Add git-cliff to flake.
- nix: Use rust-analyzer nightly
### Miscellaneous Tasks
- cargo: Downgrade libc from 0.2.154 to 0.2.153.
### Tests
- deltachat-rpc-client: Test sending vCard.
## [1.138.5] - 2024-05-16
### API-Changes
- jsonrpc: Add parse_vcard() ([#5202](https://github.com/deltachat/deltachat-core-rust/pull/5202)).
- Add Viewtype::Vcard ([#5202](https://github.com/deltachat/deltachat-core-rust/pull/5202)).
- Add make_vcard() ([#5203](https://github.com/deltachat/deltachat-core-rust/pull/5203)).
### Build system
- Add repository URL to deltachat-rpc-server packages.
### Fixes
- Parsing vCards with avatars exported by Android's "Contacts" app.
### Miscellaneous Tasks
- Rebuild node constants.
### Refactor
- contact-tools: VcardContact: rename display_name to authname.
- VcardContact: Change timestamp type to i64.
## [1.138.4] - 2024-05-15
### CI
- Run actions/setup-node before npm publish.
## [1.138.3] - 2024-05-15
### CI
- Give CI job permission to publish binaries to the release.
## [1.138.2] - 2024-05-15
### API-Changes
- deltachat-rpc-client: Add CONFIG_SYNCED constant.
### CI
- Add npm token to publish deltachat-rpc-server packages.
### Features / Changes
- Reset more settings when configuring a chatmail account.
### Tests
- Set configuration after configure() finishes.
## [1.138.1] - 2024-05-14
### Features / Changes
- Detect XCHATMAIL capability and expose it as `is_chatmail` config.
### Fixes
- Never treat message with Chat-Group-ID as a private reply.
- Always prefer Chat-Group-ID over In-Reply-To and References.
- Ignore parent message if message references itself.
### CI
- Set RUSTUP_WINDOWS_PATH_ADD_BIN to work around `nextest` issue <https://github.com/nextest-rs/nextest/issues/1493>.
- deltachat-rpc-server: Fix upload of npm packages to github releases ([#5564](https://github.com/deltachat/deltachat-core-rust/pull/5564)).
### Refactor
- Add MimeMessage.get_chat_group_id().
- Make MimeMessage.get_header() return Option<&str>.
- sql: Make open flags immutable.
- Resultify token::lookup_or_new().
### Miscellaneous Tasks
- cargo: Bump parking_lot from 0.12.1 to 0.12.2.
- cargo: Bump libc from 0.2.153 to 0.2.154.
- cargo: Bump hickory-resolver from 0.24.0 to 0.24.1.
- cargo: Bump serde_json from 1.0.115 to 1.0.116.
- cargo: Bump human-panic from 1.2.3 to 2.0.0.
- cargo: Bump brotli from 5.0.0 to 6.0.0.
## [1.138.0] - 2024-05-13
### API-Changes
- Add dc_msg_save_file() which saves file copy at the provided path ([#4309](https://github.com/deltachat/deltachat-core-rust/pull/4309)).
- Api!(jsonrpc): replace EphemeralTimer tag "variant" with "kind"
### CI
- Use rsync instead of 3rd party github action.
- Replace `black` with `ruff format`.
- Update Rust to 1.78.0.
### Documentation
- Fix references in Message.set_location() documentation.
- Remove Doxygen markup from Message.has_location().
- Add `location` module documentation.
### Features / Changes
- Delete expired path locations in ephemeral loop.
- Delete orphaned POI locations during housekeeping.
- Parsing vCards for contacts sharing ([#5482](https://github.com/deltachat/deltachat-core-rust/pull/5482)).
- contact-tools: Support parsing profile images from "PHOTO:data:image/jpeg;base64,...".
- contact-tools: Add make_vcard().
- Do not add location markers to messages with non-POI location.
- Make one-to-one chats read-only the first seconds of a SecureJoin ([#5512](https://github.com/deltachat/deltachat-core-rust/pull/5512)).
### Fixes
- Message::set_file_from_bytes(): Set Param::Filename.
- Do not fail to send encrypted quotes to unencrypted chats.
- Never prepend subject to message text when bot receives it.
- Interrupt location loop when new location is stored.
- Correct message viewtype before recoding image blob ([#5496](https://github.com/deltachat/deltachat-core-rust/pull/5496)).
- Delete POI location when disappearing message expires.
- Delete non-POI locations after `delete_device_after`, not immediately.
- Update special chats icons even if they are blocked ([#5509](https://github.com/deltachat/deltachat-core-rust/pull/5509)).
- Use ChatIdBlocked::lookup_by_contact() instead of ChatId's method when applicable.
### Miscellaneous Tasks
- cargo: Bump quote from 1.0.35 to 1.0.36.
- cargo: Bump base64 from 0.22.0 to 0.22.1.
- cargo: Bump serde from 1.0.197 to 1.0.200.
- cargo: Bump async-channel from 2.2.0 to 2.2.1.
- cargo: Bump thiserror from 1.0.58 to 1.0.59.
- cargo: Bump anyhow from 1.0.81 to 1.0.82.
- cargo: Bump chrono from 0.4.37 to 0.4.38.
- cargo: Bump imap-proto from 0.16.4 to 0.16.5.
- cargo: Bump syn from 2.0.57 to 2.0.60.
- cargo: Bump mailparse from 0.14.1 to 0.15.0.
- cargo: Bump schemars from 0.8.16 to 0.8.19.
### Other
- Build ts docs with ci + nix.
- Push docs to delta.chat instead of codespeak
- Implement jsonrpc-docs build in github action
- Rm unneeded rust install from ts docs ci
- Correct folder for js.jsonrpc docs
- Add npm install to upload-docs.yml
- Add : to upload-docs.yml
- Upload-docs npm run => npm run build
- Rm leading slash
- Rm npm install
- Merge pull request #5515 from deltachat/dependabot/cargo/quote-1.0.36
- Merge pull request #5522 from deltachat/dependabot/cargo/chrono-0.4.38
- Merge pull request #5523 from deltachat/dependabot/cargo/mailparse-0.15.0
- Add webxdc internal integration commands in jsonrpc ([#5541](https://github.com/deltachat/deltachat-core-rust/pull/5541))
- Limit quote replies ([#5543](https://github.com/deltachat/deltachat-core-rust/pull/5543))
- Stdio jsonrpc server npm package ([#5332](https://github.com/deltachat/deltachat-core-rust/pull/5332))
### Refactor
- python: Fix ruff 0.4.2 warnings.
- Move `delete_poi_location` to location module and document it.
- Remove allow_keychange.
### Tests
- Explain test_was_seen_recently false-positive and give workaround instructions ([#5474](https://github.com/deltachat/deltachat-core-rust/pull/5474)).
- Test that member is added even if "Member added" is lost.
- Test that POIs are deleted when ephemeral message expires.
- Test ts build on branch
## [1.137.4] - 2024-04-24
### API-Changes
- [**breaking**] Remove `Stream` implementation for `EventEmitter`.
- Experimental Webxdc Integration API, Maps Integration ([#5461](https://github.com/deltachat/deltachat-core-rust/pull/5461)).
### Features / Changes
- Add progressive backoff for failing IMAP connection attempts ([#5443](https://github.com/deltachat/deltachat-core-rust/pull/5443)).
- Replace event channel with broadcast channel.
- Mark contact request messages as seen on IMAP.
### Fixes
- Convert images to RGB8 (without alpha) before encoding into JPEG to fix sending of large RGBA images.
- Don't set `is_bot` for webxdc status updates ([#5445](https://github.com/deltachat/deltachat-core-rust/pull/5445)).
- Do not fail if Autocrypt Setup Message has no encryption preference to fix key transfer from K-9 Mail to Delta Chat.
- Use only CRLF in Autocrypt Setup Message.
- python: Use cached message object if `dc_get_msg()` returns `NULL`.
- python: `Message::is_outgoing`: Don't reload message from db.
- python: `_map_ffi_event`: Always check if `get_message_by_id()` returned None.
- node: Undefine `NAPI_EXPERIMENTAL` to fix build with new clang.
### Build system
- nix: Add `imap-tools` as `deltachat-rpc-client` dependency.
- nix: Add `./deltachat-contact-tools` to sources.
- nix: Update nix flake.
- deps: Update rustls to 0.21.11.
### Documentation
- Update references to SecureJoin protocols.
- Fix broken references in documentation comments.
### Refactor
- imap: remove `RwLock` from `ratelimit`.
- deltachat-ffi: Remove unused `ResultNullableExt`.
- Remove duplicate clippy exceptions.
- Group `use` at the top of the test modules.
## [1.137.3] - 2024-04-16
### API-Changes
@@ -3939,3 +4212,13 @@ https://github.com/deltachat/deltachat-core-rust/pulls?q=is%3Apr+is%3Aclosed
[1.137.1]: https://github.com/deltachat/deltachat-core-rust/compare/v1.137.0...v1.137.1
[1.137.2]: https://github.com/deltachat/deltachat-core-rust/compare/v1.137.1...v1.137.2
[1.137.3]: https://github.com/deltachat/deltachat-core-rust/compare/v1.137.2...v1.137.3
[1.137.4]: https://github.com/deltachat/deltachat-core-rust/compare/v1.137.3...v1.137.4
[1.138.0]: https://github.com/deltachat/deltachat-core-rust/compare/v1.137.4...v1.138.0
[1.138.1]: https://github.com/deltachat/deltachat-core-rust/compare/v1.138.0...v1.138.1
[1.138.2]: https://github.com/deltachat/deltachat-core-rust/compare/v1.138.1...v1.138.2
[1.138.3]: https://github.com/deltachat/deltachat-core-rust/compare/v1.138.2...v1.138.3
[1.138.4]: https://github.com/deltachat/deltachat-core-rust/compare/v1.138.3...v1.138.4
[1.138.5]: https://github.com/deltachat/deltachat-core-rust/compare/v1.138.4...v1.138.5
[1.139.0]: https://github.com/deltachat/deltachat-core-rust/compare/v1.138.5...v1.139.0
[1.139.1]: https://github.com/deltachat/deltachat-core-rust/compare/v1.139.0...v1.139.1
[1.139.1]: https://github.com/deltachat/deltachat-core-rust/compare/v1.139.1...v1.139.2
Generated
+1896 -210
View File
File diff suppressed because it is too large Load Diff
+13 -11
View File
@@ -1,6 +1,6 @@
[package]
name = "deltachat"
version = "1.137.3"
version = "1.139.2"
edition = "2021"
license = "MPL-2.0"
rust-version = "1.77"
@@ -40,15 +40,15 @@ ratelimit = { path = "./deltachat-ratelimit" }
anyhow = { workspace = true }
async-broadcast = "0.7.0"
async-channel = "2.0.0"
async-channel = "2.2.1"
async-imap = { version = "0.9.7", default-features = false, features = ["runtime-tokio"] }
async-native-tls = { version = "0.5", default-features = false, features = ["runtime-tokio"] }
async-smtp = { version = "0.9", default-features = false, features = ["runtime-tokio"] }
async_zip = { version = "0.0.12", default-features = false, features = ["deflate", "fs"] }
backtrace = "0.3"
base64 = "0.22"
brotli = { version = "5", default-features=false, features = ["std"] }
chrono = { version = "0.4.37", default-features=false, features = ["clock", "std"] }
brotli = { version = "6", default-features=false, features = ["std"] }
chrono = { workspace = true }
email = { git = "https://github.com/deltachat/rust-email", branch = "master" }
encoded-words = { git = "https://github.com/async-email/encoded-words", branch = "master" }
escaper = "0.1"
@@ -60,11 +60,14 @@ hex = "0.4.0"
hickory-resolver = "0.24"
humansize = "2"
image = { version = "0.25.1", default-features=false, features = ["gif", "jpeg", "ico", "png", "pnm", "webp", "bmp"] }
iroh = { version = "0.4.2", default-features = false }
iroh_old = { version = "0.4.2", default-features = false, package = "iroh"}
iroh-net = "0.16.2"
iroh-gossip = { version = "0.16.2", features = ["net"] }
quinn = "0.10.0"
kamadak-exif = "0.5.3"
lettre_email = { git = "https://github.com/deltachat/lettre", branch = "master" }
libc = "0.2"
mailparse = "0.14"
mailparse = "0.15"
mime = "0.3.17"
num_cpus = "1.16"
num-derive = "0.4"
@@ -73,13 +76,12 @@ once_cell = { workspace = true }
percent-encoding = "2.3"
parking_lot = "0.12"
pgp = { version = "0.11", default-features = false }
pretty_env_logger = { version = "0.5", optional = true }
qrcodegen = "1.7.0"
quick-xml = "0.31"
quoted_printable = "0.5"
rand = "0.8"
regex = { workspace = true }
reqwest = { version = "0.12.2", features = ["json"] }
reqwest = { version = "0.11.27", features = ["json"] }
rusqlite = { workspace = true, features = ["sqlcipher"] }
rust-hsluv = "0.1"
sanitize-filename = "0.5"
@@ -116,7 +118,6 @@ anyhow = { version = "1", features = ["backtrace"] } # Enable `backtrace` featur
criterion = { version = "0.5.1", features = ["async_tokio"] }
futures-lite = "2.3.0"
log = "0.4"
pretty_env_logger = "0.5"
proptest = { version = "1", default-features = false, features = ["std"] }
tempfile = "3"
testdir = "0.9.0"
@@ -168,7 +169,8 @@ harness = false
anyhow = "1"
once_cell = "1.18.0"
regex = "1.10"
rusqlite = { version = "0.31" }
rusqlite = "0.31"
chrono = { version = "0.4.38", default-features=false, features = ["alloc", "clock", "std"] }
[features]
default = ["vendored"]
@@ -177,4 +179,4 @@ vendored = [
"async-native-tls/vendored",
"rusqlite/bundled-sqlcipher-vendored-openssl",
"reqwest/native-tls-vendored"
]
]
+7 -10
View File
@@ -4,21 +4,18 @@ For example, to release version 1.116.0 of the core, do the following steps.
1. Resolve all [blocker issues](https://github.com/deltachat/deltachat-core-rust/labels/blocker).
2. Run `npm run build:core:constants` in the root of the repository
and commit generated `node/constants.js`, `node/events.js` and `node/lib/constants.js`.
2. Update the changelog: `git cliff --unreleased --tag 1.116.0 --prepend CHANGELOG.md` or `git cliff -u -t 1.116.0 -p CHANGELOG.md`.
3. Update the changelog: `git cliff --unreleased --tag 1.116.0 --prepend CHANGELOG.md` or `git cliff -u -t 1.116.0 -p CHANGELOG.md`.
4. add a link to compare previous with current version to the end of CHANGELOG.md:
3. add a link to compare previous with current version to the end of CHANGELOG.md:
`[1.116.0]: https://github.com/deltachat/deltachat-core-rust/compare/v1.115.2...v1.116.0`
5. Update the version by running `scripts/set_core_version.py 1.116.0`.
4. Update the version by running `scripts/set_core_version.py 1.116.0`.
6. Commit the changes as `chore(release): prepare for 1.116.0`.
5. Commit the changes as `chore(release): prepare for 1.116.0`.
Optionally, use a separate branch like `prep-1.116.0` for this commit and open a PR for review.
7. Tag the release: `git tag -a v1.116.0`.
6. Tag the release: `git tag -a v1.116.0`.
8. Push the release tag: `git push origin v1.116.0`.
7. Push the release tag: `git push origin v1.116.0`.
9. Create a GitHub release: `gh release create v1.116.0 -n ''`.
8. Create a GitHub release: `gh release create v1.116.0 -n ''`.
+1
View File
@@ -1,3 +1,4 @@
#![recursion_limit = "256"]
use criterion::{black_box, criterion_group, criterion_main, Criterion};
use deltachat::contact::Contact;
use deltachat::context::Context;
+1
View File
@@ -1,3 +1,4 @@
#![recursion_limit = "256"]
use std::path::PathBuf;
use criterion::{black_box, criterion_group, criterion_main, Criterion};
+1
View File
@@ -1,3 +1,4 @@
#![recursion_limit = "256"]
use std::path::Path;
use criterion::{black_box, criterion_group, criterion_main, Criterion};
+1
View File
@@ -1,3 +1,4 @@
#![recursion_limit = "256"]
use std::path::Path;
use criterion::{black_box, criterion_group, criterion_main, Criterion};
+1
View File
@@ -1,3 +1,4 @@
#![recursion_limit = "256"]
use std::path::PathBuf;
use criterion::{black_box, criterion_group, criterion_main, Criterion};
+1
View File
@@ -1,3 +1,4 @@
#![recursion_limit = "256"]
use std::path::Path;
use criterion::{black_box, criterion_group, criterion_main, Criterion};
+1
View File
@@ -1,3 +1,4 @@
#![recursion_limit = "256"]
use criterion::{criterion_group, criterion_main, Criterion};
use deltachat::context::Context;
+3 -3
View File
@@ -1,10 +1,9 @@
[package]
name = "deltachat-contact-tools"
version = "0.1.0"
version = "0.0.0" # No semver-stable versioning
edition = "2021"
description = "Contact-related tools, like parsing vcards and sanitizing name and address"
description = "Contact-related tools, like parsing vcards and sanitizing name and address. Meant for internal use in the deltachat crate."
license = "MPL-2.0"
# TODO maybe it should be called "deltachat-text-utils" or similar?
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
@@ -13,6 +12,7 @@ anyhow = { workspace = true }
once_cell = { workspace = true }
regex = { workspace = true }
rusqlite = { workspace = true } # Needed in order to `impl rusqlite::types::ToSql for EmailAddress`. Could easily be put behind a feature.
chrono = { workspace = true }
[dev-dependencies]
anyhow = { workspace = true, features = ["backtrace"] } # Enable `backtrace` feature in tests.
+395 -7
View File
@@ -29,10 +29,195 @@ use std::fmt;
use std::ops::Deref;
use anyhow::bail;
use anyhow::Context as _;
use anyhow::Result;
use chrono::{DateTime, NaiveDateTime};
use once_cell::sync::Lazy;
use regex::Regex;
// TODOs to clean up:
// - Check if sanitizing is done correctly everywhere
// - Apply lints everywhere (https://doc.rust-lang.org/cargo/reference/workspaces.html#the-lints-table)
#[derive(Debug)]
/// A Contact, as represented in a VCard.
pub struct VcardContact {
/// The email address, vcard property `email`
pub addr: String,
/// This must be the name authorized by the contact itself, not a locally given name. Vcard
/// property `fn`. Can be empty, one should use `display_name()` to obtain the display name.
pub authname: String,
/// The contact's public PGP key in Base64, vcard property `key`
pub key: Option<String>,
/// The contact's profile image (=avatar) in Base64, vcard property `photo`
pub profile_image: Option<String>,
/// The timestamp when the vcard was created / last updated, vcard property `rev`
pub timestamp: Result<i64>,
}
impl VcardContact {
/// Returns the contact's display name.
pub fn display_name(&self) -> &str {
match self.authname.is_empty() {
false => &self.authname,
true => &self.addr,
}
}
}
/// Returns a vCard containing given contacts.
///
/// Calling [`parse_vcard()`] on the returned result is a reverse operation.
pub fn make_vcard(contacts: &[VcardContact]) -> String {
fn format_timestamp(c: &VcardContact) -> Option<String> {
let timestamp = *c.timestamp.as_ref().ok()?;
let datetime = DateTime::from_timestamp(timestamp, 0)?;
Some(datetime.format("%Y%m%dT%H%M%SZ").to_string())
}
let mut res = "".to_string();
for c in contacts {
let addr = &c.addr;
let display_name = c.display_name();
res += &format!(
"BEGIN:VCARD\n\
VERSION:4.0\n\
EMAIL:{addr}\n\
FN:{display_name}\n"
);
if let Some(key) = &c.key {
res += &format!("KEY:data:application/pgp-keys;base64,{key}\n");
}
if let Some(profile_image) = &c.profile_image {
res += &format!("PHOTO:data:image/jpeg;base64,{profile_image}\n");
}
if let Some(timestamp) = format_timestamp(c) {
res += &format!("REV:{timestamp}\n");
}
res += "END:VCARD\n";
}
res
}
/// Parses `VcardContact`s from a given `&str`.
pub fn parse_vcard(vcard: &str) -> Vec<VcardContact> {
fn remove_prefix<'a>(s: &'a str, prefix: &str) -> Option<&'a str> {
let start_of_s = s.get(..prefix.len())?;
if start_of_s.eq_ignore_ascii_case(prefix) {
s.get(prefix.len()..)
} else {
None
}
}
fn vcard_property<'a>(s: &'a str, property: &str) -> Option<&'a str> {
let remainder = remove_prefix(s, property)?;
// If `s` is `EMAIL;TYPE=work:alice@example.com` and `property` is `EMAIL`,
// then `remainder` is now `;TYPE=work:alice@example.com`
// TODO this doesn't handle the case where there are quotes around a colon
let (params, value) = remainder.split_once(':')?;
// In the example from above, `params` is now `;TYPE=work`
// and `value` is now `alice@example.com`
if params
.chars()
.next()
.filter(|c| !c.is_ascii_punctuation() || *c == '_')
.is_some()
{
// `s` started with `property`, but the next character after it was not punctuation,
// so this line's property is actually something else
return None;
}
Some(value)
}
fn parse_datetime(datetime: &str) -> Result<i64> {
// According to https://www.rfc-editor.org/rfc/rfc6350#section-4.3.5, the timestamp
// is in ISO.8601.2004 format. DateTime::parse_from_rfc3339() apparently parses
// ISO.8601, but fails to parse any of the examples given.
// So, instead just parse using a format string.
// Parses 19961022T140000Z, 19961022T140000-05, or 19961022T140000-0500.
let timestamp = match DateTime::parse_from_str(datetime, "%Y%m%dT%H%M%S%#z") {
Ok(datetime) => datetime.timestamp(),
// Parses 19961022T140000.
Err(e) => match NaiveDateTime::parse_from_str(datetime, "%Y%m%dT%H%M%S") {
Ok(datetime) => datetime
.and_local_timezone(chrono::offset::Local)
.single()
.context("Could not apply local timezone to parsed date and time")?
.timestamp(),
Err(_) => return Err(e.into()),
},
};
Ok(timestamp)
}
// Remove line folding, see https://datatracker.ietf.org/doc/html/rfc6350#section-3.2
static NEWLINE_AND_SPACE_OR_TAB: Lazy<Regex> = Lazy::new(|| Regex::new("\n[\t ]").unwrap());
let unfolded_lines = NEWLINE_AND_SPACE_OR_TAB.replace_all(vcard, "");
let mut lines = unfolded_lines.lines().peekable();
let mut contacts = Vec::new();
while lines.peek().is_some() {
// Skip to the start of the vcard:
for line in lines.by_ref() {
if line.eq_ignore_ascii_case("BEGIN:VCARD") {
break;
}
}
let mut display_name = None;
let mut addr = None;
let mut key = None;
let mut photo = None;
let mut datetime = None;
for line in lines.by_ref() {
if let Some(email) = vcard_property(line, "email") {
addr.get_or_insert(email);
} else if let Some(name) = vcard_property(line, "fn") {
display_name.get_or_insert(name);
} else if let Some(k) = remove_prefix(line, "KEY;PGP;ENCODING=BASE64:")
.or_else(|| remove_prefix(line, "KEY;TYPE=PGP;ENCODING=b:"))
.or_else(|| remove_prefix(line, "KEY:data:application/pgp-keys;base64,"))
{
key.get_or_insert(k);
} else if let Some(p) = remove_prefix(line, "PHOTO;JPEG;ENCODING=BASE64:")
.or_else(|| remove_prefix(line, "PHOTO;ENCODING=BASE64;JPEG:"))
.or_else(|| remove_prefix(line, "PHOTO;TYPE=JPEG;ENCODING=b:"))
.or_else(|| remove_prefix(line, "PHOTO;ENCODING=b;TYPE=JPEG:"))
.or_else(|| remove_prefix(line, "PHOTO;ENCODING=BASE64;TYPE=JPEG:"))
.or_else(|| remove_prefix(line, "PHOTO;TYPE=JPEG;ENCODING=BASE64:"))
.or_else(|| remove_prefix(line, "PHOTO:data:image/jpeg;base64,"))
{
photo.get_or_insert(p);
} else if let Some(rev) = vcard_property(line, "rev") {
datetime.get_or_insert(rev);
} else if line.eq_ignore_ascii_case("END:VCARD") {
break;
}
}
let (authname, addr) =
sanitize_name_and_addr(display_name.unwrap_or(""), addr.unwrap_or(""));
contacts.push(VcardContact {
authname,
addr,
key: key.map(|s| s.to_string()),
profile_image: photo.map(|s| s.to_string()),
timestamp: datetime
.context("No timestamp in vcard")
.and_then(parse_datetime),
});
}
contacts
}
/// Valid contact address.
#[derive(Debug, Clone)]
pub struct ContactAddress(String);
@@ -81,14 +266,10 @@ impl rusqlite::types::ToSql for ContactAddress {
/// Make the name and address
pub fn sanitize_name_and_addr(name: &str, addr: &str) -> (String, String) {
static ADDR_WITH_NAME_REGEX: Lazy<Regex> = Lazy::new(|| Regex::new("(.*)<(.*)>").unwrap());
if let Some(captures) = ADDR_WITH_NAME_REGEX.captures(addr.as_ref()) {
let (name, addr) = if let Some(captures) = ADDR_WITH_NAME_REGEX.captures(addr.as_ref()) {
(
if name.is_empty() {
strip_rtlo_characters(
&captures
.get(1)
.map_or("".to_string(), |m| normalize_name(m.as_str())),
)
strip_rtlo_characters(captures.get(1).map_or("", |m| m.as_str()))
} else {
strip_rtlo_characters(name)
},
@@ -97,8 +278,21 @@ pub fn sanitize_name_and_addr(name: &str, addr: &str) -> (String, String) {
.map_or("".to_string(), |m| m.as_str().to_string()),
)
} else {
(strip_rtlo_characters(name), addr.to_string())
(
strip_rtlo_characters(&normalize_name(name)),
addr.to_string(),
)
};
let mut name = normalize_name(&name);
// If the 'display name' is just the address, remove it:
// Otherwise, the contact would sometimes be shown as "alice@example.com (alice@example.com)" (see `get_name_n_addr()`).
// If the display name is empty, DC will just show the address when it needs a display name.
if name == addr {
name = "".to_string();
}
(name, addr)
}
/// Normalize a name.
@@ -230,8 +424,124 @@ impl rusqlite::types::ToSql for EmailAddress {
#[cfg(test)]
mod tests {
use chrono::TimeZone;
use super::*;
#[test]
fn test_vcard_thunderbird() {
let contacts = parse_vcard(
"BEGIN:VCARD
VERSION:4.0
FN:'Alice Mueller'
EMAIL;PREF=1:alice.mueller@posteo.de
UID:a8083264-ca47-4be7-98a8-8ec3db1447ca
END:VCARD
BEGIN:VCARD
VERSION:4.0
FN:'bobzzz@freenet.de'
EMAIL;PREF=1:bobzzz@freenet.de
UID:cac4fef4-6351-4854-bbe4-9b6df857eaed
END:VCARD
",
);
assert_eq!(contacts[0].addr, "alice.mueller@posteo.de".to_string());
assert_eq!(contacts[0].authname, "Alice Mueller".to_string());
assert_eq!(contacts[0].key, None);
assert_eq!(contacts[0].profile_image, None);
assert!(contacts[0].timestamp.is_err());
assert_eq!(contacts[1].addr, "bobzzz@freenet.de".to_string());
assert_eq!(contacts[1].authname, "".to_string());
assert_eq!(contacts[1].key, None);
assert_eq!(contacts[1].profile_image, None);
assert!(contacts[1].timestamp.is_err());
assert_eq!(contacts.len(), 2);
}
#[test]
fn test_vcard_simple_example() {
let contacts = parse_vcard(
"BEGIN:VCARD
VERSION:4.0
FN:Alice Wonderland
N:Wonderland;Alice;;;Ms.
GENDER:W
EMAIL;TYPE=work:alice@example.com
KEY;TYPE=PGP;ENCODING=b:[base64-data]
REV:20240418T184242Z
END:VCARD",
);
assert_eq!(contacts[0].addr, "alice@example.com".to_string());
assert_eq!(contacts[0].authname, "Alice Wonderland".to_string());
assert_eq!(contacts[0].key, Some("[base64-data]".to_string()));
assert_eq!(contacts[0].profile_image, None);
assert_eq!(*contacts[0].timestamp.as_ref().unwrap(), 1713465762);
assert_eq!(contacts.len(), 1);
}
#[test]
fn test_make_and_parse_vcard() {
let contacts = [
VcardContact {
addr: "alice@example.org".to_string(),
authname: "Alice Wonderland".to_string(),
key: Some("[base64-data]".to_string()),
profile_image: Some("image in Base64".to_string()),
timestamp: Ok(1713465762),
},
VcardContact {
addr: "bob@example.com".to_string(),
authname: "".to_string(),
key: None,
profile_image: None,
timestamp: Ok(0),
},
];
let items = [
"BEGIN:VCARD\n\
VERSION:4.0\n\
EMAIL:alice@example.org\n\
FN:Alice Wonderland\n\
KEY:data:application/pgp-keys;base64,[base64-data]\n\
PHOTO:data:image/jpeg;base64,image in Base64\n\
REV:20240418T184242Z\n\
END:VCARD\n",
"BEGIN:VCARD\n\
VERSION:4.0\n\
EMAIL:bob@example.com\n\
FN:bob@example.com\n\
REV:19700101T000000Z\n\
END:VCARD\n",
];
let mut expected = "".to_string();
for len in 0..=contacts.len() {
let contacts = &contacts[0..len];
let vcard = make_vcard(contacts);
if len > 0 {
expected += items[len - 1];
}
assert_eq!(vcard, expected);
let parsed = parse_vcard(&vcard);
assert_eq!(parsed.len(), contacts.len());
for i in 0..parsed.len() {
assert_eq!(parsed[i].addr, contacts[i].addr);
assert_eq!(parsed[i].authname, contacts[i].authname);
assert_eq!(parsed[i].key, contacts[i].key);
assert_eq!(parsed[i].profile_image, contacts[i].profile_image);
assert_eq!(
parsed[i].timestamp.as_ref().unwrap(),
contacts[i].timestamp.as_ref().unwrap()
);
}
}
}
#[test]
fn test_contact_address() -> Result<()> {
let alice_addr = "alice@example.org";
@@ -277,4 +587,82 @@ mod tests {
assert!(EmailAddress::new("u@tt").is_ok());
assert_eq!(EmailAddress::new("@d.tt").is_ok(), false);
}
#[test]
fn test_vcard_android() {
let contacts = parse_vcard(
"BEGIN:VCARD
VERSION:2.1
N:;Bob;;;
FN:Bob
TEL;CELL:+1-234-567-890
EMAIL;HOME:bob@example.org
END:VCARD
BEGIN:VCARD
VERSION:2.1
N:;Alice;;;
FN:Alice
EMAIL;HOME:alice@example.org
END:VCARD
",
);
assert_eq!(contacts[0].addr, "bob@example.org".to_string());
assert_eq!(contacts[0].authname, "Bob".to_string());
assert_eq!(contacts[0].key, None);
assert_eq!(contacts[0].profile_image, None);
assert_eq!(contacts[1].addr, "alice@example.org".to_string());
assert_eq!(contacts[1].authname, "Alice".to_string());
assert_eq!(contacts[1].key, None);
assert_eq!(contacts[1].profile_image, None);
assert_eq!(contacts.len(), 2);
}
#[test]
fn test_vcard_local_datetime() {
let contacts = parse_vcard(
"BEGIN:VCARD\n\
VERSION:4.0\n\
FN:Alice Wonderland\n\
EMAIL;TYPE=work:alice@example.org\n\
REV:20240418T184242\n\
END:VCARD",
);
assert_eq!(contacts.len(), 1);
assert_eq!(contacts[0].addr, "alice@example.org".to_string());
assert_eq!(contacts[0].authname, "Alice Wonderland".to_string());
assert_eq!(
*contacts[0].timestamp.as_ref().unwrap(),
chrono::offset::Local
.with_ymd_and_hms(2024, 4, 18, 18, 42, 42)
.unwrap()
.timestamp()
);
}
#[test]
fn test_android_vcard_with_base64_avatar() {
// This is not an actual base64-encoded avatar, it's just to test the parsing
let contacts = parse_vcard(
"BEGIN:VCARD
VERSION:2.1
N:;Bob;;;
FN:Bob
EMAIL;HOME:bob@example.org
PHOTO;ENCODING=BASE64;JPEG:/9j/4AAQSkZJRgABAQAAAQABAAD/4gIoSUNDX1BST0ZJTEU
AAQEAAAIYAAAAAAQwAABtbnRyUkdCIFhZWiAAAAAAAAAAAAAAAABhY3NwAAAAAAAAAAAAAAAA
L8bRuAJYoZUYrI4ZY3VWwxw4Ay28AAGBISScmf/2Q==
END:VCARD
",
);
assert_eq!(contacts.len(), 1);
assert_eq!(contacts[0].addr, "bob@example.org".to_string());
assert_eq!(contacts[0].authname, "Bob".to_string());
assert_eq!(contacts[0].key, None);
assert_eq!(contacts[0].profile_image.as_deref().unwrap(), "/9j/4AAQSkZJRgABAQAAAQABAAD/4gIoSUNDX1BST0ZJTEUAAQEAAAIYAAAAAAQwAABtbnRyUkdCIFhZWiAAAAAAAAAAAAAAAABhY3NwAAAAAAAAAAAAAAAAL8bRuAJYoZUYrI4ZY3VWwxw4Ay28AAGBISScmf/2Q==");
}
}
+2 -2
View File
@@ -1,6 +1,6 @@
[package]
name = "deltachat_ffi"
version = "1.137.3"
version = "1.139.2"
description = "Deltachat FFI"
edition = "2018"
readme = "README.md"
@@ -17,7 +17,7 @@ crate-type = ["cdylib", "staticlib"]
deltachat = { path = "../", default-features = false }
deltachat-jsonrpc = { path = "../deltachat-jsonrpc", optional = true }
libc = "0.2"
human-panic = { version = "1", default-features = false }
human-panic = { version = "2", default-features = false }
num-traits = "0.2"
serde_json = "1.0"
tokio = { version = "1.37.0", features = ["rt-multi-thread"] }
+35 -3
View File
@@ -517,6 +517,7 @@ char* dc_get_blobdir (const dc_context_t* context);
* 0=Nothing else happens when the key changes.
* 1=After the key changed, `dc_chat_can_send()` returns false and `dc_chat_is_protection_broken()` returns true
* until `dc_accept_chat()` is called.
* - `is_chatmail` = 1 if the the server is a chatmail server, 0 otherwise.
* - `ui.*` = All keys prefixed by `ui.` can be used by the user-interfaces for system-specific purposes.
* The prefix should be followed by the system and maybe subsystem,
* e.g. `ui.desktop.foo`, `ui.desktop.linux.bar`, `ui.android.foo`, `ui.dc40.bar`, `ui.bot.simplebot.baz`.
@@ -4105,6 +4106,19 @@ char* dc_msg_get_subject (const dc_msg_t* msg);
char* dc_msg_get_file (const dc_msg_t* msg);
/**
* Save file copy at the user-provided path.
*
* Fails if file already exists at the provided path.
*
* @memberof dc_msg_t
* @param msg The message object.
* @param path Destination file path with filename and extension.
* @return 0 on failure, 1 on success.
*/
int dc_msg_save_file (const dc_msg_t* msg, const char* path);
/**
* Get an original attachment filename, with extension but without the path. To get the full path,
* use dc_msg_get_file().
@@ -4377,9 +4391,9 @@ int dc_msg_has_deviating_timestamp(const dc_msg_t* msg);
/**
* Check if a message has a location bound to it.
* These messages are also returned by dc_get_locations()
* and the UI may decide to display a special icon beside such messages,
* Check if a message has a POI location bound to it.
* These locations are also returned by dc_get_locations()
* The UI may decide to display a special icon beside such messages.
*
* @memberof dc_msg_t
* @param msg The message object.
@@ -5466,6 +5480,11 @@ int64_t dc_lot_get_timestamp (const dc_lot_t* lot);
*/
#define DC_MSG_WEBXDC 80
/**
* Message containing shared contacts represented as a vCard (virtual contact file)
* with email addresses and possibly other fields.
*/
#define DC_MSG_VCARD 90
/**
* @}
@@ -7314,6 +7333,19 @@ void dc_event_unref(dc_event_t* event);
/// Used in summaries.
#define DC_STR_REACTED_BY 177
/// "Establishing guaranteed end-to-end encryption, please wait…"
///
/// Used as info message.
#define DC_STR_SECUREJOIN_WAIT 190
/// "Could not yet establish guaranteed end-to-end encryption, but you may already send a message."
///
/// Used as info message.
#define DC_STR_SECUREJOIN_WAIT_TIMEOUT 191
/// "Contact"
#define DC_STR_CONTACT 200
/**
* @}
*/
+35 -2
View File
@@ -1,3 +1,4 @@
#![recursion_limit = "256"]
#![warn(unused, clippy::all)]
#![allow(
non_camel_case_types,
@@ -561,6 +562,7 @@ pub unsafe extern "C" fn dc_event_get_id(event: *mut dc_event_t) -> libc::c_int
EventType::ConfigSynced { .. } => 2111,
EventType::WebxdcStatusUpdate { .. } => 2120,
EventType::WebxdcInstanceDeleted { .. } => 2121,
EventType::WebxdcRealtimeData { .. } => 2150,
EventType::AccountsBackgroundFetchDone => 2200,
EventType::ChatlistChanged => 2300,
EventType::ChatlistItemChanged { .. } => 2301,
@@ -616,8 +618,9 @@ pub unsafe extern "C" fn dc_event_get_data1_int(event: *mut dc_event_t) -> libc:
| EventType::SecurejoinJoinerProgress { contact_id, .. } => {
contact_id.to_u32() as libc::c_int
}
EventType::WebxdcStatusUpdate { msg_id, .. } => msg_id.to_u32() as libc::c_int,
EventType::WebxdcInstanceDeleted { msg_id, .. } => msg_id.to_u32() as libc::c_int,
EventType::WebxdcRealtimeData { msg_id, .. }
| EventType::WebxdcStatusUpdate { msg_id, .. }
| EventType::WebxdcInstanceDeleted { msg_id, .. } => msg_id.to_u32() as libc::c_int,
EventType::ChatlistItemChanged { chat_id } => {
chat_id.unwrap_or_default().to_u32() as libc::c_int
}
@@ -655,6 +658,7 @@ pub unsafe extern "C" fn dc_event_get_data2_int(event: *mut dc_event_t) -> libc:
| EventType::ConnectivityChanged
| EventType::WebxdcInstanceDeleted { .. }
| EventType::IncomingMsgBunch { .. }
| EventType::WebxdcRealtimeData { .. }
| EventType::SelfavatarChanged
| EventType::AccountsBackgroundFetchDone
| EventType::ChatlistChanged
@@ -721,6 +725,7 @@ pub unsafe extern "C" fn dc_event_get_data2_str(event: *mut dc_event_t) -> *mut
| EventType::SelfavatarChanged
| EventType::WebxdcStatusUpdate { .. }
| EventType::WebxdcInstanceDeleted { .. }
| EventType::WebxdcRealtimeData { .. }
| EventType::AccountsBackgroundFetchDone
| EventType::ChatEphemeralTimerModified { .. }
| EventType::IncomingMsgBunch { .. }
@@ -3368,6 +3373,34 @@ pub unsafe extern "C" fn dc_msg_get_file(msg: *mut dc_msg_t) -> *mut libc::c_cha
.unwrap_or_else(|| "".strdup())
}
#[no_mangle]
pub unsafe extern "C" fn dc_msg_save_file(
msg: *mut dc_msg_t,
path: *const libc::c_char,
) -> libc::c_int {
if msg.is_null() || path.is_null() {
eprintln!("ignoring careless call to dc_msg_save_file()");
return 0;
}
let ffi_msg = &*msg;
let ctx = &*ffi_msg.context;
let path = to_string_lossy(path);
let r = block_on(
ffi_msg
.message
.save_file(ctx, &std::path::PathBuf::from(path)),
);
match r {
Ok(()) => 1,
Err(_) => {
r.context("Failed to save file from message")
.log_err(ctx)
.unwrap_or_default();
0
}
}
}
#[no_mangle]
pub unsafe extern "C" fn dc_msg_get_filename(msg: *mut dc_msg_t) -> *mut libc::c_char {
if msg.is_null() {
+4 -3
View File
@@ -1,6 +1,6 @@
[package]
name = "deltachat-jsonrpc"
version = "1.137.3"
version = "1.139.2"
description = "DeltaChat JSON-RPC API"
edition = "2021"
default-run = "deltachat-jsonrpc-server"
@@ -15,12 +15,13 @@ required-features = ["webserver"]
[dependencies]
anyhow = "1"
deltachat = { path = ".." }
deltachat-contact-tools = { path = "../deltachat-contact-tools" }
num-traits = "0.2"
schemars = "0.8.13"
schemars = "0.8.19"
serde = { version = "1.0", features = ["derive"] }
tempfile = "3.10.1"
log = "0.4"
async-channel = { version = "2.0.0" }
async-channel = { version = "2.2.1" }
futures = { version = "0.3.30" }
serde_json = "1"
yerpc = { version = "0.5.2", features = ["anyhow_expose", "openrpc"] }
+117 -15
View File
@@ -1,4 +1,6 @@
use std::collections::BTreeMap;
use std::path::Path;
use std::str;
use std::sync::Arc;
use std::time::Duration;
use std::{collections::HashMap, str::FromStr};
@@ -16,12 +18,14 @@ use deltachat::constants::DC_MSG_ID_DAYMARKER;
use deltachat::contact::{may_be_valid_addr, Contact, ContactId, Origin};
use deltachat::context::get_info;
use deltachat::ephemeral::Timer;
use deltachat::imex;
use deltachat::location;
use deltachat::message::get_msg_read_receipts;
use deltachat::message::{
self, delete_msgs, markseen_msgs, Message, MessageState, MsgId, Viewtype,
};
use deltachat::peer_channels::{
leave_webxdc_realtime, send_webxdc_realtime_advertisement, send_webxdc_realtime_data,
};
use deltachat::provider::get_provider_info;
use deltachat::qr::{self, Qr};
use deltachat::qr_code_generator::{generate_backup_qr, get_securejoin_qr_svg};
@@ -30,6 +34,7 @@ use deltachat::securejoin;
use deltachat::stock_str::StockMessage;
use deltachat::webxdc::StatusUpdateSerial;
use deltachat::EventEmitter;
use deltachat::{imex, info};
use sanitize_filename::is_sanitized;
use tokio::fs;
use tokio::sync::{watch, Mutex, RwLock};
@@ -41,7 +46,7 @@ pub mod types;
use num_traits::FromPrimitive;
use types::account::Account;
use types::chat::FullChat;
use types::contact::ContactObject;
use types::contact::{ContactObject, VcardContact};
use types::events::Event;
use types::http::HttpResponse;
use types::message::{MessageData, MessageObject, MessageReadReceipt};
@@ -183,6 +188,16 @@ impl CommandApi {
self.accounts.write().await.add_account().await
}
/// Imports/migrated an existing account from a database path into this account manager.
/// Returns the ID of new account.
async fn migrate_account(&self, path_to_db: String) -> Result<u32> {
self.accounts
.write()
.await
.migrate_account(std::path::PathBuf::from(path_to_db))
.await
}
async fn remove_account(&self, account_id: u32) -> Result<()> {
self.accounts
.write()
@@ -327,6 +342,11 @@ impl CommandApi {
ctx.get_info().await
}
async fn get_blob_dir(&self, account_id: u32) -> Result<Option<String>> {
let ctx = self.get_context(account_id).await?;
Ok(ctx.get_blobdir().to_str().map(|s| s.to_owned()))
}
async fn draft_self_report(&self, account_id: u32) -> Result<u32> {
let ctx = self.get_context(account_id).await?;
Ok(ctx.draft_self_report().await?.to_u32())
@@ -1425,6 +1445,23 @@ impl CommandApi {
Ok(contact_id.map(|id| id.to_u32()))
}
/// Parses a vCard file located at the given path. Returns contacts in their original order.
async fn parse_vcard(&self, path: String) -> Result<Vec<VcardContact>> {
let vcard = tokio::fs::read(Path::new(&path)).await?;
let vcard = str::from_utf8(&vcard)?;
Ok(deltachat_contact_tools::parse_vcard(vcard)
.into_iter()
.map(|c| c.into())
.collect())
}
/// Returns a vCard containing contacts with the given ids.
async fn make_vcard(&self, account_id: u32, contacts: Vec<u32>) -> Result<String> {
let ctx = self.get_context(account_id).await?;
let contacts: Vec<_> = contacts.iter().map(|&c| ContactId::new(c)).collect();
deltachat::contact::make_vcard(&ctx, &contacts).await
}
// ---------------------------------------------
// chat
// ---------------------------------------------
@@ -1729,6 +1766,37 @@ impl CommandApi {
.await
}
async fn send_webxdc_realtime_data(
&self,
account_id: u32,
instance_msg_id: u32,
data: Vec<u8>,
) -> Result<()> {
let ctx = self.get_context(account_id).await?;
send_webxdc_realtime_data(&ctx, MsgId::new(instance_msg_id), data).await
}
async fn send_webxdc_realtime_advertisement(
&self,
account_id: u32,
instance_msg_id: u32,
) -> Result<()> {
let ctx = self.get_context(account_id).await?;
let fut = send_webxdc_realtime_advertisement(&ctx, MsgId::new(instance_msg_id)).await?;
if let Some(fut) = fut {
tokio::spawn(async move {
fut.await.ok();
info!(ctx, "send_webxdc_realtime_advertisement done")
});
}
Ok(())
}
async fn leave_webxdc_realtime(&self, account_id: u32, instance_message_id: u32) -> Result<()> {
let ctx = self.get_context(account_id).await?;
leave_webxdc_realtime(&ctx, MsgId::new(instance_message_id)).await
}
async fn get_webxdc_status_updates(
&self,
account_id: u32,
@@ -1770,6 +1838,29 @@ impl CommandApi {
Ok(general_purpose::STANDARD_NO_PAD.encode(blob))
}
/// Sets Webxdc file as integration.
/// `file` is the .xdc to use as Webxdc integration.
async fn set_webxdc_integration(&self, account_id: u32, file_path: String) -> Result<()> {
let ctx = self.get_context(account_id).await?;
ctx.set_webxdc_integration(&file_path).await
}
/// Returns Webxdc instance used for optional integrations.
/// UI can open the Webxdc as usual.
/// Returns `None` if there is no integration; the caller can add one using `set_webxdc_integration` then.
/// `integrate_for` is the chat to get the integration for.
async fn init_webxdc_integration(
&self,
account_id: u32,
chat_id: Option<u32>,
) -> Result<Option<u32>> {
let ctx = self.get_context(account_id).await?;
Ok(ctx
.init_webxdc_integration(chat_id.map(ChatId::new))
.await?
.map(|msg_id| msg_id.to_u32()))
}
/// Makes an HTTP GET request and returns a response.
///
/// `url` is the HTTP or HTTPS URL.
@@ -1878,6 +1969,15 @@ impl CommandApi {
Ok(can_send)
}
/// Saves a file copy at the user-provided path.
///
/// Fails if file already exists at the provided path.
async fn save_msg_file(&self, account_id: u32, msg_id: u32, path: String) -> Result<()> {
let ctx = self.get_context(account_id).await?;
let message = Message::load_from_db(&ctx, MsgId::new(msg_id)).await?;
message.save_file(&ctx, Path::new(&path)).await
}
// ---------------------------------------------
// functions for the composer
// the composer is the message input field
@@ -1950,19 +2050,21 @@ impl CommandApi {
);
let destination_path = account_folder.join("stickers").join(collection);
fs::create_dir_all(&destination_path).await?;
let file = message.get_file(&ctx).context("no file")?;
fs::copy(
&file,
destination_path.join(format!(
"{}.{}",
msg_id,
file.extension()
.unwrap_or_default()
.to_str()
.unwrap_or_default()
)),
)
.await?;
let file = message.get_filename().context("no file?")?;
message
.save_file(
&ctx,
&destination_path.join(format!(
"{}.{}",
msg_id,
Path::new(&file)
.extension()
.unwrap_or_default()
.to_str()
.unwrap_or_default()
)),
)
.await?;
Ok(())
}
@@ -1,4 +1,5 @@
use anyhow::Result;
use deltachat::color;
use deltachat::context::Context;
use serde::Serialize;
use typescript_type_def::TypeDef;
@@ -87,3 +88,35 @@ impl ContactObject {
})
}
}
#[derive(Clone, Serialize, TypeDef, schemars::JsonSchema)]
#[serde(rename_all = "camelCase")]
pub struct VcardContact {
/// Email address.
addr: String,
/// The contact's name, or the email address if no name was given.
display_name: String,
/// Public PGP key in Base64.
key: Option<String>,
/// Profile image in Base64.
profile_image: Option<String>,
/// Contact color as hex string.
color: String,
/// Last update timestamp.
timestamp: Option<i64>,
}
impl From<deltachat_contact_tools::VcardContact> for VcardContact {
fn from(vc: deltachat_contact_tools::VcardContact) -> Self {
let display_name = vc.display_name().to_string();
let color = color::str_to_color(&vc.addr.to_lowercase());
Self {
addr: vc.addr,
display_name,
key: vc.key,
profile_image: vc.profile_image,
color: color_int_to_hex_string(color),
timestamp: vc.timestamp.ok(),
}
}
}
@@ -240,6 +240,10 @@ pub enum EventType {
status_update_serial: u32,
},
/// Data received over an ephemeral peer channel.
#[serde(rename_all = "camelCase")]
WebxdcRealtimeData { msg_id: u32, data: Vec<u8> },
/// Inform that a message containing a webxdc instance has been deleted
#[serde(rename_all = "camelCase")]
WebxdcInstanceDeleted { msg_id: u32 },
@@ -362,6 +366,10 @@ impl From<CoreEventType> for EventType {
msg_id: msg_id.to_u32(),
status_update_serial: status_update_serial.to_u32(),
},
CoreEventType::WebxdcRealtimeData { msg_id, data } => WebxdcRealtimeData {
msg_id: msg_id.to_u32(),
data,
},
CoreEventType::WebxdcInstanceDeleted { msg_id } => WebxdcInstanceDeleted {
msg_id: msg_id.to_u32(),
},
+38 -1
View File
@@ -1,3 +1,4 @@
use crate::api::VcardContact;
use anyhow::{Context as _, Result};
use deltachat::chat::Chat;
use deltachat::chat::ChatItem;
@@ -35,6 +36,10 @@ pub struct MessageObject {
parent_id: Option<u32>,
text: String,
/// Check if a message has a POI location bound to it.
/// These locations are also returned by `get_locations` method.
/// The UI may decide to display a special icon beside such messages.
has_location: bool,
has_html: bool,
view_type: MessageViewtype,
@@ -83,6 +88,8 @@ pub struct MessageObject {
download_state: DownloadState,
reactions: Option<JSONRPCReactions>,
vcard_contact: Option<VcardContact>,
}
#[derive(Serialize, TypeDef, schemars::JsonSchema)]
@@ -169,6 +176,13 @@ impl MessageObject {
Some(reactions.into())
};
let vcard_contacts: Vec<VcardContact> = message
.vcard_contacts(context)
.await?
.into_iter()
.map(Into::into)
.collect();
Ok(MessageObject {
id: msg_id.to_u32(),
chat_id: message.get_chat_id().to_u32(),
@@ -228,6 +242,8 @@ impl MessageObject {
download_state,
reactions,
vcard_contact: vcard_contacts.first().cloned(),
})
}
}
@@ -270,6 +286,11 @@ pub enum MessageViewtype {
/// Message is an webxdc instance.
Webxdc,
/// Message containing shared contacts represented as a vCard (virtual contact file)
/// with email addresses and possibly other fields.
/// Use `parse_vcard()` to retrieve them.
Vcard,
}
impl From<Viewtype> for MessageViewtype {
@@ -286,6 +307,7 @@ impl From<Viewtype> for MessageViewtype {
Viewtype::File => MessageViewtype::File,
Viewtype::VideochatInvitation => MessageViewtype::VideochatInvitation,
Viewtype::Webxdc => MessageViewtype::Webxdc,
Viewtype::Vcard => MessageViewtype::Vcard,
}
}
}
@@ -304,6 +326,7 @@ impl From<MessageViewtype> for Viewtype {
MessageViewtype::File => Viewtype::File,
MessageViewtype::VideochatInvitation => Viewtype::VideochatInvitation,
MessageViewtype::Webxdc => Viewtype::Webxdc,
MessageViewtype::Vcard => Viewtype::Vcard,
}
}
}
@@ -342,6 +365,14 @@ pub enum SystemMessageType {
LocationOnly,
InvalidUnencryptedMail,
/// 1:1 chats info message telling that SecureJoin has started and the user should wait for it
/// to complete.
SecurejoinWait,
/// 1:1 chats info message telling that SecureJoin is still running, but the user may already
/// send messages.
SecurejoinWaitTimeout,
/// Chat ephemeral message timer is changed.
EphemeralTimerChanged,
@@ -360,6 +391,9 @@ pub enum SystemMessageType {
/// Webxdc info added with `info` set in `send_webxdc_status_update()`.
WebxdcInfoMessage,
/// This message contains a users iroh node address.
IrohNodeAddr,
}
impl From<deltachat::mimeparser::SystemMessage> for SystemMessageType {
@@ -382,6 +416,9 @@ impl From<deltachat::mimeparser::SystemMessage> for SystemMessageType {
SystemMessage::WebxdcStatusUpdate => SystemMessageType::WebxdcStatusUpdate,
SystemMessage::WebxdcInfoMessage => SystemMessageType::WebxdcInfoMessage,
SystemMessage::InvalidUnencryptedMail => SystemMessageType::InvalidUnencryptedMail,
SystemMessage::IrohNodeAddr => SystemMessageType::IrohNodeAddr,
SystemMessage::SecurejoinWait => SystemMessageType::SecurejoinWait,
SystemMessage::SecurejoinWaitTimeout => SystemMessageType::SecurejoinWaitTimeout,
}
}
}
@@ -631,7 +668,7 @@ impl MessageInfo {
#[derive(
Debug, PartialEq, Eq, Copy, Clone, Serialize, Deserialize, TypeDef, schemars::JsonSchema,
)]
#[serde(rename_all = "camelCase", tag = "variant")]
#[serde(rename_all = "camelCase", tag = "kind")]
pub enum EphemeralTimer {
/// Timer is disabled.
Disabled,
+1
View File
@@ -1,3 +1,4 @@
#![recursion_limit = "256"]
pub mod api;
pub use yerpc;
+1
View File
@@ -1,3 +1,4 @@
#![recursion_limit = "256"]
use std::net::SocketAddr;
use std::path::PathBuf;
+7 -2
View File
@@ -25,12 +25,17 @@
"exports": {
".": {
"import": "./dist/deltachat.js",
"require": "./dist/deltachat.cjs"
"require": "./dist/deltachat.cjs",
"types": "./dist/deltachat.d.ts"
}
},
"license": "MPL-2.0",
"main": "dist/deltachat.js",
"name": "@deltachat/jsonrpc-client",
"repository": {
"type": "git",
"url": "https://github.com/deltachat/deltachat-core-rust.git"
},
"scripts": {
"build": "run-s generate-bindings extract-constants build:tsc build:bundle build:cjs",
"build:bundle": "esbuild --format=esm --bundle dist/deltachat.js --outfile=dist/deltachat.bundle.js",
@@ -53,5 +58,5 @@
},
"type": "module",
"types": "dist/deltachat.d.ts",
"version": "1.137.3"
"version": "1.139.2"
}
+2 -2
View File
@@ -1,6 +1,6 @@
[package]
name = "deltachat-repl"
version = "1.137.3"
version = "1.139.2"
license = "MPL-2.0"
edition = "2021"
repository = "https://github.com/deltachat/deltachat-core-rust"
@@ -11,10 +11,10 @@ anyhow = "1"
deltachat = { path = "..", features = ["internals"]}
dirs = "5"
log = "0.4.21"
pretty_env_logger = "0.5"
rusqlite = "0.31"
rustyline = "14"
tokio = { version = "1.37.0", features = ["fs", "rt-multi-thread", "macros"] }
tracing-subscriber = { version = "0.3", features = ["env-filter"] }
[features]
default = ["vendored"]
+6 -3
View File
@@ -1,3 +1,4 @@
#![recursion_limit = "256"]
//! This is a CLI program and a little testing frame. This file must not be
//! included when using Delta Chat Core as a library.
//!
@@ -31,6 +32,7 @@ use rustyline::{
};
use tokio::fs;
use tokio::runtime::Handle;
use tracing_subscriber::EnvFilter;
mod cmdline;
use self::cmdline::*;
@@ -482,9 +484,10 @@ async fn handle_cmd(
#[tokio::main]
async fn main() -> Result<(), Error> {
pretty_env_logger::formatted_timed_builder()
.parse_default_env()
.filter_module("deltachat_repl", log::LevelFilter::Info)
tracing_subscriber::fmt()
.with_env_filter(
EnvFilter::from_default_env().add_directive("deltachat_repl=info".parse()?),
)
.init();
let args = std::env::args().collect();
+1
View File
@@ -4,6 +4,7 @@
it will echo back any text send to it, it also will print to console all Delta Chat core events.
Pass --help to the CLI to see available options.
"""
from deltachat_rpc_client import events, run_bot_cli
hooks = events.HookCollection()
@@ -3,6 +3,7 @@
it will echo back any message that has non-empty text and also supports the /help command.
"""
import logging
import sys
from threading import Thread
@@ -2,6 +2,7 @@
"""
Example echo bot without using hooks
"""
import logging
import sys
+2 -2
View File
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "deltachat-rpc-client"
version = "1.137.3"
version = "1.139.2"
description = "Python client for Delta Chat core JSON-RPC interface"
classifiers = [
"Development Status :: 5 - Production/Stable",
@@ -37,7 +37,7 @@ deltachat_rpc_client = [
line-length = 120
[tool.ruff]
select = [
lint.select = [
"E", "W", # pycodestyle
"F", # Pyflakes
"N", # pep8-naming
@@ -297,6 +297,12 @@ class Account:
if event.kind == EventType.INCOMING_MSG:
return event
def wait_for_incoming_msg(self):
"""Wait for incoming message and return it.
Consumes all events before the next incoming message event."""
return self.get_message_by_id(self.wait_for_incoming_msg_event().msg_id)
def wait_for_securejoin_inviter_success(self):
while True:
event = self.wait_for_event()
@@ -2,6 +2,7 @@ from __future__ import annotations
import calendar
from dataclasses import dataclass
from tempfile import NamedTemporaryFile
from typing import TYPE_CHECKING, Optional, Union
from ._utils import AttrDict
@@ -265,3 +266,11 @@ class Chat:
location["message"] = Message(self.account, location.msg_id)
locations.append(location)
return locations
def send_contact(self, contact: Contact):
"""Send contact to the chat."""
vcard = contact.make_vcard()
with NamedTemporaryFile(suffix=".vcard") as f:
f.write(vcard.encode())
f.flush()
self._rpc.send_msg(self.account.id, self.id, {"viewtype": ViewType.VCARD, "file": f.name})
@@ -61,6 +61,7 @@ class EventType(str, Enum):
WEBXDC_INSTANCE_DELETED = "WebxdcInstanceDeleted"
CHATLIST_CHANGED = "ChatlistChanged"
CHATLIST_ITEM_CHANGED = "ChatlistItemChanged"
CONFIG_SYNCED = "ConfigSynced"
class ChatId(IntEnum):
@@ -113,6 +114,7 @@ class ViewType(str, Enum):
FILE = "File"
VIDEOCHAT_INVITATION = "VideochatInvitation"
WEBXDC = "Webxdc"
VCARD = "Vcard"
class SystemMessageType(str, Enum):
@@ -60,3 +60,6 @@ class Contact:
self.account,
self._rpc.create_chat_by_contact_id(self.account.id, self.id),
)
def make_vcard(self) -> str:
return self._rpc.make_vcard(self.account.id, [self.id])
@@ -126,8 +126,7 @@ def test_download_on_demand(acfactory: ACFactory) -> None:
alice.set_config("download_limit", "1")
event = bob.wait_for_incoming_msg_event()
msg = bob.get_message_by_id(event.msg_id)
msg = bob.wait_for_incoming_msg()
chat_id = msg.get_snapshot().chat_id
msg.get_snapshot().chat.accept()
bob.get_chat_by_id(chat_id).send_message(
@@ -135,13 +134,15 @@ def test_download_on_demand(acfactory: ACFactory) -> None:
html=base64.b64encode(os.urandom(300000)).decode("utf-8"),
)
msg_id = alice.wait_for_incoming_msg_event().msg_id
assert alice.get_message_by_id(msg_id).get_snapshot().download_state == const.DownloadState.AVAILABLE
message = alice.wait_for_incoming_msg()
snapshot = message.get_snapshot()
assert snapshot.download_state == const.DownloadState.AVAILABLE
alice.clear_all_events()
chat_id = alice.get_message_by_id(msg_id).get_snapshot().chat_id
alice._rpc.download_full_message(alice.id, msg_id)
snapshot = message.get_snapshot()
chat_id = snapshot.chat_id
alice._rpc.download_full_message(alice.id, message.id)
wait_for_chatlist_specific_item(alice, chat_id)
@@ -177,8 +178,7 @@ def test_imap_sync_seen_msgs(acfactory: ACFactory) -> None:
alice_chat_bob.send_text("hello")
event = bob.wait_for_incoming_msg_event()
msg = bob.get_message_by_id(event.msg_id)
msg = bob.wait_for_incoming_msg()
bob_chat_id = msg.get_snapshot().chat_id
msg.get_snapshot().chat.accept()
@@ -189,8 +189,7 @@ def test_imap_sync_seen_msgs(acfactory: ACFactory) -> None:
# make sure alice_second_device already received the message
alice_second_device.wait_for_incoming_msg_event()
event = alice.wait_for_incoming_msg_event()
msg = alice.get_message_by_id(event.msg_id)
msg = alice.wait_for_incoming_msg()
alice_second_device.clear_all_events()
msg.mark_seen()
+1 -1
View File
@@ -508,8 +508,8 @@ def test_reactions_for_a_reordering_move(acfactory):
"""
(ac1,) = acfactory.get_online_accounts(1)
ac2 = acfactory.new_preconfigured_account()
ac2.set_config("mvbox_move", "1")
ac2.configure()
ac2.set_config("mvbox_move", "1")
ac2.bring_online()
chat1 = acfactory.get_accepted_chat(ac1, ac2)
ac2.stop_io()
+15
View File
@@ -0,0 +1,15 @@
def test_vcard(acfactory) -> None:
alice, bob = acfactory.get_online_accounts(2)
bob_addr = bob.get_config("addr")
alice_contact_bob = alice.create_contact(bob_addr, "Bob")
alice_contact_charlie = alice.create_contact("charlie@example.org", "Charlie")
alice_chat_bob = alice_contact_bob.create_chat()
alice_chat_bob.send_contact(alice_contact_charlie)
event = bob.wait_for_incoming_msg_event()
message = bob.get_message_by_id(event.msg_id)
snapshot = message.get_snapshot()
assert snapshot.vcard_contact
assert snapshot.vcard_contact.addr == "charlie@example.org"
+1 -2
View File
@@ -22,9 +22,8 @@ skipsdist = True
skip_install = True
deps =
ruff
black
commands =
black --quiet --check --diff src/ examples/ tests/
ruff format --quiet --diff src/ examples/ tests/
ruff check src/ examples/ tests/
[pytest]
+2 -2
View File
@@ -1,6 +1,6 @@
[package]
name = "deltachat-rpc-server"
version = "1.137.3"
version = "1.139.2"
description = "DeltaChat JSON-RPC server"
edition = "2021"
readme = "README.md"
@@ -14,13 +14,13 @@ deltachat-jsonrpc = { path = "../deltachat-jsonrpc", default-features = false }
deltachat = { path = "..", default-features = false }
anyhow = "1"
env_logger = { version = "0.11.3" }
futures-lite = "2.3.0"
log = "0.4"
serde_json = "1"
serde = { version = "1.0", features = ["derive"] }
tokio = { version = "1.37.0", features = ["io-std"] }
tokio-util = "0.7.9"
tracing-subscriber = { version = "0.3", features = ["env-filter"] }
yerpc = { version = "0.5.2", features = ["anyhow_expose", "openrpc"] }
[features]
@@ -0,0 +1,3 @@
platform_package
*.tgz
package-lock.json
@@ -0,0 +1,3 @@
platform_package/*
scripts/
*.tgz
@@ -0,0 +1,79 @@
## npm package for deltachat-rpc-server
This is the successor of `deltachat-node`,
it does not use NAPI bindings but instead uses stdio executables
to let you talk to core over jsonrpc over stdio.
This simplifies cross-compilation and even reduces binary size (no CFFI layer and no NAPI layer).
## Usage
> The **minimum** nodejs version for this package is `20.11`
```
npm i @deltachat/stdio-rpc-server @deltachat/jsonrpc-client
```
```js
import { startDeltaChat } from "@deltachat/stdio-rpc-server";
import { C } from "@deltachat/jsonrpc-client";
async function main() {
const dc = await startDeltaChat("deltachat-data");
console.log(await dc.rpc.getSystemInfo());
dc.close()
}
main()
```
For a more complete example refer to https://github.com/deltachat-bot/echo/pull/69/files (TODO change link when pr is merged).
## How to use on an unsupported platform
<!-- todo instructions, will uses an env var for pointing to `deltachat-rpc-server` binary -->
<!-- todo copy parts from https://github.com/deltachat/deltachat-desktop/blob/7045c6f549e4b9d5caa0709d5bd314bbd9fd53db/docs/UPDATE_CORE.md -->
## How does it work when you install it
NPM automatically installs platform dependent optional dependencies when `os` and `cpu` fields are set correctly.
references:
- https://napi.rs/docs/deep-dive/release#3-the-native-addon-for-different-platforms-is-distributed-through-different-npm-packages, [webarchive version](https://web.archive.org/web/20240309234250/https://napi.rs/docs/deep-dive/release#3-the-native-addon-for-different-platforms-is-distributed-through-different-npm-packages)
- https://docs.npmjs.com/cli/v6/configuring-npm/package-json#cpu
- https://docs.npmjs.com/cli/v6/configuring-npm/package-json#os
When you import this package it searches for the rpc server in the following locations and order:
1. `DELTA_CHAT_RPC_SERVER` environment variable
2. in PATH
- unless `DELTA_CHAT_SKIP_PATH=1` is specified
- searches in .cargo/bin directory first
- but there an additional version check is performed
3. prebuilds in npm packages
## How do you built this package in CI
- To build platform packages, run the `build_platform_package.py` script:
```
python3 build_platform_package.py <cargo-target>
# example
python3 build_platform_package.py x86_64-apple-darwin
```
- Then pass it as an artifact to the last CI action that publishes the main package.
- upload all packages from `deltachat-rpc-server/npm-package/platform_package`.
- then publish `deltachat-rpc-server/npm-package`,
- this will run `update_optional_dependencies_and_version.js` (in the `prepack` script),
which puts all platform packages into `optionalDependencies` and updates the `version` in `package.json`
## How to build a version you can use localy on your host machine for development
You can not install the npm packet from the previous section locally, unless you have a local npm registry set up where you upload it too. This is why we have seperate scripts for making it work for local installation.
- If you just need your host platform run `python scripts/make_local_dev_version.py`
- note: this clears the `platform_package` folder
- (advanced) If you need more than one platform for local install you can just run `node scripts/update_optional_dependencies_and_version.js` after building multiple plaftorms with `build_platform_package.py`
## Thanks to nlnet
The initial work on this package was funded by nlnet as part of the [Delta Tauri](https://nlnet.nl/project/DeltaTauri/) Project.
+42
View File
@@ -0,0 +1,42 @@
import { StdioDeltaChat } from "@deltachat/jsonrpc-client";
export interface SearchOptions {
/** whether to disable looking for deltachat-rpc-server inside of $PATH */
skipSearchInPath: boolean;
/** whether to disable the DELTA_CHAT_RPC_SERVER environment variable */
disableEnvPath: boolean;
}
/**
*
* @returns absolute path to deltachat-rpc-server binary
* @throws when it is not found
*/
export function getRPCServerPath(
options?: Partial<SearchOptions>
): Promise<string>;
export type DeltaChatOverJsonRpcServer = StdioDeltaChat & {
readonly pathToServerBinary: string;
};
export interface StartOptions {
/** whether to disable outputting stderr to the parent process's stderr */
muteStdErr: boolean;
}
/**
*
* @param directory directory for accounts folder
* @param options
*/
export function startDeltaChat(directory: string, options?: Partial<SearchOptions & StartOptions> ): Promise<DeltaChatOverJsonRpcServer>
export namespace FnTypes {
export type getRPCServerPath = typeof getRPCServerPath
export type startDeltaChat = typeof startDeltaChat
}
+170
View File
@@ -0,0 +1,170 @@
//@ts-check
import { execFile, spawn } from "node:child_process";
import { stat, readdir } from "node:fs/promises";
import os from "node:os";
import { join, basename } from "node:path";
import process from "node:process";
import { promisify } from "node:util";
import {
ENV_VAR_NAME,
PATH_EXECUTABLE_NAME,
SKIP_SEARCH_IN_PATH,
} from "./src/const.js";
import {
ENV_VAR_LOCATION_NOT_FOUND,
FAILED_TO_START_SERVER_EXECUTABLE,
NPM_NOT_FOUND_SUPPORTED_PLATFORM_ERROR,
NPM_NOT_FOUND_UNSUPPORTED_PLATFORM_ERROR,
} from "./src/errors.js";
// Because this is not compiled by typescript, esm needs this stuff (` with { type: "json" };`,
// nodejs still complains about it being experimental, but deno also uses it, so treefit bets taht it will become standard)
import package_json from "./package.json" with { type: "json" };
import { createRequire } from "node:module";
function findRPCServerInNodeModules() {
const arch = os.arch();
const operating_system = process.platform;
const package_name = `@deltachat/stdio-rpc-server-${operating_system}-${arch}`;
try {
const { resolve } = createRequire(import.meta.url);
return resolve(package_name);
} catch (error) {
console.debug("findRpcServerInNodeModules", error);
if (Object.keys(package_json.optionalDependencies).includes(package_name)) {
throw new Error(NPM_NOT_FOUND_SUPPORTED_PLATFORM_ERROR(package_name));
} else {
throw new Error(NPM_NOT_FOUND_UNSUPPORTED_PLATFORM_ERROR());
}
}
}
/**
* @returns {Promise<string>}
*/
async function getLocationInPath() {
const exec = promisify(execFile);
if (os.platform() === "win32") {
const { stdout: executable } = await exec("where", [PATH_EXECUTABLE_NAME], {
shell: true,
});
return executable;
}
try {
const { stdout: executable } = await exec(
"command",
["-v", PATH_EXECUTABLE_NAME],
{ shell: true }
);
return executable;
} catch (error) {
if (error.code > 0) return "";
else throw error;
}
}
/** @type {import("./index").FnTypes.getRPCServerPath} */
export async function getRPCServerPath(
options = { skipSearchInPath: false, disableEnvPath: false }
) {
// @TODO: improve confusing naming of these options
const { skipSearchInPath, disableEnvPath } = options;
// 1. check if it is set as env var
if (process.env[ENV_VAR_NAME] && !disableEnvPath) {
try {
if (!(await stat(process.env[ENV_VAR_NAME])).isFile()) {
throw new Error(
`expected ${ENV_VAR_NAME} to point to the deltachat-rpc-server executable`
);
}
} catch (error) {
throw new Error(ENV_VAR_LOCATION_NOT_FOUND());
}
return process.env[ENV_VAR_NAME];
}
// 2. check if it can be found in PATH
if (!process.env[SKIP_SEARCH_IN_PATH] && !skipSearchInPath) {
const executable = await getLocationInPath();
// by just trying to execute it and then use "command -v deltachat-rpc-server" (unix) or "where deltachat-rpc-server" (windows) to get the path to the executable
if (executable.length > 1) {
// test if it is the right version
try {
// for some unknown reason it is in stderr and not in stdout
const { stderr } = await promisify(execFile)(
executable,
["--version"],
{ shell: true }
);
const version = stderr.slice(0, stderr.indexOf("\n"));
if (package_json.version !== version) {
throw new Error(
`version mismatch: (npm package: ${package_json.version}) (installed ${PATH_EXECUTABLE_NAME} version: ${version})`
);
} else {
return executable;
}
} catch (error) {
console.error(
"Found executable in PATH, but there was an error: " + error
);
console.error("So falling back to using prebuild...");
}
}
}
// 3. check for prebuilds
return findRPCServerInNodeModules();
}
import { StdioDeltaChat } from "@deltachat/jsonrpc-client";
/** @type {import("./index").FnTypes.startDeltaChat} */
export async function startDeltaChat(
directory,
options = {
skipSearchInPath: false,
disableEnvPath: false,
muteStdErr: false,
}
) {
const pathToServerBinary = await getRPCServerPath(options);
const server = spawn(pathToServerBinary, {
env: {
RUST_LOG: process.env.RUST_LOG || "info",
DC_ACCOUNTS_PATH: directory,
},
stdio: ["pipe", "pipe", options.muteStdErr ? "ignore" : "inherit"],
});
server.on("error", (err) => {
throw new Error(FAILED_TO_START_SERVER_EXECUTABLE(pathToServerBinary, err));
});
let shouldClose = false;
server.on("exit", () => {
if (shouldClose) {
return;
}
throw new Error("Server quit");
});
/** @type {import('./index').DeltaChatOverJsonRpcServer} */
//@ts-expect-error
const dc = new StdioDeltaChat(server.stdin, server.stdout, true);
dc.close = () => {
shouldClose = true;
if (!server.kill()) {
console.log("server termination failed");
}
};
//@ts-expect-error
dc.pathToServerBinary = pathToServerBinary;
return dc;
}
@@ -0,0 +1,19 @@
{
"license": "MPL-2.0",
"main": "index.js",
"name": "@deltachat/stdio-rpc-server",
"optionalDependencies": {},
"peerDependencies": {
"@deltachat/jsonrpc-client": "*"
},
"repository": {
"type": "git",
"url": "https://github.com/deltachat/deltachat-core-rust.git"
},
"scripts": {
"prepack": "node scripts/update_optional_dependencies_and_version.js"
},
"type": "module",
"types": "index.d.ts",
"version": "1.139.2"
}
@@ -0,0 +1,53 @@
#!/usr/bin/env python3
import subprocess
from sys import argv
from os import path, makedirs, chdir
from shutil import copy
from src.make_package import write_package_json
# ensure correct working directory
chdir(path.join(path.dirname(path.abspath(__file__)), "../"))
if len(argv) < 2:
print("First argument should be target architecture as required by cargo")
exit(1)
target = argv[1].strip()
subprocess.run(
["cargo", "build", "--release", "-p", "deltachat-rpc-server", "--target", target],
check=True,
)
newpath = "platform_package"
if not path.exists(newpath):
makedirs(newpath)
# make new folder
platform_path = "platform_package/" + target
if not path.exists(platform_path):
makedirs(platform_path)
# copy binary it over
def binary_path(binary_name):
return "../../target/" + target + "/release/" + binary_name
my_binary_name = "deltachat-rpc-server"
if not path.isfile(binary_path("deltachat-rpc-server")):
my_binary_name = "deltachat-rpc-server.exe"
if not path.isfile(binary_path("deltachat-rpc-server.exe")):
print("Did not find the build")
exit(1)
my_binary_path = binary_path(my_binary_name)
copy(my_binary_path, platform_path + "/" + my_binary_name)
# make a package.json for it
write_package_json(platform_path, target, my_binary_name)
@@ -0,0 +1,34 @@
# This script is for making a version of the npm packet that you can install locally
import subprocess
from sys import argv
from os import path, makedirs, chdir
import re
import json
import tomllib
from shutil import copy, rmtree
# ensure correct working directory
chdir(path.join(path.dirname(path.abspath(__file__)), "../"))
# get host target with "rustc -vV"
output = subprocess.run(["rustc", "-vV"], capture_output=True)
host_target = re.search('host: ([-\\w]*)', output.stdout.decode("utf-8")).group(1)
print("host target to build for is:", host_target)
# clean platform_package folder
newpath = r'platform_package'
if not path.exists(newpath):
makedirs(newpath)
else:
rmtree(path.join(path.dirname(path.abspath(__file__)), "../platform_package/"))
makedirs(newpath)
# run build_platform_package.py with the host's target to build it
subprocess.run(["python", "scripts/build_platform_package.py", host_target], capture_output=False, check=True)
# run update_optional_dependencies_and_version.js to adjust the package / make it installable locally
subprocess.run(["node", "scripts/update_optional_dependencies_and_version.js", "--local"], capture_output=False, check=True)
# typescript / npm local package installing/linking needs that this package has it's own node_modules folder
subprocess.run(["npm", "i"], capture_output=False, check=True)
@@ -0,0 +1,46 @@
import subprocess
from sys import argv
from os import path, makedirs, chdir, chmod, stat
import json
from shutil import copy
from src.make_package import write_package_json
# ensure correct working directory
chdir(path.join(path.dirname(path.abspath(__file__)), "../"))
if len(argv) < 3:
print("First argument should be target architecture as required by cargo")
print("Second argument should be the location of th built binary (binary_path)")
exit(1)
target = argv[1].strip()
binary_path = argv[2].strip()
output = subprocess.run(["rustc","--print","target-list"], capture_output=True, check=True)
available_targets = output.stdout.decode("utf-8")
if available_targets.find(target) == -1:
print("target", target, "is not known / not valid")
exit(1)
newpath = r'platform_package'
if not path.exists(newpath):
makedirs(newpath)
# make new folder
platform_path = 'platform_package/' + target
if not path.exists(platform_path):
makedirs(platform_path)
# copy binary it over
my_binary_name = path.basename(binary_path)
new_binary_path = platform_path + "/" + my_binary_name
copy(binary_path, new_binary_path)
chmod(new_binary_path, 0o555) # everyone can read & execute, nobody can write
# make a package.json for it
write_package_json(platform_path, target, my_binary_name)
@@ -0,0 +1,21 @@
def convert_cpu_arch_to_npm_cpu_arch(arch):
if arch == "x86_64":
return "x64"
if arch == "i686":
return "ia32"
if arch == "aarch64":
return "arm64"
if arch == "armv7" or arch == "arm":
return "arm"
print("architecture might not be known by nodejs, please make sure it can be returned by 'process.arch':", arch)
return arch
def convert_os_to_npm_os(os):
if os == "windows":
return "win32"
if os == "darwin" or os == "linux":
return os
if os.startswith("android"):
return "android"
print("architecture might not be known by nodejs, please make sure it can be returned by 'process.platform':", os)
return os
@@ -0,0 +1,34 @@
import tomllib
import json
from .convert_platform import convert_cpu_arch_to_npm_cpu_arch, convert_os_to_npm_os
def write_package_json(platform_path, rust_target, my_binary_name):
if len(rust_target.split("-")) == 3:
[cpu_arch, vendor, os] = rust_target.split("-")
else:
[cpu_arch, vendor, os, _env] = rust_target.split("-")
# read version
tomlfile = open("../../Cargo.toml", 'rb')
version = tomllib.load(tomlfile)['package']['version']
package_json = {
"name": "@deltachat/stdio-rpc-server-"
+ convert_os_to_npm_os(os)
+ "-"
+ convert_cpu_arch_to_npm_cpu_arch(cpu_arch),
"version": version,
"os": [convert_os_to_npm_os(os)],
"cpu": [convert_cpu_arch_to_npm_cpu_arch(cpu_arch)],
"main": my_binary_name,
"license": "MPL-2.0",
"repository": {
"type": "git",
"url": "https://github.com/deltachat/deltachat-core-rust.git",
},
}
file = open(platform_path + "/package.json", 'w')
file.write(json.dumps(package_json, indent=4))
@@ -0,0 +1,63 @@
import fs from "node:fs/promises";
import { join, dirname } from "node:path";
import { fileURLToPath } from "node:url";
const expected_cwd = join(dirname(fileURLToPath(import.meta.url)), "..");
if (process.cwd() !== expected_cwd) {
console.error(
"CWD missmatch: this script needs to be run from " + expected_cwd,
{ actual: process.cwd(), expected: expected_cwd }
);
process.exit(1);
}
// whether to use local paths instead of npm registry version number for the prebuilds in optionalDependencies
// useful for local development
const is_local = process.argv.includes("--local");
const package_json = JSON.parse(await fs.readFile("./package.json", "utf8"));
const cargo_toml = await fs.readFile("../Cargo.toml", "utf8");
const version = cargo_toml
.split("\n")
.find((line) => line.includes("version"))
.split('"')[1];
const platform_packages_dir = "./platform_package";
const platform_package_names = await Promise.all(
(await fs.readdir(platform_packages_dir)).map(async (name) => {
const p = JSON.parse(
await fs.readFile(
join(platform_packages_dir, name, "package.json"),
"utf8"
)
);
if (p.version !== version) {
console.error(
name,
"has a different version than the version of the rpc server.",
{ rpc_server: version, platform_package: p.version }
);
throw new Error("version missmatch");
}
return { folder_name: name, package_name: p.name };
})
);
package_json.version = version;
package_json.optionalDependencies = {};
for (const { folder_name, package_name } of platform_package_names) {
package_json.optionalDependencies[package_name] = is_local
? `file:${expected_cwd}/platform_package/${folder_name}` // npm seems to work better with an absolute path here
: version;
}
if (is_local) {
package_json.peerDependencies["@deltachat/jsonrpc-client"] = 'file:../../deltachat-jsonrpc/typescript'
} else {
package_json.peerDependencies["@deltachat/jsonrpc-client"] = "*"
}
await fs.writeFile("./package.json", JSON.stringify(package_json, null, 4));
@@ -0,0 +1,6 @@
//@ts-check
export const PATH_EXECUTABLE_NAME = 'deltachat-rpc-server'
export const ENV_VAR_NAME = "DELTA_CHAT_RPC_SERVER"
export const SKIP_SEARCH_IN_PATH = "DELTA_CHAT_SKIP_PATH"
@@ -0,0 +1,41 @@
//@ts-check
import { ENV_VAR_NAME } from "./const.js";
const cargoInstallCommand =
"cargo install --git https://github.com/deltachat/deltachat-core-rust deltachat-rpc-server";
export function NPM_NOT_FOUND_SUPPORTED_PLATFORM_ERROR(package_name) {
return `deltachat-rpc-server not found:
- Install it with "npm i ${package_name}"
- or download/compile deltachat-rpc-server for your platform and
- either put it into your PATH (for example with "${cargoInstallCommand}")
- or set the "${ENV_VAR_NAME}" env var to the path to deltachat-rpc-server"`;
}
export function NPM_NOT_FOUND_UNSUPPORTED_PLATFORM_ERROR() {
return `deltachat-rpc-server not found:
Unfortunately no prebuild is available for your system, so you need to provide deltachat-rpc-server yourself.
- Download or Compile deltachat-rpc-server for your platform and
- either put it into your PATH (for example with "${cargoInstallCommand}")
- or set the "${ENV_VAR_NAME}" env var to the path to deltachat-rpc-server"`;
}
export function ENV_VAR_LOCATION_NOT_FOUND(error) {
return `deltachat-rpc-server not found in ${ENV_VAR_NAME}:
Error: ${error}
Content of ${ENV_VAR_NAME}: "${process.env[ENV_VAR_NAME]}"`;
}
export function FAILED_TO_START_SERVER_EXECUTABLE(pathToServerBinary, error) {
return `Failed to start server executable at '${pathToServerBinary}',
Error: ${error}
Make sure the deltachat-rpc-server binary exists at this location
and you can start it with \`${pathToServerBinary} --version\``;
}
+12 -1
View File
@@ -1,3 +1,4 @@
#![recursion_limit = "256"]
//! Delta Chat core RPC server.
//!
//! It speaks JSON Lines over stdio.
@@ -10,6 +11,7 @@ use deltachat::constants::DC_VERSION_STR;
use deltachat_jsonrpc::api::{Accounts, CommandApi};
use futures_lite::stream::StreamExt;
use tokio::io::{self, AsyncBufReadExt, BufReader};
use tracing_subscriber::EnvFilter;
use yerpc::RpcServer as _;
#[cfg(target_family = "unix")]
@@ -27,6 +29,9 @@ async fn main() {
// "For technical reasons, stdin is implemented by using an ordinary blocking read on a separate
// thread, and it is impossible to cancel that read. This can make shutdown of the runtime hang
// until the user presses enter."
if let Err(error) = &r {
log::error!("Fatal error: {error:#}.")
}
std::process::exit(if r.is_ok() { 0 } else { 1 });
}
@@ -59,7 +64,13 @@ async fn main_impl() -> Result<()> {
#[cfg(target_family = "unix")]
let mut sigterm = signal_unix::signal(signal_unix::SignalKind::terminate())?;
env_logger::Builder::from_env(env_logger::Env::default().default_filter_or("info")).init();
// Logs from `log` crate and traces from `tracing` crate
// are configurable with `RUST_LOG` environment variable
// and go to stderr to avoid interferring with JSON-RPC using stdout.
tracing_subscriber::fmt()
.with_env_filter(EnvFilter::from_default_env())
.with_writer(std::io::stderr)
.init();
let path = std::env::var("DC_ACCOUNTS_PATH").unwrap_or_else(|_| "accounts".to_string());
log::info!("Starting with accounts directory `{}`.", path);
+29 -1
View File
@@ -23,6 +23,9 @@ ignore = [
# when upgrading.
# Please keep this list alphabetically sorted.
skip = [
{ name = "asn1-rs-derive", version = "0.4.0" },
{ name = "asn1-rs-impl", version = "0.1.0" },
{ name = "asn1-rs", version = "0.5.2" },
{ name = "async-channel", version = "1.9.0" },
{ name = "base16ct", version = "0.1.1" },
{ name = "base64", version = "<0.21" },
@@ -34,20 +37,37 @@ skip = [
{ name = "darling_core", version = "<0.14" },
{ name = "darling_macro", version = "<0.14" },
{ name = "darling", version = "<0.14" },
{ name = "der_derive", version = "0.6.1" },
{ name = "derive_more", version = "0.99.17" },
{ name = "der-parser", version = "8.2.0" },
{ name = "der", version = "0.6.1" },
{ name = "digest", version = "<0.10" },
{ name = "dlopen2", version = "0.4.1" },
{ name = "ed25519-dalek", version = "1.0.1" },
{ name = "ed25519", version = "1.5.3" },
{ name = "env_logger", version = "0.10.2" },
{ name = "event-listener", version = "2.5.3" },
{ name = "event-listener", version = "4.0.3" },
{ name = "fastrand", version = "1.9.0" },
{ name = "futures-lite", version = "1.13.0" },
{ name = "getrandom", version = "<0.2" },
{ name = "http-body", version = "0.4.6" },
{ name = "http", version = "0.2.12" },
{ name = "hyper", version = "0.14.28" },
{ name = "idna", version = "0.4.0" },
{ name = "netlink-packet-core", version = "0.5.0" },
{ name = "netlink-packet-route", version = "0.15.0" },
{ name = "nix", version = "0.26.4" },
{ name = "oid-registry", version = "0.6.1" },
{ name = "pem-rfc7468", version = "0.6.0" },
{ name = "pem", version = "1.1.1" },
{ name = "pkcs8", version = "0.9.0" },
{ name = "proc-macro-error-attr", version = "0.4.12" },
{ name = "proc-macro-error", version = "0.4.12" },
{ name = "quick-error", version = "<2.0" },
{ name = "rand_chacha", version = "<0.3" },
{ name = "rand_core", version = "<0.6" },
{ name = "rand", version = "<0.8" },
{ name = "rcgen", version = "<0.12.1" },
{ name = "redox_syscall", version = "0.3.5" },
{ name = "regex-automata", version = "0.1.10" },
{ name = "regex-syntax", version = "0.6.29" },
@@ -57,23 +77,31 @@ skip = [
{ name = "signature", version = "1.6.4" },
{ name = "spin", version = "<0.9.6" },
{ name = "spki", version = "0.6.0" },
{ name = "ssh-encoding", version = "0.1.0" },
{ name = "ssh-key", version = "0.5.1" },
{ name = "sync_wrapper", version = "0.1.2" },
{ name = "synstructure", version = "0.12.6" },
{ name = "syn", version = "1.0.109" },
{ name = "system-configuration-sys", version = "0.5.0" },
{ name = "system-configuration", version = "0.5.1" },
{ name = "time", version = "<0.3" },
{ name = "toml_edit", version = "0.21.1" },
{ name = "untrusted", version = "0.7.1" },
{ name = "wasi", version = "<0.11" },
{ name = "windows_aarch64_gnullvm", version = "<0.52" },
{ name = "windows_aarch64_msvc", version = "<0.52" },
{ name = "windows-core", version = "<0.54.0" },
{ name = "windows_i686_gnu", version = "<0.52" },
{ name = "windows_i686_msvc", version = "<0.52" },
{ name = "windows-sys", version = "<0.52" },
{ name = "windows-targets", version = "<0.52" },
{ name = "windows", version = "0.32.0" },
{ name = "windows", version = "<0.54.0" },
{ name = "windows_x86_64_gnullvm", version = "<0.52" },
{ name = "windows_x86_64_gnu", version = "<0.52" },
{ name = "windows_x86_64_msvc", version = "<0.52" },
{ name = "winnow", version = "0.5.40" },
{ name = "x509-parser", version = "<0.16.0" },
]
Generated
+17 -18
View File
@@ -48,11 +48,11 @@
"rust-analyzer-src": "rust-analyzer-src"
},
"locked": {
"lastModified": 1713421495,
"narHash": "sha256-5vVF9W1tJT+WdfpWAEG76KywktKDAW/71mVmNHEHjac=",
"lastModified": 1714112748,
"narHash": "sha256-jq6Cpf/pQH85p+uTwPPrGG8Ky/zUOTwMJ7mcqc5M4So=",
"owner": "nix-community",
"repo": "fenix",
"rev": "fd47b1f9404fae02a4f38bd9f4b12bad7833c96b",
"rev": "3ae4b908a795b6a3824d401a0702e11a7157d7e1",
"type": "github"
},
"original": {
@@ -166,11 +166,11 @@
},
"nixpkgs_2": {
"locked": {
"lastModified": 1713248628,
"narHash": "sha256-NLznXB5AOnniUtZsyy/aPWOk8ussTuePp2acb9U+ISA=",
"lastModified": 1713895582,
"narHash": "sha256-cfh1hi+6muQMbi9acOlju3V1gl8BEaZBXBR9jQfQi4U=",
"owner": "nixos",
"repo": "nixpkgs",
"rev": "5672bc9dbf9d88246ddab5ac454e82318d094bb8",
"rev": "572af610f6151fd41c212f897c71f7056e3fb518",
"type": "github"
},
"original": {
@@ -182,12 +182,11 @@
},
"nixpkgs_3": {
"locked": {
"lastModified": 1713562564,
"narHash": "sha256-NQpYhgoy0M89g9whRixSwsHb8RFIbwlxeYiVSDwSXJg=",
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "92d295f588631b0db2da509f381b4fb1e74173c5",
"type": "github"
"lastModified": 1711668574,
"narHash": "sha256-u1dfs0ASQIEr1icTVrsKwg2xToIpn7ZXxW3RHfHxshg=",
"path": "/nix/store/9fpv0kjq9a80isa1wkkvrdqsh9dpcn05-source",
"rev": "219951b495fc2eac67b1456824cc1ec1fd2ee659",
"type": "path"
},
"original": {
"id": "nixpkgs",
@@ -196,11 +195,11 @@
},
"nixpkgs_4": {
"locked": {
"lastModified": 1713537308,
"narHash": "sha256-XtTSSIB2DA6tOv+l0FhvfDMiyCmhoRbNB+0SeInZkbk=",
"lastModified": 1714076141,
"narHash": "sha256-Drmja/f5MRHZCskS6mvzFqxEaZMeciScCTFxWVLqWEY=",
"owner": "nixos",
"repo": "nixpkgs",
"rev": "5c24cf2f0a12ad855f444c30b2421d044120c66f",
"rev": "7bb2ccd8cdc44c91edba16c48d2c8f331fb3d856",
"type": "github"
},
"original": {
@@ -223,11 +222,11 @@
"rust-analyzer-src": {
"flake": false,
"locked": {
"lastModified": 1713373173,
"narHash": "sha256-octd9BFY9G/Gbr4KfwK4itZp4Lx+qvJeRRcYnN+dEH8=",
"lastModified": 1714031783,
"narHash": "sha256-xS/niQsq1CQPOe4M4jvVPO2cnXS/EIeRG5gIopUbk+Q=",
"owner": "rust-lang",
"repo": "rust-analyzer",
"rev": "46702ffc1a02a2ac153f1d1ce619ec917af8f3a6",
"rev": "56bee2ddafa6177b19c631eedc88d43366553223",
"type": "github"
},
"original": {
+16 -6
View File
@@ -525,15 +525,25 @@
};
};
devShells.default = pkgs.mkShell {
devShells.default = let
pkgs = import nixpkgs {
system = system;
overlays = [ fenix.overlays.default ];
};
in pkgs.mkShell {
buildInputs = with pkgs; [
cargo
clippy
rustc
rustfmt
rust-analyzer
(fenix.packages.${system}.complete.withComponents [
"cargo"
"clippy"
"rust-src"
"rustc"
"rustfmt"
])
cargo-deny
rust-analyzer-nightly
perl # needed to build vendored OpenSSL
git-cliff
];
};
}
+4
View File
@@ -110,6 +110,7 @@ module.exports = {
DC_MSG_IMAGE: 20,
DC_MSG_STICKER: 23,
DC_MSG_TEXT: 10,
DC_MSG_VCARD: 90,
DC_MSG_VIDEO: 50,
DC_MSG_VIDEOCHAT_INVITATION: 70,
DC_MSG_VOICE: 41,
@@ -173,6 +174,7 @@ module.exports = {
DC_STR_CONFIGURATION_FAILED: 84,
DC_STR_CONNECTED: 107,
DC_STR_CONNTECTING: 108,
DC_STR_CONTACT: 200,
DC_STR_CONTACT_NOT_VERIFIED: 36,
DC_STR_CONTACT_SETUP_CHANGED: 37,
DC_STR_CONTACT_VERIFIED: 35,
@@ -266,6 +268,8 @@ module.exports = {
DC_STR_REMOVE_MEMBER_BY_YOU: 130,
DC_STR_REPLY_NOUN: 90,
DC_STR_SAVED_MESSAGES: 69,
DC_STR_SECUREJOIN_WAIT: 190,
DC_STR_SECUREJOIN_WAIT_TIMEOUT: 191,
DC_STR_SECURE_JOIN_GROUP_QR_DESC: 120,
DC_STR_SECURE_JOIN_REPLIES: 118,
DC_STR_SECURE_JOIN_STARTED: 117,
+4
View File
@@ -110,6 +110,7 @@ export enum C {
DC_MSG_IMAGE = 20,
DC_MSG_STICKER = 23,
DC_MSG_TEXT = 10,
DC_MSG_VCARD = 90,
DC_MSG_VIDEO = 50,
DC_MSG_VIDEOCHAT_INVITATION = 70,
DC_MSG_VOICE = 41,
@@ -173,6 +174,7 @@ export enum C {
DC_STR_CONFIGURATION_FAILED = 84,
DC_STR_CONNECTED = 107,
DC_STR_CONNTECTING = 108,
DC_STR_CONTACT = 200,
DC_STR_CONTACT_NOT_VERIFIED = 36,
DC_STR_CONTACT_SETUP_CHANGED = 37,
DC_STR_CONTACT_VERIFIED = 35,
@@ -266,6 +268,8 @@ export enum C {
DC_STR_REMOVE_MEMBER_BY_YOU = 130,
DC_STR_REPLY_NOUN = 90,
DC_STR_SAVED_MESSAGES = 69,
DC_STR_SECUREJOIN_WAIT = 190,
DC_STR_SECUREJOIN_WAIT_TIMEOUT = 191,
DC_STR_SECURE_JOIN_GROUP_QR_DESC = 120,
DC_STR_SECURE_JOIN_REPLIES = 118,
DC_STR_SECURE_JOIN_STARTED = 117,
+1 -1
View File
@@ -55,5 +55,5 @@
"test:mocha": "mocha node/test/test.mjs --growl --reporter=spec --bail --exit"
},
"types": "node/dist/index.d.ts",
"version": "1.137.3"
"version": "1.139.2"
}
+2 -2
View File
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "deltachat"
version = "1.137.3"
version = "1.139.2"
description = "Python bindings for the Delta Chat Core library using CFFI against the Rust-implemented libdeltachat"
readme = "README.rst"
requires-python = ">=3.7"
@@ -46,7 +46,7 @@ deltachat = [
line-length = 120
[tool.ruff]
select = ["E", "F", "W", "YTT", "C4", "ISC", "ICN", "TID", "DTZ", "PLC", "PLE", "PLW", "PIE", "COM", "UP004", "UP010", "UP031", "UP032", "ANN204"]
lint.select = ["E", "F", "W", "YTT", "C4", "ISC", "ICN", "TID", "DTZ", "PLC", "PLE", "PLW", "PIE", "COM", "UP004", "UP010", "UP031", "UP032", "ANN204"]
line-length = 120
[tool.isort]
+4
View File
@@ -275,6 +275,7 @@ class ACSetup:
def __init__(self, testprocess, init_time) -> None:
self._configured_events = Queue()
self._account2state: Dict[Account, str] = {}
self._account2config: Dict[Account, Dict[str, str]] = {}
self._imap_cleaned: Set[str] = set()
self.testprocess = testprocess
self.init_time = init_time
@@ -336,6 +337,8 @@ class ACSetup:
if not success:
pytest.fail(f"configuring online account {acc} failed: {comment}")
self._account2state[acc] = self.CONFIGURED
if acc in self._account2config:
acc.update_config(self._account2config[acc])
return acc
def _onconfigure_start_io(self, acc):
@@ -523,6 +526,7 @@ class ACFactory:
configdict.setdefault("sentbox_watch", False)
configdict.setdefault("sync_msgs", False)
ac.update_config(configdict)
self._acsetup._account2config[ac] = configdict
self._preconfigure_key(ac, configdict["addr"])
return ac
+2 -2
View File
@@ -18,14 +18,14 @@ def test_db_busy_error(acfactory):
# make a number of accounts
accounts = acfactory.get_many_online_accounts(3)
log("created %s accounts" % len(accounts))
log(f"created {len(accounts)} accounts")
# put a bigfile into each account
for acc in accounts:
acc.bigfile = os.path.join(acc.get_blobdir(), "bigfile")
with open(acc.bigfile, "wb") as f:
f.write(b"01234567890" * 1000_000)
log("created %s bigfiles" % len(accounts))
log(f"created {len(accounts)} bigfiles")
contact_addrs = [acc.get_self_contact().addr for acc in accounts]
chat = accounts[0].create_group_chat("stress-group")
+11 -7
View File
@@ -1343,7 +1343,6 @@ def test_quote_encrypted(acfactory, lp):
for quoted_msg in msg1, msg3:
# Save the draft with a quote.
# It should be encrypted if quoted message is encrypted.
msg_draft = Message.new_empty(ac1, "text")
msg_draft.set_text("message reply")
msg_draft.quote = quoted_msg
@@ -1357,10 +1356,14 @@ def test_quote_encrypted(acfactory, lp):
chat.set_draft(None)
assert chat.get_draft() is None
# Quote should be replaced with "..." if quoted message is encrypted.
msg_in = ac2._evtracker.wait_next_incoming_message()
assert msg_in.text == "message reply"
assert msg_in.quoted_text == quoted_msg.text
assert msg_in.is_encrypted() == quoted_msg.is_encrypted()
assert not msg_in.is_encrypted()
if quoted_msg.is_encrypted():
assert msg_in.quoted_text == "..."
else:
assert msg_in.quoted_text == quoted_msg.text
def test_quote_attachment(tmp_path, acfactory, lp):
@@ -2033,14 +2036,15 @@ def test_send_receive_locations(acfactory, lp):
assert chat1.is_sending_locations()
ac1._evtracker.get_matching("DC_EVENT_SMTP_MESSAGE_SENT")
# Wait for "enabled location streaming" message.
ac2._evtracker.wait_next_incoming_message()
# First location is sent immediately as a location-only message.
ac1.set_location(latitude=2.0, longitude=3.0, accuracy=0.5)
ac1._evtracker.get_matching("DC_EVENT_LOCATION_CHANGED")
chat1.send_text("🍞")
ac1._evtracker.get_matching("DC_EVENT_SMTP_MESSAGE_SENT")
lp.sec("ac2: wait for incoming location message")
# currently core emits location changed before event_incoming message
ac2._evtracker.get_matching("DC_EVENT_LOCATION_CHANGED")
locations = chat2.get_locations()
@@ -2049,7 +2053,7 @@ def test_send_receive_locations(acfactory, lp):
assert locations[0].longitude == 3.0
assert locations[0].accuracy == 0.5
assert locations[0].timestamp > now
assert locations[0].marker == "🍞"
assert locations[0].marker is None
contact = ac2.create_contact(ac1)
locations2 = chat2.get_locations(contact=contact)
+1 -2
View File
@@ -41,12 +41,11 @@ skipsdist = True
skip_install = True
deps =
ruff
black
# pygments required by rst-lint
pygments
restructuredtext_lint
commands =
black --quiet --check --diff setup.py src/deltachat examples/ tests/
ruff format --quiet --diff setup.py src/deltachat examples/ tests/
ruff check src/deltachat tests/ examples/
rst-lint --encoding 'utf-8' README.rst
+1 -1
View File
@@ -1 +1 @@
2024-04-16
2024-05-18
+1 -1
View File
@@ -7,7 +7,7 @@ set -euo pipefail
#
# Avoid using rustup here as it depends on reading /proc/self/exe and
# has problems running under QEMU.
RUST_VERSION=1.77.1
RUST_VERSION=1.78.0
ARCH="$(uname -m)"
test -f "/lib/libc.musl-$ARCH.so.1" && LIBC=musl || LIBC=gnu
+5 -1
View File
@@ -66,7 +66,11 @@ def main():
parser = ArgumentParser(prog="set_core_version")
parser.add_argument("newversion")
json_list = ["package.json", "deltachat-jsonrpc/typescript/package.json"]
json_list = [
"package.json",
"deltachat-jsonrpc/typescript/package.json",
"deltachat-rpc-server/npm-package/package.json",
]
toml_list = [
"Cargo.toml",
"deltachat-ffi/Cargo.toml",
+8 -243
View File
@@ -14,7 +14,6 @@ use once_cell::sync::Lazy;
use crate::config::Config;
use crate::context::Context;
use crate::headerdef::HeaderDef;
use crate::tools::time;
/// `authres` is short for the Authentication-Results header, defined in
/// <https://datatracker.ietf.org/doc/html/rfc8601>, which contains info
@@ -29,45 +28,28 @@ pub(crate) async fn handle_authres(
context: &Context,
mail: &ParsedMail<'_>,
from: &str,
message_time: i64,
) -> Result<DkimResults> {
let from_domain = match EmailAddress::new(from) {
Ok(email) => email.domain,
Err(e) => {
// This email is invalid, but don't return an error, we still want to
// add a stub to the database so that it's not downloaded again
return Err(anyhow::format_err!("invalid email {}: {:#}", from, e));
}
};
let authres = parse_authres_headers(&mail.get_headers(), &from_domain);
update_authservid_candidates(context, &authres).await?;
compute_dkim_results(context, authres, &from_domain, message_time).await
compute_dkim_results(context, authres).await
}
#[derive(Debug)]
pub(crate) struct DkimResults {
/// Whether DKIM passed for this particular e-mail.
pub dkim_passed: bool,
/// Whether DKIM is known to work for e-mails coming from the sender's domain,
/// i.e. whether we expect DKIM to work.
pub dkim_should_work: bool,
/// Whether changing the public Autocrypt key should be allowed.
/// This is false if we expected DKIM to work (dkim_works=true),
/// but it failed now (dkim_passed=false).
pub allow_keychange: bool,
}
impl fmt::Display for DkimResults {
fn fmt(&self, fmt: &mut fmt::Formatter) -> fmt::Result {
write!(
fmt,
"DKIM Results: Passed={}, Works={}, Allow_Keychange={}",
self.dkim_passed, self.dkim_should_work, self.allow_keychange
)?;
if !self.allow_keychange {
write!(fmt, " KEYCHANGES NOT ALLOWED!!!!")?;
}
write!(fmt, "DKIM Results: Passed={}", self.dkim_passed)?;
Ok(())
}
}
@@ -218,10 +200,6 @@ async fn update_authservid_candidates(
context
.set_config_internal(Config::AuthservIdCandidates, Some(&new_config))
.await?;
// Updating the authservid candidates may mean that we now consider
// emails as "failed" which "passed" previously, so we need to
// reset our expectation which DKIMs work.
clear_dkim_works(context).await?
}
Ok(())
}
@@ -238,8 +216,6 @@ async fn update_authservid_candidates(
async fn compute_dkim_results(
context: &Context,
mut authres: ParsedAuthresHeaders,
from_domain: &str,
message_time: i64,
) -> Result<DkimResults> {
let mut dkim_passed = false;
@@ -272,71 +248,7 @@ async fn compute_dkim_results(
}
}
let last_working_timestamp = dkim_works_timestamp(context, from_domain).await?;
let mut dkim_should_work = dkim_should_work(last_working_timestamp)?;
if message_time > last_working_timestamp && dkim_passed {
set_dkim_works_timestamp(context, from_domain, message_time).await?;
dkim_should_work = true;
}
Ok(DkimResults {
dkim_passed,
dkim_should_work,
allow_keychange: dkim_passed || !dkim_should_work,
})
}
/// Whether DKIM in emails from this domain should be considered to work.
fn dkim_should_work(last_working_timestamp: i64) -> Result<bool> {
// When we get an email with valid DKIM-Authentication-Results,
// then we assume that DKIM works for 30 days from this time on.
let should_work_until = last_working_timestamp + 3600 * 24 * 30;
let dkim_ever_worked = last_working_timestamp > 0;
// We're using time() here and not the time when the message
// claims to have been sent (passed around as `message_time`)
// because otherwise an attacker could just put a time way
// in the future into the `Date` header and then we would
// assume that DKIM doesn't have to be valid anymore.
let dkim_should_work_now = should_work_until > time();
Ok(dkim_ever_worked && dkim_should_work_now)
}
async fn dkim_works_timestamp(context: &Context, from_domain: &str) -> Result<i64, anyhow::Error> {
let last_working_timestamp: i64 = context
.sql
.query_get_value(
"SELECT dkim_works FROM sending_domains WHERE domain=?",
(from_domain,),
)
.await?
.unwrap_or(0);
Ok(last_working_timestamp)
}
async fn set_dkim_works_timestamp(
context: &Context,
from_domain: &str,
timestamp: i64,
) -> Result<()> {
context
.sql
.execute(
"INSERT INTO sending_domains (domain, dkim_works) VALUES (?,?)
ON CONFLICT(domain) DO UPDATE SET dkim_works=excluded.dkim_works",
(from_domain, timestamp),
)
.await?;
Ok(())
}
async fn clear_dkim_works(context: &Context) -> Result<()> {
context
.sql
.execute("DELETE FROM sending_domains", ())
.await?;
Ok(())
Ok(DkimResults { dkim_passed })
}
fn parse_authservid_candidates_config(config: &Option<String>) -> BTreeSet<&str> {
@@ -349,19 +261,12 @@ fn parse_authservid_candidates_config(config: &Option<String>) -> BTreeSet<&str>
#[cfg(test)]
mod tests {
#![allow(clippy::indexing_slicing)]
use std::time::Duration;
use tokio::fs;
use tokio::io::AsyncReadExt;
use super::*;
use crate::aheader::EncryptPreference;
use crate::e2ee;
use crate::mimeparser;
use crate::peerstate::Peerstate;
use crate::securejoin::get_securejoin_qr;
use crate::securejoin::join_securejoin;
use crate::test_utils;
use crate::test_utils::TestContext;
use crate::test_utils::TestContextManager;
use crate::tools;
@@ -574,33 +479,8 @@ Authentication-Results: box.hispanilandia.net; spf=pass smtp.mailfrom=adbenitez@
let mail = mailparse::parse_mail(&bytes)?;
let from = &mimeparser::get_from(&mail.headers).unwrap().addr;
let res = handle_authres(&t, &mail, from, time()).await?;
assert!(res.allow_keychange);
}
for entry in &dir {
let mut file = fs::File::open(entry.path()).await?;
bytes.clear();
file.read_to_end(&mut bytes).await.unwrap();
let mail = mailparse::parse_mail(&bytes)?;
let from = &mimeparser::get_from(&mail.headers).unwrap().addr;
let res = handle_authres(&t, &mail, from, time()).await?;
if !res.allow_keychange {
println!(
"!!!!!! FAILURE Receiving {:?}, keychange is not allowed !!!!!!",
entry.path()
);
test_failed = true;
}
let res = handle_authres(&t, &mail, from).await?;
let from_domain = EmailAddress::new(from).unwrap().domain;
assert_eq!(
res.dkim_should_work,
dkim_should_work(dkim_works_timestamp(&t, &from_domain).await?)?
);
assert_eq!(res.dkim_passed, res.dkim_should_work);
// delta.blinzeln.de and gmx.de have invalid DKIM, so the DKIM check should fail
let expected_result = (from_domain != "delta.blinzeln.de") && (from_domain != "gmx.de")
@@ -613,9 +493,8 @@ Authentication-Results: box.hispanilandia.net; spf=pass smtp.mailfrom=adbenitez@
if res.dkim_passed != expected_result {
if authres_parsing_works {
println!(
"!!!!!! FAILURE Receiving {:?}, order {:#?} wrong result: !!!!!!",
"!!!!!! FAILURE Receiving {:?} wrong result: !!!!!!",
entry.path(),
dir.iter().map(|e| e.file_name()).collect::<Vec<_>>()
);
test_failed = true;
}
@@ -638,116 +517,7 @@ Authentication-Results: box.hispanilandia.net; spf=pass smtp.mailfrom=adbenitez@
let bytes = b"From: invalid@from.com
Authentication-Results: dkim=";
let mail = mailparse::parse_mail(bytes).unwrap();
handle_authres(&t, &mail, "invalid@rom.com", time())
.await
.unwrap();
}
#[ignore = "Disallowing keychanges is disabled for now"]
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_handle_authres_fails() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = tcm.alice().await;
let bob = tcm.bob().await;
// Bob sends Alice a message, so she gets his key
tcm.send_recv_accept(&bob, &alice, "Hi").await;
// We don't need bob anymore, let's make sure it's not accidentally used
drop(bob);
// Assume Alice receives an email from bob@example.net with
// correct DKIM -> `set_dkim_works()` was called
set_dkim_works_timestamp(&alice, "example.net", time()).await?;
// And Alice knows her server's authserv-id
alice
.set_config(Config::AuthservIdCandidates, Some("example.org"))
.await?;
tcm.section("An attacker, bob2, sends a from-forged email to Alice!");
// Sleep to make sure key reset is ignored because of DKIM failure
// and not because reordering is suspected.
tokio::time::sleep(std::time::Duration::from_millis(1100)).await;
let bob2 = tcm.unconfigured().await;
bob2.configure_addr("bob@example.net").await;
e2ee::ensure_secret_key_exists(&bob2).await?;
let chat = bob2.create_chat(&alice).await;
let mut sent = bob2
.send_text(chat.id, "Please send me lots of money")
.await;
sent.payload
.insert_str(0, "Authentication-Results: example.org; dkim=fail\n");
let received = alice.recv_msg(&sent).await;
// Assert that the error tells the user about the problem
assert!(received.error.unwrap().contains("DKIM failed"));
let bob_state = Peerstate::from_addr(&alice, "bob@example.net")
.await?
.unwrap();
// Encryption preference is still mutual.
assert_eq!(bob_state.prefer_encrypt, EncryptPreference::Mutual);
// Also check that the keypair was not changed
assert_eq!(
bob_state.public_key.unwrap(),
test_utils::bob_keypair().public
);
// Since Alice didn't change the key, Bob can't read her message
let received = tcm
.try_send_recv(&alice, &bob2, "My credit card number is 1234")
.await;
assert!(!received.text.contains("1234"));
assert!(received.error.is_some());
tcm.section("Turns out bob2 wasn't an attacker at all, Bob just has a new phone and DKIM just stopped working.");
tcm.section("To fix the key problems, Bob scans Alice's QR code.");
let qr = get_securejoin_qr(&alice.ctx, None).await.unwrap();
join_securejoin(&bob2.ctx, &qr).await.unwrap();
loop {
if let Some(mut sent) = bob2.pop_sent_msg_opt(Duration::ZERO).await {
sent.payload
.insert_str(0, "Authentication-Results: example.org; dkim=fail\n");
alice.recv_msg(&sent).await;
} else if let Some(sent) = alice.pop_sent_msg_opt(Duration::ZERO).await {
bob2.recv_msg(&sent).await;
} else {
break;
}
}
// Unfortunately, securejoin currently doesn't work with authres-checking,
// so these checks would fail:
// let contact_bob = alice.add_or_lookup_contact(&bob2).await;
// assert_eq!(
// contact_bob.is_verified(&alice.ctx).await.unwrap(),
// VerifiedStatus::BidirectVerified
// );
// let contact_alice = bob2.add_or_lookup_contact(&alice).await;
// assert_eq!(
// contact_alice.is_verified(&bob2.ctx).await.unwrap(),
// VerifiedStatus::BidirectVerified
// );
// // Bob can read Alice's messages again
// let received = tcm
// .try_send_recv(&alice, &bob2, "Can you read this again?")
// .await;
// assert_eq!(received.text.as_ref().unwrap(), "Can you read this again?");
// assert!(received.error.is_none());
Ok(())
handle_authres(&t, &mail, "invalid@rom.com").await.unwrap();
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
@@ -796,10 +566,7 @@ Authentication-Results: dkim=";
let alice = tcm.alice().await;
let bob = tcm.bob().await;
// Assume Bob received an email from something@example.net with
// correct DKIM -> `set_dkim_works()` was called
set_dkim_works_timestamp(&bob, "example.org", time()).await?;
// And Bob knows his server's authserv-id
// Bob knows his server's authserv-id
bob.set_config(Config::AuthservIdCandidates, Some("example.net"))
.await?;
@@ -821,15 +588,13 @@ Authentication-Results: dkim=";
.insert_str(0, "Authentication-Results: example.net; dkim=fail\n");
let rcvd = bob.recv_msg(&sent).await;
// Disallowing keychanges is disabled for now:
// assert!(rcvd.error.unwrap().contains("DKIM failed"));
// The message info should contain a warning:
assert!(rcvd
.id
.get_info(&bob)
.await
.unwrap()
.contains("KEYCHANGES NOT ALLOWED"));
.contains("DKIM Results: Passed=false"));
Ok(())
}
+73 -9
View File
@@ -698,7 +698,10 @@ fn encode_img(
ImageOutputFormat::Png => img.write_to(&mut buf, ImageFormat::Png)?,
ImageOutputFormat::Jpeg { quality } => {
let encoder = JpegEncoder::new_with_quality(&mut buf, quality);
img.write_with_encoder(encoder)?;
// Convert image into RGB8 to avoid the error
// "The encoder or decoder for Jpeg does not support the color type Rgba8"
// (<https://github.com/image-rs/image/issues/2211>).
img.clone().into_rgb8().write_with_encoder(encoder)?;
}
}
Ok(())
@@ -1205,6 +1208,28 @@ mod tests {
.unwrap();
}
/// Tests that RGBA PNG can be recoded into JPEG
/// by dropping alpha channel.
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_recode_image_rgba_png_to_jpeg() {
let bytes = include_bytes!("../test-data/image/screenshot-rgba.png");
send_image_check_mediaquality(
Viewtype::Image,
Some("1"),
bytes,
"png",
false, // no Exif
1920,
1080,
0,
constants::WORSE_IMAGE_SIZE,
constants::WORSE_IMAGE_SIZE * 1080 / 1920,
)
.await
.unwrap();
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_recode_image_huge_jpg() {
let bytes = include_bytes!("../test-data/image/screenshot.jpg");
@@ -1282,26 +1307,65 @@ mod tests {
let alice_msg = alice.get_last_msg().await;
assert_eq!(alice_msg.get_width() as u32, compressed_width);
assert_eq!(alice_msg.get_height() as u32, compressed_height);
check_image_size(
alice_msg.get_file(&alice).unwrap(),
compressed_width,
compressed_height,
);
let file_saved = alice
.get_blobdir()
.join("saved-".to_string() + &alice_msg.get_filename().unwrap());
alice_msg.save_file(&alice, &file_saved).await?;
check_image_size(file_saved, compressed_width, compressed_height);
let bob_msg = bob.recv_msg(&sent).await;
assert_eq!(bob_msg.get_viewtype(), Viewtype::Image);
assert_eq!(bob_msg.get_width() as u32, compressed_width);
assert_eq!(bob_msg.get_height() as u32, compressed_height);
let file = bob_msg.get_file(&bob).unwrap();
let file_saved = bob
.get_blobdir()
.join("saved-".to_string() + &bob_msg.get_filename().unwrap());
bob_msg.save_file(&bob, &file_saved).await?;
let blob = BlobObject::new_from_path(&bob, &file).await?;
let blob = BlobObject::new_from_path(&bob, &file_saved).await?;
let (_, exif) = blob.metadata()?;
assert!(exif.is_none());
let img = check_image_size(file, compressed_width, compressed_height);
let img = check_image_size(file_saved, compressed_width, compressed_height);
Ok(img)
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_send_big_gif_as_image() -> Result<()> {
let bytes = include_bytes!("../test-data/image/screenshot.gif");
let (width, height) = (1920u32, 1080u32);
let alice = TestContext::new_alice().await;
let bob = TestContext::new_bob().await;
alice
.set_config(
Config::MediaQuality,
Some(&(MediaQuality::Worse as i32).to_string()),
)
.await?;
let file = alice.get_blobdir().join("file").with_extension("gif");
fs::write(&file, &bytes)
.await
.context("failed to write file")?;
let mut msg = Message::new(Viewtype::Image);
msg.set_file(file.to_str().unwrap(), None);
let chat = alice.create_chat(&bob).await;
let sent = alice.send_msg(chat.id, &mut msg).await;
let bob_msg = bob.recv_msg(&sent).await;
// DC must detect the image as GIF and send it w/o reencoding.
assert_eq!(bob_msg.get_viewtype(), Viewtype::Gif);
assert_eq!(bob_msg.get_width() as u32, width);
assert_eq!(bob_msg.get_height() as u32, height);
let file_saved = bob
.get_blobdir()
.join("saved-".to_string() + &bob_msg.get_filename().unwrap());
bob_msg.save_file(&bob, &file_saved).await?;
let blob = BlobObject::new_from_path(&bob, &file_saved).await?;
let (file_size, _) = blob.metadata()?;
assert_eq!(file_size, bytes.len() as u64);
check_image_size(file_saved, width, height);
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_increation_in_blobdir() -> Result<()> {
let t = TestContext::new_alice().await;
+242 -44
View File
@@ -12,6 +12,7 @@ use deltachat_contact_tools::{strip_rtlo_characters, ContactAddress};
use deltachat_derive::{FromSql, ToSql};
use serde::{Deserialize, Serialize};
use strum_macros::EnumIter;
use tokio::task;
use crate::aheader::EncryptPreference;
use crate::blob::BlobObject;
@@ -38,6 +39,7 @@ use crate::mimeparser::SystemMessage;
use crate::param::{Param, Params};
use crate::peerstate::Peerstate;
use crate::receive_imf::ReceivedMsg;
use crate::securejoin::BobState;
use crate::smtp::send_msg_to_smtp;
use crate::sql;
use crate::stock_str;
@@ -126,6 +128,10 @@ pub(crate) enum CantSendReason {
/// Not a member of the chat.
NotAMember,
/// Temporary state for 1:1 chats while SecureJoin is in progress, after a timeout sending
/// messages (incl. unencrypted if we don't yet know the contact's pubkey) is allowed.
SecurejoinWait,
}
impl fmt::Display for CantSendReason {
@@ -145,6 +151,7 @@ impl fmt::Display for CantSendReason {
write!(f, "mailing list does not have a know post address")
}
Self::NotAMember => write!(f, "not a member of the chat"),
Self::SecurejoinWait => write!(f, "awaiting SecureJoin for 1:1 chat"),
}
}
}
@@ -610,7 +617,10 @@ impl ChatId {
let sort_to_bottom = true;
let ts = self
.calc_sort_timestamp(context, timestamp_sent, sort_to_bottom, false)
.await?;
.await?
// Always sort protection messages below `SystemMessage::SecurejoinWait{,Timeout}` ones
// in case of race conditions.
.saturating_add(1);
self.set_protection_for_timestamp_sort(context, protect, ts, contact_id)
.await
}
@@ -1407,6 +1417,18 @@ impl ChatId {
Ok(sort_timestamp)
}
/// Spawns a task checking after a timeout whether the SecureJoin has finished for the 1:1 chat
/// and otherwise notifying the user accordingly.
pub(crate) fn spawn_securejoin_wait(self, context: &Context, timeout: u64) {
let context = context.clone();
task::spawn(async move {
tokio::time::sleep(Duration::from_secs(timeout)).await;
let chat = Chat::load_from_db(&context, self).await?;
chat.check_securejoin_wait(&context, 0).await?;
Result::<()>::Ok(())
});
}
}
impl std::fmt::Display for ChatId {
@@ -1586,6 +1608,12 @@ impl Chat {
Some(ReadOnlyMailingList)
} else if !self.is_self_in_chat(context).await? {
Some(NotAMember)
} else if self
.check_securejoin_wait(context, constants::SECUREJOIN_WAIT_TIMEOUT)
.await?
> 0
{
Some(SecurejoinWait)
} else {
None
};
@@ -1599,6 +1627,69 @@ impl Chat {
Ok(self.why_cant_send(context).await?.is_none())
}
/// Returns the remaining timeout for the 1:1 chat in-progress SecureJoin.
///
/// If the timeout has expired, notifies the user that sending messages is possible. See also
/// [`CantSendReason::SecurejoinWait`].
pub(crate) async fn check_securejoin_wait(
&self,
context: &Context,
timeout: u64,
) -> Result<u64> {
if self.typ != Chattype::Single || self.protected != ProtectionStatus::Unprotected {
return Ok(0);
}
let (mut param0, mut param1) = (Params::new(), Params::new());
param0.set_cmd(SystemMessage::SecurejoinWait);
param1.set_cmd(SystemMessage::SecurejoinWaitTimeout);
let (param0, param1) = (param0.to_string(), param1.to_string());
let Some((param, ts_sort, ts_start)) = context
.sql
.query_row_optional(
"SELECT param, timestamp, timestamp_sent FROM msgs WHERE id=\
(SELECT MAX(id) FROM msgs WHERE chat_id=? AND param IN (?, ?))",
(self.id, &param0, &param1),
|row| {
let param: String = row.get(0)?;
let ts_sort: i64 = row.get(1)?;
let ts_start: i64 = row.get(2)?;
Ok((param, ts_sort, ts_start))
},
)
.await?
else {
return Ok(0);
};
if param == param1 {
return Ok(0);
}
let now = time();
// Don't await SecureJoin if the clock was set back.
if ts_start <= now {
let timeout = ts_start
.saturating_add(timeout.try_into()?)
.saturating_sub(now);
if timeout > 0 {
return Ok(timeout as u64);
}
}
add_info_msg_with_cmd(
context,
self.id,
&stock_str::securejoin_wait_timeout(context).await,
SystemMessage::SecurejoinWaitTimeout,
// Use the sort timestamp of the "please wait" message, this way the added message is
// never sorted below the protection message if the SecureJoin finishes in parallel.
ts_sort,
Some(now),
None,
None,
)
.await?;
context.emit_event(EventType::ChatModified(self.id));
Ok(0)
}
/// Checks if the user is part of a chat
/// and has basically the permissions to edit the chat therefore.
/// The function does not check if the chat type allows editing of concrete elements.
@@ -2235,8 +2326,9 @@ pub struct ChatInfo {
}
pub(crate) async fn update_saved_messages_icon(context: &Context) -> Result<()> {
// if there is no saved-messages chat, there is nothing to update. this is no error.
if let Some(chat_id) = ChatId::lookup_by_contact(context, ContactId::SELF).await? {
if let Some(ChatIdBlocked { id: chat_id, .. }) =
ChatIdBlocked::lookup_by_contact(context, ContactId::SELF).await?
{
let icon = include_bytes!("../assets/icon-saved-messages.png");
let blob = BlobObject::create(context, "icon-saved-messages.png", icon).await?;
let icon = blob.as_name().to_string();
@@ -2249,8 +2341,9 @@ pub(crate) async fn update_saved_messages_icon(context: &Context) -> Result<()>
}
pub(crate) async fn update_device_icon(context: &Context) -> Result<()> {
// if there is no device-chat, there is nothing to update. this is no error.
if let Some(chat_id) = ChatId::lookup_by_contact(context, ContactId::DEVICE).await? {
if let Some(ChatIdBlocked { id: chat_id, .. }) =
ChatIdBlocked::lookup_by_contact(context, ContactId::DEVICE).await?
{
let icon = include_bytes!("../assets/icon-device.png");
let blob = BlobObject::create(context, "icon-device.png", icon).await?;
let icon = blob.as_name().to_string();
@@ -2301,7 +2394,9 @@ async fn update_special_chat_name(
contact_id: ContactId,
name: String,
) -> Result<()> {
if let Some(chat_id) = ChatId::lookup_by_contact(context, contact_id).await? {
if let Some(ChatIdBlocked { id: chat_id, .. }) =
ChatIdBlocked::lookup_by_contact(context, contact_id).await?
{
// the `!= name` condition avoids unneeded writes
context
.sql
@@ -2330,6 +2425,26 @@ pub(crate) async fn update_special_chat_names(context: &Context) -> Result<()> {
Ok(())
}
/// Checks if there is a 1:1 chat in-progress SecureJoin for Bob and, if necessary, schedules a task
/// unblocking the chat and notifying the user accordingly.
pub(crate) async fn resume_securejoin_wait(context: &Context) -> Result<()> {
let Some(bobstate) = BobState::from_db(&context.sql).await? else {
return Ok(());
};
if !bobstate.in_progress() {
return Ok(());
}
let chat_id = bobstate.alice_chat();
let chat = Chat::load_from_db(context, chat_id).await?;
let timeout = chat
.check_securejoin_wait(context, constants::SECUREJOIN_WAIT_TIMEOUT)
.await?;
if timeout > 0 {
chat_id.spawn_securejoin_wait(context, timeout);
}
Ok(())
}
/// Handle a [`ChatId`] and its [`Blocked`] status at once.
///
/// This struct is an optimisation to read a [`ChatId`] and its [`Blocked`] status at once
@@ -2505,6 +2620,30 @@ async fn prepare_msg_blob(context: &Context, msg: &mut Message) -> Result<()> {
.await?
.with_context(|| format!("attachment missing for message of type #{}", msg.viewtype))?;
if msg.viewtype == Viewtype::File || msg.viewtype == Viewtype::Image {
// Correct the type, take care not to correct already very special
// formats as GIF or VOICE.
//
// Typical conversions:
// - from FILE to AUDIO/VIDEO/IMAGE
// - from FILE/IMAGE to GIF */
if let Some((better_type, _)) = message::guess_msgtype_from_suffix(&blob.to_abs_path())
{
if better_type != Viewtype::Webxdc
|| context
.ensure_sendable_webxdc_file(&blob.to_abs_path())
.await
.is_ok()
{
msg.viewtype = better_type;
}
}
} else if msg.viewtype == Viewtype::Webxdc {
context
.ensure_sendable_webxdc_file(&blob.to_abs_path())
.await?;
}
let mut maybe_sticker = msg.viewtype == Viewtype::Sticker;
if msg.viewtype == Viewtype::Image
|| maybe_sticker && !msg.param.exists(Param::ForceSticker)
@@ -2526,34 +2665,6 @@ async fn prepare_msg_blob(context: &Context, msg: &mut Message) -> Result<()> {
.set(Param::Filename, stem.to_string() + "." + blob_ext);
}
if msg.viewtype == Viewtype::File || msg.viewtype == Viewtype::Image {
// Correct the type, take care not to correct already very special
// formats as GIF or VOICE.
//
// Typical conversions:
// - from FILE to AUDIO/VIDEO/IMAGE
// - from FILE/IMAGE to GIF */
if let Some((better_type, better_mime)) =
message::guess_msgtype_from_suffix(&blob.to_abs_path())
{
if better_type != Viewtype::Webxdc
|| context
.ensure_sendable_webxdc_file(&blob.to_abs_path())
.await
.is_ok()
{
msg.viewtype = better_type;
if !msg.param.exists(Param::MimeType) {
msg.param.set(Param::MimeType, better_mime);
}
}
}
} else if msg.viewtype == Viewtype::Webxdc {
context
.ensure_sendable_webxdc_file(&blob.to_abs_path())
.await?;
}
if !msg.param.exists(Param::MimeType) {
if let Some((_, mime)) = message::guess_msgtype_from_suffix(&blob.to_abs_path()) {
msg.param.set(Param::MimeType, mime);
@@ -2587,7 +2698,9 @@ async fn prepare_msg_common(
if let Some(reason) = chat.why_cant_send(context).await? {
if matches!(
reason,
CantSendReason::ProtectionBroken | CantSendReason::ContactRequest
CantSendReason::ProtectionBroken
| CantSendReason::ContactRequest
| CantSendReason::SecurejoinWait
) && msg.param.get_cmd() == SystemMessage::SecurejoinMessage
{
// Send out the message, the securejoin message is supposed to repair the verification.
@@ -2597,6 +2710,18 @@ async fn prepare_msg_common(
}
}
// Check a quote reply is not leaking data from other chats.
// This is meant as a last line of defence, the UI should check that before as well.
// (We allow Chattype::Single in general for "Reply Privately";
// checking for exact contact_id will produce false positives when ppl just left the group)
if chat.typ != Chattype::Single && !context.get_config_bool(Config::Bot).await? {
if let Some(quoted_message) = msg.quoted_message(context).await? {
if quoted_message.chat_id != chat_id {
bail!("Bad quote reply");
}
}
}
// check current MessageState for drafts (to keep msg_id) ...
let update_msg_id = if msg.state == MessageState::OutDraft {
msg.hidden = false;
@@ -2839,17 +2964,10 @@ pub(crate) async fn create_send_msg_jobs(context: &Context, msg: &mut Message) -
.await?;
}
if let Some(last_added_location_id) = rendered_msg.last_added_location_id {
if rendered_msg.last_added_location_id.is_some() {
if let Err(err) = location::set_kml_sent_timestamp(context, msg.chat_id, now).await {
error!(context, "Failed to set kml sent_timestamp: {err:#}.");
}
if !msg.hidden {
if let Err(err) =
location::set_msg_location_id(context, msg.id, last_added_location_id).await
{
error!(context, "Failed to set msg_location_id: {err:#}.");
}
}
}
if let Some(sync_ids) = rendered_msg.sync_ids_to_delete {
@@ -4475,9 +4593,10 @@ impl Context {
}
_ => (),
}
ChatId::lookup_by_contact(self, contact_id)
ChatIdBlocked::lookup_by_contact(self, contact_id)
.await?
.with_context(|| format!("No chat for addr '{addr}'"))?
.id
}
SyncId::Grpid(grpid) => {
if let SyncAction::CreateBroadcast(name) = action {
@@ -4724,6 +4843,59 @@ mod tests {
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_quote_replies() -> Result<()> {
let alice = TestContext::new_alice().await;
let bob = TestContext::new_bob().await;
let grp_chat_id = create_group_chat(&alice, ProtectionStatus::Unprotected, "grp").await?;
let grp_msg_id = send_text_msg(&alice, grp_chat_id, "bar".to_string()).await?;
let grp_msg = Message::load_from_db(&alice, grp_msg_id).await?;
let one2one_chat_id = alice.create_chat(&bob).await.id;
let one2one_msg_id = send_text_msg(&alice, one2one_chat_id, "foo".to_string()).await?;
let one2one_msg = Message::load_from_db(&alice, one2one_msg_id).await?;
// quoting messages in same chat is okay
let mut msg = Message::new(Viewtype::Text);
msg.set_text("baz".to_string());
msg.set_quote(&alice, Some(&grp_msg)).await?;
let result = send_msg(&alice, grp_chat_id, &mut msg).await;
assert!(result.is_ok());
let mut msg = Message::new(Viewtype::Text);
msg.set_text("baz".to_string());
msg.set_quote(&alice, Some(&one2one_msg)).await?;
let result = send_msg(&alice, one2one_chat_id, &mut msg).await;
assert!(result.is_ok());
let one2one_quote_reply_msg_id = result.unwrap();
// quoting messages from groups to one-to-ones is okay ("reply privately")
let mut msg = Message::new(Viewtype::Text);
msg.set_text("baz".to_string());
msg.set_quote(&alice, Some(&grp_msg)).await?;
let result = send_msg(&alice, one2one_chat_id, &mut msg).await;
assert!(result.is_ok());
// quoting messages from one-to-one chats in groups is an error; usually this is also not allowed by UI at all ...
let mut msg = Message::new(Viewtype::Text);
msg.set_text("baz".to_string());
msg.set_quote(&alice, Some(&one2one_msg)).await?;
let result = send_msg(&alice, grp_chat_id, &mut msg).await;
assert!(result.is_err());
// ... but forwarding messages with quotes is allowed
let result = forward_msgs(&alice, &[one2one_quote_reply_msg_id], grp_chat_id).await;
assert!(result.is_ok());
// ... and bots are not restricted
alice.set_config(Config::Bot, Some("1")).await?;
let result = send_msg(&alice, grp_chat_id, &mut msg).await;
assert!(result.is_ok());
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_add_contact_to_chat_ex_add_self() {
// Adding self to a contact should succeed, even though it's pointless.
@@ -5038,6 +5210,32 @@ mod tests {
Ok(())
}
/// Tests that if member added message is completely lost,
/// member is eventually added.
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_lost_member_added() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
let bob = &tcm.bob().await;
let alice_chat_id = alice
.create_group_with_members(ProtectionStatus::Unprotected, "Group", &[bob])
.await;
let alice_sent = alice.send_text(alice_chat_id, "Hi!").await;
let bob_chat_id = bob.recv_msg(&alice_sent).await.chat_id;
assert_eq!(get_chat_contacts(bob, bob_chat_id).await?.len(), 2);
// Attempt to add member, but message is lost.
let claire_id = Contact::create(alice, "", "claire@foo.de").await?;
add_contact_to_chat(alice, alice_chat_id, claire_id).await?;
alice.pop_sent_msg().await;
let alice_sent = alice.send_text(alice_chat_id, "Hi again!").await;
bob.recv_msg(&alice_sent).await;
assert_eq!(get_chat_contacts(bob, bob_chat_id).await?.len(), 3);
Ok(())
}
/// Test that group updates are robust to lost messages and eventual out of order arrival.
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_modify_chat_lost() -> Result<()> {
+2 -1
View File
@@ -31,10 +31,11 @@ fn rgb_to_u32((r, g, b): (f64, f64, f64)) -> u32 {
///
/// Saturation is set to maximum (100.0) to make colors distinguishable, and lightness is set to
/// half (50.0) to make colors suitable both for light and dark theme.
pub(crate) fn str_to_color(s: &str) -> u32 {
pub fn str_to_color(s: &str) -> u32 {
rgb_to_u32(hsluv_to_rgb((str_to_angle(s), 100.0, 50.0)))
}
/// Returns color as a "#RRGGBB" `String` where R, G, B are hex digits.
pub fn color_int_to_hex_string(color: u32) -> String {
format!("{color:#08x}").replace("0x", "#")
}
+16 -1
View File
@@ -9,7 +9,7 @@ use base64::Engine as _;
use deltachat_contact_tools::addr_cmp;
use serde::{Deserialize, Serialize};
use strum::{EnumProperty, IntoEnumIterator};
use strum_macros::{AsRefStr, Display, EnumIter, EnumProperty, EnumString};
use strum_macros::{AsRefStr, Display, EnumIter, EnumString};
use tokio::fs;
use crate::blob::BlobObject;
@@ -254,6 +254,9 @@ pub enum Config {
/// True if account is configured.
Configured,
/// True if account is a chatmail account.
IsChatmail,
/// All secondary self addresses separated by spaces
/// (`addr1@example.org addr2@example.org addr3@example.org`)
SecondaryAddrs,
@@ -359,6 +362,9 @@ pub enum Config {
/// MsgId of webxdc map integration.
WebxdcIntegration,
/// Iroh secret key.
IrohSecretKey,
}
impl Config {
@@ -1007,6 +1013,15 @@ mod tests {
.set_config(Config::Selfavatar, Some(file.to_str().unwrap()))
.await?;
sync(&alice0, &alice1).await;
// There was a bug that a sync message creates the self-chat with the user avatar instead of
// the special icon and that remains so when the self-chat becomes user-visible. Let's check
// this.
let self_chat = alice0.get_self_chat().await;
let self_chat_avatar_path = self_chat.get_profile_image(&alice0).await?.unwrap();
assert_eq!(
self_chat_avatar_path,
alice0.get_blobdir().join("icon-saved-messages.png")
);
assert!(alice1
.get_config(Config::Selfavatar)
.await?
+13
View File
@@ -112,6 +112,11 @@ impl Context {
let mut param = LoginParam::load_candidate_params(self).await?;
let old_addr = self.get_config(Config::ConfiguredAddr).await?;
// Reset our knowledge about whether the server is a chatmail server.
// We will update it when we connect to IMAP.
self.set_config_internal(Config::IsChatmail, None).await?;
let success = configure(self, &mut param).await;
self.set_config_internal(Config::NotifyAboutWrongPw, None)
.await?;
@@ -453,6 +458,14 @@ async fn configure(ctx: &Context, param: &mut LoginParam) -> Result<()> {
progress!(ctx, 900);
if imap_session.is_chatmail() {
ctx.set_config(Config::SentboxWatch, None).await?;
ctx.set_config(Config::MvboxMove, Some("0")).await?;
ctx.set_config(Config::OnlyFetchMvbox, None).await?;
ctx.set_config(Config::ShowEmails, None).await?;
ctx.set_config(Config::E2eeEnabled, Some("1")).await?;
}
let create_mvbox = ctx.should_watch_mvbox().await?;
imap.configure_folders(ctx, &mut imap_session, create_mvbox)
+5
View File
@@ -223,6 +223,11 @@ pub(crate) const DC_BACKGROUND_FETCH_QUOTA_CHECK_RATELIMIT: u64 = 12 * 60 * 60;
/// in the group membership consistency algo to reject outdated membership changes.
pub(crate) const TIMESTAMP_SENT_TOLERANCE: i64 = 60;
/// How long a 1:1 chat can't be used for sending while the SecureJoin is in progress. This should
/// be 10-20 seconds so that we are reasonably sure that the app remains active and receiving also
/// on mobile devices. See also [`crate::chat::CantSendReason::SecurejoinWait`].
pub(crate) const SECUREJOIN_WAIT_TIMEOUT: u64 = 15;
#[cfg(test)]
mod tests {
use num_traits::FromPrimitive;
+98 -5
View File
@@ -8,10 +8,11 @@ use std::time::UNIX_EPOCH;
use anyhow::{bail, ensure, Context as _, Result};
use async_channel::{self as channel, Receiver, Sender};
use base64::Engine as _;
pub use deltachat_contact_tools::may_be_valid_addr;
use deltachat_contact_tools::{
addr_cmp, addr_normalize, normalize_name, sanitize_name_and_addr, strip_rtlo_characters,
ContactAddress,
self as contact_tools, addr_cmp, addr_normalize, normalize_name, sanitize_name_and_addr,
strip_rtlo_characters, ContactAddress, VcardContact,
};
use deltachat_derive::{FromSql, ToSql};
use rusqlite::OptionalExtension;
@@ -20,7 +21,7 @@ use tokio::task;
use tokio::time::{timeout, Duration};
use crate::aheader::EncryptPreference;
use crate::chat::{ChatId, ProtectionStatus};
use crate::chat::{ChatId, ChatIdBlocked, ProtectionStatus};
use crate::color::str_to_color;
use crate::config::Config;
use crate::constants::{Blocked, Chattype, DC_GCL_ADD_SELF, DC_GCL_VERIFIED_ONLY};
@@ -159,6 +160,35 @@ impl rusqlite::types::FromSql for ContactId {
}
}
/// Returns a vCard containing contacts with the given ids.
pub async fn make_vcard(context: &Context, contacts: &[ContactId]) -> Result<String> {
let now = time();
let mut vcard_contacts = Vec::with_capacity(contacts.len());
for id in contacts {
let c = Contact::get_by_id(context, *id).await?;
let key = Peerstate::from_addr(context, &c.addr)
.await?
.and_then(|peerstate| peerstate.peek_key(false).map(|k| k.to_base64()));
let profile_image = match c.get_profile_image(context).await? {
None => None,
Some(path) => tokio::fs::read(path)
.await
.log_err(context)
.ok()
.map(|data| base64::engine::general_purpose::STANDARD.encode(data)),
};
vcard_contacts.push(VcardContact {
addr: c.addr,
authname: c.authname,
key,
profile_image,
// Use the current time to not reveal our or contact's online time.
timestamp: Ok(now),
});
}
Ok(contact_tools::make_vcard(&vcard_contacts))
}
/// An object representing a single contact in memory.
///
/// The contact object is not updated.
@@ -1315,7 +1345,9 @@ impl Contact {
pub async fn is_profile_verified(&self, context: &Context) -> Result<bool> {
let contact_id = self.id;
if let Some(chat_id) = ChatId::lookup_by_contact(context, contact_id).await? {
if let Some(ChatIdBlocked { id: chat_id, .. }) =
ChatIdBlocked::lookup_by_contact(context, contact_id).await?
{
Ok(chat_id.is_protected(context).await? == ProtectionStatus::Protected)
} else {
// 1:1 chat does not exist.
@@ -2708,7 +2740,19 @@ Hi."#;
bob.recv_msg(&sent_msg).await;
let contact = Contact::get_by_id(&bob, *contacts.first().unwrap()).await?;
assert!(contact.was_seen_recently());
let green = ansi_term::Color::Green.normal();
assert!(
contact.was_seen_recently(),
"{}",
green.paint(
"\nNOTE: This test failure is probably a false-positive, caused by tests running in parallel.
The issue is that `SystemTime::shift()` (a utility function for tests) changes the time for all threads doing tests, and not only for the running test.
Until the false-positive is fixed:
- Use `cargo test -- --test-threads 1` instead of `cargo test`
- Or use `cargo nextest run` (install with `cargo install cargo-nextest --locked`)\n"
)
);
let self_contact = Contact::get_by_id(&bob, ContactId::SELF).await?;
assert!(!self_contact.was_seen_recently());
@@ -2803,4 +2847,53 @@ Hi."#;
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_make_vcard() -> Result<()> {
let alice = &TestContext::new_alice().await;
let bob = &TestContext::new_bob().await;
bob.set_config(Config::Displayname, Some("Bob")).await?;
let avatar_path = bob.dir.path().join("avatar.png");
let avatar_bytes = include_bytes!("../test-data/image/avatar64x64.png");
let avatar_base64 = base64::engine::general_purpose::STANDARD.encode(avatar_bytes);
tokio::fs::write(&avatar_path, avatar_bytes).await?;
bob.set_config(Config::Selfavatar, Some(avatar_path.to_str().unwrap()))
.await?;
let bob_addr = bob.get_config(Config::Addr).await?.unwrap();
let chat = bob.create_chat(alice).await;
let sent_msg = bob.send_text(chat.id, "moin").await;
alice.recv_msg(&sent_msg).await;
let bob_id = Contact::create(alice, "Some Bob", &bob_addr).await?;
let key_base64 = Peerstate::from_addr(alice, &bob_addr)
.await?
.unwrap()
.peek_key(false)
.unwrap()
.to_base64();
let fiona_id = Contact::create(alice, "Fiona", "fiona@example.net").await?;
assert_eq!(make_vcard(alice, &[]).await?, "".to_string());
let t0 = time();
let vcard = make_vcard(alice, &[bob_id, fiona_id]).await?;
let t1 = time();
// Just test that it's parsed as expected, `deltachat_contact_tools` crate has tests on the
// exact format.
let contacts = contact_tools::parse_vcard(&vcard);
assert_eq!(contacts.len(), 2);
assert_eq!(contacts[0].addr, bob_addr);
assert_eq!(contacts[0].authname, "Bob".to_string());
assert_eq!(contacts[0].key, Some(key_base64));
assert_eq!(contacts[0].profile_image, Some(avatar_base64));
let timestamp = *contacts[0].timestamp.as_ref().unwrap();
assert!(t0 <= timestamp && timestamp <= t1);
assert_eq!(contacts[1].addr, "fiona@example.net".to_string());
assert_eq!(contacts[1].authname, "".to_string());
assert_eq!(contacts[1].key, None);
assert_eq!(contacts[1].profile_image, None);
let timestamp = *contacts[1].timestamp.as_ref().unwrap();
assert!(t0 <= timestamp && timestamp <= t1);
Ok(())
}
}
+21 -13
View File
@@ -12,7 +12,7 @@ use anyhow::{bail, ensure, Context as _, Result};
use async_channel::{self as channel, Receiver, Sender};
use pgp::SignedPublicKey;
use ratelimit::Ratelimit;
use tokio::sync::{Mutex, Notify, RwLock};
use tokio::sync::{Mutex, Notify, OnceCell, RwLock};
use crate::aheader::EncryptPreference;
use crate::chat::{get_chat_cnt, ChatId, ProtectionStatus};
@@ -30,6 +30,7 @@ use crate::key::{load_self_public_key, load_self_secret_key, DcKey as _};
use crate::login_param::LoginParam;
use crate::message::{self, Message, MessageState, MsgId, Viewtype};
use crate::param::{Param, Params};
use crate::peer_channels::Iroh;
use crate::peerstate::Peerstate;
use crate::push::PushSubscriber;
use crate::quota::QuotaInfo;
@@ -288,6 +289,9 @@ pub struct InnerContext {
/// True if account has subscribed to push notifications via IMAP.
pub(crate) push_subscribed: AtomicBool,
/// Iroh for realtime peer channels.
pub(crate) iroh: OnceCell<Iroh>,
}
/// The state of ongoing process.
@@ -445,6 +449,7 @@ impl Context {
debug_logging: std::sync::RwLock::new(None),
push_subscriber,
push_subscribed: AtomicBool::new(false),
iroh: OnceCell::new(),
};
let ctx = Context {
@@ -461,18 +466,10 @@ impl Context {
return;
}
{
if self
.get_config(Config::ConfiguredAddr)
.await
.unwrap_or_default()
.filter(|s| s.ends_with(".testrun.org"))
.is_some()
{
let mut lock = self.ratelimit.write().await;
// Allow at least 1 message every second + a burst of 3.
*lock = Ratelimit::new(Duration::new(3, 0), 3.0);
}
if self.is_chatmail().await.unwrap_or_default() {
let mut lock = self.ratelimit.write().await;
// Allow at least 1 message every second + a burst of 3.
*lock = Ratelimit::new(Duration::new(3, 0), 3.0);
}
self.scheduler.start(self.clone()).await;
}
@@ -490,9 +487,17 @@ impl Context {
/// Indicate that the network likely has come back.
pub async fn maybe_network(&self) {
if let Some(iroh) = self.iroh.get() {
iroh.network_change().await;
}
self.scheduler.maybe_network().await;
}
/// Returns true if an account is on a chatmail server.
pub async fn is_chatmail(&self) -> Result<bool> {
self.get_config_bool(Config::IsChatmail).await
}
/// Does a background fetch
/// pauses the scheduler and does one imap fetch, then unpauses and returns
pub async fn background_fetch(&self) -> Result<()> {
@@ -799,6 +804,8 @@ impl Context {
res.insert("imap_server_id", format!("{server_id:?}"));
}
res.insert("is_chatmail", self.is_chatmail().await?.to_string());
if let Some(metadata) = &*self.metadata.read().await {
if let Some(comment) = &metadata.comment {
res.insert("imap_server_comment", format!("{comment:?}"));
@@ -1649,6 +1656,7 @@ mod tests {
"socks5_password",
"key_id",
"webxdc_integration",
"iroh_secret_key",
];
let t = TestContext::new().await;
let info = t.get_info().await.unwrap();
+3 -13
View File
@@ -57,11 +57,7 @@ pub(crate) async fn prepare_decryption(
autocrypt_header: None,
peerstate: None,
message_time,
dkim_results: DkimResults {
dkim_passed: false,
dkim_should_work: false,
allow_keychange: true,
},
dkim_results: DkimResults { dkim_passed: false },
});
}
@@ -86,15 +82,13 @@ pub(crate) async fn prepare_decryption(
None
};
let dkim_results = handle_authres(context, mail, from, message_time).await?;
let dkim_results = handle_authres(context, mail, from).await?;
let allow_aeap = get_encrypted_mime(mail).is_some();
let peerstate = get_autocrypt_peerstate(
context,
from,
autocrypt_header.as_ref(),
message_time,
// Disallowing keychanges is disabled for now:
true, // dkim_results.allow_keychange,
allow_aeap,
)
.await?;
@@ -287,19 +281,15 @@ pub(crate) fn keyring_from_peerstate(peerstate: Option<&Peerstate>) -> Vec<Signe
/// If we already know this fingerprint from another contact's peerstate, return that
/// peerstate in order to make AEAP work, but don't save it into the db yet.
///
/// The param `allow_change` is used to prevent the autocrypt key from being changed
/// if we suspect that the message may be forged and have a spoofed sender identity.
///
/// Returns updated peerstate.
pub(crate) async fn get_autocrypt_peerstate(
context: &Context,
from: &str,
autocrypt_header: Option<&Aheader>,
message_time: i64,
allow_change: bool,
allow_aeap: bool,
) -> Result<Option<Peerstate>> {
let allow_change = allow_change && !context.is_self_addr(from).await?;
let allow_change = !context.is_self_addr(from).await?;
let mut peerstate;
// Apply Autocrypt header
+74 -13
View File
@@ -74,12 +74,13 @@ use async_channel::Receiver;
use serde::{Deserialize, Serialize};
use tokio::time::timeout;
use crate::chat::{send_msg, ChatId};
use crate::chat::{send_msg, ChatId, ChatIdBlocked};
use crate::constants::{DC_CHAT_ID_LAST_SPECIAL, DC_CHAT_ID_TRASH};
use crate::contact::ContactId;
use crate::context::Context;
use crate::download::MIN_DELETE_SERVER_AFTER;
use crate::events::EventType;
use crate::location;
use crate::log::LogExt;
use crate::message::{Message, MessageState, MsgId, Viewtype};
use crate::mimeparser::SystemMessage;
@@ -348,16 +349,16 @@ pub(crate) async fn start_ephemeral_timers_msgids(
/// Selects messages which are expired according to
/// `delete_device_after` setting or `ephemeral_timestamp` column.
///
/// For each message a row ID, chat id and viewtype is returned.
/// For each message a row ID, chat id, viewtype and location ID is returned.
async fn select_expired_messages(
context: &Context,
now: i64,
) -> Result<Vec<(MsgId, ChatId, Viewtype)>> {
) -> Result<Vec<(MsgId, ChatId, Viewtype, u32)>> {
let mut rows = context
.sql
.query_map(
r#"
SELECT id, chat_id, type
SELECT id, chat_id, type, location_id
FROM msgs
WHERE
ephemeral_timestamp != 0
@@ -369,18 +370,21 @@ WHERE
let id: MsgId = row.get("id")?;
let chat_id: ChatId = row.get("chat_id")?;
let viewtype: Viewtype = row.get("type")?;
Ok((id, chat_id, viewtype))
let location_id: u32 = row.get("location_id")?;
Ok((id, chat_id, viewtype, location_id))
},
|rows| rows.collect::<Result<Vec<_>, _>>().map_err(Into::into),
)
.await?;
if let Some(delete_device_after) = context.get_config_delete_device_after().await? {
let self_chat_id = ChatId::lookup_by_contact(context, ContactId::SELF)
let self_chat_id = ChatIdBlocked::lookup_by_contact(context, ContactId::SELF)
.await?
.map(|c| c.id)
.unwrap_or_default();
let device_chat_id = ChatId::lookup_by_contact(context, ContactId::DEVICE)
let device_chat_id = ChatIdBlocked::lookup_by_contact(context, ContactId::DEVICE)
.await?
.map(|c| c.id)
.unwrap_or_default();
let threshold_timestamp = now.saturating_sub(delete_device_after);
@@ -389,7 +393,7 @@ WHERE
.sql
.query_map(
r#"
SELECT id, chat_id, type
SELECT id, chat_id, type, location_id
FROM msgs
WHERE
timestamp < ?1
@@ -408,7 +412,8 @@ WHERE
let id: MsgId = row.get("id")?;
let chat_id: ChatId = row.get("chat_id")?;
let viewtype: Viewtype = row.get("type")?;
Ok((id, chat_id, viewtype))
let location_id: u32 = row.get("location_id")?;
Ok((id, chat_id, viewtype, location_id))
},
|rows| rows.collect::<Result<Vec<_>, _>>().map_err(Into::into),
)
@@ -439,7 +444,7 @@ pub(crate) async fn delete_expired_messages(context: &Context, now: i64) -> Resu
// If you change which information is removed here, also change MsgId::trash() and
// which information receive_imf::add_parts() still adds to the db if the chat_id is TRASH
for (msg_id, chat_id, viewtype) in rows {
for (msg_id, chat_id, viewtype, location_id) in rows {
transaction.execute(
"UPDATE msgs
SET chat_id=?, txt='', subject='', txt_raw='',
@@ -448,6 +453,13 @@ pub(crate) async fn delete_expired_messages(context: &Context, now: i64) -> Resu
(DC_CHAT_ID_TRASH, msg_id),
)?;
if location_id > 0 {
transaction.execute(
"DELETE FROM locations WHERE independent=1 AND id=?",
(location_id,),
)?;
}
msgs_changed.push((chat_id, msg_id));
if viewtype == Viewtype::Webxdc {
webxdc_deleted.push(msg_id)
@@ -480,11 +492,13 @@ pub(crate) async fn delete_expired_messages(context: &Context, now: i64) -> Resu
/// `delete_device_after` setting being set.
async fn next_delete_device_after_timestamp(context: &Context) -> Result<Option<i64>> {
if let Some(delete_device_after) = context.get_config_delete_device_after().await? {
let self_chat_id = ChatId::lookup_by_contact(context, ContactId::SELF)
let self_chat_id = ChatIdBlocked::lookup_by_contact(context, ContactId::SELF)
.await?
.map(|c| c.id)
.unwrap_or_default();
let device_chat_id = ChatId::lookup_by_contact(context, ContactId::DEVICE)
let device_chat_id = ChatIdBlocked::lookup_by_contact(context, ContactId::DEVICE)
.await?
.map(|c| c.id)
.unwrap_or_default();
let oldest_message_timestamp: Option<i64> = context
@@ -592,6 +606,11 @@ pub(crate) async fn ephemeral_loop(context: &Context, interrupt_receiver: Receiv
.await
.log_err(context)
.ok();
location::delete_expired(context, time())
.await
.log_err(context)
.ok();
}
}
@@ -671,8 +690,10 @@ mod tests {
use super::*;
use crate::config::Config;
use crate::download::DownloadState;
use crate::location;
use crate::message::markseen_msgs;
use crate::receive_imf::receive_imf;
use crate::test_utils::TestContext;
use crate::test_utils::{TestContext, TestContextManager};
use crate::timesmearing::MAX_SECONDS_TO_LEND_FROM_FUTURE;
use crate::{
chat::{self, create_group_chat, send_text_msg, Chat, ChatItem, ProtectionStatus},
@@ -1349,4 +1370,44 @@ mod tests {
Ok(())
}
/// Tests that POI location is deleted when ephemeral message expires.
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_ephemeral_poi_location() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
let bob = &tcm.bob().await;
let chat = alice.create_chat(bob).await;
let duration = 60;
chat.id
.set_ephemeral_timer(alice, Timer::Enabled { duration })
.await?;
let sent = alice.pop_sent_msg().await;
bob.recv_msg(&sent).await;
let mut poi_msg = Message::new(Viewtype::Text);
poi_msg.text = "Here".to_string();
poi_msg.set_location(10.0, 20.0);
let alice_sent_message = alice.send_msg(chat.id, &mut poi_msg).await;
let bob_received_message = bob.recv_msg(&alice_sent_message).await;
markseen_msgs(bob, vec![bob_received_message.id]).await?;
for account in [alice, bob] {
let locations = location::get_range(account, None, None, 0, 0).await?;
assert_eq!(locations.len(), 1);
}
SystemTime::shift(Duration::from_secs(100));
for account in [alice, bob] {
delete_expired_messages(account, time()).await?;
let locations = location::get_range(account, None, None, 0, 0).await?;
assert_eq!(locations.len(), 0);
}
Ok(())
}
}
+23 -3
View File
@@ -71,7 +71,17 @@ impl EventEmitter {
/// [`try_recv`]: Self::try_recv
pub async fn recv(&self) -> Option<Event> {
let mut lock = self.0.lock().await;
lock.recv().await.ok()
loop {
match lock.recv().await {
Err(async_broadcast::RecvError::Overflowed(_)) => {
// Some events have been lost,
// but the channel is not closed.
continue;
}
Err(async_broadcast::RecvError::Closed) => return None,
Ok(event) => return Some(event),
}
}
}
/// Tries to receive an event without blocking.
@@ -86,8 +96,18 @@ impl EventEmitter {
// to avoid blocking
// in case there is a concurrent call to `recv`.
let mut lock = self.0.try_lock()?;
let event = lock.try_recv()?;
Ok(event)
loop {
match lock.try_recv() {
Err(async_broadcast::TryRecvError::Overflowed(_)) => {
// Some events have been lost,
// but the channel is not closed.
continue;
}
res @ (Err(async_broadcast::TryRecvError::Empty)
| Err(async_broadcast::TryRecvError::Closed)
| Ok(_)) => return Ok(res?),
}
}
}
}
+9
View File
@@ -279,6 +279,15 @@ pub enum EventType {
status_update_serial: StatusUpdateSerial,
},
/// Data received over an ephemeral peer channel.
WebxdcRealtimeData {
/// Message ID.
msg_id: MsgId,
/// Realtime data.
data: Vec<u8>,
},
/// Inform that a message containing a webxdc instance has been deleted.
WebxdcInstanceDeleted {
/// ID of the deleted message.
+6
View File
@@ -93,6 +93,12 @@ pub enum HeaderDef {
/// See <https://datatracker.ietf.org/doc/html/rfc8601>
AuthenticationResults,
/// Node address from iroh where direct addresses have been removed.
IrohNodeAddr,
/// Advertised gossip topic for one webxdc.
IrohGossipTopic,
#[cfg(test)]
TestHeader,
}
+58 -12
View File
@@ -5,11 +5,12 @@
use std::{
cmp::max,
cmp::min,
collections::{BTreeMap, BTreeSet, HashMap},
iter::Peekable,
mem::take,
sync::atomic::Ordering,
time::Duration,
time::{Duration, UNIX_EPOCH},
};
use anyhow::{bail, format_err, Context as _, Result};
@@ -19,8 +20,9 @@ use deltachat_contact_tools::{normalize_name, ContactAddress};
use futures::{FutureExt as _, StreamExt, TryStreamExt};
use futures_lite::FutureExt;
use num_traits::FromPrimitive;
use rand::Rng;
use ratelimit::Ratelimit;
use tokio::sync::RwLock;
use url::Url;
use crate::chat::{self, ChatId, ChatIdBlocked};
use crate::chatlist_events;
@@ -42,7 +44,7 @@ use crate::scheduler::connectivity::ConnectivityStore;
use crate::socks::Socks5Config;
use crate::sql;
use crate::stock_str;
use crate::tools::{create_id, duration_to_str};
use crate::tools::{self, create_id, duration_to_str};
pub(crate) mod capabilities;
mod client;
@@ -82,15 +84,17 @@ pub(crate) struct Imap {
pub(crate) connectivity: ConnectivityStore,
/// Rate limit for IMAP connection attempts.
conn_last_try: tools::Time,
conn_backoff_ms: u64,
/// Rate limit for successful IMAP connections.
///
/// This rate limit prevents busy loop
/// in case the server refuses connections
/// This rate limit prevents busy loop in case the server refuses logins
/// or in case connection gets dropped over and over due to IMAP bug,
/// e.g. the server returning invalid response to SELECT command
/// immediately after logging in or returning an error in response to LOGIN command
/// due to internal server error.
ratelimit: RwLock<Ratelimit>,
ratelimit: Ratelimit,
}
#[derive(Debug)]
@@ -108,6 +112,8 @@ pub(crate) struct ServerMetadata {
/// IMAP METADATA `/shared/admin` as defined in
/// <https://www.rfc-editor.org/rfc/rfc5464#section-6.2.2>.
pub admin: Option<String>,
pub iroh_relay: Option<Url>,
}
impl async_imap::Authenticator for OAuth2 {
@@ -248,8 +254,10 @@ impl Imap {
strict_tls,
login_failed_once: false,
connectivity: Default::default(),
conn_last_try: UNIX_EPOCH,
conn_backoff_ms: 0,
// 1 connection per minute + a burst of 2.
ratelimit: RwLock::new(Ratelimit::new(Duration::new(120, 0), 2.0)),
ratelimit: Ratelimit::new(Duration::new(120, 0), 2.0),
};
Ok(imap)
@@ -293,7 +301,15 @@ impl Imap {
bail!("IMAP operation attempted while it is torn down");
}
let ratelimit_duration = self.ratelimit.read().await.until_can_send();
let now = tools::Time::now();
let until_can_send = max(
min(self.conn_last_try, now)
.checked_add(Duration::from_millis(self.conn_backoff_ms))
.unwrap_or(now),
now,
)
.duration_since(now)?;
let ratelimit_duration = max(until_can_send, self.ratelimit.until_can_send());
if !ratelimit_duration.is_zero() {
warn!(
context,
@@ -316,7 +332,16 @@ impl Imap {
info!(context, "Connecting to IMAP server");
self.connectivity.set_connecting(context).await;
self.ratelimit.write().await.send();
self.conn_last_try = tools::Time::now();
const BACKOFF_MIN_MS: u64 = 2000;
const BACKOFF_MAX_MS: u64 = 80_000;
self.conn_backoff_ms = min(self.conn_backoff_ms, BACKOFF_MAX_MS / 2);
self.conn_backoff_ms = self.conn_backoff_ms.saturating_add(
rand::thread_rng().gen_range((self.conn_backoff_ms / 2)..=self.conn_backoff_ms),
);
self.conn_backoff_ms = max(BACKOFF_MIN_MS, self.conn_backoff_ms);
let connection_res: Result<Client> =
if self.lp.security == Socket::Starttls || self.lp.security == Socket::Plain {
let imap_server: &str = self.lp.server.as_ref();
@@ -364,6 +389,8 @@ impl Imap {
}
};
let client = connection_res?;
self.conn_backoff_ms = BACKOFF_MIN_MS;
self.ratelimit.send();
let imap_user: &str = self.lp.user.as_ref();
let imap_pw: &str = self.lp.password.as_ref();
@@ -1425,11 +1452,16 @@ impl Session {
let mut comment = None;
let mut admin = None;
let mut iroh_relay = None;
let mailbox = "";
let options = "";
let metadata = self
.get_metadata(mailbox, options, "(/shared/comment /shared/admin)")
.get_metadata(
mailbox,
options,
"(/shared/comment /shared/admin /shared/vendor/deltachat/irohrelay)",
)
.await?;
for m in metadata {
match m.entry.as_ref() {
@@ -1439,10 +1471,24 @@ impl Session {
"/shared/admin" => {
admin = m.value;
}
"/shared/vendor/deltachat/irohrelay" => {
if let Some(url) = m.value.as_deref().and_then(|s| Url::parse(s).ok()) {
iroh_relay = Some(url);
} else {
warn!(
context,
"Got invalid URL from iroh relay metadata: {:?}.", m.value
);
}
}
_ => {}
}
}
*lock = Some(ServerMetadata { comment, admin });
*lock = Some(ServerMetadata {
comment,
admin,
iroh_relay,
});
Ok(())
}
+8
View File
@@ -32,6 +32,14 @@ pub(crate) struct Capabilities {
/// This is supported by <https://github.com/deltachat/chatmail>
pub can_push: bool,
/// True if the server has an XCHATMAIL capability
/// indicating that it is a <https://github.com/deltachat/chatmail> server.
///
/// This can be used to hide some advanced settings in the UI
/// that are only interesting for normal email accounts,
/// e.g. the ability to move messages to Delta Chat folder.
pub is_chatmail: bool,
/// Server ID if the server supports ID capability.
pub server_id: Option<HashMap<String, String>>,
}
+1
View File
@@ -61,6 +61,7 @@ async fn determine_capabilities(
can_condstore: caps.has_str("CONDSTORE"),
can_metadata: caps.has_str("METADATA"),
can_push: caps.has_str("XDELTAPUSH"),
is_chatmail: caps.has_str("XCHATMAIL"),
server_id,
};
Ok(capabilities)
+5
View File
@@ -94,6 +94,11 @@ impl Session {
self.capabilities.can_push
}
// Returns true if IMAP server has `XCHATMAIL` capability.
pub fn is_chatmail(&self) -> bool {
self.capabilities.is_chatmail
}
/// Returns the names of all folders on the IMAP server.
pub async fn list_folders(&mut self) -> Result<Vec<async_imap::types::Name>> {
let list = self.list(Some(""), Some("*")).await?.try_collect().await?;
+52 -32
View File
@@ -193,7 +193,9 @@ pub async fn render_setup_file(context: &Context, passphrase: &str) -> Result<St
true => Some(("Autocrypt-Prefer-Encrypt", "mutual")),
};
let private_key_asc = private_key.to_asc(ac_headers);
let encr = pgp::symm_encrypt(passphrase, private_key_asc.as_bytes()).await?;
let encr = pgp::symm_encrypt(passphrase, private_key_asc.as_bytes())
.await?
.replace('\n', "\r\n");
let replacement = format!(
concat!(
@@ -284,7 +286,7 @@ pub async fn continue_key_transfer(
let file = open_file_std(context, filename)?;
let sc = normalize_setup_code(setup_code);
let armored_key = decrypt_setup_file(&sc, file).await?;
set_self_key(context, &armored_key, true, true).await?;
set_self_key(context, &armored_key, true).await?;
maybe_add_bcc_self_device_msg(context).await?;
Ok(())
@@ -293,35 +295,32 @@ pub async fn continue_key_transfer(
}
}
async fn set_self_key(
context: &Context,
armored: &str,
set_default: bool,
prefer_encrypt_required: bool,
) -> Result<()> {
async fn set_self_key(context: &Context, armored: &str, set_default: bool) -> Result<()> {
// try hard to only modify key-state
let (private_key, header) = SignedSecretKey::from_asc(armored)?;
let public_key = private_key.split_public_key()?;
let preferencrypt = header.get("Autocrypt-Prefer-Encrypt");
match preferencrypt.map(|s| s.as_str()) {
Some(headerval) => {
let e2ee_enabled = match headerval {
"nopreference" => 0,
"mutual" => 1,
_ => {
bail!("invalid Autocrypt-Prefer-Encrypt header: {:?}", header);
}
};
context
.sql
.set_raw_config_int("e2ee_enabled", e2ee_enabled)
.await?;
}
None => {
if prefer_encrypt_required {
bail!("missing Autocrypt-Prefer-Encrypt header");
if let Some(preferencrypt) = header.get("Autocrypt-Prefer-Encrypt") {
let e2ee_enabled = match preferencrypt.as_str() {
"nopreference" => 0,
"mutual" => 1,
_ => {
bail!("invalid Autocrypt-Prefer-Encrypt header: {:?}", header);
}
}
};
context
.sql
.set_raw_config_int("e2ee_enabled", e2ee_enabled)
.await?;
} else {
// `Autocrypt-Prefer-Encrypt` is not included
// in keys exported to file.
//
// `Autocrypt-Prefer-Encrypt` also SHOULD be sent
// in Autocrypt Setup Message according to Autocrypt specification,
// but K-9 6.802 does not include this header.
//
// We keep current setting in this case.
info!(context, "No Autocrypt-Prefer-Encrypt header.");
};
let self_addr = context.get_primary_self_addr().await?;
@@ -604,7 +603,7 @@ async fn export_backup_inner(
async fn import_secret_key(context: &Context, path: &Path, set_default: bool) -> Result<()> {
let buf = read_file(context, &path).await?;
let armored = std::string::String::from_utf8_lossy(&buf);
set_self_key(context, &armored, set_default, false).await?;
set_self_key(context, &armored, set_default).await?;
Ok(())
}
@@ -825,6 +824,7 @@ mod tests {
use super::*;
use crate::pgp::{split_armored_data, HEADER_AUTOCRYPT, HEADER_SETUPCODE};
use crate::receive_imf::receive_imf;
use crate::stock_str::StockMessage;
use crate::test_utils::{alice_keypair, TestContext, TestContextManager};
@@ -834,15 +834,17 @@ mod tests {
let msg = render_setup_file(&t, "hello").await.unwrap();
println!("{}", &msg);
// Check some substrings, indicating things got substituted.
// In particular note the mixing of `\r\n` and `\n` depending
// on who generated the strings.
assert!(msg.contains("<title>Autocrypt Setup Message</title"));
assert!(msg.contains("<h1>Autocrypt Setup Message</h1>"));
assert!(msg.contains("<p>This is the Autocrypt Setup Message used to"));
assert!(msg.contains("-----BEGIN PGP MESSAGE-----\r\n"));
assert!(msg.contains("Passphrase-Format: numeric9x4\r\n"));
assert!(msg.contains("Passphrase-Begin: he\n"));
assert!(msg.contains("-----END PGP MESSAGE-----\n"));
assert!(msg.contains("Passphrase-Begin: he\r\n"));
assert!(msg.contains("-----END PGP MESSAGE-----\r\n"));
for line in msg.rsplit_terminator('\n') {
assert!(line.ends_with('\r'));
}
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
@@ -1193,4 +1195,22 @@ mod tests {
Ok(())
}
/// Tests reception of Autocrypt Setup Message from K-9 6.802.
///
/// Unlike Autocrypt Setup Message sent by Delta Chat,
/// this message does not contain `Autocrypt-Prefer-Encrypt` header.
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_key_transfer_k_9() -> Result<()> {
let t = &TestContext::new().await;
t.configure_addr("autocrypt@nine.testrun.org").await;
let raw = include_bytes!("../test-data/message/k-9-autocrypt-setup-message.eml");
let received = receive_imf(t, raw, false).await?.unwrap();
let setup_code = "0655-9868-8252-5455-4232-5158-1237-5333-2638";
continue_key_transfer(t, *received.msg_ids.last().unwrap(), setup_code).await?;
Ok(())
}
}
+7
View File
@@ -38,6 +38,7 @@ use iroh::progress::ProgressEmitter;
use iroh::protocol::AuthToken;
use iroh::provider::{DataSource, Event, Provider, Ticket};
use iroh::Hash;
use iroh_old as iroh;
use tokio::fs::{self, File};
use tokio::io::{self, AsyncWriteExt, BufWriter};
use tokio::sync::broadcast::error::RecvError;
@@ -656,6 +657,12 @@ mod tests {
let text = fs::read_to_string(&path).await.unwrap();
assert_eq!(text, "i am attachment");
let path = path.with_file_name("saved.txt");
msg.save_file(&ctx1, &path).await.unwrap();
let text = fs::read_to_string(&path).await.unwrap();
assert_eq!(text, "i am attachment");
assert!(msg.save_file(&ctx1, &path).await.is_err());
// Check that both received the ImexProgress events.
ctx0.evtracker
.get_matching(|ev| matches!(ev, EventType::ImexProgress(1000)))
+2 -1
View File
@@ -94,7 +94,7 @@ pub mod webxdc;
#[macro_use]
mod dehtml;
mod authres;
mod color;
pub mod color;
pub mod html;
pub mod net;
pub mod plaintext;
@@ -106,6 +106,7 @@ pub mod receive_imf;
pub mod tools;
pub mod accounts;
pub mod peer_channels;
pub mod reaction;
/// If set IMAP/incoming and SMTP/outgoing MIME messages will be printed.
+177 -1
View File
@@ -1,4 +1,14 @@
//! Location handling.
//!
//! Delta Chat handles two kind of locations.
//!
//! There are two kinds of locations:
//! - Independent locations, also known as Points of Interest (POI).
//! - Path locations.
//!
//! Locations are sent as KML attachments.
//! Independent locations are sent in `message.kml` attachments
//! and path locations are sent in `location.kml` attachments.
use std::time::Duration;
@@ -8,6 +18,7 @@ use quick_xml::events::{BytesEnd, BytesStart, BytesText};
use tokio::time::timeout;
use crate::chat::{self, ChatId};
use crate::constants::DC_CHAT_ID_TRASH;
use crate::contact::ContactId;
use crate::context::Context;
use crate::events::EventType;
@@ -350,6 +361,7 @@ pub async fn set(context: &Context, latitude: f64, longitude: f64, accuracy: f64
)
.await?;
let mut stored_location = false;
for chat_id in chats {
context.sql.execute(
"INSERT INTO locations \
@@ -362,6 +374,7 @@ pub async fn set(context: &Context, latitude: f64, longitude: f64, accuracy: f64
chat_id,
ContactId::SELF,
)).await.context("Failed to store location")?;
stored_location = true;
info!(context, "Stored location for chat {chat_id}.");
continue_streaming = true;
@@ -369,6 +382,10 @@ pub async fn set(context: &Context, latitude: f64, longitude: f64, accuracy: f64
if continue_streaming {
context.emit_location_changed(Some(ContactId::SELF)).await?;
};
if stored_location {
// Interrupt location loop so it may send a location-only message.
context.scheduler.interrupt_location().await;
}
Ok(continue_streaming)
}
@@ -461,6 +478,58 @@ pub async fn delete_all(context: &Context) -> Result<()> {
Ok(())
}
/// Deletes expired locations.
///
/// Only path locations are deleted.
/// POIs should be deleted when corresponding message is deleted.
pub(crate) async fn delete_expired(context: &Context, now: i64) -> Result<()> {
let Some(delete_device_after) = context.get_config_delete_device_after().await? else {
return Ok(());
};
let threshold_timestamp = now.saturating_sub(delete_device_after);
let deleted = context
.sql
.execute(
"DELETE FROM locations WHERE independent=0 AND timestamp < ?",
(threshold_timestamp,),
)
.await?
> 0;
if deleted {
info!(context, "Deleted {deleted} expired locations.");
context.emit_location_changed(None).await?;
}
Ok(())
}
/// Deletes location if it is an independent location.
///
/// This function is used when a message is deleted
/// that has a corresponding `location_id`.
pub(crate) async fn delete_poi_location(context: &Context, location_id: u32) -> Result<()> {
context
.sql
.execute(
"DELETE FROM locations WHERE independent = 1 AND id=?",
(location_id as i32,),
)
.await?;
Ok(())
}
/// Deletes POI locations that don't have corresponding message anymore.
pub(crate) async fn delete_orphaned_poi_locations(context: &Context) -> Result<()> {
context.sql.execute("
DELETE FROM locations
WHERE independent=1 AND id NOT IN
(SELECT location_id from MSGS LEFT JOIN locations
ON locations.id=location_id
WHERE location_id>0 -- This check makes the query faster by not looking for locations with ID 0 that don't exist.
AND msgs.chat_id != ?)", (DC_CHAT_ID_TRASH,)).await?;
Ok(())
}
/// Returns `location.kml` contents.
pub async fn get_kml(context: &Context, chat_id: ChatId) -> Result<Option<(String, u32)>> {
let mut last_added_location_id = 0;
@@ -815,8 +884,11 @@ mod tests {
#![allow(clippy::indexing_slicing)]
use super::*;
use crate::config::Config;
use crate::message::MessageState;
use crate::receive_imf::receive_imf;
use crate::test_utils::TestContext;
use crate::test_utils::{TestContext, TestContextManager};
use crate::tools::SystemTime;
#[test]
fn test_kml_parse() {
@@ -944,6 +1016,54 @@ Content-Disposition: attachment; filename="location.kml"
Ok(())
}
/// Tests that `location.kml` is not hidden and not seen if it contains a message.
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn receive_visible_location_kml() -> Result<()> {
let alice = TestContext::new_alice().await;
receive_imf(
&alice,
br#"Subject: locations
MIME-Version: 1.0
To: <alice@example.org>
From: <bob@example.net>
Date: Tue, 21 Dec 2021 00:00:00 +0000
Chat-Version: 1.0
Message-ID: <foobar@localhost>
Content-Type: multipart/mixed; boundary="U8BOG8qNXfB0GgLiQ3PKUjlvdIuLRF"
--U8BOG8qNXfB0GgLiQ3PKUjlvdIuLRF
Content-Type: text/plain; charset=utf-8; format=flowed; delsp=no
Text message.
--U8BOG8qNXfB0GgLiQ3PKUjlvdIuLRF
Content-Type: application/vnd.google-earth.kml+xml
Content-Disposition: attachment; filename="location.kml"
<?xml version="1.0" encoding="UTF-8"?>
<kml xmlns="http://www.opengis.net/kml/2.2">
<Document addr="bob@example.net">
<Placemark><Timestamp><when>2021-11-21T00:00:00Z</when></Timestamp><Point><coordinates accuracy="1.0000000000000000">10.00000000000000,20.00000000000000</coordinates></Point></Placemark>
</Document>
</kml>
--U8BOG8qNXfB0GgLiQ3PKUjlvdIuLRF--"#,
false,
)
.await?;
let received_msg = alice.get_last_msg().await;
assert_eq!(received_msg.text, "Text message.");
assert_eq!(received_msg.state, MessageState::InFresh);
let locations = get_range(&alice, None, None, 0, 0).await?;
assert_eq!(locations.len(), 1);
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_send_locations_to_chat() -> Result<()> {
let alice = TestContext::new_alice().await;
@@ -966,6 +1086,8 @@ Content-Disposition: attachment; filename="location.kml"
let mut msg = Message::new(Viewtype::Image);
msg.set_file(file.to_str().unwrap(), None);
let sent = alice.send_msg(alice_chat.id, &mut msg).await;
let alice_msg = Message::load_from_db(&alice, sent.sender_msg_id).await?;
assert_eq!(alice_msg.has_location(), false);
let msg = bob.recv_msg_opt(&sent).await.unwrap();
assert!(msg.chat_id == bob_chat_id);
@@ -974,6 +1096,60 @@ Content-Disposition: attachment; filename="location.kml"
let bob_msg = Message::load_from_db(&bob, *msg.msg_ids.first().unwrap()).await?;
assert_eq!(bob_msg.chat_id, bob_chat_id);
assert_eq!(bob_msg.viewtype, Viewtype::Image);
assert_eq!(bob_msg.has_location(), false);
let bob_locations = get_range(&bob, None, None, 0, 0).await?;
assert_eq!(bob_locations.len(), 1);
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_delete_expired_locations() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
let bob = &tcm.bob().await;
// Alice enables deletion of messages from device after 1 week.
alice
.set_config(Config::DeleteDeviceAfter, Some("604800"))
.await?;
// Bob enables deletion of messages from device after 1 day.
bob.set_config(Config::DeleteDeviceAfter, Some("86400"))
.await?;
let alice_chat = alice.create_chat(bob).await;
// Alice enables location streaming.
// Bob receives a message saying that Alice enabled location streaming.
send_locations_to_chat(alice, alice_chat.id, 60).await?;
bob.recv_msg(&alice.pop_sent_msg().await).await;
// Alice gets new location from GPS.
assert_eq!(set(alice, 10.0, 20.0, 1.0).await?, true);
assert_eq!(get_range(alice, None, None, 0, 0).await?.len(), 1);
// 10 seconds later location sending stream manages to send location.
SystemTime::shift(Duration::from_secs(10));
delete_expired(alice, time()).await?;
maybe_send_locations(alice).await?;
bob.recv_msg_opt(&alice.pop_sent_msg().await).await;
assert_eq!(get_range(alice, None, None, 0, 0).await?.len(), 1);
assert_eq!(get_range(bob, None, None, 0, 0).await?.len(), 1);
// Day later Bob removes location.
SystemTime::shift(Duration::from_secs(86400));
delete_expired(alice, time()).await?;
delete_expired(bob, time()).await?;
assert_eq!(get_range(alice, None, None, 0, 0).await?.len(), 1);
assert_eq!(get_range(bob, None, None, 0, 0).await?.len(), 0);
// Week late Alice removes location.
SystemTime::shift(Duration::from_secs(604800));
delete_expired(alice, time()).await?;
delete_expired(bob, time()).await?;
assert_eq!(get_range(alice, None, None, 0, 0).await?.len(), 0);
assert_eq!(get_range(bob, None, None, 0, 0).await?.len(), 0);
Ok(())
}
+109 -34
View File
@@ -4,11 +4,13 @@ use std::collections::BTreeSet;
use std::path::{Path, PathBuf};
use anyhow::{ensure, format_err, Context as _, Result};
use deltachat_contact_tools::{parse_vcard, VcardContact};
use deltachat_derive::{FromSql, ToSql};
use serde::{Deserialize, Serialize};
use tokio::{fs, io};
use crate::blob::BlobObject;
use crate::chat::{Chat, ChatId};
use crate::chat::{Chat, ChatId, ChatIdBlocked};
use crate::chatlist_events;
use crate::config::Config;
use crate::constants::{
@@ -21,6 +23,7 @@ use crate::download::DownloadState;
use crate::ephemeral::{start_ephemeral_timers_msgids, Timer as EphemeralTimer};
use crate::events::EventType;
use crate::imap::markseen_on_imap_table;
use crate::location::delete_poi_location;
use crate::mimeparser::{parse_message_id, SystemMessage};
use crate::param::{Param, Params};
use crate::pgp::split_armored_data;
@@ -605,6 +608,33 @@ impl Message {
self.param.get_path(Param::File, context).unwrap_or(None)
}
/// Returns vector of vcards if the file has a vCard attachment.
pub async fn vcard_contacts(&self, context: &Context) -> Result<Vec<VcardContact>> {
if self.viewtype != Viewtype::Vcard {
return Ok(Vec::new());
}
let path = self
.get_file(context)
.context("vCard message does not have an attachment")?;
let bytes = tokio::fs::read(path).await?;
let vcard_contents = std::str::from_utf8(&bytes).context("vCard is not a valid UTF-8")?;
Ok(parse_vcard(vcard_contents))
}
/// Save file copy at the user-provided path.
pub async fn save_file(&self, context: &Context, path: &Path) -> Result<()> {
let path_src = self.get_file(context).context("No file")?;
let mut src = fs::OpenOptions::new().read(true).open(path_src).await?;
let mut dst = fs::OpenOptions::new()
.write(true)
.create_new(true)
.open(path)
.await?;
io::copy(&mut src, &mut dst).await?;
Ok(())
}
/// If message is an image or gif, set Param::Width and Param::Height
pub(crate) async fn try_calc_and_set_dimensions(&mut self, context: &Context) -> Result<()> {
if self.viewtype.has_file() {
@@ -640,13 +670,11 @@ impl Message {
Ok(())
}
/// Check if a message has a location bound to it.
/// These messages are also returned by get_locations()
/// and the UI may decide to display a special icon beside such messages,
/// Check if a message has a POI location bound to it.
/// These locations are also returned by [`location::get_range()`].
/// The UI may decide to display a special icon beside such messages.
///
/// @memberof Message
/// @param msg The message object.
/// @return 1=Message has location bound to it, 0=No location bound to message.
/// [`location::get_range()`]: crate::location::get_range
pub fn has_location(&self) -> bool {
self.location_id != 0
}
@@ -656,13 +684,17 @@ impl Message {
/// at a position different from the self-location.
/// You should not call this function
/// if you want to bind the current self-location to a message;
/// this is done by set_location() and send_locations_to_chat().
/// this is done by [`location::set()`] and [`send_locations_to_chat()`].
///
/// Typically results in the event #DC_EVENT_LOCATION_CHANGED with
/// contact_id set to ContactId::SELF.
/// Typically results in the event [`LocationChanged`] with
/// `contact_id` set to [`ContactId::SELF`].
///
/// @param latitude North-south position of the location.
/// @param longitude East-west position of the location.
/// `latitude` is the North-south position of the location.
/// `longitutde` is the East-west position of the location.
///
/// [`location::set()`]: crate::location::set
/// [`send_locations_to_chat()`]: crate::location::send_locations_to_chat
/// [`LocationChanged`]: crate::events::EventType::LocationChanged
pub fn set_location(&mut self, latitude: f64, longitude: f64) {
if latitude == 0.0 && longitude == 0.0 {
return;
@@ -1043,6 +1075,7 @@ impl Message {
filemime: Option<&str>,
) -> Result<()> {
let blob = BlobObject::create(context, suggested_name, data).await?;
self.param.set(Param::Filename, suggested_name);
self.param.set(Param::File, blob.as_name());
self.param.set_optional(Param::MimeType, filemime);
Ok(())
@@ -1112,7 +1145,7 @@ impl Message {
.get_bool(Param::GuaranteeE2ee)
.unwrap_or_default()
{
self.param.set(Param::GuaranteeE2ee, "1");
self.param.set(Param::ProtectQuote, "1");
}
let text = quote.get_text();
@@ -1398,8 +1431,8 @@ pub(crate) fn guess_msgtype_from_suffix(path: &Path) -> Option<(Viewtype, &str)>
"tif" => (Viewtype::File, "image/tiff"),
"ttf" => (Viewtype::File, "font/ttf"),
"txt" => (Viewtype::File, "text/plain"),
"vcard" => (Viewtype::File, "text/vcard"),
"vcf" => (Viewtype::File, "text/vcard"),
"vcard" => (Viewtype::Vcard, "text/vcard"),
"vcf" => (Viewtype::Vcard, "text/vcard"),
"wav" => (Viewtype::File, "audio/wav"),
"weba" => (Viewtype::File, "audio/webm"),
"webm" => (Viewtype::Video, "video/webm"),
@@ -1556,17 +1589,6 @@ pub async fn delete_msgs(context: &Context, msg_ids: &[MsgId]) -> Result<()> {
Ok(())
}
async fn delete_poi_location(context: &Context, location_id: u32) -> Result<()> {
context
.sql
.execute(
"DELETE FROM locations WHERE independent = 1 AND id=?;",
(location_id as i32,),
)
.await?;
Ok(())
}
/// Marks requested messages as seen.
pub async fn markseen_msgs(context: &Context, msg_ids: Vec<MsgId>) -> Result<()> {
if msg_ids.is_empty() {
@@ -1806,8 +1828,9 @@ pub async fn estimate_deletion_cnt(
from_server: bool,
seconds: i64,
) -> Result<usize> {
let self_chat_id = ChatId::lookup_by_contact(context, ContactId::SELF)
let self_chat_id = ChatIdBlocked::lookup_by_contact(context, ContactId::SELF)
.await?
.map(|c| c.id)
.unwrap_or_default();
let threshold_timestamp = time() - seconds;
@@ -1930,7 +1953,8 @@ pub enum Viewtype {
Text = 10,
/// Image message.
/// If the image is an animated GIF, the type DC_MSG_GIF should be used.
/// If the image is a GIF and has the appropriate extension, the viewtype is auto-changed to
/// `Gif` when sending the message.
/// File, width and height are set via dc_msg_set_file(), dc_msg_set_dimension
/// and retrieved via dc_msg_set_file(), dc_msg_set_dimension().
Image = 20,
@@ -1974,6 +1998,11 @@ pub enum Viewtype {
/// Message is an webxdc instance.
Webxdc = 80,
/// Message containing shared contacts represented as a vCard (virtual contact file)
/// with email addresses and possibly other fields.
/// Use `parse_vcard()` to retrieve them.
Vcard = 90,
}
impl Viewtype {
@@ -1991,6 +2020,7 @@ impl Viewtype {
Viewtype::File => true,
Viewtype::VideochatInvitation => false,
Viewtype::Webxdc => true,
Viewtype::Vcard => true,
}
}
}
@@ -2000,8 +2030,11 @@ mod tests {
use num_traits::FromPrimitive;
use super::*;
use crate::chat::{self, marknoticed_chat, send_text_msg, ChatItem};
use crate::chat::{
self, add_contact_to_chat, marknoticed_chat, send_text_msg, ChatItem, ProtectionStatus,
};
use crate::chatlist::Chatlist;
use crate::config::Config;
use crate::reaction::send_reaction;
use crate::receive_imf::receive_imf;
use crate::test_utils as test;
@@ -2025,8 +2058,6 @@ mod tests {
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_prepare_message_and_send() {
use crate::config::Config;
let d = test::TestContext::new().await;
let ctx = &d.ctx;
@@ -2165,8 +2196,6 @@ mod tests {
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_quote() {
use crate::config::Config;
let d = test::TestContext::new().await;
let ctx = &d.ctx;
@@ -2199,6 +2228,42 @@ mod tests {
assert_eq!(quoted_msg.get_text(), msg2.quoted_text().unwrap());
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_unencrypted_quote_encrypted_message() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
let bob = &tcm.bob().await;
let alice_group = alice
.create_group_with_members(ProtectionStatus::Unprotected, "Group chat", &[bob])
.await;
let sent = alice.send_text(alice_group, "Hi! I created a group").await;
let bob_received_message = bob.recv_msg(&sent).await;
let bob_group = bob_received_message.chat_id;
bob_group.accept(bob).await?;
let sent = bob.send_text(bob_group, "Encrypted message").await;
let alice_received_message = alice.recv_msg(&sent).await;
assert!(alice_received_message.get_showpadlock());
// Alice adds contact without key so chat becomes unencrypted.
let alice_flubby_contact_id =
Contact::create(alice, "Flubby", "flubby@example.org").await?;
add_contact_to_chat(alice, alice_group, alice_flubby_contact_id).await?;
// Alice quotes encrypted message in unencrypted chat.
let mut msg = Message::new(Viewtype::Text);
msg.set_quote(alice, Some(&alice_received_message)).await?;
chat::send_msg(alice, alice_group, &mut msg).await?;
let bob_received_message = bob.recv_msg(&alice.pop_sent_msg().await).await;
assert_eq!(bob_received_message.quoted_text().unwrap(), "...");
assert_eq!(bob_received_message.get_showpadlock(), false);
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_get_chat_id() {
// Alice receives a message that pops up as a contact request
@@ -2227,6 +2292,7 @@ mod tests {
async fn test_set_override_sender_name() {
// send message with overridden sender name
let alice = TestContext::new_alice().await;
let alice2 = TestContext::new_alice().await;
let bob = TestContext::new_bob().await;
let chat = alice.create_chat(&bob).await;
let contact_id = *chat::get_chat_contacts(&alice, chat.id)
@@ -2246,6 +2312,7 @@ mod tests {
assert_eq!(msg.get_sender_name(&contact), "over ride".to_string());
assert_ne!(contact.get_display_name(), "over ride".to_string());
chat::send_msg(&alice, chat.id, &mut msg).await.unwrap();
let sent_msg = alice.pop_sent_msg().await;
// bob receives that message
let chat = bob.create_chat(&alice).await;
@@ -2255,7 +2322,7 @@ mod tests {
.first()
.unwrap();
let contact = Contact::get_by_id(&bob, contact_id).await.unwrap();
let msg = bob.recv_msg(&alice.pop_sent_msg().await).await;
let msg = bob.recv_msg(&sent_msg).await;
assert_eq!(msg.chat_id, chat.id);
assert_eq!(msg.text, "bla blubb");
assert_eq!(
@@ -2269,6 +2336,13 @@ mod tests {
// (mailing lists may also use `Sender:`-header)
let chat = Chat::load_from_db(&bob, msg.chat_id).await.unwrap();
assert_ne!(chat.typ, Chattype::Mailinglist);
// Alice receives message on another device.
let msg = alice2.recv_msg(&sent_msg).await;
assert_eq!(
msg.get_override_sender_name(),
Some("over ride".to_string())
);
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
@@ -2469,6 +2543,7 @@ mod tests {
Viewtype::from_i32(70).unwrap()
);
assert_eq!(Viewtype::Webxdc, Viewtype::from_i32(80).unwrap());
assert_eq!(Viewtype::Vcard, Viewtype::from_i32(90).unwrap());
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
+38 -9
View File
@@ -17,11 +17,12 @@ use crate::contact::Contact;
use crate::context::Context;
use crate::e2ee::EncryptHelper;
use crate::ephemeral::Timer as EphemeralTimer;
use crate::headerdef::HeaderDef;
use crate::html::new_html_mimepart;
use crate::location;
use crate::message::{self, Message, MsgId, Viewtype};
use crate::mimeparser::SystemMessage;
use crate::param::Param;
use crate::peer_channels::create_iroh_header;
use crate::peerstate::Peerstate;
use crate::simplify::escape_message_footer_marks;
use crate::stock_str;
@@ -29,6 +30,7 @@ use crate::tools::IsNoneOrEmpty;
use crate::tools::{
create_outgoing_rfc724_mid, create_smeared_timestamp, remove_subject_prefix, time,
};
use crate::{location, peer_channels};
// attachments of 25 mb brutto should work on the majority of providers
// (brutto examples: web.de=50, 1&1=40, t-online.de=32, gmail=25, posteo=50, yahoo=25, all-inkl=100).
@@ -669,19 +671,19 @@ impl<'a> MimeFactory<'a> {
let mut is_gossiped = false;
let (main_part, parts) = match self.loaded {
Loaded::Message { .. } => {
self.render_message(context, &mut headers, &grpimage)
.await?
}
Loaded::Mdn { .. } => (self.render_mdn(context).await?, Vec::new()),
};
let peerstates = self.peerstates_for_recipients(context).await?;
let should_encrypt =
encrypt_helper.should_encrypt(context, e2ee_guaranteed, &peerstates)?;
let is_encrypted = should_encrypt && !force_plaintext;
let (main_part, parts) = match self.loaded {
Loaded::Message { .. } => {
self.render_message(context, &mut headers, &grpimage, is_encrypted)
.await?
}
Loaded::Mdn { .. } => (self.render_mdn(context).await?, Vec::new()),
};
let message = if parts.is_empty() {
// Single part, render as regular message.
main_part
@@ -960,6 +962,7 @@ impl<'a> MimeFactory<'a> {
context: &Context,
headers: &mut MessageHeaders,
grpimage: &Option<String>,
is_encrypted: bool,
) -> Result<(PartBuilder, Vec<PartBuilder>)> {
let chat = match &self.loaded {
Loaded::Message { chat } => chat,
@@ -1147,6 +1150,18 @@ impl<'a> MimeFactory<'a> {
"protection-disabled".to_string(),
));
}
SystemMessage::IrohNodeAddr => {
headers.protected.push(Header::new(
HeaderDef::IrohNodeAddr.get_headername().to_string(),
serde_json::to_string(
&context
.get_or_try_init_peer_channel()
.await?
.get_node_addr()
.await?,
)?,
));
}
_ => {}
}
@@ -1221,6 +1236,16 @@ impl<'a> MimeFactory<'a> {
.msg
.quoted_text()
.map(|quote| format_flowed_quote(&quote) + "\r\n\r\n");
if !is_encrypted
&& self
.msg
.param
.get_bool(Param::ProtectQuote)
.unwrap_or_default()
{
// Message is not encrypted but quotes encrypted message.
quoted_text = Some("> ...\r\n\r\n".to_string());
}
if quoted_text.is_none() && final_text.starts_with('>') {
// Insert empty line to avoid receiver treating user-sent quote as topquote inserted by
// Delta Chat.
@@ -1303,6 +1328,10 @@ impl<'a> MimeFactory<'a> {
let json = self.msg.param.get(Param::Arg).unwrap_or_default();
parts.push(context.build_status_update_part(json));
} else if self.msg.viewtype == Viewtype::Webxdc {
let topic = peer_channels::create_random_topic();
headers
.protected
.push(create_iroh_header(context, topic, self.msg.id).await?);
if let Some(json) = context
.render_webxdc_status_update_object(self.msg.id, None)
.await?
+187 -67
View File
@@ -36,6 +36,7 @@ use crate::simplify::{simplify, SimplifiedText};
use crate::sync::SyncItems;
use crate::tools::{
create_smeared_timestamp, get_filemeta, parse_receive_headers, smeared_time, truncate_by_lines,
validate_id,
};
use crate::{chatlist_events, location, stock_str, tools};
@@ -179,6 +180,14 @@ pub enum SystemMessage {
/// which is sent by chatmail servers.
InvalidUnencryptedMail = 13,
/// 1:1 chats info message telling that SecureJoin has started and the user should wait for it
/// to complete.
SecurejoinWait = 14,
/// 1:1 chats info message telling that SecureJoin is still running, but the user may already
/// send messages.
SecurejoinWaitTimeout = 15,
/// Self-sent-message that contains only json used for multi-device-sync;
/// if possible, we attach that to other messages as for locations.
MultiDeviceSync = 20,
@@ -190,6 +199,9 @@ pub enum SystemMessage {
/// Webxdc info added with `info` set in `send_webxdc_status_update()`.
WebxdcInfoMessage = 32,
/// This message contains a users iroh node address.
IrohNodeAddr = 40,
}
const MIME_AC_SETUP_FILE: &str = "application/autocrypt-setup";
@@ -415,8 +427,6 @@ impl MimeMessage {
if let (Some(peerstate), Ok(mail)) = (&mut decryption_info.peerstate, mail) {
if timestamp_sent > peerstate.last_seen_autocrypt
&& mail.ctype.mimetype != "multipart/report"
// Disallowing keychanges is disabled for now:
// && decryption_info.dkim_results.allow_keychange
{
peerstate.degrade_encryption(timestamp_sent);
}
@@ -506,13 +516,6 @@ impl MimeMessage {
parser.heuristically_parse_ndn(context).await;
parser.parse_headers(context).await?;
// Disallowing keychanges is disabled for now
// if !decryption_info.dkim_results.allow_keychange {
// for part in parser.parts.iter_mut() {
// part.error = Some("Seems like DKIM failed, this either is an attack or (more likely) a bug in Authentication-Results checking. Please tell us about this at https://support.delta.chat.".to_string());
// }
// }
if parser.is_mime_modified {
parser.decoded_data = mail_raw;
}
@@ -556,19 +559,25 @@ impl MimeMessage {
/// Parses avatar action headers.
async fn parse_avatar_headers(&mut self, context: &Context) {
if let Some(header_value) = self.get_header(HeaderDef::ChatGroupAvatar).cloned() {
self.group_avatar = self.avatar_action_from_header(context, header_value).await;
if let Some(header_value) = self.get_header(HeaderDef::ChatGroupAvatar) {
self.group_avatar = self
.avatar_action_from_header(context, header_value.to_string())
.await;
}
if let Some(header_value) = self.get_header(HeaderDef::ChatUserAvatar).cloned() {
self.user_avatar = self.avatar_action_from_header(context, header_value).await;
if let Some(header_value) = self.get_header(HeaderDef::ChatUserAvatar) {
self.user_avatar = self
.avatar_action_from_header(context, header_value.to_string())
.await;
}
}
fn parse_videochat_headers(&mut self) {
if let Some(value) = self.get_header(HeaderDef::ChatContent).cloned() {
if let Some(value) = self.get_header(HeaderDef::ChatContent) {
if value == "videochat-invitation" {
let instance = self.get_header(HeaderDef::ChatWebrtcRoom).cloned();
let instance = self
.get_header(HeaderDef::ChatWebrtcRoom)
.map(|s| s.to_string());
if let Some(part) = self.parts.first_mut() {
part.typ = Viewtype::VideochatInvitation;
part.param
@@ -594,6 +603,7 @@ impl MimeMessage {
| Viewtype::Audio
| Viewtype::Voice
| Viewtype::Video
| Viewtype::Vcard
| Viewtype::File
| Viewtype::Webxdc => true,
Viewtype::Unknown | Viewtype::Text | Viewtype::VideochatInvitation => false,
@@ -662,32 +672,34 @@ impl MimeMessage {
self.squash_attachment_parts();
}
if let Some(ref subject) = self.get_subject() {
let mut prepend_subject = true;
if !self.decrypting_failed {
let colon = subject.find(':');
if colon == Some(2)
|| colon == Some(3)
|| self.has_chat_version()
|| subject.contains("Chat:")
{
prepend_subject = false
if !context.get_config_bool(Config::Bot).await? {
if let Some(ref subject) = self.get_subject() {
let mut prepend_subject = true;
if !self.decrypting_failed {
let colon = subject.find(':');
if colon == Some(2)
|| colon == Some(3)
|| self.has_chat_version()
|| subject.contains("Chat:")
{
prepend_subject = false
}
}
}
// For mailing lists, always add the subject because sometimes there are different topics
// and otherwise it might be hard to keep track:
if self.is_mailinglist_message() && !self.has_chat_version() {
prepend_subject = true;
}
// For mailing lists, always add the subject because sometimes there are different topics
// and otherwise it might be hard to keep track:
if self.is_mailinglist_message() && !self.has_chat_version() {
prepend_subject = true;
}
if prepend_subject && !subject.is_empty() {
let part_with_text = self
.parts
.iter_mut()
.find(|part| !part.msg.is_empty() && !part.is_reaction);
if let Some(part) = part_with_text {
part.msg = format!("{} – {}", subject, part.msg);
if prepend_subject && !subject.is_empty() {
let part_with_text = self
.parts
.iter_mut()
.find(|part| !part.msg.is_empty() && !part.is_reaction);
if let Some(part) = part_with_text {
part.msg = format!("{} – {}", subject, part.msg);
}
}
}
}
@@ -813,8 +825,16 @@ impl MimeMessage {
.map(|s| s.to_string())
}
pub fn get_header(&self, headerdef: HeaderDef) -> Option<&String> {
self.headers.get(headerdef.get_headername())
pub fn get_header(&self, headerdef: HeaderDef) -> Option<&str> {
self.headers
.get(headerdef.get_headername())
.map(|s| s.as_str())
}
/// Returns `Chat-Group-ID` header value if it is a valid group ID.
pub fn get_chat_group_id(&self) -> Option<&str> {
self.get_header(HeaderDef::ChatGroupId)
.filter(|s| validate_id(s))
}
async fn parse_mime_recursive<'a>(
@@ -1567,9 +1587,8 @@ impl MimeMessage {
.get_header_value(HeaderDef::MessageId)
.and_then(|v| parse_message_id(&v).ok())
{
let mut to_list = get_all_addresses_from_header(&report.headers, |header_key| {
header_key == "x-failed-recipients"
});
let mut to_list =
get_all_addresses_from_header(&report.headers, "x-failed-recipients");
let to = if to_list.len() == 1 {
Some(to_list.pop().unwrap())
} else {
@@ -1917,16 +1936,11 @@ fn get_mime_type(
let mimetype = mail.ctype.mimetype.parse::<Mime>()?;
let viewtype = match mimetype.type_() {
mime::TEXT => {
if !is_attachment_disposition(mail) {
match mimetype.subtype() {
mime::PLAIN | mime::HTML => Viewtype::Text,
_ => Viewtype::File,
}
} else {
Viewtype::File
}
}
mime::TEXT => match mimetype.subtype() {
mime::VCARD if is_valid_deltachat_vcard(mail) => Viewtype::Vcard,
mime::PLAIN | mime::HTML if !is_attachment_disposition(mail) => Viewtype::Text,
_ => Viewtype::File,
},
mime::IMAGE => match mimetype.subtype() {
mime::GIF => Viewtype::Gif,
mime::SVG => Viewtype::File,
@@ -1974,6 +1988,17 @@ fn is_attachment_disposition(mail: &mailparse::ParsedMail<'_>) -> bool {
.any(|(key, _value)| key.starts_with("filename"))
}
fn is_valid_deltachat_vcard(mail: &mailparse::ParsedMail) -> bool {
let Ok(body) = &mail.get_body() else {
return false;
};
let contacts = deltachat_contact_tools::parse_vcard(body);
if let [c] = &contacts[..] {
return deltachat_contact_tools::may_be_valid_addr(&c.addr);
}
false
}
/// Tries to get attachment filename.
///
/// If filename is explicitly specified in Content-Disposition, it is
@@ -2030,36 +2055,48 @@ fn get_attachment_filename(
/// Returned addresses are normalized and lowercased.
pub(crate) fn get_recipients(headers: &[MailHeader]) -> Vec<SingleInfo> {
get_all_addresses_from_header(headers, |header_key| {
header_key == "to" || header_key == "cc"
})
let to_addresses = get_all_addresses_from_header(headers, "to");
let cc_addresses = get_all_addresses_from_header(headers, "cc");
let mut res = to_addresses;
res.extend(cc_addresses);
res
}
/// Returned addresses are normalized and lowercased.
pub(crate) fn get_from(headers: &[MailHeader]) -> Option<SingleInfo> {
let all = get_all_addresses_from_header(headers, |header_key| header_key == "from");
let all = get_all_addresses_from_header(headers, "from");
tools::single_value(all)
}
/// Returned addresses are normalized and lowercased.
pub(crate) fn get_list_post(headers: &[MailHeader]) -> Option<String> {
get_all_addresses_from_header(headers, |header_key| header_key == "list-post")
get_all_addresses_from_header(headers, "list-post")
.into_iter()
.next()
.map(|s| s.addr)
}
fn get_all_addresses_from_header(
headers: &[MailHeader],
pred: fn(String) -> bool,
) -> Vec<SingleInfo> {
/// Extracts all addresses from the header named `header`.
///
/// If multiple headers with the same name are present,
/// the last one is taken.
/// This is because DKIM-Signatures apply to the last
/// headers, and more headers may be added
/// to the beginning of the messages
/// without invalidating the signature
/// unless the header is "oversigned",
/// i.e. included in the signature more times
/// than it appears in the mail.
fn get_all_addresses_from_header(headers: &[MailHeader], header: &str) -> Vec<SingleInfo> {
let mut result: Vec<SingleInfo> = Default::default();
headers
if let Some(header) = headers
.iter()
.filter(|header| pred(header.get_key().to_lowercase()))
.filter_map(|header| mailparse::addrparse_header(header).ok())
.for_each(|addrs| {
.rev()
.find(|h| h.get_key().to_lowercase() == header)
{
if let Ok(addrs) = mailparse::addrparse_header(header) {
for addr in addrs.iter() {
match addr {
mailparse::MailAddr::Single(ref info) => {
@@ -2078,7 +2115,8 @@ fn get_all_addresses_from_header(
}
}
}
});
}
}
result
}
@@ -2378,6 +2416,22 @@ mod tests {
assert!(recipients.iter().any(|info| info.addr == "abc@bcd.com"));
assert!(recipients.iter().any(|info| info.addr == "def@def.de"));
assert_eq!(recipients.len(), 2);
// If some header is present multiple times,
// only the last one must be used.
let raw = b"From: alice@example.org\n\
TO: mallory@example.com\n\
To: mallory@example.net\n\
To: bob@example.net\n\
Content-Type: text/plain\n\
Chat-Version: 1.0\n\
\n\
Hello\n\
";
let mail = mailparse::parse_mail(&raw[..]).unwrap();
let recipients = get_recipients(&mail.headers);
assert!(recipients.iter().any(|info| info.addr == "bob@example.net"));
assert_eq!(recipients.len(), 1);
}
#[test]
@@ -3933,4 +3987,70 @@ Content-Disposition: reaction\n\
Ok(())
}
/// Tests that subject is not prepended to the message
/// when bot receives it.
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_bot_no_subject() {
let context = TestContext::new().await;
context.set_config(Config::Bot, Some("1")).await.unwrap();
let raw = br#"Message-ID: <foobar@example.org>
From: foo <foo@example.org>
Subject: Some subject
To: bar@example.org
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
/help
"#;
let message = MimeMessage::from_bytes(&context.ctx, &raw[..], None)
.await
.unwrap();
assert_eq!(message.get_subject(), Some("Some subject".to_string()));
assert_eq!(message.parts.len(), 1);
assert_eq!(message.parts[0].typ, Viewtype::Text);
// Not "Some subject – /help"
assert_eq!(message.parts[0].msg, "/help");
}
/// Tests that Delta Chat takes the last header value
/// rather than the first one if multiple headers
/// are present.
///
/// DKIM signature applies to the last N headers
/// if header name is included N times in
/// DKIM-Signature.
///
/// If the client takes the first header
/// rather than the last, it can be fooled
/// into using unsigned header
/// when signed one is present
/// but not protected by oversigning.
///
/// See
/// <https://www.zone.eu/blog/2024/05/17/bimi-and-dmarc-cant-save-you/>
/// for reference.
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_take_last_header() {
let context = TestContext::new().await;
// Mallory added second From: header.
let raw = b"From: mallory@example.org\n\
From: alice@example.org\n\
Content-Type: text/plain\n\
Chat-Version: 1.0\n\
\n\
Hello\n\
";
let mimeparser = MimeMessage::from_bytes(&context.ctx, &raw[..], None)
.await
.unwrap();
assert_eq!(
mimeparser.get_header(HeaderDef::From_).unwrap(),
"alice@example.org"
);
}
}
+5
View File
@@ -48,6 +48,11 @@ pub enum Param {
/// For Messages: message is encrypted, outgoing: guarantee E2EE or the message is not send
GuaranteeE2ee = b'c',
/// For Messages: quoted message is encrypted.
///
/// If this message is sent unencrypted, quote text should be replaced.
ProtectQuote = b'0',
/// For Messages: decrypted with validation errors or without mutual set, if neither
/// 'c' nor 'e' are preset, the messages is only transport encrypted.
ErroneousE2ee = b'e',

Some files were not shown because too many files have changed in this diff Show More