Compare commits

...
Author SHA1 Message Date
Nico de Haenandiequidoo ac13c26cfa Expect instead allow clippy large enum
Co-authored-by: iequidoo <117991069+iequidoo@users.noreply.github.com>
2026-01-24 12:22:23 +01:00
Nico de Haen ace4dc72c1 api: deprecate self_in_group from chatlist item
allow clippy::large_enum_variant
2026-01-24 10:12:56 +01:00
Nico de Haen 74fc2396b6 api: rename properties to be more consistent
There is no need to have different names for the same props in chat and
chatListItem. So let's rename them:
- is_self_in_group → self_in_group
- is_archived → archived
- is_pinned → pinned
- avatar_path → profile_image
- is_device_chat → is_device_talk
2026-01-20 15:48:16 +01:00
Simon LauxandWofWca 4ccd3cb665 api(rust and jsonrpc): marknoticed_all_chats method to mark all chats as notices, including muted ones. (#7709)
made for solving

https://github.com/deltachat/deltachat-desktop/issues/5891#issuecomment-3687566470

will also be more efficient, because desktop currently loads all fresh
messages to find out which chats to mark as noticed.
https://github.com/deltachat/deltachat-desktop/blob/76d32bfc9364d53e1ffea40fcf4a60ad9a8482ee/packages/frontend/src/components/AccountListSidebar/AccountItem.tsx#L334


# progress
- [x] implementation
- [x] write a test
- [x] make a pr to use it in desktop
https://github.com/deltachat/deltachat-desktop/pull/5923
- [x] address review comments

---------

Co-authored-by: WofWca <wofwca@protonmail.com>
2026-01-20 08:52:59 +00:00
Hocuri f5e1e2678b fix: Make it possible to leave and immediately delete a chat (#7744)
Without this PR, if you leave and immediately delete a chat, the leave
message won't be sent.

This is needed for
https://github.com/deltachat/deltachat-android/issues/4158.
2026-01-19 15:07:19 +01:00
Hocuri c3a5e3ac0d feat: In teamprofiles, don't mark chat as read on outgoing message (#7717)
Fix https://github.com/chatmail/core/issues/7704
2026-01-19 11:39:06 +00:00
Simon Laux b2f31c8148 api(rust, jsonrpc): add get_message_read_receipt_count method (#7732)
closes #7728
2026-01-19 11:37:10 +00:00
Hocuri 29c57ad065 fix: Don't remember old channel members in the database (#7716)
This PR fixes a bug that old channel members were remembered in the
database even after they left the channel. Concretely, they remained in
the `past_members` table that's only meant for groups.

Though it was not a bad bug; we're anyways not cleaning up old contacts.
2026-01-19 11:35:01 +01:00
Simon Laux 82a0d6b0ab fix: more reliable parsing of dclogin: links with ip address as host (#7734)
Also adds a test for it.

closes #7733
2026-01-18 16:11:59 +00:00
link2xt 5ff323ce15 feat(pgp): use preferred hash algorithm for signing instead of hardcoded SHA256
There is no difference for RSA and Ed25519,
the only signing keys that we generate.
The both use SHA256:
<https://github.com/rpgp/rpgp/blob/7e3b6c0af21e4a45c2a2dcba67f0b649e0b6e456/src/types/params/public.rs#L231-L234>

The only difference is for the possible future PQC signing keys
and imported NIST P-512 and NIST P-384 keys.
2026-01-18 03:59:12 +00:00
iequidoo a67a5299bf Send and apply MDNs to self (#7005)
We currently synchronize "seen" status of messages by setting `\Seen` flag on IMAP and then looking
for new `\Seen` flags using `CONDSTORE` IMAP extension. This approach has multiple disadvantages:
- It requires that the server supports `CONDSTORE` extension. For example Maddy does not support
  CONDSTORE yet: https://github.com/foxcpp/maddy/issues/727
- It leaks the seen status to the server without any encryption.
- It requires more than just store-and-forward queues and prevents replacing IMAP with simpler
  protocols like POP3 or UUCP or some HTTP-based API for queue polling.

A simpler approach is to send MDNs to self when `Config::BccSelf` (aka multidevice) is enabled,
regardless of whether the message requested and MDN. If MDN was requested and we have MDNs enabled,
then also send to the message sender, but MDN to self is sent regardless of whether read receipts
are actually enabled.

`sync_seen_flags()` and `CONDSTORE` check is better completely removed, maybe after one
release. `store_seen_flags_on_imap()` can be kept for unencrypted non-chat messages.

One potential problem with sending MDNs is that it may trigger ratelimits on some providers and
count as another recipient.
2026-01-17 20:54:35 -03:00
link2xt 659d21aa9d docs: fix formatting of indoc! link 2026-01-17 14:40:17 +00:00
link2xt 8f604e74ec fix: do not resolve ICE server hostnames during IMAP loop
Hostname resolution may timeout if DNS servers are not responding.
It is also not necessary to resolve fallback ICE server hostnames
if the user is not going to use calls.
2026-01-17 12:55:21 +00:00
Simon Laux e1ebf3e96d refactor: don't use concat! in sql statements (#7720) 2026-01-15 22:44:53 +00:00
Simon Laux 76171aea2e fix: hide incoming broadcasts in DC_GCL_FOR_FORWARDING (#7726)
you can't write to those chats, so you also can not forward to them.

Closes #7702
2026-01-15 22:26:05 +00:00
Hocuri 96b8d1720e fix: Use only lowercase letters for stats id (#7700)
If the user enables statistics-sending in the advanced settings, they
will be asked whether they also want to take part in a survey. We use a
short ID to then link the survey result to the sent statistics.

However, the survey website didn't like our base64 ids, especially the
fact that the id could contain a `-`. This PR makes it so that the id
only contains lowecase letters.
2026-01-15 18:51:04 +01:00
missytakeandiequidoo 47b49fd02e api(jsonrpc): add run_until parameter for bots (#7688)
This commit also makes testing hooks easier, as it allows to process
events and run hooks on them, until a certain event occurs.

---------

Co-authored-by: iequidoo <117991069+iequidoo@users.noreply.github.com>
2026-01-14 19:58:44 +01:00
iequidoo f50e3d6ffa feat: Don't scale up Origin of multiple and broadcast recipients when sending a message
84161f4202 promotes group members to `Origin::IncomingTo` when
accepting it, instead of `CreateChat` as before, but this changes almost nothing because it happens
rarely that the user only accepts a group and writes nothing there soon. Now if a message has
multiple recipients, i.e. it's a 3-or-more-member group, or if it's a broadcast message, we don't
scale up its recipients to `Origin::OutgoingTo`.
2026-01-14 14:32:59 -03:00
Simon Laux 2ecb537307 api!: jsonrpc: remove contacts from FullChat. To migrate load contacts on demand via get_contacts_by_ids using FullChat.contactIds (#7282)
closes #6945

### Why not deprecate it instead?
Because empty contact list is a more annoying-to-find bug in your app
than failing to build or getting undefined at runtime.

Also you would not see the deprecated hints anyway because for that you
need autogenerated types and those only exist for typescript currently.
2026-01-14 15:35:57 +00:00
Hocuri ccae73f6db feat: Don't put text into post-message (#7714)
Before this PR, when a user with current main sends a large message to a
user with an old Delta Chat (before #7431), the text will be duplicated:
One message will arrive with only the text, and one message with
attachment+text.

This PR changes this - there will be one message with only the text, and
one message with only the attachment.

If we want to guard against lost pre-messages, then we can revert this
PR in a few months, though I'm not sure that's necessary - it's unlikely
that the small pre-message gets lost but the big post-message gets
through.
2026-01-14 11:10:17 +01:00
iequidoo fce91f3ee0 feat: Disable partial search by contact address
Implement suggestion from #7477 so that there's no incremental search by contact address in UIs,
only direct matches are returned.
2026-01-13 22:13:54 -03:00
Hocuri d446a16fc6 Sync broadcast subscribers list (#7578)
fix #7497
2026-01-13 15:04:51 +01:00
iequidoo c3a6e48882 docs: set_config_from_qr() configures context for "DCACCOUNT:" and "DCLOGIN:" QRs (#7450)
Also remove "you can now call 'configure'" from the REPL output, probably users of the REPL tool can
read the code documentation to know when 'configure' should be run.
2026-01-13 01:26:35 -03:00
Simon Laux 46ec3a469b fix: logging errors in deltachat-rpc-server during startup (#7707)
Before this PR there were cases where error messages never reach the
user because logging was not initialized yet.
This PR moves log initialization to the start of the program, so that
all logged messages reach the user.
2026-01-12 00:18:23 +00:00
iequidoo fe3b1ea16d fix: Only emit TransportsModified if transports are really modified
Otherwise it's not possible to write tests reliably because sync messages may be executed multiple
times if they arrive from different transports. This should fix flaky
`test_transport_synchronization`.

Also always emit `TransportsModified` if the primary transport is changed by a sync message, even if
it doesn't contain `SyncData::Transports`.

Also don't decrease `add_timestamp` in `save_transport()` if nothing else changes, this doesn't make
sense.
2026-01-11 16:52:39 -03:00
iequidoo ed300b6f97 feat: Execute sync message before checking for primary transport update
This way, if the sync message updates transports, the check for a new primary transport is done
against the updated transport list which is more reliable.
2026-01-11 16:52:39 -03:00
iequidoo e456be4e21 fix: Send bcc-self messages to all own relays (#7656)
This fixes the bug when a new transport doesn't become primary on the 2nd device because INBOX from
the new transport isn't fully fetched. Now the `Transports` sync message is received from the old
transport, but as it has updated "From", it updates the primary transport correspondingly. NB: I/O
for the new primary transport isn't immediately started however, this needs a separate fix.
2026-01-11 16:52:39 -03:00
iequidoo ba4055b7df test(rpc-client): Replace remaining print()s with logging (#6082)
This is to fix tests failing with `OSError: [Errno 9] Bad file descriptor`. Maybe stdout closes
earlier than stderr, before the test finishes, not sure. For reference, the previous commit removing
print()s is 800edc6fce.
2026-01-11 14:52:13 -03:00
B. Petersen c06f53cb86 docs: fix chat types
- DC_CHAT_TYPE_BROADCAST does no longer exist
- DC_CHAT_TYPE_MAILINGLIST is no longer always read-only
- avoid double docs by moving everything to DC_CHAT_TYPE group
  (here, things were also updated recently)
2026-01-11 01:48:55 +01:00
link2xt 13dafa46b5 fix: take transport_id into account when marking messages with \Seen flags
Otherwise wrong IMAP client corresponding to a different transport
may pick up the job to mark the message as seen there,
and fail to do it as the message does not exist.
It may also mark the wrong message with the correct folder
and UID, but wrong IMAP server.
2026-01-09 18:40:46 +00:00
iequidoo d552250dc4 test: Port test_dont_move_sync_msgs to JSON-RPC (#7676) 2026-01-09 15:26:00 -03:00
Simon Laux 1383e790c3 feat: connectivity view: move quota up and combine with IMAP state. (#7653)
like described in
https://github.com/chatmail/core/pull/7630#discussion_r2641514867


## classical account (with legacy option mvbox)

|||
|---|---|
| <img width="891" height="635" alt="image"
src="https://github.com/user-attachments/assets/723f3dba-79dc-4b57-a14f-c5879c1a1d1d"
/>| <img width="890" height="578" alt="image"
src="https://github.com/user-attachments/assets/d45eaf35-d7b2-40d4-8c37-bbc77947c27d"
/>|

## multi transport

|||
|---|---|
|<img width="891" height="1236" alt="image"
src="https://github.com/user-attachments/assets/053cb088-7d9d-4591-b2bc-6b49399d33a0"
/> |<img width="885" height="1230" alt="image"
src="https://github.com/user-attachments/assets/c455b4f1-f521-4ae8-8884-9042af62ca46"
/>|
2026-01-09 17:20:59 +00:00
Simon Laux b536902827 fix: do not show contact address in message info (#7695)
closes #7686
2026-01-09 17:04:22 +00:00
2631745a57 feat: pre-messages / next version of download on demand (#7371)
Closes <https://github.com/chatmail/core/issues/7367>

Co-authored-by: iequidoo <dgreshilov@gmail.com>
Co-authored-by: Hocuri <hocuri@gmx.de>
2026-01-08 22:14:32 +00:00
link2xt 46bbe5f077 chore(release): prepare for 2.37.0 2026-01-08 20:45:33 +00:00
iequidoo 0f14edd5d9 feat: Don't download group messages unconditionally
There was a comment that group messages should always be downloaded to avoid inconsistent group
state, but this is solved by the group consistency algo nowadays in the sense that inconsistent
group state won't spread to other members if we send to the group.
2026-01-08 16:57:39 -03:00
Simon Laux fe6e942191 api: cffi api to create account manager with existing events channel to see events emitted during startup. dc_event_channel_new, dc_event_channel_unref, dc_event_channel_get_event_emitter and dc_accounts_new_with_event_channel (#7609)
closes #7606
2026-01-08 18:08:00 +00:00
B. Petersen 67aac12995 docs: update instructions to UI where to display the address 2026-01-08 17:19:54 +01:00
B. Petersen f2fb59f0cc test that channel summary does not have sender name 2026-01-08 17:17:16 +01:00
B. Petersen 55ab1b86f7 feat: more text instead of sender in channel summary
the sender is usually always the same and is not needed already in the summary;
making more place for the message instead
2026-01-08 17:17:16 +01:00
Hocuri ceba687df3 feat: Config option to skip seen synchronization (#7694)
At urgent request from @hpk42, this adds a config option `team_profile`.
This option is only settable via SQLite (not exposed in the UI), and the
only thing it does is disabling synchronization of seen status.

I tested manually on my Android phone that it works.

Not straigthforward to write an automatic test, because we want to test that something
does _not_ happen (i.e. that the seen status is _not_ synchronized), and
it's not clear how long to wait before we check.

Probably it's fine to just not add a test.

This is what I tried:

```python
@pytest.mark.parametrize("team_profile", [True, False])
def test_markseen_basic(team_profile, acfactory):
    """
    Test that seen status is synchronized iff `team_profile` isn't set.
    """
    alice, bob = acfactory.get_online_accounts(2)

    # Bob sets up a second device.
    bob2 = bob.clone()
    bob2.start_io()

    alice_chat_bob = alice.create_chat(bob)
    bob.create_chat(alice)
    bob2.create_chat(alice)
    alice_chat_bob.send_text("Hello Bob!")

    message = bob.wait_for_incoming_msg()
    message2 = bob2.wait_for_incoming_msg()
    assert message2.get_snapshot().state == MessageState.IN_FRESH

    message.mark_seen()

    # PROBLEM: We're not waiting 'long enough',
    # so, the 'state == MessageState.IN_SEEN' assertion below fails
    bob.create_chat(bob).send_text("Self-sent message")
    self_sent = bob2.wait_for_msg(EventType.MSGS_CHANGED)
    assert self_sent.get_snapshot().text == "Self-sent message"

    if team_profile:
        assert message2.get_snapshot().state == MessageState.IN_FRESH
    else:
        assert message2.get_snapshot().state == MessageState.IN_SEEN
```
2026-01-08 17:06:03 +01:00
link2xt 7e811469b3 api: add who_can_call_me config option 2026-01-07 22:00:54 +00:00
link2xt cdacad235e chore: update lru 0.12.3 to 0.12.5 and add RUSTSEC-2026-0002 exception
Closes https://github.com/chatmail/core/issues/7692
2026-01-07 20:18:32 +00:00
link2xt c766397abc test: regression test for vc-request encrypted by the server 2026-01-07 15:47:16 +00:00
link2xt 14a59afd5d fix: do not rely on Secure-Join header to detect {vc,vg}-request 2026-01-07 15:47:16 +00:00
dependabot[bot] 9c883e6424 chore(cargo): bump rsa from 0.9.9 to 0.9.10
Bumps [rsa](https://github.com/RustCrypto/RSA) from 0.9.9 to 0.9.10.
- [Changelog](https://github.com/RustCrypto/RSA/blob/v0.9.10/CHANGELOG.md)
- [Commits](https://github.com/RustCrypto/RSA/compare/v0.9.9...v0.9.10)

---
updated-dependencies:
- dependency-name: rsa
  dependency-version: 0.9.10
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-01-06 17:08:26 -03:00
Simon Laux 9d7db20225 refactor(ffi): replace implicit drop in cffi with explicit drop(Arc::from_raw(var)) (#7664)
for improved clarity and consistency with existing code.
2026-01-05 19:27:48 +00:00
Simon Lauxandiequidoo fdb583b5e9 api: jsonrpc api get_all_ui_config_keys to get all "ui.*" config keys (#7579)
Adds an api to get all ui config keys. There already is an option to get
all normal config keys (`"sys.config_keys"`), but before this pr there
was no way to get all `ui.*` config keys.

#### Why is this api needed?

For webxdc cleanup on desktop, which stores window position in a ui
config key (such as `ui.desktop.webxdcBounds.676464`) as soon as a
webxdc is opened since many versions now. So listing all ui keys is a
good way for us to find out which webxdc may have web data stored.
unfortunately electron does not (yet?) have a way to list all origins
that have web-data like android does, so this is the next best thing we
can do before itterating all possible ids, see also
https://github.com/deltachat/deltachat-desktop/issues/5758.

#### Why is this only a jsonrpc api and not another special/virtual
config key like `"sys.config_keys"`?

r10s indicated that `ui.*`-config keys are barely used
(https://github.com/deltachat/deltachat-desktop/issues/5790#issuecomment-3598512802),
so I thought it makes more sense to add it as dedicated api which's
existentence is checked by the typechecker, so it will be easier to not
miss it when we should remove the api again in the future.

But we could also do a dedicated special/virtual config key for it, if
you think that is better, this is easy to change.

---------

Co-authored-by: iequidoo <117991069+iequidoo@users.noreply.github.com>
2026-01-03 21:14:00 +01:00
link2xt 8d6f4b0354 chore(release): prepare for 2.36.0 2026-01-03 18:39:16 +00:00
link2xt 284469363e docs: remove references to sentbox_watch config
This config was already removed in 2.23.0.
2026-01-03 13:40:24 +00:00
dependabot[bot] 6078c79020 chore(cargo): bump criterion from 0.7.0 to 0.8.1
Bumps [criterion](https://github.com/criterion-rs/criterion.rs) from 0.7.0 to 0.8.1.
- [Release notes](https://github.com/criterion-rs/criterion.rs/releases)
- [Changelog](https://github.com/criterion-rs/criterion.rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/criterion-rs/criterion.rs/compare/criterion-plot-v0.7.0...criterion-v0.8.1)

---
updated-dependencies:
- dependency-name: criterion
  dependency-version: 0.8.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
...
2026-01-02 15:50:49 -03:00
dependabot[bot] 161e5ae358 chore(cargo): bump rustls-pki-types from 1.13.0 to 1.13.2
Bumps [rustls-pki-types](https://github.com/rustls/pki-types) from 1.13.0 to 1.13.2.
- [Release notes](https://github.com/rustls/pki-types/releases)
- [Commits](https://github.com/rustls/pki-types/compare/v/1.13.0...v/1.13.2)

---
updated-dependencies:
- dependency-name: rustls-pki-types
  dependency-version: 1.13.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-01-02 15:32:06 -03:00
dependabot[bot] a66859ebf2 chore(cargo): bump log from 0.4.28 to 0.4.29
Bumps [log](https://github.com/rust-lang/log) from 0.4.28 to 0.4.29.
- [Release notes](https://github.com/rust-lang/log/releases)
- [Changelog](https://github.com/rust-lang/log/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/log/compare/0.4.28...0.4.29)

---
updated-dependencies:
- dependency-name: log
  dependency-version: 0.4.29
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-01-02 15:29:04 -03:00
dependabot[bot] de902babc3 chore(cargo): bump hyper-util from 0.1.18 to 0.1.19
Bumps [hyper-util](https://github.com/hyperium/hyper-util) from 0.1.18 to 0.1.19.
- [Release notes](https://github.com/hyperium/hyper-util/releases)
- [Changelog](https://github.com/hyperium/hyper-util/blob/master/CHANGELOG.md)
- [Commits](https://github.com/hyperium/hyper-util/compare/v0.1.18...v0.1.19)

---
updated-dependencies:
- dependency-name: hyper-util
  dependency-version: 0.1.19
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-01-02 14:48:44 -03:00
dependabot[bot] 98a8679779 chore(cargo): bump tracing from 0.1.41 to 0.1.44
Bumps [tracing](https://github.com/tokio-rs/tracing) from 0.1.41 to 0.1.44.
- [Release notes](https://github.com/tokio-rs/tracing/releases)
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-0.1.41...tracing-0.1.44)

---
updated-dependencies:
- dependency-name: tracing
  dependency-version: 0.1.44
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-01-02 14:46:33 -03:00
dependabot[bot] 8ce3ecc809 chore(cargo): bump libc from 0.2.177 to 0.2.178
Bumps [libc](https://github.com/rust-lang/libc) from 0.2.177 to 0.2.178.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Changelog](https://github.com/rust-lang/libc/blob/0.2.178/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.177...0.2.178)

---
updated-dependencies:
- dependency-name: libc
  dependency-version: 0.2.178
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-01-02 14:39:44 -03:00
dependabot[bot] 50a1f907a5 chore(cargo): bump tempfile from 3.23.0 to 3.24.0
Bumps [tempfile](https://github.com/Stebalien/tempfile) from 3.23.0 to 3.24.0.
- [Changelog](https://github.com/Stebalien/tempfile/blob/master/CHANGELOG.md)
- [Commits](https://github.com/Stebalien/tempfile/compare/v3.23.0...v3.24.0)

---
updated-dependencies:
- dependency-name: tempfile
  dependency-version: 3.24.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-01-02 14:18:29 -03:00
dependabot[bot] c2bf2a32b5 chore(cargo): bump toml from 0.9.8 to 0.9.10+spec-1.1.0
Bumps [toml](https://github.com/toml-rs/toml) from 0.9.8 to 0.9.10+spec-1.1.0.
- [Commits](https://github.com/toml-rs/toml/compare/toml-v0.9.8...toml-v0.9.10)

---
updated-dependencies:
- dependency-name: toml
  dependency-version: 0.9.10+spec-1.1.0
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-01-02 04:41:37 -03:00
dependabot[bot] 84710384fe chore(cargo): bump uuid from 1.18.1 to 1.19.0
Bumps [uuid](https://github.com/uuid-rs/uuid) from 1.18.1 to 1.19.0.
- [Release notes](https://github.com/uuid-rs/uuid/releases)
- [Commits](https://github.com/uuid-rs/uuid/compare/v1.18.1...v1.19.0)

---
updated-dependencies:
- dependency-name: uuid
  dependency-version: 1.19.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-01-02 04:38:57 -03:00
dependabot[bot] f60fce22ed chore(cargo): bump serde_json from 1.0.145 to 1.0.147
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.145 to 1.0.147.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.145...v1.0.147)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-version: 1.0.147
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-01-02 04:36:56 -03:00
link2xt 0d2f2b3266 refactor(ffi): remove one pointer indirection for dc_accounts_t
Arc can be converted directly to raw pointers,
so there is no need to create a pointer to Arc
using a Box.
2026-01-01 15:34:55 +00:00
Casper Zandbergenandlink2xt 516f0a1a98 fix: Don't send webxdc notification for notify: "*" when chat is muted (#7658)
- webxdc notify specifically to Bob: notifies Bob even when chat is
muted
- webxdc notify to everyone ("*"): notifies Bob if he does not have the
chat muted

This aligns with how we handle notifications with quote replies.

---------

Co-authored-by: link2xt <link2xt@testrun.org>
2025-12-30 23:14:35 +01:00
link2xt 25750de4e1 feat: send sync messages over SMTP and do not move them to mvbox 2025-12-26 10:58:33 +00:00
link2xt a89ce8ce7a docs: update documentation for TransportsModified event
It is a follow-up to https://github.com/chatmail/core/pull/7643
Event is not emitted when the transports are modified on this device
and we should consistently say that this event is not only for testing.
2025-12-25 11:16:36 +00:00
Simon Laux 9ac64ea6b9 feat: connectivity view: quota for all transports (#7630)
- **show quota of all relays**
- **remove `DC_STR_STORAGE_ON_DOMAIN` stock string**
- renames the quota section to "Relay Capacity" until we come up with a
better name in
https://github.com/chatmail/core/issues/7580#issuecomment-3633803432

closes #7591

<img width="300" alt="image"
src="https://github.com/user-attachments/assets/1909dccd-e6b3-42e6-963f-004b2b464db7"
/> <img width="300" alt="image"
src="https://github.com/user-attachments/assets/1e97e67b-e0ed-492b-95a0-6ef12595abe4"
/>
2025-12-24 12:19:58 +00:00
iequidoo 294e23d82d docs: delete_chat(): Don't lie that messages aren't deleted from server
Messages are actually deleted from the server. I've checked this in Desktop.
2025-12-24 00:56:41 -03:00
link2xt 184736723f fix: reset options not available for chatmail on chatmail profiles 2025-12-24 03:23:23 +00:00
dependabot[bot] cea528ed61 chore(deps): bump cachix/install-nix-action from 31.8.4 to 31.9.0
Bumps [cachix/install-nix-action](https://github.com/cachix/install-nix-action) from 31.8.4 to 31.9.0.
- [Release notes](https://github.com/cachix/install-nix-action/releases)
- [Changelog](https://github.com/cachix/install-nix-action/blob/master/RELEASE.md)
- [Commits](https://github.com/cachix/install-nix-action/compare/0b0e072294b088b73964f1d72dfdac0951439dbd...4e002c8ec80594ecd40e759629461e26c8abed15)

---
updated-dependencies:
- dependency-name: cachix/install-nix-action
  dependency-version: 31.9.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-12-23 21:09:35 +00:00
dependabot[bot] 9b11f53da6 chore(deps): bump astral-sh/setup-uv from 7.1.4 to 7.1.6
Bumps [astral-sh/setup-uv](https://github.com/astral-sh/setup-uv) from 7.1.4 to 7.1.6.
- [Release notes](https://github.com/astral-sh/setup-uv/releases)
- [Commits](https://github.com/astral-sh/setup-uv/compare/1e862dfacbd1d6d858c55d9b792c756523627244...681c641aba71e4a1c380be3ab5e12ad51f415867)

---
updated-dependencies:
- dependency-name: astral-sh/setup-uv
  dependency-version: 7.1.6
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-12-23 21:09:01 +00:00
B. Petersen 5c339efb70 feat: add transports event to ffi 2025-12-23 12:04:32 +01:00
dependabot[bot] d71c163c7d chore(deps): bump actions/upload-artifact from 5 to 6
Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 5 to 6.
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](https://github.com/actions/upload-artifact/compare/v5...v6)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-version: '6'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-12-23 08:00:54 +00:00
dependabot[bot] 19fde9594f chore(deps): bump actions/download-artifact from 6 to 7
Bumps [actions/download-artifact](https://github.com/actions/download-artifact) from 6 to 7.
- [Release notes](https://github.com/actions/download-artifact/releases)
- [Commits](https://github.com/actions/download-artifact/compare/v6...v7)

---
updated-dependencies:
- dependency-name: actions/download-artifact
  dependency-version: '7'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-12-23 08:00:33 +00:00
iequidoo 20b3a06adf fix: inner_configure: Check Config::OnlyFetchMvbox before MvboxMove for multi-transport (#7637)
`Config::OnlyFetchMvbox` should be checked before `MvboxMove` because the latter makes no sense in
presense of `OnlyFetchMvbox` and even grayed out in the UIs in this case. Otherwise users will see
an error mentioning the wrong setting.
2025-12-23 01:22:11 -03:00
B. Petersen b0127fa381 fix: update fallback welcome message
this sets the welcome message to the one used by the current translations;
the fallback is used if that is not done for whatever reason.
2025-12-21 14:38:31 +01:00
iequidoo 6a293aebe2 test: Port test_import_export_online_all to JSON-RPC (#7411) 2025-12-19 01:17:59 -03:00
link2xt fd90493766 feat: add core version to receive_imf failure message 2025-12-18 14:44:49 +00:00
link2xt b1883c802b refactor: turn DC_VERSION_STR into &str 2025-12-18 14:44:49 +00:00
link2xt 71ee32b8b7 ci: pin GitHub Action references
Makes `zizmor` happy.
2025-12-18 14:43:27 +00:00
103 changed files with 5355 additions and 2546 deletions
+12 -12
View File
@@ -40,7 +40,7 @@ jobs:
- run: rustup override set $RUST_VERSION
shell: bash
- name: Cache rust cargo artifacts
uses: swatinem/rust-cache@v2
uses: swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5
- name: Run rustfmt
run: cargo fmt --all -- --check
- name: Run clippy
@@ -59,7 +59,7 @@ jobs:
with:
show-progress: false
persist-credentials: false
- uses: EmbarkStudios/cargo-deny-action@v2
- uses: EmbarkStudios/cargo-deny-action@76cd80eb775d7bbbd2d80292136d74d39e1b4918
with:
arguments: --all-features --workspace
command: check
@@ -91,7 +91,7 @@ jobs:
show-progress: false
persist-credentials: false
- name: Cache rust cargo artifacts
uses: swatinem/rust-cache@v2
uses: swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5
- name: Rustdoc
run: cargo doc --document-private-items --no-deps
@@ -134,10 +134,10 @@ jobs:
shell: bash
- name: Cache rust cargo artifacts
uses: swatinem/rust-cache@v2
uses: swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5
- name: Install nextest
uses: taiki-e/install-action@v2
uses: taiki-e/install-action@69e777b377e4ec209ddad9426ae3e0c1008b0ef3
with:
tool: nextest
@@ -168,13 +168,13 @@ jobs:
persist-credentials: false
- name: Cache rust cargo artifacts
uses: swatinem/rust-cache@v2
uses: swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5
- name: Build C library
run: cargo build -p deltachat_ffi
- name: Upload C library
uses: actions/upload-artifact@v5
uses: actions/upload-artifact@v6
with:
name: ${{ matrix.os }}-libdeltachat.a
path: target/debug/libdeltachat.a
@@ -194,13 +194,13 @@ jobs:
persist-credentials: false
- name: Cache rust cargo artifacts
uses: swatinem/rust-cache@v2
uses: swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5
- name: Build deltachat-rpc-server
run: cargo build -p deltachat-rpc-server
- name: Upload deltachat-rpc-server
uses: actions/upload-artifact@v5
uses: actions/upload-artifact@v6
with:
name: ${{ matrix.os }}-deltachat-rpc-server
path: ${{ matrix.os == 'windows-latest' && 'target/debug/deltachat-rpc-server.exe' || 'target/debug/deltachat-rpc-server' }}
@@ -243,7 +243,7 @@ jobs:
persist-credentials: false
- name: Download libdeltachat.a
uses: actions/download-artifact@v6
uses: actions/download-artifact@v7
with:
name: ubuntu-latest-libdeltachat.a
path: target/debug
@@ -293,7 +293,7 @@ jobs:
persist-credentials: false
- name: Download libdeltachat.a
uses: actions/download-artifact@v6
uses: actions/download-artifact@v7
with:
name: ${{ matrix.os }}-libdeltachat.a
path: target/debug
@@ -355,7 +355,7 @@ jobs:
run: pip install tox
- name: Download deltachat-rpc-server
uses: actions/download-artifact@v6
uses: actions/download-artifact@v7
with:
name: ${{ matrix.os }}-deltachat-rpc-server
path: target/debug
+47 -47
View File
@@ -34,13 +34,13 @@ jobs:
with:
show-progress: false
persist-credentials: false
- uses: cachix/install-nix-action@0b0e072294b088b73964f1d72dfdac0951439dbd # v31
- uses: cachix/install-nix-action@4e002c8ec80594ecd40e759629461e26c8abed15 # v31
- name: Build deltachat-rpc-server binaries
run: nix build .#deltachat-rpc-server-${{ matrix.arch }}-linux
- name: Upload binary
uses: actions/upload-artifact@v5
uses: actions/upload-artifact@v6
with:
name: deltachat-rpc-server-${{ matrix.arch }}-linux
path: result/bin/deltachat-rpc-server
@@ -58,13 +58,13 @@ jobs:
with:
show-progress: false
persist-credentials: false
- uses: cachix/install-nix-action@0b0e072294b088b73964f1d72dfdac0951439dbd # v31
- uses: cachix/install-nix-action@4e002c8ec80594ecd40e759629461e26c8abed15 # v31
- name: Build deltachat-rpc-server wheels
run: nix build .#deltachat-rpc-server-${{ matrix.arch }}-linux-wheel
- name: Upload wheel
uses: actions/upload-artifact@v5
uses: actions/upload-artifact@v6
with:
name: deltachat-rpc-server-${{ matrix.arch }}-linux-wheel
path: result/*.whl
@@ -82,13 +82,13 @@ jobs:
with:
show-progress: false
persist-credentials: false
- uses: cachix/install-nix-action@0b0e072294b088b73964f1d72dfdac0951439dbd # v31
- uses: cachix/install-nix-action@4e002c8ec80594ecd40e759629461e26c8abed15 # v31
- name: Build deltachat-rpc-server binaries
run: nix build .#deltachat-rpc-server-${{ matrix.arch }}
- name: Upload binary
uses: actions/upload-artifact@v5
uses: actions/upload-artifact@v6
with:
name: deltachat-rpc-server-${{ matrix.arch }}
path: result/bin/deltachat-rpc-server.exe
@@ -106,13 +106,13 @@ jobs:
with:
show-progress: false
persist-credentials: false
- uses: cachix/install-nix-action@0b0e072294b088b73964f1d72dfdac0951439dbd # v31
- uses: cachix/install-nix-action@4e002c8ec80594ecd40e759629461e26c8abed15 # v31
- name: Build deltachat-rpc-server wheels
run: nix build .#deltachat-rpc-server-${{ matrix.arch }}-wheel
- name: Upload wheel
uses: actions/upload-artifact@v5
uses: actions/upload-artifact@v6
with:
name: deltachat-rpc-server-${{ matrix.arch }}-wheel
path: result/*.whl
@@ -139,7 +139,7 @@ jobs:
run: cargo build --release --package deltachat-rpc-server --target ${{ matrix.arch }}-apple-darwin --features vendored
- name: Upload binary
uses: actions/upload-artifact@v5
uses: actions/upload-artifact@v6
with:
name: deltachat-rpc-server-${{ matrix.arch }}-macos
path: target/${{ matrix.arch }}-apple-darwin/release/deltachat-rpc-server
@@ -157,13 +157,13 @@ jobs:
with:
show-progress: false
persist-credentials: false
- uses: cachix/install-nix-action@0b0e072294b088b73964f1d72dfdac0951439dbd # v31
- uses: cachix/install-nix-action@4e002c8ec80594ecd40e759629461e26c8abed15 # v31
- name: Build deltachat-rpc-server binaries
run: nix build .#deltachat-rpc-server-${{ matrix.arch }}-android
- name: Upload binary
uses: actions/upload-artifact@v5
uses: actions/upload-artifact@v6
with:
name: deltachat-rpc-server-${{ matrix.arch }}-android
path: result/bin/deltachat-rpc-server
@@ -181,13 +181,13 @@ jobs:
with:
show-progress: false
persist-credentials: false
- uses: cachix/install-nix-action@0b0e072294b088b73964f1d72dfdac0951439dbd # v31
- uses: cachix/install-nix-action@4e002c8ec80594ecd40e759629461e26c8abed15 # v31
- name: Build deltachat-rpc-server wheels
run: nix build .#deltachat-rpc-server-${{ matrix.arch }}-android-wheel
- name: Upload wheel
uses: actions/upload-artifact@v5
uses: actions/upload-artifact@v6
with:
name: deltachat-rpc-server-${{ matrix.arch }}-android-wheel
path: result/*.whl
@@ -208,124 +208,124 @@ jobs:
with:
show-progress: false
persist-credentials: false
- uses: cachix/install-nix-action@0b0e072294b088b73964f1d72dfdac0951439dbd # v31
- uses: cachix/install-nix-action@4e002c8ec80594ecd40e759629461e26c8abed15 # v31
- name: Download Linux aarch64 binary
uses: actions/download-artifact@v6
uses: actions/download-artifact@v7
with:
name: deltachat-rpc-server-aarch64-linux
path: deltachat-rpc-server-aarch64-linux.d
- name: Download Linux aarch64 wheel
uses: actions/download-artifact@v6
uses: actions/download-artifact@v7
with:
name: deltachat-rpc-server-aarch64-linux-wheel
path: deltachat-rpc-server-aarch64-linux-wheel.d
- name: Download Linux armv7l binary
uses: actions/download-artifact@v6
uses: actions/download-artifact@v7
with:
name: deltachat-rpc-server-armv7l-linux
path: deltachat-rpc-server-armv7l-linux.d
- name: Download Linux armv7l wheel
uses: actions/download-artifact@v6
uses: actions/download-artifact@v7
with:
name: deltachat-rpc-server-armv7l-linux-wheel
path: deltachat-rpc-server-armv7l-linux-wheel.d
- name: Download Linux armv6l binary
uses: actions/download-artifact@v6
uses: actions/download-artifact@v7
with:
name: deltachat-rpc-server-armv6l-linux
path: deltachat-rpc-server-armv6l-linux.d
- name: Download Linux armv6l wheel
uses: actions/download-artifact@v6
uses: actions/download-artifact@v7
with:
name: deltachat-rpc-server-armv6l-linux-wheel
path: deltachat-rpc-server-armv6l-linux-wheel.d
- name: Download Linux i686 binary
uses: actions/download-artifact@v6
uses: actions/download-artifact@v7
with:
name: deltachat-rpc-server-i686-linux
path: deltachat-rpc-server-i686-linux.d
- name: Download Linux i686 wheel
uses: actions/download-artifact@v6
uses: actions/download-artifact@v7
with:
name: deltachat-rpc-server-i686-linux-wheel
path: deltachat-rpc-server-i686-linux-wheel.d
- name: Download Linux x86_64 binary
uses: actions/download-artifact@v6
uses: actions/download-artifact@v7
with:
name: deltachat-rpc-server-x86_64-linux
path: deltachat-rpc-server-x86_64-linux.d
- name: Download Linux x86_64 wheel
uses: actions/download-artifact@v6
uses: actions/download-artifact@v7
with:
name: deltachat-rpc-server-x86_64-linux-wheel
path: deltachat-rpc-server-x86_64-linux-wheel.d
- name: Download Win32 binary
uses: actions/download-artifact@v6
uses: actions/download-artifact@v7
with:
name: deltachat-rpc-server-win32
path: deltachat-rpc-server-win32.d
- name: Download Win32 wheel
uses: actions/download-artifact@v6
uses: actions/download-artifact@v7
with:
name: deltachat-rpc-server-win32-wheel
path: deltachat-rpc-server-win32-wheel.d
- name: Download Win64 binary
uses: actions/download-artifact@v6
uses: actions/download-artifact@v7
with:
name: deltachat-rpc-server-win64
path: deltachat-rpc-server-win64.d
- name: Download Win64 wheel
uses: actions/download-artifact@v6
uses: actions/download-artifact@v7
with:
name: deltachat-rpc-server-win64-wheel
path: deltachat-rpc-server-win64-wheel.d
- name: Download macOS binary for x86_64
uses: actions/download-artifact@v6
uses: actions/download-artifact@v7
with:
name: deltachat-rpc-server-x86_64-macos
path: deltachat-rpc-server-x86_64-macos.d
- name: Download macOS binary for aarch64
uses: actions/download-artifact@v6
uses: actions/download-artifact@v7
with:
name: deltachat-rpc-server-aarch64-macos
path: deltachat-rpc-server-aarch64-macos.d
- name: Download Android binary for arm64-v8a
uses: actions/download-artifact@v6
uses: actions/download-artifact@v7
with:
name: deltachat-rpc-server-arm64-v8a-android
path: deltachat-rpc-server-arm64-v8a-android.d
- name: Download Android wheel for arm64-v8a
uses: actions/download-artifact@v6
uses: actions/download-artifact@v7
with:
name: deltachat-rpc-server-arm64-v8a-android-wheel
path: deltachat-rpc-server-arm64-v8a-android-wheel.d
- name: Download Android binary for armeabi-v7a
uses: actions/download-artifact@v6
uses: actions/download-artifact@v7
with:
name: deltachat-rpc-server-armeabi-v7a-android
path: deltachat-rpc-server-armeabi-v7a-android.d
- name: Download Android wheel for armeabi-v7a
uses: actions/download-artifact@v6
uses: actions/download-artifact@v7
with:
name: deltachat-rpc-server-armeabi-v7a-android-wheel
path: deltachat-rpc-server-armeabi-v7a-android-wheel.d
@@ -382,7 +382,7 @@ jobs:
- name: Publish deltachat-rpc-server to PyPI
if: github.event_name == 'release'
uses: pypa/gh-action-pypi-publish@release/v1
uses: pypa/gh-action-pypi-publish@ed0c53931b1dc9bd32cbe73a98c7f6766f8a527e
publish_npm_package:
name: Build & Publish npm prebuilds and deltachat-rpc-server
@@ -406,67 +406,67 @@ jobs:
python-version: "3.11"
- name: Download Linux aarch64 binary
uses: actions/download-artifact@v6
uses: actions/download-artifact@v7
with:
name: deltachat-rpc-server-aarch64-linux
path: deltachat-rpc-server-aarch64-linux.d
- name: Download Linux armv7l binary
uses: actions/download-artifact@v6
uses: actions/download-artifact@v7
with:
name: deltachat-rpc-server-armv7l-linux
path: deltachat-rpc-server-armv7l-linux.d
- name: Download Linux armv6l binary
uses: actions/download-artifact@v6
uses: actions/download-artifact@v7
with:
name: deltachat-rpc-server-armv6l-linux
path: deltachat-rpc-server-armv6l-linux.d
- name: Download Linux i686 binary
uses: actions/download-artifact@v6
uses: actions/download-artifact@v7
with:
name: deltachat-rpc-server-i686-linux
path: deltachat-rpc-server-i686-linux.d
- name: Download Linux x86_64 binary
uses: actions/download-artifact@v6
uses: actions/download-artifact@v7
with:
name: deltachat-rpc-server-x86_64-linux
path: deltachat-rpc-server-x86_64-linux.d
- name: Download Win32 binary
uses: actions/download-artifact@v6
uses: actions/download-artifact@v7
with:
name: deltachat-rpc-server-win32
path: deltachat-rpc-server-win32.d
- name: Download Win64 binary
uses: actions/download-artifact@v6
uses: actions/download-artifact@v7
with:
name: deltachat-rpc-server-win64
path: deltachat-rpc-server-win64.d
- name: Download macOS binary for x86_64
uses: actions/download-artifact@v6
uses: actions/download-artifact@v7
with:
name: deltachat-rpc-server-x86_64-macos
path: deltachat-rpc-server-x86_64-macos.d
- name: Download macOS binary for aarch64
uses: actions/download-artifact@v6
uses: actions/download-artifact@v7
with:
name: deltachat-rpc-server-aarch64-macos
path: deltachat-rpc-server-aarch64-macos.d
- name: Download Android binary for arm64-v8a
uses: actions/download-artifact@v6
uses: actions/download-artifact@v7
with:
name: deltachat-rpc-server-arm64-v8a-android
path: deltachat-rpc-server-arm64-v8a-android.d
- name: Download Android binary for armeabi-v7a
uses: actions/download-artifact@v6
uses: actions/download-artifact@v7
with:
name: deltachat-rpc-server-armeabi-v7a-android
path: deltachat-rpc-server-armeabi-v7a-android.d
@@ -496,7 +496,7 @@ jobs:
ls -lah
- name: Upload to artifacts
uses: actions/upload-artifact@v5
uses: actions/upload-artifact@v6
with:
name: deltachat-rpc-server-npm-package
path: deltachat-rpc-server/npm-package/*.tgz
+1 -1
View File
@@ -25,7 +25,7 @@ jobs:
with:
node-version: 18.x
- name: Add Rust cache
uses: Swatinem/rust-cache@v2
uses: Swatinem/rust-cache@779680da715d629ac1d338a641029a2f4372abb5
- name: npm install
working-directory: deltachat-jsonrpc/typescript
run: npm install
+3 -3
View File
@@ -25,7 +25,7 @@ jobs:
with:
show-progress: false
persist-credentials: false
- uses: cachix/install-nix-action@0b0e072294b088b73964f1d72dfdac0951439dbd # v31
- uses: cachix/install-nix-action@4e002c8ec80594ecd40e759629461e26c8abed15 # v31
- run: nix fmt flake.nix -- --check
build:
@@ -84,7 +84,7 @@ jobs:
with:
show-progress: false
persist-credentials: false
- uses: cachix/install-nix-action@0b0e072294b088b73964f1d72dfdac0951439dbd # v31
- uses: cachix/install-nix-action@4e002c8ec80594ecd40e759629461e26c8abed15 # v31
- run: nix build .#${{ matrix.installable }}
build-macos:
@@ -105,5 +105,5 @@ jobs:
with:
show-progress: false
persist-credentials: false
- uses: cachix/install-nix-action@0b0e072294b088b73964f1d72dfdac0951439dbd # v31
- uses: cachix/install-nix-action@4e002c8ec80594ecd40e759629461e26c8abed15 # v31
- run: nix build .#${{ matrix.installable }}
@@ -23,7 +23,7 @@ jobs:
working-directory: deltachat-rpc-client
run: python3 -m build
- name: Store the distribution packages
uses: actions/upload-artifact@v5
uses: actions/upload-artifact@v6
with:
name: python-package-distributions
path: deltachat-rpc-client/dist/
@@ -42,9 +42,9 @@ jobs:
steps:
- name: Download all the dists
uses: actions/download-artifact@v6
uses: actions/download-artifact@v7
with:
name: python-package-distributions
path: dist/
- name: Publish deltachat-rpc-client to PyPI
uses: pypa/gh-action-pypi-publish@release/v1
uses: pypa/gh-action-pypi-publish@ed0c53931b1dc9bd32cbe73a98c7f6766f8a527e
+2 -2
View File
@@ -18,11 +18,11 @@ jobs:
with:
show-progress: false
persist-credentials: false
- uses: cachix/install-nix-action@0b0e072294b088b73964f1d72dfdac0951439dbd # v31
- uses: cachix/install-nix-action@4e002c8ec80594ecd40e759629461e26c8abed15 # v31
- name: Build
run: nix build .#deltachat-repl-win64
- name: Upload binary
uses: actions/upload-artifact@v5
uses: actions/upload-artifact@v6
with:
name: repl.exe
path: "result/bin/deltachat-repl.exe"
+2 -2
View File
@@ -36,7 +36,7 @@ jobs:
show-progress: false
persist-credentials: false
fetch-depth: 0 # Fetch history to calculate VCS version number.
- uses: cachix/install-nix-action@0b0e072294b088b73964f1d72dfdac0951439dbd # v31
- uses: cachix/install-nix-action@4e002c8ec80594ecd40e759629461e26c8abed15 # v31
- name: Build Python documentation
run: nix build .#python-docs
- name: Upload to py.delta.chat
@@ -55,7 +55,7 @@ jobs:
show-progress: false
persist-credentials: false
fetch-depth: 0 # Fetch history to calculate VCS version number.
- uses: cachix/install-nix-action@0b0e072294b088b73964f1d72dfdac0951439dbd # v31
- uses: cachix/install-nix-action@4e002c8ec80594ecd40e759629461e26c8abed15 # v31
- name: Build C documentation
run: nix build .#docs
- name: Upload to c.delta.chat
+1 -1
View File
@@ -19,7 +19,7 @@ jobs:
persist-credentials: false
- name: Install the latest version of uv
uses: astral-sh/setup-uv@1e862dfacbd1d6d858c55d9b792c756523627244
uses: astral-sh/setup-uv@681c641aba71e4a1c380be3ab5e12ad51f415867
- name: Run zizmor
run: uvx zizmor --format sarif . > results.sarif
+94
View File
@@ -1,5 +1,97 @@
# Changelog
## [2.37.0] - 2026-01-08
### API-Changes
- JSON-RPC API `get_all_ui_config_keys` to get all "ui.*" config keys ([#7579](https://github.com/chatmail/core/pull/7579)).
- Add `who_can_call_me` config option.
- cffi api to create account manager with existing events channel to see events emitted during startup. `dc_event_channel_new`, `dc_event_channel_unref`, `dc_event_channel_get_event_emitter` and `dc_accounts_new_with_event_channel` ([#7609](https://github.com/chatmail/core/pull/7609)).
### Features / Changes
- Config option to skip seen synchronization ([#7694](https://github.com/chatmail/core/pull/7694)).
- More text instead of sender in channel summary.
### Fixes
- Do not rely on Secure-Join header to detect {vc,vg}-request.
### Documentation
- Update instructions to UI where to display the address.
### Miscellaneous Tasks
- cargo: bump rsa from 0.9.9 to 0.9.10.
- Update lru 0.12.3 to 0.12.5 and add RUSTSEC-2026-0002 exception.
### Refactor
- ffi: Replace implicit drop in cffi with explicit `drop(Arc::from_raw(var))` ([#7664](https://github.com/chatmail/core/pull/7664)).
### Tests
- Regression test for vc-request encrypted by the server.
- Test that channel summary does not have sender name.
## [2.36.0] - 2026-01-03
### CI
- Pin GitHub Action references.
### API-Changes
- Add transports event to FFI.
### Features / Changes
- Add core version to `receive_imf` failure message.
- Connectivity view: quota for all transports ([#7630](https://github.com/chatmail/core/pull/7630)).
- Send sync messages over SMTP and do not move them to mvbox.
### Fixes
- When accepting group, add members with `Origin::IncomingTo` and sort them down in the contact list (7592).
- Update fallback welcome message.
- `inner_configure`: Check Config::OnlyFetchMvbox before MvboxMove for multi-transport ([#7637](https://github.com/chatmail/core/pull/7637)).
- Reset options not available for chatmail on chatmail profiles.
- Don't send webxdc notification for `notify: "*"` when chat is muted ([#7658](https://github.com/chatmail/core/pull/7658)).
### Documentation
- `delete_chat()`: don't lie that messages aren't deleted from server.
- Remove references to removed `sentbox_watch` config.
- Update documentation for `TransportsModified` event.
### Tests
- Contact list after accepting group with unknown contacts ([#7592](https://github.com/chatmail/core/pull/7592)).
- Port test_import_export_online_all to JSON-RPC ([#7411](https://github.com/chatmail/core/pull/7411)).
### Refactor
- Turn `DC_VERSION_STR` into `&str`.
- ffi: Remove one pointer indirection for `dc_accounts_t`.
### Miscellaneous Tasks
- deps: Bump actions/download-artifact from 6 to 7.
- deps: Bump actions/upload-artifact from 5 to 6.
- deps: Bump astral-sh/setup-uv from 7.1.4 to 7.1.6.
- deps: Bump cachix/install-nix-action from 31.8.4 to 31.9.0.
- cargo: Bump serde_json from 1.0.145 to 1.0.147.
- cargo: Bump uuid from 1.18.1 to 1.19.0.
- cargo: Bump toml from 0.9.8 to 0.9.10+spec-1.1.0.
- cargo: Bump tempfile from 3.23.0 to 3.24.0.
- cargo: Bump libc from 0.2.177 to 0.2.178.
- cargo: Bump tracing from 0.1.41 to 0.1.44.
- cargo: Bump hyper-util from 0.1.18 to 0.1.19.
- cargo: Bump log from 0.4.28 to 0.4.29.
- cargo: Bump rustls-pki-types from 1.13.0 to 1.13.2.
- cargo: Bump criterion from 0.7.0 to 0.8.1.
## [2.35.0] - 2025-12-16
### API-Changes
@@ -7449,3 +7541,5 @@ https://github.com/chatmail/core/pulls?q=is%3Apr+is%3Aclosed
[2.33.0]: https://github.com/chatmail/core/compare/v2.32.0..v2.33.0
[2.34.0]: https://github.com/chatmail/core/compare/v2.33.0..v2.34.0
[2.35.0]: https://github.com/chatmail/core/compare/v2.34.0..v2.35.0
[2.36.0]: https://github.com/chatmail/core/compare/v2.35.0..v2.36.0
[2.37.0]: https://github.com/chatmail/core/compare/v2.36.0..v2.37.0
Generated
+89 -83
View File
@@ -62,18 +62,6 @@ dependencies = [
"aes",
]
[[package]]
name = "ahash"
version = "0.8.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e89da841a80418a9b391ebaea17f5c112ffaaa96f621d2c285b5174da76b9011"
dependencies = [
"cfg-if",
"once_cell",
"version_check",
"zerocopy",
]
[[package]]
name = "aho-corasick"
version = "1.1.3"
@@ -98,6 +86,15 @@ dependencies = [
"alloc-no-stdlib",
]
[[package]]
name = "alloca"
version = "0.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e5a7d05ea6aea7e9e64d25b9156ba2fee3fdd659e34e41063cd2fc7cd020d7f4"
dependencies = [
"cc",
]
[[package]]
name = "allocator-api2"
version = "0.2.21"
@@ -1042,10 +1039,11 @@ dependencies = [
[[package]]
name = "criterion"
version = "0.7.0"
version = "0.8.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e1c047a62b0cc3e145fa84415a3191f628e980b194c2755aa12300a4e6cbd928"
checksum = "4d883447757bb0ee46f233e9dc22eb84d93a9508c9b868687b274fc431d886bf"
dependencies = [
"alloca",
"anes",
"cast",
"ciborium",
@@ -1054,6 +1052,7 @@ dependencies = [
"itertools",
"num-traits",
"oorandom",
"page_size",
"plotters",
"rayon",
"regex",
@@ -1066,9 +1065,9 @@ dependencies = [
[[package]]
name = "criterion-plot"
version = "0.6.0"
version = "0.8.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9b1bcc0dc7dfae599d84ad0b1a55f80cde8af3725da8313b528da95ef783e338"
checksum = "ed943f81ea2faa8dcecbbfa50164acf95d555afec96a27871663b300e387b2e4"
dependencies = [
"cast",
"itertools",
@@ -1304,7 +1303,7 @@ dependencies = [
[[package]]
name = "deltachat"
version = "2.35.0"
version = "2.37.0"
dependencies = [
"anyhow",
"astral-tokio-tar",
@@ -1414,7 +1413,7 @@ dependencies = [
[[package]]
name = "deltachat-jsonrpc"
version = "2.35.0"
version = "2.37.0"
dependencies = [
"anyhow",
"async-channel 2.5.0",
@@ -1435,7 +1434,7 @@ dependencies = [
[[package]]
name = "deltachat-repl"
version = "2.35.0"
version = "2.37.0"
dependencies = [
"anyhow",
"deltachat",
@@ -1451,7 +1450,7 @@ dependencies = [
[[package]]
name = "deltachat-rpc-server"
version = "2.35.0"
version = "2.37.0"
dependencies = [
"anyhow",
"deltachat",
@@ -1480,7 +1479,7 @@ dependencies = [
[[package]]
name = "deltachat_ffi"
version = "2.35.0"
version = "2.37.0"
dependencies = [
"anyhow",
"deltachat",
@@ -1997,7 +1996,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0ce92ff622d6dadf7349484f42c93271a0d49b7cc4d466a936405bacbe10aa78"
dependencies = [
"cfg-if",
"rustix 1.0.5",
"rustix 1.1.3",
"windows-sys 0.59.0",
]
@@ -2367,22 +2366,14 @@ dependencies = [
"crunchy",
]
[[package]]
name = "hashbrown"
version = "0.14.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e5274423e17b7c9fc20b6e7e208532f9b19825d82dfd615708b70edd83df41f1"
dependencies = [
"ahash",
"allocator-api2",
]
[[package]]
name = "hashbrown"
version = "0.15.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5971ac85611da7067dbfcabef3c70ebb5606018acd9e2a3903a0da507521e0d5"
dependencies = [
"allocator-api2",
"equivalent",
"foldhash",
]
@@ -2392,7 +2383,7 @@ version = "0.10.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7382cf6263419f2d8df38c55d7da83da5c18aef87fc7a7fc1fb1e344edfe14c1"
dependencies = [
"hashbrown 0.15.4",
"hashbrown",
]
[[package]]
@@ -2648,9 +2639,9 @@ dependencies = [
[[package]]
name = "hyper-util"
version = "0.1.18"
version = "0.1.19"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "52e9a2a24dc5c6821e71a7030e1e14b7b632acac55c40e9d2e082c621261bb56"
checksum = "727805d60e7938b76b826a6ef209eb70eaa1812794f9424d4a4e2d740662df5f"
dependencies = [
"bytes",
"futures-channel",
@@ -2909,7 +2900,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4b0f83760fb341a774ed326568e19f5a863af4a952def8c39f9ab92fd95b88e5"
dependencies = [
"equivalent",
"hashbrown 0.15.4",
"hashbrown",
]
[[package]]
@@ -3173,7 +3164,7 @@ dependencies = [
"iroh-metrics",
"iroh-quinn",
"iroh-quinn-proto",
"lru 0.12.3",
"lru 0.12.5",
"n0-future",
"num_enum",
"pin-project",
@@ -3267,9 +3258,9 @@ dependencies = [
[[package]]
name = "libc"
version = "0.2.177"
version = "0.2.178"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2874a2af47a2325c2001a6e6fad9b16a53b802102b528163885171cf92b15976"
checksum = "37c93d8daa9d8a012fd8ab92f088405fb202ea0b6ab73ee2482ae66af4f42091"
[[package]]
name = "libm"
@@ -3317,9 +3308,9 @@ checksum = "78b3ae25bc7c8c38cec158d1f2757ee79e9b3740fbc7ccf0e59e4b08d793fa89"
[[package]]
name = "linux-raw-sys"
version = "0.9.3"
version = "0.11.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fe7db12097d22ec582439daf8618b8fdd1a7bef6270e9af3b1ebcd30893cf413"
checksum = "df1d3c3b53da64cf5760482273a98e575c651a67eec7f77df96b5b642de8f039"
[[package]]
name = "litemap"
@@ -3344,9 +3335,9 @@ dependencies = [
[[package]]
name = "log"
version = "0.4.28"
version = "0.4.29"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "34080505efa8e45a4b816c349525ebe327ceaa8559756f0356cba97ef3bf7432"
checksum = "5e5032e24019045c762d3c0f28f5b6b8bbf38563a65908389bf7978758920897"
[[package]]
name = "loom"
@@ -3363,11 +3354,11 @@ dependencies = [
[[package]]
name = "lru"
version = "0.12.3"
version = "0.12.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d3262e75e648fce39813cb56ac41f3c3e3f65217ebf3844d818d1f9398cfb0dc"
checksum = "234cf4f4a04dc1f57e24b96cc0cd600cf2af460d4161ac5ecdd0af8e1f3b2a38"
dependencies = [
"hashbrown 0.14.5",
"hashbrown",
]
[[package]]
@@ -3768,11 +3759,10 @@ dependencies = [
[[package]]
name = "num-bigint-dig"
version = "0.8.4"
version = "0.8.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "dc84195820f291c7697304f3cbdadd1cb7199c0efc917ff5eafd71225c136151"
checksum = "e661dda6640fad38e827a6d4a310ff4763082116fe217f279885c97f511bb0b7"
dependencies = [
"byteorder",
"lazy_static",
"libm",
"num-integer",
@@ -4023,6 +4013,16 @@ dependencies = [
"sha2",
]
[[package]]
name = "page_size"
version = "0.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "30d5b2194ed13191c1999ae0704b7839fb18384fa22e49b57eeaa97d79ce40da"
dependencies = [
"libc",
"winapi",
]
[[package]]
name = "parking"
version = "2.2.1"
@@ -5050,9 +5050,9 @@ dependencies = [
[[package]]
name = "rsa"
version = "0.9.9"
version = "0.9.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "40a0376c50d0358279d9d643e4bf7b7be212f1f4ff1da9070a7b54d22ef75c88"
checksum = "b8573f03f5883dcaebdfcf4725caa1ecb9c15b2ef50c43a07b816e06799bb12d"
dependencies = [
"const-oid",
"digest",
@@ -5127,15 +5127,15 @@ dependencies = [
[[package]]
name = "rustix"
version = "1.0.5"
version = "1.1.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d97817398dd4bb2e6da002002db259209759911da105da92bec29ccb12cf58bf"
checksum = "146c9e247ccc180c1f61615433868c99f3de3ae256a30a43b49f67c2d9171f34"
dependencies = [
"bitflags 2.9.1",
"errno",
"libc",
"linux-raw-sys 0.9.3",
"windows-sys 0.59.0",
"linux-raw-sys 0.11.0",
"windows-sys 0.61.1",
]
[[package]]
@@ -5164,9 +5164,9 @@ dependencies = [
[[package]]
name = "rustls-pki-types"
version = "1.13.0"
version = "1.13.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "94182ad936a0c91c324cd46c6511b9510ed16af436d7b5bab34beab0afd55f7a"
checksum = "21e6f2ab2928ca4291b86736a8bd920a277a399bba1589409d72154ff87c1282"
dependencies = [
"web-time",
"zeroize",
@@ -5422,22 +5422,22 @@ dependencies = [
[[package]]
name = "serde_json"
version = "1.0.145"
version = "1.0.148"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "402a6f66d8c709116cf22f558eab210f5a50187f702eb4d7e5ef38d9a7f1c79c"
checksum = "3084b546a1dd6289475996f182a22aba973866ea8e8b02c51d9f46b1336a22da"
dependencies = [
"itoa",
"memchr",
"ryu",
"serde",
"serde_core",
"zmij",
]
[[package]]
name = "serde_spanned"
version = "1.0.3"
version = "1.0.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e24345aa0fe688594e73770a5f6d1b216508b4f93484c0026d521acd30134392"
checksum = "f8bbf91e5a4d6315eee45e704372590b30e260ee83af6639d64557f51b067776"
dependencies = [
"serde_core",
]
@@ -5947,14 +5947,14 @@ checksum = "55937e1799185b12863d447f42597ed69d9928686b8d88a1df17376a097d8369"
[[package]]
name = "tempfile"
version = "3.23.0"
version = "3.24.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2d31c77bdf42a745371d260a26ca7163f1e0924b64afa0b688e61b5a9fa02f16"
checksum = "655da9c7eb6305c55742045d5a8d2037996d61d8de95806335c7c86ce0f82e9c"
dependencies = [
"fastrand",
"getrandom 0.3.3",
"once_cell",
"rustix 1.0.5",
"rustix 1.1.3",
"windows-sys 0.61.1",
]
@@ -6217,14 +6217,14 @@ dependencies = [
[[package]]
name = "toml"
version = "0.9.8"
version = "0.9.10+spec-1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f0dc8b1fb61449e27716ec0e1bdf0f6b8f3e8f6b05391e8497b8b6d7804ea6d8"
checksum = "0825052159284a1a8b4d6c0c86cbc801f2da5afd2b225fa548c72f2e74002f48"
dependencies = [
"indexmap",
"serde_core",
"serde_spanned",
"toml_datetime 0.7.3",
"toml_datetime 0.7.5+spec-1.1.0",
"toml_parser",
"toml_writer",
"winnow 0.7.13",
@@ -6238,9 +6238,9 @@ checksum = "22cddaf88f4fbc13c51aebbf5f8eceb5c7c5a9da2ac40a13519eb5b0a0e8f11c"
[[package]]
name = "toml_datetime"
version = "0.7.3"
version = "0.7.5+spec-1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f2cdb639ebbc97961c51720f858597f7f24c4fc295327923af55b74c3c724533"
checksum = "92e1cfed4a3038bc5a127e35a2d360f145e1f4b971b551a2ba5fd7aedf7e1347"
dependencies = [
"serde_core",
]
@@ -6270,9 +6270,9 @@ dependencies = [
[[package]]
name = "toml_parser"
version = "1.0.4"
version = "1.0.6+spec-1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c0cbe268d35bdb4bb5a56a2de88d0ad0eb70af5384a99d648cd4b3d04039800e"
checksum = "a3198b4b0a8e11f09dd03e133c0280504d0801269e9afa46362ffde1cbeebf44"
dependencies = [
"winnow 0.7.13",
]
@@ -6285,9 +6285,9 @@ checksum = "5d99f8c9a7727884afe522e9bd5edbfc91a3312b36a77b5fb8926e4c31a41801"
[[package]]
name = "toml_writer"
version = "1.0.4"
version = "1.0.6+spec-1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "df8b2b54733674ad286d16267dcfc7a71ed5c776e4ac7aa3c3e2561f7c637bf2"
checksum = "ab16f14aed21ee8bfd8ec22513f7287cd4a91aa92e44edfe2c17ddd004e92607"
[[package]]
name = "tower"
@@ -6318,9 +6318,9 @@ checksum = "8df9b6e13f2d32c91b9bd719c00d1958837bc7dec474d94952798cc8e69eeec3"
[[package]]
name = "tracing"
version = "0.1.41"
version = "0.1.44"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "784e0ac535deb450455cbfa28a6f0df145ea1bb7ae51b821cf5e7927fdcfbdd0"
checksum = "63e71662fa4b2a2c3a26f570f037eb95bb1f85397f3cd8076caed2f026a6d100"
dependencies = [
"log",
"pin-project-lite",
@@ -6330,9 +6330,9 @@ dependencies = [
[[package]]
name = "tracing-attributes"
version = "0.1.28"
version = "0.1.31"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "395ae124c09f9e6918a2310af6038fba074bcf474ac352496d5910dd59a2226d"
checksum = "7490cfa5ec963746568740651ac6781f701c9c5ea257c58e057f3ba8cf69e8da"
dependencies = [
"proc-macro2",
"quote",
@@ -6341,9 +6341,9 @@ dependencies = [
[[package]]
name = "tracing-core"
version = "0.1.33"
version = "0.1.36"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e672c95779cf947c5311f83787af4fa8fffd12fb27e4993211a84bdfd9610f9c"
checksum = "db97caf9d906fbde555dd62fa95ddba9eecfd14cb388e4f491a66d74cd5fb79a"
dependencies = [
"once_cell",
"valuable",
@@ -6542,13 +6542,13 @@ checksum = "06abde3611657adf66d383f00b093d7faecc7fa57071cce2578660c9f1010821"
[[package]]
name = "uuid"
version = "1.18.1"
version = "1.19.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2f87b8aa10b915a06587d0dec516c282ff295b475d94abf425d62b57710070a2"
checksum = "e2e054861b4bd027cd373e18e8d8d8e6548085000e41290d95ce0c373a654b4a"
dependencies = [
"getrandom 0.3.3",
"js-sys",
"serde",
"serde_core",
"wasm-bindgen",
]
@@ -7465,6 +7465,12 @@ version = "0.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "868b928d7949e09af2f6086dfc1e01936064cc7a819253bce650d4e2a2d63ba8"
[[package]]
name = "zmij"
version = "1.0.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "aac060176f7020d62c3bcc1cdbcec619d54f48b07ad1963a3f80ce7a0c17755f"
[[package]]
name = "zune-core"
version = "0.5.0"
+3 -3
View File
@@ -1,6 +1,6 @@
[package]
name = "deltachat"
version = "2.35.0"
version = "2.37.0"
edition = "2024"
license = "MPL-2.0"
rust-version = "1.88"
@@ -116,7 +116,7 @@ webpki-roots = "0.26.8"
[dev-dependencies]
anyhow = { workspace = true, features = ["backtrace"] } # Enable `backtrace` feature in tests.
criterion = { version = "0.7.0", features = ["async_tokio"] }
criterion = { version = "0.8.1", features = ["async_tokio"] }
futures-lite = { workspace = true }
log = { workspace = true }
nu-ansi-term = { workspace = true }
@@ -199,7 +199,7 @@ rusqlite = "0.37"
sanitize-filename = "0.6"
serde = "1.0"
serde_json = "1"
tempfile = "3.23.0"
tempfile = "3.24.0"
thiserror = "2"
tokio = "1"
tokio-util = "0.7.17"
+1 -1
View File
@@ -21,7 +21,7 @@ text TEXT DEFAULT '' NOT NULL -- message text
```
Do not use macros like [`concat!`](https://doc.rust-lang.org/std/macro.concat.html)
or [`indoc!](https://docs.rs/indoc).
or [`indoc!`](https://docs.rs/indoc).
Do not escape newlines like this:
```
sql.execute(
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "deltachat_ffi"
version = "2.35.0"
version = "2.37.0"
description = "Deltachat FFI"
edition = "2018"
readme = "README.md"
+170 -94
View File
@@ -22,6 +22,7 @@ typedef struct _dc_lot dc_lot_t;
typedef struct _dc_provider dc_provider_t;
typedef struct _dc_event dc_event_t;
typedef struct _dc_event_emitter dc_event_emitter_t;
typedef struct _dc_event_channel dc_event_channel_t;
typedef struct _dc_jsonrpc_instance dc_jsonrpc_instance_t;
typedef struct _dc_backup_provider dc_backup_provider_t;
@@ -429,16 +430,13 @@ char* dc_get_blobdir (const dc_context_t* context);
* 1=send a copy of outgoing messages to self (default).
* Sending messages to self is needed for a proper multi-account setup,
* however, on the other hand, may lead to unwanted notifications in non-delta clients.
* - `sentbox_watch`= 1=watch `Sent`-folder for changes,
* 0=do not watch the `Sent`-folder (default).
* - `mvbox_move` = 1=detect chat messages,
* move them to the `DeltaChat` folder,
* and watch the `DeltaChat` folder for updates (default),
* 0=do not move chat-messages
* - `only_fetch_mvbox` = 1=Do not fetch messages from folders other than the
* `DeltaChat` folder. Messages will still be fetched from the
* spam folder and `sendbox_watch` will also still be respected
* if enabled.
* spam folder.
* 0=watch all folders normally (default)
* - `show_emails` = DC_SHOW_EMAILS_OFF (0)=
* show direct replies to chats only,
@@ -488,12 +486,15 @@ char* dc_get_blobdir (const dc_context_t* context);
* 0=use IMAP IDLE if the server supports it.
* This is a developer option used for testing polling used as an IDLE fallback.
* - `download_limit` = Messages up to this number of bytes are downloaded automatically.
* For larger messages, only the header is downloaded and a placeholder is shown.
* These messages can be downloaded fully using dc_download_full_msg() later.
* The limit is compared against raw message sizes, including headers.
* The actually used limit may be corrected
* to not mess up with non-delivery-reports or read-receipts.
* 0=no limit (default).
* For messages with large attachments, two messages are sent:
* a Pre-Message containing metadata and text and a Post-Message additionally
* containing the attachment. NB: Some "extra" metadata like avatars and gossiped
* encryption keys is stripped from post-messages to save traffic.
* Pre-Messages are shown as placeholder messages. They can be downloaded fully
* using dc_download_full_msg() later. Post-Messages are automatically
* downloaded if they are smaller than the download_limit. Other messages are
* always auto-downloaded.
* 0 = no limit (default).
* Changes affect future messages only.
* - `protect_autocrypt` = Enable Header Protection for Autocrypt header.
* This is an experimental option not compatible to other MUAs
@@ -519,6 +520,10 @@ char* dc_get_blobdir (const dc_context_t* context);
* - `webxdc_realtime_enabled` = Whether the realtime APIs should be enabled.
* 0 = WebXDC realtime API is disabled and behaves as noop.
* 1 = WebXDC realtime API is enabled (default).
* - `who_can_call_me` = Who can cause call notifications.
* 0 = Everybody (except explicitly blocked contacts),
* 1 = Contacts (default, does not include contact requests),
* 2 = Nobody (calls never result in a notification).
*
* If you want to retrieve a value, use dc_get_config().
*
@@ -575,11 +580,10 @@ int dc_set_stock_translation(dc_context_t* context, uint32_t stock_i
/**
* Set configuration values from a QR code.
* Before this function is called, dc_check_qr() should confirm the type of the
* QR code is DC_QR_ACCOUNT or DC_QR_LOGIN.
* Before this function is called, dc_check_qr() should be used to get the QR code type.
*
* Internally, the function will call dc_set_config() with the appropriate keys,
* e.g. `addr` and `mail_pw` for DC_QR_ACCOUNT and DC_QR_LOGIN.
* DC_QR_ACCOUNT and DC_QR_LOGIN QR codes configure the context, but I/O mustn't be started for such
* QR codes.
*
* @memberof dc_context_t
* @param context The context object.
@@ -890,7 +894,7 @@ int dc_preconfigure_keypair (dc_context_t* context, const cha
* the pseudo-chat DC_CHAT_ID_ARCHIVED_LINK is added if there are _any_ archived
* chats
* - the flag DC_GCL_FOR_FORWARDING sorts "Saved messages" to the top of the chatlist
* and hides the "Device chat" and contact requests.
* and hides the "Device chat", contact requests and incoming broadcasts.
* typically used on forwarding, may be combined with DC_GCL_NO_SPECIALS
* to also hide the archive link.
* - if the flag DC_GCL_NO_SPECIALS is set, archive link is not added
@@ -1559,7 +1563,7 @@ dc_array_t* dc_wait_next_msgs (dc_context_t* context);
* Mark all messages in a chat as _noticed_.
* _Noticed_ messages are no longer _fresh_ and do not count as being unseen
* but are still waiting for being marked as "seen" using dc_markseen_msgs()
* (IMAP/MDNs is not done for noticed messages).
* (read receipts aren't sent for noticed messages).
*
* Calling this function usually results in the event #DC_EVENT_MSGS_NOTICED.
* See also dc_markseen_msgs().
@@ -1611,10 +1615,10 @@ void dc_set_chat_visibility (dc_context_t* context, uint32_t ch
*
* Messages are deleted from the device and the chat database entry is deleted.
* After that, the event #DC_EVENT_MSGS_CHANGED is posted.
* Messages are deleted from the server in background.
*
* Things that are _not_ done implicitly:
*
* - Messages are **not deleted from the server**.
* - The chat or the contact is **not blocked**, so new messages from the user/the group may appear
* and the user may create the chat again.
* - **Groups are not left** - this would
@@ -3092,7 +3096,7 @@ int dc_receive_backup (dc_context_t* context, const char* qr);
/**
* Create a new account manager.
* The account manager takes an directory
* The account manager takes a directory
* where all context-databases are placed in.
* To add a context to the account manager,
* use dc_accounts_add_account() or dc_accounts_migrate_account().
@@ -3114,6 +3118,35 @@ int dc_receive_backup (dc_context_t* context, const char* qr);
*/
dc_accounts_t* dc_accounts_new (const char* dir, int writable);
/**
* Create a new account manager with an existing events channel,
* which allows you to see events emitted during startup.
*
* The account manager takes a directory
* where all context-databases are placed in.
* To add a context to the account manager,
* use dc_accounts_add_account() or dc_accounts_migrate_account().
* All account information are persisted.
* To remove a context from the account manager,
* use dc_accounts_remove_account().
*
* @memberof dc_accounts_t
* @param dir The directory to create the context-databases in.
* If the directory does not exist,
* dc_accounts_new_with_event_channel() will try to create it.
* @param writable Whether the returned account manager is writable, i.e. calling these functions on
* it is possible: dc_accounts_add_account(), dc_accounts_add_closed_account(),
* dc_accounts_migrate_account(), dc_accounts_remove_account(), dc_accounts_select_account().
* @param dc_event_channel_t Events Channel to be used for this accounts manager,
* create one with dc_event_channel_new().
* This channel is consumed by this method and can not be used again afterwards,
* so be sure to call `dc_event_channel_get_event_emitter` before.
* @return An account manager object.
* The object must be passed to the other account manager functions
* and must be freed using dc_accounts_unref() after usage.
* On errors, NULL is returned.
*/
dc_accounts_t* dc_accounts_new_with_event_channel(const char* dir, int writable, dc_event_channel_t* events_channel);
/**
* Free an account manager object.
@@ -3354,8 +3387,12 @@ void dc_accounts_set_push_device_token (dc_accounts_t* accounts, const
* Must be freed using dc_event_emitter_unref() after usage.
*
* Note: Use only one event emitter per account manager.
* Having more than one event emitter running at the same time on the same account manager
* will result in events randomly delivered to the one or to the other.
* The result of having multiple event emitters is unspecified.
* Currently events are broadcasted to all existing event emitters,
* but previous versions delivered events to only one event emitter
* and this behavior may change again in the future.
* Events emitted before creation of event emitter
* are not available to event emitter.
*/
dc_event_emitter_t* dc_accounts_get_event_emitter (dc_accounts_t* accounts);
@@ -3732,30 +3769,7 @@ uint32_t dc_chat_get_id (const dc_chat_t* chat);
/**
* Get chat type as one of the @ref DC_CHAT_TYPE constants:
*
* - @ref DC_CHAT_TYPE_SINGLE - a normal chat is a chat with a single contact,
* chats_contacts contains one record for the user. DC_CONTACT_ID_SELF
* (see dc_contact_t::id) is added _only_ for a self talk.
* These chats are created by dc_create_chat_by_contact_id().
*
* - @ref DC_CHAT_TYPE_GROUP - a group chat, chats_contacts contain all group
* members, incl. DC_CONTACT_ID_SELF.
* Groups are created by dc_create_group_chat().
*
* - @ref DC_CHAT_TYPE_MAILINGLIST - a mailing list, this is similar to groups,
* however, the member list cannot be retrieved completely
* and cannot be changed using this api.
* Mailing lists are created as needed by incoming messages
* and usually require some special server;
* they cannot be created by a function call as the other chat types.
* Moreover, for now, mailing lists are read-only.
*
* - @ref DC_CHAT_TYPE_BROADCAST - a broadcast list,
* the recipients will get messages in a one-to-one chats and
* the sender will get answers in a one-to-one as well.
* chats_contacts contain all recipients but DC_CONTACT_ID_SELF.
* Broadcasts are created by dc_create_broadcast_list().
* Get chat type as one of the @ref DC_CHAT_TYPE constants.
*
* @memberof dc_chat_t
* @param chat The chat object.
@@ -4311,6 +4325,7 @@ char* dc_msg_get_webxdc_info (const dc_msg_t* msg);
/**
* Get the size of the file. Returns the size of the file associated with a
* message, if applicable.
* If message is a pre-message, then this returns the size of the file to be downloaded.
*
* Typically, this is used to show the size of document files, e.g. a PDF.
*
@@ -5328,8 +5343,8 @@ int dc_contact_is_key_contact (dc_contact_t* contact);
*
* - If dc_contact_get_verifier_id() != 0,
* display text "Introduced by ..."
* with the name and address of the contact
* formatted by dc_contact_get_name_n_addr().
* with the name of the contact
* formatted by dc_contact_get_name().
* Prefix the text by a green checkmark.
*
* - If dc_contact_get_verifier_id() == 0 and dc_contact_is_verified() != 0,
@@ -5760,17 +5775,32 @@ int64_t dc_lot_get_timestamp (const dc_lot_t* lot);
#define DC_CHAT_TYPE_UNDEFINED 0
/**
* A one-to-one chat with a single contact. See dc_chat_get_type() for details.
* A one-to-one chat with a single contact.
*
* dc_get_chat_contacts() contains one record for the user.
* DC_CONTACT_ID_SELF is added _only_ for a self talk.
* These chats are created by dc_create_chat_by_contact_id().
*/
#define DC_CHAT_TYPE_SINGLE 100
/**
* A group chat. See dc_chat_get_type() for details.
* A group chat.
*
* dc_get_chat_contacts() contain all group members,
* including DC_CONTACT_ID_SELF.
* Groups are created by dc_create_group_chat().
*/
#define DC_CHAT_TYPE_GROUP 120
/**
* A mailing list. See dc_chat_get_type() for details.
* A mailing list.
*
* This is similar to groups,
* however, the member list cannot be retrieved completely
* and cannot be changed using an API from this library.
* Mailing lists are created as needed by incoming messages
* and usually require some special server;
* they cannot be created by a function call as the other chat types.
*/
#define DC_CHAT_TYPE_MAILINGLIST 140
@@ -5999,6 +6029,62 @@ char* dc_jsonrpc_next_response(dc_jsonrpc_instance_t* jsonrpc_instance);
*/
char* dc_jsonrpc_blocking_call(dc_jsonrpc_instance_t* jsonrpc_instance, const char *input);
/**
* @class dc_event_channel_t
*
* Opaque object that is used to create an event emitter which can be used log events during startup of an accounts manger.
* Only used for dc_accounts_new_with_event_channel().
* To use it:
* 1. create an events channel with `dc_event_channel_new()`.
* 2. get an event emitter for it with `dc_event_channel_get_event_emitter()`.
* 3. use it to create your account manager with `dc_accounts_new_with_event_channel()`, which consumes the channel.
* 4. free the empty channel wrapper object with `dc_event_channel_unref()`.
*/
/**
* Create a new event channel.
*
* @memberof dc_event_channel_t
* @return An event channel wrapper object (dc_event_channel_t).
*/
dc_event_channel_t* dc_event_channel_new();
/**
* Release/free the events channel structure.
* This function releases the memory of the `dc_event_channel_t` structure.
*
* you can call it after calling dc_accounts_new_with_event_channel,
* which took the events channel out of it already, so this just frees the underlying option.
*
* @memberof dc_event_channel_t
*/
void dc_event_channel_unref(dc_event_channel_t* event_channel);
/**
* Create the event emitter that is used to receive events.
*
* The library will emit various @ref DC_EVENT events, such as "new message", "message read" etc.
* To get these events, you have to create an event emitter using this function
* and call dc_get_next_event() on the emitter.
*
* This is similar to dc_get_event_emitter(), which, however,
* must not be called for accounts handled by the account manager.
*
* @memberof dc_event_channel_t
* @param The event channel.
* @return Returns the event emitter, NULL on errors.
* Must be freed using dc_event_emitter_unref() after usage.
*
* Note: Use only one event emitter per account manager / event channel.
* The result of having multiple event emitters is unspecified.
* Currently events are broadcasted to all existing event emitters,
* but previous versions delivered events to only one event emitter
* and this behavior may change again in the future.
* Events emitted before creation of event emitter
* are not available to event emitter.
*/
dc_event_emitter_t* dc_event_channel_get_event_emitter(dc_event_channel_t* event_channel);
/**
* @class dc_event_emitter_t
*
@@ -6702,6 +6788,16 @@ void dc_event_unref(dc_event_t* event);
*/
#define DC_EVENT_CALL_ENDED 2580
/**
* Transport relay added/deleted or default has changed.
* UI should update the list.
*
* The event is emitted when the transports are modified on another device
* using the JSON-RPC calls `add_or_update_transport`, `add_transport_from_qr`, `delete_transport`
* or `set_config(configured_addr)`.
*/
#define DC_EVENT_TRANSPORTS_MODIFIED 2600
/**
* @}
@@ -7263,22 +7359,9 @@ void dc_event_unref(dc_event_t* event);
/// `%1$s` will be replaced by the percentage used
#define DC_STR_QUOTA_EXCEEDING_MSG_BODY 98
/// "%1$s message"
///
/// Used as the message body when a message
/// was not yet downloaded completely
/// (dc_msg_get_download_state() is e.g. @ref DC_DOWNLOAD_AVAILABLE).
///
/// `%1$s` will be replaced by human-readable size (e.g. "1.2 MiB").
/// @deprecated Deprecated 2025-11-12, this string is no longer needed.
#define DC_STR_PARTIAL_DOWNLOAD_MSG_BODY 99
/// "Download maximum available until %1$s"
///
/// Appended after some separator to @ref DC_STR_PARTIAL_DOWNLOAD_MSG_BODY.
///
/// `%1$s` will be replaced by human-readable date and time.
#define DC_STR_DOWNLOAD_AVAILABILITY 100
/// "Multi Device Synchronization"
///
/// Used in subjects of outgoing sync messages.
@@ -7304,12 +7387,6 @@ void dc_event_unref(dc_event_t* event);
/// Used as a headline in the connectivity view.
#define DC_STR_OUTGOING_MESSAGES 104
/// "Storage on %1$s"
///
/// Used as a headline in the connectivity view.
///
/// `%1$s` will be replaced by the domain of the configured e-mail address.
#define DC_STR_STORAGE_ON_DOMAIN 105
/// @deprecated Deprecated 2022-04-16, this string is no longer needed.
#define DC_STR_ONE_MOMENT 106
@@ -7372,8 +7449,7 @@ void dc_event_unref(dc_event_t* event);
/// May be followed by the info-messages
/// #DC_STR_SECURE_JOIN_REPLIES, #DC_STR_CONTACT_VERIFIED and #DC_STR_MSGADDMEMBER.
///
/// `%1$s` will be replaced by name and address of the inviter,
/// `%2$s` will be replaced by the name of the inviter.
/// `%1$s` and `%2$s` will be replaced by name of the inviter.
#define DC_STR_SECURE_JOIN_STARTED 117
/// "%1$s replied, waiting for being added to the group…"
@@ -7390,7 +7466,7 @@ void dc_event_unref(dc_event_t* event);
///
/// Subtitle for verification qrcode svg image generated by the core.
///
/// `%1$s` will be replaced by name and address of the inviter.
/// `%1$s` will be replaced by name of the inviter.
#define DC_STR_SETUP_CONTACT_QR_DESC 119
/// "Scan to join %1$s"
@@ -7421,7 +7497,7 @@ void dc_event_unref(dc_event_t* event);
///
/// `%1$s` will be replaced by the old group name.
/// `%2$s` will be replaced by the new group name.
/// `%3$s` will be replaced by name and address of the contact who did the action.
/// `%3$s` will be replaced by name of the contact who did the action.
#define DC_STR_GROUP_NAME_CHANGED_BY_OTHER 125
/// "You changed the group image."
@@ -7429,7 +7505,7 @@ void dc_event_unref(dc_event_t* event);
/// "Group image changed by %1$s."
///
/// `%1$s` will be replaced by name and address of the contact who did the action.
/// `%1$s` will be replaced by name of the contact who did the action.
#define DC_STR_GROUP_IMAGE_CHANGED_BY_OTHER 127
/// "You added member %1$s."
@@ -7441,23 +7517,23 @@ void dc_event_unref(dc_event_t* event);
/// "Member %1$s added by %2$s."
///
/// `%1$s` will be replaced by name and address of the contact added to the group.
/// `%2$s` will be replaced by name and address of the contact who did the action.
/// `%1$s` will be replaced by name of the contact added to the group.
/// `%2$s` will be replaced by name of the contact who did the action.
///
/// Used in status messages.
#define DC_STR_ADD_MEMBER_BY_OTHER 129
/// "You removed member %1$s."
///
/// `%1$s` will be replaced by name and address of the contact removed from the group.
/// `%1$s` will be replaced by name of the contact removed from the group.
///
/// Used in status messages.
#define DC_STR_REMOVE_MEMBER_BY_YOU 130
/// "Member %1$s removed by %2$s."
///
/// `%1$s` will be replaced by name and address of the contact removed from the group.
/// `%2$s` will be replaced by name and address of the contact who did the action.
/// `%1$s` will be replaced by name of the contact removed from the group.
/// `%2$s` will be replaced by name of the contact who did the action.
///
/// Used in status messages.
#define DC_STR_REMOVE_MEMBER_BY_OTHER 131
@@ -7469,7 +7545,7 @@ void dc_event_unref(dc_event_t* event);
/// "Group left by %1$s."
///
/// `%1$s` will be replaced by name and address of the contact.
/// `%1$s` will be replaced by name of the contact.
///
/// Used in status messages.
#define DC_STR_GROUP_LEFT_BY_OTHER 133
@@ -7481,7 +7557,7 @@ void dc_event_unref(dc_event_t* event);
/// "Group image deleted by %1$s."
///
/// `%1$s` will be replaced by name and address of the contact.
/// `%1$s` will be replaced by name of the contact.
///
/// Used in status messages.
#define DC_STR_GROUP_IMAGE_DELETED_BY_OTHER 135
@@ -7493,7 +7569,7 @@ void dc_event_unref(dc_event_t* event);
/// "Location streaming enabled by %1$s."
///
/// `%1$s` will be replaced by name and address of the contact.
/// `%1$s` will be replaced by name of the contact.
///
/// Used in status messages.
#define DC_STR_LOCATION_ENABLED_BY_OTHER 137
@@ -7505,7 +7581,7 @@ void dc_event_unref(dc_event_t* event);
/// "Message deletion timer is disabled by %1$s."
///
/// `%1$s` will be replaced by name and address of the contact.
/// `%1$s` will be replaced by name of the contact.
///
/// Used in status messages.
#define DC_STR_EPHEMERAL_TIMER_DISABLED_BY_OTHER 139
@@ -7520,7 +7596,7 @@ void dc_event_unref(dc_event_t* event);
/// "Message deletion timer is set to %1$s s by %2$s."
///
/// `%1$s` will be replaced by the number of seconds (always >1) the timer is set to.
/// `%2$s` will be replaced by name and address of the contact.
/// `%2$s` will be replaced by name of the contact.
///
/// Used in status messages.
#define DC_STR_EPHEMERAL_TIMER_SECONDS_BY_OTHER 141
@@ -7532,7 +7608,7 @@ void dc_event_unref(dc_event_t* event);
/// "Message deletion timer is set to 1 minute by %1$s."
///
/// `%1$s` will be replaced by name and address of the contact.
/// `%1$s` will be replaced by name of the contact.
/// @deprecated 2025-11-14, this string is no longer needed
#define DC_STR_EPHEMERAL_TIMER_1_MINUTE_BY_OTHER 143
@@ -7543,7 +7619,7 @@ void dc_event_unref(dc_event_t* event);
/// "Message deletion timer is set to 1 hour by %1$s."
///
/// `%1$s` will be replaced by name and address of the contact.
/// `%1$s` will be replaced by name of the contact.
///
/// Used in status messages.
#define DC_STR_EPHEMERAL_TIMER_1_HOUR_BY_OTHER 145
@@ -7555,7 +7631,7 @@ void dc_event_unref(dc_event_t* event);
/// "Message deletion timer is set to 1 day by %1$s."
///
/// `%1$s` will be replaced by name and address of the contact.
/// `%1$s` will be replaced by name of the contact.
///
/// Used in status messages.
#define DC_STR_EPHEMERAL_TIMER_1_DAY_BY_OTHER 147
@@ -7567,7 +7643,7 @@ void dc_event_unref(dc_event_t* event);
/// "Message deletion timer is set to 1 week by %1$s."
///
/// `%1$s` will be replaced by name and address of the contact.
/// `%1$s` will be replaced by name of the contact.
///
/// Used in status messages.
#define DC_STR_EPHEMERAL_TIMER_1_WEEK_BY_OTHER 149
@@ -7584,7 +7660,7 @@ void dc_event_unref(dc_event_t* event);
/// Used in status messages.
///
/// `%1$s` will be replaced by the number of minutes (always >1) the timer is set to.
/// `%2$s` will be replaced by name and address of the contact.
/// `%2$s` will be replaced by name of the contact.
#define DC_STR_EPHEMERAL_TIMER_MINUTES_BY_OTHER 151
/// "You set message deletion timer to %1$s hours."
@@ -7599,7 +7675,7 @@ void dc_event_unref(dc_event_t* event);
/// Used in status messages.
///
/// `%1$s` will be replaced by the number of hours (always >1) the timer is set to.
/// `%2$s` will be replaced by name and address of the contact.
/// `%2$s` will be replaced by name of the contact.
#define DC_STR_EPHEMERAL_TIMER_HOURS_BY_OTHER 153
/// "You set message deletion timer to %1$s days."
@@ -7614,7 +7690,7 @@ void dc_event_unref(dc_event_t* event);
/// Used in status messages.
///
/// `%1$s` will be replaced by the number of days (always >1) the timer is set to.
/// `%2$s` will be replaced by name and address of the contact.
/// `%2$s` will be replaced by name of the contact.
#define DC_STR_EPHEMERAL_TIMER_DAYS_BY_OTHER 155
/// "You set message deletion timer to %1$s weeks."
@@ -7629,7 +7705,7 @@ void dc_event_unref(dc_event_t* event);
/// Used in status messages.
///
/// `%1$s` will be replaced by the number of weeks (always >1) the timer is set to.
/// `%2$s` will be replaced by name and address of the contact.
/// `%2$s` will be replaced by name of the contact.
#define DC_STR_EPHEMERAL_TIMER_WEEKS_BY_OTHER 157
/// "You set message deletion timer to 1 year."
@@ -7639,14 +7715,14 @@ void dc_event_unref(dc_event_t* event);
/// "Message deletion timer is set to 1 year by %1$s."
///
/// `%1$s` will be replaced by name and address of the contact.
/// `%1$s` will be replaced by name of the contact.
///
/// Used in status messages.
#define DC_STR_EPHEMERAL_TIMER_1_YEAR_BY_OTHER 159
/// "Scan to set up second device for %1$s"
///
/// `%1$s` will be replaced by name and address of the account.
/// `%1$s` will be replaced by name of the account.
#define DC_STR_BACKUP_TRANSFER_QR 162
/// "Account transferred to your second device."
+122 -51
View File
@@ -15,10 +15,9 @@ use std::collections::BTreeMap;
use std::convert::TryFrom;
use std::fmt::Write;
use std::future::Future;
use std::ops::Deref;
use std::ptr;
use std::str::FromStr;
use std::sync::{Arc, LazyLock};
use std::sync::{Arc, LazyLock, Mutex};
use std::time::{Duration, SystemTime};
use anyhow::Context as _;
@@ -4739,33 +4738,13 @@ pub unsafe extern "C" fn dc_provider_unref(provider: *mut dc_provider_t) {
/// Reader-writer lock wrapper for accounts manager to guarantee thread safety when using
/// `dc_accounts_t` in multiple threads at once.
pub struct AccountsWrapper {
inner: Arc<RwLock<Accounts>>,
}
impl Deref for AccountsWrapper {
type Target = Arc<RwLock<Accounts>>;
fn deref(&self) -> &Self::Target {
&self.inner
}
}
impl AccountsWrapper {
fn new(accounts: Accounts) -> Self {
let inner = Arc::new(RwLock::new(accounts));
Self { inner }
}
}
/// Struct representing a list of deltachat accounts.
pub type dc_accounts_t = AccountsWrapper;
pub type dc_accounts_t = RwLock<Accounts>;
#[no_mangle]
pub unsafe extern "C" fn dc_accounts_new(
dir: *const libc::c_char,
writable: libc::c_int,
) -> *mut dc_accounts_t {
) -> *const dc_accounts_t {
setup_panic!();
if dir.is_null() {
@@ -4776,7 +4755,99 @@ pub unsafe extern "C" fn dc_accounts_new(
let accs = block_on(Accounts::new(as_path(dir).into(), writable != 0));
match accs {
Ok(accs) => Box::into_raw(Box::new(AccountsWrapper::new(accs))),
Ok(accs) => Arc::into_raw(Arc::new(RwLock::new(accs))),
Err(err) => {
// We are using Anyhow's .context() and to show the inner error, too, we need the {:#}:
eprintln!("failed to create accounts: {err:#}");
ptr::null_mut()
}
}
}
pub type dc_event_channel_t = Mutex<Option<Events>>;
#[no_mangle]
pub unsafe extern "C" fn dc_event_channel_new() -> *mut dc_event_channel_t {
Box::into_raw(Box::new(Mutex::new(Some(Events::new()))))
}
/// Release the events channel structure.
///
/// This function releases the memory of the `dc_event_channel_t` structure.
///
/// you can call it after calling dc_accounts_new_with_event_channel,
/// which took the events channel out of it already, so this just frees the underlying option.
#[no_mangle]
pub unsafe extern "C" fn dc_event_channel_unref(event_channel: *mut dc_event_channel_t) {
if event_channel.is_null() {
eprintln!("ignoring careless call to dc_event_channel_unref()");
return;
}
drop(Box::from_raw(event_channel))
}
#[no_mangle]
pub unsafe extern "C" fn dc_event_channel_get_event_emitter(
event_channel: *mut dc_event_channel_t,
) -> *mut dc_event_emitter_t {
if event_channel.is_null() {
eprintln!("ignoring careless call to dc_event_channel_get_event_emitter()");
return ptr::null_mut();
}
let Some(event_channel) = &*(*event_channel)
.lock()
.expect("call to dc_event_channel_get_event_emitter() failed: mutex is poisoned")
else {
eprintln!(
"ignoring careless call to dc_event_channel_get_event_emitter()
-> channel was already consumed, make sure you call this before dc_accounts_new_with_event_channel"
);
return ptr::null_mut();
};
let emitter = event_channel.get_emitter();
Box::into_raw(Box::new(emitter))
}
#[no_mangle]
pub unsafe extern "C" fn dc_accounts_new_with_event_channel(
dir: *const libc::c_char,
writable: libc::c_int,
event_channel: *mut dc_event_channel_t,
) -> *const dc_accounts_t {
setup_panic!();
if dir.is_null() || event_channel.is_null() {
eprintln!("ignoring careless call to dc_accounts_new_with_event_channel()");
return ptr::null_mut();
}
// consuming channel enforce that you need to get the event emitter
// before initializing the account manager,
// so that you don't miss events/errors during initialisation.
// It also prevents you from using the same channel on multiple account managers.
let Some(event_channel) = (*event_channel)
.lock()
.expect("call to dc_event_channel_get_event_emitter() failed: mutex is poisoned")
.take()
else {
eprintln!(
"ignoring careless call to dc_accounts_new_with_event_channel()
-> channel was already consumed"
);
return ptr::null_mut();
};
let accs = block_on(Accounts::new_with_events(
as_path(dir).into(),
writable != 0,
event_channel,
));
match accs {
Ok(accs) => Arc::into_raw(Arc::new(RwLock::new(accs))),
Err(err) => {
// We are using Anyhow's .context() and to show the inner error, too, we need the {:#}:
eprintln!("failed to create accounts: {err:#}");
@@ -4789,17 +4860,17 @@ pub unsafe extern "C" fn dc_accounts_new(
///
/// This function releases the memory of the `dc_accounts_t` structure.
#[no_mangle]
pub unsafe extern "C" fn dc_accounts_unref(accounts: *mut dc_accounts_t) {
pub unsafe extern "C" fn dc_accounts_unref(accounts: *const dc_accounts_t) {
if accounts.is_null() {
eprintln!("ignoring careless call to dc_accounts_unref()");
return;
}
let _ = Box::from_raw(accounts);
drop(Arc::from_raw(accounts));
}
#[no_mangle]
pub unsafe extern "C" fn dc_accounts_get_account(
accounts: *mut dc_accounts_t,
accounts: *const dc_accounts_t,
id: u32,
) -> *mut dc_context_t {
if accounts.is_null() {
@@ -4816,7 +4887,7 @@ pub unsafe extern "C" fn dc_accounts_get_account(
#[no_mangle]
pub unsafe extern "C" fn dc_accounts_get_selected_account(
accounts: *mut dc_accounts_t,
accounts: *const dc_accounts_t,
) -> *mut dc_context_t {
if accounts.is_null() {
eprintln!("ignoring careless call to dc_accounts_get_selected_account()");
@@ -4832,7 +4903,7 @@ pub unsafe extern "C" fn dc_accounts_get_selected_account(
#[no_mangle]
pub unsafe extern "C" fn dc_accounts_select_account(
accounts: *mut dc_accounts_t,
accounts: *const dc_accounts_t,
id: u32,
) -> libc::c_int {
if accounts.is_null() {
@@ -4856,13 +4927,13 @@ pub unsafe extern "C" fn dc_accounts_select_account(
}
#[no_mangle]
pub unsafe extern "C" fn dc_accounts_add_account(accounts: *mut dc_accounts_t) -> u32 {
pub unsafe extern "C" fn dc_accounts_add_account(accounts: *const dc_accounts_t) -> u32 {
if accounts.is_null() {
eprintln!("ignoring careless call to dc_accounts_add_account()");
return 0;
}
let accounts = &mut *accounts;
let accounts = &*accounts;
block_on(async move {
let mut accounts = accounts.write().await;
@@ -4877,13 +4948,13 @@ pub unsafe extern "C" fn dc_accounts_add_account(accounts: *mut dc_accounts_t) -
}
#[no_mangle]
pub unsafe extern "C" fn dc_accounts_add_closed_account(accounts: *mut dc_accounts_t) -> u32 {
pub unsafe extern "C" fn dc_accounts_add_closed_account(accounts: *const dc_accounts_t) -> u32 {
if accounts.is_null() {
eprintln!("ignoring careless call to dc_accounts_add_closed_account()");
return 0;
}
let accounts = &mut *accounts;
let accounts = &*accounts;
block_on(async move {
let mut accounts = accounts.write().await;
@@ -4899,7 +4970,7 @@ pub unsafe extern "C" fn dc_accounts_add_closed_account(accounts: *mut dc_accoun
#[no_mangle]
pub unsafe extern "C" fn dc_accounts_remove_account(
accounts: *mut dc_accounts_t,
accounts: *const dc_accounts_t,
id: u32,
) -> libc::c_int {
if accounts.is_null() {
@@ -4907,7 +4978,7 @@ pub unsafe extern "C" fn dc_accounts_remove_account(
return 0;
}
let accounts = &mut *accounts;
let accounts = &*accounts;
block_on(async move {
let mut accounts = accounts.write().await;
@@ -4925,7 +4996,7 @@ pub unsafe extern "C" fn dc_accounts_remove_account(
#[no_mangle]
pub unsafe extern "C" fn dc_accounts_migrate_account(
accounts: *mut dc_accounts_t,
accounts: *const dc_accounts_t,
dbfile: *const libc::c_char,
) -> u32 {
if accounts.is_null() || dbfile.is_null() {
@@ -4933,7 +5004,7 @@ pub unsafe extern "C" fn dc_accounts_migrate_account(
return 0;
}
let accounts = &mut *accounts;
let accounts = &*accounts;
let dbfile = to_string_lossy(dbfile);
block_on(async move {
@@ -4954,7 +5025,7 @@ pub unsafe extern "C" fn dc_accounts_migrate_account(
}
#[no_mangle]
pub unsafe extern "C" fn dc_accounts_get_all(accounts: *mut dc_accounts_t) -> *mut dc_array_t {
pub unsafe extern "C" fn dc_accounts_get_all(accounts: *const dc_accounts_t) -> *mut dc_array_t {
if accounts.is_null() {
eprintln!("ignoring careless call to dc_accounts_get_all()");
return ptr::null_mut();
@@ -4968,18 +5039,18 @@ pub unsafe extern "C" fn dc_accounts_get_all(accounts: *mut dc_accounts_t) -> *m
}
#[no_mangle]
pub unsafe extern "C" fn dc_accounts_start_io(accounts: *mut dc_accounts_t) {
pub unsafe extern "C" fn dc_accounts_start_io(accounts: *const dc_accounts_t) {
if accounts.is_null() {
eprintln!("ignoring careless call to dc_accounts_start_io()");
return;
}
let accounts = &mut *accounts;
let accounts = &*accounts;
block_on(async move { accounts.write().await.start_io().await });
}
#[no_mangle]
pub unsafe extern "C" fn dc_accounts_stop_io(accounts: *mut dc_accounts_t) {
pub unsafe extern "C" fn dc_accounts_stop_io(accounts: *const dc_accounts_t) {
if accounts.is_null() {
eprintln!("ignoring careless call to dc_accounts_stop_io()");
return;
@@ -4990,7 +5061,7 @@ pub unsafe extern "C" fn dc_accounts_stop_io(accounts: *mut dc_accounts_t) {
}
#[no_mangle]
pub unsafe extern "C" fn dc_accounts_maybe_network(accounts: *mut dc_accounts_t) {
pub unsafe extern "C" fn dc_accounts_maybe_network(accounts: *const dc_accounts_t) {
if accounts.is_null() {
eprintln!("ignoring careless call to dc_accounts_maybe_network()");
return;
@@ -5001,7 +5072,7 @@ pub unsafe extern "C" fn dc_accounts_maybe_network(accounts: *mut dc_accounts_t)
}
#[no_mangle]
pub unsafe extern "C" fn dc_accounts_maybe_network_lost(accounts: *mut dc_accounts_t) {
pub unsafe extern "C" fn dc_accounts_maybe_network_lost(accounts: *const dc_accounts_t) {
if accounts.is_null() {
eprintln!("ignoring careless call to dc_accounts_maybe_network_lost()");
return;
@@ -5013,7 +5084,7 @@ pub unsafe extern "C" fn dc_accounts_maybe_network_lost(accounts: *mut dc_accoun
#[no_mangle]
pub unsafe extern "C" fn dc_accounts_background_fetch(
accounts: *mut dc_accounts_t,
accounts: *const dc_accounts_t,
timeout_in_seconds: u64,
) -> libc::c_int {
if accounts.is_null() || timeout_in_seconds <= 2 {
@@ -5032,7 +5103,7 @@ pub unsafe extern "C" fn dc_accounts_background_fetch(
}
#[no_mangle]
pub unsafe extern "C" fn dc_accounts_stop_background_fetch(accounts: *mut dc_accounts_t) {
pub unsafe extern "C" fn dc_accounts_stop_background_fetch(accounts: *const dc_accounts_t) {
if accounts.is_null() {
eprintln!("ignoring careless call to dc_accounts_stop_background_fetch()");
return;
@@ -5044,7 +5115,7 @@ pub unsafe extern "C" fn dc_accounts_stop_background_fetch(accounts: *mut dc_acc
#[no_mangle]
pub unsafe extern "C" fn dc_accounts_set_push_device_token(
accounts: *mut dc_accounts_t,
accounts: *const dc_accounts_t,
token: *const libc::c_char,
) {
if accounts.is_null() {
@@ -5067,7 +5138,7 @@ pub unsafe extern "C" fn dc_accounts_set_push_device_token(
#[no_mangle]
pub unsafe extern "C" fn dc_accounts_get_event_emitter(
accounts: *mut dc_accounts_t,
accounts: *const dc_accounts_t,
) -> *mut dc_event_emitter_t {
if accounts.is_null() {
eprintln!("ignoring careless call to dc_accounts_get_event_emitter()");
@@ -5087,16 +5158,16 @@ pub struct dc_jsonrpc_instance_t {
#[no_mangle]
pub unsafe extern "C" fn dc_jsonrpc_init(
account_manager: *mut dc_accounts_t,
account_manager: *const dc_accounts_t,
) -> *mut dc_jsonrpc_instance_t {
if account_manager.is_null() {
eprintln!("ignoring careless call to dc_jsonrpc_init()");
return ptr::null_mut();
}
let account_manager = &*account_manager;
let account_manager = Arc::from_raw(account_manager);
let cmd_api = block_on(deltachat_jsonrpc::api::CommandApi::from_arc(
account_manager.inner.clone(),
account_manager.clone(),
));
let (request_handle, receiver) = RpcClient::new();
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "deltachat-jsonrpc"
version = "2.35.0"
version = "2.37.0"
description = "DeltaChat JSON-RPC API"
edition = "2021"
license = "MPL-2.0"
+44 -12
View File
@@ -11,11 +11,11 @@ use deltachat::blob::BlobObject;
use deltachat::calls::ice_servers;
use deltachat::chat::{
self, add_contact_to_chat, forward_msgs, forward_msgs_2ctx, get_chat_media, get_chat_msgs,
get_chat_msgs_ex, marknoticed_chat, remove_contact_from_chat, Chat, ChatId, ChatItem,
MessageListOptions,
get_chat_msgs_ex, marknoticed_all_chats, marknoticed_chat, remove_contact_from_chat, Chat,
ChatId, ChatItem, MessageListOptions,
};
use deltachat::chatlist::Chatlist;
use deltachat::config::Config;
use deltachat::config::{get_all_ui_config_keys, Config};
use deltachat::constants::DC_MSG_ID_DAYMARKER;
use deltachat::contact::{may_be_valid_addr, Contact, ContactId, Origin};
use deltachat::context::get_info;
@@ -23,8 +23,8 @@ use deltachat::ephemeral::Timer;
use deltachat::imex;
use deltachat::location;
use deltachat::message::{
self, delete_msgs_ex, get_existing_msg_ids, get_msg_read_receipts, markseen_msgs, Message,
MessageState, MsgId, Viewtype,
self, delete_msgs_ex, get_existing_msg_ids, get_msg_read_receipt_count, get_msg_read_receipts,
markseen_msgs, Message, MessageState, MsgId, Viewtype,
};
use deltachat::peer_channels::{
leave_webxdc_realtime, send_webxdc_realtime_advertisement, send_webxdc_realtime_data,
@@ -416,11 +416,11 @@ impl CommandApi {
Ok(())
}
/// Set configuration values from a QR code. (technically from the URI that is stored in the qrcode)
/// Before this function is called, `checkQr()` should confirm the type of the
/// QR code is `account` or `webrtcInstance`.
/// Set configuration values from a QR code (technically from the URI stored in it).
/// Before this function is called, `check_qr()` should be used to get the QR code type.
///
/// Internally, the function will call dc_set_config() with the appropriate keys,
/// "DCACCOUNT:" and "DCLOGIN:" QR codes configure the account, but I/O mustn't be started for
/// such QR codes, consider using [`Self::add_transport_from_qr`] which also restarts I/O.
async fn set_config_from_qr(&self, account_id: u32, qr_content: String) -> Result<()> {
let ctx = self.get_context(account_id).await?;
qr::set_config_from_qr(&ctx, &qr_content).await
@@ -452,6 +452,12 @@ impl CommandApi {
Ok(result)
}
/// Returns all `ui.*` config keys that were set by the UI.
async fn get_all_ui_config_keys(&self, account_id: u32) -> Result<Vec<String>> {
let ctx = self.get_context(account_id).await?;
get_all_ui_config_keys(&ctx).await
}
async fn set_stock_strings(&self, strings: HashMap<u32, String>) -> Result<()> {
let accounts = self.accounts.read().await;
for (stock_id, stock_message) in strings {
@@ -795,11 +801,11 @@ impl CommandApi {
/// Delete a chat.
///
/// Messages are deleted from the device and the chat database entry is deleted.
/// After that, the event #DC_EVENT_MSGS_CHANGED is posted.
/// After that, a `MsgsChanged` event is emitted.
/// Messages are deleted from the server in background.
///
/// Things that are _not done_ implicitly:
///
/// - Messages are **not deleted from the server**.
/// - The chat or the contact is **not blocked**, so new messages from the user/the group may appear as a contact request
/// and the user may create the chat again.
/// - **Groups are not left** - this would
@@ -1158,10 +1164,24 @@ impl CommandApi {
Ok(None)
}
/// Mark all messages in all chats as _noticed_.
/// Skips messages from blocked contacts, but does not skip messages in muted chats.
///
/// _Noticed_ messages are no longer _fresh_ and do not count as being unseen
/// but are still waiting for being marked as "seen" using markseen_msgs()
/// (read receipts aren't sent for noticed messages).
///
/// Calling this function usually results in the event #DC_EVENT_MSGS_NOTICED.
/// See also markseen_msgs().
pub async fn marknoticed_all_chats(&self, account_id: u32) -> Result<()> {
let ctx = self.get_context(account_id).await?;
marknoticed_all_chats(&ctx).await
}
/// Mark all messages in a chat as _noticed_.
/// _Noticed_ messages are no longer _fresh_ and do not count as being unseen
/// but are still waiting for being marked as "seen" using markseen_msgs()
/// (IMAP/MDNs is not done for noticed messages).
/// (read receipts aren't sent for noticed messages).
///
/// Calling this function usually results in the event #DC_EVENT_MSGS_NOTICED.
/// See also markseen_msgs().
@@ -1428,6 +1448,18 @@ impl CommandApi {
MessageInfo::from_msg_id(&ctx, MsgId::new(message_id)).await
}
/// Returns count of read receipts on message.
///
/// This view count is meant as a feedback measure for the channel owner only.
async fn get_message_read_receipt_count(
&self,
account_id: u32,
message_id: u32,
) -> Result<usize> {
let ctx = self.get_context(account_id).await?;
get_msg_read_receipt_count(&ctx, MsgId::new(message_id)).await
}
/// Returns contacts that sent read receipts and the time of reading.
async fn get_message_read_receipts(
&self,
+4 -20
View File
@@ -10,7 +10,6 @@ use serde::{Deserialize, Serialize};
use typescript_type_def::TypeDef;
use super::color_int_to_hex_string;
use super::contact::ContactObject;
#[derive(Serialize, TypeDef, schemars::JsonSchema)]
#[serde(rename_all = "camelCase")]
@@ -48,7 +47,6 @@ pub struct FullChat {
chat_type: JsonrpcChatType,
is_unpromoted: bool,
is_self_talk: bool,
contacts: Vec<ContactObject>,
contact_ids: Vec<u32>,
/// Contact IDs of the past chat members.
@@ -61,7 +59,7 @@ pub struct FullChat {
is_device_chat: bool,
/// Note that this is different from
/// [`ChatListItem::is_self_in_group`](`crate::api::types::chat_list::ChatListItemFetchResult::ChatListItem::is_self_in_group`).
/// [`ChatListItem::self_in_group`](`crate::api::types::chat_list::ChatListItemFetchResult::ChatListItem::self_in_group`).
/// This property should only be accessed
/// when [`FullChat::chat_type`] is [`Chattype::Group`].
//
@@ -83,20 +81,6 @@ impl FullChat {
let contact_ids = get_chat_contacts(context, rust_chat_id).await?;
let past_contact_ids = get_past_chat_contacts(context, rust_chat_id).await?;
let mut contacts = Vec::with_capacity(contact_ids.len());
for contact_id in &contact_ids {
contacts.push(
ContactObject::try_from_dc_contact(
context,
Contact::get_by_id(context, *contact_id)
.await
.context("failed to load contact")?,
)
.await?,
)
}
let profile_image = match chat.get_profile_image(context).await? {
Some(path_buf) => path_buf.to_str().map(|s| s.to_owned()),
None => None,
@@ -132,7 +116,6 @@ impl FullChat {
chat_type: chat.get_type().into(),
is_unpromoted: chat.is_unpromoted(),
is_self_talk: chat.is_self_talk(),
contacts,
contact_ids: contact_ids.iter().map(|id| id.to_u32()).collect(),
past_contact_ids: past_contact_ids.iter().map(|id| id.to_u32()).collect(),
color,
@@ -150,7 +133,6 @@ impl FullChat {
}
/// cheaper version of fullchat, omits:
/// - contacts
/// - contact_ids
/// - fresh_message_counter
/// - ephemeral_timer
@@ -196,8 +178,9 @@ pub struct BasicChat {
is_self_talk: bool,
color: String,
is_contact_request: bool,
// deprecated 2026-01, use is_device_talk instead
is_device_chat: bool,
is_device_talk: bool,
is_muted: bool,
}
@@ -225,6 +208,7 @@ impl BasicChat {
color,
is_contact_request: chat.is_contact_request(),
is_device_chat: chat.is_device_talk(),
is_device_talk: chat.is_device_talk(),
is_muted: chat.is_muted(),
})
}
+13 -2
View File
@@ -17,12 +17,15 @@ use super::message::MessageViewtype;
#[derive(Serialize, TypeDef, schemars::JsonSchema)]
#[serde(tag = "kind")]
#[expect(clippy::large_enum_variant)]
pub enum ChatListItemFetchResult {
#[serde(rename_all = "camelCase")]
ChatListItem {
id: u32,
name: String,
/// deprecated 2026-01, use profile_image instead
avatar_path: Option<String>,
profile_image: Option<String>,
color: String,
chat_type: JsonrpcChatType,
last_updated: Option<i64>,
@@ -61,9 +64,14 @@ pub enum ChatListItemFetchResult {
is_self_talk: bool,
is_device_talk: bool,
is_sending_location: bool,
/// deprecated 2026-01
is_self_in_group: bool,
/// deprecated 2026-01, use archived instead
is_archived: bool,
archived: bool,
/// deprecated 2026-01, use pinned instead
is_pinned: bool,
pinned: bool,
is_muted: bool,
is_contact_request: bool,
/// contact id if this is a dm chat (for view profile entry in context menu)
@@ -105,7 +113,7 @@ pub(crate) async fn get_chat_list_item_by_id(
let visibility = chat.get_visibility();
let avatar_path = chat
let profile_image = chat
.get_profile_image(ctx)
.await?
.map(|path| path.to_str().unwrap_or("invalid/path").to_owned());
@@ -150,7 +158,8 @@ pub(crate) async fn get_chat_list_item_by_id(
Ok(ChatListItemFetchResult::ChatListItem {
id: chat_id.to_u32(),
name: chat.get_name().to_owned(),
avatar_path,
profile_image: profile_image.clone(),
avatar_path: profile_image.clone(),
color,
chat_type: chat.get_type().into(),
last_updated,
@@ -165,7 +174,9 @@ pub(crate) async fn get_chat_list_item_by_id(
is_device_talk: chat.is_device_talk(),
is_self_in_group: chat.is_self_in_chat(ctx).await?,
is_sending_location: chat.is_sending_locations(),
archived: visibility == ChatVisibility::Archived,
is_archived: visibility == ChatVisibility::Archived,
pinned: visibility == ChatVisibility::Pinned,
is_pinned: visibility == ChatVisibility::Pinned,
is_muted: chat.is_muted(),
is_contact_request: chat.is_contact_request(),
+1 -2
View File
@@ -47,8 +47,7 @@ pub struct ContactObject {
///
/// - If `verifierId` != 0,
/// display text "Introduced by ..."
/// with the name and address of the contact
/// formatted by `name_and_addr`/`nameAndAddr`.
/// with the name of the contact.
/// Prefix the text by a green checkmark.
///
/// - If `verifierId` == 0 and `isVerified` != 0,
+5 -5
View File
@@ -463,11 +463,11 @@ pub enum EventType {
/// One or more transports has changed.
///
/// This event is used for tests to detect when transport
/// synchronization messages arrives.
/// UIs don't need to use it, it is unlikely
/// that user modifies transports on multiple
/// devices simultaneously.
/// UI should update the list.
///
/// This event is emitted when transport
/// synchronization messages arrives,
/// but not when the UI modifies the transport list by itself.
TransportsModified,
}
@@ -92,6 +92,9 @@ pub struct MessageObject {
file: Option<String>,
file_mime: Option<String>,
/// The size of the file in bytes, if applicable.
/// If message is a pre-message, then this is the size of the file to be downloaded.
file_bytes: u64,
file_name: Option<String>,
+1 -1
View File
@@ -54,5 +54,5 @@
},
"type": "module",
"types": "dist/deltachat.d.ts",
"version": "2.35.0"
"version": "2.37.0"
}
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "deltachat-repl"
version = "2.35.0"
version = "2.37.0"
license = "MPL-2.0"
edition = "2021"
repository = "https://github.com/chatmail/core"
+1 -1
View File
@@ -1231,7 +1231,7 @@ pub async fn cmdline(context: Context, line: &str, chat_id: &mut ChatId) -> Resu
"setqr" => {
ensure!(!arg1.is_empty(), "Argument <qr-content> missing.");
match set_config_from_qr(&context, arg1).await {
Ok(()) => println!("Config set from QR code, you can now call 'configure'"),
Ok(()) => eprintln!("Config set from the QR code."),
Err(err) => eprintln!("Cannot set config from QR code: {err:?}"),
}
}
+1 -1
View File
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "deltachat-rpc-client"
version = "2.35.0"
version = "2.37.0"
license = "MPL-2.0"
description = "Python client for Delta Chat core JSON-RPC interface"
classifiers = [
@@ -44,8 +44,13 @@ class AttrDict(dict):
super().__setattr__(attr, val)
def _forever(_event: AttrDict) -> bool:
return False
def run_client_cli(
hooks: Optional[Iterable[Tuple[Callable, Union[type, "EventFilter"]]]] = None,
until: Callable[[AttrDict], bool] = _forever,
argv: Optional[list] = None,
**kwargs,
) -> None:
@@ -55,10 +60,11 @@ def run_client_cli(
"""
from .client import Client
_run_cli(Client, hooks, argv, **kwargs)
_run_cli(Client, until, hooks, argv, **kwargs)
def run_bot_cli(
until: Callable[[AttrDict], bool] = _forever,
hooks: Optional[Iterable[Tuple[Callable, Union[type, "EventFilter"]]]] = None,
argv: Optional[list] = None,
**kwargs,
@@ -69,11 +75,12 @@ def run_bot_cli(
"""
from .client import Bot
_run_cli(Bot, hooks, argv, **kwargs)
_run_cli(Bot, until, hooks, argv, **kwargs)
def _run_cli(
client_type: Type["Client"],
until: Callable[[AttrDict], bool] = _forever,
hooks: Optional[Iterable[Tuple[Callable, Union[type, "EventFilter"]]]] = None,
argv: Optional[list] = None,
**kwargs,
@@ -111,7 +118,7 @@ def _run_cli(
kwargs={"email": args.email, "password": args.password},
)
configure_thread.start()
client.run_forever()
client.run_until(until)
def extract_addr(text: str) -> str:
@@ -14,6 +14,7 @@ from typing import (
from ._utils import (
AttrDict,
_forever,
parse_system_add_remove,
parse_system_image_changed,
parse_system_title_changed,
@@ -91,19 +92,28 @@ class Client:
def run_forever(self) -> None:
"""Process events forever."""
self.run_until(lambda _: False)
self.run_until(_forever)
def run_until(self, func: Callable[[AttrDict], bool]) -> AttrDict:
"""Process events until the given callable evaluates to True.
The callable should accept an AttrDict object representing the
last processed event. The event is returned when the callable
evaluates to True.
"""
"""Start the event processing loop."""
self.logger.debug("Listening to incoming events...")
if self.is_configured():
self.account.start_io()
self._process_messages() # Process old messages.
return self._process_events(until_func=func) # Loop over incoming events
def _process_events(
self,
until_func: Callable[[AttrDict], bool],
until_event: EventType = False,
) -> AttrDict:
"""Process events until the given callable evaluates to True,
or until a certain event happens.
The until_func callable should accept an AttrDict object representing
the last processed event. The event is returned when the callable
evaluates to True.
"""
while True:
event = self.account.wait_for_event()
event["kind"] = EventType(event.kind)
@@ -112,10 +122,13 @@ class Client:
if event.kind == EventType.INCOMING_MSG:
self._process_messages()
stop = func(event)
stop = until_func(event)
if stop:
return event
if event.kind == until_event:
return event
def _on_event(self, event: AttrDict, filter_type: Type[EventFilter] = RawEvent) -> None:
for hook, evfilter in self._hooks.get(filter_type, []):
if evfilter.filter(event):
@@ -44,6 +44,14 @@ class Message:
read_receipts = self._rpc.get_message_read_receipts(self.account.id, self.id)
return [AttrDict(read_receipt) for read_receipt in read_receipts]
def get_read_receipt_count(self) -> int:
"""
Returns count of read receipts on message.
This view count is meant as a feedback measure for the channel owner only.
"""
return self._rpc.get_message_read_receipt_count(self.account.id, self.id)
def get_reactions(self) -> Optional[AttrDict]:
"""Get message reactions."""
reactions = self._rpc.get_message_reactions(self.account.id, self.id)
@@ -2,6 +2,7 @@
from __future__ import annotations
import logging
import os
import pathlib
import platform
@@ -204,14 +205,13 @@ def log():
class Printer:
def section(self, msg: str) -> None:
print()
print("=" * 10, msg, "=" * 10)
logging.info("\n%s %s %s", "=" * 10, msg, "=" * 10)
def step(self, msg: str) -> None:
print("-" * 5, "step " + msg, "-" * 5)
logging.info("%s step %s %s", "-" * 5, msg, "-" * 5)
def indent(self, msg: str) -> None:
print(" " + msg)
logging.info(" " + msg)
return Printer()
@@ -261,7 +261,7 @@ def get_core_python_env(tmp_path_factory):
envs[core_version] = venv
python = find_path(venv, "python")
rpc_server_path = find_path(venv, "deltachat-rpc-server")
print(f"python={python}\nrpc_server={rpc_server_path}")
logging.info(f"Paths:\npython={python}\nrpc_server={rpc_server_path}")
return python, rpc_server_path
return get_versioned_venv
+4 -3
View File
@@ -2,6 +2,7 @@ from __future__ import annotations
import imaplib
import io
import logging
import pathlib
import ssl
from contextlib import contextmanager
@@ -45,13 +46,13 @@ class DirectImap:
try:
self.conn.logout()
except (OSError, imaplib.IMAP4.abort):
print("Could not logout direct_imap conn")
logging.warning("Could not logout direct_imap conn")
def create_folder(self, foldername):
try:
self.conn.folder.create(foldername)
except errors.MailboxFolderCreateError as e:
print("Can't create", foldername, "probably it already exists:", str(e))
logging.warning(f"Cannot create '{foldername}', probably it already exists: {str(e)}")
def select_folder(self, foldername: str) -> tuple:
assert not self._idling
@@ -95,7 +96,7 @@ class DirectImap:
messages = self.get_unread_messages()
if messages:
res = self.conn.flag(messages, MailMessageFlags.SEEN, True)
print("marked seen:", messages, res)
logging.info(f"Marked seen: {messages} {res}")
def get_unread_cnt(self) -> int:
return len(self.get_unread_messages())
+45
View File
@@ -107,3 +107,48 @@ def test_no_contact_request_call(acfactory) -> None:
msg = bob.get_message_by_id(event.msg_id)
if msg.get_snapshot().text == "Hello!":
break
def test_who_can_call_me_nobody(acfactory) -> None:
alice, bob = acfactory.get_online_accounts(2)
# Bob sets "who can call me" to "nobody" (2)
bob.set_config("who_can_call_me", "2")
# Bob even accepts Alice in advance so the chat does not appear as contact request.
bob.create_chat(alice)
alice_chat_bob = alice.create_chat(bob)
alice_chat_bob.place_outgoing_call("offer")
alice_chat_bob.send_text("Hello!")
# Notification for "Hello!" message should arrive
# without the call ringing.
while True:
event = bob.wait_for_event()
# There should be no incoming call notification.
assert event.kind != EventType.INCOMING_CALL
if event.kind == EventType.INCOMING_MSG:
msg = bob.get_message_by_id(event.msg_id)
if msg.get_snapshot().text == "Hello!":
break
def test_who_can_call_me_everybody(acfactory) -> None:
"""Test that if "who can call me" setting is set to "everybody", calls arrive even in contact request chats."""
alice, bob = acfactory.get_online_accounts(2)
# Bob sets "who can call me" to "nobody" (0)
bob.set_config("who_can_call_me", "0")
alice_chat_bob = alice.create_chat(bob)
alice_chat_bob.place_outgoing_call("offer")
incoming_call_event = bob.wait_for_event(EventType.INCOMING_CALL)
incoming_call_message = Message(bob, incoming_call_event.msg_id)
# Even with the call arriving, the chat is still in the contact request mode.
incoming_chat = incoming_call_message.get_snapshot().chat
assert incoming_chat.get_basic_snapshot().is_contact_request
@@ -1,7 +1,5 @@
from __future__ import annotations
import base64
import os
from typing import TYPE_CHECKING
from deltachat_rpc_client import Account, EventType, const
@@ -129,7 +127,7 @@ def test_download_on_demand(acfactory: ACFactory) -> None:
msg.get_snapshot().chat.accept()
bob.get_chat_by_id(chat_id).send_message(
"Hello World, this message is bigger than 5 bytes",
html=base64.b64encode(os.urandom(300000)).decode("utf-8"),
file="../test-data/image/screenshot.jpg",
)
message = alice.wait_for_incoming_msg()
@@ -207,6 +207,29 @@ def test_transport_synchronization(acfactory, log) -> None:
assert ac1_clone.wait_for_incoming_msg().get_snapshot().text == "Hello!"
def test_transport_sync_new_as_primary(acfactory, log) -> None:
"""Test synchronization of new transport as primary between devices."""
ac1 = acfactory.get_online_account()
ac1_clone = ac1.clone()
ac1_clone.bring_online()
qr = acfactory.get_account_qr()
ac1.add_transport_from_qr(qr)
ac1_transports = ac1.list_transports()
assert len(ac1_transports) == 2
[transport1, transport2] = ac1_transports
ac1_clone.wait_for_event(EventType.TRANSPORTS_MODIFIED)
assert len(ac1_clone.list_transports()) == 2
assert ac1_clone.get_config("configured_addr") == transport1["addr"]
log.section("ac1 changes the primary transport")
ac1.set_config("configured_addr", transport2["addr"])
ac1_clone.wait_for_event(EventType.TRANSPORTS_MODIFIED)
assert ac1_clone.get_config("configured_addr") == transport2["addr"]
def test_recognize_self_address(acfactory) -> None:
alice, bob = acfactory.get_online_accounts(2)
@@ -696,6 +696,6 @@ def test_withdraw_securejoin_qr(acfactory):
event = alice.wait_for_event()
if (
event.kind == EventType.WARNING
and "Ignoring vg-request-with-auth message because of invalid auth code." in event.msg
and "Ignoring RequestWithAuth message because of invalid auth code." in event.msg
):
break
+363 -63
View File
@@ -5,12 +5,11 @@ import logging
import os
import socket
import subprocess
import time
from unittest.mock import MagicMock
import pytest
from deltachat_rpc_client import Contact, EventType, Message, events
from deltachat_rpc_client import EventType, events
from deltachat_rpc_client.const import DownloadState, MessageState
from deltachat_rpc_client.pytestplugin import E2EE_INFO_MSGS
from deltachat_rpc_client.rpc import JsonRpcError
@@ -333,26 +332,27 @@ def test_receive_imf_failure(acfactory) -> None:
alice_contact_bob = alice.create_contact(bob, "Bob")
alice_chat_bob = alice_contact_bob.create_chat()
bob.set_config("fail_on_receiving_full_msg", "1")
bob.set_config("simulate_receive_imf_error", "1")
alice_chat_bob.send_text("Hello!")
event = bob.wait_for_event(EventType.MSGS_CHANGED)
assert event.chat_id == bob.get_device_chat().id
msg_id = event.msg_id
message = bob.get_message_by_id(msg_id)
snapshot = message.get_snapshot()
version = bob.get_info()["deltachat_core_version"]
assert (
snapshot.text == "❌ Failed to receive a message:"
" Condition failed: `!context.get_config_bool(Config::FailOnReceivingFullMsg).await?`."
" Condition failed: `!context.get_config_bool(Config::SimulateReceiveImfError).await?`."
f" Core version {version}."
" Please report this bug to delta@merlinux.eu or https://support.delta.chat/."
)
# The failed message doesn't break the IMAP loop.
bob.set_config("fail_on_receiving_full_msg", "0")
bob.set_config("simulate_receive_imf_error", "0")
alice_chat_bob.send_text("Hello again!")
message = bob.wait_for_incoming_msg()
snapshot = message.get_snapshot()
assert snapshot.text == "Hello again!"
assert snapshot.download_state == DownloadState.DONE
assert snapshot.error is None
@@ -370,17 +370,41 @@ def test_selfavatar_sync(acfactory, data, log) -> None:
alice.set_config("selfavatar", image)
avatar_config = alice.get_config("selfavatar")
avatar_hash = os.path.basename(avatar_config)
print("Info: avatar hash is ", avatar_hash)
logging.info(f"Avatar hash is {avatar_hash}")
log.section("First device receives avatar change")
alice2.wait_for_event(EventType.SELFAVATAR_CHANGED)
avatar_config2 = alice2.get_config("selfavatar")
avatar_hash2 = os.path.basename(avatar_config2)
print("Info: avatar hash on second device is ", avatar_hash2)
logging.info(f"Avatar hash on second device is {avatar_hash2}")
assert avatar_hash == avatar_hash2
assert avatar_config != avatar_config2
def test_dont_move_sync_msgs(acfactory, direct_imap):
addr, password = acfactory.get_credentials()
ac1 = acfactory.get_unconfigured_account()
ac1.set_config("bcc_self", "1")
ac1.set_config("fix_is_chatmail", "1")
ac1.add_or_update_transport({"addr": addr, "password": password})
ac1.bring_online()
ac1_direct_imap = direct_imap(ac1)
ac1_direct_imap.select_folder("Inbox")
# Sync messages may also be sent during configuration.
inbox_msg_cnt = len(ac1_direct_imap.get_all_messages())
ac1.set_config("displayname", "Alice")
ac1.wait_for_event(EventType.MSG_DELIVERED)
ac1.set_config("displayname", "Bob")
ac1.wait_for_event(EventType.MSG_DELIVERED)
ac1_direct_imap.select_folder("Inbox")
assert len(ac1_direct_imap.get_all_messages()) == inbox_msg_cnt + 2
ac1_direct_imap.select_folder("DeltaChat")
assert len(ac1_direct_imap.get_all_messages()) == 0
def test_reaction_seen_on_another_dev(acfactory) -> None:
alice, bob = acfactory.get_online_accounts(2)
alice2 = alice.clone()
@@ -505,6 +529,103 @@ def test_import_export_backup(acfactory, tmp_path) -> None:
assert alice2.manager.get_system_info()
def test_import_export_online_all(acfactory, tmp_path, data, log) -> None:
(ac1, some1) = acfactory.get_online_accounts(2)
log.section("create some chat content")
some1_addr = some1.get_config("addr")
chat1 = ac1.create_contact(some1).create_chat()
chat1.send_text("msg1")
assert len(ac1.get_contacts()) == 1
original_image_path = data.get_path("image/avatar64x64.png")
chat1.send_file(str(original_image_path))
# Add another 100KB file that ensures that the progress is smooth enough
path = tmp_path / "attachment.txt"
with path.open("w") as file:
file.truncate(100000)
chat1.send_file(str(path))
def assert_account_is_proper(ac):
contacts = ac.get_contacts()
assert len(contacts) == 1
contact2 = contacts[0]
assert contact2.get_snapshot().address == some1_addr
chat2 = contact2.create_chat()
messages = chat2.get_messages()
assert len(messages) == 3 + E2EE_INFO_MSGS
assert messages[0 + E2EE_INFO_MSGS].get_snapshot().text == "msg1"
snapshot = messages[1 + E2EE_INFO_MSGS].get_snapshot()
assert snapshot.file_mime == "image/png"
assert os.stat(snapshot.file).st_size == os.stat(original_image_path).st_size
ac.set_config("displayname", "new displayname")
assert ac.get_config("displayname") == "new displayname"
assert_account_is_proper(ac1)
backupdir = tmp_path / "backup"
backupdir.mkdir()
log.section(f"export all to {backupdir}")
ac1.stop_io()
ac1.export_backup(backupdir)
progress = 0
files_written = []
while True:
event = ac1.wait_for_event()
if event.kind == EventType.IMEX_PROGRESS:
assert event.progress > 0 # Progress 0 indicates error.
assert event.progress < progress + 250
progress = event.progress
if progress == 1000:
break
elif event.kind == EventType.IMEX_FILE_WRITTEN:
files_written.append(event.path)
else:
logging.info(event)
assert len(files_written) == 1
assert os.path.exists(files_written[0])
ac1.start_io()
log.section("get fresh empty account")
ac2 = acfactory.get_unconfigured_account()
log.section("import backup and check it's proper")
ac2.import_backup(files_written[0])
progress = 0
while True:
event = ac2.wait_for_event()
if event.kind == EventType.IMEX_PROGRESS:
assert event.progress > 0 # Progress 0 indicates error.
assert event.progress < progress + 250
progress = event.progress
if progress == 1000:
break
else:
logging.info(event)
assert_account_is_proper(ac1)
assert_account_is_proper(ac2)
log.section(f"Second-time export all to {backupdir}")
ac1.stop_io()
ac1.export_backup(backupdir)
while True:
event = ac1.wait_for_event()
if event.kind == EventType.IMEX_PROGRESS:
assert event.progress > 0
if event.progress == 1000:
break
elif event.kind == EventType.IMEX_FILE_WRITTEN:
files_written.append(event.path)
else:
logging.info(event)
assert len(files_written) == 2
assert os.path.exists(files_written[1])
assert files_written[1] != files_written[0]
assert len(list(backupdir.glob("*.tar"))) == 2
def test_import_export_keys(acfactory, tmp_path) -> None:
alice, bob = acfactory.get_online_accounts(2)
@@ -588,60 +709,6 @@ def test_mdn_doesnt_break_autocrypt(acfactory) -> None:
assert snapshot.show_padlock
def test_reaction_to_partially_fetched_msg(acfactory, tmp_path):
"""See https://github.com/deltachat/deltachat-core-rust/issues/3688 "Partially downloaded
messages are received out of order".
If the Inbox contains X small messages followed by Y large messages followed by Z small
messages, Delta Chat first downloaded a batch of X+Z messages, and then a batch of Y messages.
This bug was discovered by @Simon-Laux while testing reactions PR #3644 and can be reproduced
with online test as follows:
- Bob enables download limit and goes offline.
- Alice sends a large message to Bob and reacts to this message with a thumbs-up.
- Bob goes online
- Bob first processes a reaction message and throws it away because there is no corresponding
message, then processes a partially downloaded message.
- As a result, Bob does not see a reaction
"""
download_limit = 300000
ac1, ac2 = acfactory.get_online_accounts(2)
ac1_addr = ac1.get_config("addr")
chat = ac1.create_chat(ac2)
ac2.set_config("download_limit", str(download_limit))
ac2.stop_io()
logging.info("sending small+large messages from ac1 to ac2")
msgs = []
msgs.append(chat.send_text("hi"))
path = tmp_path / "large"
path.write_bytes(os.urandom(download_limit + 1))
msgs.append(chat.send_file(str(path)))
for m in msgs:
m.wait_until_delivered()
logging.info("sending a reaction to the large message from ac1 to ac2")
# TODO: Find the reason of an occasional message reordering on the server (so that the reaction
# has a lower UID than the previous message). W/a is to sleep for some time to let the reaction
# have a later INTERNALDATE.
time.sleep(1.1)
react_str = "\N{THUMBS UP SIGN}"
msgs.append(msgs[-1].send_reaction(react_str))
msgs[-1].wait_until_delivered()
ac2.start_io()
logging.info("wait for ac2 to receive a reaction")
msg2 = Message(ac2, ac2.wait_for_reactions_changed().msg_id)
assert msg2.get_sender_contact().get_snapshot().address == ac1_addr
assert msg2.get_snapshot().download_state == DownloadState.AVAILABLE
reactions = msg2.get_reactions()
contacts = [Contact(ac2, int(i)) for i in reactions.reactions_by_contact]
assert len(contacts) == 1
assert contacts[0].get_snapshot().address == ac1_addr
assert list(reactions.reactions_by_contact.values())[0] == [react_str]
@pytest.mark.parametrize("n_accounts", [3, 2])
def test_download_limit_chat_assignment(acfactory, tmp_path, n_accounts):
download_limit = 300000
@@ -668,14 +735,159 @@ def test_download_limit_chat_assignment(acfactory, tmp_path, n_accounts):
path = tmp_path / "large"
path.write_bytes(os.urandom(download_limit + 1))
n_done = 0
for i in range(10):
logging.info("Sending message %s", i)
alice_group.send_file(str(path))
snapshot = bob.wait_for_incoming_msg().get_snapshot()
assert snapshot.download_state == DownloadState.AVAILABLE
if snapshot.download_state == DownloadState.DONE:
n_done += 1
# Work around lost and reordered pre-messages.
assert n_done <= 1
else:
assert snapshot.download_state == DownloadState.AVAILABLE
assert snapshot.chat == bob_group
def test_download_small_msg_first(acfactory, tmp_path):
download_limit = 70000
alice, bob0 = acfactory.get_online_accounts(2)
bob1 = bob0.clone()
bob1.set_config("download_limit", str(download_limit))
chat = alice.create_chat(bob0)
path = tmp_path / "large_enough"
path.write_bytes(os.urandom(download_limit + 1))
# Less than 140K, so sent w/o a pre-message.
chat.send_file(str(path))
chat.send_text("hi")
bob0.create_chat(alice)
assert bob0.wait_for_incoming_msg().get_snapshot().text == ""
assert bob0.wait_for_incoming_msg().get_snapshot().text == "hi"
bob1.start_io()
bob1.create_chat(alice)
assert bob1.wait_for_incoming_msg().get_snapshot().text == "hi"
assert bob1.wait_for_incoming_msg().get_snapshot().text == ""
@pytest.mark.parametrize("delete_chat", [False, True])
def test_delete_available_msg(acfactory, tmp_path, direct_imap, delete_chat):
"""
Tests `DownloadState.AVAILABLE` message deletion on the receiver side.
Also tests pre- and post-message deletion on the sender side.
"""
# Min. UI setting as of v2.35
download_limit = 163840
alice, bob = acfactory.get_online_accounts(2)
bob.set_config("download_limit", str(download_limit))
# Avoid immediate deletion from the server
alice.set_config("bcc_self", "1")
bob.set_config("bcc_self", "1")
chat_alice = alice.create_chat(bob)
path = tmp_path / "large"
path.write_bytes(os.urandom(download_limit + 1))
msg_alice = chat_alice.send_file(str(path))
msg_bob = bob.wait_for_incoming_msg()
msg_bob_snapshot = msg_bob.get_snapshot()
assert msg_bob_snapshot.download_state == DownloadState.AVAILABLE
chat_bob = bob.get_chat_by_id(msg_bob_snapshot.chat_id)
# Avoid DeleteMessages sync message
bob.set_config("bcc_self", "0")
if delete_chat:
chat_bob.delete()
else:
bob.delete_messages([msg_bob])
alice.wait_for_event(EventType.SMTP_MESSAGE_SENT)
alice.wait_for_event(EventType.SMTP_MESSAGE_SENT)
alice.set_config("bcc_self", "0")
if delete_chat:
chat_alice.delete()
else:
alice.delete_messages([msg_alice])
for acc in [bob, alice]:
if not delete_chat:
acc.wait_for_event(EventType.MSG_DELETED)
acc_direct_imap = direct_imap(acc)
# Messages may be deleted separately
while True:
acc.wait_for_event(EventType.IMAP_MESSAGE_DELETED)
while True:
event = acc.wait_for_event()
if event.kind == EventType.INFO and "Close/expunge succeeded." in event.msg:
break
if len(acc_direct_imap.get_all_messages()) == 0:
break
def test_delete_fully_downloaded_msg(acfactory, tmp_path, direct_imap):
alice, bob = acfactory.get_online_accounts(2)
# Avoid immediate deletion from the server
bob.set_config("bcc_self", "1")
chat_alice = alice.create_chat(bob)
path = tmp_path / "large"
# Big enough to be sent with a pre-message
path.write_bytes(os.urandom(300000))
chat_alice.send_file(str(path))
msg = bob.wait_for_incoming_msg()
msg_snapshot = msg.get_snapshot()
assert msg_snapshot.download_state == DownloadState.AVAILABLE
msgs_changed_event = bob.wait_for_msgs_changed_event()
assert msgs_changed_event.msg_id == msg.id
msg_snapshot = msg.get_snapshot()
assert msg_snapshot.download_state == DownloadState.DONE
bob_direct_imap = direct_imap(bob)
assert len(bob_direct_imap.get_all_messages()) == 2
# Avoid DeleteMessages sync message
bob.set_config("bcc_self", "0")
bob.delete_messages([msg])
bob.wait_for_event(EventType.MSG_DELETED)
# Messages may be deleted separately
while True:
bob.wait_for_event(EventType.IMAP_MESSAGE_DELETED)
while True:
event = bob.wait_for_event()
if event.kind == EventType.INFO and "Close/expunge succeeded." in event.msg:
break
if len(bob_direct_imap.get_all_messages()) == 0:
break
def test_imap_autodelete_fully_downloaded_msg(acfactory, tmp_path, direct_imap):
alice, bob = acfactory.get_online_accounts(2)
chat_alice = alice.create_chat(bob)
path = tmp_path / "large"
# Big enough to be sent with a pre-message
path.write_bytes(os.urandom(300000))
chat_alice.send_file(str(path))
msg = bob.wait_for_incoming_msg()
msg_snapshot = msg.get_snapshot()
assert msg_snapshot.download_state == DownloadState.AVAILABLE
msgs_changed_event = bob.wait_for_msgs_changed_event()
assert msgs_changed_event.msg_id == msg.id
msg_snapshot = msg.get_snapshot()
assert msg_snapshot.download_state == DownloadState.DONE
bob_direct_imap = direct_imap(bob)
# Messages may be deleted separately
while True:
if len(bob_direct_imap.get_all_messages()) == 0:
break
bob.wait_for_event(EventType.IMAP_MESSAGE_DELETED)
while True:
event = bob.wait_for_event()
if event.kind == EventType.INFO and "Close/expunge succeeded." in event.msg:
break
def test_markseen_contact_request(acfactory):
"""
Test that seen status is synchronized for contact request messages
@@ -699,6 +911,47 @@ def test_markseen_contact_request(acfactory):
assert message2.get_snapshot().state == MessageState.IN_SEEN
@pytest.mark.parametrize("team_profile", [True, False])
def test_no_markseen_in_team_profile(team_profile, acfactory):
"""
Test that seen status is synchronized iff `team_profile` isn't set.
"""
alice, bob = acfactory.get_online_accounts(2)
if team_profile:
bob.set_config("team_profile", "1")
# Bob sets up a second device.
bob2 = bob.clone()
bob2.start_io()
alice_chat_bob = alice.create_chat(bob)
bob_chat_alice = bob.create_chat(alice)
bob2.create_chat(alice)
alice_chat_bob.send_text("Hello Bob!")
message = bob.wait_for_incoming_msg()
message2 = bob2.wait_for_incoming_msg()
assert message2.get_snapshot().state == MessageState.IN_FRESH
message.mark_seen()
# Send a message and wait until it arrives
# in order to wait until Bob2 gets the markseen message.
# This also tests that outgoing messages
# don't mark preceeding messages as seen in team profiles.
bob_chat_alice.send_text("Outgoing message")
while True:
outgoing = bob2.wait_for_msg(EventType.MSGS_CHANGED)
if outgoing.id != 0:
break
assert outgoing.get_snapshot().text == "Outgoing message"
if team_profile:
assert message2.get_snapshot().state == MessageState.IN_FRESH
else:
assert message2.get_snapshot().state == MessageState.IN_SEEN
def test_read_receipt(acfactory):
"""
Test sending a read receipt and ensure it is attributed to the correct contact.
@@ -718,6 +971,9 @@ def test_read_receipt(acfactory):
assert len(read_receipts) == 1
assert read_receipts[0].contact_id == alice_contact_bob.id
read_receipt_cnt = read_msg.get_read_receipt_count()
assert read_receipt_cnt == 1
def test_get_http_response(acfactory):
alice = acfactory.new_configured_account()
@@ -921,6 +1177,30 @@ def test_leave_broadcast(acfactory, all_devices_online):
check_account(bob2, bob2.create_contact(alice), inviter_side=False)
def test_leave_and_delete_group(acfactory, log):
alice, bob = acfactory.get_online_accounts(2)
log.section("Alice creates a group")
alice_chat = alice.create_group("Group")
alice_chat.add_contact(bob)
assert len(alice_chat.get_contacts()) == 2 # Alice and Bob
alice_chat.send_text("hello")
log.section("Bob sees the group, and leaves and deletes it")
msg = bob.wait_for_incoming_msg().get_snapshot()
assert msg.text == "hello"
msg.chat.accept()
msg.chat.leave()
# Bob deletes the chat. This must not prevent the leave message from being sent.
msg.chat.delete()
log.section("Alice receives the delete message")
# After Bob left, only Alice will be left in the group:
while len(alice_chat.get_contacts()) != 1:
alice.wait_for_event(EventType.CHAT_MODIFIED)
def test_immediate_autodelete(acfactory, direct_imap, log):
ac1, ac2 = acfactory.get_online_accounts(2)
@@ -1053,3 +1333,23 @@ def test_synchronize_member_list_on_group_rejoin(acfactory, log):
assert chat.num_contacts() == 2
assert msg.get_snapshot().chat.num_contacts() == 2
def test_large_message(acfactory) -> None:
"""
Test sending large message without download limit set,
so it is sent with pre-message but downloaded without user interaction.
"""
alice, bob = acfactory.get_online_accounts(2)
alice_chat_bob = alice.create_chat(bob)
alice_chat_bob.send_message(
"Hello World, this message is bigger than 5 bytes",
file="../test-data/image/screenshot.jpg",
)
msg = bob.wait_for_incoming_msg()
msgs_changed_event = bob.wait_for_msgs_changed_event()
assert msg.id == msgs_changed_event.msg_id
snapshot = msg.get_snapshot()
assert snapshot.text == "Hello World, this message is bigger than 5 bytes"
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "deltachat-rpc-server"
version = "2.35.0"
version = "2.37.0"
description = "DeltaChat JSON-RPC server"
edition = "2021"
readme = "README.md"
@@ -15,5 +15,5 @@
},
"type": "module",
"types": "index.d.ts",
"version": "2.35.0"
"version": "2.37.0"
}
+9 -9
View File
@@ -24,6 +24,14 @@ use yerpc::{RpcClient, RpcSession};
#[tokio::main(flavor = "multi_thread")]
async fn main() {
// Logs from `log` crate and traces from `tracing` crate
// are configurable with `RUST_LOG` environment variable
// and go to stderr to avoid interfering with JSON-RPC using stdout.
tracing_subscriber::fmt()
.with_env_filter(EnvFilter::from_default_env())
.with_writer(std::io::stderr)
.init();
let r = main_impl().await;
// From tokio documentation:
// "For technical reasons, stdin is implemented by using an ordinary blocking read on a separate
@@ -43,7 +51,7 @@ async fn main_impl() -> Result<()> {
if let Some(arg) = args.next() {
return Err(anyhow!("Unrecognized argument {arg:?}"));
}
eprintln!("{}", &*DC_VERSION_STR);
eprintln!("{DC_VERSION_STR}");
return Ok(());
} else if first_arg.to_str() == Some("--openrpc") {
if let Some(arg) = args.next() {
@@ -64,14 +72,6 @@ async fn main_impl() -> Result<()> {
#[cfg(target_family = "unix")]
let mut sigterm = signal_unix::signal(signal_unix::SignalKind::terminate())?;
// Logs from `log` crate and traces from `tracing` crate
// are configurable with `RUST_LOG` environment variable
// and go to stderr to avoid interfering with JSON-RPC using stdout.
tracing_subscriber::fmt()
.with_env_filter(EnvFilter::from_default_env())
.with_writer(std::io::stderr)
.init();
let path = std::env::var("DC_ACCOUNTS_PATH").unwrap_or_else(|_| "accounts".to_string());
log::info!("Starting with accounts directory `{path}`.");
let writable = true;
+7 -3
View File
@@ -16,7 +16,12 @@ ignore = [
# Unmaintained rustls-pemfile
# It is a transitive dependency of iroh 0.35.0,
# this should be fixed by upgrading to iroh 1.0 once it is released.
"RUSTSEC-2025-0134"
"RUSTSEC-2025-0134",
# Old versions of "lru" are transitive dependencies of iroh 0.35.0.
# <https://rustsec.org/advisories/RUSTSEC-2026-0002>
# <https://github.com/chatmail/core/issues/7692>
"RUSTSEC-2026-0002",
]
[bans]
@@ -31,11 +36,10 @@ skip = [
{ name = "derive_more", version = "1.0.0" },
{ name = "event-listener", version = "2.5.3" },
{ name = "getrandom", version = "0.2.12" },
{ name = "hashbrown", version = "0.14.5" },
{ name = "heck", version = "0.4.1" },
{ name = "http", version = "0.2.12" },
{ name = "linux-raw-sys", version = "0.4.14" },
{ name = "lru", version = "0.12.3" },
{ name = "lru", version = "0.12.5" },
{ name = "netlink-packet-route", version = "0.17.1" },
{ name = "nom", version = "7.1.3" },
{ name = "rand_chacha", version = "0.3.1" },
+1 -1
View File
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "deltachat"
version = "2.35.0"
version = "2.37.0"
license = "MPL-2.0"
description = "Python bindings for the Delta Chat Core library using CFFI against the Rust-implemented libdeltachat"
readme = "README.rst"
-132
View File
@@ -1,7 +1,6 @@
import os
import queue
import sys
import base64
from datetime import datetime, timezone
import pytest
@@ -9,7 +8,6 @@ from imap_tools import AND
import deltachat as dc
from deltachat import account_hookimpl, Message
from deltachat.tracker import ImexTracker
from deltachat.testplugin import E2EE_INFO_MSGS
@@ -222,38 +220,6 @@ def test_webxdc_huge_update(acfactory, data, lp):
assert update["payload"] == payload
def test_webxdc_download_on_demand(acfactory, data, lp):
ac1, ac2 = acfactory.get_online_accounts(2)
acfactory.introduce_each_other([ac1, ac2])
chat = acfactory.get_accepted_chat(ac1, ac2)
msg1 = Message.new_empty(ac1, "webxdc")
msg1.set_text("message1")
msg1.set_file(data.get_path("webxdc/minimal.xdc"))
msg1 = chat.send_msg(msg1)
assert msg1.is_webxdc()
assert msg1.filename
msg2 = ac2._evtracker.wait_next_incoming_message()
assert msg2.is_webxdc()
lp.sec("ac2 sets download limit")
ac2.set_config("download_limit", "100")
assert msg1.send_status_update({"payload": base64.b64encode(os.urandom(300000))}, "some test data")
ac2_update = ac2._evtracker.wait_next_incoming_message()
assert ac2_update.download_state == dc.const.DC_DOWNLOAD_AVAILABLE
assert not msg2.get_status_updates()
ac2_update.download_full()
ac2._evtracker.get_matching("DC_EVENT_WEBXDC_STATUS_UPDATE")
assert msg2.get_status_updates()
# Get a event notifying that the message disappeared from the chat.
msgs_changed_event = ac2._evtracker.get_matching("DC_EVENT_MSGS_CHANGED")
assert msgs_changed_event.data1 == msg2.chat.id
assert msgs_changed_event.data2 == 0
def test_enable_mvbox_move(acfactory, lp):
(ac1,) = acfactory.get_online_accounts(1)
@@ -269,24 +235,6 @@ def test_enable_mvbox_move(acfactory, lp):
assert ac2._evtracker.wait_next_incoming_message().text == "message1"
def test_move_sync_msgs(acfactory):
ac1 = acfactory.new_online_configuring_account(bcc_self=True, sync_msgs=True, fix_is_chatmail=True)
acfactory.bring_accounts_online()
ac1.direct_imap.select_folder("DeltaChat")
# Sync messages may also be sent during the configuration.
mvbox_msg_cnt = len(ac1.direct_imap.get_all_messages())
ac1.set_config("displayname", "Alice")
ac1._evtracker.get_matching("DC_EVENT_MSG_DELIVERED")
ac1.set_config("displayname", "Bob")
ac1._evtracker.get_matching("DC_EVENT_MSG_DELIVERED")
ac1.direct_imap.select_folder("Inbox")
assert len(ac1.direct_imap.get_all_messages()) == 0
ac1.direct_imap.select_folder("DeltaChat")
assert len(ac1.direct_imap.get_all_messages()) == mvbox_msg_cnt + 2
def test_forward_messages(acfactory, lp):
ac1, ac2 = acfactory.get_online_accounts(2)
chat = ac1.create_chat(ac2)
@@ -826,86 +774,6 @@ def test_send_and_receive_image(acfactory, lp, data):
assert m == msg_in
def test_import_export_online_all(acfactory, tmp_path, data, lp):
(ac1, some1) = acfactory.get_online_accounts(2)
lp.sec("create some chat content")
some1_addr = some1.get_config("addr")
chat1 = ac1.create_contact(some1).create_chat()
chat1.send_text("msg1")
assert len(ac1.get_contacts()) == 1
original_image_path = data.get_path("d.png")
chat1.send_image(original_image_path)
# Add another 100KB file that ensures that the progress is smooth enough
path = tmp_path / "attachment.txt"
with path.open("w") as file:
file.truncate(100000)
chat1.send_file(str(path))
def assert_account_is_proper(ac):
contacts = ac.get_contacts()
assert len(contacts) == 1
contact2 = contacts[0]
assert contact2.addr == some1_addr
chat2 = contact2.create_chat()
messages = chat2.get_messages()
assert len(messages) == 3 + E2EE_INFO_MSGS
assert messages[0 + E2EE_INFO_MSGS].text == "msg1"
assert messages[1 + E2EE_INFO_MSGS].filemime == "image/png"
assert os.stat(messages[1 + E2EE_INFO_MSGS].filename).st_size == os.stat(original_image_path).st_size
ac.set_config("displayname", "new displayname")
assert ac.get_config("displayname") == "new displayname"
assert_account_is_proper(ac1)
backupdir = tmp_path / "backup"
backupdir.mkdir()
lp.sec(f"export all to {backupdir}")
with ac1.temp_plugin(ImexTracker()) as imex_tracker:
ac1.stop_io()
ac1.imex(str(backupdir), dc.const.DC_IMEX_EXPORT_BACKUP)
# check progress events for export
assert imex_tracker.wait_progress(1, progress_upper_limit=249)
assert imex_tracker.wait_progress(250, progress_upper_limit=499)
assert imex_tracker.wait_progress(500, progress_upper_limit=749)
assert imex_tracker.wait_progress(750, progress_upper_limit=999)
paths = imex_tracker.wait_finish()
assert len(paths) == 1
path = paths[0]
assert os.path.exists(path)
ac1.start_io()
lp.sec("get fresh empty account")
ac2 = acfactory.get_unconfigured_account()
lp.sec("get latest backup file")
path2 = ac2.get_latest_backupfile(str(backupdir))
assert path2 == path
lp.sec("import backup and check it's proper")
with ac2.temp_plugin(ImexTracker()) as imex_tracker:
ac2.import_all(path)
# check progress events for import
assert imex_tracker.wait_progress(1, progress_upper_limit=249)
assert imex_tracker.wait_progress(1000)
assert_account_is_proper(ac1)
assert_account_is_proper(ac2)
lp.sec(f"Second-time export all to {backupdir}")
ac1.stop_io()
path2 = ac1.export_all(str(backupdir))
assert os.path.exists(path2)
assert path2 != path
assert ac2.get_latest_backupfile(str(backupdir)) == path2
def test_qr_email_capitalization(acfactory, lp):
"""Regression test for a bug
that resulted in failure to propagate verification
+1 -1
View File
@@ -1 +1 @@
2025-12-16
2026-01-08
+13 -3
View File
@@ -60,8 +60,18 @@ impl Accounts {
if writable && !dir.exists() {
Accounts::create(&dir).await?;
}
let events = Events::new();
Accounts::open(events, dir, writable).await
}
Accounts::open(dir, writable).await
/// Loads or creates an accounts folder at the given `dir`.
/// Uses an existing events channel.
pub async fn new_with_events(dir: PathBuf, writable: bool, events: Events) -> Result<Self> {
if writable && !dir.exists() {
Accounts::create(&dir).await?;
}
Accounts::open(events, dir, writable).await
}
/// Get the ID used to log events.
@@ -85,14 +95,14 @@ impl Accounts {
/// Opens an existing accounts structure. Will error if the folder doesn't exist,
/// no account exists and no config exists.
async fn open(dir: PathBuf, writable: bool) -> Result<Self> {
async fn open(events: Events, dir: PathBuf, writable: bool) -> Result<Self> {
ensure!(dir.exists(), "directory does not exist");
let config_file = dir.join(CONFIG_NAME);
ensure!(config_file.exists(), "{config_file:?} does not exist");
let config = Config::from_file(config_file, writable).await?;
let events = Events::new();
let stockstrings = StockStrings::new();
let push_subscriber = PushSubscriber::new();
let accounts = config
+164 -82
View File
@@ -4,6 +4,7 @@
//! This means, the "Call ID" is a "Message ID" - similar to Webxdc IDs.
use crate::chat::ChatIdBlocked;
use crate::chat::{Chat, ChatId, send_msg};
use crate::config::Config;
use crate::constants::{Blocked, Chattype};
use crate::contact::ContactId;
use crate::context::{Context, WeakContext};
@@ -16,6 +17,8 @@ use crate::net::dns::lookup_host_with_cache;
use crate::param::Param;
use crate::tools::{normalize_text, time};
use anyhow::{Context as _, Result, ensure};
use deltachat_derive::{FromSql, ToSql};
use num_traits::FromPrimitive;
use sdp::SessionDescription;
use serde::Serialize;
use std::io::Cursor;
@@ -348,26 +351,34 @@ impl Context {
false
}
};
if let Some(chat_id_blocked) =
ChatIdBlocked::lookup_by_contact(self, from_id).await?
{
match chat_id_blocked.blocked {
Blocked::Not => {
self.emit_event(EventType::IncomingCall {
msg_id: call.msg.id,
chat_id: call.msg.chat_id,
place_call_info: call.place_call_info.to_string(),
has_video,
});
}
Blocked::Yes | Blocked::Request => {
// Do not notify about incoming calls
// from contact requests and blocked contacts.
//
// User can still access the call and accept it
// via the chat in case of contact requests.
}
}
let can_call_me = match who_can_call_me(self).await? {
WhoCanCallMe::Contacts => ChatIdBlocked::lookup_by_contact(self, from_id)
.await?
.is_some_and(|chat_id_blocked| {
match chat_id_blocked.blocked {
Blocked::Not => true,
Blocked::Yes | Blocked::Request => {
// Do not notify about incoming calls
// from contact requests and blocked contacts.
//
// User can still access the call and accept it
// via the chat in case of contact requests.
false
}
}
}),
WhoCanCallMe::Everybody => ChatIdBlocked::lookup_by_contact(self, from_id)
.await?
.is_none_or(|chat_id_blocked| chat_id_blocked.blocked != Blocked::Yes),
WhoCanCallMe::Nobody => false,
};
if can_call_me {
self.emit_event(EventType::IncomingCall {
msg_id: call.msg.id,
chat_id: call.msg.chat_id,
place_call_info: call.place_call_info.to_string(),
has_video,
});
}
let wait = call.remaining_ring_seconds();
let context = self.get_weak_context();
@@ -606,33 +617,7 @@ struct IceServer {
pub credential: Option<String>,
}
/// Creates JSON with ICE servers.
async fn create_ice_servers(
context: &Context,
hostname: &str,
port: u16,
username: &str,
password: &str,
) -> Result<String> {
// Do not use cache because there is no TLS.
let load_cache = false;
let urls: Vec<String> = lookup_host_with_cache(context, hostname, port, "", load_cache)
.await?
.into_iter()
.map(|addr| format!("turn:{addr}"))
.collect();
let ice_server = IceServer {
urls,
username: Some(username.to_string()),
credential: Some(password.to_string()),
};
let json = serde_json::to_string(&[ice_server])?;
Ok(json)
}
/// Creates JSON with ICE servers from a line received over IMAP METADATA.
/// Creates ICE servers from a line received over IMAP METADATA.
///
/// IMAP METADATA returns a line such as
/// `example.com:3478:1758650868:8Dqkyyu11MVESBqjbIylmB06rv8=`
@@ -642,20 +627,107 @@ async fn create_ice_servers(
/// while `8Dqkyyu11MVESBqjbIylmB06rv8=`
/// is the password.
pub(crate) async fn create_ice_servers_from_metadata(
context: &Context,
metadata: &str,
) -> Result<(i64, String)> {
) -> Result<(i64, Vec<UnresolvedIceServer>)> {
let (hostname, rest) = metadata.split_once(':').context("Missing hostname")?;
let (port, rest) = rest.split_once(':').context("Missing port")?;
let port = u16::from_str(port).context("Failed to parse the port")?;
let (ts, password) = rest.split_once(':').context("Missing timestamp")?;
let expiration_timestamp = i64::from_str(ts).context("Failed to parse the timestamp")?;
let ice_servers = create_ice_servers(context, hostname, port, ts, password).await?;
let ice_servers = vec![UnresolvedIceServer::Turn {
hostname: hostname.to_string(),
port,
username: ts.to_string(),
credential: password.to_string(),
}];
Ok((expiration_timestamp, ice_servers))
}
/// STUN or TURN server with unresolved DNS name.
#[derive(Debug, Clone)]
pub(crate) enum UnresolvedIceServer {
/// STUN server.
Stun { hostname: String, port: u16 },
/// TURN server with the username and password.
Turn {
hostname: String,
port: u16,
username: String,
credential: String,
},
}
/// Resolves domain names of ICE servers.
///
/// On failure to resolve, logs the error
/// and skips the server, but does not fail.
pub(crate) async fn resolve_ice_servers(
context: &Context,
unresolved_ice_servers: Vec<UnresolvedIceServer>,
) -> Result<String> {
let mut result: Vec<IceServer> = Vec::new();
// Do not use cache because there is no TLS.
let load_cache = false;
for unresolved_ice_server in unresolved_ice_servers {
match unresolved_ice_server {
UnresolvedIceServer::Stun { hostname, port } => {
match lookup_host_with_cache(context, &hostname, port, "", load_cache).await {
Ok(addrs) => {
let urls: Vec<String> = addrs
.into_iter()
.map(|addr| format!("stun:{addr}"))
.collect();
let stun_server = IceServer {
urls,
username: None,
credential: None,
};
result.push(stun_server);
}
Err(err) => {
warn!(
context,
"Failed to resolve STUN {hostname}:{port}: {err:#}."
);
}
}
}
UnresolvedIceServer::Turn {
hostname,
port,
username,
credential,
} => match lookup_host_with_cache(context, &hostname, port, "", load_cache).await {
Ok(addrs) => {
let urls: Vec<String> = addrs
.into_iter()
.map(|addr| format!("turn:{addr}"))
.collect();
let turn_server = IceServer {
urls,
username: Some(username),
credential: Some(credential),
};
result.push(turn_server);
}
Err(err) => {
warn!(
context,
"Failed to resolve TURN {hostname}:{port}: {err:#}."
);
}
},
}
}
let json = serde_json::to_string(&result)?;
Ok(json)
}
/// Creates JSON with ICE servers when no TURN servers are known.
pub(crate) async fn create_fallback_ice_servers(context: &Context) -> Result<String> {
pub(crate) fn create_fallback_ice_servers() -> Vec<UnresolvedIceServer> {
// Do not use public STUN server from https://stunprotocol.org/.
// It changes the hostname every year
// (e.g. stunserver2025.stunprotocol.org
@@ -663,36 +735,18 @@ pub(crate) async fn create_fallback_ice_servers(context: &Context) -> Result<Str
// because of bandwidth costs:
// <https://github.com/jselbie/stunserver/issues/50>
let hostname = "nine.testrun.org";
// Do not use cache because there is no TLS.
let load_cache = false;
let urls: Vec<String> = lookup_host_with_cache(context, hostname, STUN_PORT, "", load_cache)
.await?
.into_iter()
.map(|addr| format!("stun:{addr}"))
.collect();
let stun_server = IceServer {
urls,
username: None,
credential: None,
};
let hostname = "turn.delta.chat";
// Do not use cache because there is no TLS.
let load_cache = false;
let urls: Vec<String> = lookup_host_with_cache(context, hostname, STUN_PORT, "", load_cache)
.await?
.into_iter()
.map(|addr| format!("turn:{addr}"))
.collect();
let turn_server = IceServer {
urls,
username: Some("public".to_string()),
credential: Some("o4tR7yG4rG2slhXqRUf9zgmHz".to_string()),
};
let json = serde_json::to_string(&[stun_server, turn_server])?;
Ok(json)
vec![
UnresolvedIceServer::Stun {
hostname: "nine.testrun.org".to_string(),
port: STUN_PORT,
},
UnresolvedIceServer::Turn {
hostname: "turn.delta.chat".to_string(),
port: STUN_PORT,
username: "public".to_string(),
credential: "o4tR7yG4rG2slhXqRUf9zgmHz".to_string(),
},
]
}
/// Returns JSON with ICE servers.
@@ -706,11 +760,39 @@ pub(crate) async fn create_fallback_ice_servers(context: &Context) -> Result<Str
/// <https://github.com/deltachat/deltachat-desktop/issues/5447>.
pub async fn ice_servers(context: &Context) -> Result<String> {
if let Some(ref metadata) = *context.metadata.read().await {
Ok(metadata.ice_servers.clone())
let ice_servers = resolve_ice_servers(context, metadata.ice_servers.clone()).await?;
Ok(ice_servers)
} else {
Ok("[]".to_string())
}
}
/// "Who can call me" config options.
#[derive(
Debug, Default, Display, Clone, Copy, PartialEq, Eq, FromPrimitive, ToPrimitive, FromSql, ToSql,
)]
#[repr(u8)]
pub enum WhoCanCallMe {
/// Everybody can call me if they are not blocked.
///
/// This includes contact requests.
Everybody = 0,
/// Every contact who is not blocked and not a contact request, can call.
#[default]
Contacts = 1,
/// Nobody can call me.
Nobody = 2,
}
/// Returns currently configuration of the "who can call me" option.
async fn who_can_call_me(context: &Context) -> Result<WhoCanCallMe> {
let who_can_call_me =
WhoCanCallMe::from_i32(context.get_config_int(Config::WhoCanCallMe).await?)
.unwrap_or_default();
Ok(who_can_call_me)
}
#[cfg(test)]
mod calls_tests;
+1 -63
View File
@@ -2,7 +2,7 @@ use super::*;
use crate::chat::forward_msgs;
use crate::config::Config;
use crate::constants::DC_CHAT_ID_TRASH;
use crate::receive_imf::{receive_imf, receive_imf_from_inbox};
use crate::receive_imf::receive_imf;
use crate::test_utils::{TestContext, TestContextManager};
struct CallSetup {
@@ -610,65 +610,3 @@ async fn test_end_text_call() -> Result<()> {
Ok(())
}
/// Tests that partially downloaded "call ended"
/// messages are not processed.
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_no_partial_calls() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
let seen = false;
// The messages in the test
// have no `Date` on purpose,
// so they are treated as new.
let received_call = receive_imf(
alice,
b"From: bob@example.net\n\
To: alice@example.org\n\
Message-ID: <first@example.net>\n\
Chat-Version: 1.0\n\
Chat-Content: call\n\
Chat-Webrtc-Room: YWFhYWFhYWFhCg==\n\
\n\
Hello, this is a call\n",
seen,
)
.await?
.unwrap();
assert_eq!(received_call.msg_ids.len(), 1);
let call_msg = Message::load_from_db(alice, received_call.msg_ids[0])
.await
.unwrap();
assert_eq!(call_msg.viewtype, Viewtype::Call);
assert_eq!(call_state(alice, call_msg.id).await?, CallState::Alerting);
let imf_raw = b"From: bob@example.net\n\
To: alice@example.org\n\
Message-ID: <second@example.net>\n\
In-Reply-To: <first@example.net>\n\
Chat-Version: 1.0\n\
Chat-Content: call-ended\n\
\n\
Call ended\n";
receive_imf_from_inbox(
alice,
"second@example.net",
imf_raw,
seen,
Some(imf_raw.len().try_into().unwrap()),
)
.await?;
// The call is still not ended.
assert_eq!(call_state(alice, call_msg.id).await?, CallState::Alerting);
// Fully downloading the message ends the call.
receive_imf_from_inbox(alice, "second@example.net", imf_raw, seen, None)
.await
.context("Failed to fully download end call message")?;
assert_eq!(call_state(alice, call_msg.id).await?, CallState::Missed);
Ok(())
}
+206 -72
View File
@@ -12,12 +12,14 @@ use std::time::Duration;
use anyhow::{Context as _, Result, anyhow, bail, ensure};
use chrono::TimeZone;
use deltachat_contact_tools::{ContactAddress, sanitize_bidi_characters, sanitize_single_line};
use humansize::{BINARY, format_size};
use mail_builder::mime::MimePart;
use serde::{Deserialize, Serialize};
use strum_macros::EnumIter;
use crate::blob::BlobObject;
use crate::chatlist::Chatlist;
use crate::chatlist_events;
use crate::color::str_to_color;
use crate::config::Config;
use crate::constants::{
@@ -27,7 +29,9 @@ use crate::constants::{
use crate::contact::{self, Contact, ContactId, Origin};
use crate::context::Context;
use crate::debug_logging::maybe_set_logging_xdc;
use crate::download::DownloadState;
use crate::download::{
DownloadState, PRE_MSG_ATTACHMENT_SIZE_THRESHOLD, PRE_MSG_SIZE_WARNING_THRESHOLD,
};
use crate::ephemeral::{Timer as EphemeralTimer, start_chat_ephemeral_timers};
use crate::events::EventType;
use crate::key::self_fingerprint;
@@ -35,11 +39,11 @@ use crate::location;
use crate::log::{LogExt, warn};
use crate::logged_debug_assert;
use crate::message::{self, Message, MessageState, MsgId, Viewtype};
use crate::mimefactory::MimeFactory;
use crate::mimefactory::{MimeFactory, RenderedEmail};
use crate::mimeparser::SystemMessage;
use crate::param::{Param, Params};
use crate::receive_imf::ReceivedMsg;
use crate::smtp::send_msg_to_smtp;
use crate::smtp::{self, send_msg_to_smtp};
use crate::stock_str;
use crate::sync::{self, Sync::*, SyncData};
use crate::tools::{
@@ -48,7 +52,6 @@ use crate::tools::{
gm2local_offset, normalize_text, smeared_time, time, truncate_msg_text,
};
use crate::webxdc::StatusUpdateSerial;
use crate::{chatlist_events, imap};
pub(crate) const PARAM_BROADCAST_SECRET: Param = Param::Arg3;
@@ -600,6 +603,10 @@ impl ChatId {
}
/// Deletes a chat.
///
/// Messages are deleted from the device and the chat database entry is deleted.
/// After that, a `MsgsChanged` event is emitted.
/// Messages are deleted from the server in background.
pub async fn delete(self, context: &Context) -> Result<()> {
self.delete_ex(context, Sync).await
}
@@ -621,18 +628,26 @@ impl ChatId {
.sql
.transaction(|transaction| {
transaction.execute(
"UPDATE imap SET target=? WHERE rfc724_mid IN (SELECT rfc724_mid FROM msgs WHERE chat_id=?)",
(delete_msgs_target, self,),
"UPDATE imap SET target=? WHERE rfc724_mid IN (SELECT rfc724_mid FROM msgs WHERE chat_id=? AND rfc724_mid!='')",
(&delete_msgs_target, self,),
)?;
transaction.execute(
"DELETE FROM smtp WHERE msg_id IN (SELECT id FROM msgs WHERE chat_id=?)",
(self,),
"UPDATE imap SET target=? WHERE rfc724_mid IN (SELECT pre_rfc724_mid FROM msgs WHERE chat_id=? AND pre_rfc724_mid!='')",
(&delete_msgs_target, self,),
)?;
transaction.execute(
"DELETE FROM msgs_mdns WHERE msg_id IN (SELECT id FROM msgs WHERE chat_id=?)",
(self,),
)?;
transaction.execute("DELETE FROM msgs WHERE chat_id=?", (self,))?;
// If you change which information is preserved here, also change `MsgId::trash()`
// and other places it references.
transaction.execute(
"
INSERT OR REPLACE INTO msgs (id, rfc724_mid, pre_rfc724_mid, timestamp, chat_id, deleted)
SELECT id, rfc724_mid, pre_rfc724_mid, timestamp, ?, 1 FROM msgs WHERE chat_id=?
",
(DC_CHAT_ID_TRASH, self),
)?;
transaction.execute("DELETE FROM chats_contacts WHERE chat_id=?", (self,))?;
transaction.execute("DELETE FROM chats WHERE id=?", (self,))?;
Ok(())
@@ -658,7 +673,7 @@ impl ChatId {
context
.set_config_internal(Config::LastHousekeeping, None)
.await?;
context.scheduler.interrupt_inbox().await;
context.scheduler.interrupt_smtp().await;
Ok(())
}
@@ -2111,15 +2126,16 @@ pub(crate) async fn sync(context: &Context, id: SyncId, action: SyncAction) -> R
context
.add_sync_item(SyncData::AlterChat { id, action })
.await?;
context.scheduler.interrupt_inbox().await;
context.scheduler.interrupt_smtp().await;
Ok(())
}
/// Whether the chat is pinned or archived.
#[derive(Debug, Copy, Eq, PartialEq, Clone, Serialize, Deserialize, EnumIter)]
#[derive(Debug, Copy, Eq, PartialEq, Clone, Serialize, Deserialize, EnumIter, Default)]
#[repr(i8)]
pub enum ChatVisibility {
/// Chat is neither archived nor pinned.
#[default]
Normal = 0,
/// Chat is archived.
@@ -2732,7 +2748,61 @@ async fn prepare_send_msg(
Ok(row_ids)
}
/// Constructs jobs for sending a message and inserts them into the appropriate table.
/// Renders the Message or splits it into Pre- and Post-Message.
///
/// Pre-Message is a small message with metadata which announces a larger Post-Message.
/// Post-Messages are not downloaded in the background.
///
/// If pre-message is not nessesary, this returns `None` as the 0th value.
async fn render_mime_message_and_pre_message(
context: &Context,
msg: &mut Message,
mimefactory: MimeFactory,
) -> Result<(Option<RenderedEmail>, RenderedEmail)> {
let needs_pre_message = msg.viewtype.has_file()
&& mimefactory.will_be_encrypted() // unencrypted is likely email, we don't want to spam by sending multiple messages
&& msg
.get_filebytes(context)
.await?
.context("filebytes not available, even though message has attachment")?
> PRE_MSG_ATTACHMENT_SIZE_THRESHOLD;
if needs_pre_message {
info!(
context,
"Message {} is large and will be split into pre- and post-messages.", msg.id,
);
let mut mimefactory_post_msg = mimefactory.clone();
mimefactory_post_msg.set_as_post_message();
let rendered_msg = mimefactory_post_msg
.render(context)
.await
.context("Failed to render post-message")?;
let mut mimefactory_pre_msg = mimefactory;
mimefactory_pre_msg.set_as_pre_message_for(&rendered_msg);
let rendered_pre_msg = mimefactory_pre_msg
.render(context)
.await
.context("pre-message failed to render")?;
if rendered_pre_msg.message.len() > PRE_MSG_SIZE_WARNING_THRESHOLD {
warn!(
context,
"Pre-message for message {} is larger than expected: {}.",
msg.id,
rendered_pre_msg.message.len()
);
}
Ok((Some(rendered_pre_msg), rendered_msg))
} else {
Ok((None, mimefactory.render(context).await?))
}
}
/// Constructs jobs for sending a message and inserts them into the `smtp` table.
///
/// Updates the message `GuaranteeE2ee` parameter and persists it
/// in the database depending on whether the message
@@ -2765,24 +2835,11 @@ pub(crate) async fn create_send_msg_jobs(context: &Context, msg: &mut Message) -
let from = context.get_primary_self_addr().await?;
let lowercase_from = from.to_lowercase();
// Send BCC to self if it is enabled.
//
// Previous versions of Delta Chat did not send BCC self
// if DeleteServerAfter was set to immediately delete messages
// from the server. This is not the case anymore
// because BCC-self messages are also used to detect
// that message was sent if SMTP server is slow to respond
// and connection is frequently lost
// before receiving status line. NB: This is not a problem for chatmail servers, so `BccSelf`
// disabled by default is fine.
//
// `from` must be the last addr, see `receive_imf_inner()` why.
recipients.retain(|x| x.to_lowercase() != lowercase_from);
if (context.get_config_bool(Config::BccSelf).await?
|| msg.param.get_cmd() == SystemMessage::AutocryptSetupMessage)
&& (context.get_config_delete_server_after().await? != Some(0) || !recipients.is_empty())
if context.get_config_bool(Config::BccSelf).await?
|| msg.param.get_cmd() == SystemMessage::AutocryptSetupMessage
{
recipients.push(from);
smtp::add_self_recipients(context, &mut recipients, needs_encryption).await?;
}
// Default Webxdc integrations are hidden messages and must not be sent out
@@ -2803,13 +2860,32 @@ pub(crate) async fn create_send_msg_jobs(context: &Context, msg: &mut Message) -
return Ok(Vec::new());
}
let rendered_msg = match mimefactory.render(context).await {
Ok(res) => Ok(res),
Err(err) => {
message::set_msg_failed(context, msg, &err.to_string()).await?;
Err(err)
}
}?;
let (rendered_pre_msg, rendered_msg) =
match render_mime_message_and_pre_message(context, msg, mimefactory).await {
Ok(res) => Ok(res),
Err(err) => {
message::set_msg_failed(context, msg, &err.to_string()).await?;
Err(err)
}
}?;
if let (post_msg, Some(pre_msg)) = (&rendered_msg, &rendered_pre_msg) {
info!(
context,
"Message {} sizes: pre-message: {}; post-message: {}.",
msg.id,
format_size(pre_msg.message.len(), BINARY),
format_size(post_msg.message.len(), BINARY),
);
msg.pre_rfc724_mid = pre_msg.rfc724_mid.clone();
} else {
info!(
context,
"Message {} will be sent in one shot (no pre- and post-message). Size: {}.",
msg.id,
format_size(rendered_msg.message.len(), BINARY),
);
}
if needs_encryption && !rendered_msg.is_encrypted {
/* unrecoverable */
@@ -2848,38 +2924,48 @@ pub(crate) async fn create_send_msg_jobs(context: &Context, msg: &mut Message) -
context
.sql
.execute(
"UPDATE msgs SET subject=?, param=? WHERE id=?",
(&msg.subject, msg.param.to_string(), msg.id),
"UPDATE msgs SET pre_rfc724_mid=?, subject=?, param=? WHERE id=?",
(
&msg.pre_rfc724_mid,
&msg.subject,
msg.param.to_string(),
msg.id,
),
)
.await?;
let chunk_size = context.get_max_smtp_rcpt_to().await?;
let trans_fn = |t: &mut rusqlite::Transaction| {
let mut row_ids = Vec::<i64>::new();
if let Some(sync_ids) = rendered_msg.sync_ids_to_delete {
t.execute(
&format!("DELETE FROM multi_device_sync WHERE id IN ({sync_ids})"),
(),
)?;
t.execute(
"INSERT INTO imap_send (mime, msg_id) VALUES (?, ?)",
(&rendered_msg.message, msg.id),
)?;
} else {
for recipients_chunk in recipients.chunks(chunk_size) {
let recipients_chunk = recipients_chunk.join(" ");
let row_id = t.execute(
"INSERT INTO smtp (rfc724_mid, recipients, mime, msg_id) \
VALUES (?1, ?2, ?3, ?4)",
(
&rendered_msg.rfc724_mid,
recipients_chunk,
&rendered_msg.message,
msg.id,
),
)?;
}
let mut stmt = t.prepare(
"INSERT INTO smtp (rfc724_mid, recipients, mime, msg_id)
VALUES (?1, ?2, ?3, ?4)",
)?;
for recipients_chunk in recipients.chunks(chunk_size) {
let recipients_chunk = recipients_chunk.join(" ");
if let Some(pre_msg) = &rendered_pre_msg {
let row_id = stmt.execute((
&pre_msg.rfc724_mid,
&recipients_chunk,
&pre_msg.message,
msg.id,
))?;
row_ids.push(row_id.try_into()?);
}
let row_id = stmt.execute((
&rendered_msg.rfc724_mid,
&recipients_chunk,
&rendered_msg.message,
msg.id,
))?;
row_ids.push(row_id.try_into()?);
}
Ok(row_ids)
};
@@ -3120,6 +3206,36 @@ pub async fn get_chat_msgs_ex(
Ok(items)
}
/// Marks all unread messages in all chats as noticed.
/// Ignores messages from blocked contacts, but does not ignore messages in muted chats.
pub async fn marknoticed_all_chats(context: &Context) -> Result<()> {
// The sql statement here is similar to the one in get_fresh_msgs
let list = context
.sql
.query_map_vec(
"SELECT DISTINCT(c.id)
FROM msgs m
INNER JOIN chats c
ON m.chat_id=c.id
WHERE m.state=?
AND m.hidden=0
AND m.chat_id>9
AND c.blocked=0;",
(MessageState::InFresh,),
|row| {
let msg_id: ChatId = row.get(0)?;
Ok(msg_id)
},
)
.await?;
for chat_id in list {
marknoticed_chat(context, chat_id).await?;
}
Ok(())
}
/// Marks all messages in the chat as noticed.
/// If the given chat-id is the archive-link, marks all messages in all archived chats as noticed.
pub async fn marknoticed_chat(context: &Context, chat_id: ChatId) -> Result<()> {
@@ -3181,7 +3297,7 @@ pub async fn marknoticed_chat(context: &Context, chat_id: ChatId) -> Result<()>
let hidden_messages = context
.sql
.query_map_vec(
"SELECT id, rfc724_mid FROM msgs
"SELECT id FROM msgs
WHERE state=?
AND hidden=1
AND chat_id=?
@@ -3189,16 +3305,11 @@ pub async fn marknoticed_chat(context: &Context, chat_id: ChatId) -> Result<()>
(MessageState::InFresh, chat_id), // No need to check for InNoticed messages, because reactions are never InNoticed
|row| {
let msg_id: MsgId = row.get(0)?;
let rfc724_mid: String = row.get(1)?;
Ok((msg_id, rfc724_mid))
Ok(msg_id)
},
)
.await?;
for (msg_id, rfc724_mid) in &hidden_messages {
message::update_msg_state(context, *msg_id, MessageState::InSeen).await?;
imap::markseen_on_imap_table(context, rfc724_mid).await?;
}
message::markseen_msgs(context, hidden_messages).await?;
if noticed_msgs_count == 0 {
return Ok(());
}
@@ -3220,6 +3331,10 @@ pub(crate) async fn mark_old_messages_as_noticed(
context: &Context,
mut msgs: Vec<ReceivedMsg>,
) -> Result<()> {
if context.get_config_bool(Config::TeamProfile).await? {
return Ok(());
}
msgs.retain(|m| m.state.is_outgoing());
if msgs.is_empty() {
return Ok(());
@@ -3841,7 +3956,7 @@ pub(crate) async fn add_contact_to_chat_ex(
.log_err(context)
.is_ok()
{
context.scheduler.interrupt_inbox().await;
context.scheduler.interrupt_smtp().await;
}
}
context.emit_event(EventType::ChatModified(chat_id));
@@ -4002,7 +4117,7 @@ pub async fn remove_contact_from_chat(
} else {
let mut sync = Nosync;
if chat.is_promoted() {
if chat.is_promoted() && chat.typ != Chattype::OutBroadcast {
remove_from_chat_contacts_table(context, chat_id, contact_id).await?;
} else {
remove_from_chat_contacts_table_without_trace(context, chat_id, contact_id).await?;
@@ -4260,6 +4375,10 @@ pub async fn forward_msgs_2ctx(
msg.viewtype = Viewtype::Text;
}
if msg.download_state != DownloadState::Done {
msg.text += &msg.additional_text;
}
let param = &mut param;
msg.param.steal(param, Param::File);
msg.param.steal(param, Param::Filename);
@@ -4315,7 +4434,7 @@ pub async fn save_msgs(context: &Context, msg_ids: &[MsgId]) -> Result<()> {
})
.await?;
}
context.scheduler.interrupt_inbox().await;
context.scheduler.interrupt_smtp().await;
Ok(())
}
@@ -4336,12 +4455,18 @@ pub(crate) async fn save_copy_in_self_talk(
msg.param.remove(Param::WebxdcDocumentTimestamp);
msg.param.remove(Param::WebxdcSummary);
msg.param.remove(Param::WebxdcSummaryTimestamp);
msg.param.remove(Param::PostMessageFileBytes);
msg.param.remove(Param::PostMessageViewtype);
if msg.download_state != DownloadState::Done {
msg.text += &msg.additional_text;
}
if !msg.original_msg_id.is_unset() {
bail!("message already saved.");
}
let copy_fields = "from_id, to_id, timestamp_rcvd, type, txt,
let copy_fields = "from_id, to_id, timestamp_rcvd, type,
mime_modified, mime_headers, mime_compressed, mime_in_reply_to, subject, msgrmsg";
let row_id = context
.sql
@@ -4349,7 +4474,7 @@ pub(crate) async fn save_copy_in_self_talk(
&format!(
"INSERT INTO msgs ({copy_fields},
timestamp_sent,
chat_id, rfc724_mid, state, timestamp, param, starred)
txt, chat_id, rfc724_mid, state, timestamp, param, starred)
SELECT {copy_fields},
-- Outgoing messages on originating device
-- have timestamp_sent == 0.
@@ -4357,10 +4482,11 @@ pub(crate) async fn save_copy_in_self_talk(
-- so UIs display the same timestamp
-- for saved and original message.
IIF(timestamp_sent == 0, timestamp, timestamp_sent),
?, ?, ?, ?, ?, ?
?, ?, ?, ?, ?, ?, ?
FROM msgs WHERE id=?;"
),
(
msg.text,
dest_chat_id,
dest_rfc724_mid,
if msg.from_id == ContactId::SELF {
@@ -4820,12 +4946,20 @@ async fn set_contacts_by_fingerprints(
context
.sql
.transaction(move |transaction| {
transaction.execute("DELETE FROM chats_contacts WHERE chat_id=?", (id,))?;
// For broadcast channels, we only add members,
// because we don't use the membership consistency algorithm,
// and are using sync messages as a basic way to ensure consistency between devices.
// For groups, we also remove members,
// because the sync message is used in order to sync unpromoted groups.
if chat.typ != Chattype::OutBroadcast {
transaction.execute("DELETE FROM chats_contacts WHERE chat_id=?", (id,))?;
}
// We do not care about `add_timestamp` column
// because timestamps are not used for broadcast channels.
let mut statement = transaction
.prepare("INSERT INTO chats_contacts (chat_id, contact_id) VALUES (?, ?)")?;
let mut statement = transaction.prepare(
"INSERT OR IGNORE INTO chats_contacts (chat_id, contact_id) VALUES (?, ?)",
)?;
for contact_id in &contacts {
statement.execute((id, contact_id))?;
}
+215 -2
View File
@@ -1,6 +1,7 @@
use std::sync::Arc;
use super::*;
use crate::Event;
use crate::chatlist::get_archived_cnt;
use crate::constants::{DC_GCL_ARCHIVED_ONLY, DC_GCL_NO_SPECIALS};
use crate::ephemeral::Timer;
@@ -1240,6 +1241,96 @@ async fn test_unarchive_if_muted() -> Result<()> {
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_marknoticed_all_chats() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
let bob = &tcm.bob().await;
tcm.section("alice: create chats & promote them by sending a message");
let alice_chat_normal = alice
.create_group_with_members("Chat (normal)", &[alice, bob])
.await;
send_text_msg(alice, alice_chat_normal, "Hi".to_string()).await?;
let alice_chat_muted = alice
.create_group_with_members("Chat (muted)", &[alice, bob])
.await;
send_text_msg(alice, alice_chat_muted, "Hi".to_string()).await?;
set_muted(&alice.ctx, alice_chat_muted, MuteDuration::Forever).await?;
let alice_chat_archived_and_muted = alice
.create_group_with_members("Chat (archived and muted)", &[alice, bob])
.await;
send_text_msg(alice, alice_chat_archived_and_muted, "Hi".to_string()).await?;
set_muted(
&alice.ctx,
alice_chat_archived_and_muted,
MuteDuration::Forever,
)
.await?;
alice_chat_archived_and_muted
.set_visibility(&alice.ctx, ChatVisibility::Archived)
.await?;
tcm.section("bob: receive messages, accept all chats and send a reply to each messsage");
while let Some(sent_msg) = alice.pop_sent_msg_opt(Duration::default()).await {
let bob_message = bob.recv_msg(&sent_msg).await;
let bob_chat_id = bob_message.chat_id;
bob_chat_id.accept(bob).await?;
send_text_msg(bob, bob_chat_id, "reply".to_string()).await?;
}
tcm.section("alice: receive replies from bob");
while let Some(sent_msg) = bob.pop_sent_msg_opt(Duration::default()).await {
alice.recv_msg(&sent_msg).await;
}
// ensure chats have unread messages
assert_eq!(alice_chat_normal.get_fresh_msg_cnt(alice).await?, 1);
assert_eq!(alice_chat_muted.get_fresh_msg_cnt(alice).await?, 1);
assert_eq!(
alice_chat_archived_and_muted
.get_fresh_msg_cnt(alice)
.await?,
1
);
tcm.section("alice: mark as read");
alice.evtracker.clear_events();
marknoticed_all_chats(alice).await?;
tcm.section("alice: check that chats are no longer unread and that chatlist update events were received");
assert_eq!(alice_chat_normal.get_fresh_msg_cnt(alice).await?, 0);
assert_eq!(alice_chat_muted.get_fresh_msg_cnt(alice).await?, 0);
assert_eq!(
alice_chat_archived_and_muted
.get_fresh_msg_cnt(alice)
.await?,
0
);
let emitted_events = alice.evtracker.take_events();
for event in &[
EventType::ChatlistItemChanged {
chat_id: Some(alice_chat_normal),
},
EventType::ChatlistItemChanged {
chat_id: Some(alice_chat_muted),
},
EventType::ChatlistItemChanged {
chat_id: Some(alice_chat_archived_and_muted),
},
EventType::ChatlistItemChanged {
chat_id: Some(DC_CHAT_ID_ARCHIVED_LINK),
},
] {
assert!(emitted_events.iter().any(|Event { typ, .. }| typ == event));
}
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_archive_fresh_msgs() -> Result<()> {
let t = TestContext::new_alice().await;
@@ -2862,6 +2953,123 @@ async fn test_broadcast_multidev() -> Result<()> {
Ok(())
}
/// Test that, if the broadcast channel owner has multiple devices
/// and they have diverging views on the recipients,
/// it is synced when sending a member-addition message.
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_broadcast_recipients_sync1() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice1 = &tcm.alice().await;
let alice2 = &tcm.alice().await;
let bob = &tcm.bob().await;
let charlie = &tcm.charlie().await;
for a in &[alice1, alice2] {
a.set_config_bool(Config::SyncMsgs, true).await?;
}
// Alice1 creates a broadcast and adds Bob, but for some reason
// (e.g. because alice2 runs an older version of DC),
// Alice2 doesn't get to know about it
let a1_broadcast_id = create_broadcast(alice1, "Channel".to_string()).await?;
alice1.send_sync_msg().await.unwrap();
alice1.pop_sent_msg().await;
let qr = get_securejoin_qr(alice1, Some(a1_broadcast_id))
.await
.unwrap();
tcm.exec_securejoin_qr(bob, alice1, &qr).await;
// The first sync message got lost, so, alice2 doesn't know about the channel now
sync(alice1, alice2).await;
let a2_chatlist = Chatlist::try_load(alice2, 0, Some("Channel"), None).await?;
assert!(a2_chatlist.is_empty());
// Alice1 adds Charlie to the broadcast channel,
// and now, Alice2 receives the messages
join_securejoin(charlie, &qr).await.unwrap();
let request = charlie.pop_sent_msg().await;
alice1.recv_msg_trash(&request).await;
alice2.recv_msg_trash(&request).await;
let auth_required = alice1.pop_sent_msg().await;
charlie.recv_msg_trash(&auth_required).await;
alice2.recv_msg_trash(&auth_required).await;
let request_with_auth = charlie.pop_sent_msg().await;
alice1.recv_msg_trash(&request_with_auth).await;
alice2.recv_msg_trash(&request_with_auth).await;
let member_added = alice1.pop_sent_msg().await;
let a2_member_added = alice2.recv_msg(&member_added).await;
let _c_member_added = charlie.recv_msg(&member_added).await;
// Alice1 will now sync the full member list to Alice2:
sync(alice1, alice2).await;
let a2_chatlist = Chatlist::try_load(alice2, 0, Some("Channel"), None).await?;
assert_eq!(a2_chatlist.get_msg_id(0)?.unwrap(), a2_member_added.id);
let a2_bob_contact = alice2.add_or_lookup_contact_id(bob).await;
let a2_charlie_contact = alice2.add_or_lookup_contact_id(charlie).await;
let a2_chat_members = get_chat_contacts(alice2, a2_member_added.chat_id).await?;
assert!(a2_chat_members.contains(&a2_bob_contact));
assert!(a2_chat_members.contains(&a2_charlie_contact));
assert_eq!(a2_chat_members.len(), 2);
Ok(())
}
/// Test that, if the broadcast channel owner has multiple devices
/// and they have diverging views on the recipients,
/// sync messages only add members but don't remove them.
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_broadcast_recipients_sync2() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice1 = &tcm.alice().await;
let alice2 = &tcm.alice().await;
let bob = &tcm.bob().await;
let charlie = &tcm.charlie().await;
for a in &[alice1, alice2] {
a.set_config_bool(Config::SyncMsgs, true).await?;
}
let a1_broadcast_id = create_broadcast(alice1, "Channel".to_string()).await?;
sync(alice1, alice2).await;
tcm.section("Alice1 adds Bob, but Alice2 misses it for some reason");
let qr = get_securejoin_qr(alice1, Some(a1_broadcast_id))
.await
.unwrap();
tcm.exec_securejoin_qr(bob, alice1, &qr).await;
tcm.section("Alice2 adds Charlie, but Alice1 misses it for some reason");
let a2_broadcast_id = Chatlist::try_load(alice2, 0, Some("Channel"), None)
.await?
.get_chat_id(0)
.unwrap();
let qr = get_securejoin_qr(alice2, Some(a2_broadcast_id))
.await
.unwrap();
tcm.exec_securejoin_qr(charlie, alice2, &qr).await;
tcm.section("The sync messages should correct the problem");
sync(alice1, alice2).await;
sync(alice2, alice1).await;
for (alice, broadcast_id) in [(alice1, a1_broadcast_id), (alice2, a2_broadcast_id)] {
let bob_contact = alice.add_or_lookup_contact_id(bob).await;
let charlie_contact = alice.add_or_lookup_contact_id(charlie).await;
let chat_members = get_chat_contacts(alice, broadcast_id).await?;
assert!(chat_members.contains(&bob_contact));
assert!(chat_members.contains(&charlie_contact));
assert_eq!(chat_members.len(), 2);
}
Ok(())
}
/// - Create a broadcast channel
/// - Send a message into it in order to promote it
/// - Add a contact
@@ -3116,7 +3324,7 @@ async fn test_broadcast_channel_protected_listid() -> Result<()> {
.await?
.grpid;
let parsed = mimeparser::MimeMessage::from_bytes(bob, sent.payload.as_bytes(), None).await?;
let parsed = mimeparser::MimeMessage::from_bytes(bob, sent.payload.as_bytes()).await?;
assert_eq!(
parsed.get_mailinglist_header().unwrap(),
format!("My Channel <{}>", alice_list_id)
@@ -3227,6 +3435,11 @@ async fn test_remove_member_from_broadcast() -> Result<()> {
let alice_bob_contact_id = alice.add_or_lookup_contact_id(bob).await;
remove_contact_from_chat(alice, alice_chat_id, alice_bob_contact_id).await?;
// Alice must not remember old members,
// because we would like to remember the minimum information possible
let past_contacts = get_past_chat_contacts(alice, alice_chat_id).await?;
assert_eq!(past_contacts.len(), 0);
let remove_msg = alice.pop_sent_msg().await;
let rcvd = bob.recv_msg(&remove_msg).await;
assert_eq!(rcvd.text, "Member Me removed by alice@example.org.");
@@ -3311,7 +3524,7 @@ async fn test_leave_broadcast_multidevice() -> Result<()> {
remove_contact_from_chat(bob0, bob_chat_id, ContactId::SELF).await?;
let leave_msg = bob0.pop_sent_msg().await;
let parsed = MimeMessage::from_bytes(bob1, leave_msg.payload().as_bytes(), None).await?;
let parsed = MimeMessage::from_bytes(bob1, leave_msg.payload().as_bytes()).await?;
assert_eq!(parsed.parts[0].msg, "I left the group.");
let rcvd = bob1.recv_msg(&leave_msg).await;
+68 -7
View File
@@ -76,7 +76,7 @@ impl Chatlist {
/// the pseudo-chat DC_CHAT_ID_ARCHIVED_LINK is added if there are *any* archived
/// chats
/// - the flag DC_GCL_FOR_FORWARDING sorts "Saved messages" to the top of the chatlist
/// and hides the device-chat and contact requests
/// and hides the device-chat, contact requests and incoming broadcasts.
/// typically used on forwarding, may be combined with DC_GCL_NO_SPECIALS
/// - if the flag DC_GCL_NO_SPECIALS is set, archive link is not added
/// to the list (may be used eg. for selecting chats on forwarding, the flag is
@@ -224,8 +224,9 @@ impl Chatlist {
let process_rows = |rows: rusqlite::AndThenRows<_>| {
rows.filter_map(|row: std::result::Result<(_, _, Params, _), _>| match row {
Ok((chat_id, typ, param, msg_id)) => {
if typ == Chattype::Mailinglist
&& param.get(Param::ListPost).is_none_or_empty()
if typ == Chattype::InBroadcast
|| (typ == Chattype::Mailinglist
&& param.get(Param::ListPost).is_none_or_empty())
{
None
} else {
@@ -396,8 +397,6 @@ impl Chatlist {
if lastmsg.from_id == ContactId::SELF {
None
} else if chat.typ == Chattype::Group
|| chat.typ == Chattype::OutBroadcast
|| chat.typ == Chattype::InBroadcast
|| chat.typ == Chattype::Mailinglist
|| chat.is_self_talk()
{
@@ -471,10 +470,11 @@ mod tests {
use super::*;
use crate::chat::save_msgs;
use crate::chat::{
add_contact_to_chat, create_group, get_chat_contacts, remove_contact_from_chat,
send_text_msg, set_chat_name,
add_contact_to_chat, create_broadcast, create_group, get_chat_contacts,
remove_contact_from_chat, send_text_msg, set_chat_name,
};
use crate::receive_imf::receive_imf;
use crate::securejoin::get_securejoin_qr;
use crate::stock_str::StockMessage;
use crate::test_utils::TestContext;
use crate::test_utils::TestContextManager;
@@ -598,6 +598,41 @@ mod tests {
assert_eq!(chats.len(), 1);
}
/// Test that DC_CHAT_TYPE_IN_BROADCAST are hidden
/// and DC_CHAT_TYPE_OUT_BROADCAST are shown in chatlist for forwarding.
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_broadcast_visiblity_on_forward() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
let bob = &tcm.bob().await;
let alice_broadcast_a_id = create_broadcast(alice, "Channel Alice".to_string()).await?;
let qr = get_securejoin_qr(alice, Some(alice_broadcast_a_id))
.await
.unwrap();
let bob_broadcast_a_id = tcm.exec_securejoin_qr(bob, alice, &qr).await;
let bob_broadcast_b_id = create_broadcast(bob, "Channel Bob".to_string()).await?;
let chats = Chatlist::try_load(bob, DC_GCL_FOR_FORWARDING, None, None)
.await
.unwrap();
assert!(
!chats
.iter()
.any(|(chat_id, _)| chat_id == &bob_broadcast_a_id),
"alice broadcast is not shown in bobs forwarding chatlist"
);
assert!(
chats
.iter()
.any(|(chat_id, _)| chat_id == &bob_broadcast_b_id),
"bobs own broadcast is shown in his forwarding chatlist"
);
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_search_special_chat_names() {
let t = TestContext::new_alice().await;
@@ -806,6 +841,32 @@ mod tests {
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_no_summary_prefix_for_channel() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = tcm.alice().await;
let bob = tcm.bob().await;
let alice_chat_id = create_broadcast(&alice, "alice's channel".to_string()).await?;
let qr = get_securejoin_qr(&alice, Some(alice_chat_id)).await?;
tcm.exec_securejoin_qr(&bob, &alice, &qr).await;
send_text_msg(&alice, alice_chat_id, "hi".into()).await?;
let sent1 = alice.pop_sent_msg().await;
let chatlist = Chatlist::try_load(&alice, 0, None, None).await?;
let summary = chatlist.get_summary(&alice, 0, None).await?;
assert!(summary.prefix.is_none());
assert_eq!(summary.text, "hi");
bob.recv_msg(&sent1).await;
let chatlist = Chatlist::try_load(&bob, 0, None, None).await?;
let summary = chatlist.get_summary(&bob, 0, None).await?;
assert!(summary.prefix.is_none());
assert_eq!(summary.text, "hi");
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_load_broken() {
let t = TestContext::new_bob().await;
+39 -12
View File
@@ -354,7 +354,17 @@ pub enum Config {
DonationRequestNextCheck,
/// Defines the max. size (in bytes) of messages downloaded automatically.
///
/// For messages with large attachments, two messages are sent:
/// a Pre-Message containing metadata and text and a Post-Message additionally
/// containing the attachment. NB: Some "extra" metadata like avatars and gossiped
/// encryption keys is stripped from post-messages to save traffic.
/// Pre-Messages are shown as placeholder messages. They can be downloaded fully using
/// `MsgId::download_full()` later. Post-Messages are automatically downloaded if they are
/// smaller than the download_limit. Other messages are always auto-downloaded.
///
/// 0 = no limit.
/// Changes only affect future messages.
#[strum(props(default = "0"))]
DownloadLimit,
@@ -438,14 +448,24 @@ pub enum Config {
/// using this still run unmodified code.
TestHooks,
/// Return an error from `receive_imf_inner()` for a fully downloaded message. For tests.
FailOnReceivingFullMsg,
/// Return an error from `receive_imf_inner()`. For tests.
SimulateReceiveImfError,
/// Enable composing emails with Header Protection as defined in
/// <https://www.rfc-editor.org/rfc/rfc9788.html> "Header Protection for Cryptographically
/// Protected Email".
#[strum(props(default = "1"))]
StdHeaderProtectionComposing,
/// Who can call me.
///
/// The options are from the `WhoCanCallMe` enum.
#[strum(props(default = "1"))]
WhoCanCallMe,
/// Experimental option denoting that the current profile is shared between multiple team members.
/// For now, the only effect of this option is that seen flags are not synchronized.
TeamProfile,
}
impl Config {
@@ -504,7 +524,7 @@ impl Context {
.into_owned()
})
}
Config::SysVersion => Some((*constants::DC_VERSION_STR).clone()),
Config::SysVersion => Some(constants::DC_VERSION_STR.to_string()),
Config::SysMsgsizeMaxRecommended => Some(format!("{RECOMMENDED_FILE_SIZE}")),
Config::SysConfigKeys => Some(get_config_keys_string()),
_ => self.sql.get_raw_config(key.as_ref()).await?,
@@ -605,12 +625,6 @@ impl Context {
&& !self.get_config_bool(Config::Bot).await?)
}
/// Returns whether sync messages should be uploaded to the mvbox.
pub(crate) async fn should_move_sync_msgs(&self) -> Result<bool> {
Ok(self.get_config_bool(Config::MvboxMove).await?
|| !self.get_config_bool(Config::IsChatmail).await?)
}
/// Returns whether MDNs should be requested.
pub(crate) async fn should_request_mdns(&self) -> Result<bool> {
match self.get_config_bool_opt(Config::MdnsEnabled).await? {
@@ -880,7 +894,7 @@ impl Context {
{
return Ok(());
}
self.scheduler.interrupt_inbox().await;
self.scheduler.interrupt_smtp().await;
Ok(())
}
@@ -944,7 +958,7 @@ impl Context {
/// This should only be used by test code and during configure.
#[cfg(test)] // AEAP is disabled, but there are still tests for it
pub(crate) async fn set_primary_self_addr(&self, primary_new: &str) -> Result<()> {
self.quota.write().await.take();
self.quota.write().await.clear();
self.sql
.set_raw_config(Config::ConfiguredAddr.as_ref(), Some(primary_new))
@@ -953,7 +967,7 @@ impl Context {
Ok(())
}
/// Returns all primary and secondary self addresses.
/// Returns the primary self address followed by all secondary ones.
pub(crate) async fn get_all_self_addrs(&self) -> Result<Vec<String>> {
let primary_addrs = self.get_config(Config::ConfiguredAddr).await?.into_iter();
let secondary_addrs = self.get_secondary_self_addrs().await?.into_iter();
@@ -989,5 +1003,18 @@ fn get_config_keys_string() -> String {
format!(" {keys} ")
}
/// Returns all `ui.*` config keys that were set by the UI.
pub async fn get_all_ui_config_keys(context: &Context) -> Result<Vec<String>> {
let ui_keys = context
.sql
.query_map_vec(
"SELECT keyname FROM config WHERE keyname GLOB 'ui.*' ORDER BY config.id",
(),
|row| Ok(row.get::<_, String>(0)?),
)
.await?;
Ok(ui_keys)
}
#[cfg(test)]
mod config_tests;
+33 -2
View File
@@ -81,6 +81,37 @@ async fn test_ui_config() -> Result<()> {
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_get_all_ui_config_keys() -> Result<()> {
let t = TestContext::new().await;
t.set_ui_config("ui.android.screen_security", Some("safe"))
.await?;
t.set_ui_config("ui.lastchatid", Some("231")).await?;
t.set_ui_config(
"ui.desktop.webxdcBounds.528490",
Some(r#"{"x":954,"y":356,"width":378,"height":671}"#),
)
.await?;
t.set_ui_config(
"ui.desktop.webxdcBounds.556543",
Some(r#"{"x":954,"y":356,"width":378,"height":671}"#),
)
.await?;
assert_eq!(
get_all_ui_config_keys(&t).await?,
vec![
"ui.android.screen_security",
"ui.lastchatid",
"ui.desktop.webxdcBounds.528490",
"ui.desktop.webxdcBounds.556543"
]
);
Ok(())
}
/// Regression test for https://github.com/deltachat/deltachat-core-rust/issues/3012
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_set_config_bool() -> Result<()> {
@@ -237,7 +268,7 @@ async fn test_no_sync_on_self_sent_msg() -> Result<()> {
let status = "Sent via usual message";
alice0.set_config(Config::Selfstatus, Some(status)).await?;
alice0.send_sync_msg().await?;
alice0.pop_sent_sync_msg().await;
alice0.pop_sent_msg().await;
let status1 = "Synced via sync message";
alice1.set_config(Config::Selfstatus, Some(status1)).await?;
tcm.send_recv(alice0, alice1, "hi Alice!").await;
@@ -261,7 +292,7 @@ async fn test_no_sync_on_self_sent_msg() -> Result<()> {
.set_config(Config::Selfavatar, Some(file.to_str().unwrap()))
.await?;
alice0.send_sync_msg().await?;
alice0.pop_sent_sync_msg().await;
alice0.pop_sent_msg().await;
let file = alice1.dir.path().join("avatar.jpg");
let bytes = include_bytes!("../../test-data/image/avatar1000x1000.jpg");
tokio::fs::write(&file, bytes).await?;
+11 -7
View File
@@ -210,7 +210,8 @@ impl Context {
/// (i.e. [EnteredLoginParam::addr]).
pub async fn delete_transport(&self, addr: &str) -> Result<()> {
let now = time();
self.sql
let removed_transport_id = self
.sql
.transaction(|transaction| {
let primary_addr = transaction.query_row(
"SELECT value FROM config WHERE keyname='configured_addr'",
@@ -251,10 +252,11 @@ impl Context {
(addr, remove_timestamp),
)?;
Ok(())
Ok(transport_id)
})
.await?;
send_sync_transports(self).await?;
self.quota.write().await.remove(&removed_transport_id);
Ok(())
}
@@ -272,16 +274,18 @@ impl Context {
)
.await?
{
if self.get_config(Config::MvboxMove).await?.as_deref() != Some("0") {
bail!(
"To use additional relays, disable the legacy option \"Settings / Advanced / Move automatically to DeltaChat Folder\"."
);
}
// Should be checked before `MvboxMove` because the latter makes no sense in presense of
// `OnlyFetchMvbox` and even grayed out in the UIs in this case.
if self.get_config(Config::OnlyFetchMvbox).await?.as_deref() != Some("0") {
bail!(
"To use additional relays, disable the legacy option \"Settings / Advanced / Only Fetch from DeltaChat Folder\"."
);
}
if self.get_config(Config::MvboxMove).await?.as_deref() != Some("0") {
bail!(
"To use additional relays, disable the legacy option \"Settings / Advanced / Move automatically to DeltaChat Folder\"."
);
}
if self.get_config(Config::ShowEmails).await?.as_deref() != Some("2") {
bail!(
"To use additional relays, set the legacy option \"Settings / Advanced / Show Classic Emails\" to \"All\"."
+1 -4
View File
@@ -2,16 +2,13 @@
#![allow(missing_docs)]
use std::sync::LazyLock;
use deltachat_derive::{FromSql, ToSql};
use percent_encoding::{AsciiSet, NON_ALPHANUMERIC};
use serde::{Deserialize, Serialize};
use crate::chat::ChatId;
pub static DC_VERSION_STR: LazyLock<String> =
LazyLock::new(|| env!("CARGO_PKG_VERSION").to_string());
pub static DC_VERSION_STR: &str = env!("CARGO_PKG_VERSION");
/// Set of characters to percent-encode in email addresses and names.
pub(crate) const NON_ALPHANUMERIC_WITHOUT_DOT: &AsciiSet = &NON_ALPHANUMERIC.remove(b'.');
+4 -4
View File
@@ -1133,7 +1133,8 @@ VALUES (?, ?, ?, ?, ?, ?)
Origin::IncomingReplyTo
};
if query.is_some() {
let s3str_like_cmd = format!("%{}%", query.unwrap_or("").to_lowercase());
let query_lowercased = query.unwrap_or("").to_lowercase();
let s3str_like_cmd = format!("%{}%", query_lowercased);
context
.sql
.query_map(
@@ -1151,7 +1152,7 @@ ORDER BY c.origin>=? DESC, c.last_seen DESC, c.id DESC
flag_address,
minimal_origin,
&s3str_like_cmd,
&s3str_like_cmd,
&query_lowercased,
Origin::CreateChat,
),
|row| {
@@ -1658,8 +1659,7 @@ WHERE addr=?
///
/// If this returns Some(_),
/// display green checkmark in the profile and "Introduced by ..." line
/// with the name and address of the contact
/// formatted by [Self::get_name_n_addr].
/// with the name of the contact.
///
/// If this returns `Some(None)`, then the contact is verified,
/// but it's unclear by whom.
+6 -1
View File
@@ -85,10 +85,15 @@ async fn test_get_contacts() -> Result<()> {
assert_eq!(contacts.len(), 1);
assert_eq!(contacts.first(), Some(&id));
// Search by address.
// Search by address is case-insensitive, but only returns direct matches.
let contacts = Contact::get_all(&context, 0, Some("alice@example.org")).await?;
assert_eq!(contacts.len(), 1);
assert_eq!(contacts.first(), Some(&id));
let contacts = Contact::get_all(&context, 0, Some("Alice@example.org")).await?;
assert_eq!(contacts.len(), 1);
assert_eq!(contacts.first(), Some(&id));
let contacts = Contact::get_all(&context, 0, Some("alice@")).await?;
assert_eq!(contacts.len(), 0);
let contacts = Contact::get_all(&context, 0, Some("Foobar")).await?;
assert_eq!(contacts.len(), 0);
+32 -33
View File
@@ -243,9 +243,9 @@ pub struct InnerContext {
pub(crate) scheduler: SchedulerState,
pub(crate) ratelimit: RwLock<Ratelimit>,
/// Recently loaded quota information, if any.
/// Set to `None` if quota was never tried to load.
pub(crate) quota: RwLock<Option<QuotaInfo>>,
/// Recently loaded quota information for each trasnport, if any.
/// If quota was never tried to load, then the transport doesn't have an entry in the BTreeMap.
pub(crate) quota: RwLock<BTreeMap<u32, QuotaInfo>>,
/// Notify about new messages.
///
@@ -351,7 +351,7 @@ pub fn get_info() -> BTreeMap<&'static str, String> {
#[cfg(not(debug_assertions))]
res.insert("debug_assertions", "Off".to_string());
res.insert("deltachat_core_version", format!("v{}", &*DC_VERSION_STR));
res.insert("deltachat_core_version", format!("v{DC_VERSION_STR}"));
res.insert("sqlite_version", rusqlite::version().to_string());
res.insert("arch", (std::mem::size_of::<usize>() * 8).to_string());
res.insert("num_cpus", num_cpus::get().to_string());
@@ -479,7 +479,7 @@ impl Context {
events,
scheduler: SchedulerState::new(),
ratelimit: RwLock::new(Ratelimit::new(Duration::new(3, 0), 3.0)), // Allow at least 1 message every second + a burst of 3.
quota: RwLock::new(None),
quota: RwLock::new(BTreeMap::new()),
new_msgs_notify,
server_id: RwLock::new(None),
metadata: RwLock::new(None),
@@ -614,8 +614,13 @@ impl Context {
}
// Update quota (to send warning if full) - but only check it once in a while.
// note: For now this only checks quota of primary transport,
// because background check only checks primary transport at the moment
if self
.quota_needs_update(DC_BACKGROUND_FETCH_QUOTA_CHECK_RATELIMIT)
.quota_needs_update(
session.transport_id(),
DC_BACKGROUND_FETCH_QUOTA_CHECK_RATELIMIT,
)
.await
&& let Err(err) = self.update_recent_quota(&mut session).await
{
@@ -949,6 +954,10 @@ impl Context {
"show_emails",
self.get_config_int(Config::ShowEmails).await?.to_string(),
);
res.insert(
"who_can_call_me",
self.get_config_int(Config::WhoCanCallMe).await?.to_string(),
);
res.insert(
"download_limit",
self.get_config_int(Config::DownloadLimit)
@@ -1083,13 +1092,6 @@ impl Context {
.await?
.unwrap_or_default(),
);
res.insert(
"fail_on_receiving_full_msg",
self.sql
.get_raw_config("fail_on_receiving_full_msg")
.await?
.unwrap_or_default(),
);
res.insert(
"std_header_protection_composing",
self.sql
@@ -1097,6 +1099,10 @@ impl Context {
.await?
.unwrap_or_default(),
);
res.insert(
"team_profile",
self.get_config_bool(Config::TeamProfile).await?.to_string(),
);
let elapsed = time_elapsed(&self.creation_time);
res.insert("uptime", duration_to_str(elapsed));
@@ -1114,21 +1120,19 @@ impl Context {
let list = self
.sql
.query_map_vec(
concat!(
"SELECT m.id",
" FROM msgs m",
" LEFT JOIN contacts ct",
" ON m.from_id=ct.id",
" LEFT JOIN chats c",
" ON m.chat_id=c.id",
" WHERE m.state=?",
" AND m.hidden=0",
" AND m.chat_id>9",
" AND ct.blocked=0",
" AND c.blocked=0",
" AND NOT(c.muted_until=-1 OR c.muted_until>?)",
" ORDER BY m.timestamp DESC,m.id DESC;"
),
"SELECT m.id
FROM msgs m
LEFT JOIN contacts ct
ON m.from_id=ct.id
LEFT JOIN chats c
ON m.chat_id=c.id
WHERE m.state=?
AND m.hidden=0
AND m.chat_id>9
AND ct.blocked=0
AND c.blocked=0
AND NOT(c.muted_until=-1 OR c.muted_until>?)
ORDER BY m.timestamp DESC,m.id DESC",
(MessageState::InFresh, time()),
|row| {
let msg_id: MsgId = row.get(0)?;
@@ -1334,10 +1338,5 @@ impl Context {
}
}
/// Returns core version as a string.
pub fn get_version_str() -> &'static str {
&DC_VERSION_STR
}
#[cfg(test)]
mod context_tests;
+1
View File
@@ -297,6 +297,7 @@ async fn test_get_info_completeness() {
"encrypted_device_token",
"stats_last_update",
"stats_last_old_contact_id",
"simulate_receive_imf_error", // only used in tests
];
let t = TestContext::new().await;
let info = t.get_info().await.unwrap();
+169 -331
View File
@@ -1,27 +1,19 @@
//! # Download large messages manually.
use std::cmp::max;
use std::collections::BTreeMap;
use anyhow::{Result, anyhow, bail, ensure};
use deltachat_derive::{FromSql, ToSql};
use serde::{Deserialize, Serialize};
use crate::config::Config;
use crate::context::Context;
use crate::imap::session::Session;
use crate::message::{Message, MsgId, Viewtype};
use crate::mimeparser::{MimeMessage, Part};
use crate::tools::time;
use crate::{EventType, chatlist_events, stock_str};
use crate::log::warn;
use crate::message::{self, Message, MsgId, rfc724_mid_exists};
use crate::{EventType, chatlist_events};
/// Download limits should not be used below `MIN_DOWNLOAD_LIMIT`.
///
/// For better UX, some messages as add-member, non-delivery-reports (NDN) or read-receipts (MDN)
/// should always be downloaded completely to handle them correctly,
/// also in larger groups and if group and contact avatar are attached.
/// Most of these cases are caught by `MIN_DOWNLOAD_LIMIT`.
pub(crate) const MIN_DOWNLOAD_LIMIT: u32 = 163840;
pub(crate) mod post_msg_metadata;
pub(crate) use post_msg_metadata::PostMsgMetadata;
/// If a message is downloaded only partially
/// and `delete_server_after` is set to small timeouts (eg. "at once"),
@@ -29,6 +21,16 @@ pub(crate) const MIN_DOWNLOAD_LIMIT: u32 = 163840;
/// `MIN_DELETE_SERVER_AFTER` increases the timeout in this case.
pub(crate) const MIN_DELETE_SERVER_AFTER: i64 = 48 * 60 * 60;
/// From this point onward outgoing messages are considered large
/// and get a Pre-Message, which announces the Post-Message.
/// This is only about sending so we can modify it any time.
/// Current value is a bit less than the minimum auto-download setting from the UIs (which is 160
/// KiB).
pub(crate) const PRE_MSG_ATTACHMENT_SIZE_THRESHOLD: u64 = 140_000;
/// Max size for pre messages. A warning is emitted when this is exceeded.
pub(crate) const PRE_MSG_SIZE_WARNING_THRESHOLD: usize = 150_000;
/// Download state of the message.
#[derive(
Debug,
@@ -64,20 +66,8 @@ pub enum DownloadState {
InProgress = 1000,
}
impl Context {
// Returns validated download limit or `None` for "no limit".
pub(crate) async fn download_limit(&self) -> Result<Option<u32>> {
let download_limit = self.get_config_int(Config::DownloadLimit).await?;
if download_limit <= 0 {
Ok(None)
} else {
Ok(Some(max(MIN_DOWNLOAD_LIMIT, download_limit as u32)))
}
}
}
impl MsgId {
/// Schedules full message download for partially downloaded message.
/// Schedules Post-Message download for partially downloaded message.
pub async fn download_full(self, context: &Context) -> Result<()> {
let msg = Message::load_from_db(context, self).await?;
match msg.download_state() {
@@ -86,11 +76,22 @@ impl MsgId {
}
DownloadState::InProgress => return Err(anyhow!("Download already in progress.")),
DownloadState::Available | DownloadState::Failure => {
if msg.rfc724_mid().is_empty() {
return Err(anyhow!("Download not possible, message has no rfc724_mid"));
}
self.update_download_state(context, DownloadState::InProgress)
.await?;
info!(
context,
"Requesting full download of {:?}.",
msg.rfc724_mid()
);
context
.sql
.execute("INSERT INTO download (msg_id) VALUES (?)", (self,))
.execute(
"INSERT INTO download (rfc724_mid, msg_id) VALUES (?,?)",
(msg.rfc724_mid(), msg.id),
)
.await?;
context.scheduler.interrupt_inbox().await;
}
@@ -139,20 +140,9 @@ impl Message {
/// Most messages are downloaded automatically on fetch instead.
pub(crate) async fn download_msg(
context: &Context,
msg_id: MsgId,
rfc724_mid: String,
session: &mut Session,
) -> Result<()> {
let Some(msg) = Message::load_from_db_optional(context, msg_id).await? else {
// If partially downloaded message was already deleted
// we do not know its Message-ID anymore
// so cannot download it.
//
// Probably the message expired due to `delete_device_after`
// setting or was otherwise removed from the device,
// so we don't want it to reappear anyway.
return Ok(());
};
let transport_id = session.transport_id();
let row = context
.sql
@@ -161,7 +151,7 @@ pub(crate) async fn download_msg(
WHERE rfc724_mid=?
AND transport_id=?
AND target!=''",
(&msg.rfc724_mid, transport_id),
(&rfc724_mid, transport_id),
|row| {
let server_uid: u32 = row.get(0)?;
let server_folder: String = row.get(1)?;
@@ -172,11 +162,13 @@ pub(crate) async fn download_msg(
let Some((server_uid, server_folder)) = row else {
// No IMAP record found, we don't know the UID and folder.
return Err(anyhow!("Call download_full() again to try over."));
return Err(anyhow!(
"IMAP location for {rfc724_mid:?} post-message is unknown"
));
};
session
.fetch_single_msg(context, &server_folder, server_uid, msg.rfc724_mid.clone())
.fetch_single_msg(context, &server_folder, server_uid, rfc724_mid)
.await?;
Ok(())
}
@@ -209,7 +201,7 @@ impl Session {
let mut uid_message_ids: BTreeMap<u32, String> = BTreeMap::new();
uid_message_ids.insert(uid, rfc724_mid);
let (sender, receiver) = async_channel::unbounded();
self.fetch_many_msgs(context, folder, vec![uid], &uid_message_ids, false, sender)
self.fetch_many_msgs(context, folder, vec![uid], &uid_message_ids, sender)
.await?;
if receiver.recv().await.is_err() {
bail!("Failed to fetch UID {uid}");
@@ -218,41 +210,139 @@ impl Session {
}
}
impl MimeMessage {
/// Creates a placeholder part and add that to `parts`.
///
/// To create the placeholder, only the outermost header can be used,
/// the mime-structure itself is not available.
///
/// The placeholder part currently contains a text with size and availability of the message.
pub(crate) async fn create_stub_from_partial_download(
&mut self,
context: &Context,
org_bytes: u32,
) -> Result<()> {
let mut text = format!(
"[{}]",
stock_str::partial_download_msg_body(context, org_bytes).await
);
if let Some(delete_server_after) = context.get_config_delete_server_after().await? {
let until = stock_str::download_availability(
context,
time() + max(delete_server_after, MIN_DELETE_SERVER_AFTER),
)
.await;
text += format!(" [{until}]").as_str();
};
info!(context, "Partial download: {}", text);
self.do_add_single_part(Part {
typ: Viewtype::Text,
msg: text,
..Default::default()
});
Ok(())
async fn set_state_to_failure(context: &Context, rfc724_mid: &str) -> Result<()> {
if let Some(msg_id) = rfc724_mid_exists(context, rfc724_mid).await? {
// Update download state to failure
// so it can be retried.
//
// On success update_download_state() is not needed
// as receive_imf() already
// set the state and emitted the event.
msg_id
.update_download_state(context, DownloadState::Failure)
.await?;
}
Ok(())
}
async fn available_post_msgs_contains_rfc724_mid(
context: &Context,
rfc724_mid: &str,
) -> Result<bool> {
Ok(context
.sql
.query_get_value::<String>(
"SELECT rfc724_mid FROM available_post_msgs WHERE rfc724_mid=?",
(&rfc724_mid,),
)
.await?
.is_some())
}
async fn delete_from_available_post_msgs(context: &Context, rfc724_mid: &str) -> Result<()> {
context
.sql
.execute(
"DELETE FROM available_post_msgs WHERE rfc724_mid=?",
(&rfc724_mid,),
)
.await?;
Ok(())
}
async fn delete_from_downloads(context: &Context, rfc724_mid: &str) -> Result<()> {
context
.sql
.execute("DELETE FROM download WHERE rfc724_mid=?", (&rfc724_mid,))
.await?;
Ok(())
}
pub(crate) async fn msg_is_downloaded_for(context: &Context, rfc724_mid: &str) -> Result<bool> {
Ok(message::rfc724_mid_exists(context, rfc724_mid)
.await?
.is_some())
}
pub(crate) async fn download_msgs(context: &Context, session: &mut Session) -> Result<()> {
let rfc724_mids = context
.sql
.query_map_vec("SELECT rfc724_mid FROM download", (), |row| {
let rfc724_mid: String = row.get(0)?;
Ok(rfc724_mid)
})
.await?;
for rfc724_mid in &rfc724_mids {
let res = download_msg(context, rfc724_mid.clone(), session).await;
if res.is_ok() {
delete_from_downloads(context, rfc724_mid).await?;
delete_from_available_post_msgs(context, rfc724_mid).await?;
}
if let Err(err) = res {
warn!(
context,
"Failed to download message rfc724_mid={rfc724_mid}: {:#}.", err
);
if !msg_is_downloaded_for(context, rfc724_mid).await? {
// This is probably a classical email that vanished before we could download it
warn!(
context,
"{rfc724_mid} download failed and there is no downloaded pre-message."
);
delete_from_downloads(context, rfc724_mid).await?;
} else if available_post_msgs_contains_rfc724_mid(context, rfc724_mid).await? {
warn!(
context,
"{rfc724_mid} is in available_post_msgs table but we failed to fetch it,
so set the message to DownloadState::Failure - probably it was deleted on the server in the meantime"
);
set_state_to_failure(context, rfc724_mid).await?;
delete_from_downloads(context, rfc724_mid).await?;
delete_from_available_post_msgs(context, rfc724_mid).await?;
} else {
// leave the message in DownloadState::InProgress;
// it will be downloaded once it arrives.
}
}
}
Ok(())
}
/// Downloads known post-messages without pre-messages
/// in order to guard against lost pre-messages.
pub(crate) async fn download_known_post_messages_without_pre_message(
context: &Context,
session: &mut Session,
) -> Result<()> {
let rfc724_mids = context
.sql
.query_map_vec("SELECT rfc724_mid FROM available_post_msgs", (), |row| {
let rfc724_mid: String = row.get(0)?;
Ok(rfc724_mid)
})
.await?;
for rfc724_mid in &rfc724_mids {
if !msg_is_downloaded_for(context, rfc724_mid).await? {
// Download the Post-Message unconditionally,
// because the Pre-Message got lost.
// The message may be in the wrong order,
// but at least we have it at all.
let res = download_msg(context, rfc724_mid.clone(), session).await;
if res.is_ok() {
delete_from_available_post_msgs(context, rfc724_mid).await?;
}
if let Err(err) = res {
warn!(
context,
"download_known_post_messages_without_pre_message: Failed to download message rfc724_mid={rfc724_mid}: {:#}.",
err
);
}
}
}
Ok(())
}
#[cfg(test)]
@@ -260,11 +350,8 @@ mod tests {
use num_traits::FromPrimitive;
use super::*;
use crate::chat::{get_chat_msgs, send_msg};
use crate::ephemeral::Timer;
use crate::message::delete_msgs;
use crate::receive_imf::receive_imf_from_inbox;
use crate::test_utils::{E2EE_INFO_MSGS, TestContext, TestContextManager};
use crate::chat::send_msg;
use crate::test_utils::TestContext;
#[test]
fn test_downloadstate_values() {
@@ -282,29 +369,6 @@ mod tests {
);
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_download_limit() -> Result<()> {
let t = TestContext::new_alice().await;
assert_eq!(t.download_limit().await?, None);
t.set_config(Config::DownloadLimit, Some("200000")).await?;
assert_eq!(t.download_limit().await?, Some(200000));
t.set_config(Config::DownloadLimit, Some("20000")).await?;
assert_eq!(t.download_limit().await?, Some(MIN_DOWNLOAD_LIMIT));
t.set_config(Config::DownloadLimit, None).await?;
assert_eq!(t.download_limit().await?, None);
for val in &["0", "-1", "-100", "", "foo"] {
t.set_config(Config::DownloadLimit, Some(val)).await?;
assert_eq!(t.download_limit().await?, None);
}
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_update_download_state() -> Result<()> {
let t = TestContext::new_alice().await;
@@ -336,230 +400,4 @@ mod tests {
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_partial_receive_imf() -> Result<()> {
let t = TestContext::new_alice().await;
let header = "Received: (Postfix, from userid 1000); Mon, 4 Dec 2006 14:51:39 +0100 (CET)\n\
From: bob@example.com\n\
To: alice@example.org\n\
Subject: foo\n\
Message-ID: <Mr.12345678901@example.com>\n\
Chat-Version: 1.0\n\
Date: Sun, 22 Mar 2020 22:37:57 +0000\
Content-Type: text/plain";
receive_imf_from_inbox(
&t,
"Mr.12345678901@example.com",
header.as_bytes(),
false,
Some(100000),
)
.await?;
let msg = t.get_last_msg().await;
assert_eq!(msg.download_state(), DownloadState::Available);
assert_eq!(msg.get_subject(), "foo");
assert!(
msg.get_text()
.contains(&stock_str::partial_download_msg_body(&t, 100000).await)
);
receive_imf_from_inbox(
&t,
"Mr.12345678901@example.com",
format!("{header}\n\n100k text...").as_bytes(),
false,
None,
)
.await?;
let msg = t.get_last_msg().await;
assert_eq!(msg.download_state(), DownloadState::Done);
assert_eq!(msg.get_subject(), "foo");
assert_eq!(msg.get_text(), "100k text...");
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_partial_download_and_ephemeral() -> Result<()> {
let t = TestContext::new_alice().await;
let chat_id = t
.create_chat_with_contact("bob", "bob@example.org")
.await
.id;
chat_id
.set_ephemeral_timer(&t, Timer::Enabled { duration: 60 })
.await?;
// download message from bob partially, this must not change the ephemeral timer
receive_imf_from_inbox(
&t,
"first@example.org",
b"From: Bob <bob@example.org>\n\
To: Alice <alice@example.org>\n\
Chat-Version: 1.0\n\
Subject: subject\n\
Message-ID: <first@example.org>\n\
Date: Sun, 14 Nov 2021 00:10:00 +0000\
Content-Type: text/plain",
false,
Some(100000),
)
.await?;
assert_eq!(
chat_id.get_ephemeral_timer(&t).await?,
Timer::Enabled { duration: 60 }
);
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_status_update_expands_to_nothing() -> Result<()> {
let alice = TestContext::new_alice().await;
let bob = TestContext::new_bob().await;
let chat_id = alice.create_chat(&bob).await.id;
let file = alice.get_blobdir().join("minimal.xdc");
tokio::fs::write(&file, include_bytes!("../test-data/webxdc/minimal.xdc")).await?;
let mut instance = Message::new(Viewtype::File);
instance.set_file_and_deduplicate(&alice, &file, None, None)?;
let _sent1 = alice.send_msg(chat_id, &mut instance).await;
alice
.send_webxdc_status_update(instance.id, r#"{"payload":7}"#)
.await?;
alice.flush_status_updates().await?;
let sent2 = alice.pop_sent_msg().await;
let sent2_rfc724_mid = sent2.load_from_db().await.rfc724_mid;
// not downloading the status update results in an placeholder
receive_imf_from_inbox(
&bob,
&sent2_rfc724_mid,
sent2.payload().as_bytes(),
false,
Some(sent2.payload().len() as u32),
)
.await?;
let msg = bob.get_last_msg().await;
let chat_id = msg.chat_id;
assert_eq!(
get_chat_msgs(&bob, chat_id).await?.len(),
E2EE_INFO_MSGS + 1
);
assert_eq!(msg.download_state(), DownloadState::Available);
// downloading the status update afterwards expands to nothing and moves the placeholder to trash-chat
// (usually status updates are too small for not being downloaded directly)
receive_imf_from_inbox(
&bob,
&sent2_rfc724_mid,
sent2.payload().as_bytes(),
false,
None,
)
.await?;
assert_eq!(get_chat_msgs(&bob, chat_id).await?.len(), E2EE_INFO_MSGS);
assert!(
Message::load_from_db_optional(&bob, msg.id)
.await?
.is_none()
);
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_mdn_expands_to_nothing() -> Result<()> {
let bob = TestContext::new_bob().await;
let raw = b"Subject: Message opened\n\
Date: Mon, 10 Jan 2020 00:00:00 +0000\n\
Chat-Version: 1.0\n\
Message-ID: <bar@example.org>\n\
To: Alice <alice@example.org>\n\
From: Bob <bob@example.org>\n\
Content-Type: multipart/report; report-type=disposition-notification;\n\t\
boundary=\"kJBbU58X1xeWNHgBtTbMk80M5qnV4N\"\n\
\n\
\n\
--kJBbU58X1xeWNHgBtTbMk80M5qnV4N\n\
Content-Type: text/plain; charset=utf-8\n\
\n\
bla\n\
\n\
\n\
--kJBbU58X1xeWNHgBtTbMk80M5qnV4N\n\
Content-Type: message/disposition-notification\n\
\n\
Reporting-UA: Delta Chat 1.88.0\n\
Original-Recipient: rfc822;bob@example.org\n\
Final-Recipient: rfc822;bob@example.org\n\
Original-Message-ID: <foo@example.org>\n\
Disposition: manual-action/MDN-sent-automatically; displayed\n\
\n\
\n\
--kJBbU58X1xeWNHgBtTbMk80M5qnV4N--\n\
";
// not downloading the mdn results in an placeholder
receive_imf_from_inbox(&bob, "bar@example.org", raw, false, Some(raw.len() as u32)).await?;
let msg = bob.get_last_msg().await;
let chat_id = msg.chat_id;
assert_eq!(get_chat_msgs(&bob, chat_id).await?.len(), 1);
assert_eq!(msg.download_state(), DownloadState::Available);
// downloading the mdn afterwards expands to nothing and deletes the placeholder directly
// (usually mdn are too small for not being downloaded directly)
receive_imf_from_inbox(&bob, "bar@example.org", raw, false, None).await?;
assert_eq!(get_chat_msgs(&bob, chat_id).await?.len(), 0);
assert!(
Message::load_from_db_optional(&bob, msg.id)
.await?
.is_none()
);
Ok(())
}
/// Tests that fully downloading the message
/// works even if the Message-ID already exists
/// in the database assigned to the trash chat.
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_partial_download_trashed() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
let imf_raw = b"From: Bob <bob@example.org>\n\
To: Alice <alice@example.org>\n\
Chat-Version: 1.0\n\
Subject: subject\n\
Message-ID: <first@example.org>\n\
Date: Sun, 14 Nov 2021 00:10:00 +0000\
Content-Type: text/plain";
// Download message from Bob partially.
let partial_received_msg =
receive_imf_from_inbox(alice, "first@example.org", imf_raw, false, Some(100000))
.await?
.unwrap();
assert_eq!(partial_received_msg.msg_ids.len(), 1);
// Delete the received message.
// Not it is still in the database,
// but in the trash chat.
delete_msgs(alice, &[partial_received_msg.msg_ids[0]]).await?;
// Fully download message after deletion.
let full_received_msg =
receive_imf_from_inbox(alice, "first@example.org", imf_raw, false, None).await?;
// The message does not reappear.
// However, `receive_imf` should not fail.
assert!(full_received_msg.is_none());
Ok(())
}
}
+251
View File
@@ -0,0 +1,251 @@
use anyhow::{Context as _, Result};
use num_traits::ToPrimitive;
use serde::{Deserialize, Serialize};
use crate::context::Context;
use crate::log::warn;
use crate::message::Message;
use crate::message::Viewtype;
use crate::param::{Param, Params};
/// Metadata contained in Pre-Message that describes the Post-Message.
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
pub struct PostMsgMetadata {
/// size of the attachment in bytes
pub(crate) size: u64,
/// Real viewtype of message
pub(crate) viewtype: Viewtype,
/// the original file name
pub(crate) filename: String,
/// Width and height of the image or video
#[serde(skip_serializing_if = "Option::is_none")]
pub(crate) wh: Option<(i32, i32)>,
/// Duration of audio file or video in milliseconds
#[serde(skip_serializing_if = "Option::is_none")]
pub(crate) duration: Option<i32>,
}
impl PostMsgMetadata {
/// Returns `PostMsgMetadata` for messages with file attachment and `None` otherwise.
pub(crate) async fn from_msg(context: &Context, message: &Message) -> Result<Option<Self>> {
if !message.viewtype.has_file() {
return Ok(None);
}
let size = message
.get_filebytes(context)
.await?
.context("Unexpected: file has no size")?;
let filename = message
.param
.get(Param::Filename)
.unwrap_or_default()
.to_owned();
let wh = {
match (
message.param.get_int(Param::Width),
message.param.get_int(Param::Height),
) {
(None, None) => None,
(Some(width), Some(height)) => Some((width, height)),
wh => {
warn!(
context,
"Message {} misses width or height: {:?}.", message.id, wh
);
None
}
}
};
let duration = message.param.get_int(Param::Duration);
Ok(Some(Self {
size,
filename,
viewtype: message.viewtype,
wh,
duration,
}))
}
pub(crate) fn to_header_value(&self) -> Result<String> {
Ok(serde_json::to_string(&self)?)
}
pub(crate) fn try_from_header_value(value: &str) -> Result<Self> {
Ok(serde_json::from_str(value)?)
}
}
impl Params {
/// Applies data from post_msg_metadata to Params
pub(crate) fn apply_post_msg_metadata(
&mut self,
post_msg_metadata: &PostMsgMetadata,
) -> &mut Self {
self.set(Param::PostMessageFileBytes, post_msg_metadata.size);
if !post_msg_metadata.filename.is_empty() {
self.set(Param::Filename, &post_msg_metadata.filename);
}
self.set_i64(
Param::PostMessageViewtype,
post_msg_metadata.viewtype.to_i64().unwrap_or_default(),
);
if let Some((width, height)) = post_msg_metadata.wh {
self.set(Param::Width, width);
self.set(Param::Height, height);
}
if let Some(duration) = post_msg_metadata.duration {
self.set(Param::Duration, duration);
}
self
}
}
#[cfg(test)]
mod tests {
use anyhow::Result;
use pretty_assertions::assert_eq;
use crate::{
message::{Message, Viewtype},
test_utils::{TestContextManager, create_test_image},
};
use super::PostMsgMetadata;
/// Build from message with file attachment
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_build_from_file_msg() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
let mut file_msg = Message::new(Viewtype::File);
file_msg.set_file_from_bytes(alice, "test.bin", &vec![0u8; 1_000_000], None)?;
let post_msg_metadata = PostMsgMetadata::from_msg(alice, &file_msg).await?;
assert_eq!(
post_msg_metadata,
Some(PostMsgMetadata {
size: 1_000_000,
viewtype: Viewtype::File,
filename: "test.bin".to_string(),
wh: None,
duration: None,
})
);
Ok(())
}
/// Build from message with image attachment
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_build_from_image_msg() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
let mut image_msg = Message::new(Viewtype::Image);
let (width, height) = (1080, 1920);
let test_img = create_test_image(width, height)?;
image_msg.set_file_from_bytes(alice, "vacation.png", &test_img, None)?;
// this is usually done while sending,
// but we don't send it here, so we need to call it ourself
image_msg.try_calc_and_set_dimensions(alice).await?;
let post_msg_metadata = PostMsgMetadata::from_msg(alice, &image_msg).await?;
assert_eq!(
post_msg_metadata,
Some(PostMsgMetadata {
size: 1816098,
viewtype: Viewtype::Image,
filename: "vacation.png".to_string(),
wh: Some((width as i32, height as i32)),
duration: None,
})
);
Ok(())
}
/// Test that serialisation results in expected format
#[test]
fn test_serialize_to_header() -> Result<()> {
assert_eq!(
PostMsgMetadata {
size: 1_000_000,
viewtype: Viewtype::File,
filename: "test.bin".to_string(),
wh: None,
duration: None,
}
.to_header_value()?,
"{\"size\":1000000,\"viewtype\":\"File\",\"filename\":\"test.bin\"}"
);
assert_eq!(
PostMsgMetadata {
size: 5_342_765,
viewtype: Viewtype::Image,
filename: "vacation.png".to_string(),
wh: Some((1080, 1920)),
duration: None,
}
.to_header_value()?,
"{\"size\":5342765,\"viewtype\":\"Image\",\"filename\":\"vacation.png\",\"wh\":[1080,1920]}"
);
assert_eq!(
PostMsgMetadata {
size: 5_000,
viewtype: Viewtype::Audio,
filename: "audio-DD-MM-YY.ogg".to_string(),
wh: None,
duration: Some(152_310),
}
.to_header_value()?,
"{\"size\":5000,\"viewtype\":\"Audio\",\"filename\":\"audio-DD-MM-YY.ogg\",\"duration\":152310}"
);
Ok(())
}
/// Test that deserialisation from expected format works
/// This test will become important for compatibility between versions in the future
#[test]
fn test_deserialize_from_header() -> Result<()> {
assert_eq!(
serde_json::from_str::<PostMsgMetadata>(
"{\"size\":1000000,\"viewtype\":\"File\",\"filename\":\"test.bin\",\"wh\":null,\"duration\":null}"
)?,
PostMsgMetadata {
size: 1_000_000,
viewtype: Viewtype::File,
filename: "test.bin".to_string(),
wh: None,
duration: None,
}
);
assert_eq!(
serde_json::from_str::<PostMsgMetadata>(
"{\"size\":5342765,\"viewtype\":\"Image\",\"filename\":\"vacation.png\",\"wh\":[1080,1920]}"
)?,
PostMsgMetadata {
size: 5_342_765,
viewtype: Viewtype::Image,
filename: "vacation.png".to_string(),
wh: Some((1080, 1920)),
duration: None,
}
);
assert_eq!(
serde_json::from_str::<PostMsgMetadata>(
"{\"size\":5000,\"viewtype\":\"Audio\",\"filename\":\"audio-DD-MM-YY.ogg\",\"duration\":152310}"
)?,
PostMsgMetadata {
size: 5_000,
viewtype: Viewtype::Audio,
filename: "audio-DD-MM-YY.ogg".to_string(),
wh: None,
duration: Some(152_310),
}
);
Ok(())
}
}
+4 -2
View File
@@ -474,8 +474,10 @@ pub(crate) async fn delete_expired_messages(context: &Context, now: i64) -> Resu
// If you change which information is preserved here, also change `MsgId::trash()`
// and other places it references.
let mut del_msg_stmt = transaction.prepare(
"INSERT OR REPLACE INTO msgs (id, rfc724_mid, timestamp, chat_id)
SELECT ?1, rfc724_mid, timestamp, ? FROM msgs WHERE id=?1",
"
INSERT OR REPLACE INTO msgs (id, rfc724_mid, pre_rfc724_mid, timestamp, chat_id)
SELECT ?1, rfc724_mid, pre_rfc724_mid, timestamp, ? FROM msgs WHERE id=?1
",
)?;
let mut del_location_stmt =
transaction.prepare("DELETE FROM locations WHERE independent=1 AND id=?")?;
+6 -6
View File
@@ -417,13 +417,13 @@ pub enum EventType {
chat_id: ChatId,
},
/// One or more transports has changed.
/// One or more transports has changed or another transport is primary now.
///
/// This event is used for tests to detect when transport
/// synchronization messages arrives.
/// UIs don't need to use it, it is unlikely
/// that user modifies transports on multiple
/// devices simultaneously.
/// UI should update the list.
///
/// This event is emitted when a transport
/// synchronization message modifies transports,
/// but not when the UI modifies the transport list by itself.
TransportsModified,
/// Event for using in tests, e.g. as a fence between normally generated events.
+18
View File
@@ -102,6 +102,21 @@ pub enum HeaderDef {
/// used to encrypt and decrypt messages.
/// This secret is sent to a new member in the member-addition message.
ChatBroadcastSecret,
/// A message with a large attachment is split into two messages:
/// A pre-message, which contains everything but the attachment,
/// and a Post-Message.
/// The Pre-Message gets a `Chat-Post-Message-Id` header
/// referencing the Post-Message's rfc724_mid.
ChatPostMessageId,
/// Announces Post-Message metadata in a Pre-Message.
/// Contains a serialized `PostMsgMetadata` struct.
ChatPostMessageMetadata,
/// This message is preceded by a Pre-Message
/// and thus this message can be skipped while fetching messages.
/// This is an unprotected header.
ChatIsPostMessage,
/// [Autocrypt](https://autocrypt.org/) header.
Autocrypt,
@@ -147,6 +162,9 @@ pub enum HeaderDef {
impl HeaderDef {
/// Returns the corresponding header string.
///
/// Format is lower-kebab-case for easy comparisons.
/// This method is used in message receiving and testing.
pub fn get_headername(&self) -> &'static str {
self.into()
}
+100 -162
View File
@@ -23,11 +23,13 @@ use num_traits::FromPrimitive;
use ratelimit::Ratelimit;
use url::Url;
use crate::calls::{create_fallback_ice_servers, create_ice_servers_from_metadata};
use crate::calls::{
UnresolvedIceServer, create_fallback_ice_servers, create_ice_servers_from_metadata,
};
use crate::chat::{self, ChatId, ChatIdBlocked, add_device_msg};
use crate::chatlist_events;
use crate::config::Config;
use crate::constants::{self, Blocked, Chattype, ShowEmails};
use crate::constants::{self, Blocked, DC_VERSION_STR, ShowEmails};
use crate::contact::{Contact, ContactId, Modifier, Origin};
use crate::context::Context;
use crate::events::EventType;
@@ -67,7 +69,6 @@ const RFC724MID_UID: &str = "(UID BODY.PEEK[HEADER.FIELDS (\
X-MICROSOFT-ORIGINAL-MESSAGE-ID\
)])";
const BODY_FULL: &str = "(FLAGS BODY.PEEK[])";
const BODY_PARTIAL: &str = "(FLAGS RFC822.SIZE BODY.PEEK[HEADER])";
#[derive(Debug)]
pub(crate) struct Imap {
@@ -135,16 +136,15 @@ pub(crate) struct ServerMetadata {
pub iroh_relay: Option<Url>,
/// JSON with ICE servers for WebRTC calls
/// and the expiration timestamp.
///
/// If JSON is about to expire, new TURN credentials
/// should be fetched from the server
/// to be ready for WebRTC calls.
pub ice_servers: String,
/// ICE servers for WebRTC calls.
pub ice_servers: Vec<UnresolvedIceServer>,
/// Timestamp when ICE servers are considered
/// expired and should be updated.
///
/// If ICE servers are about to expire, new TURN credentials
/// should be fetched from the server
/// to be ready for WebRTC calls.
pub ice_servers_expiration_timestamp: i64,
}
@@ -615,12 +615,18 @@ impl Imap {
.context("prefetch")?;
let read_cnt = msgs.len();
let download_limit = context.download_limit().await?;
let mut uids_fetch = Vec::<(u32, bool /* partially? */)>::with_capacity(msgs.len() + 1);
let mut uids_fetch: Vec<u32> = Vec::new();
let mut available_post_msgs: Vec<String> = Vec::new();
let mut download_later: Vec<String> = Vec::new();
let mut uid_message_ids = BTreeMap::new();
let mut largest_uid_skipped = None;
let delete_target = context.get_delete_msgs_target().await?;
let download_limit: Option<u32> = context
.get_config_parsed(Config::DownloadLimit)
.await?
.filter(|&l| 0 < l);
// Store the info about IMAP messages in the database.
for (uid, ref fetch_response) in msgs {
let headers = match get_fetch_headers(fetch_response) {
@@ -632,6 +638,9 @@ impl Imap {
};
let message_id = prefetch_get_message_id(&headers);
let size = fetch_response
.size
.context("imap fetch response does not contain size")?;
// Determine the target folder where the message should be moved to.
//
@@ -706,14 +715,27 @@ impl Imap {
)
.await.context("prefetch_should_download")?
{
match download_limit {
Some(download_limit) => uids_fetch.push((
uid,
fetch_response.size.unwrap_or_default() > download_limit,
)),
None => uids_fetch.push((uid, false)),
}
uid_message_ids.insert(uid, message_id);
if headers
.get_header_value(HeaderDef::ChatIsPostMessage)
.is_some()
{
info!(context, "{message_id:?} is a post-message.");
available_post_msgs.push(message_id.clone());
if download_limit.is_none_or(|download_limit| size <= download_limit) {
download_later.push(message_id.clone());
}
largest_uid_skipped = Some(uid);
} else {
info!(context, "{message_id:?} is not a post-message.");
if download_limit.is_none_or(|download_limit| size <= download_limit) {
uids_fetch.push(uid);
uid_message_ids.insert(uid, message_id);
} else {
download_later.push(message_id.clone());
largest_uid_skipped = Some(uid);
}
};
} else {
largest_uid_skipped = Some(uid);
}
@@ -747,29 +769,10 @@ impl Imap {
};
let actually_download_messages_future = async {
let sender = sender;
let mut uids_fetch_in_batch = Vec::with_capacity(max(uids_fetch.len(), 1));
let mut fetch_partially = false;
uids_fetch.push((0, !uids_fetch.last().unwrap_or(&(0, false)).1));
for (uid, fp) in uids_fetch {
if fp != fetch_partially {
session
.fetch_many_msgs(
context,
folder,
uids_fetch_in_batch.split_off(0),
&uid_message_ids,
fetch_partially,
sender.clone(),
)
.await
.context("fetch_many_msgs")?;
fetch_partially = fp;
}
uids_fetch_in_batch.push(uid);
}
anyhow::Ok(())
session
.fetch_many_msgs(context, folder, uids_fetch, &uid_message_ids, sender)
.await
.context("fetch_many_msgs")
};
let (largest_uid_fetched, fetch_res) =
@@ -804,6 +807,30 @@ impl Imap {
chat::mark_old_messages_as_noticed(context, received_msgs).await?;
if fetch_res.is_ok() {
info!(
context,
"available_post_msgs: {}, download_later: {}.",
available_post_msgs.len(),
download_later.len(),
);
let trans_fn = |t: &mut rusqlite::Transaction| {
let mut stmt = t.prepare("INSERT OR IGNORE INTO available_post_msgs VALUES (?)")?;
for rfc724_mid in available_post_msgs {
stmt.execute((rfc724_mid,))
.context("INSERT OR IGNORE INTO available_post_msgs")?;
}
let mut stmt =
t.prepare("INSERT OR IGNORE INTO download (rfc724_mid, msg_id) VALUES (?,0)")?;
for rfc724_mid in download_later {
stmt.execute((rfc724_mid,))
.context("INSERT OR IGNORE INTO download")?;
}
Ok(())
};
context.sql.transaction(trans_fn).await?;
}
// Now fail if fetching failed, so we will
// establish a new session if this one is broken.
fetch_res?;
@@ -1110,61 +1137,22 @@ impl Session {
Ok(())
}
/// Uploads sync messages from the `imap_send` table with `\Seen` flag set.
pub(crate) async fn send_sync_msgs(&mut self, context: &Context, folder: &str) -> Result<()> {
context.send_sync_msg().await?;
while let Some((id, mime, msg_id, attempts)) = context
.sql
.query_row_optional(
"SELECT id, mime, msg_id, attempts FROM imap_send ORDER BY id LIMIT 1",
(),
|row| {
let id: i64 = row.get(0)?;
let mime: String = row.get(1)?;
let msg_id: MsgId = row.get(2)?;
let attempts: i64 = row.get(3)?;
Ok((id, mime, msg_id, attempts))
},
)
.await
.context("Failed to SELECT from imap_send")?
{
let res = self
.append(folder, Some("(\\Seen)"), None, mime)
.await
.with_context(|| format!("IMAP APPEND to {folder} failed for {msg_id}"))
.log_err(context);
if res.is_ok() {
msg_id.set_delivered(context).await?;
}
const MAX_ATTEMPTS: i64 = 2;
if res.is_ok() || attempts >= MAX_ATTEMPTS - 1 {
context
.sql
.execute("DELETE FROM imap_send WHERE id=?", (id,))
.await
.context("Failed to delete from imap_send")?;
} else {
context
.sql
.execute("UPDATE imap_send SET attempts=attempts+1 WHERE id=?", (id,))
.await
.context("Failed to update imap_send.attempts")?;
res?;
}
}
Ok(())
}
/// Stores pending `\Seen` flags for messages in `imap_markseen` table.
pub(crate) async fn store_seen_flags_on_imap(&mut self, context: &Context) -> Result<()> {
if context.get_config_bool(Config::TeamProfile).await? {
return Ok(());
}
let transport_id = self.transport_id();
let rows = context
.sql
.query_map_vec(
"SELECT imap.id, uid, folder FROM imap, imap_markseen
WHERE imap.id = imap_markseen.id AND target = folder
WHERE imap.id = imap_markseen.id
AND imap.transport_id=?
AND target = folder
ORDER BY folder, uid",
[],
(transport_id,),
|row| {
let rowid: i64 = row.get(0)?;
let uid: u32 = row.get(1)?;
@@ -1226,6 +1214,10 @@ impl Session {
return Ok(());
}
if context.get_config_bool(Config::TeamProfile).await? {
return Ok(());
}
let create = false;
let folder_exists = self
.select_with_uidvalidity(context, folder, create)
@@ -1375,7 +1367,6 @@ impl Session {
folder: &str,
request_uids: Vec<u32>,
uid_message_ids: &BTreeMap<u32, String>,
fetch_partially: bool,
received_msgs_channel: Sender<(u32, Option<ReceivedMsg>)>,
) -> Result<()> {
if request_uids.is_empty() {
@@ -1383,25 +1374,10 @@ impl Session {
}
for (request_uids, set) in build_sequence_sets(&request_uids)? {
info!(
context,
"Starting a {} FETCH of message set \"{}\".",
if fetch_partially { "partial" } else { "full" },
set
);
let mut fetch_responses = self
.uid_fetch(
&set,
if fetch_partially {
BODY_PARTIAL
} else {
BODY_FULL
},
)
.await
.with_context(|| {
format!("fetching messages {} from folder \"{}\"", &set, folder)
})?;
info!(context, "Starting UID FETCH of message set \"{}\".", set);
let mut fetch_responses = self.uid_fetch(&set, BODY_FULL).await.with_context(|| {
format!("fetching messages {} from folder \"{}\"", &set, folder)
})?;
// Map from UIDs to unprocessed FETCH results. We put unprocessed FETCH results here
// when we want to process other messages first.
@@ -1458,11 +1434,7 @@ impl Session {
count += 1;
let is_deleted = fetch_response.flags().any(|flag| flag == Flag::Deleted);
let (body, partial) = if fetch_partially {
(fetch_response.header(), fetch_response.size) // `BODY.PEEK[HEADER]` goes to header() ...
} else {
(fetch_response.body(), None) // ... while `BODY.PEEK[]` goes to body() - and includes header()
};
let body = fetch_response.body();
if is_deleted {
info!(context, "Not processing deleted msg {}.", request_uid);
@@ -1496,13 +1468,13 @@ impl Session {
context,
"Passing message UID {} to receive_imf().", request_uid
);
let res = receive_imf_inner(context, rfc724_mid, body, is_seen, partial).await;
let res = receive_imf_inner(context, rfc724_mid, body, is_seen).await;
let received_msg = match res {
Err(err) => {
warn!(context, "receive_imf error: {err:#}.");
let text = format!(
"❌ Failed to receive a message: {err:#}. Please report this bug to delta@merlinux.eu or https://support.delta.chat/."
"❌ Failed to receive a message: {err:#}. Core version v{DC_VERSION_STR}. Please report this bug to delta@merlinux.eu or https://support.delta.chat/.",
);
let mut msg = Message::new_text(text);
add_device_msg(context, None, Some(&mut msg)).await?;
@@ -1579,7 +1551,7 @@ impl Session {
if m.entry == "/shared/vendor/deltachat/turn"
&& let Some(value) = m.value
{
match create_ice_servers_from_metadata(context, &value).await {
match create_ice_servers_from_metadata(&value).await {
Ok((parsed_timestamp, parsed_ice_servers)) => {
old_metadata.ice_servers_expiration_timestamp = parsed_timestamp;
old_metadata.ice_servers = parsed_ice_servers;
@@ -1596,7 +1568,7 @@ impl Session {
info!(context, "Will use fallback ICE servers.");
// Set expiration timestamp 7 days in the future so we don't request it again.
old_metadata.ice_servers_expiration_timestamp = time() + 3600 * 24 * 7;
old_metadata.ice_servers = create_fallback_ice_servers(context).await?;
old_metadata.ice_servers = create_fallback_ice_servers();
}
return Ok(());
}
@@ -1643,7 +1615,7 @@ impl Session {
}
"/shared/vendor/deltachat/turn" => {
if let Some(value) = m.value {
match create_ice_servers_from_metadata(context, &value).await {
match create_ice_servers_from_metadata(&value).await {
Ok((parsed_timestamp, parsed_ice_servers)) => {
ice_servers_expiration_timestamp = parsed_timestamp;
ice_servers = Some(parsed_ice_servers);
@@ -1662,7 +1634,7 @@ impl Session {
} else {
// Set expiration timestamp 7 days in the future so we don't request it again.
ice_servers_expiration_timestamp = time() + 3600 * 24 * 7;
create_fallback_ice_servers(context).await?
create_fallback_ice_servers()
};
*lock = Some(ServerMetadata {
@@ -2113,17 +2085,6 @@ async fn needs_move_to_mvbox(
headers: &[mailparse::MailHeader<'_>],
) -> Result<bool> {
let has_chat_version = headers.get_header_value(HeaderDef::ChatVersion).is_some();
if !context.get_config_bool(Config::IsChatmail).await?
&& has_chat_version
&& headers
.get_header_value(HeaderDef::AutoSubmitted)
.filter(|val| val.eq_ignore_ascii_case("auto-generated"))
.is_some()
&& let Some(from) = mimeparser::get_from(headers)
&& context.is_self_addr(&from.addr).await?
{
return Ok(true);
}
if !context.get_config_bool(Config::MvboxMove).await? {
return Ok(false);
}
@@ -2259,21 +2220,6 @@ pub(crate) fn create_message_id() -> String {
format!("{}{}", GENERATED_PREFIX, create_id())
}
/// Returns chat by prefetched headers.
async fn prefetch_get_chat(
context: &Context,
headers: &[mailparse::MailHeader<'_>],
) -> Result<Option<chat::Chat>> {
let parent = get_prefetch_parent_message(context, headers).await?;
if let Some(parent) = &parent {
return Ok(Some(
chat::Chat::load_from_db(context, parent.get_chat_id()).await?,
));
}
Ok(None)
}
/// Determines whether the message should be downloaded based on prefetched headers.
pub(crate) async fn prefetch_should_download(
context: &Context,
@@ -2281,26 +2227,18 @@ pub(crate) async fn prefetch_should_download(
message_id: &str,
mut flags: impl Iterator<Item = Flag<'_>>,
) -> Result<bool> {
if message::rfc724_mid_exists(context, message_id)
.await?
.is_some()
{
markseen_on_imap_table(context, message_id).await?;
if message::rfc724_mid_download_tried(context, message_id).await? {
if let Some(from) = mimeparser::get_from(headers)
&& context.is_self_addr(&from.addr).await?
{
markseen_on_imap_table(context, message_id).await?;
}
return Ok(false);
}
// We do not know the Message-ID or the Message-ID is missing (in this case, we create one in
// the further process).
if let Some(chat) = prefetch_get_chat(context, headers).await?
&& chat.typ == Chattype::Group
&& !chat.id.is_special()
{
// This might be a group command, like removing a group member.
// We really need to fetch this to avoid inconsistent group state.
return Ok(true);
}
let maybe_ndn = if let Some(from) = headers.get_header_value(HeaderDef::From_) {
let from = from.to_ascii_lowercase();
from.contains("mailer-daemon") || from.contains("mail-daemon")
+2
View File
@@ -17,6 +17,7 @@ use crate::tools;
/// - Chat-Version to check if a message is a chat message
/// - Autocrypt-Setup-Message to check if a message is an autocrypt setup message,
/// not necessarily sent by Delta Chat.
/// - Chat-Is-Post-Message to skip it in background fetch or when it is > `DownloadLimit`.
const PREFETCH_FLAGS: &str = "(UID INTERNALDATE RFC822.SIZE BODY.PEEK[HEADER.FIELDS (\
MESSAGE-ID \
DATE \
@@ -24,6 +25,7 @@ const PREFETCH_FLAGS: &str = "(UID INTERNALDATE RFC822.SIZE BODY.PEEK[HEADER.FIE
FROM \
IN-REPLY-TO REFERENCES \
CHAT-VERSION \
CHAT-IS-POST-MESSAGE \
AUTO-SUBMITTED \
AUTOCRYPT-SETUP-MESSAGE\
)])";
+1 -1
View File
@@ -21,7 +21,7 @@ pub async fn store_self_keypair(context: &Context, keypair: &KeyPair) -> Result<
}
pub async fn parse_and_get_text(context: &Context, imf_raw: &[u8]) -> Result<String> {
let mime_parser = MimeMessage::from_bytes(context, imf_raw, None).await?;
let mime_parser = MimeMessage::from_bytes(context, imf_raw).await?;
Ok(mime_parser.parts.into_iter().next().unwrap().msg)
}
+203 -78
View File
@@ -8,6 +8,9 @@ use std::str;
use anyhow::{Context as _, Result, ensure, format_err};
use deltachat_contact_tools::{VcardContact, parse_vcard};
use deltachat_derive::{FromSql, ToSql};
use humansize::BINARY;
use humansize::format_size;
use num_traits::FromPrimitive;
use serde::{Deserialize, Serialize};
use tokio::{fs, io};
@@ -84,12 +87,10 @@ impl MsgId {
let result = context
.sql
.query_row_optional(
concat!(
"SELECT m.state, mdns.msg_id",
" FROM msgs m LEFT JOIN msgs_mdns mdns ON mdns.msg_id=m.id",
" WHERE id=?",
" LIMIT 1",
),
"SELECT m.state, mdns.msg_id
FROM msgs m LEFT JOIN msgs_mdns mdns ON mdns.msg_id=m.id
WHERE id=?
LIMIT 1",
(self,),
|row| {
let state: MessageState = row.get(0)?;
@@ -128,10 +129,12 @@ impl MsgId {
.sql
.execute(
// If you change which information is preserved here, also change
// `delete_expired_messages()` and which information `receive_imf::add_parts()`
// still adds to the db if chat_id is TRASH.
"INSERT OR REPLACE INTO msgs (id, rfc724_mid, timestamp, chat_id, deleted)
SELECT ?1, rfc724_mid, timestamp, ?, ? FROM msgs WHERE id=?1",
// `ChatId::delete_ex()`, `delete_expired_messages()` and which information
// `receive_imf::add_parts()` still adds to the db if chat_id is TRASH.
"
INSERT OR REPLACE INTO msgs (id, rfc724_mid, pre_rfc724_mid, timestamp, chat_id, deleted)
SELECT ?1, rfc724_mid, pre_rfc724_mid, timestamp, ?, ? FROM msgs WHERE id=?1
",
(self, DC_CHAT_ID_TRASH, on_server),
)
.await?;
@@ -207,10 +210,9 @@ impl MsgId {
ret += &format!("Sent: {fts}");
let from_contact = Contact::get_by_id(context, msg.from_id).await?;
let name = from_contact.get_name_n_addr();
let name = from_contact.get_display_name();
if let Some(override_sender_name) = msg.get_override_sender_name() {
let addr = from_contact.get_addr();
ret += &format!(" by ~{override_sender_name} ({addr})");
ret += &format!(" by ~{override_sender_name}");
} else {
ret += &format!(" by {name}");
}
@@ -258,7 +260,7 @@ impl MsgId {
let name = Contact::get_by_id(context, contact_id)
.await
.map(|contact| contact.get_name_n_addr())
.map(|contact| contact.get_display_name().to_owned())
.unwrap_or_default();
ret += &format!(" by {name}");
@@ -430,6 +432,10 @@ pub struct Message {
pub(crate) ephemeral_timer: EphemeralTimer,
pub(crate) ephemeral_timestamp: i64,
pub(crate) text: String,
/// Text that is added to the end of Message.text
///
/// Currently used for adding the download information on pre-messages
pub(crate) additional_text: String,
/// Message subject.
///
@@ -438,12 +444,15 @@ pub struct Message {
/// `Message-ID` header value.
pub(crate) rfc724_mid: String,
/// `Message-ID` header value of the pre-message, if any.
pub(crate) pre_rfc724_mid: String,
/// `In-Reply-To` header value.
pub(crate) in_reply_to: Option<String>,
pub(crate) is_dc_message: MessengerMessage,
pub(crate) original_msg_id: MsgId,
pub(crate) mime_modified: bool,
pub(crate) chat_visibility: ChatVisibility,
pub(crate) chat_blocked: Blocked,
pub(crate) location_id: u32,
pub(crate) error: Option<String>,
@@ -488,42 +497,42 @@ impl Message {
!id.is_special(),
"Can not load special message ID {id} from DB"
);
let msg = context
let mut msg = context
.sql
.query_row_optional(
concat!(
"SELECT",
" m.id AS id,",
" rfc724_mid AS rfc724mid,",
" m.mime_in_reply_to AS mime_in_reply_to,",
" m.chat_id AS chat_id,",
" m.from_id AS from_id,",
" m.to_id AS to_id,",
" m.timestamp AS timestamp,",
" m.timestamp_sent AS timestamp_sent,",
" m.timestamp_rcvd AS timestamp_rcvd,",
" m.ephemeral_timer AS ephemeral_timer,",
" m.ephemeral_timestamp AS ephemeral_timestamp,",
" m.type AS type,",
" m.state AS state,",
" mdns.msg_id AS mdn_msg_id,",
" m.download_state AS download_state,",
" m.error AS error,",
" m.msgrmsg AS msgrmsg,",
" m.starred AS original_msg_id,",
" m.mime_modified AS mime_modified,",
" m.txt AS txt,",
" m.subject AS subject,",
" m.param AS param,",
" m.hidden AS hidden,",
" m.location_id AS location,",
" c.blocked AS blocked",
" FROM msgs m",
" LEFT JOIN chats c ON c.id=m.chat_id",
" LEFT JOIN msgs_mdns mdns ON mdns.msg_id=m.id",
" WHERE m.id=? AND chat_id!=3",
" LIMIT 1",
),
"SELECT
m.id AS id,
rfc724_mid AS rfc724mid,
pre_rfc724_mid AS pre_rfc724mid,
m.mime_in_reply_to AS mime_in_reply_to,
m.chat_id AS chat_id,
m.from_id AS from_id,
m.to_id AS to_id,
m.timestamp AS timestamp,
m.timestamp_sent AS timestamp_sent,
m.timestamp_rcvd AS timestamp_rcvd,
m.ephemeral_timer AS ephemeral_timer,
m.ephemeral_timestamp AS ephemeral_timestamp,
m.type AS type,
m.state AS state,
mdns.msg_id AS mdn_msg_id,
m.download_state AS download_state,
m.error AS error,
m.msgrmsg AS msgrmsg,
m.starred AS original_msg_id,
m.mime_modified AS mime_modified,
m.txt AS txt,
m.subject AS subject,
m.param AS param,
m.hidden AS hidden,
m.location_id AS location,
c.archived AS visibility,
c.blocked AS blocked
FROM msgs m
LEFT JOIN chats c ON c.id=m.chat_id
LEFT JOIN msgs_mdns mdns ON mdns.msg_id=m.id
WHERE m.id=? AND chat_id!=3
LIMIT 1",
(id,),
|row| {
let state: MessageState = row.get("state")?;
@@ -550,6 +559,7 @@ impl Message {
let msg = Message {
id: row.get("id")?,
rfc724_mid: row.get::<_, String>("rfc724mid")?,
pre_rfc724_mid: row.get::<_, String>("pre_rfc724mid")?,
in_reply_to: row
.get::<_, Option<String>>("mime_in_reply_to")?
.and_then(|in_reply_to| parse_message_id(&in_reply_to).ok()),
@@ -570,10 +580,12 @@ impl Message {
original_msg_id: row.get("original_msg_id")?,
mime_modified: row.get("mime_modified")?,
text,
additional_text: String::new(),
subject: row.get("subject")?,
param: row.get::<_, String>("param")?.parse().unwrap_or_default(),
hidden: row.get("hidden")?,
location_id: row.get("location")?,
chat_visibility: row.get::<_, Option<_>>("visibility")?.unwrap_or_default(),
chat_blocked: row
.get::<_, Option<Blocked>>("blocked")?
.unwrap_or_default(),
@@ -584,9 +596,48 @@ impl Message {
.await
.with_context(|| format!("failed to load message {id} from the database"))?;
if let Some(msg) = &mut msg {
msg.additional_text =
Self::get_additional_text(context, msg.download_state, &msg.param).await?;
}
Ok(msg)
}
/// Returns additional text which is appended to the message's text field
/// when it is loaded from the database.
/// Currently this is used to add infomation to pre-messages of what the download will be and how large it is
async fn get_additional_text(
context: &Context,
download_state: DownloadState,
param: &Params,
) -> Result<String> {
if download_state != DownloadState::Done {
let file_size = param
.get(Param::PostMessageFileBytes)
.and_then(|s| s.parse().ok())
.map(|file_size: usize| format_size(file_size, BINARY))
.unwrap_or("?".to_owned());
let viewtype = param
.get_i64(Param::PostMessageViewtype)
.and_then(Viewtype::from_i64)
.unwrap_or(Viewtype::Unknown);
let file_name = param
.get(Param::Filename)
.map(sanitize_filename)
.unwrap_or("?".to_owned());
return match viewtype {
Viewtype::File => Ok(format!(" [{file_name} – {file_size}]")),
_ => {
let translated_viewtype = viewtype.to_locale_string(context).await;
Ok(format!(" [{translated_viewtype} – {file_size}]"))
}
};
}
Ok(String::new())
}
/// Returns the MIME type of an attached file if it exists.
///
/// If the MIME type is not known, the function guesses the MIME type
@@ -768,8 +819,11 @@ impl Message {
}
/// Returns the text of the message.
///
/// Currently this includes `additional_text`, but this may change in future, when the UIs show
/// the necessary info themselves.
pub fn get_text(&self) -> String {
self.text.clone()
self.text.clone() + &self.additional_text
}
/// Returns message subject.
@@ -791,7 +845,16 @@ impl Message {
}
/// Returns the size of the file in bytes, if applicable.
/// If message is a pre-message, then this returns the size of the file to be downloaded.
pub async fn get_filebytes(&self, context: &Context) -> Result<Option<u64>> {
if self.download_state != DownloadState::Done
&& let Some(file_size) = self
.param
.get(Param::PostMessageFileBytes)
.and_then(|s| s.parse().ok())
{
return Ok(Some(file_size));
}
if let Some(path) = self.param.get_file_path(context)? {
Ok(Some(get_filebytes(context, &path).await.with_context(
|| format!("failed to get {} size in bytes", path.display()),
@@ -801,6 +864,19 @@ impl Message {
}
}
/// If message is a Pre-Message,
/// then this returns the viewtype it will have when it is downloaded.
#[cfg(test)]
pub(crate) fn get_post_message_viewtype(&self) -> Option<Viewtype> {
if self.download_state != DownloadState::Done {
return self
.param
.get_i64(Param::PostMessageViewtype)
.and_then(Viewtype::from_i64);
}
None
}
/// Returns width of associated image or video file.
pub fn get_width(&self) -> i32 {
self.param.get_int(Param::Width).unwrap_or_default()
@@ -850,11 +926,10 @@ impl Message {
let contact = if self.from_id != ContactId::SELF {
match chat.typ {
Chattype::Group
| Chattype::OutBroadcast
| Chattype::InBroadcast
| Chattype::Mailinglist => Some(Contact::get_by_id(context, self.from_id).await?),
Chattype::Single => None,
Chattype::Group | Chattype::Mailinglist => {
Some(Contact::get_by_id(context, self.from_id).await?)
}
Chattype::Single | Chattype::OutBroadcast | Chattype::InBroadcast => None,
}
} else {
None
@@ -1431,6 +1506,16 @@ pub async fn get_msg_read_receipts(
.await
}
/// Returns count of read receipts on message.
///
/// This view count is meant as a feedback measure for the channel owner only.
pub async fn get_msg_read_receipt_count(context: &Context, msg_id: MsgId) -> Result<usize> {
context
.sql
.count("SELECT COUNT(*) FROM msgs_mdns WHERE msg_id=?", (msg_id,))
.await
}
pub(crate) fn guess_msgtype_from_suffix(msg: &Message) -> Option<(Viewtype, &'static str)> {
msg.param
.get(Param::Filename)
@@ -1679,11 +1764,20 @@ pub async fn delete_msgs_ex(
let target = context.get_delete_msgs_target().await?;
let update_db = |trans: &mut rusqlite::Transaction| {
trans.execute(
"UPDATE imap SET target=? WHERE rfc724_mid=?",
(target, msg.rfc724_mid),
)?;
let mut stmt = trans.prepare("UPDATE imap SET target=? WHERE rfc724_mid=?")?;
stmt.execute((&target, &msg.rfc724_mid))?;
if !msg.pre_rfc724_mid.is_empty() {
stmt.execute((&target, &msg.pre_rfc724_mid))?;
}
trans.execute("DELETE FROM smtp WHERE msg_id=?", (msg_id,))?;
trans.execute(
"DELETE FROM download WHERE rfc724_mid=?",
(&msg.rfc724_mid,),
)?;
trans.execute(
"DELETE FROM available_post_msgs WHERE rfc724_mid=?",
(&msg.rfc724_mid,),
)?;
Ok(())
};
if let Err(e) = context.sql.transaction(update_db).await {
@@ -1717,6 +1811,7 @@ pub async fn delete_msgs_ex(
msgs: deleted_rfc724_mid,
})
.await?;
context.scheduler.interrupt_smtp().await;
}
for &msg_id in msg_ids {
@@ -1751,11 +1846,11 @@ pub async fn markseen_msgs(context: &Context, msg_ids: Vec<MsgId>) -> Result<()>
"SELECT
m.chat_id AS chat_id,
m.state AS state,
m.download_state as download_state,
m.ephemeral_timer AS ephemeral_timer,
m.param AS param,
m.from_id AS from_id,
m.rfc724_mid AS rfc724_mid,
m.hidden AS hidden,
c.archived AS archived,
c.blocked AS blocked
FROM msgs m LEFT JOIN chats c ON c.id=m.chat_id
@@ -1764,10 +1859,10 @@ pub async fn markseen_msgs(context: &Context, msg_ids: Vec<MsgId>) -> Result<()>
|row| {
let chat_id: ChatId = row.get("chat_id")?;
let state: MessageState = row.get("state")?;
let download_state: DownloadState = row.get("download_state")?;
let param: Params = row.get::<_, String>("param")?.parse().unwrap_or_default();
let from_id: ContactId = row.get("from_id")?;
let rfc724_mid: String = row.get("rfc724_mid")?;
let hidden: bool = row.get("hidden")?;
let visibility: ChatVisibility = row.get("archived")?;
let blocked: Option<Blocked> = row.get("blocked")?;
let ephemeral_timer: EphemeralTimer = row.get("ephemeral_timer")?;
@@ -1776,10 +1871,10 @@ pub async fn markseen_msgs(context: &Context, msg_ids: Vec<MsgId>) -> Result<()>
id,
chat_id,
state,
download_state,
param,
from_id,
rfc724_mid,
hidden,
visibility,
blocked.unwrap_or_default(),
),
@@ -1809,31 +1904,25 @@ pub async fn markseen_msgs(context: &Context, msg_ids: Vec<MsgId>) -> Result<()>
id,
curr_chat_id,
curr_state,
curr_download_state,
curr_param,
curr_from_id,
curr_rfc724_mid,
curr_hidden,
curr_visibility,
curr_blocked,
),
_curr_ephemeral_timer,
) in msgs
{
if curr_download_state != DownloadState::Done {
if curr_state == MessageState::InFresh {
// Don't mark partially downloaded messages as seen or send a read receipt since
// they are not really seen by the user.
update_msg_state(context, id, MessageState::InNoticed).await?;
updated_chat_ids.insert(curr_chat_id);
}
} else if curr_state == MessageState::InFresh || curr_state == MessageState::InNoticed {
if curr_state == MessageState::InFresh || curr_state == MessageState::InNoticed {
update_msg_state(context, id, MessageState::InSeen).await?;
info!(context, "Seen message {}.", id);
markseen_on_imap_table(context, &curr_rfc724_mid).await?;
// Read receipts for system messages are never sent. These messages have no place to
// display received read receipt anyway. And since their text is locally generated,
// Read receipts for system messages are never sent to contacts.
// These messages have no place to display received read receipt
// anyway. And since their text is locally generated,
// quoting them is dangerous as it may contain contact names. E.g., for original message
// "Group left by me", a read receipt will quote "Group left by <name>", and the name can
// be a display name stored in address book rather than the name sent in the From field by
@@ -1841,25 +1930,35 @@ pub async fn markseen_msgs(context: &Context, msg_ids: Vec<MsgId>) -> Result<()>
//
// We also don't send read receipts for contact requests.
// Read receipts will not be sent even after accepting the chat.
if curr_blocked == Blocked::Not
let to_id = if curr_blocked == Blocked::Not
&& curr_param.get_bool(Param::WantsMdn).unwrap_or_default()
&& curr_param.get_cmd() == SystemMessage::Unknown
&& context.should_send_mdns().await?
{
Some(curr_from_id)
} else if context.get_config_bool(Config::BccSelf).await? {
Some(ContactId::SELF)
} else {
None
};
if let Some(to_id) = to_id {
context
.sql
.execute(
"INSERT INTO smtp_mdns (msg_id, from_id, rfc724_mid) VALUES(?, ?, ?)",
(id, curr_from_id, curr_rfc724_mid),
(id, to_id, curr_rfc724_mid),
)
.await
.context("failed to insert into smtp_mdns")?;
context.scheduler.interrupt_smtp().await;
}
updated_chat_ids.insert(curr_chat_id);
if !curr_hidden {
updated_chat_ids.insert(curr_chat_id);
}
}
archived_chats_maybe_noticed |=
curr_state == MessageState::InFresh && curr_visibility == ChatVisibility::Archived;
archived_chats_maybe_noticed |= curr_state == MessageState::InFresh
&& !curr_hidden
&& curr_visibility == ChatVisibility::Archived;
}
for updated_chat_id in updated_chat_ids {
@@ -2097,7 +2196,7 @@ pub(crate) async fn rfc724_mid_exists_ex(
.query_row_optional(
&("SELECT id, timestamp_sent, MIN(".to_string()
+ expr
+ ") FROM msgs WHERE rfc724_mid=?
+ ") FROM msgs WHERE rfc724_mid=?1 OR pre_rfc724_mid=?1
HAVING COUNT(*) > 0 -- Prevent MIN(expr) from returning NULL when there are no rows.
ORDER BY timestamp_sent DESC"),
(rfc724_mid,),
@@ -2112,6 +2211,32 @@ pub(crate) async fn rfc724_mid_exists_ex(
Ok(res)
}
/// Returns `true` iff there is a message
/// with the given `rfc724_mid`
/// and a download state other than `DownloadState::Available`,
/// i.e. it was already tried to download the message or it's sent locally.
pub(crate) async fn rfc724_mid_download_tried(context: &Context, rfc724_mid: &str) -> Result<bool> {
let rfc724_mid = rfc724_mid.trim_start_matches('<').trim_end_matches('>');
if rfc724_mid.is_empty() {
warn!(
context,
"Empty rfc724_mid passed to rfc724_mid_download_tried"
);
return Ok(false);
}
let res = context
.sql
.exists(
"SELECT COUNT(*) FROM msgs
WHERE rfc724_mid=? AND download_state<>?",
(rfc724_mid, DownloadState::Available),
)
.await?;
Ok(res)
}
/// Given a list of Message-IDs, returns the most relevant message found in the database.
///
/// Relevance here is `(download_state == Done, index)`, where `index` is an index of Message-ID in
+60 -81
View File
@@ -326,79 +326,7 @@ async fn test_markseen_msgs() -> Result<()> {
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_markseen_not_downloaded_msg() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
alice.set_config(Config::DownloadLimit, Some("1")).await?;
let bob = &tcm.bob().await;
let bob_chat_id = bob.create_chat(alice).await.id;
alice.create_chat(bob).await; // Make sure the chat is accepted.
tcm.section("Bob sends a large message to Alice");
let file_bytes = include_bytes!("../../test-data/image/screenshot.png");
let mut msg = Message::new(Viewtype::Image);
msg.set_file_from_bytes(bob, "a.jpg", file_bytes, None)?;
let sent_msg = bob.send_msg(bob_chat_id, &mut msg).await;
tcm.section("Alice receives a large message from Bob");
let msg = alice.recv_msg(&sent_msg).await;
assert_eq!(msg.download_state, DownloadState::Available);
assert!(!msg.param.get_bool(Param::WantsMdn).unwrap_or_default());
assert_eq!(msg.state, MessageState::InFresh);
markseen_msgs(alice, vec![msg.id]).await?;
// A not downloaded message can be seen only if it's seen on another device.
assert_eq!(msg.id.get_state(alice).await?, MessageState::InNoticed);
// Marking the message as seen again is a no op.
markseen_msgs(alice, vec![msg.id]).await?;
assert_eq!(msg.id.get_state(alice).await?, MessageState::InNoticed);
msg.id
.update_download_state(alice, DownloadState::InProgress)
.await?;
markseen_msgs(alice, vec![msg.id]).await?;
assert_eq!(msg.id.get_state(alice).await?, MessageState::InNoticed);
msg.id
.update_download_state(alice, DownloadState::Failure)
.await?;
markseen_msgs(alice, vec![msg.id]).await?;
assert_eq!(msg.id.get_state(alice).await?, MessageState::InNoticed);
msg.id
.update_download_state(alice, DownloadState::Undecipherable)
.await?;
markseen_msgs(alice, vec![msg.id]).await?;
assert_eq!(msg.id.get_state(alice).await?, MessageState::InNoticed);
assert!(
!alice
.sql
.exists("SELECT COUNT(*) FROM smtp_mdns", ())
.await?
);
alice.set_config(Config::DownloadLimit, None).await?;
// Let's assume that Alice and Bob resolved the problem with encryption.
let old_msg = msg;
let msg = alice.recv_msg(&sent_msg).await;
assert_eq!(msg.chat_id, old_msg.chat_id);
assert_eq!(msg.download_state, DownloadState::Done);
assert!(msg.param.get_bool(Param::WantsMdn).unwrap_or_default());
assert!(msg.get_showpadlock());
// The message state mustn't be downgraded to `InFresh`.
assert_eq!(msg.state, MessageState::InNoticed);
markseen_msgs(alice, vec![msg.id]).await?;
let msg = Message::load_from_db(alice, msg.id).await?;
assert_eq!(msg.state, MessageState::InSeen);
assert_eq!(
alice
.sql
.count("SELECT COUNT(*) FROM smtp_mdns", ())
.await?,
1
);
Ok(())
}
/// Message has been seen on another device when fully downloaded. `state` should be updated.
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_msg_seen_on_imap_when_downloaded() -> Result<()> {
let mut tcm = TestContextManager::new();
@@ -411,20 +339,17 @@ async fn test_msg_seen_on_imap_when_downloaded() -> Result<()> {
let mut msg = Message::new(Viewtype::Image);
msg.set_file_from_bytes(bob, "a.jpg", file_bytes, None)?;
let sent_msg = bob.send_msg(bob_chat_id, &mut msg).await;
let msg = alice.recv_msg(&sent_msg).await;
let pre_msg = bob.pop_sent_msg().await;
let msg = alice.recv_msg(&pre_msg).await;
assert_eq!(msg.download_state, DownloadState::Available);
assert_eq!(msg.state, MessageState::InFresh);
alice.set_config(Config::DownloadLimit, None).await?;
let seen = true;
let rcvd_msg = receive_imf(alice, sent_msg.payload().as_bytes(), seen)
.await
.unwrap()
.unwrap();
assert_eq!(rcvd_msg.chat_id, msg.chat_id);
let msg = Message::load_from_db(alice, *rcvd_msg.msg_ids.last().unwrap())
.await
.await?
.unwrap();
assert_eq!(rcvd_msg.chat_id, DC_CHAT_ID_TRASH);
let msg = Message::load_from_db(alice, msg.id).await?;
assert_eq!(msg.download_state, DownloadState::Done);
assert!(msg.param.get_bool(Param::WantsMdn).unwrap_or_default());
assert!(msg.get_showpadlock());
@@ -432,6 +357,60 @@ async fn test_msg_seen_on_imap_when_downloaded() -> Result<()> {
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_pre_and_post_msgs_deleted() -> Result<()> {
let reorder = false;
test_pre_and_post_msgs_deleted_ex(reorder).await
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_reordered_pre_and_post_msgs_deleted() -> Result<()> {
let reorder = true;
test_pre_and_post_msgs_deleted_ex(reorder).await
}
async fn test_pre_and_post_msgs_deleted_ex(reorder: bool) -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
let bob = &tcm.bob().await;
let alice_chat_id = alice.create_group_with_members("", &[bob]).await;
let file_bytes = include_bytes!("../../test-data/image/screenshot.gif");
let mut msg = Message::new(Viewtype::Image);
msg.set_file_from_bytes(alice, "a.jpg", file_bytes, None)?;
let full_msg = alice.send_msg(alice_chat_id, &mut msg).await;
let pre_msg = alice.pop_sent_msg().await;
let rfc724_mid_pre = bob.parse_msg(&pre_msg).await.get_rfc724_mid().unwrap();
let msg = if reorder {
let msg = bob.recv_msg(&full_msg).await;
bob.recv_msg_trash(&pre_msg).await;
Message::load_from_db(bob, msg.id).await?
} else {
let msg = bob.recv_msg(&pre_msg).await;
bob.recv_msg_trash(&full_msg).await;
msg
};
assert_ne!(rfc724_mid_pre, msg.rfc724_mid);
for (rfc724_mid, uid) in [(&rfc724_mid_pre, 1), (&msg.rfc724_mid, 2)] {
bob.sql
.execute(
"INSERT INTO imap (transport_id, rfc724_mid, folder, uid, target, uidvalidity) VALUES (1, ?, 'INBOX', ?, 'INBOX', 12345)",
(rfc724_mid, uid),
)
.await?;
}
delete_msgs(bob, &[msg.id]).await?;
assert_eq!(
bob.sql
.count("SELECT COUNT(*) FROM imap WHERE target!=''", ())
.await?,
0
);
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_get_state() -> Result<()> {
let alice = TestContext::new_alice().await;
+134 -28
View File
@@ -21,6 +21,7 @@ use crate::constants::{ASM_SUBJECT, BROADCAST_INCOMPATIBILITY_MSG};
use crate::constants::{Chattype, DC_FROM_HANDSHAKE};
use crate::contact::{Contact, ContactId, Origin};
use crate::context::Context;
use crate::download::PostMsgMetadata;
use crate::e2ee::EncryptHelper;
use crate::ensure_and_debug_assert;
use crate::ephemeral::Timer as EphemeralTimer;
@@ -59,6 +60,17 @@ pub enum Loaded {
},
}
#[derive(Debug, Clone, PartialEq)]
pub enum PreMessageMode {
/// adds the Chat-Is-Post-Message header in unprotected part
Post,
/// adds the Chat-Post-Message-ID header to protected part
/// also adds metadata and explicitly excludes attachment
Pre { post_msg_rfc724_mid: String },
/// Atomic ("normal") message.
None,
}
/// Helper to construct mime messages.
#[derive(Debug, Clone)]
pub struct MimeFactory {
@@ -94,6 +106,7 @@ pub struct MimeFactory {
/// addresses and OpenPGP keys
/// to use for encryption.
///
/// If `Some`, encrypt to self also.
/// `None` if the message is not encrypted.
encryption_pubkeys: Option<Vec<(String, SignedPublicKey)>>,
@@ -146,6 +159,9 @@ pub struct MimeFactory {
/// This field is used to sustain the topic id of webxdcs needed for peer channels.
webxdc_topic: Option<TopicId>,
/// Pre-message / post-message / atomic message.
pre_message_mode: PreMessageMode,
}
/// Result of rendering a message, ready to be submitted to a send job.
@@ -219,7 +235,6 @@ impl MimeFactory {
encryption_pubkeys = if msg.param.get_bool(Param::ForcePlaintext).unwrap_or(false) {
None
} else {
// Encrypt, but only to self.
Some(Vec::new())
};
} else if chat.is_mailing_list() {
@@ -232,6 +247,37 @@ impl MimeFactory {
// Do not encrypt messages to mailing lists.
encryption_pubkeys = None;
} else if let Some(fp) = must_have_only_one_recipient(&msg, &chat) {
let fp = fp?;
// In a broadcast channel, only send member-added/removed messages
// to the affected member
let (authname, addr) = context
.sql
.query_row(
"SELECT authname, addr FROM contacts WHERE fingerprint=?",
(fp,),
|row| {
let authname: String = row.get(0)?;
let addr: String = row.get(1)?;
Ok((authname, addr))
},
)
.await?;
let public_key_bytes: Vec<_> = context
.sql
.query_get_value(
"SELECT public_key FROM public_keys WHERE fingerprint=?",
(fp,),
)
.await?
.context("Can't send member addition/removal: missing key")?;
recipients.push(addr.clone());
to.push((authname, addr.clone()));
let public_key = SignedPublicKey::from_slice(&public_key_bytes)?;
encryption_pubkeys = Some(vec![(addr, public_key)]);
} else {
let email_to_remove = if msg.param.get_cmd() == SystemMessage::MemberRemovedFromGroup {
msg.param.get(Param::Arg)
@@ -291,13 +337,6 @@ impl MimeFactory {
for row in rows {
let (authname, addr, fingerprint, id, add_timestamp, remove_timestamp, public_key_bytes_opt) = row?;
// In a broadcast channel, only send member-added/removed messages
// to the affected member:
if let Some(fp) = must_have_only_one_recipient(&msg, &chat)
&& fp? != fingerprint {
continue;
}
let public_key_opt = if let Some(public_key_bytes) = &public_key_bytes_opt {
Some(SignedPublicKey::from_slice(public_key_bytes)?)
} else {
@@ -411,8 +450,16 @@ impl MimeFactory {
},
)
.await?;
let recipient_ids: Vec<_> = recipient_ids.into_iter().collect();
ContactId::scaleup_origin(context, &recipient_ids, Origin::OutgoingTo).await?;
let recipient_ids: Vec<_> = recipient_ids
.into_iter()
.filter(|id| *id != ContactId::SELF)
.collect();
if recipient_ids.len() == 1
&& msg.param.get_cmd() != SystemMessage::MemberRemovedFromGroup
&& chat.typ != Chattype::OutBroadcast
{
ContactId::scaleup_origin(context, &recipient_ids, Origin::OutgoingTo).await?;
}
if !msg.is_system_message()
&& msg.param.get_int(Param::Reaction).unwrap_or_default() == 0
@@ -500,6 +547,7 @@ impl MimeFactory {
sync_ids_to_delete: None,
attach_selfavatar,
webxdc_topic,
pre_message_mode: PreMessageMode::None,
};
Ok(factory)
}
@@ -515,7 +563,9 @@ impl MimeFactory {
let timestamp = create_smeared_timestamp(context);
let addr = contact.get_addr().to_string();
let encryption_pubkeys = if contact.is_key_contact() {
let encryption_pubkeys = if from_id == ContactId::SELF {
Some(Vec::new())
} else if contact.is_key_contact() {
if let Some(key) = contact.public_key(context).await? {
Some(vec![(addr.clone(), key)])
} else {
@@ -548,6 +598,7 @@ impl MimeFactory {
sync_ids_to_delete: None,
attach_selfavatar: false,
webxdc_topic: None,
pre_message_mode: PreMessageMode::None,
};
Ok(res)
@@ -779,7 +830,10 @@ impl MimeFactory {
headers.push(("Date", mail_builder::headers::raw::Raw::new(date).into()));
let rfc724_mid = match &self.loaded {
Loaded::Message { msg, .. } => msg.rfc724_mid.clone(),
Loaded::Message { msg, .. } => match &self.pre_message_mode {
PreMessageMode::Pre { .. } => create_outgoing_rfc724_mid(),
_ => msg.rfc724_mid.clone(),
},
Loaded::Mdn { .. } => create_outgoing_rfc724_mid(),
};
headers.push((
@@ -893,7 +947,7 @@ impl MimeFactory {
));
}
let is_encrypted = self.encryption_pubkeys.is_some();
let is_encrypted = self.will_be_encrypted();
// Add ephemeral timer for non-MDN messages.
// For MDNs it does not matter because they are not visible
@@ -978,6 +1032,23 @@ impl MimeFactory {
"MIME-Version",
mail_builder::headers::raw::Raw::new("1.0").into(),
));
if self.pre_message_mode == PreMessageMode::Post {
unprotected_headers.push((
"Chat-Is-Post-Message",
mail_builder::headers::raw::Raw::new("1").into(),
));
} else if let PreMessageMode::Pre {
post_msg_rfc724_mid,
} = &self.pre_message_mode
{
protected_headers.push((
"Chat-Post-Message-ID",
mail_builder::headers::message_id::MessageId::new(post_msg_rfc724_mid.clone())
.into(),
));
}
for header @ (original_header_name, _header_value) in &headers {
let header_name = original_header_name.to_lowercase();
if header_name == "message-id" {
@@ -1119,6 +1190,10 @@ impl MimeFactory {
for (addr, key) in &encryption_pubkeys {
let fingerprint = key.dc_fingerprint().hex();
let cmd = msg.param.get_cmd();
if self.pre_message_mode == PreMessageMode::Post {
continue;
}
let should_do_gossip = cmd == SystemMessage::MemberAddedToGroup
|| cmd == SystemMessage::SecurejoinMessage
|| multiple_recipients && {
@@ -1831,19 +1906,23 @@ impl MimeFactory {
let footer = if is_reaction { "" } else { &self.selfstatus };
let message_text = format!(
"{}{}{}{}{}{}",
fwdhint.unwrap_or_default(),
quoted_text.unwrap_or_default(),
escape_message_footer_marks(final_text),
if !final_text.is_empty() && !footer.is_empty() {
"\r\n\r\n"
} else {
""
},
if !footer.is_empty() { "-- \r\n" } else { "" },
footer
);
let message_text = if self.pre_message_mode == PreMessageMode::Post {
"".to_string()
} else {
format!(
"{}{}{}{}{}{}",
fwdhint.unwrap_or_default(),
quoted_text.unwrap_or_default(),
escape_message_footer_marks(final_text),
if !final_text.is_empty() && !footer.is_empty() {
"\r\n\r\n"
} else {
""
},
if !footer.is_empty() { "-- \r\n" } else { "" },
footer
)
};
let mut main_part = MimePart::new("text/plain", message_text);
if is_reaction {
@@ -1875,8 +1954,19 @@ impl MimeFactory {
// add attachment part
if msg.viewtype.has_file() {
let file_part = build_body_file(context, &msg).await?;
parts.push(file_part);
if let PreMessageMode::Pre { .. } = self.pre_message_mode {
let Some(metadata) = PostMsgMetadata::from_msg(context, &msg).await? else {
bail!("Failed to generate metadata for pre-message")
};
headers.push((
HeaderDef::ChatPostMessageMetadata.into(),
mail_builder::headers::raw::Raw::new(metadata.to_header_value()?).into(),
));
} else {
let file_part = build_body_file(context, &msg).await?;
parts.push(file_part);
}
}
if let Some(msg_kml_part) = self.get_message_kml_part() {
@@ -1921,6 +2011,8 @@ impl MimeFactory {
}
}
self.attach_selfavatar =
self.attach_selfavatar && self.pre_message_mode != PreMessageMode::Post;
if self.attach_selfavatar {
match context.get_config(Config::Selfavatar).await? {
Some(path) => match build_avatar_file(context, &path).await {
@@ -1990,6 +2082,20 @@ impl MimeFactory {
Ok(message)
}
pub fn will_be_encrypted(&self) -> bool {
self.encryption_pubkeys.is_some()
}
pub fn set_as_post_message(&mut self) {
self.pre_message_mode = PreMessageMode::Post;
}
pub fn set_as_pre_message_for(&mut self, post_message: &RenderedEmail) {
self.pre_message_mode = PreMessageMode::Pre {
post_msg_rfc724_mid: post_message.rfc724_mid.clone(),
};
}
}
fn hidden_recipients() -> Address<'static> {
+4 -4
View File
@@ -559,7 +559,7 @@ async fn test_render_reply() {
"1.0"
);
let _mime_msg = MimeMessage::from_bytes(context, rendered_msg.message.as_bytes(), None)
let _mime_msg = MimeMessage::from_bytes(context, rendered_msg.message.as_bytes())
.await
.unwrap();
}
@@ -757,7 +757,7 @@ async fn test_protected_headers_directive() -> Result<()> {
assert!(msg.get_showpadlock());
assert!(sent.payload.contains("\r\nSubject: [...]\r\n"));
let mime = MimeMessage::from_bytes(&alice, sent.payload.as_bytes(), None).await?;
let mime = MimeMessage::from_bytes(&alice, sent.payload.as_bytes()).await?;
let mut payload = str::from_utf8(&mime.decoded_data)?.splitn(2, "\r\n\r\n");
let part = payload.next().unwrap();
assert_eq!(
@@ -781,7 +781,7 @@ async fn test_hp_outer_headers() -> Result<()> {
.await?;
chat::send_text_msg(t, chat_id, "hi!".to_string()).await?;
let sent_msg = t.pop_sent_msg().await;
let msg = MimeMessage::from_bytes(t, sent_msg.payload.as_bytes(), None).await?;
let msg = MimeMessage::from_bytes(t, sent_msg.payload.as_bytes()).await?;
assert_eq!(msg.header_exists(HeaderDef::HpOuter), std_hp_composing);
for hdr in ["Date", "From", "Message-ID"] {
assert_eq!(
@@ -811,7 +811,7 @@ async fn test_dont_remove_self() -> Result<()> {
.await;
println!("{}", sent.payload);
let mime_message = MimeMessage::from_bytes(alice, sent.payload.as_bytes(), None)
let mime_message = MimeMessage::from_bytes(alice, sent.payload.as_bytes())
.await
.unwrap();
assert!(!mime_message.header_exists(HeaderDef::ChatGroupPastMembers));
+97 -51
View File
@@ -23,6 +23,7 @@ use crate::contact::ContactId;
use crate::context::Context;
use crate::decrypt::{try_decrypt, validate_detached_signature};
use crate::dehtml::dehtml;
use crate::download::PostMsgMetadata;
use crate::events::EventType;
use crate::headerdef::{HeaderDef, HeaderDefMap};
use crate::key::{self, DcKey, Fingerprint, SignedPublicKey, load_self_secret_keyring};
@@ -147,6 +148,25 @@ pub(crate) struct MimeMessage {
/// Sender timestamp in secs since epoch. Allowed to be in the future due to unsynchronized
/// clocks, but not too much.
pub(crate) timestamp_sent: i64,
pub(crate) pre_message: PreMessageMode,
}
#[derive(Debug, Clone, PartialEq)]
pub(crate) enum PreMessageMode {
/// This is a post-message.
/// It replaces its pre-message attachment if it exists already,
/// and if the pre-message does not exist, it is treated as a normal message.
Post,
/// This is a Pre-Message,
/// it adds a message preview for a Post-Message
/// and it is ignored if the Post-Message was downloaded already
Pre {
post_msg_rfc724_mid: String,
metadata: Option<PostMsgMetadata>,
},
/// Atomic ("normal") message.
None,
}
#[derive(Debug, PartialEq)]
@@ -240,12 +260,9 @@ const MIME_AC_SETUP_FILE: &str = "application/autocrypt-setup";
impl MimeMessage {
/// Parse a mime message.
///
/// If `partial` is set, it contains the full message size in bytes.
pub(crate) async fn from_bytes(
context: &Context,
body: &[u8],
partial: Option<u32>,
) -> Result<Self> {
/// This method has some side-effects,
/// such as saving blobs and saving found public keys to the database.
pub(crate) async fn from_bytes(context: &Context, body: &[u8]) -> Result<Self> {
let mail = mailparse::parse_mail(body)?;
let timestamp_rcvd = smeared_time(context);
@@ -302,7 +319,7 @@ impl MimeMessage {
);
(part, part.ctype.mimetype.parse::<Mime>()?)
} else {
// If it's a partially fetched message, there are no subparts.
// Not a valid signed message, handle it as plaintext.
(&mail, mimetype)
}
} else {
@@ -352,6 +369,16 @@ impl MimeMessage {
let mut aheader_values = mail.headers.get_all_values(HeaderDef::Autocrypt.into());
let mut pre_message = if mail
.headers
.get_header_value(HeaderDef::ChatIsPostMessage)
.is_some()
{
PreMessageMode::Post
} else {
PreMessageMode::None
};
let mail_raw; // Memory location for a possible decrypted message.
let decrypted_msg; // Decrypted signed OpenPGP message.
let secrets: Vec<String> = context
@@ -580,6 +607,39 @@ impl MimeMessage {
signatures.clear();
}
if let (Ok(mail), true) = (mail, is_encrypted)
&& let Some(post_msg_rfc724_mid) =
mail.headers.get_header_value(HeaderDef::ChatPostMessageId)
{
let post_msg_rfc724_mid = parse_message_id(&post_msg_rfc724_mid)?;
let metadata = if let Some(value) = mail
.headers
.get_header_value(HeaderDef::ChatPostMessageMetadata)
{
match PostMsgMetadata::try_from_header_value(&value) {
Ok(metadata) => Some(metadata),
Err(error) => {
error!(
context,
"Failed to parse metadata header in pre-message for {post_msg_rfc724_mid}: {error:#}."
);
None
}
}
} else {
warn!(
context,
"Expected pre-message for {post_msg_rfc724_mid} to have metadata header."
);
None
};
pre_message = PreMessageMode::Pre {
post_msg_rfc724_mid,
metadata,
};
}
let mut parser = MimeMessage {
parts: Vec::new(),
headers,
@@ -615,33 +675,27 @@ impl MimeMessage {
is_bot: None,
timestamp_rcvd,
timestamp_sent,
pre_message,
};
match partial {
Some(org_bytes) => {
parser
.create_stub_from_partial_download(context, org_bytes)
.await?;
match mail {
Ok(mail) => {
parser.parse_mime_recursive(context, mail, false).await?;
}
None => match mail {
Ok(mail) => {
parser.parse_mime_recursive(context, mail, false).await?;
}
Err(err) => {
let txt = "[This message cannot be decrypted.\n\n• It might already help to simply reply to this message and ask the sender to send the message again.\n\n• If you just re-installed Delta Chat then it is best if you re-setup Delta Chat now and choose \"Add as second device\" or import a backup.]";
Err(err) => {
let txt = "[This message cannot be decrypted.\n\n• It might already help to simply reply to this message and ask the sender to send the message again.\n\n• If you just re-installed Delta Chat then it is best if you re-setup Delta Chat now and choose \"Add as second device\" or import a backup.]";
let part = Part {
typ: Viewtype::Text,
msg_raw: Some(txt.to_string()),
msg: txt.to_string(),
// Don't change the error prefix for now,
// receive_imf.rs:lookup_chat_by_reply() checks it.
error: Some(format!("Decrypting failed: {err:#}")),
..Default::default()
};
parser.do_add_single_part(part);
}
},
let part = Part {
typ: Viewtype::Text,
msg_raw: Some(txt.to_string()),
msg: txt.to_string(),
// Don't change the error prefix for now,
// receive_imf.rs:lookup_chat_by_reply() checks it.
error: Some(format!("Decrypting failed: {err:#}")),
..Default::default()
};
parser.do_add_single_part(part);
}
};
let is_location_only = parser.location_kml.is_some() && parser.parts.is_empty();
@@ -2105,9 +2159,9 @@ pub(crate) struct Report {
///
/// It MUST be present if the original message has a Message-ID according to RFC 8098.
/// In case we can't find it (shouldn't happen), this is None.
original_message_id: Option<String>,
pub original_message_id: Option<String>,
/// Additional-Message-IDs
additional_message_ids: Vec<String>,
pub additional_message_ids: Vec<String>,
}
/// Delivery Status Notification (RFC 3464, RFC 6533)
@@ -2414,31 +2468,23 @@ async fn handle_mdn(
timestamp_sent: i64,
) -> Result<()> {
if from_id == ContactId::SELF {
warn!(
context,
"Ignoring MDN sent to self, this is a bug on the sender device."
);
// This is not an error on our side,
// we successfully ignored an invalid MDN and return `Ok`.
// MDNs to self are handled in receive_imf_inner().
return Ok(());
}
let Some((msg_id, chat_id, has_mdns, is_dup)) = context
.sql
.query_row_optional(
concat!(
"SELECT",
" m.id AS msg_id,",
" c.id AS chat_id,",
" mdns.contact_id AS mdn_contact",
" FROM msgs m ",
" LEFT JOIN chats c ON m.chat_id=c.id",
" LEFT JOIN msgs_mdns mdns ON mdns.msg_id=m.id",
" WHERE rfc724_mid=? AND from_id=1",
" ORDER BY msg_id DESC, mdn_contact=? DESC",
" LIMIT 1",
),
"SELECT
m.id AS msg_id,
c.id AS chat_id,
mdns.contact_id AS mdn_contact
FROM msgs m
LEFT JOIN chats c ON m.chat_id=c.id
LEFT JOIN msgs_mdns mdns ON mdns.msg_id=m.id
WHERE rfc724_mid=? AND from_id=1
ORDER BY msg_id DESC, mdn_contact=? DESC
LIMIT 1",
(&rfc724_mid, from_id),
|row| {
let msg_id: MsgId = row.get("msg_id")?;
+58 -71
View File
@@ -25,58 +25,54 @@ impl AvatarAction {
async fn test_mimeparser_fromheader() {
let ctx = TestContext::new_alice().await;
let mimemsg = MimeMessage::from_bytes(&ctx, b"From: g@c.de\n\nhi", None)
let mimemsg = MimeMessage::from_bytes(&ctx, b"From: g@c.de\n\nhi")
.await
.unwrap();
let contact = mimemsg.from;
assert_eq!(contact.addr, "g@c.de");
assert_eq!(contact.display_name, None);
let mimemsg = MimeMessage::from_bytes(&ctx, b"From: g@c.de \n\nhi", None)
let mimemsg = MimeMessage::from_bytes(&ctx, b"From: g@c.de \n\nhi")
.await
.unwrap();
let contact = mimemsg.from;
assert_eq!(contact.addr, "g@c.de");
assert_eq!(contact.display_name, None);
let mimemsg = MimeMessage::from_bytes(&ctx, b"From: <g@c.de>\n\nhi", None)
let mimemsg = MimeMessage::from_bytes(&ctx, b"From: <g@c.de>\n\nhi")
.await
.unwrap();
let contact = mimemsg.from;
assert_eq!(contact.addr, "g@c.de");
assert_eq!(contact.display_name, None);
let mimemsg = MimeMessage::from_bytes(&ctx, b"From: Goetz C <g@c.de>\n\nhi", None)
let mimemsg = MimeMessage::from_bytes(&ctx, b"From: Goetz C <g@c.de>\n\nhi")
.await
.unwrap();
let contact = mimemsg.from;
assert_eq!(contact.addr, "g@c.de");
assert_eq!(contact.display_name, Some("Goetz C".to_string()));
let mimemsg = MimeMessage::from_bytes(&ctx, b"From: \"Goetz C\" <g@c.de>\n\nhi", None)
let mimemsg = MimeMessage::from_bytes(&ctx, b"From: \"Goetz C\" <g@c.de>\n\nhi")
.await
.unwrap();
let contact = mimemsg.from;
assert_eq!(contact.addr, "g@c.de");
assert_eq!(contact.display_name, Some("Goetz C".to_string()));
let mimemsg =
MimeMessage::from_bytes(&ctx, b"From: =?utf-8?q?G=C3=B6tz?= C <g@c.de>\n\nhi", None)
.await
.unwrap();
let mimemsg = MimeMessage::from_bytes(&ctx, b"From: =?utf-8?q?G=C3=B6tz?= C <g@c.de>\n\nhi")
.await
.unwrap();
let contact = mimemsg.from;
assert_eq!(contact.addr, "g@c.de");
assert_eq!(contact.display_name, Some("Götz C".to_string()));
// although RFC 2047 says, encoded-words shall not appear inside quoted-string,
// this combination is used in the wild eg. by MailMate
let mimemsg = MimeMessage::from_bytes(
&ctx,
b"From: \"=?utf-8?q?G=C3=B6tz?= C\" <g@c.de>\n\nhi",
None,
)
.await
.unwrap();
let mimemsg =
MimeMessage::from_bytes(&ctx, b"From: \"=?utf-8?q?G=C3=B6tz?= C\" <g@c.de>\n\nhi")
.await
.unwrap();
let contact = mimemsg.from;
assert_eq!(contact.addr, "g@c.de");
assert_eq!(contact.display_name, Some("Götz C".to_string()));
@@ -86,7 +82,7 @@ async fn test_mimeparser_fromheader() {
async fn test_mimeparser_crash() {
let context = TestContext::new_alice().await;
let raw = include_bytes!("../../test-data/message/issue_523.txt");
let mimeparser = MimeMessage::from_bytes(&context.ctx, &raw[..], None)
let mimeparser = MimeMessage::from_bytes(&context.ctx, &raw[..])
.await
.unwrap();
@@ -98,7 +94,7 @@ async fn test_mimeparser_crash() {
async fn test_get_rfc724_mid_exists() {
let context = TestContext::new_alice().await;
let raw = include_bytes!("../../test-data/message/mail_with_message_id.txt");
let mimeparser = MimeMessage::from_bytes(&context.ctx, &raw[..], None)
let mimeparser = MimeMessage::from_bytes(&context.ctx, &raw[..])
.await
.unwrap();
@@ -112,7 +108,7 @@ async fn test_get_rfc724_mid_exists() {
async fn test_get_rfc724_mid_not_exists() {
let context = TestContext::new_alice().await;
let raw = include_bytes!("../../test-data/message/issue_523.txt");
let mimeparser = MimeMessage::from_bytes(&context.ctx, &raw[..], None)
let mimeparser = MimeMessage::from_bytes(&context.ctx, &raw[..])
.await
.unwrap();
assert_eq!(mimeparser.get_rfc724_mid(), None);
@@ -324,7 +320,7 @@ async fn test_mailparse_0_16_0_panic() {
// There should be an error, but no panic.
assert!(
MimeMessage::from_bytes(&context.ctx, &raw[..], None)
MimeMessage::from_bytes(&context.ctx, &raw[..])
.await
.is_err()
);
@@ -341,7 +337,7 @@ async fn test_parse_first_addr() {
test1\n\
";
let mimeparser = MimeMessage::from_bytes(&context.ctx, &raw[..], None).await;
let mimeparser = MimeMessage::from_bytes(&context.ctx, &raw[..]).await;
assert!(mimeparser.is_err());
}
@@ -356,7 +352,7 @@ async fn test_get_parent_timestamp() {
\n\
Some reply\n\
";
let mimeparser = MimeMessage::from_bytes(&context.ctx, &raw[..], None)
let mimeparser = MimeMessage::from_bytes(&context.ctx, &raw[..])
.await
.unwrap();
assert_eq!(
@@ -402,7 +398,7 @@ async fn test_mimeparser_with_context() {
--==break==--\n\
\n";
let mimeparser = MimeMessage::from_bytes(&context.ctx, &raw[..], None)
let mimeparser = MimeMessage::from_bytes(&context.ctx, &raw[..])
.await
.unwrap();
@@ -438,26 +434,26 @@ async fn test_mimeparser_with_avatars() {
let t = TestContext::new_alice().await;
let raw = include_bytes!("../../test-data/message/mail_attach_txt.eml");
let mimeparser = MimeMessage::from_bytes(&t, &raw[..], None).await.unwrap();
let mimeparser = MimeMessage::from_bytes(&t, &raw[..]).await.unwrap();
assert_eq!(mimeparser.user_avatar, None);
assert_eq!(mimeparser.group_avatar, None);
let raw = include_bytes!("../../test-data/message/mail_with_user_avatar.eml");
let mimeparser = MimeMessage::from_bytes(&t, &raw[..], None).await.unwrap();
let mimeparser = MimeMessage::from_bytes(&t, &raw[..]).await.unwrap();
assert_eq!(mimeparser.parts.len(), 1);
assert_eq!(mimeparser.parts[0].typ, Viewtype::Text);
assert!(mimeparser.user_avatar.unwrap().is_change());
assert_eq!(mimeparser.group_avatar, None);
let raw = include_bytes!("../../test-data/message/mail_with_user_avatar_deleted.eml");
let mimeparser = MimeMessage::from_bytes(&t, &raw[..], None).await.unwrap();
let mimeparser = MimeMessage::from_bytes(&t, &raw[..]).await.unwrap();
assert_eq!(mimeparser.parts.len(), 1);
assert_eq!(mimeparser.parts[0].typ, Viewtype::Text);
assert_eq!(mimeparser.user_avatar, Some(AvatarAction::Delete));
assert_eq!(mimeparser.group_avatar, None);
let raw = include_bytes!("../../test-data/message/mail_with_user_and_group_avatars.eml");
let mimeparser = MimeMessage::from_bytes(&t, &raw[..], None).await.unwrap();
let mimeparser = MimeMessage::from_bytes(&t, &raw[..]).await.unwrap();
assert_eq!(mimeparser.parts.len(), 1);
assert_eq!(mimeparser.parts[0].typ, Viewtype::Text);
assert!(mimeparser.user_avatar.unwrap().is_change());
@@ -467,9 +463,7 @@ async fn test_mimeparser_with_avatars() {
let raw = include_bytes!("../../test-data/message/mail_with_user_and_group_avatars.eml");
let raw = String::from_utf8_lossy(raw).to_string();
let raw = raw.replace("Chat-User-Avatar:", "Xhat-Xser-Xvatar:");
let mimeparser = MimeMessage::from_bytes(&t, raw.as_bytes(), None)
.await
.unwrap();
let mimeparser = MimeMessage::from_bytes(&t, raw.as_bytes()).await.unwrap();
assert_eq!(mimeparser.parts.len(), 1);
assert_eq!(mimeparser.parts[0].typ, Viewtype::Image);
assert_eq!(mimeparser.user_avatar, None);
@@ -485,7 +479,7 @@ async fn test_mimeparser_with_videochat() {
let t = TestContext::new_alice().await;
let raw = include_bytes!("../../test-data/message/videochat_invitation.eml");
let mimeparser = MimeMessage::from_bytes(&t, &raw[..], None).await.unwrap();
let mimeparser = MimeMessage::from_bytes(&t, &raw[..]).await.unwrap();
assert_eq!(mimeparser.parts.len(), 1);
assert_eq!(mimeparser.parts[0].typ, Viewtype::Text);
assert_eq!(mimeparser.parts[0].param.get(Param::WebrtcRoom), None);
@@ -528,7 +522,7 @@ Content-Disposition: attachment; filename=\"message.kml\"\n\
--==break==--\n\
;";
let mimeparser = MimeMessage::from_bytes(&context.ctx, &raw[..], None)
let mimeparser = MimeMessage::from_bytes(&context.ctx, &raw[..])
.await
.unwrap();
assert_eq!(
@@ -578,7 +572,7 @@ Disposition: manual-action/MDN-sent-automatically; displayed\n\
--kJBbU58X1xeWNHgBtTbMk80M5qnV4N--\n\
";
let message = MimeMessage::from_bytes(&context.ctx, &raw[..], None)
let message = MimeMessage::from_bytes(&context.ctx, &raw[..])
.await
.unwrap();
assert_eq!(
@@ -659,7 +653,7 @@ Disposition: manual-action/MDN-sent-automatically; displayed\n\
--outer--\n\
";
let message = MimeMessage::from_bytes(&context.ctx, &raw[..], None)
let message = MimeMessage::from_bytes(&context.ctx, &raw[..])
.await
.unwrap();
assert_eq!(
@@ -706,7 +700,7 @@ Additional-Message-IDs: <foo@example.com> <foo@example.net>\n\
--kJBbU58X1xeWNHgBtTbMk80M5qnV4N--\n\
";
let message = MimeMessage::from_bytes(&context.ctx, &raw[..], None)
let message = MimeMessage::from_bytes(&context.ctx, &raw[..])
.await
.unwrap();
assert_eq!(
@@ -753,7 +747,7 @@ MDYyMDYxNTE1RTlDOEE4Cj4+CnN0YXJ0eHJlZgo4Mjc4CiUlRU9GCg==
------=_Part_25_46172632.1581201680436--
"#;
let message = MimeMessage::from_bytes(&context.ctx, &raw[..], None)
let message = MimeMessage::from_bytes(&context.ctx, &raw[..])
.await
.unwrap();
assert_eq!(
@@ -797,7 +791,7 @@ MDYyMDYxNTE1RTlDOEE4Cj4+CnN0YXJ0eHJlZgo4Mjc4CiUlRU9GCg==
------=_Part_25_46172632.1581201680436--
"#;
let message = MimeMessage::from_bytes(&t, &raw[..], None).await.unwrap();
let message = MimeMessage::from_bytes(&t, &raw[..]).await.unwrap();
assert_eq!(message.parts.len(), 1);
assert_eq!(message.parts[0].typ, Viewtype::File);
@@ -839,7 +833,7 @@ iVBORw0KGgoAAAANSUhEUgAAACAAAAAeCAAAAABoYUP1AAAAAXNSR0IArs4c6QAAAo1JREFUKJFdkdFu
----11019878869865180--
"#;
let message = MimeMessage::from_bytes(&context.ctx, &raw[..], None)
let message = MimeMessage::from_bytes(&context.ctx, &raw[..])
.await
.unwrap();
assert_eq!(message.get_subject(), Some("example".to_string()));
@@ -903,7 +897,7 @@ iVBORw0KGgoAAAANSUhEUgAAACAAAAAeCAAAAABoYUP1AAAAAXNSR0IArs4c6QAAAo1JREFUKJFdkdFu
--------------779C1631600DF3DB8C02E53A--"#;
let message = MimeMessage::from_bytes(&context.ctx, &raw[..], None)
let message = MimeMessage::from_bytes(&context.ctx, &raw[..])
.await
.unwrap();
assert_eq!(message.get_subject(), Some("Test subject".to_string()));
@@ -966,7 +960,7 @@ iVBORw0KGgoAAAANSUhEUgAAACAAAAAeCAAAAABoYUP1AAAAAXNSR0IArs4c6QAAAo1JREFUKJFdkdFu
------=_NextPart_000_0003_01D622B3.CA753E60--
"#;
let message = MimeMessage::from_bytes(&context.ctx, &raw[..], None)
let message = MimeMessage::from_bytes(&context.ctx, &raw[..])
.await
.unwrap();
assert_eq!(
@@ -1064,7 +1058,7 @@ From: alice <alice@example.org>
Reply
"##;
let message = MimeMessage::from_bytes(&context.ctx, &raw[..], None)
let message = MimeMessage::from_bytes(&context.ctx, &raw[..])
.await
.unwrap();
assert_eq!(
@@ -1096,7 +1090,7 @@ From: alice <alice@example.org>
> Just a quote.
"##;
let message = MimeMessage::from_bytes(&context.ctx, &raw[..], None)
let message = MimeMessage::from_bytes(&context.ctx, &raw[..])
.await
.unwrap();
assert_eq!(
@@ -1130,7 +1124,7 @@ On 2020-10-25, Bob wrote:
> A quote.
"##;
let message = MimeMessage::from_bytes(&context.ctx, &raw[..], None)
let message = MimeMessage::from_bytes(&context.ctx, &raw[..])
.await
.unwrap();
assert_eq!(message.get_subject(), Some("Re: top posting".to_string()));
@@ -1148,7 +1142,7 @@ On 2020-10-25, Bob wrote:
async fn test_attachment_quote() {
let context = TestContext::new_alice().await;
let raw = include_bytes!("../../test-data/message/quote_attach.eml");
let mimeparser = MimeMessage::from_bytes(&context.ctx, &raw[..], None)
let mimeparser = MimeMessage::from_bytes(&context.ctx, &raw[..])
.await
.unwrap();
@@ -1166,7 +1160,7 @@ async fn test_attachment_quote() {
async fn test_quote_div() {
let t = TestContext::new_alice().await;
let raw = include_bytes!("../../test-data/message/gmx-quote.eml");
let mimeparser = MimeMessage::from_bytes(&t, raw, None).await.unwrap();
let mimeparser = MimeMessage::from_bytes(&t, raw).await.unwrap();
assert_eq!(mimeparser.parts[0].msg, "YIPPEEEEEE\n\nMulti-line");
assert_eq!(mimeparser.parts[0].param.get(Param::Quote).unwrap(), "Now?");
}
@@ -1176,7 +1170,7 @@ async fn test_allinkl_blockquote() {
// all-inkl.com puts quotes into `<blockquote> </blockquote>`.
let t = TestContext::new_alice().await;
let raw = include_bytes!("../../test-data/message/allinkl-quote.eml");
let mimeparser = MimeMessage::from_bytes(&t, raw, None).await.unwrap();
let mimeparser = MimeMessage::from_bytes(&t, raw).await.unwrap();
assert!(mimeparser.parts[0].msg.starts_with("It's 1.0."));
assert_eq!(
mimeparser.parts[0].param.get(Param::Quote).unwrap(),
@@ -1217,7 +1211,7 @@ async fn test_add_subj_to_multimedia_msg() {
async fn test_mime_modified_plain() {
let t = TestContext::new_alice().await;
let raw = include_bytes!("../../test-data/message/text_plain_unspecified.eml");
let mimeparser = MimeMessage::from_bytes(&t.ctx, raw, None).await.unwrap();
let mimeparser = MimeMessage::from_bytes(&t.ctx, raw).await.unwrap();
assert!(!mimeparser.is_mime_modified);
assert_eq!(
mimeparser.parts[0].msg,
@@ -1229,7 +1223,7 @@ async fn test_mime_modified_plain() {
async fn test_mime_modified_alt_plain_html() {
let t = TestContext::new_alice().await;
let raw = include_bytes!("../../test-data/message/text_alt_plain_html.eml");
let mimeparser = MimeMessage::from_bytes(&t.ctx, raw, None).await.unwrap();
let mimeparser = MimeMessage::from_bytes(&t.ctx, raw).await.unwrap();
assert!(mimeparser.is_mime_modified);
assert_eq!(
mimeparser.parts[0].msg,
@@ -1241,7 +1235,7 @@ async fn test_mime_modified_alt_plain_html() {
async fn test_mime_modified_alt_plain() {
let t = TestContext::new_alice().await;
let raw = include_bytes!("../../test-data/message/text_alt_plain.eml");
let mimeparser = MimeMessage::from_bytes(&t.ctx, raw, None).await.unwrap();
let mimeparser = MimeMessage::from_bytes(&t.ctx, raw).await.unwrap();
assert!(!mimeparser.is_mime_modified);
assert_eq!(
mimeparser.parts[0].msg,
@@ -1256,7 +1250,7 @@ async fn test_mime_modified_alt_plain() {
async fn test_mime_modified_alt_html() {
let t = TestContext::new_alice().await;
let raw = include_bytes!("../../test-data/message/text_alt_html.eml");
let mimeparser = MimeMessage::from_bytes(&t.ctx, raw, None).await.unwrap();
let mimeparser = MimeMessage::from_bytes(&t.ctx, raw).await.unwrap();
assert!(mimeparser.is_mime_modified);
assert_eq!(
mimeparser.parts[0].msg,
@@ -1268,7 +1262,7 @@ async fn test_mime_modified_alt_html() {
async fn test_mime_modified_html() {
let t = TestContext::new_alice().await;
let raw = include_bytes!("../../test-data/message/text_html.eml");
let mimeparser = MimeMessage::from_bytes(&t.ctx, raw, None).await.unwrap();
let mimeparser = MimeMessage::from_bytes(&t.ctx, raw).await.unwrap();
assert!(mimeparser.is_mime_modified);
assert_eq!(
mimeparser.parts[0].msg,
@@ -1288,7 +1282,7 @@ async fn test_mime_modified_large_plain() -> Result<()> {
assert!(long_txt.len() > DC_DESIRED_TEXT_LEN);
{
let mimemsg = MimeMessage::from_bytes(&t, long_txt.as_ref(), None).await?;
let mimemsg = MimeMessage::from_bytes(&t, long_txt.as_ref()).await?;
assert!(mimemsg.is_mime_modified);
assert!(
mimemsg.parts[0].msg.matches("just repeated").count()
@@ -1321,7 +1315,7 @@ async fn test_mime_modified_large_plain() -> Result<()> {
t.set_config(Config::Bot, Some("1")).await?;
{
let mimemsg = MimeMessage::from_bytes(&t, long_txt.as_ref(), None).await?;
let mimemsg = MimeMessage::from_bytes(&t, long_txt.as_ref()).await?;
assert!(!mimemsg.is_mime_modified);
assert_eq!(
format!("{}\n", mimemsg.parts[0].msg),
@@ -1368,7 +1362,7 @@ async fn test_x_microsoft_original_message_id() {
MIME-Version: 1.0\n\
\n\
Does it work with outlook now?\n\
", None)
")
.await
.unwrap();
assert_eq!(
@@ -1418,7 +1412,7 @@ async fn test_extra_imf_headers() -> Result<()> {
"Message-ID:",
"Chat-Forty-Two: 42\r\nForty-Two: 42\r\nMessage-ID:",
);
let msg = MimeMessage::from_bytes(t, payload.as_bytes(), None).await?;
let msg = MimeMessage::from_bytes(t, payload.as_bytes()).await?;
assert!(msg.headers.contains_key("chat-version"));
assert!(!msg.headers.contains_key("chat-forty-two"));
assert_ne!(msg.headers.contains_key("forty-two"), std_hp_composing);
@@ -1582,7 +1576,7 @@ async fn test_ms_exchange_mdn() -> Result<()> {
// 1. Test mimeparser directly
let mdn =
include_bytes!("../../test-data/message/ms_exchange_report_disposition_notification.eml");
let mimeparser = MimeMessage::from_bytes(&t.ctx, mdn, None).await?;
let mimeparser = MimeMessage::from_bytes(&t.ctx, mdn).await?;
assert_eq!(mimeparser.mdn_reports.len(), 1);
assert_eq!(
mimeparser.mdn_reports[0].original_message_id.as_deref(),
@@ -1608,7 +1602,6 @@ async fn test_receive_eml() -> Result<()> {
let mime_message = MimeMessage::from_bytes(
&alice,
include_bytes!("../../test-data/message/attached-eml.eml"),
None,
)
.await?;
@@ -1651,7 +1644,6 @@ Content-Disposition: reaction\n\
\n\
\u{1F44D}"
.as_bytes(),
None,
)
.await?;
@@ -1673,7 +1665,7 @@ async fn test_jpeg_as_application_octet_stream() -> Result<()> {
let context = TestContext::new_alice().await;
let raw = include_bytes!("../../test-data/message/jpeg-as-application-octet-stream.eml");
let msg = MimeMessage::from_bytes(&context.ctx, &raw[..], None)
let msg = MimeMessage::from_bytes(&context.ctx, &raw[..])
.await
.unwrap();
assert_eq!(msg.parts.len(), 1);
@@ -1691,7 +1683,7 @@ async fn test_schleuder() -> Result<()> {
let context = TestContext::new_alice().await;
let raw = include_bytes!("../../test-data/message/schleuder.eml");
let msg = MimeMessage::from_bytes(&context.ctx, &raw[..], None)
let msg = MimeMessage::from_bytes(&context.ctx, &raw[..])
.await
.unwrap();
assert_eq!(msg.parts.len(), 2);
@@ -1711,7 +1703,7 @@ async fn test_tlsrpt() -> Result<()> {
let context = TestContext::new_alice().await;
let raw = include_bytes!("../../test-data/message/tlsrpt.eml");
let msg = MimeMessage::from_bytes(&context.ctx, &raw[..], None)
let msg = MimeMessage::from_bytes(&context.ctx, &raw[..])
.await
.unwrap();
assert_eq!(msg.parts.len(), 1);
@@ -1744,7 +1736,6 @@ async fn test_time_in_future() -> Result<()> {
Content-Type: text/plain; charset=utf-8\n\
\n\
Hi",
None,
)
.await?;
@@ -1806,7 +1797,7 @@ Content-Type: text/plain; charset=utf-8
/help
"#;
let message = MimeMessage::from_bytes(&context.ctx, &raw[..], None)
let message = MimeMessage::from_bytes(&context.ctx, &raw[..])
.await
.unwrap();
assert_eq!(message.get_subject(), Some("Some subject".to_string()));
@@ -1847,7 +1838,7 @@ async fn test_take_last_header() {
Hello\n\
";
let mimeparser = MimeMessage::from_bytes(&context.ctx, &raw[..], None)
let mimeparser = MimeMessage::from_bytes(&context.ctx, &raw[..])
.await
.unwrap();
assert_eq!(
@@ -1900,9 +1891,7 @@ It DOES end with a linebreak.\r
\r
This is the epilogue. It is also to be ignored.";
let mimeparser = MimeMessage::from_bytes(&context, &raw[..], None)
.await
.unwrap();
let mimeparser = MimeMessage::from_bytes(&context, &raw[..]).await.unwrap();
assert_eq!(mimeparser.parts.len(), 2);
@@ -1948,7 +1937,7 @@ Message with a correct Message-ID hidden header
--luTiGu6GBoVLCvTkzVtmZmwsmhkNMw--
"#;
let message = MimeMessage::from_bytes(t, &raw[..], None).await.unwrap();
let message = MimeMessage::from_bytes(t, &raw[..]).await.unwrap();
assert_eq!(message.get_rfc724_mid().unwrap(), "foo@example.org");
}
@@ -2126,9 +2115,7 @@ Third alternative.
--boundary--
"#;
let message = MimeMessage::from_bytes(context, &raw[..], None)
.await
.unwrap();
let message = MimeMessage::from_bytes(context, &raw[..]).await.unwrap();
assert_eq!(message.parts.len(), 1);
assert_eq!(message.parts[0].typ, Viewtype::Text);
assert_eq!(message.parts[0].msg, "Third alternative.");
+30
View File
@@ -251,6 +251,13 @@ pub enum Param {
/// For info messages: Contact ID in added or removed to a group.
ContactAddedRemoved = b'5',
/// For (pre-)Message: ViewType of the Post-Message,
/// because pre message is always `Viewtype::Text`.
PostMessageViewtype = b'8',
/// For (pre-)Message: File byte size of Post-Message attachment
PostMessageFileBytes = b'9',
}
/// An object for handling key=value parameter lists.
@@ -441,6 +448,15 @@ impl Params {
}
self
}
/// Merge in parameters from other Params struct,
/// overwriting the keys that are in both
/// with the values from the new Params struct.
pub fn merge_in_params(&mut self, new_params: Self) -> &mut Self {
let mut new_params = new_params;
self.inner.append(&mut new_params.inner);
self
}
}
#[cfg(test)]
@@ -503,4 +519,18 @@ mod tests {
assert_eq!(p.get(Param::Height), Some("14"));
Ok(())
}
#[test]
fn test_merge() -> Result<()> {
let mut p = Params::from_str("w=12\na=5\nh=14")?;
let p2 = Params::from_str("L=1\nh=17")?;
assert_eq!(p.len(), 3);
p.merge_in_params(p2);
assert_eq!(p.len(), 4);
assert_eq!(p.get(Param::Width), Some("12"));
assert_eq!(p.get(Param::Height), Some("17"));
assert_eq!(p.get(Param::Forwarded), Some("5"));
assert_eq!(p.get(Param::IsEdited), Some("1"));
Ok(())
}
}
+9 -7
View File
@@ -17,7 +17,9 @@ use pgp::crypto::ecc_curve::ECCCurve;
use pgp::crypto::hash::HashAlgorithm;
use pgp::crypto::sym::SymmetricKeyAlgorithm;
use pgp::packet::{SignatureConfig, SignatureType, Subpacket, SubpacketData};
use pgp::types::{CompressionAlgorithm, KeyDetails, Password, PublicKeyTrait, StringToKey};
use pgp::types::{
CompressionAlgorithm, KeyDetails, Password, PublicKeyTrait, SecretKeyTrait as _, StringToKey,
};
use rand_old::{Rng as _, thread_rng};
use tokio::runtime::Handle;
@@ -31,9 +33,6 @@ pub(crate) const HEADER_SETUPCODE: &str = "passphrase-begin";
/// Preferred symmetric encryption algorithm.
const SYMMETRIC_KEY_ALGORITHM: SymmetricKeyAlgorithm = SymmetricKeyAlgorithm::AES128;
/// Preferred cryptographic hash.
const HASH_ALGORITHM: HashAlgorithm = HashAlgorithm::Sha256;
/// Split data from PGP Armored Data as defined in <https://tools.ietf.org/html/rfc4880#section-6.2>.
///
/// Returns (type, headers, base64 encoded body).
@@ -205,7 +204,8 @@ pub async fn pk_encrypt(
}
}
msg.sign(&*private_key_for_signing, Password::empty(), HASH_ALGORITHM);
let hash_algorithm = private_key_for_signing.hash_alg();
msg.sign(&*private_key_for_signing, Password::empty(), hash_algorithm);
if compress {
msg.compression(CompressionAlgorithm::ZLIB);
}
@@ -228,7 +228,8 @@ pub async fn pk_encrypt(
}
}
msg.sign(&*private_key_for_signing, Password::empty(), HASH_ALGORITHM);
let hash_algorithm = private_key_for_signing.hash_alg();
msg.sign(&*private_key_for_signing, Password::empty(), hash_algorithm);
if compress {
msg.compression(CompressionAlgorithm::ZLIB);
}
@@ -453,7 +454,8 @@ pub async fn symm_encrypt_message(
);
msg.encrypt_with_password(&mut rng, s2k, &shared_secret)?;
msg.sign(&*private_key_for_signing, Password::empty(), HASH_ALGORITHM);
let hash_algorithm = private_key_for_signing.hash_alg();
msg.sign(&*private_key_for_signing, Password::empty(), hash_algorithm);
if compress {
msg.compression(CompressionAlgorithm::ZLIB);
}
+4 -2
View File
@@ -834,6 +834,8 @@ pub(crate) async fn login_param_from_account_qr(
}
/// Sets configuration values from a QR code.
/// "DCACCOUNT:" and "DCLOGIN:" QR codes configure `context`, but I/O mustn't be started for such QR
/// codes, consider using [`Context::add_transport_from_qr`] which also restarts I/O.
pub async fn set_config_from_qr(context: &Context, qr: &str) -> Result<()> {
match check_qr(context, qr).await? {
Qr::Account { .. } => {
@@ -904,7 +906,7 @@ pub async fn set_config_from_qr(context: &Context, qr: &str) -> Result<()> {
.await?;
token::save(context, token::Namespace::Auth, None, &authcode, timestamp).await?;
context.sync_qr_code_tokens(None).await?;
context.scheduler.interrupt_inbox().await;
context.scheduler.interrupt_smtp().await;
}
Qr::ReviveVerifyGroup {
invitenumber,
@@ -936,7 +938,7 @@ pub async fn set_config_from_qr(context: &Context, qr: &str) -> Result<()> {
)
.await?;
context.sync_qr_code_tokens(Some(&grpid)).await?;
context.scheduler.interrupt_inbox().await;
context.scheduler.interrupt_smtp().await;
}
Qr::Login { address, options } => {
let mut param = login_param_from_login_qr(&address, options)?;
+31 -5
View File
@@ -59,14 +59,12 @@ pub enum LoginOptions {
/// scheme: `dclogin://user@host/?p=password&v=1[&options]`
/// read more about the scheme at <https://github.com/deltachat/interface/blob/master/uri-schemes.md#DCLOGIN>
pub(super) fn decode_login(qr: &str) -> Result<Qr> {
let url = url::Url::parse(qr).with_context(|| format!("Malformed url: {qr:?}"))?;
let qr = qr.replacen("://", ":", 1);
let url_without_scheme = qr
let url = url::Url::parse(&qr).with_context(|| format!("Malformed url: {qr:?}"))?;
let payload = qr
.get(DCLOGIN_SCHEME.len()..)
.context("invalid DCLOGIN payload E1")?;
let payload = url_without_scheme
.strip_prefix("//")
.unwrap_or(url_without_scheme);
let addr = payload
.split(['?', '/'])
@@ -365,4 +363,32 @@ mod test {
}
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_decode_dclogin_ipv4() -> anyhow::Result<()> {
let result = decode_login("dclogin://test@[127.0.0.1]?p=1234&v=1")?;
if let Qr::Login { address, options } = result {
assert_eq!(address, "test@[127.0.0.1]".to_owned());
assert_eq!(options, login_options_just_pw!("1234".to_owned()));
} else {
unreachable!("wrong type");
}
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_decode_dclogin_ipv6() -> anyhow::Result<()> {
let result =
decode_login("dclogin://test@[2001:0db8:85a3:0000:0000:8a2e:0370:7334]?p=1234&v=1")?;
if let Qr::Login { address, options } = result {
assert_eq!(
address,
"test@[2001:0db8:85a3:0000:0000:8a2e:0370:7334]".to_owned()
);
assert_eq!(options, login_options_just_pw!("1234".to_owned()));
} else {
unreachable!("wrong type");
}
Ok(())
}
}
+37 -19
View File
@@ -107,10 +107,10 @@ pub fn needs_quota_warning(curr_percentage: u64, warned_at_percentage: u64) -> b
impl Context {
/// Returns whether the quota value needs an update. If so, `update_recent_quota()` should be
/// called.
pub(crate) async fn quota_needs_update(&self, ratelimit_secs: u64) -> bool {
pub(crate) async fn quota_needs_update(&self, transport_id: u32, ratelimit_secs: u64) -> bool {
let quota = self.quota.read().await;
quota
.as_ref()
.get(&transport_id)
.filter(|quota| time_elapsed(&quota.modified) < Duration::from_secs(ratelimit_secs))
.is_none()
}
@@ -155,10 +155,13 @@ impl Context {
}
}
*self.quota.write().await = Some(QuotaInfo {
recent: quota,
modified: tools::Time::now(),
});
self.quota.write().await.insert(
session.transport_id(),
QuotaInfo {
recent: quota,
modified: tools::Time::now(),
},
);
self.emit_event(EventType::ConnectivityChanged);
Ok(())
@@ -203,27 +206,42 @@ mod tests {
let mut tcm = TestContextManager::new();
let t = &tcm.unconfigured().await;
const TIMEOUT: u64 = 60;
assert!(t.quota_needs_update(TIMEOUT).await);
assert!(t.quota_needs_update(0, TIMEOUT).await);
*t.quota.write().await = Some(QuotaInfo {
recent: Ok(Default::default()),
modified: tools::Time::now() - Duration::from_secs(TIMEOUT + 1),
});
assert!(t.quota_needs_update(TIMEOUT).await);
*t.quota.write().await = {
let mut map = BTreeMap::new();
map.insert(
0,
QuotaInfo {
recent: Ok(Default::default()),
modified: tools::Time::now() - Duration::from_secs(TIMEOUT + 1),
},
);
map
};
assert!(t.quota_needs_update(0, TIMEOUT).await);
*t.quota.write().await = Some(QuotaInfo {
recent: Ok(Default::default()),
modified: tools::Time::now(),
});
assert!(!t.quota_needs_update(TIMEOUT).await);
*t.quota.write().await = {
let mut map = BTreeMap::new();
map.insert(
0,
QuotaInfo {
recent: Ok(Default::default()),
modified: tools::Time::now(),
},
);
map
};
assert!(!t.quota_needs_update(0, TIMEOUT).await);
t.evtracker.clear_events();
t.set_primary_self_addr("new@addr").await?;
assert!(t.quota.read().await.is_none());
assert!(t.quota.read().await.is_empty());
t.evtracker
.get_matching(|evt| matches!(evt, EventType::ConnectivityChanged))
.await;
assert!(t.quota_needs_update(TIMEOUT).await);
assert!(t.quota_needs_update(0, TIMEOUT).await);
Ok(())
}
}
+13 -71
View File
@@ -392,9 +392,8 @@ mod tests {
use crate::chatlist::Chatlist;
use crate::config::Config;
use crate::contact::{Contact, Origin};
use crate::download::DownloadState;
use crate::message::{MessageState, Viewtype, delete_msgs};
use crate::receive_imf::{receive_imf, receive_imf_from_inbox};
use crate::receive_imf::receive_imf;
use crate::sql::housekeeping;
use crate::test_utils::E2EE_INFO_MSGS;
use crate::test_utils::TestContext;
@@ -748,13 +747,23 @@ Content-Disposition: reaction\n\
alice_reaction_msg.id.get_state(&alice).await?,
MessageState::InSeen
);
// Reactions don't request MDNs.
// Reactions don't request MDNs, but an MDN to self is sent.
assert_eq!(
alice
.sql
.count("SELECT COUNT(*) FROM smtp_mdns", ())
.await?,
0
1
);
assert_eq!(
alice
.sql
.count(
"SELECT COUNT(*) FROM smtp_mdns WHERE from_id=?",
(ContactId::SELF,)
)
.await?,
1
);
// Alice reacts to own message.
@@ -924,73 +933,6 @@ Content-Disposition: reaction\n\
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_partial_download_and_reaction() -> Result<()> {
let alice = TestContext::new_alice().await;
let bob = TestContext::new_bob().await;
alice
.create_chat_with_contact("Bob", "bob@example.net")
.await;
let msg_header = "From: Bob <bob@example.net>\n\
To: Alice <alice@example.org>\n\
Chat-Version: 1.0\n\
Subject: subject\n\
Message-ID: <first@example.org>\n\
Date: Sun, 14 Nov 2021 00:10:00 +0000\
Content-Type: text/plain";
let msg_full = format!("{msg_header}\n\n100k text...");
// Alice downloads message from Bob partially.
let alice_received_message = receive_imf_from_inbox(
&alice,
"first@example.org",
msg_header.as_bytes(),
false,
Some(100000),
)
.await?
.unwrap();
let alice_msg_id = *alice_received_message.msg_ids.first().unwrap();
// Bob downloads own message on the other device.
let bob_received_message = receive_imf(&bob, msg_full.as_bytes(), false)
.await?
.unwrap();
let bob_msg_id = *bob_received_message.msg_ids.first().unwrap();
// Bob reacts to own message.
send_reaction(&bob, bob_msg_id, "👍").await.unwrap();
let bob_reaction_msg = bob.pop_sent_msg().await;
// Alice receives a reaction.
alice.recv_msg_hidden(&bob_reaction_msg).await;
let reactions = get_msg_reactions(&alice, alice_msg_id).await?;
assert_eq!(reactions.to_string(), "👍1");
let msg = Message::load_from_db(&alice, alice_msg_id).await?;
assert_eq!(msg.download_state(), DownloadState::Available);
// Alice downloads full message.
receive_imf_from_inbox(
&alice,
"first@example.org",
msg_full.as_bytes(),
false,
None,
)
.await?;
// Check that reaction is still on the message after full download.
let msg = Message::load_from_db(&alice, alice_msg_id).await?;
assert_eq!(msg.download_state(), DownloadState::Done);
let reactions = get_msg_reactions(&alice, alice_msg_id).await?;
assert_eq!(reactions.to_string(), "👍1");
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_send_reaction_multidevice() -> Result<()> {
let mut tcm = TestContextManager::new();
+279 -126
View File
@@ -1,5 +1,6 @@
//! Internet Message Format reception pipeline.
use std::cmp;
use std::collections::{BTreeMap, BTreeSet, HashSet};
use std::iter;
use std::sync::LazyLock;
@@ -14,31 +15,33 @@ use mailparse::SingleInfo;
use num_traits::FromPrimitive;
use regex::Regex;
use crate::chat::{self, Chat, ChatId, ChatIdBlocked, save_broadcast_secret};
use crate::chat::{self, Chat, ChatId, ChatIdBlocked, ChatVisibility, save_broadcast_secret};
use crate::config::Config;
use crate::constants::{self, Blocked, Chattype, DC_CHAT_ID_TRASH, EDITED_PREFIX, ShowEmails};
use crate::contact::{self, Contact, ContactId, Origin, mark_contact_id_as_verified};
use crate::context::Context;
use crate::debug_logging::maybe_set_logging_xdc_inner;
use crate::download::DownloadState;
use crate::download::{DownloadState, msg_is_downloaded_for};
use crate::ephemeral::{Timer as EphemeralTimer, stock_ephemeral_timer_changed};
use crate::events::EventType;
use crate::headerdef::{HeaderDef, HeaderDefMap};
use crate::imap::{GENERATED_PREFIX, markseen_on_imap_table};
use crate::key::{DcKey, Fingerprint};
use crate::key::{self_fingerprint, self_fingerprint_opt};
use crate::log::LogExt;
use crate::log::warn;
use crate::logged_debug_assert;
use crate::log::{LogExt as _, warn};
use crate::message::{
self, Message, MessageState, MessengerMessage, MsgId, Viewtype, rfc724_mid_exists,
};
use crate::mimeparser::{AvatarAction, GossipedKey, MimeMessage, SystemMessage, parse_message_ids};
use crate::mimeparser::{
AvatarAction, GossipedKey, MimeMessage, PreMessageMode, SystemMessage, parse_message_ids,
};
use crate::param::{Param, Params};
use crate::peer_channels::{add_gossip_peer_from_header, insert_topic_stub};
use crate::reaction::{Reaction, set_msg_reaction};
use crate::rusqlite::OptionalExtension;
use crate::securejoin::{self, handle_securejoin_handshake, observe_securejoin_on_other_device};
use crate::securejoin::{
self, get_secure_join_step, handle_securejoin_handshake, observe_securejoin_on_other_device,
};
use crate::simplify;
use crate::stats::STATISTICS_BOT_EMAIL;
use crate::stock_str;
@@ -47,6 +50,7 @@ use crate::tools::{
self, buf_compress, normalize_text, remove_subject_prefix, validate_broadcast_secret,
};
use crate::{chatlist_events, ensure_and_debug_assert, ensure_and_debug_assert_eq, location};
use crate::{logged_debug_assert, mimeparser};
/// This is the struct that is returned after receiving one email (aka MIME message).
///
@@ -157,24 +161,7 @@ pub async fn receive_imf(
let mail = mailparse::parse_mail(imf_raw).context("can't parse mail")?;
let rfc724_mid = crate::imap::prefetch_get_message_id(&mail.headers)
.unwrap_or_else(crate::imap::create_message_id);
if let Some(download_limit) = context.download_limit().await? {
let download_limit: usize = download_limit.try_into()?;
if imf_raw.len() > download_limit {
let head = std::str::from_utf8(imf_raw)?
.split("\r\n\r\n")
.next()
.context("No empty line in the message")?;
return receive_imf_from_inbox(
context,
&rfc724_mid,
head.as_bytes(),
seen,
Some(imf_raw.len().try_into()?),
)
.await;
}
}
receive_imf_from_inbox(context, &rfc724_mid, imf_raw, seen, None).await
receive_imf_from_inbox(context, &rfc724_mid, imf_raw, seen).await
}
/// Emulates reception of a message from "INBOX".
@@ -186,9 +173,8 @@ pub(crate) async fn receive_imf_from_inbox(
rfc724_mid: &str,
imf_raw: &[u8],
seen: bool,
is_partial_download: Option<u32>,
) -> Result<Option<ReceivedMsg>> {
receive_imf_inner(context, rfc724_mid, imf_raw, seen, is_partial_download).await
receive_imf_inner(context, rfc724_mid, imf_raw, seen).await
}
/// Inserts a tombstone into `msgs` table
@@ -211,7 +197,6 @@ async fn get_to_and_past_contact_ids(
context: &Context,
mime_parser: &MimeMessage,
chat_assignment: &ChatAssignment,
is_partial_download: Option<u32>,
parent_message: &Option<Message>,
incoming_origin: Origin,
) -> Result<(Vec<Option<ContactId>>, Vec<Option<ContactId>>)> {
@@ -254,7 +239,7 @@ async fn get_to_and_past_contact_ids(
ChatAssignment::ExistingChat { chat_id, .. } => Some(*chat_id),
ChatAssignment::MailingListOrBroadcast => None,
ChatAssignment::OneOneChat => {
if is_partial_download.is_none() && !mime_parser.incoming {
if !mime_parser.incoming {
parent_message.as_ref().map(|m| m.chat_id)
} else {
None
@@ -484,15 +469,17 @@ async fn get_to_and_past_contact_ids(
/// downloaded again, sets `chat_id=DC_CHAT_ID_TRASH` and returns `Ok(Some(…))`.
/// If the message is so wrong that we didn't even create a database entry,
/// returns `Ok(None)`.
///
/// If `is_partial_download` is set, it contains the full message size in bytes.
pub(crate) async fn receive_imf_inner(
context: &Context,
rfc724_mid: &str,
imf_raw: &[u8],
seen: bool,
is_partial_download: Option<u32>,
) -> Result<Option<ReceivedMsg>> {
ensure!(
!context
.get_config_bool(Config::SimulateReceiveImfError)
.await?
);
if std::env::var(crate::DCC_MIME_DEBUG).is_ok() {
info!(
context,
@@ -500,16 +487,8 @@ pub(crate) async fn receive_imf_inner(
String::from_utf8_lossy(imf_raw),
);
}
if is_partial_download.is_none() {
ensure!(
!context
.get_config_bool(Config::FailOnReceivingFullMsg)
.await?
);
}
let mut mime_parser = match MimeMessage::from_bytes(context, imf_raw, is_partial_download).await
{
let mut mime_parser = match MimeMessage::from_bytes(context, imf_raw).await {
Err(err) => {
warn!(context, "receive_imf: can't parse MIME: {err:#}.");
if rfc724_mid.starts_with(GENERATED_PREFIX) {
@@ -542,7 +521,15 @@ pub(crate) async fn receive_imf_inner(
// check, if the mail is already in our database.
// make sure, this check is done eg. before securejoin-processing.
let (replace_msg_id, replace_chat_id);
if let Some(old_msg_id) = message::rfc724_mid_exists(context, rfc724_mid).await? {
if mime_parser.pre_message == mimeparser::PreMessageMode::Post {
// Post-Message just replaces the attachment and modifies Params, not the whole message.
// This is done in the `handle_post_message` method.
replace_msg_id = None;
replace_chat_id = None;
} else if let Some(old_msg_id) = message::rfc724_mid_exists(context, rfc724_mid).await? {
// This code handles the download of old partial download stub messages
// It will be removed after a transitioning period,
// after we have released a few versions with pre-messages
replace_msg_id = Some(old_msg_id);
replace_chat_id = if let Some(msg) = Message::load_from_db_optional(context, old_msg_id)
.await?
@@ -615,11 +602,7 @@ pub(crate) async fn receive_imf_inner(
&mime_parser.from,
fingerprint,
prevent_rename,
is_partial_download.is_some()
&& mime_parser
.get_header(HeaderDef::ContentType)
.unwrap_or_default()
.starts_with("multipart/encrypted"),
false,
)
.await?
{
@@ -651,29 +634,21 @@ pub(crate) async fn receive_imf_inner(
.await?
.filter(|p| Some(p.id) != replace_msg_id);
let chat_assignment = decide_chat_assignment(
context,
&mime_parser,
&parent_message,
rfc724_mid,
from_id,
&is_partial_download,
)
.await?;
let chat_assignment =
decide_chat_assignment(context, &mime_parser, &parent_message, rfc724_mid, from_id).await?;
info!(context, "Chat assignment is {chat_assignment:?}.");
let (to_ids, past_ids) = get_to_and_past_contact_ids(
context,
&mime_parser,
&chat_assignment,
is_partial_download,
&parent_message,
incoming_origin,
)
.await?;
let received_msg;
if mime_parser.get_header(HeaderDef::SecureJoin).is_some() {
if let Some(_step) = get_secure_join_step(&mime_parser) {
let res = if mime_parser.incoming {
handle_securejoin_handshake(context, &mut mime_parser, from_id)
.await
@@ -773,7 +748,6 @@ pub(crate) async fn receive_imf_inner(
to_id,
allow_creation,
&mut mime_parser,
is_partial_download,
parent_message,
)
.await?;
@@ -789,7 +763,6 @@ pub(crate) async fn receive_imf_inner(
rfc724_mid_orig,
from_id,
seen,
is_partial_download,
replace_msg_id,
prevent_rename,
chat_id,
@@ -839,6 +812,10 @@ pub(crate) async fn receive_imf_inner(
if let Some(ref sync_items) = mime_parser.sync_items {
if from_id == ContactId::SELF {
if mime_parser.was_encrypted() {
context
.execute_sync_items(sync_items, mime_parser.timestamp_sent)
.await;
// Receiving encrypted message from self updates primary transport.
let from_addr = &mime_parser.from.addr;
@@ -856,7 +833,9 @@ pub(crate) async fn receive_imf_inner(
let transport_changed = if transport_exists {
transaction.execute(
"UPDATE config SET value=? WHERE keyname='configured_addr'",
"
UPDATE config SET value=? WHERE keyname='configured_addr' AND value!=?1
",
(from_addr,),
)? > 0
} else {
@@ -872,11 +851,8 @@ pub(crate) async fn receive_imf_inner(
if transport_changed {
info!(context, "Primary transport changed to {from_addr:?}.");
context.sql.uncache_raw_config("configured_addr").await;
context.emit_event(EventType::TransportsModified);
}
context
.execute_sync_items(sync_items, mime_parser.timestamp_sent)
.await;
} else {
warn!(context, "Sync items are not encrypted.");
}
@@ -957,9 +933,7 @@ pub(crate) async fn receive_imf_inner(
let delete_server_after = context.get_config_delete_server_after().await?;
if !received_msg.msg_ids.is_empty() {
let target = if received_msg.needs_delete_job
|| (delete_server_after == Some(0) && is_partial_download.is_none())
{
let target = if received_msg.needs_delete_job || delete_server_after == Some(0) {
Some(context.get_delete_msgs_target().await?)
} else {
None
@@ -980,15 +954,84 @@ pub(crate) async fn receive_imf_inner(
),
)
.await?;
context.scheduler.interrupt_inbox().await;
}
if target.is_none() && !mime_parser.mdn_reports.is_empty() && mime_parser.has_chat_version()
{
// This is a Delta Chat MDN. Mark as read.
markseen_on_imap_table(context, rfc724_mid_orig).await?;
}
if !mime_parser.incoming && !context.get_config_bool(Config::TeamProfile).await? {
let mut updated_chats = BTreeMap::new();
let mut archived_chats_maybe_noticed = false;
for report in &mime_parser.mdn_reports {
for msg_rfc724_mid in report
.original_message_id
.iter()
.chain(&report.additional_message_ids)
{
let Some(msg_id) = rfc724_mid_exists(context, msg_rfc724_mid).await? else {
continue;
};
let Some(msg) = Message::load_from_db_optional(context, msg_id).await? else {
continue;
};
if msg.state < MessageState::InFresh || msg.state >= MessageState::InSeen {
continue;
}
if !mime_parser.was_encrypted() && msg.get_showpadlock() {
warn!(context, "MDN: Not encrypted. Ignoring.");
continue;
}
message::update_msg_state(context, msg_id, MessageState::InSeen).await?;
if let Err(e) = msg_id.start_ephemeral_timer(context).await {
error!(context, "start_ephemeral_timer for {msg_id}: {e:#}.");
}
if !mime_parser.has_chat_version() {
continue;
}
archived_chats_maybe_noticed |= msg.state < MessageState::InNoticed
&& msg.chat_visibility == ChatVisibility::Archived;
updated_chats
.entry(msg.chat_id)
.and_modify(|ts| *ts = cmp::max(*ts, msg.timestamp_sort))
.or_insert(msg.timestamp_sort);
}
}
for (chat_id, timestamp_sort) in updated_chats {
context
.sql
.execute(
"
UPDATE msgs SET state=? WHERE
state=? AND
hidden=0 AND
chat_id=? AND
timestamp<?",
(
MessageState::InNoticed,
MessageState::InFresh,
chat_id,
timestamp_sort,
),
)
.await
.context("UPDATE msgs.state")?;
// Removes all notifications for the chat in UIs. Ideally should be under
// `if chat_id.get_fresh_msg_cnt(context).await? == 0`, but this causes a not
// updated profile badge counter in the UIs as of v2.35.0. Removed notifications for
// new messages is an already existing and known problem, so let's emit the event
// unconditionally for now.
context.emit_event(EventType::MsgsNoticed(chat_id));
chatlist_events::emit_chatlist_item_changed(context, chat_id);
}
if archived_chats_maybe_noticed {
context.on_archived_chats_maybe_noticed();
}
}
}
if is_partial_download.is_none() && mime_parser.is_call() {
if mime_parser.is_call() {
context
.handle_call_msg(insert_msg_id, &mime_parser, from_id)
.await?;
@@ -1037,7 +1080,7 @@ pub(crate) async fn receive_imf_inner(
/// * `find_key_contact_by_addr`: if true, we only know the e-mail address
/// of the contact, but not the fingerprint,
/// yet want to assign the message to some key-contact.
/// This can happen during prefetch or when the message is partially downloaded.
/// This can happen during prefetch.
/// If we get it wrong, the message will be placed into the correct
/// chat after downloading.
///
@@ -1131,9 +1174,8 @@ async fn decide_chat_assignment(
parent_message: &Option<Message>,
rfc724_mid: &str,
from_id: ContactId,
is_partial_download: &Option<u32>,
) -> Result<ChatAssignment> {
let should_trash = if !mime_parser.mdn_reports.is_empty() {
let mut should_trash = if !mime_parser.mdn_reports.is_empty() {
info!(context, "Message is an MDN (TRASH).");
true
} else if mime_parser.delivery_report.is_some() {
@@ -1147,9 +1189,8 @@ async fn decide_chat_assignment(
{
info!(context, "Chat edit/delete/iroh/sync message (TRASH).");
true
} else if is_partial_download.is_none()
&& (mime_parser.is_system_message == SystemMessage::CallAccepted
|| mime_parser.is_system_message == SystemMessage::CallEnded)
} else if mime_parser.is_system_message == SystemMessage::CallAccepted
|| mime_parser.is_system_message == SystemMessage::CallEnded
{
info!(context, "Call state changed (TRASH).");
true
@@ -1218,6 +1259,44 @@ async fn decide_chat_assignment(
false
};
should_trash |= if mime_parser.pre_message == PreMessageMode::Post {
// if pre message exist, then trash after replacing, otherwise treat as normal message
let pre_message_exists = msg_is_downloaded_for(context, rfc724_mid).await?;
info!(
context,
"Message {rfc724_mid} is a post-message ({}).",
if pre_message_exists {
"pre-message exists already, so trash after replacing attachment"
} else {
"no pre-message -> Keep"
}
);
pre_message_exists
} else if let PreMessageMode::Pre {
post_msg_rfc724_mid,
..
} = &mime_parser.pre_message
{
let msg_id = rfc724_mid_exists(context, post_msg_rfc724_mid).await?;
if let Some(msg_id) = msg_id {
context
.sql
.execute(
"UPDATE msgs SET pre_rfc724_mid=? WHERE id=?",
(rfc724_mid, msg_id),
)
.await?;
}
let post_msg_exists = msg_id.is_some();
info!(
context,
"Message {rfc724_mid} is a pre-message for {post_msg_rfc724_mid} (post_msg_exists:{post_msg_exists})."
);
post_msg_exists
} else {
false
};
// Decide on the type of chat we assign the message to.
//
// The chat may not exist yet, i.e. there may be
@@ -1250,7 +1329,7 @@ async fn decide_chat_assignment(
}
} else if let Some(parent) = &parent_message {
if let Some((chat_id, chat_id_blocked)) =
lookup_chat_by_reply(context, mime_parser, parent, is_partial_download).await?
lookup_chat_by_reply(context, mime_parser, parent).await?
{
// Try to assign to a chat based on In-Reply-To/References.
ChatAssignment::ExistingChat {
@@ -1272,7 +1351,7 @@ async fn decide_chat_assignment(
}
} else if let Some(parent) = &parent_message {
if let Some((chat_id, chat_id_blocked)) =
lookup_chat_by_reply(context, mime_parser, parent, is_partial_download).await?
lookup_chat_by_reply(context, mime_parser, parent).await?
{
// Try to assign to a chat based on In-Reply-To/References.
ChatAssignment::ExistingChat {
@@ -1314,7 +1393,6 @@ async fn do_chat_assignment(
to_id: ContactId,
allow_creation: bool,
mime_parser: &mut MimeMessage,
is_partial_download: Option<u32>,
parent_message: Option<Message>,
) -> Result<(ChatId, Blocked, bool)> {
let is_bot = context.get_config_bool(Config::Bot).await?;
@@ -1365,7 +1443,6 @@ async fn do_chat_assignment(
&& let Some((new_chat_id, new_chat_id_blocked)) = create_group(
context,
mime_parser,
is_partial_download.is_some(),
create_blocked,
from_id,
to_ids,
@@ -1414,7 +1491,6 @@ async fn do_chat_assignment(
to_ids,
allow_creation || test_normal_chat.is_some(),
create_blocked,
is_partial_download.is_some(),
)
.await?
{
@@ -1496,7 +1572,6 @@ async fn do_chat_assignment(
&& let Some((new_chat_id, new_chat_id_blocked)) = create_group(
context,
mime_parser,
is_partial_download.is_some(),
Blocked::Not,
from_id,
to_ids,
@@ -1560,7 +1635,6 @@ async fn do_chat_assignment(
to_ids,
allow_creation,
Blocked::Not,
is_partial_download.is_some(),
)
.await?
{
@@ -1641,7 +1715,6 @@ async fn add_parts(
rfc724_mid: &str,
from_id: ContactId,
seen: bool,
is_partial_download: Option<u32>,
mut replace_msg_id: Option<MsgId>,
prevent_rename: bool,
mut chat_id: ChatId,
@@ -1692,8 +1765,6 @@ async fn add_parts(
}
let is_location_kml = mime_parser.location_kml.is_some();
let is_mdn = !mime_parser.mdn_reports.is_empty();
let mut group_changes = match chat.typ {
_ if chat.id.is_special() => GroupChangesInfo::default(),
Chattype::Single => GroupChangesInfo::default(),
@@ -1713,10 +1784,9 @@ async fn add_parts(
.get_rfc724_mid()
.unwrap_or(rfc724_mid.to_string());
// Extract ephemeral timer from the message or use the existing timer if the message is not fully downloaded.
let mut ephemeral_timer = if is_partial_download.is_some() {
chat_id.get_ephemeral_timer(context).await?
} else if let Some(value) = mime_parser.get_header(HeaderDef::EphemeralTimer) {
// Extract ephemeral timer from the message
let mut ephemeral_timer = if let Some(value) = mime_parser.get_header(HeaderDef::EphemeralTimer)
{
match value.parse::<EphemeralTimer>() {
Ok(timer) => timer,
Err(err) => {
@@ -1730,7 +1800,10 @@ async fn add_parts(
let state = if !mime_parser.incoming {
MessageState::OutDelivered
} else if seen || is_mdn || chat_id_blocked == Blocked::Yes || group_changes.silent
} else if seen
|| !mime_parser.mdn_reports.is_empty()
|| chat_id_blocked == Blocked::Yes
|| group_changes.silent
// No check for `hidden` because only reactions are such and they should be `InFresh`.
{
MessageState::InSeen
@@ -1919,7 +1992,6 @@ async fn add_parts(
let chat_id = if better_msg
.as_ref()
.is_some_and(|better_msg| better_msg.is_empty())
&& is_partial_download.is_none()
{
DC_CHAT_ID_TRASH
} else {
@@ -1968,10 +2040,10 @@ async fn add_parts(
}
handle_edit_delete(context, mime_parser, from_id).await?;
handle_post_message(context, mime_parser, from_id, state).await?;
if is_partial_download.is_none()
&& (mime_parser.is_system_message == SystemMessage::CallAccepted
|| mime_parser.is_system_message == SystemMessage::CallEnded)
if mime_parser.is_system_message == SystemMessage::CallAccepted
|| mime_parser.is_system_message == SystemMessage::CallEnded
{
if let Some(field) = mime_parser.get_header(HeaderDef::InReplyTo) {
if let Some(call) =
@@ -2052,6 +2124,14 @@ async fn add_parts(
}
};
if let PreMessageMode::Pre {
metadata: Some(metadata),
..
} = &mime_parser.pre_message
{
param.apply_post_msg_metadata(metadata);
};
// If you change which information is skipped if the message is trashed,
// also change `MsgId::trash()` and `delete_expired_messages()`
let trash = chat_id.is_trash() || (is_location_kml && part_is_empty && !save_mime_modified);
@@ -2064,7 +2144,7 @@ async fn add_parts(
INSERT INTO msgs
(
id,
rfc724_mid, chat_id,
rfc724_mid, pre_rfc724_mid, chat_id,
from_id, to_id, timestamp, timestamp_sent,
timestamp_rcvd, type, state, msgrmsg,
txt, txt_normalized, subject, param, hidden,
@@ -2074,7 +2154,7 @@ INSERT INTO msgs
)
VALUES (
?,
?, ?, ?, ?,
?, ?, ?, ?, ?,
?, ?, ?, ?,
?, ?, ?, ?,
?, ?, ?, ?, ?, 1,
@@ -2095,14 +2175,23 @@ RETURNING id
"#)?;
let row_id: MsgId = stmt.query_row(params![
replace_msg_id,
rfc724_mid_orig,
if let PreMessageMode::Pre {post_msg_rfc724_mid, ..} = &mime_parser.pre_message {
post_msg_rfc724_mid
} else { rfc724_mid_orig },
if let PreMessageMode::Pre {..} = &mime_parser.pre_message {
rfc724_mid_orig
} else { "" },
if trash { DC_CHAT_ID_TRASH } else { chat_id },
if trash { ContactId::UNDEFINED } else { from_id },
if trash { ContactId::UNDEFINED } else { to_id },
sort_timestamp,
if trash { 0 } else { mime_parser.timestamp_sent },
if trash { 0 } else { mime_parser.timestamp_rcvd },
if trash { Viewtype::Unknown } else { typ },
if trash {
Viewtype::Unknown
} else if let PreMessageMode::Pre {..} = mime_parser.pre_message {
Viewtype::Text
} else { typ },
if trash { MessageState::Undefined } else { state },
if trash { MessengerMessage::No } else { is_dc_message },
if trash || hidden { "" } else { msg },
@@ -2128,10 +2217,10 @@ RETURNING id
if trash { 0 } else { ephemeral_timestamp },
if trash {
DownloadState::Done
} else if is_partial_download.is_some() {
DownloadState::Available
} else if mime_parser.decrypting_failed {
DownloadState::Undecipherable
} else if let PreMessageMode::Pre {..} = mime_parser.pre_message {
DownloadState::Available
} else {
DownloadState::Done
},
@@ -2232,19 +2321,13 @@ RETURNING id
}
}
// Normally outgoing MDNs sent by us never appear in mailboxes, but Gmail saves all
// outgoing messages, including MDNs, to the Sent folder. If we detect such saved MDN,
// delete it.
let needs_delete_job =
!mime_parser.incoming && is_mdn && is_dc_message == MessengerMessage::Yes;
Ok(ReceivedMsg {
chat_id,
state,
hidden,
sort_timestamp,
msg_ids: created_db_entries,
needs_delete_job,
needs_delete_job: false,
})
}
@@ -2324,6 +2407,93 @@ async fn handle_edit_delete(
Ok(())
}
async fn handle_post_message(
context: &Context,
mime_parser: &MimeMessage,
from_id: ContactId,
state: MessageState,
) -> Result<()> {
let PreMessageMode::Post = &mime_parser.pre_message else {
return Ok(());
};
// if Pre-Message exist, replace attachment
// only replacing attachment ensures that doesn't overwrite the text if it was edited before.
let rfc724_mid = mime_parser
.get_rfc724_mid()
.context("expected Post-Message to have a message id")?;
let Some(msg_id) = message::rfc724_mid_exists(context, &rfc724_mid).await? else {
warn!(
context,
"handle_post_message: {rfc724_mid}: Database entry does not exist."
);
return Ok(());
};
let Some(original_msg) = Message::load_from_db_optional(context, msg_id).await? else {
// else: message is processed like a normal message
warn!(
context,
"handle_post_message: {rfc724_mid}: Pre-message was not downloaded yet so treat as normal message."
);
return Ok(());
};
let Some(part) = mime_parser.parts.first() else {
return Ok(());
};
// Do nothing if safety checks fail, the worst case is the message modifies the chat if the
// sender is a member.
if from_id != original_msg.from_id {
warn!(context, "handle_post_message: {rfc724_mid}: Bad sender.");
return Ok(());
}
let post_msg_showpadlock = part
.param
.get_bool(Param::GuaranteeE2ee)
.unwrap_or_default();
if !post_msg_showpadlock && original_msg.get_showpadlock() {
warn!(context, "handle_post_message: {rfc724_mid}: Not encrypted.");
return Ok(());
}
if !part.typ.has_file() {
warn!(
context,
"handle_post_message: {rfc724_mid}: First mime part's message-viewtype has no file."
);
return Ok(());
}
let mut new_params = original_msg.param.clone();
new_params
.merge_in_params(part.param.clone())
.remove(Param::PostMessageFileBytes)
.remove(Param::PostMessageViewtype);
// Don't update `chat_id`: even if it differs from pre-message's one somehow so the result
// depends on message download order, we don't want messages jumping across chats.
context
.sql
.execute(
"
UPDATE msgs SET param=?, type=?, bytes=?, error=?, state=max(state,?), download_state=?
WHERE id=?
",
(
new_params.to_string(),
part.typ,
part.bytes as isize,
part.error.as_deref().unwrap_or_default(),
state,
DownloadState::Done as u32,
original_msg.id,
),
)
.await?;
context.emit_msgs_changed(original_msg.chat_id, original_msg.id);
Ok(())
}
async fn tweak_sort_timestamp(
context: &Context,
mime_parser: &mut MimeMessage,
@@ -2413,7 +2583,6 @@ async fn lookup_chat_by_reply(
context: &Context,
mime_parser: &MimeMessage,
parent: &Message,
is_partial_download: &Option<u32>,
) -> Result<Option<(ChatId, Blocked)>> {
// If the message is encrypted and has group ID,
// lookup by reply should never be needed
@@ -2445,10 +2614,7 @@ async fn lookup_chat_by_reply(
}
// Do not assign unencrypted messages to encrypted chats.
if is_partial_download.is_none()
&& parent_chat.is_encrypted(context).await?
&& !mime_parser.was_encrypted()
{
if parent_chat.is_encrypted(context).await? && !mime_parser.was_encrypted() {
return Ok(None);
}
@@ -2465,18 +2631,7 @@ async fn lookup_or_create_adhoc_group(
to_ids: &[Option<ContactId>],
allow_creation: bool,
create_blocked: Blocked,
is_partial_download: bool,
) -> Result<Option<(ChatId, Blocked, bool)>> {
// Partial download may be an encrypted message with protected Subject header. We do not want to
// create a group with "..." or "Encrypted message" as a subject. The same is for undecipherable
// messages. Instead, assign the message to 1:1 chat with the sender.
if is_partial_download {
info!(
context,
"Ad-hoc group cannot be created from partial download."
);
return Ok(None);
}
if mime_parser.decrypting_failed {
warn!(
context,
@@ -2612,11 +2767,9 @@ async fn is_probably_private_reply(
/// than two members, a new ad hoc group is created.
///
/// On success the function returns the created (chat_id, chat_blocked) tuple.
#[expect(clippy::too_many_arguments)]
async fn create_group(
context: &Context,
mime_parser: &mut MimeMessage,
is_partial_download: bool,
create_blocked: Blocked,
from_id: ContactId,
to_ids: &[Option<ContactId>],
@@ -2698,7 +2851,7 @@ async fn create_group(
if let Some(chat_id) = chat_id {
Ok(Some((chat_id, chat_id_blocked)))
} else if is_partial_download || mime_parser.decrypting_failed {
} else if mime_parser.decrypting_failed {
// It is possible that the message was sent to a valid,
// yet unknown group, which was rejected because
// Chat-Group-Name, which is in the encrypted part, was
+56 -252
View File
@@ -10,7 +10,6 @@ use crate::chat::{
use crate::chatlist::Chatlist;
use crate::constants::DC_GCL_FOR_FORWARDING;
use crate::contact;
use crate::download::MIN_DOWNLOAD_LIMIT;
use crate::imap::prefetch_should_download;
use crate::imex::{ImexMode, imex};
use crate::securejoin::get_securejoin_qr;
@@ -19,8 +18,6 @@ use crate::test_utils::{
};
use crate::tools::{SystemTime, time};
use rand::distr::SampleString;
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_outgoing() -> Result<()> {
let context = TestContext::new_alice().await;
@@ -28,7 +25,7 @@ async fn test_outgoing() -> Result<()> {
From: alice@example.org\n\
\n\
hello";
let mimeparser = MimeMessage::from_bytes(&context.ctx, &raw[..], None).await?;
let mimeparser = MimeMessage::from_bytes(&context.ctx, &raw[..]).await?;
assert_eq!(mimeparser.incoming, false);
Ok(())
}
@@ -43,7 +40,7 @@ async fn test_bad_from() {
References: <Gr.HcxyMARjyJy.9-uvzWPTLtV@nauta.cu>\n\
\n\
hello\x00";
let mimeparser = MimeMessage::from_bytes(&context.ctx, &raw[..], None).await;
let mimeparser = MimeMessage::from_bytes(&context.ctx, &raw[..]).await;
assert!(mimeparser.is_err());
}
@@ -2842,7 +2839,7 @@ References: <second@example.net> <nonexistent@example.net> <first@example.net>
Content-Type: text/plain; charset=utf-8; format=flowed; delsp=no
Message with references."#;
let mime_parser = MimeMessage::from_bytes(&t, &mime[..], None).await?;
let mime_parser = MimeMessage::from_bytes(&t, &mime[..]).await?;
let parent = get_parent_message(&t, &mime_parser).await?.unwrap();
assert_eq!(parent.id, first.id);
@@ -3881,6 +3878,29 @@ async fn test_group_contacts_goto_bottom() -> Result<()> {
let contacts = Contact::get_all(bob, 0, None).await?;
assert_eq!(contacts.len(), 2);
assert_eq!(contacts[0], bob_fiona_id);
send_text_msg(
bob,
bob_chat_id,
"Hi Alice, stay down in my contact list".to_string(),
)
.await?;
bob.pop_sent_msg().await;
let contacts = Contact::get_all(bob, 0, None).await?;
assert_eq!(contacts[0], bob_fiona_id);
remove_contact_from_chat(bob, bob_chat_id, bob_fiona_id).await?;
bob.pop_sent_msg().await;
let contacts = Contact::get_all(bob, 0, None).await?;
// Fiona is still the 0th contact. This makes sense, maybe Bob is going to remove Alice from the
// chat too, so no need to make Alice a more "important" contact yet.
assert_eq!(contacts[0], bob_fiona_id);
send_text_msg(bob, bob_chat_id, "Alice, jump up!".to_string()).await?;
bob.pop_sent_msg().await;
let contacts = Contact::get_all(bob, 0, None).await?;
let bob_alice_id = bob.add_or_lookup_contact_id(alice).await;
assert_eq!(contacts[0], bob_alice_id);
Ok(())
}
@@ -4417,37 +4437,6 @@ async fn test_adhoc_grp_name_no_prefix() -> Result<()> {
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_download_later() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = tcm.alice().await;
alice.set_config(Config::DownloadLimit, Some("1")).await?;
assert_eq!(alice.download_limit().await?, Some(MIN_DOWNLOAD_LIMIT));
let bob = tcm.bob().await;
let bob_chat = bob.create_chat(&alice).await;
// Generate a random string so OpenPGP does not compress it.
let text =
rand::distr::Alphanumeric.sample_string(&mut rand::rng(), MIN_DOWNLOAD_LIMIT as usize);
let sent_msg = bob.send_text(bob_chat.id, &text).await;
let msg = alice.recv_msg(&sent_msg).await;
assert_eq!(msg.download_state, DownloadState::Available);
assert_eq!(msg.state, MessageState::InFresh);
let hi_msg = tcm.send_recv(&bob, &alice, "hi").await;
alice.set_config(Config::DownloadLimit, None).await?;
let msg = alice.recv_msg(&sent_msg).await;
assert_eq!(msg.download_state, DownloadState::Done);
assert_eq!(msg.state, MessageState::InFresh);
assert_eq!(alice.get_last_msg_in(msg.chat_id).await.id, hi_msg.id);
assert!(msg.timestamp_sort <= hi_msg.timestamp_sort);
Ok(())
}
/// Malice can pretend they have the same address as Alice and sends a message encrypted to Alice's
/// key but signed with another one. Alice must detect that this message is wrongly signed and not
/// treat it as Autocrypt-encrypted.
@@ -4483,158 +4472,50 @@ async fn test_outgoing_msg_forgery() -> Result<()> {
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_create_group_with_big_msg() -> Result<()> {
async fn test_pre_msg_group_consistency() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = tcm.alice().await;
let bob = tcm.bob().await;
let ba_contact = bob.add_or_lookup_contact_id(&alice).await;
let ab_chat_id = alice.create_chat(&bob).await.id;
let alice = &tcm.alice().await;
let bob = &tcm.bob().await;
let fiona = &tcm.fiona().await;
let bob_id = alice.add_or_lookup_contact_id(bob).await;
let alice_chat_id = create_group(alice, "foos").await?;
add_contact_to_chat(alice, alice_chat_id, bob_id).await?;
let file_bytes = include_bytes!("../../test-data/image/screenshot.png");
let bob_grp_id = create_group(&bob, "Group").await?;
add_contact_to_chat(&bob, bob_grp_id, ba_contact).await?;
let mut msg = Message::new(Viewtype::Image);
msg.set_file_from_bytes(&bob, "a.jpg", file_bytes, None)?;
let sent_msg = bob.send_msg(bob_grp_id, &mut msg).await;
assert!(msg.get_showpadlock());
alice.set_config(Config::DownloadLimit, Some("1")).await?;
assert_eq!(alice.download_limit().await?, Some(MIN_DOWNLOAD_LIMIT));
let msg = alice.recv_msg(&sent_msg).await;
assert_eq!(msg.download_state, DownloadState::Available);
let alice_chat = Chat::load_from_db(&alice, msg.chat_id).await?;
// Incomplete message is assigned to 1:1 chat.
assert_eq!(alice_chat.typ, Chattype::Single);
alice.set_config(Config::DownloadLimit, None).await?;
let msg = alice.recv_msg(&sent_msg).await;
assert_eq!(msg.download_state, DownloadState::Done);
assert_eq!(msg.state, MessageState::InFresh);
assert_eq!(msg.viewtype, Viewtype::Image);
assert_ne!(msg.chat_id, alice_chat.id);
let alice_grp = Chat::load_from_db(&alice, msg.chat_id).await?;
assert_eq!(alice_grp.typ, Chattype::Group);
assert_eq!(alice_grp.name, "Group");
assert_eq!(
chat::get_chat_contacts(&alice, alice_grp.id).await?.len(),
2
);
// Now Bob can send encrypted messages to Alice.
let bob_grp_id = create_group(&bob, "Group1").await?;
add_contact_to_chat(&bob, bob_grp_id, ba_contact).await?;
let mut msg = Message::new(Viewtype::Image);
msg.set_file_from_bytes(&bob, "a.jpg", file_bytes, None)?;
let sent_msg = bob.send_msg(bob_grp_id, &mut msg).await;
assert!(msg.get_showpadlock());
alice.set_config(Config::DownloadLimit, Some("1")).await?;
let msg = alice.recv_msg(&sent_msg).await;
assert_eq!(msg.download_state, DownloadState::Available);
// Until fully downloaded, an encrypted message must sit in the 1:1 chat.
assert_eq!(msg.chat_id, ab_chat_id);
alice.set_config(Config::DownloadLimit, None).await?;
let msg = alice.recv_msg(&sent_msg).await;
assert_eq!(msg.download_state, DownloadState::Done);
assert_eq!(msg.state, MessageState::InFresh);
assert_eq!(msg.viewtype, Viewtype::Image);
assert_ne!(msg.chat_id, ab_chat_id);
let alice_grp = Chat::load_from_db(&alice, msg.chat_id).await?;
assert_eq!(alice_grp.typ, Chattype::Group);
assert_eq!(alice_grp.name, "Group1");
assert_eq!(
chat::get_chat_contacts(&alice, alice_grp.id).await?.len(),
2
);
// The big message must go away from the 1:1 chat.
let msgs = chat::get_chat_msgs(&alice, ab_chat_id).await?;
assert_eq!(msgs.len(), E2EE_INFO_MSGS);
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_partial_group_consistency() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = tcm.alice().await;
let bob = tcm.bob().await;
let fiona = tcm.fiona().await;
let bob_id = alice.add_or_lookup_contact_id(&bob).await;
let alice_chat_id = create_group(&alice, "foos").await?;
add_contact_to_chat(&alice, alice_chat_id, bob_id).await?;
send_text_msg(&alice, alice_chat_id, "populate".to_string()).await?;
send_text_msg(alice, alice_chat_id, "populate".to_string()).await?;
let add = alice.pop_sent_msg().await;
bob.recv_msg(&add).await;
let bob_chat_id = bob.get_last_msg().await.chat_id;
let contacts = get_chat_contacts(&bob, bob_chat_id).await?;
bob_chat_id.accept(bob).await?;
let contacts = get_chat_contacts(bob, bob_chat_id).await?;
assert_eq!(contacts.len(), 2);
// Bob receives partial message.
let msg_id = receive_imf_from_inbox(
&bob,
"first@example.org",
b"From: Alice <alice@example.org>\n\
To: <bob@example.net>, <charlie@example.com>\n\
Chat-Version: 1.0\n\
Subject: subject\n\
Message-ID: <first@example.org>\n\
Date: Sun, 14 Nov 2021 00:10:00 +0000\
Content-Type: text/plain
Chat-Group-Member-Added: charlie@example.com",
false,
Some(100000),
)
.await?
.context("no received message")?;
let msg = Message::load_from_db(&bob, msg_id.msg_ids[0]).await?;
// Partial download does not change the member list.
assert_eq!(msg.download_state, DownloadState::Available);
assert_eq!(get_chat_contacts(&bob, bob_chat_id).await?, contacts);
// Alice sends normal message to bob, adding fiona.
add_contact_to_chat(
&alice,
alice,
alice_chat_id,
alice.add_or_lookup_contact_id(&fiona).await,
alice.add_or_lookup_contact_id(fiona).await,
)
.await?;
// This message is lost.
alice.pop_sent_msg().await;
bob.recv_msg(&alice.pop_sent_msg().await).await;
let contacts = get_chat_contacts(&bob, bob_chat_id).await?;
// Pre-message adds the new member.
let file_bytes = include_bytes!("../../test-data/image/screenshot.gif");
let mut msg = Message::new(Viewtype::Image);
msg.set_file_from_bytes(alice, "a.jpg", file_bytes, None)?;
let full_msg = alice.send_msg(alice_chat_id, &mut msg).await;
let pre_msg = alice.pop_sent_msg().await;
let msg = bob.recv_msg(&pre_msg).await;
assert_eq!(msg.download_state, DownloadState::Available);
let contacts = get_chat_contacts(bob, bob_chat_id).await?;
assert_eq!(contacts.len(), 3);
// Bob fully receives the partial message.
let msg_id = receive_imf_from_inbox(
&bob,
"first@example.org",
b"From: Alice <alice@example.org>\n\
To: Bob <bob@example.net>\n\
Chat-Version: 1.0\n\
Subject: subject\n\
Message-ID: <first@example.org>\n\
Date: Sun, 14 Nov 2021 00:10:00 +0000\
Content-Type: text/plain
Chat-Group-Member-Added: charlie@example.com",
false,
None,
)
.await?
.context("no received message")?;
let msg = Message::load_from_db(&bob, msg_id.msg_ids[0]).await?;
// After full download, the old message should not change group state.
assert_eq!(msg.download_state, DownloadState::Done);
assert_eq!(get_chat_contacts(&bob, bob_chat_id).await?, contacts);
remove_contact_from_chat(bob, bob_chat_id, bob.add_or_lookup_contact_id(fiona).await).await?;
bob.pop_sent_msg().await;
// Full message doesn't readd the removed member.
bob.recv_msg_trash(&full_msg).await;
let contacts = get_chat_contacts(bob, bob_chat_id).await?;
assert_eq!(contacts.len(), 2);
Ok(())
}
@@ -4876,48 +4757,6 @@ async fn test_references() -> Result<()> {
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_prefer_references_to_downloaded_msgs() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
let bob = &tcm.bob().await;
bob.set_config(Config::DownloadLimit, Some("1")).await?;
let fiona = &tcm.fiona().await;
let alice_bob_id = tcm.send_recv(bob, alice, "hi").await.from_id;
let alice_fiona_id = tcm.send_recv(fiona, alice, "hi").await.from_id;
let alice_chat_id = create_group(alice, "Group").await?;
add_contact_to_chat(alice, alice_chat_id, alice_bob_id).await?;
// W/o fiona the test doesn't work -- the last message is assigned to the 1:1 chat due to
// `is_probably_private_reply()`.
add_contact_to_chat(alice, alice_chat_id, alice_fiona_id).await?;
let sent = alice.send_text(alice_chat_id, "Hi").await;
let received = bob.recv_msg(&sent).await;
assert_eq!(received.download_state, DownloadState::Done);
let bob_chat_id = received.chat_id;
let file_bytes = include_bytes!("../../test-data/image/screenshot.gif");
let mut msg = Message::new(Viewtype::File);
msg.set_file_from_bytes(alice, "file", file_bytes, None)?;
let mut sent = alice.send_msg(alice_chat_id, &mut msg).await;
sent.payload = sent
.payload
.replace("References:", "X-Microsoft-Original-References:")
.replace("In-Reply-To:", "X-Microsoft-Original-In-Reply-To:");
let received = bob.recv_msg(&sent).await;
assert_eq!(received.download_state, DownloadState::Available);
assert_ne!(received.chat_id, bob_chat_id);
assert_eq!(received.chat_id, bob.get_chat(alice).await.id);
let mut msg = Message::new(Viewtype::File);
msg.set_file_from_bytes(alice, "file", file_bytes, None)?;
let sent = alice.send_msg(alice_chat_id, &mut msg).await;
let received = bob.recv_msg(&sent).await;
assert_eq!(received.download_state, DownloadState::Available);
assert_eq!(received.chat_id, bob_chat_id);
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_list_from() -> Result<()> {
let t = &TestContext::new_alice().await;
@@ -4932,7 +4771,7 @@ async fn test_list_from() -> Result<()> {
"clubinfo@donotreply.oeamtc.at"
);
let info = msg.id.get_info(t).await?;
assert!(info.contains(" by ~ÖAMTC (clubinfo@donotreply.oeamtc.at)"));
assert!(info.contains(" by ~ÖAMTC"));
Ok(())
}
@@ -5395,41 +5234,6 @@ async fn test_outgoing_plaintext_two_member_group() -> Result<()> {
Ok(())
}
/// Tests that large messages are assigned
/// to non-key-contacts if the type is not `multipart/encrypted`.
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_partial_download_key_contact_lookup() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
let bob = &tcm.bob().await;
// Create two chats with Alice, both with key-contact and email address contact.
let encrypted_chat = bob.create_chat(alice).await;
let unencrypted_chat = bob.create_email_chat(alice).await;
let seen = false;
let is_partial_download = Some(9999);
let received = receive_imf_from_inbox(
bob,
"3333@example.org",
b"From: alice@example.org\n\
To: bob@example.net\n\
Message-ID: <3333@example.org>\n\
Date: Sun, 22 Mar 2020 22:37:57 +0000\n\
\n\
hello\n",
seen,
is_partial_download,
)
.await?
.unwrap();
assert_ne!(received.chat_id, encrypted_chat.id);
assert_eq!(received.chat_id, unencrypted_chat.id);
Ok(())
}
/// Tests that outgoing unencrypted message
/// is assigned to a chat with email-contact.
///
+13 -70
View File
@@ -14,13 +14,12 @@ pub(crate) use self::connectivity::ConnectivityStore;
use crate::config::{self, Config};
use crate::contact::{ContactId, RecentlySeenLoop};
use crate::context::Context;
use crate::download::{DownloadState, download_msg};
use crate::download::{download_known_post_messages_without_pre_message, download_msgs};
use crate::ephemeral::{self, delete_expired_imap_messages};
use crate::events::EventType;
use crate::imap::{FolderMeaning, Imap, session::Session};
use crate::location;
use crate::log::{LogExt, warn};
use crate::message::MsgId;
use crate::smtp::{Smtp, send_smtp_messages};
use crate::sql;
use crate::stats::maybe_send_stats;
@@ -325,8 +324,8 @@ impl Drop for IoPausedGuard {
#[derive(Debug)]
struct SchedBox {
/// Hostname of used chatmail/email relay
host: String,
/// Address at the used chatmail/email relay
addr: String,
meaning: FolderMeaning,
conn_state: ImapConnectionState,
@@ -351,38 +350,6 @@ pub(crate) struct Scheduler {
recently_seen_loop: RecentlySeenLoop,
}
async fn download_msgs(context: &Context, session: &mut Session) -> Result<()> {
let msg_ids = context
.sql
.query_map_vec("SELECT msg_id FROM download", (), |row| {
let msg_id: MsgId = row.get(0)?;
Ok(msg_id)
})
.await?;
for msg_id in msg_ids {
if let Err(err) = download_msg(context, msg_id, session).await {
warn!(context, "Failed to download message {msg_id}: {:#}.", err);
// Update download state to failure
// so it can be retried.
//
// On success update_download_state() is not needed
// as receive_imf() already
// set the state and emitted the event.
msg_id
.update_download_state(context, DownloadState::Failure)
.await?;
}
context
.sql
.execute("DELETE FROM download WHERE msg_id=?", (msg_id,))
.await?;
}
Ok(())
}
async fn inbox_loop(
ctx: Context,
started: oneshot::Sender<()>,
@@ -481,7 +448,7 @@ async fn inbox_fetch_idle(ctx: &Context, imap: &mut Imap, mut session: Session)
}
// Update quota no more than once a minute.
if ctx.quota_needs_update(60).await
if ctx.quota_needs_update(session.transport_id(), 60).await
&& let Err(err) = ctx.update_recent_quota(&mut session).await
{
warn!(ctx, "Failed to update quota: {:#}.", err);
@@ -534,9 +501,6 @@ async fn inbox_fetch_idle(ctx: &Context, imap: &mut Imap, mut session: Session)
}
}
download_msgs(ctx, &mut session)
.await
.context("Failed to download messages")?;
session
.update_metadata(ctx)
.await
@@ -572,28 +536,6 @@ async fn fetch_idle(
};
if folder_config == Config::ConfiguredInboxFolder {
let mvbox;
let syncbox = match ctx.should_move_sync_msgs().await? {
false => &watch_folder,
true => {
mvbox = ctx.get_config(Config::ConfiguredMvboxFolder).await?;
mvbox.as_deref().unwrap_or(&watch_folder)
}
};
if ctx
.get_config(Config::ConfiguredAddr)
.await?
.unwrap_or_default()
== connection.addr
{
session
.send_sync_msgs(ctx, syncbox)
.await
.context("fetch_idle: send_sync_msgs")
.log_err(ctx)
.ok();
}
session
.store_seen_flags_on_imap(ctx)
.await
@@ -619,6 +561,11 @@ async fn fetch_idle(
delete_expired_imap_messages(ctx)
.await
.context("delete_expired_imap_messages")?;
download_known_post_messages_without_pre_message(ctx, &mut session).await?;
download_msgs(ctx, &mut session)
.await
.context("download_msgs")?;
} else if folder_config == Config::ConfiguredInboxFolder {
session.last_full_folder_scan.lock().await.take();
}
@@ -704,6 +651,7 @@ async fn fetch_idle(
Ok(session)
}
/// Simplified IMAP loop to watch non-inbox folders.
async fn simple_imap_loop(
ctx: Context,
started: oneshot::Sender<()>,
@@ -883,14 +831,9 @@ impl Scheduler {
let ctx = ctx.clone();
task::spawn(inbox_loop(ctx, inbox_start_send, inbox_handlers))
};
let host = configured_login_param
.addr
.split("@")
.last()
.context("address has no host")?
.to_owned();
let addr = configured_login_param.addr.clone();
let inbox = SchedBox {
host: host.clone(),
addr: addr.clone(),
meaning: FolderMeaning::Inbox,
conn_state,
handle,
@@ -906,7 +849,7 @@ impl Scheduler {
let meaning = FolderMeaning::Mvbox;
let handle = task::spawn(simple_imap_loop(ctx, start_send, handlers, meaning));
oboxes.push(SchedBox {
host,
addr,
meaning,
conn_state,
handle,
+97 -82
View File
@@ -343,9 +343,18 @@ impl Context {
.green {
background-color: #34c759;
}
.grey {
background-color: #808080;
}
.yellow {
background-color: #fdc625;
}
.transport {
margin-bottom: 1em;
}
.quota-list {
padding-left: 0;
}
</style>
</head>
<body>"#
@@ -375,7 +384,7 @@ impl Context {
.boxes()
.map(|b| {
(
b.host.clone(),
b.addr.clone(),
b.meaning,
b.conn_state.state.connectivity.clone(),
)
@@ -396,83 +405,85 @@ impl Context {
// =============================================================================================
// Add e.g.
// Incoming messages
// - "Inbox": Connected
// - [X] nine.testrun.org: Connected
// 1.34 GiB of 2 GiB used
// [======67%===== ]
// =============================================================================================
let watched_folders = get_watched_folder_configs(self).await?;
let incoming_messages = stock_str::incoming_messages(self).await;
ret += &format!("<h3>{incoming_messages}</h3><ul>");
for (host, folder, state) in &folders_states {
let mut folder_added = false;
if let Some(config) = folder.to_config().filter(|c| watched_folders.contains(c)) {
let f = self.get_config(config).await.log_err(self).ok().flatten();
if let Some(foldername) = f {
let detailed = &state.get_detailed();
ret += "<li>";
ret += &*detailed.to_icon();
ret += " <b>";
if folder == &FolderMeaning::Inbox {
ret += &*escaper::encode_minimal(host);
} else {
ret += &*escaper::encode_minimal(&foldername);
}
ret += ":</b> ";
ret += &*escaper::encode_minimal(&detailed.to_string_imap(self).await);
ret += "</li>";
folder_added = true;
}
}
if !folder_added && folder == &FolderMeaning::Inbox {
let detailed = &state.get_detailed();
if let DetailedConnectivity::Error(_) = detailed {
// On the inbox thread, we also do some other things like scan_folders and run jobs
// so, maybe, the inbox is not watched, but something else went wrong
ret += "<li>";
ret += &*detailed.to_icon();
ret += " ";
ret += &*escaper::encode_minimal(&detailed.to_string_imap(self).await);
ret += "</li>";
}
}
}
ret += "</ul>";
// =============================================================================================
// Add e.g.
// Outgoing messages
// Your last message was sent successfully
// =============================================================================================
let outgoing_messages = stock_str::outgoing_messages(self).await;
ret += &format!("<h3>{outgoing_messages}</h3><ul><li>");
let detailed = smtp.get_detailed();
ret += &*detailed.to_icon();
ret += " ";
ret += &*escaper::encode_minimal(&detailed.to_string_smtp(self).await);
ret += "</li></ul>";
// =============================================================================================
// Add e.g.
// Storage on testrun.org
// 1.34 GiB of 2 GiB used
// [======67%===== ]
// =============================================================================================
let domain =
&deltachat_contact_tools::EmailAddress::new(&self.get_primary_self_addr().await?)?
.domain;
let storage_on_domain =
escaper::encode_minimal(&stock_str::storage_on_domain(self, domain).await);
ret += &format!("<h3>{storage_on_domain}</h3><ul>");
let transports = self
.sql
.query_map_vec("SELECT id, addr FROM transports", (), |row| {
let transport_id: u32 = row.get(0)?;
let addr: String = row.get(1)?;
Ok((transport_id, addr))
})
.await?;
let quota = self.quota.read().await;
if let Some(quota) = &*quota {
for (transport_id, transport_addr) in transports {
let domain = &deltachat_contact_tools::EmailAddress::new(&transport_addr)
.map_or(transport_addr.clone(), |email| email.domain);
let domain_escaped = escaper::encode_minimal(domain);
ret += "<li class=\"transport\">";
let folders = folders_states
.iter()
.filter(|(folder_addr, ..)| *folder_addr == transport_addr);
for (_addr, folder, state) in folders {
let mut folder_added = false;
if let Some(config) = folder.to_config().filter(|c| watched_folders.contains(c)) {
let f = self.get_config(config).await.log_err(self).ok().flatten();
if let Some(foldername) = f {
let detailed = &state.get_detailed();
ret += &*detailed.to_icon();
ret += " <b>";
if folder == &FolderMeaning::Inbox {
ret += &*domain_escaped;
} else {
ret += &*escaper::encode_minimal(&foldername);
}
ret += ":</b> ";
ret += &*escaper::encode_minimal(&detailed.to_string_imap(self).await);
ret += "<br />";
folder_added = true;
}
}
if !folder_added && folder == &FolderMeaning::Inbox {
let detailed = &state.get_detailed();
if let DetailedConnectivity::Error(_) = detailed {
// On the inbox thread, we also do some other things like scan_folders and run jobs
// so, maybe, the inbox is not watched, but something else went wrong
ret += &*detailed.to_icon();
ret += " ";
ret += &*escaper::encode_minimal(&detailed.to_string_imap(self).await);
ret += "<br />";
}
}
}
let Some(quota) = quota.get(&transport_id) else {
ret += "</li>";
continue;
};
match &quota.recent {
Err(e) => {
ret += &escaper::encode_minimal(&e.to_string());
}
Ok(quota) => {
if !quota.is_empty() {
if quota.is_empty() {
ret += &format!(
"Warning: {domain_escaped} claims to support quota but gives no information"
);
} else {
ret += "<ul class=\"quota-list\">";
for (root_name, resources) in quota {
use async_imap::types::QuotaResourceName::*;
for resource in resources {
@@ -529,7 +540,7 @@ impl Context {
} else if percent >= QUOTA_WARN_THRESHOLD_PERCENTAGE {
"yellow"
} else {
"green"
"grey"
};
let div_width_percent = min(100, percent);
ret += &format!(
@@ -539,24 +550,28 @@ impl Context {
ret += "</li>";
}
}
} else {
let domain_escaped = escaper::encode_minimal(domain);
ret += &format!(
"<li>Warning: {domain_escaped} claims to support quota but gives no information</li>"
);
ret += "</ul>";
}
}
Err(e) => {
let error_escaped = escaper::encode_minimal(&e.to_string());
ret += &format!("<li>{error_escaped}</li>");
}
}
} else {
let not_connected = stock_str::not_connected(self).await;
ret += &format!("<li>{not_connected}</li>");
ret += "</li>";
}
ret += "</ul>";
// =============================================================================================
// Add e.g.
// Outgoing messages
// Your last message was sent successfully
// =============================================================================================
let outgoing_messages = stock_str::outgoing_messages(self).await;
ret += &format!("<h3>{outgoing_messages}</h3><ul><li>");
let detailed = smtp.get_detailed();
ret += &*detailed.to_icon();
ret += " ";
ret += &*escaper::encode_minimal(&detailed.to_string_smtp(self).await);
ret += "</li></ul>";
// =============================================================================================
ret += "</body></html>\n";
+100 -35
View File
@@ -160,7 +160,7 @@ pub async fn get_securejoin_qr(context: &Context, chat: Option<ChatId>) -> Resul
context
.sync_qr_code_tokens(Some(chat.grpid.as_str()))
.await?;
context.scheduler.interrupt_inbox().await;
context.scheduler.interrupt_smtp().await;
}
let chat_name = chat.get_name();
@@ -192,7 +192,7 @@ pub async fn get_securejoin_qr(context: &Context, chat: Option<ChatId>) -> Resul
.replace("%20", "+");
if sync_token {
context.sync_qr_code_tokens(None).await?;
context.scheduler.interrupt_inbox().await;
context.scheduler.interrupt_smtp().await;
}
format!(
"https://i.delta.chat/#{fingerprint}&i={invitenumber}&s={auth}&a={self_addr_urlencoded}&n={self_name_urlencoded}",
@@ -343,6 +343,62 @@ pub(crate) enum HandshakeMessage {
Propagate,
}
/// Step of Secure-Join protocol.
#[derive(Debug, Display, PartialEq, Eq)]
pub(crate) enum SecureJoinStep {
/// vc-request or vg-request
Request { invitenumber: String },
/// vc-auth-required or vg-auth-required
AuthRequired,
/// vc-request-with-auth or vg-request-with-auth
RequestWithAuth,
/// vc-contact-confirm
ContactConfirm,
/// vg-member-added
MemberAdded,
/// Deprecated step such as `vg-member-added-received` or `vc-contact-confirm-received`.
Deprecated,
/// Unknown step.
Unknown { step: String },
}
/// Parses message headers to find out which Secure-Join step the message represents.
///
/// Returns `None` if the message is not a Secure-Join message.
pub(crate) fn get_secure_join_step(mime_message: &MimeMessage) -> Option<SecureJoinStep> {
if let Some(invitenumber) = mime_message.get_header(HeaderDef::SecureJoinInvitenumber) {
// We do not care about presence of `Secure-Join: vc-request` or `Secure-Join: vg-request` header.
// This allows us to always treat `Secure-Join` header as protected and ignore it
// in the unencrypted part even though it is sent there for backwards compatibility.
Some(SecureJoinStep::Request {
invitenumber: invitenumber.to_string(),
})
} else if let Some(step) = mime_message.get_header(HeaderDef::SecureJoin) {
match step {
"vg-auth-required" | "vc-auth-required" => Some(SecureJoinStep::AuthRequired),
"vg-request-with-auth" | "vc-request-with-auth" => {
Some(SecureJoinStep::RequestWithAuth)
}
"vc-contact-confirm" => Some(SecureJoinStep::ContactConfirm),
"vg-member-added" => Some(SecureJoinStep::MemberAdded),
"vg-member-added-received" | "vc-contact-confirm-received" => {
Some(SecureJoinStep::Deprecated)
}
step => Some(SecureJoinStep::Unknown {
step: step.to_string(),
}),
}
} else {
None
}
}
/// Handle incoming secure-join handshake.
///
/// This function will update the securejoin state in the database as the protocol
@@ -362,9 +418,8 @@ pub(crate) async fn handle_securejoin_handshake(
if contact_id.is_special() {
return Err(Error::msg("Can not be called with special contact ID"));
}
let step = mime_message
.get_header(HeaderDef::SecureJoin)
.context("Not a Secure-Join message")?;
let step = get_secure_join_step(mime_message).context("Not a Secure-Join message")?;
info!(context, "Received secure-join message {step:?}.");
@@ -383,7 +438,7 @@ pub(crate) async fn handle_securejoin_handshake(
// will improve security (completely unrelated to the securejoin protocol)
// and is something we want to do in the future:
// https://www.rfc-editor.org/rfc/rfc9580.html#name-surreptitious-forwarding
if !matches!(step, "vg-request" | "vc-request") {
if !matches!(step, SecureJoinStep::Request { .. }) {
let mut self_found = false;
let self_fingerprint = load_self_public_key(context).await?.dc_fingerprint();
for (addr, key) in &mime_message.gossiped_keys {
@@ -403,7 +458,7 @@ pub(crate) async fn handle_securejoin_handshake(
}
match step {
"vg-request" | "vc-request" => {
SecureJoinStep::Request { ref invitenumber } => {
/*=======================================================
==== Alice - the inviter side ====
==== Step 3 in "Setup verified contact" protocol ====
@@ -413,13 +468,6 @@ pub(crate) async fn handle_securejoin_handshake(
// it just ensures, we have Bobs key now. If we do _not_ have the key because eg. MitM has removed it,
// send_message() will fail with the error "End-to-end-encryption unavailable unexpectedly.", so, there is no additional check needed here.
// verify that the `Secure-Join-Invitenumber:`-header matches invitenumber written to the QR code
let invitenumber = match mime_message.get_header(HeaderDef::SecureJoinInvitenumber) {
Some(n) => n,
None => {
warn!(context, "Secure-join denied (invitenumber missing)");
return Ok(HandshakeMessage::Ignore);
}
};
if !token::exists(context, token::Namespace::InviteNumber, invitenumber).await? {
warn!(context, "Secure-join denied (bad invitenumber).");
return Ok(HandshakeMessage::Ignore);
@@ -436,24 +484,29 @@ pub(crate) async fn handle_securejoin_handshake(
)
.await?;
let prefix = mime_message
.get_header(HeaderDef::SecureJoin)
.and_then(|step| step.get(..2))
.unwrap_or("vc");
// Alice -> Bob
send_alice_handshake_msg(
context,
autocrypt_contact_id,
&format!("{}-auth-required", &step.get(..2).unwrap_or_default()),
&format!("{prefix}-auth-required"),
)
.await
.context("failed sending auth-required handshake message")?;
Ok(HandshakeMessage::Done)
}
"vg-auth-required" | "vc-auth-required" => {
SecureJoinStep::AuthRequired => {
/*========================================================
==== Bob - the joiner's side =====
==== Step 4 in "Setup verified contact" protocol =====
========================================================*/
bob::handle_auth_required(context, mime_message).await
}
"vg-request-with-auth" | "vc-request-with-auth" => {
SecureJoinStep::RequestWithAuth => {
/*==========================================================
==== Alice - the inviter side ====
==== Steps 5+6 in "Setup verified contact" protocol ====
@@ -543,8 +596,15 @@ pub(crate) async fn handle_securejoin_handshake(
// Join group.
chat::add_contact_to_chat_ex(context, Nosync, joining_chat_id, contact_id, true)
.await?;
let chat = Chat::load_from_db(context, joining_chat_id).await?;
if chat.typ == Chattype::OutBroadcast {
// We don't use the membership consistency algorithm for broadcast channels,
// so, sync the memberlist when adding a contact
chat.sync_contacts(context).await.log_err(context).ok();
}
inviter_progress(context, contact_id, joining_chat_id, chat.typ)?;
// IMAP-delete the message to avoid handling it by another device and adding the
// member twice. Another device will know the member's key from Autocrypt-Gossip.
@@ -564,14 +624,14 @@ pub(crate) async fn handle_securejoin_handshake(
==== Bob - the joiner's side ====
==== Step 7 in "Setup verified contact" protocol ====
=======================================================*/
"vc-contact-confirm" => {
SecureJoinStep::ContactConfirm => {
context.emit_event(EventType::SecurejoinJoinerProgress {
contact_id,
progress: JoinerProgress::Succeeded.into_u16(),
});
Ok(HandshakeMessage::Ignore)
}
"vg-member-added" => {
SecureJoinStep::MemberAdded => {
let Some(member_added) = mime_message.get_header(HeaderDef::ChatGroupMemberAdded)
else {
warn!(
@@ -594,13 +654,12 @@ pub(crate) async fn handle_securejoin_handshake(
});
Ok(HandshakeMessage::Propagate)
}
"vg-member-added-received" | "vc-contact-confirm-received" => {
SecureJoinStep::Deprecated => {
// Deprecated steps, delete them immediately.
Ok(HandshakeMessage::Done)
}
_ => {
warn!(context, "invalid step: {}", step);
SecureJoinStep::Unknown { ref step } => {
warn!(context, "Invalid SecureJoin step: {step:?}.");
Ok(HandshakeMessage::Ignore)
}
}
@@ -631,17 +690,20 @@ pub(crate) async fn observe_securejoin_on_other_device(
if contact_id.is_special() {
return Err(Error::msg("Can not be called with special contact ID"));
}
let step = mime_message
.get_header(HeaderDef::SecureJoin)
.context("Not a Secure-Join message")?;
let step = get_secure_join_step(mime_message).context("Not a Secure-Join message")?;
info!(context, "Observing secure-join message {step:?}.");
if !matches!(
step,
"vg-request-with-auth" | "vc-request-with-auth" | "vg-member-added" | "vc-contact-confirm"
) {
return Ok(HandshakeMessage::Ignore);
};
match step {
SecureJoinStep::Request { .. }
| SecureJoinStep::AuthRequired
| SecureJoinStep::Deprecated
| SecureJoinStep::Unknown { .. } => {
return Ok(HandshakeMessage::Ignore);
}
SecureJoinStep::RequestWithAuth
| SecureJoinStep::MemberAdded
| SecureJoinStep::ContactConfirm => {}
}
if !encrypted_and_signed(context, mime_message, &get_self_fingerprint(context).await?) {
warn!(
@@ -673,7 +735,10 @@ pub(crate) async fn observe_securejoin_on_other_device(
mark_contact_id_as_verified(context, contact_id, Some(ContactId::SELF)).await?;
if step == "vg-member-added" || step == "vc-contact-confirm" {
if matches!(
step,
SecureJoinStep::MemberAdded | SecureJoinStep::ContactConfirm
) {
let chat_type = if mime_message
.get_header(HeaderDef::ChatGroupMemberAdded)
.is_none()
@@ -696,14 +761,14 @@ pub(crate) async fn observe_securejoin_on_other_device(
inviter_progress(context, contact_id, chat_id, chat_type)?;
}
if step == "vg-request-with-auth" || step == "vc-request-with-auth" {
if matches!(step, SecureJoinStep::RequestWithAuth) {
// This actually reflects what happens on the first device (which does the secure
// join) and causes a subsequent "vg-member-added" message to create an unblocked
// verified group.
ChatId::create_for_contact_with_blocked(context, contact_id, Blocked::Not).await?;
}
if step == "vg-member-added" {
if matches!(step, SecureJoinStep::MemberAdded) {
Ok(HandshakeMessage::Propagate)
} else {
Ok(HandshakeMessage::Ignore)
+118 -1
View File
@@ -6,8 +6,10 @@ use super::*;
use crate::chat::{CantSendReason, add_contact_to_chat, remove_contact_from_chat};
use crate::chatlist::Chatlist;
use crate::constants::Chattype;
use crate::constants::DC_CHAT_ID_TRASH;
use crate::key::self_fingerprint;
use crate::mimeparser::{GossipedKey, SystemMessage};
use crate::qr::Qr;
use crate::receive_imf::receive_imf;
use crate::stock_str::{self, messages_e2e_encrypted};
use crate::test_utils::{
@@ -1019,7 +1021,7 @@ async fn test_expired_synced_auth_token() -> Result<()> {
let qr = get_securejoin_qr(alice2, None).await?;
alice2.send_sync_msg().await.unwrap();
let sync_msg = alice2.pop_sent_sync_msg().await;
let sync_msg = alice2.pop_sent_msg().await;
// One week passes, QR code expires.
SystemTime::shift(Duration::from_secs(7 * 24 * 3600));
@@ -1253,3 +1255,118 @@ async fn test_user_deletes_chat_before_securejoin_completes() -> Result<()> {
Ok(())
}
/// Tests that vc-request is processed even if the server
/// encrypts incoming unencrypted messages with OpenPGP.
///
/// For an example of software that encrypts incoming messages
/// with OpenPGP, see <https://lacre.io/>.
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_vc_request_encrypted_at_rest() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
let bob = &tcm.bob().await;
let qr = get_securejoin_qr(alice, None).await?;
let Qr::AskVerifyContact { invitenumber, .. } = check_qr(bob, &qr).await? else {
panic!("Failed to parse QR code");
};
// Encrypted payload is this message encrypted to Alice's key:
/*
Content-Type: multipart/mixed;
boundary="1888168c99b6020b_ed9b3571859a92d7_eeabe50050fc923b"
--1888168c99b6020b_ed9b3571859a92d7_eeabe50050fc923b
Content-Type: text/plain; charset="utf-8"
Message-ID: <4f108a03-6c03-4f41-a6fb-fcd05b94c21f@localhost>
Content-Transfer-Encoding: 7bit
Secure-Join: vc-request
--1888168c99b6020b_ed9b3571859a92d7_eeabe50050fc923b--
*/
let payload = format!("To: <alice@example.org>
Subject: [...]
Date: Tue, 6 Jan 2026 08:20:47 +0000
References: <4f108a03-6c03-4f41-a6fb-fcd05b94c21f@localhost>
Chat-Version: 1.0
Autocrypt: addr=bob@example.net; prefer-encrypt=mutual; keydata=xsBNBF4wx1cBCADOwLS/xCd8iKDWUsyTfVzWby+ZGKPpamPTvdj0GFgnf0B1EBaA5//PjA
zbK5iKio6QNEmZagzJPkXPByJcAIRUm0T16tqDtCvxm+H93YEXpHi/XWOeJw9kohATSqUtsRO0pFJe
DvPiMTmQrEmHYoWDSQBfCrowZdvnMAlbJ9JjYOngcMeTxc0jxmPs5s17yFC+1OWu4fwWCyUM3wy1Jz
dKTcDWryrSkvmgFdUqJ7pJDk1HFTt+x9tvQlK3un9BXiRwv0u0zDSuI8eDH/dRLA4UL9Pq6vmJmBam
e1BPsE1PA7VzeTSJR2ooJXMT6o2AmH8PPUfRkv3OiWuh7LM5FSpHABEBAAHNETxib2JAZXhhbXBsZS
5uZXQ+wsCOBBMBCAA4BQJpXW0wFiEEzMtaqfbhFByUMWXx2xixjLz3BIcCGwMCHgAECwkIBwYVCAkK
CwIDFgIBAScCGQEACgkQ2xixjLz3BIcexwgAsOauz2xZ6QhVBQNX3Hg8OoYTn60w6b4XUOpI4UtqAT
bAmPr5VPhCUa5vzI19QCSekLZIVxUSl6C+7YF/mU8yJldKPgq1Kpr8tbnISNbUDpvuGz7NQdK0TEwx
RkPDw+ilqY7v2Rhhg0Sogi23Yrbyqvs72oorDjia6dBA7VLixpW1O9h/MfQ6WiXJcHSavoLqk2hFW1
y/AsVmArGiuT+kzRJWhTzojZLjQLHTKQii/AOQ3MhkJycyayI/igSEDWWwCcuqL8SF+PUJWGyU5PxU
2hTHDBsVItLLsqseM3WrwJRkBBdsdsR8+moHVyXJV9Gfl/4UFQCgltdEwWISgNTbAc7ATQReMMdXAQ
gAogeBLbIjaeJII3W2pxsu+SEusQkJVykbGYDtqyXV+XBZisY4GE0kTawK5mqSh+rDqquCxDgYWBRT
nGZwEKohnj2NG75pjfyVPhYMUdJt7+Ya1oiFvZlgrrOj1btjevq53yFtQolMN+X2oS8mlf9jSzIyPC
eDxJk1N1gxaAATg3ByAyB1Td0wDdFPp48ni8qzfyGZeYicvJlx74YnOaja2lnI/y+I9LsmmqiOgI8H
cbmH1od5qSnVjhcpBoTEA15YLIEkSE3C00Q5USlDS3EVg/IOu3FXnLl7v0hQ/jXyv88eycfpSfFcbM
Hot9VtJ4TIPIoSX7DQ+uU2SXJKiZNkVQARAQABwsB2BBgBCAAgBQJpXW0wAhsMFiEEzMtaqfbhFByU
MWXx2xixjLz3BIcACgkQ2xixjLz3BIcxvwf9G33yLtvekpNGeRbWkjRXSa083k/4CZpkF0Fb5led1O
hjRjw9KQdZj68qn1vVrDxygnYdHwZWy8WbwEkK+l3gyJ2v2D/Wsj+sHjQ1YMtPnx+EsRFWjKsolYfN
gJZ+IfMTasip8uJNxN5LIvI/svn43mzgeLUudddekmr/2eekhXfYz5covje3yf+u4rYDWQ78TLpJL5
bxTwiiUyV1ZIO1G2F4RNjv8wZHsGD3DasQck+OCV7jQNuyojcmq6J22lXN0clisI51mHzwkLAnI+Yh
jRj3nqwVSCxsmgfejap53JGHEL0CEy6KvepLqjFf8BiK4kwRBmI7/YWOyY2xBWdTew==
Secure-Join: vc-request
Secure-Join-Invitenumber: {invitenumber}
MIME-Version: 1.0
From: <bob@example.net>
Message-ID: <4f108a03-6c03-4f41-a6fb-fcd05b94c21f@localhost>
Content-Type: multipart/encrypted;
protocol=\"application/pgp-encrypted\";
boundary=\"1767687657/562150/131174/example.org\"
This is a MIME-encapsulated message.
--1767687657/562150/131174/example.org
Content-Type: application/pgp-encrypted
Content-Disposition: attachment
Version: 1
--1767687657/562150/131174/example.org
Content-Type: application/octet-stream
Content-Disposition: inline; filename=\"msg.asc\"
-----BEGIN PGP MESSAGE-----
wV4D5tq63hTeebASAQdAGKP3GqsznPOYBXOo8FYf5x8r0ZaJZUGkXIR+Oi32H1Mw
LdVEu0jcXmZ1fGZ7LLPtvfwllB2W9Bhiy/lCZZs+c98VYzSWaT0DHmp23WvTsBZn
0sDeAS0n2I6xPwYhUjqL8G1a9xdyvZRgHYUXhiARrXzx7G+aX8/KFjk6e2OZxiVy
PSHKh61T9wPYzPMlToPsI0GjXAzOgI9I3XQ8xPqihiUpnU6BSg+Tj26m9ZqdJe8v
aVAHxAu7aqz7eljp2H336Idgfvf1lXdA6gMA8HhHI729Ws/Mc8lcgefM9kzZWEIN
FOBYmf4JCdTghESpEJ5i+pF28ChqZ3+8lGzNmT66c2SQRTclwxFa4SS19D4mxkHq
iOMqj7LZ8R6IhnNVRD1V4mVapc7vFVZy8ViyZlmhJF0TBR/TEXlit6vtSkDKFiR9
Tb5zuFQVAwhun3i1tG3/Ii1ReflZNnteVlL1J1XnptOJ70vVUeO3qRqkbkiH8Mdm
Asvy24WKkz9jmEgdt0Hj8VUHtcOL5xmtsr0ZUCLUriylpzDh8E+0LTl2/DOgH+mI
D/gFLCzFHjI0jfXyLTGCYOPYAnvqJukLBT/X7JbCAI/62aq0K28Lp6beJhFs+bIz
gU6dGXsFMe/RpRHrIAkMAaM5xkxMDRuRJDxiUdS/X+Y8
=QrdS
-----END PGP MESSAGE-----
--1767687657/562150/131174/example.org--
");
// Alice receives vc-request, but it is encrypted
// by the server.
let received = receive_imf(alice, payload.as_bytes(), false)
.await?
.unwrap();
assert_eq!(received.chat_id, DC_CHAT_ID_TRASH);
// Test that Alice sends vc-auth-required after processing vc-request.
let sent = alice.pop_sent_msg().await;
let msg = bob.parse_msg(&sent).await;
assert_eq!(
msg.get_header(HeaderDef::SecureJoin).unwrap(),
"vc-auth-required"
);
Ok(())
}
+56 -6
View File
@@ -13,7 +13,7 @@ use crate::config::Config;
use crate::contact::{Contact, ContactId};
use crate::context::Context;
use crate::events::EventType;
use crate::log::warn;
use crate::log::{LogExt, warn};
use crate::message::Message;
use crate::message::{self, MsgId};
use crate::mimefactory::MimeFactory;
@@ -184,6 +184,9 @@ pub(crate) async fn smtp_send(
smtp: &mut Smtp,
msg_id: Option<MsgId>,
) -> SendResult {
if recipients.is_empty() {
return SendResult::Success;
}
if std::env::var(crate::DCC_MIME_DEBUG).is_ok() {
info!(context, "SMTP-sending out mime message:\n{message}");
}
@@ -495,6 +498,7 @@ async fn send_mdns(context: &Context, connection: &mut Smtp) -> Result<()> {
pub(crate) async fn send_smtp_messages(context: &Context, connection: &mut Smtp) -> Result<()> {
let ratelimited = if context.ratelimit.read().await.can_send() {
// add status updates and sync messages to end of sending queue
context.send_sync_msg().await?;
context.flush_status_updates().await?;
false
} else {
@@ -569,17 +573,32 @@ async fn send_mdn_rfc724_mid(
additional_rfc724_mids.clone(),
)
.await?;
let encrypted = mimefactory.will_be_encrypted();
let rendered_msg = mimefactory.render(context).await?;
let body = rendered_msg.message;
let addr = contact.get_addr();
let recipient = async_smtp::EmailAddress::new(addr.to_string())
.map_err(|err| format_err!("invalid recipient: {addr} {err:?}"))?;
let recipients = vec![recipient];
let mut recipients = Vec::new();
if contact_id != ContactId::SELF {
recipients.push(contact.get_addr().to_string());
}
if context.get_config_bool(Config::BccSelf).await? {
add_self_recipients(context, &mut recipients, encrypted).await?;
}
let recipients: Vec<_> = recipients
.into_iter()
.filter_map(|addr| {
async_smtp::EmailAddress::new(addr.clone())
.with_context(|| format!("Invalid recipient: {addr}"))
.log_err(context)
.ok()
})
.collect();
match smtp_send(context, &recipients, &body, smtp, None).await {
SendResult::Success => {
info!(context, "Successfully sent MDN for {rfc724_mid}.");
if !recipients.is_empty() {
info!(context, "Successfully sent MDN for {rfc724_mid}.");
}
context
.sql
.transaction(|transaction| {
@@ -666,3 +685,34 @@ async fn send_mdn(context: &Context, smtp: &mut Smtp) -> Result<bool> {
}
}
}
/// Adds self-addresses to `recipients` as necessary.
/// This doesn't check `Config::BccSelf`, it should be checked by the caller if needed.
pub(crate) async fn add_self_recipients(
context: &Context,
recipients: &mut Vec<String>,
encrypted: bool,
) -> Result<()> {
// Previous versions of Delta Chat did not send BCC self
// if DeleteServerAfter was set to immediately delete messages
// from the server. This is not the case anymore
// because BCC-self messages are also used to detect
// that message was sent if SMTP server is slow to respond
// and connection is frequently lost
// before receiving status line. NB: This is not a problem for chatmail servers, so `BccSelf`
// disabled by default is fine.
if context.get_config_delete_server_after().await? != Some(0) || !recipients.is_empty() {
// Avoid sending unencrypted messages to all transports, chatmail relays won't accept
// them. Normally the user should have a non-chatmail primary transport to send unencrypted
// messages.
if encrypted {
for addr in context.get_secondary_self_addrs().await? {
recipients.push(addr);
}
}
// `from` must be the last addr, see `receive_imf_inner()` why.
let from = context.get_primary_self_addr().await?;
recipients.push(from);
}
Ok(())
}
+59
View File
@@ -1011,6 +1011,8 @@ CREATE INDEX msgs_status_updates_index2 ON msgs_status_updates (uid);
inc_and_check(&mut migration_version, 119)?;
if dbversion < migration_version {
// This table is deprecated sinc 2025-12-25.
// Sync messages are again sent over SMTP.
sql.execute_migration(
"CREATE TABLE imap_send (
id INTEGER PRIMARY KEY AUTOINCREMENT,
@@ -1466,6 +1468,63 @@ ALTER TABLE contacts ADD COLUMN name_normalized TEXT;
.await?;
}
inc_and_check(&mut migration_version, 144)?;
if dbversion < migration_version {
sql.execute_migration_transaction(
|transaction| {
let is_chatmail = transaction
.query_row(
"SELECT value FROM config WHERE keyname='is_chatmail'",
(),
|row| {
let value: String = row.get(0)?;
Ok(value)
},
)
.optional()?
.as_deref()
== Some("1");
if is_chatmail {
transaction.execute_batch(
"DELETE FROM config WHERE keyname='only_fetch_mvbox';
DELETE FROM config WHERE keyname='show_emails';
UPDATE config SET value='0' WHERE keyname='mvbox_move'",
)?;
}
Ok(())
},
migration_version,
)
.await?;
}
inc_and_check(&mut migration_version, 145)?;
if dbversion < migration_version {
// `msg_id` in `download` table is not needed anymore,
// but we still keep it so that it's possible to import a backup into an older DC version,
// because we don't always release at the same time on all platforms.
sql.execute_migration(
"CREATE TABLE download_new (
rfc724_mid TEXT PRIMARY KEY,
msg_id INTEGER NOT NULL DEFAULT 0
) STRICT;
INSERT OR IGNORE INTO download_new (rfc724_mid, msg_id)
SELECT m.rfc724_mid, d.msg_id FROM download d
LEFT JOIN msgs m ON d.msg_id = m.id
WHERE m.rfc724_mid IS NOT NULL AND m.rfc724_mid != '';
DROP TABLE download;
ALTER TABLE download_new RENAME TO download;
CREATE TABLE available_post_msgs (
rfc724_mid TEXT PRIMARY KEY
) STRICT;
ALTER TABLE msgs ADD COLUMN pre_rfc724_mid TEXT DEFAULT '';
CREATE INDEX msgs_index9 ON msgs (pre_rfc724_mid);",
migration_version,
)
.await?;
}
let new_version = sql
.get_raw_config_int(VERSION_CFG)
.await?
+8 -5
View File
@@ -9,19 +9,20 @@ use anyhow::{Context as _, Result};
use deltachat_derive::FromSql;
use num_traits::ToPrimitive;
use pgp::types::PublicKeyTrait;
use rand::distr::SampleString as _;
use rusqlite::OptionalExtension;
use serde::Serialize;
use crate::chat::{self, ChatId, MuteDuration};
use crate::config::Config;
use crate::constants::Chattype;
use crate::constants::{Chattype, DC_VERSION_STR};
use crate::contact::{Contact, ContactId, Origin, import_vcard, mark_contact_id_as_verified};
use crate::context::{Context, get_version_str};
use crate::context::Context;
use crate::key::load_self_public_keyring;
use crate::log::LogExt;
use crate::message::{Message, Viewtype};
use crate::securejoin::QrInvite;
use crate::tools::{create_id, time};
use crate::tools::time;
pub(crate) const STATISTICS_BOT_EMAIL: &str = "self_reporting@testrun.org";
const STATISTICS_BOT_VCARD: &str = include_str!("../assets/statistics-bot.vcf");
@@ -356,7 +357,7 @@ async fn get_stats(context: &Context) -> Result<String> {
get_timestamps(context, "stats_sending_disabled_events").await?;
let stats = Statistics {
core_version: get_version_str().to_string(),
core_version: DC_VERSION_STR.to_string(),
key_create_timestamps,
stats_id: stats_id(context).await?,
is_chatmail: context.is_chatmail().await?,
@@ -390,7 +391,9 @@ pub(crate) async fn stats_id(context: &Context) -> Result<String> {
Ok(match context.get_config(Config::StatsId).await? {
Some(id) => id,
None => {
let id = create_id();
let id = rand::distr::Alphabetic
.sample_string(&mut rand::rng(), 25)
.to_lowercase();
context
.set_config_internal(Config::StatsId, Some(&id))
.await?;
+1 -1
View File
@@ -46,7 +46,7 @@ async fn test_maybe_send_stats() -> Result<()> {
r.get("contact_stats").unwrap(),
&serde_json::Value::Array(vec![])
);
assert_eq!(r.get("core_version").unwrap(), get_version_str());
assert_eq!(r.get("core_version").unwrap(), DC_VERSION_STR);
assert_eq!(maybe_send_stats(alice).await?, None);
+28 -40
View File
@@ -4,7 +4,6 @@ use std::collections::HashMap;
use std::sync::Arc;
use anyhow::{Result, bail};
use humansize::{BINARY, format_size};
use strum::EnumProperty as EnumPropertyTrait;
use strum_macros::EnumProperty;
use tokio::sync::RwLock;
@@ -17,7 +16,6 @@ use crate::contact::{Contact, ContactId};
use crate::context::Context;
use crate::message::{Message, Viewtype};
use crate::param::Param;
use crate::tools::timestamp_to_str;
/// Storage for string translations.
#[derive(Debug, Clone)]
@@ -109,15 +107,15 @@ pub enum StockMessage {
))]
DeviceMessagesHint = 70,
#[strum(props(fallback = "Welcome to Delta Chat! – \
Delta Chat looks and feels like other popular messenger apps, \
but does not involve centralized control, \
tracking or selling you, friends, colleagues or family out to large organizations.\n\n\
Technically, Delta Chat is an email application with a modern chat interface. \
Email in a new dress if you will 👻\n\n\
Use Delta Chat with anyone out of billions of people: just use their e-mail address. \
Recipients don't need to install Delta Chat, visit websites or sign up anywhere - \
however, of course, if they like, you may point them to 👉 https://get.delta.chat"))]
#[strum(props(fallback = "Get in contact!\n\n\
🙌 Tap \"QR code\" on the main screen of both devices. \
Choose \"Scan QR Code\" on one device, and point it at the other\n\n\
🌍 If not in the same room, \
scan via video call or share an invite link from \"Scan QR code\"\n\n\
Then: Enjoy your decentralized messenger experience. \
In contrast to other popular apps, \
without central control or tracking or selling you, \
friends, colleagues or family out to large organizations."))]
WelcomeMessage = 71,
#[strum(props(fallback = "Message from %1$s"))]
@@ -167,12 +165,6 @@ pub enum StockMessage {
))]
QuotaExceedingMsgBody = 98,
#[strum(props(fallback = "%1$s message"))]
PartialDownloadMsgBody = 99,
#[strum(props(fallback = "Download maximum available until %1$s"))]
DownloadAvailability = 100,
#[strum(props(fallback = "Multi Device Synchronization"))]
SyncMsgSubject = 101,
@@ -1119,21 +1111,6 @@ pub(crate) async fn quota_exceeding(context: &Context, highest_usage: u64) -> St
.replace("%%", "%")
}
/// Stock string: `%1$s message` with placeholder replaced by human-readable size.
pub(crate) async fn partial_download_msg_body(context: &Context, org_bytes: u32) -> String {
let size = &format_size(org_bytes, BINARY);
translated(context, StockMessage::PartialDownloadMsgBody)
.await
.replace1(size)
}
/// Stock string: `Download maximum available until %1$s`.
pub(crate) async fn download_availability(context: &Context, timestamp: i64) -> String {
translated(context, StockMessage::DownloadAvailability)
.await
.replace1(&timestamp_to_str(timestamp))
}
/// Stock string: `Incoming Messages`.
pub(crate) async fn incoming_messages(context: &Context) -> String {
translated(context, StockMessage::IncomingMessages).await
@@ -1144,14 +1121,6 @@ pub(crate) async fn outgoing_messages(context: &Context) -> String {
translated(context, StockMessage::OutgoingMessages).await
}
/// Stock string: `Storage on %1$s`.
/// `%1$s` will be replaced by the domain of the configured email-address.
pub(crate) async fn storage_on_domain(context: &Context, domain: &str) -> String {
translated(context, StockMessage::StorageOnDomain)
.await
.replace1(domain)
}
/// Stock string: `Not connected`.
pub(crate) async fn not_connected(context: &Context) -> String {
translated(context, StockMessage::NotConnected).await
@@ -1262,6 +1231,25 @@ pub(crate) async fn chat_unencrypted_explanation(context: &Context) -> String {
translated(context, StockMessage::ChatUnencryptedExplanation).await
}
impl Viewtype {
/// returns Localized name for message viewtype
pub async fn to_locale_string(&self, context: &Context) -> String {
match self {
Viewtype::Image => image(context).await,
Viewtype::Gif => gif(context).await,
Viewtype::Sticker => sticker(context).await,
Viewtype::Audio => audio(context).await,
Viewtype::Voice => voice_message(context).await,
Viewtype::Video => video(context).await,
Viewtype::File => file(context).await,
Viewtype::Webxdc => "Mini App".to_owned(),
Viewtype::Vcard => "👤".to_string(),
// The following shouldn't normally be shown to users, so translations aren't needed.
Viewtype::Unknown | Viewtype::Text | Viewtype::Call => self.to_string(),
}
}
}
impl Context {
/// Set the stock string for the [StockMessage].
///
-8
View File
@@ -118,14 +118,6 @@ async fn test_quota_exceeding_stock_str() -> Result<()> {
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_partial_download_msg_body() -> Result<()> {
let t = TestContext::new().await;
let str = partial_download_msg_body(&t, 1024 * 1024).await;
assert_eq!(str, "1 MiB message");
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_update_device_chats() {
let t = TestContext::new_alice().await;
+3 -4
View File
@@ -98,14 +98,13 @@ impl Summary {
let prefix = if msg.state == MessageState::OutDraft {
Some(SummaryPrefix::Draft(stock_str::draft(context).await))
} else if msg.from_id == ContactId::SELF {
if msg.is_info() || msg.viewtype == Viewtype::Call {
if msg.is_info() || msg.viewtype == Viewtype::Call || chat.typ == Chattype::OutBroadcast
{
None
} else {
Some(SummaryPrefix::Me(stock_str::self_msg(context).await))
}
} else if chat.typ == Chattype::Group
|| chat.typ == Chattype::OutBroadcast
|| chat.typ == Chattype::InBroadcast
|| chat.typ == Chattype::Mailinglist
|| chat.is_self_talk()
{
@@ -262,7 +261,7 @@ impl Message {
}
};
let text = self.text.clone();
let text = self.text.clone() + &self.additional_text;
let summary = if let Some(type_file) = type_file {
if append_text && !text.is_empty() {
+6 -6
View File
@@ -175,7 +175,7 @@ impl Context {
/// Adds most recent qr-code tokens for the given group or self-contact to the list of items to
/// be synced. If device synchronization is disabled,
/// no tokens exist or the chat is unpromoted, the function does nothing.
/// The caller should call `SchedulerState::interrupt_inbox()` on its own to trigger sending.
/// The caller should call `SchedulerState::interrupt_smtp()` on its own to trigger sending.
pub(crate) async fn sync_qr_code_tokens(&self, grpid: Option<&str>) -> Result<()> {
if !self.should_send_sync_msgs().await? {
return Ok(());
@@ -208,7 +208,7 @@ impl Context {
grpid: None,
}))
.await?;
self.scheduler.interrupt_inbox().await;
self.scheduler.interrupt_smtp().await;
Ok(())
}
@@ -675,7 +675,7 @@ mod tests {
// let alice's other device receive and execute the sync message,
// also here, self-talk should stay hidden
let sent_msg = alice.pop_sent_sync_msg().await;
let sent_msg = alice.pop_sent_msg().await;
let alice2 = TestContext::new_alice().await;
alice2.set_config_bool(Config::SyncMsgs, true).await?;
alice2.recv_msg_trash(&sent_msg).await;
@@ -722,7 +722,7 @@ mod tests {
}))
.await?;
alice1.send_sync_msg().await?.unwrap();
alice1.pop_sent_sync_msg().await
alice1.pop_sent_msg().await
} else {
let chat = alice1.get_self_chat().await;
alice1.send_text(chat.id, "Hi").await
@@ -760,7 +760,7 @@ mod tests {
.set_config(Config::Displayname, Some("Alice Human"))
.await?;
alice.send_sync_msg().await?;
alice.pop_sent_sync_msg().await;
alice.pop_sent_msg().await;
let msg = bob.recv_msg(&alice.pop_sent_msg().await).await;
assert_eq!(msg.text, "hi");
@@ -794,7 +794,7 @@ mod tests {
// group is promoted for compatibility (because the group could be created by older Core).
// TODO: assert!(msg_id.is_none());
assert!(msg_id.is_some());
let sent = alice.pop_sent_sync_msg().await;
let sent = alice.pop_sent_msg().await;
let msg = alice.parse_msg(&sent).await;
let mut sync_items = msg.sync_items.unwrap().items;
assert_eq!(sync_items.len(), 1);
+49 -37
View File
@@ -10,6 +10,7 @@ use std::path::Path;
use std::sync::{Arc, LazyLock};
use std::time::{Duration, Instant};
use anyhow::Result;
use async_channel::{self as channel, Receiver, Sender};
use chat::ChatItem;
use deltachat_contact_tools::{ContactAddress, EmailAddress};
@@ -711,44 +712,30 @@ impl TestContext {
})
}
/// Retrieves a sent sync message from the db.
///
/// This retrieves and removes a sync message which has been scheduled to send from the jobs
/// table. Messages are returned in the order they have been sent.
///
/// Panics if there is no message or on any error.
pub async fn pop_sent_sync_msg(&self) -> SentMessage<'_> {
let (id, msg_id, payload) = self
.ctx
pub async fn get_smtp_rows_for_msg<'a>(&'a self, msg_id: MsgId) -> Vec<SentMessage<'a>> {
self.ctx
.sql
.query_row(
"SELECT id, msg_id, mime \
FROM imap_send \
ORDER BY id",
(),
.query_map_vec(
"SELECT id, msg_id, mime, recipients FROM smtp WHERE msg_id=?",
(msg_id,),
|row| {
let rowid: i64 = row.get(0)?;
let _id: MsgId = row.get(0)?;
let msg_id: MsgId = row.get(1)?;
let mime: String = row.get(2)?;
Ok((rowid, msg_id, mime))
let recipients: String = row.get(3)?;
Ok((msg_id, mime, recipients))
},
)
.await
.expect("query_row failed");
self.ctx
.sql
.execute("DELETE FROM imap_send WHERE id=?", (id,))
.await
.expect("failed to remove job");
update_msg_state(&self.ctx, msg_id, MessageState::OutDelivered)
.await
.expect("failed to update message state");
SentMessage {
payload,
sender_msg_id: msg_id,
sender_context: &self.ctx,
recipients: self.get_primary_self_addr().await.unwrap(),
}
.unwrap()
.into_iter()
.map(|(msg_id, mime, recipients)| SentMessage {
payload: mime,
sender_msg_id: msg_id,
sender_context: &self.ctx,
recipients,
})
.collect()
}
/// Parses a message.
@@ -759,7 +746,7 @@ impl TestContext {
/// unlikely to be affected as the message would be processed again in exactly the
/// same way.
pub(crate) async fn parse_msg(&self, msg: &SentMessage<'_>) -> MimeMessage {
MimeMessage::from_bytes(&self.ctx, msg.payload().as_bytes(), None)
MimeMessage::from_bytes(&self.ctx, msg.payload().as_bytes())
.await
.unwrap()
}
@@ -1474,13 +1461,14 @@ impl EventTracker {
event_matcher: F,
) -> Option<EventType> {
ctx.emit_event(EventType::Test);
let mut found_event = None;
loop {
let event = self.recv().await.unwrap();
if event_matcher(&event.typ) {
return Some(event.typ);
}
if let EventType::Test = event.typ {
return None;
return found_event;
}
if event_matcher(&event.typ) {
found_event = Some(event.typ);
}
}
}
@@ -1504,6 +1492,15 @@ impl EventTracker {
pub fn clear_events(&self) {
while let Ok(_ev) = self.try_recv() {}
}
/// Takes all items from event queue and returns them.
pub fn take_events(&self) -> Vec<Event> {
let mut events = Vec::new();
while let Ok(event) = self.try_recv() {
events.push(event);
}
events
}
}
/// Gets a specific message from a chat and asserts that the chat has a specific length.
@@ -1550,7 +1547,7 @@ pub(crate) async fn mark_as_verified(this: &TestContext, other: &TestContext) {
/// alice0's side that implies sending a sync message.
pub(crate) async fn sync(alice0: &TestContext, alice1: &TestContext) {
alice0.send_sync_msg().await.unwrap();
let sync_msg = alice0.pop_sent_sync_msg().await;
let sync_msg = alice0.pop_sent_msg().await;
alice1.recv_msg_trash(&sync_msg).await;
}
@@ -1702,6 +1699,21 @@ Until the false-positive is fixed:
}
}
/// Method to create a test image file
pub(crate) fn create_test_image(width: u32, height: u32) -> Result<Vec<u8>> {
use image::{ImageBuffer, Rgb, RgbImage};
use std::io::Cursor;
let mut img: RgbImage = ImageBuffer::new(width, height);
// fill with some pattern so it stays large after compression
for (x, y, pixel) in img.enumerate_pixels_mut() {
*pixel = Rgb([(x % 255) as u8, (x + y % 255) as u8, (y % 255) as u8]);
}
let mut bytes: Vec<u8> = Vec::new();
img.write_to(&mut Cursor::new(&mut bytes), image::ImageFormat::Png)?;
Ok(bytes)
}
mod tests {
use super::*;
+1
View File
@@ -1,3 +1,4 @@
mod account_events;
mod aeap;
mod pre_messages;
mod verified_chats;
+6
View File
@@ -0,0 +1,6 @@
mod additional_text;
mod forward_and_save;
mod legacy;
mod receiving;
mod sending;
mod util;
+40
View File
@@ -0,0 +1,40 @@
use anyhow::Result;
use pretty_assertions::assert_eq;
use crate::message::Viewtype;
use crate::test_utils::TestContextManager;
use crate::tests::pre_messages::util::{
send_large_file_message, send_large_image_message, send_large_webxdc_message,
};
/// Test the addition of the download info to message text
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_additional_text_on_different_viewtypes() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
let bob = &tcm.bob().await;
let a_group_id = alice.create_group_with_members("test group", &[bob]).await;
tcm.section("Test metadata preview text for File");
let (pre_message, _, _) =
send_large_file_message(alice, a_group_id, Viewtype::File, &vec![0u8; 1_000_000]).await?;
let msg = bob.recv_msg(&pre_message).await;
assert_eq!(msg.text, "test".to_owned());
assert_eq!(msg.get_text(), "test [test.bin – 976.56 KiB]".to_owned());
tcm.section("Test metadata preview text for webxdc app");
let (pre_message, _, _) = send_large_webxdc_message(alice, a_group_id).await?;
let msg = bob.recv_msg(&pre_message).await;
assert_eq!(msg.text, "test".to_owned());
assert_eq!(msg.get_post_message_viewtype(), Some(Viewtype::Webxdc));
assert_eq!(msg.get_text(), "test [Mini App – 976.68 KiB]".to_owned());
tcm.section("Test metadata preview text for Image");
let (pre_message, _, _) = send_large_image_message(alice, a_group_id).await?;
let msg = bob.recv_msg(&pre_message).await;
assert_eq!(msg.text, "test".to_owned());
assert_eq!(msg.get_text(), "test [Image – 146.12 KiB]".to_owned());
Ok(())
}
+122
View File
@@ -0,0 +1,122 @@
//! Tests about forwarding and saving Pre-Messages
use anyhow::Result;
use pretty_assertions::assert_eq;
use crate::chat::{self};
use crate::chat::{forward_msgs, save_msgs};
use crate::chatlist::get_last_message_for_chat;
use crate::download::{DownloadState, PRE_MSG_ATTACHMENT_SIZE_THRESHOLD};
use crate::message::{Message, Viewtype};
use crate::test_utils::TestContextManager;
/// Test that forwarding Pre-Message should forward additional text to not be empty
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_forwarding_pre_message_empty_text() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
let bob = &tcm.bob().await;
let alice_group_id = alice.create_group_with_members("test group", &[bob]).await;
let pre_message = {
let mut msg = Message::new(Viewtype::File);
msg.set_file_from_bytes(alice, "test.bin", &vec![0u8; 1_000_000], None)?;
assert!(msg.get_filebytes(alice).await?.unwrap() > PRE_MSG_ATTACHMENT_SIZE_THRESHOLD);
let msg_id = chat::send_msg(alice, alice_group_id, &mut msg).await?;
let smtp_rows = alice.get_smtp_rows_for_msg(msg_id).await;
assert_eq!(smtp_rows.len(), 2);
smtp_rows.first().expect("Pre-Message exists").to_owned()
};
let bob_msg = bob.recv_msg(&pre_message).await;
assert_eq!(bob_msg.download_state, DownloadState::Available);
bob_msg.chat_id.accept(bob).await?;
tcm.section("forward pre message and check it on bobs side");
forward_msgs(bob, &[bob_msg.id], bob_msg.chat_id).await?;
let forwarded_msg_id = get_last_message_for_chat(bob, bob_msg.chat_id)
.await?
.unwrap();
let forwarded_msg = Message::load_from_db(bob, forwarded_msg_id).await?;
assert_eq!(forwarded_msg.is_forwarded(), true);
assert_eq!(forwarded_msg.download_state(), DownloadState::Done);
assert_eq!(
forwarded_msg
.param
.exists(crate::param::Param::PostMessageFileBytes),
false,
"PostMessageFileBytes not set"
);
assert_eq!(
forwarded_msg
.param
.exists(crate::param::Param::PostMessageViewtype),
false,
"PostMessageViewtype not set"
);
assert_eq!(
forwarded_msg.get_text(),
" [test.bin – 976.56 KiB]".to_owned()
);
assert_eq!(forwarded_msg.get_viewtype(), Viewtype::Text);
assert!(forwarded_msg.additional_text.is_empty());
tcm.section("check it on alices side");
let sent_forward_msg = bob.pop_sent_msg().await;
let alice_forwarded_msg = alice.recv_msg(&sent_forward_msg).await;
assert!(alice_forwarded_msg.additional_text.is_empty());
assert_eq!(alice_forwarded_msg.is_forwarded(), true);
assert_eq!(alice_forwarded_msg.download_state(), DownloadState::Done);
assert_eq!(
alice_forwarded_msg
.param
.exists(crate::param::Param::PostMessageFileBytes),
false,
"PostMessageFileBytes not set"
);
assert_eq!(
alice_forwarded_msg
.param
.exists(crate::param::Param::PostMessageViewtype),
false,
"PostMessageViewtype not set"
);
assert_eq!(
alice_forwarded_msg.get_text(),
" [test.bin – 976.56 KiB]".to_owned()
);
Ok(())
}
/// Test that forwarding Pre-Message should forward additional text to not be empty
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_saving_pre_message_empty_text() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
let bob = &tcm.bob().await;
let alice_group_id = alice.create_group_with_members("test group", &[bob]).await;
let pre_message = {
let mut msg = Message::new(Viewtype::File);
msg.set_file_from_bytes(alice, "test.bin", &vec![0u8; 1_000_000], None)?;
assert!(msg.get_filebytes(alice).await?.unwrap() > PRE_MSG_ATTACHMENT_SIZE_THRESHOLD);
let msg_id = chat::send_msg(alice, alice_group_id, &mut msg).await?;
let smtp_rows = alice.get_smtp_rows_for_msg(msg_id).await;
assert_eq!(smtp_rows.len(), 2);
smtp_rows.first().expect("Pre-Message exists").to_owned()
};
let bob_msg = bob.recv_msg(&pre_message).await;
assert_eq!(bob_msg.download_state, DownloadState::Available);
bob_msg.chat_id.accept(bob).await?;
tcm.section("save pre message and check it");
save_msgs(bob, &[bob_msg.id]).await?;
let saved_msg_id = get_last_message_for_chat(bob, bob.get_self_chat().await.id)
.await?
.unwrap();
let saved_msg = Message::load_from_db(bob, saved_msg_id).await?;
assert!(saved_msg.additional_text.is_empty());
assert!(saved_msg.get_original_msg_id(bob).await?.is_some());
assert_eq!(saved_msg.download_state(), DownloadState::Done);
assert_eq!(saved_msg.get_text(), " [test.bin – 976.56 KiB]".to_owned());
Ok(())
}
+61
View File
@@ -0,0 +1,61 @@
//! Test that downloading old stub messages still works
use anyhow::Result;
use crate::download::DownloadState;
use crate::receive_imf::receive_imf_from_inbox;
use crate::test_utils::TestContext;
// The code for downloading stub messages stays
// during the transition perios to pre-messages
// so people can still download their files shortly after they updated.
// After there are a few release with pre-message rolled out,
// we will remove the ability to download stub messages and replace the following test
// so it checks that it doesn't crash or that the messages are replaced by sth.
// like "download failed/expired, please ask sender to send it again"
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_download_stub_message() -> Result<()> {
let t = TestContext::new_alice().await;
let header = "Received: (Postfix, from userid 1000); Mon, 4 Dec 2006 14:51:39 +0100 (CET)\n\
From: bob@example.com\n\
To: alice@example.org\n\
Subject: foo\n\
Message-ID: <Mr.12345678901@example.com>\n\
Chat-Version: 1.0\n\
Date: Sun, 22 Mar 2020 22:37:57 +0000\
Content-Type: text/plain";
t.sql
.execute(
r#"INSERT INTO chats VALUES(
11001,100,'bob@example.com',0,'',2,'',
replace('C=1763151754\nt=foo','\n',char(10)),0,0,0,0,0,1763151754,0,NULL,0,'');
"#,
(),
)
.await?;
t.sql.execute(r#"INSERT INTO msgs VALUES(
11001,'Mr.12345678901@example.com','',0,
11001,11001,1,1763151754,10,10,1,0,
'[97.66 KiB message]','','',0,1763151754,1763151754,0,X'',
'','',1,0,'',0,0,0,'foo',10,replace('Hop: From: userid; Date: Mon, 4 Dec 2006 13:51:39 +0000\n\nDKIM Results: Passed=true','\n',char(10)),1,NULL,0,'');
"#, ()).await?;
let msg = t.get_last_msg().await;
assert_eq!(msg.download_state(), DownloadState::Available);
assert_eq!(msg.get_subject(), "foo");
assert!(msg.get_text().contains("[97.66 KiB message]"));
receive_imf_from_inbox(
&t,
"Mr.12345678901@example.com",
format!("{header}\n\n100k text...").as_bytes(),
false,
)
.await?;
let msg = t.get_last_msg().await;
assert_eq!(msg.download_state(), DownloadState::Done);
assert_eq!(msg.get_subject(), "foo");
assert_eq!(msg.get_text(), "100k text...");
Ok(())
}
+599
View File
@@ -0,0 +1,599 @@
//! Tests about receiving Pre-Messages and Post-Message
use anyhow::Result;
use pretty_assertions::assert_eq;
use crate::EventType;
use crate::chat;
use crate::contact;
use crate::download::{DownloadState, PRE_MSG_ATTACHMENT_SIZE_THRESHOLD, PostMsgMetadata};
use crate::message::{Message, MessageState, Viewtype, delete_msgs, markseen_msgs};
use crate::mimeparser::MimeMessage;
use crate::param::Param;
use crate::reaction::{get_msg_reactions, send_reaction};
use crate::test_utils::TestContextManager;
use crate::tests::pre_messages::util::{
send_large_file_message, send_large_image_message, send_large_webxdc_message,
};
use crate::webxdc::StatusUpdateSerial;
/// Test that mimeparser can correctly detect and parse pre-messages and Post-Messages
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_mimeparser_pre_message_and_post_message() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
let bob = &tcm.bob().await;
let alice_group_id = alice.create_group_with_members("test group", &[bob]).await;
let (pre_message, post_message, _alice_msg_id) =
send_large_file_message(alice, alice_group_id, Viewtype::File, &vec![0u8; 1_000_000])
.await?;
let parsed_pre_message = MimeMessage::from_bytes(bob, pre_message.payload.as_bytes()).await?;
let parsed_post_message = MimeMessage::from_bytes(bob, post_message.payload.as_bytes()).await?;
assert_eq!(
parsed_post_message.pre_message,
crate::mimeparser::PreMessageMode::Post,
);
assert_eq!(
parsed_pre_message.pre_message,
crate::mimeparser::PreMessageMode::Pre {
post_msg_rfc724_mid: parsed_post_message.get_rfc724_mid().unwrap(),
metadata: Some(PostMsgMetadata {
size: 1_000_000,
viewtype: Viewtype::File,
filename: "test.bin".to_string(),
wh: None,
duration: None
})
}
);
Ok(())
}
/// Test receiving pre-messages and creation of the placeholder message with the metadata
/// for file attachment
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_receive_pre_message() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
let bob = &tcm.bob().await;
let alice_group_id = alice.create_group_with_members("test group", &[bob]).await;
let (pre_message, _post_message, _alice_msg_id) =
send_large_file_message(alice, alice_group_id, Viewtype::File, &vec![0u8; 1_000_000])
.await?;
let msg = bob.recv_msg(&pre_message).await;
assert_eq!(msg.download_state(), DownloadState::Available);
assert_eq!(msg.viewtype, Viewtype::Text);
assert_eq!(msg.text, "test".to_owned());
// test that metadata is correctly returned by methods
assert_eq!(msg.get_filebytes(bob).await?, Some(1_000_000));
assert_eq!(msg.get_post_message_viewtype(), Some(Viewtype::File));
assert_eq!(msg.get_filename(), Some("test.bin".to_owned()));
Ok(())
}
/// Test receiving the Post-Message after receiving the pre-message
/// for file attachment
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_receive_pre_message_and_dl_post_message() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
let bob = &tcm.bob().await;
let alice_group_id = alice.create_group_with_members("test group", &[bob]).await;
let (pre_message, post_message, _alice_msg_id) =
send_large_file_message(alice, alice_group_id, Viewtype::File, &vec![0u8; 1_000_000])
.await?;
let msg = bob.recv_msg(&pre_message).await;
assert_eq!(msg.download_state(), DownloadState::Available);
assert_eq!(msg.viewtype, Viewtype::Text);
assert!(msg.param.exists(Param::PostMessageViewtype));
assert!(msg.param.exists(Param::PostMessageFileBytes));
assert_eq!(msg.text, "test".to_owned());
let _ = bob.recv_msg_trash(&post_message).await;
let msg = Message::load_from_db(bob, msg.id).await?;
assert_eq!(msg.download_state(), DownloadState::Done);
assert_eq!(msg.viewtype, Viewtype::File);
assert_eq!(msg.param.exists(Param::PostMessageViewtype), false);
assert_eq!(msg.param.exists(Param::PostMessageFileBytes), false);
assert_eq!(msg.text, "test".to_owned());
Ok(())
}
/// Test out of order receiving. Post-Message is received & downloaded before pre-message.
/// In that case pre-message shall be trashed.
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_out_of_order_receiving() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
let bob = &tcm.bob().await;
let alice_group_id = alice.create_group_with_members("test group", &[bob]).await;
let (pre_message, post_message, _alice_msg_id) =
send_large_file_message(alice, alice_group_id, Viewtype::File, &vec![0u8; 1_000_000])
.await?;
let msg = bob.recv_msg(&post_message).await;
assert_eq!(msg.download_state(), DownloadState::Done);
assert_eq!(msg.viewtype, Viewtype::File);
let _ = bob.recv_msg_trash(&pre_message).await;
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_lost_pre_msg() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
let bob = &tcm.bob().await;
let alice_chat_id = alice.create_group_with_members("foos", &[bob]).await;
let file_bytes = include_bytes!("../../../test-data/image/screenshot.gif");
let mut msg = Message::new(Viewtype::Image);
msg.set_file_from_bytes(alice, "a.jpg", file_bytes, None)?;
msg.set_text("populate".to_string());
let full_msg = alice.send_msg(alice_chat_id, &mut msg).await;
let _pre_msg = alice.pop_sent_msg().await;
let msg = bob.recv_msg(&full_msg).await;
assert_eq!(msg.download_state, DownloadState::Done);
assert_eq!(msg.text, "");
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_post_msg_bad_sender() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
let bob = &tcm.bob().await;
let fiona = &tcm.fiona().await;
let chat_id_alice = alice.create_group_with_members("", &[bob, fiona]).await;
let file_bytes = include_bytes!("../../../test-data/image/screenshot.gif");
let mut msg_alice = Message::new(Viewtype::Image);
msg_alice.set_file_from_bytes(alice, "a.jpg", file_bytes, None)?;
let post_msg_alice = alice.send_msg(chat_id_alice, &mut msg_alice).await;
let pre_msg_alice = alice.pop_sent_msg().await;
let msg_bob = bob.recv_msg(&pre_msg_alice).await;
assert_eq!(msg_bob.download_state, DownloadState::Available);
let msg_cnt_bob = msg_bob.chat_id.get_msg_cnt(bob).await?;
let chat_id_fiona = fiona.recv_msg(&pre_msg_alice).await.chat_id;
chat_id_fiona.accept(fiona).await?;
let mut msg_fiona = Message::new(Viewtype::Image);
msg_fiona.rfc724_mid = msg_alice.rfc724_mid.clone();
msg_fiona.set_file_from_bytes(fiona, "a.jpg", file_bytes, None)?;
let post_msg_fiona = fiona.send_msg(chat_id_fiona, &mut msg_fiona).await;
let _pre_msg = fiona.pop_sent_msg().await;
bob.recv_msg_trash(&post_msg_fiona).await;
let msg_bob = Message::load_from_db(bob, msg_bob.id).await?;
assert_eq!(msg_bob.download_state, DownloadState::Available);
assert_eq!(msg_bob.chat_id.get_msg_cnt(bob).await?, msg_cnt_bob);
bob.recv_msg_trash(&post_msg_alice).await;
let msg_bob = Message::load_from_db(bob, msg_bob.id).await?;
assert_eq!(msg_bob.download_state, DownloadState::Done);
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_lost_pre_msg_vs_new_member() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
let bob = &tcm.bob().await;
let fiona = &tcm.fiona().await;
let chat_id_alice = alice.create_group_with_members("", &[bob, fiona]).await;
let file_bytes = include_bytes!("../../../test-data/image/screenshot.gif");
let mut msg_alice = Message::new(Viewtype::Image);
msg_alice.set_file_from_bytes(alice, "a.jpg", file_bytes, None)?;
let post_msg_alice = alice.send_msg(chat_id_alice, &mut msg_alice).await;
let _pre_msg = alice.pop_sent_msg().await;
let msg_bob = bob.recv_msg(&post_msg_alice).await;
assert_eq!(msg_bob.download_state, DownloadState::Done);
let chat_id_bob = msg_bob.chat_id;
assert_eq!(chat::get_chat_contacts(bob, chat_id_bob).await?.len(), 3);
chat_id_bob.accept(bob).await?;
let sent = bob.send_text(chat_id_bob, "Hi all").await;
alice.recv_msg(&sent).await;
fiona.recv_msg_trash(&sent).await; // Undecryptable message
Ok(())
}
/// Test receiving the Post-Message after receiving an edit after receiving the pre-message
/// for file attachment
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_receive_pre_message_then_edit_and_then_dl_post_message() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
let bob = &tcm.bob().await;
let alice_group_id = alice.create_group_with_members("test group", &[bob]).await;
let (pre_message, post_message, alice_msg_id) =
send_large_file_message(alice, alice_group_id, Viewtype::File, &vec![0u8; 1_000_000])
.await?;
chat::send_edit_request(alice, alice_msg_id, "new_text".to_owned()).await?;
let edit_request = alice.pop_sent_msg().await;
let msg = bob.recv_msg(&pre_message).await;
assert_eq!(msg.download_state(), DownloadState::Available);
assert_eq!(msg.text, "test".to_owned());
let _ = bob.recv_msg_trash(&edit_request).await;
let msg = Message::load_from_db(bob, msg.id).await?;
assert_eq!(msg.download_state(), DownloadState::Available);
assert_eq!(msg.text, "new_text".to_owned());
let _ = bob.recv_msg_trash(&post_message).await;
let msg = Message::load_from_db(bob, msg.id).await?;
assert_eq!(msg.download_state(), DownloadState::Done);
assert_eq!(msg.viewtype, Viewtype::File);
assert_eq!(msg.text, "new_text".to_owned());
Ok(())
}
/// Process normal message with file attachment (neither post nor pre message)
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_receive_normal_message() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
let bob = &tcm.bob().await;
let alice_group_id = alice.create_group_with_members("test group", &[bob]).await;
let mut msg = Message::new(Viewtype::File);
msg.set_file_from_bytes(
alice,
"test.bin",
&vec![0u8; (PRE_MSG_ATTACHMENT_SIZE_THRESHOLD - 10_000) as usize],
None,
)?;
msg.set_text("test".to_owned());
let msg_id = chat::send_msg(alice, alice_group_id, &mut msg).await?;
let smtp_rows = alice.get_smtp_rows_for_msg(msg_id).await;
assert_eq!(smtp_rows.len(), 1);
let message = smtp_rows.first().expect("message exists");
let msg = bob.recv_msg(message).await;
assert_eq!(msg.download_state(), DownloadState::Done);
assert_eq!(msg.viewtype, Viewtype::File);
assert_eq!(msg.text, "test".to_owned());
Ok(())
}
/// Test receiving pre-messages and creation of the placeholder message with the metadata
/// for image attachment
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_receive_pre_message_image() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
let bob = &tcm.bob().await;
let alice_group_id = alice.create_group_with_members("test group", &[bob]).await;
let (pre_message, _post_message, _alice_msg_id) =
send_large_image_message(alice, alice_group_id).await?;
let msg = bob.recv_msg(&pre_message).await;
assert_eq!(msg.download_state(), DownloadState::Available);
assert_eq!(msg.viewtype, Viewtype::Text);
assert_eq!(msg.text, "test".to_owned());
// test that metadata is correctly returned by methods
assert_eq!(msg.get_post_message_viewtype(), Some(Viewtype::Image));
// recoded image dimensions
assert_eq!(msg.get_filebytes(bob).await?, Some(149632));
assert_eq!(msg.get_height(), 1280);
assert_eq!(msg.get_width(), 720);
Ok(())
}
/// Test receiving reaction on pre-message
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_reaction_on_pre_message() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
let bob = &tcm.bob().await;
let alice_group_id = alice.create_group_with_members("test group", &[bob]).await;
let (pre_message, post_message, alice_msg_id) =
send_large_file_message(alice, alice_group_id, Viewtype::File, &vec![0u8; 1_000_000])
.await?;
// Bob receives pre-message
let bob_msg = bob.recv_msg(&pre_message).await;
assert_eq!(bob_msg.download_state(), DownloadState::Available);
// Alice sends reaction to her own message
send_reaction(alice, alice_msg_id, "👍").await?;
// Bob receives the reaction
bob.recv_msg_hidden(&alice.pop_sent_msg().await).await;
// Test if Bob sees reaction
let reactions = get_msg_reactions(bob, bob_msg.id).await?;
assert_eq!(reactions.to_string(), "👍1");
// Bob downloads Post-Message
bob.recv_msg_trash(&post_message).await;
let msg = Message::load_from_db(bob, bob_msg.id).await?;
assert_eq!(msg.download_state(), DownloadState::Done);
// Test if Bob still sees reaction
let reactions = get_msg_reactions(bob, bob_msg.id).await?;
assert_eq!(reactions.to_string(), "👍1");
Ok(())
}
/// Tests that fully downloading the message
/// works but does not reappear when it was already deleted
/// (as in the Message-ID already exists in the database
/// and is assigned to the trash chat).
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_full_download_after_trashed() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
let bob = &tcm.bob().await;
let bob_group_id = bob.create_group_with_members("test group", &[alice]).await;
let (pre_message, post_message, _bob_msg_id) =
send_large_file_message(bob, bob_group_id, Viewtype::File, &vec![0u8; 1_000_000]).await?;
// Download message from Bob partially.
let alice_msg = alice.recv_msg(&pre_message).await;
// Delete the received message.
// Note that it remains in the database in the trash chat.
delete_msgs(alice, &[alice_msg.id]).await?;
// Fully download message after deletion.
alice.recv_msg_trash(&post_message).await;
// The message does not reappear.
let msg = Message::load_from_db_optional(bob, alice_msg.id).await?;
assert!(msg.is_none());
Ok(())
}
/// Test that webxdc updates are received for pre-messages
/// and available when the Post-Message is downloaded
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_webxdc_update_for_not_downloaded_instance() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
let bob = &tcm.bob().await;
let alice_group_id = alice.create_group_with_members("test group", &[bob]).await;
// Alice sends a larger instance and an update
let (pre_message, post_message, alice_sent_instance_msg_id) =
send_large_webxdc_message(alice, alice_group_id).await?;
alice
.send_webxdc_status_update(
alice_sent_instance_msg_id,
r#"{"payload": 7, "summary":"sum", "document":"doc"}"#,
)
.await?;
alice.flush_status_updates().await?;
let webxdc_update = alice.pop_sent_msg().await;
// Bob does not download instance but already receives update
let bob_instance = bob.recv_msg(&pre_message).await;
assert_eq!(bob_instance.download_state, DownloadState::Available);
bob.recv_msg_trash(&webxdc_update).await;
// Bob downloads instance, updates should be assigned correctly
bob.recv_msg_trash(&post_message).await;
let bob_instance = bob.get_last_msg().await;
assert_eq!(bob_instance.viewtype, Viewtype::Webxdc);
assert_eq!(bob_instance.download_state, DownloadState::Done);
assert_eq!(
bob.get_webxdc_status_updates(bob_instance.id, StatusUpdateSerial::new(0))
.await?,
r#"[{"payload":7,"document":"doc","summary":"sum","serial":1,"max_serial":1}]"#
);
let info = bob_instance.get_webxdc_info(bob).await?;
assert_eq!(info.document, "doc");
assert_eq!(info.summary, "sum");
Ok(())
}
/// Test mark seen pre-message
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_markseen_pre_msg() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
let bob = &tcm.bob().await;
let bob_chat_id = bob.create_chat(alice).await.id;
alice.create_chat(bob).await; // Make sure the chat is accepted.
tcm.section("Bob sends a large message to Alice");
let (pre_message, post_message, _bob_msg_id) =
send_large_file_message(bob, bob_chat_id, Viewtype::File, &vec![0u8; 1_000_000]).await?;
tcm.section("Alice receives a pre-message message from Bob");
let msg = alice.recv_msg(&pre_message).await;
assert_eq!(msg.download_state, DownloadState::Available);
assert!(msg.param.get_bool(Param::WantsMdn).unwrap_or_default());
assert_eq!(msg.state, MessageState::InFresh);
tcm.section("Alice marks the pre-message as read and sends a MDN");
markseen_msgs(alice, vec![msg.id]).await?;
assert_eq!(msg.id.get_state(alice).await?, MessageState::InSeen);
assert_eq!(
alice
.sql
.count("SELECT COUNT(*) FROM smtp_mdns", ())
.await?,
1
);
tcm.section("Alice downloads message");
alice.recv_msg_trash(&post_message).await;
let msg = Message::load_from_db(alice, msg.id).await?;
assert_eq!(msg.download_state, DownloadState::Done);
assert!(msg.param.get_bool(Param::WantsMdn).unwrap_or_default());
assert_eq!(
msg.state,
MessageState::InSeen,
"The message state mustn't be downgraded to `InFresh`"
);
Ok(())
}
/// Test that pre-message can start a chat
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_pre_msg_can_start_chat() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
let bob = &tcm.bob().await;
tcm.section("establishing a DM chat between alice and bob");
let bob_alice_dm_chat_id = bob.create_chat(alice).await.id;
alice.create_chat(bob).await; // Make sure the chat is accepted.
tcm.section("Alice prepares chat");
let chat_id = chat::create_group(alice, "my group").await?;
let contacts = contact::Contact::get_all(alice, 0, None).await?;
let alice_bob_id = contacts.first().expect("contact exists");
chat::add_contact_to_chat(alice, chat_id, *alice_bob_id).await?;
tcm.section("Alice sends large message to promote/start chat");
let (pre_message, _post_message, _alice_msg_id) =
send_large_file_message(alice, chat_id, Viewtype::File, &vec![0u8; 1_000_000]).await?;
tcm.section("Bob receives the pre-message message from Alice");
let msg = bob.recv_msg(&pre_message).await;
assert_eq!(msg.download_state, DownloadState::Available);
assert_ne!(msg.chat_id, bob_alice_dm_chat_id);
let chat = chat::Chat::load_from_db(bob, msg.chat_id).await?;
assert_eq!(chat.name, "my group");
Ok(())
}
/// Test that Post-Message can start a chat
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_post_msg_can_start_chat() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
let bob = &tcm.bob().await;
tcm.section("establishing a DM chat between alice and bob");
let bob_alice_dm_chat_id = bob.create_chat(alice).await.id;
alice.create_chat(bob).await; // Make sure the chat is accepted.
tcm.section("Alice prepares chat");
let chat_id = chat::create_group(alice, "my group").await?;
let contacts = contact::Contact::get_all(alice, 0, None).await?;
let alice_bob_id = contacts.first().expect("contact exists");
chat::add_contact_to_chat(alice, chat_id, *alice_bob_id).await?;
tcm.section("Alice sends large message to promote/start chat");
let (_pre_message, post_message, _bob_msg_id) =
send_large_file_message(alice, chat_id, Viewtype::File, &vec![0u8; 1_000_000]).await?;
tcm.section("Bob receives the pre-message message from Alice");
let msg = bob.recv_msg(&post_message).await;
assert_eq!(msg.download_state, DownloadState::Done);
assert_ne!(msg.chat_id, bob_alice_dm_chat_id);
let chat = chat::Chat::load_from_db(bob, msg.chat_id).await?;
assert_eq!(chat.name, "my group");
Ok(())
}
/// Test that message ordering is still correct after downloading
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_download_later_keeps_message_order() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
let bob = &tcm.bob().await;
tcm.section(
"establishing a DM chat between alice and bob and bob sends large message to alice",
);
let bob_alice_dm_chat = bob.create_chat(alice).await.id;
alice.create_chat(bob).await; // Make sure the chat is accepted.
let (pre_message, post_message, _bob_msg_id) = send_large_file_message(
bob,
bob_alice_dm_chat,
Viewtype::File,
&vec![0u8; 1_000_000],
)
.await?;
tcm.section("Alice downloads pre-message");
let msg = alice.recv_msg(&pre_message).await;
assert_eq!(msg.download_state, DownloadState::Available);
assert_eq!(msg.state, MessageState::InFresh);
assert_eq!(alice.get_last_msg_in(msg.chat_id).await.id, msg.id);
tcm.section("Bob sends hi to Alice");
let hi_msg = tcm.send_recv(bob, alice, "hi").await;
assert_eq!(alice.get_last_msg_in(msg.chat_id).await.id, hi_msg.id);
tcm.section("Alice downloads Post-Message");
alice.recv_msg_trash(&post_message).await;
let msg = Message::load_from_db(alice, msg.id).await?;
assert_eq!(msg.download_state, DownloadState::Done);
assert_eq!(alice.get_last_msg_in(msg.chat_id).await.id, hi_msg.id);
assert!(msg.timestamp_sort <= hi_msg.timestamp_sort);
Ok(())
}
/// Test that ChatlistItemChanged event is emitted when downloading Post-Message
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_chatlist_event_on_post_msg_download() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
let bob = &tcm.bob().await;
tcm.section(
"establishing a DM chat between alice and bob and bob sends large message to alice",
);
let bob_alice_dm_chat = bob.create_chat(alice).await.id;
alice.create_chat(bob).await; // Make sure the chat is accepted.
let (pre_message, post_message, _bob_msg_id) = send_large_file_message(
bob,
bob_alice_dm_chat,
Viewtype::File,
&vec![0u8; 1_000_000],
)
.await?;
tcm.section("Alice downloads pre-message");
let msg = alice.recv_msg(&pre_message).await;
assert_eq!(msg.download_state, DownloadState::Available);
assert_eq!(msg.state, MessageState::InFresh);
assert_eq!(alice.get_last_msg_in(msg.chat_id).await.id, msg.id);
tcm.section("Alice downloads Post-Message and waits for ChatlistItemChanged event ");
alice.evtracker.clear_events();
alice.recv_msg_trash(&post_message).await;
let msg = Message::load_from_db(alice, msg.id).await?;
assert_eq!(msg.download_state, DownloadState::Done);
alice
.evtracker
.get_matching(|e| {
e == &EventType::ChatlistItemChanged {
chat_id: Some(msg.chat_id),
}
})
.await;
Ok(())
}
+337
View File
@@ -0,0 +1,337 @@
//! Tests about sending pre-messages
//! - When to send a pre-message and post-message instead of a normal message
//! - Test that sent pre- and post-message contain the right Headers
//! and that they are send in the correct order (pre-message is sent first.)
use anyhow::Result;
use mailparse::MailHeaderMap;
use tokio::fs;
use crate::chat::{self, create_group, send_msg};
use crate::config::Config;
use crate::download::PRE_MSG_ATTACHMENT_SIZE_THRESHOLD;
use crate::headerdef::{HeaderDef, HeaderDefMap};
use crate::message::{Message, Viewtype};
use crate::test_utils::{self, TestContext, TestContextManager};
/// Tests that Pre-Message is sent for attachment larger than `PRE_MSG_ATTACHMENT_SIZE_THRESHOLD`
/// Also test that Pre-Message is sent first, before the Post-Message
/// And that Autocrypt-gossip and selfavatar never go into Post-Messages
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_sending_pre_message() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
let bob = &tcm.bob().await;
let fiona = &tcm.fiona().await;
let group_id = alice
.create_group_with_members("test group", &[bob, fiona])
.await;
let mut msg = Message::new(Viewtype::File);
msg.set_file_from_bytes(alice, "test.bin", &[0u8; 300_000], None)?;
msg.set_text("test".to_owned());
// assert that test attachment is bigger than limit
assert!(msg.get_filebytes(alice).await?.unwrap() > PRE_MSG_ATTACHMENT_SIZE_THRESHOLD);
let msg_id = chat::send_msg(alice, group_id, &mut msg).await?;
let smtp_rows = alice.get_smtp_rows_for_msg(msg_id).await;
// Pre-Message and Post-Message should be present
// and test that correct headers are present on both messages
assert_eq!(smtp_rows.len(), 2);
let pre_message = smtp_rows.first().expect("first element exists");
let pre_message_parsed = mailparse::parse_mail(pre_message.payload.as_bytes())?;
let post_message = smtp_rows.get(1).expect("second element exists");
let post_message_parsed = mailparse::parse_mail(post_message.payload.as_bytes())?;
assert!(
pre_message_parsed
.headers
.get_first_header(HeaderDef::ChatIsPostMessage.get_headername())
.is_none()
);
assert!(
post_message_parsed
.headers
.get_first_header(HeaderDef::ChatIsPostMessage.get_headername())
.is_some()
);
assert_eq!(
post_message_parsed
.headers
.get_header_value(HeaderDef::MessageId),
Some(format!("<{}>", msg.rfc724_mid)),
"Post-Message should have the rfc message id of the database message"
);
assert_ne!(
pre_message_parsed
.headers
.get_header_value(HeaderDef::MessageId),
post_message_parsed
.headers
.get_header_value(HeaderDef::MessageId),
"message ids of Pre-Message and Post-Message should be different"
);
let decrypted_post_message = bob.parse_msg(post_message).await;
assert_eq!(decrypted_post_message.decrypting_failed, false);
assert_eq!(
decrypted_post_message.header_exists(HeaderDef::ChatPostMessageId),
false
);
let decrypted_pre_message = bob.parse_msg(pre_message).await;
assert_eq!(
decrypted_pre_message
.get_header(HeaderDef::ChatPostMessageId)
.map(String::from),
post_message_parsed
.headers
.get_header_value(HeaderDef::MessageId)
);
assert!(
pre_message_parsed
.headers
.get_header_value(HeaderDef::ChatPostMessageId)
.is_none(),
"no Chat-Post-Message-ID header in unprotected headers of Pre-Message"
);
Ok(())
}
/// Tests that Pre-Message has autocrypt gossip headers and self avatar
/// and Post-Message doesn't have these headers
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_selfavatar_and_autocrypt_gossip_goto_pre_message() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
let bob = &tcm.bob().await;
let fiona = &tcm.fiona().await;
let group_id = alice
.create_group_with_members("test group", &[bob, fiona])
.await;
let mut msg = Message::new(Viewtype::File);
msg.set_file_from_bytes(alice, "test.bin", &[0u8; 300_000], None)?;
msg.set_text("test".to_owned());
// assert that test attachment is bigger than limit
assert!(msg.get_filebytes(alice).await?.unwrap() > PRE_MSG_ATTACHMENT_SIZE_THRESHOLD);
// simulate conditions for sending self avatar
let avatar_src = alice.get_blobdir().join("avatar.png");
fs::write(&avatar_src, test_utils::AVATAR_900x900_BYTES).await?;
alice
.set_config(Config::Selfavatar, Some(avatar_src.to_str().unwrap()))
.await?;
let msg_id = chat::send_msg(alice, group_id, &mut msg).await?;
let smtp_rows = alice.get_smtp_rows_for_msg(msg_id).await;
assert_eq!(smtp_rows.len(), 2);
let pre_message = smtp_rows.first().expect("first element exists");
let post_message = smtp_rows.get(1).expect("second element exists");
let post_message_parsed = mailparse::parse_mail(post_message.payload.as_bytes())?;
let decrypted_pre_message = bob.parse_msg(pre_message).await;
assert!(
decrypted_pre_message
.get_header(HeaderDef::ChatPostMessageId)
.is_some(),
"tested message is not a pre-message, sending order may be broken"
);
assert_ne!(decrypted_pre_message.gossiped_keys.len(), 0);
assert_ne!(decrypted_pre_message.user_avatar, None);
let decrypted_post_message = bob.parse_msg(post_message).await;
assert!(
post_message_parsed
.headers
.get_first_header(HeaderDef::ChatIsPostMessage.get_headername())
.is_some(),
"tested message is not a Post-Message, sending order may be broken"
);
assert_eq!(decrypted_post_message.gossiped_keys.len(), 0);
assert_eq!(decrypted_post_message.user_avatar, None);
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_unecrypted_gets_no_pre_message() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
let chat = alice
.create_chat_with_contact("example", "email@example.org")
.await;
let mut msg = Message::new(Viewtype::File);
msg.set_file_from_bytes(alice, "test.bin", &vec![0u8; 300_000], None)?;
msg.set_text("test".to_owned());
let msg_id = chat::send_msg(alice, chat.id, &mut msg).await?;
let smtp_rows = alice.get_smtp_rows_for_msg(msg_id).await;
assert_eq!(smtp_rows.len(), 1);
let message_bytes = smtp_rows
.first()
.expect("first element exists")
.payload
.as_bytes();
let message = mailparse::parse_mail(message_bytes)?;
assert!(
message
.headers
.get_first_header(HeaderDef::ChatIsPostMessage.get_headername())
.is_none(),
);
Ok(())
}
/// Tests that no pre message is sent for normal message
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_not_sending_pre_message_no_attachment() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
let bob = &tcm.bob().await;
let chat = alice.create_chat(bob).await;
// send normal text message
let mut msg = Message::new(Viewtype::Text);
msg.set_text("test".to_owned());
let msg_id = chat::send_msg(alice, chat.id, &mut msg).await.unwrap();
let smtp_rows = alice.get_smtp_rows_for_msg(msg_id).await;
assert_eq!(smtp_rows.len(), 1, "only one message should be sent");
let msg = smtp_rows.first().expect("first element exists");
let mail = mailparse::parse_mail(msg.payload.as_bytes())?;
assert!(
mail.headers
.get_first_header(HeaderDef::ChatIsPostMessage.get_headername())
.is_none(),
"no 'Chat-Is-Post-Message'-header should be present"
);
assert!(
mail.headers
.get_first_header(HeaderDef::ChatPostMessageId.get_headername())
.is_none(),
"no 'Chat-Post-Message-ID'-header should be present in clear text headers"
);
let decrypted_message = bob.parse_msg(msg).await;
assert!(
!decrypted_message.header_exists(HeaderDef::ChatPostMessageId),
"no 'Chat-Post-Message-ID'-header should be present"
);
// test that pre message is not send for large large text
let mut msg = Message::new(Viewtype::Text);
let long_text = String::from_utf8(vec![b'a'; 300_000])?;
assert!(long_text.len() > PRE_MSG_ATTACHMENT_SIZE_THRESHOLD.try_into().unwrap());
msg.set_text(long_text);
let msg_id = chat::send_msg(alice, chat.id, &mut msg).await.unwrap();
let smtp_rows = alice.get_smtp_rows_for_msg(msg_id).await;
assert_eq!(smtp_rows.len(), 1, "only one message should be sent");
let msg = smtp_rows.first().expect("first element exists");
let mail = mailparse::parse_mail(msg.payload.as_bytes())?;
assert!(
mail.headers
.get_first_header(HeaderDef::ChatIsPostMessage.get_headername())
.is_none()
);
assert!(
mail.headers
.get_first_header(HeaderDef::ChatPostMessageId.get_headername())
.is_none(),
"no 'Chat-Post-Message-ID'-header should be present in clear text headers"
);
let decrypted_message = bob.parse_msg(msg).await;
assert!(
!decrypted_message.header_exists(HeaderDef::ChatPostMessageId),
"no 'Chat-Post-Message-ID'-header should be present"
);
Ok(())
}
/// Tests that no pre message is sent for attachment smaller than `PRE_MSG_ATTACHMENT_SIZE_THRESHOLD`
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_not_sending_pre_message_for_small_attachment() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
let bob = &tcm.bob().await;
let chat = alice.create_chat(bob).await;
let mut msg = Message::new(Viewtype::File);
msg.set_file_from_bytes(alice, "test.bin", &vec![0u8; 100_000], None)?;
msg.set_text("test".to_owned());
// assert that test attachment is smaller than limit
assert!(msg.get_filebytes(alice).await?.unwrap() < PRE_MSG_ATTACHMENT_SIZE_THRESHOLD);
let msg_id = chat::send_msg(alice, chat.id, &mut msg).await.unwrap();
let smtp_rows = alice.get_smtp_rows_for_msg(msg_id).await;
// only one message and no "is Post-Message" header should be present
assert_eq!(smtp_rows.len(), 1);
let msg = smtp_rows.first().expect("first element exists");
let mail = mailparse::parse_mail(msg.payload.as_bytes())?;
assert!(
mail.headers
.get_first_header(HeaderDef::ChatIsPostMessage.get_headername())
.is_none()
);
assert!(
mail.headers
.get_first_header(HeaderDef::ChatPostMessageId.get_headername())
.is_none(),
"no 'Chat-Post-Message-ID'-header should be present in clear text headers"
);
let decrypted_message = bob.parse_msg(msg).await;
assert!(
!decrypted_message.header_exists(HeaderDef::ChatPostMessageId),
"no 'Chat-Post-Message-ID'-header should be present"
);
Ok(())
}
/// Tests that pre message is not send for large webxdc updates
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_render_webxdc_status_update_object_range() -> Result<()> {
let t = TestContext::new_alice().await;
let chat_id = create_group(&t, "a chat").await?;
let instance = {
let mut instance = Message::new(Viewtype::File);
instance.set_file_from_bytes(
&t,
"minimal.xdc",
include_bytes!("../../../test-data/webxdc/minimal.xdc"),
None,
)?;
let instance_msg_id = send_msg(&t, chat_id, &mut instance).await?;
assert_eq!(instance.viewtype, Viewtype::Webxdc);
Message::load_from_db(&t, instance_msg_id).await
}
.unwrap();
t.pop_sent_msg().await;
assert_eq!(t.sql.count("SELECT COUNT(*) FROM smtp", ()).await?, 0);
let long_text = String::from_utf8(vec![b'a'; 300_000])?;
assert!(long_text.len() > PRE_MSG_ATTACHMENT_SIZE_THRESHOLD.try_into().unwrap());
t.send_webxdc_status_update(instance.id, &format!("{{\"payload\": \"{long_text}\"}}"))
.await?;
t.flush_status_updates().await?;
assert_eq!(t.sql.count("SELECT COUNT(*) FROM smtp", ()).await?, 1);
Ok(())
}
+65
View File
@@ -0,0 +1,65 @@
use anyhow::Result;
use async_zip::tokio::write::ZipFileWriter;
use async_zip::{Compression, ZipEntryBuilder};
use futures::io::Cursor as FuturesCursor;
use pretty_assertions::assert_eq;
use tokio_util::compat::FuturesAsyncWriteCompatExt;
use crate::chat::{self, ChatId};
use crate::download::PRE_MSG_ATTACHMENT_SIZE_THRESHOLD;
use crate::message::{Message, MsgId, Viewtype};
use crate::test_utils::{SentMessage, TestContext, create_test_image};
pub async fn send_large_file_message<'a>(
sender: &'a TestContext,
target_chat: ChatId,
view_type: Viewtype,
content: &[u8],
) -> Result<(SentMessage<'a>, SentMessage<'a>, MsgId)> {
let mut msg = Message::new(view_type);
let file_name = if view_type == Viewtype::Webxdc {
"test.xdc"
} else {
"test.bin"
};
msg.set_file_from_bytes(sender, file_name, content, None)?;
msg.set_text("test".to_owned());
// assert that test attachment is bigger than limit
assert!(msg.get_filebytes(sender).await?.unwrap() > PRE_MSG_ATTACHMENT_SIZE_THRESHOLD);
let msg_id = chat::send_msg(sender, target_chat, &mut msg).await?;
let smtp_rows = sender.get_smtp_rows_for_msg(msg_id).await;
assert_eq!(smtp_rows.len(), 2);
let pre_message = smtp_rows.first().expect("Pre-Message exists");
let post_message = smtp_rows.get(1).expect("Post-Message exists");
Ok((pre_message.to_owned(), post_message.to_owned(), msg_id))
}
pub async fn send_large_webxdc_message<'a>(
sender: &'a TestContext,
target_chat: ChatId,
) -> Result<(SentMessage<'a>, SentMessage<'a>, MsgId)> {
let futures_cursor = FuturesCursor::new(Vec::new());
let mut buffer = futures_cursor.compat_write();
let mut writer = ZipFileWriter::with_tokio(&mut buffer);
writer
.write_entry_whole(
ZipEntryBuilder::new("index.html".into(), Compression::Stored),
&[0u8; 1_000_000],
)
.await?;
writer.close().await?;
let big_webxdc_app = buffer.into_inner().into_inner();
send_large_file_message(sender, target_chat, Viewtype::Webxdc, &big_webxdc_app).await
}
pub async fn send_large_image_message<'a>(
sender: &'a TestContext,
target_chat: ChatId,
) -> Result<(SentMessage<'a>, SentMessage<'a>, MsgId)> {
let (width, height) = (1080, 1920);
let test_img = create_test_image(width, height)?;
send_large_file_message(sender, target_chat, Viewtype::Image, &test_img).await
}

Some files were not shown because too many files have changed in this diff Show More