Compare commits

..

179 Commits

Author SHA1 Message Date
link2xt
7bc9dd6c98 chore(release): prepare for 1.155.0 2025-01-23 03:06:03 +00:00
link2xt
1a3a09dfc3 build: nix flake update fenix 2025-01-23 02:59:10 +00:00
link2xt
32459b3fdc Reapply "build: increase MSRV to 1.81.0"
This reverts commit 9d331483e9.
2025-01-23 02:59:10 +00:00
link2xt
52e9daaa1f Revert "build: downgrade Rust version used to build binaries"
This reverts commit d822da3c9f.
2025-01-23 02:59:10 +00:00
link2xt
a3734a5f87 Reapply "chore(cargo): bump rustyline from 14.0.0 to 15.0.0"
This reverts commit 6cd6aca7b8.
2025-01-23 02:59:10 +00:00
link2xt
30e1df0754 Revert "bulid(nix): use new fenix for dev shell"
This reverts commit 5069b585c8.
2025-01-23 02:59:10 +00:00
Hocuri
3959305b4a feat: Deduplicate in more places (#6464)
Deduplicate:
- In the REPL
- In `store_from_base64()`, which writes avatars received in headers
- In a few tests
- The saved messages, broadcast, device, archive icons
- The autocrypt setup message

1-2 more PRs, and we can get rid of `BlobObject::create`,
`sanitise_name()`, and some others
2025-01-22 20:39:18 +00:00
link2xt
744cab1553 feat: expire past members after 60 days 2025-01-22 20:39:15 +00:00
Hocuri
8f58c4777e feat: Keep file extension on deduplicated files (#6463)
fix #6461
2025-01-22 16:44:59 +01:00
link2xt
8dcd8aa69d api: add JSON-RPC API to get past members 2025-01-22 11:53:56 +00:00
Hocuri
65a9c4b79b File deduplication (#6332)
When receiving messages, blobs will be deduplicated with the new
function `create_and_deduplicate_from_bytes()`. For sending files, this
adds a new function `set_file_and_deduplicate()` instead of
deduplicating by default.

This is for
https://github.com/deltachat/deltachat-core-rust/issues/6265; read the
issue description there for more details.

TODO:
- [x] Set files as read-only
- [x] Don't do a write when the file is already identical
- [x] The first 32 chars or so of the 64-character hash are enough. I
calculated that if 10b people (i.e. all of humanity) use DC, and each of
them has 200k distinct blob files (I have 4k in my day-to-day account),
and we used 20 chars, then the expected value for the number of name
collisions would be ~0.0002 (and the probability that there is a least
one name collision is lower than that) [^1]. I added 12 more characters
to be on the super safe side, but this wouldn't be necessary and I could
also make it 20 instead of 32.
- Not 100% sure whether that's necessary at all - it would mainly be
necessary if we might hit a length limit on some file systems (the
blobdir is usually sth like
`accounts/2ff9fc096d2f46b6832b24a1ed99c0d6/dc.db-blobs` (53 chars), plus
64 chars for the filename would be 117).
- [x] "touch" the files to prevent them from being deleted
- [x] TODOs in the code

For later PRs:
- Replace `BlobObject::create(…)` with
`BlobObject::create_and_deduplicate(…)` in order to deduplicate
everytime core creates a file
- Modify JsonRPC to deduplicate blob files
- Possibly rename BlobObject.name to BlobObject.file in order to prevent
confusion (because `name` usually means "user-visible-name", not "name
of the file on disk").

[^1]: Calculated with both https://printfn.github.io/fend/ and
https://www.geogebra.org/calculator, both of which came to the same
result
([1](https://github.com/user-attachments/assets/bbb62550-3781-48b5-88b1-ba0e29c28c0d),

[2](https://github.com/user-attachments/assets/82171212-b797-4117-a39f-0e132eac7252))

---------

Co-authored-by: l <link2xt@testrun.org>
2025-01-21 19:42:19 +01:00
Hocuri
22a7cfe9c3 refactor: Extract group_changes_msgs() function (#6460) 2025-01-21 17:38:35 +01:00
Hocuri
1ebf2c1985 refactor: Move tests to their own files
- This [is said to lead improve compilation
  speed](https://matklad.github.io/2021/02/27/delete-cargo-integration-tests.html#Assorted-Tricks)
- When grepping for a function invocation, this makes it easy to see whether it's from a test or "real" code
- We're calling the files e.g. `chat_tests.rs` instead of `tests.rs` for the same reason why we moved `imap/mod.rs` to `imap.rs`: Otherwise, your editor always shows you that you're in the file `tests.rs` and you don't know which one.

This is only moving mimeparser and chat tests, because these were the
biggest files; we can move more files in subsequent PRs if we like it.
2025-01-20 23:10:24 +00:00
Hocuri
723ff25067 feat: Set BccSelf to true when receiving a sync message (#6434)
Fix https://github.com/deltachat/deltachat-core-rust/issues/6433

I at first only wanted to do it any outgoing messages, but @link2xt was
concerned that this may accidentally enable bcc_self, e.g. in the
following case:
- you send out a message
- it's deleted, e.g. via ephemeral messages
- Someone forwards this outgoing message to you again, e.g. via a
mailing list.
2025-01-20 22:05:29 +01:00
link2xt
2b5ce35c2d chore(release): prepare for 1.154.3 2025-01-20 20:22:40 +00:00
Hocuri
39bf3bee59 chore: Remove unused function delete_files_in_dir() (#6454) 2025-01-20 18:53:36 +01:00
link2xt
e3b9c9b209 build(nix): update rust-email hash in flake.nix 2025-01-20 16:55:47 +00:00
link2xt
74930e995d build: remove encoded-words from flake.nix 2025-01-20 16:42:04 +00:00
link2xt
8af6cdf49c chore(release): prepare for 1.154.2 2025-01-20 15:22:05 +00:00
link2xt
19a841657c fix: do not create tombstones for members removed from unpromoted groups
If we create an unpromoted group,
add a member there and then remove it
before we promote a group, there is no need to
add such member to the list of past members
and send the address of this member to the group
when it is promoted.
2025-01-20 15:12:35 +00:00
Hocuri
d4b1f8694f fix: Don't accidentally remove Self from groups (#6455) 2025-01-20 15:54:17 +01:00
bjoern
0d8c2ee9ff feat: add API to save messages (#5606)
> _took quite some time until i found the time to finish this PR and to
find a time window that does not disturb other developments too much,
but here we are:_

this PR enables UI to improve "Saved messages" hugely, bringing it on
WhatsApp's "Starred Messages" or Telegram's "Saved Messages" level. with
this PR, UIs can add the following functionality with few effort ([~100
loc on iOS](https://github.com/deltachat/deltachat-ios/pull/2527)):

- add a "Save" button in the messages context menu, allowing to save a
message
- show directly in the chat if a message was saved, eg. by a little star
★
- in "Saved Messages", show the message in its context - so with author,
avatar, date and keep incoming/outgoing state
- in "Saved Messages", a button to go to the original message can be
added
- if the original message was deleted, one can still go to the original
chat

these features were often requested, in the forum, but also in many
one-to-one discussions, recently at the global gathering.

moreover, in contrast to the old method with "forward to saved", no
traffic is wasted - the messages are saved locally, and only a small
sync messages is sent around

this is how it looks out on iOS:

<img width="260" alt="Screenshot 2025-01-17 at 00 02 51"
src="https://github.com/user-attachments/assets/902741b6-167f-4af1-9f9a-bd0439dd20d0"
/> &nbsp; &nbsp; <img width="353" alt="Screenshot 2025-01-17 at 00 05
33"
src="https://github.com/user-attachments/assets/97eceb79-6e43-4a89-9792-2e077e7141cd"
/>

technically, still a copy is done from the original message (with
already now deduplicated blobs), so that things work nicely with
deletion modes; eg. you can save an important message and preserve it
from deletion that way.

jsonrpc can be done in a subsequent PR, i was implementing the UI on iOS
where that was not needed (and most API were part of message object that
is not in use in jsonrpc atm)

@hpk42 the forward issue we discussed earller that day is already solved
(first implementation did not had an explict save_msgs() but was using
forward_msgs(SELF) as saving - with the disadvantage that forwarding to
SELF is not working, eg. if one wants the old behaviour) acutally, that
made the PR a lot nicer, as now very few existing code is changed

<details>
<summary>previous considerations and abandoned things</summary>

while working on this PR, there was also the idea to just set a flag
“starred” in the message table and not copy anything. however, while
tempting, that would result in more complexity, questions and drawbacks
in UI:

- delete-message, delete-chat, auto-deletion, clear-chat would raise
questions - what do do with the “Starred”? having a copy in “Saved
Messages” does not raise this question
- newly saved messages appear naturally as “new” in “Saved Messages”,
simply setting a flag would show them somewhere in between - unless we
do additional effort
- “Saved Messages” already has its place in the UI - and allows to
_directly_ save things there as well - not easily doable with “starring”
- one would need to re-do many things that already exist in “Saved
Messages”, at least in core
- one idea to solve some of the issues could be to have “Starred” as
well as “Saved Messages” - however, that would irritate user, one would
remember exactly what was done with a message, and understand the fine
differences

whatsapp does this “starred”, btw, so when original is deleted, starred
is deleted as well. Telegram does things similar to us, Signal does
nothing. Whatsapp has a per-chat view for starred messages - if needed,
we could do sth. like that as well, however, let’s first see how far the
“all view” goes (in contrast to whatsapp, we have profiles for
separation as well)

for the wording: “saving” is what we’re doing here, this is much more on
point as “starring” - which is more the idea of a “bookmark”, and i
think, whatsapp uses this wording to not raise false expectations
(still, i know of ppl that were quite upset that a “starred” message was
deleted when eg. the chat was cleared to save some memory)

wrt webxdc app updates: options that come into mind were: _empty_ (as
now), _snapshot_ (copy all updates) or _shortcut_ (always open
original). i am not sure what the best solution is, the easiest was
_empty_, so i went for that, as it is (a) obvious, and what is already
done with forwarding and (b) the original is easy to open now (in
contrast to forwarding).
so, might totally be that we need or want to tweak things here, but i
would leave that outside the first iteration, things are not worsened in
that area

wrt reactions: as things are detached, similar to webxdc updates, we do
not not to show the original reactions - that way, one can use reactions
for private markers (telegram is selling that feature :)

to the icon: a disk or a bookmark might be other options, but the star
is nicer and also know from other apps - and anyways a but vague UX
wise. so i think, it is fine

finally, known issue is that if a message was saved that does not exist
on another device, it does not get there. i think, that issue is a weak
one, and can be ignored mostly, most times, user will save messages soon
after receiving, and if on some devices auto-deletion is done, it is
maybe not even expected to have suddenly another copy there

</details>

EDIT: once this is merged, detailed issues about what to do should be
filed for android/desktop (however, they do not have urgency to adapt,
things will continue working as is)

---------

Co-authored-by: Hocuri <hocuri@gmx.de>
2025-01-19 15:44:57 +00:00
link2xt
3cbfb47b6e build: switch to non-git version of encoded-words 2025-01-19 10:51:46 +00:00
link2xt
0b9746b57e refactor: make memberlist update logic easier to follow 2025-01-18 22:22:19 +00:00
link2xt
fa016b36fb chore(release): prepare for 1.154.1 2025-01-15 19:23:27 +00:00
link2xt
69e01b5197 test: expect trashing of no-op "member added" in non_member_cannot_modify_member_list 2025-01-15 18:27:55 +00:00
link2xt
ffd2ec9424 chore(release): prepare for 1.154.0 2025-01-15 17:56:40 +00:00
link2xt
498979c608 fix: trash no-op addition messages
This partially restores the fix from c9cf2b7f2e
that was removed during the addition of new group consistency at de63527d94
but only for "Member added" messages.

Multiple "Member added" messages happen
when the same QR code is processed multiple times
by multiple devices.
2025-01-15 17:41:15 +00:00
link2xt
3e7b662796 feat: do not allow non-members to modify member list 2025-01-15 16:43:23 +00:00
Hocuri
6057b40910 fix: Clear the config cache after every migration (#6438)
Some migrations change the `config` table, but they don't update the
cache. While this wasn't the cause for
https://github.com/deltachat/deltachat-core-rust/issues/6432, it might
have caused a similar bug, so, let's clear the config cache after every
migration.
2025-01-15 12:35:41 +01:00
iequidoo
53572fce5c fix: migration: Set bcc_self=1 if it's unset and delete_server_after!=1 (#6432)
Users report that in a setup with Android (1.50.4 from F-Droid) and Desktop (1.48.0 x86_64 .deb
release) and chatmail account `bcc_self` was reverted to 0 on Android, resulting in messages sent
from Android not appearing on Desktop. This might happen because of the bug in migration #127, it
doesn't handle `delete_server_after` > 1. Existing chatmail configurations having
`delete_server_after` != 1 ("delete at once") should get `bcc_self` enabled, they may be multidevice
configurations:
- Before migration #127, `delete_server_after` was set to 0 upon a backup export, but
  then `bcc_self` is enabled instead (whose default is changed to 0 for chatmail).
- The user might set `delete_server_after` to a value other than 0 or 1 when that was
  possible in UIs.
So let's add another migration fixing this. But still don't check `is_chatmail` for simplicity.
2025-01-15 00:27:13 -03:00
iequidoo
53dca8ce1a refactor: Eliminate remaining repeat_vars() calls (#6359)
Using `repeat_vars()` to generate SQL statements led to some of them having more than
`SQLITE_MAX_VARIABLE_NUMBER` parameters and thus failing, so let's get rid of this pattern. But
let's not optimise for now and just repeat executing an SQL statement in a loop, all the places
where `repeat_vars()` is used seem not performance-critical and containing functions execute other
SQL statements in loops. If needed, performance can be improved by preparing a statement and
executing it in a loop. An exception is `lookup_chat_or_create_adhoc_group()` where `repeat_vars()`
can't be replaced with a query loop, there we need to replace the `SELECT` query with a read
transaction creating a temporary table which is used to perform the SELECT query then.
2025-01-14 01:14:09 -03:00
link2xt
29d7e0131e refactor: remove unnecessary is_contact_in_chat check 2025-01-14 00:27:37 +00:00
iequidoo
4ec50d1990 refactor: Add why_cant_send_ex() capable to only ignore specified conditions
Before, `Chat::why_cant_send()` just returned `CantSendReason` after the first unsuccessful check
allowing to handle the result and finally send the message if the condition is acceptable in which
case the remaining checks are not done. This didn't result in any bugs, but to make the code more
robust let's add a functional parameter to filter failed checks without early return.
2025-01-12 01:13:53 -03:00
link2xt
187274d7b7 fix: create new tombstone in chats_contacts if the row does not exist
Otherwise new members do not see past members
even if they receive info about them in every message.
2025-01-12 01:42:02 +00:00
Hocuri
5dc8788eab chore: Beta Clippy suggestions (#6422) 2025-01-11 17:58:38 +01:00
link2xt
de63527d94 feat: new group consistency algorithm
This implements new group consistency algorithm described in
<https://github.com/deltachat/deltachat-core-rust/issues/6401>

New `Chat-Group-Member-Timestamps` header is added
to send timestamps of member additions and removals.
Member is part of the chat if its addition timestamp
is greater or equal to the removal timestamp.
2025-01-11 07:52:49 +00:00
link2xt
cb43382896 ci: update Rust to 1.84.0 2025-01-10 01:58:08 +00:00
Simon Laux
a9e177f1e7 build!: remove jsonrpc feature flag
It is enabled everywhere by default since some time now. Breaking, because existing build scripts might need to be adjusted.
2025-01-09 15:56:07 +00:00
link2xt
6e8668e348 build: increase minimum supported Python version to 3.8
Python 3.7 is not supported on GitHub Actions ubuntu-latest runner:
https://github.com/actions/setup-python/issues/962

Python 3.7 has reached EOL more than 1 year ago anyway,
so not worth the effort supporting it.
2025-01-09 14:58:01 +00:00
link2xt
7f7c76f706 test: use assert_eq! to compare chatlist length 2025-01-05 23:44:34 +00:00
link2xt
3fe9a7b17f refactor: use let..else 2025-01-05 23:44:28 +00:00
link2xt
fff4020013 chore(release): prepare for 1.153.0 2025-01-05 09:08:23 +00:00
link2xt
4ffc0ca047 refactor: don't ignore get_for_contact errors 2025-01-05 02:52:19 +00:00
link2xt
3d19996f34 test: fix test_logged_ac_process_ffi_failure flakiness
This test keeps failing on macOS CI,
capturing events like `DC_EVENT_ACCOUNTS_ITEM_CHANGED`
before FailPlugin is setup.
These CI runners likely get less resources
because there is a limited number of them,
and this triggers this race condition.

Race is fixed by setting up fail plugin
before starting to capture events.
2025-01-05 01:36:09 +00:00
link2xt
7e5cec66ba refactor: simplify self_sent condition 2025-01-05 00:36:10 +00:00
link2xt
a7eab13ad6 test: message with empty To: field should have a valid to_id 2025-01-05 00:36:10 +00:00
link2xt
d26a27484b fix: default to_id to self instead of 0
If message has empty `To` field
it is a self-sent message like a message
in Saved Messages chat or a sync message.
2025-01-05 00:36:10 +00:00
link2xt
ed2a3a76b4 test: messages without recipients are assigned to self chat
Previously such messages were assigned to trash.
2025-01-05 00:36:10 +00:00
link2xt
49f5523b67 fix: allow empty To field for self-sent messages
Currently Delta Chat puts self address in the To field
to avoid the To field being empty.
There is a plan to put empty `hidden-recipients`
group there, this fix prepares the receiver for such messages.
2025-01-05 00:36:10 +00:00
link2xt
548fadc84a fix: prioritize mailing list over self-sent messages
New Delta Chat is going to send self-sent messages
with undisclosed recipients instead of placing self into the `To` field.
To avoid assigning broadcast list messages to Saved Messages chat,
we should check the mailing list headers
before attempting to assign to Saved Messages.
2025-01-05 00:36:10 +00:00
iequidoo
2bce4466d7 fix: Prefer to encrypt if E2eeEnabled even if peers have EncryptPreference::NoPreference
First of all, chatmail servers normally forbid to send unencrypted mail, so if we know the peer's
key, we should encrypt to it. Chatmail setups have `E2eeEnabled=1` by default and this isn't
possible to change in UIs, so this change fixes the chatmail case. Additionally, for chatmail, if a
peer has `EncryptPreference::Reset`, let's handle it as `EncryptPreference::NoPreference` for the
reason above. Still, if `E2eeEnabled` is 0 for a chatmail setup somehow, e.g. the user set it via
environment, let's assume that the user knows what they do and ignore `IsChatmail` flag.

NB:
- If we don't know the peer's key, we should try to send an unencrypted message as before for a
  chatmail setup.
- This change doesn't remove the "majority rule", but now the majority with
  `EncryptPreference::NoPreference` can't disable encryption if the local preference is `Mutual`. To
  disable encryption, some peer should have a missing peerstate or, for the non-chatmail case, the
  majority should have `EncryptPreference::Reset`.
2025-01-04 20:16:38 -03:00
link2xt
f31e86d203 chore: lockfile update 2025-01-04 06:49:46 +00:00
link2xt
8ec098210e fix: update shadowsocks crate to 1.22.0 to avoid panic when parsing some QR codes
`aead-cipher` feature has become optional
and is disabled by default.
We enable it to avoid breaking compatibility.
2025-01-03 23:56:47 +00:00
dependabot[bot]
62e22286bb chore(cargo): bump testdir from 0.9.1 to 0.9.3
Bumps [testdir](https://github.com/flub/testdir) from 0.9.1 to 0.9.3.
- [Changelog](https://github.com/flub/testdir/blob/main/CHANGELOG.md)
- [Commits](https://github.com/flub/testdir/compare/v0.9.1...v0.9.3)

---
updated-dependencies:
- dependency-name: testdir
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-01-03 20:50:30 -03:00
iequidoo
c596bfc44e refactor: add_parts: Remove excessive is_mdn checks 2025-01-03 00:44:55 -03:00
dependabot[bot]
379b31835b Merge pull request #6395 from deltachat/dependabot/cargo/serde-1.0.217 2025-01-03 02:27:19 +00:00
dependabot[bot]
5a69d9c355 chore(cargo): bump serde from 1.0.215 to 1.0.217
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.215 to 1.0.217.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.215...v1.0.217)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-01-03 01:57:45 +00:00
dependabot[bot]
e689db4376 Merge pull request #6387 from deltachat/dependabot/cargo/serde_json-1.0.134 2025-01-03 01:56:37 +00:00
dependabot[bot]
2d173512af Merge pull request #6396 from deltachat/dependabot/cargo/rustls-0.23.20 2025-01-03 01:56:18 +00:00
dependabot[bot]
adddc8e4ad Merge pull request #6388 from deltachat/dependabot/cargo/tokio-1.42.0 2025-01-03 01:23:31 +00:00
dependabot[bot]
29ee1fc047 Merge pull request #6390 from deltachat/dependabot/cargo/anyhow-1.0.95 2025-01-03 00:37:17 +00:00
dependabot[bot]
8a27c3edf0 chore(cargo): bump rustls from 0.23.19 to 0.23.20
Bumps [rustls](https://github.com/rustls/rustls) from 0.23.19 to 0.23.20.
- [Release notes](https://github.com/rustls/rustls/releases)
- [Changelog](https://github.com/rustls/rustls/blob/main/CHANGELOG.md)
- [Commits](https://github.com/rustls/rustls/compare/v/0.23.19...v/0.23.20)

---
updated-dependencies:
- dependency-name: rustls
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-01-03 00:04:17 +00:00
dependabot[bot]
7164786165 chore(cargo): bump tokio from 1.41.1 to 1.42.0
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.41.1 to 1.42.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.41.1...tokio-1.42.0)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-01-02 23:50:08 +00:00
dependabot[bot]
0cfd84d803 chore(cargo): bump serde_json from 1.0.133 to 1.0.134
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.133 to 1.0.134.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.133...v1.0.134)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-01-02 23:47:43 +00:00
dependabot[bot]
d25cb22ae5 Merge pull request #6380 from deltachat/dependabot/cargo/tokio-util-0.7.13 2025-01-02 23:38:44 +00:00
dependabot[bot]
e236b55fbb Merge pull request #6382 from deltachat/dependabot/cargo/tokio-rustls-0.26.1 2025-01-02 23:38:25 +00:00
dependabot[bot]
1dfb2a36e6 chore(cargo): bump anyhow from 1.0.93 to 1.0.95
Bumps [anyhow](https://github.com/dtolnay/anyhow) from 1.0.93 to 1.0.95.
- [Release notes](https://github.com/dtolnay/anyhow/releases)
- [Commits](https://github.com/dtolnay/anyhow/compare/1.0.93...1.0.95)

---
updated-dependencies:
- dependency-name: anyhow
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-01-02 22:57:38 +00:00
dependabot[bot]
15b6ed1210 Merge pull request #6398 from deltachat/dependabot/cargo/env_logger-0.11.6 2025-01-02 22:56:31 +00:00
dependabot[bot]
51e7bcf6a6 Merge pull request #6381 from deltachat/dependabot/cargo/rustls-pki-types-1.10.1 2025-01-02 22:52:38 +00:00
dependabot[bot]
e80d6ce803 Merge pull request #6392 from deltachat/dependabot/cargo/quote-1.0.38 2025-01-02 22:51:27 +00:00
dependabot[bot]
de36c05f18 Merge pull request #6379 from deltachat/dependabot/cargo/fast-socks5-0.10.0 2025-01-02 22:49:58 +00:00
dependabot[bot]
8c24dbd493 chore(cargo): bump tokio-rustls from 0.26.0 to 0.26.1
Bumps [tokio-rustls](https://github.com/rustls/tokio-rustls) from 0.26.0 to 0.26.1.
- [Release notes](https://github.com/rustls/tokio-rustls/releases)
- [Commits](https://github.com/rustls/tokio-rustls/commits)

---
updated-dependencies:
- dependency-name: tokio-rustls
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-01-02 22:05:48 +00:00
dependabot[bot]
72312a3a43 chore(cargo): bump tokio-util from 0.7.12 to 0.7.13
Bumps [tokio-util](https://github.com/tokio-rs/tokio) from 0.7.12 to 0.7.13.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-util-0.7.12...tokio-util-0.7.13)

---
updated-dependencies:
- dependency-name: tokio-util
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-01-02 22:05:21 +00:00
dependabot[bot]
06e3f0a738 Merge pull request #6394 from deltachat/dependabot/cargo/tokio-stream-0.1.17 2025-01-02 21:31:25 +00:00
dependabot[bot]
7ef4621ffd chore(cargo): bump quote from 1.0.37 to 1.0.38
Bumps [quote](https://github.com/dtolnay/quote) from 1.0.37 to 1.0.38.
- [Release notes](https://github.com/dtolnay/quote/releases)
- [Commits](https://github.com/dtolnay/quote/compare/1.0.37...1.0.38)

---
updated-dependencies:
- dependency-name: quote
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-01-02 21:23:39 +00:00
dependabot[bot]
74d586ed93 Merge pull request #6386 from deltachat/dependabot/cargo/libc-0.2.169 2025-01-02 21:17:33 +00:00
dependabot[bot]
4de5867827 Merge pull request #6378 from deltachat/dependabot/cargo/chrono-0.4.39 2025-01-02 21:15:38 +00:00
dependabot[bot]
38836e8084 Merge pull request #6384 from deltachat/dependabot/cargo/hyper-1.5.2 2025-01-02 21:14:21 +00:00
dependabot[bot]
dde79fbf98 Merge pull request #6385 from deltachat/dependabot/cargo/async-broadcast-0.7.2 2025-01-02 21:09:07 +00:00
dependabot[bot]
65af309b16 Merge pull request #6389 from deltachat/dependabot/cargo/quick-xml-0.37.2 2025-01-02 21:08:07 +00:00
dependabot[bot]
502dd1157d Merge pull request #6393 from deltachat/dependabot/cargo/syn-2.0.94 2025-01-02 21:07:37 +00:00
dependabot[bot]
1000fe5dec Merge pull request #6397 from deltachat/dependabot/cargo/proptest-1.6.0 2025-01-02 20:16:49 +00:00
iequidoo
1792d48144 fix: Don't treat location-only and sync messages as bot ones (#6357) 2025-01-02 13:14:56 -03:00
dependabot[bot]
49c09df864 chore(cargo): bump env_logger from 0.11.5 to 0.11.6
Bumps [env_logger](https://github.com/rust-cli/env_logger) from 0.11.5 to 0.11.6.
- [Release notes](https://github.com/rust-cli/env_logger/releases)
- [Changelog](https://github.com/rust-cli/env_logger/blob/main/CHANGELOG.md)
- [Commits](https://github.com/rust-cli/env_logger/compare/v0.11.5...v0.11.6)

---
updated-dependencies:
- dependency-name: env_logger
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-01-01 21:55:24 +00:00
dependabot[bot]
3d698036f5 chore(cargo): bump proptest from 1.5.0 to 1.6.0
Bumps [proptest](https://github.com/proptest-rs/proptest) from 1.5.0 to 1.6.0.
- [Release notes](https://github.com/proptest-rs/proptest/releases)
- [Changelog](https://github.com/proptest-rs/proptest/blob/main/CHANGELOG.md)
- [Commits](https://github.com/proptest-rs/proptest/commits)

---
updated-dependencies:
- dependency-name: proptest
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-01-01 21:55:08 +00:00
dependabot[bot]
bf4e11c607 chore(cargo): bump tokio-stream from 0.1.16 to 0.1.17
Bumps [tokio-stream](https://github.com/tokio-rs/tokio) from 0.1.16 to 0.1.17.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-stream-0.1.16...tokio-stream-0.1.17)

---
updated-dependencies:
- dependency-name: tokio-stream
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-01-01 21:53:58 +00:00
dependabot[bot]
9e460a106b chore(cargo): bump syn from 2.0.90 to 2.0.94
Bumps [syn](https://github.com/dtolnay/syn) from 2.0.90 to 2.0.94.
- [Release notes](https://github.com/dtolnay/syn/releases)
- [Commits](https://github.com/dtolnay/syn/compare/2.0.90...2.0.94)

---
updated-dependencies:
- dependency-name: syn
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-01-01 21:53:34 +00:00
dependabot[bot]
2d166d602b chore(cargo): bump quick-xml from 0.37.1 to 0.37.2
Bumps [quick-xml](https://github.com/tafia/quick-xml) from 0.37.1 to 0.37.2.
- [Release notes](https://github.com/tafia/quick-xml/releases)
- [Changelog](https://github.com/tafia/quick-xml/blob/master/Changelog.md)
- [Commits](https://github.com/tafia/quick-xml/compare/v0.37.1...v0.37.2)

---
updated-dependencies:
- dependency-name: quick-xml
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-01-01 21:52:20 +00:00
dependabot[bot]
fc0e7fd61f chore(cargo): bump libc from 0.2.167 to 0.2.169
Bumps [libc](https://github.com/rust-lang/libc) from 0.2.167 to 0.2.169.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Changelog](https://github.com/rust-lang/libc/blob/0.2.169/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.167...0.2.169)

---
updated-dependencies:
- dependency-name: libc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-01-01 21:51:26 +00:00
dependabot[bot]
f9a7837e87 chore(cargo): bump async-broadcast from 0.7.1 to 0.7.2
Bumps [async-broadcast](https://github.com/smol-rs/async-broadcast) from 0.7.1 to 0.7.2.
- [Release notes](https://github.com/smol-rs/async-broadcast/releases)
- [Changelog](https://github.com/smol-rs/async-broadcast/blob/master/CHANGELOG.md)
- [Commits](https://github.com/smol-rs/async-broadcast/compare/0.7.1...0.7.2)

---
updated-dependencies:
- dependency-name: async-broadcast
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-01-01 21:51:10 +00:00
dependabot[bot]
6da9838978 chore(cargo): bump hyper from 1.5.1 to 1.5.2
Bumps [hyper](https://github.com/hyperium/hyper) from 1.5.1 to 1.5.2.
- [Release notes](https://github.com/hyperium/hyper/releases)
- [Changelog](https://github.com/hyperium/hyper/blob/master/CHANGELOG.md)
- [Commits](https://github.com/hyperium/hyper/compare/v1.5.1...v1.5.2)

---
updated-dependencies:
- dependency-name: hyper
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-01-01 21:50:39 +00:00
dependabot[bot]
e45df09966 chore(cargo): bump rustls-pki-types from 1.10.0 to 1.10.1
Bumps [rustls-pki-types](https://github.com/rustls/pki-types) from 1.10.0 to 1.10.1.
- [Release notes](https://github.com/rustls/pki-types/releases)
- [Commits](https://github.com/rustls/pki-types/compare/v/1.10.0...v/1.10.1)

---
updated-dependencies:
- dependency-name: rustls-pki-types
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-01-01 21:49:52 +00:00
dependabot[bot]
56d9036d27 chore(cargo): bump fast-socks5 from 0.9.6 to 0.10.0
Bumps [fast-socks5](https://github.com/dizda/fast-socks5) from 0.9.6 to 0.10.0.
- [Release notes](https://github.com/dizda/fast-socks5/releases)
- [Commits](https://github.com/dizda/fast-socks5/compare/v0.9.6...v0.10.0)

---
updated-dependencies:
- dependency-name: fast-socks5
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-01-01 21:49:21 +00:00
dependabot[bot]
c77a09b189 chore(cargo): bump chrono from 0.4.38 to 0.4.39
Bumps [chrono](https://github.com/chronotope/chrono) from 0.4.38 to 0.4.39.
- [Release notes](https://github.com/chronotope/chrono/releases)
- [Changelog](https://github.com/chronotope/chrono/blob/main/CHANGELOG.md)
- [Commits](https://github.com/chronotope/chrono/compare/v0.4.38...v0.4.39)

---
updated-dependencies:
- dependency-name: chrono
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-01-01 21:49:05 +00:00
link2xt
25933b10c8 fix: mark holiday notice messages as bot-generated 2025-01-01 20:58:41 +00:00
link2xt
1089aea8e0 refactor: add emit_msgs_changed_without_msg_id
Added debug assertions to make sure
MsgsChanged is not emitted with 0 IDs by accident.
2025-01-01 20:50:52 +00:00
link2xt
779635d73b refactor: deprecate Param::ErroneousE2ee 2024-12-29 06:51:32 +00:00
iequidoo
21664125d7 fix: Change BccSelf default to 0 for chatmail (#6340)
Change `BccSelf` default to 0 for chatmail configurations and enable it upon a backup export. As for
`DeleteServerAfter` who was set to 0 upon a backup export before, make its default dependent on
`BccSelf` for chatmail. We don't need `BccSelf` for chatmail by default because we assume
single-device use. Also `BccSelf` is needed for "classic" email accounts even if `DeleteServerAfter`
is set to "immediately" to detect that a message was sent if SMTP server is slow to respond and
connection is lost before receiving the status line which isn't a problem for chatmail servers.
2024-12-27 22:54:36 -03:00
iequidoo
ed9c01f1f1 fix: Never change Viewtype::Sticker to Image if file has non-image extension (#6352)
Even if UIs don't call `Message::force_sticker()`, they don't want conversions of `Sticker` to
`Image` if it's obviously not an image, particularly, has non-image extension. Also UIs don't want
conversions of `Sticker` to anything other than `Image`, so let's keep the `Sticker` viewtype in
this case.
2024-12-27 22:49:42 -03:00
iequidoo
7d7a2453a9 docs: That Viewtype::Sticker may be changed to Image and how to disable that (#6352) 2024-12-27 22:49:42 -03:00
Hocuri
0cadfe34ae refactor: Remove unused parameter and return value from build_body_file(…) (#6369)
2 simple refactoring commits that remove some unused code.
2024-12-27 17:35:08 +01:00
iequidoo
137e32fe49 fix(rpc-client): Add INCOMING_REACTION to const.EventType (#6349) 2024-12-26 14:28:42 -03:00
WofWca
f8bf5a3557 feat: add IncomingWebxdcNotify.chat_id (#6356) 2024-12-25 17:49:27 +00:00
iequidoo
f61d5af468 feat: Delete vg-request-with-auth from IMAP after processing (#6208)
In multi-device case `vg-request-with-auth` left on IMAP may result in situation when Bob joins the
group, then leaves it, then second Alice device comes online and processes `vg-request-with-auth`
again and adds Bob back. So we should IMAP-delete `vg-request-with-auth`. Another device will know
the Bob's key from Autocrypt-Gossip. It's not a problem if Alice loses state (restores from an old
backup) or goes offline for long before sending `vg-member-added`, anyway it may not be delivered by
the server, rather Bob should retry sending SecureJoin messages as he is a part which wants to join,
so let's not solve this for now.
2024-12-25 14:47:17 -03:00
iequidoo
3d9aee1368 feat: Remove "jobs" from imap_markseen if folder doesn't exist (#5870)
Add a `create` param to `select_with_uidvalidity()` instead of always trying to create the folder
and return `Ok(false)` from it if the folder doesn't exist and shouldn't be created, and handle this
in `store_seen_flags_on_imap()` by just removing "jobs" from the `imap_markseen` table. Also don't
create the folder in other code paths where it's not necessary.
2024-12-24 23:37:14 -03:00
link2xt
f1302c3bc4 chore(release): prepare for 1.152.2 2024-12-24 19:20:23 +00:00
link2xt
0cc80268d2 fix: start ephemeral timer when chat is archived 2024-12-24 18:04:39 +00:00
iequidoo
64a1b8e57c fix: sanitise_name: Don't consider punctuation and control chars as part of file extension (#6362) 2024-12-24 13:38:24 -03:00
iequidoo
5772284e82 feat: Revalidate HTTP cache entries once per minute maximum
This is to avoid revalidating HTTP cache too frequently (and have many parallel revalidation tasks)
if revalidation fails or the HTTP request takes some time. The stale period >= 1 hour, so 1 more
minute won't be a problem.
2024-12-24 13:36:54 -03:00
link2xt
beb6a21ecd feat: start ephemeral timers when the chat is noticed 2024-12-24 16:05:41 +00:00
iequidoo
22bc7567d3 refactor: Remove marknoticed_chat_if_older_than()
It was called from `receive_imf` when an outgoing message is received. But
`Imap::fetch_new_messages()` already calls `chat::mark_old_messages_as_noticed()` which does the job
better (per-message).
2024-12-24 13:03:41 -03:00
iequidoo
a910808b4e feat: delete_msgs: Use transaction() instead of call_write()
Explicit transaction does the only commit (and fsync()).
2024-12-23 22:02:54 -03:00
link2xt
3d5e442145 fix: reduce number of repeat_vars() calls
SQL statements fail if the number of variables
exceeds `SQLITE_LIMIT_VARIABLE_NUMBER`.

Remaining repeat_vars() calls are difficult to replace
and use arrays passed from the UI,
e.g. forwarded message IDs or marked as seen IDs.
2024-12-22 20:23:16 +00:00
iequidoo
3af4ea1d00 feat: Emit ImexProgress(1) after receiving backup size
UIs may want to display smth like "Transferring..." after "Establishing connection between
devices..." on nonzero progress. Before, progress on the receiver side was starting with 2 after
receiving enough data.
2024-12-17 21:12:09 -03:00
link2xt
a9e38aa8fc Merge tag 'v1.152.1'
Release 1.152.1
2024-12-17 19:29:00 +00:00
link2xt
9e408c3abd chore(release): prepare for 1.152.1 2024-12-17 19:28:09 +00:00
link2xt
67e16d0222 Merge <https://github.com/deltachat/deltachat-core-rust/pull/6346>
Downgrade Rust used for release builds.
2024-12-17 19:20:40 +00:00
link2xt
5069b585c8 bulid(nix): use new fenix for dev shell 2024-12-17 18:27:38 +00:00
link2xt
6cd6aca7b8 Revert "chore(cargo): bump rustyline from 14.0.0 to 15.0.0"
This reverts commit b74ff278ce.
2024-12-17 17:21:20 +00:00
link2xt
d822da3c9f build: downgrade Rust version used to build binaries
This fixes the problem of VirusTotal
reporting the binaries built with
`nix build .#deltachat-rpc-server-win64`
as malware.
2024-12-17 17:20:48 +00:00
link2xt
9d331483e9 Revert "build: increase MSRV to 1.81.0"
This reverts commit ffe6efe819.
2024-12-17 17:20:48 +00:00
link2xt
1e1e5793dd chore: remove contrib/ directory
It only contained proxy
that some users ran in Termux
to look at IMAP traffic.

The same can be achieved with `socat`, e.g.:
  socat -v TCP-LISTEN:9999,bind=127.0.0.1 OPENSSL:nine.testrun.org:993
2024-12-15 17:00:25 +00:00
dependabot[bot]
b74ff278ce chore(cargo): bump rustyline from 14.0.0 to 15.0.0
Bumps [rustyline](https://github.com/kkawakam/rustyline) from 14.0.0 to 15.0.0.
- [Release notes](https://github.com/kkawakam/rustyline/releases)
- [Changelog](https://github.com/kkawakam/rustyline/blob/master/History.md)
- [Commits](https://github.com/kkawakam/rustyline/compare/v14.0.0...v15.0.0)

---
updated-dependencies:
- dependency-name: rustyline
  dependency-type: direct:production
  update-type: version-update:semver-major
...
2024-12-12 14:25:54 -03:00
link2xt
a305409627 chore(release): prepare for 1.152.0 2024-12-12 15:39:31 +00:00
link2xt
7d1e3c4812 fix: ignore garbage at the end of the keys 2024-12-12 15:21:58 +00:00
link2xt
2f976d8050 feat: implement stale-while-revalidate for HTTP cache 2024-12-12 14:30:45 +00:00
iequidoo
cb2157822a fix: Render "message" parts in multipart messages' HTML (#4462)
This fixes the HTML display of messages containing forwarded messages. Before, forwarded messages
weren't rendered in HTML and if a forwarded message is long and therefore truncated in the chat, it
could only be seen in the "Message Info". In #4462 it was suggested to display "Show Full
Message..." for each truncated message part and save to `msgs.mime_headers` only the corresponding
part, but this is a quite huge change and refactoring and also it may be good that currently we save
the full message structure to `msgs.mime_headers`, so i'd suggest not to change this for now.
2024-12-12 11:30:02 -03:00
iequidoo
253362899b feat: Set mime_modified for the last message part, not the first (#4462)
Otherwise the "Show Full Message..." button appears somewhere in the middle of the multipart
message, e.g. after a text in the first message bubble, but before a text in the second
bubble. Moreover, if the second/n-th bubble's text is shortened (ends with "[...]"), the user should
scroll up to click on "Show Full Message..." which doesn't look reasonable. Scrolling down looks
more acceptable (e.g. if the first bubble's text is shortened in a multipart message).

I'd even suggest to show somehow that message bubbles belong to the same multipart message, e.g. add
"[↵]" to the text of all bubbles except the last one, but let's discuss this first.
2024-12-12 11:30:02 -03:00
iequidoo
bb3075c6fd test: Record the current wrong behaviour of HTML display of multipart messages (#4462) 2024-12-12 11:30:02 -03:00
link2xt
ffe6efe819 build: increase MSRV to 1.81.0 2024-12-12 04:45:24 +00:00
link2xt
cc672b81fa fix: renew HTTP cache entry if it already exists 2024-12-11 23:39:10 +00:00
link2xt
698136b30c test: test that HTTP cache can be renewed without housekeeping 2024-12-11 23:39:10 +00:00
link2xt
33169dd49a test: actually insert pixel app into HTTP cache 2024-12-11 23:39:10 +00:00
link2xt
ee20887782 feat: cache HTTP GET requests 2024-12-11 19:34:29 +00:00
link2xt
72558af98c api!: remove dc_prepare_msg and dc_msg_is_increation 2024-12-11 19:34:29 +00:00
B. Petersen
bc3b6ae309 feat: prefix server-url in info
without the prefix,
it looks as if it is part of the Message-ID,
esp. if Message-ID is longer,
a break on different delimiters may look exactly the same.

see #6329 for some screenshots that initially confused me :)
2024-12-11 12:56:48 +01:00
link2xt
b650b96ccd chore(release): prepare for 1.151.6 2024-12-11 09:30:42 +00:00
link2xt
a373dd4e99 fix: do not subscribe to heartbeat if already subscribed via metadata 2024-12-10 12:42:53 +00:00
link2xt
7368764210 docs: move rPGP to the security section of changelog 2024-12-10 11:20:00 +00:00
dependabot[bot]
2b9722675e Merge pull request #6316 from deltachat/dependabot/cargo/fuzz/quinn-proto-0.11.9 2024-12-09 20:48:58 +00:00
dependabot[bot]
590f913310 chore(deps): bump quinn-proto from 0.11.3 to 0.11.9 in /fuzz
Bumps [quinn-proto](https://github.com/quinn-rs/quinn) from 0.11.3 to 0.11.9.
- [Release notes](https://github.com/quinn-rs/quinn/releases)
- [Commits](https://github.com/quinn-rs/quinn/compare/quinn-proto-0.11.3...quinn-proto-0.11.9)

---
updated-dependencies:
- dependency-name: quinn-proto
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-12-09 17:33:45 +00:00
link2xt
9d77f65f0e docs: update links to Node.js bindings in the README
CFFI and napi.rs bindings are not maintained,
JSON-RPC client should be used with deltachat-rpc-server instead.
2024-12-09 17:32:49 +00:00
dependabot[bot]
a13343f210 Merge pull request #6317 from deltachat/dependabot/cargo/fuzz/pgp-0.14.2 2024-12-09 15:48:49 +00:00
iequidoo
c2cbc3fe33 feat: Add info messages about implicit membership changes if group member list is recreated (#6314) 2024-12-09 12:04:26 -03:00
iequidoo
cd76f4b685 fix: Add self-addition message to chat when recreating member list
A user reported to me that after they left a group, they were implicitly readded, but there's no any
readdition message, so currently it looks in the chat like leaving it has no effect, just new
messages continue to arrive. The readdition probably happened because some member didn't receive the
user's self-removal message, anyway, at least there must be a message that the user is readded, even
if it isn't known by whom.
2024-12-09 12:04:26 -03:00
iequidoo
0501917e98 feat: Don't add "Failed to send message to ..." info messages to group chats
A NDN may arrive days after the message is sent when it's already impossible to tell which message
wasn't delivered looking at the "Failed to send" info message, so it only clutters the chat and
makes the user think they tried to send some message recently which isn't true. Moreover, the info
message duplicates the info already displayed in the error message behind the exclamation mark and
info messages do not point to the message that is failed to be sent.

Moreover it works rarely because `mimeparser.rs` only parses recipients from `x-failed-recipients`,
so it likely only works for Gmail. Postfix does not add this `X-Failed-Recipients` header. Let's
remove this parsing too. Thanks to @link2xt for pointing this out.
2024-12-09 11:01:41 -03:00
link2xt
abe81d0b84 build: add idna 0.5.0 exception into deny.toml 2024-12-09 13:33:40 +00:00
Hocuri
39be59172d test: Notifiy more prominently & in more tests about false positives when running cargo test (#6308)
This PR:
- Moves the note about the false positive to the end of the test output,
where it is more likely to be noticed
- Also notes in test_modify_chat_disordered() and
test_setup_contact_*(), in addition to the existing note in
test_was_seen_recently()
2024-12-06 15:07:57 +01:00
link2xt
f03dc6af12 refactor: factor out wait_for_all_work_done() 2024-12-06 01:22:03 +00:00
dependabot[bot]
3cb44b34e9 chore(deps): bump pgp from 0.14.0 to 0.14.2 in /fuzz
Bumps [pgp](https://github.com/rpgp/rpgp) from 0.14.0 to 0.14.2.
- [Release notes](https://github.com/rpgp/rpgp/releases)
- [Changelog](https://github.com/rpgp/rpgp/blob/main/CHANGELOG.md)
- [Commits](https://github.com/rpgp/rpgp/compare/v0.14.0...v0.14.2)

---
updated-dependencies:
- dependency-name: pgp
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-12-05 17:31:50 +00:00
link2xt
77cf536b94 chore(release): prepare for 1.151.5 2024-12-05 12:35:46 +00:00
link2xt
462dffe9ce docs: remove mention of non-existent nightly feature 2024-12-05 12:25:43 +00:00
link2xt
d89327dfc5 docs: document push module 2024-12-05 12:23:19 +00:00
link2xt
ff734ee24d chore(cargo): update rPGP to 0.14.2 2024-12-05 12:22:04 +00:00
iequidoo
8c9efc68b6 fix: Store plaintext in mime_headers of truncated sent messages (#6273)
This fixes HTML display of truncated (long) sent messages ("Show full message" in UIs). Before,
incorrect HTML was stored (with missing line breaks etc.) for them. Now stored plaintext is
formatted to HTML upon calling `MsgId::get_html()` and this results in the same HTML as on a
receiver side.
2024-12-04 23:15:05 -03:00
link2xt
e694411974 api!: remove dc_all_work_done()
Also cleaned up test_connectivity()
which tested that state does not flicker to WORKING
when there are no messages to be fetched.
The state is expected to flicker to WORKING
when checking for new messages,
so the tests were outdated since
change 3b0b2379b8
2024-12-04 14:31:55 +00:00
Hocuri
6468806d86 test: Fix panic in receive_emails benchmark (#6306)
The benchmark function (e.g. `recv_all_emails()`) is executed multiple
times on the same context. During the second iteration, all the emails
were already in the database, so, receiving them again failed.

This PR fixes that by passing in a second `iteration` counter that is
different for every invocation of the benchmark function.
2024-12-03 16:31:25 +01:00
link2xt
825455d9dc chore(release): prepare for 1.151.4 2024-12-03 14:45:31 +00:00
link2xt
6dd8f44a15 feat: encrypt notification tokens 2024-12-03 14:40:53 +00:00
link2xt
e14349ea0e chore: update lockfile so --locked build is possible again 2024-12-03 13:51:29 +00:00
link2xt
645e316faa chore(cargo): update async-smtp to 0.10.0 2024-12-03 07:05:03 +00:00
dependabot[bot]
26c46a0095 Merge pull request #6293 from deltachat/dependabot/cargo/url-2.5.4 2024-12-03 01:06:27 +00:00
link2xt
2ae98f963e chore: fixup deny.toml 2024-12-03 00:36:21 +00:00
link2xt
3b0b2379b8 fix: replace connectivity state "Connected" with "Preparing"
This better reflects that this state means
we just connected and there may me work to do.
This state is converted to DC_CONNECTIVITY_WORKING
instead of DC_CONNECTIVITY_CONNECTED state now.

Before this change when IMAP connected
to the server, it switched
from DC_CONNECTIVITY_NOT_CONNECTED
to DC_CONNECTIVITY_CONNECTING,
then to DC_CONNECTIVITY_CONNECTED (actually preparing)
then to DC_CONNECTIVITY_WORKING
and then to DC_CONNECTIVITY_CONNECTED again (actually idle).

On fast connections this resulted in flickering "Connected"
string in the status bar right before "Updating..."
and on slow connections this "Connected" state
before "Updating..." lasted for a while
leaving the user to wonder if there are no new messages
or if Delta Chat will still switch to "Updating..."
before going into "Connected" state again.
2024-12-03 00:35:38 +00:00
Hocuri
256b34dadc test: fix cargo check for receive_emails benchmark 2024-12-02 22:13:10 +01:00
Hocuri
ee0ac6389b ci: Also run cargo check without all-features 2024-12-02 22:13:10 +01:00
link2xt
191eb7efdd chore: fix typos
Applied fixes suggested by scripts/codespell.sh
2024-12-02 19:22:45 +00:00
Hocuri
587ea02ffa chore: Beta clippy suggestions (#6271)
Already apply rust beta (1.84) clippy suggestions now, before they let
CI fail in 6 weeks.

The newly used functions are available since 1.70, our MSRV is 1.77, so
we can use them.
2024-12-02 18:57:01 +00:00
dependabot[bot]
06a7c63f2d chore(cargo): bump libc from 0.2.161 to 0.2.167
Bumps [libc](https://github.com/rust-lang/libc) from 0.2.161 to 0.2.167.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Changelog](https://github.com/rust-lang/libc/blob/0.2.167/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.161...0.2.167)

---
updated-dependencies:
- dependency-name: libc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-12-02 18:22:39 +00:00
dependabot[bot]
485a765b3e chore(cargo): bump syn from 2.0.86 to 2.0.90
Bumps [syn](https://github.com/dtolnay/syn) from 2.0.86 to 2.0.90.
- [Release notes](https://github.com/dtolnay/syn/releases)
- [Commits](https://github.com/dtolnay/syn/compare/2.0.86...2.0.90)

---
updated-dependencies:
- dependency-name: syn
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-12-02 18:20:15 +00:00
dependabot[bot]
a224067c6e chore(cargo): bump serde_json from 1.0.132 to 1.0.133
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.132 to 1.0.133.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.132...v1.0.133)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-12-02 18:19:57 +00:00
dependabot[bot]
009dd89af4 chore(cargo): bump serde from 1.0.210 to 1.0.215
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.210 to 1.0.215.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.210...v1.0.215)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-12-02 18:19:33 +00:00
dependabot[bot]
16a3acbc5d chore(cargo): bump hyper from 1.5.0 to 1.5.1
Bumps [hyper](https://github.com/hyperium/hyper) from 1.5.0 to 1.5.1.
- [Release notes](https://github.com/hyperium/hyper/releases)
- [Changelog](https://github.com/hyperium/hyper/blob/master/CHANGELOG.md)
- [Commits](https://github.com/hyperium/hyper/compare/v1.5.0...v1.5.1)

---
updated-dependencies:
- dependency-name: hyper
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-12-02 18:19:03 +00:00
link2xt
ddfcd2ed2e chore(release): prepare for 1.151.3 2024-12-02 17:09:45 +00:00
dependabot[bot]
b779fc7028 Merge pull request #6299 from deltachat/dependabot/cargo/tokio-1.41.1 2024-12-02 16:59:36 +00:00
B. Petersen
6099222f0c docs: improve CFFI docs, link to corresponding JSON-RPC docs 2024-12-02 14:35:25 +01:00
dependabot[bot]
150b50fa96 chore(cargo): bump tokio from 1.41.0 to 1.41.1
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.41.0 to 1.41.1.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.41.0...tokio-1.41.1)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-12-02 01:21:00 +00:00
dependabot[bot]
8cc540098d chore(cargo): bump url from 2.5.2 to 2.5.4
Bumps [url](https://github.com/servo/rust-url) from 2.5.2 to 2.5.4.
- [Release notes](https://github.com/servo/rust-url/releases)
- [Commits](https://github.com/servo/rust-url/compare/v2.5.2...v2.5.4)

---
updated-dependencies:
- dependency-name: url
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-12-02 00:51:24 +00:00
117 changed files with 10785 additions and 7726 deletions

View File

@@ -24,7 +24,7 @@ jobs:
name: Lint Rust
runs-on: ubuntu-latest
env:
RUSTUP_TOOLCHAIN: 1.83.0
RUSTUP_TOOLCHAIN: 1.84.0
steps:
- uses: actions/checkout@v4
with:
@@ -37,8 +37,10 @@ jobs:
run: cargo fmt --all -- --check
- name: Run clippy
run: scripts/clippy.sh
- name: Check
- name: Check with all features
run: cargo check --workspace --all-targets --all-features
- name: Check with only default features
run: cargo check --all-targets
npm_constants:
name: Check if node constants are up to date
@@ -95,15 +97,15 @@ jobs:
matrix:
include:
- os: ubuntu-latest
rust: 1.83.0
rust: 1.84.0
- os: windows-latest
rust: 1.83.0
rust: 1.84.0
- os: macos-latest
rust: 1.83.0
rust: 1.84.0
# Minimum Supported Rust Version = 1.77.0
# Minimum Supported Rust Version = 1.81.0
- os: ubuntu-latest
rust: 1.77.0
rust: 1.81.0
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v4
@@ -150,7 +152,7 @@ jobs:
uses: swatinem/rust-cache@v2
- name: Build C library
run: cargo build -p deltachat_ffi --features jsonrpc
run: cargo build -p deltachat_ffi
- name: Upload C library
uses: actions/upload-artifact@v4
@@ -221,11 +223,11 @@ jobs:
- os: macos-latest
python: pypy3.10
# Minimum Supported Python Version = 3.7
# Minimum Supported Python Version = 3.8
# This is the minimum version for which manylinux Python wheels are
# built. Test it with minimum supported Rust version.
- os: ubuntu-latest
python: 3.7
python: 3.8
runs-on: ${{ matrix.os }}
steps:
@@ -275,9 +277,9 @@ jobs:
- os: macos-latest
python: pypy3.10
# Minimum Supported Python Version = 3.7
# Minimum Supported Python Version = 3.8
- os: ubuntu-latest
python: 3.7
python: 3.8
runs-on: ${{ matrix.os }}
steps:

View File

@@ -1,5 +1,305 @@
# Changelog
## [1.155.0] - 2025-01-23
### API-Changes
- Add JSON-RPC API to get past members.
### Build system
- Update Rust.
- Increase MSRV to 1.81.0
### Features / Changes
- feat: Set BccSelf to true when receiving a sync message ([#6434](https://github.com/deltachat/deltachat-core-rust/pull/6434))
- File deduplication ([#6332](https://github.com/deltachat/deltachat-core-rust/pull/6332))
### Refactor
- Move tests to their own files.
- Extract `group_changes_msgs()` function ([#6460](https://github.com/deltachat/deltachat-core-rust/pull/6460)).
## [1.154.3] - 2025-01-20
### Build system
- Remove encoded-words from flake.nix.
- nix: Update rust-email hash in flake.nix.
### Miscellaneous Tasks
- Remove unused function delete_files_in_dir() ([#6454](https://github.com/deltachat/deltachat-core-rust/pull/6454)).
## [1.154.2] - 2025-01-20
### Features / Changes
- Add API to save messages ([#5606](https://github.com/deltachat/deltachat-core-rust/pull/5606)).
### Fixes
- fix: Don't accidentally remove Self from groups ([#6455](https://github.com/deltachat/deltachat-core-rust/pull/6455)).
- Do not create tombstones for members removed from unpromoted groups.
### Build system
- Switch to non-git version of encoded-words.
### Refactor
- Make memberlist update logic easier to follow.
## [1.154.1] - 2025-01-15
### Tests
- Expect trashing of no-op "member added" in non_member_cannot_modify_member_list.
## [1.154.0] - 2025-01-15
### Features / Changes
- New group consistency algorithm.
### Fixes
- Migration: Set bcc_self=1 if it's unset and delete_server_after!=1 ([#6432](https://github.com/deltachat/deltachat-core-rust/pull/6432)).
- Clear the config cache after every migration ([#6438](https://github.com/deltachat/deltachat-core-rust/pull/6438)).
### Build system
- Increase minimum supported Python version to 3.8.
- [**breaking**] Remove jsonrpc feature flag.
### CI
- Update Rust to 1.84.0.
### Miscellaneous Tasks
- Beta Clippy suggestions ([#6422](https://github.com/deltachat/deltachat-core-rust/pull/6422)).
### Refactor
- Use let..else.
- Add why_cant_send_ex() capable to only ignore specified conditions.
- Remove unnecessary is_contact_in_chat check.
- Eliminate remaining repeat_vars() calls ([#6359](https://github.com/deltachat/deltachat-core-rust/pull/6359)).
### Tests
- Use assert_eq! to compare chatlist length.
## [1.153.0] - 2025-01-05
### Features / Changes
- Remove "jobs" from imap_markseen if folder doesn't exist ([#5870](https://github.com/deltachat/deltachat-core-rust/pull/5870)).
- Delete `vg-request-with-auth` from IMAP after processing ([#6208](https://github.com/deltachat/deltachat-core-rust/pull/6208)).
### API-Changes
- Add `IncomingWebxdcNotify.chat_id` ([#6356](https://github.com/deltachat/deltachat-core-rust/pull/6356)).
- rpc-client: Add INCOMING_REACTION to const.EventType ([#6349](https://github.com/deltachat/deltachat-core-rust/pull/6349)).
### Documentation
- Viewtype::Sticker may be changed to Image and how to disable that ([#6352](https://github.com/deltachat/deltachat-core-rust/pull/6352)).
### Fixes
- Never change Viewtype::Sticker to Image if file has non-image extension ([#6352](https://github.com/deltachat/deltachat-core-rust/pull/6352)).
- Change BccSelf default to 0 for chatmail ([#6340](https://github.com/deltachat/deltachat-core-rust/pull/6340)).
- Mark holiday notice messages as bot-generated.
- Don't treat location-only and sync messages as bot ones ([#6357](https://github.com/deltachat/deltachat-core-rust/pull/6357)).
- Update shadowsocks crate to 1.22.0 to avoid panic when parsing some QR codes.
- Prefer to encrypt if E2eeEnabled even if peers have EncryptPreference::NoPreference.
- Prioritize mailing list over self-sent messages.
- Allow empty `To` field for self-sent messages.
- Default `to_id` to self instead of 0.
### Refactor
- Remove unused parameter and return value from `build_body_file(…)` ([#6369](https://github.com/deltachat/deltachat-core-rust/pull/6369)).
- Deprecate Param::ErroneousE2ee.
- Add `emit_msgs_changed_without_msg_id`.
- Add_parts: Remove excessive `is_mdn` checks.
- Simplify `self_sent` condition.
- Don't ignore get_for_contact errors.
### Tests
- Messages without recipients are assigned to self chat.
- Message with empty To: field should have a valid to_id.
- Fix `test_logged_ac_process_ffi_failure` flakiness.
## [1.152.2] - 2024-12-24
### Features / Changes
- Emit ImexProgress(1) after receiving backup size.
- `delete_msgs`: Use `transaction()` instead of `call_write()`.
- Start ephemeral timers when the chat is noticed.
- Start ephemeral timers when the chat is archived.
- Revalidate HTTP cache entries once per minute maximum.
### Fixes
- Reduce number of `repeat_vars()` calls.
- `sanitise_name`: Don't consider punctuation and control chars as part of file extension ([#6362](https://github.com/deltachat/deltachat-core-rust/pull/6362)).
### Refactor
- Remove marknoticed_chat_if_older_than().
### Miscellaneous Tasks
- Remove contrib/ directory.
## [1.152.1] - 2024-12-17
### Build system
- Downgrade Rust version used to build binaries.
- Reduce MSRV to 1.77.0.
## [1.152.0] - 2024-12-12
### API-Changes
- [**breaking**] Remove `dc_prepare_msg` and `dc_msg_is_increation`.
### Build system
- Increase MSRV to 1.81.0.
### Features / Changes
- Cache HTTP GET requests.
- Prefix server-url in info.
- Set `mime_modified` for the last message part, not the first ([#4462](https://github.com/deltachat/deltachat-core-rust/pull/4462)).
### Fixes
- Render "message" parts in multipart messages' HTML ([#4462](https://github.com/deltachat/deltachat-core-rust/pull/4462)).
- Ignore garbage at the end of the keys.
## [1.151.6] - 2024-12-11
### Features / Changes
- Don't add "Failed to send message to ..." info messages to group chats.
- Add info messages about implicit membership changes if group member list is recreated ([#6314](https://github.com/deltachat/deltachat-core-rust/pull/6314)).
### Fixes
- Add self-addition message to chat when recreating member list.
- Do not subscribe to heartbeat if already subscribed via metadata.
### Build system
- Add idna 0.5.0 exception into deny.toml.
### Documentation
- Update links to Node.js bindings in the README.
### Refactor
- Factor out `wait_for_all_work_done()`.
### Tests
- Notifiy more prominently & in more tests about false positives when running `cargo test` ([#6308](https://github.com/deltachat/deltachat-core-rust/pull/6308)).
## [1.151.5] - 2024-12-05
### API-Changes
- [**breaking**] Remove dc_all_work_done().
### Security
- cargo: Update rPGP to 0.14.2.
This fixes [Panics on Malformed Untrusted Input](https://github.com/rpgp/rpgp/security/advisories/GHSA-9rmp-2568-59rv)
and [Potential Resource Exhaustion when handling Untrusted Messages](https://github.com/rpgp/rpgp/security/advisories/GHSA-4grw-m28r-q285).
This allows the attacker to crash the application via specially crafted messages and keys.
We recommend all users and bot operators to upgrade to the latest version.
There is no impact on the confidentiality of the messages and keys so no action other than upgrading is needed.
### Fixes
- Store plaintext in mime_headers of truncated sent messages ([#6273](https://github.com/deltachat/deltachat-core-rust/pull/6273)).
### Documentation
- Document `push` module.
- Remove mention of non-existent `nightly` feature.
### Tests
- Fix panic in `receive_emails` benchmark ([#6306](https://github.com/deltachat/deltachat-core-rust/pull/6306)).
## [1.151.4] - 2024-12-03
### Features / Changes
- Encrypt notification tokens.
### Fixes
- Replace connectivity state "Connected" with "Preparing".
### Miscellaneous Tasks
- Beta clippy suggestions ([#6271](https://github.com/deltachat/deltachat-core-rust/pull/6271)).
### Tests
- Fix `cargo check` for `receive_emails` benchmark.
### CI
- Also run cargo check without all-features.
## [1.151.3] - 2024-12-02
### API-Changes
- Remove experimental `request_internet_access` option from webxdc's `manifest.toml`.
- Add getWebxdcHref to json api ([#6281](https://github.com/deltachat/deltachat-core-rust/pull/6281)).
### CI
- Update Rust to 1.83.0.
### Documentation
- Update dc_msg_get_info_type() and dc_get_securejoin_qr() ([#6269](https://github.com/deltachat/deltachat-core-rust/pull/6269)).
- Fix references to iroh-related headers in peer_channels docs.
- Improve CFFI docs, link to corresponding JSON-RPC docs.
### Features / Changes
- Allow the user to replace maps integration ([#5678](https://github.com/deltachat/deltachat-core-rust/pull/5678)).
- Mark saved messages chat as protected.
### Fixes
- Close iroh endpoint when I/O is stopped.
- Do not add protection messages to Saved Messages chat.
- Mark Saved Messages chat as protected if it exists.
- Sync chat action even if sync message arrives before first one from contact ([#6259](https://github.com/deltachat/deltachat-core-rust/pull/6259)).
### Refactor
- Remove some .unwrap() calls.
- Create_status_update_record: Remove double check of info_msg_id.
- Use Option::or_else() to dedup emitting IncomingWebxdcNotify.
## [1.151.2] - 2024-11-26
### API-Changes
@@ -372,7 +672,7 @@ This reverts commit 6f22ce2722b51773d7fbb0d89e4764f963cafd91..
- Reset quota on configured address change ([#5908](https://github.com/deltachat/deltachat-core-rust/pull/5908)).
- Do not emit progress 1000 when configuration is cancelled.
- Assume file extensions are 32 chars max and don't contain whitespace ([#5338](https://github.com/deltachat/deltachat-core-rust/pull/5338)).
- Readd tokens.foreign_id column ([#6038](https://github.com/deltachat/deltachat-core-rust/pull/6038)).
- Re-add tokens.foreign_id column ([#6038](https://github.com/deltachat/deltachat-core-rust/pull/6038)).
### Miscellaneous Tasks
@@ -963,7 +1263,7 @@ This reverts commit 6f22ce2722b51773d7fbb0d89e4764f963cafd91..
### Tests
- deltachat-rpc-client: reenable `log_cli`.
- deltachat-rpc-client: re-enable `log_cli`.
## [1.140.0] - 2024-06-04
@@ -1900,7 +2200,7 @@ This reverts commit 6f22ce2722b51773d7fbb0d89e4764f963cafd91..
- Mark 1:1 chat as verified for Bob early. 1:1 chat with Alice is verified as soon as Alice's key is verified rather than at the end of the protocol.
- Put Message-ID into hidden headers and take it from there on receiver ([#4798](https://github.com/deltachat/deltachat-core-rust/pull/4798)). This works around servers which generate their own Message-ID and overwrite the one generated by Delta Chat.
- deltachat-repl: Enable INFO logging by default and add timestamps.
- Add `ConfigSynced` (`DC_EVENT_CONFIG_SYNCED`) event which is emitted when configuration is changed via synchronization message or synchronization message for configuration is sent. UI may refresh elments based on the configuration key which is a part of the event.
- Add `ConfigSynced` (`DC_EVENT_CONFIG_SYNCED`) event which is emitted when configuration is changed via synchronization message or synchronization message for configuration is sent. UI may refresh elements based on the configuration key which is a part of the event.
- Sync contact creation/rename across devices ([#5163](https://github.com/deltachat/deltachat-core-rust/pull/5163)).
- Encrypt MDNs ([#5175](https://github.com/deltachat/deltachat-core-rust/pull/5175)).
- Only try to configure non-strict TLS checks if explicitly set ([#5181](https://github.com/deltachat/deltachat-core-rust/pull/5181)).
@@ -5369,3 +5669,16 @@ https://github.com/deltachat/deltachat-core-rust/pulls?q=is%3Apr+is%3Aclosed
[1.151.0]: https://github.com/deltachat/deltachat-core-rust/compare/v1.150.0..v1.151.0
[1.151.1]: https://github.com/deltachat/deltachat-core-rust/compare/v1.151.0..v1.151.1
[1.151.2]: https://github.com/deltachat/deltachat-core-rust/compare/v1.151.1..v1.151.2
[1.151.3]: https://github.com/deltachat/deltachat-core-rust/compare/v1.151.2..v1.151.3
[1.151.4]: https://github.com/deltachat/deltachat-core-rust/compare/v1.151.3..v1.151.4
[1.151.5]: https://github.com/deltachat/deltachat-core-rust/compare/v1.151.4..v1.151.5
[1.151.6]: https://github.com/deltachat/deltachat-core-rust/compare/v1.151.5..v1.151.6
[1.152.0]: https://github.com/deltachat/deltachat-core-rust/compare/v1.151.6..v1.152.0
[1.152.1]: https://github.com/deltachat/deltachat-core-rust/compare/v1.152.0..v1.152.1
[1.152.2]: https://github.com/deltachat/deltachat-core-rust/compare/v1.152.1..v1.152.2
[1.153.0]: https://github.com/deltachat/deltachat-core-rust/compare/v1.152.2..v1.153.0
[1.154.0]: https://github.com/deltachat/deltachat-core-rust/compare/v1.153.0..v1.154.0
[1.154.1]: https://github.com/deltachat/deltachat-core-rust/compare/v1.154.0..v1.154.1
[1.154.2]: https://github.com/deltachat/deltachat-core-rust/compare/v1.154.1..v1.154.2
[1.154.3]: https://github.com/deltachat/deltachat-core-rust/compare/v1.154.2..v1.154.3
[1.155.0]: https://github.com/deltachat/deltachat-core-rust/compare/v1.154.3..v1.155.0

View File

@@ -27,7 +27,7 @@ add_custom_command(
PREFIX=${CMAKE_INSTALL_PREFIX}
LIBDIR=${CMAKE_INSTALL_FULL_LIBDIR}
INCLUDEDIR=${CMAKE_INSTALL_FULL_INCLUDEDIR}
${CARGO} build --target-dir=${CMAKE_BINARY_DIR}/target --release --features jsonrpc
${CARGO} build --target-dir=${CMAKE_BINARY_DIR}/target --release
WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}/deltachat-ffi
)

View File

@@ -81,7 +81,7 @@ If you want to contribute a code, follow this guide.
CI runs the tests and checks code formatting.
While it is running, self-review your PR to make sure all the changes you expect are there
and there are no accidentally commited unrelated changes and files.
and there are no accidentally committed unrelated changes and files.
Push the necessary fixup commits or force-push to your branch if needed.

699
Cargo.lock generated

File diff suppressed because it is too large Load Diff

View File

@@ -1,9 +1,9 @@
[package]
name = "deltachat"
version = "1.151.2"
version = "1.155.0"
edition = "2021"
license = "MPL-2.0"
rust-version = "1.77"
rust-version = "1.81"
repository = "https://github.com/deltachat/deltachat-core-rust"
[profile.dev]
@@ -39,20 +39,20 @@ format-flowed = { path = "./format-flowed" }
ratelimit = { path = "./deltachat-ratelimit" }
anyhow = { workspace = true }
async-broadcast = "0.7.1"
async-broadcast = "0.7.2"
async-channel = { workspace = true }
async-imap = { version = "0.10.2", default-features = false, features = ["runtime-tokio", "compress"] }
async-native-tls = { version = "0.5", default-features = false, features = ["runtime-tokio"] }
async-smtp = { version = "0.9", default-features = false, features = ["runtime-tokio"] }
async-smtp = { version = "0.10", default-features = false, features = ["runtime-tokio"] }
async_zip = { version = "0.0.17", default-features = false, features = ["deflate", "tokio-fs"] }
base64 = { workspace = true }
brotli = { version = "7", default-features=false, features = ["std"] }
bytes = "1"
chrono = { workspace = true, features = ["alloc", "clock", "std"] }
email = { git = "https://github.com/deltachat/rust-email", branch = "master" }
encoded-words = { git = "https://github.com/async-email/encoded-words", branch = "master" }
encoded-words = "0.2"
escaper = "0.1"
fast-socks5 = "0.9"
fast-socks5 = "0.10"
fd-lock = "4"
futures-lite = { workspace = true }
futures = { workspace = true }
@@ -76,7 +76,7 @@ num-traits = { workspace = true }
once_cell = { workspace = true }
parking_lot = "0.12"
percent-encoding = "2.3"
pgp = { version = "0.14.0", default-features = false }
pgp = { version = "0.14.2", default-features = false }
pin-project = "1"
qrcodegen = "1.7.0"
quick-xml = "0.37"
@@ -85,15 +85,15 @@ rand = { workspace = true }
regex = { workspace = true }
rusqlite = { workspace = true, features = ["sqlcipher"] }
rust-hsluv = "0.1"
rustls-pki-types = "1.10.0"
rustls = { version = "0.23.19", default-features = false }
rustls-pki-types = "1.10.1"
rustls = { version = "0.23.20", default-features = false }
sanitize-filename = { workspace = true }
serde_json = { workspace = true }
serde_urlencoded = "0.7.1"
serde = { workspace = true, features = ["derive"] }
sha-1 = "0.10"
sha2 = "0.10"
shadowsocks = { version = "1.21.0", default-features = false, features = ["aead-cipher-2022"] }
shadowsocks = { version = "1.22.0", default-features = false, features = ["aead-cipher", "aead-cipher-2022"] }
smallvec = "1.13.2"
strum = "0.26"
strum_macros = "0.26"
@@ -101,8 +101,8 @@ tagger = "4.3.4"
textwrap = "0.16.1"
thiserror = { workspace = true }
tokio-io-timeout = "1.2.0"
tokio-rustls = { version = "0.26.0", default-features = false }
tokio-stream = { version = "0.1.16", features = ["fs"] }
tokio-rustls = { version = "0.26.1", default-features = false }
tokio-stream = { version = "0.1.17", features = ["fs"] }
tokio-tar = { version = "0.3" } # TODO: integrate tokio into async-tar
tokio-util = { workspace = true }
tokio = { workspace = true, features = ["fs", "rt-multi-thread", "macros"] }
@@ -110,6 +110,7 @@ toml = "0.8"
url = "2"
uuid = { version = "1", features = ["serde", "v4"] }
webpki-roots = "0.26.7"
blake3 = "1.5.5"
[dev-dependencies]
anyhow = { workspace = true, features = ["backtrace"] } # Enable `backtrace` feature in tests.
@@ -120,7 +121,7 @@ nu-ansi-term = { workspace = true }
pretty_assertions = "1.4.1"
proptest = { version = "1", default-features = false, features = ["std"] }
tempfile = { workspace = true }
testdir = "0.9.0"
testdir = "0.9.3"
tokio = { workspace = true, features = ["rt-multi-thread", "macros"] }
[workspace]
@@ -150,6 +151,7 @@ harness = false
[[bench]]
name = "receive_emails"
required-features = ["internals"]
harness = false
[[bench]]
@@ -168,7 +170,7 @@ harness = false
anyhow = "1"
async-channel = "2.3.1"
base64 = "0.22"
chrono = { version = "0.4.38", default-features = false }
chrono = { version = "0.4.39", default-features = false }
deltachat-contact-tools = { path = "deltachat-contact-tools" }
deltachat-jsonrpc = { path = "deltachat-jsonrpc", default-features = false }
deltachat = { path = ".", default-features = false }
@@ -188,7 +190,7 @@ serde_json = "1"
tempfile = "3.14.0"
thiserror = "1"
tokio = "1"
tokio-util = "0.7.11"
tokio-util = "0.7.13"
tracing-subscriber = "0.3"
yerpc = "0.6.2"

View File

@@ -161,7 +161,6 @@ $ cargo bolero test fuzz_format_flowed --release=false -e afl -s NONE
## Features
- `vendored`: When using Openssl for TLS, this bundles a vendored version.
- `nightly`: Enable nightly only performance and security related features.
## Update Provider Data
@@ -178,8 +177,8 @@ Language bindings are available for:
- **C** \[[📂 source](./deltachat-ffi) | [📚 docs](https://c.delta.chat)\]
- **Node.js**
- over cffi: \[[📂 source](./node) | [📦 npm](https://www.npmjs.com/package/deltachat-node) | [📚 docs](https://js.delta.chat)\]
- over jsonrpc built with napi.rs (experimental): \[[📂 source](https://github.com/deltachat/napi-jsonrpc) | [📦 npm](https://www.npmjs.com/package/@deltachat/napi-jsonrpc)\]
- over JSON-RPC: \[[📂 source](./deltachat-rpc-client) | [📦 npm](https://www.npmjs.com/package/@deltachat/jsonrpc-client) | [📚 docs](https://js.jsonrpc.delta.chat/)\]
- over CFFI[^1]: \[[📂 source](./node) | [📦 npm](https://www.npmjs.com/package/deltachat-node) | [📚 docs](https://js.delta.chat/)\]
- **Python** \[[📂 source](./python) | [📦 pypi](https://pypi.org/project/deltachat) | [📚 docs](https://py.delta.chat)\]
- **Go**
- over jsonrpc: \[[📂 source](https://github.com/deltachat/deltachat-rpc-client-go/)\]

View File

@@ -12,18 +12,18 @@ use deltachat::{
};
use tempfile::tempdir;
async fn recv_all_emails(context: Context) -> Context {
async fn recv_all_emails(context: Context, iteration: u32) -> Context {
for i in 0..100 {
let imf_raw = format!(
"Subject: Benchmark
Message-ID: Mr.OssSYnOFkhR.{i}@testrun.org
Message-ID: Mr.{iteration}.{i}@testrun.org
Date: Sat, 07 Dec 2019 19:00:27 +0000
To: alice@example.com
From: sender@testrun.org
Chat-Version: 1.0
Chat-Disposition-Notification-To: sender@testrun.org
Chat-User-Avatar: 0
In-Reply-To: Mr.OssSYnOFkhR.{i_dec}@testrun.org
In-Reply-To: Mr.{iteration}.{i_dec}@testrun.org
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed; delsp=no
@@ -41,11 +41,11 @@ Hello {i}",
/// Receive 100 emails that remove charlie@example.com and add
/// him back
async fn recv_groupmembership_emails(context: Context) -> Context {
async fn recv_groupmembership_emails(context: Context, iteration: u32) -> Context {
for i in 0..50 {
let imf_raw = format!(
"Subject: Benchmark
Message-ID: Gr.OssSYnOFkhR.{i}@testrun.org
Message-ID: Gr.{iteration}.ADD.{i}@testrun.org
Date: Sat, 07 Dec 2019 19:00:27 +0000
To: alice@example.com, b@example.com, c@example.com, d@example.com, e@example.com, f@example.com
From: sender@testrun.org
@@ -53,13 +53,12 @@ Chat-Version: 1.0
Chat-Disposition-Notification-To: sender@testrun.org
Chat-User-Avatar: 0
Chat-Group-Member-Added: charlie@example.com
In-Reply-To: Gr.OssSYnOFkhR.{i_dec}@testrun.org
In-Reply-To: Gr.{iteration}.REMOVE.{i_dec}@testrun.org
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed; delsp=no
Hello {i}",
i = i,
i_dec = i - 1,
);
receive_imf(&context, black_box(imf_raw.as_bytes()), false)
@@ -68,7 +67,7 @@ Hello {i}",
let imf_raw = format!(
"Subject: Benchmark
Message-ID: Gr.OssSYnOFkhR.{i}@testrun.org
Message-ID: Gr.{iteration}.REMOVE.{i}@testrun.org
Date: Sat, 07 Dec 2019 19:00:27 +0000
To: alice@example.com, b@example.com, c@example.com, d@example.com, e@example.com, f@example.com
From: sender@testrun.org
@@ -76,14 +75,12 @@ Chat-Version: 1.0
Chat-Disposition-Notification-To: sender@testrun.org
Chat-User-Avatar: 0
Chat-Group-Member-Removed: charlie@example.com
In-Reply-To: Gr.OssSYnOFkhR.{i_dec}@testrun.org
In-Reply-To: Gr.{iteration}.ADD.{i}@testrun.org
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed; delsp=no
Hello {i}",
i = i,
i_dec = i - 1,
Hello {i}"
);
receive_imf(&context, black_box(imf_raw.as_bytes()), false)
.await
@@ -129,11 +126,13 @@ fn criterion_benchmark(c: &mut Criterion) {
group.bench_function("Receive 100 simple text msgs", |b| {
let rt = tokio::runtime::Runtime::new().unwrap();
let context = rt.block_on(create_context());
let mut i = 0;
b.to_async(&rt).iter(|| {
let ctx = context.clone();
i += 1;
async move {
recv_all_emails(black_box(ctx)).await;
recv_all_emails(black_box(ctx), i).await;
}
});
});
@@ -142,11 +141,13 @@ fn criterion_benchmark(c: &mut Criterion) {
|b| {
let rt = tokio::runtime::Runtime::new().unwrap();
let context = rt.block_on(create_context());
let mut i = 0;
b.to_async(&rt).iter(|| {
let ctx = context.clone();
i += 1;
async move {
recv_groupmembership_emails(black_box(ctx)).await;
recv_groupmembership_emails(black_box(ctx), i).await;
}
});
},

View File

@@ -1,80 +0,0 @@
#!/usr/bin/env python3
# Examples:
#
# Original server that doesn't use SSL:
# ./proxy.py 8080 imap.nauta.cu 143
# ./proxy.py 8081 smtp.nauta.cu 25
#
# Original server that uses SSL:
# ./proxy.py 8080 testrun.org 993 --ssl
# ./proxy.py 8081 testrun.org 465 --ssl
from datetime import datetime
import argparse
import selectors
import ssl
import socket
import socketserver
class Proxy(socketserver.ThreadingTCPServer):
allow_reuse_address = True
def __init__(self, proxy_host, proxy_port, real_host, real_port, use_ssl):
self.real_host = real_host
self.real_port = real_port
self.use_ssl = use_ssl
super().__init__((proxy_host, proxy_port), RequestHandler)
class RequestHandler(socketserver.BaseRequestHandler):
def handle(self):
print('{} - {} CONNECTED.'.format(datetime.now(), self.client_address))
total = 0
real_server = (self.server.real_host, self.server.real_port)
with socket.create_connection(real_server) as sock:
if self.server.use_ssl:
context = ssl.create_default_context()
sock = context.wrap_socket(
sock, server_hostname=real_server[0])
forward = {self.request: sock, sock: self.request}
sel = selectors.DefaultSelector()
sel.register(self.request, selectors.EVENT_READ,
self.client_address)
sel.register(sock, selectors.EVENT_READ, real_server)
active = True
while active:
events = sel.select()
for key, mask in events:
print('\n{} - {} wrote:'.format(datetime.now(), key.data))
data = key.fileobj.recv(1024)
received = len(data)
total += received
print(data)
print('{} Bytes\nTotal: {} Bytes'.format(received, total))
if data:
forward[key.fileobj].sendall(data)
else:
print('\nCLOSING CONNECTION.\n\n')
forward[key.fileobj].close()
key.fileobj.close()
active = False
if __name__ == '__main__':
p = argparse.ArgumentParser(description='Simple Python Proxy')
p.add_argument(
"proxy_port", help="the port where the proxy will listen", type=int)
p.add_argument('host', help="the real host")
p.add_argument('port', help="the port of the real host", type=int)
p.add_argument("--ssl", help="use ssl to connect to the real host",
action="store_true")
args = p.parse_args()
with Proxy('', args.proxy_port, args.host, args.port, args.ssl) as proxy:
proxy.serve_forever()

View File

@@ -1,6 +1,6 @@
[package]
name = "deltachat_ffi"
version = "1.151.2"
version = "1.155.0"
description = "Deltachat FFI"
edition = "2018"
readme = "README.md"
@@ -15,7 +15,7 @@ crate-type = ["cdylib", "staticlib"]
[dependencies]
deltachat = { workspace = true, default-features = false }
deltachat-jsonrpc = { workspace = true, optional = true }
deltachat-jsonrpc = { workspace = true }
libc = { workspace = true }
human-panic = { version = "2", default-features = false }
num-traits = { workspace = true }
@@ -30,5 +30,4 @@ yerpc = { workspace = true, features = ["anyhow_expose"] }
[features]
default = ["vendored"]
vendored = ["deltachat/vendored", "deltachat-jsonrpc/vendored"]
jsonrpc = ["dep:deltachat-jsonrpc"]

View File

@@ -418,7 +418,7 @@ char* dc_get_blobdir (const dc_context_t* context);
* - `e2ee_enabled` = 0=no end-to-end-encryption, 1=prefer end-to-end-encryption (default)
* - `mdns_enabled` = 0=do not send or request read receipts,
* 1=send and request read receipts
* default=send and request read receipts, only send but not reuqest if `bot` is set
* default=send and request read receipts, only send but not request if `bot` is set
* - `bcc_self` = 0=do not send a copy of outgoing messages to self,
* 1=send a copy of outgoing messages to self (default).
* Sending messages to self is needed for a proper multi-account setup,
@@ -722,12 +722,6 @@ char* dc_get_connectivity_html (dc_context_t* context);
int dc_get_push_state (dc_context_t* context);
/**
* Only used by the python tests.
*/
int dc_all_work_done (dc_context_t* context);
// connect
/**
@@ -969,54 +963,6 @@ uint32_t dc_create_chat_by_contact_id (dc_context_t* context, uint32_t co
uint32_t dc_get_chat_id_by_contact_id (dc_context_t* context, uint32_t contact_id);
/**
* Prepare a message for sending.
*
* Call this function if the file to be sent is still in creation.
* Once you're done with creating the file, call dc_send_msg() as usual
* and the message will really be sent.
*
* This is useful as the user can already send the next messages while
* e.g. the recoding of a video is not yet finished. Or the user can even forward
* the message with the file being still in creation to other groups.
*
* Files being sent with the increation-method must be placed in the
* blob directory, see dc_get_blobdir().
* If the increation-method is not used - which is probably the normal case -
* dc_send_msg() copies the file to the blob directory if it is not yet there.
* To distinguish the two cases, msg->state must be set properly. The easiest
* way to ensure this is to re-use the same object for both calls.
*
* Example:
* ~~~
* char* blobdir = dc_get_blobdir(context);
* char* file_to_send = mprintf("%s/%s", blobdir, "send.mp4")
*
* dc_msg_t* msg = dc_msg_new(context, DC_MSG_VIDEO);
* dc_msg_set_file(msg, file_to_send, NULL);
* dc_prepare_msg(context, chat_id, msg);
*
* // ... create the file ...
*
* dc_send_msg(context, chat_id, msg);
*
* dc_msg_unref(msg);
* free(file_to_send);
* dc_str_unref(file_to_send);
* ~~~
*
* @memberof dc_context_t
* @param context The context object as returned from dc_context_new().
* @param chat_id The chat ID to send the message to.
* @param msg The message object to send to the chat defined by the chat ID.
* On success, msg_id and state of the object are set up,
* The function does not take ownership of the object,
* so you have to free it using dc_msg_unref() as usual.
* @return The ID of the message that is being prepared.
*/
uint32_t dc_prepare_msg (dc_context_t* context, uint32_t chat_id, dc_msg_t* msg);
/**
* Send a message defined by a dc_msg_t object to a chat.
*
@@ -1041,13 +987,11 @@ uint32_t dc_prepare_msg (dc_context_t* context, uint32_t ch
* If that fails, is not possible, or the image is already small enough, the image is sent as original.
* If you want images to be always sent as the original file, use the #DC_MSG_FILE type.
*
* Videos and other file types are currently not recoded by the library,
* with dc_prepare_msg(), however, you can do that from the UI.
* Videos and other file types are currently not recoded by the library.
*
* @memberof dc_context_t
* @param context The context object as returned from dc_context_new().
* @param chat_id The chat ID to send the message to.
* If dc_prepare_msg() was called before, this parameter can be 0.
* @param msg The message object to send to the chat defined by the chat ID.
* On success, msg_id of the object is set up,
* The function does not take ownership of the object,
@@ -1064,7 +1008,6 @@ uint32_t dc_send_msg (dc_context_t* context, uint32_t ch
* @memberof dc_context_t
* @param context The context object as returned from dc_context_new().
* @param chat_id The chat ID to send the message to.
* If dc_prepare_msg() was called before, this parameter can be 0.
* @param msg The message object to send to the chat defined by the chat ID.
* On success, msg_id of the object is set up,
* The function does not take ownership of the object,
@@ -2031,6 +1974,36 @@ void dc_delete_msgs (dc_context_t* context, const uint3
void dc_forward_msgs (dc_context_t* context, const uint32_t* msg_ids, int msg_cnt, uint32_t chat_id);
/**
* Save a copy of messages in "Saved Messages".
*
* In contrast to forwarding messages,
* information as author, date and origin are preserved.
* The action completes locally, so "Saved Messages" do not show sending errors in case one is offline.
* Still, a sync message is emitted, so that other devices will save the same message,
* as long as not deleted before.
*
* To check if a message was saved, use dc_msg_get_saved_msg_id(),
* UI may show an indicator and offer an "Unsave" instead of a "Save" button then.
*
* The other way round, from inside the "Saved Messages" chat,
* UI may show the indicator and "Unsave" button checking dc_msg_get_original_msg_id()
* and offer a button to go the original message.
*
* "Unsave" is done by deleting the saved message.
* Webxdc updates are not copied on purpose.
*
* For performance reasons, esp. when saving lots of messages,
* UI should call this function from a background thread.
*
* @memberof dc_context_t
* @param context The context object.
* @param msg_ids An array of uint32_t containing all message IDs that should be saved.
* @param msg_cnt The number of messages IDs in the msg_ids array.
*/
void dc_save_msgs (dc_context_t* context, const uint32_t* msg_ids, int msg_cnt);
/**
* Resend messages and make information available for newly added chat members.
* Resending sends out the original message, however, recipients and webxdc-status may differ.
@@ -3991,7 +3964,7 @@ int dc_msg_get_viewtype (const dc_msg_t* msg);
*
* Outgoing message states:
* - @ref DC_STATE_OUT_PREPARING - For files which need time to be prepared before they can be sent,
* the message enters this state before @ref DC_STATE_OUT_PENDING.
* the message enters this state before @ref DC_STATE_OUT_PENDING. Deprecated.
* - @ref DC_STATE_OUT_DRAFT - Message saved as draft using dc_set_draft()
* - @ref DC_STATE_OUT_PENDING - The user has pressed the "send" button but the
* message is not yet sent and is pending in some way. Maybe we're offline (no checkmark).
@@ -4201,14 +4174,13 @@ char* dc_msg_get_webxdc_blob (const dc_msg_t* msg, const char*
* defaults to an empty string.
* Implementations may offer an menu or a button to open this URL.
* - internet_access:
* true if the Webxdc should get full internet access, including Webrtc.
* currently, this is only true for encrypted Webxdc's in the self chat
* that have requested internet access in the manifest.
* true if the Webxdc should get internet access;
* this is the case i.e. for experimental maps integration.
* - self_addr: address to be used for `window.webxdc.selfAddr` in JS land.
* - send_update_interval: Milliseconds to wait before calling `sendUpdate()` again since the last call.
* Should be exposed to `webxdc.sendUpdateInterval` in JS land.
* - send_update_max_size: Maximum number of bytes accepted for a serialized update object.
+ Should be exposed to `webxdc.sendUpdateMaxSize` in JS land.
* Should be exposed to `webxdc.sendUpdateMaxSize` in JS land.
*
* @memberof dc_msg_t
* @param msg The webxdc instance.
@@ -4542,20 +4514,6 @@ int dc_msg_get_info_type (const dc_msg_t* msg);
*/
char* dc_msg_get_webxdc_href (const dc_msg_t* msg);
/**
* Check if a message is still in creation. A message is in creation between
* the calls to dc_prepare_msg() and dc_send_msg().
*
* Typically, this is used for videos that are recoded by the UI before
* they can be sent.
*
* @memberof dc_msg_t
* @param msg The message object.
* @return 1=message is still in creation (dc_send_msg() was not called yet),
* 0=message no longer in creation.
*/
int dc_msg_is_increation (const dc_msg_t* msg);
/**
* Check if the message is an Autocrypt Setup Message.
@@ -4707,7 +4665,7 @@ int dc_msg_has_html (dc_msg_t* msg);
* If the download fails or succeeds,
* the event @ref DC_EVENT_MSGS_CHANGED is emitted.
*
* - @ref DC_DOWNLOAD_UNDECIPHERABLE - The message does not need any futher download action.
* - @ref DC_DOWNLOAD_UNDECIPHERABLE - The message does not need any further download action.
* It was fully downloaded, but we failed to decrypt it.
* - @ref DC_DOWNLOAD_FAILURE - Download error, the user may start over calling dc_download_full_msg() again.
*
@@ -4794,10 +4752,36 @@ void dc_msg_set_override_sender_name(dc_msg_t* msg, const char* name)
* @param file If the message object is used in dc_send_msg() later,
* this must be the full path of the image file to send.
* @param filemime The MIME type of the file. NULL if you don't know or don't care.
* @deprecated 2025-01-21 Use dc_msg_set_file_and_deduplicate instead
*/
void dc_msg_set_file (dc_msg_t* msg, const char* file, const char* filemime);
/**
* Sets the file associated with a message.
*
* If `name` is non-null, it is used as the file name
* and the actual current name of the file is ignored.
*
* If the source file is already in the blobdir, it will be renamed,
* otherwise it will be copied to the blobdir first.
*
* In order to deduplicate files that contain the same data,
* the file will be named `<hash>.<extension>`, e.g. `ce940175885d7b78f7b7e9f1396611f.jpg`.
*
* NOTE:
* - This function will rename the file. To get the new file path, call `get_file()`.
* - The file must not be modified after this function was called.
*
* @memberof dc_msg_t
* @param msg The message object. Must not be NULL.
* @param file The path of the file to attach. Must not be NULL.
* @param name The original filename of the attachment. If NULL, the current name of `file` will be used instead.
* @param filemime The MIME type of the file. NULL if you don't know or don't care.
*/
void dc_msg_set_file_and_deduplicate(dc_msg_t* msg, const char* file, const char* name, const char* filemime);
/**
* Set the dimensions associated with message object.
* Typically this is the width and the height of an image or video associated using dc_msg_set_file().
@@ -4940,6 +4924,33 @@ dc_msg_t* dc_msg_get_quoted_msg (const dc_msg_t* msg);
dc_msg_t* dc_msg_get_parent (const dc_msg_t* msg);
/**
* Get original message ID for a saved message from the "Saved Messages" chat.
*
* Can be used by UI to show a button to go the original message
* and an option to "Unsave" the message.
*
* @param msg The message object. Usually, this refers to a a message inside "Saved Messages".
* @return The message ID of the original message.
* 0 if the given message object is not a "Saved Message"
* or if the original message does no longer exist.
*/
uint32_t dc_msg_get_original_msg_id (const dc_msg_t* msg);
/**
* Check if a message was saved and return its ID inside "Saved Messages".
*
* Deleting the returned message will un-save the message.
* The state "is saved" can be used to show some icon to indicate that a message was saved.
*
* @param msg The message object. Usually, this refers to a a message outside "Saved Messages".
* @return The message ID inside "Saved Messages", if any.
* 0 if the given message object is not saved.
*/
uint32_t dc_msg_get_saved_msg_id (const dc_msg_t* msg);
/**
* Force the message to be sent in plain text.
*
@@ -5465,6 +5476,8 @@ int64_t dc_lot_get_timestamp (const dc_lot_t* lot);
/**
* Message containing a sticker, similar to image.
* NB: When sending, the message viewtype may be changed to `Image` by some heuristics like checking
* for transparent pixels.
* If possible, the UI should display the image without borders in a transparent way.
* A click on a sticker will offer to install the sticker set in some future.
*/
@@ -5569,6 +5582,8 @@ int64_t dc_lot_get_timestamp (const dc_lot_t* lot);
/**
* Outgoing message being prepared. See dc_msg_get_state() for details.
*
* @deprecated 2024-12-07
*/
#define DC_STATE_OUT_PREPARING 18
@@ -5789,6 +5804,23 @@ void dc_jsonrpc_unref(dc_jsonrpc_instance_t* jsonrpc_instance);
* returns immediately and once the result is ready it can be retrieved via dc_jsonrpc_next_response()
* the jsonrpc specification defines an invocation id that can then be used to match request and response.
*
* An overview of JSON-RPC calls is available at
* <https://js.jsonrpc.delta.chat/classes/RawClient.html>.
* Note that the page describes only the rough methods.
* Calling convention, casing etc. does vary, this is a known flaw,
* and at some point we will get to improve that :)
*
* Also, note that most calls are more high-level than this CFFI, require more database calls and are slower.
* They're more suitable for an environment that is totally async and/or cannot use CFFI, which might not be true for native apps.
*
* Notable exceptions that exist only as JSON-RPC and probably never get a CFFI counterpart:
* - getMessageReactions(), sendReaction()
* - getHttpResponse()
* - draftSelfReport()
* - getAccountFileSize()
* - importVcard(), parseVcard(), makeVcard()
* - sendWebxdcRealtimeData, sendWebxdcRealtimeAdvertisement(), leaveWebxdcRealtime()
*
* @memberof dc_jsonrpc_instance_t
* @param jsonrpc_instance jsonrpc instance as returned from dc_jsonrpc_init().
* @param request JSON-RPC request as string
@@ -5809,6 +5841,8 @@ char* dc_jsonrpc_next_response(dc_jsonrpc_instance_t* jsonrpc_instance);
/**
* Make a JSON-RPC call and return a response.
*
* See dc_jsonrpc_request() for an overview of possible calls and for more information.
*
* @memberof dc_jsonrpc_instance_t
* @param jsonrpc_instance jsonrpc instance as returned from dc_jsonrpc_init().
* @param input JSON-RPC request.
@@ -6899,7 +6933,7 @@ void dc_event_unref(dc_event_t* event);
/// "Failed to send message to %1$s."
///
/// Used in status messages.
/// Unused. Was used in group chat status messages.
/// - %1$s will be replaced by the name of the contact the message cannot be sent to
#define DC_STR_FAILED_SENDING_TO 74

View File

@@ -35,6 +35,8 @@ use deltachat::stock_str::StockMessage;
use deltachat::webxdc::StatusUpdateSerial;
use deltachat::*;
use deltachat::{accounts::Accounts, log::LogExt};
use deltachat_jsonrpc::api::CommandApi;
use deltachat_jsonrpc::yerpc::{OutReceiver, RpcClient, RpcSession};
use num_traits::{FromPrimitive, ToPrimitive};
use once_cell::sync::Lazy;
use rand::Rng;
@@ -413,16 +415,6 @@ pub unsafe extern "C" fn dc_get_push_state(context: *const dc_context_t) -> libc
block_on(ctx.push_state()) as libc::c_int
}
#[no_mangle]
pub unsafe extern "C" fn dc_all_work_done(context: *mut dc_context_t) -> libc::c_int {
if context.is_null() {
eprintln!("ignoring careless call to dc_all_work_done()");
return 0;
}
let ctx = &*context;
block_on(async move { ctx.all_work_done().await as libc::c_int })
}
#[no_mangle]
pub unsafe extern "C" fn dc_get_oauth2_url(
context: *mut dc_context_t,
@@ -986,27 +978,6 @@ pub unsafe extern "C" fn dc_get_chat_id_by_contact_id(
})
}
#[no_mangle]
pub unsafe extern "C" fn dc_prepare_msg(
context: *mut dc_context_t,
chat_id: u32,
msg: *mut dc_msg_t,
) -> u32 {
if context.is_null() || chat_id == 0 || msg.is_null() {
eprintln!("ignoring careless call to dc_prepare_msg()");
return 0;
}
let ctx = &mut *context;
let ffi_msg: &mut MessageWrapper = &mut *msg;
block_on(async move {
chat::prepare_msg(ctx, ChatId::new(chat_id), &mut ffi_msg.message)
.await
.unwrap_or_log_default(ctx, "Failed to prepare message")
})
.to_u32()
}
#[no_mangle]
pub unsafe extern "C" fn dc_send_msg(
context: *mut dc_context_t,
@@ -2008,6 +1979,26 @@ pub unsafe extern "C" fn dc_forward_msgs(
})
}
#[no_mangle]
pub unsafe extern "C" fn dc_save_msgs(
context: *mut dc_context_t,
msg_ids: *const u32,
msg_cnt: libc::c_int,
) {
if context.is_null() || msg_ids.is_null() || msg_cnt <= 0 {
eprintln!("ignoring careless call to dc_save_msgs()");
return;
}
let msg_ids = convert_and_prune_message_ids(msg_ids, msg_cnt);
let ctx = &*context;
block_on(async move {
chat::save_msgs(ctx, &msg_ids[..])
.await
.unwrap_or_log_default(ctx, "Failed to save message")
})
}
#[no_mangle]
pub unsafe extern "C" fn dc_resend_msgs(
context: *mut dc_context_t,
@@ -3723,16 +3714,6 @@ pub unsafe extern "C" fn dc_msg_get_webxdc_href(msg: *mut dc_msg_t) -> *mut libc
ffi_msg.message.get_webxdc_href().strdup()
}
#[no_mangle]
pub unsafe extern "C" fn dc_msg_is_increation(msg: *mut dc_msg_t) -> libc::c_int {
if msg.is_null() {
eprintln!("ignoring careless call to dc_msg_is_increation()");
return 0;
}
let ffi_msg = &*msg;
ffi_msg.message.is_increation().into()
}
#[no_mangle]
pub unsafe extern "C" fn dc_msg_is_setupmessage(msg: *mut dc_msg_t) -> libc::c_int {
if msg.is_null() {
@@ -3854,6 +3835,33 @@ pub unsafe extern "C" fn dc_msg_set_file(
)
}
#[no_mangle]
pub unsafe extern "C" fn dc_msg_set_file_and_deduplicate(
msg: *mut dc_msg_t,
file: *const libc::c_char,
name: *const libc::c_char,
filemime: *const libc::c_char,
) {
if msg.is_null() || file.is_null() {
eprintln!("ignoring careless call to dc_msg_set_file_and_deduplicate()");
return;
}
let ffi_msg = &mut *msg;
let ctx = &*ffi_msg.context;
ffi_msg
.message
.set_file_and_deduplicate(
ctx,
as_path(file),
to_opt_string_lossy(name).as_deref(),
to_opt_string_lossy(filemime).as_deref(),
)
.context("Failed to set file")
.log_err(&*ffi_msg.context)
.ok();
}
#[no_mangle]
pub unsafe extern "C" fn dc_msg_set_dimension(
msg: *mut dc_msg_t,
@@ -4019,6 +4027,48 @@ pub unsafe extern "C" fn dc_msg_get_parent(msg: *const dc_msg_t) -> *mut dc_msg_
}
}
#[no_mangle]
pub unsafe extern "C" fn dc_msg_get_original_msg_id(msg: *const dc_msg_t) -> u32 {
if msg.is_null() {
eprintln!("ignoring careless call to dc_msg_get_original_msg_id()");
return 0;
}
let ffi_msg: &MessageWrapper = &*msg;
let context = &*ffi_msg.context;
block_on(async move {
ffi_msg
.message
.get_original_msg_id(context)
.await
.context("failed to get original message")
.log_err(context)
.unwrap_or_default()
.map(|id| id.to_u32())
.unwrap_or(0)
})
}
#[no_mangle]
pub unsafe extern "C" fn dc_msg_get_saved_msg_id(msg: *const dc_msg_t) -> u32 {
if msg.is_null() {
eprintln!("ignoring careless call to dc_msg_get_saved_msg_id()");
return 0;
}
let ffi_msg: &MessageWrapper = &*msg;
let context = &*ffi_msg.context;
block_on(async move {
ffi_msg
.message
.get_saved_msg_id(context)
.await
.context("failed to get original message")
.log_err(context)
.unwrap_or_default()
.map(|id| id.to_u32())
.unwrap_or(0)
})
}
#[no_mangle]
pub unsafe extern "C" fn dc_msg_force_plaintext(msg: *mut dc_msg_t) {
if msg.is_null() {
@@ -4971,105 +5021,97 @@ pub unsafe extern "C" fn dc_accounts_get_event_emitter(
Box::into_raw(Box::new(emitter))
}
#[cfg(feature = "jsonrpc")]
mod jsonrpc {
use deltachat_jsonrpc::api::CommandApi;
use deltachat_jsonrpc::yerpc::{OutReceiver, RpcClient, RpcSession};
pub struct dc_jsonrpc_instance_t {
receiver: OutReceiver,
handle: RpcSession<CommandApi>,
}
use super::*;
pub struct dc_jsonrpc_instance_t {
receiver: OutReceiver,
handle: RpcSession<CommandApi>,
#[no_mangle]
pub unsafe extern "C" fn dc_jsonrpc_init(
account_manager: *mut dc_accounts_t,
) -> *mut dc_jsonrpc_instance_t {
if account_manager.is_null() {
eprintln!("ignoring careless call to dc_jsonrpc_init()");
return ptr::null_mut();
}
#[no_mangle]
pub unsafe extern "C" fn dc_jsonrpc_init(
account_manager: *mut dc_accounts_t,
) -> *mut dc_jsonrpc_instance_t {
if account_manager.is_null() {
eprintln!("ignoring careless call to dc_jsonrpc_init()");
return ptr::null_mut();
}
let account_manager = &*account_manager;
let cmd_api = block_on(deltachat_jsonrpc::api::CommandApi::from_arc(
account_manager.inner.clone(),
));
let account_manager = &*account_manager;
let cmd_api = block_on(deltachat_jsonrpc::api::CommandApi::from_arc(
account_manager.inner.clone(),
));
let (request_handle, receiver) = RpcClient::new();
let handle = RpcSession::new(request_handle, cmd_api);
let (request_handle, receiver) = RpcClient::new();
let handle = RpcSession::new(request_handle, cmd_api);
let instance = dc_jsonrpc_instance_t { receiver, handle };
let instance = dc_jsonrpc_instance_t { receiver, handle };
Box::into_raw(Box::new(instance))
}
Box::into_raw(Box::new(instance))
#[no_mangle]
pub unsafe extern "C" fn dc_jsonrpc_unref(jsonrpc_instance: *mut dc_jsonrpc_instance_t) {
if jsonrpc_instance.is_null() {
eprintln!("ignoring careless call to dc_jsonrpc_unref()");
return;
}
drop(Box::from_raw(jsonrpc_instance));
}
fn spawn_handle_jsonrpc_request(handle: RpcSession<CommandApi>, request: String) {
spawn(async move {
handle.handle_incoming(&request).await;
});
}
#[no_mangle]
pub unsafe extern "C" fn dc_jsonrpc_request(
jsonrpc_instance: *mut dc_jsonrpc_instance_t,
request: *const libc::c_char,
) {
if jsonrpc_instance.is_null() || request.is_null() {
eprintln!("ignoring careless call to dc_jsonrpc_request()");
return;
}
#[no_mangle]
pub unsafe extern "C" fn dc_jsonrpc_unref(jsonrpc_instance: *mut dc_jsonrpc_instance_t) {
if jsonrpc_instance.is_null() {
eprintln!("ignoring careless call to dc_jsonrpc_unref()");
return;
}
drop(Box::from_raw(jsonrpc_instance));
let handle = &(*jsonrpc_instance).handle;
let request = to_string_lossy(request);
spawn_handle_jsonrpc_request(handle.clone(), request);
}
#[no_mangle]
pub unsafe extern "C" fn dc_jsonrpc_next_response(
jsonrpc_instance: *mut dc_jsonrpc_instance_t,
) -> *mut libc::c_char {
if jsonrpc_instance.is_null() {
eprintln!("ignoring careless call to dc_jsonrpc_next_response()");
return ptr::null_mut();
}
let api = &*jsonrpc_instance;
block_on(api.receiver.recv())
.map(|result| serde_json::to_string(&result).unwrap_or_default().strdup())
.unwrap_or(ptr::null_mut())
}
fn spawn_handle_jsonrpc_request(handle: RpcSession<CommandApi>, request: String) {
spawn(async move {
handle.handle_incoming(&request).await;
});
#[no_mangle]
pub unsafe extern "C" fn dc_jsonrpc_blocking_call(
jsonrpc_instance: *mut dc_jsonrpc_instance_t,
input: *const libc::c_char,
) -> *mut libc::c_char {
if jsonrpc_instance.is_null() {
eprintln!("ignoring careless call to dc_jsonrpc_blocking_call()");
return ptr::null_mut();
}
#[no_mangle]
pub unsafe extern "C" fn dc_jsonrpc_request(
jsonrpc_instance: *mut dc_jsonrpc_instance_t,
request: *const libc::c_char,
) {
if jsonrpc_instance.is_null() || request.is_null() {
eprintln!("ignoring careless call to dc_jsonrpc_request()");
return;
}
let handle = &(*jsonrpc_instance).handle;
let request = to_string_lossy(request);
spawn_handle_jsonrpc_request(handle.clone(), request);
}
#[no_mangle]
pub unsafe extern "C" fn dc_jsonrpc_next_response(
jsonrpc_instance: *mut dc_jsonrpc_instance_t,
) -> *mut libc::c_char {
if jsonrpc_instance.is_null() {
eprintln!("ignoring careless call to dc_jsonrpc_next_response()");
return ptr::null_mut();
}
let api = &*jsonrpc_instance;
block_on(api.receiver.recv())
.map(|result| serde_json::to_string(&result).unwrap_or_default().strdup())
.unwrap_or(ptr::null_mut())
}
#[no_mangle]
pub unsafe extern "C" fn dc_jsonrpc_blocking_call(
jsonrpc_instance: *mut dc_jsonrpc_instance_t,
input: *const libc::c_char,
) -> *mut libc::c_char {
if jsonrpc_instance.is_null() {
eprintln!("ignoring careless call to dc_jsonrpc_blocking_call()");
return ptr::null_mut();
}
let api = &*jsonrpc_instance;
let input = to_string_lossy(input);
let res = block_on(api.handle.process_incoming(&input));
match res {
Some(message) => {
if let Ok(message) = serde_json::to_string(&message) {
message.strdup()
} else {
ptr::null_mut()
}
let api = &*jsonrpc_instance;
let input = to_string_lossy(input);
let res = block_on(api.handle.process_incoming(&input));
match res {
Some(message) => {
if let Ok(message) = serde_json::to_string(&message) {
message.strdup()
} else {
ptr::null_mut()
}
None => ptr::null_mut(),
}
None => ptr::null_mut(),
}
}

View File

@@ -1,6 +1,6 @@
[package]
name = "deltachat-jsonrpc"
version = "1.151.2"
version = "1.155.0"
description = "DeltaChat JSON-RPC API"
edition = "2021"
default-run = "deltachat-jsonrpc-server"
@@ -33,7 +33,7 @@ base64 = { workspace = true }
# optional dependencies
axum = { version = "0.7", optional = true, features = ["ws"] }
env_logger = { version = "0.11.5", optional = true }
env_logger = { version = "0.11.6", optional = true }
[dev-dependencies]
tokio = { workspace = true, features = ["full", "rt-multi-thread"] }

View File

@@ -836,6 +836,13 @@ impl CommandApi {
Ok(contacts.iter().map(|id| id.to_u32()).collect::<Vec<u32>>())
}
/// Returns contact IDs of the past chat members.
async fn get_past_chat_contacts(&self, account_id: u32, chat_id: u32) -> Result<Vec<u32>> {
let ctx = self.get_context(account_id).await?;
let contacts = chat::get_past_chat_contacts(&ctx, ChatId::new(chat_id)).await?;
Ok(contacts.iter().map(|id| id.to_u32()).collect::<Vec<u32>>())
}
/// Create a new group chat.
///
/// After creation,

View File

@@ -1,7 +1,7 @@
use std::time::{Duration, SystemTime};
use anyhow::{bail, Context as _, Result};
use deltachat::chat::{self, get_chat_contacts, ChatVisibility};
use deltachat::chat::{self, get_chat_contacts, get_past_chat_contacts, ChatVisibility};
use deltachat::chat::{Chat, ChatId};
use deltachat::constants::Chattype;
use deltachat::contact::{Contact, ContactId};
@@ -39,6 +39,10 @@ pub struct FullChat {
is_self_talk: bool,
contacts: Vec<ContactObject>,
contact_ids: Vec<u32>,
/// Contact IDs of the past chat members.
past_contact_ids: Vec<u32>,
color: String,
fresh_message_counter: usize,
// is_group - please check over chat.type in frontend instead
@@ -59,6 +63,7 @@ impl FullChat {
let chat = Chat::load_from_db(context, rust_chat_id).await?;
let contact_ids = get_chat_contacts(context, rust_chat_id).await?;
let past_contact_ids = get_past_chat_contacts(context, rust_chat_id).await?;
let mut contacts = Vec::with_capacity(contact_ids.len());
@@ -111,6 +116,7 @@ impl FullChat {
is_self_talk: chat.is_self_talk(),
contacts,
contact_ids: contact_ids.iter().map(|id| id.to_u32()).collect(),
past_contact_ids: past_contact_ids.iter().map(|id| id.to_u32()).collect(),
color,
fresh_message_counter,
is_contact_request: chat.is_contact_request(),

View File

@@ -69,7 +69,7 @@ pub enum EventType {
/// or for functions that are expected to fail (eg. autocryptContinueKeyTransfer())
/// it might be better to delay showing these events until the function has really
/// failed (returned false). It should be sufficient to report only the *last* error
/// in a messasge box then.
/// in a message box then.
Error { msg: String },
/// An action cannot be performed because the user is not in the group.
@@ -109,6 +109,7 @@ pub enum EventType {
/// Incoming webxdc info or summary update, should be notified.
#[serde(rename_all = "camelCase")]
IncomingWebxdcNotify {
chat_id: u32,
contact_id: u32,
msg_id: u32,
text: String,
@@ -343,11 +344,13 @@ impl From<CoreEventType> for EventType {
reaction: reaction.as_str().to_string(),
},
CoreEventType::IncomingWebxdcNotify {
chat_id,
contact_id,
msg_id,
text,
href,
} => IncomingWebxdcNotify {
chat_id: chat_id.to_u32(),
contact_id: contact_id.to_u32(),
msg_id: msg_id.to_u32(),
text,

View File

@@ -130,11 +130,6 @@ impl MessageObject {
let webxdc_info = if message.get_viewtype() == Viewtype::Webxdc {
Some(WebxdcMessageInfo::get_for_message(context, msg_id).await?)
} else if let (deltachat::mimeparser::SystemMessage::WebxdcInfoMessage, Some(parent_msg)) =
(message.get_info_type(), message.parent(context).await?)
{
// get webcdx info from parent message
Some(WebxdcMessageInfo::get_for_message(context, parent_msg.get_id()).await?)
} else {
None
};
@@ -278,6 +273,9 @@ pub enum MessageViewtype {
Gif,
/// Message containing a sticker, similar to image.
/// NB: When sending, the message viewtype may be changed to `Image` by some heuristics like
/// checking for transparent pixels. Use `Message::force_sticker()` to disable them.
///
/// If possible, the ui should display the image without borders in a transparent way.
/// A click on a sticker will offer to install the sticker set in some future.
Sticker,

View File

@@ -58,5 +58,5 @@
},
"type": "module",
"types": "dist/deltachat.d.ts",
"version": "1.151.2"
"version": "1.155.0"
}

View File

@@ -1,6 +1,6 @@
[package]
name = "deltachat-repl"
version = "1.151.2"
version = "1.155.0"
license = "MPL-2.0"
edition = "2021"
repository = "https://github.com/deltachat/deltachat-core-rust"
@@ -13,7 +13,7 @@ log = { workspace = true }
nu-ansi-term = { workspace = true }
qr2term = "0.3.3"
rusqlite = { workspace = true }
rustyline = "14"
rustyline = "15"
tokio = { workspace = true, features = ["fs", "rt-multi-thread", "macros"] }
tracing-subscriber = { workspace = true, features = ["env-filter"] }

View File

@@ -939,7 +939,7 @@ pub async fn cmdline(context: Context, line: &str, chat_id: &mut ChatId) -> Resu
} else {
Viewtype::File
});
msg.set_file(arg1, None);
msg.set_file_and_deduplicate(&context, Path::new(arg1), None, None)?;
msg.set_text(arg2.to_string());
chat::send_msg(&context, sel_chat.as_ref().unwrap().get_id(), &mut msg).await?;
}

View File

@@ -22,7 +22,7 @@ use log::{error, info, warn};
use nu_ansi_term::Color;
use rustyline::completion::{Completer, FilenameCompleter, Pair};
use rustyline::error::ReadlineError;
use rustyline::highlight::{Highlighter, MatchingBracketHighlighter};
use rustyline::highlight::{CmdKind as HighlightCmdKind, Highlighter, MatchingBracketHighlighter};
use rustyline::hint::{Hinter, HistoryHinter};
use rustyline::validate::Validator;
use rustyline::{
@@ -298,8 +298,8 @@ impl Highlighter for DcHelper {
self.highlighter.highlight(line, pos)
}
fn highlight_char(&self, line: &str, pos: usize, forced: bool) -> bool {
self.highlighter.highlight_char(line, pos, forced)
fn highlight_char(&self, line: &str, pos: usize, kind: HighlightCmdKind) -> bool {
self.highlighter.highlight_char(line, pos, kind)
}
}

View File

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "deltachat-rpc-client"
version = "1.151.2"
version = "1.155.0"
description = "Python client for Delta Chat core JSON-RPC interface"
classifiers = [
"Development Status :: 5 - Production/Stable",
@@ -13,7 +13,6 @@ classifiers = [
"Operating System :: POSIX :: Linux",
"Operating System :: MacOS :: MacOS X",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.7",
"Programming Language :: Python :: 3.8",
"Programming Language :: Python :: 3.9",
"Programming Language :: Python :: 3.10",
@@ -24,6 +23,7 @@ classifiers = [
"Topic :: Communications :: Email"
]
readme = "README.md"
requires-python = ">=3.8"
[tool.setuptools.package-data]
deltachat_rpc_client = [

View File

@@ -238,6 +238,11 @@ class Chat:
contacts = self._rpc.get_chat_contacts(self.account.id, self.id)
return [Contact(self.account, contact_id) for contact_id in contacts]
def get_past_contacts(self) -> list[Contact]:
"""Get past contacts for this chat."""
past_contacts = self._rpc.get_past_chat_contacts(self.account.id, self.id)
return [Contact(self.account, contact_id) for contact_id in past_contacts]
def set_image(self, path: str) -> None:
"""Set profile image of this chat.

View File

@@ -41,6 +41,7 @@ class EventType(str, Enum):
REACTIONS_CHANGED = "ReactionsChanged"
INCOMING_MSG = "IncomingMsg"
INCOMING_MSG_BUNCH = "IncomingMsgBunch"
INCOMING_REACTION = "IncomingReaction"
MSGS_NOTICED = "MsgsNoticed"
MSG_DELIVERED = "MsgDelivered"
MSG_FAILED = "MsgFailed"

View File

@@ -131,10 +131,7 @@ class Rpc:
def reader_loop(self) -> None:
try:
while True:
line = self.process.stdout.readline()
if not line: # EOF
break
while line := self.process.stdout.readline():
response = json.loads(line)
if "id" in response:
response_id = response["id"]
@@ -150,10 +147,7 @@ class Rpc:
def writer_loop(self) -> None:
"""Writer loop ensuring only a single thread writes requests."""
try:
while True:
request = self.request_queue.get()
if not request:
break
while request := self.request_queue.get():
data = (json.dumps(request) + "\n").encode()
self.process.stdin.write(data)
self.process.stdin.flush()

View File

@@ -73,22 +73,25 @@ def test_qr_securejoin(acfactory, protect, tmp_path):
qr_code = alice_chat.get_qr_code()
bob.secure_join(qr_code)
# Check that at least some of the handshake messages are deleted.
# Alice deletes "vg-request".
while True:
event = alice.wait_for_event()
if event["kind"] == "ImapMessageDeleted":
break
alice.wait_for_securejoin_inviter_success()
# Bob deletes "vg-auth-required", Alice deletes "vg-request-with-auth".
for ac in [alice, bob]:
while True:
event = ac.wait_for_event()
if event["kind"] == "ImapMessageDeleted":
break
alice.wait_for_securejoin_inviter_success()
bob.wait_for_securejoin_joiner_success()
# Test that Alice verified Bob's profile.
alice_contact_bob = alice.get_contact_by_addr(bob.get_config("addr"))
alice_contact_bob_snapshot = alice_contact_bob.get_snapshot()
assert alice_contact_bob_snapshot.is_verified
bob.wait_for_securejoin_joiner_success()
snapshot = bob.get_message_by_id(bob.wait_for_incoming_msg_event().msg_id).get_snapshot()
assert snapshot.text == "Member Me ({}) added by {}.".format(bob.get_config("addr"), alice.get_config("addr"))
assert snapshot.chat.get_basic_snapshot().is_protected == protect

View File

@@ -231,7 +231,9 @@ def test_chat(acfactory) -> None:
group.get_fresh_message_count()
group.mark_noticed()
assert group.get_contacts()
group.remove_contact(alice_chat_bob)
assert group.get_past_contacts() == []
group.remove_contact(alice_contact_bob)
assert len(group.get_past_contacts()) == 1
group.get_locations()

View File

@@ -1,6 +1,6 @@
[package]
name = "deltachat-rpc-server"
version = "1.151.2"
version = "1.155.0"
description = "DeltaChat JSON-RPC server"
edition = "2021"
readme = "README.md"

View File

@@ -65,13 +65,13 @@ so by default it uses the prebuilds.
- this will run `update_optional_dependencies_and_version.js` (in the `prepack` script),
which puts all platform packages into `optionalDependencies` and updates the `version` in `package.json`
## How to build a version you can use localy on your host machine for development
## How to build a version you can use locally on your host machine for development
You can not install the npm packet from the previous section locally, unless you have a local npm registry set up where you upload it too. This is why we have seperate scripts for making it work for local installation.
You can not install the npm packet from the previous section locally, unless you have a local npm registry set up where you upload it too. This is why we have separate scripts for making it work for local installation.
- If you just need your host platform run `python scripts/make_local_dev_version.py`
- note: this clears the `platform_package` folder
- (advanced) If you need more than one platform for local install you can just run `node scripts/update_optional_dependencies_and_version.js` after building multiple plaftorms with `build_platform_package.py`
- (advanced) If you need more than one platform for local install you can just run `node scripts/update_optional_dependencies_and_version.js` after building multiple platforms with `build_platform_package.py`
## Thanks to nlnet

View File

@@ -15,5 +15,5 @@
},
"type": "module",
"types": "index.d.ts",
"version": "1.151.2"
"version": "1.155.0"
}

View File

@@ -6,7 +6,7 @@ const expected_cwd = join(dirname(fileURLToPath(import.meta.url)), "..");
if (process.cwd() !== expected_cwd) {
console.error(
"CWD missmatch: this script needs to be run from " + expected_cwd,
"CWD mismatch: this script needs to be run from " + expected_cwd,
{ actual: process.cwd(), expected: expected_cwd }
);
process.exit(1);
@@ -40,7 +40,7 @@ const platform_package_names = await Promise.all(
"has a different version than the version of the rpc server.",
{ rpc_server: version, platform_package: p.version }
);
throw new Error("version missmatch");
throw new Error("version mismatch");
}
return { folder_name: name, package_name: p.name };
})

View File

@@ -66,7 +66,7 @@ async fn main_impl() -> Result<()> {
// Logs from `log` crate and traces from `tracing` crate
// are configurable with `RUST_LOG` environment variable
// and go to stderr to avoid interferring with JSON-RPC using stdout.
// and go to stderr to avoid interfering with JSON-RPC using stdout.
tracing_subscriber::fmt()
.with_env_filter(EnvFilter::from_default_env())
.with_writer(std::io::stderr)

View File

@@ -18,6 +18,9 @@ ignore = [
# Unmaintained instant
"RUSTSEC-2024-0384",
# idna 0.5.0
"RUSTSEC-2024-0421",
]
[bans]
@@ -37,10 +40,8 @@ skip = [
{ name = "futures-lite", version = "1.13.0" },
{ name = "getrandom", version = "<0.2" },
{ name = "http", version = "0.2.12" },
{ name = "idna", version = "0.5.0" },
{ name = "nix", version = "0.26.4" },
{ name = "num_enum_derive", version = "0.5.11" },
{ name = "num_enum", version = "0.5.11" },
{ name = "proc-macro-crate", version = "1.3.1" },
{ name = "quick-error", version = "<2.0" },
{ name = "rand_chacha", version = "<0.3" },
{ name = "rand_core", version = "<0.6" },
@@ -50,8 +51,10 @@ skip = [
{ name = "regex-syntax", version = "0.6.29" },
{ name = "sync_wrapper", version = "0.1.2" },
{ name = "syn", version = "1.0.109" },
{ name = "thiserror-impl", version = "1.0.69" },
{ name = "thiserror", version = "1.0.69" },
{ name = "time", version = "<0.3" },
{ name = "toml_edit", version = "0.19.15" },
{ name = "unicode-width", version = "0.1.11" },
{ name = "wasi", version = "<0.11" },
{ name = "windows_aarch64_gnullvm", version = "<0.52" },
{ name = "windows_aarch64_msvc", version = "<0.52" },
@@ -64,7 +67,6 @@ skip = [
{ name = "windows_x86_64_gnullvm", version = "<0.52" },
{ name = "windows_x86_64_gnu", version = "<0.52" },
{ name = "windows_x86_64_msvc", version = "<0.52" },
{ name = "winnow", version = "0.5.40" },
{ name = "winreg", version = "0.50.0" },
]
@@ -81,6 +83,7 @@ allow = [
"MIT",
"MPL-2.0",
"OpenSSL",
"Unicode-3.0",
"Unicode-DFS-2016",
"Zlib",
]
@@ -95,6 +98,5 @@ license-files = [
[sources.allow-org]
# Organisations which we allow git sources from.
github = [
"async-email",
"deltachat",
]

18
flake.lock generated
View File

@@ -47,11 +47,11 @@
"rust-analyzer-src": "rust-analyzer-src"
},
"locked": {
"lastModified": 1731393059,
"narHash": "sha256-rmzi0GHEwpzg1LGfGPO4SRD7D6QGV3UYGQxkJvn+J5U=",
"lastModified": 1737527504,
"narHash": "sha256-Z8S5gLPdIYeKwBXDaSxlJ72ZmiilYhu3418h3RSQZA0=",
"owner": "nix-community",
"repo": "fenix",
"rev": "fda8d5b59bb0dc0021ad3ba1d722f9ef6d36e4d9",
"rev": "aa13f23e3e91b95377a693ac655bbc6545ebec0d",
"type": "github"
},
"original": {
@@ -147,11 +147,11 @@
},
"nixpkgs_2": {
"locked": {
"lastModified": 1731139594,
"narHash": "sha256-IigrKK3vYRpUu+HEjPL/phrfh7Ox881er1UEsZvw9Q4=",
"lastModified": 1737469691,
"narHash": "sha256-nmKOgAU48S41dTPIXAq0AHZSehWUn6ZPrUKijHAMmIk=",
"owner": "nixos",
"repo": "nixpkgs",
"rev": "76612b17c0ce71689921ca12d9ffdc9c23ce40b2",
"rev": "9e4d5190a9482a1fb9d18adf0bdb83c6e506eaab",
"type": "github"
},
"original": {
@@ -202,11 +202,11 @@
"rust-analyzer-src": {
"flake": false,
"locked": {
"lastModified": 1731342671,
"narHash": "sha256-36eYDHoPzjavnpmEpc2MXdzMk557S0YooGms07mDuKk=",
"lastModified": 1737453499,
"narHash": "sha256-fa5AJI9mjFU2oVXqdCq2oA2pripAXbHzkUkewJRQpxA=",
"owner": "rust-lang",
"repo": "rust-analyzer",
"rev": "fc98e0657abf3ce07eed513e38274c89bbb2f8ad",
"rev": "0b68402d781955d526b80e5d479e9e47addb4075",
"type": "github"
},
"original": {

View File

@@ -88,8 +88,7 @@
cargoLock = {
lockFile = ./Cargo.lock;
outputHashes = {
"email-0.0.20" = "sha256-rV4Uzqt2Qdrfi5Ti1r+Si1c2iW1kKyWLwOgLkQ5JGGw=";
"encoded-words-0.2.0" = "sha256-KK9st0hLFh4dsrnLd6D8lC6pRFFs8W+WpZSGMGJcosk=";
"email-0.0.20" = "sha256-cfR3D5jFQpw32bGsgapK2Uwuxmht+rRK/n1ZUmCb2WA=";
"lettre-0.9.2" = "sha256-+hU1cFacyyeC9UGVBpS14BWlJjHy90i/3ynMkKAzclk=";
};
};

270
fuzz/Cargo.lock generated
View File

@@ -146,6 +146,7 @@ dependencies = [
"blake2",
"cpufeatures",
"password-hash",
"zeroize",
]
[[package]]
@@ -178,7 +179,7 @@ dependencies = [
"nom",
"num-traits",
"rusticata-macros",
"thiserror",
"thiserror 1.0.58",
"time 0.3.36",
]
@@ -190,7 +191,7 @@ checksum = "7378575ff571966e99a744addeff0bff98b8ada0dedf1956d59e634db95eaac1"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.52",
"syn 2.0.90",
"synstructure 0.13.1",
]
@@ -202,7 +203,7 @@ checksum = "7b18050c2cd6fe86c3a76584ef5e0baf286d038cda203eb6223df2cc413565f7"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.52",
"syn 2.0.90",
]
[[package]]
@@ -274,7 +275,7 @@ dependencies = [
"pin-utils",
"self_cell",
"stop-token",
"thiserror",
"thiserror 1.0.58",
"tokio",
]
@@ -285,7 +286,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9343dc5acf07e79ff82d0c37899f079db3534d99f189a1837c8e549c99405bec"
dependencies = [
"native-tls",
"thiserror",
"thiserror 1.0.58",
"tokio",
"url",
]
@@ -298,23 +299,22 @@ checksum = "3b43422f69d8ff38f95f1b2bb76517c91589a924d1559a0e935d7c8ce0274c11"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.52",
"syn 2.0.90",
]
[[package]]
name = "async-smtp"
version = "0.9.0"
version = "0.10.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8709c0d4432be428a88a06746689a9cb543e8e27ef7f61ca4d0455003a3d8c5b"
checksum = "3ee04bcf0a7ebf5594f9aff84935dc8cb0490b65055913a7a4c4d08f81e181d6"
dependencies = [
"anyhow",
"base64 0.13.1",
"futures",
"hostname",
"log",
"nom",
"pin-project",
"thiserror",
"thiserror 1.0.58",
"tokio",
]
@@ -326,7 +326,7 @@ checksum = "6e0c28dcc82d7c8ead5cb13beb15405b57b8546e93215673ff8ca0349a028107"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.52",
"syn 2.0.90",
]
[[package]]
@@ -339,7 +339,7 @@ dependencies = [
"crc32fast",
"futures-lite 2.5.0",
"pin-project",
"thiserror",
"thiserror 1.0.58",
"tokio",
"tokio-util",
]
@@ -1040,7 +1040,7 @@ checksum = "f46882e17999c6cc590af592290432be3bce0428cb0d5f8b6715e4dc7b383eb3"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.52",
"syn 2.0.90",
]
[[package]]
@@ -1108,7 +1108,7 @@ dependencies = [
"proc-macro2",
"quote",
"strsim",
"syn 2.0.52",
"syn 2.0.90",
]
[[package]]
@@ -1119,7 +1119,7 @@ checksum = "d336a2a514f6ccccaa3e09b02d41d35330c07ddf03a62165fcec10bb561c7806"
dependencies = [
"darling_core",
"quote",
"syn 2.0.52",
"syn 2.0.90",
]
[[package]]
@@ -1152,7 +1152,7 @@ dependencies = [
[[package]]
name = "deltachat"
version = "1.150.0"
version = "1.151.5"
dependencies = [
"anyhow",
"async-broadcast",
@@ -1220,7 +1220,7 @@ dependencies = [
"strum_macros",
"tagger",
"textwrap",
"thiserror",
"thiserror 1.0.58",
"tokio",
"tokio-io-timeout",
"tokio-rustls",
@@ -1263,7 +1263,7 @@ name = "deltachat_derive"
version = "2.0.0"
dependencies = [
"quote",
"syn 2.0.52",
"syn 2.0.90",
]
[[package]]
@@ -1300,7 +1300,7 @@ checksum = "8034092389675178f570469e6c3b0465d3d30b4505c294a6550db47f3c17ad18"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.52",
"syn 2.0.90",
]
[[package]]
@@ -1331,7 +1331,7 @@ dependencies = [
"darling",
"proc-macro2",
"quote",
"syn 2.0.52",
"syn 2.0.90",
]
[[package]]
@@ -1341,7 +1341,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "206868b8242f27cecce124c19fd88157fbd0dd334df2587f36417bafbc85097b"
dependencies = [
"derive_builder_core",
"syn 2.0.52",
"syn 2.0.90",
]
[[package]]
@@ -1361,7 +1361,7 @@ checksum = "cb7330aeadfbe296029522e6c40f315320aba36fc43a5b3632f3795348f3bd22"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.52",
"syn 2.0.90",
"unicode-xid",
]
@@ -1579,7 +1579,7 @@ dependencies = [
"hex",
"lazy_static",
"regex",
"thiserror",
"thiserror 1.0.58",
]
[[package]]
@@ -1670,7 +1670,7 @@ dependencies = [
"heck 0.4.1",
"proc-macro2",
"quote",
"syn 2.0.52",
"syn 2.0.90",
]
[[package]]
@@ -1690,7 +1690,7 @@ checksum = "de0d48a183585823424a4ce1aa132d174a6a81bd540895822eb4c8373a8e49e8"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.52",
"syn 2.0.90",
]
[[package]]
@@ -1814,7 +1814,7 @@ dependencies = [
"anyhow",
"async-trait",
"log",
"thiserror",
"thiserror 1.0.58",
"tokio",
"tokio-stream",
]
@@ -2061,7 +2061,7 @@ checksum = "162ee34ebcb7c64a8abebc059ce0fee27c2262618d7b60ed8faf72fef13c3650"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.52",
"syn 2.0.90",
]
[[package]]
@@ -2331,7 +2331,7 @@ dependencies = [
"ipnet",
"once_cell",
"rand 0.8.5",
"thiserror",
"thiserror 1.0.58",
"time 0.3.36",
"tinyvec",
"tokio",
@@ -2355,7 +2355,7 @@ dependencies = [
"rand 0.8.5",
"resolv-conf",
"smallvec",
"thiserror",
"thiserror 1.0.58",
"tokio",
"tracing",
]
@@ -2718,7 +2718,7 @@ dependencies = [
"rand_core 0.6.4",
"serde",
"ssh-key",
"thiserror",
"thiserror 1.0.58",
"ttl_cache",
"url",
"zeroize",
@@ -2828,7 +2828,7 @@ dependencies = [
"netlink-packet-route",
"netlink-sys",
"netwatch",
"num_enum 0.7.2",
"num_enum",
"once_cell",
"parking_lot",
"pin-project",
@@ -2848,7 +2848,7 @@ dependencies = [
"strum",
"stun-rs",
"surge-ping",
"thiserror",
"thiserror 1.0.58",
"time 0.3.36",
"tokio",
"tokio-rustls",
@@ -2880,7 +2880,7 @@ dependencies = [
"rustc-hash 2.0.0",
"rustls",
"socket2 0.5.6",
"thiserror",
"thiserror 1.0.58",
"tokio",
"tracing",
]
@@ -2898,7 +2898,7 @@ dependencies = [
"rustls",
"rustls-platform-verifier",
"slab",
"thiserror",
"thiserror 1.0.58",
"tinyvec",
"tracing",
]
@@ -2954,7 +2954,7 @@ dependencies = [
"combine",
"jni-sys",
"log",
"thiserror",
"thiserror 1.0.58",
"walkdir",
]
@@ -3186,7 +3186,7 @@ dependencies = [
"serde_bencode",
"serde_bytes",
"sha1_smol",
"thiserror",
"thiserror 1.0.58",
"tracing",
]
@@ -3349,7 +3349,7 @@ dependencies = [
"anyhow",
"byteorder",
"paste",
"thiserror",
"thiserror 1.0.58",
]
[[package]]
@@ -3363,7 +3363,7 @@ dependencies = [
"log",
"netlink-packet-core",
"netlink-sys",
"thiserror",
"thiserror 1.0.58",
"tokio",
]
@@ -3401,7 +3401,7 @@ dependencies = [
"rtnetlink",
"serde",
"socket2 0.5.6",
"thiserror",
"thiserror 1.0.58",
"time 0.3.36",
"tokio",
"tracing",
@@ -3500,7 +3500,7 @@ checksum = "9e6a0fd4f737c707bd9086cc16c925f294943eb62eb71499e9fd4cf71f8b9f4e"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.52",
"syn 2.0.90",
]
[[package]]
@@ -3543,34 +3543,13 @@ dependencies = [
"libc",
]
[[package]]
name = "num_enum"
version = "0.5.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1f646caf906c20226733ed5b1374287eb97e3c2a5c227ce668c1f2ce20ae57c9"
dependencies = [
"num_enum_derive 0.5.11",
]
[[package]]
name = "num_enum"
version = "0.7.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "02339744ee7253741199f897151b38e72257d13802d4ee837285cc2990a90845"
dependencies = [
"num_enum_derive 0.7.2",
]
[[package]]
name = "num_enum_derive"
version = "0.5.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "dcbff9bc912032c62bf65ef1d5aea88983b420f4f839db1e9b0c281a25c9c799"
dependencies = [
"proc-macro-crate 1.3.1",
"proc-macro2",
"quote",
"syn 1.0.107",
"num_enum_derive",
]
[[package]]
@@ -3579,10 +3558,10 @@ version = "0.7.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "681030a937600a36906c185595136d26abfebb4aa9c65701cefcaf8578bb982b"
dependencies = [
"proc-macro-crate 3.1.0",
"proc-macro-crate",
"proc-macro2",
"quote",
"syn 2.0.52",
"syn 2.0.90",
]
[[package]]
@@ -3813,7 +3792,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cd53dff83f26735fdc1ca837098ccf133605d794cdae66acfc2bfac3ec809d95"
dependencies = [
"memchr",
"thiserror",
"thiserror 1.0.58",
"ucd-trie",
]
@@ -3837,7 +3816,7 @@ dependencies = [
"pest_meta",
"proc-macro2",
"quote",
"syn 2.0.52",
"syn 2.0.90",
]
[[package]]
@@ -3853,15 +3832,15 @@ dependencies = [
[[package]]
name = "pgp"
version = "0.14.0"
version = "0.14.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "49bb5f77aaf8ae1ed6fe63387ad513b10cd44716fd053ecc227b9493c096cdb2"
checksum = "1877a97fd422433220ad272eb008ec55691944b1200e9eb204e3cb2cb69d34e9"
dependencies = [
"aes",
"aes-gcm",
"aes-kw",
"argon2",
"base64 0.21.7",
"base64 0.22.1",
"bitfield",
"block-padding",
"blowfish",
@@ -3897,7 +3876,7 @@ dependencies = [
"nom",
"num-bigint-dig",
"num-traits",
"num_enum 0.5.11",
"num_enum",
"ocb3",
"p256",
"p384",
@@ -3911,7 +3890,7 @@ dependencies = [
"sha3",
"signature",
"smallvec",
"thiserror",
"thiserror 1.0.58",
"twofish",
"x25519-dalek",
"x448",
@@ -3935,7 +3914,7 @@ checksum = "3c0f5fad0874fc7abcd4d750e76917eaebbecaa2c20bde22e1dbeeba8beb758c"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.52",
"syn 2.0.90",
]
[[package]]
@@ -3967,7 +3946,7 @@ dependencies = [
"mainline",
"self_cell",
"simple-dns",
"thiserror",
"thiserror 1.0.58",
"tracing",
"ureq",
"wasm-bindgen",
@@ -4021,7 +4000,7 @@ dependencies = [
"proc-macro2",
"quote",
"regex",
"syn 2.0.52",
"syn 2.0.90",
]
[[package]]
@@ -4102,12 +4081,12 @@ dependencies = [
"iroh-metrics",
"libc",
"netwatch",
"num_enum 0.7.2",
"num_enum",
"rand 0.8.5",
"serde",
"smallvec",
"socket2 0.5.6",
"thiserror",
"thiserror 1.0.58",
"time 0.3.36",
"tokio",
"tokio-util",
@@ -4199,23 +4178,13 @@ dependencies = [
"elliptic-curve",
]
[[package]]
name = "proc-macro-crate"
version = "1.3.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7f4c021e1093a56626774e81216a4ce732a735e5bad4868a03f3ed65ca0c3919"
dependencies = [
"once_cell",
"toml_edit 0.19.15",
]
[[package]]
name = "proc-macro-crate"
version = "3.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6d37c51ca738a55da99dc0c4a34860fd675453b8b36209178c2249bb13651284"
dependencies = [
"toml_edit 0.21.0",
"toml_edit",
]
[[package]]
@@ -4252,9 +4221,9 @@ checksum = "dc375e1527247fe1a97d8b7156678dfe7c1af2fc075c9a4db3690ecd2a148068"
[[package]]
name = "proc-macro2"
version = "1.0.78"
version = "1.0.92"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e2422ad645d89c99f8f3e6b88a9fdeca7fabeac836b1002371c4367c8f984aae"
checksum = "37d3544b3f2748c54e147655edb5025752e2303145b5aefb3c3ea2c78b973bb0"
dependencies = [
"unicode-ident",
]
@@ -4279,7 +4248,7 @@ checksum = "440f724eba9f6996b75d63681b0a92b06947f1457076d503a4d2e2c8f56442b8"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.52",
"syn 2.0.90",
]
[[package]]
@@ -4336,26 +4305,29 @@ dependencies = [
"quinn-udp",
"rustc-hash 1.1.0",
"rustls",
"thiserror",
"thiserror 1.0.58",
"tokio",
"tracing",
]
[[package]]
name = "quinn-proto"
version = "0.11.3"
version = "0.11.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ddf517c03a109db8100448a4be38d498df8a210a99fe0e1b9eaf39e78c640efe"
checksum = "a2fe5ef3495d7d2e377ff17b1a8ce2ee2ec2a18cde8b6ad6619d65d0701c135d"
dependencies = [
"bytes",
"getrandom 0.2.11",
"rand 0.8.5",
"ring",
"rustc-hash 1.1.0",
"rustc-hash 2.0.0",
"rustls",
"rustls-pki-types",
"slab",
"thiserror",
"thiserror 2.0.6",
"tinyvec",
"tracing",
"web-time",
]
[[package]]
@@ -4668,22 +4640,21 @@ dependencies = [
[[package]]
name = "rsa"
version = "0.9.1"
version = "0.9.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "72f1471dbb4be5de45050e8ef7040625298ccb9efe941419ac2697088715925f"
checksum = "47c75d7c5c6b673e58bf54d8544a9f432e3a925b0e80f7cd3602ab5c50c55519"
dependencies = [
"byteorder",
"const-oid",
"digest",
"num-bigint-dig",
"num-integer",
"num-iter",
"num-traits",
"pkcs1",
"pkcs8",
"rand_core 0.6.4",
"sha2",
"signature",
"spki",
"subtle",
"zeroize",
]
@@ -4702,7 +4673,7 @@ dependencies = [
"netlink-proto",
"netlink-sys",
"nix",
"thiserror",
"thiserror 1.0.58",
"tokio",
]
@@ -4791,9 +4762,9 @@ dependencies = [
[[package]]
name = "rustls"
version = "0.23.18"
version = "0.23.19"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9c9cc1d47e243d655ace55ed38201c19ae02c148ae56412ab8750e8f0166ab7f"
checksum = "934b404430bb06b3fae2cba809eb45a1ab1aecd64491213d7c3301b88393f8d1"
dependencies = [
"log",
"once_cell",
@@ -4832,6 +4803,9 @@ name = "rustls-pki-types"
version = "1.10.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "16f1201b3c9a7ee8039bcadc17b7e605e2945b27eee7631788c1bd2b0643674b"
dependencies = [
"web-time",
]
[[package]]
name = "rustls-platform-verifier"
@@ -5050,7 +5024,7 @@ checksum = "e0cd7e117be63d3c3678776753929474f3b04a43a080c744d6b0ae2a8c28e222"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.52",
"syn 2.0.90",
]
[[package]]
@@ -5125,6 +5099,7 @@ checksum = "89f599ac0c323ebb1c6082821a54962b839832b03984598375bff3975b804423"
dependencies = [
"digest",
"sha1",
"zeroize",
]
[[package]]
@@ -5182,7 +5157,7 @@ dependencies = [
"shadowsocks-crypto",
"socket2 0.5.6",
"spin 0.9.8",
"thiserror",
"thiserror 1.0.58",
"tokio",
"tokio-tfo",
"url",
@@ -5328,9 +5303,9 @@ dependencies = [
[[package]]
name = "spki"
version = "0.7.1"
version = "0.7.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "37a5be806ab6f127c3da44b7378837ebf01dadca8510a0e572460216b228bd0e"
checksum = "d91ed6c858b01f942cd56b37a94b3e0a1798290327d1236e4d9cf4eaca44d29d"
dependencies = [
"base64ct",
"der",
@@ -5415,7 +5390,7 @@ dependencies = [
"proc-macro2",
"quote",
"struct_iterable_internal",
"syn 2.0.52",
"syn 2.0.90",
]
[[package]]
@@ -5443,7 +5418,7 @@ dependencies = [
"proc-macro2",
"quote",
"rustversion",
"syn 2.0.52",
"syn 2.0.90",
]
[[package]]
@@ -5487,7 +5462,7 @@ dependencies = [
"pnet_packet",
"rand 0.8.5",
"socket2 0.5.6",
"thiserror",
"thiserror 1.0.58",
"tokio",
"tracing",
]
@@ -5505,9 +5480,9 @@ dependencies = [
[[package]]
name = "syn"
version = "2.0.52"
version = "2.0.90"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b699d15b36d1f02c3e7c69f8ffef53de37aefae075d8488d4ba1a7788d574a07"
checksum = "919d3b74a5dd0ccd15aeb8f93e7006bd9e14c295087c9896a110f490752bcf31"
dependencies = [
"proc-macro2",
"quote",
@@ -5551,7 +5526,7 @@ checksum = "c8af7666ab7b6390ab78131fb5b0fce11d6b7a6951602017c35fa82800708971"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.52",
"syn 2.0.90",
]
[[package]]
@@ -5626,7 +5601,16 @@ version = "1.0.58"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "03468839009160513471e86a034bb2c5c0e4baae3b43f79ffc55c4a5427b3297"
dependencies = [
"thiserror-impl",
"thiserror-impl 1.0.58",
]
[[package]]
name = "thiserror"
version = "2.0.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8fec2a1820ebd077e2b90c4df007bebf344cd394098a13c563957d0afc83ea47"
dependencies = [
"thiserror-impl 2.0.6",
]
[[package]]
@@ -5637,7 +5621,18 @@ checksum = "c61f3ba182994efc43764a46c018c347bc492c79f024e705f46567b418f6d4f7"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.52",
"syn 2.0.90",
]
[[package]]
name = "thiserror-impl"
version = "2.0.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d65750cab40f4ff1929fb1ba509e9914eb756131cef4210da8d5d700d26f6312"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.90",
]
[[package]]
@@ -5743,7 +5738,7 @@ checksum = "693d596312e88961bc67d7f1f97af8a70227d9f90c31bba5806eec004978d752"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.52",
"syn 2.0.90",
]
[[package]]
@@ -5823,7 +5818,7 @@ dependencies = [
"http 1.1.0",
"httparse",
"js-sys",
"thiserror",
"thiserror 1.0.58",
"tokio",
"tokio-tungstenite",
"wasm-bindgen",
@@ -5855,7 +5850,7 @@ dependencies = [
"serde",
"serde_spanned",
"toml_datetime",
"toml_edit 0.21.0",
"toml_edit",
]
[[package]]
@@ -5867,17 +5862,6 @@ dependencies = [
"serde",
]
[[package]]
name = "toml_edit"
version = "0.19.15"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1b5bb770da30e5cbfde35a2d7b9b8a2c4b8ef89548a7a6aeab5c9a576e3e7421"
dependencies = [
"indexmap",
"toml_datetime",
"winnow",
]
[[package]]
name = "toml_edit"
version = "0.21.0"
@@ -5917,7 +5901,7 @@ checksum = "34704c8d6ebcbc939824180af020566b01a7c01f80641264eba0999f6c2b6be7"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.52",
"syn 2.0.90",
]
[[package]]
@@ -5988,7 +5972,7 @@ dependencies = [
"log",
"rand 0.8.5",
"sha1",
"thiserror",
"thiserror 1.0.58",
"url",
"utf-8",
]
@@ -6217,7 +6201,7 @@ dependencies = [
"once_cell",
"proc-macro2",
"quote",
"syn 2.0.52",
"syn 2.0.90",
"wasm-bindgen-shared",
]
@@ -6251,7 +6235,7 @@ checksum = "e94f17b526d0a461a191c78ea52bbce64071ed5c04c9ffe424dcb38f74171bb7"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.52",
"syn 2.0.90",
"wasm-bindgen-backend",
"wasm-bindgen-shared",
]
@@ -6271,7 +6255,7 @@ dependencies = [
"event-listener 4.0.3",
"futures-util",
"parking_lot",
"thiserror",
"thiserror 1.0.58",
]
[[package]]
@@ -6284,6 +6268,16 @@ dependencies = [
"wasm-bindgen",
]
[[package]]
name = "web-time"
version = "1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5a6580f308b1fad9207618087a65c04e7a10bc77e02c8e84e9b00dd4b12fa0bb"
dependencies = [
"js-sys",
"wasm-bindgen",
]
[[package]]
name = "webpki-roots"
version = "0.26.7"
@@ -6393,7 +6387,7 @@ checksum = "12168c33176773b86799be25e2a2ba07c7aab9968b37541f1094dbd7a60c8946"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.52",
"syn 2.0.90",
]
[[package]]
@@ -6404,7 +6398,7 @@ checksum = "9d8dc32e0095a7eeccebd0e3f09e9509365ecb3fc6ac4d6f5f14a3f6392942d1"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.52",
"syn 2.0.90",
]
[[package]]
@@ -6693,7 +6687,7 @@ dependencies = [
"futures",
"log",
"serde",
"thiserror",
"thiserror 1.0.58",
"windows 0.52.0",
]
@@ -6742,7 +6736,7 @@ dependencies = [
"nom",
"oid-registry",
"rusticata-macros",
"thiserror",
"thiserror 1.0.58",
"time 0.3.36",
]
@@ -6802,7 +6796,7 @@ checksum = "9ce1b18ccd8e73a9321186f97e46f9f04b778851177567b1975109d26a08d2a6"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.52",
"syn 2.0.90",
]
[[package]]

View File

@@ -299,10 +299,6 @@ export class Message {
return Boolean(binding.dcn_msg_is_forwarded(this.dc_msg))
}
isIncreation() {
return Boolean(binding.dcn_msg_is_increation(this.dc_msg))
}
isInfo() {
return Boolean(binding.dcn_msg_is_info(this.dc_msg))
}

View File

@@ -9,7 +9,7 @@ const buildArgs = [
'build',
'--release',
'--features',
'vendored,jsonrpc',
'vendored',
'-p',
'deltachat_ffi'
]

View File

@@ -2374,17 +2374,6 @@ NAPI_METHOD(dcn_msg_is_forwarded) {
NAPI_RETURN_INT32(is_forwarded);
}
NAPI_METHOD(dcn_msg_is_increation) {
NAPI_ARGV(1);
NAPI_DC_MSG();
//TRACE("calling..");
int is_increation = dc_msg_is_increation(dc_msg);
//TRACE("result %d", is_increation);
NAPI_RETURN_INT32(is_increation);
}
NAPI_METHOD(dcn_msg_is_info) {
NAPI_ARGV(1);
NAPI_DC_MSG();
@@ -3555,7 +3544,6 @@ NAPI_INIT() {
NAPI_EXPORT_FUNCTION(dcn_msg_has_location);
NAPI_EXPORT_FUNCTION(dcn_msg_has_html);
NAPI_EXPORT_FUNCTION(dcn_msg_is_forwarded);
NAPI_EXPORT_FUNCTION(dcn_msg_is_increation);
NAPI_EXPORT_FUNCTION(dcn_msg_is_info);
NAPI_EXPORT_FUNCTION(dcn_msg_is_sent);
NAPI_EXPORT_FUNCTION(dcn_msg_is_setupmessage);

View File

@@ -536,7 +536,6 @@ describe('Offline Tests with unconfigured account', function () {
strictEqual(msg.getWidth(), 0, 'no message width')
strictEqual(msg.isDeadDrop(), false, 'not deaddrop')
strictEqual(msg.isForwarded(), false, 'not forwarded')
strictEqual(msg.isIncreation(), false, 'not in creation')
strictEqual(msg.isInfo(), false, 'not an info message')
strictEqual(msg.isSent(), false, 'messge is not sent')
strictEqual(msg.isSetupmessage(), false, 'not an autocrypt setup message')

View File

@@ -55,5 +55,5 @@
"test:mocha": "mocha node/test/test.mjs --growl --reporter=spec --bail --exit"
},
"types": "node/dist/index.d.ts",
"version": "1.151.2"
"version": "1.155.0"
}

View File

@@ -52,10 +52,7 @@ python3-venv` should give you a usable python installation.
First, build the core library::
cargo build --release -p deltachat_ffi --features jsonrpc
`jsonrpc` feature is required even if not used by the bindings
because `deltachat.h` includes JSON-RPC functions unconditionally.
cargo build --release -p deltachat_ffi
Create the virtual environment and activate it::

View File

@@ -4,10 +4,10 @@ build-backend = "setuptools.build_meta"
[project]
name = "deltachat"
version = "1.151.2"
version = "1.155.0"
description = "Python bindings for the Delta Chat Core library using CFFI against the Rust-implemented libdeltachat"
readme = "README.rst"
requires-python = ">=3.7"
requires-python = ">=3.8"
authors = [
{ name = "holger krekel, Floris Bruynooghe, Bjoern Petersen and contributors" },
]

View File

@@ -671,9 +671,6 @@ class Account:
def get_connectivity_html(self) -> str:
return from_dc_charpointer(lib.dc_get_connectivity_html(self._dc_context))
def all_work_done(self):
return lib.dc_all_work_done(self._dc_context)
def start_io(self):
"""start this account's IO scheduling (Rust-core async scheduler).

View File

@@ -271,8 +271,7 @@ class Chat:
:param msg: a :class:`deltachat.message.Message` instance
previously returned by
e.g. :meth:`deltachat.message.Message.new_empty` or
:meth:`prepare_file`.
e.g. :meth:`deltachat.message.Message.new_empty`.
:raises ValueError: if message can not be sent.
:returns: a :class:`deltachat.message.Message` instance as
@@ -341,37 +340,6 @@ class Chat:
raise ValueError("message could not be sent")
return Message.from_db(self.account, sent_id)
def prepare_message(self, msg):
"""prepare a message for sending.
:param msg: the message to be prepared.
:returns: a :class:`deltachat.message.Message` instance.
This is the same object that was passed in, which
has been modified with the new state of the core.
"""
msg_id = lib.dc_prepare_msg(self.account._dc_context, self.id, msg._dc_msg)
if msg_id == 0:
raise ValueError("message could not be prepared")
# modify message in place to avoid bad state for the caller
msg._dc_msg = Message.from_db(self.account, msg_id)._dc_msg
return msg
def prepare_message_file(self, path, mime_type=None, view_type="file"):
"""prepare a message for sending and return the resulting Message instance.
To actually send the message, call :meth:`send_prepared`.
The file must be inside the blob directory.
:param path: path to the file.
:param mime_type: the mime-type of this file, defaults to auto-detection.
:param view_type: "text", "image", "gif", "audio", "video", "file"
:raises ValueError: if message can not be prepared/chat does not exist.
:returns: the resulting :class:`Message` instance
"""
msg = Message.new_empty(self.account, view_type)
msg.set_file(path, mime_type)
return self.prepare_message(msg)
def send_prepared(self, message):
"""send a previously prepared message.

View File

@@ -158,12 +158,6 @@ class FFIEventTracker:
self.get_matching("DC_EVENT_CONNECTIVITY_CHANGED")
def wait_for_all_work_done(self):
while True:
if self.account.all_work_done():
return
self.get_matching("DC_EVENT_CONNECTIVITY_CHANGED")
def ensure_event_not_queued(self, event_name_regex):
__tracebackhide__ = True
rex = re.compile(f"(?:{event_name_regex}).*")

View File

@@ -108,7 +108,9 @@ class Message:
@props.with_doc
def filename(self):
"""filename if there was an attachment, otherwise empty string."""
"""file path if there was an attachment, otherwise empty string.
If you want to get the file extension or a user-visible string,
use `basename` instead."""
return from_dc_charpointer(lib.dc_msg_get_file(self._dc_msg))
def set_file(self, path, mime_type=None):
@@ -120,7 +122,8 @@ class Message:
@props.with_doc
def basename(self) -> str:
"""basename of the attachment if it exists, otherwise empty string."""
"""The user-visible name of the attachment (incl. extension)
if it exists, otherwise empty string."""
# FIXME, it does not return basename
return from_dc_charpointer(lib.dc_msg_get_filename(self._dc_msg))

View File

@@ -152,7 +152,7 @@ class TestProcess:
def get_liveconfig_producer(self):
"""provide live account configs, cached on a per-test-process scope
so that test functions can re-use already known live configs.
so that test functions can reuse already known live configs.
"""
chatmail_opt = self.pytestconfig.getoption("--chatmail")
if chatmail_opt:

View File

@@ -181,15 +181,16 @@ def test_send_file_twice_unicode_filename_mangling(tmp_path, acfactory, lp):
msg = send_and_receive_message()
assert msg.text == "withfile"
assert open(msg.filename).read() == "some data"
msg.filename.index(basename)
assert msg.filename.endswith(ext)
msg.basename.index(basename)
assert msg.basename.endswith(ext)
msg2 = send_and_receive_message()
assert msg2.text == "withfile"
assert open(msg2.filename).read() == "some data"
msg2.filename.index(basename)
assert msg2.filename.endswith(ext)
assert msg.filename != msg2.filename
msg2.basename.index(basename)
assert msg2.basename.endswith(ext)
assert msg.filename == msg2.filename # The file is deduplicated
assert msg.basename == msg2.basename
def test_send_file_html_attachment(tmp_path, acfactory, lp):
@@ -214,8 +215,8 @@ def test_send_file_html_attachment(tmp_path, acfactory, lp):
msg = ac2.get_message_by_id(ev.data2)
assert open(msg.filename).read() == content
msg.filename.index(basename)
assert msg.filename.endswith(ext)
msg.basename.index(basename)
assert msg.basename.endswith(ext)
def test_html_message(acfactory, lp):
@@ -1253,7 +1254,10 @@ def test_no_old_msg_is_fresh(acfactory, lp):
def test_prefer_encrypt(acfactory, lp):
"""Test quorum rule for encryption preference in 1:1 and group chat."""
ac1, ac2, ac3 = acfactory.get_online_accounts(3)
ac1 = acfactory.new_online_configuring_account(fix_is_chatmail=True)
ac2 = acfactory.new_online_configuring_account(fix_is_chatmail=True)
ac3 = acfactory.new_online_configuring_account(fix_is_chatmail=True)
acfactory.bring_accounts_online()
ac1.set_config("e2ee_enabled", "0")
ac2.set_config("e2ee_enabled", "1")
ac3.set_config("e2ee_enabled", "0")
@@ -1276,7 +1280,8 @@ def test_prefer_encrypt(acfactory, lp):
lp.sec("ac2: sending message to ac1")
chat2 = ac2.create_chat(ac1)
msg2 = chat2.send_text("message2")
assert not msg2.is_encrypted()
# Own preference is `Mutual` and we have the peer's key.
assert msg2.is_encrypted()
ac1._evtracker.wait_next_incoming_message()
lp.sec("ac1: sending message to group chat with ac2 and ac3")
@@ -1292,8 +1297,8 @@ def test_prefer_encrypt(acfactory, lp):
ac3.set_config("e2ee_enabled", "1")
chat3 = ac3.create_chat(ac1)
msg4 = chat3.send_text("message4")
# ac1 still does not prefer encryption
assert not msg4.is_encrypted()
# Own preference is `Mutual` and we have the peer's key.
assert msg4.is_encrypted()
ac1._evtracker.wait_next_incoming_message()
lp.sec("ac1: sending another message to group chat with ac2 and ac3")
@@ -1366,10 +1371,9 @@ def test_quote_encrypted(acfactory, lp):
msg_draft.quote = quoted_msg
chat.set_draft(msg_draft)
# Get the draft, prepare and send it.
# Get the draft and send it.
msg_draft = chat.get_draft()
msg_out = chat.prepare_message(msg_draft)
chat.send_prepared(msg_out)
chat.send_msg(msg_draft)
chat.set_draft(None)
assert chat.get_draft() is None
@@ -1899,10 +1903,11 @@ def test_connectivity(acfactory, lp):
ac1.start_io()
ac1._evtracker.wait_for_connectivity(dc.const.DC_CONNECTIVITY_CONNECTING)
ac1._evtracker.wait_for_connectivity_change(dc.const.DC_CONNECTIVITY_CONNECTING, dc.const.DC_CONNECTIVITY_CONNECTED)
ac1._evtracker.wait_for_connectivity_change(dc.const.DC_CONNECTIVITY_CONNECTING, dc.const.DC_CONNECTIVITY_WORKING)
ac1._evtracker.wait_for_connectivity_change(dc.const.DC_CONNECTIVITY_WORKING, dc.const.DC_CONNECTIVITY_CONNECTED)
lp.sec(
"Test that after calling start_io(), maybe_network() and waiting for `all_work_done()`, "
"Test that after calling start_io(), maybe_network() and waiting for `DC_CONNECTIVITY_CONNECTED`, "
"all messages are fetched",
)
@@ -1911,7 +1916,7 @@ def test_connectivity(acfactory, lp):
ac2.create_chat(ac1).send_text("Hi")
idle1.wait_for_new_message()
ac1.maybe_network()
ac1._evtracker.wait_for_all_work_done()
ac1._evtracker.wait_for_connectivity(dc.const.DC_CONNECTIVITY_CONNECTED)
msgs = ac1.create_chat(ac2).get_messages()
assert len(msgs) == 1
assert msgs[0].text == "Hi"
@@ -1927,30 +1932,6 @@ def test_connectivity(acfactory, lp):
assert len(msgs) == 2
assert msgs[1].text == "Hi 2"
lp.sec("Test that the connectivity doesn't flicker to WORKING if there are no new messages")
ac1.maybe_network()
while 1:
assert ac1.get_connectivity() == dc.const.DC_CONNECTIVITY_CONNECTED
if ac1.all_work_done():
break
ac1._evtracker.get_matching("DC_EVENT_CONNECTIVITY_CHANGED")
lp.sec("Test that the connectivity doesn't flicker to WORKING if the sender of the message is blocked")
ac1.create_contact(ac2).block()
ac1.direct_imap.select_config_folder("inbox")
with ac1.direct_imap.idle() as idle1:
ac2.create_chat(ac1).send_text("Hi")
idle1.wait_for_new_message()
ac1.maybe_network()
while 1:
assert ac1.get_connectivity() == dc.const.DC_CONNECTIVITY_CONNECTED
if ac1.all_work_done():
break
ac1._evtracker.get_matching("DC_EVENT_CONNECTIVITY_CHANGED")
lp.sec("Test that the connectivity is NOT_CONNECTED if the password is wrong")
ac1.set_config("configured_mail_pw", "abc")
@@ -1961,32 +1942,6 @@ def test_connectivity(acfactory, lp):
ac1._evtracker.wait_for_connectivity(dc.const.DC_CONNECTIVITY_NOT_CONNECTED)
def test_all_work_done(acfactory, lp):
"""
Tests that calling start_io() immediately followed by maybe_network()
and then waiting for all_work_done() reliably fetches the messages
delivered while account was offline.
In other words, connectivity should not change to a state
where all_work_done() returns true until IMAP connection goes idle.
"""
ac1, ac2 = acfactory.get_online_accounts(2)
ac1.stop_io()
ac1._evtracker.wait_for_connectivity(dc.const.DC_CONNECTIVITY_NOT_CONNECTED)
ac1.direct_imap.select_config_folder("inbox")
with ac1.direct_imap.idle() as idle1:
ac2.create_chat(ac1).send_text("Hi")
idle1.wait_for_new_message()
ac1.start_io()
ac1.maybe_network()
ac1._evtracker.wait_for_all_work_done()
msgs = ac1.create_chat(ac2).get_messages()
assert len(msgs) == 1
assert msgs[0].text == "Hi"
def test_fetch_deleted_msg(acfactory, lp):
"""This is a regression test: Messages with \\Deleted flag were downloaded again and again,
hundreds of times, because uid_next was not updated.
@@ -2340,9 +2295,8 @@ def test_group_quote(acfactory, lp):
reply_msg = Message.new_empty(msg.chat.account, "text")
reply_msg.set_text("reply")
reply_msg.quote = msg
reply_msg = msg.chat.prepare_message(reply_msg)
assert reply_msg.quoted_text == "hello"
msg.chat.send_prepared(reply_msg)
msg.chat.send_msg(reply_msg)
lp.sec("ac3: receiving reply")
received_reply = ac3._evtracker.wait_next_incoming_message()

View File

@@ -1,107 +0,0 @@
import os.path
import shutil
from filecmp import cmp
import pytest
def wait_msg_delivered(account, msg_list):
"""wait for one or more MSG_DELIVERED events to match msg_list contents."""
msg_list = list(msg_list)
while msg_list:
ev = account._evtracker.get_matching("DC_EVENT_MSG_DELIVERED")
msg_list.remove((ev.data1, ev.data2))
def wait_msgs_changed(account, msgs_list):
"""wait for one or more MSGS_CHANGED events to match msgs_list contents."""
account.log(f"waiting for msgs_list={msgs_list}")
msgs_list = list(msgs_list)
while msgs_list:
ev = account._evtracker.get_matching("DC_EVENT_MSGS_CHANGED")
for i, (data1, data2) in enumerate(msgs_list):
if ev.data1 == data1:
if data2 is None or ev.data2 == data2:
del msgs_list[i]
break
else:
account.log(f"waiting mismatch data1={data1} data2={data2}")
return ev.data2
class TestOnlineInCreation:
def test_increation_not_blobdir(self, tmp_path, acfactory, lp):
ac1, ac2 = acfactory.get_online_accounts(2)
chat = ac1.create_chat(ac2)
lp.sec("Creating in-creation file outside of blobdir")
assert str(tmp_path) != ac1.get_blobdir()
src = tmp_path / "file.txt"
src.touch()
with pytest.raises(Exception):
chat.prepare_message_file(str(src))
def test_no_increation_copies_to_blobdir(self, tmp_path, acfactory, lp):
ac1, ac2 = acfactory.get_online_accounts(2)
chat = ac1.create_chat(ac2)
lp.sec("Creating file outside of blobdir")
assert str(tmp_path) != ac1.get_blobdir()
src = tmp_path / "file.txt"
src.write_text("hello there\n")
msg = chat.send_file(str(src))
assert msg.filename.startswith(os.path.join(ac1.get_blobdir(), "file"))
assert msg.filename.endswith(".txt")
def test_forward_increation(self, acfactory, data, lp):
ac1, ac2 = acfactory.get_online_accounts(2)
chat = ac1.create_chat(ac2)
wait_msgs_changed(ac1, [(0, 0)]) # why no chat id?
lp.sec("create a message with a file in creation")
orig = data.get_path("d.png")
path = os.path.join(ac1.get_blobdir(), "d.png")
with open(path, "x") as fp:
fp.write("preparing")
prepared_original = chat.prepare_message_file(path)
assert prepared_original.is_out_preparing()
wait_msgs_changed(ac1, [(chat.id, prepared_original.id)])
lp.sec("create a new group")
chat2 = ac1.create_group_chat("newgroup")
wait_msgs_changed(ac1, [(0, 0)])
lp.sec("add a contact to new group")
chat2.add_contact(ac2)
wait_msgs_changed(ac1, [(chat2.id, None)])
lp.sec("forward the message while still in creation")
ac1.forward_messages([prepared_original], chat2)
forwarded_id = wait_msgs_changed(ac1, [(chat2.id, None)])
forwarded_msg = ac1.get_message_by_id(forwarded_id)
assert forwarded_msg.is_out_preparing()
lp.sec("finish creating the file and send it")
assert prepared_original.is_out_preparing()
shutil.copyfile(orig, path)
chat.send_prepared(prepared_original)
assert prepared_original.is_out_pending() or prepared_original.is_out_delivered()
lp.sec("check that both forwarded and original message are proper.")
wait_msgs_changed(ac1, [(chat2.id, forwarded_id), (chat.id, prepared_original.id)])
fwd_msg = ac1.get_message_by_id(forwarded_id)
assert fwd_msg.is_out_pending() or fwd_msg.is_out_delivered()
lp.sec("wait for both messages to be delivered to SMTP")
wait_msg_delivered(ac1, [(chat2.id, forwarded_id), (chat.id, prepared_original.id)])
lp.sec("wait1 for original or forwarded messages to arrive")
received_original = ac2._evtracker.wait_next_incoming_message()
assert cmp(received_original.filename, orig, shallow=False)
lp.sec("wait2 for original or forwarded messages to arrive")
received_copy = ac2._evtracker.wait_next_incoming_message()
assert received_copy.id != received_original.id
assert cmp(received_copy.filename, orig, shallow=False)

View File

@@ -378,30 +378,6 @@ class TestOfflineChat:
with pytest.raises(ValueError):
chat1.send_text("msg1")
def test_prepare_message_and_send(self, ac1, chat1):
msg = chat1.prepare_message(Message.new_empty(chat1.account, "text"))
msg.set_text("hello world")
assert msg.text == "hello world"
assert msg.id > 0
chat1.send_prepared(msg)
assert "Sent" in msg.get_message_info()
str(msg)
repr(msg)
assert msg == ac1.get_message_by_id(msg.id)
def test_prepare_file(self, ac1, chat1):
blobdir = ac1.get_blobdir()
p = os.path.join(blobdir, "somedata.txt")
with open(p, "w") as f:
f.write("some data")
message = chat1.prepare_message_file(p)
assert message.id > 0
message.set_text("hello world")
assert message.is_out_preparing()
assert message.text == "hello world"
chat1.send_prepared(message)
assert "Sent" in message.get_message_info()
def test_message_eq_contains(self, chat1):
msg = chat1.send_text("msg1")
msg2 = None
@@ -691,8 +667,7 @@ class TestOfflineChat:
assert os.path.exists(messages[1].filename)
def test_set_get_draft(self, chat1):
msg = Message.new_empty(chat1.account, "text")
msg1 = chat1.prepare_message(msg)
msg1 = Message.new_empty(chat1.account, "text")
msg1.set_text("hello")
chat1.set_draft(msg1)
msg1.set_text("obsolete")
@@ -711,21 +686,6 @@ class TestOfflineChat:
assert not res.is_ask_verifygroup()
assert res.contact_id == 10
def test_quote(self, chat1):
"""Offline quoting test"""
msg = Message.new_empty(chat1.account, "text")
msg.set_text("Multi\nline\nmessage")
assert msg.quoted_text is None
# Prepare message to assign it a Message-Id.
# Messages without Message-Id cannot be quoted.
msg = chat1.prepare_message(msg)
reply_msg = Message.new_empty(chat1.account, "text")
reply_msg.set_text("reply")
reply_msg.quote = msg
assert reply_msg.quoted_text == "Multi\nline\nmessage"
def test_group_chat_many_members_add_remove(self, ac1, lp):
lp.sec("ac1: creating group chat with 10 other members")
chat = ac1.create_group_chat(name="title1")

View File

@@ -212,8 +212,13 @@ def test_logged_ac_process_ffi_failure(acfactory):
0 / 0
cap = Queue()
ac1.log = cap.put
# Make sure the next attempt to log an event fails.
ac1.add_account_plugin(FailPlugin())
# Start capturing events.
ac1.log = cap.put
# cause any event eg contact added/changed
ac1.create_contact("something@example.org")
res = cap.get(timeout=10)

View File

@@ -1 +1 @@
2024-11-26
2025-01-23

View File

@@ -7,7 +7,7 @@ set -euo pipefail
#
# Avoid using rustup here as it depends on reading /proc/self/exe and
# has problems running under QEMU.
RUST_VERSION=1.83.0
RUST_VERSION=1.84.0
ARCH="$(uname -m)"
test -f "/lib/libc.musl-$ARCH.so.1" && LIBC=musl || LIBC=gnu

View File

@@ -11,7 +11,7 @@ set -euo pipefail
export DCC_RS_TARGET=debug
export DCC_RS_DEV="$PWD"
cargo build -p deltachat_ffi --features jsonrpc
cargo build -p deltachat_ffi
tox -c python -e py --devenv venv
venv/bin/pip install --upgrade pip

View File

@@ -12,7 +12,7 @@ export DCC_RS_DEV=`pwd`
cd python
cargo build -p deltachat_ffi --features jsonrpc
cargo build -p deltachat_ffi
# remove and inhibit writing PYC files
rm -rf tests/__pycache__

View File

@@ -8,7 +8,7 @@ set -e -x
# compile core lib
cargo build --release -p deltachat_ffi --features jsonrpc
cargo build --release -p deltachat_ffi
# Statically link against libdeltachat.a.
export DCC_RS_DEV="$PWD"
@@ -31,6 +31,6 @@ unset CHATMAIL_DOMAIN
# Try to build wheels for a range of interpreters, but don't fail if they are not available.
# E.g. musllinux_1_1 does not have PyPy interpreters as of 2022-07-10
tox --workdir "$TOXWORKDIR" -e py37,py38,py39,py310,py311,py312,py313,pypy37,pypy38,pypy39,pypy310 --skip-missing-interpreters true
tox --workdir "$TOXWORKDIR" -e py38,py39,py310,py311,py312,py313,pypy38,pypy39,pypy310 --skip-missing-interpreters true
auditwheel repair "$TOXWORKDIR"/wheelhouse/deltachat* -w "$TOXWORKDIR/wheelhouse"

View File

@@ -16,7 +16,7 @@ use image::ImageReader;
use image::{DynamicImage, GenericImage, GenericImageView, ImageFormat, Pixel, Rgba};
use num_traits::FromPrimitive;
use tokio::io::AsyncWriteExt;
use tokio::{fs, io};
use tokio::{fs, io, task};
use tokio_stream::wrappers::ReadDirStream;
use crate::config::Config;
@@ -34,6 +34,10 @@ use crate::log::LogExt;
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct BlobObject<'a> {
blobdir: &'a Path,
/// The name of the file on the disc.
/// Note that this is NOT the user-visible filename,
/// which is only stored in Param::Filename on the message.
name: String,
}
@@ -74,7 +78,7 @@ impl<'a> BlobObject<'a> {
Ok(blob)
}
// Creates a new file, returning a tuple of the name and the handle.
/// Creates a new file, returning a tuple of the name and the handle.
async fn create_new_file(
context: &Context,
dir: &Path,
@@ -88,6 +92,8 @@ impl<'a> BlobObject<'a> {
attempt += 1;
let path = dir.join(&name);
match fs::OpenOptions::new()
// Using `create_new(true)` in order to avoid race conditions
// when creating multiple files with the same name.
.create_new(true)
.write(true)
.open(&path)
@@ -139,6 +145,103 @@ impl<'a> BlobObject<'a> {
Ok(blob)
}
/// Creates a blob object by copying or renaming an existing file.
/// If the source file is already in the blobdir, it will be renamed,
/// otherwise it will be copied to the blobdir first.
///
/// In order to deduplicate files that contain the same data,
/// the file will be named `<hash>.<extension>`, e.g. `ce940175885d7b78f7b7e9f1396611f.jpg`.
/// The `original_name` param is only used to get the extension.
///
/// This is done in a in way which avoids race-conditions when multiple files are
/// concurrently created.
pub fn create_and_deduplicate(
context: &'a Context,
src: &Path,
original_name: &str,
) -> Result<BlobObject<'a>> {
// `create_and_deduplicate{_from_bytes}()` do blocking I/O, but can still be called
// from an async context thanks to `block_in_place()`.
// Tokio's "async" I/O functions are also just thin wrappers around the blocking I/O syscalls,
// so we are doing essentially the same here.
task::block_in_place(|| {
let temp_path;
let src_in_blobdir: &Path;
let blobdir = context.get_blobdir();
if src.starts_with(blobdir) || src.starts_with("$BLOBDIR/") {
src_in_blobdir = src;
} else {
info!(
context,
"Source file not in blobdir. Copying instead of moving in order to prevent moving a file that was still needed."
);
temp_path = blobdir.join(format!("tmp-{}", rand::random::<u64>()));
if std::fs::copy(src, &temp_path).is_err() {
// Maybe the blobdir didn't exist
std::fs::create_dir_all(blobdir).log_err(context).ok();
std::fs::copy(src, &temp_path).context("Copying new blobfile failed")?;
};
src_in_blobdir = &temp_path;
}
let hash = file_hash(src_in_blobdir)?.to_hex();
let hash = hash.as_str();
let hash = hash.get(0..31).unwrap_or(hash);
let new_file = if let Some(extension) = Path::new(original_name)
.extension()
.filter(|e| e.len() <= 32)
{
format!(
"$BLOBDIR/{hash}.{}",
extension.to_string_lossy().to_lowercase()
)
} else {
format!("$BLOBDIR/{hash}")
};
let blob = BlobObject {
blobdir,
name: new_file,
};
let new_path = blob.to_abs_path();
// This will also replace an already-existing file.
// Renaming is atomic, so this will avoid race conditions.
std::fs::rename(src_in_blobdir, &new_path)?;
context.emit_event(EventType::NewBlobFile(blob.as_name().to_string()));
Ok(blob)
})
}
/// Creates a new blob object with the file contents in `data`.
/// In order to deduplicate files that contain the same data,
/// the file will be named `<hash>.<extension>`, e.g. `ce940175885d7b78f7b7e9f1396611f.jpg`.
/// The `original_name` param is only used to get the extension.
///
/// The `data` will be written into the file without race-conditions.
///
/// This function does blocking I/O, but it can still be called from an async context
/// because `block_in_place()` is used to leave the async runtime if necessary.
pub fn create_and_deduplicate_from_bytes(
context: &'a Context,
data: &[u8],
original_name: &str,
) -> Result<BlobObject<'a>> {
task::block_in_place(|| {
let blobdir = context.get_blobdir();
let temp_path = blobdir.join(format!("tmp-{}", rand::random::<u64>()));
if std::fs::write(&temp_path, data).is_err() {
// Maybe the blobdir didn't exist
std::fs::create_dir_all(blobdir).log_err(context).ok();
std::fs::write(&temp_path, data).context("writing new blobfile failed")?;
};
BlobObject::create_and_deduplicate(context, &temp_path, original_name)
})
}
/// Creates a blob from a file, possibly copying it to the blobdir.
///
/// If the source file is not a path to into the blob directory
@@ -210,6 +313,9 @@ impl<'a> BlobObject<'a> {
/// this string in the database or [Params]. Eventually even
/// those conversions should be handled by the type system.
///
/// Note that this is NOT the user-visible filename,
/// which is only stored in Param::Filename on the message.
///
/// [Params]: crate::param::Params
pub fn as_name(&self) -> &str {
&self.name
@@ -279,7 +385,9 @@ impl<'a> BlobObject<'a> {
let ext: String = name
.chars()
.rev()
.take_while(|c| !c.is_whitespace())
.take_while(|c| {
(!c.is_ascii_punctuation() || *c == '.') && !c.is_whitespace() && !c.is_control()
})
.take(33)
.collect::<Vec<_>>()
.iter()
@@ -331,31 +439,25 @@ impl<'a> BlobObject<'a> {
/// Returns path to the stored Base64-decoded blob.
///
/// If `data` represents an image of known format, this adds the corresponding extension to
/// `suggested_file_stem`.
pub(crate) async fn store_from_base64(
context: &Context,
data: &str,
suggested_file_stem: &str,
) -> Result<String> {
/// If `data` represents an image of known format, this adds the corresponding extension.
///
/// Even though this function is not async, it's OK to call it from an async context.
pub(crate) fn store_from_base64(context: &Context, data: &str) -> Result<String> {
let buf = base64::engine::general_purpose::STANDARD.decode(data)?;
let ext = if let Ok(format) = image::guess_format(&buf) {
let name = if let Ok(format) = image::guess_format(&buf) {
if let Some(ext) = format.extensions_str().first() {
format!(".{ext}")
format!("file.{ext}")
} else {
String::new()
}
} else {
String::new()
};
let blob =
BlobObject::create(context, &format!("{suggested_file_stem}{ext}"), &buf).await?;
let blob = BlobObject::create_and_deduplicate_from_bytes(context, &buf, &name)?;
Ok(blob.as_name().to_string())
}
pub async fn recode_to_avatar_size(&mut self, context: &Context) -> Result<()> {
let blob_abs = self.to_abs_path();
let img_wh =
match MediaQuality::from_i32(context.get_config_int(Config::MediaQuality).await?)
.unwrap_or_default()
@@ -368,16 +470,15 @@ impl<'a> BlobObject<'a> {
let strict_limits = true;
// max_bytes is 20_000 bytes: Outlook servers don't allow headers larger than 32k.
// 32 / 4 * 3 = 24k if you account for base64 encoding. To be safe, we reduced this to 20k.
if let Some(new_name) = self.recode_to_size(
self.recode_to_size(
context,
blob_abs,
"".to_string(), // The name of an avatar doesn't matter
maybe_sticker,
img_wh,
20_000,
strict_limits,
)? {
self.name = new_name;
}
)?;
Ok(())
}
@@ -391,9 +492,9 @@ impl<'a> BlobObject<'a> {
pub async fn recode_to_image_size(
&mut self,
context: &Context,
name: String,
maybe_sticker: &mut bool,
) -> Result<()> {
let blob_abs = self.to_abs_path();
) -> Result<String> {
let (img_wh, max_bytes) =
match MediaQuality::from_i32(context.get_config_int(Config::MediaQuality).await?)
.unwrap_or_default()
@@ -405,35 +506,43 @@ impl<'a> BlobObject<'a> {
MediaQuality::Worse => (constants::WORSE_IMAGE_SIZE, constants::WORSE_IMAGE_BYTES),
};
let strict_limits = false;
if let Some(new_name) = self.recode_to_size(
let new_name = self.recode_to_size(
context,
blob_abs,
name,
maybe_sticker,
img_wh,
max_bytes,
strict_limits,
)? {
self.name = new_name;
}
Ok(())
)?;
Ok(new_name)
}
/// If `!strict_limits`, then if `max_bytes` is exceeded, reduce the image to `img_wh` and just
/// proceed with the result.
///
/// This modifies the blob object in-place.
///
/// Additionally, if you pass the user-visible filename as `name`
/// then the updated user-visible filename will be returned;
/// this may be necessary because the format may be changed to JPG,
/// i.e. "image.png" -> "image.jpg".
/// Pass an empty string if you don't care.
fn recode_to_size(
&mut self,
context: &Context,
mut blob_abs: PathBuf,
mut name: String,
maybe_sticker: &mut bool,
mut img_wh: u32,
max_bytes: usize,
strict_limits: bool,
) -> Result<Option<String>> {
) -> Result<String> {
// Add white background only to avatars to spare the CPU.
let mut add_white_bg = img_wh <= constants::BALANCED_AVATAR_SIZE;
let mut no_exif = false;
let no_exif_ref = &mut no_exif;
let res = tokio::task::block_in_place(move || {
let original_name = name.clone();
let res: Result<String> = tokio::task::block_in_place(move || {
let mut file = std::fs::File::open(self.to_abs_path())?;
let (nr_bytes, exif) = image_metadata(&file)?;
*no_exif_ref = exif.is_none();
@@ -447,7 +556,7 @@ impl<'a> BlobObject<'a> {
file.rewind()?;
ImageReader::with_format(
std::io::BufReader::new(&file),
ImageFormat::from_path(&blob_abs)?,
ImageFormat::from_path(self.to_abs_path())?,
)
}
};
@@ -455,7 +564,6 @@ impl<'a> BlobObject<'a> {
let mut img = imgreader.decode().context("image decode failure")?;
let orientation = exif.as_ref().map(|exif| exif_orientation(exif, context));
let mut encoded = Vec::new();
let mut changed_name = None;
if *maybe_sticker {
let x_max = img.width().saturating_sub(1);
@@ -467,7 +575,7 @@ impl<'a> BlobObject<'a> {
|| img.get_pixel(x_max, y_max).0[3] == 0);
}
if *maybe_sticker && exif.is_none() {
return Ok(None);
return Ok(name);
}
img = match orientation {
@@ -564,10 +672,10 @@ impl<'a> BlobObject<'a> {
if !matches!(fmt, ImageFormat::Jpeg)
&& matches!(ofmt, ImageOutputFormat::Jpeg { .. })
{
blob_abs = blob_abs.with_extension("jpg");
let file_name = blob_abs.file_name().context("No image file name (???)")?;
let file_name = file_name.to_str().context("Filename is no UTF-8 (???)")?;
changed_name = Some(format!("$BLOBDIR/{file_name}"));
name = Path::new(&name)
.with_extension("jpg")
.to_string_lossy()
.into_owned();
}
if encoded.is_empty() {
@@ -577,11 +685,12 @@ impl<'a> BlobObject<'a> {
encode_img(&img, ofmt, &mut encoded)?;
}
std::fs::write(&blob_abs, &encoded)
.context("failed to write recoded blob to file")?;
self.name = BlobObject::create_and_deduplicate_from_bytes(context, &encoded, &name)
.context("failed to write recoded blob to file")?
.name;
}
Ok(changed_name)
Ok(name)
});
match res {
Ok(_) => res,
@@ -591,7 +700,7 @@ impl<'a> BlobObject<'a> {
context,
"Cannot recode image, using original data: {err:#}.",
);
Ok(None)
Ok(original_name)
} else {
Err(err)
}
@@ -600,6 +709,17 @@ impl<'a> BlobObject<'a> {
}
}
fn file_hash(src: &Path) -> Result<blake3::Hash> {
let mut hasher = blake3::Hasher::new();
let mut src_file = std::fs::File::open(src)
.with_context(|| format!("Failed to open file {}", src.display()))?;
hasher
.update_reader(&mut src_file)
.context("update_reader")?;
let hash = hasher.finalize();
Ok(hash)
}
/// Returns image file size and Exif.
pub fn image_metadata(file: &std::fs::File) -> Result<(u64, Option<exif::Exif>)> {
let len = file.metadata()?.len();
@@ -760,16 +880,22 @@ fn add_white_bg(img: &mut DynamicImage) {
#[cfg(test)]
mod tests {
use fs::File;
use std::time::Duration;
use super::*;
use crate::chat::{self, create_group_chat, ProtectionStatus};
use crate::message::{Message, Viewtype};
use crate::sql;
use crate::test_utils::{self, TestContext};
use crate::tools::SystemTime;
fn check_image_size(path: impl AsRef<Path>, width: u32, height: u32) -> image::DynamicImage {
tokio::task::block_in_place(move || {
let img = image::open(path).expect("failed to open image");
let img = ImageReader::open(path)
.expect("failed to open image")
.with_guessed_format()
.expect("failed to guess format")
.decode()
.expect("failed to decode image");
assert_eq!(img.width(), width, "invalid width");
assert_eq!(img.height(), height, "invalid height");
img
@@ -790,22 +916,28 @@ mod tests {
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_lowercase_ext() {
let t = TestContext::new().await;
let blob = BlobObject::create(&t, "foo.TXT", b"hello").await.unwrap();
assert_eq!(blob.as_name(), "$BLOBDIR/foo.txt");
let blob = BlobObject::create_and_deduplicate_from_bytes(&t, b"hello", "foo.TXT").unwrap();
assert!(
blob.as_name().ends_with(".txt"),
"Blob {blob:?} should end with .txt"
);
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_as_file_name() {
let t = TestContext::new().await;
let blob = BlobObject::create(&t, "foo.txt", b"hello").await.unwrap();
assert_eq!(blob.as_file_name(), "foo.txt");
let blob = BlobObject::create_and_deduplicate_from_bytes(&t, b"hello", "foo.txt").unwrap();
assert_eq!(blob.as_file_name(), "ea8f163db38682925e4491c5e58d4bb.txt");
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_as_rel_path() {
let t = TestContext::new().await;
let blob = BlobObject::create(&t, "foo.txt", b"hello").await.unwrap();
assert_eq!(blob.as_rel_path(), Path::new("foo.txt"));
let blob = BlobObject::create_and_deduplicate_from_bytes(&t, b"hello", "foo.txt").unwrap();
assert_eq!(
blob.as_rel_path(),
Path::new("ea8f163db38682925e4491c5e58d4bb.txt")
);
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
@@ -865,10 +997,10 @@ mod tests {
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_create_long_names() {
let t = TestContext::new().await;
let s = "1".repeat(150);
let blob = BlobObject::create(&t, &s, b"data").await.unwrap();
let s = format!("file.{}", "a".repeat(100));
let blob = BlobObject::create_and_deduplicate_from_bytes(&t, b"data", &s).unwrap();
let blobname = blob.as_name().split('/').last().unwrap();
assert!(blobname.len() < 128);
assert!(blobname.len() < 70);
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
@@ -983,6 +1115,10 @@ mod tests {
let (stem, ext) = BlobObject::sanitise_name("a. tar.tar.gz");
assert_eq!(stem, "a. tar");
assert_eq!(ext, ".tar.gz");
let (stem, ext) = BlobObject::sanitise_name("Guia_uso_GNB (v0.8).pdf");
assert_eq!(stem, "Guia_uso_GNB (v0.8)");
assert_eq!(ext, ".pdf");
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
@@ -1003,7 +1139,7 @@ mod tests {
let strict_limits = true;
blob.recode_to_size(
&t,
blob.to_abs_path(),
"avatar.png".to_string(),
maybe_sticker,
img_wh,
20_000,
@@ -1011,7 +1147,12 @@ mod tests {
)
.unwrap();
tokio::task::block_in_place(move || {
let img = image::open(blob.to_abs_path()).unwrap();
let img = ImageReader::open(blob.to_abs_path())
.unwrap()
.with_guessed_format()
.unwrap()
.decode()
.unwrap();
assert!(img.width() == img_wh);
assert!(img.height() == img_wh);
assert_eq!(img.get_pixel(0, 0), Rgba(color));
@@ -1021,19 +1162,25 @@ mod tests {
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_selfavatar_outside_blobdir() {
async fn file_size(path_buf: &Path) -> u64 {
fs::metadata(path_buf).await.unwrap().len()
}
let t = TestContext::new().await;
let avatar_src = t.dir.path().join("avatar.jpg");
let avatar_bytes = include_bytes!("../test-data/image/avatar1000x1000.jpg");
fs::write(&avatar_src, avatar_bytes).await.unwrap();
let avatar_blob = t.get_blobdir().join("avatar.jpg");
assert!(!avatar_blob.exists());
t.set_config(Config::Selfavatar, Some(avatar_src.to_str().unwrap()))
.await
.unwrap();
assert!(avatar_blob.exists());
assert!(fs::metadata(&avatar_blob).await.unwrap().len() < avatar_bytes.len() as u64);
let avatar_cfg = t.get_config(Config::Selfavatar).await.unwrap();
assert_eq!(avatar_cfg, avatar_blob.to_str().map(|s| s.to_string()));
let avatar_blob = t.get_config(Config::Selfavatar).await.unwrap().unwrap();
let avatar_path = Path::new(&avatar_blob);
assert!(
avatar_blob.ends_with("d98cd30ed8f2129bf3968420208849d"),
"The avatar filename should be its hash, put instead it's {avatar_blob}"
);
let scaled_avatar_size = file_size(avatar_path).await;
assert!(scaled_avatar_size < avatar_bytes.len() as u64);
check_image_size(avatar_src, 1000, 1000);
check_image_size(
@@ -1042,27 +1189,32 @@ mod tests {
constants::BALANCED_AVATAR_SIZE,
);
async fn file_size(path_buf: &Path) -> u64 {
let file = File::open(path_buf).await.unwrap();
file.metadata().await.unwrap().len()
}
let mut blob = BlobObject::new_from_path(&t, &avatar_blob).await.unwrap();
let mut blob = BlobObject::new_from_path(&t, avatar_path).await.unwrap();
let maybe_sticker = &mut false;
let strict_limits = true;
blob.recode_to_size(
&t,
blob.to_abs_path(),
"avatar.jpg".to_string(),
maybe_sticker,
1000,
3000,
strict_limits,
)
.unwrap();
assert!(file_size(&avatar_blob).await <= 3000);
assert!(file_size(&avatar_blob).await > 2000);
let new_file_size = file_size(&blob.to_abs_path()).await;
assert!(new_file_size <= 3000);
assert!(new_file_size > 2000);
// The new file should be smaller:
assert!(new_file_size < scaled_avatar_size);
// And the original file should not be touched:
assert_eq!(file_size(avatar_path).await, scaled_avatar_size);
tokio::task::block_in_place(move || {
let img = image::open(avatar_blob).unwrap();
let img = ImageReader::open(blob.to_abs_path())
.unwrap()
.with_guessed_format()
.unwrap()
.decode()
.unwrap();
assert!(img.width() > 130);
assert_eq!(img.width(), img.height());
});
@@ -1082,9 +1234,9 @@ mod tests {
.await
.unwrap();
let avatar_cfg = t.get_config(Config::Selfavatar).await.unwrap().unwrap();
assert_eq!(
avatar_cfg,
avatar_src.with_extension("png").to_str().unwrap()
assert!(
avatar_cfg.ends_with("9e7f409ac5c92b942cc4f31cee2770a"),
"Avatar file name {avatar_cfg} should end with its hash"
);
check_image_size(
@@ -1368,6 +1520,7 @@ mod tests {
.set_config(Config::MediaQuality, Some(media_quality_config))
.await?;
let file = alice.get_blobdir().join("file").with_extension(extension);
let file_name = format!("file.{extension}");
fs::write(&file, &bytes)
.await
@@ -1383,7 +1536,7 @@ mod tests {
}
let mut msg = Message::new(viewtype);
msg.set_file(file.to_str().unwrap(), None);
msg.set_file_and_deduplicate(&alice, &file, Some(&file_name), None)?;
let chat = alice.create_chat(&bob).await;
if set_draft {
chat.id.set_draft(&alice, Some(&mut msg)).await.unwrap();
@@ -1439,7 +1592,7 @@ mod tests {
.await
.context("failed to write file")?;
let mut msg = Message::new(Viewtype::Image);
msg.set_file(file.to_str().unwrap(), None);
msg.set_file_and_deduplicate(&alice, &file, Some("file.gif"), None)?;
let chat = alice.create_chat(&bob).await;
let sent = alice.send_msg(chat.id, &mut msg).await;
let bob_msg = bob.recv_msg(&sent).await;
@@ -1458,35 +1611,99 @@ mod tests {
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_increation_in_blobdir() -> Result<()> {
let t = TestContext::new_alice().await;
let chat_id = create_group_chat(&t, ProtectionStatus::Unprotected, "abc").await?;
async fn test_send_gif_as_sticker() -> Result<()> {
let bytes = include_bytes!("../test-data/image/image100x50.gif");
let alice = &TestContext::new_alice().await;
let file = alice.get_blobdir().join("file").with_extension("gif");
fs::write(&file, &bytes)
.await
.context("failed to write file")?;
let mut msg = Message::new(Viewtype::Sticker);
msg.set_file_and_deduplicate(alice, &file, None, None)?;
let chat = alice.get_self_chat().await;
let sent = alice.send_msg(chat.id, &mut msg).await;
let msg = Message::load_from_db(alice, sent.sender_msg_id).await?;
// Message::force_sticker() wasn't used, still Viewtype::Sticker is preserved because of the
// extension.
assert_eq!(msg.get_viewtype(), Viewtype::Sticker);
Ok(())
}
let file = t.get_blobdir().join("anyfile.dat");
fs::write(&file, b"bla").await?;
let mut msg = Message::new(Viewtype::File);
msg.set_file(file.to_str().unwrap(), None);
let prepared_id = chat::prepare_msg(&t, chat_id, &mut msg).await?;
assert_eq!(prepared_id, msg.id);
assert!(msg.is_increation());
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_create_and_deduplicate() -> Result<()> {
let t = TestContext::new().await;
let msg = Message::load_from_db(&t, prepared_id).await?;
assert!(msg.is_increation());
let path = t.get_blobdir().join("anyfile.dat");
fs::write(&path, b"bla").await?;
let blob = BlobObject::create_and_deduplicate(&t, &path, "anyfile.dat")?;
assert_eq!(blob.name, "$BLOBDIR/ce940175885d7b78f7b7e9f1396611f.dat");
assert_eq!(path.exists(), false);
assert_eq!(fs::read(&blob.to_abs_path()).await?, b"bla");
fs::write(&path, b"bla").await?;
let blob2 = BlobObject::create_and_deduplicate(&t, &path, "anyfile.dat")?;
assert_eq!(blob2.name, blob.name);
let path_outside_blobdir = t.dir.path().join("anyfile.dat");
fs::write(&path_outside_blobdir, b"bla").await?;
let blob3 = BlobObject::create_and_deduplicate(&t, &path_outside_blobdir, "anyfile.dat")?;
assert!(path_outside_blobdir.exists());
assert_eq!(blob3.name, blob.name);
fs::write(&path, b"blabla").await?;
let blob4 = BlobObject::create_and_deduplicate(&t, &path, "anyfile.dat")?;
assert_ne!(blob4.name, blob.name);
fs::remove_dir_all(t.get_blobdir()).await?;
let blob5 = BlobObject::create_and_deduplicate(&t, &path_outside_blobdir, "anyfile.dat")?;
assert_eq!(blob5.name, blob.name);
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_increation_not_blobdir() -> Result<()> {
let t = TestContext::new_alice().await;
let chat_id = create_group_chat(&t, ProtectionStatus::Unprotected, "abc").await?;
assert_ne!(t.get_blobdir().to_str(), t.dir.path().to_str());
async fn test_create_and_deduplicate_from_bytes() -> Result<()> {
let t = TestContext::new().await;
let file = t.dir.path().join("anyfile.dat");
fs::write(&file, b"bla").await?;
let mut msg = Message::new(Viewtype::File);
msg.set_file(file.to_str().unwrap(), None);
assert!(chat::prepare_msg(&t, chat_id, &mut msg).await.is_err());
fs::remove_dir(t.get_blobdir()).await?;
let blob = BlobObject::create_and_deduplicate_from_bytes(&t, b"bla", "file")?;
assert_eq!(blob.name, "$BLOBDIR/ce940175885d7b78f7b7e9f1396611f");
assert_eq!(fs::read(&blob.to_abs_path()).await?, b"bla");
let modified1 = blob.to_abs_path().metadata()?.modified()?;
// Test that the modification time of the file is updated when a new file is created
// so that it's not deleted during housekeeping.
// We can't use SystemTime::shift() here because file creation uses the actual OS time,
// which we can't mock from our code.
tokio::time::sleep(Duration::from_millis(1100)).await;
let blob2 = BlobObject::create_and_deduplicate_from_bytes(&t, b"bla", "file")?;
assert_eq!(blob2.name, blob.name);
let modified2 = blob.to_abs_path().metadata()?.modified()?;
assert_ne!(modified1, modified2);
sql::housekeeping(&t).await?;
assert!(blob2.to_abs_path().exists());
// If we do shift the time by more than 1h, the blob file will be deleted during housekeeping:
SystemTime::shift(Duration::from_secs(65 * 60));
sql::housekeeping(&t).await?;
assert_eq!(blob2.to_abs_path().exists(), false);
let blob3 = BlobObject::create_and_deduplicate_from_bytes(&t, b"blabla", "file")?;
assert_ne!(blob3.name, blob.name);
{
// If something goes wrong and the blob file is overwritten,
// the correct content should be restored:
fs::write(blob3.to_abs_path(), b"bloblo").await?;
let blob4 = BlobObject::create_and_deduplicate_from_bytes(&t, b"blabla", "file")?;
let blob4_content = fs::read(blob4.to_abs_path()).await?;
assert_eq!(blob4_content, b"blabla");
}
Ok(())
}

File diff suppressed because it is too large Load Diff

3525
src/chat/chat_tests.rs Normal file

File diff suppressed because it is too large Load Diff

View File

@@ -144,7 +144,7 @@ impl Chatlist {
ORDER BY timestamp DESC, id DESC LIMIT 1)
WHERE c.id>9
AND c.blocked!=1
AND c.id IN(SELECT chat_id FROM chats_contacts WHERE contact_id=?2)
AND c.id IN(SELECT chat_id FROM chats_contacts WHERE contact_id=?2 AND add_timestamp >= remove_timestamp)
GROUP BY c.id
ORDER BY c.archived=?3 DESC, IFNULL(m.timestamp,c.created_timestamp) DESC, m.id DESC;",
(MessageState::OutDraft, query_contact_id, ChatVisibility::Pinned),
@@ -261,7 +261,7 @@ impl Chatlist {
WHERE c.id>9 AND c.id!=?
AND c.blocked=0
AND NOT c.archived=?
AND (c.type!=? OR c.id IN(SELECT chat_id FROM chats_contacts WHERE contact_id=?))
AND (c.type!=? OR c.id IN(SELECT chat_id FROM chats_contacts WHERE contact_id=? AND add_timestamp >= remove_timestamp))
GROUP BY c.id
ORDER BY c.id=? DESC, c.archived=? DESC, IFNULL(m.timestamp,c.created_timestamp) DESC, m.id DESC;",
(
@@ -550,7 +550,7 @@ mod tests {
let chats = Chatlist::try_load(&t, 0, Some("is:unread"), None)
.await
.unwrap();
assert!(chats.len() == 1);
assert_eq!(chats.len(), 1);
let chats = Chatlist::try_load(&t, DC_GCL_ARCHIVED_ONLY, None, None)
.await
@@ -576,7 +576,7 @@ mod tests {
.unwrap();
let chats = Chatlist::try_load(&t, 0, None, None).await.unwrap();
assert!(chats.len() == 3);
assert_eq!(chats.len(), 3);
assert!(!Chat::load_from_db(&t, chats.get_chat_id(0).unwrap())
.await
.unwrap()
@@ -585,7 +585,7 @@ mod tests {
let chats = Chatlist::try_load(&t, DC_GCL_FOR_FORWARDING, None, None)
.await
.unwrap();
assert!(chats.len() == 2); // device chat cannot be written and is skipped on forwarding
assert_eq!(chats.len(), 2); // device chat cannot be written and is skipped on forwarding
assert!(Chat::load_from_db(&t, chats.get_chat_id(0).unwrap())
.await
.unwrap()
@@ -597,7 +597,7 @@ mod tests {
let chats = Chatlist::try_load(&t, DC_GCL_FOR_FORWARDING, None, None)
.await
.unwrap();
assert!(chats.len() == 1);
assert_eq!(chats.len(), 1);
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]

View File

@@ -80,7 +80,7 @@ pub enum Config {
/// SMTP server security (e.g. TLS, STARTTLS).
SendSecurity,
/// Deprecated option for backwards compatibilty.
/// Deprecated option for backwards compatibility.
///
/// Certificate checks for SMTP are actually controlled by `imap_certificate_checks` config.
SmtpCertificateChecks,
@@ -143,7 +143,10 @@ pub enum Config {
/// Send BCC copy to self.
///
/// Should be enabled for multidevice setups.
#[strum(props(default = "1"))]
/// Default is 0 for chatmail accounts, 1 otherwise.
///
/// This is automatically enabled when importing/exporting a backup,
/// setting up a second device, or receiving a sync message.
BccSelf,
/// True if encryption is preferred according to Autocrypt standard.
@@ -202,7 +205,7 @@ pub enum Config {
/// Value 1 is treated as "delete at once": messages are deleted
/// immediately, without moving to DeltaChat folder.
///
/// Default is 1 for chatmail accounts before a backup export, 0 otherwise.
/// Default is 1 for chatmail accounts without `BccSelf`, 0 otherwise.
DeleteServerAfter,
/// Timer in seconds after which the message is deleted from the
@@ -384,6 +387,11 @@ pub enum Config {
/// Enable sending and executing (applying) sync messages. Sending requires `BccSelf` to be set
/// and `Bot` unset.
///
/// On real devices, this is usually always enabled and `BccSelf` is the only setting
/// that controls whether sync messages are sent.
///
/// In tests, this is usually disabled.
#[strum(props(default = "1"))]
SyncMsgs,
@@ -441,6 +449,12 @@ pub enum Config {
/// Enable webxdc realtime features.
#[strum(props(default = "1"))]
WebxdcRealtimeEnabled,
/// Last device token stored on the chatmail server.
///
/// If it has not changed, we do not store
/// the device token again.
DeviceToken,
}
impl Config {
@@ -513,11 +527,19 @@ impl Context {
// Default values
let val = match key {
Config::ConfiguredInboxFolder => Some("INBOX"),
Config::DeleteServerAfter => match Box::pin(self.is_chatmail()).await? {
false => Some("0"),
true => Some("1"),
Config::BccSelf => match Box::pin(self.is_chatmail()).await? {
false => Some("1"),
true => Some("0"),
},
Config::ConfiguredInboxFolder => Some("INBOX"),
Config::DeleteServerAfter => {
match !Box::pin(self.get_config_bool(Config::BccSelf)).await?
&& Box::pin(self.is_chatmail()).await?
{
true => Some("1"),
false => Some("0"),
}
}
_ => key.get_str("default"),
};
Ok(val.map(|s| s.to_string()))
@@ -664,7 +686,7 @@ impl Context {
let value = match key {
Config::Selfavatar if value.is_empty() => None,
Config::Selfavatar => {
config_value = BlobObject::store_from_base64(self, value, "avatar").await?;
config_value = BlobObject::store_from_base64(self, value)?;
Some(config_value.as_str())
}
_ => Some(value),
@@ -1099,6 +1121,30 @@ mod tests {
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_delete_server_after_default() -> Result<()> {
let t = &TestContext::new_alice().await;
// Check that the settings are displayed correctly.
assert_eq!(t.get_config(Config::BccSelf).await?, Some("1".to_string()));
assert_eq!(
t.get_config(Config::DeleteServerAfter).await?,
Some("0".to_string())
);
// Leaving emails on the server even w/o `BccSelf` is a good default at least because other
// MUAs do so even if the server doesn't save sent messages to some sentbox (like Gmail
// does).
t.set_config_bool(Config::BccSelf, false).await?;
assert_eq!(
t.get_config(Config::DeleteServerAfter).await?,
Some("0".to_string())
);
Ok(())
}
const SAVED_MESSAGES_DEDUPLICATED_FILE: &str = "969142cb84015bc135767bc2370934a.png";
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_sync() -> Result<()> {
let alice0 = TestContext::new_alice().await;
@@ -1173,7 +1219,7 @@ mod tests {
let self_chat_avatar_path = self_chat.get_profile_image(&alice0).await?.unwrap();
assert_eq!(
self_chat_avatar_path,
alice0.get_blobdir().join("icon-saved-messages.png")
alice0.get_blobdir().join(SAVED_MESSAGES_DEDUPLICATED_FILE)
);
assert!(alice1
.get_config(Config::Selfavatar)

View File

@@ -61,10 +61,7 @@ macro_rules! progress {
impl Context {
/// Checks if the context is already configured.
pub async fn is_configured(&self) -> Result<bool> {
self.sql
.get_raw_config_bool("configured")
.await
.map_err(Into::into)
self.sql.get_raw_config_bool("configured").await
}
/// Configures this account with the currently set parameters.
@@ -452,8 +449,9 @@ async fn configure(ctx: &Context, param: &EnteredLoginParam) -> Result<Configure
imap.configure_folders(ctx, &mut imap_session, create_mvbox)
.await?;
let create = true;
imap_session
.select_with_uidvalidity(ctx, "INBOX")
.select_with_uidvalidity(ctx, "INBOX", create)
.await
.context("could not read INBOX status")?;

View File

@@ -1,7 +1,7 @@
//! Contacts module
use std::cmp::{min, Reverse};
use std::collections::BinaryHeap;
use std::collections::{BinaryHeap, HashSet};
use std::fmt;
use std::path::{Path, PathBuf};
use std::time::UNIX_EPOCH;
@@ -34,7 +34,6 @@ use crate::message::MessageState;
use crate::mimeparser::AvatarAction;
use crate::param::{Param, Params};
use crate::peerstate::Peerstate;
use crate::sql::{self, params_iter};
use crate::sync::{self, Sync::*};
use crate::tools::{duration_to_str, get_abs_path, smeared_time, time, SystemTime};
use crate::{chat, chatlist_events, stock_str};
@@ -114,7 +113,8 @@ impl ContactId {
SET gossiped_timestamp=0
WHERE EXISTS (SELECT 1 FROM chats_contacts
WHERE chats_contacts.chat_id=chats.id
AND chats_contacts.contact_id=?)",
AND chats_contacts.contact_id=?
AND chats_contacts.add_timestamp >= chats_contacts.remove_timestamp)",
(self,),
)
.await?;
@@ -129,22 +129,19 @@ impl ContactId {
) -> Result<()> {
context
.sql
.execute(
&format!(
"UPDATE contacts SET origin=? WHERE id IN ({}) AND origin<?",
sql::repeat_vars(ids.len())
),
rusqlite::params_from_iter(
params_iter(&[origin])
.chain(params_iter(ids))
.chain(params_iter(&[origin])),
),
)
.transaction(|transaction| {
let mut stmt = transaction
.prepare("UPDATE contacts SET origin=?1 WHERE id = ?2 AND origin < ?1")?;
for id in ids {
stmt.execute((origin, id))?;
}
Ok(())
})
.await?;
Ok(())
}
/// Returns contact adress.
/// Returns contact address.
pub async fn addr(&self, context: &Context) -> Result<String> {
let addr = context
.sql
@@ -348,7 +345,7 @@ async fn import_vcard_contact(context: &Context, contact: &VcardContact) -> Resu
return Ok(id);
}
let path = match &contact.profile_image {
Some(image) => match BlobObject::store_from_base64(context, image, "avatar").await {
Some(image) => match BlobObject::store_from_base64(context, image) {
Err(e) => {
warn!(
context,
@@ -805,7 +802,6 @@ impl Contact {
}
let mut name = sanitize_name(name);
#[allow(clippy::collapsible_if)]
if origin <= Origin::OutgoingTo {
// The user may accidentally have written to a "noreply" address with another MUA:
if addr.contains("noreply")
@@ -1042,7 +1038,11 @@ impl Contact {
listflags: u32,
query: Option<&str>,
) -> Result<Vec<ContactId>> {
let self_addrs = context.get_all_self_addrs().await?;
let self_addrs = context
.get_all_self_addrs()
.await?
.into_iter()
.collect::<HashSet<_>>();
let mut add_self = false;
let mut ret = Vec::new();
let flag_verified_only = (listflags & DC_GCL_VERIFIED_ONLY) != 0;
@@ -1057,29 +1057,32 @@ impl Contact {
context
.sql
.query_map(
&format!(
"SELECT c.id FROM contacts c \
"SELECT c.id, c.addr FROM contacts c
LEFT JOIN acpeerstates ps ON c.addr=ps.addr \
WHERE c.addr NOT IN ({})
AND c.id>? \
WHERE c.id>?
AND c.origin>=? \
AND c.blocked=0 \
AND (iif(c.name='',c.authname,c.name) LIKE ? OR c.addr LIKE ?) \
AND (1=? OR LENGTH(ps.verified_key_fingerprint)!=0) \
ORDER BY c.last_seen DESC, c.id DESC;",
sql::repeat_vars(self_addrs.len())
),
rusqlite::params_from_iter(params_iter(&self_addrs).chain(params_slice![
(
ContactId::LAST_SPECIAL,
minimal_origin,
s3str_like_cmd,
s3str_like_cmd,
if flag_verified_only { 0i32 } else { 1i32 }
])),
|row| row.get::<_, ContactId>(0),
|ids| {
for id in ids {
ret.push(id?);
&s3str_like_cmd,
&s3str_like_cmd,
if flag_verified_only { 0i32 } else { 1i32 },
),
|row| {
let id: ContactId = row.get(0)?;
let addr: String = row.get(1)?;
Ok((id, addr))
},
|rows| {
for row in rows {
let (id, addr) = row?;
if !self_addrs.contains(&addr) {
ret.push(id);
}
}
Ok(())
},
@@ -1112,23 +1115,23 @@ impl Contact {
context
.sql
.query_map(
&format!(
"SELECT id FROM contacts
WHERE addr NOT IN ({})
AND id>?
"SELECT id, addr FROM contacts
WHERE id>?
AND origin>=?
AND blocked=0
ORDER BY last_seen DESC, id DESC;",
sql::repeat_vars(self_addrs.len())
),
rusqlite::params_from_iter(
params_iter(&self_addrs)
.chain(params_slice![ContactId::LAST_SPECIAL, minimal_origin]),
),
|row| row.get::<_, ContactId>(0),
|ids| {
for id in ids {
ret.push(id?);
(ContactId::LAST_SPECIAL, minimal_origin),
|row| {
let id: ContactId = row.get(0)?;
let addr: String = row.get(1)?;
Ok((id, addr))
},
|rows| {
for row in rows {
let (id, addr) = row?;
if !self_addrs.contains(&addr) {
ret.push(id);
}
}
Ok(())
},
@@ -1984,7 +1987,7 @@ mod tests {
use crate::chat::{get_chat_contacts, send_text_msg, Chat};
use crate::chatlist::Chatlist;
use crate::receive_imf::receive_imf;
use crate::test_utils::{self, TestContext, TestContextManager};
use crate::test_utils::{self, TestContext, TestContextManager, TimeShiftFalsePositiveNote};
#[test]
fn test_contact_id_values() {
@@ -2915,6 +2918,8 @@ Hi."#;
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_was_seen_recently() -> Result<()> {
let _n = TimeShiftFalsePositiveNote;
let mut tcm = TestContextManager::new();
let alice = tcm.alice().await;
let bob = tcm.bob().await;
@@ -2930,18 +2935,7 @@ Hi."#;
bob.recv_msg(&sent_msg).await;
let contact = Contact::get_by_id(&bob, *contacts.first().unwrap()).await?;
let green = nu_ansi_term::Color::Green.normal();
assert!(
contact.was_seen_recently(),
"{}",
green.paint(
"\nNOTE: This test failure is probably a false-positive, caused by tests running in parallel.
The issue is that `SystemTime::shift()` (a utility function for tests) changes the time for all threads doing tests, and not only for the running test.
Until the false-positive is fixed:
- Use `cargo test -- --test-threads 1` instead of `cargo test`
- Or use `cargo nextest run` (install with `cargo install cargo-nextest --locked`)\n"
)
);
assert!(contact.was_seen_recently());
let self_contact = Contact::get_by_id(&bob, ContactId::SELF).await?;
assert!(!self_contact.was_seen_recently());

View File

@@ -553,23 +553,7 @@ impl Context {
if self.scheduler.is_running().await {
self.scheduler.maybe_network().await;
// Wait until fetching is finished.
// Ideally we could wait for connectivity change events,
// but sleep loop is good enough.
// First 100 ms sleep in chunks of 10 ms.
for _ in 0..10 {
if self.all_work_done().await {
break;
}
tokio::time::sleep(std::time::Duration::from_millis(10)).await;
}
// If we are not finished in 100 ms, keep waking up every 100 ms.
while !self.all_work_done().await {
tokio::time::sleep(std::time::Duration::from_millis(100)).await;
}
self.wait_for_all_work_done().await;
} else {
// Pause the scheduler to ensure another connection does not start
// while we are fetching on a dedicated connection.
@@ -659,14 +643,36 @@ impl Context {
}
/// Emits a MsgsChanged event with specified chat and message ids
///
/// If IDs are unset, [`Self::emit_msgs_changed_without_ids`]
/// or [`Self::emit_msgs_changed_without_msg_id`] should be used
/// instead of this function.
pub fn emit_msgs_changed(&self, chat_id: ChatId, msg_id: MsgId) {
debug_assert!(!chat_id.is_unset());
debug_assert!(!msg_id.is_unset());
self.emit_event(EventType::MsgsChanged { chat_id, msg_id });
chatlist_events::emit_chatlist_changed(self);
chatlist_events::emit_chatlist_item_changed(self, chat_id);
}
/// Emits a MsgsChanged event with specified chat and without message id.
pub fn emit_msgs_changed_without_msg_id(&self, chat_id: ChatId) {
debug_assert!(!chat_id.is_unset());
self.emit_event(EventType::MsgsChanged {
chat_id,
msg_id: MsgId::new(0),
});
chatlist_events::emit_chatlist_changed(self);
chatlist_events::emit_chatlist_item_changed(self, chat_id);
}
/// Emits an IncomingMsg event with specified chat and message ids
pub fn emit_incoming_msg(&self, chat_id: ChatId, msg_id: MsgId) {
debug_assert!(!chat_id.is_unset());
debug_assert!(!msg_id.is_unset());
self.emit_event(EventType::IncomingMsg { chat_id, msg_id });
chatlist_events::emit_chatlist_changed(self);
chatlist_events::emit_chatlist_item_changed(self, chat_id);
@@ -1771,6 +1777,7 @@ mod tests {
"socks5_password",
"key_id",
"webxdc_integration",
"device_token",
];
let t = TestContext::new().await;
let info = t.get_info().await.unwrap();

View File

@@ -100,7 +100,9 @@ pub async fn maybe_set_logging_xdc(
context,
msg.get_viewtype(),
chat_id,
msg.param.get_path(Param::File, context).unwrap_or_default(),
msg.param
.get_path(Param::Filename, context)
.unwrap_or_default(),
msg.get_id(),
)
.await?;
@@ -113,11 +115,11 @@ pub async fn maybe_set_logging_xdc_inner(
context: &Context,
viewtype: Viewtype,
chat_id: ChatId,
file: Option<PathBuf>,
filename: Option<PathBuf>,
msg_id: MsgId,
) -> anyhow::Result<()> {
if viewtype == Viewtype::Webxdc {
if let Some(file) = file {
if let Some(file) = filename {
if let Some(file_name) = file.file_name().and_then(|name| name.to_str()) {
if file_name.starts_with("debug_logging")
&& file_name.ends_with(".xdc")

View File

@@ -3,7 +3,7 @@
use std::cmp::max;
use std::collections::BTreeMap;
use anyhow::{anyhow, bail, Result};
use anyhow::{anyhow, bail, ensure, Result};
use deltachat_derive::{FromSql, ToSql};
use serde::{Deserialize, Serialize};
@@ -201,7 +201,11 @@ impl Session {
bail!("Attempt to fetch UID 0");
}
self.select_with_uidvalidity(context, folder).await?;
let create = false;
let folder_exists = self
.select_with_uidvalidity(context, folder, create)
.await?;
ensure!(folder_exists, "No folder {folder}");
// we are connected, and the folder is selected
info!(context, "Downloading message {}/{} fully...", folder, uid);
@@ -436,7 +440,7 @@ mod tests {
let file = alice.get_blobdir().join("minimal.xdc");
tokio::fs::write(&file, include_bytes!("../test-data/webxdc/minimal.xdc")).await?;
let mut instance = Message::new(Viewtype::File);
instance.set_file(file.to_str().unwrap(), None);
instance.set_file_and_deduplicate(&alice, &file, None, None)?;
let _sent1 = alice.send_msg(chat_id, &mut instance).await;
alice

View File

@@ -40,20 +40,17 @@ impl EncryptHelper {
/// Determines if we can and should encrypt.
///
/// For encryption to be enabled, `e2ee_guaranteed` should be true, or strictly more than a half
/// of peerstates should prefer encryption. Own preference is counted equally to peer
/// preferences, even if message copy is not sent to self.
///
/// `e2ee_guaranteed` should be set to true for replies to encrypted messages (as required by
/// Autocrypt Level 1, version 1.1) and for messages sent in protected groups.
///
/// Returns an error if `e2ee_guaranteed` is true, but one or more keys are missing.
pub fn should_encrypt(
pub(crate) async fn should_encrypt(
&self,
context: &Context,
e2ee_guaranteed: bool,
peerstates: &[(Option<Peerstate>, String)],
) -> Result<bool> {
let is_chatmail = context.is_chatmail().await?;
let mut prefer_encrypt_count = if self.prefer_encrypt == EncryptPreference::Mutual {
1
} else {
@@ -64,10 +61,15 @@ impl EncryptHelper {
Some(peerstate) => {
let prefer_encrypt = peerstate.prefer_encrypt;
info!(context, "Peerstate for {addr:?} is {prefer_encrypt}.");
match peerstate.prefer_encrypt {
EncryptPreference::NoPreference | EncryptPreference::Reset => {}
EncryptPreference::Mutual => prefer_encrypt_count += 1,
};
if match peerstate.prefer_encrypt {
EncryptPreference::NoPreference | EncryptPreference::Reset => {
(peerstate.prefer_encrypt != EncryptPreference::Reset || is_chatmail)
&& self.prefer_encrypt == EncryptPreference::Mutual
}
EncryptPreference::Mutual => true,
} {
prefer_encrypt_count += 1;
}
}
None => {
let msg = format!("Peerstate for {addr:?} missing, cannot encrypt");
@@ -170,9 +172,11 @@ pub async fn ensure_secret_key_exists(context: &Context) -> Result<()> {
#[cfg(test)]
mod tests {
use super::*;
use crate::chat::send_text_msg;
use crate::key::DcKey;
use crate::message::{Message, Viewtype};
use crate::param::Param;
use crate::receive_imf::receive_imf;
use crate::test_utils::{bob_keypair, TestContext, TestContextManager};
mod ensure_secret_key_exists {
@@ -320,29 +324,109 @@ Sent with my Delta Chat Messenger: https://delta.chat";
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_should_encrypt() {
async fn test_should_encrypt() -> Result<()> {
let t = TestContext::new_alice().await;
assert!(t.get_config_bool(Config::E2eeEnabled).await?);
let encrypt_helper = EncryptHelper::new(&t).await.unwrap();
// test with EncryptPreference::NoPreference:
// if e2ee_eguaranteed is unset, there is no encryption as not more than half of peers want encryption
let ps = new_peerstates(EncryptPreference::NoPreference);
assert!(encrypt_helper.should_encrypt(&t, true, &ps).unwrap());
assert!(!encrypt_helper.should_encrypt(&t, false, &ps).unwrap());
assert!(encrypt_helper.should_encrypt(&t, true, &ps).await?);
// Own preference is `Mutual` and we have the peer's key.
assert!(encrypt_helper.should_encrypt(&t, false, &ps).await?);
// test with EncryptPreference::Reset
let ps = new_peerstates(EncryptPreference::Reset);
assert!(encrypt_helper.should_encrypt(&t, true, &ps).unwrap());
assert!(!encrypt_helper.should_encrypt(&t, false, &ps).unwrap());
assert!(encrypt_helper.should_encrypt(&t, true, &ps).await?);
assert!(!encrypt_helper.should_encrypt(&t, false, &ps).await?);
// test with EncryptPreference::Mutual (self is also Mutual)
let ps = new_peerstates(EncryptPreference::Mutual);
assert!(encrypt_helper.should_encrypt(&t, true, &ps).unwrap());
assert!(encrypt_helper.should_encrypt(&t, false, &ps).unwrap());
assert!(encrypt_helper.should_encrypt(&t, true, &ps).await?);
assert!(encrypt_helper.should_encrypt(&t, false, &ps).await?);
// test with missing peerstate
let ps = vec![(None, "bob@foo.bar".to_string())];
assert!(encrypt_helper.should_encrypt(&t, true, &ps).is_err());
assert!(!encrypt_helper.should_encrypt(&t, false, &ps).unwrap());
assert!(encrypt_helper.should_encrypt(&t, true, &ps).await.is_err());
assert!(!encrypt_helper.should_encrypt(&t, false, &ps).await?);
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_should_encrypt_e2ee_disabled() -> Result<()> {
let t = &TestContext::new_alice().await;
t.set_config_bool(Config::E2eeEnabled, false).await?;
let encrypt_helper = EncryptHelper::new(t).await.unwrap();
let ps = new_peerstates(EncryptPreference::NoPreference);
assert!(!encrypt_helper.should_encrypt(t, false, &ps).await?);
let ps = new_peerstates(EncryptPreference::Reset);
assert!(encrypt_helper.should_encrypt(t, true, &ps).await?);
let mut ps = new_peerstates(EncryptPreference::Mutual);
// Own preference is `NoPreference` and there's no majority with `Mutual`.
assert!(!encrypt_helper.should_encrypt(t, false, &ps).await?);
// Now the majority wants to encrypt. Let's encrypt, anyway there are other cases when we
// can't send unencrypted, e.g. protected groups.
ps.push(ps[0].clone());
assert!(encrypt_helper.should_encrypt(t, false, &ps).await?);
// Test with missing peerstate.
let ps = vec![(None, "bob@foo.bar".to_string())];
assert!(encrypt_helper.should_encrypt(t, true, &ps).await.is_err());
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_chatmail_prefers_to_encrypt() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
let bob = &tcm.bob().await;
bob.set_config_bool(Config::IsChatmail, true).await?;
let bob_chat_id = tcm
.send_recv_accept(alice, bob, "Hello from DC")
.await
.chat_id;
receive_imf(
bob,
b"From: alice@example.org\n\
To: bob@example.net\n\
Message-ID: <2222@example.org>\n\
Date: Sun, 22 Mar 3000 22:37:58 +0000\n\
\n\
Hello from another MUA\n",
false,
)
.await?;
send_text_msg(bob, bob_chat_id, "hi".to_string()).await?;
let sent_msg = bob.pop_sent_msg().await;
let msg = Message::load_from_db(bob, sent_msg.sender_msg_id).await?;
assert!(msg.get_showpadlock());
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_chatmail_can_send_unencrypted() -> Result<()> {
let mut tcm = TestContextManager::new();
let bob = &tcm.bob().await;
bob.set_config_bool(Config::IsChatmail, true).await?;
let bob_chat_id = receive_imf(
bob,
b"From: alice@example.org\n\
To: bob@example.net\n\
Message-ID: <2222@example.org>\n\
Date: Sun, 22 Mar 3000 22:37:58 +0000\n\
\n\
Hello\n",
false,
)
.await?
.unwrap()
.chat_id;
bob_chat_id.accept(bob).await?;
send_text_msg(bob, bob_chat_id, "hi".to_string()).await?;
let sent_msg = bob.pop_sent_msg().await;
let msg = Message::load_from_db(bob, sent_msg.sender_msg_id).await?;
assert!(!msg.get_showpadlock());
Ok(())
}
}

View File

@@ -84,7 +84,6 @@ use crate::location;
use crate::log::LogExt;
use crate::message::{Message, MessageState, MsgId, Viewtype};
use crate::mimeparser::SystemMessage;
use crate::sql::{self, params_iter};
use crate::stock_str;
use crate::tools::{duration_to_str, time, SystemTime};
@@ -329,23 +328,44 @@ pub(crate) async fn start_ephemeral_timers_msgids(
msg_ids: &[MsgId],
) -> Result<()> {
let now = time();
let count = context
let should_interrupt =
context
.sql
.transaction(move |transaction| {
let mut should_interrupt = false;
let mut stmt =
transaction.prepare(
"UPDATE msgs SET ephemeral_timestamp = ?1 + ephemeral_timer
WHERE (ephemeral_timestamp == 0 OR ephemeral_timestamp > ?1 + ephemeral_timer) AND ephemeral_timer > 0
AND id=?2")?;
for msg_id in msg_ids {
should_interrupt |= stmt.execute((now, msg_id))? > 0;
}
Ok(should_interrupt)
}).await?;
if should_interrupt {
context.scheduler.interrupt_ephemeral_task().await;
}
Ok(())
}
/// Starts ephemeral timer for all messages in the chat.
///
/// This should be called when chat is marked as noticed.
pub(crate) async fn start_chat_ephemeral_timers(context: &Context, chat_id: ChatId) -> Result<()> {
let now = time();
let should_interrupt = context
.sql
.execute(
&format!(
"UPDATE msgs SET ephemeral_timestamp = ? + ephemeral_timer
WHERE (ephemeral_timestamp == 0 OR ephemeral_timestamp > ? + ephemeral_timer) AND ephemeral_timer > 0
AND id IN ({})",
sql::repeat_vars(msg_ids.len())
),
rusqlite::params_from_iter(
std::iter::once(&now as &dyn crate::sql::ToSql)
.chain(std::iter::once(&now as &dyn crate::sql::ToSql))
.chain(params_iter(msg_ids)),
),
"UPDATE msgs SET ephemeral_timestamp = ?1 + ephemeral_timer
WHERE chat_id = ?2
AND ephemeral_timer > 0
AND (ephemeral_timestamp == 0 OR ephemeral_timestamp > ?1 + ephemeral_timer)",
(now, chat_id),
)
.await?;
if count > 0 {
.await?
> 0;
if should_interrupt {
context.scheduler.interrupt_ephemeral_task().await;
}
Ok(())
@@ -482,7 +502,7 @@ pub(crate) async fn delete_expired_messages(context: &Context, now: i64) -> Resu
}
for modified_chat_id in modified_chat_ids {
context.emit_msgs_changed(modified_chat_id, MsgId::new(0));
context.emit_msgs_changed_without_msg_id(modified_chat_id);
}
for msg_id in webxdc_deleted {
@@ -695,7 +715,9 @@ pub(crate) async fn start_ephemeral_timers(context: &Context) -> Result<()> {
#[cfg(test)]
mod tests {
use super::*;
use crate::chat::{marknoticed_chat, set_muted, ChatVisibility, MuteDuration};
use crate::config::Config;
use crate::constants::DC_CHAT_ID_ARCHIVED_LINK;
use crate::download::DownloadState;
use crate::location;
use crate::message::markseen_msgs;
@@ -930,7 +952,6 @@ mod tests {
// Alice sends a text message.
let mut msg = Message::new(Viewtype::Text);
chat::prepare_msg(&alice.ctx, chat_alice, &mut msg).await?;
chat::send_msg(&alice.ctx, chat_alice, &mut msg).await?;
let sent = alice.pop_sent_msg().await;
@@ -957,14 +978,12 @@ mod tests {
// Alice sends message to Bob
let mut msg = Message::new(Viewtype::Text);
chat::prepare_msg(&alice.ctx, chat_alice, &mut msg).await?;
chat::send_msg(&alice.ctx, chat_alice, &mut msg).await?;
let sent = alice.pop_sent_msg().await;
bob.recv_msg(&sent).await;
// Alice sends second message to Bob, with no timer
let mut msg = Message::new(Viewtype::Text);
chat::prepare_msg(&alice.ctx, chat_alice, &mut msg).await?;
chat::send_msg(&alice.ctx, chat_alice, &mut msg).await?;
let sent = alice.pop_sent_msg().await;
@@ -1425,4 +1444,77 @@ mod tests {
Ok(())
}
/// Tests that ephemeral timer is started when the chat is noticed.
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_noticed_ephemeral_timer() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
let bob = &tcm.bob().await;
let chat = alice.create_chat(bob).await;
let duration = 60;
chat.id
.set_ephemeral_timer(alice, Timer::Enabled { duration })
.await?;
let bob_received_message = tcm.send_recv(alice, bob, "Hello!").await;
marknoticed_chat(bob, bob_received_message.chat_id).await?;
SystemTime::shift(Duration::from_secs(100));
delete_expired_messages(bob, time()).await?;
assert!(Message::load_from_db_optional(bob, bob_received_message.id)
.await?
.is_none());
Ok(())
}
/// Tests that archiving the chat starts ephemeral timer.
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_archived_ephemeral_timer() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
let bob = &tcm.bob().await;
let chat = alice.create_chat(bob).await;
let duration = 60;
chat.id
.set_ephemeral_timer(alice, Timer::Enabled { duration })
.await?;
let bob_received_message = tcm.send_recv(alice, bob, "Hello!").await;
bob_received_message
.chat_id
.set_visibility(bob, ChatVisibility::Archived)
.await?;
SystemTime::shift(Duration::from_secs(100));
delete_expired_messages(bob, time()).await?;
assert!(Message::load_from_db_optional(bob, bob_received_message.id)
.await?
.is_none());
// Bob mutes the chat so it is not unarchived.
set_muted(bob, bob_received_message.chat_id, MuteDuration::Forever).await?;
// Now test that for already archived chat
// timer is started if all archived chats are marked as noticed.
let bob_received_message_2 = tcm.send_recv(alice, bob, "Hello again!").await;
assert_eq!(bob_received_message_2.state, MessageState::InFresh);
marknoticed_chat(bob, DC_CHAT_ID_ARCHIVED_LINK).await?;
SystemTime::shift(Duration::from_secs(100));
delete_expired_messages(bob, time()).await?;
assert!(
Message::load_from_db_optional(bob, bob_received_message_2.id)
.await?
.is_none()
);
Ok(())
}
}

View File

@@ -59,7 +59,7 @@ pub enum EventType {
/// or for functions that are expected to fail (eg. dc_continue_key_transfer())
/// it might be better to delay showing these events until the function has really
/// failed (returned false). It should be sufficient to report only the *last* error
/// in a messasge box then.
/// in a message box then.
Error(String),
/// An action cannot be performed because the user is not in the group.
@@ -109,6 +109,9 @@ pub enum EventType {
/// A webxdc wants an info message or a changed summary to be notified.
IncomingWebxdcNotify {
/// ID of the chat.
chat_id: ChatId,
/// ID of the contact sending.
contact_id: ContactId,

View File

@@ -65,6 +65,15 @@ pub enum HeaderDef {
ChatGroupMemberAdded,
ChatContent,
/// Past members of the group.
ChatGroupPastMembers,
/// Space-separated timestamps of member addition
/// for members listed in the `To` field
/// followed by timestamps of member removal
/// for members listed in the `Chat-Group-Past-Members` field.
ChatGroupMemberTimestamps,
/// Duration of the attached media file.
ChatDuration,
@@ -73,6 +82,7 @@ pub enum HeaderDef {
/// [Autocrypt](https://autocrypt.org/) header.
Autocrypt,
AutocryptGossip,
AutocryptSetupMessage,
SecureJoin,

View File

@@ -7,6 +7,8 @@
//! `MsgId.get_html()` will return HTML -
//! this allows nice quoting, handling linebreaks properly etc.
use std::mem;
use anyhow::{Context as _, Result};
use base64::Engine as _;
use lettre_email::mime::Mime;
@@ -77,21 +79,26 @@ fn get_mime_multipart_type(ctype: &ParsedContentType) -> MimeMultipartType {
struct HtmlMsgParser {
pub html: String,
pub plain: Option<PlainText>,
pub(crate) msg_html: String,
}
impl HtmlMsgParser {
/// Function takes a raw mime-message string,
/// searches for the main-text part
/// and returns that as parser.html
pub async fn from_bytes(context: &Context, rawmime: &[u8]) -> Result<Self> {
pub async fn from_bytes<'a>(
context: &Context,
rawmime: &'a [u8],
) -> Result<(Self, mailparse::ParsedMail<'a>)> {
let mut parser = HtmlMsgParser {
html: "".to_string(),
plain: None,
msg_html: "".to_string(),
};
let parsedmail = mailparse::parse_mail(rawmime)?;
let parsedmail = mailparse::parse_mail(rawmime).context("Failed to parse mail")?;
parser.collect_texts_recursive(&parsedmail).await?;
parser.collect_texts_recursive(context, &parsedmail).await?;
if parser.html.is_empty() {
if let Some(plain) = &parser.plain {
@@ -100,8 +107,8 @@ impl HtmlMsgParser {
} else {
parser.cid_to_data_recursive(context, &parsedmail).await?;
}
Ok(parser)
parser.html += &mem::take(&mut parser.msg_html);
Ok((parser, parsedmail))
}
/// Function iterates over all mime-parts
@@ -114,12 +121,13 @@ impl HtmlMsgParser {
/// therefore we use the first one.
async fn collect_texts_recursive<'a>(
&'a mut self,
context: &'a Context,
mail: &'a mailparse::ParsedMail<'a>,
) -> Result<()> {
match get_mime_multipart_type(&mail.ctype) {
MimeMultipartType::Multiple => {
for cur_data in &mail.subparts {
Box::pin(self.collect_texts_recursive(cur_data)).await?
Box::pin(self.collect_texts_recursive(context, cur_data)).await?
}
Ok(())
}
@@ -128,8 +136,35 @@ impl HtmlMsgParser {
if raw.is_empty() {
return Ok(());
}
let mail = mailparse::parse_mail(&raw).context("failed to parse mail")?;
Box::pin(self.collect_texts_recursive(&mail)).await
let (parser, mail) = Box::pin(HtmlMsgParser::from_bytes(context, &raw)).await?;
if !parser.html.is_empty() {
let mut text = "\r\n\r\n".to_string();
for h in mail.headers {
let key = h.get_key();
if matches!(
key.to_lowercase().as_str(),
"date"
| "from"
| "sender"
| "reply-to"
| "to"
| "cc"
| "bcc"
| "subject"
) {
text += &format!("{key}: {}\r\n", h.get_value());
}
}
text += "\r\n";
self.msg_html += &PlainText {
text,
flowed: false,
delsp: false,
}
.to_html();
self.msg_html += &parser.html;
}
Ok(())
}
MimeMultipartType::Single => {
let mimetype = mail.ctype.mimetype.parse::<Mime>()?;
@@ -175,14 +210,7 @@ impl HtmlMsgParser {
}
Ok(())
}
MimeMultipartType::Message => {
let raw = mail.get_body_raw()?;
if raw.is_empty() {
return Ok(());
}
let mail = mailparse::parse_mail(&raw).context("failed to parse mail")?;
Box::pin(self.cid_to_data_recursive(context, &mail)).await
}
MimeMultipartType::Message => Ok(()),
MimeMultipartType::Single => {
let mimetype = mail.ctype.mimetype.parse::<Mime>()?;
if mimetype.type_() == mime::IMAGE {
@@ -240,7 +268,7 @@ impl MsgId {
warn!(context, "get_html: parser error: {:#}", err);
Ok(None)
}
Ok(parser) => Ok(Some(parser.html)),
Ok((parser, _)) => Ok(Some(parser.html)),
}
} else {
warn!(context, "get_html: no mime for {}", self);
@@ -263,7 +291,7 @@ pub fn new_html_mimepart(html: String) -> PartBuilder {
mod tests {
use super::*;
use crate::chat;
use crate::chat::forward_msgs;
use crate::chat::{forward_msgs, save_msgs};
use crate::config::Config;
use crate::contact::ContactId;
use crate::message::{MessengerMessage, Viewtype};
@@ -274,7 +302,7 @@ mod tests {
async fn test_htmlparse_plain_unspecified() {
let t = TestContext::new().await;
let raw = include_bytes!("../test-data/message/text_plain_unspecified.eml");
let parser = HtmlMsgParser::from_bytes(&t.ctx, raw).await.unwrap();
let (parser, _) = HtmlMsgParser::from_bytes(&t.ctx, raw).await.unwrap();
assert_eq!(
parser.html,
r#"<!DOCTYPE html>
@@ -292,7 +320,7 @@ This message does not have Content-Type nor Subject.<br/>
async fn test_htmlparse_plain_iso88591() {
let t = TestContext::new().await;
let raw = include_bytes!("../test-data/message/text_plain_iso88591.eml");
let parser = HtmlMsgParser::from_bytes(&t.ctx, raw).await.unwrap();
let (parser, _) = HtmlMsgParser::from_bytes(&t.ctx, raw).await.unwrap();
assert_eq!(
parser.html,
r#"<!DOCTYPE html>
@@ -310,7 +338,7 @@ message with a non-UTF-8 encoding: äöüßÄÖÜ<br/>
async fn test_htmlparse_plain_flowed() {
let t = TestContext::new().await;
let raw = include_bytes!("../test-data/message/text_plain_flowed.eml");
let parser = HtmlMsgParser::from_bytes(&t.ctx, raw).await.unwrap();
let (parser, _) = HtmlMsgParser::from_bytes(&t.ctx, raw).await.unwrap();
assert!(parser.plain.unwrap().flowed);
assert_eq!(
parser.html,
@@ -332,7 +360,7 @@ and will be wrapped as usual.<br/>
async fn test_htmlparse_alt_plain() {
let t = TestContext::new().await;
let raw = include_bytes!("../test-data/message/text_alt_plain.eml");
let parser = HtmlMsgParser::from_bytes(&t.ctx, raw).await.unwrap();
let (parser, _) = HtmlMsgParser::from_bytes(&t.ctx, raw).await.unwrap();
assert_eq!(
parser.html,
r#"<!DOCTYPE html>
@@ -353,7 +381,7 @@ test some special html-characters as &lt; &gt; and &amp; but also &quot; and &#x
async fn test_htmlparse_html() {
let t = TestContext::new().await;
let raw = include_bytes!("../test-data/message/text_html.eml");
let parser = HtmlMsgParser::from_bytes(&t.ctx, raw).await.unwrap();
let (parser, _) = HtmlMsgParser::from_bytes(&t.ctx, raw).await.unwrap();
// on windows, `\r\n` linends are returned from mimeparser,
// however, rust multiline-strings use just `\n`;
@@ -371,7 +399,7 @@ test some special html-characters as &lt; &gt; and &amp; but also &quot; and &#x
async fn test_htmlparse_alt_html() {
let t = TestContext::new().await;
let raw = include_bytes!("../test-data/message/text_alt_html.eml");
let parser = HtmlMsgParser::from_bytes(&t.ctx, raw).await.unwrap();
let (parser, _) = HtmlMsgParser::from_bytes(&t.ctx, raw).await.unwrap();
assert_eq!(
parser.html.replace('\r', ""), // see comment in test_htmlparse_html()
r##"<html>
@@ -386,7 +414,7 @@ test some special html-characters as &lt; &gt; and &amp; but also &quot; and &#x
async fn test_htmlparse_alt_plain_html() {
let t = TestContext::new().await;
let raw = include_bytes!("../test-data/message/text_alt_plain_html.eml");
let parser = HtmlMsgParser::from_bytes(&t.ctx, raw).await.unwrap();
let (parser, _) = HtmlMsgParser::from_bytes(&t.ctx, raw).await.unwrap();
assert_eq!(
parser.html.replace('\r', ""), // see comment in test_htmlparse_html()
r##"<html>
@@ -411,7 +439,7 @@ test some special html-characters as &lt; &gt; and &amp; but also &quot; and &#x
assert!(test.find("data:").is_none());
// parsing converts cid: to data:
let parser = HtmlMsgParser::from_bytes(&t.ctx, raw).await.unwrap();
let (parser, _) = HtmlMsgParser::from_bytes(&t.ctx, raw).await.unwrap();
assert!(parser.html.contains("<html>"));
assert!(!parser.html.contains("Content-Id:"));
assert!(parser.html.contains("data:image/jpeg;base64,/9j/4AAQ"));
@@ -471,6 +499,38 @@ test some special html-characters as &lt; &gt; and &amp; but also &quot; and &#x
assert!(html.contains("this is <b>html</b>"));
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_html_save_msg() -> Result<()> {
// Alice receives a non-delta html-message
let alice = TestContext::new_alice().await;
let chat = alice
.create_chat_with_contact("", "sender@testrun.org")
.await;
let raw = include_bytes!("../test-data/message/text_alt_plain_html.eml");
receive_imf(&alice, raw, false).await?;
let msg = alice.get_last_msg_in(chat.get_id()).await;
// Alice saves the message
let self_chat = alice.get_self_chat().await;
save_msgs(&alice, &[msg.id]).await?;
let saved_msg = alice.get_last_msg_in(self_chat.get_id()).await;
assert_ne!(saved_msg.id, msg.id);
assert_eq!(
saved_msg.get_original_msg_id(&alice).await?.unwrap(),
msg.id
);
assert!(!saved_msg.is_forwarded()); // UI should not flag "saved messages" as "forwarded"
assert_ne!(saved_msg.get_from_id(), ContactId::SELF);
assert_eq!(saved_msg.get_from_id(), msg.get_from_id());
assert_eq!(saved_msg.is_dc_message, MessengerMessage::No);
assert!(saved_msg.get_text().contains("this is plain"));
assert!(saved_msg.has_html());
let html = saved_msg.get_id().get_html(&alice).await?.unwrap();
assert!(html.contains("this is <b>html</b>"));
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_html_forwarding_encrypted() {
// Alice receives a non-delta html-message

View File

@@ -13,7 +13,7 @@ use std::{
time::{Duration, UNIX_EPOCH},
};
use anyhow::{bail, format_err, Context as _, Result};
use anyhow::{bail, ensure, format_err, Context as _, Result};
use async_channel::Receiver;
use async_imap::types::{Fetch, Flag, Name, NameAttribute, UnsolicitedResponse};
use deltachat_contact_tools::ContactAddress;
@@ -41,11 +41,11 @@ use crate::mimeparser;
use crate::net::proxy::ProxyConfig;
use crate::net::session::SessionStream;
use crate::oauth2::get_oauth2_access_token;
use crate::push::encrypt_device_token;
use crate::receive_imf::{
from_field_to_contact_id, get_prefetch_parent_message, receive_imf_inner, ReceivedMsg,
};
use crate::scheduler::connectivity::ConnectivityStore;
use crate::sql;
use crate::stock_str;
use crate::tools::{self, create_id, duration_to_str};
@@ -407,7 +407,7 @@ impl Imap {
"IMAP-LOGIN as {}",
lp.user
)));
self.connectivity.set_connected(context).await;
self.connectivity.set_preparing(context).await;
info!(context, "Successfully logged into IMAP server");
return Ok(session);
}
@@ -540,10 +540,14 @@ impl Imap {
return Ok(false);
}
session
.select_with_uidvalidity(context, folder)
let create = false;
let folder_exists = session
.select_with_uidvalidity(context, folder, create)
.await
.with_context(|| format!("Failed to select folder {folder:?}"))?;
if !folder_exists {
return Ok(false);
}
if !session.new_mail && !fetch_existing_msgs {
info!(context, "No new emails in folder {folder:?}.");
@@ -835,45 +839,52 @@ impl Session {
folder: &str,
folder_meaning: FolderMeaning,
) -> Result<()> {
let uid_validity;
// Collect pairs of UID and Message-ID.
let mut msgs = BTreeMap::new();
self.select_with_uidvalidity(context, folder).await?;
let create = false;
let folder_exists = self
.select_with_uidvalidity(context, folder, create)
.await?;
if folder_exists {
let mut list = self
.uid_fetch("1:*", RFC724MID_UID)
.await
.with_context(|| format!("Can't resync folder {folder}"))?;
while let Some(fetch) = list.try_next().await? {
let headers = match get_fetch_headers(&fetch) {
Ok(headers) => headers,
Err(err) => {
warn!(context, "Failed to parse FETCH headers: {}", err);
continue;
}
};
let message_id = prefetch_get_message_id(&headers);
let mut list = self
.uid_fetch("1:*", RFC724MID_UID)
.await
.with_context(|| format!("can't resync folder {folder}"))?;
while let Some(fetch) = list.try_next().await? {
let headers = match get_fetch_headers(&fetch) {
Ok(headers) => headers,
Err(err) => {
warn!(context, "Failed to parse FETCH headers: {}", err);
continue;
if let (Some(uid), Some(rfc724_mid)) = (fetch.uid, message_id) {
msgs.insert(
uid,
(
rfc724_mid,
target_folder(context, folder, folder_meaning, &headers).await?,
),
);
}
};
let message_id = prefetch_get_message_id(&headers);
if let (Some(uid), Some(rfc724_mid)) = (fetch.uid, message_id) {
msgs.insert(
uid,
(
rfc724_mid,
target_folder(context, folder, folder_meaning, &headers).await?,
),
);
}
info!(
context,
"resync_folder_uids: Collected {} message IDs in {folder}.",
msgs.len(),
);
uid_validity = get_uidvalidity(context, folder).await?;
} else {
warn!(context, "resync_folder_uids: No folder {folder}.");
uid_validity = 0;
}
info!(
context,
"Resync: collected {} message IDs in folder {}",
msgs.len(),
folder,
);
let uid_validity = get_uidvalidity(context, folder).await?;
// Write collected UIDs to SQLite database.
context
.sql
@@ -910,15 +921,15 @@ impl Session {
.await?;
context
.sql
.execute(
&format!(
"DELETE FROM imap WHERE id IN ({})",
sql::repeat_vars(row_ids.len())
),
rusqlite::params_from_iter(row_ids),
)
.transaction(|transaction| {
let mut stmt = transaction.prepare("DELETE FROM imap WHERE id = ?")?;
for row_id in row_ids {
stmt.execute((row_id,))?;
}
Ok(())
})
.await
.context("cannot remove deleted messages from imap table")?;
.context("Cannot remove deleted messages from imap table")?;
context.emit_event(EventType::ImapMessageDeleted(format!(
"IMAP messages {uid_set} marked as deleted"
@@ -941,15 +952,15 @@ impl Session {
// Messages are moved or don't exist, IMAP returns OK response in both cases.
context
.sql
.execute(
&format!(
"DELETE FROM imap WHERE id IN ({})",
sql::repeat_vars(row_ids.len())
),
rusqlite::params_from_iter(row_ids),
)
.transaction(|transaction| {
let mut stmt = transaction.prepare("DELETE FROM imap WHERE id = ?")?;
for row_id in row_ids {
stmt.execute((row_id,))?;
}
Ok(())
})
.await
.context("cannot delete moved messages from imap table")?;
.context("Cannot delete moved messages from imap table")?;
context.emit_event(EventType::ImapMessageMoved(format!(
"IMAP messages {set} moved to {target}"
)));
@@ -995,15 +1006,15 @@ impl Session {
}
context
.sql
.execute(
&format!(
"UPDATE imap SET target='' WHERE id IN ({})",
sql::repeat_vars(row_ids.len())
),
rusqlite::params_from_iter(row_ids),
)
.transaction(|transaction| {
let mut stmt = transaction.prepare("UPDATE imap SET target='' WHERE id = ?")?;
for row_id in row_ids {
stmt.execute((row_id,))?;
}
Ok(())
})
.await
.context("cannot plan deletion of messages")?;
.context("Cannot plan deletion of messages")?;
if copy {
context.emit_event(EventType::ImapMessageMoved(format!(
"IMAP messages {set} copied to {target}"
@@ -1039,7 +1050,11 @@ impl Session {
// MOVE/DELETE operations. This does not result in multiple SELECT commands
// being sent because `select_folder()` does nothing if the folder is already
// selected.
self.select_with_uidvalidity(context, folder).await?;
let create = false;
let folder_exists = self
.select_with_uidvalidity(context, folder, create)
.await?;
ensure!(folder_exists, "No folder {folder}");
// Empty target folder name means messages should be deleted.
if target.is_empty() {
@@ -1133,29 +1148,40 @@ impl Session {
.await?;
for (folder, rowid_set, uid_set) in UidGrouper::from(rows) {
if let Err(err) = self.select_with_uidvalidity(context, &folder).await {
warn!(context, "store_seen_flags_on_imap: Failed to select {folder}, will retry later: {err:#}.");
let create = false;
let folder_exists = match self.select_with_uidvalidity(context, &folder, create).await {
Err(err) => {
warn!(
context,
"store_seen_flags_on_imap: Failed to select {folder}, will retry later: {err:#}.");
continue;
}
Ok(folder_exists) => folder_exists,
};
if !folder_exists {
warn!(context, "store_seen_flags_on_imap: No folder {folder}.");
} else if let Err(err) = self.add_flag_finalized_with_set(&uid_set, "\\Seen").await {
warn!(
context,
"Cannot mark messages {uid_set} in {folder} as seen, will retry later: {err:#}.");
continue;
} else {
info!(
context,
"Marked messages {} in folder {} as seen.", uid_set, folder
);
context
.sql
.execute(
&format!(
"DELETE FROM imap_markseen WHERE id IN ({})",
sql::repeat_vars(rowid_set.len())
),
rusqlite::params_from_iter(rowid_set),
)
.await
.context("cannot remove messages marked as seen from imap_markseen table")?;
}
context
.sql
.transaction(|transaction| {
let mut stmt = transaction.prepare("DELETE FROM imap_markseen WHERE id = ?")?;
for rowid in rowid_set {
stmt.execute((rowid,))?;
}
Ok(())
})
.await
.context("Cannot remove messages marked as seen from imap_markseen table")?;
}
Ok(())
@@ -1171,9 +1197,14 @@ impl Session {
return Ok(());
}
self.select_with_uidvalidity(context, folder)
let create = false;
let folder_exists = self
.select_with_uidvalidity(context, folder, create)
.await
.context("failed to select folder")?;
.context("Failed to select folder")?;
if !folder_exists {
return Ok(());
}
let mailbox = self
.selected_mailbox
@@ -1300,7 +1331,7 @@ impl Session {
/// Returns the last UID fetched successfully and the info about each downloaded message.
/// If the message is incorrect or there is a failure to write a message to the database,
/// it is skipped and the error is logged.
#[allow(clippy::too_many_arguments)]
#[expect(clippy::too_many_arguments)]
pub(crate) async fn fetch_many_msgs(
&mut self,
context: &Context,
@@ -1421,9 +1452,7 @@ impl Session {
let is_seen = fetch_response.flags().any(|flag| flag == Flag::Seen);
let rfc724_mid = if let Some(rfc724_mid) = uid_message_ids.get(&request_uid) {
rfc724_mid
} else {
let Some(rfc724_mid) = uid_message_ids.get(&request_uid) else {
error!(
context,
"No Message-ID corresponding to UID {} passed in uid_messsage_ids.",
@@ -1560,16 +1589,52 @@ impl Session {
};
if self.can_metadata() && self.can_push() {
let folder = context
.get_config(Config::ConfiguredInboxFolder)
.await?
.context("INBOX is not configured")?;
let device_token_changed =
context.get_config(Config::DeviceToken).await?.as_ref() != Some(&device_token);
self.run_command_and_check_ok(format!(
"SETMETADATA \"{folder}\" (/private/devicetoken \"{device_token}\")"
))
.await
.context("SETMETADATA command failed")?;
if device_token_changed {
let folder = context
.get_config(Config::ConfiguredInboxFolder)
.await?
.context("INBOX is not configured")?;
let encrypted_device_token = encrypt_device_token(&device_token)
.context("Failed to encrypt device token")?;
// We expect that the server supporting `XDELTAPUSH` capability
// has non-synchronizing literals support as well:
// <https://www.rfc-editor.org/rfc/rfc7888>.
let encrypted_device_token_len = encrypted_device_token.len();
if encrypted_device_token_len <= 4096 {
self.run_command_and_check_ok(&format_setmetadata(
&folder,
&encrypted_device_token,
))
.await
.context("SETMETADATA command failed")?;
// Store device token saved on the server
// to prevent storing duplicate tokens.
// The server cannot deduplicate on its own
// because encryption gives a different
// result each time.
context
.set_config_internal(Config::DeviceToken, Some(&device_token))
.await?;
} else {
// If Apple or Google (FCM) gives us a very large token,
// do not even try to give it to IMAP servers.
//
// Limit of 4096 is arbitrarily selected
// to be the same as required by LITERAL- IMAP extension.
//
// Dovecot supports LITERAL+ and non-synchronizing literals
// of any length, but there is no reason for tokens
// to be that large even after OpenPGP encryption.
warn!(context, "Device token is too long for LITERAL-, ignoring.");
}
}
context.push_subscribed.store(true, Ordering::Relaxed);
} else if !context.push_subscriber.heartbeat_subscribed().await {
let context = context.clone();
@@ -1581,10 +1646,17 @@ impl Session {
}
}
fn format_setmetadata(folder: &str, device_token: &str) -> String {
let device_token_len = device_token.len();
format!(
"SETMETADATA \"{folder}\" (/private/devicetoken {{{device_token_len}+}}\r\n{device_token})"
)
}
impl Session {
/// Returns success if we successfully set the flag or we otherwise
/// think add_flag should not be retried: Disconnection during setting
/// the flag, or other imap-errors, returns true as well.
/// the flag, or other imap-errors, returns Ok as well.
///
/// Returning error means that the operation can be retried.
async fn add_flag_finalized_with_set(&mut self, uid_set: &str, flag: &str) -> Result<()> {
@@ -1631,7 +1703,11 @@ impl Session {
self.close().await?;
// Before moving emails to the mvbox we need to remember its UIDVALIDITY, otherwise
// emails moved before that wouldn't be fetched but considered "old" instead.
self.select_with_uidvalidity(context, folder).await?;
let create = false;
let folder_exists = self
.select_with_uidvalidity(context, folder, create)
.await?;
ensure!(folder_exists, "No MVBOX folder {:?}??", &folder);
return Ok(Some(folder));
}
}
@@ -1642,7 +1718,10 @@ impl Session {
// Some servers require namespace-style folder names like "INBOX.DeltaChat", so we try all
// the variants here.
for folder in folders {
match self.select_with_uidvalidity(context, folder).await {
match self
.select_with_uidvalidity(context, folder, create_mvbox)
.await
{
Ok(_) => {
info!(context, "MVBOX-folder {} created.", folder);
return Ok(Some(folder));
@@ -2493,10 +2572,14 @@ async fn add_all_recipients_as_contacts(
);
return Ok(());
};
session
.select_with_uidvalidity(context, &mailbox)
let create = false;
let folder_exists = session
.select_with_uidvalidity(context, &mailbox, create)
.await
.with_context(|| format!("could not select {mailbox}"))?;
if !folder_exists {
return Ok(());
}
let recipients = session
.get_all_recipients(context)
@@ -2643,7 +2726,6 @@ mod tests {
}
}
#[allow(clippy::too_many_arguments)]
async fn check_target_folder_combination(
folder: &str,
mvbox_move: bool,
@@ -2864,4 +2946,16 @@ mod tests {
vec![("INBOX".to_string(), vec![1, 2, 3], "2:3".to_string())]
);
}
#[test]
fn test_setmetadata_device_token() {
assert_eq!(
format_setmetadata("INBOX", "foobarbaz"),
"SETMETADATA \"INBOX\" (/private/devicetoken {9+}\r\nfoobarbaz)"
);
assert_eq!(
format_setmetadata("INBOX", "foo\r\nbar\r\nbaz\r\n"),
"SETMETADATA \"INBOX\" (/private/devicetoken {15+}\r\nfoo\r\nbar\r\nbaz\r\n)"
);
}
}

View File

@@ -29,7 +29,9 @@ impl Session {
) -> Result<Self> {
use futures::future::FutureExt;
self.select_with_uidvalidity(context, folder).await?;
let create = true;
self.select_with_uidvalidity(context, folder, create)
.await?;
if self.drain_unsolicited_responses(context)? {
self.new_mail = true;

View File

@@ -34,6 +34,7 @@ impl Imap {
let watched_folders = get_watched_folders(context).await?;
let mut folder_configs = BTreeMap::new();
let mut folder_names = Vec::new();
for folder in folders {
let folder_meaning = get_folder_meaning_by_attrs(folder.attributes());
@@ -44,6 +45,7 @@ impl Imap {
// already been moved and left it in the inbox.
continue;
}
folder_names.push(folder.name().to_string());
let folder_name_meaning = get_folder_meaning_by_name(folder.name());
if let Some(config) = folder_meaning.to_config() {
@@ -91,6 +93,7 @@ impl Imap {
}
}
info!(context, "Found folders: {folder_names:?}.");
last_scan.replace(tools::Time::now());
Ok(true)
}

View File

@@ -111,7 +111,8 @@ impl ImapSession {
}
}
/// Selects a folder and takes care of UIDVALIDITY changes.
/// Selects a folder optionally creating it and takes care of UIDVALIDITY changes. Returns false
/// iff `folder` doesn't exist.
///
/// When selecting a folder for the first time, sets the uid_next to the current
/// mailbox.uid_next so that no old emails are fetched.
@@ -123,11 +124,24 @@ impl ImapSession {
&mut self,
context: &Context,
folder: &str,
) -> Result<()> {
let newly_selected = self
.select_or_create_folder(context, folder)
.await
.with_context(|| format!("failed to select or create folder {folder}"))?;
create: bool,
) -> Result<bool> {
let newly_selected = if create {
self.select_or_create_folder(context, folder)
.await
.with_context(|| format!("failed to select or create folder {folder}"))?
} else {
match self.select_folder(context, folder).await {
Ok(newly_selected) => newly_selected,
Err(err) => match err {
Error::NoFolder(..) => return Ok(false),
_ => {
return Err(err)
.with_context(|| format!("failed to select folder {folder}"))?
}
},
}
};
let mailbox = self
.selected_mailbox
.as_mut()
@@ -199,7 +213,7 @@ impl ImapSession {
}
}
return Ok(());
return Ok(true);
}
// UIDVALIDITY is modified, reset highest seen MODSEQ.
@@ -233,7 +247,7 @@ impl ImapSession {
old_uid_next,
old_uid_validity,
);
Ok(())
Ok(true)
}
}

View File

@@ -416,7 +416,7 @@ async fn import_backup_stream_inner<R: tokio::io::AsyncRead + Unpin>(
.context("cannot import unpacked database");
}
if res.is_ok() {
res = adjust_delete_server_after(context).await;
res = adjust_bcc_self(context).await;
}
fs::remove_file(unpacked_database)
.await
@@ -796,7 +796,7 @@ async fn export_database(
.to_str()
.with_context(|| format!("path {} is not valid unicode", dest.display()))?;
adjust_delete_server_after(context).await?;
adjust_bcc_self(context).await?;
context
.sql
.set_raw_config_int("backup_time", timestamp)
@@ -826,15 +826,14 @@ async fn export_database(
.await
}
/// Sets `Config::DeleteServerAfter` to "never" if needed so that new messages are present on the
/// server after a backup restoration or available for all devices in multi-device case.
/// NB: Calling this after a backup import isn't reliable as we can crash in between, but this is a
/// problem only for old backups, new backups already have `DeleteServerAfter` set if necessary.
async fn adjust_delete_server_after(context: &Context) -> Result<()> {
if context.is_chatmail().await? && !context.config_exists(Config::DeleteServerAfter).await? {
context
.set_config(Config::DeleteServerAfter, Some("0"))
.await?;
/// Sets `Config::BccSelf` (and `DeleteServerAfter` to "never" in effect) if needed so that new
/// messages are present on the server after a backup restoration or available for all devices in
/// multi-device case. NB: Calling this after a backup import isn't reliable as we can crash in
/// between, but this is a problem only for old backups, new backups already have `BccSelf` set if
/// necessary.
async fn adjust_bcc_self(context: &Context) -> Result<()> {
if context.is_chatmail().await? && !context.config_exists(Config::BccSelf).await? {
context.set_config(Config::BccSelf, Some("1")).await?;
}
Ok(())
}
@@ -1030,12 +1029,20 @@ mod tests {
let context1 = &TestContext::new_alice().await;
// Check that the setting is displayed correctly.
// Check that the settings are displayed correctly.
assert_eq!(
context1.get_config(Config::BccSelf).await?,
Some("1".to_string())
);
assert_eq!(
context1.get_config(Config::DeleteServerAfter).await?,
Some("0".to_string())
);
context1.set_config_bool(Config::IsChatmail, true).await?;
assert_eq!(
context1.get_config(Config::BccSelf).await?,
Some("0".to_string())
);
assert_eq!(
context1.get_config(Config::DeleteServerAfter).await?,
Some("1".to_string())
@@ -1058,6 +1065,10 @@ mod tests {
assert!(context2.is_configured().await?);
assert!(context2.is_chatmail().await?);
for ctx in [context1, context2] {
assert_eq!(
ctx.get_config(Config::BccSelf).await?,
Some("1".to_string())
);
assert_eq!(
ctx.get_config(Config::DeleteServerAfter).await?,
Some("0".to_string())

View File

@@ -26,12 +26,11 @@ pub async fn initiate_key_transfer(context: &Context) -> Result<String> {
/* this may require a keypair to be created. this may take a second ... */
let setup_file_content = render_setup_file(context, &setup_code).await?;
/* encrypting may also take a while ... */
let setup_file_blob = BlobObject::create(
let setup_file_blob = BlobObject::create_and_deduplicate_from_bytes(
context,
"autocrypt-setup-message.html",
setup_file_content.as_bytes(),
)
.await?;
"autocrypt-setup-message.html",
)?;
let chat_id = ChatId::create_for_contact(context, ContactId::SELF).await?;
let mut msg = Message {
@@ -39,6 +38,8 @@ pub async fn initiate_key_transfer(context: &Context) -> Result<String> {
..Default::default()
};
msg.param.set(Param::File, setup_file_blob.as_name());
msg.param
.set(Param::Filename, "autocrypt-setup-message.html");
msg.subject = stock_str::ac_setup_msg_subject(context).await;
msg.param
.set(Param::MimeType, "application/autocrypt-setup");

View File

@@ -178,6 +178,7 @@ impl BackupProvider {
}
info!(context, "Received valid backup authentication token.");
// Emit a nonzero progress so that UIs can display smth like "Transferring...".
context.emit_event(EventType::ImexProgress(1));
let blobdir = BlobDirContents::new(&context).await?;
@@ -309,6 +310,10 @@ pub async fn get_backup2(
let mut file_size_buf = [0u8; 8];
recv_stream.read_exact(&mut file_size_buf).await?;
let file_size = u64::from_be_bytes(file_size_buf);
info!(context, "Received backup file size.");
// Emit a nonzero progress so that UIs can display smth like "Transferring...".
context.emit_event(EventType::ImexProgress(1));
import_backup_stream(context, recv_stream, file_size, passphrase)
.await
.context("Failed to import backup from QUIC stream")?;
@@ -389,7 +394,8 @@ mod tests {
let file = ctx0.get_blobdir().join("hello.txt");
fs::write(&file, "i am attachment").await.unwrap();
let mut msg = Message::new(Viewtype::File);
msg.set_file(file.to_str().unwrap(), Some("text/plain"));
msg.set_file_and_deduplicate(&ctx0, &file, Some("hello.txt"), Some("text/plain"))
.unwrap();
send_msg(&ctx0, self_chat.id, &mut msg).await.unwrap();
// Prepare to transfer backup.
@@ -423,7 +429,12 @@ mod tests {
let msg = Message::load_from_db(&ctx1, *msgid).await.unwrap();
let path = msg.get_file(&ctx1).unwrap();
assert_eq!(path.with_file_name("hello.txt"), path);
assert_eq!(
// That's the hash of the file:
path.with_file_name("ac1d2d284757656a8d41dc40aae4136.txt"),
path
);
assert_eq!("hello.txt", msg.get_filename().unwrap());
let text = fs::read_to_string(&path).await.unwrap();
assert_eq!(text, "i am attachment");
@@ -434,12 +445,14 @@ mod tests {
assert!(msg.save_file(&ctx1, &path).await.is_err());
// Check that both received the ImexProgress events.
ctx0.evtracker
.get_matching(|ev| matches!(ev, EventType::ImexProgress(1000)))
.await;
ctx1.evtracker
.get_matching(|ev| matches!(ev, EventType::ImexProgress(1000)))
.await;
for ctx in [&ctx0, &ctx1] {
ctx.evtracker
.get_matching(|ev| matches!(ev, EventType::ImexProgress(1)))
.await;
ctx.evtracker
.get_matching(|ev| matches!(ev, EventType::ImexProgress(1000)))
.await;
}
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]

View File

@@ -30,7 +30,39 @@ use crate::tools::{self, time_elapsed};
pub(crate) trait DcKey: Serialize + Deserializable + PublicKeyTrait + Clone {
/// Create a key from some bytes.
fn from_slice(bytes: &[u8]) -> Result<Self> {
Ok(<Self as Deserializable>::from_bytes(Cursor::new(bytes))?)
let res = <Self as Deserializable>::from_bytes(Cursor::new(bytes));
if let Ok(res) = res {
return Ok(res);
}
// Workaround for keys imported using
// Delta Chat core < 1.0.0.
// Old Delta Chat core had a bug
// that resulted in treating CRC24 checksum
// as part of the key when reading ASCII Armor.
// Some users that started using Delta Chat in 2019
// have such corrupted keys with garbage bytes at the end.
//
// Garbage is at least 3 bytes long
// and may be longer due to padding
// at the end of the real key data
// and importing the key multiple times.
//
// If removing 10 bytes is not enough,
// the key is likely actually corrupted.
for garbage_bytes in 3..std::cmp::min(bytes.len(), 10) {
let res = <Self as Deserializable>::from_bytes(Cursor::new(
bytes
.get(..bytes.len().saturating_sub(garbage_bytes))
.unwrap_or_default(),
));
if let Ok(res) = res {
return Ok(res);
}
}
// Removing garbage bytes did not help, return the error.
Ok(res?)
}
/// Create a key from a base64 string.
@@ -565,6 +597,36 @@ i8pcjGO+IZffvyZJVRWfVooBJmWWbPB1pueo3tx8w3+fcuzpxz+RLFKaPyqXO+dD
}
}
/// Tests workaround for Delta Chat core < 1.0.0
/// which parsed CRC24 at the end of ASCII Armor
/// as the part of the key.
/// Depending on the alignment and the number of
/// `=` characters at the end of the key,
/// this resulted in various number of garbage
/// octets at the end of the key, starting from 3 octets,
/// but possibly 4 or 5 and maybe more octets
/// if the key is imported or transferred
/// using Autocrypt Setup Message multiple times.
#[test]
fn test_ignore_trailing_garbage() {
// Test several variants of garbage.
for garbage in [
b"\x02\xfc\xaa\x38\x4b\x5c".as_slice(),
b"\x02\xfc\xaa".as_slice(),
b"\x01\x02\x03\x04\x05".as_slice(),
] {
let private_key = KEYPAIR.secret.clone();
let mut binary = DcKey::to_bytes(&private_key);
binary.extend(garbage);
let private_key2 =
SignedSecretKey::from_slice(&binary).expect("Failed to ignore garbage");
assert_eq!(private_key.dc_fingerprint(), private_key2.dc_fingerprint());
}
}
#[test]
fn test_base64_roundtrip() {
let key = KEYPAIR.public.clone();

View File

@@ -707,9 +707,6 @@ pub(crate) async fn save(
))?;
if timestamp > newest_timestamp {
// okay to drop, as we use cached prepared statements
drop(stmt_test);
drop(stmt_insert);
newest_timestamp = timestamp;
newest_location_id = Some(u32::try_from(conn.last_insert_rowid())?);
}
@@ -1077,7 +1074,7 @@ Content-Disposition: attachment; filename="location.kml"
let file = alice.get_blobdir().join(file_name);
tokio::fs::write(&file, bytes).await?;
let mut msg = Message::new(Viewtype::Image);
msg.set_file(file.to_str().unwrap(), None);
msg.set_file_and_deduplicate(&alice, &file, Some("logo.png"), None)?;
let sent = alice.send_msg(alice_chat.id, &mut msg).await;
let alice_msg = Message::load_from_db(&alice, sent.sender_msg_id).await?;
assert_eq!(alice_msg.has_location(), false);
@@ -1130,6 +1127,10 @@ Content-Disposition: attachment; filename="location.kml"
assert_eq!(get_range(alice, None, None, 0, 0).await?.len(), 1);
assert_eq!(get_range(bob, None, None, 0, 0).await?.len(), 1);
// Location-only messages are "auto-generated", but they mustn't make the contact a bot.
let contact = bob.add_or_lookup_contact(alice).await;
assert!(!contact.is_bot());
// Day later Bob removes location.
SystemTime::shift(Duration::from_secs(86400));
delete_expired(alice, time()).await?;

View File

@@ -293,13 +293,7 @@ impl MsgId {
ret += ", Location sent";
}
let e2ee_errors = msg.param.get_int(Param::ErroneousE2ee).unwrap_or_default();
if 0 != e2ee_errors {
if 0 != e2ee_errors & 0x2 {
ret += ", Encrypted, no valid signature";
}
} else if 0 != msg.param.get_int(Param::GuaranteeE2ee).unwrap_or_default() {
if 0 != msg.param.get_int(Param::GuaranteeE2ee).unwrap_or_default() {
ret += ", Encrypted";
}
@@ -348,7 +342,7 @@ impl MsgId {
let server_urls = Self::get_info_server_urls(context, msg.rfc724_mid).await?;
for server_url in server_urls {
// Format as RFC 5092 relative IMAP URL.
ret += &format!("\n{server_url}");
ret += &format!("\nServer-URL: {server_url}");
}
}
let hop_info = self.hop_info(context).await?;
@@ -476,6 +470,7 @@ pub struct Message {
/// `In-Reply-To` header value.
pub(crate) in_reply_to: Option<String>,
pub(crate) is_dc_message: MessengerMessage,
pub(crate) original_msg_id: MsgId,
pub(crate) mime_modified: bool,
pub(crate) chat_blocked: Blocked,
pub(crate) location_id: u32,
@@ -542,6 +537,7 @@ impl Message {
" m.download_state AS download_state,",
" m.error AS error,",
" m.msgrmsg AS msgrmsg,",
" m.starred AS original_msg_id,",
" m.mime_modified AS mime_modified,",
" m.txt AS txt,",
" m.subject AS subject,",
@@ -598,6 +594,7 @@ impl Message {
error: Some(row.get::<_, String>("error")?)
.filter(|error| !error.is_empty()),
is_dc_message: row.get("msgrmsg")?,
original_msg_id: row.get("original_msg_id")?,
mime_modified: row.get("mime_modified")?,
text,
subject: row.get("subject")?,
@@ -626,8 +623,8 @@ impl Message {
pub fn get_filemime(&self) -> Option<String> {
if let Some(m) = self.param.get(Param::MimeType) {
return Some(m.to_string());
} else if let Some(file) = self.param.get(Param::File) {
if let Some((_, mime)) = guess_msgtype_from_suffix(Path::new(file)) {
} else if self.param.exists(Param::File) {
if let Some((_, mime)) = guess_msgtype_from_suffix(self) {
return Some(mime.to_string());
}
// we have a file but no mimetype, let's use a generic one
@@ -724,7 +721,7 @@ impl Message {
/// `contact_id` set to [`ContactId::SELF`].
///
/// `latitude` is the North-south position of the location.
/// `longitutde` is the East-west position of the location.
/// `longitude` is the East-west position of the location.
///
/// [`location::set()`]: crate::location::set
/// [`send_locations_to_chat()`]: crate::location::send_locations_to_chat
@@ -953,18 +950,6 @@ impl Message {
cmd != SystemMessage::Unknown
}
/// Whether the message is still being created.
///
/// Messages with attachments might be created before the
/// attachment is ready. In this case some more restrictions on
/// the attachment apply, e.g. if the file to be attached is still
/// being written to or otherwise will still change it can not be
/// copied to the blobdir. Thus those attachments need to be
/// created immediately in the blobdir with a valid filename.
pub fn is_increation(&self) -> bool {
self.viewtype.has_file() && self.state == MessageState::OutPreparing
}
/// Returns true if the message is an Autocrypt Setup Message.
pub fn is_setupmessage(&self) -> bool {
if self.viewtype != Viewtype::File {
@@ -1100,18 +1085,62 @@ impl Message {
self.param.set_optional(Param::MimeType, filemime);
}
/// Creates a new blob and sets it as a file associated with a message.
pub async fn set_file_from_bytes(
/// Sets the file associated with a message, deduplicating files with the same name.
///
/// If `name` is Some, it is used as the file name
/// and the actual current name of the file is ignored.
///
/// If the source file is already in the blobdir, it will be renamed,
/// otherwise it will be copied to the blobdir first.
///
/// In order to deduplicate files that contain the same data,
/// the file will be named `<hash>.<extension>`, e.g. `ce940175885d7b78f7b7e9f1396611f.jpg`.
///
/// NOTE:
/// - This function will rename the file. To get the new file path, call `get_file()`.
/// - The file must not be modified after this function was called.
pub fn set_file_and_deduplicate(
&mut self,
context: &Context,
suggested_name: &str,
file: &Path,
name: Option<&str>,
filemime: Option<&str>,
) -> Result<()> {
let name = if let Some(name) = name {
name.to_string()
} else {
file.file_name()
.map(|s| s.to_string_lossy().to_string())
.unwrap_or_else(|| "unknown_file".to_string())
};
let blob = BlobObject::create_and_deduplicate(context, file, &name)?;
self.param.set(Param::File, blob.as_name());
self.param.set(Param::Filename, name);
self.param.set_optional(Param::MimeType, filemime);
Ok(())
}
/// Creates a new blob and sets it as a file associated with a message.
///
/// In order to deduplicate files that contain the same data,
/// the file will be named `<hash>.<extension>`, e.g. `ce940175885d7b78f7b7e9f1396611f.jpg`.
///
/// NOTE: The file must not be modified after this function was called.
pub fn set_file_from_bytes(
&mut self,
context: &Context,
name: &str,
data: &[u8],
filemime: Option<&str>,
) -> Result<()> {
let blob = BlobObject::create(context, suggested_name, data).await?;
self.param.set(Param::Filename, suggested_name);
let blob = BlobObject::create_and_deduplicate_from_bytes(context, data, name)?;
self.param.set(Param::Filename, name);
self.param.set(Param::File, blob.as_name());
self.param.set_optional(Param::MimeType, filemime);
Ok(())
}
@@ -1124,12 +1153,13 @@ impl Message {
);
let vcard = contact::make_vcard(context, contacts).await?;
self.set_file_from_bytes(context, "vcard.vcf", vcard.as_bytes(), None)
.await
}
/// Updates message state from the vCard attachment.
pub(crate) async fn try_set_vcard(&mut self, context: &Context, path: &Path) -> Result<()> {
let vcard = fs::read(path).await.context("Could not read {path}")?;
let vcard = fs::read(path)
.await
.with_context(|| format!("Could not read {path:?}"))?;
if let Some(summary) = get_vcard_summary(&vcard) {
self.param.set(Param::Summary1, summary);
} else {
@@ -1272,6 +1302,35 @@ impl Message {
Ok(None)
}
/// Returns original message ID for message from "Saved Messages".
pub async fn get_original_msg_id(&self, context: &Context) -> Result<Option<MsgId>> {
if !self.original_msg_id.is_special() {
if let Some(msg) = Message::load_from_db_optional(context, self.original_msg_id).await?
{
return if msg.chat_id.is_trash() {
Ok(None)
} else {
Ok(Some(msg.id))
};
}
}
Ok(None)
}
/// Check if the message was saved and returns the corresponding message inside "Saved Messages".
/// UI can use this to show a symbol beside the message, indicating it was saved.
/// The message can be un-saved by deleting the returned message.
pub async fn get_saved_msg_id(&self, context: &Context) -> Result<Option<MsgId>> {
let res: Option<MsgId> = context
.sql
.query_get_value(
"SELECT id FROM msgs WHERE starred=? AND chat_id!=?",
(self.id, DC_CHAT_ID_TRASH),
)
.await?;
Ok(res)
}
/// Force the message to be sent in plain text.
pub fn force_plaintext(&mut self) {
self.param.set_int(Param::ForcePlaintext, 1);
@@ -1451,7 +1510,14 @@ pub async fn get_msg_read_receipts(
.await
}
pub(crate) fn guess_msgtype_from_suffix(path: &Path) -> Option<(Viewtype, &str)> {
pub(crate) fn guess_msgtype_from_suffix(msg: &Message) -> Option<(Viewtype, &'static str)> {
msg.param
.get(Param::Filename)
.or_else(|| msg.param.get(Param::File))
.and_then(|file| guess_msgtype_from_path_suffix(Path::new(file)))
}
pub(crate) fn guess_msgtype_from_path_suffix(path: &Path) -> Option<(Viewtype, &'static str)> {
let extension: &str = &path.extension()?.to_str()?.to_lowercase();
let info = match extension {
// before using viewtype other than Viewtype::File,
@@ -1625,15 +1691,15 @@ pub async fn delete_msgs(context: &Context, msg_ids: &[MsgId]) -> Result<()> {
modified_chat_ids.insert(msg.chat_id);
let target = context.get_delete_msgs_target().await?;
let update_db = |conn: &mut rusqlite::Connection| {
conn.execute(
let update_db = |trans: &mut rusqlite::Transaction| {
trans.execute(
"UPDATE imap SET target=? WHERE rfc724_mid=?",
(target, msg.rfc724_mid),
)?;
conn.execute("DELETE FROM smtp WHERE msg_id=?", (msg_id,))?;
trans.execute("DELETE FROM smtp WHERE msg_id=?", (msg_id,))?;
Ok(())
};
if let Err(e) = context.sql.call_write(update_db).await {
if let Err(e) = context.sql.transaction(update_db).await {
error!(context, "delete_msgs: failed to update db: {e:#}.");
res = Err(e);
continue;
@@ -1655,7 +1721,7 @@ pub async fn delete_msgs(context: &Context, msg_ids: &[MsgId]) -> Result<()> {
res?;
for modified_chat_id in modified_chat_ids {
context.emit_msgs_changed(modified_chat_id, MsgId::new(0));
context.emit_msgs_changed_without_msg_id(modified_chat_id);
chatlist_events::emit_chatlist_item_changed(context, modified_chat_id);
}
@@ -1685,12 +1751,12 @@ pub async fn markseen_msgs(context: &Context, msg_ids: Vec<MsgId>) -> Result<()>
.set_config_internal(Config::LastMsgId, Some(&last_msg_id.to_u32().to_string()))
.await?;
let msgs = context
.sql
.query_map(
&format!(
let mut msgs = Vec::with_capacity(msg_ids.len());
for &id in &msg_ids {
if let Some(msg) = context
.sql
.query_row_optional(
"SELECT
m.id AS id,
m.chat_id AS chat_id,
m.state AS state,
m.download_state as download_state,
@@ -1701,39 +1767,39 @@ pub async fn markseen_msgs(context: &Context, msg_ids: Vec<MsgId>) -> Result<()>
c.archived AS archived,
c.blocked AS blocked
FROM msgs m LEFT JOIN chats c ON c.id=m.chat_id
WHERE m.id IN ({}) AND m.chat_id>9",
sql::repeat_vars(msg_ids.len())
),
rusqlite::params_from_iter(&msg_ids),
|row| {
let id: MsgId = row.get("id")?;
let chat_id: ChatId = row.get("chat_id")?;
let state: MessageState = row.get("state")?;
let download_state: DownloadState = row.get("download_state")?;
let param: Params = row.get::<_, String>("param")?.parse().unwrap_or_default();
let from_id: ContactId = row.get("from_id")?;
let rfc724_mid: String = row.get("rfc724_mid")?;
let visibility: ChatVisibility = row.get("archived")?;
let blocked: Option<Blocked> = row.get("blocked")?;
let ephemeral_timer: EphemeralTimer = row.get("ephemeral_timer")?;
Ok((
(
id,
chat_id,
state,
download_state,
param,
from_id,
rfc724_mid,
visibility,
blocked.unwrap_or_default(),
),
ephemeral_timer,
))
},
|rows| rows.collect::<Result<Vec<_>, _>>().map_err(Into::into),
)
.await?;
WHERE m.id=? AND m.chat_id>9",
(id,),
|row| {
let chat_id: ChatId = row.get("chat_id")?;
let state: MessageState = row.get("state")?;
let download_state: DownloadState = row.get("download_state")?;
let param: Params = row.get::<_, String>("param")?.parse().unwrap_or_default();
let from_id: ContactId = row.get("from_id")?;
let rfc724_mid: String = row.get("rfc724_mid")?;
let visibility: ChatVisibility = row.get("archived")?;
let blocked: Option<Blocked> = row.get("blocked")?;
let ephemeral_timer: EphemeralTimer = row.get("ephemeral_timer")?;
Ok((
(
id,
chat_id,
state,
download_state,
param,
from_id,
rfc724_mid,
visibility,
blocked.unwrap_or_default(),
),
ephemeral_timer,
))
},
)
.await?
{
msgs.push(msg);
}
}
if msgs
.iter()
@@ -2106,6 +2172,9 @@ pub enum Viewtype {
Gif = 21,
/// Message containing a sticker, similar to image.
/// NB: When sending, the message viewtype may be changed to `Image` by some heuristics like
/// checking for transparent pixels. Use `Message::force_sticker()` to disable them.
///
/// If possible, the ui should display the image without borders in a transparent way.
/// A click on a sticker will offer to install the sticker set in some future.
Sticker = 23,
@@ -2181,7 +2250,8 @@ mod tests {
use super::*;
use crate::chat::{
self, add_contact_to_chat, marknoticed_chat, send_text_msg, ChatItem, ProtectionStatus,
self, add_contact_to_chat, forward_msgs, marknoticed_chat, save_msgs, send_text_msg,
ChatItem, ProtectionStatus,
};
use crate::chatlist::Chatlist;
use crate::config::Config;
@@ -2193,51 +2263,19 @@ mod tests {
#[test]
fn test_guess_msgtype_from_suffix() {
assert_eq!(
guess_msgtype_from_suffix(Path::new("foo/bar-sth.mp3")),
guess_msgtype_from_path_suffix(Path::new("foo/bar-sth.mp3")),
Some((Viewtype::Audio, "audio/mpeg"))
);
assert_eq!(
guess_msgtype_from_suffix(Path::new("foo/file.html")),
guess_msgtype_from_path_suffix(Path::new("foo/file.html")),
Some((Viewtype::File, "text/html"))
);
assert_eq!(
guess_msgtype_from_suffix(Path::new("foo/file.xdc")),
guess_msgtype_from_path_suffix(Path::new("foo/file.xdc")),
Some((Viewtype::Webxdc, "application/webxdc+zip"))
);
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_prepare_message_and_send() {
let d = test::TestContext::new().await;
let ctx = &d.ctx;
ctx.set_config(Config::ConfiguredAddr, Some("self@example.com"))
.await
.unwrap();
let chat = d.create_chat_with_contact("", "dest@example.com").await;
let mut msg = Message::new(Viewtype::Text);
let msg_id = chat::prepare_msg(ctx, chat.id, &mut msg).await.unwrap();
let _msg2 = Message::load_from_db(ctx, msg_id).await.unwrap();
assert_eq!(_msg2.get_filemime(), None);
}
/// Tests that message can be prepared even if account has no configured address.
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_prepare_not_configured() {
let d = test::TestContext::new().await;
let ctx = &d.ctx;
let chat = d.create_chat_with_contact("", "dest@example.com").await;
let mut msg = Message::new(Viewtype::Text);
assert!(chat::prepare_msg(ctx, chat.id, &mut msg).await.is_ok());
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_parse_webrtc_instance() {
let (webrtc_type, url) = Message::parse_webrtc_instance("basicwebrtc:https://foo/bar");
@@ -2357,9 +2395,9 @@ mod tests {
let mut msg = Message::new_text("Quoted message".to_string());
// Prepare message for sending, so it gets a Message-Id.
// Send message, so it gets a Message-Id.
assert!(msg.rfc724_mid.is_empty());
let msg_id = chat::prepare_msg(ctx, chat.id, &mut msg).await.unwrap();
let msg_id = chat::send_msg(ctx, chat.id, &mut msg).await.unwrap();
let msg = Message::load_from_db(ctx, msg_id).await.unwrap();
assert!(!msg.rfc724_mid.is_empty());
@@ -2522,6 +2560,41 @@ mod tests {
);
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_get_original_msg_id() -> Result<()> {
let alice = TestContext::new_alice().await;
let bob = TestContext::new_bob().await;
// normal sending of messages does not have an original ID
let one2one_chat = alice.create_chat(&bob).await;
let sent = alice.send_text(one2one_chat.id, "foo").await;
let orig_msg = Message::load_from_db(&alice, sent.sender_msg_id).await?;
assert!(orig_msg.get_original_msg_id(&alice).await?.is_none());
assert!(orig_msg.parent(&alice).await?.is_none());
assert!(orig_msg.quoted_message(&alice).await?.is_none());
// forwarding to "Saved Messages", the message gets the original ID attached
let self_chat = alice.get_self_chat().await;
save_msgs(&alice, &[sent.sender_msg_id]).await?;
let saved_msg = alice.get_last_msg_in(self_chat.get_id()).await;
assert_ne!(saved_msg.get_id(), orig_msg.get_id());
assert_eq!(
saved_msg.get_original_msg_id(&alice).await?.unwrap(),
orig_msg.get_id()
);
assert!(saved_msg.parent(&alice).await?.is_none());
assert!(saved_msg.quoted_message(&alice).await?.is_none());
// forwarding from "Saved Messages" back to another chat, detaches original ID
forward_msgs(&alice, &[saved_msg.get_id()], one2one_chat.get_id()).await?;
let forwarded_msg = alice.get_last_msg_in(one2one_chat.get_id()).await;
assert_ne!(forwarded_msg.get_id(), saved_msg.get_id());
assert_ne!(forwarded_msg.get_id(), orig_msg.get_id());
assert!(forwarded_msg.get_original_msg_id(&alice).await?.is_none());
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_markseen_msgs() -> Result<()> {
let alice = TestContext::new_alice().await;
@@ -2604,8 +2677,7 @@ mod tests {
let file_bytes = include_bytes!("../test-data/image/screenshot.png");
let mut msg = Message::new(Viewtype::Image);
msg.set_file_from_bytes(bob, "a.jpg", file_bytes, None)
.await?;
msg.set_file_from_bytes(bob, "a.jpg", file_bytes, None)?;
let sent_msg = bob.send_msg(bob_chat_id, &mut msg).await;
let msg = alice.recv_msg(&sent_msg).await;
assert_eq!(msg.download_state, DownloadState::Available);
@@ -2674,8 +2746,7 @@ mod tests {
let file_bytes = include_bytes!("../test-data/image/screenshot.png");
let mut msg = Message::new(Viewtype::Image);
msg.set_file_from_bytes(bob, "a.jpg", file_bytes, None)
.await?;
msg.set_file_from_bytes(bob, "a.jpg", file_bytes, None)?;
let sent_msg = bob.send_msg(bob_chat_id, &mut msg).await;
let msg = alice.recv_msg(&sent_msg).await;
assert_eq!(msg.download_state, DownloadState::Available);
@@ -2754,6 +2825,29 @@ mod tests {
async fn test_is_bot() -> Result<()> {
let alice = TestContext::new_alice().await;
// Alice receives an auto-generated non-chat message.
//
// This could be a holiday notice,
// in which case the message should be marked as bot-generated,
// but the contact should not.
receive_imf(
&alice,
b"From: Claire <claire@example.com>\n\
To: alice@example.org\n\
Message-ID: <789@example.com>\n\
Auto-Submitted: auto-generated\n\
Date: Fri, 29 Jan 2021 21:37:55 +0000\n\
\n\
hello\n",
false,
)
.await?;
let msg = alice.get_last_msg().await;
assert_eq!(msg.get_text(), "hello".to_string());
assert!(msg.is_bot());
let contact = Contact::get_by_id(&alice, msg.from_id).await?;
assert!(!contact.is_bot());
// Alice receives a message from Bob the bot.
receive_imf(
&alice,

View File

@@ -1,6 +1,7 @@
//! # MIME message production.
use std::collections::HashSet;
use std::path::Path;
use anyhow::{bail, Context as _, Result};
use base64::Engine as _;
@@ -66,8 +67,36 @@ pub struct MimeFactory {
selfstatus: String,
/// Vector of pairs of recipient name and address
recipients: Vec<(String, String)>,
/// Vector of actual recipient addresses.
///
/// This is the list of addresses the message should be sent to.
/// It is not the same as the `To` header,
/// because in case of "member removed" message
/// removed member is in the recipient list,
/// but not in the `To` header.
/// In case of broadcast lists there are multiple recipients,
/// but the `To` header has no members.
///
/// If `bcc_self` configuration is enabled,
/// this list will be extended with own address later,
/// but `MimeFactory` is not responsible for this.
recipients: Vec<String>,
/// Vector of pairs of recipient name and address that goes into the `To` field.
///
/// The list of actual message recipient addresses may be different,
/// e.g. if members are hidden for broadcast lists.
to: Vec<(String, String)>,
/// Vector of pairs of past group member names and addresses.
past_members: Vec<(String, String)>,
/// Timestamps of the members in the same order as in the `recipients`
/// followed by `past_members`.
///
/// If this is not empty, its length
/// should be the sum of `recipients` and `past_members` length.
member_timestamps: Vec<i64>,
timestamp: i64,
loaded: Loaded,
@@ -126,8 +155,10 @@ fn new_address_with_name(name: &str, address: String) -> Address {
impl MimeFactory {
pub async fn from_msg(context: &Context, msg: Message) -> Result<MimeFactory> {
let now = time();
let chat = Chat::load_from_db(context, msg.chat_id).await?;
let attach_profile_data = Self::should_attach_profile_data(&msg);
let undisclosed_recipients = chat.typ == Chattype::Broadcast;
let from_addr = context.get_primary_self_addr().await?;
let config_displayname = context
@@ -145,47 +176,101 @@ impl MimeFactory {
(name, None)
};
let mut recipients = Vec::with_capacity(5);
let mut recipients = Vec::new();
let mut to = Vec::new();
let mut past_members = Vec::new();
let mut member_timestamps = Vec::new();
let mut recipient_ids = HashSet::new();
let mut req_mdn = false;
if chat.is_self_talk() {
if msg.param.get_cmd() == SystemMessage::AutocryptSetupMessage {
recipients.push((from_displayname.to_string(), from_addr.to_string()));
recipients.push(from_addr.to_string());
to.push((from_displayname.to_string(), from_addr.to_string()));
}
} else if chat.is_mailing_list() {
let list_post = chat
.param
.get(Param::ListPost)
.context("Can't write to mailinglist without ListPost param")?;
recipients.push(("".to_string(), list_post.to_string()));
to.push(("".to_string(), list_post.to_string()));
recipients.push(list_post.to_string());
} else {
let email_to_remove = if msg.param.get_cmd() == SystemMessage::MemberRemovedFromGroup {
msg.param.get(Param::Arg)
} else {
None
};
context
.sql
.query_map(
"SELECT c.authname, c.addr, c.id \
FROM chats_contacts cc \
LEFT JOIN contacts c ON cc.contact_id=c.id \
WHERE cc.chat_id=? AND cc.contact_id>9;",
(msg.chat_id,),
"SELECT c.authname, c.addr, c.id, cc.add_timestamp, cc.remove_timestamp
FROM chats_contacts cc
LEFT JOIN contacts c ON cc.contact_id=c.id
WHERE cc.chat_id=? AND (cc.contact_id>9 OR (cc.contact_id=1 AND ?))",
(msg.chat_id, chat.typ == Chattype::Group),
|row| {
let authname: String = row.get(0)?;
let addr: String = row.get(1)?;
let id: ContactId = row.get(2)?;
Ok((authname, addr, id))
let add_timestamp: i64 = row.get(3)?;
let remove_timestamp: i64 = row.get(4)?;
Ok((authname, addr, id, add_timestamp, remove_timestamp))
},
|rows| {
let mut past_member_timestamps = Vec::new();
for row in rows {
let (authname, addr, id) = row?;
if !recipients_contain_addr(&recipients, &addr) {
let name = match attach_profile_data {
true => authname,
false => "".to_string(),
};
recipients.push((name, addr));
let (authname, addr, id, add_timestamp, remove_timestamp) = row?;
let addr = if id == ContactId::SELF {
from_addr.to_string()
} else {
addr
};
let name = match attach_profile_data {
true => authname,
false => "".to_string(),
};
if add_timestamp >= remove_timestamp {
if !recipients_contain_addr(&to, &addr) {
recipients.push(addr.clone());
if !undisclosed_recipients {
to.push((name, addr));
member_timestamps.push(add_timestamp);
}
}
recipient_ids.insert(id);
} else if remove_timestamp.saturating_add(60 * 24 * 3600) > now {
// Row is a tombstone,
// member is not actually part of the group.
if !recipients_contain_addr(&past_members, &addr) {
if let Some(email_to_remove) = email_to_remove {
if email_to_remove == addr {
// This is a "member removed" message,
// we need to notify removed member
// that it was removed.
recipients.push(addr.clone());
}
}
if !undisclosed_recipients {
past_members.push((name, addr));
past_member_timestamps.push(remove_timestamp);
}
}
}
recipient_ids.insert(id);
}
debug_assert!(member_timestamps.len() >= to.len());
if to.len() > 1 {
if let Some(position) = to.iter().position(|(_, x)| x == &from_addr) {
to.remove(position);
member_timestamps.remove(position);
}
}
member_timestamps.extend(past_member_timestamps);
Ok(())
},
)
@@ -226,12 +311,19 @@ impl MimeFactory {
};
let attach_selfavatar = Self::should_attach_selfavatar(context, &msg).await;
debug_assert!(
member_timestamps.is_empty()
|| to.len() + past_members.len() == member_timestamps.len()
);
let factory = MimeFactory {
from_addr,
from_displayname,
sender_displayname,
selfstatus,
recipients,
to,
past_members,
member_timestamps,
timestamp: msg.timestamp_sort,
loaded: Loaded::Message { msg, chat },
in_reply_to,
@@ -259,7 +351,10 @@ impl MimeFactory {
from_displayname: "".to_string(),
sender_displayname: None,
selfstatus: "".to_string(),
recipients: vec![("".to_string(), contact.get_addr().to_string())],
recipients: vec![contact.get_addr().to_string()],
to: vec![("".to_string(), contact.get_addr().to_string())],
past_members: vec![],
member_timestamps: vec![],
timestamp,
loaded: Loaded::Mdn {
rfc724_mid,
@@ -283,11 +378,7 @@ impl MimeFactory {
let self_addr = context.get_primary_self_addr().await?;
let mut res = Vec::new();
for (_, addr) in self
.recipients
.iter()
.filter(|(_, addr)| addr != &self_addr)
{
for addr in self.recipients.iter().filter(|&addr| *addr != self_addr) {
res.push((Peerstate::from_addr(context, addr).await?, addr.clone()));
}
@@ -356,7 +447,7 @@ impl MimeFactory {
let gossip_period = context.get_config_i64(Config::GossipPeriod).await?;
// `gossip_period == 0` is a special case for testing,
// enabling gossip in every message.
// Othewise "smeared timestamps" may result in the condition
// Otherwise "smeared timestamps" may result in the condition
// to fail even if the clock is monotonic.
if gossip_period == 0 || time() >= gossiped_timestamp + gossip_period {
Ok(true)
@@ -475,10 +566,7 @@ impl MimeFactory {
}
pub fn recipients(&self) -> Vec<String> {
self.recipients
.iter()
.map(|(_, addr)| addr.clone())
.collect()
self.recipients.clone()
}
/// Consumes a `MimeFactory` and renders it into a message which is then stored in
@@ -488,46 +576,33 @@ impl MimeFactory {
let from = new_address_with_name(&self.from_displayname, self.from_addr.clone());
let undisclosed_recipients = match &self.loaded {
Loaded::Message { chat, .. } => chat.typ == Chattype::Broadcast,
Loaded::Mdn { .. } => false,
};
let mut to = Vec::new();
if undisclosed_recipients {
for (name, addr) in &self.to {
if name.is_empty() {
to.push(Address::new_mailbox(addr.clone()));
} else {
to.push(new_address_with_name(name, addr.clone()));
}
}
let mut past_members = Vec::new(); // Contents of `Chat-Group-Past-Members` header.
for (name, addr) in &self.past_members {
if name.is_empty() {
past_members.push(Address::new_mailbox(addr.clone()));
} else {
past_members.push(new_address_with_name(name, addr.clone()));
}
}
debug_assert!(
self.member_timestamps.is_empty()
|| to.len() + past_members.len() == self.member_timestamps.len()
);
if to.is_empty() {
to.push(Address::new_group(
"hidden-recipients".to_string(),
Vec::new(),
));
} else {
let email_to_remove = match &self.loaded {
Loaded::Message { msg, .. } => {
if msg.param.get_cmd() == SystemMessage::MemberRemovedFromGroup {
msg.param.get(Param::Arg)
} else {
None
}
}
Loaded::Mdn { .. } => None,
};
for (name, addr) in &self.recipients {
if let Some(email_to_remove) = email_to_remove {
if email_to_remove == addr {
continue;
}
}
if name.is_empty() {
to.push(Address::new_mailbox(addr.clone()));
} else {
to.push(new_address_with_name(name, addr.clone()));
}
}
if to.is_empty() {
to.push(from.clone());
}
}
// Start with Internet Message Format headers in the order of the standard example
@@ -540,6 +615,32 @@ impl MimeFactory {
headers.push(Header::new_with_value("Sender".into(), vec![sender]).unwrap());
}
headers.push(Header::new_with_value("To".into(), to.clone()).unwrap());
if !past_members.is_empty() {
headers.push(
Header::new_with_value("Chat-Group-Past-Members".into(), past_members.clone())
.unwrap(),
);
}
let chat_memberlist_is_stale = if let Loaded::Message { chat, .. } = &self.loaded {
chat.member_list_is_stale(context).await?
} else {
false
};
if !self.member_timestamps.is_empty() && !chat_memberlist_is_stale {
headers.push(
Header::new_with_value(
"Chat-Group-Member-Timestamps".into(),
self.member_timestamps
.iter()
.map(|ts| ts.to_string())
.collect::<Vec<String>>()
.join(" "),
)
.unwrap(),
);
}
let subject_str = self.subject_str(context).await?;
let encoded_subject = if subject_str
@@ -652,7 +753,9 @@ impl MimeFactory {
let peerstates = self.peerstates_for_recipients(context).await?;
let is_encrypted = !self.should_force_plaintext()
&& encrypt_helper.should_encrypt(context, e2ee_guaranteed, &peerstates)?;
&& encrypt_helper
.should_encrypt(context, e2ee_guaranteed, &peerstates)
.await?;
let is_securejoin_message = if let Loaded::Message { msg, .. } = &self.loaded {
msg.param.get_cmd() == SystemMessage::SecurejoinMessage
} else {
@@ -1047,7 +1150,6 @@ impl MimeFactory {
part.body(text)
}
#[allow(clippy::cognitive_complexity)]
async fn render_message(
&mut self,
context: &Context,
@@ -1369,7 +1471,7 @@ impl MimeFactory {
// add attachment part
if msg.viewtype.has_file() {
let (file_part, _) = build_body_file(context, &msg, "").await?;
let file_part = build_body_file(context, &msg).await?;
parts.push(file_part);
}
@@ -1509,17 +1611,18 @@ pub(crate) fn wrapped_base64_encode(buf: &[u8]) -> String {
.join("\r\n")
}
async fn build_body_file(
context: &Context,
msg: &Message,
base_name: &str,
) -> Result<(PartBuilder, String)> {
async fn build_body_file(context: &Context, msg: &Message) -> Result<PartBuilder> {
let file_name = msg.get_filename().context("msg has no file")?;
let suffix = Path::new(&file_name)
.extension()
.and_then(|e| e.to_str())
.unwrap_or("dat");
let blob = msg
.param
.get_blob(Param::File, context, true)
.get_blob(Param::File, context)
.await?
.context("msg has no file")?;
let suffix = blob.suffix().unwrap_or("dat");
// Get file name to use for sending. For privacy purposes, we do
// not transfer the original filenames eg. for images; these names
@@ -1539,17 +1642,13 @@ async fn build_body_file(
),
Viewtype::Image | Viewtype::Gif => format!(
"image_{}.{}",
if base_name.is_empty() {
chrono::Utc
.timestamp_opt(msg.timestamp_sort, 0)
.single()
.map_or_else(
|| "YY-mm-dd_hh:mm:ss".to_string(),
|ts| ts.format("%Y-%m-%d_%H-%M-%S").to_string(),
)
} else {
base_name.to_string()
},
chrono::Utc
.timestamp_opt(msg.timestamp_sort, 0)
.single()
.map_or_else(
|| "YY-mm-dd_hh:mm:ss".to_string(),
|ts| ts.format("%Y-%m-%d_%H-%M-%S").to_string(),
),
&suffix,
),
Viewtype::Video => format!(
@@ -1563,18 +1662,14 @@ async fn build_body_file(
),
&suffix
),
_ => msg
.param
.get(Param::Filename)
.unwrap_or_else(|| blob.as_file_name())
.to_string(),
_ => file_name,
};
/* check mimetype */
let mimetype: mime::Mime = match msg.param.get(Param::MimeType) {
Some(mtype) => mtype.parse()?,
None => {
if let Some(res) = message::guess_msgtype_from_suffix(blob.as_rel_path()) {
if let Some(res) = message::guess_msgtype_from_suffix(msg) {
res.1.parse()?
} else {
mime::APPLICATION_OCTET_STREAM
@@ -1601,7 +1696,7 @@ async fn build_body_file(
.header(("Content-Transfer-Encoding", "base64"))
.body(encoded_body);
Ok((mail, filename_to_send))
Ok(mail)
}
async fn build_avatar_file(context: &Context, path: &str) -> Result<String> {
@@ -1905,7 +2000,7 @@ mod tests {
)
.await
.unwrap();
let new_msg = incoming_msg_to_reply_msg(
let mut new_msg = incoming_msg_to_reply_msg(
b"Received: (Postfix, from userid 1000); Mon, 4 Dec 2006 14:51:39 +0100 (CET)\n\
From: bob@example.com\n\
To: alice@example.org\n\
@@ -1931,6 +2026,9 @@ mod tests {
Original-Message-ID: <2893@example.com>\n\
Disposition: manual-action/MDN-sent-automatically; displayed\n\
\n", &t).await;
chat::send_msg(&t, new_msg.chat_id, &mut new_msg)
.await
.unwrap();
let mf = MimeFactory::from_msg(&t, new_msg).await.unwrap();
// The subject string should not be "Re: message opened"
assert_eq!("Re: Hello, Bob", mf.subject_str(&t).await.unwrap());
@@ -2077,7 +2175,7 @@ mod tests {
let mut new_msg = Message::new_text("Hi".to_string());
new_msg.chat_id = chat_id;
chat::prepare_msg(&t, chat_id, &mut new_msg).await.unwrap();
chat::send_msg(&t, chat_id, &mut new_msg).await.unwrap();
let mf = MimeFactory::from_msg(&t, new_msg).await.unwrap();
@@ -2134,7 +2232,7 @@ mod tests {
) -> String {
let t = TestContext::new_alice().await;
let mut new_msg = incoming_msg_to_reply_msg(imf_raw, &t).await;
let incoming_msg = get_chat_msg(&t, new_msg.chat_id, 0, 2).await;
let incoming_msg = get_chat_msg(&t, new_msg.chat_id, 0, 1).await;
if delete_original_msg {
incoming_msg.id.trash(&t, false).await.unwrap();
@@ -2164,6 +2262,9 @@ mod tests {
new_msg.set_quote(&t, Some(&incoming_msg)).await.unwrap();
}
chat::send_msg(&t, new_msg.chat_id, &mut new_msg)
.await
.unwrap();
let mf = MimeFactory::from_msg(&t, new_msg).await.unwrap();
mf.subject_str(&t).await.unwrap()
}
@@ -2184,9 +2285,6 @@ mod tests {
let mut new_msg = Message::new_text("Hi".to_string());
new_msg.chat_id = chat_id;
chat::prepare_msg(context, chat_id, &mut new_msg)
.await
.unwrap();
new_msg
}
@@ -2197,7 +2295,7 @@ mod tests {
let t = TestContext::new_alice().await;
let context = &t;
let msg = incoming_msg_to_reply_msg(
let mut msg = incoming_msg_to_reply_msg(
b"Received: (Postfix, from userid 1000); Mon, 4 Dec 2006 14:51:39 +0100 (CET)\n\
From: Charlie <charlie@example.com>\n\
To: alice@example.org\n\
@@ -2210,6 +2308,7 @@ mod tests {
context,
)
.await;
chat::send_msg(&t, msg.chat_id, &mut msg).await.unwrap();
let mimefactory = MimeFactory::from_msg(&t, msg).await.unwrap();
@@ -2463,8 +2562,9 @@ mod tests {
// Alice creates a group with Bob and Claire and then removes Bob.
let alice = TestContext::new_alice().await;
let claire_addr = "claire@foo.de";
let bob_id = Contact::create(&alice, "Bob", "bob@example.net").await?;
let claire_id = Contact::create(&alice, "Claire", "claire@foo.de").await?;
let claire_id = Contact::create(&alice, "Claire", claire_addr).await?;
let alice_chat_id = create_group_chat(&alice, ProtectionStatus::Unprotected, "foo").await?;
add_contact_to_chat(&alice, alice_chat_id, bob_id).await?;
@@ -2480,10 +2580,17 @@ mod tests {
.get_first_header("To")
.context("no To: header parsed")?;
let to = addrparse_header(to)?;
let mailbox = to
.extract_single_info()
.context("to: field does not contain exactly one address")?;
assert_eq!(mailbox.addr, "bob@example.net");
for to_addr in to.iter() {
match to_addr {
mailparse::MailAddr::Single(ref info) => {
// Addresses should be of existing members (Alice and Bob) and not Claire.
assert_ne!(info.addr, claire_addr);
}
mailparse::MailAddr::Group(_) => {
panic!("Group addresses are not expected here");
}
}
}
Ok(())
}
@@ -2525,8 +2632,7 @@ mod tests {
// Long messages are truncated and MimeMessage::decoded_data is set for them. We need
// decoded_data to check presence of the necessary headers.
msg.set_text("a".repeat(constants::DC_DESIRED_TEXT_LEN + 1));
msg.set_file_from_bytes(&bob, "foo.bar", "content".as_bytes(), None)
.await?;
msg.set_file_from_bytes(&bob, "foo.bar", "content".as_bytes(), None)?;
let sent = bob.send_msg(chat, &mut msg).await;
assert!(msg.get_showpadlock());
assert!(sent.payload.contains("\r\nSubject: [...]\r\n"));
@@ -2543,4 +2649,40 @@ mod tests {
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_dont_remove_self() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
let bob = &tcm.bob().await;
let first_group = alice
.create_group_with_members(ProtectionStatus::Unprotected, "First group", &[bob])
.await;
alice.send_text(first_group, "Hi! I created a group.").await;
remove_contact_from_chat(alice, first_group, ContactId::SELF).await?;
alice.pop_sent_msg().await;
let second_group = alice
.create_group_with_members(ProtectionStatus::Unprotected, "First group", &[bob])
.await;
let sent = alice
.send_text(second_group, "Hi! I created another group.")
.await;
println!("{}", sent.payload);
let mime_message = MimeMessage::from_bytes(alice, sent.payload.as_bytes(), None)
.await
.unwrap();
assert_eq!(
mime_message.get_header(HeaderDef::ChatGroupPastMembers),
None
);
assert_eq!(
mime_message.chat_group_member_timestamps().unwrap().len(),
1 // There is a timestamp for Bob, not for Alice
);
Ok(())
}
}

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@@ -47,7 +47,7 @@ pub(crate) async fn prune_connection_history(context: &Context) -> Result<()> {
Ok(())
}
/// Update the timestamp of the last successfull connection
/// Update the timestamp of the last successful connection
/// to the given `host` and `port`
/// with the given application protocol `alpn`.
///

View File

@@ -6,14 +6,17 @@ use http_body_util::BodyExt;
use hyper_util::rt::TokioIo;
use mime::Mime;
use serde::Serialize;
use tokio::fs;
use crate::blob::BlobObject;
use crate::context::Context;
use crate::net::proxy::ProxyConfig;
use crate::net::session::SessionStream;
use crate::net::tls::wrap_rustls;
use crate::tools::time;
/// HTTP(S) GET response.
#[derive(Debug)]
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct Response {
/// Response body.
pub blob: Vec<u8>,
@@ -90,9 +93,142 @@ where
Ok(sender)
}
/// Retrieves the binary contents of URL using HTTP GET request.
pub async fn read_url_blob(context: &Context, url: &str) -> Result<Response> {
let mut url = url.to_string();
/// Converts the URL to expiration and stale timestamps.
fn http_url_cache_timestamps(url: &str, mimetype: Option<&str>) -> (i64, i64) {
let now = time();
let expires = now + 3600 * 24 * 35;
let stale = if url.ends_with(".xdc") {
// WebXDCs are never stale, they just expire.
expires
} else if mimetype.is_some_and(|s| s.starts_with("image/")) {
// Cache images for 1 day.
//
// As of 2024-12-12 WebXDC icons at <https://webxdc.org/apps/>
// use the same path for all app versions,
// so may change, but it is not critical if outdated icon is displayed.
now + 3600 * 24
} else {
// Revalidate everything else after 1 hour.
//
// This includes HTML, CSS and JS.
now + 3600
};
(expires, stale)
}
/// Places the binary into HTTP cache.
async fn http_cache_put(context: &Context, url: &str, response: &Response) -> Result<()> {
let blob =
BlobObject::create_and_deduplicate_from_bytes(context, response.blob.as_slice(), "")?;
let (expires, stale) = http_url_cache_timestamps(url, response.mimetype.as_deref());
context
.sql
.insert(
"INSERT OR REPLACE INTO http_cache (url, expires, stale, blobname, mimetype, encoding)
VALUES (?, ?, ?, ?, ?, ?)",
(
url,
expires,
stale,
blob.as_name(),
response.mimetype.as_deref().unwrap_or_default(),
response.encoding.as_deref().unwrap_or_default(),
),
)
.await?;
Ok(())
}
/// Retrieves the binary from HTTP cache.
///
/// Also returns if the response is stale and should be revalidated in the background.
async fn http_cache_get(context: &Context, url: &str) -> Result<Option<(Response, bool)>> {
let now = time();
let Some((blob_name, mimetype, encoding, stale_timestamp)) = context
.sql
.query_row_optional(
"SELECT blobname, mimetype, encoding, stale
FROM http_cache WHERE url=? AND expires > ?",
(url, now),
|row| {
let blob_name: String = row.get(0)?;
let mimetype: Option<String> = Some(row.get(1)?).filter(|s: &String| !s.is_empty());
let encoding: Option<String> = Some(row.get(2)?).filter(|s: &String| !s.is_empty());
let stale_timestamp: i64 = row.get(3)?;
Ok((blob_name, mimetype, encoding, stale_timestamp))
},
)
.await?
else {
return Ok(None);
};
let is_stale = now > stale_timestamp;
let blob_object = BlobObject::from_name(context, blob_name)?;
let blob_abs_path = blob_object.to_abs_path();
let blob = match fs::read(blob_abs_path)
.await
.with_context(|| format!("Failed to read blob for {url:?} cache entry."))
{
Ok(blob) => blob,
Err(err) => {
// This should not happen, but user may go into the blobdir and remove files,
// antivirus may delete the file or there may be a bug in housekeeping.
warn!(context, "{err:?}.");
return Ok(None);
}
};
let (expires, _stale) = http_url_cache_timestamps(url, mimetype.as_deref());
let response = Response {
blob,
mimetype,
encoding,
};
// Update expiration timestamp
// to prevent deletion of the file still in use.
//
// If the response is stale, the caller should revalidate it in the background, so update
// `stale` timestamp to avoid revalidating too frequently (and have many parallel revalidation
// tasks) if revalidation fails or the HTTP request takes some time. The stale period >= 1 hour,
// so 1 more minute won't be a problem.
let stale_timestamp = if is_stale { now + 60 } else { stale_timestamp };
context
.sql
.execute(
"UPDATE http_cache SET expires=?, stale=? WHERE url=?",
(expires, stale_timestamp, url),
)
.await?;
Ok(Some((response, is_stale)))
}
/// Removes expired cache entries.
pub(crate) async fn http_cache_cleanup(context: &Context) -> Result<()> {
// Remove cache entries that are already expired
// or entries that will not expire in a year
// to make sure we don't have invalid timestamps that are way forward in the future.
context
.sql
.execute(
"DELETE FROM http_cache
WHERE ?1 > expires OR expires > ?1 + 31536000",
(time(),),
)
.await?;
Ok(())
}
/// Fetches URL and updates the cache.
///
/// URL is fetched regardless of whether there is an existing result in the cache.
async fn fetch_url(context: &Context, original_url: &str) -> Result<Response> {
let mut url = original_url.to_string();
// Follow up to 10 http-redirects
for _i in 0..10 {
@@ -139,16 +275,42 @@ pub async fn read_url_blob(context: &Context, url: &str) -> Result<Response> {
});
let body = response.collect().await?.to_bytes();
let blob: Vec<u8> = body.to_vec();
return Ok(Response {
let response = Response {
blob,
mimetype,
encoding,
});
};
info!(context, "Inserting {original_url:?} into cache.");
http_cache_put(context, &url, &response).await?;
return Ok(response);
}
Err(anyhow!("Followed 10 redirections"))
}
/// Retrieves the binary contents of URL using HTTP GET request.
pub async fn read_url_blob(context: &Context, url: &str) -> Result<Response> {
if let Some((response, is_stale)) = http_cache_get(context, url).await? {
info!(context, "Returning {url:?} from cache.");
if is_stale {
let context = context.clone();
let url = url.to_string();
tokio::spawn(async move {
// Fetch URL in background to update the cache.
info!(context, "Fetching stale {url:?} in background.");
if let Err(err) = fetch_url(&context, &url).await {
warn!(context, "Failed to revalidate {url:?}: {err:#}.");
}
});
}
return Ok(response);
}
info!(context, "Not found {url:?} in cache, fetching.");
let response = fetch_url(context, url).await?;
Ok(response)
}
/// Sends an empty POST request to the URL.
///
/// Returns response text and whether request was successful or not.
@@ -241,3 +403,125 @@ pub(crate) async fn post_form<T: Serialize + ?Sized>(
let bytes = response.collect().await?.to_bytes();
Ok(bytes)
}
#[cfg(test)]
mod tests {
use super::*;
use std::time::Duration;
use crate::sql::housekeeping;
use crate::test_utils::TestContext;
use crate::tools::SystemTime;
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_http_cache() -> Result<()> {
let t = &TestContext::new().await;
assert_eq!(http_cache_get(t, "https://webxdc.org/").await?, None);
let html_response = Response {
blob: b"<!DOCTYPE html> ...".to_vec(),
mimetype: Some("text/html".to_string()),
encoding: None,
};
let xdc_response = Response {
blob: b"PK...".to_vec(),
mimetype: Some("application/octet-stream".to_string()),
encoding: None,
};
let xdc_editor_url = "https://apps.testrun.org/webxdc-editor-v3.2.0.xdc";
let xdc_pixel_url = "https://apps.testrun.org/webxdc-pixel-v2.xdc";
http_cache_put(t, "https://webxdc.org/", &html_response).await?;
assert_eq!(http_cache_get(t, xdc_editor_url).await?, None);
assert_eq!(http_cache_get(t, xdc_pixel_url).await?, None);
assert_eq!(
http_cache_get(t, "https://webxdc.org/").await?,
Some((html_response.clone(), false))
);
http_cache_put(t, xdc_editor_url, &xdc_response).await?;
http_cache_put(t, xdc_pixel_url, &xdc_response).await?;
assert_eq!(
http_cache_get(t, xdc_editor_url).await?,
Some((xdc_response.clone(), false))
);
assert_eq!(
http_cache_get(t, xdc_pixel_url).await?,
Some((xdc_response.clone(), false))
);
assert_eq!(
http_cache_get(t, "https://webxdc.org/").await?,
Some((html_response.clone(), false))
);
// HTML is stale after 1 hour, but .xdc is not.
SystemTime::shift(Duration::from_secs(3600 + 100));
assert_eq!(
http_cache_get(t, "https://webxdc.org/").await?,
Some((html_response.clone(), true))
);
assert_eq!(
http_cache_get(t, xdc_editor_url).await?,
Some((xdc_response.clone(), false))
);
// Stale cache entry can be renewed
// even before housekeeping removes old one.
http_cache_put(t, "https://webxdc.org/", &html_response).await?;
assert_eq!(
http_cache_get(t, "https://webxdc.org/").await?,
Some((html_response.clone(), false))
);
// 35 days later pixel .xdc expires because we did not request it for 35 days and 1 hour.
// But editor is still there because we did not request it for just 35 days.
// We have not renewed the editor however, so it becomes stale.
SystemTime::shift(Duration::from_secs(3600 * 24 * 35 - 100));
// Run housekeeping to test that it does not delete the blob too early.
housekeeping(t).await?;
assert_eq!(
http_cache_get(t, xdc_editor_url).await?,
Some((xdc_response.clone(), true))
);
assert_eq!(http_cache_get(t, xdc_pixel_url).await?, None);
// If we get the blob the second time quickly, it shouldn't be stale because it's supposed
// that we've already run a revalidation task which will update the blob soon.
assert_eq!(
http_cache_get(t, xdc_editor_url).await?,
Some((xdc_response.clone(), false))
);
// But if the revalidation task hasn't succeeded after some time, the blob is stale again
// even if we continue to get it frequently.
for i in (0..100).rev() {
SystemTime::shift(Duration::from_secs(6));
if let Some((_, true)) = http_cache_get(t, xdc_editor_url).await? {
break;
}
assert!(i > 0);
}
// Test that if the file is accidentally removed from the blobdir,
// there is no error when trying to load the cache entry.
for entry in std::fs::read_dir(t.get_blobdir())? {
let entry = entry.unwrap();
let path = entry.path();
std::fs::remove_file(path).expect("Failed to remove blob");
}
assert_eq!(
http_cache_get(t, xdc_editor_url)
.await
.context("Failed to get no cache response")?,
None
);
Ok(())
}
}

View File

@@ -230,7 +230,7 @@ where
.get(9..12)
.context("HTTP status line does not contain a status code")?;
// Interpert status code according to
// Interpret status code according to
// <https://datatracker.ietf.org/doc/html/rfc7231#section-6>.
if status_code == b"407" {
Err(format_err!("Proxy Authentication Required"))
@@ -640,6 +640,9 @@ mod tests {
fn test_invalid_proxy_url() {
assert!(ProxyConfig::from_url("foobar://127.0.0.1:9050").is_err());
assert!(ProxyConfig::from_url("abc").is_err());
// This caused panic before shadowsocks 1.22.0.
assert!(ProxyConfig::from_url("ss://foo:bar@127.0.0.1:9999").is_err());
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]

View File

@@ -55,6 +55,8 @@ pub enum Param {
/// For Messages: decrypted with validation errors or without mutual set, if neither
/// 'c' nor 'e' are preset, the messages is only transport encrypted.
///
/// Deprecated on 2024-12-25.
ErroneousE2ee = b'e',
/// For Messages: force unencrypted message, a value from `ForcePlaintext` enum.
@@ -366,20 +368,15 @@ impl Params {
///
/// This parses the parameter value as a [ParamsFile] and than
/// tries to return a [BlobObject] for that file. If the file is
/// not yet a valid blob, one will be created by copying the file
/// only if `create` is set to `true`, otherwise an error is
/// returned.
/// not yet a valid blob, one will be created by copying the file.
///
/// Note that in the [ParamsFile::FsPath] case the blob can be
/// created without copying if the path already refers to a valid
/// blob. If so a [BlobObject] will be returned regardless of the
/// `create` argument.
#[allow(clippy::needless_lifetimes)]
/// blob. If so a [BlobObject] will be returned.
pub async fn get_blob<'a>(
&self,
key: Param,
context: &'a Context,
create: bool,
) -> Result<Option<BlobObject<'a>>> {
let val = match self.get(key) {
Some(val) => val,
@@ -387,10 +384,7 @@ impl Params {
};
let file = ParamsFile::from_param(context, val)?;
let blob = match file {
ParamsFile::FsPath(path) => match create {
true => BlobObject::new_from_path(context, &path).await?,
false => BlobObject::from_path(context, &path)?,
},
ParamsFile::FsPath(path) => BlobObject::new_from_path(context, &path).await?,
ParamsFile::Blob(blob) => blob,
};
Ok(Some(blob))
@@ -546,23 +540,20 @@ mod tests {
let path: PathBuf = p.get_path(Param::File, &t).unwrap().unwrap();
assert_eq!(path, fname);
// Blob does not exist yet, expect error.
assert!(p.get_blob(Param::File, &t, false).await.is_err());
fs::write(fname, b"boo").await.unwrap();
let blob = p.get_blob(Param::File, &t, true).await.unwrap().unwrap();
let blob = p.get_blob(Param::File, &t).await.unwrap().unwrap();
assert!(blob.as_file_name().starts_with("foo"));
// Blob in blobdir, expect blob.
let bar_path = t.get_blobdir().join("bar");
p.set(Param::File, bar_path.to_str().unwrap());
let blob = p.get_blob(Param::File, &t, false).await.unwrap().unwrap();
let blob = p.get_blob(Param::File, &t).await.unwrap().unwrap();
assert_eq!(blob, BlobObject::from_name(&t, "bar".to_string()).unwrap());
p.remove(Param::File);
assert!(p.get_file(Param::File, &t).unwrap().is_none());
assert!(p.get_path(Param::File, &t).unwrap().is_none());
assert!(p.get_blob(Param::File, &t, false).await.unwrap().is_none());
assert!(p.get_blob(Param::File, &t).await.unwrap().is_none());
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]

View File

@@ -7,7 +7,7 @@
//! when it's not required. Only when a webxdc subscribes to realtime data or when a reatlime message is sent,
//! the p2p machinery should be started.
//!
//! Adding peer channels to webxdc needs upfront negotation of a topic and sharing of public keys so that
//! Adding peer channels to webxdc needs upfront negotiation of a topic and sharing of public keys so that
//! nodes can connect to each other. The explicit approach is as follows:
//!
//! 1. We introduce a new [`IrohGossipTopic`](crate::headerdef::HeaderDef::IrohGossipTopic) message header with a random 32-byte TopicId,
@@ -417,7 +417,6 @@ async fn get_iroh_gossip_peers(ctx: &Context, msg_id: MsgId) -> Result<Vec<NodeA
))
})
.collect::<std::result::Result<Vec<_>, _>>()
.map_err(Into::into)
},
)
.await
@@ -629,7 +628,6 @@ mod tests {
include_bytes!("../test-data/webxdc/minimal.xdc"),
None,
)
.await
.unwrap();
send_msg(alice, alice_chat.id, &mut instance).await.unwrap();
@@ -801,7 +799,6 @@ mod tests {
include_bytes!("../test-data/webxdc/minimal.xdc"),
None,
)
.await
.unwrap();
send_msg(alice, alice_chat.id, &mut instance).await.unwrap();
@@ -935,7 +932,7 @@ mod tests {
}
}
// channel is only used to remeber if an advertisement has been sent
// channel is only used to remember if an advertisement has been sent
// bob for example does not change the channels because he never sends an
// advertisement
assert_eq!(
@@ -985,7 +982,6 @@ mod tests {
include_bytes!("../test-data/webxdc/minimal.xdc"),
None,
)
.await
.unwrap();
send_msg(alice, alice_chat.id, &mut instance).await.unwrap();
let alice_webxdc = alice.get_last_msg().await;

View File

@@ -711,9 +711,25 @@ impl Peerstate {
Origin::IncomingUnknownFrom,
)
.await?;
chat::remove_from_chat_contacts_table(context, *chat_id, contact_id)
.await?;
chat::add_to_chat_contacts_table(context, *chat_id, &[new_contact_id])
context
.sql
.transaction(|transaction| {
transaction.execute(
"UPDATE chats_contacts
SET remove_timestamp=MAX(add_timestamp+1, ?)
WHERE chat_id=? AND contact_id=?",
(timestamp, chat_id, contact_id),
)?;
transaction.execute(
"INSERT INTO chats_contacts
(chat_id, contact_id, add_timestamp)
VALUES (?1, ?2, ?3)
ON CONFLICT (chat_id, contact_id)
DO UPDATE SET add_timestamp=MAX(remove_timestamp, ?3)",
(chat_id, new_contact_id, timestamp),
)?;
Ok(())
})
.await?;
context.emit_event(EventType::ChatModified(*chat_id));

View File

@@ -21,11 +21,9 @@ use tokio::runtime::Handle;
use crate::constants::KeyGenType;
use crate::key::{DcKey, Fingerprint};
#[allow(missing_docs)]
#[cfg(test)]
pub(crate) const HEADER_AUTOCRYPT: &str = "autocrypt-prefer-encrypt";
#[allow(missing_docs)]
pub const HEADER_SETUPCODE: &str = "passphrase-begin";
/// Preferred symmetric encryption algorithm.

View File

@@ -1,10 +1,24 @@
//! # Push notifications module.
//!
//! This module is responsible for Apple Push Notification Service
//! and Firebase Cloud Messaging push notifications.
//!
//! It provides [`PushSubscriber`] type
//! which holds push notification token for the device,
//! shared by all accounts.
use std::sync::atomic::Ordering;
use std::sync::Arc;
use anyhow::Result;
use anyhow::{Context as _, Result};
use base64::Engine as _;
use pgp::crypto::aead::AeadAlgorithm;
use pgp::crypto::sym::SymmetricKeyAlgorithm;
use pgp::ser::Serialize;
use rand::thread_rng;
use tokio::sync::RwLock;
use crate::context::Context;
use crate::key::DcKey;
/// Manages subscription to Apple Push Notification services.
///
@@ -24,20 +38,85 @@ pub struct PushSubscriber {
inner: Arc<RwLock<PushSubscriberState>>,
}
/// The key was generated with
/// `rsop generate-key --profile rfc9580`
/// and public key was extracted with `rsop extract-cert`.
const NOTIFIERS_PUBLIC_KEY: &str = "-----BEGIN PGP PUBLIC KEY BLOCK-----
xioGZ03cdhsAAAAg6PasQQylEuWAp9N5PXN93rqjZdqOqN3s9RJEU/K8FZzCsAYf
GwoAAABBBQJnTdx2AhsDAh4JCAsJCAcKDQwLBRUKCQgLAhYCIiEGiJJktnCmEtXa
qsSIGRJtupMnxycz/yT0xZK9ez+YkmIAAAAAUfgg/sg0sR2mytzADFBpNAaY0Hyu
aru8ics3eUkeNn2ziL4ZsIMx+4mcM5POvD0PG9LtH8Rz/y9iItD0c2aoRBab7iri
/gDm6aQuj3xXgtAiXdaN9s+QPxR9gY/zG1t9iXgBzioGZ03cdhkAAAAgwJ0wQFsk
MGH4jklfK1fFhYoQZMjEFCRBIk+r1S+WaSDClQYYGwgAAAAsBQJnTdx2AhsMIiEG
iJJktnCmEtXaqsSIGRJtupMnxycz/yT0xZK9ez+YkmIAAAAKCRCIkmS2cKYS1WdP
EFerccH2BoIPNbrxi6hwvxxy7G1mHg//ofD90fqmeY9xTfKMYl16bqQh4R1PiYd5
LMc5VqgXHgioqTYKbltlOtWC+HDt/PrymQsN4q/aEmsM
=5jvt
-----END PGP PUBLIC KEY BLOCK-----";
/// Pads the token with spaces.
///
/// This makes it impossible to tell
/// if the user is an Apple user with shorter tokens
/// or FCM user with longer tokens by the length of ciphertext.
fn pad_device_token(s: &str) -> String {
// 512 is larger than any token, tokens seen so far have not been larger than 200 bytes.
let expected_len: usize = 512;
let payload_len = s.len();
let padding_len = expected_len.saturating_sub(payload_len);
let padding = " ".repeat(padding_len);
let res = format!("{s}{padding}");
debug_assert_eq!(res.len(), expected_len);
res
}
/// Encrypts device token with OpenPGP.
///
/// The result is base64-encoded and not ASCII armored to avoid dealing with newlines.
pub(crate) fn encrypt_device_token(device_token: &str) -> Result<String> {
let public_key = pgp::composed::SignedPublicKey::from_asc(NOTIFIERS_PUBLIC_KEY)?.0;
let encryption_subkey = public_key
.public_subkeys
.first()
.context("No encryption subkey found")?;
let padded_device_token = pad_device_token(device_token);
let literal_message = pgp::composed::Message::new_literal("", &padded_device_token);
let mut rng = thread_rng();
let chunk_size = 8;
let encrypted_message = literal_message.encrypt_to_keys_seipdv2(
&mut rng,
SymmetricKeyAlgorithm::AES128,
AeadAlgorithm::Ocb,
chunk_size,
&[&encryption_subkey],
)?;
let encoded_message = encrypted_message.to_bytes()?;
Ok(format!(
"openpgp:{}",
base64::engine::general_purpose::STANDARD.encode(encoded_message)
))
}
impl PushSubscriber {
/// Creates new push notification subscriber.
pub(crate) fn new() -> Self {
Default::default()
}
/// Sets device token for Apple Push Notification service.
/// Sets device token for Apple Push Notification service
/// or Firebase Cloud Messaging.
pub(crate) async fn set_device_token(&self, token: &str) {
self.inner.write().await.device_token = Some(token.to_string());
}
/// Retrieves device token.
///
/// The token is encrypted with OpenPGP.
///
/// Token may be not available if application is not running on Apple platform,
/// does not have Google Play services,
/// failed to register for remote notifications or is in the process of registering.
///
/// IMAP loop should periodically check if device token is available
@@ -121,3 +200,37 @@ impl Context {
}
}
}
#[cfg(test)]
mod tests {
use super::*;
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_set_device_token() {
let push_subscriber = PushSubscriber::new();
assert_eq!(push_subscriber.device_token().await, None);
push_subscriber.set_device_token("some-token").await;
let device_token = push_subscriber.device_token().await.unwrap();
assert_eq!(device_token, "some-token");
}
#[test]
fn test_pad_device_token() {
let apple_token = "0155b93b7eb867a0d8b7328b978bb15bf22f70867e39e168d03f199af9496894";
assert_eq!(pad_device_token(apple_token).trim(), apple_token);
}
#[test]
fn test_encrypt_device_token() {
let fcm_token = encrypt_device_token("fcm-chat.delta:c67DVcpVQN2rJHiSszKNDW:APA91bErcJV2b8qG0IT4aiuCqw6Al0_SbydSuz3V0CHBR1X7Fp8YzyvlpxNZIOGYVDFKejZGE1YiGSaqxmkr9ds0DuALmZNDwqIhuZWGKKrs3r7DTSkQ9MQ").unwrap();
let fcm_beta_token = encrypt_device_token("fcm-chat.delta.beta:chu-GhZCTLyzq1XseJp3na:APA91bFlsfDawdszWTyOLbxBy7KeRCrYM-SBFqutebF5ix0EZKMuCFUT_Y7R7Ex_eTQG_LbOu3Ky_z5UlTMJtI7ufpIp5wEvsFmVzQcOo3YhrUpbiSVGIlk").unwrap();
let apple_token = encrypt_device_token(
"0155b93b7eb867a0d8b7328b978bb15bf22f70867e39e168d03f199af9496894",
)
.unwrap();
assert_eq!(fcm_token.len(), fcm_beta_token.len());
assert_eq!(apple_token.len(), fcm_token.len());
}
}

View File

@@ -187,7 +187,7 @@ mod tests {
Ok(())
}
#[allow(clippy::assertions_on_constants)]
#[expect(clippy::assertions_on_constants)]
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_quota_thresholds() -> anyhow::Result<()> {
assert!(QUOTA_ALLCLEAR_PERCENTAGE > 50);

File diff suppressed because it is too large Load Diff

View File

@@ -566,6 +566,8 @@ async fn test_escaped_recipients() {
.unwrap()
.0;
// We test with non-chat message here
// because chat messages are not expected to have `Cc` header.
receive_imf(
&t,
b"From: Foobar <foobar@example.com>\n\
@@ -573,8 +575,6 @@ async fn test_escaped_recipients() {
Cc: =?utf-8?q?=3Ch2=3E?= <carl@host.tld>\n\
Subject: foo\n\
Message-ID: <asdklfjjaweofi@example.com>\n\
Chat-Version: 1.0\n\
Chat-Disposition-Notification-To: <foobar@example.com>\n\
Date: Sun, 22 Mar 2020 22:37:57 +0000\n\
\n\
hello\n",
@@ -590,11 +590,12 @@ async fn test_escaped_recipients() {
let msg = Message::load_from_db(&t, chats.get_msg_id(0).unwrap().unwrap())
.await
.unwrap();
assert_eq!(msg.is_dc_message, MessengerMessage::Yes);
assert_eq!(msg.text, "hello");
assert_eq!(msg.param.get_int(Param::WantsMdn).unwrap(), 1);
assert_eq!(msg.is_dc_message, MessengerMessage::No);
assert_eq!(msg.text, "foo hello");
}
/// Tests that `Cc` header updates display name
/// if existing contact has low enough origin.
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_cc_to_contact() {
let t = TestContext::new_alice().await;
@@ -612,6 +613,8 @@ async fn test_cc_to_contact() {
.unwrap()
.0;
// We use non-chat message here
// because chat messages are not expected to have `Cc` header.
receive_imf(
&t,
b"Received: (Postfix, from userid 1000); Mon, 4 Dec 2006 14:51:39 +0100 (CET)\n\
@@ -620,8 +623,6 @@ async fn test_cc_to_contact() {
Cc: Carl <carl@host.tld>\n\
Subject: foo\n\
Message-ID: <asdklfjjaweofi@example.com>\n\
Chat-Version: 1.0\n\
Chat-Disposition-Notification-To: <foobar@example.com>\n\
Date: Sun, 22 Mar 2020 22:37:57 +0000\n\
\n\
hello\n",
@@ -868,18 +869,10 @@ async fn test_parse_ndn_group_msg() -> Result<()> {
assert_eq!(msg.state, MessageState::OutFailed);
let msgs = chat::get_chat_msgs(&t, msg.chat_id).await?;
let msg_id = if let ChatItem::Message { msg_id } = msgs.last().unwrap() {
msg_id
} else {
let ChatItem::Message { msg_id } = *msgs.last().unwrap() else {
panic!("Wrong item type");
};
let last_msg = Message::load_from_db(&t, *msg_id).await?;
assert_eq!(
last_msg.text,
stock_str::failed_sending_to(&t, "assidhfaaspocwaeofi@gmail.com").await
);
assert_eq!(last_msg.from_id, ContactId::INFO);
assert_eq!(msg_id, msg.id);
Ok(())
}
@@ -1671,8 +1664,12 @@ async fn test_pdf_filename_simple() {
assert_eq!(msg.viewtype, Viewtype::File);
assert_eq!(msg.text, "mail body");
let file_path = msg.param.get(Param::File).unwrap();
assert!(file_path.starts_with("$BLOBDIR/simple"));
assert!(file_path.ends_with(".pdf"));
assert_eq!(
file_path,
// That's the blake3 hash of the file content:
"$BLOBDIR/24a6af459cec5d733374aeaa19a6133.pdf"
);
assert_eq!(msg.param.get(Param::Filename).unwrap(), "simple.pdf");
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
@@ -1687,8 +1684,8 @@ async fn test_pdf_filename_continuation() {
assert_eq!(msg.viewtype, Viewtype::File);
assert_eq!(msg.text, "mail body");
let file_path = msg.param.get(Param::File).unwrap();
assert!(file_path.starts_with("$BLOBDIR/test pdf äöüß"));
assert!(file_path.ends_with(".pdf"));
assert!(file_path.starts_with("$BLOBDIR/"));
assert_eq!(msg.get_filename().unwrap(), "test pdf äöüß.pdf");
}
/// HTML-images may come with many embedded images, eg. tiny icons, corners for formatting,
@@ -2208,6 +2205,30 @@ Message content",
assert_ne!(msg.chat_id, t.get_self_chat().await.id);
}
/// Tests that message with hidden recipients is assigned to Saved Messages chat.
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_hidden_recipients_self_chat() {
let t = TestContext::new_alice().await;
receive_imf(
&t,
b"Subject: s
Chat-Version: 1.0
Message-ID: <foobar@localhost>
To: hidden-recipients:;
From: <alice@example.org>
Message content",
false,
)
.await
.unwrap();
let msg = t.get_last_msg().await;
assert_eq!(msg.chat_id, t.get_self_chat().await.id);
assert_eq!(msg.to_id, ContactId::SELF);
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_no_unencrypted_name_in_self_chat() -> Result<()> {
let mut tcm = TestContextManager::new();
@@ -3227,11 +3248,11 @@ async fn test_weird_and_duplicated_filenames() -> Result<()> {
"a. tar.tar.gz",
] {
let attachment = alice.blobdir.join(filename_sent);
let content = format!("File content of {filename_sent}");
let content = "File content of tar.gz archive".to_string();
tokio::fs::write(&attachment, content.as_bytes()).await?;
let mut msg_alice = Message::new(Viewtype::File);
msg_alice.set_file(attachment.to_str().unwrap(), None);
msg_alice.set_file_and_deduplicate(&alice, &attachment, None, None)?;
let alice_chat = alice.create_chat(&bob).await;
let sent = alice.send_msg(alice_chat.id, &mut msg_alice).await;
println!("{}", sent.payload());
@@ -3245,9 +3266,10 @@ async fn test_weird_and_duplicated_filenames() -> Result<()> {
let path = msg.get_file(t).unwrap();
let path2 = path.with_file_name("saved.txt");
msg.save_file(t, &path2).await.unwrap();
assert!(
path.to_str().unwrap().ends_with(".tar.gz"),
"path {path:?} doesn't end with .tar.gz"
assert_eq!(
path.file_name().unwrap().to_str().unwrap(),
"79402cb76f44c5761888f9036992a76.gz",
"The hash of the content should always be the same"
);
assert_eq!(fs::read_to_string(&path).await.unwrap(), content);
assert_eq!(fs::read_to_string(&path2).await.unwrap(), content);
@@ -3317,6 +3339,7 @@ async fn test_outgoing_private_reply_multidevice() -> Result<()> {
let group_id = chat::create_group_chat(&bob, ProtectionStatus::Unprotected, "Group").await?;
chat::add_to_chat_contacts_table(
&bob,
time(),
group_id,
&[
bob.add_or_lookup_contact(&alice1).await.id,
@@ -3526,26 +3549,27 @@ async fn test_no_private_reply_to_blocked_account() -> Result<()> {
let alice = tcm.alice().await;
let bob = tcm.bob().await;
// =============== Bob creates a group ===============
tcm.section("Bob creates a group");
let group_id = chat::create_group_chat(&bob, ProtectionStatus::Unprotected, "Group").await?;
chat::add_to_chat_contacts_table(
&bob,
time(),
group_id,
&[bob.add_or_lookup_contact(&alice).await.id],
)
.await?;
// =============== Bob sends the first message to the group ===============
tcm.section("Bob sends the first message to the group");
let sent = bob.send_text(group_id, "Hello all!").await;
alice.recv_msg(&sent).await;
let chats = Chatlist::try_load(&bob, 0, None, None).await?;
assert_eq!(chats.len(), 1);
// =============== Bob blocks Alice ================
tcm.section("Bob blocks Alice");
Contact::block(&bob, bob.add_or_lookup_contact(&alice).await.id).await?;
// =============== Alice replies private to Bob ==============
tcm.section("Alice replies private to Bob");
let received = alice.get_last_msg().await;
assert_eq!(received.text, "Hello all!");
@@ -3559,7 +3583,7 @@ async fn test_no_private_reply_to_blocked_account() -> Result<()> {
let sent2 = alice.send_msg(alice_bob_chat.id, &mut msg_out).await;
bob.recv_msg(&sent2).await;
// ========= check that no contact request was created ============
// check that no contact request was created
let chats = Chatlist::try_load(&bob, 0, None, None).await.unwrap();
assert_eq!(chats.len(), 1);
let chat_id = chats.get_chat_id(0).unwrap();
@@ -3570,7 +3594,7 @@ async fn test_no_private_reply_to_blocked_account() -> Result<()> {
let received = bob.get_last_msg().await;
assert_eq!(received.text, "Hello all!");
// =============== Bob unblocks Alice ================
tcm.section("Bob unblocks Alice");
// test if the blocked chat is restored correctly
Contact::unblock(&bob, bob.add_or_lookup_contact(&alice).await.id).await?;
let chats = Chatlist::try_load(&bob, 0, None, None).await.unwrap();
@@ -3835,6 +3859,61 @@ async fn test_messed_up_message_id() -> Result<()> {
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_big_forwarded_with_big_attachment() -> Result<()> {
let t = &TestContext::new_bob().await;
let raw = include_bytes!("../../test-data/message/big_forwarded_with_big_attachment.eml");
let rcvd = receive_imf(t, raw, false).await?.unwrap();
assert_eq!(rcvd.msg_ids.len(), 3);
let msg = Message::load_from_db(t, rcvd.msg_ids[0]).await?;
assert_eq!(msg.get_viewtype(), Viewtype::Text);
assert_eq!(msg.get_text(), "Hello!");
assert!(!msg.has_html());
let msg = Message::load_from_db(t, rcvd.msg_ids[1]).await?;
assert_eq!(msg.get_viewtype(), Viewtype::Text);
assert!(msg
.get_text()
.starts_with("this text with 42 chars is just repeated."));
assert!(msg.get_text().ends_with("[...]"));
assert!(!msg.has_html());
let msg = Message::load_from_db(t, rcvd.msg_ids[2]).await?;
assert_eq!(msg.get_viewtype(), Viewtype::File);
assert!(msg.has_html());
let html = msg.id.get_html(t).await?.unwrap();
let tail = html
.split_once("Hello!")
.unwrap()
.1
.split_once("From: AAA")
.unwrap()
.1
.split_once("aaa@example.org")
.unwrap()
.1
.split_once("To: Alice")
.unwrap()
.1
.split_once("alice@example.org")
.unwrap()
.1
.split_once("Subject: Some subject")
.unwrap()
.1
.split_once("Date: Fri, 2 Jun 2023 12:29:17 +0000")
.unwrap()
.1;
assert_eq!(
tail.matches("this text with 42 chars is just repeated.")
.count(),
128
);
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_mua_user_adds_member() -> Result<()> {
let t = TestContext::new_alice().await;
@@ -4033,7 +4112,7 @@ async fn test_sync_member_list_on_rejoin() -> Result<()> {
remove_contact_from_chat(&alice, alice_chat_id, claire_id).await?;
alice.pop_sent_msg().await;
// readd bob
// re-add bob
add_contact_to_chat(&alice, alice_chat_id, bob_id).await?;
let add2 = alice.pop_sent_msg().await;
bob.recv_msg(&add2).await;
@@ -4080,11 +4159,15 @@ async fn test_ignore_outdated_membership_changes() -> Result<()> {
SystemTime::shift(Duration::from_secs(3600));
// Bob replies again adding Alice back.
// Bob replies again, even after some time this does not add Alice back.
//
// Bob cannot learn from Alice that Alice has left the group
// because Alice is not going to send more messages to the group.
send_text_msg(bob, bob_chat_id, "i'm bob".to_string()).await?;
let msg = &bob.pop_sent_msg().await;
alice.recv_msg(msg).await;
assert!(is_contact_in_chat(alice, alice_chat_id, ContactId::SELF).await?);
assert!(!is_contact_in_chat(alice, alice_chat_id, ContactId::SELF).await?);
Ok(())
}
@@ -4145,7 +4228,7 @@ async fn test_dont_recreate_contacts_on_add_remove() -> Result<()> {
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_recreate_contact_list_on_missing_message() -> Result<()> {
async fn test_delayed_removal_is_ignored() -> Result<()> {
let alice = TestContext::new_alice().await;
let bob = TestContext::new_bob().await;
let chat_id = create_group_chat(&alice, ProtectionStatus::Unprotected, "Group").await?;
@@ -4153,6 +4236,7 @@ async fn test_recreate_contact_list_on_missing_message() -> Result<()> {
// create chat with three members
add_to_chat_contacts_table(
&alice,
time(),
chat_id,
&[
Contact::create(&alice, "bob", "bob@example.net").await?,
@@ -4165,37 +4249,45 @@ async fn test_recreate_contact_list_on_missing_message() -> Result<()> {
let bob_chat_id = bob.recv_msg(&alice.pop_sent_msg().await).await.chat_id;
bob_chat_id.accept(&bob).await?;
// bob removes a member
// Bob removes Fiona.
let bob_contact_fiona = Contact::create(&bob, "fiona", "fiona@example.net").await?;
remove_contact_from_chat(&bob, bob_chat_id, bob_contact_fiona).await?;
let remove_msg = bob.pop_sent_msg().await;
// bob adds a new member
// Bob adds new members "blue" and "orange", but first addition message is lost.
let bob_blue = Contact::create(&bob, "blue", "blue@example.net").await?;
add_contact_to_chat(&bob, bob_chat_id, bob_blue).await?;
bob.pop_sent_msg().await;
let bob_orange = Contact::create(&bob, "orange", "orange@example.net").await?;
add_contact_to_chat(&bob, bob_chat_id, bob_orange).await?;
let add_msg = bob.pop_sent_msg().await;
// alice only receives the addition of the member
// Alice only receives the second member addition,
// but this results in addition of both members
// and removal of Fiona.
alice.recv_msg(&add_msg).await;
// since we missed a message, a new contact list should be build
assert_eq!(get_chat_contacts(&alice, chat_id).await?.len(), 3);
// readd fiona
add_contact_to_chat(&alice, chat_id, alice_fiona).await?;
// delayed removal of fiona shouldn't remove her
alice.recv_msg_trash(&remove_msg).await;
assert_eq!(get_chat_contacts(&alice, chat_id).await?.len(), 4);
// Alice re-adds Fiona.
add_contact_to_chat(&alice, chat_id, alice_fiona).await?;
assert_eq!(get_chat_contacts(&alice, chat_id).await?.len(), 5);
// Delayed removal of Fiona by Bob shouldn't remove her.
alice.recv_msg(&remove_msg).await;
assert_eq!(get_chat_contacts(&alice, chat_id).await?.len(), 5);
alice
.golden_test_chat(chat_id, "receive_imf_delayed_removal_is_ignored")
.await;
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_dont_readd_with_normal_msg() -> Result<()> {
let alice = TestContext::new_alice().await;
let bob = TestContext::new_bob().await;
let mut tcm = TestContextManager::new();
let alice = tcm.alice().await;
let bob = tcm.bob().await;
let alice_chat_id = create_group_chat(&alice, ProtectionStatus::Unprotected, "Group").await?;
@@ -4210,6 +4302,7 @@ async fn test_dont_readd_with_normal_msg() -> Result<()> {
let bob_chat_id = bob.recv_msg(&alice.pop_sent_msg().await).await.chat_id;
bob_chat_id.accept(&bob).await?;
// Bob leaves, but Alice didn't receive Bob's leave message.
remove_contact_from_chat(&bob, bob_chat_id, ContactId::SELF).await?;
bob.pop_sent_msg().await;
assert_eq!(get_chat_contacts(&bob, bob_chat_id).await?.len(), 1);
@@ -4223,12 +4316,11 @@ async fn test_dont_readd_with_normal_msg() -> Result<()> {
.await?;
bob.recv_msg(&alice.pop_sent_msg().await).await;
// Alice didn't receive Bob's leave message although a lot of time has
// passed, so Bob must readd themselves otherwise other members would think
// Bob is still here while they aren't. Bob should retry to leave if they
// think that Alice didn't re-add them on purpose (which is possible if Alice uses a classical
// MUA).
assert!(is_contact_in_chat(&bob, bob_chat_id, ContactId::SELF).await?);
// Bob received a message from Alice, but this should not re-add him to the group.
assert!(!is_contact_in_chat(&bob, bob_chat_id, ContactId::SELF).await?);
// Bob got an update that fiora is added nevertheless.
assert_eq!(get_chat_contacts(&bob, bob_chat_id).await?.len(), 2);
Ok(())
}
@@ -4236,35 +4328,50 @@ async fn test_dont_readd_with_normal_msg() -> Result<()> {
async fn test_mua_cant_remove() -> Result<()> {
let alice = TestContext::new_alice().await;
let now = time();
// Alice creates chat with 3 contacts
let date = chrono::DateTime::<chrono::Utc>::from_timestamp(now - 2000, 0)
.unwrap()
.to_rfc2822();
let msg = receive_imf(
&alice,
b"Subject: =?utf-8?q?Message_from_alice=40example=2Eorg?=\r\n\
From: alice@example.org\r\n\
To: <bob@example.net>, <claire@example.org>, <fiona@example.org> \r\n\
Date: Mon, 12 Dec 2022 14:30:39 +0000\r\n\
Message-ID: <Mr.alices_original_mail@example.org>\r\n\
Chat-Version: 1.0\r\n\
\r\n\
tst\r\n",
format!(
"Subject: =?utf-8?q?Message_from_alice=40example=2Eorg?=\r\n\
From: alice@example.org\r\n\
To: <bob@example.net>, <claire@example.org>, <fiona@example.org> \r\n\
Date: {date}\r\n\
Message-ID: <Mr.alices_original_mail@example.org>\r\n\
Chat-Version: 1.0\r\n\
\r\n\
tst\r\n"
)
.as_bytes(),
false,
)
.await?
.unwrap();
let alice_chat = Chat::load_from_db(&alice, msg.chat_id).await?;
assert_eq!(alice_chat.typ, Chattype::Group);
assert_eq!(alice_chat.member_list_is_stale(&alice).await?, false);
// Bob uses a classical MUA to answer, removing a recipient.
let date = chrono::DateTime::<chrono::Utc>::from_timestamp(now - 1000, 0)
.unwrap()
.to_rfc2822();
let bob_removes = receive_imf(
&alice,
b"Subject: Re: Message from alice\r\n\
From: <bob@example.net>\r\n\
To: <alice@example.org>, <claire@example.org>\r\n\
Date: Mon, 12 Dec 2022 14:32:39 +0000\r\n\
Message-ID: <bobs_answer_to_two_recipients@example.net>\r\n\
In-Reply-To: <Mr.alices_original_mail@example.org>\r\n\
\r\n\
Hi back!\r\n",
format!(
"Subject: Re: Message from alice\r\n\
From: <bob@example.net>\r\n\
To: <alice@example.org>, <claire@example.org>\r\n\
Date: {date}\r\n\
Message-ID: <bobs_answer_to_two_recipients@example.net>\r\n\
In-Reply-To: <Mr.alices_original_mail@example.org>\r\n\
\r\n\
Hi back!\r\n"
)
.as_bytes(),
false,
)
.await?
@@ -4272,22 +4379,29 @@ async fn test_mua_cant_remove() -> Result<()> {
assert_eq!(bob_removes.chat_id, alice_chat.id);
let group_chat = Chat::load_from_db(&alice, bob_removes.chat_id).await?;
assert_eq!(group_chat.typ, Chattype::Group);
assert_eq!(group_chat.member_list_is_stale(&alice).await?, false);
assert_eq!(
chat::get_chat_contacts(&alice, group_chat.id).await?.len(),
4
);
// But if the parent message is missing, the message must goto a new ad-hoc group.
let date = chrono::DateTime::<chrono::Utc>::from_timestamp(now, 0)
.unwrap()
.to_rfc2822();
let bob_removes = receive_imf(
&alice,
b"Subject: Re: Message from alice\r\n\
From: <bob@example.net>\r\n\
To: <alice@example.org>, <claire@example.org>\r\n\
Date: Mon, 12 Dec 2022 14:32:40 +0000\r\n\
Message-ID: <bobs_answer_to_two_recipients_1@example.net>\r\n\
In-Reply-To: <Mr.missing@example.org>\r\n\
\r\n\
Hi back!\r\n",
format!(
"Subject: Re: Message from alice\r\n\
From: <bob@example.net>\r\n\
To: <alice@example.org>, <claire@example.org>\r\n\
Date: {date}\r\n\
Message-ID: <bobs_answer_to_two_recipients_1@example.net>\r\n\
In-Reply-To: <Mr.missing@example.org>\r\n\
\r\n\
Hi back!\r\n"
)
.as_bytes(),
false,
)
.await?
@@ -4306,39 +4420,51 @@ async fn test_mua_cant_remove() -> Result<()> {
async fn test_mua_can_add() -> Result<()> {
let alice = TestContext::new_alice().await;
let now = time();
// Alice creates chat with 3 contacts
let date = chrono::DateTime::<chrono::Utc>::from_timestamp(now - 2000, 0)
.unwrap()
.to_rfc2822();
let msg = receive_imf(
&alice,
b"Subject: =?utf-8?q?Message_from_alice=40example=2Eorg?=\r\n\
From: alice@example.org\r\n\
To: <bob@example.net>, <claire@example.org>, <fiona@example.org> \r\n\
Date: Mon, 12 Dec 2022 14:30:39 +0000\r\n\
Message-ID: <Mr.alices_original_mail@example.org>\r\n\
Chat-Version: 1.0\r\n\
\r\n\
Hi!\r\n",
format!(
"Subject: =?utf-8?q?Message_from_alice=40example=2Eorg?=\r\n\
From: alice@example.org\r\n\
To: <bob@example.net>, <claire@example.org>, <fiona@example.org> \r\n\
Date: {date}\r\n\
Message-ID: <Mr.alices_original_mail@example.org>\r\n\
Chat-Version: 1.0\r\n\
\r\n\
Hi!\r\n"
)
.as_bytes(),
false,
)
.await?
.unwrap();
let alice_chat = Chat::load_from_db(&alice, msg.chat_id).await?;
assert_eq!(alice_chat.typ, Chattype::Group);
assert_eq!(alice_chat.member_list_is_stale(&alice).await?, false);
// Bob uses a classical MUA to answer, adding a recipient.
let date = chrono::DateTime::<chrono::Utc>::from_timestamp(now - 1000, 0)
.unwrap()
.to_rfc2822();
let bob_adds = receive_imf(
&alice,
b"Subject: Re: Message from alice\r\n\
From: <bob@example.net>\r\n\
To: <alice@example.org>, <claire@example.org>, <fiona@example.org>, <greg@example.host>\r\n\
Date: Mon, 12 Dec 2022 14:32:39 +0000\r\n\
Message-ID: <bobs_answer_to_two_recipients@example.net>\r\n\
In-Reply-To: <Mr.alices_original_mail@example.org>\r\n\
\r\n\
Hi back!\r\n",
false,
)
.await?
.unwrap();
&alice,
format!("Subject: Re: Message from alice\r\n\
From: <bob@example.net>\r\n\
To: <alice@example.org>, <claire@example.org>, <fiona@example.org>, <greg@example.host>\r\n\
Date: {date}\r\n\
Message-ID: <bobs_answer_to_two_recipients@example.net>\r\n\
In-Reply-To: <Mr.alices_original_mail@example.org>\r\n\
\r\n\
Hi back!\r\n").as_bytes(),
false,
)
.await?
.unwrap();
let group_chat = Chat::load_from_db(&alice, bob_adds.chat_id).await?;
assert_eq!(group_chat.typ, Chattype::Group);
@@ -4455,6 +4581,15 @@ async fn test_recreate_member_list_on_missing_add_of_self() -> Result<()> {
send_text_msg(&alice, alice_chat_id, "4th message".to_string()).await?;
bob.recv_msg(&alice.pop_sent_msg().await).await;
assert!(!is_contact_in_chat(&bob, bob_chat_id, ContactId::SELF).await?);
// Even if some time passed, Bob must not be re-added back.
SystemTime::shift(Duration::from_secs(3600));
send_text_msg(&alice, alice_chat_id, "5th message".to_string()).await?;
alice.pop_sent_msg().await;
send_text_msg(&alice, alice_chat_id, "6th message".to_string()).await?;
bob.recv_msg(&alice.pop_sent_msg().await).await;
assert!(!is_contact_in_chat(&bob, bob_chat_id, ContactId::SELF).await?);
Ok(())
}
@@ -4604,8 +4739,7 @@ async fn test_create_group_with_big_msg() -> Result<()> {
let bob_grp_id = create_group_chat(&bob, ProtectionStatus::Unprotected, "Group").await?;
add_contact_to_chat(&bob, bob_grp_id, ba_contact).await?;
let mut msg = Message::new(Viewtype::Image);
msg.set_file_from_bytes(&bob, "a.jpg", file_bytes, None)
.await?;
msg.set_file_from_bytes(&bob, "a.jpg", file_bytes, None)?;
let sent_msg = bob.send_msg(bob_grp_id, &mut msg).await;
assert!(!msg.get_showpadlock());
@@ -4641,8 +4775,7 @@ async fn test_create_group_with_big_msg() -> Result<()> {
let bob_grp_id = create_group_chat(&bob, ProtectionStatus::Unprotected, "Group1").await?;
add_contact_to_chat(&bob, bob_grp_id, ba_contact).await?;
let mut msg = Message::new(Viewtype::Image);
msg.set_file_from_bytes(&bob, "a.jpg", file_bytes, None)
.await?;
msg.set_file_from_bytes(&bob, "a.jpg", file_bytes, None)?;
let sent_msg = bob.send_msg(bob_grp_id, &mut msg).await;
assert!(msg.get_showpadlock());
@@ -4688,13 +4821,6 @@ async fn test_partial_group_consistency() -> Result<()> {
let contacts = get_chat_contacts(&bob, bob_chat_id).await?;
assert_eq!(contacts.len(), 2);
// Get initial timestamp.
let timestamp = bob_chat_id
.get_param(&bob)
.await?
.get_i64(Param::MemberListTimestamp)
.unwrap();
// Bob receives partial message.
let msg_id = receive_imf_from_inbox(
&bob,
@@ -4715,15 +4841,9 @@ Chat-Group-Member-Added: charlie@example.com",
.context("no received message")?;
let msg = Message::load_from_db(&bob, msg_id.msg_ids[0]).await?;
let timestamp2 = bob_chat_id
.get_param(&bob)
.await?
.get_i64(Param::MemberListTimestamp)
.unwrap();
// Partial download does not change the member list.
assert_eq!(msg.download_state, DownloadState::Available);
assert_eq!(timestamp, timestamp2);
assert_eq!(get_chat_contacts(&bob, bob_chat_id).await?, contacts);
// Alice sends normal message to bob, adding fiona.
@@ -4736,15 +4856,6 @@ Chat-Group-Member-Added: charlie@example.com",
bob.recv_msg(&alice.pop_sent_msg().await).await;
let timestamp3 = bob_chat_id
.get_param(&bob)
.await?
.get_i64(Param::MemberListTimestamp)
.unwrap();
// Receiving a message after a partial download recreates the member list because we treat
// such messages as if we have not seen them.
assert_ne!(timestamp, timestamp3);
let contacts = get_chat_contacts(&bob, bob_chat_id).await?;
assert_eq!(contacts.len(), 3);
@@ -4768,15 +4879,9 @@ Chat-Group-Member-Added: charlie@example.com",
.context("no received message")?;
let msg = Message::load_from_db(&bob, msg_id.msg_ids[0]).await?;
let timestamp4 = bob_chat_id
.get_param(&bob)
.await?
.get_i64(Param::MemberListTimestamp)
.unwrap();
// After full download, the old message should not change group state.
assert_eq!(msg.download_state, DownloadState::Done);
assert_eq!(timestamp3, timestamp4);
assert_eq!(get_chat_contacts(&bob, bob_chat_id).await?, contacts);
Ok(())
@@ -4799,19 +4904,13 @@ async fn test_leave_protected_group_missing_member_key() -> Result<()> {
("b@b", "bob@example.net"),
)
.await?;
// We fail to send the message.
assert!(remove_contact_from_chat(alice, group_id, ContactId::SELF)
.await
.is_err());
assert!(is_contact_in_chat(alice, group_id, ContactId::SELF).await?);
alice
.sql
.execute(
"UPDATE acpeerstates SET addr=? WHERE addr=?",
("bob@example.net", "b@b"),
)
.await?;
remove_contact_from_chat(alice, group_id, ContactId::SELF).await?;
alice.pop_sent_msg().await;
// The contact is already removed anyway.
assert!(!is_contact_in_chat(alice, group_id, ContactId::SELF).await?);
Ok(())
}
@@ -4833,12 +4932,22 @@ async fn test_protected_group_add_remove_member_missing_key() -> Result<()> {
.await?;
let fiona = &tcm.fiona().await;
let fiona_addr = fiona.get_config(Config::Addr).await?.unwrap();
mark_as_verified(alice, fiona).await;
let alice_fiona_id = alice.add_or_lookup_contact(fiona).await.id;
assert!(add_contact_to_chat(alice, group_id, alice_fiona_id)
.await
.is_err());
assert!(!is_contact_in_chat(alice, group_id, alice_fiona_id).await?);
// Sending the message failed,
// but member is added to the chat locally already.
assert!(is_contact_in_chat(alice, group_id, alice_fiona_id).await?);
let msg = alice.get_last_msg_in(group_id).await;
assert!(msg.is_info());
assert_eq!(
msg.get_text(),
stock_str::msg_add_member_local(alice, &fiona_addr, ContactId::SELF).await
);
// Now the chat has a message "You added member fiona@example.net. [INFO] !!" (with error) that
// may be confusing, but if the error is displayed in UIs, it's more or less ok. This is not a
// normal scenario anyway.
@@ -5107,8 +5216,7 @@ async fn test_prefer_references_to_downloaded_msgs() -> Result<()> {
let file_bytes = include_bytes!("../../test-data/image/screenshot.gif");
let mut msg = Message::new(Viewtype::File);
msg.set_file_from_bytes(alice, "file", file_bytes, None)
.await?;
msg.set_file_from_bytes(alice, "file", file_bytes, None)?;
let mut sent = alice.send_msg(alice_chat_id, &mut msg).await;
sent.payload = sent
.payload
@@ -5120,8 +5228,7 @@ async fn test_prefer_references_to_downloaded_msgs() -> Result<()> {
assert_eq!(received.chat_id, bob.get_chat(alice).await.id);
let mut msg = Message::new(Viewtype::File);
msg.set_file_from_bytes(alice, "file", file_bytes, None)
.await?;
msg.set_file_from_bytes(alice, "file", file_bytes, None)?;
let sent = alice.send_msg(alice_chat_id, &mut msg).await;
let received = bob.recv_msg(&sent).await;
assert_eq!(received.download_state, DownloadState::Available);
@@ -5180,7 +5287,6 @@ async fn test_receive_vcard() -> Result<()> {
.as_bytes(),
None,
)
.await
.unwrap();
let alice_bob_chat = alice.create_chat(bob).await;

View File

@@ -14,12 +14,31 @@ use crate::{context::Context, log::LogExt};
use super::InnerSchedulerState;
/// Rough connectivity status for display in the status bar in the UI.
#[derive(Debug, Clone, Copy, PartialEq, Eq, EnumProperty, PartialOrd, Ord)]
pub enum Connectivity {
/// Not connected.
///
/// This may be because we just started,
/// because we lost connection and
/// were not able to connect and log in yet
/// or because I/O is not started.
NotConnected = 1000,
/// Attempting to connect and log in.
Connecting = 2000,
/// Fetching or sending messages
/// Fetching or sending messages.
Working = 3000,
/// We are connected but not doing anything.
///
/// This is the most common state,
/// so mobile UIs display the profile name
/// instead of connectivity status in this state.
/// Desktop UI displays "Connected" in the tooltip,
/// which signals that no more messages
/// are coming in.
Connected = 4000,
}
@@ -32,13 +51,17 @@ enum DetailedConnectivity {
Error(String),
#[default]
Uninitialized,
/// Attempting to connect,
/// until we successfully log in.
Connecting,
/// Connection is just established, but there may be work to do.
Connected,
/// Connection is just established,
/// there may be work to do.
Preparing,
/// There is actual work to do, e.g. there are messages in SMTP queue
/// or we detected a message that should be downloaded.
/// or we detected a message on IMAP server that should be downloaded.
Working,
InterruptingIdle,
@@ -57,8 +80,14 @@ impl DetailedConnectivity {
DetailedConnectivity::Uninitialized => Some(Connectivity::NotConnected),
DetailedConnectivity::Connecting => Some(Connectivity::Connecting),
DetailedConnectivity::Working => Some(Connectivity::Working),
DetailedConnectivity::InterruptingIdle => Some(Connectivity::Connected),
DetailedConnectivity::Connected => Some(Connectivity::Connected),
DetailedConnectivity::InterruptingIdle => Some(Connectivity::Working),
// At this point IMAP has just connected,
// but does not know yet if there are messages to download.
// We still convert this to Working state
// so user can see "Updating..." and not "Connected"
// which is reserved for idle state.
DetailedConnectivity::Preparing => Some(Connectivity::Working),
// Just don't return a connectivity, probably the folder is configured not to be
// watched or there is e.g. no "Sent" folder, so we are not interested in it
@@ -74,9 +103,9 @@ impl DetailedConnectivity {
| DetailedConnectivity::Uninitialized
| DetailedConnectivity::NotConfigured => "<span class=\"red dot\"></span>".to_string(),
DetailedConnectivity::Connecting => "<span class=\"yellow dot\"></span>".to_string(),
DetailedConnectivity::Working
DetailedConnectivity::Preparing
| DetailedConnectivity::Working
| DetailedConnectivity::InterruptingIdle
| DetailedConnectivity::Connected
| DetailedConnectivity::Idle => "<span class=\"green dot\"></span>".to_string(),
}
}
@@ -86,10 +115,12 @@ impl DetailedConnectivity {
DetailedConnectivity::Error(e) => stock_str::error(context, e).await,
DetailedConnectivity::Uninitialized => "Not started".to_string(),
DetailedConnectivity::Connecting => stock_str::connecting(context).await,
DetailedConnectivity::Working => stock_str::updating(context).await,
DetailedConnectivity::InterruptingIdle
| DetailedConnectivity::Connected
| DetailedConnectivity::Idle => stock_str::connected(context).await,
DetailedConnectivity::Preparing | DetailedConnectivity::Working => {
stock_str::updating(context).await
}
DetailedConnectivity::InterruptingIdle | DetailedConnectivity::Idle => {
stock_str::connected(context).await
}
DetailedConnectivity::NotConfigured => "Not configured".to_string(),
}
}
@@ -107,7 +138,7 @@ impl DetailedConnectivity {
// since sending the last message, connectivity could have changed, which we don't notice
// until another message is sent
DetailedConnectivity::InterruptingIdle
| DetailedConnectivity::Connected
| DetailedConnectivity::Preparing
| DetailedConnectivity::Idle => stock_str::last_msg_sent_successfully(context).await,
DetailedConnectivity::NotConfigured => "Not configured".to_string(),
}
@@ -120,7 +151,7 @@ impl DetailedConnectivity {
DetailedConnectivity::Connecting => false,
DetailedConnectivity::Working => false,
DetailedConnectivity::InterruptingIdle => false,
DetailedConnectivity::Connected => false, // Just connected, there may still be work to do.
DetailedConnectivity::Preparing => false, // Just connected, there may still be work to do.
DetailedConnectivity::NotConfigured => true,
DetailedConnectivity::Idle => true,
}
@@ -148,8 +179,8 @@ impl ConnectivityStore {
pub(crate) async fn set_working(&self, context: &Context) {
self.set(context, DetailedConnectivity::Working).await;
}
pub(crate) async fn set_connected(&self, context: &Context) {
self.set(context, DetailedConnectivity::Connected).await;
pub(crate) async fn set_preparing(&self, context: &Context) {
self.set(context, DetailedConnectivity::Preparing).await;
}
pub(crate) async fn set_not_configured(&self, context: &Context) {
self.set(context, DetailedConnectivity::NotConfigured).await;
@@ -169,8 +200,8 @@ impl ConnectivityStore {
}
}
/// Set all folder states to InterruptingIdle in case they were `Connected` before.
/// Called during `dc_maybe_network()` to make sure that `dc_all_work_done()`
/// Set all folder states to InterruptingIdle in case they were `Idle` before.
/// Called during `dc_maybe_network()` to make sure that `all_work_done()`
/// returns false immediately after `dc_maybe_network()`.
pub(crate) async fn idle_interrupted(inbox: ConnectivityStore, oboxes: Vec<ConnectivityStore>) {
let mut connectivity_lock = inbox.0.lock().await;
@@ -179,8 +210,7 @@ pub(crate) async fn idle_interrupted(inbox: ConnectivityStore, oboxes: Vec<Conne
// returns Connected. But after dc_maybe_network(), dc_get_connectivity() must not
// return Connected until DC is completely done with fetching folders; this also
// includes scan_folders() which happens on the inbox thread.
if *connectivity_lock == DetailedConnectivity::Connected
|| *connectivity_lock == DetailedConnectivity::Idle
if *connectivity_lock == DetailedConnectivity::Idle
|| *connectivity_lock == DetailedConnectivity::NotConfigured
{
*connectivity_lock = DetailedConnectivity::InterruptingIdle;
@@ -189,9 +219,7 @@ pub(crate) async fn idle_interrupted(inbox: ConnectivityStore, oboxes: Vec<Conne
for state in oboxes {
let mut connectivity_lock = state.0.lock().await;
if *connectivity_lock == DetailedConnectivity::Connected
|| *connectivity_lock == DetailedConnectivity::Idle
{
if *connectivity_lock == DetailedConnectivity::Idle {
*connectivity_lock = DetailedConnectivity::InterruptingIdle;
}
}
@@ -507,7 +535,7 @@ impl Context {
}
/// Returns true if all background work is done.
pub async fn all_work_done(&self) -> bool {
async fn all_work_done(&self) -> bool {
let lock = self.scheduler.inner.read().await;
let stores: Vec<_> = match *lock {
InnerSchedulerState::Started(ref sched) => sched
@@ -527,4 +555,23 @@ impl Context {
}
true
}
/// Waits until background work is finished.
pub async fn wait_for_all_work_done(&self) {
// Ideally we could wait for connectivity change events,
// but sleep loop is good enough.
// First 100 ms sleep in chunks of 10 ms.
for _ in 0..10 {
if self.all_work_done().await {
break;
}
tokio::time::sleep(std::time::Duration::from_millis(10)).await;
}
// If we are not finished in 100 ms, keep waking up every 100 ms.
while !self.all_work_done().await {
tokio::time::sleep(std::time::Duration::from_millis(100)).await;
}
}
}

View File

@@ -454,6 +454,9 @@ pub(crate) async fn handle_securejoin_handshake(
.await?;
inviter_progress(context, contact_id, 800);
inviter_progress(context, contact_id, 1000);
// IMAP-delete the message to avoid handling it by another device and adding the
// member twice. Another device will know the member's key from Autocrypt-Gossip.
Ok(HandshakeMessage::Done)
} else {
// Setup verified contact.
secure_connection_established(
@@ -468,8 +471,8 @@ pub(crate) async fn handle_securejoin_handshake(
.context("failed sending vc-contact-confirm message")?;
inviter_progress(context, contact_id, 1000);
Ok(HandshakeMessage::Ignore) // "Done" would delete the message and break multi-device (the key from Autocrypt-header is needed)
}
Ok(HandshakeMessage::Ignore) // "Done" would delete the message and break multi-device (the key from Autocrypt-header is needed)
}
/*=======================================================
==== Bob - the joiner's side ====
@@ -751,7 +754,7 @@ mod tests {
use crate::imex::{imex, ImexMode};
use crate::receive_imf::receive_imf;
use crate::stock_str::{self, chat_protection_enabled};
use crate::test_utils::get_chat_msg;
use crate::test_utils::{get_chat_msg, TimeShiftFalsePositiveNote};
use crate::test_utils::{TestContext, TestContextManager};
use crate::tools::SystemTime;
use std::collections::HashSet;
@@ -798,6 +801,8 @@ mod tests {
}
async fn test_setup_contact_ex(case: SetupContactCase) {
let _n = TimeShiftFalsePositiveNote;
let mut tcm = TestContextManager::new();
let alice = tcm.alice().await;
let alice_addr = &alice.get_config(Config::Addr).await.unwrap().unwrap();
@@ -1351,6 +1356,8 @@ mod tests {
// explicit user action, the Auto-Submitted header shouldn't be present. Otherwise it would
// be strange to have it in "member-added" messages of verified groups only.
assert!(msg.get_header(HeaderDef::AutoSubmitted).is_none());
// This is a two-member group, but Alice must Autocrypt-gossip to her other devices.
assert!(msg.get_header(HeaderDef::AutocryptGossip).is_some());
{
// Now Alice's chat with Bob should still be hidden, the verified message should

View File

@@ -59,8 +59,13 @@ pub(super) async fn start_protocol(context: &Context, invite: QrInvite) -> Resul
// only become usable once the protocol is finished.
let group_chat_id = state.joining_chat_id(context).await?;
if !is_contact_in_chat(context, group_chat_id, invite.contact_id()).await? {
chat::add_to_chat_contacts_table(context, group_chat_id, &[invite.contact_id()])
.await?;
chat::add_to_chat_contacts_table(
context,
time(),
group_chat_id,
&[invite.contact_id()],
)
.await?;
}
let msg = stock_str::secure_join_started(context, invite.contact_id()).await;
chat::add_info_msg(context, group_chat_id, &msg, time()).await?;

View File

@@ -21,7 +21,6 @@ use crate::mimefactory::MimeFactory;
use crate::net::proxy::ProxyConfig;
use crate::net::session::SessionBufStream;
use crate::scheduler::connectivity::ConnectivityStore;
use crate::sql;
use crate::stock_str::unencrypted_email;
use crate::tools::{self, time_elapsed};
@@ -104,7 +103,7 @@ impl Smtp {
}
/// Connect using the provided login params.
#[allow(clippy::too_many_arguments)]
#[expect(clippy::too_many_arguments)]
pub async fn connect(
&mut self,
context: &Context,
@@ -585,18 +584,16 @@ async fn send_mdn_rfc724_mid(
info!(context, "Successfully sent MDN for {rfc724_mid}.");
context
.sql
.execute("DELETE FROM smtp_mdns WHERE rfc724_mid = ?", (rfc724_mid,))
.transaction(|transaction| {
let mut stmt =
transaction.prepare("DELETE FROM smtp_mdns WHERE rfc724_mid = ?")?;
stmt.execute((rfc724_mid,))?;
for additional_rfc724_mid in additional_rfc724_mids {
stmt.execute((additional_rfc724_mid,))?;
}
Ok(())
})
.await?;
if !additional_rfc724_mids.is_empty() {
let q = format!(
"DELETE FROM smtp_mdns WHERE rfc724_mid IN({})",
sql::repeat_vars(additional_rfc724_mids.len())
);
context
.sql
.execute(&q, rusqlite::params_from_iter(additional_rfc724_mids))
.await?;
}
Ok(true)
}
SendResult::Retry => {

Some files were not shown because too many files have changed in this diff Show More