Commit Graph

9233 Commits

Author SHA1 Message Date
l
a2df29515a feat: log the number of read/written bytes on IMAP stream read error (#6924) 2025-07-17 20:01:16 +00:00
link2xt
6df1d165dd feat: log when background fetch of all accounts finishes successfully 2025-07-17 15:46:46 +00:00
cliffmccarthy
e03e2d9a68 fix: List e-mail contacts in repl listcontacts command
- After the revisions to support key contacts, the 'listcontacts'
  command in the repl only lists key-contacts.  A separate flag now
  has to be passed to Contact::get_all() to list address contacts.
  This makes it difficult to run the example in README.md because we
  need to see the new contact so we can get its ID for the next
  command in the example, that creates a chat by ID.
- Revised 'listcontacts' command to make a second call to
  Contact::get_all() with the DC_GCL_ADDRESS flag, then print the
  e-mail contacts after the key contacts.
- Revised configuration example in top-level README.md to reflect
  current command output.

fixes #7011
2025-07-17 10:27:14 +02:00
iequidoo
8fc6ea19b4 feat: {ensure_and,logged}_debug_assert: Don't evaluate condition twice 2025-07-16 11:41:45 -03:00
iequidoo
c5c947e175 feat(repl): Print errors and debug logs to stderr
Follow-up to 545007aca5.
2025-07-16 11:39:56 -03:00
iequidoo
6d8dff54a7 fix: Ignore protected headers in outer message part (#6357)
Delta Chat always adds protected headers to the inner encrypted or signed message, so if a protected
header is only present in the outer part, it should be ignored because it's probably added by the
server or somebody else. The exceptions are Subject and List-ID because there are known cases when
they are only present in the outer message part.

Also treat any Chat-* headers as protected. This fixes e.g. a case when the server injects a
"Chat-Version" IMF header tricking Delta Chat into thinking that it's a chat message.

Also handle "Auto-Submitted" and "Autocrypt-Setup-Message" as protected headers on the receiver
side, this was apparently forgotten.
2025-07-16 11:39:06 -03:00
link2xt
a0f6bdffeb chore(release): prepare for 2.2.0 v2.2.0 2025-07-14 18:43:03 +00:00
link2xt
e6fd52afff fix: always prefer the last header
Headers are normally added at the top of the message,
e.g. when forwarding new `Received` headers are
added at the top.

When headers are protected with DKIM-Signature
and oversigning is not used,
forged headers may be added on top
so headers from the top are generally less trustworthy.

This is tested with `test_take_last_header`,
but so far last header was only preferred
for known headers. This change extends
preference of the last header to all headers.
2025-07-14 14:57:52 +00:00
Nico de Haen
0142515887 api!: In ChatListItem, replace is_group and is_(out_)broadcast with chat_type property (#7003)
- removed ChatListItem.is_broadcast
- mark ChatListItem.is_group as deprecated
2025-07-14 11:16:28 +02:00
link2xt
d45ec7f34d feat: advance next UID even if connection fails while fetching
Connection sometimes fails while processing FETCH
responses. In this case `fetch_new_messages` exits early
and does not advance next expected UID even if
some messages were processed.

This results in prefetching the same messages
after reconnection and log messages
similar to
"Not moving the message ab05c85a-e191-4fd2-a951-9972bc7e167f@localhost that we have seen before.".

With this change we advance next expected UID
even if `fetch_new_messages` returns a network error.
2025-07-13 16:32:26 +00:00
iequidoo
752f45f0f0 test: Unencrypted group creation (#6927) 2025-07-13 12:59:33 -03:00
iequidoo
0299543a86 api(jsonrpc): Add CommandApi::create_group_chat_unencrypted() (#6927) 2025-07-13 12:59:33 -03:00
iequidoo
d3908d6b36 api: Add chat::create_group_ex(), deprecate create_group_chat() (#6927)
`chat::create_group_ex()` gains an `encryption: Option<ProtectionStatus>` parameter to support
unencrypted chats.
2025-07-13 12:59:33 -03:00
iequidoo
2cf979de53 feat: Donation request device message (#6913)
A donation request device message is added if >= 100 messages have been sent and delivered. The
condition is checked every 30 days since the first message is sent. The message is added only once.
2025-07-13 11:53:14 -03:00
Hocuri
f5e8c8083d test: Tune down DELTACHAT_SAVE_TMP_DB hint (#6998)
Follow-up for https://github.com/chatmail/core/pull/6992

Since we're printing the hint to stderr now, rather than stdout (as per
link2xt's suggestion), it was too noisy. Also, it was printed once for
every test account rather than once per test.

Now, it integrates nicely with rust's hint to enable a backtrace:

```
  stderr ───

    thread 'chat::chat_tests::test_broadcasts_name_and_avatar' panicked at src/chat/chat_tests.rs:2757:5:
    assertion failed: `(left == right)`

    Diff < left / right > :
    <true
    >false


    note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
    note: If you want to examine the database files, set environment variable DELTACHAT_SAVE_TMP_DB=1

        FAIL [   0.265s] deltachat chat::chat_tests::test_broadcast
```
2025-07-12 12:52:04 +02:00
iequidoo
58b99f59f7 feat: Log failed debug assertions in all configurations
Add `logged_debug_assert` macro logging a warning if a condition is not satisfied, before invoking
`debug_assert!`, and use this macro where `Context` is accessible (i.e. don't change function
signatures for now).
Follow-up to 0359481ba4.
2025-07-12 07:27:55 -03:00
link2xt
402e42f858 chore(release): prepare for 2.1.0 v2.1.0 2025-07-11 22:56:57 +00:00
link2xt
fbae0739a6 chore(cargo): update cordyceps from 0.3.2 to 0.3.4 2025-07-11 22:07:27 +00:00
iequidoo
0359481ba4 feat: ensure_and_debug_assert{,_eq,_ne} macros combining debug_assert* and anyhow::ensure (#6907)
We have some debug assertions already, but we also want the corresponding errors in the release
configuration so that it's not less reliable than non-optimized one. This doesn't change any
function signatures, only debug assertions in functions returning `Result` are replaced.

Co-authored-by: l <link2xt@testrun.org>
2025-07-11 14:59:49 -03:00
Hocuri
6406f305b8 feat: Make it possible to leave broadcast channels (#6984)
Part of #6884.
The channel owner will not be notified in any way that you left, they
will only see that there is one member less.

For the translated stock strings, this is what we agreed on in the
group:
- Add a new "Leave Channel" stock string (will need to be done in UIs)
- Reword the existing "Are you sure you want to leave this group?"
string to "Are you sure you want to leave?" (the options are "Cancel"
and "Leave Group" / "Leave Channel", so it's clear what you are leaving)
(will need to be done in the deltachat-android repo, other UIs will pick
it up automatically)
- Reword the existing "You left the group." string to "You left". (done
here, I will adapt the strings in deltachat-android, too)

I adapted DC Android by pushing
6df2740884
to https://github.com/deltachat/deltachat-android/pull/3783.

---------

Co-authored-by: l <link2xt@testrun.org>
2025-07-11 12:34:05 +00:00
Hocuri
e5e0f0cdd7 test: Add option to save database on test failure (#6992)
I had it a few times now that I wanted to examine the database in order
to debug a test failure. Setting this environment variable makes this
easy in the future.
2025-07-11 12:01:04 +00:00
B. Petersen
0bac4acdd8 docs: update showpadlock ffi 2025-07-11 13:11:01 +02:00
Alireza Sadraii
ce5697c5f7 feat: add account ordering functionality (#6993)
New public API `set_accounts_order` allows setting the order of accounts.

The account order is stored as a list of account IDs in `accounts.toml`
under a new `accounts_order: Vec<u32>` field.
2025-07-10 22:59:27 +00:00
ivn
22258f7269 fix!: Use Viewtype::File for messages with invalid images, images of unknown size, images > 50 Mpx (#6825)
BREAKING CHANGE: messages with invalid images, images of unknown size,
huge images, will have Viewtype::File

After changing the logic of Viewtype selection, I had to fix 3 old tests
that used invalid Base64 image data.

Co-authored-by: iequidoo <117991069+iequidoo@users.noreply.github.com>
2025-07-10 18:11:55 -03:00
link2xt
5ab107866a feat: log emitted logging events with tracing 2025-07-10 00:27:24 +00:00
iequidoo
374a5ef687 feat: Don't apply chat name and avatar changes from non-members
Non-members can't modify the member list (incl. adding themselves), modify an ephemeral timer, so
they shouldn't be able to change the group name or avatar, just for consistency. Even if messages
are reordered and a group name change from a new member arrives before its addition, the new group
name will be applied on a receipt of the next message following the addition message because
Chat-Group-Name-Timestamp increases. While Delta Chat groups aimed for chatting with trusted
contacts, accepting group changes from everyone knowing Chat-Group-Id means that if any of the past
members have the key compromised, the group should be recreated which looks impractical.
2025-07-09 17:39:55 -03:00
iequidoo
1a2e355bb8 feat: migrations: Use tools::Time to measure time for logging
There's a comment in `tools` that tells to use `tools::Time` instead of `Instant` because on Android
the latter doesn't advance in the deep sleep mode. The only place except `migrations` where
`Instant` is used is tests, but we don't run CI on Android. It's unlikely that Delta Chat goes to
the deep sleep while executing migrations, but still possible, so let's use `tools::Time` as
everywhere else.
2025-07-09 17:13:07 -03:00
link2xt
192a6a2b9d chore(release): prepare for 2.0.0 v2.0.0 2025-07-09 18:31:32 +00:00
Sebastian Klähn
4ca0ce2fb2 fix: Add query to post request for account creation (#6989) 2025-07-09 18:17:17 +00:00
link2xt
ab4cb01065 fix: do not try to lookup key-contacts for unencrypted 1:1 messages 2025-07-09 17:02:31 +00:00
link2xt
661a8864b9 test: add a test reproducing chat assignment bug 2025-07-09 17:02:31 +00:00
link2xt
67f00fbb84 refactor: remove check that is always false
The check for chat_id.is_some() is inside the `else` branch
of a pattern-matching `if` that looks for `Some` pattern.
2025-07-09 17:02:30 +00:00
iequidoo
389649ea8a fix: Save msgs to key-contacts migration state and run migration periodically (#6956)
Save:
- (old contact id) -> (new contact id) mapping.
- The message id starting from which all messages are already migrated.
Run the migration from `housekeeping()` for at least 500 ms and for >= 1000 messages per run.
2025-07-09 09:10:49 -03:00
iequidoo
a87ee030fc fix: migrate_key_contacts(): Remove "id>9" from encrypted messages SELECT
+ Replace LIKE with GLOB, the latter is case-sensitive.
2025-07-09 09:10:49 -03:00
iequidoo
3f66ae91cd feat: Check images passed as File before making them Image
We don't want images having unsupported format or corrupted ones to be sent as `Image` and appear in
the "Images" tab in UIs because they can't be displayed correctly.
2025-07-08 17:43:13 -03:00
iequidoo
75b7bea78f fix: Decide on filename used for sending depending on the original Viewtype
If a user attaches an image as `File`, we should send the original filename. And vice versa, if it's
`Image` originally, we mustn't reveal the filename.

The filename used for sending is now also saved to the db, so all the sender's devices will display
the same filename in the message info.
2025-07-08 17:43:13 -03:00
iequidoo
acba27a328 fix: Treat and send images that can't be decoded as Viewtype::File
Otherwise unsupported and corrupted images are displayed in the "Images" tab in UIs and that looks
as a Delta Chat bug. This should be a rare case though, so log it as error and let the user know
that metadata isn't removed from the image at least.
2025-07-08 17:43:13 -03:00
iequidoo
cba9eb98d6 refactor: build_body_file(): Remove guessing mimetype by file extension
Guessing mimetype is already done in `chat::prepare_msg_blob()`.
2025-07-08 17:43:13 -03:00
iequidoo
da9b24d191 fix: Treat "tgs" as Viewtype::File
`Viewtype::Sticker` has special meaning: the file should be an image having fully transparent
pixels. But "tgs" (Telegram animated sticker) is a compressed JSON and isn't recognized by Core as
image.
2025-07-08 17:43:13 -03:00
Hocuri
c9c5d94666 fix: Prefer encrypted List-Id header (#6983)
If there is an encrypted List-Id header, it should be preferred over an
unencrypted List-Id header.

Part of #6884
2025-07-07 20:45:21 +00:00
Hocuri
aad8f698dd fix: Don't send ChatGroupId for broadcast channels (#6975)
Older versions of Delta Chat ignore the message if it contains a
ChatGroupId header. ("older versions" means all versions without #6901,
i.e.currently released versions)

This means that without this PR, broadcast channel messages sent from
current main don't arrive at a device running latest released DC.

Part of #6884.
2025-07-07 12:06:54 +02:00
Hocuri
35e107e87d api!: Add InBroadcastChannel, OutBroadcastChannel chattypes, add create_broadcast_channel() (#6901)
In https://github.com/chatmail/core/pull/6901, I unfortunately forgot to
document the API change when squash-merging, so, I'm doing this with the
PR here.

The API change is breaking because not adapting to the new channel types
would lead to errors.
2025-07-07 11:42:02 +02:00
link2xt
d9b361f066 docs: remove outdated comment that says MDNs are unencrypted 2025-07-06 22:25:15 +00:00
link2xt
94e75cb3b8 test: add online test for read receipts 2025-07-06 22:25:15 +00:00
link2xt
c7fb64e2f3 fix: send Autocrypt header in MDNs
Otherwise MDNs are attributed to address-contacts
rather than key-contacts.
2025-07-06 22:25:15 +00:00
link2xt
ebddabe958 api(deltachat-rpc-client): add Message.get_read_receipts() 2025-07-06 22:25:15 +00:00
cliffmccarthy
b81f7cfcab fix: Update argument documentation and handling in repl (#6979)
- Updated argument descriptions in help for import-keys,
createbroadcast, and groupimage.
- Revised import-keys to check for the required argument.
2025-07-06 19:06:14 +00:00
cliffmccarthy
9197ef04f7 fix: Update repl help and autocomplete to match implementation (#6978) 2025-07-06 19:05:29 +00:00
iequidoo
7e4d4cf680 api: Contact::get_all(): Support listing address-contacts
Also test-cover `DC_GCL_ADD_SELF`.
2025-07-03 07:10:36 -03:00
Hocuri
0a73c2b7ab feat: Show broadcast channels in their own, proper "Channel" chat (#6901)
Part of #6884 

----

- [x] Add new chat type `InBroadcastChannel` and `OutBroadcastChannel`
for incoming / outgoing channels, where the former is similar to a
`Mailinglist` and the latter is similar to a `Broadcast` (which is
removed)
- Consideration for naming: `InChannel`/`OutChannel` (without
"broadcast") would be shorter, but less greppable because we already
have a lot of occurences of `channel` in the code. Consistently calling
them `BcChannel`/`bc_channel` in the code would be both short and
greppable, but a bit arcane when reading it at first. Opinions are
welcome; if I hear none, I'll keep with `BroadcastChannel`.
- [x] api: Add create_broadcast_channel(), deprecate
create_broadcast_list() (or `create_channel()` / `create_bc_channel()`
if we decide to switch)
  - Adjust code comments to match the new behavior.
- [x] Ask Desktop developers what they use `is_broadcast` field for, and
whether it should be true for both outgoing & incoming channels (or look
it up myself)
- I added `is_out_broadcast_channel`, and deprecated `is_broadcast`, for
now
- [x] When the user changes the broadcast channel name, immediately show
this change on receiving devices
- [x] Allow to change brodacast channel avatar, and immediately apply it
on the receiving device
- [x] Make it possible to block InBroadcastChannel
- [x] Make it possible to set the avatar of an OutgoingChannel, and
apply it on the receiving side
- [x] DECIDE whether we still want to use the broadcast icon as the
default icon or whether we want to use the letter-in-a-circle
- We decided to use the letter-in-a-circle for now, because it's easier
to implement, and I need to stay in the time plan
- [x] chat.rs: Return an error if the user tries to modify a
`InBroadcastChannel`
- [x] Add automated regression tests
- [x] Grep for `broadcast` and see whether there is any other work I
need to do
- [x] Bug: Don't show `~` in front of the sender's same in broadcast
lists

----

Note that I removed the following guard:

```rust
        if !new_chat_contacts.contains(&ContactId::SELF) {
            warn!(
                context,
                "Received group avatar update for group chat {} we are not a member of.", chat.id
            );
        } else if !new_chat_contacts.contains(&from_id) {
            warn!(
                context,
                "Contact {from_id} attempts to modify group chat {} avatar without being a member.",
                chat.id,
            );
        } else [...]
```

i.e. with this change, non-members will be able to modify the avatar.
Things were slightly easier this way, and I think that this is in line
with non-members being able to modify the group name and memberlist
(they need to know the Group-Chat-Id, anyway), but I can also change it
back.
2025-07-02 20:40:30 +00:00