this PR, created together with @adbenitez, improves the connectivity
state passed to UI, which has changed a lot since multi relay:
- change the algorithm for `get_connectivity()`: this is roughly the
"best" connectivity of all relays now, so if one is connected, we're
already fine. this was discussed widely one to one, and that part
already closes#8554
- the PR adds a test for that, previously, that was untested
- additionally, do not regard unpublished relays in
`get_connectivity()`: e.g. unpublished relays are no longer given to
peers - so if only that is connected, the overall state should not be
"connected". therefore, we just ignore unpublished relays there
- in `get_connectivity_html()`, we continue showing unpublished relays,
however, we tune them down visually and flag them as such
- for the docs, remove the "range for some future use" wording. it was
never used like that, and that future will probably not arrive :)
<img width="320"
src="https://github.com/user-attachments/assets/1805c525-72d1-4197-b865-6e2d6fbc8819"
/>
---------
Co-authored-by: Hocuri <hocuri@gmx.de>
the wire format is already prepared to carry information additionally to reactions,
so let the header name reflect that as well.
in practise, we might want to use a `pinned` flag very soon
in case pinned messages got resent in channels.
it makes sense if that flag goes to the message as such,
and not to an extra message.
and then it makes sense to not introduce a new things.
ftr, this is about resending state to other members,
it is not about syncing across devices for the same user.
the code as such is not changed by this PR,
it is only a wording preparation, of a header that is not even visible.
if we semantically put things other than reactions to the wire format,
we may want to move it out of `broadcast_reactions.rs`
the UIs are currently advised
to only allow the five default reactions in broadcast channels.
this PR ensures that from sending site as well as receiving site.
soon, we probably want to make the possible reactions configurable, this
PR is mainly for some safety until then. once we have an API to change
default reactions, we can also easily test "invalid" reactions on the
receiving side; currently only sending is tested
successor of https://github.com/chatmail/core/pull/8450
---------
Co-authored-by: Jagoda Estera Ślązak <128227338+j-g00da@users.noreply.github.com>
this PR adds support for reactions in broadcast channels.
> the idea of broadcast reactions is that they are sent as usual from
subscribers to owner. after some time, the owner broadcasts them to all
subscribers, who only get to see reaction+count, not who-reacted-what
the PR is quite large, but a good share are tests and otherwise many
things are straight forward.
review should be done by-file, not by-commit. to make review easier,
here is a high-level overview:
first a change in the existing internal `Reactions` object was required.
before this PR, `Reactions` had a "contact to reaction map" only, and
the "frequencies map", that are actually mainly needed for UI, were
calculated as needed. with this PR, the "frequencies map" is the field
that always exist, the "contact map" is only available on top of that.
moreover, this PR shifts that part to the core, it was unfortunately in
the bindings before.
with that preparation things done as follows:
1. reactions from broadcast channel subscriber (`Chattype::InBroadcast`)
to broadcast channel owner (`Chattype::OutBroadcast`) are sent as usual,
only change for that step was to allow sending them at all
2. the owner receives reactions and saves them to the existing
`reactions` table as usual. additionally, the changed message is
remembered in `reactions_need_broadcast` table
3. in the IMAP loop, when ~10 minutes have passed, and
`reactions_need_broadcast` contains entries, a single, hidden message
with accumulated reactions is sent. this message may contain reactions
to different messages. for each message, all known reactions are sent as
reaction+count.
4. subscriber receive that message and save the accumulated reactions in
`reactions_broadcasted`
6. `get_message_reactions` is adapted so that `frequencies` are set
independently of who-reacted-what (the old and only field).
who-reacted-what is called `by_contact` now. it is always set for
compatibility reasons, however, it is not exhaustive for subscribers.
in general, UI should work with frequencies, the API itself, however,
has not changed.
other tweaks:
- outgoing channels are muted on creation, and UI shall allow to
unmute/mute them as all other chats. reason is that reactions are
notified, but in many cases not of large interest. this is also what
telegram is doing
- to have an intermediate feedback when reacting, the local state should
include ones own reaction, even if it is not yet broadcasted. for that,
we modify `reactions_broadcasted` using `modify_frequencies()` as needed
when sending an reaction. there are still some situations where the
update may not include ones own reaction, in this case it is added
lately by `refine_frequencies()`, so that `get_message_reactions()`
always contain SELF.
(in a first implementation, we always increased SELF reaction in
refine_frequencies(), however, that was worse and led to SELF counted
twice once the owner sent broadcast)
<details>
<summary>wire format</summary>
wire format is a JSON in the `Broadcast-Reactions:` header.
additionally, `Content-Disposition: reaction` is set to not show the
hidden message on existing devices.
using a header also allows us to broadcast reactions with resent channel
messages (on joining) later.
```
{
"messages": [
{
"id": "12345678",
"reactions": [
{ "emoji": "👍", "count": 4 },
{ "emoji": "🎉", "count": 2 }
]
},
{
"id": "23456789",
"reactions": []
}
]
}
```
for `id`, the wire format needs to use `rfc724_mid` as `msg_id` are
local only.
</details>
### known issues
- if the channel owner uses multiple devices, broadcasted reaction
updates are sent from each device. the updates are not that big, so that
is probably not a big deal. if it turns out that this is an issue, we
can think about fixes in another PR. might be done by restarting our
10-minute-wait once we see an update from another device
- we cannot set contact_id for DC_EVENT_REACTIONS_CHANGED - but i doubt
it was ever used
### for another pr
- ~~add `Broadcast-Reactions:` header also for resent channel messages,
so that new subscriber do not only get the latest messages, but also
their reactions. for that, the `Broadcast-Reactions:` header can go to
the corresponding message, no need to send extra messages. we would need
to change the sending part to send all reactions for a given message. on
receiving part, we need to make sure, `receive_broadcast_reactions()` is
called when the message actually exist.~~
EDIT: subsequent PR for resending broadcast reactions at
https://github.com/chatmail/core/pull/8496
- add api to allow only a subset of reactions, fiter incoming reactions
before broadcasting
### misc.
ui pr: https://github.com/deltachat/deltachat-ios/pull/3225 and
https://github.com/deltachat/deltachat-android/pull/4560 , which both
were tested successfully with this core PR already. desktop is meant to
be done once this is merged
---------
Co-authored-by: l <link2xt@testrun.org>
This adds a test for the unencrypted headers, because
https://github.com/chatmail/core/pull/8345/ changes how these are
rendered, and so far we didn't have any tests for them.
This change separates rendering into two separate steps:
1. Rendering of the message payload without the From, Date and Autocrypt headers.
2. Adding the From, Date and Autocrypt headers and possibly encrypting the message.
The goal is to have serializable result of the first step
that can be persisted in the database and sent later with any email address.
This way it will be possible to send queued messages over any relay.
This will make it possible not to remove all messages from the queue
when the sending relay is changed.
Currently changing `configured_addr` deletes everything from `smtp` table.
This change is however only a refactoring and does not implement any features.
Otherwise users may add up to 5 relays, but 2 oldest relays
are actually ignored.
Having the maximum number of published relays
and the number of relays used the same between all clients
makes the order of relays in the key irrelevant.
We may even remove sorting by `add_timestamp`
from `get_all_self_addrs()` in the future.
There was a bug in prefetch_should_download() that it made it return
true for pre-messages even when they were already downloaded. This meant
that pre-messages were downloaded from all relays, rather than just one,
wasting internet data.
The fix is in rfc724_mid_download_tried(), which is used by
prefetch_should_download() to determine whether a message was already
downloaded.
---------
Co-authored-by: l <link2xt@testrun.org>
If multi-relay onboarding config is set from UIs, automatically add
relays until there are 3 relays. For now, there will be a hardcoded list
of relay candidates.
- We need a list of chatmail relays that we somehow trust, and that
agree to be in the list. Then, we add all of them to the candidate list
(a new SQL table with colums "host" and "last_tried").
- Before going to IMAP IDLE: When there are less than 3 relays, fill it
up with relays from the candidate list. If creating an account fails,
try again with another relay from the list. For each candidate, we need
to remember the last time we tried to add a transport there, and try at
most once a week or so per candidate.
- For now, this will be behind an off-by-default config option, which at
least DC Android will enable when creating a new profile. UIs can then
opt in on their own pace but will likely need to disable it for tests.
- Right now, the backoff times are: Try to add a relay at most once per
hour, and try to add the same relay at most once per week
register device token if XDELTAPUSH IMAP capability is available (current relay setup provides it)
or if "maxsmtprecipients" IMAP metadata key is available (relays since May 2026 provide it),
allowing chatmail relay setups to drop XDELTAPUSH capability marker with the next release
while retaining push notification support.
The lint is reasonable, but without some work
to bring the future sizes way below the limit,
this lint is frequently triggered in unrelated PRs
that just add some local variables to async functions
or extend some structure that is passed on the stack.
- treat an Autocrypt header as a kind of Autocrypt-Gossip: addr=<self>
letting it participate in the existing cooperative key gossipping mechanics.
- speed up processing of incoming autocrypt-gossip headers by using
1 sql commit for N gossip headers instead of the current N commits.
Unifies internal naming of single/direct/1:1/normal chats
to match the naming used in the API.
Closes: #8368
Signed-off-by: Jagoda Ślązak <jslazak@jslazak.com>
---------
Signed-off-by: Jagoda Ślązak <jslazak@jslazak.com>
`unwrap_or` is eagerly evaluated,
which caused always printing the WARN log,
even if EXIF orientation value is correct
(and not ignored).
Signed-off-by: Jagoda Ślązak <jslazak@jslazak.com>
The `FolderMeaning` and the `target_folder()` function and related code
were needed for the message moving logic. The message moving logic isn't
needed anymore, so, this PR is a first step to remove it.
We only ever look at one folder by now (usually the Inbox folder), so,
any message moving logic isn't needed anymore because we will never
notice a message that is supposed to be moved.
The `target` column of the `imap` table is mostly the same as the
`folder` column now. Except when the message is supposed to be deleted,
then it's an empty string.
fix https://github.com/chatmail/core/issues/8463
Followup to https://github.com/chatmail/core/pull/8431: Re-dd `oauth2`
to the two structs that serialized and sent over the wire. This solves
an incompatibility problem where transports could not be synced to older
versions of DC, and profiles could not be transferred, because the
deserializer expected the field to be present.
I now added `#[serde(default)]`, so that with this PR, Delta Chat will
be compatible with both v2.56 and with older versions.
BREAKING CHANGE: provider lookup APIs were removed from CFFI and JSON-RPC.
also removes offline provider database code and generated provider data,
provider-specific fields in configure/transport paths, and REPL providerinfo.
Heartbeat notifications are only used on iOS for classic mail servers
but both code and privacy wise (notification server sees IP addresses from those users)
not something we want to support any longer (was discussed with bjoern).
Remove is_chatmail flag, and rely on IMAP metadata
advertising the recipients limit (usually 1000),
falling back to 50 (or fewer in some exceptional cases)
just as before when is_chatmail was false.
Also, server metadata is now keyed per transport,
which in the future eases collecting ICE servers
from all relays (instead of just any first relay).
