Since V2 a contact is its key, there is no address-to-key binding left to verify.
The JSONRPC and FFI APIs are unchanged and report nothing as verified.
BREAKING CHANGE: the statistics JSON sent to the self-reporting-bot on Android changes: Contacts have `encrypted` instead of `verified` and lose `transitive_chain` properties and message stats have `encrypted` instead of `verified` and `unverified_encrypted`, and securejoin invites lose `already_verified`. The collecting bot stores incoming reports verbatim but analysis will have to make sense of older and newer reports.
Headers like From and Autocrypt are now added late,
right before sending the message over SMTP.
This way we advertise the latest list of transports
and use the correct From address in the encrypted part
even for messages queued while being offline.
BCC-self recipients are also added late.
For unencrypted messages we only want to send a copy
to the sending address, but we don't know the sending address
when queueing the message.
Adding bcc-self recipients when dequeuing the message
also makes it possible to send copies to updated list of relays.
.execute() was returning the number of rows, so usually 1.
.insert() is returning the row ID.
In most cases it does not matter because the result is checked with .is_empty(),
but send_msg_sync() actually uses the row IDs.
"ex" means "used to be, but no longer is".
"ext" means "extended", which is the intended meaning.
"Ext" is more common not only in the Rust ecosystem but in general.
We already had several people (myself included) asking
what "ex" is supposed to mean.
To reproduce this commit, search and replace `_ex(?!\w)` -> `_ext`.
But don't change `list_transports_ex` because it's public API.
this PR adds the "message pinned state" to the "reaction state" that is
already sent together with resent channel messages since #8496.
this change is done in the first commit, and in fact only changes few
lines (one can consider to rename "broadcast_reactions.rs" and related
stuff to "broadcast_state.rs" in another PR: i did not to that in this
PR to keep diff and review simple).
moreover, in the second commit ,the "selection of messages to resent" is
changed to keep an additional focus on the pinned messages, so that the
most recent pinned messages are resent as well.
successor of #8496 and #8546
this PR adds support for reactions in broadcast channels.
> the idea of broadcast reactions is that they are sent as usual from
subscribers to owner. after some time, the owner broadcasts them to all
subscribers, who only get to see reaction+count, not who-reacted-what
the PR is quite large, but a good share are tests and otherwise many
things are straight forward.
review should be done by-file, not by-commit. to make review easier,
here is a high-level overview:
first a change in the existing internal `Reactions` object was required.
before this PR, `Reactions` had a "contact to reaction map" only, and
the "frequencies map", that are actually mainly needed for UI, were
calculated as needed. with this PR, the "frequencies map" is the field
that always exist, the "contact map" is only available on top of that.
moreover, this PR shifts that part to the core, it was unfortunately in
the bindings before.
with that preparation things done as follows:
1. reactions from broadcast channel subscriber (`Chattype::InBroadcast`)
to broadcast channel owner (`Chattype::OutBroadcast`) are sent as usual,
only change for that step was to allow sending them at all
2. the owner receives reactions and saves them to the existing
`reactions` table as usual. additionally, the changed message is
remembered in `reactions_need_broadcast` table
3. in the IMAP loop, when ~10 minutes have passed, and
`reactions_need_broadcast` contains entries, a single, hidden message
with accumulated reactions is sent. this message may contain reactions
to different messages. for each message, all known reactions are sent as
reaction+count.
4. subscriber receive that message and save the accumulated reactions in
`reactions_broadcasted`
6. `get_message_reactions` is adapted so that `frequencies` are set
independently of who-reacted-what (the old and only field).
who-reacted-what is called `by_contact` now. it is always set for
compatibility reasons, however, it is not exhaustive for subscribers.
in general, UI should work with frequencies, the API itself, however,
has not changed.
other tweaks:
- outgoing channels are muted on creation, and UI shall allow to
unmute/mute them as all other chats. reason is that reactions are
notified, but in many cases not of large interest. this is also what
telegram is doing
- to have an intermediate feedback when reacting, the local state should
include ones own reaction, even if it is not yet broadcasted. for that,
we modify `reactions_broadcasted` using `modify_frequencies()` as needed
when sending an reaction. there are still some situations where the
update may not include ones own reaction, in this case it is added
lately by `refine_frequencies()`, so that `get_message_reactions()`
always contain SELF.
(in a first implementation, we always increased SELF reaction in
refine_frequencies(), however, that was worse and led to SELF counted
twice once the owner sent broadcast)
<details>
<summary>wire format</summary>
wire format is a JSON in the `Broadcast-Reactions:` header.
additionally, `Content-Disposition: reaction` is set to not show the
hidden message on existing devices.
using a header also allows us to broadcast reactions with resent channel
messages (on joining) later.
```
{
"messages": [
{
"id": "12345678",
"reactions": [
{ "emoji": "👍", "count": 4 },
{ "emoji": "🎉", "count": 2 }
]
},
{
"id": "23456789",
"reactions": []
}
]
}
```
for `id`, the wire format needs to use `rfc724_mid` as `msg_id` are
local only.
</details>
### known issues
- if the channel owner uses multiple devices, broadcasted reaction
updates are sent from each device. the updates are not that big, so that
is probably not a big deal. if it turns out that this is an issue, we
can think about fixes in another PR. might be done by restarting our
10-minute-wait once we see an update from another device
- we cannot set contact_id for DC_EVENT_REACTIONS_CHANGED - but i doubt
it was ever used
### for another pr
- ~~add `Broadcast-Reactions:` header also for resent channel messages,
so that new subscriber do not only get the latest messages, but also
their reactions. for that, the `Broadcast-Reactions:` header can go to
the corresponding message, no need to send extra messages. we would need
to change the sending part to send all reactions for a given message. on
receiving part, we need to make sure, `receive_broadcast_reactions()` is
called when the message actually exist.~~
EDIT: subsequent PR for resending broadcast reactions at
https://github.com/chatmail/core/pull/8496
- add api to allow only a subset of reactions, fiter incoming reactions
before broadcasting
### misc.
ui pr: https://github.com/deltachat/deltachat-ios/pull/3225 and
https://github.com/deltachat/deltachat-android/pull/4560 , which both
were tested successfully with this core PR already. desktop is meant to
be done once this is merged
---------
Co-authored-by: l <link2xt@testrun.org>
This change separates rendering into two separate steps:
1. Rendering of the message payload without the From, Date and Autocrypt headers.
2. Adding the From, Date and Autocrypt headers and possibly encrypting the message.
The goal is to have serializable result of the first step
that can be persisted in the database and sent later with any email address.
This way it will be possible to send queued messages over any relay.
This will make it possible not to remove all messages from the queue
when the sending relay is changed.
Currently changing `configured_addr` deletes everything from `smtp` table.
This change is however only a refactoring and does not implement any features.
Unifies internal naming of single/direct/1:1/normal chats
to match the naming used in the API.
Closes: #8368
Signed-off-by: Jagoda Ślązak <jslazak@jslazak.com>
---------
Signed-off-by: Jagoda Ślązak <jslazak@jslazak.com>
Remove is_chatmail flag, and rely on IMAP metadata
advertising the recipients limit (usually 1000),
falling back to 50 (or fewer in some exceptional cases)
just as before when is_chatmail was false.
Also, server metadata is now keyed per transport,
which in the future eases collecting ICE servers
from all relays (instead of just any first relay).
Prevents showing "empty" chat on other devices,
when pinning a chat that did not complete securejoin yet.
Fixes: #7713
Signed-off-by: Jagoda Ślązak <jslazak@jslazak.com>
Otherwise it's not possible to see the full message text if it's too long in a received edit. On the
sender side let it still be text so that the message can be edited again.
I.e. don't fail `remove_contact_from_chat()` for such a contact because there may be a race
condition with a remote removal of the contact done without trace, but don't send a removal message
and sync message in this case and don't emit a `ChatModified` event.
If a contact is already a past member, we still send a removal message to the chat, this is safe and
protects from lost removal messages, so there's no need to complicate the code in this case.
Fix https://github.com/chatmail/core/issues/8250. Not sure why anyone
would tamper with the link, but maybe the name gets ellipsized because
it is very long, or maybe we decide to ellipsize more aggressively in
the future.
The fix is easy enough, just set the GroupNameTimestamp when creating
the channel, so that `HeaderDef::ChatGroupNameTimestamp` gets set when
sending a message and the logic in
`apply_chat_name_avatar_and_description_changes()` recognizes the
incoming name's timestamp as newer than `chat_group_name_timestamp`
(which is 0 right after joining a chat).
The test is somewhat hacky and depends on the title being the last
parameter in the invite code URL, but this is fine; if we ever change
that, then the test will fail and we need to modify it.
This also removes some wrong doc comments.
Since groups are unpromoted until sending a first message, the same fix isn't needed for groups.
`OutPreparing` is deprecated since 2024-12-07, replace it with `OutFailed`, such messages are
probably not interesting anymore. `OutMdnRcvd` is not used in the db for new messages since
a30c6ae1f7, `OutDelivered` is stored instead.
Otherwise if it's forwarded to a device that already has the original
message, the pre-message isn't received and this results in a message
having no text and `Forwarded` param.
Co-authored-by: Hocuri <hocuri@gmx.de>
Currently, the Fingerprint type implements Display, but this doesn't get
you the canonical fingerprint representation, but something
human-readable. This is confusing, and back when I first used
`Fingerprint`, I immediately wrote a bug because of this. So, instead,
make a function `human_readable()` on Fingerprint.
This comes from the discussion at
https://github.com/chatmail/core/pull/8174#discussion_r3143130722.
Remove unused info_only option from `get_chat_msgs_ex()`.
This option was meant to show an "audit log" of a group, i.e. only the
info messages. This feature was removed again from Desktop, but the
option still lingered around in Core.
This also adds a doc comment to the JsonRPC functions, because I
wanted to note somewhere that the parameter is deprecated, and I needed
some place to put this note.
This causes problems for Delta Chat Desktop at
<https://github.com/deltachat/deltachat-desktop/pull/6278>
even though the logic was originally introduced for iOS.
If the problem remains on iOS,
heuristics can be added into iOS UI.
We shouldn't just revert ad7f873c68 because then we won't be able to
normally transfer a backup from a device having clock a bit in the future.
INDEXED BY msgs_index7 is to prevent SQLite from downgrading to using msgs_index2 which has less
ordering.
Last 10 messages in a broadcast channel are resent. They are sent and encrypted only to the new member, not to other subscribers.
Close#7678
Based on https://github.com/chatmail/core/pull/7854, with the following
changes:
- Refactor and simplify code, don't reuse the existing `get_chat_msgs()`
function
- Document that Param::Arg4 is also used for resent messages
cc818d9099
- It's unclear how exactly to resend webxdc status updates. After
discussing with @r10s, don't resend webxdc's at all for now.
38d57ebb30
- Don't set fake `msg_id` in resent messages
e7d0687d90
Setting the msg_id to `u32::MAX` is hacky, and may just as well break
things as it may fix things, because some code may use the msg.id to
load information from the database, like `get_iroh_topic_for_msg()`.
From reading the code, I couldn't find any problem with leaving the
correct `msg_id`, and if there is one, then we should add a function
parameter `is_resending` that is checked in the corresponding places.
Easiest to review file-by-file rather than individual commits, probably.
I'll squash-merge this.
---------
Co-authored-by: iequidoo <dgreshilov@gmail.com>
Otherwise `Param::GuaranteeE2ee` is set only after rendering the message
and some UIs, e.g. DC Android, display the message as unencrypted while
preparing the blob and rendering and encrypting the message. NB: DC
Desktop doesn't display the message until `send_msg()` returns.
---------
Co-authored-by: Hocuri <hocuri@gmx.de>
Large size of Mimefactor.render() futures
increases the size of all callers
down to set_config() and background_fetch().
I also had to Box::pin one call to fetch_new_msg_batch
and one call to fetch_single_msg.
Before, if the user fixed their clock incorrectly set to the future, they needed to delete
previously sent messages or wait until this future comes again so that new sent messages are added
to the bottom. Strictly speaking, the problem isn't fixable because we don't know if messages were
incorrectly timestamped into the future or they are timestamped correctly and the clock is now
incorrectly set to the past. Anyway, adding messages to the middle of the chat isn't a good way to
inform the user about the problem.
We already set sort_timestamp to 0 for "Messages are end-to-end encrypted."
since 8f1bf963b4.
Do this for "Others will only see this group after you sent a first message."
and "Messages in this chat use classic email and are not encrypted." as well
so no messages can be added on top.
This is to avoid sorting incoming messages that
are slightly in the past above system messages
about SecureJoin. SecureJoin messages are
timed according to smeared timestamp,
so even in the local tests they are in the future
by a few seconds.
We set timestamp of this info message to 0
to make it always appear in the beginning of the chat.
To avoid new chats being sorted to the end of the chatlist,
we ignore such 0 and use chat creation timestamp
when sorting the chatlist.
