Closes#8693 and supsersedes #8707 from which the API
and some overall shape of this commit is inspired.
UIs call `init_transports(None)` or `init_transports(qr)`
to initialize a first transport on the fresh profile,
with more transports added in the background later.
`list_transports()` should be used instead.
Previously `addr` field might have even not correspond to any transport
as it was `Config::Addr` and not `Config::ConfiguredAddr`.
BREAKING CHANGE: list_transports() should be used instead.
UIs should use display name everywhere and avoid displaying email addresses.
BREAKING CHANGE: dc_contact_get_name_n_addr() CFFI is removed
BREAKING CHANGE: JSON-RPC contact objects don't have nameAndAddr field anymore
Also adds tests and docs to respective functions,
clarifying background fetching behaviour and the `ACCOUNTS_BACKGROUND_FETCH_DONE` event,
that came up in questions/discussions with UI devs lately.
Core no longer tracks verification, so the API has nothing left
to report and UIs should drop their checkmark and "Introduced by" code.
BREAKING CHANGE: dc_contact_is_verified() and dc_contact_get_verifier_id() are removed.
BREAKING CHANGE: the JSON-RPC Contact object loses the `isVerified` and `verifierId` fields. A bot reading `snapshot.is_verified` now gets an `AttributeError` at runtime.
BREAKING CHANGE: the Python bindings lose `Contact.is_verified()` and `Contact.get_verifier()`.
BREAKING CHANGE: DC_STR_CONTACT_VERIFIED (35) is removed, so UIs should stop registering a translation for it. A stock id core does not know is logged and otherwise ignored, so an un-updated client keeps working.
this PR adds a hint about how to reset a QR code.
the API is as-is, just now, a refactoring here is also unwanted. the
jsonrpc documentation needs to improved in general, this is known, and
also not done by this PR
closes#7985
---------
Co-authored-by: adb <asieldbenitez@gmail.com>
A second background_fetch() while one is already running returned
without emitting the event, and the FFI returned 1 for it,
so a UI waiting for an event hangs dc_get_next_event().
Emit the event in any case, so waiting for it is safe.
With I/O stopped, `background_fetch()` connected only to the transport of `configured_addr`
and we now instead fan out to all transports in a controlled loop.
If a first transport finished fetching new messages
cancel all other attempts and return.
This is meant to address the problem that amzd described
where a profile with one functioning and one hanging transport,
shows the first notification, then hangs 15 seconds waiting for the hanging transport.
meanwhile a second NSE arrives and dies, and the second message is not notified
or only generically.
Also drop the quota check from this background fetch path:
its result is in-memory only, discarded when the iOS notification service exits,
and the regular scheduler fetching refreshes it every 60s anyway.
Moreover, quota errors/running full is pretty rare
since relays generally automatically stay under quota these days.
It's another round trip for each transport of each profile and simply not necessary.
Also adds previously missing online tests.
BREAKING CHANGE: get_webxdc_blob() may fail to load icon.png or icon.jpg if image dimensions are too large.
The issue is discovered by https://github.com/Sergei768
Removing a relay now takes effect immediately:
- the profile stops fetching and advertising it,
- secondary devices immediately apply the removal through the transport sync,
Upgrading removes unpublished relays and triggers keyupdates.
BREAKING CHANGE: set_transport_unpublished() is removed: UIs call delete_transport() when the user removes a relay.
BREAKING CHANGE: list_transports_ex() and the TransportListEntry type are removed: use list_transports().
BREAKING CHANGE: delete_transport() no longer refuses to remove the primary transport: it refuses only to remove the last one and re-elects the sending transport as needed.
BREAKING CHANGE: TransportsModified is now also emitted on the device modifying the transports, not only on devices applying the synced change.
Deprecated: DC_STR_PHASING_OUT
`version_string` is meant to be displayed by UI, and comes from the
wire.
therefore, as a general precaution, ensure a string that is
regarded as a typical version string. all versions in scope are
currently v123.456.789-shortsuffix, where suffix is a-z and mostly
unused in production. that is the base. we can adapt if there is really
a need, but not for theoretical version strings. as we do not stop
processing, things are not bad even if we missed a valid usecase herr.
moreover, if `version_string` is empty, we skip the candidate - as we
cannot display something useful to the user. that little bit of care is
expected from relays :)
"ex" means "used to be, but no longer is".
"ext" means "extended", which is the intended meaning.
"Ext" is more common not only in the Rust ecosystem but in general.
We already had several people (myself included) asking
what "ex" is supposed to mean.
To reproduce this commit, search and replace `_ex(?!\w)` -> `_ext`.
But don't change `list_transports_ex` because it's public API.
> we aim to inform about updates for installations outside of any
appstore soon.
>
> there is already a PR on android at
https://github.com/deltachat/deltachat-android/pull/4582, however, the
information about "update available" is a mockup there.
>
> in general, there are 3 ideas around about how to gather the "update
available" infomation - (1) checking a central url, (2) let contacts
provide information, (3) let relay provide information. on various
one-to-one discussions, outcome is that (3) is the most reasonable way
to go.
this PR is about reading update information via IMAP metadata from the
relay.
it is up to the UI to call `get_app_version()` at a reasonable time and
frequency, see comment in the code. when called, `get_app_version()`
iterates over all known profiles and relays and checks for version
information, returning the newest for the given scope.
we do not use an event, as that is tricky wrt changes - we do not know
if other relays report later a newer version. we also do not cache
anything, to prevent bad relays avoiding us to update permanently. also
it is easier :)
<details>
<summary>outdated notes and questions</summary>
- ~~it is up to the clients to get the needed information, we could let
core filter, but it seems easy enough the other way round, and may have
debug advantages, one can iterate etc.~~ EDIT: we now filter in core,
this also makes the jsonrpc part easier, see review comments
- when is IMAP METADATA actually read? when are they ready? is that
really the correct place? i am up to change that, but beware, this is
not really my expertise, so someone else may need to take over :)
EDIT: see below, IMAP METADATA is read on connection, before fetching
starts, usually fast enough
- relay part is missing. once the format is settled and discussed
shortly here, that should be done soon as well. but this is definitely
not my expertise and needed to be done by someone else :)
- key for IMAP METADATA is `/shared/vendor/deltachat/appversions` -
shall we continue use `deltachat` for compatibility or so? `chatmail`
seems to be more correct
EDIT: we stay with the current
</details>
relay counterpart issue: https://github.com/chatmail/relay/issues/1037
cc @link2xt @Hocuri @hpk42
---------
Co-authored-by: holger krekel <holger@merlinux.eu>
The chat is always going to be encrypted, this is enough.
We actually do mark the contact as verified,
but the meaning of this verification is unclear
and we don't even display it visibly in the chat anymore.
this PR, created together with @adbenitez, improves the connectivity
state passed to UI, which has changed a lot since multi relay:
- change the algorithm for `get_connectivity()`: this is roughly the
"best" connectivity of all relays now, so if one is connected, we're
already fine. this was discussed widely one to one, and that part
already closes#8554
- the PR adds a test for that, previously, that was untested
- additionally, do not regard unpublished relays in
`get_connectivity()`: e.g. unpublished relays are no longer given to
peers - so if only that is connected, the overall state should not be
"connected". therefore, we just ignore unpublished relays there
- in `get_connectivity_html()`, we continue showing unpublished relays,
however, we tune them down visually and flag them as such
- for the docs, remove the "range for some future use" wording. it was
never used like that, and that future will probably not arrive :)
<img width="320"
src="https://github.com/user-attachments/assets/1805c525-72d1-4197-b865-6e2d6fbc8819"
/>
---------
Co-authored-by: Hocuri <hocuri@gmx.de>
this PR adds support for reactions in broadcast channels.
> the idea of broadcast reactions is that they are sent as usual from
subscribers to owner. after some time, the owner broadcasts them to all
subscribers, who only get to see reaction+count, not who-reacted-what
the PR is quite large, but a good share are tests and otherwise many
things are straight forward.
review should be done by-file, not by-commit. to make review easier,
here is a high-level overview:
first a change in the existing internal `Reactions` object was required.
before this PR, `Reactions` had a "contact to reaction map" only, and
the "frequencies map", that are actually mainly needed for UI, were
calculated as needed. with this PR, the "frequencies map" is the field
that always exist, the "contact map" is only available on top of that.
moreover, this PR shifts that part to the core, it was unfortunately in
the bindings before.
with that preparation things done as follows:
1. reactions from broadcast channel subscriber (`Chattype::InBroadcast`)
to broadcast channel owner (`Chattype::OutBroadcast`) are sent as usual,
only change for that step was to allow sending them at all
2. the owner receives reactions and saves them to the existing
`reactions` table as usual. additionally, the changed message is
remembered in `reactions_need_broadcast` table
3. in the IMAP loop, when ~10 minutes have passed, and
`reactions_need_broadcast` contains entries, a single, hidden message
with accumulated reactions is sent. this message may contain reactions
to different messages. for each message, all known reactions are sent as
reaction+count.
4. subscriber receive that message and save the accumulated reactions in
`reactions_broadcasted`
6. `get_message_reactions` is adapted so that `frequencies` are set
independently of who-reacted-what (the old and only field).
who-reacted-what is called `by_contact` now. it is always set for
compatibility reasons, however, it is not exhaustive for subscribers.
in general, UI should work with frequencies, the API itself, however,
has not changed.
other tweaks:
- outgoing channels are muted on creation, and UI shall allow to
unmute/mute them as all other chats. reason is that reactions are
notified, but in many cases not of large interest. this is also what
telegram is doing
- to have an intermediate feedback when reacting, the local state should
include ones own reaction, even if it is not yet broadcasted. for that,
we modify `reactions_broadcasted` using `modify_frequencies()` as needed
when sending an reaction. there are still some situations where the
update may not include ones own reaction, in this case it is added
lately by `refine_frequencies()`, so that `get_message_reactions()`
always contain SELF.
(in a first implementation, we always increased SELF reaction in
refine_frequencies(), however, that was worse and led to SELF counted
twice once the owner sent broadcast)
<details>
<summary>wire format</summary>
wire format is a JSON in the `Broadcast-Reactions:` header.
additionally, `Content-Disposition: reaction` is set to not show the
hidden message on existing devices.
using a header also allows us to broadcast reactions with resent channel
messages (on joining) later.
```
{
"messages": [
{
"id": "12345678",
"reactions": [
{ "emoji": "👍", "count": 4 },
{ "emoji": "🎉", "count": 2 }
]
},
{
"id": "23456789",
"reactions": []
}
]
}
```
for `id`, the wire format needs to use `rfc724_mid` as `msg_id` are
local only.
</details>
### known issues
- if the channel owner uses multiple devices, broadcasted reaction
updates are sent from each device. the updates are not that big, so that
is probably not a big deal. if it turns out that this is an issue, we
can think about fixes in another PR. might be done by restarting our
10-minute-wait once we see an update from another device
- we cannot set contact_id for DC_EVENT_REACTIONS_CHANGED - but i doubt
it was ever used
### for another pr
- ~~add `Broadcast-Reactions:` header also for resent channel messages,
so that new subscriber do not only get the latest messages, but also
their reactions. for that, the `Broadcast-Reactions:` header can go to
the corresponding message, no need to send extra messages. we would need
to change the sending part to send all reactions for a given message. on
receiving part, we need to make sure, `receive_broadcast_reactions()` is
called when the message actually exist.~~
EDIT: subsequent PR for resending broadcast reactions at
https://github.com/chatmail/core/pull/8496
- add api to allow only a subset of reactions, fiter incoming reactions
before broadcasting
### misc.
ui pr: https://github.com/deltachat/deltachat-ios/pull/3225 and
https://github.com/deltachat/deltachat-android/pull/4560 , which both
were tested successfully with this core PR already. desktop is meant to
be done once this is merged
---------
Co-authored-by: l <link2xt@testrun.org>
Largest change is in the FFI crate.
With 2024 (but not 2021) edition unsafe code
inside unsafe functions should be marked separately
so we can mark exactly the code that is unsafe.
Some CFFI functions even have no unsafe code inside.
Most interesting change is that .strdup()
functions are not marked as unsafe anymore.
They are allocating memory and return raw pointers,
but there is nothing unsafe about it.
Only using the returned raw pointers is unsafe.
This way calls to .strdup() don't have to be marked
with unsafe{} blocks.
Unifies internal naming of single/direct/1:1/normal chats
to match the naming used in the API.
Closes: #8368
Signed-off-by: Jagoda Ślązak <jslazak@jslazak.com>
---------
Signed-off-by: Jagoda Ślązak <jslazak@jslazak.com>
fix https://github.com/chatmail/core/issues/8463
Followup to https://github.com/chatmail/core/pull/8431: Re-dd `oauth2`
to the two structs that serialized and sent over the wire. This solves
an incompatibility problem where transports could not be synced to older
versions of DC, and profiles could not be transferred, because the
deserializer expected the field to be present.
I now added `#[serde(default)]`, so that with this PR, Delta Chat will
be compatible with both v2.56 and with older versions.
BREAKING CHANGE: provider lookup APIs were removed from CFFI and JSON-RPC.
also removes offline provider database code and generated provider data,
provider-specific fields in configure/transport paths, and REPL providerinfo.
Heartbeat notifications are only used on iOS for classic mail servers
but both code and privacy wise (notification server sees IP addresses from those users)
not something we want to support any longer (was discussed with bjoern).
BREAKING CHANGE: removed oauth2 module, dc_get_oauth2_url FFI function, DC_LP_AUTH flags and configured/serverflags, and the oauth2 parameter/field from SMTP/IMAP clients, JSON-RPC interfaces, and CLI tools.
also contains regenerated provider data after dropping oauth in the update script.
