With I/O stopped, `background_fetch()` connected only to the transport of `configured_addr`
and we now instead fan out to all transports in a controlled loop.
If a first transport finished fetching new messages
cancel all other attempts and return.
This is meant to address the problem that amzd described
where a profile with one functioning and one hanging transport,
shows the first notification, then hangs 15 seconds waiting for the hanging transport.
meanwhile a second NSE arrives and dies, and the second message is not notified
or only generically.
Also drop the quota check from this background fetch path:
its result is in-memory only, discarded when the iOS notification service exits,
and the regular scheduler fetching refreshes it every 60s anyway.
Moreover, quota errors/running full is pretty rare
since relays generally automatically stay under quota these days.
It's another round trip for each transport of each profile and simply not necessary.
Also adds previously missing online tests.
Removing a relay now takes effect immediately:
- the profile stops fetching and advertising it,
- secondary devices immediately apply the removal through the transport sync,
Upgrading removes unpublished relays and triggers keyupdates.
BREAKING CHANGE: set_transport_unpublished() is removed: UIs call delete_transport() when the user removes a relay.
BREAKING CHANGE: list_transports_ex() and the TransportListEntry type are removed: use list_transports().
BREAKING CHANGE: delete_transport() no longer refuses to remove the primary transport: it refuses only to remove the last one and re-elects the sending transport as needed.
BREAKING CHANGE: TransportsModified is now also emitted on the device modifying the transports, not only on devices applying the synced change.
Deprecated: DC_STR_PHASING_OUT
Requests now register before testing for shutdown,
so either the reader loop or the caller answers them.
Also failed start() winds down its threads, ignoring a broken pipe on stdin.
Adds following stock strings:
- You were removed by %1$s.
- You were added by %1$s.
and implicit equivalents:
- You were removed.
- You were added.
Closes: #8419
Signed-off-by: Jagoda Ślązak <jslazak@jslazak.com>
previously only 10 messages were sent but if joinining did not succeed (due to slow CI sometimes) nothing would be received and the test would hang.
Also cleans up the tests to not set webxdc realtime config (true by default) and restructure helpers to reduce redundant code.
unsuspecting consumers will otherwise require execnet to run pytest with deltachat-rpc-client installed.
"import py" is discouraged for pytest these days but still supported in compatibility mode.
Unifies internal naming of single/direct/1:1/normal chats
to match the naming used in the API.
Closes: #8368
Signed-off-by: Jagoda Ślązak <jslazak@jslazak.com>
---------
Signed-off-by: Jagoda Ślązak <jslazak@jslazak.com>
Heartbeat notifications are only used on iOS for classic mail servers
but both code and privacy wise (notification server sees IP addresses from those users)
not something we want to support any longer (was discussed with bjoern).
BREAKING CHANGE: removed oauth2 module, dc_get_oauth2_url FFI function, DC_LP_AUTH flags and configured/serverflags, and the oauth2 parameter/field from SMTP/IMAP clients, JSON-RPC interfaces, and CLI tools.
also contains regenerated provider data after dropping oauth in the update script.
Fix https://github.com/chatmail/core/issues/8250. Not sure why anyone
would tamper with the link, but maybe the name gets ellipsized because
it is very long, or maybe we decide to ellipsize more aggressively in
the future.
The fix is easy enough, just set the GroupNameTimestamp when creating
the channel, so that `HeaderDef::ChatGroupNameTimestamp` gets set when
sending a message and the logic in
`apply_chat_name_avatar_and_description_changes()` recognizes the
incoming name's timestamp as newer than `chat_group_name_timestamp`
(which is 0 right after joining a chat).
The test is somewhat hacky and depends on the title being the last
parameter in the invite code URL, but this is fine; if we ever change
that, then the test will fail and we need to modify it.
This also removes some wrong doc comments.
Since groups are unpromoted until sending a first message, the same fix isn't needed for groups.
`OutPreparing` is deprecated since 2024-12-07, replace it with `OutFailed`, such messages are
probably not interesting anymore. `OutMdnRcvd` is not used in the db for new messages since
a30c6ae1f7, `OutDelivered` is stored instead.
Remove unused info_only option from `get_chat_msgs_ex()`.
This option was meant to show an "audit log" of a group, i.e. only the
info messages. This feature was removed again from Desktop, but the
option still lingered around in Core.
This also adds a doc comment to the JsonRPC functions, because I
wanted to note somewhere that the parameter is deprecated, and I needed
some place to put this note.
After talking with r10s:
For spring cleaning, remove the largely-unused things that can be done a
bit.
Most private messengers (WhatsApp/Signal/...) do not have this feature,
and we do not want to become Matrix where every client has different,
partly-incompatible features.
I used some AI to draft a first version of this, and then reworked it.
This is one of multiple possibilities to fix
https://github.com/chatmail/core/issues/8041: For bots, the IncomingMsg
event is not emitted when the pre-message arrives, only when the
post-message arrives. Also, post-messages are downloaded immediately,
not after all the small messages are downloaded.
The `get_next_msgs()` API is deprecated. Instead, bots need to listen to
the IncomingMsg event in order to be notified about new events. Is this
acceptable for bots?
THE PROBLEM THAT WAS SOLVED BY THIS:
With pre-messages, it's hard for bots to wait for the message to be fully downloaded and then process it.
Up until now, bots used get_next_msgs() to query the unprocessed messages, then set last_msg_id after processing a message to that they won't process it again.
But this will now also return messages that were not fully downloaded.
ALTERNATIVES:
In the following, I will explain
the alternatives, and for why it's not so easy to just make the
`get_next_msgs()` API work. If it's not understandable, I'm happy to
elaborate more.
Core can't just completely ignore the pre-message for two reasons:
- If a post-message containing a Webxdc arrives later, and some webxdc updates arrive in the meantime, then these updates will be lost.
- The post-message doesn't contain the text (reasoning was to avoid duplicate text for people who didn't upgrade yet during the 2.43.0 rollout)
There are multiple solutions:
- Add the message as hidden in the database when the pre-message arrives.
- When the post-message arrives and we want to make it available for bots, we need to update the msg_id because of how the `get_next_msgs()` API works. This means that we need to update all webxdc updates that reference this msg_id.
- Alternatively, we could make webxdc's reference the rfc724_mid instead of the msg_id, so that we don't need stable msg_ids anymore
- Alternatively, we could deprecate `get_next_msgs()`, and ask bots to use plain events for message processing again. It's not that bad; worst case, the bot crashes and then forgets to react to some messages, but the user will just try again. And if some message makes the bot crash, then it might actually be good not to try and process it again.
- Store the pre-message text and `PostMsgMetadata` (or alternatively, the whole mime) in a new database table. Wait with processing it until the post-message arrives.
Additionally, the logic that small messages are downloaded before post-messages should be disabled for bots, in order to prevent reordering.
Rpc.start() now calls get_system_info() after launching the server
to verify it started successfully. If the server exits early (e.g.
due to an invalid accounts directory), the core error message from
stderr is captured and included in the raised JsonRpcError.
The reader_loop now unblocks pending RPC requests when the server
closes stdout, so callers never hang on a dead server.
Export JsonRpcError from the deltachat_rpc_client package.
Add test_early_failure verifying that Rpc.start() raises with
the actual core error message for invalid accounts directories.
This code does not expect the variable to be unset,
so use indexing to fail with KeyError instead.
Otherwise getenv() returns None which is then converted to "none" string by formatting
and the test only fails because of connection attempts to "none" domain.
This commit also makes testing hooks easier, as it allows to process
events and run hooks on them, until a certain event occurs.
---------
Co-authored-by: iequidoo <117991069+iequidoo@users.noreply.github.com>
This is to fix tests failing with `OSError: [Errno 9] Bad file descriptor`. Maybe stdout closes
earlier than stderr, before the test finishes, not sure. For reference, the previous commit removing
print()s is 800edc6fce.
