fix: For bots, wait with emitting IncomingMsg until the Post-Msg arrived (#8104)

I used some AI to draft a first version of this, and then reworked it.

This is one of multiple possibilities to fix
https://github.com/chatmail/core/issues/8041: For bots, the IncomingMsg
event is not emitted when the pre-message arrives, only when the
post-message arrives. Also, post-messages are downloaded immediately,
not after all the small messages are downloaded.

The `get_next_msgs()` API is deprecated. Instead, bots need to listen to
the IncomingMsg event in order to be notified about new events. Is this
acceptable for bots?

THE PROBLEM THAT WAS SOLVED BY THIS:

With pre-messages, it's hard for bots to wait for the message to be fully downloaded and then process it.

Up until now, bots used get_next_msgs() to query the unprocessed messages, then set last_msg_id after processing a message to that they won't process it again.

But this will now also return messages that were not fully downloaded.

ALTERNATIVES:

In the following, I will explain
the alternatives, and for why it's not so easy to just make the
`get_next_msgs()` API work. If it's not understandable, I'm happy to
elaborate more.

Core can't just completely ignore the pre-message for two reasons:
- If a post-message containing a Webxdc arrives later, and some webxdc updates arrive in the meantime, then these updates will be lost.
- The post-message doesn't contain the text (reasoning was to avoid duplicate text for people who didn't upgrade yet during the 2.43.0 rollout)

There are multiple solutions:
- Add the message as hidden in the database when the pre-message arrives.
  - When the post-message arrives and we want to make it available for bots, we need to update the msg_id because of how the `get_next_msgs()` API works. This means that we need to update all webxdc updates that reference this msg_id.
  - Alternatively, we could make webxdc's reference the rfc724_mid instead of the msg_id, so that we don't need stable msg_ids anymore
  - Alternatively, we could deprecate `get_next_msgs()`, and ask bots to use plain events for message processing again. It's not that bad; worst case, the bot crashes and then forgets to react to some messages, but the user will just try again. And if some message makes the bot crash, then it might actually be good not to try and process it again.
- Store the pre-message text and `PostMsgMetadata` (or alternatively, the whole mime) in a new database table. Wait with processing it until the post-message arrives.

Additionally, the logic that small messages are downloaded before post-messages should be disabled for bots, in order to prevent reordering.
This commit is contained in:
Hocuri
2026-04-10 21:10:46 +02:00
committed by GitHub
parent d6971ee4ac
commit 8b58b16cb5
6 changed files with 120 additions and 10 deletions

View File

@@ -678,7 +678,7 @@ impl CommandApi {
ChatId::new(chat_id).get_fresh_msg_cnt(&ctx).await
}
/// Gets messages to be processed by the bot and returns their IDs.
/// (deprecated) Gets messages to be processed by the bot and returns their IDs.
///
/// Only messages with database ID higher than `last_msg_id` config value
/// are returned. After processing the messages, the bot should
@@ -686,6 +686,13 @@ impl CommandApi {
/// or manually updating the value to avoid getting already
/// processed messages.
///
/// Deprecated 2026-04: This returns the message's id as soon as the first part arrives,
/// even if it is not fully downloaded yet.
/// The bot needs to wait for the message to be fully downloaded.
/// Since this is usually not the desired behavior,
/// bots should instead use the #DC_EVENT_INCOMING_MSG / [`types::events::EventType::IncomingMsg`]
/// event for getting notified about new messages.
///
/// [`markseen_msgs`]: Self::markseen_msgs
async fn get_next_msgs(&self, account_id: u32) -> Result<Vec<u32>> {
let ctx = self.get_context(account_id).await?;
@@ -698,7 +705,7 @@ impl CommandApi {
Ok(msg_ids)
}
/// Waits for messages to be processed by the bot and returns their IDs.
/// (deprecated) Waits for messages to be processed by the bot and returns their IDs.
///
/// This function is similar to [`get_next_msgs`],
/// but waits for internal new message notification before returning.
@@ -709,6 +716,13 @@ impl CommandApi {
/// To shutdown the bot, stopping I/O can be used to interrupt
/// pending or next `wait_next_msgs` call.
///
/// Deprecated 2026-04: This returns the message's id as soon as the first part arrives,
/// even if it is not fully downloaded yet.
/// The bot needs to wait for the message to be fully downloaded.
/// Since this is usually not the desired behavior,
/// bots should instead use the #DC_EVENT_INCOMING_MSG / [`types::events::EventType::IncomingMsg`]
/// event for getting notified about new messages.
///
/// [`get_next_msgs`]: Self::get_next_msgs
async fn wait_next_msgs(&self, account_id: u32) -> Result<Vec<u32>> {
let ctx = self.get_context(account_id).await?;