Adds `assert_warn`, `assert_error` and `assert_warns_or_errors`
methods to `TestContext`, that let us assert that
a certain warning or error is logged during the test.
Also asserts test logs should not contain any other
errors or warnings.
Adjusts tests accordingly.
Signed-off-by: Jagoda Ślązak <jslazak@jslazak.com>
There was a bug in prefetch_should_download() that it made it return
true for pre-messages even when they were already downloaded. This meant
that pre-messages were downloaded from all relays, rather than just one,
wasting internet data.
The fix is in rfc724_mid_download_tried(), which is used by
prefetch_should_download() to determine whether a message was already
downloaded.
---------
Co-authored-by: l <link2xt@testrun.org>
Unifies internal naming of single/direct/1:1/normal chats
to match the naming used in the API.
Closes: #8368
Signed-off-by: Jagoda Ślązak <jslazak@jslazak.com>
---------
Signed-off-by: Jagoda Ślązak <jslazak@jslazak.com>
Maybe it's even good that MDNs will be sent for pre-messages only having placeholder text with the
viewtype and size, at least this way we notify the contact that we've noticed the message. Anyway,
with adding message previews the problem will be solved for the corresponding viewtypes.
Arbitrary timeouts often result in flaky tests,
especially if CI runners may be paused.
In this case however timeout was not used by any tests
and only slowed down the tests in cases where
no message is expected but non-zero timeout is set.
Currently, the resolution of a resized image that was sent in a chat,
depends on the aspect-ratio.
Assuming the `balanced`-quality-setting is used,
a square image, that is larger than the limits for resolution and file-size,
will be resized to 1280x1280 (1,638,400 pixels),
an image with an aspect-ratio of 16:9,
will be resized to 1280x720 (921,600 pixels),
and if the aspect-ratio is 32:9, to 1280x360 (460,800 pixels).
This change makes it so, that the number of pixels,
in images with different aspect-ratios, will be similar.
Webxdc updates go to pre-message and when the message has no text,
the whole pre-message is trashed if it has a webxdc update.
The solution is not to trash pre-message in this case so preview is displayed.
receive_imf should not fail in this case,
but trash the post-message and updates.
The problem with previously used message::rfc724_mid_exists check
was that rfc724_mid_exists may return a trashed message.
This change is a preparation for ignoring
unencrypted messages by default.
New test_utils::encrypt_raw_message and
test_utils::receive_encrypted_imf are
used to encrypt the messages before
"receiving" them with receive_imf.
If we missed a message changing the primary transport, we shouldn't remove it when applying
a SyncTransports message, such a state doesn't look correct even if it's temporary.
Otherwise if it's forwarded to a device that already has the original
message, the pre-message isn't received and this results in a message
having no text and `Forwarded` param.
Co-authored-by: Hocuri <hocuri@gmx.de>
rfc724_mid_exists() returns the message id not only when the message is fully downloaded, so we need
to check this additionally. Also improve test_receive_pre_message to check that receiving the
pre-message again doesn't affect the existing message.
I used some AI to draft a first version of this, and then reworked it.
This is one of multiple possibilities to fix
https://github.com/chatmail/core/issues/8041: For bots, the IncomingMsg
event is not emitted when the pre-message arrives, only when the
post-message arrives. Also, post-messages are downloaded immediately,
not after all the small messages are downloaded.
The `get_next_msgs()` API is deprecated. Instead, bots need to listen to
the IncomingMsg event in order to be notified about new events. Is this
acceptable for bots?
THE PROBLEM THAT WAS SOLVED BY THIS:
With pre-messages, it's hard for bots to wait for the message to be fully downloaded and then process it.
Up until now, bots used get_next_msgs() to query the unprocessed messages, then set last_msg_id after processing a message to that they won't process it again.
But this will now also return messages that were not fully downloaded.
ALTERNATIVES:
In the following, I will explain
the alternatives, and for why it's not so easy to just make the
`get_next_msgs()` API work. If it's not understandable, I'm happy to
elaborate more.
Core can't just completely ignore the pre-message for two reasons:
- If a post-message containing a Webxdc arrives later, and some webxdc updates arrive in the meantime, then these updates will be lost.
- The post-message doesn't contain the text (reasoning was to avoid duplicate text for people who didn't upgrade yet during the 2.43.0 rollout)
There are multiple solutions:
- Add the message as hidden in the database when the pre-message arrives.
- When the post-message arrives and we want to make it available for bots, we need to update the msg_id because of how the `get_next_msgs()` API works. This means that we need to update all webxdc updates that reference this msg_id.
- Alternatively, we could make webxdc's reference the rfc724_mid instead of the msg_id, so that we don't need stable msg_ids anymore
- Alternatively, we could deprecate `get_next_msgs()`, and ask bots to use plain events for message processing again. It's not that bad; worst case, the bot crashes and then forgets to react to some messages, but the user will just try again. And if some message makes the bot crash, then it might actually be good not to try and process it again.
- Store the pre-message text and `PostMsgMetadata` (or alternatively, the whole mime) in a new database table. Wait with processing it until the post-message arrives.
Additionally, the logic that small messages are downloaded before post-messages should be disabled for bots, in order to prevent reordering.
It is not clear now what this is testing.
Golden test shows messages ordered
incorrectly according to the timestamps,
they should be ordered the other way round.
Comment talks about fetching from mvbox and inbox
in paralell which is a rare case that
could have happened if one message is left in the inbox
and the other message is a chat message moved to mvbox.
We never download anything that is not moved to the target folder.
The test also resides in "verified chats" tests
which are all legacy tests we kept after
replacing the concept of verified/protected chats
with key contacts in 2.x.
encryption info needs a dedicated string for "Messages are end-to-end encrypted"
as the UI will add more infomation to the info messages,
smth. as "Tap for more information".
an alternative fix would have been to let the UI render the info-message
differently, but adding another string to core causes less friction.
The webxdc file name itself isn't informative for users. Still, send and display it if the webxdc
manifest can't be parsed, it's better than sending "Mini App" and this isn't a normal case anyway.
This is not possible for webxdcs and vCards currently however, so add workarounds for them:
- Use translated "Mini App" as the webxdc name.
- Use just "👤" instead of the vCard summary (i.e. the vCard contact name).
Before this PR, when a user with current main sends a large message to a
user with an old Delta Chat (before #7431), the text will be duplicated:
One message will arrive with only the text, and one message with
attachment+text.
This PR changes this - there will be one message with only the text, and
one message with only the attachment.
If we want to guard against lost pre-messages, then we can revert this
PR in a few months, though I'm not sure that's necessary - it's unlikely
that the small pre-message gets lost but the big post-message gets
through.
The motivation is to reduce code complexity, get rid of the extra IMAP connection and cases when
messages are added to chats by Inbox and Sentbox loops in parallel which leads to various message
sorting bugs, particularly to outgoing messages breaking sorting of incoming ones which are fetched
later, but may have a smaller "Date".
If we use modules (which are actually namespaces), we can use shorter names. Another approach is to
only use modules for internal code incapsulation and use full names like deltachat-ffi does.
Create unprotected group in test_create_protected_grp_multidev
The test is renamed accordingly.
SystemMessage::ChatE2ee is added in encrypted groups
regardless of whether they are protected or not.
Previously new encrypted unprotected groups
had no message saying that messages are end-to-end encrypted
at all.
This mechanism replaces `Chat-Verified` header.
New parameter `_verified=1` in `Autocrypt-Gossip`
header marks that the sender has the gossiped key
verified.
Using `_verified=1` instead of `_verified`
because it is less likely to cause troubles
with existing Autocrypt header parsers.
This is also how https://www.rfc-editor.org/rfc/rfc2045
defines parameter syntax.
If the contact is already introduced by someone,
usually by adding to a verified group,
it should not be reverified because of another
chat message is a verified group.
This usually results is verification loops
and is not meaningful because the verifier
likely got this same contact introduced
in the same group.
this PR adds a info message "messages are end-to-end-encrypted" also for
chats created by eg. vcards. by the removal of lock icons, this is a
good place to hint for that in addition; this is also what eg. whatsapp
and others are doing
the wording itself is tweaked at
https://github.com/deltachat/deltachat-android/pull/3817 (and there is
also the rough idea to make the message a little more outstanding, by
some more dedicated colors)
~~did not test in practise, if this leads to double "e2ee info messages"
on secure join, tests look good, however.~~ EDIT: did lots of practise
tests meanwhile :)
most of the changes in this PR are about test ...
ftr, in another PR, after 2.0 reeases, there could probably quite some
code cleanup wrt set-protection, protection-disabled etc.
---------
Co-authored-by: Hocuri <hocuri@gmx.de>
This change introduces a new type of contacts
identified by their public key fingerprint
rather than an e-mail address.
Encrypted chats now stay encrypted
and unencrypted chats stay unencrypted.
For example, 1:1 chats with key-contacts
are encrypted and 1:1 chats with address-contacts
are unencrypted.
Groups that have a group ID are encrypted
and can only contain key-contacts
while groups that don't have a group ID ("adhoc groups")
are unencrypted and can only contain address-contacts.
JSON-RPC API `reset_contact_encryption` is removed.
Python API `Contact.reset_encryption` is removed.
"Group tracking plugin" in legacy Python API was removed because it
relied on parsing email addresses from system messages with regexps.
Co-authored-by: Hocuri <hocuri@gmx.de>
Co-authored-by: iequidoo <dgreshilov@gmail.com>
Co-authored-by: B. Petersen <r10s@b44t.com>
If we receive a message from non-verified contact
in a non-protected chat with a Chat-Verified header,
there is no need to upgrade the chat
to verified and display an error.
If it was an attack, an attacker could
just not send the Chat-Verified header.
Most of the time, however, it is just
message reordering.
