Fix#3507
Note that this is not intended for a release at this point! We first have to test whether it runs stable enough. If we want to make a release while we are not confident enough in authres-checking, then we have to disable it.
BTW, most of the 3000 new lines are in `test_data/messages/dkimchecks...`, not the actual code
da3a4b94 adds the results to the Message info. It currently does this by adding them to `hop_info`. Maybe we should rename `hop_info` to `extra_info` or something; this has the disadvantage that we can't rename the sql column name though.
Follow-ups for this could be:
- In `update_authservid_candidates()`: Implement the rest of the algorithm @hpk42 and me thought about. What's missing is remembering how sure we are that these are the right authserv-ids. Esp., when receiving a message sent from another account at the same domain, we can be quite sure that the authserv-ids in there are the ones of our email server. This will make authres-checking work with buzon.uy, disroot.org, yandex.ru, mailo.com, and riseup.net.
- Think about how we present this to the user - e.g. currently the only change is that we don't accept key changes, which will mean that the small lock on the message is not shown.
- And it will mean that we can fully enable AEAP, after revisiting the security implications of this, and assuming everyone (esp. @link2xt who pointed out the problems in the first place) feels comfortable with it.
* truncate incoming messages by lines,
because many linebreaks seem to cause the chat open delay on deltachat-ios
* run cargo fmt
* remove DC_DESIRED_TEXT_LINES_THRESHOLD
and use Strings instead of Cow<str>
* remove usage of clippy::indexing_slicing in truncate_by_lines (#3596)
* adjust comments
* Fix truncate_by_lines tests
* Reword indexing/slicing error
* Remove unnecessary conditional
* Fix a typo in the comment
Co-authored-by: link2xt <link2xt@testrun.org>
* force a reason when calling `set_msg_failed()`
the string is displayed to the user,
so even _some_ context as "NDN without further details"
is better than an empty string.
* make clippy happy
* add CHANGELOG entry
* clarify webxdc reference wrt info-messages
* add from_id parameter to add_info_msg_with_cmd()
* flag webxdc-info-messages as such
* set from_id to sender for webxdc-info-messages
* test additional webxdc info properties
* do not add series of similar info messages
instead, if on adding the last info message
is already from the same webxdc and sender,
just update the text
* test cleanup of webxdc info messages series
* update changelog
* make clippy happy
there is no real complexity in the args,
so allowing one more arg is probably fine.
if really wanted, we can refactor the function in another pr;
this pr is already complex enough :)
* use cleaner function names and comments
* clarify CHANGELOG
mimeparser now handles try_decrypt() errors instead of simply logging
them. If try_decrypt() returns an error, a single message bubble
with an error is added to the chat.
The case when encrypted part is found in a non-standard MIME structure
is not treated as an encryption failure anymore. Instead, encrypted
MIME part is presented as a file to the user, so they can download the
part and decrypt it manually.
Because try_decrypt() errors are handled by mimeparser now,
try_decrypt() was fixed to avoid trying to load private_keyring if the
message is not encrypted. In tests the context receiving message
usually does not have self address configured, so loading private
keyring via Keyring::new_self() fails together with the try_decrypt().
This change is aimed at decoupling parsing and
add_parts() stages to eventually separate parsing
from database changes and pipeline message parsing and
decryption.
This seems not only wasteful but genuinly has the risk someone makes
their device useless by accidentally adding a huge file.
This also re-structures the checks a little: The if-conditions are
flattened out and cheap checks are done before more expensive ones.
This makes the contact ID its own newtype instead of being a plain
u32. The change purposefully does not yet try and reap any benefits
from this yet, instead aiming for a boring change that's easy to
review. Only exception is the ToSql/FromSql as not doing that yet
would also have created churn in the database code and it is easier to
go straight for the right solution here.
Message-IDs are now retrieved only during fetching and saved into imap
table. dc_receive_imf_inner does not attempt to extract the Message-ID
anymore.
For messages without Message-ID the ID is now generated in
imap::fetch_new_messages rather than dc_receive_imf_inner,
so the same ID is used in the imap table (maintained by the imap
module) and msgs table (maintained by dc_receive_imf module).
Message-ID generation based on the Date, From and To field hashing has
been replaced with a simple dc_create_id() to avoid retrieving Date,
From, and To fields in the imap module, as it's hard to test that it
stays compatible between Delta Chat versions in this module. This
breaks jump-to-quote for quoted messages without Message-ID, which is
not critical.
Also prefetch X-Microsoft-Original-Message-ID, so retrieval of
duplicate messages with X-Microsoft-Original-Message-ID can be skipped
like it is done for messages with Message-ID header.
`imap` table maps Message-IDs to UIDs on the server. `dc_receive_imf`
no longer gets the UID of the message as an argument and does not
insert the folder and UID of the message into the `msgs`
table. `server_folder` and `server_uid` columns in `msgs` table are
deprecated.
MoveMsg and DeleteMsgOnImap jobs are removed. Now messages are moved
and deleted only in the `fetch_move_delete` procedure that consults
the `target` column of the `imap` table to determine where the message
should go.
Where the message should go is determined after prefetching by the
`imap::target_folder()` procedure. Messages are only downloaded once
they reach their target folder to avoid race conditions in multidevice
setting, such as:
1. One device trying to FETCH the message while the other tries to
MOVE it.
2. One device marking the message as \Seen in the Inbox while the
other has already copied unseen message to the Movebox and is going to
delete the \Seen message in the Inbox.
3. Device downloads the message from the Inbox while there are newer
messages in the Movebox placed there by the other device, thus
processing the messages out of order.
They contain X-MSExch-Correlation-Key header, but no
Original-Message-ID, so they cannot be used to find the original
message, but we want to recognize them as MDN nevertheless to assign
them to the trash chat.
Update gossiped_timestamp when someone else sends autocrypt gossip in
the group, so we postpone sending gossip again ourselves.
- Warn about failures to parse Autocrypt-Gossip header
- Move gossip-related methods into ChatId impl
- Fix a "gossi_pp_ed" typo
* add basic multi-device-sync functions
* generate json
* add context.parse_sync_items()
* add context.execute_sync_items()
* piggyback sync-commands message, add body for human-readable part
* avoid double json renderings
* mimeparser parses incoming .json sync-files
* do not piggyback sync-files
* execute sync items
* return status of send_sync_msg()
* send sync messages as multipart/report
* add a per-item-timestamp and also allow adding other per-item-fields in the future
* if the self-chat does not exist, create it blocked/hidden
* create tokens closer to real qr-code needs
* respect bcc_self setting, add test for that
* sync qr code tokens after promoting groups
* send sync-messages only if an experimental switch is set
* trigger send_sync_msg() after sending messages and after creating/redraw/revive qr-code
* add DC_STR_* constants to deltachat.h
* adapt to refactored qr module as of #2729
* tweak test
* use SendSyncMsgs config name instead of SendExperimentalSyncMsgs - we can remove or rename the config nevertheless, but have the option to keep it without renaming
* tweak docs
* remove currently unused effective timestamp calculation
* clarify when send_sync_msg() is called
* make sure, sync-messages are encrypted and are sent by SELF
* tweak docs, fix typos
* draft a download-api
* basic implementation
* allow partial downloads for protected chats
* use a separate column for download_state
* force a minimal timeout for delete_server_after in combination with partial messages
* add a warning if a possible download may expire by delete_server_after
* test load_imap_deletion_msgid()
* add a test for a partial download
* improve documentation and visibility
* let get_download_limit() return Result<Option>
* rusty getters
* apply MIN_DELETE_SERVER_AFTER to shown availability time
* move stub-creation to download.rs, use stock-strings, nicer logging
* make clippy happy (cargo clippy --tests)
* refine tests and comments
* fix typo
* remove superfluous closure in ffi
* respect partial_download for immediately scheduled DeleteMsgOnImap jobs
