Commit Graph

5298 Commits

Author SHA1 Message Date
Simon Laux
060c9c8aa1 cargo fmt 2024-01-31 14:04:03 +01:00
Simon Laux
727428a965 rename event and mention event in method documentation 2024-01-31 14:04:03 +01:00
Simon Laux
df455bbcf5 BackgroundFetchCompletedForAllAccounts event 2024-01-31 14:04:03 +01:00
Simon Laux
946eea4c9e add rate limit for quota check in background fetch (12h for now) 2024-01-31 14:04:03 +01:00
Simon Laux
5cdd5e0564 Apply suggestions from code review
Co-authored-by: link2xt <link2xt@testrun.org>
2024-01-31 14:04:03 +01:00
Simon Laux
8e073b9c3e log time that the function took 2024-01-31 14:04:03 +01:00
Simon Laux
ea2a692d18 fix iOS build issue 2024-01-31 14:04:03 +01:00
Simon Laux
7b68098785 feat: add background fetch method 2024-01-31 14:04:03 +01:00
Simon Laux
48f2ea717e refactor: move convert folder meaning logic in own method
also unify the error handling for the cases where it can go wrong.
2024-01-31 14:04:03 +01:00
link2xt
cb3f03fd39 feat: add support for IMAP METADATA 2024-01-31 04:16:04 +00:00
iequidoo
06f1fe18d6 fix: Delete resent messages on receiver side (#5155)
If a Delta Chat message has the Message-ID already existing in the db, but a greater "Date", it's a
resent message that can be deleted. Messages having the same "Date" mustn't be deleted because they
can be already seen messages moved back to INBOX. Also don't delete messages having lesser "Date" to
avoid deleting both messages in a multi-device setting.
2024-01-30 22:01:45 -03:00
iequidoo
1dbf924c6a feat: chat::resend_msgs: Guarantee strictly increasing time in the Date header
Use `create_smeared_timestamp()` for this. This allows to dedup messages on the receiver -- if it
sees the same Message-ID, but a different timestamp, then it's a resent message that can be deleted.
2024-01-30 22:01:45 -03:00
iequidoo
672993e69e feat: qr::check_qr(): Accept i.delta.chat invite links (#5217)
Accepts invite links with `https://i.delta.chat[/]#FINGERPRINT&a=ADDR[&OPTIONAL_PARAMS]`
scheme. Only `i.delta.chat` domain is supported now not to intersect with the common HTTPS scheme.
2024-01-28 20:33:56 -03:00
iequidoo
7cf382a3b8 fix: Treat only "Auto-Submitted: auto-generated" messages as bot-sent (#5213)
"Auto-Submitted: auto-replied" messages mustn't be considered as sent by either bots or non-bots,
e.g. MDNs have this header value and it's the same for bots and non-bots.
2024-01-26 13:03:19 -03:00
Simon Laux
0afc0dd65a fix: add tolerance for macOS and iOS changing # to %23
fixes #1969

Bug description:
macOS and iOS sometimes replace the # with %23 (uri encode it), we should be able to parse this wrong format too, see issue https://github.com/deltachat/deltachat-core-rust/issues/1969 for more info
2024-01-24 23:58:49 +01:00
link2xt
73d612a07d feat: do not drop unknown report attachments
In particular TLSRPT reports
contain files that may be interesting for admins.
Currently Delta Chat drops the attachment
so message appears as a text message without actual payload.
2024-01-24 16:13:20 +00:00
Sebastian Klähn
c5f31c3d03 fix: No new chats for MDNs with alias (#5196) (#5199)
close #5196
2024-01-22 16:51:37 +01:00
Sebastian Klähn
4bc90701cc feat: Add system message when provider does not allow unencrypted messages (#5161) (#5195)
close #5161

![Screenshot from 2024-01-19
19-56-09](https://github.com/deltachat/deltachat-core-rust/assets/39526136/27ecdd9b-1739-410b-bb26-80d5bdbbc39a)

---------

Co-authored-by: bjoern <r10s@b44t.com>
2024-01-20 11:47:23 +00:00
link2xt
2faf7fdb78 fix: BCC-to-self even if server deletion is set to "at once" 2024-01-18 10:20:01 +00:00
link2xt
6a8ea8a083 fix: set message download state to Failure on IMAP errors
Previously the message was removed from `download` table,
but message bubble was stuck in InProgress state.

Now download state is updated by the caller,
so it cannot be accidentally skipped.
2024-01-18 10:18:57 +00:00
iequidoo
589236c27b fix: chat::send_msg: Remove encryption-related params from already sent message
This allows to send existing messages (incoming and outgoing) taken from encrypted chats, to
unencrypted ones. `Param::ForcePlaintext` is removed as well -- if a message can be sent encrypted
this time, nothing bad with this.
2024-01-17 14:20:29 +01:00
iequidoo
06cccb77f8 feat: Use Quoted-Printable for the text part (#3986)
This is needed to protect from ESPs (such as gmx.at) doing their own Quoted-Printable encoding and
thus breaking messages and signatures. It's unlikely that the reader uses a MUA not supporting
Quoted-Printable encoding. And RFC 2646 "4.6" also recommends it for encrypted messages.
2024-01-16 23:46:24 -03:00
link2xt
849a873e61 feat: only try to configure non-strict TLS checks if explicitly set
Trying non-strict TLS checks is not necessary
for most servers with proper TLS setup,
but doubles the time needed to fail configuration
when the server is not responding, e.g.
when all connection attempts time out.

There is also a risk of accidentally
configuring non-strict TLS checks in a rare case
that strict TLS check configuration spuriously failed,
e.g. on a bad network.

If the server has a known broken TLS setup,
it can still be added to the provider database
or configured with non-strict TLS check manually.
User can also configure another email provider,
such as chatmail servers, instead of using the server
with invalid TLS hostname.

This change does not affect exising setups.
2024-01-15 22:54:31 +00:00
Sebastian Klähn
382fc75b1e Add more docs (#5174)
Add some docs to smtp functions
2024-01-12 11:14:05 +01:00
Sebastian Klähn
92fc9ea971 feat: Encrypt MDNs #5168 (#5175)
This PR stops MDNs from being forced to be sent unencrypted. 
If no encryption is possible (by `should_encrypt`), the fix #5152 still
applies.

close #5168
2024-01-12 10:54:54 +01:00
Sebastian Klähn
de7ac2a240 fix: emit events more reliable when starting and stopping io #5097 (#5101)
Send `EventType::ConnectivityChanged` when using the context methods
`start_io` and `stop_io`.

close #5097

---------

Co-authored-by: Septias <scoreplayer2000@gmail.comclear>
2024-01-12 09:45:34 +01:00
iequidoo
d5da2bed75 feat: Add ConfigSynced event
Add an event for a case if a multi-device synced config value changed. Maybe the app needs to
refresh smth on such an event. For uniformity it is emitted on the source device too. The value is
omitted, otherwise it would be logged which might not be good for privacy.
2024-01-11 21:53:26 -03:00
iequidoo
924d5b9377 feat: Sync contact creation/rename across devices (#5163)
Use `chat::SyncAction::Rename` for that. Anyway 1:1-s can't be renamed and a separate sync action
would only complicate the code.
2024-01-10 16:46:54 -03:00
iequidoo
bb47299ee4 fix: contact::set_blocked(): Don't fail on sync errors, just log them
Multi-device synchronisation is not critical and should not fail the local operation, in other
places sync errors are already ignored.
2024-01-10 16:46:54 -03:00
link2xt
20065d3daa docs: add a NOTE comment about KeyId backward verification race 2024-01-09 21:46:37 +00:00
link2xt
ccb267beab refactor: rename notify_peer_verified() into set_peer_verified()
It was named notify_peer_verified()
because it added info message,
but this is no longer true since
https://github.com/deltachat/deltachat-core-rust/pull/4998
(commit c6ea4e389a)
is merged.
2024-01-09 21:46:37 +00:00
link2xt
32bcb59601 refactor: do not emit ChatModified event in notify_peer_verified()
The chat is not modified at least since
c6ea4e389a
(PR https://github.com/deltachat/deltachat-core-rust/pull/4998),
even the info message is not posted there.
2024-01-09 21:46:37 +00:00
link2xt
c708c44f0a feat: mark 1:1 chat as verified for Bob early
Mark 1:1 chat as verified as soon as Alice is forward-verified
so Bob can already start sending Chat-Verified headers.
This way Alice and Bob can scan each other's QR codes
and even if all Secure-Join headers are dropped from the network,
still get forward verifications via QR-code scans
and backward verifications via Chat-Verified messages in 1:1 chat.
2024-01-09 21:46:37 +00:00
link2xt
9415a71f9d refactor: rename fingerprint_equals_sender to verify_sender_by_fingerprint 2024-01-09 21:46:37 +00:00
link2xt
1fd42f2c53 test: test recovery from lost vc-contact-confirm 2024-01-09 21:46:37 +00:00
link2xt
1e52502ab3 refactor: send Secure-Join-Fingerprint only in *-request-with-auth 2024-01-09 21:46:37 +00:00
link2xt
e855b79f9c feat: add backward_verified_key_id column to acpeerstates 2024-01-09 21:46:37 +00:00
link2xt
2f8a8f9f50 ci: update to Rust 1.75.0 and fix clippy 2024-01-08 20:01:40 +00:00
link2xt
b9a58bf625 docs: add a link to autoconfig RFC draft
This will hopefully replace deleted Mozilla documentation page
in the future.
2024-01-07 22:55:16 +00:00
iequidoo
c8075e53d2 fix: Reset message error when scheduling resending (#5119)
Before, while a message is in OutPending state after resending is requested, the user still sees the
red marker with error and it is confusing, so the user don't know the sending state of the message.
2024-01-07 15:02:31 -03:00
iequidoo
ff54cf24a1 fix: message::update_msg_state(): Reset error if message is delivered (#5119) 2024-01-05 20:43:55 -03:00
link2xt
da11542322 fix: do not remove contents from Schleuder ML messages
Before this fix actual contents of the message
reposted by Schleuder is considered a mailing list footer and removed,
not visible even in the "Show Full Message..." view.

With this change there will be two message bubbles,
one for header and one for the contents,
but it is still better than losing the contents completely.

Attempting to parse header part is out of scope for this change.
2024-01-05 15:42:56 +00:00
link2xt
4dc596e646 fix(mimefactory): do not wrap MDNs into multipart/mixed part 2024-01-05 15:34:48 +00:00
iequidoo
625887d249 fix: Split SMTP jobs already in chat::create_send_msg_jobs() (#5115)
a27e84ad89 "fix: Delete received outgoing messages from SMTP queue"
can break sending messages sent as several SMTP messages because they have a lot of recipients:
`pub(crate) const DEFAULT_MAX_SMTP_RCPT_TO: usize = 50;`

We should not cancel sending if it is such a message and we received BCC-self because it does not
mean the other part was sent successfully. For this, split such messages into separate jobs in the
`smtp` table so that only a job containing BCC-self is canceled from `receive_imf_inner()`. Although
this doesn't solve the initial problem with timed-out SMTP requests for such messages completely,
this enables fine-grained SMTP retries so we don't need to resend all SMTP messages if only some of
them failed to be sent.
2024-01-05 01:53:41 -03:00
link2xt
dcf6ffef12 fix(imap): fail fast on LIST errors
async-imap returns infinite stream of errors
in case of EOF or timeout on the input stream,
so attempting to skip and log errors results in busy loop
similar to this:

   2023-12-22T13:07:35.751Z core/event WARNING "" 2 "/__w/deltachat-core-rust/deltachat-core-rust/src/imap/scan_folders.rs:112: list_folders() can't get folder: io: timed out: timed out"
   2023-12-22T13:07:35.751Z core/event WARNING "" 2 "/__w/deltachat-core-rust/deltachat-core-rust/src/imap/scan_folders.rs:112: list_folders() can't get folder: io: timed out: timed out"
   2023-12-22T13:07:35.751Z core/event WARNING "" 2 "/__w/deltachat-core-rust/deltachat-core-rust/src/imap/scan_folders.rs:112: list_folders() can't get folder: io: timed out: timed out"
   2023-12-22T13:07:35.751Z core/event WARNING "" 2 "/__w/deltachat-core-rust/deltachat-core-rust/src/imap/scan_folders.rs:112: list_folders() can't get folder: io: timed out: timed out"
   2023-12-22T13:07:35.752Z core/event WARNING "" 2 "/__w/deltachat-core-rust/deltachat-core-rust/src/imap/scan_folders.rs:112: list_folders() can't get folder: io: timed out: timed out"
   2023-12-22T13:07:35.752Z core/event WARNING "" 2 "/__w/deltachat-core-rust/deltachat-core-rust/src/imap/scan_folders.rs:112: list_folders() can't get folder: io: timed out: timed out"
   2023-12-22T13:07:35.752Z core/event WARNING "" 2 "/__w/deltachat-core-rust/deltachat-core-rust/src/imap/scan_folders.rs:112: list_folders() can't get folder: io: timed out: timed out"
   2023-12-22T13:07:35.752Z core/event WARNING "" 2 "/__w/deltachat-core-rust/deltachat-core-rust/src/imap/scan_folders.rs:112: list_folders() can't get folder: io: timed out: timed out"
   2023-12-22T13:07:35.753Z core/event WARNING "" 2 "/__w/deltachat-core-rust/deltachat-core-rust/src/imap/scan_folders.rs:112: list_folders() can't get folder: io: timed out: timed out"
   2023-12-22T13:07:35.754Z core/event WARNING "" 2 "/__w/deltachat-core-rust/deltachat-core-rust/src/imap/scan_folders.rs:112: list_folders() can't get folder: io: timed out: timed out"
   2023-12-22T13:07:35.754Z core/event WARNING "" 2 "/__w/deltachat-core-rust/deltachat-core-rust/src/imap/scan_folders.rs:112: list_folders() can't get folder: io: timed out: timed out"

To avoid busy loop, fail fast on first error
and bubble it up instead of trying to recover.
2023-12-28 15:20:15 +00:00
iequidoo
865ede39fe fix: Properly escape target in receive_imf_inner()
The bug was made in 44227d7b86. Sql::execute() with placeholders must
be used to escape strings, one never should escape them manually as strings themselves can contain
escape symbols. Thanks to @link2xt for noticing.
2023-12-22 17:42:25 -03:00
iequidoo
a27e84ad89 fix: Delete received outgoing messages from SMTP queue (#5115)
Some SMTP servers are running slow before-queue filters, most commonly Postfix with `rspamd` filter
which is implemented as a [before-queue Milter](https://www.postfix.org/MILTER_README.html). Some of
`rspamd` plugin filters are slow on large mails.

We previously had problems with timing out during waiting for SMTP response:
https://github.com/deltachat/deltachat-core-rust/issues/1383. This is largely fixed by
https://github.com/async-email/async-smtp/pull/29 and currently we have 60-second timeout just for
reading a response but apparently it is not sufficient -- maybe connection gets killed by NAT while
we are waiting for response or `rspamd` takes more than 60 seconds for large messages.

As a result a message is resent multiple times and eventually fails with "too many retries" while
multiple BCC-self messages are received.

We should remove the message from the SMTP queue as soon as we receive it via IMAP as it is clear
the message was sent even if we did not manage to get actual SMTP server response.
2023-12-21 16:57:23 -03:00
iequidoo
b83bd26325 refactor: Don't pass seen=true to add_parts() when replacing existing message
First of all, it's just downloaded and hasn't been seen yet by the user. Also this changes nothing
as `msgs.state` isn't changed when replacing a message anyway.
2023-12-21 16:14:52 -03:00
iequidoo
44227d7b86 fix: Put Message-ID into hidden headers and take it from there on receiver (#4798)
Put a copy of Message-ID into hidden headers and prefer it over the one in the IMF header section
that servers mess up with.

This also reverts "Set X-Microsoft-Original-Message-ID on outgoing emails for amazonaws (#3077)".
2023-12-21 16:14:52 -03:00
iequidoo
6bcf022523 refactor: receive_imf_inner: Rename replace_partial_download to replace_msg_id
It's more consistent with `replace_chat_id` and the same as the corresponding parameter name of
`add_parts()`.
2023-12-21 16:14:52 -03:00