Before, only replies to chat messages were moved to the mvbox because we're removing Chat-Version
from outer headers, but there's no much sense in moving only replies and not moving original
messages and MDNs. Instead, move all encrypted messages. Users should be informed about this in UIs,
so if a user has another PGP-capable MUA, probably they should disable MvboxMove. Moreover, untying
this logic from References and In-Reply-To allows to remove them from outer headers too, the "Header
Protection for Cryptographically Protected Email" RFC even suggests such a behavior:
https://datatracker.ietf.org/doc/html/rfc9788#name-offering-more-ambitious-hea.
Instead, prefetch Secure-Join, Content-Type and Subject headers, try to find out if the message is encrypted, i.e.:
- if its Content-Type is "multipart/encrypted"
- or Subject is "..." or "[...]" as some MUAs use "multipart/mixed"; we can't only look at Subject
as it's not mandatory;
and depending on this decide on the target folder and whether the message should be
downloaded. There's no much sense in downloading unencrypted "Chat-Version"-containing messages if
`ShowEmails` is `Off` or `AcceptedContacts`, unencrypted Delta Chat messages should be considered as
usual emails, there's even the "New E-Mail" feature in UIs nowadays which sends such messages.
Don't prefetch Auto-Submitted as well, this becomes unnecessary.
Changed behavior: before, "Chat-Version"-containing messages were moved from INBOX to DeltaChat, now
such encrypted messages may remain in INBOX -- if there's no parent message or it's not
`MessengerMessage`. Don't unconditionally move encrypted messages yet because the account may be
shared with other software which doesn't and shouldn't look into the DeltaChat folder.
For multiple transports we will need to run
multiple IMAP clients in parallel.
UID validity change detected by one IMAP client
should not result in UID resync
for another IMAP client.
`login_param` module is now for user-visible entered login parameters,
while the `transport` module contains structures for internal
representation of connection candidate list
created during transport configuration.
It's used in `fetch_existing_msgs()`, but we can remove it and tell users that they need to
move/copy messages from Sentbox to Inbox so that Delta Chat adds all contacts from them. This way
users will be also informed that Delta Chat needs users to CC/BCC/To themselves to see messages sent
from other MUAs.
The motivation is to reduce code complexity, get rid of the extra IMAP connection and cases when
messages are added to chats by Inbox and Sentbox loops in parallel which leads to various message
sorting bugs, particularly to outgoing messages breaking sorting of incoming ones which are fetched
later, but may have a smaller "Date".
We do not try to delete resent messages
anymore. Previously resent messages
were distinguised by having duplicate Message-ID,
but future Date, but now we need to download
the message before we even see the Date.
We now move the message to the destination folder
but do not fetch it.
It may not be a good idea to delete
the duplicate in multi-device setups anyway,
because the device which has a message
may delete the duplicate of a message
the other device missed.
To avoid triggering IMAP busy move loop
described in the comments
we now only move the messages
from INBOX and Spam folders.
I have logs from a user where messages are prefetched for long minutes, and while it's not a problem
on its own, we can't rely that the connection overlives such a period, so make
`fetch_new_messages()` prefetch (and then actually download) messages in batches of 500 messages.
Actually this leads to fetching messages only from watched folders and Spam. Motivation:
- At least Gmail has virtual folders which aren't correctly detected as such, e.g. "Sent".
- At least Gmail has many virtual folders and scanning all of them takes significant time, 5-6 secs
in median for me. This slows down receiving new messages and consumes battery.
- Delta Chat shouldn't fetch messages from folders potentially created by other apps for their own
purposes. NB: All compatible Delta Chat forks should use the "DeltaChat" folder as mvbox.
- Fetching from folders that aren't watched, e.g. from "Sent", may lead to message ordering issues.
Socket may lose peer address when it is disconnected from
the server side. In this case debug_assert! failed
for me when running the core in debug mode on desktop.
To avoid the case of peer_addr not being available,
we now store it when LoggingStream is created.
fetch_existing option is not enabled in existing clients
and does not work with encrypted messages
without importing the key into a newely created account.
This task is not guaranteed to be cancellation-safe
and provides a stop token for safe cancellation instead.
We should always cancel the task properly
and not by racing against another future.
Otherwise following call to Handle.done()
may work on IMAP session that is in the middle
of response, for example.
Add a `create` param to `select_with_uidvalidity()` instead of always trying to create the folder
and return `Ok(false)` from it if the folder doesn't exist and shouldn't be created, and handle this
in `store_seen_flags_on_imap()` by just removing "jobs" from the `imap_markseen` table. Also don't
create the folder in other code paths where it's not necessary.
This may indicate that there was a new \Seen flag
that we don't want to skip.
Also don't drain unsolicited responses while scanning folders. Now we
only drain unsolicited responses right before IDLE and always redo the
whole fetch cycle if there have been some. Some message in the scanned
folder may not be fetched that would be previously fetched otherwise,
but it will be picked up on the next folder scan.
This change introduces new config options
`proxy_enabled` and `proxy_url`
that replace `socks5_*`.
Tested with deltachat-repl
by starting it with
`cargo run --locked -p deltachat-repl -- deltachat-db` and running
```
> set proxy_enabled 1
> set proxy_url ss://...
> setqr dcaccount:https://chatmail.example.org/new
> configure
```
Previously for each connection candidate (essentially host and port
pair) after resolving the host to a list of IPs Delta Chat iterated IP
addresses one by one. Now for IMAP and SMTP we try up to 5 IP addresses
in parallel. We start with one connection and add more connections
later. If some connection fails, e.g. we try to connect to IPv6 on IPv4
network and get "Network is unreachable" (ENETUNREACH) error, we replace
failed connection with another one immediately.
Co-authored-by: Hocuri <hocuri@gmx.de>
Apparently some providers fail TLS connection
with "no_application_protocol" alert
even when requesting "imap" protocol for IMAP connection
and "smtp" protocol for SMTP connection.
Fixes <https://github.com/deltachat/deltachat-core-rust/issues/5892>.
Recently there are many questions on the Delta Chat forum why some unexpected encrypted messages
appear in Inbox. Seems they are mainly sync messages, though that also obviously happens to
SecureJoin messages. Anyway, regardless of the `MvboxMove` setting, auto-generated outgoing messages
should be moved to the DeltaChat folder so as not to complicate co-using Delta Chat with other MUAs.
Previously Delta Chat tried all DNS resolution results
in sequence until TCP connection is established successfully,
then tried to establish TLS on top of the TCP connection.
If establishing TLS fails, the whole
connection establishment procedure failed
without trying next DNS resolution results.
In particular, in a scenario
where DNS returns incorrect result
pointing e.g. to a server
that listens on the TCP port
but does not have correpsponding TLS certificate,
Delta Chat now will fall back to the cached result
and connect successfully.
This fixes things for Gmail f.e. Before, `Imap::fetch_move_delete()` was called before looking for
Trash and returned an error because of that failing the whole `fetch_idle()` which prevented
configuring Trash in turn.
