Commit Graph

4707 Commits

Author SHA1 Message Date
link2xt
4efd0d1ef7 test: always gossip if gossip_period is set to 0
This fixes flakiness of `test_verified_group_vs_delete_server_after`.
2024-10-15 22:55:33 +00:00
link2xt
351f28361d docs: set_protection_for_timestamp_sort does not send messages
It only adds info messages.
2024-10-15 09:14:23 +00:00
link2xt
c5b78741d6 refactor: fix clippy::needless_lifetimes warnings 2024-10-15 09:14:23 +00:00
link2xt
57871bbaf8 refactor(set_protection_for_timestamp_sort): do not log bubbled up errors
Otherwise error may be logged twice.
2024-10-15 09:14:23 +00:00
link2xt
287256693c refactor: fix elided_named_lifetimes warning in beta Rust 2024-10-15 09:14:23 +00:00
iequidoo
d660f55a99 feat: Sort received outgoing message down if it's fresher than all non fresh messages
Received messages shouldn't mingle with just sent ones and appear somewhere in the middle of the
chat, so we go after the newest non fresh message.

But if a received outgoing message is older than some `InSeen` message, better sort the received
message purely by timestamp (this is an heuristic in order not to break the Gmail-like case
simulated by `verified_chats::test_old_message_4()`). We could place the received message just
before that `InSeen` message, but anyway the user may not notice it.

At least this fixes outgoing messages sorting for shared accounts where messages from other devices
should be sorted the same way as incoming ones.
2024-10-14 21:22:11 -03:00
link2xt
f1ca689f99 feat: IMAP COMPRESS support 2024-10-14 14:01:22 +00:00
iequidoo
796b0d7752 refactor: update_msg_state: Don't avoid downgrading OutMdnRcvd to OutDelivered
`OutMdnRcvd` is a "virtual" message state now, only old messages can have this state in the db, so
`update_msg_state()` can be simplified.
2024-10-14 10:24:01 -03:00
iequidoo
3c03370589 fix: Readd tokens.foreign_id column (#6038)
Otherwise backups exported from the current core and imported in versions < 1.144.0 have QR codes
not working. The breaking change which removed the column is
5a6efdff44.
2024-10-13 15:34:15 -03:00
iequidoo
8f41aed917 fix: Assume file extensions are 32 chars max and don't contain whitespace (#5338)
Before file extensions were also limited to 32 chars, but extra chars in the beginning were just cut
off, e.g. "file.with_lots_of_characters_behind_point_and_double_ending.tar.gz" was considered to
have an extension "d_point_and_double_ending.tar.gz". Better to take only "tar.gz" then.

Also don't include whitespace-containing parts in extensions. File extensions generally don't
contain whitespaces.
2024-10-11 11:04:41 -03:00
link2xt
19be12a25d chore(cargo): upgrade async_zip to 0.0.17 (#6035) 2024-10-11 00:17:40 +00:00
link2xt
6a121b87eb fix: do not emit progress 1000 when configuration is cancelled
There is already code below that emits
progress 0 or 1000 depending on whether
configuration succeeded or failed.

Before this change cancelling resulted
in progress 0 emitted,
immediately followed by progress 1000.
2024-10-10 10:34:58 +00:00
iequidoo
8acf391ffe refactor: MsgId::update_download_state: Don't fail if the message doesn't exist anymore
If a race happens and the message disappears, there's just nothing to do and no sense to
fail. Follow-up to 22e5bf8571.
2024-10-08 12:31:41 -03:00
iequidoo
aacea2de25 fix: Reset quota on configured address change (#5908) 2024-10-07 18:04:53 -03:00
link2xt
2cb8b53256 fix: emit progress 0 if get_backup() fails 2024-10-05 17:58:24 +00:00
link2xt
a592a470cf fix: make backup reception cancellable by stopping ongoing process
This is already documented in JSON-RPC API,
but in fact ongoing process was not allocated.
2024-10-05 17:58:24 +00:00
link2xt
c4d07ab99e fix: smooth progress bar for backup transfer
Before this change progress bar only started
when database is already transferred.
Database is usually the largest file
in the whole transfer, so the transfer appears
to be stuck for the sender.

With this change progress bar
starts for backup export
as soon as connection is received
and counts bytes transferred over the connection
using AsyncWrite wrapper.

Similarly for backup import,
AsyncRead wrapper counts the bytes
received and emits progress events.
2024-10-05 17:58:24 +00:00
link2xt
eddd5a0d25 fix: make it possible to cancel ongoing backup transfer 2024-10-05 17:58:24 +00:00
link2xt
0f43d5d8f4 fix: break out of accept() loop if there is an error transferring backup 2024-10-05 17:58:24 +00:00
link2xt
283a1f1653 fix: skip unconfigured folders in background_fetch()
Otherwise `background_fetch()` fails on unconfigured Mvbox,
which is typical for chatmail accounts,
and does not get to checking QUOTA ever.
2024-10-04 21:54:42 +00:00
link2xt
d33909a054 feat: reuse existing connections in background_fetch() if I/O is started 2024-10-04 21:54:42 +00:00
link2xt
8a88479d8f fix(query_row_optional): do not treat rows with NULL as missing rows
Instead of treating NULL type error
as absence of the row,
handle NULL values with SQL.
Previously we sometimes
accidentally treated a single column
being NULL as the lack of the whole row.
2024-10-04 14:43:06 +00:00
Hocuri
5711f2fe3a feat: More context for the "Cannot establish guaranteed..." info message (#6022)
The "Cannot establish guaranteed end-to-end encryption with ..." info
message can have lots of causes, and it happened twice to us now that it
took us some time to figure out which one it is.

So, include some more detail in the info message by simply adding the
non-translated error message in parantheses.

If we want to put in some more effort for nicer error messages, we
could:
- Introduce one new translated string "Cannot establish guaranteed
end-to-end encryption with …. Cause: %2$s" or similar (and remove the
old stock string)
- And/Or: Introduce new translated strings for all the possible errors
- And/Or: Maybe reword it in order to account better for the case that
the chat already is marked as g-e2ee, or use a different wording
(because if the chat is marked as g-e2ee then it might be nice to notify
the user that something may have gone wrong, but it's still working,
just that maybe the other side doesn't have us verified now)


![Screenshot_20241003-222245](https://github.com/user-attachments/assets/c064c82e-01ac-4bac-ab11-3c9ac9db5298)
2024-10-04 13:51:06 +02:00
link2xt
46922d4d9d fix: do not attempt to reference info messages
Info messages are added
at the beginning of unpromoted group chats
("Others will only see this group after you sent a first message."),
may be created by WebXDC etc.

They are not sent outside
and have local Message-ID that
is not known to other recipients
so they should be skipped when constructing
In-Reply-To and References.
2024-10-03 21:49:58 +00:00
link2xt
75fe4e106a api!: remove deprecated get_next_media() APIs 2024-10-03 21:04:03 +00:00
iequidoo
7c60ac863e feat: MsgId::get_info(): Report original filename as well 2024-10-03 15:49:03 -03:00
link2xt
22e5bf8571 fix(download_msg): do not fail if the message does not exist anymore
Without this fix IMAP loop may get stuck
trying to download non-existing message over and over
like this:
```
src/imap.rs:372: Logging into IMAP server with LOGIN.
src/imap.rs:388: Successfully logged into IMAP server
src/scheduler.rs:361: Failed to download message Msg#3467: Message Msg#3467 does not exist.
src/scheduler.rs:418: Failed fetch_idle: Failed to download messages: Message Msg#3467 does not exist
```

The whole download operation fails
due to attempt to set the state of non-existing message
to "failed". Now download of the message
will "succeed" if the message does not exist
and we don't try to set its state.
2024-10-03 17:13:53 +00:00
link2xt
c8ba516e83 refactor(decode_ideltachat): construct error message lazily 2024-10-03 15:39:27 +00:00
WofWca
9c931c22cc refactor: better log message for failed QR scan
It did not interpolate the `{prefix}`,
it just printed it in plain text.
2024-09-30 17:34:54 -03:00
link2xt
78a0d7501b feat: use Rustls instead of native TLS for HTTPS requests
HTTPS requests are used to fetch
remote images in HTML emails,
to fetch autoconfig XML,
to POST requests for `DCACCOUNT:` QR codes
to make OAuth 2 API requests
and to connect to HTTPS proxies.

Rustls is more aggressive than OpenSSL
in deprecating cryptographic algorithms
so we cannot use it for IMAP and SMTP
to avoid breaking compatibility,
but for HTTPS requests listed
above this should not result in problems.

As HTTPS requests use only strict TLS checks,
there is no `strict_tls` argument
in `wrap_rustls` function.

Rustls is already used by iroh,
so this change does not introduce new dependencies.
2024-09-26 22:35:44 +00:00
link2xt
638da904e7 refactor: merge build_tls() function into wrap_tls() 2024-09-26 22:35:44 +00:00
iequidoo
fe0c9958a6 feat: Assign message to ad-hoc group with matching name and members (#5385)
This should fix ad-hoc groups splitting when messages are fetched out of order from different
folders or otherwise reordered, or some messages are missing so that the messages reference chain is
broken, or a member was removed from the thread and readded later, etc. Even if this way two
different threads are merged, it looks acceptable, having many threads with the same name/subject
and members isn't a common use case.
2024-09-26 17:09:11 -03:00
iequidoo
c469fcb435 refactor: Move group name calculation out of create_adhoc_group() 2024-09-26 17:09:11 -03:00
link2xt
4b74c9d85f fix: avoid changing delete_server_after default for existing configurations 2024-09-26 19:18:12 +00:00
link2xt
040ac0ffe3 refactor: do not wrap shadowsocks::ProxyClientStream
Updated `shadowsocks` implements `Debug` for the type,
so there is no need to wrap it.
2024-09-26 14:28:08 +00:00
link2xt
624ae86913 api!: make QR code type for proxy not specific to SOCKS5 (#5980) 2024-09-21 18:26:07 +00:00
link2xt
b47b96d5d6 chore(cargo): update iroh to 0.25
According to
<https://www.iroh.computer/blog/iroh-0-25-0-custom-protocols-for-all>
gossip now handles updating direct addresses automatically.
2024-09-20 22:56:24 +00:00
link2xt
f6b5c5d150 feat: generate 144-bit group IDs
Instead of generating 72 random bits
and reducing them to 66 bits of Base64 characters,
generate 144 bits (18 bytes)
which is exactly 24 Base64 characters.

This should still be accepted by existing
Delta Chat clients which expect group ID
to be between 11 and 32 characters.

Message-ID creation is also simplified
to not have `Mr.` prefix
and dot in between two IDs.
Now it is a single ID followed by `@localhost`.

Some outdated documentation comments
are removed, e.g. group messages
don't start with `Gr.` already.
2024-09-20 22:38:28 +00:00
link2xt
9cc65c615c feat(smtp): more verbose SMTP connection establishment errors
The greeting is now always read manually,
even for STARTTLS connections,
so the errors returned on failure to read form the stream
are the same regardless of the connection type.
2024-09-20 20:37:47 +00:00
iequidoo
d6845bd5e9 feat: Use IMAP APPEND command to upload sync messages (#5845)
Why:
- With IMAP APPEND we can upload messages directly to the DeltaChat folder (for non-chatmail
  accounts).
- We can set the `\Seen` flag immediately so that if the user has other MUA, it doesn't alert about
  a new message if it's just a sync message (there were several such reports on the support
  forum). Though this also isn't useful for chatmail.
- We don't need SMTP envelope and overall remove some overhead on processing sync messages.
2024-09-20 17:07:45 -03:00
iequidoo
841ed43f11 feat: Don't put displayname into From/To/Sender if it equals to address (#5983)
If a displayname equals to the address, adding it looks excessive.
Moreover, it's not useful for Delta Chat receiving the message because
`sanitize_name_and_addr()` removes such a displayname anyway. Also now
at least DC Android requires specifying profile name, so there should be
a fallback for users having meaningful addresses to keep the old
behaviour when Core generates `From` w/o the profile name, and this
question has already appeared on the forum.
2024-09-20 15:59:33 -03:00
link2xt
060fd55249 feat: HTTP(S) tunneling
HTTP proxy is tested with deltachat-repl
against local Privoxy
using
```
> set proxy_url http://127.0.0.1:8118/
> setqr dcaccount:https://nine.testrun.org/new
> configure
> connect
```
2024-09-18 10:52:31 +00:00
link2xt
38c7f7300e Partially revert "test(test-data): remove public keys that can be derived from secret keys" (#5977)
This reverts commit 1caf672904.

Otherwise public key signature is regenerated each time the key is
loaded and test `key::tests::test_load_self_existing` which loads the
key twice fails when two loads happen on different seconds.

Closes #5976
2024-09-18 09:48:01 +00:00
link2xt
f7a705c6da refactor: use KeyPair::new() in create_keypair() 2024-09-16 20:51:16 +00:00
iequidoo
f497e4dd12 docs: Why search_msgs() only looks at the first kilobytes of long messages 2024-09-16 17:14:51 -03:00
iequidoo
0a63083df7 fix: Shorten message text in locally sent messages too (#2281) 2024-09-16 17:14:51 -03:00
iequidoo
5a6efdff44 fix: Save QR code token regardless of whether the group exists (#5954)
Groups promotion to other devices and QR code tokens synchronisation are not synchronised processes,
so there are reasons why a QR code token may arrive earlier than the first group message:
- We are going to upload sync messages via IMAP while group messages are sent by SMTP.
- If sync messages go to the mvbox, they can be fetched earlier than group messages from Inbox.
2024-09-16 16:40:26 -03:00
link2xt
1caf672904 test(test-data): remove public keys that can be derived from secret keys 2024-09-16 17:00:16 +00:00
link2xt
7743072411 refactor: remove addr from KeyPair 2024-09-16 17:00:16 +00:00
link2xt
c461c4f02e refactor: do not store deprecated addr and is_default into keypairs 2024-09-16 17:00:16 +00:00