Commit Graph

4261 Commits

Author SHA1 Message Date
B. Petersen
99c4d24eab cleanup jobs and Params relicts
- the `jobs` table is no longer in use,
  no need to track files on housekeeping,
  no need to clear it from repl tool

- some `Params` were used for jobs table only,
  they can be used freely for other purposes on other tables.
  param 'protection settings timestamp' was never used in practise,
  its code is removed as well, so we can free the Param as well.
2024-03-30 08:10:57 +01:00
bjoern
7bf9c4a2d9 api: remove unused dc_accounts_all_work_done() (#5384)
it was used by iOS to know when a background fetch was complete;
meanwhile the superiour `dc_accounts_background_fetch()` is used for
that.

there is still the corresponding context function `dc_all_work_done()`,
this not used by any UI as well, however, it is in use by a python
tests.

not sure, what to do with it, at a first glance, the test still seems
useful.
2024-03-30 01:18:17 +01:00
iequidoo
304e902fce fix: Don't send selfavatar in SecureJoin messages before contact verification (#5354)
Don't attach selfavatar in "v{c,g}-request" and "v{c,g}-auth-required" messages:
- These messages are deleted right after processing, so other devices won't see the avatar.
- It's also good for privacy because the contact isn't yet verified and these messages are auto-sent
  unlike usual unencrypted messages.
2024-03-28 21:29:05 -03:00
B. Petersen
0155d93622 test: Remove flaky time check from test_list_from()
the rendered time output seems different on different systems and timezomes,
eg. on my local machine, it is
`Sent: 2024.03.20 10:00:01 ` and not `Sent: 2024.03.20 09:00:01`,
maybe because of winter/summer time, idk.

as the gist of the test is to check the name,
however, i just removed the whole time check.
2024-03-28 16:40:41 +01:00
link2xt
e702c1a8ca feat: include more entries into DNS fallback cache 2024-03-25 10:30:08 +00:00
link2xt
1adea3c678 fix: put overridden sender name into message info 2024-03-25 05:19:04 +00:00
link2xt
36bdf8a67e fix: do not ignore Contact::get_by_id() error in from_field_to_contact_id() 2024-03-24 18:18:34 +00:00
link2xt
e59ff6ca74 feat: include 3 recent Message-IDs in References header
Do not include oldest reference, because chat members
which have been added later and have not seen the first message
do not have referenced message in the database.

Instead, include up to 3 recent Message-IDs.
2024-03-23 02:06:24 +00:00
iequidoo
0e5db36205 fix: Rescan folders after changing Config::SentboxWatch
If `Config::SentboxWatch` changes, the sentbox needs to be [un]configured which is done by
`Imap::scan_folders()`.
2024-03-22 19:08:05 -03:00
link2xt
b34fe8f118 feat: do not include provider hostname in Message-ID
It is leaked by anonymous mailing lists,
making it possible to tell which provider the sender is using.
Use `localhost` as the hostname instead.
2024-03-20 00:36:15 +00:00
bjoern
810be4f6c7 fix: preserve upper-/lowercase of links parsed by dehtml() (#5362)
this PR fixes a bug that lowercases all links handleld by `dehtml()`,
which is wrong.

closes #5361
2024-03-19 16:38:23 +01:00
link2xt
1ebbe26ebb api!: remove data from DC_EVENT_INCOMING_MSG_BUNCH
It is not used by existing clients
and incorrectly included all downloaded messages,
including outgoing messages and MDNs.
2024-03-19 14:21:35 +00:00
link2xt
c3a7fc4c8d test: test that reordering of Member added message results in square bracket error
This is a test reproducing the problem
in <https://github.com/deltachat/deltachat-core-rust/issues/5339>.
Fix would be to avoid reordering on the server side,
so the test checks that the unverified message
is replaced with a square bracket error
as expected if messages arrive in the wrong order.
2024-03-18 18:14:06 +00:00
iequidoo
4b4c57a480 fix: Add white background to recoded avatars (#3787)
Add white background instead of the default black one to avatars when recoding to JPEG. But not for
"usual" images to spare the CPU. The motivation is to handle correctly
"black-on-transparent-background" avatars which are quite common.
2024-03-17 22:14:08 -03:00
bjoern
b95d58208c add save_mime_headers to debug info (#5350)
as turned out on recent researches wrt a slow database, setting
`save_mime_headers` will result in 25x larger databases.

this is definetely something we want to know at least in the debug info.

(the option cannot be enabled in current UIs,
however, esp. devs find options to set this.
apart from that, save_mime_headers is only used in some python tests)

ftr: the huge database was more than 10 times slower, leading to missing
notifications on ios (as things do not finish within the few seconds iOS
gave us).
moreover, the hugeness also avoids exporting; this is fixed by
https://github.com/deltachat/deltachat-core-rust/pull/5349
2024-03-17 08:27:08 +01:00
bjoern
c468eb088e fix: on iOS, use FILE (default) instead of MEMORY (#5349)
this PR fixes one of the issues we had with an (honestly accidentally)
huge database of >2gb.

this database could not be exported on iOS, as ram memory is limited
there, leading to the app just crashing.

it is unclear if that would be better on eg. Android, however,
temp_store=FILE is not working as well there, this is the smaller
drawback there.

before merging, this PR should be tested on time with export/import on
iOS (export/import uses VACUUM which uses /tmp) EDIT: did so, works on
iphone7 with ios15
2024-03-17 08:25:55 +01:00
B. Petersen
de37135ed6 nicer summaries: prefer emoji over names 2024-03-15 19:20:33 +01:00
iequidoo
33777d8759 fix: Update MemberListTimestamp when sending a group message
`Param::MemberListTimestamp` was updated only from `receive_imf::apply_group_changes()` i.e. for
received messages. If we sent a message, that timestamp wasn't updated, so remote group membership
changes always overrode local ones. Especially that was a problem when a message is sent offline so
that it doesn't incorporate recent group membership changes.
2024-03-15 06:14:38 -03:00
link2xt
8cc348bfa4 fix: terminate ephemeral and location loop immediately on channel close
When scheduler is destroyed, e.g. during a key import,
there is some time between destroying the interrupt channel
and the loop task.

To avoid busy looping, tasks should terminate if
receiving from the interrupt loop fails
instead of treating it as the interrupt.
2024-03-15 01:26:23 +00:00
iequidoo
7d7289bd51 feat: Start IMAP loop for sentbox only if it is configured (#5105) 2024-03-09 15:06:49 +01:00
iequidoo
c0832af634 refactor: Remove deduplicate_peerstates()
There's the `UNIQUE (acpeerstates.addr)` constraint since db v94.
2024-03-08 00:42:39 -03:00
iequidoo
b6db0152b0 fix: Create new Peerstate for unencrypted message with already known Autocrypt key, but a new address
An unencrypted message with already known Autocrypt key, but sent from another address, means that
it's rather a new contact sharing the same key than the existing one changed its address, otherwise
it would already have our key to encrypt.
2024-03-08 00:42:39 -03:00
iequidoo
bc7fd4495b fix: Remove leading whitespace from Subject (#5106)
If Subject is multiline-formatted, `mailparse` adds the leading whitespace to it. The solution is to
always remove the leading whitespace, because if Subject isn't multiline-formatted, it never
contains the leading whitespace anyway. But as for the trailing whitespace -- i checked -- it's
never removed, so let's keep this as is.
2024-03-08 00:08:52 -03:00
link2xt
2030de11d9 chore: fix 2024-03-05 nightly clippy warnings 2024-03-08 02:53:47 +00:00
iequidoo
e639b58c6f refactor: Don't even parse Autocrypt header for outgoing messages (#5259)
Accordingly, there's no need in `Peerstate` for self addresses (and in the db too).
2024-03-06 19:32:37 -03:00
link2xt
7502234686 api: dc_accounts_set_push_device_token and dc_get_push_state APIs 2024-03-04 21:10:04 +00:00
link2xt
863a386d0f test: test that ASM "encrypted" with plaintext algorithm is not accepted 2024-03-04 21:10:04 +00:00
link2xt
e4b49dfdef fix: validate Group IDs and SecureJoin tokens 2024-03-04 21:10:04 +00:00
iequidoo
612aa1431e fix: Check that peer SecureJoin messages (except vc/vg-request) gossip our addr+pubkey
This fixes the following identity-misbinding attack:

It appears that Bob’s messages in the SecureJoin protocol do not properly “bind” to Alice’s public
key or fingerprint. Even though Bob’s messages carry Alice’s public key and address as a gossip in
the protected payload, Alice does not reject the message if the gossiped key is different from her
own key. As a result, Mallory could perform an identity-misbinding attack. If Mallory obtained
Alice’s QR invite code, she could change her own QR code to contain the same tokens as in Alice’s QR
code, and convince Bob to scan the modified QR code, possibly as an insider attacker. Mallory would
forward messages from Bob to Alice and craft appropriate responses for Bob on his own. In the end,
Bob would believe he is talking to Mallory, but Alice would believe she is talking to Bob.
2024-03-04 21:10:04 +00:00
link2xt
781d3abdb9 fix: make should_do_gossip() return true even if we send securejoin only to Alice 2024-03-04 21:10:04 +00:00
link2xt
78d01933ad fix: don't leak Group-ID in Message-ID
Chat assignment based on In-Reply-To and References works good enough
even if the message cannot be decrypted.
2024-03-04 21:07:10 +00:00
iequidoo
1a1467f7cf fix: Remove unsigned Chat-Group-* headers from Autocrypt-encrypted messages
These headers are opportunistically protected, so if they appear in the unencrypted part, they are
probably added by a malicious server.
2024-03-04 21:07:10 +00:00
link2xt
8d09291d1e fix: do not send Secure-Join-Group in vg-request
Secure-Join-Group is only expected by old core in vg-request-with-auth.
There is no reason to leak group ID in unencrypted vg-request.
Besides that, Secure-Join-Group is deprecated
as Alice knows Group ID corresponding to the auth code,
so the header can be removed completely eventually.
2024-03-04 21:07:10 +00:00
link2xt
4ccd2b8d02 fix: require that Autocrypt Setup Message is self-sent 2024-03-04 21:07:10 +00:00
iequidoo
794596ec69 fix: Don't log SecureJoin QRs
Delta Chat mustn't write sensitive information to unencrypted log files in local storage.
2024-03-04 21:07:10 +00:00
link2xt
3a787519b3 test: test that encrypted Message-ID overwrites X-Microsoft-Original-Message-ID 2024-03-04 21:07:10 +00:00
link2xt
c03e163ed2 fix: reject messages with protected From not corresponding to outer From 2024-03-04 21:07:10 +00:00
iequidoo
6cee295a5d fix: Don't treat forged outgoing messages as Autocrypt-encrypted 2024-03-04 21:07:10 +00:00
iequidoo
f0be7daae9 test: Add failing test on outgoing message forgery
If a message is sent from SELF, but signed with a foreign key, it mustn't be considered
Autocrypt-encrypted and shown with a padlock. Currently this is broken.
2024-03-04 21:07:10 +00:00
link2xt
0b279ec84e fix: make protected Message-ID take precedence over X-Microsoft-Original-Message-ID 2024-03-04 21:07:10 +00:00
link2xt
e919de78a3 fix: do not take Secure-Join-Auth from unprotected headers 2024-03-04 21:07:10 +00:00
link2xt
6ea675a12f fix: do not use Secure-Join-Group header
Alice already knows which auth token corresponds to which group.
There is no need to trust Bob on sending the correct group ID.
2024-03-04 21:07:10 +00:00
link2xt
b970ebe67a fix: do not compress SecureJoin messages 2024-03-04 21:07:10 +00:00
link2xt
3c4c701f9b fix: protect Secure-Join header
Secure-Join header must come from protected headers
unless it is a "vc-request" or "vg-request".
2024-03-04 21:07:10 +00:00
link2xt
01ac9c8b90 fix: ensure Autocrypt-Gossip is not taken from insecure headers 2024-03-04 21:07:10 +00:00
Hocuri
d7615b223f feat(Self-Reporting): Report number of protected/encrypted/unencrypted chats (#5292) 2024-03-04 21:07:09 +00:00
link2xt
08247a5d37 refactor: build contexts using ContextBuilder 2024-03-02 17:19:50 +00:00
link2xt
ceadd8928e api: add ContextBuilder.build() to build Context without opening 2024-03-02 17:19:50 +00:00
link2xt
5499ca52bf refactor: get rid of ImapActionResult 2024-03-02 01:31:29 +00:00
link2xt
4e8979f7c8 refactor: merge ImapConfig into Imap 2024-03-01 21:12:21 +00:00