Commit Graph

108 Commits

Author SHA1 Message Date
Hocuri
1379821b03 refactor: Move logins into SQL table (#6724)
Move all `configured_*` parameters into a new SQL table `transports`.
All `configured_*` parameters are deprecated; the only exception is
`configured_addr`, which is used to store the address of the primary
transport. Currently, there can only ever be one primary transport (i.e.
the `transports` table only ever has one row); this PR is not supposed
to change DC's behavior in any meaningful way.

This is a preparation for mt.

---------

Co-authored-by: l <link2xt@testrun.org>
2025-04-13 19:06:41 +02:00
link2xt
9f5e608c61 feat: track gossiping per (chat, fingerprint) pair
This change simplifies
updating the gossip timestamps
when we receive a message
because we only need to know
the keys received in Autocrypt-Gossip
header and which chat the message is
assigned to.
We no longer need to iterate
over the member list.

This is a preparation
for PGP contacts
and member lists that contain
key fingerprints rather than
email addresses.

This change also removes encryption preference
from Autocrypt-Gossip header.
It SHOULD NOT be gossiped
according to the Autocrypt specification
and we ignore encryption preference anyway
since 1.157.0.

test_gossip_optimization is removed
because it relied on a per-chat gossip_timestamp.
2025-04-12 02:51:11 +00:00
link2xt
3e8987b460 test: port test_delete_deltachat_folder to JSON-RPC 2025-04-09 13:28:26 +00:00
link2xt
0e874735ac test: encrypt legacy Python tests
This makes it possible to run Python tests
against chatmail servers that don't allow
any unencrypted messages.
2025-04-06 08:42:33 +00:00
link2xt
cecd3a2956 test: port test_one_account_send_bcc_setting from legacy Python to JSON-RPC 2025-04-06 07:42:34 +00:00
link2xt
c8c6beb1b6 test: port test_forward_encrypted_to_unencrypted from legacy Python to Rust 2025-04-06 07:42:34 +00:00
bjoern
97b0d09ed2 feat: get contact-id for info messages (#6714)
instead of showing addresses in info message, provide an API to get the
contact-id.

UI can then make the info message tappable and open the contact profile
in scope

the corresponding iOS PR - incl. **screencast** - is at
https://github.com/deltachat/deltachat-ios/pull/2652 ; jsonrpc can come
in a subsequent PR when things are settled on android/ios

the number of parameters in `add_info_msg_with_cmd` gets bigger and
bigger, however, i did not want to refactor this in this PR. it is also
not really adding complexity



closes #6702

---------

Co-authored-by: link2xt <link2xt@testrun.org>
Co-authored-by: Hocuri <hocuri@gmx.de>
2025-03-31 18:56:57 +02:00
link2xt
51bbdadfad feat: ignore encryption preferences
Encryption preference is sent in Autocrypt header,
but otherwise ignored.

Delta Chat always prefers encryption if it is available.
2025-03-12 16:44:52 +00:00
link2xt
339f695bd6 test(python): port test_no_old_msg_is_fresh to JSON-RPC 2025-03-12 16:44:52 +00:00
link2xt
c4e6823396 api!: remove key_gen_type config
This removes the ability to generate RSA keys.
2025-03-06 21:41:41 +00:00
link2xt
0913b6707b api!: remove save_mime_headers config option and dc_get_mime_headers()
This was only used in tests.
`msgs.mime_headers` coulmn remains
as it is used for HTML messages.
2025-03-06 21:12:18 +00:00
link2xt
8c2207d15e fix: make it impossible to overwrite default key
Replacing default key
when a profile is already part of
verified groups results in
`[The message was sent with non-verified encryption. See 'Info' for more details]`
messages for other users.

It is still possible
to import the default key before
Delta Chat generates the key.
2025-02-26 20:56:52 +00:00
link2xt
14d048bea8 feat: improve logging around IMAP IDLE 2025-01-28 00:35:28 +00:00
Hocuri
65a9c4b79b File deduplication (#6332)
When receiving messages, blobs will be deduplicated with the new
function `create_and_deduplicate_from_bytes()`. For sending files, this
adds a new function `set_file_and_deduplicate()` instead of
deduplicating by default.

This is for
https://github.com/deltachat/deltachat-core-rust/issues/6265; read the
issue description there for more details.

TODO:
- [x] Set files as read-only
- [x] Don't do a write when the file is already identical
- [x] The first 32 chars or so of the 64-character hash are enough. I
calculated that if 10b people (i.e. all of humanity) use DC, and each of
them has 200k distinct blob files (I have 4k in my day-to-day account),
and we used 20 chars, then the expected value for the number of name
collisions would be ~0.0002 (and the probability that there is a least
one name collision is lower than that) [^1]. I added 12 more characters
to be on the super safe side, but this wouldn't be necessary and I could
also make it 20 instead of 32.
- Not 100% sure whether that's necessary at all - it would mainly be
necessary if we might hit a length limit on some file systems (the
blobdir is usually sth like
`accounts/2ff9fc096d2f46b6832b24a1ed99c0d6/dc.db-blobs` (53 chars), plus
64 chars for the filename would be 117).
- [x] "touch" the files to prevent them from being deleted
- [x] TODOs in the code

For later PRs:
- Replace `BlobObject::create(…)` with
`BlobObject::create_and_deduplicate(…)` in order to deduplicate
everytime core creates a file
- Modify JsonRPC to deduplicate blob files
- Possibly rename BlobObject.name to BlobObject.file in order to prevent
confusion (because `name` usually means "user-visible-name", not "name
of the file on disk").

[^1]: Calculated with both https://printfn.github.io/fend/ and
https://www.geogebra.org/calculator, both of which came to the same
result
([1](https://github.com/user-attachments/assets/bbb62550-3781-48b5-88b1-ba0e29c28c0d),

[2](https://github.com/user-attachments/assets/82171212-b797-4117-a39f-0e132eac7252))

---------

Co-authored-by: l <link2xt@testrun.org>
2025-01-21 19:42:19 +01:00
iequidoo
2bce4466d7 fix: Prefer to encrypt if E2eeEnabled even if peers have EncryptPreference::NoPreference
First of all, chatmail servers normally forbid to send unencrypted mail, so if we know the peer's
key, we should encrypt to it. Chatmail setups have `E2eeEnabled=1` by default and this isn't
possible to change in UIs, so this change fixes the chatmail case. Additionally, for chatmail, if a
peer has `EncryptPreference::Reset`, let's handle it as `EncryptPreference::NoPreference` for the
reason above. Still, if `E2eeEnabled` is 0 for a chatmail setup somehow, e.g. the user set it via
environment, let's assume that the user knows what they do and ignore `IsChatmail` flag.

NB:
- If we don't know the peer's key, we should try to send an unencrypted message as before for a
  chatmail setup.
- This change doesn't remove the "majority rule", but now the majority with
  `EncryptPreference::NoPreference` can't disable encryption if the local preference is `Mutual`. To
  disable encryption, some peer should have a missing peerstate or, for the non-chatmail case, the
  majority should have `EncryptPreference::Reset`.
2025-01-04 20:16:38 -03:00
link2xt
72558af98c api!: remove dc_prepare_msg and dc_msg_is_increation 2024-12-11 19:34:29 +00:00
link2xt
e694411974 api!: remove dc_all_work_done()
Also cleaned up test_connectivity()
which tested that state does not flicker to WORKING
when there are no messages to be fetched.
The state is expected to flicker to WORKING
when checking for new messages,
so the tests were outdated since
change 3b0b2379b8
2024-12-04 14:31:55 +00:00
link2xt
3b0b2379b8 fix: replace connectivity state "Connected" with "Preparing"
This better reflects that this state means
we just connected and there may me work to do.
This state is converted to DC_CONNECTIVITY_WORKING
instead of DC_CONNECTIVITY_CONNECTED state now.

Before this change when IMAP connected
to the server, it switched
from DC_CONNECTIVITY_NOT_CONNECTED
to DC_CONNECTIVITY_CONNECTING,
then to DC_CONNECTIVITY_CONNECTED (actually preparing)
then to DC_CONNECTIVITY_WORKING
and then to DC_CONNECTIVITY_CONNECTED again (actually idle).

On fast connections this resulted in flickering "Connected"
string in the status bar right before "Updating..."
and on slow connections this "Connected" state
before "Updating..." lasted for a while
leaving the user to wonder if there are no new messages
or if Delta Chat will still switch to "Updating..."
before going into "Connected" state again.
2024-12-03 00:35:38 +00:00
iequidoo
33cae2815d fix: Set Config::NotifyAboutWrongPw before saving configuration (#5896)
Let's always set `Config::NotifyAboutWrongPw` before saving configuration, better if a wrong
password notification is shown once more than not shown at all. It shouldn't be a big problem
because reconfiguration is a manual action and isn't done frequently.

Also for the same reason reset `Config::NotifyAboutWrongPw` only after a successful addition of the
appropriate device message.
2024-10-25 13:14:37 -03:00
iequidoo
d6845bd5e9 feat: Use IMAP APPEND command to upload sync messages (#5845)
Why:
- With IMAP APPEND we can upload messages directly to the DeltaChat folder (for non-chatmail
  accounts).
- We can set the `\Seen` flag immediately so that if the user has other MUA, it doesn't alert about
  a new message if it's just a sync message (there were several such reports on the support
  forum). Though this also isn't useful for chatmail.
- We don't need SMTP envelope and overall remove some overhead on processing sync messages.
2024-09-20 17:07:45 -03:00
iequidoo
5b597f3a95 feat: Don't SMTP-send messages to self-chat if BccSelf is disabled
`chat::create_send_msg_jobs()` already handles `Config::BccSelf` as needed. The only exception is
Autocrypt setup messages. This change unifies the logic for the self-chat and groups only containing
`SELF`.
2024-09-15 23:48:06 -03:00
iequidoo
ff6488371c feat: Delete messages from a chatmail server immediately by default (#5805) (#5840)
I.e. treat `DeleteServerAfter == None` as "delete at once". But when a backup is exported, set
`DeleteServerAfter` to 0 so that the server decides when to delete messages, in order not to break
the multi-device case. Even if a backup is not aimed for deploying more devices, `DeleteServerAfter`
must be set to 0, otherwise the backup is half-useful because after a restoration the user wouldn't
see new messages deleted by the device after the backup was done. But if the user explicitly set
`DeleteServerAfter`, don't change it when exporting a backup. Anyway even for non-chatmail case the
app should warn the user before a backup export if they have `DeleteServerAfter` enabled.

Also do the same after a backup import. While this isn't reliable as we can crash in between, this
is a problem only for old backups, new backups already have `DeleteServerAfter` set if necessary.

---------

Co-authored-by: Hocuri <hocuri@gmx.de>
2024-09-08 16:53:56 -03:00
iequidoo
86ad5506e3 feat: Always move outgoing auto-generated messages to the mvbox
Recently there are many questions on the Delta Chat forum why some unexpected encrypted messages
appear in Inbox. Seems they are mainly sync messages, though that also obviously happens to
SecureJoin messages. Anyway, regardless of the `MvboxMove` setting, auto-generated outgoing messages
should be moved to the DeltaChat folder so as not to complicate co-using Delta Chat with other MUAs.
2024-08-06 11:33:22 -03:00
link2xt
d6d90db957 feat: new BACKUP2 transfer protocol
New protocol streams .tar into iroh-net
stream without traversing all the files first.
Reception over old backup protocol
is still supported to allow
transferring backups from old devices
to new ones, but not vice versa.
2024-07-19 03:16:57 +00:00
link2xt
f49588e64e fix: interrupt location loop when new location is stored
Otherwise location-only messages
that should be sent every 60 seconds
are never sent because location loop
waits until the end of location streaming
and is only interrupted by location streaming
ending in other chats or being enabled in other chats.
2024-04-27 20:55:05 +00:00
link2xt
4cf923ccb9 fix: do not fail to send encrypted quotes to unencrypted chats
Replace quote text with "..." instead.
2024-04-25 09:00:49 +00:00
iequidoo
e9cfcd9d1b fix: Don't try to do fetch_move_delete() if Trash is needed but not yet configured
This fixes things for Gmail f.e. Before, `Imap::fetch_move_delete()` was called before looking for
Trash and returned an error because of that failing the whole `fetch_idle()` which prevented
configuring Trash in turn.
2024-04-10 21:06:43 -03:00
link2xt
2be28f1311 test: move reaction tests to JSON-RPC 2024-04-08 19:11:16 +00:00
iequidoo
7960944b14 test: test_mvbox_sentbox_threads: Check that sentbox gets configured after setting sentbox_watch (#5105) 2024-03-22 19:08:05 -03:00
iequidoo
e16322d99d test: get_protected_chat(): Use FFIEventTracker instead of dc_wait_next_msgs() (#5207)
The way it was implemented it threw out all remaining messages after finding the next incoming
message. Better use FFIEventTracker functions, they are used in all the tests anyway.
2024-03-02 12:12:26 -03:00
iequidoo
b5f2c747e0 feat: Context::set_config(): Restart IO scheduler if needed (#5111)
Restart the IO scheduler if needed to make the new config value effective (for `MvboxMove,
OnlyFetchMvbox, SentboxWatch` currently). Also add `set_config_internal()` which doesn't affect
running the IO scheduler. The reason is that `Scheduler::start()` itself calls `set_config()`,
although not for the mentioned keys, but still, and also Rust complains about recursive async calls.
2024-02-12 15:41:11 -03:00
link2xt
2f29c56a36 fix: do not log error if watched folder is not configured
This may happen if Sent folder does not exist
but configuration option to watch it is enabled.
2024-01-31 14:04:03 +01:00
Simon Laux
1b7c5be9c5 Test server has no sentbox folder 2024-01-31 14:04:03 +01:00
iequidoo
06f1fe18d6 fix: Delete resent messages on receiver side (#5155)
If a Delta Chat message has the Message-ID already existing in the db, but a greater "Date", it's a
resent message that can be deleted. Messages having the same "Date" mustn't be deleted because they
can be already seen messages moved back to INBOX. Also don't delete messages having lesser "Date" to
avoid deleting both messages in a multi-device setting.
2024-01-30 22:01:45 -03:00
iequidoo
bd3759d55e test: test_import_export_online_all: Send the message to the existing address to avoid errors (#5220) 2024-01-29 20:34:03 -03:00
missytake
bbc6febb72 test: no timeout in SetupPlugin 2024-01-17 14:20:29 +01:00
iequidoo
c16c5e0802 test: Bring test_forward_encrypted_to_unencrypted into line with current API
Currently `Chat.send_msg()` modifies the source message and returns another message object
equivalent to the source one. That's how it works in the core and thus in Python bindings too.
2024-01-17 14:20:29 +01:00
missytake
36cab40ac1 test: add get_protected_chat to testplugin.py 2024-01-17 14:20:29 +01:00
missytake
4186d78305 test: add python test for message forwarding from encrypted to unencrypted chat 2024-01-17 14:20:29 +01:00
link2xt
828cc1fbd1 fix(connectivity): return false from all_work_done() immediately after connecting
We do not want all_work_done() to return true immediately
after calling start_io(), but only when connection goes idle.

"Connected" state is set immediately after connecting to the server,
but it does not mean there is nothing to do.

This change make all_work_done() return false
from the Connected state and introduces a new Idle
connectivity state that is only set before connection
actually goes idle. For idle state all_work_done() returns true.

From the user point of view both old Connected state
and new Idle state look the same.
2023-12-12 02:19:51 +00:00
link2xt
ab151654fb chore: remove unused import from python tests 2023-11-18 11:32:34 +01:00
link2xt
ea9556b1b9 test: port test_aeap_flow_verified to JSON-RPC 2023-11-18 11:32:34 +01:00
link2xt
f39acbc037 test: port test_qr_new_group_unblocked() to JSON-RPC 2023-11-18 11:32:34 +01:00
link2xt
005f7ff07e test: port test_qr_join_chat_with_pending_bobstate_issue4894 to JSON-RPC 2023-11-18 11:32:34 +01:00
link2xt
144ca7c171 test: port test_qr_join_chat to JSON-RPC
Disabled `verified_one_on_one_chats` is not tested
as it is not interesting, other checks are moved.
2023-11-18 11:32:34 +01:00
link2xt
7012b99d73 test: remove test_qr_setup_contact from online python tests
There is an identical `test_qr_setup_contact` in deltachat-rpc-client
test suite which also checks that contact profiles get verified.
2023-11-18 11:32:34 +01:00
B. Petersen
28952789a4 fix: raise lower auto-download limit to 160k 2023-11-07 23:27:02 +00:00
bjoern
939b4b2aab feat: add 'group created instructions' as info message (#4916)
Before, it was shown by UI when the chat is empty, however, in case of
verified groups, this is never the case any longer as the 'e2ee now
guaranteed' is always added.
2023-11-05 19:24:34 +00:00
holger krekel
196a34684d simplify "broken bobstate" as the state still passes 2023-11-01 00:58:54 +01:00
holger krekel
402fd6850c do the actual fix (thanks alex for some help) 2023-11-01 00:58:54 +01:00