Commit Graph

555 Commits

Author SHA1 Message Date
bjoern
97b0d09ed2 feat: get contact-id for info messages (#6714)
instead of showing addresses in info message, provide an API to get the
contact-id.

UI can then make the info message tappable and open the contact profile
in scope

the corresponding iOS PR - incl. **screencast** - is at
https://github.com/deltachat/deltachat-ios/pull/2652 ; jsonrpc can come
in a subsequent PR when things are settled on android/ios

the number of parameters in `add_info_msg_with_cmd` gets bigger and
bigger, however, i did not want to refactor this in this PR. it is also
not really adding complexity



closes #6702

---------

Co-authored-by: link2xt <link2xt@testrun.org>
Co-authored-by: Hocuri <hocuri@gmx.de>
2025-03-31 18:56:57 +02:00
link2xt
51bbdadfad feat: ignore encryption preferences
Encryption preference is sent in Autocrypt header,
but otherwise ignored.

Delta Chat always prefers encryption if it is available.
2025-03-12 16:44:52 +00:00
link2xt
339f695bd6 test(python): port test_no_old_msg_is_fresh to JSON-RPC 2025-03-12 16:44:52 +00:00
link2xt
c4e6823396 api!: remove key_gen_type config
This removes the ability to generate RSA keys.
2025-03-06 21:41:41 +00:00
link2xt
0913b6707b api!: remove save_mime_headers config option and dc_get_mime_headers()
This was only used in tests.
`msgs.mime_headers` coulmn remains
as it is used for HTML messages.
2025-03-06 21:12:18 +00:00
link2xt
8c2207d15e fix: make it impossible to overwrite default key
Replacing default key
when a profile is already part of
verified groups results in
`[The message was sent with non-verified encryption. See 'Info' for more details]`
messages for other users.

It is still possible
to import the default key before
Delta Chat generates the key.
2025-02-26 20:56:52 +00:00
Hocuri
a49dfeca6e refactor: Remove Message.set_file() / dc_msg_set_file() and related code (#6558)
Now that we are deduplicating everywhere, we can get rid of some code.

The old python bindings did not get an optional `name` parameter because
they are deprecated anyway, but it would be easy to add it.
2025-02-22 10:47:52 +01:00
link2xt
14d048bea8 feat: improve logging around IMAP IDLE 2025-01-28 00:35:28 +00:00
Hocuri
65a9c4b79b File deduplication (#6332)
When receiving messages, blobs will be deduplicated with the new
function `create_and_deduplicate_from_bytes()`. For sending files, this
adds a new function `set_file_and_deduplicate()` instead of
deduplicating by default.

This is for
https://github.com/deltachat/deltachat-core-rust/issues/6265; read the
issue description there for more details.

TODO:
- [x] Set files as read-only
- [x] Don't do a write when the file is already identical
- [x] The first 32 chars or so of the 64-character hash are enough. I
calculated that if 10b people (i.e. all of humanity) use DC, and each of
them has 200k distinct blob files (I have 4k in my day-to-day account),
and we used 20 chars, then the expected value for the number of name
collisions would be ~0.0002 (and the probability that there is a least
one name collision is lower than that) [^1]. I added 12 more characters
to be on the super safe side, but this wouldn't be necessary and I could
also make it 20 instead of 32.
- Not 100% sure whether that's necessary at all - it would mainly be
necessary if we might hit a length limit on some file systems (the
blobdir is usually sth like
`accounts/2ff9fc096d2f46b6832b24a1ed99c0d6/dc.db-blobs` (53 chars), plus
64 chars for the filename would be 117).
- [x] "touch" the files to prevent them from being deleted
- [x] TODOs in the code

For later PRs:
- Replace `BlobObject::create(…)` with
`BlobObject::create_and_deduplicate(…)` in order to deduplicate
everytime core creates a file
- Modify JsonRPC to deduplicate blob files
- Possibly rename BlobObject.name to BlobObject.file in order to prevent
confusion (because `name` usually means "user-visible-name", not "name
of the file on disk").

[^1]: Calculated with both https://printfn.github.io/fend/ and
https://www.geogebra.org/calculator, both of which came to the same
result
([1](https://github.com/user-attachments/assets/bbb62550-3781-48b5-88b1-ba0e29c28c0d),

[2](https://github.com/user-attachments/assets/82171212-b797-4117-a39f-0e132eac7252))

---------

Co-authored-by: l <link2xt@testrun.org>
2025-01-21 19:42:19 +01:00
link2xt
3d19996f34 test: fix test_logged_ac_process_ffi_failure flakiness
This test keeps failing on macOS CI,
capturing events like `DC_EVENT_ACCOUNTS_ITEM_CHANGED`
before FailPlugin is setup.
These CI runners likely get less resources
because there is a limited number of them,
and this triggers this race condition.

Race is fixed by setting up fail plugin
before starting to capture events.
2025-01-05 01:36:09 +00:00
iequidoo
2bce4466d7 fix: Prefer to encrypt if E2eeEnabled even if peers have EncryptPreference::NoPreference
First of all, chatmail servers normally forbid to send unencrypted mail, so if we know the peer's
key, we should encrypt to it. Chatmail setups have `E2eeEnabled=1` by default and this isn't
possible to change in UIs, so this change fixes the chatmail case. Additionally, for chatmail, if a
peer has `EncryptPreference::Reset`, let's handle it as `EncryptPreference::NoPreference` for the
reason above. Still, if `E2eeEnabled` is 0 for a chatmail setup somehow, e.g. the user set it via
environment, let's assume that the user knows what they do and ignore `IsChatmail` flag.

NB:
- If we don't know the peer's key, we should try to send an unencrypted message as before for a
  chatmail setup.
- This change doesn't remove the "majority rule", but now the majority with
  `EncryptPreference::NoPreference` can't disable encryption if the local preference is `Mutual`. To
  disable encryption, some peer should have a missing peerstate or, for the non-chatmail case, the
  majority should have `EncryptPreference::Reset`.
2025-01-04 20:16:38 -03:00
link2xt
72558af98c api!: remove dc_prepare_msg and dc_msg_is_increation 2024-12-11 19:34:29 +00:00
link2xt
e694411974 api!: remove dc_all_work_done()
Also cleaned up test_connectivity()
which tested that state does not flicker to WORKING
when there are no messages to be fetched.
The state is expected to flicker to WORKING
when checking for new messages,
so the tests were outdated since
change 3b0b2379b8
2024-12-04 14:31:55 +00:00
link2xt
3b0b2379b8 fix: replace connectivity state "Connected" with "Preparing"
This better reflects that this state means
we just connected and there may me work to do.
This state is converted to DC_CONNECTIVITY_WORKING
instead of DC_CONNECTIVITY_CONNECTED state now.

Before this change when IMAP connected
to the server, it switched
from DC_CONNECTIVITY_NOT_CONNECTED
to DC_CONNECTIVITY_CONNECTING,
then to DC_CONNECTIVITY_CONNECTED (actually preparing)
then to DC_CONNECTIVITY_WORKING
and then to DC_CONNECTIVITY_CONNECTED again (actually idle).

On fast connections this resulted in flickering "Connected"
string in the status bar right before "Updating..."
and on slow connections this "Connected" state
before "Updating..." lasted for a while
leaving the user to wonder if there are no new messages
or if Delta Chat will still switch to "Updating..."
before going into "Connected" state again.
2024-12-03 00:35:38 +00:00
Sebastian Klähn
8f3be764d2 change: Use i.delta.chat in qr codes (#6223)
As discussed in #5467 we want to use `i.delta.chat` in QR codes in favor
of `OPENPGP4FPR:` scheme. This PR does the replacement in
`get_securejoin_qr` which is used in `get_securejoin_qr_svg`.

close #5467
2024-11-19 17:32:42 +01:00
iequidoo
33cae2815d fix: Set Config::NotifyAboutWrongPw before saving configuration (#5896)
Let's always set `Config::NotifyAboutWrongPw` before saving configuration, better if a wrong
password notification is shown once more than not shown at all. It shouldn't be a big problem
because reconfiguration is a manual action and isn't done frequently.

Also for the same reason reset `Config::NotifyAboutWrongPw` only after a successful addition of the
appropriate device message.
2024-10-25 13:14:37 -03:00
iequidoo
d6845bd5e9 feat: Use IMAP APPEND command to upload sync messages (#5845)
Why:
- With IMAP APPEND we can upload messages directly to the DeltaChat folder (for non-chatmail
  accounts).
- We can set the `\Seen` flag immediately so that if the user has other MUA, it doesn't alert about
  a new message if it's just a sync message (there were several such reports on the support
  forum). Though this also isn't useful for chatmail.
- We don't need SMTP envelope and overall remove some overhead on processing sync messages.
2024-09-20 17:07:45 -03:00
link2xt
7743072411 refactor: remove addr from KeyPair 2024-09-16 17:00:16 +00:00
iequidoo
5b597f3a95 feat: Don't SMTP-send messages to self-chat if BccSelf is disabled
`chat::create_send_msg_jobs()` already handles `Config::BccSelf` as needed. The only exception is
Autocrypt setup messages. This change unifies the logic for the self-chat and groups only containing
`SELF`.
2024-09-15 23:48:06 -03:00
iequidoo
ff6488371c feat: Delete messages from a chatmail server immediately by default (#5805) (#5840)
I.e. treat `DeleteServerAfter == None` as "delete at once". But when a backup is exported, set
`DeleteServerAfter` to 0 so that the server decides when to delete messages, in order not to break
the multi-device case. Even if a backup is not aimed for deploying more devices, `DeleteServerAfter`
must be set to 0, otherwise the backup is half-useful because after a restoration the user wouldn't
see new messages deleted by the device after the backup was done. But if the user explicitly set
`DeleteServerAfter`, don't change it when exporting a backup. Anyway even for non-chatmail case the
app should warn the user before a backup export if they have `DeleteServerAfter` enabled.

Also do the same after a backup import. While this isn't reliable as we can crash in between, this
is a problem only for old backups, new backups already have `DeleteServerAfter` set if necessary.

---------

Co-authored-by: Hocuri <hocuri@gmx.de>
2024-09-08 16:53:56 -03:00
iequidoo
86ad5506e3 feat: Always move outgoing auto-generated messages to the mvbox
Recently there are many questions on the Delta Chat forum why some unexpected encrypted messages
appear in Inbox. Seems they are mainly sync messages, though that also obviously happens to
SecureJoin messages. Anyway, regardless of the `MvboxMove` setting, auto-generated outgoing messages
should be moved to the DeltaChat folder so as not to complicate co-using Delta Chat with other MUAs.
2024-08-06 11:33:22 -03:00
link2xt
d6d90db957 feat: new BACKUP2 transfer protocol
New protocol streams .tar into iroh-net
stream without traversing all the files first.
Reception over old backup protocol
is still supported to allow
transferring backups from old devices
to new ones, but not vice versa.
2024-07-19 03:16:57 +00:00
iequidoo
13f58e0ca5 fix: Delete user-deleted messages on the server even if they show up on IMAP later
Before, if the user deleted a message too quickly after sending, it was deleted only locally. The
fix is to remember for tombstones that the corresponding message should be deleted on the server
too.
2024-06-19 18:41:24 -03:00
iequidoo
72bfae9448 fix: Keep tombstones for two days before deleting (#3685)
This is a way to prevent redownloading locally deleted messages. Otherwise if a message is deleted
quickly after sending and `bcc_self` is configured, the BCC copy is downloaded and appears as a new
message as it happens for messages sent from another device.
2024-06-19 18:41:24 -03:00
link2xt
f49588e64e fix: interrupt location loop when new location is stored
Otherwise location-only messages
that should be sent every 60 seconds
are never sent because location loop
waits until the end of location streaming
and is only interrupted by location streaming
ending in other chats or being enabled in other chats.
2024-04-27 20:55:05 +00:00
link2xt
94dc65c1a2 refactor(python): fix ruff 0.4.2 warnings 2024-04-25 20:45:54 +00:00
link2xt
4cf923ccb9 fix: do not fail to send encrypted quotes to unencrypted chats
Replace quote text with "..." instead.
2024-04-25 09:00:49 +00:00
iequidoo
e9cfcd9d1b fix: Don't try to do fetch_move_delete() if Trash is needed but not yet configured
This fixes things for Gmail f.e. Before, `Imap::fetch_move_delete()` was called before looking for
Trash and returned an error because of that failing the whole `fetch_idle()` which prevented
configuring Trash in turn.
2024-04-10 21:06:43 -03:00
link2xt
2be28f1311 test: move reaction tests to JSON-RPC 2024-04-08 19:11:16 +00:00
iequidoo
304e902fce fix: Don't send selfavatar in SecureJoin messages before contact verification (#5354)
Don't attach selfavatar in "v{c,g}-request" and "v{c,g}-auth-required" messages:
- These messages are deleted right after processing, so other devices won't see the avatar.
- It's also good for privacy because the contact isn't yet verified and these messages are auto-sent
  unlike usual unencrypted messages.
2024-03-28 21:29:05 -03:00
iequidoo
7960944b14 test: test_mvbox_sentbox_threads: Check that sentbox gets configured after setting sentbox_watch (#5105) 2024-03-22 19:08:05 -03:00
link2xt
4da0c19766 test: fixup tests/test_3_offline.py::TestOfflineAccountBasic::test_wrong_db 2024-03-03 03:28:47 +00:00
iequidoo
e16322d99d test: get_protected_chat(): Use FFIEventTracker instead of dc_wait_next_msgs() (#5207)
The way it was implemented it threw out all remaining messages after finding the next incoming
message. Better use FFIEventTracker functions, they are used in all the tests anyway.
2024-03-02 12:12:26 -03:00
iequidoo
b5f2c747e0 feat: Context::set_config(): Restart IO scheduler if needed (#5111)
Restart the IO scheduler if needed to make the new config value effective (for `MvboxMove,
OnlyFetchMvbox, SentboxWatch` currently). Also add `set_config_internal()` which doesn't affect
running the IO scheduler. The reason is that `Scheduler::start()` itself calls `set_config()`,
although not for the mentioned keys, but still, and also Rust complains about recursive async calls.
2024-02-12 15:41:11 -03:00
link2xt
2f29c56a36 fix: do not log error if watched folder is not configured
This may happen if Sent folder does not exist
but configuration option to watch it is enabled.
2024-01-31 14:04:03 +01:00
Simon Laux
1b7c5be9c5 Test server has no sentbox folder 2024-01-31 14:04:03 +01:00
iequidoo
06f1fe18d6 fix: Delete resent messages on receiver side (#5155)
If a Delta Chat message has the Message-ID already existing in the db, but a greater "Date", it's a
resent message that can be deleted. Messages having the same "Date" mustn't be deleted because they
can be already seen messages moved back to INBOX. Also don't delete messages having lesser "Date" to
avoid deleting both messages in a multi-device setting.
2024-01-30 22:01:45 -03:00
iequidoo
bd3759d55e test: test_import_export_online_all: Send the message to the existing address to avoid errors (#5220) 2024-01-29 20:34:03 -03:00
missytake
bbc6febb72 test: no timeout in SetupPlugin 2024-01-17 14:20:29 +01:00
iequidoo
c16c5e0802 test: Bring test_forward_encrypted_to_unencrypted into line with current API
Currently `Chat.send_msg()` modifies the source message and returns another message object
equivalent to the source one. That's how it works in the core and thus in Python bindings too.
2024-01-17 14:20:29 +01:00
missytake
36cab40ac1 test: add get_protected_chat to testplugin.py 2024-01-17 14:20:29 +01:00
missytake
4186d78305 test: add python test for message forwarding from encrypted to unencrypted chat 2024-01-17 14:20:29 +01:00
link2xt
828cc1fbd1 fix(connectivity): return false from all_work_done() immediately after connecting
We do not want all_work_done() to return true immediately
after calling start_io(), but only when connection goes idle.

"Connected" state is set immediately after connecting to the server,
but it does not mean there is nothing to do.

This change make all_work_done() return false
from the Connected state and introduces a new Idle
connectivity state that is only set before connection
actually goes idle. For idle state all_work_done() returns true.

From the user point of view both old Connected state
and new Idle state look the same.
2023-12-12 02:19:51 +00:00
iequidoo
1f336f89a6 feat: Sync Config::Displayname across devices (#4893)
We already synchronise status/footer when we see a self-sent message with a Chat-Version
header. Would be nice to do the same for display name.

But let's do it the same way as for `Config::{MdnsEnabled,ShowEmails}`. Otherwise, if we sync the
display name using the "From" header, smth like `Param::StatusTimestamp` is needed then to reject
outdated display names. Also this timestamp needs to be updated when `Config::Displayname` is set
locally. Also this wouldn't work if system time isn't synchronised on devices. Also using multiple
approaches to sync different config values would lead to more code and bugs while having almost no
value -- using "From" only saves some bytes and allows to sync some things w/o the synchronisation
itself to be enabled. But the latter also can be a downside -- if it's usual synchonisation, you can
(potentially) disable it and share the same email account across people in some organisation
allowing them to have different display names. With using "From" for synchronisation such a
capability definitely requires a new config option.
2023-12-01 21:41:58 -03:00
link2xt
ab151654fb chore: remove unused import from python tests 2023-11-18 11:32:34 +01:00
link2xt
ea9556b1b9 test: port test_aeap_flow_verified to JSON-RPC 2023-11-18 11:32:34 +01:00
link2xt
f39acbc037 test: port test_qr_new_group_unblocked() to JSON-RPC 2023-11-18 11:32:34 +01:00
link2xt
005f7ff07e test: port test_qr_join_chat_with_pending_bobstate_issue4894 to JSON-RPC 2023-11-18 11:32:34 +01:00
link2xt
144ca7c171 test: port test_qr_join_chat to JSON-RPC
Disabled `verified_one_on_one_chats` is not tested
as it is not interesting, other checks are moved.
2023-11-18 11:32:34 +01:00
link2xt
7012b99d73 test: remove test_qr_setup_contact from online python tests
There is an identical `test_qr_setup_contact` in deltachat-rpc-client
test suite which also checks that contact profiles get verified.
2023-11-18 11:32:34 +01:00