Commit Graph

443 Commits

Author SHA1 Message Date
link2xt
37dc1f5ca0 api!: deprecate DC_GCL_VERIFIED_ONLY 2025-05-20 16:14:43 +00:00
Hocuri
1379821b03 refactor: Move logins into SQL table (#6724)
Move all `configured_*` parameters into a new SQL table `transports`.
All `configured_*` parameters are deprecated; the only exception is
`configured_addr`, which is used to store the address of the primary
transport. Currently, there can only ever be one primary transport (i.e.
the `transports` table only ever has one row); this PR is not supposed
to change DC's behavior in any meaningful way.

This is a preparation for mt.

---------

Co-authored-by: l <link2xt@testrun.org>
2025-04-13 19:06:41 +02:00
link2xt
07946a18c3 test: use QR codes to setup contact with test bots 2025-04-06 09:53:56 +00:00
link2xt
0e874735ac test: encrypt legacy Python tests
This makes it possible to run Python tests
against chatmail servers that don't allow
any unencrypted messages.
2025-04-06 08:42:33 +00:00
link2xt
b56cf72c87 api: add legacy Python bindings for make_vcard and import_vcard 2025-04-06 07:42:34 +00:00
link2xt
51bbdadfad feat: ignore encryption preferences
Encryption preference is sent in Autocrypt header,
but otherwise ignored.

Delta Chat always prefers encryption if it is available.
2025-03-12 16:44:52 +00:00
link2xt
0913b6707b api!: remove save_mime_headers config option and dc_get_mime_headers()
This was only used in tests.
`msgs.mime_headers` coulmn remains
as it is used for HTML messages.
2025-03-06 21:12:18 +00:00
link2xt
8c2207d15e fix: make it impossible to overwrite default key
Replacing default key
when a profile is already part of
verified groups results in
`[The message was sent with non-verified encryption. See 'Info' for more details]`
messages for other users.

It is still possible
to import the default key before
Delta Chat generates the key.
2025-02-26 20:56:52 +00:00
Hocuri
a49dfeca6e refactor: Remove Message.set_file() / dc_msg_set_file() and related code (#6558)
Now that we are deduplicating everywhere, we can get rid of some code.

The old python bindings did not get an optional `name` parameter because
they are deprecated anyway, but it would be easy to add it.
2025-02-22 10:47:52 +01:00
Hocuri
65a9c4b79b File deduplication (#6332)
When receiving messages, blobs will be deduplicated with the new
function `create_and_deduplicate_from_bytes()`. For sending files, this
adds a new function `set_file_and_deduplicate()` instead of
deduplicating by default.

This is for
https://github.com/deltachat/deltachat-core-rust/issues/6265; read the
issue description there for more details.

TODO:
- [x] Set files as read-only
- [x] Don't do a write when the file is already identical
- [x] The first 32 chars or so of the 64-character hash are enough. I
calculated that if 10b people (i.e. all of humanity) use DC, and each of
them has 200k distinct blob files (I have 4k in my day-to-day account),
and we used 20 chars, then the expected value for the number of name
collisions would be ~0.0002 (and the probability that there is a least
one name collision is lower than that) [^1]. I added 12 more characters
to be on the super safe side, but this wouldn't be necessary and I could
also make it 20 instead of 32.
- Not 100% sure whether that's necessary at all - it would mainly be
necessary if we might hit a length limit on some file systems (the
blobdir is usually sth like
`accounts/2ff9fc096d2f46b6832b24a1ed99c0d6/dc.db-blobs` (53 chars), plus
64 chars for the filename would be 117).
- [x] "touch" the files to prevent them from being deleted
- [x] TODOs in the code

For later PRs:
- Replace `BlobObject::create(…)` with
`BlobObject::create_and_deduplicate(…)` in order to deduplicate
everytime core creates a file
- Modify JsonRPC to deduplicate blob files
- Possibly rename BlobObject.name to BlobObject.file in order to prevent
confusion (because `name` usually means "user-visible-name", not "name
of the file on disk").

[^1]: Calculated with both https://printfn.github.io/fend/ and
https://www.geogebra.org/calculator, both of which came to the same
result
([1](https://github.com/user-attachments/assets/bbb62550-3781-48b5-88b1-ba0e29c28c0d),

[2](https://github.com/user-attachments/assets/82171212-b797-4117-a39f-0e132eac7252))

---------

Co-authored-by: l <link2xt@testrun.org>
2025-01-21 19:42:19 +01:00
link2xt
72558af98c api!: remove dc_prepare_msg and dc_msg_is_increation 2024-12-11 19:34:29 +00:00
link2xt
e694411974 api!: remove dc_all_work_done()
Also cleaned up test_connectivity()
which tested that state does not flicker to WORKING
when there are no messages to be fetched.
The state is expected to flicker to WORKING
when checking for new messages,
so the tests were outdated since
change 3b0b2379b8
2024-12-04 14:31:55 +00:00
link2xt
191eb7efdd chore: fix typos
Applied fixes suggested by scripts/codespell.sh
2024-12-02 19:22:45 +00:00
Septias
c92554dc1f fix typo 2024-10-21 11:29:55 +02:00
link2xt
7743072411 refactor: remove addr from KeyPair 2024-09-16 17:00:16 +00:00
iequidoo
ff6488371c feat: Delete messages from a chatmail server immediately by default (#5805) (#5840)
I.e. treat `DeleteServerAfter == None` as "delete at once". But when a backup is exported, set
`DeleteServerAfter` to 0 so that the server decides when to delete messages, in order not to break
the multi-device case. Even if a backup is not aimed for deploying more devices, `DeleteServerAfter`
must be set to 0, otherwise the backup is half-useful because after a restoration the user wouldn't
see new messages deleted by the device after the backup was done. But if the user explicitly set
`DeleteServerAfter`, don't change it when exporting a backup. Anyway even for non-chatmail case the
app should warn the user before a backup export if they have `DeleteServerAfter` enabled.

Also do the same after a backup import. While this isn't reliable as we can crash in between, this
is a problem only for old backups, new backups already have `DeleteServerAfter` set if necessary.

---------

Co-authored-by: Hocuri <hocuri@gmx.de>
2024-09-08 16:53:56 -03:00
link2xt
e7d4ccffe2 feat: automatic reconfiguration 2024-08-19 16:36:56 +00:00
iequidoo
d40ec88b94 test(python): Wait for bot's DC_EVENT_IMAP_INBOX_IDLE before sending messages to it (#5699)
Bot processes are run asynchronously, so we shouldn't send messages to a bot before it's fully
initialised and skipped existing messages for processing, i.e. before DC_EVENT_IMAP_INBOX_IDLE is
emitted.
2024-06-23 01:46:40 -03:00
link2xt
b5e2ded47a Revert "test: Set fetch_existing_msgs for bots (#4976)"
This reverts commit 25b8a482bc.
2024-06-20 02:00:49 +00:00
iequidoo
3b91815240 test(python): Set delete_server_after=1 ("delete immediately") for bots (#4976)
Test bots are run with `fetch_existing_msgs` set, so messages must be deleted immediately not to be
processed again after a bot redeployment.
2024-06-16 22:16:33 -03:00
iequidoo
25b8a482bc test: Set fetch_existing_msgs for bots (#4976)
A bot process is run asynchronously, so some messages can arrive before the bot is fully
initialised.
2024-06-05 18:27:20 -03:00
link2xt
7cfab9a931 test: set configuration after configure() finishes
This allows to overwrite settings changed
when XCHATMAIL capability is detected.
2024-05-15 06:27:57 +00:00
iequidoo
99cd6d10da fix(python): _map_ffi_event: Always check if get_message_by_id() returned None
As trashed messages can't be loaded from the db now, `get_message_by_id()` returns None in some
tests e.g. in `test_deleted_msgs_dont_reappear()`. A `PerAccount` hook shouldn't be called if so.
2024-04-20 05:33:56 +00:00
iequidoo
fae4cb33bc fix(python): Message::is_outgoing: Don't reload message from db
There's no need to load an updated message state from the db to implement `is_outgoing()` and also
this function is implicitly called in some tests where a message is already trashed and a call to
`dc_get_msg()` generates an unexpected error.
2024-04-20 05:33:56 +00:00
iequidoo
7a3be74350 fix(python): Use cached message object if dc_get_msg() returns NULL
Particularly, in _msgstate() and download_state(). If the message has been trashed, dc_get_msg()
returns NULL now.
2024-04-20 05:33:56 +00:00
holger krekel
acff8205e2 test: fix pytest compat (#5317)
seems pytest_report_header has changed with pytest incompatible but we
don't need it anyway so we can just leave it out.
2024-03-04 21:07:09 +00:00
iequidoo
e16322d99d test: get_protected_chat(): Use FFIEventTracker instead of dc_wait_next_msgs() (#5207)
The way it was implemented it threw out all remaining messages after finding the next incoming
message. Better use FFIEventTracker functions, they are used in all the tests anyway.
2024-03-02 12:12:26 -03:00
iequidoo
06f1fe18d6 fix: Delete resent messages on receiver side (#5155)
If a Delta Chat message has the Message-ID already existing in the db, but a greater "Date", it's a
resent message that can be deleted. Messages having the same "Date" mustn't be deleted because they
can be already seen messages moved back to INBOX. Also don't delete messages having lesser "Date" to
avoid deleting both messages in a multi-device setting.
2024-01-30 22:01:45 -03:00
iequidoo
987bdaf237 ci: Add/remove necessary newlines to fix Python lint 2024-01-26 14:46:46 -03:00
missytake
bbc6febb72 test: no timeout in SetupPlugin 2024-01-17 14:20:29 +01:00
missytake
7f7f42d721 test: Ensure that member is added before yielding chat 2024-01-17 14:20:29 +01:00
missytake
36cab40ac1 test: add get_protected_chat to testplugin.py 2024-01-17 14:20:29 +01:00
holger krekel
9ca049051c fix and streamline a little 2023-11-10 16:46:40 +01:00
Sebastian Klähn
790509676f fix: remove unused --liveconfig option
close  #4892
2023-11-10 16:46:40 +01:00
link2xt
4e21917c0e feat: enable sync messages by default 2023-11-05 21:11:47 +00:00
holger krekel
71b7b0b393 fix doc strings for qr-code joinings 2023-11-02 13:31:35 +01:00
link2xt
6aae0276da test: use instant accounts instead of mailadm 2023-10-30 00:15:22 +00:00
missytake
5a5f8b03d1 fix(python): don't automatically set the displayname to 'bot' when setting log level 2023-10-08 17:38:57 +00:00
link2xt
532e9cb09a refactor: ignore public key argument in dc_preconfigure_keypair()
Public key can be extracted from the secret key file.
2023-09-30 19:16:23 +00:00
link2xt
ef4d2a7ed0 api!(python): use dc_contact_get_verifier_id()
get_verifier() returns a Contact rather than an address now

dc_contact_get_verifier_addr() is unused.
2023-09-30 15:49:22 +00:00
link2xt
d727d85f6d chore(python): fix ruff 0.0.286 warnings 2023-08-25 20:57:44 +00:00
link2xt
81a7af10c7 chore(python): fix lint errors 2023-08-25 01:14:24 +00:00
link2xt
fbd2fc8ead test: add test_webxdc_download_on_demand
The test checks that if webxdc update is too large to
download with the current `download_limit`,
it is applied afterwards when the user manually downloads the update message.
2023-07-21 21:23:44 +00:00
link2xt
740f43a2d6 fix: do not resync IMAP after initial configuration
If there was no previous `configured_addr`,
then there is no need to run IMAP resync.
2023-07-17 16:38:09 +00:00
link2xt
88759c815b refactor(python): flatten the API of deltachat module 2023-07-03 16:19:31 +00:00
link2xt
437f8c48c4 api(python): make Contact.is_verified() return bool 2023-06-07 19:50:42 +00:00
link2xt
b2993242e4 docs(python): document pytest fixtures
These docstrings are displayed by `pytest --fixtures`
when `deltachat` package is installed from PyPI.
2023-06-05 18:34:27 +00:00
link2xt
fa87d2e225 New APIs for message processing loops
This patch adds new C APIs
dc_get_next_msgs() and dc_wait_next_msgs(),
and their JSON-RPC counterparts
get_next_msgs() and wait_next_msgs().

New configuration "last_msg_id"
tracks the last message ID processed by the bot.
get_next_msgs() returns message IDs above
the "last_msg_id".
wait_next_msgs() waits for new message notification
and calls get_next_msgs().
wait_next_msgs() can be used to build
a separate message processing loop
independent of the event loop.

Async Python API get_fresh_messages_in_arrival_order()
is deprecated in favor of get_next_messages().

Introduced Python APIs:
- Account.wait_next_incoming_message()
- Message.is_from_self()
- Message.is_from_device()

Introduced Rust APIs:
- Context.set_config_u32()
- Context.get_config_u32()
2023-04-15 21:27:45 +00:00
iequidoo
0a2200c2c8 python: Wait for initial Resync in the bot tests 2023-03-31 17:15:02 -03:00
link2xt
5db867cd1b Add IMAP_INBOX_IDLE event 2023-03-22 20:20:37 +00:00