Commit Graph
100 Commits
Author SHA1 Message Date
holger krekel 70a01a6813 feat!: remove a relay immediately instead of unpublishing it
Removing a relay now takes effect immediately:

- the profile stops fetching and advertising it,

- secondary devices immediately apply the removal through the transport sync,

Upgrading removes unpublished relays and triggers keyupdates.

BREAKING CHANGE: set_transport_unpublished() is removed: UIs call delete_transport() when the user removes a relay.

BREAKING CHANGE: list_transports_ex() and the TransportListEntry type are removed: use list_transports().

BREAKING CHANGE: delete_transport() no longer refuses to remove the primary transport: it refuses only to remove the last one and re-elects the sending transport as needed.

BREAKING CHANGE: TransportsModified is now also emitted on the device modifying the transports, not only on devices applying the synced change.

Deprecated: DC_STR_PHASING_OUT
2026-09-01 14:15:26 +02:00
holger krekel 3d61e0f349 fix: return no relay address for key-contacts without an address
`relay_addrs()` fell back to the contact address even when it is empty,
which happens for key-contacts created from a sync message or for the
self-contact, putting an empty string into the SMTP recipient list.
2026-08-29 23:11:05 +02:00
holger krekel f162749dfe feat: introduce keyupdate messages informing contacts about relay changes
When the published relay list changes, key-contacts are informed with an
unsigned message carrying the re-signed key, encrypted to a chunk of contacts
at a time. It is shaped like a receipt notification naming no message, so that
cores which know nothing about keyupdates trash it as well.

See the src/keyupdate.rs module docs for the design.
2026-08-29 23:11:05 +02:00
holger krekel 370cc5c1fd fix: trash early MDNs that reference no message
A report referencing no message can never be applied to one,
so it must not create a contact, a chat or a `last_seen` update
on its way to the trash.
2026-08-29 23:11:05 +02:00
holger krekel 08b2374fcf feat: allow to not sign asymmetrically encrypted multi-recipient messages
An unsigned message carries no intended recipient fingerprints,
so recipients of an encrypted unsigned message
learn nothing about other recipients from the PGP packets.
2026-08-29 23:11:05 +02:00
holger krekel 5176a9c633 refactor: extract shared pieces for non-chat messages
No functional changes:
Add a relay_addrs helper, share the protected headers and self-key rendering
of non-chat messages, and move insert_into_smtp from securejoin to smtp.
2026-08-29 23:11:05 +02:00
holger krekel a344cab046 refactor: rename automatic_relay_management to autorelay
"automatic_relay_management" is a long unwiedly name with no UI using the mode yet,
so let's rename it to something more succinct: autorelay
2026-08-27 16:52:38 +02:00
holger krekel 96f051a2f0 fix: produce correct wheel metadata
all published deltachat-rpc-server wheels so far fail "wheel tags"
and probably other tools.
Translate "dev" cargo-versioning to PEP440-versioning
to make CI dev wheel builds reproducable at least for the 11 non-mac targets.
2026-08-27 16:27:18 +02:00
holger krekel ea3ca3a213 fix: reliably complete configuration with progress=1000 or progress=0
this is meant to help configuration event consumers (Python, UIs)
to not hang waiting for configuration outcomes.
One test case is added that fails on main.
2026-08-24 19:09:34 +02:00
holger krekel c9880d0ba6 chore: start checking column documentation in CI and add comment for "transports.add_timestamp" 2026-08-22 18:07:16 +02:00
holger krekel 8f3337d971 refactor: move pgp tests to submodule 2026-08-19 02:00:14 +02:00
holger krekel 307ff9def1 fix(rpc): avoid hang when requests race a dying rpc-server
Requests now register before testing for shutdown,
so either the reader loop or the caller answers them.
Also failed start() winds down its threads, ignoring a broken pipe on stdin.
2026-08-17 21:07:19 +02:00
holger krekel 0119db85ad test!: rename rpc fixtures to disambiguate from ffi fixtures
fixes https://github.com/chatmail/core/issues/8583
2026-08-14 16:44:50 +02:00
holger krekel ce7c8dbca1 fix: send http requests in origin not absolute form
Absolute form is meant for proxies and for example
nginx rejects it with 400 if the host contains an underscore, breaking autoconfig discovery.
2026-08-14 12:09:24 +02:00
holger krekel 0f8c91570b feat: allow to run the test suite against underscore-domain relays
Such relays serve self-signed certificates, and are created e.g. by cmlxc deploys.
2026-08-14 12:09:24 +02:00
holger krekel feb777f704 test: load test data through the data fixture
Relative paths only worked when running from `deltachat-rpc-client`.
2026-08-14 12:09:24 +02:00
holger krekel 28a1a1da96 chore: don't swallow but log errors in three places
helps with debugging failing (flaky) tests
2026-08-14 12:04:43 +02:00
holger krekel 7add59fa76 test: abort early if DNS to chatmail domain does not work and nicer pytest startup header 2026-08-14 11:19:12 +02:00
holger krekel 56a99c227a test: add a CI-failing check that documented sql schema matches real one
finds things like "exists in the db, but is not documented" or "documented but not in the db".
2026-08-13 12:07:15 +00:00
holger krekel 3363f63316 test: remove all cache-related logic in the FFI pytest plugin 2026-08-12 22:04:15 +02:00
holger krekel 6beced4f51 test: avoid another source of random failures with direct_imap failing to connect on first try
also core src/imap.rs retries connecting (2s with backoff) but direct_imap is only used in (CI) tests
and we can just keep trying every second until pytest timeout hits.
2026-08-12 22:04:15 +02:00
holger krekel cc229372e4 test: provide complete test isolation by not re-using account addresses
As we are using a dedicated CI relay instance, we don't need to be careful with creating test addresses.
But re-using test addresses in a test running in a single worker can leak events/messages which compromises test isolation.
Instead of encountering random flaky tests because let's just not cache addresses to maximize test isolation.
2026-08-12 22:04:15 +02:00
holger krekel 5ec1d65294 test: move iroh tests into separate module 2026-08-11 18:08:58 +02:00
holger krekel 8df4fe1c17 test: fix flakyness of iroh tests by sending "forever" so that late swarm-joins still make the test work
previously only 10 messages were sent but if joinining did not succeed (due to slow CI sometimes) nothing would be received and the test would hang.

Also cleans up the tests to not set webxdc realtime config (true by default) and restructure helpers to reduce redundant code.
2026-08-11 13:13:03 +02:00
holger krekel cd42efb36d fix!: keep primary transport device-local
Devices no longer implicitely use the From address of sync messages
to determine their primary transport. Receivers have no concept of it and own devices
may disagree on which relay is reachable because of VPN or different networks:

1. Make setting a primary transport (`configured_addr`) a per-device non-synced operation.

2. Transport rows (add/remove/unpublish) keep syncing like before.

3. A device reelects a primary if a sync message unpublished/removed the current primary
   if there is a better candidate.

4. `TransportsModified` event is emitted at most once on an incoming transport sync message.

Users will notice the change in that changing primary transport in settings/advanced/relays
will not synchronize to other devices anymore.
2026-08-10 14:43:28 +02:00
holger krekel 2cacdbfd4b fix: prevent transport de-synchronization because of early fetch cancellation
Came across this while investigating more test_transport_synchronization flakiness,
sometimes missing TransportsModified events or getting a missing configured_addr.
The underlying problem was that stopping IO was triggered immediately during
receiving sync messages, potentially *canceling* the processing of the sync message,
effectively de-syncing the device's view on transports.
2026-08-10 14:20:59 +02:00
holger krekel 67437c946b test: wait for second TRANSPORTS_MODIFIED before asserting configured_addr 2026-08-10 14:20:59 +02:00
holger krekel 6934aa26ae fix: invalidate configured_addr cache before sending transport sync message
Uncaching only after send_sync_transports() could send the sync message
from the old primary address, so other devices never switched.
2026-08-10 14:20:59 +02:00
holger krekel c3bc26d52f fix: ensure same-second primary transport change propagates correctly
Without the fix, too fast transport changes are ignored at the receiver side.
2026-08-10 14:20:59 +02:00
holger krekel 15387690c8 follow link2xt review comments 2026-08-04 11:47:26 +02:00
holger krekel ba35814d34 fix: recognize self addresses in various places (instead of just the "primary") 2026-08-04 11:47:26 +02:00
holger krekel 9476f9ec7f test: fix flaky test_no_markseen_in_team_profile (#8500)
fixes https://github.com/chatmail/core/issues/8446
2026-07-31 09:12:28 +00:00
holger krekel 020e477662 test: fix flaky test_markseen_message_and_mdn test 2026-07-31 00:20:14 +02:00
holger krekel 35555ca753 api!: remove getPushState() and core's internal tracking of it.
Since https://github.com/deltachat/deltachat-ios/pull/3224 pushstate is not used
(android, desktop etc. never used it, only ios)
2026-07-30 17:32:54 +02:00
holger krekel 0bb3d88bc6 feat: stop requiring XDELTAPUSH capability for push notifications
register device token if XDELTAPUSH IMAP capability is available (current relay setup provides it)
or if "maxsmtprecipients" IMAP metadata key is available (relays since May 2026 provide it),
allowing chatmail relay setups to drop XDELTAPUSH capability marker with the next release
while retaining push notification support.
2026-07-30 17:32:54 +02:00
holger krekel 00e1d00dfa fix: send MDNs to all authentic relays of a contact, not just whatever get_addr() returns. 2026-07-27 21:54:42 +02:00
holger krekel 856ea29c47 feat: reduce unncessary gossipping of keys in group chats
- treat an Autocrypt header as a kind of Autocrypt-Gossip: addr=<self>
  letting it participate in the existing cooperative key gossipping mechanics.

- speed up processing of incoming autocrypt-gossip headers by using
  1 sql commit for N gossip headers instead of the current N commits.
2026-07-27 21:54:42 +02:00
holger krekel 6a07a2b222 feat: send Autocrypt pgp key in MDNs occassionally and when relaylist changes 2026-07-27 21:54:42 +02:00
holger krekel 411ad16ea9 fix(deltachat-rpc-client): don't depend on execnet for importing pytest plugin, remove deprecated "py" usage
unsuspecting consumers will otherwise require execnet to run pytest with deltachat-rpc-client installed.
"import py" is discouraged for pytest these days but still supported in compatibility mode.
2026-07-27 21:45:34 +02:00
holger krekel 7e5c1e8389 chore: bump version to 2.58.0-dev 2026-07-25 16:03:53 +02:00
holger krekel 758fb9daa0 chore(release): prepare for 2.57.0 2026-07-25 15:47:44 +02:00
holger krekel cd4850668f fix: migrate transports configured on 2.56 to also have a oauth:false flag
also fix test setup to include the oauth:false flag
2026-07-25 15:38:29 +02:00
holger krekel 7ec888f48a remove hermes/aco-connexion special casing and simplify/address link2xt comments 2026-07-24 13:22:13 +02:00
holger krekel eccd5ce03a address link2xt comments 2026-07-24 13:22:13 +02:00
holger krekel 3c171d539b fixup CI failures 2026-07-24 13:22:13 +02:00
holger krekel 0a71e181e4 add nauta.cu, hermes.radio and *.aco-connexion.org legacy options 2026-07-24 13:22:13 +02:00
holger krekel 1ac1917172 api!: remove provider-db handling and provider lookup APIs
BREAKING CHANGE: provider lookup APIs were removed from CFFI and JSON-RPC.

also removes offline provider database code and generated provider data,
provider-specific fields in configure/transport paths, and REPL providerinfo.
2026-07-24 13:22:13 +02:00
holger krekel 461a328915 api!: remove heartbeat push notifications
Heartbeat notifications are only used on iOS for classic mail servers
but both code and privacy wise (notification server sees IP addresses from those users)
not something we want to support any longer (was discussed with bjoern).
2026-07-24 00:04:14 +02:00
holger krekel 0e4574cd41 feat: read SMTP recipient limit from relay IMAP metadata
Remove is_chatmail flag, and rely on IMAP metadata
advertising the recipients limit (usually 1000),
falling back to 50 (or fewer in some exceptional cases)
just as before when is_chatmail was false.

Also, server metadata is now keyed per transport,
which in the future eases collecting ICE servers
from all relays (instead of just any first relay).
2026-07-22 20:31:50 +02:00
holger krekel ca1fe8e352 chore(release): prepare for 2.56.0 2026-07-21 19:17:51 +02:00
holger krekel 61898b4789 fix: revert 207c2e6e4c because some users reported problems with it 2026-07-21 18:55:22 +02:00
holger krekel 5c1e695237 api!: remove all oauth support and drop DC_LP_AUTH flags
BREAKING CHANGE: removed oauth2 module, dc_get_oauth2_url FFI function, DC_LP_AUTH flags and configured/serverflags, and the oauth2 parameter/field from SMTP/IMAP clients, JSON-RPC interfaces, and CLI tools.

also contains regenerated provider data after dropping oauth in the update script.
2026-07-21 15:55:28 +02:00
holger krekel 398ddac0b9 chore: bump version to 2.56.0-dev 2026-07-21 12:17:59 +02:00
holger krekel 6cd5b21a26 fix: don't send or process webxdc status updates in pre-messages
it makes no sense to send or receive status updates in pre-messages for large webxdc attachments because they can't be processed anyway.
2026-06-12 17:01:46 +02:00
holger krekel c26b6a017c fix: syntax error in older migration 2026-05-27 06:20:50 +00:00
holger krekel 60bc4011f7 fix: let search also return hidden contacts if search value is an email address 2026-04-07 22:39:13 +02:00
holger krekel 28cce5e31d fix: determine whether a message is an own message by looking at signature. multiple devices can temporarly have different sets of self addresses, and still need to properly recognize incoming versus outgoing messages. Disclaimer: some LLM tooling was initially involved but i went over everything by hand, and also addressed review comments. 2026-04-01 14:51:48 +02:00
holger krekel cb783ffc12 feat(rpc): add startup health-check and propagate server errors
Rpc.start() now calls get_system_info() after launching the server
to verify it started successfully. If the server exits early (e.g.
due to an invalid accounts directory), the core error message from
stderr is captured and included in the raised JsonRpcError.

The reader_loop now unblocks pending RPC requests when the server
closes stdout, so callers never hang on a dead server.

Export JsonRpcError from the deltachat_rpc_client package.

Add test_early_failure verifying that Rpc.start() raises with
the actual core error message for invalid accounts directories.
2026-02-26 18:25:31 +01:00
holger krekel f1e90c73cd chore: add dev-version bump instructions to RELEASE.md (bumping to 2.44.0-dev) 2026-02-25 10:30:42 +01:00
holger krekel c39d2f42ef fix: tolerate empty existing directory in Accounts::new() (#7886) 2026-02-24 09:22:19 +01:00
holger krekel d75a78d446 feat: introduce cross-core testing along with improvements to test frameworking 2025-12-04 14:29:16 +00:00
holger krekel b9ae74fab2 feat: deltachat_rpc_client.Rpc accepts rpc_server_path for using a particular deltachat-rpc-server (#7493)
also simplifies and make more readable popen-call to rpc-server 

addresses #7428
2025-11-25 11:02:42 +01:00
holger krekel 8eef79f95d refactor: strike events in rpc-client request handling, get result from queue 2025-11-23 21:23:53 +01:00
holger krekel 0e84cfd8ad docs: reference chatmail in the README 2025-03-21 10:42:15 +00:00
holger krekelandlink2xt f4dfc79808 test(deltachat-rpc-client): add realtime channel tests
Co-authored-by: link2xt <link2xt@testrun.org>
2024-05-25 05:46:55 +00:00
holger krekel 5d3d5d23a1 api(deltachat-rpc-client): add EventType.WEBXDC_REALTIME_DATA constant 2024-05-21 22:10:15 +00:00
holger krekel acff8205e2 test: fix pytest compat (#5317)
seems pytest_report_header has changed with pytest incompatible but we
don't need it anyway so we can just leave it out.
2024-03-04 21:07:09 +00:00
holger krekel 6345a4f5b3 fix link for securejoin 2024-01-13 12:50:16 +01:00
holger krekel 7c4c980409 for testrun.org subdomains we can allow 60 messages per minute (or lift the limit completely but maybe good to protect against wild-running bots or so) 2023-11-17 06:50:19 +00:00
holger krekel 9ca049051c fix and streamline a little 2023-11-10 16:46:40 +01:00
holger krekel 71b7b0b393 fix doc strings for qr-code joinings 2023-11-02 13:31:35 +01:00
holger krekel bd02eea66b refactor: remove unused or useless code paths in securejoin (#4897) 2023-11-02 12:29:48 +00:00
holger krekel 196a34684d simplify "broken bobstate" as the state still passes 2023-11-01 00:58:54 +01:00
holger krekel 402fd6850c do the actual fix (thanks alex for some help) 2023-11-01 00:58:54 +01:00
holger krekel 72515f440d write a failing test 2023-11-01 00:58:54 +01:00
holger krekel bfddd3fc69 fix typo 2022-12-07 09:44:54 +01:00
holger krekel cffa101419 remove usage of py in favor of pathlib 2022-05-09 21:22:19 +02:00
holger krekel 1d7d201b5b remove a log line and a return 2022-05-09 21:22:19 +02:00
holger krekel c0e4e12138 Introduce caching of configured live accounts in each test process 2022-05-09 21:22:19 +02:00
holger krekel c8bfa98b6b actually enable online account caching. previously it was creating (>100) online test accounts 2022-05-05 16:18:59 +02:00
holger krekel 32a9db6922 fix flaky test to not rely on timing 2022-05-04 13:29:08 +02:00
holger krekel 77498c17a5 reorder tests to provide a complex/offline/online distinction, also speeding up distributed test runs. 2022-05-04 13:17:08 +02:00
holger krekel 54c07f89f2 also guard the ac_process_ffi_event calling 2022-05-04 12:08:47 +02:00
holger krekel cd4d265055 safer handling of calling account hooks, refined shutdown comment 2022-05-04 12:08:47 +02:00
holger krekel 81ee69010d add a comment, fix a typo in debug message 2022-05-04 12:08:47 +02:00
holger krekel 138d5b7a02 fix segfaults with python runs -- i don't get them anymore, and can also control-c at will 2022-05-04 12:08:47 +02:00
holger krekel ca59cbc898 make direct_imap idle usage safe from tests, so we never miss finishing direct_imap idle 2022-05-03 22:56:23 +02:00
holger krekel 21f72ad845 new --debug-setup option to show events during account setup. 2022-05-03 22:50:16 +02:00
holger krekel 40fd98d580 show logs for pseudo configured accounts (for online ones they are instantiated later) 2022-05-03 22:50:16 +02:00
holger krekel 0422d751d8 avoid instatiating a full blown Account object just for showing some core info 2022-05-03 19:35:03 +02:00
holger krekel e44f68db83 prevent one strange traceback probably caused by ongoing shutdown 2022-05-03 19:35:03 +02:00
holger krekel 88cbf42c39 fix various tests 2022-05-03 19:35:03 +02:00
holger krekel 45157f79a3 fix examples 2022-05-03 19:35:03 +02:00
holger krekel f5157392b6 some renaming and some docstrings 2022-05-03 19:35:03 +02:00
holger krekel f631ec3a7c shift startio/init machinery into PendingConfigure class 2022-05-03 19:35:03 +02:00
holger krekel a9b2750ec9 slight renamings 2022-05-03 19:35:03 +02:00
holger krekel 7fbdb42695 remove _configtracker and write a tested "PendingConfigure" helper class for managing configure/started states for test accounts 2022-05-03 19:35:03 +02:00
holger krekel 026dd4480e add --extra-info CLI option (defaults to False for interactive runs) 2022-05-03 19:35:03 +02:00
holger krekel b373e7acba more renames 2022-05-03 19:35:03 +02:00
holger krekel d2ca54c167 provide for meaningful names for bringing accounts online for test functions 2022-05-03 19:35:03 +02:00