mirror of
https://github.com/chatmail/core.git
synced 2026-09-20 12:08:50 +03:00
Compare commits
4 Commits
v2.60.0
...
hpk/draft-
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
0e72fcbc47 | ||
|
|
c69c6fa4ec | ||
|
|
85fd53add2 | ||
|
|
4116d7579b |
361
draft/keyupdate-draft.md
Normal file
361
draft/keyupdate-draft.md
Normal file
@@ -0,0 +1,361 @@
|
||||
# A keyupdate push channel to maintain reliable chat connectivity
|
||||
|
||||
[Multi-relay support for chat profiles was added in March 2026][multi-relay],
|
||||
allowing them to use multiple relays for receiving and sending messages.
|
||||
While instant onboarding is being extended to multi-relay onboarding ([#8444]),
|
||||
adding and removing relays automatically is not settled,
|
||||
not least because changing a relay is unsafe today.
|
||||
This draft proposes a *keyupdate push channel*
|
||||
that shares our current key with our contacts when it changes,
|
||||
without waiting for a chat interaction.
|
||||
It helps keep chats connected now,
|
||||
and makes automatic relay changes safe enough to design later.
|
||||
The draft stays with the concept:
|
||||
how such a message is encrypted and addressed is left to the two
|
||||
implementations summarized in [Two ways to send a keyupdate](#two-ways-to-send-a-keyupdate).
|
||||
|
||||
|
||||
## Problems of maintaining reliable chat connectivity today
|
||||
|
||||
A profile's relay list lives inside its own key, as a signed notation that travels with the key.
|
||||
A contact sends to the addresses contained in the key, signed by the key holder.
|
||||
Rooted in the [Autocrypt 1](https://autocrypt.org) inline key-distribution specification,
|
||||
there is no central directory, no probe and no removal notice,
|
||||
so a contact's copy is only refreshed by traffic:
|
||||
a message we send them, or a group message from someone else
|
||||
that gossips our key onwards.
|
||||
|
||||
Reachability therefore decays exactly where traffic is thin.
|
||||
Three situations turn that decay into a cut conversation,
|
||||
or into costs of preventing one:
|
||||
|
||||
- **Mutual silence.**
|
||||
If neither side has written since our relays changed,
|
||||
the contact keeps the old list indefinitely.
|
||||
Their next message goes to a relay we no longer read,
|
||||
and they get silence rather than a bounce because the old account still exists.
|
||||
Even if a bounce comes, it's hard to effectively act on it for users.
|
||||
[#7878] is a user report of exactly this, and the reporter's objection
|
||||
is the design point:
|
||||
"people changing relays shouldn't need to know to write in every room
|
||||
they're in just to make sure everyone gets the update".
|
||||
It was closed expecting [#7865] to fix it, and [#7865] did land:
|
||||
the relay list is now carried in the key.
|
||||
That settled what gets distributed, not when,
|
||||
which is the part still missing.
|
||||
|
||||
- **Deliberate relay removal.**
|
||||
Users remove relays for good reasons:
|
||||
privacy, distrust of a provider, or wanting to use relays they chose.
|
||||
Here the cut would be immediate and total:
|
||||
every contact still on the old list would write to an address nobody reads.
|
||||
It does not happen today only because core doesn't really remove a relay,
|
||||
preventing lost messages at the costs of the "unpublished relay" workaround below.
|
||||
|
||||
- **Adding relays during degradation.**
|
||||
Relays sometimes get added precisely when the existing ones stop working,
|
||||
so the announcement would have to travel
|
||||
at the moment reachability is already impaired.
|
||||
A contact who does not learn the new relay
|
||||
keeps writing to an address that may already be dead.
|
||||
|
||||
|
||||
What exists today only narrows the gap.
|
||||
MDN key refresh ([#8481]) covers contacts who read our messages without
|
||||
replying, so the exposed population is mutual silence
|
||||
rather than all quiet contacts.
|
||||
Unpublished relays buy time, and only for removal.
|
||||
|
||||
|
||||
### The "unpublished relay" workaround is its own problem
|
||||
|
||||
"Remove" today marks a relay *unpublished*: the address stops being advertised,
|
||||
but the relay keeps a full IMAP connection and keeps receiving.
|
||||
This is deliberate ([#8384]) and it does buy safety for the cases above.
|
||||
The price is that users cannot end their relationship with a relay:
|
||||
|
||||
- **Retention is unpredictable in both directions.**
|
||||
Cleanup requires 90 days both since the removal itself
|
||||
and since the last message received there,
|
||||
and every fetched message resets the second clock,
|
||||
with no filter for blocked contacts or spam,
|
||||
so a removed relay that keeps getting mail is kept indefinitely.
|
||||
Adding a relay at the five-relay cap does the opposite
|
||||
and hard-deletes the least recently used unpublished one at once.
|
||||
|
||||
- **A relay the user removed keeps being used.**
|
||||
Unpublished transports get an IMAP loop like any other
|
||||
and count against the same cap,
|
||||
so a profile can hold connections to five relays,
|
||||
four of them ones the user pressed "Remove" on.
|
||||
Chatmail relays only delete accounts after a period without any login,
|
||||
and core keeps logging in,
|
||||
so the account there outlives the removal as well.
|
||||
|
||||
- **Nothing the user can do shortens it.**
|
||||
Real deletion is documented as internal and test-only.
|
||||
Users are told at removal time that the relay is phased out,
|
||||
but not for how long,
|
||||
and not that we keep connecting and receiving there.
|
||||
[#8384] records the friction from a user: a removed relay still advertising
|
||||
TURN servers, and a permanent connection error with "no way to get rid of
|
||||
it", patched as a symptom in [#8550] while the relay itself stayed.
|
||||
|
||||
This grows with multi-relay setups, and multi-relay onboarding ([#8444]) brings them.
|
||||
The [privacy notes being drafted for the apps][privacy notes]
|
||||
would somehow need to describe this behaviour, and the description would read badly.
|
||||
|
||||
Unpublished relays exist to protect exactly the contacts
|
||||
the new keyupdate push channel would reach,
|
||||
so the keyupdate push channel would let us drop the concept completely.
|
||||
|
||||
|
||||
## Moving on from Autocrypt1: A keyupdate push channel
|
||||
|
||||
We should be able to hand our current key to our contacts
|
||||
without waiting for a conversation to happen.
|
||||
A keyupdate is a message that does exactly that:
|
||||
it carries our current key with its signed relay list
|
||||
to the contacts who still hold an older copy,
|
||||
sent when that list changes rather than when a chat happens.
|
||||
|
||||
Whichever way such a message is sent, the same things follow from it:
|
||||
|
||||
- **Contacts acquired at any point in the past are reached**,
|
||||
and not only the ones we happen to write to,
|
||||
given they have updated to a post-keyupdate app release.
|
||||
|
||||
- **Delivery does not have to be complete to be useful.**
|
||||
In group chats, cooperative Autocrypt gossip spreads what arrived:
|
||||
members who received the fresher key pass it on to the others,
|
||||
whenever one of them next writes to the group.
|
||||
|
||||
- **"Remove" can mean actually removed.**
|
||||
Pushing keyupdates removes the last reason to keep "unpublished relays" around,
|
||||
so removing a relay can do what users intend it to do.
|
||||
|
||||
- **Automatic relay management becomes easier to design.**
|
||||
What ships under that name today is initial onboarding only
|
||||
([#8444], still off by default): no rotation, no removal.
|
||||
Designing those proved to be hard while changing a relay is unsafe,
|
||||
because any automatic change would silently cut off
|
||||
the contacts who do not hear about it.
|
||||
Keyupdates would lift that constraint and could land in the next release,
|
||||
well before automatic addition/removal mechanics are settled.
|
||||
|
||||
|
||||
## Constraints of keyupdate messages, metadata and processing
|
||||
|
||||
- **A keyupdate carries the key and nothing else.**
|
||||
The keyupdate push payload is "contacts-public":
|
||||
it is the key and its relay list,
|
||||
the same data we hand out in every chat where we participate.
|
||||
But a keyupdate is not a conversation,
|
||||
and it travels automatically to an audience nobody picks per message,
|
||||
so **the keyupdate channel must never carry user generated data
|
||||
or metadata besides the public key itself and it must not reveal
|
||||
to contacts the identity of other contacts.**
|
||||
|
||||
- **It is end-to-end encrypted.**
|
||||
In chatmail clients "public" keys are by default hidden identities,
|
||||
only transmitted in encrypted messages, with the goal of preventing
|
||||
a curious or abusive relay operator to track identities.
|
||||
|
||||
- **It goes to contacts, not to subscribers, and not to our own devices.**
|
||||
The recipients are unblocked key-contacts we share an accepted 1:1,
|
||||
group or subscribed-channel chat with.
|
||||
Subscribers of our own channels should be left out,
|
||||
because they could be a big number and it doesn't contribute to better chat connectivity.
|
||||
Our own devices learn relay changes through regular multi-device sync,
|
||||
because changing transports involves private credentials
|
||||
for accessing a relay address,
|
||||
and because we perform some merging on concurrent transport additions.
|
||||
|
||||
- **It is accepted from any unblocked key-contact.**
|
||||
The relay list lives in a direct key self-signature,
|
||||
and certificate merging verifies it and prefers the newest one,
|
||||
so merging, and not the way a keyupdate arrived,
|
||||
is the cryptographic gate for every certificate we get.
|
||||
A stale relay list is worth updating in any case,
|
||||
whatever chat state we have with that contact.
|
||||
|
||||
- **Replay changes nothing.**
|
||||
An old update can be replayed forever,
|
||||
but certificate merging keeps the direct key signature
|
||||
with the newest creation time, on a tie the one already stored,
|
||||
so a replay can not revert a relay list.
|
||||
The rest of the certificate is fixed today:
|
||||
a key carries one encryption subkey that never rotates.
|
||||
Rotating Autocrypt 2 subkeys might need a fresh look at this.
|
||||
|
||||
- **It is invisible on arrival.**
|
||||
The key should be applied on the normal Autocrypt path
|
||||
and the message then trashed rather than filed:
|
||||
no chat, no counter, and no refresh of the sender's "last seen",
|
||||
because an invisible message should not light up an online dot.
|
||||
|
||||
- **It stays quiet on clients that do not know it yet.**
|
||||
No new chat and no contact request should ever be created there.
|
||||
Old and new clients can therefore ship together,
|
||||
with coverage growing as clients update.
|
||||
|
||||
|
||||
## When and how a keyupdate goes out
|
||||
|
||||
Sending should be driven by a diff, not by an event.
|
||||
A device records the relay list it last announced,
|
||||
and a keyupdate is due only when the current relay list differs from it.
|
||||
That single decision gives us the rest:
|
||||
|
||||
- **Debounced.**
|
||||
Changes within a short window should go out as one message,
|
||||
so an automatic relay add/remove yields one keyupdate,
|
||||
and an add followed by a removal of the same yields none.
|
||||
|
||||
- **Short debouncing for user observability**:
|
||||
Around 30 seconds may be a good enough debounce window to start with.
|
||||
It is long enough that several changes usually travel in one keyupdate
|
||||
(current initial onboarding adds up to three relays sequentially),
|
||||
and short enough that someone changing relays by hand
|
||||
can still watch the effect arrive with chat peers,
|
||||
which is worth more than optimising against a few small extra messages.
|
||||
|
||||
- **Nothing on upgrade.**
|
||||
Existing profiles must start with their current relay list
|
||||
already recorded as announced.
|
||||
Otherwise the keyupdate release would send unnecessary noise
|
||||
without any actual relay changes.
|
||||
|
||||
- **Exactly one device in a multi-device setup announces.**
|
||||
A device applying a relay change received over
|
||||
multi-device sync should record the resulting list as announced without sending.
|
||||
That also systematically prevents a device catching up on a backlog of old sync
|
||||
messages from announcing historical states.
|
||||
|
||||
- **Behind real traffic.**
|
||||
The message should best leave from the SMTP loop once its queue is drained,
|
||||
so a keyupdate never delays a user message
|
||||
and is only attempted on a connection that just proved to work.
|
||||
|
||||
|
||||
|
||||
## Two ways to send a keyupdate
|
||||
|
||||
Nothing above says how a keyupdate reaches its recipients,
|
||||
and two implementations currently explore that question:
|
||||
|
||||
- [#8601] sends a single symmetrically encrypted message
|
||||
to all recipients at once, re-using the broadcast machinery core has,
|
||||
with the secret derived from our own key
|
||||
so that everyone already holding a copy computes the same value.
|
||||
Nothing in the message is recipient-specific
|
||||
and its cost is largely independent of the recipient count.
|
||||
Wire format, secret derivation, audience and sending policy
|
||||
are documented in its `src/keyupdate.rs` module docs.
|
||||
|
||||
- [#8588] (WIP) sends ordinary asymmetrically encrypted messages,
|
||||
addressed per recipient in chunks of a few dozen contacts,
|
||||
staying on paths core already has
|
||||
at a cost that grows with the number of contacts.
|
||||
|
||||
Both fit the concept described here,
|
||||
and they trade off differently on who can decrypt a keyupdate,
|
||||
what a relay observes, and how much machinery is involved,
|
||||
which is a discussion for the PRs rather than for this draft.
|
||||
|
||||
|
||||
## Open questions
|
||||
|
||||
- Should a keyupdate be rate limited for a profile whose relay list flaps on its own?
|
||||
Manual fiddling is bounded by the person
|
||||
doing it, but future automatic relay add/remove needs to think about limits.
|
||||
|
||||
- What should happen to delivery status notifications for dead addresses?
|
||||
They arrive per address and refer to one Message-ID
|
||||
but we don't do much with them. We could probably evolve to exclude such
|
||||
bounced addresses from future key updates but it shouldn't block
|
||||
a first key update implementation.
|
||||
|
||||
- Is the sending set right? Narrowing the keyupdate recipient set
|
||||
by activity would shrink what we send, but it would also disclose
|
||||
to the relay which of our contacts are close ones.
|
||||
Note that a quiet contact is indistinguishable from one who left,
|
||||
so dropping them silently loses their next message.
|
||||
|
||||
|
||||
## What a reliable keyupdate rollout might open up
|
||||
|
||||
None of the following sections is proposed here.
|
||||
But each of them becomes thinkable once changing relay setups is safer.
|
||||
|
||||
|
||||
### Maximizing reliable deletion ("forward secrecy") with Autocrypt 2
|
||||
|
||||
The keyupdate channel is not necessarily only about relays.
|
||||
Autocrypt 2 ([#8317]) introduces expiring, rotating subkeys,
|
||||
whose distribution has a similar shape as the problem above:
|
||||
a contact we never write to ends up not having our "reliable deletion" encryption keys.
|
||||
The described keyupdate mechanism could help,
|
||||
or at least it could play into the cadence of updates ...
|
||||
|
||||
|
||||
### Forward-rotating through the relay network
|
||||
|
||||
Today a relay change feels like something to be survived.
|
||||
If keyupdates make it routine, it becomes something we could choose to do:
|
||||
Better relays appear, and a profile could drift towards them
|
||||
instead of staying wherever it happened to be born.
|
||||
A safe frequency is capped by how reliably keyupdates land,
|
||||
which is the interesting part:
|
||||
rotation is exactly as feasible as the keyupdate channel is reliable.
|
||||
|
||||
|
||||
### Immortal chat connectivity: surviving total relay loss
|
||||
|
||||
One case is already out of reach of anything we send directly.
|
||||
In [#8329] both sides lost all their relays at once,
|
||||
so any announcement would go to addresses the chat partner can not access anymore.
|
||||
Cooperative Autocrypt gossip can still bridge that
|
||||
if some mutual contact remains reachable from both sides,
|
||||
which is what the reporters ended up relying on,
|
||||
and failing that an out-of-band re-exchange restores contact,
|
||||
which works since [#8358], rerunning the full securejoin protocol
|
||||
when the address is outdated.
|
||||
|
||||
Harder still is a whole region losing all its relays at once:
|
||||
nobody can announce anything to anybody, no push channel helps,
|
||||
and there may be no mutual contact left to bridge it.
|
||||
But new relays, for example in sprouting mesh networks, might become available
|
||||
and wouldn't it be useful to re-establish chat connectivity with those
|
||||
who might be able to help you, or where you can pool resources?
|
||||
|
||||
Interestingly, a keyupdate does not have to be addressed to anyone in particular.
|
||||
Sent as one ciphertext readable by whoever holds the sender's key ([#8601]),
|
||||
it does not have to be directly delivered to be useful:
|
||||
something parked now can be picked up later.
|
||||
Making it discoverable without handing everyone a way to enumerate and track profiles
|
||||
is an interesting enough challenge to make cryptographers have exciting discussions.
|
||||
But during emergencies people really just want things to work
|
||||
and we could ask users in some way "It seems you lost access to all chats.
|
||||
Do you want to enable emergency recovery mode?"
|
||||
|
||||
We could then ephemerally park keyupdates at all accessible relays,
|
||||
and scan what relay-stored key updates are decryptable for us.
|
||||
As soon as chat connectivity recovers for first bunches of people,
|
||||
they spread it further through cooperative Autocrypt gossip,
|
||||
re-establishing chats over time, scaling chat connectivity for everyone.
|
||||
|
||||
|
||||
[#7865]: https://github.com/chatmail/core/issues/7865 "Distribute information about relays in the key signature"
|
||||
[#7878]: https://github.com/chatmail/core/issues/7878 "DeltaChat seems to sometimes propagate the incorrect relays for users who switched relays"
|
||||
[#8317]: https://github.com/chatmail/core/issues/8317 "Autocrypt 2 support"
|
||||
[#8329]: https://github.com/chatmail/core/issues/8329 "Scanning QR for existing contact doesn't update relay list"
|
||||
[#8358]: https://github.com/chatmail/core/pull/8358 "fix: Rerun the full securejoin protocol if the address was outdated"
|
||||
[#8384]: https://github.com/chatmail/core/issues/8384 "Remove hidden relays automatically"
|
||||
[#8444]: https://github.com/chatmail/core/pull/8444 "feat: Basic multi-relay onboarding"
|
||||
[#8481]: https://github.com/chatmail/core/pull/8481 "Improve and speed up autocrypt/pgp gossipping with MDNs"
|
||||
[#8550]: https://github.com/chatmail/core/pull/8550 "fix: multi relay connectivity"
|
||||
[#8588]: https://github.com/chatmail/core/pull/8588 "feat: Key update messages"
|
||||
[#8601]: https://github.com/chatmail/core/pull/8601 "feat: introduce keyupdate message to inform contacts about relay changes"
|
||||
[multi-relay]: https://delta.chat/en/2026-03-31-zero#maximizing-availability-and-resilience-through-multi-path-delivery "Maximizing availability and resilience through multi-path delivery"
|
||||
[privacy notes]: https://github.com/deltachat/deltachat-pages/pull/1385 "Privacy notes being drafted for the apps"
|
||||
Reference in New Issue
Block a user