reported by bjoern while testing the branch on iOS: the background addition
would start fast enough to still show up in the configure-progress bar
of the initial onboarding screen. This commit should fix that.
Closes#8693 and supsersedes #8707 from which the API
and some overall shape of this commit is inspired.
UIs call `init_transports(None)` or `init_transports(qr)`
to initialize a first transport on the fresh profile,
with more transports added in the background later.
this PR adds a hack to allow relay operators
to differ between manually created relays and autorelays.
this is a precaution in case unexpected things happen when introducing
autorelay; relay operators then can deny creation on `password_len ==
23`, without shutting down manual creation completely.
depeding on final initTransport() is done,
it may still be that the first created relay is marked as being manual,
but that seems fine.
`password_len` of the autmatically generated password was chosen as
there is otherwise not much data sent -
chatmail avoids visible data and metadata everywhere.
this hack is about to be removed again asap,
once we have some experiences with multi relay.
---------
Co-authored-by: l <link2xt@testrun.org>
UIs should use display name everywhere and avoid displaying email addresses.
BREAKING CHANGE: dc_contact_get_name_n_addr() CFFI is removed
BREAKING CHANGE: JSON-RPC contact objects don't have nameAndAddr field anymore
No changes except that smtp::is_queue_empty is renamed into smtp::queue::is_empty,
mimefactory::QueuedEncryption is renamed into smtp::queue::Encryption
and mimefactory::QueueSideEffects is renamed to smtp::queue::SideEffects.
mimefactory module does not know anything about smtp2 table anymore.
This is mostly needed to fix flaky
tests/test_something.py::test_is_sending_finished
by making sure that is_sending_finished() return false
once we got an event that message was sent.
This removes the last usages of the deprecated `is_chatmail` and
no `XCHATMAIL` IMAP capability is read anymore.
UIs are not using it for a longer time anymore.
The self-reporting statistics now set `is_chatmail` field to
`true`: all transports have relay-typical metadata
`false`: at least one doesn't
`null`: we don't know
BREAKING CHANGE: `is_chatmail` is no longer a known config key.
don't use `is_chatmail` for determining whether to delete messages on the server:
whether a downloaded message may be dropped from the relay
depends on whether another device still needs it (BccSelf)
and on whether plaintext mail is allowed (ForceEncryption).
Also adds tests and docs to respective functions,
clarifying background fetching behaviour and the `ACCOUNTS_BACKGROUND_FETCH_DONE` event,
that came up in questions/discussions with UI devs lately.
"addr" is already deprecated and its default value was using "configured_addr".
We want to get rid of "configured_addr" too, but users should at least
not access it through deprecated config.
Core no longer tracks verification, so the API has nothing left
to report and UIs should drop their checkmark and "Introduced by" code.
BREAKING CHANGE: dc_contact_is_verified() and dc_contact_get_verifier_id() are removed.
BREAKING CHANGE: the JSON-RPC Contact object loses the `isVerified` and `verifierId` fields. A bot reading `snapshot.is_verified` now gets an `AttributeError` at runtime.
BREAKING CHANGE: the Python bindings lose `Contact.is_verified()` and `Contact.get_verifier()`.
BREAKING CHANGE: DC_STR_CONTACT_VERIFIED (35) is removed, so UIs should stop registering a translation for it. A stock id core does not know is logged and otherwise ignored, so an un-updated client keeps working.
Since V2 a contact is its key, there is no address-to-key binding left to verify.
The JSONRPC and FFI APIs are unchanged and report nothing as verified.
BREAKING CHANGE: the statistics JSON sent to the self-reporting-bot on Android changes: Contacts have `encrypted` instead of `verified` and lose `transitive_chain` properties and message stats have `encrypted` instead of `verified` and `unverified_encrypted`, and securejoin invites lose `already_verified`. The collecting bot stores incoming reports verbatim but analysis will have to make sense of older and newer reports.
Headers like From and Autocrypt are now added late,
right before sending the message over SMTP.
This way we advertise the latest list of transports
and use the correct From address in the encrypted part
even for messages queued while being offline.
BCC-self recipients are also added late.
For unencrypted messages we only want to send a copy
to the sending address, but we don't know the sending address
when queueing the message.
Adding bcc-self recipients when dequeuing the message
also makes it possible to send copies to updated list of relays.
This is similar to mimefactory::Encryption,
but does not have email addresses for asymmetrically encrypted messages.
Queued messages don't need email addresses for public keys.
Addresses are only needed to render Autocrypt-Gossip headers.
A second background_fetch() while one is already running returned
without emitting the event, and the FFI returned 1 for it,
so a UI waiting for an event hangs dc_get_next_event().
Emit the event in any case, so waiting for it is safe.
With I/O stopped, `background_fetch()` connected only to the transport of `configured_addr`
and we now instead fan out to all transports in a controlled loop.
If a first transport finished fetching new messages
cancel all other attempts and return.
This is meant to address the problem that amzd described
where a profile with one functioning and one hanging transport,
shows the first notification, then hangs 15 seconds waiting for the hanging transport.
meanwhile a second NSE arrives and dies, and the second message is not notified
or only generically.
Also drop the quota check from this background fetch path:
its result is in-memory only, discarded when the iOS notification service exits,
and the regular scheduler fetching refreshes it every 60s anyway.
Moreover, quota errors/running full is pretty rare
since relays generally automatically stay under quota these days.
It's another round trip for each transport of each profile and simply not necessary.
Also adds previously missing online tests.
this PR introduces a better quality of image recoding, consuming max.
900k instead of 500k (average is much less). the PR roughly doubles the
number of pixels sent in an image.
the old 1280px were set 8 years ago,
data and storage has improved since then,
so it is reasonable to double the number if pixels used for sending an
image.
this will be a quality boost for many images,
while not resulting in a doubled size for all of them; many images will
only be a little larger in bytes, see test.
we could always go higher, of course, but it comes at costs of relay
storage and data, so we stay conservative, even in that increase.
the "worse quality" setting is not adapted on purpose, there we really
stay at the end of what is bearable :)
cc @adbenitez
---------
Co-authored-by: l <link2xt@testrun.org>
Main change is the removal of the comparison of Chat-Disposition-Notification-To
to the From header for incoming messages.
Removed code that was settting WantsMdn for outgoing messages
is a leftover not cleaned up in ade39fe026
We do not actually use WantsMdn for outgoing messages.
BREAKING CHANGE: get_webxdc_blob() may fail to load icon.png or icon.jpg if image dimensions are too large.
The issue is discovered by https://github.com/Sergei768
This Final-Recipient was not set to the correct value anyway.
We could query the database and find out via `imap` table
which transport we have received the message on,
but it is not worth the effort as the field is not practically used
and cannot be relied on as old versions still send incorrect value.
This removes one call to get_primary_self_addr()
to make it easier to remove the concept of the "primary"
address eventually.
According to
<https://datatracker.ietf.org/doc/html/rfc8098#section-3.2.3>
Original-Recipient field values in MDNs MUST NOT be included
if the information about original recipient is not available.
Original recipient may be obtained from ORCPT parameter
of SMTP envelope or from Original-Recipient header
which MTAs are expected to convert Original-Recipient to.
The way we have been using Original-Recipient field is not correct.
Technically we should look for Original-Recipient header
on the message when downloading it from IMAP
and then copy the value into MDN,
but simply assuming it is never there is more correct
than always assuming it is the same as our current address.
I also grepped for Original-Recipient
and removed it together with Reporting-UA from the tests.
Orignal-Recipient is now only left in NDN (bounce messages) test data,
there it is correct as this field is added by MTAs
that have direct access to ORCPT parameter.
It is safe to import any keys into the keychain.
Keys can anyway be imported from vCards
and Autocrypt headers without any checks.
These checks are from the time before we had key-contacts
and maintained Autocrypt `peerstates` table.
Also marked functions that are not used outside as pub(crate).
Some functions like get_filesuffix_lc() are still used
by deltachat-repl, so the whole module cannot be made private.
Removing a relay now takes effect immediately:
- the profile stops fetching and advertising it,
- secondary devices immediately apply the removal through the transport sync,
Upgrading removes unpublished relays and triggers keyupdates.
BREAKING CHANGE: set_transport_unpublished() is removed: UIs call delete_transport() when the user removes a relay.
BREAKING CHANGE: list_transports_ex() and the TransportListEntry type are removed: use list_transports().
BREAKING CHANGE: delete_transport() no longer refuses to remove the primary transport: it refuses only to remove the last one and re-elects the sending transport as needed.
BREAKING CHANGE: TransportsModified is now also emitted on the device modifying the transports, not only on devices applying the synced change.
Deprecated: DC_STR_PHASING_OUT
`relay_addrs()` fell back to the contact address even when it is empty,
which happens for key-contacts created from a sync message or for the
self-contact, putting an empty string into the SMTP recipient list.
When the published relay list changes, key-contacts are informed with an
unsigned message carrying the re-signed key, encrypted to a chunk of contacts
at a time. It is shaped like a receipt notification naming no message, so that
cores which know nothing about keyupdates trash it as well.
See the src/keyupdate.rs module docs for the design.
