This fixes the bug when a new transport doesn't become primary on the 2nd device because INBOX from
the new transport isn't fully fetched. Now the `Transports` sync message is received from the old
transport, but as it has updated "From", it updates the primary transport correspondingly. NB: I/O
for the new primary transport isn't immediately started however, this needs a separate fix.
At urgent request from @hpk42, this adds a config option `team_profile`.
This option is only settable via SQLite (not exposed in the UI), and the
only thing it does is disabling synchronization of seen status.
I tested manually on my Android phone that it works.
Not straigthforward to write an automatic test, because we want to test that something
does _not_ happen (i.e. that the seen status is _not_ synchronized), and
it's not clear how long to wait before we check.
Probably it's fine to just not add a test.
This is what I tried:
```python
@pytest.mark.parametrize("team_profile", [True, False])
def test_markseen_basic(team_profile, acfactory):
"""
Test that seen status is synchronized iff `team_profile` isn't set.
"""
alice, bob = acfactory.get_online_accounts(2)
# Bob sets up a second device.
bob2 = bob.clone()
bob2.start_io()
alice_chat_bob = alice.create_chat(bob)
bob.create_chat(alice)
bob2.create_chat(alice)
alice_chat_bob.send_text("Hello Bob!")
message = bob.wait_for_incoming_msg()
message2 = bob2.wait_for_incoming_msg()
assert message2.get_snapshot().state == MessageState.IN_FRESH
message.mark_seen()
# PROBLEM: We're not waiting 'long enough',
# so, the 'state == MessageState.IN_SEEN' assertion below fails
bob.create_chat(bob).send_text("Self-sent message")
self_sent = bob2.wait_for_msg(EventType.MSGS_CHANGED)
assert self_sent.get_snapshot().text == "Self-sent message"
if team_profile:
assert message2.get_snapshot().state == MessageState.IN_FRESH
else:
assert message2.get_snapshot().state == MessageState.IN_SEEN
```
Adds an api to get all ui config keys. There already is an option to get
all normal config keys (`"sys.config_keys"`), but before this pr there
was no way to get all `ui.*` config keys.
#### Why is this api needed?
For webxdc cleanup on desktop, which stores window position in a ui
config key (such as `ui.desktop.webxdcBounds.676464`) as soon as a
webxdc is opened since many versions now. So listing all ui keys is a
good way for us to find out which webxdc may have web data stored.
unfortunately electron does not (yet?) have a way to list all origins
that have web-data like android does, so this is the next best thing we
can do before itterating all possible ids, see also
https://github.com/deltachat/deltachat-desktop/issues/5758.
#### Why is this only a jsonrpc api and not another special/virtual
config key like `"sys.config_keys"`?
r10s indicated that `ui.*`-config keys are barely used
(https://github.com/deltachat/deltachat-desktop/issues/5790#issuecomment-3598512802),
so I thought it makes more sense to add it as dedicated api which's
existentence is checked by the typechecker, so it will be easier to not
miss it when we should remove the api again in the future.
But we could also do a dedicated special/virtual config key for it, if
you think that is better, this is easy to change.
---------
Co-authored-by: iequidoo <117991069+iequidoo@users.noreply.github.com>
Fix get_secondary_addrs() which was using
`secondary_addrs` config that is not updated anymore.
Instead of using `secondary_addrs` config,
use the `transports` table which contains all the addresses.
And enable it by default as the standard Header Protection is backward-compatible.
Also this tests extra IMF header removal when a message has standard Header Protection since now we
can send such messages.
Omit Legacy Display Elements from "text/plain" and "text/html" (implement 4.5.3.{2,3} of
https://www.rfc-editor.org/rfc/rfc9788 "Header Protection for Cryptographically Protected Email").
`login_param` module is now for user-visible entered login parameters,
while the `transport` module contains structures for internal
representation of connection candidate list
created during transport configuration.
It's used in `fetch_existing_msgs()`, but we can remove it and tell users that they need to
move/copy messages from Sentbox to Inbox so that Delta Chat adds all contacts from them. This way
users will be also informed that Delta Chat needs users to CC/BCC/To themselves to see messages sent
from other MUAs.
The motivation is to reduce code complexity, get rid of the extra IMAP connection and cases when
messages are added to chats by Inbox and Sentbox loops in parallel which leads to various message
sorting bugs, particularly to outgoing messages breaking sorting of incoming ones which are fetched
later, but may have a smaller "Date".
We do not try to delete resent messages
anymore. Previously resent messages
were distinguised by having duplicate Message-ID,
but future Date, but now we need to download
the message before we even see the Date.
We now move the message to the destination folder
but do not fetch it.
It may not be a good idea to delete
the duplicate in multi-device setups anyway,
because the device which has a message
may delete the duplicate of a message
the other device missed.
To avoid triggering IMAP busy move loop
described in the comments
we now only move the messages
from INBOX and Spam folders.
This way, the statistics / self-reporting bot will be made into an
opt-in regular sending of statistics, where you enable the setting once
and then they will be sent automatically. The statistics will be sent to
a bot, so that the user can see exactly which data is being sent, and
how often. The chat will be archived and muted by default, so that it
doesn't disturb the user.
The collected statistics will focus on the public-key-verification that
is performed while scanning a QR code. Later on, we can add more
statistics to collect.
**Context:**
_This is just to give a rough idea; I would need to write a lot more
than a few paragraphs in order to fully explain all the context here_.
End-to-end encrypted messengers are generally susceptible to MitM
attacks. In order to mitigate against this, messengers offer some way of
verifying the chat partner's public key. However, numerous studies found
that most popular messengers implement this public-key-verification in a
way that is not understood by users, and therefore ineffective - [a 2021
"State of Knowledge" paper
concludes:](https://dl.acm.org/doi/pdf/10.1145/3558482.3581773)
> Based on our evaluation, we have determined that all current E2EE
apps, particularly when operating in opportunistic E2EE mode, are
incapable of repelling active man-in-the-middle (MitM) attacks. In
addition, we find that none of the current E2EE apps provide better and
more usable [public key verification] ceremonies, resulting in insecure
E2EE communications against active MitM attacks.
This is why Delta Chat tries to go a different route: When the user
scans a QR code (regardless of whether the QR code creates a 1:1 chat,
invites to a group, or subscribes to a broadcast channel), a
public-key-verification is performed in the background, without the user
even having to know about this.
The statistics collected here are supposed to tell us whether Delta Chat
succeeds to nudge the users into using QR codes in a way that is secure
against MitM attacks.
**Plan for statistics-sending:**
- [x] Get this PR reviewed and merged (but don't make it available in
the UI yet; if Android wants to make a release in the meantime, I will
create a PR that removes the option there)
- [x] Set the interval to 1 week again (right now, it's 1 minute for
testing)
- [ ] Write something for people who are interested in what exactly we
count, and link to it (see `TODO[blog post]` in the code)
- [ ] Prepare a short survey for participants
- [ ] Fine-tune the texts at
https://github.com/deltachat/deltachat-android/pull/3794, and get it
reviewed and merged
- [ ] After the next release, ask people to enable the
statistics-sending
Ignoring `receive_imf_inner()` errors, i.e. silently skipping messages on failures, leads to bugs
never fixed. As for temporary I/O errors, ignoring them leads to lost messages, in this case it's
better to bubble up the error and get the IMAP loop stuck. However if there's some logic error, it's
better to show it to the user so that it's more likely reported, and continue receiving messages. To
distinguish these cases, on error, try adding the message as partially downloaded with the error set
to `msgs.error`, this way the user also can retry downloading the message to finally see it if the
problem is fixed.
The setting is already removed from the UIs,
but users who had it disabled previously have
no way to enable it. After this change
encryption is effectively always preferred.
`get_connectivity()` is expected to return immediately, not when the scheduler finishes updating its
state in `start_io()/stop_io()/pause_io()`, otherwise it causes app non-responsiveness.
Instead of read-locking `SchedulerState::inner`, store the `ConnectivityStore` collection in
`Context` and fetch it from there in `get_connectivity()`. Update it every time we release a write
lock on `SchedulerState::inner`.
A donation request device message is added if >= 100 messages have been sent and delivered. The
condition is checked every 30 days since the first message is sent. The message is added only once.
Move all `configured_*` parameters into a new SQL table `transports`.
All `configured_*` parameters are deprecated; the only exception is
`configured_addr`, which is used to store the address of the primary
transport. Currently, there can only ever be one primary transport (i.e.
the `transports` table only ever has one row); this PR is not supposed
to change DC's behavior in any meaningful way.
This is a preparation for mt.
---------
Co-authored-by: l <link2xt@testrun.org>
fetch_existing option is not enabled in existing clients
and does not work with encrypted messages
without importing the key into a newely created account.
APNS tokens never expire unless
the user uninstalls the application.
Because of this in most cases
the token remains valid forever
and chatmail server never removes the token
even if it is unencrypted
or the user has removed Delta Chat profile
from the device but not the whole application.
We want to modify chatmail servers
to remember the last time the token was stored
and remove them after some time.
Before we do this, we need to modify
the client to store the device token
each time so the server knows which tokens are used
and can update their timestamps.
These were the last places in the `deltachat` crate where files were
stored without deduplication. The CFFI python bindings are the last
thing that's still missing.
Deduplicate:
- In the REPL
- In `store_from_base64()`, which writes avatars received in headers
- In a few tests
- The saved messages, broadcast, device, archive icons
- The autocrypt setup message
1-2 more PRs, and we can get rid of `BlobObject::create`,
`sanitise_name()`, and some others
Fix https://github.com/deltachat/deltachat-core-rust/issues/6433
I at first only wanted to do it any outgoing messages, but @link2xt was
concerned that this may accidentally enable bcc_self, e.g. in the
following case:
- you send out a message
- it's deleted, e.g. via ephemeral messages
- Someone forwards this outgoing message to you again, e.g. via a
mailing list.
Change `BccSelf` default to 0 for chatmail configurations and enable it upon a backup export. As for
`DeleteServerAfter` who was set to 0 upon a backup export before, make its default dependent on
`BccSelf` for chatmail. We don't need `BccSelf` for chatmail by default because we assume
single-device use. Also `BccSelf` is needed for "classic" email accounts even if `DeleteServerAfter`
is set to "immediately" to detect that a message was sent if SMTP server is slow to respond and
connection is lost before receiving the status line which isn't a problem for chatmail servers.
- **feat: add `AccountsChanged` and `AccountsItemChanged` events**
- **emit event and add tests**
closes#6106
TODO:
- [x] test receiving synced config from second device
- [x] bug: investigate how to delay the configuration event until it is
actually configured - because desktop gets the event but still shows
account as if it was unconfigured, maybe event is emitted before the
value is written to the database?
- [x] update node bindings constants
This change adds support for receiving
Autocrypt header in the protected part of encrypted message.
Autocrypt header is now also allowed in mailing lists.
Previously Autocrypt header was rejected when
List-Post header was present,
but the check for the address being equal to the From: address
is sufficient.
New experimental `protect_autocrypt` config is disabled
by default because Delta Chat with reception
support should be released first on all platforms.
