It's possible that when rebasing a PR adding a migration a merge-conflict doesn't occur if another
migration was added in the target branch. Better to have at least runtime checks that the migration
version is correct. Looks like compile-time checks are not possible because Rust doesn't allow to
redefine constants, only vars.
Bot processes are run asynchronously, so we shouldn't send messages to a bot before it's fully
initialised and skipped existing messages for processing, i.e. before DC_EVENT_IMAP_INBOX_IDLE is
emitted.
Looks like it has no sense to send any profile data (From/To names, self-status; self-avatar was
never sent even before) in MDNs, they aren't normal messages and aren't seen in a MUA. Better not to
reveal profile data to the network and even to contacts in MDNs and make them more lightweight.
Follow-up to b771311593. Since that commit names are not revealed in
verified chats, but during verification (i.e. SecureJoin) they are still sent unencrypted. Moreover,
all profile data mustn't be sent even encrypted before the contact verification, i.e. before
"v{c,g}-request-with-auth". That was done for the selfavatar in
304e902fce, now it's done for From/To names and the self-status as
well. Moreover, "v{c,g}-request" and "v{c,g}-auth-required" messages are deleted right after
processing, so other devices won't see the received profile data anyway.
Probably sync messages generated for a not yet configured account are useless because there are no
other devices yet. And even if this is not true, we don't want to depend on the order of setting
`Config::SyncMsgs` and other keys. Also w/o this the Python tests don't work if we start syncing
`Config::MvboxMove` because they don't expect that sync messages are sent while configuring
accounts.
NB: We don't restart IO from the synchronisation code, so `MvboxMove` isn't effective immediately if
`ConfiguredMvboxFolder` is unset, but only after a reconnect to IMAP.
Before, if the user deleted a message too quickly after sending, it was deleted only locally. The
fix is to remember for tombstones that the corresponding message should be deleted on the server
too.
This is a way to prevent redownloading locally deleted messages. Otherwise if a message is deleted
quickly after sending and `bcc_self` is configured, the BCC copy is downloaded and appears as a new
message as it happens for messages sent from another device.
Delta Chat -style groups have names w/o prefixes like "Re: " even if the user is added to an already
existing group, so let's remove prefixes from ad-hoc group names too. Usually it's not very
important that the group is a classic email thread existed before, this info just eats up screen
space. Also this way a group name is likely to preserve if the first message was missed.
SQLite search with `LIKE` is case-insensitive only for ASCII chars. To make it case-insensitive for
all messages, create a new column `msgs.txt_normalized` defaulting to `NULL` (so we do not bump up
the database size in a migration) and storing lowercased/normalized text there when the row is
created/updated. When doing a search, search over `IFNULL(txt_normalized, txt)`.
Before, if `Config::FetchExistingMsgs` is set, existing messages were received with the `InSeen`
state set, but for bots they must be `InFresh` and also `IncomingMsg` events should be emitted for
them so that they are processed by bots as it happens with new messages.
Otherwise it's impossible to remove a member with missing key from a protected group. In the worst
case a removed member will be added back due to the group membership consistency algo.
The "I left the group" message can't be sent to a protected group if some member's key is missing,
in this case we should remain in the group. The problem should be fixed first, then the user may
retry to leave the group.
There was a comment in `fetch_existing_msgs()`: "Bots don't want those messages". If a bot doesn't
want this setting, why enable it? It's disabled by default anyway.
`IsChatmail` is set also by `inbox_fetch_idle()`, but it isn't called during `configure()`. Setting
`IsChatmail` from `inbox_fetch_idle()` is necessary to handle client/server upgrades, but
`IsChatmail` also should be available for the app after configuring an account, e.g. DC Android
needs it to know whether to ask the user to disable battery optimisations.
When doing an AEAP transition, we mustn't just delete the old peerstate as this would break
encryption to it. This is critical for non-verified groups -- if we can't encrypt to the old
address, we can't securely remove it from the group (to add the new one instead).
It's actually `deltachat::contact::Contact::authname` by semantics. `display_name`/`name` shouldn't
be shared, it's the name given by the user locally.
Add a function parsing a vCard file at the given path.
Co-authored-by: Hocuri <hocuri@gmx.de>
Co-authored-by: Asiel Díaz Benítez <asieldbenitez@gmail.com>
This protects Bob (the joiner) of sending unexpected-unencrypted messages during an otherwise nicely
running SecureJoin.
If things get stuck, however, we do not want to block communication -- the chat is just
opportunistic as usual, but that needs to be communicated:
1. If Bob's chat with Alice is `Unprotected` and a SecureJoin is started, then add info-message
"Establishing guaranteed end-to-end encryption, please wait..." and let `Chat::can_send()` return
`false`.
2. Once the info-message "Messages are guaranteed to be e2ee from now on" is added, let
`Chat::can_send()` return `true`.
3. If after SECUREJOIN_WAIT_TIMEOUT seconds `2.` did not happen, add another info-message "Could not
yet establish guaranteed end-to-end encryption but you may already send a message" and also let
`Chat::can_send()` return `true`.
Both `2.` and `3.` require the event `ChatModified` being sent out so that UI pick up the change wrt
`Chat::can_send()` (this is the same way how groups become updated wrt `can_send()` changes).
SECUREJOIN_WAIT_TIMEOUT should be 10-20 seconds so that we are reasonably sure that the app remains
active and receiving also on mobile devices. If the app is killed during this time then we may need
to do step 3 for any pending Bob-join chats (right now, Bob can only join one chat at a time).
`ChatId::lookup_by_contact()` returns `None` for blocked chats, so it should be only used if we need
to filter out blocked chats, e.g. when building a chatlist.
E.g. the multi-device synchronisation creates the "Saved Messages" chat as blocked, in this case the
chat icon wasn't updated before and the user avatar was displayed instead.
Otherwise, e.g. if a message is a large GIF, but its viewtype is set to `Image` by the app, this GIF
will be recoded to JPEG to reduce its size. GIFs and other special viewtypes must be always detected
and sent as is.
... and fails if file already exists. The UI should open the file saving dialog, defaulting to
Downloads and original filename, when asked to save the file. After confirmation it should call
dc_msg_save_file().
This way we avoid an immediate retry if the network is not yet ready exhausting the ratelimiter's
quota of two connection attempts. Also notify the ratelimiter only after a successful connection so
that it only limits the server load, but not connection attempts.
As trashed messages can't be loaded from the db now by `Message::load_from_db()` returning an error
for such messages, errors from `Message::load_from_db()` should be logged as warnings. If it's
really an error like a db failure, it should be logged internally.
As trashed messages can't be loaded from the db now, `get_message_by_id()` returns None in some
tests e.g. in `test_deleted_msgs_dont_reappear()`. A `PerAccount` hook shouldn't be called if so.
There's no need to load an updated message state from the db to implement `is_outgoing()` and also
this function is implicitly called in some tests where a message is already trashed and a call to
`dc_get_msg()` generates an unexpected error.
This fixes things for Gmail f.e. Before, `Imap::fetch_move_delete()` was called before looking for
Trash and returned an error because of that failing the whole `fetch_idle()` which prevented
configuring Trash in turn.
If `delete_device_after` is configured, that period should be counted for webxdc instances from the
last status update, otherwise nothing prevents from deleting them. Use `msgs.timestamp_rcvd` to
store the last status update timestamp, it anyway isn't used for anything except displaying a
detailed message info. Also, as `ephemeral::select_expired_messages()` now also checks
`timestamp_rcvd`, we have an improvement that a message is guaranteed not to be deleted for the
`delete_device_after` period since its receipt. Before only the sort timestamp was checked which is
derived from the "sent" timestamp.
Let's add a 1-minute tolerance to `Params::MemberListTimestamp`.
This adds to the group membership consistency algo the following properties:
- If remote group membership changes were made by two members in parallel, both of them are applied,
no matter in which order the messages are received.
- If we remove a member locally, only explicit remote member additions/removals made in parallel are
allowed, but not the synchronisation of the member list from "To". Before, if somebody managed to
reply earlier than receiving our removal of a member, we added it back which doesn't look good.
- Always emit `ContactsChanged` from `contact::update_last_seen()` if a contact was seen recently
just for simplicity and symmetry with `RecentlySeenLoop::run()` which also may emit several events
for single contact.
- Fix sleep time calculation in `RecentlySeenLoop::run()` -- `now` must be updated on every
iteration, before the initial value was used every time which led to progressively long sleeps.
Instead, look up the 1:1 chat in `receive_imf::add_parts()`. This is a more generic approach to fix
assigning outgoing reactions to 1:1 chats in the multi-device setup. Although currently both
approaches give the same result, this way we can even implement a "react privately"
functionality. Maybe it sounds useless, but it seems better to have less reaction-specific code.
Don't attach selfavatar in "v{c,g}-request" and "v{c,g}-auth-required" messages:
- These messages are deleted right after processing, so other devices won't see the avatar.
- It's also good for privacy because the contact isn't yet verified and these messages are auto-sent
unlike usual unencrypted messages.
Add white background instead of the default black one to avatars when recoding to JPEG. But not for
"usual" images to spare the CPU. The motivation is to handle correctly
"black-on-transparent-background" avatars which are quite common.
`Param::MemberListTimestamp` was updated only from `receive_imf::apply_group_changes()` i.e. for
received messages. If we sent a message, that timestamp wasn't updated, so remote group membership
changes always overrode local ones. Especially that was a problem when a message is sent offline so
that it doesn't incorporate recent group membership changes.
An unencrypted message with already known Autocrypt key, but sent from another address, means that
it's rather a new contact sharing the same key than the existing one changed its address, otherwise
it would already have our key to encrypt.
If Subject is multiline-formatted, `mailparse` adds the leading whitespace to it. The solution is to
always remove the leading whitespace, because if Subject isn't multiline-formatted, it never
contains the leading whitespace anyway. But as for the trailing whitespace -- i checked -- it's
never removed, so let's keep this as is.
This fixes the following identity-misbinding attack:
It appears that Bob’s messages in the SecureJoin protocol do not properly “bind” to Alice’s public
key or fingerprint. Even though Bob’s messages carry Alice’s public key and address as a gossip in
the protected payload, Alice does not reject the message if the gossiped key is different from her
own key. As a result, Mallory could perform an identity-misbinding attack. If Mallory obtained
Alice’s QR invite code, she could change her own QR code to contain the same tokens as in Alice’s QR
code, and convince Bob to scan the modified QR code, possibly as an insider attacker. Mallory would
forward messages from Bob to Alice and craft appropriate responses for Bob on his own. In the end,
Bob would believe he is talking to Mallory, but Alice would believe she is talking to Bob.
If a message is sent from SELF, but signed with a foreign key, it mustn't be considered
Autocrypt-encrypted and shown with a padlock. Currently this is broken.
The way it was implemented it threw out all remaining messages after finding the next incoming
message. Better use FFIEventTracker functions, they are used in all the tests anyway.
Before group avatar was sent as an attachment. Let's do the same as with user avatar and send group
avatar as base64. Receiver code uses the same functions for user and chat avatars, so base64 avatars
are supported for most receivers already.
It was broken completely and before "fix: apply Autocrypt headers if timestamp is unchanged" that
didn't show up because the message from the second Bob's device never had "Date" greater than one
from the message sent before from the first device.
Use sync messages for that as it is done for e.g. Config::Displayname. Maybe we need to remove
self-status synchronisation via usual messages then, but let's think of it a bit.
Use sync messages for that as it is done for e.g. Config::Displayname. Maybe we need to remove
avatar synchronisation via usual messages then, but let's think of it a bit.
Even if `vc-request-with-auth` is received with a delay, the protection message must have the sort
timestamp equal to the Sent timestamp of `vc-request-with-auth`, otherwise subsequent chat messages
would also have greater sort timestamps and while it doesn't affect the chat itself (because Sent
timestamps are shown to a user), it affects the chat position in the chatlist because chats there
are sorted by sort timestamps of the last messages, so the user sees chats sorted out of
order. That's what happened in #5088 where a user restores the backup made before setting up a
verified chat with their contact and fetches new messages, including `vc-request-with-auth` and also
messages from other chats, after that.
Add a new crate `deltachat_time` with a fake `struct SystemTimeTools` for mocking
`SystemTime::now()` for test purposes. One still needs to use `std::time::SystemTime` as a struct
representing a system time. I think such a minimalistic approach is ok -- even if somebody uses the
original `SystemTime::now()` instead of the mock by mistake, that could break only tests but not the
program itself. The worst thing that can happen is that tests using `SystemTime::shift()` and
checking messages timestamps f.e. wouldn't catch the corresponding bugs, but now we don't have such
tests at all which is much worse.
The system clock may be adjusted and even go back, so caching system time in code sections where
it's not supposed to change may even protect from races/bugs.
If a time value doesn't need to be sent to another host, saved to the db or otherwise used across
program restarts, a monotonically nondecreasing clock (`Instant`) should be used. But as `Instant`
may use `libc::clock_gettime(CLOCK_MONOTONIC)`, e.g. on Android, and does not advance while being in
deep sleep mode, get rid of `Instant` in favor of using `SystemTime`, but add `tools::Time` as an
alias for it with the appropriate comment so that it's clear why `Instant` isn't used in those
places and to protect from unwanted usages of `Instant` in the future. Also this can help to switch
to another clock impl if we find any.
Restart the IO scheduler if needed to make the new config value effective (for `MvboxMove,
OnlyFetchMvbox, SentboxWatch` currently). Also add `set_config_internal()` which doesn't affect
running the IO scheduler. The reason is that `Scheduler::start()` itself calls `set_config()`,
although not for the mentioned keys, but still, and also Rust complains about recursive async calls.
Currently when a user sets up another device by logging in, a new key is created. If a message is
sent from either device outside, it cannot be decrypted by the other device.
The message is replaced with square bracket error like this:
```
<string name="systemmsg_cannot_decrypt">This message cannot be decrypted.\n\n• It might already help to simply reply to this message and ask the sender to send the message again.\n\n• If you just re-installed Delta Chat then it is best if you re-setup Delta Chat now and choose "Add as second device" or import a backup.</string>
```
(taken from Android repo `res/values/strings.xml`)
If the message is outgoing, it does not help to "simply reply to this message". Instead, we should
add a translatable device message of a special type so UI can link to the FAQ entry about second
device. But let's limit such notifications to 1 per day. And as for the undecryptable message
itself, let it go to Trash if it can't be assigned to a chat by its references.
Before it was emitted only on the source device and `test_sync()` didn't catch the bug because of
the interference of the previous call to `set_config_bool()`. Now there's a separate test that does
its job well.
iequidoo
