This way, the statistics / self-reporting bot will be made into an
opt-in regular sending of statistics, where you enable the setting once
and then they will be sent automatically. The statistics will be sent to
a bot, so that the user can see exactly which data is being sent, and
how often. The chat will be archived and muted by default, so that it
doesn't disturb the user.
The collected statistics will focus on the public-key-verification that
is performed while scanning a QR code. Later on, we can add more
statistics to collect.
**Context:**
_This is just to give a rough idea; I would need to write a lot more
than a few paragraphs in order to fully explain all the context here_.
End-to-end encrypted messengers are generally susceptible to MitM
attacks. In order to mitigate against this, messengers offer some way of
verifying the chat partner's public key. However, numerous studies found
that most popular messengers implement this public-key-verification in a
way that is not understood by users, and therefore ineffective - [a 2021
"State of Knowledge" paper
concludes:](https://dl.acm.org/doi/pdf/10.1145/3558482.3581773)
> Based on our evaluation, we have determined that all current E2EE
apps, particularly when operating in opportunistic E2EE mode, are
incapable of repelling active man-in-the-middle (MitM) attacks. In
addition, we find that none of the current E2EE apps provide better and
more usable [public key verification] ceremonies, resulting in insecure
E2EE communications against active MitM attacks.
This is why Delta Chat tries to go a different route: When the user
scans a QR code (regardless of whether the QR code creates a 1:1 chat,
invites to a group, or subscribes to a broadcast channel), a
public-key-verification is performed in the background, without the user
even having to know about this.
The statistics collected here are supposed to tell us whether Delta Chat
succeeds to nudge the users into using QR codes in a way that is secure
against MitM attacks.
**Plan for statistics-sending:**
- [x] Get this PR reviewed and merged (but don't make it available in
the UI yet; if Android wants to make a release in the meantime, I will
create a PR that removes the option there)
- [x] Set the interval to 1 week again (right now, it's 1 minute for
testing)
- [ ] Write something for people who are interested in what exactly we
count, and link to it (see `TODO[blog post]` in the code)
- [ ] Prepare a short survey for participants
- [ ] Fine-tune the texts at
https://github.com/deltachat/deltachat-android/pull/3794, and get it
reviewed and merged
- [ ] After the next release, ask people to enable the
statistics-sending
SQL statements fail if the number of variables
exceeds `SQLITE_LIMIT_VARIABLE_NUMBER`.
Remaining repeat_vars() calls are difficult to replace
and use arrays passed from the UI,
e.g. forwarded message IDs or marked as seen IDs.
Instead of treating NULL type error
as absence of the row,
handle NULL values with SQL.
Previously we sometimes
accidentally treated a single column
being NULL as the lack of the whole row.
Before, if the user deleted a message too quickly after sending, it was deleted only locally. The
fix is to remember for tombstones that the corresponding message should be deleted on the server
too.
SQLite search with `LIKE` is case-insensitive only for ASCII chars. To make it case-insensitive for
all messages, create a new column `msgs.txt_normalized` defaulting to `NULL` (so we do not bump up
the database size in a migration) and storing lowercased/normalized text there when the row is
created/updated. When doing a search, search over `IFNULL(txt_normalized, txt)`.
`ChatId::lookup_by_contact()` returns `None` for blocked chats, so it should be only used if we need
to filter out blocked chats, e.g. when building a chatlist.
API now pretends that trashed messages don't exist.
This way callers don't have to check if loaded message
belongs to trash chat.
If message may be trashed by the time it is attempted to be loaded,
callers should use Message::load_from_db_optional.
Most changes are around receive_status_update() function
because previously it relied on loading trashed status update
messages immediately after adding them to the database.
If `delete_device_after` is configured, that period should be counted for webxdc instances from the
last status update, otherwise nothing prevents from deleting them. Use `msgs.timestamp_rcvd` to
store the last status update timestamp, it anyway isn't used for anything except displaying a
detailed message info. Also, as `ephemeral::select_expired_messages()` now also checks
`timestamp_rcvd`, we have an improvement that a message is guaranteed not to be deleted for the
`delete_device_after` period since its receipt. Before only the sort timestamp was checked which is
derived from the "sent" timestamp.
When scheduler is destroyed, e.g. during a key import,
there is some time between destroying the interrupt channel
and the loop task.
To avoid busy looping, tasks should terminate if
receiving from the interrupt loop fails
instead of treating it as the interrupt.
Add a new crate `deltachat_time` with a fake `struct SystemTimeTools` for mocking
`SystemTime::now()` for test purposes. One still needs to use `std::time::SystemTime` as a struct
representing a system time. I think such a minimalistic approach is ok -- even if somebody uses the
original `SystemTime::now()` instead of the mock by mistake, that could break only tests but not the
program itself. The worst thing that can happen is that tests using `SystemTime::shift()` and
checking messages timestamps f.e. wouldn't catch the corresponding bugs, but now we don't have such
tests at all which is much worse.
I.e. from delete_msgs(). Otherwise messages must not be deleted from there, e.g. if a message is
ephemeral, but a network outage lasts longer than the ephemeral message timer, the message still
must be sent upon a successful reconnection.
Message.set_text() and Message.get_text() are modified accordingly
to accept String and return String.
Messages which previously contained None text
are now represented as messages with empty text.
Use Message.set_text("".to_string())
instead of Message.set_text(None).
Specifying msg IDs that cannot be loaded in the event payload
results in an error when the UI tries to load the message.
Instead, emit an event without IDs
to make the UI reload the whole messagelist.
Moved custom ToSql trait including Send + Sync from lib.rs to sql.rs.
Replaced most params! and paramsv! macro usage with tuples.
Replaced paramsv! and params_iterv! with params_slice!,
because there is no need to construct a vector.
This further reduces the cognitive overload of having many ways to do
something. The same is very easily done using composition. Followup
from 82ace72527.
To handle backups the UIs have to make sure they do stop the IO
scheduler and also don't accidentally restart it while working on it.
Since they have to call start_io from a bunch of locations this can be
a bit difficult to manage.
This introduces a mechanism for the core to pause IO for some time,
which is used by the imex function. It interacts well with other
calls to dc_start_io() and dc_stop_io() making sure that when resumed
the scheduler will be running or not as the latest calls to them.
This was a little more invasive then hoped due to the scheduler. The
additional abstraction of the scheduler on the context seems a nice
improvement though.
With existing approach of constructing
the SQL query dynamically I get errors like this:
ephemeral.rs:575: update failed: too many SQL variables: Error code 1: SQL error or missing database
In my case it is trying to delete 143658 messages.
This is the result of importing a Desktop backup
and enabling device auto-deletion on the phone.
Current SQLite limit is 32766 variables
as stated in <https://www.sqlite.org/limits.html>.
Gmail archives messages marked as `\Deleted` by default if those messages aren't in the Trash. But
if move them to the Trash instead, they will be auto-deleted in 30 days.
get_chat_msgs() function is split into new get_chat_msgs() without flags
and get_chat_msgs_ex() which accepts booleans instead of bitflags.
FFI call dc_get_chat_msgs() is still using bitflags for compatibility.
JSON-RPC calls get_message_ids() and get_message_list_items()
accept booleans instead of bitflags now.
