The fact that `PRAGMA incremental_vacuum` may return a zero-column
SQLITE_ROW result is documented in `sqlite3_data_count()` documentation:
<https://www.sqlite.org/c3ref/data_count.html>
Previously successful auto_vacuum worked,
but resulted in a "Failed to run incremental vacuum" log.
With existing approach of constructing
the SQL query dynamically I get errors like this:
ephemeral.rs:575: update failed: too many SQL variables: Error code 1: SQL error or missing database
In my case it is trying to delete 143658 messages.
This is the result of importing a Desktop backup
and enabling device auto-deletion on the phone.
Current SQLite limit is 32766 variables
as stated in <https://www.sqlite.org/limits.html>.
This way we can test some recently added config options that we don't want to expose in UI like
DeleteToTrash or SignUnencrypted. Note that persistent config options like DeleteToTrash should
remain anyway because they allow fine-grained (per-account) control. Having them matters for tests
also.
`<meta name="color-scheme" content="light dark" />` is a hint to the browsers
that the page can be rendered in light as well as in dark mode
and that the browser can apply corresponding defaults.
as we do not add css colors on our own,
this is sufficient for letting generated html messasge being rendered
in dark mode.
cmp. https://drafts.csswg.org/css-color-adjust/#color-scheme-propcloses#4146
standard footers meanwhile go the "contact status",
so they are no longer a reason to trigger "full message view".
this was already discussed when the HTML view was introduced at #2125
however, forgotten to change when the "contact status" was added at #2218
this pr will result in a cleaner chat view
with less "Show Full Message..." buttons
and will also save some storage
(in fact, i came over that when reviewing #4129 )
We are currently using libsqlite3-sys 0.25.2,
corresponding to SQLcipher 4.5.2
and SQLite 3.39.2.
SQLite supports ALTER TABLE DROP COLUMN since version 3.35.0,
and it has received critical database corruption bugfixes in 3.35.5.
There have been no fixes to it between SQLite 3.36.0 and 3.41.0,
so it appears stable now.
Explicit `prepare_msg` corresponding to `dc_prepare_msg`
is intended only for the case where the message is prepared
before the file is ready. It is not indented for calling
right before send_msg and requires that file path in the blobdir.
With explicit `prepare_msg` removed
all the tests still pass but there is no requirement
that the file is put into blobdir beforehand.
We do not make all transactions
[IMMEDIATE](https://www.sqlite.org/lang_transaction.html#deferred_immediate_and_exclusive_transactions)
for more parallelism -- at least read transactions can be made DEFERRED to run in parallel
w/o any drawbacks. But if we make write transactions DEFERRED also w/o any external locking,
then they are upgraded from read to write ones on the first write statement. This has some
drawbacks:
- If there are other write transactions, we block the thread and the db connection until
upgraded. Also if some reader comes then, it has to get next, less used connection with a
worse per-connection page cache.
- If a transaction is blocked for more than busy_timeout, it fails with SQLITE_BUSY.
- Configuring busy_timeout is not the best way to manage transaction timeouts, we would
prefer it to be integrated with Rust/tokio asyncs. Moreover, SQLite implements waiting
using sleeps.
- If upon a successful upgrade to a write transaction the db has been modified by another
one, the transaction has to be rolled back and retried. It is an extra work in terms of
CPU/battery.
- Maybe minor, but we lose some fairness in servicing write transactions, i.e. we service
them in the order of the first write statement, not in the order they come.
The only pro of making write transactions DEFERRED w/o the external locking is some
parallelism between them. Also we have an option to make write transactions IMMEDIATE, also
w/o the external locking. But then the most of cons above are still valid. Instead, if we
perform all write transactions under an async mutex, the only cons is losing some
parallelism for write transactions.
scripts/update-provider-database.sh checks out the provider database
and updates data.rs file,
making it easier to update.
CI uses this script to check that checked in data.rs
corresponds to the provider database repository.
the slowest test was `test_modify_chat_disordered`;
due to several sleep(), it lasts at least 11 seconds.
however, many of the sleep() are not needed and were added
just to get things done that time.
this pr removes 7 superfluous sleeps,
making the test finish in about 3 seconds,
so that this test is no longer the bottleneck when running
`cargo test` on fast machines.
(this is not only useful to safe some seconds -
but also to notice degradations as of
https://github.com/deltachat/deltachat-core-rust/issues/4051#issuecomment-1436995166 )
Although it does a little for security, it will help to protect from unwanted server-side
modifications and bugs. And now we have a time to test "multipart/signed" messages compatibility
with other MUAs.
This makes DC compatible with "multipart/signed" messages thus allowing switching to them someday
from the current "multipart/mixed" unencrypted message format.
try_decrypt() is a CPU-bound task.
When called from async function,
it should be wrapped in tokio::task::spawn_blocking().
Using tokio::task::spawn_blocking() is difficult here
because of &mail, &private_keyring and &public_keyring borrows,
so we should at least use tokio::task::block_in_place()
to avoid blocking the executor.
.call() interface is safer because it ensures
that blocking operations on SQL connection
are called within tokio::task::block_in_place().
Previously some code called blocking operations
in async context, e.g. add_parts() in receive_imf module.
The underlying implementation of .call()
can later be replaced with an implementation
that does not require block_in_place(),
e.g. a worker pool,
without changing the code using the .call() interface.
IIRC, this was written this way back when we didn't have config caching,
in order to save database accesses by getting the config outside the
for loop.
Gmail archives messages marked as `\Deleted` by default if those messages aren't in the Trash. But
if move them to the Trash instead, they will be auto-deleted in 30 days.
With the introduction of transactions in Contact::add_or_lookup(),
python tests sometimes fail to create contacts with the folowing error:
Cannot create contact: add_or_lookup: database is locked: Error code 5: The database file is locked
`PRAGMA busy_timeout=60000` does not affect
this case as the error is returned before 60 seconds pass.
DEFERRED transactions with write operations need
to be retried from scratch
if they are rolled back
due to a write operation on another connection.
Using IMMEDIATE transactions for writing
is an attempt to fix this problem
without a retry loop.
If we later need DEFERRED transactions,
e.g. for reading a snapshot without locking the database,
we may introduce another function for this.
When connection pool is organized as a stack,
it always returns most recently used connection.
Because each connection has its own page cache,
using the connection with fresh cache improves performance.
I commented out `oauth2::tests::test_oauth_from_mx`
because it requires network connection,
turned off the Wi-Fi and ran the tests.
Before the change, with a queue:
```
$ hyperfine "cargo test"
Benchmark 1: cargo test
Time (mean ± σ): 56.424 s ± 4.515 s [User: 183.181 s, System: 128.156 s]
Range (min … max): 52.123 s … 68.193 s 10 runs
```
With a stack:
```
$ hyperfine "cargo test"
Benchmark 1: cargo test
Time (mean ± σ): 29.887 s ± 1.377 s [User: 101.226 s, System: 45.573 s]
Range (min … max): 26.591 s … 31.010 s 10 runs
```
On version 1.107.1:
```
$ hyperfine "cargo test"
Benchmark 1: cargo test
Time (mean ± σ): 43.658 s ± 1.079 s [User: 202.582 s, System: 50.723 s]
Range (min … max): 41.531 s … 45.170 s 10 runs
```
