Commit Graph

3560 Commits

Author SHA1 Message Date
B. Petersen
11ca698139 allow generated html being rendered in dark mode
`<meta name="color-scheme" content="light dark" />` is a hint to the browsers
that the page can be rendered in light as well as in dark mode
and that the browser can apply corresponding defaults.

as we do not add css colors on our own,
this is sufficient for letting generated html messasge being rendered
in dark mode.

cmp. https://drafts.csswg.org/css-color-adjust/#color-scheme-prop

closes #4146
2023-03-13 11:22:06 +01:00
B. Petersen
b14e59c5f3 "full message view" not needed because of footers
standard footers meanwhile go the "contact status",
so they are no longer a reason to trigger "full message view".

this was already discussed when the HTML view was introduced at #2125
however, forgotten to change when the "contact status" was added at #2218

this pr will result in a cleaner chat view
with less "Show Full Message..." buttons
and will also save some storage
(in fact, i came over that when reviewing #4129 )
2023-03-13 10:57:56 +01:00
link2xt
4d620afdb8 Enable clippy::explicit_into_iter_loop 2023-03-12 14:51:44 +00:00
link2xt
2bd7781276 Enable cloned_instead_of_copied clippy lint 2023-03-12 14:45:46 +00:00
link2xt
a432303576 Remove explicit .iter() and .iter_mut() calls in for loops 2023-03-12 14:06:42 +00:00
link2xt
00791157e2 Drop unused columns
We are currently using libsqlite3-sys 0.25.2,
corresponding to SQLcipher 4.5.2
and SQLite 3.39.2.

SQLite supports ALTER TABLE DROP COLUMN since version 3.35.0,
and it has received critical database corruption bugfixes in 3.35.5.
There have been no fixes to it between SQLite 3.36.0 and 3.41.0,
so it appears stable now.
2023-03-09 14:29:39 +00:00
link2xt
de391155b1 Do not prepare the message explicitly in the test context
Explicit `prepare_msg` corresponding to `dc_prepare_msg`
is intended only for the case where the message is prepared
before the file is ready. It is not indented for calling
right before send_msg and requires that file path in the blobdir.

With explicit `prepare_msg` removed
all the tests still pass but there is no requirement
that the file is put into blobdir beforehand.
2023-03-09 09:37:40 +00:00
link2xt
7e5a8714a0 Add scripts/codespell.sh and spellcheck 2023-03-03 18:40:36 +00:00
iequidoo
627cf20074 Housekeeping: delete the blobs backup dir (#4104) 2023-03-03 11:06:41 -03:00
iequidoo
e767d84bea Switch to DEFERRED transactions
We do not make all transactions
[IMMEDIATE](https://www.sqlite.org/lang_transaction.html#deferred_immediate_and_exclusive_transactions)
for more parallelism -- at least read transactions can be made DEFERRED to run in parallel
w/o any drawbacks. But if we make write transactions DEFERRED also w/o any external locking,
then they are upgraded from read to write ones on the first write statement. This has some
drawbacks:
- If there are other write transactions, we block the thread and the db connection until
  upgraded. Also if some reader comes then, it has to get next, less used connection with a
  worse per-connection page cache.
- If a transaction is blocked for more than busy_timeout, it fails with SQLITE_BUSY.
- Configuring busy_timeout is not the best way to manage transaction timeouts, we would
  prefer it to be integrated with Rust/tokio asyncs. Moreover, SQLite implements waiting
  using sleeps.
- If upon a successful upgrade to a write transaction the db has been modified by another
  one, the transaction has to be rolled back and retried. It is an extra work in terms of
  CPU/battery.
- Maybe minor, but we lose some fairness in servicing write transactions, i.e. we service
  them in the order of the first write statement, not in the order they come.
The only pro of making write transactions DEFERRED w/o the external locking is some
parallelism between them. Also we have an option to make write transactions IMMEDIATE, also
w/o the external locking. But then the most of cons above are still valid. Instead, if we
perform all write transactions under an async mutex, the only cons is losing some
parallelism for write transactions.
2023-02-28 15:05:47 -03:00
link2xt
c79ded1406 Fixes for get_webxdc_blob documentation 2023-02-27 14:36:57 +00:00
link2xt
50d83ff063 Remove ResyncFolders job 2023-02-26 22:31:53 +00:00
link2xt
8e9bb8b06e Check provider database with CI
scripts/update-provider-database.sh checks out the provider database
and updates data.rs file,
making it easier to update.

CI uses this script to check that checked in data.rs
corresponds to the provider database repository.
2023-02-26 21:27:21 +00:00
link2xt
8177070673 Set minimum TLS version to 1.2 2023-02-26 18:48:32 +00:00
B. Petersen
f8e86f4503 make slowest test faster
the slowest test was `test_modify_chat_disordered`;
due to several sleep(), it lasts at least 11 seconds.

however, many of the sleep() are not needed and were added
just to get things done that time.

this pr removes 7 superfluous sleeps,
making the test finish in about 3 seconds,
so that this test is no longer the bottleneck when running
`cargo test` on fast machines.
(this is not only useful to safe some seconds -
but also to notice degradations as of
https://github.com/deltachat/deltachat-core-rust/issues/4051#issuecomment-1436995166 )
2023-02-25 15:37:10 +01:00
iequidoo
d1923d68a5 Add a config option to sign all messages with Autocrypt header (#3986)
Although it does a little for security, it will help to protect from unwanted server-side
modifications and bugs. And now we have a time to test "multipart/signed" messages compatibility
with other MUAs.
2023-02-25 10:30:35 -03:00
iequidoo
89696582ad mimeparser: Handle headers from the signed part of unencrypted signed message
This makes DC compatible with "multipart/signed" messages thus allowing switching to them someday
from the current "multipart/mixed" unencrypted message format.
2023-02-25 10:30:35 -03:00
link2xt
992a6cbfc2 mimeparser: wrap try_decrypt() into block_in_place()
try_decrypt() is a CPU-bound task.
When called from async function,
it should be wrapped in tokio::task::spawn_blocking().
Using tokio::task::spawn_blocking() is difficult here
because of &mail, &private_keyring and &public_keyring borrows,
so we should at least use tokio::task::block_in_place()
to avoid blocking the executor.
2023-02-25 11:00:05 +00:00
link2xt
1450bf5483 Remove Job::delay_seconds()
It always returned 0 for added jobs.
2023-02-25 02:45:22 +00:00
link2xt
9f804c379c Remove always zero argument to Job::new() 2023-02-25 02:42:14 +00:00
link2xt
10e8bcb73e job: remove unused variable 2023-02-25 02:35:08 +00:00
link2xt
7f2ccfb168 job: remove match with a single branch 2023-02-25 02:33:47 +00:00
link2xt
3fc67de35e Stop saving and loading jobs.param column 2023-02-25 02:33:21 +00:00
link2xt
8f0d07b93c Make smeared timestamp creation non-async
Using atomic operations instead,
so create_smeared_timestamp() can be used in sync functions,
such as SQL transactions.
2023-02-25 01:10:56 +00:00
link2xt
064f806d90 Remove UpdateRecentQuota job 2023-02-24 17:49:08 +00:00
B. Petersen
08c9deee04 add test for Context::update_contacts_timestamp(), esp. the condition and the update was not covered before 2023-02-24 16:23:33 +01:00
link2xt
d0a7e5f1b7 Update timestamps in parameters with transactions
This avoids accidentally reverting the changes
to the `param` column done by another thread.
2023-02-24 14:39:59 +00:00
link2xt
a82b09bfc2 Move TLS support to net::tls module 2023-02-24 13:23:17 +00:00
link2xt
e9668b3cfa Cache INSERT statement in receive_imf
It reduces iteration time by ~8% in message reception benchmarks
ran by `cargo criterion --bench receive_emails`.
2023-02-24 10:51:55 +00:00
link2xt
2cc85e6637 Hide some constants from public crate interface 2023-02-23 14:48:25 +00:00
link2xt
ade3d0d4eb Add more documentation comments 2023-02-23 12:55:43 +00:00
link2xt
9e28ee95e5 Document PeerstateChange 2023-02-22 15:10:33 +00:00
Hocuri
7c099c19c8 Re-disable DKIM-checks (#4076) 2023-02-22 16:03:20 +01:00
link2xt
45abaff275 Update provider database 2023-02-21 14:41:13 +00:00
iequidoo
15c9efaa95 Update provider/update.py according to the struct Provider changes 2023-02-21 10:53:15 -03:00
Hocuri
17acbca576 Correctly clear database cache after import (#4067) 2023-02-21 11:23:34 +00:00
link2xt
92c7cc40d4 sql: replace .get_conn() interface with .call()
.call() interface is safer because it ensures
that blocking operations on SQL connection
are called within tokio::task::block_in_place().

Previously some code called blocking operations
in async context, e.g. add_parts() in receive_imf module.

The underlying implementation of .call()
can later be replaced with an implementation
that does not require block_in_place(),
e.g. a worker pool,
without changing the code using the .call() interface.
2023-02-20 18:54:29 +00:00
Hocuri
710cec1beb Remove show_emails argument from prefetch_should_download() (#4064)
IIRC, this was written this way back when we didn't have config caching,
in order to save database accesses by getting the config outside the
for loop.
2023-02-20 18:06:43 +00:00
link2xt
56d10f7c42 Use transaction in update_blocked_mailinglist_contacts 2023-02-20 17:17:59 +00:00
iequidoo
3b5227c42a Move strict_tls, max_smtp_rcpt_to from Provider to ProviderOptions 2023-02-20 14:09:27 -03:00
iequidoo
604c4fcb71 Delete messages to the Trash folder for Gmail by default (#3957)
Gmail archives messages marked as `\Deleted` by default if those messages aren't in the Trash. But
if move them to the Trash instead, they will be auto-deleted in 30 days.
2023-02-20 14:09:27 -03:00
link2xt
4790ad0478 Merge switching SQLite connection pool from FIFO to LIFO 2023-02-20 16:43:19 +00:00
link2xt
eaa2ef5a44 sql: start transactions as IMMEDIATE
With the introduction of transactions in Contact::add_or_lookup(),
python tests sometimes fail to create contacts with the folowing error:

Cannot create contact: add_or_lookup: database is locked: Error code 5: The database file is locked

`PRAGMA busy_timeout=60000` does not affect
this case as the error is returned before 60 seconds pass.

DEFERRED transactions with write operations need
to be retried from scratch
if they are rolled back
due to a write operation on another connection.

Using IMMEDIATE transactions for writing
is an attempt to fix this problem
without a retry loop.

If we later need DEFERRED transactions,
e.g. for reading a snapshot without locking the database,
we may introduce another function for this.
2023-02-20 15:05:35 +00:00
link2xt
2eeacb0f8a sql: organize connection pool as a stack rather than a queue
When connection pool is organized as a stack,
it always returns most recently used connection.
Because each connection has its own page cache,
using the connection with fresh cache improves performance.

I commented out `oauth2::tests::test_oauth_from_mx`
because it requires network connection,
turned off the Wi-Fi and ran the tests.

Before the change, with a queue:
```
$ hyperfine "cargo test"
Benchmark 1: cargo test
  Time (mean ± σ):     56.424 s ±  4.515 s    [User: 183.181 s, System: 128.156 s]
  Range (min … max):   52.123 s … 68.193 s    10 runs
```

With a stack:
```
$ hyperfine "cargo test"
Benchmark 1: cargo test
  Time (mean ± σ):     29.887 s ±  1.377 s    [User: 101.226 s, System: 45.573 s]
  Range (min … max):   26.591 s … 31.010 s    10 runs
```

On version 1.107.1:
```
$ hyperfine "cargo test"
Benchmark 1: cargo test
  Time (mean ± σ):     43.658 s ±  1.079 s    [User: 202.582 s, System: 50.723 s]
  Range (min … max):   41.531 s … 45.170 s    10 runs
```
2023-02-20 15:00:37 +00:00
link2xt
446214fd7b sql: use transaction in Contact::add_or_lookup() 2023-02-19 23:16:44 +00:00
link2xt
10066b2bc7 sql: use semaphore to limit access to the connection pool
This ensures that if multiple connections are returned
to the pool at the same time, waiters get them in the order
they were placed in the queue.
2023-02-19 17:06:25 +00:00
Hocuri
641d102aba Add dc_msg_set_subject() C FFI (#4057) 2023-02-19 13:42:39 +00:00
link2xt
626ec5e793 Document the meaning of empty Message.subject 2023-02-19 13:05:39 +00:00
link2xt
85517abf58 Derive Debug for Pool 2023-02-19 02:26:26 +00:00
link2xt
75f65b06e8 Run VACUUM on the same connection as sqlcipher_export
Otherwise export_and_import_backup test fails due to SQLITE_SCHEMA error.
2023-02-19 02:26:26 +00:00