Commit Graph

3575 Commits

Author SHA1 Message Date
Hocuri
17acbca576 Correctly clear database cache after import (#4067) 2023-02-21 11:23:34 +00:00
link2xt
92c7cc40d4 sql: replace .get_conn() interface with .call()
.call() interface is safer because it ensures
that blocking operations on SQL connection
are called within tokio::task::block_in_place().

Previously some code called blocking operations
in async context, e.g. add_parts() in receive_imf module.

The underlying implementation of .call()
can later be replaced with an implementation
that does not require block_in_place(),
e.g. a worker pool,
without changing the code using the .call() interface.
2023-02-20 18:54:29 +00:00
Hocuri
710cec1beb Remove show_emails argument from prefetch_should_download() (#4064)
IIRC, this was written this way back when we didn't have config caching,
in order to save database accesses by getting the config outside the
for loop.
2023-02-20 18:06:43 +00:00
link2xt
56d10f7c42 Use transaction in update_blocked_mailinglist_contacts 2023-02-20 17:17:59 +00:00
iequidoo
3b5227c42a Move strict_tls, max_smtp_rcpt_to from Provider to ProviderOptions 2023-02-20 14:09:27 -03:00
iequidoo
604c4fcb71 Delete messages to the Trash folder for Gmail by default (#3957)
Gmail archives messages marked as `\Deleted` by default if those messages aren't in the Trash. But
if move them to the Trash instead, they will be auto-deleted in 30 days.
2023-02-20 14:09:27 -03:00
link2xt
4790ad0478 Merge switching SQLite connection pool from FIFO to LIFO 2023-02-20 16:43:19 +00:00
link2xt
eaa2ef5a44 sql: start transactions as IMMEDIATE
With the introduction of transactions in Contact::add_or_lookup(),
python tests sometimes fail to create contacts with the folowing error:

Cannot create contact: add_or_lookup: database is locked: Error code 5: The database file is locked

`PRAGMA busy_timeout=60000` does not affect
this case as the error is returned before 60 seconds pass.

DEFERRED transactions with write operations need
to be retried from scratch
if they are rolled back
due to a write operation on another connection.

Using IMMEDIATE transactions for writing
is an attempt to fix this problem
without a retry loop.

If we later need DEFERRED transactions,
e.g. for reading a snapshot without locking the database,
we may introduce another function for this.
2023-02-20 15:05:35 +00:00
link2xt
2eeacb0f8a sql: organize connection pool as a stack rather than a queue
When connection pool is organized as a stack,
it always returns most recently used connection.
Because each connection has its own page cache,
using the connection with fresh cache improves performance.

I commented out `oauth2::tests::test_oauth_from_mx`
because it requires network connection,
turned off the Wi-Fi and ran the tests.

Before the change, with a queue:
```
$ hyperfine "cargo test"
Benchmark 1: cargo test
  Time (mean ± σ):     56.424 s ±  4.515 s    [User: 183.181 s, System: 128.156 s]
  Range (min … max):   52.123 s … 68.193 s    10 runs
```

With a stack:
```
$ hyperfine "cargo test"
Benchmark 1: cargo test
  Time (mean ± σ):     29.887 s ±  1.377 s    [User: 101.226 s, System: 45.573 s]
  Range (min … max):   26.591 s … 31.010 s    10 runs
```

On version 1.107.1:
```
$ hyperfine "cargo test"
Benchmark 1: cargo test
  Time (mean ± σ):     43.658 s ±  1.079 s    [User: 202.582 s, System: 50.723 s]
  Range (min … max):   41.531 s … 45.170 s    10 runs
```
2023-02-20 15:00:37 +00:00
link2xt
446214fd7b sql: use transaction in Contact::add_or_lookup() 2023-02-19 23:16:44 +00:00
link2xt
10066b2bc7 sql: use semaphore to limit access to the connection pool
This ensures that if multiple connections are returned
to the pool at the same time, waiters get them in the order
they were placed in the queue.
2023-02-19 17:06:25 +00:00
Hocuri
641d102aba Add dc_msg_set_subject() C FFI (#4057) 2023-02-19 13:42:39 +00:00
link2xt
626ec5e793 Document the meaning of empty Message.subject 2023-02-19 13:05:39 +00:00
link2xt
85517abf58 Derive Debug for Pool 2023-02-19 02:26:26 +00:00
link2xt
75f65b06e8 Run VACUUM on the same connection as sqlcipher_export
Otherwise export_and_import_backup test fails due to SQLITE_SCHEMA error.
2023-02-19 02:26:26 +00:00
link2xt
f2b05ccc29 Reimplement connection pool on top of crossbeam 2023-02-19 02:26:26 +00:00
link2xt
e88f21c010 Remove try_many_times r2d2 bug workaround 2023-02-19 02:26:26 +00:00
link2xt
ed8e2c4818 Replace r2d2 with an own connection pool
New connection pool does not use threads
and does not remove idle connections
or create new connections at runtime.
2023-02-19 02:26:26 +00:00
link2xt
870527de1e Remove r2d2_sqlite dependency 2023-02-17 11:06:17 +00:00
link2xt
a34aeb240a Increase database timeout to 60 seconds 2023-02-16 18:37:47 +00:00
link2xt
ae19c9b331 Add more documentation 2023-02-15 21:56:33 +00:00
link2xt
7d2cca8633 sql: enable auto_vacuum on all connections 2023-02-15 18:59:47 +00:00
link2xt
c52b48b0f5 sql: update version first in migration transactions
In execute_migration transaction first update the version, and only
then execute the rest of the migration. This ensures that transaction
is immediately recognized as write transaction and there is no need
to promote it from read transaction to write transaction later, e.g.
in the case of "DROP TABLE IF EXISTS" that is a read only operation if
the table does not exist. Promoting a read transaction to write
transaction may result in an error if database is locked.
2023-02-15 18:58:44 +00:00
link2xt
893794f4e7 accounts: retry filesystem operations in migrate_account() 2023-02-15 18:57:37 +00:00
link2xt
05f0fe0a88 Derive Default where possible 2023-02-14 21:33:02 +00:00
link2xt
e11d7c0444 Remove unnecessary .into_iter() 2023-02-14 21:04:29 +00:00
link2xt
a203cde400 Remove TestHeader from non-test builds 2023-02-14 11:00:04 +00:00
link2xt
71d9716117 Remove MimeMessage::from_bytes()
It was not used anywhere except the tests.
2023-02-14 10:57:57 +00:00
link2xt
1d81457f38 Remove unused ConfiguredE2EEEnabled key
`e2ee_enabled` is always used without `configured_` prefix.
2023-02-13 21:14:47 +00:00
iequidoo
19d7632be0 Show non-deltachat emails by default for new installations 2023-02-13 14:48:53 -03:00
link2xt
2a5fa9a0d3 Documentation improvements 2023-02-13 14:01:24 +00:00
link2xt
bb702a9342 Remove outdated comment 2023-02-13 10:20:41 +00:00
link2xt
07c510c178 Remove bitflags from get_chat_msgs() interface
get_chat_msgs() function is split into new get_chat_msgs() without flags
and get_chat_msgs_ex() which accepts booleans instead of bitflags.

FFI call dc_get_chat_msgs() is still using bitflags for compatibility.

JSON-RPC calls get_message_ids() and get_message_list_items()
accept booleans instead of bitflags now.
2023-02-12 15:11:04 +00:00
link2xt
fa198c3b5e Use SOCKS5 configuration for HTTP requests 2023-02-10 23:20:11 +00:00
link2xt
c9b2ad4ffa Prefer TLS over STARTTLS during autoconfiguration 2023-02-10 11:28:11 +00:00
Hocuri
fa70d8da09 Re-enable DKIM-checks (#3935)
Re-enable keychange-denying when the From address is wrong

Reverts #3728
Closes #3735
Reopens #3700
2023-02-07 17:07:43 +01:00
link2xt
cd293e6f49 Update async-smtp to 0.8 2023-02-03 11:36:58 +00:00
link2xt
59f5cbe6f1 Merge from stable branch 2023-02-01 15:36:26 +00:00
link2xt
74abb82de2 Log server security (TLS/STARTTLS/plain) type 2023-02-01 00:01:47 +00:00
link2xt
e318f5c697 Simplify unset_empty() 2023-01-31 23:58:38 +00:00
link2xt
b62c61329a Update to base64 0.21 2023-01-31 19:24:20 +00:00
link2xt
f342dc8196 Update for new toml API 2023-01-31 13:12:54 +00:00
link2xt
fcf73165ed Inline format arguments
This feature has been stable since Rust 1.58.0.
2023-01-30 11:50:11 +03:00
link2xt
c911f1262a Remove ContextError type
Using anyhow instead.
2023-01-29 00:36:07 +00:00
link2xt
ba3e4c5dff Resultify tools::delete_file() 2023-01-29 00:35:15 +00:00
link2xt
ae564ef702 Add documentation 2023-01-28 21:30:43 +00:00
link2xt
7d3a591139 Make chunk_size variable immutable 2023-01-28 12:54:56 +00:00
link2xt
9b3f76ab5a Add remove_subject_prefix() test 2023-01-28 11:22:30 +00:00
link2xt
dab8acc7d8 Replace Result<_, Error> with Result<_> 2023-01-28 11:14:30 +00:00
link2xt
4eda53d5a1 Move SessionStream from imap to net 2023-01-27 15:51:04 +00:00