Message-IDs are now retrieved only during fetching and saved into imap
table. dc_receive_imf_inner does not attempt to extract the Message-ID
anymore.
For messages without Message-ID the ID is now generated in
imap::fetch_new_messages rather than dc_receive_imf_inner,
so the same ID is used in the imap table (maintained by the imap
module) and msgs table (maintained by dc_receive_imf module).
Message-ID generation based on the Date, From and To field hashing has
been replaced with a simple dc_create_id() to avoid retrieving Date,
From, and To fields in the imap module, as it's hard to test that it
stays compatible between Delta Chat versions in this module. This
breaks jump-to-quote for quoted messages without Message-ID, which is
not critical.
Also prefetch X-Microsoft-Original-Message-ID, so retrieval of
duplicate messages with X-Microsoft-Original-Message-ID can be skipped
like it is done for messages with Message-ID header.
This moves most common headers like From, To, Subject etc. defined in
the Internet Message Format standard at the top of the message
in the same order as used in RFC 5322.
When `smtp_send` returns `Status::Finished`,
the message should be removed from the queue even in case of
failure, such as a permanent error.
In addition to this bugfix, move the retry count increase to
the beginning of `send_msg_to_smtp` to ensure no message is
retried infinitely even in case of similar bugs.
pytest-timeout already handles all deadlocks and is configurable with
--timeout option. With this change it is possible to disable timeout
with --timeout 0 to run tests on extremely slow connections.
Also change how NO response is treated. NO response means there is an
error moving/copying the messages. When there are no matching
messages, the response is "OK No matching messages, so nothing copied"
according to some RFC 9051 examples.
- do not attempt to mark reserved meessages as seen when
messages with empty Message-ID are marked as seen on IMAP
- do not reconnect on Seen flag synchronization failures This avoid
reconnection loops in case of permanent errors in `sync_seen_flags`
Synchronizing seen flags doubles the time required to scan all
folders. Delta Chat only marks messages as Seen on Inbox or DeltaChat,
so there is no need to check for Seen flag on other folders.
This operation takes roughly 0.3 s on a moderate size database.
Calling it once before scanning all folders and scanning
the watched folder instead of each time after downloading
a message from a folder speeds up IMAP loop.
This operation takes roughly 0.3 s on a moderate size database.
Calling it once before scanning all folders and scanning
the watched folder instead of each time after downloading
a message from a folder speeds up IMAP loop.
To create encrypted account with account manager, call
dc_accounts_add_closed_account(). Open this account with
dc_context_open() using the passphrase you want to use for encryption.
When application is loaded next time and account manager is created,
it will open all accounts that have no passphrase set. For encrypted
accounts dc_context_is_open() will return 0. To open them, call
dc_context_open() with the correct passphrase. After opening, call
dc_context_start_io() on this account or just dc_accounts_start_io()
to start all accounts that are not started yet.
Support for legacy SQLite-based backup format is removed in this
commit.
- Replace .ok_or_else() and .map_err() with anyhow::Context where possible.
- Use .context() to check Option for None when it's an error
- Resultify Chatlist.get_chat_id()
- Add useful .context() to some errors
- IMAP error handling cleanup
Seen status is only synchronized on servers supporting IMAP CONDSTORE
extension. At the end of fetch loop iteration, flags are fetched for
all messages modified since previous synchronization and highest
modification sequence is stored into `imap_sync` table.
It slows down dc_get_chat_msgs() from ~50ms to seconds on Android. It
is disabled by default in SQLCipher 4.5.0, but currently SQLCipher 4.4.3
is bundled so this PRAGMA has to be disabled manually.
This is an irrecoverable error, dc_receive_imf must not fail on it
as it prevents last seen UID from advancing, so the same message
is prefetched on each iteration of IMAP loop.
Previously default of `auto_vacuum=NONE` was used.
`PRAGMA auto_vacuum=INCERMENTAL` allows running `PRAGMA
incremental_vacuum`. Unlike `VACUUM`, `PRAGMA incremental_vacuum`
frees unused pages without rebuilding the whole database, like the
`VACUUM` does, so it does not require additional disk space to build a
vacuumed copy of the database. This may even free enough space on the
disk to run `VACUUM` afterwards.
New setting will only be enabled for new databases or if the `VACUUM`
command runs successfully. Currently `VACUUM` is only executed on
backup export, but may fail nevertheless if there is not enough space
on the disk.
Also try to run `PRAGMA incremental_vacuum` during housekeeping. It
may not be the best strategy, but likely does not make any difference
under normal usage when the database only grows and there are no free
pages. Free pages are created only if enough data is deleted to free
at least one database page of 4096 bytes, for example when automatic
deletion of messages is deleted for the first time. In the future if
more data is placed into the database, like avatars and other blobs,
it may be necessary to revise this strategy, for example to keep some
free pages instead of removing all of them each time by querying
`PRAGMA freelist_pages` and running `PRAGMA incremental_vacuum(N)`.
Also resultified `get_watched_folders`.
Python test `test_moved_markseen` is modified to test using incoming
message instead of BCC-self message, because BCC-self message is
detected immediately in the Inbox.
`imap` table maps Message-IDs to UIDs on the server. `dc_receive_imf`
no longer gets the UID of the message as an argument and does not
insert the folder and UID of the message into the `msgs`
table. `server_folder` and `server_uid` columns in `msgs` table are
deprecated.
MoveMsg and DeleteMsgOnImap jobs are removed. Now messages are moved
and deleted only in the `fetch_move_delete` procedure that consults
the `target` column of the `imap` table to determine where the message
should go.
Where the message should go is determined after prefetching by the
`imap::target_folder()` procedure. Messages are only downloaded once
they reach their target folder to avoid race conditions in multidevice
setting, such as:
1. One device trying to FETCH the message while the other tries to
MOVE it.
2. One device marking the message as \Seen in the Inbox while the
other has already copied unseen message to the Movebox and is going to
delete the \Seen message in the Inbox.
3. Device downloads the message from the Inbox while there are newer
messages in the Movebox placed there by the other device, thus
processing the messages out of order.
- Check that member modifying the group is in the group themselves.
It is not allowed to readd yourself to the group or remove someone without being in the group themselves.
- Unify code for group member addition and removal.
Removing a member now recreates the group member list from the To: field.
Removed member from the Chat-Group-Member-Removed was in the To: field in previous Delta Chat versions,
so it is excluded explicity. New versions of Delta Chat put removed member in Bcc: instead.
- Apply avatar changes after updating the group member list.
This allows to check that the contact modifying the avatar is actually a group member.
They contain X-MSExch-Correlation-Key header, but no
Original-Message-ID, so they cannot be used to find the original
message, but we want to recognize them as MDN nevertheless to assign
them to the trash chat.
ASCII armored keys can be easily generated with `sq key generate` and
used to encrypt and decrypt test messages with `sq` and `gpg` without
converting them to binary using `base64 -d` first.
This function is used to lookup the chat by `References` and
`In-Reply-To` header, so it does not make sense to return trashed
message when there is another non-trashed message in one of these
headers with a real chat ID.
`indexmap` is a large dependency (4K SLoC) containing `unsafe` code.
Contact IDs are now passed around as a Vec<u32> or &[u32].
QUOTA roots are now sorted by name instead of perserving original order.
It was there since the C core, labeled with "/* last_seen is for
future use */" but never actually used. The comment was lost during
the translation from C to Rust.
`cutil.from_dc_charpointer()` is guaranteed to return `str`, while
`cutil.from_optional_dc_charpointer()` may return `None` if C function
returns `NULL`.
It was only used in two places. Rather than adding `use
pretty_assertions::*` everywhere, it's easier to remove, and it
removes two additional dependency crates.
link2xt