Commit Graph
100 Commits
Author SHA1 Message Date
Alexander Krotov db5b5d321b clippy: remove redundant imports 2020-04-13 23:02:57 +03:00
Alexander Krotov 76b7e7408a chatlist: remove indexing in get_{chat,msg}_id() 2020-04-13 23:02:57 +03:00
Alexander Krotov 324e5d0258 Use slice pattern to parse KML coordinates 2020-04-13 23:02:57 +03:00
Alexander Krotov d997bbc081 skip_forward_header: get rid of indexing with subslice patterns 2020-04-13 23:02:57 +03:00
Alexander Krotov 1c21d4f356 constants: remove unnecessary parenthesis
Rust warns about them now
2020-04-13 23:02:57 +03:00
Alexander Krotov 5f574cf283 get_next_media: replace indexing with .get() 2020-04-13 23:02:57 +03:00
Alexander Krotov 92f1e6da1e get_next_media: enumerate() instead of indexing 2020-04-13 23:02:57 +03:00
Alexander Krotov 24aa3c781b Remove indexing in Aheader::from_str 2020-04-13 23:02:57 +03:00
Alexander Krotov 32bd6109e3 Update rust-toolchain and proptest
It is required for stabilized subslice patterns.

proptest 0.9.6 fixes compatibility with the latest rustc nightly.
2020-04-13 23:02:57 +03:00
Alexander Krotov e9967c32e6 chat: filter parent messages by state instead of UID
Since introduction of server message deletion, message may not have
server UID even when it is not a draft or pending message. To handle
such messages correctly, we explicitly check message state.
2020-04-13 17:38:42 +02:00
Alexander Krotov 493a213d41 sql: move QueryReturnedNoRows and NULL handling into query_row_optional() 2020-04-13 17:38:42 +02:00
Alexander Krotov f65dbee74b Add basic test for ChatId.parent_is_encrypted() 2020-04-13 17:38:42 +02:00
Alexander Krotov f51fd1267f chat: move parent_query() and related methods from Chat to ChatId 2020-04-13 17:38:42 +02:00
Alexander Krotov d31265895d Update rPGP to 0.5.2 2020-04-03 12:57:13 +03:00
Alexander Krotov 9c2a3b8a82 Chatlist::try_load: make hide_device_expired_messages errors non-fatal 2020-04-03 12:13:42 +03:00
Alexander Krotov 916fab7d4b hide_device_expired_messages: allow missing self or device chat 2020-04-03 12:12:56 +03:00
Alexander Krotov 3163ef87c6 imap: display errors with {}, not {:?} 2020-04-01 23:27:15 +03:00
Alexander Krotov 1934181b52 Terminate new SQL statement lines with \
This way rustfmt can change indentation. However, care should be taken
to keep a space before each \
2020-04-01 20:06:27 +03:00
Alexander Krotov 1b815a7d96 Display an error if message cannot be trashed 2020-04-01 20:06:27 +03:00
Alexander Krotov 4e0a08106d ChatId.get_param: remove unnecessary type annotation 2020-04-01 20:06:27 +03:00
Alexander Krotov e8cc739fbd Reload chatlist when "delete_device_after" is set 2020-04-01 20:06:27 +03:00
Alexander Krotov fc57cbfb49 Refactor hiding of expired messages
Now there is only one function: hide_device_expired_messages().

If any messages are hidden event DC_EVENT_MSGS_CHANGED(0,0) is emitted
now, which is more correct than DC_EVENT_CHAT_MODIFIED and also triggers
chatlist reload.
2020-04-01 20:06:27 +03:00
Alexander Krotov 7522fec044 Do not emit "chat modified" events when loading chatlist
Otherwise chatlist will be loaded twice, once right after hiding expired
messages, and once in response to event emitted by try_load().
2020-04-01 20:06:27 +03:00
Alexander Krotov 237dabb907 Do not hide hidden messages in ChatId.delete_device_expired_messages()
This prevents infinite event loop, when chatlist is reloaded in response
to event, and event is emitted by hiding messages before chatlist reload.
2020-04-01 20:06:27 +03:00
Alexander Krotov 3686048ab6 chatlist: hide all expired messages before loading 2020-04-01 20:06:27 +03:00
Alexander Krotov 2bf4c5d7e7 Emit "chat modified" event when messages expire 2020-04-01 20:06:27 +03:00
Alexander Krotov 051d80b2f3 Move delete_device_expired_messages() to ChatId
This allows to check if chat is a self-talk or device chat.

Messages are deleted only in viewed chats now.
2020-04-01 20:06:27 +03:00
Alexander Krotov adaa1e856c chat: add ChatId.is_self_talk() and ChatId.is_device_talk() 2020-04-01 20:06:27 +03:00
Alexander Krotov 4daa57c98e delete_device_expired_messages: set text to DELETED for hidden messages
This makes debugging easier: DELETED messages should never be shown.
2020-04-01 20:06:27 +03:00
Alexander Krotov 7e67b2cbb3 Do not delete messages from special chats 2020-04-01 20:06:27 +03:00
Alexander Krotov 270d18a88a Move prune_tombstones() to sql:: and call from housekeeping() 2020-04-01 20:06:27 +03:00
Alexander Krotov 25f8a735a9 get_chat_msgs: remove locally expired messages
Expired messages are hidden right before retrieving messages from the
database, so expired messages are not shown to the user.
2020-04-01 20:06:27 +03:00
Alexander Krotov 9eb672ea17 Move removal of chat message tombstones to a separate function
DELETE operation may be slow compared to UPDATE. It is better to do in
a separate job.
2020-04-01 20:06:27 +03:00
Alexander Krotov 9febc762da Add chat::delete_device_expired_messages() function 2020-04-01 20:06:27 +03:00
Alexander Krotov 4b742c220c Add Context.get_config_delete_device_after() method
In contrast to get_config_delete_server_after(), value 1 does not mean
"delete at once", because it does not make sense to delete messages
immediately after receivning them.
2020-04-01 20:06:27 +03:00
Alexander Krotov 9d03d441e1 Add Config::DeleteDeviceAfter option 2020-04-01 20:06:27 +03:00
Alexander Krotov 4f73812673 Prioritize message deletion over message moving
If Delta Chat goes online and gets an expired message is in the Inbox,
it should delete it instead of moving and then deleting.
2020-04-01 20:06:27 +03:00
Alexander Krotov 33150615a1 Improve logging for server UID updates in precheck_imf()
Log all folders and UIDs and warn about UID changes without folder change.
2020-04-01 20:06:27 +03:00
Alexander Krotov 491f83c86d Remove ServerFolder and ServerUid job parameters
They were used by MarkseenMdnOnImap
2020-04-01 20:06:27 +03:00
Alexander Krotov 41f776763b Remove MarkseenMdnOnImap
MarkseenMdnOnImap stored server folder and UID which are never updated
by update_server_uid. Now hidden entries are created for MDNs, so they
should be handled as ordinary messages.
2020-04-01 20:06:27 +03:00
Alexander Krotov 65fdfac866 Remove unused dest_uid argument from Imap.mv()
It was always set to 0 because we don't know the destination UID.
2020-04-01 20:06:27 +03:00
Alexander Krotov cb0c00bc6d Log rfc724_mid in DeleteMsgOnImap 2020-04-01 20:06:27 +03:00
Alexander Krotov ad53678c19 Remove msgs.unlinked column
It is not used anymore.

Database version 64 migration introducing this column is also removed.
2020-04-01 20:06:27 +03:00
Alexander Krotov 62097765a6 update_server_uid: set server_uid even for unlinked messages
Sometimes message deletion job marks message as unlinked without
actually deleting it. It is possible if the message was already moved
into another folder, possibly by second device, but not detected there
yet. It should be detected later in the other folder, and the
server_uid in the database should be set.

Since introduction of add_imap_deletion_jobs() any expired message
record will be marked as unlinked eventually, because another message
deletion job will be scheduled even if update_server_uid() resurrects
the message once.
2020-04-01 20:06:27 +03:00
Alexander Krotov efb7280e99 Do not schedule delayed DeleteMsgOnImap jobs
All IMAP deletion jobs are scheduled either immediately, or later by
job::add_imap_deletion_jobs().
2020-04-01 20:06:27 +03:00
Alexander Krotov bdb2a47743 Revert "Automatically delete messages in 2 weeks window only"
This may result in messages not being deleted. Optimization and
traffic-saving is postponed for later, one idea is to optimize message
deletion to avoid checking if Message-ID on the server matches
Message-ID in the database.
2020-04-01 20:06:27 +03:00
Alexander Krotov c4677190be Postpone DeleteMsgOnImap on error
If job returns Status::Finished, it will be deleted. Then
add_imap_deletion_jobs will recreate it immediately if the message is
expired. To actually backoff the job, we should postpone it instead of
removing.
2020-04-01 20:06:27 +03:00
Alexander Krotov 055aba189c Automatically delete messages in 2 weeks window only
This is to avoid creating thousands of jobs when user enables
"delete_server_after" setting for the first time.

If device is offline for more than 2 weeks, some messages may not be
deleted.
2020-04-01 20:06:27 +03:00
Alexander Krotov 314c3d5e78 add_imap_deletion_jobs: check only for DeleteMsgOnImap jobs
Other jobs may have different meaning for foreign_id.

Also removed " \" at the end of lines.
2020-04-01 20:06:27 +03:00
Alexander Krotov 6db03356b5 Return AlreadyDone from Imap.delete_msg if message is gone
This way DeleteMsgOnImap will remove invalid server_uid from the database.
2020-04-01 20:06:27 +03:00
Alexander Krotov 28af919b09 Create DeleteMsgOnImap jobs before performing IMAP jobs
When "delete_server_after" setting is configured, postponed
DeleteMsgOnImap jobs are created for incoming messages.

This commit adds job::add_imap_deletion_jobs function which creates
DeleteMsgOnImap jobs right before performing IMAP jobs. This way even
messages that expired when the setting was disabled are going to be
deleted.

Job creation on message reception is unnecessary now, and even harmful
because it will create jobs with an expiration time which may later be
reduced. It is planned to be removed in following commits.
2020-04-01 20:06:27 +03:00
Alexander Krotov 8f7a456a39 Use 0 value for "delete_server_after" default.
Now 0 means "never delete", 1 means "delete at once" and other values
indicate the number of seconds after which them message should be
deleted from the server.

Configuration value interpretation is moved into
Context.get_config_delete_server_after() function.
2020-04-01 20:06:27 +03:00
Alexander Krotov 5b3bec1aac Create entries in msgs table for MDNs
At least one entry is required for DeleteMsgOnImap job. Additionally,
adding a hidden entry makes it possible to avoid redownloading the
message if it gets a new UID on the server.
2020-04-01 20:06:27 +03:00
Alexander Krotov f2aa17c9d0 Resultify MsgId.delete_from_db() 2020-04-01 20:06:27 +03:00
Alexander Krotov bc06b9e051 Do not send BCC-Self copy if we are going to remove it immediately 2020-04-01 20:06:27 +03:00
Alexander Krotov 6d216af507 Delete BCC-self messages after "delete_server_after" time 2020-04-01 20:06:27 +03:00
Alexander Krotov b2f1d9f376 Do not remove rfc724_mid for unlinked messages
Message-ID is used to send read receipts. Instead, add a separate
"unlinked" column.
2020-04-01 20:06:27 +03:00
Alexander Krotov a653e469f2 Add user-configurable option "delete_server_after"
The option sets timer in seconds after which all parts of the message
are deleted from the server.
2020-04-01 20:06:27 +03:00
Alexander Krotov 4f4241ba3a dc_receive_imf: delete all message parts if message should be deleted
DeleteMsgOnImap deletes files from the server only when the last part
is deleted. Removing only the first part of the hidden or trashed
message does not result in message deletion.
2020-04-01 20:06:27 +03:00
Alexander Krotov 2cf9c68040 Implement MsgId.unlink() and use it in DeleteMsgOnImap
Currently only trashed or hidden messages are deleted by
DeleteMsgOnImap, so it is safe to remove database records.

It is planned to delete messages on IMAP server after
user-configurable time to cleanup the server even for messages
displayed in chats. For such messages, we unlink them from the
Message-ID, but keep the database record to display them.
2020-04-01 20:06:27 +03:00
Alexander Krotov cc0f977d6f Document rfc724_mid_cnt 2020-04-01 20:06:27 +03:00
Alexander Krotov 7879952fde Delete MDNs first in MsgId.delete_from_db() 2020-04-01 20:06:27 +03:00
Alexander Krotov 4452cab987 Turn Message::Delete_from_db into MsgId method 2020-04-01 20:06:27 +03:00
Alexander Krotov 98bd64621a Refactor Imap.delete_msg() error handling
Mutable UID reference is removed. Instead, ImapActionResult is
returned immediately.

If message has changed and UID does not correspond to expected
message, ImapActionResult::Failed is returned. Temprorary IMAP errors
result in ImapActionResult::RetryLater.

On the job side, ImapActionResult::RetryLater does not result in
immediate job retry anymore. Instead, job is retried with a backoff.
2020-04-01 20:06:27 +03:00
Alexander Krotov c1c769ceb0 Add MsgId.trash() and use it to delete messages locally
In addition to moving the message into trash chat, this function
removes message text to make sure the message does not remain in the
database. Only the information necessary to delete message from the
server and avoid redownloading it should be kept, such as Message-Id
and IMAP UID.
2020-04-01 20:06:27 +03:00
Alexander Krotov d64e55c66f delete_msgs: remove explicit .iter() 2020-04-01 20:06:27 +03:00
Alexander Krotov 76fc84be37 job: document DeleteMsgOnImap 2020-04-01 20:06:27 +03:00
Alexander Krotov d78ea882c8 Update mailparse to 0.12 2020-03-31 18:11:44 +02:00
Alexander Krotov 958802a233 Add system message only if contact was removed successfully 2020-03-31 18:08:19 +02:00
Alexander Krotov a7afbf85ad Replace test Alice key with ed25519 key
Autocrypt Setup Message does not contain "==" anymore since key length
has changed.
2020-03-29 19:52:45 +03:00
Alexander Krotov a770d75e2e Fix condition in normalize_name() 2020-03-22 00:11:41 +00:00
Alexander Krotov a330104e9b Add tests that crash normalize_name() 2020-03-22 00:11:41 +00:00
Alexander Krotov e7e4821804 Reset group member list when we are added to the group
This makes test_synchronize_member_list_on_group_rejoin pass.
2020-03-19 17:03:31 +03:00
Alexander Krotov 9654802acc Add failing group rejoin test 2020-03-19 17:03:31 +03:00
Alexander Krotov 4714fb6887 Reset server_folder and server_uid in Imap.empty_folder()
This way we avoid trying to delete already deleted messages in the future.
2020-03-09 01:21:12 +03:00
Alexander Krotov 82b4647b95 Update dc_truncate() comment 2020-03-02 23:09:38 +03:00
Alexander Krotov 0c770a8b37 Fix Origin::AddressBook spelling 2020-03-02 22:57:22 +03:00
Alexander Krotov fbabe27fc1 Make add_or_lookup errors non-fatal in add_address_book() 2020-03-02 14:08:20 +01:00
Alexander Krotov 63baac3c61 Update rPGP to 0.4.1
This version fixes incorrect serialization of ECC session keys with
leading zeros.
2020-03-01 20:52:08 +03:00
Alexander Krotov 3c342339a1 imap: use parse_message_id from mimeparser 2020-02-22 16:41:07 +01:00
Alexander Krotov 33463856c5 Use mailparse::msgidparse to parse Message-IDs 2020-02-22 16:41:07 +01:00
Alexander Krotov 508b8ef2e2 Improve documentation, mostly by hiding behind pub(crate) 2020-02-20 23:37:13 +03:00
Alexander Krotov 840e321dd9 Process Permanent and Transient SMTP errors 2020-02-20 01:30:21 +01:00
Alexander Krotov d9dda44409 Add integration test for RSA and Ed25519 keys
Test that two chat clients using different key types can communicate
using Autocrypt.
2020-02-18 17:51:06 +01:00
Alexander Krotov 7368c01a8f Add key_gen_type config option 2020-02-18 17:51:06 +01:00
Alexander Krotov 21ac5be7ca Change generated key type to Ed25519
rPGP generates EdDSA and and ECDH keys using Ed25519 only, so there is
no need to specify it explicitly anywhere.
2020-02-18 17:51:06 +01:00
Alexander Krotov 66d3440675 Update const.py 2020-02-18 11:58:11 +03:00
Alexander Krotov 6b6be3b03d Fix some "`" code markup 2020-02-18 05:06:11 +03:00
Alexander Krotov bcbf192bbc Remove deprecated Chat-Group-Image header 2020-02-17 13:41:50 +01:00
Alexander Krotov 78d855c5ca Include prefer-encrypt attribute in Autocrypt-Gossip headers 2020-02-17 13:39:32 +01:00
Alexander Krotov 1fa9aa88a8 Search for Flag::Deleted and Flag::Seen with == instead of match 2020-02-17 10:50:34 +01:00
Alexander Krotov 08c77c2668 fetch_single_msg: use if let Some(...) instead of is_empty() 2020-02-17 10:50:34 +01:00
Alexander Krotov 793ebe1b0f imap: move IdleHandle from session.rs to idle.rs 2020-02-17 10:50:34 +01:00
Alexander Krotov 4c42acc7e1 Factor src/imap/session.rs out of src/imap/client.rs 2020-02-17 10:50:34 +01:00
Alexander Krotov 4eb9660bfa Move src/imap_client.rs into src/imap/client.rs 2020-02-17 10:50:34 +01:00
Alexander Krotov 8ed08f701d Do not use grpid to find last inserted row in chats table
Instead, use last_insert_rowid() function to find the row.

There is no race condition in using last_insert_rowid(), because
last_insert_rowid() returns row id last inserted in this connection. As
we hold the connection during the whole transaction, it is impossible
that some other thread will execute INSERT statement in parallel.

This commit is part of the effort to get rid of sql::get_rowid hack and
use transactions more for related SQL statements.
2020-02-17 01:35:56 +03:00
Alexander Krotov 784964efad Make sql::with_conn and sql::start_stmt public
Existing public methods that use these functions, like sql::execute, are
only suitable for executing a single statement.

Sometimes it is useful to execute multiple statements within one
connection, for example to begin a transaction, execute mutliple SELECT
and INSERT queries and commit or rollback the whole transaction.
2020-02-17 01:35:56 +03:00
Alexander Krotov adb96e72b9 Pass mutable Connection reference to with_conn callback
This makes it possible to call .transaction() method on connection.
2020-02-17 01:35:56 +03:00
Alexander Krotov 439c6f7296 Fix a typo 2020-02-17 01:35:56 +03:00
Alexander Krotov 2977ceb459 Turn deltachat::configure functions into Context methods
Now configure module is no longer public. Users should call
Context.configure() and Context.is_configured() methods.

Configure module is completely hidden from documentation unless
--document-private-items option is specified.
2020-02-16 21:17:03 +01:00