Commit Graph

2388 Commits

Author SHA1 Message Date
Alexander Krotov
270d18a88a Move prune_tombstones() to sql:: and call from housekeeping() 2020-04-01 20:06:27 +03:00
Alexander Krotov
25f8a735a9 get_chat_msgs: remove locally expired messages
Expired messages are hidden right before retrieving messages from the
database, so expired messages are not shown to the user.
2020-04-01 20:06:27 +03:00
Alexander Krotov
9eb672ea17 Move removal of chat message tombstones to a separate function
DELETE operation may be slow compared to UPDATE. It is better to do in
a separate job.
2020-04-01 20:06:27 +03:00
Alexander Krotov
9febc762da Add chat::delete_device_expired_messages() function 2020-04-01 20:06:27 +03:00
Alexander Krotov
4b742c220c Add Context.get_config_delete_device_after() method
In contrast to get_config_delete_server_after(), value 1 does not mean
"delete at once", because it does not make sense to delete messages
immediately after receivning them.
2020-04-01 20:06:27 +03:00
Alexander Krotov
9d03d441e1 Add Config::DeleteDeviceAfter option 2020-04-01 20:06:27 +03:00
B. Petersen
ff8b249cc6 For now, 'Saved messages' are auto-deleted from the server, but not from device 2020-04-01 20:06:27 +03:00
B. Petersen
248e6ea5e7 make clippy happy 2020-04-01 20:06:27 +03:00
B. Petersen
be0afdebfd target comments of @link2xt, fix ci 2020-04-01 20:06:27 +03:00
B. Petersen
9f19d20344 implement message estimating 2020-04-01 20:06:27 +03:00
B. Petersen
aea8a32ba5 add 'estimatedeletion' to repl tool 2020-04-01 20:06:27 +03:00
B. Petersen
d1a4c82937 prototype ffi for ephemeral messages 2020-04-01 20:06:27 +03:00
Alexander Krotov
4f73812673 Prioritize message deletion over message moving
If Delta Chat goes online and gets an expired message is in the Inbox,
it should delete it instead of moving and then deleting.
2020-04-01 20:06:27 +03:00
Alexander Krotov
33150615a1 Improve logging for server UID updates in precheck_imf()
Log all folders and UIDs and warn about UID changes without folder change.
2020-04-01 20:06:27 +03:00
Alexander Krotov
491f83c86d Remove ServerFolder and ServerUid job parameters
They were used by MarkseenMdnOnImap
2020-04-01 20:06:27 +03:00
Alexander Krotov
41f776763b Remove MarkseenMdnOnImap
MarkseenMdnOnImap stored server folder and UID which are never updated
by update_server_uid. Now hidden entries are created for MDNs, so they
should be handled as ordinary messages.
2020-04-01 20:06:27 +03:00
Alexander Krotov
65fdfac866 Remove unused dest_uid argument from Imap.mv()
It was always set to 0 because we don't know the destination UID.
2020-04-01 20:06:27 +03:00
Alexander Krotov
cb0c00bc6d Log rfc724_mid in DeleteMsgOnImap 2020-04-01 20:06:27 +03:00
Alexander Krotov
ad53678c19 Remove msgs.unlinked column
It is not used anymore.

Database version 64 migration introducing this column is also removed.
2020-04-01 20:06:27 +03:00
Alexander Krotov
62097765a6 update_server_uid: set server_uid even for unlinked messages
Sometimes message deletion job marks message as unlinked without
actually deleting it. It is possible if the message was already moved
into another folder, possibly by second device, but not detected there
yet. It should be detected later in the other folder, and the
server_uid in the database should be set.

Since introduction of add_imap_deletion_jobs() any expired message
record will be marked as unlinked eventually, because another message
deletion job will be scheduled even if update_server_uid() resurrects
the message once.
2020-04-01 20:06:27 +03:00
Alexander Krotov
efb7280e99 Do not schedule delayed DeleteMsgOnImap jobs
All IMAP deletion jobs are scheduled either immediately, or later by
job::add_imap_deletion_jobs().
2020-04-01 20:06:27 +03:00
Alexander Krotov
bdb2a47743 Revert "Automatically delete messages in 2 weeks window only"
This may result in messages not being deleted. Optimization and
traffic-saving is postponed for later, one idea is to optimize message
deletion to avoid checking if Message-ID on the server matches
Message-ID in the database.
2020-04-01 20:06:27 +03:00
Alexander Krotov
c4677190be Postpone DeleteMsgOnImap on error
If job returns Status::Finished, it will be deleted. Then
add_imap_deletion_jobs will recreate it immediately if the message is
expired. To actually backoff the job, we should postpone it instead of
removing.
2020-04-01 20:06:27 +03:00
Alexander Krotov
055aba189c Automatically delete messages in 2 weeks window only
This is to avoid creating thousands of jobs when user enables
"delete_server_after" setting for the first time.

If device is offline for more than 2 weeks, some messages may not be
deleted.
2020-04-01 20:06:27 +03:00
Alexander Krotov
314c3d5e78 add_imap_deletion_jobs: check only for DeleteMsgOnImap jobs
Other jobs may have different meaning for foreign_id.

Also removed " \" at the end of lines.
2020-04-01 20:06:27 +03:00
Alexander Krotov
6db03356b5 Return AlreadyDone from Imap.delete_msg if message is gone
This way DeleteMsgOnImap will remove invalid server_uid from the database.
2020-04-01 20:06:27 +03:00
Alexander Krotov
28af919b09 Create DeleteMsgOnImap jobs before performing IMAP jobs
When "delete_server_after" setting is configured, postponed
DeleteMsgOnImap jobs are created for incoming messages.

This commit adds job::add_imap_deletion_jobs function which creates
DeleteMsgOnImap jobs right before performing IMAP jobs. This way even
messages that expired when the setting was disabled are going to be
deleted.

Job creation on message reception is unnecessary now, and even harmful
because it will create jobs with an expiration time which may later be
reduced. It is planned to be removed in following commits.
2020-04-01 20:06:27 +03:00
Alexander Krotov
8f7a456a39 Use 0 value for "delete_server_after" default.
Now 0 means "never delete", 1 means "delete at once" and other values
indicate the number of seconds after which them message should be
deleted from the server.

Configuration value interpretation is moved into
Context.get_config_delete_server_after() function.
2020-04-01 20:06:27 +03:00
Alexander Krotov
5b3bec1aac Create entries in msgs table for MDNs
At least one entry is required for DeleteMsgOnImap job. Additionally,
adding a hidden entry makes it possible to avoid redownloading the
message if it gets a new UID on the server.
2020-04-01 20:06:27 +03:00
Alexander Krotov
f2aa17c9d0 Resultify MsgId.delete_from_db() 2020-04-01 20:06:27 +03:00
Alexander Krotov
bc06b9e051 Do not send BCC-Self copy if we are going to remove it immediately 2020-04-01 20:06:27 +03:00
Alexander Krotov
6d216af507 Delete BCC-self messages after "delete_server_after" time 2020-04-01 20:06:27 +03:00
Alexander Krotov
b2f1d9f376 Do not remove rfc724_mid for unlinked messages
Message-ID is used to send read receipts. Instead, add a separate
"unlinked" column.
2020-04-01 20:06:27 +03:00
Alexander Krotov
a653e469f2 Add user-configurable option "delete_server_after"
The option sets timer in seconds after which all parts of the message
are deleted from the server.
2020-04-01 20:06:27 +03:00
Alexander Krotov
4f4241ba3a dc_receive_imf: delete all message parts if message should be deleted
DeleteMsgOnImap deletes files from the server only when the last part
is deleted. Removing only the first part of the hidden or trashed
message does not result in message deletion.
2020-04-01 20:06:27 +03:00
Alexander Krotov
2cf9c68040 Implement MsgId.unlink() and use it in DeleteMsgOnImap
Currently only trashed or hidden messages are deleted by
DeleteMsgOnImap, so it is safe to remove database records.

It is planned to delete messages on IMAP server after
user-configurable time to cleanup the server even for messages
displayed in chats. For such messages, we unlink them from the
Message-ID, but keep the database record to display them.
2020-04-01 20:06:27 +03:00
Alexander Krotov
cc0f977d6f Document rfc724_mid_cnt 2020-04-01 20:06:27 +03:00
Alexander Krotov
7879952fde Delete MDNs first in MsgId.delete_from_db() 2020-04-01 20:06:27 +03:00
Alexander Krotov
4452cab987 Turn Message::Delete_from_db into MsgId method 2020-04-01 20:06:27 +03:00
Alexander Krotov
98bd64621a Refactor Imap.delete_msg() error handling
Mutable UID reference is removed. Instead, ImapActionResult is
returned immediately.

If message has changed and UID does not correspond to expected
message, ImapActionResult::Failed is returned. Temprorary IMAP errors
result in ImapActionResult::RetryLater.

On the job side, ImapActionResult::RetryLater does not result in
immediate job retry anymore. Instead, job is retried with a backoff.
2020-04-01 20:06:27 +03:00
Alexander Krotov
c1c769ceb0 Add MsgId.trash() and use it to delete messages locally
In addition to moving the message into trash chat, this function
removes message text to make sure the message does not remain in the
database. Only the information necessary to delete message from the
server and avoid redownloading it should be kept, such as Message-Id
and IMAP UID.
2020-04-01 20:06:27 +03:00
Alexander Krotov
d64e55c66f delete_msgs: remove explicit .iter() 2020-04-01 20:06:27 +03:00
Alexander Krotov
76fc84be37 job: document DeleteMsgOnImap 2020-04-01 20:06:27 +03:00
bjoern
8cd5f5990e Merge pull request #1295 from deltachat/draft_group_consistency
draft doc for healing group-inconsistencies
2020-03-31 18:27:57 +02:00
B. Petersen
6ffe54d68f do no longer ignore keypair generation test, due to the ecc-move, it is no longer expensive 2020-03-31 18:20:40 +02:00
Alexander Krotov
d78ea882c8 Update mailparse to 0.12 2020-03-31 18:11:44 +02:00
Alexander Krotov
958802a233 Add system message only if contact was removed successfully 2020-03-31 18:08:19 +02:00
bjoern
00b02efdc2 Merge pull request #1367 from deltachat/no-forward-to-device-chat
hide the device-chat from default forward-to-lists
2020-03-31 16:03:53 +02:00
Floris Bruynooghe
50569f12f5 Remove unsafe CString::yolo from ffi
CString::yolo was still used in the ffi, this was an unsafe
transitional thing.  To remove it there were two choices: 1. make
errors in creating CStrings hard errors or 2. try and be as lenient as
possible.  Given the to_string_lossy() convention adopted in the ffi
this choose the lenient option and simply skips over embedded null
bytes, leaving the rest of the strings intact.

Thus now CString::new_lossy().  It's only used for .strdup() however
so no longer a public trait.

This also cleans up the public visibility of things in the strings.rs
file:

- Rename StrExt/OptStrExt traits to what they actually do: provide
  .strdup() -> Strdup/OptStrdup.

- dc_strdup() should be an implementation detail, replace all usages
  with Strdup.strdup() method.

- Only allow visibility inside the crate for all things.

- Reduce visibility to only the module for things not used in lib.rs.
2020-03-31 09:12:41 +02:00
B. Petersen
8aa4ceb570 hide the device-chat from default forward-to-lists 2020-03-30 15:59:07 +02:00