Commit Graph

187 Commits

Author SHA1 Message Date
bjoern
2cf11bb2ea muted chats stay archived (#3184)
* do not unarchive muted chats on sending/receving messages

* add a test for unarchive muted chats

* update CHANGELOG

* improve test_unarchive_if_muted
2022-04-04 11:02:36 +02:00
link2xt
3d95272707 smtp: retry message sending automatically if loop is not interrupted 2022-04-03 18:55:29 +00:00
Hocuri
918ee47c79 Consider outgoing messages to just one receiver as "private message" (#3177) 2022-04-03 19:19:10 +02:00
link2xt
a8ab7c9c04 smtp: do not try to use stale connections
Previously first try used an old connection and. If using stale
connection, an immediate retry was performed using a new connection.

Now if connection is stale, it is closed immediately without trying to
use it.

This should reduce the delay in cases when old connection is unusable.
2022-04-03 13:11:27 +00:00
link2xt
332892b468 ephemeral: clear more fields in delete_expired_messages
These fields are cleared in other places,
but in the case of delete_device_after setting
only txt column was cleared previously.
2022-04-02 17:10:09 +00:00
holger krekel
c6d901d799 first iteration of faster sorting 2022-04-01 14:33:57 +02:00
link2xt
e9a733a789 Pass better message around instead of mutating mimeparser
This change is aimed at decoupling parsing and
add_parts() stages to eventually separate parsing
from database changes and pipeline message parsing and
decryption.
2022-03-27 11:23:45 +00:00
link2xt
0842e54f52 Add ephemeral_timestamp index for msgs table
This reduced get_chat_msgs() benchmark time from 400ms to 170ms.
2022-03-26 20:19:49 +00:00
link2xt
08d34e41c6 Return results from add_parts() via structure
Replaced mutable out parameters with explicit return of structure.
Also moved all decisions about emitted events out of add_parts(). Chat
ID is removed from created_db_entries as it is the same for all parts.
2022-03-26 16:38:08 +00:00
bjoern
8487eefe46 config_cache fixes (#3145)
* add simple backup export/import test

this test fails on current master
until the context is recrated.

* avoid config_cache races

adds needed SQL-statements to config_cache locking.

otherwise, another thread may alter the database
eg. between SELECT and the config_cache update -
resulting in the wrong value being written to config_cache.

* also update config_cache on initializing tables

VERSION_CFG is also set later, however,
not doing it here will result in bugs when we change DBVERSION at some point.

as this alters only VERSION_CFG and that is executed sequentially anyway,
race conditions between SQL and config_cache
seems not to be an issue in this case.

* clear config_cache after backup import

import replaces the whole database,
so config_cache needs to be invalidated as well.

we do that before import,
so in case a backup is imported only partly,
the cache does not add additional problems.

* update CHANGELOG
2022-03-22 22:46:29 +01:00
link2xt
26ae8accd4 Automatically unblock chats with outgoing messages 2022-03-20 18:03:10 +00:00
Hocuri
321e3e27de Introduce config caching (#3131)
* Introduce config caching

* Changelog

* Update CHANGELOG.md

Co-authored-by: bjoern <r10s@b44t.com>

* Cache a value after reading it

Co-authored-by: bjoern <r10s@b44t.com>
2022-03-21 10:13:43 +00:00
link2xt
83464a882e Optimize markseen_msgs
Use a single SELECT statement for all messages
and start ephemeral timers for all messages at once.
2022-03-20 14:57:14 +00:00
link2xt
d9e9c849e1 imap: do not delete duplicates
Currently if user moves the message into some other folder and then
moves the message back, the message is considered duplicate even
though previous copy was already deleted. This is a common problem
reported by users at least twice.

Keeping duplicates does no harm except for additional storage usage.
If the message is later deleted by the user, all the copies on the
server will be deleted. anyway.
2022-03-19 15:51:17 +00:00
Hocuri
7a56a93028 Fix long filenames containing dots (#3098) 2022-03-16 20:41:24 +01:00
Hocuri
50f13cb84b Set X-Microsoft-Original-Message-ID on outgoing emails for amazonaws (#3077) 2022-03-13 14:39:49 +01:00
link2xt
e48eef7e32 Start ephemeral timer when seen status is synchronized via imap 2022-03-12 19:28:31 +00:00
Hocuri
a907d789d6 Assign replies from different address to two-member-groups (#3119)
Holger had a case where he wrote with someone using a classing MUA.

He opened a two-member-group with this person (which also allowed him to
set the subject).

At some point the other person replied from a different email address.

What he expected: This reply should be sorted into the two-member-group.
What happened: This reply was sorted into the 1:1 chat.

---

I had added the line && chat_contacts.contains(&from_id) months ago when I wrote
this code because it seemed vaguely sensible but without any real
reason. So, let's remove it and see if it creates other problems -
my gut feeling is no.
2022-03-12 10:47:58 +00:00
Hocuri
d9441a6bdd Also resync UIDs in folders that are not configured (#2289) 2022-03-10 16:12:24 +01:00
link2xt
f28fcec81d imap: do not treat messages without Message-ID as duplicates
Message-IDs are now retrieved only during fetching and saved into imap
table. dc_receive_imf_inner does not attempt to extract the Message-ID
anymore.

For messages without Message-ID the ID is now generated in
imap::fetch_new_messages rather than dc_receive_imf_inner,
so the same ID is used in the imap table (maintained by the imap
module) and msgs table (maintained by dc_receive_imf module).

Message-ID generation based on the Date, From and To field hashing has
been replaced with a simple dc_create_id() to avoid retrieving Date,
From, and To fields in the imap module, as it's hard to test that it
stays compatible between Delta Chat versions in this module. This
breaks jump-to-quote for quoted messages without Message-ID, which is
not critical.

Also prefetch X-Microsoft-Original-Message-ID, so retrieval of
duplicate messages with X-Microsoft-Original-Message-ID can be skipped
like it is done for messages with Message-ID header.
2022-03-08 15:23:22 +00:00
bjoern
500e2d62a0 remove sentbox_move (#3111)
* remove SentboxMove

* adapt python test to removed sendbox_move option

* update CHANGELOG
2022-03-08 11:29:45 +01:00
holger krekel
63688a2f95 remove getAllUpdates() and add a typical replicatio API for the update call (#3081)
* (r10s, adb, hpk) remove getAllUpdates() and add a typical replica-API that works with increasing serials.  Streamline docs a bit.

* adapt ffi to new api

* documentation: updates serials may have gaps

* get_webxdc_status_updates() return updates larger than a given serial

* remove status_update_id from status-update-event; it is not needed (ui should update from the last known serial) and easily gets confused with last_serial

* unify wording to 'StatusUpdateSerial'

* remove legacy payload format, all known webxdc should be adapted

* add serial and max_serial to status updates

* avoid races when getting max_serial by avoiding two SQL calls

* update changelog

Co-authored-by: B. Petersen <r10s@b44t.com>
2022-03-04 20:22:48 +01:00
Floris Bruynooghe
ef841b1aa3 Securejoin: store bobstate in database instead of context
The state bob needs to maintain during a secure-join process when
exchanging messages used to be stored on the context.  This means if
the process was killed this state was lost and the securejoin process
would fail.  Moving this state into the database should help this.

This still only allows a single securejoin process at a time, this may
be relaxed in the future.  For now any previous securejoin process
that was running is killed if a new one is started (this was already
the case).

This can remove some of the complexity around BobState handling: since
the state is in the database we can already make state interactions
transactional and correct.  We no longer need the mutex around the
state handling.  This means the BobStateHandle construct that was
handling the interactions between always having a valid state and
handling the mutex is no longer needed, resulting in some nice
simplifications.

Part of #2777.
2022-03-01 23:02:40 +01:00
link2xt
7ab6d95b6c mimefactory: place common IMF headers at the top of the message
This moves most common headers like From, To, Subject etc. defined in
the Internet Message Format standard at the top of the message
in the same order as used in RFC 5322.
2022-02-23 17:51:15 +00:00
link2xt
6c32b89906 smtp: add more logging 2022-02-23 17:04:30 +00:00
bjoern
7c5eb0ae37 prepare 1.76 (#3082)
* update changelog for 1.76.0

* bump version to 1.76.0
2022-02-20 22:58:46 -05:00
Hocuri
19a32cdfd3 Let MS Exchange MDNs mark the In-Reply-To message as read (#3075)
Fix https://github.com/deltachat/deltachat-core-rust/issues/2891
2022-02-17 09:56:00 +01:00
link2xt
d708f386a1 smtp: set message state to failed when retry limit is exceeded 2022-02-13 08:59:52 +00:00
link2xt
242e8e2bb3 smtp: remove the message in case of permanent failure
When `smtp_send` returns `Status::Finished`,
the message should be removed from the queue even in case of
failure, such as a permanent error.

In addition to this bugfix, move the retry count increase to
the beginning of `send_msg_to_smtp` to ensure no message is
retried infinitely even in case of similar bugs.
2022-02-12 16:20:13 +00:00
Hocuri
bb9138708a Fix disappearing drafts (#3067) 2022-02-10 10:05:30 +01:00
Hocuri
34f5510f1f Don't directly download messages from the Spam folder (#3015)
fix #3007

My approach is:

We don't download any messages from the spam folder anymore, and only download them if they were moved out. This means that is-it-spam logic only resides in spam_target_folder(). This has some implications, see the comments.
2022-02-10 09:06:22 +01:00
link2xt
196075c031 imap: batch message deletion 2022-02-06 11:42:30 +00:00
link2xt
276daf631e imap: move messages in batches
Also change how NO response is treated. NO response means there is an
error moving/copying the messages. When there are no matching
messages, the response is "OK No matching messages, so nothing copied"
according to some RFC 9051 examples.
2022-02-05 22:15:46 +00:00
link2xt
fb19b58147 Reduce number of unsafe as conversions
Enable clippy::cast_lossless lint and get rid of
some conversions pointed out by  clippy::as_conversions.
2022-02-05 12:42:14 +00:00
bjoern
1caf3caf1b do set_visibility() in a transaction (#3053)
this avoids archived chats containing fresh messages:

before, it could happen that between the two SQL calls
a new fresh message arrives,
unarchives the chat that is immediately archived by the second SQL call -
resulting in an archive chat containing fresh messages.

as fresh messages counter are shown on app icon etc.
this is pretty weird for the user as they do not see what is "fresh".

the other way round,
there is no transaction in receive_imf(),
however, receive_imf() only unarchives chats,
so that is visible and no big issue for the user.

the issue is rare at all,
however, annoying if you get that as the badge counter may be stuck at "1"
nearly forever (until you open the archived chat in question).
2022-02-03 20:40:24 +01:00
bjoern
24e749a2c9 prepare 1.75 (#3049)
* update changelog for 1.75.0

* bump version to 1.75.0
2022-02-01 14:00:40 +01:00
link2xt
cccdc51ad4 Optimize delete_expired_imap_messages()
For me this reduced query time from 0.3 s to 0.05 s.
2022-01-31 20:34:01 +00:00
bjoern
99ddce6c3e prepare 1.74 (#3046)
* update changelog for 1.74.0

* bump version to 1.74.0
2022-01-31 19:53:50 +01:00
link2xt
f68088cfb5 imap: avoid reconnection loop when message without Message-ID is marked as seen
- do not attempt to mark reserved meessages as seen when
  messages with empty Message-ID are marked as seen on IMAP
- do not reconnect on Seen flag synchronization failures This avoid
  reconnection loops in case of permanent errors in `sync_seen_flags`
2022-01-31 00:00:00 +00:00
bjoern
a43fc47bb6 update provider database (#3043)
* update provider database

ran `./src/provider/update.py ../provider-db/_providers/ > src/provider/data.rs`

* update changelog
2022-01-31 16:07:20 +01:00
bjoern
8c1bfac53b prepare 1.73 (#3042)
* update changelog for 1.73.0

* bump version to 1.73.0
2022-01-31 15:12:44 +01:00
Floris Bruynooghe
97853c3660 Flub/watch mvbox only (#3028)
* Make set_config() look a bit nicer

* Add OnlyFetchMvbox option

* Add test for the config

* Add option to only watch mvbox

This is supposed to support having a server-side rule which moves
emails to the mvbox already.  The new option makes sure the mvbox is
wathched and also makes sure no messages are feched from folders other
than the mvbox and the spam folder if enabled.  It does not interact
with the other settings.

* Fixup ignore conditions

* Cleanup some bits

* Watch the mvbox when `WatchMvboxOnly` is set

* Rename back to only_fetch_mvbox (flub said it's OK for him)

* typo

* clippy, more typos

Co-authored-by: Hocuri <hocuri@gmx.de>
2022-01-31 13:39:48 +01:00
link2xt
f304a30193 imap: fetch Inbox before scanning other folders 2022-01-31 12:03:21 +01:00
link2xt
7eadca3959 imap: do not synchronize Seen flags on unwatched folders
Synchronizing seen flags doubles the time required to scan all
folders. Delta Chat only marks messages as Seen on Inbox or DeltaChat,
so there is no need to check for Seen flag on other folders.
2022-01-30 20:00:00 +00:00
link2xt
8aa6decbf9 imap: call delete_expired_imap_messages() less often
This operation takes roughly 0.3 s on a moderate size database.
Calling it once before scanning all folders and scanning
the watched folder instead of each time after downloading
a message from a folder speeds up IMAP loop.
2022-01-30 20:49:32 +00:00
link2xt
7cf4bcaca2 imap: call delete_expired_imap_messages() less often
This operation takes roughly 0.3 s on a moderate size database.
Calling it once before scanning all folders and scanning
the watched folder instead of each time after downloading
a message from a folder speeds up IMAP loop.
2022-01-30 20:47:32 +00:00
link2xt
e858a32aa1 smtp: cancel message sending by removing the message
This restores the logic removed in
afd8c0d879
2022-01-30 10:59:10 +00:00
B. Petersen
99f2680e2c fix splitting off text from webxdc messages
moreover, make the split check exhaustive
to avoid the same error on the next added Viewtype.
2022-01-30 11:49:09 +01:00
link2xt
0cb9e7922a Remove direct dependency on byteorder crate 2022-01-29 23:24:25 +00:00
link2xt
e73107006e smtp: replace thiserror with anyhow 2022-01-29 16:41:47 +00:00