Compare commits

..

29 Commits

Author SHA1 Message Date
bjoern
7c5eb0ae37 prepare 1.76 (#3082)
* update changelog for 1.76.0

* bump version to 1.76.0
2022-02-20 22:58:46 -05:00
link2xt
36bce6c468 Remove unused async-std feature 2022-02-19 11:30:48 +00:00
Hocuri
19a32cdfd3 Let MS Exchange MDNs mark the In-Reply-To message as read (#3075)
Fix https://github.com/deltachat/deltachat-core-rust/issues/2891
2022-02-17 09:56:00 +01:00
link2xt
d708f386a1 smtp: set message state to failed when retry limit is exceeded 2022-02-13 08:59:52 +00:00
link2xt
0f837f4bed Fix a comment typo 2022-02-12 16:29:42 +00:00
link2xt
242e8e2bb3 smtp: remove the message in case of permanent failure
When `smtp_send` returns `Status::Finished`,
the message should be removed from the queue even in case of
failure, such as a permanent error.

In addition to this bugfix, move the retry count increase to
the beginning of `send_msg_to_smtp` to ensure no message is
retried infinitely even in case of similar bugs.
2022-02-12 16:20:13 +00:00
link2xt
1d56b24b67 cargo update 2022-02-12 16:19:44 +00:00
Hocuri
bb9138708a Fix disappearing drafts (#3067) 2022-02-10 10:05:30 +01:00
Hocuri
34f5510f1f Don't directly download messages from the Spam folder (#3015)
fix #3007

My approach is:

We don't download any messages from the spam folder anymore, and only download them if they were moved out. This means that is-it-spam logic only resides in spam_target_folder(). This has some implications, see the comments.
2022-02-10 09:06:22 +01:00
link2xt
6c6d47c89c Fix CI
timeout_func_only makes pytest-rerunfailures work with pytest-timeout,
but it only works with default timeout_method.

See pytest-rerunfailures issue for details:
https://github.com/pytest-dev/pytest-rerunfailures/issues/99
2022-02-08 20:50:11 +00:00
link2xt
196075c031 imap: batch message deletion 2022-02-06 11:42:30 +00:00
link2xt
2e5e8f73c6 imap: simplify get_quota_roots() 2022-02-06 15:17:05 +00:00
link2xt
ada5d38272 imap: remove unwrap() 2022-02-06 14:07:04 +00:00
link2xt
c4b0f773db python: remove arbitrary timeouts from tests
pytest-timeout already handles all deadlocks and is configurable with
--timeout option. With this change it is possible to disable timeout
with --timeout 0 to run tests on extremely slow connections.
2022-02-06 12:52:48 +00:00
link2xt
276daf631e imap: move messages in batches
Also change how NO response is treated. NO response means there is an
error moving/copying the messages. When there are no matching
messages, the response is "OK No matching messages, so nothing copied"
according to some RFC 9051 examples.
2022-02-05 22:15:46 +00:00
link2xt
fb19b58147 Reduce number of unsafe as conversions
Enable clippy::cast_lossless lint and get rid of
some conversions pointed out by  clippy::as_conversions.
2022-02-05 12:42:14 +00:00
dependabot[bot]
13a5e3cf6f Merge pull request #3055 from deltachat/dependabot/cargo/async-std-resolver-0.20.4 2022-02-04 21:39:59 +00:00
bjoern
1caf3caf1b do set_visibility() in a transaction (#3053)
this avoids archived chats containing fresh messages:

before, it could happen that between the two SQL calls
a new fresh message arrives,
unarchives the chat that is immediately archived by the second SQL call -
resulting in an archive chat containing fresh messages.

as fresh messages counter are shown on app icon etc.
this is pretty weird for the user as they do not see what is "fresh".

the other way round,
there is no transaction in receive_imf(),
however, receive_imf() only unarchives chats,
so that is visible and no big issue for the user.

the issue is rare at all,
however, annoying if you get that as the badge counter may be stuck at "1"
nearly forever (until you open the archived chat in question).
2022-02-03 20:40:24 +01:00
dependabot[bot]
564370f79a cargo: bump async-std-resolver from 0.20.3 to 0.20.4
Bumps [async-std-resolver](https://github.com/bluejekyll/trust-dns) from 0.20.3 to 0.20.4.
- [Release notes](https://github.com/bluejekyll/trust-dns/releases)
- [Changelog](https://github.com/bluejekyll/trust-dns/blob/main/CHANGELOG.md)
- [Commits](https://github.com/bluejekyll/trust-dns/compare/v0.20.3...v0.20.4)

---
updated-dependencies:
- dependency-name: async-std-resolver
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-02-02 21:14:26 +00:00
bjoern
24e749a2c9 prepare 1.75 (#3049)
* update changelog for 1.75.0

* bump version to 1.75.0
2022-02-01 14:00:40 +01:00
link2xt
cccdc51ad4 Optimize delete_expired_imap_messages()
For me this reduced query time from 0.3 s to 0.05 s.
2022-01-31 20:34:01 +00:00
bjoern
99ddce6c3e prepare 1.74 (#3046)
* update changelog for 1.74.0

* bump version to 1.74.0
2022-01-31 19:53:50 +01:00
link2xt
f68088cfb5 imap: avoid reconnection loop when message without Message-ID is marked as seen
- do not attempt to mark reserved meessages as seen when
  messages with empty Message-ID are marked as seen on IMAP
- do not reconnect on Seen flag synchronization failures This avoid
  reconnection loops in case of permanent errors in `sync_seen_flags`
2022-01-31 00:00:00 +00:00
Hocuri
c8f56d748a Only fetch mvbox deltachat.h additions (#3045)
* Use the formatting of the rest of the file

* Add changes require restarting IO by calling dc_stop_io() and then dc_start_io(). comment
2022-01-31 17:46:18 +01:00
bjoern
a43fc47bb6 update provider database (#3043)
* update provider database

ran `./src/provider/update.py ../provider-db/_providers/ > src/provider/data.rs`

* update changelog
2022-01-31 16:07:20 +01:00
bjoern
8c1bfac53b prepare 1.73 (#3042)
* update changelog for 1.73.0

* bump version to 1.73.0
2022-01-31 15:12:44 +01:00
Floris Bruynooghe
97853c3660 Flub/watch mvbox only (#3028)
* Make set_config() look a bit nicer

* Add OnlyFetchMvbox option

* Add test for the config

* Add option to only watch mvbox

This is supposed to support having a server-side rule which moves
emails to the mvbox already.  The new option makes sure the mvbox is
wathched and also makes sure no messages are feched from folders other
than the mvbox and the spam folder if enabled.  It does not interact
with the other settings.

* Fixup ignore conditions

* Cleanup some bits

* Watch the mvbox when `WatchMvboxOnly` is set

* Rename back to only_fetch_mvbox (flub said it's OK for him)

* typo

* clippy, more typos

Co-authored-by: Hocuri <hocuri@gmx.de>
2022-01-31 13:39:48 +01:00
link2xt
f304a30193 imap: fetch Inbox before scanning other folders 2022-01-31 12:03:21 +01:00
link2xt
7eadca3959 imap: do not synchronize Seen flags on unwatched folders
Synchronizing seen flags doubles the time required to scan all
folders. Delta Chat only marks messages as Seen on Inbox or DeltaChat,
so there is no need to check for Seen flag on other folders.
2022-01-30 20:00:00 +00:00
32 changed files with 838 additions and 641 deletions

View File

@@ -1,21 +1,59 @@
# Changelog
## Unreleased
## 1.76.0
### Changes
- move messages in batches #3058
- delete messages in batches #3060
- python: remove arbitrary timeouts from tests #3059
- refactorings #3026
### Fixes
- avoid archived, fresh chats #3053
- treat "NO" IMAP response to MOVE and COPY commands as an error #3058
- Fix a bug where messages in the Spam folder created contact requests #3015
- Fix a bug where drafts disappeared after some days #3067
- Parse MS Exchange read receipts and mark the original message as read #3075
- do not retry message sending infinitely in case of permanent SMTP failure #3070
- set message state to failed when retry limit is exceeded #3072
## 1.75.0
### Changes
- optimize `delete_expired_imap_messages()` #3047
## 1.74.0
### Fixes
- avoid reconnection loop when message without Message-ID is marked as seen #3044
## 1.73.0
### API changes
- added `only_fetch_mvbox` config #3028
### Changes
- don't watch Sent folder by default #3025
- use webxdc app name in chatlist/quotes/replies etc. #3027
- refactorings #3023
- remove direct dependency on `byteorder` crate #3031
- make it possible to cancel message sending by removing the message #3034,
this was previosuly removed in 1.71.0 #2939
- always skip Seen flag synchronization when there are no updates #3039
- synchronize Seen flags only on watched folders to speed up
folder scanning #3041
- remove direct dependency on `byteorder` crate #3031
- refactorings #3023 #3013
- update provider database #3043
- improve documentation #3017 #3018 #3021
### Fixes
- fix splitting off text from webxdc messages #3032
- call slow `delete_expired_imap_messages()` less often #3037
- make synchronization of Seen status more robust in case unsolicited FETCH
result without UID is returned #3022
- fetch Inbox before scanning folders to ensure iOS does
not kill the app before it gets to fetch the Inbox in background #3040
## 1.72.0

184
Cargo.lock generated
View File

@@ -242,7 +242,7 @@ dependencies = [
"parking",
"polling",
"slab",
"socket2 0.4.3",
"socket2 0.4.4",
"waker-fn",
"winapi",
]
@@ -347,9 +347,9 @@ dependencies = [
[[package]]
name = "async-std-resolver"
version = "0.20.3"
version = "0.20.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ed4e2c3da14d8ad45acb1e3191db7a918e9505b6f155b218e70a7c9a1a48c638"
checksum = "dbf3e776afdf3a2477ef4854b85ba0dff3bd85792f685fb3c68948b4d304e4f0"
dependencies = [
"async-std",
"async-trait",
@@ -374,9 +374,9 @@ dependencies = [
[[package]]
name = "async-task"
version = "4.0.3"
version = "4.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e91831deabf0d6d7ec49552e489aed63b7456a7a3c46cff62adad428110b0af0"
checksum = "677d306121baf53310a3fd342d88dc0824f6bbeace68347593658525565abee8"
[[package]]
name = "async-trait"
@@ -408,15 +408,18 @@ dependencies = [
[[package]]
name = "autocfg"
version = "0.1.7"
version = "0.1.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1d49d90015b3c36167a20fe2810c5cd875ad504b39cff3d4eae7977e6b7c1cb2"
checksum = "0dde43e75fd43e8a1bf86103336bc699aa8d17ad1be60c76c0bdfd4828e19b78"
dependencies = [
"autocfg 1.1.0",
]
[[package]]
name = "autocfg"
version = "1.0.1"
version = "1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cdb031dd78e28731d87d56cc8ffef4a8f36ca26c38fe2de700543e627f8a464a"
checksum = "d468802bab17cbc0cc575e9b053f41e72aa36bfa6b7f55e3529ffa43161b97fa"
[[package]]
name = "backtrace"
@@ -493,9 +496,9 @@ dependencies = [
[[package]]
name = "block-buffer"
version = "0.10.0"
version = "0.10.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f1d36a02058e76b040de25a4464ba1c80935655595b661505c8b39b664828b95"
checksum = "0bf7fe51849ea569fd452f37822f606a5cabb684dc918707a0193fd4664ff324"
dependencies = [
"generic-array",
]
@@ -719,9 +722,9 @@ dependencies = [
[[package]]
name = "clipboard-win"
version = "4.4.0"
version = "4.4.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b4d3d118de1bf9678546f65e12f749b46abb5a56129d435af21fc7e42768f974"
checksum = "2f3e1238132dc01f081e1cbb9dace14e5ef4c3a51ee244bd982275fb514605db"
dependencies = [
"error-code",
"str-buf",
@@ -779,9 +782,9 @@ dependencies = [
[[package]]
name = "core-foundation"
version = "0.9.2"
version = "0.9.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6888e10551bb93e424d8df1d07f1a8b4fceb0001a3a4b048bfc47554946f47b3"
checksum = "194a7a9e6de53fa55116934067c844d9d749312f75c6f6d0980e8c252f8c2146"
dependencies = [
"core-foundation-sys",
"libc",
@@ -816,9 +819,9 @@ checksum = "fd121741cf3eb82c08dd3023eb55bf2665e5f60ec20f89760cf836ae4562e6a0"
[[package]]
name = "crc32fast"
version = "1.3.1"
version = "1.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a2209c310e29876f7f0b2721e7e26b84aff178aa3da5d091f9bfbf47669e60e3"
checksum = "b540bd8bc810d3885c6ea91e2018302f68baba2129ab3e88f32389ee9370880d"
dependencies = [
"cfg-if 1.0.0",
]
@@ -882,9 +885,9 @@ dependencies = [
[[package]]
name = "crossbeam-epoch"
version = "0.9.6"
version = "0.9.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "97242a70df9b89a65d0b6df3c4bf5b9ce03c5b7309019777fbde37e7537f8762"
checksum = "c00d6d2ea26e8b151d99093005cb442fb9a37aeaca582a03ec70946f49ab5ed9"
dependencies = [
"cfg-if 1.0.0",
"crossbeam-utils",
@@ -895,9 +898,9 @@ dependencies = [
[[package]]
name = "crossbeam-queue"
version = "0.3.3"
version = "0.3.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b979d76c9fcb84dffc80a73f7290da0f83e4c95773494674cb44b76d13a7a110"
checksum = "4dd435b205a4842da59efd07628f921c096bc1cc0a156835b4fa0bcb9a19bcce"
dependencies = [
"cfg-if 1.0.0",
"crossbeam-utils",
@@ -905,9 +908,9 @@ dependencies = [
[[package]]
name = "crossbeam-utils"
version = "0.8.6"
version = "0.8.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cfcae03edb34f947e64acdb1c33ec169824e20657e9ecb61cef6c8c74dcb8120"
checksum = "b5e5bed1f1c269533fa816a0a5492b3545209a205ca1a54842be180eb63a16a6"
dependencies = [
"cfg-if 1.0.0",
"lazy_static",
@@ -915,9 +918,9 @@ dependencies = [
[[package]]
name = "crypto-common"
version = "0.1.1"
version = "0.1.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "683d6b536309245c849479fba3da410962a43ed8e51c26b729208ec0ac2798d0"
checksum = "a4600d695eb3f6ce1cd44e6e291adceb2cc3ab12f20a33777ecd0bf6eba34e06"
dependencies = [
"generic-array",
]
@@ -1063,7 +1066,7 @@ dependencies = [
[[package]]
name = "deltachat"
version = "1.72.0"
version = "1.76.0"
dependencies = [
"ansi_term",
"anyhow",
@@ -1090,7 +1093,6 @@ dependencies = [
"hex",
"humansize",
"image",
"imap-proto",
"kamadak-exif",
"lettre_email",
"libc",
@@ -1144,7 +1146,7 @@ dependencies = [
[[package]]
name = "deltachat_ffi"
version = "1.72.0"
version = "1.76.0"
dependencies = [
"anyhow",
"async-std",
@@ -1204,13 +1206,12 @@ dependencies = [
[[package]]
name = "digest"
version = "0.10.1"
version = "0.10.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b697d66081d42af4fba142d56918a3cb21dc8eb63372c6b85d14f44fb9c5979b"
checksum = "8cb780dce4f9a8f5c087362b3a4595936b2019e7c8b30f2c3e9a7e94e6ae9837"
dependencies = [
"block-buffer 0.10.0",
"block-buffer 0.10.2",
"crypto-common",
"generic-array",
]
[[package]]
@@ -1591,9 +1592,9 @@ checksum = "fed34cd105917e91daa4da6b3728c47b068749d6a62c59811f06ed2ac71d9da7"
[[package]]
name = "futures"
version = "0.3.19"
version = "0.3.21"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "28560757fe2bb34e79f907794bb6b22ae8b0e5c669b638a1132f2592b19035b4"
checksum = "f73fe65f54d1e12b726f517d3e2135ca3125a437b6d998caf1962961f7172d9e"
dependencies = [
"futures-channel",
"futures-core",
@@ -1606,9 +1607,9 @@ dependencies = [
[[package]]
name = "futures-channel"
version = "0.3.19"
version = "0.3.21"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ba3dda0b6588335f360afc675d0564c17a77a2bda81ca178a4b6081bd86c7f0b"
checksum = "c3083ce4b914124575708913bca19bfe887522d6e2e6d0952943f5eac4a74010"
dependencies = [
"futures-core",
"futures-sink",
@@ -1616,15 +1617,15 @@ dependencies = [
[[package]]
name = "futures-core"
version = "0.3.19"
version = "0.3.21"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d0c8ff0461b82559810cdccfde3215c3f373807f5e5232b71479bff7bb2583d7"
checksum = "0c09fd04b7e4073ac7156a9539b57a484a8ea920f79c7c675d05d289ab6110d3"
[[package]]
name = "futures-executor"
version = "0.3.19"
version = "0.3.21"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "29d6d2ff5bb10fb95c85b8ce46538a2e5f5e7fdc755623a7d4529ab8a4ed9d2a"
checksum = "9420b90cfa29e327d0429f19be13e7ddb68fa1cccb09d65e5706b8c7a749b8a6"
dependencies = [
"futures-core",
"futures-task",
@@ -1633,9 +1634,9 @@ dependencies = [
[[package]]
name = "futures-io"
version = "0.3.19"
version = "0.3.21"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b1f9d34af5a1aac6fb380f735fe510746c38067c5bf16c7fd250280503c971b2"
checksum = "fc4045962a5a5e935ee2fdedaa4e08284547402885ab326734432bed5d12966b"
[[package]]
name = "futures-lite"
@@ -1654,9 +1655,9 @@ dependencies = [
[[package]]
name = "futures-macro"
version = "0.3.19"
version = "0.3.21"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6dbd947adfffb0efc70599b3ddcf7b5597bb5fa9e245eb99f62b3a5f7bb8bd3c"
checksum = "33c1e13800337f4d4d7a316bf45a567dbcb6ffe087f16424852d97e97a91f512"
dependencies = [
"proc-macro2",
"quote",
@@ -1665,21 +1666,21 @@ dependencies = [
[[package]]
name = "futures-sink"
version = "0.3.19"
version = "0.3.21"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e3055baccb68d74ff6480350f8d6eb8fcfa3aa11bdc1a1ae3afdd0514617d508"
checksum = "21163e139fa306126e6eedaf49ecdb4588f939600f0b1e770f4205ee4b7fa868"
[[package]]
name = "futures-task"
version = "0.3.19"
version = "0.3.21"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6ee7c6485c30167ce4dfb83ac568a849fe53274c831081476ee13e0dce1aad72"
checksum = "57c66a976bf5909d801bbef33416c41372779507e7a6b3a5e25e4749c58f776a"
[[package]]
name = "futures-util"
version = "0.3.19"
version = "0.3.21"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d9b5cf40b47a271f77a8b1bec03ca09044d99d2372c0de244e66430761127164"
checksum = "d8b7abd5d659d9b90c8cba917f6ec750a74e2dc23902ef9cd4cc8c8b22e6036a"
dependencies = [
"futures-channel",
"futures-core",
@@ -1753,15 +1754,14 @@ checksum = "78cc372d058dcf6d5ecd98510e7fbc9e5aec4d21de70f65fea8fecebcd881bd4"
[[package]]
name = "gloo-timers"
version = "0.2.2"
version = "0.2.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6f16c88aa13d2656ef20d1c042086b8767bbe2bdb62526894275a1b062161b2e"
checksum = "4d12a7f4e95cfe710f1d624fb1210b7d961a5fb05c4fd942f4feab06e61f590e"
dependencies = [
"futures-channel",
"futures-core",
"js-sys",
"wasm-bindgen",
"web-sys",
]
[[package]]
@@ -1895,9 +1895,9 @@ dependencies = [
[[package]]
name = "httparse"
version = "1.5.1"
version = "1.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "acd94fdbe1d4ff688b67b04eee2e17bd50995534a61539e45adfefb45e5e5503"
checksum = "9100414882e15fb7feccb4897e5f0ff0ff1ca7d1a86a23208ada4d7a18e6c6c4"
[[package]]
name = "human-panic"
@@ -2122,15 +2122,15 @@ dependencies = [
[[package]]
name = "libc"
version = "0.2.114"
version = "0.2.117"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b0005d08a8f7b65fb8073cb697aa0b12b631ed251ce73d862ce50eeb52ce3b50"
checksum = "e74d72e0f9b65b5b4ca49a346af3976df0f9c61d550727f349ecd559f251a26c"
[[package]]
name = "libm"
version = "0.2.1"
version = "0.2.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c7d73b3f436185384286bd8098d17ec07c9a7d2388a6599f824d8502b529702a"
checksum = "33a33a362ce288760ec6a508b94caaec573ae7d3bbbd91b87aa0bad4456839db"
[[package]]
name = "libsqlite3-sys"
@@ -2157,9 +2157,9 @@ checksum = "95f5690fef754d905294c56f7ac815836f2513af966aa47f2e07ac79be07827f"
[[package]]
name = "lock_api"
version = "0.4.5"
version = "0.4.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "712a4d093c9976e24e7dbca41db895dabcbac38eb5f4045393d17a95bdfb1109"
checksum = "88943dd7ef4a2e5a4bfa2753aaab3013e34ce2533d1996fb18ef591e315e2b3b"
dependencies = [
"scopeguard",
]
@@ -2229,7 +2229,7 @@ version = "0.6.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5aa361d4faea93603064a027415f07bd8e1d5c88c9fbf68bf56a285428fd79ce"
dependencies = [
"autocfg 1.0.1",
"autocfg 1.1.0",
]
[[package]]
@@ -2264,7 +2264,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a92518e98c078586bc6c934028adcca4c92a53d6a958196de835170a01d84e4b"
dependencies = [
"adler",
"autocfg 1.0.1",
"autocfg 1.1.0",
]
[[package]]
@@ -2353,7 +2353,7 @@ version = "0.2.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "090c7f9998ee0ff65aa5b723e4009f7b217707f1fb5ea551329cc4d6231fb304"
dependencies = [
"autocfg 1.0.1",
"autocfg 1.1.0",
"num-integer",
"num-traits",
]
@@ -2364,7 +2364,7 @@ version = "0.6.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5d51546d704f52ef14b3c962b5776e53d5b862e5790e40a350d366c209bd7f7a"
dependencies = [
"autocfg 0.1.7",
"autocfg 0.1.8",
"byteorder",
"lazy_static",
"libm",
@@ -2394,7 +2394,7 @@ version = "0.1.44"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d2cc698a63b549a70bc047073d2949cce27cd1c7b0a4a862d08a8031bc2801db"
dependencies = [
"autocfg 1.0.1",
"autocfg 1.1.0",
"num-traits",
]
@@ -2404,7 +2404,7 @@ version = "0.1.42"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b2021c8337a54d21aca0d59a92577a029af9431cb59b909b03252b9c164fad59"
dependencies = [
"autocfg 1.0.1",
"autocfg 1.1.0",
"num-integer",
"num-traits",
]
@@ -2415,7 +2415,7 @@ version = "0.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "12ac428b1cb17fce6f731001d307d351ec70a6d202fc2e60f7d4c5e42d8f4f07"
dependencies = [
"autocfg 1.0.1",
"autocfg 1.1.0",
"num-integer",
"num-traits",
]
@@ -2426,7 +2426,7 @@ version = "0.2.14"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9a64b1ec5cda2586e284722486d802acf1f7dbdc623e2bfc57e65ca1cd099290"
dependencies = [
"autocfg 1.0.1",
"autocfg 1.1.0",
]
[[package]]
@@ -2501,7 +2501,7 @@ version = "0.9.72"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7e46109c383602735fa0a2e48dd2b7c892b048e1bf69e5c3b1d804b7d9c203cb"
dependencies = [
"autocfg 1.0.1",
"autocfg 1.1.0",
"cc",
"libc",
"openssl-src",
@@ -2998,7 +2998,7 @@ version = "1.5.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c06aca804d41dbc8ba42dfd964f0d01334eceb64314b9ecf7c5fad5188a06d90"
dependencies = [
"autocfg 1.0.1",
"autocfg 1.1.0",
"crossbeam-deque",
"either",
"rayon-core",
@@ -3152,7 +3152,7 @@ version = "0.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bfa0f585226d2e68097d4f95d113b15b83a82e819ab25717ec0590d9584ef366"
dependencies = [
"semver 1.0.4",
"semver 1.0.5",
]
[[package]]
@@ -3257,9 +3257,9 @@ checksum = "d29ab0c6d3fc0ee92fe66e2d99f700eab17a8d57d1c1d3b748380fb20baa78cd"
[[package]]
name = "security-framework"
version = "2.5.0"
version = "2.6.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d09d3c15d814eda1d6a836f2f2b56a6abc1446c8a34351cb3180d3db92ffe4ce"
checksum = "2dc14f172faf8a0194a3aded622712b0de276821addc574fa54fc0a1167e10dc"
dependencies = [
"bitflags",
"core-foundation",
@@ -3270,9 +3270,9 @@ dependencies = [
[[package]]
name = "security-framework-sys"
version = "2.5.0"
version = "2.6.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e90dd10c41c6bfc633da6e0c659bd25d31e0791e5974ac42970267d59eba87f7"
checksum = "0160a13a177a45bfb43ce71c01580998474f556ad854dcbca936dd2841a5c556"
dependencies = [
"core-foundation-sys",
"libc",
@@ -3289,9 +3289,9 @@ dependencies = [
[[package]]
name = "semver"
version = "1.0.4"
version = "1.0.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "568a8e6258aa33c13358f81fd834adb854c6f7c9468520910a9b1e8fac068012"
checksum = "0486718e92ec9a68fbed73bb5ef687d71103b142595b406835649bebd33f72c7"
[[package]]
name = "semver-parser"
@@ -3331,9 +3331,9 @@ dependencies = [
[[package]]
name = "serde_json"
version = "1.0.78"
version = "1.0.79"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d23c1ba4cf0efd44be32017709280b32d1cea5c3f1275c3b6d9e8bc54f758085"
checksum = "8e8d9fa5c3b304765ce1fd9c4c8a3de2c8db365a5b91be52f186efc675681d95"
dependencies = [
"itoa 1.0.1",
"ryu",
@@ -3384,7 +3384,7 @@ checksum = "028f48d513f9678cda28f6e4064755b3fbb2af6acd672f2c209b62323f7aea0f"
dependencies = [
"cfg-if 1.0.0",
"cpufeatures",
"digest 0.10.1",
"digest 0.10.2",
]
[[package]]
@@ -3423,7 +3423,7 @@ checksum = "99c3bd8169c58782adad9290a9af5939994036b76187f7b4f0e6de91dbbfc0ec"
dependencies = [
"cfg-if 1.0.0",
"cpufeatures",
"digest 0.10.1",
"digest 0.10.2",
]
[[package]]
@@ -3514,9 +3514,9 @@ dependencies = [
[[package]]
name = "socket2"
version = "0.4.3"
version = "0.4.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0f82496b90c36d70af5fcd482edaa2e0bd16fade569de1330405fecbbdac736b"
checksum = "66d72b759436ae32898a2af0a14218dbf55efde3feeb170eb623637db85ee1e0"
dependencies = [
"libc",
"winapi",
@@ -3853,9 +3853,9 @@ checksum = "cda74da7e1a664f795bb1f8a87ec406fb89a02522cf6e50620d016add6dbbf5c"
[[package]]
name = "tokio"
version = "1.15.0"
version = "1.16.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fbbf1c778ec206785635ce8ad57fe52b3009ae9e0c9f574a728f3049d3e55838"
checksum = "0c27a64b625de6d309e8c57716ba93021dccf1b3b5c97edd6d3dd2d2135afc0a"
dependencies = [
"pin-project-lite",
]
@@ -3871,9 +3871,9 @@ dependencies = [
[[package]]
name = "trust-dns-proto"
version = "0.20.3"
version = "0.20.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ad0d7f5db438199a6e2609debe3f69f808d074e0a2888ee0bccb45fe234d03f4"
checksum = "ca94d4e9feb6a181c690c4040d7a24ef34018d8313ac5044a61d21222ae24e31"
dependencies = [
"async-trait",
"cfg-if 1.0.0",
@@ -3895,9 +3895,9 @@ dependencies = [
[[package]]
name = "trust-dns-resolver"
version = "0.20.3"
version = "0.20.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f6ad17b608a64bd0735e67bde16b0636f8aa8591f831a25d18443ed00a699770"
checksum = "ecae383baad9995efaa34ce8e57d12c3f305e545887472a492b838f4b5cfb77a"
dependencies = [
"cfg-if 1.0.0",
"futures-util",
@@ -3973,9 +3973,9 @@ dependencies = [
[[package]]
name = "unicode-segmentation"
version = "1.8.0"
version = "1.9.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8895849a949e7845e06bd6dc1aa51731a103c42707010a5b591c0038fb73385b"
checksum = "7e8820f5d777f6224dc4be3632222971ac30164d4a258d595640799554ebfd99"
[[package]]
name = "unicode-width"

View File

@@ -1,6 +1,6 @@
[package]
name = "deltachat"
version = "1.72.0"
version = "1.76.0"
authors = ["Delta Chat Developers (ML) <delta@codespeak.net>"]
edition = "2018"
license = "MPL-2.0"
@@ -24,7 +24,7 @@ async-imap = { git = "https://github.com/async-email/async-imap" }
async-native-tls = { version = "0.3" }
async-smtp = { git = "https://github.com/async-email/async-smtp", branch="master", features = ["socks5"] }
async-std-resolver = "0.20"
async-std = { version = "1", features = ["unstable"] }
async-std = { version = "1" }
async-tar = { version = "0.4", default-features=false }
async-trait = "0.1"
backtrace = "0.3"
@@ -38,7 +38,6 @@ escaper = "0.1"
futures = "0.3"
hex = "0.4.0"
image = { version = "0.23.5", default-features=false, features = ["gif", "jpeg", "ico", "png", "pnm", "webp", "bmp"] }
imap-proto = "0.14.3"
kamadak-exif = "0.5"
lettre_email = { git = "https://github.com/deltachat/lettre", branch = "master" }
libc = "0.2"

View File

@@ -1,6 +1,6 @@
[package]
name = "deltachat_ffi"
version = "1.72.0"
version = "1.76.0"
description = "Deltachat FFI"
authors = ["Delta Chat Developers (ML) <delta@codespeak.net>"]
edition = "2018"

View File

@@ -343,6 +343,12 @@ char* dc_get_blobdir (const dc_context_t* context);
* and watch the `DeltaChat` folder for updates (default),
* 0=do not move chat-messages
* changes require restarting IO by calling dc_stop_io() and then dc_start_io().
* - `only_fetch_mvbox` = 1=Do not fetch messages from folders other than the
* `DeltaChat` folder. Messages will still be fetched from the
* spam folder and `sendbox_watch` will also still be respected
* if enabled.
* 0=watch all folders normally (default)
* changes require restarting IO by calling dc_stop_io() and then dc_start_io().
* - `show_emails` = DC_SHOW_EMAILS_OFF (0)=
* show direct replies to chats only (default),
* DC_SHOW_EMAILS_ACCEPTED_CONTACTS (1)=

View File

@@ -224,9 +224,7 @@ class DirectImap:
""" (blocking) wait for next idle message from server. """
assert self._idling
self.account.log("imap-direct: calling idle_check")
res = self.conn.idle_check(timeout=30)
if len(res) == 0:
raise TimeoutError
res = self.conn.idle_check()
if terminate:
self.idle_done()
self.account.log("imap-direct: idle_check returned {!r}".format(res))

View File

@@ -241,7 +241,6 @@ def acfactory(pytestconfig, tmpdir, request, session_liveconfig, data):
def make_account(self, path, logid, quiet=False):
ac = Account(path, logging=self._logging)
ac._evtracker = ac.add_account_plugin(FFIEventTracker(ac))
ac._evtracker.set_timeout(30)
ac.addr = ac.get_self_contact().addr
ac.set_config("displayname", logid)
if not quiet:
@@ -483,7 +482,7 @@ class BotProcess:
def kill(self) -> None:
self.popen.kill()
def wait(self, timeout=30) -> None:
def wait(self, timeout=None) -> None:
self.popen.wait(timeout=timeout)
def fnmatch_lines(self, pattern_lines):
@@ -492,7 +491,7 @@ class BotProcess:
print("+++FNMATCH:", next_pattern)
ignored = []
while 1:
line = self.stdout_queue.get(timeout=15)
line = self.stdout_queue.get()
if line is None:
if ignored:
print("BOT stdout terminated after these lines")

View File

@@ -652,8 +652,6 @@ class TestOnlineAccount:
pre_generated_key=False,
config={"key_gen_type": str(const.DC_KEY_GEN_ED25519)}
)
# rsa key gen can be slow especially on CI, adjust timeout
ac1._evtracker.set_timeout(240)
acfactory.wait_configure_and_start_io()
chat = acfactory.get_accepted_chat(ac1, ac2)
@@ -892,11 +890,11 @@ class TestOnlineAccount:
chat = acfactory.get_accepted_chat(ac1, ac2)
chat.send_text("message1")
ac1._evtracker.get_matching("DC_EVENT_IMAP_MESSAGE_MOVED")
chat.send_text("message2")
ac1._evtracker.get_matching("DC_EVENT_IMAP_MESSAGE_MOVED")
chat.send_text("message3")
ac1._evtracker.get_matching("DC_EVENT_IMAP_MESSAGE_MOVED")
ac1._evtracker.get_matching("DC_EVENT_IMAP_MESSAGE_MOVED")
ac1._evtracker.get_matching("DC_EVENT_IMAP_MESSAGE_MOVED")
def test_forward_messages(self, acfactory, lp):
ac1, ac2 = acfactory.get_two_online_accounts()
@@ -1400,11 +1398,13 @@ class TestOnlineAccount:
assert not device_chat.can_send()
assert device_chat.get_draft() is None
def test_dont_show_emails_in_draft_folder(self, acfactory, lp):
def test_dont_show_emails(self, acfactory, lp):
"""Most mailboxes have a "Drafts" folder where constantly new emails appear but we don't actually want to show them.
So: If it's outgoing AND there is no Received header AND it's not in the sentbox, then ignore the email.
If the draft email is sent out later (i.e. moved to "Sent"), it must be shown."""
If the draft email is sent out later (i.e. moved to "Sent"), it must be shown.
Also, test that unknown emails in the Spam folder are not shown."""
ac1 = acfactory.get_online_configuring_account()
ac1.set_config("show_emails", "2")
ac1.create_contact("alice@example.org").create_chat()
@@ -1412,6 +1412,7 @@ class TestOnlineAccount:
acfactory.wait_configure(ac1)
ac1.direct_imap.create_folder("Drafts")
ac1.direct_imap.create_folder("Sent")
ac1.direct_imap.create_folder("Spam")
acfactory.wait_configure_and_start_io()
# Wait until each folder was selected once and we are IDLEing again:
@@ -1436,6 +1437,15 @@ class TestOnlineAccount:
message in Sent
""".format(ac1.get_config("configured_addr")))
ac1.direct_imap.append("Spam", """
From: unknown.address@junk.org
Subject: subj
To: {}
Message-ID: <spam.message@junk.org>
Content-Type: text/plain; charset=utf-8
Unknown message in Spam
""".format(ac1.get_config("configured_addr")))
ac1.set_config("scan_all_folders_debounce_secs", "0")
lp.sec("All prepared, now let DC find the message")
@@ -1449,6 +1459,10 @@ class TestOnlineAccount:
assert msg.text == "subj message in Sent"
assert len(msg.chat.get_messages()) == 1
assert not any("unknown.address" in c.get_name() for c in ac1.get_chats())
ac1.direct_imap.select_folder("Spam")
assert ac1.direct_imap.get_uid_by_message_id("spam.message@junk.org")
ac1.stop_io()
lp.sec("'Send out' the draft, i.e. move it to the Sent folder, and wait for DC to display it this time")
ac1.direct_imap.select_folder("Drafts")
@@ -1834,7 +1848,6 @@ class TestOnlineAccount:
lp.sec("trigger ac setup message and return setupcode")
assert ac1.get_info()["fingerprint"] != ac2.get_info()["fingerprint"]
setup_code = ac1.initiate_key_transfer()
ac2._evtracker.set_timeout(30)
ev = ac2._evtracker.get_matching("DC_EVENT_INCOMING_MSG|DC_EVENT_MSGS_CHANGED")
msg = ac2.get_message_by_id(ev.data2)
assert msg.is_setup_message()
@@ -1851,7 +1864,6 @@ class TestOnlineAccount:
def test_ac_setup_message_twice(self, acfactory, lp):
ac1 = acfactory.get_online_configuring_account()
ac2 = acfactory.clone_online_account(ac1)
ac2._evtracker.set_timeout(30)
acfactory.wait_configure_and_start_io()
lp.sec("trigger ac setup message but ignore")
@@ -2025,7 +2037,7 @@ class TestOnlineAccount:
lp.sec("ac1: send a message to group chat to promote the group")
chat.send_text("afterwards promoted")
ev = in_list.get(timeout=10)
ev = in_list.get()
assert ev.action == "chat-modified"
assert chat.is_promoted()
assert sorted(x.addr for x in chat.get_contacts()) == \
@@ -2035,29 +2047,29 @@ class TestOnlineAccount:
# note that if the above create_chat() would not
# happen we would not receive a proper member_added event
contact2 = chat.add_contact("devnull@testrun.org")
ev = in_list.get(timeout=10)
ev = in_list.get()
assert ev.action == "chat-modified"
ev = in_list.get(timeout=10)
ev = in_list.get()
assert ev.action == "chat-modified"
ev = in_list.get(timeout=10)
ev = in_list.get()
assert ev.action == "added"
assert ev.message.get_sender_contact().addr == ac1_addr
assert ev.contact.addr == "devnull@testrun.org"
lp.sec("ac1: remove address2")
chat.remove_contact(contact2)
ev = in_list.get(timeout=10)
ev = in_list.get()
assert ev.action == "chat-modified"
ev = in_list.get(timeout=10)
ev = in_list.get()
assert ev.action == "removed"
assert ev.contact.addr == contact2.addr
assert ev.message.get_sender_contact().addr == ac1_addr
lp.sec("ac1: remove ac2 contact from chat")
chat.remove_contact(ac2)
ev = in_list.get(timeout=10)
ev = in_list.get()
assert ev.action == "chat-modified"
ev = in_list.get(timeout=10)
ev = in_list.get()
assert ev.action == "removed"
assert ev.message.get_sender_contact().addr == ac1_addr
@@ -2504,8 +2516,7 @@ class TestOnlineAccount:
lp.sec("ac2: deleting all messages except third")
assert len(to_delete) == len(texts) - 1
ac2.delete_messages(to_delete)
for msg in to_delete:
ac2._evtracker.get_matching("DC_EVENT_IMAP_MESSAGE_DELETED")
ac2._evtracker.get_matching("DC_EVENT_IMAP_MESSAGE_DELETED")
ac2._evtracker.get_info_contains("close/expunge succeeded")
@@ -2682,7 +2693,13 @@ class TestOnlineAccount:
ac1.direct_imap.select_config_folder("inbox")
ac1.direct_imap.idle_start()
acfactory.get_accepted_chat(ac2, ac1).send_text("hello")
ac1.direct_imap.idle_check(terminate=True)
while True:
if len(ac1.direct_imap.idle_check(terminate=True)) > 1:
# If length is 1, it's [(b'OK', b'Still here')]
# Could happen on very slow network.
#
# More is usually [(1, b'EXISTS'), (1, b'RECENT')]
break
ac1.direct_imap.conn.move(["*"], folder) # "*" means "biggest UID in mailbox"
lp.sec("Everything prepared, now see if DeltaChat finds the message (" + variant + ")")

View File

@@ -79,7 +79,7 @@ addopts = -v -ra --strict-markers
norecursedirs = .tox
xfail_strict=true
timeout = 90
timeout_method = thread
timeout_func_only = True
markers =
ignored: ignore this test in default test runs, use --ignored to run.

View File

@@ -474,22 +474,21 @@ impl ChatId {
self
);
if visibility == ChatVisibility::Archived {
context
.sql
.execute(
"UPDATE msgs SET state=? WHERE chat_id=? AND state=?;",
paramsv![MessageState::InNoticed, self, MessageState::InFresh],
)
.await?;
}
context
.sql
.execute(
"UPDATE chats SET archived=? WHERE id=?;",
paramsv![visibility, self],
)
.transaction(move |transaction| {
if visibility == ChatVisibility::Archived {
transaction.execute(
"UPDATE msgs SET state=? WHERE chat_id=? AND state=?;",
paramsv![MessageState::InNoticed, self, MessageState::InFresh],
)?;
}
transaction.execute(
"UPDATE chats SET archived=? WHERE id=?;",
paramsv![visibility, self],
)?;
Ok(())
})
.await?;
context.emit_event(EventType::MsgsChanged {
@@ -961,7 +960,7 @@ impl std::fmt::Display for ChatId {
/// well as query for a [ChatId].
impl rusqlite::types::ToSql for ChatId {
fn to_sql(&self) -> rusqlite::Result<rusqlite::types::ToSqlOutput> {
let val = rusqlite::types::Value::Integer(self.0 as i64);
let val = rusqlite::types::Value::Integer(i64::from(self.0));
let out = rusqlite::types::ToSqlOutput::Owned(val);
Ok(out)
}
@@ -971,7 +970,7 @@ impl rusqlite::types::ToSql for ChatId {
impl rusqlite::types::FromSql for ChatId {
fn column_result(value: rusqlite::types::ValueRef) -> rusqlite::types::FromSqlResult<Self> {
i64::column_result(value).and_then(|val| {
if 0 <= val && val <= std::u32::MAX as i64 {
if 0 <= val && val <= i64::from(std::u32::MAX) {
Ok(ChatId::new(val as u32))
} else {
Err(rusqlite::types::FromSqlError::OutOfRange(val))

View File

@@ -74,6 +74,13 @@ pub enum Config {
#[strum(props(default = "0"))]
SentboxMove, // If `MvboxMove` is true, this config is ignored. Currently only used in tests.
/// Watch for new messages in the "Mvbox" (aka DeltaChat folder) only.
///
/// This will not entirely disable other folders, e.g. the spam folder will also still
/// be watched for new messages.
#[strum(props(default = "0"))]
OnlyFetchMvbox,
#[strum(props(default = "0"))] // also change ShowEmails.default() on changes
ShowEmails,
@@ -225,6 +232,11 @@ impl Context {
Ok(self.get_config_int(key).await? != 0)
}
pub(crate) async fn should_watch_mvbox(&self) -> Result<bool> {
Ok(self.get_config_bool(Config::MvboxMove).await?
|| self.get_config_bool(Config::OnlyFetchMvbox).await?)
}
/// Gets configured "delete_server_after" value.
///
/// `None` means never delete the message, `Some(0)` means delete
@@ -233,7 +245,7 @@ impl Context {
match self.get_config_int(Config::DeleteServerAfter).await? {
0 => Ok(None),
1 => Ok(Some(0)),
x => Ok(Some(x as i64)),
x => Ok(Some(i64::from(x))),
}
}
@@ -255,7 +267,7 @@ impl Context {
pub async fn get_config_delete_device_after(&self) -> Result<Option<i64>> {
match self.get_config_int(Config::DeleteDeviceAfter).await? {
0 => Ok(None),
x => Ok(Some(x as i64)),
x => Ok(Some(i64::from(x))),
}
}
@@ -281,31 +293,25 @@ impl Context {
}
}
self.emit_event(EventType::SelfavatarChanged);
Ok(())
}
Config::DeleteDeviceAfter => {
let ret = self
.sql
.set_raw_config(key, value)
.await
.map_err(Into::into);
let ret = self.sql.set_raw_config(key, value).await;
// Force chatlist reload to delete old messages immediately.
self.emit_event(EventType::MsgsChanged {
msg_id: MsgId::new(0),
chat_id: ChatId::new(0),
});
ret
ret?
}
Config::Displayname => {
let value = value.map(improve_single_line_input);
self.sql.set_raw_config(key, value.as_deref()).await?;
Ok(())
}
_ => {
self.sql.set_raw_config(key, value).await?;
Ok(())
}
}
Ok(())
}
pub async fn set_config_bool(&self, key: Config, value: bool) -> Result<()> {

View File

@@ -443,7 +443,7 @@ async fn configure(ctx: &Context, param: &mut LoginParam) -> Result<()> {
progress!(ctx, 900);
let create_mvbox = ctx.get_config_bool(Config::MvboxMove).await?;
let create_mvbox = ctx.should_watch_mvbox().await?;
imap.configure_folders(ctx, create_mvbox).await?;

View File

@@ -358,6 +358,7 @@ impl Context {
let sentbox_watch = self.get_config_int(Config::SentboxWatch).await?;
let mvbox_move = self.get_config_int(Config::MvboxMove).await?;
let sentbox_move = self.get_config_int(Config::SentboxMove).await?;
let only_fetch_mvbox = self.get_config_int(Config::OnlyFetchMvbox).await?;
let folders_configured = self
.sql
.get_raw_config_int("folders_configured")
@@ -422,6 +423,7 @@ impl Context {
res.insert("sentbox_watch", sentbox_watch.to_string());
res.insert("mvbox_move", mvbox_move.to_string());
res.insert("sentbox_move", sentbox_move.to_string());
res.insert("only_fetch_mvbox", only_fetch_mvbox.to_string());
res.insert("folders_configured", folders_configured.to_string());
res.insert("configured_sentbox_folder", configured_sentbox_folder);
res.insert("configured_mvbox_folder", configured_mvbox_folder);

View File

@@ -193,7 +193,6 @@ pub(crate) async fn dc_receive_imf_inner(
&mut mime_parser,
imf_raw,
incoming,
incoming_origin,
server_folder,
&to_ids,
&rfc724_mid,
@@ -415,7 +414,6 @@ async fn add_parts(
mime_parser: &mut MimeMessage,
imf_raw: &[u8],
incoming: bool,
incoming_origin: Origin,
server_folder: &str,
to_ids: &[u32],
rfc724_mid: &str,
@@ -432,7 +430,6 @@ async fn add_parts(
) -> Result<ReceivedMsg> {
let mut chat_id = None;
let mut chat_id_blocked = Blocked::Not;
let mut incoming_origin = incoming_origin;
let parent = get_parent_message(context, mime_parser).await?;
@@ -676,7 +673,6 @@ async fn add_parts(
if chat_id_blocked != Blocked::Not {
if chat_id_blocked != create_blocked {
chat_id.set_blocked(context, create_blocked).await?;
chat_id_blocked = create_blocked;
}
if create_blocked == Blocked::Request && parent.is_some() {
// we do not want any chat to be created implicitly. Because of the origin-scale-up,
@@ -687,9 +683,6 @@ async fn add_parts(
context,
"Message is a reply to a known message, mark sender as known.",
);
if !incoming_origin.is_known() {
incoming_origin = Origin::IncomingReplyTo;
}
}
}
}
@@ -701,15 +694,6 @@ async fn add_parts(
} else {
MessageState::InFresh
};
let is_spam = (chat_id_blocked == Blocked::Request)
&& !incoming_origin.is_known()
&& (is_dc_message == MessengerMessage::No)
&& context.is_spam_folder(server_folder).await?;
if is_spam {
chat_id = Some(DC_CHAT_ID_TRASH);
info!(context, "Message is probably spam (TRASH)");
}
} else {
// Outgoing
@@ -1156,8 +1140,8 @@ INSERT INTO msgs
stmt.execute(paramsv![
rfc724_mid,
chat_id,
if trash { 0 } else { from_id as i32 },
if trash { 0 } else { to_id as i32 },
if trash { 0 } else { i64::from(from_id) },
if trash { 0 } else { i64::from(to_id) },
sort_timestamp,
sent_timestamp,
rcvd_timestamp,
@@ -2363,12 +2347,10 @@ fn dc_create_incoming_rfc724_mid(mime: &MimeMessage) -> String {
#[cfg(test)]
mod tests {
use chat::get_chat_contacts;
use mailparse::MailHeaderMap;
use super::*;
use crate::chat::get_chat_contacts;
use crate::chat::{get_chat_msgs, ChatItem, ChatVisibility};
use crate::chatlist::Chatlist;
use crate::constants::{DC_CONTACT_ID_INFO, DC_GCL_NO_SPECIALS};
@@ -2977,7 +2959,7 @@ mod tests {
&headers,
"some-other-message-id",
std::iter::empty(),
ShowEmails::Off
ShowEmails::Off,
)
.await
.unwrap());
@@ -4298,76 +4280,6 @@ YEAAAAAA!.
assert_eq!(msg.text.unwrap(), "Reply");
}
#[async_std::test]
async fn test_dont_show_spam() {
async fn is_shown(t: &TestContext, raw: &[u8], server_folder: &str) -> bool {
let mail = mailparse::parse_mail(raw).unwrap();
dc_receive_imf(t, raw, server_folder, false).await.unwrap();
t.get_last_msg().await.rfc724_mid
== mail.get_headers().get_first_value("Message-Id").unwrap()
}
let t = TestContext::new_alice().await;
t.set_config(Config::ConfiguredSpamFolder, Some("Spam"))
.await
.unwrap();
t.set_config(Config::ShowEmails, Some("2")).await.unwrap();
assert!(
is_shown(
&t,
b"Message-Id: abcd1@exmaple.com\n\
From: bob@example.org\n\
Chat-Version: 1.0\n",
"Inbox",
)
.await,
);
assert!(
is_shown(
&t,
b"Message-Id: abcd2@exmaple.com\n\
From: bob@example.org\n",
"Inbox",
)
.await,
);
assert!(
is_shown(
&t,
b"Message-Id: abcd3@exmaple.com\n\
From: bob@example.org\n\
Chat-Version: 1.0\n",
"Spam",
)
.await,
);
assert!(
// Note the `!`:
!is_shown(
&t,
b"Message-Id: abcd4@exmaple.com\n\
From: bob@example.org\n",
"Spam",
)
.await,
);
Contact::create(&t, "", "bob@example.org").await.unwrap();
assert!(
is_shown(
&t,
b"Message-Id: abcd5@exmaple.com\n\
From: bob@example.org\n",
"Spam",
)
.await,
);
}
#[async_std::test]
async fn test_dont_show_all_outgoing_msgs_in_self_chat() {
// Regression test for <https://github.com/deltachat/deltachat-android/issues/1940>:

View File

@@ -71,7 +71,7 @@ pub(crate) fn dc_gm2local_offset() -> i64 {
/* returns the offset that must be _added_ to an UTC/GMT-time to create the localtime.
the function may return negative values. */
let lt = Local::now();
lt.offset().local_minus_utc() as i64
i64::from(lt.offset().local_minus_utc())
}
// timesmearing

View File

@@ -451,12 +451,11 @@ pub(crate) async fn delete_expired_imap_messages(context: &Context) -> Result<()
.execute(
"UPDATE imap
SET target=''
WHERE EXISTS (
SELECT * FROM msgs
WHERE rfc724_mid=imap.rfc724_mid
AND ((download_state = 0 AND timestamp < ?) OR
(download_state != 0 AND timestamp < ?) OR
(ephemeral_timestamp != 0 AND ephemeral_timestamp <= ?))
WHERE rfc724_mid IN (
SELECT rfc724_mid FROM msgs
WHERE ((download_state = 0 AND timestamp < ?) OR
(download_state != 0 AND timestamp < ?) OR
(ephemeral_timestamp != 0 AND ephemeral_timestamp <= ?))
)",
paramsv![threshold_timestamp, threshold_timestamp_extended, now],
)

View File

@@ -9,7 +9,7 @@ use std::{
collections::{BTreeMap, BTreeSet},
};
use anyhow::{anyhow, bail, format_err, Context as _, Result};
use anyhow::{bail, format_err, Context as _, Result};
use async_imap::types::{
Fetch, Flag, Mailbox, Name, NameAttribute, Quota, QuotaRoot, UnsolicitedResponse,
};
@@ -458,15 +458,9 @@ impl Imap {
.await
.context("fetch_new_messages")?;
self.move_messages(context, watch_folder)
self.move_delete_messages(context, watch_folder)
.await
.context("move_messages")?;
self.delete_messages(context, watch_folder)
.await
.context("delete_messages")?;
self.sync_seen_flags(context, watch_folder)
.await
.context("sync_seen_flags")?;
.context("move_delete_messages")?;
Ok(())
}
@@ -667,6 +661,11 @@ impl Imap {
folder: &str,
fetch_existing_msgs: bool,
) -> Result<bool> {
if should_ignore_folder(context, folder).await? {
info!(context, "Not fetching from {}", folder);
return Ok(false);
}
let new_emails = self.select_with_uidvalidity(context, folder).await?;
if !new_emails && !fetch_existing_msgs {
@@ -749,6 +748,11 @@ impl Imap {
// message, move it to the movebox and then download the second message before
// downloading the first one, if downloading from inbox before moving is allowed.
if folder == target
// Never download messages directly from the spam folder.
// If the sender is known, the message will be moved to the Inbox or Mvbox
// and then we download the message from there.
// Also see `spam_target_folder()`.
&& !context.is_spam_folder(folder).await?
&& prefetch_should_download(
context,
&headers,
@@ -825,18 +829,128 @@ impl Imap {
Ok(read_cnt > 0)
}
/// Moves messages.
/// Deletes batch of messages identified by their UID from the currently
/// selected folder.
async fn delete_message_batch(
&mut self,
context: &Context,
uid_set: &str,
row_ids: Vec<i64>,
) -> Result<()> {
// mark the message for deletion
self.add_flag_finalized_with_set(uid_set, "\\Deleted")
.await?;
context
.sql
.execute(
format!(
"DELETE FROM imap WHERE id IN ({})",
row_ids.iter().map(|_| "?").collect::<Vec<&str>>().join(",")
),
rusqlite::params_from_iter(row_ids),
)
.await
.context("cannot remove deleted messages from imap table")?;
context.emit_event(EventType::ImapMessageDeleted(format!(
"IMAP messages {} marked as deleted",
uid_set
)));
self.config.selected_folder_needs_expunge = true;
Ok(())
}
/// Moves batch of messages identified by their UID from the currently
/// selected folder to the target folder.
async fn move_message_batch(
&mut self,
context: &Context,
set: &str,
row_ids: Vec<i64>,
target: &str,
) -> Result<()> {
if self.config.can_move {
let session = self
.session
.as_mut()
.context("no session while attempting to MOVE messages")?;
match session.uid_mv(set, &target).await {
Ok(()) => {
// Messages are moved or don't exist, IMAP returns OK response in both cases.
context
.sql
.execute(
format!(
"DELETE FROM imap WHERE id IN ({})",
row_ids.iter().map(|_| "?").collect::<Vec<&str>>().join(",")
),
rusqlite::params_from_iter(row_ids),
)
.await
.context("cannot delete moved messages from imap table")?;
context.emit_event(EventType::ImapMessageMoved(format!(
"IMAP messages {} moved to {}",
set, target
)));
return Ok(());
}
Err(err) => {
warn!(
context,
"Cannot move message, fallback to COPY/DELETE {} to {}: {}",
set,
target,
err
);
}
}
} else {
info!(
context,
"Server does not support MOVE, fallback to COPY/DELETE {} to {}", set, target
);
}
// Server does not support MOVE or MOVE failed.
// Copy the message to the destination folder and mark the record for deletion.
let session = self
.session
.as_mut()
.context("no session while attempting to COPY messages")?;
match session.uid_copy(&set, &target).await {
Ok(()) => {
context
.sql
.execute(
format!(
"UPDATE imap SET target='' WHERE id IN ({})",
row_ids.iter().map(|_| "?").collect::<Vec<&str>>().join(",")
),
rusqlite::params_from_iter(row_ids),
)
.await
.context("cannot plan deletion of copied messages")?;
context.emit_event(EventType::ImapMessageMoved(format!(
"IMAP messages {} copied to {}",
set, target
)));
Ok(())
}
Err(err) => Err(err.into()),
}
}
/// Moves and deletes messages as planned in the `imap` table.
///
/// This is the only place where messages are moved on the IMAP server.
async fn move_messages(&mut self, context: &Context, folder: &str) -> Result<()> {
let rows = context
/// This is the only place where messages are moved or deleted on the IMAP server.
async fn move_delete_messages(&mut self, context: &Context, folder: &str) -> Result<()> {
let mut rows = context
.sql
.query_map(
"SELECT id, uid, target FROM imap
WHERE folder = ?
AND target != folder
AND target != '' -- Not planned for deletion.
ORDER BY id",
ORDER BY target, uid",
paramsv![folder],
|row| {
let rowid: i64 = row.get(0)?;
@@ -846,147 +960,62 @@ impl Imap {
},
|rows| rows.collect::<Result<Vec<_>, _>>().map_err(Into::into),
)
.await?;
.await?
.into_iter()
.peekable();
self.prepare(context).await?;
self.select_folder(context, Some(folder)).await?;
for (rowid, uid, target) in rows {
// TODO: batch moves of messages with the same destination.
let set = uid.to_string();
while let Some((_, _, target)) = rows.peek().cloned() {
// Construct next request for the target folder.
let mut uid_set = String::new();
let mut rowid_set = Vec::new();
if self.config.can_move {
if let Some(session) = &mut self.session {
match session.uid_mv(&set, &target).await {
Ok(_) => {
context.emit_event(EventType::ImapMessageMoved(format!(
"IMAP message {}/{} moved to {}",
folder, uid, target
)));
context
.sql
.execute("DELETE FROM imap WHERE id=?", paramsv![rowid])
.await?;
continue;
}
Err(async_imap::error::Error::No(text)) => {
// "NO" response, probably the message is moved already.
info!(
context,
"IMAP message {}/{} cannot be moved: {}", folder, uid, text
);
context
.sql
.execute("DELETE FROM imap WHERE id=?", paramsv![rowid])
.await?;
continue;
}
Err(err) => {
warn!(
context,
"Cannot move message, fallback to COPY/DELETE {}/{} to {}: {}",
folder,
uid,
target,
err
);
}
while uid_set.len() < 1000 {
// Construct a new range.
if let Some((start_rowid, start_uid, _)) =
rows.next_if(|(_, _, start_target)| start_target == &target)
{
rowid_set.push(start_rowid);
let mut end_uid = start_uid;
while let Some((next_rowid, next_uid, _)) =
rows.next_if(|(_, next_uid, next_target)| {
next_target == &target && *next_uid == end_uid + 1
})
{
end_uid = next_uid;
rowid_set.push(next_rowid);
}
let uid_range = UidRange {
start: start_uid,
end: end_uid,
};
if !uid_set.is_empty() {
uid_set.push(',');
}
uid_set.push_str(&uid_range.to_string());
} else {
bail!("No session while attempting to move the message");
}
} else {
info!(
context,
"Server does not support MOVE, fallback to COPY/DELETE {}/{} to {}",
folder,
uid,
target
);
}
// Server does not support MOVE or MOVE failed.
// Copy the message to the destination folder and mark the record for deletion.
if let Some(session) = &mut self.session {
match session.uid_copy(&set, &target).await {
Ok(_) => {
context.emit_event(EventType::ImapMessageMoved(format!(
"IMAP message {}/{} copied to {}",
folder, uid, target
)));
// Plan deletion of the original message.
context
.sql
.execute("UPDATE imap SET target='' WHERE id=?", paramsv![rowid])
.await?;
}
Err(async_imap::error::Error::No(text)) => {
// "NO" response, probably the message is moved already.
info!(
context,
"IMAP message {}/{} cannot be copied: {}", folder, uid, text
);
context
.sql
.execute("DELETE FROM imap WHERE id=?", paramsv![rowid])
.await?;
continue;
}
Err(err) => {
warn!(
context,
"Could not copy message {}/{}: {}", folder, uid, err
);
// Break the loop to avoid moving messages out of order.
// We can't proceed until this message is moved or copied.
break;
}
}
} else {
bail!("No session while attempting to copy the message");
}
}
Ok(())
}
/// Deletes messages that are marked as planned for deletion in `imap` table.
///
/// This is the only place where messages are deleted from the IMAP server.
async fn delete_messages(&mut self, context: &Context, folder: &str) -> Result<()> {
let rows = context
.sql
.query_map(
"SELECT id, uid FROM imap
WHERE folder=? AND target=''
ORDER BY uid ASC
LIMIT 50", // Do not try to delete too many messages at once.
paramsv![folder],
|row| {
let rowid: i64 = row.get(0)?;
let uid: u32 = row.get(1)?;
Ok((rowid, uid))
},
|rows| rows.collect::<Result<Vec<_>, _>>().map_err(Into::into),
)
.await?;
if rows.is_empty() {
return Ok(());
}
for (rowid, uid) in rows {
match self.delete_msg(context, folder, uid).await {
ImapActionResult::Failed | ImapActionResult::RetryLater => {
warn!(context, "Deletion of message {}/{} failed", folder, uid);
break;
}
ImapActionResult::Success => {
context
.sql
.execute("DELETE FROM imap WHERE id=?", paramsv![rowid])
.await?;
}
}
// Empty target folder name means messages should be deleted.
if target.is_empty() {
self.delete_message_batch(context, &uid_set, rowid_set)
.await
.with_context(|| format!("cannot delete batch of messages {:?}", &uid_set))?;
} else {
self.move_message_batch(context, &uid_set, rowid_set, &target)
.await
.with_context(|| {
format!(
"cannot move batch of messages {:?} to folder {:?}",
&uid_set, target
)
})?;
}
}
@@ -1023,33 +1052,23 @@ impl Imap {
.as_ref()
.with_context(|| format!("No mailbox selected, folder: {}", folder))?;
let remote_highest_modseq = if let Some(remote_highest_modseq) = mailbox.highest_modseq {
remote_highest_modseq
} else {
// Check if the mailbox supports MODSEQ.
// We are not interested in actual value of HIGHESTMODSEQ.
if mailbox.highest_modseq.is_none() {
info!(
context,
"Mailbox {} does not support mod-sequences, skipping flag synchronization.", folder
);
return Ok(());
};
let mut highest_modseq = get_modseq(context, folder)
.await
.with_context(|| format!("failed to get MODSEQ for folder {}", folder))?;
if highest_modseq >= remote_highest_modseq {
info!(
context,
"MODSEQ {} is already new, HIGHESTMODSEQ={}, skipping seen flag update",
highest_modseq,
remote_highest_modseq
);
return Ok(());
}
let mut updated_chat_ids = BTreeSet::new();
let uid_validity = get_uidvalidity(context, folder)
.await
.with_context(|| format!("failed to get UID validity for folder {}", folder))?;
let mut highest_modseq = get_modseq(context, folder)
.await
.with_context(|| format!("failed to get MODSEQ for folder {}", folder))?;
let mut list = session
.uid_fetch("1:*", format!("(FLAGS) (CHANGEDSINCE {})", highest_modseq))
.await
@@ -1084,10 +1103,6 @@ impl Imap {
}
}
if remote_highest_modseq > highest_modseq {
// We haven't seen the message with the highest MODSEQ, maybe it was deleted already.
highest_modseq = remote_highest_modseq;
}
set_modseq(context, folder, highest_modseq)
.await
.with_context(|| format!("failed to set MODSEQ for folder {}", folder))?;
@@ -1295,7 +1310,8 @@ impl Imap {
let folder = folder.clone();
// safe, as we checked above that there is a body.
let body = body.unwrap();
let body = body
.context("we checked that message has body right above, but it has vanished")?;
let is_seen = msg.flags().any(|flag| flag == Flag::Seen);
match dc_receive_imf_inner(
@@ -1350,7 +1366,7 @@ impl Imap {
bail!("Can't set flag, should reconnect");
}
let session = self.session.as_mut().context("No session").unwrap();
let session = self.session.as_mut().context("No session")?;
let query = format!("+FLAGS ({})", flag);
let mut responses = session
.uid_store(uid_set, &query)
@@ -1428,39 +1444,6 @@ impl Imap {
}
}
pub async fn delete_msg(
&mut self,
context: &Context,
folder: &str,
uid: u32,
) -> ImapActionResult {
if let Some(imapresult) = self
.prepare_imap_operation_on_msg(context, folder, uid)
.await
{
return imapresult;
}
// we are connected, and the folder is selected
let display_imap_id = format!("{}/{}", folder, uid);
// mark the message for deletion
if let Err(err) = self.add_flag_finalized(uid, "\\Deleted").await {
warn!(
context,
"Cannot mark message {} as \"Deleted\": {}.", display_imap_id, err
);
ImapActionResult::RetryLater
} else {
context.emit_event(EventType::ImapMessageDeleted(format!(
"IMAP Message {} marked as deleted",
display_imap_id
)));
self.config.selected_folder_needs_expunge = true;
ImapActionResult::Success
}
}
pub async fn ensure_configured_folders(
&mut self,
context: &Context,
@@ -1584,23 +1567,6 @@ impl Imap {
Ok(())
}
/// Update HIGHESTMODSEQ on selected mailbox.
///
/// Should be called when MODSEQ is seen on the response, such as IDLE response.
pub(crate) fn update_modseq(&mut self, modseq: u64) {
self.config.selected_mailbox =
self.config
.selected_mailbox
.as_ref()
.map(|mailbox| Mailbox {
highest_modseq: Some(std::cmp::max(
mailbox.highest_modseq.unwrap_or_default(),
modseq,
)),
..mailbox.clone()
});
}
/// Return whether the server sent an unsolicited EXISTS response.
/// Drains all responses from `session.unsolicited_responses` in the process.
/// If this returns `true`, this means that new emails arrived and you should
@@ -1627,29 +1593,38 @@ impl Imap {
self.config.can_check_quota
}
pub async fn get_quota_roots(
pub(crate) async fn get_quota_roots(
&mut self,
mailbox_name: &str,
) -> Result<(Vec<QuotaRoot>, Vec<Quota>)> {
if let Some(session) = self.session.as_mut() {
let quota_roots = session.get_quota_root(mailbox_name).await?;
Ok(quota_roots)
} else {
Err(anyhow!("Not connected to IMAP, no session"))
}
let session = self.session.as_mut().context("no session")?;
let quota_roots = session.get_quota_root(mailbox_name).await?;
Ok(quota_roots)
}
}
/// Returns target folder for a message found in the Spam folder.
async fn spam_target_folder(
async fn should_move_out_of_spam(
context: &Context,
folder: &str,
headers: &[mailparse::MailHeader<'_>],
) -> Result<Option<Config>> {
if let Some(chat) = prefetch_get_chat(context, headers).await? {
if chat.blocked != Blocked::Not {
) -> Result<bool> {
if headers.get_header_value(HeaderDef::ChatVersion).is_some() {
// If this is a chat message (i.e. has a ChatVersion header), then this might be
// a securejoin message. We can't find out at this point as we didn't prefetch
// the SecureJoin header. So, we always move chat messages out of Spam.
// Two possibilities to change this would be:
// 1. Remove the `&& !context.is_spam_folder(folder).await?` check from
// `fetch_new_messages()`, and then let `dc_receive_imf()` check
// if it's a spam message and should be hidden.
// 2. Or add a flag to the ChatVersion header that this is a securejoin
// request, and return `true` here only if the message has this flag.
// `dc_receive_imf()` can then check if the securejoin request is valid.
return Ok(true);
}
if let Some(msg) = get_prefetch_parent_message(context, headers).await? {
if msg.chat_blocked != Blocked::Not {
// Blocked or contact request message in the spam folder, leave it there.
return Ok(None);
return Ok(false);
}
} else {
// No chat found.
@@ -1657,20 +1632,40 @@ async fn spam_target_folder(
from_field_to_contact_id(context, &mimeparser::get_from(headers), true).await?;
if blocked_contact {
// Contact is blocked, leave the message in spam.
return Ok(None);
return Ok(false);
}
if let Some(chat_id_blocked) = ChatIdBlocked::lookup_by_contact(context, from_id).await? {
if chat_id_blocked.blocked != Blocked::Not {
return Ok(None);
return Ok(false);
}
} else if from_id != DC_CONTACT_ID_SELF {
// No chat with this contact found.
return Ok(None);
return Ok(false);
}
}
if needs_move_to_mvbox(context, headers).await? {
Ok(true)
}
/// Returns target folder for a message found in the Spam folder.
/// If this returns None, the message will not be moved out of the
/// Spam folder, and as `fetch_new_messages()` doesn't download
/// messages from the Spam folder, the message will be ignored.
async fn spam_target_folder(
context: &Context,
folder: &str,
headers: &[mailparse::MailHeader<'_>],
) -> Result<Option<Config>> {
if !should_move_out_of_spam(context, headers).await? {
return Ok(None);
}
if needs_move_to_mvbox(context, headers).await?
// If OnlyFetchMvbox is set, we don't want to move the message to
// the inbox or sentbox where we wouldn't fetch it again:
|| context.get_config_bool(Config::OnlyFetchMvbox).await?
{
Ok(Some(Config::ConfiguredMvboxFolder))
} else if needs_move_to_sentbox(context, folder, headers).await? {
Ok(Some(Config::ConfiguredSentboxFolder))
@@ -2005,7 +2000,7 @@ async fn mark_seen_by_uid(
.sql
.query_row_optional(
"SELECT id, chat_id FROM msgs
WHERE rfc724_mid IN (
WHERE id > 9 AND rfc724_mid IN (
SELECT rfc724_mid FROM imap
WHERE folder=?1
AND uidvalidity=?2
@@ -2152,6 +2147,21 @@ pub async fn get_config_last_seen_uid(context: &Context, folder: &str) -> Result
}
}
/// Whether to ignore fetching messages from a folder.
///
/// This caters for the [`Config::OnlyFetchMvbox`] setting which means mails from folders
/// not explicitly watched should not be fetched.
async fn should_ignore_folder(context: &Context, folder: &str) -> Result<bool> {
if !context.get_config_bool(Config::OnlyFetchMvbox).await? {
return Ok(false);
}
if context.is_sentbox(folder).await? {
// Still respect the SentboxWatch setting.
return Ok(!context.get_config_bool(Config::SentboxWatch).await?);
}
Ok(!(context.is_mvbox(folder).await? || context.is_spam_folder(folder).await?))
}
/// Builds a list of sequence/uid sets. The returned sets have each no more than around 1000
/// characters because according to <https://tools.ietf.org/html/rfc2683#section-3.2.1.5>
/// command lines should not be much more than 1000 chars (servers should allow at least 8000 chars)
@@ -2406,7 +2416,7 @@ mod tests {
("Spam", true, true, "DeltaChat"),
];
// These are the same as above, but all messages in Spam stay in Spam
// These are the same as above, but non-chat messages in Spam stay in Spam
const COMBINATIONS_REQUEST: &[(&str, bool, bool, &str)] = &[
("INBOX", false, false, "INBOX"),
("INBOX", false, true, "INBOX"),
@@ -2417,9 +2427,9 @@ mod tests {
("Sent", true, false, "Sent"),
("Sent", true, true, "DeltaChat"),
("Spam", false, false, "Spam"),
("Spam", false, true, "Spam"),
("Spam", false, true, "INBOX"),
("Spam", true, false, "Spam"),
("Spam", true, true, "Spam"),
("Spam", true, true, "DeltaChat"),
];
#[async_std::test]

View File

@@ -3,7 +3,6 @@ use super::Imap;
use anyhow::{bail, Context as _, Result};
use async_imap::extensions::idle::IdleResponse;
use async_std::prelude::*;
use imap_proto::types::{AttributeValue, Response};
use std::time::{Duration, SystemTime};
use crate::{context::Context, scheduler::InterruptInfo};
@@ -72,13 +71,6 @@ impl Imap {
match fut.await {
Ok(Event::IdleResponse(IdleResponse::NewData(x))) => {
info!(context, "Idle has NewData {:?}", x);
if let Response::Fetch(_message, attrs) = x.parsed() {
for attr in attrs {
if let AttributeValue::ModSeq(modseq) = attr {
self.update_modseq(*modseq);
}
}
}
}
Ok(Event::IdleResponse(IdleResponse::Timeout)) => {
info!(context, "Idle-wait timeout or interruption");
@@ -165,7 +157,6 @@ impl Imap {
// in anything. If so, we behave as if IDLE had data but
// will have already fetched the messages so perform_*_fetch
// will not find any new.
match self.fetch_new_messages(context, &watch_folder, false).await {
Ok(res) => {
info!(context, "fetch_new_messages returned {:?}", res);

View File

@@ -10,7 +10,8 @@ use async_std::prelude::*;
use super::{get_folder_meaning, get_folder_meaning_by_name};
impl Imap {
pub(crate) async fn scan_folders(&mut self, context: &Context) -> Result<()> {
/// Returns true if folders were scanned, false if scanning was postponed.
pub(crate) async fn scan_folders(&mut self, context: &Context) -> Result<bool> {
// First of all, debounce to once per minute:
let mut last_scan = context.last_full_folder_scan.lock().await;
if let Some(last_scan) = *last_scan {
@@ -20,7 +21,7 @@ impl Imap {
.await?;
if elapsed_secs < debounce_secs {
return Ok(());
return Ok(false);
}
}
info!(context, "Starting full folder scan");
@@ -98,24 +99,26 @@ impl Imap {
}
last_scan.replace(Instant::now());
Ok(())
Ok(true)
}
}
pub(crate) async fn get_watched_folder_configs(context: &Context) -> Result<Vec<Config>> {
let mut res = vec![Config::ConfiguredInboxFolder];
if context.get_config_bool(Config::SentboxWatch).await? {
res.push(Config::ConfiguredSentboxFolder);
}
if context.should_watch_mvbox().await? {
res.push(Config::ConfiguredMvboxFolder);
}
Ok(res)
}
pub(crate) async fn get_watched_folders(context: &Context) -> Result<Vec<String>> {
let mut res = Vec::new();
if let Some(inbox_folder) = context.get_config(Config::ConfiguredInboxFolder).await? {
res.push(inbox_folder);
}
let folder_watched_configured = &[
(Config::SentboxWatch, Config::ConfiguredSentboxFolder),
(Config::MvboxMove, Config::ConfiguredMvboxFolder),
];
for (watched, configured) in folder_watched_configured {
if context.get_config_bool(*watched).await? {
if let Some(folder) = context.get_config(*configured).await? {
res.push(folder);
}
for folder_config in get_watched_folder_configs(context).await? {
if let Some(folder) = context.get_config(folder_config).await? {
res.push(folder);
}
}
Ok(res)

View File

@@ -197,7 +197,7 @@ impl Job {
"UPDATE jobs SET desired_timestamp=?, tries=?, param=? WHERE id=?;",
paramsv![
self.desired_timestamp,
self.tries as i64,
i64::from(self.tries),
self.param.to_string(),
self.job_id as i32,
],
@@ -676,7 +676,7 @@ fn get_backoff_time_offset(tries: u32, action: Action) -> i64 {
if seconds < 1 {
seconds = 1;
}
seconds as i64
i64::from(seconds)
}
}
}

View File

@@ -7,7 +7,8 @@
clippy::all,
clippy::indexing_slicing,
clippy::wildcard_imports,
clippy::needless_borrow
clippy::needless_borrow,
clippy::cast_lossless
)]
#![allow(
clippy::match_bool,

View File

@@ -253,7 +253,8 @@ impl LoginParam {
sql.set_raw_config(key, Some(&self.imap.server)).await?;
let key = format!("{}mail_port", prefix);
sql.set_raw_config_int(key, self.imap.port as i32).await?;
sql.set_raw_config_int(key, i32::from(self.imap.port))
.await?;
let key = format!("{}mail_user", prefix);
sql.set_raw_config(key, Some(&self.imap.user)).await?;
@@ -273,7 +274,8 @@ impl LoginParam {
sql.set_raw_config(key, Some(&self.smtp.server)).await?;
let key = format!("{}send_port", prefix);
sql.set_raw_config_int(key, self.smtp.port as i32).await?;
sql.set_raw_config_int(key, i32::from(self.smtp.port))
.await?;
let key = format!("{}send_user", prefix);
sql.set_raw_config(key, Some(&self.smtp.user)).await?;

View File

@@ -183,7 +183,7 @@ impl rusqlite::types::ToSql for MsgId {
format_err!("Invalid MsgId {}", self.0).into(),
));
}
let val = rusqlite::types::Value::Integer(self.0 as i64);
let val = rusqlite::types::Value::Integer(i64::from(self.0));
let out = rusqlite::types::ToSqlOutput::Owned(val);
Ok(out)
}
@@ -194,7 +194,7 @@ impl rusqlite::types::FromSql for MsgId {
fn column_result(value: rusqlite::types::ValueRef) -> rusqlite::types::FromSqlResult<Self> {
// Would be nice if we could use match here, but alas.
i64::column_result(value).and_then(|val| {
if 0 <= val && val <= std::u32::MAX as i64 {
if 0 <= val && val <= i64::from(std::u32::MAX) {
Ok(MsgId::new(val as u32))
} else {
Err(rusqlite::types::FromSqlError::OutOfRange(val))

View File

@@ -1203,6 +1203,9 @@ impl MimeMessage {
if let Some(_disposition) = report_fields.get_header_value(HeaderDef::Disposition) {
let original_message_id = report_fields
.get_header_value(HeaderDef::OriginalMessageId)
// MS Exchange doesn't add an Original-Message-Id header. Instead, they put
// the original message id into the In-Reply-To header:
.or_else(|| report.headers.get_header_value(HeaderDef::InReplyTo))
.and_then(|v| parse_message_id(&v).ok());
let additional_message_ids = report_fields
.get_header_value(HeaderDef::AdditionalMessageIds)
@@ -1480,8 +1483,8 @@ async fn update_gossip_peerstates(
pub(crate) struct Report {
/// Original-Message-ID header
///
/// It MUST be present if the original message has a Message-ID according to RFC 8098, but MS
/// Exchange does not add it nevertheless, in which case it is `None`.
/// It MUST be present if the original message has a Message-ID according to RFC 8098.
/// In case we can't find it (shouldn't happen), this is None.
original_message_id: Option<String>,
/// Additional-Message-IDs
additional_message_ids: Vec<String>,
@@ -3183,10 +3186,32 @@ Message.
#[async_std::test]
async fn test_ms_exchange_mdn() -> Result<()> {
let t = TestContext::new_alice().await;
let raw =
t.set_config(Config::ShowEmails, Some("2")).await?;
let original =
include_bytes!("../test-data/message/ms_exchange_report_original_message.eml");
dc_receive_imf(&t, original, "INBOX", false).await?;
let original_msg_id = t.get_last_msg().await.id;
// 1. Test mimeparser directly
let mdn =
include_bytes!("../test-data/message/ms_exchange_report_disposition_notification.eml");
let mimeparser = MimeMessage::from_bytes(&t.ctx, raw).await?;
assert!(!mimeparser.mdn_reports.is_empty());
let mimeparser = MimeMessage::from_bytes(&t.ctx, mdn).await?;
assert_eq!(mimeparser.mdn_reports.len(), 1);
assert_eq!(
mimeparser.mdn_reports[0].original_message_id.as_deref(),
Some("d5904dc344eeb5deaf9bb44603f0c716@posteo.de")
);
assert!(mimeparser.mdn_reports[0].additional_message_ids.is_empty());
// 2. Test that marking the original msg as read works
dc_receive_imf(&t, mdn, "INBOX", false).await?;
assert_eq!(
original_msg_id.get_state(&t).await?,
MessageState::OutMdnRcvd
);
Ok(())
}
}

View File

@@ -366,7 +366,7 @@ static P_EXAMPLE_COM: Lazy<Provider> = Lazy::new(|| {
}
});
// fastmail.md: fastmail.com
// fastmail.md: 123mail.org, 150mail.com, 150ml.com, 16mail.com, 2-mail.com, 4email.net, 50mail.com, airpost.net, allmail.net, bestmail.us, cluemail.com, elitemail.org, emailcorner.net, emailengine.net, emailengine.org, emailgroups.net, emailplus.org, emailuser.net, eml.cc, f-m.fm, fast-email.com, fast-mail.org, fastem.com, fastemail.us, fastemailer.com, fastest.cc, fastimap.com, fastmail.cn, fastmail.co.uk, fastmail.com, fastmail.com.au, fastmail.de, fastmail.es, fastmail.fm, fastmail.fr, fastmail.im, fastmail.in, fastmail.jp, fastmail.mx, fastmail.net, fastmail.nl, fastmail.org, fastmail.se, fastmail.to, fastmail.tw, fastmail.uk, fastmail.us, fastmailbox.net, fastmessaging.com, fea.st, fmail.co.uk, fmailbox.com, fmgirl.com, fmguy.com, ftml.net, h-mail.us, hailmail.net, imap-mail.com, imap.cc, imapmail.org, inoutbox.com, internet-e-mail.com, internet-mail.org, internetemails.net, internetmailing.net, jetemail.net, justemail.net, letterboxes.org, mail-central.com, mail-page.com, mailandftp.com, mailas.com, mailbolt.com, mailc.net, mailcan.com, mailforce.net, mailftp.com, mailhaven.com, mailingaddress.org, mailite.com, mailmight.com, mailnew.com, mailsent.net, mailservice.ms, mailup.net, mailworks.org, ml1.net, mm.st, myfastmail.com, mymacmail.com, nospammail.net, ownmail.net, petml.com, postinbox.com, postpro.net, proinbox.com, promessage.com, realemail.net, reallyfast.biz, reallyfast.info, rushpost.com, sent.as, sent.at, sent.com, speedpost.net, speedymail.org, ssl-mail.com, swift-mail.com, the-fastest.net, the-quickest.com, theinternetemail.com, veryfast.biz, veryspeedy.net, warpmail.net, xsmail.com, yepmail.net, your-mail.com
static P_FASTMAIL: Lazy<Provider> = Lazy::new(|| Provider {
id: "fastmail",
status: Status::Preparation,
@@ -389,13 +389,6 @@ static P_FASTMAIL: Lazy<Provider> = Lazy::new(|| Provider {
port: 465,
username_pattern: Email,
},
Server {
protocol: Smtp,
socket: Starttls,
hostname: "smtp.fastmail.com",
port: 587,
username_pattern: Email,
},
],
config_defaults: None,
strict_tls: true,
@@ -716,8 +709,8 @@ static P_MAIL_DE: Lazy<Provider> = Lazy::new(|| Provider {
static P_MAIL_RU: Lazy<Provider> = Lazy::new(|| {
Provider {
id: "mail.ru",
status: Status::Ok,
before_login_hint: "Не рекомендуется использовать mail.ru, потому что он разряжает вашу батарею быстрее, чем другие провайдеры.",
status: Status::Preparation,
before_login_hint: "Вам необходимо сгенерировать \"пароль для внешнего приложения\" в веб-интерфейсе mail.ru, чтобы mail.ru работал с Delta Chat.",
after_login_hint: "",
overview_page: "https://providers.delta.chat/mail-ru",
server: vec![
@@ -905,7 +898,7 @@ static P_NAVER: Lazy<Provider> = Lazy::new(|| Provider {
oauth2_authorizer: None,
});
// outlook.com.md: hotmail.com, outlook.com, office365.com, outlook.com.tr, live.com
// outlook.com.md: hotmail.com, outlook.com, office365.com, outlook.com.tr, live.com, outlook.de
static P_OUTLOOK_COM: Lazy<Provider> = Lazy::new(|| Provider {
id: "outlook.com",
status: Status::Ok,
@@ -1495,7 +1488,123 @@ pub(crate) static PROVIDER_DATA: Lazy<HashMap<&'static str, &'static Provider>>
("example.com", &*P_EXAMPLE_COM),
("example.org", &*P_EXAMPLE_COM),
("example.net", &*P_EXAMPLE_COM),
("123mail.org", &*P_FASTMAIL),
("150mail.com", &*P_FASTMAIL),
("150ml.com", &*P_FASTMAIL),
("16mail.com", &*P_FASTMAIL),
("2-mail.com", &*P_FASTMAIL),
("4email.net", &*P_FASTMAIL),
("50mail.com", &*P_FASTMAIL),
("airpost.net", &*P_FASTMAIL),
("allmail.net", &*P_FASTMAIL),
("bestmail.us", &*P_FASTMAIL),
("cluemail.com", &*P_FASTMAIL),
("elitemail.org", &*P_FASTMAIL),
("emailcorner.net", &*P_FASTMAIL),
("emailengine.net", &*P_FASTMAIL),
("emailengine.org", &*P_FASTMAIL),
("emailgroups.net", &*P_FASTMAIL),
("emailplus.org", &*P_FASTMAIL),
("emailuser.net", &*P_FASTMAIL),
("eml.cc", &*P_FASTMAIL),
("f-m.fm", &*P_FASTMAIL),
("fast-email.com", &*P_FASTMAIL),
("fast-mail.org", &*P_FASTMAIL),
("fastem.com", &*P_FASTMAIL),
("fastemail.us", &*P_FASTMAIL),
("fastemailer.com", &*P_FASTMAIL),
("fastest.cc", &*P_FASTMAIL),
("fastimap.com", &*P_FASTMAIL),
("fastmail.cn", &*P_FASTMAIL),
("fastmail.co.uk", &*P_FASTMAIL),
("fastmail.com", &*P_FASTMAIL),
("fastmail.com.au", &*P_FASTMAIL),
("fastmail.de", &*P_FASTMAIL),
("fastmail.es", &*P_FASTMAIL),
("fastmail.fm", &*P_FASTMAIL),
("fastmail.fr", &*P_FASTMAIL),
("fastmail.im", &*P_FASTMAIL),
("fastmail.in", &*P_FASTMAIL),
("fastmail.jp", &*P_FASTMAIL),
("fastmail.mx", &*P_FASTMAIL),
("fastmail.net", &*P_FASTMAIL),
("fastmail.nl", &*P_FASTMAIL),
("fastmail.org", &*P_FASTMAIL),
("fastmail.se", &*P_FASTMAIL),
("fastmail.to", &*P_FASTMAIL),
("fastmail.tw", &*P_FASTMAIL),
("fastmail.uk", &*P_FASTMAIL),
("fastmail.us", &*P_FASTMAIL),
("fastmailbox.net", &*P_FASTMAIL),
("fastmessaging.com", &*P_FASTMAIL),
("fea.st", &*P_FASTMAIL),
("fmail.co.uk", &*P_FASTMAIL),
("fmailbox.com", &*P_FASTMAIL),
("fmgirl.com", &*P_FASTMAIL),
("fmguy.com", &*P_FASTMAIL),
("ftml.net", &*P_FASTMAIL),
("h-mail.us", &*P_FASTMAIL),
("hailmail.net", &*P_FASTMAIL),
("imap-mail.com", &*P_FASTMAIL),
("imap.cc", &*P_FASTMAIL),
("imapmail.org", &*P_FASTMAIL),
("inoutbox.com", &*P_FASTMAIL),
("internet-e-mail.com", &*P_FASTMAIL),
("internet-mail.org", &*P_FASTMAIL),
("internetemails.net", &*P_FASTMAIL),
("internetmailing.net", &*P_FASTMAIL),
("jetemail.net", &*P_FASTMAIL),
("justemail.net", &*P_FASTMAIL),
("letterboxes.org", &*P_FASTMAIL),
("mail-central.com", &*P_FASTMAIL),
("mail-page.com", &*P_FASTMAIL),
("mailandftp.com", &*P_FASTMAIL),
("mailas.com", &*P_FASTMAIL),
("mailbolt.com", &*P_FASTMAIL),
("mailc.net", &*P_FASTMAIL),
("mailcan.com", &*P_FASTMAIL),
("mailforce.net", &*P_FASTMAIL),
("mailftp.com", &*P_FASTMAIL),
("mailhaven.com", &*P_FASTMAIL),
("mailingaddress.org", &*P_FASTMAIL),
("mailite.com", &*P_FASTMAIL),
("mailmight.com", &*P_FASTMAIL),
("mailnew.com", &*P_FASTMAIL),
("mailsent.net", &*P_FASTMAIL),
("mailservice.ms", &*P_FASTMAIL),
("mailup.net", &*P_FASTMAIL),
("mailworks.org", &*P_FASTMAIL),
("ml1.net", &*P_FASTMAIL),
("mm.st", &*P_FASTMAIL),
("myfastmail.com", &*P_FASTMAIL),
("mymacmail.com", &*P_FASTMAIL),
("nospammail.net", &*P_FASTMAIL),
("ownmail.net", &*P_FASTMAIL),
("petml.com", &*P_FASTMAIL),
("postinbox.com", &*P_FASTMAIL),
("postpro.net", &*P_FASTMAIL),
("proinbox.com", &*P_FASTMAIL),
("promessage.com", &*P_FASTMAIL),
("realemail.net", &*P_FASTMAIL),
("reallyfast.biz", &*P_FASTMAIL),
("reallyfast.info", &*P_FASTMAIL),
("rushpost.com", &*P_FASTMAIL),
("sent.as", &*P_FASTMAIL),
("sent.at", &*P_FASTMAIL),
("sent.com", &*P_FASTMAIL),
("speedpost.net", &*P_FASTMAIL),
("speedymail.org", &*P_FASTMAIL),
("ssl-mail.com", &*P_FASTMAIL),
("swift-mail.com", &*P_FASTMAIL),
("the-fastest.net", &*P_FASTMAIL),
("the-quickest.com", &*P_FASTMAIL),
("theinternetemail.com", &*P_FASTMAIL),
("veryfast.biz", &*P_FASTMAIL),
("veryspeedy.net", &*P_FASTMAIL),
("warpmail.net", &*P_FASTMAIL),
("xsmail.com", &*P_FASTMAIL),
("yepmail.net", &*P_FASTMAIL),
("your-mail.com", &*P_FASTMAIL),
("firemail.at", &*P_FIREMAIL_DE),
("firemail.de", &*P_FIREMAIL_DE),
("five.chat", &*P_FIVE_CHAT),
@@ -1539,6 +1648,7 @@ pub(crate) static PROVIDER_DATA: Lazy<HashMap<&'static str, &'static Provider>>
("office365.com", &*P_OUTLOOK_COM),
("outlook.com.tr", &*P_OUTLOOK_COM),
("live.com", &*P_OUTLOOK_COM),
("outlook.de", &*P_OUTLOOK_COM),
("posteo.de", &*P_POSTEO),
("posteo.af", &*P_POSTEO),
("posteo.at", &*P_POSTEO),
@@ -1743,4 +1853,4 @@ pub(crate) static PROVIDER_IDS: Lazy<HashMap<&'static str, &'static Provider>> =
});
pub static PROVIDER_UPDATED: Lazy<chrono::NaiveDate> =
Lazy::new(|| chrono::NaiveDate::from_ymd(2022, 1, 11));
Lazy::new(|| chrono::NaiveDate::from_ymd(2022, 1, 31));

View File

@@ -11,6 +11,7 @@ use crate::dc_tools::maybe_add_time_based_warnings;
use crate::ephemeral::delete_expired_imap_messages;
use crate::imap::Imap;
use crate::job::{self, Thread};
use crate::log::LogExt;
use crate::smtp::{send_smtp_messages, Smtp};
use self::connectivity::ConnectivityStore;
@@ -169,25 +170,49 @@ async fn fetch_idle(ctx: &Context, connection: &mut Imap, folder: Config) -> Int
warn!(ctx, "{:#}", err);
}
// Scan other folders before fetching from watched folder. This may result in the
// messages being moved into the watched folder, for example from the Spam folder to
// the Inbox folder.
if folder == Config::ConfiguredInboxFolder {
// Only scan on the Inbox thread in order to prevent parallel scans, which might lead to duplicate messages
if let Err(err) = connection.scan_folders(ctx).await {
// Don't reconnect, if there is a problem with the connection we will realize this when IDLEing
// but maybe just one folder can't be selected or something
warn!(ctx, "{}", err);
}
}
// fetch
// Fetch the watched folder.
if let Err(err) = connection.fetch_move_delete(ctx, &watch_folder).await {
connection.trigger_reconnect(ctx).await;
warn!(ctx, "{:#}", err);
return InterruptInfo::new(false);
}
// Scan additional folders only after finishing fetching the watched folder.
//
// On iOS the application has strictly limited time to work in background, so we may not
// be able to scan all folders before time is up if there are many of them.
if folder == Config::ConfiguredInboxFolder {
// Only scan on the Inbox thread in order to prevent parallel scans, which might lead to duplicate messages
match connection.scan_folders(ctx).await {
Err(err) => {
// Don't reconnect, if there is a problem with the connection we will realize this when IDLEing
// but maybe just one folder can't be selected or something
warn!(ctx, "{}", err);
}
Ok(true) => {
// Fetch the watched folder again in case scanning other folder moved messages
// there.
//
// In most cases this will select the watched folder and return because there are
// no new messages. We want to select the watched folder anyway before going IDLE
// there, so this does not take additional protocol round-trip.
if let Err(err) = connection.fetch_move_delete(ctx, &watch_folder).await {
connection.trigger_reconnect(ctx).await;
warn!(ctx, "{:#}", err);
return InterruptInfo::new(false);
}
}
Ok(false) => {}
}
}
// Synchronize Seen flags.
connection
.sync_seen_flags(ctx, &watch_folder)
.await
.context("sync_seen_flags")
.ok_or_log(ctx);
connection.connectivity.set_connected(ctx).await;
// idle
@@ -350,7 +375,7 @@ impl Scheduler {
}))
};
if ctx.get_config_bool(Config::MvboxMove).await? {
if ctx.should_watch_mvbox().await? {
let ctx = ctx.clone();
mvbox_handle = Some(task::spawn(async move {
simple_imap_loop(

View File

@@ -5,6 +5,7 @@ use async_std::sync::{Mutex, RwLockReadGuard};
use crate::dc_tools::time;
use crate::events::EventType;
use crate::imap::scan_folders::get_watched_folder_configs;
use crate::quota::{
QUOTA_ERROR_THRESHOLD_PERCENTAGE, QUOTA_MAX_AGE_SECONDS, QUOTA_WARN_THRESHOLD_PERCENTAGE,
};
@@ -362,17 +363,14 @@ impl Context {
[
(
Config::ConfiguredInboxFolder,
None,
inbox.state.connectivity.clone(),
),
(
Config::ConfiguredMvboxFolder,
Some(Config::MvboxMove),
mvbox.state.connectivity.clone(),
),
(
Config::ConfiguredSentboxFolder,
Some(Config::SentboxWatch),
sentbox.state.connectivity.clone(),
),
],
@@ -391,20 +389,12 @@ impl Context {
// - "Sent": Connected
// =============================================================================================
let watched_folders = get_watched_folder_configs(self).await?;
ret += &format!("<h3>{}</h3><ul>", stock_str::incoming_messages(self).await);
for (folder, watch, state) in &folders_states {
let w = if let Some(watch_config) = *watch {
self.get_config(watch_config)
.await
.ok_or_log(self)
.flatten()
== Some("1".to_string())
} else {
true
};
for (folder, state) in &folders_states {
let mut folder_added = false;
if w {
if watched_folders.contains(folder) {
let f = self.get_config(*folder).await.ok_or_log(self).flatten();
if let Some(foldername) = f {

View File

@@ -343,19 +343,48 @@ pub(crate) async fn send_msg_to_smtp(
return Err(err);
}
let (body, recipients, msg_id) = context
// Increase retry count as soon as we have an SMTP connection. This ensures that the message is
// eventually removed from the queue by exceeding retry limit even in case of an error that
// keeps happening early in the message sending code, e.g. failure to read the message from the
// database.
context
.sql
.execute(
"UPDATE smtp SET retries=retries+1 WHERE id=?",
paramsv![rowid],
)
.await
.context("failed to update retries count")?;
let (body, recipients, msg_id, retries) = context
.sql
.query_row(
"SELECT mime, recipients, msg_id FROM smtp WHERE id=?",
"SELECT mime, recipients, msg_id, retries FROM smtp WHERE id=?",
paramsv![rowid],
|row| {
let mime: String = row.get(0)?;
let recipients: String = row.get(1)?;
let msg_id: MsgId = row.get(2)?;
Ok((mime, recipients, msg_id))
let retries: i64 = row.get(3)?;
Ok((mime, recipients, msg_id, retries))
},
)
.await?;
if retries > 6 {
message::set_msg_failed(
context,
msg_id,
Some("Number of retries exceeded the limit."),
)
.await;
context
.sql
.execute("DELETE FROM smtp WHERE id=?", paramsv![rowid])
.await
.context("failed to remove message with exceeded retry limit from smtp table")?;
bail!("Number of retries exceeded the limit");
}
let recipients_list = recipients
.split(' ')
.filter_map(
@@ -421,27 +450,16 @@ pub(crate) async fn send_msg_to_smtp(
};
match status {
Status::Finished(res) => {
context
.sql
.execute("DELETE FROM smtp WHERE id=?", paramsv![rowid])
.await?;
if res.is_ok() {
msg_id.set_delivered(context).await?;
context
.sql
.execute("DELETE FROM smtp WHERE id=?", paramsv![rowid])
.await?;
}
res
}
Status::RetryNow | Status::RetryLater => {
context
.sql
.execute(
"UPDATE smtp SET retries=retries+1 WHERE id=?",
paramsv![rowid],
)
.await
.context("failed to update retries count")?;
Err(format_err!("Retry"))
}
Status::RetryNow | Status::RetryLater => Err(format_err!("Retry")),
}
}
@@ -454,10 +472,6 @@ pub(crate) async fn send_smtp_messages(
connection: &mut Smtp,
) -> anyhow::Result<()> {
context.send_sync_msg().await?; // Add sync message to the end of the queue if needed.
context
.sql
.execute("DELETE FROM smtp WHERE retries > 5", paramsv![])
.await?;
let rowids = context
.sql
.query_map(

View File

@@ -787,7 +787,7 @@ async fn maybe_add_from_param(
async fn prune_tombstones(sql: &Sql) -> Result<()> {
sql.execute(
"DELETE FROM msgs
WHERE (chat_id=? OR hidden)
WHERE chat_id=?
AND NOT EXISTS (
SELECT * FROM imap WHERE msgs.rfc724_mid=rfc724_mid AND target!=''
)",
@@ -906,6 +906,23 @@ mod tests {
}
}
/// Regression test for a bug where housekeeping deleted drafts since their
/// `hidden` flag is set.
#[async_std::test]
async fn test_housekeeping_dont_delete_drafts() {
let t = TestContext::new_alice().await;
let chat = t.create_chat_with_contact("bob", "bob@example.com").await;
let mut new_draft = Message::new(Viewtype::Text);
new_draft.set_text(Some("This is my draft".to_string()));
chat.id.set_draft(&t, Some(&mut new_draft)).await.unwrap();
housekeeping(&t).await.unwrap();
let loaded_draft = chat.id.get_draft(&t).await.unwrap();
assert_eq!(loaded_draft.unwrap().text.unwrap(), "This is my draft");
}
/// Regression test.
///
/// Previously the code checking for existence of `config` table

View File

@@ -75,7 +75,7 @@ impl StatusUpdateId {
impl rusqlite::types::ToSql for StatusUpdateId {
fn to_sql(&self) -> rusqlite::Result<rusqlite::types::ToSqlOutput> {
let val = rusqlite::types::Value::Integer(self.0 as i64);
let val = rusqlite::types::Value::Integer(i64::from(self.0));
let out = rusqlite::types::ToSqlOutput::Owned(val);
Ok(out)
}

View File

@@ -1,43 +1,43 @@
Return-Path: <anonymous@example.org>
Return-Path: <bob@example.net>
Delivered-To: anonymous@posteo.de
Received: from proxy02.posteo.name ([127.0.0.1])
by dovecot16.posteo.name (Dovecot) with LMTP id Cp2uFxP1sWHbCQEAchYRkQ
for <anonymous@posteo.de>; Thu, 09 Dec 2021 13:25:38 +0100
by dovecot16.posteo.name (Dovecot) with LMTP id GaxcARout2HxiwMAchYRkQ
for <anonymous@posteo.de>; Mon, 13 Dec 2021 12:35:32 +0100
Received: from proxy02.posteo.de ([127.0.0.1])
by proxy02.posteo.name (Dovecot) with LMTP id MWsaCwrvsWG0wgEAGFAyLg
; Thu, 09 Dec 2021 13:25:38 +0100
Received: from mailin06.posteo.de (unknown [10.0.1.6])
by proxy02.posteo.de (Postfix) with ESMTPS id 4J8tXy0KkMz120l
for <anonymous@posteo.de>; Thu, 9 Dec 2021 13:25:38 +0100 (CET)
Received: from mx04.posteo.de (mailin06.posteo.de [127.0.0.1])
by mailin06.posteo.de (Postfix) with ESMTPS id F24DE215B8
for <anonymous@posteo.de>; Thu, 9 Dec 2021 13:25:37 +0100 (CET)
by proxy02.posteo.name (Dovecot) with LMTP id q/LiCqwqt2FMTQEAGFAyLg
; Mon, 13 Dec 2021 12:35:32 +0100
Received: from mailin05.posteo.de (unknown [10.0.1.5])
by proxy02.posteo.de (Postfix) with ESMTPS id 4JCKFJ1LCLz1214
for <anonymous@posteo.de>; Mon, 13 Dec 2021 12:35:32 +0100 (CET)
Received: from mx03.posteo.de (mailin05.posteo.de [127.0.0.1])
by mailin05.posteo.de (Postfix) with ESMTPS id 1B26420012
for <anonymous@posteo.de>; Mon, 13 Dec 2021 12:35:32 +0100 (CET)
X-Virus-Scanned: amavisd-new at posteo.de
X-Spam-Flag: NO
X-Spam-Score: 0.011
X-Spam-Level:
X-Spam-Status: No, score=0.011 tagged_above=-1000 required=7
tests=[HTML_MESSAGE=0.001, T_POSTEO_TLSINY=0.01] autolearn=disabled
X-Posteo-Antispam-Signature: v=1; e=base64; a=aes-256-gcm; d=27yedFdXeAzOobR4x685XJ/5e6WQmX8PP5pSnOlGU2a9Ismhk38wb5AS44xh1yeL5PUxla78UEsHwGkPR0IyPRlHWaLMFLd5CJZN3GzFfrj/2CuB+cd1hOLpp9hRmCebc3rchuDr
X-Posteo-Antispam-Signature: v=1; e=base64; a=aes-256-gcm; d=RzB41PpqvrD+cuxf3UAqQLiXQL4MHazHZcKeOYJw75deIl7zxtrLXqfAZCeq2IPKt/njRRONUbfuvNdvLxg4mBJ0Rnb53wFOKOqtEpTzxYoQff3yqBpGSohr0DBG26PyBHi7ba/7
Authentication-Results: posteo.de; dmarc=none (p=none dis=none) header.from=example.org
X-Posteo-TLS-Received-Status: TLSv1.2
Received: from mail.example.org (mail.example.org [0.0.0.0])
by mx04.posteo.de (Postfix) with ESMTPS id 4J8tXx38vRz10yw
for <anonymous@posteo.at>; Thu, 9 Dec 2021 13:25:37 +0100 (CET)
Received: from [192.168.1.11] (port=22105 helo=mail.example.org)
by mx03.posteo.de (Postfix) with ESMTPS id 4JCKFH2ZM7zyx0
for <alice@example.org>; Mon, 13 Dec 2021 12:35:31 +0100 (CET)
Received: from [192.168.1.11] (port=27040 helo=mail.example.org)
by mail.example.org with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384
(Exim 4.94.2)
(envelope-from <anonymous@example.org>)
id 1mvIUG-0007VC-2U
for anonymous@posteo.at; Thu, 09 Dec 2021 13:25:24 +0100
From: Anonymous <anonymous@example.org>
To: Anonymous <anonymous@posteo.at>
(envelope-from <bob@example.net>)
id 1mwjc6-0003hM-2K
for alice@example.org; Mon, 13 Dec 2021 12:35:26 +0100
From: Anonymous_2 <bob@example.net>
To: Anonymous_1 <alice@example.org>
Subject: Gelesen: Test message
Thread-Topic: Test message
Thread-Index: AQHX7Dt/+5f88Aokk0KrqG0hbF8dN6wqFvxh
Date: Thu, 9 Dec 2021 12:25:24 +0000
Message-ID: <1711fc3548cd4b2699ccd4fffac17713@anonymous>
In-Reply-To: <75dd051097b02468183707ad0dd62ebd@posteo.de>
Thread-Index: AQHX8BVZ9B3+kB6CNUCtJ9eQIONSNawwSqpi
Date: Mon, 13 Dec 2021 11:35:26 +0000
Message-ID: <59b1d0c94a8d4834b7ab779a76647d44@mail.example.org>
In-Reply-To: <d5904dc344eeb5deaf9bb44603f0c716@posteo.de>
Accept-Language: de-AT, de-DE, en-US
Content-Language: de-DE
X-MS-Has-Attach:
@@ -45,29 +45,29 @@ X-MS-TNEF-Correlator:
x-ms-exchange-transport-fromentityheader: Hosted
x-originating-ip: [192.168.120.215]
Content-Type: multipart/report;
boundary="_000_1711fc3548cd4b2699ccd4fffac17713anonymous_";
boundary="_000_59b1d0c94a8d4834b7ab779a76647d44mailexampleorg_";
report-type=disposition-notification
MIME-Version: 1.0
--_000_1711fc3548cd4b2699ccd4fffac17713anonymous_
--_000_59b1d0c94a8d4834b7ab779a76647d44mailexampleorg_
Content-Type: multipart/alternative;
boundary="_002_1711fc3548cd4b2699ccd4fffac17713anonymous_"
boundary="_002_59b1d0c94a8d4834b7ab779a76647d44mailexampleorg_"
--_002_1711fc3548cd4b2699ccd4fffac17713anonymous_
--_002_59b1d0c94a8d4834b7ab779a76647d44mailexampleorg_
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
Ihre Nachricht
An: Anonymous
An: Anonymous_2
Betreff: Test message
Gesendet: Mittwoch, 8. Dezember 2021 14:57:05 (UTC+01:00) Amsterdam, Ber=
lin, Bern, Rom, Stockholm, Wien
Gesendet: Montag, 13. Dezember 2021 12:33:58 (UTC+01:00) Amsterdam, Berl=
in, Bern, Rom, Stockholm, Wien
wurde am Donnerstag, 9. Dezember 2021 13:24:34 (UTC+01:00) Amsterdam, Berl=
in, Bern, Rom, Stockholm, Wien gelesen.
wurde am Montag, 13. Dezember 2021 12:34:40 (UTC+01:00) Amsterdam, Berlin,=
Bern, Rom, Stockholm, Wien gelesen.
--_002_1711fc3548cd4b2699ccd4fffac17713anonymous_
--_002_59b1d0c94a8d4834b7ab779a76647d44mailexampleorg_
Content-Type: text/html; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
@@ -83,26 +83,26 @@ ding-left: 4pt; border-left: #800000 2px solid; } --></style>
<font size=3D"2"><span style=3D"font-size:10pt;">
<div class=3D"PlainText">Ihre Nachricht <br>
<br>
&nbsp;&nbsp; An: Anonymous<br>
&nbsp;&nbsp; An: Anonymous_2<br>
&nbsp;&nbsp; Betreff: Test message<br>
&nbsp;&nbsp; Gesendet: Mittwoch, 8. Dezember 2021 14:57:05 (UTC&#43;01:00) =
Amsterdam, Berlin, Bern, Rom, Stockholm, Wien<br>
&nbsp;&nbsp; Gesendet: Montag, 13. Dezember 2021 12:33:58 (UTC&#43;01:00) A=
msterdam, Berlin, Bern, Rom, Stockholm, Wien<br>
<br>
&nbsp;wurde am Donnerstag, 9. Dezember 2021 13:24:34 (UTC&#43;01:00) Amster=
dam, Berlin, Bern, Rom, Stockholm, Wien gelesen.</div>
&nbsp;wurde am Montag, 13. Dezember 2021 12:34:40 (UTC&#43;01:00) Amsterdam=
, Berlin, Bern, Rom, Stockholm, Wien gelesen.</div>
</span></font>
</body>
</html>
--_002_1711fc3548cd4b2699ccd4fffac17713anonymous_--
--_002_59b1d0c94a8d4834b7ab779a76647d44mailexampleorg_--
--_000_1711fc3548cd4b2699ccd4fffac17713anonymous_
--_000_59b1d0c94a8d4834b7ab779a76647d44mailexampleorg_
Content-Type: message/disposition-notification
Final-recipient: RFC822; anonymous@example.org
Final-recipient: RFC822; bob@example.net
Disposition: automatic-action/MDN-sent-automatically; displayed
X-MSExch-Correlation-Key: coNC5vaCQkiAOjek1v1Uew==
X-Display-Name: Anonymous
X-MSExch-Correlation-Key: nf7/jgN6Qk+WzsrkY5s9WA==
X-Display-Name: Anonymous_2
--_000_1711fc3548cd4b2699ccd4fffac17713anonymous_--
--_000_59b1d0c94a8d4834b7ab779a76647d44mailexampleorg_--

View File

@@ -0,0 +1,34 @@
Received: We have to put a Received header here. Otherwise, the message would be ignored
because DC thinks it's a draft, and the test fails.
Alternatively, we could configure the Sentobox folder in the test.
MIME-Version: 1.0
Content-Type: multipart/alternative;
boundary="=_3293d145f71bf71c4bb97415536759d6"
Date: Mon, 13 Dec 2021 12:33:58 +0100
From: Anonymous_1 <alice@example.org>
To: Anonymous_2 <bob@example.net>
Subject: Test message
Return-Receipt-To: Anonymous_1 <alice@example.org>
Disposition-Notification-To: Anonymous_1 <alice@example.org>
Message-ID: <d5904dc344eeb5deaf9bb44603f0c716@posteo.de>
X-Sender: alice@example.org
User-Agent: Posteo Webmail
--=_3293d145f71bf71c4bb97415536759d6
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset=US-ASCII
This is a test!
Best regards
--=_3293d145f71bf71c4bb97415536759d6
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html; charset=UTF-8
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN">
<html><body style=3D'font-size: 10pt; font-family: Verdana,Geneva,sans-seri=
f'>
This is a test!<br /><br />Best regards
</body></html>
--=_3293d145f71bf71c4bb97415536759d6--