Commit Graph

3039 Commits

Author SHA1 Message Date
bjoern
500e2d62a0 remove sentbox_move (#3111)
* remove SentboxMove

* adapt python test to removed sendbox_move option

* update CHANGELOG
2022-03-08 11:29:45 +01:00
holger krekel
63688a2f95 remove getAllUpdates() and add a typical replicatio API for the update call (#3081)
* (r10s, adb, hpk) remove getAllUpdates() and add a typical replica-API that works with increasing serials.  Streamline docs a bit.

* adapt ffi to new api

* documentation: updates serials may have gaps

* get_webxdc_status_updates() return updates larger than a given serial

* remove status_update_id from status-update-event; it is not needed (ui should update from the last known serial) and easily gets confused with last_serial

* unify wording to 'StatusUpdateSerial'

* remove legacy payload format, all known webxdc should be adapted

* add serial and max_serial to status updates

* avoid races when getting max_serial by avoiding two SQL calls

* update changelog

Co-authored-by: B. Petersen <r10s@b44t.com>
2022-03-04 20:22:48 +01:00
Floris Bruynooghe
ef841b1aa3 Securejoin: store bobstate in database instead of context
The state bob needs to maintain during a secure-join process when
exchanging messages used to be stored on the context.  This means if
the process was killed this state was lost and the securejoin process
would fail.  Moving this state into the database should help this.

This still only allows a single securejoin process at a time, this may
be relaxed in the future.  For now any previous securejoin process
that was running is killed if a new one is started (this was already
the case).

This can remove some of the complexity around BobState handling: since
the state is in the database we can already make state interactions
transactional and correct.  We no longer need the mutex around the
state handling.  This means the BobStateHandle construct that was
handling the interactions between always having a valid state and
handling the mutex is no longer needed, resulting in some nice
simplifications.

Part of #2777.
2022-03-01 23:02:40 +01:00
link2xt
0e50bc1443 Fix 1.59 clippy warnings 2022-02-27 13:29:02 +00:00
Hocuri
7c4a6ddcdf Add AcManager (#3073)
* Add AcManager

See https://github.com/deltachat/deltachat-core-rust/pull/2901#issuecomment-998285039

This reduces boilerplate code again therefore, improving the
signal-noise-ratio and reducing the mental barrier to start
writing a unit test.

Slightly off-topic:

I didn't add any advanced functions like `manager.get("alice");` because
they're not needed yet; however, once we have the AcManager we can
think about fancy things like:

```rust
acm.send_text(&alice, "Hi Bob, this is Alice!", &bob);
```
which automatically lets bob receive the message.

However, this may be less useful than it seems at first, since most of
the tests I looked at wouldn't benefit from it, so at least I won't do
it until I have a test that would benefit from it.

* Remove unnecessary RefCell

* Rename AcManager to TestContextManager

* Don't store TestContext's in a vec for now as we don't need this; we can re-add it later

* Rename acm -> tcm
2022-02-23 19:34:47 +01:00
link2xt
7ab6d95b6c mimefactory: place common IMF headers at the top of the message
This moves most common headers like From, To, Subject etc. defined in
the Internet Message Format standard at the top of the message
in the same order as used in RFC 5322.
2022-02-23 17:51:15 +00:00
link2xt
6c32b89906 smtp: add more logging 2022-02-23 17:04:30 +00:00
Hocuri
19a32cdfd3 Let MS Exchange MDNs mark the In-Reply-To message as read (#3075)
Fix https://github.com/deltachat/deltachat-core-rust/issues/2891
2022-02-17 09:56:00 +01:00
link2xt
d708f386a1 smtp: set message state to failed when retry limit is exceeded 2022-02-13 08:59:52 +00:00
link2xt
0f837f4bed Fix a comment typo 2022-02-12 16:29:42 +00:00
link2xt
242e8e2bb3 smtp: remove the message in case of permanent failure
When `smtp_send` returns `Status::Finished`,
the message should be removed from the queue even in case of
failure, such as a permanent error.

In addition to this bugfix, move the retry count increase to
the beginning of `send_msg_to_smtp` to ensure no message is
retried infinitely even in case of similar bugs.
2022-02-12 16:20:13 +00:00
Hocuri
bb9138708a Fix disappearing drafts (#3067) 2022-02-10 10:05:30 +01:00
Hocuri
34f5510f1f Don't directly download messages from the Spam folder (#3015)
fix #3007

My approach is:

We don't download any messages from the spam folder anymore, and only download them if they were moved out. This means that is-it-spam logic only resides in spam_target_folder(). This has some implications, see the comments.
2022-02-10 09:06:22 +01:00
link2xt
196075c031 imap: batch message deletion 2022-02-06 11:42:30 +00:00
link2xt
2e5e8f73c6 imap: simplify get_quota_roots() 2022-02-06 15:17:05 +00:00
link2xt
ada5d38272 imap: remove unwrap() 2022-02-06 14:07:04 +00:00
link2xt
276daf631e imap: move messages in batches
Also change how NO response is treated. NO response means there is an
error moving/copying the messages. When there are no matching
messages, the response is "OK No matching messages, so nothing copied"
according to some RFC 9051 examples.
2022-02-05 22:15:46 +00:00
link2xt
fb19b58147 Reduce number of unsafe as conversions
Enable clippy::cast_lossless lint and get rid of
some conversions pointed out by  clippy::as_conversions.
2022-02-05 12:42:14 +00:00
bjoern
1caf3caf1b do set_visibility() in a transaction (#3053)
this avoids archived chats containing fresh messages:

before, it could happen that between the two SQL calls
a new fresh message arrives,
unarchives the chat that is immediately archived by the second SQL call -
resulting in an archive chat containing fresh messages.

as fresh messages counter are shown on app icon etc.
this is pretty weird for the user as they do not see what is "fresh".

the other way round,
there is no transaction in receive_imf(),
however, receive_imf() only unarchives chats,
so that is visible and no big issue for the user.

the issue is rare at all,
however, annoying if you get that as the badge counter may be stuck at "1"
nearly forever (until you open the archived chat in question).
2022-02-03 20:40:24 +01:00
link2xt
cccdc51ad4 Optimize delete_expired_imap_messages()
For me this reduced query time from 0.3 s to 0.05 s.
2022-01-31 20:34:01 +00:00
link2xt
f68088cfb5 imap: avoid reconnection loop when message without Message-ID is marked as seen
- do not attempt to mark reserved meessages as seen when
  messages with empty Message-ID are marked as seen on IMAP
- do not reconnect on Seen flag synchronization failures This avoid
  reconnection loops in case of permanent errors in `sync_seen_flags`
2022-01-31 00:00:00 +00:00
bjoern
a43fc47bb6 update provider database (#3043)
* update provider database

ran `./src/provider/update.py ../provider-db/_providers/ > src/provider/data.rs`

* update changelog
2022-01-31 16:07:20 +01:00
Floris Bruynooghe
97853c3660 Flub/watch mvbox only (#3028)
* Make set_config() look a bit nicer

* Add OnlyFetchMvbox option

* Add test for the config

* Add option to only watch mvbox

This is supposed to support having a server-side rule which moves
emails to the mvbox already.  The new option makes sure the mvbox is
wathched and also makes sure no messages are feched from folders other
than the mvbox and the spam folder if enabled.  It does not interact
with the other settings.

* Fixup ignore conditions

* Cleanup some bits

* Watch the mvbox when `WatchMvboxOnly` is set

* Rename back to only_fetch_mvbox (flub said it's OK for him)

* typo

* clippy, more typos

Co-authored-by: Hocuri <hocuri@gmx.de>
2022-01-31 13:39:48 +01:00
link2xt
f304a30193 imap: fetch Inbox before scanning other folders 2022-01-31 12:03:21 +01:00
link2xt
7eadca3959 imap: do not synchronize Seen flags on unwatched folders
Synchronizing seen flags doubles the time required to scan all
folders. Delta Chat only marks messages as Seen on Inbox or DeltaChat,
so there is no need to check for Seen flag on other folders.
2022-01-30 20:00:00 +00:00
link2xt
8aa6decbf9 imap: call delete_expired_imap_messages() less often
This operation takes roughly 0.3 s on a moderate size database.
Calling it once before scanning all folders and scanning
the watched folder instead of each time after downloading
a message from a folder speeds up IMAP loop.
2022-01-30 20:49:32 +00:00
link2xt
7cf4bcaca2 imap: call delete_expired_imap_messages() less often
This operation takes roughly 0.3 s on a moderate size database.
Calling it once before scanning all folders and scanning
the watched folder instead of each time after downloading
a message from a folder speeds up IMAP loop.
2022-01-30 20:47:32 +00:00
link2xt
e858a32aa1 smtp: cancel message sending by removing the message
This restores the logic removed in
afd8c0d879
2022-01-30 10:59:10 +00:00
B. Petersen
99f2680e2c fix splitting off text from webxdc messages
moreover, make the split check exhaustive
to avoid the same error on the next added Viewtype.
2022-01-30 11:49:09 +01:00
B. Petersen
7a9a323bac test sending webxdc+text 2022-01-30 11:49:09 +01:00
link2xt
0cb9e7922a Remove direct dependency on byteorder crate 2022-01-29 23:24:25 +00:00
link2xt
e73107006e smtp: replace thiserror with anyhow 2022-01-29 16:41:47 +00:00
B. Petersen
d342d59e65 use webxdc app name in chatlist/quotes/replies/etc
this uses `get_webxdc_info().name` for chatlist etc.
the previuosly used static strings comes from a time
where we just did not had the correct name.

i was also thinking about adding `get_webxdc_info().summary`,
however, as this information is dynamic,
that may open several issues, eg. quoted text may change
so that the answer is out of context.
2022-01-29 16:48:24 +01:00
link2xt
2690fa2da5 Don't watch Sent folder by default 2022-01-29 11:35:02 +00:00
Hocuri
f6807d6b22 Fix set_config_bool() (#3013) 2022-01-27 12:12:16 +01:00
Hocuri
5f939c3123 Fix: Run migrations after importing backup again (#3006)
As of #2980, the migrations were not run after importing a backup. This lead to errors like "Failed to send message: no such table: smtp" if you imported a backup from a previous DC version.
2022-01-24 18:36:28 +03:00
B. Petersen
8ca54f616e raise webxdc sending limit to 640 kb
as discussed in dev chat,
100 kb is too low if one wants to use any game framework.
also, many game that doesn't have poor graphics
or uses magic CSS-tricks to do fancy draws,
will have more than 100 kb probably just in tiles and sprites.

even for regions with low resources,
100 kb is low for a game -
esp. if we compare that with stickers that are send around as well
are not deduplicated and also have several 100 kb in size -
for much less effect.

we should still encourage ppl to create tiny apps,
however, a too low limit also restricts possibilities wrt adaption.
2022-01-23 12:48:25 +01:00
link2xt
f7f899f0a4 smtp: retry immediately if connection is stale 2022-01-22 21:17:01 +00:00
B. Petersen
05a3c0c89b webxdc: synchronous state for read-only-chats
already before,
we did _not_ sent updates to contact requests or other read-only-chats.
however, we _did_ modify the local database,
so that getAllUpdates() returns an update that was not sent out to other peers.

this is fixed by checking can_send() soon.

that way, all peers have the same state
also for contact requests or other read-only-chats
and webxdc in contact requests can be opened as usual.

further ui improvements may be needed for contact requests
(maybe allow the webxdc to know about that state or
maybe show a warning in the ui somewhere),
however, this is not part of this pr.
2022-01-22 21:28:47 +01:00
link2xt
f21691c122 Add "database_encrypted" field to Context.get_info() 2022-01-22 14:43:37 +00:00
bjoern
7f43d3bb37 let sending invalid webxdc fail (#2993)
* let sending invalid webxdc fail

invalid webxdc can still be send as Viewtype::File, however
(maybe one want to discuss errors or so ;)

* clarify the supported zip compression methods
2022-01-22 11:19:21 +01:00
bjoern
315e4215d9 make update messages work if a key is missing (#2998)
* add a test for unencrypted replies to encrypted webxdc instances

* make update messages work if a key is missing

even in opportunistic chats,
replies to encrypted messages are forced to be encrypted,
if that is not possbile, message sending fails.

while this is okay to not leak previously send text messages,
the quotes as used by webxdc are more artificial,
currently only the static text "Webxdc".

* changelog ...
2022-01-22 10:56:15 +01:00
Floris Bruynooghe
260cb78e3a Re-write the blob filename creation loop
This was written in a way which attempted to avoid easily creating an
infinite loop.  But really that's a python idiom and doesn't work very
well in Rust.  Worse, as shown by #2972 it is really easy to still get
this wrong.  Instead do this the rust way, this way the compiler can
also reason properly about the branches and what is unreachable
removing some bogus dead code.
2022-01-20 21:55:32 +01:00
link2xt
a1f04d2129 imex: use param2 for passphrase 2022-01-16 13:22:08 +00:00
bjoern
9b562eebcd handle parent for webxdc info-messages (#2984)
* set parent for webxdc info-messages

* test parent() for info-messages

* add dc_msg_get_parent() ffi
2022-01-19 11:46:32 +01:00
bjoern
8d3227a92b fix webxdc forwarding and drafts (#2979)
* fix forwarding webxdc instances, add a test for that

* adapt webxdc test to fail on info-messages added when in draft mode

* do not add info-messages when in draft-mode

* half the number of instance-loads per webxdc update send/receive
2022-01-17 14:23:35 +01:00
Hocuri
c6d855084e Save "configured" flag later (#2974)
While experimenting with encrypted storage, once configuring failed between 920 and 940. But as the "configured" config had already been written after progress 910, some part of the code thought we are configured, some didn't.
2022-01-16 20:37:28 +01:00
Hocuri
827b3f8aeb Create parent directory if creating a new file fails (#2978)
With this PR, my encrypted-storage Android PR now works, at least I
couldn't find any further bugs.

Without it, configuring fails with: `Failed to create blob
icon-saved-messages-1803424689.png in
/data/user/0/com.b44t.messenger.beta/files/accounts/0e402b37dcd14a9586aea46294c908f2/dc.db-blobs:
No such file or directory (os error 2)`.

Also see https://github.com/deltachat/deltachat-core-rust/pull/2972.
2022-01-16 20:37:02 +01:00
link2xt
afd8c0d879 Add smtp table
It replaces SendMsgToSmtp job.

Prepared outgoing SMTP payloads are stored in the database now rather
than files in blobdir.
2022-01-01 19:14:53 +00:00
B. Petersen
6316ee7c9b add editable "summary" to dc_msg_get_webxdc_info()
the summary can be modified by the apps using
`sendUpdate({summary: "foo", payload: ...})`

the summary is updated when there is no newer update
and chat will be informed by the change as usual by
`DC_EVENT_MSGS_CHANGED`.
2022-01-16 00:30:53 +01:00