Commit Graph

16 Commits

Author SHA1 Message Date
link2xt
12823c2213 Add imap table to keep track of message UIDs
`imap` table maps Message-IDs to UIDs on the server. `dc_receive_imf`
no longer gets the UID of the message as an argument and does not
insert the folder and UID of the message into the `msgs`
table. `server_folder` and `server_uid` columns in `msgs` table are
deprecated.

MoveMsg and DeleteMsgOnImap jobs are removed. Now messages are moved
and deleted only in the `fetch_move_delete` procedure that consults
the `target` column of the `imap` table to determine where the message
should go.

Where the message should go is determined after prefetching by the
`imap::target_folder()` procedure.  Messages are only downloaded once
they reach their target folder to avoid race conditions in multidevice
setting, such as:

1. One device trying to FETCH the message while the other tries to
MOVE it.

2. One device marking the message as \Seen in the Inbox while the
other has already copied unseen message to the Movebox and is going to
delete the \Seen message in the Inbox.

3. Device downloads the message from the Inbox while there are newer
messages in the Movebox placed there by the other device, thus
processing the messages out of order.
2021-12-19 00:00:00 +00:00
link2xt
01db8d0130 test_utils: fix Alice's address in alice_keypair()
The key stored in test-data has @example.org identity.
2021-12-05 09:14:58 +00:00
link2xt
2b7bf11b05 Rust documentation improvements
Document all public modules and some methods.

Make some internal public symbols private.
2021-08-22 15:34:14 +02:00
link2xt
a47c0486ae Store mime_headers as BLOBs
Raw MIME messages may contain non-ASCII characters. Attempting to
store them as TEXT by using String::from_utf8_lossy results in
non-ASCII characters being replaced with Unicode U+FFFD "REPLACEMENT
CHARACTER" which is later incorrectly decoded when attempting to parse
`mime_headers` content into HTML.
2021-06-19 17:49:26 +03:00
link2xt
8610b0c945 sql: switch from sqlx to rusqlite 2021-04-25 22:33:14 +03:00
Friedel Ziegelmayer
6bb5721f29 feat: improve internal sql interface
Switches from rusqlite to sqlx to have a fully async based interface
to sqlite.

Co-authored-by: B. Petersen <r10s@b44t.com>
Co-authored-by: Hocuri <hocuri@gmx.de>
Co-authored-by: link2xt <link2xt@testrun.org>
2021-04-06 16:06:11 +02:00
B. Petersen
6f4090fbf6 make clippy happy and avoid unneeded evaluation 2021-03-04 23:24:59 +01:00
B. Petersen
29cbbf9ce8 let get_html() return first instead of last text
usually, there is at most one text/html and one text/plain part.

multiple text/plain parts,
are known only by mailinglist software that wants to add a footer
but cannot alter the original content part -
maybe because it is html and things are hard to stitch
or maybe because the content part is cryptographically signed
and cannot be altered therefore.
2021-03-04 23:24:59 +01:00
link2xt
a698a8dd84 Update to Rust 1.50
Also run rustfmt, fix new clippy warnings.
2021-03-01 16:48:33 +03:00
link2xt
a88893f262 clippy: fix needless_borrow 2021-02-14 00:55:23 +03:00
Floris Bruynooghe
1e2ec8e264 Adapt to new TestContext api
The newer API is more consistent with it's naming.  This is somehow
fallout from a master rebase.
2021-01-30 18:36:47 +01:00
bjoern
1e6d8063c8 add dc_msg_set_html() api (#2153)
* draft dc_msg_set_html() api

* implement setting 'html to be send'

* test sending html-parts

* more flexible html-partbuilder

* write html-parts to database and also send them

* add 'sendhtml' command to repl tool
2021-01-27 12:56:22 +01:00
Floris Bruynooghe
355e0145c0 Depend on anyhow directly
This removes the proxy via crate::error to depend on anyhow directly.
There is no benefit to this indirection and this makes it simpler to
see which error types are used.
2021-01-24 17:29:52 +03:00
bjoern
e9c582c4e4 forward html-messages properly (#2151)
* add failing tests for forwarding html-mails

* let MsgId.get_html() return an option

* write html-part to local database on forwarding

* add html-part to forwarded non-dc messages

* read HTML-parts from encrypted messages

* avoid clone()

* Received:-header is no longer needed since #2152

* Update src/html.rs

Co-authored-by: Floris Bruynooghe <flub@devork.be>

* Update src/html.rs

Co-authored-by: Floris Bruynooghe <flub@devork.be>

* Update src/mimeparser.rs

Co-authored-by: Floris Bruynooghe <flub@devork.be>

* prefer 'orig' over 'org' as abbreviation for 'original'

* improve comment on tests

* prefer 'try_into()' over 'as u32' to avoid panics on bad data

* simplify ffi

Co-authored-by: Floris Bruynooghe <flub@devork.be>
2021-01-23 23:21:18 +01:00
Alexander Krotov
5e8e77dfb6 clippy: forbid wildcard_imports 2021-01-12 12:06:13 +03:00
bjoern
e2688f6355 add option to access original message (#2125)
* draft API to deal with uncut message texts

* add column mime_modified

* add mime_modified flag to MimeParser and save it in the database

* save mime_headers also when mime_modified is set

* cargo fmt

* set mime_modified on parsed html-texts and when there are multiple alternative-parts; add test for that

* prototype functions, add to repl and ffi

* use correct mime_modified flag

* basically parse Mime-Structure to HTML

* add basic tests for HTML-parsing

* convert text/plain to html for getting original

* respect charset for plain texts

* make test more specific

* fix handling non-utf-8 charsets for plain messages

* add test for plain_to_html()

* add failing test for plaintext linkify

* linkify urls in plain text

* fix regex

* plain text linkify: add failing test for encapsulated links as <https://domain.com>

* plain text linkify: make encapsulated links as <https://domain.com> work

* plain text linkify: require word boundary at beginning of link, add tests for that

* plain text linkify: linkify emails

* plain text: support format=flowed

* plain text: support quotes

* make clippy happy

* set mime-modified also when simplify() cuts non-html messages, add tests for that

* streamline mime recursion

* repl tool: write original html to file for further processing

* convert cid:- to data:-protocol

* add a test for cid: to data: conversion

* make clippy happy

* fix html-tests to work with windows-lineends

* clarify what the returned html-code may contain

* add some more detailed doc comments

* add mime_modified column only if not exist

this additional check is needed
as the column may added with another dbversion in
some shipped beta-versions.

* incorporate documentation suggestions from review

* rename get_original_mime_html() to more simple get_html()

* rename api is_mime_modified() to more simple has_html(); internally, mime_modified-flag stays as-is, however

* rename MimeS to MimeMultipartType

* do not set mime-modified flag for encrypted messages that need extra-handling for saved mime-structure

* fix typo

* move get_msg_html() to MsgId.get_html()

* incorporate more documentation suggestions from review

* remove unused return value from collect_texts_recursive()

* avoid mime_modified being mutable in write-parts-loop

* move 'use futures::future::FutureExt' atop of html.rs

* move attributes defining plain-text to a dedicated structure

* more PlainText to separate file

* escape cid when building regex

* let dc_get_msg_html() return NULL when calling with bad param
2021-01-11 17:40:35 +01:00