Commit Graph

236 Commits

Author SHA1 Message Date
Hocuri
396ec131fc Implement receiving mailing lists (#1964)
* Copypaste-merge my old work

* Start implementing mailinglists the new way

* Create pseudo contact

* Fine-tune docs

* Remove some unnecessary changes

* style

* Make a stock str

* Fix a crash. Yes, this line caused a panic when reconfiguring on Android

(without a reasonable error log). Also, I could not receive any messages
anymore.

* rfmt

* Add tests and make them pass

* Even more tests

* rfmt

* Enhance test and fix bug

* Don't update the sender name when prefetching because maybe it's a mailing list

* Use an enum instead of numbers for the decision

* Don't remove anyone from mailing lists

* Fix bug in the regex

* Adjust error msg

* Compile error after rebase

* Correctly emit event

* Add dc_msg_is_mailing_list so that you can find out whether messages in the deaddrop belong to mailing lists.

* Add received headers to unit tests

* Comments, small tweaks

* Use dc_msg_get_override_sender_name instead of dc_msg_get_sender_name

* Add dc_msg_get_sender_first_name() because sometimes the first name was not correctly shown in mailing lists

* small fixes, don't let the user modify mailing list groups

* Hide contacts for addresses like noreply@github.com and reply+AEJ...@reply.github.com

When testing mailing lists, I noticed that sometimes a mailing list
contact got a name (like, hocuri <noreply@github.com>). It turned out
that ages ago, I had accidentally written an email to - in this example
- hocuri <noreply@github.com> and it had been added to the contacts
list.

This hides email addresses from the contacts list that are obviously not
meant to be written at and prevents updating the names.

* Comment, clippy

* Replace u32 with ChatId

* Resolve lost of small issues from the reviews

* remove dc_msg_get_sender_first_name

* add dc_msg_get_real_chat_id()

this allows to check if a contact request belongs to a mailing list
and to show name/avatar/whatever of the mailinglist.

another approach was to duplicate some chat-apis for messages
(eg. dc_msg_is_mailing_list()) however that would require far more new apis.

the idea to change the behavior of dc_msg_get_chat_id() would be
more clean, however, that easily breaks existing implementations
and can led to hard to find bugs.

* remove now unused Message.is_mailing_list()

* if a name for a mailing list is missing, use List-ID

* fix comment

* fix error message

* document how dc_get_chat_contacts() works for mailing lists

* refine decide api (#2185)

* add DC_DECIDE* constants to deltachat.h, tweak documentation

* use StartChat/Block/NotNow instead of Yes/Never/NotNow

* decide_on_contact_request works on ctx/msg-id

functions working on message-objects
usually do not read or write directly on the database.

therefore, decide_on_contact_request()
should not work with message-objects as well,
it is even a bit misleading, as eg. chat-id of the object is not modified.

instead, the function works on context,
similar to dc_send_msg(), dc_create_chat() and so on.
for now, i moved it to context, could maybe be part od MsgId.

* Update src/chatlist.rs

Co-authored-by: Hocuri <hocuri@gmx.de>

Co-authored-by: Hocuri <hocuri@gmx.de>

* refine documentation

* re-add accidentally deleted Param::MailingList

* remove pseudo-contact in domain @mailing.list

1. the pseudo-contact was added to the mailing list contacts,
   which is not needed.
   might be that we want to add the recent contacts there in a subsequent pr
   (we do not know all contacts of a mailing list)

2. the pseudo-contact was used to block
   mailing lists; this is done by setting the chat to Blocked::Manually now

3- the pseudo-contact was used for unblocking;
   as it is very neat not to require additional ui for mailing list unblocking,
   might be that we introduce a similar pseudo-contact for this limited purpose
   in a subsequent pr, however, the pseudo-contact needs to exist only
   during unblocking then, maybe also the special domain is not needed,
   we'll see :)

* Move dc_decide_on_contact_request() up to the dc_context section as it's a member of dc_context now

More specifically, to the "handle messages" section

* re-introduce Chattype::Mailinglist (#2195)

* re-introduce Chattype::Mailinglist

* exhaustive chattype-check in fetch_existing_msgs() and get_summary2()

* exhaustive chattype-check in ndn_maybe_add_info_msg()

* exhaustive chattype-check in message::fill()

* remove dc_chat_is_mailing_list() from ffi

Co-authored-by: B. Petersen <r10s@b44t.com>
2021-02-07 23:37:04 +01:00
bjoern
0ef6a3060a fix From:-header bug (#2193)
* add failing test for From:-header bug

the test fails on the last check for quote-encapsulated utf-8 strings.

* Update mailparse

Co-authored-by: link2xt <link2xt@testrun.org>
2021-02-05 20:57:38 +01:00
Floris Bruynooghe
355e0145c0 Depend on anyhow directly
This removes the proxy via crate::error to depend on anyhow directly.
There is no benefit to this indirection and this makes it simpler to
see which error types are used.
2021-01-24 17:29:52 +03:00
bjoern
e9c582c4e4 forward html-messages properly (#2151)
* add failing tests for forwarding html-mails

* let MsgId.get_html() return an option

* write html-part to local database on forwarding

* add html-part to forwarded non-dc messages

* read HTML-parts from encrypted messages

* avoid clone()

* Received:-header is no longer needed since #2152

* Update src/html.rs

Co-authored-by: Floris Bruynooghe <flub@devork.be>

* Update src/html.rs

Co-authored-by: Floris Bruynooghe <flub@devork.be>

* Update src/mimeparser.rs

Co-authored-by: Floris Bruynooghe <flub@devork.be>

* prefer 'orig' over 'org' as abbreviation for 'original'

* improve comment on tests

* prefer 'try_into()' over 'as u32' to avoid panics on bad data

* simplify ffi

Co-authored-by: Floris Bruynooghe <flub@devork.be>
2021-01-23 23:21:18 +01:00
Floris Bruynooghe
83dd1c6232 New version of clippy has a lot of new lints
Lots of new clippy lints due to toolchain upgrade.

Made the Message::error field pub(crate) again, it was the odd one out
and it seemed a reasonable way to shut up clippy.
2021-01-19 11:20:33 +01:00
Alexander Krotov
5e8e77dfb6 clippy: forbid wildcard_imports 2021-01-12 12:06:13 +03:00
bjoern
e2688f6355 add option to access original message (#2125)
* draft API to deal with uncut message texts

* add column mime_modified

* add mime_modified flag to MimeParser and save it in the database

* save mime_headers also when mime_modified is set

* cargo fmt

* set mime_modified on parsed html-texts and when there are multiple alternative-parts; add test for that

* prototype functions, add to repl and ffi

* use correct mime_modified flag

* basically parse Mime-Structure to HTML

* add basic tests for HTML-parsing

* convert text/plain to html for getting original

* respect charset for plain texts

* make test more specific

* fix handling non-utf-8 charsets for plain messages

* add test for plain_to_html()

* add failing test for plaintext linkify

* linkify urls in plain text

* fix regex

* plain text linkify: add failing test for encapsulated links as <https://domain.com>

* plain text linkify: make encapsulated links as <https://domain.com> work

* plain text linkify: require word boundary at beginning of link, add tests for that

* plain text linkify: linkify emails

* plain text: support format=flowed

* plain text: support quotes

* make clippy happy

* set mime-modified also when simplify() cuts non-html messages, add tests for that

* streamline mime recursion

* repl tool: write original html to file for further processing

* convert cid:- to data:-protocol

* add a test for cid: to data: conversion

* make clippy happy

* fix html-tests to work with windows-lineends

* clarify what the returned html-code may contain

* add some more detailed doc comments

* add mime_modified column only if not exist

this additional check is needed
as the column may added with another dbversion in
some shipped beta-versions.

* incorporate documentation suggestions from review

* rename get_original_mime_html() to more simple get_html()

* rename api is_mime_modified() to more simple has_html(); internally, mime_modified-flag stays as-is, however

* rename MimeS to MimeMultipartType

* do not set mime-modified flag for encrypted messages that need extra-handling for saved mime-structure

* fix typo

* move get_msg_html() to MsgId.get_html()

* incorporate more documentation suggestions from review

* remove unused return value from collect_texts_recursive()

* avoid mime_modified being mutable in write-parts-loop

* move 'use futures::future::FutureExt' atop of html.rs

* move attributes defining plain-text to a dedicated structure

* more PlainText to separate file

* escape cid when building regex

* let dc_get_msg_html() return NULL when calling with bad param
2021-01-11 17:40:35 +01:00
Alexander Krotov
f98aa0d906 Make maybe_remove_bad_parts() non-async 2020-12-22 20:51:57 +03:00
Alexander Krotov
179dd0f3a1 Fix "dehtlm_failed" typo 2020-12-22 20:51:42 +03:00
Hocuri
03232eb79c Adapt other tests to the fact that now the subject is prepended more often 2020-12-18 18:44:16 +01:00
Hocuri
9edc6702f1 Fix #2078 2020-12-18 18:44:16 +01:00
Hocuri
2b207e1375 rustfmt 2020-12-18 18:35:54 +01:00
Hocuri
4d2c2130e8 Replace all t.ctx with t
Not sure if that's the way to go because, you know, merge conflicts, but
we could do this now.
2020-12-18 18:35:54 +01:00
Hocuri
ec83fae314 Parse name="quote" divs (#2104)
fix #1560 Replies in html-only format are not converted nicely wrt Quoting
2020-12-13 18:02:20 +01:00
Alexander Krotov
518e87b0cf Compare Chat-Disposition-Notification-To: address case-insensitively
Otherwise users with uppercase letters in their addresses never receive
read receipts.
2020-12-06 23:29:11 +03:00
Alexander Krotov
fc1f44c6d6 Spellcheck 2020-12-06 04:26:20 +03:00
B. Petersen
f774665921 be more graceful when the apostrophed-encoding is used erroneous - either by other MUA or by Delta Chat up to core1.50 2020-12-06 04:24:39 +03:00
B. Petersen
8fcb8c3788 add failing test for erroneous apostrophed-encoding 2020-12-06 04:24:39 +03:00
B. Petersen
af045c245d warn about encoding errors; this requires Context 2020-12-04 12:25:45 +01:00
B. Petersen
0bdd1b7dc2 avoid unwrap() when falling back to latin1 charset 2020-12-04 11:42:14 +01:00
B. Petersen
e77706f7d0 add test apostrophed-encoding with cp1252 charset (aka ANSI aka Windows-1252) 2020-12-04 01:28:24 +01:00
B. Petersen
8de73c4566 handle apostrophed-encoding with non-utf-8 charsets 2020-12-04 01:19:03 +01:00
B. Petersen
56e6c2712b add failing test for apostroped windows-1251 encoding 2020-12-04 00:19:19 +01:00
B. Petersen
b510d74c4a add test for apostrophed-encoded filenames 2020-12-03 17:39:00 +01:00
B. Petersen
966712019f add test for bad encoded-words delimiter 2020-12-03 17:38:55 +01:00
B. Petersen
f919e4962d add test for cyrillic encoded-words filenames 2020-12-03 17:01:32 +01:00
B. Petersen
412645e1ce add test for binary word-encoded filenames 2020-12-03 17:01:32 +01:00
B. Petersen
1321a78f87 streamline test 2020-11-24 14:40:28 +01:00
B. Petersen
210d8bad04 allow specification of both, and and also do the apostrope-decoding when specified by a trailing asterix in the key. add tests for that. 2020-11-24 14:40:28 +01:00
B. Petersen
e19d2cccfc prefer 'if let' over 'unwrap()' 2020-11-24 14:40:28 +01:00
B. Petersen
a29dc514d3 decode filenames as CHARSET'LANG'test%2E%70%64%66 2020-11-24 14:40:28 +01:00
Alexander Krotov
66907c17d3 mimeparser: preserve quotes in messages with attachments 2020-11-08 12:01:35 +03:00
Alexander Krotov
11d9fcad35 Display a quote if top posting is detected
Previously quote at the end was always displayed as [...].
2020-10-26 18:03:30 +03:00
Hocuri
26aeacc6be @link2xt's review 2020-10-19 15:23:18 +02:00
Hocuri
4d2542cee5 Don't show HTML if there is no content and there is a file attached
Fix https://github.com/deltachat/deltachat-core-rust/issues/1982
2020-10-19 13:07:55 +02:00
Alexander Krotov
67cddedf7e Switch from lazy_static to once_cell 2020-10-18 15:47:21 +03:00
Alexander Krotov
19697e255e Parse normal MUA messages consisting of only a quote 2020-10-16 14:57:02 +02:00
B. Petersen
45dae1ff0c protect against attackers dropping the protect-this-chat message by not showing unprotected messages directly; this is done by checking the Chat-Verified flag on each incoming message. moreover, make sure, the flag is signed+encrypted (it must be read from the protected headers). 2020-10-13 14:59:27 +02:00
B. Petersen
12cf89735c handling incoming protection-changes messages, always add info-msg 2020-10-13 14:59:26 +02:00
B. Petersen
d240bbcd07 if a message is replaced by an error, this also removes special is_system_message states 2020-10-13 14:59:26 +02:00
B. Petersen
5e07a36cd2 add/send info-message on protection changes 2020-10-13 14:59:26 +02:00
Hocuri
be88b946b6 Peek reipients, fetch existing messages
Read all of an e-mail accounts messages and extract all To/CC addresses
if the From was from our own account.
Then, fetch existing messages from the server and show them.

Also, I fixed two other things:
- just by chance my test failed because of an completely unrelated bug.
The bug: bcc_self messages were not marked as read if mvbox_move was set
to true.
- add some color to the test output (minor change)
2020-10-10 15:56:30 +03:00
Alexander Krotov
20182b027e Add quote API
Sticky encryption rule, requiring that all replies to encrypted messages
are encrypted, applies only to messages with a quote now.

Co-Authored-By: B. Petersen <r10s@b44t.com>
2020-10-10 12:41:43 +03:00
René Rössler
9739c0305b Accessible msg error and type changes 2020-10-08 11:51:04 +02:00
Alexander Krotov
332a387c98 Fix nightly clippy warnings 2020-09-19 17:49:32 +03:00
Floris Bruynooghe
ee27c7d9d4 Run clippy on tests and examples 2020-09-05 18:13:16 +02:00
Alexander Krotov
0f164861c7 Fix clippy::iter_next_slice errors 2020-08-28 02:13:23 +03:00
Alexander Krotov
763334d0aa Sort message replies after parent message 2020-08-22 13:57:29 +03:00
Alexander Krotov
be922eef0f Format plain text as Format=Flowed DelSp=No
This avoids triggering spam filters which require that lines are wrapped
to 78 characters.
2020-08-22 13:57:10 +03:00
B. Petersen
77c61ab25b fix threading in interation with non-delta-clients
threading was broken in core43 as this flags unencrypted messages as errors
and errors are not replied-to.

the fix is not to mark missing signatures for unencrypted messages as errors.
2020-08-18 11:43:29 +02:00