Commit Graph
100 Commits
Author SHA1 Message Date
B. Petersen 02bc334af5 update changelog for 1.51 2021-03-23 18:51:26 +01:00
B. Petersen 36aef6499d update provider database 2021-03-18 21:55:33 +01:00
B. Petersen 4ba9c2fafa fix clippy error on generating rust code from python 2021-03-18 21:55:33 +01:00
B. Petersen db3f87dd77 add a test that fails if 'References:'-header is missing 2021-03-14 14:38:53 +01:00
B. Petersen f43555b41c set References header to Message-ID on top-level messages to add some resilience against smtp-server changing Message-ID header 2021-03-14 14:38:53 +01:00
B. Petersen 4ab90f7069 add DoxygenLayout.xml file
`DoxygenLayout.xml` is picked up by doxygen automatically (as `Doxyfile`)
and its template can be generated by `doxygen -l DoxygenLayout.xml`.

after that, the following tweaks were done:
- remove all xml-parts but header (other defaults are fine so far)
- rename "Modules" header to "Constants" (there are no modules :)
- remove useless subitems from "Classes", eg. by name only shows as "d"
- reorder tabs so that "Classes" becomes the first one
2021-03-13 22:19:12 +03:00
B. Petersen 6963fd877d allow one additional boolean
the function is not complex and is only called once.
refactoring that seems to be a bit too much effort
and at least out of scope of this pr.
2021-03-12 11:30:25 +01:00
B. Petersen 045fbab7cd remove subsequent images inside multipart/related 2021-03-12 11:30:25 +01:00
B. Petersen f69bcc71ed mark child-parts of multipart/related as such 2021-03-12 11:30:25 +01:00
B. Petersen c1fddebc06 add test for multipart/related mails 2021-03-12 11:30:25 +01:00
B. Petersen 8703da83f5 comment in more detail about DC_DESIRED_TEXT_LEN and use old limit that had worked for some while okayish 2021-03-04 23:26:02 +01:00
B. Petersen 4ae86b4e61 truncate long texts and make the whole text accessible via has_html()/get_html() 2021-03-04 23:26:02 +01:00
B. Petersen e418d89c79 add failing test to test truncating 2021-03-04 23:26:02 +01:00
B. Petersen 6f4090fbf6 make clippy happy and avoid unneeded evaluation 2021-03-04 23:24:59 +01:00
B. Petersen 29cbbf9ce8 let get_html() return first instead of last text
usually, there is at most one text/html and one text/plain part.

multiple text/plain parts,
are known only by mailinglist software that wants to add a footer
but cannot alter the original content part -
maybe because it is html and things are hard to stitch
or maybe because the content part is cryptographically signed
and cannot be altered therefore.
2021-03-04 23:24:59 +01:00
B. Petersen cd3c2a6c6c add test with multiple text parts
the test has multiple text parts because
the content text is signed and
the mailinglist software wants to add some text
(and cannot alter signed part without breaking the signature)
2021-03-04 23:24:59 +01:00
B. Petersen 27a7fae9c6 add a test checking get_fresh_msgs() in combination with mute_until 2021-03-03 12:02:58 +01:00
B. Petersen 1deaf87b24 fix mute-condition in get_fresh_msgs() 2021-03-03 12:02:58 +01:00
B. Petersen 75adbd2c8f add mute options to repl tool 2021-03-03 12:02:58 +01:00
B. Petersen 5c2d6c22a0 remove additional text parts if we think they belong to a mailinglist footer 2021-02-28 21:03:39 +01:00
B. Petersen 40d7f3ff71 add failing test for mailinglists with footer in an extra mimepart 2021-02-28 21:03:39 +01:00
B. Petersen c3dd47beba add test for muting/unmuting wrt fresh messages 2021-02-27 21:08:53 +01:00
B. Petersen f9c5ad817b resultify get_fresh_msgs(), this will make eg. test fail on bad sql 2021-02-27 21:08:53 +01:00
B. Petersen 81cd577bf0 exclude muted chats from notified-list 2021-02-27 21:08:53 +01:00
B. Petersen f789de7044 get mailinglist name from From: if sender indicates a notification list 2021-02-27 00:03:30 +01:00
B. Petersen b035a721ef add a failing test for mailing list names hidden in 'From' 2021-02-27 00:03:30 +01:00
B. Petersen 38169b2aad check From: address without creating/altering a contact-record if it is not SELF 2021-02-24 14:53:52 +01:00
B. Petersen f973b75d6a unblock mailinglists via existing block-api
blocked mailinglists addresses are added to the contact table
before blocked contact list is created -
(this allows unblocking of blocked lists in previous testing releases,
however, more importantly, it keeps all blocking/unblocking code inside
contacts)

on unblocking such a contact,
the corresponding chat is unblocked as well.
2021-02-23 17:57:26 +01:00
B. Petersen a9aeea0ffc add failing test trying to unblock mailinglist 2021-02-23 17:57:26 +01:00
B. Petersen 79e72418bb resultify Contact::get_all_blocked() 2021-02-23 17:57:26 +01:00
B. Petersen 01af3b7547 fix sorting of blocked contacts (successor of #2206) 2021-02-23 17:57:26 +01:00
bjoern 6d93d7af63 add decision- and blocking-functions to repl, cleanup (#2258)
* deprecate mostly unused dc_get_blocked_cnt() api

instead, the size returned by get_blocked_contacts() should be checked,
this is safer and allows easier adaption of blocking rules.

ui or python seems not to use dc_get_blocked_cnt(),
however, there is one test in node,
therefore, the function will continue working for now
(by just returning Contact::get_all_blocked().len() then)

* add decision api to repl tool

* add block/unblock api to repl tool

* unify usage of @deprecated doxygen command
2021-02-22 14:58:49 +01:00
B. Petersen 6792523fcd perfer X-Microsoft-Original-Message-ID also for delete-check and uid-sync 2021-02-19 00:25:46 +03:00
B. Petersen 0fa90a81e5 prefer X-Microsoft-Original-Message-ID, if set
outlooks SMTP-server change the Message-ID of messages
and put the original Message-ID to X-Microsoft-Original-Message-ID.

the changed Message-ID has some issues:

- outgoing messages with bcc_self enabled are shown twice
  as the self-copy got the changed Message-ID while the database uses
  the original one

- read receipts do not work as they refer to the changed message id

- in general, sender and recipient see different Message-IDs

the issues can be fixed by

(1) let all receivers use the original Message-ID

	this is what this pr is doing
    and this should fix all issues with delta-to-delta communication,
	including groups, group-images etc.
    there may be issues left in communication
	with other MUAs as they are using another Message-ID.

(2) ftr: updating the Message-ID in the database of the sender to the new one

	this requires bcc_self always enabled (which is not the case)
    and may also result easily in race conditions
    (Bob answers before Alice sees its self-sent message),
    however, has the advantage of better compatibility with other MUA.

if needed, the compatibility with other MUA could be improved by remembering
both Messages-IDs, maybe we could treat the modified as References or so,
however, i think, this could be part of another PR if we know better about
real, in the wild issues.
2021-02-19 00:25:46 +03:00
bjoern 046a2a8eae set correct name for mailchimp mailinglists (#2243)
* add test for mailchimp mailinglists

* pass MimeMessage to create_or_lookup_mailinglist() (as of the other create*() routines) to allow more flexible name processing; document the function

* get mailing list name for mailchimp from From:-header

* make clippy happy

* add comment to '.list-id.mcsv.net' suffix
2021-02-16 10:24:44 +01:00
B. Petersen dab91574f2 adapt repl-tool to new mailinglist api 2021-02-15 23:11:02 +01:00
B. Petersen f77beaf4fc remove parameter having always the same value from create_or_lookup_mailinglist() 2021-02-15 21:38:31 +01:00
B. Petersen ffe68cadec update CHANGELOG.md 2021-02-15 21:38:31 +01:00
B. Petersen d4e90c7fff add a test for handling mailinglists without ListId-header 2021-02-15 21:38:31 +01:00
B. Petersen 0c2b3e838e support mailinglists based on Sender: header 2021-02-15 21:38:31 +01:00
B. Petersen e7c6667347 introduce MailinglistType for more exhaustive checking 2021-02-15 21:38:31 +01:00
B. Petersen 0b3fb9c0a3 prepend subject whenever we think sth. is a mailinglist, not only for a subset of mailing lists 2021-02-15 21:38:31 +01:00
B. Petersen 3ead349ccf add recent PRs between 1.50 and now to CHANGELOG 2021-02-15 11:07:39 +01:00
B. Petersen b505d2666b if incoming Sender:-header ist set, renaming of contacts is prevented as for List-Id:-header 2021-02-14 18:45:31 +01:00
B. Petersen 1f59b5cd15 on sending, set Sender:-header if we have an overridden name 2021-02-14 18:45:31 +01:00
B. Petersen 5c684eb3c1 add dc_msg_set_override_sender_name() api
with mailinglists, we already receive and handle per-message-names,
this api allows this also eg. for bots based on the deltachat api.
2021-02-14 18:45:31 +01:00
B. Petersen ced3a56da4 add XEP-0392 to standards.md 2021-02-13 13:37:11 +03:00
B. Petersen 982dc53dc1 add some tests for lookup_by_contact_id() 2021-02-12 18:43:34 +01:00
B. Petersen ad640e163c add dc_contact_get_auth_name() 2021-02-12 11:53:29 +01:00
B. Petersen a5c8c977db bump version to 1.51.0-alpha.0
1.51.0-alpha.0 is somewhere between 1.50 and 1.51,
as various beta-versions come or already came out with the new core,
this is helps in logs/debugging.
2021-02-10 15:47:25 +01:00
B. Petersen c41f1b42df update docs for dc_contact_get_name() 2021-02-09 16:52:26 +01:00
B. Petersen 6a4624be25 add a test to make sure, contacts cannot be added/removed to one-to-one chats 2021-02-09 12:54:09 +01:00
B. Petersen 5922069b77 rename mailinglists, remove real_group_exists()
renaming mailing lists and setting mailinglists images
(possible locally, but not synced)
was blocked because the function real_group_exists() checked
the chattype on a numerically.

it is not 100% clear to me,
why this function exist at all,
it just checks if a record with type=120 (group) exists under the given id -
this is the same as loading the chat and check chat.typ afterwards.

might be a premature optimisation relict from core-c
or made some error handling easier there.
2021-02-09 12:54:09 +01:00
B. Petersen 6aeda98c0a comment on the quoted-encded test 2021-02-08 22:06:31 +01:00
B. Petersen 5acf8e1aac support videochat-services not suppporting rooms
some videochat-services do not support adding random rooms to the url,
users let people in manually, so they can always use the same url.

as hacks as hiding the genrated random room behind `#$ROOM` or `?=foo=$ROOM`
does not always work, this pr introduces the parameter `$NOROOM` that is
just replaced by the empty string.

see https://support.delta.chat/t/videochat-with-webex/1412
for further details.
2021-02-07 19:43:36 +01:00
B. Petersen a34cfd56b4 update link to 'Go' repo 2021-02-06 02:02:08 +01:00
bjoernandlink2xt 0ef6a3060a fix From:-header bug (#2193)
* add failing test for From:-header bug

the test fails on the last check for quote-encapsulated utf-8 strings.

* Update mailparse

Co-authored-by: link2xt <link2xt@testrun.org>
2021-02-05 20:57:38 +01:00
B. Petersen 6b9b39b953 the virtualenv.pypa.io-link without .html extension shows a 404 2021-01-27 16:35:32 +03:00
B. Petersen 1ac44e5a77 fix internal link in README.rst
there must not be a space between link text and the link in angle brackets:
https://sublime-and-sphinx-guide.readthedocs.io/en/latest/references.html
2021-01-27 13:18:38 +01:00
bjoern 1e6d8063c8 add dc_msg_set_html() api (#2153)
* draft dc_msg_set_html() api

* implement setting 'html to be send'

* test sending html-parts

* more flexible html-partbuilder

* write html-parts to database and also send them

* add 'sendhtml' command to repl tool
2021-01-27 12:56:22 +01:00
B. Petersen e6dd963ebb #define removed ffi-funcion dc_contact_get_first_name as dc_contact_get_display_name to make updating easier 2021-01-26 03:02:19 +03:00
bjoernandFloris Bruynooghe e9c582c4e4 forward html-messages properly (#2151)
* add failing tests for forwarding html-mails

* let MsgId.get_html() return an option

* write html-part to local database on forwarding

* add html-part to forwarded non-dc messages

* read HTML-parts from encrypted messages

* avoid clone()

* Received:-header is no longer needed since #2152

* Update src/html.rs

Co-authored-by: Floris Bruynooghe <flub@devork.be>

* Update src/html.rs

Co-authored-by: Floris Bruynooghe <flub@devork.be>

* Update src/mimeparser.rs

Co-authored-by: Floris Bruynooghe <flub@devork.be>

* prefer 'orig' over 'org' as abbreviation for 'original'

* improve comment on tests

* prefer 'try_into()' over 'as u32' to avoid panics on bad data

* simplify ffi

Co-authored-by: Floris Bruynooghe <flub@devork.be>
2021-01-23 23:21:18 +01:00
B. Petersen 2a8c418d54 remove dc_contact_get_first_name() api
instead, dc_contact_get_display_name() should be used.

dc_contact_get_first_name() was created to save some space on the screen,
esp. on mobile devices,
however, this does not always work and has issues on its own
with some names ("Dr. Strangelove", ":) Name" and so on).

as with mailing lists, more apis with first_name() would be needed,
we decided to drop that instead of following that way.

it is also less an issue as some years ago as screens have become larger,
if really, needed, the ui can handle that more gracefully,
however, just using dc_contact_get_display_name() should be fine as well.
2021-01-23 18:50:22 +03:00
bjoern 5561aada45 update get_info() (#2156)
* add failing test checking for missing get_info() values

* add missing get_info() values

* remove unused imap_folder resp. ImapFolder config
2021-01-19 22:30:19 +01:00
B. Petersen dae20d90ed add missing import 2021-01-18 17:45:57 +01:00
bjoern f7ecf34ead document DC_STR in ffi (#2143)
* fix typo

* document DC_STR_* constants and related parts
2021-01-18 14:07:28 +01:00
B. Petersen e077ee9238 make clippy happy 2021-01-15 07:13:58 +03:00
B. Petersen 662735c233 word-encode group-name-changed if needed, use more meaningful names for the parameter 2021-01-15 07:13:58 +03:00
B. Petersen fef2a48054 add a maybe_encode_words() function
maybe_encode_words() calls encode_words()
when needs_encoding() returns true.
2021-01-15 07:13:58 +03:00
bjoern e2688f6355 add option to access original message (#2125)
* draft API to deal with uncut message texts

* add column mime_modified

* add mime_modified flag to MimeParser and save it in the database

* save mime_headers also when mime_modified is set

* cargo fmt

* set mime_modified on parsed html-texts and when there are multiple alternative-parts; add test for that

* prototype functions, add to repl and ffi

* use correct mime_modified flag

* basically parse Mime-Structure to HTML

* add basic tests for HTML-parsing

* convert text/plain to html for getting original

* respect charset for plain texts

* make test more specific

* fix handling non-utf-8 charsets for plain messages

* add test for plain_to_html()

* add failing test for plaintext linkify

* linkify urls in plain text

* fix regex

* plain text linkify: add failing test for encapsulated links as <https://domain.com>

* plain text linkify: make encapsulated links as <https://domain.com> work

* plain text linkify: require word boundary at beginning of link, add tests for that

* plain text linkify: linkify emails

* plain text: support format=flowed

* plain text: support quotes

* make clippy happy

* set mime-modified also when simplify() cuts non-html messages, add tests for that

* streamline mime recursion

* repl tool: write original html to file for further processing

* convert cid:- to data:-protocol

* add a test for cid: to data: conversion

* make clippy happy

* fix html-tests to work with windows-lineends

* clarify what the returned html-code may contain

* add some more detailed doc comments

* add mime_modified column only if not exist

this additional check is needed
as the column may added with another dbversion in
some shipped beta-versions.

* incorporate documentation suggestions from review

* rename get_original_mime_html() to more simple get_html()

* rename api is_mime_modified() to more simple has_html(); internally, mime_modified-flag stays as-is, however

* rename MimeS to MimeMultipartType

* do not set mime-modified flag for encrypted messages that need extra-handling for saved mime-structure

* fix typo

* move get_msg_html() to MsgId.get_html()

* incorporate more documentation suggestions from review

* remove unused return value from collect_texts_recursive()

* avoid mime_modified being mutable in write-parts-loop

* move 'use futures::future::FutureExt' atop of html.rs

* move attributes defining plain-text to a dedicated structure

* more PlainText to separate file

* escape cid when building regex

* let dc_get_msg_html() return NULL when calling with bad param
2021-01-11 17:40:35 +01:00
bjoern 0e1ca4323c Merge pull request #2134 from deltachat/sync-db-version
use dbversion=71 for updating ConfiguredProvider
2021-01-10 01:06:26 +01:00
B. Petersen 1b9cd18e33 use dbversion=71 for updating ConfiguredProvider
in addition to merged pr #2123,
dbversion=70 was also used by pr #2125.

pr #2125 is not yet merged, however beta-versions were shipped with
that pr, so, dbversion=70 is already in use on some systems and on these
systems ConfiguredProvider won't be set.

the other way round, that ConfiguredProvider is set twice on some systems
affects only dev-installations and should not harm.

once this pr is merged, #2125 needs a rebase and will use dbversion=72 for
its changes to add the new column also n the dev-installations :)
(we need to check that the column to add does not exist, however)
2021-01-10 00:40:59 +01:00
B. Petersen 6edff503aa add reference to RFC 3676 to standards.md
RFC 3676 is understood for incoming messages
and used for outgoing messages.
2021-01-08 19:33:52 +03:00
bjoern c243b89f7d Merge pull request #2121 from deltachat/strict-tls-default
Enable strict TLS by default
2020-12-27 00:47:54 +01:00
bjoern 8979232cfb Merge pull request #2111 from deltachat/add-subj-to-multimedia-msg
Add subject to multimedia message text
2020-12-21 13:56:30 +01:00
bjoern 09113e2579 correct and refine docs for DC_MSG_IMAGE (#2099)
* correct and refine docs for DC_MSG_IMAGE

* unify 'eg.' to 'e.g.'
2020-12-06 20:34:03 +01:00
B. Petersen f774665921 be more graceful when the apostrophed-encoding is used erroneous - either by other MUA or by Delta Chat up to core1.50 2020-12-06 04:24:39 +03:00
B. Petersen 7b291c1416 use correct key when sending filenames
adding an asterisk to the filename key
REQUIRES two apostrope-delimiters in the value as of charset'lang'name,
see rfc2231

as we do use the value this way, we MUST not add the asterisk.

things worked correctly in the past as
consuming MUAs are pretty graceful.

i assume the error comes from the c to rust conversion -
core-c did add the asterisk, however also did the apostrope-encoding.
core-rust leaves the asterisk but changes encoding.
2020-12-06 04:24:39 +03:00
B. Petersen 8fcb8c3788 add failing test for erroneous apostrophed-encoding 2020-12-06 04:24:39 +03:00
bjoern cdb7f1dd9f Merge pull request #2094 from deltachat/fix-non-utf8-filenames
test and fix non-utf8 filenames
2020-12-04 16:32:03 +01:00
B. Petersen af045c245d warn about encoding errors; this requires Context 2020-12-04 12:25:45 +01:00
B. Petersen 0bdd1b7dc2 avoid unwrap() when falling back to latin1 charset 2020-12-04 11:42:14 +01:00
B. Petersen c6f6751c89 mention used RFCs in standards.md 2020-12-04 01:45:00 +01:00
B. Petersen e77706f7d0 add test apostrophed-encoding with cp1252 charset (aka ANSI aka Windows-1252) 2020-12-04 01:28:24 +01:00
B. Petersen 8de73c4566 handle apostrophed-encoding with non-utf-8 charsets 2020-12-04 01:19:03 +01:00
B. Petersen 56e6c2712b add failing test for apostroped windows-1251 encoding 2020-12-04 00:19:19 +01:00
B. Petersen b510d74c4a add test for apostrophed-encoded filenames 2020-12-03 17:39:00 +01:00
B. Petersen 966712019f add test for bad encoded-words delimiter 2020-12-03 17:38:55 +01:00
B. Petersen f919e4962d add test for cyrillic encoded-words filenames 2020-12-03 17:01:32 +01:00
B. Petersen 412645e1ce add test for binary word-encoded filenames 2020-12-03 17:01:32 +01:00
bjoern d8babe2d0c Merge pull request #2080 from deltachat/fix-filenames
fix decoding of continued filenames
2020-12-02 11:41:52 +01:00
B. Petersen 4ef2a7c8d7 prefer exhaustive 'match' over 'if' 2020-11-29 14:47:54 +03:00
B. Petersen 75d79dc79c use different avatar-sizes for different media_quality settings 2020-11-29 14:47:54 +03:00
B. Petersen 2720d34594 use split off image recoding function for avatars 2020-11-29 14:47:54 +03:00
B. Petersen 7c15e4e948 fix rotation for small images and avatars 2020-11-29 14:47:54 +03:00
B. Petersen 4f836950bc split off image recoding to separate function 2020-11-29 14:47:54 +03:00
B. Petersen 1321a78f87 streamline test 2020-11-24 14:40:28 +01:00
B. Petersen 210d8bad04 allow specification of both, and and also do the apostrope-decoding when specified by a trailing asterix in the key. add tests for that. 2020-11-24 14:40:28 +01:00
B. Petersen e19d2cccfc prefer 'if let' over 'unwrap()' 2020-11-24 14:40:28 +01:00
B. Petersen a29dc514d3 decode filenames as CHARSET'LANG'test%2E%70%64%66 2020-11-24 14:40:28 +01:00