Commit Graph

236 Commits

Author SHA1 Message Date
B. Petersen
de20e4c9dd basic w30 sending and receiving 2022-01-13 11:11:50 +01:00
link2xt
30cb0cbcfd Reduce number of AsRef generics
They result in compilation of duplicate code.
2021-12-31 13:57:45 +00:00
Hocuri
4136217249 Start making it possible to write to mailing lists (#2736)
See #748, #1964 and 3ba4c6718e/draft/mailing_list_managers.md

Also fix #2735: Assign outgoing messages from other devices to the mailing list
2021-12-31 14:01:30 +01:00
link2xt
12823c2213 Add imap table to keep track of message UIDs
`imap` table maps Message-IDs to UIDs on the server. `dc_receive_imf`
no longer gets the UID of the message as an argument and does not
insert the folder and UID of the message into the `msgs`
table. `server_folder` and `server_uid` columns in `msgs` table are
deprecated.

MoveMsg and DeleteMsgOnImap jobs are removed. Now messages are moved
and deleted only in the `fetch_move_delete` procedure that consults
the `target` column of the `imap` table to determine where the message
should go.

Where the message should go is determined after prefetching by the
`imap::target_folder()` procedure.  Messages are only downloaded once
they reach their target folder to avoid race conditions in multidevice
setting, such as:

1. One device trying to FETCH the message while the other tries to
MOVE it.

2. One device marking the message as \Seen in the Inbox while the
other has already copied unseen message to the Movebox and is going to
delete the \Seen message in the Inbox.

3. Device downloads the message from the Inbox while there are newer
messages in the Movebox placed there by the other device, thus
processing the messages out of order.
2021-12-19 00:00:00 +00:00
Hocuri
d47680733b Fix: Also show the hop_info for encrypted messages (#2923)
Before, the hop_info was shown only for unencrypted messages.

Credits: The bug was noticed by link2xt

Follow-up for #2751
2021-12-28 13:23:41 +01:00
Sebastian Klähn
2a11f8f59d add hop_info (#2751) 2021-12-22 16:08:12 +01:00
link2xt
dae80cbe35 Recognize MS Exchange read receipts as read receipts
They contain X-MSExch-Correlation-Key header, but no
Original-Message-ID, so they cannot be used to find the original
message, but we want to recognize them as MDN nevertheless to assign
them to the trash chat.
2021-12-11 21:44:54 +00:00
link2xt
01db8d0130 test_utils: fix Alice's address in alice_keypair()
The key stored in test-data has @example.org identity.
2021-12-05 09:14:58 +00:00
link2xt
54e79409e6 Optimize Autocrypt gossip
Update gossiped_timestamp when someone else sends autocrypt gossip in
the group, so we postpone sending gossip again ourselves.

- Warn about failures to parse Autocrypt-Gossip header
- Move gossip-related methods into ChatId impl
- Fix a "gossi_pp_ed" typo
2021-11-01 01:17:51 +03:00
bjoern
65f09c238b add multi-device sync (#2669)
* add basic multi-device-sync functions

* generate json

* add context.parse_sync_items()

* add context.execute_sync_items()

* piggyback sync-commands message, add body for human-readable part

* avoid double json renderings

* mimeparser parses incoming .json sync-files

* do not piggyback sync-files

* execute sync items

* return status of send_sync_msg()

* send sync messages as multipart/report

* add a per-item-timestamp and also allow adding other per-item-fields in the future

* if the self-chat does not exist, create it blocked/hidden

* create tokens closer to real qr-code needs

* respect bcc_self setting, add test for that

* sync qr code tokens after promoting groups

* send sync-messages only if an experimental switch is set

* trigger send_sync_msg() after sending messages and after creating/redraw/revive qr-code

* add DC_STR_* constants to deltachat.h

* adapt to refactored qr module as of #2729

* tweak test

* use SendSyncMsgs config name instead of SendExperimentalSyncMsgs - we can remove or rename the config nevertheless, but have the option to keep it without renaming

* tweak docs

* remove currently unused effective timestamp calculation

* clarify when send_sync_msg() is called

* make sure, sync-messages are encrypted and are sent by SELF

* tweak docs, fix typos
2021-10-25 12:40:32 +02:00
bjoern
0f2095947c download on demand (#2631)
* draft a download-api

* basic implementation

* allow partial downloads for protected chats

* use a separate column for download_state

* force a minimal timeout for delete_server_after in combination with partial messages

* add a warning if a possible download may expire by delete_server_after

* test load_imap_deletion_msgid()

* add a test for a partial download

* improve documentation and visibility

* let get_download_limit() return Result<Option>

* rusty getters

* apply MIN_DELETE_SERVER_AFTER to shown availability time

* move stub-creation to download.rs, use stock-strings, nicer logging

* make clippy happy (cargo clippy --tests)

* refine tests and comments

* fix typo

* remove superfluous closure in ffi

* respect partial_download for immediately scheduled DeleteMsgOnImap jobs
2021-09-13 21:12:00 +02:00
Hocuri
46956caf75 Fix: Recognize ndns that put the headers into "message/global-headers" part (Improve ndn detection) (#2598)
I sent a message, and the ndn (Non Delivery Notification) was not parsed correctly, so here comes
the fix and the test.
2021-09-12 21:02:51 +02:00
link2xt
7745db8310 Ignore MDNs sent to self
These sometimes arrive happen due to a bug in previous versions of
Delta Chat.
2021-09-12 02:18:50 +03:00
link2xt
ab1de69fbc mimeparser: rename MimeMessage.get() into MimeMessage.get_header() 2021-08-22 23:21:22 +03:00
link2xt
2b7bf11b05 Rust documentation improvements
Document all public modules and some methods.

Make some internal public symbols private.
2021-08-22 15:34:14 +02:00
link2xt
3440daca1a Reduce message length limit to 5000 chars (#2615)
- Use the same limit for info: full text can be read in HTML anyway.
- Remove DC_MAX_GET_{TEXT,INFO}_LEN constants from deltachat.h
- Fix a typo: s/DC_ELLIPSE/DC_ELLIPSIS/
- Do not truncate the text when loading from the database.
- Update the documentation: limit is in Rust chars, not bytes
2021-08-21 21:02:14 +02:00
link2xt
ade39fe026 fix: do not set WantsMdn param for outgoing messages
This bug sometimes results in sending read receipts to self in
multi-device setups.

It happens consistently in a setup where the first device is
configured to move messages to DeltaChat folder and the second device
is not. When both devices receive BCC-self message simultaneously, the
first device moves the message to DeltaChat folder, while the second
device tries to mark the message as seen in the Inbox. Regardless of
whether the second device marks the message as seen successfully or
fails because the message is already moved by the first device,
`Job.markseen_msg_on_imap()` sends the read receipt to the From
address.
2021-08-15 20:39:28 +03:00
link2xt
20bf41b4e6 Set timestamps for system messages
Previously system messages were always added to the end of the chat,
even if the message triggering them was sent earlier.  This is
especially important for messages about disappearing timer reset
triggered by classic email messages, as they should be placed right
after the message resetting the timer.
2021-08-08 23:03:38 +03:00
link2xt
065b574d93 Remove deaddrop chat
Contact request chats are not merged into a single virtual "deaddrop"
chat anymore. Instead, they are shown in the chatlist the same way as
other chats, but sending of messages to them is not allowed and MDNs
are not sent automatically until the chat is "accepted" by the user.

New API:
- dc_chat_is_contact_request(): returns true if chat is a contact
request.  In this case option to accept and block the chat via
dc_accept_chat() and dc_block_chat() should be shown in the UI.
- dc_accept_chat(): accept contact request and unblock the chat
- dc_block_chat(): decline contact request and block the chat

Removed API:
- dc_create_chat_by_msg_id(): deprecated 2021-02-07 in favor of
  dc_decide_on_contact_request()
- dc_marknoticed_contact(): deprecated 2021-02-07 in favor of
  dc_decide_on_contact_request()
- dc_decide_on_contact_request(): this call requires a message ID from
  deaddrop chat as input. As deaddrop chat is removed, this call can't
  be used anymore.
- dc_msg_get_real_chat_id(): use dc_msg_get_chat_id() instead, the
  only difference between these calls was in handling of deaddrop chat
- removed DC_CHAT_ID_DEADDROP and DC_STR_DEADDROP constants
2021-07-24 15:58:14 +03:00
link2xt
aec8332544 mimeparser: use mailparse to parse RFC 2231 filenames
mailparse supports RFC 2231 since version 0.13.5, so there is no need
for our own code to support it.
2021-07-17 14:31:16 +03:00
link2xt
8ce9a78d6c chatlist: resultify get_msg_id, get_summary and get_summary2
Avoid using MsgId::new(0) in place of `None` in the Rust part.
Zero ID is only used in FFI part now.
2021-06-26 17:04:55 +03:00
link2xt
f820671d53 Use Auto-Submitted: auto-generated header to identify bots
New `dc_msg_is_bot()` C API and corresponding `Message.is_bot()`
Python API can be used to check if incoming message is sent by a bot,
e.g. to avoid two echo bots replying indefinitely to each other.

"Bot" flag is not set for outgoing messages, but may be set for
BCC-self messages. For now documentation says that `dc_msg_is_bot()`
return value is unspecified for outgoing messages. It can be better
specified later if needed for specific applications, e.g. sharing an
account with a helper bot.
2021-06-19 17:36:20 +03:00
B. Petersen
475fa24876 move links to angle brackets to avoid rustdoc errors 2021-06-19 01:39:04 +03:00
B. Petersen
4fe3a80f96 allow stickers with gif-images 2021-06-11 13:08:48 +02:00
link2xt
c5539de4da More robust In-Reply-To parsing
Use `parse_message_id` on the header contents before using it.

Database still stores the raw value.
2021-06-07 21:38:00 +02:00
Hocuri
e3e2adeea5 Fix all outgoing messages popping up in selfchat (#2456)
Fix https://github.com/deltachat/deltachat-android/issues/1940, fix https://github.com/deltachat/deltachat-core-rust/issues/2220 (I assume these are the same bug)

The problem was:
- Gmail adds a header `Bcc: <self-address>` to our bcc-self message
- `to_id` was just set to the first recipient, in this case _self_
- it was seen that `to_id` is _self_, so it's a self-sent message
2021-05-28 20:02:46 +02:00
link2xt
adac903818 Debloat the binary by using less AsRef arguments
Using `impl AsRef<str>` as the argument instead of `&str` makes it
possible to call the function with `&str`, `String` and other types
that implement `AsRef` trait.

The cost of it is that compiled binary contains mulitple versions of
the same function, one for each variant of types. If function contains
multiple generic `impl AsRef` arguments, the number of versions possibly
compiled into binary grows exponentially with the number of arguments.

Simple way to avoid it is to call `.as_ref()` on the caller side to
convert the argument to `&str`. In most cases even adding a `&` and
relying on `Deref` coercion is sufficient.

This patch changes many functions that accepted `impl AsRef<str>` and
`impl AsRef<Path>` to accept `&str` and `&Path` instead.

In some places `.clone()` calls are removed. Calling `.clone()` on
`String` and passing `String` to a function accepting `impl
AsRef<str>` is completely unnecessary as `&str` reference could be
passed instead. There is no clippy warning against it yet, but
changing argument type to `&str` allowed to find these cases.

The result of debloating is not impressive, several hundred kilobytes
are saved, which is about 3% of the `.so` binary, but the code is
cleaner too.
2021-05-09 16:25:11 +03:00
link2xt
30c463e0ba Add extension to avatars base64-encoded in headers 2021-05-03 01:32:37 +02:00
link2xt
bba437523a Process Chat-User-Avatar headers with embedded base64 images 2021-05-01 07:40:56 +03:00
link2xt
f76bc44cdc mimeparser: parse hidden headers 2021-05-01 07:40:56 +03:00
link2xt
8610b0c945 sql: switch from sqlx to rusqlite 2021-04-25 22:33:14 +03:00
Hocuri
179a2a50e6 Parse <blockquote> tags for better quote detection (#2313) 2021-04-07 18:45:00 +02:00
Friedel Ziegelmayer
6bb5721f29 feat: improve internal sql interface
Switches from rusqlite to sqlx to have a fully async based interface
to sqlite.

Co-authored-by: B. Petersen <r10s@b44t.com>
Co-authored-by: Hocuri <hocuri@gmx.de>
Co-authored-by: link2xt <link2xt@testrun.org>
2021-04-06 16:06:11 +02:00
B. Petersen
6963fd877d allow one additional boolean
the function is not complex and is only called once.
refactoring that seems to be a bit too much effort
and at least out of scope of this pr.
2021-03-12 11:30:25 +01:00
B. Petersen
045fbab7cd remove subsequent images inside multipart/related 2021-03-12 11:30:25 +01:00
B. Petersen
f69bcc71ed mark child-parts of multipart/related as such 2021-03-12 11:30:25 +01:00
B. Petersen
4ae86b4e61 truncate long texts and make the whole text accessible via has_html()/get_html() 2021-03-04 23:26:02 +01:00
B. Petersen
e418d89c79 add failing test to test truncating 2021-03-04 23:26:02 +01:00
link2xt
a698a8dd84 Update to Rust 1.50
Also run rustfmt, fix new clippy warnings.
2021-03-01 16:48:33 +03:00
B. Petersen
5c2d6c22a0 remove additional text parts if we think they belong to a mailinglist footer 2021-02-28 21:03:39 +01:00
link2xt
f8755b505e Fix clippy::unnecessary_wraps warnings
This lint is enabled by default in 1.50 toolchain.
2021-02-25 12:35:45 +03:00
B. Petersen
0fa90a81e5 prefer X-Microsoft-Original-Message-ID, if set
outlooks SMTP-server change the Message-ID of messages
and put the original Message-ID to X-Microsoft-Original-Message-ID.

the changed Message-ID has some issues:

- outgoing messages with bcc_self enabled are shown twice
  as the self-copy got the changed Message-ID while the database uses
  the original one

- read receipts do not work as they refer to the changed message id

- in general, sender and recipient see different Message-IDs

the issues can be fixed by

(1) let all receivers use the original Message-ID

	this is what this pr is doing
    and this should fix all issues with delta-to-delta communication,
	including groups, group-images etc.
    there may be issues left in communication
	with other MUAs as they are using another Message-ID.

(2) ftr: updating the Message-ID in the database of the sender to the new one

	this requires bcc_self always enabled (which is not the case)
    and may also result easily in race conditions
    (Bob answers before Alice sees its self-sent message),
    however, has the advantage of better compatibility with other MUA.

if needed, the compatibility with other MUA could be improved by remembering
both Messages-IDs, maybe we could treat the modified as References or so,
however, i think, this could be part of another PR if we know better about
real, in the wild issues.
2021-02-19 00:25:46 +03:00
B. Petersen
e7c6667347 introduce MailinglistType for more exhaustive checking 2021-02-15 21:38:31 +01:00
B. Petersen
0b3fb9c0a3 prepend subject whenever we think sth. is a mailinglist, not only for a subset of mailing lists 2021-02-15 21:38:31 +01:00
link2xt
57841cdcc0 Remove &Context from Peerstate and MimeParser
Passing Context around explicitly removes the need for explicit lifetimes.
2021-02-14 14:19:55 +03:00
link2xt
a88893f262 clippy: fix needless_borrow 2021-02-14 00:55:23 +03:00
Floris Bruynooghe
46a3226e43 Bunch of feedback: renames, simple functions, no cow 2021-02-12 18:28:51 +01:00
Floris Bruynooghe
29f184b4c4 Strongly type stock strings
This changes the internal stock strings API to be more strongly typed,
ensuring that the caller can not construct the stock string in the
wrong way.

The old approach left it to the callers to figure out how a stock
string should be created, now each stock string has their specific
arguments and callers can not make mistakes.  In particular all the
subtleties and different ways of calling stock_system_msg() disappear.

This could not use a trait for stock strings, as this would not allow
having per-message typed arguments.  So we needed a type per message
with a custom method, only by convention this method is .stock_str().
The type is a enum without variants to avoid allowing someone to
create the type.

Sadly the fallback string and substitutions are still far away from
each other, but it is now only one place which needs to know how to
construct the string instead of many.
2021-02-12 18:28:51 +01:00
link2xt
0601b05cb7 Use footer as a contact status 2021-02-11 13:57:49 +03:00
B. Petersen
6aeda98c0a comment on the quoted-encded test 2021-02-08 22:06:31 +01:00