Commit Graph

514 Commits

Author SHA1 Message Date
link2xt
016fb2ceb2 Remove indexmap dependency
`indexmap` is a large dependency (4K SLoC) containing `unsafe` code.

Contact IDs are now passed around as a Vec<u32> or &[u32].

QUOTA roots are now sorted by name instead of perserving original order.
2021-11-22 16:40:06 +01:00
link2xt
5c571520a0 contact: use last_seen column
It was there since the C core, labeled with "/* last_seen is for
future use */" but never actually used. The comment was lost during
the translation from C to Rust.
2021-11-21 21:14:17 +03:00
link2xt
c4d1a639b0 Remove itertools dependency
Collecting into Vec of &str and joining may even be faster according
to benchmarks:
https://gist.github.com/green-s/fbd0d374b290781ac9b3f8ff03e3245d
2021-11-20 11:42:52 +01:00
bjoern
eebb2a3b68 do not assume 'no ephemeral timer' on partial downloads (#2811)
* do not assume 'no ephemeral timer' on partial downloads

* add a test to check that ephemeral timers are not disabled on partial downloads
2021-11-15 11:45:06 +01:00
link2xt
1379f8a055 Factor apply_group_changes out of create_or_lookup_group
`apply_group_changes` is executed regardless of whether the group is
created via `create_or_lookup_group` or found via
`lookup_chat_by_reply`. This change removes the need for
`lookup_chat_by_reply` to return `None` when group ID exists in the
database to let `create_or_lookup_group` run. As a side effect of this
Delta Chat replies to ad hoc groups are now correctly assigned to
chats when there are multiple groups with the same group ID, such as
ad hoc groups with the same member lists.
2021-11-07 18:34:44 +03:00
bjoern
4968f72dfb fix permanently hiding of one-to-one chats after secure-join (#2791)
* test one-to-one chats on setup-contact/secure-join

only one chat is created after scanning a QR code:

- on setup-contact, one-to-ones are created on both sided

- on secure-join, the joined group chat is created;
  one-to-ones are not created intitally,
  but should become visible on receiving messages

* make sure, Alice creates the chat with Bob on setup-contact

not totally sure if that change in #2508 was on-purpose,
however, all yet released versions
did create the one-to-one chat also on the Inviter's (Alice) side,
so, let's stay with that,
i do not see many reasons to change that.

* unblock hidden (Blocked::Yes) one-to-one chats

one-to-one chats may be hidden by secure-join,
in case someone later writes a message to it
(not unlikely), the chat needs to be shown.

before, messages are just not shown,
the corresponding chat did not appear.

the 'Blocked' wording of a 'Chat' must not be mixed with the
'Blocking' of a contact. 'Chat-Blocking' is mostly a visibility thing,
that may change as messages come in.

this change should not affect _really_ blocked contacts -
they are filtered out already before
and their messages are usually not even downloaded.
also, before allow_creation is checked,
that may disallow chat creation for show_emails reasons.

all in all, it just does the same
as if the user has manualy deleted the chat before and it would be created.

* simplify test
2021-11-06 18:46:10 +01:00
link2xt
54e79409e6 Optimize Autocrypt gossip
Update gossiped_timestamp when someone else sends autocrypt gossip in
the group, so we postpone sending gossip again ourselves.

- Warn about failures to parse Autocrypt-Gossip header
- Move gossip-related methods into ChatId impl
- Fix a "gossi_pp_ed" typo
2021-11-01 01:17:51 +03:00
link2xt
90d8c8baf5 Only apply ephemeral timers to non-special chats 2021-10-31 02:19:27 +03:00
link2xt
1dee17f980 Resultify dc_receive_imf::save_locations() 2021-10-31 02:19:27 +03:00
link2xt
6aeb21d3af dc_receive_imf: do not ignore lookup_by_contact errors 2021-10-31 02:19:27 +03:00
link2xt
4747ae2f1c Make is_dc_message and allow_creation immutable 2021-10-31 02:19:27 +03:00
link2xt
7968f55191 Trash messages instead of hiding them 2021-10-31 02:19:27 +03:00
link2xt
8ef6b6089f receive_imf: simplify message state calculation
- Make `state` variable immutable.
- Don't mark contact requests as fresh when fetching existing messages: this is not needed since there is no contact request chat anymore.
- Don't mark messages from blocked chats as noticed, this is not needed since messages go to blocked chat anyway instead of contact request chat.
2021-10-28 01:59:54 +03:00
bjoern
3b7b8ea0f1 non-blocking group QR joins (#2508)
* refactor: cleanup send_handshake_msg()

- rename to send_alice_handshake_msg() as used by Alice only

- remove dead code from Bob
  (Bob's code is at BobState::send_handshake_message() since some time)

- take a contact_id and not a chat_id;
  this makes things less confusing when
  info-messages are put to the final group chat

* always directly return chat-id from dc_join_securejoin()

* take care not to create a group twice

* adapt documentation

* add info-msg on group invites; add inviter directly after creation

* document existing 'joinqr' command in repl tool

* do not create empty one-to-one chats for group-joins

* refactor: cleanup fingerprint_equals_sender()

- the function takes a contact_id directly now.
  before it consumes the first contact of a one-to-one chat -
  which may be easily confused with the group-chat in creation.
  moreover, the conversion contact_id -> chat_id -> contact_id
  is unneeded overhead.

* show info-messages in destination chat for alice

* fingerprint_equals_sender() returns Err on database failure

* tweak documentation

* clarify what an 'unfinished tasks' task is.

* add regression test for create_for_contact_with_blocked()

* rename Blocked::Manually to better fitting Blocked::Yes

* tweak test_secure_join() and make sure, Alice and Bob have only on chat after a group-join
2021-10-26 16:34:07 +02:00
bjoern
65f09c238b add multi-device sync (#2669)
* add basic multi-device-sync functions

* generate json

* add context.parse_sync_items()

* add context.execute_sync_items()

* piggyback sync-commands message, add body for human-readable part

* avoid double json renderings

* mimeparser parses incoming .json sync-files

* do not piggyback sync-files

* execute sync items

* return status of send_sync_msg()

* send sync messages as multipart/report

* add a per-item-timestamp and also allow adding other per-item-fields in the future

* if the self-chat does not exist, create it blocked/hidden

* create tokens closer to real qr-code needs

* respect bcc_self setting, add test for that

* sync qr code tokens after promoting groups

* send sync-messages only if an experimental switch is set

* trigger send_sync_msg() after sending messages and after creating/redraw/revive qr-code

* add DC_STR_* constants to deltachat.h

* adapt to refactored qr module as of #2729

* tweak test

* use SendSyncMsgs config name instead of SendExperimentalSyncMsgs - we can remove or rename the config nevertheless, but have the option to keep it without renaming

* tweak docs

* remove currently unused effective timestamp calculation

* clarify when send_sync_msg() is called

* make sure, sync-messages are encrypted and are sent by SELF

* tweak docs, fix typos
2021-10-25 12:40:32 +02:00
link2xt
ee3b40a59a Remove double reference from lookup_chat_by_reply argument 2021-10-23 12:34:36 +00:00
link2xt
e511b87955 Remove unnecessary mut qualifier 2021-10-23 12:21:24 +00:00
link2xt
6e3c2fc839 dc_receive_imf: simplify timestamp calculation
Reduce the number of mutable variables and out parameters in
`add_parts`.

Also don't call `dc_create_smeared_timestamp` if there is no `Date`
header. Timestamps are only supposed to be "created" when the message
is sent, not received, to make sure sent messages are sorted properly
in MUAs that only use the date for sorting. Delta Chat uses database
IDs for sorting in addition to timestamps, so it can sort messages
with equal timestamps properly.

Update `dc_smeared_time` documentation.

Turn `dc_smeared_time` and `dc_create_smeared_timestamp` comments into
documentation comments.
2021-10-09 13:44:59 +03:00
bjoern
941b8caa8b move create_multiuser_record() to ChatId (#2706)
this is needed for targeting "non-blocking group QR joins"
as create_multiuser_record() would also be needed from other places.

this will make rebasing/rewriting and finally reviewing #2508 easier.
2021-09-27 20:24:25 +02:00
link2xt
3096193d58 ephemeral: always apply timers from system messages
Also extend the tests to catch regressions.
2021-09-25 02:51:23 +03:00
bjoern
43d1d9b1b3 emit MsgsChanged(chat_id, 0) on full downloads (#2696)
before, MsgsChanged(chat_id, new_msg_id) was emitted,
but that does not cover the deleted message.

in theory, we could emit both,
however, that would just be a waste of refresh in uis.

also before, events were used this way,
however, also the documentations are updated to
reflect reality better.
2021-09-20 20:50:22 +02:00
link2xt
085a899de2 Fix ephemeral timer rollback protection
Implement get_previous_message() that only looks for the last
Message-ID in the References: header instead of using
get_parent_message() which falls back to using other Message-IDs from
References and In-Reply-To field.
2021-09-20 01:54:51 +03:00
link2xt
4e8724694a Notify about incoming contact requests 2021-09-19 04:06:01 +03:00
link2xt
47bf67e658 Resultification 2021-09-18 21:56:02 +03:00
link2xt
b1af486e10 Log all decisions when applying ephemeral timer to chats
This should make it easier to debug problems related to timer
rollbacks and reported failure to disable the timer in some chats.
2021-09-18 00:48:33 +03:00
bjoern
bffb41326c better names for more mailinglist-types (#2685)
* add a test for xing mailinglists

* strip long hash-prefixes from mailinglist name if we got the name from the List-Id as a last resort

* add a test for newsletter@ mailinglists

these mailinglists have the list-name in `From:`
and can be detected by addresses starting with `newsletter@`.

* if not list-name is set, use the `From:` name for addresses starting with `newsletter@`

this is similar to what we do with `notifications@`

* Update src/dc_receive_imf.rs

Co-authored-by: Hocuri <hocuri@gmx.de>

* add an example to the regex

Co-authored-by: Hocuri <hocuri@gmx.de>
2021-09-16 18:43:37 +02:00
bjoern
0f2095947c download on demand (#2631)
* draft a download-api

* basic implementation

* allow partial downloads for protected chats

* use a separate column for download_state

* force a minimal timeout for delete_server_after in combination with partial messages

* add a warning if a possible download may expire by delete_server_after

* test load_imap_deletion_msgid()

* add a test for a partial download

* improve documentation and visibility

* let get_download_limit() return Result<Option>

* rusty getters

* apply MIN_DELETE_SERVER_AFTER to shown availability time

* move stub-creation to download.rs, use stock-strings, nicer logging

* make clippy happy (cargo clippy --tests)

* refine tests and comments

* fix typo

* remove superfluous closure in ffi

* respect partial_download for immediately scheduled DeleteMsgOnImap jobs
2021-09-13 21:12:00 +02:00
Hocuri
46956caf75 Fix: Recognize ndns that put the headers into "message/global-headers" part (Improve ndn detection) (#2598)
I sent a message, and the ndn (Non Delivery Notification) was not parsed correctly, so here comes
the fix and the test.
2021-09-12 21:02:51 +02:00
link2xt
6f3dd7f0c2 Use saturating addition for ephemeral timers
Integer overflows crash the application by default.

On a first sight this is only a potential crash that can't be
triggered, because timestamps are stored as i64 and ephemeral timer
duration is u32.
2021-09-12 19:23:54 +03:00
bjoern
da2f30786b get correct names of .xt.local mailinglists (#2665)
* add a test for .xt.local mailinglists

* get correct names of .xt.local mailinglists

these mailinglist probably come from the xt:Commerce system
and are pretty widely used.

i have not seen an .xt.local mailinglist with name set in List-Id,
however, if that happens, it will still be taken.

only if the name is unset,
we use the name from the From-header.
2021-09-12 15:34:29 +02:00
link2xt
5f065b245f Resultification 2021-09-05 06:18:38 +03:00
bjoern
3c43d790a3 update chat/contact data only when there was no newer update (#2642)
* check update timestamps for signatures, user-avatars, ephemeral-settings, last-subject

* check update timestamp for group-avatars

* check update timestamp for group-names

* check update timestamp for memberlist

* check update timestamp for protection-settings

* add a more advanced test

* add another more advanced test

* set last-subject-timestamp more carefully

* bubble up errros from set_*timestamp() and check for from_id==0 before

* simplify Params::set_i64()

* remove comment that is more confusing than helpful

* use update_timestamp() wording consistently
2021-09-04 22:16:39 +02:00
link2xt
ab1de69fbc mimeparser: rename MimeMessage.get() into MimeMessage.get_header() 2021-08-22 23:21:22 +03:00
link2xt
2b7bf11b05 Rust documentation improvements
Document all public modules and some methods.

Make some internal public symbols private.
2021-08-22 15:34:14 +02:00
Hocuri
223aeb7b1a Fix: Make emails forwarded by GMX readable (#2600)
Recognizing these emails as forwarded would probably be too complicated and require too much special-casing, but now the user can access the email text via "Show full message".

fix #2599

Co-authored-by: B. Petersen <r10s@b44t.com>
2021-08-14 18:05:17 +02:00
link2xt
20bf41b4e6 Set timestamps for system messages
Previously system messages were always added to the end of the chat,
even if the message triggering them was sent earlier.  This is
especially important for messages about disappearing timer reset
triggered by classic email messages, as they should be placed right
after the message resetting the timer.
2021-08-08 23:03:38 +03:00
link2xt
5a5b80c960 Resultify get_chat_id_by_grpid and create_or_lookup_mailinglist
Use `Option` instead of `Error` to indicate that no chat ID is found.
2021-08-08 16:26:02 +03:00
link2xt
9c2035538c dc_receive_imf: use None instead of ChatId::new(0) 2021-08-07 00:37:31 +03:00
link2xt
0f0072f5a2 dc_receive_imf: don't create chats when MDNs are received 2021-08-01 13:21:16 +03:00
link2xt
ffb17c4e61 Fix nightly clippy errors 2021-08-01 01:46:17 +03:00
link2xt
5287a3de40 dc_receive_imf: avoid cloning rfc724_mid String 2021-07-31 22:23:56 +03:00
link2xt
6f644f5c7c dc_receive_imf: extract insert_msg_id from created_db_entries
This makes add_parts() accept one argument less
2021-07-31 22:23:56 +03:00
link2xt
31b930b2fa add_parts: make mime_in_reply_to and mime_references immutable 2021-07-31 22:23:56 +03:00
link2xt
0574aeb768 dc_receive_imf: return ChatId from add_parts() 2021-07-31 22:23:56 +03:00
link2xt
bf68bc14a4 dc_receive_imf: remove dead code
mimeparser ensures that mails have at least one part.

Besides that, add_parts() handles the case of no parts just fine.
2021-07-31 22:23:56 +03:00
link2xt
065b574d93 Remove deaddrop chat
Contact request chats are not merged into a single virtual "deaddrop"
chat anymore. Instead, they are shown in the chatlist the same way as
other chats, but sending of messages to them is not allowed and MDNs
are not sent automatically until the chat is "accepted" by the user.

New API:
- dc_chat_is_contact_request(): returns true if chat is a contact
request.  In this case option to accept and block the chat via
dc_accept_chat() and dc_block_chat() should be shown in the UI.
- dc_accept_chat(): accept contact request and unblock the chat
- dc_block_chat(): decline contact request and block the chat

Removed API:
- dc_create_chat_by_msg_id(): deprecated 2021-02-07 in favor of
  dc_decide_on_contact_request()
- dc_marknoticed_contact(): deprecated 2021-02-07 in favor of
  dc_decide_on_contact_request()
- dc_decide_on_contact_request(): this call requires a message ID from
  deaddrop chat as input. As deaddrop chat is removed, this call can't
  be used anymore.
- dc_msg_get_real_chat_id(): use dc_msg_get_chat_id() instead, the
  only difference between these calls was in handling of deaddrop chat
- removed DC_CHAT_ID_DEADDROP and DC_STR_DEADDROP constants
2021-07-24 15:58:14 +03:00
Hocuri
6e7f63dba7 Fix which chats messages are assigned to (#2465)
fix https://github.com/deltachat/deltachat-core-rust/issues/2463
fix #2116

The email could be private (i.e. only sent to me) or non-private (i.e. also sent to sb else). Also, it could be a classical email or a chat message. Below, I'm describing for each of the four combinations whether they should be assigned to a chat by create_or_lookup_group() or lookup_chat_by_reply(). Because I needed to use these function names a lot, I shortened them to l:group() and l:by_reply() in this PR description.

(!) means todo, (! -> Done) means I fixed something.

## Private classical email: 
l:group() and l:by_reply() must both take care not to put it into group
l:group() no (! -> Done), l:by_reply no (! -> Done)
except for classical MUA replies to two-member-groups, they should be put into the parent group

### wrt alias-support:

A private classical email is very probably not going to be an answer to email that went to an alias address:

Suppose Alice writes to support@example.com.
support@example.com forwards to bob@example.com and claire@example.com.
When Bob answers, he will _probably_ answer `To: alice@example.com, support@example.com` (=> it's a non-private classical email).

With this PR, if he does only answer `To: alice@example.com`, (=> it's a private classical email), Alice's DC will show the answer in the private chat with Bob. Which actually makes more sense than showing it in the support@example.com chat I think. Also, if it was shown in the support@example.com chat, then Alice would answer in the support@example.com chat, then Claire would get Alice's message but not Bob's and therefore miss some context.
That being said - **I could change this**. Pretty easily actually, I would just have to remove the call to `is_probably_private_reply()` from `lookup_chat_by_reply()`. Didn't think through all edge cases yet, but should work.

## Private chat message:
l:group() has to put private chat messages into the group: It won't mistakenly put a message there (because it can rely on Chat-Group-Id). Currently, `is_probably_private_reply()` returns true for private chat messages, but l:group still has to put them into the group chat. (it's not nice that the function called is_probably_private_reply returns true for all private chat messages, but I didn't find any nicer solution) l:by_reply() must not assign it to sth special
l:group yes, l:by_reply no

By the way, for chat messages, `try_getting_grpid()` doesn't look at InReplyTo or References, in order not to put private chat-message replies into the group chat.

For alias-support, the same goes as for the private classical emails.

## Non-private classical email:
Just put it into any group, and if there is none, create one. 

_Off-topic:
We currently don't look at the recipients lists, which means that a message can easily be assigned to a group although it was not sent to all group members. One day we could somehow compare the recipients list with the members list, but that needs some more discussing (esp. what do we do if they don't match? Create a new group? Show a hint in the UI?) and is nothing for a bug-fixing PR like this one._

l_group yes, l_by_reply yes, even for outgoing messages (! -> Done, this also was the issue reported by @gerryfrancis in the testing group:

Outgoing messages were not put into a chat by In-Reply-To/References, which is correct for chat messages, but for non-private classical outgoing emails, a new ad-hoc group was created everytime.

So, I added this: https://github.com/deltachat/deltachat-core-rust/pull/2465/files#diff-e7606b521f6710ddc6e5236ba5d7eefc917b7ad744b9e71762fd42830c55485bR703-R711)

## Non-private chat message:
Can be put into chat by l:by_reply() because it must be a group message
l:group yes, l:by_reply yes (to make alias support work if the support person uses DC)

Nothing to test or fix here; we have to put chat group messages into the group, which is trivial. And we have to make sure that alias-support works, which already was well tested.
2021-07-19 16:02:11 +02:00
hendrik
a5065c21af fixed can_send() for users not in group 2021-07-14 23:10:58 +02:00
link2xt
8700cf0aba dc_receive_imf: remove cleanup() closure
Do not send any events in case of `add_parts` error.
2021-06-28 09:16:10 +03:00
link2xt
a6ad457065 dc_receive_imf: fix a typo ("reveive") 2021-06-27 20:52:35 +03:00