feat: Make broadcast lists create their own chat - UIs need to ask for
the name when creating broadcast lists now (see
https://github.com/deltachat/deltachat-android/pull/2653)
That's quite a minimal approach: Add a List-ID header to outgoing
broadcast lists, so that the receiving Delta Chat shows them as a
separate chat, as talked about with @r10s and @hpk42.
Done:
- [x] Fix an existing bug that the chat name isn't updated when the
broadcast/mailing list name changes (I already started this locally)
To be done in other PRs:
- [ ] Right now the receiving side shows "Mailing list" in the subtitle
of such a chat, it would be nicer if it showed "Broadcast list" (or
alternatively, rename "Broadcast list" to "Mailing list", too)
- [ ] The UIs should probably ask for a name before creating the
broadcast list, since it will actually be sent over the wire. (Android
PR: https://github.com/deltachat/deltachat-android/pull/2653)
Fixes https://github.com/deltachat/deltachat-core-rust/issues/4597
BREAKING CHANGE: This means that UIs need to ask for the name when creating a broadcast list, similar to https://github.com/deltachat/deltachat-android/pull/2653.
If the user makes a backup from a UI that supports the experimental verified 1:1 chats (e.g. nightly Android) and imports it into a UI that doesn't, then this config should be cleared.
We already talked about this a long time ago after @Simon-Laux noticed this problem, but I think we didn't actually solve it back then?
Best to review with whitespace changes disabled.
Directly unwrap in TestContext::get_chat()
Turns out that all usages of get_chat() directly unwrapped, because in a
test it doesn't make sense to handle the error of there being no chat.
Don't show a contact as verified if their key changed in the meantime
If a contact's key changed since the verification, then it's very
unlikely that they still have the old, verified key. So, don't show them
as verified anymore.
This also means that you can't add a contact like this to a verified
group, which is good.
The documentation actually already described this (new) behavior:
```rust
/// and if the key has not changed since this verification.
```
so, this adapts the code to the documentation.
Correctly handle messages with old timestamps for verified chats:
* They must not be sorted over a protection-changed info message
* If they change the protection, then they must not be sorted over existing other messages, because then the protection-changed info message would also be above these existing messages.
This PR fixes this:
1. Even seen messages can't be sorted into already-noticed messages anymore. **This also changes DC's behavior in the absence of verified 1:1 chats**. Before this PR, messages that are marked as seen when they are downloaded will always be sorted by their timestamp, even if it's very old.
2. protection-changed info messages are always sorted to the bottom.
**Edit:**
3. There is an exception to rule 1: Outgoing messages are still allowed to be sorted purely by their timestamp, and don't influence old messages. This is to the problem described at [*].
Together, these rules also make sure that the protection-changed info message is always right above the message causing the change.
[*] If we receive messages from two different folders, e.g. `Sent` and `Inbox`, then this will lead to wrong message ordering in many cases. I need to think about this more, or maybe someone else has an idea. One new idea that came to my mind is:
* Always sort noticed messages under the newest info message (this PR sorts them under the newest noticed message, master sorts them purely by their sent timestamp)
* Always sort unnoticed messages under the newest noticed message (that's the same behavior as in this PR and on master)
* Always sort protection-changed info messages to the bottom (as in this PR)
However, after a talk with @link2xt we instead decided to add rule 3. (see above) because it seemed a little bit easier.
If the verification is broken, `can_send()` is false.
But if the user was typing a message right when a verification-breaking message came in, the UI still needs to be able to save it as a draft.
Steps to reproduce the bug:
- Set a draft
- Your chat partner breaks verification
- Go back to the chats list
- Go to the chat again
- Accept the breakage
- Expected: The draft is still there
- Bug behavior: The draft is gone
1:1 chats are automatically created as protected if the contact is
verified, there is no need to explicitly do this.
Plus, by removing this call, the test also tests that automatically
creating 1:1 chats as protected works.
* Don't let blocking be bypassed using groups
Fix#4313
* Fix another bug: A blocked group was sometimes not unblocked when an unblocked contact sent a message into it.
* Small performance improvement by not unnecessarily loading the peerstate
* Remove wrong info message "{contact} verified" when scanning a QR code with just an email
I think that this was a bug in the original C code and then slipped
through two refactorings.
IIRC, this was written this way back when we didn't have config caching,
in order to save database accesses by getting the config outside the
for loop.
* Print chats after a test failed again
E.g.
```
========== Chats of bob: ==========
Single#Chat#10: alice@example.org [alice@example.org]
--------------------------------------------------------------------------------
Msg#10: (Contact#Contact#10): hellooo [FRESH]
Msg#11: (Contact#Contact#10): hellooo without mailing list [FRESH]
--------------------------------------------------------------------------------
========== Chats of alice: ==========
Single#Chat#10: bob@example.net [bob@example.net]
--------------------------------------------------------------------------------
Msg#10: Me (Contact#Contact#Self): hellooo √
Msg#11: Me (Contact#Contact#Self): hellooo without mailing list √
--------------------------------------------------------------------------------
```
I found this very useful sometimes, so, let's try to re-introduce it (it
was removed in #3449)
* Add failing test for TestContext::drop()
* Do not panic in TestContext::drop() if runtime is dropped
Co-authored-by: link2xt <link2xt@testrun.org>
* Because both only make problems with mailing lists, it's easiest to just disable them. If we want, we can make them work properly with mailing lists one day and re-enable them, but this needs some further thoughts.
Part of #3701
* Use load_from_db() in more tests
* clippy
* Changelog
* Downgrade warning to info, improve message
* Use lifetimes instead of cloning
This esp. speeds up receive_imf a bit when we recreate the member list (recreate_member_list == true).
It's a preparation for https://github.com/deltachat/deltachat-core-rust/issues/3768, which will be a one-line-fix, but recreate the member list more often, so that we want to optimize this case a bit.
It also adds a benchmark for this case. It's not that easy to make the benchmark non-flaky, but by closing all other programs and locking the CPU to 1.5GHz it worked. It is consistently a few percent faster than ./without-optim:
```
Receive messages/Receive 100 Chat-Group-Member-{Added|Removed} messages
time: [52.257 ms 52.569 ms 52.941 ms]
change: [-3.5301% -2.6181% -1.6697%] (p = 0.00 < 0.05)
Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
4 (4.00%) high mild
3 (3.00%) high severe
```
That's a bug which @Simon-Laux and probably also @hpk42 had, where one malformed incoming (Spam-) mail blocked the receiving of all emails coming after it.
The problem was that from_field_to_contact_id() returned ContactId::UNDEFINED, and then lookup_by_contact() returned Err.
* Treat multiple From addresses as if there was no From: addr
* changelog
* Don't send invalid emails through the whole receive_imf pipeline
Instead, directly create a trash entry for them.
* Don't create trash entries for randomly generated Message-Id's
* clippy
* fix typo
Co-authored-by: link2xt <link2xt@testrun.org>
* Add DC_EVENT_INCOMING_MSG event
* Fix lots of compile errors
* Docs
* Changelog
* Fix python tests
Adding DC_EVENT_INCOMING_MSG_BUNCH made the python tests fail because they use `get_matching("DC_EVENT_INCOMING_MSG")`, which also matches DC_EVENT_INCOMING_MSG_BUNCH, so the tests got confused.
This fixes `get_matching()` to only match whole event names.
* Also fix test_ac_setup_message_twice()
The built regex was ^EVENT_NAME1|EVENT_NAME2$, which becomes parsed as
"^EVENT_NAME1" OR "EVENT_NAME2$". Introduce a group (parentheses) to fix
this.
* desktop will use DC_EVENT_INCOMING_MSG_BUNCH,
so I would not call it experimental anymore
* add generated node constants
* msg_ids in the event as Vec<u32>
number[] in js land
this is way more convinient than a json encoded string.
* Apply suggestions from code review
Co-authored-by: bjoern <r10s@b44t.com>
Co-authored-by: Simon Laux <mobile.info@simonlaux.de>
Co-authored-by: Simon Laux <Simon-Laux@users.noreply.github.com>
Co-authored-by: bjoern <r10s@b44t.com>
Seems like consume_events() didn't work properly, i.e. in some cases it
didn't see the latest events and failed to consume them. So, the
IncomingMsg event from receiving DC_MAILINGLIST stayed in the events
channel, which made this fail:
```rust
// Check that no notification is displayed for blocked mailing list message.
while let Ok(event) = t.evtracker.try_recv() {
assert!(!matches!(event.typ, EventType::IncomingMsg { .. }));
}
```
Fix it by explicitly waiting for the first IncomingMsg event.
Fix#3507
Note that this is not intended for a release at this point! We first have to test whether it runs stable enough. If we want to make a release while we are not confident enough in authres-checking, then we have to disable it.
BTW, most of the 3000 new lines are in `test_data/messages/dkimchecks...`, not the actual code
da3a4b94 adds the results to the Message info. It currently does this by adding them to `hop_info`. Maybe we should rename `hop_info` to `extra_info` or something; this has the disadvantage that we can't rename the sql column name though.
Follow-ups for this could be:
- In `update_authservid_candidates()`: Implement the rest of the algorithm @hpk42 and me thought about. What's missing is remembering how sure we are that these are the right authserv-ids. Esp., when receiving a message sent from another account at the same domain, we can be quite sure that the authserv-ids in there are the ones of our email server. This will make authres-checking work with buzon.uy, disroot.org, yandex.ru, mailo.com, and riseup.net.
- Think about how we present this to the user - e.g. currently the only change is that we don't accept key changes, which will mean that the small lock on the message is not shown.
- And it will mean that we can fully enable AEAP, after revisiting the security implications of this, and assuming everyone (esp. @link2xt who pointed out the problems in the first place) feels comfortable with it.
Very small PR; Motivation: Easier navigation using Go-To-definition.
Because, using go-to-definition of rust-analyzer on parse() doesn't take you to the actual parse() implementation but its trait definiton. On the other hand, it's very easy to find EmailAddress::new().
#3491 introduced a bug that your address is only replaced in the first group you write to, which was rather hard to fix. In order to be able to release something, we agreed to revert it and instead only replace the contacts in verified groups (and in broadcast lists, if the signing key is verified).
Highlights:
* Revert "Only do the AEAP transition in the chat where it happened"
This reverts commit 22f4cd7b79.
* Only do the transition for verified groups (and broadcast lists)
To be exact, only do the transition if the signing key fingerpring is
verified. And only do it in verified groups and broadcast lists
* Slightly adapt string to this change
* Changelog
It happened multiple times now that I wanted to quickly execute a test, but because of a warning that had become an error, it didn't execute.
This turns warnings into warnings again; our CI will fail if there is a warning, anyway (because of RUSTFLAGS: -Dwarnings)
Not completely sure it's worth it since some other dependencies still
depend on it. Anyway, proc macros are said to be bad for compile times, I just typed out what the proc macro generates and it's only 8 more lines, and we're already doing it this way in e.g. action_by_contact() and collect_texts_recursive() (the latter needs the boxed future both for the trait and for recursion).
I added this poison_sender and poison_receiver stuff back when we had event listeners (which were called "sinks", too, but anyway), i.e. user-definable closures that were run in the events loop. This was to make sure that the test fails if the closure panics. But since we don't have them anymore and this code isn't supposed to panic anyway:
```rust
while let Some(event) = events.recv().await {
for sender in senders.read().await.iter() {
sender.try_send(event.clone()).ok();
}
}
```
* Rename aeap-mvp.rst to aeap-mvp.md
* Update aeap-mvp.md
* Apply suggestions from hpk's review
Co-authored-by: holger krekel <holger@merlinux.eu>
* Additions to holger's review
* Decide on TODOs
* Drop the "If we are going to assign a message to a chat, but the sender is not a member of this chat" condition again
Co-authored-by: holger krekel <holger@merlinux.eu>
Hocuriandiequidoo
