* Print chats after a test failed again
E.g.
```
========== Chats of bob: ==========
Single#Chat#10: alice@example.org [alice@example.org]
--------------------------------------------------------------------------------
Msg#10: (Contact#Contact#10): hellooo [FRESH]
Msg#11: (Contact#Contact#10): hellooo without mailing list [FRESH]
--------------------------------------------------------------------------------
========== Chats of alice: ==========
Single#Chat#10: bob@example.net [bob@example.net]
--------------------------------------------------------------------------------
Msg#10: Me (Contact#Contact#Self): hellooo √
Msg#11: Me (Contact#Contact#Self): hellooo without mailing list √
--------------------------------------------------------------------------------
```
I found this very useful sometimes, so, let's try to re-introduce it (it
was removed in #3449)
* Add failing test for TestContext::drop()
* Do not panic in TestContext::drop() if runtime is dropped
Co-authored-by: link2xt <link2xt@testrun.org>
* Because both only make problems with mailing lists, it's easiest to just disable them. If we want, we can make them work properly with mailing lists one day and re-enable them, but this needs some further thoughts.
Part of #3701
* Use load_from_db() in more tests
* clippy
* Changelog
* Downgrade warning to info, improve message
* Use lifetimes instead of cloning
This esp. speeds up receive_imf a bit when we recreate the member list (recreate_member_list == true).
It's a preparation for https://github.com/deltachat/deltachat-core-rust/issues/3768, which will be a one-line-fix, but recreate the member list more often, so that we want to optimize this case a bit.
It also adds a benchmark for this case. It's not that easy to make the benchmark non-flaky, but by closing all other programs and locking the CPU to 1.5GHz it worked. It is consistently a few percent faster than ./without-optim:
```
Receive messages/Receive 100 Chat-Group-Member-{Added|Removed} messages
time: [52.257 ms 52.569 ms 52.941 ms]
change: [-3.5301% -2.6181% -1.6697%] (p = 0.00 < 0.05)
Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
4 (4.00%) high mild
3 (3.00%) high severe
```
That's a bug which @Simon-Laux and probably also @hpk42 had, where one malformed incoming (Spam-) mail blocked the receiving of all emails coming after it.
The problem was that from_field_to_contact_id() returned ContactId::UNDEFINED, and then lookup_by_contact() returned Err.
* Treat multiple From addresses as if there was no From: addr
* changelog
* Don't send invalid emails through the whole receive_imf pipeline
Instead, directly create a trash entry for them.
* Don't create trash entries for randomly generated Message-Id's
* clippy
* fix typo
Co-authored-by: link2xt <link2xt@testrun.org>
* Add DC_EVENT_INCOMING_MSG event
* Fix lots of compile errors
* Docs
* Changelog
* Fix python tests
Adding DC_EVENT_INCOMING_MSG_BUNCH made the python tests fail because they use `get_matching("DC_EVENT_INCOMING_MSG")`, which also matches DC_EVENT_INCOMING_MSG_BUNCH, so the tests got confused.
This fixes `get_matching()` to only match whole event names.
* Also fix test_ac_setup_message_twice()
The built regex was ^EVENT_NAME1|EVENT_NAME2$, which becomes parsed as
"^EVENT_NAME1" OR "EVENT_NAME2$". Introduce a group (parentheses) to fix
this.
* desktop will use DC_EVENT_INCOMING_MSG_BUNCH,
so I would not call it experimental anymore
* add generated node constants
* msg_ids in the event as Vec<u32>
number[] in js land
this is way more convinient than a json encoded string.
* Apply suggestions from code review
Co-authored-by: bjoern <r10s@b44t.com>
Co-authored-by: Simon Laux <mobile.info@simonlaux.de>
Co-authored-by: Simon Laux <Simon-Laux@users.noreply.github.com>
Co-authored-by: bjoern <r10s@b44t.com>
Seems like consume_events() didn't work properly, i.e. in some cases it
didn't see the latest events and failed to consume them. So, the
IncomingMsg event from receiving DC_MAILINGLIST stayed in the events
channel, which made this fail:
```rust
// Check that no notification is displayed for blocked mailing list message.
while let Ok(event) = t.evtracker.try_recv() {
assert!(!matches!(event.typ, EventType::IncomingMsg { .. }));
}
```
Fix it by explicitly waiting for the first IncomingMsg event.
Fix#3507
Note that this is not intended for a release at this point! We first have to test whether it runs stable enough. If we want to make a release while we are not confident enough in authres-checking, then we have to disable it.
BTW, most of the 3000 new lines are in `test_data/messages/dkimchecks...`, not the actual code
da3a4b94 adds the results to the Message info. It currently does this by adding them to `hop_info`. Maybe we should rename `hop_info` to `extra_info` or something; this has the disadvantage that we can't rename the sql column name though.
Follow-ups for this could be:
- In `update_authservid_candidates()`: Implement the rest of the algorithm @hpk42 and me thought about. What's missing is remembering how sure we are that these are the right authserv-ids. Esp., when receiving a message sent from another account at the same domain, we can be quite sure that the authserv-ids in there are the ones of our email server. This will make authres-checking work with buzon.uy, disroot.org, yandex.ru, mailo.com, and riseup.net.
- Think about how we present this to the user - e.g. currently the only change is that we don't accept key changes, which will mean that the small lock on the message is not shown.
- And it will mean that we can fully enable AEAP, after revisiting the security implications of this, and assuming everyone (esp. @link2xt who pointed out the problems in the first place) feels comfortable with it.
Very small PR; Motivation: Easier navigation using Go-To-definition.
Because, using go-to-definition of rust-analyzer on parse() doesn't take you to the actual parse() implementation but its trait definiton. On the other hand, it's very easy to find EmailAddress::new().
#3491 introduced a bug that your address is only replaced in the first group you write to, which was rather hard to fix. In order to be able to release something, we agreed to revert it and instead only replace the contacts in verified groups (and in broadcast lists, if the signing key is verified).
Highlights:
* Revert "Only do the AEAP transition in the chat where it happened"
This reverts commit 22f4cd7b79.
* Only do the transition for verified groups (and broadcast lists)
To be exact, only do the transition if the signing key fingerpring is
verified. And only do it in verified groups and broadcast lists
* Slightly adapt string to this change
* Changelog
It happened multiple times now that I wanted to quickly execute a test, but because of a warning that had become an error, it didn't execute.
This turns warnings into warnings again; our CI will fail if there is a warning, anyway (because of RUSTFLAGS: -Dwarnings)
Not completely sure it's worth it since some other dependencies still
depend on it. Anyway, proc macros are said to be bad for compile times, I just typed out what the proc macro generates and it's only 8 more lines, and we're already doing it this way in e.g. action_by_contact() and collect_texts_recursive() (the latter needs the boxed future both for the trait and for recursion).
I added this poison_sender and poison_receiver stuff back when we had event listeners (which were called "sinks", too, but anyway), i.e. user-definable closures that were run in the events loop. This was to make sure that the test fails if the closure panics. But since we don't have them anymore and this code isn't supposed to panic anyway:
```rust
while let Some(event) = events.recv().await {
for sender in senders.read().await.iter() {
sender.try_send(event.clone()).ok();
}
}
```
* Rename aeap-mvp.rst to aeap-mvp.md
* Update aeap-mvp.md
* Apply suggestions from hpk's review
Co-authored-by: holger krekel <holger@merlinux.eu>
* Additions to holger's review
* Decide on TODOs
* Drop the "If we are going to assign a message to a chat, but the sender is not a member of this chat" condition again
Co-authored-by: holger krekel <holger@merlinux.eu>
* Do ephemeral deletion in background loop
1. in start_io start ephemeral async task, in stop_io cancel ephemeral async task
2. start ephemeral async task which loops like this:
- wait until next time a message deletion is needed or an interrupt occurs (see 3.)
- perform delete_expired_messages including sending MSGS_CHANGED events
3. on new messages (incoming or outgoing) with ephemeral timer:
- interrupt ephemeral async task
* Changelog
* Fix and improve test
* no return value needed
* address @link2xt review comments
* slight normalization: have only one place where we wait for interrupt_receiver
* simplify sql statement -- and don't exit the ephemeral_task if there is an sql problem but rather wait
* Remove now-unused `ephemeral_task` JoinHandle
The JoinHandle is now inside the Scheduler.
* fix clippy
* Revert accidental move of the line
* Add log
Co-authored-by: holger krekel <holger@merlinux.eu>
Co-authored-by: link2xt <link2xt@testrun.org>
* Configure: Try "imap.*"/"smtp.*"/"mail.*" first
Fix half of #3158.
Try "imap.ex.org"/"smtp.ex.org" and "mail.ex.org" first because if a server exists
under this address, it's likely the correct one.
Try "ex.org" last because if it's wrong and the server is configured to
not answer at all, configuration may be stuck for several minutes.
* Changelog
* Add test
Today, we solved an issue with holger's phone that he couldn't export
his account anymore because during the `VACUUM` the Android system
killed the app because of OOM. The solution was to drop the table
`backup_blobs`, so let's automatically do this in migration
This table was used back in the olden days when we backuped by exporting
the dbfile and then putting all blobs into it. During import, the
`backup_blobs` table should have been dropped, seems like this didn't
work here.
Holger had a case where he wrote with someone using a classing MUA.
He opened a two-member-group with this person (which also allowed him to
set the subject).
At some point the other person replied from a different email address.
What he expected: This reply should be sorted into the two-member-group.
What happened: This reply was sorted into the 1:1 chat.
---
I had added the line && chat_contacts.contains(&from_id) months ago when I wrote
this code because it seemed vaguely sensible but without any real
reason. So, let's remove it and see if it creates other problems -
my gut feeling is no.
* Add AcManager
See https://github.com/deltachat/deltachat-core-rust/pull/2901#issuecomment-998285039
This reduces boilerplate code again therefore, improving the
signal-noise-ratio and reducing the mental barrier to start
writing a unit test.
Slightly off-topic:
I didn't add any advanced functions like `manager.get("alice");` because
they're not needed yet; however, once we have the AcManager we can
think about fancy things like:
```rust
acm.send_text(&alice, "Hi Bob, this is Alice!", &bob);
```
which automatically lets bob receive the message.
However, this may be less useful than it seems at first, since most of
the tests I looked at wouldn't benefit from it, so at least I won't do
it until I have a test that would benefit from it.
* Remove unnecessary RefCell
* Rename AcManager to TestContextManager
* Don't store TestContext's in a vec for now as we don't need this; we can re-add it later
* Rename acm -> tcm
fix#3007
My approach is:
We don't download any messages from the spam folder anymore, and only download them if they were moved out. This means that is-it-spam logic only resides in spam_target_folder(). This has some implications, see the comments.
As of #2980, the migrations were not run after importing a backup. This lead to errors like "Failed to send message: no such table: smtp" if you imported a backup from a previous DC version.
While experimenting with encrypted storage, once configuring failed between 920 and 940. But as the "configured" config had already been written after progress 910, some part of the code thought we are configured, some didn't.
With this PR, my encrypted-storage Android PR now works, at least I
couldn't find any further bugs.
Without it, configuring fails with: `Failed to create blob
icon-saved-messages-1803424689.png in
/data/user/0/com.b44t.messenger.beta/files/accounts/0e402b37dcd14a9586aea46294c908f2/dc.db-blobs:
No such file or directory (os error 2)`.
Also see https://github.com/deltachat/deltachat-core-rust/pull/2972.
We had an unhelpful log in the Testing group:
- it repeatedly says "src/imap.rs:1010: dc_receive_imf error: add_parts error" but the actual error is not shown
- it says "deltachat-ffi/src/lib.rs:186: dc_get_config(): invalid key" but it doesn't say what key it's trying to set
When there is an outgoing message in a chat, mark all older messages in this chat as seen.
Android already has a similar behavior, however, this led to the issue https://github.com/deltachat/deltachat-android/issues/2163 and should be changed back.
--
From the issue description at https://github.com/deltachat/deltachat-android/issues/2163, I implemented these fixes:
> Core should take care that if the last message in a chat is not fresh|noticed, no messages in the chat can be fresh. [...] Do this [...] in a function that's called at the end of fetch_new_messages(). Then dc_receive_imf() wouldn't get slower by this and we could re-use the same function for migration.
So, I didn't do this inside `dc_receive_imf()` in order not to make it take even longer. This obviously has the downside of higher complexity.
And I think we should implement this:
> On Androd, show the unread badge when unread!=0 again (see deltachat/deltachat-android@618af02). Then the user can see that there is a chat with an unread message and click it to get rid of it.
because it shouldn't be the UI's job to decide whether an unread badge is shown, but the core's.
Generally we could also just remove the device message as we have the
connectivity view, OTOH if you are not using DC a lot, it may be useful
to be notified without opening DC.
And apart from this one bug it's working fine.
Hocuri
