This esp. speeds up receive_imf a bit when we recreate the member list (recreate_member_list == true).
It's a preparation for https://github.com/deltachat/deltachat-core-rust/issues/3768, which will be a one-line-fix, but recreate the member list more often, so that we want to optimize this case a bit.
It also adds a benchmark for this case. It's not that easy to make the benchmark non-flaky, but by closing all other programs and locking the CPU to 1.5GHz it worked. It is consistently a few percent faster than ./without-optim:
```
Receive messages/Receive 100 Chat-Group-Member-{Added|Removed} messages
time: [52.257 ms 52.569 ms 52.941 ms]
change: [-3.5301% -2.6181% -1.6697%] (p = 0.00 < 0.05)
Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
4 (4.00%) high mild
3 (3.00%) high severe
```
It's a w/a for "Space added before long group names after MIME serialization/deserialization"
issue. DC itself never creates group names with leading/trailing whitespace, so it can be safely
removed. On the sender side there's no trim() because group names anyway go through
improve_single_line_input(). And I believe we should send the exact name we have in our db. Also
there's no check for leading/trailing whitespace because there may be existing user databases with
group names having such whitespaces.
Since switch to async we don't have spurious "database is busy"
errors anymore. Since an error is irrecoverable in most cases,
we can skip the message. The cost of this is we may
accidentally skip a correct message if I/O fails, but
the advantage is that we are guaranteed to never confuse
irrecoverable error with recoverable one and get stuck in
infinite loop redownloading the same message over and over.
That's a bug which @Simon-Laux and probably also @hpk42 had, where one malformed incoming (Spam-) mail blocked the receiving of all emails coming after it.
The problem was that from_field_to_contact_id() returned ContactId::UNDEFINED, and then lookup_by_contact() returned Err.
* Treat multiple From addresses as if there was no From: addr
* changelog
* Don't send invalid emails through the whole receive_imf pipeline
Instead, directly create a trash entry for them.
* Don't create trash entries for randomly generated Message-Id's
* clippy
* fix typo
Co-authored-by: link2xt <link2xt@testrun.org>
If we fetch messages out of order, then f.e. reactions don't work because if we process a reaction not yet having the corresponding message processed, the reaction is thrown away.
* allow deleting referenced contacts in UI
we are quite often getting requests of users
who want to get rid of some contact in the "new chat" list.
there is already a "delete" option,
but it does not work for referenced contacts -
however, it is not obvious for users that a contact is in use,
esp. of some mailing list or larger chat, old contacts, whatever.
this pr revives an old idea [^1] of "soft deleting" referenced contacts -
this way, the user can remove the annoying entry
without the need to understand complicated things
and finally saying that deletion is impossible :)
once the contact is reused, it will reappear,
however, this is already explained in the confirmation dialog of the UIs.
technically, this pr was simpler as expected as we already have
a Origin::Hidden, that is just reused here.
[^1]: https://github.com/deltachat/deltachat-core/pull/542
* update rust doccomment
* update changelog
* avoid races on contact deletion
chats may be created between checking for "no chats" and contact deletion.
this is prevented by putting the statement into an EXCLUSIVE transaction.
* fix failing python test
* Add DC_EVENT_INCOMING_MSG event
* Fix lots of compile errors
* Docs
* Changelog
* Fix python tests
Adding DC_EVENT_INCOMING_MSG_BUNCH made the python tests fail because they use `get_matching("DC_EVENT_INCOMING_MSG")`, which also matches DC_EVENT_INCOMING_MSG_BUNCH, so the tests got confused.
This fixes `get_matching()` to only match whole event names.
* Also fix test_ac_setup_message_twice()
The built regex was ^EVENT_NAME1|EVENT_NAME2$, which becomes parsed as
"^EVENT_NAME1" OR "EVENT_NAME2$". Introduce a group (parentheses) to fix
this.
* desktop will use DC_EVENT_INCOMING_MSG_BUNCH,
so I would not call it experimental anymore
* add generated node constants
* msg_ids in the event as Vec<u32>
number[] in js land
this is way more convinient than a json encoded string.
* Apply suggestions from code review
Co-authored-by: bjoern <r10s@b44t.com>
Co-authored-by: Simon Laux <mobile.info@simonlaux.de>
Co-authored-by: Simon Laux <Simon-Laux@users.noreply.github.com>
Co-authored-by: bjoern <r10s@b44t.com>
The problem was that a message without Autocrypt key or with a wrong
signature resets peerstate regardless of what DKIM check says. I
inserted sleep(1.1) to make sure reset always happens and make the bug
reproducible, then fixed it by forbidding reset if DKIM check fails.
https://github.com/deltachat/deltachat-core-rust/pull/3731
The way to create a Context is now rather burdensome, users have to
create and import a bunch of things just to get a Context. So let's
introduce a builder.
Notice that the builder can only produce an open context, if the
context can not be opened it is dropped. This is on purpose, the
Context itself can become RAII again at some point by doing this.
Only the FFI needs to have the concept of an open and a closed
Context.
Fix#3507
Note that this is not intended for a release at this point! We first have to test whether it runs stable enough. If we want to make a release while we are not confident enough in authres-checking, then we have to disable it.
BTW, most of the 3000 new lines are in `test_data/messages/dkimchecks...`, not the actual code
da3a4b94 adds the results to the Message info. It currently does this by adding them to `hop_info`. Maybe we should rename `hop_info` to `extra_info` or something; this has the disadvantage that we can't rename the sql column name though.
Follow-ups for this could be:
- In `update_authservid_candidates()`: Implement the rest of the algorithm @hpk42 and me thought about. What's missing is remembering how sure we are that these are the right authserv-ids. Esp., when receiving a message sent from another account at the same domain, we can be quite sure that the authserv-ids in there are the ones of our email server. This will make authres-checking work with buzon.uy, disroot.org, yandex.ru, mailo.com, and riseup.net.
- Think about how we present this to the user - e.g. currently the only change is that we don't accept key changes, which will mean that the small lock on the message is not shown.
- And it will mean that we can fully enable AEAP, after revisiting the security implications of this, and assuming everyone (esp. @link2xt who pointed out the problems in the first place) feels comfortable with it.
* let search_msgs() return unaccepted requests
unaccepted chat requests are shown in the chatlist,
it should be returned by search_msgs() an by the other search functions as well.
form the view of the user, the search acts like a filter,
so there is no reason to hide things additionally.
also, the user may remember a word in a chat request,
maybe even an archived one (there is no need to accept a request before archiving)
that one wants to search later on.
* test searching for unaccepted requests
* simplyfy expression; `c.blocked!=1` is also what is used in similar statements
* jsonrpc: typescript client: export constants
under `C` enum,
similar to how its exported from `deltachat-node`
* add pr number to changelog
* fix tests
* fix changelog entry position
* jsonrpc in cffi also sends events now
* add pr id to changelog
* jsonrpc: new format for events and better typescript autocompletion (#3663)
* jsonrpc: new format for events and better typescript autocompletion
* adjust doc comments
