This fixes flaky tests/test_securejoin.py::test_qr_scan_updates_new_relay_address
which hanged during the second securejoin on fast network (e.g. when testing against local container)
as Bob received securejoin pubkey message from Alice but did not merge the new relay list.
It is only doing SQL, so does not need to be async.
This way insert_into_smtp() can use a single transaction
to insert a tombstone and enqueue the message.
as recently discussed with @adbenitez and @Amzd, it is better to add an
info message also for unpinning a message:
otherwise the message "Alice pinned a message" may look like a bug if
there is nothing actually pinned as Bob unpinned for whatever reason
afterwards.
technically, this is a simple change (famous last words) as a messages
were always sent on pinning as well as on unpinning, but for unpinning,
they were additionally hidden.
now the "unpin" info message is visible, however, still tuned down: it
is not notified and it does not bubble up in the chatlist (both similar
to many info messages about group changes)
ftr, this is a minor change, and it should not block ongoing releases
wrt pinned messages
join_next() failing means something went really wrong, e.g. panic inside a task.
We also don't cancel the tasks other than by dropping JoinSet,
so join_next() should never return an error.
It removes compromised emSign CA,
see https://emudhra.we-have-a-problem.com/ for details.
We have duplicate dependency because of iroh 0.35
still dependencing on the old version,
but most important is that IMAP/SMTP connections
are using the new version.
spec.md already documents Header Confidentiality Policy
that hides Chat-Version header in encrypted messages,
but so far we have copied the header into unprotected headers for compatibility.
Since 2.45.0 prefetch does not look at the unprotected Chat-Version
do decide if the message should be downloaded
(change b94792706a)
and since 2.50 show_emails setting which controlled
whether only "chat messages" are processed
is removed (change 63596a4940).
Without connection candidates IMAP busy loops when you try to stop I/O
because interrupt channel gets closed and recv() always returns an error immediately,
but no connection is attempted afterwards so IMAP loop keeps calling .prepare()
in a loop without ever yielding control to executor, so it cannot even be cancelled.
New JSON-RPC test test_pseudo_transport_start_stop_io reproduces the problem
from CFFI test tests/test_3_offline.py::TestOfflineChat::test_send_lots_of_offline_msgs
which added one transport without connection candidates and started I/O.
A reaction summary of a broadcast channel is sent as an empty reaction
to the latest channel message. When another device of the channel owner
received it, it removed the owner's own reaction on that message and
queued the message for the next summary. So the owner's devices sent
the summary to each other, and to all subscribers, every 10 minutes
forever.
self.parts.remove(i) can panic if "i" is out of range,
but now it is obvious that it will not because "i" is returned from position(),
and clippy is happy.
This is a minor fix, worst case an image with very large dimensions will not be classified as a File.
Mostly removing `clippy::arithmetic_side_effects`.
while showing reactions for channel summaries was regarded as "let's
try" when channel reactions were introduced, meanwhile, after some
experience, it only clutters and spams the chatlist.
for the subscriber, this is anyways only "you reacted", which is often
wrong as there are other reactions. for the owner it is about some names
they do not know. the idea of having some "feedback" for the owner is
still fulfilled as reactions are (optionally) notified - so owner can
still have the "ping ping ping" and relax :)
finallly, this is also what telegram is doing with their channels
Also moved python version to environment variable
so status checks will be named
"JSON-RPC Python tests (ubuntu-latest, latest)" instead of
"JSON-RPC Python tests (ubuntu-latest, 3.14)" and we don't have to
update the list of required status checks each time the version changes.
We already do the same for Rust.
Remove the two unnecessary DB calls that fetch the message
(which could possibly be different from the current `Message`).
Now the only async thing in `get_summary_text_ext` that remains
is `get_webxdc_info`.
See https://github.com/chatmail/core/pull/8826.
The timeout currently owns each IMAP and SMTP JoinHandle. When 30
seconds elapse, dropping the timeout drops the handle and detaches the
task instead of cancelling it. On Haiku, a network task stuck behind a
missed or delayed socket wakeup can therefore outlive scheduler
shutdown, retaining connections and interfering with restart or
active-account deletion.
Borrow each handle while waiting, then explicitly abort and await it if
shutdown times out. This guarantees that scheduler shutdown does not
complete while a timed-out network task is still running.
Classic email servers sometimes start rejecting all messages and ask the user to solve web CAPTCHA, but still allow to connect.
In this case we want to switch to another SMTP server by moving the transport to the end of the list returned by smtp::sorted_transports.
We disconnect after any message sending failure, so next message or resent same message will be sent over another other transport.
Permanent SMTP error does not necessarily mean that switching to another server will help.
For example if the user tries to send a large message, any server will reject it, and it is not a failure of the server.
We do not try to distinguish between error messages, so it is expected that we may
sometimes switch to another server when it is unnecessary.
Reencoding animated WebPs into JPEG loses all frames except the first one.
Avatars we still reencode because we don't want animated avatars anyway.
animated.webp is produced with this script using Pillow 12.3.0:
```
#!/usr/bin/env python3
from PIL import Image, ImageOps
with Image.open("avatar900x900.png") as im:
frames = [im.rotate(x) for x in range(10, 360, 10)]
im.save(
"animated.webp",
"webp",
append_images=frames,
lossless=True,
duration=200,
save_all=True,
)
```
Resulting image is 677K, just slightly larger than the size needed to trigger reencoding.
The test fails on main with ~150MB memory remaining allocated
after sending/receiving two 20MB messages,
and stays well below when explicitely setting libc's M_MMAP_THRESHOLD.
this PR adds a new `msg_get_summary_text(add_forwarded, add_type_emoji,
approx_chars)` api, replacing the old `get_summarytext(approx_chars)`.
checking the concrete existing and new usecases, the new options are
required, eg. for:
- pinned messages: they do not show the user name (therefore, there
should not be a "forward" indicator), and may or may not not e.g.
display image left of string (this makes type emoji optional)
- notifications, quotes: they show user name (therefore, "forward"
indicator needed) and often display e.g. image a bit off (so type emoji
is potentially useful, cmp. whatsapp, but potentially also optional)
- copy to clipboard: clipboard should get forwarded and emoji, but text
should not be truncated and linends should be preserved - esp. the
latter was a silent bug in at least android. to support that better,
`approx_chars` can be set to `0` to skip truncation and unwrapping.
| Use case | `add_forwarded` | `add_type_emoji` | `approx_chars>0` |
| --- | :---: | :---: | :---: |
| Notification | ✅ | ✅ (potentially ❌) | ✅ |
| Quote | ✅ | ✅ (potentially ❌) | ✅ |
| Pinned message | ❌ | ❌ (rarely ✅) | ✅ |
| Copy to clipboard | ✅ | ✅ | ❌ |
| Chatlist (core internal) | ✅ | ✅ | ✅ |
| Reaction summary (core internal) | ❌ | ✅ | ✅ |
(the get_summary() returning a "lot" can maybe be removed at some point,
in favor of adding the usernames directly to get_summary_text() and
changing add_forwarded to add_usernames. but that is out of scope)
jsonrpc can be added easily once we settle on name etc.
references: https://github.com/chatmail/core/pull/8812 ,
https://github.com/chatmail/core/pull/8546
cc @Amzd , @adbenitez
---------
Co-authored-by: Hocuri <hocuri@gmx.de>
mail-builder changed how such filename is encoded in Content-Disposition between versions 0.5.0 and 1.0.0.
We can parse both encodings already and the test was not failing, but it is worth adding this corner case to the tests.
mail-builder 0.5.0 encoded the filename like this:
Content-Disposition: attachment;
filename="=?utf-8?Q?very_long_filename_with_emoji_=F0=9F=98=80_very_long_file_name_with_emoji_=F0=9F=AB=A0_very_long_file_name_with_emoji.tar.gz?="
mail-builder 1.0.0 encodes the filename like this:
Content-Disposition: attachment;
filename*0*=UTF-8''very%20long%20filename%20with%20emoji%20%F0%9F%98%80%20;
filename*1*=very%20long%20file%20name%20with%20emoji%20%F0%9F%AB%A0%20very;
filename*2*=%20long%20file%20name%20with%20emoji.tar.gz
link2xt
B. Petersen
gluek
WofWca
pancake
Hocuri
holger krekel