We don't want images having unsupported format or corrupted ones to be sent as `Image` and appear in
the "Images" tab in UIs because they can't be displayed correctly.
If a user attaches an image as `File`, we should send the original filename. And vice versa, if it's
`Image` originally, we mustn't reveal the filename.
The filename used for sending is now also saved to the db, so all the sender's devices will display
the same filename in the message info.
Otherwise unsupported and corrupted images are displayed in the "Images" tab in UIs and that looks
as a Delta Chat bug. This should be a rare case though, so log it as error and let the user know
that metadata isn't removed from the image at least.
this PR scaled avatars using the Triangle-filter,
resulting in often better image quality and smaller files (5%).
it comes at high costs,
therefore, we do not do that unconditionally for each image sent, see
comment in the code
and https://github.com/chatmail/core/pull/6815
---------
Co-authored-by: iequidoo <117991069+iequidoo@users.noreply.github.com>
up to now, avatars were restricted to 20k, 256x256 pixel.
resulting in often poor avatar quality.
(the limitation comes from times
where unencrypted outlook inner headers were a thing;
this has changed meanwhile)
to increase quality while keeping a reasonable small size,
for "balanced quality",
we increase the allowed width to 512x512 and the allowed size to 60k.
for "worse quality", things stay unchanged at 128x128 pixel and 20k.
Now that we are deduplicating everywhere, we can get rid of some code.
The old python bindings did not get an optional `name` parameter because
they are deprecated anyway, but it would be easy to add it.
With iOS and Desktop copying the file to a to a temp file with the name
of `get_filename()`, it should be sanitized first.
The PR can be reviewed commit-by-commit or all at once.
There was a bug that file extensions were removed when recoding an
avatar. The problem was that `recode_avatar` used `name` to check for
the extension, but some functions passed an empty string.
There even were two tests from before the decision to keep the
extensions that tested for the faulty behavior.
These were the last places in the `deltachat` crate where files were
stored without deduplication. The CFFI python bindings are the last
thing that's still missing.
Deduplicate:
- In the REPL
- In `store_from_base64()`, which writes avatars received in headers
- In a few tests
- The saved messages, broadcast, device, archive icons
- The autocrypt setup message
1-2 more PRs, and we can get rid of `BlobObject::create`,
`sanitise_name()`, and some others
When receiving messages, blobs will be deduplicated with the new
function `create_and_deduplicate_from_bytes()`. For sending files, this
adds a new function `set_file_and_deduplicate()` instead of
deduplicating by default.
This is for
https://github.com/deltachat/deltachat-core-rust/issues/6265; read the
issue description there for more details.
TODO:
- [x] Set files as read-only
- [x] Don't do a write when the file is already identical
- [x] The first 32 chars or so of the 64-character hash are enough. I
calculated that if 10b people (i.e. all of humanity) use DC, and each of
them has 200k distinct blob files (I have 4k in my day-to-day account),
and we used 20 chars, then the expected value for the number of name
collisions would be ~0.0002 (and the probability that there is a least
one name collision is lower than that) [^1]. I added 12 more characters
to be on the super safe side, but this wouldn't be necessary and I could
also make it 20 instead of 32.
- Not 100% sure whether that's necessary at all - it would mainly be
necessary if we might hit a length limit on some file systems (the
blobdir is usually sth like
`accounts/2ff9fc096d2f46b6832b24a1ed99c0d6/dc.db-blobs` (53 chars), plus
64 chars for the filename would be 117).
- [x] "touch" the files to prevent them from being deleted
- [x] TODOs in the code
For later PRs:
- Replace `BlobObject::create(…)` with
`BlobObject::create_and_deduplicate(…)` in order to deduplicate
everytime core creates a file
- Modify JsonRPC to deduplicate blob files
- Possibly rename BlobObject.name to BlobObject.file in order to prevent
confusion (because `name` usually means "user-visible-name", not "name
of the file on disk").
[^1]: Calculated with both https://printfn.github.io/fend/ and
https://www.geogebra.org/calculator, both of which came to the same
result
([1](https://github.com/user-attachments/assets/bbb62550-3781-48b5-88b1-ba0e29c28c0d),
[2](https://github.com/user-attachments/assets/82171212-b797-4117-a39f-0e132eac7252))
---------
Co-authored-by: l <link2xt@testrun.org>
Even if UIs don't call `Message::force_sticker()`, they don't want conversions of `Sticker` to
`Image` if it's obviously not an image, particularly, has non-image extension. Also UIs don't want
conversions of `Sticker` to anything other than `Image`, so let's keep the `Sticker` viewtype in
this case.
Before file extensions were also limited to 32 chars, but extra chars in the beginning were just cut
off, e.g. "file.with_lots_of_characters_behind_point_and_double_ending.tar.gz" was considered to
have an extension "d_point_and_double_ending.tar.gz". Better to take only "tar.gz" then.
Also don't include whitespace-containing parts in extensions. File extensions generally don't
contain whitespaces.
Before this change progress bar only started
when database is already transferred.
Database is usually the largest file
in the whole transfer, so the transfer appears
to be stuck for the sender.
With this change progress bar
starts for backup export
as soon as connection is received
and counts bytes transferred over the connection
using AsyncWrite wrapper.
Similarly for backup import,
AsyncRead wrapper counts the bytes
received and emits progress events.
Otherwise, e.g. if a message is a large GIF, but its viewtype is set to `Image` by the app, this GIF
will be recoded to JPEG to reduce its size. GIFs and other special viewtypes must be always detected
and sent as is.
... and fails if file already exists. The UI should open the file saving dialog, defaulting to
Downloads and original filename, when asked to save the file. After confirmation it should call
dc_msg_save_file().
Otherwise an error
"The encoder or decoder for Jpeg does not support the color type `Rgba8`"
is returned if image has an alpha channel.
This is caused by the recent change of JPEG encoder
in `image` crate: <https://github.com/image-rs/image/issues/2211>
Add white background instead of the default black one to avatars when recoding to JPEG. But not for
"usual" images to spare the CPU. The motivation is to handle correctly
"black-on-transparent-background" avatars which are quite common.
Use sync messages for that as it is done for e.g. Config::Displayname. Maybe we need to remove
avatar synchronisation via usual messages then, but let's think of it a bit.
Check if a sticker has at least one fully transparent corner and otherwise change the Sticker type
to Image. This would fix both Android and iOS at the same time and prevent similar bug on future
platforms that may get this bug like Ubuntu Touch.
