These were the last places in the `deltachat` crate where files were
stored without deduplication. The CFFI python bindings are the last
thing that's still missing.
When receiving messages, blobs will be deduplicated with the new
function `create_and_deduplicate_from_bytes()`. For sending files, this
adds a new function `set_file_and_deduplicate()` instead of
deduplicating by default.
This is for
https://github.com/deltachat/deltachat-core-rust/issues/6265; read the
issue description there for more details.
TODO:
- [x] Set files as read-only
- [x] Don't do a write when the file is already identical
- [x] The first 32 chars or so of the 64-character hash are enough. I
calculated that if 10b people (i.e. all of humanity) use DC, and each of
them has 200k distinct blob files (I have 4k in my day-to-day account),
and we used 20 chars, then the expected value for the number of name
collisions would be ~0.0002 (and the probability that there is a least
one name collision is lower than that) [^1]. I added 12 more characters
to be on the super safe side, but this wouldn't be necessary and I could
also make it 20 instead of 32.
- Not 100% sure whether that's necessary at all - it would mainly be
necessary if we might hit a length limit on some file systems (the
blobdir is usually sth like
`accounts/2ff9fc096d2f46b6832b24a1ed99c0d6/dc.db-blobs` (53 chars), plus
64 chars for the filename would be 117).
- [x] "touch" the files to prevent them from being deleted
- [x] TODOs in the code
For later PRs:
- Replace `BlobObject::create(…)` with
`BlobObject::create_and_deduplicate(…)` in order to deduplicate
everytime core creates a file
- Modify JsonRPC to deduplicate blob files
- Possibly rename BlobObject.name to BlobObject.file in order to prevent
confusion (because `name` usually means "user-visible-name", not "name
of the file on disk").
[^1]: Calculated with both https://printfn.github.io/fend/ and
https://www.geogebra.org/calculator, both of which came to the same
result
([1](https://github.com/user-attachments/assets/bbb62550-3781-48b5-88b1-ba0e29c28c0d),
[2](https://github.com/user-attachments/assets/82171212-b797-4117-a39f-0e132eac7252))
---------
Co-authored-by: l <link2xt@testrun.org>
with this PR, when an `.xdc` with `request_integration = map` in the
manifest is added to the "Saved Messages" chat, it is used _locally_ as
an replacement for the shipped maps.xdc (other devices will see the
`.xdc` but not use it)
this allows easy development and adapting the map to use services that
work better in some area.
there are lots of known discussions and ideas about adding more barriers
of safety. however, after internal discussions, we decided to move
forward and also to allow internet, if requested by an integration (as
discussed at
https://github.com/deltachat/deltachat-core-rust/pull/3516).
the gist is to ease development and to make users who want to adapt,
actionable _now_, without making things too hard and adding too high
barriers or stressing our own resources/power too much.
note, that things are still experimental and will be the next time -
without the corresponding switch being enabled, nothing will work at
all, so we can be quite relaxed here :)
for android/ios, things will work directly. for desktop, allow_internet
needs to be accepted unconditionally from core. for the future, we might
add a question before using an integration and/or add signing. or sth.
completely different - but for now, the thing is to get started.
nb: "integration" field in the webxdc-info is experimental as well and
should not be used in UIs at all currently, it may vanish again and is
there mainly for simplicity of the code; therefore, no need to document
that.
successor of https://github.com/deltachat/deltachat-core-rust/pull/5461
this is how it looks like currently - again, please note that all that
is an experiment!
<img width=320
src=https://github.com/deltachat/deltachat-core-rust/assets/9800740/f659c891-f46a-4e28-9d0a-b6783d69be8d>
<img width=320
src=https://github.com/deltachat/deltachat-core-rust/assets/9800740/54549b3c-a894-4568-9e27-d5f1caea2d22>
... when going out of experimental, there are loots of ideas, eg.
changing "Start" to "integrate"
this PR removes most usages of the `descr` parameter.
- to avoid noise in different branches etc. (as annoying on similar, at
a first glance simple changes), i left the external API stable
- also, the effort to do a database migration seems to be over the top,
so the column is left and set to empty strings on future updates - maybe
we can recycle the column at some point ;)
closes#6245
add `update.href` property option to update objects send via
`Context::send_webxdc_status_update()`.
when set together with `update.info`,
UI can implement the info message as a link that is passed to the webxdc
via `window.location.href`.
for that purpose, UI will read the link back from
`Message::get_webxdc_href()`.
Practically,
this allows e.g. an calendar.xdc
to emits clickable update messages
opening the calendar at the correct date.
closes#6219
documentation at https://github.com/webxdc/website/pull/90
this PR adds support for the property `update.notify` to notify about
changes in `update.info` or `update.summary`. the property can be set to
an array of addresses [^1]
core emits then the event `IncomingWebxdcNotify`, resulting in all UIs
to display a system notification, maybe even via PUSH.
for using the existing `update.info` and `update.summary`: the message
is no secret and should be visible to all group members as usual, to not
break the UX of having same group messages on all devices of all users -
as known already from the normal messages.
also, that way, there is no question what happens if user have disabled
notifications as the change is presented in the chat as well
doc counterpart at https://github.com/webxdc/website/pull/90closes#6217
[^1]: addresses come in either via the payload as currently or as an
explicit sender in the future - this does not affect this PR. same for
translations, see discussions at #6217 and #6097
---------
Co-authored-by: adb <asieldbenitez@gmail.com>
Co-authored-by: l <link2xt@testrun.org>
as discussed in several chats, this PR starts making it possible to use
Webxdc as integrations to the main app. In other word: selected parts of
the main app can be integrated as Webxdc, eg. Maps [^1]
this PR contains two parts:
- draft an Webxdc Integration API
- use the Webxdc Integration API to create a Maps Integration
to be clear: a Webxdc is not part of this PR. the PR is about marking a
Webxdc being used as a Map - and core then feeds the Webxdc with
location data. from the view of the Webxdc, the normal
`sendUpdate()`/`setUpdateListener()` is used.
things are still marked as "experimental", idea is to get that in to
allow @adbenitez and @nicodh to move forward on the integrations into
android and desktop, as well as improving the maps.xdc itself.
good news is that we currently can change the protocol between Webxdc
and core at any point :)
# Webxdc Integration API
see `dc_init_webxdc_integration()` in `deltachat.h` for overview and
documentation.
rust code is mostly in `webxdc/integration.rs` that is called by other
places as needed. current [user of the API is
deltachat-ios](https://github.com/deltachat/deltachat-ios/pull/1912),
android/desktop will probably follow.
the jsonrpc part is missing and can come in another PR when things are
settled and desktop is really starting [^2] (so we won't need to do all
iterations twice :) makes also sense, when this is done by someone
actually trying that out on desktop
while the API is prepared to allow other types of integrations (photo
editor, compose tools ...) internally, we currently ignore the type. if
that gets more crazy, we probably also need a dedicated table for the
integrations and not just a single param.
# Maps Integration
rust code is mostly in `webxdc/maps_integration.rs` that is called by
`webxdc/integration.rs` as needed.
EDIT: the idea of having a split here, is that
`webxdc/maps_integration.rs` really can focus on the json part, on the
communication with the .xdc, including tests
this PR is basic implementation, enabling to move forward on
integrations on iOS, but also on desktop and android.
the current implementation allows already the following:
- global and per-chat maps
- add and display POIs
- show positions and tracks of the last 24 hours
the current maps.xdc uses leaflet, and is in some regards better than
the current android/desktop implementations (much faster, show age of
positions, fade out positions, always show names of POIs, clearer UI).
however, we are also not bound to leaflet, it can be anything
> [**screenshots of the current
state**](https://github.com/deltachat/deltachat-ios/pull/1912)
> 👆
to move forward faster and to keep this PR small, the following will go
to a subsequent PR:
- consider allowing webxdc to use a different timewindow for the
location
- delete POIs
- jsonrpc
[^1]: maps are a good example as anyways barely native (see android
app), did cause a lot of pain on many levels in the past (technically,
bureaucratically), and have a comparable simple api
[^2]: only going for jsonrpc would only make sense if large parts of
android/ios would use jsonrpc, we're not there
---------
Co-authored-by: link2xt <link2xt@testrun.org>
