Change `BccSelf` default to 0 for chatmail configurations and enable it upon a backup export. As for
`DeleteServerAfter` who was set to 0 upon a backup export before, make its default dependent on
`BccSelf` for chatmail. We don't need `BccSelf` for chatmail by default because we assume
single-device use. Also `BccSelf` is needed for "classic" email accounts even if `DeleteServerAfter`
is set to "immediately" to detect that a message was sent if SMTP server is slow to respond and
connection is lost before receiving the status line which isn't a problem for chatmail servers.
Even if UIs don't call `Message::force_sticker()`, they don't want conversions of `Sticker` to
`Image` if it's obviously not an image, particularly, has non-image extension. Also UIs don't want
conversions of `Sticker` to anything other than `Image`, so let's keep the `Sticker` viewtype in
this case.
In multi-device case `vg-request-with-auth` left on IMAP may result in situation when Bob joins the
group, then leaves it, then second Alice device comes online and processes `vg-request-with-auth`
again and adds Bob back. So we should IMAP-delete `vg-request-with-auth`. Another device will know
the Bob's key from Autocrypt-Gossip. It's not a problem if Alice loses state (restores from an old
backup) or goes offline for long before sending `vg-member-added`, anyway it may not be delivered by
the server, rather Bob should retry sending SecureJoin messages as he is a part which wants to join,
so let's not solve this for now.
Add a `create` param to `select_with_uidvalidity()` instead of always trying to create the folder
and return `Ok(false)` from it if the folder doesn't exist and shouldn't be created, and handle this
in `store_seen_flags_on_imap()` by just removing "jobs" from the `imap_markseen` table. Also don't
create the folder in other code paths where it's not necessary.
This is to avoid revalidating HTTP cache too frequently (and have many parallel revalidation tasks)
if revalidation fails or the HTTP request takes some time. The stale period >= 1 hour, so 1 more
minute won't be a problem.
It was called from `receive_imf` when an outgoing message is received. But
`Imap::fetch_new_messages()` already calls `chat::mark_old_messages_as_noticed()` which does the job
better (per-message).
SQL statements fail if the number of variables
exceeds `SQLITE_LIMIT_VARIABLE_NUMBER`.
Remaining repeat_vars() calls are difficult to replace
and use arrays passed from the UI,
e.g. forwarded message IDs or marked as seen IDs.
UIs may want to display smth like "Transferring..." after "Establishing connection between
devices..." on nonzero progress. Before, progress on the receiver side was starting with 2 after
receiving enough data.
It only contained proxy
that some users ran in Termux
to look at IMAP traffic.
The same can be achieved with `socat`, e.g.:
socat -v TCP-LISTEN:9999,bind=127.0.0.1 OPENSSL:nine.testrun.org:993
This fixes the HTML display of messages containing forwarded messages. Before, forwarded messages
weren't rendered in HTML and if a forwarded message is long and therefore truncated in the chat, it
could only be seen in the "Message Info". In #4462 it was suggested to display "Show Full
Message..." for each truncated message part and save to `msgs.mime_headers` only the corresponding
part, but this is a quite huge change and refactoring and also it may be good that currently we save
the full message structure to `msgs.mime_headers`, so i'd suggest not to change this for now.
Otherwise the "Show Full Message..." button appears somewhere in the middle of the multipart
message, e.g. after a text in the first message bubble, but before a text in the second
bubble. Moreover, if the second/n-th bubble's text is shortened (ends with "[...]"), the user should
scroll up to click on "Show Full Message..." which doesn't look reasonable. Scrolling down looks
more acceptable (e.g. if the first bubble's text is shortened in a multipart message).
I'd even suggest to show somehow that message bubbles belong to the same multipart message, e.g. add
"[↵]" to the text of all bubbles except the last one, but let's discuss this first.
without the prefix,
it looks as if it is part of the Message-ID,
esp. if Message-ID is longer,
a break on different delimiters may look exactly the same.
see #6329 for some screenshots that initially confused me :)
A user reported to me that after they left a group, they were implicitly readded, but there's no any
readdition message, so currently it looks in the chat like leaving it has no effect, just new
messages continue to arrive. The readdition probably happened because some member didn't receive the
user's self-removal message, anyway, at least there must be a message that the user is readded, even
if it isn't known by whom.
A NDN may arrive days after the message is sent when it's already impossible to tell which message
wasn't delivered looking at the "Failed to send" info message, so it only clutters the chat and
makes the user think they tried to send some message recently which isn't true. Moreover, the info
message duplicates the info already displayed in the error message behind the exclamation mark and
info messages do not point to the message that is failed to be sent.
Moreover it works rarely because `mimeparser.rs` only parses recipients from `x-failed-recipients`,
so it likely only works for Gmail. Postfix does not add this `X-Failed-Recipients` header. Let's
remove this parsing too. Thanks to @link2xt for pointing this out.
This PR:
- Moves the note about the false positive to the end of the test output,
where it is more likely to be noticed
- Also notes in test_modify_chat_disordered() and
test_setup_contact_*(), in addition to the existing note in
test_was_seen_recently()
This fixes HTML display of truncated (long) sent messages ("Show full message" in UIs). Before,
incorrect HTML was stored (with missing line breaks etc.) for them. Now stored plaintext is
formatted to HTML upon calling `MsgId::get_html()` and this results in the same HTML as on a
receiver side.
Also cleaned up test_connectivity()
which tested that state does not flicker to WORKING
when there are no messages to be fetched.
The state is expected to flicker to WORKING
when checking for new messages,
so the tests were outdated since
change 3b0b2379b8
The benchmark function (e.g. `recv_all_emails()`) is executed multiple
times on the same context. During the second iteration, all the emails
were already in the database, so, receiving them again failed.
This PR fixes that by passing in a second `iteration` counter that is
different for every invocation of the benchmark function.
This better reflects that this state means
we just connected and there may me work to do.
This state is converted to DC_CONNECTIVITY_WORKING
instead of DC_CONNECTIVITY_CONNECTED state now.
Before this change when IMAP connected
to the server, it switched
from DC_CONNECTIVITY_NOT_CONNECTED
to DC_CONNECTIVITY_CONNECTING,
then to DC_CONNECTIVITY_CONNECTED (actually preparing)
then to DC_CONNECTIVITY_WORKING
and then to DC_CONNECTIVITY_CONNECTED again (actually idle).
On fast connections this resulted in flickering "Connected"
string in the status bar right before "Updating..."
and on slow connections this "Connected" state
before "Updating..." lasted for a while
leaving the user to wonder if there are no new messages
or if Delta Chat will still switch to "Updating..."
before going into "Connected" state again.
Already apply rust beta (1.84) clippy suggestions now, before they let
CI fail in 6 weeks.
The newly used functions are available since 1.70, our MSRV is 1.77, so
we can use them.
A sync message for accepting or blocking a 1:1 chat may arrive before the first message from the
contact, when it does not exist yet. This frequently happens in non-chatmail accounts that have
moving to the DeltaChat folder disabled because Delta Chat unconditionally uploads sync messages to
the DeltaChat folder. Let's create a hidden contact in this case and a 1:1 chat for it.
this partly reverts experimental #3516
that allowed any .xdc sent to "Saved Messages" to request internet.
this helped on pushing map integration forward.
meanwhile, however, we have that map integration (#5461 and #5678),
that implies `info.internet_access` being set.
experimental `manifest.request_internet_access` is no longer needed therefore.
future will tell, if we revive the option at some point or
go for more intrations ('sending' is discussed often :) -
but currently it is not needed.
with this PR, when an `.xdc` with `request_integration = map` in the
manifest is added to the "Saved Messages" chat, it is used _locally_ as
an replacement for the shipped maps.xdc (other devices will see the
`.xdc` but not use it)
this allows easy development and adapting the map to use services that
work better in some area.
there are lots of known discussions and ideas about adding more barriers
of safety. however, after internal discussions, we decided to move
forward and also to allow internet, if requested by an integration (as
discussed at
https://github.com/deltachat/deltachat-core-rust/pull/3516).
the gist is to ease development and to make users who want to adapt,
actionable _now_, without making things too hard and adding too high
barriers or stressing our own resources/power too much.
note, that things are still experimental and will be the next time -
without the corresponding switch being enabled, nothing will work at
all, so we can be quite relaxed here :)
for android/ios, things will work directly. for desktop, allow_internet
needs to be accepted unconditionally from core. for the future, we might
add a question before using an integration and/or add signing. or sth.
completely different - but for now, the thing is to get started.
nb: "integration" field in the webxdc-info is experimental as well and
should not be used in UIs at all currently, it may vanish again and is
there mainly for simplicity of the code; therefore, no need to document
that.
successor of https://github.com/deltachat/deltachat-core-rust/pull/5461
this is how it looks like currently - again, please note that all that
is an experiment!
<img width=320
src=https://github.com/deltachat/deltachat-core-rust/assets/9800740/f659c891-f46a-4e28-9d0a-b6783d69be8d>
<img width=320
src=https://github.com/deltachat/deltachat-core-rust/assets/9800740/54549b3c-a894-4568-9e27-d5f1caea2d22>
... when going out of experimental, there are loots of ideas, eg.
changing "Start" to "integrate"
following `RELEASE.md`, after merging, the following is needed:
6. Tag the release: `git tag --annotate v1.151.2`.
7. Push the release tag: `git push origin v1.151.2`.
8. Create a GitHub release: `gh release create v1.151.2 --notes ''`.
this PR adds the `href` from `update.href` to the IncomingWebxdcNotify
event (DC_EVENT_INCOMING_WEBXDC_NOTIFY in cffi)
purpose is to add a "Start" button to the notifications that allow
starting the app immediately with the given href
using long options make things less mystical, it is clearer what
happens.
(apart from that i also disallowed `git -a` on my machine in general, as
`git commit -a` is considered harmful. as my approach to disallow that
is a bit greedy and disallows `-a` just for any git commands, this is
the only place where i regularly struggle :)
Text parts are using quoted-printable encoding
which takes care of wrapping long lines,
so using format=flowed is unnecessary.
This improves compatibility with receivers
which do not support format=flowed.
Receiving format=flowed messages is still possible, receiver side of
Delta Chat is unchanged.
- **feat: add `AccountsChanged` and `AccountsItemChanged` events**
- **emit event and add tests**
closes#6106
TODO:
- [x] test receiving synced config from second device
- [x] bug: investigate how to delay the configuration event until it is
actually configured - because desktop gets the event but still shows
account as if it was unconfigured, maybe event is emitted before the
value is written to the database?
- [x] update node bindings constants
this PR removes most usages of the `descr` parameter.
- to avoid noise in different branches etc. (as annoying on similar, at
a first glance simple changes), i left the external API stable
- also, the effort to do a database migration seems to be over the top,
so the column is left and set to empty strings on future updates - maybe
we can recycle the column at some point ;)
closes#6245
add `update.href` property option to update objects send via
`Context::send_webxdc_status_update()`.
when set together with `update.info`,
UI can implement the info message as a link that is passed to the webxdc
via `window.location.href`.
for that purpose, UI will read the link back from
`Message::get_webxdc_href()`.
Practically,
this allows e.g. an calendar.xdc
to emits clickable update messages
opening the calendar at the correct date.
closes#6219
documentation at https://github.com/webxdc/website/pull/90
it may be handy for an xdc to have only one list of all adresses, or
there may just be bugs.
in any case, do not notify SELF, e.g. in a multi-device setup; we're
also not doing this for other messages.
this is also a preparation for having an option to notify ALL.
I found the old documentation rather hard to understand. The new doc
string:
- uses whole sentences, leaving less space for misinterpretation
- explicitly mentions that it can happen that there is no
webxdc-info-message
- is clearly structured using bullet points.
this PR adds support for the property `update.notify` to notify about
changes in `update.info` or `update.summary`. the property can be set to
an array of addresses [^1]
core emits then the event `IncomingWebxdcNotify`, resulting in all UIs
to display a system notification, maybe even via PUSH.
for using the existing `update.info` and `update.summary`: the message
is no secret and should be visible to all group members as usual, to not
break the UX of having same group messages on all devices of all users -
as known already from the normal messages.
also, that way, there is no question what happens if user have disabled
notifications as the change is presented in the chat as well
doc counterpart at https://github.com/webxdc/website/pull/90closes#6217
[^1]: addresses come in either via the payload as currently or as an
explicit sender in the future - this does not affect this PR. same for
translations, see discussions at #6217 and #6097
---------
Co-authored-by: adb <asieldbenitez@gmail.com>
Co-authored-by: l <link2xt@testrun.org>
this PR adds the address to be used by the UI for
`window.webxdc.selfAddr` to webxdc-info. UIs need to be changed
accordingly and must not use configured_addr any longer.
the address is created by sha256(private-key + rfc724_mid) , which
results in different addresses for each webxdc, without the option to
find out the real address of the user.
this also returns the same address for a multi-device-setup - sending
totally random self address around might be an alternative, however
would require connectivity (both devices may be offline on first start).
for existing app, after the change, there will be a new user, resulting
eg. in a new highscore, otherwise, things should be mostly fine. this
assumption is also important as we might change the thing another time
when it comes to multi-transport.
ftr, addresses look like
`0f187e3f420748b03e3da76543e9a84ecff822687ce7e94f250c04c7c50398bc` now
when this is merged, we need to adapt #6230 and file issues for all UI
to use `info.selfAddr`
closes#6216
`receive_imf() is only used in tests and the REPL, which enables the
"internals" feature. This PR marks it as such, so that it's clear not
only from the comment that this function is not used for anything else.
If a message partially downloaded before is already IMAP-seen upon a full download, it should be
updated to `InSeen`. OTOH if it's not IMAP-seen, but already `InNoticed` locally, its state should
be preserved. So we take the maximum of two states.
This fixes sending MDNs for big messages when they are downloaded and really seen. Otherwise MDNs
are not sent for big encrypted messages because they "don't want MDN" until downloaded.
Add a test on what happens currently when apps call `markseen_msgs()` for not downloaded encrypted
messages. Such messages are marked as seen, but MDNs aren't sent for them. Also currently when such
a message is downloaded, it remains `InSeen` despite the full content hasn't yet been seen by the
user.
As discussed in #5467 we want to use `i.delta.chat` in QR codes in favor
of `OPENPGP4FPR:` scheme. This PR does the replacement in
`get_securejoin_qr` which is used in `get_securejoin_qr_svg`.
close#5467
This is needed for iOS (https://github.com/deltachat/deltachat-ios/pull/2393), see comment in the code. An alternative would be
to add an API `invalidate_config_cache()` or to do nothing and just
assume that things will be fine.
There is a change in behavior for the case
when name is the same as the suffix
(`name_len` == `namespc_len`),
but normally `files_in_use` should not contain empty filenames.
The message may be deleted while chatlist item is loading.
In this case displaying "No messages" is better than failing.
Ideally loading of the chatlist item
should happen in 1 database transaction and
always return some message if chat is not empty,
but this requires large refactoring.
this PR changes `DC_CERTCK_ACCEPT_*` to the same values in cffi as rust
does. and regards the same values as deprecated afterwards
there is some confusion about what is deprecated and what not, see
https://github.com/deltachat/deltachat-android/issues/3408
iOS needs to be adapted as it was following the docs in the CFFI before,
same desktop. both need to be graceful on reading and strict on writing.
~~**this PR is considered harmful,** so we should not merge that in
during 1.48 release, there is no urgency, things are fine (wondering if
it isn't even worth the effort, however, having different values and
deprecations is a call for trouble in the future ...)~~
---------
Co-authored-by: iequidoo <117991069+iequidoo@users.noreply.github.com>
due to async processing,
it may happen getConnectivityHtml() is called from UI before startIO() is actually called.
eg. on iOS, we may delay startIo() if another process is still processing a PUSH notification -
when during this time, the connectivity view is opened,
it is weird if a big error "CONTACT THE DEVELOPERS!11!!!" is shown :)
also, there is not really a function is_connected(),
for $reasons, as this turned out to be flacky,
so it is not even easy to check the state before calling getConnectivityHtml()
it is not worth in doing too much special,
we are talking about rare situaton,
also, the connectivity view gets updated some moments later.
Before I was getting
```
error: attribute 'targetPlatforms' missing
at /nix/store/dyzl40h25l04565n90psbhzgnc5vp2xr-source/pkgs/build-support/rust/build-rust-package/default.nix:162:7:
161| meta.platforms or lib.platforms.all
162| rustc.targetPlatforms;
| ^
163| };
```
This was probably an upstream issues as discussed in here
https://discourse.nixos.org/t/error-attribute-targetplatforms-missing-after-updating-inputs/54494
After this update it is fixed.
80 characters are a bit limited in practise ...
On Mon, 3 Jan, 2022 at 8:34 PM "Anonymous The Mighty" <anonymous@example.com> wrote:
... already breaks the limit. it is good to allow up to 40 additional characters
for name + email address.
allowing any length, however, may catch too much,
as the line could also be a normal paragraph with important content,
so 120 characters seems reasonable.
the idea of adding more complexity here would probably lead only to, well more complexity -
things can anyways go wrong -
and, we have the "show full message..." button for exactly that purpose,
so that the user can access everything as original.
so, if things go wrong sometimes,
this is expected and fine.
There were many cases in which "member added/removed" messages were added to chats even if they
actually do nothing because a member is already added or removed. But primarily this fixes a
scenario when Alice has several devices and shares an invite link somewhere, and both their devices
handle the SecureJoin and issue `ChatGroupMemberAdded` messages so all other members see a
duplicated group member addition.
This change adds support for receiving
Autocrypt header in the protected part of encrypted message.
Autocrypt header is now also allowed in mailing lists.
Previously Autocrypt header was rejected when
List-Post header was present,
but the check for the address being equal to the From: address
is sufficient.
New experimental `protect_autocrypt` config is disabled
by default because Delta Chat with reception
support should be released first on all platforms.
Over the past years, it happend two times that a user came to me worried
about a false-positive "Cannot login as ***. Please check if the e-mail
address and the password are correct." message.
I'm not sure why this happened, but this PR makes the logic for
showing this notification stricter:
- Before: The notification is shown if connection fails two times in a
row, and the second error contains the word "authentication".
- Now: The notification is shown if the connection fails two times in a
row, and _both_ error messages contain the word "authentication".
The second commit just renames `login_failed_once` to
`authentication_failed_once` in order to reflect this change.
Delta Chat for Android does not support Android 4 anymore,
so there is no reason to keep using unsupported NDK.
r27 is the latest LTS version of Android NDK.
Tested:
- `nix build .#deltachat-rpc-server-arm64-v8a-android`
- `nix build .#deltachat-rpc-server-armv6l-linux`
`nix build .#deltachat-rpc-server-x86_64-android`
and
`nix build .#deltachat-rpc-server-x86-android`
still fail, but we do not build it in CI.
Explicit check for `-----BEGIN PGP MESSAGE-----` is unnecessary
and not sufficient to ensure that the message is valid.
We have already checked the MIME type,
so ASCII-armored OpenPGP message should be inside.
If it's not, decryption will fail anyway.
close#2338
Concat error messages when receiving new ndns.
This PR adds a newline followed by the new NDN error to the error text.
Maybe we should use something more prominent like
```
-----------------------------------------------------------------------
```
or more newlines, but I'm not sure. This maybe has to be tested on a
real device to see what works best.
3f9242a saves name from all QR codes to `name` (i.e. manually edited name), but for SecureJoin QR
codes the name should be saved to `authname` because such QR codes are generated by the
inviter. Other QR codes may be generated locally and not only by Delta Chat, so the name from them
mustn't go to `authname` and be revealed to the network or other contacts.
If the number of retries for message is exceeded,
do not fail when marking it as failed if the message does not exist.
Otherwise we may never delete the message from SMTP queue
because corresponding msg_id is not valid anymore.
The server should decode the URL and according to RFC 3986
query parameters may or may not be URL-encoded,
but at some servers don't decode the dot correctly.
`@` is decoded correctly by autoconfig.murena.io
This may indicate that there was a new \Seen flag
that we don't want to skip.
Also don't drain unsolicited responses while scanning folders. Now we
only drain unsolicited responses right before IDLE and always redo the
whole fetch cycle if there have been some. Some message in the scanned
folder may not be fetched that would be previously fetched otherwise,
but it will be picked up on the next folder scan.
Just a small refactoring. Instead of rebinding res all the time just use
`and` and `and_then`how they are inteded to be used. Improves code
readability imo.
This adds a function to `Message`:
```rust
pub fn new_text(text: String) -> Self {
Message {
viewtype: Viewtype::Text,
text,
..Default::default()
}
}
```
I keep expecting that a function like this must exist and being
surprised that it doesn't.
Open question is whether it should be `pub` or `pub(crate)` - I made it
`pub` for now because it may be useful for others and we currently we
aren't thinking about the Rust API that much, anyway, but I can make it
`pub(crate)`, too (then it can't be used in deltachat-jsonrpc and
deltachat-repl).
I replaced some usages of Message::new(Viewtype::Text), but not all yet,
I'm going to do this in a follow-up, which will remove another around 65
LOC.
Right now, when there is an SMTP connection error, the connectivity view
will always show "Error: SMTP connection failure: SMTP failed to
connect".
Instead, I just used the same method that is used in imap connect()
already.
0a63083df7 (fix: Shorten message text in locally sent messages too)
sets `msgs.mime_modified` for long outgoing messages, but forgets to save full message text.
I.e. add the "Messages are guaranteed to be end-to-end encrypted from now on." message and mark the
chat as protected again because no user action is required in this case. There are a couple of
problems though:
- If the program crashes earlier than the protection is restored, the chat remains
protection-broken. But this problem already exists because `ChatId::set_protection()` is never
retried.
- If multiple old unverified messages are received, protection messages added in between don't
annihilate, so they clutter the chat.
Let's always set `Config::NotifyAboutWrongPw` before saving configuration, better if a wrong
password notification is shown once more than not shown at all. It shouldn't be a big problem
because reconfiguration is a manual action and isn't done frequently.
Also for the same reason reset `Config::NotifyAboutWrongPw` only after a successful addition of the
appropriate device message.
Otherwise it always fails with SQLITE_READONLY:
```
WARNING src/sql.rs:769: Failed to run incremental vacuum: attempt to write a readonly database: Error code 8: Attempt to write a readonly database.
```
this PR adds a function that can be used to create any QR code, in a raw
form.
this can be used to create add-contact as well as add-second-device QR
codes (eg. `dc_create_qr_svg(dc_get_securejoin_qr())`) - as well as for
other QR codes as proxies.
the disadvantage of the rich-formatted QR codes as created by
`dc_get_securejoin_qr_svg()` and `dc_backup_provider_get_qr_svg()` were:
- they do not look good and cannot interact with UI layout wise (but
also tapping eg. an address is not easily possible)
- esp. text really looks bad. even with
[some](e5dc8fe3d8)
[hacks](https://github.com/deltachat/deltachat-android/pull/2215) it
[stays buggy](https://github.com/deltachat/deltachat-ios/issues/2200);
the bugs mainly come from different SVG implementation, all need their
own quirks
- accessibility is probably bad as well
we thought that time, SVG is a great thing for QR codes, but apart from
basic geometrics, it is not.
so, we avoid text, this also means to avoid putting an avatar in the
middle of the QR code (we can put some generic symbol there, eg.
different ones for add-contact and add-second-device).
while this looks like a degradation, also other messengers use more raw
QR codes. also, we removed many data from the QR code anyway, eg. the
email address is no longer there. that time, sharing QR images was more
a thing, meanwhile we have invite links, that are much better for that
purpose.
in theory, we could also leave the SVG path completely and go for PNG -
which we did not that time as PNG and text looks bad, as the system font
is not easily usable :) but going for PNG would add further challenges
as passing binary data around, and also UI-implemtation-wise, that would
be a larger step. so, let's stay with SVG in a form we know is
compatible.
the old QR code functions are deprecated.
this PR allows setting a "private tag" for a profile, see
https://github.com/deltachat/deltachat-android/pull/3373 for a possible
UI.
currently, the core does not do anything with the tag (so, it could also
be a ui.-config option), however, this may change in the future - it
might bet synced, and become also otherwise useful in core. also, having
this in core is better documentation-wise, as otherwise each UI easily
does its own things :)
This adds "stale-while-revalidate" in-memory cache for DNS. Instead of
calling `tokio::net::lookup_host` we use previous result of
`tokio::net::lookup_host` immediately and spawn revalidation task in the
background. This way all lookups after the first successful one return
immediately.
Most of the time results returned by resolvers are the same anyway, but
with this cache we avoid waiting 60 second timeout if DNS request is
lost. Common reason result may be different is round-robin DNS load
balancing and switching from IPv4 to IPv6 network. For round-robin DNS
we don't break load balancing but simply use a different result, and for
IPv6 we anyway likely have a result in persistent cache and can use IPv4
otherwise.
Especially frequent should be the case when you send a message over SMTP
and SMTP connection is stale (older than 60 s), so we open a new one.
With this change new connection will be set up faster as you don't need
to wait for DNS resolution, so message will be sent faster.
rust-analyzer was showing warnings here because it is always also
building in the Test configuration, and EventType has a
```rust
#[cfg(test)]
Test,
```
variant, which was not matched.
On main, when running `cargo build`, the following warning is emitted:
> warning:
/home/jonathan/deltachat-android/jni/deltachat-core-rust/deltachat-ffi/Cargo.toml:
`default-features` is ignored for deltachat, since `default-features`
was not specified for `workspace.dependencies.deltachat`, this could
become a hard error in the future
This is because when referring to a workspace dependency, it's not
possible to remove features, it's only possible to add features, so that
the `vendored` feature was always enabled with no possibility to disable
it.
This PR restores the wanted behavior of enabling vendoring by default
with the possibility to disable it with "default-features = false".
It fixes `nix build .#python-docs` by not passing
`--no-default-features` when building deltachat with nix.
If a message from an old contact's setup is received, the outdated Autocrypt header isn't applied,
so the contact verification preserves. But the chat protection breaks because the old message is
sorted to the bottom as it mustn't be sorted over the protection info message (which is `InNoticed`
moreover). Would be nice to preserve the chat protection too e.g. add a "protection broken" message,
then the old message and then a new "protection enabled" message, but let's record the current
behaviour first.
To avoid reordering, wait for "member removed" message
to be received before sending "member added".
The test failed at least once
because email server may reorder the messages internally
while delivering.
Otherwise instead of "old address"
ac2 may receive "member added",
resulting in this failure:
```
> assert msg_in_1.text == msg_out.text
E AssertionError: assert 'Member Me (c...hat.computer.' == 'old address'
E - old address
E + Member Me (ci-hfpxxe@***) added by ci-8e7mkr@***.
```
This ensures we do not get stuck trying DNS resolver results
when we have a known to work IP address in the cache
and DNS resolver returns garbage
either because it is a captive portal
or if it maliciously wants to get us stuck
trying a long list of unresponsive IP addresses.
This also limits the number of results we try to 10 overall.
If there are more results, we will retry later
with new resolution results.
It is impossible to set no display name anyway
in Delta Chat Android at least
because we don't want email addresses
in the UI.
This test does not work with long domains
that may get wrapped, so better remove it
instead of trying to prevent wrapping of domains.
Received messages shouldn't mingle with just sent ones and appear somewhere in the middle of the
chat, so we go after the newest non fresh message.
But if a received outgoing message is older than some `InSeen` message, better sort the received
message purely by timestamp (this is an heuristic in order not to break the Gmail-like case
simulated by `verified_chats::test_old_message_4()`). We could place the received message just
before that `InSeen` message, but anyway the user may not notice it.
At least this fixes outgoing messages sorting for shared accounts where messages from other devices
should be sorted the same way as incoming ones.
Otherwise backups exported from the current core and imported in versions < 1.144.0 have QR codes
not working. The breaking change which removed the column is
5a6efdff44.
Before file extensions were also limited to 32 chars, but extra chars in the beginning were just cut
off, e.g. "file.with_lots_of_characters_behind_point_and_double_ending.tar.gz" was considered to
have an extension "d_point_and_double_ending.tar.gz". Better to take only "tar.gz" then.
Also don't include whitespace-containing parts in extensions. File extensions generally don't
contain whitespaces.
There is already code below that emits
progress 0 or 1000 depending on whether
configuration succeeded or failed.
Before this change cancelling resulted
in progress 0 emitted,
immediately followed by progress 1000.
Before this change progress bar only started
when database is already transferred.
Database is usually the largest file
in the whole transfer, so the transfer appears
to be stuck for the sender.
With this change progress bar
starts for backup export
as soon as connection is received
and counts bytes transferred over the connection
using AsyncWrite wrapper.
Similarly for backup import,
AsyncRead wrapper counts the bytes
received and emits progress events.
Instead of treating NULL type error
as absence of the row,
handle NULL values with SQL.
Previously we sometimes
accidentally treated a single column
being NULL as the lack of the whole row.
The "Cannot establish guaranteed end-to-end encryption with ..." info
message can have lots of causes, and it happened twice to us now that it
took us some time to figure out which one it is.
So, include some more detail in the info message by simply adding the
non-translated error message in parantheses.
If we want to put in some more effort for nicer error messages, we
could:
- Introduce one new translated string "Cannot establish guaranteed
end-to-end encryption with …. Cause: %2$s" or similar (and remove the
old stock string)
- And/Or: Introduce new translated strings for all the possible errors
- And/Or: Maybe reword it in order to account better for the case that
the chat already is marked as g-e2ee, or use a different wording
(because if the chat is marked as g-e2ee then it might be nice to notify
the user that something may have gone wrong, but it's still working,
just that maybe the other side doesn't have us verified now)
Info messages are added
at the beginning of unpromoted group chats
("Others will only see this group after you sent a first message."),
may be created by WebXDC etc.
They are not sent outside
and have local Message-ID that
is not known to other recipients
so they should be skipped when constructing
In-Reply-To and References.
Without this fix IMAP loop may get stuck
trying to download non-existing message over and over
like this:
```
src/imap.rs:372: Logging into IMAP server with LOGIN.
src/imap.rs:388: Successfully logged into IMAP server
src/scheduler.rs:361: Failed to download message Msg#3467: Message Msg#3467 does not exist.
src/scheduler.rs:418: Failed fetch_idle: Failed to download messages: Message Msg#3467 does not exist
```
The whole download operation fails
due to attempt to set the state of non-existing message
to "failed". Now download of the message
will "succeed" if the message does not exist
and we don't try to set its state.
HTTPS requests are used to fetch
remote images in HTML emails,
to fetch autoconfig XML,
to POST requests for `DCACCOUNT:` QR codes
to make OAuth 2 API requests
and to connect to HTTPS proxies.
Rustls is more aggressive than OpenSSL
in deprecating cryptographic algorithms
so we cannot use it for IMAP and SMTP
to avoid breaking compatibility,
but for HTTPS requests listed
above this should not result in problems.
As HTTPS requests use only strict TLS checks,
there is no `strict_tls` argument
in `wrap_rustls` function.
Rustls is already used by iroh,
so this change does not introduce new dependencies.
This should fix ad-hoc groups splitting when messages are fetched out of order from different
folders or otherwise reordered, or some messages are missing so that the messages reference chain is
broken, or a member was removed from the thread and readded later, etc. Even if this way two
different threads are merged, it looks acceptable, having many threads with the same name/subject
and members isn't a common use case.
Instead of generating 72 random bits
and reducing them to 66 bits of Base64 characters,
generate 144 bits (18 bytes)
which is exactly 24 Base64 characters.
This should still be accepted by existing
Delta Chat clients which expect group ID
to be between 11 and 32 characters.
Message-ID creation is also simplified
to not have `Mr.` prefix
and dot in between two IDs.
Now it is a single ID followed by `@localhost`.
Some outdated documentation comments
are removed, e.g. group messages
don't start with `Gr.` already.
The greeting is now always read manually,
even for STARTTLS connections,
so the errors returned on failure to read form the stream
are the same regardless of the connection type.
Why:
- With IMAP APPEND we can upload messages directly to the DeltaChat folder (for non-chatmail
accounts).
- We can set the `\Seen` flag immediately so that if the user has other MUA, it doesn't alert about
a new message if it's just a sync message (there were several such reports on the support
forum). Though this also isn't useful for chatmail.
- We don't need SMTP envelope and overall remove some overhead on processing sync messages.
If a displayname equals to the address, adding it looks excessive.
Moreover, it's not useful for Delta Chat receiving the message because
`sanitize_name_and_addr()` removes such a displayname anyway. Also now
at least DC Android requires specifying profile name, so there should be
a fallback for users having meaningful addresses to keep the old
behaviour when Core generates `From` w/o the profile name, and this
question has already appeared on the forum.
This reverts commit 1caf672904.
Otherwise public key signature is regenerated each time the key is
loaded and test `key::tests::test_load_self_existing` which loads the
key twice fails when two loads happen on different seconds.
Closes#5976
Groups promotion to other devices and QR code tokens synchronisation are not synchronised processes,
so there are reasons why a QR code token may arrive earlier than the first group message:
- We are going to upload sync messages via IMAP while group messages are sent by SMTP.
- If sync messages go to the mvbox, they can be fetched earlier than group messages from Inbox.
`chat::create_send_msg_jobs()` already handles `Config::BccSelf` as needed. The only exception is
Autocrypt setup messages. This change unifies the logic for the self-chat and groups only containing
`SELF`.
This makes possible to schedule one more sending of the message, the existing jobs are not
cancelled. Otherwise it's complicated to implement bots that resend messages when a new member joins
the group.
This change removes OAuth2 for Gmail
as Delta Chat does not have a working
client ID anymore.
Tests are adjusted to test against Yandex
and MX queries for OAuth2 are always disabled
because they were only used to detect Google Workspace.
This change introduces new config options
`proxy_enabled` and `proxy_url`
that replace `socks5_*`.
Tested with deltachat-repl
by starting it with
`cargo run --locked -p deltachat-repl -- deltachat-db` and running
```
> set proxy_enabled 1
> set proxy_url ss://...
> setqr dcaccount:https://chatmail.example.org/new
> configure
```
And also:
- Make it `pub(crate)`.
- Use it in `should_request_mdns()` as using `config_exists()` there isn't correct because the
latter doesn't look at environment.
I.e. treat `DeleteServerAfter == None` as "delete at once". But when a backup is exported, set
`DeleteServerAfter` to 0 so that the server decides when to delete messages, in order not to break
the multi-device case. Even if a backup is not aimed for deploying more devices, `DeleteServerAfter`
must be set to 0, otherwise the backup is half-useful because after a restoration the user wouldn't
see new messages deleted by the device after the backup was done. But if the user explicitly set
`DeleteServerAfter`, don't change it when exporting a backup. Anyway even for non-chatmail case the
app should warn the user before a backup export if they have `DeleteServerAfter` enabled.
Also do the same after a backup import. While this isn't reliable as we can crash in between, this
is a problem only for old backups, new backups already have `DeleteServerAfter` set if necessary.
---------
Co-authored-by: Hocuri <hocuri@gmx.de>
There are providers in the provider database
that do not have servers specified.
For such providers default list should be tried
just like when configuring unknown providers.
Documentation comment says forward and backward verification is set,
but the code was not doing it.
`vc-contact-confirm` and `vg-member-added` messages
indicate that other device finished securejoin protocol
so we know Bob has our key marked as verified.
bcc_self has been enabled by default
since core version 1.95.0
by merging
PR <https://github.com/deltachat/deltachat-core-rust/pull/3612>.
However deltachat.h documentation
still incorrectly said that bcc_self is disabled by default.
This change replaces
usage of `reqwest` and `hyper-util`
with custom connection establishment code
so it is done in the same way
as for IMAP and SMTP connections.
This way we control HTTP, IMAP and SMTP
connection establishment
and schedule connection attempts
to resolved IP addresses
in the same way for all 3 protocols.
With current implementation
every time connection fails
we take the next delay from `delays` iterator.
In the worst case first 4 DNS results
immediately refuse connection
and we start fifth connection attempt
with 1 year timeout,
effectively continuing all remaining
connection attempts without concurrency.
With new implementation
new connection attempts are
added to `connection_attempt_set`
independently of connection failures
and after 10 seconds
we always end up with five
parallel connection attempts
as long as there are enough IP addresses.
Previously for each connection candidate (essentially host and port
pair) after resolving the host to a list of IPs Delta Chat iterated IP
addresses one by one. Now for IMAP and SMTP we try up to 5 IP addresses
in parallel. We start with one connection and add more connections
later. If some connection fails, e.g. we try to connect to IPv6 on IPv4
network and get "Network is unreachable" (ENETUNREACH) error, we replace
failed connection with another one immediately.
Co-authored-by: Hocuri <hocuri@gmx.de>
This ensures we don't add multiple Auto-Submitted headers
when bots send vg-request or vc-request messages.
The change fixes failing
receive_imf::tests::test_bot_accepts_another_group_after_qr_scan
test.
This fixes the bug that sometimes made QR scans fail.
The problem was:
When sorting headers into unprotected/hidden/protected, the From: header
was added twice for all messages: Once into unprotected_headers and once
into protected_headers. For messages that are `is_encrypted && verified
|| is_securejoin_message`, the display name is removed before pushing it
into unprotected_headers.
Later, duplicate headers are removed from unprotected_headers right
before prepending unprotected_headers to the message. But since the
unencrypted From: header got modified a bit when removing the display
name, it's not exactly the same anymore, so it's not removed from
unprotected_headers and consequently added again.
I.e. to all messages except "v{c,g}-request" as they sent out on a QR code scanning which is a
manual action and "vg-member-added" as formally this message is auto-submitted, but the member
addition is a result of an explicit user action. Otherwise it would be strange to have the
Auto-Submitted header in "member-added" messages of verified groups only.
Before the fix HTTP client
had no connection timeout,
so it only had a chance
to test one IPv6 and one IPv4
address if the first addresses timed out.
Now it can test at least 4 addresses
of each family and more if some addresses
refuse connection rather than time out.
Otherwise if DNS server returns incorrect results,
we may never try preloaded DNS results.
For example, we may get our first results
from a captive portal.
To test, add `127.0.0.1 example.org`
and try to create an account.
Without this change we only try 127.0.0.1 and fail.
With this change preloaded DNS results are tried as well.
this PR adds the type DC_QR_SOCKS5_PROXY to `dc_check_qr()` for
**supporting telegram proxy QR codes**. if returned, the UI should ask
the user if they want to us the proxy and call
`dc_set_config_from_qr();` afterwards (plus maybe `dc_configure()`).
idea is to improve our proxy story, follow ups may be:
- in UI, - move proxy out of "Account & Password", as a **separate
"Proxy Activity"** (it should stay in "Advanced" for now, however, below
"Server", which might be moved up)
- allow **opening the "Proxy Activity" from the welcome screens**
three-dot-menu (that would also solve a long standing issue that
entering the email address bypasses the proxy
- show proxy usage in the "Connectivity View" and/or add an **icon** to
the main chatlist screen (beside three-dot menu) in case some proxy is
in use; tapping this icon will open the "Proxy Activity"
- the the new "Proxy Activity", add a **share / show proxy QR code**
button. that would generate invite links in the form
`https://i.delta.chat/socks#...` - so that tapping then opens the app.
support for these links need to be added to core then.
- handle a list of proxies in core, offer selection in UI. the list
could be one for all profiles and could be filled eg. by normal invite
links or other channels
---------
Co-authored-by: iequidoo <117991069+iequidoo@users.noreply.github.com>
Fix a typo in the config name
(by using `Config::` to avoid it)
and make sure we don't panic on unknown values.
Also test that we don't panic on unknown
`configured_imap_certificate_checks` values.
Apparently some providers fail TLS connection
with "no_application_protocol" alert
even when requesting "imap" protocol for IMAP connection
and "smtp" protocol for SMTP connection.
Fixes <https://github.com/deltachat/deltachat-core-rust/issues/5892>.
configured_imap_certificate_checks=0 means
accept invalid certificates unless provider database
says otherwise or SOCKS5 is enabled.
It should not be saved into the database anymore.
This bug was introduced in
<https://github.com/deltachat/deltachat-core-rust/pull/5854>
(commit 6b4532a08e)
and affects released core 1.142.4, 1.142.5 and 1.142.6.
Fix reverts faulty fix from
<https://github.com/deltachat/deltachat-core-rust/pull/5886>
(commit a268946f8d)
which changed the way configured_imap_certificate_checks=0
is interpreted and introduced problems
for existing setups with configured_imap_certificate_checks=0:
<https://github.com/deltachat/deltachat-core-rust/issues/5889>.
Existing test from previous fix is not reverted
and still applies.
Regression test is added to check that
configured_imap_certificate_checks
is not "0" for new accounts.
If user has not set any settings manually
and provider is not configured,
default to strict TLS checks.
Bug was introduced in
<https://github.com/deltachat/deltachat-core-rust/pull/5854>
(commit 6b4532a08e)
and affects released core 1.142.4 and 1.142.5.
The problem only affects accounts configured
using these core versions with provider
not in the provider database or when using advanced settings.
`imap::Session::store_seen_flags_on_imap()` handles messages from multiple folders, so not being
able to select one folder mustn't fail the whole function.
`cargo install` ignores lockfile by default.
Without lockfile current build fails
due to iroh-net 0.21.0 depending on `derive_more` 1.0.0-beta.6
but failing to compile with `derive_more` 1.0.0.-beta.7.
This particular error will be fixed by upgrading to iroh 0.22.0,
but using lockfile will avoid similar problems in the future.
- To avoid receiving undecryptable MDNs by bots and replying to them if the bot's key changes.
- MDNs from bots don't look useful in general, usually the user expects some reply from the bot, not
just that the message is read.
Recently there are many questions on the Delta Chat forum why some unexpected encrypted messages
appear in Inbox. Seems they are mainly sync messages, though that also obviously happens to
SecureJoin messages. Anyway, regardless of the `MvboxMove` setting, auto-generated outgoing messages
should be moved to the DeltaChat folder so as not to complicate co-using Delta Chat with other MUAs.
Otherwise cargo emits these warnings:
warning: .../deltachat-core-rust/deltachat-ffi/Cargo.toml: `default-features` is ignored for deltachat, since `default-features` was not specified for `workspace.dependencies.deltachat`, this could become a hard error in the future
warning: .../deltachat-core-rust/deltachat-rpc-server/Cargo.toml: `default-features` is ignored for deltachat, since `default-features` was not specified for `workspace.dependencies.deltachat`, this could become a hard error in the future
warning: .../deltachat-core-rust/deltachat-rpc-server/Cargo.toml: `default-features` is ignored for deltachat-jsonrpc, since `default-features` was not specified for `workspace.dependencies.deltachat-jsonrpc`, this could become a hard error in the future
Previously Delta Chat tried to use local part of email address as well.
This configuration is very uncommon,
but trying it doubled the time of configuration try
in the worst case, e.g. when the password is typed in incorrectly.
Single function smtp::connect::connect_stream
returns a stream of a single `dyn` type
that can be a TLS, STARTTLS or plaintext
connection established over SOCKS5 or directly.
This way we can't get an account with missing blobs if there's not enough disk space.
Also delete already unpacked files if all files weren't unpacked successfully. Still, there are some
minor problems remaining:
- If a db wasn't imported successfully, unpacked blobs aren't deleted because we don't know at which
step the import failed and whether the db will reference the blobs after restart.
- If `delete_and_reset_all_device_msgs()` fails, the whole `import_backup()` fails also, but after a
restart delete_and_reset_all_device_msgs() isn't retried. Probably errors from it should be
ignored at all.
Previously Delta Chat tried all DNS resolution results
in sequence until TCP connection is established successfully,
then tried to establish TLS on top of the TCP connection.
If establishing TLS fails, the whole
connection establishment procedure failed
without trying next DNS resolution results.
In particular, in a scenario
where DNS returns incorrect result
pointing e.g. to a server
that listens on the TCP port
but does not have correpsponding TLS certificate,
Delta Chat now will fall back to the cached result
and connect successfully.
while adapting strings for the recent change about read receipts,
https://github.com/deltachat/deltachat-core-rust/pull/5712 , it turns
out in discussions eg. at
https://github.com/deltachat/deltachat-android/issues/3179 that
untranslated english for the read receipts seem to be sufficient or even
better:
- do not reveal the sender's language
- unexpected languages are confusing - even if you chat in english, you
may get Chinese read receipts
- many clients do not show the text anyways, iirc, eg. Outlook display
the read receipts in context, and Delta Chat of course as well
- afaik, we're leaving comparable `multipart/report` untranslated as
well (sync, but also webxdc updates are practically english only)
- less code, fewer translations needed :)
There are many reasons why we may fail to find valid signatures in a message, e.g. we don't yet know
a public key attached in the same message, anyway, if From is forged, the message must be rejected.
Also always take the displayname from encrypted From, even if no valid signatures are found.
This is a hint for apps that a WebXDC icon should be shown in the summary, e.g. in the
chatlist. Otherwise it's not clear when it should be shown, e.g. it shouldn't be shown in a reaction
summary.
For hardcoded built-in DNS results
there is no cache entry in `dns_cache` table
so they cannot be prioritized if DNS resolution
never returned these results yet.
If there is no entry, a new one should be created.
SQL UPSERT does this.
Follow-up to 3cf78749df "Emit DC_EVENT_MSGS_CHANGED for
DC_CHAT_ID_ARCHIVED_LINK when the number of archived chats with unread messages increases (#3940)".
In general we don't want to make an extra db query to know if a noticied chat is
archived. Emitting events should be cheap, better to allow false-positive `MsgsChanged` events.
`Context::send_sync_msg()` mustn't be called from multiple tasks in parallel to avoid sending the
same sync items twice because sync items are removed from the db only after successful
sending. Let's guarantee this by calling `send_sync_msg()` only from the SMTP loop. Before
`send_sync_msg()` could be called in parallel from the SMTP loop and another task doing
e.g. `chat::sync()` which led to `test_multidevice_sync_chat` being flaky because of events
triggered by duplicated sync messages.
New protocol streams .tar into iroh-net
stream without traversing all the files first.
Reception over old backup protocol
is still supported to allow
transferring backups from old devices
to new ones, but not vice versa.
The "Chat-Group-Member-Removed" header is added to ad-hoc group messages as well, so we should check
for its presense before creating an ad-hoc group as we do for DC-style groups.
Before, update sending might be delayed due to rate limits and later merged into large
messages. This is undesirable for apps that want to send large files over WebXDC updates because the
message with aggregated update may be too large for actual sending and hit the provider limit or
require multiple attempts on a flaky SMTP connection.
So, don't aggregate updates if the size of an aggregated update will exceed the limit of 100
KiB. This is a soft limit, so it may be exceeded if a single update is larger and it limits only the
update JSON size, so the message with all envelopes still may be larger. Also the limit may be
exceeded when updates are sent together with the WebXDC instance when resending it as the instance
size isn't accounted to not complicate the code. At least this is not worse than the previous
behaviour when all updates were attached.
Bridge bots like matterdelta need to set a quoted text without referencing the quoted message, this
makes easier bridging messages from other platforms to Delta Chat or even bridging Delta Chat groups
in different accounts where you can not set a quoted message by the message id from another account.
Even though SMTP ALPN is not officially registered (unlike IMAP),
it is an obvious choice that will allow
to multiplex SMTP and other protocols on the same TLS port.
Newer node-gyp uses newer gyp
which requires newer python
that is not available in Debian 10
container that is
used for building node prebuilds
with old glibc.
Follow-up to 5fa7cff46. Let's still not send a sync message if the contact wasn't modified. This is
not very important, but just for consistency with the `chat::rename_ex()` behaviour.
This way it's clearer which key is which and also adding the key fingerprint to the file name avoids
overwriting another previously exported key. I think this is better than adding an incremental
number as we do for backups, there's no need to export a key several times to different files.
For purposes of building a message it's better to consider the self-chat as verified. Particularly,
this removes unencrypted name from the "From" header.
If currently there are no multi-device bots, let's disable sync messages for bots at all. Another
option is to auto-disable sync messages when `Config::Bot` is set, so sync messages can be reenabled
if needed. But let's leave this option for the future.
About the first TODO: I tried this out, but it didn't actually improve
things, for two reasons:
1. The trick with `#![cfg_attr(not(test),
warn(clippy::indexing_slicing))]` that enables the lint everywhere
except for tests doesn't work with workspace-wide lints. (Context: We
want to lint against indexing because it might panic, but in a test
panicking is fine, so we don't want to enable the lint in tests).
2. Most of our crates have different sets of lints right now, so it
would only be very few crates that use the workspace-wide list of lints.
About the second TODO:
It's not feasible right now to fully parse vCards, and for our
good-enough parser the current behavior is fine, I think. If we fail to
parse some realworld vCards because of this, we can still improve it.
Notification server uses APNS server
for heartbeat notifications,
so registering FCM tokens there
will result in failing to notify them
and unregistering them anyway.
It's possible that when rebasing a PR adding a migration a merge-conflict doesn't occur if another
migration was added in the target branch. Better to have at least runtime checks that the migration
version is correct. Looks like compile-time checks are not possible because Rust doesn't allow to
redefine constants, only vars.
Instead of constructing lists of protected,
unprotected and hidden headers,
construct a single list of headers
and then sort them into separate lists
based on the well-defined policy.
This also fixes the bug
where Subject was not present in the IMF header
for signed-only messages.
Closes#5713
Bot processes are run asynchronously, so we shouldn't send messages to a bot before it's fully
initialised and skipped existing messages for processing, i.e. before DC_EVENT_IMAP_INBOX_IDLE is
emitted.
we're always checking the configuration encrypted.
saying it is 'preferred' encrypted is misleading,
therfore, just remove it.
i do not think, it is worth saying that we do not query 'http',
this is clear from the source code.
moreover, fix two typos.
Looks like it has no sense to send any profile data (From/To names, self-status; self-avatar was
never sent even before) in MDNs, they aren't normal messages and aren't seen in a MUA. Better not to
reveal profile data to the network and even to contacts in MDNs and make them more lightweight.
Follow-up to b771311593. Since that commit names are not revealed in
verified chats, but during verification (i.e. SecureJoin) they are still sent unencrypted. Moreover,
all profile data mustn't be sent even encrypted before the contact verification, i.e. before
"v{c,g}-request-with-auth". That was done for the selfavatar in
304e902fce, now it's done for From/To names and the self-status as
well. Moreover, "v{c,g}-request" and "v{c,g}-auth-required" messages are deleted right after
processing, so other devices won't see the received profile data anyway.
Probably sync messages generated for a not yet configured account are useless because there are no
other devices yet. And even if this is not true, we don't want to depend on the order of setting
`Config::SyncMsgs` and other keys. Also w/o this the Python tests don't work if we start syncing
`Config::MvboxMove` because they don't expect that sync messages are sent while configuring
accounts.
NB: We don't restart IO from the synchronisation code, so `MvboxMove` isn't effective immediately if
`ConfiguredMvboxFolder` is unset, but only after a reconnect to IMAP.
Before, if the user deleted a message too quickly after sending, it was deleted only locally. The
fix is to remember for tombstones that the corresponding message should be deleted on the server
too.
This is a way to prevent redownloading locally deleted messages. Otherwise if a message is deleted
quickly after sending and `bcc_self` is configured, the BCC copy is downloaded and appears as a new
message as it happens for messages sent from another device.
Delta Chat -style groups have names w/o prefixes like "Re: " even if the user is added to an already
existing group, so let's remove prefixes from ad-hoc group names too. Usually it's not very
important that the group is a classic email thread existed before, this info just eats up screen
space. Also this way a group name is likely to preserve if the first message was missed.
SQLite search with `LIKE` is case-insensitive only for ASCII chars. To make it case-insensitive for
all messages, create a new column `msgs.txt_normalized` defaulting to `NULL` (so we do not bump up
the database size in a migration) and storing lowercased/normalized text there when the row is
created/updated. When doing a search, search over `IFNULL(txt_normalized, txt)`.
Before, if `Config::FetchExistingMsgs` is set, existing messages were received with the `InSeen`
state set, but for bots they must be `InFresh` and also `IncomingMsg` events should be emitted for
them so that they are processed by bots as it happens with new messages.
Since commit c0a17df344
(PR https://github.com/deltachat/deltachat-core-rust/pull/3402)
`send_smtp_messages` returns an error
as soon as it encounters the first message it failed to send.
Since this worked like this for about 2 years
without any problems, there is no need to revert the change,
but outdated comment should be removed.
If a display name should be protected (i.e. opportunistically encrypted), only put the corresponding
address to the unprotected headers. We protect the From display name only for verified chats,
otherwise this would be incompatible with Thunderbird and K-9 who don't use display names from the
encrypted part. Still, we always protect To display names as compatibility seems less critical here.
When receiving a messge, overwrite the From display name but not the whole From field as that would
allow From forgery. For the To field we don't really care. Anyway as soon as we receive a message
from the user, the display name will be corrected.
Co-authored-by: iequidoo <dgreshilov@gmail.com>
Otherwise it's impossible to remove a member with missing key from a protected group. In the worst
case a removed member will be added back due to the group membership consistency algo.
The "I left the group" message can't be sent to a protected group if some member's key is missing,
in this case we should remain in the group. The problem should be fixed first, then the user may
retry to leave the group.
There was a comment in `fetch_existing_msgs()`: "Bots don't want those messages". If a bot doesn't
want this setting, why enable it? It's disabled by default anyway.
`IsChatmail` is set also by `inbox_fetch_idle()`, but it isn't called during `configure()`. Setting
`IsChatmail` from `inbox_fetch_idle()` is necessary to handle client/server upgrades, but
`IsChatmail` also should be available for the app after configuring an account, e.g. DC Android
needs it to know whether to ask the user to disable battery optimisations.
Before this fix LogSink was dropped immediately,
resulting in no logs printed for contexts created using
TextContext::new_alice(), but printed for contexts created
using TextContext::new().
`!to_ids().is_empty()` check is needed in cases of 1:1 chat creation
because otherwise `to_id` is undefined,
but in case of outgoing group message without recipients
observed on a second device creating a group should be allowed.
Previously OpenSSL was pinned due to problems
with Zig toolchain used back then
and then not unpinned even after switching from Zig to Nix
due to <https://github.com/alexcrichton/openssl-src-rs/issues/235>.
The problem should be fixed now
and we can try to unpin OpenSSL again.
When doing an AEAP transition, we mustn't just delete the old peerstate as this would break
encryption to it. This is critical for non-verified groups -- if we can't encrypt to the old
address, we can't securely remove it from the group (to add the new one instead).
-remove `DELTA_CHAT_SKIP_PATH` environment variable
- remove version check / search for dc rpc server in $PATH
- remove `options.skipSearchInPath`
- add `options.takeVersionFromPATH`
async-broadcast returns Overflowed error once
if channel overflow happened.
Public APIs such as get_next_event JSON-RPC method
are only expecting an error if the channel is closed,
so we should not propagate overflow error outside.
In particular, Delta Chat Desktop
stop receiving events completely if an error
is returned once.
If overflow happens, we should ignore it
and try again until we get an event or an error because
the channel is closed (in case of recv())
or empty (in case of try_recv()).
If multiple headers with the same name are present,
we must take the last one.
This is the one that is DKIM-signed if
this header is DKIM-signed at all.
Ideally servers should prevent adding
more From, To and Cc headers by oversigning
them, but unfortunately it is not common
and OpenDKIM in its default configuration
does not oversign any headers.
closes#5533
adds the functions that were still missing for migration to jsonrpc (the
ones that the cffi already had, so just should be quick to review ;)
It's actually `deltachat::contact::Contact::authname` by semantics. `display_name`/`name` shouldn't
be shared, it's the name given by the user locally.
Add a function parsing a vCard file at the given path.
Co-authored-by: Hocuri <hocuri@gmx.de>
Co-authored-by: Asiel Díaz Benítez <asieldbenitez@gmail.com>
When there are no parent references,
Delta Chat inserts Message-ID into References.
Such references should be ignored
because otherwise fully downloaded message
may be assigned to the same chat as previously incorrectly assigned
partially downloaded message.
Fixes receive_imf::tests::test_create_group_with_big_msg
Chat-Group-ID always correctly identifies the chat
message was sent to, while In-Reply-To and References
may point to a message that has itself been incorrectly
assigned to a chat.
This protects Bob (the joiner) of sending unexpected-unencrypted messages during an otherwise nicely
running SecureJoin.
If things get stuck, however, we do not want to block communication -- the chat is just
opportunistic as usual, but that needs to be communicated:
1. If Bob's chat with Alice is `Unprotected` and a SecureJoin is started, then add info-message
"Establishing guaranteed end-to-end encryption, please wait..." and let `Chat::can_send()` return
`false`.
2. Once the info-message "Messages are guaranteed to be e2ee from now on" is added, let
`Chat::can_send()` return `true`.
3. If after SECUREJOIN_WAIT_TIMEOUT seconds `2.` did not happen, add another info-message "Could not
yet establish guaranteed end-to-end encryption but you may already send a message" and also let
`Chat::can_send()` return `true`.
Both `2.` and `3.` require the event `ChatModified` being sent out so that UI pick up the change wrt
`Chat::can_send()` (this is the same way how groups become updated wrt `can_send()` changes).
SECUREJOIN_WAIT_TIMEOUT should be 10-20 seconds so that we are reasonably sure that the app remains
active and receiving also on mobile devices. If the app is killed during this time then we may need
to do step 3 for any pending Bob-join chats (right now, Bob can only join one chat at a time).
`ChatId::lookup_by_contact()` returns `None` for blocked chats, so it should be only used if we need
to filter out blocked chats, e.g. when building a chatlist.
E.g. the multi-device synchronisation creates the "Saved Messages" chat as blocked, in this case the
chat icon wasn't updated before and the user avatar was displayed instead.
- [x] figgure out how to build the packages (that it installs native
optional package automatically)
- [X] Make the gluecode
- [x] expose both the lowerlevel api that desktop uses (~~send objects
and receive objects~~, getting path of rpc-server is enough)
- [X] and the higher level api needed for bots (jsonrpc client)
- [X] typescript types
- [x] automatically pick the right binary from npm or allow getting it
from env var, or give out an error (throw error)
- [x] find out how to dev locally (use local built core in dc desktop) -
there is the question of how to link the typescript client and the task
to add a search in the cargo target folder for a debug build or a
different way, find out some good flow that we can use and document for
dc desktop + locally built core development
- [x] build the packages in ci
- [x] fix that deltachat-rpc-server is not executable
postponed:
- [ ] publish from ci
- [ ] add key/token to deploy to npm
Closes#4694
## Related prs
- https://github.com/deltachat-bot/echo/pull/69
- https://github.com/deltachat/deltachat-desktop/pull/3567
---------
Co-authored-by: link2xt <link2xt@testrun.org>
this PR checks if the quotes are used in a reasonable way:
- quoted messages should be send to the same chat
- or to one-to-one chats
if the quote's chat ID is not the same than the sending chat _and_ the
sending chat is not a one-to-one chat, sending is aborted.
usually, the UIs does not allow quoting in other ways, so, this check is
only a "last defence line" to avoid leaking data in case the UI has
bugs, as recently in
https://github.com/deltachat/deltachat-android/issues/3032
@iequidoo @link2xt @adbenitez i am not 100% sure about this PR, maybe
i've overseen a reasonable usecase where other quotes make sense
---------
Co-authored-by: link2xt <link2xt@testrun.org>
Otherwise, e.g. if a message is a large GIF, but its viewtype is set to `Image` by the app, this GIF
will be recoded to JPEG to reduce its size. GIFs and other special viewtypes must be always detected
and sent as is.
Otherwise location-only messages
that should be sent every 60 seconds
are never sent because location loop
waits until the end of location streaming
and is only interrupted by location streaming
ending in other chats or being enabled in other chats.
This is similar to `test_modify_chat_disordered`,
but tests that recovery works in the simplest case
where next message is not modifying group membership.
2024-04-27 07:51:08 +00:00
245 changed files with 29012 additions and 11428 deletions
If you have a feature request, create a new topic on the [forum](https://support.delta.chat/).
## Contributing code
## Code contributions
If you want to contribute a code, [open a Pull Request](https://github.com/deltachat/deltachat-core-rust/pulls).
If you want to contribute a code, follow this guide.
If you have write access to the repository,
push a branch named `<username>/<feature>`
so it is clear who is responsible for the branch,
and open a PR proposing to merge the change.
Otherwise fork the repository and create a branch in your fork.
1.**Select an issue to work on.**
If you have an write access to the repository, assign the issue to yourself.
Otherwise state in the comment that you are going to work on the issue
to avoid duplicate work.
If the issue does not exist yet, create it first.
2.**Write the code.**
Follow the [coding conventions](STYLE.md) when writing the code.
3.**Commit the code.**
If you have write access to the repository,
push a branch named `<username>/<feature>`
so it is clear who is responsible for the branch,
and open a PR proposing to merge the change.
Otherwise fork the repository and create a branch in your fork.
Commit messages follow the [Conventional Commits] notation.
We use [git-cliff] to generate the changelog from commit messages before the release.
With **`git cliff --unreleased`**, you can check how the changelog entry for your commit will look.
The following prefix types are used:
-`feat`: Features, e.g. "feat: Pause IO for BackupProvider". If you are unsure what's the category of your commit, you can often just use `feat`.
-`fix`: Bug fixes, e.g. "fix: delete `smtp` rows when message sending is cancelled"
-`api`: API changes, e.g. "api(rust): add `get_msg_read_receipts(context, msg_id)`"
-`refactor`: Refactorings, e.g. "refactor: iterate over `msg_ids` without `.iter()`"
-`perf`: Performance improvements, e.g. "perf: improve SQLite performance with `PRAGMA synchronous=normal`"
-`test`: Test changes and improvements to the testing framework.
-`build`: Build system and tool configuration changes, e.g. "build(git-cliff): put "ci" commits into "CI" section of changelog"
-`ci`: CI configuration changes, e.g. "ci: limit artifact retention time for `libdeltachat.a` to 1 day"
-`docs`: Documentation changes, e.g. "docs: add contributing guidelines"
-`chore`: miscellaneous tasks, e.g. "chore: add `.DS_Store` to `.gitignore`"
Release preparation commits are marked as "chore(release): prepare for X.Y.Z"
as described in [releasing guide](RELEASE.md).
Use a `!` to mark breaking changes, e.g. "api!: Remove `dc_chat_can_send`".
Alternatively, breaking changes can go into the commit description, e.g.:
```
fix: Fix race condition and db corruption when a message was received during backup
BREAKING CHANGE: You have to call `dc_stop_io()`/`dc_start_io()` before/after `dc_imex(DC_IMEX_EXPORT_BACKUP)`
```
4. [**Open a Pull Request**](https://github.com/deltachat/deltachat-core-rust/pulls).
Refer to the corresponding issue.
If you intend to squash merge the PR from the web interface,
make sure the PR title follows the conventional commits notation
as it will end up being a commit title.
Otherwise make sure each commit title follows the conventional commit notation.
5.**Make sure all CI checks succeed.**
CI runs the tests and checks code formatting.
While it is running, self-review your PR to make sure all the changes you expect are there
and there are no accidentally committed unrelated changes and files.
Push the necessary fixup commits or force-push to your branch if needed.
6.**Ask for review.**
Use built-in GitHub feature to request a review from suggested reviewers.
If you do not have write access to the repository, ask for review in the comments.
7.**Merge the PR.**
Once a PR has an approval and passes CI, it can be merged.
PRs from a branch created in the main repository,
i.e. authored by those who have write access, are merged by their authors.
This is to ensure that PRs are merged as intended by the author,
e.g. as a squash merge, by rebasing from the web interface or manually from the command line.
If you have multiple changes in one PR, do a rebase merge.
Otherwise, you should usually do a squash merge.
If PR author does not have write access to the repository,
maintainers who reviewed the PR can merge it.
If you do not have access to the repository and created a PR from a fork,
ask the maintainers to merge the PR and say how it should be merged.
## Other ways to contribute
For other ways to contribute, refer to the [website](https://delta.chat/en/contribute).
You can find the list of good first issues
and a link to this guide
on the contributing page: <https://github.com/deltachat/deltachat-core-rust/contribute>
### Coding conventions
We format the code using `rustfmt`. Run `cargo fmt` prior to committing the code.
Run `scripts/clippy.sh` to check the code for common mistakes with [Clippy].
Commit messages follow the [Conventional Commits] notation.
We use [git-cliff] to generate the changelog from commit messages before the release.
With **`git cliff --unreleased`**, you can check how the changelog entry for your commit will look.
The following prefix types are used:
-`feat`: Features, e.g. "feat: Pause IO for BackupProvider". If you are unsure what's the category of your commit, you can often just use `feat`.
-`fix`: Bug fixes, e.g. "fix: delete `smtp` rows when message sending is cancelled"
-`api`: API changes, e.g. "api(rust): add `get_msg_read_receipts(context, msg_id)`"
-`refactor`: Refactorings, e.g. "refactor: iterate over `msg_ids` without `.iter()`"
-`perf`: Performance improvements, e.g. "perf: improve SQLite performance with `PRAGMA synchronous=normal`"
-`test`: Test changes and improvements to the testing framework.
-`build`: Build system and tool configuration changes, e.g. "build(git-cliff): put "ci" commits into "CI" section of changelog"
-`ci`: CI configuration changes, e.g. "ci: limit artifact retention time for `libdeltachat.a` to 1 day"
-`docs`: Documentation changes, e.g. "docs: add contributing guidelines"
-`chore`: miscellaneous tasks, e.g. "chore: add `.DS_Store` to `.gitignore`"
Release preparation commits are marked as "chore(release): prepare for vX.Y.Z".
If you intend to squash merge the PR from the web interface,
make sure the PR title follows the conventional commits notation
as it will end up being a commit title.
Otherwise make sure each commit title follows the conventional commit notation.
#### Breaking Changes
Use a `!` to mark breaking changes, e.g. "api!: Remove `dc_chat_can_send`".
Alternatively, breaking changes can go into the commit description, e.g.:
```
fix: Fix race condition and db corruption when a message was received during backup
BREAKING CHANGE: You have to call `dc_stop_io()`/`dc_start_io()` before/after `dc_imex(DC_IMEX_EXPORT_BACKUP)`
```
#### Multiple Changes in one PR
If you have multiple changes in one PR, create multiple conventional commits, and then do a rebase merge. Otherwise, you should usually do a squash merge.
- over jsonrpc built with napi.rs (experimental): \[[📂 source](https://github.com/deltachat/napi-jsonrpc) | [📦 npm](https://www.npmjs.com/package/@deltachat/napi-jsonrpc)\]
We format the code using `rustfmt`. Run `cargo fmt` prior to committing the code.
Run `scripts/clippy.sh` to check the code for common mistakes with [Clippy].
[Clippy]: https://doc.rust-lang.org/clippy/
## SQL
Multi-line SQL statements should be formatted using string literals,
for example
```
sql.execute(
"CREATE TABLE messages (
id INTEGER PRIMARY KEY AUTOINCREMENT,
text TEXT DEFAULT '' NOT NULL -- message text
) STRICT",
)
.await?;
```
Do not use macros like [`concat!`](https://doc.rust-lang.org/std/macro.concat.html)
or [`indoc!](https://docs.rs/indoc).
Do not escape newlines like this:
```
sql.execute(
"CREATE TABLE messages ( \
id INTEGER PRIMARY KEY AUTOINCREMENT, \
text TEXT DEFAULT '' NOT NULL \
) STRICT",
)
.await?;
```
Escaping newlines
is prone to errors like this if space before backslash is missing:
```
"SELECT foo\
FROM bar"
```
Literal above results in `SELECT fooFROM bar` string.
This style also does not allow using `--` comments.
---
Declare new SQL tables with [`STRICT`](https://sqlite.org/stricttables.html) keyword
to make SQLite check column types.
Declare primary keys with [`AUTOINCREMENT`](https://www.sqlite.org/autoinc.html) keyword.
This avoids reuse of the row IDs and can avoid dangerous bugs
like forwarding wrong message because the message was deleted
and another message took its row ID.
Declare all new columns as `NOT NULL`
and set the `DEFAULT` value if it is optional so the column can be skipped in `INSERT` statements.
Dealing with `NULL` values both in SQL and in Rust is tricky and we try to avoid it.
If column is already declared without `NOT NULL`, use `IFNULL` function to provide default value when selecting it.
Use `HAVING COUNT(*) > 0` clause
to [prevent aggregate functions such as `MIN` and `MAX` from returning `NULL`](https://stackoverflow.com/questions/66527856/aggregate-functions-max-etc-return-null-instead-of-no-rows).
Don't delete unused columns too early, but maybe after several months/releases, unused columns are
still used by older versions, so deleting them breaks downgrading the core or importing a backup in
an older version. Also don't change the column type, consider adding a new column with another name
instead. Finally, never change column semantics, this is especially dangerous because the `STRICT`
/// or for functions that are expected to fail (eg. autocryptContinueKeyTransfer())
/// it might be better to delay showing these events until the function has really
/// failed (returned false). It should be sufficient to report only the *last* error
/// in a messasge box then.
/// in a message box then.
Error{msg: String},
/// An action cannot be performed because the user is not in the group.
@@ -98,6 +98,24 @@ pub enum EventType {
contact_id: u32,
},
/// Incoming reaction, should be notified.
#[serde(rename_all = "camelCase")]
IncomingReaction{
contact_id: u32,
msg_id: u32,
reaction: String,
},
/// Incoming webxdc info or summary update, should be notified.
#[serde(rename_all = "camelCase")]
IncomingWebxdcNotify{
chat_id: u32,
contact_id: u32,
msg_id: u32,
text: String,
href: Option<String>,
},
/// There is a fresh message. Typically, the user will show an notification
/// when receiving this message.
///
@@ -240,6 +258,15 @@ pub enum EventType {
status_update_serial: u32,
},
/// Data received over an ephemeral peer channel.
#[serde(rename_all = "camelCase")]
WebxdcRealtimeData{msg_id: u32,data: Vec<u8>},
/// Advertisement received over an ephemeral peer channel.
/// This can be used by bots to initiate peer-to-peer communication from their side.
#[serde(rename_all = "camelCase")]
WebxdcRealtimeAdvertisementReceived{msg_id: u32},
/// Inform that a message containing a webxdc instance has been deleted
#[serde(rename_all = "camelCase")]
WebxdcInstanceDeleted{msg_id: u32},
@@ -259,6 +286,23 @@ pub enum EventType {
/// If `chat_id` is set to None, then all currently visible chats need to be rerendered, and all not-visible items need to be cleared from cache if the UI has a cache.
#[serde(rename_all = "camelCase")]
ChatlistItemChanged{chat_id: Option<u32>},
/// Inform that the list of accounts has changed (an account removed or added or (not yet implemented) the account order changes)
///
/// This event is only emitted by the account manager
AccountsChanged,
/// Inform that an account property that might be shown in the account list changed, namely:
/// - is_configured (see is_configured())
/// - displayname
/// - selfavatar
/// - private_tag
///
/// This event is emitted from the account whose property changed.
AccountsItemChanged,
/// Inform than some events have been skipped due to event channel overflow.
EventChannelOverflow{n: u64},
}
implFrom<CoreEventType>forEventType{
@@ -290,6 +334,28 @@ impl From<CoreEventType> for EventType {
- Then pass it as an artifact to the last CI action that publishes the main package.
- upload all packages from `deltachat-rpc-server/npm-package/platform_package`.
- then publish `deltachat-rpc-server/npm-package`,
- this will run `update_optional_dependencies_and_version.js` (in the `prepack` script),
which puts all platform packages into `optionalDependencies` and updates the `version` in `package.json`
## How to build a version you can use locally on your host machine for development
You can not install the npm packet from the previous section locally, unless you have a local npm registry set up where you upload it too. This is why we have separate scripts for making it work for local installation.
- If you just need your host platform run `python scripts/make_local_dev_version.py`
- note: this clears the `platform_package` folder
- (advanced) If you need more than one platform for local install you can just run `node scripts/update_optional_dependencies_and_version.js` after building multiple platforms with `build_platform_package.py`
## Thanks to nlnet
The initial work on this package was funded by nlnet as part of the [Delta Tauri](https://nlnet.nl/project/DeltaTauri/) Project.
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
