The main reason for this change is the app picker
that Delta Chat clients use, which utilizes
the `fetch_url` function.
Sometimes we get an error from the server,
but we have no way to figure out that it's an error,
other than inspecting the body, which we don't (and shouldn't) do.
This results in us attempting to send webxdc apps
that are not even valid .zip files.
Another, arguably even worse thing is that
we also put the error responses to the cache,
so it's not easy to recover from such an error.
So, let's just return an error if the response code
is not a successful response code.
Four new APIs `add_transport_from_qr()`, `add_transport()`,
`list_transports()`, `delete_transport()`, as described in the draft at
"API".
The `add_tranport*()` APIs automatically stops and starts I/O; for
`configure()` the stopping and starting is done in the JsonRPC bindings,
which is not where things like this should be done I think, the bindings
should just translate the APIs.
This also completely disables AEAP for now.
I won't be available for a week, but if you want to merge this already,
feel free to just commit all review suggestions and squash-merge.
Deduplicate:
- In the REPL
- In `store_from_base64()`, which writes avatars received in headers
- In a few tests
- The saved messages, broadcast, device, archive icons
- The autocrypt setup message
1-2 more PRs, and we can get rid of `BlobObject::create`,
`sanitise_name()`, and some others
This is to avoid revalidating HTTP cache too frequently (and have many parallel revalidation tasks)
if revalidation fails or the HTTP request takes some time. The stale period >= 1 hour, so 1 more
minute won't be a problem.
This adds "stale-while-revalidate" in-memory cache for DNS. Instead of
calling `tokio::net::lookup_host` we use previous result of
`tokio::net::lookup_host` immediately and spawn revalidation task in the
background. This way all lookups after the first successful one return
immediately.
Most of the time results returned by resolvers are the same anyway, but
with this cache we avoid waiting 60 second timeout if DNS request is
lost. Common reason result may be different is round-robin DNS load
balancing and switching from IPv4 to IPv6 network. For round-robin DNS
we don't break load balancing but simply use a different result, and for
IPv6 we anyway likely have a result in persistent cache and can use IPv4
otherwise.
Especially frequent should be the case when you send a message over SMTP
and SMTP connection is stale (older than 60 s), so we open a new one.
With this change new connection will be set up faster as you don't need
to wait for DNS resolution, so message will be sent faster.
This ensures we do not get stuck trying DNS resolver results
when we have a known to work IP address in the cache
and DNS resolver returns garbage
either because it is a captive portal
or if it maliciously wants to get us stuck
trying a long list of unresponsive IP addresses.
This also limits the number of results we try to 10 overall.
If there are more results, we will retry later
with new resolution results.
HTTPS requests are used to fetch
remote images in HTML emails,
to fetch autoconfig XML,
to POST requests for `DCACCOUNT:` QR codes
to make OAuth 2 API requests
and to connect to HTTPS proxies.
Rustls is more aggressive than OpenSSL
in deprecating cryptographic algorithms
so we cannot use it for IMAP and SMTP
to avoid breaking compatibility,
but for HTTPS requests listed
above this should not result in problems.
As HTTPS requests use only strict TLS checks,
there is no `strict_tls` argument
in `wrap_rustls` function.
Rustls is already used by iroh,
so this change does not introduce new dependencies.
This change introduces new config options
`proxy_enabled` and `proxy_url`
that replace `socks5_*`.
Tested with deltachat-repl
by starting it with
`cargo run --locked -p deltachat-repl -- deltachat-db` and running
```
> set proxy_enabled 1
> set proxy_url ss://...
> setqr dcaccount:https://chatmail.example.org/new
> configure
```
This change replaces
usage of `reqwest` and `hyper-util`
with custom connection establishment code
so it is done in the same way
as for IMAP and SMTP connections.
This way we control HTTP, IMAP and SMTP
connection establishment
and schedule connection attempts
to resolved IP addresses
in the same way for all 3 protocols.
Before the fix HTTP client
had no connection timeout,
so it only had a chance
to test one IPv6 and one IPv4
address if the first addresses timed out.
Now it can test at least 4 addresses
of each family and more if some addresses
refuse connection rather than time out.
Otherwise if DNS server returns incorrect results,
we may never try preloaded DNS results.
For example, we may get our first results
from a captive portal.
To test, add `127.0.0.1 example.org`
and try to create an account.
Without this change we only try 127.0.0.1 and fail.
With this change preloaded DNS results are tried as well.
Apparently some providers fail TLS connection
with "no_application_protocol" alert
even when requesting "imap" protocol for IMAP connection
and "smtp" protocol for SMTP connection.
Fixes <https://github.com/deltachat/deltachat-core-rust/issues/5892>.
