Compare commits

...

18 Commits

Author SHA1 Message Date
link2xt
ddd4fc49a2 chore(release): prepare for 2.35.0 2025-12-16 22:22:20 +00:00
link2xt
7d5bedde4d fix: take transport_id into account when using imap table 2025-12-16 22:08:33 +00:00
iequidoo
e34fee72a0 feat: lookup_host_with_cache(): Don't return empty address list (#7596)
All users of this function expect a nonempty list to be returned, otherwise they fail with hard to
understand errors like "No connection attempts were made", so it's better to fail early if DNS
resolution failed and the cache doesn't contain resolutions as well.
2025-12-16 16:03:17 -03:00
link2xt
7ba4a43253 feat: add transport addresses to IMAP URLs in message info 2025-12-16 16:49:49 +00:00
Simon Laux
a09fd4577a fix: add explicit limit for adding relays (5 at the moment) (#7611)
closes https://github.com/chatmail/core/issues/7608
2025-12-15 10:35:23 +00:00
Simon Laux
525a3539d2 misc: log entered login params and actual used params on configuration failure (#7610)
#7587 removed "used_account_settings" and "entered_account_settings"
from Context.get_info(). link2xt pointed out that
entered_account_settings can still be useful to debug login issues, so
tis pr adds logs both on failed configuration attempts.

example warning event:
```
Warning src/configure.rs:292: configure failed: entered params myself@merlinux.eu imap:unset:***:unset:0:Automatic:AUTH_NORMAL smtp:unset:0:unset:0:Automatic:AUTH_NORMAL cert_automatic, used params myself@merlinux.eu imap:[mailcow.testrun.org:993:tls:myself@merlinux.eu, mailcow.testrun.org:143:starttls:myself@merlinux.eu] smtp:[mailcow.testrun.org:465:tls:myself@merlinux.eu, mailcow.testrun.org:587:starttls:myself@merlinux.eu] provider:none cert_automatic
```

---------

Co-authored-by: iequidoo <117991069+iequidoo@users.noreply.github.com>
2025-12-15 08:37:45 +00:00
Simon Laux
fbcdd45015 feat: add ip addresses of known public chatmail relays from https://chatmail.at/relays to dns cache (#7607)
closes #7597
2025-12-15 08:29:24 +00:00
iequidoo
1ea8ed6442 fix: Use fallback ICE servers if server can't IMAP METADATA (#7382) 2025-12-14 19:44:11 +00:00
iequidoo
f6817131b8 fix: Don't use fallback servers if got TURN servers from IMAP METADATA 2025-12-14 19:44:10 +00:00
iequidoo
28fc1d2ff2 feat: Use turn.delta.chat as fallback TURN server (#7382) 2025-12-14 19:44:10 +00:00
Simon Laux
5925f72316 fix: remove now redundant "used_account_settings" and "entered_account_settings" from Context.get_info() (#7587)
follow up to https://github.com/chatmail/core/pull/7583
2025-12-13 21:21:55 +00:00
Simon Laux
8dfa5fc37e api: add blob dir size to storage info (#7605)
closes #7598
2025-12-12 20:15:12 +00:00
B. Petersen
49b04e8789 feat: improve error messages on adding relays 2025-12-12 18:28:14 +01:00
link2xt
d87d87f467 fix: do not set normalized name for existing chats and contacts in a migration
We got a report that application is not responding after update
on Android and getting killed before it can start,
suspected to be a slow SQL migration:
<https://github.com/chatmail/core/issues/7602>

This change removes calculation of normalized names for
existing chats and contacts added in
<https://github.com/chatmail/core/pull/7548>
to exclude the possibility of this migration being slow.
New chats and contacts will still get normalized names
and all chats and contacts will get it when they are renamed.
2025-12-12 15:44:51 +00:00
iequidoo
bf72b3ad49 fix: Remove SecurejoinWait info message when received Alice's key (#7585)
And don't add a `SecurejoinWait` info message at all if we know Alice's key from the start. If we
don't remove this info message, it appears in the chat after "Messages are end-to-end encrypted..."
which is quite confusing when Bob can already send messages to Alice.
2025-12-12 04:01:32 -03:00
iequidoo
30f2981259 fix: get_chat_msgs_ex(): Don't match on "S=" (Cmd) in param payload 2025-12-12 04:01:32 -03:00
link2xt
121bfd1fa8 ci: update Rust to 1.92.0 2025-12-11 21:23:53 +00:00
link2xt
9e2a4325e9 chore: apply Rust 1.92.0 clippy suggestions 2025-12-11 21:23:53 +00:00
32 changed files with 429 additions and 191 deletions

View File

@@ -20,7 +20,7 @@ permissions: {}
env:
RUSTFLAGS: -Dwarnings
RUST_VERSION: 1.91.0
RUST_VERSION: 1.92.0
# Minimum Supported Rust Version
MSRV: 1.88.0

View File

@@ -1,5 +1,42 @@
# Changelog
## [2.35.0] - 2025-12-16
### API-Changes
- Add blob dir size to storage info ([#7605](https://github.com/chatmail/core/pull/7605)).
### Features / Changes
- Use `turn.delta.chat` as fallback TURN server ([#7382](https://github.com/chatmail/core/pull/7382)).
- Add ip addresses of known public chatmail relays from https://chatmail.at/relays to DNS cache ([#7607](https://github.com/chatmail/core/pull/7607)).
- Improve error messages on adding relays.
- Add transport addresses to IMAP URLs in message info.
- `lookup_host_with_cache()`: Don't return empty address list ([#7596](https://github.com/chatmail/core/pull/7596)).
### Fixes
- `get_chat_msgs_ex()`: Don't match on "S=" (Cmd) in param payload.
- Remove `SecurejoinWait` info message when received Alice's key ([#7585](https://github.com/chatmail/core/pull/7585)).
- Do not set normalized name for existing chats and contacts in a migration.
- Remove now redundant "used_account_settings" and "entered_account_settings" from `Context.get_info()` ([#7587](https://github.com/chatmail/core/pull/7587)).
- Don't use fallback servers if got TURN servers from IMAP METADATA.
- Use fallback ICE servers if server can't IMAP METADATA ([#7382](https://github.com/chatmail/core/pull/7382)).
- Add explicit limit for adding relays (5 at the moment) ([#7611](https://github.com/chatmail/core/pull/7611)).
- Take `transport_id` into account when using `imap` table.
### CI
- Update Rust to 1.92.0.
### Miscellaneous Tasks
- Apply Rust 1.92.0 clippy suggestions.
### Other
- Log entered login params and actual used params on configuration failure ([#7610](https://github.com/chatmail/core/pull/7610)).
## [2.34.0] - 2025-12-11
### API-Changes
@@ -7411,3 +7448,4 @@ https://github.com/chatmail/core/pulls?q=is%3Apr+is%3Aclosed
[2.32.0]: https://github.com/chatmail/core/compare/v2.31.0..v2.32.0
[2.33.0]: https://github.com/chatmail/core/compare/v2.32.0..v2.33.0
[2.34.0]: https://github.com/chatmail/core/compare/v2.33.0..v2.34.0
[2.35.0]: https://github.com/chatmail/core/compare/v2.34.0..v2.35.0

12
Cargo.lock generated
View File

@@ -1304,7 +1304,7 @@ dependencies = [
[[package]]
name = "deltachat"
version = "2.34.0"
version = "2.35.0"
dependencies = [
"anyhow",
"astral-tokio-tar",
@@ -1388,6 +1388,7 @@ dependencies = [
"tracing",
"url",
"uuid",
"walkdir",
"webpki-roots",
]
@@ -1413,7 +1414,7 @@ dependencies = [
[[package]]
name = "deltachat-jsonrpc"
version = "2.34.0"
version = "2.35.0"
dependencies = [
"anyhow",
"async-channel 2.5.0",
@@ -1429,13 +1430,12 @@ dependencies = [
"tempfile",
"tokio",
"typescript-type-def",
"walkdir",
"yerpc",
]
[[package]]
name = "deltachat-repl"
version = "2.34.0"
version = "2.35.0"
dependencies = [
"anyhow",
"deltachat",
@@ -1451,7 +1451,7 @@ dependencies = [
[[package]]
name = "deltachat-rpc-server"
version = "2.34.0"
version = "2.35.0"
dependencies = [
"anyhow",
"deltachat",
@@ -1480,7 +1480,7 @@ dependencies = [
[[package]]
name = "deltachat_ffi"
version = "2.34.0"
version = "2.35.0"
dependencies = [
"anyhow",
"deltachat",

View File

@@ -1,6 +1,6 @@
[package]
name = "deltachat"
version = "2.34.0"
version = "2.35.0"
edition = "2024"
license = "MPL-2.0"
rust-version = "1.88"
@@ -111,6 +111,7 @@ toml = "0.9"
tracing = "0.1.41"
url = "2"
uuid = { version = "1", features = ["serde", "v4"] }
walkdir = "2.5.0"
webpki-roots = "0.26.8"
[dev-dependencies]

View File

@@ -1,6 +1,6 @@
[package]
name = "deltachat_ffi"
version = "2.34.0"
version = "2.35.0"
description = "Deltachat FFI"
edition = "2018"
readme = "README.md"

View File

@@ -1,6 +1,6 @@
[package]
name = "deltachat-jsonrpc"
version = "2.34.0"
version = "2.35.0"
description = "DeltaChat JSON-RPC API"
edition = "2021"
license = "MPL-2.0"
@@ -19,7 +19,6 @@ yerpc = { workspace = true, features = ["anyhow_expose", "openrpc"] }
typescript-type-def = { version = "0.5.13", features = ["json_value"] }
tokio = { workspace = true }
sanitize-filename = { workspace = true }
walkdir = "2.5.0"
base64 = { workspace = true }
[dev-dependencies]

View File

@@ -35,14 +35,13 @@ use deltachat::qr_code_generator::{generate_backup_qr, get_securejoin_qr_svg};
use deltachat::reaction::{get_msg_reactions, send_reaction};
use deltachat::securejoin;
use deltachat::stock_str::StockMessage;
use deltachat::storage_usage::get_storage_usage;
use deltachat::storage_usage::{get_blobdir_storage_usage, get_storage_usage};
use deltachat::webxdc::StatusUpdateSerial;
use deltachat::EventEmitter;
use sanitize_filename::is_sanitized;
use tokio::fs;
use tokio::sync::{watch, Mutex, RwLock};
use types::login_param::EnteredLoginParam;
use walkdir::WalkDir;
use yerpc::rpc;
pub mod types;
@@ -330,13 +329,7 @@ impl CommandApi {
async fn get_account_file_size(&self, account_id: u32) -> Result<u64> {
let ctx = self.get_context(account_id).await?;
let dbfile = ctx.get_dbfile().metadata()?.len();
let total_size = WalkDir::new(ctx.get_blobdir())
.max_depth(2)
.into_iter()
.filter_map(|entry| entry.ok())
.filter_map(|entry| entry.metadata().ok())
.filter(|metadata| metadata.is_file())
.fold(0, |acc, m| acc + m.len());
let total_size = get_blobdir_storage_usage(&ctx);
Ok(dbfile + total_size)
}

View File

@@ -54,5 +54,5 @@
},
"type": "module",
"types": "dist/deltachat.d.ts",
"version": "2.34.0"
"version": "2.35.0"
}

View File

@@ -1,6 +1,6 @@
[package]
name = "deltachat-repl"
version = "2.34.0"
version = "2.35.0"
license = "MPL-2.0"
edition = "2021"
repository = "https://github.com/chatmail/core"

View File

@@ -430,12 +430,12 @@ async fn handle_cmd(
}
"oauth2" => {
if let Some(addr) = ctx.get_config(config::Config::Addr).await? {
let oauth2_url =
get_oauth2_url(&ctx, &addr, "chat.delta:/com.b44t.messenger").await?;
if oauth2_url.is_none() {
println!("OAuth2 not available for {}.", &addr);
if let Some(oauth2_url) =
get_oauth2_url(&ctx, &addr, "chat.delta:/com.b44t.messenger").await?
{
println!("Open the following url, set mail_pw to the generated token and server_flags to 2:\n{oauth2_url}");
} else {
println!("Open the following url, set mail_pw to the generated token and server_flags to 2:\n{}", oauth2_url.unwrap());
println!("OAuth2 not available for {}.", &addr);
}
} else {
println!("oauth2: set addr first.");

View File

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "deltachat-rpc-client"
version = "2.34.0"
version = "2.35.0"
license = "MPL-2.0"
description = "Python client for Delta Chat core JSON-RPC interface"
classifiers = [

View File

@@ -221,3 +221,58 @@ def test_recognize_self_address(acfactory) -> None:
bob_chat.send_text("Hello!")
msg = alice.wait_for_incoming_msg().get_snapshot()
assert msg.chat == alice.create_chat(bob)
def test_transport_limit(acfactory) -> None:
"""Test transports limit."""
account = acfactory.get_online_account()
qr = acfactory.get_account_qr()
limit = 5
for _ in range(1, limit):
account.add_transport_from_qr(qr)
assert len(account.list_transports()) == limit
with pytest.raises(JsonRpcError):
account.add_transport_from_qr(qr)
second_addr = account.list_transports()[1]["addr"]
account.delete_transport(second_addr)
# test that adding a transport after deleting one works again
account.add_transport_from_qr(qr)
def test_message_info_imap_urls(acfactory, log) -> None:
"""Test that message info contains IMAP URLs of where the message was received."""
alice, bob = acfactory.get_online_accounts(2)
log.section("Alice adds ac1 clone removes second transport")
qr = acfactory.get_account_qr()
for i in range(3):
alice.add_transport_from_qr(qr)
# Wait for all transports to go IDLE after adding each one.
for _ in range(i + 1):
alice.bring_online()
new_alice_addr = alice.list_transports()[2]["addr"]
alice.set_config("configured_addr", new_alice_addr)
# Enable multi-device mode so messages are not deleted immediately.
alice.set_config("bcc_self", "1")
# Bob creates chat, learning about Alice's currently selected transport.
# This is where he will send the message.
bob_chat = bob.create_chat(alice)
# Alice changes the transport again.
alice.set_config("configured_addr", alice.list_transports()[3]["addr"])
bob_chat.send_text("Hello!")
msg = alice.wait_for_incoming_msg()
for alice_transport in alice.list_transports():
addr = alice_transport["addr"]
assert (addr == new_alice_addr) == (addr in msg.get_info())

View File

@@ -90,12 +90,9 @@ def test_lowercase_address(acfactory) -> None:
assert account.get_config("configured_addr") == addr
assert account.list_transports()[0]["addr"] == addr
for param in [
account.get_info()["used_account_settings"],
account.get_info()["entered_account_settings"],
]:
assert addr in param
assert addr_upper not in param
param = account.get_info()["used_transport_settings"]
assert addr in param
assert addr_upper not in param
def test_configure_ip(acfactory) -> None:
@@ -733,7 +730,7 @@ def test_configured_imap_certificate_checks(acfactory):
alice = acfactory.new_configured_account()
# Certificate checks should be configured (not None)
assert "cert_strict" in alice.get_info().used_account_settings
assert "cert_strict" in alice.get_info().used_transport_settings
# "cert_old_automatic" is the value old Delta Chat core versions used
# to mean user entered "imap_certificate_checks=0" (Automatic)
@@ -746,7 +743,7 @@ def test_configured_imap_certificate_checks(acfactory):
#
# Core 1.142.4, 1.142.5 and 1.142.6 saved this value due to bug.
# This test is a regression test to prevent this happening again.
assert "cert_old_automatic" not in alice.get_info().used_account_settings
assert "cert_old_automatic" not in alice.get_info().used_transport_settings
def test_no_old_msg_is_fresh(acfactory):

View File

@@ -1,6 +1,6 @@
[package]
name = "deltachat-rpc-server"
version = "2.34.0"
version = "2.35.0"
description = "DeltaChat JSON-RPC server"
edition = "2021"
readme = "README.md"

View File

@@ -15,5 +15,5 @@
},
"type": "module",
"types": "index.d.ts",
"version": "2.34.0"
"version": "2.35.0"
}

View File

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "deltachat"
version = "2.34.0"
version = "2.35.0"
license = "MPL-2.0"
description = "Python bindings for the Delta Chat Core library using CFFI against the Rust-implemented libdeltachat"
readme = "README.rst"

View File

@@ -1295,16 +1295,17 @@ def test_configure_error_msgs_invalid_server(acfactory):
ev = ac2._evtracker.get_matching("DC_EVENT_CONFIGURE_PROGRESS")
if ev.data1 == 0:
break
err_lower = ev.data2.lower()
# Can't connect so it probably should say something about "internet"
# again, should not repeat itself
# If this fails then probably `e.msg.to_lowercase().contains("could not resolve")`
# in configure.rs returned false because the error message was changed
# (i.e. did not contain "could not resolve" anymore)
assert (ev.data2.count("internet") + ev.data2.count("network")) == 1
assert (err_lower.count("internet") + err_lower.count("network")) == 1
# Should mention that it can't connect:
assert ev.data2.count("connect") == 1
assert err_lower.count("connect") == 1
# The users do not know what "configuration" is
assert "configuration" not in ev.data2.lower()
assert "configuration" not in err_lower
def test_status(acfactory):

View File

@@ -1 +1 @@
2025-12-11
2025-12-16

View File

@@ -7,7 +7,7 @@ set -euo pipefail
#
# Avoid using rustup here as it depends on reading /proc/self/exe and
# has problems running under QEMU.
RUST_VERSION=1.91.0
RUST_VERSION=1.92.0
ARCH="$(uname -m)"
test -f "/lib/libc.musl-$ARCH.so.1" && LIBC=musl || LIBC=gnu

View File

@@ -663,9 +663,7 @@ pub(crate) async fn create_fallback_ice_servers(context: &Context) -> Result<Str
// because of bandwidth costs:
// <https://github.com/jselbie/stunserver/issues/50>
// We use nine.testrun.org for a default STUN server.
let hostname = "nine.testrun.org";
// Do not use cache because there is no TLS.
let load_cache = false;
let urls: Vec<String> = lookup_host_with_cache(context, hostname, STUN_PORT, "", load_cache)
@@ -673,14 +671,27 @@ pub(crate) async fn create_fallback_ice_servers(context: &Context) -> Result<Str
.into_iter()
.map(|addr| format!("stun:{addr}"))
.collect();
let ice_server = IceServer {
let stun_server = IceServer {
urls,
username: None,
credential: None,
};
let json = serde_json::to_string(&[ice_server])?;
let hostname = "turn.delta.chat";
// Do not use cache because there is no TLS.
let load_cache = false;
let urls: Vec<String> = lookup_host_with_cache(context, hostname, STUN_PORT, "", load_cache)
.await?
.into_iter()
.map(|addr| format!("turn:{addr}"))
.collect();
let turn_server = IceServer {
urls,
username: Some("public".to_string()),
credential: Some("o4tR7yG4rG2slhXqRUf9zgmHz".to_string()),
};
let json = serde_json::to_string(&[stun_server, turn_server])?;
Ok(json)
}

View File

@@ -3090,7 +3090,7 @@ pub async fn get_chat_msgs_ex(
WHERE m.chat_id=?
AND m.hidden=0
AND (
m.param GLOB \"*S=*\"
m.param GLOB '*\nS=*' OR param GLOB 'S=*'
OR m.from_id == ?
OR m.to_id == ?
);",

View File

@@ -45,6 +45,10 @@ use crate::transport::{
use crate::{EventType, stock_str};
use crate::{chat, provider};
/// Maximum number of relays
/// see <https://github.com/chatmail/core/issues/7608>
pub(crate) const MAX_TRANSPORT_RELAYS: usize = 5;
macro_rules! progress {
($context:tt, $progress:expr, $comment:expr) => {
assert!(
@@ -269,17 +273,50 @@ impl Context {
.await?
{
if self.get_config(Config::MvboxMove).await?.as_deref() != Some("0") {
bail!("Cannot use multi-transport with mvbox_move enabled.");
bail!(
"To use additional relays, disable the legacy option \"Settings / Advanced / Move automatically to DeltaChat Folder\"."
);
}
if self.get_config(Config::OnlyFetchMvbox).await?.as_deref() != Some("0") {
bail!("Cannot use multi-transport with only_fetch_mvbox enabled.");
bail!(
"To use additional relays, disable the legacy option \"Settings / Advanced / Only Fetch from DeltaChat Folder\"."
);
}
if self.get_config(Config::ShowEmails).await?.as_deref() != Some("2") {
bail!("Cannot use multi-transport with disabled fetching of classic emails.");
bail!(
"To use additional relays, set the legacy option \"Settings / Advanced / Show Classic Emails\" to \"All\"."
);
}
if self
.sql
.count("SELECT COUNT(*) FROM transports", ())
.await?
>= MAX_TRANSPORT_RELAYS
{
bail!(
"You have reached the maximum number of relays ({}).",
MAX_TRANSPORT_RELAYS
)
}
}
let provider = configure(self, param).await?;
let provider = match configure(self, param).await {
Err(error) => {
// Log entered and actual params
let configured_param = get_configured_param(self, param).await;
warn!(
self,
"configure failed: Entered params: {}. Used params: {}. Error: {error}.",
param.to_string(),
configured_param
.map(|param| param.to_string())
.unwrap_or("error".to_owned())
);
return Err(error);
}
Ok(provider) => provider,
};
self.set_config_internal(Config::NotifyAboutWrongPw, Some("1"))
.await?;
on_configure_completed(self, provider).await?;

View File

@@ -23,7 +23,6 @@ use crate::imap::{FolderMeaning, Imap, ServerMetadata};
use crate::key::self_fingerprint;
use crate::log::warn;
use crate::logged_debug_assert;
use crate::login_param::EnteredLoginParam;
use crate::message::{self, MessageState, MsgId};
use crate::net::tls::TlsSessionStore;
use crate::peer_channels::Iroh;
@@ -816,11 +815,6 @@ impl Context {
/// Returns information about the context as key-value pairs.
pub async fn get_info(&self) -> Result<BTreeMap<&'static str, String>> {
let l = EnteredLoginParam::load(self).await?;
let l2 = ConfiguredLoginParam::load(self).await?.map_or_else(
|| "Not configured".to_string(),
|(_transport_id, param)| param.to_string(),
);
let secondary_addrs = self.get_secondary_self_addrs().await?.join(", ");
let all_transports: Vec<String> = ConfiguredLoginParam::load_all(self)
.await?
@@ -910,8 +904,6 @@ impl Context {
.unwrap_or_else(|| "<unset>".to_string()),
);
res.insert("proxy_enabled", proxy_enabled.to_string());
res.insert("entered_account_settings", l.to_string());
res.insert("used_account_settings", l2);
res.insert("used_transport_settings", all_transports);
if let Some(server_id) = &*self.server_id.read().await {

View File

@@ -153,11 +153,15 @@ pub(crate) async fn download_msg(
return Ok(());
};
let transport_id = session.transport_id();
let row = context
.sql
.query_row_optional(
"SELECT uid, folder FROM imap WHERE rfc724_mid=? AND target!=''",
(&msg.rfc724_mid,),
"SELECT uid, folder FROM imap
WHERE rfc724_mid=?
AND transport_id=?
AND target!=''",
(&msg.rfc724_mid, transport_id),
|row| {
let server_uid: u32 = row.get(0)?;
let server_folder: String = row.get(1)?;

View File

@@ -123,7 +123,7 @@ struct OAuth2 {
access_token: String,
}
#[derive(Debug)]
#[derive(Debug, Default)]
pub(crate) struct ServerMetadata {
/// IMAP METADATA `/shared/comment` as defined in
/// <https://www.rfc-editor.org/rfc/rfc5464#section-6.2.1>.
@@ -916,7 +916,7 @@ impl Session {
context
.sql
.transaction(move |transaction| {
transaction.execute("DELETE FROM imap WHERE folder=?", (folder,))?;
transaction.execute("DELETE FROM imap WHERE transport_id=? AND folder=?", (transport_id, folder,))?;
for (uid, (rfc724_mid, target)) in &msgs {
// This may detect previously undetected moved
// messages, so we update server_folder too.
@@ -1054,14 +1054,16 @@ impl Session {
///
/// This is the only place where messages are moved or deleted on the IMAP server.
async fn move_delete_messages(&mut self, context: &Context, folder: &str) -> Result<()> {
let transport_id = self.transport_id();
let rows = context
.sql
.query_map_vec(
"SELECT id, uid, target FROM imap
WHERE folder = ?
AND target != folder
ORDER BY target, uid",
(folder,),
WHERE folder = ?
AND transport_id = ?
AND target != folder
ORDER BY target, uid",
(folder, transport_id),
|row| {
let rowid: i64 = row.get(0)?;
let uid: u32 = row.get(1)?;
@@ -1277,10 +1279,10 @@ impl Session {
};
let is_seen = fetch.flags().any(|flag| flag == Flag::Seen);
if is_seen
&& let Some(chat_id) = mark_seen_by_uid(context, folder, uid_validity, uid)
&& let Some(chat_id) = mark_seen_by_uid(context, transport_id, folder, uid_validity, uid)
.await
.with_context(|| {
format!("failed to update seen status for msg {folder}/{uid}")
format!("Transport {transport_id}: Failed to update seen status for msg {folder}/{uid}")
})?
{
updated_chat_ids.insert(chat_id);
@@ -1546,17 +1548,17 @@ impl Session {
Ok(())
}
/// Retrieves server metadata if it is supported.
/// Retrieves server metadata if it is supported, otherwise uses fallback one.
///
/// We get [`/shared/comment`](https://www.rfc-editor.org/rfc/rfc5464#section-6.2.1)
/// and [`/shared/admin`](https://www.rfc-editor.org/rfc/rfc5464#section-6.2.2)
/// metadata.
pub(crate) async fn fetch_metadata(&mut self, context: &Context) -> Result<()> {
if !self.can_metadata() {
return Ok(());
}
pub(crate) async fn update_metadata(&mut self, context: &Context) -> Result<()> {
let mut lock = context.metadata.write().await;
if !self.can_metadata() {
*lock = Some(Default::default());
}
if let Some(ref mut old_metadata) = *lock {
let now = time();
@@ -1565,31 +1567,33 @@ impl Session {
return Ok(());
}
info!(context, "ICE servers expired, requesting new credentials.");
let mailbox = "";
let options = "";
let metadata = self
.get_metadata(mailbox, options, "(/shared/vendor/deltachat/turn)")
.await?;
let mut got_turn_server = false;
for m in metadata {
if m.entry == "/shared/vendor/deltachat/turn"
&& let Some(value) = m.value
{
match create_ice_servers_from_metadata(context, &value).await {
Ok((parsed_timestamp, parsed_ice_servers)) => {
old_metadata.ice_servers_expiration_timestamp = parsed_timestamp;
old_metadata.ice_servers = parsed_ice_servers;
got_turn_server = false;
}
Err(err) => {
warn!(context, "Failed to parse TURN server metadata: {err:#}.");
if self.can_metadata() {
info!(context, "ICE servers expired, requesting new credentials.");
let mailbox = "";
let options = "";
let metadata = self
.get_metadata(mailbox, options, "(/shared/vendor/deltachat/turn)")
.await?;
for m in metadata {
if m.entry == "/shared/vendor/deltachat/turn"
&& let Some(value) = m.value
{
match create_ice_servers_from_metadata(context, &value).await {
Ok((parsed_timestamp, parsed_ice_servers)) => {
old_metadata.ice_servers_expiration_timestamp = parsed_timestamp;
old_metadata.ice_servers = parsed_ice_servers;
got_turn_server = true;
}
Err(err) => {
warn!(context, "Failed to parse TURN server metadata: {err:#}.");
}
}
}
}
}
if !got_turn_server {
info!(context, "Will use fallback ICE servers.");
// Set expiration timestamp 7 days in the future so we don't request it again.
old_metadata.ice_servers_expiration_timestamp = time() + 3600 * 24 * 7;
old_metadata.ice_servers = create_fallback_ice_servers(context).await?;
@@ -2357,6 +2361,7 @@ pub(crate) async fn prefetch_should_download(
/// Returns updated chat ID if any message was marked as seen.
async fn mark_seen_by_uid(
context: &Context,
transport_id: u32,
folder: &str,
uid_validity: u32,
uid: u32,
@@ -2367,12 +2372,13 @@ async fn mark_seen_by_uid(
"SELECT id, chat_id FROM msgs
WHERE id > 9 AND rfc724_mid IN (
SELECT rfc724_mid FROM imap
WHERE folder=?1
AND uidvalidity=?2
AND uid=?3
WHERE transport_id=?
AND folder=?
AND uidvalidity=?
AND uid=?
LIMIT 1
)",
(&folder, uid_validity, uid),
(transport_id, &folder, uid_validity, uid),
|row| {
let msg_id: MsgId = row.get(0)?;
let chat_id: ChatId = row.get(1)?;

View File

@@ -171,12 +171,17 @@ impl MsgId {
context
.sql
.query_map_vec(
"SELECT folder, uid FROM imap WHERE rfc724_mid=?",
"SELECT transports.addr, imap.folder, imap.uid
FROM imap
LEFT JOIN transports
ON transports.id = imap.transport_id
WHERE imap.rfc724_mid=?",
(rfc724_mid,),
|row| {
let folder: String = row.get("folder")?;
let uid: u32 = row.get("uid")?;
Ok(format!("</{folder}/;UID={uid}>"))
let addr: String = row.get(0)?;
let folder: String = row.get(1)?;
let uid: u32 = row.get(2)?;
Ok(format!("<{addr}/{folder}/;UID={uid}>"))
},
)
.await

View File

@@ -40,7 +40,7 @@
//! used for successful connection timestamp of
//! retrieving them from in-memory cache is used.
use anyhow::{Context as _, Result};
use anyhow::{Context as _, Result, ensure};
use std::collections::HashMap;
use std::net::{IpAddr, Ipv4Addr, Ipv6Addr, SocketAddr};
use std::str::FromStr;
@@ -506,10 +506,6 @@ static DNS_PRELOAD: LazyLock<HashMap<&'static str, Vec<IpAddr>>> = LazyLock::new
"mail.nubo.coop",
vec![IpAddr::V4(Ipv4Addr::new(79, 99, 201, 10))],
),
(
"mehl.cloud",
vec![IpAddr::V4(Ipv4Addr::new(95, 217, 223, 172))],
),
(
"mx.freenet.de",
vec![
@@ -680,6 +676,72 @@ static DNS_PRELOAD: LazyLock<HashMap<&'static str, Vec<IpAddr>>> = LazyLock::new
IpAddr::V4(Ipv4Addr::new(185, 230, 214, 164)),
],
),
// Known public chatmail relays from https://chatmail.at/relays
(
"mehl.cloud",
vec![IpAddr::V4(Ipv4Addr::new(95, 217, 223, 172))],
),
(
"mailchat.pl",
vec![IpAddr::V4(Ipv4Addr::new(46, 62, 144, 137))],
),
(
"chatmail.woodpeckersnest.space",
vec![IpAddr::V4(Ipv4Addr::new(85, 215, 162, 146))],
),
(
"chatmail.culturanerd.it",
vec![IpAddr::V4(Ipv4Addr::new(82, 165, 94, 165))],
),
(
"chatmail.hackea.org",
vec![IpAddr::V4(Ipv4Addr::new(82, 165, 11, 85))],
),
(
"chika.aangat.lahat.computer",
vec![IpAddr::V4(Ipv4Addr::new(71, 19, 150, 113))],
),
(
"tarpit.fun",
vec![IpAddr::V4(Ipv4Addr::new(152, 53, 86, 246))],
),
(
"d.gaufr.es",
vec![IpAddr::V4(Ipv4Addr::new(51, 77, 140, 91))],
),
(
"chtml.ca",
vec![IpAddr::V4(Ipv4Addr::new(51, 222, 156, 177))],
),
(
"chatmail.au",
vec![IpAddr::V4(Ipv4Addr::new(45, 124, 54, 79))],
),
(
"sombras.chat",
vec![IpAddr::V4(Ipv4Addr::new(82, 25, 70, 154))],
),
(
"e2ee.wang",
vec![IpAddr::V4(Ipv4Addr::new(139, 84, 233, 161))],
),
(
"chat.privittytech.com",
vec![IpAddr::V4(Ipv4Addr::new(35, 154, 144, 0))],
),
("e2ee.im", vec![IpAddr::V4(Ipv4Addr::new(45, 137, 99, 57))]),
(
"chatmail.email",
vec![IpAddr::V4(Ipv4Addr::new(57, 128, 220, 120))],
),
(
"danneskjold.de",
vec![IpAddr::V4(Ipv4Addr::new(46, 62, 216, 132))],
),
(
"darkrun.dev",
vec![IpAddr::V4(Ipv4Addr::new(72, 11, 149, 146))],
),
])
});
@@ -788,7 +850,7 @@ pub(crate) async fn lookup_host_with_cache(
}
};
if load_cache {
let addrs = if load_cache {
let mut cache = lookup_cache(context, hostname, port, alpn, now).await?;
if let Some(ips) = DNS_PRELOAD.get(hostname) {
for ip in ips {
@@ -799,10 +861,15 @@ pub(crate) async fn lookup_host_with_cache(
}
}
Ok(merge_with_cache(resolved_addrs, cache))
merge_with_cache(resolved_addrs, cache)
} else {
Ok(resolved_addrs)
}
resolved_addrs
};
ensure!(
!addrs.is_empty(),
"Could not find DNS resolutions for {hostname}:{port}. Check server hostname and your network"
);
Ok(addrs)
}
/// Merges results received from DNS with cached results.

View File

@@ -538,9 +538,9 @@ async fn inbox_fetch_idle(ctx: &Context, imap: &mut Imap, mut session: Session)
.await
.context("Failed to download messages")?;
session
.fetch_metadata(ctx)
.update_metadata(ctx)
.await
.context("Failed to fetch metadata")?;
.context("update_metadata")?;
session
.register_token(ctx)
.await

View File

@@ -5,14 +5,16 @@ use anyhow::{Context as _, Result};
use super::HandshakeMessage;
use super::qrinvite::QrInvite;
use crate::chat::{self, ChatId, is_contact_in_chat};
use crate::chatlist_events;
use crate::constants::{Blocked, Chattype};
use crate::contact::Origin;
use crate::context::Context;
use crate::events::EventType;
use crate::key::self_fingerprint;
use crate::message::{Message, Viewtype};
use crate::log::LogExt;
use crate::message::{Message, MsgId, Viewtype};
use crate::mimeparser::{MimeMessage, SystemMessage};
use crate::param::Param;
use crate::param::{Param, Params};
use crate::securejoin::{ContactId, encrypted_and_signed, verify_sender_by_fingerprint};
use crate::stock_str;
use crate::sync::Sync::*;
@@ -48,16 +50,16 @@ pub(super) async fn start_protocol(context: &Context, invite: QrInvite) -> Resul
ContactId::scaleup_origin(context, &[invite.contact_id()], Origin::SecurejoinJoined).await?;
context.emit_event(EventType::ContactsChanged(None));
let has_key = context
.sql
.exists(
"SELECT COUNT(*) FROM public_keys WHERE fingerprint=?",
(invite.fingerprint().hex(),),
)
.await?;
// Now start the protocol and initialise the state.
{
let has_key = context
.sql
.exists(
"SELECT COUNT(*) FROM public_keys WHERE fingerprint=?",
(invite.fingerprint().hex(),),
)
.await?;
// `joining_chat_id` is `Some` if group chat
// already exists and we are in the chat.
let joining_chat_id = match invite {
@@ -142,20 +144,22 @@ pub(super) async fn start_protocol(context: &Context, invite: QrInvite) -> Resul
Ok(joining_chat_id)
}
QrInvite::Contact { .. } => {
// For setup-contact the BobState already ensured the 1:1 chat exists because it
// uses it to send the handshake messages.
chat::add_info_msg_with_cmd(
context,
private_chat_id,
&stock_str::securejoin_wait(context).await,
SystemMessage::SecurejoinWait,
None,
time(),
None,
None,
None,
)
.await?;
// For setup-contact the BobState already ensured the 1:1 chat exists because it is
// used to send the handshake messages.
if !has_key {
chat::add_info_msg_with_cmd(
context,
private_chat_id,
&stock_str::securejoin_wait(context).await,
SystemMessage::SecurejoinWait,
None,
time(),
None,
None,
None,
)
.await?;
}
Ok(private_chat_id)
}
}
@@ -177,6 +181,38 @@ async fn insert_new_db_entry(context: &Context, invite: QrInvite, chat_id: ChatI
.await
}
async fn delete_securejoin_wait_msg(context: &Context, chat_id: ChatId) -> Result<()> {
if let Some((msg_id, param)) = context
.sql
.query_row_optional(
"
SELECT id, param FROM msgs
WHERE timestamp=(SELECT MAX(timestamp) FROM msgs WHERE chat_id=? AND hidden=0)
AND chat_id=? AND hidden=0
LIMIT 1
",
(chat_id, chat_id),
|row| {
let id: MsgId = row.get(0)?;
let param: String = row.get(1)?;
let param: Params = param.parse().unwrap_or_default();
Ok((id, param))
},
)
.await?
&& param.get_cmd() == SystemMessage::SecurejoinWait
{
let on_server = false;
msg_id.trash(context, on_server).await?;
context.emit_event(EventType::MsgDeleted { chat_id, msg_id });
context.emit_msgs_changed_without_msg_id(chat_id);
chatlist_events::emit_chatlist_item_changed(context, chat_id);
context.emit_msgs_changed_without_ids();
chatlist_events::emit_chatlist_changed(context);
}
Ok(())
}
/// Handles `vc-auth-required` and `vg-auth-required` handshake messages.
///
/// # Bob - the joiner's side
@@ -213,6 +249,11 @@ pub(super) async fn handle_auth_required(
info!(context, "Fingerprint verified.",);
let chat_id = private_chat_id(context, &invite).await?;
delete_securejoin_wait_msg(context, chat_id)
.await
.context("delete_securejoin_wait_msg")
.log_err(context)
.ok();
send_handshake_message(context, &invite, chat_id, BobHandshakeMsg::RequestWithAuth).await?;
context
.sql

View File

@@ -100,6 +100,17 @@ async fn test_setup_contact_ex(case: SetupContactCase) {
bob_chat.why_cant_send(&bob).await.unwrap(),
Some(CantSendReason::MissingKey)
);
// Check Bob's info messages.
let msg_cnt = 2;
let mut i = 0..msg_cnt;
let msg = get_chat_msg(&bob, bob_chat.get_id(), i.next().unwrap(), msg_cnt).await;
assert!(msg.is_info());
assert_eq!(msg.get_text(), messages_e2e_encrypted(&bob).await);
let msg = get_chat_msg(&bob, bob_chat.get_id(), i.next().unwrap(), msg_cnt).await;
assert!(msg.is_info());
assert_eq!(msg.get_text(), stock_str::securejoin_wait(&bob).await);
let contact_alice_id = bob.add_or_lookup_contact_no_key(&alice).await.id;
let sent = bob.pop_sent_msg().await;
assert!(!sent.payload.contains("Bob Examplenet"));
@@ -272,15 +283,10 @@ async fn test_setup_contact_ex(case: SetupContactCase) {
assert!(contact_alice.get_name().is_empty());
assert_eq!(contact_alice.is_bot(), case == SetupContactCase::AliceIsBot);
// Check Bob got expected info messages in his 1:1 chat.
let msg_cnt = 2;
let mut i = 0..msg_cnt;
let msg = get_chat_msg(&bob, bob_chat.get_id(), i.next().unwrap(), msg_cnt).await;
// The `SecurejoinWait` info message has been removed, but the e2ee notice remains.
let msg = get_chat_msg(&bob, bob_chat.get_id(), 0, 1).await;
assert!(msg.is_info());
assert_eq!(msg.get_text(), messages_e2e_encrypted(&bob).await);
let msg = get_chat_msg(&bob, bob_chat.get_id(), i.next().unwrap(), msg_cnt).await;
assert!(msg.is_info());
assert_eq!(msg.get_text(), stock_str::securejoin_wait(&bob).await);
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]

View File

@@ -19,7 +19,7 @@ use crate::log::warn;
use crate::message::MsgId;
use crate::provider::get_provider_info;
use crate::sql::Sql;
use crate::tools::{Time, inc_and_check, normalize_text, time_elapsed};
use crate::tools::{Time, inc_and_check, time_elapsed};
use crate::transport::ConfiguredLoginParam;
const DBVERSION: i32 = 68;
@@ -1456,52 +1456,14 @@ CREATE INDEX imap_sync_index ON imap_sync(transport_id, folder);
inc_and_check(&mut migration_version, 143)?;
if dbversion < migration_version {
let trans_fn = |t: &mut rusqlite::Transaction| {
t.execute_batch(
"
sql.execute_migration(
"
ALTER TABLE chats ADD COLUMN name_normalized TEXT;
ALTER TABLE contacts ADD COLUMN name_normalized TEXT;
",
)?;
let mut stmt = t.prepare("UPDATE chats SET name_normalized=? WHERE id=?")?;
for res in t
.prepare("SELECT id, name FROM chats LIMIT 10000")?
.query_map((), |row| {
let id: u32 = row.get(0)?;
let name: String = row.get(1)?;
Ok((id, name))
})?
{
let (id, name) = res?;
if let Some(name_normalized) = normalize_text(&name) {
stmt.execute((name_normalized, id))?;
}
}
let mut stmt = t.prepare("UPDATE contacts SET name_normalized=? WHERE id=?")?;
for res in t
.prepare(
"
SELECT id, IIF(name='', authname, name) FROM contacts
ORDER BY last_seen DESC LIMIT 10000
",
)?
.query_map((), |row| {
let id: u32 = row.get(0)?;
let name: String = row.get(1)?;
Ok((id, name))
})?
{
let (id, name) = res?;
if let Some(name_normalized) = normalize_text(&name) {
stmt.execute((name_normalized, id))?;
}
}
Ok(())
};
sql.execute_migration_transaction(trans_fn, migration_version)
.await?;
migration_version,
)
.await?;
}
let new_version = sql

View File

@@ -2,6 +2,7 @@
use crate::{context::Context, message::MsgId};
use anyhow::Result;
use humansize::{BINARY, format_size};
use walkdir::WalkDir;
/// Storage Usage Report
/// Useful for debugging space usage problems in the deltachat database.
@@ -14,11 +15,15 @@ pub struct StorageUsage {
/// count and total size of status updates
/// for the 10 webxdc apps with the most size usage in status updates
pub largest_webxdc_data: Vec<(MsgId, u64, u64)>,
/// Total size of all files in the blobdir
pub blobdir_size: u64,
}
impl std::fmt::Display for StorageUsage {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
writeln!(f, "Storage Usage:")?;
let blobdir_size = format_size(self.blobdir_size, BINARY);
writeln!(f, "[Blob Directory Size]: {blobdir_size}")?;
let human_db_size = format_size(self.db_size, BINARY);
writeln!(f, "[Database Size]: {human_db_size}")?;
writeln!(f, "[Largest Tables]:")?;
@@ -46,6 +51,10 @@ impl std::fmt::Display for StorageUsage {
/// Get storage usage information for the Context's database
pub async fn get_storage_usage(ctx: &Context) -> Result<StorageUsage> {
let context_clone = ctx.clone();
let blobdir_size =
tokio::task::spawn_blocking(move || get_blobdir_storage_usage(&context_clone));
let page_size: u64 = ctx
.sql
.query_get_value("PRAGMA page_size", ())
@@ -101,9 +110,23 @@ pub async fn get_storage_usage(ctx: &Context) -> Result<StorageUsage> {
)
.await?;
let blobdir_size = blobdir_size.await?;
Ok(StorageUsage {
db_size: page_size * page_count,
largest_tables,
largest_webxdc_data,
blobdir_size,
})
}
/// Returns storage usage of the blob directory
pub fn get_blobdir_storage_usage(ctx: &Context) -> u64 {
WalkDir::new(ctx.get_blobdir())
.max_depth(2)
.into_iter()
.filter_map(|entry| entry.ok())
.filter_map(|entry| entry.metadata().ok())
.filter(|metadata| metadata.is_file())
.fold(0, |acc, m| acc + m.len())
}