Compare commits

..

7 Commits

Author SHA1 Message Date
Hocuri
232f8f24d1 fix: Manipulate sort_timestamp to not be 0 2026-03-28 16:35:01 +01:00
link2xt
58aafef935 fix: do not sort received messages below the last seen one 2026-03-28 11:24:39 +01:00
link2xt
be2b2bd561 fix: never sort the message before chat joining timestamp
This is to avoid sorting incoming messages that
are slightly in the past above system messages
about SecureJoin. SecureJoin messages are
timed according to smeared timestamp,
so even in the local tests they are in the future
by a few seconds.
2026-03-28 10:54:37 +01:00
link2xt
3e8acee642 fix: always sort "Messages are end-to-end encrypted" notice to the beginning
We set timestamp of this info message to 0
to make it always appear in the beginning of the chat.
To avoid new chats being sorted to the end of the chatlist,
we ignore such 0 and use chat creation timestamp
when sorting the chatlist.
2026-03-28 10:24:44 +01:00
link2xt
1f9f0d7393 test: use load_imf_email() more 2026-03-28 10:24:44 +01:00
link2xt
c5e53fa1a2 test: do not rely on loading newest chat in load_imf_email()
We know which message was added from the return value
of receive_imf(). It may be that the first chat
in the chatlist is not the one where the message was received
if there is a pinned chat or if
just received message is old.
2026-03-28 10:24:44 +01:00
link2xt
f98c021ad1 test: remove test_old_message_5
It is not clear now what this is testing.
Golden test shows messages ordered
incorrectly according to the timestamps,
they should be ordered the other way round.

Comment talks about fetching from mvbox and inbox
in paralell which is a rare case that
could have happened if one message is left in the inbox
and the other message is a chat message moved to mvbox.
We never download anything that is not moved to the target folder.

The test also resides in "verified chats" tests
which are all legacy tests we kept after
replacing the concept of verified/protected chats
with key contacts in 2.x.
2026-03-28 10:24:32 +01:00
53 changed files with 438 additions and 1085 deletions

View File

@@ -14,7 +14,7 @@ jobs:
steps:
- name: Dependabot metadata
id: metadata
uses: dependabot/fetch-metadata@v3.0.0
uses: dependabot/fetch-metadata@v2.4.0
with:
github-token: "${{ secrets.GITHUB_TOKEN }}"
- name: Approve a PR

View File

@@ -1,23 +0,0 @@
# Check that PRs are made against the -dev version.
#
# If this fails, push commit to update the version to -dev to main.
name: Check for -dev version
on:
pull_request:
permissions: {}
jobs:
check_dev_version:
name: Check that current version ends with -dev
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
with:
show-progress: false
persist-credentials: false
- name: Run version-checking script
run: scripts/check-dev-version.py

View File

@@ -1,61 +1,5 @@
# Changelog
## [2.48.0] - 2026-03-30
### Fixes
- Fix reordering problems in multi-relay setups by not sorting received messages below the last seen one.
- Always sort "Messages are end-to-end encrypted" notice to the beginning.
- Make Message-ID of pre-messages stable across resends ([#8007](https://github.com/chatmail/core/pull/8007)).
- Delete `imap_markseen` entries not corresponding to any `imap` rows.
- Cleanup `imap` and `imap_sync` records without transport in housekeeping.
- When receiving MDN, mark all preceding messages as noticed, even having same timestamp ([#7928](https://github.com/chatmail/core/pull/7928)).
- Remove migration 108 preventing upgrades from core 1.86.0 to the latest version.
### Features / Changes
- Improve IMAP loop logs.
- Add decryption error to the device message about outgoing message decryption failure.
- Log received message sort timestamp.
### Performance
- Move sorting outside of SQL query in `store_seen_flags_on_imap`.
### API-Changes
- Add JSON-RPC API `markfresh_chat()`.
- ffi: Correctly declare `dc_event_channel_new()` as having no params ([#7831](https://github.com/chatmail/core/pull/7831)).
### Refactor
- Remove `wal_checkpoint_mutex`, lock `write_mutex` before getting sql connection instead.
- Replace async `RwLock` with sync `RwLock` for stock strings.
- Cleanup remaining Autocrypt Setup Message processing in `mimeparser`.
- SecureJoin: do not check for self address in forwarding protection.
- Fix clippy warnings.
### CI
- Update {c,py}.delta.chat website deployments.
- Use environments for {rs,cffi,js.jsonrpc}.delta.chat deployments.
- Fix https://docs.zizmor.sh/audits/#bot-conditions.
### Documentation
- Add SQL performance tips to STYLE.md.
### Tests
- Remove `test_old_message_5`.
- Do not rely on loading newest chat in `load_imf_email()`.
- Use `load_imf_email()` more.
- The message is sorted correctly in the chat even if it arrives late.
### Miscellaneous Tasks
- cargo: update rustls-webpki to 0.103.10.
## [2.47.0] - 2026-03-24
### Fixes
@@ -8040,4 +7984,3 @@ https://github.com/chatmail/core/pulls?q=is%3Apr+is%3Aclosed
[2.45.0]: https://github.com/chatmail/core/compare/v2.44.0..v2.45.0
[2.46.0]: https://github.com/chatmail/core/compare/v2.45.0..v2.46.0
[2.47.0]: https://github.com/chatmail/core/compare/v2.46.0..v2.47.0
[2.48.0]: https://github.com/chatmail/core/compare/v2.47.0..v2.48.0

54
Cargo.lock generated
View File

@@ -827,9 +827,9 @@ dependencies = [
[[package]]
name = "chrono"
version = "0.4.44"
version = "0.4.43"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c673075a2e0e5f4a1dde27ce9dee1ea4558c7ffe648f576438a20ca1d2acc4b0"
checksum = "fac4744fb15ae8337dc853fee7fb3f4e48c0fbaa23d0afe49c447b4fab126118"
dependencies = [
"iana-time-zone",
"num-traits",
@@ -1307,7 +1307,7 @@ dependencies = [
[[package]]
name = "deltachat"
version = "2.49.0-dev"
version = "2.48.0-dev"
dependencies = [
"anyhow",
"astral-tokio-tar",
@@ -1416,7 +1416,7 @@ dependencies = [
[[package]]
name = "deltachat-jsonrpc"
version = "2.49.0-dev"
version = "2.48.0-dev"
dependencies = [
"anyhow",
"async-channel 2.5.0",
@@ -1437,7 +1437,7 @@ dependencies = [
[[package]]
name = "deltachat-repl"
version = "2.49.0-dev"
version = "2.48.0-dev"
dependencies = [
"anyhow",
"deltachat",
@@ -1453,7 +1453,7 @@ dependencies = [
[[package]]
name = "deltachat-rpc-server"
version = "2.49.0-dev"
version = "2.48.0-dev"
dependencies = [
"anyhow",
"deltachat",
@@ -1482,7 +1482,7 @@ dependencies = [
[[package]]
name = "deltachat_ffi"
version = "2.49.0-dev"
version = "2.48.0-dev"
dependencies = [
"anyhow",
"deltachat",
@@ -2860,9 +2860,9 @@ dependencies = [
[[package]]
name = "image"
version = "0.25.10"
version = "0.25.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "85ab80394333c02fe689eaf900ab500fbd0c2213da414687ebf995a65d5a6104"
checksum = "e6506c6c10786659413faa717ceebcb8f70731c0a60cbae39795fdf114519c1a"
dependencies = [
"bytemuck",
"byteorder-lite",
@@ -3260,9 +3260,9 @@ dependencies = [
[[package]]
name = "libc"
version = "0.2.184"
version = "0.2.182"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "48f5d2a454e16a5ea0f4ced81bd44e4cfc7bd3a507b61887c99fd3538b28e4af"
checksum = "6800badb6cb2082ffd7b6a67e6125bb39f18782f793520caee8cb8846be06112"
[[package]]
name = "libm"
@@ -3483,9 +3483,9 @@ dependencies = [
[[package]]
name = "moxcms"
version = "0.8.1"
version = "0.7.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bb85c154ba489f01b25c0d36ae69a87e4a1c73a72631fc6c0eb6dde34a73e44b"
checksum = "ddd32fa8935aeadb8a8a6b6b351e40225570a37c43de67690383d87ef170cd08"
dependencies = [
"num-traits",
"pxfm",
@@ -4235,18 +4235,18 @@ dependencies = [
[[package]]
name = "pin-project"
version = "1.1.11"
version = "1.1.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f1749c7ed4bcaf4c3d0a3efc28538844fb29bcdd7d2b67b2be7e20ba861ff517"
checksum = "677f1add503faace112b9f1373e43e9e054bfdd22ff1a63c1bc485eaec6a6a8a"
dependencies = [
"pin-project-internal",
]
[[package]]
name = "pin-project-internal"
version = "1.1.11"
version = "1.1.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d9b20ed30f105399776b9c883e68e536ef602a16ae6f596d2c473591d6ad64c6"
checksum = "6e918e4ff8c4549eb882f14b3a4bc8c8bc93de829416eacf579f1207a8fbf861"
dependencies = [
"proc-macro2",
"quote",
@@ -4615,9 +4615,9 @@ dependencies = [
[[package]]
name = "proptest"
version = "1.11.0"
version = "1.10.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4b45fcc2344c680f5025fe57779faef368840d0bd1f42f216291f0dc4ace4744"
checksum = "37566cb3fdacef14c0737f9546df7cfeadbfbc9fef10991038bf5015d0c80532"
dependencies = [
"bitflags 2.11.0",
"num-traits",
@@ -4729,9 +4729,9 @@ dependencies = [
[[package]]
name = "quote"
version = "1.0.45"
version = "1.0.44"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "41f2619966050689382d2b44f664f4bc593e129785a36d6ee376ddf37259b924"
checksum = "21b2ebcf727b7760c461f091f9f0f539b77b8e87f2fd88131e7f1b433b3cece4"
dependencies = [
"proc-macro2",
]
@@ -5988,9 +5988,9 @@ checksum = "55937e1799185b12863d447f42597ed69d9928686b8d88a1df17376a097d8369"
[[package]]
name = "tempfile"
version = "3.27.0"
version = "3.26.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "32497e9a4c7b38532efcdebeef879707aa9f794296a4f0244f6f69e9bc8574bd"
checksum = "82a72c767771b47409d2345987fda8628641887d5466101319899796367354a0"
dependencies = [
"fastrand",
"getrandom 0.3.3",
@@ -6144,9 +6144,9 @@ checksum = "1f3ccbac311fea05f86f61904b462b55fb3df8837a366dfc601a0161d0532f20"
[[package]]
name = "tokio"
version = "1.50.0"
version = "1.49.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "27ad5e34374e03cfffefc301becb44e9dc3c17584f414349ebe29ed26661822d"
checksum = "72a2903cd7736441aac9df9d7688bd0ce48edccaadf181c3b90be801e81d3d86"
dependencies = [
"bytes",
"libc",
@@ -6403,9 +6403,9 @@ dependencies = [
[[package]]
name = "tracing-subscriber"
version = "0.3.23"
version = "0.3.22"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cb7f578e5945fb242538965c2d0b04418d38ec25c79d160cd279bf0731c8d319"
checksum = "2f30143827ddab0d256fd843b7a66d164e9f271cfa0dde49142c5ca0ca291f1e"
dependencies = [
"matchers",
"nu-ansi-term",

View File

@@ -1,6 +1,6 @@
[package]
name = "deltachat"
version = "2.49.0-dev"
version = "2.48.0-dev"
edition = "2024"
license = "MPL-2.0"
rust-version = "1.88"
@@ -181,7 +181,7 @@ harness = false
anyhow = "1"
async-channel = "2.5.0"
base64 = "0.22"
chrono = { version = "0.4.44", default-features = false }
chrono = { version = "0.4.43", default-features = false }
deltachat-contact-tools = { path = "deltachat-contact-tools" }
deltachat-jsonrpc = { path = "deltachat-jsonrpc", default-features = false }
deltachat = { path = ".", default-features = false }
@@ -198,7 +198,7 @@ rusqlite = "0.37"
sanitize-filename = "0.6"
serde = "1.0"
serde_json = "1"
tempfile = "3.27.0"
tempfile = "3.25.0"
thiserror = "2"
tokio = "1"
tokio-util = "0.7.18"

View File

@@ -68,12 +68,6 @@ keyword doesn't help here.
Consider adding context to `anyhow` errors for SQL statements using `.context()` so that it's
possible to understand from logs which statement failed. See [Errors](#errors) for more info.
When changing complex SQL queries, test them on a new database with `EXPLAIN QUERY PLAN`
to make sure that indexes are used and large tables are not going to be scanned.
Never run `ANALYZE` on the databases,
this makes query planner unpredictable
and may make performance significantly worse: <https://github.com/chatmail/core/issues/6585>
## Errors
Delta Chat core mostly uses [`anyhow`](https://docs.rs/anyhow/) errors.

View File

@@ -1,6 +1,6 @@
[package]
name = "deltachat_ffi"
version = "2.49.0-dev"
version = "2.48.0-dev"
description = "Deltachat FFI"
edition = "2018"
readme = "README.md"

View File

@@ -364,14 +364,18 @@ uint32_t dc_get_id (dc_context_t* context);
* To get these events, you have to create an event emitter using this function
* and call dc_get_next_event() on the emitter.
*
* Events are broadcasted to all existing event emitters.
* Events emitted before creation of event emitter
* are not available to event emitter.
*
* @memberof dc_context_t
* @param context The context object as created by dc_context_new().
* @return Returns the event emitter, NULL on errors.
* Must be freed using dc_event_emitter_unref() after usage.
*
* Note: Use only one event emitter per context.
* The result of having multiple event emitters is unspecified.
* Currently events are broadcasted to all existing event emitters,
* but previous versions delivered events to only one event emitter
* and this behavior may change again in the future.
* Events emitted before creation of event emitter
* may or may not be available to event emitter.
*/
dc_event_emitter_t* dc_get_event_emitter(dc_context_t* context);
@@ -3319,14 +3323,18 @@ void dc_accounts_set_push_device_token (dc_accounts_t* accounts, const
* This is similar to dc_get_event_emitter(), which, however,
* must not be called for accounts handled by the account manager.
*
* Events are broadcasted to all existing event emitters.
* Events emitted before creation of event emitter
* are not available to event emitter.
*
* @memberof dc_accounts_t
* @param accounts The account manager as created by dc_accounts_new().
* @return Returns the event emitter, NULL on errors.
* Must be freed using dc_event_emitter_unref() after usage.
*
* Note: Use only one event emitter per account manager.
* The result of having multiple event emitters is unspecified.
* Currently events are broadcasted to all existing event emitters,
* but previous versions delivered events to only one event emitter
* and this behavior may change again in the future.
* Events emitted before creation of event emitter
* are not available to event emitter.
*/
dc_event_emitter_t* dc_accounts_get_event_emitter (dc_accounts_t* accounts);
@@ -4973,6 +4981,17 @@ uint32_t dc_msg_get_original_msg_id (const dc_msg_t* msg);
*/
uint32_t dc_msg_get_saved_msg_id (const dc_msg_t* msg);
/**
* Force the message to be sent in plain text.
*
* This API is for bots, there is no need to expose it in the UI.
*
* @memberof dc_msg_t
* @param msg The message object.
*/
void dc_msg_force_plaintext (dc_msg_t* msg);
/**
* @class dc_contact_t
*
@@ -5960,14 +5979,21 @@ void dc_event_channel_unref(dc_event_channel_t* event_channel);
* To get these events, you have to create an event emitter using this function
* and call dc_get_next_event() on the emitter.
*
* Events are broadcasted to all existing event emitters.
* Events emitted before creation of event emitter
* are not available to event emitter.
* This is similar to dc_get_event_emitter(), which, however,
* must not be called for accounts handled by the account manager.
*
* @memberof dc_event_channel_t
* @param The event channel.
* @return Returns the event emitter, NULL on errors.
* Must be freed using dc_event_emitter_unref() after usage.
*
* Note: Use only one event emitter per account manager / event channel.
* The result of having multiple event emitters is unspecified.
* Currently events are broadcasted to all existing event emitters,
* but previous versions delivered events to only one event emitter
* and this behavior may change again in the future.
* Events emitted before creation of event emitter
* are not available to event emitter.
*/
dc_event_emitter_t* dc_event_channel_get_event_emitter(dc_event_channel_t* event_channel);

View File

@@ -2826,7 +2826,7 @@ pub unsafe extern "C" fn dc_array_search_id(
// Returns 1 if location belongs to the track of the user,
// 0 if location was reported independently.
#[no_mangle]
pub unsafe extern "C" fn dc_array_is_independent(
pub unsafe fn dc_array_is_independent(
array: *const dc_array_t,
index: libc::size_t,
) -> libc::c_int {
@@ -4054,6 +4054,16 @@ pub unsafe extern "C" fn dc_msg_get_saved_msg_id(msg: *const dc_msg_t) -> u32 {
})
}
#[no_mangle]
pub unsafe extern "C" fn dc_msg_force_plaintext(msg: *mut dc_msg_t) {
if msg.is_null() {
eprintln!("ignoring careless call to dc_msg_force_plaintext()");
return;
}
let ffi_msg = &mut *msg;
ffi_msg.message.force_plaintext();
}
// dc_contact_t
/// FFI struct for [dc_contact_t]

View File

@@ -1,6 +1,6 @@
[package]
name = "deltachat-jsonrpc"
version = "2.49.0-dev"
version = "2.48.0-dev"
description = "DeltaChat JSON-RPC API"
edition = "2021"
license = "MPL-2.0"

View File

@@ -678,7 +678,7 @@ impl CommandApi {
ChatId::new(chat_id).get_fresh_msg_cnt(&ctx).await
}
/// (deprecated) Gets messages to be processed by the bot and returns their IDs.
/// Gets messages to be processed by the bot and returns their IDs.
///
/// Only messages with database ID higher than `last_msg_id` config value
/// are returned. After processing the messages, the bot should
@@ -686,13 +686,6 @@ impl CommandApi {
/// or manually updating the value to avoid getting already
/// processed messages.
///
/// Deprecated 2026-04: This returns the message's id as soon as the first part arrives,
/// even if it is not fully downloaded yet.
/// The bot needs to wait for the message to be fully downloaded.
/// Since this is usually not the desired behavior,
/// bots should instead use the #DC_EVENT_INCOMING_MSG / [`types::events::EventType::IncomingMsg`]
/// event for getting notified about new messages.
///
/// [`markseen_msgs`]: Self::markseen_msgs
async fn get_next_msgs(&self, account_id: u32) -> Result<Vec<u32>> {
let ctx = self.get_context(account_id).await?;
@@ -705,7 +698,7 @@ impl CommandApi {
Ok(msg_ids)
}
/// (deprecated) Waits for messages to be processed by the bot and returns their IDs.
/// Waits for messages to be processed by the bot and returns their IDs.
///
/// This function is similar to [`get_next_msgs`],
/// but waits for internal new message notification before returning.
@@ -716,13 +709,6 @@ impl CommandApi {
/// To shutdown the bot, stopping I/O can be used to interrupt
/// pending or next `wait_next_msgs` call.
///
/// Deprecated 2026-04: This returns the message's id as soon as the first part arrives,
/// even if it is not fully downloaded yet.
/// The bot needs to wait for the message to be fully downloaded.
/// Since this is usually not the desired behavior,
/// bots should instead use the #DC_EVENT_INCOMING_MSG / [`types::events::EventType::IncomingMsg`]
/// event for getting notified about new messages.
///
/// [`get_next_msgs`]: Self::get_next_msgs
async fn wait_next_msgs(&self, account_id: u32) -> Result<Vec<u32>> {
let ctx = self.get_context(account_id).await?;

View File

@@ -54,5 +54,5 @@
},
"type": "module",
"types": "dist/deltachat.d.ts",
"version": "2.49.0-dev"
"version": "2.48.0-dev"
}

View File

@@ -1,6 +1,6 @@
[package]
name = "deltachat-repl"
version = "2.49.0-dev"
version = "2.48.0-dev"
license = "MPL-2.0"
edition = "2021"
repository = "https://github.com/chatmail/core"

View File

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "deltachat-rpc-client"
version = "2.49.0-dev"
version = "2.48.0-dev"
license = "MPL-2.0"
description = "Python client for Delta Chat core JSON-RPC interface"
classifiers = [

View File

@@ -405,15 +405,7 @@ class Account:
@futuremethod
def wait_next_messages(self) -> list[Message]:
"""(deprecated) Wait for new messages and return a list of them. Meant for bots.
Deprecated 2026-04: This returns the message's id as soon as the first part arrives,
even if it is not fully downloaded yet.
The bot needs to wait for the message to be fully downloaded.
Since this is usually not the desired behavior,
bots should instead use the `EventType.INCOMING_MSG`
event for getting notified about new messages.
"""
"""Wait for new messages and return a list of them."""
next_msg_ids = yield self._rpc.wait_next_msgs.future(self.id)
return [Message(self, msg_id) for msg_id in next_msg_ids]

View File

@@ -1047,7 +1047,6 @@ def test_no_old_msg_is_fresh(acfactory):
assert ac1.create_chat(ac2).get_fresh_message_count() == 1
assert len(list(ac1.get_fresh_messages())) == 1
ac1_clone.wait_for_incoming_msg_event()
ac1.wait_for_event(EventType.IMAP_INBOX_IDLE)
logging.info("Send a message from ac1_clone to ac2 and check that ac1 marks the first message as 'noticed'")

View File

@@ -1,6 +1,6 @@
[package]
name = "deltachat-rpc-server"
version = "2.49.0-dev"
version = "2.48.0-dev"
description = "DeltaChat JSON-RPC server"
edition = "2021"
readme = "README.md"

View File

@@ -18,7 +18,7 @@ import { startDeltaChat } from "@deltachat/stdio-rpc-server";
import { C } from "@deltachat/jsonrpc-client";
async function main() {
const dc = startDeltaChat("deltachat-data");
const dc = await startDeltaChat("deltachat-data");
console.log(await dc.rpc.getSystemInfo());
dc.close()
}

View File

@@ -15,7 +15,7 @@ export interface SearchOptions {
*/
export function getRPCServerPath(
options?: Partial<SearchOptions>
): string;
): Promise<string>;
@@ -33,15 +33,8 @@ export interface StartOptions {
* @param directory directory for accounts folder
* @param options
*/
export function startDeltaChat(directory: string, options?: Partial<SearchOptions & StartOptions> ): DeltaChatOverJsonRpcServer
export function startDeltaChat(directory: string, options?: Partial<SearchOptions & StartOptions> ): Promise<DeltaChatOverJsonRpcServer>
export class DeltaChatOverJsonRpc extends StdioDeltaChat {
constructor(
directory: string,
options?: Partial<SearchOptions & StartOptions>
);
readonly pathToServerBinary: string;
}
export namespace FnTypes {
export type getRPCServerPath = typeof getRPCServerPath

View File

@@ -1,6 +1,6 @@
//@ts-check
import { spawn } from "node:child_process";
import { statSync } from "node:fs";
import { stat } from "node:fs/promises";
import os from "node:os";
import process from "node:process";
import { ENV_VAR_NAME, PATH_EXECUTABLE_NAME } from "./src/const.js";
@@ -36,7 +36,7 @@ function findRPCServerInNodeModules() {
}
/** @type {import("./index").FnTypes.getRPCServerPath} */
export function getRPCServerPath(options = {}) {
export async function getRPCServerPath(options = {}) {
const { takeVersionFromPATH, disableEnvPath } = {
takeVersionFromPATH: false,
disableEnvPath: false,
@@ -45,7 +45,7 @@ export function getRPCServerPath(options = {}) {
// 1. check if it is set as env var
if (process.env[ENV_VAR_NAME] && !disableEnvPath) {
try {
if (!statSync(process.env[ENV_VAR_NAME]).isFile()) {
if (!(await stat(process.env[ENV_VAR_NAME])).isFile()) {
throw new Error(
`expected ${ENV_VAR_NAME} to point to the deltachat-rpc-server executable`
);
@@ -68,49 +68,41 @@ export function getRPCServerPath(options = {}) {
import { StdioDeltaChat } from "@deltachat/jsonrpc-client";
/** @type {import("./index").FnTypes.startDeltaChat} */
export function startDeltaChat(directory, options = {}) {
return new DeltaChatOverJsonRpc(directory, options);
}
export class DeltaChatOverJsonRpc extends StdioDeltaChat {
/**
*
* @param {string} directory
* @param {Partial<import("./index").SearchOptions & import("./index").StartOptions>} options
*/
constructor(directory, options = {}) {
const pathToServerBinary = getRPCServerPath(options);
const server = spawn(pathToServerBinary, {
env: {
RUST_LOG: process.env.RUST_LOG,
DC_ACCOUNTS_PATH: directory,
},
stdio: ["pipe", "pipe", options.muteStdErr ? "ignore" : "inherit"],
});
server.on("error", (err) => {
throw new Error(
FAILED_TO_START_SERVER_EXECUTABLE(pathToServerBinary, err)
);
});
let shouldClose = false;
server.on("exit", () => {
if (shouldClose) {
return;
}
throw new Error("Server quit");
});
super(server.stdin, server.stdout, true);
this.close = () => {
shouldClose = true;
if (!server.kill()) {
console.log("server termination failed");
}
};
this.pathToServerBinary = pathToServerBinary;
}
export async function startDeltaChat(directory, options = {}) {
const pathToServerBinary = await getRPCServerPath(options);
const server = spawn(pathToServerBinary, {
env: {
RUST_LOG: process.env.RUST_LOG,
DC_ACCOUNTS_PATH: directory,
},
stdio: ["pipe", "pipe", options.muteStdErr ? "ignore" : "inherit"],
});
server.on("error", (err) => {
throw new Error(FAILED_TO_START_SERVER_EXECUTABLE(pathToServerBinary, err));
});
let shouldClose = false;
server.on("exit", () => {
if (shouldClose) {
return;
}
throw new Error("Server quit");
});
/** @type {import('./index').DeltaChatOverJsonRpcServer} */
//@ts-expect-error
const dc = new StdioDeltaChat(server.stdin, server.stdout, true);
dc.close = () => {
shouldClose = true;
if (!server.kill()) {
console.log("server termination failed");
}
};
//@ts-expect-error
dc.pathToServerBinary = pathToServerBinary;
return dc;
}

View File

@@ -15,5 +15,5 @@
},
"type": "module",
"types": "index.d.ts",
"version": "2.49.0-dev"
"version": "2.48.0-dev"
}

View File

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "deltachat"
version = "2.49.0-dev"
version = "2.48.0-dev"
license = "MPL-2.0"
description = "Python bindings for the Delta Chat Core library using CFFI against the Rust-implemented libdeltachat"
readme = "README.rst"

View File

@@ -254,6 +254,10 @@ class Message:
"""Quote setter."""
lib.dc_msg_set_quote(self._dc_msg, quoted_message._dc_msg)
def force_plaintext(self) -> None:
"""Force the message to be sent in plain text."""
lib.dc_msg_force_plaintext(self._dc_msg)
@property
def error(self) -> Optional[str]:
"""Error message."""

View File

@@ -288,7 +288,6 @@ def test_use_new_verified_group_after_going_online(acfactory, data, tmp_path, lp
assert open(contact.get_profile_image(), "rb").read() == open(avatar_path, "rb").read()
lp.sec("ac2_offl: sending message")
chat2.accept()
msg_out = chat2.send_text("hello")
lp.sec("ac1: receiving message")

View File

@@ -1 +1 @@
2026-04-08
2026-03-24

View File

@@ -1,23 +0,0 @@
#!/usr/bin/env python3
#
# Script to check that current version ends with -dev.
# Meant to be run in CI to check that PRs are made against the -dev version.
# If the version is not -dev, it was forgotten to be updated
# after making a release.
from pathlib import Path
import tomllib
import sys
def main():
with Path("Cargo.toml").open("rb") as fp:
cargo_toml = tomllib.load(fp)
version = cargo_toml["package"]["version"]
if not version.endswith("-dev"):
print(f"Current version {version} does not end with -dev", file=sys.stderr)
sys.exit(1)
if __name__ == "__main__":
main()

View File

@@ -10,8 +10,8 @@ use anyhow::{Context as _, Result, ensure, format_err};
use base64::Engine as _;
use futures::StreamExt;
use image::ImageReader;
use image::codecs::jpeg::JpegEncoder;
use image::{DynamicImage, GenericImage, GenericImageView, ImageFormat, Pixel, Rgba};
use image::{codecs::jpeg::JpegEncoder, metadata::Orientation};
use num_traits::FromPrimitive;
use tokio::{fs, task};
use tokio_stream::wrappers::ReadDirStream;
@@ -362,10 +362,7 @@ impl<'a> BlobObject<'a> {
return Ok(name);
}
let mut img = imgreader.decode().context("image decode failure")?;
let orientation = exif
.as_ref()
.map(|exif| exif_orientation(exif, context))
.unwrap_or(Orientation::NoTransforms);
let orientation = exif.as_ref().map(|exif| exif_orientation(exif, context));
let mut encoded = Vec::new();
if *vt == Viewtype::Sticker {
@@ -384,7 +381,13 @@ impl<'a> BlobObject<'a> {
return Ok(name);
}
}
img.apply_orientation(orientation);
img = match orientation {
Some(90) => img.rotate90(),
Some(180) => img.rotate180(),
Some(270) => img.rotate270(),
_ => img,
};
// max_wh is the maximum image width and height, i.e. the resolution-limit.
// target_wh target-resolution for resizing the image.
@@ -548,17 +551,18 @@ fn image_metadata(file: &std::fs::File) -> Result<(u64, Option<exif::Exif>)> {
Ok((len, exif))
}
fn exif_orientation(exif: &exif::Exif, context: &Context) -> Orientation {
if let Some(orientation) = exif.get_field(exif::Tag::Orientation, exif::In::PRIMARY)
&& let Some(val) = orientation.value.get_uint(0)
&& let Ok(val) = TryInto::<u8>::try_into(val)
{
return Orientation::from_exif(val).unwrap_or({
warn!(context, "Exif orientation value ignored: {val:?}.");
Orientation::NoTransforms
});
fn exif_orientation(exif: &exif::Exif, context: &Context) -> i32 {
if let Some(orientation) = exif.get_field(exif::Tag::Orientation, exif::In::PRIMARY) {
// possible orientation values are described at http://sylvana.net/jpegcrop/exif_orientation.html
// we only use rotation, in practise, flipping is not used.
match orientation.value.get_uint(0) {
Some(3) => return 180,
Some(6) => return 90,
Some(8) => return 270,
other => warn!(context, "Exif orientation value ignored: {other:?}."),
}
}
Orientation::NoTransforms
0
}
/// All files in the blobdir.

View File

@@ -305,7 +305,7 @@ async fn test_recode_image_2() {
has_exif: true,
original_width: 2000,
original_height: 1800,
orientation: Some(Orientation::Rotate270),
orientation: 270,
compressed_width: 1800,
compressed_height: 2000,
..Default::default()
@@ -336,28 +336,6 @@ async fn test_recode_image_2() {
assert_correct_rotation(&img_rotated);
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_recode_image_vflipped() {
let bytes = include_bytes!("../../test-data/image/rectangle200x180-vflipped.jpg");
let img_rotated = SendImageCheckMediaquality {
viewtype: Viewtype::Image,
media_quality_config: "0",
bytes,
extension: "jpg",
has_exif: true,
original_width: 200,
original_height: 180,
orientation: Some(Orientation::FlipVertical),
compressed_width: 200,
compressed_height: 180,
..Default::default()
}
.test()
.await
.unwrap();
assert_correct_rotation(&img_rotated);
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_recode_image_bad_exif() {
// `exiftool` reports for this file "Bad offset for IFD0 XResolution", still Exif must be
@@ -552,7 +530,7 @@ struct SendImageCheckMediaquality<'a> {
pub(crate) has_exif: bool,
pub(crate) original_width: u32,
pub(crate) original_height: u32,
pub(crate) orientation: Option<Orientation>,
pub(crate) orientation: i32,
pub(crate) res_viewtype: Option<Viewtype>,
pub(crate) compressed_width: u32,
pub(crate) compressed_height: u32,
@@ -568,7 +546,7 @@ impl SendImageCheckMediaquality<'_> {
let has_exif = self.has_exif;
let original_width = self.original_width;
let original_height = self.original_height;
let orientation = self.orientation.unwrap_or(Orientation::NoTransforms);
let orientation = self.orientation;
let res_viewtype = self.res_viewtype.unwrap_or(Viewtype::Image);
let compressed_width = self.compressed_width;
let compressed_height = self.compressed_height;

View File

@@ -497,27 +497,6 @@ impl ChatId {
Ok(())
}
/// Adds info message to the beginning of the chat.
///
/// Used for messages such as
/// "Others will only see this group after you sent a first message."
pub(crate) async fn add_start_info_message(self, context: &Context, text: &str) -> Result<()> {
let sort_timestamp = 0;
add_info_msg_with_cmd(
context,
self,
text,
SystemMessage::Unknown,
Some(sort_timestamp),
time(),
None,
None,
None,
)
.await?;
Ok(())
}
/// Archives or unarchives a chat.
pub async fn set_visibility(self, context: &Context, visibility: ChatVisibility) -> Result<()> {
self.set_visibility_ex(context, Sync, visibility).await
@@ -3632,7 +3611,7 @@ pub(crate) async fn create_group_ex(
// Add "Messages in this chat use classic email and are not encrypted." message.
stock_str::chat_unencrypted_explanation(context)
};
chat_id.add_start_info_message(context, &text).await?;
add_info_msg(context, chat_id, &text).await?;
}
if let (true, true) = (sync.into(), !grpid.is_empty()) {
let id = SyncId::Grpid(grpid);
@@ -4131,56 +4110,61 @@ pub async fn remove_contact_from_chat(
delete_broadcast_secret(context, chat_id).await?;
}
ensure!(
matches!(
chat.typ,
Chattype::Group | Chattype::OutBroadcast | Chattype::InBroadcast
),
"Cannot remove members from non-group chats."
);
if matches!(
chat.typ,
Chattype::Group | Chattype::OutBroadcast | Chattype::InBroadcast
) {
if !chat.is_self_in_chat(context).await? {
let err_msg = format!(
"Cannot remove contact {contact_id} from chat {chat_id}: self not in group."
);
context.emit_event(EventType::ErrorSelfNotInGroup(err_msg.clone()));
bail!("{err_msg}");
} else {
let mut sync = Nosync;
if !chat.is_self_in_chat(context).await? {
let err_msg =
format!("Cannot remove contact {contact_id} from chat {chat_id}: self not in group.");
context.emit_event(EventType::ErrorSelfNotInGroup(err_msg.clone()));
bail!("{err_msg}");
}
if chat.is_promoted() && chat.typ != Chattype::OutBroadcast {
remove_from_chat_contacts_table(context, chat_id, contact_id).await?;
} else {
remove_from_chat_contacts_table_without_trace(context, chat_id, contact_id).await?;
}
let mut sync = Nosync;
// We do not return an error if the contact does not exist in the database.
// This allows to delete dangling references to deleted contacts
// in case of the database becoming inconsistent due to a bug.
if let Some(contact) = Contact::get_by_id_optional(context, contact_id).await? {
if chat.is_promoted() {
let addr = contact.get_addr();
let fingerprint = contact.fingerprint().map(|f| f.hex());
if chat.is_promoted() && chat.typ != Chattype::OutBroadcast {
remove_from_chat_contacts_table(context, chat_id, contact_id).await?;
} else {
remove_from_chat_contacts_table_without_trace(context, chat_id, contact_id).await?;
}
// We do not return an error if the contact does not exist in the database.
// This allows to delete dangling references to deleted contacts
// in case of the database becoming inconsistent due to a bug.
if let Some(contact) = Contact::get_by_id_optional(context, contact_id).await? {
if chat.is_promoted() {
let addr = contact.get_addr();
let fingerprint = contact.fingerprint().map(|f| f.hex());
let res =
send_member_removal_msg(context, &chat, contact_id, addr, fingerprint.as_deref())
let res = send_member_removal_msg(
context,
&chat,
contact_id,
addr,
fingerprint.as_deref(),
)
.await;
if contact_id == ContactId::SELF {
res?;
} else if let Err(e) = res {
warn!(
context,
"remove_contact_from_chat({chat_id}, {contact_id}): send_msg() failed: {e:#}."
);
if contact_id == ContactId::SELF {
res?;
} else if let Err(e) = res {
warn!(
context,
"remove_contact_from_chat({chat_id}, {contact_id}): send_msg() failed: {e:#}."
);
}
} else {
sync = Sync;
}
}
context.emit_event(EventType::ChatModified(chat_id));
if sync.into() {
chat.sync_contacts(context).await.log_err(context).ok();
}
} else {
sync = Sync;
}
}
context.emit_event(EventType::ChatModified(chat_id));
if sync.into() {
chat.sync_contacts(context).await.log_err(context).ok();
} else {
bail!("Cannot remove members from non-group chats.");
}
Ok(())

View File

@@ -7,7 +7,7 @@ use crate::constants::{DC_GCL_ARCHIVED_ONLY, DC_GCL_NO_SPECIALS};
use crate::ephemeral::Timer;
use crate::headerdef::HeaderDef;
use crate::imex::{ImexMode, has_backup, imex};
use crate::message::{Message, MessengerMessage, delete_msgs};
use crate::message::{MessengerMessage, delete_msgs};
use crate::mimeparser::{self, MimeMessage};
use crate::receive_imf::receive_imf;
use crate::securejoin::{get_securejoin_qr, join_securejoin};
@@ -2792,7 +2792,7 @@ async fn test_broadcast_members_cant_see_each_other() -> Result<()> {
assert_eq!(parsed.decoded_data_contains("bob@example.net"), false);
let parsed_by_bob = bob.parse_msg(&vc_pubkey).await;
assert!(parsed_by_bob.decryption_error.is_some());
assert!(parsed_by_bob.decrypting_failed);
charlie.recv_msg_trash(&vc_pubkey).await;
}
@@ -2821,7 +2821,7 @@ async fn test_broadcast_members_cant_see_each_other() -> Result<()> {
assert_eq!(parsed.decoded_data_contains("bob@example.net"), false);
let parsed_by_bob = bob.parse_msg(&member_added).await;
assert!(parsed_by_bob.decryption_error.is_some());
assert!(parsed_by_bob.decrypting_failed);
let rcvd = charlie.recv_msg(&member_added).await;
assert_eq!(rcvd.param.get_cmd(), SystemMessage::MemberAddedToGroup);
@@ -2836,7 +2836,7 @@ async fn test_broadcast_members_cant_see_each_other() -> Result<()> {
assert_eq!(parsed.decoded_data_contains("bob@example.net"), false);
let parsed_by_bob = bob.parse_msg(&hi_msg).await;
assert!(parsed_by_bob.decryption_error.is_none());
assert_eq!(parsed_by_bob.decrypting_failed, false);
}
tcm.section("Alice removes Charlie. Bob must not see it.");
@@ -2853,7 +2853,7 @@ async fn test_broadcast_members_cant_see_each_other() -> Result<()> {
assert_eq!(parsed.decoded_data_contains("bob@example.net"), false);
let parsed_by_bob = bob.parse_msg(&member_removed).await;
assert!(parsed_by_bob.decryption_error.is_some());
assert!(parsed_by_bob.decrypting_failed);
let rcvd = charlie.recv_msg(&member_removed).await;
assert_eq!(rcvd.param.get_cmd(), SystemMessage::MemberRemovedFromGroup);
@@ -3768,7 +3768,14 @@ async fn test_leave_broadcast_multidevice() -> Result<()> {
// The contact should be marked as verified.
check_direct_chat_is_hidden_and_contact_is_verified(alice, bob0).await;
check_direct_chat_is_hidden_and_contact_is_verified(bob0, alice).await;
check_direct_chat_is_hidden_and_contact_is_verified(bob1, alice).await;
// TODO: There is a known bug in `observe_securejoin_on_other_device()`:
// When Bob joins a group or broadcast with his first device,
// then a chat with Alice will pop up on his second device.
// When it's fixed, the 2 following lines can be replaced with
// `check_direct_chat_is_hidden_and_contact_is_verified(bob1, alice).await;`
let bob1_alice_contact = bob1.add_or_lookup_contact_no_key(alice).await;
assert!(bob1_alice_contact.is_verified(bob1).await.unwrap());
tcm.section("Alice sends first message to broadcast.");
let sent_msg = alice.send_text(alice_chat_id, "Hello!").await;
@@ -6105,118 +6112,3 @@ async fn test_leftgrps() -> Result<()> {
Ok(())
}
/// Tests that if the message arrives late,
/// it can still be sorted above the last seen message.
///
/// Versions 2.47 and below always sorted incoming messages
/// after the last seen message, but with
/// the introduction of multi-relay it is possible
/// that some messages arrive only to some relays
/// and are fetched after the already arrived seen message.
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_late_message_above_seen() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
let bob = &tcm.bob().await;
let charlie = &tcm.charlie().await;
let alice_chat_id = alice
.create_group_with_members("Group", &[bob, charlie])
.await;
let alice_sent = alice.send_text(alice_chat_id, "Hello everyone!").await;
let bob_chat_id = bob.recv_msg(&alice_sent).await.chat_id;
bob_chat_id.accept(bob).await?;
let charlie_chat_id = charlie.recv_msg(&alice_sent).await.chat_id;
charlie_chat_id.accept(charlie).await?;
// Bob sends a message, but the message is delayed.
let bob_sent = bob.send_text(bob_chat_id, "Hello from Bob!").await;
SystemTime::shift(Duration::from_secs(1000));
let charlie_sent = charlie
.send_text(charlie_chat_id, "Hello from Charlie!")
.await;
// Alice immediately receives a message from Charlie and reads it.
let alice_received_from_charlie = alice.recv_msg(&charlie_sent).await;
assert_eq!(
alice_received_from_charlie.get_text(),
"Hello from Charlie!"
);
message::markseen_msgs(alice, vec![alice_received_from_charlie.id]).await?;
// Bob message arrives later, it should be above the message from Charlie.
let alice_received_from_bob = alice.recv_msg(&bob_sent).await;
assert_eq!(alice_received_from_bob.get_text(), "Hello from Bob!");
// The last message in the chat is still from Charlie.
let last_msg = alice.get_last_msg_in(alice_chat_id).await;
assert_eq!(last_msg.get_text(), "Hello from Charlie!");
Ok(())
}
/// Tests that start message for unpromoted groups sticks to the top of the chat.
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_unpromoted_group_start_message() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
let bob = &tcm.bob().await;
// Start messages are disabled for test context by default,
// but this test is specifically about start messages.
alice.set_config(Config::SkipStartMessages, None).await?;
// Shift the clock forward, so we can rewind it back later.
SystemTime::shift(Duration::from_secs(3600));
// Alice creates unpromoted group with Bob.
let chat_id = create_group(alice, "Group").await?;
let bob_id = alice.add_or_lookup_contact_id(bob).await;
add_contact_to_chat(alice, chat_id, bob_id).await?;
let [
ChatItem::Message {
msg_id: e2ee_msg_id,
},
ChatItem::Message {
msg_id: info_msg_id,
},
] = get_chat_msgs(alice, chat_id).await?[..]
else {
panic!("Expected two messages in the chat");
};
let msg = Message::load_from_db(alice, e2ee_msg_id).await?;
assert_eq!(msg.text, "Messages are end-to-end encrypted.");
let msg = Message::load_from_db(alice, info_msg_id).await?;
assert_eq!(
msg.text,
"Others will only see this group after you sent a first message."
);
// Alice rewinds the clock.
SystemTime::shift_back(Duration::from_secs(3600));
let text_msg_id = send_text_msg(alice, chat_id, "Hello".to_string()).await?;
let [
ChatItem::Message {
msg_id: e2ee_msg_id2,
},
ChatItem::Message {
msg_id: info_msg_id2,
},
ChatItem::Message {
msg_id: text_msg_id2,
},
] = get_chat_msgs(alice, chat_id).await?[..]
else {
panic!("Expected three messages in the chat");
};
assert_eq!(e2ee_msg_id2, e2ee_msg_id);
assert_eq!(info_msg_id2, info_msg_id);
assert_eq!(text_msg_id2, text_msg_id);
Ok(())
}

View File

@@ -261,7 +261,6 @@ impl Context {
.await?;
send_sync_transports(self).await?;
self.quota.write().await.remove(&removed_transport_id);
self.restart_io_if_running().await;
Ok(())
}

View File

@@ -1182,9 +1182,7 @@ VALUES (?, ?, ?, ?, ?, ?)
let mut ret = Vec::new();
let flag_add_self = (listflags & constants::DC_GCL_ADD_SELF) != 0;
let flag_address = (listflags & constants::DC_GCL_ADDRESS) != 0;
let minimal_origin = if context.get_config_bool(Config::Bot).await?
|| query.is_some_and(may_be_valid_addr)
{
let minimal_origin = if context.get_config_bool(Config::Bot).await? {
Origin::Unknown
} else {
Origin::IncomingReplyTo

View File

@@ -420,16 +420,12 @@ async fn test_delete() -> Result<()> {
Contact::delete(&alice, contact_id).await?;
let contact = Contact::get_by_id(&alice, contact_id).await?;
assert_eq!(contact.origin, Origin::Hidden);
// Hidden contacts are found when searching by email address
assert_eq!(
Contact::get_all(&alice, 0, Some("bob@example.net"))
.await?
.len(),
1
0
);
// Hidden contacts are not found by a non-address query
assert_eq!(Contact::get_all(&alice, 0, Some("bob")).await?.len(), 0);
// Delete chat.
chat.get_id().delete(&alice).await?;
@@ -487,7 +483,7 @@ async fn test_delete_and_recreate_contact() -> Result<()> {
Contact::get_all(&t, 0, Some("bob@example.net"))
.await?
.len(),
1
0
);
let contact_id3 = t.add_or_lookup_contact_id(&bob).await;

View File

@@ -1142,17 +1142,10 @@ ORDER BY m.timestamp DESC,m.id DESC",
Ok(list)
}
/// (deprecated) Returns a list of messages with database ID higher than requested.
/// Returns a list of messages with database ID higher than requested.
///
/// Blocked contacts and chats are excluded,
/// but self-sent messages and contact requests are included in the results.
///
/// Deprecated 2026-04: This returns the message's id as soon as the first part arrives,
/// even if it is not fully downloaded yet.
/// The bot needs to wait for the message to be fully downloaded.
/// Since this is usually not the desired behavior,
/// bots should instead use the [`EventType::IncomingMsg`]
/// event for getting notified about new messages.
pub async fn get_next_msgs(&self) -> Result<Vec<MsgId>> {
let last_msg_id = match self.get_config(Config::LastMsgId).await? {
Some(s) => MsgId::new(s.parse()?),
@@ -1201,7 +1194,7 @@ ORDER BY m.timestamp DESC,m.id DESC",
Ok(list)
}
/// (deprecated) Returns a list of messages with database ID higher than last marked as seen.
/// Returns a list of messages with database ID higher than last marked as seen.
///
/// This function is supposed to be used by bot to request messages
/// that are not processed yet.
@@ -1211,13 +1204,6 @@ ORDER BY m.timestamp DESC,m.id DESC",
/// shortly after notification or notification is manually triggered
/// to interrupt waiting.
/// Notification may be manually triggered by calling [`Self::stop_io`].
///
/// Deprecated 2026-04: This returns the message's id as soon as the first part arrives,
/// even if it is not fully downloaded yet.
/// The bot needs to wait for the message to be fully downloaded.
/// Since this is usually not the desired behavior,
/// bots should instead use the #DC_EVENT_INCOMING_MSG / [`EventType::IncomingMsg`]
/// event for getting notified about new messages.
pub async fn wait_next_msgs(&self) -> Result<Vec<MsgId>> {
self.new_msgs_notify.notified().await;
let list = self.get_next_msgs().await?;

View File

@@ -730,19 +730,10 @@ impl Imap {
info!(context, "{message_id:?} is a post-message.");
available_post_msgs.push(message_id.clone());
let is_bot = context.get_config_bool(Config::Bot).await?;
if is_bot && download_limit.is_none_or(|download_limit| size <= download_limit)
{
uids_fetch.push(uid);
uid_message_ids.insert(uid, message_id);
} else {
if download_limit.is_none_or(|download_limit| size <= download_limit) {
// Download later after all the small messages are downloaded,
// so that large messages don't delay receiving small messages
download_later.push(message_id.clone());
}
largest_uid_skipped = Some(uid);
if download_limit.is_none_or(|download_limit| size <= download_limit) {
download_later.push(message_id.clone());
}
largest_uid_skipped = Some(uid);
} else {
info!(context, "{message_id:?} is not a post-message.");
if download_limit.is_none_or(|download_limit| size <= download_limit) {

View File

@@ -1137,16 +1137,4 @@ mod tests {
Ok(())
}
/// Tests importing a backup from Delta Chat 1.30.3 for Android (core v1.86.0).
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_import_ancient_backup() -> Result<()> {
let mut tcm = TestContextManager::new();
let context = &tcm.unconfigured().await;
let backup_path = Path::new("test-data/core-1.86.0-backup.tar");
imex(context, ImexMode::ImportBackup, backup_path, None).await?;
Ok(())
}
}

View File

@@ -1319,7 +1319,7 @@ impl Message {
}
/// Force the message to be sent in plain text.
pub(crate) fn force_plaintext(&mut self) {
pub fn force_plaintext(&mut self) {
self.param.set_int(Param::ForcePlaintext, 1);
}

View File

@@ -232,7 +232,11 @@ impl MimeFactory {
if chat.is_self_talk() {
to.push((from_displayname.to_string(), from_addr.to_string()));
encryption_pubkeys = Some(Vec::new());
encryption_pubkeys = if msg.param.get_bool(Param::ForcePlaintext).unwrap_or(false) {
None
} else {
Some(Vec::new())
};
} else if chat.is_mailing_list() {
let list_post = chat
.param

View File

@@ -86,9 +86,7 @@ pub(crate) struct MimeMessage {
/// messages to this address to post them to the list.
pub list_post: Option<String>,
pub chat_disposition_notification_to: Option<SingleInfo>,
/// Decryption error if decryption of the message has failed.
pub decryption_error: Option<String>,
pub decrypting_failed: bool,
/// Valid signature fingerprint if a message is an
/// Autocrypt encrypted and signed message and corresponding intended recipient fingerprints
@@ -373,7 +371,7 @@ impl MimeMessage {
hop_info += "\n\n";
hop_info += &dkim_results.to_string();
let from_is_not_self_addr = !context.is_self_addr(&from.addr).await?;
let incoming = !context.is_self_addr(&from.addr).await?;
let mut aheader_values = mail.headers.get_all_values(HeaderDef::Autocrypt.into());
@@ -438,7 +436,7 @@ impl MimeMessage {
};
let mut autocrypt_header = None;
if from_is_not_self_addr {
if incoming {
// See `get_all_addresses_from_header()` for why we take the last valid header.
for val in aheader_values.iter().rev() {
autocrypt_header = match Aheader::from_str(val) {
@@ -469,7 +467,7 @@ impl MimeMessage {
None
};
let mut public_keyring = if from_is_not_self_addr {
let mut public_keyring = if incoming {
if let Some(autocrypt_header) = autocrypt_header {
vec![autocrypt_header.public_key]
} else {
@@ -654,15 +652,6 @@ impl MimeMessage {
.into_iter()
.last()
.map(|(fp, recipient_fps)| (fp, recipient_fps.into_iter().collect::<HashSet<_>>()));
let incoming = if let Some((ref sig_fp, _)) = signature {
sig_fp.hex() != key::self_fingerprint(context).await?
} else {
// rare case of getting a cleartext message
// so we determine 'incoming' flag by From-address
from_is_not_self_addr
};
let mut parser = MimeMessage {
parts: Vec::new(),
headers,
@@ -675,7 +664,7 @@ impl MimeMessage {
from,
incoming,
chat_disposition_notification_to,
decryption_error: mail.err().map(|err| format!("{err:#}")),
decrypting_failed: mail.is_err(),
// only non-empty if it was a valid autocrypt message
signature,
@@ -916,7 +905,7 @@ impl MimeMessage {
&& let Some(ref subject) = self.get_subject()
{
let mut prepend_subject = true;
if self.decryption_error.is_none() {
if !self.decrypting_failed {
let colon = subject.find(':');
if colon == Some(2)
|| colon == Some(3)
@@ -957,7 +946,7 @@ impl MimeMessage {
self.parse_attachments();
// See if an MDN is requested from the other side
if self.decryption_error.is_none()
if !self.decrypting_failed
&& !self.parts.is_empty()
&& let Some(ref dn_to) = self.chat_disposition_notification_to
{
@@ -1089,7 +1078,7 @@ impl MimeMessage {
#[cfg(test)]
/// Returns whether the decrypted data contains the given `&str`.
pub(crate) fn decoded_data_contains(&self, s: &str) -> bool {
assert!(self.decryption_error.is_none());
assert!(!self.decrypting_failed);
let decoded_str = str::from_utf8(&self.decoded_data).unwrap();
decoded_str.contains(s)
}

View File

@@ -19,6 +19,7 @@ use tokio_io_timeout::TimeoutStream;
use url::Url;
use crate::config::Config;
use crate::constants::NON_ALPHANUMERIC_WITHOUT_DOT;
use crate::context::Context;
use crate::net::connect_tcp;
use crate::net::session::SessionStream;
@@ -92,12 +93,13 @@ impl HttpConfig {
}
fn to_url(&self, scheme: &str) -> String {
let host = utf8_percent_encode(&self.host, NON_ALPHANUMERIC_WITHOUT_DOT);
if let Some((user, password)) = &self.user_password {
let user = utf8_percent_encode(user, NON_ALPHANUMERIC);
let password = utf8_percent_encode(password, NON_ALPHANUMERIC);
format!("{scheme}://{user}:{password}@{}:{}", self.host, self.port)
format!("{scheme}://{user}:{password}@{host}:{}", self.port)
} else {
format!("{scheme}://{}:{}", self.host, self.port)
format!("{scheme}://{host}:{}", self.port)
}
}
}
@@ -141,12 +143,13 @@ impl Socks5Config {
}
fn to_url(&self) -> String {
let host = utf8_percent_encode(&self.host, NON_ALPHANUMERIC_WITHOUT_DOT);
if let Some((user, password)) = &self.user_password {
let user = utf8_percent_encode(user, NON_ALPHANUMERIC);
let password = utf8_percent_encode(password, NON_ALPHANUMERIC);
format!("socks5://{user}:{password}@{}:{}", self.host, self.port)
format!("socks5://{user}:{password}@{host}:{}", self.port)
} else {
format!("socks5://{}:{}", self.host, self.port)
format!("socks5://{host}:{}", self.port)
}
}
}
@@ -562,20 +565,6 @@ mod tests {
user_password: None
})
);
let proxy_config = ProxyConfig::from_url("socks5://my-proxy.example.org").unwrap();
assert_eq!(
proxy_config,
ProxyConfig::Socks5(Socks5Config {
host: "my-proxy.example.org".to_string(),
port: 1080,
user_password: None
})
);
assert_eq!(
proxy_config.to_url(),
"socks5://my-proxy.example.org:1080".to_string()
);
}
#[test]
@@ -609,20 +598,6 @@ mod tests {
user_password: None
})
);
let proxy_config = ProxyConfig::from_url("http://my-proxy.example.org").unwrap();
assert_eq!(
proxy_config,
ProxyConfig::Http(HttpConfig {
host: "my-proxy.example.org".to_string(),
port: 80,
user_password: None
})
);
assert_eq!(
proxy_config.to_url(),
"http://my-proxy.example.org:80".to_string()
);
}
#[test]
@@ -656,20 +631,6 @@ mod tests {
user_password: None
})
);
let proxy_config = ProxyConfig::from_url("https://my-proxy.example.org").unwrap();
assert_eq!(
proxy_config,
ProxyConfig::Https(HttpConfig {
host: "my-proxy.example.org".to_string(),
port: 443,
user_password: None
})
);
assert_eq!(
proxy_config.to_url(),
"https://my-proxy.example.org:443".to_string()
);
}
#[test]

View File

@@ -892,32 +892,6 @@ async fn test_set_proxy_config_from_qr() -> Result<()> {
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_dont_encode_hyphen_in_proxy_hostnames() -> Result<()> {
let mut tcm = TestContextManager::new();
let t = &tcm.alice().await;
let qr_text = "socks5://my-proxy.example.org";
let qr = check_qr(t, qr_text).await?;
assert_eq!(
qr,
Qr::Proxy {
url: "socks5://my-proxy.example.org".to_string(),
host: "my-proxy.example.org".to_string(),
port: 1080,
}
);
set_config_from_qr(t, "socks5://my-proxy.example.org").await?;
assert_eq!(
t.get_config(Config::ProxyUrl).await?,
Some("socks5://my-proxy.example.org:1080".to_string())
);
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_decode_shadowsocks() -> Result<()> {
let ctx = TestContext::new().await;

View File

@@ -14,9 +14,7 @@ use mailparse::SingleInfo;
use num_traits::FromPrimitive;
use regex::Regex;
use crate::chat::{
self, Chat, ChatId, ChatIdBlocked, ChatVisibility, is_contact_in_chat, save_broadcast_secret,
};
use crate::chat::{self, Chat, ChatId, ChatIdBlocked, ChatVisibility, save_broadcast_secret};
use crate::config::Config;
use crate::constants::{self, Blocked, Chattype, DC_CHAT_ID_TRASH, EDITED_PREFIX, ShowEmails};
use crate::contact::{self, Contact, ContactId, Origin, mark_contact_id_as_verified};
@@ -47,7 +45,6 @@ use crate::securejoin::{
self, get_secure_join_step, handle_securejoin_handshake, observe_securejoin_on_other_device,
};
use crate::simplify;
use crate::smtp::msg_has_pending_smtp_job;
use crate::stats::STATISTICS_BOT_EMAIL;
use crate::stock_str;
use crate::sync::Sync::*;
@@ -585,7 +582,14 @@ pub(crate) async fn receive_imf_inner(
(rfc724_mid_orig, &self_addr),
)
.await?;
if !msg_has_pending_smtp_job(context, msg_id).await? {
if !context
.sql
.exists(
"SELECT COUNT(*) FROM smtp WHERE rfc724_mid=?",
(rfc724_mid_orig,),
)
.await?
{
msg_id.set_delivered(context).await?;
}
return Ok(None);
@@ -723,7 +727,7 @@ pub(crate) async fn receive_imf_inner(
let show_emails = ShowEmails::from_i32(context.get_config_int(Config::ShowEmails).await?)
.unwrap_or_default();
let allow_creation = if mime_parser.decryption_error.is_some() {
let allow_creation = if mime_parser.decrypting_failed {
false
} else if is_dc_message == MessengerMessage::No
&& !context.get_config_bool(Config::IsChatmail).await?
@@ -878,23 +882,11 @@ UPDATE config SET value=? WHERE keyname='configured_addr' AND value!=?1
.is_some_and(|part| part.typ == Viewtype::Webxdc)
{
can_info_msg = false;
if mime_parser.pre_message == PreMessageMode::Post
&& let Some(msg_id) = message::rfc724_mid_exists(context, rfc724_mid_orig).await?
{
// The messsage is a post-message and pre-message exists.
// Assign status update to existing message because just received post-message will be trashed.
Some(
Message::load_from_db(context, msg_id)
.await
.context("Failed to load webxdc instance that we just checked exists")?,
)
} else {
Some(
Message::load_from_db(context, insert_msg_id)
.await
.context("Failed to load just created webxdc instance")?,
)
}
Some(
Message::load_from_db(context, insert_msg_id)
.await
.context("Failed to load just created webxdc instance")?,
)
} else if let Some(field) = mime_parser.get_header(HeaderDef::InReplyTo) {
if let Some(instance) =
message::get_by_rfc724_mids(context, &parse_message_ids(field)).await?
@@ -1018,11 +1010,11 @@ UPDATE config SET value=? WHERE keyname='configured_addr' AND value!=?1
&& msg.chat_visibility == ChatVisibility::Archived;
updated_chats
.entry(msg.chat_id)
.and_modify(|pos| *pos = cmp::max(*pos, (msg.timestamp_sort, msg.id)))
.or_insert((msg.timestamp_sort, msg.id));
.and_modify(|ts| *ts = cmp::max(*ts, msg.timestamp_sort))
.or_insert(msg.timestamp_sort);
}
}
for (chat_id, (timestamp_sort, msg_id)) in updated_chats {
for (chat_id, timestamp_sort) in updated_chats {
context
.sql
.execute(
@@ -1031,13 +1023,12 @@ UPDATE msgs SET state=? WHERE
state=? AND
hidden=0 AND
chat_id=? AND
(timestamp,id)<(?,?)",
timestamp<?",
(
MessageState::InNoticed,
MessageState::InFresh,
chat_id,
timestamp_sort,
msg_id,
),
)
.await
@@ -1071,12 +1062,7 @@ UPDATE msgs SET state=? WHERE
let fresh = received_msg.state == MessageState::InFresh
&& mime_parser.is_system_message != SystemMessage::CallAccepted
&& mime_parser.is_system_message != SystemMessage::CallEnded;
let is_bot = context.get_config_bool(Config::Bot).await?;
let is_pre_message = matches!(mime_parser.pre_message, PreMessageMode::Pre { .. });
let skip_bot_notify = is_bot && is_pre_message;
let important =
mime_parser.incoming && fresh && !is_old_contact_request && !skip_bot_notify;
let important = mime_parser.incoming && fresh && !is_old_contact_request;
for msg_id in &received_msg.msg_ids {
chat_id.emit_msg_event(context, *msg_id, important);
}
@@ -1225,18 +1211,14 @@ async fn decide_chat_assignment(
{
info!(context, "Call state changed (TRASH).");
true
} else if let Some(ref decryption_error) = mime_parser.decryption_error
&& !mime_parser.incoming
{
} else if mime_parser.decrypting_failed && !mime_parser.incoming {
// Outgoing undecryptable message.
let last_time = context
.get_config_i64(Config::LastCantDecryptOutgoingMsgs)
.await?;
let now = tools::time();
let update_config = if last_time.saturating_add(24 * 60 * 60) <= now {
let txt = format!(
"⚠️ It seems you are using Delta Chat on multiple devices that cannot decrypt each other's outgoing messages. To fix this, on the older device use \"Settings / Add Second Device\" and follow the instructions. (Error: {decryption_error}, {rfc724_mid})."
);
let txt = "⚠️ It seems you are using Delta Chat on multiple devices that cannot decrypt each other's outgoing messages. To fix this, on the older device use \"Settings / Add Second Device\" and follow the instructions.";
let mut msg = Message::new_text(txt.to_string());
chat::add_device_msg(context, None, Some(&mut msg))
.await
@@ -2310,7 +2292,7 @@ RETURNING id
if trash { 0 } else { ephemeral_timestamp },
if trash {
DownloadState::Done
} else if mime_parser.decryption_error.is_some() {
} else if mime_parser.decrypting_failed {
DownloadState::Undecipherable
} else if let PreMessageMode::Pre {..} = mime_parser.pre_message {
DownloadState::Available
@@ -2374,7 +2356,7 @@ RETURNING id
info!(
context,
"Message has {icnt} parts and is assigned to chat #{chat_id}, timestamp={sort_timestamp}."
"Message has {icnt} parts and is assigned to chat #{chat_id}."
);
if !chat_id.is_trash() && !hidden {
@@ -2580,22 +2562,7 @@ WHERE id=?
),
)
.await?;
if context.get_config_bool(Config::Bot).await? {
if original_msg.hidden {
// No need to emit an event about the changed message
} else if !original_msg.chat_id.is_trash() {
let fresh = original_msg.state == MessageState::InFresh;
let important = mime_parser.incoming && fresh;
original_msg
.chat_id
.emit_msg_event(context, original_msg.id, important);
context.new_msgs_notify.notify_one();
}
} else {
context.emit_msgs_changed(original_msg.chat_id, original_msg.id);
}
context.emit_msgs_changed(original_msg.chat_id, original_msg.id);
Ok(())
}
@@ -2738,7 +2705,7 @@ async fn lookup_or_create_adhoc_group(
allow_creation: bool,
create_blocked: Blocked,
) -> Result<Option<(ChatId, Blocked, bool)>> {
if mime_parser.decryption_error.is_some() {
if mime_parser.decrypting_failed {
warn!(
context,
"Not creating ad-hoc group for message that cannot be decrypted."
@@ -2960,7 +2927,7 @@ async fn create_group(
if let Some(chat_id) = chat_id {
Ok(Some((chat_id, chat_id_blocked)))
} else if mime_parser.decryption_error.is_some() {
} else if mime_parser.decrypting_failed {
// It is possible that the message was sent to a valid,
// yet unknown group, which was rejected because
// Chat-Group-Name, which is in the encrypted part, was
@@ -3155,18 +3122,17 @@ async fn apply_group_changes(
}
}
apply_chat_name_avatar_and_description_changes(
context,
mime_parser,
from_id,
is_from_in_chat,
chat,
&mut send_event_chat_modified,
&mut better_msg,
)
.await?;
if is_from_in_chat {
apply_chat_name_avatar_and_description_changes(
context,
mime_parser,
from_id,
chat,
&mut send_event_chat_modified,
&mut better_msg,
)
.await?;
// Avoid insertion of `from_id` into a group with inappropriate encryption state.
if from_is_key_contact != chat.grpid.is_empty()
&& chat.member_list_is_stale(context).await?
@@ -3340,7 +3306,6 @@ async fn apply_chat_name_avatar_and_description_changes(
context: &Context,
mime_parser: &MimeMessage,
from_id: ContactId,
is_from_in_chat: bool,
chat: &mut Chat,
send_event_chat_modified: &mut bool,
better_msg: &mut Option<String>,
@@ -3369,8 +3334,7 @@ async fn apply_chat_name_avatar_and_description_changes(
let chat_group_name_timestamp = chat.param.get_i64(Param::GroupNameTimestamp).unwrap_or(0);
let group_name_timestamp = group_name_timestamp.unwrap_or(mime_parser.timestamp_sent);
// To provide group name consistency, compare names if timestamps are equal.
if is_from_in_chat
&& (chat_group_name_timestamp, grpname) < (group_name_timestamp, &chat.name)
if (chat_group_name_timestamp, grpname) < (group_name_timestamp, &chat.name)
&& chat
.id
.update_timestamp(context, Param::GroupNameTimestamp, group_name_timestamp)
@@ -3391,19 +3355,14 @@ async fn apply_chat_name_avatar_and_description_changes(
.get_header(HeaderDef::ChatGroupNameChanged)
.is_some()
{
if is_from_in_chat {
let old_name = &sanitize_single_line(old_name);
better_msg.get_or_insert(
if matches!(chat.typ, Chattype::InBroadcast | Chattype::OutBroadcast) {
stock_str::msg_broadcast_name_changed(context, old_name, grpname)
} else {
stock_str::msg_grp_name(context, old_name, grpname, from_id).await
},
);
} else {
// Attempt to change group name by non-member, trash it.
*better_msg = Some(String::new());
}
let old_name = &sanitize_single_line(old_name);
better_msg.get_or_insert(
if matches!(chat.typ, Chattype::InBroadcast | Chattype::OutBroadcast) {
stock_str::msg_broadcast_name_changed(context, old_name, grpname)
} else {
stock_str::msg_grp_name(context, old_name, grpname, from_id).await
},
);
}
}
@@ -3426,8 +3385,7 @@ async fn apply_chat_name_avatar_and_description_changes(
let new_timestamp = timestamp_in_header.unwrap_or(mime_parser.timestamp_sent);
// To provide consistency, compare descriptions if timestamps are equal.
if is_from_in_chat
&& (old_timestamp, &old_description) < (new_timestamp, &new_description)
if (old_timestamp, &old_description) < (new_timestamp, &new_description)
&& chat
.id
.update_timestamp(context, Param::GroupDescriptionTimestamp, new_timestamp)
@@ -3448,13 +3406,8 @@ async fn apply_chat_name_avatar_and_description_changes(
.get_header(HeaderDef::ChatGroupDescriptionChanged)
.is_some()
{
if is_from_in_chat {
better_msg
.get_or_insert(stock_str::msg_chat_description_changed(context, from_id).await);
} else {
// Attempt to change group description by non-member, trash it.
*better_msg = Some(String::new());
}
better_msg
.get_or_insert(stock_str::msg_chat_description_changed(context, from_id).await);
}
}
@@ -3464,46 +3417,39 @@ async fn apply_chat_name_avatar_and_description_changes(
&& value == "group-avatar-changed"
&& let Some(avatar_action) = &mime_parser.group_avatar
{
if is_from_in_chat {
// this is just an explicit message containing the group-avatar,
// apart from that, the group-avatar is send along with various other messages
better_msg.get_or_insert(
if matches!(chat.typ, Chattype::InBroadcast | Chattype::OutBroadcast) {
stock_str::msg_broadcast_img_changed(context)
} else {
match avatar_action {
AvatarAction::Delete => {
stock_str::msg_grp_img_deleted(context, from_id).await
}
AvatarAction::Change(_) => {
stock_str::msg_grp_img_changed(context, from_id).await
}
// this is just an explicit message containing the group-avatar,
// apart from that, the group-avatar is send along with various other messages
better_msg.get_or_insert(
if matches!(chat.typ, Chattype::InBroadcast | Chattype::OutBroadcast) {
stock_str::msg_broadcast_img_changed(context)
} else {
match avatar_action {
AvatarAction::Delete => stock_str::msg_grp_img_deleted(context, from_id).await,
AvatarAction::Change(_) => {
stock_str::msg_grp_img_changed(context, from_id).await
}
},
);
} else {
// Attempt to change group avatar by non-member, trash it.
*better_msg = Some(String::new());
}
}
},
);
}
if let Some(avatar_action) = &mime_parser.group_avatar
&& is_from_in_chat
&& chat
if let Some(avatar_action) = &mime_parser.group_avatar {
info!(context, "Group-avatar change for {}.", chat.id);
if chat
.param
.update_timestamp(Param::AvatarTimestamp, mime_parser.timestamp_sent)?
{
info!(context, "Group-avatar change for {}.", chat.id);
match avatar_action {
AvatarAction::Change(profile_image) => {
chat.param.set(Param::ProfileImage, profile_image);
}
AvatarAction::Delete => {
chat.param.remove(Param::ProfileImage);
}
};
chat.update_param(context).await?;
*send_event_chat_modified = true;
{
match avatar_action {
AvatarAction::Change(profile_image) => {
chat.param.set(Param::ProfileImage, profile_image);
}
AvatarAction::Delete => {
chat.param.remove(Param::ProfileImage);
}
};
chat.update_param(context).await?;
*send_event_chat_modified = true;
}
}
Ok(())
@@ -3617,14 +3563,7 @@ async fn create_or_lookup_mailinglist_or_broadcast(
chattype,
&listid,
name,
if chattype == Chattype::InBroadcast {
// If we joined the broadcast, we have scanned a QR code.
// Even if 1:1 chat does not exist or is in a contact request,
// create the channel as unblocked.
Blocked::Not
} else {
create_blocked
},
create_blocked,
param,
mime_parser.timestamp_sent,
)
@@ -3810,12 +3749,10 @@ async fn apply_out_broadcast_changes(
let mut added_removed_id: Option<ContactId> = None;
if from_id == ContactId::SELF {
let is_from_in_chat = true;
apply_chat_name_avatar_and_description_changes(
context,
mime_parser,
from_id,
is_from_in_chat,
chat,
&mut send_event_chat_modified,
&mut better_msg,
@@ -3904,12 +3841,10 @@ async fn apply_in_broadcast_changes(
let mut send_event_chat_modified = false;
let mut better_msg = None;
let is_from_in_chat = is_contact_in_chat(context, chat.id, from_id).await?;
apply_chat_name_avatar_and_description_changes(
context,
mime_parser,
from_id,
is_from_in_chat,
chat,
&mut send_event_chat_modified,
&mut better_msg,

View File

@@ -13,10 +13,9 @@ use crate::constants::DC_GCL_FOR_FORWARDING;
use crate::contact;
use crate::imap::prefetch_should_download;
use crate::imex::{ImexMode, imex};
use crate::key;
use crate::securejoin::get_securejoin_qr;
use crate::test_utils::{
E2EE_INFO_MSGS, TestContext, TestContextManager, alice_keypair, get_chat_msg, mark_as_verified,
E2EE_INFO_MSGS, TestContext, TestContextManager, get_chat_msg, mark_as_verified,
};
use crate::tools::{SystemTime, time};
@@ -3330,7 +3329,7 @@ async fn test_outgoing_undecryptable() -> Result<()> {
assert!(
dev_msg
.text
.starts_with("⚠️ It seems you are using Delta Chat on multiple devices that cannot decrypt each other's outgoing messages. To fix this, on the older device use \"Settings / Add Second Device\" and follow the instructions. (Error:")
.contains("⚠️ It seems you are using Delta Chat on multiple devices that cannot decrypt each other's outgoing messages. To fix this, on the older device use \"Settings / Add Second Device\" and follow the instructions.")
);
let raw = include_bytes!("../../test-data/message/thunderbird_encrypted_signed.eml");
@@ -4378,42 +4377,39 @@ async fn test_recreate_member_list_on_missing_add_of_self() -> Result<()> {
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_keep_member_list_if_possibly_nomember() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
let bob = &tcm.bob().await;
let alice_chat_id = create_group(alice, "Group").await?;
let alice = TestContext::new_alice().await;
let bob = TestContext::new_bob().await;
let alice_chat_id = create_group(&alice, "Group").await?;
add_contact_to_chat(
alice,
&alice,
alice_chat_id,
alice.add_or_lookup_contact_id(bob).await,
alice.add_or_lookup_contact_id(&bob).await,
)
.await?;
send_text_msg(alice, alice_chat_id, "populate".to_string()).await?;
send_text_msg(&alice, alice_chat_id, "populate".to_string()).await?;
let bob_chat_id = bob.recv_msg(&alice.pop_sent_msg().await).await.chat_id;
let fiona = &tcm.fiona().await;
let fiona = TestContext::new_fiona().await;
add_contact_to_chat(
alice,
&alice,
alice_chat_id,
alice.add_or_lookup_contact_id(fiona).await,
alice.add_or_lookup_contact_id(&fiona).await,
)
.await?;
let fiona_chat_id = fiona.recv_msg(&alice.pop_sent_msg().await).await.chat_id;
fiona_chat_id.accept(fiona).await?;
fiona_chat_id.accept(&fiona).await?;
SystemTime::shift(Duration::from_secs(60));
chat::set_chat_name(fiona, fiona_chat_id, "Renamed").await?;
// Message about chat name change from non-member is trashed.
bob.recv_msg_trash(&fiona.pop_sent_msg().await).await;
chat::set_chat_name(&fiona, fiona_chat_id, "Renamed").await?;
bob.recv_msg(&fiona.pop_sent_msg().await).await;
// Bob missed the message adding fiona, but mustn't recreate the member list or apply the group
// name change.
assert_eq!(get_chat_contacts(bob, bob_chat_id).await?.len(), 2);
assert!(is_contact_in_chat(bob, bob_chat_id, ContactId::SELF).await?);
let bob_alice_contact = bob.add_or_lookup_contact_id(alice).await;
assert!(is_contact_in_chat(bob, bob_chat_id, bob_alice_contact).await?);
let chat = Chat::load_from_db(bob, bob_chat_id).await?;
assert_eq!(get_chat_contacts(&bob, bob_chat_id).await?.len(), 2);
assert!(is_contact_in_chat(&bob, bob_chat_id, ContactId::SELF).await?);
let bob_alice_contact = bob.add_or_lookup_contact_id(&alice).await;
assert!(is_contact_in_chat(&bob, bob_chat_id, bob_alice_contact).await?);
let chat = Chat::load_from_db(&bob, bob_chat_id).await?;
assert_eq!(chat.get_name(), "Group");
Ok(())
}
@@ -5366,46 +5362,6 @@ async fn test_outgoing_unencrypted_chat_assignment() {
assert_eq!(received.chat_id, chat.id);
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_incoming_reply_with_date_in_past() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
let msg0 = receive_imf(
alice,
b"From: bob@example.net\n\
To: alice@example.org\n\
Message-ID: <message@example.net>\n\
Date: Sun, 22 Mar 2020 22:22:22 +0000\n\
\n\
This device has an atomic clock\n",
false,
)
.await?
.unwrap();
let msg1 = receive_imf(
alice,
b"From: bob@example.net\n\
To: alice@example.org\n\
Message-ID: <message1@example.net>\n\
In-Reply-To: <message@example.net>\n\
Date: Sun, 22 Mar 2020 11:11:11 +0000\n\
\n\
And this one has a wind-up clock\n",
false,
)
.await?
.unwrap();
assert_eq!(msg1.chat_id, msg0.chat_id);
assert!(msg1.sort_timestamp >= msg0.sort_timestamp);
assert_eq!(
alice.get_last_msg_in(msg0.chat_id).await.id,
*msg1.msg_ids.last().unwrap()
);
Ok(())
}
/// Tests Bob receiving a message from Alice
/// in a new group she just created
/// with only Alice and Bob.
@@ -5605,90 +5561,3 @@ async fn test_calendar_alternative() -> Result<()> {
Ok(())
}
/// Tests that outgoing encrypted messages are detected
/// by verifying own signature, completely ignoring From address.
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_outgoing_determined_by_signature() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
let bob = &tcm.bob().await;
// alice_dev2: same key, different address.
let different_from = "very@different.from";
assert!(!alice.is_self_addr(different_from).await?);
let alice_dev2 = &tcm.unconfigured().await;
alice_dev2.configure_addr(different_from).await;
key::store_self_keypair(alice_dev2, &alice_keypair()).await?;
assert_ne!(
alice.get_config(Config::Addr).await?.unwrap(),
different_from
);
// Send message from alice_dev2 and check alice sees it as outgoing
let chat_id = alice_dev2.create_chat_id(bob).await;
let sent_msg = alice_dev2.send_text(chat_id, "hello from new device").await;
let msg = alice.recv_msg(&sent_msg).await;
assert_eq!(msg.state, MessageState::OutDelivered);
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_mark_message_as_delivered_only_after_sent_out_fully() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
let bob = &tcm.bob().await;
alice.set_config_bool(Config::BccSelf, true).await?;
let alice_chat_id = alice.create_chat_id(bob).await;
let file_bytes = include_bytes!("../../test-data/image/screenshot.gif");
let mut msg = Message::new(Viewtype::Image);
msg.set_file_from_bytes(alice, "a.jpg", file_bytes, None)?;
let msg_id = chat::send_msg(alice, alice_chat_id, &mut msg)
.await
.unwrap();
let (pre_msg_id, pre_msg_payload) = first_row_in_smtp_queue(alice).await;
assert_eq!(msg_id, pre_msg_id);
assert!(pre_msg_payload.len() < file_bytes.len());
assert_eq!(msg_id.get_state(alice).await?, MessageState::OutPending);
// Alice receives her own pre-message because of bcc_self
// This should not yet mark the message as delivered,
// because not everything was sent,
// but it does remove the pre-message from the SMTP queue
receive_imf(alice, pre_msg_payload.as_bytes(), false).await?;
assert_eq!(msg_id.get_state(alice).await?, MessageState::OutPending);
let (post_msg_id, post_msg_payload) = first_row_in_smtp_queue(alice).await;
assert_eq!(msg_id, post_msg_id);
assert!(post_msg_payload.len() > file_bytes.len());
assert_eq!(msg_id.get_state(alice).await?, MessageState::OutPending);
// Alice receives her own post-message because of bcc_self
// This should now mark the message as delivered,
// because everything was sent by now.
receive_imf(alice, post_msg_payload.as_bytes(), false).await?;
assert_eq!(msg_id.get_state(alice).await?, MessageState::OutDelivered);
Ok(())
}
/// Queries the first sent message in the SMTP queue
/// without removing it from the SMTP queue.
/// This simulates the case that a message is successfully sent out,
/// but the 'OK' answer from the server doesn't arrive,
/// so that the SMTP row stays in the database.
pub(crate) async fn first_row_in_smtp_queue(alice: &TestContext) -> (MsgId, String) {
alice
.sql
.query_row_optional("SELECT msg_id, mime FROM smtp ORDER BY id", (), |row| {
let msg_id: MsgId = row.get(0)?;
let mime: String = row.get(1)?;
Ok((msg_id, mime))
})
.await
.expect("query_row_optional failed")
.expect("No SMTP row found")
}

View File

@@ -450,8 +450,10 @@ pub(crate) async fn handle_securejoin_handshake(
) {
let mut self_found = false;
let self_fingerprint = load_self_public_key(context).await?.dc_fingerprint();
for key in mime_message.gossiped_keys.values() {
if key.public_key.dc_fingerprint() == self_fingerprint {
for (addr, key) in &mime_message.gossiped_keys {
if key.public_key.dc_fingerprint() == self_fingerprint
&& context.is_self_addr(addr).await?
{
self_found = true;
break;
}
@@ -839,6 +841,13 @@ pub(crate) async fn observe_securejoin_on_other_device(
inviter_progress(context, contact_id, chat_id, chat_type)?;
}
if matches!(step, SecureJoinStep::RequestWithAuth) {
// This actually reflects what happens on the first device (which does the secure
// join) and causes a subsequent "vg-member-added" message to create an unblocked
// verified group.
ChatId::create_for_contact_with_blocked(context, contact_id, Blocked::Not).await?;
}
if matches!(step, SecureJoinStep::MemberAdded) {
Ok(HandshakeMessage::Propagate)
} else {

View File

@@ -465,7 +465,11 @@ pub(crate) async fn send_msg_to_smtp(
match status {
SendResult::Retry => Err(format_err!("Retry")),
SendResult::Success => {
if !msg_has_pending_smtp_job(context, msg_id).await? {
if !context
.sql
.exists("SELECT COUNT(*) FROM smtp WHERE msg_id=?", (msg_id,))
.await?
{
msg_id.set_delivered(context).await?;
}
Ok(())
@@ -474,16 +478,6 @@ pub(crate) async fn send_msg_to_smtp(
}
}
pub(crate) async fn msg_has_pending_smtp_job(
context: &Context,
msg_id: MsgId,
) -> Result<bool, Error> {
context
.sql
.exists("SELECT COUNT(*) FROM smtp WHERE msg_id=?", (msg_id,))
.await
}
/// Attempts to send queued MDNs.
async fn send_mdns(context: &Context, connection: &mut Smtp) -> Result<()> {
loop {

View File

@@ -16,6 +16,7 @@ use crate::constants::ShowEmails;
use crate::context::Context;
use crate::key::DcKey;
use crate::log::warn;
use crate::message::MsgId;
use crate::provider::get_provider_info;
use crate::sql::Sql;
use crate::tools::{Time, inc_and_check, time_elapsed};
@@ -733,6 +734,12 @@ impl Sql {
Ok(())
}
async fn set_db_version_in_cache(&self, version: i32) -> Result<()> {
let mut lock = self.config_cache.write().await;
lock.insert(VERSION_CFG.to_string(), Some(format!("{version}")));
Ok(())
}
async fn execute_migration(&self, query: &str, version: i32) -> Result<()> {
self.execute_migration_transaction(
|transaction| {
@@ -1605,11 +1612,51 @@ CREATE INDEX msgs_status_updates_index2 ON msgs_status_updates (uid);
.await?;
}
// Migration 108 is removed, it was using high level code
// to split SMTP queue messages into chunks with smaller number of recipients
// and started to fail later as high level code started
// expecting `transports` table that is only added in future migrations.
// Migration 108 was not changing the database schema.
if dbversion < 108 {
let version = 108;
let chunk_size = context.get_max_smtp_rcpt_to().await?;
sql.transaction(move |trans| {
Sql::set_db_version_trans(trans, version)?;
let id_max =
trans.query_row("SELECT IFNULL((SELECT MAX(id) FROM smtp), 0)", (), |row| {
let id_max: i64 = row.get(0)?;
Ok(id_max)
})?;
while let Some((id, rfc724_mid, mime, msg_id, recipients, retries)) = trans
.query_row(
"SELECT id, rfc724_mid, mime, msg_id, recipients, retries FROM smtp \
WHERE id<=? LIMIT 1",
(id_max,),
|row| {
let id: i64 = row.get(0)?;
let rfc724_mid: String = row.get(1)?;
let mime: String = row.get(2)?;
let msg_id: MsgId = row.get(3)?;
let recipients: String = row.get(4)?;
let retries: i64 = row.get(5)?;
Ok((id, rfc724_mid, mime, msg_id, recipients, retries))
},
)
.optional()?
{
trans.execute("DELETE FROM smtp WHERE id=?", (id,))?;
let recipients = recipients.split(' ').collect::<Vec<_>>();
for recipients in recipients.chunks(chunk_size) {
let recipients = recipients.join(" ");
trans.execute(
"INSERT INTO smtp (rfc724_mid, mime, msg_id, recipients, retries) \
VALUES (?, ?, ?, ?, ?)",
(&rfc724_mid, &mime, msg_id, recipients, retries),
)?;
}
}
Ok(())
})
.await
.with_context(|| format!("migration failed for version {version}"))?;
sql.set_db_version_in_cache(version).await?;
}
if dbversion < 109 {
sql.execute_migration(

View File

@@ -57,9 +57,9 @@ async fn test_key_contacts_migration_autocrypt() -> Result<()> {
);
assert_eq!(pgp_bob.get_verifier_id(&t).await?, None);
// Hidden address-contact can't be looked up by name.
// Hidden address-contact can't be looked up.
assert!(
Contact::get_all(&t, constants::DC_GCL_ADDRESS, Some("bob"))
Contact::get_all(&t, constants::DC_GCL_ADDRESS, Some("bob@example.net"))
.await?
.is_empty()
);
@@ -113,8 +113,12 @@ async fn test_key_contacts_migration_email2() -> Result<()> {
)?)).await?;
t.sql.run_migrations(&t).await?;
// Hidden key-contact can't be looked up by name.
assert!(Contact::get_all(&t, 0, Some("bob")).await?.is_empty());
// Hidden key-contact can't be looked up.
assert!(
Contact::get_all(&t, 0, Some("bob@example.net"))
.await?
.is_empty()
);
let pgp_bob = Contact::get_by_id(&t, ContactId::new(11001)).await?;
assert_eq!(pgp_bob.is_key_contact(), true);
assert_eq!(pgp_bob.origin, Origin::Hidden);
@@ -152,9 +156,9 @@ async fn test_key_contacts_migration_verified() -> Result<()> {
t.sql.run_migrations(&t).await?;
// Hidden address-contact can't be looked up by name.
// Hidden address-contact can't be looked up.
assert!(
Contact::get_all(&t, constants::DC_GCL_ADDRESS, Some("bob"))
Contact::get_all(&t, constants::DC_GCL_ADDRESS, Some("bob@example.net"))
.await?
.is_empty()
);

View File

@@ -40,7 +40,6 @@ use crate::message::{Message, MessageState, MsgId, update_msg_state};
use crate::mimeparser::{MimeMessage, SystemMessage};
use crate::receive_imf::receive_imf;
use crate::securejoin::{get_securejoin_qr, join_securejoin};
use crate::smtp::msg_has_pending_smtp_job;
use crate::stock_str::StockStrings;
use crate::tools::time;
@@ -659,7 +658,10 @@ impl TestContext {
.execute("DELETE FROM smtp WHERE id=?;", (rowid,))
.await
.expect("failed to remove job");
if !msg_has_pending_smtp_job(self, msg_id)
if !self
.ctx
.sql
.exists("SELECT COUNT(*) FROM smtp WHERE msg_id=?", (msg_id,))
.await
.expect("Failed to check for more jobs")
{

View File

@@ -4,19 +4,16 @@ use pretty_assertions::assert_eq;
use crate::EventType;
use crate::chat;
use crate::chat::send_msg;
use crate::config::Config;
use crate::contact;
use crate::download::{DownloadState, PRE_MSG_ATTACHMENT_SIZE_THRESHOLD, PostMsgMetadata};
use crate::message::{Message, MessageState, Viewtype, delete_msgs, markseen_msgs};
use crate::mimeparser::MimeMessage;
use crate::param::Param;
use crate::reaction::{get_msg_reactions, send_reaction};
use crate::receive_imf::receive_imf;
use crate::summary::assert_summary_texts;
use crate::test_utils::TestContextManager;
use crate::tests::pre_messages::util::{
big_webxdc_app, send_large_file_message, send_large_image_message, send_large_webxdc_message,
send_large_file_message, send_large_image_message, send_large_webxdc_message,
};
use crate::webxdc::StatusUpdateSerial;
@@ -516,99 +513,6 @@ async fn test_webxdc_update_for_not_downloaded_instance() -> Result<()> {
Ok(())
}
/// Tests receiving of a large webxdc with updates attached to the the .xdc message.
///
/// Updates are sent in a post message and should be assigned to xdc instance
/// once post message is downloaded.
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_webxdc_updates_in_post_message_after_pre_message() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
let bob = &tcm.bob().await;
let alice_chat_id = alice.create_chat_id(bob).await;
let big_webxdc_app = big_webxdc_app().await?;
let mut alice_instance = Message::new(Viewtype::Webxdc);
alice_instance.set_file_from_bytes(alice, "test.xdc", &big_webxdc_app, None)?;
alice_instance.set_text("Test".to_string());
alice_chat_id
.set_draft(alice, Some(&mut alice_instance))
.await?;
alice
.send_webxdc_status_update(alice_instance.id, r#"{"payload":42, "info":"i"}"#)
.await?;
send_msg(alice, alice_chat_id, &mut alice_instance).await?;
let post_message = alice.pop_sent_msg().await;
let pre_message = alice.pop_sent_msg().await;
let bob_instance = bob.recv_msg(&pre_message).await;
assert_eq!(bob_instance.download_state, DownloadState::Available);
bob.recv_msg_trash(&post_message).await;
let bob_instance = Message::load_from_db(bob, bob_instance.id).await?;
assert_eq!(bob_instance.download_state, DownloadState::Done);
assert_eq!(
bob.get_webxdc_status_updates(bob_instance.id, StatusUpdateSerial::new(0))
.await?,
r#"[{"payload":42,"info":"i","serial":1,"max_serial":1}]"#
);
Ok(())
}
/// Tests receiving of a large webxdc post-message with updates attached
/// to the the .xdc post-message when pre-message arrives later.
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_webxdc_updates_in_post_message_without_pre_message() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = &tcm.alice().await;
let bob = &tcm.bob().await;
let alice_chat_id = alice.create_chat_id(bob).await;
let big_webxdc_app = big_webxdc_app().await?;
let mut alice_instance = Message::new(Viewtype::Webxdc);
alice_instance.set_file_from_bytes(alice, "test.xdc", &big_webxdc_app, None)?;
alice_instance.set_text("Test".to_string());
alice_chat_id
.set_draft(alice, Some(&mut alice_instance))
.await?;
alice
.send_webxdc_status_update(alice_instance.id, r#"{"payload":42, "info":"i"}"#)
.await?;
send_msg(alice, alice_chat_id, &mut alice_instance).await?;
let post_message = alice.pop_sent_msg().await;
let pre_message = alice.pop_sent_msg().await;
// Bob receives post-message first.
let bob_instance = bob.recv_msg(&post_message).await;
assert_eq!(bob_instance.download_state, DownloadState::Done);
assert_eq!(
bob.get_webxdc_status_updates(bob_instance.id, StatusUpdateSerial::new(0))
.await?,
r#"[{"payload":42,"info":"i","serial":1,"max_serial":1}]"#
);
// Bob may still receive pre-message later.
bob.recv_msg_trash(&pre_message).await;
let bob_instance = Message::load_from_db(bob, bob_instance.id).await?;
assert_eq!(bob_instance.download_state, DownloadState::Done);
assert_eq!(
bob.get_webxdc_status_updates(bob_instance.id, StatusUpdateSerial::new(0))
.await?,
r#"[{"payload":42,"info":"i","serial":1,"max_serial":1}]"#
);
Ok(())
}
/// Test mark seen pre-message
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_markseen_pre_msg() -> Result<()> {
@@ -797,46 +701,3 @@ async fn test_chatlist_event_on_post_msg_download() -> Result<()> {
Ok(())
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_bot_pre_message_notifications() -> Result<()> {
let mut tcm = TestContextManager::new();
let alice = tcm.alice().await;
let bob = tcm.bob().await;
bob.set_config_bool(Config::Bot, true).await?;
let alice_group_id = alice.create_group_with_members("test group", &[&bob]).await;
let (pre_message, post_message, _alice_msg_id) = send_large_file_message(
&alice,
alice_group_id,
Viewtype::File,
&vec![0u8; (PRE_MSG_ATTACHMENT_SIZE_THRESHOLD + 1) as usize],
)
.await?;
// Bob receives pre-message
bob.evtracker.clear_events();
receive_imf(&bob, pre_message.payload().as_bytes(), false).await?;
// Verify Bob does NOT get an IncomingMsg event for the pre-message
assert!(
bob.evtracker
.get_matching_opt(&bob, |e| matches!(e, EventType::IncomingMsg { .. }))
.await
.is_none()
);
// Bob receives post-message
receive_imf(&bob, post_message.payload().as_bytes(), false).await?;
// Verify Bob DOES get an IncomingMsg event for the complete message
bob.evtracker
.get_matching(|e| matches!(e, EventType::IncomingMsg { .. }))
.await;
let msg = bob.get_last_msg().await;
assert_eq!(msg.download_state, DownloadState::Done);
Ok(())
}

View File

@@ -79,7 +79,7 @@ async fn test_sending_pre_message() -> Result<()> {
);
let decrypted_post_message = bob.parse_msg(post_message).await;
assert!(decrypted_post_message.decryption_error.is_none());
assert_eq!(decrypted_post_message.decrypting_failed, false);
assert_eq!(
decrypted_post_message.header_exists(HeaderDef::ChatPostMessageId),
false

View File

@@ -37,7 +37,10 @@ pub async fn send_large_file_message<'a>(
Ok((pre_message.to_owned(), post_message.to_owned(), msg_id))
}
pub async fn big_webxdc_app() -> Result<Vec<u8>> {
pub async fn send_large_webxdc_message<'a>(
sender: &'a TestContext,
target_chat: ChatId,
) -> Result<(SentMessage<'a>, SentMessage<'a>, MsgId)> {
let futures_cursor = FuturesCursor::new(Vec::new());
let mut buffer = futures_cursor.compat_write();
let mut writer = ZipFileWriter::with_tokio(&mut buffer);
@@ -48,14 +51,7 @@ pub async fn big_webxdc_app() -> Result<Vec<u8>> {
)
.await?;
writer.close().await?;
Ok(buffer.into_inner().into_inner())
}
pub async fn send_large_webxdc_message<'a>(
sender: &'a TestContext,
target_chat: ChatId,
) -> Result<(SentMessage<'a>, SentMessage<'a>, MsgId)> {
let big_webxdc_app = big_webxdc_app().await?;
let big_webxdc_app = buffer.into_inner().into_inner();
send_large_file_message(sender, target_chat, Viewtype::Webxdc, &big_webxdc_app).await
}

Binary file not shown.

Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.5 KiB