Commit Graph
54 Commits
Author SHA1 Message Date
iequidoo 4aae48b0a1 Gracefully terminate deltachat-rpc-server on ctrl-c and SIGTERM (#4323) 2023-04-18 12:10:39 -04:00
iequidoo a8b790a5db deltachat-rpc-server: Drop Accounts object when terminating (#4323)
Otherwise it continues to keep the sender side of the event emitter which prevents events_task from
stopping.
2023-04-18 12:10:39 -04:00
iequidoo 4dfe34eedc fix review comments 2023-04-16 15:32:36 -04:00
iequidoo 430a71288f Reduce + recode images to JPEG if they are too huge (#3956)
I.e. > 500K for the balanced media quality and 130K for the worse one. This can remove animation and
transparency from PNG/WebP, but then a user always can send an image as a file.

Also don't reduce wide/high images if they aren't huge. Among other benefits, this way most of PNG
screenshots aren't touched.

Also remove Exif from all images, not from JPEGs only.
2023-04-16 15:32:36 -04:00
iequidoo 350509d5d1 Remove metadata from avatars and JPEG images before sending (#4027)
If there's an Exif, rewrite the file to remove it. This implies recoding now though.
2023-04-16 15:32:36 -04:00
iequidoo f1eeb1df8c Cleanly terminate deltachat-rpc-server (#4234)
Do it as per "Thread safety" section in deltachat-ffi/deltachat.h. Also terminate on ctrl-c.
2023-04-12 17:20:42 -04:00
iequidoo 36bec9c295 Update "accounts.toml" atomically (#4295) 2023-04-06 22:52:25 -04:00
iequidoo c8988f5a55 maybe_add_time_based_warnings(): Use release date instead of the provider DB update one (#4213)
bjoern <r10s@b44t.com> wrote:
> maybe_add_time_based_warnings() requires some date guaranteed to be in the near past. based on
this known date we check if the system clock is wrong (if earlier than known date) and if the used
Delta Chat version may be outdated (1 year passed since known date). while this does not catch all
situations, it catches quite some errors with comparable few effort.
>
> figuring out the date guaranteed to be in the near past is a bit tricky. that time, we added
get_provider_update_timestamp() for exactly that purpose - it is checked manually by some dev and
updated from time to time, usually before a release.
>
> however, meanwhile, the provider-db gets updated less frequently - things might be settled a bit
more - and, get_provider_update_timestamp() was also changed to return the date of the last commit,
instead of last run. while that seem to be more on-purpose, we cannot even do an “empty” database
update to update the known date.
>
> as get_provider_update_timestamp() is not used for anything else, maybe we should completely
remove that function and replace it by get_last_release_timestamp that is then updated by
./scripts/set_core_version.py - the result of that is reviewed manually anyway, so that seems to be
a good place (i prefer manual review here and mistrust further automation as also dev or ci clocks
may be wrong :)
2023-04-06 22:00:02 -04:00
iequidoo 41456aa2ab jsonrpc API: Fix documentation of search_messages() (#4288) 2023-04-03 14:17:39 -04:00
iequidoo b56a7bc139 Fix CHANGELOG.md after c401780c 2023-04-01 09:13:08 -03:00
iequidooandbjoern c401780c15 Compress mime_headers column with HTML emails stored in database (#4077)
Co-authored-by: bjoern <r10s@b44t.com>
2023-03-31 19:45:34 -03:00
iequidoo df96a1daac test_moved_markseen: Expect DC_EVENT_MSGS_CHANGED also 2023-03-31 17:15:02 -03:00
iequidoo 0a2200c2c8 python: Wait for initial Resync in the bot tests 2023-03-31 17:15:02 -03:00
iequidoo eb5ddf270f receive_imf: Mark special messages as seen (#3054)
Exactly: delivery reports, webxdc status updates.
2023-03-25 15:47:38 -03:00
iequidoo 87252ab053 cargo: bump async_zip from 0.0.9 to 0.0.11
Bumps [async_zip](https://github.com/Majored/rs-async-zip) from 0.0.9 to 0.0.11.
- [Release notes](https://github.com/Majored/rs-async-zip/releases)
- [Commits](https://github.com/Majored/rs-async-zip/compare/v0.0.9...v0.0.11)

---
updated-dependencies:
- dependency-name: async_zip
  dependency-type: direct:production
  update-type: version-update:semver-patch
...
2023-03-23 00:33:25 +00:00
iequidoo 6b67f8bcfd Support non-persistent configuration with DELTACHAT_* env #3986
This way we can test some recently added config options that we don't want to expose in UI like
DeleteToTrash or SignUnencrypted. Note that persistent config options like DeleteToTrash should
remain anyway because they allow fine-grained (per-account) control. Having them matters for tests
also.
2023-03-14 12:57:06 -03:00
iequidoo 627cf20074 Housekeeping: delete the blobs backup dir (#4104) 2023-03-03 11:06:41 -03:00
iequidoo b149df1993 test_{delete,trash}_multiple_messages: Continue waiting for events if not all messages disappeared 2023-02-28 15:05:47 -03:00
iequidoo e767d84bea Switch to DEFERRED transactions
We do not make all transactions
[IMMEDIATE](https://www.sqlite.org/lang_transaction.html#deferred_immediate_and_exclusive_transactions)
for more parallelism -- at least read transactions can be made DEFERRED to run in parallel
w/o any drawbacks. But if we make write transactions DEFERRED also w/o any external locking,
then they are upgraded from read to write ones on the first write statement. This has some
drawbacks:
- If there are other write transactions, we block the thread and the db connection until
  upgraded. Also if some reader comes then, it has to get next, less used connection with a
  worse per-connection page cache.
- If a transaction is blocked for more than busy_timeout, it fails with SQLITE_BUSY.
- Configuring busy_timeout is not the best way to manage transaction timeouts, we would
  prefer it to be integrated with Rust/tokio asyncs. Moreover, SQLite implements waiting
  using sleeps.
- If upon a successful upgrade to a write transaction the db has been modified by another
  one, the transaction has to be rolled back and retried. It is an extra work in terms of
  CPU/battery.
- Maybe minor, but we lose some fairness in servicing write transactions, i.e. we service
  them in the order of the first write statement, not in the order they come.
The only pro of making write transactions DEFERRED w/o the external locking is some
parallelism between them. Also we have an option to make write transactions IMMEDIATE, also
w/o the external locking. But then the most of cons above are still valid. Instead, if we
perform all write transactions under an async mutex, the only cons is losing some
parallelism for write transactions.
2023-02-28 15:05:47 -03:00
iequidoo d1923d68a5 Add a config option to sign all messages with Autocrypt header (#3986)
Although it does a little for security, it will help to protect from unwanted server-side
modifications and bugs. And now we have a time to test "multipart/signed" messages compatibility
with other MUAs.
2023-02-25 10:30:35 -03:00
iequidoo 89696582ad mimeparser: Handle headers from the signed part of unencrypted signed message
This makes DC compatible with "multipart/signed" messages thus allowing switching to them someday
from the current "multipart/mixed" unencrypted message format.
2023-02-25 10:30:35 -03:00
iequidoo 15c9efaa95 Update provider/update.py according to the struct Provider changes 2023-02-21 10:53:15 -03:00
iequidoo 3b5227c42a Move strict_tls, max_smtp_rcpt_to from Provider to ProviderOptions 2023-02-20 14:09:27 -03:00
iequidoo 604c4fcb71 Delete messages to the Trash folder for Gmail by default (#3957)
Gmail archives messages marked as `\Deleted` by default if those messages aren't in the Trash. But
if move them to the Trash instead, they will be auto-deleted in 30 days.
2023-02-20 14:09:27 -03:00
iequidoo 19d7632be0 Show non-deltachat emails by default for new installations 2023-02-13 14:48:53 -03:00
iequidoo 4c5ed3df2c Add a test for verified groups with multiple devices on a joining side (#3836)
There was the following bug:
- Bob has two devices, the second is offline.
- Alice creates a verified group and sends a QR invitation to Bob.
- Bob joins the group.
- Bob's second devices goes online, but sees a contact request instead of the verified group.
- The "member added" message is not a system message but a plain text message.
- Sending a message fails as the key is missing -- message info says "proper enc-key for <Alice>
  missing, cannot encrypt".
2023-01-25 20:28:49 -03:00
iequidoo 39601be452 observe_securejoin_on_other_device(): Add handling of "v*-request-with-auth" messages (#3836)
This fixes the case with multiple devices on the joining side: if we observe such a message from
another device, we mark an inviter as verified and an accepted contact thus causing a subsequent
"vg-member-added" message -- in case of a verified group -- to create it properly.
2023-01-25 20:28:49 -03:00
iequidoo 76aaecb8f2 Add gossips to all Securejoin messages (#3836)
It worked before only because of the gossip-by-timeout logic in the branch above. And that is why
not always.
2023-01-25 20:28:49 -03:00
iequidoo 4b2faeedab ChatId::create_multiuser_record(): Log blocked of created chat 2023-01-25 20:28:49 -03:00
iequidoo 3c3710420b Don't unpin chat on sending / receiving not fresh messages (#3967)
This bug was introduced by 3cf78749df - there are three visibility
states: Archived, Pinned and Normal - in the database, this "visibility" is named historically
"archived" ... the original code has an AND archived=1 therefore.
2023-01-18 16:57:38 -03:00
iequidoo 3cf78749df Emit DC_EVENT_MSGS_CHANGED for DC_CHAT_ID_ARCHIVED_LINK when the number of archived chats with
unread messages increases (#3940)
2023-01-16 16:05:47 -03:00
iequidoo badbf766bb Move event emitting for a new message to a separate function 2023-01-16 16:05:47 -03:00
iequidoo 5f883a4445 Prepare to remove "vc-contact-confirm-received", "vg-member-added-received" messages from Securejoin
protocol
2023-01-12 15:13:30 -03:00
iequidoo 8dc6ff268d check_verified_properties(): Don't ignore fails of Peerstate::set_verified()
- Return Result from set_verified() so that it can't be missed.
- Pass Fingerprint to set_verified() by value to avoid cloning it inside. This optimises out an
  extra clone() if we already have a value that can be moved at the caller side. However, this may
  add an extra clone() if set_verified() fails, but let's not optimise the fail scenario.
2023-01-12 15:13:30 -03:00
iequidoo 57d7df530b Add a test that a new verified member is seen on the second device going online (#3836)
- Alice has two devices, the second is offline.
- Alice creates a verified group and sends a QR invitation to Bob.
- Bob joins the group and sends a message there. Alice sees it.
- Alice's second devices goes online, but doesn't see Bob in the group.
2023-01-12 15:13:30 -03:00
iequidoo 13b2fe8d30 import_self_keys(): Log set_self_key() error as DC_EVENT_INFO
It isn't an error actually since we just skip the file.
2023-01-12 15:13:30 -03:00
iequidoo d644988845 Securejoin: Fix adding and handling Autocrypt-Gossip headers (#3836)
- If bcc_self is set, gossip headers must be added despite of the number of group members.
- If another device observes Secure-Join, instead of looking for Secure-Join-Fingerprint in
  "vg-member-added"/"vc-contact-confirm" messages it must use keys from Autocrypt-Gossip headers as
  described in the Countermitm doc
  (https://countermitm.readthedocs.io/en/latest/new.html#joining-a-verified-group-secure-join).
2023-01-12 15:13:30 -03:00
iequidoo 27c6cfc958 Log messages in info!() if DCC_MIME_DEBUG is set
Using println!() leads to reordered output on terminal. Moreover, println!() prints to stdout which
is not for logging.
2023-01-12 15:13:30 -03:00
iequidoo 6d9d31cad1 Add timeouts to HTTP requests (#3908) 2023-01-11 14:29:49 -03:00
iequidoo 6dc790f447 Don't parse the message again after detached signatures validation
If we move the detached signatures validation code out of try_decrypt(), we don't need to convert an
already parsed signed message part to Vec and then parse it back. Also this simplifies the
try_decrypt() semantics and return type. It can't make a good coffee anyway.
2022-12-26 18:48:38 +04:00
iequidoo ba82ce2798 Validate signatures in try_decrypt() even if the message isn't encrypted (#3844)
This way we don't need a separate code path for signatures validation for unencrypted
messages. Also, now we degrade encryption only if there are no valid signatures, so the code for
upgrading encryption back isn't needed.
2022-12-21 13:24:10 -03:00
iequidoo a3fe105256 Prefer encryption for the peer if the message is encrypted or signed with the known key (#3844)
Note that if the message is encrypted, we don't check whether it's signed with an attached key
currently, otherwise a massive refactoring of the code is needed because for encrypted messages a
signature is checked and discarded first now.
2022-12-19 19:33:39 +04:00
iequidoo 03c273e30f Don't send GroupNameChanged message if the group name doesn't change in terms of
improve_single_line_input() (#3650)
2022-12-18 21:27:18 +04:00
iequidoo c3a0bb2b77 Fix cargo clippy and doc errors after Rust update to 1.66 2022-12-16 02:46:04 +04:00
iequidoo 21f1439ad8 Treat attached PGP keys as peer keys with mutual encryption preference (#3778) 2022-12-13 04:57:45 +04:00
iequidoo 4cbcd3c606 Revert "mimeparser: assume all Thunderbird users prefer encryption" except for the test (#3778)
This partially reverts commit b341cfd4d9.
2022-12-13 04:57:45 +04:00
iequidoo 1f14767fe9 Revert "Fix misplaced info! message" (#3778)
This reverts commit 08de326930.
2022-12-13 04:57:45 +04:00
iequidoo de96500c1a Add a test on reactions after a reordering MOVE to DeltaChat folder (#3756) 2022-12-04 04:40:11 +04:00
iequidoo 9b881cdd19 Fetch messages in the order of their INTERNALDATE (#3756)
When a batch of messages is moved from Inbox to DeltaChat folder with a single MOVE command, their
UIDs may be reordered (e.g. Gmail is known for that) which leads to that messages are processed by
receive_imf in the wrong order. But the INTERNALDATE attribute is preserved during a MOVE according
to RFC3501. So, use it for sorting fetched messages.
2022-12-04 04:40:11 +04:00
iequidoo f6a502a8e3 Remove the remaining AsRef<str> (#3669)
Using &str instead of AsRef is better for compile times, binary size and code complexity.
2022-11-28 17:02:05 -03:00
iequidoo 9783da5d8e receive_imf: trim() "Chat-Group-Name{,-Changed}:" headers content (#3650)
It's a w/a for "Space added before long group names after MIME serialization/deserialization"
issue. DC itself never creates group names with leading/trailing whitespace, so it can be safely
removed. On the sender side there's no trim() because group names anyway go through
improve_single_line_input(). And I believe we should send the exact name we have in our db. Also
there's no check for leading/trailing whitespace because there may be existing user databases with
group names having such whitespaces.
2022-11-24 08:18:36 -03:00
iequidoo 0b44886b62 Add a test for bug "Partially downloaded messages are received out of order" (#3688)
+ add Message.download_state property.
2022-11-23 23:51:50 +04:00
iequidoo 36991b5c8a Add Python API to send reactions (#3762) 2022-11-23 23:51:50 +04:00
iequidoo 25be8ccd05 fetch_new_messages(): fetch messages sequentially (#3688)
If we fetch messages out of order, then f.e. reactions don't work because if we process a reaction not yet having the corresponding message processed, the reaction is thrown away.
2022-11-17 15:51:59 -03:00