Commit Graph

106 Commits

Author SHA1 Message Date
link2xt
bf9eb35f53 feat: add scripts/zig-ffi.sh
This script can be used to compile the libraries
used in the Android application without the NDK.
2023-06-26 15:36:49 +00:00
link2xt
dd806b2d88 test: add make-python-testenv.sh script
This scripts makes it easy to (re)create python testing environment.
2023-06-19 16:24:03 +00:00
link2xt
b4209fac2e ci(concourse): install venv before trying to use it
This is a follow-up to cbe1671104

I changed `apt-get` arguments for x86_64 glibc builds,
but forgot to change it for x86_64 musl, aarch64 glibc and aarch64 musl.

Because of this, `upload-wheels` task failed with a message:
```
The virtual environment was not created successfully because ensurepip is not
available.  On Debian/Ubuntu systems, you need to install the python3-venv
package using the following command.

    apt install python3.11-venv

You may need to use sudo with that command.  After installing the python3-venv
package, recreate your virtual environment.
```
2023-06-16 19:15:05 +00:00
link2xt
cbe1671104 ci(concourse): install devpi in a virtual environment
This commit solves the "error: externally-managed-environment"
which started appearing since Debian 12 release.
`debian` is used as an Docker image to run devpi.
2023-06-13 16:51:53 +00:00
link2xt
a2599ef08a ci: run cargo check with musl libc 2023-06-06 22:45:17 +00:00
link2xt
22d0a4bb32 build: use Rust 1.70.0 to compile deltachat-rpc-server releases 2023-06-06 22:45:17 +00:00
link2xt
f2b02b7bb0 docs: document how to get Concourse CI secrets from pass 2023-06-01 19:50:42 +00:00
link2xt
24d52c5909 build(set_core_version.py): expect release date in the changelog
git-cliff adds the date to the changelog automatically,
so set_core_version.py only needs to check that the date is in place.
2023-05-16 22:47:28 +00:00
link2xt
15ce54edfb Update set_core_version.py with new release process 2023-05-10 12:00:14 +00:00
link2xt
7918a6801e scripts/deny.sh: update package cache before running cargo deny 2023-05-05 21:23:10 +00:00
link2xt
b369a30544 Pass scripts/run-python-test.sh arguments to pytest 2023-04-14 16:24:08 +00:00
iequidoo
c8988f5a55 maybe_add_time_based_warnings(): Use release date instead of the provider DB update one (#4213)
bjoern <r10s@b44t.com> wrote:
> maybe_add_time_based_warnings() requires some date guaranteed to be in the near past. based on
this known date we check if the system clock is wrong (if earlier than known date) and if the used
Delta Chat version may be outdated (1 year passed since known date). while this does not catch all
situations, it catches quite some errors with comparable few effort.
>
> figuring out the date guaranteed to be in the near past is a bit tricky. that time, we added
get_provider_update_timestamp() for exactly that purpose - it is checked manually by some dev and
updated from time to time, usually before a release.
>
> however, meanwhile, the provider-db gets updated less frequently - things might be settled a bit
more - and, get_provider_update_timestamp() was also changed to return the date of the last commit,
instead of last run. while that seem to be more on-purpose, we cannot even do an “empty” database
update to update the known date.
>
> as get_provider_update_timestamp() is not used for anything else, maybe we should completely
remove that function and replace it by get_last_release_timestamp that is then updated by
./scripts/set_core_version.py - the result of that is reviewed manually anyway, so that seems to be
a good place (i prefer manual review here and mistrust further automation as also dev or ci clocks
may be wrong :)
2023-04-06 22:00:02 -04:00
link2xt
fd7cc83537 Merge tag 'v1.112.4'
Release 1.112.4
2023-03-31 01:13:56 +00:00
link2xt
6c57bc9438 Fix call to auditwheel in scripts/run_all.sh
This bugs prevents CI from building wheels.
2023-03-31 01:02:41 +00:00
link2xt
ae5f72cf4f Remove install_python_bindings.py script and update docs
`install_python_bindings.py` was not used by CI
and scripts, except for `scripts/run-python-test.sh`
which only used it to invoke `cargo`.

Instead of using an additional script,
run cargo directly.

The documentation is updated to remove
references to `install_python_bindings.py`.
The section "Installing bindings from source"
was in fact incorrect as it suggested
running `tox --devenv` first and only then
compiling the library with `install_python_bindings.py`.
Now it explicitly says to run cargo first
and then install the package without using `tox`.
`tox` is still used for running the tests
and setting up development environment.
2023-03-31 00:27:35 +00:00
link2xt
4223cac7a5 set_core_version.py: add newline to the end of package.json 2023-03-30 20:38:47 +00:00
link2xt
3400f5641e Build zig-rpc-server for i686
cargo-zigbuild is replaced with a simple zig-cc wrapper.
cargo-zigbuild still tries to use i386-linux-musl triple,
while in zig 0.11 x86-linux-musl should be used [1].

This also gives us more control over compilation flags used
and prevents changes due to cargo-zigbuild upgrades,
as its version was not pinned.

[1] Merged PR "all: rename i386 to x86 "
    <https://github.com/ziglang/zig/pull/13101>
2023-03-29 10:56:22 +00:00
link2xt
c84155cbd4 Add scripts/deny.sh
This can be used to manually run `cargo deny`
without specifying all the parameters manually.

Similar to existing scripts/clippy.sh
2023-03-25 19:54:24 +00:00
link2xt
635c73ffc6 python: use locally installed auditwheel
`auditwheel` is a part of manylinux images,
there is no need to create a virtual environment
and install it again.
2023-03-23 23:00:46 +00:00
link2xt
124a0e90e1 Release 1.112.0 2023-03-23 13:34:24 +00:00
link2xt
c4657991c8 ci: build all deltachat-rpc-server binaries without NDK 2023-03-21 22:17:14 +00:00
link2xt
3eadc86217 Update Rust in coredeps docker image to 1.68.0 2023-03-18 21:08:40 +00:00
link2xt
a32a3b8cca Construct HashMaps in provider database from array
This saves 450 lines according to `cargo llvm-lines --release`.
2023-03-17 18:30:37 +00:00
link2xt
73e7ee5c13 Build armv7 deltachat-rpc-server binaries without NDK 2023-03-13 23:09:25 +00:00
link2xt
0ac61690cf Enable all features when running clippy 2023-03-11 12:57:38 +00:00
link2xt
fe60b2dd2d Update scripts/anroid-rpc-server.sh
Ported changes from deltachat-android/scripts/ndk-make.sh
2023-03-04 09:35:56 +00:00
link2xt
7e5a8714a0 Add scripts/codespell.sh and spellcheck 2023-03-03 18:40:36 +00:00
Floris Bruynooghe
f5c334a1e4 ci: Make sure clippy script check everything
Also disable missing_docs_in_private_items as this is like 300+ errors
currently.
2023-03-02 21:33:20 +01:00
link2xt
a2f1df052b Autoformat create-provider-data-rs.py 2023-02-26 21:27:21 +00:00
link2xt
0086232bbb Rename update-provider-database.py into create-provider-data-rs.py 2023-02-26 21:27:21 +00:00
link2xt
8e9bb8b06e Check provider database with CI
scripts/update-provider-database.sh checks out the provider database
and updates data.rs file,
making it easier to update.

CI uses this script to check that checked in data.rs
corresponds to the provider database repository.
2023-02-26 21:27:21 +00:00
link2xt
c02c5ffe2c Format docs_wheels.yml with prettier 2023-02-24 22:31:04 +00:00
link2xt
c8ce4ce5aa Switch to "vX.Y.Z" tagging scheme
Previously we used tags like "1.109.0" and "py-1.109.0".
Since Python bindings are always released
at the same time as the core,
it is easier to use a single tag for them.

"v" prefix is added to make matching by "v*" easier
in CI and scripts.
2023-02-24 16:48:37 +00:00
adbenitez
a2e088b415 Compile deltachat-rpc-server for Android
Co-Authored-By: link2xt <link2xt@testrun.org>
2023-02-24 15:17:40 +00:00
link2xt
880ee63fed Document aarch64-unknown-linux-musl.sh 2023-02-22 20:39:46 +00:00
link2xt
cff4a9dce3 ci: cross-compile deltachat-rpc-server for aarch64 Linux 2023-02-22 19:50:42 +00:00
link2xt
c9db41a7f6 python: build Python 3.11 wheels 2023-02-16 14:36:23 +00:00
link2xt
0885cad089 Add deltachat-repl to scripts/set_core_version.py 2023-02-13 20:32:25 +00:00
link2xt
1a49bc85b8 Add scripts/clippy.sh
The script makes it easier to manually check REPL and benchmark code
with clippy without the need to copy-paste all the flags.
2023-02-11 16:07:52 +00:00
link2xt
fac7b064b4 Refine Python CI
Add lint environment to `deltachat-rpc-client/`
and set line length to 120, same as in `python/`.

Switch from flake8 to ruff.

Fix ruff warnings.
2023-01-20 16:53:21 +00:00
link2xt
0e849609f4 ci: python: do not try to upload source distributions
They are not useful because you still need a statically
linked rust library besides the C FFI module, and
they are not built by tox 4 without changes.
2023-01-09 16:20:56 +00:00
Hocuri
d722b6ba19 Only go to 1.64 for now 2022-12-19 17:15:17 +01:00
Hocuri
04f68fddd9 Go back to Rust 1.61
as 1.65 makes the iOS build fail.

For Android, it would actually be enough to go back to 1.64, but let's
try what's needed for iOS.
2022-12-19 12:41:19 +01:00
link2xt
4df588668a Bump MSRV to 1.61.0
This is required by `image` crate.

Also update current Rust version to 1.65.0.
2022-12-04 12:24:17 +00:00
link2xt
75ed4fe398 JSON-RPC stdio server
It speaks JSON-RPC serialized into JSON Lines over stdio.
2022-11-06 20:33:24 +00:00
Simon Laux
43150195d4 fix set_core_version.py script to also update version in deltachat-jsonrpc/typescript/package.json 2022-09-07 10:40:18 +02:00
link2xt
e7b6c689ce Keep deltachat-jsonrpc version in sync 2022-08-28 17:30:20 +00:00
link2xt
db496b82fb Add --features jsonrpc to scripts/run_all.sh
Python wheels cannot be built without it
as they expect everything defined in deltachat.h
to be in the C library.
2022-08-22 00:10:30 +00:00
link2xt
8b2ece63ab scripts/coverage.sh: enable -Cdebuginfo=2
It is needed to generate .gcno and .gcda files.

Debug info has been disabled in a8c389c3b4
which broke coverage script.
2022-08-14 18:37:12 +00:00
Franz Heinzmann
0887acf1bf Integrate JSON-RPC API in core (#3463)
* integrate json-rpc repo
https://github.com/deltachat/deltachat-jsonrpc

* get target dir from cargo

* fix clippy

* use node 16 in ci
use `npm i` instead of `npm ci`
try fix ci script
and fix a doc comment

* fix get_provider_info docs

* refactor function name

* fix formatting
make test  pass
fix clippy

* update .gitignore

* change now returns event names as id
directly, no conversion method or number ids anymore

also longer timeout for requesting test accounts from mailadm

* fix compile after rebase

* add json api to cffi and expose it in dc node

* add some files to npm ignore
that don't need to be in the npm package

* add jsonrpc crate to set_core_version

* add jsonrpc feature flag

* call a jsonrpc function in segfault example

* break loop on empty response

* fix closing segfault
thanks again to link2xt for figguring this out

* activate other tests again

* remove selectAccount  from highlevel client

* put jsonrpc stuff in own module

* disable jsonrpc by default

* add @deltachat/jsonrpc-client
to make sure its dependencies are installed, too
whwn installing dc-node

* commit types.ts
that dc-node has everything it needs to provide @deltachat/jsonrpc-client
without an extra ts compile step

* improve naming

* Changes for tokio compat, upgrade to yerpc 0.3

This also changes the webserver binary to use axum in place of tide.

* Improvements to typescript package

* Improve docs.

* improve docs, fix example

* Fix CFFI for JSON-RPC changes

* use stable toolchain not 1.56.0

* fix ci

* try to fix ci

* remove emtpy file
allow unused code for new_from_arc

* expose anyhow errors
feature name was wrong

* use multi-threaded runtime in JSON-RPC webserver

* improve test setup and code style

* don't wait for IO on webserver start

* Bump yerpc to 0.3.1 with fix for axum server

* update todo document
remove specific api stuff for now,
we now have the an incremental aproach on moving
not the all at-once effort I though it would be

* remove debug logs

* changelog entry about the jsonrpc

* Fix method name casings and cleanups

* Improve JSON-RPC CI, no need to build things multiple times

* Naming consistency: Use DeltaChat not Deltachat

* Improve documentation

* fix docs

* adress dig's comments
- description in cargo.toml
- impl From<EventType> for EventTypeName
- rename `CommandApi::new_from_arc` -> `CommandApi::from_arc`
- pre-allocate if we know the entry count already
- remove unused enumerate
- remove unused serde attribute comment
- rename `FullChat::from_dc_chat_id` -> `FullChat::try_from_dc_chat_id`

* make it more idiomatic:
rename `ContactObject::from_dc_contact -> `ContactObject::try_from_dc_contact`

* apply link2xt's suggestions:
- unref jsonrpc_instance in same thread it was created in
- increase `max_queue_size` from 1 to 1000

* reintroduce segfault test script

* remove unneeded context
thanks to link2xt for pointing that out

* Update deltachat-ffi/deltachat.h

Co-authored-by: bjoern <r10s@b44t.com>

* Update deltachat-ffi/deltachat.h

Co-authored-by: bjoern <r10s@b44t.com>

* make sure to use dc_str_unref instead of free
on cstrings returned/owned by rust

* Increase online test timeouts for CI

* fix the typos
thanks to ralphtheninja for finding them

* restore same configure behaviour as desktop:
make configure restart io with the old configuration if it had one on error

* found another segfault:
this time in batch_set_config

* remove print from test

* make dcn_json_rpc_request return undefined instead of not returning
this might have been the cause for the second segfault

* add set_config_from_qr to jsonrpc

* add `add_device_message` to jsonrpc

* jsonrpc: add `get_fresh_msgs` and `get_fresh_msg_cnt`

* jsonrpc: add dm_chat_contact to ChatListItemFetchResult

* add webxdc methods to jsonrpc:
- `webxdc_send_status_update`
- `webxdc_get_status_updates`
- `message_get_webxdc_info`

* add `chat_get_media` to jsonrpc
also add viewtype wrapper enum and use it in `MessageObject`,
additionally to using it in `chat_get_media`

* use camelCase in all js object properties

* Add check_qr function to jsonrpc

* Fixed clippy errors and formatting

* Fixed formatting

* fix changelog ordering after rebase

* fix compile after merging in master branch

Co-authored-by: Simon Laux <mobile.info@simonlaux.de>
Co-authored-by: Simon Laux <Simon-Laux@users.noreply.github.com>
Co-authored-by: bjoern <r10s@b44t.com>
Co-authored-by: flipsimon <28535045+flipsimon@users.noreply.github.com>
2022-08-04 16:56:37 +00:00