mirror of
https://github.com/SpartanJ/eepp.git
synced 2026-10-27 00:28:34 +03:00
feat(vfs): add ZIP browsing and optional shared module builds
- Add a read-only ZipProvider with streaming reads, cached metadata, change notifications, and zip:///archive.zip!/member URIs. - Support ZIP navigation and copying files out of archives in eefm. - Extend Zip with explicit read-only access, optional resource registration, and improved stream, CRC, and error handling. - Add EE_VFS_API exports and --with-shared-vfs support; reject combining shared VFS with static eepp. - Fix polling watcher teardown from its own thread. - Add regression tests and document ZIP and shared module behavior.
This commit is contained in:
+63
-1
@@ -2,12 +2,13 @@
|
||||
|
||||
## Introduction
|
||||
|
||||
`eepp-vfs` is an optional static module that gives applications one asynchronous, URI-based API
|
||||
`eepp-vfs` is an optional module that gives applications one asynchronous, URI-based API
|
||||
for local files, packed resources and remote servers:
|
||||
|
||||
| Scheme | Provider | Notes |
|
||||
| --- | --- | --- |
|
||||
| `file://` | `LocalProvider` | Native filesystem (POSIX and Win32). |
|
||||
| `zip://` | `ZipProvider` | Read-only browsing and streaming reads of individual local ZIP archives. |
|
||||
| `res://` | `ResourceProvider` | Read-only view of every open `Pack` (`System::PackFileIndex`). |
|
||||
| `sftp://` | `SFTPProvider` | Built by default (`--without-vfs-sftp` disables it), libssh2 based, honors `~/.ssh/config`. |
|
||||
| `mem://` | `MemoryProvider` | In-memory filesystem, mainly for tests. |
|
||||
@@ -26,6 +27,12 @@ are useful on their own:
|
||||
|
||||
Link the module with `eepp_module_vfs_add()` in Premake and include `<eepp/vfs.hpp>`. The UI model
|
||||
lives in `<eepp/vfs/ui/vfsdirectorymodel.hpp>` and is not part of the aggregate header.
|
||||
The default is `eepp-vfs-static`; generate with `--with-shared-vfs` to build and link `eepp-vfs`
|
||||
as a shared library. Public types and non-inline functions use `EE_VFS_API`, controlled by
|
||||
`EE_VFS_EXPORTS` for the DLL and `EE_VFS_STATIC` for static builds, independently of core eepp
|
||||
exports. The Premake helper sets the consumer flags and dependencies. The MinGW helper accepts
|
||||
`--with-shared-vfs` as well. Combining it with `--with-static-eepp` is rejected: a shared VFS
|
||||
must use the same eepp DLL as the application, including its resource and UI singletons.
|
||||
|
||||
## Tasks, results and threads
|
||||
|
||||
@@ -256,6 +263,61 @@ on the server's `PATH`, or set `SFTPProviderConfig::watchAgentCommand`. The eepp
|
||||
same connection.
|
||||
- The efsw-agent is not deployed automatically.
|
||||
|
||||
## ZIP archives (`zip://`)
|
||||
|
||||
`ZipProvider` is registered by default. Build archive locations with `makeURI()` rather than
|
||||
concatenating paths: the backing file URI and the path inside the archive are separate identities.
|
||||
Locations use `zip:///path/archive.zip!/member`. The host, when present, identifies a local UNC
|
||||
archive. Literal `!` characters in the backing path are doubled, so a directory named `packages!`
|
||||
uses `packages!!/` before the archive boundary. Member names use ordinary URI escaping.
|
||||
`withAppendedSegment()` appends a member; eefm handles crossing the archive root with Up.
|
||||
|
||||
```cpp
|
||||
URI archive = URI::fromFSPath( "/home/me/downloads/package.zip" );
|
||||
URI root = ZipProvider::makeURI( archive );
|
||||
URI folder = root.withAppendedSegment( "docs" );
|
||||
|
||||
vfs.listAll( folder ).then( ui, []( VFSResult<std::vector<VFSFileInfo>>&& entries ) { ... } );
|
||||
vfs.readAll( folder.withAppendedSegment( "README.txt" ) );
|
||||
vfs.copy( folder, URI::fromFSPath( "/home/me/extracted-docs" ) );
|
||||
```
|
||||
|
||||
Directories exist both for explicit directory entries (including empty directories) and implicitly
|
||||
for the parents of files. Listings include uncompressed file sizes and entry timestamps. Permission bits are unspecified
|
||||
(the archive API does not expose them), so copies receive the destination filesystem's defaults. Browsing
|
||||
never registers an archive in `PackRegistry` or `PackFileIndex`, so archive contents do not become
|
||||
application resources. Opening a missing archive reports `NotFound` and never creates a file.
|
||||
|
||||
The provider is read-only: creating, replacing, renaming or deleting entries returns `ReadOnly`.
|
||||
Files stream through independent readers without extracting the entire entry into memory.
|
||||
Sequential reads retain the decompression position; seeking backwards replays the compressed
|
||||
prefix with bounded scratch memory. `Seek` is advertised; `RandomAccess` is not. CRC failures are
|
||||
reported as `IO`, including when the final read requests exactly the remaining file size.
|
||||
|
||||
Changes to the backing file invalidate its cached index for new operations. Open handles keep
|
||||
their original reader and metadata; modifying an archive in place can cause their reads to fail.
|
||||
The normal VFS polling backend detects archive changes. Archives on network mounts use the local
|
||||
I/O scheduler's mount isolation and timeouts.
|
||||
|
||||
Only `file://` backing archives are supported initially. Password-protected entries and unsupported
|
||||
compression methods report `Unsupported`. ZIP symlink attributes are not followed: their stored
|
||||
data is exposed as a regular file. Unsafe names (absolute paths, traversal components, backslashes
|
||||
or drive/stream colons) are excluded. Indexes are limited to 262,144 entries including implicit
|
||||
directories, with at most eight cached indexes; existing handles can retain older indexes.
|
||||
|
||||
In eefm, Enter or double-click on a local `.zip` file enters it. Files inside open through the usual
|
||||
application flow, and copying entries out uses VFS transfers. Up at the archive root returns to its
|
||||
containing folder and selects the archive. History and saved tabs retain archive locations. The
|
||||
location bar displays a readable address such as
|
||||
`zip:///home/me/downloads/package.zip!/docs`, which can also be typed or pasted there.
|
||||
|
||||
For lower-level archive access, construct `Zip archive(false)` to opt out of the resource registry
|
||||
and call `archive.open(path, Zip::OpenMode::ReadOnly)`. `entryCount()`, `entry()` and `openStream()`
|
||||
provide metadata and streaming access. Streams borrow the archive and must be destroyed before it;
|
||||
callers sharing an archive across threads must serialize access. `open()` never creates a missing
|
||||
file; writing starts explicitly with `create()`. The default constructor retains resource-pack
|
||||
registration for the engine and other existing pack consumers.
|
||||
|
||||
## Resources (`res://`)
|
||||
|
||||
`System::PackFileIndex` (formerly `System::VirtualFileSystem`) indexes the files of every open `Pack`. `res:///fonts/a.ttf` maps to the pack path
|
||||
|
||||
@@ -16,12 +16,14 @@ class EE_API IOStreamZip : public IOStream {
|
||||
public:
|
||||
static IOStreamZip* New( Zip* pack, const std::string& path );
|
||||
|
||||
/** @brief Open a file from a zip file
|
||||
/** @brief Open a file from a zip file. The pack must outlive the stream.
|
||||
** @param pack Pack to open from path
|
||||
** @param path Path of the file in the pack file
|
||||
**/
|
||||
IOStreamZip( Zip* pack, const std::string& path );
|
||||
|
||||
IOStreamZip( Zip* pack, Uint64 index );
|
||||
|
||||
virtual ~IOStreamZip();
|
||||
|
||||
ios_size read( char* data, ios_size size );
|
||||
@@ -37,9 +39,10 @@ class EE_API IOStreamZip : public IOStream {
|
||||
bool isOpen();
|
||||
|
||||
protected:
|
||||
std::string mPath;
|
||||
struct zip* mZip;
|
||||
struct zip_file* mFile;
|
||||
Uint64 mIndex;
|
||||
ios_size mSize;
|
||||
ios_size mPos;
|
||||
};
|
||||
|
||||
|
||||
@@ -14,6 +14,9 @@ class EE_API Pack : protected Mutex {
|
||||
public:
|
||||
Pack();
|
||||
|
||||
/** Standalone archives can opt out of PackRegistry and PackFileIndex. */
|
||||
explicit Pack( bool registerResource );
|
||||
|
||||
virtual ~Pack();
|
||||
|
||||
/** Creates a new pack file */
|
||||
@@ -81,6 +84,7 @@ class EE_API Pack : protected Mutex {
|
||||
|
||||
protected:
|
||||
bool mIsOpen;
|
||||
bool mRegisterResource;
|
||||
|
||||
void onPackOpened();
|
||||
|
||||
|
||||
@@ -2,26 +2,66 @@
|
||||
#define EE_SYSTEMCZIP_HPP
|
||||
|
||||
#include <eepp/system/pack.hpp>
|
||||
#include <memory>
|
||||
|
||||
struct zip;
|
||||
|
||||
namespace EE { namespace System {
|
||||
|
||||
/** @brief Zip files package manager. */
|
||||
/** ZIP archive reader/writer and optional resource pack. Use Zip(false) and OpenMode::ReadOnly
|
||||
* for standalone browsing. Entry streams borrow the archive and must be destroyed before it.
|
||||
* Serialize access to an archive and its streams when sharing them between threads. */
|
||||
class EE_API Zip : public Pack {
|
||||
public:
|
||||
static Zip* New();
|
||||
|
||||
struct Entry {
|
||||
std::string name;
|
||||
Uint64 size{ 0 };
|
||||
Int64 modified{ 0 };
|
||||
Uint32 crc{ 0 };
|
||||
};
|
||||
|
||||
enum class OpenMode : Uint8 { ReadOnly, ReadWrite };
|
||||
|
||||
enum class Error : Uint8 {
|
||||
None,
|
||||
NotFound,
|
||||
PermissionDenied,
|
||||
InvalidArchive,
|
||||
Unsupported,
|
||||
ReadOnly,
|
||||
IO
|
||||
};
|
||||
|
||||
Zip();
|
||||
|
||||
explicit Zip( bool registerResource );
|
||||
|
||||
~Zip();
|
||||
|
||||
/** Creates a new pack file */
|
||||
bool create( const std::string& path );
|
||||
|
||||
/** Open a pack file */
|
||||
/** Opens an existing archive for reading and writing. Missing files are never created;
|
||||
* use create() explicitly. */
|
||||
bool open( const std::string& path );
|
||||
|
||||
/** ReadOnly supports Unicode native paths and cannot modify or create an archive. */
|
||||
bool open( const std::string& path, OpenMode mode );
|
||||
|
||||
Uint64 entryCount() const;
|
||||
|
||||
bool entry( Uint64 index, Entry& result );
|
||||
|
||||
/** Opens an entry without extracting it into memory. Backward seeks replay the prefix;
|
||||
* scratch space is bounded. Reads/seeks return -1 on error, including CRC failures. */
|
||||
std::unique_ptr<IOStream> openStream( Uint64 index );
|
||||
|
||||
Error error() const { return mError; }
|
||||
|
||||
const std::string& errorMessage() const { return mErrorMessage; }
|
||||
|
||||
/** Close the pack file */
|
||||
bool close();
|
||||
|
||||
@@ -80,8 +120,20 @@ class EE_API Zip : public Pack {
|
||||
struct zip* mZip;
|
||||
|
||||
std::string mZipPath;
|
||||
std::string mErrorMessage;
|
||||
OpenMode mOpenMode{ OpenMode::ReadWrite };
|
||||
Error mError{ Error::None };
|
||||
|
||||
struct zip* getZip();
|
||||
|
||||
private:
|
||||
bool openReadOnly( const std::string& path );
|
||||
|
||||
void setError( int code, int systemCode = 0 );
|
||||
|
||||
bool writable();
|
||||
|
||||
bool extractStream( IOStream& stream, Uint8* data, size_t size );
|
||||
};
|
||||
|
||||
}} // namespace EE::System
|
||||
|
||||
+64
-23
@@ -193,6 +193,10 @@ newoption {
|
||||
description = "Set the shared data directory (default: /usr/share/ecode)",
|
||||
}
|
||||
newoption { trigger = "with-static-cpp", description = "Builds statically libstdc++" }
|
||||
newoption { trigger = "with-shared-vfs", description = "Builds and links the VFS module as a shared library instead of a static library." }
|
||||
if _OPTIONS["with-static-eepp"] and _OPTIONS["with-shared-vfs"] then
|
||||
error("--with-shared-vfs requires eepp-shared; it cannot be combined with --with-static-eepp")
|
||||
end
|
||||
newoption { trigger = "without-vfs-sftp", description = "Disables sftp:// support in the eepp-vfs module (otherwise the vendored libssh2 is built)." }
|
||||
newoption { trigger = "with-lto", description = "Enables Link-Time Optimization for release builds." }
|
||||
|
||||
@@ -1085,8 +1089,8 @@ function vfs_sftp_enabled()
|
||||
return _OPTIONS["without-vfs-sftp"] == nil and not os.is_real("emscripten")
|
||||
end
|
||||
|
||||
function eepp_module_vfs_add()
|
||||
links { "eepp-vfs-static", "efsw-static" }
|
||||
function eepp_module_vfs_dependencies()
|
||||
links { "efsw-static" }
|
||||
includedirs { "src/modules/vfs/include/", "src/thirdparty/efsw/include" }
|
||||
-- The ssh_config evaluator (always built) resolves host names through Winsock.
|
||||
if os.is_real("windows") or os.is_real("mingw32") or os.is_real("mingw64") then
|
||||
@@ -1111,6 +1115,58 @@ function eepp_module_vfs_add()
|
||||
end
|
||||
end
|
||||
|
||||
function eepp_module_vfs_add()
|
||||
includedirs { "src/modules/vfs/include/", "src/thirdparty/efsw/include" }
|
||||
if vfs_sftp_enabled() then
|
||||
defines { "EE_VFS_SFTP" }
|
||||
end
|
||||
if _OPTIONS["with-shared-vfs"] then
|
||||
links { "eepp-vfs" }
|
||||
else
|
||||
links { "eepp-vfs-static" }
|
||||
defines { "EE_VFS_STATIC" }
|
||||
eepp_module_vfs_dependencies()
|
||||
end
|
||||
end
|
||||
|
||||
function build_vfs_module( shared )
|
||||
local name = shared and "eepp-vfs" or "eepp-vfs-static"
|
||||
project(name)
|
||||
kind(shared and "SharedLib" or "StaticLib")
|
||||
language "C++"
|
||||
set_targetdir("libs/" .. os.get_real() .. "/")
|
||||
includedirs { "include", "src/modules/vfs/include/", "src/modules/vfs/src/", "src/thirdparty/efsw/include" }
|
||||
files { "src/modules/vfs/src/**.cpp" }
|
||||
defines { shared and "EE_VFS_EXPORTS" or "EE_VFS_STATIC" }
|
||||
if shared then
|
||||
eepp_module_vfs_dependencies()
|
||||
links { "eepp-shared" }
|
||||
end
|
||||
if vfs_sftp_enabled() then
|
||||
defines { "EE_VFS_SFTP" }
|
||||
includedirs { "src/thirdparty/libssh2/include" }
|
||||
else
|
||||
excludes { "src/modules/vfs/src/eepp/vfs/sftp/**.cpp" }
|
||||
end
|
||||
if _OPTIONS["with-static-eepp"] then
|
||||
defines { "EE_STATIC" }
|
||||
end
|
||||
if not is_vs() then
|
||||
buildoptions{ "-std=c++20" }
|
||||
else
|
||||
buildoptions{ "/std:c++20" }
|
||||
end
|
||||
build_base_cpp_configuration( name )
|
||||
|
||||
if shared then
|
||||
postsymlinklib("../libs/" .. os.get_real() .. "/", "../../bin/", "eepp-vfs" )
|
||||
postsymlinklib("../libs/" .. os.get_real() .. "/", "../../bin/unit_tests/", "eepp-vfs" )
|
||||
if os.is_real("mingw32") or os.is_real("mingw64") or (os.is("windows") and not is_vs()) then
|
||||
links { "winpthread" }
|
||||
end
|
||||
end
|
||||
end
|
||||
|
||||
function eepp_module_backward_add( ecode_mode )
|
||||
files { "src/modules/backward/src/backward.cpp" }
|
||||
includedirs { "src/thirdparty" }
|
||||
@@ -1761,27 +1817,10 @@ solution "eepp"
|
||||
end
|
||||
build_base_cpp_configuration( "efsw-agent" )
|
||||
|
||||
project "eepp-vfs-static"
|
||||
kind "StaticLib"
|
||||
language "C++"
|
||||
set_targetdir("libs/" .. os.get_real() .. "/")
|
||||
includedirs { "include", "src/modules/vfs/include/", "src/modules/vfs/src/", "src/thirdparty/efsw/include" }
|
||||
files { "src/modules/vfs/src/**.cpp" }
|
||||
if vfs_sftp_enabled() then
|
||||
defines { "EE_VFS_SFTP" }
|
||||
includedirs { "src/thirdparty/libssh2/include" }
|
||||
else
|
||||
excludes { "src/modules/vfs/src/eepp/vfs/sftp/**.cpp" }
|
||||
end
|
||||
if _OPTIONS["with-static-eepp"] then
|
||||
defines { "EE_STATIC" }
|
||||
end
|
||||
if not is_vs() then
|
||||
buildoptions{ "-std=c++20" }
|
||||
else
|
||||
buildoptions{ "/std:c++20" }
|
||||
end
|
||||
build_base_cpp_configuration( "eepp-vfs" )
|
||||
build_vfs_module( false )
|
||||
if _OPTIONS["with-shared-vfs"] then
|
||||
build_vfs_module( true )
|
||||
end
|
||||
|
||||
project "languages-syntax-highlighting-static"
|
||||
kind "StaticLib"
|
||||
@@ -2297,6 +2336,7 @@ solution "eepp"
|
||||
links { "util" }
|
||||
end
|
||||
files { "src/tests/unit_tests/*.cpp",
|
||||
"src/tools/eefm/location.cpp",
|
||||
"src/tools/eproc/process_info.cpp",
|
||||
"src/tools/eproc/process_model.cpp",
|
||||
"src/tools/eproc/process_table_state.cpp",
|
||||
@@ -2313,6 +2353,7 @@ solution "eepp"
|
||||
links { "Cocoa.framework" }
|
||||
end
|
||||
eepp_module_vfs_add()
|
||||
links { "efsw-static" }
|
||||
eepp_module_backward_add( false )
|
||||
|
||||
configuration { "linux" }
|
||||
|
||||
+64
-24
@@ -51,6 +51,10 @@ newoption {
|
||||
description = "Set the shared data directory",
|
||||
}
|
||||
newoption { trigger = "with-static-cpp", description = "Builds statically libstdc++" }
|
||||
newoption { trigger = "with-shared-vfs", description = "Builds and links the VFS module as a shared library instead of a static library." }
|
||||
if _OPTIONS["with-static-eepp"] and _OPTIONS["with-shared-vfs"] then
|
||||
error("--with-shared-vfs requires eepp-shared; it cannot be combined with --with-static-eepp")
|
||||
end
|
||||
newoption { trigger = "without-vfs-sftp", description = "Disables sftp:// support in the eepp-vfs module (otherwise the vendored libssh2 is built)." }
|
||||
newoption { trigger = "with-lto", description = "Enables Link-Time Optimization for release builds." }
|
||||
|
||||
@@ -1006,8 +1010,8 @@ function vfs_sftp_enabled()
|
||||
return _OPTIONS["without-vfs-sftp"] == nil and not os.istarget("emscripten")
|
||||
end
|
||||
|
||||
function eepp_module_vfs_add()
|
||||
links { "eepp-vfs-static", "efsw-static" }
|
||||
function eepp_module_vfs_dependencies()
|
||||
links { "efsw-static" }
|
||||
incdirs { "src/modules/vfs/include/", "src/thirdparty/efsw/include" }
|
||||
-- The ssh_config evaluator (always built) resolves host names through Winsock.
|
||||
filter "system:windows"
|
||||
@@ -1033,6 +1037,57 @@ function eepp_module_vfs_add()
|
||||
end
|
||||
end
|
||||
|
||||
function eepp_module_vfs_add()
|
||||
incdirs { "src/modules/vfs/include/", "src/thirdparty/efsw/include" }
|
||||
if vfs_sftp_enabled() then
|
||||
defines { "EE_VFS_SFTP" }
|
||||
end
|
||||
if _OPTIONS["with-shared-vfs"] then
|
||||
links { "eepp-vfs" }
|
||||
else
|
||||
links { "eepp-vfs-static" }
|
||||
defines { "EE_VFS_STATIC" }
|
||||
eepp_module_vfs_dependencies()
|
||||
end
|
||||
end
|
||||
|
||||
function build_vfs_module( shared )
|
||||
local name = shared and "eepp-vfs" or "eepp-vfs-static"
|
||||
project(name)
|
||||
kind(shared and "SharedLib" or "StaticLib")
|
||||
language "C++"
|
||||
cppdialect "C++20"
|
||||
incdirs { "include", "src/modules/vfs/include/", "src/modules/vfs/src/", "src/thirdparty/efsw/include" }
|
||||
files { "src/modules/vfs/src/**.cpp" }
|
||||
defines { shared and "EE_VFS_EXPORTS" or "EE_VFS_STATIC" }
|
||||
if shared then
|
||||
eepp_module_vfs_dependencies()
|
||||
links { "eepp-shared" }
|
||||
end
|
||||
if vfs_sftp_enabled() then
|
||||
defines { "EE_VFS_SFTP" }
|
||||
incdirs { "src/thirdparty/libssh2/include" }
|
||||
else
|
||||
excludes { "src/modules/vfs/src/eepp/vfs/sftp/**.cpp" }
|
||||
end
|
||||
if _OPTIONS["with-static-eepp"] then
|
||||
defines { "EE_STATIC" }
|
||||
end
|
||||
build_base_cpp_configuration( name )
|
||||
target_dir_lib("")
|
||||
filter "action:not vs*"
|
||||
buildoptions { "-Wall" }
|
||||
filter { "action:export-compile-commands", "system:macosx" }
|
||||
buildoptions { "-std=c++20" }
|
||||
|
||||
if shared then
|
||||
postsymlinklib_arch( "eepp-vfs" )
|
||||
filter { "system:windows", "action:not vs*" }
|
||||
links { "winpthread" }
|
||||
filter {}
|
||||
end
|
||||
end
|
||||
|
||||
function eepp_module_backward_add( ecode_mode )
|
||||
files { "src/modules/backward/src/backward.cpp" }
|
||||
incdirs { "src/thirdparty" }
|
||||
@@ -1202,7 +1257,7 @@ function postsymlinklib_arch(name)
|
||||
postsymlinklib( _MAIN_SCRIPT_DIR .. "/libs/" .. os.target() .. "/arm64/", _MAIN_SCRIPT_DIR .. "/bin/", name, "options:arch=arm64" )
|
||||
postsymlinklib( _MAIN_SCRIPT_DIR .. "/libs/" .. os.target() .. "/arm64/", _MAIN_SCRIPT_DIR .. "/bin/", name, "options:arch=aarch64" )
|
||||
postsymlinklib( _MAIN_SCRIPT_DIR .. "/libs/" .. os.target() .. "/universal/", _MAIN_SCRIPT_DIR .. "/bin/", name, "architecture:universal" )
|
||||
if name == "eepp" then
|
||||
if name == "eepp" or name == "eepp-vfs" then
|
||||
postsymlinklib( _MAIN_SCRIPT_DIR .. "/libs/" .. os.target() .. "/x86/", _MAIN_SCRIPT_DIR .. "/bin/unit_tests/", name, "architecture:x86" )
|
||||
postsymlinklib( _MAIN_SCRIPT_DIR .. "/libs/" .. os.target() .. "/x86_64/", _MAIN_SCRIPT_DIR .. "/bin/unit_tests/", name, "architecture:x86_64" )
|
||||
postsymlinklib( _MAIN_SCRIPT_DIR .. "/libs/" .. os.target() .. "/arm/", _MAIN_SCRIPT_DIR .. "/bin/unit_tests/", name, "architecture:ARM" )
|
||||
@@ -1781,27 +1836,10 @@ workspace "eepp"
|
||||
links { "CoreFoundation.framework", "CoreServices.framework" }
|
||||
filter {}
|
||||
|
||||
project "eepp-vfs-static"
|
||||
kind "StaticLib"
|
||||
language "C++"
|
||||
cppdialect "C++20"
|
||||
incdirs { "include", "src/modules/vfs/include/", "src/modules/vfs/src/", "src/thirdparty/efsw/include" }
|
||||
files { "src/modules/vfs/src/**.cpp" }
|
||||
if vfs_sftp_enabled() then
|
||||
defines { "EE_VFS_SFTP" }
|
||||
incdirs { "src/thirdparty/libssh2/include" }
|
||||
else
|
||||
excludes { "src/modules/vfs/src/eepp/vfs/sftp/**.cpp" }
|
||||
end
|
||||
if _OPTIONS["with-static-eepp"] then
|
||||
defines { "EE_STATIC" }
|
||||
end
|
||||
build_base_cpp_configuration( "eepp-vfs" )
|
||||
target_dir_lib("")
|
||||
filter "action:not vs*"
|
||||
buildoptions { "-Wall" }
|
||||
filter { "action:export-compile-commands", "system:macosx" }
|
||||
buildoptions { "-std=c++20" }
|
||||
build_vfs_module( false )
|
||||
if _OPTIONS["with-shared-vfs"] then
|
||||
build_vfs_module( true )
|
||||
end
|
||||
|
||||
project "languages-syntax-highlighting-static"
|
||||
kind "StaticLib"
|
||||
@@ -2289,6 +2327,7 @@ workspace "eepp"
|
||||
incdirs { "src/modules/eterm/include/", "src/thirdparty", "src/modules/vfs/src/" }
|
||||
language "C++"
|
||||
files { "src/tests/unit_tests/*.cpp",
|
||||
"src/tools/eefm/location.cpp",
|
||||
"src/tools/eproc/process_info.cpp",
|
||||
"src/tools/eproc/process_model.cpp",
|
||||
"src/tools/eproc/process_table_state.cpp",
|
||||
@@ -2317,6 +2356,7 @@ workspace "eepp"
|
||||
links { "util" }
|
||||
filter {}
|
||||
eepp_module_vfs_add()
|
||||
links { "efsw-static" }
|
||||
eepp_module_backward_add( false )
|
||||
|
||||
filter { "system:linux" }
|
||||
|
||||
@@ -3,6 +3,7 @@ cd "$(dirname "$0")" || exit
|
||||
|
||||
ARCH=32
|
||||
BACKEND=sdl2
|
||||
VFS_SHARED_ARG=
|
||||
MAKE_ARGS=()
|
||||
for i in "$@"; do
|
||||
case $i in
|
||||
@@ -10,6 +11,9 @@ for i in "$@"; do
|
||||
CONFIG="${i#*=}"
|
||||
MAKE_ARGS+=("$i")
|
||||
;;
|
||||
--with-shared-vfs)
|
||||
VFS_SHARED_ARG="--with-shared-vfs"
|
||||
;;
|
||||
--backend=*)
|
||||
BACKEND="${i#*=}"
|
||||
;;
|
||||
@@ -78,7 +82,7 @@ detect_backend_name()
|
||||
fi
|
||||
}
|
||||
|
||||
PREMAKE5_ARGS="--file=../../premake5.lua --os=windows --cc=gcc --windows-mingw-build --with-backend=$(detect_backend_arg) $PREMAKE5_ARCH gmake"
|
||||
PREMAKE5_ARGS="--file=../../premake5.lua --os=windows --cc=gcc --windows-mingw-build --with-backend=$(detect_backend_arg) $PREMAKE5_ARCH $VFS_SHARED_ARG gmake"
|
||||
|
||||
if command -v premake5 &> /dev/null
|
||||
then
|
||||
|
||||
@@ -1,7 +1,10 @@
|
||||
#include <eepp/system/iostreamzip.hpp>
|
||||
|
||||
#include <algorithm>
|
||||
#include <array>
|
||||
#include <eepp/system/zip.hpp>
|
||||
#include <libzip/zip.h>
|
||||
#include <libzip/zipint.h>
|
||||
#include <limits>
|
||||
|
||||
namespace EE { namespace System {
|
||||
|
||||
@@ -10,61 +13,62 @@ IOStreamZip* IOStreamZip::New( Zip* pack, const std::string& path ) {
|
||||
}
|
||||
|
||||
IOStreamZip::IOStreamZip( Zip* pack, const std::string& path ) :
|
||||
mPath( path ), mZip( pack->getZip() ), mFile( NULL ), mPos( 0 ) {
|
||||
struct zip_stat zs;
|
||||
int err = zip_stat( mZip, path.c_str(), 0, &zs );
|
||||
IOStreamZip( pack, static_cast<Uint64>( pack->exists( path ) ) ) {}
|
||||
|
||||
if ( !err ) {
|
||||
mFile = zip_fopen_index( mZip, zs.index, 0 );
|
||||
IOStreamZip::IOStreamZip( Zip* pack, Uint64 index ) :
|
||||
mZip( pack->getZip() ), mFile( nullptr ), mIndex( index ), mSize( 0 ), mPos( 0 ) {
|
||||
struct zip_stat info;
|
||||
zip_stat_init( &info );
|
||||
if ( mZip && zip_stat_index( mZip, index, 0, &info ) == 0 &&
|
||||
info.size <= static_cast<Uint64>( std::numeric_limits<ios_size>::max() ) ) {
|
||||
mSize = static_cast<ios_size>( info.size );
|
||||
mFile = zip_fopen_index( mZip, index, 0 );
|
||||
}
|
||||
}
|
||||
|
||||
IOStreamZip::~IOStreamZip() {
|
||||
if ( isOpen() ) {
|
||||
if ( mFile )
|
||||
zip_fclose( mFile );
|
||||
}
|
||||
}
|
||||
|
||||
ios_size IOStreamZip::read( char* data, ios_size size ) {
|
||||
int res = -1;
|
||||
|
||||
if ( isOpen() ) {
|
||||
res = zip_fread( mFile, reinterpret_cast<void*>( &data[0] ), size );
|
||||
|
||||
if ( -1 != res ) {
|
||||
mPos += size;
|
||||
if ( !mFile || size < 0 )
|
||||
return -1;
|
||||
auto count = zip_fread( mFile, data, static_cast<Uint64>( size ) );
|
||||
if ( count > 0 ) {
|
||||
mPos += count;
|
||||
// libzip validates the CRC only on the read past EOF. Validate before reporting the
|
||||
// last bytes: offset-based callers may never request another read on this stream.
|
||||
if ( mPos >= mSize ) {
|
||||
char extra;
|
||||
if ( mPos != mSize || zip_fread( mFile, &extra, 1 ) != 0 )
|
||||
return -1;
|
||||
}
|
||||
}
|
||||
|
||||
return -1 != res ? res : 0;
|
||||
return count;
|
||||
}
|
||||
|
||||
ios_size IOStreamZip::write( const char* data, ios_size size ) {
|
||||
return 0;
|
||||
ios_size IOStreamZip::write( const char*, ios_size ) {
|
||||
return -1;
|
||||
}
|
||||
|
||||
ios_size IOStreamZip::seek( ios_size position ) {
|
||||
if ( isOpen() && mPos != position ) {
|
||||
if ( !mFile || position < 0 || position > mSize )
|
||||
return -1;
|
||||
if ( position < mPos ) {
|
||||
zip_fclose( mFile );
|
||||
|
||||
struct zip_stat zs;
|
||||
int err = zip_stat( mZip, mPath.c_str(), 0, &zs );
|
||||
|
||||
if ( !err ) {
|
||||
mFile = zip_fopen_index( mZip, zs.index, 0 );
|
||||
|
||||
if ( 0 != position ) {
|
||||
ScopedBuffer ptr( position );
|
||||
read( (char*)ptr.get(), position );
|
||||
}
|
||||
|
||||
mPos = position;
|
||||
|
||||
return position;
|
||||
}
|
||||
mFile = zip_fopen_index( mZip, mIndex, 0 );
|
||||
mPos = 0;
|
||||
if ( !mFile )
|
||||
return -1;
|
||||
}
|
||||
|
||||
return 0;
|
||||
std::array<char, 16384> scratch;
|
||||
while ( mPos < position ) {
|
||||
const auto count = std::min<ios_size>( position - mPos, scratch.size() );
|
||||
if ( read( scratch.data(), count ) <= 0 )
|
||||
return -1;
|
||||
}
|
||||
return mPos;
|
||||
}
|
||||
|
||||
ios_size IOStreamZip::tell() {
|
||||
@@ -72,13 +76,11 @@ ios_size IOStreamZip::tell() {
|
||||
}
|
||||
|
||||
ios_size IOStreamZip::getSize() {
|
||||
struct zip_stat zs;
|
||||
int err = zip_stat( mZip, mPath.c_str(), 0, &zs );
|
||||
return !err ? zs.size : 0;
|
||||
return mSize;
|
||||
}
|
||||
|
||||
bool IOStreamZip::isOpen() {
|
||||
return NULL != mFile;
|
||||
return mFile != nullptr;
|
||||
}
|
||||
|
||||
}} // namespace EE::System
|
||||
|
||||
@@ -1,15 +1,19 @@
|
||||
#include <eepp/system/pack.hpp>
|
||||
#include <eepp/system/packregistry.hpp>
|
||||
#include <eepp/system/packfileindex.hpp>
|
||||
#include <eepp/system/packregistry.hpp>
|
||||
|
||||
namespace EE { namespace System {
|
||||
|
||||
Pack::Pack() : Mutex(), mIsOpen( false ) {
|
||||
PackRegistry::instance()->add( this );
|
||||
Pack::Pack() : Pack( true ) {}
|
||||
|
||||
Pack::Pack( bool registerResource ) : mIsOpen( false ), mRegisterResource( registerResource ) {
|
||||
if ( mRegisterResource )
|
||||
PackRegistry::instance()->add( this );
|
||||
}
|
||||
|
||||
Pack::~Pack() {
|
||||
PackRegistry::instance()->remove( this );
|
||||
if ( mRegisterResource )
|
||||
PackRegistry::instance()->remove( this );
|
||||
}
|
||||
|
||||
bool Pack::isOpen() const {
|
||||
@@ -17,11 +21,13 @@ bool Pack::isOpen() const {
|
||||
}
|
||||
|
||||
void Pack::onPackOpened() {
|
||||
PackFileIndex::instance()->onResourceAdd( this );
|
||||
if ( mRegisterResource )
|
||||
PackFileIndex::instance()->onResourceAdd( this );
|
||||
}
|
||||
|
||||
void Pack::onPackClosed() {
|
||||
PackFileIndex::instance()->onResourceRemove( this );
|
||||
if ( mRegisterResource )
|
||||
PackFileIndex::instance()->onResourceRemove( this );
|
||||
}
|
||||
|
||||
}} // namespace EE::System
|
||||
|
||||
+223
-128
@@ -1,8 +1,18 @@
|
||||
#include <eepp/system/zip.hpp>
|
||||
|
||||
#include <algorithm>
|
||||
#include <cerrno>
|
||||
#include <eepp/system/filesystem.hpp>
|
||||
#include <eepp/system/iostreamzip.hpp>
|
||||
#include <eepp/system/zip.hpp>
|
||||
#include <eepp/system/lock.hpp>
|
||||
#include <libzip/zip.h>
|
||||
#include <libzip/zipint.h>
|
||||
#include <limits>
|
||||
|
||||
#if EE_PLATFORM == EE_PLATFORM_WIN
|
||||
#include <io.h>
|
||||
#else
|
||||
#include <unistd.h>
|
||||
#endif
|
||||
|
||||
namespace EE { namespace System {
|
||||
|
||||
@@ -10,56 +20,61 @@ Zip* Zip::New() {
|
||||
return eeNew( Zip, () );
|
||||
}
|
||||
|
||||
Zip::Zip() : mZip( NULL ) {}
|
||||
Zip::Zip() : Zip( true ) {}
|
||||
|
||||
Zip::Zip( bool registerResource ) : Pack( registerResource ), mZip( nullptr ) {}
|
||||
|
||||
Zip::~Zip() {
|
||||
close();
|
||||
}
|
||||
|
||||
bool Zip::create( const std::string& path ) {
|
||||
if ( !FileSystem::fileExists( path ) ) {
|
||||
int err;
|
||||
mZip = zip_open( path.c_str(), ZIP_CREATE, &err );
|
||||
|
||||
if ( 0 == checkPack() ) {
|
||||
mZipPath = path;
|
||||
|
||||
mIsOpen = true;
|
||||
|
||||
onPackOpened();
|
||||
|
||||
return true;
|
||||
}
|
||||
} else {
|
||||
return open( path );
|
||||
if ( !close() )
|
||||
return false;
|
||||
mError = Error::None;
|
||||
mErrorMessage.clear();
|
||||
mOpenMode = OpenMode::ReadWrite;
|
||||
int error = ZIP_ER_OK;
|
||||
mZip = zip_open( path.c_str(), ZIP_CREATE, &error );
|
||||
if ( !mZip ) {
|
||||
setError( error, errno );
|
||||
return false;
|
||||
}
|
||||
|
||||
return false;
|
||||
mZipPath = path;
|
||||
mIsOpen = true;
|
||||
onPackOpened();
|
||||
return true;
|
||||
}
|
||||
|
||||
bool Zip::open( const std::string& path ) {
|
||||
if ( FileSystem::fileExists( path ) ) {
|
||||
int err;
|
||||
mZip = zip_open( path.c_str(), 0, &err );
|
||||
return open( path, OpenMode::ReadWrite );
|
||||
}
|
||||
|
||||
if ( 0 == checkPack() ) {
|
||||
mZipPath = path;
|
||||
|
||||
mIsOpen = true;
|
||||
|
||||
onPackOpened();
|
||||
|
||||
return true;
|
||||
}
|
||||
bool Zip::open( const std::string& path, OpenMode mode ) {
|
||||
if ( !close() )
|
||||
return false;
|
||||
mError = Error::None;
|
||||
mErrorMessage.clear();
|
||||
mOpenMode = mode;
|
||||
if ( mode == OpenMode::ReadOnly ) {
|
||||
if ( !openReadOnly( path ) )
|
||||
return false;
|
||||
} else {
|
||||
return create( path );
|
||||
int error = ZIP_ER_OK;
|
||||
mZip = zip_open( path.c_str(), 0, &error );
|
||||
if ( !mZip ) {
|
||||
setError( error, errno );
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
return false;
|
||||
mZipPath = path;
|
||||
mIsOpen = true;
|
||||
onPackOpened();
|
||||
return true;
|
||||
}
|
||||
|
||||
bool Zip::close() {
|
||||
if ( 0 == checkPack() ) {
|
||||
if ( mZip ) {
|
||||
// zip_close() writes the pending changes. When that fails the archive is left allocated
|
||||
// (and unchanged on disk): release it and report the failure.
|
||||
bool written = zip_close( mZip ) == 0;
|
||||
@@ -81,42 +96,50 @@ bool Zip::close() {
|
||||
return written;
|
||||
}
|
||||
|
||||
return false;
|
||||
return true;
|
||||
}
|
||||
|
||||
bool Zip::addFile( const std::string& path, const std::string& inpack ) {
|
||||
ScopedBuffer file;
|
||||
|
||||
FileSystem::fileGet( path, file );
|
||||
if ( !writable() || !FileSystem::fileGet( path, file ) )
|
||||
return false;
|
||||
|
||||
return addFile( file.get(), file.length(), inpack );
|
||||
}
|
||||
|
||||
bool Zip::addFile( const Uint8* data, const Uint32& dataSize, const std::string& inpack ) {
|
||||
if ( !writable() )
|
||||
return false;
|
||||
if ( 0 == checkPack() ) {
|
||||
struct zip_source* zs = zip_source_buffer( mZip, (const void*)data, dataSize, 0 );
|
||||
|
||||
if ( NULL != zs ) {
|
||||
Int32 Result = (Int32)zip_add( mZip, inpack.c_str(), zs );
|
||||
if ( Result < 0 ) {
|
||||
zip_source_free( zs );
|
||||
return false;
|
||||
}
|
||||
|
||||
std::string path = mZipPath;
|
||||
bool written = close();
|
||||
open( path );
|
||||
bool reopened = open( path );
|
||||
|
||||
if ( -1 != Result && written )
|
||||
if ( written && reopened )
|
||||
return true;
|
||||
} else
|
||||
zip_source_free( zs );
|
||||
}
|
||||
}
|
||||
|
||||
return false;
|
||||
}
|
||||
|
||||
bool Zip::addFile( std::vector<Uint8>& data, const std::string& inpack ) {
|
||||
return addFile( reinterpret_cast<const Uint8*>( &data[0] ), (Uint32)data.size(), inpack );
|
||||
return addFile( data.data(), (Uint32)data.size(), inpack );
|
||||
}
|
||||
|
||||
bool Zip::addFiles( std::map<std::string, std::string> paths ) {
|
||||
if ( !writable() )
|
||||
return false;
|
||||
for ( std::map<std::string, std::string>::iterator itr = paths.begin(); itr != paths.end();
|
||||
++itr )
|
||||
if ( !addFile( itr->first, itr->second ) )
|
||||
@@ -125,6 +148,8 @@ bool Zip::addFiles( std::map<std::string, std::string> paths ) {
|
||||
}
|
||||
|
||||
bool Zip::eraseFile( const std::string& path ) {
|
||||
if ( !writable() )
|
||||
return false;
|
||||
std::vector<std::string> tmpv;
|
||||
tmpv.push_back( path );
|
||||
|
||||
@@ -132,6 +157,8 @@ bool Zip::eraseFile( const std::string& path ) {
|
||||
}
|
||||
|
||||
bool Zip::eraseFiles( const std::vector<std::string>& paths ) {
|
||||
if ( !writable() )
|
||||
return false;
|
||||
Int32 Ex;
|
||||
Uint32 i = 0;
|
||||
|
||||
@@ -150,87 +177,53 @@ bool Zip::eraseFiles( const std::vector<std::string>& paths ) {
|
||||
}
|
||||
|
||||
bool Zip::extractFile( const std::string& path, const std::string& dest ) {
|
||||
lock();
|
||||
|
||||
bool Ret;
|
||||
|
||||
ScopedBuffer data;
|
||||
|
||||
Ret = extractFileToMemory( path, data );
|
||||
|
||||
if ( Ret )
|
||||
FileSystem::fileWrite( dest, data.get(), data.length() );
|
||||
|
||||
unlock();
|
||||
|
||||
return Ret;
|
||||
return extractFileToMemory( path, data ) &&
|
||||
FileSystem::fileWrite( dest, data.get(), data.length() );
|
||||
}
|
||||
|
||||
bool Zip::extractFileToMemory( const std::string& path, std::vector<Uint8>& data ) {
|
||||
lock();
|
||||
|
||||
bool Ret = false;
|
||||
Int32 Pos = exists( path );
|
||||
|
||||
if ( 0 == checkPack() && -1 != Pos ) {
|
||||
|
||||
data.clear();
|
||||
|
||||
struct zip_stat zs;
|
||||
int err = zip_stat( mZip, path.c_str(), 0, &zs );
|
||||
|
||||
if ( !err ) {
|
||||
struct zip_file* zf = zip_fopen_index( mZip, zs.index, 0 );
|
||||
|
||||
if ( NULL != zf ) {
|
||||
data.resize( zs.size );
|
||||
|
||||
int Result =
|
||||
(Int32)zip_fread( zf, reinterpret_cast<void*>( &data[0] ), data.size() );
|
||||
|
||||
zip_fclose( zf );
|
||||
|
||||
if ( -1 != Result )
|
||||
Ret = true;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
unlock();
|
||||
|
||||
return Ret;
|
||||
Lock guard( *this );
|
||||
const auto index = exists( path );
|
||||
if ( index < 0 )
|
||||
return false;
|
||||
IOStreamZip stream( this, index );
|
||||
if ( !stream.isOpen() || static_cast<Uint64>( stream.getSize() ) > data.max_size() )
|
||||
return false;
|
||||
data.resize( static_cast<size_t>( stream.getSize() ) );
|
||||
return extractStream( stream, data.data(), data.size() );
|
||||
}
|
||||
|
||||
bool Zip::extractFileToMemory( const std::string& path, ScopedBuffer& data ) {
|
||||
lock();
|
||||
Lock guard( *this );
|
||||
const auto index = exists( path );
|
||||
if ( index < 0 )
|
||||
return false;
|
||||
IOStreamZip stream( this, index );
|
||||
if ( !stream.isOpen() ||
|
||||
static_cast<Uint64>( stream.getSize() ) > std::numeric_limits<size_t>::max() )
|
||||
return false;
|
||||
data.reset( static_cast<size_t>( stream.getSize() ) );
|
||||
return extractStream( stream, data.get(), data.size() );
|
||||
}
|
||||
|
||||
bool Ret = false;
|
||||
Int32 Pos = exists( path );
|
||||
Int32 Result = 0;
|
||||
|
||||
if ( 0 == checkPack() && -1 != Pos ) {
|
||||
struct zip_stat zs;
|
||||
int err = zip_stat( mZip, path.c_str(), 0, &zs );
|
||||
|
||||
if ( !err ) {
|
||||
struct zip_file* zf = zip_fopen_index( mZip, zs.index, 0 );
|
||||
|
||||
if ( NULL != zf ) {
|
||||
data.reset( zs.size );
|
||||
|
||||
Result = (Int32)zip_fread( zf, (void*)data.get(), data.length() );
|
||||
|
||||
zip_fclose( zf );
|
||||
|
||||
if ( -1 != Result )
|
||||
Ret = true;
|
||||
}
|
||||
bool Zip::extractStream( IOStream& stream, Uint8* data, size_t size ) {
|
||||
size_t done = 0;
|
||||
while ( done < size ) {
|
||||
const auto count = stream.read( reinterpret_cast<char*>( data + done ),
|
||||
std::min<size_t>( size - done, 65536 ) );
|
||||
if ( count <= 0 ) {
|
||||
setError( ZIP_ER_READ );
|
||||
return false;
|
||||
}
|
||||
done += static_cast<size_t>( count );
|
||||
}
|
||||
|
||||
unlock();
|
||||
|
||||
return Ret;
|
||||
char extra;
|
||||
if ( stream.read( &extra, 1 ) != 0 ) {
|
||||
setError( ZIP_ER_CRC );
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
Int32 Zip::exists( const std::string& path ) {
|
||||
@@ -245,21 +238,15 @@ Int8 Zip::checkPack() {
|
||||
}
|
||||
|
||||
std::vector<std::string> Zip::getFileList() {
|
||||
std::vector<std::string> tmpv;
|
||||
|
||||
Int32 numfiles = zip_get_num_files( mZip );
|
||||
|
||||
for ( Int32 i = 0; i < numfiles; i++ ) {
|
||||
struct zip_stat zs;
|
||||
|
||||
if ( -1 != zip_stat_index( mZip, i, 0, &zs ) ) {
|
||||
if ( zs.size > 0 ) {
|
||||
tmpv.push_back( std::string( zs.name ) );
|
||||
}
|
||||
}
|
||||
std::vector<std::string> files;
|
||||
const auto count = entryCount();
|
||||
files.reserve( static_cast<size_t>( count ) );
|
||||
Entry info;
|
||||
for ( Uint64 i = 0; i < count; ++i ) {
|
||||
if ( entry( i, info ) && !info.name.ends_with( '/' ) )
|
||||
files.push_back( std::move( info.name ) );
|
||||
}
|
||||
|
||||
return tmpv;
|
||||
return files;
|
||||
}
|
||||
|
||||
/** @return The file path of the opened package */
|
||||
@@ -275,4 +262,112 @@ zip* Zip::getZip() {
|
||||
return mZip;
|
||||
}
|
||||
|
||||
bool Zip::openReadOnly( const std::string& path ) {
|
||||
// fopenUtf8 preserves Windows Unicode paths. zip_fdopen consumes a duplicated descriptor,
|
||||
// so the FILE owned here can always be closed normally.
|
||||
FILE* file = FileSystem::fopenUtf8( path, "rb" );
|
||||
if ( !file ) {
|
||||
setError( errno == ENOENT ? ZIP_ER_NOENT : ZIP_ER_OPEN, errno );
|
||||
return false;
|
||||
}
|
||||
#if EE_PLATFORM == EE_PLATFORM_WIN
|
||||
int descriptor = _dup( _fileno( file ) );
|
||||
#else
|
||||
int descriptor = dup( fileno( file ) );
|
||||
#endif
|
||||
const int duplicateError = errno;
|
||||
std::fclose( file );
|
||||
if ( descriptor < 0 ) {
|
||||
setError( ZIP_ER_OPEN, duplicateError );
|
||||
return false;
|
||||
}
|
||||
int error = ZIP_ER_OK;
|
||||
mZip = zip_fdopen( descriptor, 0, &error );
|
||||
if ( !mZip ) {
|
||||
// This vendored libzip consumes descriptor after fdopen succeeds. Only its two early
|
||||
// ZIP_ER_OPEN failures (dup/fdopen) leave it for the caller to close.
|
||||
const int systemError = errno;
|
||||
if ( error == ZIP_ER_OPEN ) {
|
||||
#if EE_PLATFORM == EE_PLATFORM_WIN
|
||||
_close( descriptor );
|
||||
#else
|
||||
::close( descriptor );
|
||||
#endif
|
||||
}
|
||||
setError( error, systemError );
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
Uint64 Zip::entryCount() const {
|
||||
if ( !mZip )
|
||||
return 0;
|
||||
const auto count = zip_get_num_entries( mZip, 0 );
|
||||
return count == std::numeric_limits<Uint64>::max() ? 0 : count;
|
||||
}
|
||||
|
||||
bool Zip::entry( Uint64 index, Entry& result ) {
|
||||
struct zip_stat info;
|
||||
zip_stat_init( &info );
|
||||
if ( !mZip || zip_stat_index( mZip, index, 0, &info ) != 0 ) {
|
||||
setError( ZIP_ER_INCONS );
|
||||
return false;
|
||||
}
|
||||
result.name = info.name;
|
||||
result.size = info.size;
|
||||
result.modified = static_cast<Int64>( info.mtime );
|
||||
result.crc = info.crc;
|
||||
return true;
|
||||
}
|
||||
|
||||
std::unique_ptr<IOStream> Zip::openStream( Uint64 index ) {
|
||||
struct zip_stat info;
|
||||
zip_stat_init( &info );
|
||||
if ( !mZip || zip_stat_index( mZip, index, 0, &info ) != 0 ) {
|
||||
setError( ZIP_ER_INCONS );
|
||||
return {};
|
||||
}
|
||||
if ( info.size > static_cast<Uint64>( std::numeric_limits<ios_size>::max() ) ) {
|
||||
setError( ZIP_ER_COMPNOTSUPP );
|
||||
return {};
|
||||
}
|
||||
auto stream = std::make_unique<IOStreamZip>( this, index );
|
||||
if ( !stream->isOpen() ) {
|
||||
int code = 0;
|
||||
int systemCode = 0;
|
||||
zip_error_get( mZip, &code, &systemCode );
|
||||
setError( code, systemCode );
|
||||
return {};
|
||||
}
|
||||
return stream;
|
||||
}
|
||||
|
||||
void Zip::setError( int code, int systemCode ) {
|
||||
if ( code == ZIP_ER_NOENT )
|
||||
mError = Error::NotFound;
|
||||
else if ( zip_error_get_sys_type( code ) == ZIP_ET_SYS &&
|
||||
( systemCode == EACCES || systemCode == EPERM ) )
|
||||
mError = Error::PermissionDenied;
|
||||
else if ( code == ZIP_ER_NOZIP || code == ZIP_ER_INCONS || code == ZIP_ER_EOF )
|
||||
mError = Error::InvalidArchive;
|
||||
else if ( code == ZIP_ER_COMPNOTSUPP || code == ZIP_ER_MULTIDISK ||
|
||||
code == ZIP_ER_ENCRNOTSUPP || code == ZIP_ER_NOPASSWD || code == ZIP_ER_WRONGPASSWD )
|
||||
mError = Error::Unsupported;
|
||||
else
|
||||
mError = Error::IO;
|
||||
char message[256];
|
||||
zip_error_to_str( message, sizeof( message ), code, systemCode );
|
||||
mErrorMessage = message;
|
||||
}
|
||||
|
||||
bool Zip::writable() {
|
||||
if ( mOpenMode == OpenMode::ReadOnly ) {
|
||||
mError = Error::ReadOnly;
|
||||
mErrorMessage = "ZIP archive is read-only";
|
||||
return false;
|
||||
}
|
||||
return mZip != nullptr;
|
||||
}
|
||||
|
||||
}} // namespace EE::System
|
||||
|
||||
@@ -11,6 +11,7 @@
|
||||
#include <eepp/vfs/providers/localprovider.hpp>
|
||||
#include <eepp/vfs/providers/memoryprovider.hpp>
|
||||
#include <eepp/vfs/providers/resourceprovider.hpp>
|
||||
#include <eepp/vfs/providers/zipprovider.hpp>
|
||||
#include <eepp/vfs/transfer.hpp>
|
||||
#include <eepp/vfs/virtualfilesystem.hpp>
|
||||
#include <eepp/vfs/watch.hpp>
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
#ifndef EE_VFS_CAPABILITIES_HPP
|
||||
#define EE_VFS_CAPABILITIES_HPP
|
||||
|
||||
#include <eepp/config.hpp>
|
||||
#include <eepp/vfs/config.hpp>
|
||||
|
||||
namespace EE { namespace VFS {
|
||||
|
||||
@@ -36,7 +36,7 @@ enum class VFSCapability : Uint32 {
|
||||
|
||||
/** Set of VFSCapability flags. Capabilities may vary per URI (for example per SFTP server), so
|
||||
* providers report them per URI. */
|
||||
class ProviderCapabilities {
|
||||
class EE_VFS_API ProviderCapabilities {
|
||||
public:
|
||||
constexpr ProviderCapabilities() noexcept = default;
|
||||
|
||||
|
||||
@@ -0,0 +1,22 @@
|
||||
#ifndef EE_VFS_CONFIG_HPP
|
||||
#define EE_VFS_CONFIG_HPP
|
||||
|
||||
#include <eepp/config.hpp>
|
||||
|
||||
#ifndef EE_VFS_API
|
||||
#if defined( EE_VFS_STATIC )
|
||||
#define EE_VFS_API
|
||||
#elif EE_PLATFORM == EE_PLATFORM_WIN
|
||||
#if defined( EE_VFS_EXPORTS )
|
||||
#define EE_VFS_API __declspec( dllexport )
|
||||
#else
|
||||
#define EE_VFS_API __declspec( dllimport )
|
||||
#endif
|
||||
#elif defined( __GNUC__ ) && __GNUC__ >= 4
|
||||
#define EE_VFS_API __attribute__( ( visibility( "default" ) ) )
|
||||
#else
|
||||
#define EE_VFS_API
|
||||
#endif
|
||||
#endif
|
||||
|
||||
#endif
|
||||
@@ -2,6 +2,7 @@
|
||||
#define EE_VFS_DIRECTORYHANDLE_HPP
|
||||
|
||||
#include <eepp/core/cancellationtoken.hpp>
|
||||
#include <eepp/vfs/config.hpp>
|
||||
#include <eepp/vfs/error.hpp>
|
||||
#include <eepp/vfs/fileinfo.hpp>
|
||||
#include <memory>
|
||||
@@ -14,7 +15,7 @@ namespace EE { namespace VFS {
|
||||
* endOfDirectory is explicit: an empty batch does not imply EOF, and the final batch may carry
|
||||
* entries. "." and ".." are never reported.
|
||||
*/
|
||||
struct DirectoryBatch {
|
||||
struct EE_VFS_API DirectoryBatch {
|
||||
std::vector<VFSFileInfo> entries;
|
||||
bool endOfDirectory{ false };
|
||||
};
|
||||
@@ -23,7 +24,7 @@ struct DirectoryBatch {
|
||||
* a read is pending and must release native resources in their destructor if close() was never
|
||||
* called. Calls are serialized by the handle's single consumer; implementations need not support
|
||||
* concurrent read() calls. */
|
||||
class DirectoryHandleImpl {
|
||||
class EE_VFS_API DirectoryHandleImpl {
|
||||
public:
|
||||
virtual ~DirectoryHandleImpl() = default;
|
||||
|
||||
@@ -37,7 +38,7 @@ class DirectoryHandleImpl {
|
||||
* Move-only. Destroying an open handle releases it without waiting. read() after EOF returns an
|
||||
* empty batch with endOfDirectory set.
|
||||
*/
|
||||
class DirectoryHandle {
|
||||
class EE_VFS_API DirectoryHandle {
|
||||
public:
|
||||
DirectoryHandle() = default;
|
||||
|
||||
|
||||
@@ -1,10 +1,10 @@
|
||||
#ifndef EE_VFS_ERROR_HPP
|
||||
#define EE_VFS_ERROR_HPP
|
||||
|
||||
#include <eepp/config.hpp>
|
||||
#include <eepp/core/result.hpp>
|
||||
#include <eepp/core/task.hpp>
|
||||
#include <eepp/core/uri.hpp>
|
||||
#include <eepp/vfs/config.hpp>
|
||||
#include <string>
|
||||
#include <string_view>
|
||||
|
||||
@@ -38,13 +38,13 @@ enum class VFSErrorCode : Uint8 {
|
||||
};
|
||||
|
||||
/** @return A stable, human readable identifier for code (for example "NotFound"). */
|
||||
const char* toString( VFSErrorCode code );
|
||||
EE_VFS_API const char* toString( VFSErrorCode code );
|
||||
|
||||
/** @brief Typed VFS failure.
|
||||
*
|
||||
* nativeCode keeps the provider error (errno, Win32 error, SFTP status) for diagnostics only.
|
||||
*/
|
||||
struct VFSError {
|
||||
struct EE_VFS_API VFSError {
|
||||
URI uri;
|
||||
std::string message;
|
||||
Int64 nativeCode{ 0 };
|
||||
|
||||
@@ -3,6 +3,7 @@
|
||||
|
||||
#include <eepp/core/cancellationtoken.hpp>
|
||||
#include <eepp/vfs/capabilities.hpp>
|
||||
#include <eepp/vfs/config.hpp>
|
||||
#include <eepp/vfs/error.hpp>
|
||||
#include <eepp/vfs/fileinfo.hpp>
|
||||
#include <memory>
|
||||
@@ -16,7 +17,7 @@ namespace EE { namespace VFS {
|
||||
* task completes. Implementations must release native resources in their destructor if close()
|
||||
* was never called.
|
||||
*/
|
||||
class FileHandleImpl {
|
||||
class EE_VFS_API FileHandleImpl {
|
||||
public:
|
||||
virtual ~FileHandleImpl() = default;
|
||||
|
||||
@@ -44,7 +45,7 @@ class FileHandleImpl {
|
||||
* Offset-based operations avoid a shared mutable seek position, so independent reads can be
|
||||
* pipelined. read() returning 0 bytes for a non-empty buffer means end of file.
|
||||
*/
|
||||
class FileHandle {
|
||||
class EE_VFS_API FileHandle {
|
||||
public:
|
||||
FileHandle() = default;
|
||||
|
||||
|
||||
@@ -3,8 +3,8 @@
|
||||
|
||||
#include <array>
|
||||
#include <cstring>
|
||||
#include <eepp/config.hpp>
|
||||
#include <eepp/core/uri.hpp>
|
||||
#include <eepp/vfs/config.hpp>
|
||||
#include <initializer_list>
|
||||
#include <optional>
|
||||
#include <string>
|
||||
@@ -22,7 +22,7 @@ enum class FileType : Uint8 {
|
||||
Socket,
|
||||
};
|
||||
|
||||
const char* toString( FileType type );
|
||||
EE_VFS_API const char* toString( FileType type );
|
||||
|
||||
/** @brief Opaque, provider-defined revision token.
|
||||
*
|
||||
@@ -33,7 +33,7 @@ const char* toString( FileType type );
|
||||
*
|
||||
* Stored inline (no heap) because it travels inside every VFSFileInfo.
|
||||
*/
|
||||
class FileRevision {
|
||||
class EE_VFS_API FileRevision {
|
||||
public:
|
||||
static constexpr size_t Capacity = 32;
|
||||
|
||||
@@ -92,7 +92,7 @@ class FileRevision {
|
||||
* The entry does not store its URI: listings hold thousands of entries and the caller always
|
||||
* knows the URI it listed or stat'ed. Use uriIn( directory ) for an entry of a listing.
|
||||
*/
|
||||
class VFSFileInfo {
|
||||
class EE_VFS_API VFSFileInfo {
|
||||
public:
|
||||
/** Last path segment (verbatim, not percent-encoded). */
|
||||
std::string name;
|
||||
|
||||
@@ -1,13 +1,13 @@
|
||||
#ifndef EE_VFS_OPTIONS_HPP
|
||||
#define EE_VFS_OPTIONS_HPP
|
||||
|
||||
#include <eepp/config.hpp>
|
||||
#include <eepp/vfs/config.hpp>
|
||||
#include <eepp/vfs/fileinfo.hpp>
|
||||
#include <optional>
|
||||
|
||||
namespace EE { namespace VFS {
|
||||
|
||||
struct StatOptions {
|
||||
struct EE_VFS_API StatOptions {
|
||||
/** When false the entry itself is described (lstat semantics). */
|
||||
bool followSymlinks{ true };
|
||||
};
|
||||
@@ -20,7 +20,7 @@ enum class DirectoryMetadata : Uint8 {
|
||||
Full,
|
||||
};
|
||||
|
||||
struct DirectoryOpenOptions {
|
||||
struct EE_VFS_API DirectoryOpenOptions {
|
||||
DirectoryMetadata metadata{ DirectoryMetadata::Full };
|
||||
/** Resolve the target type of symlinks (VFSFileInfo::targetType). May cost a stat per link. */
|
||||
bool resolveSymlinkTargets{ true };
|
||||
@@ -52,7 +52,7 @@ constexpr bool hasFlag( OpenFlags flags, OpenFlags flag ) noexcept {
|
||||
return ( static_cast<Uint8>( flags ) & static_cast<Uint8>( flag ) ) != 0;
|
||||
}
|
||||
|
||||
struct OpenOptions {
|
||||
struct EE_VFS_API OpenOptions {
|
||||
OpenFlags flags{ OpenFlags::Read };
|
||||
/** Permissions for newly created files; provider default (0666 & ~umask) when unset. */
|
||||
std::optional<Uint32> createPermissions;
|
||||
@@ -62,21 +62,21 @@ struct OpenOptions {
|
||||
OpenOptions( OpenFlags flags ) : flags( flags ) {}
|
||||
};
|
||||
|
||||
struct MkDirOptions {
|
||||
struct EE_VFS_API MkDirOptions {
|
||||
std::optional<Uint32> permissions;
|
||||
};
|
||||
|
||||
struct RemoveOptions {
|
||||
struct EE_VFS_API RemoveOptions {
|
||||
/** VFS-level only: remove directory contents recursively. Providers receive false. */
|
||||
bool recursive{ false };
|
||||
};
|
||||
|
||||
struct RenameOptions {
|
||||
struct EE_VFS_API RenameOptions {
|
||||
/** Replace an existing destination. Atomic when the provider reports AtomicReplace. */
|
||||
bool overwrite{ false };
|
||||
};
|
||||
|
||||
struct WriteOptions {
|
||||
struct EE_VFS_API WriteOptions {
|
||||
/** Write a sibling temporary file and atomically rename it over the destination. */
|
||||
bool atomic{ true };
|
||||
/** When atomic replacement is unsupported, fall back to an in-place write instead of failing
|
||||
@@ -102,7 +102,7 @@ enum class OverwritePolicy : Uint8 {
|
||||
Skip,
|
||||
};
|
||||
|
||||
struct CopyOptions {
|
||||
struct EE_VFS_API CopyOptions {
|
||||
OverwritePolicy overwrite{ OverwritePolicy::Fail };
|
||||
bool recursive{ true };
|
||||
/** Copy permission bits when both sides support them. */
|
||||
|
||||
@@ -3,6 +3,7 @@
|
||||
|
||||
#include <eepp/core/cancellationtoken.hpp>
|
||||
#include <eepp/vfs/capabilities.hpp>
|
||||
#include <eepp/vfs/config.hpp>
|
||||
#include <eepp/vfs/directoryhandle.hpp>
|
||||
#include <eepp/vfs/error.hpp>
|
||||
#include <eepp/vfs/filehandle.hpp>
|
||||
@@ -12,7 +13,7 @@
|
||||
|
||||
namespace EE { namespace VFS {
|
||||
|
||||
struct SpaceInfo {
|
||||
struct EE_VFS_API SpaceInfo {
|
||||
Uint64 capacity{ 0 };
|
||||
Uint64 free{ 0 };
|
||||
/** Free space available to the current (non-privileged) user. */
|
||||
@@ -33,7 +34,7 @@ struct SpaceInfo {
|
||||
* - Cancellation is cooperative: a cancelled operation completes with VFSErrorCode::Cancelled.
|
||||
* - Providers never touch UI or model state.
|
||||
*/
|
||||
class VFSProvider {
|
||||
class EE_VFS_API VFSProvider {
|
||||
public:
|
||||
virtual ~VFSProvider() = default;
|
||||
|
||||
|
||||
@@ -5,6 +5,7 @@
|
||||
#include <eepp/core/task.hpp>
|
||||
#include <eepp/system/threadpool.hpp>
|
||||
#include <eepp/system/time.hpp>
|
||||
#include <eepp/vfs/config.hpp>
|
||||
#include <eepp/vfs/error.hpp>
|
||||
#include <functional>
|
||||
#include <memory>
|
||||
@@ -28,10 +29,10 @@ class MountWatchdog;
|
||||
* The kernel cannot interrupt a call on an unreachable hard NFS mount, so its thread stays blocked;
|
||||
* nobody waits for it anymore, and its result is discarded if it ever arrives.
|
||||
*/
|
||||
class EE_API LocalIOScheduler {
|
||||
class EE_VFS_API LocalIOScheduler {
|
||||
public:
|
||||
/** A file system mounted from another machine. */
|
||||
struct NetworkMount {
|
||||
struct EE_VFS_API NetworkMount {
|
||||
/** Where it is mounted (a native path). */
|
||||
std::string root;
|
||||
/** What is mounted, for messages ("server:/export", "//server/share"). */
|
||||
@@ -40,7 +41,7 @@ class EE_API LocalIOScheduler {
|
||||
|
||||
/** Where the calls of a path run: its network mount's lane, or the shared pool. Resolve it
|
||||
* once for a handle (an open file or directory) and reuse it for the handle's calls. */
|
||||
class Route {
|
||||
class EE_VFS_API Route {
|
||||
public:
|
||||
bool isNetworkMount() const { return mLane != nullptr; }
|
||||
|
||||
@@ -128,7 +129,7 @@ class EE_API LocalIOScheduler {
|
||||
|
||||
/** A call queued on a lane: run once on the lane's thread, or failed by the watchdog, and
|
||||
* only the first of the two completes its task. */
|
||||
class Call {
|
||||
class EE_VFS_API Call {
|
||||
public:
|
||||
virtual ~Call() = default;
|
||||
|
||||
@@ -154,7 +155,7 @@ class EE_API LocalIOScheduler {
|
||||
|
||||
/** What the routing needs, shared with the calls that resolve their route on the pool (which
|
||||
* can outlive the scheduler). */
|
||||
struct Routing {
|
||||
struct EE_VFS_API Routing {
|
||||
MountLookup lookup;
|
||||
std::shared_ptr<MountWatchdog> watchdog;
|
||||
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
#ifndef EE_VFS_PROVIDERS_LOCALPROVIDER_HPP
|
||||
#define EE_VFS_PROVIDERS_LOCALPROVIDER_HPP
|
||||
|
||||
#include <eepp/vfs/config.hpp>
|
||||
#include <eepp/vfs/provider.hpp>
|
||||
#include <eepp/vfs/providers/localioscheduler.hpp>
|
||||
#include <memory>
|
||||
@@ -19,7 +20,7 @@ namespace EE { namespace VFS {
|
||||
* Revisions are derived from device, inode (file index), modification time and size, so both
|
||||
* in-place edits and atomic replacements are detected.
|
||||
*/
|
||||
class LocalProvider : public VFSProvider {
|
||||
class EE_VFS_API LocalProvider : public VFSProvider {
|
||||
public:
|
||||
explicit LocalProvider( std::shared_ptr<LocalIOScheduler> scheduler );
|
||||
|
||||
|
||||
@@ -2,6 +2,7 @@
|
||||
#define EE_VFS_PROVIDERS_MEMORYPROVIDER_HPP
|
||||
|
||||
#include <eepp/core/executor.hpp>
|
||||
#include <eepp/vfs/config.hpp>
|
||||
#include <eepp/vfs/provider.hpp>
|
||||
#include <map>
|
||||
#include <memory>
|
||||
@@ -21,7 +22,7 @@ namespace EE { namespace VFS {
|
||||
*
|
||||
* Open file handles keep their data alive when the file is removed or replaced, like POSIX inodes.
|
||||
*/
|
||||
class MemoryProvider : public VFSProvider {
|
||||
class EE_VFS_API MemoryProvider : public VFSProvider {
|
||||
public:
|
||||
explicit MemoryProvider( std::string scheme = "mem", Executor* completionExecutor = nullptr );
|
||||
|
||||
|
||||
@@ -2,6 +2,7 @@
|
||||
#define EE_VFS_PROVIDERS_RESOURCEPROVIDER_HPP
|
||||
|
||||
#include <eepp/system/threadpool.hpp>
|
||||
#include <eepp/vfs/config.hpp>
|
||||
#include <eepp/vfs/provider.hpp>
|
||||
#include <memory>
|
||||
|
||||
@@ -16,7 +17,7 @@ namespace EE { namespace VFS {
|
||||
* Revisions change whenever a pack is opened or closed. Sizes are reported once a file is opened
|
||||
* (packs expose no cheap size query), so listing and stat leave them unknown.
|
||||
*/
|
||||
class ResourceProvider : public VFSProvider {
|
||||
class EE_VFS_API ResourceProvider : public VFSProvider {
|
||||
public:
|
||||
explicit ResourceProvider( std::shared_ptr<System::ThreadPool> pool );
|
||||
|
||||
|
||||
@@ -0,0 +1,66 @@
|
||||
#ifndef EE_VFS_PROVIDERS_ZIPPROVIDER_HPP
|
||||
#define EE_VFS_PROVIDERS_ZIPPROVIDER_HPP
|
||||
|
||||
#include <eepp/vfs/config.hpp>
|
||||
#include <eepp/vfs/provider.hpp>
|
||||
#include <eepp/vfs/providers/localioscheduler.hpp>
|
||||
#include <memory>
|
||||
|
||||
namespace EE { namespace VFS {
|
||||
|
||||
class ZipProviderImpl;
|
||||
|
||||
/** Read-only zip:// filesystem over individual local archives. Use makeURI() to identify an
|
||||
* archive and a member path: zip:///path/archive.zip!/member. Literal bangs in the backing
|
||||
* path are doubled to distinguish them from the archive boundary.
|
||||
* Metadata is indexed without decompressing entries. Open handles retain their archive snapshot;
|
||||
* new operations notice changes to the backing file. At most eight archive indexes are cached.
|
||||
* Archives on network mounts use LocalIOScheduler's lanes and buffer-safe timeout delivery. */
|
||||
class EE_VFS_API ZipProvider : public VFSProvider {
|
||||
public:
|
||||
explicit ZipProvider( std::shared_ptr<LocalIOScheduler> scheduler );
|
||||
|
||||
~ZipProvider() override;
|
||||
|
||||
/** Builds an archive location. member is an unescaped, '/'-separated path inside it. Only
|
||||
* file:// backing archives are currently supported; other inputs return an empty URI.
|
||||
* Root is "/". */
|
||||
static URI makeURI( const URI& archive, std::string_view member = "/" );
|
||||
|
||||
/** Decodes and validates the backing file:// URI of a zip:// location. */
|
||||
static VFSResult<URI> archiveURI( const URI& uri );
|
||||
|
||||
/** Validates a ZIP location and returns its normalized path within the archive. */
|
||||
static VFSResult<std::string> memberPath( const URI& uri );
|
||||
|
||||
std::string_view scheme() const override { return "zip"; }
|
||||
|
||||
ProviderCapabilities capabilities( const URI& uri ) const override;
|
||||
|
||||
VFSTask<VFSFileInfo> stat( const URI& uri, const StatOptions& options,
|
||||
CancellationToken token ) override;
|
||||
|
||||
VFSTask<DirectoryHandle> openDirectory( const URI& uri, const DirectoryOpenOptions& options,
|
||||
CancellationToken token ) override;
|
||||
|
||||
VFSTask<FileHandle> open( const URI& uri, const OpenOptions& options,
|
||||
CancellationToken token ) override;
|
||||
|
||||
VFSTask<void> mkdir( const URI& uri, const MkDirOptions& options,
|
||||
CancellationToken token ) override;
|
||||
|
||||
VFSTask<void> remove( const URI& uri, const RemoveOptions& options,
|
||||
CancellationToken token ) override;
|
||||
|
||||
VFSTask<void> rename( const URI& from, const URI& to, const RenameOptions& options,
|
||||
CancellationToken token ) override;
|
||||
|
||||
VFSTask<URI> canonicalize( const URI& uri, CancellationToken token ) override;
|
||||
|
||||
private:
|
||||
std::shared_ptr<ZipProviderImpl> mImpl;
|
||||
};
|
||||
|
||||
}} // namespace EE::VFS
|
||||
|
||||
#endif
|
||||
@@ -2,6 +2,7 @@
|
||||
#define EE_VFS_SFTP_SFTPPROVIDER_HPP
|
||||
|
||||
#include <eepp/core/task.hpp>
|
||||
#include <eepp/vfs/config.hpp>
|
||||
#include <eepp/vfs/provider.hpp>
|
||||
#include <eepp/vfs/ssh/sshconfig.hpp>
|
||||
#include <eepp/vfs/watch.hpp>
|
||||
@@ -17,7 +18,7 @@ class VirtualFileSystem;
|
||||
|
||||
/** Unknown host key presented for a user decision. Changed (mismatching) host keys are never
|
||||
* offered: they always fail with VFSErrorCode::HostKeyRejected. */
|
||||
struct SSHHostKeyInfo {
|
||||
struct EE_VFS_API SSHHostKeyInfo {
|
||||
std::string host;
|
||||
/** Key algorithm, for example "ecdsa-sha2-nistp256". */
|
||||
std::string keyType;
|
||||
@@ -37,7 +38,7 @@ enum class SSHHostKeyDecision : Uint8 {
|
||||
};
|
||||
|
||||
/** Application policy for unknown host keys (for example a UI dialog). */
|
||||
class SSHHostKeyVerifier {
|
||||
class EE_VFS_API SSHHostKeyVerifier {
|
||||
public:
|
||||
virtual ~SSHHostKeyVerifier() = default;
|
||||
|
||||
@@ -45,12 +46,12 @@ class SSHHostKeyVerifier {
|
||||
virtual Task<SSHHostKeyDecision> verify( const SSHHostKeyInfo& info ) = 0;
|
||||
};
|
||||
|
||||
struct SSHPrompt {
|
||||
struct EE_VFS_API SSHPrompt {
|
||||
std::string text;
|
||||
bool echo{ false };
|
||||
};
|
||||
|
||||
struct SSHAuthenticationRequest {
|
||||
struct EE_VFS_API SSHAuthenticationRequest {
|
||||
enum class Kind : Uint8 {
|
||||
Password,
|
||||
/** Passphrase of an encrypted identity file. */
|
||||
@@ -74,7 +75,7 @@ struct SSHAuthenticationRequest {
|
||||
|
||||
/** Supplies secrets on demand. Secrets are used for one authentication attempt and never stored
|
||||
* by the VFS (and never appear in URIs). */
|
||||
class SSHAuthenticationHandler {
|
||||
class EE_VFS_API SSHAuthenticationHandler {
|
||||
public:
|
||||
virtual ~SSHAuthenticationHandler() = default;
|
||||
|
||||
@@ -83,7 +84,7 @@ class SSHAuthenticationHandler {
|
||||
request( const SSHAuthenticationRequest& request ) = 0;
|
||||
};
|
||||
|
||||
struct SFTPProviderConfig {
|
||||
struct EE_VFS_API SFTPProviderConfig {
|
||||
std::shared_ptr<SSHHostKeyVerifier> hostKeyVerifier;
|
||||
std::shared_ptr<SSHAuthenticationHandler> authenticationHandler;
|
||||
/** Builds the OpenSSH configuration context for a host. Defaults to
|
||||
@@ -115,7 +116,7 @@ class SFTPProviderImpl;
|
||||
* connection too. All SSH sessions are driven by one I/O thread in non-blocking mode;
|
||||
* provider methods never block the caller.
|
||||
*/
|
||||
class SFTPProvider : public VFSProvider {
|
||||
class EE_VFS_API SFTPProvider : public VFSProvider {
|
||||
public:
|
||||
explicit SFTPProvider( SFTPProviderConfig config = {} );
|
||||
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
#ifndef EE_VFS_SSH_SSHCONFIG_HPP
|
||||
#define EE_VFS_SSH_SSHCONFIG_HPP
|
||||
|
||||
#include <eepp/config.hpp>
|
||||
#include <eepp/system/time.hpp>
|
||||
#include <eepp/vfs/config.hpp>
|
||||
#include <functional>
|
||||
#include <map>
|
||||
#include <memory>
|
||||
@@ -18,7 +18,7 @@ namespace EE { namespace VFS {
|
||||
using EE::System::Time;
|
||||
|
||||
/** @brief OpenSSH-style pattern matching (ssh_config(5) "PATTERNS"). */
|
||||
class SSHPattern {
|
||||
class EE_VFS_API SSHPattern {
|
||||
public:
|
||||
enum class ListMatch : Int8 {
|
||||
/** A negated pattern matched: the whole list is rejected. */
|
||||
@@ -38,7 +38,7 @@ class SSHPattern {
|
||||
};
|
||||
|
||||
/** One "Keyword arguments" line of an ssh_config file. */
|
||||
struct SSHConfigLine {
|
||||
struct EE_VFS_API SSHConfigLine {
|
||||
/** Lowercased keyword. */
|
||||
std::string keyword;
|
||||
/** Arguments split with OpenSSH quoting rules. */
|
||||
@@ -53,7 +53,7 @@ struct SSHConfigLine {
|
||||
* Parsing only tokenizes: directives are kept in file order so the evaluator can apply OpenSSH's
|
||||
* ordering, first-value-wins, Host/Match activation and Include semantics.
|
||||
*/
|
||||
class SSHConfigFile {
|
||||
class EE_VFS_API SSHConfigFile {
|
||||
public:
|
||||
static SSHConfigFile parse( std::string_view text, std::string path = {} );
|
||||
|
||||
@@ -79,7 +79,7 @@ enum class SSHStrictHostKeyChecking : Uint8 {
|
||||
No,
|
||||
};
|
||||
|
||||
struct SSHJumpHost {
|
||||
struct EE_VFS_API SSHJumpHost {
|
||||
std::string user;
|
||||
std::string host;
|
||||
/** 0 when not specified (the jump host's own config / default applies). */
|
||||
@@ -93,7 +93,7 @@ struct SSHJumpHost {
|
||||
* Paths are fully expanded (tilde, tokens, environment). The raw evaluated option values are kept
|
||||
* in `options` for diagnostics and `ssh -G` style dumps.
|
||||
*/
|
||||
struct SSHResolvedConfig {
|
||||
struct EE_VFS_API SSHResolvedConfig {
|
||||
std::string originalHost;
|
||||
/** Effective connection host (HostName applied, canonicalized when enabled). */
|
||||
std::string hostName;
|
||||
@@ -157,7 +157,7 @@ struct SSHResolvedConfig {
|
||||
|
||||
/** Environment of an evaluation. Everything that touches the system is injectable so tests stay
|
||||
* deterministic and never execute commands or resolve names. */
|
||||
struct SSHConfigContext {
|
||||
struct EE_VFS_API SSHConfigContext {
|
||||
/** Host as given by the user (URI host). */
|
||||
std::string originalHost;
|
||||
/** Command-line style overrides (from the URI): they win over every config file. */
|
||||
@@ -211,12 +211,16 @@ struct SSHConfigContext {
|
||||
* cycle/depth protection, hostname canonicalization and the canonical/final re-parse pass, and
|
||||
* context-aware token and environment expansion.
|
||||
*/
|
||||
class SSHConfigEvaluator {
|
||||
class EE_VFS_API SSHConfigEvaluator {
|
||||
public:
|
||||
explicit SSHConfigEvaluator( SSHConfigContext context );
|
||||
|
||||
~SSHConfigEvaluator();
|
||||
|
||||
SSHConfigEvaluator( const SSHConfigEvaluator& ) = delete;
|
||||
|
||||
SSHConfigEvaluator& operator=( const SSHConfigEvaluator& ) = delete;
|
||||
|
||||
/** @return The resolved configuration, or an error message for fatal problems (for example
|
||||
* canonicalization failure with CanonicalizeFallbackLocal no). */
|
||||
std::variant<SSHResolvedConfig, std::string> evaluate();
|
||||
|
||||
@@ -4,13 +4,14 @@
|
||||
#include <eepp/core/cancellationtoken.hpp>
|
||||
#include <eepp/core/executor.hpp>
|
||||
#include <eepp/core/small_unique_function.hpp>
|
||||
#include <eepp/vfs/config.hpp>
|
||||
#include <eepp/vfs/error.hpp>
|
||||
#include <memory>
|
||||
#include <optional>
|
||||
|
||||
namespace EE { namespace VFS {
|
||||
|
||||
struct TransferProgress {
|
||||
struct EE_VFS_API TransferProgress {
|
||||
URI currentSource;
|
||||
URI currentDestination;
|
||||
Uint64 bytesDone{ 0 };
|
||||
@@ -19,7 +20,7 @@ struct TransferProgress {
|
||||
std::optional<Uint64> filesTotal;
|
||||
};
|
||||
|
||||
struct TransferResult {
|
||||
struct EE_VFS_API TransferResult {
|
||||
Uint64 bytesTransferred{ 0 };
|
||||
Uint64 filesTransferred{ 0 };
|
||||
Uint64 filesSkipped{ 0 };
|
||||
@@ -37,7 +38,7 @@ class TransferState;
|
||||
}
|
||||
|
||||
/** @brief RAII handle of a progress subscription. */
|
||||
class TransferProgressSubscription {
|
||||
class EE_VFS_API TransferProgressSubscription {
|
||||
public:
|
||||
TransferProgressSubscription() = default;
|
||||
|
||||
@@ -69,7 +70,7 @@ class TransferProgressSubscription {
|
||||
* coalesced: a slow subscriber receives the latest snapshot rather than every intermediate one.
|
||||
* Transfers keep running when the Transfer object is destroyed; call cancel() to stop them.
|
||||
*/
|
||||
class Transfer {
|
||||
class EE_VFS_API Transfer {
|
||||
public:
|
||||
Transfer() = default;
|
||||
|
||||
|
||||
@@ -6,6 +6,7 @@
|
||||
#include <eepp/system/regex.hpp>
|
||||
#include <eepp/ui/models/model.hpp>
|
||||
#include <eepp/ui/models/variant.hpp>
|
||||
#include <eepp/vfs/config.hpp>
|
||||
#include <eepp/vfs/virtualfilesystem.hpp>
|
||||
#include <functional>
|
||||
#include <memory>
|
||||
@@ -39,7 +40,8 @@ using UI::Models::Variant;
|
||||
* All public methods must be called on the UI thread, which must be the thread the executor runs
|
||||
* functions on.
|
||||
*/
|
||||
class VFSDirectoryModel : public Model, public std::enable_shared_from_this<VFSDirectoryModel> {
|
||||
class EE_VFS_API VFSDirectoryModel : public Model,
|
||||
public std::enable_shared_from_this<VFSDirectoryModel> {
|
||||
public:
|
||||
enum Column : Uint8 { Name, Size, Modified, Type, Permissions, Owner, Group, ColumnCount };
|
||||
|
||||
@@ -83,6 +85,10 @@ class VFSDirectoryModel : public Model, public std::enable_shared_from_this<VFSD
|
||||
|
||||
~VFSDirectoryModel() override;
|
||||
|
||||
VFSDirectoryModel( const VFSDirectoryModel& ) = delete;
|
||||
|
||||
VFSDirectoryModel& operator=( const VFSDirectoryModel& ) = delete;
|
||||
|
||||
/** Navigates to a directory. */
|
||||
void setURI( URI uri );
|
||||
|
||||
|
||||
@@ -2,6 +2,7 @@
|
||||
#define EE_VFS_VIRTUALFILESYSTEM_HPP
|
||||
|
||||
#include <eepp/system/threadpool.hpp>
|
||||
#include <eepp/vfs/config.hpp>
|
||||
#include <eepp/vfs/provider.hpp>
|
||||
#include <eepp/vfs/providers/localioscheduler.hpp>
|
||||
#include <eepp/vfs/transfer.hpp>
|
||||
@@ -31,7 +32,7 @@ using EE::System::ThreadPool;
|
||||
* Provider work never runs on that pool, so waits there cannot starve provider I/O. Copies and
|
||||
* moves run on a pool of their own: long transfers never delay the other operations.
|
||||
*/
|
||||
class VirtualFileSystem {
|
||||
class EE_VFS_API VirtualFileSystem {
|
||||
public:
|
||||
struct Config {
|
||||
/** Worker threads for blocking local I/O (LocalProvider and similar). */
|
||||
@@ -45,7 +46,7 @@ class VirtualFileSystem {
|
||||
* counts as not responding: the call fails with TimedOut, and so do further calls until
|
||||
* the server answers again (see LocalIOScheduler). */
|
||||
System::Time networkMountTimeout{ System::Seconds( 15 ) };
|
||||
/** Register file:// and res:// (and sftp:// when built with SFTP support). */
|
||||
/** Register file://, zip:// and res:// (and sftp:// when built with SFTP support). */
|
||||
bool registerDefaultProviders{ true };
|
||||
};
|
||||
|
||||
|
||||
@@ -5,6 +5,7 @@
|
||||
#include <eepp/core/executor.hpp>
|
||||
#include <eepp/core/small_unique_function.hpp>
|
||||
#include <eepp/system/time.hpp>
|
||||
#include <eepp/vfs/config.hpp>
|
||||
#include <eepp/vfs/error.hpp>
|
||||
#include <memory>
|
||||
#include <mutex>
|
||||
@@ -29,11 +30,11 @@ enum class WatchEventType : Uint8 {
|
||||
RescanRequired,
|
||||
};
|
||||
|
||||
const char* toString( WatchEventType type );
|
||||
EE_VFS_API const char* toString( WatchEventType type );
|
||||
|
||||
/** One change, relative to the watched root ("dir/file.txt", '/' separated, never leading '/').
|
||||
*/
|
||||
struct WatchEvent {
|
||||
struct EE_VFS_API WatchEvent {
|
||||
std::string relativePath;
|
||||
std::string oldRelativePath;
|
||||
WatchEventType type{ WatchEventType::Modified };
|
||||
@@ -52,16 +53,16 @@ enum class WatchStatus : Uint8 {
|
||||
Failed,
|
||||
};
|
||||
|
||||
const char* toString( WatchStatus status );
|
||||
EE_VFS_API const char* toString( WatchStatus status );
|
||||
|
||||
/** A coalesced group of events for one watch. Events are in arrival order. */
|
||||
struct WatchEventBatch {
|
||||
struct EE_VFS_API WatchEventBatch {
|
||||
URI root;
|
||||
std::vector<WatchEvent> events;
|
||||
WatchStatus status{ WatchStatus::Active };
|
||||
};
|
||||
|
||||
struct WatchOptions {
|
||||
struct EE_VFS_API WatchOptions {
|
||||
bool recursive{ false };
|
||||
bool followSymlinks{ false };
|
||||
bool allowOutOfScopeLinks{ false };
|
||||
@@ -84,7 +85,7 @@ class WatchSubscriptionState;
|
||||
* Thread-safe. Events are accumulated and delivered as one batch per executor dispatch, which
|
||||
* coalesces bursts without adding a latency timer.
|
||||
*/
|
||||
class WatchSink {
|
||||
class EE_VFS_API WatchSink {
|
||||
public:
|
||||
virtual ~WatchSink() = default;
|
||||
|
||||
@@ -101,14 +102,14 @@ class WatchSink {
|
||||
};
|
||||
|
||||
/** Backend-side registration of one logical watch. Destroying it stops native watching. */
|
||||
class WatchRegistration {
|
||||
class EE_VFS_API WatchRegistration {
|
||||
public:
|
||||
virtual ~WatchRegistration() = default;
|
||||
};
|
||||
|
||||
/** @brief Notification engine for a set of URIs. Backends adapt efsw (local), a remote
|
||||
* efsw-agent or polling into URI space; they never mutate UI or model state. */
|
||||
class WatchBackend {
|
||||
class EE_VFS_API WatchBackend {
|
||||
public:
|
||||
virtual ~WatchBackend() = default;
|
||||
|
||||
@@ -125,7 +126,7 @@ class WatchBackend {
|
||||
* that the callback is not running and will not run again (a callback may destroy its own
|
||||
* handle).
|
||||
*/
|
||||
class WatchHandle {
|
||||
class EE_VFS_API WatchHandle {
|
||||
public:
|
||||
WatchHandle() = default;
|
||||
|
||||
@@ -167,7 +168,7 @@ class WatchHandle {
|
||||
* Backends are registered per scheme. A scheme without a dedicated backend uses the fallback
|
||||
* backend (normally polling through the VFS) when WatchOptions::allowPolling is set.
|
||||
*/
|
||||
class WatchManager {
|
||||
class EE_VFS_API WatchManager {
|
||||
public:
|
||||
WatchManager();
|
||||
|
||||
|
||||
@@ -0,0 +1,614 @@
|
||||
#include <eepp/vfs/providers/zipprovider.hpp>
|
||||
|
||||
#include <algorithm>
|
||||
#include <array>
|
||||
#include <eepp/core/containers.hpp>
|
||||
#include <eepp/system/zip.hpp>
|
||||
#include <eepp/vfs/detail/memoryhandles.hpp>
|
||||
#include <eepp/vfs/providers/localprovider.hpp>
|
||||
#include <limits>
|
||||
#include <mutex>
|
||||
|
||||
using namespace EE::System;
|
||||
|
||||
namespace EE { namespace VFS {
|
||||
|
||||
namespace {
|
||||
|
||||
constexpr ProviderCapabilities
|
||||
ZipCapabilities( static_cast<Uint32>( VFSCapability::Read ) |
|
||||
static_cast<Uint32>( VFSCapability::List ) |
|
||||
static_cast<Uint32>( VFSCapability::Seek ) |
|
||||
static_cast<Uint32>( VFSCapability::CaseSensitive ) );
|
||||
constexpr Uint64 NoEntry = std::numeric_limits<Uint64>::max();
|
||||
constexpr size_t MaxEntries = 262144;
|
||||
|
||||
VFSError readOnly( const URI& uri ) {
|
||||
return VFSError( VFSErrorCode::ReadOnly, uri, "ZIP archives are read-only" );
|
||||
}
|
||||
|
||||
VFSError readerError( const Zip& reader, const URI& uri ) {
|
||||
VFSErrorCode code = VFSErrorCode::IO;
|
||||
switch ( reader.error() ) {
|
||||
case Zip::Error::NotFound:
|
||||
code = VFSErrorCode::NotFound;
|
||||
break;
|
||||
case Zip::Error::PermissionDenied:
|
||||
code = VFSErrorCode::PermissionDenied;
|
||||
break;
|
||||
case Zip::Error::InvalidArchive:
|
||||
code = VFSErrorCode::InvalidOperation;
|
||||
break;
|
||||
case Zip::Error::Unsupported:
|
||||
code = VFSErrorCode::Unsupported;
|
||||
break;
|
||||
default:
|
||||
break;
|
||||
}
|
||||
return VFSError( code, uri, reader.errorMessage() );
|
||||
}
|
||||
|
||||
/** Archive paths are untrusted. Reject names that could leave a copy destination or turn into
|
||||
* native Windows paths/streams; do not resolve these names into aliases of safe entries. */
|
||||
bool safeEntry( std::string_view name ) {
|
||||
if ( name.empty() || name.size() > 4096 || name.front() == '/' ||
|
||||
name.find_first_of( std::string_view( "\\:\0", 3 ) ) != std::string_view::npos )
|
||||
return false;
|
||||
size_t start = 0;
|
||||
while ( start < name.size() ) {
|
||||
size_t end = name.find( '/', start );
|
||||
if ( end == std::string_view::npos )
|
||||
end = name.size();
|
||||
auto part = name.substr( start, end - start );
|
||||
// Windows trims trailing dots/spaces; an all-dot/space segment can become a parent.
|
||||
if ( part.empty() || part.find_first_not_of( ". " ) == std::string_view::npos )
|
||||
return false;
|
||||
start = end + 1;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
/** Parsed archive identity and normalized path within it. */
|
||||
struct ZipAddress {
|
||||
URI archive;
|
||||
std::string member;
|
||||
|
||||
static VFSResult<ZipAddress> parse( const URI& uri ) {
|
||||
if ( uri.getScheme() != "zip" || uri.getAuthority() != uri.getHost() ||
|
||||
!uri.getFragment().empty() || uri.getPath().find( '\0' ) != std::string::npos )
|
||||
return Err( VFSError( VFSErrorCode::InvalidURI, uri, "Invalid ZIP location" ) );
|
||||
if ( !uri.getRawQuery().empty() )
|
||||
return Err( VFSError( VFSErrorCode::Unsupported, uri, "Unsupported ZIP backing URI" ) );
|
||||
const auto& path = uri.getPath();
|
||||
if ( path.empty() || path.front() != '/' )
|
||||
return Err( VFSError( VFSErrorCode::InvalidURI, uri, "ZIP path must be absolute" ) );
|
||||
std::string backing;
|
||||
backing.reserve( path.size() );
|
||||
size_t boundary = std::string::npos;
|
||||
for ( size_t i = 0; i < path.size(); ++i ) {
|
||||
if ( path[i] == '!' ) {
|
||||
if ( i + 1 < path.size() && path[i + 1] == '!' ) {
|
||||
backing += '!';
|
||||
++i;
|
||||
continue;
|
||||
}
|
||||
if ( i + 1 == path.size() || path[i + 1] == '/' ) {
|
||||
boundary = i + 1;
|
||||
break;
|
||||
}
|
||||
}
|
||||
backing += path[i];
|
||||
}
|
||||
if ( boundary == std::string::npos || backing.empty() || backing.back() == '/' )
|
||||
return Err(
|
||||
VFSError( VFSErrorCode::InvalidURI, uri, "ZIP path needs archive.zip!/member" ) );
|
||||
ZipAddress result;
|
||||
result.archive.setScheme( "file" );
|
||||
result.archive.setHost( uri.getHost() );
|
||||
result.archive.setRawPath( std::move( backing ) );
|
||||
URI member;
|
||||
member.setScheme( "zip" );
|
||||
member.setRawPath( boundary == path.size() ? "/" : path.substr( boundary ) );
|
||||
member.normalize();
|
||||
result.member = member.getPath();
|
||||
return Ok( std::move( result ) );
|
||||
}
|
||||
};
|
||||
|
||||
struct ZipNode {
|
||||
VFSFileInfo info;
|
||||
std::vector<size_t> children;
|
||||
Uint64 entry{ NoEntry };
|
||||
};
|
||||
|
||||
/** Immutable directory index. Entry streams open independent readers, so their I/O and teardown
|
||||
* never wait for another handle on the same archive. */
|
||||
struct ZipArchive {
|
||||
std::string path;
|
||||
std::vector<ZipNode> nodes;
|
||||
UnorderedMap<std::string, size_t> paths;
|
||||
FileRevision revision;
|
||||
LocalIOScheduler::Route route;
|
||||
Uint64 stamp{ 0 };
|
||||
|
||||
VFSFileInfo info( std::string name, bool directory, Uint64 entry = NoEntry,
|
||||
const Zip::Entry* metadata = nullptr ) const {
|
||||
VFSFileInfo value;
|
||||
value.name = std::move( name );
|
||||
value.type = directory ? FileType::Directory : FileType::Regular;
|
||||
value.hidden = !value.name.empty() && value.name[0] == '.';
|
||||
// Read-only is a provider capability, not the stored entry's mode. The archive API does
|
||||
// not expose Unix permissions: leave them unknown so copies use writable native defaults.
|
||||
if ( metadata ) {
|
||||
value.setModifiedTime( metadata->modified );
|
||||
if ( !directory )
|
||||
value.setSize( metadata->size );
|
||||
}
|
||||
value.revision = FileRevision::fromValues(
|
||||
{ stamp, entry, metadata ? metadata->crc : 0, metadata ? metadata->size : 0 } );
|
||||
return value;
|
||||
}
|
||||
|
||||
VFSResult<void> index( Zip& reader, const URI& uri, const CancellationToken& token ) {
|
||||
const auto count = reader.entryCount();
|
||||
if ( count > MaxEntries )
|
||||
return Err(
|
||||
VFSError( VFSErrorCode::InvalidOperation, uri, "ZIP has too many entries" ) );
|
||||
nodes.push_back( { info( "/", true ), {}, NoEntry } );
|
||||
paths.emplace( "", 0 );
|
||||
Zip::Entry metadata;
|
||||
for ( Uint64 entry = 0; entry < count; ++entry ) {
|
||||
if ( token.isCancellationRequested() )
|
||||
return Err( cancelledError( uri ) );
|
||||
if ( !reader.entry( entry, metadata ) )
|
||||
return Err( readerError( reader, uri ) );
|
||||
std::string_view name( metadata.name );
|
||||
if ( !safeEntry( name ) )
|
||||
continue;
|
||||
const bool directory = name.back() == '/';
|
||||
if ( directory )
|
||||
name.remove_suffix( 1 );
|
||||
size_t start = 0;
|
||||
size_t parent = 0;
|
||||
while ( start < name.size() ) {
|
||||
size_t end = name.find( '/', start );
|
||||
if ( end == std::string_view::npos )
|
||||
end = name.size();
|
||||
const bool last = end == name.size();
|
||||
const bool folder = !last || directory;
|
||||
std::string path( name.substr( 0, end ) );
|
||||
auto found = paths.find( path );
|
||||
size_t node;
|
||||
if ( found == paths.end() ) {
|
||||
if ( nodes.size() >= MaxEntries ) {
|
||||
return Err( VFSError( VFSErrorCode::InvalidOperation, uri,
|
||||
"ZIP has too many directories and entries" ) );
|
||||
}
|
||||
node = nodes.size();
|
||||
paths.emplace( std::move( path ), node );
|
||||
nodes[parent].children.push_back( node );
|
||||
nodes.push_back(
|
||||
{ info( std::string( name.substr( start, end - start ) ), folder ),
|
||||
{},
|
||||
NoEntry } );
|
||||
} else {
|
||||
node = found->second;
|
||||
}
|
||||
ZipNode& value = nodes[node];
|
||||
// A directory wins a file/directory collision so that every listed child remains
|
||||
// reachable. Duplicate file names resolve to their first central-directory entry.
|
||||
if ( folder && value.info.type != FileType::Directory ) {
|
||||
value.info = info( std::move( value.info.name ), true );
|
||||
value.entry = NoEntry;
|
||||
}
|
||||
if ( last && ( folder || value.info.type != FileType::Directory ) &&
|
||||
value.entry == NoEntry ) {
|
||||
value.info = info( std::move( value.info.name ), folder, entry, &metadata );
|
||||
value.entry = entry;
|
||||
}
|
||||
parent = node;
|
||||
start = end + 1;
|
||||
}
|
||||
}
|
||||
return Ok();
|
||||
}
|
||||
|
||||
VFSResult<size_t> find( const std::string& member, const URI& uri ) const {
|
||||
auto found = paths.find( member );
|
||||
if ( found != paths.end() ) {
|
||||
if ( uri.getPath().ends_with( '/' ) &&
|
||||
nodes[found->second].info.type != FileType::Directory )
|
||||
return Err( VFSError( VFSErrorCode::NotDirectory, uri ) );
|
||||
return Ok( found->second );
|
||||
}
|
||||
// Distinguish a missing member from a path through a regular file.
|
||||
size_t slash = member.find( '/' );
|
||||
while ( slash != std::string::npos ) {
|
||||
auto parent = paths.find( member.substr( 0, slash ) );
|
||||
if ( parent != paths.end() && nodes[parent->second].info.type != FileType::Directory )
|
||||
return Err( VFSError( VFSErrorCode::NotDirectory, uri ) );
|
||||
slash = member.find( '/', slash + 1 );
|
||||
}
|
||||
return Err( VFSError( VFSErrorCode::NotFound, uri ) );
|
||||
}
|
||||
};
|
||||
|
||||
class ZipFileImpl : public FileHandleImpl, public std::enable_shared_from_this<ZipFileImpl> {
|
||||
template <typename T, typename F> auto guarded( CancellationToken token, F&& fn ) {
|
||||
return [self = shared_from_this(), token = std::move( token ),
|
||||
fn = std::forward<F>( fn )]() mutable -> VFSResult<T> {
|
||||
if ( token.isCancellationRequested() )
|
||||
return Err( cancelledError( self->mUri ) );
|
||||
std::lock_guard<std::mutex> lock( self->mMutex );
|
||||
if ( !self->mStream )
|
||||
return Err( VFSError( VFSErrorCode::InvalidOperation, self->mUri, "File closed" ) );
|
||||
return fn( *self, token );
|
||||
};
|
||||
}
|
||||
|
||||
public:
|
||||
ZipFileImpl( std::shared_ptr<LocalIOScheduler> scheduler, std::shared_ptr<ZipArchive> archive,
|
||||
std::unique_ptr<Zip> reader, std::unique_ptr<IOStream> stream, URI uri,
|
||||
size_t node ) :
|
||||
mScheduler( std::move( scheduler ) ),
|
||||
mArchive( std::move( archive ) ),
|
||||
mReader( std::move( reader ) ),
|
||||
mStream( std::move( stream ) ),
|
||||
mUri( std::move( uri ) ),
|
||||
mNode( node ) {}
|
||||
|
||||
VFSTask<size_t> read( Uint64 offset, std::span<Uint8> data, CancellationToken token ) override {
|
||||
if ( !mArchive->route.isNetworkMount() ) {
|
||||
return mScheduler->submit<size_t>(
|
||||
mArchive->route, guarded<size_t>( std::move( token ),
|
||||
[offset, data]( ZipFileImpl& self,
|
||||
const CancellationToken& token ) {
|
||||
return self.readInto( offset, data, token );
|
||||
} ) );
|
||||
}
|
||||
// A mount's read can outlive its timeout. Only the winning completion copies into data.
|
||||
return mScheduler->submitCommitted<size_t>(
|
||||
mArchive->route,
|
||||
guarded<std::vector<Uint8>>(
|
||||
std::move( token ),
|
||||
[offset, size = data.size()]( ZipFileImpl& self, const CancellationToken& token )
|
||||
-> VFSResult<std::vector<Uint8>> {
|
||||
std::vector<Uint8> buffer( size );
|
||||
auto result = self.readInto( offset, buffer, token );
|
||||
if ( !result )
|
||||
return Err( std::move( result ).error() );
|
||||
buffer.resize( result.value() );
|
||||
return Ok( std::move( buffer ) );
|
||||
} ),
|
||||
[data]( VFSResult<std::vector<Uint8>>&& result ) -> VFSResult<size_t> {
|
||||
if ( !result )
|
||||
return Err( std::move( result ).error() );
|
||||
std::copy( result.value().begin(), result.value().end(), data.begin() );
|
||||
return Ok( result.value().size() );
|
||||
} );
|
||||
}
|
||||
|
||||
VFSTask<size_t> write( Uint64, std::span<const Uint8>, CancellationToken ) override {
|
||||
return readyError<size_t>( readOnly( mUri ) );
|
||||
}
|
||||
|
||||
VFSTask<void> truncate( Uint64 ) override { return readyError<void>( readOnly( mUri ) ); }
|
||||
|
||||
VFSTask<void> flush() override { return readyValue<void>(); }
|
||||
|
||||
VFSTask<VFSFileInfo> stat() override {
|
||||
return readyValue<VFSFileInfo>( mArchive->nodes[mNode].info );
|
||||
}
|
||||
|
||||
VFSTask<void> close() override {
|
||||
return mScheduler->submit<void>( mArchive->route,
|
||||
[self = shared_from_this()]() -> VFSResult<void> {
|
||||
std::lock_guard<std::mutex> lock( self->mMutex );
|
||||
self->mStream.reset();
|
||||
self->mReader.reset();
|
||||
return Ok();
|
||||
} );
|
||||
}
|
||||
|
||||
ProviderCapabilities capabilities() const override { return ZipCapabilities; }
|
||||
|
||||
private:
|
||||
VFSResult<size_t> readInto( Uint64 offset, std::span<Uint8> data,
|
||||
const CancellationToken& token ) {
|
||||
const auto& node = mArchive->nodes[mNode];
|
||||
if ( offset >= node.info.size().value_or( 0 ) )
|
||||
return Ok( size_t( 0 ) );
|
||||
if ( offset < static_cast<Uint64>( mStream->tell() ) ) {
|
||||
auto stream = mReader->openStream( node.entry );
|
||||
if ( !stream )
|
||||
return Err( readerError( *mReader, mUri ) );
|
||||
mStream = std::move( stream );
|
||||
}
|
||||
std::array<char, 16384> scratch;
|
||||
while ( static_cast<Uint64>( mStream->tell() ) < offset ) {
|
||||
if ( token.isCancellationRequested() )
|
||||
return Err( cancelledError( mUri ) );
|
||||
const auto count = std::min<Uint64>( offset - mStream->tell(), scratch.size() );
|
||||
if ( mStream->read( scratch.data(), count ) <= 0 )
|
||||
return Err( VFSError( VFSErrorCode::IO, mUri, "Could not seek ZIP entry" ) );
|
||||
}
|
||||
size_t done = 0;
|
||||
while ( done < data.size() ) {
|
||||
if ( token.isCancellationRequested() )
|
||||
return Err( cancelledError( mUri ) );
|
||||
const auto count = std::min<size_t>( data.size() - done, 65536 );
|
||||
const auto got = mStream->read( reinterpret_cast<char*>( data.data() + done ), count );
|
||||
if ( got < 0 )
|
||||
return Err( VFSError( VFSErrorCode::IO, mUri,
|
||||
"Could not read ZIP entry (corrupt data or CRC)" ) );
|
||||
if ( got == 0 )
|
||||
break;
|
||||
done += static_cast<size_t>( got );
|
||||
}
|
||||
return Ok( done );
|
||||
}
|
||||
|
||||
std::shared_ptr<LocalIOScheduler> mScheduler;
|
||||
// Streams borrow the reader and are destroyed before it.
|
||||
std::shared_ptr<ZipArchive> mArchive;
|
||||
std::unique_ptr<Zip> mReader;
|
||||
std::unique_ptr<IOStream> mStream;
|
||||
std::mutex mMutex;
|
||||
URI mUri;
|
||||
size_t mNode;
|
||||
};
|
||||
|
||||
} // namespace
|
||||
|
||||
class ZipProviderImpl : public std::enable_shared_from_this<ZipProviderImpl> {
|
||||
public:
|
||||
explicit ZipProviderImpl( std::shared_ptr<LocalIOScheduler> scheduler ) :
|
||||
scheduler( std::move( scheduler ) ) {}
|
||||
|
||||
template <typename T, typename F>
|
||||
VFSTask<T> run( const URI& uri, CancellationToken token, F&& fn ) {
|
||||
if ( token.isCancellationRequested() )
|
||||
return readyError<T>( cancelledError( uri ) );
|
||||
auto address = ZipAddress::parse( uri );
|
||||
if ( !address )
|
||||
return readyError<T>( std::move( address ).error() );
|
||||
auto native = LocalProvider::toNativePath( address.value().archive );
|
||||
if ( !native )
|
||||
return readyError<T>( std::move( native ).error() );
|
||||
std::string member = std::move( address ).value().member;
|
||||
while ( !member.empty() && member.front() == '/' )
|
||||
member.erase( member.begin() );
|
||||
while ( !member.empty() && member.back() == '/' )
|
||||
member.pop_back();
|
||||
std::string path = std::move( native ).value();
|
||||
return scheduler->submitResolved<T>(
|
||||
path, true,
|
||||
[self = shared_from_this(), uri, path, member = std::move( member ),
|
||||
token = std::move( token ), fn = std::forward<F>( fn )]() mutable -> VFSResult<T> {
|
||||
if ( token.isCancellationRequested() )
|
||||
return Err( cancelledError( uri ) );
|
||||
auto archive = self->load( path, uri, token );
|
||||
if ( !archive )
|
||||
return Err( std::move( archive ).error() );
|
||||
return fn( std::move( archive ).value(), member, uri );
|
||||
} );
|
||||
}
|
||||
|
||||
std::shared_ptr<LocalIOScheduler> scheduler;
|
||||
|
||||
private:
|
||||
struct Cached {
|
||||
std::string path;
|
||||
std::shared_ptr<ZipArchive> archive;
|
||||
Uint64 used{ 0 };
|
||||
};
|
||||
|
||||
VFSResult<std::shared_ptr<ZipArchive>> load( const std::string& path, const URI& uri,
|
||||
const CancellationToken& token ) {
|
||||
const URI backing = URI::fromFSPath( path );
|
||||
auto metadata = LocalProvider::statSync( backing, path, true );
|
||||
if ( !metadata )
|
||||
return Err( std::move( metadata ).error() );
|
||||
if ( !metadata.value().isRegularFile() )
|
||||
return Err( VFSError( VFSErrorCode::InvalidOperation, uri,
|
||||
"ZIP backing file is not a regular file" ) );
|
||||
const auto revision = metadata.value().revision;
|
||||
{
|
||||
std::lock_guard<std::mutex> lock( mMutex );
|
||||
for ( auto& cached : mCache ) {
|
||||
if ( cached.path == path && cached.archive &&
|
||||
cached.archive->revision == revision ) {
|
||||
cached.used = ++mUsed;
|
||||
return Ok( cached.archive );
|
||||
}
|
||||
}
|
||||
}
|
||||
auto archive = std::make_shared<ZipArchive>();
|
||||
archive->revision = revision;
|
||||
archive->path = path;
|
||||
archive->stamp = std::hash<std::string_view>{}(
|
||||
std::string_view( reinterpret_cast<const char*>( revision.data() ), revision.size() ) );
|
||||
archive->route = scheduler->resolvedRouteFor( path, true );
|
||||
Zip reader( false );
|
||||
if ( !reader.open( path, Zip::OpenMode::ReadOnly ) )
|
||||
return Err( readerError( reader, uri ) );
|
||||
auto indexed = archive->index( reader, uri, token );
|
||||
if ( !indexed )
|
||||
return Err( std::move( indexed ).error() );
|
||||
auto after = LocalProvider::statSync( backing, path, true );
|
||||
if ( !after || after.value().revision != revision )
|
||||
return Err( VFSError( VFSErrorCode::ChangedExternally, uri,
|
||||
"Archive changed while it was opened" ) );
|
||||
std::shared_ptr<ZipArchive> retired;
|
||||
{
|
||||
std::lock_guard<std::mutex> lock( mMutex );
|
||||
auto slot = mCache.begin();
|
||||
for ( auto it = mCache.begin(); it != mCache.end(); ++it ) {
|
||||
if ( it->path == path ) {
|
||||
slot = it;
|
||||
break;
|
||||
}
|
||||
if ( it->used < slot->used )
|
||||
slot = it;
|
||||
}
|
||||
retired = std::move( slot->archive );
|
||||
slot->path = path;
|
||||
slot->archive = archive;
|
||||
slot->used = ++mUsed;
|
||||
}
|
||||
// Free a retired directory index outside the cache lock.
|
||||
return Ok( std::move( archive ) );
|
||||
}
|
||||
|
||||
std::mutex mMutex;
|
||||
std::array<Cached, 8> mCache;
|
||||
Uint64 mUsed{ 0 };
|
||||
};
|
||||
|
||||
ZipProvider::ZipProvider( std::shared_ptr<LocalIOScheduler> scheduler ) :
|
||||
mImpl( std::make_shared<ZipProviderImpl>( std::move( scheduler ) ) ) {}
|
||||
|
||||
ZipProvider::~ZipProvider() = default;
|
||||
|
||||
URI ZipProvider::makeURI( const URI& archive, std::string_view member ) {
|
||||
if ( archive.getScheme() != "file" || !archive.getRawQuery().empty() ||
|
||||
!archive.getFragment().empty() || archive.getAuthority() != archive.getHost() ||
|
||||
archive.getPath().empty() || archive.getPath().front() != '/' ||
|
||||
archive.getPath().find( '\0' ) != std::string::npos )
|
||||
return {};
|
||||
URI uri;
|
||||
uri.setScheme( "zip" );
|
||||
uri.setHost( archive.getHost() );
|
||||
std::string path;
|
||||
path.reserve( archive.getPath().size() + member.size() + 2 );
|
||||
// Escape literal bangs in the backing path, so even a directory named "packages!" is
|
||||
// unambiguous. Only the first unescaped "!/" separates the archive from its member.
|
||||
for ( char c : archive.getPath() ) {
|
||||
path += c;
|
||||
if ( c == '!' )
|
||||
path += '!';
|
||||
}
|
||||
URI entry;
|
||||
entry.setScheme( "zip" );
|
||||
entry.setRawPath( member.empty() || member.front() != '/' ? "/" + std::string( member )
|
||||
: std::string( member ) );
|
||||
entry.normalize();
|
||||
path += '!';
|
||||
path += entry.getPath();
|
||||
uri.setRawPath( std::move( path ) );
|
||||
return uri;
|
||||
}
|
||||
|
||||
VFSResult<URI> ZipProvider::archiveURI( const URI& uri ) {
|
||||
auto address = ZipAddress::parse( uri );
|
||||
if ( !address )
|
||||
return Err( std::move( address ).error() );
|
||||
return Ok( std::move( address ).value().archive );
|
||||
}
|
||||
|
||||
VFSResult<std::string> ZipProvider::memberPath( const URI& uri ) {
|
||||
auto address = ZipAddress::parse( uri );
|
||||
if ( !address )
|
||||
return Err( std::move( address ).error() );
|
||||
return Ok( std::move( address ).value().member );
|
||||
}
|
||||
|
||||
ProviderCapabilities ZipProvider::capabilities( const URI& ) const {
|
||||
return ZipCapabilities;
|
||||
}
|
||||
|
||||
VFSTask<VFSFileInfo> ZipProvider::stat( const URI& uri, const StatOptions&,
|
||||
CancellationToken token ) {
|
||||
return mImpl->run<VFSFileInfo>( uri, std::move( token ),
|
||||
[]( const std::shared_ptr<ZipArchive>& archive,
|
||||
const std::string& member,
|
||||
const URI& uri ) -> VFSResult<VFSFileInfo> {
|
||||
auto node = archive->find( member, uri );
|
||||
if ( !node )
|
||||
return Err( std::move( node ).error() );
|
||||
return Ok( archive->nodes[node.value()].info );
|
||||
} );
|
||||
}
|
||||
|
||||
VFSTask<DirectoryHandle> ZipProvider::openDirectory( const URI& uri, const DirectoryOpenOptions&,
|
||||
CancellationToken token ) {
|
||||
return mImpl->run<DirectoryHandle>(
|
||||
uri, std::move( token ),
|
||||
[]( const std::shared_ptr<ZipArchive>& archive, const std::string& member,
|
||||
const URI& uri ) -> VFSResult<DirectoryHandle> {
|
||||
auto node = archive->find( member, uri );
|
||||
if ( !node )
|
||||
return Err( std::move( node ).error() );
|
||||
const auto& folder = archive->nodes[node.value()];
|
||||
if ( folder.info.type != FileType::Directory )
|
||||
return Err( VFSError( VFSErrorCode::NotDirectory, uri ) );
|
||||
std::vector<VFSFileInfo> entries;
|
||||
entries.reserve( folder.children.size() );
|
||||
for ( auto child : folder.children )
|
||||
entries.push_back( archive->nodes[child].info );
|
||||
return Ok( DirectoryHandle(
|
||||
uri, std::make_shared<detail::VectorDirectoryImpl>( std::move( entries ) ) ) );
|
||||
} );
|
||||
}
|
||||
|
||||
VFSTask<FileHandle> ZipProvider::open( const URI& uri, const OpenOptions& options,
|
||||
CancellationToken token ) {
|
||||
if ( hasFlag( options.flags, OpenFlags::Write ) ||
|
||||
hasFlag( options.flags, OpenFlags::Create ) ||
|
||||
hasFlag( options.flags, OpenFlags::Truncate ) )
|
||||
return readyError<FileHandle>( readOnly( uri ) );
|
||||
return mImpl->run<FileHandle>(
|
||||
uri, std::move( token ),
|
||||
[scheduler = mImpl->scheduler]( std::shared_ptr<ZipArchive> archive,
|
||||
const std::string& member,
|
||||
const URI& uri ) -> VFSResult<FileHandle> {
|
||||
auto node = archive->find( member, uri );
|
||||
if ( !node )
|
||||
return Err( std::move( node ).error() );
|
||||
const auto& info = archive->nodes[node.value()];
|
||||
if ( info.info.type == FileType::Directory )
|
||||
return Err( VFSError( VFSErrorCode::IsDirectory, uri ) );
|
||||
auto reader = std::make_unique<Zip>( false );
|
||||
if ( !reader->open( archive->path, Zip::OpenMode::ReadOnly ) )
|
||||
return Err( readerError( *reader, uri ) );
|
||||
auto current =
|
||||
LocalProvider::statSync( URI::fromFSPath( archive->path ), archive->path, true );
|
||||
if ( !current || current.value().revision != archive->revision )
|
||||
return Err( VFSError( VFSErrorCode::ChangedExternally, uri,
|
||||
"Archive changed while opening the entry" ) );
|
||||
auto stream = reader->openStream( info.entry );
|
||||
if ( !stream )
|
||||
return Err( readerError( *reader, uri ) );
|
||||
return Ok( FileHandle( uri, std::make_shared<ZipFileImpl>(
|
||||
scheduler, std::move( archive ), std::move( reader ),
|
||||
std::move( stream ), uri, node.value() ) ) );
|
||||
} );
|
||||
}
|
||||
|
||||
VFSTask<void> ZipProvider::mkdir( const URI& uri, const MkDirOptions&, CancellationToken ) {
|
||||
return readyError<void>( readOnly( uri ) );
|
||||
}
|
||||
|
||||
VFSTask<void> ZipProvider::remove( const URI& uri, const RemoveOptions&, CancellationToken ) {
|
||||
return readyError<void>( readOnly( uri ) );
|
||||
}
|
||||
|
||||
VFSTask<void> ZipProvider::rename( const URI& from, const URI&, const RenameOptions&,
|
||||
CancellationToken ) {
|
||||
return readyError<void>( readOnly( from ) );
|
||||
}
|
||||
|
||||
VFSTask<URI> ZipProvider::canonicalize( const URI& uri, CancellationToken token ) {
|
||||
return mImpl->run<URI>( uri, std::move( token ),
|
||||
[]( const std::shared_ptr<ZipArchive>& archive,
|
||||
const std::string& member, const URI& uri ) -> VFSResult<URI> {
|
||||
auto found = archive->find( member, uri );
|
||||
if ( !found )
|
||||
return Err( std::move( found ).error() );
|
||||
auto backing = ZipProvider::archiveURI( uri );
|
||||
URI normalized = ZipProvider::makeURI( backing.value(), member );
|
||||
return Ok( std::move( normalized ) );
|
||||
} );
|
||||
}
|
||||
|
||||
}} // namespace EE::VFS
|
||||
@@ -18,7 +18,7 @@ namespace EE { namespace VFS { namespace sftp {
|
||||
* time. Speaks the agent protocol over Unix-domain sockets; elsewhere (Windows named pipes,
|
||||
* Pageant) supported() is false and libssh2's own agent support is used.
|
||||
*/
|
||||
class SSHAgent {
|
||||
class EE_VFS_API SSHAgent {
|
||||
public:
|
||||
struct Identity {
|
||||
/** Public key blob (SSH wire format). */
|
||||
|
||||
@@ -3,6 +3,7 @@
|
||||
#include <eepp/vfs/detail/filewriter.hpp>
|
||||
#include <eepp/vfs/providers/localprovider.hpp>
|
||||
#include <eepp/vfs/providers/resourceprovider.hpp>
|
||||
#include <eepp/vfs/providers/zipprovider.hpp>
|
||||
#include <eepp/vfs/transferengine.hpp>
|
||||
#include <eepp/vfs/virtualfilesystem.hpp>
|
||||
#include <eepp/vfs/watchers/localwatchbackend.hpp>
|
||||
@@ -69,6 +70,7 @@ VirtualFileSystem::~VirtualFileSystem() {
|
||||
|
||||
void VirtualFileSystem::registerDefaultProviders() {
|
||||
registerProvider( std::make_shared<LocalProvider>( mLocalIO ) );
|
||||
registerProvider( std::make_shared<ZipProvider>( mLocalIO ) );
|
||||
registerProvider( std::make_shared<ResourceProvider>( mIOPool ) );
|
||||
// Network mounts are polled through the VFS (whose calls time out), not by efsw.
|
||||
auto polling = std::make_shared<PollingWatchBackend>( *this );
|
||||
|
||||
@@ -26,7 +26,7 @@ class LocalWatchBackendImpl;
|
||||
* Directories on a network mount go to networkBackend instead (the VFS polling backend): efsw's
|
||||
* own threads would block on an unreachable server, while VFS calls time out.
|
||||
*/
|
||||
class LocalWatchBackend : public WatchBackend {
|
||||
class EE_VFS_API LocalWatchBackend : public WatchBackend {
|
||||
public:
|
||||
/** @param pool Where watches are registered and removed.
|
||||
* @param networkBackend Watches directories on network mounts; without it efsw does.
|
||||
|
||||
@@ -12,7 +12,7 @@ namespace EE { namespace VFS {
|
||||
|
||||
namespace detail {
|
||||
|
||||
class PollingWatchBackendImpl {
|
||||
class PollingWatchBackendImpl : public std::enable_shared_from_this<PollingWatchBackendImpl> {
|
||||
public:
|
||||
struct Snapshot {
|
||||
struct Entry {
|
||||
@@ -52,8 +52,12 @@ class PollingWatchBackendImpl {
|
||||
mStopping = true;
|
||||
}
|
||||
mWake.notify_all();
|
||||
if ( mThread.joinable() )
|
||||
mThread.join();
|
||||
if ( mThread.joinable() ) {
|
||||
if ( mThread.get_id() == std::this_thread::get_id() )
|
||||
mThread.detach();
|
||||
else
|
||||
mThread.join();
|
||||
}
|
||||
}
|
||||
|
||||
/** Takes the initial snapshot on the polling thread and resolves once the watch is active.
|
||||
@@ -88,7 +92,9 @@ class PollingWatchBackendImpl {
|
||||
|
||||
void ensureThread() {
|
||||
if ( !mThread.joinable() )
|
||||
mThread = std::thread( [this] { loop(); } );
|
||||
// A synchronous watch callback may release the backend on this thread. Keep its
|
||||
// state alive until the loop observes stop(), without joining the current thread.
|
||||
mThread = std::thread( [self = shared_from_this()] { self->loop(); } );
|
||||
}
|
||||
|
||||
void loop() {
|
||||
@@ -101,7 +107,9 @@ class PollingWatchBackendImpl {
|
||||
auto snapshot = take( *pending.watch );
|
||||
lock.lock();
|
||||
if ( !snapshot ) {
|
||||
lock.unlock();
|
||||
pending.resolve( Err( std::move( snapshot ).error() ) );
|
||||
lock.lock();
|
||||
continue;
|
||||
}
|
||||
pending.watch->snapshot = std::move( snapshot ).value();
|
||||
@@ -139,8 +147,10 @@ class PollingWatchBackendImpl {
|
||||
while ( !mPendingAdds.empty() ) {
|
||||
auto pending = std::move( mPendingAdds.front() );
|
||||
mPendingAdds.pop_front();
|
||||
lock.unlock();
|
||||
pending.resolve( Err( VFSError( VFSErrorCode::InvalidOperation, pending.watch->root,
|
||||
"Watch backend shutting down" ) ) );
|
||||
lock.lock();
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -21,7 +21,7 @@ class PollingWatchBackendImpl;
|
||||
* successful poll emits RescanRequired. Recursive polling is limited to MaxEntries entries per
|
||||
* watch; larger trees degrade to the top directories that fit.
|
||||
*/
|
||||
class PollingWatchBackend : public WatchBackend {
|
||||
class EE_VFS_API PollingWatchBackend : public WatchBackend {
|
||||
public:
|
||||
static constexpr size_t MaxEntries = 100000;
|
||||
|
||||
|
||||
@@ -12,6 +12,7 @@
|
||||
#include <eepp/vfs/providers/localioscheduler.hpp>
|
||||
#include <eepp/vfs/providers/localprovider.hpp>
|
||||
#include <eepp/vfs/watchers/localwatchbackend.hpp>
|
||||
#include <eepp/vfs/watchers/pollingwatchbackend.hpp>
|
||||
#include <mutex>
|
||||
#include <optional>
|
||||
#include <thread>
|
||||
@@ -460,12 +461,15 @@ UTEST( VFSResource, zipPackWithSeveralFiles ) {
|
||||
EXPECT_TRUE( zip.addFile( a, "vfs-zip/a.txt" ) );
|
||||
EXPECT_TRUE( zip.addFile( b, "vfs-zip/sub/b.txt" ) );
|
||||
EXPECT_TRUE( zip.addFile( c, "vfs-zip/sub/c.txt" ) );
|
||||
std::vector<Uint8> empty;
|
||||
EXPECT_TRUE( zip.addFile( empty, "vfs-zip/empty.txt" ) );
|
||||
EXPECT_TRUE( zip.addFile( empty, "vfs-zip/empty-dir/" ) );
|
||||
EXPECT_TRUE( zip.close() );
|
||||
}
|
||||
|
||||
Zip zip;
|
||||
ASSERT_TRUE( zip.open( zipPath ) );
|
||||
EXPECT_EQ( 3u, zip.getFileList().size() );
|
||||
EXPECT_EQ( 4u, zip.getFileList().size() );
|
||||
std::vector<Uint8> data;
|
||||
EXPECT_TRUE( zip.extractFileToMemory( "vfs-zip/sub/c.txt", data ) );
|
||||
EXPECT_STDSTREQ( "c3", std::string( data.begin(), data.end() ) );
|
||||
@@ -473,6 +477,8 @@ UTEST( VFSResource, zipPackWithSeveralFiles ) {
|
||||
VirtualFileSystem vfs;
|
||||
EXPECT_STDSTREQ( "a1", readString( vfs, URI( "res:///vfs-zip/a.txt" ) ) );
|
||||
EXPECT_STDSTREQ( "b2", readString( vfs, URI( "res:///vfs-zip/sub/b.txt" ) ) );
|
||||
EXPECT_STDSTREQ( "", readString( vfs, URI( "res:///vfs-zip/empty.txt" ) ) );
|
||||
EXPECT_FALSE( PackFileIndex::instance()->fileExists( "vfs-zip/empty-dir" ) );
|
||||
auto listed = waitFor( vfs.listAll( URI( "res:///vfs-zip/sub" ) ) );
|
||||
ASSERT_TRUE( listed.hasValue() );
|
||||
EXPECT_EQ( 2u, listed.value().size() );
|
||||
@@ -1290,3 +1296,45 @@ UTEST( LocalIOScheduler, linksIntoAMountRunOnItsLane ) {
|
||||
gate->open();
|
||||
}
|
||||
#endif
|
||||
|
||||
namespace {
|
||||
|
||||
class ReleasePollingBackendSink : public WatchSink {
|
||||
public:
|
||||
URI uri;
|
||||
std::mutex gate;
|
||||
std::shared_ptr<PollingWatchBackend> backend;
|
||||
|
||||
void push( WatchEvent&& ) override {}
|
||||
|
||||
void push( std::vector<WatchEvent>&& ) override {}
|
||||
|
||||
void rescanRequired() override {}
|
||||
|
||||
void setStatus( WatchStatus ) override {
|
||||
std::lock_guard<std::mutex> lock( gate );
|
||||
backend.reset();
|
||||
}
|
||||
|
||||
const URI& root() const override { return uri; }
|
||||
};
|
||||
|
||||
} // namespace
|
||||
|
||||
UTEST( VFSWatch, pollingCallbackCanReleaseTheLastBackendOwner ) {
|
||||
VirtualFileSystem vfs;
|
||||
auto memory = std::make_shared<MemoryProvider>();
|
||||
vfs.registerProvider( memory );
|
||||
const URI root( "mem://polling-teardown/dir" );
|
||||
memory->putFile( root.withAppendedSegment( "file" ), "contents" );
|
||||
auto sink = std::make_shared<ReleasePollingBackendSink>();
|
||||
sink->uri = root;
|
||||
sink->backend = std::make_shared<PollingWatchBackend>( vfs );
|
||||
std::weak_ptr<PollingWatchBackend> weak = sink->backend;
|
||||
std::unique_lock<std::mutex> hold( sink->gate );
|
||||
auto task = sink->backend->watch( root, {}, sink );
|
||||
hold.unlock();
|
||||
ASSERT_EQ( TaskState::Ready, task.waitFor( std::chrono::seconds( 5 ) ) );
|
||||
ASSERT_TRUE( waitFor( std::move( task ) ).hasValue() );
|
||||
EXPECT_TRUE( weak.expired() );
|
||||
}
|
||||
|
||||
@@ -0,0 +1,440 @@
|
||||
#include "../../tools/eefm/location.hpp"
|
||||
#include "utest.hpp"
|
||||
#include "vfs_test_utils.hpp"
|
||||
#include <array>
|
||||
#include <cerrno>
|
||||
#include <eepp/system/base64.hpp>
|
||||
#include <eepp/system/packfileindex.hpp>
|
||||
#include <eepp/system/zip.hpp>
|
||||
#include <eepp/vfs/providers/zipprovider.hpp>
|
||||
#include <fstream>
|
||||
|
||||
using namespace EE;
|
||||
using namespace EE::System;
|
||||
using namespace EE::VFS;
|
||||
using namespace vfstest;
|
||||
|
||||
namespace {
|
||||
|
||||
bool makeZip( const std::string& path,
|
||||
std::initializer_list<std::pair<std::string, std::string>> files ) {
|
||||
Zip archive;
|
||||
if ( !archive.create( path ) )
|
||||
return false;
|
||||
for ( const auto& [name, data] : files ) {
|
||||
if ( !archive.addFile( reinterpret_cast<const Uint8*>( data.data() ), data.size(), name ) )
|
||||
return false;
|
||||
}
|
||||
return archive.close();
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
UTEST( ZipProvider, archiveURIsPreserveIdentityAndMemberNames ) {
|
||||
TempDir temp;
|
||||
const URI file = URI::fromFSPath( temp.path + "Mixed Case !!#%?.zip" );
|
||||
const URI root = ZipProvider::makeURI( file );
|
||||
const URI member = root.withAppendedSegment( "dir" ).withAppendedSegment( "Name #%?.txt" );
|
||||
const URI roundTrip( member.toString() );
|
||||
EXPECT_TRUE( member == roundTrip );
|
||||
auto backing = ZipProvider::archiveURI( roundTrip );
|
||||
ASSERT_TRUE( backing.hasValue() );
|
||||
EXPECT_TRUE( file == backing.value() );
|
||||
EXPECT_TRUE( file == ZipProvider::archiveURI( member.getParent().getParent() ).value() );
|
||||
EXPECT_FALSE( ZipProvider::archiveURI( root.getParent() ).hasValue() );
|
||||
EXPECT_FALSE( root == ZipProvider::makeURI( URI::fromFSPath( temp.path + "other.zip" ) ) );
|
||||
EXPECT_TRUE( member.isSameOrChildOf( root ) );
|
||||
const URI bangArchive = URI::fromFSPath( temp.path + "packages!/archive!.zip" );
|
||||
const URI bangMember = ZipProvider::makeURI( bangArchive, "/folder!/name!.txt" );
|
||||
EXPECT_TRUE( bangArchive == ZipProvider::archiveURI( URI( bangMember.toString() ) ).value() );
|
||||
EXPECT_STDSTREQ( "/folder!/name!.txt", ZipProvider::memberPath( bangMember ).value() );
|
||||
const URI clamped = ZipProvider::makeURI( file, "/../../inside" );
|
||||
EXPECT_TRUE( file == ZipProvider::archiveURI( clamped ).value() );
|
||||
EXPECT_STDSTREQ( "/inside", ZipProvider::memberPath( clamped ).value() );
|
||||
EXPECT_EQ( VFSErrorCode::InvalidURI,
|
||||
codeOf( ZipProvider::archiveURI( URI( "zip://nothex/" ) ) ) );
|
||||
EXPECT_EQ(
|
||||
VFSErrorCode::InvalidURI,
|
||||
codeOf( ZipProvider::archiveURI( ZipProvider::makeURI( URI( "sftp://host/a.zip" ) ) ) ) );
|
||||
}
|
||||
|
||||
UTEST( ZipProvider, listsImplicitAndEmptyDirectoriesWithMetadata ) {
|
||||
TempDir temp;
|
||||
const std::string path = temp.path + "archive.zip";
|
||||
ASSERT_TRUE( makeZip( path, { { "top.txt", "hello" },
|
||||
{ "dir/leaf.txt", "leaf" },
|
||||
{ "empty/", "" },
|
||||
{ ".hidden", "secret" },
|
||||
{ "dir/Upper #%.txt", "mixed" } } ) );
|
||||
VirtualFileSystem vfs;
|
||||
const URI root = ZipProvider::makeURI( URI::fromFSPath( path ) );
|
||||
auto listed = waitFor( vfs.listAll( root ) );
|
||||
ASSERT_TRUE( listed.hasValue() );
|
||||
EXPECT_TRUE( sortedNames( listed.value() ) ==
|
||||
std::vector<std::string>( { ".hidden", "dir", "empty", "top.txt" } ) );
|
||||
auto folder = waitFor( vfs.stat( root.withAppendedSegment( "dir" ) ) );
|
||||
ASSERT_TRUE( folder.hasValue() );
|
||||
EXPECT_TRUE( folder.value().isDirectory() );
|
||||
auto file = waitFor( vfs.stat( root.withAppendedSegment( "top.txt" ) ) );
|
||||
ASSERT_TRUE( file.hasValue() );
|
||||
EXPECT_EQ( 5u, file.value().size().value_or( 0 ) );
|
||||
EXPECT_TRUE( file.value().modifiedTime().has_value() );
|
||||
EXPECT_FALSE( file.value().revision.empty() );
|
||||
EXPECT_FALSE( file.value().permissions().has_value() );
|
||||
EXPECT_FALSE( folder.value().permissions().has_value() );
|
||||
auto empty = waitFor( vfs.listAll( root.withAppendedSegment( "empty" ) ) );
|
||||
ASSERT_TRUE( empty.hasValue() );
|
||||
EXPECT_TRUE( empty.value().empty() );
|
||||
EXPECT_STDSTREQ( "mixed",
|
||||
readString( vfs, root.withAppendedSegment( "dir" ).withAppendedSegment(
|
||||
"Upper #%.txt" ) ) );
|
||||
auto opened = waitFor( vfs.openDirectory( root ) );
|
||||
ASSERT_TRUE( opened.hasValue() );
|
||||
auto handle = std::move( opened ).value();
|
||||
size_t count = 0;
|
||||
bool end = false;
|
||||
while ( !end ) {
|
||||
auto batch = waitFor( handle.read( 1 ) );
|
||||
ASSERT_TRUE( batch.hasValue() );
|
||||
count += batch.value().entries.size();
|
||||
end = batch.value().endOfDirectory;
|
||||
}
|
||||
EXPECT_EQ( 4u, count );
|
||||
EXPECT_TRUE( waitFor( handle.close() ).hasValue() );
|
||||
}
|
||||
|
||||
UTEST( ZipProvider, streamingReadsSeekBackwardsAndCopyAcrossProviders ) {
|
||||
TempDir temp;
|
||||
std::string content( 300000, 'x' );
|
||||
Uint32 random = 1;
|
||||
for ( auto& byte : content ) {
|
||||
random = random * 1664525u + 1013904223u;
|
||||
byte = static_cast<char>( random >> 24 );
|
||||
}
|
||||
const std::string path = temp.path + "archive.zip";
|
||||
ASSERT_TRUE( makeZip(
|
||||
path, { { "dir/large.txt", content }, { "dir/second.txt", "second" }, { "zero", "" } } ) );
|
||||
VirtualFileSystem vfs;
|
||||
const URI root = ZipProvider::makeURI( URI::fromFSPath( path ) );
|
||||
const URI file = root.withAppendedSegment( "dir" ).withAppendedSegment( "large.txt" );
|
||||
auto opened = waitFor( vfs.open( file ) );
|
||||
ASSERT_TRUE( opened.hasValue() );
|
||||
auto handle = std::move( opened ).value();
|
||||
std::array<Uint8, 31> buffer;
|
||||
for ( Uint64 offset : { 0ull, 100000ull, 15ull, 299980ull, 300000ull } ) {
|
||||
auto result = waitFor( handle.read( offset, buffer ) );
|
||||
ASSERT_TRUE( result.hasValue() );
|
||||
const std::string expected = content.substr( offset, buffer.size() );
|
||||
EXPECT_EQ( expected.size(), result.value() );
|
||||
EXPECT_TRUE( expected == std::string( buffer.begin(), buffer.begin() + result.value() ) );
|
||||
}
|
||||
EXPECT_TRUE( waitFor( handle.close() ).hasValue() );
|
||||
EXPECT_EQ( VFSErrorCode::InvalidOperation, codeOf( waitFor( handle.read( 0, buffer ) ) ) );
|
||||
EXPECT_STDSTREQ( "", readString( vfs, root.withAppendedSegment( "zero" ) ) );
|
||||
EXPECT_TRUE( content == readString( vfs, file ) );
|
||||
const URI destination = URI::fromFSPath( temp.path + "extracted" );
|
||||
auto copied =
|
||||
waitFor( vfs.copy( root.withAppendedSegment( "dir" ), destination ).completion() );
|
||||
ASSERT_TRUE( copied.hasValue() );
|
||||
EXPECT_TRUE( content == readString( vfs, destination.withAppendedSegment( "large.txt" ) ) );
|
||||
EXPECT_STDSTREQ( "second", readString( vfs, destination.withAppendedSegment( "second.txt" ) ) );
|
||||
EXPECT_TRUE( waitFor( vfs.readAll( file, {}, 100 ) ).hasError() );
|
||||
// An in-place edit can break a backward seek. Keep the old stream alive when reopening
|
||||
// fails: this is an I/O failure, and must not turn every later call into "File closed".
|
||||
auto reopened = waitFor( vfs.open( file ) );
|
||||
ASSERT_TRUE( reopened.hasValue() );
|
||||
auto changing = std::move( reopened ).value();
|
||||
ASSERT_TRUE( waitFor( changing.read( 200000, buffer ) ).hasValue() );
|
||||
ASSERT_TRUE( FileSystem::fileWrite( path, std::string{} ) );
|
||||
EXPECT_TRUE( waitFor( changing.read( 0, buffer ) ).hasError() );
|
||||
auto eof = waitFor( changing.read( content.size(), buffer ) );
|
||||
ASSERT_TRUE( eof.hasValue() );
|
||||
EXPECT_EQ( 0u, eof.value() );
|
||||
}
|
||||
|
||||
UTEST( ZipProvider, readOnlyErrorsMissingFilesAndCancellation ) {
|
||||
TempDir temp;
|
||||
const std::string path = temp.path + "archive.zip";
|
||||
ASSERT_TRUE( makeZip( path, { { "file", "contents" } } ) );
|
||||
VirtualFileSystem vfs;
|
||||
const URI root = ZipProvider::makeURI( URI::fromFSPath( path ) );
|
||||
const URI file = root.withAppendedSegment( "file" );
|
||||
EXPECT_EQ( VFSErrorCode::ReadOnly, codeOf( waitFor( vfs.open( file, OpenFlags::Write ) ) ) );
|
||||
EXPECT_EQ( VFSErrorCode::ReadOnly,
|
||||
codeOf( waitFor( vfs.makeDir( root.withAppendedSegment( "new" ) ) ) ) );
|
||||
EXPECT_EQ( VFSErrorCode::ReadOnly, codeOf( waitFor( vfs.remove( file ) ) ) );
|
||||
EXPECT_EQ( VFSErrorCode::ReadOnly,
|
||||
codeOf( waitFor( vfs.rename( file, root.withAppendedSegment( "renamed" ) ) ) ) );
|
||||
EXPECT_EQ( VFSErrorCode::IsDirectory, codeOf( waitFor( vfs.open( root ) ) ) );
|
||||
EXPECT_EQ( VFSErrorCode::NotDirectory, codeOf( waitFor( vfs.openDirectory( file ) ) ) );
|
||||
const URI fileWithSlash = ZipProvider::makeURI( URI::fromFSPath( path ), "/file/" );
|
||||
EXPECT_EQ( VFSErrorCode::NotDirectory, codeOf( waitFor( vfs.stat( fileWithSlash ) ) ) );
|
||||
EXPECT_EQ( VFSErrorCode::NotDirectory, codeOf( waitFor( vfs.open( fileWithSlash ) ) ) );
|
||||
EXPECT_EQ( VFSErrorCode::NotDirectory,
|
||||
codeOf( waitFor( vfs.stat( file.withAppendedSegment( "child" ) ) ) ) );
|
||||
EXPECT_EQ( VFSErrorCode::NotFound,
|
||||
codeOf( waitFor( vfs.stat( root.withAppendedSegment( "missing" ) ) ) ) );
|
||||
CancellationSource cancelled;
|
||||
cancelled.cancel();
|
||||
EXPECT_EQ( VFSErrorCode::Cancelled,
|
||||
codeOf( waitFor( vfs.open( file, {}, cancelled.token() ) ) ) );
|
||||
const URI missing = ZipProvider::makeURI( URI::fromFSPath( temp.path + "missing.zip" ) );
|
||||
EXPECT_EQ( VFSErrorCode::NotFound, codeOf( waitFor( vfs.stat( missing ) ) ) );
|
||||
EXPECT_FALSE( FileSystem::fileExists( temp.path + "missing.zip" ) );
|
||||
ASSERT_TRUE( FileSystem::fileWrite( temp.path + "bad.zip", "this is not a ZIP" ) );
|
||||
EXPECT_EQ( VFSErrorCode::InvalidOperation,
|
||||
codeOf( waitFor( vfs.stat(
|
||||
ZipProvider::makeURI( URI::fromFSPath( temp.path + "bad.zip" ) ) ) ) ) );
|
||||
EXPECT_FALSE( vfs.capabilities( file ).has( VFSCapability::Write ) );
|
||||
auto opened = waitFor( vfs.open( file ) );
|
||||
ASSERT_TRUE( opened.hasValue() );
|
||||
auto handle = std::move( opened ).value();
|
||||
const std::array<Uint8, 1> byte{ 'x' };
|
||||
EXPECT_EQ( VFSErrorCode::ReadOnly, codeOf( waitFor( handle.write( 0, byte ) ) ) );
|
||||
EXPECT_EQ( VFSErrorCode::ReadOnly, codeOf( waitFor( handle.truncate( 0 ) ) ) );
|
||||
}
|
||||
|
||||
UTEST( ZipProvider, browsingNeverRegistersResourcesAndNoticesArchiveReplacement ) {
|
||||
TempDir temp;
|
||||
const std::string path = temp.path + "archive.zip";
|
||||
ASSERT_TRUE( makeZip( path, { { "zip-provider-only.txt", "old" } } ) );
|
||||
const Uint64 generation = PackFileIndex::instance()->getGeneration();
|
||||
VirtualFileSystem vfs;
|
||||
const URI root = ZipProvider::makeURI( URI::fromFSPath( path ) );
|
||||
EXPECT_STDSTREQ( "old",
|
||||
readString( vfs, root.withAppendedSegment( "zip-provider-only.txt" ) ) );
|
||||
EXPECT_EQ( generation, PackFileIndex::instance()->getGeneration() );
|
||||
EXPECT_FALSE( PackFileIndex::instance()->fileExists( "zip-provider-only.txt" ) );
|
||||
auto oldInfo = waitFor( vfs.stat( root.withAppendedSegment( "zip-provider-only.txt" ) ) );
|
||||
ASSERT_TRUE( oldInfo.hasValue() );
|
||||
// Replace only after handles have closed (portable to Windows, where an open file can
|
||||
// prevent replacement). A fresh operation must discard the cached directory index.
|
||||
ASSERT_TRUE( FileSystem::fileRemove( path ) );
|
||||
ASSERT_TRUE( makeZip( path, { { "new.txt", "new contents" } } ) );
|
||||
auto listing = waitFor( vfs.listAll( root ) );
|
||||
ASSERT_TRUE( listing.hasValue() );
|
||||
EXPECT_TRUE( sortedNames( listing.value() ) == std::vector<std::string>( { "new.txt" } ) );
|
||||
EXPECT_STDSTREQ( "new contents", readString( vfs, root.withAppendedSegment( "new.txt" ) ) );
|
||||
EXPECT_EQ(
|
||||
VFSErrorCode::NotFound,
|
||||
codeOf( waitFor( vfs.stat( root.withAppendedSegment( "zip-provider-only.txt" ) ) ) ) );
|
||||
}
|
||||
|
||||
UTEST( Zip, standaloneReaderIsReadOnlyAndHasBoundedSeek ) {
|
||||
TempDir temp;
|
||||
const auto path = temp.path + "archive.zip";
|
||||
const std::string content( 200000, 'a' );
|
||||
ASSERT_TRUE( makeZip( path, { { "large", content } } ) );
|
||||
std::string original;
|
||||
ASSERT_TRUE( FileSystem::fileGet( path, original ) );
|
||||
{
|
||||
Zip reader( false );
|
||||
ASSERT_TRUE( reader.open( path, Zip::OpenMode::ReadOnly ) );
|
||||
EXPECT_TRUE( reader.isOpen() );
|
||||
const std::array<Uint8, 1> byte{ 'x' };
|
||||
EXPECT_FALSE( reader.addFile( byte.data(), byte.size(), "forbidden" ) );
|
||||
EXPECT_TRUE( reader.error() == Zip::Error::ReadOnly );
|
||||
EXPECT_FALSE( reader.eraseFile( "large" ) );
|
||||
EXPECT_EQ( 1u, reader.entryCount() );
|
||||
Zip::Entry entry;
|
||||
ASSERT_TRUE( reader.entry( 0, entry ) );
|
||||
EXPECT_EQ( content.size(), entry.size );
|
||||
auto stream = reader.openStream( 0 );
|
||||
ASSERT_TRUE( stream != nullptr );
|
||||
EXPECT_EQ( 199999, stream->seek( 199999 ) );
|
||||
char value = 0;
|
||||
EXPECT_EQ( 1, stream->read( &value, 1 ) );
|
||||
EXPECT_EQ( int( 'a' ), int( value ) );
|
||||
EXPECT_EQ( 0, stream->seek( 0 ) );
|
||||
EXPECT_EQ( 1, stream->read( &value, 1 ) );
|
||||
}
|
||||
std::string after;
|
||||
ASSERT_TRUE( FileSystem::fileGet( path, after ) );
|
||||
EXPECT_TRUE( original == after );
|
||||
Zip missing( false );
|
||||
EXPECT_FALSE( missing.open( temp.path + "missing.zip", Zip::OpenMode::ReadOnly ) );
|
||||
EXPECT_TRUE( missing.error() == Zip::Error::NotFound );
|
||||
EXPECT_FALSE( FileSystem::fileExists( temp.path + "missing.zip" ) );
|
||||
EXPECT_FALSE( missing.open( temp.path + "missing.zip" ) );
|
||||
EXPECT_FALSE( FileSystem::fileExists( temp.path + "missing.zip" ) );
|
||||
ASSERT_TRUE( FileSystem::fileWrite( temp.path + "invalid.zip", "not an archive" ) );
|
||||
errno = EACCES;
|
||||
EXPECT_FALSE( missing.open( temp.path + "invalid.zip", Zip::OpenMode::ReadOnly ) );
|
||||
EXPECT_TRUE( missing.error() == Zip::Error::InvalidArchive );
|
||||
errno = EACCES;
|
||||
EXPECT_FALSE( missing.open( temp.path + "invalid.zip" ) );
|
||||
EXPECT_TRUE( missing.error() == Zip::Error::InvalidArchive );
|
||||
}
|
||||
|
||||
UTEST( ZipLocation, readableAddressesRoundTripAndRootReturnsToContainingFolder ) {
|
||||
TempDir temp;
|
||||
const URI archive = URI::fromFSPath( temp.path + "Mixed # %!?.zip" );
|
||||
const URI root = ZipProvider::makeURI( archive );
|
||||
const URI member = root.withAppendedSegment( "a # %.txt" );
|
||||
EXPECT_TRUE( eefm::Location::parse( eefm::Location::text( root ).toUtf8() ) == root );
|
||||
EXPECT_TRUE( eefm::Location::parse( eefm::Location::text( member ).toUtf8() ) == member );
|
||||
EXPECT_TRUE( URI( eefm::Location::text( member ).toUtf8() ) == member );
|
||||
EXPECT_TRUE( eefm::Location::text( root ).toUtf8().starts_with( "zip:///" ) );
|
||||
EXPECT_TRUE( ZipProvider::archiveURI( URI( "zip:///tmp/archive.zip!/docs" ) ).hasValue() );
|
||||
EXPECT_TRUE( eefm::Location::parent( root ) == archive.getParent() );
|
||||
EXPECT_TRUE( eefm::Location::parent( member ) == root );
|
||||
EXPECT_STDSTREQ( archive.getLastPathSegment(), eefm::Location::entryName( root ) );
|
||||
EXPECT_STDSTREQ( archive.getLastPathSegment(), eefm::Location::displayName( root ).toUtf8() );
|
||||
}
|
||||
|
||||
UTEST( ZipProvider, unsafeNamesAreExcludedAndCollisionsAreDeterministic ) {
|
||||
TempDir temp;
|
||||
// Stored ZIP with traversal/absolute/drive names, duplicate entries, and a file/directory
|
||||
// collision. Kept as a fixture because Zip::addFile refuses duplicate names.
|
||||
const std::string encoded = "UEsDBBQAAAAAADIIUV2PKKQfBAAAAAQAAAAIAAAAc2FmZS50eHRzYWZlUEsDBBQAAA"
|
||||
"AAADIIUV37OSuCAwAAAAMAAAAN"
|
||||
"AAAALi4vZXNjYXBlLnR4dGJhZFBLAwQUAAAAAAAyCFFd+"
|
||||
"zkrggMAAAADAAAADQAAAC9hYnNvbHV0ZS50eHRiYWRQSwME"
|
||||
"FAAAAAAAMghRXfs5K4IDAAAAAwAAABQAAABkaXIvLi4vLi4vZXNjYXBlLnR4dGJhZF"
|
||||
"BLAwQUAAAAAAAyCFFd+zkrggMA"
|
||||
"AAADAAAACwAAAEM6ZHJpdmUudHh0YmFkUEsDBBQAAAAAADIIUV37OSuCAwAAAAMAAA"
|
||||
"AOAAAAZGlyXGVzY2FwZS50eHRi"
|
||||
"YWRQSwMEFAAAAAAAMghRXfs5K4IDAAAAAwAAAA4AAAAuLiAvZXNjYXBlLnR4dGJhZF"
|
||||
"BLAwQUAAAAAAAyCFFdAAAAAAAA"
|
||||
"AAAAAAAABgAAAGVtcHR5L1BLAwQUAAAAAAAyCFFdEDafjAQAAAAEAAAACQAAAGNvbG"
|
||||
"xpc2lvbmZpbGVQSwMEFAAAAAAA"
|
||||
"MghRXSlUsyIFAAAABQAAAA8AAABjb2xsaXNpb24vY2hpbGRjaGlsZFBLAwQUAAAAAA"
|
||||
"AyCFFdV+5xkgUAAAAFAAAACQAA"
|
||||
"AGR1cGxpY2F0ZWZpcnN0UEsDBBQAAAAAADIIUV1pER+"
|
||||
"2BgAAAAYAAAAJAAAAZHVwbGljYXRlc2Vjb25kUEsBAhQDFAAA"
|
||||
"AAAAMghRXY8opB8EAAAABAAAAAgAAAAAAAAAAAAAAIABAAAAAHNhZmUudHh0UEsBAh"
|
||||
"QDFAAAAAAAMghRXfs5K4IDAAAA"
|
||||
"AwAAAA0AAAAAAAAAAAAAAIABKgAAAC4uL2VzY2FwZS50eHRQSwECFAMUAAAAAAAyCF"
|
||||
"Fd+zkrggMAAAADAAAADQAAAAAA"
|
||||
"AAAAAAAAgAFYAAAAL2Fic29sdXRlLnR4dFBLAQIUAxQAAAAAADIIUV37OSuCAwAAAA"
|
||||
"MAAAAUAAAAAAAAAAAAAACAAYYA"
|
||||
"AABkaXIvLi4vLi4vZXNjYXBlLnR4dFBLAQIUAxQAAAAAADIIUV37OSuCAwAAAAMAAA"
|
||||
"ALAAAAAAAAAAAAAACAAbsAAABD"
|
||||
"OmRyaXZlLnR4dFBLAQIUAxQAAAAAADIIUV37OSuCAwAAAAMAAAAOAAAAAAAAAAAAAA"
|
||||
"CAAecAAABkaXJcZXNjYXBlLnR4"
|
||||
"dFBLAQIUAxQAAAAAADIIUV37OSuCAwAAAAMAAAAOAAAAAAAAAAAAAACAARYBAAAuLi"
|
||||
"AvZXNjYXBlLnR4dFBLAQIUAxQA"
|
||||
"AAAAADIIUV0AAAAAAAAAAAAAAAAGAAAAAAAAAAAAEAD9QUUBAABlbXB0eS9QSwECFA"
|
||||
"MUAAAAAAAyCFFdEDafjAQAAAAE"
|
||||
"AAAACQAAAAAAAAAAAAAAgAFpAQAAY29sbGlzaW9uUEsBAhQDFAAAAAAAMghRXSlUsy"
|
||||
"IFAAAABQAAAA8AAAAAAAAAAAAA"
|
||||
"AIABlAEAAGNvbGxpc2lvbi9jaGlsZFBLAQIUAxQAAAAAADIIUV1X7nGSBQAAAAUAAA"
|
||||
"AJAAAAAAAAAAAAAACAAcYBAABk"
|
||||
"dXBsaWNhdGVQSwECFAMUAAAAAAAyCFFdaREftgYAAAAGAAAACQAAAAAAAAAAAAAAgA"
|
||||
"HyAQAAZHVwbGljYXRlUEsFBgAA"
|
||||
"AAAMAAwAtQIAAB8CAAAAAA==";
|
||||
std::string bytes;
|
||||
ASSERT_NE( size_t( -1 ), Base64::decode( encoded, bytes ) );
|
||||
ASSERT_TRUE( FileSystem::fileWrite( temp.path + "unsafe.zip", bytes ) );
|
||||
VirtualFileSystem vfs;
|
||||
const URI root = ZipProvider::makeURI( URI::fromFSPath( temp.path + "unsafe.zip" ) );
|
||||
auto listed = waitFor( vfs.listAll( root ) );
|
||||
ASSERT_TRUE( listed.hasValue() );
|
||||
EXPECT_TRUE( sortedNames( listed.value() ) ==
|
||||
std::vector<std::string>( { "collision", "duplicate", "empty", "safe.txt" } ) );
|
||||
EXPECT_STDSTREQ( "first", readString( vfs, root.withAppendedSegment( "duplicate" ) ) );
|
||||
EXPECT_STDSTREQ(
|
||||
"child",
|
||||
readString( vfs, root.withAppendedSegment( "collision" ).withAppendedSegment( "child" ) ) );
|
||||
const auto destination = URI::fromFSPath( temp.path + "out" );
|
||||
ASSERT_TRUE( waitFor( vfs.copy( root, destination ).completion() ).hasValue() );
|
||||
EXPECT_FALSE( FileSystem::fileExists( temp.path + "escape.txt" ) );
|
||||
EXPECT_STDSTREQ( "safe", readString( vfs, destination.withAppendedSegment( "safe.txt" ) ) );
|
||||
}
|
||||
|
||||
UTEST( ZipProvider, crcFailureIsReportedEvenForAnExactSizedRead ) {
|
||||
TempDir temp;
|
||||
const std::string encoded =
|
||||
"UEsDBBQAAAAAADIIUV0Ks4YUDwAAAA8AAAALAAAAY2hlY2tlZC50eHRjaGVja2VkIHBheWxvYWRQSwECFAMUAAAAAA"
|
||||
"Ay"
|
||||
"CFFdCrOGFA8AAAAPAAAACwAAAAAAAAAAAAAAgAEAAAAAY2hlY2tlZC50eHRQSwUGAAAAAAEAAQA5AAAAOAAAAAAA";
|
||||
std::string bytes;
|
||||
ASSERT_NE( size_t( -1 ), Base64::decode( encoded, bytes ) );
|
||||
const auto payload = bytes.find( "checked payload" );
|
||||
ASSERT_NE( std::string::npos, payload );
|
||||
bytes[payload] ^= 1;
|
||||
ASSERT_TRUE( FileSystem::fileWrite( temp.path + "corrupt.zip", bytes ) );
|
||||
VirtualFileSystem vfs;
|
||||
const URI root = ZipProvider::makeURI( URI::fromFSPath( temp.path + "corrupt.zip" ) );
|
||||
auto opened = waitFor( vfs.open( root.withAppendedSegment( "checked.txt" ) ) );
|
||||
ASSERT_TRUE( opened.hasValue() );
|
||||
auto handle = std::move( opened ).value();
|
||||
std::array<Uint8, 15> data;
|
||||
EXPECT_EQ( VFSErrorCode::IO, codeOf( waitFor( handle.read( 0, data ) ) ) );
|
||||
}
|
||||
|
||||
UTEST( ZipProvider, openHandlesSurviveCacheEvictionAndProviderRemoval ) {
|
||||
TempDir temp;
|
||||
const auto path = temp.path + "original.zip";
|
||||
ASSERT_TRUE( makeZip( path, { { "file", "original" }, { "other", "second" } } ) );
|
||||
VirtualFileSystem vfs;
|
||||
const URI root = ZipProvider::makeURI( URI::fromFSPath( path ) );
|
||||
auto first = waitFor( vfs.open( root.withAppendedSegment( "file" ) ) );
|
||||
auto second = waitFor( vfs.open( root.withAppendedSegment( "other" ) ) );
|
||||
ASSERT_TRUE( first.hasValue() );
|
||||
ASSERT_TRUE( second.hasValue() );
|
||||
auto a = std::move( first ).value();
|
||||
auto b = std::move( second ).value();
|
||||
for ( int i = 0; i < 9; ++i ) {
|
||||
const auto other = temp.path + String::toString( i ) + ".zip";
|
||||
ASSERT_TRUE( makeZip( other, { { "x", "x" } } ) );
|
||||
ASSERT_TRUE(
|
||||
waitFor( vfs.stat( ZipProvider::makeURI( URI::fromFSPath( other ) ) ) ).hasValue() );
|
||||
}
|
||||
vfs.unregisterProvider( "zip" );
|
||||
std::array<Uint8, 8> left;
|
||||
std::array<Uint8, 6> right;
|
||||
auto readingA = a.read( 0, left );
|
||||
auto readingB = b.read( 0, right );
|
||||
ASSERT_TRUE( waitFor( std::move( readingA ) ).hasValue() );
|
||||
ASSERT_TRUE( waitFor( std::move( readingB ) ).hasValue() );
|
||||
EXPECT_TRUE( std::string( left.begin(), left.end() ) == "original" );
|
||||
EXPECT_TRUE( std::string( right.begin(), right.end() ) == "second" );
|
||||
EXPECT_TRUE( waitFor( a.close() ).hasValue() );
|
||||
EXPECT_TRUE( waitFor( b.close() ).hasValue() );
|
||||
}
|
||||
|
||||
UTEST( ZipProvider, emptyArchivesAndUnicodeBackingFiles ) {
|
||||
TempDir temp;
|
||||
const std::string empty = std::string( "PK\x05\x06", 4 ) + std::string( 18, '\0' );
|
||||
const auto path = temp.path + "empty.zip";
|
||||
ASSERT_TRUE( FileSystem::fileWrite( path, empty ) );
|
||||
VirtualFileSystem vfs;
|
||||
auto listing = waitFor( vfs.listAll( ZipProvider::makeURI( URI::fromFSPath( path ) ) ) );
|
||||
ASSERT_TRUE( listing.hasValue() );
|
||||
EXPECT_TRUE( listing.value().empty() );
|
||||
const auto unicode = temp.path + "archivo-ñ-雪.zip";
|
||||
// Build at an ASCII path with the legacy writer, then rename through the portable VFS.
|
||||
const auto source = temp.path + "source.zip";
|
||||
ASSERT_TRUE( makeZip(
|
||||
source, { { "file", "unicode archive" }, { "directorio-ñ/雪.txt", "unicode member" } } ) );
|
||||
ASSERT_TRUE(
|
||||
waitFor( vfs.rename( URI::fromFSPath( source ), URI::fromFSPath( unicode ) ) ).hasValue() );
|
||||
EXPECT_STDSTREQ( "unicode archive",
|
||||
readString( vfs, ZipProvider::makeURI( URI::fromFSPath( unicode ) )
|
||||
.withAppendedSegment( "file" ) ) );
|
||||
const URI member = ZipProvider::makeURI( URI::fromFSPath( unicode ), "/directorio-ñ/雪.txt" );
|
||||
EXPECT_STDSTREQ( "unicode member", readString( vfs, URI( member.toString() ) ) );
|
||||
}
|
||||
|
||||
UTEST( ZipProvider, pollingWatchNoticesBackingArchiveReplacement ) {
|
||||
TempDir temp;
|
||||
const auto path = temp.path + "watched.zip";
|
||||
ASSERT_TRUE( makeZip( path, { { "before", "old" } } ) );
|
||||
VirtualFileSystem vfs;
|
||||
const URI root = ZipProvider::makeURI( URI::fromFSPath( path ) );
|
||||
WatchOptions options;
|
||||
options.pollingInterval = Milliseconds( 50 );
|
||||
auto opened = waitFor( vfs.watch( root, options ) );
|
||||
ASSERT_TRUE( opened.hasValue() );
|
||||
WatchHandle watch( std::move( opened ).value() );
|
||||
EXPECT_EQ( WatchStatus::Polling, watch.status() );
|
||||
TestLoop loop;
|
||||
EventCollector collector;
|
||||
watch.onEvents( loop, collector.callback() );
|
||||
ASSERT_TRUE( FileSystem::fileRemove( path ) );
|
||||
ASSERT_TRUE( makeZip( path, { { "after", "new contents" } } ) );
|
||||
EXPECT_TRUE( loop.runUntil( [&] { return collector.has( WatchEventType::Added, "after" ); } ) );
|
||||
EXPECT_TRUE( collector.has( WatchEventType::Removed, "before" ) );
|
||||
EXPECT_STDSTREQ( "new contents", readString( vfs, root.withAppendedSegment( "after" ) ) );
|
||||
}
|
||||
Vendored
+2
@@ -257,6 +257,8 @@ _zip_readcdir(FILE *fp, off_t buf_offset, unsigned char *buf, unsigned char *eoc
|
||||
|
||||
left = cd->size;
|
||||
i=0;
|
||||
if (cd->nentry == 0 && left == 0)
|
||||
return cd;
|
||||
do {
|
||||
if ((_zip_dirent_read(cd->entry+i, fp, bufp, &left, 0, error)) < 0) {
|
||||
cd->nentry = i;
|
||||
|
||||
@@ -8,6 +8,7 @@
|
||||
#include "placescontroller.hpp"
|
||||
#include "trash.hpp"
|
||||
#include <eepp/system/sha1.hpp>
|
||||
#include <eepp/vfs/providers/zipprovider.hpp>
|
||||
|
||||
namespace eefm {
|
||||
|
||||
@@ -70,6 +71,12 @@ void FileOperations::openRow( FileTab* tab, size_t row ) {
|
||||
mApp.panes().navigate( tab, tab->model->entryURI( row ) );
|
||||
return;
|
||||
}
|
||||
const URI entry = tab->model->entryURI( row );
|
||||
if ( entry.getScheme() == "file" &&
|
||||
String::toLower( FileSystem::fileExtension( tab->model->entry( row ).name ) ) == "zip" ) {
|
||||
mApp.panes().navigate( tab, ZipProvider::makeURI( entry ) );
|
||||
return;
|
||||
}
|
||||
// Enter / double click on a multiple selection opens all the selected files.
|
||||
auto rows = tab->selectedRows();
|
||||
if ( std::find( rows.begin(), rows.end(), row ) == rows.end() )
|
||||
|
||||
@@ -1,14 +1,16 @@
|
||||
#include "location.hpp"
|
||||
#include <eepp/system/filesystem.hpp>
|
||||
#include <eepp/vfs/providers/zipprovider.hpp>
|
||||
|
||||
using namespace EE;
|
||||
using namespace EE::System;
|
||||
using namespace EE::VFS;
|
||||
|
||||
namespace eefm {
|
||||
|
||||
URI Location::parse( const std::string& text ) {
|
||||
std::string trimmed = String::trim( text );
|
||||
if ( trimmed.find( "://" ) != std::string::npos ) {
|
||||
if ( trimmed.find( "://" ) != std::string::npos || trimmed.starts_with( "zip:" ) ) {
|
||||
URI uri( trimmed );
|
||||
if ( uri.getPath().empty() )
|
||||
uri.setRawPath( "/" );
|
||||
@@ -17,14 +19,46 @@ URI Location::parse( const std::string& text ) {
|
||||
return URI::fromFSPath( FileSystem::expandTilde( trimmed ) );
|
||||
}
|
||||
|
||||
URI Location::parent( const URI& uri ) {
|
||||
if ( uri.getScheme() == "zip" ) {
|
||||
auto archive = ZipProvider::archiveURI( uri );
|
||||
auto member = ZipProvider::memberPath( uri );
|
||||
if ( archive && member ) {
|
||||
if ( member.value() == "/" )
|
||||
return archive.value().getParent();
|
||||
URI entry;
|
||||
entry.setRawPath( std::move( member ).value() );
|
||||
return ZipProvider::makeURI( archive.value(), entry.getParent().getPath() );
|
||||
}
|
||||
}
|
||||
return uri.getParent();
|
||||
}
|
||||
|
||||
std::string Location::entryName( const URI& uri ) {
|
||||
if ( uri.getScheme() == "zip" ) {
|
||||
auto archive = ZipProvider::archiveURI( uri );
|
||||
auto member = ZipProvider::memberPath( uri );
|
||||
if ( archive && member && member.value() == "/" )
|
||||
return archive.value().getLastPathSegment();
|
||||
}
|
||||
return uri.getLastPathSegment();
|
||||
}
|
||||
|
||||
String Location::displayName( const URI& uri ) {
|
||||
std::string name = uri.getLastPathSegment();
|
||||
std::string name = entryName( uri );
|
||||
if ( uri.getScheme() == "zip" && !name.empty() )
|
||||
return String::fromUtf8( name );
|
||||
if ( name.empty() || uri.getPath() == "/" )
|
||||
name = uri.getScheme() == "file" ? std::string( "/" ) : uri.getHost();
|
||||
return String::fromUtf8( name );
|
||||
}
|
||||
|
||||
String Location::text( const URI& uri ) {
|
||||
if ( uri.getScheme() == "zip" ) {
|
||||
// URI::toString() omits an empty authority for non-file schemes. Spell it explicitly
|
||||
// in the location bar; both spellings parse into the same provider URI.
|
||||
return String::fromUtf8( uri.getSchemeAndAuthority() + uri.getPathEtc() );
|
||||
}
|
||||
if ( uri.getScheme() == "file" ) {
|
||||
std::string path = uri.getFSPath();
|
||||
FileSystem::dirAddSlashAtEnd( path );
|
||||
|
||||
@@ -7,21 +7,29 @@
|
||||
|
||||
namespace eefm {
|
||||
|
||||
using namespace EE;
|
||||
|
||||
/** How locations are typed, shown and copied. */
|
||||
class Location {
|
||||
public:
|
||||
/** A typed location: a URI, or a local path ("~" expands to the home folder). */
|
||||
static EE::URI parse( const std::string& text );
|
||||
static URI parse( const std::string& text );
|
||||
|
||||
/** Parent location; leaving an archive root returns to the archive's containing folder. */
|
||||
static URI parent( const URI& uri );
|
||||
|
||||
/** Entry to select when leaving this location (the archive's name at its root). */
|
||||
static std::string entryName( const URI& uri );
|
||||
|
||||
/** The name of a folder for titles: its last segment, "/" or the host for roots. */
|
||||
static EE::String displayName( const EE::URI& uri );
|
||||
static String displayName( const URI& uri );
|
||||
|
||||
/** The location bar text: a local path with a trailing slash, the URI otherwise. */
|
||||
static EE::String text( const EE::URI& uri );
|
||||
static String text( const URI& uri );
|
||||
|
||||
/** A location for Copy Location: a plain path (no trailing slash) for local files and
|
||||
* folders, the URI otherwise. Copying files (Ctrl+C) puts URIs in the clipboard instead. */
|
||||
static std::string clipboardText( const EE::URI& uri );
|
||||
static std::string clipboardText( const URI& uri );
|
||||
};
|
||||
|
||||
} // namespace eefm
|
||||
|
||||
@@ -568,7 +568,7 @@ void PaneManager::navigate( FileTab* tab, URI uri, bool recordHistory ) {
|
||||
void PaneManager::goBack() {
|
||||
auto* tab = currentTab();
|
||||
if ( tab && tab->historyIndex > 0 ) {
|
||||
tab->selectLater( tab->model->uri().getLastPathSegment() );
|
||||
tab->selectLater( Location::entryName( tab->model->uri() ) );
|
||||
--tab->historyIndex;
|
||||
navigate( tab, tab->history[tab->historyIndex], false );
|
||||
}
|
||||
@@ -586,10 +586,10 @@ void PaneManager::goUp() {
|
||||
auto* tab = currentTab();
|
||||
if ( !tab )
|
||||
return;
|
||||
URI parent = tab->model->uri().getParent();
|
||||
URI parent = Location::parent( tab->model->uri() );
|
||||
if ( parent != tab->model->uri() ) {
|
||||
// Like Dolphin, the folder we leave stays selected in its parent.
|
||||
tab->selectLater( tab->model->uri().getLastPathSegment() );
|
||||
tab->selectLater( Location::entryName( tab->model->uri() ) );
|
||||
navigate( tab, parent );
|
||||
}
|
||||
}
|
||||
@@ -649,7 +649,7 @@ void PaneManager::updateChrome( Pane& pane ) {
|
||||
return;
|
||||
mUI->find<UIWidget>( "back" )->setEnabled( tab->historyIndex > 0 );
|
||||
mUI->find<UIWidget>( "forward" )->setEnabled( tab->historyIndex + 1 < tab->history.size() );
|
||||
mUI->find<UIWidget>( "up" )->setEnabled( uri.getParent() != uri );
|
||||
mUI->find<UIWidget>( "up" )->setEnabled( Location::parent( uri ) != uri );
|
||||
mApp.window()->setTitle( Location::text( uri ) + " — eefm" );
|
||||
if ( auto* panel = mApp.places().panel() )
|
||||
panel->setCurrent( uri );
|
||||
|
||||
Reference in New Issue
Block a user