mirror of
https://github.com/SpartanJ/eepp.git
synced 2026-10-27 08:37:58 +03:00
feat(vfs): add runtime libarchive support and eefm extraction
- Add a read-only ArchiveProvider with optional runtime-loaded libarchive 3.x - Stream archive entries with bounded indexes and backing-file revision checks - Expose available formats and filters without a build-time dependency - Extend eefm archive browsing and add extraction context-menu actions - Merge extraction directories while requiring confirmation before overwriting files - Add archive, extraction, naming and transfer-conflict regression tests
This commit is contained in:
@@ -9,6 +9,7 @@ for local files, packed resources and remote servers:
|
||||
| --- | --- | --- |
|
||||
| `file://` | `LocalProvider` | Native filesystem (POSIX and Win32). |
|
||||
| `zip://` | `ZipProvider` | Read-only browsing and streaming reads of individual local ZIP archives. |
|
||||
| `archive://` | `ArchiveProvider` | Read-only local archives through optional runtime-loaded libarchive 3.x. |
|
||||
| `res://` | `ResourceProvider` | Read-only view of every open `Pack` (`System::PackFileIndex`). |
|
||||
| `sftp://` | `SFTPProvider` | Built by default (`--without-vfs-sftp` disables it), libssh2 based, honors `~/.ssh/config`. |
|
||||
| `mem://` | `MemoryProvider` | In-memory filesystem, mainly for tests. |
|
||||
@@ -171,6 +172,11 @@ when a destination appears while the copy is running. The `Overwrite` policy req
|
||||
Creating a file with `Fail` or `Skip` requires `Rename`. Moving a symlink through the copy fallback
|
||||
is unsupported; native same-filesystem renames still move the link itself.
|
||||
|
||||
`MergeDirectories` merges existing destination directories but reports `AlreadyExists` for files,
|
||||
including files that appear during the copy. It supports copying an archive root into an existing
|
||||
folder without overwriting files. The default `Fail` policy also rejects existing directories;
|
||||
`Overwrite` and `Skip` merge directories and apply their respective policies to files.
|
||||
|
||||
Native renames also preserve an existing destination when safe replacement is unavailable.
|
||||
POSIX systems without an atomic no-replace primitive return `Unsupported` for no-overwrite
|
||||
renames, rather than using an existence check followed by a replacing rename.
|
||||
@@ -344,6 +350,77 @@ callers sharing an archive across threads must serialize access. `open()` never
|
||||
file; writing starts explicitly with `create()`. The default constructor retains resource-pack
|
||||
registration for the engine and other existing pack consumers.
|
||||
|
||||
## Runtime archives (`archive://`)
|
||||
|
||||
`ArchiveProvider` is registered by default but loads libarchive only when used, on an I/O worker.
|
||||
It requires a compatible libarchive **3.x** shared library, with no libarchive development headers
|
||||
or build-time link dependency. Missing libraries, unsupported ABI versions and missing required
|
||||
symbols return `Unsupported`; the rest of the VFS, including bundled ZIP support, remains available.
|
||||
Emscripten and iOS have no runtime backend.
|
||||
|
||||
```cpp
|
||||
URI backing = URI::fromFSPath( "/home/me/downloads/source.tar.gz" );
|
||||
URI root = ArchiveProvider::makeURI( backing );
|
||||
vfs.listAll( root );
|
||||
vfs.copy( root.withAppendedSegment( "source" ), URI::fromFSPath( "/home/me/source" ) );
|
||||
|
||||
auto provider = std::dynamic_pointer_cast<ArchiveProvider>( vfs.provider( "archive" ) );
|
||||
provider->support().then( ui, []( VFSResult<ArchiveSupport>&& support ) {
|
||||
// version, formats and compression filters from the installed runtime library.
|
||||
} );
|
||||
```
|
||||
|
||||
Locations use `archive:///path/source.tar.gz!/member`, with the same bang escaping, member path
|
||||
rules and archive-root navigation as `zip://`. Only local `file://` backing archives are accepted.
|
||||
An optional constructor `libraryPath` selects an exact library; otherwise the provider tries the
|
||||
platform's usual libarchive names (including `/usr/lib/libarchive.2.dylib` on macOS).
|
||||
|
||||
Supported readers are enabled individually: TAR (including GNU and PAX variants), CPIO, ar, 7z,
|
||||
RAR/RAR5, ISO9660, CAB, LHA, XAR and ZIP, when supplied by the loaded library. Compression filters
|
||||
include gzip, bzip2, xz, LZMA, lzip, zstd and lz4 when built into that library. `support()` reports
|
||||
the enabled set. External decompression programs are never enabled. Recognition does not guarantee
|
||||
every entry can be decoded: encrypted or unsupported entry data and detected corruption report
|
||||
errors when read. Integrity checks follow the installed libarchive; there is no extra checksum pass.
|
||||
In particular, libarchive 3.8.2's gzip reader does not verify the gzip trailer CRC.
|
||||
|
||||
`eefm` recognizes `.tar`, `.tar.gz`, `.tgz`, other common compressed TAR suffixes, `.7z`, `.rar`,
|
||||
`.iso`, `.cab`, `.cpio`, `.ar`, `.deb`, `.xar` and `.lha`/`.lzh` as browsing candidates. Extensions
|
||||
are case-insensitive; the archive contents determine whether opening succeeds. A `.deb` opens its
|
||||
outer ar container; nested archives currently need to be copied out before browsing. ZIP-based
|
||||
formats continue to use the bundled `ZipProvider`.
|
||||
|
||||
Right-click a single recognized local archive in `eefm` for the **Extract** submenu. The first
|
||||
action extracts into a folder named after the archive (for example `package.tar.gz` into
|
||||
`package/`); **Extract Here** uses its containing folder. **Extract To...** accepts a destination
|
||||
folder path or URI, including writable SFTP locations. Extraction uses VFS transfers with progress,
|
||||
cancellation and confirmation before overwriting existing files. Hidden entries are included.
|
||||
It follows VFS copy semantics: links to files are copied as contents, while directory symlinks and
|
||||
unsupported special entries are skipped and reported as a partial transfer. Links are not recreated.
|
||||
|
||||
The provider is read-only; moves out fail before copying because source removal is unavailable.
|
||||
Indexing scans headers and skips entry data without extracting it into memory. Compressed archives
|
||||
may require decompression during indexing. Each opened file has an independent streaming reader;
|
||||
opening a late entry or seeking backwards can replay the archive prefix, so repeated random reads
|
||||
can be expensive. For plain and gzip TAR, an explicit file close returns one idle reader to its
|
||||
index: opening later entries can continue forward, avoiding repeated prefix decompression during
|
||||
sequential directory copies. Other codecs and formats are released on close, so large decoder
|
||||
windows or solid blocks are not retained by the index. `Seek` is supported, `RandomAccess` is not.
|
||||
Idle readers close their native files; reuse reopens at the saved input position and checks the
|
||||
backing revision again. Cached decoder state therefore does not block deleting or replacing the
|
||||
archive on Windows. Data is not cached as extracted files.
|
||||
|
||||
Symbolic and hard links resolve only inside the archive, with a 40-link limit; native filesystem
|
||||
targets are never followed. Device nodes and FIFOs can be listed but cannot be opened as files.
|
||||
Unsafe member names are excluded using the same checks as ZIP. Indexes have a 262,144-entry limit
|
||||
and a conservative 64 MiB metadata budget, including implicit directories and stored paths, with
|
||||
at most eight cached indexes. These limits cover the provider's index, not libarchive's own decoder
|
||||
working memory. Open handles can retain older indexes.
|
||||
|
||||
Backing-file revisions invalidate cached indexes for new operations. Existing readers retain their
|
||||
opened file and library; reopening for a backward seek after replacement reports `ChangedExternally`.
|
||||
In-place modifications may cause read errors. Network mounts use the local scheduler's isolated
|
||||
lanes and buffer-safe timeouts, and archive changes can be watched through the VFS polling backend.
|
||||
|
||||
## Resources (`res://`)
|
||||
|
||||
`System::PackFileIndex` (formerly `System::VirtualFileSystem`) indexes the files of every open `Pack`. `res:///fonts/a.ttf` maps to the pack path
|
||||
|
||||
@@ -8,6 +8,7 @@
|
||||
#include <eepp/vfs/fileinfo.hpp>
|
||||
#include <eepp/vfs/options.hpp>
|
||||
#include <eepp/vfs/provider.hpp>
|
||||
#include <eepp/vfs/providers/archiveprovider.hpp>
|
||||
#include <eepp/vfs/providers/localprovider.hpp>
|
||||
#include <eepp/vfs/providers/memoryprovider.hpp>
|
||||
#include <eepp/vfs/providers/resourceprovider.hpp>
|
||||
|
||||
@@ -100,6 +100,8 @@ enum class OverwritePolicy : Uint8 {
|
||||
Overwrite,
|
||||
/** Leave existing destination files untouched and continue. */
|
||||
Skip,
|
||||
/** Merge existing directories, but fail with AlreadyExists for existing files. */
|
||||
MergeDirectories,
|
||||
};
|
||||
|
||||
struct EE_VFS_API CopyOptions {
|
||||
|
||||
@@ -0,0 +1,83 @@
|
||||
#ifndef EE_VFS_PROVIDERS_ARCHIVEPROVIDER_HPP
|
||||
#define EE_VFS_PROVIDERS_ARCHIVEPROVIDER_HPP
|
||||
|
||||
#include <eepp/vfs/config.hpp>
|
||||
#include <eepp/vfs/provider.hpp>
|
||||
#include <eepp/vfs/providers/localioscheduler.hpp>
|
||||
#include <memory>
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
namespace EE { namespace VFS {
|
||||
|
||||
class ArchiveProviderImpl;
|
||||
|
||||
/** Runtime capabilities of the installed libarchive 3.x library. Only built-in decoders are
|
||||
* enabled: the provider never delegates decompression to external commands. */
|
||||
struct EE_VFS_API ArchiveSupport {
|
||||
std::string version;
|
||||
std::vector<std::string> formats;
|
||||
std::vector<std::string> filters;
|
||||
};
|
||||
|
||||
/** Read-only archive:// filesystem backed by optional runtime-loaded libarchive 3.x.
|
||||
* No libarchive headers or link dependency are required. Locations have the form
|
||||
* archive:///path/archive.tar.gz!/member (use makeURI()). Local archives only.
|
||||
* Indexing and reads run through LocalIOScheduler, including mount isolation and safe timeouts.
|
||||
* Entry data streams with bounded buffers. Backward seeks reopen and replay the archive;
|
||||
* RandomAccess is not advertised. Readers retain the runtime library until they are closed.
|
||||
* Missing/incompatible libraries return Unsupported. Emscripten has no runtime backend. */
|
||||
class EE_VFS_API ArchiveProvider : public VFSProvider {
|
||||
public:
|
||||
/** libraryPath optionally selects an exact runtime library; empty uses platform defaults.
|
||||
* Loading happens lazily on an I/O thread, never in the constructor. */
|
||||
explicit ArchiveProvider( std::shared_ptr<LocalIOScheduler> scheduler,
|
||||
std::string libraryPath = {} );
|
||||
|
||||
~ArchiveProvider() override;
|
||||
|
||||
/** Builds an archive location. member is an unescaped, '/'-separated path inside it. Only
|
||||
* file:// backing archives are currently supported; other inputs return an empty URI.
|
||||
* Root is "/". */
|
||||
static URI makeURI( const URI& archive, std::string_view member = "/" );
|
||||
|
||||
/** Decodes and validates the backing file:// URI of an archive:// location. */
|
||||
static VFSResult<URI> archiveURI( const URI& uri );
|
||||
|
||||
/** Validates an archive location and returns its normalized path within the archive. */
|
||||
static VFSResult<std::string> memberPath( const URI& uri );
|
||||
|
||||
std::string_view scheme() const override { return "archive"; }
|
||||
|
||||
/** Reports the actual installed library's capabilities, or Unsupported when unavailable. */
|
||||
VFSTask<ArchiveSupport> support( CancellationToken token = {} );
|
||||
|
||||
ProviderCapabilities capabilities( const URI& uri ) const override;
|
||||
|
||||
VFSTask<VFSFileInfo> stat( const URI& uri, const StatOptions& options,
|
||||
CancellationToken token ) override;
|
||||
|
||||
VFSTask<DirectoryHandle> openDirectory( const URI& uri, const DirectoryOpenOptions& options,
|
||||
CancellationToken token ) override;
|
||||
|
||||
VFSTask<FileHandle> open( const URI& uri, const OpenOptions& options,
|
||||
CancellationToken token ) override;
|
||||
|
||||
VFSTask<void> mkdir( const URI& uri, const MkDirOptions& options,
|
||||
CancellationToken token ) override;
|
||||
|
||||
VFSTask<void> remove( const URI& uri, const RemoveOptions& options,
|
||||
CancellationToken token ) override;
|
||||
|
||||
VFSTask<void> rename( const URI& from, const URI& to, const RenameOptions& options,
|
||||
CancellationToken token ) override;
|
||||
|
||||
VFSTask<URI> canonicalize( const URI& uri, CancellationToken token ) override;
|
||||
|
||||
private:
|
||||
std::shared_ptr<ArchiveProviderImpl> mImpl;
|
||||
};
|
||||
|
||||
}} // namespace EE::VFS
|
||||
|
||||
#endif
|
||||
@@ -46,7 +46,8 @@ class EE_VFS_API VirtualFileSystem {
|
||||
* counts as not responding: the call fails with TimedOut, and so do further calls until
|
||||
* the server answers again (see LocalIOScheduler). */
|
||||
System::Time networkMountTimeout{ System::Seconds( 15 ) };
|
||||
/** Register file://, zip:// and res:// (and sftp:// when built with SFTP support). */
|
||||
/** Register file://, zip://, archive:// and res:// (and sftp:// when built with SFTP
|
||||
* support). */
|
||||
bool registerDefaultProviders{ true };
|
||||
};
|
||||
|
||||
|
||||
@@ -0,0 +1,111 @@
|
||||
#ifndef _FILE_OFFSET_BITS
|
||||
#define _FILE_OFFSET_BITS 64
|
||||
#endif
|
||||
#include "archiveinput.hpp"
|
||||
#include <eepp/core/string.hpp>
|
||||
#include <eepp/system/filesystem.hpp>
|
||||
#include <limits>
|
||||
#if EE_PLATFORM == EE_PLATFORM_WIN
|
||||
#ifndef NOMINMAX
|
||||
#define NOMINMAX
|
||||
#endif
|
||||
#include <fcntl.h>
|
||||
#include <io.h>
|
||||
#include <windows.h>
|
||||
#endif
|
||||
|
||||
namespace EE { namespace VFS { namespace detail {
|
||||
|
||||
ArchiveInput::ArchiveInput( const std::string& path ) : mPath( path ) {
|
||||
if ( !openFile() )
|
||||
return;
|
||||
#if EE_PLATFORM == EE_PLATFORM_WIN
|
||||
int result = _fseeki64( mFile, 0, SEEK_END );
|
||||
#else
|
||||
int result = ::fseeko( mFile, 0, SEEK_END );
|
||||
#endif
|
||||
mSize = result == 0 ? tell() : -1;
|
||||
if ( mSize < 0 || seek( 0 ) < 0 ) {
|
||||
std::fclose( mFile );
|
||||
mFile = nullptr;
|
||||
}
|
||||
}
|
||||
|
||||
bool ArchiveInput::openFile() {
|
||||
#if EE_PLATFORM == EE_PLATFORM_WIN
|
||||
HANDLE handle =
|
||||
CreateFileW( String::fromUtf8( mPath ).toWideString().c_str(), GENERIC_READ,
|
||||
FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE, nullptr, OPEN_EXISTING,
|
||||
FILE_ATTRIBUTE_NORMAL | FILE_FLAG_SEQUENTIAL_SCAN, nullptr );
|
||||
if ( handle == INVALID_HANDLE_VALUE )
|
||||
return false;
|
||||
int fd = _open_osfhandle( reinterpret_cast<std::intptr_t>( handle ), _O_RDONLY | _O_BINARY );
|
||||
if ( fd < 0 ) {
|
||||
CloseHandle( handle );
|
||||
return false;
|
||||
}
|
||||
mFile = _fdopen( fd, "rb" );
|
||||
if ( !mFile )
|
||||
_close( fd );
|
||||
#else
|
||||
mFile = System::FileSystem::fopenUtf8( mPath, "rb" );
|
||||
#endif
|
||||
return mFile != nullptr;
|
||||
}
|
||||
|
||||
bool ArchiveInput::suspend() {
|
||||
if ( !mFile )
|
||||
return false;
|
||||
mPosition = tell();
|
||||
int result = std::fclose( mFile );
|
||||
mFile = nullptr;
|
||||
return result == 0 && mPosition >= 0;
|
||||
}
|
||||
|
||||
bool ArchiveInput::resume() {
|
||||
if ( mFile )
|
||||
return true;
|
||||
if ( !openFile() )
|
||||
return false;
|
||||
if ( seek( mPosition ) < 0 ) {
|
||||
std::fclose( mFile );
|
||||
mFile = nullptr;
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
ArchiveInput::~ArchiveInput() {
|
||||
if ( mFile )
|
||||
std::fclose( mFile );
|
||||
}
|
||||
|
||||
std::intptr_t ArchiveInput::read( char* data, size_t size ) {
|
||||
const size_t got = std::fread( data, 1, size, mFile );
|
||||
return got == 0 && std::ferror( mFile ) ? -1 : static_cast<std::intptr_t>( got );
|
||||
}
|
||||
|
||||
Int64 ArchiveInput::seek( Int64 position ) {
|
||||
if ( position < 0 || !mFile )
|
||||
return -1;
|
||||
#if EE_PLATFORM == EE_PLATFORM_WIN
|
||||
return _fseeki64( mFile, position, SEEK_SET ) == 0 ? position : -1;
|
||||
#else
|
||||
if ( static_cast<Uint64>( position ) >
|
||||
static_cast<Uint64>( std::numeric_limits<off_t>::max() ) )
|
||||
return -1;
|
||||
return ::fseeko( mFile, static_cast<off_t>( position ), SEEK_SET ) == 0 ? position : -1;
|
||||
#endif
|
||||
}
|
||||
|
||||
Int64 ArchiveInput::tell() const {
|
||||
if ( !mFile )
|
||||
return -1;
|
||||
#if EE_PLATFORM == EE_PLATFORM_WIN
|
||||
return _ftelli64( mFile );
|
||||
#else
|
||||
return ::ftello( mFile );
|
||||
#endif
|
||||
}
|
||||
|
||||
}}} // namespace EE::VFS::detail
|
||||
@@ -0,0 +1,48 @@
|
||||
#ifndef EE_VFS_ARCHIVE_ARCHIVEINPUT_HPP
|
||||
#define EE_VFS_ARCHIVE_ARCHIVEINPUT_HPP
|
||||
|
||||
#include <cstdint>
|
||||
#include <cstdio>
|
||||
#include <eepp/vfs/config.hpp>
|
||||
#include <string>
|
||||
|
||||
namespace EE { namespace VFS { namespace detail {
|
||||
|
||||
/** Archive input needs checked 64-bit seeks and, on Windows, sharing that allows replacement
|
||||
* while an active or idle reader keeps the old file open. It never exposes a CRT file to
|
||||
* libarchive. */
|
||||
class EE_VFS_API ArchiveInput {
|
||||
public:
|
||||
explicit ArchiveInput( const std::string& path );
|
||||
|
||||
~ArchiveInput();
|
||||
|
||||
ArchiveInput( const ArchiveInput& ) = delete;
|
||||
|
||||
ArchiveInput& operator=( const ArchiveInput& ) = delete;
|
||||
|
||||
bool isOpen() const { return mFile != nullptr; }
|
||||
|
||||
std::intptr_t read( char* data, size_t size );
|
||||
|
||||
Int64 seek( Int64 position );
|
||||
|
||||
Int64 tell() const;
|
||||
|
||||
Int64 size() const { return mSize; }
|
||||
|
||||
bool suspend();
|
||||
|
||||
bool resume();
|
||||
|
||||
private:
|
||||
bool openFile();
|
||||
|
||||
std::string mPath;
|
||||
std::FILE* mFile = nullptr;
|
||||
Int64 mSize = 0;
|
||||
Int64 mPosition = 0;
|
||||
};
|
||||
|
||||
}}} // namespace EE::VFS::detail
|
||||
#endif
|
||||
@@ -0,0 +1,293 @@
|
||||
#include "libarchive.hpp"
|
||||
#include <algorithm>
|
||||
#include <cerrno>
|
||||
#include <eepp/system/sys.hpp>
|
||||
#include <limits>
|
||||
|
||||
namespace EE { namespace VFS { namespace detail {
|
||||
|
||||
namespace {
|
||||
constexpr std::array<const char*, 11> Formats = { "tar", "cpio", "ar", "7zip", "rar", "rar5",
|
||||
"iso9660", "cab", "lha", "xar", "zip" };
|
||||
constexpr std::array<const char*, 11> Filters = { "none", "gzip", "bzip2", "xz", "lzma", "lzip",
|
||||
"zstd", "lz4", "compress", "rpm", "uu" };
|
||||
} // namespace
|
||||
|
||||
VFSResult<std::shared_ptr<LibArchive>> LibArchive::load( const std::string& path, const URI& uri ) {
|
||||
#if EE_PLATFORM == EE_PLATFORM_EMSCRIPTEN || EE_PLATFORM == EE_PLATFORM_IOS
|
||||
(void)path;
|
||||
#else
|
||||
auto library = std::make_shared<LibArchive>();
|
||||
if ( !path.empty() ) {
|
||||
library->mLibrary = System::Sys::loadObject( path );
|
||||
} else {
|
||||
#if EE_PLATFORM == EE_PLATFORM_WIN
|
||||
const char* names[] = { "archive.dll", "libarchive-13.dll", "libarchive.dll" };
|
||||
#elif EE_PLATFORM == EE_PLATFORM_MACOS
|
||||
const char* names[] = { "/usr/lib/libarchive.2.dylib", "libarchive.13.dylib",
|
||||
"libarchive.dylib" };
|
||||
#else
|
||||
const char* names[] = { "libarchive.so.13", "libarchive.so" };
|
||||
#endif
|
||||
for ( const char* name : names ) {
|
||||
library->mLibrary = System::Sys::loadObject( name );
|
||||
if ( library->mLibrary )
|
||||
break;
|
||||
}
|
||||
}
|
||||
if ( library->mLibrary && library->bind() )
|
||||
return Ok( std::move( library ) );
|
||||
#endif
|
||||
return Err( VFSError( VFSErrorCode::Unsupported, uri,
|
||||
"A compatible libarchive 3.x runtime library is not available" ) );
|
||||
}
|
||||
|
||||
LibArchive::~LibArchive() {
|
||||
#if EE_PLATFORM != EE_PLATFORM_EMSCRIPTEN && EE_PLATFORM != EE_PLATFORM_IOS
|
||||
if ( mLibrary )
|
||||
System::Sys::unloadObject( mLibrary );
|
||||
#endif
|
||||
}
|
||||
|
||||
bool LibArchive::bind() {
|
||||
#if EE_PLATFORM == EE_PLATFORM_EMSCRIPTEN || EE_PLATFORM == EE_PLATFORM_IOS
|
||||
return false;
|
||||
#else
|
||||
auto bind = [this]( auto& fn, const std::string& name ) {
|
||||
fn = reinterpret_cast<std::decay_t<decltype( fn )>>(
|
||||
System::Sys::loadFunction( mLibrary, name ) );
|
||||
return fn != nullptr;
|
||||
};
|
||||
if ( !bind( mVersion, "archive_version_number" ) || mVersion() < 3000000 ||
|
||||
mVersion() >= 4000000 )
|
||||
return false;
|
||||
#define EE_ARCHIVE_BIND( member, symbol ) \
|
||||
if ( !bind( member, symbol ) ) \
|
||||
return false
|
||||
EE_ARCHIVE_BIND( mVersionString, "archive_version_string" );
|
||||
EE_ARCHIVE_BIND( readNew, "archive_read_new" );
|
||||
EE_ARCHIVE_BIND( readFree, "archive_read_free" );
|
||||
EE_ARCHIVE_BIND( readOpen, "archive_read_open2" );
|
||||
EE_ARCHIVE_BIND( setSeek, "archive_read_set_seek_callback" );
|
||||
EE_ARCHIVE_BIND( nextHeader, "archive_read_next_header" );
|
||||
EE_ARCHIVE_BIND( readData, "archive_read_data" );
|
||||
EE_ARCHIVE_BIND( errorString, "archive_error_string" );
|
||||
EE_ARCHIVE_BIND( errorNumber, "archive_errno" );
|
||||
EE_ARCHIVE_BIND( formatCode, "archive_format" );
|
||||
EE_ARCHIVE_BIND( filterCount, "archive_filter_count" );
|
||||
EE_ARCHIVE_BIND( filterCode, "archive_filter_code" );
|
||||
EE_ARCHIVE_BIND( entryPath, "archive_entry_pathname_utf8" );
|
||||
EE_ARCHIVE_BIND( entryLink, "archive_entry_symlink_utf8" );
|
||||
EE_ARCHIVE_BIND( entryHardLink, "archive_entry_hardlink_utf8" );
|
||||
EE_ARCHIVE_BIND( entrySize, "archive_entry_size" );
|
||||
EE_ARCHIVE_BIND( entrySizeSet, "archive_entry_size_is_set" );
|
||||
EE_ARCHIVE_BIND( entryType, "archive_entry_filetype" );
|
||||
EE_ARCHIVE_BIND( entryPermissions, "archive_entry_perm" );
|
||||
EE_ARCHIVE_BIND( entryTime, "archive_entry_mtime" );
|
||||
EE_ARCHIVE_BIND( entryTimeSet, "archive_entry_mtime_is_set" );
|
||||
#undef EE_ARCHIVE_BIND
|
||||
// Probe each decoder on its own disposable reader. A WARN can register an external-command
|
||||
// fallback, so that reader must never be used to open untrusted input.
|
||||
auto probe = [&]( auto& functions, const auto& names, const char* prefix ) {
|
||||
for ( size_t i = 0; i < names.size(); ++i ) {
|
||||
Enable enable = nullptr;
|
||||
if ( !bind( enable, std::string( prefix ) + names[i] ) )
|
||||
continue;
|
||||
archive* reader = readNew();
|
||||
if ( !reader )
|
||||
continue;
|
||||
if ( enable( reader ) == 0 )
|
||||
functions[i] = enable;
|
||||
readFree( reader );
|
||||
}
|
||||
};
|
||||
probe( mFormats, Formats, "archive_read_support_format_" );
|
||||
probe( mFilters, Filters, "archive_read_support_filter_" );
|
||||
return true;
|
||||
#endif
|
||||
}
|
||||
|
||||
ArchiveSupport LibArchive::support() const {
|
||||
ArchiveSupport support;
|
||||
support.version = mVersionString();
|
||||
for ( size_t i = 0; i < Formats.size(); ++i ) {
|
||||
if ( mFormats[i] )
|
||||
support.formats.emplace_back( Formats[i] );
|
||||
}
|
||||
for ( size_t i = 0; i < Filters.size(); ++i ) {
|
||||
if ( mFilters[i] )
|
||||
support.filters.emplace_back( Filters[i] );
|
||||
}
|
||||
return support;
|
||||
}
|
||||
|
||||
VFSResult<void> LibArchive::configure( archive* reader, const URI& uri ) const {
|
||||
if ( !reader )
|
||||
return Err( VFSError( VFSErrorCode::IO, uri, "Could not allocate archive reader" ) );
|
||||
for ( auto enable : mFormats ) {
|
||||
if ( enable && enable( reader ) != 0 )
|
||||
return Err( error( reader, uri ) );
|
||||
}
|
||||
for ( auto enable : mFilters ) {
|
||||
if ( enable && enable( reader ) != 0 )
|
||||
return Err( error( reader, uri ) );
|
||||
}
|
||||
return Ok();
|
||||
}
|
||||
|
||||
VFSError LibArchive::error( archive* reader, const URI& uri ) const {
|
||||
const char* message = errorString( reader );
|
||||
int number = errorNumber( reader );
|
||||
return VFSError( number == EACCES || number == EPERM ? VFSErrorCode::PermissionDenied
|
||||
: VFSErrorCode::IO,
|
||||
uri, message ? message : "Could not read archive", number );
|
||||
}
|
||||
|
||||
LibArchiveReader::LibArchiveReader( std::shared_ptr<LibArchive> library, const std::string& path,
|
||||
URI uri ) :
|
||||
mLibrary( std::move( library ) ), mFile( path ), mUri( std::move( uri ) ) {}
|
||||
|
||||
LibArchiveReader::~LibArchiveReader() {
|
||||
if ( mReader )
|
||||
mLibrary->readFree( mReader );
|
||||
}
|
||||
|
||||
VFSResult<void> LibArchiveReader::open( CancellationToken token ) {
|
||||
mToken = std::move( token );
|
||||
if ( mToken.isCancellationRequested() )
|
||||
return Err( cancelledError( mUri ) );
|
||||
if ( !mFile.isOpen() )
|
||||
return Err( VFSError( VFSErrorCode::IO, mUri, "Could not open archive backing file" ) );
|
||||
mReader = mLibrary->readNew();
|
||||
auto configured = mLibrary->configure( mReader, mUri );
|
||||
if ( !configured )
|
||||
return configured;
|
||||
if ( mLibrary->setSeek( mReader, &seekCallback ) != 0 ||
|
||||
mLibrary->readOpen( mReader, this, nullptr, &readCallback, &skipCallback, nullptr ) != 0 )
|
||||
return Err( error() );
|
||||
return Ok();
|
||||
}
|
||||
|
||||
VFSResult<bool> LibArchiveReader::next( CancellationToken token ) {
|
||||
mToken = std::move( token );
|
||||
if ( mToken.isCancellationRequested() )
|
||||
return Err( cancelledError( mUri ) );
|
||||
int status = mLibrary->nextHeader( mReader, &mEntry );
|
||||
if ( status == 1 )
|
||||
return Ok( false );
|
||||
// WARN can mean an entry was only partially decoded. Never silently publish that index.
|
||||
if ( status != 0 )
|
||||
return Err( error() );
|
||||
return Ok( true );
|
||||
}
|
||||
|
||||
VFSResult<size_t> LibArchiveReader::read( std::span<Uint8> data, CancellationToken token ) {
|
||||
mToken = std::move( token );
|
||||
if ( mToken.isCancellationRequested() )
|
||||
return Err( cancelledError( mUri ) );
|
||||
auto size = mLibrary->readData( mReader, data.data(), data.size() );
|
||||
if ( size < 0 )
|
||||
return Err( error() );
|
||||
return Ok( static_cast<size_t>( size ) );
|
||||
}
|
||||
|
||||
bool LibArchiveReader::canReuse() const {
|
||||
if ( ( mLibrary->formatCode( mReader ) & 0xff0000 ) != 0x30000 )
|
||||
return false;
|
||||
for ( int i = 0; i < mLibrary->filterCount( mReader ); ++i ) {
|
||||
const int code = mLibrary->filterCode( mReader, i );
|
||||
if ( code != 0 && code != 1 )
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
VFSResult<void> LibArchiveReader::resume( CancellationToken token ) {
|
||||
mToken = std::move( token );
|
||||
if ( mToken.isCancellationRequested() )
|
||||
return Err( cancelledError( mUri ) );
|
||||
if ( !mFile.resume() )
|
||||
return Err( VFSError( VFSErrorCode::IO, mUri, "Could not reopen archive input" ) );
|
||||
return Ok();
|
||||
}
|
||||
|
||||
VFSResult<VFSFileInfo> LibArchiveReader::info() const {
|
||||
VFSFileInfo info;
|
||||
// POSIX archive type values, independent of the host's stat constants.
|
||||
switch ( mLibrary->entryType( mEntry ) ) {
|
||||
case 0100000:
|
||||
info.type = FileType::Regular;
|
||||
break;
|
||||
case 0040000:
|
||||
info.type = FileType::Directory;
|
||||
break;
|
||||
case 0120000:
|
||||
info.type = FileType::Symlink;
|
||||
break;
|
||||
case 0010000:
|
||||
info.type = FileType::FIFO;
|
||||
break;
|
||||
case 0020000:
|
||||
info.type = FileType::CharacterDevice;
|
||||
break;
|
||||
case 0060000:
|
||||
info.type = FileType::BlockDevice;
|
||||
break;
|
||||
default:
|
||||
break;
|
||||
}
|
||||
if ( const char* link = mLibrary->entryLink( mEntry ) ) {
|
||||
if ( std::char_traits<char>::length( link ) > 4096 )
|
||||
return Err(
|
||||
VFSError( VFSErrorCode::InvalidOperation, mUri, "Archive link is too long" ) );
|
||||
info.symlinkTarget = link;
|
||||
}
|
||||
if ( mLibrary->entrySizeSet( mEntry ) && mLibrary->entrySize( mEntry ) >= 0 )
|
||||
info.setSize( static_cast<Uint64>( mLibrary->entrySize( mEntry ) ) );
|
||||
if ( mLibrary->entryTimeSet( mEntry ) )
|
||||
info.setModifiedTime( mLibrary->entryTime( mEntry ) );
|
||||
info.setPermissions( mLibrary->entryPermissions( mEntry ) & 07777 );
|
||||
return Ok( std::move( info ) );
|
||||
}
|
||||
|
||||
VFSError LibArchiveReader::error() const {
|
||||
return mToken.isCancellationRequested() ? cancelledError( mUri )
|
||||
: mLibrary->error( mReader, mUri );
|
||||
}
|
||||
|
||||
LibArchive::Count LibArchiveReader::readCallback( archive*, void* context, const void** buffer ) {
|
||||
auto& self = *static_cast<LibArchiveReader*>( context );
|
||||
if ( self.mToken.isCancellationRequested() )
|
||||
return -1;
|
||||
*buffer = self.mBuffer.data();
|
||||
return self.mFile.read( self.mBuffer.data(), self.mBuffer.size() );
|
||||
}
|
||||
|
||||
LibArchive::Offset LibArchiveReader::skipCallback( archive*, void* context,
|
||||
LibArchive::Offset amount ) {
|
||||
auto& self = *static_cast<LibArchiveReader*>( context );
|
||||
if ( self.mToken.isCancellationRequested() || amount <= 0 )
|
||||
return 0;
|
||||
const auto position = self.mFile.tell();
|
||||
const auto size = self.mFile.size();
|
||||
if ( position < 0 || size < position )
|
||||
return 0;
|
||||
const auto skip = std::min<Int64>( amount, size - position );
|
||||
return self.mFile.seek( position + skip ) == position + skip ? skip : 0;
|
||||
}
|
||||
|
||||
LibArchive::Offset LibArchiveReader::seekCallback( archive*, void* context,
|
||||
LibArchive::Offset offset, int whence ) {
|
||||
auto& self = *static_cast<LibArchiveReader*>( context );
|
||||
if ( self.mToken.isCancellationRequested() )
|
||||
return -30;
|
||||
Int64 base = whence == SEEK_SET ? 0
|
||||
: whence == SEEK_CUR ? self.mFile.tell()
|
||||
: self.mFile.size();
|
||||
if ( base < 0 || ( offset < 0 && offset < -base ) ||
|
||||
( offset > 0 && offset > std::numeric_limits<Int64>::max() - base ) )
|
||||
return -25;
|
||||
return self.mFile.seek( base + offset );
|
||||
}
|
||||
|
||||
}}} // namespace EE::VFS::detail
|
||||
@@ -0,0 +1,137 @@
|
||||
#ifndef EE_VFS_ARCHIVE_LIBARCHIVE_HPP
|
||||
#define EE_VFS_ARCHIVE_LIBARCHIVE_HPP
|
||||
|
||||
#include "archiveinput.hpp"
|
||||
#include <array>
|
||||
#include <cstdint>
|
||||
#include <ctime>
|
||||
#include <eepp/vfs/providers/archiveprovider.hpp>
|
||||
#if EE_PLATFORM != EE_PLATFORM_WIN
|
||||
#include <sys/types.h>
|
||||
#endif
|
||||
|
||||
struct archive;
|
||||
struct archive_entry;
|
||||
|
||||
namespace EE { namespace VFS { namespace detail {
|
||||
|
||||
/** Private subset of libarchive 3.x's C ABI. No struct layout is imported. Reader callbacks use
|
||||
* eepp's UTF-8 file I/O and cancellation; all native resources die before the loader reference. */
|
||||
class EE_VFS_API LibArchive {
|
||||
public:
|
||||
#if EE_PLATFORM == EE_PLATFORM_WIN && !defined( _WIN64 )
|
||||
using Count = long;
|
||||
#else
|
||||
using Count = std::intptr_t;
|
||||
#endif
|
||||
using Offset = std::int64_t;
|
||||
#if EE_PLATFORM == EE_PLATFORM_WIN
|
||||
using Mode = unsigned short;
|
||||
#else
|
||||
using Mode = mode_t;
|
||||
#endif
|
||||
using Enable = int ( * )( archive* );
|
||||
using Read = Count ( * )( archive*, void*, const void** );
|
||||
using Skip = Offset ( * )( archive*, void*, Offset );
|
||||
using Seek = Offset ( * )( archive*, void*, Offset, int );
|
||||
|
||||
static VFSResult<std::shared_ptr<LibArchive>> load( const std::string& path, const URI& uri );
|
||||
|
||||
LibArchive() = default;
|
||||
|
||||
~LibArchive();
|
||||
|
||||
LibArchive( const LibArchive& ) = delete;
|
||||
|
||||
LibArchive& operator=( const LibArchive& ) = delete;
|
||||
|
||||
ArchiveSupport support() const;
|
||||
|
||||
VFSResult<void> configure( archive* reader, const URI& uri ) const;
|
||||
|
||||
VFSError error( archive* reader, const URI& uri ) const;
|
||||
|
||||
archive* ( *readNew )() = nullptr;
|
||||
int ( *readFree )( archive* ) = nullptr;
|
||||
int ( *readOpen )( archive*, void*, int ( * )( archive*, void* ), Read, Skip,
|
||||
int ( * )( archive*, void* ) ) = nullptr;
|
||||
int ( *setSeek )( archive*, Seek ) = nullptr;
|
||||
int ( *nextHeader )( archive*, archive_entry** ) = nullptr;
|
||||
Count ( *readData )( archive*, void*, size_t ) = nullptr;
|
||||
const char* ( *errorString )( archive* ) = nullptr;
|
||||
int ( *errorNumber )( archive* ) = nullptr;
|
||||
int ( *formatCode )( archive* ) = nullptr;
|
||||
int ( *filterCount )( archive* ) = nullptr;
|
||||
int ( *filterCode )( archive*, int ) = nullptr;
|
||||
const char* ( *entryPath )( archive_entry* ) = nullptr;
|
||||
const char* ( *entryLink )( archive_entry* ) = nullptr;
|
||||
const char* ( *entryHardLink )( archive_entry* ) = nullptr;
|
||||
Offset ( *entrySize )( archive_entry* ) = nullptr;
|
||||
int ( *entrySizeSet )( archive_entry* ) = nullptr;
|
||||
Mode ( *entryType )( archive_entry* ) = nullptr;
|
||||
Mode ( *entryPermissions )( archive_entry* ) = nullptr;
|
||||
std::time_t ( *entryTime )( archive_entry* ) = nullptr;
|
||||
int ( *entryTimeSet )( archive_entry* ) = nullptr;
|
||||
|
||||
private:
|
||||
bool bind();
|
||||
|
||||
void* mLibrary = nullptr;
|
||||
int ( *mVersion )() = nullptr;
|
||||
const char* ( *mVersionString )() = nullptr;
|
||||
std::array<Enable, 11> mFormats{};
|
||||
std::array<Enable, 11> mFilters{};
|
||||
};
|
||||
|
||||
class EE_VFS_API LibArchiveReader {
|
||||
public:
|
||||
LibArchiveReader( std::shared_ptr<LibArchive> library, const std::string& path, URI uri );
|
||||
|
||||
~LibArchiveReader();
|
||||
|
||||
LibArchiveReader( const LibArchiveReader& ) = delete;
|
||||
|
||||
LibArchiveReader& operator=( const LibArchiveReader& ) = delete;
|
||||
|
||||
VFSResult<void> open( CancellationToken token );
|
||||
|
||||
VFSResult<bool> next( CancellationToken token );
|
||||
|
||||
VFSResult<size_t> read( std::span<Uint8> data, CancellationToken token );
|
||||
|
||||
void setURI( const URI& uri ) { mUri = uri; }
|
||||
|
||||
/** Only retain the bounded decoders for plain/gzip TAR after a file closes. */
|
||||
bool canReuse() const;
|
||||
|
||||
bool suspend() { return mFile.suspend(); }
|
||||
|
||||
VFSResult<void> resume( CancellationToken token );
|
||||
|
||||
const char* path() const { return mLibrary->entryPath( mEntry ); }
|
||||
|
||||
const char* hardLink() const { return mLibrary->entryHardLink( mEntry ); }
|
||||
|
||||
VFSResult<VFSFileInfo> info() const;
|
||||
|
||||
private:
|
||||
static LibArchive::Count readCallback( archive*, void* self, const void** buffer );
|
||||
|
||||
static LibArchive::Offset skipCallback( archive*, void* self, LibArchive::Offset amount );
|
||||
|
||||
static LibArchive::Offset seekCallback( archive*, void* self, LibArchive::Offset offset,
|
||||
int whence );
|
||||
|
||||
VFSError error() const;
|
||||
|
||||
std::shared_ptr<LibArchive> mLibrary;
|
||||
ArchiveInput mFile;
|
||||
URI mUri;
|
||||
CancellationToken mToken;
|
||||
archive* mReader = nullptr;
|
||||
archive_entry* mEntry = nullptr;
|
||||
std::array<char, 16384> mBuffer;
|
||||
};
|
||||
|
||||
}}} // namespace EE::VFS::detail
|
||||
#endif
|
||||
@@ -0,0 +1,108 @@
|
||||
#ifndef EE_VFS_DETAIL_ARCHIVELOCATION_HPP
|
||||
#define EE_VFS_DETAIL_ARCHIVELOCATION_HPP
|
||||
|
||||
#include <eepp/vfs/error.hpp>
|
||||
|
||||
namespace EE { namespace VFS { namespace detail {
|
||||
|
||||
/** Parsed archive identity and normalized path within it. */
|
||||
class ArchiveLocation {
|
||||
public:
|
||||
URI archive;
|
||||
std::string member;
|
||||
|
||||
/** Archive paths are untrusted. Reject names that could leave a copy destination or turn
|
||||
* into native Windows paths/streams; do not alias unsafe entries to safe names. */
|
||||
static bool safeEntry( std::string_view name ) {
|
||||
if ( name.empty() || name.size() > 4096 || name.front() == '/' ||
|
||||
name.find_first_of( std::string_view( "\\:\0", 3 ) ) != std::string_view::npos )
|
||||
return false;
|
||||
size_t start = 0;
|
||||
while ( start < name.size() ) {
|
||||
size_t end = name.find( '/', start );
|
||||
if ( end == std::string_view::npos )
|
||||
end = name.size();
|
||||
auto part = name.substr( start, end - start );
|
||||
// Windows trims trailing dots/spaces; an all-dot/space segment can become a parent.
|
||||
if ( part.empty() || part.find_first_not_of( ". " ) == std::string_view::npos )
|
||||
return false;
|
||||
start = end + 1;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
static VFSResult<ArchiveLocation> parse( const URI& uri, std::string_view scheme ) {
|
||||
if ( uri.getScheme() != scheme || uri.getAuthority() != uri.getHost() ||
|
||||
!uri.getFragment().empty() || uri.getPath().find( '\0' ) != std::string::npos )
|
||||
return Err( VFSError( VFSErrorCode::InvalidURI, uri, "Invalid archive location" ) );
|
||||
if ( !uri.getRawQuery().empty() )
|
||||
return Err(
|
||||
VFSError( VFSErrorCode::Unsupported, uri, "Unsupported archive backing URI" ) );
|
||||
const auto& path = uri.getPath();
|
||||
if ( path.empty() || path.front() != '/' )
|
||||
return Err(
|
||||
VFSError( VFSErrorCode::InvalidURI, uri, "archive path must be absolute" ) );
|
||||
std::string backing;
|
||||
backing.reserve( path.size() );
|
||||
size_t boundary = std::string::npos;
|
||||
for ( size_t i = 0; i < path.size(); ++i ) {
|
||||
if ( path[i] == '!' ) {
|
||||
if ( i + 1 < path.size() && path[i + 1] == '!' ) {
|
||||
backing += '!';
|
||||
++i;
|
||||
continue;
|
||||
}
|
||||
if ( i + 1 == path.size() || path[i + 1] == '/' ) {
|
||||
boundary = i + 1;
|
||||
break;
|
||||
}
|
||||
}
|
||||
backing += path[i];
|
||||
}
|
||||
if ( boundary == std::string::npos || backing.empty() || backing.back() == '/' )
|
||||
return Err(
|
||||
VFSError( VFSErrorCode::InvalidURI, uri, "archive path needs archive!/member" ) );
|
||||
ArchiveLocation result;
|
||||
result.archive.setScheme( "file" );
|
||||
result.archive.setHost( uri.getHost() );
|
||||
result.archive.setRawPath( std::move( backing ) );
|
||||
URI member;
|
||||
member.setScheme( std::string( scheme ) );
|
||||
member.setRawPath( boundary == path.size() ? "/" : path.substr( boundary ) );
|
||||
member.normalize();
|
||||
result.member = member.getPath();
|
||||
return Ok( std::move( result ) );
|
||||
}
|
||||
static URI make( std::string_view scheme, const URI& archive, std::string_view member ) {
|
||||
if ( archive.getScheme() != "file" || !archive.getRawQuery().empty() ||
|
||||
!archive.getFragment().empty() || archive.getAuthority() != archive.getHost() ||
|
||||
archive.getPath().empty() || archive.getPath().front() != '/' ||
|
||||
archive.getPath().find( '\0' ) != std::string::npos )
|
||||
return {};
|
||||
URI uri;
|
||||
uri.setScheme( std::string( scheme ) );
|
||||
uri.setHost( archive.getHost() );
|
||||
std::string path;
|
||||
path.reserve( archive.getPath().size() + member.size() + 2 );
|
||||
// Escape literal bangs in the backing path, so even a directory named "packages!" is
|
||||
// unambiguous. Only the first unescaped "!/" separates the archive from its member.
|
||||
for ( char c : archive.getPath() ) {
|
||||
path += c;
|
||||
if ( c == '!' )
|
||||
path += '!';
|
||||
}
|
||||
URI entry;
|
||||
entry.setScheme( std::string( scheme ) );
|
||||
entry.setRawPath( member.empty() || member.front() != '/' ? "/" + std::string( member )
|
||||
: std::string( member ) );
|
||||
entry.normalize();
|
||||
path += '!';
|
||||
path += entry.getPath();
|
||||
uri.setRawPath( std::move( path ) );
|
||||
return uri;
|
||||
}
|
||||
};
|
||||
|
||||
}}} // namespace EE::VFS::detail
|
||||
|
||||
#endif
|
||||
@@ -0,0 +1,697 @@
|
||||
#include <eepp/vfs/providers/archiveprovider.hpp>
|
||||
|
||||
#include <algorithm>
|
||||
#include <array>
|
||||
#include <eepp/core/containers.hpp>
|
||||
#include <eepp/vfs/archive/libarchive.hpp>
|
||||
#include <eepp/vfs/detail/archivelocation.hpp>
|
||||
#include <eepp/vfs/detail/memoryhandles.hpp>
|
||||
#include <eepp/vfs/providers/localprovider.hpp>
|
||||
#include <limits>
|
||||
#include <mutex>
|
||||
|
||||
namespace EE { namespace VFS {
|
||||
|
||||
using detail::ArchiveLocation;
|
||||
using detail::LibArchive;
|
||||
using detail::LibArchiveReader;
|
||||
|
||||
namespace {
|
||||
constexpr ProviderCapabilities ArchiveCapabilities =
|
||||
VFSCapability::Read | VFSCapability::List | VFSCapability::Seek | VFSCapability::CaseSensitive;
|
||||
constexpr size_t MaxEntries = 262144;
|
||||
constexpr size_t MaxMetadata = 64 * 1024 * 1024;
|
||||
constexpr Uint64 NoEntry = std::numeric_limits<Uint64>::max();
|
||||
|
||||
VFSError readOnly( const URI& uri ) {
|
||||
return VFSError( VFSErrorCode::ReadOnly, uri, "Archives are read-only" );
|
||||
}
|
||||
|
||||
struct ArchiveNode {
|
||||
VFSFileInfo info;
|
||||
std::vector<size_t> children;
|
||||
std::string hardLink;
|
||||
Uint64 entry = NoEntry;
|
||||
};
|
||||
|
||||
/** Immutable index; readers are independent and retain the library and revision they opened. */
|
||||
struct ArchiveIndex {
|
||||
std::string path;
|
||||
std::vector<ArchiveNode> nodes;
|
||||
UnorderedMap<std::string, size_t> paths;
|
||||
std::shared_ptr<LibArchive> library;
|
||||
FileRevision revision;
|
||||
LocalIOScheduler::Route route;
|
||||
|
||||
struct IdleReader {
|
||||
std::unique_ptr<LibArchiveReader> reader;
|
||||
Uint64 entry = NoEntry;
|
||||
};
|
||||
|
||||
IdleReader takeReader( Uint64 entry ) {
|
||||
std::lock_guard<std::mutex> lock( mReaderMutex );
|
||||
if ( mIdle.reader && mIdle.entry < entry )
|
||||
return std::move( mIdle );
|
||||
return {};
|
||||
}
|
||||
|
||||
void recycleReader( std::unique_ptr<LibArchiveReader> reader, Uint64 entry ) {
|
||||
if ( !reader )
|
||||
return;
|
||||
IdleReader retired;
|
||||
{
|
||||
std::lock_guard<std::mutex> lock( mReaderMutex );
|
||||
retired = std::move( mIdle );
|
||||
mIdle = { std::move( reader ), entry };
|
||||
}
|
||||
// Native reader teardown never holds the index lock. At most one idle reader is kept,
|
||||
// allowing sequential copies to advance without replaying each entry's archive prefix.
|
||||
}
|
||||
|
||||
VFSResult<size_t> find( std::string member, const URI& uri, bool follow = true,
|
||||
std::string* resolved = nullptr ) const {
|
||||
// Resolve links only within the archive. URI normalization clamps parent components at
|
||||
// the member root; no target is ever passed to a native filesystem API.
|
||||
for ( size_t links = 0; links <= 40; ++links ) {
|
||||
size_t start = 0;
|
||||
bool redirected = false;
|
||||
while ( start < member.size() ) {
|
||||
size_t end = member.find( '/', start );
|
||||
if ( end == std::string::npos )
|
||||
end = member.size();
|
||||
auto found = paths.find( member.substr( 0, end ) );
|
||||
if ( found == paths.end() )
|
||||
return Err( VFSError( VFSErrorCode::NotFound, uri ) );
|
||||
const auto& node = nodes[found->second];
|
||||
if ( ( node.info.isSymlink() && ( follow || end < member.size() ) ) ||
|
||||
!node.hardLink.empty() ) {
|
||||
const std::string& target =
|
||||
node.hardLink.empty() ? node.info.symlinkTarget : node.hardLink;
|
||||
if ( target.empty() || target.size() > 4096 ||
|
||||
target.find_first_of( "\\:" ) != std::string::npos )
|
||||
return Err(
|
||||
VFSError( VFSErrorCode::Unsupported, uri, "Invalid archive link" ) );
|
||||
std::string path;
|
||||
if ( !node.hardLink.empty() || target.front() == '/' )
|
||||
path = "/" + target;
|
||||
else
|
||||
path = "/" + member.substr( 0, start ) + target;
|
||||
path += member.substr( end );
|
||||
URI normalized;
|
||||
normalized.setRawPath( std::move( path ) );
|
||||
normalized.normalize();
|
||||
member = normalized.getPath();
|
||||
while ( !member.empty() && member.front() == '/' )
|
||||
member.erase( member.begin() );
|
||||
redirected = true;
|
||||
break;
|
||||
}
|
||||
if ( end < member.size() && !node.info.isDirectory() )
|
||||
return Err( VFSError( VFSErrorCode::NotDirectory, uri ) );
|
||||
start = end + 1;
|
||||
}
|
||||
if ( redirected )
|
||||
continue;
|
||||
auto found = paths.find( member );
|
||||
if ( found == paths.end() )
|
||||
return Err( VFSError( VFSErrorCode::NotFound, uri ) );
|
||||
if ( uri.getPath().ends_with( '/' ) && !nodes[found->second].info.isDirectory() )
|
||||
return Err( VFSError( VFSErrorCode::NotDirectory, uri ) );
|
||||
if ( resolved )
|
||||
*resolved = std::move( member );
|
||||
return Ok( found->second );
|
||||
}
|
||||
return Err( VFSError( VFSErrorCode::TooManySymlinks, uri ) );
|
||||
}
|
||||
|
||||
VFSResult<void> index( LibArchiveReader& reader, const URI& uri, CancellationToken token ) {
|
||||
VFSFileInfo root;
|
||||
root.name = "/";
|
||||
root.type = FileType::Directory;
|
||||
root.revision = revision;
|
||||
nodes.push_back( { std::move( root ), {}, {}, NoEntry } );
|
||||
paths.emplace( "", 0 );
|
||||
size_t budget = MaxMetadata;
|
||||
for ( Uint64 ordinal = 0;; ++ordinal ) {
|
||||
auto next = reader.next( token );
|
||||
if ( !next )
|
||||
return Err( std::move( next ).error() );
|
||||
if ( !next.value() )
|
||||
break;
|
||||
if ( ordinal >= MaxEntries )
|
||||
return Err( VFSError( VFSErrorCode::InvalidOperation, uri,
|
||||
"Archive has too many entries" ) );
|
||||
const char* raw = reader.path();
|
||||
if ( !raw )
|
||||
return Err( VFSError( VFSErrorCode::Unsupported, uri,
|
||||
"Archive filename cannot be decoded as UTF-8" ) );
|
||||
std::string_view name( raw );
|
||||
while ( name.starts_with( "./" ) )
|
||||
name.remove_prefix( 2 );
|
||||
if ( name == "." || name.empty() )
|
||||
continue;
|
||||
if ( !ArchiveLocation::safeEntry( name ) )
|
||||
continue;
|
||||
if ( name.ends_with( '/' ) )
|
||||
name.remove_suffix( 1 );
|
||||
auto info = reader.info();
|
||||
if ( !info )
|
||||
return Err( std::move( info ).error() );
|
||||
VFSFileInfo metadata = std::move( info ).value();
|
||||
const char* hardLink = reader.hardLink();
|
||||
if ( metadata.symlinkTarget.size() > 4096 ||
|
||||
( hardLink && std::char_traits<char>::length( hardLink ) > 4096 ) )
|
||||
return Err(
|
||||
VFSError( VFSErrorCode::InvalidOperation, uri, "Archive link is too long" ) );
|
||||
size_t start = 0, parent = 0;
|
||||
while ( start < name.size() ) {
|
||||
size_t end = name.find( '/', start );
|
||||
if ( end == std::string_view::npos )
|
||||
end = name.size();
|
||||
bool last = end == name.size();
|
||||
bool folder = !last || metadata.isDirectory();
|
||||
std::string key( name.substr( 0, end ) );
|
||||
auto found = paths.find( key );
|
||||
size_t node;
|
||||
if ( found == paths.end() ) {
|
||||
const size_t cost =
|
||||
2 * ( sizeof( ArchiveNode ) + key.size() + end - start + 128 );
|
||||
if ( nodes.size() >= MaxEntries || cost > budget )
|
||||
return Err( VFSError( VFSErrorCode::InvalidOperation, uri,
|
||||
"Archive metadata budget exceeded" ) );
|
||||
budget -= cost;
|
||||
node = nodes.size();
|
||||
paths.emplace( std::move( key ), node );
|
||||
nodes[parent].children.push_back( node );
|
||||
VFSFileInfo info;
|
||||
info.name = std::string( name.substr( start, end - start ) );
|
||||
info.type = folder ? FileType::Directory : FileType::Unknown;
|
||||
info.hidden = info.name.starts_with( '.' );
|
||||
info.revision = revision;
|
||||
nodes.push_back( { std::move( info ), {}, {}, NoEntry } );
|
||||
} else {
|
||||
node = found->second;
|
||||
}
|
||||
auto& value = nodes[node];
|
||||
if ( folder && !value.info.isDirectory() ) {
|
||||
VFSFileInfo directory;
|
||||
directory.name = std::move( value.info.name );
|
||||
directory.hidden = value.info.hidden;
|
||||
directory.type = FileType::Directory;
|
||||
directory.revision = revision;
|
||||
value.info = std::move( directory );
|
||||
value.entry = NoEntry;
|
||||
value.hardLink.clear();
|
||||
}
|
||||
// As with ZIP, directories win collisions and duplicate files keep the first.
|
||||
if ( last && ( folder || !value.info.isDirectory() ) && value.entry == NoEntry ) {
|
||||
size_t cost =
|
||||
2 * ( metadata.symlinkTarget.size() +
|
||||
( hardLink ? std::char_traits<char>::length( hardLink ) : 0 ) );
|
||||
if ( cost > budget )
|
||||
return Err( VFSError( VFSErrorCode::InvalidOperation, uri,
|
||||
"Archive metadata budget exceeded" ) );
|
||||
budget -= cost;
|
||||
metadata.name = std::move( value.info.name );
|
||||
metadata.hidden = metadata.name.starts_with( '.' );
|
||||
metadata.revision = FileRevision::fromValues(
|
||||
{ std::hash<std::string_view>{}( std::string_view(
|
||||
reinterpret_cast<const char*>( revision.data() ), revision.size() ) ),
|
||||
ordinal } );
|
||||
value.info = std::move( metadata );
|
||||
value.entry = ordinal;
|
||||
if ( hardLink )
|
||||
value.hardLink = hardLink;
|
||||
}
|
||||
parent = node;
|
||||
start = end + 1;
|
||||
}
|
||||
}
|
||||
return Ok();
|
||||
}
|
||||
|
||||
private:
|
||||
std::mutex mReaderMutex;
|
||||
IdleReader mIdle;
|
||||
};
|
||||
|
||||
class ArchiveFile : public FileHandleImpl, public std::enable_shared_from_this<ArchiveFile> {
|
||||
public:
|
||||
ArchiveFile( std::shared_ptr<LocalIOScheduler> scheduler, std::shared_ptr<ArchiveIndex> index,
|
||||
std::unique_ptr<LibArchiveReader> reader, size_t node, URI uri ) :
|
||||
mScheduler( std::move( scheduler ) ),
|
||||
mIndex( std::move( index ) ),
|
||||
mReader( std::move( reader ) ),
|
||||
mUri( std::move( uri ) ),
|
||||
mNode( static_cast<Uint32>( node ) ) {}
|
||||
|
||||
static VFSResult<std::unique_ptr<LibArchiveReader>> openReader( ArchiveIndex& index,
|
||||
size_t node, const URI& uri,
|
||||
CancellationToken token,
|
||||
bool reuse = true ) {
|
||||
auto current = LocalProvider::statSync( URI::fromFSPath( index.path ), index.path, true );
|
||||
if ( !current || current.value().revision != index.revision )
|
||||
return Err( VFSError( VFSErrorCode::ChangedExternally, uri,
|
||||
"Archive changed while opening entry" ) );
|
||||
auto idle =
|
||||
reuse ? index.takeReader( index.nodes[node].entry ) : ArchiveIndex::IdleReader{};
|
||||
auto reader = std::move( idle.reader );
|
||||
Uint64 first = reader ? idle.entry + 1 : 0;
|
||||
if ( reader ) {
|
||||
reader->setURI( uri );
|
||||
auto resumed = reader->resume( token );
|
||||
if ( !resumed )
|
||||
return Err( std::move( resumed ).error() );
|
||||
} else {
|
||||
reader = std::make_unique<LibArchiveReader>( index.library, index.path, uri );
|
||||
auto opened = reader->open( token );
|
||||
if ( !opened )
|
||||
return Err( std::move( opened ).error() );
|
||||
first = 0;
|
||||
}
|
||||
current = LocalProvider::statSync( URI::fromFSPath( index.path ), index.path, true );
|
||||
if ( !current || current.value().revision != index.revision )
|
||||
return Err( VFSError( VFSErrorCode::ChangedExternally, uri,
|
||||
"Archive changed while opening entry" ) );
|
||||
reader->setURI( uri );
|
||||
for ( Uint64 i = first; i <= index.nodes[node].entry; ++i ) {
|
||||
auto next = reader->next( token );
|
||||
if ( !next )
|
||||
return Err( std::move( next ).error() );
|
||||
if ( !next.value() )
|
||||
return Err( VFSError( VFSErrorCode::IO, uri, "Archive entry is missing" ) );
|
||||
}
|
||||
return Ok( std::move( reader ) );
|
||||
}
|
||||
|
||||
VFSTask<size_t> read( Uint64 offset, std::span<Uint8> data, CancellationToken token ) override {
|
||||
auto work = [self = shared_from_this(), offset,
|
||||
token]( std::span<Uint8> buffer ) -> VFSResult<size_t> {
|
||||
std::lock_guard<std::mutex> lock( self->mMutex );
|
||||
auto result = self->readInto( offset, buffer, token );
|
||||
if ( !result )
|
||||
self->mReusable = false;
|
||||
return result;
|
||||
};
|
||||
if ( !mIndex->route.isNetworkMount() ) {
|
||||
return mScheduler->submit<size_t>(
|
||||
mIndex->route,
|
||||
[work = std::move( work ), data]() mutable { return work( data ); } );
|
||||
}
|
||||
// A mount can return after its timeout, when data no longer exists. Commit only if
|
||||
// this call still owns completion; its work touches an independent buffer.
|
||||
return mScheduler->submitCommitted<size_t>(
|
||||
mIndex->route,
|
||||
[work = std::move( work ),
|
||||
size = data.size()]() mutable -> VFSResult<std::vector<Uint8>> {
|
||||
std::vector<Uint8> buffer( size );
|
||||
auto result = work( buffer );
|
||||
if ( !result )
|
||||
return Err( std::move( result ).error() );
|
||||
buffer.resize( result.value() );
|
||||
return Ok( std::move( buffer ) );
|
||||
},
|
||||
[data]( VFSResult<std::vector<Uint8>>&& result ) -> VFSResult<size_t> {
|
||||
if ( !result )
|
||||
return Err( std::move( result ).error() );
|
||||
std::copy( result.value().begin(), result.value().end(), data.begin() );
|
||||
return Ok( result.value().size() );
|
||||
} );
|
||||
}
|
||||
|
||||
VFSTask<size_t> write( Uint64, std::span<const Uint8>, CancellationToken ) override {
|
||||
return readyError<size_t>( readOnly( mUri ) );
|
||||
}
|
||||
|
||||
VFSTask<void> truncate( Uint64 ) override { return readyError<void>( readOnly( mUri ) ); }
|
||||
|
||||
VFSTask<void> flush() override { return readyValue<void>(); }
|
||||
|
||||
VFSTask<VFSFileInfo> stat() override {
|
||||
return readyValue<VFSFileInfo>( mIndex->nodes[mNode].info );
|
||||
}
|
||||
|
||||
VFSTask<void> close() override {
|
||||
return mScheduler->submit<void>(
|
||||
mIndex->route, [self = shared_from_this()]() -> VFSResult<void> {
|
||||
std::lock_guard<std::mutex> lock( self->mMutex );
|
||||
if ( self->mReusable && self->mReader && self->mReader->canReuse() &&
|
||||
self->mReader->suspend() ) {
|
||||
self->mIndex->recycleReader( std::move( self->mReader ),
|
||||
self->mIndex->nodes[self->mNode].entry );
|
||||
} else {
|
||||
self->mReader.reset();
|
||||
}
|
||||
return Ok();
|
||||
} );
|
||||
}
|
||||
|
||||
ProviderCapabilities capabilities() const override { return ArchiveCapabilities; }
|
||||
|
||||
private:
|
||||
VFSResult<size_t> readInto( Uint64 offset, std::span<Uint8> data, CancellationToken token ) {
|
||||
if ( token.isCancellationRequested() )
|
||||
return Err( cancelledError( mUri ) );
|
||||
if ( !mReader )
|
||||
return Err( VFSError( VFSErrorCode::InvalidOperation, mUri, "File closed" ) );
|
||||
if ( data.empty() )
|
||||
return Ok( size_t( 0 ) );
|
||||
const auto size = mIndex->nodes[mNode].info.size();
|
||||
if ( size && offset >= *size )
|
||||
return Ok( size_t( 0 ) );
|
||||
if ( offset < mPosition ) {
|
||||
auto reopened = openReader( *mIndex, mNode, mUri, token, false );
|
||||
if ( !reopened )
|
||||
return Err( std::move( reopened ).error() );
|
||||
mReader = std::move( reopened ).value();
|
||||
mPosition = 0;
|
||||
}
|
||||
std::array<Uint8, 16384> scratch;
|
||||
while ( mPosition < offset ) {
|
||||
auto got = mReader->read( std::span<Uint8>( scratch ).first(
|
||||
std::min<Uint64>( scratch.size(), offset - mPosition ) ),
|
||||
token );
|
||||
if ( !got )
|
||||
return got;
|
||||
if ( got.value() == 0 )
|
||||
return Err( VFSError( VFSErrorCode::IO, mUri, "Truncated archive entry" ) );
|
||||
mPosition += got.value();
|
||||
}
|
||||
size_t done = 0;
|
||||
const size_t wanted = size ? std::min<Uint64>( data.size(), *size - offset ) : data.size();
|
||||
while ( done < wanted ) {
|
||||
auto got = mReader->read(
|
||||
data.subspan( done, std::min<size_t>( wanted - done, 65536 ) ), token );
|
||||
if ( !got )
|
||||
return got;
|
||||
if ( got.value() == 0 ) {
|
||||
if ( size )
|
||||
return Err( VFSError( VFSErrorCode::IO, mUri, "Truncated archive entry" ) );
|
||||
break;
|
||||
}
|
||||
done += got.value();
|
||||
mPosition += got.value();
|
||||
}
|
||||
// Check the entry's end marker too: some compression/checksum errors surface only
|
||||
// when a read advances past the final output byte.
|
||||
if ( size && mPosition == *size ) {
|
||||
auto end = mReader->read( std::span<Uint8>( scratch ).first( 1 ), token );
|
||||
if ( !end )
|
||||
return end;
|
||||
if ( end.value() != 0 )
|
||||
return Err(
|
||||
VFSError( VFSErrorCode::IO, mUri, "Archive entry exceeds declared size" ) );
|
||||
}
|
||||
return Ok( done );
|
||||
}
|
||||
|
||||
std::shared_ptr<LocalIOScheduler> mScheduler;
|
||||
std::shared_ptr<ArchiveIndex> mIndex;
|
||||
std::unique_ptr<LibArchiveReader> mReader;
|
||||
std::mutex mMutex;
|
||||
URI mUri;
|
||||
Uint64 mPosition = 0;
|
||||
Uint32 mNode; // The index is capped at MaxEntries, well below UINT32_MAX.
|
||||
bool mReusable = true;
|
||||
};
|
||||
} // namespace
|
||||
|
||||
class ArchiveProviderImpl : public std::enable_shared_from_this<ArchiveProviderImpl> {
|
||||
public:
|
||||
ArchiveProviderImpl( std::shared_ptr<LocalIOScheduler> scheduler, std::string path ) :
|
||||
scheduler( std::move( scheduler ) ), mLibraryPath( std::move( path ) ) {}
|
||||
|
||||
VFSResult<std::shared_ptr<LibArchive>> library( const URI& uri ) {
|
||||
std::call_once( mLoad, [&] {
|
||||
auto loaded = LibArchive::load( mLibraryPath, uri );
|
||||
if ( loaded )
|
||||
mLibrary = std::move( loaded ).value();
|
||||
} );
|
||||
if ( !mLibrary )
|
||||
return Err(
|
||||
VFSError( VFSErrorCode::Unsupported, uri,
|
||||
"A compatible libarchive 3.x runtime library is not available" ) );
|
||||
return Ok( mLibrary );
|
||||
}
|
||||
|
||||
template <typename T, typename F>
|
||||
VFSTask<T> run( const URI& uri, CancellationToken token, F&& fn ) {
|
||||
if ( token.isCancellationRequested() )
|
||||
return readyError<T>( cancelledError( uri ) );
|
||||
auto address = ArchiveLocation::parse( uri, "archive" );
|
||||
if ( !address )
|
||||
return readyError<T>( std::move( address ).error() );
|
||||
auto native = LocalProvider::toNativePath( address.value().archive );
|
||||
if ( !native )
|
||||
return readyError<T>( std::move( native ).error() );
|
||||
std::string path = std::move( native ).value();
|
||||
std::string member = std::move( address ).value().member;
|
||||
while ( !member.empty() && member.front() == '/' )
|
||||
member.erase( member.begin() );
|
||||
while ( !member.empty() && member.back() == '/' )
|
||||
member.pop_back();
|
||||
return scheduler->submitResolved<T>(
|
||||
path, true,
|
||||
[self = shared_from_this(), uri, path, member = std::move( member ),
|
||||
token = std::move( token ), fn = std::forward<F>( fn )]() mutable -> VFSResult<T> {
|
||||
if ( token.isCancellationRequested() )
|
||||
return Err( cancelledError( uri ) );
|
||||
auto index = self->load( path, uri, token );
|
||||
if ( !index )
|
||||
return Err( std::move( index ).error() );
|
||||
return fn( std::move( index ).value(), member, uri, token );
|
||||
} );
|
||||
}
|
||||
|
||||
std::shared_ptr<LocalIOScheduler> scheduler;
|
||||
|
||||
private:
|
||||
struct Cached {
|
||||
std::shared_ptr<ArchiveIndex> index;
|
||||
Uint64 used = 0;
|
||||
};
|
||||
|
||||
VFSResult<std::shared_ptr<ArchiveIndex>> load( const std::string& path, const URI& uri,
|
||||
CancellationToken token ) {
|
||||
auto loaded = library( uri );
|
||||
if ( !loaded )
|
||||
return Err( std::move( loaded ).error() );
|
||||
const URI backing = URI::fromFSPath( path );
|
||||
auto before = LocalProvider::statSync( backing, path, true );
|
||||
if ( !before )
|
||||
return Err( std::move( before ).error() );
|
||||
if ( !before.value().isRegularFile() )
|
||||
return Err( VFSError( VFSErrorCode::InvalidOperation, uri,
|
||||
"Archive backing file is not a regular file" ) );
|
||||
const auto revision = before.value().revision;
|
||||
{
|
||||
std::lock_guard<std::mutex> lock( mMutex );
|
||||
for ( auto& cached : mCache ) {
|
||||
if ( cached.index && cached.index->path == path &&
|
||||
cached.index->revision == revision ) {
|
||||
cached.used = ++mUsed;
|
||||
return Ok( cached.index );
|
||||
}
|
||||
}
|
||||
}
|
||||
auto index = std::make_shared<ArchiveIndex>();
|
||||
index->path = path;
|
||||
index->revision = revision;
|
||||
index->library = std::move( loaded ).value();
|
||||
index->route = scheduler->resolvedRouteFor( path, true );
|
||||
LibArchiveReader reader( index->library, path, uri );
|
||||
auto opened = reader.open( token );
|
||||
if ( !opened )
|
||||
return Err( std::move( opened ).error() );
|
||||
auto indexed = index->index( reader, uri, token );
|
||||
if ( !indexed )
|
||||
return Err( std::move( indexed ).error() );
|
||||
auto after = LocalProvider::statSync( backing, path, true );
|
||||
if ( !after || after.value().revision != revision )
|
||||
return Err( VFSError( VFSErrorCode::ChangedExternally, uri,
|
||||
"Archive changed while indexing" ) );
|
||||
std::shared_ptr<ArchiveIndex> retired;
|
||||
{
|
||||
std::lock_guard<std::mutex> lock( mMutex );
|
||||
auto slot = mCache.begin();
|
||||
for ( auto it = mCache.begin(); it != mCache.end(); ++it ) {
|
||||
if ( it->index && it->index->path == path ) {
|
||||
slot = it;
|
||||
break;
|
||||
}
|
||||
if ( it->used < slot->used )
|
||||
slot = it;
|
||||
}
|
||||
retired = std::move( slot->index );
|
||||
slot->index = index;
|
||||
slot->used = ++mUsed;
|
||||
}
|
||||
return Ok( std::move( index ) );
|
||||
}
|
||||
|
||||
std::string mLibraryPath;
|
||||
std::shared_ptr<LibArchive> mLibrary;
|
||||
std::once_flag mLoad;
|
||||
std::mutex mMutex;
|
||||
std::array<Cached, 8> mCache;
|
||||
Uint64 mUsed = 0;
|
||||
};
|
||||
|
||||
ArchiveProvider::ArchiveProvider( std::shared_ptr<LocalIOScheduler> scheduler,
|
||||
std::string libraryPath ) :
|
||||
mImpl( std::make_shared<ArchiveProviderImpl>( std::move( scheduler ),
|
||||
std::move( libraryPath ) ) ) {}
|
||||
|
||||
ArchiveProvider::~ArchiveProvider() = default;
|
||||
|
||||
URI ArchiveProvider::makeURI( const URI& archive, std::string_view member ) {
|
||||
return ArchiveLocation::make( "archive", archive, member );
|
||||
}
|
||||
|
||||
VFSResult<URI> ArchiveProvider::archiveURI( const URI& uri ) {
|
||||
auto address = ArchiveLocation::parse( uri, "archive" );
|
||||
if ( !address )
|
||||
return Err( std::move( address ).error() );
|
||||
return Ok( std::move( address ).value().archive );
|
||||
}
|
||||
|
||||
VFSResult<std::string> ArchiveProvider::memberPath( const URI& uri ) {
|
||||
auto address = ArchiveLocation::parse( uri, "archive" );
|
||||
if ( !address )
|
||||
return Err( std::move( address ).error() );
|
||||
return Ok( std::move( address ).value().member );
|
||||
}
|
||||
|
||||
VFSTask<ArchiveSupport> ArchiveProvider::support( CancellationToken token ) {
|
||||
return mImpl->scheduler->submit<ArchiveSupport>(
|
||||
LocalIOScheduler::Route{},
|
||||
[impl = mImpl, token = std::move( token )]() -> VFSResult<ArchiveSupport> {
|
||||
if ( token.isCancellationRequested() )
|
||||
return Err( cancelledError( {} ) );
|
||||
auto library = impl->library( {} );
|
||||
if ( !library )
|
||||
return Err( std::move( library ).error() );
|
||||
return Ok( library.value()->support() );
|
||||
} );
|
||||
}
|
||||
|
||||
ProviderCapabilities ArchiveProvider::capabilities( const URI& ) const {
|
||||
return ArchiveCapabilities;
|
||||
}
|
||||
|
||||
VFSTask<VFSFileInfo> ArchiveProvider::stat( const URI& uri, const StatOptions& options,
|
||||
CancellationToken token ) {
|
||||
return mImpl->run<VFSFileInfo>(
|
||||
uri, std::move( token ),
|
||||
[options]( const std::shared_ptr<ArchiveIndex>& index, const std::string& member,
|
||||
const URI& uri, CancellationToken ) -> VFSResult<VFSFileInfo> {
|
||||
auto found = index->find( member, uri, options.followSymlinks );
|
||||
if ( !found )
|
||||
return Err( std::move( found ).error() );
|
||||
auto info = index->nodes[found.value()].info;
|
||||
if ( !member.empty() )
|
||||
info.name = member.substr( member.find_last_of( '/' ) + 1 );
|
||||
info.hidden = info.name.starts_with( '.' );
|
||||
return Ok( std::move( info ) );
|
||||
} );
|
||||
}
|
||||
|
||||
VFSTask<DirectoryHandle> ArchiveProvider::openDirectory( const URI& uri,
|
||||
const DirectoryOpenOptions&,
|
||||
CancellationToken token ) {
|
||||
return mImpl->run<DirectoryHandle>(
|
||||
uri, std::move( token ),
|
||||
[]( const std::shared_ptr<ArchiveIndex>& index, const std::string& member, const URI& uri,
|
||||
CancellationToken token ) -> VFSResult<DirectoryHandle> {
|
||||
auto found = index->find( member, uri );
|
||||
if ( !found )
|
||||
return Err( std::move( found ).error() );
|
||||
const auto& folder = index->nodes[found.value()];
|
||||
if ( !folder.info.isDirectory() )
|
||||
return Err( VFSError( VFSErrorCode::NotDirectory, uri ) );
|
||||
std::vector<VFSFileInfo> entries;
|
||||
entries.reserve( folder.children.size() );
|
||||
for ( auto child : folder.children ) {
|
||||
if ( token.isCancellationRequested() )
|
||||
return Err( cancelledError( uri ) );
|
||||
auto info = index->nodes[child].info;
|
||||
if ( info.isSymlink() || !index->nodes[child].hardLink.empty() ) {
|
||||
auto target =
|
||||
index->find( member.empty() ? info.name : member + "/" + info.name, uri );
|
||||
if ( target ) {
|
||||
if ( !index->nodes[child].hardLink.empty() ) {
|
||||
std::string name = std::move( info.name );
|
||||
info = index->nodes[target.value()].info;
|
||||
info.name = std::move( name );
|
||||
info.hidden = info.name.starts_with( '.' );
|
||||
} else {
|
||||
info.targetType = index->nodes[target.value()].info.type;
|
||||
if ( auto size = index->nodes[target.value()].info.size() )
|
||||
info.setSize( *size );
|
||||
}
|
||||
}
|
||||
}
|
||||
entries.push_back( std::move( info ) );
|
||||
}
|
||||
return Ok( DirectoryHandle(
|
||||
uri, std::make_shared<detail::VectorDirectoryImpl>( std::move( entries ) ) ) );
|
||||
} );
|
||||
}
|
||||
|
||||
VFSTask<FileHandle> ArchiveProvider::open( const URI& uri, const OpenOptions& options,
|
||||
CancellationToken token ) {
|
||||
if ( hasFlag( options.flags, OpenFlags::Write ) ||
|
||||
hasFlag( options.flags, OpenFlags::Create ) ||
|
||||
hasFlag( options.flags, OpenFlags::Truncate ) )
|
||||
return readyError<FileHandle>( readOnly( uri ) );
|
||||
return mImpl->run<FileHandle>(
|
||||
uri, std::move( token ),
|
||||
[scheduler = mImpl->scheduler]( std::shared_ptr<ArchiveIndex> index,
|
||||
const std::string& member, const URI& uri,
|
||||
CancellationToken token ) -> VFSResult<FileHandle> {
|
||||
auto found = index->find( member, uri );
|
||||
if ( !found )
|
||||
return Err( std::move( found ).error() );
|
||||
const auto& node = index->nodes[found.value()];
|
||||
if ( node.info.isDirectory() )
|
||||
return Err( VFSError( VFSErrorCode::IsDirectory, uri ) );
|
||||
if ( !node.info.isRegularFile() || node.entry == NoEntry )
|
||||
return Err( VFSError( VFSErrorCode::Unsupported, uri,
|
||||
"Archive entry is not a regular file" ) );
|
||||
auto reader = ArchiveFile::openReader( *index, found.value(), uri, token );
|
||||
if ( !reader )
|
||||
return Err( std::move( reader ).error() );
|
||||
return Ok( FileHandle( uri, std::make_shared<ArchiveFile>(
|
||||
scheduler, std::move( index ),
|
||||
std::move( reader ).value(), found.value(), uri ) ) );
|
||||
} );
|
||||
}
|
||||
|
||||
VFSTask<void> ArchiveProvider::mkdir( const URI& uri, const MkDirOptions&, CancellationToken ) {
|
||||
return readyError<void>( readOnly( uri ) );
|
||||
}
|
||||
|
||||
VFSTask<void> ArchiveProvider::remove( const URI& uri, const RemoveOptions&, CancellationToken ) {
|
||||
return readyError<void>( readOnly( uri ) );
|
||||
}
|
||||
|
||||
VFSTask<void> ArchiveProvider::rename( const URI& uri, const URI&, const RenameOptions&,
|
||||
CancellationToken ) {
|
||||
return readyError<void>( readOnly( uri ) );
|
||||
}
|
||||
|
||||
VFSTask<URI> ArchiveProvider::canonicalize( const URI& uri, CancellationToken token ) {
|
||||
return mImpl->run<URI>( uri, std::move( token ),
|
||||
[]( const std::shared_ptr<ArchiveIndex>& index,
|
||||
const std::string& member, const URI& uri,
|
||||
CancellationToken ) -> VFSResult<URI> {
|
||||
std::string resolved;
|
||||
auto found = index->find( member, uri, true, &resolved );
|
||||
if ( !found )
|
||||
return Err( std::move( found ).error() );
|
||||
auto backing = ArchiveProvider::archiveURI( uri );
|
||||
return Ok( ArchiveProvider::makeURI( backing.value(), resolved ) );
|
||||
} );
|
||||
}
|
||||
|
||||
}} // namespace EE::VFS
|
||||
@@ -4,6 +4,7 @@
|
||||
#include <array>
|
||||
#include <eepp/core/containers.hpp>
|
||||
#include <eepp/system/zip.hpp>
|
||||
#include <eepp/vfs/detail/archivelocation.hpp>
|
||||
#include <eepp/vfs/detail/memoryhandles.hpp>
|
||||
#include <eepp/vfs/providers/localprovider.hpp>
|
||||
#include <limits>
|
||||
@@ -48,73 +49,6 @@ VFSError readerError( const Zip& reader, const URI& uri ) {
|
||||
return VFSError( code, uri, reader.errorMessage() );
|
||||
}
|
||||
|
||||
/** Archive paths are untrusted. Reject names that could leave a copy destination or turn into
|
||||
* native Windows paths/streams; do not resolve these names into aliases of safe entries. */
|
||||
bool safeEntry( std::string_view name ) {
|
||||
if ( name.empty() || name.size() > 4096 || name.front() == '/' ||
|
||||
name.find_first_of( std::string_view( "\\:\0", 3 ) ) != std::string_view::npos )
|
||||
return false;
|
||||
size_t start = 0;
|
||||
while ( start < name.size() ) {
|
||||
size_t end = name.find( '/', start );
|
||||
if ( end == std::string_view::npos )
|
||||
end = name.size();
|
||||
auto part = name.substr( start, end - start );
|
||||
// Windows trims trailing dots/spaces; an all-dot/space segment can become a parent.
|
||||
if ( part.empty() || part.find_first_not_of( ". " ) == std::string_view::npos )
|
||||
return false;
|
||||
start = end + 1;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
/** Parsed archive identity and normalized path within it. */
|
||||
struct ZipAddress {
|
||||
URI archive;
|
||||
std::string member;
|
||||
|
||||
static VFSResult<ZipAddress> parse( const URI& uri ) {
|
||||
if ( uri.getScheme() != "zip" || uri.getAuthority() != uri.getHost() ||
|
||||
!uri.getFragment().empty() || uri.getPath().find( '\0' ) != std::string::npos )
|
||||
return Err( VFSError( VFSErrorCode::InvalidURI, uri, "Invalid ZIP location" ) );
|
||||
if ( !uri.getRawQuery().empty() )
|
||||
return Err( VFSError( VFSErrorCode::Unsupported, uri, "Unsupported ZIP backing URI" ) );
|
||||
const auto& path = uri.getPath();
|
||||
if ( path.empty() || path.front() != '/' )
|
||||
return Err( VFSError( VFSErrorCode::InvalidURI, uri, "ZIP path must be absolute" ) );
|
||||
std::string backing;
|
||||
backing.reserve( path.size() );
|
||||
size_t boundary = std::string::npos;
|
||||
for ( size_t i = 0; i < path.size(); ++i ) {
|
||||
if ( path[i] == '!' ) {
|
||||
if ( i + 1 < path.size() && path[i + 1] == '!' ) {
|
||||
backing += '!';
|
||||
++i;
|
||||
continue;
|
||||
}
|
||||
if ( i + 1 == path.size() || path[i + 1] == '/' ) {
|
||||
boundary = i + 1;
|
||||
break;
|
||||
}
|
||||
}
|
||||
backing += path[i];
|
||||
}
|
||||
if ( boundary == std::string::npos || backing.empty() || backing.back() == '/' )
|
||||
return Err(
|
||||
VFSError( VFSErrorCode::InvalidURI, uri, "ZIP path needs archive.zip!/member" ) );
|
||||
ZipAddress result;
|
||||
result.archive.setScheme( "file" );
|
||||
result.archive.setHost( uri.getHost() );
|
||||
result.archive.setRawPath( std::move( backing ) );
|
||||
URI member;
|
||||
member.setScheme( "zip" );
|
||||
member.setRawPath( boundary == path.size() ? "/" : path.substr( boundary ) );
|
||||
member.normalize();
|
||||
result.member = member.getPath();
|
||||
return Ok( std::move( result ) );
|
||||
}
|
||||
};
|
||||
|
||||
struct ZipNode {
|
||||
VFSFileInfo info;
|
||||
std::vector<size_t> children;
|
||||
@@ -163,7 +97,7 @@ struct ZipArchive {
|
||||
if ( !reader.entry( entry, metadata ) )
|
||||
return Err( readerError( reader, uri ) );
|
||||
std::string_view name( metadata.name );
|
||||
if ( !safeEntry( name ) )
|
||||
if ( !detail::ArchiveLocation::safeEntry( name ) )
|
||||
continue;
|
||||
const bool directory = name.back() == '/';
|
||||
if ( directory )
|
||||
@@ -369,7 +303,7 @@ class ZipProviderImpl : public std::enable_shared_from_this<ZipProviderImpl> {
|
||||
VFSTask<T> run( const URI& uri, CancellationToken token, F&& fn ) {
|
||||
if ( token.isCancellationRequested() )
|
||||
return readyError<T>( cancelledError( uri ) );
|
||||
auto address = ZipAddress::parse( uri );
|
||||
auto address = detail::ArchiveLocation::parse( uri, "zip" );
|
||||
if ( !address )
|
||||
return readyError<T>( std::move( address ).error() );
|
||||
auto native = LocalProvider::toNativePath( address.value().archive );
|
||||
@@ -471,43 +405,18 @@ ZipProvider::ZipProvider( std::shared_ptr<LocalIOScheduler> scheduler ) :
|
||||
ZipProvider::~ZipProvider() = default;
|
||||
|
||||
URI ZipProvider::makeURI( const URI& archive, std::string_view member ) {
|
||||
if ( archive.getScheme() != "file" || !archive.getRawQuery().empty() ||
|
||||
!archive.getFragment().empty() || archive.getAuthority() != archive.getHost() ||
|
||||
archive.getPath().empty() || archive.getPath().front() != '/' ||
|
||||
archive.getPath().find( '\0' ) != std::string::npos )
|
||||
return {};
|
||||
URI uri;
|
||||
uri.setScheme( "zip" );
|
||||
uri.setHost( archive.getHost() );
|
||||
std::string path;
|
||||
path.reserve( archive.getPath().size() + member.size() + 2 );
|
||||
// Escape literal bangs in the backing path, so even a directory named "packages!" is
|
||||
// unambiguous. Only the first unescaped "!/" separates the archive from its member.
|
||||
for ( char c : archive.getPath() ) {
|
||||
path += c;
|
||||
if ( c == '!' )
|
||||
path += '!';
|
||||
}
|
||||
URI entry;
|
||||
entry.setScheme( "zip" );
|
||||
entry.setRawPath( member.empty() || member.front() != '/' ? "/" + std::string( member )
|
||||
: std::string( member ) );
|
||||
entry.normalize();
|
||||
path += '!';
|
||||
path += entry.getPath();
|
||||
uri.setRawPath( std::move( path ) );
|
||||
return uri;
|
||||
return detail::ArchiveLocation::make( "zip", archive, member );
|
||||
}
|
||||
|
||||
VFSResult<URI> ZipProvider::archiveURI( const URI& uri ) {
|
||||
auto address = ZipAddress::parse( uri );
|
||||
auto address = detail::ArchiveLocation::parse( uri, "zip" );
|
||||
if ( !address )
|
||||
return Err( std::move( address ).error() );
|
||||
return Ok( std::move( address ).value().archive );
|
||||
}
|
||||
|
||||
VFSResult<std::string> ZipProvider::memberPath( const URI& uri ) {
|
||||
auto address = ZipAddress::parse( uri );
|
||||
auto address = detail::ArchiveLocation::parse( uri, "zip" );
|
||||
if ( !address )
|
||||
return Err( std::move( address ).error() );
|
||||
return Ok( std::move( address ).value().member );
|
||||
|
||||
@@ -198,7 +198,8 @@ VFSResult<bool> TransferEngine::copyNode( const Node& node, const URI& source,
|
||||
if ( existing ) {
|
||||
if ( existing.value().isDirectory() )
|
||||
return Err( VFSError( VFSErrorCode::IsDirectory, destination ) );
|
||||
if ( mOptions.overwrite == OverwritePolicy::Fail )
|
||||
if ( mOptions.overwrite == OverwritePolicy::Fail ||
|
||||
mOptions.overwrite == OverwritePolicy::MergeDirectories )
|
||||
return Err( VFSError( VFSErrorCode::AlreadyExists, destination ) );
|
||||
if ( mOptions.overwrite == OverwritePolicy::Skip ) {
|
||||
++mResult.filesSkipped;
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
#include <algorithm>
|
||||
#include <eepp/vfs/detail/await.hpp>
|
||||
#include <eepp/vfs/detail/filewriter.hpp>
|
||||
#include <eepp/vfs/providers/archiveprovider.hpp>
|
||||
#include <eepp/vfs/providers/localprovider.hpp>
|
||||
#include <eepp/vfs/providers/resourceprovider.hpp>
|
||||
#include <eepp/vfs/providers/zipprovider.hpp>
|
||||
@@ -71,6 +72,7 @@ VirtualFileSystem::~VirtualFileSystem() {
|
||||
void VirtualFileSystem::registerDefaultProviders() {
|
||||
registerProvider( std::make_shared<LocalProvider>( mLocalIO ) );
|
||||
registerProvider( std::make_shared<ZipProvider>( mLocalIO ) );
|
||||
registerProvider( std::make_shared<ArchiveProvider>( mLocalIO ) );
|
||||
registerProvider( std::make_shared<ResourceProvider>( mIOPool ) );
|
||||
// Network mounts are polled through the VFS (whose calls time out), not by efsw.
|
||||
auto polling = std::make_shared<PollingWatchBackend>( *this );
|
||||
|
||||
@@ -0,0 +1,553 @@
|
||||
#include <eepp/config.hpp>
|
||||
#if EE_PLATFORM == EE_PLATFORM_WIN
|
||||
#ifndef NOMINMAX
|
||||
#define NOMINMAX
|
||||
#endif
|
||||
#include <windows.h>
|
||||
#include <winioctl.h>
|
||||
#endif
|
||||
|
||||
#include "../../tools/eefm/location.hpp"
|
||||
#include "utest.hpp"
|
||||
#include "vfs_test_utils.hpp"
|
||||
#include <array>
|
||||
#include <cstdio>
|
||||
#include <eepp/system/base64.hpp>
|
||||
#include <eepp/system/compression.hpp>
|
||||
#include <eepp/system/iostreamfile.hpp>
|
||||
#include <eepp/system/iostreammemory.hpp>
|
||||
#include <eepp/system/packfileindex.hpp>
|
||||
#include <eepp/vfs/providers/archiveprovider.hpp>
|
||||
#include <filesystem>
|
||||
#include <fstream>
|
||||
|
||||
using namespace EE;
|
||||
using namespace EE::System;
|
||||
using namespace EE::VFS;
|
||||
using namespace vfstest;
|
||||
|
||||
namespace {
|
||||
|
||||
// Independent USTAR fixture writer. GNU/PAX extensions are explicit entries so the tests do
|
||||
// not depend on a system tar command or on the same reader being used to produce fixtures.
|
||||
struct TarFixture {
|
||||
std::string bytes;
|
||||
|
||||
void entry( std::string_view name, std::string_view data, char type = '0',
|
||||
std::string_view link = {}, std::optional<Uint64> headerSize = {} ) {
|
||||
std::array<char, 512> header{};
|
||||
std::copy_n( name.data(), std::min<size_t>( name.size(), 100 ), header.data() );
|
||||
std::snprintf( header.data() + 100, 8, "%07o", type == '5' ? 0755 : 0644 );
|
||||
std::snprintf( header.data() + 108, 8, "%07o", 1000 );
|
||||
std::snprintf( header.data() + 116, 8, "%07o", 1000 );
|
||||
std::snprintf( header.data() + 124, 12, "%011llo",
|
||||
static_cast<unsigned long long>( headerSize.value_or( data.size() ) &
|
||||
077777777777ULL ) );
|
||||
std::snprintf( header.data() + 136, 12, "%011o", 1700000000 );
|
||||
std::fill_n( header.data() + 148, 8, ' ' );
|
||||
header[156] = type;
|
||||
std::copy_n( link.data(), std::min<size_t>( link.size(), 100 ), header.data() + 157 );
|
||||
std::copy_n( "ustar", 5, header.data() + 257 );
|
||||
header[263] = header[264] = '0';
|
||||
unsigned sum = 0;
|
||||
for ( unsigned char byte : header )
|
||||
sum += byte;
|
||||
std::snprintf( header.data() + 148, 7, "%06o", sum );
|
||||
header[155] = ' ';
|
||||
bytes.append( header.data(), header.size() );
|
||||
bytes.append( data );
|
||||
bytes.append( ( 512 - data.size() % 512 ) % 512, '\0' );
|
||||
}
|
||||
|
||||
bool write( const std::string& path, bool gzip = false ) const {
|
||||
std::string data = bytes + std::string( 1024, '\0' );
|
||||
if ( !gzip )
|
||||
return FileSystem::fileWrite( path, data );
|
||||
IOStreamMemory input( static_cast<const char*>( data.data() ), data.size() );
|
||||
IOStreamFile output( path, "wb" );
|
||||
return output.isOpen() &&
|
||||
Compression::compress( output, input, Compression::MODE_GZIP ) == Compression::OK;
|
||||
}
|
||||
|
||||
static std::string pax( std::string value ) {
|
||||
size_t size = value.size() + 3;
|
||||
while ( std::to_string( size ).size() + value.size() + 2 != size )
|
||||
size = std::to_string( size ).size() + value.size() + 2;
|
||||
return std::to_string( size ) + " " + value + "\n";
|
||||
}
|
||||
};
|
||||
|
||||
bool available( int* utest_result, VirtualFileSystem& vfs ) {
|
||||
auto provider = std::dynamic_pointer_cast<ArchiveProvider>( vfs.provider( "archive" ) );
|
||||
if ( !provider ) {
|
||||
*utest_result = UTEST_TEST_FAILURE;
|
||||
return false;
|
||||
}
|
||||
auto support = waitFor( provider->support() );
|
||||
if ( !support ) {
|
||||
if ( !support.error().is( VFSErrorCode::Unsupported ) )
|
||||
*utest_result = UTEST_TEST_FAILURE;
|
||||
UTEST_PRINT_INFO(
|
||||
"libarchive runtime unavailable; integration requires an installed library" );
|
||||
if ( *utest_result != UTEST_TEST_FAILURE )
|
||||
*utest_result = UTEST_TEST_SKIPPED;
|
||||
return false;
|
||||
}
|
||||
if ( support.value().version.empty() ||
|
||||
std::find( support.value().formats.begin(), support.value().formats.end(), "tar" ) ==
|
||||
support.value().formats.end() ) {
|
||||
*utest_result = UTEST_TEST_FAILURE;
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
UTEST( ArchiveProvider, locationsAndEefmExtensions ) {
|
||||
TempDir temp;
|
||||
const URI backing = URI::fromFSPath( temp.path + "Archive! #%.tar.gz" );
|
||||
const URI root = ArchiveProvider::makeURI( backing );
|
||||
const URI member = root.withAppendedSegment( "dir" ).withAppendedSegment( "entry! #%" );
|
||||
EXPECT_TRUE( ArchiveProvider::archiveURI( URI( member.toString() ) ).value() == backing );
|
||||
EXPECT_STDSTREQ( "/dir/entry! #%", ArchiveProvider::memberPath( member ).value() );
|
||||
EXPECT_TRUE( eefm::Location::parse( eefm::Location::text( member ).toUtf8() ) == member );
|
||||
EXPECT_TRUE( eefm::Location::parent( root ) == backing.getParent() );
|
||||
EXPECT_STDSTREQ( backing.getLastPathSegment(), eefm::Location::entryName( root ) );
|
||||
EXPECT_STDSTREQ( backing.getLastPathSegment(), eefm::Location::displayName( root ).toUtf8() );
|
||||
EXPECT_TRUE( eefm::Location::parent( member ) == root.withAppendedSegment( "dir" ) );
|
||||
for ( const char* name : { "archive.tar", "archive.TAR.GZ", "archive.tgz", "archive.tar.xz",
|
||||
"archive.tar.bz2", "archive.7z", "archive.rar", "package.deb" } ) {
|
||||
const URI file = temp.uri().withAppendedSegment( name );
|
||||
EXPECT_TRUE( eefm::Location::archiveRoot( file ) == ArchiveProvider::makeURI( file ) );
|
||||
}
|
||||
EXPECT_TRUE( ArchiveProvider::makeURI( URI( "sftp://host/a.tar" ) ).empty() );
|
||||
EXPECT_EQ( VFSErrorCode::InvalidURI,
|
||||
codeOf( ArchiveProvider::archiveURI( URI( "archive:///no-boundary" ) ) ) );
|
||||
}
|
||||
|
||||
UTEST( ArchiveProvider, largeSparseTarUses64BitInputOffsets ) {
|
||||
TempDir temp;
|
||||
VirtualFileSystem vfs;
|
||||
if ( !available( utest_result, vfs ) )
|
||||
return;
|
||||
const Uint64 largeSize = Uint64( 3 ) * 1024 * 1024 * 1024;
|
||||
TarFixture header;
|
||||
header.entry( "large", "", '0', {}, largeSize );
|
||||
const std::string path = temp.path + "large.tar";
|
||||
ASSERT_TRUE( FileSystem::fileWrite( path, header.bytes ) );
|
||||
#if EE_PLATFORM == EE_PLATFORM_WIN
|
||||
// Mark sparse before extending: the test must not allocate 3 GiB on disk.
|
||||
HANDLE file = CreateFileW( String::fromUtf8( path ).toWideString().c_str(), GENERIC_WRITE,
|
||||
FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE, nullptr,
|
||||
OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, nullptr );
|
||||
ASSERT_TRUE( file != INVALID_HANDLE_VALUE );
|
||||
DWORD returned = 0;
|
||||
bool sparse =
|
||||
DeviceIoControl( file, FSCTL_SET_SPARSE, nullptr, 0, nullptr, 0, &returned, nullptr ) != 0;
|
||||
CloseHandle( file );
|
||||
if ( !sparse )
|
||||
UTEST_SKIP( "Temporary filesystem does not support sparse files" );
|
||||
#endif
|
||||
TarFixture tail;
|
||||
tail.entry( "tail", "payload" );
|
||||
{
|
||||
const std::filesystem::path nativePath( std::u8string( path.begin(), path.end() ) );
|
||||
std::fstream stream( nativePath, std::ios::binary | std::ios::in | std::ios::out );
|
||||
stream.seekp( static_cast<std::streamoff>( 512 + largeSize ) );
|
||||
const std::string end = tail.bytes + std::string( 1024, '\0' );
|
||||
stream.write( end.data(), end.size() );
|
||||
ASSERT_TRUE( stream.good() );
|
||||
}
|
||||
const URI root = ArchiveProvider::makeURI( URI::fromFSPath( path ) );
|
||||
auto listed = waitFor( vfs.listAll( root ) );
|
||||
ASSERT_TRUE( listed.hasValue() );
|
||||
EXPECT_EQ( 2u, listed.value().size() );
|
||||
EXPECT_STDSTREQ( "payload", readString( vfs, root.withAppendedSegment( "tail" ) ) );
|
||||
EXPECT_EQ(
|
||||
largeSize,
|
||||
waitFor( vfs.stat( root.withAppendedSegment( "large" ) ) ).value().size().value_or( 0 ) );
|
||||
}
|
||||
|
||||
UTEST( ArchiveProvider, missingLibraryIsOptionalAndReadOnly ) {
|
||||
auto pool = ThreadPool::createShared( 1 );
|
||||
auto scheduler = std::make_shared<LocalIOScheduler>( pool, Seconds( 1 ) );
|
||||
TempDir temp;
|
||||
ArchiveProvider provider( scheduler, temp.path + "missing-libarchive" );
|
||||
EXPECT_EQ( VFSErrorCode::Unsupported, codeOf( waitFor( provider.support() ) ) );
|
||||
const URI file =
|
||||
ArchiveProvider::makeURI( URI::fromFSPath( temp.path + "archive.tar" ), "/file" );
|
||||
EXPECT_EQ( VFSErrorCode::Unsupported, codeOf( waitFor( provider.stat( file, {}, {} ) ) ) );
|
||||
EXPECT_EQ( VFSErrorCode::ReadOnly,
|
||||
codeOf( waitFor( provider.open( file, OpenFlags::Write, {} ) ) ) );
|
||||
EXPECT_EQ( VFSErrorCode::ReadOnly, codeOf( waitFor( provider.remove( file, {}, {} ) ) ) );
|
||||
CancellationSource cancelled;
|
||||
cancelled.cancel();
|
||||
EXPECT_EQ( VFSErrorCode::Cancelled,
|
||||
codeOf( waitFor( provider.support( cancelled.token() ) ) ) );
|
||||
EXPECT_EQ( VFSErrorCode::Cancelled,
|
||||
codeOf( waitFor( provider.stat( file, {}, cancelled.token() ) ) ) );
|
||||
}
|
||||
|
||||
UTEST( ArchiveProvider, tarAndGzipListStreamSeekAndCopy ) {
|
||||
TempDir temp;
|
||||
VirtualFileSystem vfs;
|
||||
if ( !available( utest_result, vfs ) )
|
||||
return;
|
||||
TarFixture fixture;
|
||||
std::string content( 200000, 'x' );
|
||||
for ( size_t i = 0; i < content.size(); ++i )
|
||||
content[i] = static_cast<char>( 'a' + i % 26 );
|
||||
fixture.entry( "./dir/file.txt", content );
|
||||
fixture.entry( "empty/", "", '5' );
|
||||
fixture.entry( "zero", "" );
|
||||
const auto generation = PackFileIndex::instance()->getGeneration();
|
||||
for ( bool gzip : { false, true } ) {
|
||||
const std::string path = temp.path + ( gzip ? "archivo-ñ-雪.tar.gz" : "archive.tar" );
|
||||
ASSERT_TRUE( fixture.write( path, gzip ) );
|
||||
const URI root = ArchiveProvider::makeURI( URI::fromFSPath( path ) );
|
||||
auto list = waitFor( vfs.listAll( root ) );
|
||||
ASSERT_TRUE( list.hasValue() );
|
||||
EXPECT_TRUE( sortedNames( list.value() ) ==
|
||||
std::vector<std::string>( { "dir", "empty", "zero" } ) );
|
||||
EXPECT_TRUE(
|
||||
waitFor( vfs.listAll( root.withAppendedSegment( "empty" ) ) ).value().empty() );
|
||||
const URI file = root.withAppendedSegment( "dir" ).withAppendedSegment( "file.txt" );
|
||||
auto opened = waitFor( vfs.open( file ) );
|
||||
ASSERT_TRUE( opened.hasValue() );
|
||||
auto handle = std::move( opened ).value();
|
||||
std::array<Uint8, 32> bytes{};
|
||||
for ( Uint64 offset : { 150000u, 0u, 199990u, 5u } ) {
|
||||
auto got = waitFor( handle.read( offset, bytes ) );
|
||||
ASSERT_TRUE( got.hasValue() );
|
||||
EXPECT_EQ( std::min<size_t>( bytes.size(), content.size() - offset ), got.value() );
|
||||
EXPECT_TRUE( std::string( bytes.begin(), bytes.begin() + got.value() ) ==
|
||||
content.substr( offset, got.value() ) );
|
||||
}
|
||||
EXPECT_FALSE( handle.capabilities().has( VFSCapability::RandomAccess ) );
|
||||
EXPECT_TRUE( waitFor( handle.close() ).hasValue() );
|
||||
EXPECT_STDSTREQ( "", readString( vfs, root.withAppendedSegment( "zero" ) ) );
|
||||
const URI destination = URI::fromFSPath( temp.path + ( gzip ? "gzip-copy" : "tar-copy" ) );
|
||||
EXPECT_TRUE(
|
||||
waitFor( vfs.copy( root.withAppendedSegment( "dir" ), destination ).completion() )
|
||||
.hasValue() );
|
||||
EXPECT_TRUE( content == readString( vfs, destination.withAppendedSegment( "file.txt" ) ) );
|
||||
const URI refused = destination.withAppendedSegment( "must-not-exist" );
|
||||
EXPECT_EQ( VFSErrorCode::Unsupported,
|
||||
codeOf( waitFor( vfs.move( file, refused ).completion() ) ) );
|
||||
EXPECT_EQ( VFSErrorCode::NotFound, codeOf( waitFor( vfs.stat( refused ) ) ) );
|
||||
}
|
||||
EXPECT_EQ( generation, PackFileIndex::instance()->getGeneration() );
|
||||
}
|
||||
|
||||
UTEST( ArchiveProvider, extractRootIntoExistingFolder ) {
|
||||
TempDir temp;
|
||||
VirtualFileSystem vfs;
|
||||
if ( !available( utest_result, vfs ) )
|
||||
return;
|
||||
TarFixture tar;
|
||||
tar.entry( ".hidden", "hidden" );
|
||||
tar.entry( "sub/file.txt", "contents" );
|
||||
tar.entry( "empty/", "", '5' );
|
||||
ASSERT_TRUE( tar.write( temp.path + "archive.tar.gz", true ) );
|
||||
const URI root =
|
||||
eefm::Location::archiveRoot( temp.uri().withAppendedSegment( "archive.tar.gz" ) );
|
||||
CopyOptions options;
|
||||
options.overwrite = OverwritePolicy::MergeDirectories;
|
||||
auto copied = waitFor( vfs.copy( root, temp.uri(), options ).completion() );
|
||||
ASSERT_TRUE( copied );
|
||||
EXPECT_FALSE( copied.value().partial() );
|
||||
EXPECT_EQ( 2u, copied.value().filesTransferred );
|
||||
EXPECT_STDSTREQ( "hidden", readString( vfs, temp.uri().withAppendedSegment( ".hidden" ) ) );
|
||||
EXPECT_STDSTREQ( "contents",
|
||||
readString( vfs, temp.uri().withAppendedSegment( "sub" ).withAppendedSegment(
|
||||
"file.txt" ) ) );
|
||||
auto empty = waitFor( vfs.stat( temp.uri().withAppendedSegment( "empty" ) ) );
|
||||
ASSERT_TRUE( empty );
|
||||
EXPECT_TRUE( empty.value().isDirectory() );
|
||||
EXPECT_STDSTREQ( "hidden", readString( vfs, root.withAppendedSegment( ".hidden" ) ) );
|
||||
}
|
||||
|
||||
UTEST( ArchiveProvider, concurrentReadersAndSequentialDirectoryCopy ) {
|
||||
TempDir temp;
|
||||
VirtualFileSystem vfs;
|
||||
if ( !available( utest_result, vfs ) )
|
||||
return;
|
||||
TarFixture fixture;
|
||||
for ( int i = 0; i < 64; ++i )
|
||||
fixture.entry( "dir/file-" + std::to_string( i ), "contents " + std::to_string( i ) );
|
||||
const std::string path = temp.path + "sequential.tar.gz";
|
||||
ASSERT_TRUE( fixture.write( path, true ) );
|
||||
const URI root = ArchiveProvider::makeURI( URI::fromFSPath( path ) );
|
||||
const URI directory = root.withAppendedSegment( "dir" );
|
||||
auto first = waitFor( vfs.open( directory.withAppendedSegment( "file-0" ) ) );
|
||||
auto second = waitFor( vfs.open( directory.withAppendedSegment( "file-1" ) ) );
|
||||
ASSERT_TRUE( first.hasValue() );
|
||||
ASSERT_TRUE( second.hasValue() );
|
||||
auto a = std::move( first ).value();
|
||||
auto b = std::move( second ).value();
|
||||
std::array<Uint8, 4> bytes{};
|
||||
EXPECT_EQ( 4u, waitFor( a.read( 2, bytes ) ).value() );
|
||||
EXPECT_TRUE( waitFor( b.close() ).hasValue() );
|
||||
// Closing one reader must not change another's position or its backward-seek behavior.
|
||||
EXPECT_EQ( 4u, waitFor( a.read( 0, bytes ) ).value() );
|
||||
EXPECT_TRUE( std::string( bytes.begin(), bytes.end() ) == "cont" );
|
||||
EXPECT_TRUE( waitFor( a.close() ).hasValue() );
|
||||
const URI output = temp.uri().withAppendedSegment( "copied" );
|
||||
auto copied = waitFor( vfs.copy( directory, output ).completion() );
|
||||
ASSERT_TRUE( copied.hasValue() );
|
||||
EXPECT_EQ( 64u, copied.value().filesTransferred );
|
||||
for ( int i = 0; i < 64; ++i ) {
|
||||
EXPECT_STDSTREQ(
|
||||
"contents " + std::to_string( i ),
|
||||
readString( vfs, output.withAppendedSegment( "file-" + std::to_string( i ) ) ) );
|
||||
}
|
||||
}
|
||||
|
||||
UTEST( ArchiveProvider, gnuAndPaxNamesAndSizeOverrides ) {
|
||||
TempDir temp;
|
||||
VirtualFileSystem vfs;
|
||||
if ( !available( utest_result, vfs ) )
|
||||
return;
|
||||
TarFixture fixture;
|
||||
const std::string longName = "gnu/" + std::string( 180, 'n' );
|
||||
fixture.entry( "././@LongLink", longName + '\0', 'L' );
|
||||
fixture.entry( "placeholder", "gnu content" );
|
||||
const std::string paxName = "pax/" + std::string( 180, 'p' );
|
||||
fixture.entry( "PaxHeader", TarFixture::pax( "path=" + paxName ) + TarFixture::pax( "size=11" ),
|
||||
'x' );
|
||||
fixture.entry( "placeholder", "pax content", '0', {}, 0 );
|
||||
fixture.entry( "tail", "tail content" );
|
||||
const auto path = temp.path + "extensions.tar.gz";
|
||||
ASSERT_TRUE( fixture.write( path, true ) );
|
||||
const URI root = ArchiveProvider::makeURI( URI::fromFSPath( path ) );
|
||||
EXPECT_STDSTREQ( "gnu content", readString( vfs, ArchiveProvider::makeURI(
|
||||
URI::fromFSPath( path ), longName ) ) );
|
||||
EXPECT_STDSTREQ( "pax content", readString( vfs, ArchiveProvider::makeURI(
|
||||
URI::fromFSPath( path ), paxName ) ) );
|
||||
EXPECT_STDSTREQ( "tail content", readString( vfs, root.withAppendedSegment( "tail" ) ) );
|
||||
}
|
||||
|
||||
UTEST( ArchiveProvider, runtimeFiltersAndSevenZip ) {
|
||||
TempDir temp;
|
||||
VirtualFileSystem vfs;
|
||||
if ( !available( utest_result, vfs ) )
|
||||
return;
|
||||
auto provider = std::dynamic_pointer_cast<ArchiveProvider>( vfs.provider( "archive" ) );
|
||||
auto support = waitFor( provider->support() );
|
||||
ASSERT_TRUE( support.hasValue() );
|
||||
struct Fixture {
|
||||
const char* name;
|
||||
const char* filter;
|
||||
const char* format;
|
||||
const char* data;
|
||||
};
|
||||
// Generated independently using Python tarfile/bz2/lzma and libarchive's 7zip writer.
|
||||
for ( const auto& fixture :
|
||||
{ Fixture{ "archive.tar.bz2", "bzip2", "tar",
|
||||
"QlpoOTFBWSZTWRGkBHYAAG97gMmAAAhAAEWAAARnJN4gCAggAFQ0jUxDATR6maCSKaGmg0GTQfTiO"
|
||||
"hBOCEIlVsTWUcECGBjRxOwjZ0UMctBnNo9/TCY+TXCvVVOZEQGxdyRThQkBGkBHYA==" },
|
||||
Fixture{ "archive.tar.xz", "xz", "tar",
|
||||
"/Td6WFoAAATm1rRGAgAhARYAAAB0L+Wj4Cf/AF1dADMaSda2OCGvAe5uBXM7wPyAzzewSHXXbqSo/"
|
||||
"hdgBa5T3GsxIACRvOjdMLsxnrOfIgBYUuJzInp0sdefWq+cGCAurmuD6EUsI9irRPY0Dg3L73/"
|
||||
"zP4rTIqzcpMEqAAAAAAB5/rNswT5g4QABeYBQAAAA5DeTlbHEZ/sCAAAAAARZWg==" },
|
||||
Fixture{ "archive.7z", "none", "7zip",
|
||||
"N3q8ryccAAP0BzRLBwAAAAAAAAA0AAAAAAAAAF5/"
|
||||
"nFVwYXlsb2FkAQQGAAEJBwAHCwEAAQEADAcACAoBFWosQgAABQERCwBmAGkAbABlAAAAFQYBACGAA"
|
||||
"IAAAA==" } } ) {
|
||||
if ( std::find( support.value().filters.begin(), support.value().filters.end(),
|
||||
fixture.filter ) == support.value().filters.end() ||
|
||||
std::find( support.value().formats.begin(), support.value().formats.end(),
|
||||
fixture.format ) == support.value().formats.end() )
|
||||
continue;
|
||||
const std::string path = temp.path + fixture.name;
|
||||
std::string bytes;
|
||||
Base64::decode( fixture.data, bytes );
|
||||
ASSERT_TRUE( FileSystem::fileWrite( path, bytes ) );
|
||||
EXPECT_STDSTREQ( "payload", readString( vfs, ArchiveProvider::makeURI(
|
||||
URI::fromFSPath( path ), "/file" ) ) );
|
||||
}
|
||||
}
|
||||
|
||||
UTEST( ArchiveProvider, recognizedFormatCanHaveAnUnavailableCodec ) {
|
||||
TempDir temp;
|
||||
VirtualFileSystem vfs;
|
||||
if ( !available( utest_result, vfs ) )
|
||||
return;
|
||||
auto provider = std::dynamic_pointer_cast<ArchiveProvider>( vfs.provider( "archive" ) );
|
||||
auto support = waitFor( provider->support() );
|
||||
ASSERT_TRUE( support.hasValue() );
|
||||
if ( std::find( support.value().formats.begin(), support.value().formats.end(), "7zip" ) ==
|
||||
support.value().formats.end() )
|
||||
UTEST_SKIP( "7zip reader not available" );
|
||||
std::string bytes;
|
||||
Base64::decode(
|
||||
"N3q8ryccAANoCeHQEQAAAAAAAAA8AAAAAAAAAMgYqcgAOBhLmXUPRgxO4E///"
|
||||
"d8QAAEEBgABCREABwsBAAEjAwEBBV0AAIAADAcACAoBFWosQgAABQERCwBmAGkAbABlAAAAFQYBACGAAIAAAA==",
|
||||
bytes );
|
||||
const std::string path = temp.path + "lzma.7z";
|
||||
ASSERT_TRUE( FileSystem::fileWrite( path, bytes ) );
|
||||
auto result =
|
||||
waitFor( vfs.readAll( ArchiveProvider::makeURI( URI::fromFSPath( path ), "/file" ) ) );
|
||||
if ( std::find( support.value().filters.begin(), support.value().filters.end(), "lzma" ) ==
|
||||
support.value().filters.end() ) {
|
||||
EXPECT_TRUE( result.hasError() );
|
||||
} else {
|
||||
ASSERT_TRUE( result.hasValue() );
|
||||
EXPECT_STDSTREQ( "payload", std::string( result.value().begin(), result.value().end() ) );
|
||||
}
|
||||
}
|
||||
|
||||
UTEST( ArchiveProvider, aggregateMetadataIsBounded ) {
|
||||
TempDir temp;
|
||||
VirtualFileSystem vfs;
|
||||
if ( !available( utest_result, vfs ) )
|
||||
return;
|
||||
TarFixture fixture;
|
||||
std::string nested;
|
||||
for ( int i = 0; i < 100; ++i )
|
||||
nested += "directory-name-long-enough-for-budget/";
|
||||
for ( int i = 0; i < 512; ++i ) {
|
||||
fixture.entry( "././@LongLink", std::to_string( i ) + "/" + nested + "file" + '\0', 'L' );
|
||||
fixture.entry( "file", "" );
|
||||
}
|
||||
const std::string path = temp.path + "metadata.tar";
|
||||
ASSERT_TRUE( fixture.write( path ) );
|
||||
EXPECT_EQ(
|
||||
VFSErrorCode::InvalidOperation,
|
||||
codeOf( waitFor( vfs.listAll( ArchiveProvider::makeURI( URI::fromFSPath( path ) ) ) ) ) );
|
||||
}
|
||||
|
||||
UTEST( ArchiveProvider, unsafePathsLinksAndCollisionsStayInsideArchive ) {
|
||||
TempDir temp;
|
||||
VirtualFileSystem vfs;
|
||||
if ( !available( utest_result, vfs ) )
|
||||
return;
|
||||
TarFixture fixture;
|
||||
fixture.entry( "dir/file", "inside" );
|
||||
fixture.entry( "dir/file", "duplicate" );
|
||||
fixture.entry( "collision", "hidden" );
|
||||
fixture.entry( "collision/child", "child" );
|
||||
fixture.entry( "../evil", "evil" );
|
||||
fixture.entry( "/absolute", "evil" );
|
||||
fixture.entry( "C:drive", "evil" );
|
||||
fixture.entry( "back\\slash", "evil" );
|
||||
fixture.entry( "file-link", "", '2', "dir/file" );
|
||||
fixture.entry( ".hidden-link", "", '2', "dir/file" );
|
||||
fixture.entry( "dir-link", "", '2', "dir" );
|
||||
fixture.entry( "hard-link", "", '1', "dir/file" );
|
||||
fixture.entry( "cycle", "", '2', "cycle" );
|
||||
fixture.entry( "outside", "", '2', "../../secret" );
|
||||
fixture.entry( "pipe", "", '6' );
|
||||
const auto path = temp.path + "links.tar";
|
||||
ASSERT_TRUE( fixture.write( path ) );
|
||||
const URI root = ArchiveProvider::makeURI( URI::fromFSPath( path ) );
|
||||
auto listed = waitFor( vfs.listAll( root ) );
|
||||
ASSERT_TRUE( listed.hasValue() );
|
||||
EXPECT_EQ( 9u, listed.value().size() );
|
||||
const URI hidden = root.withAppendedSegment( ".hidden-link" );
|
||||
auto hiddenInfo = waitFor( vfs.stat( hidden ) );
|
||||
ASSERT_TRUE( hiddenInfo.hasValue() );
|
||||
EXPECT_TRUE( hiddenInfo.value().hidden );
|
||||
EXPECT_STDSTREQ( ".hidden-link", hiddenInfo.value().name );
|
||||
EXPECT_TRUE( waitFor( vfs.canonicalize( hidden ) ).value() ==
|
||||
root.withAppendedSegment( "dir" ).withAppendedSegment( "file" ) );
|
||||
EXPECT_STDSTREQ( "inside", readString( vfs, root.withAppendedSegment( "file-link" ) ) );
|
||||
EXPECT_STDSTREQ( "inside", readString( vfs, root.withAppendedSegment( "hard-link" ) ) );
|
||||
EXPECT_STDSTREQ(
|
||||
"inside",
|
||||
readString( vfs, root.withAppendedSegment( "dir-link" ).withAppendedSegment( "file" ) ) );
|
||||
EXPECT_STDSTREQ(
|
||||
"child",
|
||||
readString( vfs, root.withAppendedSegment( "collision" ).withAppendedSegment( "child" ) ) );
|
||||
EXPECT_EQ( VFSErrorCode::TooManySymlinks,
|
||||
codeOf( waitFor( vfs.open( root.withAppendedSegment( "cycle" ) ) ) ) );
|
||||
EXPECT_EQ( VFSErrorCode::NotFound,
|
||||
codeOf( waitFor( vfs.open( root.withAppendedSegment( "outside" ) ) ) ) );
|
||||
EXPECT_EQ( VFSErrorCode::Unsupported,
|
||||
codeOf( waitFor( vfs.open( root.withAppendedSegment( "pipe" ) ) ) ) );
|
||||
EXPECT_TRUE(
|
||||
waitFor( vfs.stat( root.withAppendedSegment( "file-link" ), StatOptions{ false } ) )
|
||||
.value()
|
||||
.isSymlink() );
|
||||
}
|
||||
|
||||
UTEST( ArchiveProvider, missingInvalidCorruptAndTruncatedArchivesReportErrors ) {
|
||||
TempDir temp;
|
||||
VirtualFileSystem vfs;
|
||||
if ( !available( utest_result, vfs ) )
|
||||
return;
|
||||
EXPECT_EQ( VFSErrorCode::NotFound, codeOf( waitFor( vfs.stat( ArchiveProvider::makeURI(
|
||||
URI::fromFSPath( temp.path + "missing.tar" ) ) ) ) ) );
|
||||
ASSERT_TRUE( FileSystem::fileWrite( temp.path + "invalid.tar", "not an archive" ) );
|
||||
EXPECT_TRUE( waitFor( vfs.listAll( ArchiveProvider::makeURI(
|
||||
URI::fromFSPath( temp.path + "invalid.tar" ) ) ) )
|
||||
.hasError() );
|
||||
TarFixture fixture;
|
||||
fixture.entry( "file", "contents" );
|
||||
fixture.bytes[10] ^= 1;
|
||||
ASSERT_TRUE( fixture.write( temp.path + "checksum.tar" ) );
|
||||
EXPECT_TRUE( waitFor( vfs.listAll( ArchiveProvider::makeURI(
|
||||
URI::fromFSPath( temp.path + "checksum.tar" ) ) ) )
|
||||
.hasError() );
|
||||
TarFixture truncated;
|
||||
truncated.entry( "file", "short", '0', {}, 65536 );
|
||||
ASSERT_TRUE( truncated.write( temp.path + "truncated.tar" ) );
|
||||
EXPECT_TRUE( waitFor( vfs.listAll( ArchiveProvider::makeURI(
|
||||
URI::fromFSPath( temp.path + "truncated.tar" ) ) ) )
|
||||
.hasError() );
|
||||
}
|
||||
|
||||
UTEST( ArchiveProvider, truncatedGzipIsReported ) {
|
||||
TempDir temp;
|
||||
VirtualFileSystem vfs;
|
||||
if ( !available( utest_result, vfs ) )
|
||||
return;
|
||||
TarFixture fixture;
|
||||
fixture.entry( "file", "contents" );
|
||||
const std::string path = temp.path + "corrupt.tar.gz";
|
||||
ASSERT_TRUE( fixture.write( path, true ) );
|
||||
std::string bytes;
|
||||
ASSERT_TRUE( FileSystem::fileGet( path, bytes ) );
|
||||
ASSERT_GT( bytes.size(), 8u );
|
||||
const URI file = ArchiveProvider::makeURI( URI::fromFSPath( path ), "/file" );
|
||||
bytes.resize( bytes.size() - 6 );
|
||||
ASSERT_TRUE( FileSystem::fileWrite( path, bytes ) );
|
||||
EXPECT_TRUE( waitFor( vfs.readAll( file ) ).hasError() );
|
||||
}
|
||||
|
||||
UTEST( ArchiveProvider, replacementCacheEvictionAndReaderLifetime ) {
|
||||
TempDir temp;
|
||||
VirtualFileSystem vfs;
|
||||
if ( !available( utest_result, vfs ) )
|
||||
return;
|
||||
TarFixture fixture;
|
||||
fixture.entry( "file", "original" );
|
||||
const auto path = temp.path + "original.tar";
|
||||
ASSERT_TRUE( fixture.write( path ) );
|
||||
const URI root = ArchiveProvider::makeURI( URI::fromFSPath( path ) );
|
||||
auto opened = waitFor( vfs.open( root.withAppendedSegment( "file" ) ) );
|
||||
ASSERT_TRUE( opened.hasValue() );
|
||||
auto handle = std::move( opened ).value();
|
||||
for ( int i = 0; i < 10; ++i ) {
|
||||
const auto other = temp.path + std::to_string( i ) + ".tar";
|
||||
ASSERT_TRUE( fixture.write( other ) );
|
||||
ASSERT_TRUE( waitFor( vfs.listAll( ArchiveProvider::makeURI( URI::fromFSPath( other ) ) ) )
|
||||
.hasValue() );
|
||||
}
|
||||
vfs.unregisterProvider( "archive" );
|
||||
std::array<Uint8, 8> bytes{};
|
||||
auto read = waitFor( handle.read( 0, bytes ) );
|
||||
ASSERT_TRUE( read.hasValue() );
|
||||
EXPECT_TRUE( std::string( bytes.begin(), bytes.end() ) == "original" );
|
||||
EXPECT_TRUE( waitFor( handle.close() ).hasValue() );
|
||||
|
||||
// Invalidate an index after replacement, including same-length changes.
|
||||
VirtualFileSystem next;
|
||||
EXPECT_STDSTREQ( "original", readString( next, root.withAppendedSegment( "file" ) ) );
|
||||
ASSERT_TRUE( FileSystem::fileRemove( path ) );
|
||||
TarFixture replacement;
|
||||
replacement.entry( "file", "replaced" );
|
||||
ASSERT_TRUE( replacement.write( path ) );
|
||||
EXPECT_STDSTREQ( "replaced", readString( next, root.withAppendedSegment( "file" ) ) );
|
||||
}
|
||||
@@ -346,6 +346,49 @@ UTEST( VFSTransfer, crossProviderCopyAndProgress ) {
|
||||
EXPECT_STDSTREQ( "new", readString( vfs, destination.withAppendedSegment( "a.txt" ) ) );
|
||||
}
|
||||
|
||||
UTEST( VFSTransfer, directoryMergeRequiresExplicitFileOverwrite ) {
|
||||
TempDir tmp;
|
||||
VirtualFileSystem vfs;
|
||||
auto mem = std::make_shared<MemoryProvider>();
|
||||
vfs.registerProvider( mem );
|
||||
const URI source( "mem://s/tree" );
|
||||
const URI nested = tmp.uri().withAppendedSegment( "nested" );
|
||||
const URI file = nested.withAppendedSegment( "file.txt" );
|
||||
mem->putDirectory( source.withAppendedSegment( "empty" ) );
|
||||
mem->putFile( source.withAppendedSegment( "nested" ).withAppendedSegment( "file.txt" ), "new" );
|
||||
ASSERT_TRUE( waitFor( vfs.makeDir( nested ) ) );
|
||||
ASSERT_TRUE( waitFor( vfs.writeAll( tmp.uri().withAppendedSegment( "unrelated" ), "keep" ) ) );
|
||||
// The ordinary Fail policy still rejects the existing root directory.
|
||||
EXPECT_EQ( VFSErrorCode::AlreadyExists,
|
||||
codeOf( waitFor( vfs.copy( source, tmp.uri() ).completion() ) ) );
|
||||
CopyOptions options;
|
||||
options.overwrite = OverwritePolicy::MergeDirectories;
|
||||
auto copied = waitFor( vfs.copy( source, tmp.uri(), options ).completion() );
|
||||
ASSERT_TRUE( copied );
|
||||
EXPECT_EQ( 1u, copied.value().filesTransferred );
|
||||
EXPECT_STDSTREQ( "new", readString( vfs, file ) );
|
||||
EXPECT_STDSTREQ( "keep", readString( vfs, tmp.uri().withAppendedSegment( "unrelated" ) ) );
|
||||
ASSERT_TRUE(
|
||||
waitFor( vfs.stat( tmp.uri().withAppendedSegment( "empty" ) ) ).value().isDirectory() );
|
||||
|
||||
ASSERT_TRUE( waitFor( vfs.writeAll( file, "existing" ) ) );
|
||||
auto conflict = waitFor( vfs.copy( source, tmp.uri(), options ).completion() );
|
||||
ASSERT_FALSE( conflict );
|
||||
EXPECT_EQ( VFSErrorCode::AlreadyExists, conflict.error().code );
|
||||
EXPECT_TRUE( conflict.error().uri == file );
|
||||
EXPECT_STDSTREQ( "existing", readString( vfs, file ) );
|
||||
// The policy also protects a file copied directly, without a directory root.
|
||||
EXPECT_EQ(
|
||||
VFSErrorCode::AlreadyExists,
|
||||
codeOf( waitFor(
|
||||
vfs.copy( source.withAppendedSegment( "nested" ).withAppendedSegment( "file.txt" ),
|
||||
file, options )
|
||||
.completion() ) ) );
|
||||
options.overwrite = OverwritePolicy::Overwrite;
|
||||
ASSERT_TRUE( waitFor( vfs.copy( source, tmp.uri(), options ).completion() ) );
|
||||
EXPECT_STDSTREQ( "new", readString( vfs, file ) );
|
||||
}
|
||||
|
||||
UTEST( VFSTransfer, moveNeverRemovesUncopiedSources ) {
|
||||
VirtualFileSystem vfs;
|
||||
auto memA = std::make_shared<MemoryProvider>( "mema" );
|
||||
@@ -748,7 +791,8 @@ UTEST( VFSSafety, unsafeReplacementIsRefusedWithoutAtomicReplace ) {
|
||||
}
|
||||
|
||||
UTEST( VFSSafety, concurrentDestinationRespectsFailAndSkip ) {
|
||||
for ( auto policy : { OverwritePolicy::Fail, OverwritePolicy::Skip } ) {
|
||||
for ( auto policy :
|
||||
{ OverwritePolicy::Fail, OverwritePolicy::Skip, OverwritePolicy::MergeDirectories } ) {
|
||||
VirtualFileSystem vfs;
|
||||
auto destination = std::make_shared<MemoryProvider>();
|
||||
auto source = std::make_shared<RacingReadProvider>( destination );
|
||||
@@ -760,7 +804,7 @@ UTEST( VFSSafety, concurrentDestinationRespectsFailAndSkip ) {
|
||||
auto result =
|
||||
waitFor( vfs.move( URI( "race://t/source" ), URI( "mem://t/destination" ), options )
|
||||
.completion() );
|
||||
if ( policy == OverwritePolicy::Fail ) {
|
||||
if ( policy != OverwritePolicy::Skip ) {
|
||||
ASSERT_FALSE( result.hasValue() );
|
||||
EXPECT_EQ( VFSErrorCode::AlreadyExists, result.error().code );
|
||||
} else {
|
||||
|
||||
@@ -302,6 +302,52 @@ UTEST( Zip, standaloneReaderIsReadOnlyAndHasBoundedSeek ) {
|
||||
EXPECT_TRUE( missing.error() == Zip::Error::InvalidArchive );
|
||||
}
|
||||
|
||||
UTEST( ZipProvider, extractRootIntoExistingFolder ) {
|
||||
TempDir temp;
|
||||
VirtualFileSystem vfs;
|
||||
const URI file = temp.uri().withAppendedSegment( "archive.zip" );
|
||||
ASSERT_TRUE(
|
||||
makeZip( file.getFSPath(),
|
||||
{ { ".hidden", "hidden" }, { "sub/file.txt", "contents" }, { "empty", "" } } ) );
|
||||
CopyOptions options;
|
||||
options.overwrite = OverwritePolicy::MergeDirectories;
|
||||
auto copied =
|
||||
waitFor( vfs.copy( ZipProvider::makeURI( file ), temp.uri(), options ).completion() );
|
||||
ASSERT_TRUE( copied );
|
||||
EXPECT_FALSE( copied.value().partial() );
|
||||
EXPECT_EQ( 3u, copied.value().filesTransferred );
|
||||
EXPECT_STDSTREQ( "hidden", readString( vfs, temp.uri().withAppendedSegment( ".hidden" ) ) );
|
||||
EXPECT_STDSTREQ( "contents",
|
||||
readString( vfs, temp.uri().withAppendedSegment( "sub" ).withAppendedSegment(
|
||||
"file.txt" ) ) );
|
||||
EXPECT_STDSTREQ( "", readString( vfs, temp.uri().withAppendedSegment( "empty" ) ) );
|
||||
// The backing archive remains usable after extraction into its containing folder.
|
||||
EXPECT_STDSTREQ( "hidden", readString( vfs, ZipProvider::makeURI( file, "/.hidden" ) ) );
|
||||
}
|
||||
|
||||
UTEST( ZipLocation, extractionFolderNames ) {
|
||||
TempDir temp;
|
||||
for ( const auto& [file, folder] : { std::pair{ "Mixed Case.zip", "Mixed Case" },
|
||||
{ "release.TAR.GZ", "release" },
|
||||
{ "package.tar.bz2", "package" },
|
||||
{ "package.tar.xz", "package" },
|
||||
{ "package.tar.lzma", "package" },
|
||||
{ "package.tar.lz", "package" },
|
||||
{ "package.tar.zst", "package" },
|
||||
{ "package.tar.lz4", "package" },
|
||||
{ "package.tar.Z", "package" },
|
||||
{ "package.tgz", "package" },
|
||||
{ "release.v1.7z", "release.v1" },
|
||||
{ "a! #%雪.epub", "a! #%雪" },
|
||||
{ ".hidden.zip", ".hidden" },
|
||||
{ ".tar.gz", "archive" },
|
||||
{ "..zip", "archive" },
|
||||
{ "...zip", "archive" } } ) {
|
||||
EXPECT_STDSTREQ(
|
||||
folder, eefm::Location::archiveFolderName( temp.uri().withAppendedSegment( file ) ) );
|
||||
}
|
||||
}
|
||||
|
||||
UTEST( ZipLocation, zipBasedFormatsUseTheSameProvider ) {
|
||||
TempDir temp;
|
||||
VirtualFileSystem vfs;
|
||||
|
||||
@@ -393,6 +393,41 @@ void FileOperations::showContextMenu( FileTab* tab, bool background ) {
|
||||
->setId( "open-other" );
|
||||
}
|
||||
}
|
||||
if ( !inTrash && uris.size() == 1 && !anyFolder &&
|
||||
!Location::archiveRoot( uris[0] ).empty() ) {
|
||||
const URI file = uris[0];
|
||||
const std::string folderName = Location::archiveFolderName( file );
|
||||
const URI folder = file.getParent().withAppendedSegment( folderName );
|
||||
const int paneIndex = tab->pane->index;
|
||||
UIPopUpMenu* extract = UIPopUpMenu::New();
|
||||
extract->add( String::format( "Extract to \"%s/\"", folderName.c_str() ) )
|
||||
->setId( "extract-folder" );
|
||||
extract->add( mUI->i18n( "eefm_extract_here", "Extract Here" ) )
|
||||
->setId( "extract-here" );
|
||||
extract->add( mUI->i18n( "eefm_extract_to", "Extract To..." ) )->setId( "extract-to" );
|
||||
extract->on(
|
||||
Event::OnItemClicked, [this, file, folder, paneIndex]( const Event* event ) {
|
||||
const std::string& id = event->getNode()->getId();
|
||||
if ( id == "extract-here" ) {
|
||||
extractArchive( file, file.getParent(), paneIndex );
|
||||
} else if ( id == "extract-folder" ) {
|
||||
extractArchive( file, folder, paneIndex );
|
||||
} else if ( id == "extract-to" ) {
|
||||
Dialogs::promptText(
|
||||
mUI, mUI->i18n( "eefm_extract_to", "Extract To..." ),
|
||||
mUI->i18n( "eefm_extract_destination",
|
||||
"Destination folder (path or URI):" ),
|
||||
Location::text( folder ).toUtf8(), false,
|
||||
[this, file, paneIndex]( std::optional<std::string> destination ) {
|
||||
if ( !destination || String::trim( *destination ).empty() )
|
||||
return;
|
||||
extractArchive( file, Location::parse( *destination ), paneIndex );
|
||||
} );
|
||||
}
|
||||
} );
|
||||
menu->addSubMenu( mUI->i18n( "eefm_extract", "Extract" ), mApp.icon( "mime-archive" ),
|
||||
extract );
|
||||
}
|
||||
// After the open actions, as Dolphin.
|
||||
if ( menu->getCount() > 0 )
|
||||
menu->addSeparator();
|
||||
@@ -640,22 +675,35 @@ void FileOperations::transferFiles( const std::vector<URI>& sources, const URI&
|
||||
auto [base, extension] = FileSystem::fileNameSplitExtension( name );
|
||||
target = destination.withAppendedSegment( base + " (copy)" + extension );
|
||||
}
|
||||
startTransfer( source, target, move, paneIndex );
|
||||
startTransfer( source, target, move ? TransferKind::Move : TransferKind::Copy, paneIndex );
|
||||
}
|
||||
}
|
||||
|
||||
void FileOperations::startTransfer( const URI& source, const URI& destination, bool move,
|
||||
void FileOperations::extractArchive( const URI& file, const URI& destination, int paneIndex ) {
|
||||
URI root = Location::archiveRoot( file );
|
||||
if ( root.empty() )
|
||||
return;
|
||||
CopyOptions options;
|
||||
options.overwrite = OverwritePolicy::MergeDirectories;
|
||||
// Copy the root as one transfer: entries are streamed sequentially, and an empty archive
|
||||
// still creates its destination folder. Existing files require an explicit overwrite.
|
||||
startTransfer( root, destination, TransferKind::Extract, paneIndex, options );
|
||||
}
|
||||
|
||||
void FileOperations::startTransfer( const URI& source, const URI& destination, TransferKind kind,
|
||||
int paneIndex, const CopyOptions& options ) {
|
||||
const bool move = kind == TransferKind::Move;
|
||||
String label = kind == TransferKind::Extract ? mUI->i18n( "eefm_extracting", "Extracting " )
|
||||
: move ? mUI->i18n( "eefm_moving", "Moving " )
|
||||
: mUI->i18n( "eefm_copying", "Copying " );
|
||||
Uint64 id = ++mLastTransferId;
|
||||
ActiveTransfer active{
|
||||
id,
|
||||
paneIndex,
|
||||
( move ? mUI->i18n( "eefm_moving", "Moving " ) : mUI->i18n( "eefm_copying", "Copying " ) ) +
|
||||
Location::displayName( source ),
|
||||
move ? mVFS->move( source, destination, options )
|
||||
: mVFS->copy( source, destination, options ),
|
||||
{},
|
||||
{} };
|
||||
ActiveTransfer active{ id,
|
||||
paneIndex,
|
||||
std::move( label ) + Location::displayName( source ),
|
||||
move ? mVFS->move( source, destination, options )
|
||||
: mVFS->copy( source, destination, options ),
|
||||
{},
|
||||
{} };
|
||||
active.subscription = active.transfer.onProgress(
|
||||
[this, id]( const TransferProgress& progress ) {
|
||||
auto it = mTransfers.find( id );
|
||||
@@ -669,7 +717,7 @@ void FileOperations::startTransfer( const URI& source, const URI& destination, b
|
||||
mTransfers.emplace( id, std::move( active ) );
|
||||
updateTransfers();
|
||||
std::move( completion )
|
||||
.then( mApp.executor(), [this, id, source, destination, move, paneIndex,
|
||||
.then( mApp.executor(), [this, id, source, destination, kind, paneIndex,
|
||||
options]( VFSResult<TransferResult>&& result ) {
|
||||
auto it = mTransfers.find( id );
|
||||
String label = it != mTransfers.end() ? it->second.label : String();
|
||||
@@ -690,25 +738,33 @@ void FileOperations::startTransfer( const URI& source, const URI& destination, b
|
||||
URI folder = destination.getParent();
|
||||
for ( auto& pane : mApp.panes().all() ) {
|
||||
for ( auto& tab : pane->tabList ) {
|
||||
if ( tab->model->uri() == folder.getParent() &&
|
||||
folder != tab->model->uri() )
|
||||
if ( ( tab->model->uri() == folder.getParent() &&
|
||||
folder != tab->model->uri() ) ||
|
||||
( kind == TransferKind::Extract && tab->model->uri() == destination ) )
|
||||
tab->model->refresh();
|
||||
}
|
||||
}
|
||||
return;
|
||||
}
|
||||
if ( result.error().is( VFSErrorCode::AlreadyExists ) &&
|
||||
options.overwrite == OverwritePolicy::Fail ) {
|
||||
( options.overwrite == OverwritePolicy::Fail ||
|
||||
options.overwrite == OverwritePolicy::MergeDirectories ) ) {
|
||||
String question =
|
||||
kind == TransferKind::Extract
|
||||
? mUI->i18n( "eefm_extract_overwrite",
|
||||
"The extraction destination contains existing files. "
|
||||
"Overwrite them?\n" ) +
|
||||
Location::text( destination )
|
||||
: String( String::format( "\"%s\" already exists. Overwrite it?",
|
||||
destination.getLastPathSegment().c_str() ) );
|
||||
Dialogs::confirm(
|
||||
mUI, mUI->i18n( "eefm_file_exists", "File Already Exists" ),
|
||||
String::format( "\"%s\" already exists. Overwrite it?",
|
||||
destination.getLastPathSegment().c_str() ),
|
||||
[this, source, destination, move, paneIndex, options]( bool overwrite ) {
|
||||
mUI, mUI->i18n( "eefm_file_exists", "File Already Exists" ), question,
|
||||
[this, source, destination, kind, paneIndex, options]( bool overwrite ) {
|
||||
if ( !overwrite )
|
||||
return;
|
||||
CopyOptions retry = options;
|
||||
retry.overwrite = OverwritePolicy::Overwrite;
|
||||
startTransfer( source, destination, move, paneIndex, retry );
|
||||
startTransfer( source, destination, kind, paneIndex, retry );
|
||||
} );
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -82,6 +82,8 @@ class FileOperations {
|
||||
void cancelTransfers();
|
||||
|
||||
private:
|
||||
enum class TransferKind : Uint8 { Copy, Move, Extract };
|
||||
|
||||
struct ActiveTransfer {
|
||||
Uint64 id;
|
||||
int pane;
|
||||
@@ -128,7 +130,9 @@ class FileOperations {
|
||||
void transferFiles( const std::vector<URI>& sources, const URI& destination, bool move,
|
||||
int paneIndex );
|
||||
|
||||
void startTransfer( const URI& source, const URI& destination, bool move, int paneIndex,
|
||||
void extractArchive( const URI& file, const URI& destination, int paneIndex );
|
||||
|
||||
void startTransfer( const URI& source, const URI& destination, TransferKind kind, int paneIndex,
|
||||
const EE::VFS::CopyOptions& options = {} );
|
||||
|
||||
void updateTransfers();
|
||||
|
||||
+51
-10
@@ -1,6 +1,8 @@
|
||||
#include "location.hpp"
|
||||
#include <eepp/system/filesystem.hpp>
|
||||
#include <eepp/vfs/providers/archiveprovider.hpp>
|
||||
#include <eepp/vfs/providers/zipprovider.hpp>
|
||||
#include <string_view>
|
||||
|
||||
using namespace EE;
|
||||
using namespace EE::System;
|
||||
@@ -8,9 +10,17 @@ using namespace EE::VFS;
|
||||
|
||||
namespace eefm {
|
||||
|
||||
namespace {
|
||||
|
||||
constexpr std::string_view TarSuffixes[] = { ".tar.gz", ".tar.bz2", ".tar.xz", ".tar.lzma",
|
||||
".tar.lz", ".tar.zst", ".tar.lz4", ".tar.z" };
|
||||
|
||||
} // namespace
|
||||
|
||||
URI Location::parse( const std::string& text ) {
|
||||
std::string trimmed = String::trim( text );
|
||||
if ( trimmed.find( "://" ) != std::string::npos || trimmed.starts_with( "zip:" ) ) {
|
||||
if ( trimmed.find( "://" ) != std::string::npos || trimmed.starts_with( "zip:" ) ||
|
||||
trimmed.starts_with( "archive:" ) ) {
|
||||
URI uri( trimmed );
|
||||
if ( uri.getPath().empty() )
|
||||
uri.setRawPath( "/" );
|
||||
@@ -25,28 +35,59 @@ URI Location::archiveRoot( const URI& file ) {
|
||||
const std::string extension = String::toLower( FileSystem::fileExtension( file.getPath() ) );
|
||||
if ( extension == "zip" || extension == "jar" || extension == "apk" || extension == "epub" )
|
||||
return ZipProvider::makeURI( file );
|
||||
if ( extension == "tar" || extension == "tgz" || extension == "tbz" || extension == "tbz2" ||
|
||||
extension == "txz" || extension == "tlz" || extension == "tzst" || extension == "7z" ||
|
||||
extension == "rar" || extension == "iso" || extension == "cab" || extension == "cpio" ||
|
||||
extension == "ar" || extension == "deb" || extension == "xar" || extension == "lha" ||
|
||||
extension == "lzh" )
|
||||
return ArchiveProvider::makeURI( file );
|
||||
const std::string name = String::toLower( file.getLastPathSegment() );
|
||||
for ( const auto suffix : TarSuffixes ) {
|
||||
if ( name.ends_with( suffix ) )
|
||||
return ArchiveProvider::makeURI( file );
|
||||
}
|
||||
return {};
|
||||
}
|
||||
|
||||
std::string Location::archiveFolderName( const URI& file ) {
|
||||
std::string name = file.getLastPathSegment();
|
||||
const std::string lower = String::toLower( name );
|
||||
size_t suffix = name.find_last_of( '.' );
|
||||
for ( const auto tarSuffix : TarSuffixes ) {
|
||||
if ( lower.ends_with( tarSuffix ) ) {
|
||||
suffix = name.size() - tarSuffix.size();
|
||||
break;
|
||||
}
|
||||
}
|
||||
if ( suffix != std::string::npos )
|
||||
name.resize( suffix );
|
||||
if ( name.empty() || name == "." || name == ".." )
|
||||
return "archive";
|
||||
return name;
|
||||
}
|
||||
|
||||
URI Location::parent( const URI& uri ) {
|
||||
if ( uri.getScheme() == "zip" ) {
|
||||
auto archive = ZipProvider::archiveURI( uri );
|
||||
auto member = ZipProvider::memberPath( uri );
|
||||
if ( uri.getScheme() == "zip" || uri.getScheme() == "archive" ) {
|
||||
const bool zip = uri.getScheme() == "zip";
|
||||
auto archive = zip ? ZipProvider::archiveURI( uri ) : ArchiveProvider::archiveURI( uri );
|
||||
auto member = zip ? ZipProvider::memberPath( uri ) : ArchiveProvider::memberPath( uri );
|
||||
if ( archive && member ) {
|
||||
if ( member.value() == "/" )
|
||||
return archive.value().getParent();
|
||||
URI entry;
|
||||
entry.setRawPath( std::move( member ).value() );
|
||||
return ZipProvider::makeURI( archive.value(), entry.getParent().getPath() );
|
||||
return zip ? ZipProvider::makeURI( archive.value(), entry.getParent().getPath() )
|
||||
: ArchiveProvider::makeURI( archive.value(), entry.getParent().getPath() );
|
||||
}
|
||||
}
|
||||
return uri.getParent();
|
||||
}
|
||||
|
||||
std::string Location::entryName( const URI& uri ) {
|
||||
if ( uri.getScheme() == "zip" ) {
|
||||
auto archive = ZipProvider::archiveURI( uri );
|
||||
auto member = ZipProvider::memberPath( uri );
|
||||
if ( uri.getScheme() == "zip" || uri.getScheme() == "archive" ) {
|
||||
const bool zip = uri.getScheme() == "zip";
|
||||
auto archive = zip ? ZipProvider::archiveURI( uri ) : ArchiveProvider::archiveURI( uri );
|
||||
auto member = zip ? ZipProvider::memberPath( uri ) : ArchiveProvider::memberPath( uri );
|
||||
if ( archive && member && member.value() == "/" )
|
||||
return archive.value().getLastPathSegment();
|
||||
}
|
||||
@@ -55,7 +96,7 @@ std::string Location::entryName( const URI& uri ) {
|
||||
|
||||
String Location::displayName( const URI& uri ) {
|
||||
std::string name = entryName( uri );
|
||||
if ( uri.getScheme() == "zip" && !name.empty() )
|
||||
if ( ( uri.getScheme() == "zip" || uri.getScheme() == "archive" ) && !name.empty() )
|
||||
return String::fromUtf8( name );
|
||||
if ( name.empty() || uri.getPath() == "/" )
|
||||
name = uri.getScheme() == "file" ? std::string( "/" ) : uri.getHost();
|
||||
@@ -63,7 +104,7 @@ String Location::displayName( const URI& uri ) {
|
||||
}
|
||||
|
||||
String Location::text( const URI& uri ) {
|
||||
if ( uri.getScheme() == "zip" ) {
|
||||
if ( uri.getScheme() == "zip" || uri.getScheme() == "archive" ) {
|
||||
// URI::toString() omits an empty authority for non-file schemes. Spell it explicitly
|
||||
// in the location bar; both spellings parse into the same provider URI.
|
||||
return String::fromUtf8( uri.getSchemeAndAuthority() + uri.getPathEtc() );
|
||||
|
||||
@@ -15,9 +15,14 @@ class Location {
|
||||
/** A typed location: a URI, or a local path ("~" expands to the home folder). */
|
||||
static URI parse( const std::string& text );
|
||||
|
||||
/** Browsable ZIP archive root for a local .zip, .jar, .apk or .epub file; empty otherwise. */
|
||||
/** Browsable archive root for a recognized local archive filename; empty otherwise.
|
||||
* Runtime archive support is checked asynchronously by the provider when navigating. */
|
||||
static URI archiveRoot( const URI& file );
|
||||
|
||||
/** Suggested extraction folder, without the archive suffix (including compound TAR suffixes).
|
||||
*/
|
||||
static std::string archiveFolderName( const URI& file );
|
||||
|
||||
/** Parent location; leaving an archive root returns to the archive's containing folder. */
|
||||
static URI parent( const URI& uri );
|
||||
|
||||
|
||||
Reference in New Issue
Block a user