add shared WebView resource cache:

Introduce WebResourceCache with partitioned sharing, per-document sessions,
navigation generations, leases, and canonical request keys.

Coalesce concurrent document, stylesheet, font, and image requests while
discarding stale navigation subscribers independently. Retain completed
resources with TTL/LRU and byte-budget policies, and periodically prune
expired unleased entries from UIWebView.

Route UISceneNode, UIWebView, UIImage, CSS background images, and remote
drawable resolution through the shared cache. Preserve ResourceScope entries
as explicit overrides and allow multiple WebViews to share a cache partition.

Keep image decode and texture replacement on the guarded graphics-thread
completion path. Exclude volatile Cookie and Referer values from cache
identity while preserving partition isolation and content-affecting variants.

Start TTL retention when the final document lease is released so Back/Forward
navigation can reuse resources regardless of how long the previous page was
open.

Avoid unstable pointers and iterators into unordered_dense storage when
tracking leases and pruning entries. Add ASan regression coverage for dense
map growth, navigation, eviction, partition isolation, request coalescing,
retry behavior, and cache retention.

Fully document cache partitions, sessions, generations, request identity,
ownership, threading, and maintenance behavior.
This commit is contained in:
Martín Lucas Golini
2026-07-22 02:22:38 -03:00
parent f7dba042df
commit 2d8693295c
16 changed files with 1339 additions and 219 deletions
@@ -1,6 +1,6 @@
# eepp shared-resource ownership architecture
Status: active implementation baseline; Stage 0 through Stage 5 complete, 2026-07-21.
Status: active implementation baseline; Stage 0 through Stage 6 complete, 2026-07-21.
This document freezes the contracts that must be true before the public texture API is changed. The
implementation may refine names and small mechanics, but changing an invariant below requires an
@@ -752,6 +752,28 @@ Exit criteria:
### Stage 6: WebResourceCache and document leases
Status: complete, 2026-07-21. Each UISceneNode now owns a WebResourceCache document session with
an explicit cache partition and navigation generation. UIWebView advances that session at the
existing navigation boundary. Document, stylesheet, remote font, image, and CSS background image
requests share canonical fragment-free keys that include the partition, request method/body and
headers, resource kind, and image decode options.
Concurrent requests coalesce into one fetch and one image decode/upload. Subscribers retain their
own document generation, so navigation or destruction removes stale delivery without cancelling a
request needed by another session. Applications can install one cache and explicit partition into
multiple WebViews to share eligible public resources; distinct partitions and content-affecting
headers remain isolated. Scene ResourceScope entries remain an explicit override but fetched Web
resources are retained only by consumers, document leases, and the cache.
Completed entries use monotonic TTL and LRU timestamps plus a configurable byte budget. Active
document leases are not evicted; the TTL starts when the final document lease is released, allowing
Back/Forward navigation to reuse resources regardless of how long the previous document remained
open. Navigation releases only that session's previous leases, and UIWebView performs throttled
cache maintenance so expired unleased entries are collected even when no new requests complete. Failed loads
retry, redirect/final cookies are delivered only to current subscribers, and image upload is
dispatched through the scene's guarded main-thread resource queue. Common lease lists use inline
small vectors, and completed entries release starter request bodies, headers, and dispatchers.
Implement cache partitions, canonical keys/origins, per-document sessions and leases, in-flight
coalescing, per-subscriber generation guards, retries, TTL/LRU, and byte budgets. Integrate WebView
navigation at its existing document replacement boundary.
@@ -837,8 +859,6 @@ Remove raw-owning `ResourceManager<T>` only when no subclass or consumer depends
## 12. Next implementation deliverable
Stage 2 is complete with the repository-wide TexturePtr API and holder migration while temporary
factory retention preserves existing global lookup behavior. The next coding deliverable is Stage
3: introduce catalogs and Graphics scopes, migrate semantic name lookup out of TextureFactory, move
intentional persistence into catalogs/caches, and then remove the factory's temporary strong
retention.
Stage 7 migrates the remaining ResourceManager families one at a time: fonts and font caches,
themes/icons, shaders/programs, nine-patches, atlases, and any remaining raw-owning manager. The
raw-owning ResourceManager template is removed only after its final consumer is migrated.
+1
View File
@@ -180,6 +180,7 @@
#include <eepp/ui/uiwidgettablerow.hpp>
#include <eepp/ui/uiwindow.hpp>
#include <eepp/ui/undostack.hpp>
#include <eepp/ui/webresourcecache.hpp>
#include <eepp/ui/widgetcommandexecuter.hpp>
#endif
-3
View File
@@ -145,9 +145,6 @@ class EE_API UINodeDrawable : public Drawable {
Origin mOrigin{ Origin::PaddingBox };
Clip mClip{ Clip::BorderBox };
Attachment mAttachment{ Attachment::Scroll };
std::shared_ptr<std::atomic<bool>> mAsyncDrawableAlive;
Uint64 mRemoteDrawableLoadId{ 0 };
virtual void onPositionChange();
virtual void onColorFilterChange();
+18
View File
@@ -13,6 +13,7 @@
#include <eepp/ui/drawableresolver.hpp>
#include <eepp/ui/keyboardshortcut.hpp>
#include <eepp/ui/layoutinvalidation.hpp>
#include <eepp/ui/webresourcecache.hpp>
#include <atomic>
#include <functional>
@@ -850,6 +851,21 @@ class EE_API UISceneNode : public SceneNode {
Network::CookieManager& getCookieManager() { return mCookieManager; }
const WebResourceCachePtr& getWebResourceCache() const { return mWebResourceCache; }
UISceneNode* setWebResourceCache( WebResourceCachePtr cache, CachePartitionId partition = 0 );
DocumentSessionId getDocumentSessionId() const { return mDocumentSessionId; }
Uint64 beginDocumentNavigation( const URI& uri );
Uint64 getDocumentGeneration() const;
void requestWebResource( WebResourceRequest request, WebResourceCache::Callback callback );
Graphics::TexturePtr requestWebTexture( WebResourceRequest request,
WebResourceCache::Callback callback = {} );
void invalidateAsyncResourceLoads();
virtual void invalidate( Node* invalidator );
@@ -918,6 +934,8 @@ class EE_API UISceneNode : public SceneNode {
std::shared_ptr<AsyncResourceLoadState> mAsyncResourceLoadState;
Graphics::ResourceScopePtr mResourceScope;
DrawableResolver mDrawableResolver;
WebResourceCachePtr mWebResourceCache;
DocumentSessionId mDocumentSessionId{ 0 };
KeyBindings mKeyBindings;
std::map<std::string, KeyBindingCommand> mKeyBindingCommands;
UnorderedSet<UIWidget*> mDirtyStyle;
+6
View File
@@ -7,6 +7,7 @@
#include <eepp/system/time.hpp>
#include <eepp/ui/layoutinvalidation.hpp>
#include <eepp/ui/uiscrollview.hpp>
#include <eepp/ui/webresourcecache.hpp>
#include <functional>
#include <memory>
@@ -72,6 +73,10 @@ class EE_API UIWebView : public UIScrollView {
UISceneNode* getDocumentSceneNode() const;
const WebResourceCachePtr& getWebResourceCache() const;
UIWebView* setWebResourceCache( WebResourceCachePtr cache, CachePartitionId partition = 0 );
void setStyleSheetDefaultMarker( Uint32 marker );
void setUserAgent( const std::string& userAgent );
@@ -117,6 +122,7 @@ class EE_API UIWebView : public UIScrollView {
Uint64 mNavigationGeneration{ 0 };
std::string mUserAgent;
Time mDefaultTimeout{ Seconds( 30 ) };
Time mWebResourceCachePruneElapsed;
Uint32 mStyleSheetDefaultMarker{ 0 };
void loadURI( URI uri, bool isHistoryNav );
+302
View File
@@ -0,0 +1,302 @@
#ifndef EE_UI_WEBRESOURCECACHE_HPP
#define EE_UI_WEBRESOURCECACHE_HPP
#include <eepp/graphics/resource.hpp>
#include <eepp/graphics/texture.hpp>
#include <eepp/network/http.hpp>
#include <eepp/network/uri.hpp>
#include <eepp/system/time.hpp>
#include <functional>
#include <memory>
#include <string>
namespace EE { namespace UI {
/** Shared ownership handle for a WebResourceCache.
*
* A cache can be installed into more than one UIWebView. Sharing the cache does not by itself make
* all resources visible between those views: sessions must also use the same CachePartitionId. */
using WebResourceCachePtr = std::shared_ptr<class WebResourceCache>;
/** Identifies one document consumer of a WebResourceCache.
*
* UISceneNode creates a session for its current document. A session tracks the current navigation
* generation and the cache entries leased by that document. IDs are process-wide and opaque. */
using DocumentSessionId = Uint64;
/** Identifies a cache sharing and privacy boundary.
*
* Sessions in the same partition may reuse and coalesce resources. Sessions in different
* partitions never share entries, even when requesting the same URI. A partition should therefore
* represent one intentionally shared HTTP state, normally one cookie jar/authentication context.
*
* For example, browser tabs that share cookies should use the same WebResourceCache and
* CachePartitionId. A private tab or a view logged into a different account must use another
* partition. Passing zero to createSession() creates a new private partition automatically.
* Partition IDs are process-wide and opaque.
*
* @code
* auto cache = WebResourceCache::New();
* auto normalProfile = cache->createPartition();
* firstTab->setWebResourceCache( cache, normalProfile );
* secondTab->setWebResourceCache( cache, normalProfile ); // May reuse firstTab resources.
* privateTab->setWebResourceCache( cache, cache->createPartition() ); // Fully isolated entries.
* @endcode */
using CachePartitionId = Uint64;
/** Canonical network origin used to describe a document navigation.
*
* Origins compare scheme, host, and port. Consequently HTTP and HTTPS origins are distinct, as are
* equal hosts using different ports. Paths, queries, and fragments are not part of an origin. */
struct EE_API WebOriginKey {
/** Lowercase URI scheme, such as "http" or "https". */
std::string scheme;
/** Lowercase host name. */
std::string host;
/** URI port, or the URI implementation's default/empty port value. */
Uint16 port{ 0 };
/** @return True when no origin components have been assigned. */
bool empty() const { return scheme.empty() && host.empty() && port == 0; }
/** @return True when both values describe the same scheme, host, and port. */
bool operator==( const WebOriginKey& other ) const;
/** @return True when at least one origin component differs. */
bool operator!=( const WebOriginKey& other ) const { return !( *this == other ); }
/** Creates an origin key from a URI, normalizing the scheme and host to lowercase. */
static WebOriginKey fromURI( const Network::URI& uri );
};
/** Semantic type of a cached web resource.
*
* The kind is part of cache identity, so one URI requested as a document and as a stylesheet uses
* distinct entries. Images additionally carry texture decode options in WebResourceRequest. */
enum class WebResourceKind : Uint8 {
/** Top-level HTML or another document response. */
Document,
/** CSS stylesheet response. */
StyleSheet,
/** Downloadable font response. */
Font,
/** Image response decoded into a Graphics::Texture. */
Image
};
/** Current lifecycle state of a cache entry. */
enum class WebResourceLoadState : Uint8 {
/** Entry exists but has not started loading. */
Empty,
/** One fetch is active; additional subscribers join it. */
Loading,
/** Resource completed successfully and is available for reuse. */
Ready,
/** Last fetch failed and remains subject to the retry delay. */
Failed,
/** Entry was cleared and any late fetch completion will be ignored. */
Cancelled
};
/** Complete input required to identify and fetch one web resource variant.
*
* Cache identity includes the partition, canonical URI, resource kind, method, non-navigation
* headers, body, and image decode options. Cookie and Referer are deliberately excluded from entry
* identity: the partition isolates cookie contexts, while these two values naturally change during
* navigation. They are still sent with the HTTP request.
*
* Transport policy such as timeout, proxy, and completionDispatcher controls a fetch but does not
* describe the resulting resource and is therefore not part of cache identity. */
struct EE_API WebResourceRequest {
/** Dispatches a completion function to the thread on which it is safe to finalize a resource.
*/
using CompletionDispatcher = std::function<void( std::function<void()> )>;
/** Absolute URI to fetch. Fragments are removed and the URI is normalized for cache lookup. */
Network::URI uri;
/** Semantic resource type. */
WebResourceKind kind{ WebResourceKind::Document };
/** HTTP method. The method is part of cache identity. */
Network::Http::Request::Method method{ Network::Http::Request::Method::Get };
/** HTTP request headers. See the type documentation for cache-key rules. */
Network::Http::Request::FieldTable headers;
/** HTTP request body. The complete body is part of cache identity. */
std::string body;
/** Maximum duration allowed for the HTTP request. */
System::Time timeout{ System::Seconds( 5 ) };
/** Whether HTTPS certificates must be validated. */
bool validateCertificate{ true };
/** Optional HTTP proxy URI. An empty URI uses a direct connection. */
Network::URI proxy;
/** Whether HTTP redirects should be followed. */
bool followRedirect{ true };
/** Image-only texture wrapping policy. True selects ClampToEdge. */
bool clampToEdge{ true };
/** Image-only option controlling mipmap generation. */
bool mipmaps{ false };
/** Image-only option requesting texture compression. */
bool compressTexture{ false };
/** Image-only SVG rasterization scale. */
Float svgScale{ 1.f };
/** Image-only completion dispatcher.
*
* Image decode/upload completion must execute where a graphics context is current. This policy
* is not part of cache identity and only the request that starts a coalesced load supplies it.
*/
CompletionDispatcher completionDispatcher;
};
/** Result delivered to a WebResourceCache subscriber.
*
* Data resources set data, while image resources set texture. A texture request may already have
* returned the same TexturePtr as a transparent placeholder before this result is delivered. */
struct EE_API WebResourceResult {
/** True when the request completed and produced a usable resource. */
bool success{ false };
/** Final cache-entry state observed by this completion. */
WebResourceLoadState state{ WebResourceLoadState::Empty };
/** Numeric HTTP response status. */
int status{ 0 };
/** Failure description. Empty for successful results. */
std::string error;
/** Set-Cookie value from the final response, or from its redirect chain when applicable. */
std::string setCookie;
/** Shared immutable response body for documents, stylesheets, and fonts. */
std::shared_ptr<const std::string> data;
/** Shared texture for image resources. */
Graphics::TexturePtr texture;
};
/** Shared HTTP-backed document resource cache.
*
* The cache separates three concepts:
* - A partition defines which HTTP/cookie context may share entries.
* - A session represents one active document using entries from that partition.
* - A navigation generation prevents an obsolete document from receiving asynchronous results.
*
* Entries are strongly retained by the cache after their final document lease is released. Their
* TTL starts at that release point, allowing Back/Forward navigation to reuse them. Expired entries
* are removed by prune(); UIWebView invokes it periodically. A byte budget additionally evicts the
* least-recently-used unleased entries. Active leases and in-flight loads are not evicted.
*
* Requests for the same key are coalesced into one HTTP operation. Each current subscriber receives
* the result, while callbacks belonging to obsolete navigation generations are discarded. Public
* operations are internally synchronized. Subscriber callbacks run without the cache mutex held. */
class EE_API WebResourceCache : public std::enable_shared_from_this<WebResourceCache> {
public:
/** Receives the completed result of a data or texture request. */
using Callback = std::function<void( const WebResourceResult& )>;
/** @return A newly allocated shared web-resource cache. */
static WebResourceCachePtr New();
/** Creates an empty cache with a 30-second TTL, 1-second retry delay, and 64 MiB budget. */
WebResourceCache();
/** Cancels cache delivery and releases all retained entries and sessions. */
~WebResourceCache();
/** Allocates a new process-wide partition identifier.
* @return An opaque, non-zero partition ID. */
CachePartitionId createPartition();
/** Creates a document session.
* @param partition Partition to join. Zero creates a new private partition.
* @return An opaque, non-zero document session ID. */
DocumentSessionId createSession( CachePartitionId partition = 0 );
/** Destroys a session and releases all entries leased by it.
*
* Releasing the final lease starts each entry's TTL; it does not normally erase entries
* immediately. Pending callbacks for the destroyed session will not be delivered. */
void destroySession( DocumentSessionId session );
/** Begins a new document navigation in an existing session.
*
* Previous document leases are released and the generation is incremented. Pending subscribers
* from older generations become stale, but shared in-flight requests continue for other current
* subscribers.
* @param session Session performing the navigation.
* @param uri Destination document URI, used to record its origin.
* @return The new generation, or zero when the session does not exist. */
Uint64 beginNavigation( DocumentSessionId session, const Network::URI& uri );
/** @return The current generation for a session, or zero when it does not exist. */
Uint64 getSessionGeneration( DocumentSessionId session ) const;
/** @return The partition used by a session, or zero when it does not exist. */
CachePartitionId getSessionPartition( DocumentSessionId session ) const;
/** Requests a document, stylesheet, font, or other non-texture response body.
*
* A ready cache hit may invoke callback synchronously. A miss starts or joins an asynchronous
* fetch. The callback is omitted when the session/generation is stale.
* @param session Requesting document session.
* @param generation Generation returned by beginNavigation().
* @param request Resource and transport description.
* @param callback Optional completion callback. */
void requestData( DocumentSessionId session, Uint64 generation, WebResourceRequest request,
Callback callback );
/** Requests an image as a shared texture.
*
* On a miss, this creates and returns a transparent placeholder immediately. The same
* TexturePtr is populated after download and decode. A ready hit returns the existing texture
* without a new HTTP request. This function must be called where texture creation is valid, and
* the request must provide an appropriate completionDispatcher for asynchronous GPU upload.
* @return The cached/new texture, or null for an invalid or stale session. */
Graphics::TexturePtr requestTexture( DocumentSessionId session, Uint64 generation,
WebResourceRequest request, Callback callback = {} );
/** Sets the retention duration used when an entry's final document lease is released. */
void setTTL( System::Time ttl );
/** @return The current post-lease retention duration. */
System::Time getTTL() const;
/** Sets how long a failed entry suppresses another fetch attempt. */
void setRetryDelay( System::Time delay );
/** @return The current failed-request retry delay. */
System::Time getRetryDelay() const;
/** Sets the maximum retained response/texture byte estimate.
*
* Applying a smaller budget immediately evicts least-recently-used unleased entries where
* possible. Active entries may temporarily make retained bytes exceed the budget. */
void setByteBudget( std::size_t bytes );
/** @return The configured retained-byte budget. */
std::size_t getByteBudget() const;
/** @return Estimated bytes retained by ready data bodies and decoded image textures. */
std::size_t getRetainedBytes() const;
/** @return Number of cache entries in all states and partitions. */
std::size_t getEntryCount() const;
/** @return Number of HTTP requests currently owned by the cache. */
std::size_t getInFlightCount() const;
/** Removes expired entries and enforces the byte budget.
*
* Active document leases and loading entries are preserved. UIWebView calls this periodically;
* other cache owners should provide their own maintenance point. */
void prune();
/** Removes every entry immediately and suppresses delivery from outstanding fetches.
*
* Sessions remain valid but no longer lease entries. This does not cancel the underlying HTTP
* operation; a late completion is ignored because its entry no longer exists. */
void clear();
/** Completion supplied to a custom Fetcher. */
using FetchCompletion = std::function<void( Network::Http::Response )>;
/** Custom fetch implementation, primarily intended for deterministic tests.
*
* The fetcher must eventually invoke FetchCompletion exactly once. Production uses
* Http::requestAsync when no custom fetcher is installed. */
using Fetcher = std::function<void( const WebResourceRequest&, FetchCompletion )>;
/** Installs or clears a custom HTTP fetch implementation. */
void setFetcher( Fetcher fetcher );
private:
/** Private implementation containing synchronized sessions and cache entries. */
struct Impl;
/** Exclusive implementation state. */
std::unique_ptr<Impl> mImpl;
};
}} // namespace EE::UI
#endif
+2
View File
@@ -58,6 +58,7 @@
../../include/eepp/graphics/drawable.hpp
../../include/eepp/graphics/drawableresource.hpp
../../include/eepp/ui/drawableresolver.hpp
../../include/eepp/ui/webresourcecache.hpp
../../include/eepp/graphics/fontbmfont.hpp
../../include/eepp/graphics/font.hpp
../../include/eepp/graphics/fontfamily.hpp
@@ -556,6 +557,7 @@
../../src/eepp/graphics/drawablegroup.cpp
../../src/eepp/graphics/drawableresource.cpp
../../src/eepp/ui/drawableresolver.cpp
../../src/eepp/ui/webresourcecache.cpp
../../src/eepp/graphics/fontbmfont.cpp
../../src/eepp/graphics/font.cpp
../../src/eepp/graphics/fontfamily.cpp
+2
View File
@@ -58,6 +58,7 @@
../../include/eepp/graphics/drawable.hpp
../../include/eepp/graphics/drawableresource.hpp
../../include/eepp/ui/drawableresolver.hpp
../../include/eepp/ui/webresourcecache.hpp
../../include/eepp/graphics/fontbmfont.hpp
../../include/eepp/graphics/font.hpp
../../include/eepp/graphics/fontfamily.hpp
@@ -547,6 +548,7 @@
../../src/eepp/graphics/drawablegroup.cpp
../../src/eepp/graphics/drawableresource.cpp
../../src/eepp/ui/drawableresolver.cpp
../../src/eepp/ui/webresourcecache.cpp
../../src/eepp/graphics/fontbmfont.cpp
../../src/eepp/graphics/font.cpp
../../src/eepp/graphics/fontfamily.cpp
+2
View File
@@ -57,6 +57,7 @@
../../include/eepp/graphics/drawable.hpp
../../include/eepp/graphics/drawableresource.hpp
../../include/eepp/ui/drawableresolver.hpp
../../include/eepp/ui/webresourcecache.hpp
../../include/eepp/graphics/fontbmfont.hpp
../../include/eepp/graphics/font.hpp
../../include/eepp/graphics/fontmanager.hpp
@@ -539,6 +540,7 @@
../../src/eepp/graphics/drawablegroup.cpp
../../src/eepp/graphics/drawableresource.cpp
../../src/eepp/ui/drawableresolver.cpp
../../src/eepp/ui/webresourcecache.cpp
../../src/eepp/graphics/fontbmfont.cpp
../../src/eepp/graphics/font.cpp
../../src/eepp/graphics/fontmanager.cpp
+15 -1
View File
@@ -103,7 +103,21 @@ DrawablePtr DrawableResolver::resolve( const std::string& name, bool firstSearch
drawable = texture ? texture->clone() : DrawablePtr{};
} else if ( String::startsWith( name, "http://" ) || String::startsWith( name, "https://" ) ) {
TexturePtr texture = resourceScope.findTexture( name );
if ( !texture && Engine::instance()->isSharedGLContextEnabled() ) {
if ( mSceneNode && Engine::instance()->isSharedGLContextEnabled() ) {
if ( !texture ) {
WebResourceRequest request;
request.uri = URI( name );
request.kind = WebResourceKind::Image;
texture = mSceneNode->requestWebTexture(
std::move( request ), [name]( const WebResourceResult& result ) {
if ( !result.success )
Log::debug( "DrawableResolver::resolve: could not download image: %s. "
"Error: %d\n%s",
name, result.status, result.error );
} );
}
}
if ( !mSceneNode && !texture && Engine::instance()->isSharedGLContextEnabled() ) {
texture = TextureFactory::instance()->createEmptyTexture(
1, 1, 4, Color::Transparent, false, Texture::ClampMode::ClampToEdge, false, false,
name );
+23 -53
View File
@@ -64,10 +64,7 @@ UIImage* UIImage::NewWithTag( const std::string& tag ) {
}
UIImage::UIImage( const std::string& tag ) :
UIWidget( tag ),
mScaleType( UIScaleType::None ),
mColor(),
mAlignOffset( 0, 0 ) {
UIWidget( tag ), mScaleType( UIScaleType::None ), mColor(), mAlignOffset( 0, 0 ) {
mFlags |= UI_AUTO_SIZE;
applyDefaultTheme();
@@ -379,65 +376,38 @@ void UIImage::loadRemoteDrawable( const Network::URI& uri ) {
return;
std::string url = uri.toString();
ResourceScopePtr resourceScope = scene->getResourceScope();
if ( TexturePtr texture = resourceScope->findTexture( url ) ) {
TextureDrawable* textureDrawable =
if ( TexturePtr texture = scene->getResourceScope()->findTexture( url ) ) {
TextureDrawable* current =
mDrawable && mDrawable->getDrawableType() == Drawable::TEXTUREDRAWABLE
? static_cast<TextureDrawable*>( mDrawable.get() )
: nullptr;
if ( !textureDrawable || textureDrawable->getTexture() != texture ) {
if ( !current || current->getTexture() != texture ) {
++mRemoteImageLoadId;
setDrawable( std::move( texture ) );
}
return;
}
auto resourceState = scene->getAsyncResourceLoadState();
Uint64 resourceGeneration =
resourceState ? resourceState->generation.load( std::memory_order_acquire ) : 0;
Uint64 loadId = ++mRemoteImageLoadId;
if ( !mAsyncImageAlive )
mAsyncImageAlive = std::make_shared<std::atomic<bool>>( true );
auto alive = mAsyncImageAlive;
TexturePtr texture = TextureFactory::instance()->createEmptyTexture(
1, 1, 4, Color::Transparent, false, Texture::ClampMode::ClampToEdge, false, false, url );
if ( texture ) {
resourceScope->publishLocal( url, texture );
setDrawable( texture );
}
Http::Request::FieldTable headers;
if ( !scene->getReferer().empty() )
headers["referer"] = scene->getReferer().toString();
Http::getAsync(
[resourceState, resourceGeneration, alive, loadId, texture, url = std::move( url ),
this]( const Http&, Http::Request&, Http::Response& response ) {
if ( !UISceneNode::isAsyncResourceLoadCurrent( resourceState, resourceGeneration ) ||
!alive || !alive->load( std::memory_order_acquire ) || texture == nullptr )
return;
if ( response.isOK() && !response.getBody().empty() ) {
std::string imageData( response.getBody() );
UISceneNode::runAsyncResourceOnMainThread(
resourceState, resourceGeneration,
[alive, loadId, texture, imageData = std::move( imageData ),
this]( UISceneNode* ) mutable {
if ( !alive || !alive->load( std::memory_order_acquire ) ||
loadId != mRemoteImageLoadId )
return;
Image image( reinterpret_cast<const Uint8*>( imageData.data() ),
imageData.size() );
if ( image.getPixels() != nullptr )
texture->replace( &image );
} );
} else {
WebResourceRequest request;
request.uri = uri;
request.kind = WebResourceKind::Image;
request.proxy = Http::getEnvProxyURI();
TexturePtr texture = scene->requestWebTexture(
std::move( request ), [url = std::move( url )]( const WebResourceResult& result ) {
if ( !result.success )
Log::debug( "UIImage::loadRemoteDrawable: could not download image: %s. Error: "
"%d\n%s",
url, response.getStatus(), response.getBody() );
}
},
uri, Seconds( 5 ), {}, headers, "", true, Http::getEnvProxyURI() );
url, result.status, result.error );
} );
if ( texture ) {
TextureDrawable* current =
mDrawable && mDrawable->getDrawableType() == Drawable::TEXTUREDRAWABLE
? static_cast<TextureDrawable*>( mDrawable.get() )
: nullptr;
if ( !current || current->getTexture() != texture ) {
++mRemoteImageLoadId;
setDrawable( std::move( texture ) );
}
}
}
void UIImage::onSizeChange() {
+20 -54
View File
@@ -418,9 +418,6 @@ UINodeDrawable::LayerDrawable::LayerDrawable( UINodeDrawable* container ) :
mAttachment( Attachment::Scroll ) {}
UINodeDrawable::LayerDrawable::~LayerDrawable() {
if ( mAsyncDrawableAlive )
mAsyncDrawableAlive->store( false, std::memory_order_release );
mResourceChangeConnection.disconnect();
}
@@ -677,65 +674,34 @@ bool UINodeDrawable::LayerDrawable::loadRemoteDrawable( const std::string& value
return true;
std::string url = uri.toString();
ResourceScopePtr resourceScope = scene->getResourceScope();
if ( TexturePtr texture = resourceScope->findTexture( url ) ) {
TextureDrawable* textureDrawable =
if ( TexturePtr texture = scene->getResourceScope()->findTexture( url ) ) {
TextureDrawable* current =
mDrawable && mDrawable->getDrawableType() == Drawable::TEXTUREDRAWABLE
? static_cast<TextureDrawable*>( mDrawable.get() )
: nullptr;
if ( !textureDrawable || textureDrawable->getTexture() != texture ) {
++mRemoteDrawableLoadId;
if ( !current || current->getTexture() != texture )
setDrawable( std::move( texture ) );
}
return true;
}
auto resourceState = scene->getAsyncResourceLoadState();
Uint64 resourceGeneration =
resourceState ? resourceState->generation.load( std::memory_order_acquire ) : 0;
Uint64 loadId = ++mRemoteDrawableLoadId;
if ( !mAsyncDrawableAlive )
mAsyncDrawableAlive = std::make_shared<std::atomic<bool>>( true );
auto alive = mAsyncDrawableAlive;
TexturePtr texture = TextureFactory::instance()->createEmptyTexture(
1, 1, 4, Color::Transparent, false, Texture::ClampMode::ClampToEdge, false, false, url );
if ( texture ) {
resourceScope->publishLocal( url, texture );
setDrawable( texture );
}
Http::Request::FieldTable headers;
if ( !scene->getReferer().empty() )
headers["referer"] = scene->getReferer().toString();
Http::getAsync(
[resourceState, resourceGeneration, alive, loadId, texture, url = std::move( url ),
this]( const Http&, Http::Request&, Http::Response& response ) {
if ( !UISceneNode::isAsyncResourceLoadCurrent( resourceState, resourceGeneration ) ||
!alive || !alive->load( std::memory_order_acquire ) || texture == nullptr )
return;
if ( response.isOK() && !response.getBody().empty() ) {
std::string imageData( response.getBody() );
UISceneNode::runAsyncResourceOnMainThread(
resourceState, resourceGeneration,
[alive, loadId, texture, imageData = std::move( imageData ),
this]( UISceneNode* ) mutable {
if ( !alive || !alive->load( std::memory_order_acquire ) ||
loadId != mRemoteDrawableLoadId )
return;
Image image( reinterpret_cast<const Uint8*>( imageData.data() ),
imageData.size() );
if ( image.getPixels() != nullptr )
texture->replace( &image );
} );
} else {
WebResourceRequest request;
request.uri = uri;
request.kind = WebResourceKind::Image;
request.proxy = Http::getEnvProxyURI();
TexturePtr texture = scene->requestWebTexture(
std::move( request ), [url = std::move( url )]( const WebResourceResult& result ) {
if ( !result.success )
Log::debug( "UINodeDrawable::LayerDrawable::loadRemoteDrawable: could not "
"download image: %s. Error: %d\n%s",
url, response.getStatus(), response.getBody() );
}
},
uri, Seconds( 5 ), {}, headers, "", true, Http::getEnvProxyURI() );
url, result.status, result.error );
} );
if ( texture ) {
TextureDrawable* current =
mDrawable && mDrawable->getDrawableType() == Drawable::TEXTUREDRAWABLE
? static_cast<TextureDrawable*>( mDrawable.get() )
: nullptr;
if ( !current || current->getTexture() != texture )
setDrawable( std::move( texture ) );
}
return true;
}
+154 -60
View File
@@ -143,6 +143,7 @@ UISceneNode::UISceneNode( EE::Window::Window* window ) :
mAsyncResourceLoadState( std::make_shared<AsyncResourceLoadState>() ),
mResourceScope( ResourceScope::New() ),
mDrawableResolver( *this ),
mWebResourceCache( WebResourceCache::New() ),
mKeyBindings( mWindow->getInput() ) {
// Reset size since the SceneNode already set it but needs to set the size from zero to emit
// the required events to its children.
@@ -160,6 +161,7 @@ UISceneNode::UISceneNode( EE::Window::Window* window ) :
mRoot->setParent( this )->setPosition( 0, 0 )->setId( "uiscenenode_root_node" );
mRoot->enableReportSizeChangeToChildren();
mAsyncResourceLoadState->owner.store( this, std::memory_order_release );
mDocumentSessionId = mWebResourceCache->createSession();
resizeNode( mWindow );
}
@@ -175,6 +177,8 @@ UISceneNode::~UISceneNode() {
mHostUISceneNode->unregisterChildUISceneNode( this );
clearFontFaces();
if ( mWebResourceCache && mDocumentSessionId )
mWebResourceCache->destroySession( mDocumentSessionId );
eeSAFE_DELETE( mUIThemeManager );
eeSAFE_DELETE( mUIIconThemeManager );
@@ -762,6 +766,94 @@ UISceneNode* UISceneNode::setResourceScope( ResourceScopePtr resourceScope ) {
return this;
}
UISceneNode* UISceneNode::setWebResourceCache( WebResourceCachePtr cache,
CachePartitionId partition ) {
if ( !cache )
cache = WebResourceCache::New();
if ( cache == mWebResourceCache &&
( partition == 0 || partition == cache->getSessionPartition( mDocumentSessionId ) ) )
return this;
if ( mWebResourceCache && mDocumentSessionId )
mWebResourceCache->destroySession( mDocumentSessionId );
mWebResourceCache = std::move( cache );
mDocumentSessionId = mWebResourceCache->createSession( partition );
return this;
}
Uint64 UISceneNode::beginDocumentNavigation( const URI& uri ) {
return mWebResourceCache && mDocumentSessionId
? mWebResourceCache->beginNavigation( mDocumentSessionId, uri )
: 0;
}
Uint64 UISceneNode::getDocumentGeneration() const {
return mWebResourceCache && mDocumentSessionId
? mWebResourceCache->getSessionGeneration( mDocumentSessionId )
: 0;
}
void UISceneNode::requestWebResource( WebResourceRequest request,
WebResourceCache::Callback callback ) {
if ( !mWebResourceCache || !mDocumentSessionId )
return;
if ( !mReferer.empty() )
request.headers.emplace( "referer", mReferer.toString() );
std::string cookie = mCookieManager.getCookieHeader( request.uri.getAuthority() );
if ( !cookie.empty() )
request.headers["Cookie"] = std::move( cookie );
auto resourceState = mAsyncResourceLoadState;
Uint64 resourceGeneration =
resourceState ? resourceState->generation.load( std::memory_order_acquire ) : 0;
auto wrapped = [resourceState, resourceGeneration, callback = std::move( callback ),
authority = request.uri.getAuthority()]( const WebResourceResult& result ) {
if ( !UISceneNode::isAsyncResourceLoadCurrent( resourceState, resourceGeneration ) )
return;
UISceneNode* scene = resourceState->owner.load( std::memory_order_acquire );
if ( !scene )
return;
if ( !result.setCookie.empty() )
scene->mCookieManager.storeCookiesFromHeader( authority, result.setCookie );
if ( callback )
callback( result );
};
mWebResourceCache->requestData( mDocumentSessionId, getDocumentGeneration(),
std::move( request ), std::move( wrapped ) );
}
TexturePtr UISceneNode::requestWebTexture( WebResourceRequest request,
WebResourceCache::Callback callback ) {
if ( !mWebResourceCache || !mDocumentSessionId )
return {};
if ( !mReferer.empty() )
request.headers.emplace( "referer", mReferer.toString() );
std::string cookie = mCookieManager.getCookieHeader( request.uri.getAuthority() );
if ( !cookie.empty() )
request.headers["Cookie"] = std::move( cookie );
auto resourceState = mAsyncResourceLoadState;
Uint64 resourceGeneration =
resourceState ? resourceState->generation.load( std::memory_order_acquire ) : 0;
request.completionDispatcher = [resourceState,
resourceGeneration]( std::function<void()> completion ) {
UISceneNode::runAsyncResourceOnMainThread(
resourceState, resourceGeneration,
[completion = std::move( completion )]( UISceneNode* ) { completion(); } );
};
auto wrapped = [resourceState, resourceGeneration, callback = std::move( callback ),
authority = request.uri.getAuthority()]( const WebResourceResult& result ) {
if ( !UISceneNode::isAsyncResourceLoadCurrent( resourceState, resourceGeneration ) )
return;
UISceneNode* scene = resourceState->owner.load( std::memory_order_acquire );
if ( !scene )
return;
if ( !result.setCookie.empty() )
scene->mCookieManager.storeCookiesFromHeader( authority, result.setCookie );
if ( callback )
callback( result );
};
return mWebResourceCache->requestTexture( mDocumentSessionId, getDocumentGeneration(),
std::move( request ), std::move( wrapped ) );
}
static std::string getErrorContext( size_t offset, std::string_view content ) {
static constexpr std::size_t CONTEXT_LENGTH = 50;
std::size_t minVal = offset >= CONTEXT_LENGTH ? offset - CONTEXT_LENGTH : 0;
@@ -1632,47 +1724,48 @@ void UISceneNode::loadFontFaces( const StyleSheetStyleVector& styles, URI baseUR
auto resourceState = mAsyncResourceLoadState;
Uint64 resourceGeneration =
resourceState ? resourceState->generation.load( std::memory_order_acquire ) : 0;
Http::getAsync(
[resourceState, resourceGeneration, internalFontName, authorFamily, fontStyle,
fontWeight, path]( const Http&, Http::Request&, Http::Response& response ) {
if ( !UISceneNode::isAsyncResourceLoadCurrent( resourceState,
resourceGeneration ) )
return;
WebResourceRequest request;
request.uri = URI( path );
request.kind = WebResourceKind::Font;
request.timeout = Seconds( 5 );
requestWebResource( std::move( request ), [resourceState, resourceGeneration,
internalFontName, authorFamily, fontStyle,
fontWeight,
path]( const WebResourceResult& result ) {
if ( !UISceneNode::isAsyncResourceLoadCurrent( resourceState, resourceGeneration ) )
return;
if ( response.isOK() && !response.getBody().empty() ) {
std::string fontData( response.getBody() );
UISceneNode::runAsyncResourceOnMainThread(
resourceState, resourceGeneration,
[fontData = std::move( fontData ), internalFontName, authorFamily,
fontStyle, fontWeight]( UISceneNode* scene ) mutable {
FontTrueType* font = FontTrueType::New( internalFontName );
if ( font->loadFromMemory( &fontData[0], fontData.size() ) &&
font->loaded() ) {
font->setVariableFontWeight( fontWeight );
scene->registerFontFaceAlias( authorFamily, fontStyle,
fontWeight, font );
scene->mFontFaces.push_back( font );
if ( scene->mRoot )
scene->mRoot->reloadFontFamily();
} else {
eeSAFE_DELETE( font );
}
} );
} else {
UISceneNode::runAsyncResourceOnMainThread(
resourceState, resourceGeneration,
[internalFontName, path, status = response.getStatus(),
statusDescription =
std::string( response.getStatusDescription() )]( UISceneNode* ) {
Log::error( "UISceneNode::loadFontFaces: Failed to load font "
"\"%s\", from: %s. Request response status code: %d "
"(%s)",
internalFontName, path, status,
statusDescription.c_str() );
} );
}
},
URI( path ), Seconds( 5 ) );
if ( result.success && result.data && !result.data->empty() ) {
std::string fontData( *result.data );
UISceneNode::runAsyncResourceOnMainThread(
resourceState, resourceGeneration,
[fontData = std::move( fontData ), internalFontName, authorFamily,
fontStyle, fontWeight]( UISceneNode* scene ) mutable {
FontTrueType* font = FontTrueType::New( internalFontName );
if ( font->loadFromMemory( &fontData[0], fontData.size() ) &&
font->loaded() ) {
font->setVariableFontWeight( fontWeight );
scene->registerFontFaceAlias( authorFamily, fontStyle, fontWeight,
font );
scene->mFontFaces.push_back( font );
if ( scene->mRoot )
scene->mRoot->reloadFontFamily();
} else {
eeSAFE_DELETE( font );
}
} );
} else {
UISceneNode::runAsyncResourceOnMainThread(
resourceState, resourceGeneration,
[internalFontName, path, status = result.status,
statusDescription = result.error]( UISceneNode* ) {
Log::error( "UISceneNode::loadFontFaces: Failed to load font "
"\"%s\", from: %s. Request response status code: %d "
"(%s)",
internalFontName, path, status, statusDescription.c_str() );
} );
}
} );
} else if ( VFS::instance()->fileExists( path ) ) {
FontTrueType* font =
FontTrueType::New( makeInternalFontName( authorFamily, fontStyle, fontWeight ) );
@@ -1786,26 +1879,27 @@ void UISceneNode::loadCSS( URI uri, std::optional<Time> defer ) {
Uint64 resourceGeneration =
resourceState ? resourceState->generation.load( std::memory_order_acquire ) : 0;
URI baseURL = getURIFromURL( url );
Http::getAsync(
[resourceState, resourceGeneration, url, baseURL = std::move( baseURL )](
const Http&, Http::Request&, Http::Response& response ) {
if ( !UISceneNode::isAsyncResourceLoadCurrent( resourceState, resourceGeneration ) )
return;
if ( !response.getBody().empty() &&
response.getStatus() == Http::Response::Status::Ok ) {
std::string css( response.getBody() );
UISceneNode::runAsyncResourceOnMainThread(
resourceState, resourceGeneration,
[css = std::move( css ), url, baseURL]( UISceneNode* scene ) mutable {
scene->combineStyleSheet( css, true, String::hash( url ), baseURL );
Log::debug( "UISceneNode::loadCSS: Loaded - %s", url );
} );
} else {
Log::debug( "UISceneNode::loadCSS: Failed to load %s - %s", url,
response.getStatusDescription() );
}
},
uri, Seconds( 5 ) );
WebResourceRequest request;
request.uri = uri;
request.kind = WebResourceKind::StyleSheet;
request.timeout = Seconds( 5 );
requestWebResource( std::move( request ), [resourceState, resourceGeneration, url,
baseURL = std::move( baseURL )](
const WebResourceResult& result ) {
if ( !UISceneNode::isAsyncResourceLoadCurrent( resourceState, resourceGeneration ) )
return;
if ( result.success && result.data && !result.data->empty() ) {
std::string css( *result.data );
UISceneNode::runAsyncResourceOnMainThread(
resourceState, resourceGeneration,
[css = std::move( css ), url, baseURL]( UISceneNode* scene ) mutable {
scene->combineStyleSheet( css, true, String::hash( url ), baseURL );
Log::debug( "UISceneNode::loadCSS: Loaded - %s", url );
} );
} else {
Log::debug( "UISceneNode::loadCSS: Failed to load %s - %s", url, result.error );
}
} );
} else if ( VFS::instance()->fileExists( uri.getPath() ) ) {
IOStream* stream = VFS::instance()->getFileFromPath( uri.getPath() );
CSS::StyleSheetParser parser;
+31 -42
View File
@@ -266,6 +266,12 @@ void UIWebView::scheduledUpdate( const Time& time ) {
if ( mDocumentScene ) {
mDocumentScene->update( time );
updateDocumentMetricsIfNeeded();
mWebResourceCachePruneElapsed += time;
if ( mWebResourceCachePruneElapsed >= Seconds( 1 ) ) {
mWebResourceCachePruneElapsed = Time::Zero;
if ( const auto& cache = mDocumentScene->getWebResourceCache() )
cache->prune();
}
}
}
@@ -290,6 +296,8 @@ void UIWebView::loadURI( URI uri, const std::string& method, const std::string&
void UIWebView::loadURI( URI uri, bool isHistoryNav, const std::string& method,
const std::string& body, const Http::Request::FieldTable& headers ) {
Uint64 generation = beginNavigationLoad();
if ( mDocumentScene )
mDocumentScene->beginDocumentNavigation( uri );
mIsLoading = true;
if ( !isHistoryNav )
@@ -348,61 +356,32 @@ void UIWebView::loadDocumentAsync( const URI& url, const std::string& method,
}
auto reqHeaders = headers;
auto& cookieManager = ui->getCookieManager();
if ( cookieManager.hasCookie( url.getAuthority() ) ) {
std::string cookieHeader = cookieManager.getCookieHeader( url.getAuthority() );
if ( !cookieHeader.empty() )
reqHeaders["Cookie"] = cookieHeader;
}
if ( !mUserAgent.empty() )
reqHeaders["User-Agent"] = mUserAgent;
Http::AsyncResponseCallback responseCb =
[loadState, generation, url]( const Http&, Http::Request&, Http::Response& response ) {
WebResourceRequest request;
request.uri = url;
request.kind = WebResourceKind::Document;
request.method = method == "POST" ? Http::Request::Method::Post : Http::Request::Method::Get;
request.headers = std::move( reqHeaders );
request.body = body;
request.timeout = mDefaultTimeout;
ui->requestWebResource(
std::move( request ), [loadState, generation, url]( const WebResourceResult& result ) {
UIWebView* self = resolveNavigationLoad( loadState, generation );
if ( !self )
return;
auto ui = self->getDocumentSceneNode();
if ( !ui )
return;
if ( response.isOK() ) {
std::string data = response.getBody();
if ( response.hasField( "set-cookie" ) ) {
ui->getCookieManager().storeCookiesFromHeader(
url.getAuthority(), response.getField( "set-cookie" ) );
}
if ( result.success && result.data ) {
std::string data( *result.data );
self->loadDocumentData( url, std::move( data ), generation );
} else {
if ( !self->isNavigationLoadCurrent( generation ) )
return;
self->mIsLoading = false;
NavigationEvent ev( self, Event::OnNavigationError, url, false,
response.getStatusDescription() );
NavigationEvent ev( self, Event::OnNavigationError, url, false, result.error );
self->sendEvent( &ev );
}
};
Http::Request::ProgressCallback progressCb =
[loadState, generation,
url]( const Http&, const Http::Request&, const Http::Response& response,
const Http::Request::Status& status, std::size_t, std::size_t ) {
UIWebView* self = resolveNavigationLoad( loadState, generation );
if ( !self )
return false;
auto ui = self->getDocumentSceneNode();
if ( !ui )
return false;
if ( status == Http::Request::Status::Redirect && response.hasField( "set-cookie" ) ) {
ui->getCookieManager().storeCookiesFromHeader( url.getAuthority(),
response.getField( "set-cookie" ) );
}
return true;
};
Http::requestAsync( responseCb, url, mDefaultTimeout,
method == "POST" ? Http::Request::Method::Post : Http::Request::Method::Get,
progressCb, reqHeaders, body, true, {} );
} );
}
void UIWebView::loadDocumentData( URI url, std::string data ) {
@@ -546,6 +525,16 @@ UISceneNode* UIWebView::getDocumentSceneNode() const {
return mDocumentScene;
}
const WebResourceCachePtr& UIWebView::getWebResourceCache() const {
return mDocumentScene->getWebResourceCache();
}
UIWebView* UIWebView::setWebResourceCache( WebResourceCachePtr cache, CachePartitionId partition ) {
if ( mDocumentScene )
mDocumentScene->setWebResourceCache( std::move( cache ), partition );
return this;
}
bool UIWebView::updateDocumentViewportMetrics() {
if ( !mDocumentScene || !mDocumentLayout || !mDocContainer || mUpdatingDocumentViewportMetrics )
return false;
+476
View File
@@ -0,0 +1,476 @@
#include <eepp/core/small_vector.hpp>
#include <eepp/graphics/image.hpp>
#include <eepp/graphics/texturefactory.hpp>
#include <eepp/system/clock.hpp>
#include <eepp/ui/webresourcecache.hpp>
#include <algorithm>
#include <atomic>
#include <mutex>
#include <vector>
using namespace EE::Graphics;
using namespace EE::Network;
using namespace EE::System;
namespace EE { namespace UI {
namespace {
std::atomic<Uint64> sNextDocumentSessionId{ 1 };
std::atomic<Uint64> sNextCachePartitionId{ 1 };
std::string canonicalURI( URI uri ) {
uri.setFragment( "" );
uri.normalize();
return uri.toString();
}
std::string makeRequestKey( CachePartitionId partition, const WebResourceRequest& request ) {
std::string key =
String::format( "%llu\n%u\n%u\n%s\n", static_cast<unsigned long long>( partition ),
static_cast<Uint32>( request.kind ), static_cast<Uint32>( request.method ),
canonicalURI( request.uri ).c_str() );
for ( const auto& header : request.headers ) {
std::string name = String::toLower( header.first );
// The partition identifies the cookie/authentication context. Cookie values evolve while a
// document is being used, and Referer changes with navigation history; including either one
// would duplicate an otherwise reusable resource on Back/Forward navigation.
if ( name == "cookie" || name == "referer" )
continue;
key += name;
key += ':';
key += header.second;
key += '\n';
}
key += request.body;
key += String::format( "\n%d:%d:%d:%.4f", request.clampToEdge, request.mipmaps,
request.compressTexture, request.svgScale );
return key;
}
} // namespace
bool WebOriginKey::operator==( const WebOriginKey& other ) const {
return scheme == other.scheme && host == other.host && port == other.port;
}
WebOriginKey WebOriginKey::fromURI( const URI& uri ) {
return { String::toLower( uri.getScheme() ), String::toLower( uri.getHost() ), uri.getPort() };
}
struct WebResourceCache::Impl {
struct Subscriber {
DocumentSessionId session{ 0 };
Uint64 generation{ 0 };
Callback callback;
};
struct Entry {
WebResourceLoadState state{ WebResourceLoadState::Empty };
WebResourceRequest request;
std::shared_ptr<const std::string> data;
TexturePtr texture;
std::vector<Subscriber> subscribers;
SmallVector<DocumentSessionId, 4> activeLeases;
Time lastUsed;
Time expiresAt;
Time retryAt;
std::size_t retainedBytes{ 0 };
int status{ 0 };
std::string error;
};
struct Session {
CachePartitionId partition{ 0 };
Uint64 generation{ 0 };
WebOriginKey origin;
};
mutable std::mutex mutex;
Clock clock;
UnorderedMap<DocumentSessionId, Session> sessions;
UnorderedMap<std::string, Entry> entries;
Time ttl{ Seconds( 30 ) };
Time retryDelay{ Seconds( 1 ) };
std::size_t byteBudget{ 64 * 1024 * 1024 };
std::size_t retainedBytes{ 0 };
std::size_t inFlightCount{ 0 };
Fetcher fetcher;
Time now() const { return clock.getElapsedTime(); }
bool isCurrent( const Subscriber& subscriber ) const {
auto it = sessions.find( subscriber.session );
return it != sessions.end() && it->second.generation == subscriber.generation;
}
void releaseSessionLeases( DocumentSessionId session ) {
const Time releasedAt = now();
for ( auto& cacheEntry : entries ) {
Entry& entry = cacheEntry.second;
auto lease = std::find( entry.activeLeases.begin(), entry.activeLeases.end(), session );
if ( lease != entry.activeLeases.end() ) {
entry.activeLeases.erase( lease );
if ( entry.activeLeases.empty() ) {
entry.lastUsed = releasedAt;
entry.expiresAt = releasedAt + ttl;
}
}
}
}
void lease( DocumentSessionId session, Entry& entry ) {
if ( std::find( entry.activeLeases.begin(), entry.activeLeases.end(), session ) ==
entry.activeLeases.end() )
entry.activeLeases.push_back( session );
entry.lastUsed = now();
entry.expiresAt = entry.lastUsed + ttl;
}
UnorderedMap<std::string, Entry>::iterator
eraseEntry( UnorderedMap<std::string, Entry>::iterator it ) {
retainedBytes -= std::min( retainedBytes, it->second.retainedBytes );
return entries.erase( it );
}
void pruneLocked() {
Time current = now();
for ( auto it = entries.begin(); it != entries.end(); ) {
if ( it->second.state != WebResourceLoadState::Loading &&
it->second.activeLeases.empty() && it->second.expiresAt <= current ) {
it = eraseEntry( it );
} else {
++it;
}
}
while ( retainedBytes > byteBudget ) {
auto oldest = entries.end();
for ( auto it = entries.begin(); it != entries.end(); ++it ) {
if ( it->second.state == WebResourceLoadState::Loading ||
!it->second.activeLeases.empty() )
continue;
if ( oldest == entries.end() || it->second.lastUsed < oldest->second.lastUsed )
oldest = it;
}
if ( oldest == entries.end() )
break;
eraseEntry( oldest );
}
}
};
WebResourceCachePtr WebResourceCache::New() {
return std::make_shared<WebResourceCache>();
}
WebResourceCache::WebResourceCache() : mImpl( std::make_unique<Impl>() ) {}
WebResourceCache::~WebResourceCache() {
clear();
}
CachePartitionId WebResourceCache::createPartition() {
return sNextCachePartitionId.fetch_add( 1, std::memory_order_relaxed );
}
DocumentSessionId WebResourceCache::createSession( CachePartitionId partition ) {
if ( partition == 0 )
partition = createPartition();
DocumentSessionId id = sNextDocumentSessionId.fetch_add( 1, std::memory_order_relaxed );
std::lock_guard<std::mutex> lock( mImpl->mutex );
mImpl->sessions.emplace( id, Impl::Session{ partition } );
return id;
}
void WebResourceCache::destroySession( DocumentSessionId session ) {
std::lock_guard<std::mutex> lock( mImpl->mutex );
auto it = mImpl->sessions.find( session );
if ( it == mImpl->sessions.end() )
return;
mImpl->releaseSessionLeases( session );
mImpl->sessions.erase( it );
mImpl->pruneLocked();
}
Uint64 WebResourceCache::beginNavigation( DocumentSessionId session, const URI& uri ) {
std::lock_guard<std::mutex> lock( mImpl->mutex );
auto it = mImpl->sessions.find( session );
if ( it == mImpl->sessions.end() )
return 0;
mImpl->releaseSessionLeases( session );
it->second.origin = WebOriginKey::fromURI( uri );
return ++it->second.generation;
}
Uint64 WebResourceCache::getSessionGeneration( DocumentSessionId session ) const {
std::lock_guard<std::mutex> lock( mImpl->mutex );
auto it = mImpl->sessions.find( session );
return it != mImpl->sessions.end() ? it->second.generation : 0;
}
CachePartitionId WebResourceCache::getSessionPartition( DocumentSessionId session ) const {
std::lock_guard<std::mutex> lock( mImpl->mutex );
auto it = mImpl->sessions.find( session );
return it != mImpl->sessions.end() ? it->second.partition : 0;
}
void WebResourceCache::requestData( DocumentSessionId session, Uint64 generation,
WebResourceRequest request, Callback callback ) {
std::string key;
WebResourceResult immediate;
bool hasImmediate = false;
bool startRequest = false;
Fetcher fetcher;
{
std::lock_guard<std::mutex> lock( mImpl->mutex );
auto sessionIt = mImpl->sessions.find( session );
if ( sessionIt == mImpl->sessions.end() || sessionIt->second.generation != generation )
return;
key = makeRequestKey( sessionIt->second.partition, request );
auto entryIt = mImpl->entries.try_emplace( key ).first;
auto& entry = entryIt->second;
mImpl->lease( session, entry );
if ( entry.state == WebResourceLoadState::Ready ) {
immediate = { true, entry.state, entry.status, {}, {}, entry.data, entry.texture };
hasImmediate = true;
} else if ( entry.state == WebResourceLoadState::Failed && entry.retryAt > mImpl->now() ) {
immediate = { false, entry.state, entry.status, entry.error, {}, {}, entry.texture };
hasImmediate = true;
} else {
if ( callback )
entry.subscribers.push_back( { session, generation, std::move( callback ) } );
if ( entry.state != WebResourceLoadState::Loading ) {
entry.state = WebResourceLoadState::Loading;
entry.request = request;
entry.error.clear();
++mImpl->inFlightCount;
startRequest = true;
fetcher = mImpl->fetcher;
}
}
}
if ( hasImmediate ) {
if ( callback )
callback( immediate );
return;
}
if ( !startRequest )
return;
std::shared_ptr<std::string> redirectCookie;
if ( !fetcher )
redirectCookie = std::make_shared<std::string>();
auto weak = weak_from_this();
auto process = [weak, key, redirectCookie]( Http::Response& response ) {
auto self = weak.lock();
if ( !self )
return;
std::vector<Impl::Subscriber> subscribers;
WebResourceResult result;
{
std::lock_guard<std::mutex> lock( self->mImpl->mutex );
auto entryIt = self->mImpl->entries.find( key );
if ( entryIt == self->mImpl->entries.end() ||
entryIt->second.state != WebResourceLoadState::Loading )
return;
auto& entry = entryIt->second;
self->mImpl->inFlightCount--;
entry.status = static_cast<int>( response.getStatus() );
entry.lastUsed = self->mImpl->now();
entry.expiresAt = entry.lastUsed + self->mImpl->ttl;
std::string setCookie = response.getField( "set-cookie" );
if ( setCookie.empty() && redirectCookie )
setCookie = *redirectCookie;
if ( response.isOK() && !response.getBody().empty() ) {
if ( entry.request.kind == WebResourceKind::Image ) {
Image image( reinterpret_cast<const Uint8*>( response.getBody().data() ),
response.getBody().size() );
if ( image.getPixels() && entry.texture ) {
entry.texture->replace( &image );
entry.retainedBytes =
static_cast<std::size_t>( entry.texture->getSize().getWidth() *
entry.texture->getSize().getHeight() * 4 );
} else {
entry.state = WebResourceLoadState::Failed;
entry.error = "Invalid image data";
}
} else {
entry.data = std::make_shared<const std::string>( response.getBody() );
entry.retainedBytes = entry.data->size();
}
if ( entry.state != WebResourceLoadState::Failed ) {
entry.state = WebResourceLoadState::Ready;
self->mImpl->retainedBytes += entry.retainedBytes;
result = { true, entry.state, entry.status, {}, std::move( setCookie ),
entry.data, entry.texture };
} else {
result = {
false, entry.state, entry.status, entry.error, std::move( setCookie ),
{}, entry.texture };
}
} else {
entry.state = WebResourceLoadState::Failed;
entry.error = response.getStatusDescription();
result = { false, entry.state, entry.status, entry.error, std::move( setCookie ),
{}, entry.texture };
}
if ( entry.state == WebResourceLoadState::Failed )
entry.retryAt = entry.lastUsed + self->mImpl->retryDelay;
entry.request = {};
subscribers.swap( entry.subscribers );
self->mImpl->pruneLocked();
for ( auto it = subscribers.begin(); it != subscribers.end(); ) {
if ( !self->mImpl->isCurrent( *it ) )
it = subscribers.erase( it );
else
++it;
}
}
for ( auto& subscriber : subscribers ) {
if ( subscriber.callback )
subscriber.callback( result );
}
};
auto complete = [process = std::move( process ), kind = request.kind,
dispatcher = request.completionDispatcher]( Http::Response response ) mutable {
if ( kind == WebResourceKind::Image && dispatcher ) {
auto sharedResponse = std::make_shared<Http::Response>( std::move( response ) );
dispatcher(
[process = std::move( process ), sharedResponse = std::move( sharedResponse )]() {
process( *sharedResponse );
} );
} else {
process( response );
}
};
if ( fetcher ) {
fetcher( request, std::move( complete ) );
} else {
Http::Request::ProgressCallback progress =
[redirectCookie]( const Http&, const Http::Request&, const Http::Response& response,
const Http::Request::Status& status, std::size_t, std::size_t ) {
if ( status == Http::Request::Status::Redirect &&
response.hasField( "set-cookie" ) )
*redirectCookie = response.getField( "set-cookie" );
return true;
};
Http::requestAsync(
[complete = std::move( complete )]( const Http&, Http::Request&,
Http::Response& response ) mutable {
complete( std::move( response ) );
},
request.uri, request.timeout, request.method, progress, request.headers, request.body,
request.validateCertificate, request.proxy, request.followRedirect );
}
}
TexturePtr WebResourceCache::requestTexture( DocumentSessionId session, Uint64 generation,
WebResourceRequest request, Callback callback ) {
request.kind = WebResourceKind::Image;
std::string key;
TexturePtr texture;
{
std::lock_guard<std::mutex> lock( mImpl->mutex );
auto sessionIt = mImpl->sessions.find( session );
if ( sessionIt == mImpl->sessions.end() || sessionIt->second.generation != generation )
return {};
key = makeRequestKey( sessionIt->second.partition, request );
auto entry = mImpl->entries.find( key );
if ( entry != mImpl->entries.end() )
texture = entry->second.texture;
}
if ( !texture ) {
texture = TextureFactory::instance()->createEmptyTexture(
1, 1, 4, Color::Transparent, false,
request.clampToEdge ? Texture::ClampMode::ClampToEdge : Texture::ClampMode::ClampRepeat,
request.mipmaps, request.compressTexture, canonicalURI( request.uri ) );
std::lock_guard<std::mutex> lock( mImpl->mutex );
auto sessionIt = mImpl->sessions.find( session );
if ( sessionIt == mImpl->sessions.end() || sessionIt->second.generation != generation )
return {};
auto entry = mImpl->entries.try_emplace( key ).first;
if ( !entry->second.texture )
entry->second.texture = texture;
else
texture = entry->second.texture;
}
requestData( session, generation, std::move( request ), std::move( callback ) );
return texture;
}
void WebResourceCache::setTTL( Time ttl ) {
std::lock_guard<std::mutex> lock( mImpl->mutex );
mImpl->ttl = ttl;
}
Time WebResourceCache::getTTL() const {
std::lock_guard<std::mutex> lock( mImpl->mutex );
return mImpl->ttl;
}
void WebResourceCache::setRetryDelay( Time delay ) {
std::lock_guard<std::mutex> lock( mImpl->mutex );
mImpl->retryDelay = delay;
const Time retryAt = mImpl->now() + delay;
for ( auto& entry : mImpl->entries ) {
if ( entry.second.state == WebResourceLoadState::Failed )
entry.second.retryAt = retryAt;
}
}
Time WebResourceCache::getRetryDelay() const {
std::lock_guard<std::mutex> lock( mImpl->mutex );
return mImpl->retryDelay;
}
void WebResourceCache::setByteBudget( std::size_t bytes ) {
std::lock_guard<std::mutex> lock( mImpl->mutex );
mImpl->byteBudget = bytes;
mImpl->pruneLocked();
}
std::size_t WebResourceCache::getByteBudget() const {
std::lock_guard<std::mutex> lock( mImpl->mutex );
return mImpl->byteBudget;
}
std::size_t WebResourceCache::getRetainedBytes() const {
std::lock_guard<std::mutex> lock( mImpl->mutex );
return mImpl->retainedBytes;
}
std::size_t WebResourceCache::getEntryCount() const {
std::lock_guard<std::mutex> lock( mImpl->mutex );
return mImpl->entries.size();
}
std::size_t WebResourceCache::getInFlightCount() const {
std::lock_guard<std::mutex> lock( mImpl->mutex );
return mImpl->inFlightCount;
}
void WebResourceCache::prune() {
std::lock_guard<std::mutex> lock( mImpl->mutex );
mImpl->pruneLocked();
}
void WebResourceCache::clear() {
std::lock_guard<std::mutex> lock( mImpl->mutex );
for ( auto& entry : mImpl->entries ) {
entry.second.state = WebResourceLoadState::Cancelled;
entry.second.subscribers.clear();
}
mImpl->entries.clear();
mImpl->retainedBytes = 0;
mImpl->inFlightCount = 0;
}
void WebResourceCache::setFetcher( Fetcher fetcher ) {
std::lock_guard<std::mutex> lock( mImpl->mutex );
mImpl->fetcher = std::move( fetcher );
}
}} // namespace EE::UI
@@ -0,0 +1,261 @@
#include "utest.hpp"
#include <eepp/ui/webresourcecache.hpp>
#include <vector>
using namespace EE;
using namespace EE::Network;
using namespace EE::System;
using namespace EE::UI;
namespace {
Http::Response response( Http::Response::Status status, const std::string& body = {} ) {
Http::Response::FieldTable fields;
return Http::Response::createFakeResponse( fields, status, body );
}
WebResourceRequest requestFor( const char* uri ) {
WebResourceRequest request;
request.uri = URI( uri );
request.kind = WebResourceKind::StyleSheet;
return request;
}
} // namespace
UTEST( WebResourceCache, coalescesSubscribersInOnePartition ) {
auto cache = WebResourceCache::New();
std::vector<WebResourceCache::FetchCompletion> completions;
int fetches = 0;
cache->setFetcher( [&fetches, &completions]( const WebResourceRequest&,
WebResourceCache::FetchCompletion completion ) {
++fetches;
completions.emplace_back( std::move( completion ) );
} );
constexpr CachePartitionId partition = 7;
auto first = cache->createSession( partition );
auto second = cache->createSession( partition );
auto firstGeneration = cache->beginNavigation( first, URI( "https://example.com/a" ) );
auto secondGeneration = cache->beginNavigation( second, URI( "https://example.com/b" ) );
int deliveries = 0;
bool allSuccessful = true;
cache->requestData( first, firstGeneration, requestFor( "https://example.com/site.css#one" ),
[&deliveries, &allSuccessful]( const WebResourceResult& result ) {
allSuccessful &= result.success;
++deliveries;
} );
cache->requestData( second, secondGeneration, requestFor( "https://example.com/site.css#two" ),
[&deliveries, &allSuccessful]( const WebResourceResult& result ) {
allSuccessful &= result.success;
++deliveries;
} );
EXPECT_EQ( 1, fetches );
EXPECT_EQ( 1u, completions.size() );
auto ok = Http::Response::Status::Ok;
completions.front()( response( ok, "body" ) );
EXPECT_EQ( 2, deliveries );
EXPECT_TRUE( allSuccessful );
cache->requestData( first, firstGeneration, requestFor( "https://example.com/site.css" ),
[&deliveries]( const WebResourceResult& ) { ++deliveries; } );
EXPECT_EQ( 1, fetches );
EXPECT_EQ( 3, deliveries );
EXPECT_EQ( 0u, cache->getInFlightCount() );
}
UTEST( WebResourceCache, staleDocumentSubscriberDoesNotBlockCurrentDocument ) {
auto cache = WebResourceCache::New();
WebResourceCache::FetchCompletion completion;
cache->setFetcher(
[&completion]( const WebResourceRequest&, WebResourceCache::FetchCompletion cb ) {
completion = std::move( cb );
} );
auto session = cache->createSession( 11 );
auto oldGeneration = cache->beginNavigation( session, URI( "https://example.com/old" ) );
int staleDeliveries = 0;
int currentDeliveries = 0;
cache->requestData( session, oldGeneration, requestFor( "https://example.com/shared.css" ),
[&staleDeliveries]( const WebResourceResult& ) { ++staleDeliveries; } );
auto currentGeneration =
cache->beginNavigation( session, URI( "https://example.com/current" ) );
cache->requestData( session, currentGeneration, requestFor( "https://example.com/shared.css" ),
[&currentDeliveries]( const WebResourceResult& ) { ++currentDeliveries; } );
auto ok = Http::Response::Status::Ok;
completion( response( ok, "body" ) );
EXPECT_EQ( 0, staleDeliveries );
EXPECT_EQ( 1, currentDeliveries );
}
UTEST( WebResourceCache, partitionsDoNotShareEntries ) {
auto cache = WebResourceCache::New();
int fetches = 0;
cache->setFetcher(
[&fetches]( const WebResourceRequest&, WebResourceCache::FetchCompletion completion ) {
++fetches;
auto ok = Http::Response::Status::Ok;
completion( response( ok, "body" ) );
} );
auto first = cache->createSession( 1 );
auto second = cache->createSession( 2 );
auto firstGeneration = cache->beginNavigation( first, URI( "https://example.com/a" ) );
auto secondGeneration = cache->beginNavigation( second, URI( "https://example.com/b" ) );
cache->requestData( first, firstGeneration, requestFor( "https://example.com/site.css" ), {} );
cache->requestData( second, secondGeneration, requestFor( "https://example.com/site.css" ),
{} );
EXPECT_EQ( 2, fetches );
}
UTEST( WebResourceCache, decodeAndRequestVariantsUseDistinctEntries ) {
auto cache = WebResourceCache::New();
int fetches = 0;
cache->setFetcher(
[&fetches]( const WebResourceRequest&, WebResourceCache::FetchCompletion completion ) {
++fetches;
auto ok = Http::Response::Status::Ok;
completion( response( ok, "body" ) );
} );
auto session = cache->createSession( 5 );
auto generation = cache->beginNavigation( session, URI( "https://example.com/" ) );
auto base = requestFor( "https://example.com/resource" );
cache->requestData( session, generation, base, {} );
base.svgScale = 2.f;
cache->requestData( session, generation, base, {} );
base.headers["Accept-Language"] = "es";
cache->requestData( session, generation, base, {} );
EXPECT_EQ( 3, fetches );
}
UTEST( WebResourceCache, navigationHeadersDoNotDuplicateRetainedEntries ) {
auto cache = WebResourceCache::New();
int fetches = 0;
cache->setFetcher(
[&fetches]( const WebResourceRequest&, WebResourceCache::FetchCompletion completion ) {
++fetches;
auto ok = Http::Response::Status::Ok;
completion( response( ok, "body" ) );
} );
auto session = cache->createSession( 9 );
auto generation = cache->beginNavigation( session, URI( "https://news.example/" ) );
auto request = requestFor( "https://news.example/logo.png" );
request.headers["referer"] = "https://news.example/";
cache->requestData( session, generation, request, {} );
cache->beginNavigation( session, URI( "https://other.example/" ) );
generation = cache->beginNavigation( session, URI( "https://news.example/" ) );
request.headers["Referer"] = "https://other.example/";
request.headers["Cookie"] = "session=now-present";
cache->requestData( session, generation, request, {} );
EXPECT_EQ( 1, fetches );
EXPECT_EQ( 1u, cache->getEntryCount() );
}
UTEST( WebResourceCache, ttlStartsWhenFinalDocumentLeaseIsReleased ) {
auto cache = WebResourceCache::New();
cache->setTTL( Time::Zero );
int fetches = 0;
cache->setFetcher(
[&fetches]( const WebResourceRequest&, WebResourceCache::FetchCompletion completion ) {
++fetches;
auto ok = Http::Response::Status::Ok;
completion( response( ok, "body" ) );
} );
auto session = cache->createSession();
auto generation = cache->beginNavigation( session, URI( "https://news.example/" ) );
auto request = requestFor( "https://news.example/logo.png" );
cache->requestData( session, generation, request, {} );
// Simulate a document that remained open beyond the entry's original deadline. Retention
// starts when navigation releases the final lease, using the policy active at that time.
cache->setTTL( Seconds( 30 ) );
cache->beginNavigation( session, URI( "https://other.example/" ) );
cache->prune();
EXPECT_EQ( 1u, cache->getEntryCount() );
generation = cache->beginNavigation( session, URI( "https://news.example/" ) );
cache->requestData( session, generation, request, {} );
EXPECT_EQ( 1, fetches );
}
UTEST( WebResourceCache, navigationReleasesLeasesAfterDenseMapGrowth ) {
auto cache = WebResourceCache::New();
cache->setFetcher(
[]( const WebResourceRequest&, WebResourceCache::FetchCompletion completion ) {
auto ok = Http::Response::Status::Ok;
completion( response( ok, "body" ) );
} );
auto session = cache->createSession();
auto generation = cache->beginNavigation( session, URI( "https://example.com/" ) );
for ( int i = 0; i < 512; ++i ) {
cache->requestData(
session, generation,
requestFor( String::format( "https://example.com/resource/%d", i ).c_str() ), {} );
}
EXPECT_EQ( 512u, cache->getEntryCount() );
cache->setTTL( Time::Zero );
cache->beginNavigation( session, URI( "https://other.example/" ) );
cache->prune();
EXPECT_EQ( 0u, cache->getEntryCount() );
}
UTEST( WebResourceCache, failedLoadsCanRetry ) {
auto cache = WebResourceCache::New();
int fetches = 0;
cache->setFetcher(
[&fetches]( const WebResourceRequest&, WebResourceCache::FetchCompletion completion ) {
++fetches;
auto status = fetches == 1 ? Http::Response::Status::ConnectionFailed
: Http::Response::Status::Ok;
completion( response( status, fetches == 1 ? "" : "body" ) );
} );
auto session = cache->createSession();
auto generation = cache->beginNavigation( session, URI( "https://example.com/" ) );
int successful = 0;
auto callback = [&successful]( const WebResourceResult& result ) {
if ( result.success )
++successful;
};
cache->requestData( session, generation, requestFor( "https://example.com/site.css" ),
callback );
EXPECT_EQ( 1, fetches );
EXPECT_EQ( 0, successful );
cache->requestData( session, generation, requestFor( "https://example.com/site.css" ),
callback );
EXPECT_EQ( 1, fetches );
cache->setRetryDelay( Time::Zero );
cache->requestData( session, generation, requestFor( "https://example.com/site.css" ),
callback );
EXPECT_EQ( 2, fetches );
EXPECT_EQ( 1, successful );
}
UTEST( WebResourceCache, ttlAndByteBudgetEvictOnlyUnleasedEntries ) {
auto cache = WebResourceCache::New();
cache->setFetcher(
[]( const WebResourceRequest&, WebResourceCache::FetchCompletion completion ) {
auto ok = Http::Response::Status::Ok;
completion( response( ok, "12345678" ) );
} );
auto session = cache->createSession();
auto generation = cache->beginNavigation( session, URI( "https://example.com/" ) );
cache->requestData( session, generation, requestFor( "https://example.com/a.css" ), {} );
EXPECT_EQ( 1u, cache->getEntryCount() );
cache->setByteBudget( 1 );
EXPECT_EQ( 1u, cache->getEntryCount() );
cache->beginNavigation( session, URI( "https://example.com/next" ) );
cache->prune();
EXPECT_EQ( 0u, cache->getEntryCount() );
cache->setByteBudget( 1024 );
cache->setTTL( Time::Zero );
generation = cache->getSessionGeneration( session );
cache->requestData( session, generation, requestFor( "https://example.com/b.css" ), {} );
EXPECT_EQ( 1u, cache->getEntryCount() );
cache->beginNavigation( session, URI( "https://example.com/final" ) );
cache->prune();
EXPECT_EQ( 0u, cache->getEntryCount() );
}