Module picture

A picture represents a dynamically rendered image.

Overview

Pictures consists of:

• A number of primitives that are drawn onto the picture.
• A composite operation describing how to composite this picture into its parent.
• A configuration describing how to draw the primitives on this picture (e.g. in screen space or local space).

The tree of pictures are generated during scene building.

Depending on their composite operations pictures can be rendered into intermediate targets or folded into their parent picture.

Picture caching

Pictures can be cached to reduce the amount of rasterization happening per frame.

When picture caching is enabled, the scene is cut into a small number of slices, typically:

• content slice
• UI slice
• background UI slice which is hidden by the other two slices most of the time.

Each of these slice is made up of fixed-size large tiles of 2048x512 pixels (or 128x128 for the UI slice).

Tiles can be either cached rasterized content into a texture or “clear tiles” that contain only a solid color rectangle rendered directly during the composite pass.

Invalidation

Each tile keeps track of the elements that affect it, which can be:

• primitives
• clips
• image keys
• opacity bindings
• transforms

These dependency lists are built each frame and compared to the previous frame to see if the tile changed.

The tile’s primitive dependency information is organized in a quadtree, each node storing an index buffer of tile primitive dependencies.

The union of the invalidated leaves of each quadtree produces a per-tile dirty rect which defines the scissor rect used when replaying the tile’s drawing commands and can be used for partial present.

Display List shape

WR will first look for an iframe item in the root stacking context to apply picture caching to. If that’s not found, it will apply to the entire root stacking context of the display list. Apart from that, the format of the display list is not important to picture caching. Each time a new scroll root is encountered, a new picture cache slice will be created. If the display list contains more than some arbitrary number of slices (currently 8), the content will all be squashed into a single slice, in order to save GPU memory and compositing performance.

Compositor Surfaces

Sometimes, a primitive would prefer to exist as a native compositor surface. This allows a large and/or regularly changing primitive (such as a video, or webgl canvas) to be updated each frame without invalidating the content of tiles, and can provide a significant performance win and battery saving.

Since drawing a primitive as a compositor surface alters the ordering of primitives in a tile, we use ‘overlay tiles’ to ensure correctness. If a tile has a compositor surface, and that tile has primitives that overlap the compositor surface rect, the tile switches to be drawn in alpha mode.

We rely on only promoting compositor surfaces that are opaque primitives. With this assumption, the tile(s) that intersect the compositor surface get a ‘cutout’ in the rectangle where the compositor surface exists (not the entire tile), allowing that tile to be drawn as an alpha tile after the compositor surface.

Tiles are only drawn in overlay mode if there is content that exists on top of the compositor surface. Otherwise, we can draw the tiles in the normal fast path before the compositor surface is drawn. Use of the per-tile valid and dirty rects ensure that we do a minimal amount of per-pixel work here to blend the overlay tile (this is not always optimal right now, but will be improved as a follow up).

Structs

BackdropInfo

Stores information about the calculated opaque backdrop of this slice.

BackdropSurface

BindingInfo

Information about the state of a binding.

BlitReason

A set of flags describing why a picture may need a backing surface.

ClusterFlags

A set of flags describing why a picture may need a backing surface.

CompositorSurface

Wrapper struct around an external surface descriptor with a little more information that the picture caching code needs.

DeferredDirtyTest 🔒

In some cases, we need to know the dirty rect of all tiles in order to correctly invalidate a primitive.

DirtyRegion

Represents the dirty region of a tile cache picture, relative to a “visibility” spatial node. At the moment the visibility node is world space, but the plan is to switch to raster space.

ExternalNativeSurface

Information about a native compositor surface cached between frames.

ExternalNativeSurfaceKey

Hash key for an external native compositor surface

ImageDependency

Information stored an image dependency

MatrixKey 🔒

A comparable transform matrix, that compares with epsilon checks.

NativeSurface

Represents the native surfaces created for a picture cache, if using a native compositor. An opaque and alpha surface is always created, but tiles are added to a surface based on current opacity. If the calculated opacity of a tile changes, the tile is invalidated and attached to a different native surface. This means that we don’t need to invalidate the entire surface if only some tiles are changing opacity. It also means we can take advantage of opaque tiles on cache slices where only some of the tiles are opaque. There is an assumption that creating a native surface is cheap, and only when a tile is added to a surface is there a significant cost. This assumption holds true for the current native compositor implementations on Windows and Mac.

OrderedPictureChild

Information about a preserve-3D hierarchy child that has been plane-split and ordered according to the view direction.

PictureFlags

Flags describing properties for a given PicturePrimitive

PicturePrimitive

PictureScratchBuffer

PrimitiveCluster

Descriptor for a cluster of primitives. For now, this is quite basic but will be extended to handle more spatial clustering of primitives.

PrimitiveComparer 🔒

A helper struct to compare a primitive and all its sub-dependencies.

PrimitiveComparisonKey 🔒

A key for storing primitive comparison results during tile dependency tests.

PrimitiveDependencyIndex

An index into the prims array in a TileDescriptor.

PrimitiveDependencyInfo 🔒

Information about the dependencies of a single primitive instance.

PrimitiveDescriptor

Defines a key that uniquely identifies a primitive instance.

PrimitiveList

A list of primitive instances that are added to a picture This ensures we can keep a list of primitives that are pictures, for a fast initial traversal of the picture tree without walking the instance list.

RasterConfig

ScaleOffsetKey 🔒

A comparable scale-offset, that compares with epsilon checks.

SliceId

The key that identifies a tile cache instance. For now, it’s simple the index of the slice as it was created during scene building.

SpatialNodeComparer 🔒

A helper for comparing spatial nodes between frames. The comparisons are done by-value, so that if the shape of the spatial node tree changes, invalidations aren’t done simply due to the spatial node index changing between display lists.

SpatialNodeKey

A dependency for a transform is defined by the spatial node index + frame it was used

SubSlice

A SubSlice represents a potentially overlapping set of tiles within a picture cache. Most picture cache instances will have only a single sub-slice. The exception to this is when a picture cache has compositor surfaces, in which case sub slices are used to interleave content under or order the compositor surface(s).

SubSliceIndex

Defines which sub-slice (effectively a z-index) a primitive exists on within a picture cache instance.

SurfaceAllocInfo 🔒

Information from get_surface_rects about the allocated size, UV sampling parameters etc for an off-screen surface

SurfaceIndex

SurfaceInfo

Information about an offscreen surface. For now, it contains information about the size and coordinate system of the surface. In the future, it will contain information about the contents of the surface, which will allow surfaces to be cached / retained between frames and display lists.

Tile

Information about a cached tile.

TileCacheInstance

Represents a cache of tiles that make up a picture primitives.

TileCacheParams

Information that is required to reuse or create a new tile cache. Created during scene building and passed to the render backend / frame builder.

TileCoordinate

Unit for tile coordinates.

TileDescriptor

Uniquely describes the content of this tile, in a way that can be (reasonably) efficiently hashed and compared.

TileId

A stable ID for a given tile, to help debugging. These are also used as unique identifiers for tile surfaces when using a native compositor.

TileKey

Uniquely identifies a tile within a picture cache slice

TileNode

A node in the dirty rect tracking quadtree.

TilePostUpdateContext 🔒

TilePostUpdateState 🔒

TilePreUpdateContext 🔒

TileUpdateDirtyContext 🔒

TileUpdateDirtyState 🔒

Enums

BackdropKind

Binding

Information stored in a tile descriptor for a binding.

CompareHelperResult

Optional extra information returned by is_same when logging is enabled.

InvalidationReason

Debugging information about why a tile was invalidated

Picture3DContext

Enum value describing the place of a picture in a 3D context.

PictureCompositeMode

Specifies how this Picture should be composited onto the target it belongs to.

PrimitiveCompareResult

The result of a primitive dependency comparison. Size is a u8 since this is a hot path in the code, and keeping the data small is a performance win.

PrimitiveDependency 🔒

ResolvedSurfaceTexture

This is the same as a SurfaceTextureDescriptor but has been resolved into a texture cache handle (if appropriate) that can be used by the batching and compositing code in the renderer.

SubpixelMode

Specify whether a surface allows subpixel AA text rendering.

SurfacePromotionFailure 🔒

SurfaceTextureDescriptor

A descriptor for the kind of texture that a picture cache tile will be drawn into.

TileModification 🔒

The kind of modification that a tile wants to do

TileNodeKind

Details for a node in a quadtree that tracks dirty rects for a tile.

TileSurface

The backing surface for this tile.

TransformKey 🔒

A comparable / hashable version of a coordinate space mapping. Used to determine if a transform dependency for a tile has changed.

Constants

MAX_BLUR_RADIUS 🔒

MAX_COMPOSITOR_SURFACES

The maximum number of compositor surfaces that are allowed per picture cache. This is an arbitrary number that should be enough for common cases, but low enough to prevent performance and memory usage drastically degrading in pathological cases.

MAX_COMPOSITOR_SURFACES_SIZE 🔒

Maximum size of a compositor surface.

MAX_SURFACE_SIZE

The maximum size per axis of a surface, in DevicePixel coordinates. Render tasks larger than this size are scaled down to fit, which may cause some blurriness.

TILE_SIZE_DEFAULT

The size in device pixels of a normal cached tile.

TILE_SIZE_SCROLLBAR_HORIZONTAL

The size in device pixels of a tile for horizontal scroll bars

TILE_SIZE_SCROLLBAR_VERTICAL

The size in device pixels of a tile for vertical scroll bars

Statics

NEXT_TILE_ID 🔒

Used to get unique tile IDs, even when the tile cache is destroyed between display lists / scenes.

Functions

calculate_screen_uv

calculate_uv_rect_kind

clamp 🔒

clampf 🔒

get_relative_scale_offset 🔒

get_surface_rects 🔒

get_transform_key 🔒

request_render_task 🔒

Type Aliases

ColorBinding

ColorBindingInfo

OpacityBinding

OpacityBindingInfo

TileOffset

TileRect