Expand description
A picture represents a dynamically rendered image.
Overview
Pictures consists of:
- A number of primitives that are drawn onto the picture.
- A composite operation describing how to composite this picture into its parent.
- A configuration describing how to draw the primitives on this picture (e.g. in screen space or local space).
The tree of pictures are generated during scene building.
Depending on their composite operations pictures can be rendered into intermediate targets or folded into their parent picture.
Picture caching
Pictures can be cached to reduce the amount of rasterization happening per frame.
When picture caching is enabled, the scene is cut into a small number of slices, typically:
- content slice
- UI slice
- background UI slice which is hidden by the other two slices most of the time.
Each of these slice is made up of fixed-size large tiles of 2048x512 pixels (or 128x128 for the UI slice).
Tiles can be either cached rasterized content into a texture or “clear tiles” that contain only a solid color rectangle rendered directly during the composite pass.
Invalidation
Each tile keeps track of the elements that affect it, which can be:
- primitives
- clips
- image keys
- opacity bindings
- transforms
These dependency lists are built each frame and compared to the previous frame to see if the tile changed.
The tile’s primitive dependency information is organized in a quadtree, each node storing an index buffer of tile primitive dependencies.
The union of the invalidated leaves of each quadtree produces a per-tile dirty rect which defines the scissor rect used when replaying the tile’s drawing commands and can be used for partial present.
Display List shape
WR will first look for an iframe item in the root stacking context to apply picture caching to. If that’s not found, it will apply to the entire root stacking context of the display list. Apart from that, the format of the display list is not important to picture caching. Each time a new scroll root is encountered, a new picture cache slice will be created. If the display list contains more than some arbitrary number of slices (currently 8), the content will all be squashed into a single slice, in order to save GPU memory and compositing performance.
Compositor Surfaces
Sometimes, a primitive would prefer to exist as a native compositor surface. This allows a large and/or regularly changing primitive (such as a video, or webgl canvas) to be updated each frame without invalidating the content of tiles, and can provide a significant performance win and battery saving.
Since drawing a primitive as a compositor surface alters the ordering of primitives in a tile, we use ‘overlay tiles’ to ensure correctness. If a tile has a compositor surface, and that tile has primitives that overlap the compositor surface rect, the tile switches to be drawn in alpha mode.
We rely on only promoting compositor surfaces that are opaque primitives. With this assumption, the tile(s) that intersect the compositor surface get a ‘cutout’ in the rectangle where the compositor surface exists (not the entire tile), allowing that tile to be drawn as an alpha tile after the compositor surface.
Tiles are only drawn in overlay mode if there is content that exists on top of the compositor surface. Otherwise, we can draw the tiles in the normal fast path before the compositor surface is drawn. Use of the per-tile valid and dirty rects ensure that we do a minimal amount of per-pixel work here to blend the overlay tile (this is not always optimal right now, but will be improved as a follow up).
Structs
- Stores information about the calculated opaque backdrop of this slice.
- Information about the state of a binding.
- A set of flags describing why a picture may need a backing surface.
- A set of flags describing why a picture may need a backing surface.
- Wrapper struct around an external surface descriptor with a little more information that the picture caching code needs.
- In some cases, we need to know the dirty rect of all tiles in order to correctly invalidate a primitive.
- Represents the dirty region of a tile cache picture.
- Information about a native compositor surface cached between frames.
- Hash key for an external native compositor surface
- Information stored an image dependency
- A comparable transform matrix, that compares with epsilon checks.
- Represents the native surfaces created for a picture cache, if using a native compositor. An opaque and alpha surface is always created, but tiles are added to a surface based on current opacity. If the calculated opacity of a tile changes, the tile is invalidated and attached to a different native surface. This means that we don’t need to invalidate the entire surface if only some tiles are changing opacity. It also means we can take advantage of opaque tiles on cache slices where only some of the tiles are opaque. There is an assumption that creating a native surface is cheap, and only when a tile is added to a surface is there a significant cost. This assumption holds true for the current native compositor implementations on Windows and Mac.
- Information about a preserve-3D hierarchy child that has been plane-split and ordered according to the view direction.
- Flags describing properties for a given PicturePrimitive
- Descriptor for a cluster of primitives. For now, this is quite basic but will be extended to handle more spatial clustering of primitives.
- A helper struct to compare a primitive and all its sub-dependencies.
- A key for storing primitive comparison results during tile dependency tests.
- An index into the prims array in a TileDescriptor.
- Information about the dependencies of a single primitive instance.
- Defines a key that uniquely identifies a primitive instance.
- A list of primitive instances that are added to a picture This ensures we can keep a list of primitives that are pictures, for a fast initial traversal of the picture tree without walking the instance list.
- A comparable scale-offset, that compares with epsilon checks.
- The key that identifies a tile cache instance. For now, it’s simple the index of the slice as it was created during scene building.
- A helper for comparing spatial nodes between frames. The comparisons are done by-value, so that if the shape of the spatial node tree changes, invalidations aren’t done simply due to the spatial node index changing between display lists.
- A dependency for a transform is defined by the spatial node index + frame it was used
- A SubSlice represents a potentially overlapping set of tiles within a picture cache. Most picture cache instances will have only a single sub-slice. The exception to this is when a picture cache has compositor surfaces, in which case sub slices are used to interleave content under or order the compositor surface(s).
- Defines which sub-slice (effectively a z-index) a primitive exists on within a picture cache instance.
- Information from
get_surface_rectsabout the allocated size, UV sampling parameters etc for an off-screen surface - Information about an offscreen surface. For now, it contains information about the size and coordinate system of the surface. In the future, it will contain information about the contents of the surface, which will allow surfaces to be cached / retained between frames and display lists.
- Information about a cached tile.
- Represents a cache of tiles that make up a picture primitives.
- Information that is required to reuse or create a new tile cache. Created during scene building and passed to the render backend / frame builder.
- Unit for tile coordinates.
- Uniquely describes the content of this tile, in a way that can be (reasonably) efficiently hashed and compared.
- A stable ID for a given tile, to help debugging. These are also used as unique identifiers for tile surfaces when using a native compositor.
- Uniquely identifies a tile within a picture cache slice
- A node in the dirty rect tracking quadtree.
Enums
- Information stored in a tile descriptor for a binding.
- Optional extra information returned by is_same when logging is enabled.
- Debugging information about why a tile was invalidated
- Enum value describing the place of a picture in a 3D context.
- Specifies how this Picture should be composited onto the target it belongs to.
- The result of a primitive dependency comparison. Size is a u8 since this is a hot path in the code, and keeping the data small is a performance win.
- This is the same as a
SurfaceTextureDescriptorbut has been resolved into a texture cache handle (if appropriate) that can be used by the batching and compositing code in the renderer. - Specify whether a surface allows subpixel AA text rendering.
- A descriptor for the kind of texture that a picture cache tile will be drawn into.
- The kind of modification that a tile wants to do
- Details for a node in a quadtree that tracks dirty rects for a tile.
- The backing surface for this tile.
- A comparable / hashable version of a coordinate space mapping. Used to determine if a transform dependency for a tile has changed.
Constants
- The maximum number of compositor surfaces that are allowed per picture cache. This is an arbitrary number that should be enough for common cases, but low enough to prevent performance and memory usage drastically degrading in pathological cases.
- Maximum size of a compositor surface.
- The maximum size per axis of a surface, in WorldPixel coordinates.
- The size in device pixels of a normal cached tile.
- The size in device pixels of a tile for horizontal scroll bars
- The size in device pixels of a tile for vertical scroll bars
Statics
- Used to get unique tile IDs, even when the tile cache is destroyed between display lists / scenes.