Rendering events and counters in the Debugger

One of Gameface's prominent features is its integration with Google Chrome's devtools. With the click of a button, you can inspect a page running inside the Gameface's player in the same way you can inspect a webpage running in Chrome. The inspector allows you to see the state of the DOM tree, inspect individual elements, create performance recordings to identify performance problems or see the resources that the page loads during startup.

Recently we've introduced a plethora of enhancements to the Performance tab of the devtools. These enhancements will help us identify problems quicker and allow us to see what is going on internally in Gameface when a page is being rendered. These improvements can also greatly benefit Gameface users and we will showcase some of them here.

We will go over the new improvements, explain why they are useful, how they are used practically, and what problems can be detected with their help. All of the new features are in the "Performance" tab and to take advantage of them; you'll have to first create a recording of the HTML page from the Google Chrome Dev Tools.

Rendering Trace markers

Internally Gameface uses the proprietary rendering library Renoir. Up until now, Renoir had no way of communicating with the devtools, and hence event tracing in it was impossible. This is no longer the case and now we can see Renoir's internal markers being tracked and displayed in the Performance tab of the Chrome devtools.

These new markers can give better insights into the performance characteristics of the rendering library for a given page.

The major markers visible in the recordings are the following ones:

"Paint" - the top-level marker of all rendering-related work. This is essentially the time that Gameface spends in ViewRenderer::Paint function
"Process Frontend Commands Only" - the work during rendering is divided into two big groups - frontend (where we decide what graphics API calls we need) and backend (where we do the actual interaction with the graphics API). This marker is for the time we spend in the frontend.
"Execute Backend Buffers" - the actual execution of the generated graphics API commands.
"Batch Commands" - The time taken by Renoir to evaluate which of the draw commands can be done in a single draw call.
"Process Layer" - the generation of a backend command for each layer in the page. Gameface creates a separate layer for each group of elements that need to be drawn together in their parent layer. This happens when there is some filter, opacity, or blend mode applied to a DOM node.

These are the main spots to keep an eye on for potential performance problems when it comes to the rendering. The concrete time values of the markers can vary based on the complexity of the UI, but it is a good idea to consider them during UI development to identify performance issues quickly.

A handy feature is the ability to relate some trace markers ("Process Layers" for example) to the DOM node that is responsible for it. Simply hovering a "Process Layer" marker will highlight the DOM node in question that has been drawn in it. The highlighting happens in the corresponding view's viewport. Clicking on it and inspecting the "Node" field in the summary tab below will show you exactly which node this event is for. Clicking on the link itself will even bring you to the node in the "Elements" tab.

All of this allows you to quickly identify the node responsible for a given layer. This way you can reason about which parts of your UI are taking the most time to be drawn.

The rendering markers are also thread-aware. If the inspected view was initialized with the ViewSettings::ExecuteCommandProcessingWithLayout option set to true, the frontend command will be executed on the Layout thread just after "Layout" and "RecordRendering" have finished. This can be made visible thanks to the rendering markers.

Object creation and destruction markers

Renoir tries to minimize the GPU object creations (textures, index and vertex buffers, and constant buffers) by reusing already created resources. In some situations, however, this is not possible with the default capacity of the internal caches. It is crucial to know when a UI is causing some cache to be trashed and constant recreation of objects (most often textures) is happening.

We've introduced trace markers that mark the creation and destruction of different GPU objects to address this issue. You can see when a texture, vertex, or index buffer is created and destroyed. The trace markers are the following:

Texture Create/Destroy - for textures
VB Create/Destroy - for vertex buffers
IB Create/Destroy - for index buffers

These events also carry some meta information about the object that has been created. Most notably, this is the type of object which gives information about the object usage. These types can be inspected by clicking on the corresponding event and examining the "Type" field in the "Summary" tab below.

For textures, the possible types are:

ScratchTexture - temporary textures needed mostly for storing intermediate results when blurring elements
LayerTexture - textures in which Renoir draws the layers of DOM nodes.
ImageTexture - textures that store images used in the HTML/CSS
SurfaceTexture - textures created by Gameface for drawing some auxiliary things like some SVGs or shadow shapes for example.
CompositorTexture - textures created by Renoir to draw elements with coh-composition-id
GlyphAtlas - textures that store the character glyphs used for text rendering
GradientCacheTexture - textures that store the colors needed for some gradients

For vertex/index buffers, the possible types are:

GeometryBuffer -- buffers that store the geometry needed for all of the rendered basic shapes
PathBuffer - buffers that store tessellated geometry of paths used in the HTML/CSS
GlyphBuffer - buffers that store the geometry used for rendering the character glyphs. Those are created and destroyed in a single frame.

The creation/destruction markers can help you detect abnormal behavior in the object creation and destruction. In general, creating a texture for each frame and destroying it at the end can be a hint that the cache's capacity is too small.

With the ability to relate DOM nodes to "Process Layer" markers, you can even deduce which node has caused the creation of a given texture. For example, notice which "Process Layer" event has a "Texture Create" marker for a ScratchTexture that occurred. Then see to which node this "Process Layer" marker belongs. This way, we can see which DOM nodes cause potential trashing of the texture caches in Renoir.

In the below GIF, you can see the DOM node that has caused the creation of a texture:

Textures counters

The textures counters are another new feature in the Performance tab of the devtools, closely related to the trace markers for object creation and destruction.

The counters can be activated by enabling the "Counters" checkbox. This will show a separate panel near the bottom of the screen. The panel contains several charts displaying the counts of different texture types and how these counts evolve over time. The texture counts of only some texture types are tracked.

These counters present yet another opportunity to detect problems with a page. A large and unexpected amount of image textures might indicate that some of the images on a page are not released for some reason. On the other hand, excessive creation and destruction of scratch textures will present itself with constantly changing scratch textures count. For example, we might have three textures during rendering but two between frames. This means that we need three textures during the frame rendering, but in the end, if the cache capacity is exceeded, textures have to be destroyed.

Memory counters

The other types of counters are the CPU and GPU memory ones. They give us the ability to get a better picture of the memory resources used by Renoir. These counters can be seen by enabling the "Memory" checkbox.

Currently, we track two types of memory usage:

Frame memory - every frame Renoir allocates transient resources with a lifetime of a single frame in the so-called "frame memory". This memory is wiped after each frame and for the next one, Renoir starts allocating linearly from the start of the chunk. The counter lets you judge just how much memory Renoir needs for each frame.

GPU memory - this is the total amount of estimated GPU memory that Renoir uses for the allocated GPU objects.

Both memory types can vary drastically based on the complexity of the UI implemented on the page. Having the memory usage exposed allows you to quantify this complexity and judge what resources you might need to render your UI.

The GPU memory usage can also be used to spot problems, as discussed before. Large spikes in the memory lasting for short periods of time might indicate that there is some constant GPU resource recreation.

Precise Scratch Texture Manager monitoring

This section is a little more advanced but can help you fully leverage Gameface's capabilities.

As previously said, Renoir allocates two types of temporary textures - layer textures and scratch textures. The allocation behavior is controlled by the so-called "Scratch Texture Manager". This is an internal system of Renoir that decides when to allocate a new temporary texture and when to reuse an already created one. We can think about this manager as a cache with a set capacity that prevents constant recreation of textures that might be needed only for a single frame. When the memory for the temporary textures exceeds a certain limit, the scratch texture manager will deallocate some of the resources. There are different limits by default for the two types of textures:

for layer textures, the limit is 16 Megabytes
for scratch textures, the limit is 8 Megabytes

These limits can be changed through the View::QueueSetCacheBytesSize(InternalCaches cache, unsigned capacity) API by passing ICACHE_ScratchLayers or ICACHE_ScratchTextures as the cache argument. Setting up an appropriate cache capacity for your use case is crucial to avoid constant texture recreation.

In the Performance tab, we now can monitor the state of the scratch texture manager, the currently used memory by it, and just how full the manager's caches are. The "Scratch Texture Manager" checkbox allows you to inspect the relevant state of the caches and see how the memory usage evolves.

In the charts panel for the scratch texture manager, there are 4 charts that give you information:

"STM (Scratch textures) Memory" - the current memory used for scratch textures.
"STM (Scratch textures) Limit" - the cache capacity limit for scratch textures
"STM (Layer textures) Memory" - the current memory used for layer textures
"STM (Layer textures) Limit" - the cache capacity limit for layer textures

It is advisable to inspect only the charts for a single type of texture at a time - otherwise, the charts panel becomes cluttered with information that will be hard to read. The charts displayed can be controlled with the checkboxes above the panel.

The dashed lines show the caches' limit, while the solid ones indicate the current memory used. When the current memory goes over the limit, the scratch texture manager will destroy some of the textures at the end of the frame. Depending on your use case, you may have to adjust the caches' capacities to avoid their trashing. This new panel in the Performance tab will help you decide on the exact size of the caches.

Screenshots

The last new feature that we'll touch upon are the "Screenshots" enabled again from the checkbox in the upper part of the Chrome Dev Tools. The screenshots recording works the same way it does in Chrome. Enable the checkbox and do a performance recording. Cohtml will encode a screenshot of each frame and send the data to the inspector. Then, when examining the recorded data, you'll be able to see how the UI texture changes each frame.

The screenshots are taken only if the "Screenshot" checkbox is enabled. Screenshots of the UI texture won't be included in the JSON data if the recording was done without them.

The screenshot capturing is helpful when you want to show some visual problem that happens in specific circumstances. You can, for example, set up a page that reproduces the issue and then run the page while performing a recording with screenshot capturing enabled. The rendered page for each frame will also be saved in the resulting JSON file for the profiling session.

Summary

To summarize, the new features in the integration of Gameface with the Chrome Dev Tools are:

Rendering trace markers that allow you to inspect what is happening during Renoir's work.
Ability to relate rendering events to concrete DOM nodes
Tracking of GPU objects creation and destruction while having meta information in each event about the usage of the created object
Tracking of GPU objects counts and how they develop over time
Tracking of GPU and CPU memory used by Renoir
Providing insight into the state of the Scratch Texture Manager of Renoir
Ability to record screenshots for each rendered frame

These features aim to allow developers to understand what is going on internally in Gameface and Renoir, where are the system resources (CPU and GPU time and memory) going, and find potential problems in a given HTML page.