The Meta group, parent company of Facebook, has developed its own chip to accelerate the real-time display of its future avatars in its VR headsets. Meta has indeed developed a technology called Pixel Codec Avatar (PiCA)which significantly improves the naturalness of artificial faces in VR, both from the point of view of image precision and animation.
The first version of this system ran on a very powerful graphics card (Nvidia Titan X), a GPU too energy-intensive to be integrated into a VR headset. To support PiCA in a virtual reality headset, the company’s engineers have therefore developed a specific chip. It is a processor integrating a CPU with RISC-V instruction set, a little memory and above all a neural network accelerator which takes care of the calculations of this codec in real time. Engraved in 7 nm, this chip of only 2.56 mm² focuses on its task (calculating the field of view in space taking into account eye tracking and generating PiCA data), 3D calculation and display being well supported by the GPU of the SoC of the Quest 2, platform which was used for its initial development.
If the chip was developed for PiCA, its development involved, in turn, completely rethinking this technology to take full advantage of this custom chip. It is likely that the codec could have been supported by the AI part of a chip, like the Hexagon of a Snapdragon… But the fact of having developed this codec/chip symbiosis allows Meta to do drastically reduce energy consumption and, in the process, the thermal footprint of its technology.
We note in passing that Meta took advantage of this development of a relatively simple chip (small 32-bit processor, only 2 MB of SRAM) to get their hands dirty in the RISC-V instruction set. To the obvious detriment of ARM.
Source : Road to VR