Nvidia has just presented a new artificial intelligence technology capable of almost instantaneously generating a 3D scene from “a few dozen photos”.
baptized Instant neural radiance fields (Instant NeRF), it is capable of training “a small neural network” in the space of a few seconds from a slew of photos, provided, however, that you know the angles of the shots well. The resulting model is then used to generate all the data necessary to create the 3D environment of the scene in question. A calculation that, according to Nvidia, requires only a few “tens of microseconds”.
In a YouTube video, the American company demonstrates this new technology, which obviously relies on internal tools such as Cuda Toolkit Where Tiny Cuda Neural Networks.
Thus, we see a person in the middle of a room who is photographed from several sides, and, magic of technology, we navigate around him in a three-dimensional space.
Creating 3D scenes from photos is nothing new. The main innovation, according to Nvidia, is the execution speed. The calculation would be more than a thousand times faster than with the usual technologies, which would require hours to achieve the same result. There are many possible uses.
“The technology could be used to train robots and self-driving cars to understand the size and shape of real-world objects by capturing 2D images or video footage.
It could also be used in architecture and entertainment to quickly generate digital representations of real environments that creators can modify and expand.”explains Nvidia in a blog post.
This transformation also does not need a lot of computing power. A single graphics card would be sufficient, as the neural network is small. But there is still a downside: it is preferable that the photographed scene is motionless.
“If there is too much movement during the 2D image capture process, the AI-generated 3D scene will be blurry”, warns the company. But isn’t the future always a bit so?
Source : Nvidia