Bluffing! This Artificial Intelligence turns words into images

Entering a sentence and seeing photorealistic images appear on the screen to illustrate it, this is the feat that an Artificial Intelligence developed by Nvidia achieves. Baptized GauGAN 2, this is an evolved version of this AI, since GauGAN had already been presented in 2019. At the time, AI already knew how to transform sketches into photorealistic images. Nvidia also offers a software baptized Canvas to use this process. The only constraint is that the software requires a powerful graphics card from the brand to be able to function.

To turn words into pictures, GauGAN 2 uses a special generative AI model that has been trained from 10 million landscape images. So he doesn’t really know how to improvise if the keywords have multiple meanings or if they’re too subtle, but that’s just the beginning.

Mix keywords and sketches

Better yet, it is possible to mix the functions of Canvas and those of the keywords to combine text and drawings. The process makes it possible to go further in the personalization of the rendering based on the sketch.

On his blog, the founder has published a short video that shows GauGAN 2 at work. On a web page, the user gradually enters keywords, such as ocean waves hitting rocks on the beach (” the waves of the ocean hit the rocks on the beach ”) and we see the corresponding images displayed as and when typing. Impressive ! Nvidia offers to test the AI ​​from a Navigator but during our attempts, the page did not answer.