Published on
Updated
Reading 3 min.
A dark mountain overlooking red plains, surrounding a strange blue shape: a trio of French artists presented to AFP an image generated by artificial intelligence from the brain activity of one of them. “A first”, according to the collective.
“In my head, I thought very strongly of a volcano“, says Pierre Fautrel, one of the members of “Obvious” who took part in the experiment.
During an hour in an MRI machine at the Brain Institute of the Pitié Salpêtrière hospital in Paris, he imagined several scenes, each inspired by a short description. During the process, his brain activity was recorded and then processed by a specially trained AI.
If Pierre Fautrel confesses that the work generated is not exactly what he had in mind, “she kept the semantic elements: a flaming mountain with flowing lava with a landscape on a light background“.
For almost a year, he, Hugo Caselles-Dupré and Gauthier Vernier have devoted all their energy to “Mind to Image”, a project that seems to be science fiction: transcribing the the imagination of an artist thanks to generative AI.
“Technical prowess”
To achieve this, these thirty-year-olds started from an existing “open source” model, MindEye, which associates images seen with brain activity, and then trained their own AI in several stages.
First, by broadcasting portraits and landscapes to one of the artists in the MRI to observe which areas of their brain are activated, and using this data to feed the AI.
This attempts to reconstruct the original images solely from brain data, without having seen them during training.
A process repeated many times for around ten hours “to create a database“, explains Hugo Caselles-Dupré.
Secondly, they repeated the exercise based solely on memories of these images, before trying the experiment on pure imagination while reading “prompt“, descriptive texts written before entering the MRI.
“We have known for around ten years that it is possible to reconstruct an image seen from the activity of this visual cortex.“, explains Alizée Lopez-Persem, researcher at the Brain Institute and Inserm. “But an +imagined+ image, no. It represents a challenge“.
Several dozen hours are necessary to sort through the information collected in the MRI before providing it to the AI.
But, once this work is done, the generation of images “is more or less instantaneous, on very powerful computers“, summarizes Hugo Caselles-Dupré.
“Two years ago, I would never have believed that this could exist“, testifies Charles Mellerio, neuroradiologist who participates in the trio’s research project.
For him, this “technical prowess” is linked to a double progress in recent years: that of medical imaging, which has made a leap forward “in terms of resolution and precision”, and the dazzling development of generative AI.
Surreal AI
The trio of artists also drew inspiration from the surrealist movement, which celebrates its 100th anniversary in 2024, to give a particular tone to the creations of its algorithm.
“It’s a movement where we try to move as quickly as possible between the mental image and the plastic realization.“, notes Pierre Fautrel, who sees in their experimentation a means of “reinterpreting surrealism”.
“For us, there are real links between art and science“, adds Hugo Caselles-Dupré, who recognizes that this type of technology “can be very scary if used in the wrong way“.
In the future, the three artists hope to extend their experimentation to other formats, by reconstructing sound or video. Until then, they will present their different creations in October at the Danysz gallery in Paris.
In 2018, they stunned the art world by selling more than 400,000 euros at Christie’s in New York, a work presented as the first produced by artificial intelligence software, “Edmond de Belamy”, a fictional portrait with blurred contours printed on canvas.