AI at the service of manga and animation: eldorado or illusion? (5/5)

Useful, fun, futile or catastrophic… “Machine learning” excites the curiosity and creativity of manga publishers and animation producers. In the fifth and final part of our investigation, we looked at the services that artificial intelligence provides or is about to provide to fans of manga and animation.

Episode #5: AI: to the delight of the fans?

Audience of geeks par excellence, fans of manga and Japanese animation are very aware of the latest technological advances. Better they actively contribute to it. Many independent projects have thus emerged in recent years with unimaginable opportunities until recently.

Take the example of cosplay, the art of dressing up as a character from your favorite license. There is no guide strictly speaking to realize the costume of so and so. Worse: on a 2D manga drawing, we don’t necessarily have all the angles of view. Difficult to make a costume pattern if you do not visualize the whole of the latter.

In the research paper “Anime-to-Real Clothing: Cosplay Costume Generation via Image-to-Image Translation“, researchers Koya Tango and Marie Katsura from Doshisha University and Hayato Maki and Ryosuke Goto from the Goto ZOZO Institute of Technology have implemented an AI protocol capable of interpreting animation scenes to generate renderings realistic costumes on the fly, and even identify assimilable objects (e.g. a belt, a leather purse, etc.), thus facilitating the creation of patterns.For Toshihiro Miura, editor-in-chief of the magazine Morning, “this type of initiative is beneficial not only for the fans but also for the authors who will undeniably be touched to see even more costumes taken from their works. And even more so if they manage to faithfully reproduce the spirit of a character as drawn by the artist”.

If in 2021 more than one in two comics sold in France is a manga, it remains difficult to navigate among the impressive number of releases (more than 300 volumes in September 2022). Of course, publishers accompany these releases with recommendations, specialized booksellers provide ad hoc recommendations to their customers.

Journalists and influencers offer reviews or lists of manga by theme, allowing readers to target certain titles, but far from the main blockbusters it can be difficult to find recommendations.

https://mangaki.fr/ is a manga and anime recommendation solution based on “machine learning”, where fans can feed the algorithm by indicating the manga they liked and thus create cohorts of taste associations. The aim of the researchers is to allow manga to be discovered by readers who do not necessarily have a circle of advice close to them.

Finally, at a time when social networks are omnipresent, the question of one’s avatar (or profile picture, PP) online is anything but innocuous. Fans of manga and more particularly of the license Bleach (published in France by Glénat editions), look no further, the MangaGAN tool, developed by Chinese scholars, is made for you. It is one of the first GAN (Generative Adversarial Network) systems trained with a base of drawings made by Tite Kubo. This database contains a sample of 448 pairs of eyes, 109 noses, 179 mouths and 106 faces drawn by the famous mangaka.

Many AIs are trained to produce “mangaified” versions of selfies or portraits. As the developed one by Chinese researchers from Damo Academy, a team from Alibaba and the Institute of Computing Technology at Peking University.

Now that Netflix is ​​churning out live-action TV series adapted from manga licenses, many fans are using “machine learning” to transpose 2D characters into realistic, human-like versions. A kind of pre-casting. The Japanese youtuber AI Mikan specializes in this type of video made with https://www.artbreeder.com/

Or this type of fan creation, which imagines via machine learning a human casting for Saint Seiya.

Conclusion

“Machine learning” is not intended to replace humans, says Toshihiro Miura, the magazine’s editor-in-chief Morning (Kodansha). In the future, “if AI can be brought to help during the creation process then it could be a tool at the service of authors to create even more new stories”. For the moment, no publisher contacted is considering using these new technologies to replace publishers in the various local markets.

Manga and animation are fields which, by their nature, attract a geek and tech-savvy audience, so it is not surprising that the subject of “machine learning” and artificial intelligence is very abundant in this medium. . Both with academics, industrialists and isolated fans. Publishers have understood this well: Shueisha, the publisher of the famous Weekly Shōnen Jump (One Piece, naruto, Jujutsu Kaisen, My Hero Academiaetc.), launched an annual contest, “Mirai no jump“, where fans are invited to submit innovative application prototypes. More than ever, contributions from fans around the world are welcome because, beyond the mathematical or algorithmic side of “machine learning”, databases to train the models are crucial Community projects like the catalog of character portraits from the animation Daf:re form the soil that will see the functionalities of tomorrow hatch.