AI at the service of manga and animation: eldorado or illusion? (4/5)

Useful, fun, futile or catastrophic… Machine learning excites the curiosity and creativity of manga publishers and animation producers. In the fourth part of our investigation, we looked at the contributions of neural networks and other new generation algorithms in the animation world.

Episode #4: AI at the service of the world of animation?

In the world of animation, in constant shortage of skilled labor, any help to reduce the workload of animators, chara designers, set artists or intervalists is welcome. The studios have understood this well and they have launched projects in all directions, mainly in partnership with research labs, in order to automate their productions as much as possible. Especially since with streaming services like ADN, Crunchyroll, Disney+ and others… the demand for Japanese animation has never been greater.

One of the first AI-assisted animation tasks is working around sets.

Many artists refer to photos of actual sets to draw the backgrounds of their scenes, both for realistic and fanciful scenes. The Toei animation studios, more particularly the Oizumi studio teams under the leadership of Katsuhiro Takagi, have chosen to integrate “machine learning”. They teamed up with Preferred Network Inc. to develop a tool called Scenify. This not only makes it possible to scan a photo, but also to transform it into several layers of “anime” style drawings in order to save a lot of time for the teams in charge of the sets. It only remains for humans to make a fine tuning. A time saver on which Toei animation has not yet communicated, but which opens the door to other tests.

Another example of machine learning integration. The job of an intervalist is to draw the missing drawings between the key drawings (those made by the key animator). It is often novice animators who are entrusted with this task, which is often daunting but crucial for the quality of an animation. In the cinema, it is customary to film in 24 frames per second. In the world of animation, only the Hollywood productions of Disney and consort allow themselves such a level of detail. In Japan, for reasons of production time and cost, it is customary to have fewer frames per second. However, to overcome the shortage of animators, more and more companies are interested in “machine learning” to automatically generate some of these images.

For the facilitator sandaFrench artist working at Toei Animation, “this kind of initiative has already been tested in Japan, but for the moment they do not offer convincing results: often, the renderings are not natural. The quality of an animation does not depends not only on the number of images, but also on the fluidity and naturalness released by the latter.The animator himself decides how many interval images to put in his animations, which is part of the leg of each animator”.

the Singaporean software Cacani was even used to generate part of the intervals of an episode of the 5th part of Jojo’s Adventure, directed by David Production studio.

Still in the field of animation, with the progress of display on new screens and the democratization of HD or even ultra-HD (4K and 8K) formats, old productions are not of sufficient definition to be used on these media. . The studios authoring often realize what is called upscaling. However, to increase the resolution without losing quality, it is necessary to recalculate a lot of information that makes up each image of each animation sequence. It’s a titanic task, well helped by “machine learning” algorithms. Daisuke Urushihara, the CEO of the company RADIUS5, announced that their system ofupscale under “machine learning” could provide results upscaled to 8K in a few seconds per scene, greatly reducing the resource requirements for this type of task. And this is all the more important since many old sources have been lost, making it impossible to re-scan the period 16 or 25mm reels in high definition. Some well-trained AIs can reproduce the level of a reel scan from low definition digital files.

The promises of improved restorations of old anime are the ones that seem to be of most interest to fans today. Whether they are nostalgic for their VHS or Laser Discs, or whether it is a new generation curious to discover the references and inspirations of their favorite animators, everyone has an appetite for remastered old series. And since we are talking about enthusiasts, the transition is self-evident, the fifth and last part of our investigation: “AI at the service of manga and anime: Eldorado or illusion?”, will be devoted to fans of all horizon.