MoStyle: combining 2D and 3D animation for greater creativity

On the occasion of World Art Day, Pierre Bénard, a teacher-researcher in the Manao project team shared by Inria and LaBRI, looks back at the genesis of the ANR MoStyle project, which began in 2021 and will come to an end at the end of the year: "The initial idea was to bring together two very different forms of animation, 2D and 3D. In many modern productions, these two forms coexist, and we wanted to help animators specifically with movement design, while leaving them full artistic control over their creations."

The main aim of MoStyle, which stands for "Motion Stylization", is to determine how digital tools can help designers of 2D or 3D animated films. In these sectors, the stylisation of movement is crucial to improving legibility or charm.

MoStyle aims to provide 2D animators with controls and previews similar to those available in 3D, and to enable 3D specialists to give their work a 2D feel. The whole art of traditional 2D animation, since its birth at the beginning of the 20th century, has been to give the impression of movements in volume from simple 2D drawings. Conversely, some modern 3D productions are inspired by 2D to add a graphic touch. This is particularly true of "Spider-man: New Generation" and its sequel, where certain jerky movements and effects typical of comic books sometimes give the impression of being hand-drawn in 2D, whereas in fact they are computer-generated images.

In reality, 2D and 3D artists work in very different ways, which led the participants in the MoStyle project to divide their work into two very distinct branches.

On the 2D side, the team took as its starting point the traditional way that animators design character movements. Generally, a lead animator draws the most important key poses. Then an intervallist produces the many intermediate drawings between two poses to give the illusion of continuous movement, with 12 to 24 drawings for one second of animation. The succession of drawings on paper must give an impression of volume, so that the spectator imagines a 'real' character moving and turning, and not just a sketch on a sheet of paper. Here, the tools developed in MoStyle are designed to help the artist right from the preparatory drawings phase," explains Pierre Bénard. Even before all the intermediate drawings are made by the intervallist, he can see what the movement is going to look like." In other words, the programme itself generates glimpses of movement between two poses.

Automating tedious tasks to save time by imitating work done by a human being... This is reminiscent of many of the promises made about generative artificial intelligence. But there's nothing like that here, says Pierre Bénard: "Our aim is not to do away with the work of the interval operator. So MoStyle will not be used in the final phase of the animation, but at the sketching stage. The project is above all a tool for working upstream, to help with the design of an animated film. Of course, it is not the project that will produce the final result. But the lead animator will be able to use this movement preview, created using the software, to share his work with his collaborators (art director, producer, camera operator), before embarking on the actual production.

What's more, the system does not use automatic learning. There was a practical reason for this choice in the first place," says Pierre Bénard. AIs need data to 'learn', and here we work on preparatory work, sketches whose style is specific to each artist, and not the final animation. So there was nothing to feed their algorithms." The other reason is more ethical. Wishing to leave the animators in complete control of their creation, the Inria team discussed with them how to assist them effectively. "Artists express a great deal of reticence about AI in this environment.

As far as 3D is concerned, the problem is a little different, because it involves helping to reproduce the effects typical of 2D animation with computer-generated images. It's a tricky challenge, as Pierre Bénard points out: "Admittedly, empirical techniques already exist, but each animator has his or her own, and the process has not been theorised". In traditional 3D animation, it is possible to simulate motion blur effects that are reminiscent of a real shot, but which generally don't exist in 2D animation. On the other hand, animators who want to add a 2D style will use various "smear" effects grouped into three categories by the researchers: deforming an object in the direction of its movement to accentuate the impression of speed; adding lines of movement as in comics; repeating a drawing several times and superimposing it.

MoStyle's strength lies in unifying these three techniques and making them available in software that applies them directly to 3D animations. This facilitates the work of animators, who can design and, above all, control these effects as they see fit. In practical terms, this process takes the form of a plug-in integrated into the free Blender modelling software. The tool works well for a large number of movements, but not every time", explains Pierre Bénard. As long as the movement is fluid, everything's fine. But if there's a sharp discontinuity, such as a bounce, the object looks less pretty and the movement doesn't retract properly.

What are the future prospects for MoStyle? Of course, the 2D and 3D branches are pursuing the same goal, but they are not following the same methods and developments. The 3D part has already been integrated and made available. It has even been downloaded nearly 6,000 times to date. As for the 2D part, it remains more experimental. However, Praxinos, a company that develops animation software, is working with Inria researchers to integrate these tools. The next challenge is to try to bring these two branches even closer together to explore all the variations of movement in the different types of animation... Watch this space!