Disney researches technology to change actor faces with extremely high quality, which can be applied to blockbuster movies in the near future.

Within just a few short years, neural network applications and face-changing software have developed dramatically and produced unbelievably authentic products. Not only stop at the level of still image collage as photoshop has been doing very well for many years, the power of the neural network is even “so big” that it can manipulate even complex video ( known through deepfake concept).

Surely this will be a tool that will be applied more widely in the future, especially in the field of film. Recently, Disney has announced research on a completely new high-quality “face-to-face” system, promising to be a powerful weapon for Hollywood’s creative and post-production team in the future. far.

One of the biggest challenges that deepfake videos face is that they need huge databases. These data include images of faces, portraits of millions, even billions of different people. Each person owns a few dozen to a few hundred separate expressions. The larger the database, the higher the input image quality, the more authentic the final deepfake will be.

However, most of the current photos are collected from the Internet with limited resolution. 4K videos are also unable to handle too blurry or sharp shots, leading to the quality of the face-changing process.

Therefore, the first task that Disney needs to handle is to create an extremely high quality image source, specifically for changing the faces of actors. In the study “High-quality face swapping techniques in tricks”, published at the Eurographics Symposium 2020, ETH Zurich’s research team and Disney Research Studios shared more details on the issue. this. Accordingly, they offer a series of innovative and more advanced approaches for the process of automatically changing faces with sharp quality, eligible for later use in blockbusters.

The input image quality is especially important for deepfake videos as well as face swapping.

First, the new algorithm Disney researchers devised will edit the original video to facilitate the process of changing the actor’s face later. The motion in that video will be transformed more smoothly and smoothly, eliminating all possible problems. For example, even the lips of a character are a bit shaky, which can adversely affect the quality of face swapping.

Besides, the research team also improved a lot of other factors. Including the process of mixing different faces with techniques with greater accuracy, creating a more standard product with light, more harmonious contrast. This algorithm will also keep the frame motion smooth during face swap, completely eliminating the situation of new faces that can be faulty, glitched and reveal faces in the original video.

With the development of machine learning, it is not surprising that every day, scientists find a new application of this technology. That will help them quickly solve problems that were previously extremely complex and time-consuming. Since the first deepfake videos appeared on the Internet, the technical experts have already seen the potential that this technology possesses for the film and television industry. In particular, swapping faces between stunt actors and main stars is no longer a very strange job for the post-production team.

When the deepfake reaches perfection, the post-production team will probably no longer have to struggle with turning the stunt’s face into the main star’s face.

However, to do that at the present time is a whole cumbersome stage, from makeup, rotation design, to complex processing tasks on the computer, and sometimes I have to go back to some scenes to get another angle on their faces. The whole process, plus relying too much on the VFX team, often costs a significant amount of movie studios.

With Disney’s new research, completed and unused footage may become data to “teach” their algorithm, and then it will find a reasonable face swap. Best. However, this also poses a serious problem: We will hardly be able to distinguish deepfake videos in the future, when this tool is perfect and widely used. The prospect of the Internet again flooding in fake news, in a more sophisticated form – video, is entirely possible and will make it difficult to filter information from users.

According to Gizmodo