This algorithm can change the human speech in the video by editing the translation

Tram Ho

Making fake videos is getting easier and easier, and the latest results in this area are both impressive and scary.

Recently, researchers from Stanford University, Max Planck Institute of Informatics, Princetone University, and Adobe have developed a new algorithm capable of altering human speech in a video by compiling Re-edit the text in the translation of the video.

This method is said to be able to change the video while retaining the characteristics of the speaker. To accomplish the impossible, first, the algorithm will read phonemes and pronunciation of characters and words in the original video, then create a model of the head of the speaker to parody. correctly the person's voice and movements.

Once you recompile the translation, the algorithm will perform a search for the lip segment that creates the words you type, and replace the original phrase. But the replaced part can have quite a bit of pause and cropping, because small segments of video in the entire clip are put together to create a suitable lip motion. Therefore, the algorithm will apply some smart smoothing techniques to help the edited video look more natural.

Currently, this algorithm only works when you train it with an original video that is at least 40 minutes long.

In the video explaining this new method, Stanford's Ohad Fried has shown us how easy it is to replace a phrase without interfering with the quality of the speech.

Algorithm to edit videos through text

Naturally, this algorithm makes people worry that anyone will be able to edit a speech (including speeches of celebrities, such as politicians), insert There was some bad information in it, and it made it look as natural as the original – as if deepfake didn't make us nervous enough! However, Fried said that photo editing software has also encountered worries, and then everything is "in place", "The Earth is still spinning."

With that in mind, along with many options to identify edited videos, Fried believes that the tool he develops will be very useful for video producers, reducing time and effort. spend to go back to the parts that need to be changed in the speech or other videos that people say stand in front of the camera.

He added that there may be many options such as inserting digital watermark into a video to avoid spoofing, and the study itself will encourage others to develop similar solutions:

It is necessary to develop some types of watermarks that help confirm any content that has been edited and provides a complete record of the corrections that have been made. Moreover, researchers can develop better forensic tools, such as digital or conventional fingerprint techniques to determine whether a video has been tampered with or not for many other secret purposes. together. In fact, this study and similar studies provide specific insights into video editing issues, which are needed to develop better editing detection tools. ”

Although researchers are optimistic about the positive applications of this algorithm, never assume that it will be able to be used on mass products without proper protection measures.

Share the news now

Source : ttvn