“Despite using Artificial Intelligence, it was a fairly traditional process”

He deepfake by Lola Flores who stars in the new Cruzcampo campaign it is simply spectacular. Leaving behind haunting valleys and technical failures, the virtual replica of one of our most legendary artists demonstrates – as the impressive youth versions of the protagonists of ‘30 coins‘- that Spain is a true power in the creation of deepfakes. In this case, the result had everything going for it: family support (also providing graphic material), enough time and a strong brand behind it to finance the process.

Ogilvy is the advertising agency behind the project, and Metropolitan the special effects and post-production studio that has carried it out. A deepfake made in Barcelona. We have spoken with Juan Pedro Moreno and Javi Senovilla, creative directors of the first campaign; and Ramon Arteman and Nico Roig, director and member of the 3D department respectively of the second to give us details about the process and explain what difficulties they have had to overcome until reaching the final result.

5,000 images of Lola and two softwares to polish the image

The project, Juan Pedro Moreno, creative director of the spot, tells us, has been long, since the plan was for the campaign to come out in March of last year, “but the pandemic has given us room to have some room for maneuver and polish even the smallest detail “. The technical process of deepfake In itself, however, it has been much shorter: just “three or four weeks”, which took a little longer because their clients wanted to “go testing, changing funds, seeing different adjustments.” AND by having extra time thanks to the delay caused by the pandemic, they were able to increase the time spent on final touch-ups and the smallest details.

To begin with, Metropolitana had to compile images to serve as a basis: “The idea was to have as many ranges of face angles and expressions as possible. All the hours we had of footage were reduced to approximately 5,000 images with which we feed artificial intelligence“, says Ramón Arteman, director and co-founder of Metropolitana. That footage brought a problem:” The delicate thing was the resolution, the images were in PAL in low resolution, which led to obtaining a material that did not always have the best quality we wanted for the shot “, so many of them had to be tweaked to optimize them.

The fact that Lola Flores had such a long career in time was, paradoxically, problematic: “An added difficulty was that there were many images of Lola Flores’ entire life, but we had to choose a specific time to serve as a reference, something that we did between ourselves, the agency and the family itself. They showed us about a hundred photos and asked us which one we liked. We took an image of Lola that was like a totem “, explains Ramón.

Lola Flores 1 First image agreed between all parts of the production and the family of Lola Flores that would serve as a reference to create the deepfake. Image provided by Metropolitana.

That first reference was a photo of Lola when she was about fifty years old, but finally a later and higher quality one was agreed upon. “We have that photo as the Holy Grail. From it we compared eyes, eyebrows, chin, general figure … That was the photo that we did not stop looking to compare the result we were achieving“, explains the director of Metropolitana.

The next step was to record an actress creating a 3D model, “a mold on which a surgical framework would be woven on which Lola’s face would be recreated”, says Javier Senovilla, one of the creative directors of the spot. The actress would be the basis of all the gestures on which the images of the real Lola Flores would later go, “which then it would be outlined by digitally retouching expressions of the singer and details of the skin, such as the part where the hair meets the forehead, which had to be retouched separately to bridge the differences between the model’s physique and that of Lola Flores, “says Roig.

Lola Flores 2 On the left, the image that was finally used as a reference by the team in charge of the deepfake, with Lola Flores in her fifties. On the right, a frame of the final result. Image provided by Metropolitana.

In this process, two specific programs to create deepfakes. On the one hand, FaceSwap, an open source software based on TensorFlow, Keras and Python. It has a very visual appearance that makes work easier, at least in the first phase. For the second phase, something was required that allowed us to go much deeper into the details, in a more artisanal way, and using code. That second software was DeepFaceLab, used for those long finishing touches.

Nico Roig, from Metropolitana’s 3D department and one of those in charge of bringing Lola Flores back to life in a certain way, explains that “DeepFaceLab gave us better results when it came to controlling all the parameters, which is the quid of the question “, and that” the final touch-up, not only of the deepfake but its final integration, composition, etc; it took us almost as long as his own deepfake. The last 10% took almost as long as all of the above. Having time is what allowed us to work on it like this to keep everyone happy: Cruzcampo, the agency, the production team, the director, Lola’s family … “.

Above, recordings of the actress on which the Above, recordings of the actress on which the ‘deepfake’ is based. Below, footage of Lola Flores in positions similar to those of the actress.

This last part, the family, was key from the very beginning, when it was just a script: Javier Senovilla, creative co-director of the spot, tells us that “Rosario Flores told us not to change a single comma, these are things that my mother could have say”. From there they were advising on such minute details as the correction of gestures, the shape of the eyebrows “or comments that we would never have taken into account, such as that the hairstyle would never have been in the middle,” he says. Even Lolita imitated her mother’s voice in a process that also went through deep post-production. “Then we would experiment with recording”, says Moreno, “doing tests with limiters to modify the pitch, and using special software we modulated the timbre to bring it closer to the original”

In that final work, little appreciable touches for those who see the ad, but typical of those who pay attention to detail, such as optimize the look so that it was “more feline” or adjust the size of the ears, since those of the actress did not fit completely in Lola’s. They also purposely made an obstruction of the face with their hands, something that is not done in the deepfake “traditional” (all the quotation marks in the world to talk about a “deepfake traditional “) to, according to Nico,” convince the undecided. “” It was a fairly artisanal and subjective process, “he concludes.

During the last months, with the video in development or already created and waiting for the advertising campaign to start, from Metropolitana they eagerly waited for the moment to see the light. “We didn’t expect it to make so much noise, the response is being great, “says Ramón. 16,000 retweets and 30,000 ‘likes’ in his first five hours posted on Twitter. In that waiting time they were able to test its effectiveness by showing the spot to those who did not know the nature of the deepfake, always with good results: but Moreno comments that “all doubts were dispelled when his own daughters gave the go-ahead to the recreation we had done”.

Source: Xataka by feeds.weblogssl.com.

*The article has been translated based on the content of Xataka by feeds.weblogssl.com. If there is any problem regarding the content, copyright, please leave a report below the article. We will try to process as quickly as possible to protect the rights of the author. Thank you very much!

*We just want readers to access information more quickly and easily with other multilingual content, instead of information only available in a certain language.

*We always respect the copyright of the content of the author and always include the original link of the source article.If the author disagrees, just leave the report below the article, the article will be edited or deleted at the request of the author. Thanks very much! Best regards!