This Canadian VFX studio is using AI to cosmetically alter your favourite actors

TORONTO — Wondering how some stars seemingly never age? A Toronto VFX company says the answer is in a growing post-production trend in which cosmetic touch-ups are routinely made on actors’ skin and hairlines using technology.

Matt Panousis, partner and chief operating officer of the visual effects and tech studio Monsters Aliens Robots Zombies (MARZ), says the industry has been shifting in the direction of subtle, virtualcosmetic alterations for some time.

“What you discover as you go deeper is a lot of these celebrities are brands,” says Panousis. “Like a brand that needs to remain consistent — so does a celebrity’s face.”

The studio, which began as a startup in 2019 and does various visual effects work including bringing a disembodied hand known as “The Thing” to life in Netflix’s “Wednesday,” says the cosmetic side of VFX, whether fixing imperfections or de-aging actors for flashback scenes,is a large part of their business.

About a year and a half ago, they began using software called Vanity AI — a project that took the company three years to build and which they now plan to license to others — for work such as removing acne, crow’s feet, and under-eye bags.

In one example, they helped smooth out laugh lines and fade forehead wrinkles on Vancouver-born actor Joshua Jackson for a flashback sequence in the Peacock series “Dr. Death,” in which he portrays American neurosurgeon Christopher Duntsch. In the scene, the doctor is shown as a football hopeful during his university days,which takes place several years prior to his infamouscareer in medicine.

In another, an otherwise visible wig line is made invisible on actress Kathryn Newton, who dons a brunette hairpiece to portray Cassie Lang, the daughter of Avenger Scott Lang who goes on to become a superhero herself in this weekend’s “Ant-Man and the Wasp: Quantumania.”

“There are storyline touch-ups and then there’s the purely cosmetic, so when we say 80 to 85% of all productions use visual effects, a big chunk of that is just the cosmetic stuff,” says Panousis. “Think about it, why do people like Snapchat filters? Celebrities are people too.” 

Removing shine, blemishes, and de-aging is among the cosmetic work that has grown in demand, says Panousis — and the expectation is there even for TV productions, which typically work on tighter timelines and budgets. 

“These guys have a fraction of the budget and a fraction of the time that their film counterparts have,” he says. 

“It’s only gotten worse and with the proliferation of streaming and the amount of content coming out. There are not enough VFX artists in the world to do the work.”

What makes Vanity AI faster, Panousis says, is the way the tool can predict where the ideal placement of cosmetic fixes should be rather than continually modifying a face frame by frame.  

Through a series of dials, users can make alternationsthat can be applied to multiple frames until it’s a moving, fully altered shot — a process that would previously take days, compared to mere hours with their app,according to MARZ.

Panousis adds that as the demands for cosmetic changes and time-jumping storylines become more popular, the celebrity who fancies a more touched-up look is now an industry expectation they’ve also had to meet, albeit a soft-spoken one.

“Sometimes it’s contracted, which is typically for your major A-lister celebrities who are the extreme cases,” says Panousis who declined to name names due to contractual obligations.

”They’ll have it built into the contracts that say, unless there’s a certain budget associated with some sort of cosmetic enhancement, they won’t do the project.” 

“So much of the industry’s work is entirely invisible and Vanity AI as an application, is meant to be entirely invisible — you’re not really supposed to know if a person has done some work if they’ve done a good job,” says Panousis.

“It’s great for an actor who had a breakout of acne, or another who experienced a bad hangover, whatever areas you select, you can clean it very simply.”

Panousis acknowledges that in an era of synthetic media like deepfakes, there are other AI solutions to making cosmetic changes. Competitors exist in varying degrees, one of the more notable being Disney’s FRAN by Disney Research Studios which allows users to re-age and de-age someone at little cost. 

Still, he says Vanity AI meets Hollywood’s high-quality expectations given its use on multi-million dollar projects such as “Spider-Man: No Way Home,” and “Stranger Things 4.”

This year will mark the first time MARZ will make the capabilities of Vanity AI available to large and small production studios around the world to “democratize the effects.”

“Democratization means bringing down the barriers to entry and making the work accessible,” says Panousis. 

“If you think about it, quality VFX is something that’s only ever been available to the biggest and best productions, but if we can price it properly for these different segments in a way that is digestible, we believe that the tool is absolutely going to be used in other segments of entertainment.” 

Panousis ultimately envisions Vanity AI eventually being affordable for the average indie creator as well.

The goal for MARZ is to license out the app, which Panousis acknowledges may mean giving others the ability to perform the same work they do. 

“We’ve got to get the tech into others’ hands and that could very well be our competitors and that could be other VFX companies,” he says. 

This report by The Canadian Press was first published Feb. 17, 2023. 

Noel Ransome, The Canadian Press