I just watched Biggie Smalls perform ‘live’ in the metaverse

Smalls’s hyperrealistic avatar is not just an impressive technical feat. It is also a critical test of two important questions that we will soon face if metaverse platforms gain momentum: whether people will pay to watch an avatar of a deceased artist perform and whether this business is ethical.

Smalls has not been the first deceased artist to be resurrected. The controversial, but very popular, method of reanimating dead musicians has been Hologram performances. Buddy Holly, Whitney Houston and Amy Winehouse were all transformed into holograms for gigs that were held after their deaths. One of the most notable hologram shows was by Smalls’s rival Tupac Shakur, who died in 1996 but “performed” at Coachella in 2012.

Holograms, however, are inherently limited. To create the illusion of 3D, they require that the audience sit at a certain angle. The metaverse allows people to view a more realistic avatar and interact with it. This is something that the Smalls team hopes to be able offer in the near future.

What was remarkable about Smalls’s performance Friday was his realism. His movements, mannerisms and facial expressions were astonishingly lifelike.

But there was a few hiccups that reminded viewers that Smalls was an avatar. Smalls was often seen in scenes with live rappers and would sometimes stumble into his co-performers. Smalls would sometimes wander away from the center circle where he was performing if a rapper supported his lyrics. He didn’t respond to the other rappers like a human performer would.

Smalls’s avatar was more “natural” off-screen, in pre-recorded digital segments where his likeness roamed through ’90s-era Brooklyn. His movements weren’t unnatural, his clothes were wrinkled, and his head turned and hands moved in ways that made it hard to tell this person was a digital creation.

The technology behind this visual feat took years to create, according Remington Scott, the VFX director responsible. Scott is the founder of Hyperreal, the studio behind the motion capture that made Andy Serkis’s Gollum character come to life in The Lord of the Rings. In this case, an actor was used, but an avatar used the same techniques. Scott states that “when we used this technology for feature films, it would have taken six months and millions of dollar.” “Now, we can do it in six weeks and at a much lower cost.”

The team created a database of “micro-expression reference materials,” analyzed “pore-level resolution imagery,” and tracked the elasticity of sub-skin layers to understand how Smalls’s facial skin moved, Scott explains. These tiny changes in facial expression were essential to create a real avatar.

All that research paid off. “I have seen the avatar through the building process… and it feels very real to me. Voletta Wallace, his mother, shared her thoughts via email. “I see my son’s characteristics in the detailing.” “The avatar turned out not to be all I hoped for,” Scott says. Scott recalls that Wallace was present when Smalls’ avatar was unveiled to him. Scott said that there was no one in the room who didn’t see it. “At that moment we surpassed all technical achievements and were in the realm emotional real simulations

Part of the reason Smalls was a prime candidate for a VR concert is that he was a star without any live recorded performances. Elliot Osagie (founder of Willingie), a digital media company, said that Biggie only released two albums and never toured. Fans were able to see their hero in person and introduce a new generation of fans to the legendary rapper through the virtual performance.

That’s where Wallace, who is also executor of his estate (estimated to be worth around $160 million), comes in. It was an emotional project but there was no doubt that it was also a business opportunity. Scott said that Wallace and her son’s estate were looking for “opportunities” to bring Smalls back to his fans. Smalls could be a part of the metaverse, which is dominated by younger generations. Wallace confirms this: “I envision more concerts, videos of his music, commercials, animation, films, and more opportunities in the metaverse.”

Wallace, Hyperreal, Willingie, and Meta refused to disclose how much Wallace’s estate paid for the avatar, or how much Meta paid for exclusively hosting the VR concert. Meta also did not respond to MIT Technology Review’s questions about its role in the concert but did insist that the event–which was held on the company’s flagship metaverse platform, Horizon Worlds–was not held in the metaverse, but rather in virtual reality. Meta didn’t respond when asked what the metaverse was.

However, Scott states that ownership is what distinguishes his avatars from the traditional ones. He says that other avatars don’t have rights for actors and performers. “But our model is to reverse that. We create digital identities for talent, then we move forward.” Smalls’s estate had full input in creating his digital twin. But how can you ensure that an artist has the right to decide what can and cannot be reproduced? “That’s the million–or should we say billions-of dollars question,” says Theo Tzanidis. He is a senior lecturer on digital marketing at the University of the West of Scotland. He has written extensively about the holograms and metaverse music.

For the most part, celebrities and artists do not currently include clauses in contracts or wills about how they would like their likeness used in the metaverse or by artificial intelligence, but Tzanidis would not be surprised if the practice were to begin soon.

We don’t know if Smalls would consent to the use of his likeness. And there is no way that he could have imagined a platform like Horizon Worlds. To Osagie it is important that an avatar stays true to the era of the artist and does not do anything that they couldn’t. Osagie uses Miles Davis’s career as an example for his upcoming metaverse project. If you want to tell a story about Miles Davis’ music, that would be great. You could not animate Drake’s avatar and have him play cards with Drake. For me, the real line is that the artist is doing

That may make some sense. In a future where avatars are becoming increasingly lifelike, businesses expand, and the line between real life and metaverse is blurred, Miles Davis may be able to play cards with Drake with or without the approval either of their estates.

Even the creators of Smalls’s concert took creative liberties. One scene showed Smalls’s avatar on the balcony of what is presumably his apartment; the camera pans over a portrait of former president Barack Obama embracing Smalls, an event that could not have happened because Obama was elected more than 10 years after the musician’s death. Smalls is shown answering a phone, a product that was not available in his lifetime at least twice.

Tzanidis thinks the lack of legal framework is problematic. It goes beyond the traditional boundaries of art. Tzanidis believes that there is a problem with the lack of a legal framework. What if you could learn from experts in your field? What happens when ?”

can be re-created?

This vision is already being realized: a virtual version of American golfer Jack Nicklaus will launch soon on an unnamed virtual platform. Fans will be able interact with him and he’ll share stories about his victories and offer tips.

Nicklaus was involved in the creation and maintenance of his avatar. Smalls was not. There is no way to verify that his wishes were the same as his mother’s. Tzanidis states that there is no metaverse rulebook. “There should be .”

Osagie states that Smalls’s Thursday concert is not the end. Scott and he are looking at other gigs and games as well as performing at Coachella by Smalls. Scott is excited about the possibility. Scott says that the metaverse is another reality and Biggie is still alive in it. He is excited about the prospect. “I think a lot of fans will love that world.”