Indepth: The world of virtual pop stars - Will AI & algorithms change the music biz?

In 2018, a Japanese man by name Akihiko Kondo made news by ‘marrying’ Hatsune Miku, a virtual pop star. He was one among the many who married fictional characters in a country known for its uncanny fascination for anime and cartoons. Kondo identified himself as fictosexual, a catch-all term for those who are sexually attracted to fictional characters. Such cases no longer raise any eyebrows as they have become commonplace in the zeitgeist of the times that we live in, with virtual influencers and virtual pop stars assuming real-life personas.

Recently, the record company Capitol Music Group dropped ‘virtual rapper’ FN Meka following allegations of racial stereotyping, including the use of N-word. Industry Blackout, a black activist group, had sent the record company an open letter alleging that FN Meka was not only offensive, but it was a direct “insult to the Black community and our culture”.

The AI-powered robot-singer, fashioned like a cyborg and looking like an alien, had already done something that would make any real-life singer worth his salt squirm with envy. The ‘singer’ amassed 9 million TikTok followers and 500,000 subscribers on Spotify!

We can expect more such AI-powered virtual singers, especially in the metaverse, say experts. “As technology advances to more and more futuristic and real-world simulations, we can expect to see more unprecedented creations, which can pleasantly or unpleasantly surprise us,” noted Lumos Labs Founder Kaavya Prasad.

With metaverse gaining rapid traction and the creator economy becoming a fully-fledged revenue model, we can expect to see emerging technologies being used in diverse circumstances, she adds.

“We can definitely expect more such Augmented Reality-powered virtual singers,” opines DaveAI CTO-Co-founder Dr Ananth. “Today, AI can produce extremely realistic and interesting art, including music, imagery and other experiences. But more and more live people will be using avatars to express their virtual and digital selfs.”

Ashray Malhotra , CEO-Co-founder, Rephrase.ai, points out that we are at very early stages of virtual humans being a part of our lives. “Be it in the Metaverse or a part of the usual workflow in our work lives, augmented versions of ourselves and completely digitally generated characters are only going to expand their presence in our lives.”

With real-world simulations and futuristic technologies becoming part of the metaverse, we are likely to witness more Augmented Reality-powered virtual singers in the future, says Gaurav Dagaonkar, co-founder and CEO, Hoopr.ai

“The experience of such creations can arouse a mixed bag of feelings though these will definitely spur the creators to come up with such AI-powered creations - not only in the music space but across the range of other categories and segments. Further, as the world has just started to explore the potential of digital twins, the influence of AI-based characters is only going to expand from hereon. So don’t get surprised if tomorrow you encounter a digital avatar of your favourite actor, composer, or producer as the metaverse is all poised to take over the world,” Gaurav adds. 

Kaavya Prasad feels that within the metaverse, we have seen virtual art galleries, virtual concerts, workplaces, retail stores, and many more similar implementations are to be expected in the near future. Hence, she adds, virtual artists should not be beyond the scope of emerging technology.

“Collaboration of artificial intelligence (AI), augmented reality (AR), and virtual reality (VR) has led us to have these virtual artists such as Miquela, Lu do Magalu, etc., who also exhibit the extent of innovations that technology can achieve today and it gives developers and production houses an opportunity to innovate too,” adds Prasad.

According to her, the presently thriving creators’ economy can include not just the mainstream singers, actors, artists, etc., but also the producers, composers, AI researchers, developers, and more as that is what decentralisation implies.

She feels that going forward we can also expect virtual actors, movies, gaming simulations, restaurants with virtual servers, offices with colleagues joining in virtually and having virtual conference rooms, playrooms, etc. The future of the metaverse, she adds, is not bound by physical world limitations and is an inclusive space for all creators.

The pitfalls

In the case of FN Meka, it wasn’t all the work of AI. Though a human gave the voice, FN Meka and his music (lyrics, chords, sound, melody, and tempo) was derived in part from Artificial Intelligence. There was ‘some human involvement’, admits Anthony Martini, the co-founder of Factory New, the virtual record label company behind FN Meka. An anonymous black man sang the songs, he admitted in an interview.

He, however, does not rule out the possibility of a fully computer-abled singer bursting into the scene in the future. He said his company is working “towards the ability to have a computer come up with and perform its own words – and even collaborate with other computers as co-writers.”

Will the emergence of AI-powered singers minus any human control create more FN Meka-like controversies? With little human supervision or control over such creations, will we witness revolting and outlandish lyrics, leading to social unrest and controversies? Experts feel that it as much a legal problem as it is an ethical one. It is also about the sentience of AI-powered entities that we will witness in the days ahead. How intelligent can AI-created ‘tech beings’ be? Can they be sentient, after all? (That question reminds us of the recent sacking of a Google engineer because he argued so convincingly that its AI chat-bot was sentient.)

Undoubtedly, says Gaurav Dagaonkar, both legal and ethical challenges are associated with creations based on AI technology. According to him, given the nascent nature of the metaverse, the regulatory framework related to virtual worlds is nonexistent. “This means no one knows what is right and what steps could possibly land them in legal trouble. This void is also stunting the growth of the metaverse and could spell trouble for segment stakeholders in the future, especially for content creators and technology developers. The ethical facet of the problem is also significant, but thankfully we can easily manage the issue with the help of technology. For example, one can easily restrict and control the lyrics produced by AI within the preset levels of tolerance although doing the same with a lyricist is not possible. This means that the censoring of AI-based lyrics can be done without creating much fuss though the same done to lyrics written by human beings could easily snowball into a controversy,” he says.

This is definitely an ethical dilemma, agrees Ananth. But it is easier to restrict or censor AI-produced lyrics than human ones. On the flip side, tolerances for AI mistakes are way lower than human mistakes, he contends.

Kaavya Prasad says that with all technological advancements come certain caveats as at the end of the day. According to her, virtual worlds are still very nascent and not well structured or regulated as the real world. Thus, she adds, we come at a crossroad when it comes to AI ethics and regulation.

In the case of FN Meka, she adds, the developers behind this AI were not the ideal representatives for it. According to her, in the digital world, one can choose how one presents itself, yet virtual influencers and creators are somehow inspired by real-world people.

She feels that AI and other related technologies are meant to enhance our lives, not to supplant it.

“The presentation of FN Meka, chosen by its developers, who could not have completely understood the rich history of the community they chose for it, was yet another example of how AI ethics is a field where the real world has a lot to learn and explore. With the way the technology is advancing in the metaverse and AI domains, it would feel like these virtual robots and virtual worlds are taking over, but the metaverse is not here to replace reality, but rather to access knowledge, learning, and entertainment etc., that you cannot in the real world. It's to enhance the way we live, not substitute it,” Prasad explains.

Artificial intelligence should be treated like a tool which aids human creativity instead of considering it as an owner of IP in itself, maintains Ashray Malhotra.

There are other ethical challenges. For instance: Does an artist really need to be a human to be considered an actual artist, who can perform and record music, and build a fanbase?

The artist is always human, maintains Dr Ananth. “The copyright to the AI entities’ creations will lie with the creators, or made public. What AI creates, may appear like art, but it is only a very good mimicry of the world we show it, along with some chaos thrown in. But without the appreciation and feedback of humans, AI is also equally likely to produce strange things or repetitive things. In that case, the humans who guide it, may be the owners of the copyright,” he adds.

“We imagine it to be something like, say Adobe Photoshop. If a company makes bad, unethical content using Adobe Photoshop, the ethical and legal challenges of this should be borne by the company. Similarly, when it comes to either AI music as in this case, or AI video content as in the case of Rephrase.ai, the final content is the intellectual property of the company or the individual which uses this software. The major change is that as technologies become even more powerful, it requires even more careful use to ensure that we use these capabilities for good,” says Malhotra.

At least given the legal infrastructure today, says Ashray Malhotra, the answer to this ethical challenge is fairly clear. “Since AI technologies are used by some company or person, the intellectual property of the content resides with that entity. Hence, to create a particular kind of music or video, you could either use an actual outsourced band or video editing team, do-it-yourself or in-house or use an AI technology to create content,” he says.

The power of AI

According to Kaavya Prasad, the inclusivity of metaverse and Web3 world makes it possible for sophisticated algorithms that can write a whole song and also make it possible for a virtual artist to voice it. Global singer, Grimes, she points out, formed a new AI girl group, NPC, which has already released its first song in November of 2021.

“Additionally, virtual singers can be found as far back as 1996 with Kyoko Date, a Japanese 3D CG character, also termed as the world’s first virtual pop star. Therefore, virtual artists have been in the game, in some capacity, but are gaining more traction and sophistication now as technology is advancing and expanding to include more users. As user experience is moving towards simplification and easier interfaces, people have more tools and easy-to-understand technology to play around with. Presently, there is also a number of song writing software already available on the market. Further, autotune has become a fairly common part of the music industry and sampling of each other’s music for their own use is rampant, with legal processes in place as well. Thus, we can definitely expect to see more AI-assisted singers with their own lyrics and can even have real world and AI collaborations in the future,” she says.

And what about the ethics of these situations?

According to Gaurav Dagaonkar, there is no denying the fact that human beings will continue to remain at the epicentre of all creations produced and powered by new-age technologies including AI. In fact, he adds, whatever AI creates is the result of the things and exposure offered to the technology by us. Whenever the AI technology is used to create lyrics, music, or some other stuff,  the Intellectual Property Rights will remain with the particular person or firm that is using the technology for content creation. Therefore, this ethical challenge can be easily addressed with the currently existing technology and intellectual property provisions we have across the world.” 

We would have to trust the record labels and production companies to abstain from unethical practices for now, says Prasad. As the industry is expanding, we can expect to have more eyeballs on the space, thus leading to more regulatory clarity with regard to intellectual property, copyrights, patents, AI-bias, risk of surveillance, and so forth, but we need to trust the systems in place for now.

There are also other challenges. For instance: Does an artist really need to be a human to be considered an actual artist, who can perform and record music, and build a fanbase?

This would be a highly subjective concept as to what constitutes an artist, says Kaavya.

When it comes to real-world artists, she adds, it is also based on preferences of the consumers. “But as a solution building their own Lumos Metaverse, we would like to see similar innovations grow into more diverse solutions, thus, futuristic ideas such as virtual artists should not be discouraged; rather they could instead be acknowledged as a different genre altogether. Hence, the popularity they gain and the traction they receive would be a result of consumer behaviour, as is the case with the real world. We already see these virtual characters having millions of followers and association with brands and record labels, therefore, constricting any form of talent expressed in a certain way which is new and unfamiliar should not be encouraged and should not be grounds to restrict any technology,” says Prasad.

But if we go into the influence that these AI artists have or the conduct they should incorporate into their day-to-day activities and promotions, there is much social and cultural ground to be covered, she maintains.

“As we can see in the case of FN Meka, social and cultural data for AI can be a difficult feat to process and quantify, which can then lead to developers using their own knowledge and own experience which might be limited in some cases. There are certain other challenges to consider here as well; creators often have a global presence, a global audience they cater to and being cognizant of global codes of conduct or significant cultural and social values equates to a lot of raw data that needs to be quantified for an AI-based artist. Additionally, there is a human touch that is completely lost in these instances which can again lead to followers or fans feeling a certain gap between the artist and themselves, which would be quite the opposite of what real-world artists strive for. Furthermore, following a virtual artist can lead to mental fatigue and unrealistic expectations for aspiring fans or followers, both in terms of appearances and how hard should young developers work to have their creations liked as much. Though it is common knowledge that these virtual artists are created through a programme, their appearances itself imply a certain bias with respect to how a person should look like in that particular scenario,” she says.

Virtual record companies

“Even with all the money labels devote to finding talent, the success rate is a pitiful 1%. Now we can literally custom-create artists using elements proven to work, greatly increasing the odds of success,” said Anthony Martini.

Is it an indication that we can expect more virtual record companies? And how will the world adapt to such AI-created singers and virtual record companies?

We will definitely see a surge in these companies, maintains Dr Ananth. In some sense, he adds, companies will have much more control over the AI creators than human ones, and it will be profitable to them in the long run. “But, it remains to be seen whether the public at large would accept the AI creations and AI personas and also whether the experts who created the AI can adapt the algorithms in time to the catch the next trend in music.”

Agreeing with Ananth on this, Kaavya says: “We can definitely foresee the emergence of more aspects of the virtual creator industry, including virtual record labels, singers, actors, etc. This could not only be a prelude to virtual singers but also a prelude to a hybrid entertainment industry, mixed with AI-based and real-world talent.”

With regard to custom-created artists, she says, it will be possible through technology but you cannot custom-create consumer preference.

There is enormous scope and possibilities that will definitely help in the emergence of more companies taking a plunge into the segment, says Gaurav Dagaonkar. He, however, is not sure whether the public will accept such AI-powered singers.

“Given the high level of control over AI-powered content, these firms can be tremendously benefitted from the prospects of the creator's economy. However, it will be interesting to witness whether the public at large will accept AI-powered creations with open arms. The acceptance of users is paramount and will ultimately define the future prospects of the AI-based content industry. As we move from physical reality to virtual reality, the transition period is more likely to power the hybrid economy where both physical and virtual realities come together and complement each other rather than merely jostling to replace one another,” he adds.

“Though there are tried and tested genres of art such as pop music and commercial films that follow a certain pattern, we cannot be 100% sure that these AI models would be a guarantee of success. Furthermore, the real world is a place which will always be there and the existing systems would always thrive considering they have the human touch, the grassroot level knowledge, and time. Creating an active virtual artist with music albums, promotions, virtual interactions, etc., is not an easy task even in the current technology landscape. With the metaverse, numerous obstacles have been removed and to some extent, stigma around virtual realities, gaming, digital concepts, and more is lessening by the day. But we cannot expect it to lead to a massive change or to AI singers overtaking real world artists,” she says.

“With the current conversation and fear for AI sentience and the highly complicated ethical implications, we have to be informed and aware when dealing with these innovations and ensure that there is a distinction between real world and virtual world to avoid the risk of mental health problems that can be exaggerated as we include more of virtual worlds into our daily lives,” she concludes.