Meet Soul Machines, the company that produces the “virtual workforce” for the metaverse

Translated from the article by reporter Alexis Ong, editor of the technology magazine The Verge.

Sam is not a flesh-and-blood character, but a digital doll created by New Zealand-based technology company Soul Machines. Designed to conduct short conversations with users, Sam operates with a “digital brain” and can analyze facial expressions via webcam. At one point, Sam asked me to smile but didn’t seem to notice the brightest “baring teeth” action. I wonder if I’m not good at expressing my emotions. When Sam asked me if I understood anything about robotics, I replied, “ No, but you were going to tell me about it .”

“ Great answer! ‘, Sam raised his voice. “ You should do my job! “. Sam then explains that his lines and actions are not recorded, but are all outputted in real time similar to natural interactions. Humans OS 2.0-powered virtual dolls can use their hands, and will probably soon be able to take advantage of full-body movements as well.

Sam asked, “ Does that bring me closer to you? ”

When we think of the metaverse, we often think of it as the virtual world that allows the user, or rather the avatars – the user’s selves, to live, work, and participate in virtual activities. But it’s Soul Machines that fills the metaverse with virtual populations, a concept Soul Machines co-founder Greg Cross calls a ” virtual workforce .” This will be a large number of “Sams”, which operate similarly to the NPCs in the games you still play.

” When we play games, we often create our own personality, when we interact with children’s friends, we bring a different personality than when we go out drinking with friends. ” , Mr. Cross explained. “ As human beings, we always adjust our personalities and our roles to suit the circumstances. We can create those concepts on a virtual doll.”

At the moment, Soul Machines is focusing on creating virtual dolls that are applied in customer care and present in front of the crowd. However, they have also collaborated with many well-known individuals in the film industry (such as “Iron Man” Robert Downey Jr.) or Black Eyed Peas artist will.i.am.

Soul Machines isn’t the only business looking to create virtual dolls. The market also witnessed another player, the AI ​​Foundation (AIF) with the backing of many leading industry experts. However, Soul Machines stands apart from the others on two points.

Next to Mr. Cross, Soul Machines co-founder is Mark Sagar, a famous Hollywood visual effects expert. His groundbreaking research catapulted complex brain science into the art of virtual anthropomorphism, becoming an integral part of today’s cinematic techniques.

A virtual brain that can learn more knowledge is the second big difference. However, at the present time, these artificial intelligence virtual dolls still need direction from real people to be able to go in the right direction and take reasonable actions with the actual circumstances. The ultimate goal of Soul Machines is to teach a virtual doll to make decisions based on goals (and in the future, on the merits of each action). However, this is still a distant dream at the moment.

“ At some point in the future, you can create a version of yourself or even multiple versions of yourself, and they can run free, make money for you, make money. for your business, while you engage in other less boring activities, ” said Mr. Cross.

The first image that crossed my mind was the resource “farm” bots in online RPGs. But Cross goes a step further, suggesting that one’s self could be involved in playing Call of Duty, for example. “ These types of virtual dummies, what we call virtual people are under the control of real people, they can mimic actions under the direction of real people, ” he explains.

Mr. Cross’s mention of the Call of Duty game aroused a bit of curiosity, because this is an online game that relies heavily on player skills and is heavily influenced by cheats/hacks. “ The charm of online multiplayer is that you can play against another player, and know you’re competing against another human brain, ” said Mark Johnson, a technical culture researcher. Digital and emerging workforces. “ In this respect, I don’t know what the difference is between a virtual dummy and a good console… and while it’s interesting, you just look at how many people prefer to play against humans versus people who like to play against the console. see, no matter how good the machine is .”

Most online games do not allow players to use bots (automated software) or third-party services. “ There are many obvious reasons for such rules, using ‘virtual dolls’ would both break the law and force us to think more deeply about the definition around play, work, factors. equity and how we spend our time, ” said researcher Johnson.

Mr. Cross emphasized that Soul Machines only wants the good of humanity, and affirmed that no powerful individual would take advantage of virtual doll technology to amplify extremes. “ I think it’s important for companies and communities to learn from each generation of technology we’ve created, and think about where we want to go in the future, ” said Mr. Cross. at the same time mentioning two areas, health care and education, aspects with inadequate infrastructure and resources.

“ We don’t see that virtual dolls will replace health professionals and educators, we see technology as a system that strengthens them, ” said Mr. Cross. This will be an opportunity for the underprivileged in society to access an “ alternative version ” of education and healthcare, and Soul Machines’ virtual dolls should, in theory, help the majority of the community. co-experience things they may not have had.

In addition, a survey conducted by the Soul Machines research team shows that users are more willing to disclose sensitive factors, such as financial status or health status, to virtual dolls than to real people.

Augmented reality technology, virtual reality has been applied in education, but the use of virtual teachers to replace teacher love is still controversial. “ Children learn to socialize, regulate emotions and personality through observing adults around them – perhaps observing teachers for about 30 hours a week for 12 years, ” said Nick Kelly, expert. in education and cognitive system design at the Queensland University of Technology. He admits technology can improve the learning environment, but this is a complex and potentially dangerous issue.

“ No matter how realistically constructed the virtual dolls are, how high-resolution they are, or how cleverly programmed to simulate the action of interest, they cannot replace real teachers who know how to be human – whether in the present moment or in the immediate future , “said researcher Kelly.

In addition to the appearance of the virtual workforce that can influence the way users see reality, digital dolls must operate under certain conditions. “ When creating a virtual character representing a large corporation, it is obvious that the virtual character will not be able to express negative emotions ,” said Mr. Cross. Having a smiling face of the corporation will change the way we interact with customer service, and at the same time carry the risk of underestimating the improper treatment of corporations and businesses. workers are people.

Once entering the metaverse, companies like Soul Machines will provide a huge pool of human resources, creating virtual faces of real companies from all industries.

The artist version will.i.am was created by Soul Machines.

But if virtual dolls become the “backbone” workforce of the metaverse, the next evolution of the gaming industry will have to be as disruptive as healthcare or education. Games are no longer a mere pastime, but have become a billion-dollar industry with a unique ecosystem of developers, artists, content creators, and even Hollywood stars.

The purchase and sale of virtual items with high liquidity, mechanisms to entice players to spend money, etc. bring significant profits to developers. However, anthropomorphizing this money machine with virtual dolls will likely have unintended social and cultural consequences.

“ In recent years, the blockbuster game industry has not created a good reputation for its unethical methods of making money, even when looking at games from the perspective of buying virtual items (microtransactions), treating games as translations. online service , inventory , item betting or whatever, ” said researcher Johnson. ” All of these events bring with them ethical questions that have so far emerged and are slowly being answered .”

Soul Machines’ work also raises questions about how virtual versions of real people work in the metaverse. Can an entity representing a person in one system be free to move into another? How will the being affect the virtual life and the real life? How will human relationships change?

Limiting virtual dolls to the form of NPC helpers would be a reasonable first step to probe the situation. According to Cross, Soul Machines will gradually build virtual assistants that can respond in real time, based on input signals they receive around. Step by step, they will gradually evolve to become more modern, becoming solid bricks for the metaverse.

“ A virtual workforce will become more functional as the metaverse continues to evolve, ” said Cross.

If we consider the virtual workforce in the metaverse as a tool, it is obvious to take full advantage of the power of an AI system that can be present anytime, anywhere. This point is still beyond the reach of modern technology, we probably won’t live long enough to see the future Soul Machines envisioned. But Sam, or some part of the Sam system today, would be able to have that honor.