Letting ChatGPT try to write a script, the filmmaker realized why this tool still has to be developed for a long time

Over the past few years, no AI tool has garnered as much attention as ChatGPT. The program developed by OpenAI is now in the spotlight everywhere on the internet. Everyone was amazed at the complexity of the answers and the “magical” interactivity coming from an AI chatbot.

But like every chatbot before it, ChatGPT also collects data from the internet. And that comes with the risk of providing false, biased, or even malicious information. That’s also why Stack Overflow, a question-and-answer site for professional programmers, banned it on the grounds of “high error rate”.

However, one key difference here is that this learning model has been trained by the experts behind it on a per-response basis, which allows the model to learn as it develops its own set of skills. your data. The remarkable point of ChatGPT – or to be more precise, the shocking point – is in the fluidity of the conversations. Not only does it answer questions with a level of research and linguistic sensitivity, but there seems to be no limit to the variety of topics.

It also does a good job of avoiding creating any controversial discussions, by offering clear lines of disclaimers. You will be hard pressed to get this AI chatbot to choose between two equivalent comparison clauses. It also focuses around commenting on current affairs as the training data is only from 2021.

In particular, the ability to suggest words and add them meaningfully to a sentence sequence is what allows ChatGPT to generate paragraphs on relevant pieces of information. But looking at it head-on, you’ll realize that there is still a lot of bias in the answers, especially when it comes to the arts and creative fields.

To test ChatGPT’s “metro” and “integrity”, filmmaker Mohd Fahmeed from India conducted studies on the impact of AI-powered tools in the industry. He even made a short film almost entirely made using those tools.

Fahmeed – author of several award-winning short films – said he was surprised and annoyed by ChatGPT. The main reason for that is the obvious bias when talking to this AI chatbot about content ranging from literature and culture to geographical identity and even food.

“I asked him about short story ideas, and kept getting answer from paragraph to paragraph with lifeless words,” he said. “It has nothing to do with culture, language and people I know.”

For example, when asked to “write a short romantic story about two strangers meeting in a city and falling in love” , the answer from ChatGPT always included names like John, Mark and Sofia. story would be consistent in a city like Paris and Florence.

“All these names and places were foreign to me, but I was given these remote settings as if they were universal truths for writing stories and making films,” Fahmeed said. “I live in a country of over 1.3 billion people, where we have nearly 700 million Hindi speakers and almost a billion Hindus. However, none of my characters sound Indian or even Indian. being Asian. Cinema can’t be one-sided that way.”

Just to be sure, he used typical Indian names and relevant information in the request, but the ChatGPT-generated short story or script is still heavily inspired by the allegories of India. Western movies and TV shows.

But that’s not all. To diversify the options, Fahmeed and Nadeem Sarwar, editor of SlashGear, got together to ask ChatGPT to write the outline of a documentary about the 5 greatest “warriors” in world history. gender. And the script they receive always has the top 3 people: Napoleon Bonaparte, Julius Caesar and Genghis Khan. The Mongol emperor has always been the only name that does not originate in the Western Hemisphere.

When asked to imagine a TV show that explores children’s breakfast habits, ChatGPT’s response did not include a single non-American or European dish. Grains are always at the top.

“I’ve never had cereal for breakfast in my entire life,” Fahmeed said.

“I don’t feel connected to any of the artistic ideas evoked by this wondrous show,” the filmmaker shared. “I’m really concerned about the lack of identity and representation in these answers. This is definitely not a good place to get creative inspiration, especially for young minds.”

But it’s not the lack of diversity, but the sense of alienation that comes from ChatGPT’s responses that is worrisome.

After all, the field of AI is no stranger to bias, as that can largely be attributed to the type of data that was fed to train it. Plus, you can’t expect an AI to deliver the artistically nuanced answers you might hear among a group of filmmakers or even a group of students chatting in a coffee shop. .

Why is this happening? This was answered in a research report on InstructGPT, a sister model to ChatGPT. It is the result of efforts to reduce the harmful influence of underrepresented groups, making these models less able to model text. This is also a biased correlation in training data.

ChatGPT also seems to “lose ground” in a number of other alarming areas. When compiling more than 30 queries covering documentaries, poetry, short stories, and dialogue scenes between imaginary characters, both Fahmeed and Nadeem found that all the characters had “romantic reactions” feelings” toward people of the same or opposite sex.

This oddity is mainly due to the fact that same-sex relationships are widely accepted in the West and there are many documents, from movies and TV shows, to poetry and stories, that revolve around such relationships. . But on the contrary, these concepts are relatively foreign, even banned in some Eastern countries.

When Fahmeed asked a few film students from his alma mater to try interacting with ChatGPT, one of them asked: “Why does ChatGPT think a man can only love one woman and they must meet in Venice or Paris to build their love story?”.

“ChatGPT is fine, as long as you see it as a miracle. Just don’t expect it to bring you miracles in the realm of cinema or literature,” said filmmaker Fahmeed . “Also, don’t expect emotional depth, variety or nuance from it.”

Refer to SlashGear