AI gives the visually challenged a chance to enjoy movies

Despite losing her eyesight at 52, Beijing resident Zhang Hua has maintained her enthusiasm for the cinema.

For more than a decade, she has been a regular visitor to the China Braille Library to enjoy "barrier-free movies", a special version tailored for people with visual impairments. These movies, typically screened on Tuesdays, feature added audio narratives that explain visual elements such as character movements and settings.

However, the main regret for her and fellow movie fans is that most of these films were released several years or even decades ago, primarily due to copyright issues.

A new service jointly launched by the technology company Tencent and the library will expand their options. Titled "Barrier-free Cinemas", the service, aimed at benefiting China's more than 17 million visually impaired individuals, is scheduled to release over 600 films and TV dramas.

Most highlighted projects include actress-director Jia Ling's YOLO, this year's highest-grossing film, the crime series The Long Season, and the sci-fi TV series Three-Body, adapted from Liu Cixin's Hugo Award-winning novel.

Li Daren, deputy senior president of Tencent Video, the streaming site that provides the service, reveals that cutting-edge artificial intelligence technology will be employed to speed up production processes.

"In the past, creating a special version of a feature-length movie required around 30 days, with professionals needing to watch the film 13 to 15 times, write the narrative script, and perform the voice-over job," he says.

The AI models developed by Tencent can automatically generate a script based on the video content. "After the wording is polished, the AI models can also generate the voice-over automatically, allowing our volunteer workers to make slight adjustments and double-check at the end. The entire process can be reduced to 10 days," he adds.

The service also plans to recruit a total of 100 volunteers, with most expected to be selected among movie fans in colleges and within the company, to create barrier-free versions of the selected programs, including actress Liu Yifei's costume drama A Dream of Splendor, the romance series The Oath of Love, and actor Wu Lei's Love Like the Galaxy.

He Chuan, deputy director of the China Braille Library, says that such movies, known as "audio description" in the West, can be traced back to their earliest exploration in the 1970s.

He recalls that the library began to explore the production of barrier-free movies in 2003, with the first title being director Ridley Scott's historical epic, Gladiator.

But it was the historical epic The Founding of a Republic, a tribute to the 60th anniversary of the founding of the People's Republic of China, that became the first of its kind to be officially released in the barrier-free format on discs in 2011.Since then, more than 860 such movies have been produced and published, adds He.

"Besides incorporating voice-overs, the remade video content also includes footage of a host performing sign language, framed in a small window at the bottom right. This practice allows people with hearing impairments to comprehend the audio background," he says.

With the experience accumulated over past years, He says the library has established a set of criteria to ensure the quality.

"To write the narrative script is akin to a translation task. The script needs to be accurate, beautifully crafted, and the writer should exercise restraint when incorporating hints within scenes. The creators must discern when to speak and when to hold back," he explains.

Acknowledging that achieving such nuance and delicacy remains a challenging task for AI, he says they will conduct further training on AI models to make digital creations more humanlike.

"Over the past decade, we have noticed that watching films is the preferred form of entertainment for the visually impaired, garnering larger audiences than activities such as reading or learning musical instruments," says He.

Nevertheless, with the rapid advancement of the Chinese film industry, he says that they have observed movies becoming increasingly challenging to narrate due to the inclusion of more action sequences, elaborate stunts and complex artistic expressions.

"It's a significant challenge but we are confident that we will improve in the future," he says.