Caution urged around rapid embrace of transformative tech

Artificial intelligence has transformed the educational landscape and reshaped how learners think and learn, yet like a double-edged sword, it has also sparked concerns over its negative impact on learners as well as academic norms.

AI is becoming an ever-increasing presence among children, according to the sixth China Internet Usage Survey Report for Minors published by the Institute of Journalism and Communication of the Chinese Academy of Social Sciences and Social Sciences Press.

Nearly 90 percent of internet users under 18 have heard of AI technology, 42.4 percent express a strong interest in it, and 19 percent have used generative AI products, the survey said.

However, some parents worry that developing AI courses will increase students' academic burden while doing little in enhancing their digital literacy.

Including AI in college and high school entrance examinations will make AI education overly focused on exams, according to Xiong Bingqi, director of the 21st Century Education Research Institute.

The 2024 Report on the Internet Use of Chinese Minors went so far as to caution that AI is jeopardizing the cognitive abilities of children in a way that can't be easily detected.

The report pointed out that children are more likely to fall into information cocoons and cultivate stereotypes with the widespread use of intelligent recommendation algorithms.

Excessive use of generative AI applications such as ChatGPT and Sora may hinder children's ability to discern real information from fake, the report said.

Li Yi, director of the Beijing Municipal Education Commission, in an interview with China Education Daily, said, "The introduction of AI raises questions as to how to ensure data security, prevent algorithm biases, balance the roles of AI and human teachers, and avoid compromising students' social skills and critical thinking abilities as a result of excessive reliance on it."

The use of AI also gives rise to a series of ethical challenges regarding ensuring the fairness of assessment results. Yang Zhiming, a professor from Hunan Normal University, used ChatGPT to write an essay on innovative talent cultivation as part of a college entrance examination.

When the essay was assessed, 13 human experts gave an average score of 49 out of 60 points, whereas ChatGPT scored itself 54 points, and Chinese technology company iFlytek's AI chatbot even gave a score of 60.

"AI can serve as an assistant, but cannot be given the final say, because doing so may pose significant risks, just as no one would dare to fully rely on a prescription written by AI," Yang cautioned.

In college, AI has been embraced by professors and students to assist their academic research.

A 2024 study on the use of generative AI among Chinese university teachers and students by MyCOS, a higher education consulting firm, suggested that among the over 3,000 university teachers and students interviewed, almost all had used generative AI in their studies and work, with only one percent of university lecturers and students never having used generative AI.

The study showed that university students mainly use generative AI for retrieving and compiling information (54 percent), analyzing and visualizing data (54 percent), and translation (34 percent). In addition, nearly 30 percent of university students write papers or complete assignments primarily by using AI.

Despite the benefits, AI technology is not without risks and limitations. Many university students have reported that the references of AI-generated papers are fake with links to papers inaccessible, authors' names made up and the publication information inaccurate.

Overreliance on AI technology in completing academic assignments may lead to an integrity crisis. Excessive and indiscriminate use of AI tools could alienate students to being mere puppets to technology, Wang Zhengqing, a professor at the School of Education at Southwest University told 21st Century News.

Incidents of AI-assisted paper writing have prompted a debate on whether it constitutes a violation of academic ethics.

Last March, a professor at a university in Beijing published a Science Citation Index paper in an international academic journal, which was later found to contain a common ChatGPT prompter, sparking concerns on how AI may harm the accuracy and rigor of academic research.

Several universities have announced that they will pilot a system for the detection of the use of generative AI in undergraduate theses.

In November, Fudan University issued a regulation on the use of AI tools in writing undergraduate theses, which allows students to use AI in areas that don't require innovation, such as literature retrieval and formatting, but limits its use in research design and data analysis, data collection, visualizing results and translation and language polishing.

Shen Yang, a professor at the School of Journalism and Communication and the College of AI at Tsinghua University, was quoted in Global Times, saying, "Universities should establish clear standards, integrate AI into teaching practices, train students' logical thinking ability, enhance the transparency of AI technology and guide students to use AI responsibly."