Q&A with MOSS artificial intelligence research team

What is the difference between MOSS and chatbots like Siri, Xiaodu and Xiaoai that we use every day?

The relationship between the two is like comparing a smartphone to an old-feature phone. Chatbots were actually weak in artificial intelligence, and they were designed for chatting, just like traditional feature phones can only be used for making calls. In comparison, large-scale language models, like ChatGPT and MOSS, can do many things besides chatting.

As far as ChatGPT and MOSS are concerned, they have a general ability that can help humans accomplish various things, but they are presented in the form of dialogue. It can complete most tasks in the field of natural language processing, including machine translation, information extraction and error correction. They can also interact with the outside world and learn to create by using external tools. These are things that existing chatbots do not have.

What are the main differences between MOSS and ChatGPT?

The biggest difference lies in the parameter scale of the two models. ChatGPT has as many as 175 billion parameters, while the parameter volume of MOSS is an order of magnitude smaller, about one-10th of the former.

Another big difference is the ability to iterate. The larger the number of users and the more interactive data, the stronger the iterative ability of the model. ChatGPT, as the leader of the new round of the artificial intelligence race, is far ahead and collects a large amount of user interaction data.

The above-mentioned difference in the number of parameters and the amount of interactive data further lead to the gap in factual knowledge reserves between MOSS and ChatGPT, which shows that MOSS is more prone to errors when answering factual questions.

Yet when you pay attention to models such as ChatGPT and MOSS, you should pay more attention to their understanding ability, learning ability and thinking ability, rather than just their factual knowledge reserve. From the perspective of logical ability, I think the performance of MOSS is actually not bad.

Where is the technological innovation of MOSS?

MOSS is characterized by its small scale, which makes it easier to adapt to personalized models. In China, most enterprises have a very large demand for access to AI services, but if such a large model as ChatGPT is directly migrated, enterprises will not be able to use it. The scale of MOSS is very suitable. It can be deployed privately within the enterprise, and can be transformed into productivity after some data fine-tuning.

More specialized capabilities can be endowed on MOSS, such as access to external knowledge bases, and the ability to add searching or domain-specific tools. Just like human beings, when we do not have enough abilities, we can use tools to amplify our abilities. For example, we can look up dictionaries and use search engines to make up for an insufficiency of memory. The same is true for MOSS. If its own knowledge is not so large, we have to think of other ways to make it better at using various tools, so as to empower all walks of life. This may be the main difference between MOSS and ChatGPT in the future.

MOSS currently has a pure English interface, and the Chinese level is obviously not as good as the English level. What is the reason?

Our primary goal in developing the MOSS model is to verify the technical route. At present, there are many high-quality English datasets that are publicly available in the open source community, but few high-quality Chinese datasets. In order to verify the technical route first, we launched this English interface version. We have begun to construct some high-quality Chinese data by ourselves. Although it will take some time, we believe that MOSS will have good Chinese comprehension and generation capabilities in the future. Our goal is also to build a large-scale Chinese language model with Chinese characteristics.

What are the team's short-term goals and ultimate expectations for MOSS?

The short-term goal is to hope that MOSS will become a domestically advanced conversational language model. We hope to insist on being a research institution that is not driven by profit, share the research results with the academic community free of charge, and open the research results to the industry under the premise of legal compliance, letting them do customization or applications in special fields. If all goes well, we will make MOSS open source around the end of March.

In the long run, we look forward to using MOSS as a pedestal leading to artificial general intelligence. I am very optimistic that artificial general intelligence will not be very far from science fiction. It will become a reality maybe in five to 10 years. By that time, we will accept artificial general intelligence as we accept search engines now.

?