Data Generation, Testing and Evaluation of Chinese Natural Language Processing in the Cloud

Data Generation, Testing and Evaluation of Chinese Natural Language Processing in the Cloud

연구 분야: Artificial Intelligence

학회: 2022 IEEE 7th International Conference on Smart Cloud (SmartCloud)

With the rapid development of artificial intelligence, natural language processing, as an important branch, has also become a hot research field. A series of super large-scale pre-trained models represented by BERT and GPT have made great progress in natural language understanding and natural language generation, even some of the experimental accuracy exceed the human benchmark. However, these models will also make some mistakes and even fairness problems when they have the language ability equivalent to human beings. In order to verify whether the models can truly understand natural language, the evaluation of these models is particularly important. More methods are needed to evaluate the model. The language model-based evaluation tools often require a lot of computing resources. In this paper, we propose a method for testing and evaluation of Chinese natural language processing in cloud, generate testing data and design tests for Chinese data and test two pre-trained models. The experimental results show that our method can find defects of the model, though it has high performance on specific dataset.