Maintenance of Shanghai Actual Population Data Based on Data Warehouse

Maintenance of Shanghai Actual Population Data Based on Data Warehouse

연구 분야: Databases

학회: 2022 3rd International Conference on Big Data, Artificial Intelligence and Internet of Things Engineering (ICBAIE)

At present, there are problems of untimely and inaccurate data updating in the actual population database in Shanghai. The specific manifestation is that there are a large number of people in Shanghai but not registered (referred to as missing registration) and people who have left Shanghai but the information has not been canceled from the actual population database (referred to as not canceled). In order to solve the above problems, this paper proposes to build a data warehouse based on the continuously updated population big data and the existing actual population database to update the actual population database in time. Firstly, the paper use Kettle to process the acquired data source and load the suspicious person data (suspected unregistered or suspected uncanceled) from the data source into the data warehouse by comparing it with the registered data in the actual population database. Then, a data fusion algorithm is proposed to fuse and analyze the data in the data warehouse, and analyze the high-suspected unregistered personnel and the high-suspected unregistered personnel to update the Shanghai actual population database in time. This provides support for maintaining accurate and real-time population databases and solves the problem of inaccurate data in the real population database to a certain extent.