PostMan: A Productive System for Spatio-temporal Data Management and Analysis

PostMan: A Productive System for Spatio-temporal Data Management and Analysis

연구 분야: Databases

학회: Data Science and Engineering

In daily life, there is an increasing demand for efficient management and analysis of spatio-temporal data. However, current systems struggle to balance multi-functionality, scalability, and computational efficiency in this domain. To address this challenge, we introduce PostMan: a productive spatio-temporal data management system. PostMan is based on Apache Spark and Apache Hadoop HDFS. It extensively, efficiently, and scalably supports spatio-temporal data types and operators across multiple API levels. To realize effective data management and analysis, PostMan designs the unified partition management and hybrid index. Based on this, PostMan has designed and implemented a variety of optimization strategies for vector and raster operators. PostMan also introduces a two-phase static partitioning (TPSP) method to maintain load balance before and after partition filtering during the query process. In the first phase, partitions are generated using an enhanced R*-Tree algorithm, while the second phase allocates partitions by modeling the task as an optimization problem solved through greedy algorithms. For faster computation, PostMan introduces processes and program interfaces for GPU accelerated spatio-temporal operators in Spark. Moreover, extensive evaluations using real-world datasets show PostMan’s notable efficiency and scalability advantages (e.g., 13%-36% improvement) over baseline systems, as well as their constituent techniques. Finally, PostMan has been deployed on the public cloud in a Software as a Service (SaaS) model, garnering substantial attention from customers.