Modern Experiment Management Systems Architecture for Scientific Big Data

Modern Experiment Management Systems Architecture for Scientific Big Data

연구 분야: Databases

학회: International Conference on Software Testing, Machine Learning and Complex Process Analysis

Experiment management systems (EMS) have more than twenty-five years of history. From small desktop prototypes to large-scale distributed systems, they become more and more complicated. The new chapter of EMS was uncovered with the age of Big Data surrounded by a special ecosystem to extract, analyze and store data. The big data ecosystem considers new elements that must be taken into account to expand the functionality for EMS to support all data lifecycle stages. One of the challenges is to highlight the key points of a huge variety of EMS evolving through time. Such systems do not usually follow a unified pattern because of special needs for each project. This paper introduces the conceptual high-level architecture as an example of a unified pattern of building EMS for big data ecosystems. The architecture does not consider to be used with the grid computing approach.