Extracting deltas from column oriented NoSQL databases for different incremental applications and diverse data targets

摘要

This paper describes the Change Data Capture (CDC) problems in the context of column-oriented NoSQL databases (CoNoSQLDBs). CDC is a term mostly used by ETL tools and data warehousing environments (DW) to depict a data processing of extracting data changes made at the data sources. Based on analyzing the impacts and constraints caused by the core features of CoNoSQLDBs, we propose a logical change data (delta) model and the corresponding delta representations which could work with different incremental applications and diverse data targets. Moreover, we present five feasible CDC approaches, i.e. Timestamp-based approach, Audit-column approach, Log-based approach, Trigger-based approach and Snapshot differential approach and indicate the performance winners under different circumstances.