Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

A Case for Enrichment in Data Management Systems

A Case for Enrichment in Data Management Systems We describe ENRICHDB, a new DBMS technology designed for emerging domains (e.g., sensor-driven smart spaces and social media analytics) that require incoming data to be enriched using expensive functions prior to its usage. To support online processing, today, such enrichment is performed outside of DBMSs, as a static data processing workflow prior to its ingestion into a DBMS. Such a strategy could result in a significant delay from the time when data arrives and when it is enriched and ingested into the DBMS, especially when the enrichment complexity is high. Also, enriching at ingestion could result in wastage of resources if applications do not use/require all data to be enriched. ENRICHDB's design represents a significant departure from the above, where we explore seamless integration of data enrichment all through the data processing pipeline - at ingestion, triggered based on events in the background, and progressively during query processing. The cornerstone of ENRICHDB is a powerful enrichment data and query model that encapsulates enrichment as an operator inside a DBMS enabling it to co-optimize enrichment with query processing. This paper describes this data model and provides a summary of the system implementation. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png ACM SIGMOD Record Association for Computing Machinery

A Case for Enrichment in Data Management Systems

Download PDF
6 pages

Loading next page...
 
/lp/association-for-computing-machinery/a-case-for-enrichment-in-data-management-systems-7rco5ODTXZ

References

References for this paper are not available at this time. We will be adding them shortly, thank you for your patience.

Publisher
Association for Computing Machinery
Copyright
Copyright © 2022 Copyright is held by the owner/author(s)
ISSN
0163-5808
DOI
10.1145/3552490.3552497
Publisher site
See Article on Publisher Site

Abstract

We describe ENRICHDB, a new DBMS technology designed for emerging domains (e.g., sensor-driven smart spaces and social media analytics) that require incoming data to be enriched using expensive functions prior to its usage. To support online processing, today, such enrichment is performed outside of DBMSs, as a static data processing workflow prior to its ingestion into a DBMS. Such a strategy could result in a significant delay from the time when data arrives and when it is enriched and ingested into the DBMS, especially when the enrichment complexity is high. Also, enriching at ingestion could result in wastage of resources if applications do not use/require all data to be enriched. ENRICHDB's design represents a significant departure from the above, where we explore seamless integration of data enrichment all through the data processing pipeline - at ingestion, triggered based on events in the background, and progressively during query processing. The cornerstone of ENRICHDB is a powerful enrichment data and query model that encapsulates enrichment as an operator inside a DBMS enabling it to co-optimize enrichment with query processing. This paper describes this data model and provides a summary of the system implementation.

Journal

ACM SIGMOD Record – Association for Computing Machinery

Published: Jul 29, 2022

References

  • https://airflow
    airflow, Apache
  • https://github
    paper, Full; code,
  • github
    postgresql, IMV implementation of
  • Progressive approach to relational entity resolution
    Altowim, Y.
  • An empirical comparison of supervised learning algorithms
    Caruana, R.; Niculescu-Mizil, A.
  • Bayes optimal multilabel classification via probabilistic classifier chains
    Cheng, W.
  • Uncertainty annotated databases - A lightweight approach for approximating certain answers
    Feng, S.
  • Cleaning denial constraint violations through relaxation
    Giannakopoulou, S.
  • The madlib analytics library orMAD skills, the SQL
    Hellerstein, J.M.
  • Online aggregation
    Hellerstein, J.M.
  • Top-k multiclass SVM
    Lapin, M.
  • LOCATER: cleaning wifi connectivity datasets for semantic localization
    Lin, Y.
  • Progressive duplicate detection
    Papenbrock, T.
  • The impact of columnar in-memory databases on enterprise systems
    Plattner, H.
  • Probabilistic databases
    Suciu, D.
  • Least squares support vector machine classifiers
    Suykens, J. A. K.
  • An IDEA: an ingestion framework for data enrichment in asterixdb
    X.Wang,
  • Pay-as-you-go entity resolution
    E.Whang, S.
  • Stacked generalization
    H.Wolpert, D.
  • Discretized streams: fault-tolerant streaming computation at scale
    Zaharia, M.
  • I/o-efficient statistical computing with RIOT
    Zhang, Y.