Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

A Low-cost Disk Solution Enabling LSM-tree to Achieve High Performance for Mixed Read/Write Workloads

A Low-cost Disk Solution Enabling LSM-tree to Achieve High Performance for Mixed Read/Write... LSM-tree has been widely used in data management production systems for write-intensive workloads. However, as read and write workloads co-exist under LSM-tree, data accesses can experience long latency and low throughput due to the interferences to buffer caching from the compaction, a major and frequent operation in LSM-tree. After a compaction, the existing data blocks are reorganized and written to other locations on disks. As a result, the related data blocks that have been loaded in the buffer cache are invalidated since their referencing addresses are changed, causing serious performance degradations. To re-enable high-speed buffer caching during intensive writes, we propose Log-Structured buffered-Merge tree (simplified as LSbM-tree) by adding a compaction buffer on disks to minimize the cache invalidations on buffer cache caused by compactions. The compaction buffer efficiently and adaptively maintains the frequently visited datasets. In LSbM, strong locality objects can be effectively kept in the buffer cache with minimum or no harmful invalidations. With the help of a small on-disk compaction buffer, LSbM achieves a high query performance by enabling effective buffer caching, while retaining all the merits of LSM-tree for write-intensive data processing and providing high bandwidth of disks for range queries. We have implemented LSbM based on LevelDB. We show that with a standard buffer cache and a hard disk, LSbM can achieve 2x performance improvement over LevelDB. We have also compared LSbM with other existing solutions to show its strong cache effectiveness. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png ACM Transactions on Storage (TOS) Association for Computing Machinery

A Low-cost Disk Solution Enabling LSM-tree to Achieve High Performance for Mixed Read/Write Workloads

Download PDF
26 pages

Loading next page...
 
/lp/association-for-computing-machinery/a-low-cost-disk-solution-enabling-lsm-tree-to-achieve-high-performance-pfB3R7To5l

References

References for this paper are not available at this time. We will be adding them shortly, thank you for your patience.

Publisher
Association for Computing Machinery
Copyright
Copyright © 2018 ACM
ISSN
1553-3077
eISSN
1553-3093
DOI
10.1145/3162615
Publisher site
See Article on Publisher Site

Abstract

LSM-tree has been widely used in data management production systems for write-intensive workloads. However, as read and write workloads co-exist under LSM-tree, data accesses can experience long latency and low throughput due to the interferences to buffer caching from the compaction, a major and frequent operation in LSM-tree. After a compaction, the existing data blocks are reorganized and written to other locations on disks. As a result, the related data blocks that have been loaded in the buffer cache are invalidated since their referencing addresses are changed, causing serious performance degradations. To re-enable high-speed buffer caching during intensive writes, we propose Log-Structured buffered-Merge tree (simplified as LSbM-tree) by adding a compaction buffer on disks to minimize the cache invalidations on buffer cache caused by compactions. The compaction buffer efficiently and adaptively maintains the frequently visited datasets. In LSbM, strong locality objects can be effectively kept in the buffer cache with minimum or no harmful invalidations. With the help of a small on-disk compaction buffer, LSbM achieves a high query performance by enabling effective buffer caching, while retaining all the merits of LSM-tree for write-intensive data processing and providing high bandwidth of disks for range queries. We have implemented LSbM based on LevelDB. We show that with a standard buffer cache and a hard disk, LSbM can achieve 2x performance improvement over LevelDB. We have also compared LSbM with other existing solutions to show its strong cache effectiveness.

Journal

ACM Transactions on Storage (TOS) – Association for Computing Machinery

Published: Apr 12, 2018

Keywords: LSM-tree

References

  • Compaction management in distributed key-value datastores
    Ahmad, Muhammad Yousuf; Kemme, Bettina
  • Cassandra
    Apache,
  • HBASE
    Apache,
  • Workload analysis of a large-scale key-value store
    Atikoglu, Berk; Xu, Yuehai; Frachtenberg, Eitan; Jiang, Song; Paleczny, Mike
  • Riak
    Basho,
  • Network applications of bloom filters: A survey
    Broder, Andrei; Mitzenmacher, Michael
  • Bigtable: A distributed storage system for structured data
    Chang, Fay; Dean, Jeffrey; Ghemawat, Sanjay; Hsieh, Wilson C.; Wallach, Deborah A.; Burrows, Mike; Chandra, Tushar; Fikes, Andrew; Gruber, Robert E.
  • Essential roles of exploiting internal parallelism of flash memory based solid state drives in high-speed data processing
    Chen, Feng; Lee, Rubao; Zhang, Xiaodong
  • Benchmarking cloud serving systems with YCSB
    Cooper, Brian F.; Silberstein, Adam; Tam, Erwin; Ramakrishnan, Raghu; Sears, Russell
  • RocksDB
    Facebook,
  • Scaling concurrent log-structured data stores
    Golan-Gueta, Guy; Bortnikov, Edward; Hillel, Eshcar; Keidar, Idit
  • LevelDB
    Google,
  • Re-enabling high-speed caching for LSM-trees
    Guo, Lei; Teng, Dejun; Lee, Rubao; Chen, Feng; Ma, Siyuan; Zhang, Xiaodong
  • Incremental organization for data recording and warehousing
    Jagadish, H. V.; Narayan, P. P. S.; Seshadri, Sridhar; Sudarshan, S.; Kanneganti, Rama
  • Tree indexing on solid state drives
    Li, Yinan; He, Bingsheng; Yang, Robin Jun; Luo, Qiong; Yi, Ke
  • Towards accurate and fast evaluation of multi-stage log-structured designs
    Lim, Hyeontaek; Andersen, David G.; Kaminsky, Michael
  • WiscKey: Separating keys from values in SSD-conscious storage
    Lu, Lanyue; Pillai, Thanumalayan Sankaranarayana; Arpaci-Dusseau, Andrea C.; Arpaci-Dusseau, Remzi H.
  • Compaction Improvements in Cassandra 2
    Mcguire, Ryan
  • The log-structured merge-tree (LSM-tree)
    ONeil, Patrick; Cheng, Edward; Gawlick, Dieter; ONeil, Elizabeth
  • bLSM: A general purpose log structured merge tree
    Sears, Russell; Ramakrishnan, Raghu
  • Building workload-independent storage with VT-trees
    Shetty, Pradeep J.; Spillane, Richard P.; Malpani, Ravikant R.; Andrews, Binesh; Seyster, Justin; Zadok, Erez
  • LSbM-tree: Re-enabling buffer caching in data management for mixed reads and writes
    Teng, Dejun; Guo, Lei; Lee, Rubao; Chen, Feng; Ma, Siyuan; Zhang, Yanfeng; Zhang, Xiaodong
  • An efficient design and implementation of LSM-tree based key-value store on open-channel SSD
    Wang, Peng; Sun, Guangyu; Jiang, Song; Ouyang, Jian; Lin, Shiding; Zhang, Chen; Cong, Jason
  • LSM-trie: An LSM-tree-based ultra-large key-value store for small data
    Wu, Xingbo; Xu, Yuehai; Shao, Zili; Jiang, Song
  • Mega-kv: A case for GPUS to maximize the throughput of in-memory key-value stores
    Zhang, Kai; Wang, Kaibo; Yuan, Yuan; Guo, Lei; Lee, Rubao; Zhang, Xiaodong