Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Countering Fragmentation in an Enterprise Storage System

Countering Fragmentation in an Enterprise Storage System As a file system ages, it can experience multiple forms of fragmentation. Fragmentation of the free space in the file system can lower write performance and subsequent read performance. Client operations as well as internal operations, such as deduplication, can fragment the layout of an individual file, which also impacts file read performance. File systems that allow sub-block granular addressing can gather intra-block fragmentation, which leads to wasted free space. Similarly, wasted space can also occur when a file system writes a collection of blocks out to object storage as a single large object, because the constituent blocks can become free at different times. The impact of fragmentation also depends on the underlying storage media. This article studies each form of fragmentation in the NetApp® WAFL®file system, and explains how the file system leverages a storage virtualization layer for defragmentation techniques that physically relocate blocks efficiently, including those in read-only snapshots. The article analyzes the effectiveness of these techniques at reducing fragmentation and improving overall performance across various storage media. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png ACM Transactions on Storage (TOS) Association for Computing Machinery

Countering Fragmentation in an Enterprise Storage System

Download PDF
35 pages

Loading next page...
 
/lp/association-for-computing-machinery/countering-fragmentation-in-an-enterprise-storage-system-UOMBKTHkO0

References

References for this paper are not available at this time. We will be adding them shortly, thank you for your patience.

Publisher
Association for Computing Machinery
Copyright
Copyright © 2020 ACM
ISSN
1553-3077
eISSN
1553-3093
DOI
10.1145/3366173
Publisher site
See Article on Publisher Site

Abstract

As a file system ages, it can experience multiple forms of fragmentation. Fragmentation of the free space in the file system can lower write performance and subsequent read performance. Client operations as well as internal operations, such as deduplication, can fragment the layout of an individual file, which also impacts file read performance. File systems that allow sub-block granular addressing can gather intra-block fragmentation, which leads to wasted free space. Similarly, wasted space can also occur when a file system writes a collection of blocks out to object storage as a single large object, because the constituent blocks can become free at different times. The impact of fragmentation also depends on the underlying storage media. This article studies each form of fragmentation in the NetApp® WAFL®file system, and explains how the file system leverages a storage virtualization layer for defragmentation techniques that physically relocate blocks efficiently, including those in read-only snapshots. The article analyzes the effectiveness of these techniques at reducing fragmentation and improving overall performance across various storage media.

Journal

ACM Transactions on Storage (TOS) – Association for Computing Machinery

Published: Jan 16, 2020

Keywords: Storage system

References

  • DFS: A de-fragmented file system
    Ahn, Woo Hyun; Kim, Kyungbaek; Choi, Yongjin; Park, Daeyeon
  • Commercial fault tolerance: A tale of two systems
    Bartlett, Wendy; Spainhower, Lisa
  • LightNVM: The Linux open-channel SSD subsystem
    Bjørling, Matias; González, Javier; Bonnet, Philippe
  • File systems fated for senescence? Nonsense, says science! In Proceedings of the Conference on File and Storage Technologies (FAST’17)
    Conway, Alexander; Bakshi, Ainesh; Jiao, Yizheng; Jannen, William; Zhan, Yang; Yuan, Jun; Bender, Michael A.; Johnson, Rob; Kuszmaul, Bradley C.; Porter, Donald E.
  • A Paging Experiment with the Multics System
    Corbato, Fernando J.
  • Row-diagonal parity for double disk failure correction
    Corbett, Peter; English, Bob; Goel, Atul; Grcanac, Tomislav; Kleiman, Steven; Leong, James; Sankar, Sunitha
  • To waffinity and beyond: A scalable architecture for incremental parallelization of file system code
    Curtis-Maury, Matthew; Devadas, Vinay; Fang, Vania; Kulkarni, Aditya
  • Scalable write allocation in the WAFL file system
    Curtis-Maury, Matthew; Kesavan, Ram; Bhattacharjee, Mrinal
  • FlexVol: Flexible, efficient file volume virtualization in WAFL
    Edwards, John K.; Ellard, Daniel; Everhart, Craig; Fair, Robert; Hamilton, Eric; Kahn, Andy; Kanevsky, Arkady; Lentini, James; Prakash, Ashish; Smith, Keith A.
  • RAID triple parity
    Goel, Atul; Corbett, Peter
  • Improving file system performance of mobile storage systems using a decoupled defragmenter
    Hahn, Sangwook Shane; Lee, Sungjin; Ji, Cheng; Chang, Li-Pin; Yee, Inhyuk; Shi, Liang; Xue, Chun Jason; Kim, Jihong
  • The unwritten contract of solid state drives
    He, Jun; Kannan, Sudarsun; Arpaci-Dusseau, Andrea C.; Arpaci-Dusseau, Remzi H.
  • SMaRT: An approach to shingled magnetic recording translation
    He, Weiping; Du, David H. C.
  • File system design for an NFS file server appliance
    Hitz, Dave; Lau, James; Malcolm, Michael
  • An empirical study of file-system fragmentation in mobile storage systems
    Ji, Cheng; Chang, Li-Pin; Shi, Liang; Wu, Chao; Li, Qiao; Xue, Chun Jason
  • Geriatrix: Aging what you see and what you don’t see
    Kadekodi, Saurabh; Nagarajan, Vaishnavh; Ganger, Gregory R.; Gibson, Garth A.
  • Efficient search for free blocks in the WAFL file system
    Kesavan, Ram; Curtis-Maury, Matthew; Bhattacharjee, Mrinal
  • Storage gardening: Using a virtualization layer for efficient defragmentation in the WAFL file system
    Kesavan, Ram; Curtis-Maury, Matthew; Devadas, Vinay; Mishra, Kesari
  • Algorithms and data structures for efficient free space reclamation in WAFL
    Kesavan, Ram; Singh, Rohit; Grusecki, Travis; Patel, Yuvraj
  • Efficient free space reclamation in WAFL
    Kesavan, Ram; Singh, Rohit; Grusecki, Travis; Patel, Yuvraj
  • SHRD: Improving spatial locality in flash storage accesses by sequentializing in host and randomizing in device
    Kim, Hyukjoong; Shin, Dongkun; Jeong, Yunho; Kim, Kyung Ho
  • Enable Cloud-Connected All-Flash Arrays with ONTAP 9
    Lantz, John
  • F2FS: A new file system for flash storage
    Lee, Changman; Sim, Dongho; Hwang, Joo Young; Cho, Sangyeun
  • ReSSD: A software layer for resuscitating SSDs from poor small random write performance
    Lee, Youngjae; Kim, Jin-Soo; Maeng, Seungryoul
  • ReconFS: A reconstructable file system on flash storage
    Lu, Youyou; Shu, Jiwu; Wang, Wei
  • The new ext4 filesystem: Current status and future plans
    Mathur, Avantika; Cao, Mingming; Bhattacharya, Suparna; Dilger, Andreas; Tomas, Alex; Vivier, Laurent
  • A fast file system for UNIX
    McKusick, Marshall K.; Joy, William N.; Leffler, Samuel J.; Fabry, Robert S.
  • A study of practical deduplication
    Meyer, Dutch T.; Bolosky, William J.
  • ZFS at OpenSolaris Community
    Microsystems, SUN
  • A survey of software techniques for using non-volatile memories for storage and main memory systems
    Mittal, Sparsh; Vetter, Jeffrey S.
  • Volume Move Express Guide
    NetApp, ; Inc,
  • A case for redundant arrays of inexpensive disks (RAID)
    Patterson, David A.; Gibson, Garth; Katz, Randy H.
  • SnapMirror: File system based asynchronous mirroring for disaster recovery
    Patterson, Hugo; Manley, Stephen; Federwisch, Mike; Hitz, Dave; Kleiman, Steve; Owara, Shane
  • BTRFS: The Linux B-tree filesystem
    Rodeh, Ohad; Bacik, Josef; Mason, Chris
  • The design and implementation of a log-structured file system
    Rosenblum, Mendel; Ousterhout, John K.
  • ext4 online defragmentation
    Sato, Takashi
  • Jacqueline Chang, Sara McMains, and Venkata Padmanabhan
    Seltzer, Margo; Smith, Keith A.; Balakrishnan, Hari
  • File system aging-Increasing the relevance of file system benchmarks
    Smith, Keith A.; Seltzer, Margo I.
  • Storage Performance Council-1 Benchmark
    Council, Storage Performance
  • The Private Lives of Disk Drives
    Sundaram, Rajesh
  • SCSI Storage Interfaces
    T1, Technical Committee
  • TRIM Specification
    T1, Technical Committee
  • Ceph: A scalable, high-performance distributed file system
    Weil, Sage A.; Brandt, Scott A.; Miller, Ethan L.; Long, Darrell D. E.; Maltzhan, Carlos
  • ReiserFS
    Wikipedia,
  • NOVA: A log-structured file system for hybrid volatile/non-volatile main memories
    Xu, Jian; Swanson, Steven
  • Performance characterization of hyperscale applications on NVMe SSDs
    Xu, Qiumin; Siyamwala, Huzefa; Ghosh, Mrinmoy; Awasthi, Manu; Suri, Tameesh; Guz, Zvika; Shayesteh, Anahita; Balakrishnan, Vijay
  • Performance analysis of NVMe SSDs and their implication on real world databases
    Xu, Qiumin; Siyamwala, Huzefa; Ghosh, Mrinmoy; Suri, Tameesh; Awasthi, Manu; Guz, Zvika; Shayesteh, Anahita; Balakrishnan, Vijay
  • Don’t stack your log on my log
    Yang, Jingpei; Plasson, Ned; Gillis, Greg; Talagala, Nisha; Sundararaman, Swaminathan
  • yFS: A journaling file system design for handling large data sets with reduced seeking
    Zhang, Zhihui; Ghose, Kanad
  • LSDM: Improving the performance of mobile storage with a log-structured address remapping device driver
    Zuck, Aviad; Kishon, Oren; Toledo, Sivan