Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

On the Lifecycle of the File

On the Lifecycle of the File Users and Operating Systems (OSs) have vastly different views of files. OSs use files to persist data and structured information. To accomplish this, OSs treat files as named collections of bytes managed in hierarchical file systems. Despite their critical role in computing, little attention is paid to the lifecycle of the file, the evolution of file contents, or the evolution of file metadata. In contrast, users have rich mental models of files: they group files into projects, send data repositories to others, work on documents over time, and stash them aside for future use. Current OSs and Revision Control Systems ignore such mental models, persisting a selective, manually designated history of revisions. Preserving the mental model allows applications to better match how users view their files, making file processing and archiving tools more effective. We propose two mechanisms that OSs can adopt to better preserve the mental model: File Lifecycle Events (FLEs) that record a file’s progression and Complex File Events (CFEs) that combine them into meaningful patterns. We present the Complex File Events Engine (CoFEE), which uses file system monitoring and an extensible rulebase (Drools) to detect FLEs and convert them into complex ones. CFEs are persisted in NoSQL stores for later querying. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png ACM Transactions on Storage (TOS) Association for Computing Machinery

On the Lifecycle of the File

Download PDF
45 pages

Loading next page...
 
/lp/association-for-computing-machinery/on-the-lifecycle-of-the-file-htrdSO67o8
Publisher
Association for Computing Machinery
Copyright
Copyright © 2019 ACM
ISSN
1553-3077
eISSN
1553-3093
DOI
10.1145/3295463
Publisher site
See Article on Publisher Site

Abstract

Users and Operating Systems (OSs) have vastly different views of files. OSs use files to persist data and structured information. To accomplish this, OSs treat files as named collections of bytes managed in hierarchical file systems. Despite their critical role in computing, little attention is paid to the lifecycle of the file, the evolution of file contents, or the evolution of file metadata. In contrast, users have rich mental models of files: they group files into projects, send data repositories to others, work on documents over time, and stash them aside for future use. Current OSs and Revision Control Systems ignore such mental models, persisting a selective, manually designated history of revisions. Preserving the mental model allows applications to better match how users view their files, making file processing and archiving tools more effective. We propose two mechanisms that OSs can adopt to better preserve the mental model: File Lifecycle Events (FLEs) that record a file’s progression and Complex File Events (CFEs) that combine them into meaningful patterns. We present the Complex File Events Engine (CoFEE), which uses file system monitoring and an extensible rulebase (Drools) to detect FLEs and convert them into complex ones. CFEs are persisted in NoSQL stores for later querying.

Journal

ACM Transactions on Storage (TOS) – Association for Computing Machinery

Published: Feb 18, 2019

Keywords: Files

References

  • Generating realistic impressions for file-system benchmarking
    Agrawal, Nitin; Arpaci-Dusseau, Andrea C.; Arpaci-Dusseau, Remzi H.
  • A five-year study of file-system metadata
    Agrawal, Nitin; Bolosky, William J.; Douceur, John R.; Lorch, Jacob R.
  • Understanding windows file system transactions
    Allred, Christian
  • Putting lipstick on pig: Enabling database-style workflow provenance
    Amsterdamer, Yael; Davidson, Susan B.; Deutch, Daniel; Milo, Tova; Stoyanovich, Julia; Tannen, Val
  • Apache Subversion
    Apache,
  • Back Up Your Mac With Time Machine
    Apple, ; Inc,
  • Amazon Simple Storage Service Developer Guide (API Version 2006-03-01): Using Versioning
    AWS.,
  • Anti-ransomware
    Bhashkar, Aishwary; Kulkarni, Abhijit P.; Jagdale, Prakash D.
  • Evaluation of filesystem provenance visualization tools
    Borkin, M. A.; Yeh, C. S.; Boyd, M.; Macko, P.; Gajos, K. Z.; Seltzer, M.; Pfister, H.
  • Btrfs Wiki
    Btrfs,
  • It’s about the data: Provenance as a tool for assessing data fitness
    Chapman, Adriane; Allen, M. David; Blaustein, Barbara
  • Nonlinear revision control for images
    Chen, Hsiang-Ting; Wei, Li-Yi; Chang, Chun-Fa
  • Shared information and program plagiarism detection
    Chen, X.; Francia, B.; Li, M.; McKinnon, B.; Seker, A.
  • Version models for software configuration management
    Conradi, Reidar; Westfechtel, Bernhard
  • ShieldFS: A self-healing, ransomware-aware filesystem
    Continella, Andrea; Guagnelli, Alessandro; Zingaro, Giovanni; De Pasquale, Giulio; Barenghi, Alessandro; Zanero, Stefano
  • Wayback: A user-level versioning file system for Linux
    Cornell, Brian; Dinda, Peter A.; Bustamante, Fabián E.
  • TOPS-20 User’s Guide
    Corporation, Digital Equipment
  • Building a cloud file system
    Darcy, Jeff
  • Provenance in DISC systems: Reducing space overhead at runtime
    Diestelkämper, Ralf; Herschel, Melanie; Jadhav, Priyanka
  • Cardinal: Novel software for studying file management behavior
    Dinneen, Jesse David; Odoni, Fabian; Frissen, Ilja; Julien, Charles-Antoine
  • A large-scale study of file-system contents
    Douceur, John R.; Bolosky, William J.
  • Auditable versioned data storage outsourcing
    Esiner, Ertem; Datta, Anwitaman
  • Finding similar files in large document repositories
    Forman, George; Eshghi, Kave; Chiocchetti, Stephane
  • Scaling SPADE to ”Big Provenance
    Gehani, Ashish; Kazmi, Hasanat; Irshad, Hassaan
  • GNU Make
    GNU.,
  • Google Drive—Cloud Storage 8 File Backup for Photos, Docs 8 More
    Google,
  • A file is not a file: Understanding the I/O behavior of apple desktop applications
    Harter, Tyler; Dragga, Chris; Vaughn, Michael; Arpaci-Dusseau, Andrea C.; Arpaci-Dusseau, Remzi H.
  • File System Design for an NFS File Server Appliance
    Hitz, Dave; Lau, James; Malcolm, Michael
  • BioLite, a lightweight bioinformatics framework with automated tracking of diagnostics and provenance
    Howison, Mark; Sinnott-Armstrong, Nicholas A.; Dunn, Casey W.
  • BVSSD: Build built-in versioning flash-based solid state drives
    Huang, Ping; Zhou, Ke; Wang, Hua; Li, Chun Hua
  • MarFS, a near-POSIX interface to cloud objects
    Inman, Jeff; Vining, Will; Ransom, Garrett; Grider, Gary
  • Drools—Business Rules Management System
    Community, JBoss
  • Provenance query patterns for many-task scientific computing
    Jr, Luiz M. R. Gadelha
  • Efficient data deduplication system considering file modification pattern
    Jung, Ho Min; Park, Sang Yong; Lee, Jeong Gun; Ko, Young Woong
  • Using file relationships in malware classification
    Karampatziakis, Nikos; Stokes, Jack W.; Thomas, Anil; Marinescu, Mady
  • Ontology-assisted provenance visualization for supporting enterprise search of engineering and business files
    Khan, Saiful; Kanturska, Urszula; Waters, Tom; Eaton, James; Bañares-Alcántara, Renè
  • Versioning version trees: The provenance of actions that affect multiple versions
    Koop, David
  • Deleting, Moving and Renaming
    Küng, Stefan; Onken, Lübbe; Large, Simon
  • Architectures for controller based CDP
    Laden, Guy; Ta-Shma, Paula; Yaffe, Eitan; Factor, Michael; Fienblit, Shachar
  • Provenance-based recommendations for visual data exploration
    Lahmar, Houssem Ben; Herschel, Melanie
  • Scientific user behavior and data-sharing trends in a petascale file system
    Lim, Seung-Hwan; Sim, Hyogi; Gunasekaran, Raghul; Vazhkudai, Sudharshan S.
  • A hybrid algorithm of backward hashing and automaton tracking for virus scanning
    Lin, Po-Ching; Lin, Ying-Dar; Lai, Yuan-Cheng
  • A search strategy for large document bases
    Lucarella, D.
  • Provenance map orbiter: Interactive exploration of large provenance graphs
    Macko, Peter; Seltzer, Margo
  • A general-purpose provenance library
    Macko, Peter; Seltzer, Margo
  • Finding similar files in a large file system
    Manber, Udi
  • A fast file system for UNIX
    McKusick, Marshall K.; Joy, William N.; Leffler, Samuel J.; Fabry, Robert S.
  • Description of How Word Creates Temporary Files
    Microsoft,
  • Cross-checking semantic correctness: The case of finding file system bugs
    Min, Changwoo; Kashyap, Sanidhya; Lee, Byoungyoung; Song, Chengyu; Kim, Taesoo
  • PROV-DM: The PROV Data Model
    Moreau, Luc; (Eds, Paolo Missier
  • Provenance-aware storage systems
    Muniswamy-Reddy, Kiran-Kumar; Holland, David A.; Braun, Uri; Seltzer, Margo
  • A versatile and user-oriented versioning file system
    Muniswamy-Reddy, Kiran-Kumar; Wright, Charles P.; Himmer, Andrew; Zadok, Erez
  • Bolt Protocol
    Technology, Neo; Inc,
  • Cypher Query Language Reference
    Neo,
  • Neo4j Database
    Neo,
  • JPathwatch: A Java Library for Monitoring Directory and File Changes
    Pachler, Uwe
  • A survey and classification of storage deduplication systems
    Paulo, João; Pereira, José
  • Ext3cow: A time-shifting file system for regulatory compliance
    Peterson, Zachary N. J.; Burns, Randal C.
  • Git-log: Show Commit Logs (version 2
    Project, Git
  • A comparison of file system workloads
    Roselli, Drew; Lorch, Jacob R.; Anderson, Thomas E.
  • Hashing and data fingerprinting in digital forensics
    Roussev, V.
  • Elephant: The file system that never forgets
    Santry, D. J.; Feeley, M. J.; Hutchinson, N. C.; Veitch, A. C.
  • Visualizing provenance using comics
    Schreiber, Andreas; Struminski, Regina
  • Hierarchical file systems are dead
    Seltzer, Margo; Murphy, Nicholas
  • File system aging—Increasing the relevance of file system benchmarks
    Smith, Keith A.; Seltzer, Margo I.
  • Metadata efficiency in versioning file systems
    Soules, Craig A. N.; Goodson, Garth R.; Strunk, John D.; Ganger, Gregory R.
  • Story book: An efficient extensible provenance framework
    Spillane, R.; Sears, R.; Yalamanchili, C.; Gaikwad, S.; Chinni, M.; Zadok, E.
  • A file provenance system
    Sultana, Salmin; Bertino, Elisa
  • Re: Merge with git-pasky II
    Torvalds, Linus
  • A nine year study of file system and storage benchmarking
    Traeger, Avishay; Zadok, Erez; Joukov, Nikolai; Wright, Charles P.
  • Efficient signature based malware detection on mobile devices
    Venugopal, Deepak; Hu, Guoning
  • Compressing provenance graphs
    Xie, Yulai; Muniswamy-Reddy, Kiran-Kumar; Long, Darrell D. E.; Amer, Ahmed; Feng, Dan; Tan, Zhipeng
  • JNotify: File system events library for Java
    Yadan, Omry