Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Editorial

Editorial Editorial: Need for Artifact Verified Articles in ACM Transactions One of the complaints by many researchers in the field of design automation, and embedded sys- tems including myself, has been that the results in most papers are hard to reproduce. Often, we find papers with astounding performance numbers—speed, latency, power consumption, accuracy, and many other performance characteristics. Unfortunately, when one contacts the authors for the code, or the models, they are unavailable—because the graduate student working on it has gradu- ated by the time of publication and has not archived the artifacts properly, or because the system built has dependence on licensed software that may not be available to the one asking for the code, or may be the authors are unwilling to provide the artifacts. The result of this unavailability of research artifacts are multiple folds—when comparing new results against reported results in the literature, one must quote numbers from the papers that were obtained in very different circumstances, with different—possibility better—implementation of al- gorithms or worse implementations, or the dataset on which results were obtained are drastically of different characteristics, and so on. Thus, the comparison becomes between apples and oranges. In many cases, the authors may http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png ACM Transactions on Embedded Computing Systems (TECS) Association for Computing Machinery

Loading next page...
 
/lp/association-for-computing-machinery/editorial-AiX2NZf0Q2

References

References for this paper are not available at this time. We will be adding them shortly, thank you for your patience.

Publisher
Association for Computing Machinery
Copyright
Copyright © 2018 ACM
ISSN
1539-9087
eISSN
1558-3465
DOI
10.1145/3282437
Publisher site
See Article on Publisher Site

Abstract

Editorial: Need for Artifact Verified Articles in ACM Transactions One of the complaints by many researchers in the field of design automation, and embedded sys- tems including myself, has been that the results in most papers are hard to reproduce. Often, we find papers with astounding performance numbers—speed, latency, power consumption, accuracy, and many other performance characteristics. Unfortunately, when one contacts the authors for the code, or the models, they are unavailable—because the graduate student working on it has gradu- ated by the time of publication and has not archived the artifacts properly, or because the system built has dependence on licensed software that may not be available to the one asking for the code, or may be the authors are unwilling to provide the artifacts. The result of this unavailability of research artifacts are multiple folds—when comparing new results against reported results in the literature, one must quote numbers from the papers that were obtained in very different circumstances, with different—possibility better—implementation of al- gorithms or worse implementations, or the dataset on which results were obtained are drastically of different characteristics, and so on. Thus, the comparison becomes between apples and oranges. In many cases, the authors may

Journal

ACM Transactions on Embedded Computing Systems (TECS)Association for Computing Machinery

Published: Nov 19, 2018

There are no references for this article.