Access the full text.
Sign up today, get DeepDyve free for 14 days.
V Vovk (2015)
Cross-conformal predictorsAnn. Math. Artif. Intell., 74
L Breiman (2001)
Random forestsMach. Learn., 45
H Papadopoulos, H Haralambous (2011)
Reliable prediction intervals with regression neural networksNeural Netw., 24
U Johansson, H Boström, T Löfström, H Linusson (2014)
Regression conformal prediction with random forestsMach. Learn., 97
J Demšar (2006)
Statistical comparisons of classifiers over multiple data setsJ. Mach. Learn. Res., 7
H Papadopoulos (2008)
Inductive conformal prediction: Theory and application to neural networksTools in Artificial Intelligence, 18
H Papadopoulos, A Gammerman, V Vovk (2008)
In: Proceedings of the IASTED International Conference on Artificial Intelligence and Applications (AIA 2008), pp. 64–69
H Papadopoulos, V Vovk, A Gammerman (2011)
Regression conformal prediction with nearest neighboursJ. Artif. Intell. Res., 40
L Breiman (1996)
Bagging predictorsMach. Learn., 24
H Boström, H Linusson, T Löfström, U Johansson (2016)
In: Conformal and Probabilistic Prediction with Applications - 5th International Symposium, COPA 2016, Madrid, Spain, April 20-22, 2016, Proceedings, pp. 75–89
The conformal prediction framework allows for specifying the probability of making incorrect predictions by a user-provided confidence level. In addition to a learning algorithm, the framework requires a real-valued function, called nonconformity measure, to be specified. The nonconformity measure does not affect the error rate, but the resulting efficiency, i.e., the size of output prediction regions, may vary substantially. A recent large-scale empirical evaluation of conformal regression approaches showed that using random forests as the learning algorithm together with a nonconformity measure based on out-of-bag errors normalized using a nearest-neighbor-based difficulty estimate, resulted in state-of-the-art performance with respect to efficiency. However, the nearest-neighbor procedure incurs a significant computational cost. In this study, a more straightforward nonconformity measure is investigated, where the difficulty estimate employed for normalization is based on the variance of the predictions made by the trees in a forest. A large-scale empirical evaluation is presented, showing that both the nearest-neighbor-based and the variance-based measures significantly outperform a standard (non-normalized) nonconformity measure, while no significant difference in efficiency between the two normalized approaches is observed. The evaluation moreover shows that the computational cost of the variance-based measure is several orders of magnitude lower than when employing the nearest-neighbor-based nonconformity measure. The use of out-of-bag instances for calibration does, however, result in nonconformity scores that are distributed differently from those obtained from test instances, questioning the validity of the approach. An adjustment of the variance-based measure is presented, which is shown to be valid and also to have a significant positive effect on the efficiency. For conformal regression forests, the variance-based nonconformity measure is hence a computationally efficient and theoretically well-founded alternative to the nearest-neighbor procedure.
Annals of Mathematics and Artificial Intelligence – Springer Journals
Published: Mar 1, 2017
Read and print from thousands of top scholarly journals.
Already have an account? Log in
Bookmark this article. You can see your Bookmarks on your DeepDyve Library.
To save an article, log in first, or sign up for a DeepDyve account if you don’t already have one.
Copy and paste the desired citation format or use the link below to download a file formatted for EndNote
Access the full text.
Sign up today, get DeepDyve free for 14 days.
All DeepDyve websites use cookies to improve your online experience. They were placed on your computer when you launched this website. You can change your cookie settings through your browser.