Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Single- and Multi-FPGA Acceleration of Dense Stereo Vision for Planetary Rovers

Single- and Multi-FPGA Acceleration of Dense Stereo Vision for Planetary Rovers Increased mobile autonomy is a vital requisite for future planetary exploration rovers. Stereo vision is a key enabling technology in this regard, as it can passively reconstruct in three dimensions the surroundings of a rover and facilitate the selection of science targets and the planning of safe routes. Nonetheless, accurate dense stereo algorithms are computationally demanding. When executed on the low-performance, radiation-hardened CPUs typically installed on rovers, slow stereo processing severely limits the driving speed and hence the science that can be conducted in situ. Aiming to decrease execution time while increasing the accuracy of stereo vision embedded in future rovers, this article proposes HW/SW co-design and acceleration on resource-constrained, space-grade FPGAs. In a top-down approach, we develop a stereo algorithm based on the space sweep paradigm, design its parallel HW architecture, implement it with VHDL, and demonstrate feasible solutions even on small-sized devices with our multi-FPGA partitioning methodology. To meet all cost, accuracy, and speed requirements set by the European Space Agency for this system, we customize our HW/SW co-processor by design space exploration and testing on a Mars-like dataset. Implemented on Xilinx Virtex technology, or European NG-MEDIUM devices, the FPGA kernel processes a 1,120 1,120 stereo pair in 1.7s3.1s, utilizing only 5.49.3 LUT6 and 200312 RAMB18. The proposed system exhibits up to 32 speedup over desktop CPUs, or 2,810 over space-grade LEON3, and achieves a mean reconstruction error less than 2cm up to 4m depth. Excluding errors exceeding 2cm (which are less than 4% of the total), the mean error is under 8mm. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png ACM Transactions on Embedded Computing Systems (TECS) Association for Computing Machinery

Single- and Multi-FPGA Acceleration of Dense Stereo Vision for Planetary Rovers

Download PDF
27 pages

Loading next page...
 
/lp/association-for-computing-machinery/single-and-multi-fpga-acceleration-of-dense-stereo-vision-for-P1Mq30HHeu
Publisher
Association for Computing Machinery
Copyright
Copyright © 2019 ACM
ISSN
1539-9087
eISSN
1558-3465
DOI
10.1145/3312743
Publisher site
See Article on Publisher Site

Abstract

Increased mobile autonomy is a vital requisite for future planetary exploration rovers. Stereo vision is a key enabling technology in this regard, as it can passively reconstruct in three dimensions the surroundings of a rover and facilitate the selection of science targets and the planning of safe routes. Nonetheless, accurate dense stereo algorithms are computationally demanding. When executed on the low-performance, radiation-hardened CPUs typically installed on rovers, slow stereo processing severely limits the driving speed and hence the science that can be conducted in situ. Aiming to decrease execution time while increasing the accuracy of stereo vision embedded in future rovers, this article proposes HW/SW co-design and acceleration on resource-constrained, space-grade FPGAs. In a top-down approach, we develop a stereo algorithm based on the space sweep paradigm, design its parallel HW architecture, implement it with VHDL, and demonstrate feasible solutions even on small-sized devices with our multi-FPGA partitioning methodology. To meet all cost, accuracy, and speed requirements set by the European Space Agency for this system, we customize our HW/SW co-processor by design space exploration and testing on a Mars-like dataset. Implemented on Xilinx Virtex technology, or European NG-MEDIUM devices, the FPGA kernel processes a 1,120 1,120 stereo pair in 1.7s3.1s, utilizing only 5.49.3 LUT6 and 200312 RAMB18. The proposed system exhibits up to 32 speedup over desktop CPUs, or 2,810 over space-grade LEON3, and achieves a mean reconstruction error less than 2cm up to 4m depth. Excluding errors exceeding 2cm (which are less than 4% of the total), the mean error is under 8mm.

Journal

ACM Transactions on Embedded Computing Systems (TECS) – Association for Computing Machinery

Published: Mar 18, 2019

Keywords: Planetary rovers

References

  • Accurate hardware-based stereo vision
    Ambrosch, Kristian; Kubinger, Wilfried
  • Autonomy for mars rovers: Past, present, and future
    Bajracharya, Max; Maimone, Mark W.; Helmick, Daniel
  • Real-time stereo vision system using semi-global matching disparity estimation: Architecture and FPGA-implementation
    Banz, Christian
  • Video-rate stereo matching using Markov random field TRW-S inference on a hybrid CPU+FPGA computing platform
    Choi, Jungwook; Rutenbar, Rob A.
  • An efficient hardware-oriented stereo matching algorithm
    Cocorullo, Giuseppe; Corsonello, Pasquale; Frustaci, Fabio; Perri, Stefania
  • A space-sweep approach to true multi-image matching
    Collins, Robert T.
  • Reconfigurable hardware implementation of a phase-correlation stereo algorithm
    Darabiha, Ahmad; MacLean, W. James; Rose, Jonathan
  • Planetary Rovers: Robotic Exploration of the Solar System
    Ellery, Alex
  • Real-time plane-sweeping stereo with multiple sweeping directions
    Gallup, David; Frahm, Jan-Michael; Mordohai, Philippos; Yang, Qingxiong; Pollefeys, Marc
  • Variable baseline/resolution stereo
    Gallup, David; Frahm, Jan-Michael; Mordohai, Philippos; Pollefeys, Marc
  • An FPGA-based processing pipeline for high-definition stereo video
    Greisen, Pierre; Heinzle, Simon; Gross, Markus; Burg, Andreas P.
  • Evaluation of stereo matching costs on images with radiometric differences
    Hirschmüller, Heiko; Scharstein, Daniel
  • Fast and accurate stereo vision system on FPGA
    Jin, Minxi; Maruyama, Tsutomu
  • FPGA design and implementation of a real-time stereo vision system
    Jin, Seunghun
  • SPARTAN: Developing a vision system for future autonomous space exploration robots
    Kostavelis, Ioannis
  • From eFPGA cores to RHBD System-On-Chip FPGA (NanoXplore’s presentation of the NG-MEDIUM rad-hard FPGA)
    Mauff, Joel Le
  • Real-time ray tracing on coarse-grained reconfigurable processor
    Lee, Jaedon; Shin, Youngsam; Lee, Won-Jong; Ryu, Soojung; Kim, Jeongwook
  • Hardware implementation of stereo correspondence algorithm for the ExoMars mission
    Lentaris, George; Diamantopoulos, Dionysios; Siozios, Kostas; Soudris, Dimitrios; Rodrigálvarez, Marcos Avilés
  • High performance embedded computing in space: Evaluation of platforms for vision-based navigation
    Lentaris, George; Maragos, Konstantinos; Stratakos, Ioannis; Papadopoulos, Lazaros; Papanikolaou, Odysseas; Soudris, Dimitrios; Lourakis, Manolis; Zabulis, Xenophon; Gonzalez-Arjona, David; Furano, Gianluca
  • HW/SW co-design and FPGA acceleration of visual odometry algorithms for rover navigation on mars
    Lentaris, George; Stamoulias, Ioannis; Soudris, Dimitrios; Lourakis, Manolis
  • 3D entity-based stereo matching with ground control points and joint second-order smoothness prior
    Liu, Jing; Li, Chunpeng; Mei, Feng; Wang, Zhaoqi
  • Accurate scale factor estimation in 3D reconstruction
    Lourakis, Manolis; Zabulis, Xenophon
  • Overview of the Mars exploration rovers’ autonomous mobility and vision capabilities
    Maimone, Mark W.; Leger, P. Chris; Biesiadecki, Jeffrey J.
  • Computer vision on mars
    Matthies, Larry
  • Harsh environments: Space radiation environment, effects, and mitigation
    Maurer, Richard H.; Fraeman, Martin E.; Martin, Mark N.; Roth, David R.
  • Towards automatic visual obstacle avoidance
    Moravec, Hans P.
  • Accurate stereo matching by two-step energy minimization
    Mozerov, Mikhail G.; van de Weijer, Joost
  • An FPGA-based real-time system for 3D stereo matching, combining absolute differences and census with aggregation and belief propagation
    Papadimitriou, Kyprianos; Thomas, Sotiris; Dollas, Apostolos
  • FPGA implementation of an efficient similarity-based adaptive window algorithm for real-time stereo matching
    Pérez-Patricio, Madaín; Aguilar-González, Abiel
  • Detailed real-time urban 3D reconstruction from video
    Pollefeys, Marc
  • Iso-disparity surfaces for general stereo configurations
    Pollefeys, Marc; Sinha, Sudipta
  • High level synthesis of stereo matching: Productivity, performance, and software constraints
    Rupnow, Kyle; Liang, Yun; Li, Yinan; Min, Dongbo; Do, Minh; Chen, Deming
  • A taxonomy and evaluation of dense two-frame stereo correspondence algorithms
    Scharstein, Daniel; Szeliski, Richard
  • Middlebury Stereo Evaluation, Version 2
    Scharstein, Daniel; Szeliski, Richard
  • Large-scale outdoor 3D reconstruction on a mobile device
    Schöps, Thomas; Sattler, Torsten; Häne, Christian; Pollefeys, Marc
  • Hardware acceleration for an accurate stereo vision system using mini-census adaptive support region
    Shan, Yi
  • HAPS Family of Physical Prototyping Solutions
    Synopsys,
  • Real-time architecture for a robust multi-scale stereo engine on FPGA
    Tomasi, Matteo; Vanegas, Mauricio; Barranco, Francisco; Diaz, Javier; Ros, Eduardo
  • A hardware-efficient architecture for accurate real-time disparity map estimation
    Ttofis, Christos; Kyrkou, Christos; Theocharides, Theocharis
  • Multiview stereo via volumetric graph-cuts and occlusion robust photo-consistency
    Vogiatzis, George; Esteban, Carlos Hernández; Torr, Philip H. S.; Cipolla, Roberto
  • Real-time high-quality stereo vision system in FPGA
    Wang, Wenqiang; Yan, Jing; Xu, Ningyi; Wang, Yu; Hsu, Feng-Hsiung
  • Stereo matching with color-weighted correlation, hierarchical belief propagation, and occlusion handling
    Yang, Qingxiong
  • Fast stereo matching using adaptive guided filtering
    Yang, Qingqing; Ji, Pan; Li, Dongxiao; Yao, Shaojun; Zhang, Ming
  • Multi-resolution real-time stereo on commodity graphics hardware
    Yang, Ruigang; Pollefeys, Marc
  • Adaptive support-weight approach for correspondence search
    Yoon, Kuk-Jin; Kweon, In So
  • Increasing the accuracy of the space-sweeping approach to stereo reconstruction, using spherical backprojection surfaces
    Zabulis, Xenophon; Kordelas, Georgios; Müller, Karsten; Smolic, Aljoscha
  • Accurate image-guided stereo matching with efficient matching cost and disparity refinement
    Zhan, Yunlong; Gu, Yuzhang; Huang, Kui; Zhang, Cheng; Hu, Keli
  • Low-cost FPGA stereo vision system for real time disparity maps calculation
    Zicari, Paolo; Perri, Stefania; Corsonello, Pasquale; Cocorullo, Giuseppe