Dense Reconstruction Algorithm of Sparse Light-field Based on Optical Flow Method

LIU Jian; SONG Na; PAN Jinxiao; Abdulr Ghaffa; YANG Ming

doi:10.15953/j.ctta.2021.052

Volume 31 Issue 2

Mar. 2022

Turn off MathJax

Article Contents

Abstract

References

CT Theory and Applications > 2022 > 31(2): 173-185. > DOI: 10.15953/j.ctta.2021.052

LIU J, SONG N, PAN J X, et al. Dense reconstruction algorithm of sparse light-field based on optical flow method[J]. CT Theory and Applications, 2022, 31(2): 173-185. DOI: 10.15953/j.ctta.2021.052. (in Chinese).

Citation:

PDF (3091 KB)

Dense Reconstruction Algorithm of Sparse Light-field Based on Optical Flow Method

LIU Jian^{1, 2, 3,},
SONG Na¹,
PAN Jinxiao^{2, 3},
Abdulr Ghaffa^{4, 5},
YANG Ming^{1, 2, 3, ,}

1.
School of Science, North University of China, Taiyuan 030051, China
2.
State Key Laboratory of Dynamic Testing Technology, North University of China, Taiyuan 030051, China
3.
Shanxi Key Laboratory of Signal Capturing and Processing, North University of China, Taiyuan 030051, China
4.
State Key Laboratory of Geomechanics and Geotechnical Engineering, Institute of Rock and Soil Mechanics, Chinese Academy of Sciences, Wuhan 430071, China
5.
University of Chinese Academy of Sciences, Beijing 100049, China

More Information

Received Date: November 29, 2021
Accepted Date: January 16, 2022
Available Online: January 20, 2022
Published Date: March 31, 2022

Graphical Abstract

Abstract

Abstract

Light field imaging plays a vital role in three-dimensional reconstruction, synthetic aperture de occlusion, and holographic imaging. The light field dense reconstruction algorithm can make up for the shortage of light field imaging hardware and realize the dense reconstruction of the sparse light field. Based on the basic principle of two-dimensional optical flow and the light field biplane model, a new mathematical model of a four-dimensional light field optical flow constraint equation is proposed in this paper. The position coordinates of the new viewing angle are determined by using the light field optical flow solved by the constraint equation, and the intensity values of the new coordinates are obtained point by point through interpolation calculation and image inversion, so as to finally obtain a high-quality new viewing angle synthetic images. The experimental results show that the proposed method can realize high-quality reconstruction of the texture, shadow, and color information in the extended baseline scene. The quantitative evaluation results show that the algorithm can accomplish the task of dense light field reconstruction in complex settings. This algorithm in this paper is only applicable to the case of linear optical flow constraint and one-dimensional viewing angle light field. The subsequent related research will focus on the case of nonlinear optical flow constraint and multi-dimensional viewing angle light field.
- light field,
- light field reconstruction,
- LK optical flow method

FullText(HTML)

References (22)

References

[1]	IHRKE I, RESTREPO J, MIGNARD-DEBISE L. Principles of light field imaging: Briefly revisiting 25 years of research[J]. IEEE Signal Processing Magazine, 2016, 33(5): 59−69. doi: 10.1109/MSP.2016.2582220
[2]	REN N, LEVOY M, BREDIF M, et al. Light field photography with a hand-held plenoptic camera[J]. Stanford University Cstr, 2005, 2(1): 1−11.
[3]	WILBURN B, JOSHI N, VAISH V, et al. High performance imaging using large camera arrays[J]. ACM Transactions on Graphics, 2005, 24(3): 765−776. doi: 10.1145/1073204.1073259
[4]	STA B, LU Z, WZA B, et al. Quality assessment of DIBR-synthesized views: An overview[J]. Neurocomputing, 2021, 423: 158−178. doi: 10.1016/j.neucom.2020.09.062
[5]	ZHOU W, LIU G, SHI J, et al. Depth-guided view synthesis for light field reconstruction from a single image[J], Image and Vision Computing, 2020, 95(c): 103874.
[6]	JING X, MA Y, ZHAO Q, et al. Light field reconstruction using dynamically generated filters[J]. Springer, Cham, 2020: 3−13.
[7]	焦鹏飞, 李亮, 赵骥. 压缩感知在医学图像重建中的最新进展[J]. CT理论与应用研究, 2012,21(1): 133−147. JIAO P F, LI L, ZHAO J. New advances of compressed sensing in medical image reconstruction[J]. CT Theory and Applications, 2012, 21(1): 133−147. (in Chinese).
[8]	CAO X, GENG Z, et al. Dictionary-based light field acquisition using sparse camera array[J]. Optics Express, 2014, 22(20): 24081−24095. doi: 10.1364/OE.22.024081
[9]	王飞, 王昊, 卞耀明, 等. 深度学习在计算成像中的应用[J]. 光学学报, 2020,40(1): 0111002. doi: 10.3788/AOS202040.0111002 WANG F, WANG H, BIAN Y M, et al. Applications of deep learning in computational imaging[J]. Acta Optica Sinica, 2020, 40(1): 0111002. (in Chinese). doi: 10.3788/AOS202040.0111002
[10]	FLYNN J, NEULANDER I, PHILBIN J, et al. Deep stereo: Learning to predict new views from the world's imagery[C]//IEEE Computer Society, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Washington D C, USA: 2016: 5515-5524.
[11]	李坤, 毛亚丽, 马健, 等. 稀疏光场的密集重建方法[J]. 中国科技论文, 2018,13(20): 2317−2322. doi: 10.3969/j.issn.2095-2783.2018.20.006 LI K, MAO Y L, MA J, et al. Dense reconstruction for sparsely-sampled light field[J]. China Sciencepapter, 2018, 13(20): 2317−2322. (in Chinese). doi: 10.3969/j.issn.2095-2783.2018.20.006
[12]	王伟健. 基于DIBR的虚拟视点合成算法研究[D]. 武汉: 华中科技大学, 2016.
[13]	GIBSON J J. The perception of the visual world[M]. Oxford, England: Houghton Mifflin, 1950.
[14]	都雪静, 张美欧. 基于光流法与背景建模法融合的车道线识别算法研究[J]. 重庆理工大学学报(自然科学), 2021,35(3): 29−35, 99. DU X J, ZHANG M O. Based on fusing optical flow method and background, odeling for lane line recognition[J]. Journal of Chongqing University of Technology (Natural Science), 2021, 35(3): 29−35, 99. (in Chinese).
[15]	HARTMANN C, WANG J, OPRISTESCU D, et al. Implementation and evaluation of optical flow methods for two-dimensional deformation measurement in comparison to digital image correlation[J]. Optics & Lasers in Engineering, 2018, 107(aug.): 127−141.
[16]	於小杰, 贺勇, 刘盛华. 一种用于无人机室内定位的改进ORB光流算法[J]. 计算机工程与应用, 2021, 57(4): 266-271. SHI X J, HE Y, LIU S H. An improved orb optical flow algorithm for UAV indoor positioning[J]. Computer Engineering and Applications, 2021, 57(4): 266-271. (in Chinese).
[17]	GANG Z, TANG S, LI J. Face landmark point tracking using LK pyramid optical flow[C]//Tenth International Conference on Machine Vision (ICMV 2017). 2018.
[18]	赵于平, 金熙, 赵松年. 光场成像中的景深扩展方法与信息处理[J]. 现代电子技术, 2016,39(4): 102−107. ZHAO Y P, JING X, ZHAO S N. Scene depth extension method and information processing in light field imaging[J]. Modern Electronics Technique, 2016, 39(4): 102−107. (in Chinese).
[19]	李亚楠, 赵耀, 林春雨, 等. 基于图像分割的金字塔Lucas-Kanade光流法提取深度信息[J]. 铁道学报, 2015,37(1): 63−68. doi: 10.3969/j.issn.1001-8360.2015.01.010 LI Y N, ZHAO Y, LIN C Y, et al. Depth information extraction with pyramid lucas-kanade optical flow method based on image segmentation[J]. Journal of the China Railway Society, 2015, 37(1): 63−68. (in Chinese). doi: 10.3969/j.issn.1001-8360.2015.01.010
[20]	HONAUER K, JOHANNSEN O, KONDERMANN D, et al. A dataset and evaluation methodology for depth estimation on 4d light fields[M]. Asian Conference on Computer Vision, 2016: 19-34.
[21]	佟雨兵, 张其善, 祁云平. 基于PSNR与SSIM联合的图像质量评价模型[J]. 中国图象图形学报, 2006,11(12): 1758−1763. doi: 10.11834/jig.2006012307 TONG Y B, ZHANG Q S, QI Y P. Image quality assessing by combining PSNR with SSIM[J]. Journal of Image and Graphics, 2006, 11(12): 1758−1763. (in Chinese). doi: 10.11834/jig.2006012307
[22]	夏正德, 宋娜, 刘宾, 等. 基于字典学习的稠密光场重建算法[J]. 物理学报, 2020,69(6): 064201. doi: 10.7498/aps.69.20191621 XIA Z D, SONG N, LIU B, et al. Dense light field reconstruction algorithm based on dictionary learning[J]. Acta Physica Sinica, 2020, 69(6): 064201. (in Chinese). doi: 10.7498/aps.69.20191621

Cited By

Get Citation

PDF

XML

Article views (554) PDF downloads (52)

Dense Reconstruction Algorithm of Sparse Light-field Based on Optical Flow Method

Abstract

References

Catalog

Related

Dense Reconstruction Algorithm of Sparse Light-field Based on Optical Flow Method

Abstract

References

Catalog

Related

Export File

Citation

Format

Content