Dense Reconstruction Algorithm of Sparse Light-field Based on Optical Flow Method
-
摘要: 光场成像在三维重建、合成孔径去遮挡和全息成像等应用中具有重要作用。光场稠密重建算法能够弥补光场成像硬件的不足，实现稀疏光场的稠密重建。本文从二维光流基本原理出发，以光场双平面模型为基础，提出一种新的四维光场光流约束方程数学模型，利用该约束方程求解得到的光场光流确定新视角的位置坐标，并通过插值计算及图像反演逐点获得新坐标的强度值，最终获得高质量的新视角合成图像。实验结果表明，本文方法能够对长基线场景中的纹理、阴影及色彩等信息进行高质量重建，定量评价结果表明该算法能够完成复杂场景的光场稠密重建任务。本文算法仅适用于线性光场光流的约束以及一维视角光场的情形，后续相关研究将围绕非线性光流约束以及多维视角光场的情形展开。Abstract: Light field imaging plays a vital role in three-dimensional reconstruction, synthetic aperture de occlusion, and holographic imaging. The light field dense reconstruction algorithm can make up for the shortage of light field imaging hardware and realize the dense reconstruction of the sparse light field. Based on the basic principle of two-dimensional optical flow and the light field biplane model, a new mathematical model of a four-dimensional light field optical flow constraint equation is proposed in this paper. The position coordinates of the new viewing angle are determined by using the light field optical flow solved by the constraint equation, and the intensity values of the new coordinates are obtained point by point through interpolation calculation and image inversion, so as to finally obtain a high-quality new viewing angle synthetic images. The experimental results show that the proposed method can realize high-quality reconstruction of the texture, shadow, and color information in the extended baseline scene. The quantitative evaluation results show that the algorithm can accomplish the task of dense light field reconstruction in complex settings. This algorithm in this paper is only applicable to the case of linear optical flow constraint and one-dimensional viewing angle light field. The subsequent related research will focus on the case of nonlinear optical flow constraint and multi-dimensional viewing angle light field.
-
Keywords:
- light field /
- light field reconstruction /
- LK optical flow method
-
-
![]()
图 1 二维成像到四维成像扩展
Figure 1. Extension of two-dimensional imaging to four-dimensional imaging
![]()
图 8 包含高低频信息场景的稠密光场重建
Figure 8. Dense light field reconstruction of scenes containing high and low frequency information
表 1 不同场景稠密光场重建PSNR值对比结果
Table 1 Comparison of PSNR values of dense light field reconstruction in different scenes
场景 Table Bicycle Town Boardgames Rosemary Vinyl 文献22算法 34.865 30.773 34.015 30.813 35.367 34.614 本文算法1*9 39.326 34.817 38.979 44.117 41.105 46.855 本文算法1*7 40.074 35.030 39.780 44.228 42.412 47.389 
下载: 导出CSV
表 2 不同场景稠密光场重建SSIM值对比结果
Table 2 Comparison of SSIM values of dense light field reconstruction in different scenes
场景 Table Bicycle Town Boardgames Rosemary Vinyl 文献22算法 0.932 0.886 0.947 0.934 0.970 0.942 本文算法1*9 0.960 0.958 0.982 0.993 0.987 0.994 本文算法1*7 0.963 0.959 0.984 0.994 0.988 0.995
下载: 导出CSV
-
[1] IHRKE I, RESTREPO J, MIGNARD-DEBISE L. Principles of light field imaging: Briefly revisiting 25 years of research[J]. IEEE Signal Processing Magazine, 2016, 33(5): 59−69. doi: 10.1109/MSP.2016.2582220
[2] REN N, LEVOY M, BREDIF M, et al. Light field photography with a hand-held plenoptic camera[J]. Stanford University Cstr, 2005, 2(1): 1−11.
[3] WILBURN B, JOSHI N, VAISH V, et al. High performance imaging using large camera arrays[J]. ACM Transactions on Graphics, 2005, 24(3): 765−776. doi: 10.1145/1073204.1073259
[4] STA B, LU Z, WZA B, et al. Quality assessment of DIBR-synthesized views: An overview[J]. Neurocomputing, 2021, 423: 158−178. doi: 10.1016/j.neucom.2020.09.062
[5] ZHOU W, LIU G, SHI J, et al. Depth-guided view synthesis for light field reconstruction from a single image[J], Image and Vision Computing, 2020, 95(c): 103874.
[6] JING X, MA Y, ZHAO Q, et al. Light field reconstruction using dynamically generated filters[J]. Springer, Cham, 2020: 3−13.
[7] 焦鹏飞, 李亮, 赵骥. 压缩感知在医学图像重建中的最新进展[J]. CT理论与应用研究, 2012,21(1): 133−147. JIAO P F, LI L, ZHAO J. New advances of compressed sensing in medical image reconstruction[J]. CT Theory and Applications, 2012, 21(1): 133−147. (in Chinese).
[8] CAO X, GENG Z, et al. Dictionary-based light field acquisition using sparse camera array[J]. Optics Express, 2014, 22(20): 24081−24095. doi: 10.1364/OE.22.024081
[9] 王飞, 王昊, 卞耀明, 等. 深度学习在计算成像中的应用[J]. 光学学报, 2020,40(1): 0111002. doi: 10.3788/AOS202040.0111002 WANG F, WANG H, BIAN Y M, et al. Applications of deep learning in computational imaging[J]. Acta Optica Sinica, 2020, 40(1): 0111002. (in Chinese). doi: 10.3788/AOS202040.0111002
[10] FLYNN J, NEULANDER I, PHILBIN J, et al. Deep stereo: Learning to predict new views from the world's imagery[C]//IEEE Computer Society, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Washington D C, USA: 2016: 5515-5524.
[11] 李坤, 毛亚丽, 马健, 等. 稀疏光场的密集重建方法[J]. 中国科技论文, 2018,13(20): 2317−2322. doi: 10.3969/j.issn.2095-2783.2018.20.006 LI K, MAO Y L, MA J, et al. Dense reconstruction for sparsely-sampled light field[J]. China Sciencepapter, 2018, 13(20): 2317−2322. (in Chinese). doi: 10.3969/j.issn.2095-2783.2018.20.006
[12] 王伟健. 基于DIBR的虚拟视点合成算法研究[D]. 武汉: 华中科技大学, 2016. [13] GIBSON J J. The perception of the visual world[M]. Oxford, England: Houghton Mifflin, 1950.
[14] 都雪静, 张美欧. 基于光流法与背景建模法融合的车道线识别算法研究[J]. 重庆理工大学学报(自然科学), 2021,35(3): 29−35, 99. DU X J, ZHANG M O. Based on fusing optical flow method and background, odeling for lane line recognition[J]. Journal of Chongqing University of Technology (Natural Science), 2021, 35(3): 29−35, 99. (in Chinese).
[15] HARTMANN C, WANG J, OPRISTESCU D, et al. Implementation and evaluation of optical flow methods for two-dimensional deformation measurement in comparison to digital image correlation[J]. Optics & Lasers in Engineering, 2018, 107(aug.): 127−141.
[16] 於小杰, 贺勇, 刘盛华. 一种用于无人机室内定位的改进ORB光流算法[J]. 计算机工程与应用, 2021, 57(4): 266-271. SHI X J, HE Y, LIU S H. An improved orb optical flow algorithm for UAV indoor positioning[J]. Computer Engineering and Applications, 2021, 57(4): 266-271. (in Chinese).
[17] GANG Z, TANG S, LI J. Face landmark point tracking using LK pyramid optical flow[C]//Tenth International Conference on Machine Vision (ICMV 2017). 2018.
[18] 赵于平, 金熙, 赵松年. 光场成像中的景深扩展方法与信息处理[J]. 现代电子技术, 2016,39(4): 102−107. ZHAO Y P, JING X, ZHAO S N. Scene depth extension method and information processing in light field imaging[J]. Modern Electronics Technique, 2016, 39(4): 102−107. (in Chinese).
[19] 李亚楠, 赵耀, 林春雨, 等. 基于图像分割的金字塔Lucas-Kanade光流法提取深度信息[J]. 铁道学报, 2015,37(1): 63−68. doi: 10.3969/j.issn.1001-8360.2015.01.010 LI Y N, ZHAO Y, LIN C Y, et al. Depth information extraction with pyramid lucas-kanade optical flow method based on image segmentation[J]. Journal of the China Railway Society, 2015, 37(1): 63−68. (in Chinese). doi: 10.3969/j.issn.1001-8360.2015.01.010
[20] HONAUER K, JOHANNSEN O, KONDERMANN D, et al. A dataset and evaluation methodology for depth estimation on 4d light fields[M]. Asian Conference on Computer Vision, 2016: 19-34.
[21] 佟雨兵, 张其善, 祁云平. 基于PSNR与SSIM联合的图像质量评价模型[J]. 中国图象图形学报, 2006,11(12): 1758−1763. doi: 10.11834/jig.2006012307 TONG Y B, ZHANG Q S, QI Y P. Image quality assessing by combining PSNR with SSIM[J]. Journal of Image and Graphics, 2006, 11(12): 1758−1763. (in Chinese). doi: 10.11834/jig.2006012307
[22] 夏正德, 宋娜, 刘宾, 等. 基于字典学习的稠密光场重建算法[J]. 物理学报, 2020,69(6): 064201. doi: 10.7498/aps.69.20191621 XIA Z D, SONG N, LIU B, et al. Dense light field reconstruction algorithm based on dictionary learning[J]. Acta Physica Sinica, 2020, 69(6): 064201. (in Chinese). doi: 10.7498/aps.69.20191621
计量
- 文章访问数: 554
- HTML全文浏览量: 279
- PDF下载量: 52
