Light Field Compression by Residual CNN-Assisted JPEG
Document Type
Conference Proceeding
Publication Date
9-22-2021
Department
Department of Electrical and Computer Engineering; Department of Computer Science
Abstract
Light field (LF) imaging has gained significant attention due to its recent success in 3-dimensional (3D) displaying and rendering as well as augmented and virtual reality usage. Because of the two extra dimensions, LFs are much larger than conventional images. We develop a JPEG-assisted learning-based technique to reconstruct an LF from a JPEG bitstream with a bit per pixel ratio of 0.0047 on average. For compression, we keep the LF's center view and use JPEG compression with 50% quality. Our reconstruction pipeline consists of a small JPEG enhancement network (JPEG-Hance), a depth estimation network (Depth-Net), followed by view synthesizing by warping the enhanced center view. Our pipeline is significantly faster than using video compression on pseudo-sequences extracted from an LF, both in compression and decompression, while maintaining effective performance. We show that with a 1% compression time cost and 18x speedup for decompression, our methods reconstructed LFs have better structural similarity index metric (SSIM) and comparable peak signal-to-noise ratio (PSNR) compared to the state-of-the-art video compression techniques used to compress LFs.
Publication Title
Proceedings of the International Joint Conference on Neural Networks
ISBN
9780738133669
Recommended Citation
Hedayati, E.,
Havens, T. C.,
&
Bos, J.
(2021).
Light Field Compression by Residual CNN-Assisted JPEG.
Proceedings of the International Joint Conference on Neural Networks,
2021-July.
http://doi.org/10.1109/IJCNN52387.2021.9534210
Retrieved from: https://digitalcommons.mtu.edu/michigantech-p/15425