CN110191359A - A kind of light field image compression method chosen based on crucial sub-aperture image - Google Patents
A kind of light field image compression method chosen based on crucial sub-aperture image Download PDFInfo
- Publication number
- CN110191359A CN110191359A CN201910409994.5A CN201910409994A CN110191359A CN 110191359 A CN110191359 A CN 110191359A CN 201910409994 A CN201910409994 A CN 201910409994A CN 110191359 A CN110191359 A CN 110191359A
- Authority
- CN
- China
- Prior art keywords
- sub
- aperture
- image
- light field
- crucial
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T9/00—Image coding
- G06T9/002—Image coding using neural networks
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs
- H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
- H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10052—Images from lightfield camera
Abstract
The present invention relates to a kind of light field image compression methods chosen based on crucial sub-aperture image, comprising: the light field image for capturing light-field camera is decomposed into sub-aperture pattern matrix；According to the architectural characteristic of sub-aperture pattern matrix, chooses wherein a small number of sub-aperture images and be used as crucial sub-aperture image, remaining majority sub-aperture image is as non-key sub-aperture image；By crucial sub-aperture image according to progressive scan sequential build at single pseudo- video sequence；Compressed encoding transmission is carried out to pseudo- video sequence based on HEVC；Pass through the non-key sub-aperture images of majority that convolutional neural networks interpolation reconstruction end to end does not compile transmission out using a small number of crucial sub-aperture images have been decoded；The complete sub-aperture pattern matrix of reconstruction is restored to light field image；Realize the Efficient Compression coding and decoding and rebuilding of light field image.The method of the present invention can efficiently reduce memory space and transmission bandwidth needed for light field image under the premise of guaranteeing objective reconstruction quality, realize higher compression performance.
Description
Technical field
The present invention relates to light field image compression field, especially a kind of light field image chosen based on crucial sub-aperture image
Compression method.
Background technique
In recent years, light field image is attracted extensive attention since it can provide three-dimensional scenic feeling of immersion abundant, true to nature, is being given pleasure to
The fields such as pleasure, medicine, agricultural, industry have broad application prospects.Can only capture in scene compared to traditional images acquisition has
The two-dimensional projection in angular range is limited, optical field imaging can collect the light radiation information from all directions, demultiplexing tradition
The angle information lost in photography.The high-dimensional expression of light field image provides richer scene information, efficiently contributes to solve
Certainly traditional computer visual problem, such as depth perception, three-dimensional reconstruction, scene understanding.However, the high information quantity of light field image and
High-dimensional feature makes it have huge data volume.Efficient Compression is not carried out to light field image to be unable to satisfy at all with massive information
The requirement of the light field image being characterized stored and transmitted.Meanwhile light field image and traditional natural image and video have difference
Structure feature and statistical property.Traditional natural image and video compression technology are not directly adaptable to use light field image.Therefore, light
Field picture is collapsed into research topic that is urgent and having important theoretical significance and practical application value.
Domestic and international research worker successively proposes a series of light field image compression method around light field image.Its
In, most intuitive method is directly to be compressed using the compression method of various still images to light field original image, such as static
Standard of image compression (JPEG).Another kind of method is to carry out rarefaction dimensionality reduction to light field original image using the method for compressed sensing
It is compressed again afterwards.But field information is to be characterized in the way of two dimensional image multiplexing to four-dimensional field information, light field
There is the correlation of deeper grade inside original image.Meanwhile light field image camera array or light-field camera itself construct it is several
There is geometry redundancies for what constraint.Above-mentioned compression method does not account for the characteristic of light field image itself, and compression performance is simultaneously paid no attention to
Think, there is also biggish rooms for promotion.
Video encoding standard HEVC (High Efficiency Video Coding) is under the premise of identical video quality
About 50% is reduced than H.264/AVC high-grade (High Profile), supports various specifications video, in compression ratio, calculates complexity
Balance well is realized between degree, robustness and system delay, and new development opportunity is carried out for light field image compression strap.However,
The core of HEVC is mainly around how improving natural image and video quality this theme is unfolded, and visual redundancy degree disappears
Except method fails to fully consider the unique architectural characteristic of light field image.Therefore, light field image compression effect directly is carried out using HEVC
Fruit is unsatisfactory.
By foregoing description, although light field image compression obtains certain progress, due to answering for light field image structure
Polygamy, light field image compression needs to solve also in the exploratory stage there are many more problem on the whole.
Summary of the invention
It is a primary object of the present invention to overcome drawbacks described above in the prior art, propose a kind of based on crucial sub-aperture figure
As the light field image compression method chosen.The method of the present invention fully considers the architectural characteristic of input light ground subaperture image array,
The performance of light field image compression method can be effectively improved.
The present invention adopts the following technical scheme:
A kind of light field image compression method chosen based on crucial sub-aperture image, steps are as follows:
1) light field image to be compressed is decomposed into sub-aperture pattern matrix；
2) for selected part sub-aperture image as crucial sub-aperture image, remainder is non-key sub-aperture image；
3) by crucial sub-aperture picture construction at single pseudo- video sequence；
4) compressed encoding transmission is carried out to pseudo- video sequence based on HEVC；
5) the pseudo- video sequence of decoding and rebuilding is decomposed into crucial sub-aperture image；
6) it utilizes and has decoded the non-key sub-aperture image of crucial sub-aperture image reconstruction；
7) the sub-aperture pattern matrix of reconstruction is restored to light field image.
The light field image to be compressed is shot to obtain by hand-held light-field camera.
It is realized by MATLAB light field tool box and the light field image to be compressed is decomposed into sub-aperture pattern matrix.
It is realized by MATLAB light field tool box and the sub-aperture pattern matrix of the reconstruction is restored to light field image.
According to the design feature of sub-aperture pattern matrix, crucial sub-aperture image is chosen.
The key sub-aperture image uses progressive scan sequential build for single pseudo- video sequence.
Interpolation reconstruction is carried out to non-key sub-aperture image using convolutional neural networks end to end.
The convolutional neural networks end to end, any two figure of the input sub-aperture pattern matrix with a line or same row
Picture, can interpolation go out any number of medial view.
By the above-mentioned description of this invention it is found that compared with prior art, the invention has the following beneficial effects:
1, the method for the present invention utilizes the architectural characteristic of light field sub-aperture pattern matrix to be compressed, need to only choose a small number of crucial sons
Aperture sub-aperture image carries out encoding and decoding based on HEVC platform using the single pseudo- video sequence of progressive scan mode building, with
Achieve the purpose that remove redundancy.
2, in view of there is the parallax variation of regularity in the method for the present invention between sub-aperture image, in decoding end using
The a small number of crucial sub-aperture images of decoding go out the non-pass of majority of uncoded transmission by convolutional neural networks interpolation reconstruction end to end
Key sub-aperture image improves code efficiency to save code stream.
Detailed description of the invention
Fig. 1 is the main flow chart of the method for the present invention；
Fig. 2 is that the crucial sub-aperture image of the method for the present invention chooses schematic diagram.
Specific embodiment
Below by way of specific embodiment, the invention will be further described.
Referring to Fig. 1, it is a kind of based on crucial sub-aperture image choose light field image compression method, to input light field image into
Row re-sampling operations are broken down into sub-aperture pattern matrix, choose a small number of crucial sub-aperture picture constructions into single pseudo- video
Sequence carries out Efficient Compression coding based on HEVC, and decoding end interpolation reconstruction goes out whole sub-aperture images, and is restored to light field
Image, specific implementation step are as follows:
1) light field image is extracted as input, is broken down into sub-aperture pattern matrix.
Specifically, light field image to be compressed is carried out resampling using MATLAB light field tool box, it is decomposed into sub-aperture figure
As array format, it is equivalent to the natural image shot with different view to Same Scene, is only deposited between sub-aperture image
In visual angle difference.
2) sub-aperture image is divided into crucial sub-aperture image and non-key sub-aperture image.
Specifically, the design feature of zygote subaperture image array, chooses crucial sub-aperture image, according to Fig. 2
The shown a small number of sub-aperture images for choosing oblique line mark are as crucial sub-aperture image, and remaining majority sub-aperture image is as non-pass
Key sub-aperture image.
3) by crucial sub-aperture picture construction at single pseudo- video sequence.
Specifically, using progressive scan mode, according to sequence from top to bottom, from left to right by the crucial sub-aperture of selection
Picture construction is at single pseudo- video sequence.
4) compressed encoding transmission is carried out to pseudo- video sequence based on HEVC.It is specific as follows:
Specifically, using LowDelay_P pattern-coding structure, only first frame is according to intraframe coding method in coding side
It is encoded, subsequent each frame is all encoded as general P frame, as much as possible removal redundancy, to the pseudo- video constructed
Sequence carries out coding transmission.
5) the pseudo- video sequence of decoding and rebuilding is decomposed into crucial sub-aperture image.
6) it is based on the non-key sub-aperture image of convolutional neural networks interpolation reconstruction.
Specifically, being completed using convolutional neural networks end to end using crucial sub-aperture image has been decoded in decoding end
To the interpolation reconstruction of all non-key sub-aperture images, to rebuild entire sub-aperture pattern matrix.
7) the complete sub-aperture pattern matrix of reconstruction is restored to light field image, light field image is rebuild in output.
Method proposed by the invention can efficiently reduce light field image under the premise of guaranteeing objective reconstruction quality
Required memory space and transmission bandwidth realize higher compression performance.
The above is only a specific embodiment of the present invention, but the design concept of the present invention is not limited to this, all to utilize this
Design makes a non-material change to the present invention, and should all belong to behavior that violates the scope of protection of the present invention.
Claims (8)
1. a kind of light field image compression method chosen based on crucial sub-aperture image, it is characterized in that, steps are as follows:
1) light field image to be compressed is decomposed into sub-aperture pattern matrix；
2) for selected part sub-aperture image as crucial sub-aperture image, remainder is non-key sub-aperture image；
3) by crucial sub-aperture picture construction at single pseudo- video sequence；
4) compressed encoding transmission is carried out to pseudo- video sequence based on HEVC；
5) the pseudo- video sequence of decoding and rebuilding is decomposed into crucial sub-aperture image；
6) it utilizes and has decoded the non-key sub-aperture image of crucial sub-aperture image reconstruction；
7) the sub-aperture pattern matrix of reconstruction is restored to light field image.
2. the light field image compression method according to claim 1 chosen based on crucial sub-aperture image, which is characterized in that
The light field image to be compressed is shot to obtain by hand-held light-field camera.
3. the light field image compression method according to claim 1 chosen based on crucial sub-aperture image, which is characterized in that
It is realized by MATLAB light field tool box and the light field image to be compressed is decomposed into sub-aperture pattern matrix.
4. the light field image compression method according to claim 1 chosen based on crucial sub-aperture image, which is characterized in that
It is realized by MATLAB light field tool box and the sub-aperture pattern matrix of the reconstruction is restored to light field image.
5. the light field image compression method according to claim 1 chosen based on crucial sub-aperture image, which is characterized in that
According to the design feature of sub-aperture pattern matrix, crucial sub-aperture image is chosen.
6. the light field image compression method according to claim 1 chosen based on crucial sub-aperture image, which is characterized in that
The key sub-aperture image uses progressive scan sequential build for single pseudo- video sequence.
7. the light field image compression method according to claim 1 chosen based on crucial sub-aperture image, which is characterized in that
Interpolation reconstruction is carried out to non-key sub-aperture image using convolutional neural networks end to end.
8. the light field image compression method according to claim 7 chosen based on crucial sub-aperture image, described end-to-end
Convolutional neural networks, input sub-aperture pattern matrix with a line or same row any two images, can interpolation go out arbitrary number
The medial view of amount.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910409994.5A CN110191359A (en) | 2019-05-16 | 2019-05-16 | A kind of light field image compression method chosen based on crucial sub-aperture image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910409994.5A CN110191359A (en) | 2019-05-16 | 2019-05-16 | A kind of light field image compression method chosen based on crucial sub-aperture image |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110191359A true CN110191359A (en) | 2019-08-30 |
Family
ID=67716432
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910409994.5A Pending CN110191359A (en) | 2019-05-16 | 2019-05-16 | A kind of light field image compression method chosen based on crucial sub-aperture image |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110191359A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113965757A (en) * | 2021-10-21 | 2022-01-21 | 上海师范大学 | Light field image coding method and device based on EPI (intrinsic similarity) and storage medium |
| WO2022016350A1 (en) * | 2020-07-21 | 2022-01-27 | Oppo广东移动通信有限公司 | Light field image processing method, light field image encoder and decoder, and storage medium |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104469372A (en) * | 2014-11-06 | 2015-03-25 | 中国科学院计算技术研究所 | Method and system for compressing light field image collected by microlens array |
| CN106973293A (en) * | 2017-04-21 | 2017-07-21 | 中国科学技术大学 | The light field image coding method predicted based on parallax |
| CN107105278A (en) * | 2017-04-21 | 2017-08-29 | 中国科学技术大学 | The coding and decoding video framework that motion vector is automatically generated |
| CN107295264A (en) * | 2017-08-01 | 2017-10-24 | 清华大学深圳研究生院 | One kind is based on homography conversion light-field data compression method |
| CN107770537A (en) * | 2017-11-02 | 2018-03-06 | 中国科学技术大学 | Based on the light field image compression method linearly rebuild |
| CN108184064A (en) * | 2018-01-04 | 2018-06-19 | 中国科学技术大学 | A kind of visual angle image array division methods |
| CN109118474A (en) * | 2018-07-07 | 2019-01-01 | 福州大学 | A kind of image drawing method of multiple views sparseness measuring |
-
2019
- 2019-05-16 CN CN201910409994.5A patent/CN110191359A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104469372A (en) * | 2014-11-06 | 2015-03-25 | 中国科学院计算技术研究所 | Method and system for compressing light field image collected by microlens array |
| CN106973293A (en) * | 2017-04-21 | 2017-07-21 | 中国科学技术大学 | The light field image coding method predicted based on parallax |
| CN107105278A (en) * | 2017-04-21 | 2017-08-29 | 中国科学技术大学 | The coding and decoding video framework that motion vector is automatically generated |
| CN107295264A (en) * | 2017-08-01 | 2017-10-24 | 清华大学深圳研究生院 | One kind is based on homography conversion light-field data compression method |
| CN107770537A (en) * | 2017-11-02 | 2018-03-06 | 中国科学技术大学 | Based on the light field image compression method linearly rebuild |
| CN108184064A (en) * | 2018-01-04 | 2018-06-19 | 中国科学技术大学 | A kind of visual angle image array division methods |
| CN109118474A (en) * | 2018-07-07 | 2019-01-01 | 福州大学 | A kind of image drawing method of multiple views sparseness measuring |
Non-Patent Citations (1)
| Title |
|---|
| 马晓辉等: ""基于多视点伪序列的光场图像压缩"", 《信号处理》 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2022016350A1 (en) * | 2020-07-21 | 2022-01-27 | Oppo广东移动通信有限公司 | Light field image processing method, light field image encoder and decoder, and storage medium |
| CN113965757A (en) * | 2021-10-21 | 2022-01-21 | 上海师范大学 | Light field image coding method and device based on EPI (intrinsic similarity) and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106664414B (en) | For the candidate method and device generated of single sample mode in Video coding | |
| TWI239207B (en) | Method and device for coding digital images using intra-mode block prediction | |
| CN102939763B (en) | Calculating disparity for three-dimensional images | |
| JP7431742B2 (en) | Method and apparatus for encoding/decoding a point cloud representing a three-dimensional object | |
| CN108921910B (en) | JPEG coding compressed image restoration method based on scalable convolutional neural network | |
| CN102396222B (en) | Adaptive entropy coding for images and videos using set partitioning in generalized hierarchical trees | |
| JP6360969B2 (en) | Image encoding and decoding method and apparatus | |
| TW201223290A (en) | Image data compression and decompression methods and display driving device using the same | |
| CN102356635A (en) | Video acquisition and processing systems | |
| CN101883284B (en) | Video encoding/decoding method and system based on background modeling and optional differential mode | |
| CN112308754B (en) | Multi-decoding-end parallel extraction method for high-capacity robust watermark application | |
| CN110191359A (en) | A kind of light field image compression method chosen based on crucial sub-aperture image | |
| WO2023082834A1 (en) | Video compression method and apparatus, and computer device and storage medium | |
| EP3777180A1 (en) | A method and apparatus for encoding/decoding a point cloud representing a 3d object | |
| CN108777794A (en) | The coding method of image and device, storage medium, electronic device | |
| Hu et al. | An adaptive two-layer light field compression scheme using GNN-based reconstruction | |
| CN103561255B (en) | A kind of Nakedness-yet stereoscopic display method | |
| Fang et al. | 3dac: Learning attribute compression for point clouds | |
| CN110062231A (en) | Method for compressing image based on multilayer convolutional neural networks | |
| CN106210443A (en) | Network controls the method for display content synchronization | |
| Yang et al. | Real-time 3d video compression for tele-immersive environments | |
| CN104935928B (en) | A kind of efficient image compression method based on spatial domain down-sampling pattern | |
| CN103780908B (en) | Efficient H264 decoding method | |
| CN111080729B (en) | Training picture compression network construction method and system based on Attention mechanism | |
| CN106559668A (en) | A kind of low code rate image compression method based on intelligent quantization technology |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20190830 |
|
| WD01 | Invention patent application deemed withdrawn after publication |