CN105163129B - Gradient map guiding based on depth resampling 3D HEVC decoding methods - Google Patents
Gradient map guiding based on depth resampling 3D HEVC decoding methods Download PDFInfo
- Publication number
- CN105163129B CN105163129B CN201510608870.1A CN201510608870A CN105163129B CN 105163129 B CN105163129 B CN 105163129B CN 201510608870 A CN201510608870 A CN 201510608870A CN 105163129 B CN105163129 B CN 105163129B
- Authority
- CN
- China
- Prior art keywords
- mrow
- msub
- mtd
- video
- pixel
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Abstract
The present invention relates to a kind of guiding of gradient map based on depth resampling 3D HEVC decoding methods.This method of the present invention is encoded to texture video by original resolution, and resolution decreasing coding is carried out to deep video；Coding side is using the method for the piecemeal medium filtering of gradient map guiding to deep video down-sampling；Decoding end is up-sampled using the method for the neighborhood valuation of gradient map guiding to deep video；The texture video and up-sampling deep video finally obtained to decoding carries out View Synthesis, obtains required multi-view point video.The inventive method protects the edge of deep video, improves the quality of deep video resampling, reduce algorithm complex on the premise of coding efficiency is kept.
Description
Technical field
The invention belongs to 3D technical field of video coding, more particularly to a kind of guiding of gradient map based on depth resampling
3D-HEVC decoding methods.
Background technology
In recent years since, many Video Applications based on 3D, such as mobile 3 D video, free viewpoint video and three-dimensional video-frequency show
Show etc. that technology is developed rapidly.But the 3D Video Applications based on multiple views technology bring substantial amounts of video data and needed
Store or transmit, constrain its further development.With the coding standard 3D- of the multiple views plus depth data based on HEVC
HEVC formal issue, bring the Efficient Compression coding of 3D videos.However, with the application of high definition and ultra high-definition video, use
Amount amount is continuously increased, and the video data volume has been continuously increased unavoidably, and the channel width of transmission of video be it is limited,
It is therefore desirable to reduce code check using new method.3D-HEVC video coding-decoding methods based on depth resampling are that one kind can
The method of 3D video compression efficiencies is further improved, thus is received significant attention.
The existing 3D-HEVC method for video coding based on depth resampling, its depth Downsapling method often use arest neighbors
Domain Downsapling method (nearest neighbor downsampling), bilinearity Downsapling method (bilinear
) or bicubic Downsapling method (bicubic downsampling) downsampling.These Downsapling methods it is common
Shortcoming is not consider the protection of the depth value of edge pixel during down-sampling, therefore destroys the depth at the edge of original depth-map
Angle value, it so will result in the decline of the depth plot quality after up-sampling.And conventional depth top sampling method, i.e., match simultaneously
The edge of texture maps and depth map comes to depth edge interpolation, so as to reach edge-protected purpose.Their the shortcomings that is algorithm
Complexity it is high, it is difficult to realize that video is handled in real time, and do not consider that the continuity at edge causes the edge for easily producing fracture.
In addition, the characteristics of depth map is the object for splitting each inner flat by significant edge, therefore the Gradient Features of depth map are
Edge pixel has higher gradient, and non-edge pixels has relatively low gradient, can be very good to distinguish using this feature
Edge pixel and non-edge pixels.Therefore the present invention proposes new depth Downsapling method.Gradient map is primarily based on by piecemeal
Depth map be categorized as edge block and non-edge block, median filter method then is used to the different pixels set in these blocks,
The depth value at edge in original depth-map can preferably be retained；When making depth up-sampling simultaneously, in order to recover continuous
Depth edge, using the depth estimation method guided based on gradient map.It is rectangular block first by each pixel-expansion of depth map, so
The depth value of four drift angles of rectangular block is estimated afterwards.Compare the gradient for being expanded pixel and the gradient of other three neighborhood territory pixels is equal
Value, if the former is more than the latter, illustrates that being expanded pixel is different from its neighborhood territory pixel, it should it is edge pixel, its drift angle depth
Value is to be expanded the depth value of pixel；Conversely, it is similar to its neighborhood territory pixel to be expanded pixel, then non-edge pixels is should be, its
Drift angle depth value is the intermediate value for being expanded pixel and other neighborhood territory pixel depth values.Then in level, vertical and diagonal
It is upper to enter row interpolation using the drift angle depth value of valuation.Top sampling method proposed by the present invention Protect edge information and can make well
Obtaining edge has continuity.Further it is proposed that deep video piecemeal down-sampling and extension up-sampling method so that can
Neatly to be sampled according to zoom factor to depth map.Deep video Downsapling method proposed by the present invention and up-sampling side
Method, its algorithm complex is low, is adapted to use in 3D-HEVC encoder and decoder.
The content of the invention
The purpose of the present invention is the deficiency for the existing 3D-HEVC method for video coding based on depth resampling, is carried
Gone out a kind of guiding of gradient map based on depth resampling 3D-HEVC decoding methods.
The inventive method is encoded to texture video by original resolution, and resolution decreasing coding is carried out to deep video.
In order to retain the depth value at edge in original depth-map, for deep video down-sampling, its gradient map is firstly generated, according to contracting
Put the block that coefficient is divided into original gradient figure non-overlapping copies.Depth map is divided into edge block and Fei Bian under gradient map guiding
Edge block, the depth value of down-sampling finally is obtained using median filter method to the different pixels set in these blocks.In order to obtain
Continuous edge, up-sampled for deep video, firstly generate its gradient map, it is then according to zoom factor that low resolution is deep
The each pixel-expansion spent in figure is rectangular block, obtains up-sampling the template of depth map.Then four drift angles of rectangular block are estimated
Depth value, compare the average of the gradient for being expanded pixel and other three neighborhood territory pixel gradients, it is right if the former is more than the latter
The drift angle depth value for answering position is the depth value for being expanded pixel；If the latter is more than the former, the drift angle depth value of correspondence position
To be expanded the intermediate value of the depth value of pixel and other three neighborhood territory pixels.Then, the drift angle pair formed using four drift angles
Average depth value, to level, vertically, the blank pixel on diagonal enters row interpolation.Remaining blank pixel uses its most low coverage
From interpolation depth value.Finally with the texture video and up-sampling typically obtained based on depth map Rendering algorithms to decoding
Deep video carries out View Synthesis, obtains required multi-view point video.
The present invention includes coding method and coding/decoding method.
Coding method comprises the concrete steps that：
Step (1), texture video encoded with original resolution, inputted by the reference sequence of coding profile
3D-HEVC encoders；
Step (2), the piecemeal median filter method progress down-sampling by deep video using gradient map guiding, are then fed into
3D-HEVC encoders；
The bit stream of step (3), the bit stream that texture video is encoded and deep video coding is closed by reference encoder order
Exported after and；
Coding/decoding method comprises the concrete steps that：
Step (I), the bit stream of Video coding decoded, be texture video and depth by reference encoder Sequential output
Video；
Step (II), the deep video by decoding, the neighborhood estimation method guided using gradient map are up-sampled, obtained
The decoding video of size identical with original video；
Step (III), using typically based on depth map Rendering algorithms by the depth of decoded texture video and up-sampling
Video carries out View Synthesis, obtains required multi-view point video.
The present invention has the beneficial effect that：
(1) present invention is used based on the method for partition of zoom factor to deep video resampling, therefore is gone for not
Same sample rate；(2) the depth Downsapling method in 3D-HEVC method for video coding proposed by the present invention is according to depth map edge
The Gradient Features of block and non-edge block, retain the marginal value in original depth-map in the method for medium filtering.(3) present invention proposes
3D-HEVC method for video coding in depth top sampling method according to the Gradient Features of adjacent pixel in depth map, adopted in estimation
The depth value of four drift angles of block of pixels is each extended in sample depth map template, then makees level, vertically, diagonal in block respectively
On depth interpolation, the depth value of rest of pixels is finally obtained according to the method for nearest-neighbor.Up-sampling can so be kept
The continuity at depth map edge, reduce divergent margin point.
Brief description of the drawings
Fig. 1 is coding and decoding methods flow chart in the present invention；
Fig. 2 is the flow chart of depth map Downsapling method in the present invention；
Fig. 3 is the schematic diagram of depth map Downsapling method in the present invention；
Fig. 4 is the schematic diagram of depth map top sampling method in the present invention.
Embodiment
The present invention is further illustrated with reference to the accompanying drawings and examples.
Gradient map guiding based on depth resampling 3D-HEVC decoding methods, using 3D Video coding HTM models, survey
Try conditioned reference JCT-3V universal test condition (JCT3V-E1100), the multiple views of 2 viewpoints carried using HTM models
Encode the configuration file baseCfg_2view.cfg of (MV-HEVC), down-sampling and up-sampling zoom factor s=4.
Specific implementation step is referring to the drawings 1~4.
The specific coding method of 3D-HEVC videos is as shown in Figure 1：
Step (1), texture video encoded with original resolution, inputted by the reference sequence of coding profile
3D-HEVC encoders.
Step (2), the piecemeal median filter method progress down-sampling by deep video using gradient map guiding, are then fed into
3D-HEVC encoders.As shown in Fig. 2 its step is：
A. for every two field picture of deep video, the horizontal direction of each pixel and vertical is generated with Sobel operators first
Gradient on direction：
Wherein, GhAnd GvIt is horizontal and vertical gradient respectively.Then the Grad of each pixel is：
B. as shown in figure 3, depth map is divided into the block that the sizes of non-overlapping copies is 4 × 4, the then depth map after piecemeal
For：
GB(x, y)=G (4x, 4y) (3)
Wherein x, y are piecemeal depth map GBThe centre coordinate of (x, y).
C. all block B (x, y) are divided into edge block Be(x, y) and non-edge block Bne(x,y):
Wherein, p=0.5, Ntotal(x, y) is the number of all pixels in a block, Nh(x, y) is inside block B (x, y)
The pixel with high gradient value number, be calculated as follows；
Wherein, λ=2, corresponding to zoom factor s=4.GB(i, j) is the gradient of pixel (i, j) inside block B (x, y), GBavg
(x, y) is block B (x, y) average gradient.
D. corresponding with the gradient map of piecemeal, depth map is divided into non-overlapped edge block and non-edge block.For side
Edge block Be(x, y), the intermediate value for calculating it are:
Wherein, { DBh(i, j) } represent inside block B (x, y) there is high gradient GBhThe pixel set of (x, y), it meets following
Condition：
DBh(i,j):GBh(i,j)≥λGBavg(x,y) (7)
Wherein, λ GBavgThe same formula of definition (5) of (x, y).
For non-edge block Bne(x, y), the intermediate value for calculating it are：
Wherein, { DB(i, j) } represent the set of all pixels depth value inside block B (x, y)；The result of formula (6) and formula (8)
Constitute the value of the depth map of down-sampling.
The bit stream of step (3), the bit stream that texture video is encoded and deep video coding is closed by reference encoder order
Exported after and.
The specific coding/decoding method of 3D-HEVC videos is as shown in Figure 1：
Step (I), the bit stream of Video coding decoded, be texture video and depth by reference encoder Sequential output
Video；
Step (II), the deep video by decoding, the neighborhood estimation method guided using gradient map are up-sampled, obtained
The decoding video of size identical with original video.As shown in figure 4, left figure is the depth map of the resolution decreasing obtained after decoding, it is right
Figure is the block that each pixel-expansion of left figure is 4 × 4 sizes, so as to the depth map template up-sampled.Left figure pixel a expands
The rectangular block for 4 × 4 sizes being made up of for right figure A1~A12 pixels is opened up, its valuation step is：
1. the gradient map of depth map is generated according to step (a)；
2. first from four drift angles A1, A4, A13 and A16 of left figure estimation right figure rectangular block value.If a coordinate for (x,
Y), then A1 coordinate is (4 (x-1)+Isosorbide-5-Nitrae (y-1)+1), and its value is estimated by { e, f, h, a }；A4 coordinate for (4 (x-1)+1,
4y), its value is estimated by { f, g, a, b }；A13 coordinate is (4x, 4 (y-1)+1), and its value is estimated by { h, a, i, c }；A16
Coordinate be (4x, 4y), its value is estimated by { a, b, c, d }.The method of this four drift angle valuations is similar, below with A16
Exemplified by be explained.If pixel a depth value is D (x, y) in former depth map, Grad is G (x, y), its three neighborhood pictures
Plain b, c, d depth value is respectively D (x, y+1), D (x+1, y), D (x+1, y+1), and the average of their gradients is Gavg(x,y).Note
Dmedian(x, y) is the intermediate value for gathering { D (x, y), D (x, y+1), D (x+1, y), D (x+1, y+1) }, then the depth of the estimation of pixel A 16
Angle value D (4x, 4y) is：
Here parameter ε=1.
3. after the depth value that four drift angles of block are 2. obtained by step, the blank picture in horizontal direction between A1 and A4
Plain A2 and A3 valuations are：
Blank pixel A14 and A15 valuation in horizontal direction between A13 and A16 is：
Blank pixel A5 and A9 valuation in vertical direction between A1 and A13 is：
Blank pixel A8 and A12 valuation in vertical direction between A4 and A16 is：
Blank pixel A7 and A10 valuation between 45 ° of diagonally adjacent A4 and A13 is：
Blank pixel A6 and A11 valuation between -45 ° of diagonally adjacent A1 and A16 is：
Remaining blank pixel uses the depth value of the interpolating pixel away from its nearest neighbours.
Step (III), using typically based on depth map Rendering algorithms by the depth of decoded texture video and up-sampling
Video carries out View Synthesis, obtains required multi-view point video.
Claims (2)
1. gradient map guiding based on depth resampling 3D-HEVC decoding methods, it is characterised in that：This method is to texture video
Encoded by original resolution, resolution decreasing coding is carried out to deep video；In piecemeal of the coding side using gradient map guiding
The method of value filtering is to deep video down-sampling；Decoding end is using the method for the neighborhood valuation of gradient map guiding on deep video
Sampling；The texture video and up-sampling deep video finally obtained to decoding carries out View Synthesis, obtains required multiple views and regards
Frequently；
Coding method comprises the concrete steps that：
Step (1), texture video encoded with original resolution, 3D- is inputted by the reference sequence of coding profile
HEVC encoders；
Step (2), the piecemeal median filter method progress down-sampling by deep video using gradient map guiding, are then fed into 3D-
HEVC encoders；
After the bit stream of step (3), the bit stream that texture video is encoded and deep video coding is merged by reference encoder order
Output；
Coding/decoding method comprises the concrete steps that：
Step (4), the bit stream of Video coding decoded, be that texture video and depth regard by reference encoder Sequential output
Frequently；
Step (5), the deep video by decoding, the neighborhood estimation method guided using gradient map are up-sampled, and are obtained and original
The decoding video of the identical size of beginning video；
Step (6), by the deep video of decoded texture video and up-sampling carry out View Synthesis, obtain required multiple views
Video；
The piecemeal median filter method of gradient map guiding described in step (2) carries out down-sampling, comprises the following steps that：
A. for every two field picture of deep video, firstly generate each pixel horizontally and vertically on gradient：
<mrow>
<mo>&dtri;</mo>
<mi>G</mi>
<mo>=</mo>
<mrow>
<mo>(</mo>
<msub>
<mi>G</mi>
<mi>h</mi>
</msub>
<mo>,</mo>
<msub>
<mi>G</mi>
<mi>v</mi>
</msub>
<mo>)</mo>
</mrow>
<mo>-</mo>
<mo>-</mo>
<mo>-</mo>
<mrow>
<mo>(</mo>
<mn>1</mn>
<mo>)</mo>
</mrow>
</mrow>
Wherein, GhAnd GvIt is horizontally oriented respectively and the gradient in vertical direction；Then the Grad of each pixel is：
<mrow>
<mi>G</mi>
<mo>=</mo>
<msqrt>
<mrow>
<msubsup>
<mi>G</mi>
<mi>h</mi>
<mn>2</mn>
</msubsup>
<mo>+</mo>
<msubsup>
<mi>G</mi>
<mi>v</mi>
<mn>2</mn>
</msubsup>
</mrow>
</msqrt>
<mo>-</mo>
<mo>-</mo>
<mo>-</mo>
<mrow>
<mo>(</mo>
<mn>2</mn>
<mo>)</mo>
</mrow>
</mrow>
B. zoom factor is set as s, is s × s block by the size that depth map is divided into non-overlapping copies, then the depth map after piecemeal
For：
GB(x, y)=G (sx, sy) (3)
Wherein x, y are piecemeal depth map GBThe centre coordinate of (x, y)；G (sx, sy) is the coordinate of former depth map；
C. all block B (x, y) that depth map divides are divided into edge block Be(x, y) and non-edge block Bne(x,y):
<mrow>
<mi>B</mi>
<mrow>
<mo>(</mo>
<mi>x</mi>
<mo>,</mo>
<mi>y</mi>
<mo>)</mo>
</mrow>
<mo>=</mo>
<mfenced open = "{" close = "">
<mtable>
<mtr>
<mtd>
<mrow>
<msub>
<mi>B</mi>
<mi>e</mi>
</msub>
<mrow>
<mo>(</mo>
<mi>x</mi>
<mo>,</mo>
<mi>y</mi>
<mo>)</mo>
</mrow>
<mo>,</mo>
</mrow>
</mtd>
<mtd>
<mrow>
<mi>i</mi>
<mi>f</mi>
<mi> </mi>
<msub>
<mi>N</mi>
<mi>h</mi>
</msub>
<mrow>
<mo>(</mo>
<mi>x</mi>
<mo>,</mo>
<mi>y</mi>
<mo>)</mo>
</mrow>
<mo>></mo>
<mi>p</mi>
<mo>*</mo>
<msub>
<mi>N</mi>
<mrow>
<mi>t</mi>
<mi>o</mi>
<mi>t</mi>
<mi>a</mi>
<mi>l</mi>
</mrow>
</msub>
<mrow>
<mo>(</mo>
<mi>x</mi>
<mo>,</mo>
<mi>y</mi>
<mo>)</mo>
</mrow>
</mrow>
</mtd>
</mtr>
<mtr>
<mtd>
<mrow>
<msub>
<mi>B</mi>
<mrow>
<mi>n</mi>
<mi>e</mi>
</mrow>
</msub>
<mrow>
<mo>(</mo>
<mi>x</mi>
<mo>,</mo>
<mi>y</mi>
<mo>)</mo>
</mrow>
<mo>,</mo>
</mrow>
</mtd>
<mtd>
<mrow>
<mi>o</mi>
<mi>t</mi>
<mi>h</mi>
<mi>e</mi>
<mi>r</mi>
<mi>w</mi>
<mi>i</mi>
<mi>s</mi>
<mi>e</mi>
</mrow>
</mtd>
</mtr>
</mtable>
</mfenced>
<mo>-</mo>
<mo>-</mo>
<mo>-</mo>
<mrow>
<mo>(</mo>
<mn>4</mn>
<mo>)</mo>
</mrow>
</mrow>
Wherein, p is percentage parameter, Ntotal(x, y) is the number of all pixels in a block, Nh(x, y) is block B (x, y)
The number of the pixel with high gradient value of the inside, is calculated as follows：
<mrow>
<mtable>
<mtr>
<mtd>
<mrow>
<msub>
<mi>N</mi>
<mi>h</mi>
</msub>
<mrow>
<mo>(</mo>
<mi>x</mi>
<mo>,</mo>
<mi>y</mi>
<mo>)</mo>
</mrow>
<mo>=</mo>
<munder>
<mo>&Sigma;</mo>
<mrow>
<mo>(</mo>
<mi>i</mi>
<mo>,</mo>
<mi>j</mi>
<mo>)</mo>
<mo>&Element;</mo>
<mi>B</mi>
<mo>(</mo>
<mi>x</mi>
<mo>,</mo>
<mi>y</mi>
<mo>)</mo>
</mrow>
</munder>
<mi>&rho;</mi>
<mo>&lsqb;</mo>
<msub>
<mi>G</mi>
<mi>B</mi>
</msub>
<mrow>
<mo>(</mo>
<mi>i</mi>
<mo>,</mo>
<mi>j</mi>
<mo>)</mo>
</mrow>
<mo>,</mo>
<msub>
<mi>G</mi>
<mrow>
<mi>B</mi>
<mi>a</mi>
<mi>v</mi>
<mi>g</mi>
</mrow>
</msub>
<mrow>
<mo>(</mo>
<mi>x</mi>
<mo>,</mo>
<mi>y</mi>
<mo>)</mo>
</mrow>
<mo>&rsqb;</mo>
</mrow>
</mtd>
</mtr>
<mtr>
<mtd>
<mrow>
<mi>w</mi>
<mi>i</mi>
<mi>t</mi>
<mi>h</mi>
<mi>&rho;</mi>
<mo>&lsqb;</mo>
<msub>
<mi>G</mi>
<mi>B</mi>
</msub>
<mrow>
<mo>(</mo>
<mi>i</mi>
<mo>,</mo>
<mi>j</mi>
<mo>)</mo>
</mrow>
<mo>,</mo>
<msub>
<mi>G</mi>
<mrow>
<mi>B</mi>
<mi>a</mi>
<mi>v</mi>
<mi>g</mi>
</mrow>
</msub>
<mrow>
<mo>(</mo>
<mi>x</mi>
<mo>,</mo>
<mi>y</mi>
<mo>)</mo>
</mrow>
<mo>&rsqb;</mo>
<mo>=</mo>
<mfenced open = "{" close = "">
<mtable>
<mtr>
<mtd>
<mrow>
<mn>1</mn>
<mo>,</mo>
</mrow>
</mtd>
<mtd>
<mrow>
<mi>i</mi>
<mi>f</mi>
<mi> </mi>
<msub>
<mi>G</mi>
<mi>B</mi>
</msub>
<mrow>
<mo>(</mo>
<mi>i</mi>
<mo>,</mo>
<mi>j</mi>
<mo>)</mo>
</mrow>
<mo>&GreaterEqual;</mo>
<msub>
<mi>&lambda;G</mi>
<mrow>
<mi>B</mi>
<mi>a</mi>
<mi>v</mi>
<mi>g</mi>
</mrow>
</msub>
<mrow>
<mo>(</mo>
<mi>x</mi>
<mo>,</mo>
<mi>y</mi>
<mo>)</mo>
</mrow>
</mrow>
</mtd>
</mtr>
<mtr>
<mtd>
<mrow>
<mn>0</mn>
<mo>,</mo>
</mrow>
</mtd>
<mtd>
<mrow>
<mi>o</mi>
<mi>t</mi>
<mi>h</mi>
<mi>e</mi>
<mi>r</mi>
<mi>w</mi>
<mi>i</mi>
<mi>s</mi>
<mi>e</mi>
</mrow>
</mtd>
</mtr>
</mtable>
</mfenced>
</mrow>
</mtd>
</mtr>
</mtable>
<mo>-</mo>
<mo>-</mo>
<mo>-</mo>
<mrow>
<mo>(</mo>
<mn>5</mn>
<mo>)</mo>
</mrow>
</mrow>
Wherein, λ is the exponent number of down-sampling, and λ=1,2,3 correspond to zoom factor for 2,4,8；GB(i, j) is picture inside block B (x, y)
The gradient of plain (i, j), GBavg(x, y) is block B (x, y) average gradient；
D. corresponding with the gradient map of piecemeal, depth map is divided into non-overlapped edge block and non-edge block；For edge block Be
(x, y), the intermediate value for calculating it are：
<mrow>
<msub>
<mi>D</mi>
<mrow>
<mi>d</mi>
<mi>o</mi>
<mi>w</mi>
<mi>n</mi>
</mrow>
</msub>
<mrow>
<mo>(</mo>
<mi>x</mi>
<mo>,</mo>
<mi>y</mi>
<mo>)</mo>
</mrow>
<mo>=</mo>
<munder>
<mrow>
<mi>m</mi>
<mi>e</mi>
<mi>d</mi>
<mi>i</mi>
<mi>a</mi>
<mi>n</mi>
</mrow>
<mrow>
<mo>(</mo>
<mi>i</mi>
<mo>,</mo>
<mi>j</mi>
<mo>)</mo>
<mo>&Element;</mo>
<mi>B</mi>
<mo>(</mo>
<mi>x</mi>
<mo>,</mo>
<mi>y</mi>
<mo>)</mo>
</mrow>
</munder>
<mo>{</mo>
<msub>
<mi>D</mi>
<mrow>
<mi>B</mi>
<mi>h</mi>
</mrow>
</msub>
<mrow>
<mo>(</mo>
<mi>i</mi>
<mo>,</mo>
<mi>j</mi>
<mo>)</mo>
</mrow>
<mo>}</mo>
<mo>-</mo>
<mo>-</mo>
<mo>-</mo>
<mrow>
<mo>(</mo>
<mn>6</mn>
<mo>)</mo>
</mrow>
</mrow>
Wherein, { DBh(i, j) } represent inside block B (x, y) there is high gradient GBhThe pixel set of (x, y), it meets following condition：
DBh(i,j):GBh(i,j)≥λGBavg(x,y) (7)
Wherein, λ GBavgThe same formula of definition (5) of (x, y)；
For non-edge block Bne(x, y), the intermediate value for calculating it are：
<mrow>
<msub>
<mi>D</mi>
<mrow>
<mi>d</mi>
<mi>o</mi>
<mi>w</mi>
<mi>n</mi>
</mrow>
</msub>
<mrow>
<mo>(</mo>
<mi>x</mi>
<mo>,</mo>
<mi>y</mi>
<mo>)</mo>
</mrow>
<mo>=</mo>
<munder>
<mrow>
<mi>m</mi>
<mi>e</mi>
<mi>d</mi>
<mi>i</mi>
<mi>a</mi>
<mi>n</mi>
</mrow>
<mrow>
<mo>(</mo>
<mi>i</mi>
<mo>,</mo>
<mi>j</mi>
<mo>)</mo>
<mo>&Element;</mo>
<mi>B</mi>
<mo>(</mo>
<mi>x</mi>
<mo>,</mo>
<mi>y</mi>
<mo>)</mo>
</mrow>
</munder>
<mo>{</mo>
<msub>
<mi>D</mi>
<mi>B</mi>
</msub>
<mrow>
<mo>(</mo>
<mi>i</mi>
<mo>,</mo>
<mi>j</mi>
<mo>)</mo>
</mrow>
<mo>}</mo>
<mo>-</mo>
<mo>-</mo>
<mo>-</mo>
<mrow>
<mo>(</mo>
<mn>8</mn>
<mo>)</mo>
</mrow>
</mrow>
Wherein, { DB(i, j) } represent the set of all pixels depth value inside block B (x, y)；The result of formula (6) and formula (8) is formed
The value of the depth map of down-sampling；
The neighborhood estimation method using gradient map guiding described in step (5) is up-sampled, and is comprised the following steps that：
1. according to the gradient map of the generation depth map of step (a)；
2. zoom factor is set as s, by the block that each interpolation pixel-expansion is s × s sizes, the depth artwork after being up-sampled
Plate；
3. a pixel coordinate of the depth map of decoding output is set as (x, y), then in its depth map template after up-sampling
The coordinate of top left corner pixel is (s (x-1)+1, s (y-1)+1), and the coordinate of upper right corner pixel is (s (x-1)+1, sy), the lower left corner
The coordinate of pixel is (sx, s (y-1)+1), and the coordinate of lower right corner pixel is (sx, sy)；To the upper left corner, the upper right corner, the lower left corner and
4, lower right corner pixel enters row interpolation using the method for neighborhood valuation, specific as follows：
If the coordinate of pixel to be valuated is (sx, sy), the depth value of (x, y) is D (x, y) in depth map, Grad be G (x,
y)；Then three neighborhood territory pixels (x, y+1) of the pixel, (x+1, y), the depth value of (x+1, y+1) are respectively D (x, y+1), D (x+
1, y), D (x+1, y+1), gradient mean value Gavg(x,y)；Remember Dmedian(x, y) be set D (x, y), D (x, y+1), D (x+1,
Y), D (x+1, y+1) } intermediate value, then the depth value D (sx, sy) estimated is：
<mrow>
<mi>D</mi>
<mrow>
<mo>(</mo>
<mi>s</mi>
<mi>x</mi>
<mo>,</mo>
<mi>s</mi>
<mi>y</mi>
<mo>)</mo>
</mrow>
<mo>=</mo>
<mfenced open = "{" close = "">
<mtable>
<mtr>
<mtd>
<mrow>
<mi>D</mi>
<mrow>
<mo>(</mo>
<mi>x</mi>
<mo>,</mo>
<mi>y</mi>
<mo>)</mo>
</mrow>
<mo>,</mo>
</mrow>
</mtd>
<mtd>
<mrow>
<mi>i</mi>
<mi>f</mi>
<mi> </mi>
<mi>G</mi>
<mrow>
<mo>(</mo>
<mi>x</mi>
<mo>,</mo>
<mi>y</mi>
<mo>)</mo>
</mrow>
<mo>&GreaterEqual;</mo>
<msub>
<mi>&epsiv;G</mi>
<mrow>
<mi>a</mi>
<mi>v</mi>
<mi>g</mi>
</mrow>
</msub>
<mrow>
<mo>(</mo>
<mi>x</mi>
<mo>,</mo>
<mi>y</mi>
<mo>)</mo>
</mrow>
</mrow>
</mtd>
</mtr>
<mtr>
<mtd>
<mrow>
<msub>
<mi>D</mi>
<mrow>
<mi>m</mi>
<mi>e</mi>
<mi>d</mi>
<mi>i</mi>
<mi>a</mi>
<mi>n</mi>
</mrow>
</msub>
<mrow>
<mo>(</mo>
<mi>x</mi>
<mo>,</mo>
<mi>y</mi>
<mo>)</mo>
</mrow>
<mo>,</mo>
</mrow>
</mtd>
<mtd>
<mrow>
<mi>o</mi>
<mi>t</mi>
<mi>h</mi>
<mi>e</mi>
<mi>r</mi>
<mi>w</mi>
<mi>i</mi>
<mi>s</mi>
<mi>e</mi>
</mrow>
</mtd>
</mtr>
</mtable>
</mfenced>
<mo>-</mo>
<mo>-</mo>
<mo>-</mo>
<mrow>
<mo>(</mo>
<mn>9</mn>
<mo>)</mo>
</mrow>
</mrow>
Here parameter ε=1；
4. the depth value of four drift angles of block is 3. obtained by step, (s (x-1)+1, s (y-1)+1) and (s in horizontal direction
(x-1)+1, sy) between blank pixel valuation be：
<mrow>
<msub>
<mi>D</mi>
<mn>1</mn>
</msub>
<mo>=</mo>
<mfrac>
<mn>1</mn>
<mn>2</mn>
</mfrac>
<mo>&lsqb;</mo>
<mi>D</mi>
<mrow>
<mo>(</mo>
<mi>s</mi>
<mo>(</mo>
<mrow>
<mi>x</mi>
<mo>-</mo>
<mn>1</mn>
</mrow>
<mo>)</mo>
<mo>+</mo>
<mn>1</mn>
<mo>,</mo>
<mi>s</mi>
<mo>(</mo>
<mrow>
<mi>y</mi>
<mo>-</mo>
<mn>1</mn>
</mrow>
<mo>)</mo>
<mo>+</mo>
<mn>1</mn>
<mo>)</mo>
</mrow>
<mo>+</mo>
<mi>D</mi>
<mrow>
<mo>(</mo>
<mi>s</mi>
<mo>(</mo>
<mrow>
<mi>x</mi>
<mo>-</mo>
<mn>1</mn>
</mrow>
<mo>)</mo>
<mo>+</mo>
<mn>1</mn>
<mo>,</mo>
<mi>s</mi>
<mi>y</mi>
<mo>)</mo>
</mrow>
<mo>&rsqb;</mo>
<mo>-</mo>
<mo>-</mo>
<mo>-</mo>
<mrow>
<mo>(</mo>
<mn>10</mn>
<mo>)</mo>
</mrow>
</mrow>
The blank pixel valuation of (sx, s (y-1)+1) between (sx, sy) is in horizontal direction：
<mrow>
<msub>
<mi>D</mi>
<mn>2</mn>
</msub>
<mo>=</mo>
<mfrac>
<mn>1</mn>
<mn>2</mn>
</mfrac>
<mo>&lsqb;</mo>
<mi>D</mi>
<mrow>
<mo>(</mo>
<mi>s</mi>
<mi>x</mi>
<mo>,</mo>
<mi>s</mi>
<mo>(</mo>
<mrow>
<mi>y</mi>
<mo>-</mo>
<mn>1</mn>
</mrow>
<mo>)</mo>
<mo>+</mo>
<mn>1</mn>
<mo>)</mo>
</mrow>
<mo>+</mo>
<mi>D</mi>
<mrow>
<mo>(</mo>
<mi>s</mi>
<mi>x</mi>
<mo>,</mo>
<mi>s</mi>
<mi>y</mi>
<mo>)</mo>
</mrow>
<mo>&rsqb;</mo>
<mo>-</mo>
<mo>-</mo>
<mo>-</mo>
<mo>-</mo>
<mrow>
<mo>(</mo>
<mn>11</mn>
<mo>)</mo>
</mrow>
</mrow>
Blank pixel valuation in vertical direction between (s (x-1)+1, s (y-1)+1) and (sx, s (y-1)+1) is：
<mrow>
<msub>
<mi>D</mi>
<mn>3</mn>
</msub>
<mo>=</mo>
<mfrac>
<mn>1</mn>
<mn>2</mn>
</mfrac>
<mo>&lsqb;</mo>
<mi>D</mi>
<mrow>
<mo>(</mo>
<mi>s</mi>
<mo>(</mo>
<mrow>
<mi>x</mi>
<mo>-</mo>
<mn>1</mn>
</mrow>
<mo>)</mo>
<mo>+</mo>
<mn>1</mn>
<mo>,</mo>
<mi>s</mi>
<mo>(</mo>
<mrow>
<mi>y</mi>
<mo>-</mo>
<mn>1</mn>
</mrow>
<mo>)</mo>
<mo>+</mo>
<mn>1</mn>
<mo>)</mo>
</mrow>
<mo>+</mo>
<mi>D</mi>
<mrow>
<mo>(</mo>
<mi>s</mi>
<mi>x</mi>
<mo>,</mo>
<mi>s</mi>
<mo>(</mo>
<mrow>
<mi>y</mi>
<mo>-</mo>
<mn>1</mn>
</mrow>
<mo>)</mo>
<mo>+</mo>
<mn>1</mn>
<mo>)</mo>
</mrow>
<mo>&rsqb;</mo>
<mo>-</mo>
<mo>-</mo>
<mo>-</mo>
<mrow>
<mo>(</mo>
<mn>12</mn>
<mo>)</mo>
</mrow>
</mrow>
The blank pixel valuation of (s (x-1)+1, sy) between (sx, sy) is in vertical direction：
<mrow>
<msub>
<mi>D</mi>
<mn>4</mn>
</msub>
<mo>=</mo>
<mfrac>
<mn>1</mn>
<mn>2</mn>
</mfrac>
<mo>&lsqb;</mo>
<mi>D</mi>
<mrow>
<mo>(</mo>
<mi>s</mi>
<mo>(</mo>
<mrow>
<mi>x</mi>
<mo>-</mo>
<mn>1</mn>
</mrow>
<mo>)</mo>
<mo>+</mo>
<mn>1</mn>
<mo>,</mo>
<mi>s</mi>
<mi>y</mi>
<mo>)</mo>
</mrow>
<mo>+</mo>
<mi>D</mi>
<mrow>
<mo>(</mo>
<mi>s</mi>
<mi>x</mi>
<mo>,</mo>
<mi>s</mi>
<mi>y</mi>
<mo>)</mo>
</mrow>
<mo>&rsqb;</mo>
<mo>-</mo>
<mo>-</mo>
<mo>-</mo>
<mrow>
<mo>(</mo>
<mn>13</mn>
<mo>)</mo>
</mrow>
</mrow>
It is positioned at the 45 ° of diagonally adjacent blank pixel valuations of (s (x-1)+1, sy) between (sx, s (y-1)+1)：
<mrow>
<msub>
<mi>D</mi>
<mn>5</mn>
</msub>
<mo>=</mo>
<mfrac>
<mn>1</mn>
<mn>2</mn>
</mfrac>
<mo>&lsqb;</mo>
<mi>D</mi>
<mrow>
<mo>(</mo>
<mi>s</mi>
<mo>(</mo>
<mrow>
<mi>x</mi>
<mo>-</mo>
<mn>1</mn>
</mrow>
<mo>)</mo>
<mo>+</mo>
<mn>1</mn>
<mo>,</mo>
<mi>s</mi>
<mi>y</mi>
<mo>)</mo>
</mrow>
<mo>+</mo>
<mi>D</mi>
<mrow>
<mo>(</mo>
<mi>s</mi>
<mi>x</mi>
<mo>,</mo>
<mi>s</mi>
<mo>(</mo>
<mrow>
<mi>y</mi>
<mo>-</mo>
<mn>1</mn>
</mrow>
<mo>)</mo>
<mo>+</mo>
<mn>1</mn>
<mo>)</mo>
</mrow>
<mo>&rsqb;</mo>
<mo>-</mo>
<mo>-</mo>
<mo>-</mo>
<mrow>
<mo>(</mo>
<mn>14</mn>
<mo>)</mo>
</mrow>
</mrow>
It is positioned at -45 ° of diagonally adjacent blank pixel valuations of (s (x-1)+1, s (y-1)+1) between (sx, sy)：
<mrow>
<msub>
<mi>D</mi>
<mn>6</mn>
</msub>
<mo>=</mo>
<mfrac>
<mn>1</mn>
<mn>2</mn>
</mfrac>
<mo>&lsqb;</mo>
<mi>D</mi>
<mrow>
<mo>(</mo>
<mi>s</mi>
<mo>(</mo>
<mrow>
<mi>x</mi>
<mo>-</mo>
<mn>1</mn>
</mrow>
<mo>)</mo>
<mo>+</mo>
<mn>1</mn>
<mo>,</mo>
<mi>s</mi>
<mo>(</mo>
<mrow>
<mi>y</mi>
<mo>-</mo>
<mn>1</mn>
</mrow>
<mo>)</mo>
<mo>+</mo>
<mn>1</mn>
<mo>)</mo>
</mrow>
<mo>+</mo>
<mi>D</mi>
<mrow>
<mo>(</mo>
<mi>s</mi>
<mi>x</mi>
<mo>,</mo>
<mi>s</mi>
<mi>y</mi>
<mo>)</mo>
</mrow>
<mo>&rsqb;</mo>
<mo>-</mo>
<mo>-</mo>
<mo>-</mo>
<mrow>
<mo>(</mo>
<mn>15</mn>
<mo>)</mo>
</mrow>
</mrow>
Remaining blank pixel uses the depth value of the interpolating pixel away from its nearest neighbours.
2. gradient map as claimed in claim 1 guiding based on depth resampling 3D-HEVC decoding methods, it is characterised in that
View Synthesis described in step (6), which uses, is typically based on depth map rendering intent.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510608870.1A CN105163129B (en) | 2015-09-22 | 2015-09-22 | Gradient map guiding based on depth resampling 3D HEVC decoding methods |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510608870.1A CN105163129B (en) | 2015-09-22 | 2015-09-22 | Gradient map guiding based on depth resampling 3D HEVC decoding methods |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105163129A CN105163129A (en) | 2015-12-16 |
| CN105163129B true CN105163129B (en) | 2018-01-23 |
Family
ID=54803869
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510608870.1A Active CN105163129B (en) | 2015-09-22 | 2015-09-22 | Gradient map guiding based on depth resampling 3D HEVC decoding methods |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105163129B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105635742B (en) * | 2015-12-30 | 2018-09-07 | 杭州电子科技大学 | The method for resampling based on depth enhancing towards 3D Video codings |
| CN108111851B (en) * | 2016-11-25 | 2020-12-22 | 华为技术有限公司 | Deblocking filtering method and terminal |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103581687A (en) * | 2013-09-11 | 2014-02-12 | 北京交通大学长三角研究院 | Self-adaptive depth image coding method based on compressed sensing |
| CN103905812A (en) * | 2014-03-27 | 2014-07-02 | 北京工业大学 | Texture/depth combination up-sampling method |
| CN104537627A (en) * | 2015-01-08 | 2015-04-22 | 北京交通大学 | Depth image post-processing method |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8643701B2 (en) * | 2009-11-18 | 2014-02-04 | University Of Illinois At Urbana-Champaign | System for executing 3D propagation for depth image-based rendering |
-
2015
- 2015-09-22 CN CN201510608870.1A patent/CN105163129B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103581687A (en) * | 2013-09-11 | 2014-02-12 | 北京交通大学长三角研究院 | Self-adaptive depth image coding method based on compressed sensing |
| CN103905812A (en) * | 2014-03-27 | 2014-07-02 | 北京工业大学 | Texture/depth combination up-sampling method |
| CN104537627A (en) * | 2015-01-08 | 2015-04-22 | 北京交通大学 | Depth image post-processing method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105163129A (en) | 2015-12-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP3816928A1 (en) | Image super-resolution reconstruction method, image super-resolution reconstruction apparatus, and computer-readable storage medium | |
| US9736455B2 (en) | Method and apparatus for downscaling depth data for view plus depth data compression | |
| EP3692510B1 (en) | A method and apparatus for reconstructing a point cloud representing a 3d object | |
| US9525858B2 (en) | Depth or disparity map upscaling | |
| CN100576934C (en) | Virtual visual point synthesizing method based on the degree of depth and block information | |
| KR100806201B1 (en) | Generating method for three-dimensional video formation using hierarchical decomposition of depth image, and device for the same, and system and storage medium therefor | |
| CN104156957B (en) | Stable and high-efficiency high-resolution stereo matching method | |
| US20220108483A1 (en) | Video based mesh compression | |
| CN103077542B (en) | A kind of compression method for interest region of depth map | |
| EP2747427B1 (en) | Method, apparatus and computer program usable in synthesizing a stereoscopic image | |
| CN105141940B (en) | A kind of subregional 3D method for video coding | |
| CN103763564B (en) | Depth map encoding method based on edge lossless compress | |
| CN103606136B (en) | Based on the video super resolution of key frame and non local constraint | |
| CN105163129B (en) | Gradient map guiding based on depth resampling 3D HEVC decoding methods | |
| CN103971354A (en) | Method for reconstructing low-resolution infrared image into high-resolution infrared image | |
| CN104780383B (en) | A kind of 3D HEVC multi-resolution video coding methods | |
| Bidgoli et al. | OSLO: On-the-Sphere Learning for Omnidirectional images and its application to 360-degree image compression | |
| CN109345444B (en) | Super-resolution stereoscopic image construction method with enhanced depth perception | |
| CN103905812A (en) | Texture/depth combination up-sampling method | |
| EP2775723A1 (en) | Method, apparatus and computer program for generating a multiview image-plus-depth format | |
| Hu et al. | Depth map super-resolution using synthesized view matching for depth-image-based rendering | |
| CN112634127A (en) | Unsupervised stereo image redirection method | |
| Hung et al. | Dual edge-confined inpainting of 3D depth map using color image's edges and depth image's edges | |
| CN104394399B (en) | Three limit filtering methods of deep video coding | |
| Graziosi et al. | Depth map up-sampling based on edge layers |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20201217 Address after: Room 3003-1, building 1, Gaode land center, Jianggan District, Hangzhou City, Zhejiang Province Patentee after: Zhejiang Zhiduo Network Technology Co.,Ltd. Address before: 310018 No. 2 street, Xiasha Higher Education Zone, Hangzhou, Zhejiang Patentee before: HANGZHOU DIANZI University Effective date of registration: 20201217 Address after: 314500 Tongxiang, Jiaxing, Zhejiang, Wutong Street East Road (East) 55, Tongxiang chamber of Commerce Building 1 unit 1702, 1703 room -A-218 Patentee after: Jiaxing Baoqiao Machinery Technology Co.,Ltd. Address before: Room 3003-1, building 1, Gaode land center, Jianggan District, Hangzhou City, Zhejiang Province Patentee before: Zhejiang Zhiduo Network Technology Co.,Ltd. |
|
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20210423 Address after: 310018 room 205, building 2, dongfangming building, Qiantang New District, Hangzhou City, Zhejiang Province Patentee after: Hangzhou aoguan Network Technology Co.,Ltd. Address before: 314500 Tongxiang, Jiaxing, Zhejiang, Wutong Street East Road (East) 55, Tongxiang chamber of Commerce Building 1 unit 1702, 1703 room -A-218 Patentee before: Jiaxing Baoqiao Machinery Technology Co.,Ltd. |
|
| TR01 | Transfer of patent right |