CN111050180A - Parallel implementation method for depth map parallax estimation - Google Patents
Parallel implementation method for depth map parallax estimation Download PDFInfo
- Publication number
- CN111050180A CN111050180A CN201911271204.8A CN201911271204A CN111050180A CN 111050180 A CN111050180 A CN 111050180A CN 201911271204 A CN201911271204 A CN 201911271204A CN 111050180 A CN111050180 A CN 111050180A
- Authority
- CN
- China
- Prior art keywords
- processing element
- block
- depth map
- parallel
- sad
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
- H04N19/436—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation using parallelised computational arrangements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
- H04N19/51—Motion estimation or motion compensation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/593—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving spatial prediction techniques
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/597—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding specially adapted for multi-view video sequence encoding
Abstract
The invention provides a parallel implementation method for depth map disparity estimation, which comprises the following steps: constructing a parallel structure for depth map disparity estimation based on a 4 x 4 two-dimensional array of contiguously interconnected processing elements; reading the pixel value of the basic viewpoint and the pixel value of the reference viewpoint into the processing element of the parallel structure, performing block matching operation of time domain adjacent blocks and space domain adjacent blocks in parallel, and calculating to obtain SAD (sum of absolute difference) values of the adjacent blocks; and according to the SAD value, selecting an optimal time domain adjacent block and an optimal space domain adjacent block in parallel, and performing motion compensation prediction and parallax compensation prediction. According to the depth map disparity estimation method, a parallel structure is constructed, the depth map disparity estimation algorithm of three-dimensional high-efficiency video coding is realized in a parallel mode, the calculation efficiency of the disparity estimation process is improved, and the depth map coding time is shortened.
Description
Technical Field
The invention relates to the technical field of video processing, in particular to a parallel implementation method of a three-dimensional high-efficiency video coding depth map parallax estimation algorithm.
Background
The parallax refers to the position deviation of the same pixel point in the projection of different viewpoints. Disparity Estimation (DE) is a process of finding spatial correlation of image data between different viewpoints and calculating a Disparity Vector (DV). By adopting the parallax estimation, redundant data among a plurality of viewpoints can be greatly eliminated, and the method is an important link for inter-viewpoint motion prediction, residual prediction, illumination compensation and viewpoint synthesis prediction in three-dimensional video coding.
Three-dimensional High Efficiency Video Coding (3D High Efficiency Video Coding, 3D-HEVC for short) proposed by JCT-3V is designed based on the algorithm basis of the extended portion MVC (Multi-view Video Coding, MVC for short) of H.264/AVC and the H.265/HEVC standard, and a Multi-view Video plus Depth (MVD for short) format is adopted to code texture images and Depth images of multiple views. In 3D-HEVC, Disparity vectors of a depth map are obtained using a Neighboring block Disparity estimation (NBDV) based method. For example, for an N × N base view coding block, the disparity vector acquisition is a search and matching process for 2 temporal neighboring blocks of the base view and 5 spatial neighboring blocks of the reference view. By adopting the block matching mode, the problems that more reference frame information is needed, higher calculation amount and higher data throughput are brought, and the efficiency of video coding is seriously influenced compared with the standard two-dimensional video coding are brought. If an excessively large coding block is selected to reduce the amount of calculation, continuity prediction errors are caused, and view synthesis is affected.
The above drawbacks are expected to be overcome by those skilled in the art.
Therefore, a parallel implementation method for disparity estimation of depth map for three-dimensional efficient video coding is needed.
Disclosure of Invention
Technical problem to be solved
In order to solve the above problems in the prior art, the present invention provides a parallel implementation method for disparity estimation of a depth map, which solves the problem of time consumption in disparity estimation in the prior art.
(II) technical scheme
In order to achieve the purpose, the invention adopts the main technical scheme that:
an embodiment of the present invention provides a parallel implementation method for disparity estimation of a depth map, including the following steps:
s1, constructing a parallel structure for disparity estimation of the three-dimensional high-efficiency video coding depth map based on the 4 x 4 two-dimensional processing element array of the adjacent interconnection;
s2, reading basic viewpoint pixel values and reference viewpoint pixel values into the processing elements of the parallel structure based on the acquired video image;
s3, carrying out block matching operation of time domain adjacent blocks and space domain adjacent blocks in parallel in the processing element of the parallel structure, and calculating to obtain SAD value of each adjacent block;
s4, selecting the time domain minimum SAD value to obtain the optimal time domain adjacent block according to the SAD value, performing motion compensation prediction, selecting the minimum space domain SAD value to obtain the optimal space domain adjacent block, and performing parallax compensation prediction;
and S5, calculating to obtain the disparity vector of the depth map according to the motion compensation prediction result and the disparity compensation prediction result.
In an embodiment of the present invention, before the reading of the base viewpoint pixel value and the reference viewpoint pixel value in step S2, the method further includes:
converting the basic viewpoint image and the reference viewpoint image of the test sequence of the video into binary data which can be identified by the array;
distributing the binary data in a document, and storing the binary data in a data input storage (DIM) as the current pixel value of the basic viewpoint and the current pixel value of the reference viewpoint;
and storing the reconstructed image processed according to the frame before the basic viewpoint in the video in a data output storage (DOM) as the reference pixel value of the basic viewpoint.
In an embodiment of the present invention, the reading the base viewpoint pixel values and the reference viewpoint pixel values into the processing elements of the parallel structure in step S2 includes:
accessing the DIM by processing element PE00 of a first row and a first column in the two-dimensional processing element array in an contiguously interconnected manner to read a corresponding base view current prediction block pixel value in a size of 8 × 8;
accessing the DIM by way of contiguous interconnections through processing element PE10 of a second row and a first column in the two-dimensional processing element array to read 5 corresponding reference view spatial neighboring block pixel values at a size of 10 × 10;
accessing the DOM by processing element PE03 of the first row and fourth column in the two-dimensional array of processing elements in a contiguous interconnect reads 2 corresponding base view temporal neighboring block pixel values at a size of 10 x 10.
In an embodiment of the present invention, after the step S2 is completed, before the step S3 is performed, the method further includes:
the processing element PE00 issues the current prediction block to the data storage addresses [0] - [63] of the processing element PE01 and PE02 in parallel, and issues the current prediction block to the data storage addresses [0] - [63] of the processing element PE11, PE20, PE21, PE30 and PE31 via the processing element PE 10;
the processing element PE03 sequentially issues the 2 basic viewpoint time domain adjacent blocks to data storage addresses [100] - [199] of PE01 and PE 02;
the processing element PE10 issues the 5 reference viewpoint spatial neighboring blocks to data storage addresses [100] - [199] of the processing elements PE11, PE20, PE21, PE30 and PE31 respectively;
the processing elements PE01, PE02, PE11, PE20, PE21, PE30 and PE31 are put in parallel into the same handshake signal after the completion of data transmission.
In one embodiment of the present invention, in step S3, the performing block matching operations on the temporal neighboring blocks and the spatial neighboring blocks in parallel in the processing unit of the parallel structure to calculate the SAD value of each neighboring block includes:
the processing elements PE01 and PE02 perform block matching operation on the basic viewpoint prediction blocks in the respective data storage addresses [0] to [63] and the basic viewpoint time domain neighboring blocks in the storage addresses [100] to [199], calculate SAD values of the basic viewpoint prediction blocks, and store the results into PE 12;
processing elements PE11, PE20, PE21, PE30 and PE31 perform block matching operations on the basic view spatial neighboring blocks of respective data memory addresses [0] - [63] and reference view spatial neighboring blocks in memory addresses [100] - [199] in parallel, calculate their SAD values, and store the results in processing element PE 22;
according to the method, the SAD value (Sum of Absolute Differences, SAD for short) is calculated according to a formula (1), and the method comprises the following steps:
(1)
wherein f isk(i, j) is the pixel value of the ith row and jth column of the basic view prediction block, gk(i, j) is a pixel value of the ith row and the jth column of the basic view reference block or the reference view reference block, (i, j) is a motion vector, and N is the size of the coding block.
In an embodiment of the present invention, in step S4, selecting an optimal temporal neighboring block from a temporal minimum SAD value according to the SAD value, performing motion compensation prediction, selecting an optimal spatial neighboring block from a minimum spatial SAD value, and performing disparity compensation prediction includes:
the processing element PE12 compares the SAD values sent by PE01 and PE02 when receiving the SAD values, stores the relatively small SAD values into the processing element PE12, finally obtains the optimal time domain adjacent block, performs motion compensation prediction calculation, and stores the result into the processing element PE 32;
the processing element PE22 compares the SAD values sent from PE11, PE20, PE21, PE30 and PE31, stores the relatively small SAD value in the processing element PE22, obtains the optimal spatial neighboring blocks, performs the disparity compensated prediction calculation, and stores the result in the processing element PE 32.
In an embodiment of the present invention, in step S5, the calculating a disparity vector of a depth map according to the motion compensated prediction and the disparity compensated prediction result includes:
after receiving the motion compensation prediction result and the disparity compensation prediction result respectively sent from PE12 and PE22, the processing element PE32 calculates the motion compensation prediction result and the disparity compensation prediction result to obtain a disparity vector of the depth map.
Advantageous effects
The invention has the beneficial effects that: according to the parallel implementation method for the disparity estimation of the depth map, provided by the embodiment of the invention, the parallel implementation is performed on the disparity estimation algorithm of the three-dimensional high-efficiency video coding by constructing the parallel structure, so that the calculation efficiency of the disparity estimation process is improved, the parallel calculation requirement of the disparity vector calculation process can be met, and the depth map coding time is shortened.
Drawings
FIG. 1 is a schematic diagram of the reference viewpoint time domain neighboring block and spatial neighboring block according to the neighboring block disparity vector NBDV method in accordance with an embodiment of the present invention;
fig. 2 is a flowchart of a parallel implementation method for disparity estimation of a depth map according to an embodiment of the present invention;
FIG. 3 is a diagram illustrating a parallel structure for disparity estimation of a depth map according to an embodiment of the present invention;
FIG. 4 is a diagram illustrating a 10 × 10 search window according to an embodiment of the present invention.
Detailed Description
For the purpose of better explaining the present invention and to facilitate understanding, the present invention will be described in detail by way of specific embodiments with reference to the accompanying drawings.
All technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. The terminology used in the description of the invention herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items.
In a related embodiment of the present invention, the disparity vector of the depth map is obtained using a method based on NBDV-independent encoding.
FIG. 1 is a schematic diagram of the reference viewpoint time domain neighboring block and spatial neighboring block position according to the neighboring block disparity vector NBDV method in an embodiment of the present invention. As shown in fig. 1, for an N × N basic view coding block, the acquisition of disparity vector is a search and matching process for 2 temporal neighboring blocks of the current view and 5 spatial neighboring blocks of the reference view. In fig. 1, positions T0 and T1 are temporal neighboring blocks, and positions a0, a1, B0, B1 and B2 are spatial neighboring blocks. In the conventional software method, NBDV performs search of an optimal matching block, SAD calculation, in the order of T0, T1, a1, B1, B0, a0, and B2. By adopting the block matching mode, the problems of long coding time and low efficiency are brought. Meanwhile, motion-compensated prediction (MCP) is required to be performed on time-domain neighboring blocks for disparity estimation of the 3D-HEVC depth map, and disparity-compensated prediction (DCP) is required to be performed on spatial-domain neighboring blocks, so that the computational complexity of prediction among views of the depth map is greatly increased.
In order to more accurately select a proper coding block size and realize a disparity estimation algorithm, an embodiment of the invention provides a disparity estimation algorithm by referring to HTM (3D-HEVC Test Model) reference software and different Test sequences of an official depth map: a series of comparative experiments were performed by balloon, Kendo, Newapaper and Poznan Stree and the results showed that almost all tested sequences selected 8 × 8 block sizes with depth of 3 for inter-view prediction 65% after CU partitioning. Therefore, the method selects the PU with the size of 8 multiplied by 8 to be used for the parallel realization of the disparity estimation of the depth map of the 3D-HEVC, and can obviously reduce the computational complexity under the condition of ensuring the prediction accuracy. For a PU of 8 × 8 size, the search window pixel block size for disparity estimation is 10 × 10, and the number of pixels involved in calculation reaches 700, resulting in a large access amount of disparity estimation data of the depth map and intensive calculation. However, analysis shows that the computation process of MCP, DCP and block matching has the characteristics of less global data, relatively independent pixel processing among blocks and a large number of same operations at the same time, so that the computation process can adopt a parallel idea, namely that a plurality of block matching processes are performed at the same time in parallel. Therefore, the parallel structure of the depth map disparity estimation algorithm capable of supporting the 3D-HEVC standard is designed, the original framework cannot be applied, the framework is designed again, and the framework can meet the parallel calculation requirement of the disparity vector calculation process.
Fig. 2 is a flowchart of a parallel implementation method for disparity estimation of a depth map for three-dimensional efficient video coding according to an embodiment of the present invention, as shown in fig. 2, the method includes the following steps:
as shown in fig. 2, in step S1, a parallel structure for three-dimensional high-efficiency video coding depth map disparity estimation is constructed based on a 4 × 4 two-dimensional processing element array of contiguous interconnects;
as shown in fig. 2, in step S2, reading base viewpoint pixel values and reference viewpoint pixel values into the processing elements of the parallel structure based on the acquired video image;
as shown in fig. 2, in step S3, the processing element of the parallel structure performs block matching operations on the temporal neighboring blocks and the spatial neighboring blocks in parallel, and calculates the SAD value of each neighboring block;
as shown in fig. 2, in step S4, according to the SAD value, selecting the minimum temporal SAD value to obtain the optimal temporal neighboring block, performing motion compensation prediction, selecting the minimum spatial SAD value to obtain the optimal spatial neighboring block, and performing disparity compensation prediction;
as shown in fig. 2, in step S5, a disparity vector is calculated based on the motion compensated prediction and the disparity compensated prediction results.
In the technical scheme provided by the embodiment of the invention shown in fig. 2, the depth map parallax estimation algorithm is realized in a parallelization manner by constructing a parallel structure, so that the calculation efficiency of the parallax estimation process is improved, the parallel calculation requirement of the parallax vector calculation process can be met, and the encoding time is shortened.
The specific implementation of each step of the embodiment shown in fig. 2 is explained in detail below:
in step S1, a parallel structure of depth map disparity estimation for three-dimensional efficient video coding is constructed based on a 4 × 4 two-dimensional array of contiguously interconnected processing elements.
In an embodiment of the present invention, 16 Processing elements (PE for short) are taken as an example, and fig. 3 is an architecture diagram of a depth map disparity estimation parallel structure for three-dimensional efficient video coding in an embodiment of the present invention. As shown in fig. 3, 4 × 4 PEs are included, i.e., the first row is PE00, PE01, PE02, and PE03, respectively, the second row is PE10, PE11, PE12, and PE13, the third row is PE20, PE21, PE22, and PE23, respectively, and the fourth row is PE30, PE31, PE32, and PE33, respectively.
In an embodiment of the present invention, before reading the base viewpoint pixel value and the reference viewpoint pixel value in step S2, the method further includes:
firstly, converting a test sequence of a video into binary data which can be identified by the array; secondly, dividing the binary data into 396 lines (64 pixels per line) from left to right and from top to bottom in an 8 × 8 coding block size, and storing the 396 lines by lines at an address where data input storage DIM [0000] starts, as the base view current image pixel value; converting a reference viewpoint image in a video test sequence into binary data which can be identified by the array, dividing the binary data into 396 rows (100 pixels in each row) from left to right and from top to bottom according to the size of a 10 multiplied by 10 search window, and storing the 396 rows in an address where data input storage DIM [1000] starts as the reference viewpoint pixel value; and storing pixel values of a reconstructed image obtained after encoding a frame of image on the current viewpoint into addresses starting from data output storage DOM [0000] according to the size of a 10 x 10 search window and lines (100 pixels per line) to serve as the reference pixel values of the current viewpoint.
The search window is a pixel block of 10 × 10 size determined on the reference frame centering on the SAD target block, as shown in fig. 4.
In step S2, the base viewpoint pixel values and the reference viewpoint pixel values are read into the processing elements of the parallel structure based on the acquired video image.
In an embodiment of the present invention, the reading the base viewpoint pixel value and the reference viewpoint pixel value in this step includes:
a. accessing the DIM by a processing element PE00 in a first row and a first column of the two-dimensional processing element array in a contiguous interconnection manner, reading 8 × 8 pixels in an x-th row from [0000] address to addresses [0] - [63] as a basic view Prediction Unit (PU);
b. accessing the DIM by the processing element PE10 in the first column and the second row in the two-dimensional processing element array in a manner of contiguous interconnection, and sequentially reading pixels of 5 rows 10 × 10 spatial neighboring blocks with row numbers of x-19, x-18, x-17, x-1, x +17 to addresses [100] - [599] from [1000] addresses as search windows of spatial neighboring blocks B2, B1, B0, A1 and A0;
c. the processing element PE03 accesses the DOM in a contiguous interconnect manner, starting at the address [0000], and sequentially reads 2 lines 10 × 10 of temporal neighboring block pixels with line number x, x +19 into the addresses [0] - [199], as the search windows of temporal neighboring blocks T1 and T0, respectively.
The above is the data loading process.
In an embodiment of the present invention, after step S2 and before step S3 is executed, the method further includes:
a. after reading the basic viewpoint PU, the processing element PE00 issues the basic viewpoint PU to the data storage addresses [0] - [63] of the processing elements PE01, PE10 and PE02 in a parallel manner, and then issues the basic viewpoint PU to the data storage addresses [0] - [63] of the processing elements PE11, PE20, PE21, PE30 and PE31 in a parallel manner by the PE 10;
b. processing element PE10 issues data of addresses [100] - [199] to data storage addresses [100] - [199] of PE31, issues data of addresses [200] - [299] to data storage addresses [100] - [199] of PE21, issues data of addresses [300] - [399] to data storage addresses [100] - [199] of PE20, issues data of addresses [400] - [499] to data storage addresses [100] - [199] of PE11, and issues data of addresses [500] - [599] to data storage addresses [100] - [199] of PE30 for the read reference viewpoint search window;
c. while PE10 issues data, processing element PE03 issues data of addresses [0] - [99] to data storage addresses [100] - [199] of PE02 and data of addresses [100] - [199] to data storage addresses [100] - [199] of PE01 for the read basic viewpoint search window;
d. the processing elements PE01, PE02, PE11, PE20, PE21, PE30 and PE31 are put in parallel into the same handshake signal after the completion of data transmission.
The above is the process of data transmission.
In an embodiment of the present invention, the performing, in the processing unit of the parallel structure, block matching operations of the time domain neighboring blocks and the spatial domain neighboring blocks in parallel in step S3, and calculating the SAD value of each neighboring block specifically includes:
the processing elements PE01 and PE02 take out the pixel value of the PU of the basic viewpoint from the respective local data storage addresses [0] - [63], perform block matching operation with the search window data of the time domain adjacent blocks in the storage addresses [100] - [199] according to a full search strategy, calculate to obtain the respective minimum SAD value, and store the minimum SAD value and the corresponding pixel value of the matched block into the processing element PE12 in the third row and the third column;
the processing elements PE11, PE12, PE13, PE21, and PE22 take out the PU pixel value of the basic viewpoint from the respective data storage addresses [0] - [63], perform block matching operation with the adjacent block search window data in the hollow domains in the storage addresses [100] - [199] according to the full search strategy, calculate the respective minimum SAD value, and store the minimum SAD value and the corresponding pixel value of the matched block in the processing element PE22 in the third row and the fourth column.
In an embodiment of the present invention, the step S4 of selecting the optimal temporal neighboring block with the minimum temporal SAD value according to the SAD value, performing motion compensation prediction, selecting the optimal spatial neighboring block with the minimum spatial SAD value, and performing disparity compensation prediction specifically includes:
the processing element PE12 compares the SAD values sent from PE01 and PE02 when receiving the SAD values, stores the relatively small SAD value and the pixel value of the corresponding matching block into the processing element PE12 as the optimal time domain neighboring block, and then calculates the MV by performing motion compensation by using the MCP method and stores the MV into PE 32;
the processing element PE22 compares the SAD values sent from PE11, PE20, PE21, PE30 and PE31, stores the relatively small SAD value into the processing element PE22, finally obtains the optimal spatial neighboring block, and then calculates DV by performing disparity compensation using MVP method and stores the DV into the processing element PE 32.
In one embodiment of the present invention, step S5 includes:
the processing element PE32 receives the motion compensation MV and the disparity compensation DV sent from PE12 and PE22, respectively, and calculates them to obtain the disparity vector of the depth map.
It should be noted that the full Search algorithm is an exhaustive Search performed in a Search Range (SR). The search range is referred to as a search area or a search window. The full search requires a predetermined search area within which to compare a given encoded block of the predicted frame with all candidate blocks in the reference frame search window, finding the matching block with the smallest matching error. For a prediction block of size 8 × 8, the search window size is at least 10 × 10, as shown in fig. 4. The searching process is to use a Current prediction Block (CB) to perform pixel-by-pixel matching with a Reference Block (RB) having the same size in a search window, and in HEVC, a SAD value is used to characterize a matching degree between CB and RB, which is defined as formula (1):
(1)
wherein f isk(i, j) is the pixel value of the ith row and jth column of the basic view prediction block, gk(i, j) is the pixel value of the ith row and the jth column of the basic view reference block or the reference view reference block, and N is the size of the coding block.
The offset (i, j) between CB and RB is called a Motion Vector (MV). The smaller the SAD value, the higher the matching degree, and the position of the reference block with the smallest cost is the best matching block. The calculation formula of the cost is shown as (2):
(2)cost＝distortion+λ×Rate
where distorsion is the SAD value of the current reference block, λ is the input coefficient, and Rate is the coding cost of the offset.
For 8 × 8 sized CBs, a 10 × 10 sized search window can generate 9 8 × 8 sized RBs for top left, top right, left, middle, right, bottom left, bottom right, and bottom right, respectively, that is, SAD values for 9 blocks need to be calculated.
The process of calculating motion vectors between the same viewpoint CB and RB is called motion-compensated prediction (MCP), and the predicted vector becomes MV. The process of calculating motion vectors between the different views CB and RB is called disparity-compensated prediction (DCP), and the vector obtained by prediction becomes DV. The disparity vector of the depth map is composed of two components, horizontal and vertical, DV can be used as the depth value of the depth map, and the horizontal component is obtained by equation (3):
(3)dv＝(s*v+o)>>n
where v is the depth sample value, s is the scale factor, o is the position offset, and n is a shift parameter that depends on the required precision.
The present embodiment is designed based on an 8 × 8 block size, and therefore, in order to calculate the depth map disparity vector based on the NBDV method, a SAD matching process of 63 RBs for 2 temporal reference windows and 5 spatial reference windows needs to be completed for each CB. In HTM this calculation is done serially, which causes significant computation delay. The invention uses 7 PEs to complete the calculation process of 63 SADs at the same time through parallel design, thereby achieving the purposes of improving the calculation efficiency of the algorithm and shortening the time.
In summary, the method provided by the embodiment of the present invention designs a parallel structure for disparity estimation and calculation of a depth map, so as to improve the parallelism as much as possible and shorten the encoding time.
Since each functional module of the parallel structure for depth map disparity estimation for three-dimensional high-efficiency video coding of fig. 3 corresponds to the steps of the above-mentioned exemplary embodiment of the parallel implementation method for depth map disparity estimation for three-dimensional high-efficiency video coding shown in fig. 2, for details not disclosed in the structural embodiment of the present disclosure, please refer to the above-mentioned embodiment of the parallel implementation method for depth map disparity estimation for three-dimensional high-efficiency video coding of the present disclosure.
It should be understood that the above description of specific embodiments of the present invention is only for the purpose of illustrating the technical lines and features of the present invention, and is intended to enable those skilled in the art to understand the contents of the present invention and to implement the present invention, but the present invention is not limited to the above specific embodiments. It is intended that all such changes and modifications as fall within the scope of the appended claims be embraced therein.
Claims (7)
1. A parallel implementation method for disparity estimation of a depth map, comprising:
s1, constructing a parallel structure for disparity estimation of the three-dimensional high-efficiency video coding depth map based on the 4 x 4 two-dimensional processing element array of the adjacent interconnection;
s2, reading basic viewpoint pixel values and reference viewpoint pixel values into the processing elements of the parallel structure based on the acquired video image;
s3, carrying out block matching operation of time domain adjacent blocks and space domain adjacent blocks in parallel in the processing element of the parallel structure, and calculating to obtain SAD value of each adjacent block;
s4, selecting the time domain minimum SAD value to obtain the optimal time domain adjacent block according to the SAD value, performing motion compensation prediction, selecting the minimum space domain SAD value to obtain the optimal space domain adjacent block, and performing parallax compensation prediction;
and S5, calculating to obtain the disparity vector of the depth map according to the motion compensation prediction result and the disparity compensation prediction result.
2. The method of claim 1, wherein before the reading of the base view pixel value and the reference view pixel value in step S2, further comprising:
converting the basic viewpoint image and the reference viewpoint image of the test sequence of the video into binary data which can be identified by the array;
distributing the binary data in a document, and storing the binary data in a data input storage (DIM) as the current pixel value of the basic viewpoint and the current pixel value of the reference viewpoint;
and storing the reconstructed image processed according to the frame before the basic viewpoint in the video in a data output storage (DOM) as the reference pixel value of the basic viewpoint.
3. The method of claim 2, wherein the reading of the base view pixel values and the reference view pixel values into the processing elements of the parallel structure in step S2 comprises:
accessing the DIM by processing element PE00 of a first row and a first column in the two-dimensional processing element array in an contiguously interconnected manner to read a corresponding base view current prediction block pixel value in a size of 8 × 8;
accessing the DIM by way of contiguous interconnections through processing element PE10 of a second row and a first column in the two-dimensional processing element array to read 5 corresponding reference view spatial neighboring block pixel values at a size of 10 × 10;
accessing the DOM by processing element PE03 of the first row and fourth column in the two-dimensional array of processing elements in a contiguous interconnect reads 2 corresponding base view temporal neighboring block pixel values at a size of 10 x 10.
4. The method of claim 3, wherein before performing step 3 after step S2 is completed, further comprising:
the processing element PE00 issues the current prediction block to the data storage addresses [0] - [63] of the processing element PE01 and PE02 in parallel, and issues the current prediction block to the data storage addresses [0] - [63] of the processing element PE11, PE20, PE21, PE30 and PE31 via the processing element PE 10;
the processing element PE03 sequentially issues the 2 basic viewpoint time domain neighboring blocks to data storage addresses [100] - [199] of PE01 and PE02, respectively;
the processing element PE10 issues the 5 reference viewpoint spatial neighboring blocks to data storage addresses [100] - [199] of the processing elements PE11, PE20, PE21, PE30 and PE31 respectively;
the processing elements PE01, PE02, PE11, PE20, PE21, PE30 and PE31 are put in parallel into the same handshake signal after the completion of data transmission.
5. The method of claim 1, wherein step S3 includes:
the processing elements PE01 and PE02 perform block matching operation on the basic viewpoint prediction blocks in the respective data storage addresses [0] to [63] and the basic viewpoint time domain neighboring blocks in the storage addresses [100] to [199], calculate SAD values of the basic viewpoint prediction blocks, and store the results into PE 12;
the processing elements PE11, PE20, PE21, PE30, and PE31 perform block matching operations on the base view spatial neighboring blocks of the respective data memory addresses [0] - [63] and the reference view spatial neighboring blocks in the memory addresses [100] - [199], calculate their SAD values, and store the results in the processing element PE 22.
6. The method of claim 1, wherein step S4 includes:
the processing element PE12 compares the SAD values sent by PE01 and PE02 when receiving the SAD values, stores the relatively small SAD values into the processing element PE12, finally obtains the optimal time domain adjacent block, performs motion compensation prediction calculation, and stores the result into the processing element PE 32;
the processing element PE22 compares the SAD values sent from PE11, PE20, PE21, PE30 and PE31, stores the relatively small SAD value in the processing element PE22, obtains the optimal spatial neighboring blocks, performs the disparity compensated prediction calculation, and stores the result in the processing element PE 32.
7. The method of claim 1, wherein step S5 includes:
after receiving the motion compensation prediction result and the disparity compensation prediction result respectively sent from PE12 and PE22, the processing element PE32 calculates the motion compensation prediction result and the disparity compensation prediction result to obtain a disparity vector of the depth map.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911271204.8A CN111050180A (en) | 2020-03-05 | 2020-03-05 | Parallel implementation method for depth map parallax estimation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911271204.8A CN111050180A (en) | 2020-03-05 | 2020-03-05 | Parallel implementation method for depth map parallax estimation |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111050180A true CN111050180A (en) | 2020-04-21 |
Family
ID=70235914
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911271204.8A Pending CN111050180A (en) | 2020-03-05 | 2020-03-05 | Parallel implementation method for depth map parallax estimation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111050180A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111680618A (en) * | 2020-06-04 | 2020-09-18 | 西安邮电大学 | Dynamic gesture recognition method based on video data characteristics, storage medium and device |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150332474A1 (en) * | 2014-05-18 | 2015-11-19 | Edge 3 Technologies, Inc. | Orthogonal and Collaborative Disparity Decomposition |
| CN110381321A (en) * | 2019-08-23 | 2019-10-25 | 西安邮电大学 | A kind of interpolation calculation Parallel Implementation method for motion compensation |
| CN110650346A (en) * | 2019-09-26 | 2020-01-03 | 西安邮电大学 | 3D-HEVC depth map motion estimation parallel implementation method and structure |
-
2020
- 2020-03-05 CN CN201911271204.8A patent/CN111050180A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150332474A1 (en) * | 2014-05-18 | 2015-11-19 | Edge 3 Technologies, Inc. | Orthogonal and Collaborative Disparity Decomposition |
| CN110381321A (en) * | 2019-08-23 | 2019-10-25 | 西安邮电大学 | A kind of interpolation calculation Parallel Implementation method for motion compensation |
| CN110650346A (en) * | 2019-09-26 | 2020-01-03 | 西安邮电大学 | 3D-HEVC depth map motion estimation parallel implementation method and structure |
Non-Patent Citations (2)
| Title |
|---|
| 谢晓燕、周金娜、朱筠、刘新闯、王安琪: "一种改进的相邻块视差矢量快速获取方法", 《西安邮电大学学报》 * |
| 谢晓燕、王昱、石鹏飞、朱筠: "3D-HEVC深度建模模式的优化及并行设计", 《小型微型计算机系统》 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111680618A (en) * | 2020-06-04 | 2020-09-18 | 西安邮电大学 | Dynamic gesture recognition method based on video data characteristics, storage medium and device |
| CN111680618B (en) * | 2020-06-04 | 2023-04-18 | 西安邮电大学 | Dynamic gesture recognition method based on video data characteristics, storage medium and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US4937666A (en) | Circuit implementation of block matching algorithm with fractional precision | |
| US10268901B2 (en) | Quasi-parametric optical flow estimation | |
| ES2664746T3 (en) | Effective prediction using partition coding | |
| US11539976B2 (en) | Motion estimation using collocated blocks | |
| US8160146B2 (en) | Video coding method and apparatus for calculating motion vectors of the vertices of a patch of an image and transmitting information of horizontal and vertical components of the motion vectors | |
| US20090085846A1 (en) | Image processing device and method performing motion compensation using motion estimation | |
| BR122020014854B1 (en) | EFFECTIVE PARTITION ENCODING WITH HIGH LEVEL OF PARTITION FREEDOM | |
| US5936672A (en) | Half pixel motion estimator | |
| KR100994983B1 (en) | Apparatus and method for estimation of high speed motion | |
| TW502535B (en) | Methods and apparatus for motion estimation in compressed domain | |
| US20060044316A1 (en) | High performance memory and system organization for digital signal processing | |
| CN110650346B (en) | 3D-HEVC depth map motion estimation parallel implementation method and structure | |
| CN110662072B (en) | Motion information candidate list construction method and device and readable storage medium | |
| CN111050180A (en) | Parallel implementation method for depth map parallax estimation | |
| US9443319B2 (en) | Image processor for motion searches | |
| CN110381321B (en) | Interpolation calculation parallel implementation method for motion compensation | |
| US5982910A (en) | Method and circuit arrangement for undersampling in the case of movement estimation | |
| EP2046047A1 (en) | Method and device for performing motion estimation | |
| KR20150110541A (en) | Interpolation method and corresponding device | |
| CN110933441A (en) | Parallel implementation method for depth map contour prediction | |
| US20110249188A1 (en) | Method of block-based motion estimation | |
| US10448043B2 (en) | Motion estimation method and motion estimator for estimating motion vector of block of current frame | |
| US8200032B2 (en) | Image processing method and related apparatus for performing image processing operation according to image blocks in horizontal direction | |
| KR100359091B1 (en) | Motion estimation device | |
| US20040179604A1 (en) | Motion vector selection based on a preferred point |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200421 |
|
| RJ01 | Rejection of invention patent application after publication |