CN111432194B - Disparity map hole filling method and device, electronic equipment and storage medium - Google Patents
Disparity map hole filling method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN111432194B CN111432194B CN202010168220.0A CN202010168220A CN111432194B CN 111432194 B CN111432194 B CN 111432194B CN 202010168220 A CN202010168220 A CN 202010168220A CN 111432194 B CN111432194 B CN 111432194B
- Authority
- CN
- China
- Prior art keywords
- pixel
- filled
- disparity
- value
- parallax
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
- H04N13/106—Processing image signals
- H04N13/122—Improving the 3D impression of stereoscopic images by modifying image signal contents, e.g. by filtering or adding monoscopic depth cues
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Image Processing (AREA)
- Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)
Abstract
The embodiment of the application provides a method and a device for filling a disparity map hole, wherein the method comprises the following steps: for each pixel to be filled, determining a left effective parallax pixel of the pixel to be filled and a right effective parallax pixel of the pixel to be filled; calculating the parallax value difference corresponding to the pixel to be filled and the original pixel value difference corresponding to the pixel to be filled, and calculating the parallax estimation value of the pixel to be filled based on the parallax value difference corresponding to the pixel to be filled and the original pixel value difference corresponding to the pixel to be filled; and taking the parallax estimation value of at least part of the pixels to be filled as the parallax value of the pixels to be filled. Meanwhile, the relevance between the difference of the parallax values of the effective parallax pixels on the two sides of the pixel to be filled and the parallax value of the pixel to be filled and the relevance between the difference of the pixel values of the original image pixels corresponding to the effective parallax pixels on the two sides and the parallax value of the pixel to be filled are considered. And the condition of stripe flaws and the condition of highlight spots are avoided, and the filling of the parallax image holes is completed in short time.
Description
Technical Field
The application relates to the field of image processing, in particular to a disparity map hole filling method and device and an electronic equipment storage medium.
Background
The disparity map is an image used in the stereo matching technique to represent a translational relationship between a target image and a reference image. The steps of consistency detection, occlusion detection, mismatch detection, etc. in the stereo matching technology all depend on the disparity map. Since the reference image does not contain all information required in the target image, a hole appears in the target image, and the disparity map comprises a region to be filled corresponding to the hole of the target image. Before the step of performing a disparity map-dependent process in the stereo matching technology by using a disparity map, disparity map hole filling needs to be performed on a region to be filled in the disparity map, that is, for each pixel to be filled in the region to be filled in the disparity map, a disparity value of the pixel to be filled is re-determined, so that the pixel to be filled has the determined disparity value.
Currently, a scan line filling algorithm or a variable size average filtering algorithm is generally adopted for filling the disparity map holes. The scanning line filling algorithm directly selects the minimum value of the left effective parallax value and the right effective parallax value of the pixel to be filled in the region to be filled, so that an obvious streak defect phenomenon is generated. On one hand, for each pixel to be filled, the parallax value of the pixel to be filled can be determined after repeated iterative filtering, so that the time consumption for filling the parallax image hole is long. On the other hand, a highlight spot may appear at a pixel where the determined disparity value is wrong.
Disclosure of Invention
The embodiment of the application provides a disparity map hole filling method and device, electronic equipment and an electronic equipment storage medium.
The embodiment of the application provides a method for filling a disparity map hole, which comprises the following steps:
for each pixel to be filled in the area to be filled of the disparity map, determining a left effective disparity pixel of the pixel to be filled and a right effective disparity pixel of the pixel to be filled;
for each pixel to be filled, calculating a disparity value difference amount corresponding to the pixel to be filled and an original pixel value difference amount corresponding to the pixel to be filled, wherein the disparity value difference amount corresponding to the pixel to be filled is a difference amount between a disparity value of a left effective disparity pixel of the pixel to be filled and a disparity value of a right effective disparity pixel of the pixel to be filled, and the original pixel value difference amount corresponding to the pixel to be filled is a difference amount between a pixel value of an original pixel corresponding to the left effective disparity pixel of the pixel to be filled and a pixel value of an original image corresponding to the right effective disparity pixel of the pixel to be filled;
for each pixel to be filled, calculating a parallax estimation value of the pixel to be filled based on a parallax value difference corresponding to the pixel to be filled and an original pixel value difference corresponding to the pixel to be filled;
for each pixel to be filled in at least part of pixels to be filled, taking the parallax estimated value of the pixel to be filled as the parallax value of the pixel to be filled.
The embodiment of the application provides a parallax map hole filling device, includes:
an effective parallax pixel determination unit configured to determine, for each pixel to be filled in a region to be filled of a parallax map, a left effective parallax pixel of the pixel to be filled and a right effective parallax pixel of the pixel to be filled;
a difference amount calculation unit configured to calculate, for each pixel to be filled, a disparity value difference amount corresponding to the pixel to be filled and an original pixel value difference amount corresponding to the pixel to be filled, where the disparity value difference amount corresponding to the pixel to be filled is a difference amount between a disparity value of a left effective disparity pixel of the pixel to be filled and a disparity value of a right effective disparity pixel of the pixel to be filled, and the original pixel value difference amount corresponding to the pixel to be filled is a difference amount between a pixel value of an original pixel corresponding to the left effective disparity pixel of the pixel to be filled and a pixel value of an original image pixel corresponding to the right effective disparity pixel of the pixel to be filled;
a parallax estimation value calculation unit configured to calculate, for each of the pixels to be filled, a parallax estimation value of the pixel to be filled based on a parallax value difference amount corresponding to the pixel to be filled and an original pixel value difference amount corresponding to the pixel to be filled;
the parallax value filling unit is configured to take the parallax estimation value of each pixel to be filled in at least part of the pixels to be filled as the parallax value of the pixel to be filled.
The disparity map hole filling method and device provided by the embodiment of the application realize that when the disparity map hole is filled, the relevance between the disparity value of the left effective disparity pixel of the pixel to be filled and the disparity value of the right effective disparity pixel of the pixel to be filled and the disparity value of the pixel to be filled, the relevance between the difference between the pixel value of the original image pixel corresponding to the left effective disparity pixel of the pixel to be filled and the pixel value of the original image pixel corresponding to the right effective disparity pixel of the pixel to be filled and the relevance between the disparity value of the pixel to be filled are considered at the same time, the disparity estimation value of each pixel to be filled is calculated, and the disparity value of each pixel to be filled is further determined. Therefore, on one hand, the stripe flaw condition and the highlight spot condition are avoided, and on the other hand, the parallax image hole filling is completed in short time.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and together with the description, serve to explain the principles of the application.
Fig. 1 is a flowchart illustrating a disparity map hole filling method provided in an embodiment of the present application;
fig. 2 is a block diagram illustrating a structure of a disparity map hole filling apparatus provided in an embodiment of the present application;
fig. 3 shows a block diagram of an electronic device according to an embodiment of the present application.
Detailed Description
The present application will be described in further detail with reference to the following drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the relevant invention and not restrictive of the invention. It should be noted that, for convenience of description, only the portions related to the related invention are shown in the drawings.
It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict. The present application will be described in detail below with reference to the embodiments with reference to the attached drawings.
Fig. 1 shows a flowchart of a disparity map hole filling method provided in an embodiment of the present application, where the method includes:
For each pixel in the disparity map, the pixel value of the pixel is the disparity value. The parallax values of the pixels located in the region other than the region to be filled are valid, and the pixels located in the region other than the region to be filled are valid parallax pixels. The disparity value of the effective disparity pixel can be referred to as an effective disparity value, and the disparity value of the effective disparity pixel in the disparity map is kept unchanged.
For each pixel to be filled in the disparity map, a left effective disparity pixel of the pixel to be filled and a right effective disparity pixel of the pixel to be filled are determined.
In the application, for each pixel to be filled in the area to be filled in the disparity map, it is determined whether at least one effective disparity pixel located on the left side of the pixel to be filled exists in a currently scanned line, and if so, an effective disparity pixel closest to the pixel to be filled in all the effective disparity pixels located on the left side of the pixel to be filled in the currently scanned line is taken as the left effective disparity pixel of the pixel to be filled. If not, the next line of the currently scanned line is searched continuously. And searching for the effective parallax pixel on the left side of the pixel to be filled by searching in at least one row. When searching for the first time, the current scanning line is the line where the pixel to be filled is located. Similarly, the right effective parallax pixel of each pixel to be filled can be found.
For each pixel to be filled, there is typically at least one valid parallax pixel on the left side of the pixel to be filled and on the right side of the pixel to be filled in the row in which the pixel to be filled is located.
For each pixel to be filled, when at least one effective parallax pixel located on the left side of the pixel to be filled exists in the row of the pixel to be filled, the left effective parallax pixel of the pixel to be filled may be an effective parallax pixel closest to the pixel to be filled, among all effective parallax pixels located on the left side of the pixel to be filled in the row of the pixel to be filled.
For each pixel to be filled, when at least one effective parallax pixel located on the right side of the pixel to be filled exists in the row of the pixel to be filled, the right effective parallax pixel of the pixel to be filled may be an effective parallax pixel located closest to the pixel to be filled, among all effective parallax pixels located on the right side of the pixel to be filled in the row of the pixel to be filled.
And 102, calculating the parallax value difference corresponding to each pixel to be filled and the original pixel value difference corresponding to each pixel to be filled.
In this application, for each pixel to be filled, the disparity value difference amount corresponding to the pixel to be filled is the difference amount between the disparity value of the left effective disparity pixel of the pixel to be filled and the disparity value of the right effective disparity pixel of the pixel to be filled.
For each pixel to be filled, when the disparity value difference corresponding to the pixel to be filled is calculated, the difference obtained by subtracting another disparity value from any one of the disparity value of the left effective disparity pixel of the pixel to be filled and the disparity value of the right effective disparity pixel of the pixel to be filled can be calculated, and the absolute value of the calculated difference is used as the disparity value difference corresponding to the pixel to be filled.
In this application, for each pixel to be filled, the difference amount of the original pixel value corresponding to the pixel to be filled is the difference amount between the pixel value of the original image pixel corresponding to the left effective parallax pixel of the pixel to be filled and the pixel value of the original image pixel corresponding to the right effective parallax pixel of the pixel to be filled.
In the present application, the original image corresponding to the disparity map may be a target image of a reference image and a target image used for generating the disparity map.
For each pixel in the disparity map, the pixel corresponds to an original pixel in the original image corresponding to the disparity map, and the position of the pixel in the disparity map is the same as the position of the original pixel corresponding to the pixel in the original image corresponding to the disparity map.
For each pixel to be filled, when calculating the difference amount of the original pixel value corresponding to the pixel to be filled, a difference between any one of the pixel value of the original image pixel corresponding to the left effective parallax pixel of the pixel to be filled and the pixel value of the original image pixel corresponding to the right effective parallax pixel of the pixel to be filled minus another pixel value may be calculated, for example, a difference between the pixel value of the original image pixel corresponding to the left effective parallax pixel of the pixel to be filled and the pixel value of the original image pixel corresponding to the right effective parallax pixel of the pixel to be filled is calculated, and an absolute value of the calculated difference is used as the difference amount of the original pixel value corresponding to the pixel to be filled.
And 103, calculating the parallax estimation value of the pixel to be filled based on the parallax value difference corresponding to the pixel to be filled and the corresponding original pixel value difference.
In this application, for each pixel to be filled in the area to be filled in the disparity map, when the disparity estimation value of the pixel to be filled is calculated based on the disparity value difference corresponding to the pixel to be filled and the original pixel value difference corresponding to the pixel to be filled, the disparity value difference corresponding to the pixel to be filled may be compared with a disparity value difference threshold, and the original pixel value difference corresponding to the pixel to be filled may be compared with an original pixel value difference threshold. And calculating the parallax estimation value of the pixel to be filled according to the comparison result of the parallax value difference corresponding to the pixel to be filled and the parallax value difference threshold value and the comparison result of the original pixel value difference corresponding to the pixel to be filled and the original pixel value difference threshold value.
The following describes a process of calculating a disparity estimation value of a pixel to be filled, taking the pixel to be filled as an example:
the parallax value of the left effective parallax pixel of the pixel to be filled utilizes dplIndicating that the disparity value of the right effective disparity pixel of the pixel to be filled utilizes dprAnd (4) showing. The disparity value difference corresponding to the pixel to be filled is represented by Δ d, where Δ d ═ dpl-dprL. Disparity value difference threshold value utilization deltadAnd (4) showing.
The pixel value of the original image pixel corresponding to the left effective parallax pixel of the pixel to be filled utilizes IplIndicating that the pixel value of the original image pixel corresponding to the right effective parallax pixel of the pixel to be filled utilizes IprAnd (4) showing. The difference of the original pixel value corresponding to the pixel to be filled is represented by Δ I ═ Ipl-IprL. Threshold value utilization delta for difference of original pixel valuesiAnd (4) showing.
When Δ d is smaller than δdAnd Δ i is less than δiAnd if so, the parallax value of the left effective parallax pixel of the pixel to be filled is similar to the parallax value of the right effective parallax pixel of the pixel to be filled, and meanwhile, the pixel value of the original image pixel corresponding to the left effective parallax pixel of the pixel to be filled is similar to the pixel value of the original image pixel corresponding to the right effective parallax pixel of the pixel to be filled.
When Δ d is smaller than δdAnd Δ i is less than δiThen it may be located to the left of the pixel to be filledAnd taking the parallax value of the side effective parallax pixel and one parallax value in a value interval with the parallax value of the right effective parallax pixel of the pixel to be filled as end points as the parallax estimated value of the pixel to be filled.
When Δ d<δdAnd Δ i>δiWhen the parallax image is processed, the pixel value of the original image pixel corresponding to the left effective parallax pixel of the pixel to be filled and the pixel value of the original image pixel corresponding to the right effective parallax pixel of the pixel to be filled have transition or mutation to a certain extent on the original image corresponding to the parallax image, and meanwhile, the difference of the parallax values is small, so that the left effective parallax pixel of the pixel to be filled, the pixel to be filled and the right effective parallax pixel of the pixel to be filled may be in a gradual change region or an inclined plane. The plurality of disparity values at the fade area or slope are of a fade effect and continuity.
The parallax estimation value of the pixel to be filled utilizes dp *And (4) showing. The disparity value of the pixel to be filled and the disparity value of the left effective disparity pixel of the pixel to be filled utilize | dp *-dplI represents that the difference between the parallax value of the right effective parallax pixel of the pixel to be filled and the parallax value of the left effective parallax pixel of the pixel to be filled utilizes dpr-dplAnd | represents.
Distance between the pixel to be filled and the left effective parallax pixel of the pixel to be filled is distlAnd (4) showing. Distance between the pixel to be filled and the right effective parallax pixel of the pixel to be filled is determined by distrAnd (4) showing. The sum of the distance between the pixel to be filled and the left effective parallax pixel of the pixel to be filled and the distance between the pixel to be filled and the right effective parallax pixel of the pixel to be filled utilizes distl+distrAnd (4) showing.
When Δ d<δdAnd Δ i>δiIn order to make the left effective parallax pixel of the pixel to be filled, the pixel to be filled and the right effective parallax pixel of the pixel to be filled have gradual change effect and continuity, d needs to be madep *-dplI and | dpr-dplThe ratio of | is | dp *-dpl|/|dpr-dplI and distlAnd distl+distrIs the ratio of (i.e. dist)l/distl+distrAnd (4) approaching. Can change the distlAnd distl+distrMultiplying the ratio by a predetermined coefficient close to 1, e.g., 0.9, and taking the result as | dp *-dplI and | dpr-dprThe ratio of | is | dp *-dpl|/|dpr-dpl|，dpr、dplAre known and can be calculated from | dp *-dpl|/|dpr-dplL, calculating dp *. When d isplIs less than dprWhen it is, then dp *Is greater than dplAnd is less than dpr. When d isplGreater than dprWhen d is greater thanp *Is greater than dprAnd is less than dpl。
When Δ d<δdAnd Δ i>δiAnd calculating the parallax estimation value of the pixel to be filled by considering the distance between the pixel to be filled and the left effective parallax pixel of the pixel to be filled and the correlation between the distance between the pixel to be filled and the right effective parallax pixel of the pixel to be filled and the parallax value of the pixel to be filled. When the calculated parallax estimated value of the pixel to be filled is used as the parallax value of the pixel to be filled, and the filling of the parallax value of the pixel to be filled is completed, the gradual change effect and the continuity of the pixel to be filled in the inclined plane or the gradual change area or the inclined plane can be maintained.
When Δ d>δdAnd Δ i>δiWhen the pixel to be filled is in the filled area, there is a significant abrupt change between the parallax value of the left effective parallax pixel of the pixel to be filled and the parallax value of the right effective parallax pixel of the pixel to be filled, which may be caused by the pixel to be filled being located at the edge of the area to be filled.
When Δ d>δdAnd Δ i>δiThen, the disparity value of the left effective disparity pixel of the pixel to be filled and the disparity value of the left effective disparity pixel to be filled can be comparedAnd taking the smallest disparity value in the disparity values of the effective disparity pixels on the right side of the filling pixel as a reference disparity value, and taking a disparity value which is larger than the reference disparity value and has a difference with the reference disparity value smaller than a disparity threshold value or a disparity value which is smaller than the reference disparity value and has a difference with the reference disparity value smaller than the disparity threshold value as the disparity estimation value of the pixel to be filled.
When Δ d>δdAnd Δ i<δiWhen the pixel to be filled is in the filled area, the disparity value of the left effective disparity pixel of the pixel to be filled and the disparity value of the right effective disparity pixel of the pixel to be filled have abrupt changes, but the pixel value of the original pixel corresponding to the left effective disparity pixel of the pixel to be filled and the pixel value of the original image pixel corresponding to the right effective disparity pixel of the pixel to be filled do not have abrupt changes in the original image, which may be caused by the boundary of a plurality of different color sub-areas in the filled area where the area to be filled is large and the pixel to be filled is located.
When Δ d>δdAnd Δ i<δiThen, the smallest disparity value of the left effective disparity pixel of the pixel to be filled and the disparity value of the right effective disparity pixel of the pixel to be filled may be used as the reference disparity value, and a disparity value that is greater than the reference disparity value and whose difference from the reference disparity value is less than a disparity threshold value or a disparity value that is less than the reference disparity value and whose difference from the reference disparity value is less than a disparity threshold value may be used as the disparity estimation value of the pixel to be filled.
In some embodiments, for each pixel to be filled, when the disparity value difference amount corresponding to the pixel to be filled is less than a disparity value difference amount threshold and the original pixel value difference amount corresponding to the pixel to be filled is less than an original pixel value difference amount threshold, calculating an average value of the disparity value of the left effective disparity pixel of the pixel to be filled and the disparity value of the right effective disparity pixel of the pixel to be filled; and taking the calculated average value as the parallax estimated value of the pixel to be filled.
When Δ d is smaller than δdAnd Δ i is less than δiThen, the parallax value d of the left effective parallax pixel of the pixel to be filled can be calculatedplThe parallax value d of the right effective parallax pixel with the pixel to be filledprAnd, the left effective parallax pixel d of the pixel to be filledplAnd the disparity value d of the right effective disparity pixel of the pixel to be filledprAnd 2 is divided to obtain the parallax estimated value of the pixel to be filled.
When Δ d is smaller than δdAnd Δ i is less than δiThen, the parallax estimation value d of the pixel to be filledp *Can be expressed as:
in some embodiments, for each pixel to be filled, when the disparity value difference amount corresponding to the pixel to be filled is less than the disparity value difference amount threshold and the original pixel value difference amount corresponding to the pixel to be filled is greater than the original pixel value difference amount threshold, calculating the disparity estimation value of the pixel to be filled based on the disparity value difference amount corresponding to the pixel to be filled and the original pixel value difference amount corresponding to the pixel to be filled comprises: and calculating the parallax estimation value of the pixel to be filled based on the parallax value difference corresponding to the pixel to be filled, the first distance corresponding to the pixel to be filled and the second distance corresponding to the pixel to be filled.
For each pixel to be filled, the disparity value difference amount corresponding to the pixel to be filled is the difference amount between the disparity value of the left effective disparity pixel of the pixel to be filled and the disparity value of the right effective disparity pixel of the pixel to be filled.
For each pixel to be filled, the difference obtained by subtracting the other disparity value from any one of the disparity value of the left effective disparity pixel of the pixel to be filled and the disparity value of the right effective disparity pixel of the pixel to be filled can be calculated, and the absolute value of the calculated difference is used as the difference between the disparity value of the left effective disparity pixel of the pixel to be filled and the disparity value of the right effective disparity pixel of the pixel to be filled, that is, the absolute value of the calculated difference is used as the disparity value difference corresponding to the pixel to be filled.
For each pixel to be filled, the first distance corresponding to the pixel to be filled is the distance between the pixel to be filled and the left effective parallax pixel of the pixel to be filled or the right effective parallax pixel of the pixel to be filled.
For each pixel to be filled, the second distance corresponding to the pixel to be filled is the sum of the distance between the pixel to be filled and the left effective parallax pixel of the pixel to be filled and the distance between the pixel to be filled and the right effective parallax pixel of the pixel to be filled.
In the present application, a first preset ratio and a second preset ratio may be preset, the first preset ratio is less than 1/2, and the second preset ratio is greater than 1/2 and less than 1. And when the first distance corresponding to the pixel to be filled is smaller than 1/2 of the second distance corresponding to the pixel to be filled, calculating the parallax estimation value of the pixel to be filled by utilizing a first preset proportion, and when the first distance corresponding to the pixel to be filled is larger than 1/2 of the second distance corresponding to the pixel to be filled, calculating the parallax estimation value of the pixel to be filled by utilizing a second preset proportion.
For each pixel to be filled, when the first distance corresponding to the pixel to be filled is the distance between the pixel to be filled and the left effective parallax pixel of the pixel to be filled, if the first distance corresponding to the pixel to be filled is smaller than 1/2 of the second distance corresponding to the pixel to be filled, the product of the parallax value difference amount corresponding to the pixel to be filled and the first preset proportion can be used as the difference amount between the parallax estimation value of the pixel to be filled and the parallax value of the left effective parallax pixel of the pixel to be filled.
And under the condition that the first distance corresponding to the pixel to be filled is smaller than 1/2 of the second distance corresponding to the pixel to be filled, if the parallax value of the left effective parallax pixel of the pixel to be filled is larger than the parallax value of the right effective parallax pixel of the pixel to be filled, taking the difference obtained by subtracting the calculated difference amount from the parallax value of the left effective parallax pixel of the pixel to be filled as the parallax estimated value of the pixel to be filled. And if the parallax value of the left effective parallax pixel of the pixel to be filled is smaller than the parallax value of the right effective parallax pixel of the pixel to be filled, taking the sum of the parallax value of the left effective parallax pixel of the pixel to be filled and the calculated difference amount as the parallax estimated value of the pixel to be filled.
If the first distance corresponding to the pixel to be filled is equal to 1/2 of the second distance corresponding to the pixel to be filled, 1/2 of the disparity value difference amount corresponding to the pixel to be filled can be used as the difference amount between the disparity estimation value of the pixel to be filled and the disparity value of the left effective disparity pixel of the pixel to be filled. And calculating the parallax estimation value of the pixel to be filled according to the calculated difference and the parallax value of the effective parallax pixel on the left side of the pixel to be filled.
If the first distance corresponding to the pixel to be filled is greater than 1/2 of the second distance corresponding to the pixel to be filled, a product of the disparity value difference corresponding to the pixel to be filled and a second preset proportion can be used as the difference between the disparity estimation value of the pixel to be filled and the disparity value of the left effective disparity pixel of the pixel to be filled. And calculating the parallax estimation value of the pixel to be filled according to the calculated difference and the parallax value of the effective parallax pixel on the left side of the pixel to be filled.
For each pixel to be filled, when the first distance corresponding to the pixel to be filled is the distance between the pixel to be filled and the right effective parallax pixel of the pixel to be filled, the same way as the above-mentioned process of calculating the parallax estimation value of the pixel to be filled can be used to calculate the difference between the parallax estimation value of the pixel to be filled and the parallax value of the right effective parallax pixel of the pixel to be filled. And calculating the parallax estimation value of the pixel to be filled according to the calculated difference and the parallax value of the effective parallax pixel on the right side of the pixel to be filled.
In some embodiments, for each pixel to be filled, when the disparity value difference amount corresponding to the pixel to be filled is less than the disparity value difference amount threshold and the original pixel value difference amount corresponding to the pixel to be filled is greater than the original pixel value difference amount threshold, calculating the disparity estimation value of the pixel to be filled based on the disparity value difference amount corresponding to the pixel to be filled, the first distance corresponding to the pixel to be filled, and the second distance corresponding to the pixel to be filled includes: calculating the distance ratio corresponding to the pixel to be filled; and calculating the parallax estimation value of the pixel to be filled based on the distance ratio corresponding to the pixel to be filled and the parallax difference corresponding to the pixel to be filled.
For each pixel to be filled, the distance ratio corresponding to the pixel to be filled is the ratio of the first distance corresponding to the pixel to be filled to the second distance corresponding to the pixel to be filled, i.e., the quotient of the first distance corresponding to the pixel to be filled and the second distance corresponding to the pixel to be filled.
For each pixel to be filled, when the first distance corresponding to the pixel to be filled is the distance between the pixel to be filled and the left effective parallax pixel of the pixel to be filled, the product of the distance ratio corresponding to the pixel to be filled and the parallax value difference corresponding to the pixel to be filled may be used as the difference between the parallax estimation value of the pixel to be filled and the parallax value of the left effective parallax pixel of the pixel to be filled. And calculating the parallax estimation value of the pixel to be filled according to the calculated difference and the parallax value of the effective parallax pixel on the left side of the pixel to be filled.
When Δ d<δdAnd Δ i>δiIn order to make the left effective parallax pixel of the pixel to be filled, and the right effective parallax pixel of the pixel to be filled have gradual change effect and continuity, d can be setp *-dplI and | dpr-dplI.e. ratio of |, to distance i.e. distlAnd distl+distrAre equal in the ratio of i.e. | dp *-dpl|/|dpr-dpl|＝distl/distl+distr。
distlRepresenting a first distance, dist, corresponding to a pixel to be filledl+distrRepresents a second distance, dist, corresponding to the pixel to be filledl/distl+distrRepresents the corresponding distance ratio, | d, of the pixel to be filledp *-dplL represents the parallax estimation value of the pixel to be filled and the left side of the pixel to be filledThe amount of difference between disparity values of effective disparity pixels, | dpr-dplAnd | represents the disparity value difference corresponding to the pixel to be filled. | dpr-dpl|.(distl/distl+distr) And the disparity value of the pixel to be filled is represented as the difference between the disparity estimation value of the pixel to be filled and the disparity value of the left effective disparity pixel of the pixel to be filled.
When d isplIs less than dprWhen it is, then dp *Greater than dplAnd is less than dprThe parallax estimation value d of the pixel to be filledp *Can be expressed as:
dp *＝dpl+|dpr-dpl|.(distl/distl+distr)
when d isplGreater than dprWhen it is, then dp *Greater than dprAnd is less than dplThe parallax estimation value d of the pixel to be filledp *Can be expressed as:
dp *＝dpl-|dpr-dpl|.(distl/distl+distr)
for each pixel to be filled, when the first distance corresponding to the pixel to be filled is the distance between the pixel to be filled and the right effective parallax pixel of the pixel to be filled, similarly to the above calculation process, when the first distance corresponding to the pixel to be filled is the distance between the pixel to be filled and the right effective parallax pixel of the pixel to be filled, the distance ratio corresponding to the pixel to be filled can be calculated, and the product of the distance ratio corresponding to the pixel to be filled and the parallax value difference corresponding to the pixel to be filled is taken as the difference between the parallax estimation value of the pixel to be filled and the parallax value difference of the right effective parallax pixel of the pixel to be filled. And calculating the parallax estimation value of the pixel to be filled according to the calculated difference and the parallax value of the effective parallax pixel on the right side of the pixel to be filled.
In some embodiments, for each pixel to be filled, when the disparity value difference corresponding to the pixel to be filled is greater than the disparity value difference threshold, the smallest disparity value of the left effective disparity pixel of the pixel to be filled and the disparity value of the right effective disparity pixel of the pixel to be filled is used as the disparity estimation value of the pixel to be filled.
When Δ d>δdThen, the parallax estimation value d of the pixel to be filledp *Can be expressed as:
dp *＝min(dpl,dpr)
and 104, taking the parallax estimation value of at least part of the pixels to be filled as the parallax value of the pixels to be filled.
In this application, after the disparity estimation value of each pixel to be filled is calculated, for each pixel to be filled in at least part of the pixels to be filled, the disparity estimation value of the pixel to be filled may be used as the disparity value of the pixel to be filled.
In this application, for each pixel to be filled in all pixels to be filled, the disparity estimation value of the pixel to be filled is used as the disparity value of the pixel to be filled, so as to complete the filling of the disparity value of the pixel to be filled. Therefore, each pixel to be filled has a parallax value, and the filling of the parallax map hole is completed.
In the present application, for each pixel to be filled with a low confidence of the disparity estimation value, the disparity value of the pixel to be filled with the low confidence of the disparity estimation value may be determined by searching for an original pixel having a similar pixel color of an original image pixel corresponding to the pixel to be filled with the low confidence of the disparity estimation value.
The pixel to be filled with the low confidence coefficient of the parallax estimation value is the pixel to be filled with the corresponding parallax difference value difference quantity larger than the parallax difference value difference quantity threshold value.
For each pixel to be filled except for the pixel to be filled with low confidence of the parallax estimation value, the parallax estimation value of the pixel to be filled can be used as the parallax value of the pixel to be filled.
For each pixel to be filled with low confidence of the parallax estimation value, an original image pixel meeting the following conditions can be found in an original image corresponding to the parallax image: the distance between the original pixel and the original pixel corresponding to the pixel to be filled with low confidence coefficient of the parallax estimation value is smaller than a distance threshold, and the absolute value of the difference between the pixel value of the original pixel and the pixel value of the original pixel corresponding to the pixel to be filled with low confidence coefficient of the parallax estimation value is smaller than a threshold. Then, the parallax value of the pixel corresponding to the found original pixel in the parallax map may be set as the parallax value with low confidence of the parallax estimation value.
In some embodiments, after regarding, for each pixel to be filled in at least some pixels to be filled, the disparity estimation value of the pixel to be filled as the disparity value of the pixel to be filled, the method further includes: for each target pixel to be filled in the area to be filled of the disparity map, searching out an original image pixel with the pixel color most similar to the pixel color of the original image pixel corresponding to the target pixel to be filled from an area, associated with the target pixel to be filled, in the original image corresponding to the disparity map, wherein the target pixel to be filled is the pixel to be filled with the disparity estimation value of which the confidence coefficient is smaller than the confidence coefficient threshold value, and the area associated with the target pixel to be filled is an area with a preset area and taking the original image pixel corresponding to the target pixel to be filled as the center; and taking the parallax value of the pixel corresponding to the searched original pixel in the parallax map as the parallax value of the target pixel to be filled.
In the present application, for a target pixel to be filled with a low confidence of the disparity estimation value, that is, a confidence smaller than the confidence threshold, if the disparity estimation value of the target pixel to be filled is directly adopted as the disparity value of the target pixel to be filled, the consistency of the disparity values in the region to be filled in the disparity map may be affected and an error may occur at the edge of the region to be filled in the disparity map.
In this application, for each target pixel to be filled, the disparity value of the target pixel to be filled may be determined by searching for an original pixel having the most similar pixel color of an original image pixel corresponding to the target pixel to be filled.
For each non-target pixel to be filled, the disparity estimation value of the non-target pixel to be filled can be used as the disparity value of the non-target pixel to be filled.
In the present application, a pixel to be filled that satisfies one of the following conditions may be determined as a target pixel to be filled for which the confidence of the disparity estimation value is low: the disparity value difference corresponding to the pixel to be filled is greater than the disparity value difference threshold, the original pixel value difference corresponding to the pixel to be filled is greater than the original pixel value difference threshold, the disparity value difference corresponding to the pixel to be filled is greater than the disparity value difference threshold, and the original pixel value difference corresponding to the pixel to be filled is less than the original pixel value difference threshold.
When the pixel to be filled satisfies Δ d>δdAnd Δ i>δi、Δd>δdAnd Δ i<δiAnd when one of the two conditions is satisfied, the pixel to be filled is a target pixel to be filled.
Taking a target pixel to be filled as an example, a process of determining a disparity value of the target pixel to be filled is described:
firstly, the area of the original image corresponding to the disparity map and associated with the target pixel to be filled is determined. The region associated with the target pixel to be filled is a region of a preset area centered on the original image pixel corresponding to the target pixel to be filled.
The region with the original image pixel corresponding to the target pixel to be filled as the center and the preset area may be a circular region with the original image pixel corresponding to the target pixel to be filled as the center and the radius as the preset length.
Then, an original pixel with the pixel color most similar to the pixel color of the original image pixel corresponding to the target pixel to be filled is found out from the area associated with the target pixel to be filled.
And the original image pixels in the area associated with the target pixel to be filled except the original image pixel corresponding to the target pixel to be filled are other original image pixels.
And the similarity between the pixel colors of other original image pixels in the area associated with the target pixel to be filled and the pixel color of the original image pixel corresponding to the target pixel to be filled is associated with the absolute value of the difference between the pixel value of the other original image pixel and the pixel value of the original image pixel corresponding to the target pixel to be filled. The larger the absolute value, the smaller the similarity.
For each other original image pixel, an absolute value of a difference between a pixel value of the other original image pixel and a pixel value of the original image pixel corresponding to the target pixel to be filled may be calculated, and the absolute value is taken as an absolute value corresponding to the other original image pixel.
The other original image pixel with the smallest corresponding absolute value may be used as the original image pixel with the most similar pixel color to the original image pixel corresponding to the target pixel to be filled in the region associated with the target pixel to be filled, so that the original image pixel with the most similar pixel color to the pixel color of the original image pixel corresponding to the target pixel to be filled in the region associated with the target pixel to be filled in the original image corresponding to the disparity map is found.
And finally, taking the parallax value of the pixel corresponding to the searched original pixel in the parallax map as the parallax value of the target pixel to be filled.
The original pixel with the pixel color most similar to the pixel color of the original image pixel corresponding to the target pixel to be filled is found out from the area associated with the target pixel to be filled, the parallax value of the pixel corresponding to the found original pixel is taken as the parallax value of the target pixel to be filled, which can be called as two-dimensional similarity filling.
In some embodiments, further comprising: after determining the disparity value of each pixel to be filled, the method further comprises the following steps: and performing median filtering processing on the area to be filled to remove salt and pepper noise in the area to be filled.
Considering that after the holes of the disparity map of the area to be filled are filled, a certain amount of salt and pepper noise may exist in the disparity area, performing median filtering on the area to be filled, for example, performing median filtering with a radius of 3 on the area to be filled, to remove the salt and pepper noise in the area to be filled.
Please refer to fig. 2, which shows a block diagram of a disparity map hole filling apparatus according to an embodiment of the present application. The disparity map hole filling device includes: an effective parallax pixel determination unit 201, a disparity amount calculation unit 202, a parallax estimation value calculation unit 203, and a parallax value filling unit 204.
The effective parallax pixel determination unit 201 is configured to determine, for each pixel to be filled in a region to be filled of a parallax map, a left effective parallax pixel of the pixel to be filled and a right effective parallax pixel of the pixel to be filled;
the difference calculating unit 202 is configured to calculate, for each pixel to be filled, a disparity value difference amount corresponding to the pixel to be filled and an original pixel value difference amount corresponding to the pixel to be filled, where the disparity value difference amount corresponding to the pixel to be filled is a difference amount between a disparity value of a left effective disparity pixel of the pixel to be filled and a disparity value of a right effective disparity pixel of the pixel to be filled, and the original pixel value difference amount corresponding to the pixel to be filled is a difference amount between a pixel value of an original pixel corresponding to the left effective disparity pixel of the pixel to be filled and a pixel value of an original image pixel corresponding to the right effective disparity pixel of the pixel to be filled;
the parallax estimation value calculation unit 203 is configured to calculate, for each pixel to be filled, a parallax estimation value of the pixel to be filled based on a parallax value difference amount corresponding to the pixel to be filled and an original pixel value difference amount corresponding to the pixel to be filled;
the disparity value filling unit 204 is configured to, for each pixel to be filled of at least some pixels to be filled, take the disparity estimation value of the pixel to be filled as the disparity value of the pixel to be filled.
In some embodiments, the disparity estimation value calculation unit 203 includes:
the first calculation module is used for calculating the average value of the parallax value of the left effective parallax pixel of the pixel to be filled and the parallax value of the right effective parallax pixel of the pixel to be filled when the parallax value difference corresponding to the pixel to be filled is smaller than the parallax value difference threshold and the original pixel value difference corresponding to the pixel to be filled is smaller than the original pixel value difference threshold;
and taking the calculated average value as the parallax estimated value of the pixel to be filled.
In some embodiments, the disparity estimation value calculation unit 203 includes:
a second calculating module configured to calculate a disparity estimation value of the pixel to be filled based on a disparity value difference amount corresponding to the pixel to be filled, a first distance corresponding to the pixel to be filled and a second distance corresponding to the pixel to be filled when the disparity value difference amount corresponding to the pixel to be filled is smaller than a disparity value difference amount threshold and an original pixel value difference amount corresponding to the pixel to be filled is larger than an original pixel value difference amount threshold, wherein the disparity value difference amount corresponding to the pixel to be filled is a difference amount between a disparity value of a left effective disparity pixel of the pixel to be filled and a disparity value of a right effective disparity pixel of the pixel to be filled, and the first distance is a distance between the pixel to be filled and the left effective disparity pixel of the pixel to be filled or the right effective disparity pixel of the pixel to be filled, the second distance is the sum of the distance between the pixel to be filled and the left effective parallax pixel of the pixel to be filled and the distance between the pixel to be filled and the right effective parallax pixel of the pixel to be filled.
In some embodiments, the second computing module is further configured to: calculating a distance ratio corresponding to the pixel to be filled, wherein the distance ratio corresponding to the pixel to be filled is a ratio of a first distance corresponding to the pixel to be filled to a second distance corresponding to the pixel to be filled; and calculating the parallax estimation value of the pixel to be filled based on the distance ratio corresponding to the pixel to be filled and the parallax difference corresponding to the pixel to be filled.
In some embodiments, the disparity estimation value calculation unit 203 includes:
and the third calculation module is configured to, when the disparity value difference corresponding to the pixel to be filled is greater than a disparity value difference threshold, take the smallest disparity value of the left effective disparity pixel of the pixel to be filled and the disparity value of the right effective disparity pixel of the pixel to be filled as the disparity estimation value of the pixel to be filled.
In some embodiments, the disparity map hole filling apparatus further comprises:
the searching unit is configured to, after regarding each pixel to be filled in at least part of pixels to be filled, use the parallax estimation value of the pixel to be filled as the parallax value of the pixel to be filled, for each target pixel to be filled in the region to be filled in the parallax map, find out an original image pixel having a pixel color most similar to the pixel color of the original image pixel corresponding to the target pixel to be filled from a region, associated with the target pixel to be filled, in the original image corresponding to the parallax map, wherein the target pixel to be filled is the pixel to be filled whose confidence coefficient of the parallax estimation value is smaller than a confidence coefficient threshold value, and the region associated with the target pixel to be filled is a region of a preset area with the original image pixel corresponding to the target pixel to be filled as a center; and taking the parallax value of the pixel corresponding to the searched original pixel in the parallax map as the parallax value of the target pixel to be filled.
In some embodiments, the disparity map hole filling apparatus further comprises:
and the filtering unit is configured to perform median filtering processing on the region to be filled so as to remove salt and pepper noise in the region to be filled.
Fig. 3 is a block diagram of an electronic device according to this embodiment. Electronic device 300 includes a processing component 322 that further includes one or more processors, and memory resources, represented by memory 332, for storing instructions, such as application programs, that are executable by processing component 322. The application programs stored in memory 332 may include one or more modules that each correspond to a set of instructions. Further, the processing component 322 is configured to execute instructions to perform the above-described methods.
The electronic device 300 may also include a power component 326 configured to perform power management of the electronic device 300, a wired or wireless network interface 350 configured to connect the electronic device 300 to a network, and an input/output (I/O) interface 358. The electronic device 300 may operate based on an operating system stored in the memory 332, such as Windows Server, MacOS XTM, UnixTM, LinuxTM, FreeBSDTM, or the like.
In an exemplary embodiment, a storage medium comprising instructions, such as a memory comprising instructions, executable by an electronic device to perform the above method is also provided. Alternatively, the storage medium may be a non-transitory computer readable storage medium, which may be, for example, a ROM, a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.
Other embodiments of the present application will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice within the art to which the invention pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the application being indicated by the following claims.
It will be understood that the present application is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the application is limited only by the appended claims.
Claims (10)
1. A disparity map hole filling method is characterized by comprising the following steps:
for each pixel to be filled in the area to be filled of the disparity map, determining a left effective disparity pixel of the pixel to be filled and a right effective disparity pixel of the pixel to be filled;
for each pixel to be filled, calculating a disparity value difference amount corresponding to the pixel to be filled and an original pixel value difference amount corresponding to the pixel to be filled, wherein the disparity value difference amount corresponding to the pixel to be filled is a difference amount between a disparity value of a left effective disparity pixel of the pixel to be filled and a disparity value of a right effective disparity pixel of the pixel to be filled, and the original pixel value difference amount corresponding to the pixel to be filled is a difference amount between a pixel value of an original pixel corresponding to the left effective disparity pixel of the pixel to be filled and a pixel value of an original image corresponding to the right effective disparity pixel of the pixel to be filled;
for each pixel to be filled, calculating a parallax estimation value of the pixel to be filled based on a parallax value difference corresponding to the pixel to be filled and an original pixel value difference corresponding to the pixel to be filled;
for each pixel to be filled in at least part of pixels to be filled, taking the parallax estimated value of the pixel to be filled as the parallax value of the pixel to be filled.
2. The method of claim 1, wherein calculating the disparity estimation value of the pixel to be filled based on the disparity value difference amount corresponding to the pixel to be filled and the original pixel value difference amount corresponding to the pixel to be filled comprises:
when the disparity value difference corresponding to the pixel to be filled is smaller than a disparity value difference threshold and the original pixel value difference corresponding to the pixel to be filled is smaller than an original pixel value difference threshold, calculating the average value of the disparity value of the left effective disparity pixel of the pixel to be filled and the disparity value of the right effective disparity pixel of the pixel to be filled;
and taking the calculated average value as the parallax estimated value of the pixel to be filled.
3. The method of claim 1, wherein calculating the disparity estimation value of the pixel to be filled based on the disparity value difference amount corresponding to the pixel to be filled and the original pixel value difference amount corresponding to the pixel to be filled comprises:
when the disparity value difference corresponding to the pixel to be filled is smaller than a disparity value difference threshold and the original pixel value difference corresponding to the pixel to be filled is larger than an original pixel value difference threshold, calculating a disparity estimation value of the pixel to be filled based on the disparity value difference corresponding to the pixel to be filled, a first distance corresponding to the pixel to be filled and a second distance corresponding to the pixel to be filled, wherein the first distance is a distance between the pixel to be filled and the left effective disparity pixel or the right effective disparity pixel, and the second distance is a sum of a distance between the pixel to be filled and the left effective disparity pixel and a distance between the pixel to be filled and the right effective disparity pixel.
4. The method of claim 3, wherein calculating the disparity estimation value of the pixel to be filled based on the disparity value difference corresponding to the pixel to be filled, the first distance corresponding to the pixel to be filled, and the second distance corresponding to the pixel to be filled comprises:
calculating a distance ratio corresponding to the pixel to be filled, wherein the distance ratio corresponding to the pixel to be filled is a ratio of a first distance corresponding to the pixel to be filled to a second distance corresponding to the pixel to be filled;
and calculating the parallax estimation value of the pixel to be filled based on the distance ratio corresponding to the pixel to be filled and the parallax difference corresponding to the pixel to be filled.
5. The method of claim 1, wherein calculating the disparity estimation value of the pixel to be filled based on the disparity value difference amount corresponding to the pixel to be filled and the original pixel value difference amount corresponding to the pixel to be filled comprises:
and when the disparity value difference corresponding to the pixel to be filled is greater than a disparity value difference threshold, taking the minimum disparity value of the left effective disparity pixel of the pixel to be filled and the disparity value of the right effective disparity pixel of the pixel to be filled as the disparity estimation value of the pixel to be filled.
6. The method according to one of claims 1 to 5, wherein after taking the disparity estimate value of the pixel to be filled as the disparity value of the pixel to be filled for each pixel to be filled of at least some pixels to be filled, the method further comprises:
for each target pixel to be filled in the area to be filled of the disparity map, finding out an original image pixel with the pixel color most similar to the pixel color of the original image pixel corresponding to the target pixel to be filled from an area, associated with the target pixel to be filled, in the original image corresponding to the disparity map, wherein the target pixel to be filled is the pixel to be filled with the disparity estimation value of which the confidence coefficient is smaller than a confidence coefficient threshold value, and the area associated with the target pixel to be filled is an area with a preset area and taking the original image pixel corresponding to the target pixel to be filled as the center; and taking the parallax value of the pixel corresponding to the searched original pixel in the parallax map as the parallax value of the target pixel to be filled.
7. The method of claim 6, wherein after determining the disparity value for each pixel to be filled, the method further comprises:
and carrying out median filtering processing on the area to be filled so as to remove salt and pepper noise in the area to be filled.
8. A disparity map hole filling apparatus, comprising:
an effective parallax pixel determination unit configured to determine, for each pixel to be filled in a region to be filled of a parallax map, a left effective parallax pixel of the pixel to be filled and a right effective parallax pixel of the pixel to be filled;
a difference amount calculation unit configured to calculate, for each pixel to be filled, a disparity value difference amount corresponding to the pixel to be filled and an original pixel value difference amount corresponding to the pixel to be filled, where the disparity value difference amount corresponding to the pixel to be filled is a difference amount between a disparity value of a left effective disparity pixel of the pixel to be filled and a disparity value of a right effective disparity pixel of the pixel to be filled, and the original pixel value difference amount corresponding to the pixel to be filled is a difference amount between a pixel value of an original pixel corresponding to the left effective disparity pixel of the pixel to be filled and a pixel value of an original image pixel corresponding to the right effective disparity pixel of the pixel to be filled;
a parallax estimation value calculation unit configured to calculate, for each of the pixels to be filled, a parallax estimation value of the pixel to be filled based on a parallax value difference amount corresponding to the pixel to be filled and an original pixel value difference amount corresponding to the pixel to be filled;
the parallax value filling unit is configured to take the parallax estimation value of each pixel to be filled in at least part of the pixels to be filled as the parallax value of the pixel to be filled.
9. An electronic device, comprising:
a processor;
a memory for storing the processor-executable instructions;
wherein the processor is configured to execute the instructions to implement the method of any one of claims 1 to 7.
10. A storage medium having instructions that, when executed by a processor of an electronic device, enable the electronic device to perform the method of any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010168220.0A CN111432194B (en) | 2020-03-11 | 2020-03-11 | Disparity map hole filling method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010168220.0A CN111432194B (en) | 2020-03-11 | 2020-03-11 | Disparity map hole filling method and device, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111432194A CN111432194A (en) | 2020-07-17 |
| CN111432194B true CN111432194B (en) | 2021-07-23 |
Family
ID=71547751
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010168220.0A Active CN111432194B (en) | 2020-03-11 | 2020-03-11 | Disparity map hole filling method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111432194B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2023225825A1 (en) * | 2022-05-23 | 2023-11-30 | 上海玄戒技术有限公司 | Position difference graph generation method and apparatus, electronic device, chip, and medium |
| CN114866758B (en) * | 2022-05-31 | 2024-02-23 | 星宸科技股份有限公司 | Parallax image filling method and image processing device |
Family Cites Families (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102307312B (en) * | 2011-08-31 | 2013-06-05 | 四川虹微技术有限公司 | Method for performing hole filling on destination image generated by depth-image-based rendering (DIBR) technology |
| CN102427547B (en) * | 2011-11-15 | 2013-11-06 | 清华大学 | Multi-angle stereo rendering apparatus |
| US9235879B2 (en) * | 2012-06-29 | 2016-01-12 | Hong Kong Applied Science And Technology Research Institute Co., Ltd. | Apparatus, system, and method for temporal domain hole filling based on background modeling for view synthesis |
| CN102930593B (en) * | 2012-09-28 | 2016-01-13 | 上海大学 | Based on the real-time drawing method of GPU in a kind of biocular systems |
| CN103051908B (en) * | 2012-12-26 | 2014-11-05 | 四川虹微技术有限公司 | Disparity map-based hole filling device |
| US10497140B2 (en) * | 2013-08-15 | 2019-12-03 | Intel Corporation | Hybrid depth sensing pipeline |
| CN104065946B (en) * | 2014-06-17 | 2015-10-21 | 四川虹微技术有限公司 | Based on the gap filling method of image sequence |
| CN104661014B (en) * | 2015-01-29 | 2016-09-28 | 四川虹微技术有限公司 | The gap filling method that space-time combines |
| CN109887008B (en) * | 2018-08-31 | 2022-09-13 | 河海大学常州校区 | Method, device and equipment for parallax stereo matching based on forward and backward smoothing and O (1) complexity |
-
2020
- 2020-03-11 CN CN202010168220.0A patent/CN111432194B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN111432194A (en) | 2020-07-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US7324687B2 (en) | Color segmentation-based stereo 3D reconstruction system and process | |
| CN109447154B (en) | Picture similarity detection method, device, medium and electronic equipment | |
| US8279930B2 (en) | Image processing apparatus and method, program, and recording medium | |
| Cigla et al. | Information permeability for stereo matching | |
| CN111432194B (en) | Disparity map hole filling method and device, electronic equipment and storage medium | |
| US9196021B2 (en) | Video enhancement using related content | |
| CN113793356B (en) | Lane line detection method and device | |
| Hsia | Improved depth image-based rendering using an adaptive compensation method on an autostereoscopic 3-D display for a Kinect sensor | |
| Wang et al. | Reference-free DIBR-synthesized video quality metric in spatial and temporal domains | |
| US9659372B2 (en) | Video disparity estimate space-time refinement method and codec | |
| CN112967191B (en) | Image processing method, device, electronic equipment and storage medium | |
| GB2600808A (en) | Denoising images rendered using Monte Carlo renderings | |
| Mukherjee et al. | A hybrid algorithm for disparity calculation from sparse disparity estimates based on stereo vision | |
| CN112053394A (en) | Image processing method, image processing device, electronic equipment and storage medium | |
| Shabanian et al. | A new hybrid stereo disparity estimation algorithm with guided image filtering-based cost aggregation | |
| JP2005339535A (en) | Calculation of dissimilarity measure | |
| Islam et al. | Robust enhancement of depth images from depth sensors | |
| EP2927872A1 (en) | Method and device for processing a video sequence | |
| US20210295056A1 (en) | Method, apparatus, electronic device, and computer storage medium for video processing | |
| CN111476810B (en) | Image edge detection method and device, electronic equipment and storage medium | |
| Seychell et al. | Efficient object selection using depth and texture information | |
| CN113538467A (en) | Image segmentation method and device and training method and device of image segmentation model | |
| CN110740308B (en) | Time consistent reliability delivery system | |
| JP6827317B2 (en) | Image processing equipment and programs | |
| Xu et al. | Automatic Depth Prediction from 2D Videos Based on Non-parametric Learning and Bi-directional Depth Propagation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |