JP2014211829A - Image processing apparatus, image processing circuit, image processing program, and image processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To make it possible to estimate an occlusion area accurately at a high speed.SOLUTION: In an image processing apparatus 10, an estimation unit 12 moves windows in a depth map acquired by an acquisition unit 11. The estimation unit 12 determines whether or not an attention pixel is a pixel in an occlusion area on the basis of the total sum of the differences between distances corresponding to the attention pixel in each of the windows and distances corresponding to each of non-attention pixels, and estimates the occlusion area.

Description

  The present invention relates to an image processing apparatus, an image processing circuit, an image processing program, and an image processing method.

  When viewing a 3D (three dimensions) image, the user may feel uncomfortable or uncomfortable. When a person views an object in real space, the person's focus is in the vicinity of the object in the central visual field. For this reason, an object outside the central visual field is recognized in a state of being out of focus. In other words, in a real space, it is rare to recognize a plurality of objects having greatly different distances from a person at the same level. For this reason, when a plurality of objects having greatly different depths (that is, corresponding to distances) are viewed in the image, the observer feels tired because the degree of contradiction between convergence and adjustment is large.

  Therefore, in order to suppress the degree of contradiction between congestion and adjustment in this situation, a filtering process is generally performed. In this filtering process, a distance image is used. For example, the distance image includes two images taken from two viewpoints (hereinafter, an image taken from the right viewpoint is referred to as a “right image”, and an image taken from the left viewpoint is illustrated in FIG. (Referred to as “left image”). FIG. 1 is a diagram illustrating an example of a left image and a right image. Specifically, the origin of the right image and the origin of the left image are matched, and the difference (that is, parallax) between the coordinates of the reference point of the right image and the coordinates of the corresponding point of the left image corresponding to the reference point is An image having the reference point value is a “right-side distance image”. Conversely, the origin of the right image is aligned with the origin of the left image, and the difference between the coordinates of the reference point of the left image and the coordinates of the corresponding point of the right image corresponding to the reference point (that is, parallax) The image with the value of is the “left distance image”. FIG. 2 is a diagram illustrating an example of the left distance image. The left distance image shown in FIG. 2 is created based on the right image and the left image shown in FIG. In the distance image, the value of each pixel (that is, the parallax value) is indicated by shading, and the darker means farther and the thinner it means closer.

  Here, it is desired to improve the accuracy of the distance image in order to improve the quality of the image subjected to the filtering process. In order to improve the accuracy of the distance image, it is desired to estimate the “occlusion region” in the distance image with high accuracy. The “occlusion area” is an image area that exists in one of the right image and the left image but does not exist in the other. In FIG. 2, the portion indicated by an ellipse is an occlusion region.

  Conventionally, there is a method of performing block matching processing as a method of estimating an occlusion region in a distance image with high accuracy. In this conventional method, as shown in FIG. 3, with reference to one of the right distance image and the left distance image, it is determined whether or not each of the one partial region exists in the other, and the non-existing region is determined. Specify as an occlusion area. FIG. 3 is a diagram for explaining a conventional method of estimating an occlusion area by block matching processing.

JP 2002-099918 A JP2012-010255A JP 2012-089136 A

  However, in the conventional occlusion region estimation method, first, a left distance image and a right distance image are created by block matching processing, and further, block matching processing is performed between the left distance image and the right distance image. Since this block matching process has a large amount of calculation, it takes a long time for the estimation process.

  The disclosed technique has been made in view of the above, and an object thereof is to provide an image processing apparatus, an image processing circuit, an image processing program, and an image processing method capable of estimating an occlusion area with high accuracy and high speed. To do.

  In the aspect of the disclosure, the window is moved in the distance image based on the first image and the second image having a different viewpoint from the first image, and the distance corresponding to the target pixel in each window and each non-target pixel The occlusion area is estimated by determining whether or not the pixel of interest is a pixel in the occlusion area based on the sum of the differences from the distance corresponding to.

  According to the disclosed aspect, the occlusion area can be estimated with high accuracy and high speed.

FIG. 1 is a diagram illustrating an example of a left image and a right image. FIG. 2 is a diagram illustrating an example of the left distance image. FIG. 3 is a diagram for explaining an occlusion region estimation method using block matching processing. FIG. 4 is a block diagram illustrating an example of the image processing apparatus according to the first embodiment. FIG. 5 is a flowchart illustrating an example of a processing operation of the image processing apparatus according to the first embodiment. FIG. 6 is a diagram illustrating an example of an occlusion area estimation result by the image processing apparatus according to the first embodiment. FIG. 7 is a block diagram illustrating an example of an image processing apparatus according to the second embodiment. FIG. 8 is a flowchart illustrating an example of a processing operation of the image processing apparatus according to the second embodiment. FIG. 9 is a diagram for explaining an estimated width calculation process in the image processing apparatus according to the second embodiment. FIG. 10 is a diagram for explaining an estimated width calculation process in the image processing apparatus according to the second embodiment. FIG. 11 is a block diagram illustrating an example of an image processing apparatus according to the third embodiment. FIG. 12 is a flowchart illustrating an example of a processing operation of the image processing apparatus according to the third embodiment. FIG. 13 is a diagram for explaining the estimation width calculation process in the image processing apparatus according to the third embodiment. FIG. 14 is a diagram illustrating an example of a distance image after the occlusion area correction process according to the third embodiment. FIG. 15 is a diagram illustrating a hardware configuration example of the image processing apparatus.

  Hereinafter, embodiments of an image processing apparatus, an image processing circuit, an image processing program, and an image processing method disclosed in the present application will be described in detail with reference to the drawings. Note that the image processing apparatus, the image processing circuit, the image processing program, and the image processing method disclosed in the present application are not limited by this embodiment. Moreover, the same code | symbol is attached | subjected to the structure which has the same function in embodiment, and the overlapping description is abbreviate | omitted. In the embodiment, equivalent processing steps are denoted by the same reference numerals, and redundant description is omitted.

[Example 1]
[Configuration example of image processing apparatus]
FIG. 4 is a block diagram illustrating an example of the image processing apparatus according to the first embodiment. In FIG. 4, the image processing apparatus 10 includes an acquisition unit 11, an estimation unit 12, and a correction unit 13.

  The acquisition unit 11 includes a first image (for example, a left image) captured at a first viewpoint, and a second image (for example, a right image) captured at a second viewpoint different from the first viewpoint. A distance image (for example, a left distance image or a right distance image) is acquired. Here, the description will be made on the assumption that the distance image is calculated outside the image processing apparatus 10, but the present invention is not limited to this, and the acquisition unit 11 obtains the first image and the second image. The distance image may be calculated (formed) based on the acquired first image and second image.

  The estimation unit 12 estimates the occlusion region in the distance image acquired by the acquisition unit 11 based on the distance variation between adjacent pixels in the distance image.

  Specifically, the estimation unit 12 sequentially moves “windows” in the distance image acquired by the acquisition unit 11. The window has a size corresponding to, for example, nine pixels including one “target pixel” and eight pixels (non-target pixel) surrounding the target pixel. Then, when the estimation unit 12 sets a window at a certain position of the distance image, the pixel corresponding to the center of the window is set as the target pixel, and the difference between the distance corresponding to the target pixel and the distance corresponding to each non-target pixel is calculated. calculate. That is, eight differences are calculated. Then, the estimation unit 12 calculates the sum of the calculated eight differences. And the estimation part 12 estimates an occlusion area | region by determining whether the attention pixel corresponding to the said sum total is a pixel of an occlusion area | region based on the calculated sum total.

  Specifically, when the calculated sum is larger than the threshold, the estimation unit 12 determines that the pixel of interest corresponding to the sum is a pixel in the occlusion area. On the other hand, when the calculated sum is equal to or less than the threshold, the estimation unit 12 determines that the pixel of interest corresponding to the sum is not a pixel in the occlusion area. That is, the estimation unit 12 determines that the target pixel corresponding to the sum is a pixel in the “unstable region” when the calculated sum is larger than the threshold, and the “stable region” when the calculated sum is equal to or less than the threshold. It is determined that the pixel is. That is, the “unstable region” is a region where the distance variation is larger than that of the “stable region”. Then, the estimating unit 12 determines that the “unstable region” is an occlusion region.

  The correction unit 13 corrects the distance image based on the occlusion area estimated by the estimation unit 12.

[Operation example of image processing apparatus]
An operation example of the image processing apparatus 10 having the above configuration will be described. FIG. 5 is a flowchart illustrating an example of a processing operation of the image processing apparatus according to the first embodiment. FIG. 6 is a diagram illustrating an example of an occlusion area estimation result by the image processing apparatus according to the first embodiment.

  In the image processing apparatus 10, the estimation unit 12 sets a window at the initial position in the distance locus image acquired by the acquisition unit 11 (step S101). The initial position is, for example, (1, 1) pixels when the pixel in the leftmost column (X ′ coordinate) and the bottom row (Y ′ coordinate) of the distance image is (0, 0) pixels. This is the position where the center of the window is aligned.

  The estimation unit 12 calculates the difference between the distance corresponding to the target pixel in the window set in step S101 and the distance corresponding to each non-target pixel (step S102).

  The estimation unit 12 calculates the sum of the differences calculated in step S102 (step S103).

  The estimation unit 12 determines whether or not the sum calculated in step S103 is larger than a predetermined threshold (step S104).

  When the sum is larger than the threshold value (Yes at Step S104), the estimation unit 12 determines that the target pixel is a pixel in the occlusion area, and sets “1” as the target pixel (Step S105).

  On the other hand, when the sum is equal to or less than the threshold (No at Step S104), the estimation unit 12 determines that the target pixel is not a pixel in the occlusion area, and sets “0” to the target pixel (Step S106). FIG. 6 shows an example of a distance image in which “1” or “0”, which is information related to the occlusion area, is set for each pixel of interest. In the distance image obtained here, in addition to the distance information, information related to the occlusion area is set.

  Then, the estimation unit 12 determines whether or not the currently set window position is the final position (step S107). If the window position is not the final position (No in step S107), the estimation unit 12 moves the window according to the window movement rule. (Step S108). And the estimation part 12 repeats the process of step S102-S107 in the position after a movement. Here, for example, the window moving rule moves the window one pixel at a time to the right, and when the window reaches the right end of the distance image, moves the window up one pixel while returning the window to the left end of the distance image. For example, when the distance image has a size of M pixels × N pixels, the final position is a position where the center of the window is aligned with (M−1, N−1) pixels.

  If the estimation unit 12 determines that the currently set window position is the final position (Yes in step S107), the estimation unit 12 ends the occlusion area estimation process.

  As described above, according to the present embodiment, in the image processing apparatus 10, the estimation unit 12 sequentially moves windows within the distance image acquired by the acquisition unit 11. Then, the estimation unit 12 determines whether or not the target pixel is a pixel in the occlusion area based on the sum of the differences between the distance corresponding to the target pixel and the distance corresponding to each non-target pixel in each window. To estimate the occlusion area.

  With the configuration of the image processing apparatus 10, the occlusion area can be estimated using one of the left distance image and the right distance image, so that the amount of calculation of the estimation process can be reduced. Further, based on a parameter that can be calculated with a small amount of calculation, that is, the sum of the difference between the distance corresponding to the target pixel and the distance corresponding to each non-target pixel in each window, and having a specific value, The occlusion area can be estimated. Therefore, the occlusion area can be estimated at high speed and with high accuracy.

  In the image processing apparatus 10, the correction unit 13 corrects the distance image based on the occlusion area estimated by the estimation unit 12.

  With this configuration of the image processing apparatus 10, the distance image can be corrected based on the occlusion area accurately estimated by the estimation unit 12, so that the accuracy of the distance image can be improved. For example, by using the distance image with improved accuracy and performing filtering processing on the right image and the left image, the degree of contradiction between the congestion and the adjustment can be reduced. As a result, it is possible to provide a 3D image that can reduce the feeling of fatigue felt by the observer.

[Example 2]
In the second embodiment, the estimated width of the occlusion area is calculated, and the estimated occlusion area is corrected based on the estimated width.

[Configuration example of image processing apparatus]
FIG. 7 is a block diagram illustrating an example of an image processing apparatus according to the second embodiment. In FIG. 7, the image processing apparatus 30 includes a width calculation unit 31 and a correction unit 32.

  The width calculation unit 31 estimates the occlusion region based on the distance corresponding to each of the first pixel and the second pixel located on both sides (that is, left and right) across the occlusion region estimated by the estimation unit 12. Calculate the width. The width calculation unit 31 calculates the estimated width of the occlusion area for each row (that is, the Y ′ coordinate) of the occlusion area estimated by the estimation unit 12. Hereinafter, the first pixel and the second pixel may be referred to as a “left pixel” and a “right pixel”, respectively. Further, the distance corresponding to the “left pixel” may be referred to as “first distance”, and the distance corresponding to the “right pixel” may be referred to as “second distance”.

  The correction unit 32 corrects the occlusion area estimated by the estimation unit 12 based on the estimated width calculated by the width calculation unit 31. The width calculation unit 31 corrects the occlusion region for each row (that is, Y ′ coordinate) of the occlusion region estimated by the estimation unit 12.

  Then, the correction unit 32 corrects the distance image based on the corrected occlusion area.

[Operation example of image processing apparatus]
An operation example of the image processing apparatus 30 having the above configuration will be described. FIG. 8 is a flowchart illustrating an example of a processing operation of the image processing apparatus according to the second embodiment. FIGS. 9 and 10 are diagrams for explaining the estimated width calculation process in the image processing apparatus according to the second embodiment.

  In the image processing apparatus 30, the width calculation unit 31 sets the “target partial region” as an initial position for the occlusion region estimated by the estimation unit 12 (step S 201). The target partial region is a pixel group having the same Y ′ coordinate in the occlusion region. The initial position of the target partial area is the position where the Y ′ coordinate in the occlusion area estimated by the estimation unit 12 is the minimum value (see FIG. 9).

  The width calculation unit 31 specifies the first distance corresponding to the left pixel and the second distance corresponding to the right pixel for the target partial region set in step S201 (step S202). That is, as illustrated in FIG. 9, the width calculation unit 31 identifies the first distance corresponding to the pixel p1 and the second distance corresponding to the pixel p2 for the target partial region at the initial position.

The width calculation unit 31 calculates the estimated width α based on the first distance and the second distance specified in step S202 (step S203). Specifically, the width calculation unit 31 sets the larger one of the first distance and the second distance as Z1, sets the smaller one as Z0, and substitutes the estimated width α in the following equation (1). Is calculated.
α = b × (Z1-Z0) / Z0 (1)
b is the distance between the left camera that captured the left image and the right camera that captured the right image.

Here, as shown in FIG. 10, when a front object (foreground) exists at a position in front of the rear object (background) (P (X, Z) in FIG. 10), the distance from the camera position to the front object ( Z0), the distance from the camera position to the rear object (Z1), and the separation distance b between the left and right cameras are determined. In FIG. 10, O ₁ indicates the left projection plane, and _Or indicates the right projection plane. F is the distance from the left camera position to the left projection plane, or the distance from the right camera position to the right projection plane. _Xl is the distance between the intersection of the line segment connecting the position of the left camera and P (X, Z) and the left projection plane and the center point of the left projection plane. _Xr is the distance between the intersection of the line segment connecting the right camera position and P (X, Z) and the right projection plane and the center point of the right projection plane.

  Returning to FIG. 8, the correction unit 32 compares the set width of the target partial region with the estimated width α, and determines whether or not the set width of the target partial region is larger than the estimated width α (step S <b> 204). Here, for example, the estimated width α itself calculated in step S203 is not used, but a value obtained by converting the estimated width α of the real space calculated in step S203 into a distance image space may be used.

  When the width of the target partial region is larger than the estimated width α (Yes at Step S204), the correcting unit 32 corrects the occlusion region based on the estimated width α (Step S205). That is, the correction unit 32 changes the pixel state from the pixel at the end of the target partial region to the pixel in the stable region until the width of the target partial region reaches the estimated width α. Specifically, the correction unit 32 stabilizes the pixel state from the pixel in the unstable region in order from the right end pixel so that the pixel from the left end of the target partial region to the pixel corresponding to the estimated width α is a stable region. You may change into the pixel of an area | region. Alternatively, the correcting unit 32 changes the state of the pixels in order from the leftmost pixel to the stable region pixel so that the pixel from the rightmost pixel of the target partial region to the pixel corresponding to the estimated width α is a stable region. You may change to Alternatively, the correction unit 32 changes the state of the pixels in order from both the right end and the left end so that the pixel group corresponding to the estimated width α existing in the center of the target partial region becomes a pixel in the stable region. May be changed to a pixel in the stable region.

  Moreover, the correction | amendment part 32 does not correct | amend the width | variety of an attention partial area | region, when the width | variety of an attention partial area | region is below the estimation width | variety (alpha) (step S204 negative). Then, the process proceeds to step S206. Here, the target partial region after the correction in step S205 or the target partial region when the width of the target partial region is equal to or smaller than the estimated width α is the determined occlusion region.

  The width calculation unit 31 determines whether or not the current target partial region corresponds to the final position (step S206). The final position of the target partial area is a position where the Y ′ coordinate in the occlusion area estimated by the estimation unit 12 is the maximum value.

  When determining that the position is not the final position (No at Step S206), the width calculating unit 31 moves the target partial area according to the movement rule of the target partial area (Step S207). Then, the width calculation unit 31 repeats the processes of steps S202, S203, and S206, and the correction unit 32 repeats the processes of steps S204 and S205.

  If the width calculation unit 31 determines that the position is the final position (Yes at step S206), the occlusion area correction process is terminated.

  As described above, according to the present exemplary embodiment, in the image processing apparatus 30, the width calculation unit 31 includes each of the first pixel and the second pixel that are located on both sides of the occlusion region estimated by the estimation unit 12. The estimated width of the occlusion area is calculated based on the distance corresponding to. Then, the correction unit 32 corrects the occlusion area estimated by the estimation unit 12 based on the estimated width calculated by the width calculation unit 31.

  With the configuration of the image processing device 30, the estimated width of the occlusion area can be accurately calculated based on the distance corresponding to the pixels of the stable area located on both sides of the occlusion area. Since the estimated occlusion area can be corrected based on the estimated width calculated with high accuracy, the accuracy of the estimated occlusion area can be further improved.

  Then, the correction unit 32 corrects the distance image based on the corrected occlusion area.

  With the configuration of the image processing device 30, the distance image can be corrected based on the occlusion region with improved accuracy, and thus the accuracy of the distance image can be improved. For example, by using the distance image with improved accuracy and performing filtering processing on the right image and the left image, the degree of contradiction between the congestion and the adjustment can be reduced. As a result, it is possible to provide a 3D image that can reduce the feeling of fatigue felt by the observer.

[Example 3]
In the third embodiment, as in the second embodiment, the estimated width of the occlusion area is calculated, and the estimated occlusion area is corrected based on the estimated width. However, in the second embodiment, for example, information on the occlusion area shown in FIG. 6 is corrected. In the third embodiment, in the distance image, the estimated occlusion area is corrected and the distance in the estimated occlusion area is corrected. I do.

[Configuration example of image processing apparatus]
FIG. 11 is a block diagram illustrating an example of an image processing apparatus according to the third embodiment. In FIG. 11, the image processing apparatus 50 includes a correction unit 51.

  For the occlusion region estimated by the estimation unit 12, the correction unit 51 includes a “first region” having the estimated width α calculated by the width calculation unit 31 and a “second region” excluding the first region. Is set. Here, the first area is a confirmed occlusion area.

  Then, the correction unit 51 corrects the distance corresponding to the pixel closer to the left pixel than the right pixel in the second region by the first distance. Further, the correction unit 51 corrects the distance corresponding to the pixel closer to the right pixel than the left pixel in the second region by the second distance. Further, the correction unit 51 corrects the distance corresponding to the pixel in the first region by the larger one of the first distance and the second distance.

  When the width of the occlusion region estimated by the estimation unit 12 is equal to or less than the estimated width α, the correction unit 51 sets the distance corresponding to the pixel in the occlusion region estimated by the estimation unit 12 to the first distance. And the larger one of the second distances is corrected.

[Operation example of image processing apparatus]
An operation example of the image processing apparatus 50 having the above configuration will be described. FIG. 12 is a flowchart illustrating an example of a processing operation of the image processing apparatus according to the third embodiment. FIG. 13 is a diagram for explaining the estimation width calculation process in the image processing apparatus according to the third embodiment. FIG. 14 is a diagram illustrating an example of a distance image after the occlusion area correction process according to the third embodiment.

  In the image processing device 50, when the width of the target partial region is larger than the estimated width α (Yes in step S204), the correcting unit 51 includes a first region having the estimated width α in the center of the target partial region, and the first region A second area excluding is set (step S301). For example, the first area A401 and the second area A402 shown in the right figure are set for the target partial area shown in the left figure of FIG. 13 (the estimated occlusion area A400 in the figure). The second area A402 includes an area A403 and an area A404.

  Then, the correction unit 51 corrects the distance corresponding to the pixel closer to the left pixel than the right pixel in the second region A402 (that is, the pixel in the region A403) by the first distance (step S302). . Further, the correction unit 51 corrects the distance corresponding to the pixel closer to the right pixel than the left pixel in the second region A402 (that is, the pixel in the region A404) by the second distance (step S302). . Further, the correction unit 51 corrects the distance corresponding to the pixel in the first area A401 by the larger one of the first distance and the second distance (step S302).

  On the other hand, when the width of the target partial area is equal to or smaller than the estimated width α (No in step S204), the correcting unit 51 sets the distance corresponding to the pixel of the target partial area as the larger one of the first distance and the second distance. (Step S303).

  When the correction unit 51 performs the above correction processing, for example, a corrected distance image as shown in FIG. 14 is obtained.

  In the above description, the case where the first area is set at the center of the target partial area has been described. However, the present invention is not limited to this. For example, the first area may be set according to the right end of the target partial area, or the first area may be set according to the left end of the target partial area.

  As described above, according to the present embodiment, in the image processing apparatus 50, the width calculation unit 31 includes each of the first pixel and the second pixel that are located on both sides of the occlusion region estimated by the estimation unit 12. The estimated width of the occlusion area is calculated based on the distance corresponding to. Then, the correction unit 51 sets a first region having the estimated width calculated by the width calculation unit 31 and a second region excluding the first region for the occlusion region estimated by the estimation unit 12. Then, the correction unit 51 corrects the distance corresponding to the pixel closer to the first pixel than the second pixel in the second region by the first distance. Then, the correction unit 51 corrects the distance corresponding to the pixel closer to the second pixel than the first pixel by the second distance. Then, the correction unit 51 corrects the distance corresponding to the pixel in the first region by the larger one of the first distance and the second distance.

  With the configuration of the image processing device 50, the estimated width of the occlusion area can be calculated with high accuracy based on the distance corresponding to the pixels of the stable area located on both sides of the occlusion area. Since the estimated occlusion area can be corrected based on the estimated width calculated with high accuracy, the accuracy of the estimated occlusion area can be further improved. In addition, since the distance corresponding to the estimated pixel of the occlusion area can be corrected using the distance corresponding to the pixel of the stable area, the accuracy of the distance image can be improved. For example, by using the distance image with improved accuracy and performing filtering processing on the right image and the left image, the degree of contradiction between the congestion and the adjustment can be reduced. As a result, it is possible to provide a 3D image that can reduce the feeling of fatigue felt by the observer.

[Other embodiments]
[1] In the first to third embodiments, the description has been made on the assumption that there is one estimated occlusion region. However, the present invention is not limited to this. When there are a plurality of estimated occlusion areas, the processing described in the first to third embodiments may be performed on each occlusion area.

  [2] Each component of each part illustrated in the first to third embodiments is not necessarily physically configured as illustrated. In other words, the specific form of distribution / integration of each part is not limited to the one shown in the figure, and all or a part thereof may be functionally or physically distributed / integrated in arbitrary units according to various loads and usage conditions. Can be configured.

  Furthermore, various processing functions performed in each device are performed on a CPU (Central Processing Unit) (or a microcomputer such as an MPU (Micro Processing Unit), MCU (Micro Controller Unit), etc.) in whole or in part. You may make it perform. Various processing functions may be executed entirely or arbitrarily on a program that is analyzed and executed by a CPU (or a microcomputer such as an MPU or MCU) or hardware based on wired logic. .

  The image processing apparatuses according to the first to third embodiments can be realized by the following hardware configuration, for example.

  FIG. 15 is a diagram illustrating a hardware configuration example of the image processing apparatus. As illustrated in FIG. 15, the image processing apparatus 100 includes a processor 101, a memory 102, a camera 103, and a display device 104.

  Examples of the processor 101 include a central processing unit (CPU), a digital signal processor (DSP), and a field programmable gate array (FPGA). Further, examples of the memory 102 include a RAM (Random Access Memory) such as an SDRAM (Synchronous Dynamic Random Access Memory), a ROM (Read Only Memory), a flash memory, and the like.

  Various processing functions performed by the image processing apparatuses according to the first to third embodiments may be realized by executing programs stored in various memories such as a nonvolatile storage medium by a processor included in the amplification apparatus. . That is, a program corresponding to each process executed by the acquisition unit 11, the estimation unit 12, the width calculation unit 31, and the correction units 13, 32, and 51 is recorded in the memory 102, and each program is executed by the processor 101. May be. Further, the right image and the left image may be taken by the camera 103. Further, the right image, the left image, the distance image, and the corrected distance image may be displayed on the display device 104.

  The processor 101 may be realized as a one-chip image processing circuit.

10, 30, 50 Image processing apparatus 11 Acquisition unit 12 Estimation unit 13, 32, 51 Correction unit 31 Width calculation unit

Claims (7)

A memory and a processor connected to the memory;
The processor is
Moving a window within a distance image based on a first image and a second image having a different viewpoint from the first image;
Based on the sum of the difference between the distance corresponding to the target pixel in each window and the distance corresponding to each non-target pixel, it is determined whether the target pixel is a pixel in the occlusion area, and the occlusion area is determined. presume,
An image processing apparatus. The processor is
Based on the distance corresponding to each of the first pixel and the second pixel located on both sides across the estimated occlusion area, the estimated width of the occlusion area is calculated,
Correcting the estimated occlusion area based on the estimated width;
The image processing apparatus according to claim 1. The processor is
Based on the distance corresponding to each of the first pixel and the second pixel located on both sides across the estimated occlusion area, the estimated width of the occlusion area is calculated,
A first area having the estimated width and a second area excluding the first area with respect to the estimated occlusion area;
The distance corresponding to the pixel closer to the first pixel than the second pixel in the second region is corrected by the distance corresponding to the first pixel, and the first pixel is corrected to the first pixel. A distance corresponding to a pixel close to the second pixel is corrected with a distance corresponding to the second pixel, and a distance corresponding to a pixel in the first region is corrected to a distance corresponding to the first pixel and the first Correct with the larger of the distances corresponding to 2 pixels,
The image processing apparatus according to claim 1. The processor determines that the pixel of interest corresponding to the sum is a pixel in the occlusion region when the sum is greater than a threshold value.
The image processing apparatus according to claim 1, wherein the image processing apparatus is an image processing apparatus. Moving a window within a distance image based on a first image and a second image having a different viewpoint from the first image;
Based on the sum of the difference between the distance corresponding to the target pixel in each window and the distance corresponding to each non-target pixel, it is determined whether the target pixel is a pixel in the occlusion area, and the occlusion area is determined. presume,
Image processing circuit. Moving a window within a distance image based on a first image and a second image having a different viewpoint from the first image;
Based on the sum of the difference between the distance corresponding to the target pixel in each window and the distance corresponding to each non-target pixel, it is determined whether the target pixel is a pixel in the occlusion area, and the occlusion area is determined. presume,
An image processing program for causing an image processing apparatus to execute processing. Moving a window within a distance image based on a first image and a second image having a different viewpoint from the first image;
Based on the sum of the difference between the distance corresponding to the target pixel in each window and the distance corresponding to each non-target pixel, it is determined whether the target pixel is a pixel in the occlusion area, and the occlusion area is determined. presume,
Image processing method.

Images