JP2015141615A - Image processor and method thereof - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve the estimation accuracy of distance information near a boundary between a foreground region and a background region of an image.SOLUTION: An input part inputs image data obtained by photographing a subject, the subject and distance information of the background of the subject. A boundary information generation part generates boundary information between the foreground region and the background region of the image data. A correction region setting part sets a correction region for correcting the distance information on the basis of the boundary information. A transparency calculation part calculates the transparency of a pixel of the correction region. A distance information correction part corrects distance information corresponding to the pixel of the correction region on the basis of the transparency of the pixel of the correction region.

Description

  The present invention relates to image processing of image data having distance information.

  The distance information (depth information) in an image can be estimated using a distance image sensor mounted on a monocular camera or multi-viewpoint image information acquired from a plurality of cameras. Depth information is applied to a wide range of fields, such as conversion from 2D images to 3D images, medical technology, robot control, and 3D television stereoscopic technology.

  Most of the estimation of distance information uses a method of estimating a distance using a plurality of images. This method uses multiple cameras to photograph subjects (objects and landscapes) from different viewpoints and different distances, and estimates the distance from the geometric relationship between the cameras and the subjects and the displacement of the subject pixels in the image. To do.

  On the other hand, a monocular method for estimating a distance from a single two-dimensional still image has been studied. In monocular viewing, the distance is estimated using the texture, contrast, etc. of the image.

However, each method includes several factors that affect the estimation accuracy of distance information.
(1) In a region where there is no texture in the object, the matching degree of pixels having the same color becomes ambiguous, and the distance estimation accuracy decreases.
(2) An object that undergoes specular reflection or a low-luminance region has a large luminance value or color that changes when the viewpoint changes, making it difficult to measure the degree of coincidence by color and affects distance estimation.
(3) The subject has an area that is hidden by the observation position. In other words, if there is a state (occlusion) in which the object in the foreground hides the object behind it, not all pixels have corresponding points, and searching for corresponding points in the hidden area will cause distance estimation errors. growing.
(4) When the contour of an object is unclear, distance estimation near the contour becomes inaccurate.

  Regarding factors 1 to 3, there are many studies, and distance estimation accuracy has been improved. However, the problem of distance estimation regarding factor 4 has not been improved.

  The problem of distance estimation will be described with reference to FIG. Fig. 1 (A) shows an example of an image consisting of an object area and a background area.The object area close to the imaging device overlaps the background area far from the imaging device, the object area has a texture pattern, and a part of the object area is It is an example of an image having color information with a close background. In such an image, due to factor 4, distance estimation near the contour becomes inaccurate.

  FIG. 1B shows an estimation result of ideal distance information. The distance from the photographing apparatus is presented as a difference in lightness, and a subject with higher lightness is closer to the photographing apparatus. In FIG. 1B, the background area and the object area have uniform distance values. The black line shown in FIG. 1B is a boundary line that represents the boundary of distance information between the background region and the object region, and the boundary line basically corresponds to the boundary between the background region and the object region.

  FIG. 1C shows an example of the estimation result of distance information. It can be said that the estimation result of distance information is better as it is closer to FIG. However, as shown in Fig. 1 (C), near the boundary between the background area and the object area, the object area enters the background area, or conversely, the background area enters the object area. However, there are areas that are not sufficient.

JP 2012-079251 A

  An object of the present invention is to improve the estimation accuracy of distance information in the vicinity of the boundary between a foreground region and a background region of an image.

  The present invention has the following configuration as one means for achieving the above object.

  The image processing according to the present invention inputs image data obtained by photographing a subject, and distance information of the subject and the background of the subject, generates boundary information between a foreground region and a background region of the image data, and Based on the information, a correction area for correcting the distance information is set, the transparency of the pixel in the correction area is calculated, and the distance information corresponding to the pixel is corrected based on the transparency of the pixel in the correction area.

  According to the present invention, it is possible to improve the estimation accuracy of distance information in the vicinity of the boundary between the foreground region and the background region of the image.

The figure explaining the subject of distance estimation. FIG. 3 is a block diagram illustrating a configuration example of a photographing apparatus having an image processing function according to an embodiment. The block diagram explaining the process structural example of an image process part. The flowchart explaining the correction process of the distance information by an image process part. The figure which shows the example of a setting of a correction | amendment area | region. The flowchart explaining the process of a transparency calculation part. The figure which shows an example of transparency (alpha). The flowchart explaining the process of a distance information correction | amendment part. The figure which shows the example of correction | amendment of distance information. The flowchart explaining the process of the distance information correction | amendment part which correct | amends the distance information corresponding to an attention pixel in steps. 9 is a flowchart for explaining distance information correction processing by an image processing unit according to the second embodiment. The flowchart explaining the process of a correction area | region setting part. 10 is a flowchart for explaining distance information correction processing by the image processing unit according to the third embodiment. The flowchart explaining correction of the correction area | region by a correction area | region setting part.

  Hereinafter, image processing according to an embodiment of the present invention will be described in detail with reference to the drawings. In the following, image processing for improving the estimation accuracy of distance information according to the present invention will be described. However, the constituent elements described in the embodiments are exemplifications, and the scope of the present invention is determined by the claims. It is not limited by the individual embodiments.

[Imaging device]
A configuration example of a photographing apparatus having the image processing function of the embodiment will be described with reference to the block diagram of FIG.

  The microprocessor (CPU) 1101 executes a program stored in the ROM of the internal memory 1111 using the RAM of the internal memory 1111 as a work memory, and comprehensively controls each component described later via the system bus 1103.

  The imaging unit 1102 captures an image of the subject and acquires distance information of the subject and the background of the subject using a conventional distance estimation method. The distance information represents the distance between the subject or the background and the photographing apparatus for each pixel of the photographed image. Acquisition of subject distance information may be realized by providing a distance measurement unit as part of the imaging unit 1102.

  The display unit 1104 is a liquid crystal display (LCD) that displays captured images, character strings, user interfaces (UI), and the like. The display control unit 1105 performs display control for displaying a captured image, a character string, a UI, and the like on the display unit 104 in accordance with an instruction from the CPU 1101.

  The operation unit 1106 is a button, a dial, or a numeric keypad for inputting user instructions. The CPU 1101 receives a user instruction via the operation unit 1106. Further, the touch screen function added to the display unit 104 can be used as a part of the operation unit 1106.

  The imaging unit control unit 1107 controls the imaging optical system of the imaging unit 1102 in accordance with an instruction from the CPU 1101, and performs focusing, shutter opening / closing, aperture adjustment, and the like. The digital signal processing unit 1108 performs various processes such as development processing including demosaicing, white balance adjustment, gamma correction, and noise reduction on the imaging data input from the imaging unit 1102.

  The image processing unit 1109 is configured to perform image processing such as color gamut mapping for mapping the imaging data output from the imaging unit 102 or the image data output from the digital signal processing unit 108 into a standard color space, or an image in accordance with a user instruction. Apply processing. The image processing unit 1109 stores the image data to be displayed on the display unit 1104 in a predetermined area of the internal memory 1111, and compresses / decompresses the image data to be stored in the medium (eg, recording medium such as HDD, memory card, USB memory) 1113 Input to part 1110. Further, the image processing unit 1109 acquires boundary information between the foreground area and the background area of the image data, which will be described in detail later, in accordance with an instruction from the CPU 1101.

  The compression / decompression unit 1110 performs various processes such as format conversion and encoding with data compression on the imaged data or the image processed image data. Further, the compression / decompression unit 1110 records distance information and boundary information in a predetermined area of the image format (for example, Exif manufacturer tag) in accordance with an instruction from the CPU 1101.

  The external memory control unit 1112 stores the image data output from the compression / decompression unit 1110 in various media 1113. Further, the external memory control unit 1112 can read image data stored in various media 1113 in accordance with an instruction from the CPU 1101 and supply the read image data to the compression / decompression unit 1110 and the image processing unit 1109. In that case, the compression / decompression unit 1110 performs format conversion accompanying decoding and data expansion in accordance with instructions from the CPU 1101, and the image processing unit 1109 performs various processes such as development processing, color gamut mapping, resolution conversion, and gamma correction.

[Image processing unit]
A processing configuration example of the image processing unit 1109 will be described with reference to the block diagram of FIG.

  The input unit 11 inputs image data and distance information between the subject and the background included in the image represented by the image data from the imaging unit 1102. The boundary information generation unit 12 generates boundary information indicating the boundary between the foreground area and the background area included in the input image data.

  Based on the boundary information, the correction area setting unit 14 sets an area of input image data whose distance information is to be corrected (hereinafter referred to as a correction area). The transparency calculation unit 15 calculates the transparency of each pixel of the input image data included in the correction area.

  The distance information correction unit 16 corrects the distance information using transparency for each pixel in the correction area. The output unit 17 adds the distance information corrected by the distance information correction unit 16 to the image data and stores it in the internal memory 1111 or the medium 1113.

  The distance information correction processing by the image processing unit 1109 will be described with reference to the flowchart of FIG.

  The input unit 11 inputs image data to be processed and distance information (S11). The boundary information generation unit 12 extracts an object image area (hereinafter referred to as an object area) from the input image data as a foreground area (S12), and determines a boundary between the extracted object area and an area other than the object area (background area). The boundary information shown is generated (S13). The boundary information may be vector data indicating a boundary line, or a list of pixels corresponding to the boundary.

  Further, the boundary information generation unit 12 displays an image represented by the image data on the display unit 1104, acquires a boundary line between the object region and the background region that the user inputs with reference to the image via the operation unit 1106, The acquired boundary line can also be used as boundary information.

  Next, the correction area setting unit 14 sets a correction area for distance information based on the boundary information (S14). The transparency calculation unit 15 calculates the transparency of the pixels included in the correction area (S15). Then, the distance information correction unit 16 calculates the corrected distance information of the correction area based on the transparency (S16).

  In this embodiment, the transparency (hereinafter also referred to as “alpha channel”) represents the probability that a pixel corresponds to the foreground or the background. In the following, setting of a correction area, calculation of transparency, and correction of distance information will be described in detail.

● Correction area setting Figure 5 shows an example of correction area setting. The correction area setting unit 14 sets an area having a predetermined width whose center is the boundary line as a correction area. In FIG. 5, the object region is represented by white and the background region is represented by dark gray. The boundary line is represented by a solid black line, but the actual boundary line does not have a color. The light gray area spreading on both sides of the boundary line is the correction area.

● Transparency calculation The transparency calculation unit 15 calculates transparency based on the theory that the pixel values of the image data near the boundary are a linear mixture of the foreground color and the background color (equation (1)) (also called “mating process”) ).
I = αF + (1-α) B… (1)
Where I is the pixel value,
F is foreground color,
B is the background color,
α is transparency (alpha channel).

  Since it is unclear whether a pixel near the boundary belongs to the foreground area or the background area, it is necessary to consider the blending (blending) of the foreground color and the background color. The transparency calculation unit 15 estimates the transparency α of the pixel in the correction area based on the assumption that the pixel value in the correction area is a linear mixture of the foreground color F and the background color B.

  If the transparency range is 0 ≦ α ≦ 1, the pixel belonging to the foreground area is α = 1, and the pixel belonging to the background area is α = 0. A pixel with transparency 0 <α <1 means that the foreground corresponds to a region that blocks a part of the background.

  The processing of the transparency calculation unit 15 will be described with reference to the flowchart of FIG.

Transparency calculation unit 15 is the shortest distance from the target pixel I of the correction area, it searches for a pixel I _F of pixel I _B and the object area of the background region (S151). Then, to get the pixel value B and F thereof pixels I _B and I _F (S152), based on the equation (1), to calculate the transparency α of the pixel of interest I (S153).

  Next, the transparency calculation unit 15 repeats the processes of steps S151 to S153 until the calculation of the transparency α is completed for all the pixels in the correction region based on the determination in step S154.

  FIG. 7 shows an example of the transparency α. FIG. 7A shows an example of an input image, and FIG. 7B shows an example of an alpha channel image generated by matting processing. In FIG. 7B, a white area is an area set as the object area by the correction area setting unit 14, and the transparency of the area is set to α = 1. A region indicated by dark gray is a region set as a background region by the correction region setting unit 14, and the transparency of the region is set to α = 0.

  In FIG. 7B, the light gray area is an area set as the correction area by the correction area setting unit 14, and the transparency of each pixel in the area is calculated by the transparency calculation unit 15, and 0 ≦ α ≦ 1 The transparency of the range is set.

Correction of distance information The processing of the distance information correction unit 16 will be described with reference to the flowchart of FIG.

The distance information correction unit 16 acquires the transparency α of the target pixel I in the correction area (S161). Then, in the shortest distance from the pixel of interest I, it searches the pixels I _F of pixel I _B and the object area of the background region (S162). Then, the search result of the pixel I _B and I _F (hereinafter, non-corrected pixel) to obtain the distance information D _B and D _F corresponding to (S163).

Next, the distance information correction unit 16 calculates the corrected distance information D ′ for the target pixel I using Equation (2) using the transparency α of the target pixel I and the distance information corresponding to the non-corrected pixel. That is, it is determined whether or not the transparency α is a predetermined threshold Th (for example, Th = 0.5) (S164). If the transparency is equal to or greater than the threshold (α ≧ Th), the target pixel I is regarded as a pixel closer to the object region. The correction distance information D ′ at is calculated (S165). If the transparency is less than the threshold (α <Th), the correction distance information D ′ for the target pixel is calculated with the target pixel I as a pixel closer to the background area (S166).
if (α ≧ Th)
D '= αD _F + (1-α) D _B ;
else
D '= (1-α) D _F + αD _B ;… (2)

  That is, the distance information correction unit 16 adds the distance information corresponding to the two non-corrected pixels, with the transparency α and (1-α) of the target pixel I as weights, and the corrected distance information D ′ in the target pixel I. To do.

  Next, the distance information correction unit 16 repeats the processes of steps S161 to S166 until the calculation of the correction distance information D ′ is completed for all the pixels in the correction region based on the determination in step S167.

  FIG. 9 shows an example of correction of distance information. FIG. 9A shows an example of an image composed of an object region and a background region. An object region close to the photographing device overlaps a background region far from the photographing device, and the object region has a texture pattern. Note that the background area and the object area have substantially uniform distance values. FIG. 9B shows a distance estimation result before correction, and shows a state where the background area has entered the object area and the distance estimation accuracy near the boundary is low. FIG. 9C shows an example of the corrected distance information, and the distance estimation accuracy near the boundary is improved by the correction.

  In this way, in order to solve the problem of low distance estimation accuracy near the boundary between the object area and the background area, a distance information correction area is set near the boundary, and distance information is corrected based on transparency to improve distance estimation accuracy. Can be achieved. Therefore, even when the contour of the object is unclear, the distance estimation accuracy near the contour can be improved.

  The distance information with improved distance estimation accuracy obtained in this way is added to the image data and stored in the internal memory 1111 or the medium 1113, for example. Then, the processing that is performed by the image processor 1109 or an external computer device to generate a free viewpoint composite image using multiple viewpoint images, or the refocus that focuses on an arbitrary part such as an object or background, and deceives others Used for processing.

[Modification]
The boundary information generation unit 12 may generate a binary separated image of the object region and the background region instead of generating boundary information. If the input image data includes multiple object areas and these object areas overlap each other, the boundary information between the object areas is generated between the object areas, and the distance information is corrected in the same manner as described above. do it.

  In the above description, an example of searching for an uncorrected pixel at the shortest distance from the target pixel has been described. As another search method, for example, a pixel is searched in the eight directions of the left, right, up, down, and diagonal directions centering on the pixel of interest, and a pixel closer to the pixel of interest among the pixels of the searched object region and background region is uncorrected pixel May be adopted. In this way, although the improvement of the distance estimation accuracy is slightly reduced, the search time can be shortened compared to the case of searching for uncorrected pixels in all directions.

  In the above description, the example in which the distance information corresponding to the target pixel is corrected using the transparency α of the target pixel and the distance information corresponding to the neighboring non-corrected pixels is described for all the pixels in the correction region. According to this correction method, the distance information corresponding to the pixel in the correction area is corrected to distance information between the distance information of the object area and the distance information of the background area, with the transparency α of the pixel as a weight. If the transparency changes smoothly from the object region to the background region, the corrected distance information also changes smoothly from the object region to the background region. On the other hand, distance information corresponding to the target pixel can be corrected step by step.

  The processing of the distance information correction unit 16 that corrects the distance information corresponding to the target pixel in stages will be described with reference to the flowchart of FIG.

  The distance information correction unit 16 acquires the transparency α of the target pixel I in the correction area (S261), compares the transparency α with a predetermined threshold Th ′ (for example, Th ′ = 0.5) (S262), and determines whether or not based on the comparison result. Select a correction pixel. That is, when the transparency is equal to or greater than the threshold (α ≧ Th ′) (S263), the pixel in the object region that is in the vicinity (for example, the shortest distance) of the target pixel I is selected (S264). If the transparency is less than the threshold (α <Th ′) (S263), a pixel in the background area that is in the vicinity (for example, the shortest distance) of the pixel of interest I is selected (S265). Then, the correction distance information D ′ for the target pixel is calculated from the distance information corresponding to the selected pixel (S266).

  The distance information correction unit 16 may select, for example, a pixel in the object region or background region that is the shortest distance from the target pixel, or an object region or background region included in a predetermined distance range centered on the target pixel. One pixel or a plurality of pixels may be selected. When a plurality of pixels are selected, the average value of the distance information corresponding to the selected pixels may be corrected distance information D ′, or weighting of distance information with the distance (number of pixels) between the target pixel and each selected pixel as a weight The average value may be used as the correction distance information D ′.

  Next, the distance information correction unit 16 repeats the processing of steps S261 to S266 until the calculation of the correction distance information D ′ is completed for all the pixels in the correction region based on the determination in step S267.

  As described above, based on the transparency of the pixel in the correction area, the distance estimation accuracy can be improved by gradually correcting the distance information corresponding to the pixel to the object area or the background area.

  The image processing according to the second embodiment of the present invention will be described below. Note that the same reference numerals in the second embodiment denote the same parts as in the first embodiment, and a detailed description thereof will be omitted.

  In the first embodiment, an area having a predetermined width with the boundary line being substantially in the center is set as a correction area for distance information. In the second embodiment, the correction area is set based on the reliability of the distance information.

  The distance information correction processing by the image processing unit 1109 in the second embodiment will be described with reference to the flowchart of FIG.

  The input unit 11 inputs image data to be processed, distance information, and reliability information (S21). The boundary information generation unit 12 extracts an object region from the input image data (S22), and generates boundary information indicating a boundary between the extracted object region and another region (background region) (S23). The boundary information may be vector data indicating a boundary line, or a list of pixels corresponding to the boundary.

  Next, the correction area setting unit 14 sets a correction area for the distance information based on the boundary information and the reliability information (S24). The transparency calculation unit 15 calculates the transparency of the pixels included in the correction area (S25). Then, the distance information correction unit 16 corrects the distance information of the correction area based on the transparency (S26).

  Hereinafter, the setting of the correction area will be described in detail. The calculation of transparency and the correction of distance information are the same as in the first embodiment, and detailed description thereof is omitted.

Correction Area Setting Processing of the correction area setting unit 14 will be described with reference to the flowchart of FIG.

  The correction area setting unit 14 determines one pixel on the boundary line indicated by the boundary information as a target pixel (S241), and obtains reliability information (hereinafter, reliability R) of distance information corresponding to the target pixel (S242). Then, the reliability R is compared with a predetermined threshold value Rth, and the comparison result is recorded (S243). Then, by the determination in step S244, the processing in steps S241 to S243 is repeated until the comparison result between the reliability R and the threshold value Rth is recorded for all pixels on the boundary line. Hereinafter, a pixel having a reliability of distance information less than a threshold (R <Rth) is referred to as a “low reliability pixel”.

  When the reliability of the distance information is determined for all the pixels on the boundary line, the correction area setting unit 14 expands the area of the low reliability area around the low reliability pixels on the boundary line, and sets the correction area of the distance information. decide.

  In other words, comparing the reliability R of the distance information corresponding to the pixel adjacent to one pixel (adjacent pixel) and the threshold Rth (hereinafter referred to as reliability determination), centering on the low reliability pixel on the boundary line, the determination result is a bitmap. Record as an image. Subsequently, the reliability determination of the pixel adjacent to the two pixels is performed and the determination result is recorded,..., The reliability determination of the pixel adjacent to the n pixel is performed and the determination result is recorded (S245),. Then, for all low-reliability pixels on the boundary line, when all the pixels to be determined in one reliability determination satisfy R ≧ Rth (S246), the region enlargement centering on the pixel ends.

  Next, it is determined whether or not there is a low-reliability pixel on the boundary line where region expansion has not been performed (S247). If there is a low-reliability pixel, the center pixel is moved (S248), and the process returns to step S245 to perform region expansion. .

  As in the first embodiment, a provisional correction area having a predetermined width centered on the boundary line is set, and in step S246, all pixels to be determined in one reliability determination are outside the provisional correction area. May end the area expansion. In addition, in the area expansion centered on each low-reliability pixel on the boundary line, pixels whose determination results are already recorded in the bitmap image (and pixels outside the provisional correction area) need to be subjected to reliability determination. Absent.

  When there is no low-reliability pixel on the boundary line that has not been subjected to region expansion and the region expansion is completed, a bitmap image in which low-reliability pixels including low-reliability pixels on the boundary line are recorded is obtained. The correction area setting unit 14 sets the pixel area formed by the low reliability pixels indicated by the bitmap image as a correction area for the distance information.

  In Example 2, since the correction area is determined based on the reliability information, the correction area is narrowed in the area where the reliability of the distance information is high, and the correction area is widened in the area where the reliability is high. The width of the region may vary depending on the region.

  As described above, since the region with the low reliability of the distance information is extracted as the correction region of the distance information, the distance information is not corrected to the region with the high reliability of the distance information near the boundary, and the distance estimation accuracy is further improved. be able to.

  Hereinafter, image processing according to the third embodiment of the present invention will be described. Note that the same reference numerals in the third embodiment denote the same parts as in the first and second embodiments, and a detailed description thereof will be omitted.

  In the first embodiment, an area having a predetermined width around the boundary line is set as a correction area for distance information. In the second embodiment, a correction area is set based on the reliability of distance information. In the third embodiment, a correction area is set, and the correction area is corrected based on the transparency α of the pixels included in the correction area.

  The distance information correction processing by the image processing unit 1109 in the third embodiment will be described with reference to the flowchart of FIG.

  The input unit 11 inputs image data to be processed and distance information (S31). The boundary information generation unit 12 extracts an object region from the input image data (S32), and generates boundary information indicating a boundary between the extracted object region and another region (background region) (S33). The boundary information may be vector data indicating a boundary line, or a list of pixels corresponding to the boundary.

  Next, the correction area setting unit 14 sets a temporary correction area for the distance information based on the boundary information (S34). The transparency calculation unit 15 calculates the transparency α of the pixels included in the provisional correction area (S35). The correction area setting unit 14 sets a correction area obtained by correcting the provisional correction area based on the transparency α (S36). Then, the distance information correction unit 16 corrects the distance information of the correction area based on the transparency α (S37).

Correction Area Setting Correction area correction by the correction area setting unit 14 will be described with reference to the flowchart of FIG.

  The correction area setting unit 14 acquires the transparency α of the target pixel in the provisional correction area (S361), and compares the acquired transparency α with a predetermined range (S362). The predetermined range is, for example, more than 0.1 and less than 0.9 (0.1 <α <0.9), more than 0.2 and less than 0.8 (0.2 <α <0.8).

  When the transparency α of the target pixel is included in the predetermined range (S363), the correction area setting unit 14 leaves the pixel in the correction area (S364). When the transparency α of the target pixel is not included in the predetermined range (S363), the pixel is excluded from the correction area (S365). In other words, pixels whose transparency α is included in the predetermined range are left in the correction region as pixels included in the boundary region, and pixels whose transparency α is not included in the predetermined range are excluded from the correction region as pixels of the background region or object region. To do.

  Next, the correction area setting unit 14 repeats the processes from step S361 to S365 until the determination of step S366 completes the residual / exclusion process based on the transparency α for all the pixels in the provisional correction area. When the remaining / excluded process is completed, the remaining pixels correspond to the corrected correction area.

  In this way, since the correction area is corrected based on the transparency α, the distance estimation is not performed without correcting the distance information to pixels with the transparency α included in the provisional correction area 0 or 1, or pixels close to 0 or 1. It is possible to increase the accuracy and speed up the correction process.

[Other Examples]
The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various recording media, and a computer (or CPU, MPU, etc.) of the system or apparatus reads the program. It is a process to be executed.

Claims (15)

Input means for inputting image data of a subject and distance information of the subject and the background of the subject;
Generating means for generating boundary information between the foreground region and the background region of the image data;
Setting means for setting a correction area for correcting the distance information based on the boundary information;
Calculating means for calculating the transparency of the pixels in the correction region;
An image processing apparatus comprising: a correction unit that corrects distance information corresponding to a pixel based on the transparency of the pixel in the correction region.   2. The image processing apparatus according to claim 1, wherein the generation unit extracts an area of an object image as the foreground from the image data, and uses an area other than the area of the object image as the background area.   3. The image processing apparatus according to claim 1, wherein the setting unit sets a region having a predetermined width with a boundary line indicated by the boundary information as a substantially center as the correction region. The input means further inputs the reliability of the distance information,
2. The setting unit sets, as the correction area, an area obtained by area expansion of a pixel area corresponding to distance information whose reliability is less than a predetermined first threshold from a boundary line indicated by the boundary information. An image processing apparatus according to claim 2.   The setting means sets a region having a predetermined width whose center is a boundary line indicated by the boundary information as a temporary correction region, and sets a region in which pixels whose transparency is not included in the predetermined range are excluded from the temporary correction region. 3. The image processing apparatus according to claim 1, wherein the image processing apparatus is set as the correction area.   The calculation means calculates the transparency between 0 and 1 for the pixels in the correction area, with the transparency of the pixels in the object area being 1 and the transparency of the pixels in the background area being 0. An image processing apparatus according to any one of the above.   The correction means includes the distance information corresponding to the pixels in the foreground region in the vicinity of the target pixel in the correction region, the distance information corresponding to the pixels in the background region in the vicinity of the target pixel, and the 7. The image processing apparatus according to claim 1, wherein the corrected distance information is calculated based on transparency.   The correction means searches for a pixel in the foreground area that is the shortest distance from the target pixel and a pixel in the background area that is the shortest distance from the target pixel, and sets the transparency of the target pixel as a weight, 8. The image processing apparatus according to claim 7, wherein a result obtained by adding the distance information corresponding to a pixel is used as the corrected distance information.   The correction unit selects a pixel in the foreground region in the vicinity of the target pixel when the transparency of the target pixel is equal to or greater than a predetermined second threshold, and the target when the transparency is less than the second threshold. 8. The image processing apparatus according to claim 7, wherein a pixel in the background area in the vicinity of a pixel is selected, and the corrected distance information is calculated from the distance information corresponding to the selected pixel.   10. The image processing according to claim 9, wherein the correction unit selects a plurality of pixels in the foreground area or the background area and uses an average value of the distance information corresponding to the selected pixels as the corrected distance information. apparatus.   The correction means selects a plurality of pixels in the foreground area or the background area, and weights and averages the distance information corresponding to the selected pixel by using the distance between the selected pixel and the target pixel as a weight. 10. The image processing apparatus according to claim 9, wherein the corrected distance information is used.   The distance information corrected by the correction means is added to the image data, and output means for outputting the image data with the distance information added is described in any one of claims 1 to 11. Image processing device. Input image data of the subject and distance information of the subject and the background of the subject,
Generating boundary information between the foreground region and the background region of the image data;
Set a correction area for correcting the distance information based on the boundary information,
Calculating the transparency of the pixels in the correction area;
An image processing method for correcting distance information corresponding to a pixel based on transparency of the pixel in the correction area.   13. A program for causing a computer to function as each unit of the image processing apparatus according to claim 1.   15. A computer-readable recording medium on which the program according to claim 14 is recorded.