JPH11183142A - Method and apparatus for picking up three-dimensional image - Google Patents

Abstract

PROBLEM TO BE SOLVED: To avoid generation of an occlusion and improve measurement accuracy for distance information by obtaining a plurality of parallax images for an object to be measured and calculating a distance value at each pixel of a reference parallax image, based on a stereoscopic image pair of the reference parallax image and each parallax image. SOLUTION: A pickup part-moving mechanism part 2 makes an image pickup part 1 carry out a repeated stepped movement, that is, repeatedly stops and moves the image pickup part 1. A plurality of parallax images are obtained by picking up images at each position where the image pickup part 1 stops, which are stored in a luminance value memory 4. The stop position of the image pickup part 1 is set at several tens points via an equal distance. An image to be used as a luminance image information is selected from the plurality of parallax images to be a reference parallax image. A corresponding pixel-searching part 6 forms a first stereoscopic image pair from the parallax image selected on the basis of data of a pickup position memory 3 and the reference parallax image, searches for corresponding pixels, calculates distance information and further calculates an estimated parallax value from a second stereoscopic image pair.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image pickup method and apparatus for obtaining a three-dimensional image composed of luminance information and distance information, and more particularly, to measuring a shape of an object to be measured when performing shape measurement using stereo vision. The present invention relates to a three-dimensional image capturing method and a three-dimensional image capturing apparatus that can take measures against occlusion and improve the accuracy of measured distance information even when a (measured object) has a relatively complicated shape.

[0002]

2. Description of the Related Art As a method of measuring the shape of a target object, a passive method represented by stereo vision and an active method of irradiating a target object with light and detecting reflected light thereof are used.
(Laser radar, active stereo, moiré, interference). In general, the measurement accuracy of the active method is higher, but the measurement range is determined within the reach of the laser power, for example, due to the limitations of the light emitting means that emits light. In many cases, measurement is not possible. On the other hand, the passive method is versatile and has the advantage that there are few restrictions on the target object.

[0003] Among the passive methods, shape measurement by stereo vision is known as a method with relatively high accuracy.
Shape measurement by stereo vision applies the principle of triangulation,
For example, as shown in FIG. 4, one parallax image (luminance image) serving as luminance information is obtained from two parallax image pairs at each angle of view of the observation systems A and B obtained by imaging the object to be measured from different positions. The distance value from the imaging point at each pixel is calculated.

According to the shape measurement by stereo vision described above, the problem of occlusion occurs because the optical axis of the observation system for obtaining each parallax image is different. This is particularly troublesome when trying to make one-to-one correspondence between the luminance value and the distance value. That is, as shown in FIG. 4, a phenomenon (occlusion) occurs in which the area that is visible at the angle of view of the observation system A that obtains the luminance information of the measured object 40 is not visible at the angle of view of the other observation system B. However, it is not possible to make one-to-one correspondence between the luminance value and the distance value for each pixel in the luminance information, and as a result, there is a pixel having no distance information.

In order to cope with such an occlusion problem, as a method of measuring distance information when an object to be measured having an occlusion region has a relatively complicated shape, the object to be measured is measured from a plurality of different positions. There has been proposed a method of acquiring a plurality of parallax images by photographing and repeatedly performing a distance calculation process to calculate a distance of an occlusion portion.

That is, in this method, a continuous stereo image of a measured object is created from a plurality of parallax images obtained by imaging the measured object from a plurality of different positions (a horizontal cutting line of each parallax image is formed). Then, the continuous stereo image is decomposed into directional components to calculate the depth of the portion without occlusion, and the depth is calculated again from the continuous stereo image excluding the portion where the depth is determined. The depth of the occlusion part is calculated by repeating the processing. To calculate the depth, a continuous stereo image is scanned with straight lines having various inclinations, the variance on the straight line is obtained, and when the variance is equal to or less than a reference value, the object is set at a depth corresponding to the direction (inclination) of the straight line. Is considered to exist (Japanese Unexamined Patent Publication No. 6-74).
723).

[0007]

However, in the above-mentioned method, when calculating the depth, all of the plurality of captured parallax images can be formed into a continuous stereo image without selecting parallax images in an area where occlusion does not occur. Has adopted. In this case, some of the adopted parallax image pairs may cause a reduction in distance measurement accuracy. The reason will be described below.

That is, as shown in FIG.
In stereo vision (stereo image pair) composed of a pair of parallax images obtained by imaging from different positions via
Λ is the length of one side of the pixel size of the captured image, B is the base line length that is the distance between the lens center positions in each parallax image,
Assuming that the focal length of the lens is F, the parallax d, which is the difference between the pixels of the corresponding pixels in the stereo image pair, and the measured object 4
The relationship with the distance value z from the plane where 0 is located to the center of the lens 50 can be expressed by Equation 1.

[0009]

[Equation 1] d = B · F / λ · z

What is directly measured from the stereo image pair is the parallax d (pixel difference). According to Equation 1, when the base line length B is small, the obtained parallax d itself becomes small, and the error generated in the parallax increases. Therefore, in the creation of the continuous stereo image described above, it is expected that the distance information (depth) based on the straight line calculated using the parallax that may have a large error will naturally have a large error. In FIG. 5, the captured image plane is described in front of the lens 50. However, in an actual optical system, the image plane (imaging plane) is located on the rear side above the image forming center symmetrical with respect to the lens 50. Plane) exists.

The present invention has been made in view of the above-mentioned circumstances, and provides a three-dimensional image pickup method and a three-dimensional image pickup apparatus capable of avoiding occlusion as much as possible and improving the accuracy of measuring distance information. The purpose is to:

[0012]

According to a first aspect of the present invention, there is provided a three-dimensional image pickup method for measuring luminance information and distance information with respect to an object to be measured, which is performed by the following procedure. I have. First, a plurality of parallax images are obtained by photographing the object to be measured from a plurality of imaging positions. Next, one of the parallax images is used as luminance information, and necessary for stereoscopic viewing from an image pair of a reference parallax image serving as the luminance information and a first parallax image at an imaging position closest to the imaging position. The first corresponding pixel search is performed, and the first
In the range where the occlusion does not occur, the respective distances from the imaging position to each pixel of the luminance information are calculated from the parallax of the image pair and stored as distance information. A corresponding pixel search is again performed from the image pair of the reference parallax image and the second parallax image at the second closest imaging position to the imaging position.
The distance information corresponding to each pixel is calculated in the range where the occlusion does not occur, and the distance information rewriting process of rewriting the already stored distance information is performed for the pixel whose distance has been calculated. This distance information rewriting process is sequentially repeated for the image pair of the reference parallax image and each of the parallax images whose proximity to the imaging position is the third or later. And the first
Are stored as occlusion pixels having no distance information.

According to a second aspect of the present invention, in the three-dimensional image capturing method according to the first aspect, the reference parallax image is a parallax image at an intermediate position with respect to the measured object among a plurality of parallax images. It is characterized by.

According to a third aspect of the present invention, in the three-dimensional image pickup method according to the first aspect, the corresponding pixel search in the distance information rewriting process is based on the distance information stored by the immediately preceding distance information rewriting process. It is characterized by performing.

According to a fourth aspect of the present invention, in the three-dimensional image capturing method according to the first aspect, for a pixel in a range where the first occlusion occurs, distance information of a pixel closest to the pixel is obtained. It is characterized by assigning.

According to a fifth aspect of the present invention, there is provided a three-dimensional image pickup apparatus for measuring luminance information and distance information with respect to an object to be measured. Imaging unit. The imaging unit captures the object to be measured from a plurality of imaging positions, and enables acquisition of a plurality of parallax images. Corresponding pixel search means. The corresponding pixel searching means uses one of the plurality of parallax images as luminance information, sequentially selects an image pair of a reference parallax image serving as luminance information and each of the other parallax images, and searches for a corresponding pixel required for stereoscopic viewing. Is what you do. Distance information calculation means. The distance information calculating means calculates a distance from an imaging position of a pixel in the reference parallax image from the parallax of each image pair. Distance value memory. This distance value memory stores distance information for pixels whose distances have been calculated by the distance information calculation means. Occlusion area processing unit. The occlusion area processing section stores pixels whose distances could not be calculated by the distance information calculating means as occlusion pixels. 3D image memory. The three-dimensional image memory stores the three-dimensional information by associating distance information or occlusion pixels with each pixel of the reference parallax image. Distance information rewriting means. The distance information rewriting means uses, for the distance information of each pixel of the three-dimensional image memory, a pixel pair having a maximum base line length in imaging each image pair serving as a basis for distance calculation by the distance information calculation means. It is rewritten to distance information.

According to a sixth aspect of the present invention, in the three-dimensional image pickup apparatus according to the fifth aspect, a distance value assigning means for assigning a distance value of a nearest pixel to the occlusion pixel is provided. And

According to the present invention, the first corresponding pixel search required for stereoscopic viewing is performed from the image pair of the reference parallax image and the first parallax image at the closest imaging position. Occlusion can be avoided as much as possible by stereo vision with a short reference length.

Further, since the corresponding pixel search is performed a plurality of times from the image pair of the reference parallax image and the parallax image and the distance information is rewritten, the distance information is rewritten to the distance information based on stereo vision having a long reference length. The accuracy of information measurement can be improved.

The range in which occlusion occurs can be reduced by using the reference parallax image as a parallax image at an intermediate position with respect to the measured object in the plurality of parallax images.

The processing speed can be increased by performing the corresponding pixel search in the distance information rewriting process based on the distance information stored in the immediately preceding distance information rewriting process.

By assigning distance information of the nearest pixel to the pixel in the range where the first occlusion has occurred in the first corresponding pixel search, the rounding of the edge can be avoided by avoiding the rounding of the edge. You can save.

[0023]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of a three-dimensional image pickup apparatus and a three-dimensional image pickup method according to the present invention will be described with reference to the drawings. FIG. 1 is a block diagram illustrating a configuration of an OA camera as a three-dimensional image capturing device, and FIG. 2 is a flowchart illustrating an operation of the three-dimensional image capturing device.

The three-dimensional image pickup device is a device for measuring luminance information and distance information with respect to an object to be measured. The image pickup section 1 obtains an image by picking up an object to be measured, and the image pickup section 1 obtains a plurality of parallax images. An imaging section moving mechanism section 2 that translates the section 1 within a certain distance range; and an imaging section moving mechanism section 2
, An imaging position memory 3 for storing each imaging position, a luminance value memory 4 for storing each parallax image, an output luminance value memory 5 for selectively storing one of the parallax images as a reference parallax image, and a reference parallax image. Corresponding pixel search unit (corresponding pixel search means) for searching for a corresponding pixel in an image pair of an image and a parallax image
6, a distance information calculating unit (distance information calculating means) 7 for calculating distance information of each pixel of the reference parallax image from the image pair, a distance value memory 8 for storing distance information, and occlusion in the reference parallax image from the image pair. An occlusion area processing unit 9 for detecting and processing pixels in the area, a three-dimensional image memory 10 for storing luminance information and distance information of the measured object,
It is comprised including.

The imaging section 1 linearly moves in steps within a certain distance range by the imaging section moving mechanism section 2, images the object to be measured from different positions for each step, and forms a plurality of parallax images. It is supposed to get. In the corresponding pixel search unit 6, the output luminance value memory 5
, The image pair of the reference parallax image and each of the other parallax images, which is selected as the luminance information, is sequentially selected, and a search for a corresponding pixel required for stereoscopic viewing is performed.

By combining this reference parallax image with each of the other parallax images, a corresponding pixel search is performed for various image pairs from a short base line to a long base line, so that an occlusion generation area is small. It is possible to obtain distance information from a pair of images having a short base line length to a unit having a high distance value measurement accuracy (a pair of images having a long base line). That is, according to an image pair having a short base line length (close to the optical axis), almost all pixels in the captured images (the reference parallax image and the parallax image) are present in both images because the angle of view in the imaging is substantially the same. Occlusion in which only pixels are obtained can be avoided. Further, according to the image pair having a long base line length, the value of the parallax in Expression 1 described above becomes large, and the error due to the parallax can be reduced, and the accuracy of distance value measurement can be increased.

The output luminance value memory 5 is located at an intermediate position with respect to the object to be measured so that distance information can be added to each pixel of luminance information, in other words, an occlusion area is generated as little as possible. The image at the imaging position is selected as the reference parallax image (luminance information). In other words, if the reference parallax image is an image obtained at an imaging position that is an intermediate position with respect to the measured object, and a stereo image pair is formed from the image and a parallax image having a slightly shifted angle of view, the measured object Can be stereoscopically imaged from almost the front, and the occurrence of occlusion areas can be minimized.

The distance information calculator 7 calculates the distance from the image pickup position of the pixel of the luminance image information from the image pair of the reference parallax image and each parallax image. In this case, in an image pair of the reference parallax image and each parallax image, a portion where an occlusion area occurs has no corresponding pixel, and therefore cannot be calculated. Then, in the distance value memory 8, the distance information is stored for the pixels for which the distance can be calculated by the distance information calculation unit 7, and the pixels for which the distance cannot be calculated by the distance information calculation unit 7 are stored as occlusion pixels. Further, the occlusion area processing section 9 is provided with a distance value assignment processing means,
The distance value of the nearest pixel is assigned to each pixel that has been determined to be an occlusion pixel.

The three-dimensional image memory 10 stores distance information or three-dimensional information in which occlusion pixels are associated with each pixel of the reference parallax image as luminance image information. The distance information of each pixel in the three-dimensional image memory 10 can be rewritten to distance information based on a pixel pair having a maximum base line length in imaging each image pair, which is the basis of distance calculation in the distance information calculation unit 7. Is configured.

Next, an imaging method using the three-dimensional image imaging apparatus will be described with reference to FIGS. When the start of imaging is selected by a switch (not shown) of the three-dimensional imaging apparatus, the imaging unit moving mechanism unit 2 performs a step movement in which the imaging unit 1 repeatedly stops and moves (step 101). Imaging is performed at each position where the imaging unit 1 is stopped (Step 102), a plurality of parallax images are obtained, and the obtained luminance images are stored in the luminance value memory 4 (Step 103) repeatedly. (Step 10
4). The position at which the imaging unit 1 stops is set to several tens at equal intervals within a fixed linear distance in which the imaging unit 1 can move in parallel. Therefore, the luminance value memory 4 stores the luminance information of each pixel of each parallax image.

Next, an image used as luminance image information is selected from a plurality of parallax images as a reference parallax image (step 105). As described above, the reference parallax image is captured at the center position with respect to the measured object, that is, the parallax image captured at the intermediate stop position of the moving imaging unit 1 so that the occlusion region is generated as little as possible. The image is used as a reference parallax image. The luminance value information of each pixel of the selected reference parallax image is stored in the output luminance value memory 5 (Step 106).

Subsequently, the corresponding pixel search unit 6 selects a parallax image captured at a position closest to the capturing position of the reference parallax image based on the data in the capturing position memory 3, and selects the parallax image and the luminance image information. Form a first stereo image pair with the reference parallax image selected as (step 1)
07). In this case, since the optical axes of the reference parallax image and the parallax image are located at the closest positions, the occurrence of the occlusion region can be minimized.

Then, in this stereo image pair, a corresponding pixel search for searching for a corresponding pixel is performed (step 1).
08). At the time of searching for a corresponding pixel, since the optical axes of the images are close to each other, the corresponding pixel of the parallax image exists at a position close to the pixel position of the reference parallax image, so that the search area of the corresponding pixel search can be set small. . As a method of searching for a corresponding pixel, a window having a certain size near the target pixel is provided, and a pixel having the smallest difference in luminance value between pixels in this window, that is, a pixel that minimizes Dw shown in Equation 2 For example, a corresponding pixel may be used.

[0034]

(Equation 2)

In Equation 2, W is the size of the set correlation window, and I is
ref is the luminance value of the pixel in the reference parallax image of the stereo image pair, and Is is the luminance value of the pixel in the parallax image forming the stereo image pair. If the corresponding pixel can be searched, distance information (distance value z) is calculated by Expression 3 (step 109). Expression 3 is a modification of Expression 1, and is the same as Expression 1 with respect to the notation. This operation is performed for all pixels of the reference parallax image.

[0036]

[Equation 3] z = B · F / λ · d

The corresponding pixel can be searched for in the stereo image pair, and the distance value of each pixel for which the distance value has been calculated by Equation 3 is
It is stored in the distance value memory 8 (step 110). If the distance value cannot be calculated due to occlusion, information indicating that the pixel is an occlusion pixel is stored in the distance value memory 8.

Next, in addition to the parallax image used as one of the stereo image pair in the previous step, a parallax image taken at a position closest to the imaging position where the reference parallax image was taken is taken as a parallax image of the second stereo image pair. (Step 11
1). Since the distance value previously stored in the distance value memory 8 (the distance value overwritten in step 115 described later) has been calculated in the previous step, the expected value of the parallax d can be calculated by Expression 1 (step). 11
2). By searching for the corresponding pixel using the parallax d (step 113), the search time can be reduced (the processing time can be shortened).

The distance information of each pixel of the reference parallax image is calculated from the corresponding pixel in the stereo image pair (step 114).
Procedure is the same as in the previous step, but since the baseline length of the second stereo image pair is longer than the baseline length of the first stereo image pair in the previous step, distance information with higher measurement accuracy can be obtained. Becomes On the other hand, the number of pixels for which a distance value cannot be calculated due to the occurrence of occlusion increases. For the pixels for which the distance value has been calculated by the second stereo image pair, for the pixels that have already been calculated by the first stereo image pair and have been written to the distance value memory 8,
The data is overwritten with the newly calculated distance value (step 115).

Thereafter, the same stereo image pair is set for all the captured parallax images, and the first stereo image pair is set by performing an overwriting operation using the distance value information calculated in each step. For each pixel of the reference parallax image in an area where occlusion does not occur, distance information based on the stereo image pair having the longest base line length can be obtained (step 116).

Even after forming a stereo image pair by using all the parallax images and calculating the distance information for each pixel,
Depending on the imaging target, occlusion may occur, that is, the occlusion pixel may be stored in the distance value memory 8. Such an area is considered to be an edge area (see the occlusion area in FIG. 1) that is visible from the position where the reference parallax image (luminance image information) is captured.

A method of processing these occlusion pixels will be described. As shown in FIG.
Is the actual distance value (depth information) in a certain horizontal section of the measured object when the measured object 40 exists in front of
Is as shown in FIG. Here, the shaded area corresponds to the occlusion area. When pixels in the occlusion area are associated with distance values obtained by taking weighted averages from surrounding pixels, unnecessary rounding of edges occurs as shown in FIG. 5B. When such “rounding” occurs, when the image of the measured object 40 is to be separated from the image of the background object 41 using the distance information, the edge of the boundary between the background image and the object image can be detected. Separation becomes difficult.

Therefore, in the occlusion area processing section 9 of the three-dimensional image pickup apparatus of the present invention, the operation of allocating the distance value of the nearest pixel to the pixel of the occlusion area (step 118) is performed (step 119). Thus, the profile of the distance information is as shown in FIG. In this case, although the error may increase as the distance information of each pixel, the edge portion is preserved, and the image of the measured object can be easily separated from the background image.

When the processing of the occlusion area is completed,
A distance value is obtained for each pixel of the reference parallax image (luminance information image) (step 117), and information of the luminance value and the distance value is stored in a three-dimensional image memory as a three-dimensional image having each pixel (step 120). To end the operation.

In the three-dimensional image pickup apparatus described above, as a means for obtaining a plurality of parallax images, the image pickup unit 1 is moved in parallel in a stepwise manner by a moving mechanism to obtain parallax images having different image pickup positions. A configuration may be adopted in which a plurality of parallax images are obtained by installing the optical system and imaging.

The luminance information used in searching for the corresponding pixel includes infrared reflected light obtained by an imaging device having sensitivity in the infrared region in addition to the normal R, G, and B signals used for the luminance information image. Information may be used. Generally, since different materials have different reflection spectra, it is conceivable that a corresponding pixel search can be performed by using a plurality of spectra even when a corresponding point search is impossible with ordinary visible light.

The corresponding pixel is detected by minimizing the difference between the luminance values in the correlation window when searching for the corresponding pixel. However, the corresponding amount is calculated by calculating a characteristic amount such as detecting a linear component by Hough transform or the like. Pixel detection may be performed.

According to the above-described three-dimensional image capturing method, a plurality of parallax images are first collectively obtained and the corresponding pixel search and the distance value calculation are performed. After obtaining the reference parallax image by performing the above-described image capturing, the image capturing is performed from a different viewpoint, and the corresponding pixel search and the distance value calculation are performed each time, so that the capacity of the luminance value memory can be reduced.

[0049]

According to the present invention, a plurality of parallax images are acquired for an object to be measured, and a distance value at each pixel of the reference parallax image is calculated from a stereo image pair of the reference parallax image and each parallax image. By doing so, the distance value can be calculated from the stereo image pair having the maximum baseline length in a range where occlusion does not occur, so that distance information with high measurement accuracy can be obtained while avoiding occlusion. For pixels in the area where occlusion could not be avoided,
By assigning the distance information of the nearest pixel to the pixel, the edge can be preserved, and the object image can be easily separated from the background image using the distance information.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of an OA camera as an example of an embodiment of a three-dimensional image capturing apparatus according to the present invention.

FIG. 2 is a flowchart for explaining the operation of the three-dimensional image pickup device.

FIGS. 3A, 3B, and 3C are explanatory diagrams of distance information in a horizontal cross section with respect to an object to be measured for explaining how to add a distance value to a pixel for which occlusion cannot be avoided;

FIG. 4 is an explanatory plan view of an imaging plane for explaining occlusion occurring in a measured object in stereo vision.

FIG. 5 is an optical explanatory diagram showing a relationship between a base line length and a distance and a parallax in stereo vision.

[Description of Signs] 1 ... Imaging unit 2 ... Imaging unit moving mechanism unit 3 ... Imaging position memory 4 ... Luminance value memory 5 ... Output luminance value memory 6 ... Corresponding pixel searching unit (corresponding pixel searching means) 7
... Distance information calculation unit (distance information calculation means) 8 ... Distance value memory 9 ... Occlusion area processing unit 10 ... 3D image memory

Claims (6)

[Claims] 1. A three-dimensional image capturing method for measuring luminance information and distance information for an object to be measured, wherein said object to be measured is photographed from a plurality of image pickup positions to obtain a plurality of parallax images, And the first parallax image necessary for stereoscopic viewing from an image pair of a reference parallax image serving as the luminance information and a first parallax image at an imaging position closest to the imaging position. Is performed, and in the range where the first occlusion does not occur, each distance from the imaging position to each pixel of the luminance information is calculated from the parallax of the image pair and stored as distance information. A corresponding pixel search is again performed from the image pair with the second parallax image at the closest imaging position, and each distance corresponding to each pixel is calculated in a range where the second occlusion does not occur, The distance information rewriting process for rewriting the distance information already stored is performed for the pixels for which the separation has been calculated. For the distance information rewriting process, each parallax image in which the distance between the reference parallax image and the imaging position is the third or later. A three-dimensional image pickup method, wherein the pixels in the range where the first occlusion occurs are stored as occlusion pixels having no distance information. 2. The three-dimensional image pickup method according to claim 1, wherein the reference parallax image is a parallax image at an intermediate position with respect to a measured object among a plurality of parallax images. 3. The three-dimensional image pickup method according to claim 1, wherein the corresponding pixel search in the distance information rewriting process is performed based on the distance information stored by the immediately preceding distance information rewriting process. 4. The three-dimensional image pickup method according to claim 1, wherein distance information of a pixel closest to the pixel in a range where the first occlusion occurs is assigned. 5. An apparatus for measuring luminance information and distance information for an object to be measured, wherein the apparatus measures an image of the object to be measured from a plurality of imaging positions,
An imaging unit capable of acquiring a plurality of parallax images, and one of the plurality of parallax images as luminance information, sequentially selecting an image pair of a reference parallax image serving as luminance information and each of the other parallax images, for stereo viewing. Corresponding pixel searching means for searching for a necessary corresponding pixel; distance information calculating means for calculating a distance from an imaging position of a pixel in the reference parallax image from the parallax of each image pair; and a distance in the distance information calculating means. A distance value memory that stores distance information for the pixel that could be calculated, an occlusion area processing unit that stores a pixel whose distance could not be calculated by the distance information calculation unit as an occlusion pixel, and for each pixel of the reference parallax image. A three-dimensional image memory that stores three-dimensional information in correspondence with distance information or occlusion pixels, and distance information of each pixel of the three-dimensional image memory. And a distance information rewriting means for rewriting the distance information by a pixel pair having a maximum base line length in imaging each image pair as a basis of the distance calculation by the distance information calculating means. Image capturing device. 6. A three-dimensional image pickup apparatus according to claim 5, further comprising a distance value assigning means for assigning a distance value of a nearest pixel to said occlusion pixel.