JPH0674762A - Distance measuring apparatus - Google Patents

Abstract

PURPOSE:To provide a distance measuring apparatus in which stereo images imaged at different focal points are utilized in the measurement of distance in order to obtain highly accurate information of an object which is present in a range wider than the depth of focus of an optical system. CONSTITUTION:An imaging system comprising focal point operating sections 2a, 2b, lenses 3a, 3b, and imaging elements 5a, 5b produces a stereo image at same focal point. A focal point setting unit 1 sets a plurality of focal points for producing stereo images. A corresponding point detector 9 subjects the stereo images to corresponding point detection and a three-dimensional coordinate calculating unit 10 calculates three-dimensional coordinates from the corresponding points thus detected. A selector 14 subjects the three-dimensional coordinates, calculated through the three-dimensional calculating unit 10, to decision based on the focal point and focal depth thus obtaining distance information.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a distance measuring device,
In particular, the present invention relates to a distance measuring device that performs measurement using a plurality of images including parallax.

[0002]

2. Description of the Related Art Conventionally, in the field of computer vision and the like, it has been required to obtain distance information from a captured image. For obtaining distance information from an image, for example, a method of detecting corresponding points from a plurality of images having parallax using correlation and calculating distance information by the same principle as triangulation as shown in FIG. Are known. This method is described, for example, in “Trends of stereoscopic measurement methods using images” by Kokichi Sugihara, [17th Conference on Image Engineering Conference, 2-1, pp. 31-36, (1986)]. By the above method, distance information can be obtained from the image.

[0003]

However, when the image used for these processes is obtained by the imaging optical system, the following problems occur. Since the image obtained through the imaging optical system is affected by the depth of focus, it is possible to obtain a sharp image of an object included within the depth of focus with respect to the optical axis direction of the optical system. However, with respect to the object out of the depth of focus, a sharp image cannot be obtained and a blurred image is obtained. However, it is difficult to detect corresponding points with high accuracy for a blurred image. Therefore, it can be seen that it is difficult to accurately detect corresponding points and obtain distance information for an object existing in a range wider than the depth of focus of the optical system.

The present invention has been made in view of the above problems, and provides a distance measuring device capable of obtaining accurate distance information even for an object existing in a range wider than the depth of focus of an optical system. The purpose is to

[0005]

That is, the present invention corresponds to an image pickup device for picking up a stereo image at the same focus position, a focus position setting means for setting a plurality of focus positions for picking up the stereo image, and the stereo image. Corresponding point detecting means for detecting points, three-dimensional coordinate calculating means for calculating three-dimensional coordinates from the corresponding points detected by the corresponding point detecting means, and three-dimensional coordinates calculated by the three-dimensional coordinate calculating means for the focal point position. And a determination unit that obtains distance information according to the determination based on the depth of focus.

[0006]

In the distance measuring device of the present invention, the image pickup device picks up a stereo image at the same focus position, and the focus position setting means sets a plurality of focus positions for picking up the stereo image. Corresponding point detection means performs corresponding point detection on the stereo image, and three-dimensional coordinate calculation means calculates three-dimensional coordinates from the corresponding points detected by the corresponding point detection means. Then, the three-dimensional coordinates calculated by the three-dimensional coordinate calculating means are judged by the judging means based on the focal position and the focal depth, and the distance information is obtained.

[0007]

Embodiments of the present invention will be described below with reference to the drawings.

According to the present invention, a plurality of stereo images as shown in FIG. 3A are photographed at different focal positions as shown in FIG. 3B, and stereo images photographed at the different focal positions. Is used for distance measurement, it is possible to obtain accurate distance information even for an object existing in a range wider than the depth of focus of the optical system.

In the first embodiment, the corresponding points are detected and the three-dimensional coordinates are calculated for the stereo image as shown in FIG. 3A, and the focus position and the depth of focus when the stereo image is photographed. To use. Then, the three-dimensional coordinates are accepted only when the obtained three-dimensional coordinates are within the depth of focus.

Further, as shown in FIG. 3B, the same processing is performed on each of the stereo images obtained at a plurality of different focus positions. By integrating the three-dimensional coordinates obtained at each focal position, the three-dimensional coordinates of the entire object existing in a range wider than the focal depth of the optical system are not used due to the effect of the depth of focus, without using the image of the blurred portion. You can get the coordinates.

Referring to FIGS. 1 and 2, the first aspect of the present invention is described.
An example will be described. FIG. 1 is a block diagram showing the configuration of the distance measuring device of the present invention, and 1 is a focal position setting device. The focus position setting device 1 includes a focus position operation unit 2a.
And 2b to drive the lenses 3a and 3b, respectively, to pick up an image at a predetermined focus position. Selectors 4a and 4b
Is from the image pickup devices 5a and 5b to the A / D converters 6a and 6
Receive the A / D converted image in b. Also, the selector 4a
And 4b, by an instruction from the focus position setting device 1, an image memory _{7a 1, 7a 2, ...,} 7a m and 7b _1, 7
Stored in b ₂ , ..., 7b _m .

The selectors 8a and 8b follow the instructions from the focus position setting device 1 and store the memories 7a ₁ , 7a ₂ ,.
Images are picked up from 7a _m and 7b ₁ , 7b ₂ , ..., 7b _m and sent to the corresponding point detecting device 9. Three-dimensional coordinate calculation device 10
The density value corresponding to the three-dimensional coordinates is obtained from the corresponding point detection device 9 and is output to the selector 11. This selector 11
Is a memory 12 according to an instruction from the focus position setting device 1.
Stored in ₁ , 12 ₂ , ..., 12 _m .

The outputs of the memories 12 ₁ , 12 ₂ , ..., 12 _m are supplied to a selector 14 via a selector 13.
The density value selected by the selection device 14 is stored in the memory 15.
Stored in. Then, the projection device 16 projects the density value corresponding to the three-dimensional coordinates stored in the memory 15, that is, the three-dimensional image in an appropriate direction, and outputs it to the monitor 18 via the D / A converter 17. Next, the operation of the first embodiment will be described with reference to the flowchart of FIG.

The focus position setting device 1 uses the first image i
Initially set to 1 in the sense of imaging _a-1 and i _b-1 (step S1). Next, the focus position setting device 1
The lens 3a is driven through focus position operating unit 2a, images the n-th image i _an, at the focal position f _n. Further, the focus position setting device 1 uses the focus position operation unit 2b to move the lens 3
b is driven and the nth image i _bn is imaged at the focus position f _n (step S2).

The A / D converter 6a is an image pickup device 5a.
The image from is converted into A / D and sent to the selector 4a. Similarly, the A / D converter 6b converts the image from the image sensor 5b into A / D
D-convert and send to the selector 4b. The selector 4a receives n from the focus position setting device 1 and stores the image i _an in the memory 7a.
Store in _n . Further, the selector 4b receives n from the focus position setting device 1 and stores the image i _bn in the memory 7b _n (step S3).

Next, the focus position setting device 1 adds 1 to n representing the image number (step S4). The focus position setting device 1 compares n representing the image number with the scheduled image number m (step S5), and repeats steps S2 to S4 while n ≦ m holds.

By the operations in steps S1 to S5 described above, the memories 7a ₁ , 7a ₂ , ..., 7a _n ,.
The _m, image i _a-1 focus position f _1, image i _a-2 of the focal position f _2, ..., the image i _an, the focal position f _n, ..., an image of the image i _am focus position f _m Stored in memory 7b ₁ ,
_{7b 2, ..., 7b n,} ..., the 7b _m, image i _b-1 focus position f _1, image i _b-2 of the focal position f _2, ..., image i _bn focus position f _n, ..., The image of the image i _{bm at} the focus position f _m is stored.

Next, the focus position setting device 1 sets n to 1 in the sense of processing the _first images i _a-1 and i _b-1 (step S6). The selector 8a receives n from the focus position setting device 1, retrieves the image i _an from the memory 7a _n , and sends it to the corresponding point detection device 9. Similarly, the selector 8b receives n from the focus position setting device 1, extracts the image i _bn from the memory 7b _n , and sends it to the corresponding point detection device 9. Corresponding point detecting device 9 performs corresponding point detection on the image i _an, and the image i _bn, l _n pieces of image i _an, in a coordinate of the corresponding point (x _a
^k , Y _a ^k ) And the coordinates (x _b ^k in the image i _bn ，
y _b ^k ) Is obtained (step S7).

The three-dimensional coordinate calculation device 10 receives the (x _a ^k , Y _a ^k ) And (x _b ^k , Y _b ^k ) And l
get _n Then, the three-dimensional coordinate calculation device 10 sets k to 1 in the sense that the first corresponding point is processed (step S8). In addition, the three-dimensional coordinate calculation device 10
The coordinates (x _a ^k) in the image i _an of the k-th corresponding point , Y _a
^k ) And the coordinates (x _b ^k in the image i _bn , Y _b ^k ) From
Based on the same principle as triangulation, the three-dimensional coordinates (X ^k , Y ^k , Z ^k ) Is calculated (step S
9).

The above-mentioned three-dimensional coordinate calculation device 10 is provided with the obtained three-dimensional coordinates (X ^k , Y ^k , Z ^k ) Concentration value F
_n (X ^k , Y ^k , Z ^k ), The concentration value i _an (x _a ^k , Y
_a ^k ), Or the density value i _bn (x _a ^k , Y _a ^k ) Is substituted (step S10). Next, the selector 11 instructs the density value F corresponding to the three-dimensional coordinates from the three-dimensional coordinate calculation device 10.
_n (X ^k , Y ^k , Z ^k ) Get. At the same time, by receiving n from the focus position setting device 1, the density value F _n (X ^k , Y
^k , Z ^k ) Is stored in the memory 12 _n (step S1)
1).

Further, the three-dimensional coordinate calculating apparatus 10 adds 1 to k indicating the corresponding point number (step S12). Then, the three-dimensional coordinate calculation device 10 uses k representing the corresponding point number.
And l _n representing the number of corresponding points are compared (step S1
3), while the k ≦ l _n are composed step S9~S
Repeat 12.

On the other hand, the density values F _n (X ^k corresponding to 1 _n three-dimensional coordinates obtained from the n-th image are stored in the memory 12 _n by the above-described operations of steps S8 to S13. , Y
^k , Z ^k ) Is stored. The focus position setting device 1 adds 1 to n representing the image number (step S14).

Here, the focus position setting device 1 compares n representing the image number with the planned image number m (step S1).
5), while n ≦ m holds, steps S7 to S
Repeat 14. Further, by the operation of step S6~S15 described above, the memory _{12 1, 12 2, ...,} 12 n,
..., 12 _m , density values F ₁ (X ^k corresponding to l ₁ three-dimensional coordinates , Y ^k , Z ^k ), And the density value F ₂ (X ^k corresponding to the three-dimensional coordinates of l ₂ pieces) , Y ^k , Z ^k ), ..., Intensity values F _n (X ^k corresponding to three _n- dimensional coordinates , Y ^k , Z ^k ), ..., Concentration value F _m (X ^k corresponding to the 1 _m three-dimensional coordinates , Y ^k , Z ^k ) Are stored respectively.

Next, the focus position setting device 1 uses the l-th image
Density value F corresponding to the three-dimensional coordinates obtained from the image
₁(X^k , Y^k , Z^k ) Means processing 1
It is set (step S16). And the selection device 14
Is the darkness corresponding to the three-dimensional coordinates obtained from the nth image.
Degree value F_n(X^k , Y^k , Z^k ), The first concentration value
1 is set to k to mean that processing is performed (step
S17). In addition, the selection device 14 uses three-dimensional coordinates (X^k ，
Y^k , Z^k ) Z^k Is the focus position f_nDepth of focus centered on
Degree f_dWhether it is within the width of
S18). Where depth of focus f_dWithin the range of three
Density value F corresponding to dimensional coordinates_n(X^k , Y^k , Z^k )
Store in memory 15, otherwise do nothing (step
S19).

Next, the selection device 14 adds 1 to k representing the corresponding point number (step S20). This selection device 1
4 is k representing the number of corresponding points and l _n representing the number of corresponding points
Comparing the door (step S21), and while the k ≦ l _n are composed repeats steps S18 to S20.

By the operations in steps S17 to S21 described above, the density value F _n (X ^k corresponding to the three-dimensional coordinates obtained from the n-th image is obtained. , Y ^k , Z ^k ) In the selection device 1
The one selected by 4 is stored in the memory 15.
Then, the focus position setting device 1 adds 1 to n representing the image number (step S22). Further, the focus position setting device 1 compares n representing the image number with the planned image number m (step S23), and repeats steps S17 to S22 while n ≦ m holds.

Next, among the density values corresponding to the three-dimensional coordinates obtained from all the images, the one selected by the selection device 14 is stored in the memory 15 by the operations of steps S16 to S23 described above. . The projection device 16 uses the density values corresponding to the three-dimensional coordinates stored in the memory 15,
That is, a three-dimensional image is projected in an appropriate direction and output to the monitor 18 through the D / A converter 17 (step S2).
4).

By causing the distance measuring device having such a configuration to perform the above operation, accurate distance information can be obtained even for an object existing in a range wider than the depth of focus of the optical system.

Further, in general, if a plurality of images having different focal positions are to be photographed, the contrast method used for AF (autofocus) of a camera is used to obtain tentative distance information even for a single-eye image. You can get it. However, the contrast method alone cannot distinguish a distance shorter than the focal depth of the optical system, and the resolution is limited by the focal depth. Therefore, according to the present invention,
It is possible to obtain accurate distance information even for a distance shorter than the focal length of the optical system. Next, a second embodiment of the present invention will be described.

In this second embodiment, FIG.
The stereo image as shown in FIG.
3B, a contrast curve for each area is obtained for a stereo image with different focus positions as shown in FIG. 3B, and the contrast curves shown in FIGS. Obtain a distance map as shown in b). However, this distance map cannot distinguish the distances below the depth of focus, and the resolution is limited by the depth of focus. By using this distance map to detect corresponding points, the three-dimensional coordinates of the entire object existing in a range wider than the depth of focus of the optical system can be accurately measured without using the image of the part that is blurred due to the influence of the depth of focus. You can get better.

FIG. 6 is a block diagram of the second embodiment, and FIG. 7 is a second diagram.
3 is a flowchart illustrating the operation of the embodiment of FIG. still,
In the following embodiments, the same parts as those in the first embodiment described above are designated by the same reference numerals, and the description thereof will be omitted to avoid duplication.

In the second embodiment, with respect to the distance measuring device having the configuration of FIG. 20b, memories 21a and 21b, area limiting devices 22a and 22b are provided, and selectors 11 and 1
3, memories 12 ₁ , 12 ₂ , ..., 12 _m , selection device 14
The part that is deleted is different.

The focus position setting device 1 uses the first image i
_In the sense that _a-1 and _ib-1 are imaged, 1 is initially set to n (step S25). Then, the focus position setting device 1
Drives the lens 3a through the focus position operation unit 2a,
The n-th image i _an, captured at a focal position f _n. Similarly, the focus position setting device 1 drives the lens 3b through the focus position operation unit 2b to _{move the} nth image i _bn to the focus position f.
_The image is captured at _n (step S26).

The A / D converter 6a is connected to the image pickup device 5a.
The image from is converted into A / D and sent to the selector 4a. Similarly, the A / D converter 6b converts the image from the image sensor 5b into A / D
D-convert and send to the selector 4b. The selector 4a receives n from the focus position setting device 1 and stores the image i _an in the memory 7a _n.
To store. Further, the selector 4b is the focus position setting device 1
To _n to store the image i _bn in the memory 7b _n (step 27).

The focus position setting device 1 adds 1 to n representing the image number (step 28). Here, the focus position setting device 1 compares n representing the image number with the planned image number m (step S29), and repeats steps S26 to S28 as long as n ≦ m holds.

By the operations of steps S25 to S29 described above, the memories 7a ₁ , 7a ₂ , ..., 7a _n ,.
The _m, image i _a-1 focus position f _1, image i _a-2 of the focal position f _2, ..., the image i _an, the focal position f _n, ..., an image of the image i _am focus position f _m Is stored. On the other hand, the memory _{7b 1, 7b 2, ...,} 7b n, ..., the 7b _m, image i _b-1 focus position f _1, image i _b-2 of the focal position f _2, ...,
The image i _{bn at the} focus position f _n , ..., The image i at the focus position f _m
Each _bm image is stored.

The distance map creating apparatus 20a stores C ¹ which stores the contrast value of the area shown in FIG. , C ² ,,, C
^{q ,,,} C ^l The initial value 0 is set to (step S3
0). Then, the focus position setting device 1 uses the first image i
_{In order} to process _a-1 , n is initialized to 1 (step S31). Then, the selector 8a sends the image _ian to the area dividing device 19a by n from the focus position setting device 1. This area dividing device 19a is used for the image i _an
Were divided as shown in FIG. 4, the image a _n ¹ , Image a _n
² ,…, Image a _n ^{q ,,,} image a _n ^l To get The area dividing device 19a uses the image a _n ¹ , Image a _n ² ,,, image a _n
^{q ,,,} image a _n ^l Is sent to the distance map creating device 20a (step S32).

The distance map creating apparatus 20a sets q to 1 in the sense that the process is performed on the first area (step S33). The distance map creating device 20a displays the image a _n ^q. Contrast value C _n ^{q for} (Step S3
It corresponds to 4). Where C ^q ≤ C _n ^q It is determined whether or not (step S35). And C ^q ≤ C _n ^q If (Yes), C ^q To C _n ^q By substituting for f ^q Is input to f _n (step S36). On the other hand, C ^q ≤ C _n ^q If not (No), do nothing.

Next, the distance map generator 20a adds 1 to q representing the area (step S37), and then compares q representing the area with the last area 1 (step S38).
Here, while q ≦ l holds, steps 34 to S are performed.
Repeat 37. When q ≦ l is not satisfied, the focus position setting device 1 adds 1 to n representing the image number (step S39) and compares n representing the image number with the planned image number m (step S39). S40). Where n ≦ m
Steps S32 to S39 are repeated while is satisfied.

By the operations of steps S31 to S40 described above, the focus position f ^{1 of} each area by the contrast method is obtained. , F ² ,…, F ^q ,…, F ^l Is obtained. The distance map creating device 20a uses the focus position f ¹ , F ² ,…, F ^q ，
…, F ¹ Is stored in the memory 21a (step S4
1).

The distance map generator 20b is shown in FIG.
C that stores the contrast value of the area¹ , C² ,,, C
^q ,,, C^l To the initial value 0 (step S4
2). The focus position setting device 1 uses the first image i_b-1To
In the sense that it means, 1 is initially set to n (step S
43). The selector 8b is provided with the n from the focus position setting device 1.
By image i_bnIs sent to the area dividing device 19b. That
Then, the area dividing device 19b uses the image i_bnShown in Figure 4
Image b_n ¹ , Image b_n ² , Image b
_n ^q , Image b_n ^l To get This area dividing device 19b
Is the image b_n ¹ , Image b_n ² , Image b_n ^q ,,, picture
Image b_n ^l To the distance map creating device 20b (step
S44).

Next, the distance map creating device 20b is the first
Set 1 to q in the sense that processing will be performed on the area
(Step S45), the distance map creating device 20b displays the image b._n ^q
Contrast value C for_n ^q (Step S4
6). And C^q ≤C_n ^q Is determined (step S4
7), C^q ≤C_n ^q Then C^q To C_n ^q Substituting
f representing the focus position of q^q To f_n(Step S4
8). On the other hand, C^q ≤C_n ^q If not, step S49
Go to.

The distance map creating device 20b uses q to represent the area.
Is added to 1 (step S49), and then q
And the last area 1 are compared (step S50). Here, while q ≦ l holds, steps S46 to S
Repeat 49.

Next, the focus position setting device 1 adds 1 to n representing the image number (step S51) and compares n representing the image number with the planned image number m (step S5).
2). Here, steps S44 to S51 are repeated while n ≦ m holds. In addition, steps S43 to S5
By the operation of 2, the focus position f ^{1 of} each area by the contrast method , F ² ,…, F ^q ,…, F ^l Is obtained. The distance map creating device 20b uses the focus position f ¹ , F ² ，
…, F ^q ,…, F ¹ Is stored in the memory 21b (step S53).

The focus position setting device 1 uses the first image i
_{In order} to process _a-1 and _ib-1 , n is initialized to 1 (step S54). The selector 8a uses the n from the focus position setting device 1 to _{select the} image i _an from the area limiting device 2
Send to 2a. Area limiting device 22a refers to the distance map stored in the memory 21a, the focal position in the image i _an, to the region of the f _n and the image i _'an, (step S5
5). Further, the selector 8b sends the image i _bn to the area limiting device 22b by n from the focus position setting device 1. The area limiting device 22b refers to the distance map stored in the memory 21b, and the focus position is f _n in the image i _bn.
Area is _designated as image _i'bn (step S56).

The area limiting device 22a applies the image _i'an to the corresponding point detecting device 9, while the area limiting device 22b uses the image i'an.
_bn is sent to the corresponding point detecting device 9. This corresponding point detection device 9
Performs corresponding point detection on the image i _'an, and the image i' _bn, l _n pieces of image i of the corresponding point 'coordinates in the _an, (x _a
^k , Y _a ^k ) And the coordinates (x _b ^k in the image i ′ _bn , Y _b
^k ) Is obtained (step S57).

The three-dimensional coordinate calculation device 10 receives the (x _a ^k , Y _a ^k ) And (x _b ^k , Y _b ^k ) And l
_In the sense that _n is obtained and processing is performed on the first corresponding point,
1 is set to k (step S58). Then, the three-dimensional coordinate calculation device 10 uses the image i ′ _an of the k-th corresponding point.
Coordinates within (x _a ^k , Y _a ^k ) And the coordinates (x _b ^k in the image i ′ _bn , Y _b ^k ), Based on the same principle as triangulation, the three-dimensional coordinates (X ^k , Y ^k ，
Z ^k ) Is obtained (step S59).

The three-dimensional coordinate calculation device 10 uses the obtained three-dimensional coordinates (X ^k , Y ^k , Z ^k ) Concentration value F
_n (X ^k , Y ^k , Z ^k ), The density value i ′ _an (x _a ^k ，
y _a ^k ), Or the density value i ′ _bn (x _a ^k , Y _a ^k ) Is substituted (step S60). Next, the three-dimensional coordinate calculation device 10 determines the density value F _n (X ^k corresponding to the three-dimensional coordinates. ，
Y ^k , Z ^k ) Is stored in the memory 15 (step S6
1), 1 is added to k indicating the corresponding point number (step S
62).

Next, the three-dimensional coordinate calculation device 10 compares k representing the number of the corresponding point with l _n representing the number of the corresponding points (step S63). Here, while the k ≦ l _n are composed repeats steps S59～S62. Further, by the operations of steps S58 to S63, the density values F _n (X ^k corresponding to 1 _n three-dimensional coordinates obtained from the n-th image are stored in the memory 15. , Y ^k , Z ^k ) Is stored, and the focus position setting device 1 adds 1 to n representing the image number (step S64).

The focus position setting device 1 compares n representing the image number with the planned image number m (step S65).
Here, as long as n ≦ m holds, steps S55 to S55
Repeat S64. Further, by the operations of steps S54 to S65, the density value F ₁ (X ^k corresponding to l ₁ three-dimensional coordinates is stored in the memory 15. , Y ^k , Z ^k ), L ₂ corresponding to three-dimensional coordinates F ₂ (X ^k , Y ^k , Z ^k ),
..., density values F _n (X ^k corresponding to l _n three-dimensional coordinates ，
Y ^k , Z ^k ), ..., density values corresponding to l _m pieces of three-dimensional coordinates F _m (X ^k , Y ^k , Z ^k ) Is stored.

After that, the projection device 16 projects the density value corresponding to the three-dimensional coordinates stored in the memory 15, that is, the three-dimensional image in an appropriate direction and outputs it to the monitor 18 through the D / A converter 17. Yes (step S66).

As described above, according to the second embodiment, accurate distance information can be obtained even for an object existing in a range wider than the depth of focus of the optical system. Further, accurate distance information can be obtained even for a distance shorter than the depth of focus of the optical system. Next, a third embodiment of the present invention will be described.

In the third embodiment, a stereo image with a long focal depth is created from stereo images with different focal positions as shown in FIG. 3, and corresponding points are detected to detect the focal depth of the optical system. It is possible to obtain accurate distance information even for objects existing in a wider range.

A method for obtaining a long focal depth image by an image processing method is described in, for example, Japanese Patent Application No. 63-39936. Referring to FIG. 8, the above-mentioned Japanese Patent Application No. 63-
A case of obtaining a long focal depth image based on the method described in No. 39936 will be described.

In the image input optical system, an image input when the focus position is set on a certain object plane is determined to be blurred depending on the distance relationship between the object and the focusing surface. Here, when the object has a structure larger than the depth of focus of the image input optical system, the image input when the focus position is set on a certain object plane has a different blurred state depending on the location. Next, when the focus position is changed, an image having a blurry state distribution different from that of the previously input image is input. In other words, the part that was blurred in the previously input image is in focus in the next input image,
The reverse also occurs. This is determined by the distance relationship between the structure of the object and the set focal position, as described above.

Therefore, by inputting an image while changing the position of the in-focus point according to an appropriate setting range, it is possible to obtain a plurality of images in which different parts of the object are in focus. When all of these are added together, an almost uniformly blurred image is obtained regardless of the location in the image.

Then, an appropriate restoration process is performed on the combined image next to obtain an image in which the focus is uniform, that is, a long focal depth image, regardless of the position in the image.

FIG. 9 is a block diagram of the third embodiment, and FIG. 10 is a flow chart for explaining the operation of the third embodiment. In the third embodiment, in addition to the selectors 8a and 8b, adders 23a and 23a are added to the distance measuring device having the configuration of FIG.
b, memories 24a and 24b, recovery processing devices 25a and 25
b are provided, selectors 11 and 13, memories 12 ₁ and 1
2 ₂ , ..., 12 _m , _except that the selection device 14 is deleted.

The focus position setting device 1 uses the first image i
Initially set to 1 in the sense that _a-1 and _ib-1 are imaged (step S67). This focus position setting device 1
Lenses 3a, 3b through the focus position operating means 2a, 2b
To drive the n-th image i _an , i _bn to the focus position f _n
The image is picked up at (step S68).

Next, the A / D converters 6a and 6b perform A / D conversion on the images from the image pickup devices 5a and 5b and send them to the selectors 4a and 4b, respectively. Then, the selectors 4a and 4b
Receives n from the focus position setting device 1 and the images i _an , i
_bn is stored in the memories 7a _n and 7b _n (step S6)
9). After that, the focus position setting device 1 adds 1 to n representing the image number (step S70).

The focus position setting device 1 compares n representing the image number with the planned image number m (step S71), and n
Steps S68 to S70 are repeated as long as ≤m holds. Further, by the operation of step S67～S71, memory _{7a 1, 7a 2, ...,} 7a n, ..., the 7a _m, image i _a-1 focus position f _1, an image focus position f ₂ i _a-2 ,
..., the image i _an of the focus position f _n , ..., the image i _am of the focus position f _m is stored, while the memories 7b ₁ and 7 are stored.
_{b 2, ..., 7b n,} ..., 7b in the _m, the image i _b-1 focus position f _1, image i _b-2 of the focal position f _2, ..., the focal position f
The image i _bn , ... Of _n , and the image i _bm of the focus position f _m are respectively stored.

The adder 23a uses the images i _an (n = 1, 2, ...,), which are stored in the memory 7a _n and have different focus positions.
m) is weighted to obtain _a uniformly blurred image i _a ^br over the entire image and stored in the memory 24a (step S72). Then, the recovery processing device 25a uses the memory 24
The uniformly blurred image i _a ^{br of a} is subjected to a restoration process to obtain _a long focal depth image i _a ^{df in} which an object having different focus positions is uniformly focused and stored in the memory 24a ( Step S73).

Next, the adder 23b _{causes the} image i _bn (n = 1, n) with different focus positions stored in the memory 7b _n .
2, ..., M) are weighted to obtain a uniformly blurred image i _b ^br over the entire image and stored in the memory 24b (step S74). Then, the recovery processing device 25b
The uniformly blurred image i _b ^br in the memory 24b is subjected to a recovery process to obtain a long focal depth image i _b ^{df in} which an object having different focus positions is uniformly focused and stored in the memory 24b. (Step S75).

The corresponding point detecting device 9 uses the image i _a ^df and the image i _a ^df.
Corresponding point detection is performed for _b ^df , and the image i of l corresponding points is detected.
coordinates in _a ^df (x _a ^k , Y _a ^k ) And the coordinates (x _b ^k in the image i _b ^df , Y _b ^k ) Is obtained (step S76). The three-dimensional coordinate calculation device 10 receives from the corresponding point detection device 9 (x _a
^k , Y _a ^k ) And (x _b ^k , Y _b ^k ) And l. In addition, the three-dimensional coordinate calculation apparatus 10 sets k to 1 in the sense that processing is performed on the first corresponding point (step S7).
7). Further, the three-dimensional coordinate calculation device 10 uses the coordinates (x _a ^k) in the image i _a ^df of the k-th corresponding point. , Y _a ^k ) And the coordinates (x _b ^k in the image i _b ^df , Y _b ^k ), Based on the same principle as triangulation, the three-dimensional coordinates (X
^k , Y ^k , Z ^k ) Is obtained (step S78).

The three-dimensional coordinate calculation device 10 uses the obtained three-dimensional coordinate (X ^k , Y ^k , Z ^k ) Concentration value F (X ^k ，
Y ^k , Z ^k ), The density value i _a ^df (x _a ^k , Y _a ^k ) Or
Or the density value i _b ^df (x _a ^k , Y _a ^k ) Is substituted (step S79). The three-dimensional coordinate calculation device 10 uses the density value F (X ^k corresponding to the three-dimensional coordinates. , Y ^k , Z ^k ) Memory 1
5 (step S80). Furthermore, the three-dimensional coordinate calculation device 10 adds 1 to k indicating the corresponding point number (step S81).

Here, the three-dimensional coordinate calculation apparatus 10 compares k representing the corresponding point number with l representing the number of corresponding points (step S82). In step S82, k
Steps S78 to S81 are repeated while ≦ l is satisfied.

By the operations of steps S77 to S82 described above, the density values F (X ^k corresponding to 1 three-dimensional coordinates are stored in the memory 15. , Y ^k , Z ^k ) Is stored. Then, the projection device 16 projects the density value corresponding to the three-dimensional coordinates stored in the memory 15, that is, the three-dimensional image in an appropriate direction, and outputs it to the monitor 18 through the D / A converter 17 (step S83). ). As described above, according to the third embodiment, it is possible to obtain accurate information even for an object existing in a range wider than the depth of focus of the optical system. Next, a fourth embodiment of the present invention will be described.

In the fourth embodiment, as shown in FIGS. 11 (a) and 11 (b), the case where the object is an object that transmits light will be described. Basically, also in the fourth embodiment, as shown in FIG. 11B, a stereo image obtained at a plurality of different focal positions is used for distance measurement, so that a range wider than the focal depth of the optical system can be obtained. It is possible to obtain accurate distance information even for an object existing in.

However, when the object is a transmissive object, there is a problem in the obtained image that a blurred image of the object out of the focus position is superimposed on the image of the object existing at the focus position. The superimposed blurred image makes accurate corresponding point detection difficult. Therefore, recovery processing is performed using a three-dimensional OTF (Optical Transfer Function) obtained from the characteristics of the imaging optical system to erase the blurred image before performing correspondence detection.

FIG. 12 is a block diagram of the fourth embodiment, and FIG. 13 is a flow chart for explaining the operation of the fourth embodiment.
In the fourth embodiment, recovery processing devices 26a and 26b are provided for the distance measuring device having the configuration of FIG. 1, and selectors 11 and 13, memories 12 ₁ , 12 ₂ , ..., 12 _m and a selection device 14 are provided. The deleted part is different.

The focus position setting device 1 uses the first image i
Initially set to 1 in the sense that the image of _a-1 and i _b-1 is taken (step S84). This focus position setting device 1
Lenses 3a, 3b through the focus position operating means 2a, 2b
_Is driven to image the nth images i _an and i _bn at the focal position f _n (step S85). After that, A
The / D converters 6a and 6b perform A / D conversion on the images from the image pickup devices 5a and 5b, and send the images to the selectors 4a and 4b, respectively. The selectors 4a and 4b are used for the focus position setting devices 1 to n.
In response, the images i _an and i _bn are stored in the memories 7a _n and 7b _n.
(Step S86). Next, the focus position setting device 1 adds 1 to n representing the image number (step S87).

Next, the focus position setting device 1 compares n representing the image number with the planned image number m (step S8).
8). Here, while n ≦ m holds, step S
85 to S87 are repeated. Then, steps S84 to S8
By the operation of 8, the memories 7a ₁ , 7a ₂ , ..., 7
a _n, ..., the 7a _m, image i _a-1 focus position f _1, image i _a-2 of the focal position f _2, ..., image i of the focal position f _n
An image of the image i _{am at} the focal position f _m is stored, and the images i _b-1 and focal position f _{1 at} the focal position f ₁ are stored in the memories 7b ₁ , 7b ₂ , ..., 7b _n , ..., 7b _m. image i of f _2
b-2 , ..., Image i _bn of focus position f _n , ..., Focus position f
The images of _m and i _bm are stored.

Next, the recovery processing device 26a has the memory 7
_{a 1, 7a 2, ...,} 7a n, ..., from 7a _m, image i
_{, a} , i _a-2 , ..., I _an , ..., i _am are taken out, restoration processing is performed using the three-dimensional OTF of the imaging optical system, and images i ′ _a-1 , i ′ _a-2 , ..., _i'an , ..., _i'am are obtained. The restoration processing device 26a uses the images i'a _-1 , i'a _-2 ,
_{..., i 'an, ...,} i' the _am memory 7a _1, 7a _2,
..., 7a _n, ..., stored in the 7a _m (step S8
9). Here, the operation of the recovery processing device 26a will be described.

First, with reference to FIG. 14, the recovery processing principle will be described one-dimensionally for simplicity. Image degradation is shown in Figure 14.
As shown in (a), it is represented by the product of the frequency component F of the original image and the OTF H of the optical system for each frequency component. Therefore, the estimation of the original image is represented by the product of the frequency component G of the deteriorated image and the frequency component of the inverse transformation H ⁻¹ of the OTF of the optical system, as shown in FIG. In the same example,
The recovery process needs to be performed in a three-dimensional space represented by the optical axis direction and a plane orthogonal to the optical axis direction.

Therefore, the restoration processing device 26a handles the images i _a-1 , i _a-2 , ..., I _an , ..., I _am as a three-dimensional image g in which they are stacked in the optical axis direction. Recovery processor 26
In a, the three-dimensional image g is subjected to three-dimensional Fourier transform to obtain a frequency component G. Further, the product of the frequency component G and the inverse transformation H ⁻¹ of the three-dimensional OTF for each frequency component is calculated to obtain the frequency component F ′. The restoration processing device 26a performs a three-dimensional inverse Fourier transform on the frequency component F'to obtain a three-dimensional estimated original image f '.
To get In addition, the three-dimensional images f ′ stacked in the optical axis direction
, I ′ _a-1 , i ′ _a-2 , ..., i ′ _an , ..., i ′
_am

Similarly, the recovery processing device 26b has a memory 7
_{b 1, 7b 2, ...,} 7b n, ..., from 7b _m, image i
_{b-1, i b-2} , ..., i bn, ..., taken out i _bm, performs restoration processing using three-dimensional OTF of the imaging optical system, the image _{i 'b-1, i'} b-2, , _I'bn , ..., _i'bm are obtained. Then, the images i'b _-1 , _{i'b -2} , ..., _i'bn ,
, I ′ _bm are stored in the memories 7b ₁ , 7b ₂ , ..., 7b _n ,
..., 7b _m are stored (step S90). The operation of the recovery processing device 26b is similar to the operation of the recovery processing device 26a.

Next, the focus position setting device 1 sets n to 1 in the sense that the _first images i'a _-1 and _i'b-1 are processed (step S91). Then, the selector 8a,
The memory 8a receives the n from the focus position setting device 1b.
Images i ′ _an and i ′ _bn are taken out from _n and 7b _n and sent to the corresponding point detecting device 9. The corresponding point detection device 9 _{displays the} image i ′ _an
And the image i ′ _bn are subjected to corresponding point detection, and the coordinates (x _a ^{k of the} l _n corresponding points in the image i ′ _an , Y _a ^k ) And the coordinates (x _b ^k in the image i ′ _bn , Y _b ^k ) Is obtained (step S92).

The three-dimensional coordinate calculation apparatus 10 receives the (x _a ^k , Y _a ^k ) And (x _b ^k , Y _b ^k ) And l
_{After n} is obtained, k is set to 1 in the sense that the first corresponding point is processed (step S93). Next, the three-dimensional coordinate calculation device 10 determines the image i ′ _an of the k-th corresponding point.
Coordinates within (x _a ^k , Y _a ^k ) And the coordinates (x _b ^k in the image i ′ _bn , Y _b ^k ) Based on the same principle as triangulation, the three-dimensional coordinate (X ^k , Y ^k , Z ^k )
Is calculated (step S94).

The above-mentioned three-dimensional coordinate calculation device 10 uses the obtained three-dimensional coordinate (X ^k , Y ^k , Z ^k ) Concentration value F
_n (X ^k , Y ^k , Z ^k ), The density value i ′ _an (x _a ^k ，
y _a ^k ), Or the density value i ′ _bn (x _a ^k , Y _a ^k ) Is substituted (step S95). Three-dimensional coordinate calculation device 10
Is the density value F _n (X ^k for three-dimensional coordinates) , Y ^k ，
Z ^k ) Is obtained and stored in the memory 15 (step S9).
6). Then, the three-dimensional coordinate calculation device 10 adds 1 to k indicating the corresponding point number (step S97).

Next, the three-dimensional coordinate calculation apparatus 10 compares k representing the number of the corresponding point with l _n representing the number of the corresponding points (step S98), and step k is satisfied while k ≦ l _n. Repeat S94 to S97. Then, step S9
By the operations from 3 to S98, the density value F corresponding to 1 _n three-dimensional coordinates obtained from the n-th image is stored in the memory 15.
_n (X ^k , Y ^k , Z ^k ) Is stored. Next, the focus position setting device 1 adds 1 to n representing the image number (step S99).

Here, the focus position setting device 1 compares n representing the image number with the planned image number m (step S10).
0) and n ≦ m are satisfied, steps S92 to S
Repeat 99. Further, by the operations of steps S91 to S100, the memory 15 stores the density values F ₁ (X ^k corresponding to three _1- dimensional coordinates). , Y ^k , Z ^k ), L ₂ corresponding to three-dimensional coordinates F ₂ (X ^k , Y ^k , Z ^k ),
..., density values F _n (X ^k corresponding to l _n three-dimensional coordinates ，
Y ^k , Z ^k ), ..., density values corresponding to l _m pieces of three-dimensional coordinates F _m (X ^k , Y ^k , Z ^k ) Is stored. Then, the projection device 16 projects the density value corresponding to the three-dimensional coordinates stored in the memory 15, that is, the three-dimensional image in an appropriate direction, and outputs it to the monitor 18 through the D / A converter 17 (step S101). ).

As described above, according to the fourth embodiment, even if the object is a transmissive object, accurate distance information can be obtained for the object existing in a range wider than the depth of focus of the optical system. Can be obtained.

Further, generally, if a plurality of images having different focal positions are photographed, it is possible to obtain a three-dimensional image and obtain distance information by performing restoration processing even for a monocular image. However, this alone cannot separate a distance shorter than the depth of focus of the optical system, and the resolution is limited by the depth of focus. According to the present invention, it is possible to obtain accurate distance information even for a distance shorter than the depth of focus of the optical system.

[0084]

As described above, according to the present invention, even for an object existing in a range wider than the depth of focus of the optical system,
A distance measuring device that can obtain accurate distance information can be provided.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of a distance measuring device according to a first embodiment of the present invention.

FIG. 2 is a flowchart illustrating the operation of the first embodiment.

FIG. 3 is a diagram showing an example of a stereo image.

FIG. 4 is a diagram showing an example in which the stereo image of FIG. 3 is divided into small areas.

FIG. 5 is a diagram showing a distance map.

FIG. 6 is a block diagram showing a configuration of a distance measuring device in a second embodiment of the present invention.

FIG. 7 is a flowchart illustrating the operation of the second embodiment.

FIG. 8 is a diagram illustrating a method of obtaining a long focal depth image by an image processing method.

FIG. 9 is a block diagram showing a configuration of a distance measuring device in a third embodiment of the present invention.

FIG. 10 is a flowchart illustrating the operation of the third embodiment.

FIG. 11 is a diagram showing an example of a stereo image when an object is an object that transmits light.

FIG. 12 is a block diagram showing a configuration of a distance measuring device according to a fourth embodiment of the present invention.

FIG. 13 is a flowchart illustrating the operation of the fourth embodiment.

FIG. 14 is a diagram illustrating an operation of the recovery processing apparatus.

FIG. 15 is a diagram showing a method of calculating distance information according to the same principle as triangulation according to a conventional technique.

[Explanation of symbols]

1 ... Focus position setting device, 2a, 2b ... Focus position operation unit,
3a, 3b ... Lenses, 4a, 4b, 8a, 8b, 11,
Reference numeral 13 ... Selector, 5a, 5b ... Image pickup element, 6a, 6b ...
A / D _{converter, 7a 1, 7a 2, ...} , 7a m, 7b 1,
7b ₂ , ..., 7b _m , 12 ₁ , 12 ₂ , ..., 12 _m , 1
5, 21a, 21b, 24a, 24b ... Memory, 9 ... Corresponding point detection device, 10 ... Three-dimensional coordinate calculation device, 14 ... Selection device, 16 ... Projection device, 17 ... D / A converter, 18 ... Monitor, 19a , 19b ... Area dividing device, 20a, 20b
... distance map creating device, 22a, 22b ... region limiting device,
23a, 23b ... Adder, 25a, 25b, 26a, 2
6b ... Recovery processing device.

Claims (1)

[Claims] 1. An image pickup device for picking up a stereo image at the same focus position, a focus position setting means for setting a plurality of focus positions for picking up the stereo image, and a corresponding point detecting means for detecting corresponding points in the stereo image. The three-dimensional coordinate calculating means for calculating three-dimensional coordinates from the corresponding points detected by the corresponding point detecting means, and the three-dimensional coordinate calculated by the three-dimensional coordinate calculating means according to the determination based on the focus position and the depth of focus. A distance measuring device, comprising: