JP2000310531A - Distance image creating device and method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To create a high-quality distance image by providing a camera with a zoom adjusting function and adjusting focus which changes according to the zoomed state of the camera. SOLUTION: A distance image creating device is provided with image pickup means 2 and 3 which comprise one camera 2 with a focus adjusting function or more and are formed of a plurality of cameras 2, a focus detecting means 5 to compare an image picked up by the camera 2 with a focus adjusting function with points corresponding to points in the image picked by the camera 2 in an image of which focus is not adjusted, to create an evaluating value indicating similarity, and to detect an appropriately focused state, and a distance image creating means 7 to create a distance image indicating the distance between any camera and an object of which image to be picked up through the use of the image in a focused state detected by the focus detecting means 5 and the image of which focus is not adjusted.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] 1. Field of the Invention [0002] The present invention relates to a distance image generating apparatus and method for generating a distance image indicating a distance between a camera that captures an object to be imaged and a target object using a zoom adjustment function.

[0002]

2. Description of the Related Art A reference camera arranged at a predetermined position,
In a multi-view stereo system that includes a detection camera arranged around the reference camera and determines the distance between the reference camera and the imaging target, the foreground is included in a twin-lens stereo system when there are a plurality of imaging targets. It is possible to generate a more accurate distance image by compensating for defects such as an image with occlusion or a periodic pattern that is weak in which the imaging target object is hidden by the background imaging target object. Specifically, Japanese Patent Application No. 9-99333 filed by the present applicant
And the method disclosed in Japanese Patent Application No. 9-288479.

In the above-mentioned multi-lens camera system, the reference camera is provided with a zoom adjustment function so that the zoom state is variable. In a multi-lens camera system including a reference camera having a zoom adjustment function, by adding a function of estimating an optimum zoom scale (magnification) and other parameters related to zooming such as lens distortion and lens center, and performing image conversion. The distance between a reference camera and an object to be imaged is measured (Japanese Patent Application No. 10-17 / 1998).
No. 3522).

[0004]

However, in the above-described multi-lens camera system, when the distance between the reference camera and the subject is measured by changing the zoom scale of the reference camera, the focus of the reference image captured by the reference camera is measured. Is the same as the detected image, and only when the image quality equivalent to that at the time of calibration of the multi-lens camera system is obtained. Here, the calibration is
This is a process for obtaining a point on each detected image corresponding to a point on the reference image when the reference camera and each of the detection cameras have substantially the same viewing angle, and is a process performed on the premise of generating a distance image.

In view of the above, the present invention has been proposed in view of the above situation, and has a camera having a zoom adjustment function, and adjusts a focus that changes according to the zoom state of the camera to achieve high quality. It is an object of the present invention to provide a distance image generating apparatus and method capable of generating a distance image of the range.

[0006]

A distance image generating apparatus according to the present invention for solving the above-mentioned problems includes an image pickup means having at least one camera having a focus adjustment function and comprising a plurality of cameras; Focus detection that compares an image captured by a camera having the same with a point on an image that has not been subjected to focus adjustment corresponding to the point on the image to generate an evaluation value indicating the degree of similarity to detect an appropriate focus state Means, an image in a focus state detected by the focus detection means, and an image on which focus adjustment has not been performed, and a distance image for generating a distance image indicating a distance between one of the cameras and the imaging target object. Generating means.

According to this distance image generation device, when a distance image is generated using an image requiring focus adjustment,
An evaluation value is generated according to the focus adjustment of the camera, and a distance image with the smallest evaluation value is generated.

[0008] Further, a distance image generating method according to the present invention comprises:
Among a plurality of cameras including one or more cameras that capture an image of an imaging target using the focus adjustment function and generate an image,
A first step of detecting a focus state of the camera having the focus adjustment function, and a focus adjustment corresponding to an image when the focus adjustment is performed and a point on the image by changing the focus state within a predetermined range. A second step of generating an evaluation value indicating the degree of similarity by comparing points on an image that has not been subjected to the above, a third step of detecting a focus state at an appropriate evaluation value, And a fourth step of generating a distance image indicating a distance between the camera having the focus adjustment function and the imaging target object by using the image on which focus adjustment has been performed and the image on which focus adjustment has not been performed. Features.

According to this distance image generation method, when a distance image is generated using an image requiring focus adjustment,
An evaluation value is generated according to the focus adjustment of the camera, and a distance image with the smallest evaluation value is generated.

[0010]

Embodiments of the present invention will be described below in detail with reference to the drawings.

A distance image generating apparatus 1 to which the present invention can be applied
Has a reference camera 2 and a detection camera 3 arranged around the reference camera 2 as shown in FIGS.

As shown in FIG. 1, the reference camera 2 is arranged at a substantially central position among a plurality of cameras, and captures an object to be imaged to generate a grayscale image (hereinafter, referred to as a reference image). . The reference camera 2 has a zoom mechanism for increasing or decreasing a viewing angle by zooming when capturing an image of an imaging target. The zoom state of the zoom mechanism is detected by a detection mechanism including, for example, a potentiometer. The potentiometer generates an output voltage according to the zoom state of the zoom mechanism, encodes the output voltage, and generates a sensor signal.

As a detection mechanism for detecting the zoom state, not only a potentiometer but also a rotary encoder or the like which is used by connecting to a lens in the reference camera 2 from the outside can be used.

The detection camera 3 is a camera arranged around the reference camera 2 and captures an image of an object to be imaged to generate a grayscale image (hereinafter, referred to as a detection image).

If the reference image B and the detected images D1 to D4 captured by the reference camera 2 and the detection camera 3 are fixed to a state where the zoom scales of the reference camera 2 and the detection camera 3 are used at the time of calibration, FIG. As shown in the figure, the reference image B of the picture corresponding to the arrangement of each camera
And the detection images D1 to D4.

As shown in FIG. 2, the distance image generating apparatus 1 includes a zoom sensor signal detecting section 4 for detecting a zoom state of the reference camera 2 and a focus sensor signal detecting section for detecting a focus state of the reference camera 2. A frame memory 6 in which a reference image and a detected image captured by the reference camera 2 and the detection camera 3 are stored;
A distance image generation unit 7 that generates a distance image using the reference image and the detected image from the camera, and a control unit 8 that controls the focus of the reference camera 2 are provided.

The zoom sensor signal detecting section 4 is connected to the reference camera 2, and receives a sensor signal from the above-described detecting mechanism. The zoom sensor signal detection unit 4 obtains the relationship between the output voltage Vn and the zoom scale of the reference camera 2 using, for example, a potentiometer using the input sensor signal.

Here, the zoom scale is expressed as a ratio of the same object size in the reference image when the zoom state changes with respect to the object size in the reference image at an output voltage V _calib described later. More specifically, the zoom scale determines a corresponding point in the reference image at each output voltage Vn with respect to a feature point on the reference image at the output voltage V _calib by a so-called image matching method or the like, and _obtains the reference image at the output voltage V _calib . And the output voltage V
It is obtained by finding the positional relationship between n and the corresponding point on the reference image.

As shown in FIG. 4, the relationship between the output voltage Vn and the zoom scale S is such that the zoom scale increases as the output voltage increases. That is, the enlargement ratio of the image increases. “V _calib ” in the figure _indicates a zoom when calibration is performed to obtain a point of the detected image corresponding to a point on the reference image when the reference camera 2 and the detection camera 3 are positioned and an image of the same object is taken. This is an output voltage value corresponding to the scale S _calib (= 1). The zoom scale S _calib corresponding to the output voltage V _calib is used as a reference zoom scale, and the output voltages V ₀ , V _{1 ,.}
When Vn is shifted in the positive direction with respect to the output voltage V _calib , the reference image is zoomed in, and when Vn is shifted in the negative direction with respect to the output voltage V _calib , it indicates that zoom-out is performed. That is, FIG.
The output voltages V ₀ , V ₁ ,..., Vn indicate that the zoom scale is zoomed in to S ₀ , S ₁ ,. In addition, since mechanical backlash exists in the zoom operation in the reference camera 2, the relationship between the output voltage Vn and the zoom scale is obtained separately for zooming in and zooming out.

In order to obtain the relationship between the output voltage Vn and the zoom scale S shown in FIG. 4, the process is stopped by changing the zoom scale of the reference camera 2 and plotting the zoom scale with respect to the output voltage at each point. .

A characteristic curve indicating the relationship between the output voltage Vn and the zoom scale S may be obtained by interpolating the measurement results at several points of the output voltage Vn.

Here, the calibration is performed by setting the viewing angle between the reference camera 2 and the detection camera 3 to a predetermined angle.

The zoom sensor signal detecting section 4 prepares, for example, a table of the relationship between the output voltage Vn and the zoom scale shown in FIG. A scale signal indicating a zoom scale is generated and output to the distance image generation unit 7.

The relationship between the output voltage Vn and the zoom scale S may be prepared by preparing a table of the relationship between the zoom scale and the image center and the relationship between the zoom scale and the lens distortion. .

Further, the zoom sensor signal detecting section 4 receives an image center signal indicating the image center (Xc, Yc) of the reference camera 2 from the sensor signal from the reference camera 2 and a lens distortion signal indicating the lens distortion κ of the reference camera 2. Is generated and output to the distance image generation unit 7. Further, the zoom sensor signal detection unit 4 prepares, for example, a table of the relationship between the sensor signal, the relationship with the image center, and the lens distortion as in the case of the above-described zoom scale, and stores the sensor signal from the reference camera 2. By detecting the signal, an image center signal and a lens distortion signal are generated and output to the distance image generation unit 7.

When a rotary encoder is used as a detection mechanism built in the reference camera 2, the zoom sensor signal detection unit 4 counts the number of pulses output from the rotary encoder as a sensor signal. , Zoom scale, image center and lens distortion.

The frame memory 6 is connected to the reference camera 2 and each detection camera 3, and receives a reference image and a detection image. These frame memories 6 A / D convert the input reference image and detected image and output the distance image generation unit 7.

The focus sensor signal detecting section 5 has a sensor signal S for detecting a focus state from the reference camera 2.
f _zoom is input. The focus sensor signal detection unit 5 detects the focus state of the reference camera 2 using the sensor signal Sf _zoom , generates a focus sensor signal, outputs the focus sensor signal to the distance image generation unit 7, and outputs the focus sensor signal to the control unit 8. .

The distance image generator 7 uses the scale signal, the image center signal and the lens distortion signal from the sensor signal detector 4 and the focus sensor signal from the focus sensor signal detector 5 to zoom the reference camera 2. A distance image indicating the distance between the reference camera 2 and the imaging target is generated according to the state and the focus state.

The distance image generation unit 7 is configured to generate a distance image based on the reference camera 2 and each detection camera 3.
Is performed to obtain a point on each detected image corresponding to a point on the reference image when is set to be substantially the same viewing angle. Thereby, the distance image generation unit 7 can obtain a corresponding point on the detected image corresponding to a point on the reference image when the imaging target is imaged.

Further, the distance image generating section 7 generates a focus control signal _Sopt indicating an optimum focus.
At this time, the distance image generation unit 7 uses the focus sensor signal from the focus sensor signal detection unit 5 to generate a focus control signal _Sopt for performing focus adjustment, and outputs the focus control signal _Sopt to the control unit 8.

The control unit 8 compares the focus sensor signal from the focus sensor signal detection unit 5 with the focus control signal from the distance image generation unit 7 to move the position of the lens built in the reference camera 2. Is generated.

The distance image generating apparatus 1 configured as described above
Accordingly, when performing the focus adjustment and performing the process of generating the distance image, the process illustrated in the flowchart in FIG. 5 is performed.

According to this flowchart, first, in step ST1, the user or the reference camera 2 is automatically turned on.
Performs a zoom operation.

In the next step ST2, the zoom sensor signal Sz is detected by the zoom sensor signal detector 4 using the sensor signal Sz from the reference camera 2 in response to the zoom operation performed by the reference camera 2, and the distance image The generator 7 optimizes the zoom scale based on the zoom sensor signal Sz. The detailed processing for optimizing the zoom scale will be described later.

In step ST3, the distance image generation unit 7 performs a process of generating a distance image with the zoom scale optimized in step ST2, and determines the accuracy of the generated distance image. At this time, the distance image generation unit 7
An image captured by the reference camera 2 is compared with a point on the image captured by the detection camera 3 corresponding to a point on the image, and an evaluation value such as SSAD (Sum of squared absolute distance) indicating similarity is used. , To evaluate the accuracy of the range image. And
When the distance image generation unit 7 determines that the accuracy of the distance image is equal to or higher than the predetermined value, the process ends, and when it determines that the accuracy of the distance image is not equal to or higher than the predetermined value, the process proceeds to step ST4.

In step ST4, the distance image generator 7 generates a focus control signal _Sopt indicating the optimum focus in order to correct the focus of the reference camera 2 because the accuracy of the distance image is not sufficient. Control unit 8
Output to At this time, the distance image generation unit 7 outputs the focus sensor signal Sf from the focus sensor signal detection unit 5.
_A focus control signal S _opt is generated using _zoom .

The control unit 8 controls the distance image generation unit 7
A focus drive signal is generated using the focus control signal S _opt from the camera and the focus sensor signal Sf _zoom from the focus sensor signal detector 5, and the focus adjustment is performed by driving a lens built in the reference camera 2. It returns to step ST2.

When the focus adjustment is performed by the reference camera 2 and the step ST2 is performed again, the distance image generation unit 7 again optimizes the zoom scale of the reference image changed by the focus adjustment in the step ST4.
If it is determined in step ST3 that the accuracy of the distance image is sufficient, the process ends. The details of the process of adjusting the focus in step ST4 will be described later.

Next, the process of optimizing the zoom scale in the above-described step ST2 will be described with reference to the flowchart of FIG.

First, in step ST11, the zoom state of the reference camera 2 is detected by the detection mechanism. Then, the detection mechanism generates a sensor signal according to the zoom state, and outputs the sensor signal to the zoom sensor signal detection unit 4.

In step ST12, a sensor signal is input to the zoom sensor signal detector 4. For example, when a potentiometer is used as a detecting mechanism, the zoom scale Sv, the image center (Xc, Yc) and the lens distortion κ with respect to the output voltage Vn are determined. It is obtained with reference to a conversion table or the like and output to the distance image generation unit 7.

In step ST13, the distance image generation unit 7 performs an initialization process on the zoom scale S, the optimum evaluation value E _min , and the zoom scale optimum value S _{opt in} order to optimize the zoom scale when generating the distance image. .
That is, the initialization processing in step ST3 is as follows.
This process is a prerequisite for a process of optimizing a zoom scale S described later to obtain an optimum zoom scale value S _opt . Note that the above-described optimum evaluation value E _min is determined by using the SSAD or the like obtained when a distance image is generated.

Then, in the distance image generation unit 7, S _range
Is a constant that designates a scale range to be subjected to an optimization process described later, and is converted into a zoom scale obtained by subtracting “S _range ” from a zoom scale Sv whose value is adjusted by the optimization process described later. In this step ST13, the zoom scale optimum value S
Let _opt be “S”. Further, the distance image generation unit 7 sets the optimum evaluation value E _min indicating the similarity between the point on the reference image and the point on the detected image corresponding to the point on the reference image to a predetermined value E
Set to.

In step ST14, the distance image generation unit 7 uses the zoom scale S obtained by performing the initialization processing in step ST13, the image center (Xc, Yc) and the lens distortion κ obtained in step ST1 as a reference. By performing image scale conversion processing, the reference image B shown in FIG. 7A is converted into a reference image B ′ shown in FIG. 7B. Note that the detected images D1 to D4 are not subjected to scale conversion.

That is, when performing the scale conversion process in step ST14, the distance image
As shown in (2), the processing shown in steps ST21 to ST23 is performed.

First, in step ST21, the lens distortion κ at the zoom scale Sv and the image center (Xc, Y
Remove the distortion of the reference image using c). As a result, as shown in FIG. 9A, distortion is removed from the reference image as compared with before the distortion is removed. At this time, the distance image generation unit 7
Removes the distortion of the reference image assuming that the distortion of the reference image changes in proportion to the cube of the distance from the center of the image.

In step ST22, the distance image generation section 7 performs scale conversion of the reference image. When performing the scale conversion, the distance image generation unit 7 uses the zoom scale S and the image center (Xc, Yc) of the reference image from which the distortion has been removed in step ST21. At this time, the distance image generation unit 7 performs scale conversion using a conversion formula represented by Xi = Xc + (Xi−Xc) / S Yi = Yc + (Yi−Yc) / S. Here, (Xi, Yi) is the coordinates of the ith pixel in the XY direction on the reference image.

As a result, as shown in FIG. 9B, the pattern of the reference image is reduced as compared with that before the scale conversion, and has the same size as the pattern of the reference image used at the time of calibration.

In step ST23, the distance image generator 7 _sets the image center (X
c, Yc) and lens distortion κ using step ST22.
The distortion is added to the scale-converted reference image. As a result, as shown in FIG. 9C, the reference image is converted into an image equivalent to the reference image captured in the zoom state at the time of calibration.

Returning to FIG. 6, in step ST14, the distance image generation unit 7 performs the distance image generation process and the evaluation value calculation process using the reference image after the scale conversion process. At this time, a distance image is generated by using a relationship between a point on the reference image and a corresponding point on the detected image created as a table by calibrating in advance, and an evaluation value E is calculated using the following equation. .

[0052]

(Equation 1)

Where I (x) is the luminance value of the reference image,
I ′ (x ′) is the luminance value of the detected image.

At this time, step ST21 to step S
By performing the process shown in T23 on the reference image, the effective area of the distance image becomes smaller than the reference image captured by the reference camera 2. Therefore, when calculating the evaluation value E using the reference image subjected to the scale conversion processing, the effective area of the reference image captured by the reference camera 2 is considered. That is, by performing the scale conversion process, the number of pixels forming a predetermined region of the reference image captured by the reference camera 2 and the number of pixels of the reference image corresponding to the predetermined region after the scale conversion is performed Therefore, when the evaluation value E is obtained by the above equation, an average of the evaluation values E in an arbitrary region is used as the evaluation value E in each small region.

Next, in step ST15, the distance image generator 7 performs a zoom scale optimization process. That is, in step ST15a, the distance image generation unit 7
Is the optimal evaluation value E _min determined in step ST13,
By comparing the evaluation value E calculated in step ST14, the flow proceeds to step ST15b when it is determined evaluation value E is smaller than the optimum evaluation value E _min, it is determined that is not smaller evaluation value E than the optimum evaluation value E _min When step ST1
Proceed to 6.

In step ST15b, the evaluation value E selected in step ST15a is replaced with the optimum evaluation value E.
_min as well as the converted optimal evaluation value E _min
Is updated to the optimum value _Sopt .

In step ST16, the distance image generating section 7 executes the above-described steps ST14 and ST15.
The zoom scale S at the time of performing the processing indicated by
+ S _range ”and compare the size with“ S> Sv +
When it is determined that “S _range ”, the process proceeds to step ST17, and when it is determined that “S ≦ Sv + S _range ”, the process proceeds to step ST18. In step ST18, the zoom scale S is incremented to “S + S _step ”.

That is, the distance image generation unit 7 outputs
_{In the range of} ≦ S ≦ + S _range (S _start ≦ S ≦ S _end ), the above-described steps ST14 to ST1 are performed for each S _step.
Step 6 is repeated. That is, the scale conversion of the reference image is performed in step ST14, and the evaluation value E and the optimum evaluation value E obtained from the scale-converted reference image and the detected image are obtained.
_When the evaluation value E is small, the optimum evaluation value E _min is updated to the evaluation value E at any time and the optimum value S
By updating _opt to the zoom scale S, an optimum zoom scale within a predetermined zoom scale range is detected.

The range image generation unit 7 calculates ± S
_{When calculating} the optimum value _Sopt of the zoom scale by changing the zoom scale in S _step units within the _range of the range, for example, as shown in FIG.
By extracting a plurality of points near the minimum of the evaluation value E with respect to _{start to} S _end and interpolating these points, the optimum value S _{opt is obtained.}
May be determined. Furthermore, the distance image generation unit 7 may extract a plurality of points in the vicinity of the minimum of the evaluation value E, and determine an optimum value S _opt by approximating the points with a quadratic function.

In step ST17, the distance image generation unit 7 determines the optimum value S _opt obtained in step ST15.
Is generated. At this time, the distance image generation unit 7 performs the scale conversion processing shown in steps ST21 to ST23 using the optimum value S _opt of the zoom scale, and uses the reference image and the detected image subjected to the scale conversion processing to generate the distance image. Generate When outputting the generated distance image, the distance image generating unit 7
The inverse conversion is performed on the zoom scale, the lens distortion, and the image center, and the conversion is performed so that the zoom scale, the lens distortion, and the image center when the object to be imaged is captured by the reference camera 2.

It should be noted that the optimum value S is obtained by using a method of performing scale conversion not on the reference image but on the detected image.
It is also possible to calculate _opt . By using such a method, it is possible to eliminate the need for inverse transformation such as scale transformation performed when outputting a distance image.

In such a distance image generating apparatus 1, the zoom sensor signal detecting section 4 detects the zoom state of the reference camera 2 to obtain the relationship between the output voltage Vn, the zoom scale, the lens distortion, and the image center. The evaluation value E is obtained by performing scale conversion of the reference image by using the optimum value S of the zoom scale within a predetermined range ± S _{range.
Find opt} .

It is also possible to optimize the zoom scale by a method in which the scale conversion is applied not to the reference image but to the detected image.

Next, an example of processing for generating a focus control signal by the distance image generation unit 7 and generating a focus drive signal by the control unit 8 to adjust the focus of the reference camera 2 in step ST4 shown in FIG. This will be described with reference to the flowchart of FIG.

According to this flowchart, first, in step ST 31, the distance image generator 7 sets the focus sensor signal Sf from the focus sensor signal detector 5
Perform processing to read _zoom .

In step ST32, the distance image generator 7 initializes various parameters used in the following processing using the focus sensor signal Sf _zoom input in step ST31. At this time, the distance image generation unit 7 performs step ST5 to step ST39 described below.
Is set to “Sf _zoom −α”, the minimum value of SSAD SSAD _{min is set} to a constant “1.0 * 10 ¹¹ ”, and the optimum value Sf _opt of the zoom scale after focus adjustment is set to “Sf”.

In step ST33, the distance image generation section 7 executes step S33 in the same manner as in step ST14.
A distance image is generated for the variable Sf obtained by performing the initialization process at T32, and an SSAD is generated.

In step ST34, the distance image generator 7 performs an optimization process regarding focus. That is, in step ST34a, the distance image generation unit 7
The SSAD _min determined in step ST32 and the step ST32
Comparing with SSAD calculated in 33, SSAD is smaller than SSAD _min .
When it is determined that is smaller, the process proceeds to step ST34b,
When it is determined that SSAD is not smaller than SSAD _{min, the} process proceeds to step ST35.

[0069] In step ST34b, updates the SSAD selected in the above step ST34a as SSAD _min, and updates the variable Sf in converted SSAD _min to optimal values Sf _opt.

In step ST35, the distance image generation section 7 performs the above-described steps ST33 and ST34.
The variable Sf at the time of performing the process indicated by “Sf _zoom +
α ”, the zoom scale Sf is set to“ Sf
When it is determined that the variable Sf is not larger than “ _zoom + α”, the process proceeds to step ST36, and when it is determined that the variable Sf is larger than “Sf _zoom + α”, the process proceeds to step ST38.

In step ST36, the distance image generation unit 7 performs a process of advancing the variable Sf by Sf _step .

In step ST37, the distance image generation unit 7 controls the focus of the reference camera 2 by causing the control unit 8 to generate a focus drive signal in accordance with Sf proceeded in step ST35, and then proceeds to step ST33.
Return to

In step ST35, the variable Sf
Is determined to be larger than Sf _zoom + α in step ST3.
In step 8, a focus control signal is generated using the optimum value Sf _opt held when the variable Sf is determined to be larger than Sf _zoom + α, and is output to the control unit 8. Accordingly, the control unit 8, referring to generates a focus drive signal to the focus sensor signal and a focus control signal from the focus sensor signal detecting unit 5, performs processing for focus control by using the optimum values Sf _opt To end the processing.

As described above, the distance image generation unit 7 performs the processing shown in steps ST35 to ST36 until it determines in step ST37 that the variable Sf is within the range of Sf _zoom ± α. That is, the distance image generation unit 7
_When the optimum value Sf _opt is obtained by changing the variable Sf in Sf _step units within the range of _zoom ± α, for example, as shown in FIG. 12, a plurality of points near the minimum of the SSAD for Sf-α to Sf + α are extracted, By interpolating these points, the optimum value Sf _opt is determined. Further, the distance image generation unit 7
A plurality of points near the minimum of D may be extracted, and these points may be approximated by a quadratic function to determine an optimum value Sf _opt .

As described above, the distance image generation device 1
The focus sensor signal is generated by the focus sensor signal detection unit 5, an optimum value Sf _opt when SSAD is minimum is detected while changing the focus, and the focus control signal is generated by the distance image generation unit 7. Since the focus control process is performed by the control unit 8, even if the focus has not been adjusted after performing the zoom operation, a distance image with the minimum SSAD can be generated, and a high-accuracy distance image can be generated. Can be generated.

In the description of the present invention, an example has been described in which a distance image is generated when focus adjustment is performed in accordance with a change in the zoom scale of the reference camera 2 or the detection camera 3. Even if the image size changes due to focus adjustment, a high-accuracy distance image can be generated by optimizing the zoom scale again in step ST2.

At this time, the distance image generation device 1 uses, for example, a reference image B and detection images D1 to D1 as shown in FIG.
In D4, the zoom scale is converted only for the reference image B imaged by operating the zoom, and FIG.
As shown in FIG. 13, a reference image B ′ is obtained, and a focus adjustment process is further performed on the reference image B ′ to obtain a reference image B ″ shown in FIG. 13C. At this time, the reference image B ″ shown in FIG. B ″ is the reference image B ′ shown in FIG.
The size has changed as compared with, and in this case, it is smaller.

On the other hand, when the zoom scale is changed in S _step units within the _range of ± S _range to obtain the optimum value _Sopt of the zoom scale, for example, as shown in FIG.
The optimum value S _opt is determined according to the optimum evaluation value E _min for the actually obtained zoom scales S _{start to} S _end , and the zoom scales S _{start ′ to} S _end for the reference image B ″ after the focus is adjusted. _According to the optimal evaluation value E _{min for '
Find the} optimal S _{opt '} . Accordingly, the distance image generation device 1 generates a distance image of the image by changing the zoom scale, and even when the zoom scale of the image changes by performing focus adjustment, the optimum zoom scale is S _{opt ′.} And a distance image can be generated.

Further, according to the distance image generating device 1,
The process of generating a distance image by optimizing the zoom scale after zooming shown in FIG. 6 is performed, and the process of generating a distance image by obtaining the SSAD at the optimum value Sf _opt when adjusting the focus shown in FIG. 11 is performed. By performing the process of optimizing the zoom scale after the focus adjustment described with reference to FIG. 14 to generate the distance image, a more accurate distance image can be generated.

[0080]

As described above in detail, the range image generating apparatus according to the present invention provides an image captured by a camera having a focus adjustment function and an image which has not been subjected to focus adjustment corresponding to a point on the image. A focus detection unit that generates an evaluation value indicating the degree of similarity by comparing the above points to detect an optimal focus state, an image in the focus state detected by the focus detection unit, and no focus adjustment has been performed A distance image generating means for generating a distance image indicating a distance between any of the cameras and the imaging target using the image, so that when generating a distance image using an image requiring focus adjustment, An evaluation value is generated according to the focus adjustment of the camera, and a distance image with the smallest evaluation value can be generated. Be adjusted dregs, it is possible to produce high quality range image.

Further, the distance image generating method according to the present invention
An evaluation value indicating the degree of similarity by comparing the image when the focus is adjusted by changing the focus state within a predetermined range and a point on the image on which the focus is not adjusted corresponding to a point on the image. , A third step of detecting a focus state at an appropriate evaluation value, and an image that has been adjusted in focus in an appropriate focus state and an image that has not been adjusted in focus. Since a fourth step of generating a distance image indicating a distance between the camera having the focus adjustment function and the imaging target object is included, when a distance image is generated using an image requiring focus adjustment, It is possible to generate a distance image with the smallest evaluation value, even if the focus that changes according to the zoom state of the camera is adjusted. It is possible to produce high quality range image.

[Brief description of the drawings]

FIG. 1 is a perspective view showing an example of the arrangement of a reference camera and a detection camera constituting a range image generating apparatus to which the present invention is applied.

FIG. 2 is a block diagram illustrating a configuration of a range image generation device to which the present invention has been applied.

FIG. 3 is a diagram illustrating an image captured when a zoom state of a reference camera and a detection camera is fixed.

FIG. 4 is a diagram illustrating a relationship between an output voltage generated by a zoom sensor signal detection unit and a zoom scale of a reference image.

FIG. 5 is a flowchart illustrating a process of performing focus adjustment and generating a distance image by the distance image generation device to which the present invention is applied.

FIG. 6 is a flowchart illustrating a process of a distance image generation unit when a distance image is generated by the distance image generation device to which the present invention is applied.

7A and 7B are diagrams for explaining that a scale conversion process is performed on a reference image. FIG. 7A illustrates a reference image and a detected image before the scale conversion process, and FIG. 7B illustrates a reference image after the scale conversion process. And a detected image.

FIG. 8 is a flowchart illustrating a process of performing a scale conversion process on a reference image.

9 is a diagram showing a reference image on which a scale conversion process is performed according to the flowchart shown in FIG. 8; FIG.
Shows reference images before and after distortion removal, (b) shows reference images before and after scale conversion, and (c) shows reference images before and after distortion addition.

FIG. 10 is a diagram showing evaluation values when the zoom scale is changed within a predetermined range ± S _range .

FIG. 11 is a flowchart illustrating a process of generating a focus control signal by a distance image generation unit, generating a focus drive signal by the control unit, and adjusting the focus of the reference camera.

FIG. 12 is a diagram illustrating a relationship between a focus state and SSAD.

13A illustrates a reference image and a detected image before scale conversion processing, FIG. 13B illustrates a reference image and a detected image after scale conversion processing, and FIG. 13C illustrates a reference image and a detected image after focus adjustment. It is a figure showing an image.

FIG. 14 is a diagram showing an evaluation value when the zoom scale is changed within ± S _range within a predetermined range, and an evaluation value according to the zoom scale changed by performing focus adjustment.

[Explanation of symbols]

Reference Signs List 1 distance image generation device, 2 reference camera, 3 detection camera, 5 focus sensor signal detection unit, 7 distance image generation unit

Claims (8)

[Claims] 1. A camera having a focus adjusting function.
An image pickup unit having a plurality of cameras having the above-mentioned features, and comparing an image picked up by the camera having the focus adjustment function with a point on an image on which focus adjustment corresponding to a point on the image has not been performed. A focus detection unit that generates an evaluation value indicating the focus state and detects an appropriate focus state, an image in the focus state detected by the focus detection unit, and an image on which focus adjustment has not been performed. A distance image generation device comprising: a distance image generation unit that generates a distance image indicating a distance between a camera and an imaging target. 2. The image pickup means includes a camera having a zoom adjustment function, and the focus detection means evaluates using an image on which focus adjustment is performed after zoom adjustment is performed by the camera having the zoom adjustment function. 2. The distance image generating apparatus according to claim 1, wherein a value is generated to detect an appropriate focus state. 3. A zoom state detecting means for detecting a zoom state of a camera having the zoom adjustment function and calculating at least a zoom scale; a scale converting means for converting a zoom scale of an image taken by any of the cameras; The scale conversion unit compares the image on which the zoom scale has been converted and a point on the image on which the scale conversion has not been performed corresponding to a point on the image to generate an evaluation value indicating the degree of similarity. A zoom scale detecting means for detecting a proper zoom scale, wherein the distance image generating means uses an image on the zoom scale detected by the zoom scale detecting means and an image on which no scale conversion has been performed. 3. The distance image generation device according to claim 2, wherein a distance image indicating a distance between one of the cameras and the imaging target is generated. . 4. The zoom state detecting means detects a zoom scale of a camera having the focus adjusting function after focus adjustment.
The distance image generation device according to claim 1. 5. A first method for detecting a focus state of a camera having the focus adjustment function among a plurality of cameras including one or more cameras that generate an image by capturing an image of an object using the focus adjustment function. Step: changing the focus state within a predetermined range, comparing the image at the time of the focus adjustment with a point on the image that has not been subjected to the focus adjustment corresponding to a point on the image, and calculating the similarity. A second step of generating an evaluation value indicating the following: a third step of detecting a focus state at an appropriate evaluation value; an image having focus adjusted in the appropriate focus state; and an image not having focus adjusted. And a fourth step of generating a distance image indicating a distance between the camera having the focus adjustment function and the imaging target object using To generate a distance image. 6. The method according to claim 5, wherein in the first step, an evaluation value is generated using an image on which focus adjustment has been performed after zoom adjustment, and a focus state at an appropriate evaluation value is detected. Distance image generation method. 7. A step of detecting a zoom state of a camera having a zoom adjustment function and calculating at least a zoom scale; a step of converting a zoom scale of an image captured by any one of the cameras; and a step of converting the zoom scale. Detecting an appropriate zoom scale by generating an evaluation value indicating the degree of similarity by comparing the applied image and a point on the image not subjected to scale conversion corresponding to the point on the image, 6. A distance image indicating a distance between any one of the cameras and the object to be imaged, using the image on the appropriate zoom scale and the image on which the zoom scale has not been converted. Distance image generation method. 8. The distance image generation method according to claim 7, wherein in the step of calculating the zoom scale, the zoom scale of the camera that has performed the focus adjustment is detected.