JP4872035B2 - Imaging apparatus, captured image distance measuring method, captured image distance measuring program, and recording medium - Google Patents

Description

  The present invention relates to an imaging apparatus having a function of measuring a distance to a subject, and more specifically, to accurately measure the distance to a subject by accurately determining whether or not the posture of the subject is inclined. It relates to the technology that has been made possible.

Currently, various methods for distance measurement and shape restoration have been proposed, but many require large-scale devices and special environments. For example, in the stereo method using the principle of triangulation described in Non-Patent Documents 1 and 2, a method using a plurality of cameras and a method using a laser projector have been proposed. Non-Patent Document 2 discloses an illuminance difference stereo method for obtaining a shape from a plurality of images taken under different illumination conditions. However, it is necessary to control the light source and to illuminate with an infinite light source. There is a problem that it is difficult to realize the light source in an actual environment. The shape-from-shading method that restores the shape from the shadows observed in a single image of Non-Patent Document 2 requires only a single light source, but assumes an infinity light source as in the illuminance difference stereo method. Control is necessary. However, it is difficult to realize these conditions in a real environment. Non-Patent Document 3 discloses Cyber Modeler as a method for restoring a three-dimensional shape using an inexpensive apparatus. This is done by placing an object on a turntable on which a reference mark is printed, taking multiple images with a commercially available digital camera while rotating the turntable, calculating the position of the camera from the reference mark, and A three-dimensional shape is calculated from a silhouette by acquiring an image. Although this device is inexpensive, there is a problem that there is a restriction on the environment, for example, a rotating table is necessary and background paper must be installed to extract the silhouette.
Non-Patent Document 4 uses a fact that near-infrared light is emitted from LEDs arranged around the camera and the intensity of the reflected light is almost inversely proportional to the distance to the object. A technique for obtaining a three-dimensional shape is disclosed.
Further, in Patent Document 1, an imaging instruction is input in an image input apparatus that includes an imaging unit that captures an image to be captured with a set exposure time, and a light emitting unit that irradiates the imaging target with light. Then, the exposure time is set to the first time, the light emitting means irradiates the imaging target with light, the imaging means takes the first image of the imaging target, the exposure time is set to the first time, The light emitting unit does not irradiate the imaging target with light, the imaging unit captures a second image of the imaging target, obtains the difference between the first image and the second image, and the light emitting unit irradiates the imaging target. A technique for obtaining a third image that is an image representing the intensity distribution of light reflected by the imaging target of light and that includes depth information of the imaging target is disclosed.
Further, Patent Document 2 uses an image acquisition device that can acquire both a reflected light image and a natural light image, extracts and refers to reflection characteristic information from the natural image acquired by the image acquisition device, and refers to the reflected light image. A technique for acquiring accurate distance information by correction is disclosed.
JP 2004-328657 A Japanese Patent Laid-Open No. 2003-185412 Seiji Iguchi, Kosuke Sato, “Three-dimensional image measurement”, Shoshodo, 1990 BKP Horn, NTT Human Interface Laboratory Project RVT translation, “Robot Vision”, Asakura Shoten, 1993 Kota Fujimura, Tomohiro Ohkami, Tomoya Terauchi, Tetsuichi Emi, Tsutomu Fukuzaki, “Software Technology for Easily Generating 3D Data from Digital Camera Images”, SANYO TECHNICAL REVIEW, Vol. 35, No.1, Jun. 2003 Shunichi Numasaki, Miwako Doi, “An interface that conveys feelings by motion-motion processor”, IPSJ Magazine, Vol.41, No.2, Feb. 2000

However, the prior art disclosed in Patent Document 1 is based on the difference between the first image obtained by irradiating the subject with light and the second image obtained without irradiating the subject with light. 3 is obtained, and the distance is measured based on the third image. However, there is a problem that the distance cannot be measured accurately because the reflection characteristics of the subject are not uniform.
The prior art disclosed in Patent Document 2 is to measure the distance by correcting the reflection characteristics of the subject in order to solve the problem of Patent Document 1, but in either case, the subject is attached to the imaging device. However, it is difficult to distinguish between the illuminance change due to distance and the illuminance change due to inclination when tilting in the direction of approaching or separating, and the distance between the imaging device and the subject cannot be measured accurately. There is a problem.
In view of such problems, the present invention corrects the distance to the subject more accurately by correcting the distance according to the inclination angle even when the subject is inclined toward or away from the imaging device. It is another object of the present invention to provide an imaging apparatus and an algorithm capable of easily measuring.

In order to solve this problem, the present invention provides an imaging optical system that guides imaging light from a subject along a predetermined optical path, and receives imaging light from the imaging optical system and converts it into an electrical signal. Imaging device, signal processing means for performing a required process on the electrical signal output from the imaging element, image storage means for storing the image signal processed by the signal processing means, and light for irradiating the subject Flashing means for controlling and each of the means for controlling each of the means, and the control means flashes a first subject image obtained by natural light and a subject in natural light by the flashing means. an imaging apparatus for measuring the distance to the object from the imaging device based on the third object image obtained from the difference between the subject image, the control means, the third object image on the imaging element Nitsu A vector from the center of the lens of the imaging optical system to the target pixel on the image sensor, a vector from the center of the lens to an adjacent pixel adjacent to the target pixel on the image sensor, and the target pixel. Between a first point on the subject corresponding to the pixel of interest and a second point on the subject corresponding to the adjacent pixel, obtained based on a distance between the center of the lens and A tilt amount of the subject is calculated based on the vector, and a distance to the subject is corrected based on the tilt amount .
The present invention is an image pickup apparatus including an image pickup optical system, an image pickup device, a signal processing unit, an image storage unit, a flash unit, and a control unit, and particularly flashes a first subject image obtained by natural light and a subject in natural light. When measuring the distance from the imaging device to the subject by acquiring the third subject image from the second subject image obtained when flashing by means, the amount of inclination of the subject is calculated to correct the distance error It is.
According to a second aspect of the present invention, the control means obtains a third subject image by obtaining a difference between the first subject image obtained from the imaging optical system and the second subject image, and the flashlight. A distance calculation unit that calculates a distance to the subject based on an image obtained by the image acquisition unit, and an image obtained by the flash image acquisition unit from the center of the lens of the imaging optical system to the image sensor Based on the vector of the pixel of interest, the vector from the center of the lens to the adjacent pixel adjacent to the pixel of interest on the image sensor, and the distance between the pixel of interest and the center of the lens. calculates the inclination of the object based on a vector between the second point on the subject corresponding to the adjacent pixels first point and on the subject corresponding to the pixel of interest Image image inclination calculation means, and distance correction means for correcting the distance to the subject obtained by the distance calculation means based on the amount of inclination of the subject calculated by the captured image inclination calculation means. It is characterized by.
The control means of the present invention obtains a third subject image by obtaining a difference between the first subject image and the second subject image, calculates a distance to the subject based on the image, and controls the inclination of the subject. An accurate distance is derived by calculating and correcting the distance to the subject based on the amount of inclination.

According to a third aspect of the present invention, the distance from the lens along the optical axis of the small area on the subject is proportional to the square root of the radiant intensity of the flash means and the reflectance of the target object, and inversely proportional to the square root of the difference in illuminance. It is characterized by.
The distance from the lens along the optical axis of the small area on the subject is proportional to the square root of the radiant intensity of the flashing means and the reflectance of the target object, and inversely proportional to the square root of the difference in illuminance.
According to a fourth aspect of the present invention, the distance from the imaging optical system to the imaging element is f, the lens diameter of the imaging optical system is d, and a light beam traveling from a small area on the subject surface toward the center of the lens is an optical axis of the lens. Α, the angle between the ray and the normal of the subject surface, θ, the radiant intensity of the flash means, I, the irradiance on the image sensor when the flash means is used, E ^F _image , Assuming that the irradiance on the image sensor when the flash means is not used is E ^NF _image , the irradiance of the small area on the image sensor E _image = E ^F _image −E ^NF _image , and the flash means Is assumed to be the same position as the center of the lens , and the object surface is assumed to be a diffuse reflection surface having a reflectance r. Distance z from
It is calculated | required by.
The reflected light becomes weaker in proportion to the cosine of the angle formed by the object surface normal and the line of sight. Therefore, if the inclination of the surface is not taken into consideration, the calculation result will be a result farther than the actual distance. Therefore, in the algorithm of the present invention, the angle α between the light beam from the small area on the object surface toward the center of the lens and the optical axis of the lens and the angle θ between the light beam and the normal of the object surface are used to derive the distance. It is a point introduced in. Thereby, since the change of the illumination intensity by the inclination of a to-be-photographed object can be correct | amended, an exact distance can be measured.

According to a fifth aspect of the present invention, in the method for measuring the distance to the subject, a difference between the first subject image obtained by natural light and the second subject image obtained when the subject in natural light is flashed by the flash means is obtained. The flash image acquisition procedure for obtaining the third subject image, the distance calculation procedure for calculating the distance to the subject based on the image obtained by the flash image acquisition procedure, and the image obtained by the flash image acquisition procedure A vector from the center of the lens of the imaging optical system to a pixel of interest on the imaging device, a vector from the center of the lens to an adjacent pixel adjacent to the pixel of interest on the imaging device, and the pixel of interest The first point on the subject corresponding to the pixel of interest and the pixel corresponding to the adjacent pixel obtained based on the distance between the center of the lens and A captured image inclination calculation procedure for calculating the inclination of the object based on a vector between the second point on Utsushitai, said distance based on the inclination amount of the object calculated by the captured image gradient algorithm A distance correction procedure for correcting the distance to the subject obtained by the calculation procedure.
According to a sixth aspect of the present invention, there is provided a control means for measuring the distance to the subject, the first subject image obtained by natural light and the second subject image obtained when the subject in natural light is flashed by the flash means. The flash image acquisition means for obtaining the third subject image by obtaining the difference, the distance calculation means for calculating the distance to the subject based on the image obtained by the flash image acquisition means, and the flash image acquisition means Regarding the image , a vector from the center of the lens of the imaging optical system to the target pixel on the image sensor, a vector from the center of the lens to an adjacent pixel adjacent to the target pixel on the image sensor, and the target pixel And the distance between the lens and the center of the lens, the first point on the subject corresponding to the pixel of interest and the object corresponding to the adjacent pixel obtained based on Captured image inclination calculation means for calculating a tilt of the object based on a vector between the second points on the body, said distance calculating means based on the inclination amount of the object calculated by the captured image inclination calculation means It is made to function as distance correction means for correcting the distance to the subject obtained by the above.
The control unit is composed of a CPU. The CPU is operated by a program stored in the ROM. Therefore, the captured image distance measurement program of the present invention is configured to execute a series of functions of a flash image acquisition unit, a distance calculation unit, a captured image tilt calculation unit, and a distance correction unit.
A seventh aspect is characterized in that the captured image distance measuring program according to the sixth aspect is recorded in a computer-readable format.

According to the invention of claim 1, the third subject image is obtained from the first subject image obtained by natural light and the second subject image obtained when the subject in natural light is flashed by the flash means, Since the error in the distance due to the inclination of the subject is corrected, the distance to the subject can be accurately measured by a simple method.
Further, in claims 2 and 5, the control means obtains a third subject image by obtaining a difference between the first subject image and the second subject image, and calculates a distance to the subject based on the image. Since the inclination of the subject is calculated and the distance to the subject is corrected based on the amount of inclination, the correct distance can be calculated by correcting the brightness even if the subject is inclined.
According to the third and fourth aspects of the present invention, in the algorithm of the present invention, an angle α formed by a light beam traveling from a small region on the object surface toward the center of the lens with the optical axis of the lens, and an angle θ formed between the light beam and the normal line of the object surface. Is introduced in the equation for deriving the distance, so that the result of correcting the distance according to the tilt angle of the subject can be calculated.
Further, in claim 6, the captured image distance measurement program of the present invention is configured to execute a series of functions of the flash image acquisition means, the distance calculation means, the captured image inclination calculation means, and the distance correction means. In addition to the basic functions of the imaging device, the program can accurately calculate the distance to the subject even if the subject is tilted.
According to a seventh aspect of the present invention, by recording the captured image distance measurement program on a recording medium in a computer-readable format, the program can be operated anywhere by carrying the recording medium.

Hereinafter, the present invention will be described in detail with reference to embodiments shown in the drawings. However, the components, types, combinations, shapes, relative arrangements, and the like described in this embodiment are merely illustrative examples and not intended to limit the scope of the present invention only unless otherwise specified. .
FIG. 1 is a schematic diagram for explaining a method for obtaining distance information according to the present invention. In order to obtain distance information, a dedicated device and a special environment are required, which is difficult to carry out easily. Therefore, in the present invention, as shown in FIG. 1, an image 4 a captured by irradiating flash light 1 a using a flash 1 in natural light 3 and an image 4 b captured using only natural light 3 without using the flash 1. By taking the difference, it is possible to obtain an image 4c using only the flash light 1a from which the natural light 3 is removed. The reflected light from the flash light 1a becomes weaker in proportion to the square of the distance. By utilizing this property, the distance can be obtained from the difference image 4c. Further, the intensity of the reflected light varies depending on the inclination of the object surface, but the present invention also proposes this correction method. According to the present invention, anyone can easily obtain distance information using a general digital camera.
FIG. 2 is a block diagram showing an internal configuration of the digital camera 2 of the present invention. The same components will be described with the same reference numerals. The digital camera 2 includes a plurality of lenses, a diaphragm, and the like, and includes an optical system (imaging optical system) 10 that guides imaging light from a subject to a CCD (imaging device) 11 along a predetermined optical path, and an optical system 10. A CCD (imaging device) 11 that receives the imaged light and converts it into an electrical signal, an A / D converter 12 that converts the analog signal converted by the CCD 11 into a digital signal, and an A / D converter 12 A signal processing circuit (signal processing means) 13 for performing required processing on the digital signal, a frame memory (image storage means) 14 for storing an image signal processed by the signal processing circuit 13, and a light for irradiating the subject. And a control section (control means) 15 for controlling all circuits. In the block diagram of FIG. 2, only the main parts for explaining the present invention are shown, and the other parts are omitted.

Next, the schematic operation of the digital camera 2 of the present invention will be described. An optical image reflected from a subject (not shown) is focused by the optical system 10 and projected onto the surface of the CCD 11 as light intensity. The CCD 11 converts the intensity of the light into an intensity of an electric signal as a two-dimensional image and transmits it to the A / D converter 12. The A / D converter 12 converts the signal into a predetermined bit (generally 8 bits) according to the signal level and transmits it to the signal processing circuit 13. The signal processing circuit 13 performs signal processing such as white balance on the basis of the digital signal and then stores it in the frame memory 14. The flash 1 emits light to illuminate the subject when the subject is underexposed. All these controls are controlled by the control unit 15.
FIG. 3 is a functional block diagram of the control unit of FIG. This control unit 15 uses the flash 1 in the natural light 3 of FIG. 1 obtained from the optical system 10 and irradiates the flash light 1 a and captures the image 4 a (first subject image) without using the flash 1. A flash image acquisition unit (flash image acquisition unit) 22 that obtains an image 4c (third subject image) by obtaining a difference between images 4b (second subject image) captured only with natural light 3 and a flash image acquisition unit 22 A captured image distance calculation unit (distance calculation means) 23 that calculates the distance to the subject based on the obtained image and a direction in which the camera approaches or separates from the camera based on the image obtained by the flash image acquisition unit 22 A captured image tilt calculation unit (captured image tilt calculation means) 24 that calculates the tilt of the captured image and a captured image distance calculation based on the tilt amount of the subject calculated by the captured image tilt calculation unit 24. Distance correcting operation unit for correcting the distance to the subject obtained by the parts 23 and (distance correction means) 25, and includes a.
In the present invention, the digital camera 2 is used to capture the same subject in two ways, when the flash light 1a is used and when it is not used, and distance measurement is performed using the difference in brightness between the two images. That is, the pixel value of the image captured by the digital camera 2 is proportional to the irradiance on the CCD 11. Since the irradiance is linear, the difference between the irradiance obtained from the image captured by irradiating the flash light 1a and the irradiance obtained from the image captured without irradiating the flash light 1a in the same environment is taken. Thus, the irradiance value of only the reflected light from the flash light 1a can be obtained. The distance information can be acquired by utilizing the fact that the intensity of the reflected light becomes weak in proportion to the square of the distance to the object. Since the present invention does not need to know the light source environment other than the flash, there is no need to prepare a special environment such as a dark room, and there is an advantage that distance measurement can be easily performed with an inexpensive apparatus.

  FIG. 4 is a flowchart for explaining one method for obtaining distance information according to the present invention. This will be described with reference to FIG. First, in the natural light 3 of FIG. 1 obtained from the optical system 10, an image 4 b (this image is referred to as an image A) captured only with the natural light 3 without using the flash 1 is stored in the memory (S 1). Next, an image 4a (this image is referred to as an image B) taken by irradiating the flash light 1a using the flash 1 in the natural light 3 of FIG. 1 obtained from the optical system 10 is stored in the memory (S2). (The photographing order of the images A and B may be reversed.) Then, the flash image acquisition unit 22 takes the difference between the images B and A and stores the image in the memory as the image C (S3). Next, based on the image C stored in the flash image acquisition unit 22, the captured image distance calculation unit 23 calculates the distance to the subject (S4). Further, based on the image C stored in the flash image acquisition unit 22, the captured image tilt calculation unit 24 calculates the tilt of the subject (S5). Then, the distance correction calculation unit 25 corrects the distance to the subject calculated by the captured image distance calculation unit 23 from the distance to the subject and the inclination of the subject (S6), and the corrected distance data is displayed on the display unit of the digital camera 2. (S7).

FIG. 5 is a diagram for explaining the camera coordinate system of the present invention. The derivation of the distance calculation formula will be described with reference to FIG. First, the following conditions are assumed in the present invention.
(A) The reflection on the surface of the object 32 is only diffuse reflection, and not specular reflection.
(B) The positions of the centers of the flash 1 and the lens 31 are equal.
Consider the relationship between the radiance at one point on the object 32 and the irradiance at one point on the CCD image 30 corresponding to that point. Consider a lens 31 with a diameter d that is a distance f away from the CCD image plane 30. The area of the small region on the surface of the object 32 is δO, and the area of the corresponding small region on the CCD image 30 is δI. It is assumed that a light beam 35 from the small region δO on the object toward the center of the lens 31 forms an angle α with the optical axis 33, and an angle formed by the light beam 35 and the surface normal 34 is θ. If the small area δO on the object is at a distance from the lens 31 along the optical axis 33, the irradiance of the corresponding small image area is given by the following equation.
(1)
Here, L _object is the radiance of the small object region. When the bidirectional reflectance distribution function ρ (θ _i , φ _i ; θ _e , φ _e ) is introduced, the equation (1) is

(2)
It becomes. Here, E _object is the irradiance on the object. Here, the bidirectional reflectance distribution function represents the reflection characteristics of the surface with respect to a certain gaze direction (θ _i , φ _e ) and light irradiation direction (θ _i , φ _i ), that is, radiance with respect to irradiance. Is a function representing the ratio of.

Next, consider the case where the light source is only a flash. When the radiant intensity of the flash is I, the irradiance due to the flash light on the object is
(3)
It becomes. Here, θ _f is an angle formed by the light ray 35 directed from the flash toward the small object region δO and the surface ray 34. Also, δω is the solid angle when looking at the minute area on the object from the flash,
(4)
It can be expressed as Considering equations (3) and (4), equation (2) is
(5)
It becomes.

If the irradiance on the image when the flash is used is E ^F _image , and the irradiance on the image when the flash is not used is E ^NF _image , E _image = E ^F _image −E ^NF _image .
Here, since it is assumed that the position of the flash is the same as the center of the lens, α _f = α and θ _f = θ can be set. Assuming that the object surface is a diffuse reflection surface with a reflectance r, the bidirectional reflectance distribution function is
(6)
Therefore, from equation (5),
(7)
Get.
From equation (7), it can be seen that the distance is proportional to the square root of the flash radiation intensity and the reflectance of the target object, and inversely proportional to the square root of the illuminance difference. As described above, since the irradiance of the CCD surface 30 is proportional to the luminance value of the image, the distance can be obtained from the difference between the two images.
However, since θ is an unknown number, the distance cannot be obtained at this stage, so the surface normal is estimated. In equation (7), the angle θ formed by the line-of-sight direction and the normal of the object surface is an unknown quantity, and therefore it is necessary to estimate this.

FIG. 6 is an explanatory diagram for estimating θ. Points on the object appearing on the i th pixel and i + 1 pixel on the CCD 30 are P _i and P _{i + 1} , respectively, and vectors from the lens center to the i th pixel and i + 1 pixel on the CCD 30 are α _i , Let α _{i + 1 be} the irradiance of only the flash obtained from the difference between the images, and let it be E _i and E _{i + 1} , respectively. Further, it is assumed that the surface of the object is locally flat and the normal vector is n. Vector here
Is
(8)
It can be expressed. Since Pi _{Pi + 1} and n are vertical,
(9)
It is. Than this,
(10)
Get. Also,
(11)
Because it can be expressed as
(12)

Get. Substituting equation (12) into equation (10),
(13)
It becomes. From equation (7)
(14)
Therefore, when Equation (13) and Equation (14) are combined,
(15)
It becomes. If the right side is a,
(16)
And
Use a
(17)
It can be expressed. As long as the viewing angle of the camera does not exceed 180 [deg], z _i is negative and is usually less than 180 [deg]. Therefore, the direction of the vector parallel to the object surface can be determined only from known information according to Equation (17). Will be obtained.
Using this method, it is possible to estimate θ by obtaining vectors parallel to the surface in the four neighborhoods of the pixel of interest, and obtaining a three-dimensional surface normal from their outer product.

Example 1
A white screen (SANYO KA-LCV-101KZ) shown in FIG. 7 as an object was imaged from the front at a position about 1.55 [m] away. The white screen 35 is close to diffuse reflection with uniform reflection characteristics, and is a suitable target for the assumption (b) described in FIG. The results are shown in FIG. FIG. 8A shows the result seen from the xz axis, and FIG. 8B shows the result seen from the xyz axis. From this result, it can be seen that although there is variation, it is distributed almost on a plane. It is considered that the main cause of this variation is that noise has not been completely removed. There is a slightly protruding part near the center, which is considered to be due to the influence of the specular reflection component.
(Example 2)
Next, the same object was imaged from a position inclined 30 [deg] from the front. The distance from the center is the same as in the first embodiment. The results are shown in FIG. FIG. 9A shows the result viewed from the xz axis, and FIG. 9B shows the result viewed from the xyz axis. The reflected light becomes weaker in proportion to the cosine of the angle formed by the object surface normal and the line of sight. Therefore, if the inclination of the surface is not taken into consideration, the calculation result is a result that is farther than the actual distance. Since the present invention includes a portion considering the inclination described with reference to FIG. 5, this experiment was performed to verify the effectiveness of this portion. The results are shown in FIG. Although the variation is slightly larger than the result of Example 1, it can be seen that the distance between the centers is substantially the same and distributed at a position close to a plane inclined by 30 [deg]. The reason why the variation is larger than that of the first embodiment is that the surface normal is estimated based on the illuminance ratio with the adjacent pixels, and therefore the influence of noise becomes strong when the surface inclination is large even with the same noise intensity. It is possible to end up.

The present invention is not limited only to the above-described embodiments. Each function constituting the image pickup apparatus of the above-described embodiment is programmed, written in advance on a recording medium such as a CD-ROM, and the CD-ROM or the like is stored in a medium driving apparatus such as a CD-ROM drive mounted on a computer. Needless to say, the object of the present invention can be achieved by mounting these programs, storing these programs in a memory or storage device of a computer, and executing them.
In this case, the program itself read from the recording medium realizes the functions of the above-described embodiment, and the program and the recording medium recording the program also constitute the present invention.
As a recording medium for storing the program, a semiconductor medium (for example, ROM, nonvolatile memory card, etc.), an optical medium (for example, DVD, MO, MD, CD, etc.), a magnetic medium (for example, magnetic tape, flexible disk, etc.) ) Or the like.
Further, not only the functions of the above-described embodiment are realized by executing the loaded program, but also the above-described implementation by cooperating with the operating system or other application programs based on the instructions of the program. The case where the function of the form is realized is also included.
In the case of distribution to the market, the program is stored and distributed on a portable recording medium, or the program is stored in a storage device of a server computer connected via the Internet or the like. It can also be transferred to a computer. In this case, the storage device of this server computer is also included in the recording medium of the present invention.
In the computer, the functions of the above-described embodiments are realized by installing a program on a portable recording medium or a transferred program on a recording medium connected to the computer and executing the installed program. Is done.

As described above, according to the present invention, the third subject image 4c is obtained from the first subject image 4b obtained by natural light and the second subject image 4a obtained when the subject in natural light is flashed by the flash 1. Since the error in the distance due to the inclination of the subject is corrected, the distance to the subject can be accurately measured by a simple method.
Further, the control unit 15 obtains a third subject image 4c by obtaining a difference between the first subject image 4b and the second subject image 4a, calculates a distance to the subject based on the image, and calculates the subject. Since the distance to the subject is corrected based on the amount of inclination, the correct distance can be calculated by correcting the brightness even if the subject is tilted.
Further, according to the algorithm of the present invention, the angle α between the light beam 35 from the small region σO on the object surface toward the center of the lens 31 and the optical axis 33 of the lens 31 and the angle between the light beam 35 and the normal line 34 of the object surface. Since θ is introduced in the equation for deriving the distance, the result of correcting the distance according to the inclination angle of the subject can be calculated.
The captured image distance measurement program of the present invention is configured to execute a series of functions of the flash image acquisition unit 22, the captured image distance calculation unit 23, the captured image tilt calculation unit 24, and the distance correction calculation unit 25. Therefore, in addition to the basic function as a digital camera, it is possible to accurately calculate the distance to the subject even if the subject is inclined.
Further, by recording the captured image distance measurement program on a recording medium in a computer-readable format, the program can be operated anywhere by carrying the recording medium.

It is a schematic diagram explaining one method for obtaining distance information of the present invention. It is a block diagram which shows the structure inside the digital camera 2 of this invention. It is a functional block diagram of a control part of the present invention. It is a flowchart explaining one method of obtaining distance information of the present invention. It is a figure explaining the camera coordinate system of this invention. It is explanatory drawing in the case of estimating (theta) of this invention. It is the figure of the white screen used for the Example. It is a figure which shows the result of Example 1. It is a figure which shows the result of Example 2.

Explanation of symbols

  1 flash, 1a flash light, 3 natural light, 4a image (first subject image), 4b image (second subject image), 4c image (third subject image), 10 optical system, 15 control unit, 22 flash Image acquisition unit (flash image acquisition unit), 23 captured image distance calculation unit (distance calculation unit), 24 captured image tilt calculation unit (captured image tilt calculation unit), 25 distance correction calculation unit (distance correction unit)

Claims (7)

An imaging optical system that guides imaging light from a subject along a predetermined optical path, an imaging element that receives imaging light from the imaging optical system and converts it into an electrical signal, and an electrical signal output from the imaging element Signal processing means for performing the above processing, image storage means for storing the image signal processed by the signal processing means, flash means for emitting light for irradiating the subject, and control means for controlling the means. Prepared,
Imaging based on the third subject image obtained from the difference between the first subject image obtained by the natural light by the control means and the second subject image obtained by flashing the subject in the natural light by the flash means. In an imaging device that measures the distance from the device to the subject,
Wherein,
About the third subject image on the image sensor
A vector from the center of the lens of the imaging optical system to the target pixel on the image sensor, a vector from the center of the lens to an adjacent pixel adjacent to the target pixel on the image sensor, the target pixel and the lens The distance between the center of
The inclination amount of the subject is calculated based on a vector between the first point on the subject corresponding to the target pixel and the second point on the subject corresponding to the adjacent pixel, obtained based on An image pickup apparatus that corrects a distance to the subject based on the amount of inclination . The control means is a flash image acquisition means for obtaining a third subject image by obtaining a difference between the first subject image and the second subject image obtained from the imaging optical system;
Distance computing means for computing the distance to the subject based on the image obtained by the flash image obtaining means;
About the image obtained by the flash image acquisition means,
A vector from the center of the lens of the imaging optical system to the target pixel on the image sensor, a vector from the center of the lens to an adjacent pixel adjacent to the target pixel on the image sensor, the target pixel and the lens The distance between the center of
The inclination of the subject is calculated based on a vector between the first point on the subject corresponding to the target pixel and the second point on the subject corresponding to the adjacent pixel obtained based on A captured image tilt calculation means;
2. A distance correction unit that corrects a distance to the subject obtained by the distance calculation unit based on an inclination amount of the subject calculated by the captured image tilt calculation unit. The imaging device described in 1.   The distance from the lens along the optical axis of the small area on the subject is proportional to the square root of the radiant intensity of the flash means and the reflectance of the target object, and is inversely proportional to the square root of the difference in illuminance. Item 3. The imaging device according to Item 1 or 2. The distance from the image pickup optical system to the image pickup element is f, the lens diameter of the image pickup optical system is d, and the angle between the light beam from a small area on the subject surface toward the center of the lens and the optical axis of the lens is α , The angle between the ray and the normal of the subject surface is θ, the radiant intensity of the flash means is I, the irradiance on the image sensor when the flash means is used, E ^F _image , and the flash means is used when the irradiance on the imaging device when not the ^E _{NF image,} an irradiance _{^{E image = E F image -E NF}} image of the small area on the image sensor, the lens position of said flash unit assumed to be the same position as the center of, and the when the object surface is assumed to be diffuse reflection surface of the reflection factor r, distance from the lens along the optical axis of the small region on the object Z
The imaging device according to claim 1, wherein the imaging device is obtained by: In the method of measuring the distance to the subject,
A flash image acquisition procedure for obtaining a third subject image by obtaining a difference between a first subject image obtained by natural light and a second subject image obtained by flashing a subject in natural light by the flash means;
A distance calculation procedure for calculating the distance to the subject based on the image obtained by the flash image acquisition procedure;
For images obtained by the flash image acquisition procedure,
A vector from the center of the lens of the imaging optical system to the target pixel on the image sensor, a vector from the center of the lens to an adjacent pixel adjacent to the target pixel on the image sensor, the target pixel and the lens The distance between the center of
The inclination of the subject is calculated based on a vector between the first point on the subject corresponding to the target pixel and the second point on the subject corresponding to the adjacent pixel obtained based on A captured image tilt calculation procedure, and a distance correction procedure for correcting the distance to the subject obtained by the distance calculation procedure based on the tilt amount of the subject calculated by the captured image tilt calculation procedure. The captured image distance measuring method characterized by this. Control means to measure the distance to the subject,
Flash image acquisition means for obtaining a third subject image by obtaining a difference between a first subject image obtained by natural light and a second subject image obtained by flashing a subject in natural light by the flash means, the flash Distance calculation means for calculating the distance to the subject based on the image obtained by the image acquisition means, the image obtained by the flash image acquisition means,
A vector from the center of the lens of the imaging optical system to the target pixel on the image sensor, a vector from the center of the lens to an adjacent pixel adjacent to the target pixel on the image sensor, the target pixel and the lens And a vector between a first point on the subject corresponding to the pixel of interest and a second point on the subject corresponding to the adjacent pixel, obtained based on a distance between A captured image tilt calculating means for calculating the tilt of the subject based on the distance, and a distance for correcting the distance to the subject obtained by the distance calculating means based on the tilt amount of the subject calculated by the captured image tilt calculating means A captured image distance measuring program which functions as a correction unit.   A recording medium in which the captured image distance measurement program according to claim 6 is recorded in a computer-readable format.