JP2009047497A - Stereoscopic imaging device, control method of stereoscopic imaging device, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To restrict irradiation of ranging light to a subject, in a stereoscopic imaging device equipped with an imaging means for ranging by a TOF method and an imaging means for ranging by stereo matching. <P>SOLUTION: The first distance image generation part 30 generates a distance image D1 from data for the distance image acquired by the first imaging part 2A. A stereo matching part 31 detects a corresponding point from a standard image and a reference image acquired by the second and third imaging parts 2B, 2C, and the second distance image generation part 32 generates a distance image D2. In this case, imaging is performed by the second and third imaging parts 2B, 2C before imaging by the first imaging part 2A, and a face detection part 39 detects a prescribed subject such as a face from an image acquired thereby. When the prescribed subject is detected, imaging is performed only by the second and third imaging parts, and only the second distance image generation part 32 is allowed to generate a distance image D2. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a stereoscopic imaging apparatus that acquires a distance image representing a stereoscopic shape of a subject, a control method for the stereoscopic imaging apparatus, and a program for causing a computer to execute the control method for the stereoscopic imaging apparatus.

  In an imaging device that captures an image of a subject and obtains an image, the distance from the imaging device to the subject is measured by measuring the time until ranging light such as near infrared rays emitted toward the subject is reflected by the subject and returns. A distance image representing a three-dimensional shape of a subject is generated by measuring a distance. The method of measuring the distance to the subject (ranging) using the reflection of light in this way is called a TOF (Time Of Flight) method, and various methods for measuring the distance using the TOF method are proposed. Has been.

  In addition, the subject is imaged using at least two cameras provided at different positions, and corresponding pixels among a plurality of images (a reference image by the reference camera and a reference image by the reference camera) acquired thereby. By searching (stereo matching) and applying the principle of triangulation to the difference in position (parallax) between the corresponding pixel on the reference image and the pixel on the reference image, the reference camera or reference camera applies the pixel to the pixel. A method for measuring a distance to a corresponding point on a subject and generating a distance image representing a three-dimensional shape of the subject has also been proposed.

  Here, when performing stereo matching, as shown in FIG. 7, the points on the real space mapped to a certain point Pa on the reference image G1 are points P2, P2, and P3 from the point O2. Since there are a plurality of lines on the line of sight, the point Pa ′ on the reference image G2 corresponding to the point Pa exists on a straight line (epipolar line) that is a map of the points P1, P2, P3, etc. in the real space. Corresponding points are searched for. In FIG. 7, the point O2 is the viewpoint of the base camera, and the point O3 is the viewpoint of the reference camera. When performing stereo matching in this way, it is necessary to obtain corresponding points for all the pixels on the epipolar line, so that it takes a long time to search for corresponding points.

  For this reason, another camera arranged at a different position is added to two cameras (referred to as stereo cameras), and a silhouette of the subject is created from the images acquired by the added cameras. A technique has been proposed in which feature points are detected from an acquired image, and corresponding points are searched based on the feature points only in the range of the silhouette of the subject, thereby shortening the search time for the corresponding points (Patent Document 1). reference).

  Also, if a stereo camera and millimeter wave radar are used and an obstacle is detected by the millimeter wave radar and no obstacle is detected in the image acquired by the stereo camera, it is acquired by the stereo camera based on the detection result by the millimeter wave radar. There has been proposed a technique for improving the obstacle detection accuracy and shortening the search time for corresponding points by limiting the search range for corresponding points in the image (see Patent Document 2).

In addition, although it is an infrared camera, an object region is extracted from a first image acquired by one stereo camera, and based on a correlation value between the extracted region and a second image acquired by the other stereo camera. When extracting a subject area from the second image and searching for corresponding points in the range of the subject area to calculate a distance to the subject to generate a distance image, the search range is narrowed according to the calculated distance. Accordingly, a method for preventing the correspondence between the corresponding points has also been proposed (see Patent Document 3).
JP 2000-331160 A JP 2006-234513 A JP 2003-322521 A

  By the way, both a camera for distance measurement by the TOF method and two cameras for distance measurement by stereo matching are prepared, and two distance images of a distance image by the TOF method and a distance image by the stereo matching are acquired. It is possible. However, a subject such as a person may feel annoyed by the person who is the subject at the time of shooting when the distance measuring light is irradiated. In addition, there are subjects that are not preferable to be irradiated with distance measuring light as well as a person.

  The present invention has been made in view of the above circumstances, and in a stereoscopic imaging apparatus including an imaging unit for measuring a distance by the TOF method and an imaging unit for measuring a distance by stereo matching, The purpose is to be able to limit the irradiation.

A stereoscopic imaging device according to the present invention is a first imaging apparatus that irradiates a subject with distance measuring light, images reflected light of the distance measuring light from the subject, and obtains data for a distance image representing the three-dimensional shape of the subject. Imaging means;
First distance image generation means for generating a first distance image representing the three-dimensional shape of the subject based on the distance image data;
Second imaging means for acquiring a reference image of the subject for calculating a second distance image by imaging the subject;
At least one third imaging means for acquiring a reference image of the subject for calculation of the second distance image by imaging the subject from a position different from the second imaging means;
Corresponding point search means for searching for corresponding points of pixels between the reference image and the reference image;
Second distance image generation means for generating the second distance image representing the three-dimensional shape of the subject based on the searched corresponding points;
Before imaging by the first imaging unit, the second and third imaging units perform imaging, a predetermined subject is detected from an image acquired by the imaging, and a detection result of the predetermined subject is output Subject detection means;
When the predetermined subject is detected, only the second and third imaging means perform imaging, and only the second distance image generating means generates the second distance image. The apparatus further comprises second and third imaging means, corresponding point searching means, and control means for controlling the second distance image generating means.

  In the stereoscopic imaging apparatus according to the present invention, when the predetermined subject is detected, the corresponding point search means uses a distance range including the predetermined subject in the reference image and the reference image as the corresponding point search range. It is good also as a means to set.

The method for controlling a stereoscopic imaging apparatus according to the present invention irradiates a subject with distance measuring light, images reflected light of the distance measuring light from the subject, and obtains data for a distance image representing the three-dimensional shape of the subject. First imaging means;
First distance image generation means for generating a first distance image representing the three-dimensional shape of the subject based on the distance image data;
Second imaging means for acquiring a reference image of the subject for calculating a second distance image by imaging the subject;
At least one third imaging means for acquiring a reference image of the subject for calculation of the second distance image by imaging the subject from a position different from the second imaging means;
Corresponding point search means for searching for corresponding points of pixels between the reference image and the reference image;
A control method for a stereoscopic imaging apparatus, comprising: a second distance image generation unit configured to generate the second distance image representing the stereoscopic shape of the subject based on the searched corresponding points;
Prior to imaging by the first imaging means, the second and third imaging means perform imaging,
Detecting a predetermined subject from an image acquired by the imaging and outputting a detection result of the predetermined subject;
When the predetermined subject is detected, only the second and third imaging means perform imaging,
Only the second distance image generating means generates the second distance image.

  In addition, you may provide as a program for making a computer perform the control method of the three-dimensional imaging device by this invention.

  According to the present invention, when a predetermined subject is detected from an image acquired by performing imaging with the second and third imaging means before imaging by the first imaging means, the second and third Only the image capturing means performs imaging so that only the second distance image is generated. For this reason, when a predetermined subject is detected, it is not necessary to generate a first distance image by irradiating the predetermined subject with distance measuring light. In particular, when the predetermined subject is a person, it is not necessary to irradiate the person with the distance measuring light, so that it is possible to prevent the person who is the subject from being annoyed by the distance measuring light during imaging.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a front perspective view showing an external configuration of a stereoscopic imaging apparatus according to a first embodiment of the present invention, and FIG. 2 is a rear perspective view thereof. As shown in FIGS. 1 and 2, the stereoscopic imaging apparatus 1 according to the first embodiment includes first to third imaging units 2 </ b> A, 2 </ b> B, and 2 </ b> C on the front side thereof. The first imaging unit 2A is disposed at a substantially central position on the front surface of the stereoscopic imaging device 1, and performs imaging for acquiring the first distance image D1 based on irradiating the subject with distance measuring light. Is. In addition, a distance measuring light irradiation unit 3 such as an LED for irradiating distance measuring light composed of infrared light toward the subject is provided at the upper and lower positions of the first imaging unit 2A.

  The second and third imaging units 2B and 2C are arranged at the left and right positions of the first imaging unit 2A, and are used as reference images necessary for performing stereo matching to generate the second distance image D2. Imaging for obtaining G1 and reference image G2 is performed.

  In addition, a release button 4 for performing imaging is provided on the top surface of the stereoscopic imaging device 1. Further, on the back side, a monitor 5 that performs various displays including an image acquired by imaging is provided.

  When the subject is imaged by such a stereoscopic imaging apparatus 1, the imaging ranges by the imaging units 2A, 2B, and 2C are as shown in FIG. In FIG. 3, points O1 to O3 are viewpoints of the imaging units 2A, 2B, and 2C, respectively. Further, the optical axis of the imaging unit 2B is the z axis, the vertical direction of the paper is the x axis, and the vertical direction is the y axis. H is a subject.

  4 is a schematic block diagram showing the internal configuration of the first imaging unit 2A, and FIG. 5 is a schematic diagram showing the internal configuration of the second and third imaging units 2B and 2C.

  The first to third imaging units 2A, 2B, and 2C include lenses 10A, 10B, and 10C, diaphragms 11A, 11B, and 11C, shutters 12A, 12B, and 12C, CCDs 13A, 13B, and 13C, an analog front end (AFE) 14A, and the like. 14B, 14C and A / D converters 15A, 15B, 15C, respectively.

  The lenses 10A, 10B, and 10C are configured by a plurality of functional lenses such as a focus lens for focusing on a subject and a zoom lens for realizing a zoom function, and their positions are adjusted by a lens driving unit (not shown). The

  The apertures 11A, 11B, and 11C are adjusted in aperture diameter based on aperture value data obtained by AE processing by an aperture drive unit (not shown).

  The shutters 12A, 12B, and 12C are mechanical shutters, and are driven by a shutter drive unit (not shown) according to the shutter speed obtained by the AE process.

  The CCDs 13A, 13B, and 13C have a photoelectric surface in which a large number of light receiving elements are two-dimensionally arranged, and subject light is imaged on the photoelectric surface and subjected to photoelectric conversion to acquire an analog imaging signal. In addition, color filters in which R, G, and B color filters are regularly arranged are arranged on the front surfaces of the CCDs 13B and 13C.

  The AFEs 14A, 14B, and 14C perform processing for removing noise of the analog imaging signal and processing for adjusting the gain of the analog imaging signal (hereinafter referred to as analog processing) for the analog imaging signals output from the CCDs 13A, 13B, and 13C. Apply.

  The A / D converters 15A, 15B, and 15C convert the analog imaging signals that have been subjected to the analog processing by the AFEs 14A, 14B, and 14C into digital signals. Note that the image data obtained by the CCDs 13B and 13C of the imaging units 2B and 2C and converted into digital signals is data having R, G, and B density values for each pixel. Note that the image represented by the image data acquired by the imaging unit 2B is the standard image G1, and the image represented by the image data acquired by the imaging unit 2C is the reference image G2.

  Further, the imaging unit 2A includes a modulation unit 18 for modulating the distance measuring light emitted from the distance measuring light irradiation unit 3. Further, a drive control unit 19 for controlling the drive of the CCD 13A, the AFE 14A, and the modulation unit 18 is provided. Specifically, in response to an instruction from the CPU 33, the drive control unit 19 instructs the modulation unit 18 to perform modulation, causes the ranging light irradiation unit 3 to emit modulated light, and adjusts the timing of each unit of the imaging unit 2A. Drive to obtain data for distance image.

  FIG. 6 is a schematic block diagram showing the internal configuration of the stereoscopic imaging apparatus 1 according to the first embodiment. The stereoscopic imaging device 1 includes an imaging control unit 22, an image processing unit 23, a compression / decompression processing unit 24, a frame memory 25, a media control unit 26, an internal memory 27, and a display control unit 28.

  The imaging control unit 22 includes an AF processing unit and an AE processing unit. The AF processing unit determines the focal lengths of the lenses 10A, 10B, and 10C based on images (pre-standard images and pre-reference images) acquired by the imaging units 2B and 2C by half-pressing the release button 4, and the imaging units 2A and 2C Output to 2B and 2C. The AE processing unit determines an aperture value and a shutter speed based on the pre-image, and outputs them to the imaging units 2A, 2B, and 2C.

  The image processing unit 23 performs image processing such as white balance adjustment processing, tone correction, sharpness correction, and color correction on the digital image data acquired by the imaging units 2B and 2C. Note that reference codes G1 and G2 before processing are also used for the standard image and the reference image after processing in the image processing unit 23.

  The compression / decompression processing unit 24 applies, for example, JPEG to image data representing the base image G1 and the reference image G2 that have been subjected to image processing by the image processing unit 23 and image data of a distance image generated as described later. The image file is generated by performing compression processing in a compression format such as. A tag storing incidental information such as shooting date and time is added to the image file based on the Exif format or the like.

  The frame memory 25 performs various processes including the above-described image processing on the distance image data acquired by the imaging unit 2A and the image data representing the reference image G1 and the reference image G2 acquired by the imaging units 2B and 2C. This is a working memory used when performing.

  The media control unit 26 accesses the recording medium 29 and controls writing and reading of an image file such as a distance image.

  The internal memory 27 stores various constants set in the stereoscopic imaging apparatus 1, a program executed by the CPU 33, and the like.

  The display control unit 28 is for displaying the image data stored in the frame memory 25 on the monitor 5 and displaying the image recorded on the recording medium 29 on the monitor 5.

  In addition, the stereoscopic imaging device 1 includes a first distance image generation unit 30, a stereo matching unit 31, a second distance image generation unit 32, and a CPU 33.

  The first distance image generation unit 30 calculates the distance from the stereoscopic imaging device 1 at each point on the subject using the distance image data acquired by the imaging unit 2A to generate a distance image. Specifically, the phase difference between the emitted light and the reflected light of the distance measuring light irradiation unit 3 is calculated from the distance image data obtained by imaging the reflected light of the distance measuring light reflected by the subject. Based on this phase difference, the distance from the imaging unit 2A to the subject is calculated. Then, the calculated distance is associated with each pixel of the CCD 13A to generate the first distance image D1. The pixel value of each pixel in the first distance image D1 represents the distance from the imaging unit 2A to the subject.

  As shown in FIG. 7, the stereo matching unit 31 has a plurality of points in the real space mapped to a certain point Pa on the reference image G1 on the line of sight from the point O2, such as points P1, P2, and P3. Therefore, based on the fact that the point Pa ′ on the reference image R corresponding to the point Pa exists on a straight line (epipolar line) that is a map of the points P1, P2, P3, etc. in the real space, the reference image Corresponding points between G1 and the reference image G2 are searched on the reference image G2. In FIG. 7, the point O2 is the viewpoint of the imaging unit 2B that is the reference camera, and the point O3 is the viewpoint of the imaging unit 2C that is the reference camera. Here, the viewpoint is the center of the CCD.

  Further, when searching for the corresponding points, the stereo matching unit 31 searches the standard image G1 and the reference image G2 for regions within a predetermined distance range based on a predetermined distance on the distance image D1. Set the corresponding point search range. The setting of the corresponding point search range will be described in detail below.

  FIG. 8 is a diagram for explaining the relationship between the imaging units 2B and 2C. Here, the viewpoints O2 and O3, which are the intersections of the optical axes of the imaging units 2B and 2C, and the image plane that is the plane from which the base image G1 and the reference image G2 are obtained, are used as the origin, and the imaging units 2B and 2B on the image plane Let 2C coordinate systems be (u, v) and (u ', v'), respectively. In addition, it is assumed that the optical axes of the imaging units 2B and 2C are parallel, and the u-axis and u′-axis on the image plane face the same direction on the same straight line. Further, the focal length of the imaging units 2B and 2C is assumed to be f, and the baseline length is assumed to be b. The focal length f and the baseline length b are calculated in advance as calibration parameters and stored in the internal memory 27.

  At this time, the position (X, Y, Z) in the three-dimensional space is expressed by the following formulas (1) to (3) with reference to the coordinate system of the imaging unit 2B.

X = b · u / (u−u ′) (1)
Y = b · v / (u−u ′) (2)
Z = b · f / (u−u ′) (3)
Here, u−u ′ is a lateral shift amount (parallax) of the projection point on the image plane of the imaging units 2B and 2C. Further, it can be seen from the expression (3) that the distance Z, which is the depth, is inversely proportional to the parallax.

  FIG. 9 is a diagram for explaining the setting of the search range. Here, it is assumed that a predetermined distance on the distance image D1 is La, and a predetermined distance range based on the distance La is L0 to L1. Specifically, for example, the distance range L0 to L1 can use a value of about 10 cm before and after the distance La as a reference. Note that the predetermined distance La may be a value set in advance by the user, or a representative value such as an average value of distances in the entire distance image D1 or in the central area may be used.

Here, if the value in the u-axis direction of the coordinates of the target point Pt to be searched for corresponding points on the screen of the imaging unit 2B is ut, the corresponding point P1 of the target point Pt on the image plane of the imaging unit 2C. Is on the u ′ axis. In the present embodiment, the corresponding points are searched in the distance ranges L0 to L1 with the distance La as a reference. For this reason, u ′ coordinates u0 and u1 on the image plane of the imaging unit 2C corresponding to the distances L0 and L1 are obtained, respectively. That is, from the above equation (3),
L0 = b · f / (ut−u0) (4)
L1 = b · f / (ut−u1) (5)
Because
u0 = -b. f / L0 + ut (6)
u1 = -b. f / L1 + ut (7)
The coordinates u0 and u1 can be obtained as follows. Therefore, the stereo matching unit 31 sets the range of u0 to u1 calculated by the equations (6) and (7) as the corresponding point search range on the reference image G2.

  The second distance image generation unit 32 uses the corresponding points obtained by the stereo matching unit 31 to calculate the distances from the imaging units 2B and 2C to the subject by the above formulas (1) to (3). The second distance image D2 is generated by associating the determined distance with the pixels of the CCDs 13B and 13C. The pixel value of each pixel of the second distance image D2 represents the distance from the imaging units 2B and 2C to the subject. The generated distance image D2 is recorded on the recording medium 29.

  Note that when the distance image D2 is generated, the corresponding point may not be found on the reference image G2. In such a case, the second distance image generation unit 32 applies the pixel value (that is, the distance) of the corresponding pixel in the distance image D1 to the uncorresponding pixel for which the corresponding point is not found on the distance image D2. Thus, the third distance image D3 is generated. In this case, the third distance image D3 is recorded on the recording medium 29.

  The CPU 33 controls each unit of the stereoscopic imaging device 1 in accordance with a signal from the input unit 34 including the release button 4.

  The data bus 35 is connected to each unit constituting the stereoscopic imaging device 1 and the CPU 33, and exchanges various data and various information in the stereoscopic imaging device 1.

  Next, processing performed in the first embodiment will be described. FIG. 10 is a flowchart showing processing performed in the first embodiment. It is assumed that AF processing and AE processing in the imaging control unit 22 based on the half-pressing operation of the release button 4 has already been performed, and here, processing after the release button 4 is fully pressed and imaging is performed. Will be described.

  When the release button 4 is fully pressed, the CPU 33 starts processing, and the imaging unit 2A emits distance measuring light toward the subject from the distance measuring light irradiation unit 3 according to an instruction from the CPU 33 (step ST1). The reflected light from the subject of the distance light is imaged, and data for the distance image is acquired (step ST2). Then, the first distance image generation unit 30 generates a first distance image D1 from the distance image data (step ST3).

  On the other hand, the imaging units 2B and 2C image the subject according to an instruction from the CPU 33, and the image processing unit 23 performs image processing on the acquired image data to acquire the reference image G1 and the reference image G2 (step ST4). Note that the processes of steps ST1 to ST3 and step ST4 may be performed in parallel or sequentially. In the present embodiment, it is performed in parallel.

  Next, the stereo matching unit 31 sets an area within a predetermined distance range L0 to L1 with reference to a predetermined distance La on the distance image D1 as a corresponding point search range (step ST5). Corresponding points are searched in the searched range (step ST6). In addition, since the process of step ST5 cannot be performed until the distance image D1 is generated, when the distance image D1 is not generated at the time when the standard image G1 and the reference image G2 are acquired, the process of the distance image D1 is performed. It is necessary to wait for the process of step ST5 until the generation is completed.

  And the 2nd distance image generation part 32 produces | generates the 2nd distance image D2 based on the searched corresponding point (step ST7). Further, it is determined whether or not there is a pixel for which no corresponding point is found on the distance image D2 (step ST8). When step ST8 is affirmed, the pixel value of the pixel for which no corresponding point is found in the distance image D2 is set. On the other hand, the pixel value of the corresponding pixel in the distance image D1 is applied to generate the third distance image D3 (step ST9). If step ST8 is negative, the process proceeds to step ST10.

  Then, in response to an instruction from the CPU 33, the media control unit 26 records the standard image G1, the reference image G2, and the distance image D2 or D3 on the recording medium 29 (image recording: step ST10), and the process ends.

  As described above, in the first embodiment, an area within the predetermined distance range L0 to L1 with the predetermined distance La on the first distance image D1 as a reference is set as the corresponding point search range. Thus, the corresponding points are searched. As described above, since the first distance image D1 is used for setting the search range of the corresponding points, the reference image G1 and the reference image G2 are determined in advance without depending on the size of the subject to be imaged. Only a region within a certain distance range can be set as a corresponding point search range. Therefore, the search range for corresponding points can be reliably narrowed, and as a result, the calculation time for searching for corresponding points can be shortened. Further, since the search range can be set in units of pixels on the standard image G1 and the reference image G2, the corresponding points can be searched with high accuracy.

  In addition, for the pixel for which no corresponding point is found in the second distance image D2, the pixel value of the corresponding pixel in the first distance image D1 is applied to generate the third distance image D3. A range image can be generated well.

  Next, a second embodiment of the present invention will be described. FIG. 11 is a front perspective view showing an external configuration of a stereoscopic imaging apparatus according to the second embodiment of the present invention. Note that in the second embodiment, the same reference numerals are assigned to the same components as those in the first embodiment, and detailed description thereof is omitted. In the stereoscopic imaging apparatus 1A according to the second embodiment, the imaging units 2B and 2C are attached to the mounts 6B and 6C that can slide to the left and right of the stereoscopic imaging apparatus 1A, and the mounts 6B and 6C are left and right as shown in FIG. This is different from the first embodiment in that the base line length of the imaging units 2B and 2C can be changed by sliding the image pickup unit 2B.

  FIG. 13 is a schematic block diagram showing the internal configuration of the stereoscopic imaging apparatus according to the second embodiment. As shown in FIG. 13, the stereoscopic imaging apparatus 1 </ b> A according to the second embodiment includes a drive mechanism 36 for sliding the mounts 6 </ b> B and 6 </ b> C and a mount drive control unit 37 for controlling the drive of the drive mechanism 36. Is different from the stereoscopic imaging apparatus 1 according to the first embodiment. The drive mechanism 36 and the mount drive control unit 37 correspond to the baseline length changing unit.

  The drive mechanism 36 includes a motor and a slide mechanism (not shown), and slides the mounts 6B and 6C so as to change the base line length of the imaging units 2B and 2C according to an instruction from the mount drive control unit 37. Note that the base line length of the imaging units 2B and 2C can be changed either automatically or manually. In the case of manual operation, the user changes the baseline length by operating the input unit 34. Further, it is assumed that whether the base line length of the imaging units 2B and 2C is changed automatically or manually is set in advance by the user.

  The function of the mount drive control unit 37 will be described in the process performed in the second embodiment below.

  FIG. 14 is a flowchart showing processing performed in the second embodiment. It is assumed that AF processing and AE processing in the imaging control unit 22 based on the half-pressing operation of the release button 4 has already been performed, and here, processing after the release button 4 is fully pressed and imaging is performed. Will be described.

  When the release button 4 is fully pressed, the CPU 33 starts processing, and stores an initial baseline length value, which is the current baseline length of the imaging units 2B and 2C, in the internal memory 27 (step ST21). Next, when the release button 4 is fully pressed, the imaging unit 2A irradiates distance measuring light toward the subject from the distance measuring light irradiation unit 3 according to an instruction from the CPU 33 (step ST22), and further depends on the subject of the distance measuring light. The reflected light is imaged to acquire distance image data (step ST23). Then, the first distance image generation unit 30 generates the first distance image D1 from the distance image data (step ST24).

  On the other hand, the imaging units 2B and 2C image a subject according to an instruction from the CPU 33, and the image processing unit 23 performs image processing on the acquired image data to acquire the reference image G1 and the reference image G2 (step ST25). Note that the processes of steps ST22 to 24 and step ST25 may be performed in parallel or sequentially. In the present embodiment, it is performed in parallel.

  Then, the mount drive control unit 37 performs a baseline length change process (step ST26). Since the base line length changing process cannot be performed until the distance image D1 is generated, if the distance image D1 is not generated at the time when the base image G1 and the reference image G2 are acquired, the baseline length change process is performed. It is necessary to wait for the process of step ST26 until the generation is completed.

  FIG. 15 is a flowchart of the base line length changing process. First, the mount drive control unit 37 calculates a subject distance that is a distance from the stereoscopic imaging apparatus 1A to the subject and a subject range that is a range including the subject in the distance image D1 based on the distance image D1 (step ST41). . Specifically, the distance image D1 is divided into a plurality of regions (for example, 3 × 3), and a representative value such as an average value of the distance in the center region is calculated as the subject distance. Then, in the distance image D1, an area including pixels within a distance range of, for example, ± 10% with respect to the calculated subject distance is calculated as the subject area.

  The distance image D1 may be displayed on the monitor 5, the user may select a subject area, and a representative value such as an average value of distances in the selected subject area may be calculated as the subject distance.

  Next, the mount drive control unit 37 determines whether or not the calculated subject distance is greater than a predetermined distance TL1 determined in advance (step ST42). The predetermined distance TL1 is set in advance by the user and stored in the internal memory 27. If step ST42 is affirmed, it is determined whether or not the base line length is set to be automatically changed (step ST43). If step ST43 is negative, an operation to change the baseline length using the input unit 34 by the user is accepted (step ST44). Then, it is determined whether or not the baseline length has been changed to a preset limit range (step ST45).

  FIG. 16A shows the base image G1 and the reference image G2 before the change of the base line length, and FIG. 16B shows the base image G1 and the reference image G2 when the base line length is changed. As shown in FIG. 16 (a), the subject H located at the approximate center of the base image G1 and the reference image G2 is gradually changed so that the base line length is increased, so that the base image G1 and the reference image G2 are gradually changed. Move to the end. Here, the ends of the base image G1 and the reference image G2 are often distorted due to the distortion of the lenses of the imaging units 2B and 2C and the distortion of the CCDs 13B and 13C. For this reason, in the second embodiment, the base line length is changed so that the subject H is not located in a region within the predetermined range around the base image G1 and the reference image G2 (the hatched portion shown in FIG. 16B). The range is limited.

  Accordingly, the mount drive control unit 37 detects a subject from the standard image G1 and the reference image G2, and determines whether or not the detected subject is located in a region within a predetermined range around the standard image G1 and the reference image G2. Thus, the process of step ST45 is performed.

  If step ST45 is negative, it is determined whether or not the user has instructed to end the operation for changing the baseline length (step ST46). This determination is performed by, for example, performing an inquiry display of “Do you want to end the change of the baseline length” on the monitor 5 and determining whether or not the user has input YES to the inquiry display. If step ST46 is negative, the process returns to step ST44 and the processes after step ST44 are repeated. If step ST46 is positive, the mounts 6B and 6C are fixed at that position (step ST47), and the process proceeds to steps ST22 and 25 of FIG. Moreover, also when step ST45 is affirmed, it progresses to the process of step ST47.

  On the other hand, if step ST43 is affirmed, the mount drive control unit 37 drives the drive mechanism 36 to change the base line length to a preset limit range (step ST48), and the process proceeds to step ST47.

  If step ST42 is negative, it is determined whether the baseline length is different from the initial value (step ST49). If step ST49 is positive, the calibration parameter is changed based on the current baseline length (step ST50), and the process proceeds to step ST27 shown in FIG. Moreover, also when step ST49 is denied, it progresses to the process of step ST27.

  Returning to FIG. 14, the CPU 33 performs the processes of step ST27 to step ST32 and ends the process. In addition, since the process of step ST27-step ST32 is the same as the process of step ST5-step ST10 in 1st Embodiment, detailed description is abbreviate | omitted here.

  As described above, in the second embodiment, when the subject distance is larger than the predetermined distance TL1, the base line length, which is the distance between the image capturing units 2B and 2C, is changed. Here, when the base line length is increased, it is possible to improve the accuracy of distance calculation for a subject located at a position away from the imaging units 2B and 2C. For this reason, it is possible to improve the accuracy of generation of the distance image D2 using the standard image G1 and the reference image G2.

  Next, a third embodiment of the present invention will be described. Note that the external configuration of the stereoscopic imaging device according to the third embodiment is the same as the external configuration of the stereoscopic imaging device 1 according to the first embodiment, and thus detailed description thereof is omitted here. FIG. 17 is a schematic block diagram showing the internal configuration of the stereoscopic imaging apparatus according to the third embodiment. As shown in FIG. 17, the stereoscopic imaging apparatus 1 </ b> B according to the third embodiment is provided with a zoom magnification changing unit 38 that controls driving of the zoom lenses of the lenses 10 </ b> B and 10 </ b> C of the imaging units 2 </ b> B and 2 </ b> C. It differs from the stereoscopic imaging device 1 according to the form.

  The function of the zoom magnification changing unit 38 will be described in the processing performed in the third embodiment below.

  FIG. 18 is a flowchart showing processing performed in the third embodiment. It is assumed that AF processing and AE processing in the imaging control unit 22 based on the half-pressing operation of the release button 4 has already been performed, and here, processing after the release button 4 is fully pressed and imaging is performed. Will be described.

  When the release button 4 is fully pressed, the CPU 33 starts processing, and stores in the internal memory 27 the initial zoom value that is the current zoom position of the zoom lenses 10B and 10C of the imaging units 2B and 2C (step ST61). ). Next, when the release button 4 is fully pressed, the imaging unit 2A irradiates distance measuring light toward the subject from the distance measuring light irradiation unit 3 according to an instruction from the CPU 33 (step ST62), and further depends on the subject of the distance measuring light. The reflected light is imaged to acquire distance image data (step ST63). Then, the first distance image generation unit 30 generates the first distance image D1 from the distance image data (step ST64).

  Further, the imaging units 2B and 2C capture the subject according to an instruction from the CPU 33, and the image processing unit 23 performs image processing on the acquired image data to acquire the reference image G1 and the reference image G2 (step ST65). Note that the processes of steps ST62 to ST64 and step ST65 may be performed in parallel or sequentially. In the present embodiment, it is performed in parallel.

  Then, the zoom magnification changing unit 38 performs zoom magnification changing processing (step ST66). Since the zoom magnification changing process cannot be performed until the distance image D1 is generated, if the distance image D1 has not been generated at the time when the standard image G1 and the reference image G2 are acquired, the distance image D1 is displayed. It is necessary to wait for the process of step ST66 until the generation is completed.

  FIG. 19 is a flowchart of zoom magnification change processing. First, similarly to the mount drive control unit 37 in the second embodiment, the zoom magnification changing unit 38 determines the subject distance that is the distance from the stereoscopic imaging apparatus 1A to the subject and the subject in the distance image D1 based on the distance image D1. A subject range that is an included range is calculated (step ST81).

  Next, the zoom magnification changing unit 38 determines whether or not the calculated subject distance is larger than a predetermined distance TL2 determined in advance (step ST82). The predetermined distance TL2 is set in advance by the user and stored in the internal memory 27. If step ST82 is affirmed, it is determined whether or not the zoom magnification is automatically changed (step ST83). If step ST83 is negative, a zoom magnification change operation by the user using the input unit 34 is accepted (step ST84). Then, it is determined whether or not the zoom magnification has been changed to a preset limit range (step ST85). Specifically, as in the second embodiment, a subject is detected from the base image G1 and the reference image G2, and the detected subject is located in a region within a predetermined range around the base image G1 and the reference image G2. By determining whether or not, the process of step ST85 is performed.

  If step ST85 is negative, it is determined whether or not the user has instructed to end the zoom magnification change operation (step ST86). This determination is made by, for example, performing an inquiry display “Do you want to end the zoom magnification change?” On the monitor 5 and determining whether or not the user has entered YES for the inquiry display. If step ST86 is negative, the process returns to step ST84 and the processes after step ST84 are repeated. If step ST86 is positive, the zoom is fixed at that position (step ST87), and the process proceeds to steps ST62 and 65 shown in FIG. Moreover, also when step ST85 is affirmed, it progresses to the process of step ST87.

  On the other hand, if step ST83 is affirmed, the zoom magnification changing unit 38 changes the zoom magnification to the limit range (step ST88), and the process proceeds to step ST87.

  If step ST82 is negative, it is determined whether the zoom magnification is different from the initial value (step ST89). If step ST89 is positive, the calibration parameter is changed based on the current zoom magnification (step ST90), and the process proceeds to step ST67 shown in FIG. Moreover, also when step ST89 is denied, it progresses to the process of step ST67.

  Returning to FIG. 18, the CPU 33 performs the processes of step ST67 to step ST72 and ends the process. In addition, since the process of step ST67-step ST72 is the same as the process of step ST5-step ST10 in 1st Embodiment, detailed description is abbreviate | omitted here.

  As described above, in the third embodiment, when the subject distance is larger than the predetermined distance TL2, the zoom magnification of the imaging units 2B and 2C is changed to be increased. Here, if the zoom magnification is increased, it is possible to improve the accuracy of distance calculation for a subject located at a position away from the imaging units 2B and 2C. For this reason, it is possible to improve the accuracy of generation of the distance image D2 using the standard image G1 and the reference image G2.

  Next, a fourth embodiment of the present invention will be described. Note that the external configuration of the stereoscopic imaging device according to the fourth embodiment is the same as the external configuration of the stereoscopic imaging device 1 according to the first embodiment, and thus detailed description thereof is omitted here. FIG. 20 is a schematic block diagram showing the internal configuration of the stereoscopic imaging apparatus according to the fourth embodiment. As shown in FIG. 20, the stereoscopic imaging apparatus 1C according to the fourth embodiment includes a face detection unit 39, and images the subject with the imaging units 2B and 2C before imaging with all of the imaging units 2A to 2C. A pre-standard image and a pre-reference image are obtained, and it is determined whether or not a human face is included in the pre-standard image and the pre-reference image. If a face is included, imaging using the imaging unit 2A is prohibited. This is different from the first embodiment. Note that the pre-standard image and the pre-reference image can be quickly determined whether or not a face is included by reducing the number of pixels as compared with the standard image G1 and the reference image G2.

  The face detection unit 39 uses a template matching method, a method using a face discriminator obtained by machine learning learning using a large number of sample images of a face, and the like to enclose a face on the pre-standard image and the pre-reference image. A range area is detected as a face area. The detection result of the face area includes information indicating whether or not a face is included in the pre-standard image and the pre-reference image, the number of faces when the face is included, and the face area on the pre-standard image and the pre-reference image. Contains the coordinate value representing the position.

  Next, processing performed in the fourth embodiment will be described. FIG. 21 is a flowchart showing processing performed in the fourth embodiment. The CPU 33 starts processing by turning on the power of the stereoscopic imaging apparatus 1C, and starts monitoring whether or not the release button 4 is half-pressed (step ST101). If step ST101 is affirmed, the imaging units 2B and 2C capture the subject in accordance with an instruction from the CPU 33 to obtain a pre-standard image and a pre-reference image (step ST102). Next, the face detection unit 39 detects a face area from the pre-standard image and the pre-reference image, generates a detection result, and stores the detection result in the internal memory 27 (step ST103). Note that the imaging control unit 22 performs AF processing and AE processing based on at least one of the pre-standard image and the pre-reference image. Subsequently, it is determined whether or not the release button 4 has been fully pressed (step ST104). If step ST104 is negative, the process returns to step ST101, and the processes after step ST101 are repeated.

  If step ST104 is affirmed, the CPU 33 determines whether or not a face region is included in the pre-standard image and the pre-reference image based on the detection result of the face detection unit 39 (step ST105). If step ST105 is negative, the distance image D2 or D3 is generated using the imaging units 2A to 2C (step ST106), and the process ends. Note that the process in step ST106 is the same as the process in steps ST1 to ST10 in the first embodiment, and thus detailed description thereof is omitted here.

  On the other hand, when step ST105 is affirmed, only the imaging units 2B and 2C image the subject according to an instruction from the CPU 33, and the image processing unit 23 performs image processing on the acquired image data, and the reference image G1 and the reference Image G2 is acquired (step ST107). Next, the stereo matching unit 31 detects the correspondence between the face areas included in the standard image G1 and the reference image G2 (step ST108). Note that the positions of the face areas of the standard image G1 and the reference image G2 may be obtained by detecting the face areas of the pre-standard image and the pre-reference image by causing the face detection unit 39 to detect the face areas. May be.

  FIG. 22 is a diagram for explaining the correspondence between face areas. As shown in FIG. 22 (a), when the standard image G1 and the reference image G2 include only one face area F1, F2, respectively, the face areas F1, F2 of the standard image G1 and the reference image G2 correspond to each other. It is attached. On the other hand, as shown in FIG. 22B, when a plurality of face areas are included in the standard image G1 and the reference image G2, face areas having the same size are associated with each other. Specifically, the face area F11 of the standard image G1 is associated with the face area F12 of the reference image G2, and the face area F13 of the standard image G1 is associated with the face area F14 of the reference image G2. In addition, as shown in FIG. 22C, when a plurality of face areas are included in the standard image G1 and the reference image G2 and the sizes of the plurality of face areas are the same, based on the positional relationship between the face areas. Face areas are associated with each other. Specifically, the face area F21 of the reference image G1 is the face area F22 of the reference image G2, the face area F23 of the reference image G1 is the face area F24 of the reference image G2, and the face area F25 of the reference image G1 is the reference image G2. Are respectively associated with the face area F26.

  Next, as shown in FIG. 23, the stereo matching unit 31 sets the center lines C1 and C2 of the face area in the standard image G1 and the reference image G2, calculates the shift between the center lines C1 and C2 as the parallax d, Using the parallax d, the distance from the stereoscopic imaging device 1C to the center lines C1 and C2 of the face area is calculated (step ST109). Then, an area within a predetermined distance range based on the calculated distance is set as a corresponding point search range (step ST110), and a corresponding point is searched in the set search range (step ST111). As the distance range in this case, since the center line of the face passes near the nose of the person, the distance range should be about 5 cm before and 20 cm after the calculated distance so that the entire face falls within the distance range. Is preferred. Note that the range of the parallax d calculated in step ST109 may be set as the corresponding point search range.

  And the 2nd distance image generation part 32 produces | generates the 2nd distance image D2 based on the searched corresponding point (step ST112). Then, in response to an instruction from the CPU 33, the media control unit 26 records the standard image G1, the reference image G2, and the distance image D2 on the recording medium 29 (image recording: step ST113), and ends the process.

  As described above, in the fourth embodiment, when a face area is detected from the pre-standard image and the pre-reference image, imaging using the imaging unit 2A is not performed. For this reason, the person is not irradiated with the distance measuring light, and as a result, it is possible to prevent the person who is the subject from being bothered by the distance measuring light.

  Although the face area is detected from the pre-standard image and the pre-reference image in the fourth embodiment, the subject to be detected is not limited to the face.

  Further, in the fourth embodiment, the deviation of the center line of the detected face area is calculated as the parallax d, and the distance image D2 is generated using the calculated parallax d. However, the reference image G1 and the reference image are generated. A corresponding point of G2 may be searched, and the distance image D2 may be generated based on the searched corresponding point.

  In the fourth embodiment, the distance image D2 or D3 is generated in step ST106 by the same process as in the first embodiment. At this time, the first distance image D1 is also generated. You may make it record on the recording medium 29. FIG. Alternatively, the second distance image D2 may be generated by searching for the corresponding point without setting the corresponding point search range using the distance image D1.

  By the way, the distance images D2 and D3 generated as described above may cause erroneous pixel correspondence due to problems inherent in stereo matching. The cause of pixel miscorrespondence is that the imaging units 2B and 2C are capturing the same subject at different positions, and thus can be viewed from the imaging unit 2B, but cannot be viewed from the imaging unit 2C. For example, a hidden point may be generated, or a plurality of corresponding points may be generated when the subject has a simple color. Thus, when an erroneous correspondence occurs, the distance image D2 or D3 cannot be generated with high accuracy. Hereinafter, an embodiment for solving this problem will be described as a fifth embodiment.

  FIG. 24 is a schematic block diagram showing the internal configuration of the stereoscopic imaging apparatus according to the fifth embodiment. As shown in FIG. 24, the stereoscopic imaging apparatus 1D according to the fifth embodiment is different from the first embodiment in that a pixel value changing unit 40 is provided. The function of the pixel value changing unit 40 will be described in the process performed in the following fifth embodiment.

  FIG. 25 is a flowchart showing processing performed in the fifth embodiment. Note that in the fifth embodiment, the processing up to the generation of the distance image D2 or the distance image D3 is the same as the processing in steps ST1 to ST9 in the first embodiment, so here the first embodiment. Only the processes after step ST9 will be described.

  Following the processing in step ST9, the pixel value changing unit 40 calculates the absolute value of the difference value between the pixel values of the corresponding pixels in the distance image D1 and the distance image D2 or D3 (step ST121). Then, it is determined whether or not there is a pixel whose absolute value of the calculated difference value is equal to or greater than a predetermined threshold value Th1 (step ST122).

  If step ST122 is negative, the base image G1, the reference image G2, and the distance image D2 or D3 are recorded on the recording medium 29, assuming that there is no pixel miscorrespondence to the distance image D2 or the distance image D3 (image recording: Step ST123), the process ends.

  On the other hand, when step ST122 is affirmed, the pixel value changing process is performed with the pixel having the absolute value of the difference value equal to or greater than the threshold Th1 as an erroneously-corresponding pixel (step ST124). Here, the pixel value changing process includes a process of deleting the pixel value of the erroneously corresponding pixel, a process of replacing the pixel value of the erroneously corresponding pixel with the pixel value of the corresponding pixel of the distance image D1, and pixels around the erroneously corresponding pixel. Any one of the processes of interpolating the pixel value of the erroneously-corresponding pixel by the pixel value is performed.

  Then, in response to an instruction from the CPU 33, the media control unit 26 records the distance image (D4) on which the pixel value changing process has been performed, on the recording medium 29 together with the reference image G1 and the reference image G3 (step ST125), and performs the processing. finish.

  As described above, in the fifth embodiment, the absolute value of the difference value between the corresponding pixels in the distance image D1 and the distance image D2 or D3 is calculated, and the absolute value of the difference value exceeds the threshold value Th1. Since the miscorresponding pixel is detected and the pixel value of the miscorresponding pixel is changed to a predetermined pixel value, the pixel value of the miscorresponding pixel can be corrected and the distance image D4 can be generated with high accuracy.

  In the fifth embodiment, the pixel value changing process is performed after the process of step ST9 in the first embodiment. However, the pixel value changing process is also performed in the second to fourth embodiments. Of course you can.

  In the first to fifth embodiments, only one third imaging unit 2C for obtaining the reference image G2 is provided. However, a plurality of third imaging units 2C are provided, and a plurality of references are provided. An image may be acquired to generate the second distance image D2.

  The stereoscopic imaging apparatuses 1, 1A, 1B, 1C, and 1D according to the embodiments of the present invention have been described above. However, the computer includes the first distance image generation unit 30, the stereo matching unit 31, and the second distance image. 10, 14, 15, 18, 19, 21, 25 are made to function as means corresponding to the generation unit 32, the mount drive control unit 37, the zoom magnification change unit 38, the face detection unit 39, and the pixel value change unit 40. A program for performing such processing is also one embodiment of the present invention. A computer-readable recording medium in which such a program is recorded is also one embodiment of the present invention.

1 is a front perspective view showing an external configuration of a stereoscopic imaging apparatus according to a first embodiment of the present invention. 1 is a rear perspective view showing an external configuration of a stereoscopic imaging apparatus according to a first embodiment of the present invention. The figure which shows the imaging range of the 1st to 3rd imaging part The figure which shows the internal structure of a 1st imaging part. The figure which shows the internal structure of the 2nd and 3rd imaging part. 1 is a schematic block diagram showing an internal configuration of a stereoscopic imaging device according to a first embodiment. Diagram for explaining stereo matching The figure for demonstrating the relationship between the 2nd and 3rd imaging part. Diagram for explaining setting of search range The flowchart which shows the process performed in 1st Embodiment. Front perspective view showing the external structure of a stereoscopic imaging apparatus according to a second embodiment of the present invention (No. 1) Front perspective view showing the external structure of a stereoscopic imaging apparatus according to a second embodiment of the present invention (No. 2) Schematic block diagram showing the internal configuration of the stereoscopic imaging apparatus according to the second embodiment The flowchart which shows the process performed in 2nd Embodiment. The flowchart which shows the baseline length change process in 2nd Embodiment Diagram for explaining the limitation range of baseline length change The schematic block diagram which shows the internal structure of the stereo imaging device by 3rd Embodiment. The flowchart which shows the process performed in 3rd Embodiment The flowchart which shows the zoom magnification change process in 3rd Embodiment. Schematic block diagram showing the internal configuration of a stereoscopic imaging apparatus according to the fourth embodiment The flowchart which shows the process performed in 4th Embodiment A diagram for explaining the correspondence between face areas The figure for demonstrating calculation of the parallax in 4th Embodiment. Schematic block diagram showing the internal configuration of a stereoscopic imaging apparatus according to the fifth embodiment The flowchart which shows the process performed in 5th Embodiment

Explanation of symbols

1, 1A, 1B, 1C, 1D Stereoscopic imaging device 2A, 2B, 2C Imaging unit 3 Ranging light irradiation unit 4 Release button 5 Monitor 13A, 13B, 13C CCD
30 1st distance image generation part 31 Stereo matching part 32 2nd distance image generation part 33 CPU
36 Drive mechanism 37 Mount drive control unit 38 Zoom magnification change unit 39 Face detection unit 40 Pixel value change unit

Claims (4)

First imaging means for irradiating a subject with distance measuring light, imaging reflected light of the distance measuring light from the subject, and obtaining data for a distance image representing the three-dimensional shape of the subject;
First distance image generation means for generating a first distance image representing the three-dimensional shape of the subject based on the distance image data;
Second imaging means for acquiring a reference image of the subject for calculating a second distance image by imaging the subject;
At least one third imaging means for acquiring a reference image of the subject for calculation of the second distance image by imaging the subject from a position different from the second imaging means;
Corresponding point search means for searching for corresponding points of pixels between the reference image and the reference image;
Second distance image generation means for generating the second distance image representing the three-dimensional shape of the subject based on the searched corresponding points;
Before imaging by the first imaging unit, the second and third imaging units perform imaging, a predetermined subject is detected from an image acquired by the imaging, and a detection result of the predetermined subject is output Subject detection means;
When the predetermined subject is detected, only the second and third imaging means perform imaging, and only the second distance image generating means generates the second distance image. A stereoscopic imaging apparatus further comprising second and third imaging means, the corresponding point searching means, and a control means for controlling the second distance image generating means.   The corresponding point search means is a means for setting, when the predetermined subject is detected, a distance range including the predetermined subject in the reference image and the reference image as the corresponding point search range. Item 3. The stereoscopic imaging device according to Item 1. First imaging means for irradiating a subject with distance measuring light, imaging reflected light of the distance measuring light from the subject, and obtaining data for a distance image representing the three-dimensional shape of the subject;
First distance image generation means for generating a first distance image representing the three-dimensional shape of the subject based on the distance image data;
Second imaging means for acquiring a reference image of the subject for calculating a second distance image by imaging the subject;
At least one third imaging means for acquiring a reference image of the subject for calculation of the second distance image by imaging the subject from a position different from the second imaging means;
Corresponding point search means for searching for corresponding points of pixels between the reference image and the reference image;
A control method for a stereoscopic imaging apparatus, comprising: a second distance image generation unit configured to generate the second distance image representing the stereoscopic shape of the subject based on the searched corresponding points;
Prior to imaging by the first imaging means, the second and third imaging means perform imaging,
Detecting a predetermined subject from an image acquired by the imaging and outputting a detection result of the predetermined subject;
When the predetermined subject is detected, only the second and third imaging means perform imaging,
A control method for a stereoscopic imaging apparatus, wherein only the second distance image generation means generates the second distance image. First imaging means for irradiating a subject with distance measuring light, imaging reflected light of the distance measuring light from the subject, and obtaining data for a distance image representing the three-dimensional shape of the subject;
First distance image generation means for generating a first distance image representing the three-dimensional shape of the subject based on the distance image data;
Second imaging means for acquiring a reference image of the subject for calculating a second distance image by imaging the subject;
At least one third imaging means for acquiring a reference image of the subject for calculation of the second distance image by imaging the subject from a position different from the second imaging means;
Corresponding point search means for searching for corresponding points of pixels between the reference image and the reference image;
A program for causing a computer to execute a control method of a stereoscopic imaging apparatus including second distance image generation means for generating the second distance image representing the stereoscopic shape of the subject based on the searched corresponding points. There,
A procedure for causing the second and third imaging means to perform imaging before imaging by the first imaging means;
A procedure for detecting a predetermined subject from an image acquired by the imaging and outputting a detection result of the predetermined subject;
A procedure for causing only the second and third imaging means to perform imaging when the predetermined subject is detected;
A program that causes only the second distance image generating means to generate the second distance image.

Images