JP6957162B2 - Distance measuring device and moving object - Google Patents

Description

The present invention relates to a distance measuring device and a moving body capable of performing high-precision distance measurement together with image acquisition, and more particularly to a distance measuring device and a moving body that measures a distance by an imaging surface phase difference method.

In a moving body including a vehicle, a drone or a robot, in order to support the movement of the moving body, it is required to measure the distance between an obstacle and the moving body around the moving body, for example. In particular, in order to perform movement support such as collision avoidance and tracking of other objects, recognition processing using an image of an obstacle is also required. Therefore, in a moving body, it is used as an imaging device to measure the distance to the obstacle. Cameras are heavily used. In a camera, an imaging surface phase difference method is known as a method of acquiring not only an image of an obstacle (subject) used for recognition processing but also a distance (see, for example, Patent Documents 1 and 2). In the imaging surface phase difference method, the parallax of a pair of images formed by the light flux passing through two different regions (partial pupils) in the exit pupil of the optical system of the camera is obtained, and the subject is obtained from the parallax based on the principle of triangulation. Measure the distance to. In a camera using the image pickup surface phase difference method, for example, each pixel of the image pickup device has two photoelectric conversion units (photodiodes). In the imaging surface phase difference method, the parallax of a pair of images is obtained from an electrical signal (hereinafter referred to as "image signal") in which an optical image formed by light flux incident on two photoelectric conversion units is converted, while two The image signals obtained from the photoelectric conversion unit are added up to acquire an image of the subject.

By the way, so-called still cameras and video cameras are required to have high-quality images. Each of the two luminous fluxes passing through the corresponding two partial pupils is incident on the two photoelectric conversion units in the pixel. Therefore, in order to obtain a high-quality image, an image obtained by enlarging the two partial pupils in the exit pupil and adding the image signals obtained from the two photoelectric conversion units is formed by the luminous flux passing through the entire exit pupil. It is necessary to set it so that it is the same as the image.

Japanese Unexamined Patent Publication No. 2013-190622 Japanese Unexamined Patent Publication No. 2001-21792

However, as shown in FIG. 16, when the two partial pupils 160 and 161 are enlarged, the baseline length, which is the distance between the centers of gravity of the two partial pupils 160 and 161, becomes shorter, and the image signal 162 obtained from the two photoelectric conversion units becomes shorter. , 163 reduces the parallax of the pair of images. As a result, in the imaging surface phase difference method in which the distance from the parallax to the subject is measured based on the principle of triangulation, the accuracy of the measured distance may decrease.

An object of the present invention is to provide a distance measuring device capable of acquiring an image and measuring a distance with high accuracy.

In order to achieve the above object, the distance measuring device of the present invention has an optical system having a diaphragm, an image pickup device having an image pickup element in which a plurality of pixels are arranged, and a parallax output from each of the plurality of pixels. A distance measuring device including a distance information acquisition unit that acquires distance information of a subject based on an image signal of a pair of optical images, and the aperture is defined with first and second distance measuring light beams. A first diaphragm hole and a second diaphragm hole for imaging are provided, and a third diaphragm hole for defining an imaging light beam is provided, and each of the plurality of pixels receives the first ranging light beam and receives the pair. a first photoelectric conversion unit that outputs one image signal of the optical image of a second photoelectric conversion unit in which the second by receiving the distance measuring light beam and outputs the other of the image signal of the optical image of said pair , by receiving the light beam for the imaging have a third photoelectric conversion unit to output an image signal of an optical image of a subject, the third photoelectric conversion unit, with respect to the direction of the parallax, the first photoelectric conversion unit and characterized Rukoto sandwiched between the second photoelectric conversion unit.

Further, in order to achieve the above object, the moving body of the present invention includes a distance measuring device and an alarm device that issues an alarm based on the distance measuring result of the distance measuring device, and the distance measuring device has a diaphragm. The distance information of the subject is acquired based on the image signal of the optical system having the optical system, the image pickup device having the image pickup element in which a plurality of pixels are arranged, and the image signal of a pair of optical images having a difference in output from each of the plurality of pixels. The diaphragm has a first diaphragm hole and a second diaphragm hole that define the first and second ranging light beams, and a third diaphragm hole that defines the imaging light beam. Each of the plurality of pixels is provided with a first photoelectric conversion unit that receives the first diaphragm-finding light beam and outputs one image signal of the pair of optical images, and the second measurement. The second photoelectric conversion unit that receives the light beam for distance and outputs the other image signal of the pair of optical images, and the third photoelectric conversion unit that receives the light beam for imaging and outputs the image signal of the optical image of the subject. The third photoelectric conversion unit is sandwiched between the first photoelectric conversion unit and the second photoelectric conversion unit in the direction of the diaphragm .

In addition, other aspects of the present invention will be clarified in each embodiment described below.

According to the present invention, it is possible to provide a distance measuring device capable of acquiring an image and measuring a distance with high accuracy.

It is a block diagram which shows schematic structure of the distance measuring apparatus which concerns on 1st Embodiment of this invention. It is a front view which shows the structure of the image sensor in FIG. 1 schematically. It is a figure for demonstrating the principle of the distance measurement by the imaging surface phase difference method. It is a figure which shows the exit pupil of the optical system in 1st Embodiment of this invention. It is a figure for demonstrating the arrangement of each PD in a pixel for image pickup and distance measurement. It is a figure for demonstrating the modification of the exit pupil of an optical system, and the modification of the arrangement of each PD in the pixel for imaging and distance measurement. It is a figure for demonstrating the state of the diaphragm of the luminous flux for distance measurement in the 2nd Embodiment of this invention. It is a figure for demonstrating the state of the diaphragm of the luminous flux for distance measurement in the 2nd Embodiment of this invention. It is a block diagram which shows schematic structure of the distance measuring apparatus which concerns on 2nd Embodiment of this invention. It is a figure for demonstrating the limitation of the incident angle range of the luminous flux for distance measurement by a diaphragm hole. It is a figure for demonstrating the limitation of the incident angle range of the luminous flux for distance measurement by a diaphragm hole. 9 is a view of the diaphragm in FIG. 9 viewed from the direction of the optical axis. It is a figure for demonstrating the configuration of the driving support system in the automobile as a moving body which concerns on this Embodiment. It is a figure for demonstrating the configuration of the driving support system in the automobile as a moving body which concerns on this Embodiment. It is a flowchart which shows the collision avoidance processing as an operation example of the driving support system in this embodiment. It is a figure for demonstrating the relationship between each partial pupil and the baseline length in the exit pupil of an optical system.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. The components described in the present embodiment are merely examples, and the scope of the present invention is not limited by the components described in the present embodiment.

First, the first embodiment of the present invention will be described. FIG. 1 is a block diagram schematically showing a configuration of a distance measuring device according to a first embodiment of the present invention.

In FIG. 1, the distance measuring device includes a camera 10 having an optical system 11 and an image sensor 12 in which a large number of pixels are arranged, an image analysis unit 13, and a distance information acquisition unit 14. The optical system 11 has, for example, two lenses 11a and 11b arranged along the optical axis, and forms an optical image of a subject on the image pickup device 12. As shown in FIG. 2, a large number of pixels included in the image pickup device 12 are divided into a plurality of imaging pixels 12a and a plurality of distance measuring pixels 12b (first pixel, second pixel). In addition, in order to eliminate the complexity, FIG. 2 only partially shows the mode of arrangement of each pixel on the upper left side, and the mode of arrangement of each pixel on the entire surface of the image sensor 12 is not shown. Each imaging pixel 12a and each distance measuring pixel 12b each has a photoelectric conversion element as a photoelectric conversion unit, for example, a photodiode (hereinafter, referred to as “PD”). In the present embodiment, the PD is composed of a photoelectric conversion film or the like formed of silicon or a thin film having light absorption characteristics. Each of the plurality of imaging pixels 12a receives a light flux passing through a partial region of the exit pupil of the optical system 11 (a different region in the exit pupil; hereinafter referred to as “partial pupil”) to form an optical image of the subject. .. Further, each of the plurality of distance measuring pixels 12b receives a light flux passing through two different partial pupils in the exit pupil of the optical system 11. In the image sensor 12, for example, according to the Bayer arrangement, imaging pixels 12a having a spectral sensitivity of G (green) are arranged as two diagonal pixels out of four pixels in 2 rows × 2 columns, and R as the other 2 pixels. Image pickup pixels 12a having spectral sensitivities of (red) and B (blue) are arranged one by one. The spectral sensitivity of a specific color possessed by each imaging pixel 12a is imparted by a primary color system color filter possessed by each imaging pixel 12a. Further, in the image sensor 12, in some pixels of 2 rows × 2 columns, the imaging pixels 12a having two diagonal spectral sensitivities of G are left as they are, and the imaging pixels 12a having the spectral sensitivities of R and B are left as they are. Is replaced with the distance measuring pixel 12b. In the image sensor 12, two diagonal distance measuring pixels 12b form a pair of optical images by receiving light flux passing through the two partial pupils in some of the pixels of 2 rows × 2 columns. Further, each optical image is photoelectrically converted to output an image signal (output signal). The image analysis unit 13 performs image processing on the output image signal. The distance information acquisition unit 14 calculates the parallax of the pair of optical images from the image signal subjected to the image processing, and further calculates the distance to the subject based on the calculated parallax. That is, the distance measuring device measures the distance to the subject by the imaging surface phase difference method. When calculating the parallax from the image signal, it is not always necessary to perform image processing on the image signal output from the image sensor 12. For example, when the parallax can be calculated directly from the image signal, the image sensor 12 outputs the image signal directly to the distance information acquisition unit 14 without going through the image analysis unit 13.

In the image sensor 12 described above, the two distance measuring pixels 12b receive the light flux passing through each partial pupil to form a pair of optical images, but one imaging and distance measuring pixel 12c described later is two. A pair of optical images may be formed by receiving the light flux passing through the partial pupil. In this case, the imaging / ranging pixel 12c has at least two PDs, and each of the two PDs receives the light flux that has passed through the two partial pupils. Further, the imaging / ranging pixel 12c synthesizes the luminous flux passing through the two partial pupils received by the two PDs to form an optical image of the subject. Therefore, the imaging and ranging pixels 12c may be arranged in almost the entire area of the image sensor 12. In the present embodiment, the lenses 11a and 11b of the optical system 11 are fixed in the camera 10, and the so-called autofocus function is omitted. That is, the focus is fixed on the camera 10. However, the camera 10 may have an autofocus function, and in this case, the autofocus is executed based on the distance to the subject calculated by the distance information acquisition unit 14.

FIG. 3 is a diagram for explaining the principle of distance measurement by the imaging surface phase difference method. Specifically, FIG. 3A shows a case where distance measurement is performed using an optical image formed by a pair of two distance measuring pixels 12b. FIG. 3B shows a case where each pixel in FIG. 3A is viewed from the direction of the optical axis. FIG. 3C shows a case where distance measurement is performed using a pair of optical images formed by one imaging / ranging pixel 12c. In addition, in FIGS. 3A and 3C, the image pickup pixel 12a, each distance measurement pixel 12b, and the image pickup / distance measurement pixel 12c are drawn in a state of being viewed from the side.

First, in FIG. 3A, the exit pupils 30 of the optical system 11 are located near both ends of the exit pupils 30 in the horizontal direction (horizontal direction, first direction in the figure) (hereinafter, referred to as “parallax direction”). It has two partial pupils (hereinafter referred to as "distance measuring pupils") 31 and 32 that are located. Further, the exit pupil 30 has a partial pupil (hereinafter, referred to as “imaging pupil”) 33 which is sandwiched between the distance measuring pupils 31 and 32 in the parallax direction and is located substantially at the center of the exit pupil 30. Luminous fluxes 34, 35 (first luminous flux, second luminous flux) are emitted from each of the distance measuring pupils 31 and 32 and are incident on each of the pair of distance measuring pixels 12b. Further, the image pickup light flux 36 is emitted from the image pickup pupil 33 and is incident on the image pickup pixel 12a. The two distance measuring pixels 12b are a microlens 37 (first microlens, second microlens) and PD38 (first photoelectric conversion unit, second photoelectric conversion unit) facing the exit pupil 30 via the microlens 37. ) And. Further, the two ranging pixels 12b are arranged between the microlens 37 and the PD38 and have an opening 39 (first opening, second opening) that partially confronts the microlens 37. It has a first light-shielding film and a second light-shielding film). The aperture 39 (first aperture) of the imaging pixel 12b on the left side of FIG. 3B is arranged eccentrically in the parallax direction (first direction) from the center of the pixel or the optical axis of the microlens 37. Further, the opening 39 (second opening) of the imaging pixel 12b on the right side of FIG. 3B is a parallax direction (second direction opposite to the first direction) from the center of the pixel or the optical axis of the microlens 37. ) Is eccentric. Further, the PD38 of the imaging pixel 12b on the left side of FIG. 3B and the PD38 of the imaging pixel 12b on the right side of FIG. 3B have different incident angle sensitivity characteristics. The scope of the claims and the center of the pixel in the present specification are the center of gravity of the pixel. For example, when the pixel is rectangular, the intersection of the two diagonal lines corresponds to the center of the pixel. The imaging pixel 12a has a microlens 41 and a PD 42 facing the exit pupil 30 via the microlens 41. Further, the imaging pixel 12a has a light-shielding film 66 (third light-shielding film) arranged between the microlens 41 and the PD42 and having an opening 65 (third aperture) that partially confronts the microlens 41. The aperture 65 (third aperture) of the imaging pixel 12a is arranged on the center of the pixel or on the optical axis of the microlens 41. The PD42 of the imaging pixel 12a has different incident angle sensitivity characteristics from the PD38 of the two imaging pixels 12b.

In the pair of ranging pixels 12b, two microlenses 37 are arranged near the image plane of the optical system 11, and each microlens 37 transfers the ranging light fluxes 34 and 35 to the two light-shielding films 40 (each opening 39). Condensing. The optical system 11 and the two microlenses 37 are configured such that the exit pupil 30 and the two light-shielding films 40 (each opening 39) are optically conjugated. Therefore, the shapes of the openings 39 of the two light-shielding films 40 are projected by the two microlenses 37 onto the two distance measuring pupils 31 and 32 in the exit pupil 30. That is, the arrangement (position, size) of the two distance measuring pupils 31 and 32 is realized by the position and size of the respective openings 39 of the light-shielding film 40. Further, in the imaging pixel 12a, the microlens 41 is arranged near the image plane of the optical system 11, so that the exit pupil 30 and the light-shielding film 66 (aperture 65) of the optical system 11 and the microlens 41 are optically coupled. It is composed of. Therefore, the shape of the aperture 65 is projected by the microlens 41 onto the imaging pupil 33 in the exit pupil 30. That is, the arrangement (position, size) of the imaging pupil 33 is realized by the position and size of the opening 65 of the light-shielding film 66. The two PD38s of the pair of distance measuring pixels 12b output an image signal in which the optical image formed on each microlens 37 is converted by the distance measuring fluxes 34 and 35 passing through the distance measuring pupils 31 and 32, respectively. do. The parallax of a pair of optical images is calculated by performing image shift detection calculation processing (correlation processing, phase difference detection processing) or the like on the output image signal. Further, the defocus amount and the distance to the subject are calculated from the parallax based on the principle of triangulation (see, for example, US Patent Application Publication No. 2015/0092988). Further, the PD 42 of the imaging pixel 12a outputs an image signal obtained by converting an optical image formed on the microlens 41 by the imaging light flux 36 passing through the imaging pupil 33, and an image of the subject is formed from the image signal. Will be done. Although the light-shielding film 40 and the light-shielding film 66 are provided in FIGS. 3 (A) and 3 (B), these light-shielding films may be omitted. In this case, by making the positions and sizes of the PD 38 and PD 42 the same as the positions and sizes of the openings 39 and 65, the arrangement of the distance measuring pupils 31 and 32 and the arrangement of the imaging pupil 33 described above are realized. be able to.

Further, in FIG. 3C, the ranging luminous fluxes 34 and 35 are emitted from each of the ranging pupils 31 and 32 and incident on the imaging and ranging pixels 12c. An imaging light beam 36 is emitted from the imaging pupil 33 and is incident on the same imaging / ranging pixel 12c. The imaging / ranging pixel 12c has a microlens 43 and PDs 44 to 46 facing the exit pupil 30 via the microlens 43. In the imaging / ranging pixel 12c, the microlens 43 is arranged near the image plane of the optical system 11, and the microlens 43 focuses the ranging light fluxes 34 and 35 on the two PDs 44 and 45. The optical system 11 and the microlens 43 are configured such that the exit pupil 30 and PDs 44 to 46 are optically conjugated. Therefore, the microlens 43 projects the shape of the PD 44 onto the distance measuring pupil 31 at the exit pupil 30. Further, the microlens 43 projects the shape of the PD 45 onto the distance measuring pupil 32 in the exit pupil 30. Further, the microlens 43 projects the shape of the PD 46 onto the imaging pupil 33 in the exit pupil 30. That is, the arrangement (position, size) of the distance measuring pupils 31 and 32 and the imaging pupil 33 is realized by the positions and sizes of PDs 44 to 46. The PDs 44 and 45 of the imaging and ranging pixels 12c output an image signal obtained by converting an optical image formed on the microlens 43 by the ranging luminous fluxes 34 and 35 passing through the ranging pupils 31 and 32, respectively. .. Here, too, the parallax of the pair of optical images is calculated by performing image shift detection calculation processing (correlation processing, phase difference detection processing) or the like on the output image signal. Further, the defocus amount and the distance to the subject are calculated from the parallax based on the principle of triangulation. Further, the PD 46 of the imaging / ranging pixel 12c outputs an image signal in which an optical image formed on the microlens 43 is converted by an imaging luminous flux 36 passing through the imaging pupil 33. The PDs 44 to 46 have different incident angle sensitivity characteristics.

FIG. 4 is a diagram showing exit pupils of the optical system according to the first embodiment of the present invention. The horizontal direction in the figure corresponds to the parallax direction.

In FIG. 4A, the exit pupil 30 has two elliptical shapes that are symmetrically located in the parallax direction with respect to the center of the exit pupil 30 (the optical axis of the optical system 11) and are located near both ends of the exit pupil 30. It has parallax eyes 31 and 32. Further, the exit pupil 30 has a perfect circular imaging pupil 33 that is sandwiched between the distance measuring pupils 31 and 32 in the parallax direction and is located substantially in the center of the exit pupil 30. The ratio of the distance L between the centers of gravity with respect to the parallax directions of the distance measuring pupils 31 and 32 to the exit pupil width W, which is the length of the exit pupil 30 with respect to the parallax direction, is 0.6 or more and 0.9 or less. Further, the ratio of the distance measuring pupil widths Wa and Wb (partial pupil width) to the exit pupil width W, which are the lengths of the distance measuring pupils 31 and 32 in the parallax direction, is 0.1 or more and 0.4 or less. be. Further, each of the elliptical distance measuring pupils 31 and 32 has a long side in the vertical direction (direction perpendicular to the parallax direction) in the figure. The ratio of the distance measurement pupil height H (partial pupil height), which is the length of each distance measurement pupil 31 and 32 in the vertical direction in the figure, to the distance measurement pupil widths Wa and Wb (hereinafter, “aspect ratio””. ) Is 1 or more, preferably 2 or more. The sizes and shapes of the distance measuring pupils 31 and 32 are arbitrary as long as they comply with the above-mentioned restrictions on the distance measuring pupil widths Wa and Wb and the distance measuring pupil height H. For example, as shown in FIG. 4 (B). In addition, the distance measuring pupils 31 and 32 may be slightly smaller. Further, as shown in FIG. 4C, the distance measuring pupils 31 and 32 and the imaging pupil 33 may have the same shape (round shape). However, the size of the distance measuring pupils 31 and 32 is to acquire the distance information of the subject whose intensity of each image signal is accurate based on the distance measuring fluxes 34 and 35 passing through the distance measuring pupils 31 and 32, respectively. It is necessary that the size is as high as possible.

According to the present embodiment, the ratio of the distance L (baseline length) between the centers of gravity with respect to the parallax direction of each distance measuring pupils 31 and 32 to the exit pupil width W of the exit pupil 30 is 0.6 or more and 0.9 or less. Therefore, the baseline length can be increased. As a result, the accuracy of the distance to the subject measured by using the imaging surface phase difference method can be improved. Further, since the ratio of the distance measuring pupil widths Wa and Wb to the exit pupil width W is 0.1 or more and 0.4 or less, the arrangement of the distance measuring pupils 31 and 32 with respect to the parallax direction of the exit pupil 30 is arranged. You can increase the degree of freedom. As a result, the distance measuring pupils 31 and 32 can be positioned near both ends of the exit pupil 30 in the parallax direction, so that the baseline length can be reliably increased. Further, if the pupil widths Wa and Wb for distance measurement are too small, the amount of light of the luminous fluxes 34 and 35 for distance measurement is significantly reduced, and the S / N ratio of the obtained image signal for distance measurement is reduced for measurement. Distance accuracy is reduced. However, as described above, since the ratio of the distance measuring pupil widths Wa and Wb to the exit pupil width W is 0.1 or more, it is necessary to prevent the light amount of the distance measuring fluxes 34 and 35 from being significantly reduced. Can be done. Further, when the distance measuring pupil widths Wa and Wb are increased, that is, when the distance measuring pupils 31 and 32 are increased, the baseline length is shortened and the accuracy of the measured distance is lowered. However, as described above, since the ratio of the distance measuring pupil widths Wa and Wb to the exit pupil width W is 0.4 or less, it is possible to prevent the baseline length from becoming short. Further, since the aspect ratios of the distance measuring pupils 31 and 32 are 1 or more, the amount of the distance measuring fluxes 34 and 35 passing through the distance measuring pupils 31 and 32 can be increased. As a result, the S / N ratio of the image signal obtained from the optical images formed by the ranging light fluxes 34 and 35 can be increased, and the distance information of the subject can be obtained with high accuracy. Further, the exit pupil 30 has an imaging pupil 33 sandwiched between the distance measuring pupils 31 and 32 in the parallax direction, and the optical image formed by the imaging luminous flux 36 passing through the imaging pupil 33 is converted. An image of the subject is formed from the image signal. As a result, the aperture can be reduced and the depth of focus of the subject can be deepened as compared with the case where the light flux passing through the entire region of the exit pupil 30 is used, and an image of the subject suitable for the recognition process can be obtained.

As described above, the arrangement of the two distance measuring pupils 31 and 32 is realized by the position and size of the respective openings 39 of the two light shielding films 40 in the two distance measuring pixels 12b. Further, the arrangement of the imaging pupil 33 is realized by the position and size of the opening 65 of the light-shielding film 66 in the imaging pixel 12a. Alternatively, the arrangement of the two ranging pupils 31 and 32 and the imaging pupil 33 is realized by the positions and sizes of PDs 44 to 46 in the imaging and ranging pixels 12c. Therefore, in the image pickup / distance measurement pixel 12c, as shown in FIG. 5A, the two PDs 44 and 45 are in the vertical direction (parallax direction) in the figure so as to correspond to the vertically long distance measurement pupils 31 and 32. It has a rectangular shape with long sides in the vertical direction). Further, the two PDs 44 and 45 are arranged apart from each other in the parallax direction so as to correspond to the two distance measuring pupils 31 and 32 located near both ends of the exit pupil 30. Further, the PD 46 has a square shape so as to correspond to the perfect circular imaging pupil 33, and the PD 46 has an imaging and ranging pixel 12c so as to correspond to the imaging pupil 33 located substantially in the center of the exit pupil 30. It is placed approximately in the center. As shown in FIG. 5A, when one imaging / ranging pixel 12c has two PDs 44 and 45, for calculating the parallax of a pair of optical images from one imaging / ranging pixel 12c. Since the image signal is obtained, many of the image signals can be obtained. That is, the resolution of the image signal can be increased. As a result, the formed image can be made high quality. Moreover, the resolution of the distance information can be increased.

The imaging / ranging pixel 12c may have only one of PD44 and PD45. For example, as shown in FIG. 5 (B), one imaging / ranging pixel 12c (lower in the figure) has PD44, and the other imaging / ranging pixel 12c (upper in the figure) has PD45. Has. In this case, the ranging light flux 34 passing through the ranging pupil 31 is received by the PD44 of one imaging and ranging pixel 12c, and the PD44 is converted into an optical image formed by the ranging light flux 34. Output the image signal. Further, the ranging light flux 35 passing through the ranging pupil 32 is received by the PD45 of another imaging / ranging pixel 12c, and the PD45 is an image obtained by converting the optical image formed by the ranging light flux 35. Output a signal. Further, the parallax of the pair of optical images is calculated from the image signals output from the PD44 of one imaging / ranging pixel 12c and the PD45 of the other imaging / ranging pixel 12c. As shown in FIG. 5B, when the imaging / ranging pixel 12c has only one of PD44 and PD45, the number of PDs possessed by one imaging / ranging pixel 12c can be reduced to two. can. As a result, there is a margin in the arrangement of each PD, and each PD can be enlarged. As a result, the amount of light received by each PD can be increased to improve the sensitivity of each PD, the formed image can be made high quality even in an environment where the amount of light is insufficient, and the distance can be calculated. The accuracy can be improved.

In the present embodiment, the exit pupils 30 have the distance measuring pupils 31 and 32 arranged in the horizontal direction in the drawing, but further have two distance measuring pupils arranged in the vertical direction in the drawing. You may be. For example, as shown in FIG. 6A, the exit pupil 30 has two elliptical distance measuring pupils 47 and 48 arranged in the vertical direction in the figure in addition to the distance measuring pupils 31 and 32. The two distance measuring pupils 47 and 48 are located near both ends of the exit pupil 30 in the vertical direction in the drawing. As a result, the parallax of the pair of optical images can be calculated not only in the horizontal direction in the drawing but also in the vertical direction in the drawing, and the accuracy of measuring the distance to the horizontal line or the diagonal line in the subject can be improved. In this case, as shown in FIG. 6B, the imaging / ranging pixel 12c has a rectangular shape having a long side in the horizontal direction in the drawing so as to correspond to the two elliptical ranging pupils 47 and 48. Has PD49,50 exhibiting. Further, the two PD49s and 50s are arranged apart from each other in the vertical direction in the drawing so as to correspond to the two distance measuring pupils 47 and 48 located near both ends in the vertical direction in the drawing of the exit pupil 30. ..

Further, in the present embodiment, the imaging pixel 12a has a primary color system color filter, but the color filter may be a complementary color system instead of the primary color system. Since the complementary color filter has a larger amount of light to be transmitted than the primary color filter, the sensitivity of the PD 42 can be improved. On the other hand, the luminous flux received by the PD 38 in the distance measuring pixel 12b is limited to the luminous flux passing through the aperture 39, and the sizes of the PDs 44 and 45 are limited in the imaging / ranging pixel 12c. However, the distance measuring pixel 12b and the imaging / distance measuring pixel 12c do not have a color filter or have a complementary color filter. As a result, the amount of light flux received by the PD38 or PD44, 45 is not so limited. Therefore, the sensitivity of PD38 and PD44,45 does not decrease significantly.

Next, a second embodiment of the present invention will be described. Since the configuration and operation of the second embodiment of the present invention are basically the same as those of the first embodiment described above, the description of the overlapping configuration and operation is omitted, and different configurations and actions are described below. The action will be explained.

When the distance measuring device is applied to a moving body, for example, an automobile or a drone, the distance measuring device is preferably smaller and lighter, and the camera 10 is also required to be further miniaturized accordingly. At this time, the image sensor 12 is also made smaller, and the imaging pixels 12a and the distance measuring pixels 12b of the image sensor 12 are also made smaller. Here, when the distance measuring pixel 12b is miniaturized, for example, when the size (pixel size) of the pixel becomes several times the wavelength of visible light, the distance measuring luminous flux 34, 35 incident on the microlens 37 The spread due to the diffraction of the lens becomes large. Specifically, as shown in FIG. 7A, the luminous flux 35 for ranging when the luminous flux 35 incident on the microlens 37 has a large pixel size and almost no diffraction occurs due to diffraction. It spreads more than (see the luminous flux shown by the solid line in the figure) (see the luminous flux shown by the broken line in the figure). As a result, the amount of light of the light beam passing through the opening 39 decreases, the amount of light received by the distance measuring light beam 35 received by the PD 38 decreases, and the sensitivity of the PD 38 to the distance measuring light beam 35 decreases. As a result, the incident angle sensitivity when the distance measuring luminous flux 35 is diffracted and expanded as compared with the incident angle sensitivity characteristic in the ideal state when the pixel size is large and almost no diffraction occurs (see the characteristic shown by the solid line in the figure). The absolute value of sensitivity in the characteristics (see the characteristics shown by the broken line in the figure) becomes smaller. Then, the SNR (S / N ratio) of the obtained image signal is lowered, and the distance measurement accuracy is lowered. Further, when the distance measuring light flux 35 received by the PD 38 is used for forming an image of a subject, the image quality of the formed image is deteriorated, and the recognition accuracy in the recognition process is lowered. In the figure (graph) showing the incident angle sensitivity characteristic, the vertical axis indicates the sensitivity and the horizontal axis indicates the incident angle. The angle of incidence at the intersection with the vertical axis on the horizontal axis is the angle of incidence of the main ray of each luminous flux.

In order to improve the sensitivity of the PD 38 to the luminous flux 35 for distance measurement, it is preferable to increase the width Wc of the aperture 39 to increase the amount of light of the luminous flux passing through the aperture 39, as shown in FIG. 7 (B). However, when the width Wc of the opening 39 is increased, the range of the incident angle of the light flux that can pass through the opening 39 also widens (see the light flux shown by the broken line in the figure). As a result, the incident angle of the incident angle sensitivity characteristic (see the characteristic shown by the broken line in the figure) when the width Wc of the opening 39 is enlarged as compared with the incident angle sensitivity characteristic in the ideal state (see the characteristic shown by the solid line in the figure). Since the range is widened, the baseline length is shortened and the distance measurement accuracy is lowered.

Correspondingly, in the present embodiment, as shown in FIG. 7C, the width Wc of the opening 39 is increased in order to improve the sensitivity of the PD 38, and the range of the incident angle of the incident angle sensitivity characteristic is increased. limit. That is, in order to limit the incident angle of the luminous flux passing through the opening 39, the luminous flux 35 for ranging is narrowed down to limit the incident angle distribution (width) of the luminous flux 35 for ranging (the luminous flux shown by the solid line in the figure). reference). Specifically, the aperture hole 52 of the aperture 51 is used to narrow the incident angle range (incident angle distribution on the image plane of the optical system 11) of the luminous flux 35 for distance measurement. At this time, even if the distance measuring light flux 35 that has passed through the diaphragm hole 52 is diffracted by the microlens 37, the distance measuring light beam 35 is an opening having an enlarged width Wc on the light shielding film 40 (on the first light shielding film). It does not spread more than 39. As a result, the PD 38 can sufficiently receive the luminous flux 35 for ranging. By taking the same measures, the PD 38 can sufficiently receive the luminous flux 34 for distance measurement. From the above, it is possible to narrow the range of the incident angle of the incident angle sensitivity characteristic while maintaining the absolute value of the sensitivity of the incident angle sensitivity characteristic high, and it is possible to prevent the ranging accuracy from being lowered.

Further, when the imaging and ranging pixels 12c are miniaturized, for example, when the pixel size is about several times the wavelength of visible light, the spread due to the diffraction of the ranging light fluxes 34 and 35 incident on the microlens 43 is widened. growing. Specifically, as shown in FIG. 8A, the luminous flux 35 for ranging when the luminous flux 35 incident on the microlens 43 has a large pixel size and almost no diffraction occurs due to diffraction. It spreads more than (see the luminous flux shown by the solid line in the figure) (see the luminous flux shown by the broken line in the figure). As a result, the amount of light received by the PD 45 for the distance measuring flux 35 is reduced, and the sensitivity of the PD 45 to the distance measuring flux 35 is reduced. As a result, the incident angle sensitivity when the distance measuring luminous flux 35 is diffracted and expanded as compared with the incident angle sensitivity characteristic in the ideal state when the pixel size is large and almost no diffraction occurs (see the characteristic shown by the solid line in the figure). The absolute value of sensitivity in the characteristics (see the characteristics shown by the broken line in the figure) becomes smaller. Then, the SNR of the obtained image signal is lowered, and the distance measurement accuracy is lowered. Further, when the distance measuring light flux 35 received by the PD 45 is used for forming an image of a subject, the image quality of the formed image is deteriorated, and the recognition accuracy in the recognition process is lowered.

In order to improve the sensitivity of the PD45 to the luminous flux 35 for distance measurement, it is preferable to increase the width Wd of the PD45 to increase the amount of light flux received by the PD45, as shown in FIG. 8B. However, when the width Wd of the PD45 is increased, the range of the incident angle of the luminous flux that can be incident on the PD45 also expands (see the luminous flux indicated by the broken line in the figure). As a result, the range of the incident angle of the incident angle sensitivity characteristic (see the characteristic shown by the broken line in the figure) when the width Wd of the PD45 is enlarged as compared with the incident angle sensitivity characteristic in the ideal state (see the characteristic shown by the solid line in the figure). As a result, the baseline length becomes shorter and the distance measurement accuracy decreases.

Correspondingly, in the present embodiment, as shown in FIG. 8C, the width Wd of the PD45 is increased in order to improve the sensitivity of the PD45, and the range of the incident angle of the incident angle sensitivity characteristic is limited. do. That is, in order to limit the incident angle of the luminous flux incident on the PD 45, the luminous flux 35 for ranging is narrowed down to limit the incident angle distribution (width) of the luminous flux 35 for ranging (see the luminous flux shown by the solid line in the figure). ). Specifically, the aperture hole 52 of the aperture 51 is used to narrow the incident angle range (incident angle distribution on the image plane of the optical system 11) of the luminous flux 35 for distance measurement. At this time, even if the distance measuring light flux 35 that has passed through the aperture hole 52 is diffracted by the microlens 37, the distance measuring light flux 35 does not spread more than the PD 45 whose width Wd is expanded on the plane including the PD 45. .. As a result, the PD 45 can sufficiently receive the luminous flux 35 for ranging. By taking the same measures, the PD44 can sufficiently receive the luminous flux 34 for distance measurement. From the above, it is possible to narrow the range of the incident angle of the incident angle sensitivity characteristic while maintaining the absolute value of the sensitivity of the incident angle sensitivity characteristic high, and to improve the distance measurement accuracy.

FIG. 9 is a block diagram schematically showing a configuration of a distance measuring device according to a second embodiment of the present invention. In FIG. 9, the distance measuring device includes a camera 10 having an optical system 53 and an image sensor 12 in which a large number of pixels are arranged, an image analysis unit 13, and a distance information acquisition unit 14. The optical system 53 has, for example, two lenses 11a and 11b arranged along the optical axis, and forms an optical image of a subject on the image pickup device 12. Further, the optical system 53 has an aperture 51 arranged between the two lenses 11a and 11b. The diaphragm 51 has two diaphragm holes 52 corresponding to the two distance measuring pupils 31 and 32. When the distance measuring pixels 12b are arranged on the image pickup element 12, the optical system 53 and the microlens 37 are configured so that the two diaphragm holes 52 of the diaphragm 51 and the light-shielding film 40 (opening 39) are optically coupled. NS. Further, when the imaging and ranging pixels 12c are arranged on the image sensor 12, the optical system 53 and the microlens 43 are configured so that the two aperture holes 52 of the aperture 51 and the PDs 44 and 45 are optically conjugated. NS.

As described above, the two diaphragm holes 52 define the luminous flux 34, 35 for distance measurement passing through the distance measurement pupils 31 and 32. For example, as shown in FIG. 10, each aperture hole 52 has an incident angle sensitivity characteristic in which the range of the incident angle is limited by defining the incident angle range of each of the distance measuring luminous fluxes 34 and 35 (hereinafter, “limited incident”). (Refer to the characteristic shown by the solid line in the figure). At this time, the incident angle sensitivity characteristics of the opening 39 with the expanded width Wc and the PD44, 45 with the expanded width Wd (hereinafter referred to as "incident angle sensitivity characteristics at the time of width expansion") (characteristics shown by the broken lines in the figure). Let We be the half-value width of (see). The half-value width We is the incident angle range when the sensitivity becomes half the sensitivity Hb of the maximum sensitivity Ha in the incident angle sensitivity characteristic when the width is expanded. Further, the half width of the limited incident angle sensitivity characteristic is defined as Wf. The full width at half maximum Wf is the incident angle range when the sensitivity becomes half the sensitivity Hb of the maximum sensitivity Ha in the limited incident angle sensitivity characteristic. In the present embodiment, the size (width in the parallax direction) of each aperture hole 52 is defined so that the half-value width Wf of the limited incident angle sensitivity characteristic is smaller than the half-value width We of the incident angle sensitivity characteristic when the width is expanded. Will be done. Specifically, when the image pickup and distance measurement pixels 12c are arranged on the image sensor 12, the width of each aperture hole 52 in the parallax direction (horizontal direction in the figure) is Wg, as shown in FIG. At this time, the width Wg is smaller than the width Wh in the parallax direction in the aperture 51 of the luminous flux at the incident angle (see the luminous flux shown by the broken line in the figure) corresponding to the half-value width We of the incident angle sensitivity characteristic at the time of width expansion. Is stipulated in. Further, the size, shape, and position of the two diaphragm holes 52 are defined so that the distance measuring luminous fluxes 34 and 35 have the same polarization state and the same wavelength.

FIG. 12 is a view of the diaphragm in FIG. 9 viewed from the direction of the optical axis. In FIG. 12A, the disc-shaped diaphragm 51 has two diaphragm holes 52 (first diaphragm hole and second diaphragm hole) arranged apart from each other on the diameter in the parallax direction (horizontal direction in the drawing). Have. The positions, sizes and shapes of the two diaphragm holes 52 are set so as to realize the positions, sizes and shapes of the distance measuring pupils 31 and 32. As shown in FIG. 12A, when the diaphragm 51 has only two diaphragm holes 52, the light flux 36 for imaging does not occur. Therefore, the distance measuring luminous fluxes 34 and 35 are used not only for measuring the distance to the subject but also for forming an image of the subject. As shown in FIG. 12B, the diaphragm 51 may have not only the two diaphragm holes 52 but also another diaphragm hole 54 (third diaphragm hole) substantially in the center. In this case, the position, size and shape of the other diaphragm holes 54 are set so as to realize the position, size and shape of the imaging pupil 33. The shape of each aperture hole 52 is not particularly limited, but as shown in FIG. 12C, the parallax of the length in the direction perpendicular to the parallax direction of each aperture hole 52 (vertical direction in the figure). The ratio to the length with respect to the direction may be 1 or more. In this case, the aspect ratio of the distance measuring pupils 31 and 32 can be set to 1 or more, and thus the amount of the distance measuring flux 34 and 35 passing through the distance measuring pupils 31 and 32 is increased for measurement. The intensity of the image signal obtained from the distance fluxes 34 and 35 can be increased. Further, as shown in FIG. 12D, the diaphragm 51 may have a diaphragm mechanism 55 for adjusting the opening of another diaphragm hole 54. The diaphragm mechanism 55 is composed of two shielding blades 56, and each shielding blade 56 is configured to be slidable in the vertical direction in the drawing. As a result, the light flux 36 for imaging that passes through the pupil 33 for imaging can be narrowed down, so that the depth of focus of the subject can be deepened, and a high-quality image of the subject can be easily obtained. Further, as shown in FIG. 12E, the diaphragm 51 may have a diaphragm mechanism 57 for adjusting the opening of the diaphragm hole 52. Each diaphragm mechanism 57 is composed of two shielding blades 58, and each shielding blade 58 is configured to be slidable in the vertical direction in the drawing. As a result, the amount of light of the distance measuring fluxes 34 and 35 passing through the distance measuring pupils 31 and 32 can be freely adjusted.

In the present embodiment, the opening of the diaphragm (opening diaphragm) 51 is referred to as a "throttle hole 52", but the opening of the diaphragm 51 does not necessarily have to be a through hole. That is, the aperture may be a transparent glass plate on which a light-shielding film is formed, a liquid crystal shutter, or an electrochromic device. Further, in the present embodiment, the transmission type aperture diaphragm is illustrated as the diaphragm 51, but a reflection type aperture diaphragm may be used instead. In this case, a member in which the portion other than the aperture hole on the mirror is covered with a light absorbing material may be used, or a member in which only the portion of the aperture hole on a plate (metal plate or the like) that does not reflect light is covered with the mirror. You may use it. In the claims and the present specification, a portion (transparent glass, mirror) corresponding to a diaphragm hole of a reflective aperture diaphragm is also referred to as a “diaphragm hole”.

Next, a third embodiment of the present invention will be described. The third embodiment of the present invention is an application of the distance measuring device according to the first embodiment and the second embodiment described above to an automobile as a moving body.

13 and 14 are diagrams for schematically explaining the configuration of a driving support system in an automobile as a moving body according to the present embodiment.

In FIGS. 13 and 14, the vehicle 59 includes a distance measuring device 60 having a camera 10, an image analysis unit 13, and a distance information acquisition unit 14, and a vehicle position determination unit 61. The vehicle position determination unit 61 determines the relative position of the vehicle 59 with respect to the preceding vehicle based on the distance measuring result calculated by the distance measuring device 60, for example, the distance to the preceding vehicle. The image analysis unit 13, the distance information acquisition unit 14, and the vehicle position determination unit 61 can be implemented by software (program) or by hardware, and can be implemented by combining software and hardware. good. For example, the processing of each part may be realized by storing a program in the memory of a computer (microcomputer, FPGA, etc.) built in the camera 10 and causing the computer to execute the program. Further, a dedicated processor such as an ASIC that realizes all or part of the processing of each part by a logic circuit may be provided.

Further, the vehicle 59 includes a vehicle information acquisition device 62 (moving body information acquisition device), a control device 63, and an alarm device 64, and the vehicle position determination unit 61 includes a vehicle information acquisition device 62, a control device 63, and the like. It is connected to the alarm device 64. The vehicle position determination unit 61 acquires at least one of the vehicle speed (speed), yaw rate, steering angle, and the like of the vehicle 59 from the vehicle information acquisition device 62 as vehicle information (moving body information). The control device 63 controls the vehicle 59 based on the determination result of the vehicle position determination unit 61. The alarm device 64 issues an alarm based on the determination result of the vehicle position determination unit 61. The control device 63 is, for example, an ECU (engine control unit). For example, when there is a high possibility of a collision with a vehicle in front as a determination result of the vehicle position determination unit 61, the control device 63 avoids the collision by applying the brake of the vehicle 59, releasing the accelerator, suppressing the engine output, or the like. Perform vehicle control to reduce damage. Further, for example, when there is a high possibility of collision with a vehicle in front, the warning device 64 sounds an alarm such as a sound, displays warning information on the screen of a car navigation system or the like, and gives vibration to the seat belt or steering. To warn the user. In the present embodiment, the camera 10 of the distance measuring device 60 images the surroundings of the vehicle 59, for example, the front or the rear. The control device 63 may be configured to control the vehicle 59 based on not only the distance measurement result of the distance measuring device 60 but also the vehicle information acquired by the vehicle information acquisition device 62.

FIG. 15 is a flowchart showing a collision avoidance process as an operation example of the driving support system according to the present embodiment. The operation of each part of the driving support system will be described below according to this flowchart.

First, in step S1, the camera 10 is used to acquire image signals of a plurality of images (for example, two images for distance measurement and one image for imaging). Next, in step S2, vehicle information is acquired from the vehicle information acquisition device 62. The vehicle information here is information including at least one of the vehicle speed, yaw rate, steering angle, and the like of the vehicle 59. Next, in step S3, feature analysis (recognition processing) is performed on at least one of the acquired image signals. Specifically, the image analysis unit 13 analyzes the feature quantities such as the amount and direction of the edge, the density value, the color, and the brightness value in the image signal to analyze the object (automobile, bicycle, pedestrian, lane, guardrail). , Brake lamp, etc.) is recognized (detected). The feature amount analysis of the image may be performed for each of a plurality of image signals, or only a part of the plurality of image signals (for example, only one image signal for imaging). You may go against it.

In the following step S4, the distance information acquisition unit 14 obtains the parallax between the plurality of images captured by the camera 10, thereby acquiring the distance information to the object existing in the captured image. The distance information acquisition unit 14 acquires the distance information. As for the parallax calculation method, since the SSDA method, the area correlation method, and the like already exist as known techniques, detailed description thereof will be omitted in the present embodiment. The parallax of steps S2, S3, and S4 may be acquired in the order of the above steps, or each step may be performed in parallel. Here, the distance to the object or the amount of defocus existing in the captured image can be calculated from the parallax obtained in step S4, the internal parameters of the camera 10, and the external parameters. In the present embodiment, the distance information is defined as information on the position of the object such as the distance to the object, the defocus amount, and the parallax (image deviation amount). The distance information may also be referred to as depth information or depth information.

After that, in step S5, it is determined whether or not the obtained distance information is within the predetermined setting, that is, whether or not there is an obstacle within the set distance, and the possibility of a front or rear collision is determined. I do. When an obstacle exists within the set distance, it is determined that there is a possibility of collision, and the control device 63 performs an avoidance operation of the vehicle 59 (step S6). Specifically, the control device 63 and the alarm device 64 are notified that there is a possibility of collision. At this time, the control device 63 controls at least one of the moving direction and the moving speed of the vehicle 59. For example, braking is applied, that is, a control signal for generating a braking force is generated and output to each wheel of the vehicle 59, and the output of the engine is suppressed to avoid a collision with a vehicle in front and to prevent a collision. Make a reduction. In addition, the alarm device 64 notifies the user of the danger by sound, video, vibration, or the like. After that, this process ends. On the other hand, if there is no obstacle within the set distance, it is determined that there is no possibility of collision, and this process is terminated.

According to the process of FIG. 15, it is possible to effectively detect an obstacle. That is, it is possible to accurately detect obstacles, avoid collisions, and reduce damage.

In the present embodiment, collision avoidance based on distance information has been described, but based on the distance information, a vehicle that follows the preceding vehicle, maintains the center in the lane, or suppresses deviation from the lane. The present invention can also be applied to. Further, the present invention can be applied not only to the driving support of the vehicle 59 but also to the autonomous driving of the vehicle 59. Further, the distance measuring device 60 in the present invention can be applied not only to a vehicle such as an automobile but also to a moving body such as a ship, an aircraft, a drone or an industrial robot. Further, the distance measuring device 60 in the present invention can be applied not only to moving objects but also to devices that widely use object recognition, such as devices used in intersection monitoring systems and intelligent transportation systems (ITS). For example, the present invention may be applied to an intersection surveillance camera which is a non-mobile object in an intersection surveillance system.

Although the present invention has been described above using the respective embodiments, the present invention is not limited to the above-described embodiments.

The present invention supplies a program that realizes one or more functions of each of the above-described embodiments to a system or device via a network or storage medium, and one or more processors of the computer of the system or device implements the program. It can also be realized by the process of reading and executing. The present invention can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

10 Camera 11,53 Optical system 12 Image sensor 12a Image sensor 12b Distance measurement pixel 12c Imaging and distance measurement pixel 13 Image analysis unit 14 Distance information acquisition unit 30 Exit pupil 31, 32, 47, 48 Distance measurement pupil 33 Eyes for imaging 34,35 Light beam for distance measurement 36 Light beam for imaging 37,41,43 Microlens 38,42,44-46,49,50 PD
39, 65 Aperture 40, 66 Light-shielding film 51 Aperture 52 Aperture hole 54 Other aperture hole 59 Vehicle 60 Distance measuring device

Claims (6)

Distance information of the subject based on an image signal of an optical system having an aperture, an image sensor having an image sensor in which a plurality of pixels are arranged , and a pair of optical images having parallax output from each of the plurality of pixels. It is a distance measuring device including a distance information acquisition unit for acquiring
It said to stop,
The first diaphragm hole and the second diaphragm hole that define the first and second ranging light fluxes ,
A third diaphragm hole that defines the luminous flux for imaging is provided.
Each of the plurality of pixels
A first photoelectric conversion unit that receives the first ranging light flux and outputs one image signal of the pair of optical images .
A second photoelectric conversion unit that receives the second luminous flux for distance measurement and outputs the other image signal of the pair of optical images .
By receiving the light beam for the imaging have a third photoelectric conversion unit to output an image signal of an optical image of an object,
The third photoelectric conversion unit, with respect to the direction of the parallax, the distance measuring apparatus according to claim Rukoto sandwiched between the first photoelectric conversion unit and the second photoelectric conversion unit. The first diaphragm hole and the second diaphragm hole are arranged along the direction of the parallax.
The width of the incident angle distribution of the light flux passing through the first diaphragm hole in the direction of the parallax is smaller than the half width of the incident angle sensitivity characteristic of the first photoelectric conversion unit in the direction of the parallax.
The claim is characterized in that the width of the incident angle distribution of the light flux passing through the second diaphragm hole in the direction of the parallax is smaller than the half width of the incident angle sensitivity characteristic of the second photoelectric conversion unit in the direction of the parallax. 1. The distance measuring device according to 1. The first diaphragm hole and the second diaphragm hole are arranged along the direction of the parallax.
Width in the direction of the parallax of the optical image of the first throttle hole on the plane including the first photoelectric conversion section is smaller than the width in the direction of the parallax of the first photoelectric conversion unit,
Claims wherein the width in the direction of the parallax of the optical image of the second throttle hole on the plane including the second photoelectric conversion portion, wherein the smaller than the width in the direction of the parallax of the second photoelectric conversion unit Item 1. The distance measuring device according to item 1. The first photoelectric conversion unit and the second photoelectric conversion unit have a first shape and have a first shape.
The distance measuring device according to any one of claims 1 to 3, wherein the third photoelectric conversion unit has a second shape different from the first shape. The first diaphragm hole and the second diaphragm hole are arranged apart from each other in the direction of the parallax and have a third shape.
The measurement according to any one of claims 1 to 4, wherein the third diaphragm hole is arranged substantially in the center of the diaphragm and has a fourth shape different from the third shape. Distance device. It ’s a mobile body,
Distance measuring device and
It is provided with an alarm device that issues an alarm based on the distance measurement result of the distance measuring device.
The distance measuring device is
An image pickup device having an optical system having an aperture and an image pickup element in which a plurality of pixels are arranged,
It has a distance information acquisition unit that acquires distance information of a subject based on an image signal of a pair of optical images having parallax output from each of the plurality of pixels.
It said to stop,
The first diaphragm hole and the second diaphragm hole that define the first and second ranging light fluxes ,
A third diaphragm hole that defines the luminous flux for imaging is provided.
Each of the plurality of pixels
A first photoelectric conversion unit that receives the first ranging light flux and outputs one image signal of the pair of optical images.
A second photoelectric conversion unit that receives the second luminous flux for distance measurement and outputs the other image signal of the pair of optical images.
It has a third photoelectric conversion unit that receives the light flux for imaging and outputs an image signal of an optical image of the subject.
The third photoelectric conversion unit is a moving body characterized in that it is sandwiched between the first photoelectric conversion unit and the second photoelectric conversion unit in the direction of the parallax.

Images