JP2010243463A - Stereo camera and vehicle-outside monitor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To accurately detect three-dimensional positions of subjects being at a long distance and at a short distance, by using two imaging elements to simultaneously acquire stereo images for a long distance and for a short distance. <P>SOLUTION: A first imager 2 and a second imager 3 are provided with imaging elements 26 and 36, one for each. A stereo image for a long distance is formed out of an image of S-polarized light components detected by the first imager 2 and an image of S-polarized light components detected by the second imager 3. A stereo image for a short distance is formed out of an image of P-polarized light components detected by the first imager 2 and an image of P-polarized light components detected by the second imager 3. A stereo camera 1 is thereby simplified, miniaturized, and reduced in cost. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a stereo camera device that obtains a distance image having three-dimensional distance information and detects a plurality of three-dimensional objects from the distance image, and a vehicle exterior monitoring device using the stereo camera device.

  Conventionally, a stereo camera is known as a technique for measuring the three-dimensional position of a subject. The stereo camera calculates the three-dimensional position information of the subject by analyzing the captured images of two cameras with different viewpoints. In a stereo camera, the distance between two cameras (the distance between the optical axes) is the baseline length, and the three-dimensional position information of the subject is taken by the two cameras, which is the difference in image coordinates due to the difference in the viewpoints of the two cameras. Can be easily calculated from the parallax and the baseline length. In this stereo camera, the three-dimensional position can be calculated only for a subject that is shown in common in the two cameras.

  When a subject far away to some extent is photographed with a stereo camera, the parallax between the images taken by the two cameras becomes too small, and the error in the position of the subject to be measured becomes large. For this reason, a subject whose position can be measured with sufficient accuracy is limited to a subject whose distance from the stereo camera is closer than the measurement limit distance.

  On the other hand, the width of the field of view of the captured image differs depending on the focal length of the lens used in the stereo camera. An area range in which the position of the subject can be measured is also determined according to the focal length of the lens. The shorter the focal length of the lens, the wider the field of view of the photographed image. However, the parallax of a distant subject becomes smaller, and the measurement limit distance becomes smaller. On the contrary, when the focal length of the lens is increased, the field of view of the captured image is narrowed, but the parallax is enlarged even in a distant subject, and thus the measurement limit distance is increased.

  In order to accurately detect the three-dimensional positions of the long-distance subject and the short-distance subject, the vehicle exterior monitoring device disclosed in Patent Document 1 is a short-distance set of two stereo cameras and a short-distance subject. A total of four CCD cameras, including a set of two stereo cameras, are mounted on the vehicle, and three-dimensional measurement processing is performed by the stereo method. In this case, as a typical example of a range in which an object can be detected by a stereo camera, the distance is about 20 m to 100 m in front of the vehicle for a long distance, and about 5 m to 30 m in front of the vehicle for a short distance.

  In the case where a long-distance stereo camera and a short-distance stereo camera are used as in the vehicle exterior monitoring device shown in Patent Document 1, it is necessary to provide four sets of each image sensor. However, the image pickup element requires a mounting component and a circuit board to constitute a camera, and thus causes an increase in size and is a component that dominates the cost among the constituent members of the camera. For this reason, there is a problem that the size increases and the cost increases.

  The present invention solves such a problem and obtains a stereo image for a long distance and a short distance at the same time using two sets of image sensors to accurately determine a three-dimensional position of a subject at a long distance and a short distance. It is an object of the present invention to provide a stereo camera device that can be detected and an out-of-vehicle monitoring device using the stereo camera device.

  The stereo camera device according to the present invention includes a first imaging unit and a second imaging unit that is separated from the first imaging unit by a predetermined distance, and the first imaging unit includes a first lens and a first lens. 2 lenses, a first light guiding means, a first area dividing filter, and a first image sensor, and the optical axes of the first lens and the second lens are parallel to each other at a predetermined distance. The first light guide means deflects light from the first lens by 90 degrees with respect to the optical axis of the first lens, and deflects the deflected light by 90 degrees to produce S-polarized light. The component light is guided to the first region dividing filter, and the P-polarized component light is transmitted through the second lens and guided to the first region dividing filter. The filter has two polarizer regions that transmit only the light of the S-polarized component and the P-polarized component, and the first The light from the light means is separated into an S-polarized component and a P-polarized component, and the first image sensor receives the S-polarized component and the P-polarized component separated by the first area dividing filter and receives the S-polarized light. The second imaging unit includes a third lens, a fourth lens, a second light guide, a second area dividing filter, and a second imaging element. The third lens and the fourth lens are arranged so that their optical axes are parallel to each other at a predetermined distance, and the second light guide unit deflects light from the third lens by 90 degrees. And deflecting the deflected light by 90 degrees to guide the S-polarized component light to the second region dividing filter, and transmitting the P-polarized component light out of the light that has passed through the fourth lens. 2 region dividing filter, the second region dividing filter is light of S polarization component and P polarization component. Two polarizer regions that transmit only light, and separates light from the second light guide means into S-polarized component and P-polarized component, and the second imaging device includes the second region dividing filter Receiving the light of the S-polarized component and the P-polarized component separated in step S, detecting the images of the S-polarized component and the P-polarized component, and detecting the image of the S-polarized component detected by the first imaging unit and the second imaging A stereo image for a long distance is formed from the S-polarized component image detected by the first imaging unit, and the P-polarized component image detected by the first imaging unit and the P-polarized component detected by the second imaging unit A short-distance stereo image is formed from the above image.

  The first light guide means includes a first reflection mirror and a first polarization beam splitter, and the first reflection mirror deflects light from the first lens by 90 degrees with respect to the optical axis. The first polarizing beam splitter reflects light from the first reflecting mirror to deflect S-polarized component light by 90 degrees, and P-polarized component light out of the light passing through the second lens. The second light guide means includes a second reflecting mirror and a second polarizing beam splitter, and the second reflecting mirror transmits light from the third lens with respect to the optical axis. The second polarization beam splitter reflects light from the second reflection mirror to deflect S-polarized component light by 90 degrees, and P of the light passing through the fourth lens is deflected by 90 degrees. It is characterized by transmitting light of a polarization component.

  The first reflection mirror of the first light guide means and the first polarization beam splitter are integrated into a prism configuration, and the second reflection mirror of the second light guide means and the second polarization beam splitter are integrated. It is characterized by having a prism structure.

  The first lens to the fourth lens are preferably arranged in the order of the first lens, the second lens, the third lens, and the fourth lens.

  In the first area dividing filter, the polarizer area that transmits the S-polarized component and the polarizer area that transmits the light of the P-polarized component are divided into two according to the arrangement direction of the first lens and the second lens. In the second area dividing filter, the polarizer area that transmits the S-polarized component and the polarizer area that transmits the light of the P-polarized component are divided into two according to the arrangement direction of the third lens and the fourth lens. It is characterized by being.

  Further, the first area dividing filter alternately includes a polarizer area that transmits a plurality of S-polarized components and a polarizer area that transmits light of a P-polarized component according to the arrangement direction of the first lens and the second lens. The second region dividing filter includes a polarizer region that transmits a plurality of S-polarized components and a polarizer region that transmits light of the P-polarized components according to the arrangement direction of the third lens and the fourth lens. It is desirable to have alternately.

  Each polarizer region of the first region dividing filter and the second region dividing filter is composed of a multilayer structure in which a plurality of transparent materials having different refractive indexes are laminated on a transparent substrate, and is repeated in one direction for each layer. It has a one-dimensional periodic uneven shape.

  The vehicle exterior monitoring device of the present invention includes the stereo camera device and is mounted on a vehicle.

  It is desirable to have a half-wave plate between the second lens and the first light guiding unit and the fourth lens and the second light guiding unit of the stereo camera device of the outside monitoring apparatus.

  In the present invention, each of the first imaging unit and the second imaging unit has one imaging device, and the image of the S polarization component detected by the first imaging unit and the S detected by the second imaging unit. A stereo image for a long distance is formed from the image of the polarization component, and the stereo for a short distance is formed from the image of the P polarization component detected by the first imaging unit and the image of the P polarization component detected by the second imaging unit. By forming an image, a stereo image for a long distance and a short distance can be formed by two imaging elements, and the stereo camera device can be simplified and downsized, and the cost can be reduced.

  Further, since a stereo image for a long distance and a short distance can be formed, a highly accurate distance measurement can be performed over a wide distance range.

  In addition, the light guiding means of the first image pickup unit and the second image pickup unit is configured by a reflecting mirror and a polarizing beam splitter, and the reflecting mirror and the polarizing beam splitter are integrated into a prism configuration so that the light can be rotated or shifted. Even in this case, the deviation of the optical path is canceled, and a low-bust high configuration can be realized.

  Further, a first reflecting mirror that forms a stereo image for a long distance by arranging the first lens to the fourth lens in the order of the first lens, the second lens, the third lens, and the fourth lens. Since the directions of the reflecting surfaces of the second reflecting mirror and the second reflecting mirror are the same, it is possible to realize a stereo camera optical system that is less susceptible to the influence of the luminance distribution due to the reflecting characteristics of the reflecting mirror.

  In addition, the S-polarized component light and the P-polarized component light are divided into two by a region dividing filter to obtain an S-polarized component image and a P-polarized component image, so that the parallax between the long-distance and short-distance stereo images is obtained. A wide detection range can be secured.

  Furthermore, by having alternately a polarizer region that transmits a plurality of S-polarized light components and a polarizer region that transmits light of a P-polarized light component in the region dividing filter, the entire image pickup device can be used as an image pickup range. A visual field can be secured.

  In addition, by using the stereo camera device of the present invention in an out-of-vehicle monitoring device mounted on a vehicle, a stereo image of a long-distance subject and a short-distance subject can be obtained, and a high-precision distance over a wide distance range Measurements can be made.

  Further, by providing a half-wave plate between the second lens and the first light guiding unit of the stereo camera device used for the vehicle outside monitoring apparatus and on the fourth lens and the second light guiding unit, It is possible to obtain a high-precision stereo image by preventing the occurrence of illusion due to a window or the like when capturing an image, and to accurately measure the distance to the subject.

It is a block diagram of the stereo camera apparatus of this invention. It is a block diagram of the outside monitoring apparatus mounted in the vehicle. It is the perspective view seen from the front side of a stereo camera device. It is a perspective view which shows the structure of a region division | segmentation filter and an image pick-up element. It is a perspective view which shows the structure of the polarizer area | region which comprises an area | region division filter. It is a flowchart which shows a process when measuring the distance to a to-be-photographed object. It is a figure which shows the image of the long distance and short distance imaged with the stereo camera apparatus. It is a figure which shows the distance image which synthesize | combined the image of a long distance and a short distance. It is a perspective view which shows the other structure of an area | region division filter. It is a block diagram of the 2nd stereo camera apparatus. It is a block diagram of the 3rd stereo camera apparatus. It is a block diagram of the 4th stereo camera apparatus.

  FIG. 1 is a configuration diagram of a stereo camera device of the present invention. This stereo camera device 1 forms a stereo image for a long distance and a short distance, and is used for an out-of-vehicle monitoring device 5 mounted on a vehicle 4 such as an automobile as shown in FIG. The outside monitoring device 5 captures an object within a predetermined range outside the vehicle, recognizes and monitors the outside object from the captured image, and processes the stereo camera device 1 and the image captured by the stereo camera device 1. The image processor 6 for calculating the three-dimensional distance distribution information and the distance information from the image processor 6 are input, the road shape and the three-dimensional positions of a plurality of three-dimensional objects are detected at high speed from the distance information, and the detection result The computer includes an image processing computer 7 that identifies a preceding vehicle and an obstacle based on the above and performs a judgment process of a collision warning. When the image processing computer 7 is connected to sensors such as a speed sensor 8 and a steering angle sensor 9 for detecting the current running state of the vehicle, and the recognized object becomes an obstacle of the host vehicle 4. In addition to displaying the warning on the display 10 installed in front of the driver and warning the driver, automatic collision avoidance control of the vehicle body can be performed by connecting an external device that controls actuators (not shown). Yes. It is explanatory drawing regarding the installation position with respect to the vehicle 4 of the stereo camera apparatus 1. FIG. It is desirable to be placed at a position that does not block the driver's field of view, such as the rear stage of the rearview mirror.

  As shown in FIG. 1, the stereo camera device 1 includes a first imaging unit 2 and a second imaging unit 3 that is separated from the first imaging unit 2 by a predetermined distance. The first imaging unit 2 includes a first lens group (hereinafter referred to as a first lens) 21, a second lens group (hereinafter referred to as a second lens) 22, a reflection mirror 23, a polarization beam splitter 24, and a region division. A filter 25 and an image sensor 26 are included. The reflection mirror 23 is composed of a prism having a reflection surface 23a that deflects the optical path by 90 degrees with respect to the optical axis of the first lens 21, and polarizes light passing through the first lens 21 by 90 degrees. The polarizing beam splitter 24 is arranged in parallel with the reflecting surface 23a of the reflecting mirror 23 at a predetermined distance, and reflects the S-polarized component light and transmits the P-polarized component light from a right-angle prism having a polarizing beam splitter surface 24a. Thus, only the S-polarized light component of the light reflected by the reflection mirror 23 is reflected by the polarization beam splitter surface 24a and deflected by 90 degrees. The second lens 22 is provided on the front surface of the polarization beam splitter 24 so that the optical axis thereof is parallel to the optical axis of the first lens 21, and only the light of the P-polarized component among the light passing through the second lens 22. Passes through the polarization beam splitter surface 24a of the polarization beam splitter 24. The region dividing filter 25 has two polarizer regions that transmit only S-polarized component light and P-polarized component light, respectively, and the S-polarized component light reflected by the polarizing beam splitter surface 24a of the polarizing beam splitter 24 and the polarized beam splitter. The P-polarized light component transmitted through the surface 24a is incident to separate the S-polarized light component and the P-polarized light component. The image sensor 26 receives the S-polarized component light and the P-polarized component light separated by the region dividing filter 25 and outputs an image signal.

  The second imaging unit 3 includes a third lens group (hereinafter referred to as a third lens) 31, a fourth lens group (hereinafter referred to as a fourth lens) 32, a reflection mirror 33, a polarization beam splitter 34, and a region division. A filter 35 and an image sensor 36 are included. The reflection mirror 33 has a reflection surface 33a that deflects the optical path by 90 degrees with respect to the optical axis of the third lens 31, and polarizes the light passing through the third lens 31 by 90 degrees. The polarization beam splitter 34 is arranged in parallel with the reflection surface 33a of the reflection mirror 33 at a predetermined distance, and reflects the S-polarized component light and transmits the P-polarized component light from a right-angle prism having a polarization beam splitter surface 34a. Thus, only the S-polarized light component of the light reflected by the reflection mirror 33 is reflected by the polarization beam splitter surface 34a and deflected by 90 degrees. The fourth lens 32 is provided on the front surface of the polarization beam splitter 34 so that the optical axis is parallel to the optical axis of the third lens 31, and only the light of the P-polarized component out of the light passing through the fourth lens 32. Is transmitted through the polarization beam splitter surface 34 a of the polarization beam splitter 34. The region dividing filter 35 has two polarizer regions that respectively transmit only the S-polarized component light and the P-polarized component light, and the S-polarized component light reflected by the polarizing beam splitter surface 34a of the polarizing beam splitter 34 and the polarized beam splitter. The P-polarized light component transmitted through the surface 34a is incident to separate the S-polarized light component and the P-polarized light component. The image sensor 36 receives the S-polarized component light and the P-polarized component light separated by the region dividing filter 35 and outputs an image signal.

  Each component of the stereo camera device 1 will be described. The first lens 21 of the first imaging unit 2 and the third lens 31 of the second imaging unit 3 have a long focal length and are used for long distances, and the second lens 22 of the first imaging unit 2 is used. The fourth lens 32 of the second imaging unit 3 has a short focal length and is used for short distance use. That is, the short-distance stereo image forms a wide-angle image by using the second lens 22 of the first imaging unit 2 and the fourth lens 32 of the second imaging unit 3 that have a short focal length, and thus a long-distance image. The stereo image for use uses the first lens 21 of the first imaging unit 2 and the third lens 31 of the second imaging unit 3 that have a long focal length, and the reflection mirrors 23 and 33 and the polarization beam splitters 24 and 34. A clear image of a long-distance subject is formed by the optical path via This increases the degree of freedom in designing the lens, and as a result, the number of lenses can be reduced and the cost can be reduced.

  Further, the first lens 21 and the second lens 22 of the first imaging unit 2, the third lens 31 and the fourth lens 32 of the second imaging unit 3, and the first lens 21 and the second lens 32. The reflecting surface 23a and the second imaging of the reflecting mirror 23 of the first imaging unit 2 that form a long-distance stereo image by arranging the lens 22, the third lens 31, and the fourth lens 32 in a substantially line. The direction of the reflection surface 33a of the reflection mirror 33 of the unit 3, the polarization beam splitter surface 24a of the polarization beam splitter 24 of the first imaging unit 2, and the polarization beam splitter surface 34a of the polarization beam splitter 34 of the second imaging unit 32 Make the same. In this way, it is possible to realize a stereo camera optical system that is not easily affected by the luminance distribution caused by the reflection characteristics of the reflection surfaces 23a and 33a of the reflection mirrors 23 and 33 and the polarization beam splitter surfaces 24a and 34a. That is, when the amount of reflected light is different within the reflecting surface as the characteristics of the reflecting mirrors 23 and 33 and the polarizing beam splitters 24 and 34, a light and dark imbalance occurs in the left and right stereo images. Therefore, it is possible to form a high-quality stereo image and reduce the influence on ranging accuracy.

  When the long-distance and short-distance stereo camera devices 1 are mounted on the vehicle 4, it is necessary to arrange the captured images so that they are not kicked by a hood or the like. Since the second lens 22 of the first imaging unit 2 and the fourth lens 32 of the second imaging unit 3 that form an image for short distance are wide-angle lenses, they are arranged upward when mounted on the vehicle 4. It is desirable to be done. Therefore, as shown in the perspective view of FIG. 3, the second lens 22 of the first imaging unit 2 and the fourth lens 32 of the second imaging unit 3 are the first lens of the first imaging unit 2. 21 and the third lens 31 of the second imaging unit 3 are arranged at positions above the third lens 31.

  The reflecting mirror 23 and the polarizing beam splitter 24 of the first imaging unit 2 have a prism configuration in which the reflecting surface 23a of the reflecting mirror 23 and the polarizing beam splitter surface 24a of the polarizing beam splitter 24 are opposed to each other. The reflecting mirror 33 and the polarizing beam splitter 34 of the second camera unit 2 also have a prism configuration in which the reflecting surface 33a of the reflecting mirror 33 and the polarizing beam splitter surface 34a of the polarizing beam splitter 34 are opposed to each other. With this configuration, it is possible to realize a highly robust configuration in which the deviation of the optical path is canceled even if the prism is rotated or shifted.

  As shown in the perspective view of FIG. 4, the region dividing filters 25 and 35 are close to the first lens 21 of the first imaging unit 2 for long distance and the third lens 31 of the second imaging unit 3. Is divided into a polarizer region 25a (35a) and a polarizer region 25b (35b) in accordance with the arrangement of the second lens 22 of the first imaging unit 2 and the fourth lens 32 of the second imaging unit 3. ing. That is, a polarizer region 25a that transmits only the S-polarized component light from the first lens 21 of the first imaging unit 2 and a polarizer region 25b that transmits only the P-polarized component light from the second lens 22. The region dividing filter 35 includes a polarizer region 35 a that transmits only the S-polarized component light from the third lens 31 of the second imaging unit 3 for long distances, and P from the second lens 32. It has a polarizer region 35b that transmits only the light of the polarization component.

The dividing lines 25c (35c) of the two areas of the area dividing filters 25 and 35 are divided into areas so as to be parallel to the arrangement direction of the first lens 21 to the fourth lens 32, and the first lens 21 is divided.
A wide parallax detection range can be secured for any of the images of the fourth lens 32. In addition, regarding the upper and lower layouts of the region division patterns, it is desirable that the light is transmitted from the second lens 22 and the fourth lens 32 in the upper region. That is, the second lens 22 and the fourth lens 32 are wide-angle lenses for a short distance, and when mounted on the vehicle 4, it is necessary to arrange the captured image so as not to be kicked by a hood or the like.

  The image pickup devices 26 and 36 generally have a rectangular image pickup region, but the image pickup devices 26 and 36 correspond to the arrangement of the two divided polarizer regions 25a (35a) and the polarizer regions 25b (35b) of the region dividing filters 25 and 35. 36 is also divided into an imaging area 26a (36a) and an imaging area 26b (36b) so that a wide parallax detection range can be secured.

  The center of the polarizer region 25a of the area dividing filter 25 and the center of the imaging region 26a of the image sensor 26 coincide with the optical axis of the first lens 21, and the center of the polarizer region 25b and the center of the imaging region 26b are the second. Are arranged so as to coincide with the optical axis of the lens 22. That is, the first lens 21 and the second lens 22 are arranged so as to be shifted in different directions by 1/4 of the image sensor size with respect to the short direction of the image sensor 26. Similarly, the center of the polarizer region 35a of the region dividing filter 35 and the center of the imaging region 36a of the image sensor 36 coincide with the optical axis of the third lens 31, and the center of the polarizer region 35b and the center of the imaging region 36b are the first. The four lenses 32 are arranged so as to coincide with the optical axis. That is, the third lens 31 and the fourth lens 32 are also arranged so as to be shifted in different directions by ¼ of the image sensor size with respect to the short direction of the image sensor 36. In general, the amount of light decreases and aberration occurs toward the periphery of a lens. However, by arranging the lens as described above, it is possible to obtain stable characteristics that are not affected by lens aberration or light amount imbalance.

  The polarizer regions 25a (35a) and the polarizer regions 25b (35b) of the region dividing filters 25 and 35 are made of a polarizer made of, for example, a photonic crystal, and the polarizer regions 25a and 25b of the region dividing filter 25 are shown in FIG. As shown in FIG. 5 (a), a transparent high-refractive index medium layer 252 and a low-refractive index medium layer 253 are alternately arranged on a transparent substrate 251 formed with periodic groove rows while preserving the shape of the interface. It is formed by laminating. The high refractive index medium layer 252 and the low refractive index medium layer 253 have periodicity in the X direction perpendicular to the groove rows of the transparent substrate 251, but are uniform in the Y direction parallel to the groove rows. There may be a periodic or aperiodic structure having a length larger than the X direction. Such a fine periodic structure (photonic crystal) can be produced with high reproducibility and high uniformity by using a method called a self-cloning technique.

As shown in FIG. 4, the polarizer regions 25a and 25b made of the photonic crystal are arranged on an XY plane in an orthogonal coordinate system having a Z axis parallel to the optical axes 11a and 11b and an XY axis orthogonal to the Z axis. A multilayer structure in which two or more kinds of transparent materials are alternately stacked in the Z-axis direction on one parallel substrate 251, for example, an alternating multilayer film of Ta ₂ O ₅ and SiO ₂ , and the polarizer regions 25 a and 25 _b include Each film has a concavo-convex shape, and this concavo-convex shape is periodically repeated in one direction in the XY plane. As shown in FIG. 5B, the polarizer region 25a has a groove direction parallel to the Y-axis direction, and the polarizer region 25b has a groove direction parallel to the X-axis direction. The direction of the groove is 90 degrees different between the polarizer region 25a and the polarizer region 25b. That is, from the input light incident on the XY plane, polarized components having different polarization directions are transmitted by the polarizer region 25a and the polarizer region 25b, and equal amounts of unpolarized components are respectively transmitted by the polarizer region 25a and the polarizer region 25b. It is designed to be transparent. Note that, although two types of concave and convex grooves are provided in the region dividing filter 25, the concave and convex grooves may have a plurality of types. Thus, by forming the polarizer regions 25a and 25b with a photonic crystal, the polarizer regions 25a and 25b can be used stably for a long period of time with excellent ultraviolet deterioration. The polarizer regions 35a and 35b of the region dividing filter 35 are formed in the same manner.

  The opening area and the transmission axis of the polarizer regions 25a (35a) and the polarizer regions 25b (35b) of the region dividing filters 25 and 35 can be freely determined depending on the size and direction of the groove pattern to be processed on the transparent substrate 251 (351) first. Can be designed. Pattern formation can be performed by various methods such as electron beam lithography, photolithography, interference exposure, and nanoprinting. In any case, the direction of the groove can be determined with high accuracy for each minute region. Therefore, it is possible to form a polarizer region in which micropolarizers having different transmission axes are combined, and a polarizer in which a plurality of polarizer regions are arranged. In addition, since only a specific region having a concavo-convex pattern operates as a polarizer, if the surrounding region is flat or isotropic in the plane, the light is not a polarization-dependent medium. To Penetrate. Therefore, a polarizer can be formed only in a specific region.

  The area dividing filters 25 and 35 are disposed in proximity to the image sensors 26 and 36. More preferably, the filter structure surface is attached to the image pickup devices 26 and 36 mounted on the die by bonding or the like with the filter structure surface facing the image pickup device surface. In general, the light from the lens is directed to the image sensor with finite light of a converging system. Therefore, when the distance between the area dividing filters 25 and 35 and the image sensors 26 and 36 is increased, the light near the boundary between the area dividing filters 25 and 35 is in each area. In contrast, crosstalk noise is generated. Therefore, by arranging them in close proximity as described above, a stable stereo camera device in which such crosstalk does not occur can be realized.

  Next, processing when the stereo camera device 1 configured as described above is mounted on the vehicle 4 to capture an image of a subject and measure the distance to the subject will be described with reference to the flowchart of FIG.

  When imaging in front of the vehicle 4 is started by the stereo camera device 1, the light transmitted through the first lens 21 of the first imaging unit 2 is reflected on the reflecting mirror surface 23 a of the reflecting mirror 23 and the polarizing beam splitter surface of the polarizing beam splitter 24. Only the light of the S polarization component is received by the polarization region 26 a of the image sensor 26 through the polarized polarizer region 25 a of the region dividing filter 25 from 24 a. Further, only the light of the P-polarized component is received by the polarization region 26 b of the image sensor 26 through the polarization beam splitter surface 24 a and the polarizer region 25 b of the region dividing filter 25. The light that has passed through the third lens 31 of the second imaging unit 3 passes through the polarizing mirror region 35a of the region dividing filter 35 from the reflecting mirror surface 33a of the reflecting mirror 33 and the polarizing beam splitter surface 34a of the polarizing beam splitter 34. Only the polarization component light is received by the polarization region 36 a of the image sensor 36. Further, only the light of the P-polarized component is received by the polarization region 36 b of the image sensor 36 through the polarization beam splitter surface 34 a and the polarizer region 35 b of the region dividing filter 35. By this light reception, the first image 12a of the long-distance subject, the first image 12b of the short-distance subject, the second image 12c of the long-distance subject, and the second image 12d of the short-distance subject are imaged. , 36 (step S1).

  Based on the first image 12a of the long-distance subject imaged in the imaging region 26a of the image sensor 26 and the second image 12c of the long-distance subject imaged in the imaging region 36a of the image sensor 36, The parallax of the subject is detected, and the first image 12b of the short-distance subject imaged in the imaging region 26b of the imaging device 26 and the second image 12d of the short-distance subject imaged in the imaging region 36b of the imaging device 36. The parallax of the subject at a short distance is detected based on (Step S2). Based on the detected distant object parallax, the distance to the distant object is calculated based on the later-described equation (1) to obtain a long-distance measurement result. The distance to the subject is calculated and used as a short distance measurement result (step S3). Then, the obtained long distance measurement result and short distance measurement result are combined and output to the display 10 as one distance image (step S4).

  An example of the first image 12a and the second image 12c of the captured long-distance subject is shown in FIG. 7A, and an example of the first image 12b and the second image 12d of the short-distance subject is shown. 7 (b). In general, the image through the lens is inverted, but here, the images 12a to 12d are shown rotated forward for easy understanding. As shown in FIG. 7A, the images 12a and 12c for long distance have a long focal length, so that the image of the subject appears large. On the other hand, as shown in FIG. 7B, the images 12b and 12c for short distance have a short focal length, so the optical magnification is low, and the subject looks small and is observed in a wide field of view. An example of the distance image 13 obtained by combining the long distance measurement result and the short distance measurement result is shown in FIG. The distance image 13 is displayed with the distance in shades of color, and the short distance measurement result and the long distance measurement result are combined and output. The distance to the subject can be visually recognized by the color density of the distance image 13. The result of the long-distance measurement corresponds to a part 13a of the distance image 13, and the distance of the long-distance image 13a is higher than that of other regions because of the narrow field of view.

The captured images 12 a to 12 d are temporarily stored in the memory of the image processing computer 7 mounted on the vehicle 4. The image processing computer 7 accesses the image data stored in the memory, for example, cuts out the effective area of the images 12b and 12d, and pays attention to the minute area 14a of the image 12b and the minute area 14b of the image 12d corresponding thereto, The parallax δ of the subject in these minute areas 14a and 14b is calculated. Since this parallax δ occurs only in the horizontal direction, the parallax δ in the horizontal direction may be calculated here. This parallax δ is obtained in units of pixels (pixels). Between the distance A between the stereo camera device 1 and the subject and the parallax δ, the relationship of the following equation (1) is established from the distance between the lenses (baseline length) B and the focal length f of the lens.
A = B · f / δ (1)
By substituting the parallax δ into this equation (1), the distance A to the subject can be obtained.
Then, the parallax detection is performed while scanning the minute region of interest over the entire image, and the distance image 13 shown in FIG. 8 can be formed by combining them. Accordingly, the distance to the vehicle ahead and the distance to the pedestrian can be measured, and this can be used for warning information for the driver of the vehicle 4 and feedback information for vehicle operation.

  In this way, the two image pickup devices 26 and 36 of the stereo camera device 1 can form a long-distance and a short-distance stereo image, and the stereo camera device 1 can be reduced in size, simplified, and low-cost as compared with the related art. Can be achieved. Further, by forming stereo images for long distance and short distance, highly accurate distance measurement can be performed over a wide distance range.

  In the above description, the area dividing filters 25 and 35 are divided into the polarizer area 25a (35a) and the polarizer area 25b (35b). However, as shown in FIG. 35a) and the polarizer regions 25b (35b) may be alternately arranged in parallel strips. Thus, by arranging the polarizer regions 25a (35a) and the polarizer regions 25b (35b) in parallel strips alternately, the entire image sensor 26 (36) can be used as an imaging range. A wide field of view can be secured. In addition, since the strips are arranged in a direction parallel to the arrangement direction of the first lens 21 to the fourth lens 32, a horizontal stereo image necessary for parallax detection can be obtained with high accuracy.

  In the above description, the S-polarized component and the P-polarized component incident on the region dividing filters 25 and 35 are separated by the polarization beam splitters 24 and 34. However, as shown in FIG. Polarizers 27a and 37a for S-polarization are provided in front or rear of the third lens 31, and polarizers 27b and 37b for P-polarization are provided in front or rear of the second lens 22 and the fourth lens 32. Instead of the polarization beam splitters 24 and 34, prisms 28 and 38 having reflective surfaces that reflect the S-polarized component and transmit the P-polarized component may be used.

  Further, as shown in FIG. 11, the second imaging unit 3 is arranged with respect to the first imaging unit 2, and the base lengths of the first lens 21 for long distance and the third lens 31 are set close to each other. The distance may be longer than the base line length of the second lens 22 and the fourth lens 32 for distance. Thus, by increasing the base line length of the first lens 21 and the third lens 31 for a long distance, the distance accuracy of the long distance can be improved.

  Also, as shown in FIG. 12, half-wave plates 29 and 39 are disposed between the second lens 22 for short distance and the polarizing beam splitter 24, and between the fourth lens 32 and the polarizing beam splitter 34. You may do it. The first lens 21 to the fourth lens 32 detect light having the same polarization direction, but particularly light having an S polarization component is detected. That is, the stereo camera device 1 is an optical system in which the light of the P polarization component is cut. For example, when the stereo camera device 1 is disposed inside the vehicle 4 and the outside of the vehicle is monitored, the S-polarized component such as the window of the vehicle 4 becomes dazzling. By removing at 29 and 39, distance measurement for long distance and short distance can be performed with high accuracy.

  Further, in the above description, the case where the region dividing filters 25 and 35 are configured using a photonic crystal has been described, but a wire grid type polarizer may be used. A wire grid type polarizer is a polarizer formed by periodically arranging thin metal wires, and has been conventionally used in the millimeter wave region of electromagnetic waves. The structure of the wire grid polarizer has a structure in which fine metal wires that are sufficiently thin compared to the wavelength of the input light are arranged at intervals that are sufficiently short compared to the wavelength. It is already known that when light is incident on such a structure, polarized light parallel to the metal thin wire is reflected and polarized light orthogonal thereto is transmitted. With respect to the direction of the fine metal wire, since it can be produced by changing each region independently in one substrate, the characteristics of the wire grid polarizer can be changed for each region. If this is utilized, it can be set as the structure which changed the direction of the transmission axis for every area | region.

  As a method for manufacturing the wire grid, a thin metal can be left by forming a metal film on a substrate and performing patterning by lithography. As another manufacturing method, a groove is formed on the substrate by lithography, and a metal is deposited by vacuum deposition from a direction perpendicular to the direction of the groove and inclined from the normal line of the substrate (a direction oblique to the substrate surface). Can be produced. In vacuum vapor deposition, particles flying from the vapor deposition source hardly collide with other molecules or atoms in the middle of the vapor deposition, and the particles travel linearly from the vapor deposition source to the substrate. On the other hand, at the bottom part (concave part) of the groove, the film is shielded by the convex part and hardly formed. Therefore, by controlling the amount of film formation, a metal film can be formed only on the convex portion of the groove formed on the substrate, and a thin metal wire can be produced. The wire metal used in the wire grid polarizer is preferably aluminum or silver, but the same phenomenon can be realized even with other metals such as tungsten. In addition, as lithography, optical lithography, electron beam lithography, X-ray lithography, and the like can be given. However, since an interval between thin lines is about 100 nm when an operation with visible light is assumed, electron beam lithography or X-ray lithography is more preferable. . In addition, vacuum deposition is desirable for metal film formation. However, since the directionality of particles incident on the substrate is important, sputtering in a high vacuum atmosphere or collimation sputtering using a collimator is also possible. Since this wire grid type polarizer is a semiconductor process as well as a photonic crystal polarizer, it is possible to accurately produce a boundary line between two divided regions.

DESCRIPTION OF SYMBOLS 1; Stereo camera apparatus, 2; 1st imaging part, 3; 2nd imaging part, 4;
5: Outside vehicle monitoring device, 6: Image processor, 8: Computer for image processing,
10; display; 21; first lens; 22; second lens;
23; reflecting mirror, 24; polarizing beam splitter, 25; area dividing filter,
26; Image pickup device, 31; Third lens, 32; Fourth lens, 33; Reflection mirror,
34; polarizing beam splitter, 35; area dividing filter, 36; imaging device.

JP-A-5-265547

Claims (9)

A first imaging unit and a second imaging unit separated from the first imaging unit by a predetermined distance;
The first imaging unit includes a first lens, a second lens, a first light guiding unit, a first area dividing filter, and a first imaging element, and the first lens and the second lens The lens is arranged so that the optical axis is parallel with a predetermined distance, and the first light guide unit deflects light from the first lens by 90 degrees with respect to the optical axis of the first lens. And deflecting the deflected light by 90 degrees to guide the S-polarized component light to the first region dividing filter, and transmitting the P-polarized component light out of the light passing through the second lens, The first region division filter has two polarizer regions that transmit only the light of the S-polarization component and the P-polarization component, and the light from the first light guide unit is guided to the first region division filter. The first image sensor is separated by the first area dividing filter, and is separated into an S-polarized component and a P-polarized component. By receiving the S-polarized light component and P-polarized light component detects the image of the S-polarized light component and P-polarized light component,
The second imaging unit includes a third lens, a fourth lens, a second light guiding unit, a second area dividing filter, and a second imaging element, and the third lens and the fourth lens The lens is arranged so that the optical axes are parallel to each other at a predetermined distance, and the second light guide unit deflects light from the third lens by 90 degrees and deflects the deflected light by 90 degrees. S-polarized component light is guided to the second region dividing filter, P-polarized component light out of the light passing through the fourth lens is transmitted to the second region dividing filter, and the second region dividing filter is transmitted. The region dividing filter has two polarizer regions that transmit only the light of the S-polarized component and the P-polarized component, and separates the light from the second light guiding means into the S-polarized component and the P-polarized component, The second image sensor includes light of an S polarization component and a P polarization component separated by the second region dividing filter. Receiving and detects the image of the S-polarized light component and P-polarized light component,
A stereo image for a long distance is formed from the S-polarized component image detected by the first imaging unit and the S-polarized component image detected by the second imaging unit, and is detected by the first imaging unit. A stereo camera device for forming a short-distance stereo image from an image of a P-polarized component to be detected and an image of a P-polarized component detected by the second imaging unit. The first light guide means includes a first reflection mirror and a first polarization beam splitter, and the first reflection mirror deflects light from the first lens by 90 degrees with respect to the optical axis. The first polarizing beam splitter reflects light from the first reflecting mirror to deflect S-polarized component light by 90 degrees, and P-polarized component light out of the light passing through the second lens. Through
The second light guide means includes a second reflection mirror and a second polarization beam splitter, and the second reflection mirror deflects light from the third lens by 90 degrees with respect to the optical axis. The second polarizing beam splitter reflects the light from the second reflecting mirror to deflect the S-polarized component light by 90 degrees, and the P-polarized component light out of the light passing through the fourth lens. The stereo camera device according to claim 1, wherein the stereo camera device is transmitted.   The first reflection mirror of the first light guide means and the first polarization beam splitter are integrated into a prism configuration, and the second reflection mirror of the second light guide means and the second polarization beam splitter are integrated. 3. The stereo camera device according to claim 2, wherein a prism structure is provided.   The first to fourth lenses are arranged in the order of a first lens, a second lens, a third lens, and a fourth lens. The stereo camera device described in 1. In the first region dividing filter, a polarizer region that transmits an S-polarized component and a polarizer region that transmits light of a P-polarized component are divided into two according to the arrangement direction of the first lens and the second lens. ,
In the second region dividing filter, a polarizer region that transmits an S-polarized component and a polarizer region that transmits light of a P-polarized component are divided into two according to the arrangement direction of the third lens and the fourth lens. The stereo camera device according to claim 4, wherein the stereo camera device is provided. The first region dividing filter alternately includes a polarizer region that transmits a plurality of S-polarized components and a polarizer region that transmits light of a P-polarized component according to the arrangement direction of the first lens and the second lens. Have
The second region dividing filter alternately includes a polarizer region that transmits a plurality of S-polarized components and a polarizer region that transmits light of a P-polarized component according to the arrangement direction of the third lens and the fourth lens. The stereo camera device according to claim 4, wherein the stereo camera device is provided.   Each polarizer region of the first region dividing filter and the second region dividing filter is composed of a multilayer structure in which a plurality of transparent materials having different refractive indexes are laminated on a transparent substrate, and is repeated in one direction for each layer. The stereo camera device according to claim 1, which has a one-dimensional periodic uneven shape.   An out-of-vehicle monitoring device comprising the stereo camera device according to claim 1 and mounted on a vehicle.   9. A half-wave plate is provided between the second lens and the first light guide unit of the stereo camera device and between the fourth lens and the second light guide unit. Outside monitoring device.

Images