JP2020020657A - Stereo camera device, stereo camera system, and moving object - Google Patents

Abstract

To provide a stereo camera device with which, even when imaging via a light transmissive member, it is possible to maintain accurate distance measurement performance without executing a stereo calibration process.SOLUTION: The stereo camera device comprises: at least two camera units for imaging a subject on the basis of imaging light having passed through a transparent member having a cyclic distortion characteristic; and a fixing unit for fixing each camera unit at a position where the phases of image movement based on the cyclic distortion characteristic of the transparent member in the camera images imaged by each camera unit match. Thus, the phases of image movement in the camera images due to the transparent member can be matched in each camera unit. Therefore, it is possible to reduce a parallax error due to the distortion of the transparent member and maintain accurate distance measurement performance without executing a stereo calibration process.SELECTED DRAWING: Figure 5

Description

  The present invention relates to a stereo camera device, a stereo camera system, and a moving object.

  In recent years, in-vehicle stereo camera devices have been put to practical use. This in-vehicle stereo camera device generates a distance image in front of the vehicle using a stereo camera device provided in a vehicle such as an automobile. Further, the in-vehicle stereo camera device measures the distance to the recognized obstacle based on the generated distance image. The in-vehicle stereo camera device warns the driver based on the measurement result and automatically controls a control device such as a brake or a steering. As a result, it is possible to perform driving assistance that automatically enables, for example, collision prevention or inter-vehicle distance control.

  Patent Document 1 (Japanese Patent Application Laid-Open No. 2004-132870) discloses a stereo camera adjustment device that performs stereo calibration in a vehicle-mounted state.

  Patent Document 2 (Japanese Patent Application Laid-Open No. 07-010569) discloses a method of manufacturing a float glass sheet that can suppress periodic “wrinkles (distortion)” generated in a manufacturing process of the float glass sheet.

  However, when the stereo camera device performs imaging through a light transmitting member such as a float plate glass that transmits light, the distance measurement accuracy between obstacles and the like is affected by distortion generated in a manufacturing process of the light transmitting member. There is a problem of decline.

  For this reason, as disclosed in Patent Document 1, in the case of an in-vehicle stereo camera device, a stereo calibration process for performing stereo calibration in a vehicle-mounted state is required. In the case of the technique disclosed in Patent Literature 1, a stereo calibration process is added during the vehicle manufacturing process, and there is a problem that the vehicle manufacturing process is complicated.

  In the case of the method for manufacturing a float glass sheet disclosed in Patent Document 2, it can be said that the float glass can be manufactured while suppressing distortion, but it is difficult to completely eliminate even minute distortion. For this reason, even when an image is taken by a stereo camera device via a float glass manufactured by the technique disclosed in Patent Document 2, the above-described stereo calibration process is still required, and the manufacturing process of the vehicle is complicated. Is concerned.

  SUMMARY OF THE INVENTION The present invention has been made in view of the above-described problems, and enables accurate distance measurement performance to be maintained without performing a stereo calibration process even when imaging is performed through a light transmitting member, thereby manufacturing a vehicle or the like. It is an object of the present invention to provide a stereo camera device, a stereo camera system, and a moving object that can prevent inconvenience of complicating processes.

  In order to solve the above-described problems and achieve the object, the present invention provides at least two camera units that capture an image of a subject based on imaging light transmitted through a transmission member having periodic distortion characteristics, and each camera A fixing unit that fixes each camera unit at a position where the phase of image movement based on the periodic distortion characteristic of the transmission member in the camera image captured by the unit matches.

  ADVANTAGE OF THE INVENTION According to this invention, even when imaging via a light transmission member, accurate distance measurement performance can be maintained without performing a stereo calibration process. Further, since the stereo calibration process can be omitted, there is an effect that the inconvenience of complicating the manufacturing process of the vehicle or the like can be prevented.

FIG. 1 is a diagram illustrating a state in which a windshield portion of a vehicle provided with a stereo camera system according to an embodiment is viewed from a side. FIG. 2 is a diagram illustrating a schematic configuration of the stereo camera system according to the embodiment. FIG. 3 is a block diagram of a stereo camera device provided in the stereo camera system according to the embodiment. FIG. 4 is a diagram for explaining the principle of distance measurement by the camera units arranged in parallel. FIG. 5 is a diagram illustrating a state where the stereo camera device of the stereo camera system according to the embodiment is installed inside the windshield of the vehicle. FIG. 6 is a diagram for explaining the image movement amount and the parallax error amount of the camera images of the left and right camera units generated by the windshield. FIG. 7 is a view of left and right camera units provided in the vehicle as viewed along the optical axis direction from outside the vehicle via a windshield. FIG. 8 is a diagram for explaining an imaging state of a camera image captured via a windshield that is inclined with respect to the optical axis of the stereo camera device. FIG. 9 is a diagram illustrating an imaging state of a camera image when a stripe pattern in an oblique direction exists on the windshield. FIG. 10 is a diagram for explaining a distortion cycle measuring process. FIG. 11 is a diagram for explaining a search operation of a pixel (corresponding point) moving between the image without glass and the image with glass.

  Hereinafter, embodiments of a stereo camera device, a stereo camera system, and a moving object will be described.

(System configuration)
FIG. 1 is a diagram of a state in which a windshield portion of a vehicle provided with a stereo camera system according to an embodiment is viewed from a side. As shown in FIG. 1, a stereo camera system 2 that captures an image in a predetermined imaging range in a direction in which the vehicle 1 moves forward is provided inside (inside the vehicle) a windshield (an example of a transmissive member) 1a of the vehicle 1. I have. The stereo camera system 2 includes two image sensors as described later with reference to FIG. 3, and captures two images for the left eye and the right eye.

  FIG. 2 is a diagram illustrating a schematic configuration of the stereo camera system 2 according to the embodiment. As shown in FIG. 2, the stereo camera system 2 includes a stereo camera device 20 and a vehicle ECU (Engine Control Unit) 50.

  The stereo camera device 20 captures two images for the left eye and the right eye in a predetermined imaging range in the direction in which the vehicle 1 moves forward. Further, the stereo camera device 20 detects, for example, an obstacle in the traveling direction of the vehicle 1 based on the captured left-eye and right-eye images. Then, based on the detection result of an obstacle or the like and CAN (Controller Area Network) information to be described later, the stereo camera device 20 performs, for example, engine control, braking control, running lane keeping assist, steering assist, and the like of the vehicle 1. Vehicle control data for performing driving support for the vehicle 1 is supplied to the vehicle ECU 50.

  The vehicle ECU 50 is an example of a control unit, and based on vehicle control data supplied from the stereo camera device 20, for example, engine control, braking control, and driving on the vehicle 1 such as running lane keeping assist and steering assist. Perform support control. In the following, the vehicle 1 will be described as an example of a moving object, but the stereo camera system 2 of the present embodiment can be applied to a ship, an aircraft, a robot, and the like.

(Configuration of imaging device)
FIG. 3 is a block diagram of the stereo camera device 20. As shown in FIG. 3, the stereo camera device 20 includes a camera unit 20a for the left eye, a camera unit 20b for the right eye, and an image processing device 30. The camera units 20a and 20b are assembled parallel (horizontally) by a camera housing, which is an example of a fixed unit, and captures a moving image (or a still image) of the imaging target area. In addition, as will be described later, the camera housing is positioned at a position where the phase of the image movement based on the periodic distortion characteristic of the transmission member such as the windshield in the camera images captured by the camera units 20a and 20b matches. The camera units 20a and 20b are fixed.

  The camera units 20a and 20b each have a lens 21, an image sensor 22, and a sensor controller 23. The image sensor 22 is, for example, a charge coupled device (CCD) image sensor or a complementary metal-oxide semiconductor (CMOS) image sensor. The sensor controller 23 performs, for example, exposure control of the image sensor 22, image reading control, communication with an external circuit, and transmission control of image data.

  The image processing device 30 includes, for example, a data bus line 300, a serial bus line 302, a CPU (Central Processing Unit) 304, an FPGA (Field-Programmable Gate Array) 306, a ROM (Read Only Memory) 308, and a RAM (Random Access Memory) 310. , A serial IF (Interface) 312 and a data IF (Interface) 314.

  The image processing device 30 is connected to the above-described camera units 20a and 20b via a data bus line 300 and a serial bus line 302. A storage unit such as the ROM 308 stores a vehicle control program. The CPU 304 controls the entire operation of the image processing apparatus 30 and executes image processing and image recognition processing by executing a vehicle control program stored in a storage unit such as the ROM 308.

  Brightness image data of each camera image captured by each image sensor 22 of the camera units 20a and 20b is written to the RAM 310 of the image processing device 30 via the data bus line 300. Sensor exposure value change control data, image reading parameter change control data, and various setting data from the CPU 304 or the FPGA 306 are transmitted and received via the serial bus line 302.

  The FPGA 306 generates a parallax image by performing processing that requires real-time processing on the image data stored in the RAM 310, such as gamma correction, distortion correction (parallelization of left and right images), and parallax calculation by block matching. , Is written into the RAM 310 again. The parallax image is information in which the vertical position, the horizontal position, and the depth position of the object are associated with each other.

  The CPU 304 controls each sensor controller 23 of the stereo camera system 2 and performs overall control of the image processing device 30. The CPU 304 acquires, for example, CAN (Controller Area Network) information of the vehicle 1 (own vehicle) as parameters (vehicle speed, acceleration, steering angle, yaw rate, etc.) via the data IF 314.

  Further, the CPU 304 reads the parallax image written in the RAM 310 based on the vehicle control program, and recognizes, for example, an object (an obstacle) such as a preceding vehicle, a person, a guardrail, and a road surface. Then, the CPU 304 forms vehicle control data based on the object recognition result and the CAN information of the vehicle 1 (own vehicle), and supplies the vehicle control data to the vehicle ECU 50 via the serial IF 312.

  The vehicle ECU 50 performs, for example, engine control, braking control, and driving support control for the vehicle 1 such as running lane keeping assist and steering assist based on the vehicle control data supplied from the image processing device 30.

(Distance measurement by stereo camera system)
Here, the principle of distance measurement by the camera units 20a and 20b arranged in parallel will be described with reference to FIG. In FIG. 4, the camera _{C 0} corresponds to the camera unit 20a described above, the camera _{C 1} corresponds to the camera unit 20b described above. The focal length of the camera C ₀ is “f”, the optical center is “O ₀ ”, and the imaging surface is “S ₀ ”. The focal length of the camera C ₁ is "f", the optical center "O _1", the imaging surface is "S _1". The cameras C ₀ and C ₁ are arranged in parallel at a distance B.

The image of the subject “A” located at a distance “d” in the optical axis direction from the optical center “O ₀ ” of the camera C ₀ is represented by a straight line “A” × the optical center “O ₀ ” and the imaging surface “ An image is formed at the intersection point “P ₀ ” of “S ₀ ”. Similarly, in the case of the camera C _1, subject "A" is focused on the intersection of the imaging surface "S _1", "P _1".

The intersection of a straight line (shown by a dashed straight line in FIG. 4) that passes through the optical center “O ₁ ” of the camera “C ₁ ” and is parallel to the straight line “A-O ₀ ” and “P ₁ ” ₀₀ ”, and the distance between the intersection“ P ₀₀ ”and the point“ P ₁ ”at which the subject“ A ”forms an image is“ r ”.

The intersection point “P ₀₀ ” is a position corresponding to the point “P ₀ ” where the subject “A” forms an image on the camera C ₀ . Distance "r", in the case of photographing the same subject "A" in the two cameras C ₀ and the camera C _1, shows the positional displacement amount of the camera image. The displacement amount of each camera image is called “parallax”.

Since the triangles AO ₀ -O ₁ and the triangles: O ₁ -P ₀₀ -P ₁ are similar, the base length is “B”, the focal length is “f”, the base length “B” and the focal length are If “f” is known, the distance “d” can be calculated from the parallax “r” by an arithmetic expression of “d = Bf / r”. The “base line length” is a length of a base line that is a straight line that is orthogonal to each optical axis of each of the camera units 20a and 20b and that connects each optical axis.

The above is the principle of distance measurement in the stereo camera system 2. In order to establish this principle, the two cameras C ₀ and the camera C ₁ as shown in FIG. 4, it is necessary to accurately position while maintaining the parallel relationship.

(Effect of windshield)
Next, when the in-vehicle stereo camera device 20 is installed outside the vehicle, higher durability is required for high temperature, low temperature, waterproof, dustproof, and the like. For this reason, the stereo camera device 20 is often installed inside the windshield 1a of the vehicle as shown in FIG. The stereo camera system 2 installed inside the vehicle receives an imaging light from outside the vehicle via the windshield 1a to perform imaging. The windshield 1a has a complicated curved surface shape, and its shape is more distorted than a sophisticated optical component such as a lens of the stereo camera device 20.

  As an example, in the case of the windshield 1a having a thickness of 1.5 mm or less, fine distortion (almost in the manufacturing process (float method)) in which the pitch is about 25 mm and the depth of the valley is about 0.1 μm to 0.3 μm. Periodic wrinkles (corrugation) occur. It is difficult to completely eliminate such distortion in the manufacturing process.

  For this reason, distortion also occurs in the camera image captured through the windshield 1a. In addition, the distortion characteristics also change depending on the installation position and orientation of the stereo camera system 2 with respect to the vehicle 1. Therefore, in order to correct the distortion generated in the camera image, after installing the stereo camera system 2 at a predetermined position of the vehicle 1, a stereo calibration process is performed using the image captured through the windshield 1a. I needed to.

(Parallax error due to image distortion)
Next, a change in parallax due to image distortion will be described. Image distortion means that the image position changes as compared to an ideal pinhole camera. By performing imaging through the windshield 1a, a phenomenon occurs in which the amount of image position movement changes according to the position in the image.

Consider a direction parallel to the base line direction (the direction of the base line length B in FIG. 4) (hereinafter, referred to as a lateral direction) as a position in the image. The base line direction is a direction (the above-described lateral direction) of a line parallel to a straight line connecting the optical center “O ₀ ” and the optical center “O ₁ ” of the camera C ₀ and the camera C ₁ shown in FIG. When the left and right camera units 20a and 20b are attached to the vehicle 1 as shown in FIG. 1, the X direction in FIG. 4 is the width direction of the vehicle 1 (= left and right direction of the vehicle 1), and the Y direction is the The height direction (= vertical direction) and the Z direction are forward directions (depth direction: front-back direction) of the vehicle 1. In FIG. 4, the Z direction corresponds to the optical axis direction of each of the camera units 20a and 20b. Similarly, the X direction corresponds to the above-described baseline direction.

  The image position movement amount is a plus or minus one-dimensional amount, and the distortion characteristics on the image are as shown in FIGS. 6A and 6B. 6A and 6B, the horizontal axis represents the horizontal position (x) in the image, and the vertical axis represents the image position movement amount (Δx). In addition, the solid line curve indicates the image position movement amount of the left-eye camera unit 20a, and the solid line curve indicates the image position movement amount of the right-eye camera unit 20b.

  When the left and right camera units 20a and 20b maintain an ideal parallel relationship, parallax between the left and right camera images occurs only in the horizontal direction, and the subject at infinity is captured at the same position. However, when the left and right camera images are distorted, the stereo parallax is the difference between the image positions on the left and right camera images, and thus the parallax error corresponding to the difference between the image movement amounts of the left and right camera units 20a and 20b. Occurs.

  The example of FIG. 6A shows a parallax error when the image position of the left and right camera images has a large displacement, and the example of FIG. 6B shows a small difference between the image positions of the left and right camera images. 11 shows a parallax error in the case where has occurred. The arrows in FIGS. 6A and 6B indicate the magnitude of the parallax error. As shown in FIG. 6A, the image position of one of the left and right camera images is shifted in the plus direction, and the image position of the other camera image is shifted in the minus direction. In addition, when the image positions of the left and right camera images are shifted in the opposite direction, the parallax error increases.

  On the other hand, as shown in FIG. 6B, when the image movement amounts of the left and right camera images are substantially in the same direction and the curves overlap each other, even if the movement amount itself is large, the left and right image position movement amount is not large. The error of the parallax, which is the difference, becomes smaller. This means that even if the windshield 1a has the same distortion, the parallax error can be reduced by aligning the image movement phases of the left and right camera images.

  The fixed positions of the left and right camera units 20a and 20b are adjusted so that the amount of phase shift of the image movement of each camera image is equal to or less than a predetermined amount (phase difference) such as 30 degrees or less. Good. Thereby, the parallax error can be made equal to or less than a predetermined value.

(Strain cycle and baseline length)
Next, if the relative positions of the wrinkles of the windshield 1a in front of the single camera and the single camera in the left and right camera units 20a, 20b match, the same shape is formed immediately in front of the left and right camera units 20a, 20b. Is present, the phases of the image position movement amounts of the camera images can be made uniform (the phases can be made the same).

  FIGS. 7A to 7C are views of the left and right camera units 20a and 20b provided in the vehicle viewed from outside the vehicle 1 via the windshield 1a along the optical axis direction. is there. The symbols “+” shown in FIGS. 7A to 7C indicate a glass region (hereinafter, referred to as a plus region) in which the image position movement amount has a distortion in the plus direction. 7A to 7C indicate a glass region (hereinafter, referred to as a minus region) in which the image position movement amount has a distortion in the minus direction.

  As shown in FIG. 7A, in the case where the plus areas having the image position movement amount plus distortion are located in front of the left and right camera units 20a and 20b, and the minus areas are located on the left and right of each plus area, As shown in FIG. 6B, the directions of the image position movement amounts in the left and right camera images match, and the parallax error is reduced.

  This is not limited to the case where the plus region is located “in front of” the left and right camera units 20a and 20b, and the plus region and the minus region are located at the same position of the left and right camera units 20a and 20b as shown in FIG. If the region is located, the parallax error similarly becomes smaller. Note that the example of FIG. 7B is an example in which a plus area is located on the left side of the left and right camera units 20a and 20b, and a minus area is located on the right side. Such an arrangement of the plus region and the minus region is established if the interval between the plus regions on the windshield 1a (distortion period: distortion characteristic) matches the base line length of the stereo camera device 20.

  Furthermore, as shown in FIG. 7C, the case where the base line length is in a relationship of an integral multiple of the period ratio of the distortion period (distortion period ratio) is also established. The example of FIG. 7C is an example in which the base line length is twice the distortion period. Hereinafter, the magnification of the base line length with respect to the distortion period is referred to as “base line length magnification”. Even if there is some variation in the distortion period or the amplitude, if the worst case in which the image movement amount becomes the maximum and minimum combination on the left and right can be avoided, the phase of the left and right camera images becomes as shown in FIG. By inverting the parallax error amount, the parallax error amount can be significantly reduced to half or less of the maximum parallax error amount.

(Upper limit of baseline length magnification)
Next, the upper limit of the base line length magnification will be described. When the distortion period is constant, the base line length magnification may be an integer multiple. However, when the distortion period varies, the larger the baseline length magnification, the greater the phase shift between the left and right camera images due to the variation in the distortion period. For example, when the representative period is 30 mm and the distortion period of the windshield 1a is 33 mm (+ 10%), if the base line length is 1 and the base line length is 30 mm, the phase difference between the left and right is (33−30) / 30 = The shift is 10%. On the other hand, if the base line length magnification is 5 and the base line length is 150 mm, the phase difference between the left and right camera images is ((33−30) / 30) × 5 = 50%, as shown in FIG. As described above, the phases of the left and right camera images are inverted to generate the maximum parallax error.

  In order to avoid such inconvenience, it is necessary to make the product of the fluctuation rate of the distortion period (period fluctuation rate) and the baseline length magnification sufficiently smaller than 50%. For the purpose of aligning the phases of the left and right camera images, it is desirable to reduce the baseline length magnification. However, since the length of the base line is proportional to the height of the distance measurement accuracy of the stereo camera device 20, the base line length is preferably longer.

  For this reason, in the stereo camera system according to the embodiment, the product of the variation rate of the distortion period of the windshield 1a and the magnification of the baseline length of the stereo camera device 20 with respect to the distortion period of the windshield 1a is set to be smaller than 50%. . Each of the camera units 20a and 20b is fixed to a position that realizes a base line length that can maintain the distance measurement accuracy required for the stereo camera device 20 by a camera housing that is a fixing unit.

(Other considerations)
Next, in many cases, the windshield 1a is inclined with respect to the optical axis of the stereo camera device 20, as shown in FIG. Therefore, when, for example, a vertical stripe pattern (a vertical stripe pattern of the windshield 1a) exists on the windshield 1a, the camera images captured by the left and right camera units 20a and 20b are respectively shown in FIG. As described above, the camera image is a camera image having a shape of a lower constriction (a shape obtained by rotating a fan shape or an isosceles trapezoid by 180 degrees) in which the width on the lower side is smaller than the width on the upper side of the vertical stripes.

  Here, even if the windshield 1a rotates around the base line direction of the stereo camera device 20, the distance between the left and right camera units 20a, 20b and the windshield 1a does not change. For this reason, as in the description using FIGS. 7A to 7C, if the period of the stripe pattern in the stereo base line direction and the base line length have an integral multiple, the stereo camera device 20 and the windshield 1a The relative positional relationship of the upper stripe pattern is the same on the left and right. As a result, the striped areas on the left and right camera images have the same fan shape as shown in FIG. 8, and the parallax error can be reduced as described above.

  Further, when the stripe pattern of the windshield 1a is not a direction orthogonal to the base line direction but a diagonal stripe pattern as shown in FIG. 9, if the base line length is an integral multiple of the period in the base line direction, the left and right sides are also left and right. Since the striped patterns captured by the camera images indicate relative positional relationships, the distortion phases on each camera image match.

  In addition, since the windshield 1a, which is one of the curved glass surfaces, has a smooth curved surface shape, the period of the distortion along the glass surface does not exactly match the period of the distortion shown in the image. However, on a baseline length scale of about several tens of millimeters, the windshield 1a is substantially close to a plane, and as described above, even if the distortion phases of the left and right camera images do not exactly match, the opposite phases (FIG. 6 (a) alone is effective in reducing parallax errors.

(Formation process of stereo camera device)
Hereinafter, a process of forming the stereo camera device 20 in the stereo camera system 2 of the embodiment will be described. In the stereo camera system 2 of the embodiment, first, a typical distortion period of the windshield 1a is measured and determined. Then, a base line length between the camera units 20a and 20b of the stereo camera device 20 is determined based on the determined cycle, and the positions of the camera units 20a and 20b are fixed so as to have the determined base line length. The device 20 is formed.

(Measurement process of strain period)
In the distortion period measurement step, a stereo camera device (corresponding to the above-described stereo camera device) provided in the measurement device captures an image of a test chart or the like through the windshield and measures the distortion of the windshield. Specifically, as shown in FIG. 10, a test chart 60 is arranged in front of a fixed stereo camera device 70. Further, a windshield 80 (corresponding to the above-described windshield 1a) is installed between the stereo camera device 70 and the test chart 60 at a distance d [mm] from the stereo camera device 70. Then, in the state where the windshield 80 is installed (the state where the glass is present) and the state where the windshield 80 is not installed (the state where there is no glass), the test chart 60 is imaged.

  Next, the CPU of the measuring device compares the image with glass obtained by imaging the test chart 60 with glass and the image without glass obtained by imaging the test chart 60 with no glass. I do. Then, the CPU of the measuring device searches for a pixel (point) that moves between the image without glass and the image with glass (corresponding point search).

FIG. 11 is a diagram for explaining the corresponding point search operation. In FIG 11, the image position (x, y) indicates the image position of the glass without the image, the image position (x ^{1, y 1)} indicates the image position of the glass there images. The position of the point corresponding to the root of the arrow shown in FIG. 11 indicates the image position before the movement of the glassless image, and the position of the point corresponding to the tip of the arrow shown in FIG. This indicates the image position of the moved image with glass.

Since the user wants to know the period in the base line direction, the CPU of the measuring device has virtually set a number of horizontal lines on the image and inserted the windshield 80 as shown by the dotted horizontal lines in FIG. Is calculated in the horizontal direction at each image position “Δx = x ¹ −x”.

  When the x coordinate on each horizontal line is associated with the image movement amount data in the horizontal direction (x direction), many curves of the camera image shown in FIG. 6 can be collected. The CPU of the measuring instrument performs frequency analysis on the curve of “x−Δx” to detect the peak position of the power spectrum. The reciprocal of the peak frequency corresponds to the distortion period [pix] on the image on the line.

  The CPU of the measuring device sets the distortion period α [pix] on the image, the distance d [mm] between the stereo camera device 70 and the windshield 80, and the focal length f [pix] of the stereo camera device 70, Is calculated using the equation of “d × (α / f)”. In addition, the CPU of the measuring device measures the distortion characteristics of a large number of windshields 80 manufactured under the same conditions, and calculates an average value of a large number of periods. Then, the CPU of the measuring device determines the calculated average value as a representative distortion period of each windshield 80.

(Operation of determining baseline length)
Next, it is assumed that the result of measuring the strain periods of the many windshields 80 as described above is, for example, 30 mm ± 5%. In addition, it is assumed that the base line length, which is the distance between the optical axes of the left and right camera units included in the stereo camera device 70, is required to be, for example, 80 mm or more from the ranging accuracy of the stereo camera device 70.

  The minimum value which is an integral multiple of 30 and is 80 or more is 30 × 3 = 90. Assuming that the base line length magnification is 3, 5 (%) × 3 = 15 (%), so that the image movement amounts of the left and right camera units do not become in opposite phases even if the period variation is considered. For this reason, the base line length of the stereo camera device 70 is fixed at 90 mm.

(Effects of Embodiment)
Thereby, the phase of the image movement of the camera image by the windshield 80 can be matched between the left and right camera units. Therefore, the parallax error due to the distortion of the windshield 80 can be reduced without performing a special calibration process (stereo calibration process), and accurate distance measurement performance can be maintained. Further, since the stereo calibration process of the stereo camera device 70 can be dispensed with, it is possible to prevent inconvenience in complicating the manufacturing process of a vehicle, a ship, an aircraft, or the like provided with the stereo camera device. Further, since the stereo calibration process can be omitted, it is possible to prevent (suppress) an inconvenience of increasing the manufacturing cost of the vehicle.

  Since a subject at a short distance other than infinity originally has parallax, even when the position movement curves of the left and right camera images overlap (see FIG. 6B), the movement amounts are different, resulting in a parallax error. However, since the parallax itself increases as the distance decreases, the parallax error due to the distortion of the windshield 70 is smaller than the error with respect to the parallax.

  Generally, it is more important to reduce the parallax error of a long-distance subject that is greatly affected by the distortion of the windshield 70. Therefore, as in the stereo camera system according to the embodiment, it is very effective to make the phases of the amount of position movement of the left and right camera images uniform.

  In addition, not only the windshield 70 of the vehicle, but also the stereo camera device 20 is often covered with a transparent member such as a glass plate or an acrylic plate, for example, for dust prevention, waterproofing, breakage prevention, and the like. For example, in a case where the left and right camera sections are covered with one glass plate, this embodiment is very effective.

(Modification)
In the description of the above-described embodiment, the stereo calibration process is not required. However, the parallax error caused by the windshield 80 is reduced by the combination of the above-described distortion period and the base line length. A stereo calibration process may be performed. Thereby, the distance measurement performance of the stereo camera device 70 can be improved.

  Finally, the above-described embodiments have been presented by way of example, and are not intended to limit the scope of the present invention. This new embodiment can be implemented in other various forms, and various omissions, replacements, and changes can be made without departing from the spirit of the invention. The embodiments and the modifications of the embodiments are included in the scope and the gist of the invention, and are included in the invention described in the claims and the equivalents thereof.

REFERENCE SIGNS LIST 1 vehicle 1a windshield 2 stereo camera system 20 stereo camera device 20a camera unit for left eye 20b camera unit for right eye 50 vehicle ECU
60 chart 70 stereo camera device 80 windshield

JP-A-2004-132870 JP-A-07-010569

Claims (8)

At least two camera units for imaging a subject based on imaging light transmitted through a transmission member having periodic distortion characteristics,
A stereo camera device comprising: a fixing unit that fixes each of the camera units at a position where a phase of an image movement based on a periodic distortion characteristic of the transmitting member in a camera image captured by each of the camera units matches. The stereo camera device according to claim 1, wherein the fixing unit fixes each of the camera units at a position where the phase of the image movement is equal to or less than 30 degrees. The fixing part,
The product of the variation rate of the strain period of the transmission member and the base length, which is the length of a straight line connecting the optical axes in a direction perpendicular to the optical axis of each camera unit, multiplied by the magnification of the strain period of the transmission member. 3. The stereo camera device according to claim 1, wherein each of the camera units is fixed at a position smaller than 50%. The fixing part,
In the direction orthogonal to the optical axis of each of the camera units, fix each of the camera units at a position where the base line length, which is the length of a straight line connecting the optical axes, is an integral multiple of the distortion period of the transmitting member. The stereo camera device according to claim 1 or 2, wherein: The fixing unit, fixing the camera unit at a position where the base line length realizes a length that can maintain a ranging system required as the stereo camera device,
The stereo camera device according to claim 3 or 4, wherein: The stereo camera device according to any one of claims 1 to 5,
A control unit that controls a control target based on a camera image captured by each camera unit of the stereo camera device,
Stereo camera system having The stereo camera device according to any one of claims 1 to 5,
A control unit that performs at least a predetermined traveling control based on a camera image captured by each of the camera units of the stereo camera device,
A mobile body comprising: The transmission member is a windshield,
The moving body according to claim 7, wherein:

Images