JP2017116321A - Calibration device for onboard stereo camera - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a calibration device for onboard stereo cameras that performs automatic calibration easily with sufficient frequency.SOLUTION: Provided is a calibration device 1 for onboard stereo cameras that calibrates two cameras that a stereo camera mounted onboard a vehicle has, the calibration device 1 comprising: a brightness detection unit 3 for detecting brightness around the vehicle; a brightness determination unit 13 for determining whether the brightness detected by the brightness detection unit 3 is less than or equal to a threshold; a light projection device 5 for carrying out preset light projection to the imaging range of a stereo camera 4 when it is determined by the brightness determination unit 13 that the brightness is less than or equal to the threshold; and a calibration unit 17 for calibrating two cameras 41, 42 that the stereo camera 4 has, on the basis of the captured image of the stereo camera 4, when the light projection device 5 carried out light projection.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a calibration device for an in-vehicle stereo camera.

  Conventionally, it is known that in-vehicle stereo cameras having two cameras are subject to vibrations and temperature changes on a daily basis, so that relative displacement occurs between the two cameras. In the apparatus described in Patent Document 1 below, in a stereo camera having two cameras installed so that their optical axes are parallel to each other, spot light is projected onto a measurement object of the stereo camera, and the two cameras When the spot lights picked up in step (b) are not on the corresponding line (on the epopular line), it is determined that a relative positional shift has occurred between the two cameras.

JP 2008-1643338 A

  By the way, when a relative positional shift occurs between the two cameras of the in-vehicle stereo camera, it is necessary to bring the vehicle to a dealer or the like for calibration. In order to avoid such trouble, there is a demand for an apparatus that easily performs automatic calibration with sufficient frequency.

  Accordingly, an object of the present invention is to provide a calibration device for an in-vehicle stereo camera that can easily perform automatic calibration with sufficient frequency for the in-vehicle stereo camera.

  One aspect of the present invention is a calibration device for an in-vehicle stereo camera that calibrates two cameras included in a stereo camera mounted on a vehicle, and includes a brightness detection unit that detects brightness around the vehicle; A brightness determination unit that determines whether or not the brightness detected by the brightness detection unit is less than or equal to a threshold value, and imaging by a stereo camera when the brightness determination unit determines that the brightness is less than or equal to the threshold value A light projecting device that performs light projection preset in a range, and a calibration that performs calibration of the two cameras of the stereo camera based on the captured image of the stereo camera when the light projecting device performs light projection A section.

  According to one embodiment of the present invention, automatic calibration can be easily performed with sufficient frequency for an in-vehicle stereo camera.

It is a block diagram which shows the calibration apparatus for vehicle-mounted stereo cameras which concern on this embodiment. It is a side view for demonstrating the light projection of the light projection pattern by the vehicle-mounted light projector. (A) It is a figure which shows the picked-up image by the left camera of a stereo camera. (B) It is a figure which shows the picked-up image by the right camera of a stereo camera. It is a flowchart which shows the process of a calibration apparatus.

  DESCRIPTION OF EMBODIMENTS Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a block diagram showing a calibration apparatus 1 for an in-vehicle stereo camera according to this embodiment. A calibration apparatus 1 shown in FIG. 1 is mounted on a vehicle such as a passenger car and calibrates the in-vehicle stereo camera 4.

[Configuration of calibration device]
The calibration device 1 includes an ECU [Electronic Control Unit] 2 that controls the device centrally. The ECU 2 is an electronic control unit including a CPU [Central Processing Unit], a ROM [Read Only Memory], a RAM [Random Access Memory], and the like. In the ECU 2, various processes are executed by loading a program stored in the ROM into the RAM and executing it by the CPU. The ECU 2 may be composed of a plurality of electronic control units. The ECU 2 is connected to the brightness detection unit 3, the stereo camera 4, and the light projecting device 5.

  The brightness detection unit 3 is an illuminance sensor that is mounted on a vehicle and detects the brightness around the vehicle. The illuminance sensor may be the same as that used for the on / off determination of the auto headlight of the vehicle. The brightness detection unit 3 transmits the detected brightness (illuminance) around the vehicle to the ECU 2. The brightness detection unit 3 is not limited to the illuminance sensor, and the stereo camera 4 may be used as the brightness detection unit. For example, the brightness around the vehicle can be detected by using luminance information included in the captured image of the stereo camera 4.

  The stereo camera 4 is an imaging device that images the periphery of the vehicle. The stereo camera 4 has two left and right cameras (left camera 41 and right camera 42) arranged to reproduce binocular parallax. The left camera 41 and the right camera 42 are provided on the back side of the windshield of the vehicle as an example, and image the front of the vehicle. The left camera 41 and the right camera 42 are provided so that their optical axes are parallel to each other. The left camera 41 and the right camera 42 are provided a predetermined distance apart in the width direction of the vehicle. The stereo camera 4 transmits imaging information in front of the vehicle to the ECU 2. The imaging information of the stereo camera 4 includes information in the depth direction. Note that the stereo camera 4 may have two or more cameras. Further, a plurality of stereo cameras 4 may be provided so as to be able to image the rear side of the vehicle, the left side of the vehicle, and the right side of the vehicle.

  The light projecting device 5 is a projector-type headlight that performs preset light projection. The light projecting device 5 is provided at the front end of the vehicle. The light projecting device 5 projects light to the imaging range of the stereo camera 4 (imaging range common to the left camera 41 and the right camera 42). The light projecting device 5 can project a light projecting pattern having a preset shape in accordance with a control signal transmitted from the ECU 2. The light projection pattern will be described later. Note that a plurality of light projecting devices 5 may be provided behind the vehicle, on the left side of the vehicle, or on the right side of the vehicle so as to be able to project light.

  Next, the functional configuration of the ECU 2 will be described. The ECU 2 includes a camera memory 11, a distortion correction calculation unit 12, a brightness determination unit 13, a projection pattern storage unit 14, a projection control unit 15, a pattern determination unit 16, and a calibration unit 17.

  The camera memory 11 is a memory that stores image data captured by the stereo camera 4. The camera memory 11 is formed in a RAM mounted on the vehicle. The distortion correction calculation unit 12 is a circuit that calculates distortion correction so that the imaging results of the left camera 41 and the right camera 42 of the stereo camera 4 are in parallel equivalence.

  The brightness determination unit 13 determines whether the brightness around the vehicle detected by the brightness detection unit 3 is equal to or less than a threshold value. The threshold value is a value set in advance to determine the start of calibration of the stereo camera 4. This threshold value may be a fixed value.

  The light projection pattern storage unit 14 is a storage unit that stores a light projection pattern projected by the light projecting device 5. The projection pattern storage unit 14 is formed in the HDD mounted on the vehicle. The light projection pattern is a pattern having a shape in which light is drawn on a road surface or the like in a dark place such as in a tunnel. The projection pattern may be an appropriate shape for calibrating the left camera 41 and the right camera 42 of the stereo camera 4.

  The light projecting control unit 15 controls the light projecting by the light projecting device 5 by transmitting a control signal to the light projecting device 5. The light projection control unit 15 performs light projection of the light projection pattern stored in the light projection pattern storage unit 14 when the brightness determination unit 13 determines that the brightness around the vehicle is equal to or less than the threshold value. A control signal is transmitted to the projector 5. The light projecting device 5 projects a light projecting pattern within the imaging range of the stereo camera 4.

  Here, FIG. 2 is a side view for explaining the projection of the projection pattern by the in-vehicle projector. FIG. 2 shows the vehicle M, the road surface R, the imaging range W of the stereo camera 4, and the light projection pattern P projected onto the road surface R by the light projecting device 5. As shown in FIG. 2, the light projecting device 5 projects the light projecting pattern P onto the road surface R included in the imaging range W of the stereo camera 4. The light projection pattern P shown in FIG. 2 shows a smiley face as an example, but a checker pattern, random dots, or the like can be adopted.

  The left camera 41 and the right camera 42 included in the stereo camera 4 capture the light projection pattern P, respectively. FIG. 3A is a diagram illustrating an image captured by the left camera 41 of the stereo camera 4. FIG. 3B is a diagram illustrating an image captured by the right camera 42 of the stereo camera 4. 3A and 3B show white lines L1 to L3. The white line L1 and the white line L2 are boundary lines that form a travel lane in which the vehicle M travels. The white line L2 corresponds to a lane boundary line between the traveling lane and the adjacent lane. The white line L3 is a boundary line that forms an adjacent lane together with the white line L2. For example, the light projection pattern P is projected so as to overlap the white line L1.

  The pattern determination unit 16 determines (recognizes) the light projection pattern P based on the captured image of the stereo camera 4. Based on the information on the projection pattern stored in the projection pattern storage unit 14, the pattern determination unit 16 captures an image captured by the left camera 41 shown in FIG. 3A and the right camera 42 shown in FIG. The light projection pattern P included in each captured image is determined by performing image processing (for example, a well-known pattern recognition process) on the captured images.

  The calibration unit 17 calibrates the left camera 41 and the right camera 42 included in the stereo camera 4 based on the captured image of the stereo camera 4. The calibration unit 17 performs calibration using the light projection pattern projected by the light projecting device 5 (light projection pattern captured by the stereo camera 4). That is, since the left camera 41 and the right camera 42 of the stereo camera 4 are provided so that their optical axes are parallel, when the relative positions of the left camera 41 and the right camera 42 are not displaced, The projection pattern P in FIG. 3A and the projection pattern P in FIG. 3B have the same position in the vertical direction of the captured image. On the other hand, when there is a relative positional shift between the left camera 41 and the right camera 42, the vertical direction of the captured image of the light projection pattern P in FIG. 3A and the light projection pattern P in FIG. Deviation occurs in position. The calibration unit 17 calculates the displacement of the position of the projection pattern P as a parallelization error.

  The calibration unit 17 calibrates the left camera 41 and the right camera 42 so that the parallelization error is equal to or less than the allowable threshold by appropriately adjusting the parameters of the left camera 41 and the right camera 42. The allowable threshold value is a preset threshold value for determining whether or not the error is allowable for ensuring the function of the stereo camera 4. The allowable threshold value may be a fixed value.

[Processing in calibration equipment]
Next, processing of the calibration apparatus 1 according to the present embodiment will be described with reference to FIG. FIG. 4 is a flowchart showing processing of the calibration apparatus 1. The flowchart shown in FIG. 4 is started when the vehicle engine is started.

  As shown in FIG. 4, the ECU 2 of the calibration device 1 determines whether or not the brightness around the vehicle M is equal to or less than the threshold by the brightness determination unit 13 as S10. The brightness determination unit 13 performs the above determination based on the brightness around the vehicle M detected by the brightness detection unit 3. When it is determined that the brightness around the vehicle M is not less than or equal to the threshold (S10: NO), the ECU 2 ends the current process. Thereafter, the ECU 2 repeats the process from S10 again after a predetermined time (for example, 1 second) has elapsed. When it is determined that the brightness around the vehicle M is equal to or less than the threshold (S10: YES), the ECU 2 proceeds to S12.

  In S <b> 12, the ECU 2 performs light projection of a light projection pattern (preset light projection) by the light projecting device 5. The ECU 2 transmits a control signal from the light projecting control unit 15 to the light projecting device 5 based on the information on the light projecting pattern stored in the light projecting pattern storage unit 14. The light projecting device 5 projects a preset light projecting pattern based on the transmitted control signal. When the light projecting device 5 projects the light projection pattern (when the light projection pattern is captured by the stereo camera 4), the ECU 2 proceeds to S14. The light projecting device 5 continues projecting the light projecting pattern until the calibration of the stereo camera 4 in S14 is completed.

  In S <b> 14, the ECU 2 performs calibration of the stereo camera 4 by the calibration unit 17. The calibration unit 17 calculates a parallelization error between the left camera 41 and the right camera 42 included in the stereo camera 4 by using the projection pattern projected by the projector 5. The calibration unit 17 uses the light projection pattern projected by the light projecting device 5 to perform calibration so that the parallelization error between the left camera 41 and the right camera 42 is equal to or less than an allowable threshold.

  The calibration unit 17 repeats the calibration until the parallelization error between the left camera 41 and the right camera 42 is equal to or less than an allowable threshold value. At this time, the light projecting device 5 may project the light projection pattern each time calibration is performed. The ECU 2 performs projection of a projection pattern by the projector 5, calculation of a parallelization error between the left camera 41 and the right camera 42 by the calibration unit 17, and calibration of the left camera 41 and the right camera 42 by the calibration unit 17. Repeat until the parallelization error is below the allowable threshold.

  The ECU 2 ends the current process when the parallelization error of the calibration of the left camera 41 and the right camera 42 is equal to or less than the allowable threshold. Thereafter, the ECU 2 repeats the process from S10 again after a predetermined time (for example, 6 hours) has elapsed.

<Operational effects of the calibration device according to the present embodiment>
According to the calibration device 1 according to the present embodiment described above, when the brightness around the vehicle M is equal to or less than the threshold value, by projecting a preset projection pattern from the projector 5, Since the left camera 41 and the right camera 42 of the stereo camera 4 are calibrated, it is not necessary to bring the vehicle M to a dealer or the like, and automatic calibration is easily performed with sufficient frequency to the in-vehicle stereo camera 4. be able to. Further, in this calibration device 1, since it is difficult to capture the projection pattern when the periphery of the vehicle M is bright, calibration using the projection pattern is performed when the brightness of the periphery of the vehicle M is equal to or less than a threshold value. By doing so, the accuracy of calibration can be improved.

  As mentioned above, although embodiment of this invention was described, this invention is not restricted to embodiment mentioned above. The present invention can be implemented in various forms including various modifications and improvements based on the knowledge of those skilled in the art including the above-described embodiments. As a modified example from the present invention, whether or not the brightness around the vehicle M is equal to or less than the threshold value according to the time or the position of the vehicle M in the map information without providing a brightness detection unit such as an illuminance sensor. If it is estimated that the brightness around the vehicle M is less than or equal to the threshold value, a mode of performing calibration is conceivable.

  DESCRIPTION OF SYMBOLS 1 ... Calibration apparatus, 2 ... ECU, 3 ... Brightness detection part, 4 ... Stereo camera, 5 ... Projection apparatus, 11 ... Camera memory, 12 ... Distortion correction calculating part, 13 ... Brightness determination part, 14 ... Projection pattern storage unit, 15 ... Projection control unit, 16 ... Pattern determination unit, 17 ... Calibration unit, 41 ... Left camera, 42 ... Right camera, L1-L3 ... White line, M ... Vehicle, P ... Projection pattern , R: road surface, W: imaging range of a stereo camera.

Claims (1)

A calibration device for an in-vehicle stereo camera that calibrates two cameras of a stereo camera mounted on a vehicle,
A brightness detector for detecting brightness around the vehicle;
A brightness determination unit for determining whether the brightness detected by the brightness detection unit is equal to or less than a threshold;
A light projecting device that performs light projection preset in an imaging range of the stereo camera when the brightness determination unit determines that the brightness is equal to or less than the threshold;
A calibration unit that calibrates the two cameras of the stereo camera based on a captured image of the stereo camera when the light projecting device performs the light projection;
A calibration device for an in-vehicle stereo camera.

Images