JP2024049283A - Calibration device and calibration method for stereo camera - Google Patents

Abstract

When the optical axis of a stereo camera is calibrated based on collimator light emitted from a pair of collimators, no adjustment is required even if the parallelism of the optical axes of both collimators is misaligned.
[Solution] The stereo camera calibration device includes a stereo camera unit 5 having cameras 5a, 5b, a collimator unit 2 having collimators 2a, 2b, a reference camera unit 2 having reference cameras 2a, 2b, and a control unit 4, wherein the control unit 4 determines the coordinates of the imaging planes of the reference cameras 3a, 3b that receive collimated light emitted from the collimators 2a, 2b, calculates the amount of displacement of the optical axes of the collimators 2a, 2b from the difference between both coordinates, determines the coordinates of the imaging planes of the cameras 5a, 5b that receive the collimated light emitted from the collimators 2a, 2b, calculates the amount of displacement of the optical axes of the cameras 5a, 5b from the difference between both coordinates, and calculates the calibration values of the cameras 5a, 5b from the difference between both displacements.
[Selected Figure] Figure 1

Description

The present invention relates to a stereo camera calibration device and calibration method that calculates the amount of displacement of the optical axis between a first camera and a second camera that constitute a stereo camera unit.

Conventionally, image processing using the so-called stereo method has been known as a technology for recognizing space three-dimensionally. This image processing using the stereo method uses the principle of triangulation to determine distance from the parallax when the same object is imaged using a stereo camera unit equipped with two cameras arranged with a predetermined baseline length.

In image processing using the stereo method, two image signals from a stereo camera unit are sequentially shifted and overlapped to determine the position where the two image signals match. Therefore, it is desirable that there be only a shift in corresponding positions between the two images caused by parallax. Therefore, to accurately calculate the three-dimensional position of an object using a stereo camera unit, it is necessary to calibrate at least the parameters that indicate the positional relationship between the two cameras (hereinafter referred to as "extrinsic parameters").

When calibrating (sometimes called "calibrating") the external parameters between the left and right cameras, as shown in FIG. 8, first, a calibration board Cb is presented in front of the left and right cameras C1, C2 at a distance L that serves as a reference in the depth (z) direction. Then, the two left and right cameras C1, C2 take images of the calibration board Cb, and the external parameters between the left and right cameras C1, C2, particularly the parameters related to the parallax (x) direction, are calibrated based on the images.

To accurately calibrate the external parameters of the left and right cameras C1 and C2, it is necessary to ensure a distance L in the depth (z) direction of several to several tens of meters. However, it is difficult to ensure such sufficient space during the camera calibration process.

As a countermeasure, for example, Patent Document 1 (JP 2019-90755 A) discloses a technology in which a pair of collimators that provide optical conditions similar to those when a calibration board is placed at infinity in the depth direction of the stereo camera are arranged corresponding to the two cameras installed in the stereo camera. The technology disclosed in this document measures the parallax equivalent to infinity by capturing the collimated light (parallel light) emitted from each collimator with each camera of the stereo camera, and then calibrates the stereo camera efficiently and in a space-saving manner based on the measurement data.

JP 2019-90755 A

Incidentally, the technology disclosed in the above-mentioned Patent Document 1 is based on the premise that the collimated light emitted from each collimator is always parallel. The parallelism of a pair of collimators supported by a support is prone to shift over time, so it needs to be inspected and adjusted periodically.

When operating a stereo camera calibration device in a camera calibration process, periodically inspecting and adjusting the parallelism of a pair of collimators supported by a support requires pausing the line that continuously inspects the cameras at regular intervals to adjust the optical axes of the pair of collimators. This results in a decrease in production efficiency.

The present invention aims to provide a stereo camera calibration device and method that can achieve high production efficiency when calibrating the optical axis of a stereo camera unit based on collimator light emitted from a pair of collimators, without the need to temporarily suspend the camera inspection line to adjust the optical axis of the pair of collimators even if the parallelism of the optical axes of the pair of collimators shifts over time.

The present invention relates to a stereo camera calibration device that includes a stereo camera unit having a first camera and a second camera arranged with a predetermined baseline length, a collimator unit having a first collimator and a second collimator fixed to an arm at a distance equal to the baseline length, and a control unit having a camera displacement amount calculation unit that determines the coordinates of the imaging planes of the first camera and the second camera of the stereo camera unit that receive collimated light emitted from the first collimator and the second collimator, and calculates the displacement amount of the optical axis of the first camera and the second camera from the difference between the two coordinates, the stereo camera calibration device including a first reference camera and The control unit further includes a reference camera unit having a second reference camera, and the control unit further includes a collimator displacement amount calculation unit that calculates the coordinates of the imaging plane of the first reference camera and the second reference camera of the reference camera unit that receive the collimated light emitted from the first collimator and the second collimator, and calculates the displacement amount of the optical axis of the first collimator and the second collimator from the difference between the two coordinates, and a calibration value calculation unit that calculates the calibration value of the first camera and the second camera of the stereo camera unit from the difference between the displacement amount calculated by the collimator displacement amount calculation unit and the displacement amount calculated by the camera displacement amount calculation unit.

The present invention relates to a method for calibrating a stereo camera, the method including a stereo camera unit having a first camera and a second camera arranged with a predetermined baseline length, a collimator unit having a first collimator and a second collimator fixed to an arm at a distance equal to the baseline length, and a control unit that obtains coordinates of the imaging plane of the first camera and the second camera of the stereo camera unit that receives collimated light emitted from the first collimator and the second collimator, and calculates the amount of displacement of the optical axis of the first camera and the second camera from the difference between the two coordinates, the method further including a reference camera unit having a first reference camera and a second reference camera fixed with the same baseline length as the baseline length, the control unit obtains coordinates of the imaging plane of the first reference camera and the second reference camera of the reference camera unit that receives collimated light emitted from the first collimator and the second collimator, calculates the amount of displacement of the optical axis of the first collimator and the second collimator from the difference between the two coordinates, and calculates the calibration value of the first camera and the second camera of the stereo camera unit from the difference between the two displacements.

According to the present invention, the coordinates of the imaging plane of the first and second reference cameras of the reference camera unit that receive the collimated light emitted from the first and second collimators of the collimator unit are obtained, and the displacement amount of the optical axis of the first and second collimators is calculated from the difference between these two coordinates. The coordinates of the imaging plane of the first and second cameras of the stereo camera unit that receive the collimated light emitted from the first and second collimators are obtained, and the displacement amount of the optical axis of the first and second cameras is calculated from the difference between these coordinates. The calibration values of the first and second cameras of the stereo camera unit are calculated from the difference between these displacement amounts. Therefore, even if the parallelism of the optical axes of the first and second collimators shifts over time, there is no need to temporarily suspend the camera inspection line to adjust the optical axes of the pair of collimators, and high production efficiency can be obtained.

Schematic diagram of stereo camera unit calibration device Schematic plan view of the collimator unit A schematic plan view showing a state in which collimated light from a collimator unit is received by a reference camera. Schematic diagram showing the calibration of the stereo camera unit on the inspection line FIG. 1 is an explanatory diagram showing a state in which a pair of parallel collimated lights are received by an imaging element of a stereo camera. FIG. 1 is an explanatory diagram showing a state in which a pair of parallel collimated lights are received by the imaging elements of an inclined stereo camera. FIG. 1 is an explanatory diagram showing the calibration of a stereo camera to be calibrated; Flowchart showing a collimator displacement calculation routine Flowchart showing stereo camera calibration processing routine Schematic diagram showing the calibration of extrinsic parameters of a conventional stereo camera unit.

An embodiment of the present invention will be described below with reference to the drawings. As shown in FIG. 1, a stereo camera unit calibration device 1 includes a collimator unit 2, a reference camera unit 3, and a control unit 4. Reference numeral 5 denotes a stereo camera unit to be calibrated. In this embodiment, measuring the deviation of the external parameters of the first and second cameras 5a and 5b provided in the stereo camera unit 5 to be calibrated from a reference position is referred to as "calibration." Correcting image data captured by the first and second cameras 5a and 5b through image processing based on the calibration results is referred to as "correction."

As shown in FIG. 2A, the collimator unit 2 has a first collimator 2a, a second collimator 2b, and a drive unit 2c, and the first and second collimators 2a and 2b are fixed to the ends of an arm 2d that protrudes from the drive unit 2c to the left and right in the figure via a mount base (not shown). Each collimator 2a and 2b has a light source and a collimator lens, and generates and emits parallel light by passing light emitted from the light source through the collimator lens. The collimators 2a and 2b are fixed at a distance equal to the base length of the first camera 5a and the second camera 5b (see FIG. 4) of the stereo camera unit 5 to be calibrated, and the optical axes are initially adjusted to be parallel.

The drive unit 2c rotates the arm 2d in the plan view shown in FIG. 2A and translates it left and right in accordance with a drive signal from the control unit 4. The rotation of the arm 2d is performed, for example, by driving a built-in motor. The translation of the arm 2d is performed, for example, by using a rack-and-pinion mechanism and driving a pinion that meshes with a rack formed on the arm 2d with a motor.

The reference camera unit 3 includes a first reference camera 3a and a second reference camera 3b (see FIG. 2B), and an image processor 3c. Each of the reference cameras 3a and 3b includes an imaging element such as a CCD or CMOS and an objective lens, and the imaging element is disposed at the focal length of the objective lens. In addition, both of the reference cameras 3a and 3b have the same baseline length as the first and second cameras 5a and 5b of the stereo camera unit 5 to be calibrated, and are fixed to an arm (not shown). In FIG. 2B, F1 is the light receiving surface of the imaging element provided in the first camera 5a, and F2 is the light receiving surface of the imaging element provided in the second camera 5b. The image processor 3c converts the analog signals received by the imaging elements of the first and second cameras 5a and 5b into digital signals.

The control unit 4 is composed of a microcontroller equipped with a CPU, RAM, ROM, rewritable non-volatile memory (flash memory or EEPROM), and peripheral devices. The ROM stores programs and fixed data necessary for the CPU to execute each process. The RAM is provided as a work area for the CPU, and temporarily stores various data for the CPU. The CPU is also called an MPU (Microprocessor) or a processor. A GPU (Graphics Processing Unit) or a GSP (Graph Streaming Processor) may be used instead of the CPU. Alternatively, a selective combination of the CPU, GPU, and GSP may be used.

The image processing unit 3c of the reference camera unit 3 is connected to the input side of the control unit 4 via a bus line. In addition, the drive unit 2c of the collimator unit 2 and the image processing/correction unit 5c of the calibration target stereo camera unit 5, which will be described later, are connected to the output side of the control unit 4. Note that data communication between the control unit 4 and the image processing/correction unit 5c of the calibration target stereo camera unit 5 may be possible wirelessly.

As shown in FIG. 3, the first and second reference cameras 3a and 3b of the reference camera unit 3 are suspended from the ceiling 11 of the calibration process section Pc set up on the inspection line. The first and second reference cameras 3a and 3b are oriented in the direction of movement of the conveyor 12 installed on the inspection line and are supported in a horizontal state perpendicular to the conveyor 12.

As shown in FIG. 3, the collimator unit 2 is suspended from the ceiling 11 while being supported by the lifting device 14. As shown by the dashed line in the figure, when the collimator unit 2 is supported by the lifting device 14 and stopped at the standby position at the raised end, the first and second collimators 2a, 2b face the first and second reference cameras 3a, 3b of the reference camera unit 3 with a predetermined distance between them on the moving direction (downstream) side of the conveyor 12.

The conveyor 12 sequentially transports the stereo camera units 5 in the production process. The first and second cameras 5a, 5b of the stereo camera unit 5 to be calibrated are fixed to a fixture or another product (collectively referred to as the "camera support machine 13" below). Analog signals received by the image pickup elements of the first and second cameras 5a, 5b are converted to digital signals by the image processing/correction unit 5c. This image processing/correction unit 5c has the function of correcting images according to the calibration, as described below.

As shown by the dashed line in FIG. 3, the collimator 2 moved to the upper end by the lifting device 14 is in a standby state. In this standby state, the first and second collimators 2a, 2b of the collimator unit 2 face the first and second reference cameras 3a, 3b at a predetermined distance from the moving direction side of the conveyor 12. As the first and second cameras 5a, 5b of the stereo camera unit 5 approach the calibration process section Pc, the lifting device 14 operates to lower the collimator unit 2, and when the stereo camera unit 5 reaches the calibration process section Pc, it stops at the lower end. In this state, the first and second cameras 5a, 5b are presented in front of the first and second collimators 2a, 2b.

Pallets 15 are placed at a predetermined interval on the conveyor 12 of the manufacturing inspection line. The camera support machine 13 is placed on the pallets 15. The control unit 4 controls the lifting operation of the lifting device 14 supporting the collimator unit 2 in synchronization with the timing when the first and second cameras 5a, 5b of the stereo camera unit 5, which moves while fixed to the camera support machine 13, approach the calibration process section Pc.

That is, when the first and second cameras 5a, 5b of the stereo camera unit 5 reach the calibration process section Pc, the control unit 4 operates the lifting device 14 to move the collimator unit 2 to the calibration position at the lower end. Then, the first and second collimators 2a, 2b are presented in front of the first and second cameras 5a, 5b provided on the stereo camera unit 5 to be calibrated. Next, the control unit 4 operates the lifting device 14 to move the collimator unit 2 to the standby position at the upper end, and the first and second collimators 2a, 2b are directly facing the first and second reference cameras 3a, 3b of the reference camera unit 3.

When the collimator unit 2 is stopped at the standby position at the raised end, the control unit 4 causes the image pickup elements of the first and second reference cameras 3a and 3b of the reference camera unit 3 to receive the collimator light emitted from the first and second collimators 2a and 2b of the collimator unit 2. Then, the control unit 4 detects the reference convergence coordinates P1 and P2 based on the image signal from the image processing unit 3c at that time.

When the collimator unit 2 descends to the calibration position, the control unit 4 causes the image sensors of the first and second cameras 5a and 5b of the calibration target stereo camera unit 5 to receive the collimator light emitted from the first and second collimators 2a and 2b of the collimator unit 2. Then, the control unit 4 detects the calibration convergence coordinates P1' and P2' based on the image signal from the image processing/correction unit 5c at that time.

Now, with reference to Figures 4A and 4B, the characteristics of collimated light will be briefly explained using the objective lenses 5aa, 5ba provided in the first and second cameras 5a, 5b of the stereo camera unit 5 to be calibrated that receives the collimated light, and the image sensors 5ab, 5bb arranged at the focal length positions of the objective lenses.

As shown in Figure 4A, when collimated light (parallel light) is incident parallel to the optical axis of the camera, this collimated light is imaged on the optical axis of the image pickup elements 5ab and 5bb by the objective lenses 5aa and 5ba. Therefore, the calibration convergence coordinates P1' and P2' are the centers of the image pickup surfaces F1 and F2. This is also the case when the collimated light is moved in parallel.

On the other hand, when the first and second cameras 5a and 5b are tilted as a whole as shown in FIG. 4B, the incidence angle of the collimated light changes relatively, so that the imaging position for the image pickup elements 5ab and 5bb shifts from the optical axis. Therefore, the calibration convergence coordinates P1' and P2' become positions different from the centers of the image pickup surfaces F1 and F2. However, the first and second collimators 2a and 2b are not always fixed in a fixed position, and tilt is likely to occur due to deterioration over time of the mount base supporting the first and second collimators 2a and 2b due to continuous operation. Note that, although FIG. 2A and FIG. 2B show a state in which only the second collimator 2b is tilted, there are also cases in which only the first collimator 2a is tilted, or in which both the first and second collimators 2a and 2b are tilted.

As a result, the parallelism of the collimated light emitted from the first and second collimators 2a and 2b is no longer guaranteed. However, as shown in FIG. 3, on the inspection line, the collimator unit 2 continuously calibrates the external parameters of the stereo camera unit 5 to be calibrated, which is transported on the conveyor 12. Therefore, in order to periodically adjust the parallelism of the collimated light emitted from the first and second collimators 2a and 2b, the inspection line needs to be stopped each time, resulting in a decrease in production efficiency.

In this embodiment, even if the parallelism of the collimated light emitted from the first and second collimators 2a and 2b provided in the collimator unit 2 is not guaranteed due to deterioration, it is possible to calibrate the external parameters of the first and second cameras 5a and 5b provided in the stereo camera unit 5 to be calibrated without making any modifications.

The calibration of the stereo camera unit 5 to be calibrated by the control unit 4 is specifically performed according to the collimator displacement amount calculation routine shown in FIG. 6 and the stereo camera unit calibration processing routine shown in FIG. 7. The processing in the flowchart shown in FIG. 6 corresponds to the collimator displacement amount calculation unit of the present invention.

The flowchart shown in FIG. 6 is started every preset period (number of units) when the stereo camera unit 5 to be calibrated passes through the calibration process section Pc on the inspection line. First, in step S1, the control unit 4 causes the image pickup elements of the first and second reference cameras 3a and 3b of the reference camera unit 3 to receive the collimator light emitted from the first and second collimators 2a and 2b of the collimator unit 2 stopped at the standby position.

Next, proceeding to step S2, the control unit 4 detects the reference convergence coordinates P1, P2 of the collimator light imaged on the imaging planes F1, F2 based on the image signal from the image processing unit 3c that processes the image received by the imaging element. Then, the drive unit 2c rotates and/or horizontally moves the arm 2d to adjust the reference convergence coordinate P1 (or P2) to fall within a predetermined range with respect to the center of the optical axis of the imaging plane F1 (or F2). As a result, the collimated light from the first collimator 2a (second collimator 2b) becomes approximately parallel to the optical axis of the first reference camera 3a (second reference camera 3b) of the reference camera unit 3.

Then, the process proceeds to step S3, where the control unit 4 detects the reference convergence coordinates P1 and P2, calculates the displacement amount d of the optical axis, and ends the routine.

The displacement amount d is the difference between the coordinates (x, y) of P1 and P2. Since the optical axes of the first and second reference cameras 3a and 3b of the reference camera unit 3 are set parallel, when the displacement amount d is 0, it indicates that the optical axes of the first and second collimators 2a and 2b are parallel. Also, as shown in FIG. 2B, when the displacement amount d is other than 0, it indicates that the optical axes of the first and second collimators 2a and 2b are misaligned.

This displacement amount d is read when executing the stereo camera unit calibration processing routine shown in FIG. 7. This routine is executed in synchronization with the timing when the stereo camera unit 5 to be calibrated, which is fixed to the camera support machine 13 transported by the conveyor 12, approaches the calibration process section Pc.

First, in step S11, the control unit 4 operates the lifting device 14 to lower the collimator unit 2 to the calibration position when the first and second cameras 5a and 5b of the stereo camera unit 5 to be calibrated approach the calibration process unit Pc. Then, the first and second collimators 2a and 2b of the collimator unit 2 are presented in front of the first and second cameras 5a and 5b of the stereo camera unit 5 to be calibrated. Next, the collimator light emitted from the first and second collimators 2a and 2b of the collimator unit 2 is received by the imaging elements of the first and second cameras 5a and 5b of the stereo camera unit 5 to be calibrated.

Then, proceeding to step S12, the control unit 4 detects the calibrated convergence coordinates P1', P2' of the collimator light imaged on the imaging planes F1', F2' based on the image signal from the image processing/correction unit 5c, which processes the image received by the imaging element. The control unit 4 then rotates and/or horizontally moves the arm 2d using the drive unit 2c, adjusting the convergence coordinates P1, P1' (or P2, P2') to be the same coordinate.

The positional relationship between the calibration target stereo camera unit 5 and the collimator unit 2 transported by the conveyor 12 is not always the same, and deviations in the positional relationship can also occur due to installation errors in the camera support machine 13 to which the calibration target stereo camera unit 5 is fixed. Therefore, the positional relationship between the calibration target stereo camera unit 5 and the collimator unit 2 may be tilted relative to each other. Therefore, if collimated light is emitted to the first and second cameras 5a and 5b of the calibration target stereo camera unit 5 with both the first and second collimators 2a and 2b tilted, distortion is likely to occur in the images captured by the first and second cameras 5a and 5b.

In this case, as shown in FIG. 4A, if collimated light is emitted with the first and second collimators 2a and 2b facing the first and second cameras 5a and 5b of the stereo camera unit 5 to be calibrated, the collimated light will be parallel to the optical axes of the first and second cameras 5a and 5b, minimizing image distortion. However, if the optical axes of the first and second collimators 2a and 2b of the collimator unit 2 are not parallel, it is difficult to set both the first and second collimators 2a and 2b parallel to the optical axes of the first and second cameras 5a and 5b of the stereo camera unit 5 to be calibrated.

Therefore, in this embodiment, first, in step S2 of the flowchart shown in FIG. 6, the inclination of the collimator unit 2 is adjusted so that the reference convergence coordinate P1 (or P2) receives light at the center of the optical axis. Then, in step S12, the convergence coordinates P1, P1' (or P2, P2') are adjusted to be the same coordinate, and the displacement amount d' is calculated in step S13 described later using the adjusted calibrated convergence coordinate P1' (or P2') as a reference. This makes it possible to minimize image distortion caused by the inclination of the optical axis of the first and second collimators 2a, 2b relative to the first and second reference cameras 3a, 3b.

Next, the process proceeds to step S13, where the control unit 4 detects the calibration convergence coordinates P1' and P2', calculates the displacement amount d', and proceeds to step S14. The displacement amount d' is the difference between the coordinates (x, y) of P1' and P2'. The process in step S13 corresponds to the camera displacement amount calculation unit of the present invention.

The first and second collimators 2a and 2b of the collimator unit 2 presented in front of the first and second cameras 5a and 5b provided in the calibration target stereo camera unit 5 shown in FIG. 5 are the same as those presented in front of the first and second reference cameras 3a and 3b provided in the reference camera unit 3 shown in FIG. 2B. Therefore, when the inclination of the collimator unit 2 is adjusted so that the convergence coordinates P1 and P1' (or P2 and P2') are the same coordinates, the displacement amount d' appears in the calibration convergence coordinate P2' (or P1') on the side where the coordinates were not adjusted.

Then, proceed to step S14 and read the displacement amount d. Next, proceed to step S15 and compare the displacement amounts d and d'. If the optical axes of the first and second cameras 5a and 5b of the stereo camera unit 5 to be calibrated are parallel and there is no change in the optical axes of the first and second collimators 2a and 2b of the collimator unit 2, the displacement amounts d and d' match (d = d'). On the other hand, if the displacement amounts d and d' do not match (d ≠ d'), it is determined that the optical axes of the first and second cameras 5a and 5b of the stereo camera unit 5 to be calibrated are misaligned.

If the displacements d and d' are the same (d = d'), jump to step S20. If the displacements d and d' are different (d ≠ d'), proceed to step S16. In step S16, a calibration value ε is calculated from the difference between the displacements d and d' (ε = d - d'). In step S12, the control unit 4 adjusts the convergence coordinates P1 and P1' (or P2 and P2') so that they are the same coordinates. Therefore, this calibration value ε becomes a calibration value for the reference position of the first camera 5a (or the second camera 5b) that indicates the calibrated convergence coordinate P2' (or P1') on the side whose coordinates have not been adjusted in step S12. The process in step S16 corresponds to the calibration value calculation unit of the present invention.

Next, the process proceeds to step S17, where the image offset amount Δε that makes the calibration value ε zero is calculated, and in step S18, this image offset amount Δε is stored in the non-volatile memory of the stereo camera unit 5 to be calibrated. This image offset amount Δε becomes the correction amount corresponding to the deviation of the external parameters. The process in step S17 corresponds to the image offset amount calculation unit of the present invention.

Then, proceed to step S19, correct the image of the calibrated first camera 5a (or second camera 5b) based on the image offset amount Δε stored in the non-volatile memory, and proceed to step S20. Note that the processing in step S19 after the first and second cameras 5a, 5b of the stereo camera unit 5 to be calibrated have passed through the calibration process section Pc is independently executed by the image processing/correction section 5c based on the image offset amount Δε stored in the non-volatile memory of the stereo camera unit 5 to be calibrated.

When the process proceeds from step S15 or step S19 to step S20, the control unit 4 operates the lifting device 14 to raise the collimator unit 2 to the standby position, and ends the routine.

As described above, this embodiment includes a collimator unit 2, a reference camera unit 3, and a control unit 4. The control unit 4 first receives collimated light from the first and second collimators 2a and 2b of the collimator unit 2 with the first and second reference cameras 3a and 3b of the reference camera unit 3, and calculates the displacement d of the collimated light caused by the positional shift of the first and second collimators 2a and 2b from the difference between their position coordinates (reference convergence coordinates) P1 and P2.

Next, the control unit 4 irradiates the collimated light from the first and second collimators 2a and 2b of the same collimator unit 2 onto the first and second cameras 5a and 5b of the calibration target stereo camera unit 5, which is fixed to the camera support mechanism 13, and receives the light, and calculates the displacement amount d' from the difference between the position coordinates (calibration convergence coordinates) P1' and P2'.

This displacement amount d' includes both factors due to the optical axis misalignment of the first and second collimators 2a and 2b, and factors due to the external parameters. Therefore, the difference between the displacement amounts d and d' is calculated, and the factors due to the optical axis misalignment of the first and second collimators 2a and 2b are removed to calculate the calibration value ε due to the external parameters of the first and second cameras 5a and 5b of the stereo camera unit 5 to be calibrated.

Therefore, even if the optical axis of the first and second collimators 2a, 2b of the collimator unit 2 is misaligned, there is no need to adjust the optical axes of the first and second collimators 2a, 2b. As a result, the calibration of the first and second cameras 5a, 5b of the stereo camera unit 5 to be calibrated can be performed continuously without having to temporarily stop the inspection line to adjust the optical axes of the first and second collimators 2a, 2b. This allows for high production efficiency.

1...Stereo camera unit calibration device,
2...Collimator unit,
2a, 2b...first and second collimators,
2c...Drive unit,
2d...arm,
3...Reference camera unit,
3a, 3b...first and second reference cameras,
3c...image processing unit,
4...Control unit,
5... Calibration target stereo camera unit,
5aa, 5ba...objective lens,
5a, 5b...first and second cameras,
5a, 5bb...imaging elements,
5c...Image processing/correction unit,
11...Ceiling,
12...Conveyor,
13...Camera support device,
14...Lifting device,
15...Palette,
F1, F2, F1', F2'...imaging surface,
L: distance,
P1, P2...reference convergence coordinates,
P1', P2'...calibration convergence coordinates,
d, d': displacement amount,
Δε: image offset amount,
ε…calibration value

Claims (5)

a stereo camera unit having a first camera and a second camera arranged with a predetermined baseline length;
a collimator unit having a first collimator and a second collimator fixed to an arm at a distance equal to the base length;
a control unit having a camera displacement amount calculation unit that calculates coordinates of an imaging plane of the first camera and the second camera of the stereo camera unit that receive collimated light emitted from the first collimator and the second collimator, and calculates a displacement amount of an optical axis of the first camera and the second camera from a difference between the coordinates,
a reference camera unit including a first reference camera and a second reference camera fixed with the same baseline length as the baseline length;
The control unit is
a collimator displacement amount calculation unit that calculates coordinates of an imaging plane of a first reference camera and a second reference camera of the reference camera unit that receive collimated light emitted from the first collimator and the second collimator, and calculates a displacement amount of an optical axis of the first collimator and the second collimator from a difference between the two coordinates;
The stereo camera calibration device further comprises a calibration value calculation unit that calculates a calibration value of the first camera and the second camera of the stereo camera unit from a difference between the displacement amount calculated by the collimator displacement amount calculation unit and the displacement amount calculated by the camera displacement amount calculation unit. the collimator unit has a drive unit that displaces the arm,
2. The stereo camera calibration device according to claim 1, wherein the control unit drives the drive unit to adjust the collimated light emitted from the first collimator and the second collimator of the collimator unit so that one of the collimated light beams emitted from the first collimator and the second collimator of the collimator unit is received at coordinates within a preset range of an imaging plane of one of the first reference camera and the second reference camera of the reference camera unit that receives the collimated light. 3. The stereo camera calibration device according to claim 2, wherein the control unit drives the drive unit to adjust so that one of the collimated lights emitted from the first collimator and the second collimator of the collimator unit is received at coordinates on an imaging surface of one of the first camera and the second camera of the stereo camera unit that receives the collimated light, the coordinates matching the coordinates received on an imaging surface of one of the first reference camera and the second reference camera of the reference camera unit. The control unit is
4. The stereo camera calibration device according to claim 1, further comprising an image offset amount calculation unit that calculates an image offset amount that makes the calibration value calculated by the calibration value calculation unit zero. a stereo camera unit having a first camera and a second camera arranged with a predetermined baseline length;
a collimator unit having a first collimator and a second collimator fixed to an arm at a distance equal to the base length;
a control unit that calculates coordinates of an imaging plane of the first camera and the second camera of the stereo camera unit that receive collimated light emitted from the first collimator and the second collimator, and calculates a displacement amount of an optical axis of the first camera and the second camera from a difference between the two coordinates,
a reference camera unit including a first reference camera and a second reference camera fixed with the same baseline length as the baseline length;
The control unit is
determining coordinates of an imaging plane of a first reference camera and a second reference camera of the reference camera unit which receive collimated light emitted from the first collimator and the second collimator, and calculating a displacement amount of an optical axis of the first collimator and the second collimator from a difference between the two coordinates;
A stereo camera calibration method, comprising: calculating a calibration value of the first camera and the second camera of the stereo camera unit from a difference between the two displacement amounts.

Images