JP2020053898A - Calibration apparatus, calibration system, camera, movable body, and calibration value adjustment method - Google Patents

Abstract

To improve the accuracy in calibrating a camera mounted on a movable body.SOLUTION: A calibration apparatus 1 comprises: a camera 2; a memory 3 that stores three-dimensional positions of four or more calibration points located at at least two or more different distances from the camera 2 and a mounting position and a mounting angle of the camera 2; a processor 4 that creates a display image in which four or more markers for calibration corresponding respectively to the four or more calibration points are displayed superimposed on an image photographed by the camera 2; and an interface unit 5 that displays the display image and gives the processor 4, from the outside, an instruction to change superimposition display positions of the respective markers for calibration. The processor 4 calculates calibration values for performing six-axis adjustment of the mounting position and mounting angle of the camera 2, so as to reduce an index representing the difference between estimated display positions on the photographed image of the four or more calibration points estimated from the three-dimensional positions of the calibration points, the mounting position, and the mounting angle, and the display positions of the markers for calibration on the display image.SELECTED DRAWING: Figure 1

Description

  The present disclosure relates to a calibration device, a calibration system, a camera, a moving object, and a calibration value adjustment method.

  2. Description of the Related Art Conventionally, a technique for detecting an object or the like around a vehicle using an image captured by a camera attached to the vehicle has been known. The mounting position and the mounting angle of the camera mounted on the vehicle may deviate from the design values due to a mounting error or a temporal change after the mounting. For this reason, there has been proposed a parameter adjustment method capable of easily and accurately adjusting parameters relating to a mounting position and a mounting angle of a camera used for calibrating a captured image (for example, see Patent Document 1).

  For example, according to the adjustment method described in Patent Document 1, an object is arranged at a predetermined position, and a camera is arranged at a mounting position according to a real image of the object captured by the camera and a parameter stored in a storage unit. And the adjustment image of the object imaged by is displayed in a superimposed manner. Then, specific parameters such as height and depression angle are adjusted so that the real image of the object and the adjustment image match.

JP 2000-209577 A

  However, in the conventional technology, there is room for improving the calibration accuracy in the technology for calibrating the mounting position and the mounting angle of the camera mounted on the vehicle.

  Accordingly, it is an object of the present invention to focus on these points and to provide a calibration device, a calibration system, a camera, a moving object, and a calibration value adjustment method with improved calibration accuracy.

  The calibration device according to the present disclosure includes a camera mounted on a moving body, a memory, a processor, and an interface unit. The memory stores three-dimensional positions of four or more calibration points located at least two or more different distances from the camera, and a mounting position and a mounting angle of the camera. The processor generates a display image in which four or more calibration markers respectively corresponding to the four or more calibration points are superimposed on a captured image of the camera. The interface unit displays the display image and externally gives the processor an instruction to change the superimposed display position of each of the calibration markers. The processor estimates an estimated display position of the four or more calibration points on the captured image from the three-dimensional position of the calibration point, the mounting position, and the mounting angle. The processor adjusts the mounting position and the mounting angle of the camera by six axes so as to reduce an index indicating a difference between the estimated display position and the superimposed display position of the calibration marker on the display image. Calculate the calibration value.

  The calibration system of the present disclosure includes a calibration device and a calibration point. The calibration device includes a camera mounted on the moving object, a memory, a processor, and an interface unit. The memory stores three-dimensional positions of four or more calibration points located at least two or more different distances from the camera, and a mounting position and a mounting angle of the camera. The processor generates a display image in which four or more calibration markers respectively corresponding to the four or more calibration points are superimposed on a captured image of the camera. The interface unit displays the display image and externally gives the processor an instruction to change the superimposed display position of each of the calibration markers. The processor estimates an estimated display position of the four or more calibration points on the captured image from the three-dimensional position of the calibration point, the mounting position, and the mounting angle. The processor adjusts the mounting position and the mounting angle of the camera by six axes so as to reduce an index indicating a difference between the estimated display position and the superimposed display position of the calibration marker on the display image. Calculate the calibration value. The calibration points are four or more real space calibration points corresponding to the four or more calibration points stored in the memory.

  A camera according to an embodiment of the present disclosure is a camera mounted on a moving object, and includes an image sensor, a memory, and a processor. The memory stores three-dimensional positions of four or more calibration points located at least two or more different distances from the camera, and a mounting position and a mounting angle of the camera. The processor generates a display image in which four or more calibration markers respectively corresponding to the four or more calibration points are superimposed on an image captured by the image sensor. The processor displays the display image on an external interface unit, and receives an instruction to change a superimposed display position of each of the calibration markers from the interface unit. The processor estimates an estimated display position of the four or more calibration points on the captured image from the three-dimensional position of the calibration point, the mounting position, and the mounting angle. The processor adjusts the mounting position and the mounting angle of the camera by six axes so as to reduce an index indicating a difference between the estimated display position and the superimposed display position of the calibration marker on the display image. Calculate the calibration value.

  The moving object of the present disclosure includes a calibration device. The calibration device includes a camera, a memory, a processor, and an interface unit. The memory stores three-dimensional positions of four or more calibration points located at least two or more different distances from the camera, and a mounting position and a mounting angle of the camera. The processor generates a display image in which four or more calibration markers respectively corresponding to the four or more calibration points are superimposed on a captured image of the camera. The interface unit displays the display image and externally gives the processor an instruction to change the superimposed display position of each of the calibration markers. The processor estimates an estimated display position of the four or more calibration points on the captured image from the three-dimensional position of the calibration point, the mounting position, and the mounting angle. The processor adjusts the mounting position and the mounting angle of the camera by six axes so as to reduce an index indicating a difference between the estimated display position and the superimposed display position of the calibration marker on the display image. Calculate the calibration value.

  The calibration value adjustment method according to the present disclosure includes acquiring a captured image including four or more calibration points located at least two or more different distances with a camera mounted on a moving body. The calibration value adjustment method includes generating a display image in which four or more calibration markers respectively corresponding to the four or more calibration points are superimposed on the captured image of the camera. Further, the calibration value adjusting method includes receiving, from the outside, a change instruction for changing a superimposed display position of the calibration marker at a position corresponding to the calibration point on the display image. Further, the calibration value adjusting method includes obtaining coordinates of the calibration marker in the display image. Further, the calibration value adjusting method includes estimating an estimated display position of the calibration point on the captured image from a pre-stored mounting position and mounting angle of the camera, and a three-dimensional position of the calibration point. Including. Further, the calibration value adjusting method may include setting the mounting position and the mounting angle of the camera on six axes so as to reduce the difference between the superimposed display position of the calibration marker acquired from the display image and the estimated display position. Calculating a calibration value to be adjusted.

  According to the embodiments of the present disclosure, it is possible to improve the calibration accuracy of a camera mounted on a moving body.

FIG. 3 is a functional block diagram illustrating a schematic calibration of the calibration device. FIG. 2 is a diagram illustrating an arrangement of components of a calibration system including the calibration device of FIG. 1 mounted on a vehicle. 4 is a flowchart illustrating an example of a calibration value adjustment process performed by a control unit illustrated in FIG. 1. FIG. 2 is a diagram illustrating an arrangement of a vehicle equipped with the calibration device illustrated in FIG. 1 and a first example of a calibration pattern. FIG. 2 is a diagram illustrating a first example of a display image displayed on the interface unit in FIG. 1. FIG. 4 is a diagram illustrating a first example of an adjustment confirmation screen displayed on the interface unit in FIG. 1. It is a figure explaining the 2nd example of a pattern for calibration. It is a figure showing the 2nd example of a display picture. It is a figure showing the 2nd example of an adjustment check screen. It is a figure explaining the 3rd example of a pattern for calibration. It is a figure showing the 3rd example of a display picture. It is a figure showing the 3rd example of an adjustment check screen. It is a figure explaining the 4th example of a pattern for calibration. It is a figure showing the 4th example of a display picture. It is a figure showing the 4th example of an adjustment check screen. It is a figure explaining the 5th example of a pattern for calibration. It is a figure showing the 5th example of a display picture. It is a figure showing the 5th example of an adjustment check screen. It is a figure explaining the 6th example of a pattern for calibration. It is a figure showing the 6th example of a display picture. It is a figure explaining the 7th example of a pattern for calibration. It is a figure showing the 7th example of a display picture. It is a figure explaining the 8th example of a pattern for calibration. It is a figure showing the 8th example of a display picture. FIG. 2 is a functional block diagram illustrating a schematic configuration of a camera.

  Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. The drawings used in the following description are schematic. The dimensional ratios and the like in the drawings do not always match actual ones.

<Configuration of calibration device>
As illustrated in FIG. 1, a calibration device 1 according to an embodiment of the present disclosure includes a camera 2, a memory 3, a control unit 4 (processor), and an interface unit 5. The calibration device 1 is a device for calibrating an image of the camera 2 mounted on the moving body 100 (see FIG. 2). As described later, the memory 3 and the control unit 4 can be included in the camera 2.

  The “moving object” in the present disclosure includes a vehicle, a ship, and an aircraft. The “vehicle” in the present disclosure includes an automobile and an industrial vehicle, but is not limited thereto, and may include a railway vehicle, a living vehicle, and a fixed-wing aircraft traveling on a runway. Automobiles include, but are not limited to, passenger cars, trucks, buses, motorcycles, trolley buses, and the like, and may include other vehicles traveling on roads. Industrial vehicles include industrial vehicles for agriculture and construction. Industrial vehicles include, but are not limited to, forklifts and golf carts. Industrial vehicles for agriculture include, but are not limited to, tractors, tillers, transplanters, binders, combines, and lawnmowers. Industrial vehicles for construction include, but are not limited to, bulldozers, scrapers, excavators, crane trucks, dump trucks, and road rollers. Vehicles include those that run manually. The classification of the vehicle is not limited to the above. For example, an automobile may include an industrial vehicle that can travel on a road, and a plurality of classifications may include the same vehicle. Ships in the present disclosure include marine jets, boats, and tankers. The aircraft according to the present disclosure includes a fixed wing aircraft and a rotary wing aircraft.

  The camera 2 captures an image outside the moving body 100 and outputs the captured image to the control unit 4 as an electric signal. The camera 2 includes, but is not limited to, a rear camera, a front camera, a left side camera, a right side camera, and the like. The rear camera, the front camera, the left side camera, and the right side camera are installed on the vehicle such that the rear, front, left, and right peripheral areas of the vehicle can be imaged. For example, the rear camera, the front camera, the left side camera, and the right side camera can each perform wide-angle shooting. In one embodiment, as shown in FIG. 2, the camera 2 is a rear camera attached to the moving body 100 so that an image of the rear of the moving body 100 can be taken. Hereinafter, the camera 2 will be described as a rear camera. The camera 2 may be attached in a posture in which the optical axis O is inclined downward from the horizontal direction as shown in FIG.

  The camera 2 includes an imaging optical system 2a and an imaging device 2b. The imaging optical system 2a includes one or more optical elements such as lenses. The imaging optical system 2a forms an image of a subject on the light receiving surface of the imaging element 2b. The imaging element 2b includes a CCD image sensor (Charge-Coupled Device Image Sensor) and a CMOS image sensor (Complementary MOS Image Sensor). The imaging element 2b can convert the image of the subject into an electric signal and output the electric signal to the control unit 4.

  The memory 3 may be configured using one or more of a semiconductor memory, a magnetic memory, an optical memory, and the like. Semiconductor memory may include volatile memory and non-volatile memory. The magnetic memory may include, for example, a hard disk and a magnetic tape. The optical memory may include, for example, a CD (Compact Disc), a DVD (Digital Versatile Disc), and a BD (Blu-ray (registered trademark) Disc).

  The memory 3 stores the three-dimensional positions of four or more calibration points, and the mounting position (X, Y, Z) and mounting angle (ψ, θ, φ) of the camera 2. The four or more calibration points include calibration points located at at least two or more different distances from the camera 2. Information on the three-dimensional positions of four or more calibration points is predetermined for calibration of the camera 2 and stored in the memory 3. The mounting position (X, Y, Z) and mounting angle (ψ, θ, φ) of the camera 2 define a relative position and angle relationship with the moving body 100 on which the camera 2 is mounted. The mounting position (X, Y, Z) and mounting angle (ψ, θ, φ) of the camera 2 stored in the memory 3 are values determined at the time of designing the moving body 100 on which the camera 2 is mounted, or past values. Are values updated by the calibration process.

Prestored 4 points or more calibration points and P _i into the memory 3, the coordinates _{(X pi, Y pi, Z} pi) and. Here, i is an integer from 1 to n. n corresponds to the number of calibration points. In the present embodiment, the description is made on the assumption that n = 4. However, the present invention is not limited to this, and n ≧ 5. Each calibration point P _i is arranged at a predetermined position in the photographing range of the camera 2. Each calibration point P _i may be a point on a horizontal plane on which the tire of the moving body 100 is mounted. The horizontal plane on which the tire of the moving body 100 is mounted is also referred to as an arrangement surface on which the moving body 100 is arranged.

The mounting position (X, Y, Z) is the position of the camera 2 with respect to the moving body 100 stored in the memory 3. In the present embodiment, as shown in FIG. 2, the position of the camera 2 is perpendicular to the Y-axis direction that is a direction toward the rear of the moving body 100, the Z direction that is a vertical direction, and the Y-axis direction and the Z-axis direction. The direction is indicated using the X-axis direction. The mounting position (X ₀ , Y ₀ , Z ₀ ) stored in the memory 3 at a certain point in time can be used as an initial value of the mounting position (X, Y, Z) when performing a repetitive calculation described later. The initial value of the mounting position (X, Y, Z) at the time of performing the repetitive calculation is a design value (X _D , Y _D , Z _D ) of the mounting position when the moving body 100 mounted with the camera 2 is manufactured. May be used.

The mounting angle (ψ, θ, φ) is the angle of the camera 2 with respect to the moving body 100 stored in the memory 3. The angle of the camera 2 uses a tilt angle ψ, which is a rotation angle around the X axis, a roll angle θ, which is a rotation angle around the Y axis, and a pan angle φ, which is a rotation angle around the Z axis. Shown. The mounting angles (ψ ₀ , θ ₀ , φ ₀ ) stored in the memory 3 at a certain point in time can be used as initial values of the mounting angles (θ, θ, φ) when performing a repetitive calculation described later. As the mounting angles (ψ, θ, φ), design values (ψ _D , θ _D , φ _D ) of the mounting angles when assembling the moving body 100 on which the camera 2 is mounted can be used.

  The control unit 4 includes one or a plurality of processors. The processing performed by the control unit 4 can be rephrased as the processing performed by the processor. The processor includes a general-purpose processor that executes a specific function by reading a specific program, and a special-purpose processor specialized for a specific process. The dedicated processor includes an application specific integrated circuit (ASIC). The processor includes a programmable logic device (PLD; Programmable Logic Device). The PLD includes an FPGA (Field-Programmable Gate Array). The control unit 4 may be any one of SoC (System-on-a-Chip) and SiP (System In a Package) in which one or more processors cooperate.

  As shown in FIG. 1, the control unit 4 can include each functional block of an image processing unit 41 and an arithmetic processing unit 42. The image processing unit 41 performs image processing on an image captured by a camera. The arithmetic processing unit performs arithmetic processing for calculating a calibration value. The processing performed by each functional block can be rephrased as the processing performed by the control unit 4. The control unit 4 is connected to the camera 2, the memory 3, and the interface unit 5 via a wired or wireless communication line, respectively, and can transmit and receive information. The control unit 4 can control the camera 2, the memory 3, and the interface unit 5. In the normal operation mode in which the calibration value is not adjusted, the control unit 4 determines the latest mounting position (X, Y, Z) and mounting angle (ψ, θ, φ) stored in the memory 3 based on the calibration value. Thus, the calibration of the image captured by the camera 2 can be performed.

  The memory 3 and the control unit 4 are stored in the same housing as shown in FIG. 2 and can be housed inside the moving body 100 which is a vehicle. For example, the memory 3 and the control unit 4 can be arranged below the front windshield. As described above, the memory 3 and the control unit 4 may be built in the camera 2.

  The interface unit 5 is a device for a user of the calibration device 1 to perform input and output with respect to the calibration device 1. The interface unit 5 can be configured to include, for example, a touch panel. The interface unit 5 can display a display image 10 captured by the camera 2 and processed by the image processing unit 41 of the control unit 4. The interface unit 5 can receive an input from the user using a fingertip of the user or a specific tool such as a mouse or a stylus pen. The interface unit 5 can share the touch panel display device of the moving body 100 with another device. In this embodiment, the “user” is, for example, a worker at a car dealership or a service base.

<Calibration value adjustment processing>
Next, a calibration value adjustment process performed by the control unit 4 of the calibration device 1 will be described in detail with reference to the flowchart of FIG.

  This calibration value adjustment process is performed after the camera 2 is attached to the moving body 100 at the time of manufacturing, in order to adjust the mounting position (X, Y, Z) and mounting angle (ψ, θ, φ) of the camera 2. It is assumed that it is performed at a place other than the manufacturing facility. For example, when the moving body 100 is a vehicle, such a calibration value may be calculated at an automobile dealer or a service base.

  Prior to adjustment of the calibration value, as shown in FIG. 4, the moving body 100 and the calibration sheet 6 are accurately positioned so as to have a predetermined positional relationship. The calibration sheet 6 can be a movable sheet. The calibration sheet 6 is arranged on a horizontal plane (arrangement plane) on which the tires of the moving body 100 are mounted. The calibration sheet 6 is arranged at a predetermined angle from a reference position of the vehicle at a predetermined distance. The reference position can be the position where the camera 2 is attached or the center of the rear end of the moving body 100. In FIG. 4, phantom lines along the X-axis direction and the Y-axis direction are indicated by broken lines. The two dashed lines intersect at the rear end of the mobile unit 100. In the example of FIG. 4, the calibration sheet can be arranged with the intersection of the broken lines on the installation surface as the reference position indicating the origin in the real space.

FIG. 4 shows a calibration pattern M _{1 according} to a first example drawn on the calibration sheet 6. Calibration pattern M ₁ according to the first example has the shape of a quadrilateral which is displayed by a solid line representing the outer periphery. Each vertex of the quadrilateral of the calibration pattern M ₁ is used as a calibration point P ₁ to P _4. That is, the calibration point P ₁ to P ₄ are located on the outer periphery of the quadrilateral. Calibration points P ₁ to P ₄ are calibration points of the real space corresponding to the calibration point P ₁ to P ₄ stored in the memory 3. When the positioning is correctly performed, the three-dimensional positions of the _four calibration points P _{1 to} P ₄ on the calibration sheet 6 coincide with the three-dimensional positions of the calibration points P _{1 to} P ₄ stored in the memory 3. As an example, the three-dimensional position of each of the calibration points P _{1 to} P ₄ is expressed in units of meters (1, 2, 0), (−1, 2, 0), (−1, 1, 0), (1, 1,0). 3-dimensional position in the real space of each calibration point P ₁ to P ₄ is not limited thereto.

A calibration point P ₁ to P ₄ that are included in the calibration apparatus 1 and the calibration pattern M ₁ constitutes a calibration system. The calibration device 1 and the calibration points P _{1 to} P ₄ accurately positioned with respect to the calibration device enable adjustment processing of the calibration value.

  When accurate positioning is performed between the moving body 100 and the calibration sheet 6. The control unit 4 acquires a captured image from the camera 2 (Step S01). The image processing unit 41 of the control unit 4 can perform arbitrary processing such as distortion correction, brightness adjustment, contrast adjustment, and gamma correction on the captured image acquired from the camera 2.

The image processing unit 41 further displays _four adjustment frames C _{1 to} C ₄ (calibration markers) corresponding to the four calibration points P _{1 to} P ₄ stored in the memory 3 respectively. The display image 10 is generated (Step S02). The image processing unit 41 causes the interface unit 5 to display the display image 10 generated as shown in FIG.

In order to display the adjustment frames C _{1 to} C ₄ , the control unit 4 calculates estimated display positions f _{1 to} f _{4 at} which _four calibration points P _{1 to} P ₄ are estimated to be located on the display image 10. The estimated display positions f _{1 to} f ₄ are two-dimensional quantities having values of the X coordinate and the Y coordinate on the display image 10. The control unit 4 converts the estimated display positions f _{1 to} f ₄ into three-dimensional positions (X _pi , Y _pi , Z _pi ) of _four calibration points P _{1 to} P ₄ stored in the memory 3 (i = 1 4) and the mounting position (X ₀ , Y ₀ , Z ₀ ) and mounting angle (ψ ₀ , θ ₀ , φ ₀ ) of the camera 2. When the memory 3 has a mounting position (X _D , Y _D , Z _D ) and a mounting angle (ψ _D , θ _D , φ _D ) which are design values at the time of manufacturing, the control unit 4 estimates using these. it may calculate the display position f ₁ ~f _4. The calculation of the estimated display positions f _{1 to} f ₄ may be performed by the arithmetic processing unit 42. The image processing unit 41 may be the estimated display position f ₁ ~f ₄ to display the adjustment frame C ₁ -C ₄ so that the center.

Adjustment frame C ₁ -C _4, as shown in FIG. 5 is a frame shape such as circular or elliptical form having a transparent inner region (interior portion of the frame). Interior region of the adjusting frame C ₁ -C ₄ is greater than the intersecting region a region of the calibration points P ₁ to P ₄ are in two adjacent sides of the calibration pattern M ₁ on the display image 10.

If the mounting position (X ₀ , Y ₀ , Z ₀ ) and mounting angle (ψ ₀ , θ ₀ , φ ₀ ) of the camera 2 stored in the memory 3 are correct, the calibration points P _{1 to} P ₄ on the display image 10 and coordinates the first displayed center coordinates of the adjustment frame C ₁ -C ₄ matches. On the other hand, the mounting position (X ₀ , Y ₀ , Z ₀ ) and mounting angle (ψ ₀ , θ) in which the actual mounting position (X, Y, Z) and mounting angle (）, θ, φ) are stored in the memory 3. ₀ , φ ₀ ), the coordinates of the calibration points P _{1 to} P ₄ do not match the center coordinates of the adjustment frames C _{1 to} C ₄ displayed first. Therefore, the control unit 4 changes the positions of the adjustment frames C _{1 to} C ₄ from the interface unit 5 by the user in order to adjust the mounting position (X, Y, Z) and the mounting angle (ψ, θ, φ). Is received (step S03). In the above, in the case of a circle or an ellipse, the center coordinate means the coordinate of the center. In the case of a rectangle or a diamond, it means the coordinates of the center of gravity. Center coordinates of the adjustment frame C ₁ -C ₄ are representative of superimposed display position of the adjustment frame C ₁ -C _4.

Here, an example of a position change process of adjusting frame C ₁ -C ₄ via the interface unit 5. As shown in FIG. 5, a selection button 12, a position adjustment button 13, and an OK button 14 are displayed on the interface unit 5 together with the display image 10. These buttons can be buttons or icons on a touch panel. The selection button 12, the position adjustment button 13, and the OK button 14 are operated on the GUI to activate processing described below.

The user of the calibration device 1 gives an instruction to change the superimposed display position of the respective adjustment frames C _{1 to} C _{4 on} the display image 10 to the control unit 4 via the interface unit 5 from the outside. First, the user selects the adjustment frame C ₁ -C ₄ for adjusting the selection button 12. Pressing the selection button 12, one of the adjustment frame C ₁ -C ₄ is selected. The selected adjustment frame C ₁ -C ₄ was is presented to the user by a change in the display mode such as a color change or blinking. Each time the user presses the selection button 12, sequentially switched adjusting frame C ₁ -C ₄ are selected.

The user can use the position adjustment buttons 13 to change the positions of the selected adjustment frames C _{1 to} C ₄ . As shown in FIG. 5, the position adjustment button 13 is composed of four buttons indicating up, down, left, and right directions, and by pressing these buttons, the adjustment frame C being selected in each of the up, down, left, and right directions. moving the ₁ -C ₄ may be aligned with. The user, the adjustment frame C ₁ -C _4, corresponding calibration point P ₁ to P ₄ is moved so as to enter the interior region of the adjusting frame C ₁ -C _4. The user, each calibration point P ₁ to P ₄ are adjusted to be in the center of the inside region as possible adjustment frame C ₁ -C _4.

The size of the inner region of the adjustment frame C ₁ -C ₄ is determined from the adjustment accuracy. The size of the inner region of the adjustment frame C ₁ -C _4, the adjustment specifications, in other words, is set to fit tolerances. That is, the greater the tolerance adjustment frame C ₁ -C ₄ is large. Smaller tolerances, adjusting frame C ₁ -C ₄ is small.

When the adjustment of the positions of all the adjustment frames C _{1 to} C ₄ is completed, the user presses the OK button 14. Upon depression of the OK button 14, the control unit 4, from the interface unit 5, change instruction and superimposed display position of each of the adjustment frame C ₁ -C _4, superimposed on each adjustment frame C ₁ -C ₄ on the display image 10 Receives center coordinates that are display positions.

The control unit 4 having acquired the center coordinates of the adjustment frames C _{1 to} C ₄ adjusts the mounting position (X, Y, Z) and the mounting angle (ψ, θ, φ) of the camera 2 by six axes. Is calculated (step S04). The six-axis adjustment means adjusting the position of each of the three orthogonal axes in the three-dimensional space in the respective directions and the angle around each of the three axes. Hereinafter, a method of calculating the calibration value will be described.

The arithmetic processing unit 42 determines the center coordinates of the changed adjustment frames C _{1 to} C ₄ , that is, the superimposed display positions, as the actual display positions I _{1 to} I ₄ on the display image 10 of the calibration points P _{1 to} P _4. I do. The actual display positions I _{1 to} I ₄ are two-dimensional quantities having the values of the X coordinate and the Y coordinate on the display image 10. The arithmetic processing unit 42 sets the mounting position (X, Y, Z) and mounting angle (ψ) so as to reduce the index indicating the difference between the estimated display positions f _{1 to} f ₄ and the actual display positions I _{1 to} I _4. , Θ, φ). For the calculation of the calibration value, the least square method can be used as follows.

First, the position and orientation α, which is a six-dimensional quantity including the parameters of the mounting position (X, Y, Z) and the mounting angle (ψ, θ, φ) to be calibrated, is defined as follows.
α = [X, Y, Z, ψ, θ, φ] ^T (1)
The estimated display positions f _{1 to} f ₄ are functions of the position and orientation α.

となる。 If the actual position and orientation of the camera 2 coincides with the design values or the position and orientation α that is configured most recently, then I _i = f _i (α) (i = 1 to 4). However, due to a shift of the mounting position (X, Y, Z) or a shift of the mounting angle (，, θ, φ) of the camera 2, there is a difference between the actual display position I _i and the estimated display position f _i (α). , Error L _i (α). That is,

Becomes

Therefore, the arithmetic processing unit 42 performs an arithmetic operation to minimize the error L _i (α) by the least square method. The error L _i (α) is minimized by minimizing an index corresponding to the error L _i (α). Specifically, by the sum of squares for each calibration point P _i of the error L _i (alpha) is the error function e (alpha), and calculates the position and orientation alpha to minimize this, position (X, Y , Z) and posture (ψ, θ, φ) calibration values can be calculated. The error function e (α) is an index corresponding to the error _Li (α). The error function e (α) is expressed by the following equation.

The minimization of the error function e (α) can be performed by an iterative calculation. That is, the arithmetic processing unit 42 adjusts the parameters of each position and orientation α in a direction to decrease the error function e (α). The arithmetic processing unit 42 recalculates the estimated display position f _i (α), the error L _i (α), and the error function e (α) of the calibration point P _i using the changed position and orientation α. The arithmetic processing unit 42 adjusts the position and orientation α until the error function e (α) becomes the minimum value or its vicinity, and estimates the display position f _i (α) of the calibration point P _i and the error L _i (α). And the recalculation of the error function e (α) is repeatedly executed.

  The fact that the error function e (α) is at or near the minimum value can be determined by various methods. For example, the arithmetic processing unit 42 may set a threshold value for the error function e (α), and terminate the iterative calculation when the error function e (α) becomes smaller than the threshold value. The arithmetic processing unit 42 calculates an error function e (α) before and after each repetition calculation or a change amount of a parameter of each position and orientation α, and when the change amount becomes a predetermined value or less, the error function e ( α) may be determined to have converged to or near the minimum value, and the repetition of the calculation may be terminated. Each parameter of the position and orientation α at the end of the iterative calculation becomes a calibration value of the mounting position (X, Y, Z) and the mounting angle (ψ, θ, φ).

  Various well-known calculation methods such as a steepest descent method, a Newton method, a Gauss-Newton method, and a Levenberg-Marquardt method can be used for the method of performing the iterative calculation.

  As an example, when the Gauss-Newton method is used, the arithmetic processing unit 42 searches for the minimum value of the error function e (α), so that the convergence condition set in parameters such as the error function e (α) or the position and orientation α is used. Iterative calculation is performed until is satisfied. As an example, the search for the minimum value of the error function e (α) can be performed by iterative calculation until the following Δα is converged.

ここで、Δαは反復計算による位置姿勢αの変化量である。Ｊは、αにおけるＬ_i（α）のヤコビアンであり、次のように表すことができる。

Here, Δα is the amount of change in the position and orientation α by the iterative calculation. J is the Jacobian of L _i (α) at α and can be expressed as:

反復計算は、Δαが所定の閾値Ｔｈ未満になるまで繰り返される。閾値Ｔｈは、適宜定めることができる。

The iterative calculation is repeated until Δα becomes less than the predetermined threshold Th. The threshold value Th can be determined as appropriate.

誤差関数ｅ（α）を最小化するため、演算処理部４２は目標計画法を採用することができる。 In the above description, the arithmetic processing unit 42 performs the calculation to minimize the sum of the squares of the error Li (α) as the error function e (α). However, the method for obtaining the optimum value of the position and orientation α is not limited to this. For example, the arithmetic processing unit 42 can derive the attitude position of the camera 2 using the minimum absolute value method. In the minimum absolute value method, an error function e (α) which is an index corresponding to an error is represented by the following equation.

In order to minimize the error function e (α), the arithmetic processing unit 42 can adopt a target programming method.

The control unit 4 calculates the calibration values (X _C , Y _C , Z _C ) of the mounting position and the calibration values (ψ _C , θ _C , φ _C ) of the mounting angle in the arithmetic processing unit 42 and stores them in the memory 3. Can be stored. The memory 3 stores these calibration values in place of the calibration values (X _C , Y _C , Z _C ) of the mounting position and the calibration values (ψ _C , θ _C , φ _C ) of the mounting angle as the respective calibration values. design value of the position _{(X D, Y D, Z} D) and mounting angle of the design value _{(X D, Y D, Z} D) may store only the difference from.

  Next, the image processing unit 41 of the control unit 4 causes the interface unit 5 to display the adjustment confirmation screen 20 as shown in FIG. 6 (Step S05).

On the adjustment confirmation screen 20, translucent shading is superimposed on the image captured by the camera 2. The adjustment confirmation screen 20 includes a hatched area Ah in which translucent hatching is displayed and a non-hatched area Ac without hatching. Network without meshing region Ac is four, which is stored in the memory 3 the three-dimensional position of calibration points _{P 1 ~P 4 (X pi,} Y pi, Z pi) (i = 1~4), and the calibration of the mounting position The area is calculated based on the values (X _C , Y _C , Z _C ) and the calibration values of the mounting angle (θ _C , θ _C , φ _C ). Network without meshing region Ac is based on the calibration point P ₁ to P estimated display position f ₁ ~f ₄ after calibration value adjustment of _4, a region and its neighboring area to the calibration pattern M ₁ is displayed. If the respective calibration values have been accurately calculated, the calibration pattern M ₁ is located in the non-shaded area Ac on the adjustment confirmation screen 20. The width of the unshaded area Ac is determined based on the required calibration accuracy or tolerance. The non-shaded area Ac indicates the range of the allowable error.

The interface unit 5 displays a BACK button 21 and an OK button 22 on the adjustment confirmation screen 20. The BACK button 21 and the OK button 22 can be buttons or icons on the touch panel. The user of the calibration device 1, the adjustment confirmation screen 20, the entire calibration pattern M ₁ is visually confirmed whether or not included in the shaded region without the Ac, either BACK button 21 and OK button 22 Is pressed.

The user, at least a portion of the calibration pattern M ₁ is, if overlaps the shaded region Ah, it can be determined that the accuracy of the calculated calibration value is not good. In this case, the user presses the BACK button 21. When the BACK button 21 is pressed, the control unit 4 receives the signal from the interface unit 5 (Step S06: No), and returns the process to Step S03. That is, the control unit 4 to display the display image 10 to the interface unit 5 receives the position change of the adjustment frame C ₁ -C ₄ again.

The user, if the entire calibration pattern M ₁ is determined to be included in the shaded region without the Ac, and presses an OK button 22. When the OK button 22 is pressed, the interface unit 5 transmits an adjustment end instruction to the control unit 4 (Step S06: Yes). Thus, the calibration value adjustment processing ends.

The calibration values (X _C , Y _C , Z _C ) of the mounting position and the calibration values (搭載_C , θ _C , φ _C ) of the mounting angle stored in the memory 3 are stored in the camera 2 after the calibration value adjustment processing is completed. Is used to calibrate the image captured by the camera 2 during the normal operation of.

  As described above, according to the present embodiment, it is possible to adjust the calibration values for the six axes of the mounting position (X, Y, Z) of the camera 2 and the mounting angles (ψ, θ, φ) of the camera 2. . In the calibration device 1, for example, when only the height (Z) and depression angle (tilt angle: の) of the camera are to be adjusted, for example, displacement in the horizontal direction (X-axis direction) or pan direction (φ ) Or rotation in the roll direction (θ) cannot adjust the parameters in those directions. Further, the height (Z) and the depression angle (ψ) to be adjusted cannot be accurately calibrated. Since the calibration device 1 of the present disclosure can perform six-axis adjustment, accurate adjustment of the calibration value is possible, and the calibration accuracy of the camera 2 can be improved.

  In addition, in a case where a target object such as a pedestrian is detected from a captured image using the camera 2 and a distance to the target object is calculated, if the mounting conditions of the camera 2 in the six-axis direction are not known accurately. In some cases, an error generated in the distance calculation becomes large. The camera 2 whose calibration value has been adjusted using the calibration device 1 of the present disclosure can perform calibration in six axial directions, so that errors can be reduced. Therefore, the calibration device 1 of the present disclosure is particularly effective for the camera 2 used for such an application.

Further, the calibration device 1 of the present disclosure, in the calculation of the calibration value, to use more than four points of the calibration points P ₁ to P ₄ located at least two or more different distances. Since the distances of the calibration points P _{1 to} P ₄ from the camera 2 are different, the accuracy of the calibration can be made higher than when the calibration points P _{1 to} P ₄ are arranged at the same distance. Further, when the number of calibration points is three, there may be a plurality of solutions minimizing the error function e (α), so that an accurate calibration value may not be obtained. By setting the calibration points P _{1 to} P ₄ to four or more, the calibration device 1 of the present disclosure can more accurately and uniquely determine a calibration value.

In addition, in the calibration device 1 of the present disclosure, by using the portable calibration sheet 6, it is relatively easy to use the calibration device 6 in a manufacturing facility such as a factory without requiring a dedicated facility. High-precision adjustment can be performed. Further, while confirming the user in the display image 10, by moving the adjustment frame C ₁ -C _4, since the calculation of the calibration value, the calibration apparatus 1 the actual display position of the calibration point P ₁ to P ₄ The processing of the control unit 4 is simpler than performing the determination of I _{1 to} I ₄ automatically. Further, since the user visually judges the image, it is hardly affected by changes in the working environment such as brightness and direction of light.

Further, an adjustment frame C ₁ -C _4, only match the calibration point P ₁ to P ₄ in a simple GUI while viewing the display image 10 (Graphical User Interface), since the completion of the adjustment of the calibration values, the user The adjustment of the calibration value can be easily performed without any specialized knowledge. Further, the adjustment confirmation screen 20 is provided, and the quality of the calibration value can be determined based on a simple determination criterion of whether or not it is in contact with the shaded area Ah. .

<Variations of calibration patterns>
The calibration apparatus 1 may use a different calibration patterns and calibration pattern M ₁ shown in FIG. Hereinafter, the case of using a different calibration patterns M ₂ ~M _8, illustrating a display example by the interface unit 5. In the following description, since the common embodiment and many parts with calibration pattern M ₁ described above, it is possible to omit the description of the common parts. Further, in the embodiment the same as or similar to components using a calibration pattern M ₁ described above are denoted by the same reference numerals.

(Second example)
Calibration pattern M ₂ shown in Figure 7, the dots indicating the calibration points P ₁ to P ₄ in the four vertices of a quadrilateral are arranged. Dots have a size. The dot is, for example, a small circular area whose inside is filled. During the calibration points P ₁ to P ₄ adjacent are connected by a straight line except for the vicinity of the dot.

When the user adjusts the calibration value, the interface unit 5, as shown in FIG. 8, and displays the display image 10 obtained by superimposing the adjusted frame C ₁ -C ₄ in the image captured by the camera 2. The user, using the position adjustment button 13, when adjusting the position of the adjustment frame C ₁ -C _4, a dot indicating a calibration point P ₁ to P ₄ corresponding, within each adjustment frame C ₁ -C ₄ wherein in the region, and, each adjustment frame C ₁ -C ₄ a straight line portion representing the edges of the adjacent quadrilateral not to be in contact. Using the calibration pattern M _2, by thus adjusting the adjusting frame C ₁ -C _4, clearly space between the dots indicating the calibration points P ₁ to P ₄ and the adjusting frame C ₁ -C ₄ it can. Thus, the user easily visually recognize that the calibration points P ₁ to P ₄ enters the inside of the adjusting frame C ₁ -C _4.

The interface unit 5 can display an adjustment confirmation screen 20 shown in FIG. The user confirms whether the entire dot and solid line corresponds to the calibration points P ₁ to P ₄ of the calibration pattern M ₂ are included in the shaded region without the Ac, termination or redo the adjustment Can be determined.

(Third example)
Calibration pattern M ₃ shown in FIG. 10, dots are arranged for indicating the calibration points P ₁ to P ₄ only four vertices of a quadrilateral. Solid lines indicating each side of the quadrilateral are not displayed.

When the user adjusts the calibration value, the interface unit 5, as shown in FIG. 11 displays the display image 10 obtained by superimposing the adjusted frame C ₁ -C ₄ in the image captured by the camera 2. The user, using the position adjustment button 13, when adjusting the position of the adjustment frame C ₁ -C _4, a dot indicating a calibration point P ₁ to P ₄ corresponding, within each adjustment frame C ₁ -C ₄ Include it in the area.

The interface unit 5 can display an adjustment confirmation screen 20 shown in FIG. The adjustment confirmation screen 20, the four imaginary lines 23a~23d connecting the estimated display position f ₁ ~f ₄ calibration point P ₁ to P ₄ are displayed. The user can have a point of intersection of the four imaginary lines 23 a to 23 d, to check whether the overlaps with dots indicating the calibration points P ₁ to P _4, to determine the end or redo the adjustment . And the straight line that intersects, by combining the dot, the user, the quality of the match between the actual display position I ₁ ~I ₄ calibration point P ₁ to P ₄ and the estimated display position f ₁ ~f ₄ visually It becomes easier to make a decision.

(4th example)
Calibration pattern M ₄ shown in FIG. 13 is displayed as a quadrilateral at least partially internally filled. In the illustrated example, the quadrilateral is a rectangle that is entirely filled. Four vertices of a quadrilateral corresponds to a calibration point P ₁ to P _4.

When the user adjusts the calibration value, the interface unit 5, as shown in FIG. 14 displays the display image 10 obtained by superimposing the adjusted frame C ₁ -C ₄ in the image captured by the camera 2. The user, using the position adjustment button 13, when adjusting the position of the adjustment frame C ₁ -C _4, each vertex of the quadrilateral to be included in the interior region of each adjustment frame C ₁ -C _4. Since the quadrilateral is filled, the difference in contrast between the unfilled region and the unfilled region is clear. Therefore, the user easily visually recognize the position of the calibration point P ₁ to P _4.

The interface unit 5 can display an adjustment confirmation screen 20 shown in FIG. The user can entire calibration pattern M ₄ is to confirm whether or not included in the shaded region without the Ac, determines completion or redo the adjustment.

(Fifth example)
Calibration pattern M ₅ shown in FIG. 16, dots are arranged for indicating the calibration points P ₁ to P ₄ in the four vertices of a quadrilateral. A dot is a small circular area whose interior is filled. Furthermore, the calibration pattern M ₅ are the portions other than the four corners of the quadrilateral are filled. Portions are filled with the calibration pattern M ₅ has a shape obtained by removing a square from each of rectangular corners.

When the user adjusts the calibration value, the interface unit 5, as shown in FIG. 17 displays the display image 10 obtained by superimposing the adjusted frame C ₁ -C ₄ in the image captured by the camera 2. The user, using the position adjustment button 13, when adjusting the position of the adjustment frame C ₁ -C _4, a dot indicating a calibration point P ₁ to P ₄ corresponding, within each adjustment frame C ₁ -C ₄ wherein in the region, and, each adjustment frame C ₁ -C ₄ is adjusted so as not to touch the filled portion of the calibration pattern M _5. Using the calibration pattern M _5, by thus adjusting the adjusting frame C ₁ -C _4, clearly space between the dots indicating the calibration points P ₁ to P ₄ and the adjusting frame C ₁ -C ₄ it can. Thus, it becomes easy to visually recognize that the calibration points P ₁ to P ₄ enters the inside of the adjusting frame C ₁ -C _4. Furthermore, the calibration pattern M _5, because it has a except filled portion in the vicinity of the calibration points P ₁ to P _4, the user further easily recognize the calibration points P ₁ to P _4.

The interface unit 5 can display an adjustment confirmation screen 20 shown in FIG. The user can entire calibration pattern M ₅ is to check whether included in the shaded region without the Ac, determines completion or redo the adjustment.

(Sixth example)
Calibration pattern M ₆ shown in FIG. 19, the dots indicating the calibration points P ₁ to P ₄ located on the four vertices of a quadrilateral, in that a filling portion excluding the vicinity of the four corners of a quadrilateral, in FIG. 16 similar to the calibration pattern M _5. However, the calibration pattern M ₆ is different from the calibration pattern M _5, the four corners of the filled portion is in the quadrant shape to the removed shape rather than a square.

When the user adjusts the calibration value, the interface unit 5, as shown in FIG. 20 displays the display image 10 obtained by superimposing the adjusted frame C ₁ -C ₄ in the image captured by the camera 2. The user, using the position adjustment button 13, when adjusting the position of the adjustment frame C ₁ -C _4, a dot indicating a calibration point P ₁ to P ₄ corresponding, within each adjustment frame C ₁ -C ₄ wherein in the region, and, each adjustment frame C ₁ -C ₄ is adjusted so as not to touch the filled portion of the calibration pattern M _6. Here, in the calibration pattern M ₆ , the shape of the four corners where there is no painting has a shape corresponding to the outer periphery of the adjustment frames C _{1 to} C ₄ . That is, the shape of the filled portion without having an arcuate shape along the outer periphery of the adjusting frame C ₁ -C _4. Therefore, the user positions the calibration points P ₁ to P _4, while viewing the display image 10, can be adjusted so as not to touch the fill portion of the calibration pattern M _6, further facilitated.

(Seventh example)
In the example of the display image 10 described above, the adjustment frame C ₁ -C ₄ was assumed to have a circular or elliptical. However, the shape of the adjusting frame C ₁ -C ₄ is not limited to the circular or elliptical. In calibration sheet 6 of FIG. 21, similar to FIG. 13, the calibration pattern M ₇ rectangles inside filled is placed. In contrast, in the display image 10 shown in FIG. 22, the adjustment frame C ₁ -C ₄ may be a rectangular shape. Even using the adjustment frame C ₁ -C ₄ rectangular shape, it is possible to similarly position change of the adjustment frame C ₁ -C ₄ in the case of using a circular or elliptical adjustment frame.

(Eighth example)
Further, in the calibration pattern M ₁ ~M ₇ described above, the calibration points P ₁ to P ₄ was located at the apex of the periphery of the quadrilateral. However, the calibration points P ₁ to P ₄ may be arranged in other than a vertex of a quadrilateral. In calibration sheet 6 of FIG. 23, the calibration pattern M _8, respectively on the four sides of the quadrilateral has a calibration point P ₁ to P _4. In the display image 10 shown in FIG. 24, it is possible to adjust the position of the adjustment frame C ₁ -C ₄ similarly to the other examples.

<Camera>
The functions of the control unit and the memory of the calibration device according to the present disclosure can be incorporated in a camera mounted on the moving body 100. FIG. 25 illustrates a configuration of a camera 50 according to an embodiment of the present disclosure. The camera 50 includes an imaging optical system 51, an imaging device 52, a memory 53, and a control unit 54 (processor). The imaging optical system 51, the imaging device 52, the memory 53, and the control unit 54 (processor) correspond to the imaging optical system 2a, the imaging device 2b, the memory 3, and the control unit 4 of the calibration device 1 in FIG. 1, respectively. Since the function is the same as that of the component, the description is omitted. The interface unit 55 in FIG. 25 corresponds to the interface unit 5 in FIG. The interface unit 55 is located outside the camera 50. As the interface unit 55, a touch panel or the like of another device can be used. With the configuration as described above, the camera 50 of the present disclosure can have the same operation and effect as the calibration device 1 of FIG.

  Although the embodiments according to the present disclosure have been described based on the drawings and examples, it should be noted that those skilled in the art can easily make various changes or modifications based on the present disclosure. Therefore, it should be noted that these variations or modifications are included in the scope of the present disclosure. For example, functions included in each component or each step can be rearranged so as not to be logically inconsistent, and a plurality of components or steps can be combined into one or divided. It is. Embodiments according to the present disclosure can be realized as a method, a program, or a storage medium on which the program is executed by a processor included in the device. It should be understood that these are also included in the scope of the present disclosure.

  In the present disclosure, the X axis, the Y axis, and the Z axis are provided for convenience of description, and may be interchanged with each other. The configuration according to the present disclosure has been described using an orthogonal coordinate system including the X axis, the Y axis, and the Z axis. The positional relationship between the components according to the present disclosure is not limited to the orthogonal relationship.

In the present disclosure, the calibration pattern does not have to be drawn on the calibration sheet 6. For example, the floor of the place of adjusting the calibration value of the calibration apparatus 1 may be a calibration pattern M ₁ those taped. Moving this case, in order to accurately position the moving body 100 with respect to the calibration pattern M _1, a tire stopper for fixing the position of car tires is provided, the user of the calibration apparatus 1 is in the position of the tire stop The body 100 is accurately fixed.

In the above embodiments, the memory 3 of the calibration apparatus 1 was configured to store three-dimensional position of calibration points P ₁ to P ₄ which is determined in advance. Three-dimensional position of calibration points P ₁ to P ₄ can also be entered when adjusting the calibration value. For example, there calibration sheet 6 of a plurality of use, may be stored three-dimensional position of calibration points P ₁ to P ₄ in the memory 3 of the calibration apparatus 1 according to the calibration sheet 6 to be used.

  The GUI of the interface unit 5 is only an example. Each button displayed on the touch panel of the interface unit 5 can be arranged at an arbitrary position. For adjusting the position of the interface unit 5, other input means may be used instead of the touch panel.

  The calibration marker used in the display image 10 is not limited to a frame shape such as a circle, an ellipse, and a rectangle. Calibration markers of various shapes can be employed. For example, as the calibration marker, for example, a marker in which a straight line crosses in a cross shape can be used. In that case, the position of the calibration marker can be adjusted by moving the point where the cross-shaped straight line intersects so as to coincide with the calibration point.

DESCRIPTION OF SYMBOLS 1 Calibration apparatus 2 Camera 3 Memory 4 Control part (processor)
5 Interface section 6 Calibration sheet 10 Display image 12 Selection button 13 Position adjustment button 14 OK button 20 Adjustment confirmation screen 21 BACK button 22 OK button 23a, 23b, 23c, 23d Virtual line 41 Image processing section 42 Arithmetic processing section 50 Camera 51 Lens 52 Image sensor 53 Memory 54 Control unit (processor)
100 vehicle Ah Shaded areas Ac Shaded region without C ₁ -C ₄ adjusting frame (calibration marker)
M ₁ ~M ₈ calibration pattern O optical axis P ₁ to P ₄ calibration point f ₁ ~f ₄ Estimation display position I ₁ I4 display position

Claims (16)

A camera mounted on a moving object,
A memory for storing three-dimensional positions of four or more calibration points located at least two or more different distances from the camera, and a mounting position and a mounting angle of the camera;
A processor that generates a display image, in which four or more calibration markers respectively corresponding to the four or more calibration points are superimposed and displayed on the captured image of the camera,
An interface unit that displays the display image, and externally issues a change instruction of a superimposed display position of each of the calibration markers to the processor,
The processor further includes: an estimated display position on the captured image of the four or more calibration points estimated from the three-dimensional position of the calibration point, the mounting position, and the mounting angle; and the calibration on the display image. A calibration device for calculating a calibration value for adjusting the mounting position and the mounting angle of the camera by six axes so as to reduce an index indicating a difference between the display marker and the superimposed display position.   The calibration device according to claim 1, wherein the processor repeatedly executes a calculation for reducing the index, and ends the repetition of the calculation by determining convergence of the index. A camera mounted on a moving object,
A memory for storing three-dimensional positions of four or more calibration points located at least two or more different distances from the camera, and a mounting position and a mounting angle of the camera;
A processor that generates a display image, in which four or more calibration markers respectively corresponding to the four or more calibration points are superimposed on the captured image of the camera, and
An interface unit that displays the display image, and that externally issues a change instruction of a superimposed display position of each of the calibration markers to the processor,
The processor further includes: an estimated display position on the captured image of the four or more calibration points estimated from the three-dimensional position of the calibration point, the mounting position, and the mounting angle; and the calibration on the display image. A calibration device that calculates a calibration value for adjusting the mounting position and the mounting angle of the camera by six axes, so as to reduce an index indicating a difference between the camera marker and the superimposed display position,
Four or more real space calibration points corresponding to the four or more calibration points stored in the memory;
A calibration system comprising:   The calibration system according to claim 3, wherein the calibration point is represented by a dot having a size, and the calibration marker is a frame having an inner area larger than the dot in the display image.   The calibration system according to claim 3, wherein the calibration point is located on an outer periphery of a quadrilateral.   The calibration system according to claim 5, wherein the quadrilateral is represented by a solid line representing an outer periphery, and a vertex of the quadrilateral represents the calibration point.   The calibration system according to claim 5, wherein the quadrilateral is configured by the calibration points corresponding to the vertices and a straight line connecting the calibration points except for the vicinity of the calibration points.   The calibration system according to claim 5, wherein the calibration points are indicated by only four points located at the vertices of the quadrilateral.   The calibration system of claim 5, wherein the quadrilateral is at least partially filled.   The calibration system according to claim 9, wherein the quadrilateral has an unfilled area at four corners, and the calibration point is located at a vertex in the area.   The calibration system according to claim 10, wherein the shape of the unfilled area at the four corners of the quadrilateral corresponds to the shape of the calibration marker.   The calibration system according to any one of claims 3 to 11, wherein the calibration points are arranged on a movable sheet.   The interface unit displays an adjustment confirmation screen indicating an allowable error range of the calibration point together with the calibration point based on the three-dimensional position of the calibration point and the calibration value, and the calibration value is set in the allowable error range. 13. The calibration system according to any one of claims 3 to 12, wherein the determination whether the inside is inside or outside is received. A camera mounted on a moving object,
An image sensor;
A memory for storing three-dimensional positions of four or more calibration points located at least two or more different distances from the camera, and a mounting position and a mounting angle of the camera;
A processor that generates a display image, in which four or more calibration markers respectively corresponding to the four or more calibration points are superimposed and displayed on a captured image captured by the image sensor,
The processor causes the display image to be displayed on an external interface unit, and receives from the interface unit an instruction to change the superimposed display position of each of the calibration markers,
The processor further includes: an estimated display position on the captured image of the four or more calibration points estimated from the three-dimensional position of the calibration point, the mounting position, and the mounting angle; and the calibration on the display image. A camera for calculating a calibration value for adjusting the mounting position and the mounting angle of the camera by six axes so as to reduce an index indicating a difference between the mounting marker and the superimposed display position.   A camera, a three-dimensional position of four or more calibration points located at least two or more different distances from the camera, and a memory for storing a mounting position and a mounting angle of the camera; A processor for generating a display image, in which four or more calibration markers respectively corresponding to the calibration points or more are displayed in a superimposed manner, and displaying the display image, and providing the processor with each of the calibration markers An interface unit for externally giving an instruction to change the superimposed display position of the three-dimensional position of the calibration point, the mounting position and the mounting position and the mounting points and the four or more of the calibration points estimated from the mounting angle. To reduce an index indicating a difference between an estimated display position on a captured image and the superimposed display position of the calibration marker on the display image, Moving body provided with a calibration device configured to calculate the mounting position and the calibration value, wherein the mounting angle adjusting six axes of the camera. Acquiring a captured image including four or more calibration points located at least two or more different distances with a camera mounted on a moving body;
Generating a display image in which four or more calibration markers respectively corresponding to the four or more calibration points are superimposed on the captured image of the camera,
A step of externally receiving a change instruction to change a superimposed display position of the calibration marker at a position corresponding to the calibration point on the display image;
Obtaining the coordinates of the calibration marker in the display image,
Estimating an estimated display position of the calibration point on the captured image from a pre-stored mounting position and mounting angle of the camera, and a three-dimensional position of the calibration point;
Calculating a calibration value for adjusting the mounting position and mounting angle of the camera by six axes so as to reduce the difference between the superimposed display position of the calibration marker acquired from the display image and the estimated display position; ,
And a calibration value adjustment method.

Images