CN114833821A

CN114833821A - Camera parameter calibration method, system and computer program product

Info

Publication number: CN114833821A
Application number: CN202210341774.5A
Authority: CN
Inventors: 安书鹏
Original assignee: Autonavi Software Co Ltd
Current assignee: Autonavi Software Co Ltd
Priority date: 2022-03-29
Filing date: 2022-03-29
Publication date: 2022-08-02

Abstract

The embodiment of the disclosure discloses a camera parameter calibration method, a system and a computer program product, wherein the method comprises the following steps: sending a mechanical arm control signal to a mechanical arm for fixing a camera to be calibrated so as to control the mechanical arm to drive the camera to be fixed to move along a preset motion track; sending a camera control signal to the camera to be calibrated so as to control the camera to be calibrated to acquire camera calibration data in the process of moving along with the mechanical arm; acquiring the camera calibration data; the camera calibration data comprises calibration plate images collected by the camera to be fixed; and determining the camera parameters of the camera to be calibrated based on the camera calibration data. The technical scheme can be suitable for parameter calibration of batch cameras in a mass production scene, and the parameter calibration efficiency can be improved.

Description

Camera parameter calibration method, system and computer program product

Technical Field

The present disclosure relates to the field of vision technologies, and in particular, to a method, a system, and a computer program product for calibrating camera parameters.

Background

The camera calibration is a basic link of machine vision application such as vision measurement, three-dimensional reconstruction and the like, and the accuracy and precision of the calibration result directly determine whether the vision system can work normally. The camera calibration refers to a process of shooting an image of a calibration plate by using a camera, and calculating camera parameters by using three-dimensional coordinates of known characteristic points in the calibration plate and corresponding image coordinates on the image. The camera calibration comprises the acquisition of camera calibration data and camera parameter calculation.

In the prior art, in the process of acquiring calibration data of a camera, the camera is moved manually, images of a calibration plate are acquired from different angles at different positions, and then internal parameters and external parameters of the camera are calculated by using a parameter calibration algorithm. However, the calibration method takes a long time, and the trajectory of the manual movement may be inconsistent when each camera device is calibrated, which may not be applied to a large-scale mass production scenario. Therefore, how to efficiently calibrate the camera parameters in a mass production scenario is one of the main technical problems that needs to be solved currently.

Disclosure of Invention

The embodiment of the disclosure provides a camera parameter calibration method, a system and a computer program product.

In a first aspect, an embodiment of the present disclosure provides a camera parameter calibration method, where the method includes:

sending a mechanical arm control signal to a mechanical arm for fixing a camera to be calibrated so as to control the mechanical arm to drive the camera to be fixed to move along a preset motion track;

sending a camera control signal to the camera to be calibrated so as to control the camera to be calibrated to acquire camera calibration data in the process of moving along with the mechanical arm;

Acquiring the camera calibration data; the camera calibration data comprises calibration plate images collected by the camera to be fixed;

and determining the camera parameters of the camera to be calibrated based on the camera calibration data.

Further, the camera parameters include internal parameters of the camera to be calibrated, and the sending of a mechanical arm control signal to a mechanical arm for fixing the camera to be calibrated so as to control the mechanical arm to drive the camera to be fixed to move along a preset motion trajectory includes:

and sending a first mechanical arm control signal to the mechanical arm so as to control the mechanical arm to move from one of the plurality of track points to the next and stay for a first preset time after each track point is reached.

Further, sending a camera control signal to the camera to be calibrated to control the camera to be calibrated to acquire camera calibration data in the process of moving along with the mechanical arm, including:

and when the mechanical arm reaches the track point and stays, sending a first camera control signal to the camera to be calibrated so as to control the camera to be calibrated to acquire an image of the calibration plate.

Further, after the mechanical arm reaches one of the track points and stays at a first preset time, the method further comprises the following steps:

Sending a second mechanical arm control signal to the mechanical arm to control the mechanical arm to drive the camera to be calibrated to stay for a second preset time after the camera to be calibrated rotates for a preset angle at the track point;

and sending a second camera control signal to the camera to be calibrated during the period that the mechanical arm drives the camera to be calibrated to rotate by a preset angle and stay, so as to control the camera to be calibrated to acquire an image of the calibration plate.

Further, the camera parameters include internal parameters of the camera to be calibrated, and a mechanical arm control signal is sent to a mechanical arm for fixing the camera to be calibrated, so as to control the mechanical arm to drive the camera to be fixed to move along a preset motion track, including:

sending a first mechanical arm control signal to the mechanical arm to control the mechanical arm to move from one of the plurality of track points to the next and stay for a first preset time after each track point is reached;

and after the mechanical arm reaches one track point and stays for a first preset time, sending a second mechanical arm control signal to the mechanical arm so as to control the mechanical arm to drive the camera to be calibrated to stay for a second preset time after rotating for a first preset angle.

and sending a camera control signal to the camera to be calibrated so as to control the camera to be calibrated to acquire a calibration plate image as camera calibration data in the process that the mechanical arm moves according to the first mechanical arm control signal and/or the second mechanical arm control signal at a preset frequency.

Further, determining the camera parameters of the camera to be calibrated based on the camera calibration data comprises:

and processing the camera calibration data by using a preset internal reference calibration algorithm to obtain the internal reference of the camera to be calibrated.

Further, the camera parameters comprise external parameters of the camera to be calibrated, inertial navigation equipment is further arranged on the camera to be calibrated, and the camera calibration data further comprise inertial navigation data and data acquisition time; determining camera parameters of the camera to be calibrated based on the camera calibration data, including:

and processing the camera calibration data by using a preset external reference calibration algorithm to obtain the external reference of the camera to be calibrated.

Further, a mechanical arm control signal is sent to a mechanical arm used for fixing the camera to be calibrated so as to control the mechanical arm to drive the camera to be fixed to move along a preset motion track, and the method comprises the following steps:

sending a third mechanical arm control signal to the mechanical arm to control the mechanical arm to move to a preset preparation point, and then sequentially performing the following rotation actions around an X axis, a Y axis and a Z axis: the positive direction is rotated by a second preset angle, and the negative direction is rotated by a second preset angle which is 2 times, and then the rotation returns to the positive direction;

sending a fourth mechanical arm control signal to the mechanical arm to control the mechanical arm to perform the following translation actions at the preset preparation point along the X-axis direction, the Y-axis direction and the Z-axis direction in sequence: the positive direction is translated for a preset distance, and the negative direction is translated for a preset distance which is 2 times and then returns to the positive direction;

the sending a camera control signal to the camera to be calibrated to control the camera to be calibrated to acquire camera calibration data in the process of moving along with the mechanical arm includes:

and sending a third camera control signal to the camera to be calibrated so as to control the camera to be calibrated to acquire calibration plate images and acquire inertial navigation data in the process that the mechanical arm moves according to the third mechanical arm control signal and/or the fourth mechanical arm control signal at a preset frequency.

Furthermore, a multi-split clamp is arranged on the mechanical arm and used for fixing a plurality of cameras to be calibrated.

In a second aspect, an embodiment of the present disclosure provides a camera parameter calibration apparatus, including:

the camera calibration device comprises a first sending module, a second sending module and a control module, wherein the first sending module is configured to send a mechanical arm control signal to a mechanical arm used for fixing a camera to be calibrated so as to control the mechanical arm to drive the camera to be fixed to move along a preset motion track;

the second sending module is configured to send a camera control signal to the camera to be calibrated so as to control the camera to be calibrated to acquire camera calibration data in the process of moving along with the mechanical arm;

an acquisition module configured to acquire the camera calibration data; the camera calibration data comprises calibration plate images collected by the camera to be fixed;

a determination module configured to determine camera parameters of the camera to be calibrated based on the camera calibration data.

The functions can be realized by hardware, and the functions can also be realized by executing corresponding software by hardware. The hardware or software includes one or more modules corresponding to the above-described functions.

In one possible design, the apparatus includes a structure including a memory for storing one or more computer instructions that enable the apparatus to perform the corresponding method described above, and a processor configured to execute the computer instructions stored in the memory. The apparatus may also include a communication interface for the apparatus to communicate with other devices or a communication network.

In a third aspect, an embodiment of the present disclosure provides a camera calibration system, including: a camera to be calibrated, a mechanical arm and control equipment; wherein the content of the first and second substances,

the camera to be calibrated is fixed at the tail end of the mechanical arm;

the mechanical arm drives the camera to be calibrated to move along a preset motion track under the control of the control equipment;

the control equipment sends a mechanical arm control signal to the mechanical arm and sends a camera control signal to the camera to be calibrated so as to control the mechanical arm to drive the camera to be fixed to move along a preset motion track and control the camera to be calibrated to acquire camera calibration data in the process of moving along with the mechanical arm;

the control equipment also acquires the camera calibration data and determines the camera parameters of the camera to be calibrated based on the camera calibration data; the camera calibration data comprises calibration plate images collected by the camera to be fixed.

Further, the camera parameters comprise external parameters of the camera to be calibrated;

in the external reference calibration process of the camera to be calibrated, the control equipment sends a third camera control signal to the camera to be calibrated so as to control the camera to be calibrated to acquire calibration plate images and inertial navigation data at a preset frequency;

The control equipment also sends a third mechanical arm control signal to the mechanical arm so as to control the mechanical arm to move to a preset preparation point and then sequentially rotate around an X axis, a Y axis and a Z axis as follows: the positive direction is rotated by a second preset angle, and the negative direction is rotated by a second preset angle which is 2 times, and then the rotation returns to the positive direction;

the control equipment sends a fourth mechanical arm control signal to the mechanical arm so as to control the mechanical arm to perform the following translation actions at the preset preparation point in the X-axis direction, the Y-axis direction and the Z-axis direction in sequence: the positive direction translates for a preset distance, and the negative direction translates for 2 times of the preset distance and then returns to the positive direction.

In a fourth aspect, the disclosed embodiments provide an electronic device comprising a memory, a processor, and a computer program stored on the memory, wherein the processor executes the computer program to implement the method of any of the above aspects.

In a fifth aspect, the disclosed embodiments provide a computer-readable storage medium for storing computer instructions for any one of the above apparatuses, which when executed by a processor, are configured to implement the method of any one of the above aspects.

In a sixth aspect, the disclosed embodiments provide a computer program product comprising computer instructions for implementing the method of any one of the above aspects when executed by a processor.

The technical scheme provided by the embodiment of the disclosure can have the following beneficial effects:

in the embodiment of the disclosure, in the process of calibrating the camera parameters, the camera to be calibrated is fixed on the mechanical arm, the mechanical arm drives the camera to be calibrated to move along the preset motion track under the control of the mechanical arm control signal, in the process of moving along with the mechanical arm, the camera to be calibrated acquires camera calibration data under the control of the camera control signal, and then the control device calculates the camera parameters of the camera to be calibrated based on the camera calibration data. In this way, the mechanical arm drives the camera to be calibrated to move along the preset motion track, and the camera to be calibrated can acquire camera calibration data from different angles at a fixed position under the control of the camera control signal, so that the motion track of each camera to be calibrated and the position of the acquired camera calibration data have consistency, and therefore, the method can be suitable for parameter calibration of batch cameras in a mass production scene, and can also improve the efficiency of parameter calibration.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

Drawings

Other features, objects, and advantages of the present disclosure will become more apparent from the following detailed description of non-limiting embodiments when taken in conjunction with the accompanying drawings. In the drawings:

FIG. 1 shows a flow diagram of a camera parameter calibration method according to an embodiment of the present disclosure;

FIG. 2 shows a block diagram of a camera parameter calibration system according to an embodiment of the present disclosure;

FIGS. 3(a) and 3(b) are schematic diagrams illustrating application scenarios of camera internal reference calibration and camera external reference calibration according to an embodiment of the present disclosure;

fig. 4 shows a block diagram of a camera parameter calibration apparatus according to an embodiment of the present disclosure;

fig. 5 is a schematic structural diagram of an electronic device suitable for implementing a camera parameter calibration method according to an embodiment of the present disclosure.

Detailed Description

Hereinafter, exemplary embodiments of the present disclosure will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily implement them. Also, for the sake of clarity, parts not relevant to the description of the exemplary embodiments are omitted in the drawings.

In the present disclosure, it is to be understood that terms such as "including" or "having," etc., are intended to indicate the presence of the disclosed features, numbers, steps, actions, components, parts, or combinations thereof, and do not preclude the possibility that one or more other features, numbers, steps, actions, components, parts, or combinations thereof are present or added.

It should be further noted that the embodiments and features of the embodiments in the present disclosure may be combined with each other without conflict. The present disclosure will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.

The details of the embodiments of the present disclosure are described in detail below with reference to specific embodiments.

Fig. 1 shows a flowchart of a camera parameter calibration method according to an embodiment of the present disclosure. As shown in fig. 1, the camera parameter calibration method includes the following steps:

in step S101, sending a mechanical arm control signal to a mechanical arm for fixing a camera to be calibrated, so as to control the mechanical arm to drive the camera to be fixed to move along a preset motion track;

in step S102, sending a camera control signal to the camera to be calibrated to control the camera to be calibrated to acquire camera calibration data in a process of moving along with the mechanical arm;

in step S103, acquiring the camera calibration data; the camera calibration data comprises calibration plate images collected by the camera to be fixed;

in step S104, camera parameters of the camera to be calibrated are determined based on the camera calibration data.

In this embodiment, the mechanical arm may be a mechanical arm with 6 degrees of freedom, and a clamp for fixing the camera to be calibrated may be provided at a distal end of the mechanical arm. The cameras to be calibrated may be any cameras, and may be one or more cameras requiring parameter calibration in batches, for example. The embodiment of the disclosure can be executed on a control device, the control device can be located in the same physical space with the mechanical arm, and also can be located in different physical spaces, that is, the control device can control the mechanical arm in a short distance, and also can control the mechanical arm in a remote way.

In some embodiments, the control device may control the robotic arm through a network. After receiving the mechanical arm control signal of the control equipment, the mechanical arm executes corresponding action based on the control.

In some embodiments, the control device may further perform network communication or wired connection with the camera to be calibrated, the control device may control the camera to be calibrated to perform image acquisition, and the mechanical arm may drive the camera to be calibrated to move or rotate. In other embodiments, the method of the present disclosure may be executed on other devices, such as a cloud or a remote server, and the control device may execute a control program received from the cloud or the remote server to control the camera and the robot arm to be calibrated, and transmit camera calibration data acquired by the camera to be calibrated in the process to the cloud or the remote server to calibrate the camera parameters.

In some embodiments, the robot arm control signal is used to control the robot arm to perform corresponding actions, for example, the robot arm may move along a preset motion track based on the robot arm control signal, and the actions of the robot arm on the preset motion track may include, but are not limited to, translational actions in various directions and rotational actions in various different angles in various directions. In some embodiments, the robot arm control signal may be a set of continuous control signals, each of which may include, but is not limited to, a translation direction, a translation distance, a rotation direction, a rotation angle, and a motion time. In the embodiment of the present disclosure, sending the robot arm control signal to the robot arm may be understood as sending the set of control signals to the robot arm continuously from the first of the set of control signals, where the set of control signals may be sent only once or may be sent repeatedly for multiple times, which may be specifically determined according to a requirement for actually performing parameter calibration, and is not limited herein.

In some embodiments, the camera control signals are used to control the camera to be calibrated to take successive shots or to take shots at appropriate times. In the calibration process of the camera parameters, a camera is used for collecting images of the calibration plate, and internal parameters and/or external parameters of the camera are calibrated based on the images. In the embodiment of the present disclosure, the camera may be fixed at the end of the robot arm, and the robot arm is controlled by the control device to transport the camera to the image capturing area, in which the calibration plate may be disposed, and the camera may be aligned with the calibration plate when capturing an image. The mechanical arm drives the camera to be calibrated to capture images of the calibration plate from different angles at different positions under the control of the control equipment, dozens of images of the calibration plate at different positions and/or different angles can be collected under normal conditions, and then calibration of camera parameters is completed by executing a corresponding parameter calibration algorithm.

In some embodiments, camera calibration data collected by the camera to be calibrated, which may include, but is not limited to, calibration plate images, may be used as input to the parametric calibration algorithm. In other embodiments, the camera calibration data may further include inertial navigation data and a data acquisition time, which may be an acquisition time of the calibration plate image and the inertial navigation data. The different parameter calibration algorithms may use different camera calibration data, and the camera calibration data may be acquired according to the requirements of the parameter calibration algorithm to be used currently. The camera to be calibrated can transmit the acquired camera calibration data to the control equipment in a USB or network mode, and the control equipment can directly execute a parameter calibration algorithm to calculate the parameters of the camera to be calibrated; the control device may also upload camera calibration data received from the camera to the cloud or the remote server, and the cloud or the remote server calculates parameters of the camera to be calibrated by executing a parameter calibration algorithm.

In an optional implementation manner of this embodiment, a multi-split fixture is disposed on the mechanical arm, and is used to fix a plurality of cameras to be calibrated.

In the optional implementation mode, a plurality of clamps can be arranged at the tail end of the mechanical arm, and each clamp can clamp one camera to be calibrated, so that parameter calibration of the plurality of cameras can be realized in one control process, and the parameter calibration efficiency is further improved.

In an optional implementation manner of this embodiment, in step S104, that is, the step of determining the camera parameters of the camera to be calibrated based on the camera calibration data further includes the following steps:

In this optional implementation manner, calibration of the camera internal parameters may be implemented by using the method provided in the embodiment of the present disclosure. Camera intrinsic parameters are parameters related to the camera's own characteristics, such as the focal length of the camera, pixel size, etc. In the calibration process of the camera internal reference, calibration plate images can be acquired from different positions and different angles of the same position, and based on the acquired calibration plate images, the calibration plate images are processed by using an internal reference calibration algorithm to obtain the internal reference of the camera to be calibrated.

In an optional implementation manner of this embodiment, the camera parameters include internal parameters of the camera to be calibrated, and step S101 is to send a mechanical arm control signal to a mechanical arm for fixing the camera to be calibrated so as to control the mechanical arm to drive the camera to be fixed to move along a preset motion trajectory, and further includes the following steps:

In an optional implementation manner of this embodiment, step S102, that is, the step of sending a camera control signal to the camera to be calibrated to control the camera to be calibrated to acquire camera calibration data in a process of moving along with the mechanical arm, further includes the following steps:

In the optional implementation manner, in the process of calibrating the internal reference of the camera by using the method provided by the embodiment of the present disclosure, a motion trajectory may be preset, and a plurality of track points may be set on the motion trajectory, in the process of calibrating the internal reference, a control signal of the first mechanical arm may be sent to the mechanical arm to control the mechanical arm to start from a first track point of the plurality of track points and move from one track point to another track point, and the mechanical arm may stay for a certain time after moving to each track point, for example, stay for 200 milliseconds. It should be noted that, the fact that the mechanical arm moves to a certain track point means that the tail end of the mechanical arm moves to the track point, and the camera to be calibrated is fixed at the tail end of the mechanical arm, which is equivalent to that the mechanical arm conveys the camera to be calibrated to each track point, and during the stay period of each track point, the camera to be calibrated can acquire an image of the calibration plate under the control of the control signal of the first camera. The camera to be fixed may capture one or more calibration plate images at the same location.

It should be noted that, a plurality of track points in the preset motion trajectory are all at different positions in front of the calibration board, and the vertical distance between each track point and the calibration board and/or the linear distance between each track point and the calibration board may be different or partially the same.

Because the mechanical arm stays for a certain time at each track point, and the camera to be calibrated acquires the calibration plate image during the stay period of the mechanical arm, the acquired calibration plate image is clearer and is suitable for calibrating the internal reference of the camera.

In an optional implementation manner of this embodiment, after the mechanical arm reaches one of the track points and stops for a first predetermined time, the method further includes the following steps:

sending a second mechanical arm control signal to the mechanical arm to control the mechanical arm to drive the camera to be calibrated to stay for a second preset time after the camera to be calibrated rotates for a preset angle on the track point;

In this optional implementation manner, in order to meet the requirement of the internal reference calibration algorithm on the calibration data of the camera and to enable the internal reference calibration algorithm to calculate the internal reference of the camera more accurately, a second mechanical arm control signal may be sent to the mechanical arm again after the mechanical arm stays at a track point for a predetermined time, so as to control the mechanical arm to rotate around the X axis, the Y axis and/or the Z axis at the track point for a certain angle respectively, the mechanical arm may stay for a certain time again once rotating, and during the stay period, the camera to be calibrated may be controlled to acquire one or more calibration board images by sending a second camera control signal. By the method, the camera to be calibrated can acquire calibration plate images from different positions and different angles of the same position in the internal reference calibration process, so that the internal reference obtained by calibration is more accurate.

In some embodiments, the first predetermined angle may be less than 20 degrees. The first predetermined time and the second predetermined time may be the same or different.

and after the mechanical arm reaches one track point and stays for a first preset time, a second mechanical arm control signal is sent to the mechanical arm so as to control the mechanical arm to drive the camera to be calibrated to stay for a second preset time after rotating for a first preset angle.

This alternative implementation differs from the internal reference calibration process mentioned in the previous embodiment. In the embodiment, a motion track can be preset, a plurality of track points can be set on the motion track, in the internal reference calibration process, a first mechanical arm control signal can be sent to the mechanical arm, the mechanical arm can move from one track point to another track point from the first track point of the plurality of track points, and the mechanical arm can stay for a certain time after moving to each track point, for example, the mechanical arm can stay for 100 milliseconds. It should be noted that, the fact that the mechanical arm moves to a certain track point means that the tail end of the mechanical arm moves to the track point, and the camera to be calibrated is fixed at the tail end of the mechanical arm, which is equivalent to that the mechanical arm stops for a period of time after the camera to be calibrated is conveyed to each track point.

In this alternative implementation, different from the previous implementation, in this implementation, the control device sends a camera control signal to the camera to be calibrated after the start of the internal reference calibration, so as to control the camera to be calibrated to acquire the calibration plate image at a predetermined frequency. That is, in this embodiment, regardless of whether the camera to be calibrated is carried to the locus point, data acquisition is started as soon as the camera control signal is received, and for example, calibration plate images may be acquired at a frequency of 20 frames per second.

It should be noted that the time for sending the camera control signal to the camera to be calibrated and the time for sending the first mechanical arm control signal to the mechanical arm may be the same or different, and the sequence may be adjustable.

Because the mechanical arm stays for a certain time at each track point, and the camera to be calibrated acquires the image of the calibration plate at a preset frequency, the camera to be calibrated can acquire the image of the calibration plate in a static state during the stay of the mechanical arm, and the image of the calibration plate acquired in the static state is clear and is suitable for calibrating the internal reference of the camera.

The control equipment can acquire all calibration plate images acquired by the camera to be calibrated, and can screen out the calibration plate images acquired in the static state by comparing pixel point difference values between a plurality of continuous calibration plate images in front and at back, further select one of the calibration plate images acquired in each static state, input the calibration plate images into an internal reference calibration algorithm, and calculate to obtain the internal reference of the camera to be calibrated.

Similarly, in order to meet the requirement of the internal reference calibration algorithm on the calibration data of the camera and enable the internal reference calibration algorithm to calculate the internal reference of the camera more accurately, after the mechanical arm stays at a track point for a predetermined time, a second mechanical arm control signal may be sent to the mechanical arm again to control the mechanical arm to rotate around the X axis, the Y axis and/or the Z axis at the track point for a first predetermined angle respectively, and the mechanical arm may stay for a certain time each time. In this embodiment, since the camera to be calibrated continuously acquires images at a predetermined frequency based on the camera control signal, the camera to be calibrated continuously acquires data while the robot arm moves according to the second robot arm control signal, and the camera to be calibrated can acquire one or more images of the calibration plate in a stationary state while the robot arm is stopped. By the method, the camera to be calibrated can acquire calibration plate images from different positions and different angles of the same position in the internal reference calibration process, so that the internal reference obtained by calibration is more accurate.

In an optional implementation manner of this embodiment, the camera parameters include external parameters of the camera to be calibrated, the camera to be calibrated is further provided with an inertial navigation device, and the camera calibration data further includes inertial navigation data and data acquisition time; step S104, namely, the step of determining the camera parameters of the camera to be calibrated based on the camera calibration data, further includes the following steps:

In this optional implementation, the embodiment of the present disclosure may also be used to calibrate external parameters of the camera. The camera external parameters may be parameters in a world coordinate system, such as the position, rotational direction, etc. of the camera, which may be divided into a rotation matrix and a translation matrix, which together describe how to convert points from the world coordinate system to the camera coordinate system.

In the external reference calibration process of the camera, the external reference calibration algorithm can use inertial navigation data and data acquisition time acquired by inertial navigation equipment arranged on the camera to be fixed besides the calibration plate image acquired by the camera to be fixed. The data acquisition time may include, but is not limited to, calibration plate image acquisition time and inertial navigation data acquisition time. In some embodiments, inertial navigation data may be acquired at the same time the calibration plate image is acquired.

For the same camera or the same group of cameras, after the calibration of the internal parameters of the camera is completed by executing the method provided by the embodiment of the disclosure, the calibration of the external parameters of the camera is completed by executing the method again. It is understood that the calibration of the external reference of the camera may be completed first, and then the calibration of the internal reference may be completed, which may be specifically set according to an actual situation, and is not limited herein.

In an optional implementation manner of this embodiment, step S101, that is, a step of sending a robot arm control signal to a robot arm for fixing a camera to be calibrated to control the robot arm to drive the camera to be fixed to move along a preset movement track, further includes the following steps:

sending a third mechanical arm control signal to the mechanical arm to control the mechanical arm to move to a preset preparation point, and then sequentially performing the following rotation actions around an X axis, a Y axis and a Z axis: the positive direction rotates by a second preset angle, and the negative direction rotates by a second preset angle which is 2 times, and then the rotation returns to the positive direction;

Step S102, namely, the step of sending a camera control signal to the camera to be calibrated to control the camera to be calibrated to acquire camera calibration data in the process of moving along with the mechanical arm, further includes the following steps:

In this optional implementation manner, this embodiment is applicable to the calibration process of the external reference of the camera, and in this embodiment, a preset preparation point may be preset, and the preset preparation point may be, for example, right opposite to the central point of the calibration board and be about 2 meters away from the calibration board. In the external parameter calibration process, a third mechanical arm control signal is sent to the mechanical arm, the mechanical arm is firstly moved to a preset preparation point, and then the mechanical arm is controlled to rotate around an X axis, a Y axis and a Z axis in sequence at the preset preparation point. For example, the X-axis is rotated forward by a second predetermined angle, then rotated backward by a second predetermined angle that is 2 times larger than the X-axis (corresponding to a reverse rotation of the X-axis by the second predetermined angle from the original angle), and then rotated forward by the second predetermined angle back to the original angle. Similarly, the above-described rotation motions are respectively performed around the Y-axis and the Z-axis. To obtain more data, after one pass is completed, the above actions may be repeated a number of times, e.g., 2-5 times.

In some embodiments, the second predetermined angle may be greater than the first predetermined angle, for example the first predetermined angle may be in the range of 1-20 degrees, and the second predetermined angle may be greater than or equal to 30 degrees.

It should be noted that, the moving of the mechanical arm to the preset preparation point means that the tail end of the mechanical arm moves to the preset preparation point, and the tail end of the mechanical arm drives the camera to be calibrated to rotate by a certain angle.

After the rotation action is completed, a fourth mechanical arm control signal can be sent to the mechanical arm, so that the mechanical arm is controlled to sequentially perform translation actions from the preset preparation point to the X-axis direction, the Y-axis direction and the Z-axis direction. In some embodiments, the robot arm may be controlled to move forward along the X axis for a predetermined distance, for example, 1 meter, then move backward for a predetermined distance 2 times (corresponding to moving backward for a predetermined distance), and then return to positive. Similarly, the mechanical arm can be controlled to respectively perform positive and negative translation along the Y axis and the Z axis and then return to the positive direction.

In this alternative embodiment, the control device may send a third camera control signal to the camera to be calibrated after the external reference calibration is started or after a third robot arm control signal is sent to the robot arm, so as to control the camera to be calibrated to acquire the calibration plate image and the inertial navigation data at a predetermined frequency. In this embodiment, the camera to be calibrated may start to acquire the camera calibration data after receiving the third camera control signal before or after the movement of the robot arm, for example, the calibration board image may be acquired at a frequency of 20 frames per second, and the inertial navigation data may be acquired while acquiring the calibration board image. The camera to be calibrated can transmit the acquired calibration plate image, inertial navigation data and data acquisition time to the control equipment together.

It should be noted that the time for sending the third camera control signal to the camera to be calibrated and the time for sending the third robot arm control signal to the robot arm may be the same or different, and the sequence may be adjustable.

Because the camera to be calibrated collects the calibration plate image and the inertial navigation data at a preset frequency in the motion process of the mechanical arm, although in the external reference calibration process, the external reference calibration algorithm does not need to process the collected image content, namely, does not need a clear calibration plate image, the mechanical arm can not stay at any position in the process, and only the camera to be calibrated collects the calibration plate image and the inertial navigation data at the position of the mechanical arm at a certain collection frequency.

The control equipment can determine the position of the camera to be calibrated when the calibration plate image and the inertial navigation data are acquired based on the acquisition time of the calibration plate image and the inertial navigation data and the motion track of the mechanical arm, and the control equipment inputs the calibration plate image, the inertial navigation data, the data acquisition time, the acquisition position and the like into an external parameter calibration algorithm, so that the external parameter of the camera to be calibrated can be obtained through calculation.

Fig. 2 shows a block diagram of a camera parameter calibration system according to an embodiment of the present disclosure. As shown in fig. 2, the camera parameter calibration system includes: a camera 201 to be calibrated, a mechanical arm 202 and a control device 203; wherein the content of the first and second substances,

The camera 201 to be calibrated is fixed at the tail end of the mechanical arm 202;

the mechanical arm 202 drives the camera 201 to be calibrated to move along a preset motion track under the control of the control device 203;

the control device 203 sends a mechanical arm control signal to the mechanical arm 202 and a camera control signal to the camera 201 to be calibrated so as to control the mechanical arm 202 to drive the camera to be fixed to move along a preset motion track and control the camera 201 to be calibrated to acquire camera calibration data in the process of moving along with the mechanical arm 202;

the control device 203 further obtains the camera calibration data, and determines the camera parameters of the camera 201 to be calibrated based on the camera calibration data; the camera calibration data comprises calibration plate images acquired by the camera to be fixed.

In this embodiment, the robot arm 202 may be a robot arm 202 having 6 degrees of freedom, and the tip of the robot arm 202 may be provided with a jig for fixing the camera 201 to be calibrated. The camera 201 to be calibrated may be any camera, for example, one or more of the control devices 203 in the camera that needs to perform parameter calibration in batch may be an electronic device with a control function, such as an upper computer, a server, and the like. The control device 203 may be located in the same physical space as the robot arm 202, or may be located in a different physical space, that is, the control device 203 may control the robot arm 202 at a short distance, or may control the robot arm 202 at a remote distance.

In some embodiments, the control device 203 may control the robotic arm 202 through a network. The robot arm 202, after receiving a robot arm control signal of the control device 203, performs a corresponding action based on the control of the robot arm 202.

In some embodiments, the control device 203 may further perform network communication or wired connection with the camera 201 to be calibrated, the control device 203 may control the camera 201 to be calibrated to perform image acquisition, and the mechanical arm 202 may drive the camera 201 to be calibrated to move or rotate. In other embodiments, the above method of the present disclosure may be executed on other devices, such as a cloud or a remote server, and the control device 203 may execute a control program received from the cloud or the remote server to control the camera 201 to be calibrated and the robot arm 202, and transmit the camera calibration data acquired by the camera 201 to be calibrated in the process to the cloud or the remote server to calibrate the camera parameters.

In some embodiments, the robot arm control signal is used to control the robot arm 202 to perform corresponding actions, for example, the robot arm 202 may move along a preset motion track based on the robot arm control signal, and the actions of the robot arm 202 on the preset motion track may include, but are not limited to, translational actions in various directions and rotational actions in various different angles in various directions. In some embodiments, the robot arm control signal may be a set of continuous control signals, each of which may include, but is not limited to, a translation direction, a translation distance, a rotation direction, a rotation angle, and a motion time, etc. In the embodiment of the present disclosure, sending the robot arm control signal to the robot arm 202 may be understood as sending the set of control signals to the robot arm 202 continuously from the first of a set of control signals, where the set of control signals may be sent only once or may be sent repeatedly for multiple times, and may specifically be determined according to a requirement for actually performing parameter calibration, which is not limited herein.

In some embodiments, the camera control signals are used to control the camera 201 to be calibrated to take successive shots or to take shots at appropriate times. In the calibration process of the camera parameters, the camera 201 to be calibrated is used for collecting the image of the calibration plate, and the internal reference and/or the external reference of the camera are calibrated based on the image. In the embodiment of the present disclosure, the camera 201 to be calibrated may be fixed at the end of the robot arm 202, and the control device 203 controls the robot arm 202 to transport the camera to an image acquisition area, in which a calibration board may be disposed, and the camera may be aligned with the calibration board when acquiring an image. The mechanical arm 202, under the control of the control device 203, drives the camera 201 to be calibrated to capture images of the calibration plate from different angles at different positions, and can acquire dozens of images of the calibration plate at different positions and/or different angles under normal conditions, thereby completing calibration of camera parameters by executing a corresponding parameter calibration algorithm.

In some embodiments, camera calibration data collected by the camera 201 to be calibrated may be used as input to the parameter calibration algorithm, which may include, but is not limited to, calibration plate images. In other embodiments, the camera calibration data may further include inertial navigation data and a data acquisition time, which may be an acquisition time of the calibration plate image and the inertial navigation data. The different parameter calibration algorithms may use different camera calibration data, and the camera calibration data may be acquired according to the requirements of the parameter calibration algorithm to be used currently. The camera 201 to be calibrated can transmit the acquired camera calibration data to the control device 203 through USB or network, and the control device 203 can directly execute the parameter calibration algorithm to calculate the parameters of the camera 201 to be calibrated; the control device 203 may also upload camera calibration data received from the camera to a cloud or a remote server, and the cloud or the remote server performs a parameter calibration algorithm to calculate parameters of the camera 201 to be calibrated.

In the embodiment of the present disclosure, in the process of calibrating the camera parameters, the camera 201 to be calibrated is fixed on the mechanical arm 202, the mechanical arm 202 drives the camera 201 to be calibrated to move along the preset motion trajectory under the control of the control device 203, in the process of moving along with the mechanical arm 202, the camera 201 to be calibrated acquires camera calibration data under the control of the control device 203, and then the control device 203 calculates the camera parameters of the camera 201 to be calibrated based on the camera calibration data. In this way, the mechanical arm 202 drives the camera 201 to be calibrated to move along the preset motion track, and the camera 201 to be calibrated can acquire camera calibration data from different angles at a fixed position under the control of the control device 203, so that the motion track of each camera 201 to be calibrated and the position of the acquired camera calibration data have consistency, and therefore, the method can be suitable for parameter calibration of batch cameras in a mass production scene, and can also improve the efficiency of parameter calibration.

In an optional implementation manner of this embodiment, a multi-split fixture is disposed on the mechanical arm 202, and is used for fixing a plurality of cameras 201 to be calibrated.

In this optional implementation manner, a plurality of clamps may be arranged at the end of the robot 202, and each clamp may clamp one camera 201 to be calibrated, so that parameter calibration of a plurality of cameras may be implemented in one control process, and the efficiency of parameter calibration is further improved.

In an optional implementation manner of this embodiment, the control device 203 may process the camera calibration data by using a preset internal reference calibration algorithm to obtain the internal reference of the camera 201 to be calibrated.

In this optional implementation manner, calibration of the camera internal parameters may be implemented by using the camera parameter calibration system provided in the embodiment of the present disclosure. The camera intrinsic parameters are parameters related to the characteristics of the camera itself, such as the focal length, pixel size, etc. of the camera. In the calibration process of the camera internal reference, calibration plate images can be acquired from different positions and different angles of the same position, and based on the acquired calibration plate images, the calibration plate images are processed by using an internal reference calibration algorithm to obtain the internal reference of the camera 201 to be calibrated.

In an optional implementation manner of this embodiment, in the process of controlling the mechanical arm 202 and the camera to be fixed, the control device 203 sends a first mechanical arm control signal to the mechanical arm 202 to control the mechanical arm 202 to move from one of the multiple track points to the next, and stops for a first predetermined time after each track point is reached; and during the time that the mechanical arm 202 reaches the track point and stops, the control device 203 sends a first camera control signal to the camera 201 to be calibrated so as to control the camera 201 to be calibrated to acquire an image of the calibration board.

In this optional implementation manner, in the process of calibrating the camera internal parameters by using the camera parameter calibration system provided in the embodiment of the present disclosure, a motion trajectory may be preset, and a plurality of trace points may be set on the motion trajectory, in the process of calibrating the internal parameters, the control device 203 may send a first mechanical arm control signal to the mechanical arm 202 to control the mechanical arm 202 to start from a first trace point of the plurality of trace points and move from one trace point to another trace point, and may stay for a certain time after moving to each trace point, for example, may stay for 200 milliseconds. It should be noted that, the moving of the mechanical arm 202 to a certain track point means that the end of the mechanical arm 202 moves to the track point, and the camera 201 to be calibrated is fixed at the end of the mechanical arm 202, so that it is equivalent to that the mechanical arm 202 transports the camera 201 to be calibrated to each track point, and during the stay period of each track point of the mechanical arm 202, the camera to be fixed can acquire an image of the calibration board under the control of the first camera control signal. The camera to be fixed may capture one or more calibration plate images at the same location.

It should be noted that, a plurality of track points in the preset motion track are all at different positions in front of the calibration board, and the vertical distance from each track point to the calibration board and/or the linear distance from the calibration board may be different or partially the same.

Because the mechanical arm 202 stays at each track point for a certain time, and the camera 201 to be calibrated acquires the calibration plate image during the stay of the mechanical arm 202, the acquired calibration plate image is clearer and is suitable for calibrating the internal reference of the camera.

In an optional implementation manner of this embodiment, in the process of controlling the mechanical arm 202 and the camera to be calibrated, after the mechanical arm 202 reaches one track point and stays at a first predetermined time, the control device 203 further sends a second mechanical arm control signal to the mechanical arm 202 to control the mechanical arm 202 to drive the camera 201 to be calibrated to rotate by a predetermined angle and stay at a second predetermined time; and during the period that the mechanical arm 202 drives the camera 201 to be calibrated to rotate by a preset angle and stay, the control device 203 sends a second camera control signal to the camera 201 to be calibrated so as to control the camera 201 to be calibrated to acquire an image of the calibration board.

In this optional implementation manner, in order to meet the requirement of the internal reference calibration algorithm on the calibration data of the camera and to enable the internal reference calibration algorithm to calculate the internal reference of the camera more accurately, after the mechanical arm 202 stays at a track point for a predetermined time, the control device 203 may send a second mechanical arm control signal to the mechanical arm 202 again to control the mechanical arm 202 to rotate around the X axis, the Y axis, and/or the Z axis at the track point for a certain angle respectively, and stay for a certain time again once per rotation, and during the stay, the control device 203 may control the camera 201 to be calibrated to acquire one or more calibration board images by sending a second camera control signal. In this way, the camera 201 to be calibrated can acquire calibration plate images from different positions and different angles of the same position in the internal reference calibration process, so that the calibrated internal reference is more accurate.

In an optional implementation manner of this embodiment, in the process of controlling the mechanical arm 202 and the camera to be fixed by the control device 203, the control device 203 sends a camera control signal to the camera 201 to be calibrated to control the camera 201 to be calibrated to acquire the image of the calibration board at a predetermined frequency; and the control device 203 further sends a first robot control signal to the robot arm 202 to control the robot arm 202 to move from one of the plurality of trace points to the next and stay for a first predetermined time after reaching each of the trace points; after the mechanical arm 202 reaches one of the track points and stays at a first preset time, the control device 203 further sends a second mechanical arm control signal to the mechanical arm 202 to control the mechanical arm 202 to drive the camera 201 to be calibrated to rotate by a first preset angle and stay at a second preset time.

This alternative implementation differs from the internal reference calibration process mentioned in the previous embodiment. In the embodiment, a motion trajectory may also be preset, a plurality of trajectory points may be set on the motion trajectory, in the internal reference calibration process, the control device 203 may start from a first trajectory point of the plurality of trajectory points by sending a first mechanical arm control signal to the mechanical arm 202, move from one trajectory point to another trajectory point, and may stay for a certain time after moving to each trajectory point, for example, may stay for 200 milliseconds. It should be noted that, the moving of the mechanical arm 202 to a certain track point means that the end of the mechanical arm 202 moves to the track point, and the camera 201 to be calibrated is fixed at the end of the mechanical arm 202, which is equivalent to that the mechanical arm 202 stays for a period of time after the camera 201 to be calibrated is conveyed to each track point by the mechanical arm 202.

Unlike the previous embodiment, in this embodiment, the control device 203 sends a camera control signal to the camera 201 to be calibrated after the start of the internal reference calibration, so as to control the camera 201 to be calibrated to acquire the calibration plate image at a predetermined frequency. That is, in this embodiment, regardless of whether the camera 201 to be calibrated is carried to the locus point, data acquisition is started as long as the camera control signal of the control device 203 is received, and for example, calibration board images may be acquired at a frequency of 20 frames per second.

It should be noted that the time for sending the camera control signal to the camera 201 to be calibrated and the time for sending the first robot arm control signal to the robot arm 202 may be the same or different, and the sequence may be adjustable.

Since the mechanical arm 202 stays at each track point for a certain time and the camera 201 to be calibrated acquires the calibration plate image at a predetermined frequency, the camera 201 to be calibrated can acquire the calibration plate image in a static state during the stay of the mechanical arm 202, and the calibration plate image acquired in the static state is clear and suitable for calibrating the internal reference of the camera.

The control device 203 may acquire all calibration board images acquired by the camera 201 to be calibrated, and select a calibration board image in a static state from the calibration board images, so that the selected calibration board image is input to the internal reference calibration algorithm for processing, and the internal reference of the camera 201 to be calibrated is obtained through calculation. In some embodiments, the control device 203 may select the calibration board images acquired in the static state by comparing pixel point differences between several consecutive calibration board images before and after, and then select one of the calibration board images acquired in each static state, input the selected calibration board image to the internal reference calibration algorithm, and calculate the internal reference of the camera 201 to be calibrated.

Similarly, in order to meet the requirement of the internal reference calibration algorithm on the calibration data of the camera and to enable the internal reference calibration algorithm to calculate the internal reference of the camera more accurately, the control device 203 may send a second robot arm control signal to the robot arm 202 again after the robot arm 202 stays at a track point for a predetermined time, so as to control the robot arm 202 to rotate around the X axis, the Y axis and/or the Z axis at the track point by a first predetermined angle respectively, and stay for a certain time every time of rotation, and during the stay, the control device 203 may control the camera 201 to be calibrated to acquire one or more calibration board images by sending the second camera control signal. In this way, the camera 201 to be calibrated can acquire calibration plate images from different positions and different angles of the same position in the internal reference calibration process, so that the calibrated internal reference is more accurate.

In an optional implementation manner of this embodiment, the camera 201 to be calibrated is further provided with an inertial navigation device, and the camera calibration data further includes inertial navigation data and data acquisition time; the control device 203 processes the camera calibration data by using a preset external reference calibration algorithm to obtain the external reference of the camera 201 to be calibrated.

For the same camera or the same group of cameras, after the camera calibration system provided by the embodiment of the disclosure is executed to complete the camera internal parameter calibration, the camera calibration process is executed again, so that the calibration of the camera external parameters is completed. It is understood that the calibration of the external reference of the camera may be completed first, and then the calibration of the internal reference may be completed, which may be specifically set according to an actual situation, and is not limited herein.

In an optional implementation manner of this embodiment, in the process that the control device 203 controls the mechanical arm 202 and the camera to be fixed, the control device 203 sends a third camera control signal to the camera 201 to be calibrated to control the camera 201 to be calibrated to acquire a calibration plate image and acquire inertial navigation data at a predetermined frequency; the control device 203 further sends a third robot arm control signal to the robot arm 202 to control the robot arm 202 to move to a preset preparation point, and then sequentially perform the following rotation motions around the X axis, the Y axis, and the Z axis: the positive direction is rotated by a second preset angle, and the negative direction is rotated by a second preset angle which is 2 times, and then the rotation returns to the positive direction; the control device 203 further sends a fourth robot arm control signal to the robot arm 202 to control the robot arm 202 to perform the following translation motions in the X-axis direction, the Y-axis direction and the Z-axis direction at the preset preparation point: the positive direction translates for a preset distance, and the negative direction translates for 2 times of the preset distance and then returns to the positive direction.

In this optional implementation manner, this embodiment is applicable to the calibration process of the external reference of the camera, and in this embodiment, a preset preparation point may be preset, and the preset preparation point may be, for example, right opposite to the central point of the calibration board and be about 2 meters away from the calibration board. In the external reference calibration process, the control device 203 may move to a preset preparation point by sending a third robot control signal to the robot 202, and then control the robot 202 to rotate around the X axis, the Y axis, and the Z axis at the preset preparation point. For example, the X-axis is rotated forward by a second predetermined angle, then rotated backward by a second predetermined angle that is 2 times larger than the X-axis (corresponding to a reverse rotation of the X-axis by the second predetermined angle from the original angle), and then rotated forward by the second predetermined angle back to the original angle. Similarly, the above-described rotation motions are performed around the Y-axis and the Z-axis, respectively. To obtain more data, after one pass is completed, the above actions may be repeated a number of times, e.g., 2-5 times.

It should be noted that, the moving of the mechanical arm 202 to the preset preparation point means that the end of the mechanical arm 202 moves to the preset preparation point, and the end of the mechanical arm 202 drives the camera 201 to be calibrated to rotate by a certain angle.

After the completion of the above-described rotation motion, the control device 203 may further control the robot arm 202 to sequentially perform the translation motion in the X-axis, Y-axis, and Z-axis directions from the preset preparation point by sending a fourth robot arm control signal to the robot arm 202. In some embodiments, the robotic arm 202 may be controlled to move positively along the X-axis a predetermined distance, for example, 1 meter, then move negatively a predetermined distance 2 times (corresponding to moving negatively a predetermined distance after returning to positive), and then return to positive. Similarly, the control device 203 may control the robotic arm 202 to translate positively and negatively along the Y-axis and Z-axis, respectively, and then return to positive.

In this alternative embodiment, the control device 203 may send a third camera control signal to the camera 201 to be calibrated after the external reference calibration is started or after sending a third robot arm control signal to the robot arm 202, so as to control the camera 201 to be calibrated to acquire calibration plate images and inertial navigation data at a predetermined frequency. In this embodiment, the camera 201 to be calibrated may start to acquire camera calibration data before or after the movement of the robot arm 202 and after receiving the third camera control signal, for example, may acquire calibration plate images at a frequency of 20 frames per second and acquire inertial navigation data at the same time as acquiring the calibration plate images. The camera 201 to be calibrated may transmit the acquired calibration plate image, inertial navigation data, and data acquisition time to the control device 203 together.

It should be noted that the time for the control device 203 to send the third camera control signal to the camera 201 to be calibrated and the time for sending the third robot arm control signal to the robot arm 202 may be the same or different, and the sequence may be adjustable.

Since the camera 201 to be calibrated acquires the calibration plate image and the inertial navigation data at a predetermined frequency in the motion process of the mechanical arm 202, although in the external reference calibration process, the external reference calibration algorithm does not need to process the acquired image content, that is, a clear calibration plate image is not needed, the mechanical arm 202 may not stay at any position in the process as long as the camera 201 to be calibrated acquires the calibration plate image and the inertial navigation data at the position where the mechanical arm 202 arrives at a certain acquisition frequency.

The control device 203 can determine the position of the camera 201 to be calibrated when the calibration plate image and the inertial navigation data are acquired based on the acquisition time of the calibration plate image and the inertial navigation data and the motion track of the mechanical arm 202, and the control device 203 inputs the calibration plate image, the inertial navigation data, the data acquisition time, the acquisition position and the like to the external reference calibration algorithm, so that the external reference of the camera 201 to be calibrated can be calculated.

Fig. 3(a) and 3(b) are schematic diagrams illustrating application scenarios of camera internal reference calibration and camera external reference calibration according to an embodiment of the present disclosure. Assuming that the robot arm 202 is provided with three-by-three clamps, three cameras 201 to be calibrated are clamped together. The control device 203 sends a mechanical arm control signal moving along a preset motion track to the mechanical arm 202, and the control device 203 also sends camera control signals to the three cameras 201 to be calibrated at the same time so as to control the cameras 201 to be calibrated to acquire images at the frequency of 20 frames per second.

As shown in fig. 3(a), during the internal reference calibration process, the control device controls the mechanical arm to move as follows:

1. the tail end of the mechanical arm moves to a motion track point 1 under the control of the control equipment, a calibration plate is arranged in front of the motion track point 1, and the motion track point 1 is 1 meter away from the calibration plate.

2. After the tail end of the mechanical arm is stationary at the motion track point 1 for 100 milliseconds, the mechanical arm respectively rotates around X, Y and the Z axis in the positive direction and the negative direction under the control of the control equipment, and the mechanical arm is stationary again for 100 milliseconds every time, and the rotation angle can be in the range of 1-20 degrees.

3. And the tail end of the mechanical arm moves to a track point 2, after the mechanical arm is stationary for 100 milliseconds, the mechanical arm respectively rotates around X, Y and the Z axis in a positive direction and a negative direction under the control of the control equipment, and the mechanical arm is stationary again for 100 milliseconds every time, and the rotation angle can be in the range of 1-20 degrees.

4. And by analogy, the tail end of the mechanical arm moves to track points 3-12 respectively, and the motion is executed at each track point.

In the motion process, the camera to be calibrated acquires images at the frequency of 20 frames per second, and transmits the acquired images back to the control end.

After the completion of the movement as shown in fig. 3(a), a calibration process of the camera external parameter is performed.

As shown in fig. 3(b), during the calibration of the external camera parameter, the control device controls the mechanical arm to move as follows:

1. the tail end of the mechanical arm drives the camera to be calibrated to move to a preparation point.

2. The tail end of the mechanical arm rotates around the X axis positively for 30 degrees, negatively for 60 degrees and then positively for 30 degrees.

3. The tail end of the mechanical arm rotates around the Y axis for 30 degrees in the positive direction, 60 degrees in the negative direction and 30 degrees in the positive direction.

4. The tail end of the mechanical arm rotates 30 degrees around the Z axis in the positive direction, rotates 60 degrees in the negative direction and rotates 30 degrees in the positive direction.

5. Repeating the steps 2-4 again or more.

6. The tail end of the mechanical arm translates 1 meter leftwards, 2 meters rightwards and 1 meter leftwards.

7. The tail end of the mechanical arm translates forwards for 1 meter, translates backwards for 2 meters and translates forwards for 1 meter.

8. The tail end of the mechanical arm translates upwards for 1 meter, translates downwards for 2 meters and translates upwards for 1 meter.

The following are embodiments of the disclosed apparatus that may be used to perform embodiments of the disclosed methods.

Fig. 4 shows a block diagram of a camera parameter calibration apparatus according to an embodiment of the present disclosure. The apparatus may be implemented as part or all of an electronic device through software, hardware, or a combination of both. As shown in fig. 4, the camera parameter calibration apparatus includes:

the first sending module 401 is configured to send a mechanical arm control signal to a mechanical arm for fixing a camera to be calibrated, so as to control the mechanical arm to drive the camera to be fixed to move along a preset motion track;

a second sending module 402, configured to send a camera control signal to the camera to be calibrated, so as to control the camera to be calibrated to acquire camera calibration data in a process of moving along with the mechanical arm;

an acquisition module 403 configured to acquire the camera calibration data; the camera calibration data comprises calibration plate images collected by the camera to be fixed;

a determining module 404 configured to determine camera parameters of the camera to be calibrated based on the camera calibration data.

In an optional implementation manner of this embodiment, the determining module includes:

and the first processing submodule is configured to process the camera calibration data by using a preset internal reference calibration algorithm so as to obtain the internal reference of the camera to be calibrated.

In an optional implementation manner of this embodiment, the camera parameter includes an internal parameter of the camera to be calibrated, and the first sending module includes:

a first sending submodule configured to send a first robot control signal to the robot arm to control the robot arm to move from one of a plurality of track points to a next and to stay for a first predetermined time after reaching each of the track points.

In an optional implementation manner of this embodiment, the second sending module includes:

and the second sending sub-module is configured to send a first camera control signal to the camera to be calibrated during the time when the mechanical arm reaches the track point and stays, so as to control the camera to be calibrated to acquire an image of the calibration board.

In an optional implementation manner of this embodiment, after the mechanical arm reaches one of the track points and stops for a first predetermined time, the apparatus further includes:

The third sending submodule is configured to send a second mechanical arm control signal to the mechanical arm so as to control the mechanical arm to drive the camera to be calibrated to rotate by a preset angle and stay for a second preset time;

and the fourth sending submodule is configured to send a second camera control signal to the camera to be calibrated during the period that the mechanical arm drives the camera to be calibrated to rotate by a preset angle and stay, so as to control the camera to be calibrated to acquire an image of the calibration plate.

a fifth sending submodule configured to send a first robot arm control signal to the robot arm to control the robot arm to move from one of the plurality of track points to the next and stay for a first predetermined time after reaching each of the track points;

and the sixth sending submodule is configured to send a second mechanical arm control signal to the mechanical arm after the mechanical arm reaches one track point and stays at the first preset time, so as to control the mechanical arm to drive the camera to be calibrated to stay at the second preset time after rotating by the first preset angle.

a seventh sending sub-module, configured to send a camera control signal to the camera to be calibrated, so as to control the camera to be calibrated to acquire a calibration plate image as camera calibration data at a predetermined frequency in a process that the mechanical arm moves according to the first mechanical arm control signal and/or the second mechanical arm control signal.

In an optional implementation manner of this embodiment, the camera parameters include external parameters of the camera to be calibrated, the camera to be calibrated is further provided with an inertial navigation device, and the camera calibration data further includes inertial navigation data and data acquisition time; the determining module includes:

and the second processing submodule is configured to process the camera calibration data by using a preset external reference calibration algorithm so as to obtain the external reference of the camera to be calibrated.

In an optional implementation manner of this embodiment, the first sending module includes:

an eighth sending submodule configured to send a third robot arm control signal to the robot arm to control the robot arm to perform the following rotation motions around the X axis, the Y axis, and the Z axis in sequence after moving to a preset preparation point: the positive direction is rotated by a second preset angle, and the negative direction is rotated by a second preset angle which is 2 times, and then the rotation returns to the positive direction;

A ninth sending submodule configured to send a fourth robot arm control signal to the robot arm to control the robot arm to perform the following translation motions in the X-axis direction, the Y-axis direction and the Z-axis direction at the preset preparation point in sequence: the positive direction is translated for a preset distance, and the negative direction is translated for a preset distance which is 2 times and then returns to the positive direction;

the second sending module includes:

and the tenth sending submodule is configured to send a third camera control signal to the camera to be calibrated so as to control the camera to be calibrated to acquire calibration plate images and acquire inertial navigation data in the process that the mechanical arm moves according to the third mechanical arm control signal and/or the fourth mechanical arm control signal at a preset frequency.

The camera parameter calibration device in the embodiment of the present disclosure corresponds to the camera parameter calibration method, and specific details may refer to the description of the camera parameter calibration method, which is not described herein again.

As shown in fig. 5, the electronic device 500 includes a processing unit 501, which may be implemented as a CPU, GPU, FPGA, NPU, or the like processing unit. The processing unit 501 may perform various processes in the embodiments of any one of the methods described above of the present disclosure according to a program stored in a Read Only Memory (ROM)502 or a program loaded from a storage section 508 into a Random Access Memory (RAM) 503. In the RAM503, various programs and data necessary for the operation of the electronic apparatus 500 are also stored. The processing unit 501, the ROM502, and the RAM503 are connected to each other by a bus 504. An input/output (I/O) interface 505 is also connected to bus 504.

The following components are connected to the I/O interface 505: an input portion 506 including a keyboard, a mouse, and the like; an output portion 507 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage portion 508 including a hard disk and the like; and a communication section 509 including a network interface card such as a LAN card, a modem, or the like. The communication section 509 performs communication processing via a network such as the internet. The driver 510 is also connected to the I/O interface 505 as necessary. A removable medium 511 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 510 as necessary, so that a computer program read out therefrom is mounted into the storage section 508 as necessary.

In particular, according to embodiments of the present disclosure, any of the methods described above with reference to embodiments of the present disclosure may be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product comprising a computer program tangibly embodied on a machine-readable medium, the computer program comprising program code for performing any of the methods of the embodiments of the present disclosure. In such an embodiment, the computer program may be downloaded and installed from a network through the communication section 509, and/or installed from the removable medium 511.

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowcharts or block diagrams may represent a module, a program segment, or a portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The units or modules described in the embodiments of the present disclosure may be implemented by software or hardware. The units or modules described may also be provided in a processor, and the names of the units or modules do not in some cases constitute a limitation of the units or modules themselves.

As another aspect, the present disclosure also provides a computer-readable storage medium, which may be the computer-readable storage medium included in the apparatus in the above-described embodiment; or it may be a separate computer readable storage medium not incorporated into the device. The computer readable storage medium stores one or more programs for use by one or more processors in performing the methods described in the present disclosure.

The foregoing description is only exemplary of the preferred embodiments of the disclosure and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the invention in the present disclosure is not limited to the specific combination of the above-mentioned features, but also encompasses other embodiments in which any combination of the above-mentioned features or their equivalents is possible without departing from the inventive concept. For example, the above features and (but not limited to) the features disclosed in this disclosure having similar functions are replaced with each other to form the technical solution.

Claims

1. A camera parameter calibration method comprises the following steps:

2. The method according to claim 1, wherein the camera parameters include internal parameters of the camera to be calibrated, and the sending of the robot arm control signal to the robot arm for fixing the camera to be calibrated to control the robot arm to drive the camera to be fixed to move along a preset motion trajectory includes:

3. The method of claim 2, wherein sending camera control signals to the camera to be calibrated to control the camera to be calibrated to acquire camera calibration data during movement with the robotic arm comprises:

4. The method of claim 2, wherein after the robotic arm reaches one of the trajectory points and dwells for a first predetermined time, the method further comprises:

5. The method according to claim 1, wherein the camera parameters include internal parameters of the camera to be calibrated, and a mechanical arm control signal is sent to a mechanical arm for fixing the camera to be calibrated so as to control the mechanical arm to drive the camera to be fixed to move along a preset motion trajectory, including:

6. The method of claim 5, wherein sending camera control signals to the camera to be calibrated to control the camera to be calibrated to acquire camera calibration data during movement with the robotic arm comprises:

7. The method according to any one of claims 1-6, wherein determining camera parameters of the camera to be calibrated based on the camera calibration data comprises:

8. The method according to any one of claims 1 to 6, wherein the camera parameters comprise external parameters of the camera to be calibrated, an inertial navigation device is further arranged on the camera to be calibrated, the camera calibration data further comprises inertial navigation data and data acquisition time; determining camera parameters of the camera to be calibrated based on the camera calibration data, including:

9. The method of claim 8, wherein sending a robot arm control signal to a robot arm for fixing the camera to be calibrated so as to control the robot arm to drive the camera to be fixed to move along a preset motion track comprises:

sending a third mechanical arm control signal to the mechanical arm to control the mechanical arm to move to a preset preparation point and then sequentially rotate around an X axis, a Y axis and a Z axis as follows: the positive direction is rotated by a second preset angle, and the negative direction is rotated by a second preset angle which is 2 times, and then the rotation returns to the positive direction;

10. The method according to any one of claims 1 to 6 and 9, wherein a multi-split clamp is arranged on the mechanical arm and used for fixing a plurality of cameras to be calibrated.

11. A camera calibration system, comprising: a camera to be calibrated, a mechanical arm and control equipment; wherein the content of the first and second substances,

the camera to be calibrated is fixed at the tail end of the mechanical arm;

12. The camera calibration system of claim 11, wherein the camera parameters include external parameters of the camera to be calibrated;

13. A computer program product comprising computer instructions, wherein the computer instructions, when executed by a processor, implement the method of any one of claims 1-10.