CN114494446A - External parameter calibration method and system and electronic equipment - Google Patents
External parameter calibration method and system and electronic equipment Download PDFInfo
- Publication number
- CN114494446A CN114494446A CN202011146976.1A CN202011146976A CN114494446A CN 114494446 A CN114494446 A CN 114494446A CN 202011146976 A CN202011146976 A CN 202011146976A CN 114494446 A CN114494446 A CN 114494446A
- Authority
- CN
- China
- Prior art keywords
- odometer
- visual
- calibration
- camera module
- mobile robot
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C22/00—Measuring distance traversed on the ground by vehicles, persons, animals or other moving solid bodies, e.g. using odometers, using pedometers
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Theoretical Computer Science (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Manipulator (AREA)
Abstract
An external parameter calibration method, a system and electronic equipment thereof are used for calibrating external parameters between a wheel type odometer and a camera module which are carried on a mobile robot. The external reference calibration method comprises the following steps: controlling the movement of a mobile device according to a plurality of preset poses to respectively and correspondingly move the mobile robot, and controlling the camera module of the mobile robot to synchronously acquire a plurality of vision calibration images, wherein the mobile device is rigidly connected with the wheel type odometer of the mobile robot; obtaining a plurality of odometer poses of the wheel type odometer relative to a reference coordinate system through coordinate transformation according to the plurality of preset poses; obtaining a plurality of visual poses of the camera module relative to the reference coordinate system through a visual positioning technology according to the plurality of visual calibration images; and calculating external parameters between the wheel type odometer and the camera module based on the position and the corresponding visual position of the odometer.
Description
Technical Field
The invention relates to the technical field of multi-sensor calibration, in particular to an external reference calibration method, a system and electronic equipment thereof.
Background
With the rapid development of mobile robotics and unmanned technologies, the multi-sensor fusion positioning technology has become a hot spot of current research. Because the mobile robot carrying the wheel type odometer can obtain the positioning information of the robot according to dead reckoning, and the camera module carried by the mobile robot can obtain the positioning information of the Visual Odometer (VO) of the mobile robot according to the Visual positioning technology, the two positioning information are fused, and more accurate and robust positioning can be obtained. In order to improve the fusion positioning precision between the wheel type odometer and the camera module, external parameters of the two sensors need to be calibrated.
At present, the method for calibrating the external reference between the wheel type odometer and the camera module generally comprises the steps of firstly enabling the wheel type odometer to move on a floor for a certain track, then carrying out pose recursion through a coding value of the wheel type odometer and calculating pose transformation relation of adjacent moments; meanwhile, the pose change relation of the camera module at adjacent moments is obtained through a visual positioning technology, so that external parameters between the wheel type odometer and the camera module are calculated according to the pose change relation of the wheel type odometer and the pose change relation of the camera module, and the external parameters of the two sensors are calibrated.
However, due to the influence of the friction force of the floor material, the pose recurrence precision of the wheel type odometer is low, and the internal parameter error of the wheel type odometer is also transmitted to the calculated external parameter, so that the calculated external parameter precision is also low. In addition, the external reference calibration method needs to occupy a large space, needs a long operation time, and is not suitable for mass production calibration of the mobile robot.
Disclosure of Invention
The invention has an advantage of providing an external reference calibration method, a system and an electronic device thereof, which can reduce the external reference calibration error between the wheel type odometer and the camera module and are beneficial to improving the external reference calibration precision between the wheel type odometer and the camera module.
Another advantage of the present invention is to provide an external reference calibration method, a system and an electronic device thereof, wherein in an embodiment of the present invention, the external reference calibration method can improve external reference calibration efficiency between the wheel-type odometer and the camera module.
Another advantage of the present invention is to provide an external reference calibration method, a system and an electronic device thereof, wherein in an embodiment of the present invention, the external reference calibration method only needs to occupy a small space to complete external reference calibration between the wheel-type odometer and the camera module, which is beneficial to mass production calibration of products.
Another advantage of the present invention is to provide an external parameter calibration method, a system and an electronic device thereof, wherein in an embodiment of the present invention, the external parameter calibration method can rapidly and accurately acquire data, so that both the efficiency and the accuracy of external parameter calibration can be improved.
Another advantage of the present invention is to provide an external reference calibration method, a system and an electronic device thereof, wherein in an embodiment of the present invention, the external reference calibration method can increase corresponding calibration constraints in combination with specific application scenarios, which is helpful for simplifying an algorithm of external reference calibration, greatly reducing the calculation amount of external reference calibration, and improving calibration efficiency.
Another advantage of the present invention is to provide an external reference calibration method, a system and an electronic device thereof, wherein the external reference calibration method does not need to adopt a complex structure and a huge amount of calculation, and has low requirements on software and hardware. Therefore, the present invention successfully and effectively provides a solution, which not only provides an external reference calibration method, a system thereof and an electronic device, but also increases the practicability and reliability of the external reference calibration method, the system thereof and the electronic device.
To achieve at least one of the above advantages or other advantages and objects, the present invention provides an external reference calibration method for calibrating an external reference between a wheel type odometer mounted on a mobile robot and a camera module, wherein the external reference calibration method includes:
controlling the movement of a mobile device according to a plurality of preset poses to respectively and correspondingly move the mobile robot, and controlling the camera module of the mobile robot to synchronously acquire a plurality of visual calibration images, wherein the mobile device is rigidly connected with the wheel-type odometer of the mobile robot;
obtaining a plurality of odometer poses of the wheel type odometer relative to a reference coordinate system through coordinate transformation according to the plurality of preset poses;
obtaining a plurality of visual poses of the camera module relative to the reference coordinate system through a visual positioning technology according to the plurality of visual calibration images; and
and calculating external parameters between the wheel type odometer and the camera module based on the odometer pose and the corresponding visual pose.
According to an embodiment of the invention, the moving means is controlled to translate and rotate the mobile robot in a motion plane.
According to an embodiment of the invention, the mobile device is a robot system, wherein the robot system comprises a robot arm, a base and a connection station, wherein one end of the robot arm is mounted to the base and the other end of the robot arm is connected to the connection station, wherein the connection station is adapted to mount the mobile robot such that the connection station is rigidly connected to the wheeled odometer.
According to an embodiment of the present invention, the coordinate system of the wheel odometer of the mobile robot coincides with the coordinate system of the connection table of the robot arm system, so that the preset pose is the pose of the wheel odometer with respect to the base.
According to an embodiment of the present invention, the camera module is controlled to synchronously capture a pattern with visual feature points for obtaining the visual calibration image with corner feature.
According to an embodiment of the present invention, the pattern with the visual feature points is a label pattern or a checkerboard pattern.
According to an embodiment of the present invention, the external reference calibration method further includes the steps of:
providing a calibration box, wherein the calibration box is internally provided with the pattern with the visual characteristic points, so that when the mobile robot is moved in the calibration box, the camera module can shoot the pattern with the visual characteristic points to obtain the visual calibration image with the angular point characteristics.
According to an embodiment of the invention, the calibration box has a cubic shape, and the top and bottom surfaces of the calibration box are parallel to the plane of motion.
According to an embodiment of the present invention, the pattern with the visual feature points is attached to three side inner walls of the calibration box.
According to another aspect of the present invention, the present invention further provides an external reference calibration system for calibrating an external reference between a wheeled odometer mounted on a mobile robot and a camera module, wherein the external reference calibration system comprises:
the control module is used for controlling the movement of a mobile device according to a plurality of preset poses so as to respectively and correspondingly move the mobile robot and controlling the camera module of the mobile robot to synchronously acquire a plurality of vision calibration images, wherein the mobile device is rigidly connected with the wheel type odometer of the mobile robot;
the coordinate conversion module is used for obtaining a plurality of odometer poses of the wheel type odometer relative to a reference coordinate system through coordinate conversion according to the plurality of preset poses;
the visual positioning module is used for obtaining a plurality of visual poses of the camera module relative to the reference coordinate system through a visual positioning technology according to the plurality of visual calibration images; and
and the external parameter calculation module is used for calculating the external parameters between the wheel type odometer and the camera module based on the odometer pose and the corresponding visual pose.
According to another aspect of the present invention, the present invention further provides an electronic device comprising:
at least one processor configured to execute instructions; and
a memory communicatively coupled to the at least one processor, wherein the memory has at least one instruction, wherein the instruction is executed by the at least one processor to cause the at least one processor to perform some or all of the steps of an external reference calibration method for calibrating external reference between a wheeled odometer and a camera module mounted on a mobile robot, and the external reference calibration method comprises the steps of:
controlling the movement of a mobile device according to a plurality of preset poses to respectively and correspondingly move the mobile robot, and controlling the camera module of the mobile robot to synchronously acquire a plurality of vision calibration images, wherein the mobile device is rigidly connected with the wheel type odometer of the mobile robot;
obtaining a plurality of odometer poses of the wheel type odometer relative to a reference coordinate system through coordinate transformation according to the plurality of preset poses;
obtaining a plurality of visual poses of the camera module relative to the reference coordinate system through a visual positioning technology according to the plurality of visual calibration images; and
and calculating external parameters between the wheel type odometer and the camera module based on the odometer pose and the corresponding visual pose.
According to another aspect of the present invention, there is further provided an electronic apparatus for calibrating external parameters between a wheel odometer mounted on a mobile robot and a camera module, including:
an electronic device body, wherein the electronic device body comprises a mobile device, wherein the mobile device is suitable for being rigidly connected with the wheel-type odometer of the mobile robot; and
at least one external parameter calibration system, wherein the external parameter calibration system is configured on the electronic device body, and the external parameter calibration system comprises:
the control module is used for controlling the movement of the mobile device according to a plurality of preset poses so as to respectively and correspondingly move the mobile robot and controlling the camera module of the mobile robot to synchronously acquire a plurality of vision calibration images;
the coordinate conversion module is used for obtaining a plurality of odometer poses of the wheel type odometer relative to a reference coordinate system through coordinate conversion according to the plurality of preset poses;
the visual positioning module is used for obtaining a plurality of visual poses of the camera module relative to the reference coordinate system through a visual positioning technology according to the plurality of visual calibration images; and
and the external parameter calculation module is used for calculating the external parameters between the wheel type odometer and the camera module based on the odometer pose and the corresponding visual pose.
According to an embodiment of the invention, the mobile device is a robot system, wherein the robot system comprises a robot arm, a base and a connection station, wherein one end of the robot arm is mounted to the base and the other end of the robot arm is connected to the connection station, wherein the connection station is adapted to mount the mobile robot such that the connection station is rigidly connected to the wheeled odometer.
According to an embodiment of the present invention, the electronic device body further includes a calibration box, wherein a pattern with visual feature points is disposed in the calibration box, so that when the mobile robot is moved in the calibration box, the camera module can capture the pattern with visual feature points to obtain the visual calibration image with corner features.
Further objects and advantages of the invention will be fully apparent from the ensuing description and drawings.
These and other objects, features and advantages of the present invention will become more fully apparent from the following detailed description, the accompanying drawings and the claims.
Drawings
Fig. 1 is a schematic flow chart of an external reference calibration method according to an embodiment of the present invention.
FIG. 2 shows a block diagram schematic of an external reference calibration system according to an embodiment of the invention.
FIG. 3 shows a block diagram schematic of an electronic device according to an embodiment of the invention.
FIG. 4 shows one example of another electronic device in accordance with an embodiment of the invention.
Fig. 5 shows a schematic structural diagram of the calibration box of the electronic device according to the above embodiment of the invention.
Fig. 6 is a schematic diagram illustrating a coordinate system transformation relationship in the external reference calibration method according to the above embodiment of the invention.
Detailed Description
The following description is presented to disclose the invention so as to enable any person skilled in the art to practice the invention. The preferred embodiments described below are by way of example only, and other obvious variations will occur to those skilled in the art. The basic principles of the invention, as defined in the following description, may be applied to other embodiments, variations, modifications, equivalents, and other technical solutions without departing from the spirit and scope of the invention.
In the present invention, the terms "a" and "an" in the claims and the description should be understood as meaning "one or more", that is, one element may be one in number in one embodiment, and the element may be more than one in number in another embodiment. The terms "a" and "an" should not be construed as limiting the number unless the number of such elements is explicitly recited as one in the present disclosure, but rather the terms "a" and "an" should not be construed as being limited to only one of the number.
In the description of the present invention, it is to be understood that the terms "first," "second," and the like are used for descriptive purposes only and are not to be construed as indicating or implying relative importance. In the description of the present invention, it should be noted that, unless explicitly stated or limited otherwise, the terms "connected" and "connected" are to be interpreted broadly, and may be, for example, fixedly connected, detachably connected, or integrally connected; can be mechanically or electrically connected; they may be directly connected or indirectly connected through an intermediate. The specific meanings of the above terms in the present invention can be understood by those skilled in the art according to specific situations.
In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above are not necessarily intended to refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, various embodiments or examples and features of different embodiments or examples described in this specification can be combined and combined by one skilled in the art without contradiction.
At present, the existing method for calibrating external parameters between a wheel-type odometer and a camera module generally comprises the steps of enabling the wheel-type odometer to move on a floor for a certain track, then carrying out pose recursion through a coding value of the wheel-type odometer and calculating a pose transformation relation of adjacent moments. However, due to the influence of the friction force of the floor material, the pose recurrence precision of the wheel type odometer is low, and the internal parameter error of the wheel type odometer is also transmitted to the calculated external parameter, so that the calculated external parameter precision is also low. In addition, the external reference calibration method needs to occupy a large space, needs a long operation time, and is not suitable for mass production calibration of the mobile robot.
Therefore, in order to improve the efficiency and the precision of external reference calibration, the application provides an external reference calibration method for a mobile robot, which can directly obtain the pose transformation amount of a wheel type odometer of the mobile robot through the preset pose transformation of a mobile device, thereby effectively shortening the time required by the external reference calibration and improving the precision of the pose transformation amount of the wheel type odometer.
Illustrative method
Referring to fig. 1 of the drawings, an external reference calibration method according to an embodiment of the present invention is illustrated for calibrating external reference between a wheel type odometer and a camera module mounted on a mobile robot. Specifically, as shown in fig. 1, the external reference calibration method may include the steps of:
s100: controlling the movement of a mobile device according to a plurality of preset poses to respectively and correspondingly move the mobile robot, and controlling the camera module of the mobile robot to synchronously acquire a plurality of vision calibration images, wherein the mobile device is rigidly connected with the wheel type odometer of the mobile robot;
s200: according to the preset pose, a plurality of odometer poses of the wheel type odometer relative to a reference coordinate system are obtained through coordinate system conversion;
s300, obtaining a plurality of visual poses of the camera module relative to the reference coordinate system through a visual positioning technology according to the visual calibration image; and
s400: and calculating external parameters between the wheel type odometer and the camera module based on the odometer pose and the corresponding visual pose.
It should be noted that, in the external reference calibration method of the present application, the mobile device is used to move the mobile robot according to the preset pose, and the mobile device is rigidly connected to the wheel type odometer of the mobile robot, so that the external reference calibration method of the present invention can obtain the pose of the wheel type odometer based on the preset pose directly through coordinate system transformation, avoid the pose of the wheel type odometer from recurrence error due to the influence of floor material, and completely eliminate the internal reference error of the wheel type odometer, thereby ensuring that the external reference calibration accuracy of the external reference calibration method of the present application to the mobile robot is greatly improved.
In addition, according to the external reference calibration method, the pose of the wheel type odometer can be obtained only through simple coordinate system conversion calculation without calculating the pose transformation relation of adjacent moments through pose recursion of the coded value of the wheel type odometer, so that the calculation amount for obtaining the pose of the odometer is greatly reduced, and therefore the external reference calibration efficiency of the mobile robot can be greatly improved.
It is worth mentioning that according to the working principle of the wheel-type odometer of the mobile robot, the wheel-type odometer can only accurately acquire pose changes on a plane, that is, the mobile robot with the wheel-type odometer is generally applied to application scenes where machines such as a sweeping robot, an inspection robot, an intelligent logistics vehicle or an intelligent automobile move on a flat ground or a floor. Therefore, in the step S100 of the external reference calibration method of the above-described embodiment of the present application: the moving device is preferably controlled to translate and rotate the mobile robot within a movement plane so that the moving device moves in a plurality of the preset poses, respectively. It will be appreciated that there is both translation in the plane of motion and rotation in the plane of motion between any two adjacent said preset poses to ensure that a plurality of valid said odometry poses are obtained.
In addition, just because the mobile robot only translates and rotates on the motion plane, but not translates or rotates in a space outside the motion plane, the pose transformation of the mobile robot is simplified, so when the external parameters between the wheel type odometer and the camera module of the mobile robot are solved based on the odometer pose and the visual pose, a model for solving the external parameters is simplified, the calculation amount required for solving the external parameters is greatly reduced, and the external parameter calibration method is favorable for further improving the external parameter calibration efficiency of the mobile robot.
Illustratively, as shown in fig. 4, the moving device in the external reference calibration method of the present application may be, but is not limited to, implemented as a robot arm system 810, wherein the robot arm system includes a robot arm 811, a base 812 and a connecting station 813, wherein one end of the robot arm 811 is mounted to the base 812, and the other end of the robot arm 811 is connected to the connecting station 813, wherein the connecting station 813 is adapted to mount the mobile robot to rigidly connect the wheel odometer, so as to move the connecting station 813 according to the preset pose with respect to the base 812 through the robot arm 811, thereby moving the mobile robot mounted to the connecting station 813. It is to be understood that the robot arm of the robot arm system may be, but is not limited to being, implemented as an articulated robot arm, a rigid robot arm, or a flexible robot arm as long as the requirement of the preset posture can be satisfied.
Preferably, the coordinate system of the wheel odometer of the mobile robot coincides with the coordinate system of the connection station of the robot arm system, i.e. the coordinate system of the wheel odometer of the mobile robot and the coordinate system of the connection station of the robot arm system are preferably embodied in one and the same coordinate system, so that the predetermined pose is the connection station being in phaseWith respect to the attitude of the base, that is, the attitude of the wheeled odometer with respect to the base. For example, a coordinate system O is constructed with the base of the robot arm system as an origin2-x2y2z2Such that said predetermined pose is said connecting table with respect to said coordinate system O2-x2y2z2I.e. the wheeled odometer with respect to the coordinate system O2-x2y2z2The pose of (1).
It is noted that in order to accurately calculate the external reference between the wheel type odometer and the camera module of the mobile robot, the odometer pose of the wheel type odometer and the visual pose of the camera module must be compared and calculated in the same reference coordinate system. Since the pose of the camera module obtained by the visual positioning technique is usually the camera module relative to the environmental coordinate system O1-x1y1z1In said step S200 and said step S300 of said external reference calibration method of the present application, said reference coordinate system is preferably implemented as said environmental coordinate system O1-x1y1z1At the moment, the preset pose is converted from the coordinate system O only through the coordinate system2-x2y2z2By converting to the environment coordinate system, the pose of the wheel type odometer relative to the environment coordinate system can be obtained. Of course, in other examples of the present application, the external reference calibration method may also convert the camera module with respect to the environmental coordinate system O through coordinate system transformation1-x1y1z1Is converted into the pose of the camera module relative to the coordinate system O2-x2y2z2The pose of (a) can still be converted into the pose of the camera module and the pose of the wheel odometer with respect to the same reference coordinate system (i.e., the coordinate system O)2-x2y2z2) To ensure that normal external parameter calculation is performed subsequently.
More preferably, said plane of motionParallel to the coordinate system O2-x2y2z2X in (2)2y2And the plane is used for simplifying parameter change in the preset pose, so that the difficulty of external parameter calculation is reduced, and the corresponding calculation amount is further reduced.
Most preferably, the environment coordinate system O1-x1y1z1X in (2)1Axis, y1Axis and z1The axes being parallel to the coordinate system O in turn2-x2y2z2X in (2)2Axis, y2Axis and z2And the axis is used for simplifying the conversion complexity of a subsequent coordinate system and improving the efficiency of external parameter calibration. In other words, the plane of motion is parallel to the ambient coordinate system O1-x1y1z1X of1y1And (4) a plane.
It should be noted that the camera module of the mobile robot can directly shoot a natural environment to obtain the visual calibration image containing natural features, so as to extract the natural features from the visual calibration image through the visual positioning technology to solve the pose of the camera module relative to the environment coordinate system. However, since the pose accuracy obtained by the natural features is low, in order to improve the visual pose accuracy of the camera module, in the step S100 of the external reference calibration method according to the above embodiment of the present invention:
the camera module is controlled to synchronously shoot the pattern with the visual characteristic points, and the pattern is used for obtaining the visual calibration image with the corner point characteristics.
It can be understood that the accuracy of the visual pose obtained by the visual positioning technology by using the corner point features extracted from the visual calibration image is higher than that by using the natural features extracted from the visual calibration image, so that the accuracy of the external reference calibration method for the mobile robot is improved.
Preferably, the pattern with the visual feature points may be, but is not limited to being, implemented as a label pattern, such as a two-dimensional code pattern. Of course, in other examples of the present application, the pattern with the visual feature points may also be implemented as a checkerboard pattern.
In addition, according to the above embodiment of the present application, the external reference calibration method may further include the steps of:
providing a calibration box, wherein the pattern with the visual characteristic points is arranged in the calibration box, so that when the mobile robot is moved in the calibration box, the camera module can shoot the pattern with the visual characteristic points to obtain the visual calibration image with the corner point characteristics.
Preferably, as shown in fig. 4 and 5, the calibration box has a cubic shape, and both the top and bottom surfaces of the calibration box are parallel to the movement plane. Of course, in other examples of the invention, the calibration boxes may also be implemented in other types of shapes, such as cylinders, cones, etc.
More preferably, the pattern with the visual characteristic points is attached to three side inner walls of the calibration box, so that the camera module of the mobile robot moving on the motion plane can shoot the pattern with the visual characteristic points in various poses.
Exemplarily, the bottom angle of the calibration box is taken as an origin O1Taking three sides of the calibration box as x1Axis, y1Axis and z1And (3) the axis constructs the environment coordinate system so as to determine the pose of the pattern with the visual characteristic points in the calibration box, and the visual pose of the camera module relative to the environment coordinate system can be solved according to the visual calibration image.
In particular, the internal parameters of the camera module of the mobile robot are calibrated, and the camera module is corrected for image distortion. Therefore, the relative motion of the mechanical arm is controlled for n times in each external reference calibration, and the external reference calibration method can obtain the pose transformation quantity of the adjacent odometer according to the adjacent motion of the mechanical arm; meanwhile, the external reference calibration method can also obtain adjacent visual pose transformation quantity according to the detected pose of the two-dimensional code pattern; and finally solving the external reference relation between the camera module and the wheel type odometer according to the n relative transformation pairs.
It should be noted that, since the odometer pose of the wheel type odometer is directly obtained by coordinate system transformation using the preset pose of the mobile device, and the mobile robot does not need to be moved on the floor to obtain a pose recursion through the code value of the wheel type odometer, the size of the calibration box does not need to be large, for example, the side length of the calibration box only needs 1.5m to meet the requirement of external reference calibration, so that the external reference calibration method of the present application does not need to occupy a large space, and is particularly suitable for mass production calibration of products.
For example, it is assumed that the camera module is calibrated with internal parameters and the image is corrected for distortion before data processing. And controlling the mechanical arm to perform k times of relative motion every time of external parameter calibration so as to obtain pose transformation quantities of adjacent odometers according to the adjacent motion of the mechanical arm, obtain pose transformation quantities of adjacent camera modules according to the detected two-dimensional code pose, and further obtain the external parameter relation between the camera modules and the wheel-type odometers according to k relative transformation pairs. For example, fig. 6 shows a relative conversion relationship of a coordinate system, in which p and q represent a translation amount and a rotation amount, respectively;
and
respectively a translation component and a rotation component from the wheel type odometer to the external parameter of the camera module;
and
the translation component and the rotation component of the wheel type odometer from the k moment to the k +1 moment are respectively;
and
the translation component and the rotation component of the camera module from the k moment to the k +1 moment are respectively.
First, the motion at two adjacent time points can obtain the following equations (1) and (2):
then, to
Is decomposed into rotation amount around xy axis and rotation amount around z axis
Obtaining formula (3) according to formula (1):
further, formula (4) is obtained from the quaternion multiplication formula:
wherein the content of the first and second substances,
and
determined by the following equations (5) and (6):
then, the formula (4) is derived to obtain the formula (7):
wherein, q is due toyxIs a quaternion, so that formula (8) is satisfied:
further, q is obtained from two constraint conditionsyx。
Then, q of the external parameter is obtainedzConverting the component and the translation component to obtain the formula (9):
converting equation (9) into a matrix form (10):
then, for k relative motions, a 2k × 4 matrix G is constructed, resulting in equation (11):
further, equation (12) can be obtained from equation (10):
finally, the least square method is used for solving
And a course angle external parameter alpha; and optimizing the obtained external parameters by using a nonlinear optimization method to finally obtain the external parameters R and t of the wheel type odometer and the camera module.
Illustrative System
Referring to fig. 2 of the drawings, an external reference calibration system for calibrating external reference between a wheeled odometer mounted on a mobile robot and a camera module according to an embodiment of the present invention is illustrated. Specifically, as shown in fig. 2, the external reference calibration system 500 includes: a control module 510, configured to control movement of a mobile device according to a plurality of preset poses to respectively and correspondingly move the mobile robot, and control the camera module of the mobile robot to synchronously acquire a plurality of vision calibration images, where the mobile device is rigidly connected to the wheel-type odometer of the mobile robot; a coordinate transformation module 520, configured to obtain a plurality of odometer poses of the wheel type odometer relative to a reference coordinate system through coordinate transformation according to the plurality of preset poses; a visual positioning module 530, configured to obtain, according to the multiple visual calibration images, multiple visual poses of the camera module relative to the reference coordinate system through a visual positioning technique; and an external parameter calculating module 540, configured to calculate an external parameter between the wheel-type odometer and the camera module based on the odometer pose and the corresponding visual pose.
Illustrative electronic device
Next, an electronic apparatus according to an embodiment of the present invention is described with reference to fig. 3. As shown in fig. 3, the electronic device 90 includes one or more processors 91 and memory 92.
The processor 91 may be a Central Processing Unit (CPU) or other form of processing unit having data processing capabilities and/or instruction execution capabilities, and may control other components in the electronic device 90 to perform desired functions. In other words, the processor 91 comprises one or more physical devices configured to execute instructions. For example, the processor 91 may be configured to execute instructions that are part of: one or more applications, services, programs, routines, libraries, objects, components, data structures, or other logical constructs. Such instructions may be implemented to perform a task, implement a data type, transform the state of one or more components, implement a technical effect, or otherwise arrive at a desired result.
The processor 91 may include one or more processors configured to execute software instructions. Additionally or alternatively, the processor 91 may include one or more hardware or firmware logic machines configured to execute hardware or firmware instructions. The processors of the processor 91 may be single core or multicore, and the instructions executed thereon may be configured for serial, parallel, and/or distributed processing. The various components of the processor 91 may optionally be distributed over two or more separate devices, which may be remotely located and/or configured for coordinated processing. Aspects of the processor 91 may be virtualized and executed by remotely accessible networked computing devices configured in a cloud computing configuration.
The memory 92 may include one or more computer program products that may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory. The volatile memory may include, for example, Random Access Memory (RAM), cache memory (cache), and/or the like. The non-volatile memory may include, for example, Read Only Memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer readable storage medium and executed by the processor 11 to implement some or all of the steps of the above-described exemplary methods of the present invention described above, and/or other desired functions.
In other words, the memory 92 comprises one or more physical devices configured to hold machine-readable instructions executable by the processor 91 to implement the methods and processes described herein. In implementing these methods and processes, the state of the memory 92 may be transformed (e.g., to hold different data). The memory 92 may include removable and/or built-in devices. The memory 92 may include optical memory (e.g., CD, DVD, HD-DVD, blu-ray disc, etc.), semiconductor memory (e.g., RAM, EPROM, EEPROM, etc.), and/or magnetic memory (e.g., hard-disk drive, floppy-disk drive, tape drive, MRAM, etc.), among others. The memory 92 may include volatile, nonvolatile, dynamic, static, read/write, read-only, random-access, sequential-access, location-addressable, file-addressable, and/or content-addressable devices.
It is understood that the memory 92 comprises one or more physical devices. However, aspects of the instructions described herein may alternatively be propagated by a communication medium (e.g., an electromagnetic signal, an optical signal, etc.) that is not held by a physical device for a limited period of time. Aspects of the processor 91 and the memory 92 may be integrated together into one or more hardware logic components. These hardware logic components may include, for example, Field Programmable Gate Arrays (FPGAs), program and application specific integrated circuits (PASIC/ASIC), program and application specific standard products (PSSP/ASSP), system on a chip (SOC), and Complex Programmable Logic Devices (CPLDs).
In one example, as shown in FIG. 3, the electronic device 90 may also include an input device 93 and an output device 94, which may be interconnected via a bus system and/or other form of connection mechanism (not shown). The input device 93 may be, for example, a camera module or the like for capturing image data or video data. As another example, the input device 93 may include or interface with one or more user input devices such as a keyboard, mouse, touch screen, or game controller. In some embodiments, the input device 93 may include or interface with a selected Natural User Input (NUI) component. Such component parts may be integrated or peripheral and the transduction and/or processing of input actions may be processed on-board or off-board. Example NUI components may include a microphone for speech and/or voice recognition; the infrared, color, stereo display and/or depth camera module is used for machine vision and/or gesture recognition; a head tracker, eye tracker, accelerometer and/or gyroscope for motion detection and/or intent recognition; and an electric field sensing component for assessing brain activity and/or body movement; and/or any other suitable sensor.
The output device 94 may output various information including the classification result and the like to the outside. The output devices 94 may include, for example, a display, speakers, printer, and the like, as well as a communication network and its connected remote output devices.
Of course, the electronic device 90 may further comprise the communication means, wherein the communication means may be configured to communicatively couple the electronic device 90 with one or more other computer devices. The communication means may comprise wired and/or wireless communication devices compatible with one or more different communication protocols. As a non-limiting example, the communication subsystem may be configured for communication via a wireless telephone network or a wired or wireless local or wide area network. In some embodiments, the communications device may allow the electronic device 90 to send and/or receive messages to and/or from other devices via a network such as the internet.
It will be appreciated that the configurations and/or approaches described herein are exemplary in nature, and that these specific embodiments or examples are not to be considered in a limiting sense, because numerous variations are possible. The specific routines or methods described herein may represent one or more of any number of processing strategies. As such, various acts illustrated and/or described may be performed in the sequence illustrated and/or described, in other sequences, in parallel, or omitted. Also, the order of the above-described processes may be changed.
Of course, for simplicity, only some of the components of the electronic device 90 relevant to the present invention are shown in fig. 3, omitting components such as buses, input/output interfaces, and the like. In addition, the electronic device 90 may include any other suitable components, depending on the particular application.
According to another aspect of the present invention, an embodiment of the present invention further provides another electronic device for calibrating external parameters between the wheel-type odometer and the camera module mounted on the mobile robot. Illustratively, as shown in fig. 4 and 5, the electronic device includes an electronic device body 800 and at least one external reference calibration system 500 as described above. The electronic device body 800 comprises a mobile device 810, wherein the mobile device 810 is adapted to be rigidly connected with the wheeled odometer of the mobile robot. The external reference calibration system 500 is configured on the electronic device body 800, and the external reference calibration system 500 comprises the following components which are mutually connected in a communication way: the control module is used for controlling the movement of the mobile device 810 according to a plurality of preset poses to respectively and correspondingly move the mobile robot, and controlling the camera module of the mobile robot to synchronously acquire a plurality of vision calibration images; the coordinate conversion module is used for obtaining a plurality of odometer poses of the wheel type odometer relative to a reference coordinate system through coordinate conversion according to the plurality of preset poses; the visual positioning module is used for obtaining a plurality of visual poses of the camera module relative to the reference coordinate system through a visual positioning technology according to the plurality of visual calibration images; and the external parameter calculation module is used for calculating the external parameters between the wheel type odometer and the camera module based on the odometer pose and the corresponding visual pose.
In one example of the present invention, as shown in fig. 4, the mobile device 810 is a robot arm system, wherein the robot arm system comprises a robot arm 811, a base 812 and a connection station 813, wherein one end of the robot arm 811 is mounted to the base 812 and the other end of the robot arm 811 is connected to the connection station 813, wherein the connection station 813 is adapted to mount the mobile robot such that the connection station 813 is rigidly connected to the wheeled odometer.
In an example of the present invention, as shown in fig. 4 and 5, the electronic device body 800 further includes a calibration box 820, wherein a pattern 821 with visual feature points is disposed in the calibration box 820, so that when the mobile robot is moved in the calibration box 820, the camera module can capture the pattern 821 with visual feature points to obtain the visual calibration image with corner features.
It should also be noted that in the apparatus, devices and methods of the present invention, the components or steps may be broken down and/or re-combined. These decompositions and/or recombinations are to be regarded as equivalents of the present invention.
The previous description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other aspects without departing from the scope of the invention. Thus, the present invention is not intended to be limited to the aspects shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
It will be appreciated by persons skilled in the art that the embodiments of the invention described above and shown in the drawings are given by way of example only and are not limiting of the invention. The objects of the invention have been fully and effectively accomplished. The functional and structural principles of the present invention have been shown and described in the examples, and any variations or modifications of the embodiments of the present invention may be made without departing from the principles.
Claims (14)
1. An external reference calibration method for calibrating an external reference between a wheeled odometer mounted on a mobile robot and a camera module, the external reference calibration method comprising the steps of:
controlling the movement of a mobile device according to a plurality of preset poses to respectively and correspondingly move the mobile robot, and controlling the camera module of the mobile robot to synchronously acquire a plurality of vision calibration images, wherein the mobile device is rigidly connected with the wheel type odometer of the mobile robot;
obtaining a plurality of odometer poses of the wheel type odometer relative to a reference coordinate system through coordinate transformation according to the plurality of preset poses;
obtaining a plurality of visual poses of the camera module relative to the reference coordinate system through a visual positioning technology according to the plurality of visual calibration images; and
and calculating external parameters between the wheel type odometer and the camera module based on the odometer pose and the corresponding visual pose.
2. The external reference calibration method according to claim 1, wherein the mobile device is controlled to translate and rotate the mobile robot in a motion plane.
3. The extrinsic parameter calibration method according to claim 2, wherein the mobile device is a robotic arm system, wherein the robotic arm system includes a robotic arm, a base, and a connection station, wherein one end of the robotic arm is mounted to the base and the other end of the robotic arm is connected to the connection station, wherein the connection station is adapted to mount the mobile robot such that the connection station is rigidly connected to the wheeled odometer.
4. The extrinsic calibration method according to claim 3, wherein a coordinate system of the wheel odometer of the mobile robot coincides with a coordinate system of the connection stage of the robot arm system such that the preset pose is a pose of the wheel odometer with respect to the base.
5. The extrinsic calibration method according to any one of claims 2 to 4, wherein the camera module is controlled to synchronously capture a pattern with visual feature points for obtaining the visual calibration image with corner feature.
6. The extrinsic reference calibration method according to claim 5, wherein the pattern with visual feature points is a label pattern or a checkerboard pattern.
7. The external reference calibration method according to claim 6, further comprising the steps of:
providing a calibration box, wherein the calibration box is internally provided with the pattern with the visual characteristic points, so that when the mobile robot is moved in the calibration box, the camera module can shoot the pattern with the visual characteristic points to obtain the visual calibration image with the angular point characteristics.
8. The external reference calibration method as claimed in claim 7, wherein the calibration box has a cubic shape, and the top and bottom surfaces of the calibration box are parallel to the movement plane.
9. The referencing method of claim 8, wherein the pattern with the visual feature points is affixed to three side walls of the calibration box.
10. An external reference calibration system for calibrating external reference between a wheeled odometer mounted on a mobile robot and a camera module, wherein the external reference calibration system comprises:
the control module is used for controlling the movement of a mobile device according to a plurality of preset poses so as to respectively and correspondingly move the mobile robot and controlling the camera module of the mobile robot to synchronously acquire a plurality of vision calibration images, wherein the mobile device is rigidly connected with the wheel type odometer of the mobile robot;
the coordinate conversion module is used for obtaining a plurality of odometer poses of the wheel type odometer relative to a reference coordinate system through coordinate conversion according to the plurality of preset poses;
the visual positioning module is used for obtaining a plurality of visual poses of the camera module relative to the reference coordinate system through a visual positioning technology according to the plurality of visual calibration images; and
and the external parameter calculation module is used for calculating the external parameters between the wheel type odometer and the camera module based on the position and the corresponding visual position of the odometer.
11. An electronic device, comprising:
at least one processor configured to execute instructions; and
a memory communicatively coupled to the at least one processor, wherein the memory has at least one instruction, wherein the instruction is executed by the at least one processor to cause the at least one processor to perform some or all of the steps of an external reference calibration method for calibrating external reference between a wheeled odometer and a camera module mounted on a mobile robot, and the external reference calibration method comprises the steps of:
controlling the movement of a mobile device according to a plurality of preset poses to respectively and correspondingly move the mobile robot, and controlling the camera module of the mobile robot to synchronously acquire a plurality of vision calibration images, wherein the mobile device is rigidly connected with the wheel type odometer of the mobile robot;
obtaining a plurality of odometer poses of the wheel type odometer relative to a reference coordinate system through coordinate transformation according to the plurality of preset poses;
obtaining a plurality of visual poses of the camera module relative to the reference coordinate system through a visual positioning technology according to the plurality of visual calibration images; and
and calculating external parameters between the wheel type odometer and the camera module based on the odometer pose and the corresponding visual pose.
12. An electronic equipment for demarcating external parameters between a wheel type odometer and a camera module carried on a mobile robot, comprising:
an electronic device body, wherein the electronic device body comprises a mobile device, wherein the mobile device is suitable for being rigidly connected with the wheel-type odometer of the mobile robot; and
at least one external reference calibration system, wherein the external reference calibration system is configured on the electronic device body, and the external reference calibration system comprises:
the control module is used for controlling the movement of the mobile device according to a plurality of preset poses so as to respectively and correspondingly move the mobile robot and controlling the camera module of the mobile robot to synchronously acquire a plurality of vision calibration images;
the coordinate conversion module is used for obtaining a plurality of odometer poses of the wheel type odometer relative to a reference coordinate system through coordinate conversion according to the plurality of preset poses;
the visual positioning module is used for obtaining a plurality of visual poses of the camera module relative to the reference coordinate system through a visual positioning technology according to the plurality of visual calibration images; and
and the external parameter calculation module is used for calculating the external parameters between the wheel type odometer and the camera module based on the odometer pose and the corresponding visual pose.
13. The electronic device of claim 12, wherein the mobile device is a robotic arm system, wherein the robotic arm system comprises a robotic arm, a base, and a connection station, wherein one end of the robotic arm is mounted to the base and the other end of the robotic arm is connected to the connection station, wherein the connection station is adapted to mount the mobile robot such that the connection station is rigidly connected to the wheeled odometer.
14. The electronic device of claim 12 or 13, wherein the electronic device body further comprises a calibration box, wherein a pattern with visual feature points is disposed in the calibration box, so that when the mobile robot is moved in the calibration box, the camera module can capture the pattern with visual feature points to obtain the visual calibration image with corner features.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011146976.1A CN114494446A (en) | 2020-10-23 | 2020-10-23 | External parameter calibration method and system and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011146976.1A CN114494446A (en) | 2020-10-23 | 2020-10-23 | External parameter calibration method and system and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114494446A true CN114494446A (en) | 2022-05-13 |
Family
ID=81471032
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011146976.1A Pending CN114494446A (en) | 2020-10-23 | 2020-10-23 | External parameter calibration method and system and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114494446A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115908589A (en) * | 2023-02-23 | 2023-04-04 | 深圳佑驾创新科技有限公司 | Multi-sensor calibration system and method |
-
2020
- 2020-10-23 CN CN202011146976.1A patent/CN114494446A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115908589A (en) * | 2023-02-23 | 2023-04-04 | 深圳佑驾创新科技有限公司 | Multi-sensor calibration system and method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP6915009B2 (en) | Sensor calibration methods and equipment, computer equipment, media and vehicles | |
| CN109544629B (en) | Camera position and posture determining method and device and electronic equipment | |
| CN109910016B (en) | Visual acquisition calibration method, device and system based on multi-degree-of-freedom mechanical arm | |
| CN110880189B (en) | Combined calibration method and combined calibration device thereof and electronic equipment | |
| CN111415387B (en) | Camera pose determining method and device, electronic equipment and storage medium | |
| US20170307354A1 (en) | Position and orientation measuring apparatus, information processing apparatus and information processing method | |
| CN109214980A (en) | A kind of 3 d pose estimation method, device, equipment and computer storage medium | |
| Goldberg et al. | Stereo and IMU assisted visual odometry on an OMAP3530 for small robots | |
| CN116645649B (en) | Vehicle pose and size estimation method, device and storage medium | |
| Boren et al. | Exponential growth of ROS [ROS topics] | |
| CN113858217B (en) | Multi-robot interaction three-dimensional visual pose perception method and system | |
| US11729367B2 (en) | Wide viewing angle stereo camera apparatus and depth image processing method using the same | |
| CN113570715B (en) | Sensor fusion-based rotary laser real-time positioning modeling system and method | |
| US20240051122A1 (en) | Robot movement and online trajectory optimization | |
| WO2020024182A1 (en) | Parameter processing method and apparatus, camera device and aircraft | |
| CN114494446A (en) | External parameter calibration method and system and electronic equipment | |
| CN110728716B (en) | Calibration method and device and aircraft | |
| US20220114813A1 (en) | Detecting obstacle | |
| CN111290574B (en) | Method and device for controlling unmanned aerial vehicle by using gestures and readable storage medium | |
| KR102565444B1 (en) | Method and apparatus for identifying object | |
| CN110421563A (en) | A kind of industrial robot builds figure positioning system and robot | |
| KR102639524B1 (en) | Map merging method of electronic apparatus | |
| WO2023130465A1 (en) | Aerial vehicle, image processing method and apparatus, and movable platform | |
| CN116148883B (en) | SLAM method, device, terminal equipment and medium based on sparse depth image | |
| Nyman et al. | Multi-camera platform calibration using multi-linear constraints |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |