CN113643358A - External parameter calibration method, device, storage medium and system of camera - Google Patents
External parameter calibration method, device, storage medium and system of camera Download PDFInfo
- Publication number
- CN113643358A CN113643358A CN202110914889.4A CN202110914889A CN113643358A CN 113643358 A CN113643358 A CN 113643358A CN 202110914889 A CN202110914889 A CN 202110914889A CN 113643358 A CN113643358 A CN 113643358A
- Authority
- CN
- China
- Prior art keywords
- calibration
- coordinate system
- camera
- reference object
- world coordinate
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Length Measuring Devices By Optical Means (AREA)
Abstract
The application relates to a method, a device, a storage medium and a system for calibrating external parameters of a camera, belonging to the technical field of computers, wherein the method comprises the following steps: responding to an instruction for calibrating the camera, and controlling the camera to acquire an image of the current calibration environment to obtain an environment image; the ground clearance of the calibration reference object in the calibration environment is the same as the equipment height of the mobile equipment; determining the position of a standard reference object in the environment image under a camera coordinate system; determining an external reference calibration result of the camera based on the position of the calibration reference object in the camera coordinate system and the position of the calibration reference object in the world coordinate system; the problem that the world coordinate system of the camera is inconsistent with the world coordinate system of the mobile device, so that errors exist when environmental data acquired by the camera and the environmental data acquired by the mobile device are fused or compared can be solved; the position under the camera coordinate system can be ensured to be converted into the world coordinate system of the mobile device, and the accuracy of subsequent data fusion or data comparison can be improved.
Description
[ technical field ] A method for producing a semiconductor device
The application relates to a camera external reference calibration method, device, storage medium and system, and belongs to the technical field of computers.
[ background of the invention ]
The existing self-moving equipment often has the functions of environment identification, path planning, map construction and the like. At this time, the self-moving device usually collects environment data of a working area, and analyzes and processes the environment data to implement a corresponding function.
However, the range of collecting environmental data from mobile devices is limited. On the basis of the above, a camera for monitoring the self-moving device can be additionally arranged in the working area. At this time, the environmental data collected from the mobile device and the environmental data collected in the camera monitoring process are fused to analyze the fused environmental data, so that the accuracy of data analysis can be improved.
The camera needs to be externally calibrated in advance before use. The traditional external reference calibration mode comprises the following steps: marking the position under the world coordinate system on the ground; and calculating a homography matrix according to the pixel positions of the positions in the world coordinate systems in the camera coordinate system to obtain an external reference calibration result of the camera.
However, since the mobile device has a certain height and the computed external reference calibration result is obtained by converting the data in the camera coordinate system to the ground, the position of the mobile device in the world coordinate system may have an error from the actual position.
[ summary of the invention ]
The application provides a camera external reference calibration method, a camera external reference calibration device, a camera external reference calibration storage medium and a camera external reference calibration system, which can solve the problem that camera coordinates acquired by a camera are converted into a world coordinate system established based on the ground, the world coordinate system of a mobile device is established based on a plane where the device height of the mobile device is located, and errors exist when environment data acquired by the camera coordinates and the environment data acquired by the mobile device are fused or compared. The application provides the following technical scheme:
in a first aspect, a method for calibrating external parameters of a camera is provided, the method comprising:
responding to an instruction for calibrating the camera, and controlling the camera to acquire an image of the current calibration environment to obtain an environment image of the calibration environment; at least one calibration reference object is placed in the calibration environment, and the height of the calibration reference object from the ground is the same as the height of the mobile equipment;
determining the position of the calibration reference object in the environment image under a camera coordinate system;
determining an external reference calibration result of the camera based on the position of the calibration reference object in a camera coordinate system and the position of the calibration reference object in a world coordinate system; and the external reference calibration result is used for converting the position of the calibration reference object in the camera coordinate system into the position in the world coordinate system.
Optionally, the determining the position of the calibration reference in the environment image under the camera coordinate system includes:
displaying the environment image in a user interface;
in response to the selection operation of the image position in the environment image, the position of the calibration reference object indicated by the selection operation in a camera coordinate system is obtained.
Optionally, the number of positions in the camera coordinate system is at least two; the method further comprises the following steps:
determining the corresponding relation between the position of the calibration reference object under a camera coordinate system and the position of the calibration reference object under a world coordinate system according to the execution sequence of the selection operation; the execution order corresponds to the positions in the world coordinate system in a one-to-one mode.
Optionally, the determining the position of the calibration reference in the environment image under the camera coordinate system includes:
identifying a designated position of a calibration reference object in the environment image by using an image identification algorithm, and determining the position of the designated position in a camera coordinate system as the position of the calibration reference object in the camera coordinate system; accordingly, in the calibration environment, the position of the calibration reference object in the world coordinate system is the position of the designated position in the world coordinate system.
Optionally, the number of the positions of the calibration reference object in the camera coordinate system is 4, and each position of the calibration reference object in the camera coordinate system corresponds to one position in the world coordinate system;
the determining an external reference calibration result of the camera based on the position of the calibration reference object in the camera coordinate system and the position of the calibration reference object in the world coordinate system comprises:
and calculating a homography matrix of the camera by using each position of the calibration reference object in a camera coordinate system and each position of the calibration reference object in a world coordinate system to obtain the external reference calibration result.
In a second aspect, an external reference calibration apparatus for a camera is provided, the apparatus including a processor and a memory; the memory stores a program, which is loaded and executed by the processor to implement the camera external reference calibration method provided by the first aspect.
In a third aspect, a computer-readable storage medium is provided, in which a program is stored, and the program is used for implementing the external reference calibration method of the camera provided by the first aspect when being executed by a processor.
In a fourth aspect, an external reference calibration system for a camera is provided, the system comprising:
at least one calibration reference placed within a calibration environment, the calibration reference having a same ground clearance height as a device height from a mobile device;
the camera to be calibrated, wherein the acquisition range of the camera comprises the calibration reference object;
an external reference calibration apparatus of a camera communicatively connected to the camera to be calibrated, the external reference calibration apparatus of the camera comprising the external reference calibration apparatus of the camera of claim 6.
Optionally, the designated position of the calibration reference is located at a position under a world coordinate system marked in advance.
Optionally, the system further comprises a laser ground instrument; the calibration reference object is a cube;
for each pre-marked position under the world coordinate system, when the calibration reference object is placed at the position under the world coordinate system, the laser ground instrument emits two mutually intersected laser lines which are respectively parallel to the x axis and the y axis of the world coordinate system;
when the designated position of the calibration reference object is placed at a position under the world coordinate system, the edge of the calibration reference object is tangent to the two laser lines, and the designated position is overlapped with the position under the world coordinate system.
The beneficial effects of this application include at least: controlling a camera to acquire an image of a current calibration environment by responding to a camera calibration instruction to obtain an environment image of the calibration environment; at least one calibration reference object is placed in the calibration environment, and the height of the calibration reference object from the ground is the same as the height of the mobile equipment; determining the position of a standard reference object in the environment image under a camera coordinate system; determining an external reference calibration result of the camera based on the position of the calibration reference object in the camera coordinate system and the position of the calibration reference object in the world coordinate system; the external reference calibration result is used for converting the position of the calibration reference object in the camera coordinate system into the position in the world coordinate system; the problem that the camera coordinate acquired by the camera is converted into a world coordinate system established based on the ground, the world coordinate system of the mobile equipment is established based on the plane where the equipment height of the mobile equipment is located, and errors exist when environmental data acquired by the camera coordinate system and the environmental data acquired by the mobile equipment are fused or compared can be solved; because the calibration reference object with the same ground clearance height as the equipment height of the mobile equipment is arranged in the calibration environment, and the camera calibration is carried out according to the calibration reference object, the position under the camera coordinate system can be ensured to be converted to the position under the world coordinate system determined based on the equipment height of the mobile equipment, namely to be converted to the position under the world coordinate system of the mobile equipment, and the accuracy of subsequent data fusion or data comparison can be improved.
The foregoing description is only an overview of the technical solutions of the present application, and in order to make the technical solutions of the present application more clear and clear, and to implement the technical solutions according to the content of the description, the following detailed description is made with reference to the preferred embodiments of the present application and the accompanying drawings.
[ description of the drawings ]
Fig. 1 is a schematic structural diagram of an operating system of a self-moving device according to an embodiment of the present application;
FIG. 2 is a schematic structural diagram of an external reference calibration system of a camera according to an embodiment of the present application;
FIG. 3 is a flowchart of a method for calibrating an external parameter of a camera according to an embodiment of the present application;
FIG. 4 is a schematic diagram of an environmental image captured by a camera provided by an embodiment of the present application;
FIG. 5 is a block diagram of an external reference calibration apparatus of a camera according to an embodiment of the present application;
fig. 6 is a block diagram of an external reference calibration apparatus of a camera according to another embodiment of the present application.
[ detailed description ] embodiments
The following detailed description of embodiments of the present application will be made with reference to the accompanying drawings and examples. The following examples are intended to illustrate the present application but are not intended to limit the scope of the present application.
First, several terms referred to in the present application will be described.
Pixel coordinates are as follows: refers to the location of the pixel in the image. The vertex in the upper left corner of the pixel coordinate system is generally the origin, the u-axis is horizontally to the right, and the v-axis is vertically to the bottom. In the pixel coordinate system, the coordinates of each pixel are expressed in pixels. However, the representation method of the pixels cannot reflect the physical size of the object in the image, and therefore, the pixel coordinates need to be converted into image coordinates.
Image coordinate system: translating the origin of the pixel coordinate system to the center of the image to obtain the origin of the image coordinate system, wherein the x axis of the image coordinate system is parallel to the u axis of the pixel coordinate system, and the directions of the x axis and the u axis are the same; the y-axis of the image coordinate system is parallel to the v-axis of the pixel coordinate system and has the same direction.
Camera coordinate system: the optical axis of the camera is taken as the Z-axis, and the central position of the light ray in the optical system of the camera is the origin Oc (i.e., the center of the lens of the camera). The horizontal axis Xc of the camera coordinate system is parallel to the x-axis of the image coordinate system and the vertical axis Yc of the camera coordinate system is parallel to the y-axis of the image coordinate system. The distance OcOi between the origin of the camera coordinate system and the origin of the image coordinate system is f (i.e., the focal length).
World coordinate system: refers to a mapping relationship between the image and the real object. The origin of the world coordinate system is Ow, and the three-dimensional coordinate axes Xw, Yw, Zw of the world coordinate system are not necessarily parallel to other camera coordinate systems, image coordinate systems, or pixel coordinate systems, but have a certain angle and a certain translation. When the camera coordinate system is translated and rotated around the X, Y, Z axis according to certain parameters, the coordinates in the world coordinate system can be obtained. The manner of translation and rotation around the X, Y, Z axis is the external reference calibration result of the camera.
In other words, the camera's external parameters are used to indicate the rotational and translational transformation relationship of the camera with respect to the world coordinate system.
Conventional mobile devices are usually provided with sensors to collect environmental data of the current working area. The sensors include, but are not limited to, a laser radar sensor, an obstacle avoidance sensor, a distance measurement sensor, a vision sensor, etc., and the present embodiment does not list the types of the sensors. The mobile device can realize the functions of map construction, obstacle avoidance, path planning, target identification and the like of a working area based on environmental data acquired by a sensor.
However, the sensing range of the sensor installed on the mobile device is often small, and the posture of the collected data is fixed, so when the corresponding function is realized by using the environmental data collected by the sensor, the problem of poor effect of realizing the corresponding function may be caused by inaccurate analysis of the environmental data.
Based on this, referring to fig. 1, the present application proposes an operating system of a self-moving device, which includes a self-moving device 10 and a camera 20, wherein the self-moving device 10 and the camera 20 are both disposed in the same operating area. The installation height of the camera 20 is higher than the device height of the self-moving device 10, so that when a sensor is installed on the self-moving device 10, the camera 20 can acquire environmental data of a work area in a posture different from the acquisition posture of the sensor. Then, the environmental data collected by the camera 20 and the environmental data collected by the camera are fused, and the fused environmental data is used for analysis, so that the realization effect of the corresponding function is improved.
Before the camera 20 is used, external reference calibration is required, namely, the rotation and translation transformation relation of the camera 20 relative to the world coordinate system is determined. In the conventional camera calibration method, the planes of the Xw axis and the Yw axis of the world coordinate system are located on the plane of the ground, and at this time, the camera coordinates acquired by the camera 20 are converted into the world coordinate system established based on the ground. The world coordinate system of the mobile device 10 is established based on the plane of the device height of the mobile device 10, which causes an error problem in the fusion of the environment data when the environment data is fused.
Based on the above technical problem, the present embodiment provides an external reference calibration system and method for a camera, which are described below.
Fig. 2 is a schematic structural diagram of an external reference calibration system of a camera according to an embodiment of the present application, where as can be seen from fig. 2, the system at least includes: a calibration reference 30, a camera 20 to be calibrated and an external reference calibration device 40.
The calibration reference 30 is placed within the calibration environment. The number of calibration references 30 is at least one, and the ground clearance h of each calibration reference 30 is the same as the height of the mobile device.
In this embodiment, the device height is a height between a plane in which the Xw axis and the Yw axis are located in a world coordinate system established by the mobile device (i.e., the world coordinate system of the mobile device) and the ground. The Xw axis and the Yw axis are generally in the plane of the top surface of the mobile device. In practical implementation, the device height may also be an installation height of a sensor installed on the self-moving device, that is, distances between the Xw axis and the Yw axis in the world coordinate system of the self-moving device and the ground are the installation height of the sensor, and the embodiment does not limit a manner of determining the device height of the self-moving device. The world coordinate system in the present application refers to a world coordinate system of a mobile device, and the height between the plane where the Xw axis and the Yw axis of the world coordinate system are located and the ground is the device height.
The height of the calibration reference 30 from the ground refers to the distance between the plane of the calibration reference 30 farthest from the ground and the ground.
In one example, the designated location of the calibration reference 30 is located at a pre-marked location in the world coordinate system on the ground. At this time, the height from the ground of the specified position of the calibration reference 30 is the same as the height of the device from the mobile device.
Wherein the designated location is distinguished from other locations on the calibration reference 30. Such as: referring to fig. 2, the calibration reference 30 is a cube having a 2 x 2 grid on its top surface, with a 2 x 2 grid at the center of the grid being designated.
Optionally, there are one or at least two designated positions on each calibration reference 30.
When there is one designated position on each calibration reference 30, the number of calibration references 30 is the same as the number of designated positions. Such as: in fig. 2 there are 4 calibration references, each having a designated position which is the central position of a 2 x 2 checkerboard.
At least two designated positions on each calibration reference 30, the number of calibration references 30 is less than the number of designated positions. Such as: the calibration reference 30 is a cube, the upper surface of the cube has a 3 × 3 checkerboard, and the designated positions are 4 vertices of a middle cell of the 3 × 3 checkerboard, and the embodiment does not limit the implementation manner of the calibration reference 30 and the implementation manner of the designated positions.
In this embodiment, the specified position is taken as an intersection of the checkerboards for explanation, in actual implementation, the calibration reference 30 may further be provided with a light emitting device, and the specified position may also be a light emitting position of the light emitting device, and the implementation manner of the specified position is not limited in this embodiment.
Alternatively, the locations under the world coordinate system marked on the ground are marked using a laser ground and tape measure. Such as: two laser lines which are mutually intersected are emitted by a laser ground instrument and are respectively parallel to the x axis and the y axis of a world coordinate system, four points with world coordinates of (0,0), (0,1), (1,0) and (1,1) are drawn on the ground by using a measuring tape, and the position under the world coordinate system is obtained.
In order to ensure the accuracy of placing the designated position of the calibration reference 30 at a position under the world coordinate system, the external reference calibration system further comprises a laser ground instrument (not shown in the figure). Schematically, the calibration reference is a cube. At the moment, for each pre-marked position under the world coordinate system, when a calibration reference object is placed at the position under the world coordinate system, the laser ground instrument emits two mutually intersected laser lines which are respectively parallel to the x axis and the y axis of the world coordinate system; when the designated position of the calibration reference object is placed at a position under the world coordinate system, the edge of the calibration reference object is tangent to the two laser lines, and the designated position is overlapped with the position under the world coordinate system.
In other examples, the calibration reference 30 may be placed on other objects instead of directly on the ground, so as to ensure that the height of the calibration reference 30 from the ground is the same as the height of the mobile device, and the present embodiment does not limit the placement manner of the calibration reference 30.
The acquisition range of the camera 20 comprises a calibration reference 30. In this embodiment, the height of the camera 20 is higher than the height of the mobile device, so that the camera 20 can acquire the plane of the calibration reference object 30 farthest from the ground, thereby acquiring the image of the designated position.
The external reference calibration device 40 is in communication connection with the camera 20, the external reference calibration device 40 may be a computer, a tablet computer, a notebook computer, or a mobile phone, and the like, and the embodiment does not limit the type of the external reference calibration device.
In this embodiment, the external reference calibration device 40 is configured to respond to an instruction for calibrating the camera, and control the camera to perform image acquisition on the current calibration environment to obtain an environment image of the calibration environment; determining the position of a standard reference object in the environment image under a camera coordinate system; and determining an external reference calibration result of the camera based on the position of the calibration reference object in the camera coordinate system and the position of the calibration reference object in the world coordinate system.
At least one calibration reference object is placed in the calibration environment, and the height of the calibration reference object from the ground is the same as the height of the mobile equipment. The external reference calibration result is used for converting the position of the calibration reference object in the camera coordinate system into the position in the world coordinate system.
In this embodiment, by setting the calibration reference object having the same height from the ground as the device height of the mobile device in the calibration environment, it can be ensured that the position in the camera coordinate system is converted to the position in the world coordinate system determined based on the device height of the mobile device, that is, to the position in the world coordinate system of the mobile device, and the accuracy of subsequent data fusion or data comparison can be improved.
Based on the external reference calibration system of the camera shown in fig. 2, the external reference calibration method of the camera provided by the present application is described below. The present application is described by taking as an example that the external reference calibration method of the camera provided by each embodiment is used in the external reference calibration device 40 in the external reference calibration system shown in fig. 2.
Fig. 3 is a flowchart of an external reference calibration method for a camera according to an embodiment of the present application, where the method includes at least the following steps:
step 301, in response to an instruction for calibrating a camera, controlling the camera to acquire an image of a current calibration environment to obtain an environment image of the calibration environment; at least one calibration reference object is placed in the calibration environment, and the height of the calibration reference object from the ground is the same as the height of the self-moving equipment.
In one example, the external reference calibration device is provided with a calibration control, and when a calibration operation acting on the calibration control is received, an instruction for calibrating the camera is generated.
Optionally, the calibration control may be an entity button disposed on the calibration reference device, or the external calibration reference device has a touch display screen, and the calibration control is a virtual touch button displayed on the touch display screen.
The environment image collected by the camera comprises an image of a calibration reference. Such as: the image of the environment acquired by the camera is shown in fig. 4, which includes 4 images of the calibration references 30, each having a 2 × 2 checkerboard on the upper surface.
Step 302, determining the position of the calibration reference object in the environment image under the camera coordinate system.
In one example, determining a position of a calibration reference in an environmental image in a camera coordinate system includes: displaying an environment image in a user interface; in response to the selection operation of the image position in the environment image, the position of the calibration reference object indicated by the selection operation in the camera coordinate system is acquired.
When the number of the positions under the camera coordinate system is at least two, determining the corresponding relation between the position of the calibration reference object under the camera coordinate system and the position of the calibration reference object under the world coordinate system; the execution order corresponds to the positions in the world coordinate system one by one.
Such as: referring to fig. 4, the first execution order corresponds to a position (0,0) in the world coordinate system, the second execution order corresponds to a position (0,1) in the world coordinate system, the third execution order corresponds to a position (1,0) in the world coordinate system, and the fourth execution order corresponds to a position (1,1) in the world coordinate system. At this time, the position 41 in the camera coordinate system instructed by the selection operation executed for the first time is a position in the camera coordinate system corresponding to the position (0,0) in the world coordinate system; the position 42 in the camera coordinate system instructed by the selection operation performed for the second time is a position in the camera coordinate system corresponding to the position (0,1) in the world coordinate system; the position 43 in the camera coordinate system instructed by the selection operation performed for the third time is a position in the camera coordinate system corresponding to the position (1,0) in the world coordinate system; the position 44 in the camera coordinate system indicated by the selection operation performed for the fourth time is the position in the camera coordinate system corresponding to the position (1,1) in the world coordinate system.
In practical implementation, the one-to-one correspondence between the execution order and the position in the world coordinate system may be other ways, and this embodiment is not listed here.
Optionally, the correspondence between the position in the camera coordinate system and the position in the world coordinate system may also be manually adjusted by the user, such as: displaying a position in a world coordinate system and a position in a camera coordinate system of a selection operation instruction in a user interface; and then, receiving a connection operation between the position in the world coordinate system and the position in the camera coordinate system to obtain a corresponding relationship between the position in the camera coordinate system and the position in the world coordinate system.
In another example, determining a position of a calibration reference in an environmental image in a camera coordinate system includes: identifying the designated position of the calibration reference object in the environment image by using an image identification algorithm, and determining the position of the designated position in the camera coordinate system as the position of the calibration reference object in the camera coordinate system; accordingly, in the calibration environment, the position of the calibration reference object in the world coordinate system is the position of the specified position in the world coordinate system.
At this time, the designated position is distinguished from other positions of the calibration reference, and the distinction can be recognized by the image recognition algorithm.
Optionally, the image recognition algorithm includes, but is not limited to: scale-invariant feature transform (SIFT) algorithm, or Speeded Up Robust Features (SURF), etc., and the implementation of the image recognition algorithm is not limited in this embodiment.
Step 303, determining an external reference calibration result of the camera based on the position of the calibration reference object in the camera coordinate system and the position of the calibration reference object in the world coordinate system; the external reference calibration result is used for converting the position of the calibration reference object in the camera coordinate system into the position in the world coordinate system.
In one example, the outlier calibration result is a homography matrix of the camera. Accordingly, the number of positions in the camera coordinate system is 4, and each position of the calibration reference object in the camera coordinate system corresponds to a position in the world coordinate system. At this time, determining an external reference calibration result of the camera based on the position of the calibration reference object in the camera coordinate system and the position of the calibration reference object in the world coordinate system includes: and calculating a homography matrix of the camera by using each position of the calibration reference object in the camera coordinate system and each position of the calibration reference object in the world coordinate system to obtain an external reference calibration result.
Homography (Homography) transformation is used to describe the position mapping relationship of an object between the world coordinate system and the pixel coordinate system, and the corresponding transformation matrix is called Homography matrix. The homography matrix may be defined as:
wherein M is an internal reference matrix of the camera, the internal reference matrix is known, and the position under the camera coordinate system can be converted into a pixel coordinate system by using the internal reference matrix, so that a final homography matrix can be obtained by using the world coordinate system and the pixel coordinate system.
In summary, in the external reference calibration method for the camera provided in this embodiment, the camera is controlled to perform image acquisition on the current calibration environment by responding to the instruction for calibrating the camera, so as to obtain the environment image of the calibration environment; at least one calibration reference object is placed in the calibration environment, and the height of the calibration reference object from the ground is the same as the height of the mobile equipment; determining the position of a standard reference object in the environment image under a camera coordinate system; determining an external reference calibration result of the camera based on the position of the calibration reference object in the camera coordinate system and the position of the calibration reference object in the world coordinate system; the external reference calibration result is used for converting the position of the calibration reference object in the camera coordinate system into the position in the world coordinate system; the problem that the camera coordinate acquired by the camera is converted into a world coordinate system established based on the ground, the world coordinate system of the mobile equipment is established based on the plane where the equipment height of the mobile equipment is located, and errors exist when environmental data acquired by the camera coordinate system and the environmental data acquired by the mobile equipment are fused or compared can be solved; because the calibration reference object with the same ground clearance height as the equipment height of the mobile equipment is arranged in the calibration environment, and the camera calibration is carried out according to the calibration reference object, the position under the camera coordinate system can be ensured to be converted to the position under the world coordinate system determined based on the equipment height of the mobile equipment, namely to be converted to the position under the world coordinate system of the mobile equipment, and the accuracy of subsequent data fusion or data comparison can be improved.
Fig. 5 is a block diagram of an external reference calibration apparatus of a camera according to an embodiment of the present application. The device is used in an external reference calibration device 40 in an external reference calibration system shown in FIG. 2, and the device at least comprises the following modules: an image acquisition module 510, a position determination module 520, and an external reference calibration module 530.
An image acquisition module 510, configured to respond to an instruction for calibrating the camera, and control the camera to perform image acquisition on a current calibration environment to obtain an environment image of the calibration environment; at least one calibration reference object is placed in the calibration environment, and the height of the calibration reference object from the ground is the same as the height of the mobile equipment;
a position determining module 520, configured to determine a position of the calibration reference in the environment image in a camera coordinate system;
an external reference calibration module 530, configured to determine an external reference calibration result of the camera based on the position of the calibration reference object in the camera coordinate system and the position of the calibration reference object in the world coordinate system; and the external reference calibration result is used for converting the position of the calibration reference object in the camera coordinate system into the position in the world coordinate system.
For relevant details reference is made to the above-described method embodiments.
It should be noted that: in the external reference calibration apparatus for a camera provided in the above embodiment, when performing external reference calibration of the camera, only the division of the functional modules is illustrated, and in practical applications, the function distribution may be completed by different functional modules according to needs, that is, the internal structure of the external reference calibration apparatus for a camera is divided into different functional modules, so as to complete all or part of the functions described above. In addition, the external reference calibration device of the camera provided by the above embodiment and the external reference calibration method of the camera belong to the same concept, and specific implementation processes thereof are detailed in the method embodiment and are not described herein again.
Fig. 6 is a block diagram of an external reference calibration apparatus of a camera according to an embodiment of the present application. The apparatus includes an external reference calibration apparatus 40 of the external reference calibration system shown in FIG. 2. The apparatus comprises at least a processor 601 and a memory 602.
Processor 601 may include one or more processing cores such as: 4 core processors, 8 core processors, etc. The processor 601 may be implemented in at least one hardware form of a DSP (Digital Signal Processing), an FPGA (Field-Programmable Gate Array), and a PLA (Programmable Logic Array). The processor 601 may also include a main processor and a coprocessor, where the main processor is a processor for Processing data in an awake state, and is also called a Central Processing Unit (CPU); a coprocessor is a low power processor for processing data in a standby state. In some embodiments, the processor 601 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content required to be displayed on the display screen. In some embodiments, processor 601 may also include an AI (Artificial Intelligence) processor for processing computational operations related to machine learning.
The memory 602 may include one or more computer-readable storage media, which may be non-transitory. The memory 602 may also include high-speed random access memory, as well as non-volatile memory, such as one or more magnetic disk storage devices, flash memory storage devices. In some embodiments, a non-transitory computer readable storage medium in the memory 602 is used to store at least one instruction for execution by the processor 601 to implement the camera calibration method provided by the method embodiments herein.
In some embodiments, the external reference calibration device may further include: a peripheral interface and at least one peripheral. The processor 601, memory 602 and peripheral interface may be connected by a bus or signal lines. Each peripheral may be connected to the peripheral interface via a bus, signal line, or circuit board. Illustratively, peripheral devices include, but are not limited to: radio frequency circuit, touch display screen, audio circuit, power supply, etc.
Of course, the external reference calibration apparatus may also include fewer or more components, which is not limited in this embodiment.
Optionally, the present application further provides a computer-readable storage medium, in which a program is stored, and the program is loaded and executed by a processor to implement the external reference calibration method of the camera according to the above method embodiment.
Optionally, the present application further provides a computer product, which includes a computer-readable storage medium, where a program is stored in the computer-readable storage medium, and the program is loaded and executed by a processor to implement the camera external reference calibration method according to the foregoing method embodiment.
The technical features of the embodiments described above may be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the embodiments described above are not described, but should be considered as being within the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (10)
1. An external reference calibration method of a camera, the method comprising:
responding to an instruction for calibrating the camera, and controlling the camera to acquire an image of the current calibration environment to obtain an environment image of the calibration environment; at least one calibration reference object is placed in the calibration environment, and the height of the calibration reference object from the ground is the same as the height of the mobile equipment;
determining the position of the calibration reference object in the environment image under a camera coordinate system;
determining an external reference calibration result of the camera based on the position of the calibration reference object in a camera coordinate system and the position of the calibration reference object in a world coordinate system; and the external reference calibration result is used for converting the position of the calibration reference object in the camera coordinate system into the position in the world coordinate system.
2. The method of claim 1, wherein the determining the position of the calibration reference in the environment image under a camera coordinate system comprises:
displaying the environment image in a user interface;
in response to the selection operation of the image position in the environment image, the position of the calibration reference object indicated by the selection operation in a camera coordinate system is obtained.
3. The method of claim 2, wherein the number of positions in the camera coordinate system is at least two; the method further comprises the following steps:
determining the corresponding relation between the position of the calibration reference object under a camera coordinate system and the position of the calibration reference object under a world coordinate system according to the execution sequence of the selection operation; the execution order corresponds to the positions in the world coordinate system in a one-to-one mode.
4. The method of claim 1, wherein the determining the position of the calibration reference in the environment image under a camera coordinate system comprises:
identifying a designated position of a calibration reference object in the environment image by using an image identification algorithm, and determining the position of the designated position in a camera coordinate system as the position of the calibration reference object in the camera coordinate system; accordingly, in the calibration environment, the position of the calibration reference object in the world coordinate system is the position of the designated position in the world coordinate system.
5. The method according to any one of claims 1 to 3, wherein the number of the positions of the calibration reference in the camera coordinate system is 4, and each position of the calibration reference in the camera coordinate system corresponds to a position in the world coordinate system;
the determining an external reference calibration result of the camera based on the position of the calibration reference object in the camera coordinate system and the position of the calibration reference object in the world coordinate system comprises:
and calculating a homography matrix of the camera by using each position of the calibration reference object in a camera coordinate system and each position of the calibration reference object in a world coordinate system to obtain the external reference calibration result.
6. An external reference calibration device of a camera is characterized by comprising a processor and a memory; the memory stores a program that is loaded and executed by the processor to implement the camera external reference calibration method according to any one of claims 1 to 5.
7. A computer-readable storage medium, characterized in that the storage medium has stored therein a program which, when being executed by a processor, is adapted to implement the method for extrinsic calibration of a camera according to any one of claims 1 to 5.
8. An external reference calibration system for a camera, the system comprising:
at least one calibration reference placed within a calibration environment, the calibration reference having a same ground clearance height as a device height from a mobile device;
the camera to be calibrated, wherein the acquisition range of the camera comprises the calibration reference object;
an external reference calibration apparatus of a camera communicatively connected to the camera to be calibrated, the external reference calibration apparatus of the camera comprising the external reference calibration apparatus of the camera of claim 6.
9. The system of claim 8, wherein the designated location of the calibration reference is at a pre-marked location in the world coordinate system.
10. The system of claim 9, further comprising a laser ground instrument; the calibration reference object is a cube;
for each pre-marked position under the world coordinate system, when the calibration reference object is placed at the position under the world coordinate system, the laser ground instrument emits two mutually intersected laser lines which are respectively parallel to the x axis and the y axis of the world coordinate system;
when the designated position of the calibration reference object is placed at a position under the world coordinate system, the edge of the calibration reference object is tangent to the two laser lines, and the designated position is overlapped with the position under the world coordinate system.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110914889.4A CN113643358B (en) | 2021-08-10 | 2021-08-10 | External parameter calibration method, device, storage medium and system of camera |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110914889.4A CN113643358B (en) | 2021-08-10 | 2021-08-10 | External parameter calibration method, device, storage medium and system of camera |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113643358A true CN113643358A (en) | 2021-11-12 |
| CN113643358B CN113643358B (en) | 2023-07-07 |
Family
ID=78420757
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110914889.4A Active CN113643358B (en) | 2021-08-10 | 2021-08-10 | External parameter calibration method, device, storage medium and system of camera |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113643358B (en) |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150341629A1 (en) * | 2014-05-21 | 2015-11-26 | GM Global Technology Operations LLC | Automatic calibration of extrinsic and intrinsic camera parameters for surround-view camera system |
| CN105678783A (en) * | 2016-01-25 | 2016-06-15 | 西安科技大学 | Data fusion calibration method of catadioptric panorama camera and laser radar |
| CN109102546A (en) * | 2018-07-16 | 2018-12-28 | 珠海市微半导体有限公司 | A kind of scaling method of the robot camera based on more scaling boards |
| CN110322513A (en) * | 2018-03-30 | 2019-10-11 | 杭州海康威视数字技术股份有限公司 | A kind of Camera extrinsic scaling method, device and electronic equipment |
| CN111360810A (en) * | 2018-12-25 | 2020-07-03 | 深圳市优必选科技有限公司 | External parameter calibration method and device for robot sensor, robot and storage medium |
| CN112414444A (en) * | 2019-08-22 | 2021-02-26 | 阿里巴巴集团控股有限公司 | Data calibration method, computer equipment and storage medium |
| CN112509125A (en) * | 2020-12-14 | 2021-03-16 | 广州迦恩科技有限公司 | Three-dimensional reconstruction method based on artificial markers and stereoscopic vision |
| CN112882053A (en) * | 2021-01-21 | 2021-06-01 | 清华大学深圳国际研究生院 | Method for actively calibrating external parameters of laser radar and encoder |
| CN112949478A (en) * | 2021-03-01 | 2021-06-11 | 浙江国自机器人技术股份有限公司 | Target detection method based on holder camera |
| CN113052919A (en) * | 2021-04-23 | 2021-06-29 | 深圳市商汤科技有限公司 | Calibration method and device of visual sensor, electronic equipment and storage medium |
-
2021
- 2021-08-10 CN CN202110914889.4A patent/CN113643358B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150341629A1 (en) * | 2014-05-21 | 2015-11-26 | GM Global Technology Operations LLC | Automatic calibration of extrinsic and intrinsic camera parameters for surround-view camera system |
| CN105678783A (en) * | 2016-01-25 | 2016-06-15 | 西安科技大学 | Data fusion calibration method of catadioptric panorama camera and laser radar |
| CN110322513A (en) * | 2018-03-30 | 2019-10-11 | 杭州海康威视数字技术股份有限公司 | A kind of Camera extrinsic scaling method, device and electronic equipment |
| CN109102546A (en) * | 2018-07-16 | 2018-12-28 | 珠海市微半导体有限公司 | A kind of scaling method of the robot camera based on more scaling boards |
| CN111360810A (en) * | 2018-12-25 | 2020-07-03 | 深圳市优必选科技有限公司 | External parameter calibration method and device for robot sensor, robot and storage medium |
| CN112414444A (en) * | 2019-08-22 | 2021-02-26 | 阿里巴巴集团控股有限公司 | Data calibration method, computer equipment and storage medium |
| CN112509125A (en) * | 2020-12-14 | 2021-03-16 | 广州迦恩科技有限公司 | Three-dimensional reconstruction method based on artificial markers and stereoscopic vision |
| CN112882053A (en) * | 2021-01-21 | 2021-06-01 | 清华大学深圳国际研究生院 | Method for actively calibrating external parameters of laser radar and encoder |
| CN112949478A (en) * | 2021-03-01 | 2021-06-11 | 浙江国自机器人技术股份有限公司 | Target detection method based on holder camera |
| CN113052919A (en) * | 2021-04-23 | 2021-06-29 | 深圳市商汤科技有限公司 | Calibration method and device of visual sensor, electronic equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113643358B (en) | 2023-07-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109918977B (en) | Method, device and equipment for determining idle parking space | |
| CN112654886B (en) | External parameter calibration method, device, equipment and storage medium | |
| CN110568447B (en) | Visual positioning method, device and computer readable medium | |
| CN107223269B (en) | Three-dimensional scene positioning method and device | |
| US20230244227A1 (en) | Data processing method, control apparatus and storage medium | |
| CN113763478B (en) | Unmanned vehicle camera calibration method, device, equipment, storage medium and system | |
| CN113696188B (en) | Hand-eye calibration data acquisition method and device, electronic equipment and storage medium | |
| CN112489099A (en) | Point cloud registration method and device, storage medium and electronic equipment | |
| CN114004882A (en) | Two-dimensional map generation method and device, terminal equipment and storage medium | |
| WO2016187752A1 (en) | Method and device for measuring antenna attitude | |
| CN114898055A (en) | Virtual reality-based remote intelligent inspection method and system for extra-high voltage converter station | |
| CN113643358B (en) | External parameter calibration method, device, storage medium and system of camera | |
| CN112505652A (en) | Target detection method, device and storage medium | |
| CN116912417A (en) | Texture mapping method, device, equipment and storage medium based on three-dimensional reconstruction of human face | |
| CN116386373A (en) | Vehicle positioning method and device, storage medium and electronic equipment | |
| CN116203976A (en) | Indoor inspection method and device for transformer substation, unmanned aerial vehicle and storage medium | |
| CN115063489A (en) | External parameter calibration method, device, equipment and storage medium | |
| CN113446940A (en) | Point cloud scanning method, device and equipment | |
| CN113639639A (en) | Data processing method and device for position data and storage medium | |
| CN113177988A (en) | Calibration method, device, equipment and storage medium for dome camera and laser | |
| CN113177975A (en) | Depth calculation method and three-dimensional modeling method based on dome camera and laser radar | |
| CN112652056A (en) | 3D information display method and device | |
| CN111260781A (en) | Method and device for generating image information and electronic equipment | |
| WO2023103884A1 (en) | Object model establishment method and apparatus, and electronic device and storage medium | |
| CN116736272A (en) | Method, device, equipment and medium for determining joint calibration parameters |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |