WO2022077239A1

WO2022077239A1 - Camera parameter calibration method, image processing method and apparatus, and storage medium

Info

Publication number: WO2022077239A1
Application number: PCT/CN2020/120707
Authority: WO
Inventors: 张明磊; 梁家斌
Original assignee: 深圳市大疆创新科技有限公司
Priority date: 2020-10-13
Filing date: 2020-10-13
Publication date: 2022-04-21
Also published as: CN113490966A; CN113490966B

Abstract

A camera parameter calibration method, an image processing method and apparatus, and a storage medium. The method comprises: acquiring multiple images captured by a camera during the process of an unmanned aerial vehicle moving along a preset route (S101); calculating calibration parameters of the camera by means of the multiple images, the calibration parameters of the camera comprising an intrinsic parameter and/or a distortion parameter of the camera (S102); and transmitting the calibration parameters of the camera to the camera such that the camera associatively stores the calibration parameters of the camera and a lens identifier of the camera (S103).

Description

Camera parameter calibration method, image processing method, device and storage medium

technical field

The present application relates to the technical field of camera calibration, and in particular, to a camera parameter calibration method, an image processing method, a device and a storage medium.

Background technique

In the related fields of stereo vision and 3D reconstruction, it is necessary to perform mathematical modeling on the process of camera imaging to establish a mapping relationship between 3D points in space and 2D points on an image. Camera calibration can be used to calculate the parameters required in the mathematical modeling process, including the camera's intrinsic parameters, extrinsic parameters or distortion parameters.

The existing camera calibration usually adopts the pre-calibration method, that is, in a specific calibration environment (usually arranged indoors, with a calibration board set at a fixed position), a camera is used to shoot a certain number of images in the calibration environment, and these images are used for calibration. However, the calibration environment is very different from the application environment, and the calibration parameters calculated by using the images captured in the calibration environment will have errors; in addition, if the lens of the camera is an interchangeable lens, the replacement of the lens will cause a large change in the calibration parameters. Use the same calibration parameters to apply to all lenses, and re-calibration of new lenses needs to be returned to the factory.

SUMMARY OF THE INVENTION

Based on this, the present application provides a camera parameter calibration method, an image processing method, a device and a storage medium.

In a first aspect, the present application provides a method for calibrating camera parameters, the camera is mounted on a drone, the camera includes a body and a lens, and the method includes:

acquiring a plurality of images collected by the camera during the movement of the drone along the preset route;

The calibration parameters of the camera are calculated by using the plurality of images, and the calibration parameters of the camera include internal parameters and/or distortion parameters of the camera;

The calibration parameters of the camera are transmitted to the camera, so that the camera stores the calibration parameters of the camera in association with the lens identification of the camera.

In a second aspect, the present application provides an image processing method for processing an image collected by a camera, where the camera includes a body and a lens, and the method includes:

According to the lens identification of the camera, determine the calibration parameter of the camera corresponding to the lens identification, and the calibration parameter of the camera includes the internal parameter and/or the distortion parameter of the camera;

acquiring images captured by the camera;

The calibration parameters of the camera are stored in association with the images collected by the camera.

In a third aspect, the present application provides a camera parameter calibration device, the camera is mounted on a drone, the camera includes a body and a lens, and the device includes: a memory and a processor;

the memory is used to store computer programs;

The processor is configured to execute the computer program and implement the following steps when executing the computer program:

The calibration parameters of the camera are transmitted to the camera, so that the camera stores the calibration parameters of the camera in association with the lens identifier of the camera.

In a fourth aspect, the present application provides an image processing device for processing images collected by a camera, the camera includes a body and a lens, and the device includes: a memory and a processor;

the memory is used to store computer programs;

acquiring images captured by the camera;

In a fifth aspect, the present application provides a computer-readable storage medium, where the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor enables the processor to calibrate the camera parameters as described above method.

In a sixth aspect, the present application provides a computer-readable storage medium, where the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor implements the image processing method as described above.

The embodiments of the present application provide a method for calibrating camera parameters, an image processing method, a device, and a storage medium. On the one hand, to obtain multiple images collected by the camera during the movement of the drone along a preset route, the camera is mounted on the unmanned aerial vehicle. camera, the camera includes a body and a lens; the calibration parameters of the camera are calculated by using the plurality of images, and the calibration parameters of the camera include the internal parameters and/or distortion parameters of the camera; the calibration parameters of the camera are It is transmitted to the camera, so that the camera stores the calibration parameters of the camera in association with the lens identifier of the camera. Since the multiple images used to calculate the calibration parameters of the camera are collected by the camera mounted on the UAV during the movement of the UAV along the preset route, the scene of image collection used for calibration is closer to the actual application Therefore, calculating the calibration parameters of the camera by using multiple images collected in the calibration scene that is closer to the actual application scenario will help to improve the accuracy of the calibration parameters and help ensure the relative accuracy and stability of the calibration parameters; The calibration parameters of the camera are transmitted to the camera, so that the camera stores the calibration parameters of the camera in association with the lens identification of the camera, which is beneficial to realize the matching between the lens and the calibration parameters in the image acquisition system with interchangeable lenses. In the case of replacing a new lens, the user can calibrate by himself, and establish the relationship between the new lens identification and the calibration parameters, without returning to the factory for calibration. On the other hand, according to the lens identification of the camera, the calibration parameters of the camera corresponding to the lens identification are determined, and the calibration parameters of the camera include internal parameters and/or distortion parameters of the camera; The image collected; the calibration parameters of the camera are stored in association with the image collected by the camera. Since the lens is marked with the corresponding calibration parameters of the camera, after the image captured by the camera is acquired, the calibration parameters of the camera are stored in association with the image captured by the camera. In this way, the lens can be The collected images can correspond to the calibration parameters of the camera corresponding to the lens, which can avoid the separation of the calibration parameters from the images collected by the lens. When using these images, the calibration parameters of the corresponding camera can be obtained, so that a more accurate space can be established The mapping relationship between 3D points in and 2D points on the image.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not limiting of the present application.

Description of drawings

In order to explain the technical solutions of the embodiments of the present application more clearly, the following briefly introduces the accompanying drawings used in the description of the embodiments. For those of ordinary skill, other drawings can also be obtained from these drawings without any creative effort.

1 is a schematic flowchart of an embodiment of a method for calibrating camera parameters of the present application;

2 is a schematic flowchart of another embodiment of a method for calibrating camera parameters of the present application;

3 is a schematic flowchart of an embodiment of an image processing method of the present application;

4 is a schematic structural diagram of an embodiment of an apparatus for calibrating camera parameters of the present application;

FIG. 5 is a schematic structural diagram of an embodiment of an image processing apparatus of the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

The flowcharts shown in the figures are for illustration only, and do not necessarily include all contents and operations/steps, nor do they have to be performed in the order described. For example, some operations/steps can also be decomposed, combined or partially combined, so the actual execution order may be changed according to the actual situation.

In the related fields of stereo vision and 3D reconstruction, it is necessary to perform mathematical modeling on the process of camera imaging to establish a mapping relationship between 3D points in space and 2D points on an image. Camera calibration can be used to calculate the parameters required in the mathematical modeling process. Existing camera calibration usually adopts a pre-calibration method, that is, in a specific calibration environment, a camera is used to capture a certain number of images of the calibration environment, and these images are used for calibration. However, the calibration environment is very different from the application environment. The camera is usually relatively far away from the object to be photographed when the drone is shooting, while the camera is usually relatively close to the calibration board during indoor pre-calibration, so the calculated calibration parameters will have errors; if The lens of the camera is an interchangeable lens, and the replacement of the lens will cause a large change in the calibration parameters, so that the unified calibration parameters cannot be used for all lenses, and the re-calibration needs to be returned to the factory.

Some embodiments of the present application will be described in detail below with reference to the accompanying drawings. The embodiments described below and features in the embodiments may be combined with each other without conflict.

Referring to FIG. 1 , FIG. 1 is a schematic flowchart of an embodiment of a method for calibrating camera parameters of the present application. The camera is mounted on a drone, and the camera includes a body and a lens. The camera is mounted on the drone. When the drone moves, the camera moves with it. The camera includes a body and a lens. The lens and the body can be fixed or detachable.

The method includes: step S101, step S102 and step S103.

Step S101: Acquire a plurality of images collected by the camera during the movement of the UAV along a preset route.

Step S102: Calculate and obtain calibration parameters of the camera by using the plurality of images, where the calibration parameters of the camera include internal parameters and/or distortion parameters of the camera.

In this embodiment, when the camera captures images, the camera is mounted on the drone, and the drone moves along a preset route, and the camera captures multiple images in the process of following the movement of the drone.

The preset route may be a pre-planned route that facilitates the acquisition of multiple images that can improve the stability and accuracy of the calibration parameters. For example, the drone moves along a preset route, the orientation of the photosensitive elements when the camera captures images is not exactly the same, the orientation of the main optical axis of the camera is not exactly the same, the cameras are not at the same height when capturing images, and so on.

Usually, through camera calibration, we can know: (1) External parameters (also called external parameter matrix), the external parameters can explain how points in the real world (such as points in the world coordinate system) are rotated and translated, and then fall into On a point in another real world (such as a point in the camera coordinate system); (2) internal parameters (also known as internal parameter matrix), the internal parameters can indicate that after the above (1), the point that falls to another real world ( For example, the point of the camera coordinate system), how does it continue to pass through the lens of the camera and become pixel points through pinhole imaging and electronic conversion; (3) Distortion parameters (also called distortion matrix), the distortion parameters can explain why the above one The pixel point does not fall in the position where the theoretical calculation should fall, and it also produces a certain offset and deformation. Among them, the intrinsic parameters and distortion parameters are parameters related to the characteristics of the camera itself. In this embodiment, the calibration parameters of the camera calculated by using the plurality of images mainly include: internal parameters of the camera, or distortion parameters of the camera, or internal parameters and distortion parameters of the camera.

Internal parameters include the following:

Among them, f _x , f _y are the focal lengths of the camera, x ₀ , y ₀ are the coordinates of the principal point of the image; s is the coordinate axis tilt parameter, ideally 0.

In geometric optics and cathode ray tube displays, distortion is a shift from a straight line projection. To put it simply, theoretically a straight line projected onto the picture remains a straight line, but in fact a straight line projected onto the picture cannot remain a straight line, which is an optical distortion. Distortion can generally be divided into two categories, including radial distortion and tangential distortion. Real lenses generally have radial distortion and tangential distortion. The distortion parameters include radial distortion coefficients and tangential distortion coefficients, wherein k ₁ , k ₂ , and k ₃ are radial distortion coefficients, and p ₁ and p ₂ are tangential distortion coefficients. Radial distortion occurs in the process of converting the camera coordinate system to the physical coordinate system of the image, which is mainly caused by the shape of the lens, while tangential distortion occurs in the production process or assembly process of the camera, which is mainly due to the difference between the photoreceptor plane and the lens. caused by parallelism.

Compared with capturing images in a specific calibration environment when calibrating cameras in the prior art, the scene of image capture used for calibration in this embodiment is closer to the actual application scenario, and the camera on the UAV can be far away from the shooting object, which is helpful. Because of covering the various positions of the sensor, the multiple images collected by the camera during the movement of the UAV along the preset route are obtained, and the multiple images collected in the calibration scene which is closer to the actual application scene are used to calculate The calibration parameters of the camera help to improve the accuracy of the calibration parameters and help to ensure the relative accuracy and stability of the calibration parameters.

Step S103: Transmit the calibration parameters of the camera to the camera, so that the camera stores the calibration parameters of the camera in association with the lens identifier of the camera.

The lens identifier can be an identifier that can uniquely identify the lens, for example, the lens model, production time, date of manufacture, etc., or a combination thereof, can be used as the lens identifier. In this embodiment, the calibration parameters of the camera are transmitted to the camera, so that the camera associates and stores the calibration parameters of the camera and the lens identification of the camera, which is beneficial to realize the difference between the lens and the calibration parameters in the image acquisition system with interchangeable lenses. The matching between different lenses matches the corresponding calibration parameters, and further can also match the images captured by the lens. In the case of replacing a new lens, the user can calibrate by himself, and establish the relationship between the new lens identification and the calibration parameters, without returning to the factory for calibration.

The embodiment of the present application acquires multiple images collected by the camera during the movement of the UAV along the preset route, the camera is mounted on the UAV, and the camera includes a body and a lens; the calibration of the camera is calculated by using the multiple images parameters, the calibration parameters of the camera include internal parameters and/or distortion parameters of the camera; the calibration parameters of the camera are transmitted to the camera, so that the camera can compare the calibration parameters of the camera with the camera The lens ID is stored associatively. Since the multiple images used to calculate the calibration parameters of the camera are collected by the camera mounted on the UAV during the movement of the UAV along the preset route, the scene of image collection used for calibration is closer to the actual application In the scene, the camera on the drone can be far away from the shooting object, which helps to cover the various positions of the sensor. Therefore, using the multiple images collected in the calibration scene that is closer to the actual application scene to calculate the calibration parameters of the camera is helpful. In order to improve the accuracy of the calibration parameters, it is helpful to ensure the relative accuracy and stability of the calibration parameters; because the calibration parameters of the camera are also transmitted to the camera, so that the camera can associate the calibration parameters of the camera with the lens identification of the camera. It is beneficial to realize the matching between the lens and the calibration parameters in the image acquisition system of the interchangeable lens. Different lenses match the corresponding calibration parameters, and can also match the image captured by the lens. In the case of replacing a new lens, the user can calibrate by himself, and establish the relationship between the new lens identification and the calibration parameters, without returning to the factory for calibration.

In an embodiment, the plurality of images include a first image and a second image, and the orientations of the photosensitive elements of the camera corresponding to the first image and the second image are different. Or, the principal optical axes of the cameras corresponding to the first image and the second image are oriented differently. Alternatively, the heights of the cameras corresponding to the first image and the second image are different.

If the photosensitive elements of the camera are uniformly oriented in the same direction, during the solution process using the aerial triangulation algorithm, there are multiple solutions for the calculation of the coordinates of the image principal point of the camera, and the image principal point cannot be correctly self-calibrated. Moreover, in the process of actually using the camera to capture images, the photosensitive elements of the camera are not always oriented in one direction, and the photosensitive elements of the camera are oriented differently, which is closer to and matches the actual situation of the actual captured image. In this way, the calculated calibration parameters of the camera can be made more accurate and stable, and in particular, it can be helpful for the accurate and stable stabilization of the principal point coordinates x ₀ and y ₀ in the internal parameters of the camera.

If the cameras are uniformly facing downward (that is, the orientation of the main optical axis is the same), and images are collected at a uniform height, there are multiple solutions to the calculation of the focal length of the camera during the solution process using the aerial triangulation algorithm, and the focal length cannot be calibrated correctly. Moreover, in the process of collecting images with a camera in practical application, the main optical axis of the camera is not always in one direction, and the height of the camera is not always maintained at the same height, which is closer to and matches the actual situation of collecting images, through In this way, the calculated calibration parameters of the camera can be made more accurate and stable, and in particular, it can help to make the focal lengths f _x and f _y in the internal parameters of the camera accurate and stable.

In one embodiment, when the camera captures images at different heights, the positioning sensor on the drone can obtain position information.

In one embodiment, the preset route is a curved route, a tic-tac-toe route or a circumnavigation route. Bending route, tic-tac-toe route, or circumnavigation route, the drone moves along the above-mentioned crooked route, tic-tac-toe route or circumnavigation route, the orientation of the photosensitive element when the camera captures images is not exactly the same, or the main optical axis of the camera is oriented Not exactly the same, or the cameras were not at the same height when the image was captured, etc.

For example: take the UAV as an example, in the specific application, the preset route can be: (1) at different flight heights, at a certain inclination angle (for example, the angle between the camera orientation and the vertical direction is 45 degrees or 30 degrees) or other angles); or, (2) acquisition of a five-way inclined flight path for 3D reconstruction; or, (3) a tilt flight path with a tic-tac-toe turning nose; or (4) a cloud-like flight during a tic-tac-toe flight The stage swings in different directions, so that the camera on the head can shoot at a certain angle of inclination; and so on.

In an embodiment, the shooting parameters when the camera collects the plurality of images meets preset shooting parameter requirements, and the preset shooting parameter requirements include: the focusing distance of the camera is greater than or equal to a preset distance threshold; The focal length of the camera remains unchanged; or, the shutter speed of the camera is greater than or equal to a preset speed threshold.

The shooting parameters can be parameters used by the camera when shooting images, such as shutter, focus distance, aperture, sensitivity (also known as ISO value), exposure value (EV, Exposure Value), whether to turn on the flash, etc. The focusing distance refers to the distance between objects and images. In order to make the image clear, the camera needs to focus to make the image clear within a certain range of depth. The camera is mounted on the drone to collect images, usually far from the object to be photographed, and the photogrammetry specification clearly requires that the focus distance should be set to infinity for image acquisition in aerial surveys.

Therefore, when shooting objects with a long distance, choosing a focus distance greater than or equal to the preset distance threshold helps to calibrate the scene closer to the application scene. The images captured by the camera with the focal length unchanged, and the more stable calibration parameters of the camera are calculated by using these images. The shutter speed of the camera is greater than or equal to the preset speed threshold, so that a clear image can be obtained and motion blur of the image can be reduced as much as possible.

In one embodiment, the plurality of images are acquired by the camera using a global shutter or a mechanical shutter. In this way, the jelly effect of the imaging can be avoided.

Using images to detect fast-moving objects, when the detected object passes quickly through the screen, the object is obviously deformed like jelly, this phenomenon is the so-called "jelly effect". The "jelly effect" is usually a phenomenon that occurs when shooting with rolling shutter. The rolling shutter reads pixels in a line-by-line manner. If the object to be photographed moves or vibrates at a high speed relative to the camera, use the rolling shutter to shoot. If the line scan speed is insufficient, the result may appear "slanted", "wobbly" or "partially exposed".

The global shutter is the parallel reading of pixels on the entire sensor, so it can avoid the jelly effect of imaging. The mechanical shutter can control the light entry time. When shooting fast-moving objects, the light entry time can be controlled very low, and the fast-moving objects can be easily grasped. Therefore, the mechanical shutter can also avoid the jelly effect of imaging.

In an embodiment, the shooting scene when the camera collects the plurality of images meets the preset shooting scene requirements, and the preset shooting scene requirements include: the lighting parameters of the shooting scene meet the preset lighting parameter requirements; or , the elevation parameters of the shooting scene meet the preset elevation parameter requirements.

The preset shooting scene requirements may refer to shooting scene requirements that help to collect multiple images that can improve the stability and accuracy of the calibration parameters. The preset shooting scene requirements include but are not limited to: preset lighting parameter requirements, preset elevation parameter requirements, preset shooting object requirements, and so on. The illumination parameters of the shooting scene meet the preset illumination parameter requirements, including but not limited to: the illumination should not be too bright nor too dark, and the image is not overexposed or the brightness is too dark, which is not conducive to calibrating the parameters. The elevation parameter can reflect whether the shooting scene has the characteristics of ups and downs. It can be required that the ground surface has a certain level of ups and downs, such as an urban environment with artificial buildings, so that even if the absolute height of the camera remains unchanged, the height between the camera and the shooting object is changing.

In one embodiment, in order to obtain better calibration accuracy, the number of the plurality of images is greater than a preset number threshold. The preset number threshold may be determined according to specific conditions, for example, the preset number threshold may be 300 sheets, 500 sheets, and the like.

In one embodiment, the camera lens is replaceable. With the existing calibration method, if the lens of the camera is detachable or replaceable, the disassembly or replacement of the lens will cause a large change in the calibration parameters of the camera, and there will be large errors in the parameters pre-calibrated in the factory. The method of example, because the calibration parameters of the camera are transmitted to the camera, so that the camera stores the calibration parameters of the camera and the lens identification of the camera in association, which is beneficial to realize the difference between the lens and the calibration parameter in the image acquisition system of the interchangeable lens. In the case of replacing a new lens, the user can calibrate by himself, and establish the relationship between the new lens identification and the calibration parameters, without returning to the factory for calibration.

In order to ensure that the calibrated parameters of the camera can be better used in the future, it is possible to check whether the accuracy of the calibration parameters meets the requirements for use, that is, step S103, before transmitting the calibration parameters of the camera to the camera, further It may include: step S104, as shown in FIG. 2 .

Step S104: Check whether the precision of the calibration parameter meets the usage requirements.

At this time, in step S103, the transmitting the calibration parameters of the camera to the camera may include: if the accuracy of the calibration parameters of the camera meets the usage requirements, transmitting the calibration parameters of the camera to the camera.

If the accuracy does not meet the usage requirements, re-calibration can be performed, that is, the method may further include: if the accuracy of the calibration parameters of the camera does not meet the usage requirements, outputting prompt information, the prompt information is used to prompt the user to re-calibrate. Among them, when re-calibrating, you can change the shooting scene and choose the weather with better lighting.

In order to check the accuracy of the calibration parameters, a target point whose actual three-dimensional position is known can be preselected in the shooting scene. When shooting an image, the target point needs to be shot, and at least one of the multiple images includes the pixel point corresponding to the target point. Among them, the target point can select the image control point and check point of the known three-dimensional position that has been deployed, which can reduce the cost and simplify the process.

At this time, in step S104, the checking whether the accuracy of the calibration parameter meets the usage requirements may include: checking the accuracy of the calibration parameter by using the actual three-dimensional position of the target point and the pixel position of the pixel point corresponding to the target point meet the usage requirements.

Knowing the actual three-dimensional position of the target point, through the calculated external parameters, internal parameters, and distortion parameters of the camera, the pixel position of the pixel point calculated by the target point can be obtained, and the calculated pixel position of the pixel point of the target point can be obtained. The position is compared with the pixel position of the pixel corresponding to the target point in the captured image, and it is judged whether the error between the two meets the requirements for use. The accuracy does not meet the requirements for use.

In an embodiment, the calibration parameters of the camera and the lens identifier of the camera are associated and stored in the storage space of the body in the form of a relationship list. In this way, it is helpful to find the corresponding calibration parameters according to the lens identification of the camera.

Wherein, the lens identifier includes the SN code of the lens. The SN code of the lens is the unique product serial number of the lens, which can uniquely identify the lens. It is usually written near the bayonet of the lens. Through the coding of this serial number, some basic information of the lens and the authenticity of the lens can be queried. The SN code of the lens is used as the lens identification, which can uniquely identify the lens.

In an embodiment, in step S103, the transmitting the calibration parameters of the camera to the camera may include: transmitting the calibration parameters of the camera to the control device of the UAV, so that the unmanned The human-machine control device transmits the calibration parameters of the camera to the camera. For example: the control device is a remote controller, the device executing the method of this embodiment is a personal computer PC, and the process of step S103 may be: the PC may first write the calibration parameters into the SD card, store the calibration parameters in the remote controller through the SD card, and then The remote control is transmitted to the drone by wired or wireless means; or, the PC is transferred to the remote control, and the remote control is forwarded to the drone by wired or wireless means; or the PC is directly transmitted to the drone by wireless or wired means UAV; alternatively, it can be transmitted to the UAV, the UAV can transmit it to the camera after receiving it, or it can be directly transmitted to the camera through the above method; and so on.

Referring to FIG. 3, FIG. 3 is a schematic flowchart of an embodiment of the image processing method of the present application. The image processing method of this embodiment is a method on the UAV side, which is used to process the images collected by the camera, that is, the calibration parameter transmission of the camera. For the camera, after the camera stores the calibration parameters of the camera in association with the lens identifier of the camera, the method of performing correlation processing on the captured image. The camera includes a body and a lens. For a detailed description of the same contents in the image processing method according to the embodiment of the present application and the above-mentioned camera parameter calibration method, please refer to the content part of the above-mentioned camera parameter calibration method, which will not be repeated here. The following mainly describes in detail the differences between the image processing method according to the embodiment of the present application and the above-mentioned camera parameter calibration method.

The method includes: step S201, step S202 and step S203.

Step S201: Determine the calibration parameters of the camera corresponding to the lens identification according to the lens identification of the camera, where the calibration parameters of the camera include internal parameters and/or distortion parameters of the camera.

Step S202: Acquire an image captured by the camera.

Step S203: Associate and store the calibration parameters of the camera with the images collected by the camera.

In this embodiment of the present application, the calibration parameters of the camera corresponding to the lens identification are determined according to the lens identification of the camera, and the calibration parameters of the camera include internal parameters and/or distortion parameters of the camera; The image collected; the calibration parameters of the camera are stored in association with the image collected by the camera. Since the lens is marked with the corresponding calibration parameters of the camera, after the image captured by the camera is acquired, the calibration parameters of the camera are stored in association with the image captured by the camera. In this way, the lens can be The collected images can correspond to the calibration parameters of the camera corresponding to the lens, which can avoid the separation of the calibration parameters from the images collected by the lens. When using these images, the calibration parameters of the corresponding camera can be obtained, so that a more accurate space can be established The mapping relationship between 3D points in and 2D points on the image.

In an embodiment, in step S203, the storing the calibration parameters of the camera in association with the images collected by the camera may include: storing the calibration parameters of the camera in the XMP field of the image and/or or the Eixf field.

XMP (Extensible Metadata Platform): The extensible metadata platform is a set of standards for the creation, processing and exchange of metadata. When making and managing documents or images, if metadata (such as title, abstract, author, copyright, etc. information) is included in the relevant documents, the metadata can be used to summarize, classify, search, etc., and various reference materials can also be obtained. , making the process very convenient. XMP is a unified metadata standard. The XMP field of the image is used to save the metadata of the image. In this embodiment of the present application, the calibration parameters of the camera are stored as metadata in the XMP field of the image, so as to facilitate the subsequent search for the calibration parameters of the camera through the metadata.

Eixf (Exchangeable image file format): Exchangeable image file format, is a file format specially set for digital camera photos, which can record the attribute information and shooting data of digital photos. There are fields that identify camera information and lens information.

Wherein, the lens identifier includes the SN code of the lens.

In an embodiment, in step S201, the determining the calibration parameters of the camera according to the lens identification of the camera may include: determining the lens identification corresponding to the lens identification according to the lens identification of the camera and a relationship list. Calibration parameters of the camera, the relationship list is used to indicate the corresponding relationship between the lens identifier and the calibration parameter. Optionally, the relationship list is stored in the storage space of the fuselage.

If the lens identifier does not exist in the relationship list, the method may further include: if the lens identifier does not exist in the relationship list, or if there is no calibration parameter of the camera corresponding to the lens identifier, calibrating The initial value of the parameter is stored in association with the image captured by the camera.

The initial value of the calibration parameter may be the calibration parameter calibrated when the camera leaves the factory, and is generally stored in a non-volatile storage medium inside the camera body when the camera leaves the factory. If there is no lens identifier in the relationship list, or there is no calibration parameter of the camera corresponding to the lens identifier, the initial value of the calibration parameter is stored in association with the image collected by the camera. When the image for calibration is collected and acquired, the image is stored in the initial value of the calibration parameter.

Referring to FIG. 4, FIG. 4 is a schematic structural diagram of an embodiment of the camera parameter calibration device of the present application. The camera is mounted on a drone, and the camera includes a body and a lens. It should be noted that the camera parameters of this embodiment are The calibration device can perform the steps in the above-mentioned camera parameter calibration method. For a detailed description of the relevant content, please refer to the relevant content of the above-mentioned camera parameter calibration method, which will not be repeated here.

The camera parameter calibration device 100 includes: a memory 1 and a processor 2; the processor 2 and the memory 1 are connected through a bus.

Wherein, the processor 2 may be a microcontroller unit, a central processing unit or a digital signal processor, and so on.

Wherein, the memory 1 may be a Flash chip, a read-only memory, a magnetic disk, an optical disk, a U disk, a mobile hard disk, or the like.

The memory 1 is used to store a computer program; the processor 2 is used to execute the computer program and implement the following steps when executing the computer program:

Obtain multiple images collected by the camera during the movement of the UAV along the preset route; use the multiple images to calculate and obtain the calibration parameters of the camera, and the calibration parameters of the camera include the internal parameters of the camera. parameters and/or distortion parameters; transmitting the calibration parameters of the camera to the camera, so that the camera stores the calibration parameters of the camera in association with the lens identification of the camera.

Wherein, the multiple images include a first image and a second image, and the orientations of the photosensitive elements of the camera corresponding to the first image and the second image are different; or, the first image and the second image have different orientations; The principal optical axes of the cameras corresponding to the two images are oriented differently; or the heights of the cameras corresponding to the first image and the second image are different.

Wherein, the preset route is a curved route, a zigzag route or a circumnavigation route.

Wherein, the shooting parameters when the camera collects the multiple images meet the preset shooting parameter requirements, and the preset shooting parameter requirements include: the focusing distance of the camera is greater than or equal to a preset distance threshold; the focal length of the camera unchanged; or the shutter speed of the camera is greater than or equal to a preset speed threshold.

The multiple images are acquired by the camera using a global shutter or a mechanical shutter.

Wherein, the shooting scene when the camera collects the multiple images meets the preset shooting scene requirements, and the preset shooting scene requirements include: the lighting parameters of the shooting scene meet the preset lighting parameter requirements; or the shooting scene The elevation parameters meet the preset elevation parameter requirements.

Wherein, the number of the plurality of images is greater than a preset number threshold.

Wherein, the lens of the camera can be replaced.

Wherein, when executing the computer program, the processor implements the following steps: checking whether the accuracy of the calibration parameters meets the requirements for use; if the accuracy of the calibration parameters of the camera meets the requirements for use, calibrate the camera parameters are transmitted to the camera.

Wherein, when executing the computer program, the processor implements the following steps: if the accuracy of the calibration parameters of the camera does not meet the usage requirements, output prompt information, where the prompt information is used to prompt the user to re-calibrate.

Wherein, at least one of the multiple images includes a pixel point corresponding to a target point, and the actual three-dimensional position of the target point is known; when the processor executes the computer program, the processor implements the following steps: using the The actual three-dimensional position of the target point and the pixel position of the pixel point corresponding to the target point are used to check whether the accuracy of the calibration parameter meets the requirements for use.

Wherein, the calibration parameters of the camera and the lens identifier of the camera are associated and stored in the storage space of the fuselage in the form of a relationship list.

Wherein, the lens identifier includes the SN code of the lens.

Wherein, the calibration device further includes a communication circuit, and when the processor executes the computer program, the processor implements the following steps: controlling the communication circuit to transmit the calibration parameters of the camera to the control device of the UAV, so that the control device of the drone transmits the calibration parameters of the camera to the camera.

Referring to FIG. 5 , FIG. 5 is a schematic structural diagram of an embodiment of an image processing apparatus of the present application. The image processing apparatus is used to process images collected by a camera, and the camera includes a body and a lens. The image processing apparatus of this embodiment may be mounted on a drone, and the image processing apparatus may be an intelligent device capable of processing images captured by a camera, or may be a camera with an image processing function. It should be noted that the image processing apparatus of this embodiment can execute the steps in the above-mentioned image processing method. For a detailed description of the relevant content, please refer to the relevant content of the above-mentioned image processing method, which will not be repeated here.

The image processing apparatus 200 includes: a memory 11 and a processor 22; the processor 22 is connected to the memory 11 through a bus.

Wherein, the processor 22 may be a microcontroller unit, a central processing unit or a digital signal processor, and so on.

Wherein, the memory 11 may be a Flash chip, a read-only memory, a magnetic disk, an optical disk, a U disk, a mobile hard disk, or the like.

The memory 11 is used to store a computer program; the processor 22 is used to execute the computer program and implement the following steps when executing the computer program:

According to the lens identification of the camera, determine the calibration parameters of the camera corresponding to the lens identification, and the calibration parameters of the camera include the internal parameters and/or distortion parameters of the camera; obtain the image collected by the camera; The calibration parameters of the camera are stored in association with the images collected by the camera.

Wherein, when executing the computer program, the processor implements the following steps: storing the calibration parameters of the camera in the XMP field and/or the Eixf field of the image.

Wherein, the lens identifier includes the SN code of the lens.

Wherein, when executing the computer program, the processor implements the following steps: determining the calibration parameters of the camera corresponding to the lens identification according to the lens identification of the camera and a relationship list, and the relationship list is used to indicate Correspondence between lens identification and calibration parameters.

Wherein, the relationship list is stored in the storage space of the fuselage.

Wherein, when executing the computer program, the processor implements the following steps: if there is no lens identifier in the relationship list, or there is no calibration parameter of the camera corresponding to the lens identifier, then the calibration parameter is The initial value is stored in association with the image captured by the camera.

The present application also provides a computer-readable storage medium, where the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor enables the processor to calibrate the camera parameters as described in any one of the above method. For a detailed description of the relevant content, please refer to the above-mentioned relevant content section, which will not be repeated here.

Wherein, the computer-readable storage medium may be an internal storage unit of the above-mentioned camera parameter calibration device, such as a hard disk or a memory. The computer-readable storage medium may also be an external storage device, such as an equipped plug-in hard disk, smart memory card, secure digital card, flash memory card, and the like.

The present application also provides another computer-readable storage medium, where the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor implements the image processing method described in any one of the above . For a detailed description of the relevant content, please refer to the above-mentioned relevant content section, which will not be repeated here.

Wherein, the computer-readable storage medium may be an internal storage unit of the above-mentioned image processing apparatus, such as a hard disk or a memory. The computer-readable storage medium may also be an external storage device, such as an equipped plug-in hard disk, smart memory card, secure digital card, flash memory card, and the like.

It should be understood that the terms used in the specification of the present application are only for the purpose of describing particular embodiments and are not intended to limit the present application.

It will also be understood that, as used in this specification and the appended claims, the term "and/or" refers to and including any and all possible combinations of one or more of the associated listed items.

The above are only specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of various equivalents within the technical scope disclosed in the present application. Modifications or substitutions shall be covered by the protection scope of this application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

A method for calibrating camera parameters, characterized in that the camera is mounted on a drone, the camera includes a body and a lens, and the method includes:

acquiring a plurality of images collected by the camera during the movement of the drone along the preset route;

The calibration parameters of the camera are calculated by using the plurality of images, and the calibration parameters of the camera include internal parameters and/or distortion parameters of the camera;

The calibration parameters of the camera are transmitted to the camera, so that the camera stores the calibration parameters of the camera in association with the lens identifier of the camera.
The method according to claim 1, wherein the plurality of images comprises a first image and a second image, and the orientations of the photosensitive elements of the camera corresponding to the first image and the second image are different; Alternatively, the principal optical axes of the cameras corresponding to the first image and the second images are oriented differently; or the heights of the cameras corresponding to the first image and the second image are different.
The method according to claim 1, wherein the preset route is a curved route, a tic-tac-toe route or a circumnavigation route.
The method according to claim 1, wherein the shooting parameters when the camera collects the plurality of images meet preset shooting parameter requirements, and the preset shooting parameter requirements include:

The focusing distance of the camera is greater than or equal to a preset distance threshold;

The focal length of the camera does not change; or

The shutter speed of the camera is greater than or equal to a preset speed threshold.
The method according to claim 1, wherein the plurality of images are acquired by the camera using a global shutter or a mechanical shutter.
The method according to claim 1, wherein a shooting scene when the camera collects the plurality of images meets a preset shooting scene requirement, and the preset shooting scene requirement includes:

The lighting parameters of the shooting scene meet the preset lighting parameter requirements; or

The elevation parameters of the shooting scene meet the preset elevation parameter requirements.
The method of claim 1, wherein the number of the plurality of images is greater than a preset number threshold.
The method of claim 1, wherein the camera lens is replaceable.
The method according to claim 1, wherein before transmitting the calibration parameters of the camera to the camera, the method further comprises:

Check whether the accuracy of the calibration parameters meets the requirements for use;

The transmitting the calibration parameters of the camera to the camera includes:

If the accuracy of the calibration parameters of the camera meets the usage requirements, the calibration parameters of the camera are transmitted to the camera.
The method according to claim 9, wherein the method further comprises:

If the accuracy of the calibration parameters of the camera does not meet the usage requirements, prompt information is output, and the prompt information is used to prompt the user to re-calibrate.
The method according to claim 9, wherein at least one of the plurality of images includes a pixel point corresponding to a target point, and the actual three-dimensional position of the target point is known;

The checking whether the accuracy of the calibration parameters meets the requirements for use includes:

Whether the accuracy of the calibration parameter meets the usage requirements is checked by using the actual three-dimensional position of the target point and the pixel position of the pixel point corresponding to the target point.
The method according to claim 1, wherein the calibration parameters of the camera and the lens identifier of the camera are stored in association with each other in the form of a relationship list.
The method according to claim 1, wherein the calibration parameters of the camera are stored in the storage space of the fuselage in association with the lens identifier of the camera.
The method according to claim 1, wherein the shot identification comprises an SN code of the shot.
The method according to claim 1, wherein the transmitting the calibration parameters of the camera to the camera comprises:

The calibration parameters of the camera are transmitted to the control device of the drone, so that the control device of the drone transmits the calibration parameters of the camera to the camera.
An image processing method for processing images collected by a camera, the camera comprising a body and a lens, characterized in that the method includes:

According to the lens identification of the camera, determine the calibration parameter of the camera corresponding to the lens identification, and the calibration parameter of the camera includes the internal parameter and/or the distortion parameter of the camera;

acquiring images captured by the camera;

The calibration parameters of the camera are stored in association with the images collected by the camera.
The method according to claim 16, wherein the storing the calibration parameters of the camera in association with the images collected by the camera comprises:

The calibration parameters of the camera are stored in the XMP field and/or the Eixf field of the image.
The method according to claim 16, wherein the determining the calibration parameters of the camera according to the lens identifier of the camera comprises:

A calibration parameter of the camera corresponding to the lens identification is determined according to the lens identification of the camera and a relationship list, and the relationship list is used to indicate the corresponding relationship between the lens identification and the calibration parameter.
The method according to claim 18, wherein the relationship list is stored in a storage space of the fuselage.
The method of claim 19, wherein the method further comprises:

If the lens identifier does not exist in the relationship list, or there is no calibration parameter of the camera corresponding to the lens identifier, the initial value of the calibration parameter is stored in association with the image collected by the camera.
A camera parameter calibration device, characterized in that the camera is mounted on a drone, the camera includes a body and a lens, and the device includes: a memory and a processor;

the memory is used to store computer programs;

The processor is configured to execute the computer program and implement the following steps when executing the computer program:

acquiring a plurality of images collected by the camera during the movement of the drone along the preset route;

The calibration parameters of the camera are calculated by using the plurality of images, and the calibration parameters of the camera include internal parameters and/or distortion parameters of the camera;

The calibration parameters of the camera are transmitted to the camera, so that the camera stores the calibration parameters of the camera in association with the lens identifier of the camera.
The device according to claim 21, wherein the plurality of images comprises a first image and a second image, and the orientations of the photosensitive elements of the camera corresponding to the first image and the second image are different; Alternatively, the principal optical axes of the cameras corresponding to the first image and the second images are oriented differently; or the heights of the cameras corresponding to the first image and the second image are different.
The device according to claim 21, wherein the preset route is a curved route, a tic-tac-toe route or a circumnavigation route.
The device according to claim 21, wherein the shooting parameters when the camera collects the multiple images meets preset shooting parameter requirements, and the preset shooting parameter requirements include:

The focusing distance of the camera is greater than or equal to a preset distance threshold;

The focal length of the camera does not change; or

The shutter speed of the camera is greater than or equal to a preset speed threshold.
The apparatus according to claim 21, wherein the plurality of images are acquired by the camera using a global shutter or a mechanical shutter.
The device according to claim 21, wherein a shooting scene when the camera collects the plurality of images satisfies a preset shooting scene requirement, and the preset shooting scene requirement includes:

The lighting parameters of the shooting scene meet the preset lighting parameter requirements; or

The elevation parameters of the shooting scene meet the preset elevation parameter requirements.
The apparatus according to claim 21, wherein the number of the plurality of images is greater than a preset number threshold.
21. The device of claim 21, wherein the camera lens is replaceable.
The device according to claim 21, wherein, when the processor executes the computer program, the following steps are implemented:

Check whether the accuracy of the calibration parameters meets the requirements for use;

If the accuracy of the calibration parameters of the camera meets the usage requirements, the calibration parameters of the camera are transmitted to the camera.
The apparatus according to claim 29, wherein when the processor executes the computer program, the following steps are implemented:

If the accuracy of the calibration parameters of the camera does not meet the usage requirements, prompt information is output, and the prompt information is used to prompt the user to re-calibrate.
The device according to claim 29, wherein at least one of the plurality of images includes a pixel point corresponding to a target point, and the actual three-dimensional position of the target point is known;

When the processor executes the computer program, the following steps are implemented:

Use the actual three-dimensional position of the target point and the pixel position of the pixel point corresponding to the target point to check whether the accuracy of the calibration parameter meets the usage requirements.
The apparatus according to claim 21, wherein the calibration parameters of the camera and the lens identifiers of the camera are stored in association with each other in the form of a relationship list.
The device according to claim 21, wherein the calibration parameters of the camera are stored in the storage space of the fuselage in association with the lens identifier of the camera.
The apparatus according to claim 21, wherein the lens identifier comprises an SN code of the lens.
The device according to claim 21, wherein the calibration device further comprises a communication circuit, and the processor implements the following steps when executing the computer program:

The communication circuit is controlled to transmit the calibration parameters of the camera to the control device of the drone, so that the control device of the drone transmits the calibration parameters of the camera to the camera.
An image processing device for processing images collected by a camera, the camera comprising a body and a lens, characterized in that the device comprises: a memory and a processor;

the memory is used to store computer programs;

The processor is configured to execute the computer program and implement the following steps when executing the computer program:

According to the lens identification of the camera, determine the calibration parameter of the camera corresponding to the lens identification, and the calibration parameter of the camera includes the internal parameter and/or the distortion parameter of the camera;

acquiring images captured by the camera;

The calibration parameters of the camera are stored in association with the images collected by the camera.
The apparatus according to claim 36, wherein when the processor executes the computer program, the following steps are implemented:

The calibration parameters of the camera are stored in the XMP field and/or the Eixf field of the image.
The apparatus according to claim 36, wherein when the processor executes the computer program, the following steps are implemented:

A calibration parameter of the camera corresponding to the lens identification is determined according to the lens identification of the camera and a relationship list, and the relationship list is used to indicate the corresponding relationship between the lens identification and the calibration parameter.
The device according to claim 38, wherein the relationship list is stored in the storage space of the body.
The apparatus according to claim 39, wherein when the processor executes the computer program, the following steps are implemented:

If the lens identifier does not exist in the relationship list, or there is no calibration parameter of the camera corresponding to the lens identifier, the initial value of the calibration parameter is stored in association with the image collected by the camera.
A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor implements the process according to any one of claims 1-15 The calibration method of the camera parameters.
A computer-readable storage medium, characterized in that, the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor implements the process according to any one of claims 16-20 image processing method.