CN109697737B - Camera calibration method, device, electronic equipment and computer-readable storage medium - Google Patents
Camera calibration method, device, electronic equipment and computer-readable storage medium Download PDFInfo
- Publication number
- CN109697737B CN109697737B CN201811452677.3A CN201811452677A CN109697737B CN 109697737 B CN109697737 B CN 109697737B CN 201811452677 A CN201811452677 A CN 201811452677A CN 109697737 B CN109697737 B CN 109697737B
- Authority
- CN
- China
- Prior art keywords
- camera
- calibration
- aperture value
- target
- camera module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
Abstract
The application relates to a camera calibration method, a camera calibration device, an electronic device and a computer readable storage medium. The method comprises the following steps: the method comprises the steps of obtaining different aperture values of a main camera in a camera module in sequence, shooting a calibration plate through the camera module according to the different aperture values, obtaining a set of calibration images corresponding to each aperture value, wherein the set of calibration images comprise calibration images respectively shot by the main camera and at least two auxiliary cameras in the camera module, and calibrating the camera module according to a set of calibration images corresponding to each aperture value, so as to obtain camera calibration parameters corresponding to the camera module under each aperture value. The calibration plate can be shot through the camera modules with different aperture values, calibration processing is carried out according to the obtained calibration image corresponding to each aperture value, calibration parameters corresponding to the camera modules under different aperture values are obtained, and the calibration accuracy of the camera can be improved.
Description
Technical Field
The present application relates to the field of image technologies, and in particular, to a method and an apparatus for calibrating a camera, an electronic device, and a computer-readable storage medium.
Background
Before the camera leaves the factory, the camera needs to be calibrated to obtain calibration parameters of the camera, so that the camera can process images according to the calibration parameters, and the processed images can restore objects in a three-dimensional space. However, in the use process of the camera, different shooting conditions can affect the imaging effect of the image, and the problem of low calibration accuracy of the camera exists.
Disclosure of Invention
The embodiment of the application provides a camera calibration method, a camera calibration device, electronic equipment and a computer-readable storage medium, which can improve the accuracy of camera calibration.
A camera calibration method, comprising:
sequentially acquiring different aperture values of a main camera in a camera module, and shooting a calibration plate by the camera module according to the different aperture values to obtain a group of calibration images corresponding to each aperture value, wherein the group of calibration images comprise calibration images respectively shot by the main camera and at least two auxiliary cameras in the camera module;
and calibrating the camera module according to a group of calibration images corresponding to each aperture value to obtain camera calibration parameters corresponding to the camera module under each aperture value.
A camera calibration device comprises:
the camera module is used for shooting a calibration plate according to different aperture values to obtain a group of calibration images corresponding to each aperture value, wherein the group of calibration images comprise calibration images respectively shot by the main camera and at least two auxiliary cameras in the camera module;
and the calibration processing module is used for calibrating the camera module according to a group of calibration images corresponding to each aperture value to obtain camera calibration parameters corresponding to the camera module under each aperture value.
An electronic device comprising a camera module, a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the following steps:
sequentially acquiring different aperture values of a main camera in a camera module, and shooting a calibration plate by the camera module according to the different aperture values to obtain a group of calibration images corresponding to each aperture value, wherein the group of calibration images comprise calibration images respectively shot by the main camera and at least two auxiliary cameras in the camera module;
and calibrating the camera module according to a group of calibration images corresponding to each aperture value to obtain camera calibration parameters corresponding to the camera module under each aperture value.
A computer-readable storage medium, on which a computer program is stored which, when executed by a processor, carries out the steps of:
sequentially acquiring different aperture values of a main camera in a camera module, and shooting a calibration plate by the camera module according to the different aperture values to obtain a group of calibration images corresponding to each aperture value, wherein the group of calibration images comprise calibration images respectively shot by the main camera and at least two auxiliary cameras in the camera module;
and calibrating the camera module according to a group of calibration images corresponding to each aperture value to obtain camera calibration parameters corresponding to the camera module under each aperture value.
According to the camera calibration method, the camera calibration device, the electronic equipment and the computer-readable storage medium, different aperture values of a main camera in a camera module are sequentially obtained, the camera module shoots a calibration plate with different aperture values to obtain a group of calibration images corresponding to each aperture value, wherein the group of calibration images comprise calibration images respectively shot by the main camera and at least two auxiliary cameras in the camera module, and the camera module is calibrated according to the group of calibration images corresponding to each aperture value to obtain camera calibration parameters corresponding to the camera module under each aperture value. Because can shoot with different aperture values through the camera module calibration board, carry out calibration according to the demarcation image that each aperture value that obtains corresponds and handle, obtain the camera module and demarcate the parameter under different aperture values correspond, can improve the camera and demarcate the accuracy.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a diagram of an application environment for a camera calibration method in one embodiment;
FIG. 2 is a flow diagram of a camera calibration method in one embodiment;
FIG. 3 is a schematic diagram of a camera calibration method in one embodiment;
FIG. 4 is a flowchart illustrating the process of capturing images with the camera module according to an embodiment;
FIG. 5 is a flow diagram of a camera calibration method in one embodiment;
FIG. 6 is a flow diagram of a correction process performed on a captured image in one embodiment;
FIG. 7 is a flow chart of a camera calibration method in yet another embodiment;
FIG. 8 is a block diagram showing the structure of a camera calibration apparatus according to an embodiment;
FIG. 9 is a block diagram showing an internal configuration of an electronic apparatus according to an embodiment;
FIG. 10 is a schematic diagram of an image processing circuit in one embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
It will be understood that, as used herein, the terms "first," "second," and the like may be used herein to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish one element from another. For example, a first camera may be referred to as a second camera, and similarly, a second camera may be referred to as a first camera, without departing from the scope of the present application. The first camera and the second camera are both cameras, but they are not the same camera.
Fig. 1 is a schematic diagram of an application environment of the camera calibration method in one embodiment. As shown in fig. 1, the application environment includes an electronic device 110 and a calibration board 120. The calibration plate (chart plate) 120 has a chart pattern thereon. The electronic device 110 may sequentially obtain different aperture values of a main camera in the camera module, capture the calibration plate 120 with different aperture values through the camera module, obtain a set of calibration images corresponding to each aperture value of the camera module, and calibrate the camera module according to a set of calibration images corresponding to each aperture value, so as to obtain camera calibration parameters corresponding to the camera module under each aperture value. Wherein, the calibration image that main camera and two at least supplementary cameras were shot respectively in having contained the camera module in a set of calibration image. The main camera may be a color camera with an adjustable aperture value, and the auxiliary camera may be, but is not limited to, one or more of a color camera, a black-and-white camera, a telephoto camera, a wide-angle camera, or a depth camera. The calibration board 120 may be a three-dimensional calibration board or a two-dimensional calibration board.
Fig. 2 is a flow diagram of a camera calibration method in one embodiment. The camera calibration method in the present embodiment is described by taking the electronic device in fig. 1 as an example. As shown in fig. 2, the camera calibration method includes steps 202 to 204.
The camera module is an assembly including at least one camera. The camera module can be arranged in the electronic equipment or arranged outside the electronic equipment, so that the electronic equipment can acquire images through the camera module. In this application embodiment, the camera module can contain at least three camera, specifically, the camera module contains main camera and two at least supplementary cameras. The auxiliary camera can be but not limited to one or more of a color camera, a black-and-white camera, a telephoto camera, a wide-angle camera or a depth camera. For example, the camera module may include two color cameras and one depth camera, and the camera module may also include a color camera, a black-and-white camera, a depth camera, and the like, but is not limited thereto. The aperture value is a relative value of the focal length and the light transmission diameter of the camera. The aperture of the camera is a device for controlling the amount of light entering a light sensing surface in the camera, and the size of the aperture can be expressed by an aperture value. The larger the aperture value is, the smaller the light transmission diameter, i.e., the aperture is, the smaller the light input amount per unit time is; conversely, the smaller the aperture value, the larger the light transmission diameter, i.e., the aperture, and the larger the amount of light entering per unit time.
The main camera is a camera having a plurality of aperture values so that the size of the aperture can be adjusted for photographing. A calibration plate refers to a pattern with an array of fixed pitch images. Specifically, the calibration plate may be a three-dimensional calibration plate with at least three calibration surfaces, or may be a flat plate with only one calibration surface. When the electronic equipment calibrates the camera, at least more than three calibration pictures are needed, therefore, when the calibration plate is a three-dimensional calibration plate, the electronic equipment shoots the calibration plate through the camera module by using different aperture values, a group of calibration images corresponding to each aperture value can comprise a main camera in the camera module and at least two calibration images shot by at least two auxiliary cameras respectively, and the calibration images comprise three different calibration images; when the calibration plate is a flat plate with only one calibration surface, the electronic equipment can shoot the calibration plate at least three angles through the camera module under each aperture value, so that at least three calibration images shot by three cameras in the camera module can be contained in a group of calibration images corresponding to each aperture value.
The electronic equipment can acquire the different aperture values of the main camera in the camera module in sequence, and the calibration plate is shot by the camera module according to the different aperture values to obtain a set of calibration images corresponding to each aperture value of the camera module. Specifically, the electronic device may obtain the adjusted aperture value of the main camera after adjusting the aperture value of the main camera each time, where the adjustment of the aperture value of the main camera may be performed by a user or may be performed automatically by the electronic device.
And 204, calibrating the camera module according to a group of calibration images corresponding to each aperture value to obtain camera calibration parameters corresponding to the camera module under each aperture value.
The calibration processing refers to the operation of solving parameters in a geometric model imaged by the camera, and the shot image can restore an object in a three-dimensional space through the geometric model imaged by the camera. Specifically, the calibration process may include a monocular calibration process for finding internal parameters, external parameters, distortion coefficients, and the like of a single camera, and a binocular calibration process for finding external parameters between two cameras. The camera calibration parameters refer to camera parameters obtained after the camera module is calibrated. The camera calibration parameters can be used for correcting the image acquired by the camera, so that the corrected image can restore the object in the three-dimensional space. Specifically, the camera calibration parameters may include a monocular calibration parameter of a single camera in the camera module and a binocular calibration parameter between the main camera and the auxiliary camera in the camera module, where the monocular calibration parameter may include an internal parameter, an external parameter, a distortion coefficient, and the like, and the binocular calibration parameter is the external parameter between the two cameras.
The electronic device calibrates the camera module according to a set of calibration images corresponding to each aperture value, and specifically, the electronic device may calibrate the camera module by using a conventional camera calibration method, a camera self-calibration method, a calibration method for a Zhang friend between the conventional calibration method and the self-calibration method, and the like, so as to obtain calibration parameters corresponding to the camera module. For example, when the electronic device acquires a set of calibration images corresponding to the first aperture value by adjusting the aperture value of the main camera, and the set of calibration images respectively includes the calibration image a acquired by the main camera, the calibration image B acquired by the auxiliary camera 1, and the calibration image C acquired by the auxiliary camera 2, the electronic device performs calibration processing according to the calibration images a, B, and C, and the obtained camera calibration parameters include internal parameters, external parameters, distortion coefficients, and the like corresponding to the main camera and the auxiliary cameras 1 and 2, respectively, and external parameters between the main camera and the auxiliary camera 1, and external parameters between the main camera and the auxiliary camera 2.
In the embodiment of the application, different aperture values of a main camera in a camera module are sequentially acquired, the calibration plate is shot by the camera module according to different aperture values, a set of calibration images corresponding to each aperture value is obtained, wherein the set of calibration images comprise calibration images respectively shot by the main camera and at least two auxiliary cameras in the camera module, the camera module is calibrated according to the set of calibration images corresponding to each aperture value, the camera calibration parameters corresponding to the camera module under each aperture value are obtained, the situation that the camera with multiple aperture values is calibrated only once is avoided, and the accuracy of camera calibration is improved. Furthermore, in the use process of the camera module, the corresponding camera calibration parameters can be acquired according to the adopted aperture value to process the image, and the accuracy of image processing can be improved.
In an embodiment, in the provided camera calibration method, a process of shooting the calibration plate with different aperture values through the camera module includes: under each aperture value of camera module, shoot the calibration plate of at least three angle.
When the calibration plate is a flat plate having only one calibration surface, the calibration plate can be rotated through a plurality of angles by the rotation shaft. In at least three angles of the calibration plate, one of the angles can be 0 degree, and the other angles are ± θ degrees. The value of theta can be determined according to actual requirements on the basis of ensuring decoupling between postures. For example, the electronic device captures calibration plates at five angles of 0 degree, -20 degree, +20 degree, -30 degree and +30 degree at any aperture value, and obtains a set of calibration images corresponding to the aperture value.
Fig. 3 is a schematic diagram of the camera module capturing images at least three angles according to an embodiment. As shown in fig. 3, the optical axis of each camera in the camera module in the electronic device 110 is perpendicular to the rotation axis of the calibration board 120, the calibration board 120 can rotate by a plurality of angles through the rotation axis, one of the angles can be 0 degree, and the other two rotation angles are ± θ degrees, and the electronic device 110 can shoot the rotated calibration board 120.
Electronic equipment shoots at least three angle of calibration board under each aperture value of camera module, can obtain a set of demarcation image that each aperture value of camera module corresponds, contains three at least demarcation images that main camera and supplementary camera were shot respectively in the camera module in a set of demarcation image.
As shown in fig. 4, in an embodiment, a process of shooting a calibration plate with different aperture values by a camera module in a camera calibration method may include steps 402 to 406, where:
The aperture value of the main camera typically has a corresponding adjustment range. The minimum aperture value is the aperture value when the light-passing diameter of the main camera is maximum; the maximum aperture value is the aperture value when the main camera has the smallest light-passing diameter. The electronic device may acquire a minimum aperture value and a maximum aperture value of the main camera.
The target aperture value is an aperture value corresponding to a calibration image for performing calibration processing on the camera. The electronic equipment determines a target aperture value according to the minimum aperture value and the maximum aperture value, and specifically, the electronic equipment can obtain an aperture value as the target aperture value every other preset aperture difference value from the minimum aperture value; the electronic equipment can also obtain an aperture value as a target aperture value every preset number of aperture values from the minimum aperture value; the electronic device may further directly obtain, according to the preset aperture value, an aperture value that is the same as the preset aperture value among the minimum aperture value and the maximum aperture value as the target aperture value, and the like, which is not limited thereto, wherein the preset aperture difference value, the preset number, and the preset aperture value may be set according to the actual application requirement, and are not limited herein. For example, when the preset aperture value is 3, the minimum aperture value of the main camera is 3, and the maximum aperture value is 9, the electronic apparatus may acquire one aperture value as the target aperture value every preset aperture value of 3 from the minimum aperture value of 3, and the target aperture values are 3, 6, and 9.
And step 406, shooting the calibration plate by the camera module at different target aperture values.
After the electronic equipment determines the target aperture value according to the minimum aperture value and the maximum aperture value, the aperture value of the main camera can be adjusted according to the target aperture value, the calibration plate is shot by the camera module according to different target aperture values, and a set of calibration images corresponding to each target aperture of the camera module respectively are obtained.
The camera module is calibrated, shooting of the camera module is avoided, shooting of the calibrated images is avoided at each aperture value of the main camera, and efficiency of image acquisition and calibration processing can be improved.
In one embodiment, the target aperture value includes a first target aperture value and a second target aperture value, and the process of shooting the calibration plate at different target aperture values through the camera module may include: the calibration plate is adjusted according to at least three angles, and after the angle of the calibration plate is adjusted at each time, the calibration plate is shot by the camera module according to the first target aperture value and the second target aperture value.
The electronic equipment adjusts the calibration plate according to at least three angles, and after the angle of the calibration plate is adjusted at each time, the calibration plate is shot by the camera module according to the first target aperture value and the second target aperture value respectively. In an embodiment, the calibration board may be adjusted according to the first angle, the second angle, and the third angle, and the process of shooting the calibration board by the electronic device through the camera module with different target aperture values may include: adjusting the calibration plate to a first angle, acquiring a first target aperture value of a main camera after adjustment, shooting the calibration plate by the first target aperture value through the camera module, acquiring a second target aperture value of the main camera after shooting, and shooting the calibration plate by the second target aperture value through the camera module; adjusting the calibration plate to a second angle, acquiring a first target aperture value of the main camera after adjustment, shooting the calibration plate by the first target aperture value through the camera module, acquiring a second target aperture value of the main camera after shooting, and shooting the calibration plate by the second target aperture value through the camera module; adjusting the calibration plate to a third angle, acquiring a first target aperture value of the main camera after adjustment, shooting the calibration plate by the first target aperture value through the camera module, acquiring a second target aperture value of the main camera after shooting, and shooting the calibration plate by the second target aperture value through the camera module; thereby obtaining a group of calibration images corresponding to the first target aperture value and the second target aperture value respectively. The electronic equipment adjusts the calibration plate according to at least three angles, adjusts the back at every turn to the calibration plate, shoots the calibration plate through the camera module with first target aperture value and second target aperture value, can reduce the regulation number of times of calibration plate, improves the efficiency that the camera was markd.
As shown in fig. 5, in an embodiment, in the camera calibration method, the auxiliary cameras included in the camera module include a first auxiliary camera and a second auxiliary camera, and in the method, the calibration process is performed on the camera module according to a set of calibration images corresponding to each aperture value, and a process of obtaining camera calibration parameters corresponding to the camera module at each aperture value may include steps 502 to 508. Wherein:
The electronic equipment shoots the calibration plate through the camera module according to different aperture values, a set of calibration images corresponding to each aperture value are obtained, each set of calibration images contain calibration images obtained by shooting by each camera in the camera module respectively, and the electronic equipment can obtain the calibration images corresponding to each camera in the camera module respectively.
And step 504, performing monocular calibration processing on each camera according to the calibration image corresponding to each camera to obtain a monocular calibration parameter corresponding to each aperture value.
The monocular calibration processing is an operation of calibrating a single camera and calculating internal parameters, external parameters, distortion coefficients and the like of the camera. The electronic equipment can perform monocular calibration processing on each camera according to the calibration image corresponding to each camera respectively to obtain the monocular calibration parameters corresponding to each camera under each aperture value. The monocular calibration parameters comprise internal parameters, external parameters, distortion coefficients and the like corresponding to the camera under the aperture value. For example, in the above example, when a set of calibration images acquired by the electronic device through the camera module at the first aperture value includes the calibration image a acquired by the main camera, the calibration image B acquired by the first auxiliary camera, and the calibration image C acquired by the second auxiliary camera, the electronic device performs monocular calibration processing on the main camera, the first auxiliary camera, and the second auxiliary camera according to the calibration image a, the calibration image B, and the calibration image C, respectively, so as to obtain monocular calibration parameters corresponding to the respective cameras at the first aperture value.
And 506, performing binocular calibration processing on the main camera and the first auxiliary camera according to the monocular calibration parameters corresponding to the main camera and the first auxiliary camera under each aperture value to obtain a first binocular calibration parameter corresponding to each aperture value.
The binocular calibration processing refers to an operation of calibrating at least two cameras and solving an external parameter between the at least two cameras. The first dual targeting parameter is used for representing a parameter of a relative distance between the main camera and the first auxiliary camera. Specifically, the first dual targeting parameter may be an extrinsic parameter between the main camera and the first auxiliary camera. The first dual-target fixed parameters comprise a rotation matrix and a translation matrix between the main camera and the first auxiliary camera. The electronic equipment can calculate a first double-target fixed parameter according to the monocular calibration parameters respectively corresponding to the main camera and the first auxiliary camera under each aperture value. In an embodiment, the electronic device may also perform binocular calibration processing on the main camera and the first auxiliary camera according to calibration images corresponding to the main camera and the first auxiliary camera respectively at each aperture value.
And step 508, performing binocular calibration processing on the main camera and the second auxiliary camera according to the monocular calibration parameters corresponding to the main camera and the second auxiliary camera under each aperture value to obtain a second binocular calibration parameter corresponding to each aperture value.
Similarly, a second bi-targeting parameter is used for a parameter representing the relative distance between the main camera and the second auxiliary camera. In particular, the second bi-targeting parameter may be an extrinsic parameter between the main camera and the second auxiliary camera. The second dual-target fixed parameters comprise a rotation matrix and a translation matrix between the main camera and the second auxiliary camera. The electronic equipment can calculate second double-target fixed parameters according to the monocular calibration parameters respectively corresponding to the main camera and the second auxiliary camera under each aperture value.
Therefore, the electronic equipment can obtain the monocular calibration parameters, the first double-target parameter and the second double-target parameter corresponding to each aperture value, and the calibration accuracy of the camera module with the three cameras can be improved. Furthermore, in the use process of the camera module, the electronic device can acquire one or more calibration parameters of corresponding monocular calibration parameters, first double-target calibration parameters or second double-target calibration parameters according to the acquired aperture value adopted by the main camera, and processes the image, so that the accuracy of image processing can be improved.
As shown in fig. 6, in one embodiment, the provided camera calibration method may further include steps 602 to 606. Wherein:
The initial image is an image which is acquired by the camera module and is not corrected through calibration parameters. When the camera module is used for image acquisition, a main camera and at least one auxiliary camera are generally adopted for shooting, and then an initial image can be an image acquired by one or more cameras in the camera module. And the acquired aperture value is the aperture value corresponding to the main camera when the initial image is acquired. When the electronic equipment collects an initial image through the camera module, acquiring a collection aperture value of a main camera; when the initial image is the image collected by the main camera, the electronic device can also directly obtain the collection aperture value corresponding to the initial image.
In one embodiment, the electronic device may further acquire a target application program initiating a capture instruction for capturing the initial image when the initial image is captured by the camera module, and acquire a capture aperture value of the main camera when it is determined that the target application program belongs to the first type of application program.
An application refers to a computer program that can accomplish certain tasks. Specifically, the application program has a function of calling a camera to acquire an image. For example, the Instagram may capture an image through a camera, the WeChat may capture an image through a camera, or scan a two-dimensional code image, the payment application, or the lock screen application may capture a face image through a camera for recognition, and the like, but are not limited thereto. The acquisition instruction can be generated by clicking a button on the display screen by a user, or generated by pressing a control on the touch screen by the user, and the like, and when the electronic equipment acquires the initial image through the camera module, the electronic equipment acquires a target application program initiating the acquisition instruction for acquiring the initial image. The electronic device may classify the application, and specifically, the electronic device may classify the application according to a requirement for image accuracy, a requirement for image processing speed, and the like. The first type of application may be an application that requires a higher image accuracy or a lower image processing speed. The electronic device can judge whether the target application program belongs to the first type of application program or not when the target application program corresponding to the acquisition instruction is acquired, and acquire the acquisition aperture value of the main camera when the target application program belongs to the first type of application program.
And step 604, acquiring a first target calibration parameter corresponding to the acquired aperture value from the camera calibration parameter corresponding to each aperture value of the camera module.
The electronic device calibrates the camera module according to a set of calibration images corresponding to each aperture value to obtain camera calibration parameters corresponding to the camera module at each aperture value, and then the electronic device can obtain corresponding first target calibration parameters from the camera calibration parameters corresponding to each aperture value according to the acquired aperture value. In one embodiment, when there is no camera calibration parameter corresponding to the collected aperture value, the electronic device may acquire the camera calibration parameter corresponding to the aperture value closest to the collected aperture value as the first target calibration parameter.
And 606, correcting the initial image according to the first target calibration parameter, and outputting the obtained target image.
The electronic device corrects the initial image according to the first target calibration parameter, and specifically, the electronic device may correct the initial image according to a monocular calibration parameter corresponding to each camera in the first target calibration parameter and a binocular calibration parameter between the cameras, so as to obtain a processed target image. For example, when the electronic device acquires an initial image M1 and an initial image N1 through a main camera and a first auxiliary camera in a camera module, the electronic device obtains corresponding first target calibration parameters according to an acquisition aperture value corresponding to the initial image M, corrects the initial image M1 according to monocular calibration parameters corresponding to the main camera in the first target calibration parameters to obtain an image M2, corrects the initial image N1 according to the monocular calibration parameters corresponding to the first auxiliary camera to obtain an image N2, and further corrects the image M2 and the image N2 according to a first dual target calibration parameter between the main camera and the first auxiliary camera to obtain a target image.
The acquisition aperture value when the camera module acquires the initial image is acquired, the corresponding first target calibration parameter is acquired according to the acquisition aperture value, the initial image is corrected to obtain the target image, the accuracy of the target image can be improved, the influence on the definition and the like of the image is avoided when the initial image is processed by adopting the unified calibration parameter, and the image processing effect is optimized.
In one embodiment, the provided camera calibration method may further include: and when the target application program is judged to belong to the second type of application program, acquiring a preset second target calibration parameter, correcting the initial image according to the second target calibration parameter, and outputting the obtained target image.
The camera usually has a default aperture value when in actual application, and when just starting the camera, the camera collects images through the default aperture value. The preset second target calibration parameter is a camera calibration parameter corresponding to the default aperture value. The electronic device may categorize applications that capture images according to a default aperture value as a second type of application. The electronic device may also classify applications that require less accuracy in image processing or more speed in image processing as a second class of applications. When the target application program belongs to the first type of application program, the electronic equipment acquires a first target calibration parameter corresponding to the acquired aperture value, and corrects the initial image according to the first target calibration parameter; and when the target application program is judged to belong to the second type of application program, the electronic equipment corrects the initial image according to a preset second target calibration parameter and outputs the obtained target image.
The electronic equipment judges the classification of the target application program according to the target application program initiating the acquisition instruction, so that the corresponding calibration parameters are obtained to correct the initial image, and when the target application program belongs to the second type of application program, the preset second target calibration parameters are directly adopted to process the initial image, so that the image processing efficiency can be improved.
In an embodiment, the auxiliary camera in the provided camera calibration method includes a depth camera, and the electronic device performs calibration processing on the camera module according to a set of calibration images corresponding to each aperture value, and the obtained camera calibration parameters of the camera module corresponding to each aperture value include calibration parameters between the main camera and the depth camera and calibration parameters between the main camera and other auxiliary cameras. In the use process of the camera module, the electronic equipment can calibrate the depth image acquired by the depth camera and the image acquired by the main camera to obtain first depth information of the image acquired by the main camera; the electronic equipment can also perform calibration processing according to images acquired by the main camera and other auxiliary cameras, and obtain second depth information of the images according to binocular ranging, so that the electronic equipment can process the acquired images according to the first depth information or the second depth information according to actual requirements, for example, when the images have scenes with unclear textures, the electronic equipment can process the images according to the first depth information; when the depth information of the photographed object in the image is large, the electronic device may process the image according to the second depth information.
The camera module of the electronic equipment comprises the depth camera, and the camera calibration parameters obtained by calibrating the camera module comprise calibration information of the depth camera and the main camera and calibration information of the main camera and other auxiliary cameras, so that after the initial image is corrected by the electronic equipment according to the camera calibration parameters, two groups of depth information can be respectively obtained, and the electronic equipment selects the depth information to process the image according to actual conditions, so that the accuracy of image processing can be improved.
Fig. 7 is a flow chart of a camera calibration method in one embodiment. For example, the camera calibration method includes:
And step 704, sequentially acquiring different aperture values of the main camera at the first angle of the calibration plate and shooting through the camera module.
And step 706, sequentially acquiring different aperture values of the main camera at a second angle of the calibration plate and shooting through the camera module.
And 708, sequentially acquiring different aperture values of the main camera at a third angle of the calibration plate and shooting through the camera module.
And step 710, acquiring a group of calibration images shot by each aperture value for calibration processing.
And 712, storing the calibration parameters corresponding to each aperture value.
Through different angles of the calibration plate, the camera module shoots the calibration plate through different aperture values to obtain a set of calibration images corresponding to the camera module under each aperture value, so that the camera module is calibrated according to a set of calibration images corresponding to each aperture value to obtain camera calibration parameters corresponding to each aperture value, the calibration accuracy of the camera can be improved, the adjustment times of the calibration plate are reduced, and the calibration efficiency of the camera is improved.
It should be understood that although the various steps in the flowcharts of fig. 2, 4-7 are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least some of the steps in fig. 2, 4-7 may include multiple sub-steps or multiple stages that are not necessarily performed at the same time, but may be performed at different times, and the order of performance of the sub-steps or stages is not necessarily sequential, but may be performed in turn or alternating with other steps or at least some of the sub-steps or stages of other steps.
Fig. 8 is a block diagram of a camera calibration apparatus according to an embodiment. As shown in fig. 8, the camera calibration device includes an image capturing module 802 and a calibration processing module 804. Wherein:
the image shooting module 802 is configured to sequentially obtain different aperture values of a main camera in the camera module, and shoot the calibration plate with the different aperture values through the camera module to obtain a set of calibration images corresponding to each aperture value, where the set of calibration images includes calibration images shot by the main camera and at least two auxiliary cameras in the camera module respectively;
and the calibration processing module 804 is configured to perform calibration processing on the camera module according to a group of calibration images corresponding to each aperture value, so as to obtain a camera calibration parameter corresponding to the camera module under each aperture value.
In one embodiment, the image capturing module 802 may be further configured to capture calibration plates at least three angles at each aperture value of the camera module.
In one embodiment, the image capturing module 802 may be further configured to obtain a minimum aperture value and a maximum aperture value of the main camera; determining a target aperture value according to the minimum aperture value and the maximum aperture value; shoot the calibration plate through the camera module with different target aperture values.
In one embodiment, the image capture module 802 may also be configured to adjust the calibration plate according to at least three angles; after the angle to the calibration plate is adjusted at every turn, the calibration plate is shot through the camera module by the first target aperture value and the second target aperture value.
In an embodiment, the calibration processing module 804 may be further configured to obtain calibration images corresponding to the cameras in the camera module according to a group of calibration images corresponding to each aperture value; performing monocular calibration processing on each camera according to the calibration image corresponding to each camera respectively to obtain a monocular calibration parameter corresponding to each aperture value; performing binocular calibration processing on the main camera and the first auxiliary camera according to the monocular calibration parameters corresponding to the main camera and the first auxiliary camera under each aperture value to obtain first binocular calibration parameters corresponding to each aperture value; and carrying out binocular calibration processing on the main camera and the second auxiliary camera according to the monocular calibration parameters corresponding to the main camera and the second auxiliary camera under each aperture value to obtain second binocular calibration parameters corresponding to each aperture value.
In an embodiment, the provided camera calibration apparatus may further include an image processing module 806, where the image processing module 806 is configured to obtain a collection aperture value of the main camera when the initial image is collected by the camera module; acquiring a first target calibration parameter corresponding to the acquired aperture value from the camera calibration parameter corresponding to each aperture value of the camera module; and correcting the initial image according to the first target calibration parameter, and outputting the obtained target image.
In one embodiment, the image processing module 806 may be further configured to obtain a collection aperture value of the main camera when it is determined that the target application belongs to the first type of application; acquiring a first target calibration parameter corresponding to the acquired aperture value from camera calibration parameters corresponding to the camera module under each aperture value; and correcting the initial image according to the first target calibration parameter, and outputting the obtained target image.
In an embodiment, the image processing module 806 may be further configured to, when it is determined that the target application program belongs to the second type of application program, obtain a preset second target calibration parameter, perform correction processing on the initial image according to the second target calibration parameter, and output the obtained target image.
The camera calibration device provided by the embodiment of the application is used for acquiring different aperture values of a main camera in a camera module in sequence, the calibration plate is shot by the camera module according to different aperture values, a group of calibration images corresponding to each aperture value is obtained, wherein one group of calibration images comprise calibration images respectively shot by the main camera and at least two auxiliary cameras, the camera module is calibrated according to a group of calibration images corresponding to each aperture value, the camera calibration parameters corresponding to the camera module under each aperture value are obtained, the situation that the camera with a plurality of aperture values is calibrated only once is avoided, and the camera calibration accuracy is improved.
The division of the modules in the camera calibration device is merely used for illustration, and in other embodiments, the camera calibration device may be divided into different modules as needed to complete all or part of the functions of the camera calibration device.
Fig. 9 is a schematic diagram of an internal structure of an electronic device in one embodiment. As shown in fig. 9, the electronic device includes a processor and a memory connected by a system bus. The electronic equipment further comprises a camera module. Wherein, the processor is used for providing calculation and control capability and supporting the operation of the whole electronic equipment. The memory may include a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The computer program is executable by a processor for implementing a camera calibration method provided in the following embodiments. The internal memory provides a cached execution environment for the operating system computer programs in the non-volatile storage medium. The electronic device may be a mobile phone, a tablet computer, or a personal digital assistant or a wearable device, etc.
The implementation of each module in the camera-calibration apparatus provided in the embodiments of the present application may be in the form of a computer program. The computer program may be run on a terminal or a server. The program modules constituted by the computer program may be stored on the memory of the terminal or the server. Which when executed by a processor, performs the steps of the method described in the embodiments of the present application.
The embodiment of the application also provides the electronic equipment. The electronic device includes therein an Image Processing circuit, which may be implemented using hardware and/or software components, and may include various Processing units defining an ISP (Image Signal Processing) pipeline. FIG. 10 is a schematic diagram of an image processing circuit in one embodiment. As shown in fig. 10, for convenience of explanation, only aspects of the image processing technology related to the embodiments of the present application are shown.
As shown in fig. 10, the image processing circuit includes an ISP processor 1040 and control logic 1050. The image data captured by the imaging device 1010 is first processed by the ISP processor 1040, and the ISP processor 1040 analyzes the image data to capture image statistics that may be used to determine and/or control one or more parameters of the imaging device 1010. The imaging device 1010 may include a camera having one or more lenses 1012 and an image sensor 1014. The image sensor 1014 may include an array of color filters (e.g., Bayer filters), and the image sensor 1014 may acquire light intensity and wavelength information captured with each imaging pixel of the image sensor 1014 and provide a set of raw image data that may be processed by the ISP processor 1040. The sensor 1020 (e.g., a gyroscope) may provide parameters of the acquired image processing (e.g., anti-shake parameters) to the ISP processor 1040 based on the type of sensor 1020 interface. The sensor 1020 interface may utilize an SMIA (Standard Mobile Imaging Architecture) interface, other serial or parallel camera interfaces, or a combination of the above.
In addition, the image sensor 1014 may also send raw image data to the sensor 1020, the sensor 1020 may provide the raw image data to the ISP processor 1040 based on the type of interface of the sensor 1020, or the sensor 1020 may store the raw image data in the image memory 1030.
The ISP processor 1040 processes the raw image data pixel by pixel in a variety of formats. For example, each image pixel may have a bit depth of 8, 10, 12, or 14 bits, and ISP processor 1040 may perform one or more image processing operations on the raw image data, gathering statistical information about the image data. Wherein the image processing operations may be performed with the same or different bit depth precision.
Upon receiving raw image data from image sensor 1014 interface or from sensor 1020 interface or from image memory 1030, ISP processor 1040 may perform one or more image processing operations, such as temporal filtering. The processed image data may be sent to image memory 1030 for additional processing before being displayed. ISP processor 1040 receives processed data from image memory 1030 and performs image data processing on the processed data in the raw domain and in the RGB and YCbCr color spaces. The image data processed by ISP processor 1040 may be output to display 1070 for viewing by a user and/or further processed by a Graphics Processing Unit (GPU). Further, the output of ISP processor 1040 may also be sent to image memory 1030, and display 1070 may read image data from image memory 1030. In one embodiment, image memory 1030 may be configured to implement one or more frame buffers. Further, the output of the ISP processor 1040 may be transmitted to the encoder/decoder 1060 for encoding/decoding the image data. The encoded image data may be saved and decompressed before being displayed on a display 1070 device. The encoder/decoder 1060 may be implemented by a CPU or GPU or coprocessor.
The statistics determined by the ISP processor 1040 may be sent to the control logic 1050 unit. For example, the statistical data may include image sensor 1014 statistics such as auto-exposure, auto-white balance, auto-focus, flicker detection, black level compensation, lens 1012 shading correction, and the like. Control logic 1050 may include a processor and/or microcontroller that executes one or more routines (e.g., firmware) that may determine control parameters of imaging device 1010 and ISP processor 1040 based on the received statistical data. For example, the control parameters of the imaging device 1010 may include sensor 1020 control parameters (e.g., gain, integration time for exposure control, anti-shake parameters, etc.), camera flash control parameters, lens 1012 control parameters (e.g., focal length for focusing or zooming), or a combination of these parameters. The ISP control parameters may include gain levels and color correction matrices for automatic white balance and color adjustment (e.g., during RGB processing), and lens 1012 shading correction parameters.
In an embodiment of the present application, the image processing circuit may include at least three imaging devices (cameras) 1010, and the above-described camera calibration method may be implemented using the image processing technique in fig. 10.
The embodiment of the application also provides a computer readable storage medium. One or more non-transitory computer-readable storage media containing computer-executable instructions that, when executed by one or more processors, cause the processors to perform the steps of the camera calibration method.
A computer program product comprising instructions which, when run on a computer, cause the computer to perform a camera calibration method.
Any reference to memory, storage, database, or other medium used by embodiments of the present application may include non-volatile and/or volatile memory. Suitable non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM), which acts as external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), Enhanced SDRAM (ESDRAM), synchronous Link (Synchlink) DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and bus dynamic RAM (RDRAM).
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the present application. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (10)
1. A camera calibration method, comprising:
sequentially acquiring different aperture values of a main camera in a camera module, acquiring a minimum aperture value and a maximum aperture value of the main camera, determining a target aperture value according to the minimum aperture value and the maximum aperture value, and shooting a calibration plate by the camera module at different target aperture values to obtain a group of calibration images corresponding to each target aperture value, wherein the group of calibration images comprise calibration images respectively shot by the main camera and at least two auxiliary cameras in the camera module;
and calibrating the camera module according to a group of calibration images corresponding to each target aperture value to obtain camera calibration parameters corresponding to each target aperture value of the camera module.
2. The method of claim 1, wherein said capturing a calibration plate with different target aperture values by said camera module comprises:
and shooting the calibration plate at least three angles under each target aperture value of the camera module.
3. The method of claim 1, wherein the target aperture value comprises a first target aperture value and a second target aperture value;
pass through the camera module is with different target aperture values the calibration plate is shot, include:
adjusting the calibration plate according to at least three angles;
after the angle of the calibration plate is adjusted every time, the calibration plate is shot by the first target aperture value and the second target aperture value through the camera module.
4. The method of claim 1, wherein the auxiliary cameras comprise a first auxiliary camera and a second auxiliary camera;
the calibrating the camera module according to a set of calibration images corresponding to each target aperture value to obtain the camera calibration parameters corresponding to the camera module at each target aperture value includes:
obtaining a calibration image corresponding to each camera in the camera module according to a group of calibration images corresponding to each target aperture value;
performing monocular calibration processing on each camera according to the calibration image corresponding to each camera to obtain a monocular calibration parameter corresponding to each target aperture value;
performing binocular calibration processing on the main camera and the first auxiliary camera according to the monocular calibration parameters corresponding to the main camera and the first auxiliary camera under each target aperture value to obtain a first binocular calibration parameter corresponding to each target aperture value;
and carrying out binocular calibration processing on the main camera and the second auxiliary camera according to the monocular calibration parameters corresponding to the main camera and the second auxiliary camera under each target aperture value to obtain second binocular calibration parameters corresponding to each target aperture value.
5. The method according to any one of claims 1 to 4, further comprising:
acquiring an acquisition aperture value of the main camera when an initial image is acquired through the camera module;
acquiring a first target calibration parameter corresponding to the acquired aperture value from a camera calibration parameter corresponding to each target aperture value of the camera module;
and correcting the initial image according to the first target calibration parameter, and outputting the obtained target image.
6. The method of claim 5, wherein the obtaining the acquisition aperture value of the main camera while acquiring the initial image through the camera module comprises:
when an initial image is collected through the camera module, a target application program initiating a collection instruction for collecting the initial image is obtained;
when the target application program is judged to belong to a first type of application program, acquiring the acquisition aperture value of the main camera;
the method further comprises the following steps:
and when the target application program is judged to belong to the second type of application program, acquiring a preset second target calibration parameter, correcting the initial image according to the second target calibration parameter, and outputting the obtained target image.
7. A camera calibration device is characterized by comprising:
the image shooting module is used for sequentially obtaining different aperture values of a main camera in a camera module, obtaining a minimum aperture value and a maximum aperture value of the main camera, determining a target aperture value according to the minimum aperture value and the maximum aperture value, and shooting a calibration plate by the camera module according to different target aperture values to obtain a group of calibration images corresponding to each target aperture value, wherein the group of calibration images comprise calibration images respectively shot by the main camera and at least two auxiliary cameras in the camera module;
and the calibration processing module is used for calibrating the camera module according to a group of calibration images corresponding to each target aperture value to obtain camera calibration parameters corresponding to each target aperture value of the camera module.
8. The apparatus of claim 7,
the image shooting module is also used for shooting the calibration plates at least three angles under each target aperture value of the camera module.
9. An electronic device comprising a camera module, a memory and a processor, the memory having stored therein a computer program that, when executed by the processor, causes the processor to perform the steps of the camera calibration method according to any one of claims 1 to 6.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811452677.3A CN109697737B (en) | 2018-11-30 | 2018-11-30 | Camera calibration method, device, electronic equipment and computer-readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811452677.3A CN109697737B (en) | 2018-11-30 | 2018-11-30 | Camera calibration method, device, electronic equipment and computer-readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109697737A CN109697737A (en) | 2019-04-30 |
| CN109697737B true CN109697737B (en) | 2021-08-17 |
Family
ID=66230213
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811452677.3A Active CN109697737B (en) | 2018-11-30 | 2018-11-30 | Camera calibration method, device, electronic equipment and computer-readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109697737B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110264524B (en) * | 2019-05-24 | 2023-07-21 | 联想（上海）信息技术有限公司 | Calibration method, device, system and storage medium |
| CN112348899A (en) * | 2019-08-07 | 2021-02-09 | 虹软科技股份有限公司 | Calibration parameter obtaining method and device, processor and electronic equipment |
| CN110675458B (en) * | 2019-10-08 | 2023-03-31 | Oppo广东移动通信有限公司 | Method and device for calibrating camera and storage medium |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106204574B (en) * | 2016-07-07 | 2018-12-21 | 兰州理工大学 | Camera pose self-calibrating method based on objective plane motion feature |
| EP3327481B1 (en) * | 2016-11-28 | 2019-07-03 | InterDigital CE Patent Holdings | Method for modelling an imaging device, corresponding computer program product and computer-readable carrier medium |
| CN108055471B (en) * | 2017-11-16 | 2020-05-15 | 深圳市维海德技术股份有限公司 | Aperture correction method and device |
| CN107958469A (en) * | 2017-12-28 | 2018-04-24 | 北京安云世纪科技有限公司 | A kind of scaling method of dual camera, device, system and mobile terminal |
| CN108668078B (en) * | 2018-04-28 | 2019-07-30 | Oppo广东移动通信有限公司 | Image processing method, device, computer readable storage medium and electronic equipment |
-
2018
- 2018-11-30 CN CN201811452677.3A patent/CN109697737B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN109697737A (en) | 2019-04-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107948519B (en) | Image processing method, device and equipment | |
| CN109767467B (en) | Image processing method, image processing device, electronic equipment and computer readable storage medium | |
| CN111246089B (en) | Jitter compensation method and apparatus, electronic device, computer-readable storage medium | |
| CN110166695B (en) | Camera anti-shake method and device, electronic equipment and computer readable storage medium | |
| CN108055452B (en) | Image processing method, device and equipment | |
| CN107223330B (en) | Depth information acquisition method and device and image acquisition equipment | |
| CN109712192B (en) | Camera module calibration method and device, electronic equipment and computer readable storage medium | |
| CN109685853B (en) | Image processing method, image processing device, electronic equipment and computer readable storage medium | |
| CN109559352B (en) | Camera calibration method, device, electronic equipment and computer-readable storage medium | |
| CN109600548B (en) | Image processing method and device, electronic equipment and computer readable storage medium | |
| CN110290323B (en) | Image processing method, image processing device, electronic equipment and computer readable storage medium | |
| CN110278360B (en) | Image processing method and device, electronic equipment and computer readable storage medium | |
| CN112004029B (en) | Exposure processing method, exposure processing device, electronic apparatus, and computer-readable storage medium | |
| CN111246100B (en) | Anti-shake parameter calibration method and device and electronic equipment | |
| CN109584312B (en) | Camera calibration method, device, electronic equipment and computer-readable storage medium | |
| CN109559353B (en) | Camera module calibration method and device, electronic equipment and computer readable storage medium | |
| CN107704798B (en) | Image blurring method and device, computer readable storage medium and computer device | |
| CN109697737B (en) | Camera calibration method, device, electronic equipment and computer-readable storage medium | |
| CN109963080B (en) | Image acquisition method and device, electronic equipment and computer storage medium | |
| CN109598763B (en) | Camera calibration method, device, electronic equipment and computer-readable storage medium | |
| US20220180484A1 (en) | Image processing method, electronic device, and computer-readable storage medium | |
| CN109951641B (en) | Image shooting method and device, electronic equipment and computer readable storage medium | |
| CN109584311B (en) | Camera calibration method, device, electronic equipment and computer-readable storage medium | |
| CN109598764A (en) | Camera calibration method and device, electronic equipment, computer readable storage medium | |
| CN112087571A (en) | Image acquisition method and device, electronic equipment and computer readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |