CN111343360B - Correction parameter obtaining method - Google Patents

Correction parameter obtaining method Download PDF

Info

Publication number
CN111343360B
CN111343360B CN201811544026.7A CN201811544026A CN111343360B CN 111343360 B CN111343360 B CN 111343360B CN 201811544026 A CN201811544026 A CN 201811544026A CN 111343360 B CN111343360 B CN 111343360B
Authority
CN
China
Prior art keywords
lens
corrected
image
points
lenses
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201811544026.7A
Other languages
Chinese (zh)
Other versions
CN111343360A (en
Inventor
任健
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hangzhou Hikvision Digital Technology Co Ltd
Original Assignee
Hangzhou Hikvision Digital Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hangzhou Hikvision Digital Technology Co Ltd filed Critical Hangzhou Hikvision Digital Technology Co Ltd
Priority to CN201811544026.7A priority Critical patent/CN111343360B/en
Publication of CN111343360A publication Critical patent/CN111343360A/en
Application granted granted Critical
Publication of CN111343360B publication Critical patent/CN111343360B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/80—Camera processing pipelines; Components thereof
    • H04N23/81—Camera processing pipelines; Components thereof for suppressing or minimising disturbance in the image signal generation
    • G06T5/80—
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/50—Constructional details
    • H04N23/55—Optical parts specially adapted for electronic image sensors; Mounting thereof
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30204—Marker
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30244—Camera pose

Abstract

The embodiment of the invention provides a correction parameter obtaining method, which comprises the following steps: obtaining a first image when the lens in the lens group to be corrected respectively shoots target paper, wherein the lens in the lens group to be corrected is as follows: the method comprises the steps that lenses with overlapped focal length sections exist in multi-lens equipment, pixel points in all first images are the same in size, the focal length and the object distance are the same when the first images are obtained by shooting through lenses in a lens group to be corrected, and a first calibration point is arranged on target paper; determining the coordinate position of an imaging point of a first calibration point in a first image corresponding to each lens in a lens group to be corrected; and calculating translation correction parameters of each first lens relative to the second lens based on the coordinate position of the imaging point of the first calibration point in each first image. So as to obtain correction parameters for correcting images acquired by different lenses in the multi-lens device.

Description

Correction parameter obtaining method
Technical Field
The present invention relates to the field of parameter calibration technologies, and in particular, to a method and an apparatus for obtaining a calibration parameter, and an electronic device.
Background
At present, in order to extend the range of focal lengths supported by image capturing devices such as video cameras and cameras, a multi-lens setting mode is mostly adopted, so that the set lenses supporting different focal lengths are used to capture images of different focal lengths, and the extension of the range of focal lengths supported by the image capturing devices is achieved. The image capturing device with multiple lenses may be referred to as a multi-lens device.
In the lens installation process of the multi-lens device, the problem of shooting angle difference between different lenses is inevitable. Due to the difference in the shooting angles between different lenses in a multi-lens device, when the multi-lens device switches lenses to capture images for viewing by a user, there is a difference between the images captured by the different lenses, for example: a shot object in a scene is imaged at the upper left corner of an image collected by a lens before switching, the shot object is imaged at the lower right corner of the image collected by the lens after switching, and poor experience can be brought to a user due to the difference between the images.
Disclosure of Invention
An object of the embodiments of the present invention is to provide a method and an apparatus for obtaining correction parameters, and an electronic device, so as to obtain correction parameters for correcting images acquired by different lenses in the multi-lens device. The specific technical scheme is as follows:
in one aspect, an embodiment of the present invention provides a method for obtaining a correction parameter, where the method includes:
obtaining first images of respective shooting target papers of lenses in a lens group to be corrected, wherein the lenses in the lens group to be corrected are as follows: the method comprises the steps that lenses with overlapped focal length sections exist in multi-lens equipment, pixel points in all first images are the same in size, the focal length and the object distance are the same when the first images are obtained through shooting by the lenses in a lens group to be corrected, and first calibration points are arranged on target paper;
determining the coordinate position of the imaging point of the first calibration point in the first image aiming at the first image corresponding to each lens in the lens group to be corrected;
calculating and obtaining a translation correction parameter of each first lens relative to a second lens based on the coordinate position of the imaging point of the first calibration point in each first image, wherein the second lens is: the first lens is a lens in the lens group to be corrected: and the lenses except the second lens in the lens group to be corrected.
Optionally, after the step of calculating a translational correction parameter of each first lens with respect to the second lens based on the coordinate position of the imaging point of the first calibration point in each first image, the method further includes:
obtaining a focal length and an object distance when a first image is obtained by shooting through the lenses in the lens group to be corrected, wherein the focal length and the object distance are used as a first focal length and a first object distance;
and storing the corresponding relation between the translation correction parameter of the first lens relative to the second lens and the first focal length and the first object distance for each first lens in the lens group to be corrected.
Optionally, the step of calculating a translational correction parameter of each first lens with respect to the second lens based on the coordinate position of the imaging point of the first calibration point in each first image includes:
selecting a lens from the lenses in the lens group to be corrected as a second lens;
calculating the translational correction parameters of each first lens relative to the second lens by adopting the following steps:
determining a position coordinate of an imaging point of the first calibration point in a first image corresponding to a first target lens as a first position coordinate, wherein the first target lens is one of the first lenses;
calculating to obtain a horizontal difference value and a horizontal correction direction between the first position coordinate and the second position coordinate, and a vertical difference value and a vertical correction direction, wherein the second position coordinate is: determining the position coordinate of an imaging point of the first calibration point in a first image corresponding to the second lens;
and determining the calculated horizontal difference value and horizontal correction direction between the first position coordinate and the second position coordinate, and the calculated vertical difference value and vertical correction direction as the translation correction parameters of the first target lens relative to the second lens.
Optionally, the target paper is further provided with at least two second calibration points and at least two third calibration points, and a connecting line of the at least two second calibration points is perpendicular to a connecting line of the at least two third calibration points and intersects with the first calibration point;
before the step of determining the calculated horizontal difference and horizontal correction direction, and vertical difference and vertical correction direction between the first position coordinate and the second position coordinate as the translation correction parameters of the first target lens relative to the second lens, the method further includes:
determining position coordinates of imaging points of the at least two second calibration points and imaging points of the at least two third calibration points in the first image corresponding to the first target lens;
determining position coordinates of imaging points of the at least two second calibration points and imaging points of the at least two third calibration points in the first image corresponding to the second lens;
determining a cropping size and a cropping position corresponding to the first target lens based on position coordinates of imaging points of the at least two second calibration points and the at least two third calibration points in the first image corresponding to the first target lens, position coordinates of imaging points of the at least two second calibration points and the at least two third calibration points in the first image corresponding to the second lens, and a preset translation parameter calculation formula;
the step of determining the calculated horizontal difference and horizontal correction direction between the first position coordinate and the second position coordinate, and the calculated vertical difference and vertical correction direction as the translational correction parameter of the first target lens relative to the second lens includes:
and determining the calculated horizontal difference value and horizontal correction direction between the first position coordinate and the second position coordinate, the calculated vertical difference value and vertical correction direction, and the cutting size and cutting position corresponding to the first target lens as the translation correction parameters of the first target lens relative to the second lens.
Optionally, before the step of obtaining the first image of the target paper taken by each lens of the lens group to be corrected, the method further comprises:
acquiring second images of the lenses in the lens group to be corrected when the lenses respectively shoot target paper, wherein the target paper is also provided with at least two second calibration points and at least two third calibration points, the connecting line of the at least two second calibration points is perpendicular to the connecting line of the at least two third calibration points and intersects with the first calibration point, and the focal lengths of the lenses in the lens group to be corrected when the lenses shoot the second images are the same;
determining position coordinates of an imaging point of the first calibration point, imaging points of at least two second calibration points and imaging points of at least two third calibration points in the second image corresponding to each lens in the lens group to be corrected;
for a second image corresponding to each lens in the lens group to be corrected, calculating to obtain a rotation error of the lens corresponding to the second image based on an imaging point of the first calibration point, respective imaging points of at least two second calibration points, respective position coordinates of respective imaging points of at least two third calibration points, and a preset rotation error calculation formula in the second image;
based on the rotation error of each lens in the lens group to be corrected, calculating to obtain the rotation correction parameters of each third lens relative to the fourth lens, wherein the fourth lens is: one lens of the lens group to be corrected, the third lens is: and the lens groups to be corrected except the fourth lens.
Optionally, the first calibration point is a bulls-eye of the target paper, at least two of the second calibration points are respectively located at two sides of the first calibration point, and at least two of the third calibration points are respectively located at two sides of the first calibration point.
Optionally, after the step of calculating rotation correction parameters of the third lenses relative to the fourth lenses based on the rotation error of each lens in the lens group to be corrected, the method further includes:
obtaining a focal length when a second image is obtained by shooting through the lenses in the lens group to be corrected, and taking the focal length as a second focal length;
and storing the corresponding relation between the rotation correction parameter of the third lens relative to the fourth lens and the second focal length for each third lens in the lens group to be corrected.
Optionally, when the number of the second calibration points is two, and the number of the third calibration points is two;
the preset rotation error calculation formula is as follows:
Figure BDA0001908891680000041
wherein α (N) denotes a rotation error of a lens corresponding to the second image at a focal length of N, and (D)x,Dy) And (F)x,Fy) Position coordinates respectively identifying imaging points of two of the second calibration points in the second image, the (C)x,Cy) And (E)x,Ey) Position coordinates respectively identifying imaging points of two of the third calibration points in the second image, the (G)x,Gy) Identifying position coordinates of an imaged point of the first calibration point in the second imageAnd N is a positive number.
Optionally, the step of calculating rotation correction parameters of each third lens relative to the fourth lens based on the rotation error of each lens in the lens group to be corrected includes:
selecting a lens from the lenses in the lens group to be corrected as a fourth lens;
calculating the difference value of the rotation error of the third lens and the rotation error of the fourth lens as the rotation difference value of the third lens aiming at each third lens;
and determining the calculated rotation difference value of the third lens as a rotation correction parameter of the third lens relative to the fourth lens for each third lens.
Optionally, before the step of determining, for each third lens, the calculated difference of the third lens as a rotation correction parameter of the third lens relative to the fourth lens, the method further includes:
for each third lens, determining a cutting position corresponding to the third lens and a cutting size corresponding to the third lens based on a central point of a second image corresponding to the third lens, a rotation difference value of the third lens and a resolution of the third lens;
the step of determining the calculated difference between the third lens and each third lens as the rotation correction parameter of the third lens relative to the fourth lens includes:
and determining the calculated difference value of the third lens, the cutting position corresponding to the third lens and the cutting size corresponding to the third lens as the rotation correction parameters of the third lens relative to the fourth lens for each third lens.
On the other hand, an embodiment of the present invention provides a correction parameter obtaining apparatus, where the apparatus includes:
the first obtaining module is used for obtaining a first image when the lenses in the lens group to be corrected respectively shoot target paper, wherein the lenses in the lens group to be corrected are as follows: lenses with overlapped focal length sections exist in the multi-lens device, pixel points in the first images are the same in size, the focal length and the object distance are the same when the first images are obtained by shooting through the lenses in the lens group to be corrected, and the target paper is provided with a first calibration point;
the first determining module is used for determining the coordinate position of an imaging point of the first calibration point in a first image corresponding to each lens in the lens group to be corrected;
a first calculating module, configured to calculate, based on a coordinate position of an imaging point of the first calibration point in each first image, a translational correction parameter of each first lens with respect to a second lens, where the second lens is: the first lens is a lens in the lens group to be corrected: and the lenses except the second lens in the lens group to be corrected.
Optionally, the apparatus further comprises:
a second obtaining module, configured to obtain a focal length and an object distance when the lenses in the lens group to be corrected take a first image as a first focal length and a first object distance after calculating a translational correction parameter of each first lens relative to the second lens based on the coordinate position of the imaging point of the first calibration point in each first image;
the first storage module is used for storing the corresponding relation between the translation correction parameter of the first lens relative to the second lens and the first focal length and the first object distance for each first lens in the lens group to be corrected.
Optionally, the first computing module includes:
the first selection unit is used for selecting a lens from the lenses in the lens group to be corrected as a second lens;
the first calculation unit is used for calculating and obtaining the translation correction parameters of each first lens relative to the second lens by adopting the following steps: the method comprises the following steps:
a first determining subunit, configured to determine, as a first position coordinate, a position coordinate of an imaging point of the first calibration point in a first image corresponding to a first target lens, where the first target lens is a lens of the first lens;
a first calculating subunit, configured to calculate a horizontal difference value and a horizontal correction direction between the first position coordinate and a second position coordinate, and a vertical difference value and a vertical correction direction, where the second position coordinate is: determining the position coordinate of an imaging point of the first calibration point in a first image corresponding to the second lens;
and the second determining subunit is configured to determine, as the translation correction parameter of the first target lens relative to the second lens, the calculated horizontal difference value and horizontal correction direction between the first position coordinate and the second position coordinate, and the calculated vertical difference value and vertical correction direction.
Optionally, the target paper is further provided with at least two second calibration points and at least two third calibration points, and a connecting line of the at least two second calibration points is perpendicular to a connecting line of the at least two third calibration points and intersects with the first calibration point;
the first computing module further comprises:
a third determining subunit, configured to determine, before determining, as the translational correction parameter of the first target lens relative to the second lens, the horizontal difference value and the horizontal correction direction between the first position coordinate and the second position coordinate, and the vertical difference value and the vertical correction direction obtained by the calculation, position coordinates of an imaging point of the at least two second calibration points and an imaging point of the at least two third calibration points in the first image corresponding to the first target lens; determining position coordinates of imaging points of the at least two second calibration points and imaging points of the at least two third calibration points in the first image corresponding to the second lens;
a fourth determining subunit, configured to determine a cropping size and a cropping position corresponding to the first target lens, based on position coordinates of the imaging points of the at least two second calibration points and the at least two third calibration points in the first image corresponding to the first target lens, the position coordinates of the imaging points of the at least two second calibration points and the imaging points of the at least two third calibration points in the first image corresponding to the second lens, and a preset translation parameter calculation formula;
the second determining subunit is specifically configured to
And determining the calculated horizontal difference value and horizontal correction direction between the first position coordinate and the second position coordinate, the calculated vertical difference value and vertical correction direction, and the cutting size and cutting position corresponding to the first target lens as the translation correction parameters of the first target lens relative to the second lens.
Optionally, the apparatus further comprises:
a third obtaining module, configured to obtain second images when the lenses in the lens group to be corrected respectively shoot target paper before obtaining first images when the lenses in the lens group to be corrected respectively shoot the target paper, where the target paper is further provided with at least two second calibration points and at least two third calibration points, a connection line of the at least two second calibration points is perpendicular to a connection line of the at least two third calibration points and intersects with the first calibration point, and focal lengths of the lenses in the lens group to be corrected when the lenses shoot the second images are the same;
a second determining module, configured to determine, for a second image corresponding to each lens of the lens group to be corrected, position coordinates of an imaging point of the first calibration point, imaging points of at least two second calibration points, and imaging points of at least two third calibration points in the second image;
a second calculating module, configured to calculate, for a second image corresponding to each lens in the lens group to be corrected, a rotation error of the lens corresponding to the second image based on an imaging point of the first calibration point in the second image, an imaging point of each of at least two second calibration points, a position coordinate of an imaging point of each of at least two third calibration points, and a preset rotation error calculation formula;
a third calculating module, configured to calculate, based on a rotation error of each lens in the lens group to be corrected, a rotation correction parameter of each third lens relative to a fourth lens, where the fourth lens is: one lens of the lens group to be corrected, the third lens is: and the lens groups to be corrected except the fourth lens.
Optionally, the first calibration point is a bulls-eye of the target paper, at least two of the second calibration points are respectively located at two sides of the first calibration point, and at least two of the third calibration points are respectively located at two sides of the first calibration point.
Optionally, the apparatus further comprises:
a fourth obtaining module, configured to obtain, after the rotation correction parameter of each third lens with respect to the fourth lens is calculated and obtained based on the rotation error of each lens in the lens group to be corrected, a focal length when the lens in the lens group to be corrected obtains a second image by shooting, as the second focal length;
and the second storage module is used for storing the corresponding relation between the rotation correction parameter of the third lens relative to the fourth lens and the second focal length aiming at each third lens in the lens group to be corrected.
Optionally, when the number of the second calibration points is two, and the number of the third calibration points is two;
the preset rotation error calculation formula is as follows:
Figure BDA0001908891680000081
wherein α (N) denotes a rotation error of a lens corresponding to the second image in a case where the focal length is N, and (D)x,Dy) And (F)x,Fy) Position coordinates respectively identifying imaging points of two of the second calibration points in the second image, the (C)x,Cy) And (E)x,Ey) Position coordinates respectively identifying imaging points of two of the third calibration points in the second image, the (G)x,Gy) Identifying an imaged point of the first calibration point in the second imagePosition coordinates, N is a positive number.
Optionally, the third computing module comprises:
the second selection unit is used for selecting one lens from the lenses in the lens group to be corrected as a fourth lens;
a second calculation unit, configured to calculate, for each third lens, a difference between a rotation error of the third lens and a rotation error of the fourth lens, as a rotation difference of the third lens;
and the first determining unit is used for determining the calculated rotation difference value of the third lens as a rotation correction parameter of the third lens relative to the fourth lens for each third lens.
Optionally, the third computing module further comprises:
a second determining unit, configured to determine, for each third lens, a clipping position corresponding to the third lens and a clipping size corresponding to the third lens based on a central point of a second image corresponding to the third lens, a rotation difference value of the third lens, and a resolution of the third lens before determining, for each third lens, the calculated difference value of the third lens as a rotation correction parameter of the third lens with respect to the fourth lens;
the first determining unit is specifically configured to:
and determining the calculated difference value of the third lens, the cutting position corresponding to the third lens and the cutting size corresponding to the third lens as the rotation correction parameters of the third lens relative to the fourth lens for each third lens.
On the other hand, the embodiment of the invention provides electronic equipment, which comprises a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory complete mutual communication through the communication bus;
a memory for storing a computer program;
the processor is configured to implement any of the steps of the correction parameter obtaining method provided in the embodiments of the present invention when executing the computer program stored in the memory.
In another aspect, an embodiment of the present invention provides a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the steps of the correction parameter obtaining method provided in the embodiment of the present invention are implemented.
In the technical scheme provided by the embodiment of the invention, first images of the lens group to be corrected when the lens respectively shoots target paper are obtained, wherein the lenses in the lens group to be corrected are as follows: the method comprises the steps that lenses with overlapped focal length sections exist in multi-lens equipment, pixel points in all first images are the same in size, the focal length and the object distance are the same when the first images are obtained by shooting through lenses in a lens group to be corrected, and a first calibration point is arranged on target paper; determining the coordinate position of an imaging point of a first calibration point in a first image corresponding to each lens in a lens group to be corrected; calculating translation correction parameters of each first lens relative to a second lens based on the coordinate position of an imaging point of a first calibration point in each first image, wherein the second lens is as follows: a lens of the lens group to be corrected, the first lens being: and the lens except the second lens in the lens group to be corrected.
Therefore, in the embodiment of the present invention, the position coordinates of the imaging point of the first calibration point set on the target paper in the first image may be used to determine the difference of the shooting angles between the lenses in the lens group to be corrected, and based on the difference of the shooting angles between the lenses in the lens group to be corrected, the first lens, which is the other lens except for the lens to be corrected, and the translation correction parameter relative to the second lens, which is the other lens to be corrected, are obtained. In the subsequent image correction process, the image acquired by the first lens is corrected by using the translation correction parameter of the first lens relative to the second lens, so that the obtained corrected image and the image acquired by the second lens have the same shooting angle, that is, the positions of all points in the shot scene in the obtained corrected image are the same as the positions in the image acquired by the second lens. In addition, in the embodiment of the invention, the first calibration point in the target paper can occupy a complete pixel point in the first image, so that the correction parameter determined based on the first image containing the target paper can be more accurate, and the subsequent image corrected by using the correction parameter has better effect. Of course, it is not necessary for any product or method of practicing the invention to achieve all of the above-described advantages at the same time.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
FIG. 1A is a schematic diagram of a positional relationship between two lenses with a mounting distance in a multi-lens apparatus;
FIG. 1B is a schematic diagram of a positional relationship of image content represented by images captured by two lenses having a mounting distance in a multi-lens device;
FIG. 2 is a schematic flow chart of a method for obtaining calibration parameters according to an embodiment of the present invention;
FIG. 3 is a schematic view of a target paper according to an embodiment of the present invention;
FIGS. 4A and 4B are schematic diagrams of a first image (including an image point of a first calibration point) collected by a lens of a lens group to be calibrated;
FIG. 4C is a schematic diagram illustrating a position relationship between the second first image obtained after the translation of the second first image shown in FIGS. 4A and 4B and the second first image before the translation;
FIG. 5 is a schematic diagram showing a positional relationship between image contents represented by images captured by two lenses having a relative rotation error;
FIG. 6 is a schematic diagram of a rotation correction parameter obtaining process according to an embodiment of the present invention;
FIG. 7 is a schematic structural diagram of a calibration parameter obtaining apparatus according to an embodiment of the present invention;
fig. 8 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It can be understood that, in order to better satisfy the user's requirement, that is, to extend the range of the focal length when the image capturing device supports capturing an image, a plurality of lenses respectively supporting different focal lengths may be provided in an image capturing device at present, and the image capturing device may automatically switch the lenses according to actual conditions or based on user instructions, so as to capture the target scene monitored by using different lenses, thereby ensuring the capturing effect, that is, the image effect of the captured image, for example, ensuring that the sharpness value of the captured image is not lower than a preset sharpness value, and that the capturing regions corresponding to the two switched lenses have an overlapping region. In the embodiment of the present invention, an image capturing device provided with a plurality of lenses respectively supporting different focal lengths may be referred to as a multi-lens device. The resolutions of the plurality of lenses provided by the multi-lens apparatus may all be the same.
The multi-lens device can shoot a scene by using different lenses supporting different focal lengths arranged in the multi-lens device, and the acquisition regions corresponding to the lenses in the multi-lens device have overlapping regions. On the premise of ignoring the slight difference, it can be considered that different lenses arranged inside the multi-lens device are equal in distance from the target in the shot scene, that is, the object distance is equal. The minute difference is a difference between a lens provided inside the multi-lens apparatus and a lens of another external apparatus.
In order to ensure the quality of the shot image, for example, the sharpness value is not lower than the preset sharpness value, when the lenses are switched, the Zoom motor and the Focus motor corresponding to the two lenses are driven, so that the focal lengths of the two lenses are kept consistent and the image is clear. The Zoom motor and the Focus motor are parts in the lens, the Zoom motor is used for controlling Zoom of the lens, and the Focus motor is used for controlling focusing of the lens, so that an imaging picture corresponding to the lens is clear. At this time, it can be considered that when the lenses are switched, it is necessary to ensure that the focal lengths of the two lenses to be switched are equal, and the number of pixels occupied by the shooting object in the scene in the imaging pictures corresponding to the two lenses is equal. The sizes of the pixel points in the images obtained by the two lenses through respective shooting are ensured to be the same.
In one case, the installation distance between lenses in the multi-lens apparatus is inevitable. Because the optical axes of a plurality of lenses cannot be ensured to be on the same straight line, the installation distance exists between the lenses. The installation distance exists between the lenses, so that when different lenses are used for shooting the same scene, the acquisition regions corresponding to different lenses are different, namely the acquisition regions represented by the images acquired by different lenses are different. The difference is obvious in a near-focus scene and not obvious in a far-focus scene. The above-mentioned near focus scene is: with the same resolution, in order to capture a high quality image, for example: shooting a scene in which the definition value exceeds a preset definition value and the required focal length does not exceed the preset focal length; the far focus scene is as follows: with the same resolution, in order to capture a high quality image, for example: and shooting to obtain an image with the definition value exceeding the preset definition value, wherein the required focal length exceeds the preset focal length.
As shown in fig. 1A, a schematic diagram of a positional relationship between two lenses of a lens a and a lens B having a mounting distance R in a multi-lens apparatus is shown. Because the installation distance R exists between the two lenses of the lens a and the lens B, the difference in shooting angle exists when the lens a and the lens B shoot images, as shown in fig. 1A, the region identified by F is the acquisition region corresponding to the lens a, the region identified by E is the acquisition region corresponding to the lens B, the region identified by S is the overlapping region of the acquisition regions corresponding to the lens a and the lens B, P identifies the focal length of the lens a and the lens B, T identifies the object distance of the lens a and the lens B, SensorA identifies the image sensor of the lens a, and SensorB identifies the image sensor of the lens B. As shown in fig. 1B, the two images can represent that there is a difference between the two corresponding lens capturing regions, that is, the difference between the two lenses in the horizontal direction is K, the difference between the two lenses in the vertical direction is J, the sizes of the two lenses are the same, and the two lenses represent that there is an overlapping region between the two lens capturing regions, but there is a difference between the two lens capturing angles. Because there is shooting angle difference between the camera lenses, when the camera lenses are switched, the situation that displayed images jump appears can occur, and poor use experience can be brought to users.
Therefore, when the lenses are switched, in order to avoid the difference in the shooting angle between the lenses, a poor use experience is brought to the user. The embodiment of the invention provides a correction parameter obtaining method, a correction parameter obtaining device and electronic equipment, and aims to obtain correction parameters for correcting images acquired by different lenses in multi-lens equipment.
The correction parameter obtaining method provided by the embodiment of the invention can be applied to any type of electronic equipment, and the electronic equipment can be equipment connected with the multi-lens equipment, such as a computer, a mobile phone and the like; or the multi-lens apparatus described above. The functional software for implementing the correction parameter obtaining method provided by the embodiment of the present invention may exist in the form of special client software, or may exist in the form of a plug-in of existing application software, which is all possible.
For the lenses in the multi-lens setting with the installation distance, the optical axes of the lenses are all parallel, and the corresponding acquisition regions of the lenses have a translation error, as shown in fig. 1A and 1B. Under the condition of a certain focal length and an object distance, when the lenses are switched, images acquired by the two switched lenses are corrected, so that imaging points of the same shooting object in a scene are located at the same position of the corrected images, namely, relative translation errors do not exist between the corrected images. In one case, one of the two lenses to be switched may be used as a reference lens, the reference lens is considered to have no translational error, then a relative translational error between the other lens to be switched and the reference lens is determined, and an image captured by the other lens may be corrected based on the relative translational error between the other lens and the reference lens. That is, an image acquired by one of the two lenses which are switched is used as a reference image, that is, an image which does not need to be corrected, and an image acquired by the other lens is corrected, so that the position of a shooting object in the corrected image is the same as the position of the shooting object in the reference image. For example: the position of the pixel point at the upper left corner of the image (including the reference image and the corrected image) is taken as the coordinate origin, the position of a photographic object in the reference image is (x1, y1), and the position of the photographic object in the corrected image is (x1, y 1). In the embodiment of the present invention, it is necessary to obtain and store the correction parameter, which enables the position of the photographic object in the corrected image to be the same as the position of the photographic object in the reference image, for the focal length and the object distance. Based on the above principle, as shown in fig. 2, an embodiment of the present invention provides a method for obtaining a correction parameter, which may include the following steps:
s201: acquiring first images of the lenses in the lens group to be corrected when the lenses respectively shoot target paper;
wherein, the lens in the lens group to be corrected is: the method comprises the steps that lenses with overlapped focal length sections exist in multi-lens equipment, pixel points of all first images are the same in size, the focal length and the object distance are the same when the first images are obtained by shooting through the lenses in a lens group to be corrected, and a first calibration point is arranged on target paper;
in one case, the lenses provided in the multi-lens apparatus may be zoom lenses, that is, each lens may support multiple focal lengths, and the multiple focal lengths supported by each lens may be referred to as a focal length segment, that is, each lens corresponds to a focal length segment. In another case, it is possible that the lenses provided in the multi-lens apparatus include both a zoom lens and a fixed focus, that is, both a lens capable of supporting a plurality of focal lengths and a lens capable of supporting one focal length.
In the embodiment of the invention, in the multi-lens device, at least two focal length sections corresponding to the lenses are overlapped. For example: the multi-lens device A comprises a first lens, a second lens, a third lens and a fourth lens, wherein the first lens corresponds to a focal length section [ a, c ], the second lens corresponds to a focal length section [ b, e ], the third lens corresponds to a focal length section [ e, m ] and the fourth lens corresponds to a focal length section [ m, n ], wherein a is more than b, less than c, less than e, and less than m, n. The focal length section corresponding to the first lens is overlapped with the focal length section corresponding to the second lens in the number [ b, c ], the focal length section corresponding to the second lens is overlapped with the focal length section corresponding to the third lens in the number [ e ], and the focal length section corresponding to the third lens is overlapped with the focal length section corresponding to the fourth lens in the number [ m ]. In theory, in a multi-lens device, there may be coincidence of focal lengths corresponding to three or more lenses, which is also possible.
The lenses with overlapped focal length sections in the multi-lens device can form a lens group to be corrected. In accordance with the above example, the first lens and the second lens can be referred to as a lens group to be corrected, the second lens and the third lens can be referred to as a lens group to be corrected, and the third lens and the fourth lens can be referred to as a lens group to be corrected.
Before the multi-lens device leaves the factory, the correction parameter obtaining process provided by the embodiment of the invention is executed for the multi-lens device, so that the correction parameters for correcting the images acquired by different lenses in the multi-lens device are obtained for the multi-lens device.
In the embodiment of the invention, the target paper can be shot by utilizing each lens in the multi-lens device, so that the target paper can completely fill the imaging picture of each lens as much as possible. Before shooting, aligning the multi-lens device to target paper, and performing focusing operation to enable imaging pictures corresponding to all lenses in a lens group to be corrected to be focused clearly, wherein for example, the definition of the imaging pictures exceeds a preset definition value; and adjusting the focal length of each lens in the lens group to be corrected to ensure that the number of pixel points occupied by the target paper in the imaging picture corresponding to each lens is the same, and the sizes of the pixel points in the imaging picture corresponding to each lens are the same, namely adjusting the focal length of each lens in the lens group to be corrected to be the same. Under the condition of the same focal length and the same object distance, the target paper is shot by utilizing all the lenses in the lens group to be corrected and sent to the electronic equipment, and the electronic equipment obtains a first image when the lenses in the lens group to be corrected respectively shoot the target paper. When the lenses in the lens group to be corrected respectively shoot target paper, the relative positions of the lenses in the lens group to be corrected are not changed. The lenses in the lens group to be corrected are shot for the same target paper, and it can be determined that the distances between the lenses in the lens group to be corrected and the target paper are equal, that is, the object distances corresponding to the lenses in the lens group to be corrected are equal.
The target paper can be made of transparent plastic plates, and can be square (including square and rectangle) or round. The surface of the target paper can be carved with a circle which takes the center of the plastic plate as the center of a circle and takes the preset length as the radius; and two diameters perpendicular to each other are engraved in the circle. The outer side of the circle in the target paper is carved with a square grid with preset size. When the target paper is square, the two mutually perpendicular diameters are respectively parallel or perpendicular to the edge of the target paper. In one case, in order to better distinguish the directions on the target paper, different identifiers may be set for the directions on the target paper, for example, when the target paper is square, different identifiers may be set for the four corners of the target paper. To ensure the accuracy of subsequent resulting calibration parameters.
In an embodiment of the present invention, the target paper is provided with a first calibration point, and the first calibration point may be a center of a circle carved by the target paper, that is, a target center of the target paper, or may be any position of the target paper. When the lens in the lens group to be corrected shoots the target paper, an image is obtained, and the first calibration point can occupy a complete pixel point in the image. One aspect, as shown in fig. 3, is a schematic view of a target paper according to an embodiment of the present invention. The target paper can be square, and a circle which takes the center of the target paper as the center of a circle and takes the preset length as the radius is carved on the surface of the target paper; the outer side of the circle carved on the surface of the target paper is carved with a square grid with preset size, each corner of the target paper is provided with different marks, as shown in figure 3, the four corners of the target paper are marked by the number of the solid square grids, and the four corners of the target paper are respectively marked by O, X, Y and Z.
Based on the schematic diagram of the target paper shown in fig. 3, the process of adjusting the focal length of each lens in the lens group to be corrected so that the number of pixel points occupied by the target paper in the imaging picture corresponding to each lens is the same and the size of the pixel points in the imaging picture corresponding to each lens is the same will be described:
assuming that the lens group to be corrected comprises a lens C and a lens D, recording an imaging point O corresponding to the O point in an imaging picture corresponding to the lens C1Coordinates of points
Figure BDA0001908891680000151
Imaging point X corresponding to X point1Coordinates of points
Figure BDA0001908891680000152
Imaging point Y corresponding to Y point1Coordinates of points
Figure BDA0001908891680000153
Calculated to obtain O1Point and X1Distance between points as a first distance
Figure BDA0001908891680000154
O1Point and Y1Distance between points as second distance
Figure BDA0001908891680000155
Adjusting the focal length of the lens D to enable an imaging point O corresponding to the O point in an imaging picture corresponding to the lens D2Imaging point X corresponding to point X2The distance between the points is equal to the first distance, and the imaging point O corresponding to the O point2Imaging point Y corresponding to point Y2The distance between the points is equal to the second distance, namely the number of pixel points occupied by the target paper in the imaging picture corresponding to the lens D and the imaging picture corresponding to the lens D of the target paper are obtainedThe number of occupied pixels in the plane is equal.
S202: determining the coordinate position of an imaging point of a first calibration point in a first image corresponding to each lens in a lens group to be corrected;
s203: and calculating to obtain translation correction parameters of each first lens relative to the second lens based on the coordinate position of the imaging point of the first calibration point in each first image.
Wherein, the second camera lens is: a lens of the lens group to be corrected, the first lens being: and the lens except the second lens in the lens group to be corrected.
It can be understood that when the lenses in the lens group to be corrected respectively shoot the target paper, the first calibration point set on the target paper has a corresponding imaging point in the imaging picture corresponding to each lens, that is, each first image includes the imaging point of the first calibration point. In the embodiment of the present invention, any feasible position determining manner may be adopted to determine the position coordinates of the imaging point of the first calibration point in each first image.
In the embodiment of the invention, the resolutions corresponding to the lenses in the lens group to be corrected are the same, when the first image is obtained by shooting, the object distances and the focal lengths of the lenses in the lens group to be corrected are the same, and the sizes of the pixel points in the first image are the same. In order to better ensure the accuracy of the determined position relationship between the first images, a coordinate system where the first images are located can be uniformly set. In one case, a coordinate system where each first image is located may be set, where the pixel points in the pth row and mth column in the first image are used as the origin of coordinates, and the directions of the horizontal axes and the vertical axes of the coordinate system where each first image is located are the same. Wherein P and M are both positive integers. For convenience of calculation, the position of the pixel point at the upper left corner (or the lower left corner, or the upper right corner or the lower right corner) in each first image may be set as the origin of coordinates, the row where the pixel point serving as the origin of coordinates is located is taken as the horizontal axis, and the column where the pixel point serving as the origin of coordinates is located is taken as the vertical axis.
In one case, the position relationship between the first images may be represented by positions of imaging points of the first calibration points in the first images, and further, a translational relationship between the first images may be determined based on the position relationship between the first images, so that imaging points corresponding to the same shooting object in the first images coincide. Based on the determined translation relation between the first images, translation correction parameters of the first lenses relative to the second lens can be determined.
When the lenses are switched, the purpose of correcting the image collected by one of the two lenses for lens switching is as follows: the imaging positions of the same shooting object (such as a first calibration point) in the scene in the images corresponding to different lenses are the same. In one case, it may be considered that a lens in the lens group to be corrected has no translational error, that is, serves as a second lens, and then, with reference to the second lens, the relative translational error of the other lenses in the lens group to be corrected, that is, the first lens, with respect to the second lens, is calculated; furthermore, the relative translational error can be used as a translational correction parameter of each first lens in each lens group to be corrected relative to the lens without translational error.
In one implementation, the step of calculating a translational correction parameter of each first lens with respect to the second lens based on a coordinate position of an imaging point of the first calibration point in each first image may include:
selecting a lens from the lenses in the lens group to be corrected as a second lens;
calculating the translational correction parameters of each first lens relative to the second lens by adopting the following steps:
determining position coordinates of an imaging point of a first calibration point in a first image corresponding to a first target lens as first position coordinates, wherein the first target lens is one lens in the first lens;
calculating to obtain a horizontal difference value and a horizontal correction direction between the first position coordinate and the second position coordinate, and a vertical difference value and a vertical correction direction, wherein the second position coordinate is as follows: determining the position coordinate of an imaging point of a first calibration point in a first image corresponding to a second lens;
and determining the calculated horizontal difference value and horizontal correction direction between the first position coordinate and the second position coordinate, and the calculated vertical difference value and vertical correction direction as the translation correction parameters of the first target lens relative to the second lens.
When one lens is selected from the lenses in the lens group to be corrected as the second lens, one lens may be randomly selected as the second lens, or a lens having a focal length including the maximum focal length may be selected as the second lens. The lens group to be corrected, which has the focal length including the maximum focal length, is selected as the second lens, so that management of correction parameters can be facilitated to a certain extent, and the subsequent image correction process is facilitated.
As shown in fig. 4A and 4B, the schematic diagrams of first images (including imaging points of the first calibration point) collected by the lenses in the lens group to be corrected are shown, where fig. 4A and 4B can represent the position of the imaging point of the first calibration point in each first image and the positional relationship between the imaging points of the first calibration point in each first image, and the images collected by the lenses in the lens group to be corrected respectively are: the size of the first image and the second image is 4 x 4, the border of the first image is marked by a solid line, and the border of the second image is marked by a dotted line. The black squares identify the imaging points of the first index points. The solid lines and the dashed lines in fig. 4A and 4B are only for distinguishing the different first images and are not meant to be limiting.
Based on fig. 4A and 4B, a description will be given of a process of obtaining the horizontal difference value and the horizontal correction direction, and the vertical difference value and the vertical correction direction between the first position coordinates and the second position coordinates by the above calculation:
the position of a pixel point at the lower left corner of each first image is taken as an origin, a horizontal axis is taken as the row of the pixel point, a coordinate system is established by taking the row of the pixel point as a vertical axis, the position coordinate of an imaging point of a first calibration point in the first image is (1,1), and the position coordinate of the imaging point of the first calibration point in the second image is (0, 2).
Taking the lens corresponding to the first image as the second lens, in order to make the position of the imaging point of the first calibration point in the first image second identical to the position of the imaging point of the first calibration point in the first image, the position coordinate of the imaging point of the first calibration point in the first image second needs to be converted from (0,2) to (1,1), at this time, the horizontal difference value and the horizontal correction direction between the first position coordinate and the second position coordinate, and the vertical difference value and the vertical correction direction can be calculated; the horizontal difference between the first position coordinate and the second position coordinate is 1 pixel point, the horizontal correction direction mark is negative, the vertical difference between the first position coordinate and the second position coordinate is 1 pixel point, and the vertical correction direction mark is positive. Specifically, it can be identified as: based on the coordinate system where the second image is located, the position of the imaging point of the first calibration point in the second image is translated by 1 pixel point in the direction of increasing the abscissa coordinate value, and is translated by 1 pixel point in the direction of decreasing the ordinate coordinate value, that is, based on the coordinate system where the second image is located, the first image is translated by 1 pixel point in the direction of increasing the abscissa coordinate value, and is translated by 1 pixel point in the direction of decreasing the ordinate coordinate value.
The second first image and the translated second first image are in a position relationship, as shown in fig. 4C. The positions of the pixels of the translated second first image and the positions of the pixels of the second first image have an overlapping region, for example, the positions of the light gray squares and the black squares in fig. 4C, and the positions of the pixels of the translated second first image and the positions of the pixels of the second first image have a non-overlapping region, for example, the positions of the dark gray squares.
In an implementation manner, since the resolution corresponding to each lens is fixed, in order to make the imaging positions of the first calibration points in the first images corresponding to each lens in the lens group to be corrected the same, that is, the position coordinates of the imaging points of the first calibration points in each first image are the same, at this time, the first image corresponding to the first target lens needs to be cropped, so that the imaging points of the first calibration points in the first target image can be translated to the position coordinates where the position coordinates of the imaging points of the first calibration points in the first image corresponding to the second lens are the same. The target paper is also provided with at least two second calibration points and at least two third calibration points, and the connecting line of the at least two second calibration points is vertical to the connecting line of the at least two third calibration points and is intersected with the first calibration point; before the step of determining the calculated horizontal difference and horizontal correction direction between the first position coordinate and the second position coordinate, and the calculated vertical difference and vertical correction direction as the translational correction parameters of the first target lens relative to the second lens, the method may further include:
determining position coordinates of imaging points of at least two second calibration points and imaging points of at least two third calibration points in a first image corresponding to a first target lens;
determining position coordinates of imaging points of at least two second calibration points and imaging points of at least two third calibration points in the first image corresponding to the second lens;
determining a cutting size and a cutting position corresponding to the first target lens based on position coordinates of imaging points of at least two second calibration points and at least two third calibration points in a first image corresponding to the first target lens, position coordinates of the imaging points of at least two second calibration points and at least two third calibration points in a first image corresponding to the second lens and a preset translation parameter calculation formula;
the step of determining the calculated horizontal difference and horizontal correction direction between the first position coordinate and the second position coordinate, and the calculated vertical difference and vertical correction direction as the translational correction parameters of the first target lens relative to the second lens may include:
and determining the calculated horizontal difference value and horizontal correction direction between the first position coordinate and the second position coordinate, the vertical difference value and vertical correction direction, and the corresponding cutting size and cutting position of the first target lens as translation correction parameters of the first target lens relative to the second lens.
In the embodiment of the present invention, the preset translation parameter calculation formula may be identified as follows, and formula (1) is:
Figure BDA0001908891680000191
from the above formula, the following formula (2) can be derived:
Figure BDA0001908891680000192
wherein (x)a，ya) Identifying the position coordinates of an imaging point of a first calibration point in a first image corresponding to a first target lens; w is aaIdentifying the distance between two imaging points which are farthest away from each other in the imaging points of at least two third calibration points in the first image corresponding to the first target lens (such as the imaging points of the point D and the point F shown in FIG. 3); h isaIdentifying the distance between two imaging points which are farthest away from each other in the imaging points of at least two second calibration points in the first image corresponding to the first target lens (such as the imaging points of points C and E shown in fig. 3); (x)b，yb) Identifying the position coordinates of the imaging point of the first calibration point in the first image corresponding to the second lens; w is abIdentifying the distance between two imaging points which are farthest away from each other in the imaging points of at least two third calibration points in the first image corresponding to the second lens (such as the imaging points of points D and F shown in fig. 3); h isbIdentifying the distance between two imaging points which are farthest away from each other in the imaging points of at least two second calibration points in the first image corresponding to the second lens (such as the imaging points of points C and E shown in fig. 3); (L)0,M0) Identifying resolutions of the first target shot and the second shot; down identifies the size of a lower frame of a first image corresponding to the first target shot, which needs to be cut; up identifies the size of the upper frame of the first image corresponding to the first target shot to be cut; left identifies the size of the left frame of the first image corresponding to the first target lens to be cut, and right identifies the size of the right frame of the first image corresponding to the first target lens to be cut.
And then, the calculated size to be cut of each edge (lower frame, upper frame, left frame and right frame) of the first image corresponding to the first target lens can be used as the cutting size corresponding to the first target lens, and further, the cutting position corresponding to the first target lens can be determined according to the size to be cut of each edge. And then combining the calculated horizontal difference value and horizontal correction direction between the first position coordinate and the second position coordinate, and the vertical difference value and the vertical correction direction as the translation correction parameters of the first target lens relative to the second lens.
Taking the upper border of the first image corresponding to the first target lens as an example, a process of determining a clipping position corresponding to the first target lens according to the size of each edge to be clipped is described: determining the size up of the upper frame of the first image corresponding to the first target shot to be cut, determining the up line and the up +1 line from the pixel point contained in the line of the upper frame as the starting line, and determining the position between the up line and the up +1 line as the cutting position of the upper frame of the first image corresponding to the first target shot.
Wherein, when the target paper is the rectangle, can mark to four angles of the target paper of rectangle, and then when shooing to this target paper, can distinguish lower frame, last frame, left frame and right frame in the first image through the mark of four angles of the target paper of rectangle. The lower frame, the upper frame, the left frame and the right frame are relative. For example: for the guaranteed shooting shown in fig. 3, in the obtained first image, the image frame corresponding to the side identified by "O" and "X" may be used as the upper frame of the first image, the image frame corresponding to the side identified by "O" and "Y" may be used as the left frame of the first image, the image frame corresponding to the side identified by "X" and "Z" may be used as the right frame of the first image, and the image frame corresponding to the side identified by "Y" and "Z" may be used as the lower frame of the first image.
It can be understood that, in the image correction stage, after the image captured by the first target lens is cropped based on the translation correction parameters, the cropped image needs to be stretched. For example: the 1920 x 1080 image is cropped 100 pixels into 1820 x 980 original image, but 1920 x 1080 is still output when displayed.
The embodiments of the present invention do not limit the resolution of the lenses in the multi-lens apparatus as long as the resolution of each lens is the same. The resolution of the lenses in the multi-lens device may be 1920 × 1080, 3840 × 2160 or other sizes.
In the embodiment of the present invention, in order to better improve the performance and user experience of the multi-lens device, different correction parameters need to be obtained for different object distances on the premise of the same focal length, that is, on the premise of the same focal length, the correction parameter obtaining procedure provided in the embodiment of the present invention is executed for different object distances. When the lenses are switched in the using process of the multi-lens device, the correction parameters corresponding to the actual object distance can be used according to the actual object distance, and the images collected by the switched lenses are corrected. In one case, the object distance corresponding to the multi-lens device, i.e., the object distance corresponding to each lens in the lens group to be corrected, can be changed by using the distance-increasing lens.
In the shooting process, the object distance is the same, and the imaging effect of the lens is different when the focal distance is different. In one case, the lenses in the multi-lens device may be all zoom lenses, and there may be multiple focal lengths that are overlapped between the lenses in the lens group to be corrected, and in order to ensure that the lenses in the lens group to be corrected are switched at any focal length, a better switching effect may be achieved, so that the difference between the images is smaller. In the embodiment of the present invention, in order to better improve the performance of the multi-lens device and the user experience, different correction parameters need to be obtained for different focal lengths on the premise that the object distances are the same, that is, on the premise that the object distances are the same, the correction parameter obtaining process provided in the embodiment of the present invention is performed for different focal lengths. When the lenses are switched in the using process of the multi-lens device, the images collected by the switched lenses can be corrected by using the correction parameters corresponding to the actual focal length according to the actual focal length.
The process of correcting the image by using the correction parameters may flexibly adopt any feasible correction mode, and the embodiment of the present invention does not limit the correction mode adopted in the process of correcting the image by using the correction parameters.
In consideration of the fact that the correction parameter obtaining method provided by the embodiment of the invention is used for obtaining a set of correction parameters for each object distance or focal distance, the workload is large. In order to reduce workload, in the embodiment of the present invention, in a case that a focal length is fixed, for a first preset number of object distances, by using the correction parameter obtaining method provided in the embodiment of the present invention, a translational correction parameter of each first lens with respect to the second lens, which corresponds to each object distance in the first preset number of object distances, is obtained, and for translational correction parameters corresponding to other object distances, the translational correction parameters may be obtained by calculating the obtained translational correction parameters and a preset equal proportion algorithm. In one case, the first preset number may be 3, and the correction parameter obtaining method provided in the embodiment of the present invention may be utilized to obtain the translational correction parameters of each first lens corresponding to the object distances of 6 meters, 30 meters, and 100 meters with respect to the second lens, and further calculate the translational correction parameters of each first lens corresponding to other object distances with respect to the second lens based on a preset equal proportion algorithm and the obtained translational correction parameters of each first lens corresponding to the 3 object distances with respect to the second lens, for example, when the object distance M is between 6 and 30 meters, the preset equal proportion algorithm (1) may be identified as follows:
Figure BDA0001908891680000221
Figure BDA0001908891680000222
wherein (W)M,HM) Marking the horizontal difference value and the vertical difference value of the first lens relative to the second lens corresponding to the object distance M, (W)6,H6) Marking the horizontal difference of the first lens relative to the second lens corresponding to the object distance of 6 metersValue and vertical difference (W)30,H30) And identifying the horizontal difference value and the vertical difference value of the first lens relative to the second lens corresponding to the object distance of 30 meters.
For the same first lens, at any object distance and focal length, the horizontal correction direction and the vertical correction direction in the translational correction parameters of the first lens relative to the second lens are the same.
Similarly, the method for obtaining the correction parameters provided by the embodiment of the present invention can be used to obtain the translational correction parameters of each first lens relative to the second lens corresponding to each focal length in the second preset number of focal lengths, with respect to the second preset number of focal lengths, under the condition that the object distance is fixed. And then calculating the translational correction parameters of the first lenses corresponding to the other object distances relative to the second lens based on a preset equal proportion algorithm and the obtained translational correction parameters of the first lenses corresponding to the second preset number of object distances relative to the second lens. The calculation process is similar to the process of obtaining the translational correction parameters of each first lens relative to the second lens corresponding to other object distances through calculation, and details are not repeated herein.
In the embodiment of the present invention, the position coordinates of the imaging point of the first calibration point set on the target paper in the first image may be used to determine the difference of the shooting angles between the lenses in the lens group to be corrected, and based on the difference of the shooting angles between the lenses in the lens group to be corrected, the first lens, which is the other lens except for the lens used as the reference, and the translation correction parameter relative to the second lens, which is the other lens used as the reference, are obtained. In the subsequent image correction process, the image acquired by the first lens is corrected by using the translation correction parameter of the first lens relative to the second lens, so that the obtained corrected image and the image acquired by the second lens have the same shooting angle, that is, the positions of all points in the shot scene in the obtained corrected image are the same as the positions in the image acquired by the second lens. In addition, in the embodiment of the invention, the first calibration point in the target paper can occupy a complete pixel point in the first image, so that the correction parameter determined based on the first image containing the target paper can be more accurate, and the subsequent image corrected by using the correction parameter has better effect.
In one implementation, after the step of calculating a translational correction parameter of each first lens with respect to the second lens based on the coordinate position of the imaging point of the first calibration point in each first image, the method may further include:
obtaining a focal length and an object distance when a first image is obtained by shooting through a lens in a lens group to be corrected, wherein the focal length and the object distance are used as a first focal length and a first object distance;
and storing the corresponding relation between the translation correction parameter of the first lens relative to the second lens and the first focal length and the first object distance for each first lens in the lens group to be corrected.
In the embodiment of the invention, in order to facilitate the subsequent shooting process of the multi-lens device, when the lenses are switched, the accurate translation correction parameters are quickly determined based on the actual object distance and the actual focal distance when the lenses are switched, after the translation correction parameters of each first lens relative to the second lens are obtained through calculation, the first focal distance and the first object distance when the first image is obtained through shooting are also required to be obtained, further, the first focal distance and the first object distance are stored for each first lens in the lens group to be corrected, the translation correction parameters of the first lens relative to the second lens are established and stored, and the corresponding relation between the translation correction parameters of the first lens relative to the second lens and the first focal distance and the first object distance is established and stored.
In one implementation, when lenses of a multi-lens apparatus are mounted, a rotation error may exist between the lenses in addition to a mounting distance. Also, since a plurality of lenses are provided in the multi-lens apparatus, a rotation error of each lens may be different, that is, a relative rotation error exists between the lenses of the multi-lens apparatus. When the lenses are switched, a relative rotation error exists between the two lenses for switching, and a relative rotation error also exists between the collected images, so that poor experience can be brought to a user. Then, when the lenses are switched, rotation error correction needs to be performed on the images captured by the two lenses that are switched. Fig. 5 is a schematic diagram showing a position relationship between image contents represented by images captured by two lenses with relative rotation errors. Before the step of obtaining the first image of the target paper taken by each lens of the lens group to be corrected, as shown in fig. 6, the method may further include:
s601: acquiring second images of the target paper shot by the lenses in the lens group to be corrected;
the target paper is also provided with at least two second calibration points and at least two third calibration points, the connecting line of the at least two second calibration points is perpendicular to the connecting line of the at least two third calibration points and intersects with the first calibration point, and the focal lengths of the lenses in the lens group to be corrected are the same when the lenses shoot to obtain a second image;
in the embodiment of the invention, the target paper can be shot by utilizing each lens in the multi-lens device, so that the target paper can completely fill the imaging picture of each lens as much as possible. Before shooting, aligning the multi-lens device to target paper, and performing focusing operation to enable imaging pictures corresponding to all lenses in a lens group to be corrected to be focused clearly, wherein for example, the definition of the imaging pictures exceeds a preset definition value; and shooting the target paper by utilizing each lens in the lens group to be corrected, and sending the target paper to the electronic equipment, wherein the electronic equipment obtains a second image when the lens in the lens group to be corrected respectively shoots the target paper. When the lenses in the lens group to be corrected respectively shoot target paper, the relative positions of the lenses in the lens group to be corrected are not changed. The lenses in the lens group to be corrected are shot for the same target paper, and it can be determined that the distances between the lenses in the lens group to be corrected and the target paper are equal, that is, the object distances corresponding to the lenses in the lens group to be corrected are equal.
In one implementation, in order to facilitate the subsequent calculation process, the first calibration point may be a target center of the target paper, that is, a center of a circle inscribed on the surface of the target paper, the at least two second calibration points are respectively located at two sides of the first calibration point, and the at least two third calibration points are respectively located at two sides of the first calibration point. In order to better ensure the accuracy of the calculated correction parameters, the distances between the first calibration point and the at least two second calibration points and between the first calibration point and the at least two third calibration points are not less than a preset distance. In one case, the distances between the first index point and the at least two second index points and between the at least two third index points may be equal.
S602: determining position coordinates of an imaging point of a first calibration point, imaging points of at least two second calibration points and imaging points of at least two third calibration points in a second image corresponding to each lens in the lens group to be corrected;
s603: calculating a rotation error of the lens corresponding to the second image based on an imaging point of a first calibration point, respective imaging points of at least two second calibration points, respective position coordinates of respective imaging points of at least two third calibration points and a preset rotation error calculation formula in the second image corresponding to each lens in the lens group to be corrected;
in the embodiment of the present invention, for convenience of calculation, the position of the pixel point at the upper left corner (or the lower left corner, or the upper right corner, or the lower right corner) in each first image may be set as the origin of coordinates, the row where the pixel point serving as the origin of coordinates is located is taken as the horizontal axis, and the column where the pixel point serving as the origin of coordinates is located is taken as the vertical axis.
Theoretically, in the second image corresponding to each lens in the lens group to be corrected, the relationship between the imaging point of the first calibration point, the imaging point of each of the at least two second calibration points, and the imaging point of each of the at least two third calibration points may exist as follows: in the first case, in the second image corresponding to each lens in the lens group to be corrected, the imaging point of the first calibration point set in the target paper and the imaging points of at least two second calibration points are the same row of pixel points, and the imaging point of the first calibration point and the imaging points of at least two third calibration points are the same row of pixel points; or, in the second case, in the second image corresponding to each lens in the lens group to be corrected, the imaging point of the first calibration point set in the target paper and the imaging points of at least two second calibration points are the same row of pixel points, and the imaging point of the first calibration point and the imaging points of at least two third calibration points are the same row of pixel points.
When the lenses in the multi-lens device have rotation errors, the relationship among the imaging point of the first calibration point, the imaging points of the at least two second calibration points, and the imaging points of the at least two third calibration points in the second image corresponding to each lens in the lens group to be corrected does not satisfy the above two conditions.
Moreover, the larger the rotation error of the lens in the multi-lens device is, the larger the included angle between the straight line where the imaging point of the first calibration point and the respective imaging points of the at least two second calibration points in the second image corresponding to the lens are located and the straight line where the imaging point of the first calibration point and the respective imaging points of the at least two second calibration points are located theoretically is, taking the first case as an example: the larger the included angle between the imaging point of the first calibration point in the second image corresponding to the lens and the longitudinal axis between the straight line where the imaging points of the at least two second calibration points are located and the coordinate axis is; and the larger the included angle between the straight line where the imaging point of the first calibration point and the respective imaging points of the at least two third calibration points in the second image corresponding to the lens are located and the straight line where the imaging point of the first calibration point and the respective imaging points of the at least two third calibration points are located theoretically is, taking the first case as an example: the larger the included angle between the imaging point of the first calibration point in the second image corresponding to the lens and the longitudinal axis between the straight line where the imaging points of the at least two third calibration points are located and the coordinate axis is. Accordingly, the larger the rotation angle of the lens.
And when a relative rotation error exists between the two lenses, an included angle exists between a straight line where an imaging point of the first calibration point and a straight line where respective imaging points of the at least two second calibration points are located in the second image corresponding to the two lenses, and an included angle exists between a straight line where an imaging point of the first calibration point and a straight line where respective imaging points of the at least two third calibration points are located in the second image corresponding to the two lenses.
In the embodiment of the invention, the rotation error of each lens in the multi-lens device can be characterized as follows: the angle of rotation about the optical axis of the lens when each lens is actually mounted. The relative rotation error between the lenses in a multi-lens device can be characterized as: the difference between the rotation angles around the optical axis of the lens when the respective lenses are actually mounted. Wherein the optical axes of the lenses in the multi-lens device are all parallel. And when the relationship among the imaging point of the first calibration point, the imaging points of the at least two second calibration points and the imaging points of the at least two third calibration points in the second image corresponding to the lens meets any one of the two conditions, the rotation error of the lens is considered to be 0.
Based on the above principle, in the embodiment of the present invention, for each second image, the position relationship between the imaging point of the first calibration point and the imaging point of each of the at least two second calibration points, and the position relationship between the imaging point of the first calibration point and the imaging point of each of the at least two third calibration points may be determined through the position coordinates of the imaging point of the first calibration point, the imaging point of each of the at least two second calibration points, and the imaging point of each of the at least two third calibration points of the second image; and further obtaining a rotation error corresponding to the second image.
In one implementation, as shown in fig. 3, the target paper may include two second calibration points, which are point C and point E, respectively, and two third calibration points, which are point D and point F, respectively, when there are two second calibration points and there are two third calibration points;
the preset rotation error calculation formula (2) is:
Figure BDA0001908891680000261
wherein α (N) denotes a rotation error of the lens corresponding to the second image in the case where the focal length is N, (D)x,Dy) And (F)x,Fy) Position coordinates of imaging points respectively identifying two second calibration points in the second image, (C)x,Cy) And (E)x,Ey) Position coordinates of imaging points that respectively identify two third calibration points in the second image, (G)x,Gy) And identifying the position coordinates of the imaging point of the first calibration point in the second image, wherein N is a positive number.
S604: and calculating the rotation correction parameters of the third lenses relative to the fourth lenses based on the rotation errors of each lens in the lens group to be corrected.
Wherein, the fourth camera lens is: one lens in the lens group to be corrected, the third lens is: and the lens except the fourth lens in the lens group to be corrected.
In the embodiment of the present invention, the purpose of performing rotation error correction on images acquired by two switched lenses is: so that there is no relative rotation error between the images captured by the two lenses that are switched, for example: and no included angle exists between the central lines of the images collected by the two switched lenses. In view of the above, one of the two lenses to be switched may be used as a reference lens, and the reference lens may be regarded as having no rotation error, and a relative rotation error of the other lens with respect to the reference lens may be calculated with respect to the other lens except the reference lens, and further, a rotation correction parameter of the other lens with respect to the reference lens may be determined based on the calculated relative rotation error of the other lens with respect to the reference lens.
As shown in fig. 5, the left side shows an imaging picture corresponding to the fourth lens, and the right side shows an imaging picture corresponding to the third lens, and it is considered that there is no rotation error of the fourth lens, and as shown in fig. 5, the relative rotation error of the third lens with respect to the fourth lens is an angle α.
Based on the above principle, the step of calculating rotation correction parameters of the third lenses relative to the fourth lenses based on the rotation error of each lens in the lens group to be corrected may include:
selecting a lens from the lenses in the lens group to be corrected as a fourth lens;
calculating the difference value of the rotation error of the third lens and the rotation error of the fourth lens as the rotation difference value of the third lens aiming at each third lens;
and determining the calculated rotation difference value of the third lens as a rotation correction parameter of the third lens relative to the fourth lens for each third lens.
When one lens is selected from the lenses in the lens group to be corrected as the fourth lens, one lens can be randomly selected as the fourth lens, and the lens with the supported focal length including the maximum focal length in the lens group to be corrected can also be selected as the fourth lens. The lens group to be corrected, which has the focal length including the maximum focal length, is selected as the fourth lens, so that management of correction parameters can be facilitated to a certain extent, and the subsequent image correction process is facilitated. In one case, when a lens is selected as the fourth lens from the lenses in the lens group to be corrected, the above-mentioned lens as the second lens may be selected as the fourth lens.
Specifically, for each third lens, calculating a difference between a rotation error of the third lens and a rotation error of the fourth lens, as the rotation difference of the third lens, the equation (3) may be:
αrelative to each other＝α1-α2 (3)
Wherein alpha isRelative to each otherThe rotation difference of the third lens is identified, α 1 identifies the rotation error of the third lens, and α 2 identifies the rotation error of the fourth lens.
In the process of shooting, when the focal length of the lens is different, the rotation error between the shot images is different. In one case, the lenses in the multi-lens device may be all zoom lenses, and there may be multiple focal lengths that are overlapped between the lenses in the lens group to be corrected, and in order to ensure that the lenses in the lens group to be corrected are switched at any focal length, a better switching effect may be achieved, so that the difference between the images is smaller. Theoretically, the correction parameter obtaining method provided by the embodiment of the present invention can be respectively applied to each third lens in the lens group to be corrected to obtain the rotation correction parameters of the third lens relative to the fourth lens at different focal lengths. Wherein, the different focal lengths refer to: and the focal length supported by the third lens and the focal length supported by the fourth lens are overlapped.
As for each third lens, the calibration parameter obtaining method provided by the embodiment of the present invention is respectively used to obtain the rotation calibration parameters of the third lens relative to the fourth lens at different focal lengths, the workload is large, and in order to reduce the workload to a certain extent, the calibration parameter obtaining method provided by the embodiment of the present invention can be used to obtain the rotation calibration parameters of the third lens relative to the fourth lens at the target focal length for each third lens; and the rotation correction parameters of the third lens relative to the fourth lens at other focal lengths can be obtained by calculating the rotation correction parameters of the third lens relative to the fourth lens at the target focal length. Wherein, the target focal length may include: the third lens supports a focal length which is coincident with the focal length supported by the fourth lens, and the maximum focal length and the minimum focal length in the focal lengths are coincident with each other, and the average focal length of the maximum focal length and the minimum focal length is equal to the average focal length.
The rotation correction parameter obtained by the above calculation with respect to the fourth lens at the target focal length can be calculated by using a preset equal-proportion algorithm.
It can be understood that, as for the rotation error between the lenses, the influence of the object distance is not so great, and the influence of the object distance on the rotation error between the lenses can be ignored to some extent.
In the embodiment of the present invention, when there is a relative rotation error between lenses in a multi-lens device, in order to ensure the accuracy of the obtained translational correction parameters between lenses, it is necessary to first ensure that the rotational error between the lenses in the lens group to be corrected is corrected, and then calculate the translational correction parameters between the lenses in the lens group to be corrected.
In one implementation, after the step of calculating rotation correction parameters of the third lenses relative to the fourth lens based on the rotation error of each lens of the lens group to be corrected, the method may further include:
obtaining a focal length of a lens group to be corrected when a second image is obtained by shooting, and taking the focal length as a second focal length;
and storing the corresponding relation between the rotation correction parameter of the third lens relative to the fourth lens and the second focal length for each third lens in the lens group to be corrected.
In the embodiment of the invention, in order to facilitate the subsequent shooting process of the multi-lens device, when the lenses are switched, the accurate rotation correction parameters are quickly determined based on the actual object distance and the actual focal length when the lenses are switched, after the rotation correction parameters of the third lenses relative to the fourth lens are obtained through calculation, the second focal length when the second image is obtained through shooting is also required to be obtained, and further, the second focal length is stored for each third lens in the lens group to be corrected, the rotation correction parameters of the third lenses relative to the fourth lens are established and stored, and the corresponding relation between the rotation correction parameters of the third lenses relative to the fourth lens and the second focal length is established and stored.
In one implementation, before the step of determining, for each third lens, the calculated difference of the third lens as a rotation correction parameter of the third lens relative to the fourth lens, the method may further include:
for each third lens, determining a cutting position corresponding to the third lens and a cutting size corresponding to the third lens based on a central point of a second image corresponding to the third lens, a rotation difference value of the third lens and a resolution of the third lens;
the step of determining the calculated difference of the third lens as the rotation correction parameter of the third lens relative to the fourth lens for each third lens may include:
and determining the calculated difference value of the third lens, the cutting position corresponding to the third lens and the cutting size corresponding to the third lens as the rotation correction parameters of the third lens relative to the fourth lens for each third lens.
When the image is subjected to rotation correction, the size of the sensor for acquiring the image is fixed for the lens, and after the third lens is rotated based on the rotation difference value of the third lens, a black edge may appear in the acquired image.
In the embodiment of the present invention, the resolution (i.e., the number of row pixels and the number of column pixels) of the second image cropped with respect to the fourth shot and the third shot can be calculated according to the rotation difference of the third shot and the resolution of the third shot, i.e., the resolution of the second image before cropping (i.e., the number of row pixels and the number of column pixels). And then, based on the calculated resolution of the second image after the third lens is cut, the cutting sizes and the cutting positions of the upper part, the lower part, the left part and the right part of the imaging picture before the third lens is cut can be obtained according to the central symmetry principle.
In one case, in order to ensure the image quality, it is necessary to specify a cutting size and a cutting position that maximize the resolution of the second image after cutting. The determination process may be as follows: as shown in fig. 5, the solid line rectangle shown on the right side in fig. 5 is the second image before cropping corresponding to the third shot, and the dotted line rectangle shown on the right side in fig. 5 is the second image after cropping, and in this case, the resolution of the second image after cropping can be guaranteed to be the maximum. At this time, the resolution between the two images, that is, the number of row pixels and the number of column pixels, has the following relation (4):
Figure BDA0001908891680000301
wherein L is1Identifying the number of line pixels, M, of the cropped second image1And identifying the number of column pixel points of the second image obtained after cutting, identifying the number of row pixel points of the second image before cutting corresponding to the third lens, identifying the number of column pixel points of the second image before cutting corresponding to the third lens, and identifying the relative rotation error of the third lens relative to the fourth lens by alpha.
The following relationship (4) is modified to yield equation (5):
Figure BDA0001908891680000302
at this time, the second image before cropping corresponding to the third shot needs to be cropped up and down (M-M) with the center point of the second image before cropping as the center of symmetry12) line of pixel points, left and right cutting (L-L) of the second image before cutting1And/2) arranging pixel points to obtain a second image after cutting.
Corresponding to the above method embodiment, an embodiment of the present invention provides a correction parameter obtaining apparatus, as shown in fig. 7, where the apparatus includes:
a first obtaining module 710, configured to obtain a first image of a target paper taken by each lens in a lens group to be corrected, where the lenses in the lens group to be corrected are: the method comprises the steps that lenses with overlapped focal length sections exist in multi-lens equipment, pixel points in all first images are the same in size, the focal length and the object distance are the same when the first images are obtained through shooting by the lenses in a lens group to be corrected, and first calibration points are arranged on target paper;
a first determining module 720, configured to determine, for a first image corresponding to each lens of the lens group to be corrected, a coordinate position of an imaging point of the first calibration point in the first image;
a first calculating module 730, configured to calculate, based on a coordinate position of an imaging point of the first calibration point in each first image, a translational correction parameter of each first lens with respect to a second lens, where the second lens is: the first lens is a lens in the lens group to be corrected: and the lenses except the second lens in the lens group to be corrected.
In the embodiment of the present invention, the position coordinates of the imaging point of the first calibration point set on the target paper in the first image may be used to determine the difference of the shooting angles between the lenses in the lens group to be corrected, and based on the difference of the shooting angles between the lenses in the lens group to be corrected, the first lens, which is the other lens except for the lens used as the reference, and the translation correction parameter relative to the second lens, which is the other lens used as the reference, are obtained. In the subsequent image correction process, the image acquired by the first lens is corrected by using the translation correction parameter of the first lens relative to the second lens, so that the obtained corrected image and the image acquired by the second lens have the same shooting angle, that is, the positions of all points in the shot scene in the obtained corrected image are the same as the positions in the image acquired by the second lens. In addition, in the embodiment of the invention, the first calibration point in the target paper can occupy a complete pixel point in the first image, so that the correction parameter determined based on the first image containing the target paper can be more accurate, and the subsequent image corrected by using the correction parameter has better effect.
In one implementation, the apparatus further comprises:
a second obtaining module, configured to obtain a focal length and an object distance when the lenses in the lens group to be corrected take a first image as a first focal length and a first object distance after calculating a translational correction parameter of each first lens relative to the second lens based on the coordinate position of the imaging point of the first calibration point in each first image;
the first storage module is used for storing the corresponding relation between the translation correction parameter of the first lens relative to the second lens and the first focal length and the first object distance for each first lens in the lens group to be corrected.
In one implementation, the first computing module 730 includes:
the first selection unit is used for selecting a lens from the lenses in the lens group to be corrected as a second lens;
the first calculation unit is used for calculating and obtaining the translation correction parameters of each first lens relative to the second lens by adopting the following steps: the method comprises the following steps:
a first determining subunit, configured to determine, as a first position coordinate, a position coordinate of an imaging point of the first calibration point in a first image corresponding to a first target lens, where the first target lens is a lens of the first lens;
a first calculating subunit, configured to calculate a horizontal difference value and a horizontal correction direction between the first position coordinate and a second position coordinate, and a vertical difference value and a vertical correction direction, where the second position coordinate is: determining the position coordinate of an imaging point of the first calibration point in a first image corresponding to the second lens;
and the second determining subunit is configured to determine, as the translation correction parameter of the first target lens relative to the second lens, the calculated horizontal difference value and horizontal correction direction between the first position coordinate and the second position coordinate, and the calculated vertical difference value and vertical correction direction.
In one implementation manner, the target paper is further provided with at least two second calibration points and at least two third calibration points, and a connecting line of the at least two second calibration points is perpendicular to a connecting line of the at least two third calibration points and intersects with the first calibration point;
the first calculating module 730 further comprises:
a third determining subunit, configured to determine, before determining, as the translational correction parameter of the first target lens relative to the second lens, the horizontal difference value and the horizontal correction direction between the first position coordinate and the second position coordinate, and the vertical difference value and the vertical correction direction obtained by the calculation, position coordinates of an imaging point of the at least two second calibration points and an imaging point of the at least two third calibration points in the first image corresponding to the first target lens; determining position coordinates of imaging points of the at least two second calibration points and imaging points of the at least two third calibration points in the first image corresponding to the second lens;
a fourth determining subunit, configured to determine a cropping size and a cropping position corresponding to the first target lens, based on position coordinates of the imaging points of the at least two second calibration points and the at least two third calibration points in the first image corresponding to the first target lens, the position coordinates of the imaging points of the at least two second calibration points and the imaging points of the at least two third calibration points in the first image corresponding to the second lens, and a preset translation parameter calculation formula;
the second determining subunit is specifically configured to
And determining the calculated horizontal difference value and horizontal correction direction between the first position coordinate and the second position coordinate, the calculated vertical difference value and vertical correction direction, and the cutting size and cutting position corresponding to the first target lens as the translation correction parameters of the first target lens relative to the second lens.
In one implementation, the apparatus further comprises:
a third obtaining module, configured to obtain second images of the lenses in the lens group to be corrected when the lenses take target paper, before obtaining the first images of the lenses in the lens group to be corrected when the lenses take the target paper, where the target paper is further provided with at least two second calibration points and at least two third calibration points, a connection line of the at least two second calibration points is perpendicular to a connection line of the at least two third calibration points and intersects with the first calibration point, and focal lengths of the lenses in the lens group to be corrected when the lenses take the second images are the same;
a second determining module, configured to determine, for a second image corresponding to each lens of the lens group to be corrected, position coordinates of an imaging point of the first calibration point, imaging points of at least two second calibration points, and imaging points of at least two third calibration points in the second image;
a second calculating module, configured to calculate, for a second image corresponding to each lens in the lens group to be corrected, a rotation error of the lens corresponding to the second image based on an imaging point of the first calibration point in the second image, an imaging point of each of at least two second calibration points, a position coordinate of an imaging point of each of at least two third calibration points, and a preset rotation error calculation formula;
a third calculating module, configured to calculate a rotation correction parameter of each third lens relative to a fourth lens based on a rotation error of each lens in the lens group to be corrected, where the fourth lens is: one lens of the lens group to be corrected, the third lens is: and the lens groups to be corrected except the fourth lens.
In one implementation, the first calibration point is a target center of the target paper, at least two of the second calibration points are respectively located at two sides of the first calibration point, and at least two of the third calibration points are respectively located at two sides of the first calibration point.
In one implementation, the apparatus further comprises:
a fourth obtaining module, configured to obtain, after the rotation correction parameter of each third lens with respect to the fourth lens is calculated and obtained based on the rotation error of each lens in the lens group to be corrected, a focal length when the lens in the lens group to be corrected obtains a second image by shooting, as the second focal length;
and the second storage module is used for storing the corresponding relation between the rotation correction parameter of the third lens relative to the fourth lens and the second focal length aiming at each third lens in the lens group to be corrected.
In one implementation, when the second index points are two, and the third index points are two;
the preset rotation error calculation formula is as follows:
Figure BDA0001908891680000341
wherein α (N) denotes a rotation error of a lens corresponding to the second image in a case where the focal length is N, and (D)x,Dy) And (F)x,Fy) Position coordinates respectively identifying imaging points of two of the second calibration points in the second image, the (C)x,Cy) And (E)x,Ey) Position coordinates respectively identifying imaging points of two of the third calibration points in the second image, the (G)x,Gy) Identifying an imaged point of the first calibration point in the second imagePosition coordinates, N is a positive number.
In one implementation, the third computing module includes:
the second selection unit is used for selecting one lens from the lenses in the lens group to be corrected as a fourth lens;
a second calculation unit, configured to calculate, for each third lens, a difference between a rotation error of the third lens and a rotation error of the fourth lens, as a rotation difference of the third lens;
and the first determining unit is used for determining the calculated rotation difference value of the third lens as a rotation correction parameter of the third lens relative to the fourth lens for each third lens.
In one implementation, the third computing module further comprises:
a second determining unit, configured to determine, for each third lens, a clipping position corresponding to the third lens and a clipping size corresponding to the third lens based on a central point of a second image corresponding to the third lens, a rotation difference value of the third lens, and a resolution of the third lens before determining, for each third lens, the calculated difference value of the third lens as a rotation correction parameter of the third lens with respect to the fourth lens;
the first determining unit is specifically configured to:
and determining the calculated difference value of the third lens, the cutting position corresponding to the third lens and the cutting size corresponding to the third lens as the rotation correction parameters of the third lens relative to the fourth lens for each third lens.
Corresponding to the above method embodiments, the embodiment of the present invention further provides an electronic device, as shown in fig. 8, including a processor 810, a communication interface 820, a memory 830 and a communication bus 840, where the processor 810, the communication interface 820 and the memory 830 communicate with each other through the communication bus 840,
a memory 830 for storing a computer program;
the processor 810 is configured to, when executing the computer program stored in the memory 830, implement any of the above-mentioned correction parameter obtaining method steps provided by the embodiment of the present invention:
obtaining first images of respective shooting target papers of lenses in a lens group to be corrected, wherein the lenses in the lens group to be corrected are as follows: the method comprises the steps that lenses with overlapped focal length sections exist in multi-lens equipment, pixel points in all first images are the same in size, the focal length and the object distance are the same when the first images are obtained through shooting by the lenses in a lens group to be corrected, and first calibration points are arranged on target paper;
determining the coordinate position of the imaging point of the first calibration point in the first image aiming at the first image corresponding to each lens in the lens group to be corrected;
calculating and obtaining a translation correction parameter of each first lens relative to a second lens based on the coordinate position of the imaging point of the first calibration point in each first image, wherein the second lens is: the first lens is a lens in the lens group to be corrected: and the lenses except the second lens in the lens group to be corrected.
In the embodiment of the present invention, the position coordinates of the imaging point of the first calibration point set on the target paper in the first image may be used to determine the difference of the shooting angles between the lenses in the lens group to be corrected, and based on the difference of the shooting angles between the lenses in the lens group to be corrected, the first lens, which is the other lens except for the lens used as the reference, and the translation correction parameter relative to the second lens, which is the other lens used as the reference, are obtained. In the subsequent image correction process, the image acquired by the first lens is corrected by using the translation correction parameter of the first lens relative to the second lens, so that the obtained corrected image and the image acquired by the second lens have the same shooting angle, that is, the positions of all points in the shot scene in the obtained corrected image are the same as the positions in the image acquired by the second lens. In addition, in the embodiment of the invention, the first calibration point in the target paper can occupy a complete pixel point in the first image, so that the correction parameter determined based on the first image containing the target paper can be more accurate, and the subsequent image corrected by using the correction parameter has better effect.
The communication bus mentioned in the electronic device may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown, but this does not mean that there is only one bus or one type of bus.
The communication interface is used for communication between the electronic equipment and other equipment.
The Memory may include a Random Access Memory (RAM) or a Non-Volatile Memory (NVM), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the processor.
The Processor may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; but also Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components.
Corresponding to the above method embodiments, the present invention provides a computer-readable storage medium, in which a computer program is stored, and the computer program, when executed by a processor, implements any of the above correction parameter obtaining method steps provided by the present invention.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
All the embodiments in the present specification are described in a related manner, and the same and similar parts among the embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the system embodiment, since it is substantially similar to the method embodiment, the description is simple, and for the relevant points, reference may be made to the partial description of the method embodiment.
The above description is only for the preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention shall fall within the protection scope of the present invention.

Claims (10)

1. A method for obtaining correction parameters, the method comprising:
obtaining first images of respective shooting target papers of lenses in a lens group to be corrected, wherein the lenses in the lens group to be corrected are as follows: the method comprises the steps that lenses with overlapped focal length sections exist in multi-lens equipment, pixel points in all first images are the same in size, the focal length and the object distance are the same when the first images are obtained through shooting by the lenses in a lens group to be corrected, and first calibration points are arranged on target paper;
determining the coordinate position of the imaging point of the first calibration point in the first image aiming at the first image corresponding to each lens in the lens group to be corrected;
calculating and obtaining a translation correction parameter of each first lens relative to a second lens based on the coordinate position of the imaging point of the first calibration point in each first image, wherein the second lens is: the first lens is a lens in the lens group to be corrected: and the lenses except the second lens in the lens group to be corrected.
2. The method of claim 1, wherein after the step of calculating a translational correction parameter for each first lens relative to the second lens based on the coordinate position of the imaging point of the first calibration point in each first image, the method further comprises:
obtaining a focal length and an object distance when a first image is obtained by shooting through the lenses in the lens group to be corrected, wherein the focal length and the object distance are used as a first focal length and a first object distance;
and storing the corresponding relation between the translation correction parameter of the first lens relative to the second lens and the first focal length and the first object distance for each first lens in the lens group to be corrected.
3. The method according to claim 1, wherein the step of calculating a translational correction parameter of each first lens with respect to the second lens based on the coordinate position of the imaging point of the first calibration point in each first image comprises:
selecting a lens from the lenses in the lens group to be corrected as a second lens;
calculating the translational correction parameters of each first lens relative to the second lens by adopting the following steps:
determining a position coordinate of an imaging point of the first calibration point in a first image corresponding to a first target lens as a first position coordinate, wherein the first target lens is one of the first lenses;
calculating to obtain a horizontal difference value and a horizontal correction direction between the first position coordinate and the second position coordinate, and a vertical difference value and a vertical correction direction, wherein the second position coordinate is: determining the position coordinate of an imaging point of the first calibration point in a first image corresponding to the second lens;
and determining the calculated horizontal difference value and horizontal correction direction between the first position coordinate and the second position coordinate, and the calculated vertical difference value and vertical correction direction as the translation correction parameters of the first target lens relative to the second lens.
4. The method according to claim 3, wherein the target paper is further provided with at least two second calibration points and at least two third calibration points, wherein a connecting line of the at least two second calibration points is perpendicular to a connecting line of the at least two third calibration points and intersects with the first calibration point;
before the step of determining the calculated horizontal difference and horizontal correction direction, and vertical difference and vertical correction direction between the first position coordinate and the second position coordinate as the translation correction parameters of the first target lens relative to the second lens, the method further includes:
determining position coordinates of imaging points of the at least two second calibration points and imaging points of the at least two third calibration points in the first image corresponding to the first target lens;
determining position coordinates of imaging points of the at least two second calibration points and imaging points of the at least two third calibration points in the first image corresponding to the second lens;
determining a cropping size and a cropping position corresponding to the first target lens based on position coordinates of imaging points of the at least two second calibration points and the at least two third calibration points in the first image corresponding to the first target lens, position coordinates of imaging points of the at least two second calibration points and the at least two third calibration points in the first image corresponding to the second lens, and a preset translation parameter calculation formula;
the step of determining the calculated horizontal difference and horizontal correction direction between the first position coordinate and the second position coordinate, and the calculated vertical difference and vertical correction direction as the translational correction parameter of the first target lens relative to the second lens includes:
and determining the calculated horizontal difference value and horizontal correction direction between the first position coordinate and the second position coordinate, the calculated vertical difference value and vertical correction direction, and the cutting size and cutting position corresponding to the first target lens as the translation correction parameters of the first target lens relative to the second lens.
5. The method according to any one of claims 1 to 4, wherein before the step of obtaining the first image of the target paper taken by each of the lenses of the lens group to be corrected, the method further comprises:
acquiring second images of the lenses in the lens group to be corrected when the lenses respectively shoot target paper, wherein the target paper is also provided with at least two second calibration points and at least two third calibration points, the connecting line of the at least two second calibration points is perpendicular to the connecting line of the at least two third calibration points and intersects with the first calibration point, and the focal lengths of the lenses in the lens group to be corrected when the lenses shoot the second images are the same;
determining position coordinates of an imaging point of the first calibration point, imaging points of at least two second calibration points and imaging points of at least two third calibration points in the second image corresponding to each lens in the lens group to be corrected;
for a second image corresponding to each lens in the lens group to be corrected, calculating to obtain a rotation error of the lens corresponding to the second image based on an imaging point of the first calibration point, respective imaging points of at least two second calibration points, respective position coordinates of respective imaging points of at least two third calibration points, and a preset rotation error calculation formula in the second image;
based on the rotation error of each lens in the lens group to be corrected, calculating to obtain the rotation correction parameters of each third lens relative to the fourth lens, wherein the fourth lens is: one lens of the lens group to be corrected, the third lens is: and the lens groups to be corrected except the fourth lens.
6. The method of claim 5, wherein the first index point is a bulls-eye of the backing paper, at least two of the second index points are located on either side of the first index point, and at least two of the third index points are located on either side of the first index point.
7. The method as claimed in claim 5, wherein after the step of calculating rotation correction parameters of the respective third lenses with respect to the fourth lenses based on the rotation errors of each lens of the lens groups to be corrected, the method further comprises:
obtaining a focal length when a second image is obtained by shooting through the lenses in the lens group to be corrected, and taking the focal length as a second focal length;
and storing the corresponding relation between the rotation correction parameter of the third lens relative to the fourth lens and the second focal length for each third lens in the lens group to be corrected.
8. The method of claim 5, wherein when the second index points are two and the third index points are two;
the preset rotation error calculation formula is as follows:
Figure FDA0001908891670000031
wherein α (N) denotes a rotation error of a lens corresponding to the second image in a case where the focal length is N, and (D)x,Dy) And (F)x,Fy) Position coordinates respectively identifying imaging points of two of the second calibration points in the second image, the (C)x,Cy) And (E)x,Ey) Respectively identifying two of the second imagesPosition coordinates of an imaging point of the third calibration point, said (G)x,Gy) And identifying the position coordinates of the imaging point of the first calibration point in the second image, wherein N is a positive number.
9. The method as claimed in claim 5, wherein the step of calculating rotation correction parameters of the third lens relative to the fourth lens based on the rotation errors of each lens in the lens group to be corrected comprises:
selecting a lens from the lenses in the lens group to be corrected as a fourth lens;
calculating the difference value of the rotation error of the third lens and the rotation error of the fourth lens as the rotation difference value of the third lens aiming at each third lens;
and determining the calculated rotation difference value of the third lens as a rotation correction parameter of the third lens relative to the fourth lens for each third lens.
10. The method of claim 9, wherein before the step of determining the calculated difference of the third lens for each third lens as the rotation correction parameter of the third lens relative to the fourth lens, the method further comprises:
for each third lens, determining a cutting position corresponding to the third lens and a cutting size corresponding to the third lens based on a central point of a second image corresponding to the third lens, a rotation difference value of the third lens and a resolution of the third lens;
the step of determining the calculated difference of the third lens as the rotation correction parameter of the third lens relative to the fourth lens for each third lens includes:
and determining the calculated difference value of the third lens, the cutting position corresponding to the third lens and the cutting size corresponding to the third lens as the rotation correction parameters of the third lens relative to the fourth lens for each third lens.
CN201811544026.7A 2018-12-17 2018-12-17 Correction parameter obtaining method Active CN111343360B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201811544026.7A CN111343360B (en) 2018-12-17 2018-12-17 Correction parameter obtaining method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201811544026.7A CN111343360B (en) 2018-12-17 2018-12-17 Correction parameter obtaining method

Publications (2)

Publication Number Publication Date
CN111343360A CN111343360A (en) 2020-06-26
CN111343360B true CN111343360B (en) 2022-05-17

Family

ID=71186882

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201811544026.7A Active CN111343360B (en) 2018-12-17 2018-12-17 Correction parameter obtaining method

Country Status (1)

Country Link
CN (1) CN111343360B (en)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113592956B (en) * 2021-07-30 2023-12-19 武汉精测电子集团股份有限公司 Multi-lens combined calibration method and device based on microscopic detection machine
CN113752260B (en) * 2021-09-07 2023-12-26 京东方科技集团股份有限公司 Material taking positioning correction method and device
CN117014733B (en) * 2023-10-08 2024-04-12 荣耀终端有限公司 Shooting correction method, device and equipment

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104835159A (en) * 2015-05-07 2015-08-12 中国科学院长春光学精密机械与物理研究所 Digital image correction method for continuous variable-focal-length optical imaging system
CN106454015A (en) * 2015-08-04 2017-02-22 宁波舜宇光电信息有限公司 Image synthesis method for multi-lens camera module
TW201800822A (en) * 2016-06-24 2018-01-01 聚晶半導體股份有限公司 Method and system for multi-lens module alignment

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6603885B1 (en) * 1998-04-30 2003-08-05 Fuji Photo Film Co., Ltd. Image processing method and apparatus
US20180007345A1 (en) * 2015-01-16 2018-01-04 Imra Europe S.A.S. Self-rectification of stereo camera

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104835159A (en) * 2015-05-07 2015-08-12 中国科学院长春光学精密机械与物理研究所 Digital image correction method for continuous variable-focal-length optical imaging system
CN106454015A (en) * 2015-08-04 2017-02-22 宁波舜宇光电信息有限公司 Image synthesis method for multi-lens camera module
TW201800822A (en) * 2016-06-24 2018-01-01 聚晶半導體股份有限公司 Method and system for multi-lens module alignment

Also Published As

Publication number Publication date
CN111343360A (en) 2020-06-26

Similar Documents

Publication Publication Date Title
JP6263623B2 (en) Image generation method and dual lens apparatus
US9316840B2 (en) Methods and apparatus for reducing plenoptic camera artifacts
WO2018214365A1 (en) Image correction method, apparatus, device, and system, camera device, and display device
US9325899B1 (en) Image capturing device and digital zooming method thereof
WO2021208371A1 (en) Multi-camera zoom control method and apparatus, and electronic system and storage medium
US7962033B2 (en) Methods and apparatus for full-resolution light-field capture and rendering
CN109474780B (en) Method and device for image processing
US20180213217A1 (en) Equipment and method for promptly performing calibration and verification of intrinsic and extrinsic parameters of a plurality of image capturing elements installed on electronic device
CN111343360B (en) Correction parameter obtaining method
CN109409147B (en) Bar code recognition method and device
JP2010226157A (en) Camera, and image correction method
CN110099220B (en) Panoramic stitching method and device
WO2018228466A1 (en) Focus region display method and apparatus, and terminal device
CN111279393A (en) Camera calibration method, device, equipment and storage medium
US20070024714A1 (en) Whiteboard camera apparatus and methods
CN109087253B (en) Image correction method and device
CN105472263B (en) Image acquisition method and the image capture equipment for using the method
CN110136205B (en) Parallax calibration method, device and system of multi-view camera
JP2013126135A (en) Stereo image generation device, stereo image generation method and computer program for stereo image generation
CN114363522A (en) Photographing method and related device
CN112422942B (en) White balance synchronization method, lens module and electronic equipment
JP6283329B2 (en) Augmented Reality Object Recognition Device
JP5964772B2 (en) Lens information registration system, lens information server and camera body used for lens information registration system
CN206364891U (en) High definition linear array panoramic camera
CN110581977B (en) Video image output method and device and three-eye camera

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant