CN114494034A - Image distortion correction method, device and equipment - Google Patents
Image distortion correction method, device and equipment Download PDFInfo
- Publication number
- CN114494034A CN114494034A CN202111571459.3A CN202111571459A CN114494034A CN 114494034 A CN114494034 A CN 114494034A CN 202111571459 A CN202111571459 A CN 202111571459A CN 114494034 A CN114494034 A CN 114494034A
- Authority
- CN
- China
- Prior art keywords
- distortion correction
- foreground object
- image
- pixel coordinates
- preset
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012937 correction Methods 0.000 title claims abstract description 321
- 238000000034 method Methods 0.000 title claims abstract description 82
- 238000004590 computer program Methods 0.000 claims description 16
- 238000013507 mapping Methods 0.000 claims description 11
- 238000003860 storage Methods 0.000 claims description 6
- 238000006243 chemical reaction Methods 0.000 claims description 5
- 239000000126 substance Substances 0.000 claims description 4
- 230000001131 transforming effect Effects 0.000 claims description 4
- 230000000007 visual effect Effects 0.000 abstract description 5
- 230000000875 corresponding effect Effects 0.000 description 28
- 230000008569 process Effects 0.000 description 15
- 238000010586 diagram Methods 0.000 description 10
- 230000006870 function Effects 0.000 description 8
- 238000012545 processing Methods 0.000 description 6
- 230000009466 transformation Effects 0.000 description 5
- 230000008859 change Effects 0.000 description 4
- 238000003384 imaging method Methods 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 4
- 230000004075 alteration Effects 0.000 description 3
- 238000013528 artificial neural network Methods 0.000 description 3
- 238000001514 detection method Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000011218 segmentation Effects 0.000 description 3
- 238000011161 development Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 238000003702 image correction Methods 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 241001465754 Metazoa Species 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 210000000746 body region Anatomy 0.000 description 1
- 230000001427 coherent effect Effects 0.000 description 1
- 230000003138 coordinated effect Effects 0.000 description 1
- 210000003128 head Anatomy 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/80—Geometric correction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/08—Projecting images onto non-planar surfaces, e.g. geodetic screens
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/18—Image warping, e.g. rearranging pixels individually
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
Abstract
The invention discloses an image distortion correction method, a device and equipment, wherein the method comprises the following steps: acquiring an image to be corrected, and determining pixel coordinates of a foreground object in the image to be corrected; determining at least one distortion correction grid corresponding to the pixel coordinates of the foreground object in a preset distortion correction network; the preset distortion correction network and the image to be corrected have the same size, the distortion correction network comprises a plurality of distortion correction grids, and the distortion correction grids are combined in an overlapping mode; and correcting the pixel coordinates of the foreground object in the corresponding distortion correction grid according to a preset correction model to obtain the pixel coordinates of the corrected foreground object. The correction method has robustness and universality, and by presetting the correction model, the foreground object does not need to be segmented, and the distortion state of the foreground object does not need to be judged, so that the performance of image distortion correction is improved, and good visual experience is provided for a user.
Description
Technical Field
The invention relates to the technical field of internet, in particular to an image distortion correction method, device and equipment.
Background
With the development of information technology, the demand of people for camera shooting of terminal equipment is rapidly increased, so that the rapid development of camera functions is promoted, and the camera shooting function of intelligent terminals including mobile phones and tablets is more and more powerful, and becomes one of the important functions of the terminal equipment. Among them, the lens field angle of the terminal device greatly affects the image capturing function, but the problem of distortion of the captured image is more and more significant as the lens field angle is larger and larger.
Disclosure of Invention
The invention provides an image distortion correction method, device and equipment, the method can directly finish the correction process to foreground objects of different forms by performing image distortion correction on a preset distortion correction grid, and a preset correction model does not need to be reset according to the change of the form of the foreground object, so that the method has stronger robustness and universality.
In a first aspect, an embodiment of the present invention provides an image distortion correction method, including:
acquiring pixel coordinates of a foreground object in an image to be corrected;
determining at least one distortion correction grid corresponding to the pixel coordinates of the foreground object in a preset distortion correction network; the preset distortion correction network and the image to be corrected have the same size, the distortion correction network comprises a plurality of distortion correction grids, and the distortion correction grids are combined in an overlapping mode;
and correcting the pixel coordinates of the foreground object in the corresponding distortion correction grid according to a preset correction model to obtain the pixel coordinates of the corrected foreground object.
In one or some optional embodiments, the determining at least one distortion correction mesh to which the pixel coordinates of the foreground object correspond in a preset distortion correction mesh includes:
acquiring a pixel coordinate set of each distortion correction grid in the preset distortion correction network;
for each distortion correction network, determining that the pixel coordinates of the foreground object correspond to the distortion correction mesh based on the pixel coordinates of the foreground object belonging to a set of pixel coordinates of the corresponding distortion correction mesh.
In one or some optional embodiments, the obtaining a set of pixel coordinates of each of the orthotics meshes in the preset orthotics network includes:
acquiring original pixel coordinates of each pixel point in the preset distortion correction network;
for each of the preset orthotics meshes:
and transforming the original pixel coordinates of each pixel point in the distortion correction grid, determining the transformed pixel coordinates of each pixel point, and obtaining a pixel coordinate set of the distortion correction grid.
In one or some alternative embodiments, the method further comprises:
determining the original pixel coordinates of a foreground object in an image to be corrected;
and carrying out coordinate conversion on the original pixel coordinates of the foreground object in the image to obtain the pixel coordinates of the foreground object in the image.
In one or some alternative embodiments, the original pixel coordinates are coordinate-transformed using the following equation 1:
wherein xi represents the row coordinate of the original pixel coordinate, yi represents the column coordinate of the original pixel coordinate;
w represents the width of the distortion correction mesh, and h represents the height of the distortion correction mesh;
u0i denotes the row coordinates of the pixel coordinates of the foreground object and v0i denotes the column coordinates of the pixel coordinates of the foreground object.
In one or some optional embodiments, the preset correction model is a spherical projection model; the correcting the pixel coordinates of the foreground object according to a preset correction model to obtain the pixel coordinates of the corrected foreground object includes:
substituting the pixel coordinates of the foreground object into a spherical mapping relation formula of the spherical projection model to obtain pixel coordinates of the corrected foreground object;
the spherical projection model has the following spherical mapping relation formula:
wherein the content of the first and second substances,
θ0＝atan(v0i/u0i) U0i denotes row coordinates of pixel coordinates of the foreground object, v0i denotes column coordinates of pixel coordinates of the foreground object, u1i denotes row coordinates of pixel coordinates of the foreground object after correction, v1i denotes column coordinates of pixel coordinates of the foreground object after correction, γiIs the scaling factor of the distortion correction mesh.
In one or some alternative embodiments, the method further comprises: obtaining the scaling coefficient gamma of each distortion correction network in the preset distortion correction networki：
Acquiring distortion parameters of a camera corresponding to the preset distortion correction network;
mapping the distortion parameter of the camera to the preset distortion correction network to obtain the distortion parameter value of the central position of each distortion correction grid in the preset distortion correction network;
obtaining the zoom factor gamma of each distortion correction network according to the distortion parameter value and the preset correction coefficient value of the central position of each distortion correction networki。
In one or some alternative embodiments, the preset correction model is a cylindrical projection model or a perspective projection model.
In one or some optional embodiments, before acquiring the pixel coordinates of the foreground object in the image to be corrected, the method further comprises:
judging whether the image to be corrected comprises a foreground object or not;
and if so, determining the pixel coordinates of the foreground object in the image to be corrected.
In one or some alternative embodiments, the shape of each of the predetermined distortion correction meshes is an ellipse, a circle, a square or a rectangle.
In a second aspect, an embodiment of the present invention provides an image distortion correcting apparatus, including:
the coordinate determination module is used for acquiring the pixel coordinates of the foreground object in the image to be corrected;
the matching module is used for determining at least one distortion correction grid corresponding to the pixel coordinates of the foreground object in a preset distortion correction network; the preset distortion correction network and the image to be corrected have the same size, the distortion correction network comprises a plurality of distortion correction grids, and the distortion correction grids are combined in an overlapping mode;
and the correction module is used for correcting the pixel coordinates of the foreground object in the corresponding correction grid according to a preset correction model to obtain the pixel coordinates of the corrected foreground object.
In a third aspect, an embodiment of the present invention provides an image distortion correcting apparatus including the above-described image distortion correcting device.
In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored, which when executed by a processor implements the image distortion correction method as described above.
In a fifth aspect, an embodiment of the present invention provides a computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor executes the computer program to implement the image distortion correction method as described above.
The technical scheme provided by the embodiment of the invention has the beneficial effects that at least:
the image distortion correction method provided by the invention identifies the pixel coordinates of the foreground object in the image to be corrected, determines the distortion correction grid corresponding to the pixel coordinates of the foreground object, completes the pixel coordinate correction of the foreground object in the corresponding distortion correction grid according to the preset correction model, and finally obtains the complete corrected image. The image distortion correction is carried out on the preset distortion correction grid, the correction process can be directly completed on the foreground objects in different forms, and the preset correction model does not need to be reset according to the change of the forms of the foreground objects, so that the method has stronger robustness and universality. Compared with the image distortion correction method in the prior art, the method has the advantages that the foreground object does not need to be segmented, the distortion state of the foreground object does not need to be judged, the performance of image distortion correction is improved, the speed and the efficiency of image distortion correction are improved, and the image distortion correction can be carried out in real time, so that the stretching deformation of the foreground object is prevented timely, the foreground object is guaranteed to have good presentation effects under different field angles, the real state of the foreground object is highly restored, and good visual experience is provided for users.
Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.
The technical solution of the present invention is further described in detail by the accompanying drawings and embodiments.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the principles of the invention and not to limit the invention. In the drawings:
fig. 1 is a schematic flowchart of an image distortion correction method provided in an embodiment of the present invention;
fig. 2 is a schematic structural diagram of a preset distortion correction grid according to an embodiment of the present invention;
FIG. 3 is a flowchart illustrating another image distortion correction method according to an embodiment of the present invention;
fig. 4 is a schematic structural diagram of an image distortion correction apparatus provided in an embodiment of the present invention.
Detailed Description
Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the exemplary embodiments below are not intended to represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.
The research of the application shows that the main reason for the image distortion is that the large-view-field short-focal-length optical system does not satisfy the pinhole model, the focal height of the image surface is inconsistent with the ideal image height after the light passes through the camera module, so that the deviation exists between the actual imaging point and the ideal imaging point, and the deformation of geometric shapes such as stretching and distortion can occur in the imaging content. For example, when a mobile phone camera, especially a wide-angle camera, is used to shoot a portrait of multiple people, the area with a large field angle, that is, the face at the edge of the picture, may have an obvious stretching phenomenon compared with the face at the center of the picture, or the legs and arms of people at the edge may be obviously thickened, and the imaging effect is not good, so that the distortion of the image or video needs to be corrected, and the true state of the portrait is restored to the maximum extent, so that the image or video better conforms to the vision habit of human eyes, and the user has good visual experience.
There is an image distortion correction method that requires first identifying a foreground object, including a human image, and performing an algorithm constraint correction process on a foreground object region and a background region. In the constraint of the foreground object region, the shape of the foreground object is corrected based on different projection algorithms, for example, the foreground object is a portrait, and the shape of the head and body regions of the portrait can be corrected by adopting different projection algorithms respectively; and, through the constraint of the background area, the image content of the background area is ensured not to cause background distortion or faults due to foreground correction.
The invention researches and finds that the image distortion correction method can solve the problem of deformation or distortion of a foreground object and can realize the natural, coherent and coordinated effects of the foreground and the background to a certain extent, but the method needs to identify and separate the foreground and the background firstly and then carries out correction processing based on the extracted distortion state of the foreground object, so that the efficiency is not high; in addition, the method needs to establish different image distortion correction models for each group of images, and has no universality, so that the robustness of the image distortion correction method is not strong.
The invention provides an image distortion correction method, device and equipment, the method can directly finish the correction process to foreground objects of different forms by performing image distortion correction on a preset distortion correction grid, and a preset correction model does not need to be reset according to the change of the form of the foreground object, so that the method has stronger robustness and universality.
Referring to fig. 1, an image distortion correction method according to an embodiment of the present invention includes the following steps:
s101: and acquiring the pixel coordinates of the foreground object in the image to be corrected.
In the embodiment of the present invention, the image to be corrected may be an image directly captured by a camera or an image in a video captured by a camera, or may be an image obtained from another original fixed carrier, such as an optical disc, a hard disk, or a cloud, and the size of the image to be corrected may be a preset size. The image to be corrected may include a foreground object region and a background region, where an image of the foreground object region is a foreground object. The number of foreground objects in the image to be corrected may be one or more. The foreground object in the image to be corrected can be obtained by means of the prior art, for example, a neural network detection technique is used to determine the foreground object in the image to be corrected. The foreground object in the image to be corrected may include a human image, an animal image, a plant image, or other detectable objects. For example, when the foreground object is a human image, a Multi-task Convolutional Deep Neural Network (Multi-task Convolutional Deep Neural Network) model can be used to perform face region detection and face keypoint detection on the image to be corrected, so as to identify the foreground object. In the image to be corrected, the part except the foreground object is the background, such as a mountain, the sky, a building, an indoor or outdoor environment, and the like. The background is typically farther from the camera in object space than the foreground objects. Accordingly, foreground objects are typically closer to the camera in object space than to the background.
In the embodiment of the present invention, the pixel coordinate in the image to be corrected is the pixel coordinate of each pixel point in the image to be corrected, and then the pixel coordinate of the foreground object is the coordinate of the pixel point corresponding to the foreground object in the image to be corrected.
S102: and determining at least one distortion correction mesh corresponding to the pixel coordinates of the foreground object in a preset distortion correction network.
In the embodiment of the present invention, before the image distortion correction method is executed, a distortion correction network may be set in advance. The size of the preset distortion correction network may be predetermined according to the size of the image to be corrected, and assuming that the size of the image to be corrected is W × H, the size of the preset distortion correction network may also be W × H, where W represents the width of the image to be corrected, and H represents the length of the image to be corrected. The preset distortion correction network may preset a plurality of distortion correction meshes, and the plurality of distortion correction meshes are overlapped and combined. Wherein the shape of the distortion correction mesh may be elliptical, circular, square or rectangular, among other shapes known in the art. Referring to fig. 2, the aberration correcting network includes a plurality of overlapping elliptical aberration correcting meshes. The smaller the size of the distortion correction mesh in the distortion correction network is, the finer the distortion correction can be, but the smaller the size of the distortion correction mesh is, the greater the number of the distortion correction meshes in the distortion correction network is, and the performance consumption in the distortion correction process can be increased.
After the distortion correction network and the plurality of distortion correction networks thereof are set, the pixel coordinates of each distortion correction mesh in the preset distortion correction network can be obtained according to the pixel coordinate determination method in the prior art, so that at least one distortion correction mesh corresponding to the pixel coordinates of the foreground object in the preset distortion correction network can be determined based on the comparison between the pixel coordinates of each foreground object and the pixel coordinates in each distortion correction network. In the embodiment of the present invention, since the distortion correction meshes in the preset distortion correction network are arranged in an overlapping combination, the pixel coordinates of the foreground object may correspond to one or more distortion correction meshes, and certainly, due to the limitation of the size and the number of overlapping layers of the distortion correction meshes in the distortion correction network, the one or more pixel coordinates of the foreground object may not be in any distortion correction mesh, which does not affect the execution of the image distortion correction method proposed by the present invention.
S103: and correcting the pixel coordinates of the foreground object in the corresponding distortion correction grid according to a preset correction model to obtain the pixel coordinates of the corrected foreground object.
After the distortion correction mesh corresponding to the pixel coordinate of the foreground object is determined in step S102, the pixel coordinate of the foreground object may be corrected in each distortion correction mesh according to the preset correction model of the distortion correction mesh, so as to obtain the pixel coordinate of the corrected foreground object. And when the pixel coordinates of the corrected foreground objects in all the distortion correction grids are obtained, completing the image distortion correction process of the image to be corrected to obtain the corrected image.
The image distortion correction method provided by the invention can be applied to any electronic equipment, including but not limited to various smart phones, digital cameras, personal computers, notebook computers, tablet computers and the like. The electronic equipment is provided with a camera, the electronic equipment obtains an image to be corrected through real-time shooting of the camera, and executes the image distortion correction method of the embodiment of the application on the image to be corrected so as to perform distortion correction on the image to be corrected and obtain a corrected image.
In the embodiment of the invention, because the distortion correction network and each distortion correction grid thereof are preset, after the foreground image is obtained, the foreground image does not need to be segmented, namely, different objects in the foreground object do not need to be identified separately, and only the distortion correction grid of each pixel point comprising the foreground object needs to be subjected to pixel coordinate correction, so that the distortion correction of the foreground object can be completed, and the corrected image is obtained. For example, assuming that the image to be corrected is a group photo portrait image, only a plurality of portraits in the identified foreground object are needed, and different distortion correction meshes corresponding to the foreground object are determined, and it is not necessary to segment different portraits based on a portrait segmentation technology, and as long as the pixel coordinates of the foreground object including the plurality of portraits are obtained, the pixel coordinates can be corrected in the corresponding distortion correction meshes, and finally the corrected portrait image is obtained. Therefore, the problem of inaccurate image correction caused by segmentation errors during object segmentation is avoided, and the performance loss of distortion correction is reduced.
The image distortion correction method provided by the invention identifies the pixel coordinates of the foreground object in the image to be corrected, determines the distortion correction grid corresponding to the pixel coordinates of the foreground object, completes the pixel coordinate correction of the foreground object in the corresponding distortion correction grid according to the preset correction model, and finally obtains the complete corrected image. The image distortion correction is carried out on the preset distortion correction grid, the correction process can be directly completed on the foreground objects in different forms, and the preset correction model does not need to be reset according to the change of the forms of the foreground objects, so that the method has stronger robustness and universality. Compared with the image distortion correction method in the prior art, the foreground object does not need to be segmented, the distortion state of the foreground object does not need to be judged, the performance of image distortion correction is improved, the speed and the efficiency of image distortion correction are improved, and the image distortion correction can be carried out in real time, so that the stretching deformation of the foreground object is prevented in time, the good presentation effect of the foreground object under different field angles is ensured, the real state of the foreground object is highly restored, and good visual experience is provided for a user.
In a specific embodiment, in step S101, the pixel coordinates of the foreground object in the image to be corrected can be obtained by:
performing coordinate conversion on the original pixel coordinates of the foreground object in the image to be corrected based on the following formula 1 to obtain the pixel coordinates of the foreground object in the image:
wherein: xi represents the row coordinates of the original pixel coordinates of the foreground object, yi represents the column coordinates of the original pixel coordinates of the foreground object;
w represents the width of the distortion correction mesh, and h represents the height of the distortion correction mesh;
u0i denotes row coordinates of pixel coordinates of the foreground object, and v0i denotes column coordinates of pixel coordinates of the foreground object.
In the embodiment of the present invention, the determining method of the original pixel coordinate of the image to be corrected may adopt a method in the prior art. For example, a certain corner point or a center point in the image to be corrected may be used as an origin, and a horizontal coordinate axis and a vertical coordinate axis may be set to obtain a row coordinate and a column coordinate of the original pixel coordinate, thereby obtaining the original pixel coordinate. For example, the coordinates of the corner point at the top left corner in the image to be corrected are (0, 0), the original pixel coordinates of the pixel point adjacent to the right side of the corner point are (1, 0), the original pixel coordinates of the pixel point adjacent to the lower side of the corner point are (0, 1), and so on, since all foreground objects in the image to be corrected have been determined before, the original pixel coordinates of each foreground object can be obtained.
In the embodiment of the present invention, before the step of performing the coordinate transformation to obtain the pixel coordinates of the foreground object, the preset distortion correction network and each distortion correction mesh therein are obtained in advance, and therefore, the step of performing the coordinate transformation may be performed based on the above formula 1. And performing coordinate transformation on the original pixel coordinates of the foreground object so as to perform the subsequent coordinate correction step.
Correspondingly, the specific process of executing the step S102 to determine at least one distortion correction mesh corresponding to the pixel coordinates of the foreground object in the preset distortion correction mesh may include the following steps:
acquiring a pixel coordinate set of each distortion correction grid in the preset distortion correction network;
for each distortion correction network, determining that the pixel coordinates of the foreground object correspond to the distortion correction mesh based on the pixel coordinates of the foreground object belonging to a set of pixel coordinates of the corresponding distortion correction mesh.
Wherein it may be determined that the pixel coordinates of the foreground object belong to a set of pixel coordinates of a corresponding distortion correction mesh by:
and judging whether the pixel coordinate of the foreground object belongs to the pixel coordinate set of the distortion correction grid or not, and if so, determining that the pixel coordinate of the foreground object belongs to the pixel coordinate set of the corresponding distortion correction grid.
In the embodiment of the present invention, after obtaining the preset distortion correction network and each distortion correction mesh thereof, the pixel coordinate set of each distortion correction mesh may be predetermined in the following manner:
acquiring original pixel coordinates of each pixel point in a preset distortion correction network;
for each of the preset distortion correction meshes:
and transforming the original pixel coordinates of each pixel point in the distortion correction grid according to the formula 1, and determining the transformed pixel coordinates of each pixel point to obtain a pixel coordinate set of the distortion correction grid.
In the embodiment of the present invention, the determination manner of the original pixel coordinate of each pixel point in the preset distortion correction network is similar to the determination manner of the original pixel coordinate of the image to be corrected, and the specific implementation process may refer to the detailed description of the determination manner of the original pixel coordinate of the image to be corrected, which is not described herein again.
In the embodiment of the present invention, it is assumed that the preset distortion correction network includes N distortion correction meshes, and the pixel coordinate set Q0 of the ith distortion correction mesh isNi(xi,yi)＝[u0i(xi,yi),v0i(xi,yi)]TWherein, U0i (x)i,yi) Line coordinates of a pixel point having original pixel coordinates (xi, yi) in the distortion correction mesh, v0i (x)i,yi) And the column coordinates of a pixel point of which the original pixel coordinate is (xi, yi) in the distortion correction grid are expressed.
Because the position of each distortion correction grid in the preset distortion correction network is determined, after the original pixel coordinates of each pixel point in the preset distortion correction network are obtained, all the pixel points and the original pixel coordinates thereof in each distortion correction grid can be determined according to the position of each pixel point.
The process of obtaining the pixel coordinates of each pixel point in the distortion correction grid by performing coordinate transformation on the pixel points in each distortion correction grid is similar to the process of determining the pixel coordinates of the foreground object, and is not repeated here. And when the pixel coordinate of each pixel point in the distortion correction grid is obtained according to the formula 1, obtaining a pixel coordinate set of the distortion correction grid.
In an embodiment of the present invention, the preset correction model may be a spherical projection model; correspondingly, the specific implementation process of correcting the pixel coordinates of the foreground object according to the preset correction model in the corresponding distortion correction mesh described in the step S103 to obtain the pixel coordinates of the corrected foreground object may include the following steps:
in the distortion correction mesh, substituting the pixel coordinates of the foreground object into a spherical mapping relation formula 2 of the spherical projection model to obtain the pixel coordinates of the corrected foreground object:
wherein the content of the first and second substances,
θ0＝atan(v0i/u0i) U0i denotes row coordinates of pixel coordinates of the foreground object, v0i denotes column coordinates of pixel coordinates of the foreground object, u1i denotes row coordinates of pixel coordinates of the foreground object after correction, v1i denotes column coordinates of pixel coordinates of the foreground object after correction, γiAnd the scaling coefficient of the ith distortion correction grid in the preset distortion correction network.
Wherein, the preset distortion correction network has a scaling factor gamma of each distortion correction networkiThe method can be specifically obtained by the following steps:
acquiring distortion parameters of a camera corresponding to the preset distortion correction network;
mapping the distortion parameter of the camera to the preset distortion correction network to obtain the distortion parameter value of the central position of each distortion correction grid in the preset distortion correction network;
obtaining the zoom factor gamma of each distortion correction network according to the distortion parameter value and the preset correction coefficient value of the central position of each distortion correction networki。
In the embodiment of the present invention, the distortion parameter of the camera may be directly obtained from a camera manufacturer, or may be obtained by calibrating the camera according to a camera calibration method in the prior art. The camera calibration method may be a linear calibration method, a nonlinear optimization calibration method, a zhangying calibration method, or other common calibration methods, and the camera calibration method in the embodiment of the present invention may not be specifically limited as long as the distortion parameter of the camera can be obtained.
In the embodiment of the invention, because the distortion rates of different positions of the images shot by the camera are different, the smaller the distortion rate of the position close to the optical center of the camera is, the distortion parameter of the camera corresponding to the preset distortion correction network comprises the distortion parameter value of each pixel point position in the preset distortion correction network, the distortion parameter value of different pixel point positions can be obtained by obtaining the distortion parameter of the camera, and further the distortion parameter value of the center position of each distortion correction network can be obtained. The scaling coefficient of each distortion correction grid can be obtained by multiplying the distortion parameter value of the center position of the distortion correction grid with the preset correction coefficient value. Since the distortion rate is smaller near the position in the camera optics, the scaling factor of the distortion correction mesh closer to the center of the preset distortion correction network is smaller, that is, the scaling factor of the distortion correction mesh closer to the edge of the preset distortion correction network is larger than that of the distortion correction mesh closer to the center of the preset distortion correction network.
In the embodiment of the present invention, since the scaling coefficient of each distortion correction mesh in the preset distortion correction network is determined according to the distortion parameter of the camera, the difference of the scaling coefficients of the distortion correction meshes at adjacent or overlapping positions approaches the difference of the distortion parameter values of adjacent regions in the distortion correction network. Therefore, the pixel coordinates of the foreground object are corrected by each distortion correction grid, the adjacent area of the foreground object in the corrected image can realize smooth transition, and the problem that the corrected image is burred or the image is not connected is avoided.
In one embodiment, the preset correction model may also adopt other stereoscopic projection models in the prior art, such as a cylindrical projection model or a perspective projection model. Based on the above detailed description about the spherical projection model, those skilled in the art may correct the pixel coordinates of the foreground object in the distortion correction mesh by using a cylindrical projection model or a perspective projection model, to obtain the pixel coordinates of the corrected foreground object, so as to obtain a corrected image. The specific implementation of the cylindrical projection model or the perspective projection model can be combined with the spherical projection model and the related description in the prior art, and will not be described herein again.
In one embodiment, referring to fig. 3, before performing step S101, the image distortion correcting method may further include:
s100: judging whether the image to be corrected comprises a foreground object or not;
if yes, executing step S101; if not, the image distortion correction is finished.
Since some images may not have foreground objects but only contain background, when it is determined that the image to be corrected does not include foreground objects, the distortion correction process is stopped, and thus, the time for performing image correction processing can be saved. And system resources are saved. Certainly, if the image to be corrected does not include a foreground object, the background in the image to be corrected may also be corrected by using an image distortion correction method in the prior art to obtain an image with a better visual effect, and the specific implementation process may refer to detailed description in the prior art and is not described herein again.
Based on the same inventive concept, embodiments of the present invention further provide an image distortion correction apparatus, and as the principle of the problem solved by the apparatus is similar to that of the image distortion correction method, the implementation of the apparatus may refer to the implementation of the image distortion correction method, and repeated details are omitted.
Referring to fig. 4, an image distortion correcting apparatus according to an embodiment of the present invention includes: a coordinate determination module 100, a matching module 200 and a rectification module 300; wherein:
a coordinate determination module 100, configured to obtain pixel coordinates of a foreground object in an image to be corrected;
a matching module 200, configured to determine at least one distortion correction mesh corresponding to the pixel coordinate of the foreground object in a preset distortion correction network; the preset distortion correction network and the image to be corrected have the same size, the distortion correction network comprises a plurality of distortion correction grids, and the distortion correction grids are combined in an overlapping mode;
and the correcting module 300 is configured to correct the pixel coordinates of the foreground object according to a preset correcting model in the corresponding correcting grid, so as to obtain the pixel coordinates of the corrected foreground object.
In one or some optional embodiments, the matching module 200 is specifically configured to obtain a pixel coordinate set of each orthotics mesh in the preset orthotics network;
for each distortion correction network, determining that the pixel coordinates of the foreground object correspond to the distortion correction mesh based on the pixel coordinates of the foreground object belonging to a set of pixel coordinates of the corresponding distortion correction mesh.
In one or some optional embodiments, the matching module 200 is specifically configured to obtain an original pixel coordinate of each pixel point in the preset distortion correction network;
for each of the preset orthotics meshes:
and transforming the original pixel coordinates of each pixel point in the distortion correction grid, determining the transformed pixel coordinates of each pixel point, and obtaining a pixel coordinate set of the distortion correction grid.
In one or some optional embodiments, the coordinate determination module 100 is specifically configured to:
determining the original pixel coordinates of a foreground object in an image to be corrected;
and carrying out coordinate conversion on the original pixel coordinates of the foreground object in the image to obtain the pixel coordinates of the foreground object in the image.
In one or some alternative embodiments, the coordinate determination module 100 is specifically configured to perform coordinate transformation on the original pixel coordinates by using the following formula 1:
wherein xi represents the row coordinate of the original pixel coordinate, yi represents the column coordinate of the original pixel coordinate;
w represents the width of the distortion correction mesh, and h represents the height of the distortion correction mesh;
u0i denotes the row coordinates of the pixel coordinates of the foreground object and v0i denotes the column coordinates of the pixel coordinates of the foreground object.
In one or some optional embodiments, the preset correction model is a spherical projection model; the orthotic module 300 is specifically configured to:
substituting the pixel coordinates of the foreground object into a spherical mapping relation formula of the spherical projection model to obtain the pixel coordinates of the corrected foreground object;
the spherical projection model has the following spherical mapping relation formula:
wherein the content of the first and second substances,
θ0＝atan(v0i/u0i) U0i denotes row coordinates of pixel coordinates of the foreground object, v0i denotes column coordinates of pixel coordinates of the foreground object, u1i denotes row coordinates of pixel coordinates of the foreground object after correction, v1i denotes column coordinates of pixel coordinates of the foreground object after correction, γiIs the scaling factor of the distortion correction mesh.
In one or some alternative embodiments, the correcting module 300 is further configured to obtain the scaling factor γ of each of the preset aberration correcting networks by the following methodi：
Acquiring distortion parameters of a camera corresponding to the preset distortion correction network;
mapping the distortion parameter of the camera to the preset distortion correction network to obtain the distortion parameter value of the central position of each distortion correction grid in the preset distortion correction network;
obtaining the zoom factor gamma of each distortion correction network according to the distortion parameter value and the preset correction coefficient value of the central position of each distortion correction networki。
In one or some optional embodiments, the image distortion correction apparatus further includes a foreground object determining module, configured to determine whether a foreground object is included in the image to be corrected before obtaining pixel coordinates of the foreground object in the image to be corrected;
and if so, determining the pixel coordinates of the foreground object in the image to be corrected.
In one or some alternative embodiments, the shape of each of the predetermined network of orthoses is an ellipse, a circle, a square or a rectangle.
Embodiments of the present invention also provide a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the method for correcting image distortion is implemented.
Embodiments of the present invention further provide a computer device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, the image distortion correction method is implemented.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present invention without departing from the spirit and scope of the invention. Thus, if such modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include such modifications and variations.
Claims (10)
1. An image distortion correction method, comprising:
acquiring pixel coordinates of a foreground object in an image to be corrected;
determining at least one distortion correction grid corresponding to the pixel coordinates of the foreground object in a preset distortion correction network; the preset distortion correction network and the image to be corrected have the same size, the distortion correction network comprises a plurality of distortion correction grids, and the distortion correction grids are combined in an overlapping mode;
and correcting the pixel coordinates of the foreground object in the corresponding distortion correction grid according to a preset correction model to obtain the pixel coordinates of the corrected foreground object.
2. The image distortion correction method of claim 1, wherein the determining at least one distortion correction mesh to which the pixel coordinates of the foreground object correspond in a preset distortion correction mesh comprises:
acquiring a pixel coordinate set of each distortion correction grid in the preset distortion correction network;
for each distortion correction network, determining that the pixel coordinates of the foreground object correspond to the distortion correction mesh based on the pixel coordinates of the foreground object belonging to a set of pixel coordinates of the corresponding distortion correction mesh.
3. The method for image distortion correction according to claim 2, wherein the obtaining a set of pixel coordinates of each distortion correction mesh in the preset distortion correction mesh comprises:
acquiring original pixel coordinates of each pixel point in the preset distortion correction network;
for each of the preset orthotics meshes:
and transforming the original pixel coordinates of each pixel point in the distortion correction grid, determining the transformed pixel coordinates of each pixel point, and obtaining a pixel coordinate set of the distortion correction grid.
4. The image distortion correction method according to any one of claims 1 to 3, wherein the obtaining pixel coordinates of a foreground object in the image to be corrected includes:
determining the original pixel coordinates of a foreground object in an image to be corrected;
and carrying out coordinate conversion on the original pixel coordinates of the foreground object in the image to obtain the pixel coordinates of the foreground object in the image.
5. The image distortion correction method according to any one of claims 1 to 4, wherein the coordinates of the original pixel are subjected to coordinate conversion using the following formula 1:
wherein xi represents the row coordinate of the original pixel coordinate, yi represents the column coordinate of the original pixel coordinate;
w represents the width of the distortion correction mesh, and h represents the height of the distortion correction mesh;
u0i denotes the row coordinates of the pixel coordinates of the foreground object and v0i denotes the column coordinates of the pixel coordinates of the foreground object.
6. The image distortion correction method according to any one of claims 1 to 5, wherein the preset correction model is a spherical projection model; the correcting the pixel coordinates of the foreground object according to a preset correction model to obtain the pixel coordinates of the corrected foreground object includes:
substituting the pixel coordinates of the foreground object into a spherical mapping relation formula of the spherical projection model to obtain the pixel coordinates of the corrected foreground object;
the spherical projection model has the following spherical mapping relation formula:
wherein the content of the first and second substances,
θ0＝atan(v0i/u0i) U0i denotes row coordinates of pixel coordinates of the foreground object, v0i denotes column coordinates of pixel coordinates of the foreground object, u1i denotes row coordinates of pixel coordinates of the foreground object after correction, v1i denotes column coordinates of pixel coordinates of the foreground object after correction, γiIs the scaling factor of the distortion correction mesh.
7. The image distortion correction method of claim 6, further comprising: obtaining the distortion of the preset distortion correction network by the following methodScaling factor gamma of variable correction networki：
Acquiring distortion parameters of a camera corresponding to the preset distortion correction network;
mapping the distortion parameter of the camera to the preset distortion correction network to obtain the distortion parameter value of the central position of each distortion correction grid in the preset distortion correction network;
obtaining the zoom factor gamma of each distortion correction network according to the distortion parameter value and the preset correction coefficient value of the central position of each distortion correction networki。
8. An image distortion correction apparatus, comprising:
the coordinate determination module is used for acquiring the pixel coordinates of the foreground object in the image to be corrected;
the matching module is used for determining at least one distortion correction grid corresponding to the pixel coordinates of the foreground object in a preset distortion correction network; the preset distortion correction network and the image to be corrected have the same size, the distortion correction network comprises a plurality of distortion correction grids, and the distortion correction grids are combined in an overlapping mode;
and the correction module is used for correcting the pixel coordinates of the foreground object in the corresponding correction grid according to a preset correction model to obtain the pixel coordinates of the corrected foreground object.
9. A computer-readable storage medium on which a computer program is stored which, when being executed by a processor, implements the image distortion correction method according to any one of claims 1 to 7.
10. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the image distortion correction method according to any one of claims 1 to 7 when executing the program.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111571459.3A CN114494034A (en) | 2021-12-21 | 2021-12-21 | Image distortion correction method, device and equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111571459.3A CN114494034A (en) | 2021-12-21 | 2021-12-21 | Image distortion correction method, device and equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114494034A true CN114494034A (en) | 2022-05-13 |
Family
ID=81493965
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111571459.3A Pending CN114494034A (en) | 2021-12-21 | 2021-12-21 | Image distortion correction method, device and equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114494034A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2024041027A1 (en) * | 2022-08-26 | 2024-02-29 | 腾讯音乐娱乐科技（深圳）有限公司 | Panoramic-image processing method, and computer device and storage medium |
-
2021
- 2021-12-21 CN CN202111571459.3A patent/CN114494034A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2024041027A1 (en) * | 2022-08-26 | 2024-02-29 | 腾讯音乐娱乐科技（深圳）有限公司 | Panoramic-image processing method, and computer device and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP5437311B2 (en) | Image correction method, image correction system, angle estimation method, and angle estimation device | |
| CN110493525B (en) | Zoom image determination method and device, storage medium and terminal | |
| CN109003311B (en) | Calibration method of fisheye lens | |
| WO2016065632A1 (en) | Image processing method and device | |
| CN108122191B (en) | Method and device for splicing fisheye images into panoramic image and panoramic video | |
| CN104506762B (en) | Optical field acquisition control method and device, optical field acquisition equipment | |
| CN107665483B (en) | Calibration-free convenient monocular head fisheye image distortion correction method | |
| CN102479379B (en) | Image rectification method and relevant image rectification system | |
| CN108717704B (en) | Target tracking method based on fisheye image, computer device and computer readable storage medium | |
| CN112261387B (en) | Image fusion method and device for multi-camera module, storage medium and mobile terminal | |
| CN108848367B (en) | Image processing method and device and mobile terminal | |
| CN110099220B (en) | Panoramic stitching method and device | |
| WO2007129446A1 (en) | Image processing method, image processing program, image processing device, and imaging device | |
| CN112511767B (en) | Video splicing method and device, and storage medium | |
| CN112686824A (en) | Image correction method, image correction device, electronic equipment and computer readable medium | |
| CN111507894B (en) | Image stitching processing method and device | |
| CN112470192A (en) | Dual-camera calibration method, electronic device and computer-readable storage medium | |
| CN111866523A (en) | Panoramic video synthesis method and device, electronic equipment and computer storage medium | |
| CN104994280B (en) | A kind of preview image processing method and user terminal | |
| CN114339042A (en) | Image processing method and device based on multiple cameras and computer readable storage medium | |
| CN112215906A (en) | Image processing method and device and electronic equipment | |
| CN114494034A (en) | Image distortion correction method, device and equipment | |
| CN113965664A (en) | Image blurring method, storage medium and terminal device | |
| CN111353945B (en) | Fisheye image correction method, device and storage medium | |
| CN103929584B (en) | Method for correcting image and image calibrating circuit |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |