CN118096606A

CN118096606A - Image processing method and device, electronic equipment and storage medium

Info

Publication number: CN118096606A
Application number: CN202211496557.XA
Authority: CN
Inventors: 吕凭乐; 毛子靖; 涂仲轩; 孙恒
Original assignee: Beijing Xiaomi Mobile Software Co Ltd
Current assignee: Beijing Xiaomi Mobile Software Co Ltd
Priority date: 2022-11-24
Filing date: 2022-11-24
Publication date: 2024-05-28

Abstract

The disclosure relates to an image processing method and device, an electronic device and a storage medium. The image processing method comprises the following steps: performing roller shutter effect correction (RSC) on a first grid corresponding to a first image generated by the image module to obtain a second grid; determining the amount of distortion introduced when the RSC is performed; correcting the second grid according to the distortion quantity to obtain a third grid; and correcting the first image according to the third grid to obtain a second image.

Description

Image processing method and device, electronic equipment and storage medium

Technical Field

The disclosure relates to the field of information technology, and in particular, to an image processing method and device, an electronic device and a storage medium.

Background

Images can be acquired by using a camera, but the camera is limited by the hardware structure of the camera or the volume of the electronic equipment in which the camera is positioned, and the acquired images can generate some distortion relative to an acquisition object.

To remove or attenuate such distortion, various corrections are made to the acquired image. However, in the correction process, there may be a problem of excessive correction, thereby introducing distortion again, so that the correction effect of the corrected image is not ideal.

Disclosure of Invention

The embodiment of the disclosure provides an image processing method and device, electronic equipment and a storage medium.

A first aspect of an embodiment of the present disclosure provides an image processing method, including:

Carrying out roller shutter effect correction (Rolling Shutter Correction, RSC) on a first grid corresponding to the first image generated by the image module to obtain a second grid;

Determining the amount of distortion introduced when the RSC is performed;

correcting the second grid according to the distortion quantity to obtain a third grid;

and correcting the first image according to the third grid to obtain a second image.

Based on the above scheme, the determining the distortion amount introduced in the RSC includes:

Determining a first rotation amount of a target object in the first image according to image coordinates of the target object in two adjacent frames generated by the image module;

determining a second rotation amount of the image module when generating two adjacent frames of the first images;

An affine transformation matrix reflecting distortion introduced at the RSC is determined based on the first rotation amount and the second rotation amount.

Based on the above-described aspect, the determining an affine transformation matrix reflecting distortion introduced at the RSC according to the first rotation amount and the second rotation amount includes:

determining a confidence level based on the first rotation amount and the second rotation amount;

Obtaining third coordinates of each target point according to the confidence coefficient, the first coordinates of each target point in the first image before the RSC and the second coordinates of each target point after the RSC; wherein the target point is a predetermined imaging point on the target object;

and determining the affine transformation matrix according to the second coordinates and the third coordinates of each target point.

Based on the above-mentioned scheme, the determining the affine transformation matrix according to the second coordinates and the third coordinates of each of the target points includes:

determining a first barycentric point coordinate according to the second coordinate of each target point;

Determining a second gravity point coordinate according to the third coordinate of each target point;

Determining the affine transformation matrix according to the second coordinates, the first barycentric point coordinates, the first weight values, the third coordinates, the second barycentric point coordinates and the second weight values of each target point; wherein the first weight values of any two target points are equal; the second weight values of any two target points are equal.

Based on the above scheme, the correcting the second grid according to the distortion amount to obtain a third grid includes:

and multiplying each grid point in the second grid with the affine transformation matrix to obtain the third grid.

Based on the above scheme, the method further comprises:

performing coordinate transformation on a third image acquired by the image module to obtain the first image; the left vertex of the third image is located at the origin of the image coordinate system, and the center point of the first image is located at the origin of the image coordinate system.

Based on the above scheme, the correcting RSC based on the roller shutter effect on the first grid corresponding to the first image, to obtain the second grid includes:

When the electronic equipment adopts an image module to collect a face image, rolling blind effect correction RSC is carried out on a first grid corresponding to the first image containing the face image, and a second grid is obtained.

A second aspect of an embodiment of the present disclosure provides an image processing apparatus, the apparatus including:

the first correction module is used for carrying out rolling curtain effect correction (RSC) on a first grid corresponding to the first image generated by the image module to obtain a second grid;

A determining module, configured to determine an amount of distortion introduced during the RSC;

the second correction module is used for correcting the second grid according to the distortion quantity to obtain a third grid;

and the third correction module is used for correcting the first image according to the third grid to obtain a second image.

Based on the above scheme, the determining module includes:

The first determining unit is used for determining a first rotation amount of the target object according to the image coordinates of the target object in the first images of two adjacent frames generated by the image module;

The second determining unit is used for determining a second rotation amount of the image module when two adjacent frames of the first images are generated;

A third determining unit configured to determine an affine transformation matrix reflecting distortion introduced at the RSC based on the first rotation amount and the second rotation amount.

Based on the above, the third determining unit is configured to determine a confidence according to the first rotation amount and the second rotation amount; obtaining third coordinates of each target point according to the confidence coefficient, the first coordinates of each target point in the first image before the RSC and the second coordinates of each target point after the RSC; and determining the affine transformation matrix according to the second coordinates and the third coordinates of each target point.

Based on the above scheme, the third determining unit is specifically configured to determine a first barycentric point coordinate according to the second coordinates of each of the target points; determining a second gravity point coordinate according to the third coordinate of each target point; determining the affine transformation matrix according to the second coordinates, the first barycentric point coordinates, the first weight values, the third coordinates, the second barycentric point coordinates and the second weight values of each target point; wherein the first weight values of any two target points are equal; the second weight values of any two target points are equal.

Based on the above solution, the third correction module is specifically configured to multiply each grid point in the second grid with the affine transformation matrix to obtain the third grid.

Based on the above scheme, the device further comprises:

The preprocessing module is used for carrying out coordinate transformation on the third image acquired by the image module to obtain the first image; the left vertex of the third image is located at the origin of the image coordinate system, and the center point of the first image is located at the origin of the image coordinate system.

Based on the above scheme, the first correction module is specifically configured to perform roller shutter effect correction RSC on a first grid corresponding to the first image including the face image when the electronic device collects the face image by using the image module, so as to obtain a second grid.

A third aspect of an embodiment of the present disclosure provides an electronic device, including:

A memory for storing processor-executable instructions;

A processor connected to the memory;

Wherein the processor is configured to perform the image processing method as provided in any of the claims of the first aspect.

A fourth aspect of the disclosed embodiments provides a non-transitory computer-readable storage medium, which when executed by a processor of a computer, enables the computer to perform the image processing method as provided by any of the technical aspects of the first aspect.

According to the technical scheme provided by the embodiment of the disclosure, after RSC, the distortion amount of distortion introduced during RSC can be determined; and performing inverse correction on the RSC according to the distortion quantity, so that distortion introduced by at least part of the RSC is reduced, and the quality of image processing is improved.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description, serve to explain the principles of the disclosure.

FIG. 1 is a flow chart of an image processing method according to an exemplary embodiment;

FIG. 2 is an image and grid change schematic of one RSC and R-RSC shown in accordance with an exemplary embodiment;

FIG. 3 is a flow chart illustrating a method of image processing according to an exemplary embodiment;

FIG. 4 is a flow chart illustrating a method of image processing according to an exemplary embodiment;

Fig. 5 is a schematic structural view of an image processing apparatus according to an exemplary embodiment;

fig. 6 is a schematic diagram of an electronic device according to an exemplary embodiment.

Detailed Description

Reference will now be made in detail to exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numbers in different drawings refer to the same or similar elements, unless otherwise indicated. The implementations described in the following exemplary examples are not representative of all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus consistent with some aspects of the disclosure as detailed in the accompanying claims.

As shown in fig. 1, an embodiment of the present disclosure provides an image processing method, including:

S1110: performing roller shutter effect correction (RSC) on a first grid corresponding to a first image generated by the image module to obtain a second grid;

S1120: determining the amount of distortion introduced when the RSC is performed;

S1130: correcting the second grid according to the distortion quantity to obtain a third grid;

S1140: and correcting the first image according to the third grid to obtain a second image.

The image processing method can be used for correcting image distortion after an original image is acquired by an image module adopting a rolling shutter (rolling shutter).

When the image module adopting the rolling shutter is exposed, a progressive scanning mode is adopted, so that the exposure time of pixels in different rows in the same image is different, the relative position between an acquisition object and the image module can be changed although the exposure time is shorter, and the state of the acquisition object acquired by the pixels in different rows at different moments can be caused, so that image distortion is generated. In the embodiment of the disclosure, RSC is performed on the first grid corresponding to the first image. RSC herein is to reduce distortion by changing coordinates in the image.

The grid points on the first grid have a correspondence with the image coordinates in the first image. If the first image is an image which has undergone coordinate transformation, the first grid is a special-shaped grid. If the first image has not been subjected to image processing, the first grid is still a standard grid. The standard grid is as follows: a horizontal and vertical grid. Any adjacent three grid points in a standard grid may lie on the same straight line. The first mesh is a warped mesh (mesh) that can be used for the change in pixel coordinates. If the first image has undergone image processing, the first grid has changed from a standard grid to a shaped grid. For example, the image may be subjected to optical image anti-shake (Optical Image Stabilization, OIS) correction and/or lens distortion correction (Lens Distortion correction, LDC) before RSC is performed.

The first grid may be laid over the first image and cover each pixel of the first image.

The position of one or more grid points in the first grid will change after RSC correction, so that a second grid will be obtained. The positions of one or more grid points in the second grid are changed, thereby changing from a standard grid to a special-shaped grid.

The RSC herein may employ any algorithm in the related art that corrects for the roller blind effect.

Illustratively, RSC may be performed by the following functional relationship:

K is an internal reference matrix of the image module; r _tj is the rotation matrix of the image module for acquiring the exposure time tj of the object X on the jth frame of image. The transpose of the rotation matrix of the image module for the exposure time ti of the acquisition point X on the ith frame image. x _i is the projection coordinates on the i-th frame image; x _j is the projection coordinates on the j-th frame image. In some embodiments, where tj is the exposure time of the center line pixel on the first image, then R _tj is a rotation matrix of the center line exposure time of the first image. And x _i may be the pixel coordinates of the acquired object on the second image after RSC correction. In an ideal state, if the relative position between the image module and the acquisition object remains static in the acquisition process of one frame of image, it is indicated that even if the image module is subjected to line-by-line exposure, the pixels of different lines acquire the images of the acquisition object in the same or similar state, and distortion is introduced after RSC. If the relative position between the image module and the acquisition is in a non-ideal state, the relative position can have relative motion in the acquisition process of one frame of image, only the motion with different magnitudes can reduce distortion introduced by a part of rolling shutter after passing through the RSC, but part of image frames or part of pixels in one frame of image are also possibly because of the distortion introduced by the RSC. In this case, it may be necessary to achieve the elimination of distortion by inverse correction.

To solve such RSC-introduced distortion, in the embodiment of the present disclosure, a step of obtaining a distortion amount due to RSC-introduced distortion is introduced, and the distortion amount determined based on this step is corrected by a deformed mesh (mesh) to obtain a third mesh finally transformed by coordinates. The deformation of the second grid based on the amount of deformation may be referred to as a reverse roller shutter effect correction (R-RSC). The correction direction of the second grid according to the amount of distortion is opposite to the correction direction of the first grid at the time of performing RSC in step S1130, so that the distortion introduced by RSC can be at least partially canceled.

FIG. 2 is a schematic illustration of a face image and a warped mesh sequentially performing RSC and R-RSC. The grid of the leftmost image shown in fig. 2 is not a standard grid, then it is illustrated that other image processing is being undergone before the RSC is continued, so that the grid is changed from a standard grid to a special-shaped grid. After the third grid is obtained, coordinate transformation is carried out on each pixel point in the first image according to the third grid. In the process of transforming the coordinates of the pixel points, the difference value of the pixel values is processed or downsampled according to various interpolation algorithms. For example, interpolation processing of pixel values is performed based on spline curves, bezier curves, or the like.

In the embodiment of the disclosure, the steps of determining the distortion amount and reversely correcting the RSC are introduced after the RSC, so that the distortion introduced by at least part of the RSC is reduced, and the quality of image processing is improved.

In some embodiments, the S1120 may include:

s1121: determining a first rotation amount of a target object in the first image according to image coordinates of the target object in two adjacent frames generated by the image module;

s1122: determining a second rotation amount of the image module when generating two adjacent frames of the first images;

S1123: an affine transformation matrix reflecting distortion introduced at the RSC is determined based on the first rotation amount and the second rotation amount.

For example, the image coordinates of the target object in the two adjacent frames of images may be the quaternion of the center point of the target object, or the average value of the quaternions of the boundaries of the target object in the two adjacent frames of images.

The image coordinate system may include an X-axis and a Y-axis; the X-axis may correspond to a row of the image; the Y-axis may correspond to a column of images. The position of coordinates in the two frames of images changes in the X axis and the Y axis, and the quaternion values in the X axis and the Y axis can be obtained. The position change of the target object on the Z axis can be directly based on the rotation quantity acquired by the attitude sensors such as gyroscopes and the like of the same equipment of the image module. The first rotation amount can be calculated simply in this way. The first rotation amount may reflect a rotation variation amount of the target object with respect to the image module.

For example, in the process of collecting two adjacent frames of images by the image module, the gesture sensor may be used as a rotation amount (i.e. a second rotation amount) of the image module according to a difference value of detection values of the gesture sensor.

Illustratively, q _t is a quaternion of the target object of the t-th frame image; q _t-1 is a quaternion of a target object of the t-1 th frame image; the first amount of rotation may beQ _Gyrot is a quaternion detected by an attitude sensor such as a gyroscope and the like when the frame image is the t frame; q _Gyrot-1 is a quaternion detected by an attitude sensor such as a gyroscope and the like when the t-1 frame image is formed; the second rotation amount may be

In the implementation of the present disclosure, the difference between the first rotation amount and the second rotation amount is represented as a relative rotation of the target object with respect to the image module.

In the embodiment of the present disclosure, therefore, an affine transformation matrix that performs inverse correction of RSC can be obtained from the first rotation amount and the second rotation amount.

Further, the determining an affine transformation matrix reflecting distortion introduced by the RSC according to the first rotation amount and the second rotation amount includes:

The magnitude of the confidence reflects the magnitude of the amount of distortion introduced by the RSC.

The determining a confidence from the first rotation amount and the second rotation amount may include:

Determining a rotational differential value based on the first rotational amount and the second rotational amount;

and determining the confidence according to the differential value.

The rotation difference value may be determined according to the first rotation amount and the second rotation amount by using the following function:

Or/>

The confidence level is determined according to the differential value, and can be determined by adopting the following functions:

Wherein con is the confidence; the fdc _para is a cardinal parameter, which may typically be an empirical or experimental value. The fdc _para can range from 1 to plus infinity. Illustratively, the value of fdc _para may be determined by the processing effect of one or more test images before the image module is shipped and written to the camera parameters of the image module.

And obtaining a third coordinate of each target point according to the confidence coefficient, a first coordinate of each target point in the first image before the RSC, and a second coordinate of each target point after the target point passes through the RSC, wherein the third coordinate of each target point can be determined by adopting the following formula:

P_{stand_after_RSC_andARSC}＝P_{stand_after_RSC}+(P_stand-P_{stand_after_RSC})*con；

Or alternatively

P_{stand_after_RSC_andARSC}＝P_stand+(P_{stand_after_RSC}-P_stand)*con。

Wherein P _{stand_after_RSC_andARSC} is the third coordinate of the target point; p _{stand_after_RSC} is the second coordinate of the target point; p _stand is the first coordinate of the target point.

In one embodiment, the predetermined imaging point may include: contour points of the target object. For example, the target object is a human face; the target point may be: one or more facial contour points in the facial imaging. The facial contour points may include: facial outline points and/or facial feature outline points, etc.

After the third coordinates and the second coordinates of each target point are determined, an affine transformation matrix can be determined from the second coordinates and the third coordinates.

In one embodiment, the determining the affine transformation matrix according to the second coordinate, the first barycentric coordinate, the first weight value, the third coordinate, the second barycentric coordinate, and the second weight value of each of the target points may be determined by using the following formula:

M is a jet conversion matrix; p _i is the second coordinate of the i-th target point; q _j is the third coordinate of the jth target point. First weight of the ith target point of omega _i; second weight of the jth target point of ω _j. p ^* is the first barycentric point coordinate; q ^* is the second centroid point coordinate.

Since in the embodiment of the disclosure, any one pixel point in the first image is equally important, the first weights of any two target points are equal, and the second weights of any two target points are equal.

Because the first weights of any two target points are equal and the second weights of any two target points are equal, p ^* is the center point coordinate of the first image after RSC; and q ^* is the center point coordinates of the second image obtained after the RSC-based inverse correction.

In one embodiment, the first weight and the second weight are equal.

Illustratively, ω _i＝ω_j =1.

In some embodiments, the correcting the second grid according to the distortion amount to obtain a third grid includes:

In some embodiments, the method further comprises:

For better processing of the image, coordinate normalization is performed after the original image (i.e., the third image) coordinate transformation is obtained from the image module. For example, the center point of the first image is at the origin of the image coordinate system.

As RSC and inverse correction of RSC are continued thereafter, the amount of calculation of the coordinate correlation can be reduced.

The above-described image processing method may be applied in any scene, but the above-described operations are performed in a preset scene due to the amount of computation involved in image processing. Thus, in some embodiments, the correcting RSC based on the rolling effect on the first grid corresponding to the first image, to obtain the second grid includes: when the electronic equipment adopts an image module to collect a face image, rolling blind effect correction RSC is carried out on a first grid corresponding to the first image containing the face image, and the second grid is obtained.

The user has little concern on the distortion of scenery, the RSC is acquired by using the distant view of the rear camera, and the RSC step can be omitted, so that when the front camera projection is adopted to acquire the face, the RSC needs to be reversely corrected after the RSC is performed.

Referring to fig. 4, an embodiment of the disclosure provides an image processing method, and the embodiment is exemplified by a face image, and may specifically include the following steps:

step one: the standardization of coordinates of the face frame image may specifically include:

the 4-vertex coordinates of the face image are normalized (i.e., normalized) to [ -FRMHEIGHT: FRMHEIGHT, -frmWidth: frmWidth ] to ensure that the origin is positioned at the center point of the image;

Then the anti-shake (Optical Image Stabilization, OIS) of the optical image of 4 points is required to be compensated reversely, and the coordinate position of the corrected lens distortion is obtained and recorded as The first image can be obtained from the image after coordinate normalization and OIS compensation.

Step two: coordinates of the Face frame points after RSC are obtained, face _stand is taken as input, and the coordinates of each coordinate point after the rolling effect (rolling shutter effect) is removed are calculated according to x _j＝K·R_tj·R_ti ^T·x_i and are recorded as Face _{stand_after_RSC}.

Notably, are: r _ti is replaced by a rotation matrix corresponding to the row exposure time of the pixel point when each point is calculated; r _tj is a rotation matrix corresponding to the exposure time of the central line of the frame where the current face imaging is located;

Step three: obtaining confidence coefficient of the face distortion correction degree;

The degree of face distortion may be different due to the difference in the previous rotation angle of the foreground (face) and the background.

When the movement of the face and the camera tends to be the same, the distortion of the face is basically introduced by RSC, and the rolling cutter needs to be corrected reversely;

When the movement of the face and the background tends to be the same, the distortion of the face is basically introduced by the rolling router itself, RSC is needed, and the rolling router is not needed to be corrected reversely.

Therefore, the degree of face distortion correction needs to be dynamically obtained. Acquiring quaternions of face orientations of current frame and previous frame Quaternion/>, of gyroscope (gyro) orientation at corresponding moments

According toThe difference in rotation amounts of the foreground (face) and the background is calculated and used as a parameter x of the smoothing function.

And obtaining distortion correction confidence of the face of the current frame according to the Con=1- (1/fdc _para)^x), wherein fdc _para represents the radix parameter of the smoothing function.

Step four: estimating affine transformation matrix according to coordinate positions of face frame points before and after RSC and confidence degree

Firstly, according to the confidence obtained by the calculation in the step three, confidence is calculated according to the speed _{stand_after_RSC_andARSC}＝Face_{stand_after_RsC}+(Face_stand-Face_{stand_after_RSC})

Calculating the position of a target point in the face image:

According to Face _{stand_after_RSC_andARSC} and Face _{stand_after_RSC}, according to AndEstimating an affine transformation matrix, wherein p _i represents points in Face _{stand_after_RSC} and q _j represents points in Face _{stand_after_RSC_andARSC}; omega _i and omega _j represent weights.

Step five: the affine transformation matrix is applied to the warped mesh to obtain a mesh (grid _warp) for image warping (warp), and the mesh can be used for image warping to correct face distortion.

The deformation amount introduced by the RSC is determined by acquiring the portrait by the front camera, and the reverse correction is performed based on the deformation amount, so that the deformation of the face image introduced by the RSC is reduced, the face deformation is reduced, and the image processing quality is improved.

As shown in fig. 5, an embodiment of the present disclosure provides an image processing apparatus including:

A first correction module 110, configured to perform roller shutter effect correction RSC on a first grid corresponding to the first image generated by the image module, to obtain a second grid;

a determining module 120 configured to determine an amount of skew, wherein the amount of skew indicates skew introduced by the RSC;

A second correction module 130, configured to correct the second grid according to the distortion amount, so as to obtain a third grid;

And a third correction module 140, configured to correct the first image according to the third grid, so as to obtain a second image.

In some embodiments, the first correction module 110, the determination module 120, the second correction module 130, and the third correction module 140 may be program modules; the above-described operations can be performed by program modules when executed by a processor.

In other embodiments, the first correction module 110, the determination module 120, the second correction module 130, and the third correction module 140 may be soft-hard combination modules; the soft and hard combined module comprises a programmable array; the programmable array includes, but is not limited to: a field programmable array and/or a complex programmable array.

In still other embodiments the first correction module 110, the determination module 120, the second correction module 130, and the third correction module 140 may be purely hardware modules; the pure hardware modules include, but are not limited to: an application specific integrated circuit.

As can be appreciated, the determining module 120 includes:

It is understood that the third determining unit is configured to determine a confidence level according to the first rotation amount and the second rotation amount; obtaining third coordinates of each target point according to the confidence coefficient, the first coordinates of each target point in the first image before the RSC and the second coordinates of each target point after the RSC; and determining the affine transformation matrix according to the second coordinates and the third coordinates of each target point.

It is to be appreciated that the third determining unit is specifically configured to determine the first barycentric coordinates according to the second coordinates of each of the target points; determining a second gravity point coordinate according to the third coordinate of each target point; determining the affine transformation matrix according to the second coordinates, the first barycentric point coordinates, the first weight values, the third coordinates, the second barycentric point coordinates and the second weight values of each target point; wherein the first weight values of any two target points are equal; the second weight values of any two target points are equal.

It may be appreciated that the third correction module 140 is specifically configured to multiply each grid point in the second grid by the affine transformation matrix to obtain the third grid.

It will be appreciated that the apparatus further comprises:

As can be appreciated, the first correction module 110 is specifically configured to perform a roller shutter effect correction RSC on a first grid corresponding to the first image including the face image when the electronic device collects the face image by using the image module, so as to obtain the second grid.

An embodiment of the present disclosure provides an electronic device, including:

A memory for storing processor-executable instructions;

A processor connected to the memory;

wherein the processor is configured to perform the image processing method provided in any of the foregoing technical solutions, and may specifically include the method shown in any of fig. 1, fig. 3, and fig. 4.

The electronic device includes, but is not limited to, various fixed terminals and/or mobile terminals.

The processor may be coupled to the memory via a bus including, but not limited to: IPS bus and/or I ² C bus, etc.

In some embodiments, the electronic device further comprises a network interface, likewise connected by a bus equal to the processor. The network interface may be for the electronic device to connect to a network.

The disclosed embodiments provide a non-transitory computer readable storage medium, which when executed by a processor of a computer, enables the computer to perform the software package upgrade method provided by any of the foregoing technical solutions, such as the methods shown in any of fig. 1,3, and 4.

Referring to fig. 6, an embodiment of the present disclosure provides an electronic device, which is the aforementioned display device. Electronic device 800 may include one or more of the following components: a processing component 802, a memory 804, a power component 806, a multimedia component 808, a multimedia data component 810, an input/output (I/O) interface 812, a sensor component 814, and a communication component 816. The processing component 802 generally controls overall operation of the electronic device 800, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to perform all or part of the steps of the methods described above. Further, the processing component 802 can include one or more modules that facilitate interactions between the processing component 802 and other components. For example, the processing component 802 can include a multimedia module to facilitate interaction between the multimedia component 808 and the processing component 802.

The memory 804 is configured to store various types of data to support operations at the device 800. Examples of such data include instructions for any application or method operating on the electronic device 800, contact data, phonebook data, messages, pictures, videos, and so forth. The memory 804 may be implemented by any type or combination of volatile or nonvolatile memory devices such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disk.

The power supply component 806 provides power to the various components of the electronic device 800. Power components 806 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power for electronic device 800.

The multimedia component 808 includes a screen between the electronic device 800 and the user that provides an output interface. In some embodiments, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensor may sense not only the boundary of a touch or sliding action, but also the duration and pressure associated with the touch or sliding operation. In some embodiments, the multimedia component 808 includes a front camera and/or a rear camera. The front camera and/or the rear camera may receive external multimedia data when the device 800 is in an operational state, such as a photographing state or a video state. Each front camera and rear camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

The multimedia data component 810 is configured to output and/or input multimedia data signals. For example, the multimedia data component 810 includes a Microphone (MIC) configured to receive external multimedia data signals when the electronic device 800 is in an operational state, such as a call state, a recording state, and a voice recognition state. The received multimedia data signals may be further stored in memory 804 or transmitted via communications component 816.

In some embodiments, the multimedia data component 810 further comprises a speaker for outputting multimedia data signals.

The I/O interface 812 provides an interface between the processing component 802 and peripheral interface modules, which may be a keyboard, click wheel, operating buttons, etc. These operating buttons may include, but are not limited to: homepage operation button, volume operation button, start operation button and lock operation button.

The sensor assembly 814 includes one or more sensors for providing status assessment of various aspects of the electronic device 800. For example, the sensor assembly 814 may detect an on/off state of the device 800, a relative positioning of the components, such as a display and keypad of the electronic device 800, the sensor assembly 814 may also detect a change in position of the electronic device 800 or a component of the electronic device 800, the presence or absence of a user's contact with the electronic device 800, an orientation or acceleration/deceleration of the electronic device 800, and a change in temperature of the electronic device 800. The sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 814 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an acceleration sensor, a gyroscopic sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

The communication component 816 is configured to facilitate communication between the electronic device 800 and other devices, either wired or wireless. The electronic device 800 may access a wireless network based on a communication standard, such as Wi-Fi,2G, or 3G, or a combination thereof. In one exemplary embodiment, the communication component 816 receives broadcast signals or broadcast related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the communication component 816 further includes a Near Field Communication (NFC) module to facilitate short range communications. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, ultra Wideband (UWB) technology, bluetooth (BT) technology, and other technologies.

In an exemplary embodiment, the apparatus 800 may be implemented by one or more Application Specific Integrated Circuits (ASICs), digital Signal Processors (DSPs), digital Signal Processing Devices (DSPDs), programmable Logic Devices (PLDs), field Programmable Gate Arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic elements for performing the image processing methods provided in any one of the embodiments described above.

The image processing method may include: performing roller shutter effect correction (RSC) on a first grid corresponding to a first image generated by the image module to obtain a second grid; determining the amount of distortion introduced when the RSC is performed; correcting the second grid according to the distortion quantity to obtain a third grid; and correcting the first image according to the third grid to obtain a second image.

As can be appreciated, the determining the amount of distortion includes:

It is to be understood that the determining an affine transformation matrix reflecting distortion introduced at the RSC based on the first rotation amount and the second rotation amount includes:

It is understood that said determining said affine transformation matrix from the second coordinates and said third coordinates of each of said target points comprises:

It may be appreciated that correcting the second grid according to the distortion amount, to obtain a third grid includes:

It will be appreciated that the method further comprises:

As can be appreciated, the correcting RSC based on the roller shutter effect performed on the first grid corresponding to the first image, to obtain a second grid includes:

when the electronic equipment adopts an image module to collect a face image, rolling blind effect correction RSC is carried out on a first grid corresponding to the first image containing the face image, and the second grid is obtained.

Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This disclosure is intended to cover any adaptations, uses, or adaptations of the disclosure following the general principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.

It is to be understood that the present disclosure is not limited to the precise arrangements and instrumentalities shown in the drawings, and that various modifications and changes may be effected without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. An image processing method, the method comprising:

Performing roller shutter effect correction (RSC) on a first grid corresponding to a first image generated by the image module to obtain a second grid;

Determining the amount of distortion introduced when the RSC is performed;

2. The method of claim 1, wherein said determining the amount of distortion introduced in said RSC comprises:

3. The method according to claim 2, wherein the determining an affine transformation matrix reflecting distortion introduced at the RSC according to the first rotation amount and the second rotation amount includes:

4. A method according to claim 3, wherein said determining said affine transformation matrix from the second coordinates and said third coordinates of each of said target points comprises:

5. The method according to any one of claims 2 to 4, wherein correcting the second grid according to the amount of distortion to obtain a third grid comprises:

6. The method according to any one of claims 1 to 4, further comprising:

7. The method according to any one of claims 1 to 4, wherein the correcting RSC based on the roller shutter effect on the first grid corresponding to the first image to obtain the second grid includes:

8. An image processing apparatus, characterized in that the apparatus comprises:

9. The apparatus of claim 8, wherein the determining module comprises:

10. The apparatus according to claim 9, wherein the third determining unit is configured to determine a confidence level based on the first rotation amount and the second rotation amount; obtaining third coordinates of each target point according to the confidence coefficient, the first coordinates of each target point in the first image before the RSC and the second coordinates of each target point after the RSC; and determining the affine transformation matrix according to the second coordinates and the third coordinates of each target point.

11. The apparatus according to claim 10, wherein the third determining unit is specifically configured to determine the first barycentric point coordinates according to the second coordinates of each of the target points; determining a second gravity point coordinate according to the third coordinate of each target point; determining the affine transformation matrix according to the second coordinates, the first barycentric point coordinates, the first weight values, the third coordinates, the second barycentric point coordinates and the second weight values of each target point; wherein the first weight values of any two target points are equal; the second weight values of any two target points are equal.

12. The apparatus according to any one of claims 9 to 11, wherein the third correction module is configured to, in particular, multiply each grid point in the second grid with the affine transformation matrix to obtain the third grid.

13. The apparatus according to any one of claims 8 to 12, further comprising:

14. The apparatus according to any one of claims 8 to 11, wherein the first correction module is specifically configured to, when an electronic device collects a face image using an image module, perform a rolling-curtain effect correction RSC on a first grid corresponding to the first image including the face image, to obtain the second grid.

15. An electronic device, comprising:

A memory for storing processor-executable instructions;

A processor connected to the memory;

wherein the processor is configured to perform the image processing method of any one of claims 1 to 7.

16. A non-transitory computer readable storage medium, which when executed by a processor of a computer, causes the computer to perform the image processing method of any one of claims 1 to 7.