CN111860527A - Image correction method, image correction device, computer device, and storage medium - Google Patents
Image correction method, image correction device, computer device, and storage medium Download PDFInfo
- Publication number
- CN111860527A CN111860527A CN201911016421.2A CN201911016421A CN111860527A CN 111860527 A CN111860527 A CN 111860527A CN 201911016421 A CN201911016421 A CN 201911016421A CN 111860527 A CN111860527 A CN 111860527A
- Authority
- CN
- China
- Prior art keywords
- image
- corner
- corrected
- coordinates
- acquiring
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/22—Image preprocessing by selection of a specific region containing or referencing a pattern; Locating or processing of specific regions to guide the detection or recognition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/10—Character recognition
- G06V30/14—Image acquisition
- G06V30/148—Segmentation of character regions
- G06V30/153—Segmentation of character regions using recognition of characters or words
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/24—Aligning, centring, orientation detection or correction of the image
- G06V10/247—Aligning, centring, orientation detection or correction of the image by affine transforms, e.g. correction due to perspective effects; Quadrilaterals, e.g. trapezoids
Abstract
The invention provides an image correction method, an image correction device, a computer device and a storage medium. The image correction method comprises the following steps: acquiring an image to be corrected, and acquiring corner coordinates in the image to be corrected; calculating a perspective correction matrix according to the corner coordinates and the target corner coordinates; and carrying out perspective transformation on the image to be corrected by utilizing the perspective correction matrix to obtain a corrected image. By adopting the technical scheme of the invention, the problems of inaccuracy of the rotation angle of the certificate calculated according to the inclined direction of the characters and incapability of correcting perspective shooting in the related technology can be solved, and the accuracy of image correction is improved.
Description
Technical Field
The present invention relates to the field of image processing technologies, and in particular, to an image correction method, an image correction apparatus, a computer device, and a storage medium.
Background
The document OCR (Optical Character Recognition) automatically recognizes the characters in the document, and to perform the document OCR, it is first required to take a document picture and then recognize the characters in the document picture. When a photographer shoots the certificate, the horizontal placement of the certificate cannot be guaranteed, and the certificate can be inclined or even placed upside down. The identification is directly carried out on the non-horizontal certificates, the identification accuracy is low, and even the identification cannot be carried out.
In the related art, the OCR method for the certificate generally adopts two methods to solve the problem of improper placement of the certificate:
1. when the certificate is shot, a certificate area frame is displayed on an application program, a photographer needs to ensure that the boundary of the certificate accurately corresponds to a prompt frame on the application program, and the shot certificate is in a horizontal state. The method has higher requirements on the photographer, normal shooting cannot be performed if the certificates are not aligned accurately, and user experience is poor.
2. The photographer can shoot at will without limiting the placing direction of the certificate, and the certificate is corrected by the algorithm. The rotation angle of the document is usually calculated according to the oblique direction of the text, but two problems exist: (1) the rotation angle is not accurately calculated, so that the correction is insufficient; (2) when shooting, the mobile phone is not parallel to the certificate, so that the problem of perspective exists, and under the condition, the inclination state of the certificate cannot be described by a single angle, so that the correction is inevitably inaccurate.
Disclosure of Invention
The present invention is directed to solving at least one of the problems of the prior art or the related art.
To this end, an aspect of the present invention is to propose an image correction method.
Another aspect of the present invention is to provide an image correction apparatus.
Yet another aspect of the invention is directed to a computer device.
Yet another aspect of the present invention is to provide a computer-readable storage medium.
In view of the above, according to an aspect of the present invention, there is provided an image correction method, including: acquiring an image to be corrected, and acquiring corner coordinates of the image to be corrected; calculating a perspective correction matrix according to the corner coordinates and the target corner coordinates; and carrying out perspective transformation on the image to be corrected by utilizing the perspective correction matrix to obtain a corrected image.
The image correction method provided by the invention obtains an image to be corrected, wherein the image to be corrected is a non-horizontal and oblique image, four corner coordinates of the image to be corrected are obtained by utilizing a model (such as a depth learning model), a perspective correction matrix is obtained according to the corner coordinates and target corner coordinates, and the target corner coordinates are the corner coordinates of an expected horizontal image. Furthermore, the perspective correction matrix is used for mapping each pixel value in the image to be corrected to the corresponding pixel value in the horizontal state, namely obtaining the corrected horizontal image. By adopting the technical scheme of the invention, the problems of inaccuracy of the rotation angle of the certificate calculated according to the inclined direction of the characters and incapability of correcting perspective shooting in the related technology can be solved, and the accuracy of image correction is improved.
It should be noted that the image to be corrected is an image of a certificate or a business card that needs to be corrected, and may be a picture taken by a user directly or extracted from a picture taken by the user. For example, when the user needs to upload a certificate image for identity verification, the user directly aligns the camera with the certificate to shoot, and only the certificate image but not other images are shot in the picture. Or, the user does not directly aim the camera at the certificate for shooting, the shot picture contains other images (such as a desktop on which the certificate is placed) besides the certificate image, and at this time, the certificate image, namely the image to be corrected, needs to be extracted from the picture.
The image correction method according to the present invention may further include the following technical features:
in the above technical solution, the method further comprises: detecting the corrected image to obtain a character area; and carrying out optical character recognition on the character area to acquire character information.
In the technical scheme, after image correction is carried out, the corrected image is detected, the character area is obtained, then the character area is identified to obtain character information, the characters in the picture are extracted quickly and accurately, and the problems of low efficiency and high error rate caused by manual input are solved.
In any of the above technical solutions, the method further includes: obtaining the confidence coefficient of the corner point coordinates; and under the condition that the confidence coefficient is greater than or equal to the confidence coefficient threshold value, calculating a perspective correction matrix according to the corner point coordinates and the target corner point coordinates.
In the technical scheme, the confidence coefficient of the corner coordinates, namely the accuracy of the corner coordinates, is obtained, and under the condition that the confidence coefficient is greater than or equal to a confidence coefficient threshold value, the perspective correction matrix is calculated according to the corner coordinates and the target corner coordinates, so that the image correction error caused by the inaccuracy of the corner coordinates is avoided, and the correction accuracy is improved.
In any of the above technical solutions, the method further includes: acquiring a training image, and marking the corner coordinates of the training image; and rotating the training image by any angle, and establishing a model for acquiring the corner coordinates and the confidence coefficient of the image to be corrected by using the training image and the corner coordinates of the training image.
In the technical scheme, a model for obtaining the coordinates and confidence degrees of the corners of the image to be corrected is established, specifically, four corner marking data of a training image are prepared, the coordinates of the four corners are marked manually, a deep learning model is adopted to carry out regression training of the four corners, and the training image is randomly rotated by any angle in the training process. When the image is corrected, the model is directly used for obtaining the coordinates and the confidence coefficient of the corner points so as to improve the efficiency of image correction.
In any of the above technical solutions, the step of obtaining coordinates of four corner points of the certificate in the image to be corrected specifically includes: defining the numbers of an upper left corner, an upper right corner, a lower right corner and a lower left corner of the target corner as No. 1, No. 2, No. 3 and No. 4 respectively; and acquiring coordinates of four corner points of the image to be corrected and the numbers of the four corner points by using the model.
In the technical scheme, target angular points are defined, namely upper left, upper right, lower right and lower left angular points of the corrected certificate are respectively No. 1, No. 2, No. 3 and No. 4, four angular point coordinates of the image to be corrected and numbers to which the four angular points belong are obtained by utilizing a model, no matter how the image to be corrected rotates, the angular point 1 corresponds to the upper left angular point of the corrected certificate, the angular point 2 corresponds to the upper right angular point of the corrected certificate, the angular point 3 corresponds to the lower right angular point of the corrected certificate, and the angular point 4 corresponds to the lower left angular point of the corrected certificate. The corner point number information can ensure that the corrected image is horizontal and forward.
According to another aspect of the present invention, there is provided an image correction apparatus comprising: the information acquisition unit is used for acquiring an image to be corrected and acquiring the corner coordinates of the image to be corrected; the calculation unit is used for calculating a perspective correction matrix according to the corner coordinates and the target corner coordinates; and the correcting unit is used for carrying out perspective transformation on the image to be corrected by utilizing the perspective correction matrix to obtain the corrected image.
According to the image correction device provided by the invention, the information acquisition unit acquires an image to be corrected, the image to be corrected is a non-horizontal and oblique image, the model (such as a depth learning model) is used for acquiring four corner coordinates of the image to be corrected, the calculation unit acquires a perspective correction matrix according to the corner coordinates and target corner coordinates, and the target corner coordinates are the corner coordinates of an expected horizontal image. Furthermore, the correction unit maps each pixel value in the image to be corrected to a corresponding pixel value in a horizontal state by using the perspective correction matrix, that is, a corrected horizontal image is obtained. By adopting the technical scheme of the invention, the problems of inaccuracy of the rotation angle of the certificate calculated according to the inclined direction of the characters and incapability of correcting perspective shooting in the related technology can be solved, and the accuracy of image correction is improved.
It should be noted that the image to be corrected is an image of a certificate or a business card that needs to be corrected, and may be a picture taken by a user directly or extracted from a picture taken by the user. For example, when the user needs to upload a certificate image for identity verification, the user directly aligns the camera with the certificate to shoot, and only the certificate image but not other images are shot in the picture. Or, the user does not directly aim the camera at the certificate for shooting, the shot picture contains other images (such as a desktop on which the certificate is placed) besides the certificate image, and at this time, the certificate image, namely the image to be corrected, needs to be extracted from the picture.
The image correction device according to the present invention may further include:
in the above technical solution, the method further comprises: the detection unit is used for detecting the corrected image and acquiring a character area; and the information identification unit is used for carrying out optical character identification on the character area to acquire character information.
In the technical scheme, after image correction is carried out, the detection unit detects the corrected image to obtain the character region, and then the information identification unit identifies the character region to obtain the character information, so that the characters in the image can be quickly and accurately extracted, and the problems of low efficiency and high error rate caused by manual input are solved.
In any of the above technical solutions, the information obtaining unit is further configured to obtain a confidence level of the corner coordinates; and the calculation unit is specifically used for calculating the perspective correction matrix according to the corner point coordinates and the target corner point coordinates under the condition that the confidence coefficient is greater than or equal to the confidence coefficient threshold value.
In the technical scheme, the information acquisition unit acquires the confidence coefficient of the corner coordinates, namely the accuracy of the corner coordinates, and the calculation unit calculates the perspective correction matrix according to the corner coordinates and the target corner coordinates under the condition that the confidence coefficient is greater than or equal to a confidence coefficient threshold value, so that the image correction error caused by the inaccuracy of the corner coordinates is avoided, and the correction accuracy is improved.
In any of the above technical solutions, the method further includes: the model establishing unit is used for acquiring a training image and marking the corner coordinates of the training image; and rotating the training image at any angle, and establishing a model for acquiring the corner coordinates and the confidence coefficient of the image to be corrected by using the training image and the corner coordinates of the training image.
In the technical scheme, a model establishing unit establishes a model for acquiring the coordinates and confidence degrees of the corners of the image to be corrected, specifically, four corner marking data of a training image are prepared, the coordinates of the four corners are marked manually, a deep learning model is adopted to carry out regression training of the four corners, and the training image is randomly rotated by any angle in the training process. When the image is corrected, the model is directly used for obtaining the coordinates and the confidence coefficient of the corner points so as to improve the efficiency of image correction.
In any of the above technical solutions, the acquiring of coordinates of four corner points of a certificate in an image to be corrected by an information acquiring unit specifically includes: defining the numbers of an upper left corner, an upper right corner, a lower right corner and a lower left corner of the target corner as No. 1, No. 2, No. 3 and No. 4 respectively; and acquiring coordinates of four corner points of the image to be corrected and the numbers of the four corner points by using the model.
In the technical scheme, target angular points are defined, namely upper left, upper right, lower right and lower left angular points of the corrected certificate are respectively No. 1, No. 2, No. 3 and No. 4, four angular point coordinates of the image to be corrected and numbers to which the four angular points belong are obtained by utilizing a model, no matter how the image to be corrected rotates, the angular point 1 corresponds to the upper left angular point of the corrected certificate, the angular point 2 corresponds to the upper right angular point of the corrected certificate, the angular point 3 corresponds to the lower right angular point of the corrected certificate, and the angular point 4 corresponds to the lower left angular point of the corrected certificate. The corner point number information can ensure that the corrected image is horizontal and forward.
According to a further aspect of the present invention, a computer device is provided, which comprises a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the steps of the image correction method according to any one of the above-mentioned technical solutions when executing the computer program.
The computer device provided by the present invention implements the steps of the image correction method according to any of the above-mentioned technical solutions when the processor executes the computer program, and therefore, the computer device includes all the advantageous effects of the image correction method according to any of the above-mentioned technical solutions.
According to yet another aspect of the present invention, a computer-readable storage medium is proposed, on which a computer program is stored, which computer program, when being executed by a processor, carries out the steps of the image correction method according to any one of the above-mentioned claims.
The present invention provides a computer-readable storage medium, wherein when being executed by a processor, a computer program implements the steps of the image correction method according to any of the above technical solutions, and therefore the computer-readable storage medium includes all the advantages of the image correction method according to any of the above technical solutions.
Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention.
Drawings
The above and/or additional aspects and advantages of the present invention will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:
FIG. 1 is a flow chart illustrating an image correction method according to a first embodiment of the present invention;
figure 2 shows a schematic diagram of corner coordinates and target corner coordinates of an embodiment of the invention;
FIG. 3 is a flow chart illustrating an image correction method according to a second embodiment of the present invention;
FIG. 4 is a flow chart showing an image correction method according to a third embodiment of the present invention;
FIG. 5 is a flow chart showing an image correction method according to a fourth embodiment of the present invention;
FIG. 6 shows a schematic diagram of a deep learning model based credential correction method of a fifth embodiment of the present invention;
fig. 7 shows a schematic block diagram of an image correction apparatus of a first embodiment of the present invention;
fig. 8 is a schematic block diagram showing an image correction apparatus of a second embodiment of the present invention;
fig. 9 shows a schematic block diagram of an image correction apparatus of a third embodiment of the present invention.
Detailed Description
In order that the above objects, features and advantages of the present invention can be more clearly understood, a more particular description of the invention will be rendered by reference to the appended drawings. It should be noted that the embodiments of the present invention and features of the embodiments may be combined with each other without conflict.
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention, however, the present invention may be practiced in other ways than those specifically described herein, and therefore the scope of the present invention is not limited to the specific embodiments disclosed below.
An embodiment of the first aspect of the present invention provides an image correction method, which is described in detail by the following embodiments.
First embodiment, fig. 1 is a flowchart illustrating an image correction method according to a first embodiment of the present invention. Wherein, the method comprises the following steps:
102, acquiring an image to be corrected, and acquiring corner coordinates of the image to be corrected;
104, calculating a perspective correction matrix according to the corner coordinates and the target corner coordinates;
and 106, carrying out perspective transformation on the image to be corrected by utilizing the perspective correction matrix to obtain a corrected image.
The image correction method provided by the invention obtains an image to be corrected by shooting, wherein the image to be corrected is a non-horizontal and oblique image, four corner coordinates of the image to be corrected are obtained by utilizing a model (such as a depth learning model), a perspective correction matrix is obtained according to the corner coordinates and target corner coordinates, the target corner coordinates are the corner coordinates of an expected horizontal image, for example, as shown in fig. 2, four corners of the image to be corrected are A, B, C, D, and the target corners are A ', B', C 'and D'. The coordinates of the corner point A are [ u, v, w ], the coordinates of the corner point A ' are [ x ', y ', w ' ], and the relationship between the coordinates of the corner point A and the coordinates of the target corner point A ' is shown in formula (1), so that the perspective correction matrix can be calculated according to the coordinates of the corner point and the coordinates of the target corner point. Furthermore, the perspective correction matrix is used for mapping each pixel value in the image to be corrected to the corresponding pixel value of the image in the horizontal state, namely, the corrected horizontal image is obtained. By adopting the embodiment of the invention, the problems of inaccuracy of the rotation angle of the certificate calculated according to the inclined direction of the characters and incapability of correcting perspective shooting in the related technology can be solved, and the accuracy of image correction is improved.
Wherein the content of the first and second substances,
is a perspective correction matrix.
It should be noted that the image to be corrected is an image of a certificate or a business card that needs to be corrected, and may be a picture taken by a user directly or extracted from a picture taken by the user. For example, when the user needs to upload a certificate image for identity verification, the user directly aligns the camera with the certificate to shoot, and only the certificate image but not other images are shot in the picture. Or, the user does not directly aim the camera at the certificate for shooting, the shot picture contains other images (such as a desktop on which the certificate is placed) besides the certificate image, and at this time, the certificate image, namely the image to be corrected, needs to be extracted from the picture.
Further, step 102, acquiring an image to be corrected, and acquiring corner coordinates of the image to be corrected, specifically includes: defining the numbers of an upper left corner, an upper right corner, a lower right corner and a lower left corner of the target corner as No. 1, No. 2, No. 3 and No. 4 respectively; and acquiring coordinates of four corner points of the image to be corrected and the numbers of the four corner points by using the model. Defining target angular points, namely upper left, upper right, lower right and lower left angular points of the corrected certificate as No. 1, No. 2, No. 3 and No. 4 respectively, acquiring four angular point coordinates of the image to be corrected by using a model, and numbers to which the four angular points belong, wherein the angular point No. 1 corresponds to the upper left angular point of the corrected certificate, the angular point No. 2 corresponds to the upper right angular point of the corrected certificate, the angular point No. 3 corresponds to the lower right angular point of the corrected certificate, and the angular point No. 4 corresponds to the lower left angular point of the corrected certificate no matter how the image to be corrected rotates. The corner point number information can ensure that the corrected image is horizontal and forward.
Second embodiment, fig. 3 is a flowchart illustrating an image correction method according to a second embodiment of the present invention. Wherein, the method comprises the following steps:
In the embodiment, after image correction is performed, the corrected image is detected, the character region is obtained, and then the character region is identified to obtain character information, so that characters in the picture can be extracted quickly and accurately, and the problems of low efficiency and high error rate caused by manual input are solved.
For example, identification of documents or business cards, etc., which may include identification cards, driving licenses, bank cards, etc., may be performed using the method. In the scenes of submitting identity information by a bank, uploading and checking registration information of a car booking driver and the like, a user shoots a certificate image which can be an inclined image or a non-horizontal image, namely, the shooting level of the user is not required, and four corner point coordinates of the certificate image and the confidence coefficient of each corner point coordinate are obtained. When the confidence of the corner points is greater than the confidence threshold, a perspective correction matrix is calculated according to the corner point coordinates and the target corner point coordinates, each pixel point of the certificate image is converted by the perspective correction matrix to obtain a horizontal certificate image, and then characters in the horizontal certificate image are extracted quickly and accurately by means of OCR, so that the problems of low manual input efficiency and high error rate are solved.
Further, step 302, acquiring an image to be corrected, and acquiring corner coordinates of the image to be corrected, specifically includes: defining the numbers of an upper left corner, an upper right corner, a lower right corner and a lower left corner of the target corner as No. 1, No. 2, No. 3 and No. 4 respectively; and acquiring coordinates of four corner points of the image to be corrected and the numbers of the four corner points by using the model.
Third embodiment, fig. 4 is a flowchart illustrating an image correction method according to a third embodiment of the present invention. Wherein, the method comprises the following steps:
In the embodiment, the confidence coefficient of the corner coordinates, that is, the accuracy of the corner coordinates is obtained, and under the condition that the confidence coefficient is greater than or equal to the confidence coefficient threshold, the perspective correction matrix is calculated according to the corner coordinates and the target corner coordinates, so that the image correction error caused by the inaccuracy of the corner coordinates is avoided, and the correction accuracy is improved.
Further, step 402, acquiring an image to be corrected, and acquiring corner coordinates of the image to be corrected, specifically including: defining the numbers of an upper left corner, an upper right corner, a lower right corner and a lower left corner of the target corner as No. 1, No. 2, No. 3 and No. 4 respectively; and acquiring coordinates of four corner points of the image to be corrected and the numbers of the four corner points by using the model.
Fourth embodiment, fig. 5 is a flowchart illustrating an image correction method according to a fourth embodiment of the present invention. Wherein, the method comprises the following steps:
In the embodiment, a model for obtaining the coordinates and confidence of the corners of the image to be corrected is established, specifically, four corner marking data of a training image are prepared, the coordinates of the four corners are marked manually, a deep learning model is adopted to perform regression training of the four corners, and the training image is randomly rotated by any angle in the training process. When the image is corrected, the model is directly used for obtaining the coordinates and the confidence coefficient of the corner points so as to improve the efficiency of image correction.
Further, step 504, acquiring an image to be corrected, and acquiring corner coordinates of the image to be corrected, specifically includes: defining the numbers of an upper left corner, an upper right corner, a lower right corner and a lower left corner of the target corner as No. 1, No. 2, No. 3 and No. 4 respectively; and acquiring coordinates of four corner points of the image to be corrected and the numbers of the four corner points by using the model.
The method does not directly calculate the rotation angle of the certificate, but positions four corner points of the certificate by using a deep learning method, can calculate a transformation matrix according to coordinates of the four corner points of the certificate, can perform perspective transformation on an original oblique image by using the transformation matrix, and can obtain the certificate in a horizontal state after transformation. FIG. 6 shows a schematic diagram of a certificate correction method based on deep learning model according to a fifth embodiment of the present invention. Wherein, the method comprises the following steps:
Preparing a model: and preparing marking data of four corner points of the certificate, and marking the coordinates of the four corner points of the certificate. And (3) carrying out four-corner point regression training by adopting deep learning, and randomly rotating the certificate by any angle in the training process.
Image sampling: and shooting the certificate image.
Angular point detection: and inputting the shot certificate image by using the model, and outputting the confidence coefficient of the corner point and the coordinates of the corner point in the image area.
Matrix transformation: and calculating a perspective transformation matrix according to the coordinates of the four corner points. And carrying out perspective change on the original non-horizontal image by utilizing a perspective transformation matrix to obtain a horizontal certificate image.
And field detection: detecting the text area of the horizontal certificate.
Copying characters for OCR: and identifying the character area.
An embodiment of the second aspect of the present invention provides an image correction apparatus, which is described in detail by the following embodiments.
First embodiment, fig. 7 shows a schematic block diagram of an image correction apparatus 700 according to a first embodiment of the present invention. The image correction apparatus 700 includes:
an information obtaining unit 702, configured to obtain an image to be corrected, and obtain corner coordinates of the image to be corrected;
a calculating unit 704, configured to calculate a perspective correction matrix according to the corner coordinates and the target corner coordinates;
The correcting unit 706 is configured to perform perspective transformation on the image to be corrected by using the perspective correction matrix to obtain a corrected image.
In the image correction apparatus 700 provided by the present invention, the information obtaining unit 702 obtains an image to be corrected, which is a non-horizontal and oblique image, by shooting, four corner coordinates of the image to be corrected are obtained by using a model (e.g. a depth learning model), and the calculating unit 704 obtains a perspective correction matrix according to the corner coordinates and target corner coordinates, which are the corner coordinates of an expected horizontal image. Further, the correction unit 706 maps each pixel value in the image to be corrected to a corresponding pixel value of the image in the horizontal state by using the perspective correction matrix, that is, obtains a corrected horizontal image. By adopting the embodiment of the invention, the problems of inaccuracy of the rotation angle of the certificate calculated according to the inclined direction of the characters and incapability of correcting perspective shooting in the related technology can be solved, and the accuracy of image correction is improved.
It should be noted that the image to be corrected is an image of a certificate or a business card that needs to be corrected, and may be a picture taken by a user directly or extracted from a picture taken by the user. For example, when the user needs to upload a certificate image for identity verification, the user directly aligns the camera with the certificate to shoot, and only the certificate image but not other images are shot in the picture. Or, the user does not directly aim the camera at the certificate for shooting, the shot picture contains other images (such as a desktop on which the certificate is placed) besides the certificate image, and at this time, the certificate image, namely the image to be corrected, needs to be extracted from the picture.
In this embodiment, the information obtaining unit 702 is further configured to obtain a confidence level of the corner coordinates; the calculating unit 704 is specifically configured to calculate a perspective correction matrix according to the corner coordinates and the target corner coordinates when the confidence is greater than or equal to the confidence threshold. The information acquisition unit 702 acquires the confidence coefficient of the corner coordinates, namely the accuracy of the corner coordinates, and the calculation unit 704 calculates the perspective correction matrix according to the corner coordinates and the target corner coordinates when the confidence coefficient is greater than or equal to a confidence coefficient threshold value, so that the image correction error caused by inaccuracy of the corner coordinates is avoided, and the correction accuracy is improved.
The information acquiring unit 702 acquires coordinates of four corner points of a certificate in an image to be corrected, and specifically includes: defining the numbers of an upper left corner, an upper right corner, a lower right corner and a lower left corner of the target corner as No. 1, No. 2, No. 3 and No. 4 respectively; and acquiring coordinates of four corner points of the image to be corrected and the numbers of the four corner points by using the model. Defining target angular points, namely upper left, upper right, lower right and lower left angular points of the corrected certificate as No. 1, No. 2, No. 3 and No. 4 respectively, acquiring four angular point coordinates of the image to be corrected by using a model, and numbers to which the four angular points belong, wherein the angular point No. 1 corresponds to the upper left angular point of the corrected certificate, the angular point No. 2 corresponds to the upper right angular point of the corrected certificate, the angular point No. 3 corresponds to the lower right angular point of the corrected certificate, and the angular point No. 4 corresponds to the lower left angular point of the corrected certificate no matter how the image to be corrected rotates. The corner point number information can ensure that the corrected image is horizontal and forward.
Second embodiment, fig. 8 is a schematic block diagram of an image correction apparatus 700 according to a second embodiment of the present invention. The image correction apparatus 700 includes:
an information obtaining unit 702, configured to obtain an image to be corrected, and obtain corner coordinates of the image to be corrected;
a calculating unit 704, configured to calculate a perspective correction matrix according to the corner coordinates and the target corner coordinates;
the correcting unit 706 is configured to perform perspective transformation on the image to be corrected by using a perspective correction matrix to obtain a corrected image;
a detection unit 708, configured to detect the corrected image and obtain a text region;
the information recognition unit 710 is configured to perform optical character recognition on the text region to obtain text information.
In this embodiment, after image correction, the detection unit 708 detects the corrected image to obtain a text region, and then the information identification unit 710 identifies the text region to obtain text information, so as to quickly and accurately extract the text in the image, thereby avoiding the problems of low efficiency and high error rate caused by manual input.
In this embodiment, the information obtaining unit 702 is further configured to obtain a confidence level of the corner coordinates; the calculating unit 704 is specifically configured to calculate a perspective correction matrix according to the corner coordinates and the target corner coordinates when the confidence is greater than or equal to the confidence threshold.
The information acquiring unit 702 acquires coordinates of four corner points of a certificate in an image to be corrected, and specifically includes: defining the numbers of an upper left corner, an upper right corner, a lower right corner and a lower left corner of the target corner as No. 1, No. 2, No. 3 and No. 4 respectively; and acquiring coordinates of four corner points of the image to be corrected and the numbers of the four corner points by using the model.
Third embodiment, fig. 9 shows a schematic block diagram of an image correction apparatus 700 according to a third embodiment of the present invention. The image correction apparatus 700 includes:
an information obtaining unit 702, configured to obtain an image to be corrected, and obtain corner coordinates of the image to be corrected;
a calculating unit 704, configured to calculate a perspective correction matrix according to the corner coordinates and the target corner coordinates;
the correcting unit 706 is configured to perform perspective transformation on the image to be corrected by using a perspective correction matrix to obtain a corrected image;
a detection unit 708, configured to detect the corrected image and obtain a text region;
an information recognition unit 710, configured to perform optical character recognition on the text region to obtain text information;
a model establishing unit 712, configured to obtain a training image and label corner coordinates of the training image; and rotating the training image at any angle, and establishing a model for acquiring the corner coordinates and the confidence coefficient of the image to be corrected by using the corner coordinates of the training image and the training image.
In this embodiment, the model establishing unit 712 establishes a model for obtaining the coordinates and confidence of the corners of the image to be corrected, specifically, four corner labeling data of a training image are prepared, coordinates of the four corners are manually labeled, a deep learning model is used to perform regression training of the four corners, and the training image is randomly rotated by any angle in the training process. When the image is corrected, the model is directly used for obtaining the coordinates and the confidence coefficient of the corner points so as to improve the efficiency of image correction.
In this embodiment, the information obtaining unit 702 is further configured to obtain a confidence level of the corner coordinates; the calculating unit 704 is specifically configured to calculate a perspective correction matrix according to the corner coordinates and the target corner coordinates when the confidence is greater than or equal to the confidence threshold.
The information acquiring unit 702 acquires coordinates of four corner points of a certificate in an image to be corrected, and specifically includes: defining the numbers of an upper left corner, an upper right corner, a lower right corner and a lower left corner of the target corner as No. 1, No. 2, No. 3 and No. 4 respectively; and acquiring coordinates of four corner points of the image to be corrected and the numbers of the four corner points by using the model.
In an embodiment of the third aspect of the present invention, a computer device is provided, which includes a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the image correction method according to any of the above embodiments when executing the computer program.
The computer device provided by the present invention implements the steps of the image correction method according to any of the above embodiments when the processor executes the computer program, and therefore, the computer device includes all the advantageous effects of the image correction method according to any of the above embodiments.
An embodiment of the fourth aspect of the present invention proposes a computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, implements the steps of the image correction method according to any one of the above embodiments.
The present invention provides a computer-readable storage medium, which when executed by a processor implements the steps of the image correction method according to any of the above embodiments, and therefore includes all the advantageous effects of the image correction method according to any of the above embodiments.
In the description herein, the terms "first" and "second" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance unless explicitly stated or limited otherwise; the terms "connected," "mounted," "secured," and the like are to be construed broadly and include, for example, fixed connections, removable connections, or integral connections; may be directly connected or indirectly connected through an intermediate. The specific meanings of the above terms in the present invention can be understood by those skilled in the art according to specific situations.
In the description herein, the description of the terms "one embodiment," "some embodiments," "specific embodiments," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (12)
1. An image correction method, comprising:
acquiring an image to be corrected, and acquiring corner coordinates of the image to be corrected;
calculating a perspective correction matrix according to the corner point coordinates and the target corner point coordinates;
and carrying out perspective transformation on the image to be corrected by utilizing the perspective correction matrix to obtain a corrected image.
2. The image correction method according to claim 1, characterized by further comprising:
detecting the corrected image to obtain a character area;
and carrying out optical character recognition on the character area to obtain character information.
3. The image correction method according to claim 1 or 2, characterized by further comprising:
obtaining the confidence coefficient of the corner point coordinates;
and under the condition that the confidence coefficient is greater than or equal to a confidence coefficient threshold value, calculating the perspective correction matrix according to the corner point coordinates and the target corner point coordinates.
4. The image correction method according to claim 3, characterized by further comprising:
acquiring a training image, and marking the coordinates of the corner points of the training image;
and rotating the training image at any angle, and establishing a model for acquiring the corner coordinates of the image to be corrected and the confidence coefficient by using the training image and the corner coordinates of the training image.
5. The image correction method according to claim 4, wherein the step of obtaining coordinates of four corner points of the certificate in the image to be corrected specifically comprises:
defining the numbers of an upper left corner, an upper right corner, a lower right corner and a lower left corner of the target corner as No. 1, No. 2, No. 3 and No. 4 respectively;
And acquiring coordinates of the four corner points of the image to be corrected and numbers of the four corner points by using the model.
6. An image correction apparatus characterized by comprising:
the device comprises an information acquisition unit, a correction unit and a correction unit, wherein the information acquisition unit is used for acquiring an image to be corrected and acquiring the coordinates of the corner points of the image to be corrected;
the calculation unit is used for calculating a perspective correction matrix according to the corner point coordinates and the target corner point coordinates;
and the correcting unit is used for carrying out perspective transformation on the image to be corrected by utilizing the perspective correction matrix to obtain a corrected image.
7. The image correction device according to claim 6, characterized by further comprising:
the detection unit is used for detecting the corrected image and acquiring a character area;
and the information identification unit is used for carrying out optical character identification on the character area to acquire character information.
8. The image correction apparatus according to claim 6 or 7,
the information acquisition unit is further used for acquiring the confidence coefficient of the corner point coordinates;
the calculation unit is specifically configured to calculate the perspective correction matrix according to the corner point coordinates and the target corner point coordinates when the confidence is greater than or equal to a confidence threshold.
9. The image correction device according to claim 8, characterized by further comprising:
the model establishing unit is used for acquiring a training image and marking the corner coordinates of the training image; and rotating the training image at any angle, and establishing a model for acquiring the corner coordinates of the image to be corrected and the confidence coefficient by using the training image and the corner coordinates of the training image.
10. The image correction device according to claim 9, wherein the information acquiring unit acquires coordinates of four corner points of the certificate in the image to be corrected, and specifically includes:
defining the numbers of an upper left corner, an upper right corner, a lower right corner and a lower left corner of the target corner as No. 1, No. 2, No. 3 and No. 4 respectively;
and acquiring coordinates of the four corner points of the image to be corrected and numbers of the four corner points by using the model.
11. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the image correction method according to any one of claims 1 to 5 when executing the computer program.
12. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the image correction method according to any one of claims 1 to 5.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911016421.2A CN111860527A (en) | 2019-10-24 | 2019-10-24 | Image correction method, image correction device, computer device, and storage medium |
| PCT/CN2020/122362 WO2021078133A1 (en) | 2019-10-24 | 2020-10-21 | Systems and methods for image processing |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911016421.2A CN111860527A (en) | 2019-10-24 | 2019-10-24 | Image correction method, image correction device, computer device, and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111860527A true CN111860527A (en) | 2020-10-30 |
Family
ID=72970664
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911016421.2A Pending CN111860527A (en) | 2019-10-24 | 2019-10-24 | Image correction method, image correction device, computer device, and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111860527A (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111860489A (en) * | 2019-12-09 | 2020-10-30 | 北京嘀嘀无限科技发展有限公司 | Certificate image correction method, device, equipment and storage medium |
| CN112468716A (en) * | 2020-11-02 | 2021-03-09 | 航天信息股份有限公司 | Camera visual angle correction method and device, storage medium and electronic equipment |
| CN113012407A (en) * | 2021-02-18 | 2021-06-22 | 上海电机学院 | Eye screen distance prompt myopia prevention system based on machine vision |
| CN113111880A (en) * | 2021-05-12 | 2021-07-13 | 中国平安人寿保险股份有限公司 | Certificate image correction method and device, electronic equipment and storage medium |
| CN113436079A (en) * | 2021-06-23 | 2021-09-24 | 平安科技（深圳）有限公司 | Certificate image detection method and device, electronic equipment and storage medium |
| CN115457559A (en) * | 2022-08-19 | 2022-12-09 | 上海通办信息服务有限公司 | Method, device and equipment for intelligently correcting text and license pictures |
| CN115937003A (en) * | 2022-11-02 | 2023-04-07 | 深圳市新良田科技股份有限公司 | Image processing method, image processing device, terminal equipment and readable storage medium |
-
2019
- 2019-10-24 CN CN201911016421.2A patent/CN111860527A/en active Pending
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111860489A (en) * | 2019-12-09 | 2020-10-30 | 北京嘀嘀无限科技发展有限公司 | Certificate image correction method, device, equipment and storage medium |
| CN112468716A (en) * | 2020-11-02 | 2021-03-09 | 航天信息股份有限公司 | Camera visual angle correction method and device, storage medium and electronic equipment |
| CN113012407A (en) * | 2021-02-18 | 2021-06-22 | 上海电机学院 | Eye screen distance prompt myopia prevention system based on machine vision |
| CN113111880A (en) * | 2021-05-12 | 2021-07-13 | 中国平安人寿保险股份有限公司 | Certificate image correction method and device, electronic equipment and storage medium |
| CN113111880B (en) * | 2021-05-12 | 2023-10-17 | 中国平安人寿保险股份有限公司 | Certificate image correction method, device, electronic equipment and storage medium |
| CN113436079A (en) * | 2021-06-23 | 2021-09-24 | 平安科技（深圳）有限公司 | Certificate image detection method and device, electronic equipment and storage medium |
| CN115457559A (en) * | 2022-08-19 | 2022-12-09 | 上海通办信息服务有限公司 | Method, device and equipment for intelligently correcting text and license pictures |
| CN115457559B (en) * | 2022-08-19 | 2024-01-16 | 上海通办信息服务有限公司 | Method, device and equipment for intelligently correcting texts and license pictures |
| CN115937003A (en) * | 2022-11-02 | 2023-04-07 | 深圳市新良田科技股份有限公司 | Image processing method, image processing device, terminal equipment and readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111860527A (en) | Image correction method, image correction device, computer device, and storage medium | |
| CN110738602B (en) | Image processing method and device, electronic equipment and readable storage medium | |
| CN109657665B (en) | Invoice batch automatic identification system based on deep learning | |
| CN107798299B (en) | Bill information identification method, electronic device and readable storage medium | |
| US20190295267A1 (en) | Detecting specified image identifiers on objects | |
| US9014459B2 (en) | Identification method for valuable file and identification device thereof | |
| US8811751B1 (en) | Method and system for correcting projective distortions with elimination steps on multiple levels | |
| US9082192B2 (en) | Text image trimming method | |
| US8897600B1 (en) | Method and system for determining vanishing point candidates for projective correction | |
| US9280691B2 (en) | System for determining alignment of a user-marked document and method thereof | |
| CN107590495B (en) | Answer sheet picture correction method and device, readable storage medium and electronic equipment | |
| CN110869944B (en) | Reading test cards using mobile devices | |
| CN111860489A (en) | Certificate image correction method, device, equipment and storage medium | |
| US8913836B1 (en) | Method and system for correcting projective distortions using eigenpoints | |
| CN109509378B (en) | A kind of online testing method for supporting handwriting input | |
| CN106845508A (en) | The methods, devices and systems of release in a kind of detection image | |
| WO2018233171A1 (en) | Method and apparatus for entering document information, computer device and storage medium | |
| JP6542230B2 (en) | Method and system for correcting projected distortion | |
| JP4594952B2 (en) | Character recognition device and character recognition method | |
| CN112115907A (en) | Method, device, equipment and medium for extracting structured information of fixed layout certificate | |
| CN108197624A (en) | The recognition methods of certificate image rectification and device, computer storage media | |
| CN114694161A (en) | Text recognition method and equipment for specific format certificate and storage medium | |
| US20190102617A1 (en) | System and method of training a classifier for determining the category of a document | |
| CN112733773A (en) | Object detection method and device, computer equipment and storage medium | |
| US10032073B1 (en) | Detecting aspect ratios of document pages on smartphone photographs by learning camera view angles |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |